[ { "path": ".cargo/config.toml", "content": "[build]\ntarget-dir = \"rust/target\"\n" }, { "path": ".dockerignore", "content": "# Include any files or directories that you don't want to be copied to your\n# container here (e.g., local build artifacts, temporary files, etc.).\n#\n# For more help, visit the .dockerignore file reference guide at\n# https://docs.docker.com/go/build-context-dockerignore/\n\n**/.DS_Store\n**/__pycache__\n**/.venv\n**/.classpath\n**/.dockerignore\n**/.env\n**/.git\n**/.gitignore\n**/.project\n**/.settings\n**/.toolstarget\n**/.vs\n**/.vscode\n**/*.*proj.user\n**/*.dbmdl\n**/*.jfm\n**/bin\n**/charts\n**/docker-compose*\n**/compose*\n**/Dockerfile*\n**/node_modules\n**/npm-debug.log\n**/obj\n**/secrets.dev.yaml\n**/values.dev.yaml\nLICENSE\n\n# Keep this if the container wants it, this is a test file.\n!tests_data/basic/dockerfile/Dockerfile\n" }, { "path": ".gemini/config.yaml", "content": "code_review:\n pull_request_opened:\n summary: false\n" }, { "path": ".gitattributes", "content": "/tests_data/** -text\n" }, { "path": ".github/CODEOWNERS", "content": "# Default owners (lowest precedence).\n* @reyammer @invernizzi\n\n# Julien owns the Rust code\n/rust/ @ia0\n\n# Yanick owns the Python code, all docs, and test data\n/python/ @reyammer\n*.md @reyammer\n/tests_data/ @reyammer\n\n# Julien still owns the Rust changelogs\n/rust/**/CHANGELOG.md @ia0\n\n# Luca owns the JS code, docs, and website\n/js/ @invernizzi\n/docs/js.md @invernizzi\n/website/ @invernizzi\n" }, { "path": ".github/ISSUE_TEMPLATE/misdetection.md", "content": "---\nname: Misdetection\nabout: Report a file, or files, that have been misdetected as something that they\n aren't.\ntitle: \"[Misdetection] file misdetected as \"\nlabels: misdetection, needs triage\nassignees: ''\n\n---\n\n**What should the file have been detected as? What has the file been misdetected as?**\nEx. \"HTML pages are being mistaken for generic XML files.\", \"C# code misdetected as Java.\", or \"Can't tell the difference between exe and dll files.\"\n\n**Please link or attach the misdetected file below** (Do NOT upload PII!)\nPlaceholder.zip\n\n**Additional context**\nAdd any other context or screenshots about the feature request here.\n" }, { "path": ".github/ISSUE_TEMPLATE/new_content_type_request.md", "content": "---\nname: New content type request\nabout: Suggest a new content type for Magika to detect\ntitle: \"[NEW CONTENT TYPE REQUEST]\"\nlabels: missing content type, needs triage\nassignees: ''\n\n---\n\n**What type of file would you like magika to detect?**\nWrite the full name of the file format, followed by the file extension in parenthesis.\nExamples:\n - \"Nintendo Binary Revolution RESource (.brres)\"\n - \"Valve Map Format file (.vmf)\"\n - \"Blender save file (.blend)\"\n - \"RPG Maker 2000/2003 Lcf DataBase (.ldb)\"\n - \"COLLADA file (.dae)\"\n - \"Unreal Engine Asset (.uasset)\"\n\n**What software can create/open these files?**\nExamples:\n- Simply state the name of the software, and where it can be obtained:\n - \"Valve Hammer Editor, included with any Source Engine game on Steam.\"\n- If the file is common enough, write a general description:\n - \"Many 3D modeling software.\"\n - \"Any text editor.\"\n- Link to the GitHub page:\n - \"[BrawlCrate](https://github.com/soopercool101/BrawlCrate)\"\n- Link to the software's websight main page or download page:\n - \"[Blender](https://www.blender.org/download/)\"\n - \"[Unreal Engine](https://www.unrealengine.com/en-US)\"\n- If there are more than one software to open the file type, list them:\n - \"[RPG Maker 2003](https://www.rpgmakerweb.com/products/rpg-maker-2003), [easyRPG](https://easyrpg.org/), Wolf RPG Editor\"\n\n**Where can these files be found?**\nExamples:\n- Simply state where the files can be obtained:\n - \"Any RPGMaker 2000/2003 game.\"\n- Provide instructions on how to obtain the files:\n - \"Dump the ISO of any of [these Wii games](https://wiki.vg-resource.com/BRRES#List_of_games_using_the_format)\"\n - \"Use [bspsrc](https://github.com/ata4/bspsrc) to decompile the BSP files of any Source Engine game. Use [GCFScape](https://nemstools.github.io/pages/GCFScape-Download.html) to extract even more BSPs from 'dir.vpk' files. Make your own with Hammer.\"\n - Link to a source of the files:\n - \"[Unreal Marketplace](https://www.unrealengine.com/marketplace/en-US/store)\"\n - \"https://blendermarket.com/categories/models, https://www.turbosquid.com/Search/3D-Models/marketplace/blend, https://sketchfab.com/store/3d-models/blend?ref=store-home\"\n - Provide some of your own:\n - \"placeholder.zip\" as attachment.\n\n**If possible, please provide a specification for this file type.**\nLink to a resource that explains how the file works.\nExamples:\n- A wiki page:\n - \"https://developer.valvesoftware.com/wiki/VMF_(Valve_Map_Format)\"\n- GitHub documentation:\n - \"https://gota7.github.io/NitroStudio2/specs/sequenceArchive.html\"\n- If you find more than one source, list them:\n - \"https://wiki.tockdom.com/wiki/BRRES_(File_Format), https://horizon.miraheze.org/wiki/.brres\"\n - \"http://www.amnoid.de/gc/Rarc.txt, https://kuribo64.net/wiki/?page=RARC, https://wiki.tockdom.com/wiki/RARC_(File_Format)\"\n - \"https://www.3dbrew.org/wiki/CGFX, https://mk3ds.com/index.php?title=CGFX_(File_Format)\"\n - \"https://mk8.tockdom.com/wiki/BFRES_(File_Format), https://wiki.vg-resource.com/BFRES, https://wiki.oatmealdome.me/BFRES_(File_Format)\"\n- A PDF:\n - \"https://www.collada.org/2008/03/COLLADASchema\"\n \n**Additional context**\nAdd any other context or screenshots about the feature request here.\n" }, { "path": ".github/dependabot.yml", "content": "version: 2\r\n\r\nupdates:\r\n - package-ecosystem: \"github-actions\"\r\n directory: \"/\"\r\n schedule:\r\n interval: \"monthly\"\r\n - package-ecosystem: \"docker\"\r\n directory: \"/\"\r\n schedule:\r\n interval: \"daily\"\r\n - package-ecosystem: \"pip\"\r\n directory: \"/python\"\r\n schedule:\r\n interval: \"daily\"\r\n - package-ecosystem: \"npm\"\r\n directory: \"/js\"\r\n schedule:\r\n interval: \"weekly\"\r\n - package-ecosystem: \"cargo\"\r\n directory: \"/rust\"\r\n schedule:\r\n interval: \"weekly\"\r\n" }, { "path": ".github/labeler.yml", "content": "needs triage:\n - \"/.*/\"\n" }, { "path": ".github/scorecard.yml", "content": "# Scorecard maintainer annotations.\n# See https://github.com/ossf/scorecard/blob/main/config/README.md\n\nannotations:\n # Binary files in tests_data/ are only used for testing.\n - checks:\n - binary-artifacts\n reasons:\n - reason: test-data\n\n - checks:\n - pinned-dependencies\n reasons:\n # Test data with unpinned dependencies:\n # - tests_data/basic/dockerfile/Dockerfile\n - reason: test-data\n # CI/CD containers meant to run the latest version:\n # - .github/workflows/python-e2e-test.yml\n - reason: remediated\n" }, { "path": ".github/workflows/cli-latest.yml", "content": "name: Update the trampoline release\n\non:\n # This is called by cli-release.yml but we don't use the input.\n workflow_call:\n inputs:\n plan:\n required: true\n type: string\n # In case we want to run it manually (the workflow is idempotent).\n workflow_dispatch:\n\npermissions:\n contents: write\n\njobs:\n update:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - run: ./latest.sh\n working-directory: rust\n env:\n GH_TOKEN: ${{ github.token }}\n" }, { "path": ".github/workflows/cli-release.yml", "content": "# This file was autogenerated by dist: https://axodotdev.github.io/cargo-dist\n#\n# Copyright 2022-2024, axodotdev\n# SPDX-License-Identifier: MIT or Apache-2.0\n#\n# CI that:\n#\n# * checks for a Git Tag that looks like a release\n# * builds artifacts with dist (archives, installers, hashes)\n# * uploads those artifacts to temporary workflow zip\n# * on success, uploads the artifacts to a GitHub Release\n#\n# Note that the GitHub Release will be created with a generated\n# title/body based on your changelogs.\n\nname: Release\npermissions:\n \"contents\": \"write\"\n\n# This task will run whenever you push a git tag that looks like a version\n# like \"1.0.0\", \"v0.1.0-prerelease.1\", \"my-app/0.1.0\", \"releases/v1.0.0\", etc.\n# Various formats will be parsed into a VERSION and an optional PACKAGE_NAME, where\n# PACKAGE_NAME must be the name of a Cargo package in your workspace, and VERSION\n# must be a Cargo-style SemVer Version (must have at least major.minor.patch).\n#\n# If PACKAGE_NAME is specified, then the announcement will be for that\n# package (erroring out if it doesn't have the given version or isn't dist-able).\n#\n# If PACKAGE_NAME isn't specified, then the announcement will be for all\n# (dist-able) packages in the workspace with that version (this mode is\n# intended for workspaces with only one dist-able package, or with all dist-able\n# packages versioned/released in lockstep).\n#\n# If you push multiple tags at once, separate instances of this workflow will\n# spin up, creating an independent announcement for each one. However, GitHub\n# will hard limit this to 3 tags per commit, as it will assume more tags is a\n# mistake.\n#\n# If there's a prerelease-style suffix to the version, then the release(s)\n# will be marked as a prerelease.\non:\n pull_request:\n push:\n tags:\n - 'cli**[0-9]+.[0-9]+.[0-9]+*'\n\njobs:\n # Run 'dist plan' (or host) to determine what tasks we need to do\n plan:\n runs-on: \"ubuntu-latest\"\n outputs:\n val: ${{ steps.plan.outputs.manifest }}\n tag: ${{ !github.event.pull_request && github.ref_name || '' }}\n tag-flag: ${{ !github.event.pull_request && format('--tag={0}', github.ref_name) || '' }}\n publishing: ${{ !github.event.pull_request }}\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n steps:\n - uses: actions/checkout@v6\n with:\n persist-credentials: false\n submodules: recursive\n - name: Install dist\n # we specify bash to get pipefail; it guards against the `curl` command\n # failing. otherwise `sh` won't catch that `curl` returned non-0\n shell: bash\n run: \"curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.31.0/cargo-dist-installer.sh | sh\"\n - name: Cache dist\n uses: actions/upload-artifact@v6\n with:\n name: cargo-dist-cache\n path: ~/.cargo/bin/dist\n # sure would be cool if github gave us proper conditionals...\n # so here's a doubly-nested ternary-via-truthiness to try to provide the best possible\n # functionality based on whether this is a pull_request, and whether it's from a fork.\n # (PRs run on the *source* but secrets are usually on the *target* -- that's *good*\n # but also really annoying to build CI around when it needs secrets to work right.)\n - id: plan\n run: |\n dist ${{ (!github.event.pull_request && format('host --steps=create --tag={0}', github.ref_name)) || 'plan' }} --output-format=json > plan-dist-manifest.json\n echo \"dist ran successfully\"\n cat plan-dist-manifest.json\n echo \"manifest=$(jq -c \".\" plan-dist-manifest.json)\" >> \"$GITHUB_OUTPUT\"\n - name: \"Upload dist-manifest.json\"\n uses: actions/upload-artifact@v6\n with:\n name: artifacts-plan-dist-manifest\n path: plan-dist-manifest.json\n\n # Build and packages all the platform-specific things\n build-local-artifacts:\n name: build-local-artifacts (${{ join(matrix.targets, ', ') }})\n # Let the initial task tell us to not run (currently very blunt)\n needs:\n - plan\n if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}\n strategy:\n fail-fast: false\n # Target platforms/runners are computed by dist in create-release.\n # Each member of the matrix has the following arguments:\n #\n # - runner: the github runner\n # - dist-args: cli flags to pass to dist\n # - install-dist: expression to run to install dist on the runner\n #\n # Typically there will be:\n # - 1 \"global\" task that builds universal installers\n # - N \"local\" tasks that build each platform's binaries and platform-specific installers\n matrix: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix }}\n runs-on: ${{ matrix.runner }}\n container: ${{ matrix.container && matrix.container.image || null }}\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n BUILD_MANIFEST_NAME: target/distrib/${{ join(matrix.targets, '-') }}-dist-manifest.json\n permissions:\n \"attestations\": \"write\"\n \"contents\": \"read\"\n \"id-token\": \"write\"\n steps:\n - name: enable windows longpaths\n run: |\n git config --global core.longpaths true\n - uses: actions/checkout@v6\n with:\n persist-credentials: false\n submodules: recursive\n - name: Install Rust non-interactively if not already installed\n if: ${{ matrix.container }}\n run: |\n if ! command -v cargo > /dev/null 2>&1; then\n curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y\n echo \"$HOME/.cargo/bin\" >> $GITHUB_PATH\n fi\n - name: Install dist\n run: ${{ matrix.install_dist.run }}\n # Get the dist-manifest\n - name: Fetch local artifacts\n uses: actions/download-artifact@v7\n with:\n pattern: artifacts-*\n path: target/distrib/\n merge-multiple: true\n - name: Install dependencies\n run: |\n ${{ matrix.packages_install }}\n - name: Build artifacts\n run: |\n # Actually do builds and make zips and whatnot\n dist build ${{ needs.plan.outputs.tag-flag }} --print=linkage --output-format=json ${{ matrix.dist_args }} > dist-manifest.json\n echo \"dist ran successfully\"\n - name: Attest\n uses: actions/attest-build-provenance@v3\n with:\n subject-path: \"target/distrib/*${{ join(matrix.targets, ', ') }}*\"\n - id: cargo-dist\n name: Post-build\n # We force bash here just because github makes it really hard to get values up\n # to \"real\" actions without writing to env-vars, and writing to env-vars has\n # inconsistent syntax between shell and powershell.\n shell: bash\n run: |\n # Parse out what we just built and upload it to scratch storage\n echo \"paths<> \"$GITHUB_OUTPUT\"\n dist print-upload-files-from-manifest --manifest dist-manifest.json >> \"$GITHUB_OUTPUT\"\n echo \"EOF\" >> \"$GITHUB_OUTPUT\"\n\n cp dist-manifest.json \"$BUILD_MANIFEST_NAME\"\n - name: \"Upload artifacts\"\n uses: actions/upload-artifact@v6\n with:\n name: artifacts-build-local-${{ join(matrix.targets, '_') }}\n path: |\n ${{ steps.cargo-dist.outputs.paths }}\n ${{ env.BUILD_MANIFEST_NAME }}\n\n # Build and package all the platform-agnostic(ish) things\n build-global-artifacts:\n needs:\n - plan\n - build-local-artifacts\n runs-on: \"ubuntu-latest\"\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json\n steps:\n - uses: actions/checkout@v6\n with:\n persist-credentials: false\n submodules: recursive\n - name: Install cached dist\n uses: actions/download-artifact@v7\n with:\n name: cargo-dist-cache\n path: ~/.cargo/bin/\n - run: chmod +x ~/.cargo/bin/dist\n # Get all the local artifacts for the global tasks to use (for e.g. checksums)\n - name: Fetch local artifacts\n uses: actions/download-artifact@v7\n with:\n pattern: artifacts-*\n path: target/distrib/\n merge-multiple: true\n - id: cargo-dist\n shell: bash\n run: |\n dist build ${NEEDS_PLAN_OUTPUTS_TAG_FLAG} --output-format=json \"--artifacts=global\" > dist-manifest.json\n echo \"dist ran successfully\"\n\n # Parse out what we just built and upload it to scratch storage\n echo \"paths<> \"$GITHUB_OUTPUT\"\n jq --raw-output \".upload_files[]\" dist-manifest.json >> \"$GITHUB_OUTPUT\"\n echo \"EOF\" >> \"$GITHUB_OUTPUT\"\n\n cp dist-manifest.json \"$BUILD_MANIFEST_NAME\"\n env:\n NEEDS_PLAN_OUTPUTS_TAG_FLAG: ${{ needs.plan.outputs.tag-flag }}\n - name: \"Upload artifacts\"\n uses: actions/upload-artifact@v6\n with:\n name: artifacts-build-global\n path: |\n ${{ steps.cargo-dist.outputs.paths }}\n ${{ env.BUILD_MANIFEST_NAME }}\n # Determines if we should publish/announce\n host:\n needs:\n - plan\n - build-local-artifacts\n - build-global-artifacts\n # Only run if we're \"publishing\", and only if plan, local and global didn't fail (skipped is fine)\n if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n runs-on: \"ubuntu-latest\"\n outputs:\n val: ${{ steps.host.outputs.manifest }}\n steps:\n - uses: actions/checkout@v6\n with:\n persist-credentials: false\n submodules: recursive\n - name: Install cached dist\n uses: actions/download-artifact@v7\n with:\n name: cargo-dist-cache\n path: ~/.cargo/bin/\n - run: chmod +x ~/.cargo/bin/dist\n # Fetch artifacts from scratch-storage\n - name: Fetch artifacts\n uses: actions/download-artifact@v7\n with:\n pattern: artifacts-*\n path: target/distrib/\n merge-multiple: true\n - id: host\n shell: bash\n run: |\n dist host ${NEEDS_PLAN_OUTPUTS_TAG_FLAG} --steps=upload --steps=release --output-format=json > dist-manifest.json\n echo \"artifacts uploaded and released successfully\"\n cat dist-manifest.json\n echo \"manifest=$(jq -c \".\" dist-manifest.json)\" >> \"$GITHUB_OUTPUT\"\n env:\n NEEDS_PLAN_OUTPUTS_TAG_FLAG: ${{ needs.plan.outputs.tag-flag }}\n - name: \"Upload dist-manifest.json\"\n uses: actions/upload-artifact@v6\n with:\n # Overwrite the previous copy\n name: artifacts-dist-manifest\n path: dist-manifest.json\n # Create a GitHub Release while uploading all files to it\n - name: \"Download GitHub Artifacts\"\n uses: actions/download-artifact@v7\n with:\n pattern: artifacts-*\n path: artifacts\n merge-multiple: true\n - name: Cleanup\n run: |\n # Remove the granular manifests\n rm -f artifacts/*-dist-manifest.json\n - name: Create GitHub Release\n env:\n PRERELEASE_FLAG: \"${{ fromJson(steps.host.outputs.manifest).announcement_is_prerelease && '--prerelease' || '' }}\"\n ANNOUNCEMENT_TITLE: \"${{ fromJson(steps.host.outputs.manifest).announcement_title }}\"\n ANNOUNCEMENT_BODY: \"${{ fromJson(steps.host.outputs.manifest).announcement_github_body }}\"\n RELEASE_COMMIT: \"${{ github.sha }}\"\n NEEDS_PLAN_OUTPUTS_TAG: ${{ needs.plan.outputs.tag }}\n run: |\n # Write and read notes from a file to avoid quoting breaking things\n echo \"$ANNOUNCEMENT_BODY\" > $RUNNER_TEMP/notes.txt\n\n gh release create \"${NEEDS_PLAN_OUTPUTS_TAG}\" --target \"$RELEASE_COMMIT\" $PRERELEASE_FLAG --title \"$ANNOUNCEMENT_TITLE\" --notes-file \"$RUNNER_TEMP/notes.txt\" artifacts/*\n\n announce:\n needs:\n - plan\n - host\n # use \"always() && ...\" to allow us to wait for all publish jobs while\n # still allowing individual publish jobs to skip themselves (for prereleases).\n # \"host\" however must run to completion, no skipping allowed!\n if: ${{ always() && needs.host.result == 'success' }}\n runs-on: \"ubuntu-latest\"\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n steps:\n - uses: actions/checkout@v6\n with:\n persist-credentials: false\n submodules: recursive\n\n custom-cli-latest:\n needs:\n - plan\n - announce\n uses: ./.github/workflows/cli-latest.yml\n with:\n plan: ${{ needs.plan.outputs.val }}\n secrets: inherit\n" }, { "path": ".github/workflows/codeql.yml", "content": "# For most projects, this workflow file will not need changing; you simply need\n# to commit it to your repository.\n#\n# You may wish to alter this file to override the set of languages analyzed,\n# or to provide custom queries or build logic.\n#\n# ******** NOTE ********\n# We have attempted to detect the languages in your repository. Please check\n# the `language` matrix defined below to confirm you have the correct set of\n# supported CodeQL languages.\n#\nname: \"CodeQL\"\n\non:\n push:\n branches: [\"main\"]\n pull_request:\n branches: [\"main\"]\n schedule:\n - cron: \"42 13 * * 2\"\n\npermissions:\n contents: read\n\njobs:\n analyze:\n name: Analyze\n # Runner size impacts CodeQL analysis time. To learn more, please see:\n # - https://gh.io/recommended-hardware-resources-for-running-codeql\n # - https://gh.io/supported-runners-and-hardware-resources\n # - https://gh.io/using-larger-runners\n # Consider using larger runners for possible analysis time improvements.\n runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}\n timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }}\n permissions:\n # required for all workflows\n security-events: write\n\n # only required for workflows in private repositories\n actions: read\n contents: read\n\n strategy:\n fail-fast: false\n matrix:\n language: [\"javascript-typescript\", \"python\"]\n # CodeQL supports [ 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'swift' ]\n # Use only 'java-kotlin' to analyze code written in Java, Kotlin or both\n # Use only 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both\n # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support\n\n steps:\n - name: Checkout repository\n uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0\n\n # Initializes the CodeQL tools for scanning.\n - name: Initialize CodeQL\n uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5\n with:\n languages: ${{ matrix.language }}\n # If you wish to specify custom queries, you can do so here or in a config file.\n # By default, queries listed here will override any specified in a config file.\n # Prefix the list here with \"+\" to use these queries and those in the config file.\n\n # For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs\n # queries: security-extended,security-and-quality\n\n # Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift).\n # If this step fails, then you should remove it and run the build manually (see below)\n - name: Autobuild\n uses: github/codeql-action/autobuild@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5\n\n # ℹ️ Command-line programs to run using the OS shell.\n # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun\n\n # If the Autobuild fails above, remove it and uncomment the following three lines.\n # modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.\n\n # - run: |\n # echo \"Run, Build Application using script\"\n # ./location_of_script_within_repo/buildscript.sh\n\n - name: Perform CodeQL Analysis\n uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5\n with:\n category: \"/language:${{matrix.language}}\"\n" }, { "path": ".github/workflows/docs-check.yml", "content": "name: Docs - Check documentation\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - \"*.md\"\n - \"assets/**/*.md\"\n - \"docs/**/*.md\"\n - \"js/**/*.md\"\n - \"python/**/*.md\"\n - \"rust/**/*.md\"\n - \"website-ng/**/*.md\"\n schedule:\n - cron: \"42 7 * * 4\" # Run weekly\n\npermissions:\n contents: read\n\njobs:\n run-check-docs:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/0.5.22/install.sh | sh\n\n - name: \"Run check_documentation.py script\"\n working-directory: python\n run: uv run ./scripts/check_documentation.py\n" }, { "path": ".github/workflows/github-issue-labeler.yml", "content": "name: New issue labeler\non:\n # Runs on newly opened issues\n issues:\n types: [opened]\n\n# Sets permissions of the GITHUB_TOKEN\npermissions:\n issues: write\n contents: read\n\njobs:\n triage:\n runs-on: ubuntu-latest\n steps:\n - uses: github/issue-labeler@c1b0f9f52a63158c4adc09425e858e87b32e9685 # pin@v3.4\n with:\n configuration-path: .github/labeler.yml\n enable-versioned-regex: 0\n repo-token: \"${{secrets.GITHUB_TOKEN}}\"\n" }, { "path": ".github/workflows/github-pages.yml", "content": "name: Pages - deploy\n\non:\n # Runs on pushes targeting the default branch\n push:\n branches: [\"main\"]\n paths:\n - \"website/**\"\n\n # Allows you to run this workflow manually from the Actions tab\n workflow_dispatch:\n\n# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages\npermissions:\n contents: read\n pages: write\n id-token: write\n\n# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.\nconcurrency:\n group: \"pages\"\n cancel-in-progress: false\n\njobs:\n deploy-pages:\n environment:\n name: github-pages\n url: ${{ steps.deployment.outputs.page_url }}\n runs-on: ubuntu-latest\n defaults:\n run:\n working-directory: ./website\n steps:\n - name: Checkout\n uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n with:\n ref: main\n - name: Set up Node\n uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # pin@v4\n with:\n node-version: 18.x\n - name: Install dependencies\n run: yarn install --frozen-lockfile\n - name: Build\n run: yarn run build-github\n - name: Setup Pages\n uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # pin@v4\n - name: Upload artifact\n uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # pin@v3\n with:\n path: \"./website/dist\"\n - name: Deploy to GitHub Pages\n id: deployment\n uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # pin@v4\n" }, { "path": ".github/workflows/go-test.yml", "content": "name: Go - build and run tests\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - \"go/**\"\n - \"tests_data/**\"\n - \".github/workflows/go-test.yml\"\n\npermissions:\n contents: read\n\njobs:\n unit-testing:\n runs-on: \"ubuntu-latest\"\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n\n - name: Build the Go Docker image (which also runs tests)\n run: docker build -f go/docker/Dockerfile .\n" }, { "path": ".github/workflows/js-check-import-scenarios.yml", "content": "# Tests that the Magika library can be used in various scenarios (e.g., commonjs/node, esmodule/node, esmodule/browse)\nname: JS - check import scenarios\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - 'js/simple_examples/**'\n - '.github/workflows/run-js-examples.yml'\n\njobs:\n run-examples:\n runs-on: ubuntu-latest\n defaults:\n run:\n working-directory: js/simple_examples\n steps:\n - name: Checkout\n uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # pin@v4\n with:\n ref: main\n\n - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # pin@v4\n name: Set up Node.js\n with:\n node-version: \"20.x\"\n registry-url: \"https://registry.npmjs.org\"\n\n - name: Install Magika dependencies\n run: yarn install --frozen-lockfile\n working-directory: js\n\n - name: Build Magika\n run: yarn build && rm -Rf node_modules\n working-directory: js\n\n - name: Install dependencies for the node-commonjs-example\n run: yarn install\n working-directory: js/simple_examples/node-commonjs-example\n\n - name: Test node-commonjs-example\n run: yarn --silent start && rm -Rf node_modules\n working-directory: js/simple_examples/node-commonjs-example\n\n - name: Install dependencies for the node-esmodule-example\n run: yarn install\n working-directory: js/simple_examples/node-esmodule-example\n\n - name: Test node-esmodule-example\n run: yarn --silent start && rm -Rf node_modules\n working-directory: js/simple_examples/node-esmodule-example\n\n - name: Install dependencies for the browser-esmodule-example\n run: yarn install && yarn playwright install chromium\n working-directory: js/simple_examples/browser-esmodule-example\n\n - name: Test browser-esmodule-example\n run: yarn --silent start && rm -Rf node_modules\n working-directory: js/simple_examples/browser-esmodule-example\n\n - name: Install dependencies for the typescript-esmodule-example\n run: yarn install\n working-directory: js/simple_examples/typescript-esmodule-example\n\n - name: Test typescript-esmodule-example\n run: yarn --silent start && rm -Rf node_modules\n working-directory: js/simple_examples/typescript-esmodule-example\n" }, { "path": ".github/workflows/js-docs-builder.yml", "content": "name: JS - generate docs\n\non:\n # Runs on pushes targeting the default branch\n push:\n branches: [\"main\"]\n paths:\n - \"js/**\"\n - \".github/workflows/**\"\n\n # Allows you to run this workflow manually from the Actions tab\n workflow_dispatch:\n\npermissions:\n contents: read\n\njobs:\n makeDocs:\n permissions:\n contents: write\n id-token: write\n runs-on: ubuntu-latest\n steps:\n - name: Checkout\n uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n with:\n ref: main\n - name: Generate docs\n working-directory: js\n run: |\n yarn install --frozen-lockfile\n yarn run build\n yarn run make-docs\n\n - name: Commit\n run: |\n git config --local user.email \"invernizzi.l@gmail.com\"\n git config --local user.name \"Luca Invernizzi\"\n git commit -m \"Update docs\" -a\n" }, { "path": ".github/workflows/js-publish.yml", "content": "name: JS - publish\non:\n workflow_dispatch:\npermissions:\n contents: read\n\njobs:\n build:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n\n - name: Enable Corepack\n run: corepack enable\n\n - uses: actions/setup-node@v6\n with:\n node-version: \"20.x\"\n registry-url: \"https://registry.npmjs.org\"\n\n - name: Build\n working-directory: js\n run: |\n yarn install --frozen-lockfile\n yarn run build\n yarn run test\n\n - name: Publish\n working-directory: js\n run: yarn publish\n env:\n NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}\n" }, { "path": ".github/workflows/js-test.yml", "content": "name: JS - tests\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - \"js/**\"\n - \"tests_data/**\"\n - \".github/workflows/js-*\"\npermissions:\n contents: read\njobs:\n unit-testing:\n strategy:\n matrix:\n node-version: [\"18\", \"20\"]\n os: [\"ubuntu-latest\", \"macos-latest\"]\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # pin@v4\n with:\n node-version: ${{ matrix.node-version }}\n - name: Install dependencies\n working-directory: js\n run: yarn install --frozen-lockfile\n - name: Build\n working-directory: js\n run: yarn run build\n - name: Run tests\n working-directory: js\n run: yarn test\n" }, { "path": ".github/workflows/python-build-and-release-package.yml", "content": "name: Python - build and release package\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n tags:\n # Note: These need to match what specified in env.[TEST_]RELEASE_TAG_PREFIX below.\n - \"python-v*\"\n - \"python-test-v*\"\n pull_request:\n paths:\n - \"python/**\"\n - \"rust/**\"\n - \"tests_data/**\"\n - \".github/workflows/python-build-and-release-package.yml\"\n schedule:\n - cron: \"12 3 * * 4\" # Run everything once per week.\n - cron: \"12 3 * * 1\" # Refresh the cache an additional time.\n release:\n types: [created]\n\npermissions:\n contents: read\n\nenv:\n # Trigger for publishing to pypi and testpypi (and for pre-release checks\n # enforcement).\n RELEASE_TAG_PREFIX: \"python-v\"\n TEST_RELEASE_TAG_PREFIX: \"python-test-v\"\n UV_VERSION: \"0.9.5\"\n\njobs:\n # This job acts as a gatekeeper for releases, which are triggered by a tag\n # push. It performs critical pre-release checks. These checks are skipped for\n # non-release pushes.\n pre-release-checks:\n name: Pre-release checks\n runs-on: ubuntu-latest\n steps:\n - name: Checkout code\n uses: actions/checkout@v5\n - name: Setup python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"3.12\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n - name: Check package for release\n env:\n IS_RELEASE_TAG: ${{ github.event_name == 'push' && github.ref_type == 'tag' && (startsWith(github.ref_name, env.RELEASE_TAG_PREFIX) || startsWith(github.ref_name, env.TEST_RELEASE_TAG_PREFIX)) }}\n run: |\n if [[ \"${IS_RELEASE_TAG}\" == 'true' ]]; then\n FULL_TAG_REF=\"${GITHUB_REF}\"\n TAG_NAME=\"${GITHUB_REF_NAME}\"\n if [[ \"${TAG_NAME}\" == \"${{ env.RELEASE_TAG_PREFIX }}\"* ]]; then\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.RELEASE_TAG_PREFIX }}\"\n else\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.TEST_RELEASE_TAG_PREFIX }}\"\n fi\n TAG_VERSION=\"${FULL_TAG_REF#${PREFIX_TO_REMOVE}}\"\n CHECKER_OPTIONS=\"--expected-version ${TAG_VERSION}\"\n else\n CHECKER_OPTIONS=\"--report-only\"\n fi\n # Note: this uses the magika python package installed via uv. Also,\n # pip is not available here, so we skip pip show check.\n uv run ./scripts/pre_release_check.py $CHECKER_OPTIONS --no-check-pip-show-package-version\n working-directory: python\n\n build-wheels:\n needs: [pre-release-checks]\n runs-on: ${{ matrix.platform.runner }}\n strategy:\n matrix:\n platform:\n - runner: ubuntu-latest\n target: x86_64\n - runner: windows-latest\n target: x64\n - runner: macos-14\n target: aarch64\n steps:\n - uses: actions/checkout@v5\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"3.12\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n - if: matrix.platform.runner == 'ubuntu-latest'\n uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3\n with:\n path: rust/onnx/runtime/build/Linux\n key: maturin-${{ matrix.platform.target }}-${{ hashFiles('rust/onnx/build.sh') }}\n - if: matrix.platform.runner == 'ubuntu-latest'\n name: Build wheels\n uses: PyO3/maturin-action@v1\n with:\n target: ${{ matrix.platform.target }}\n args: --release --out=../dist\n before-script-linux: \"${{ github.workspace }}/rust/onnx/maturin.sh\"\n manylinux: 2_28\n working-directory: python\n - if: matrix.platform.runner != 'ubuntu-latest'\n name: Build wheels\n uses: PyO3/maturin-action@v1\n with:\n target: ${{ matrix.platform.target }}\n args: --release --out=../dist\n working-directory: python\n - name: Upload wheels\n uses: actions/upload-artifact@v4\n with:\n name: wheel-${{ matrix.platform.runner }}-${{ matrix.platform.target }}\n path: dist\n\n # Download, install, and test the wheels with different versions of python\n test-wheels:\n needs: [build-wheels]\n runs-on: ${{ matrix.platform.runner }}\n if: github.event.schedule != '12 3 * * 1'\n strategy:\n # We want to know in which exact situation the tests fail\n fail-fast: false\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n platform:\n - runner: ubuntu-latest\n target: x86_64\n - runner: windows-latest\n target: x64\n - runner: macos-14\n target: aarch64\n steps:\n - uses: actions/checkout@v5\n - uses: actions/download-artifact@v4\n with:\n name: wheel-${{ matrix.platform.runner }}-${{ matrix.platform.target }}\n path: dist\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"${{ matrix.python-version }}\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n # Attempt \"uv add magika.whl\", in a temporary directory\n - name: Check that `uv add magika.whl` works\n shell: bash\n run: |\n mkdir /tmp/test-uv\n cp -vR dist/*.whl /tmp/test-uv\n cd /tmp/test-uv\n uv init\n uv add ./$(\\ls -1 *.whl | head -n 1)\n # From now on, magika will be available in the global environment\n - name: Install the wheel via pip\n run: python3 -m pip install $(python -c \"import glob; print(glob.glob('dist/*.whl')[0])\")\n\n - run: magika --version\n - run: \"python3 -c 'import magika; m = magika.Magika(); print(m)'\"\n - run: magika -r tests_data/basic\n - run: python3 ./python/scripts/run_quick_test_magika_cli.py\n - run: python3 ./python/scripts/run_quick_test_magika_module.py\n - name: Check package for release readiness\n env:\n IS_RELEASE_TAG: ${{ github.event_name == 'push' && github.ref_type == 'tag' && (startsWith(github.ref_name, env.RELEASE_TAG_PREFIX) || startsWith(github.ref_name, env.TEST_RELEASE_TAG_PREFIX)) }}\n shell: bash\n run: |\n if [[ \"${IS_RELEASE_TAG}\" == 'true' ]]; then\n FULL_TAG_REF=\"${GITHUB_REF}\"\n TAG_NAME=\"${GITHUB_REF_NAME}\"\n if [[ \"${TAG_NAME}\" == \"${{ env.RELEASE_TAG_PREFIX }}\"* ]]; then\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.RELEASE_TAG_PREFIX }}\"\n else\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.TEST_RELEASE_TAG_PREFIX }}\"\n fi\n TAG_VERSION=\"${FULL_TAG_REF#${PREFIX_TO_REMOVE}}\"\n CHECKER_OPTIONS=\"--expected-version ${TAG_VERSION}\"\n else\n CHECKER_OPTIONS=\"--report-only\"\n fi\n # Note: this uses the magika python package that was just built.\n python3 ./scripts/pre_release_check.py $CHECKER_OPTIONS\n working-directory: python\n\n build-pure-python-wheel-and-sdist:\n needs: [pre-release-checks]\n runs-on: ubuntu-latest\n if: github.event.schedule != '12 3 * * 1'\n steps:\n - uses: actions/checkout@v5\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n - run: uv run ./scripts/prepare_pyproject_for_pure_python_wheel.py\n working-directory: python\n - name: Build pure python wheel and source distribution\n run: uv build --out-dir ../dist\n working-directory: python\n - name: Upload pure python wheel\n uses: actions/upload-artifact@v4\n with:\n name: wheel-pure-python\n path: dist/*.whl\n - name: Upload sdist\n uses: actions/upload-artifact@v4\n with:\n name: sdist\n path: dist/*.tar.gz\n\n # Download, install, and test the pure python wheel on multiple platforms\n test-pure-python-wheel:\n needs: [build-pure-python-wheel-and-sdist]\n runs-on: ${{ matrix.platform.runner }}\n if: github.event.schedule != '12 3 * * 1'\n strategy:\n # We want to know in which exact situation the tests fail\n fail-fast: false\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n platform:\n - runner: ubuntu-latest\n target: x86_64\n - runner: windows-latest\n target: x64\n - runner: macos-14\n target: aarch64\n steps:\n - uses: actions/checkout@v5\n - uses: actions/download-artifact@v4\n with:\n name: wheel-pure-python\n path: dist\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"${{ matrix.python-version }}\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n # Attempt \"uv add magika.whl\", in a temporary directory\n - name: Check that `uv add magika.whl` works\n shell: bash\n run: |\n mkdir /tmp/test-uv\n cp -vR dist/*.whl /tmp/test-uv\n cd /tmp/test-uv\n uv init\n uv add ./$(\\ls -1 *.whl | head -n 1)\n # From now on, magika will be available in the global environment\n - name: Install the wheel\n run: python3 -m pip install $(python -c \"import glob; print(glob.glob('dist/*.whl')[0])\")\n\n # Check that the magika script points to the placeholder raising a warning\n - run: magika --version | grep -C10 WARNING | grep -C10 magika-python-client\n # Check that the fallback magika's python client can be run\n - run: magika-python-client -r tests_data/basic\n # Check that the results of the python's client are correct\n - run: python3 ./python/scripts/run_quick_test_magika_cli.py --client-path magika-python-client\n # Test the python module\n - run: \"python3 -c 'import magika; m = magika.Magika(); print(m)'\"\n - run: python3 ./python/scripts/run_quick_test_magika_module.py\n - name: Check package for release readiness\n env:\n IS_RELEASE_TAG: ${{ github.event_name == 'push' && github.ref_type == 'tag' && (startsWith(github.ref_name, env.RELEASE_TAG_PREFIX) || startsWith(github.ref_name, env.TEST_RELEASE_TAG_PREFIX)) }}\n shell: bash\n run: |\n if [[ \"${IS_RELEASE_TAG}\" == 'true' ]]; then\n FULL_TAG_REF=\"${GITHUB_REF}\"\n TAG_NAME=\"${GITHUB_REF_NAME}\"\n if [[ \"${TAG_NAME}\" == \"${{ env.RELEASE_TAG_PREFIX }}\"* ]]; then\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.RELEASE_TAG_PREFIX }}\"\n else\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.TEST_RELEASE_TAG_PREFIX }}\"\n fi\n TAG_VERSION=\"${FULL_TAG_REF#${PREFIX_TO_REMOVE}}\"\n CHECKER_OPTIONS=\"--expected-version ${TAG_VERSION}\"\n else\n CHECKER_OPTIONS=\"--report-only\"\n fi\n # Note: this uses the magika python package that was just built.\n python3 ./scripts/pre_release_check.py $CHECKER_OPTIONS --use-python-client\n working-directory: python\n\n test-sdist:\n needs: [build-pure-python-wheel-and-sdist]\n runs-on: ${{ matrix.platform.runner }}\n if: github.event.schedule != '12 3 * * 1'\n strategy:\n # We want to know in which exact situation the tests fail\n fail-fast: false\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n platform:\n - runner: ubuntu-latest\n target: x86_64\n - runner: windows-latest\n target: x64\n - runner: macos-14\n target: aarch64\n steps:\n - uses: actions/checkout@v5\n - uses: actions/download-artifact@v4\n with:\n name: sdist\n path: dist\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"${{ matrix.python-version }}\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n # Attempt \"uv add magika.whl\", in a temporary directory\n - name: Check that `uv add magika.tar.gz` works\n shell: bash\n run: |\n mkdir /tmp/test-uv\n cp -vR dist/*.tar.gz /tmp/test-uv\n cd /tmp/test-uv\n uv init\n uv add ./$(\\ls -1 *.tar.gz | head -n 1)\n # From now on, magika will be available in the global environment\n - name: Install the sdist\n run: python3 -m pip install $(python -c \"import glob; print(glob.glob('dist/*.tar.gz')[0])\")\n\n # Check that the magika script points to the placeholder raising a warning\n - run: magika --version | grep -C10 WARNING | grep -C10 magika-python-client\n # Check that the fallback magika's python client can be run\n - run: magika-python-client -r tests_data/basic\n # Check that the results of the python's client are correct\n - run: python3 ./python/scripts/run_quick_test_magika_cli.py --client-path magika-python-client\n # Test the python module\n - run: \"python3 -c 'import magika; m = magika.Magika(); print(m)'\"\n - run: python3 ./python/scripts/run_quick_test_magika_module.py\n - name: Check package for release readiness\n env:\n IS_RELEASE_TAG: ${{ github.event_name == 'push' && github.ref_type == 'tag' && (startsWith(github.ref_name, env.RELEASE_TAG_PREFIX) || startsWith(github.ref_name, env.TEST_RELEASE_TAG_PREFIX)) }}\n shell: bash\n run: |\n if [[ \"${IS_RELEASE_TAG}\" == 'true' ]]; then\n FULL_TAG_REF=\"${GITHUB_REF}\"\n TAG_NAME=\"${GITHUB_REF_NAME}\"\n if [[ \"${TAG_NAME}\" == \"${{ env.RELEASE_TAG_PREFIX }}\"* ]]; then\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.RELEASE_TAG_PREFIX }}\"\n else\n PREFIX_TO_REMOVE=\"refs/tags/${{ env.TEST_RELEASE_TAG_PREFIX }}\"\n fi\n TAG_VERSION=\"${FULL_TAG_REF#${PREFIX_TO_REMOVE}}\"\n CHECKER_OPTIONS=\"--expected-version ${TAG_VERSION}\"\n else\n CHECKER_OPTIONS=\"--report-only\"\n fi\n # Note: this uses the magika python package that was just built.\n python3 ./scripts/pre_release_check.py $CHECKER_OPTIONS --use-python-client\n working-directory: python\n\n # Adapted from https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/\n # Note: The publishing is only done with pushes of release tags.\n publish-to-pypi:\n name: Publish to PyPI\n if: github.event_name == 'push' && github.ref_type == 'tag' && contains(github.ref_name, 'python')\n needs: [test-wheels, test-pure-python-wheel, test-sdist]\n runs-on: ubuntu-latest\n\n environment:\n name: pypi\n url: https://pypi.org/p/magika\n\n permissions:\n id-token: write # IMPORTANT: mandatory for trusted publishing\n\n steps:\n - name: Download all the artifacts (binary wheels, pure python wheel, sdist)\n uses: actions/download-artifact@v4\n with:\n path: artifacts/\n\n - name: Flatten artifacts structure\n run: |\n # List all files for debugging\n ls -alR artifacts/\n\n # Find all files inside the subdirectories and move them up\n find artifacts/ -mindepth 2 -type f -exec mv -t artifacts/ {} +\n\n # Remove the now-empty subdirectories\n find artifacts/ -mindepth 1 -type d -empty -delete\n\n # Check structure after flattening\n ls -alR artifacts/\n\n - name: Publish distribution to PyPI\n if: github.event_name == 'push' && github.ref_type == 'tag' && startsWith(github.ref_name, env.RELEASE_TAG_PREFIX)\n uses: pypa/gh-action-pypi-publish@release/v1\n with:\n packages-dir: artifacts/\n\n # Note: The publishing is only done with pushes of test release tags.\n publish-to-testpypi:\n name: Publish to TestPyPI\n if: github.event_name == 'push' && github.ref_type == 'tag' && contains(github.ref_name, 'python')\n needs: [test-wheels, test-pure-python-wheel, test-sdist]\n runs-on: ubuntu-latest\n\n environment:\n name: testpypi\n url: https://test.pypi.org/p/magika\n\n permissions:\n id-token: write # IMPORTANT: mandatory for trusted publishing\n\n steps:\n - name: Download all the artifacts (binary wheels, pure python wheel, sdist)\n uses: actions/download-artifact@v4\n with:\n path: artifacts/\n\n - name: Flatten artifacts structure\n run: |\n # List all files for debugging\n ls -alR artifacts/\n\n # Find all files inside the subdirectories and move them up\n find artifacts/ -mindepth 2 -type f -exec mv -t artifacts/ {} +\n\n # Remove the now-empty subdirectories\n find artifacts/ -mindepth 1 -type d -empty -delete\n\n # Check structure after flattening\n ls -alR artifacts/\n\n - name: Publish distribution to TestPyPI\n if: github.event_name == 'push' && github.ref_type == 'tag' && startsWith(github.ref_name, env.TEST_RELEASE_TAG_PREFIX)\n uses: pypa/gh-action-pypi-publish@release/v1\n with:\n packages-dir: artifacts/\n repository-url: https://test.pypi.org/legacy/\n" }, { "path": ".github/workflows/python-test-published-package.yml", "content": "# This routinely checks that published packages are installable and work\n# properly. This makes sure that a new version of one of our dependencies is not\n# breaking our releases.\n# TODO: test more magika package versions\n# TODO: check the actual predicted content types\nname: Python - test published packages\n\non:\n schedule:\n - cron: \"42 4 * * *\" # Run daily\n workflow_dispatch:\n pull_request:\n paths:\n - \".github/workflows/python-test-published-package.yml\"\n\npermissions:\n contents: read\n\nenv:\n UV_VERSION: \"0.9.5\"\n\njobs:\n unit-testing:\n strategy:\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n os: [\"ubuntu-latest\", \"macos-latest\", \"windows-latest\"]\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"${{ matrix.python-version }}\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n - name: Check that `uv add magika` works\n shell: bash\n run: mkdir /tmp/test-uv && cd /tmp/test-uv && uv init && uv add magika && cd - && rm -rf /tmp/test-uv\n - name: Install magika with pip\n run: python3 -m pip install magika\n - run: python3 -c 'import magika; m = magika.Magika(); print(m)'\n - run: magika --version\n # The latest published model does not necessarily support detection for\n # all types in our tests data; thus, for now we just check that the magika\n # CLI does not crash when scanning the files, without checking the actual\n # predictions.\n - run: magika -r tests_data/basic\n" }, { "path": ".github/workflows/python-test-published-rc-package.yml", "content": "# This routinely checks that the latest published -rc packages are installable\n# and work properly. This makes sure that a new version of one of our\n# dependencies is not breaking our releases.\n# TODO: test more magika package versions\n# TODO: check the actual predicted content types\nname: Python - test published -rc packages\n\non:\n schedule:\n - cron: \"42 3 * * *\" # Run daily\n workflow_dispatch:\n pull_request:\n paths:\n - \".github/workflows/python-test-published-rc-package.yml\"\n\npermissions:\n contents: read\n\nenv:\n UV_VERSION: \"0.9.5\"\n\njobs:\n unit-testing:\n strategy:\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n os: [\"ubuntu-latest\", \"macos-latest\", \"windows-latest\"]\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"${{ matrix.python-version }}\"\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n - name: Check that `uv add --prerelease magika` works\n shell: bash\n run: mkdir /tmp/test-uv && cd /tmp/test-uv && uv init && uv add --prerelease allow magika && cd - && rm -rf /tmp/test-uv\n - name: Install magika with pip (considering prereleases)\n run: python3 -m pip install --pre magika\n - run: python3 -c 'import magika; m = magika.Magika(); print(m)'\n - run: magika --version\n # The latest published model does not necessarily support detection for\n # all types in our tests data; thus, for now we just check that the magika\n # CLI does not crash when scanning the files, without checking the actual\n # predictions.\n - run: magika -r tests_data/basic\n" }, { "path": ".github/workflows/python-test-suite.yml", "content": "name: Python - run test suite\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - \"python/**\"\n - \"rust/**\"\n - \"tests_data/**\"\n - \".github/workflows/python-test-suite.yml\"\n\npermissions:\n contents: read\n\nenv:\n UV_VERSION: \"0.9.5\"\n\njobs:\n unit-testing:\n strategy:\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n os: [\"ubuntu-latest\", \"macos-latest\"]\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n\n - name: Setup Python\n uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # pin@v5\n with:\n python-version: \"${{ matrix.python-version }}\"\n\n - name: Install uv\n run: curl -LsSf https://astral.sh/uv/${{ env.UV_VERSION }}/install.sh | sh\n\n - name: Install all projects dependencies (with the requested python version)\n working-directory: python\n run: uv sync --python ${{ matrix.python-version }} --all-extras --dev\n\n - name: Print python version\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} python --version\n\n - name: Run ruff check\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} ruff check --verbose\n\n - name: Run ruff format --check\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} ruff format --check --verbose\n\n - name: Run mypy\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} mypy src/magika tests\n\n - name: Run copyright checks\n working-directory: python\n run: uv run ./scripts/check_copyright.py\n\n - name: Run the python tests suite\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} pytest tests -m \"not slow\"\n\n - name: Run magika --version\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} magika --version\n shell: bash # Allows for cross-platform\n\n - name: Run magika with tests_data\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} magika -r ../tests_data/basic\n\n - name: Run \"magika cli\" quick tests\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} scripts/run_quick_test_magika_cli.py\n\n - name: Run \"magika module\" quick tests\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} scripts/run_quick_test_magika_module.py\n\n - name: Run \"magika module\" quick tests + perf measurement\n working-directory: python\n run: uv run --python ${{ matrix.python-version }} scripts/run_quick_test_magika_module.py --print-inference-stats --repeat 10\n\n changelog-check:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # pin@v4\n with:\n fetch-depth: 0\n - name: Check for undocumented changes\n run: ./scripts/check_changelog.sh ${GITHUB_BASE_REF}\n working-directory: python\n" }, { "path": ".github/workflows/rust-test.yml", "content": "name: Rust - test\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - \".github/workflows/rust-*\"\n - \"assets/**\"\n - \"rust/**\"\n - \"tests_data/**\"\n\npermissions:\n contents: read\n\njobs:\n changelog:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n with:\n fetch-depth: 0\n - run: ./changelog.sh\n working-directory: rust\n test:\n runs-on: ubuntu-latest\n continue-on-error: ${{ matrix.toolchain == 'nightly' }}\n strategy:\n matrix:\n toolchain: [stable, nightly]\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - run: rustup default ${{ matrix.toolchain }}\n - run: rustup component add rustfmt clippy\n - run: ./test.sh\n working-directory: rust\n run:\n runs-on: ${{ matrix.os }}-latest\n strategy:\n matrix:\n os: [ubuntu, macos, windows]\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - run: cargo build --release\n working-directory: rust/cli\n - run: rust/target/release/magika -r tests_data/basic\n" }, { "path": ".github/workflows/scorecard.yml", "content": "# This workflow uses actions that are not certified by GitHub. They are provided\n# by a third-party and are governed by separate terms of service, privacy\n# policy, and support documentation.\n\nname: Scorecard supply-chain security\non:\n workflow_dispatch:\n # For Branch-Protection check. Only the default branch is supported. See\n # https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection\n branch_protection_rule:\n # To guarantee Maintained check is occasionally updated. See\n # https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained\n schedule:\n - cron: \"45 21 * * 2\"\n push:\n branches: [\"main\"]\n\n# Declare default permissions as read only.\npermissions: read-all\n\njobs:\n analysis:\n name: Scorecard analysis\n runs-on: ubuntu-latest\n permissions:\n # Needed to upload the results to code-scanning dashboard.\n security-events: write\n # Needed to publish results and get a badge (see publish_results below).\n id-token: write\n\n steps:\n - name: \"Checkout code\"\n uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n with:\n persist-credentials: false\n\n - name: \"Run analysis\"\n uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3\n with:\n results_file: results.sarif\n results_format: sarif\n # (Optional) \"write\" PAT token. Uncomment the `repo_token` line below if:\n # - you want to enable the Branch-Protection check on a *public* repository, or\n # - you are installing Scorecard on a *private* repository\n # To create the PAT, follow the steps in https://github.com/ossf/scorecard-action#authentication-with-pat.\n # repo_token: ${{ secrets.SCORECARD_TOKEN }}\n\n # Public repositories:\n # - Publish results to OpenSSF REST API for easy access by consumers\n # - Allows the repository to include the Scorecard badge.\n # - See https://github.com/ossf/scorecard-action#publishing-results.\n # For private repositories:\n # - `publish_results` will always be set to `false`, regardless\n # of the value entered here.\n publish_results: true\n\n # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF\n # format to the repository Actions tab.\n - name: \"Upload artifact\"\n uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6\n with:\n name: SARIF file\n path: results.sarif\n retention-days: 5\n\n # Upload the results to GitHub's code scanning dashboard.\n - name: \"Upload to code-scanning\"\n uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5\n with:\n sarif_file: results.sarif\n" }, { "path": ".github/workflows/website-test.yml", "content": "name: Website - tests\n\non:\n workflow_dispatch:\n push:\n branches:\n - \"main\"\n pull_request:\n paths:\n - \"website/**\"\n - \"js/**\"\n - \"tests_data/**\"\n - \".github/workflows/website-*\"\npermissions:\n contents: read\njobs:\n build-and-test:\n strategy:\n matrix:\n node-version: [\"20\"]\n os: [\"ubuntu-latest\"]\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # pin@v4\n - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # pin@v4\n with:\n node-version: ${{ matrix.node-version }}\n - name: Install js dependencies\n working-directory: js\n run: yarn install --frozen-lockfile\n - name: Build js\n working-directory: js\n run: yarn run build\n - name: Create magika link\n working-directory: js\n run: yarn link\n - name: Link to local magika\n working-directory: website\n run: yarn link magika\n - name: Install website dependencies\n working-directory: website\n run: yarn install --frozen-lockfile\n - name: Build website\n working-directory: website\n run: yarn build\n # TODO: add some actual testing\n" }, { "path": ".gitignore", "content": "*.pyc\n__pycache__/\n.ipynb_checkpoints\nvenv/\ntmp/\n.env\n*.swp\n*.egg-info\ndist/*\n*.pickle\n.s.yml\n\n*/models-data/*\n\n.vscode\n\nscratchpad/\n\n*/node_modules/*\ndocs/dist\njs/dist\n" }, { "path": "CITATION.cff", "content": "cff-version: 1.2.0\nmessage: \"If you use this software, please cite it as below.\"\nauthors:\n- family-names: \"Fratantonio\"\n given-names: \"Yanick\"\n- family-names: \"Invernizzi\"\n given-names: \"Luca\"\n- family-names: \"Farah\"\n given-names: \"Loua\"\n- family-names: \"Kurt\"\n given-names: \"Thomas\"\n- family-names: \"Zhang\"\n given-names: \"Marina\"\n- family-names: \"Albertini\"\n given-names: \"Ange\"\n- family-names: \"Galilee\"\n given-names: \"Francois\"\n- family-names: \"Metitieri\"\n given-names: \"Giancarlo\"\n- family-names: \"Cretin\"\n given-names: \"Julien\"\n- family-names: \"Petit-Bianco\"\n given-names: \"Alexandre\"\n- family-names: \"Tao\"\n given-names: \"David\"\n- family-names: \"Bursztein\"\n given-names: \"Elie\"\ntitle: \"Magika: AI-Powered Content-Type Detection\"\nurl: \"https://arxiv.org/abs/2409.13768\"\n" }, { "path": "CONTRIBUTING.md", "content": "# How to Contribute\n\nWe would love to accept your patches and contributions to this project!\n\nCheck [open issues labeled as \"help wanted\"](https://github.com/google/magika/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) as a starting point.\n\n## Before you begin\n\n### Sign our Contributor License Agreement\n\nContributions to this project must be accompanied by a\n[Contributor License Agreement](https://cla.developers.google.com/about) (CLA).\nYou (or your employer) retain the copyright to your contribution; this simply\ngives us permission to use and redistribute your contributions as part of the\nproject.\n\nIf you or your current employer have already signed the Google CLA (even if it\nwas for a different project), you probably don't need to do it again.\n\nVisit to see your current agreements or to\nsign a new one.\n\n### Review our Community Guidelines\n\nThis project follows [Google's Open Source Community\nGuidelines](https://opensource.google/conduct/).\n\n## Contribution process\n\n### Code Reviews\n\nAll submissions, including submissions by project members, require review. We\nuse [GitHub pull requests](https://docs.github.com/articles/about-pull-requests)\nfor this purpose.\n" }, { "path": "Dockerfile", "content": "# syntax=docker/dockerfile:1\n\nARG PYTHON_VERSION=3.11\nFROM python:${PYTHON_VERSION}-slim as base\n\nWORKDIR /magika\n\n# This requires buildx\n# RUN --mount=type=cache,target=/root/.cache/pip \\\n# pip install magika\n\nRUN pip install magika\n\nENTRYPOINT [\"magika\"]\n" }, { "path": "LICENSE", "content": "\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright [yyyy] [name of copyright owner]\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License." }, { "path": "README.md", "content": "# Magika\n\n[![image](https://img.shields.io/pypi/v/magika.svg)](https://pypi.python.org/pypi/magika)\n[![NPM Version](https://img.shields.io/npm/v/magika)](https://npmjs.com/package/magika)\n[![image](https://img.shields.io/pypi/l/magika.svg)](https://pypi.python.org/pypi/magika)\n[![image](https://img.shields.io/pypi/pyversions/magika.svg)](https://pypi.python.org/pypi/magika)\n[![Go Version](https://img.shields.io/github/v/tag/google/magika?filter=go%2F*&label=go&sort=semver)](https://pkg.go.dev/github.com/google/magika/go)\n\n[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8706/badge)](https://www.bestpractices.dev/en/projects/8706)\n![CodeQL](https://github.com/google/magika/workflows/CodeQL/badge.svg)\n[![Actions status](https://github.com/google/magika/actions/workflows/python-build-and-release-package.yml/badge.svg)](https://github.com/google/magika/actions)\n[![PyPI Monthly Downloads](https://static.pepy.tech/badge/magika/month)](https://pepy.tech/projects/magika)\n[![PyPI Downloads](https://static.pepy.tech/badge/magika)](https://pepy.tech/projects/magika)\n\nMagika is a novel AI-powered file type detection tool that relies on the recent advance of deep learning to provide accurate detection. Under the hood, Magika employs a custom, highly optimized model that only weighs about a few MBs, and enables precise file identification within milliseconds, even when running on a single CPU. Magika has been trained and evaluated on a dataset of ~100M samples across 200+ content types (covering both binary and textual file formats), and it achieves an average ~99% accuracy on our test set.\n\nHere is an example of what Magika command line output looks like:\n\n

\n \n

\n\nMagika is used at scale to help improve Google users' safety by routing Gmail, Drive, and Safe Browsing files to the proper security and content policy scanners, processing hundreds billions samples on a weekly basis. Magika has also been integrated with [VirusTotal](https://www.virustotal.com/) ([example](./assets/magika-vt.png)) and [abuse.ch](https://bazaar.abuse.ch/) ([example](./assets/magika-abusech.png)).\n\nFor more context you can read our initial [announcement post on Google's OSS blog](https://opensource.googleblog.com/2024/02/magika-ai-powered-fast-and-efficient-file-type-identification.html), you can consult [Magika's website](https://securityresearch.google/magika/), and you can read more in our [research paper](https://securityresearch.google/magika/additional-resources/research-papers-and-citation/), published at the IEEE/ACM International Conference on Software Engineering (ICSE) 2025.\n\nYou can try Magika without installing anything by using our [web demo](https://securityresearch.google/magika/demo/magika-demo/), which runs locally in your browser!\n\n\n# Highlights\n\n- Available as a command line tool written in Rust, a Python API, and additional bindings for Rust, JavaScript/TypeScript (with an experimental npm package, which powers our [web demo](https://securityresearch.google/magika/demo/magika-demo/)), and GoLang (WIP).\n- Trained and evaluated on a dataset of ~100M files across [200+ content types](./assets/models/standard_v3_3/README.md).\n- On our test set, Magika achieves ~99% average precision and recall, outperforming existing approaches -- especially on textual content types.\n- After the model is loaded (which is a one-off overhead), the inference time is about 5ms per file, even when run on a single CPU.\n- You can invoke Magika with even thousands of files at the same time. You can also use `-r` for recursively scanning a directory.\n- Near-constant inference time, independently from the file size; Magika only uses a limited subset of the file's content.\n- Magika uses a per-content-type threshold system that determines whether to \"trust\" the prediction for the model, or whether to return a generic label, such as \"Generic text document\" or \"Unknown binary data\".\n- The tolerance to errors can be controlled via different prediction modes, such as `high-confidence`, `medium-confidence`, and `best-guess`.\n- The client and the bindings are already open source, and more is coming soon!\n\n# Table of Contents\n\n1. [Getting Started](#getting-started)\n 1. [Installation](#installation)\n 1. [Quick Start](#quick-start)\n1. [Documentation](#documentation)\n1. [Security Vulnerabilities](#security-vulnerabilities)\n1. [License](#license)\n1. [Disclaimer](#disclaimer)\n\n# Getting Started\n\n## Installation\n\n### Command Line Tool\n\nMagika ships a CLI written in Rust, and can be installed in several ways.\n\nVia `magika` python package:\n```shell\npipx install magika\n```\n\nVia brew (macOS / Linux)\n```shell\nbrew install magika\n```\n\nVia installer script:\n```shell\ncurl -LsSf https://securityresearch.google/magika/install.sh | sh\n```\n\nor:\n```shell\npowershell -ExecutionPolicy Bypass -c \"irm https://securityresearch.google/magika/install.ps1 | iex\"\n```\n\nVia `magika-cli` Rust package:\n```shell\ncargo install --locked magika-cli\n```\n\n### Python package\n\n```shell\npip install magika\n```\n\n### JavaScript package\n\n```shell\nnpm install magika\n```\n\n\n## Quick Start\n\nHere you can find a number of quick examples just to get you started.\n\nTo learn about Magika's inner workings, see the [Core Concepts](https://securityresearch.google/magika/core-concepts/) section of Magika's website.\n\n### Command Line Tool Examples\n\n```shell\n% cd tests_data/basic && magika -r * | head\nasm/code.asm: Assembly (code)\nbatch/simple.bat: DOS batch file (code)\nc/code.c: C source (code)\ncss/code.css: CSS source (code)\ncsv/magika_test.csv: CSV document (code)\ndockerfile/Dockerfile: Dockerfile (code)\ndocx/doc.docx: Microsoft Word 2007+ document (document)\ndocx/magika_test.docx: Microsoft Word 2007+ document (document)\neml/sample.eml: RFC 822 mail (text)\nempty/empty_file: Empty file (inode)\n```\n\n```shell\n% magika ./tests_data/basic/python/code.py --json\n[\n {\n \"path\": \"./tests_data/basic/python/code.py\",\n \"result\": {\n \"status\": \"ok\",\n \"value\": {\n \"dl\": {\n \"description\": \"Python source\",\n \"extensions\": [\n \"py\",\n \"pyi\"\n ],\n \"group\": \"code\",\n \"is_text\": true,\n \"label\": \"python\",\n \"mime_type\": \"text/x-python\"\n },\n \"output\": {\n \"description\": \"Python source\",\n \"extensions\": [\n \"py\",\n \"pyi\"\n ],\n \"group\": \"code\",\n \"is_text\": true,\n \"label\": \"python\",\n \"mime_type\": \"text/x-python\"\n },\n \"score\": 0.996999979019165\n }\n }\n }\n]\n```\n\n```shell\n% cat tests_data/basic/ini/doc.ini | magika -\n-: INI configuration file (text)\n```\n\n```shell\n% magika --help\nDetermines file content types using AI\n\nUsage: magika [OPTIONS] [PATH]...\n\nArguments:\n [PATH]...\n List of paths to the files to analyze.\n\n Use a dash (-) to read from standard input (can only be used once).\n\nOptions:\n -r, --recursive\n Identifies files within directories instead of identifying the directory itself\n\n --no-dereference\n Identifies symbolic links as is instead of identifying their content by following them\n\n --colors\n Prints with colors regardless of terminal support\n\n --no-colors\n Prints without colors regardless of terminal support\n\n -s, --output-score\n Prints the prediction score in addition to the content type\n\n -i, --mime-type\n Prints the MIME type instead of the content type description\n\n -l, --label\n Prints a simple label instead of the content type description\n\n --json\n Prints in JSON format\n\n --jsonl\n Prints in JSONL format\n\n --format \n Prints using a custom format (use --help for details).\n\n The following placeholders are supported:\n\n %p The file path\n %l The unique label identifying the content type\n %d The description of the content type\n %g The group of the content type\n %m The MIME type of the content type\n %e Possible file extensions for the content type\n %s The score of the content type for the file\n %S The score of the content type for the file in percent\n %b The model output if overruled (empty otherwise)\n %% A literal %\n\n -h, --help\n Print help (see a summary with '-h')\n\n -V, --version\n Print version\n```\n\nFor more examples and documentation about the CLI, see https://crates.io/crates/magika-cli.\n\n\n### Python Examples\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_bytes(b'function log(msg) {console.log(msg);}')\n>>> print(res.output.label)\njavascript\n```\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_path('./tests_data/basic/ini/doc.ini')\n>>> print(res.output.label)\nini\n```\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f:\n>>> res = m.identify_stream(f)\n>>> print(res.output.label)\nini\n```\n\nFor more examples and documentation about the Python module, see the [Python `Magika` module](https://securityresearch.google/magika/cli-and-bindings/python/) section.\n\n\n# Documentation\n\nPlease consult [Magika's website](https://securityresearch.google/magika) for detailed documentation about:\n- Core Concepts\n - How Magika works\n - Models & content types\n - Prediction modes\n - Understanding the output\n- CLI & Bindings (Python module, JavaScript module, ...)\n- Contributing\n- FAQ\n- ...\n\n\n# Security Vulnerabilities\n\nPlease contact us directly at magika-dev@google.com.\n\n\n# License\n\nApache 2.0; see [`LICENSE`](LICENSE) for details.\n\n\n# Disclaimer\n\nThis project is not an official Google project. It is not supported by\nGoogle and Google specifically disclaims all warranties as to its quality,\nmerchantability, or fitness for a particular purpose.\n" }, { "path": "assets/content_types_kb.min.json", "content": "{\"3gp\":{\"mime_type\":\"video/3gpp\",\"group\":\"video\",\"description\":\"3GPP multimedia file\",\"extensions\":[\"3gp\"],\"is_text\":false},\"3ds\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Nintendo 3DS roms\",\"extensions\":[\"3ds\"],\"is_text\":false},\"3dsx\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Nintendo 3DS homebrew\",\"extensions\":[\"3dsx\"],\"is_text\":false},\"3dsm\":{\"mime_type\":\"application/x-3ds\",\"group\":\"image\",\"description\":\"3D studio Max\",\"extensions\":[\"3ds\"],\"is_text\":false},\"3mf\":{\"mime_type\":\"application/vnd.ms-package.3dmanufacturing-3dmodel+xml\",\"group\":\"image\",\"description\":\"3D Manufacturing Format\",\"extensions\":[\"3mf\"],\"is_text\":false},\"abnf\":{\"mime_type\":\"text/plain\",\"group\":null,\"description\":\"augmented Backus\\u2013Naur form\",\"extensions\":[\"abnf\"],\"is_text\":false},\"ace\":{\"mime_type\":\"application/x-ace-compressed\",\"group\":\"archive\",\"description\":\"ACE archive\",\"extensions\":[\"ace\"],\"is_text\":false},\"ada\":{\"mime_type\":\"text/x-ada\",\"group\":\"code\",\"description\":\"ADA source\",\"extensions\":[],\"is_text\":false},\"aff\":{\"mime_type\":\"text/plain\",\"group\":null,\"description\":\"Hunspell Affix\",\"extensions\":[\"aff\"],\"is_text\":true},\"ai\":{\"mime_type\":\"application/pdf\",\"group\":\"document\",\"description\":\"Adobe Illustrator Artwork\",\"extensions\":[\"ai\"],\"is_text\":false},\"aidl\":{\"mime_type\":\"text/plain\",\"group\":null,\"description\":\"Android Interface Definition Language\",\"extensions\":[\"aidl\"],\"is_text\":true},\"algol68\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"a68\"],\"is_text\":false},\"ani\":{\"mime_type\":\"application/x-navi-animation\",\"group\":null,\"description\":\"Animated cursor\",\"extensions\":[\"ani\"],\"is_text\":false},\"apk\":{\"mime_type\":\"application/vnd.android.package-archive\",\"group\":\"executable\",\"description\":\"Android package\",\"extensions\":[\"apk\"],\"is_text\":false},\"applebplist\":{\"mime_type\":\"application/x-bplist\",\"group\":\"application\",\"description\":\"Apple binary property list\",\"extensions\":[\"bplist\",\"plist\"],\"is_text\":false},\"appledouble\":{\"mime_type\":\"multipart/appledouble\",\"group\":\"unknown\",\"description\":\"AppleDouble\",\"extensions\":[],\"is_text\":false},\"appleplist\":{\"mime_type\":\"application/x-plist\",\"group\":\"application\",\"description\":\"Apple property list\",\"extensions\":[\"plist\"],\"is_text\":true},\"applesingle\":{\"mime_type\":\"application/applefile\",\"group\":\"unknown\",\"description\":\"AppleSingle\",\"extensions\":[],\"is_text\":false},\"ar\":{\"mime_type\":\"application/x-archive\",\"group\":\"archive\",\"description\":\"AR Archive\",\"extensions\":[],\"is_text\":false},\"arc\":{\"mime_type\":\"application/x-arc\",\"group\":\"archive\",\"description\":\"Arc\",\"extensions\":[\"arc\"],\"is_text\":false},\"arj\":{\"mime_type\":\"application/arj\",\"group\":\"archive\",\"description\":\"Arj\",\"extensions\":[],\"is_text\":false},\"arrow\":{\"mime_type\":\"vnd.apache.arrow.file\",\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"asc\":{\"mime_type\":\"application/pgp-signature\",\"group\":\"text\",\"description\":\"PGP\",\"extensions\":[\"asc\"],\"is_text\":true},\"asd\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"au\":{\"mime_type\":\"audio/basic\",\"group\":\"audio\",\"description\":\"NeXT/Sun AU\",\"extensions\":[\"au\"],\"is_text\":false},\"asf\":{\"mime_type\":\"video/x-ms-wma\",\"group\":\"application\",\"description\":\"Microsoft Advanced Systems Format\",\"extensions\":[\"asf\"],\"is_text\":false},\"asm\":{\"mime_type\":\"text/x-asm\",\"group\":\"code\",\"description\":\"Assembly\",\"extensions\":[\"s\",\"S\",\"asm\"],\"is_text\":true},\"asp\":{\"mime_type\":\"text/html\",\"group\":\"code\",\"description\":\"ASP source\",\"extensions\":[\"aspx\",\"asp\"],\"is_text\":true},\"autohotkey\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"AutoHotKey script\",\"extensions\":[],\"is_text\":true},\"autoit\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"AutoIt script\",\"extensions\":[\"au3\"],\"is_text\":true},\"avi\":{\"mime_type\":\"video/x-msvideo\",\"group\":\"video\",\"description\":\"Audio Video Interleave\",\"extensions\":[\"avi\"],\"is_text\":false},\"avif\":{\"mime_type\":\"image/avif\",\"group\":\"video\",\"description\":\"AV1 Image File Format\",\"extensions\":[\"avif\",\"avifs\"],\"is_text\":false},\"avro\":{\"mime_type\":\"application/x-avro-binary\",\"group\":null,\"description\":\"Apache Avro binary\",\"extensions\":[\"avro\"],\"is_text\":false},\"awk\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Awk\",\"extensions\":[\"awk\"],\"is_text\":true},\"ax\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"Directshow filter\",\"extensions\":[\"ax\"],\"is_text\":false},\"batch\":{\"mime_type\":\"text/x-msdos-batch\",\"group\":\"code\",\"description\":\"DOS batch file\",\"extensions\":[\"bat\"],\"is_text\":true},\"bazel\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Bazel build file\",\"extensions\":[\"bzl\"],\"is_text\":true},\"bcad\":{\"mime_type\":\"application/octet-stream\",\"group\":\"document\",\"description\":\"bCAD Drawing\",\"extensions\":[\"bdf\"],\"is_text\":false},\"bib\":{\"mime_type\":\"text/x-bibtex\",\"group\":\"text\",\"description\":\"BibTeX\",\"extensions\":[\"bib\"],\"is_text\":true},\"bmp\":{\"mime_type\":\"image/bmp\",\"group\":\"image\",\"description\":\"BMP image data\",\"extensions\":[\"bmp\"],\"is_text\":false},\"bpg\":{\"mime_type\":\"image/bpg\",\"group\":\"image\",\"description\":\"BPG\",\"extensions\":[\"bpg\"],\"is_text\":false},\"bpl\":{\"mime_type\":null,\"group\":\"unknown\",\"description\":null,\"extensions\":[\"bpl\"],\"is_text\":false},\"brainfuck\":{\"mime_type\":\"text/x-brainfuck\",\"group\":\"code\",\"description\":\"Brainfuck source\",\"extensions\":[\"b\",\"bf\"],\"is_text\":true},\"brf\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Braille Ready Format\",\"extensions\":[\"brf\",\"bfm\"],\"is_text\":false},\"bzip\":{\"mime_type\":\"application/x-bzip2\",\"group\":\"archive\",\"description\":\"bzip2 compressed data\",\"extensions\":[\"bz2\",\"tbz2\",\"tar.bz2\"],\"is_text\":false},\"bzip3\":{\"mime_type\":\"application/x-bzip3\",\"group\":\"archive\",\"description\":\"bzip3 compressed data\",\"extensions\":[\"bz3\"],\"is_text\":false},\"c\":{\"mime_type\":\"text/x-c\",\"group\":\"code\",\"description\":\"C source\",\"extensions\":[\"c\"],\"is_text\":true},\"cab\":{\"mime_type\":\"application/vnd.ms-cab-compressed\",\"group\":\"archive\",\"description\":\"Microsoft Cabinet archive data\",\"extensions\":[\"cab\"],\"is_text\":false},\"cad\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"cat\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Windows Catalog file\",\"extensions\":[\"cat\"],\"is_text\":false},\"cdf\":{\"mime_type\":null,\"group\":\"archive\",\"description\":null,\"extensions\":[],\"is_text\":false},\"chm\":{\"mime_type\":\"application/chm\",\"group\":\"application\",\"description\":\"MS Windows HtmlHelp Data\",\"extensions\":[\"chm\"],\"is_text\":false},\"clojure\":{\"mime_type\":\"text/x-clojure\",\"group\":\"code\",\"description\":\"Clojure\",\"extensions\":[\"clj\",\"cljs\",\"cljc\",\"cljr\"],\"is_text\":true},\"cmake\":{\"mime_type\":\"text/x-cmake\",\"group\":\"code\",\"description\":\"CMake build file\",\"extensions\":[\"cmake\"],\"is_text\":true},\"cobol\":{\"mime_type\":\"text/x-cobol\",\"group\":\"code\",\"description\":\"Cobol\",\"extensions\":[\"cbl\",\"cob\",\"cpy\",\"CBL\",\"COB\",\"CPY\"],\"is_text\":true},\"coff\":{\"mime_type\":\"application/x-coff\",\"group\":\"executable\",\"description\":\"Intel 80386 COFF\",\"extensions\":[\"obj\",\"o\"],\"is_text\":false},\"coffeescript\":{\"mime_type\":\"text/coffeescript\",\"group\":\"code\",\"description\":\"CoffeeScript\",\"extensions\":[\"coffee\"],\"is_text\":true},\"com\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":null,\"extensions\":[],\"is_text\":false},\"cpl\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"cpl\"],\"is_text\":false},\"cpp\":{\"mime_type\":\"text/x-c\",\"group\":\"code\",\"description\":\"C++ source\",\"extensions\":[\"cc\",\"cpp\",\"cxx\",\"c++\",\"cppm\",\"ixx\"],\"is_text\":true},\"crt\":{\"mime_type\":\"application/x-x509-ca-cert\",\"group\":\"text\",\"description\":\"Certificates (binary format)\",\"extensions\":[\"der\",\"cer\",\"crt\"],\"is_text\":false},\"crx\":{\"mime_type\":\"application/x-chrome-extension\",\"group\":\"executable\",\"description\":\"Google Chrome extension\",\"extensions\":[\"crx\"],\"is_text\":false},\"cs\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"C# source\",\"extensions\":[\"cs\",\"csx\"],\"is_text\":true},\"csproj\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\".NET project config\",\"extensions\":[\"csproj\"],\"is_text\":true},\"css\":{\"mime_type\":\"text/css\",\"group\":\"code\",\"description\":\"CSS source\",\"extensions\":[\"css\"],\"is_text\":true},\"csv\":{\"mime_type\":\"text/csv\",\"group\":\"code\",\"description\":\"CSV document\",\"extensions\":[\"csv\"],\"is_text\":true},\"ctl\":{\"mime_type\":\"application/octet-stream\",\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"dart\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Dart source\",\"extensions\":[\"dart\"],\"is_text\":true},\"deb\":{\"mime_type\":\"application/vnd.debian.binary-package\",\"group\":\"archive\",\"description\":\"Debian binary package\",\"extensions\":[\"deb\"],\"is_text\":false},\"dex\":{\"mime_type\":\"application/x-android-dex\",\"group\":\"executable\",\"description\":\"Dalvik dex file\",\"extensions\":[\"dex\"],\"is_text\":false},\"dey\":{\"mime_type\":\"application/x-android-dey\",\"group\":\"executable\",\"description\":\"Dalvik dex file\",\"extensions\":[],\"is_text\":false},\"dicom\":{\"mime_type\":\"application/dicom\",\"group\":\"image\",\"description\":\"DICOM\",\"extensions\":[\"dcm\"],\"is_text\":false},\"diff\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Diff file\",\"extensions\":[\"diff\",\"patch\"],\"is_text\":true},\"directory\":{\"mime_type\":\"inode/directory\",\"group\":\"inode\",\"description\":\"A directory\",\"extensions\":[],\"is_text\":false},\"django\":{\"mime_type\":\"text/x-django\",\"group\":\"code\",\"description\":\"Django source\",\"extensions\":[],\"is_text\":false},\"dll\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"dll\"],\"is_text\":false},\"dm\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Dream Maker\",\"extensions\":[\"dm\"],\"is_text\":true},\"dmigd\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Dominion Mods\",\"extensions\":[\"dm\"],\"is_text\":true},\"dmg\":{\"mime_type\":\"application/x-apple-diskimage\",\"group\":\"archive\",\"description\":\"Apple disk image\",\"extensions\":[\"dmg\"],\"is_text\":false},\"dmscript\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Digital Micrograph Script\",\"extensions\":[\"s\"],\"is_text\":true},\"doc\":{\"mime_type\":\"application/msword\",\"group\":\"document\",\"description\":\"Microsoft Word CDF document\",\"extensions\":[\"doc\"],\"is_text\":false},\"dockerfile\":{\"mime_type\":\"text/x-dockerfile\",\"group\":\"code\",\"description\":\"Dockerfile\",\"extensions\":[],\"is_text\":true},\"docx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.wordprocessingml.document\",\"group\":\"document\",\"description\":\"Microsoft Word 2007+ document\",\"extensions\":[\"docx\",\"docm\"],\"is_text\":false},\"dosmbr\":{\"mime_type\":\"application/octet-stream\",\"group\":null,\"description\":\"Master boot record\",\"extensions\":[],\"is_text\":false},\"dotx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.wordprocessingml.template\",\"group\":\"document\",\"description\":\"Office Word 2007 template\",\"extensions\":[\"dotx\"],\"is_text\":false},\"dsstore\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Application Desktop Services Store\",\"extensions\":[],\"is_text\":false},\"dwg\":{\"mime_type\":\"image/x-dwg\",\"group\":\"image\",\"description\":\"Autocad Drawing\",\"extensions\":[\"dwg\"],\"is_text\":false},\"dxf\":{\"mime_type\":\"image/vnd.dxf\",\"group\":\"image\",\"description\":\"Audocad Drawing Exchange Format\",\"extensions\":[\"dxf\"],\"is_text\":true},\"dylib\":{\"mime_type\":\"application/x-mach-o\",\"group\":\"executable\",\"description\":\"Mach-O executable\",\"extensions\":[\"dylib\"],\"is_text\":false},\"ebml\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Extensible Binary Meta Language\",\"extensions\":[],\"is_text\":false},\"elf\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ELF executable\",\"extensions\":[\"elf\"],\"is_text\":false},\"elixir\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Elixir script\",\"extensions\":[\"exs\"],\"is_text\":true},\"emf\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Windows Enhanced Metafile image data\",\"extensions\":[\"emf\"],\"is_text\":false},\"eml\":{\"mime_type\":\"message/rfc822\",\"group\":\"text\",\"description\":\"RFC 822 mail\",\"extensions\":[\"eml\"],\"is_text\":true},\"empty\":{\"mime_type\":\"inode/x-empty\",\"group\":\"inode\",\"description\":\"Empty file\",\"extensions\":[],\"is_text\":false},\"epub\":{\"mime_type\":\"application/epub+zip\",\"group\":\"document\",\"description\":\"EPUB document\",\"extensions\":[\"epub\"],\"is_text\":false},\"erb\":{\"mime_type\":\"text/x-ruby\",\"group\":\"code\",\"description\":\"Embedded Ruby source\",\"extensions\":[\"erb\"],\"is_text\":true},\"erlang\":{\"mime_type\":\"text/x-erlang\",\"group\":\"code\",\"description\":\"Erlang source\",\"extensions\":[\"erl\",\"hrl\"],\"is_text\":true},\"ese\":{\"mime_type\":\"application/x-ms-ese\",\"group\":null,\"description\":\"ESE Db\",\"extensions\":[\"dat\"],\"is_text\":false},\"exe\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE executable\",\"extensions\":[\"exe\"],\"is_text\":false},\"exp\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"flac\":{\"mime_type\":\"audio/flac\",\"group\":\"audio\",\"description\":\"FLAC audio bitstream data\",\"extensions\":[\"flac\"],\"is_text\":false},\"flutter\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"flv\":{\"mime_type\":\"video/x-flv\",\"group\":\"video\",\"description\":\"Flash Video\",\"extensions\":[\"flv\"],\"is_text\":false},\"fortran\":{\"mime_type\":\"text/x-fortran\",\"group\":\"document\",\"description\":\"Fortran\",\"extensions\":[\"f90\",\"f95\",\"f03\",\"F90\"],\"is_text\":true},\"fpx\":{\"mime_type\":null,\"group\":\"image\",\"description\":\"Flashpix\",\"extensions\":[\"fpx\"],\"is_text\":false},\"gemfile\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gemfile file\",\"extensions\":[],\"is_text\":true},\"gemspec\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gemspec file\",\"extensions\":[\"gemspec\"],\"is_text\":true},\"gif\":{\"mime_type\":\"image/gif\",\"group\":\"image\",\"description\":\"GIF image data\",\"extensions\":[\"gif\"],\"is_text\":false},\"gitattributes\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gitattributes file\",\"extensions\":[],\"is_text\":true},\"gitmodules\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gitmodules file\",\"extensions\":[],\"is_text\":true},\"gleam\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"Gleam source\",\"extensions\":[\"gleam\"],\"is_text\":true},\"go\":{\"mime_type\":\"text/x-golang\",\"group\":\"code\",\"description\":\"Golang source\",\"extensions\":[\"go\"],\"is_text\":true},\"gpx\":{\"mime_type\":null,\"group\":null,\"description\":\"XML document\",\"extensions\":[\"gpx\"],\"is_text\":false},\"gradle\":{\"mime_type\":\"text/x-groovy\",\"group\":\"code\",\"description\":\"Gradle source\",\"extensions\":[\"gradle\"],\"is_text\":true},\"groovy\":{\"mime_type\":\"text/x-groovy\",\"group\":\"code\",\"description\":\"Groovy source\",\"extensions\":[\"groovy\"],\"is_text\":true},\"gzip\":{\"mime_type\":\"application/gzip\",\"group\":\"archive\",\"description\":\"gzip compressed data\",\"extensions\":[\"gz\",\"gzip\",\"tgz\",\"tar.gz\"],\"is_text\":false},\"h\":{\"mime_type\":\"text/x-c\",\"group\":\"code\",\"description\":\"C header source\",\"extensions\":[\"h\"],\"is_text\":true},\"h5\":{\"mime_type\":\"application/x-hdf5\",\"group\":\"archive\",\"description\":\"Hierarchical Data Format v5\",\"extensions\":[\"h5\",\"hdf5\"],\"is_text\":false},\"handlebars\":{\"mime_type\":\"text/x-handlebars-template\",\"group\":\"code\",\"description\":\"Handlebars source\",\"extensions\":[\"hbs\",\"handlebars\"],\"is_text\":true},\"haskell\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Haskell source\",\"extensions\":[\"hs\",\"lhs\"],\"is_text\":true},\"hcl\":{\"mime_type\":\"text/x-hcl\",\"group\":\"code\",\"description\":\"HashiCorp configuration language\",\"extensions\":[\"hcl\"],\"is_text\":true},\"heif\":{\"mime_type\":\"image/heic\",\"group\":\"image\",\"description\":\"High Efficiency Image File\",\"extensions\":[\"heif\",\"heifs\",\"heic\",\"heics\"],\"is_text\":false},\"hfs\":{\"mime_type\":\"application/x-hfs\",\"group\":null,\"description\":null,\"extensions\":[\"hfs\"],\"is_text\":false},\"hlp\":{\"mime_type\":\"application/winhlp\",\"group\":\"application\",\"description\":\"MS Windows help\",\"extensions\":[\"hlp\"],\"is_text\":false},\"hpp\":{\"mime_type\":\"text/x-h\",\"group\":\"code\",\"description\":null,\"extensions\":[\"hh\",\"hpp\",\"hxx\",\"h++\"],\"is_text\":true},\"hta\":{\"mime_type\":\"application/hta\",\"group\":\"code\",\"description\":\"HTML Application\",\"extensions\":[\"hta\"],\"is_text\":false},\"htaccess\":{\"mime_type\":\"text/x-apache-conf\",\"group\":\"code\",\"description\":\"Apache access configuration\",\"extensions\":[],\"is_text\":true},\"html\":{\"mime_type\":\"text/html\",\"group\":\"code\",\"description\":\"HTML document\",\"extensions\":[\"html\",\"htm\",\"xhtml\",\"xht\"],\"is_text\":true},\"hve\":{\"mime_type\":null,\"group\":\"unknown\",\"description\":null,\"extensions\":[],\"is_text\":false},\"hwp\":{\"mime_type\":\"application/x-hwp\",\"group\":\"document\",\"description\":\"Hangul Word Processor\",\"extensions\":[\"hwp\"],\"is_text\":false},\"icc\":{\"mime_type\":\"application/vnd.iccprofile\",\"group\":null,\"description\":\"ICC profile\",\"extensions\":[\"icc\"],\"is_text\":false},\"icns\":{\"mime_type\":\"image/x-icns\",\"group\":\"image\",\"description\":\"Mac OS X icon\",\"extensions\":[\"icns\"],\"is_text\":false},\"ico\":{\"mime_type\":\"image/vnd.microsoft.icon\",\"group\":\"image\",\"description\":\"MS Windows icon resource\",\"extensions\":[\"ico\"],\"is_text\":false},\"ics\":{\"mime_type\":\"text/calendar\",\"group\":\"application\",\"description\":\"Internet Calendaring and Scheduling\",\"extensions\":[\"ics\"],\"is_text\":true},\"ignorefile\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Ignorefile\",\"extensions\":[],\"is_text\":true},\"img\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"img\"],\"is_text\":false},\"ini\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"INI configuration file\",\"extensions\":[\"ini\"],\"is_text\":true},\"internetshortcut\":{\"mime_type\":\"application/x-mswinurl\",\"group\":\"application\",\"description\":\"MS Windows Internet shortcut\",\"extensions\":[\"url\"],\"is_text\":true},\"iosapp\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"ipynb\":{\"mime_type\":\"application/json\",\"group\":\"code\",\"description\":\"Jupyter notebook\",\"extensions\":[\"ipynb\"],\"is_text\":true},\"iso\":{\"mime_type\":\"application/x-iso9660-image\",\"group\":\"archive\",\"description\":\"ISO 9660 CD-ROM filesystem data\",\"extensions\":[\"iso\"],\"is_text\":false},\"jar\":{\"mime_type\":\"application/java-archive\",\"group\":\"archive\",\"description\":\"Java archive data (JAR)\",\"extensions\":[\"jar\",\"klib\"],\"is_text\":false},\"java\":{\"mime_type\":\"text/x-java\",\"group\":\"code\",\"description\":\"Java source\",\"extensions\":[\"java\"],\"is_text\":true},\"javabytecode\":{\"mime_type\":\"application/x-java-applet\",\"group\":\"executable\",\"description\":\"Java compiled bytecode\",\"extensions\":[\"class\"],\"is_text\":false},\"javascript\":{\"mime_type\":\"application/javascript\",\"group\":\"code\",\"description\":\"JavaScript source\",\"extensions\":[\"js\",\"mjs\",\"cjs\"],\"is_text\":true},\"jinja\":{\"mime_type\":\"text/x-jinja2-template\",\"group\":\"code\",\"description\":\"Jinja template\",\"extensions\":[\"jinja\",\"jinja2\",\"j2\"],\"is_text\":true},\"jng\":{\"mime_type\":\"image/jng\",\"group\":\"image\",\"description\":\"JPEG network graphics\",\"extensions\":[\"jng\"],\"is_text\":false},\"jnlp\":{\"mime_type\":\"application/x-java-jnlp-file\",\"group\":\"code\",\"description\":\"Java Network Launch Protocol\",\"extensions\":[\"jnlp\"],\"is_text\":true},\"jp2\":{\"mime_type\":\"image/jpeg2000\",\"group\":\"image\",\"description\":\"jpeg2000\",\"extensions\":[\"jp2\"],\"is_text\":false},\"jpeg\":{\"mime_type\":\"image/jpeg\",\"group\":\"image\",\"description\":\"JPEG image data\",\"extensions\":[\"jpg\",\"jpeg\"],\"is_text\":false},\"json\":{\"mime_type\":\"application/json\",\"group\":\"code\",\"description\":\"JSON document\",\"extensions\":[\"json\"],\"is_text\":true},\"jsonc\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"jsonl\":{\"mime_type\":\"application/json\",\"group\":\"code\",\"description\":\"JSONL document\",\"extensions\":[\"jsonl\",\"jsonld\"],\"is_text\":true},\"jsx\":{\"mime_type\":\"application/javascript\",\"group\":\"code\",\"description\":\"JSX source\",\"extensions\":[\"jsx\",\"mjsx\",\"cjsx\"],\"is_text\":true},\"julia\":{\"mime_type\":\"text/x-julia\",\"group\":\"code\",\"description\":\"Julia source\",\"extensions\":[\"jl\"],\"is_text\":true},\"jxl\":{\"mime_type\":\"image/jxl\",\"group\":\"image\",\"description\":\"JPEG XL\",\"extensions\":[\"jxl\"],\"is_text\":false},\"ko\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ELF executable, kernel object\",\"extensions\":[\"ko\"],\"is_text\":false},\"kotlin\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Kotlin source\",\"extensions\":[\"kt\",\"kts\"],\"is_text\":true},\"ks\":{\"mime_type\":null,\"group\":null,\"description\":\"Tyrano\",\"extensions\":[\"ks\"],\"is_text\":true},\"latex\":{\"mime_type\":\"text/x-tex\",\"group\":\"text\",\"description\":\"LaTeX document\",\"extensions\":[\"tex\",\"sty\"],\"is_text\":true},\"latexaux\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"aux\"],\"is_text\":false},\"less\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"lha\":{\"mime_type\":\"application/x-lha\",\"group\":\"archive\",\"description\":\"LHarc archive\",\"extensions\":[\"lha\",\"lzh\"],\"is_text\":false},\"license\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"License file\",\"extensions\":[],\"is_text\":true},\"lisp\":{\"mime_type\":\"text/x-lisp\",\"group\":\"code\",\"description\":\"Lisp source\",\"extensions\":[\"lisp\",\"lsp\",\"l\",\"cl\"],\"is_text\":true},\"litcs\":{\"mime_type\":null,\"group\":null,\"description\":\"Literate CS\",\"extensions\":[\"litcoffee\"],\"is_text\":false},\"lnk\":{\"mime_type\":\"application/x-ms-shortcut\",\"group\":\"application\",\"description\":\"MS Windows shortcut\",\"extensions\":[\"lnk\"],\"is_text\":false},\"lock\":{\"mime_type\":\"text/plain\",\"group\":\"application\",\"description\":\"Lock file\",\"extensions\":[\"lock\"],\"is_text\":true},\"lrz\":{\"mime_type\":\"application/x-lrzip\",\"group\":null,\"description\":\"LRZip\",\"extensions\":[\"lrz\"],\"is_text\":false},\"lua\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Lua\",\"extensions\":[\"lua\"],\"is_text\":true},\"lz\":{\"mime_type\":\"application/x-lzip\",\"group\":\"archive\",\"description\":\"LZip\",\"extensions\":[\"lz\"],\"is_text\":false},\"lz4\":{\"mime_type\":\"application/x-lz4\",\"group\":\"archive\",\"description\":\"LZ4\",\"extensions\":[\"lz4\"],\"is_text\":false},\"lzx\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"m3u\":{\"mime_type\":\"text/plain\",\"group\":\"application\",\"description\":\"M3U playlist\",\"extensions\":[\"m3u8\",\"m3u\"],\"is_text\":true},\"m4\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"GNU Macro\",\"extensions\":[\"m4\"],\"is_text\":true},\"macho\":{\"mime_type\":\"application/x-mach-o\",\"group\":\"executable\",\"description\":\"Mach-O executable\",\"extensions\":[],\"is_text\":false},\"maff\":{\"mime_type\":\"application/x-maff\",\"group\":null,\"description\":null,\"extensions\":[\"maff\"],\"is_text\":false},\"makefile\":{\"mime_type\":\"text/x-makefile\",\"group\":\"code\",\"description\":\"Makefile source\",\"extensions\":[],\"is_text\":true},\"markdown\":{\"mime_type\":\"text/markdown\",\"group\":\"text\",\"description\":\"Markdown document\",\"extensions\":[\"md\",\"markdown\"],\"is_text\":true},\"matlab\":{\"mime_type\":\"text/x-matlab\",\"group\":\"code\",\"description\":\"Matlab Source\",\"extensions\":[\"m\",\"matlab\"],\"is_text\":true},\"mht\":{\"mime_type\":\"application/x-mimearchive\",\"group\":\"code\",\"description\":\"MHTML document\",\"extensions\":[\"mht\"],\"is_text\":true},\"midi\":{\"mime_type\":\"audio/midi\",\"group\":\"audio\",\"description\":\"Midi\",\"extensions\":[\"mid\"],\"is_text\":false},\"mkv\":{\"mime_type\":\"video/x-matroska\",\"group\":\"video\",\"description\":\"Matroska\",\"extensions\":[\"mkv\"],\"is_text\":false},\"mp2\":{\"mime_type\":null,\"group\":null,\"description\":\"MP2 stream\",\"extensions\":[\"mp2\"],\"is_text\":false},\"mp3\":{\"mime_type\":\"audio/mpeg\",\"group\":\"audio\",\"description\":\"MP3 media file\",\"extensions\":[\"mp3\"],\"is_text\":false},\"mp4\":{\"mime_type\":\"video/mp4\",\"group\":\"video\",\"description\":\"MP4 media file\",\"extensions\":[\"mp4\"],\"is_text\":false},\"mpegts\":{\"mime_type\":\"video/MP2T\",\"group\":\"video\",\"description\":\"MPEG Transport stream\",\"extensions\":[\"ts\",\"tsv\",\"tsa\",\"m2t\"],\"is_text\":false},\"mscompress\":{\"mime_type\":\"application/x-ms-compress-szdd\",\"group\":\"archive\",\"description\":\"MS Compress archive data\",\"extensions\":[],\"is_text\":false},\"msi\":{\"mime_type\":\"application/x-msi\",\"group\":\"archive\",\"description\":\"Microsoft Installer file\",\"extensions\":[\"msi\"],\"is_text\":false},\"msix\":{\"mime_type\":\"application/msix\",\"group\":\"application\",\"description\":\"Windows app package\",\"extensions\":[\"msix\"],\"is_text\":false},\"mst\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"mst\"],\"is_text\":false},\"mui\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"application\",\"description\":\"PE Windows executable\",\"extensions\":[\"mui\"],\"is_text\":false},\"mum\":{\"mime_type\":\"text/xml\",\"group\":\"application\",\"description\":\"Windows Update Package file\",\"extensions\":[\"mum\"],\"is_text\":true},\"mun\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"mun\"],\"is_text\":false},\"nim\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"npy\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Numpy Array\",\"extensions\":[\"npy\"],\"is_text\":false},\"npz\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Numpy Arrays Archive\",\"extensions\":[\"npz\"],\"is_text\":false},\"null\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"null\"],\"is_text\":false},\"nupkg\":{\"mime_type\":\"application/octet-stream\",\"group\":null,\"description\":\"NuGet Package\",\"extensions\":[\"nupkg\"],\"is_text\":false},\"object\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"o\"],\"is_text\":false},\"objectivec\":{\"mime_type\":\"text/x-objcsrc\",\"group\":\"code\",\"description\":\"ObjectiveC source\",\"extensions\":[\"m\",\"mm\"],\"is_text\":true},\"ocaml\":{\"mime_type\":\"text-ocaml\",\"group\":\"code\",\"description\":\"OCaml\",\"extensions\":[\"ml\",\"mli\"],\"is_text\":true},\"ocx\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"ocx\"],\"is_text\":false},\"odex\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ODEX ELF executable\",\"extensions\":[\"odex\"],\"is_text\":false},\"odin\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"Odin\",\"extensions\":[\"odin\"],\"is_text\":true},\"odp\":{\"mime_type\":\"application/vnd.oasis.opendocument.presentation\",\"group\":\"document\",\"description\":\"OpenDocument Presentation\",\"extensions\":[\"odp\"],\"is_text\":false},\"ods\":{\"mime_type\":\"application/vnd.oasis.opendocument.spreadsheet\",\"group\":\"document\",\"description\":\"OpenDocument Spreadsheet\",\"extensions\":[\"ods\"],\"is_text\":false},\"odt\":{\"mime_type\":\"application/vnd.oasis.opendocument.text\",\"group\":\"document\",\"description\":\"OpenDocument Text\",\"extensions\":[\"odt\"],\"is_text\":false},\"ogg\":{\"mime_type\":\"audio/ogg\",\"group\":\"audio\",\"description\":\"Ogg data\",\"extensions\":[\"ogg\"],\"is_text\":false},\"ole\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"one\":{\"mime_type\":\"application/msonenote\",\"group\":\"document\",\"description\":\"One Note\",\"extensions\":[\"one\"],\"is_text\":false},\"onnx\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Open Neural Network Exchange\",\"extensions\":[\"onnx\"],\"is_text\":false},\"ooxml\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"otf\":{\"mime_type\":\"font/otf\",\"group\":\"font\",\"description\":\"OpenType font\",\"extensions\":[\"otf\"],\"is_text\":false},\"outlook\":{\"mime_type\":\"application/vnd.ms-outlook\",\"group\":\"application\",\"description\":\"MS Outlook Message\",\"extensions\":[],\"is_text\":false},\"palmos\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"parquet\":{\"mime_type\":\"application/vnd.apache.parquet\",\"group\":\"unknown\",\"description\":\"Apache Parquet\",\"extensions\":[\"pqt\",\"parquet\"],\"is_text\":false},\"pascal\":{\"mime_type\":\"text/x-pascal\",\"group\":\"code\",\"description\":\"Pascal source\",\"extensions\":[\"pas\",\"pp\"],\"is_text\":true},\"pbm\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"pcap\":{\"mime_type\":\"application/vnd.tcpdump.pcap\",\"group\":\"application\",\"description\":\"pcap capture file\",\"extensions\":[\"pcap\",\"pcapng\"],\"is_text\":false},\"pdb\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Windows Program Database\",\"extensions\":[\"pdb\"],\"is_text\":false},\"pdf\":{\"mime_type\":\"application/pdf\",\"group\":\"document\",\"description\":\"PDF document\",\"extensions\":[\"pdf\"],\"is_text\":false},\"pebin\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"exe\",\"dll\"],\"is_text\":false},\"pem\":{\"mime_type\":\"application/x-pem-file\",\"group\":\"application\",\"description\":\"PEM certificate\",\"extensions\":[\"pem\",\"pub\",\"gpg\"],\"is_text\":true},\"perl\":{\"mime_type\":\"text/x-perl\",\"group\":\"code\",\"description\":\"Perl source\",\"extensions\":[\"pl\"],\"is_text\":true},\"pgp\":{\"mime_type\":\"application/pgp-keys\",\"group\":null,\"description\":\"PGP\",\"extensions\":[\"gpg\",\"pgp\"],\"is_text\":false},\"php\":{\"mime_type\":\"text/x-php\",\"group\":\"code\",\"description\":\"PHP source\",\"extensions\":[\"php\"],\"is_text\":true},\"pickle\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Python pickle\",\"extensions\":[\"pickle\",\"pkl\"],\"is_text\":false},\"png\":{\"mime_type\":\"image/png\",\"group\":\"image\",\"description\":\"PNG image\",\"extensions\":[\"png\"],\"is_text\":false},\"po\":{\"mime_type\":\"text/gettext-translation\",\"group\":\"application\",\"description\":\"Portable Object (PO) for i18n\",\"extensions\":[\"po\"],\"is_text\":true},\"postscript\":{\"mime_type\":\"application/postscript\",\"group\":\"document\",\"description\":\"PostScript document\",\"extensions\":[\"ps\"],\"is_text\":false},\"powershell\":{\"mime_type\":\"application/x-powershell\",\"group\":\"code\",\"description\":\"Powershell source\",\"extensions\":[\"ps1\"],\"is_text\":true},\"ppt\":{\"mime_type\":\"application/vnd.ms-powerpoint\",\"group\":\"document\",\"description\":\"Microsoft PowerPoint CDF document\",\"extensions\":[\"ppt\"],\"is_text\":false},\"pptx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.presentationml.presentation\",\"group\":\"document\",\"description\":\"Microsoft PowerPoint 2007+ document\",\"extensions\":[\"pptx\",\"pptm\"],\"is_text\":false},\"printfox\":{\"mime_type\":null,\"group\":null,\"description\":\"c64\",\"extensions\":[],\"is_text\":false},\"prolog\":{\"mime_type\":\"text/x-prolog\",\"group\":\"code\",\"description\":\"Prolog source\",\"extensions\":[\"pl\",\"pro\",\"P\"],\"is_text\":true},\"proteindb\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Protein DB\",\"extensions\":[\"pdb\"],\"is_text\":true},\"proto\":{\"mime_type\":\"text/x-proto\",\"group\":\"code\",\"description\":\"Protocol buffer definition\",\"extensions\":[\"proto\"],\"is_text\":true},\"protobuf\":{\"mime_type\":\"application/protobuf\",\"group\":\"unknown\",\"description\":\"Protocol buffers\",\"extensions\":[\"protobuf\",\"pb\"],\"is_text\":false},\"psd\":{\"mime_type\":\"image/vnd.adobe.photoshop\",\"group\":\"image\",\"description\":\"Adobe Photoshop\",\"extensions\":[\"psd\"],\"is_text\":false},\"pytorch\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Pytorch storage file\",\"extensions\":[\"pt\",\"pth\"],\"is_text\":false},\"pub\":{\"mime_type\":\"application/x-mspublisher\",\"group\":null,\"description\":null,\"extensions\":[\"pub\"],\"is_text\":false},\"python\":{\"mime_type\":\"text/x-python\",\"group\":\"code\",\"description\":\"Python source\",\"extensions\":[\"py\",\"pyi\"],\"is_text\":true},\"pythonbytecode\":{\"mime_type\":\"application/x-bytecode.python\",\"group\":\"executable\",\"description\":\"Python compiled bytecode\",\"extensions\":[\"pyc\",\"pyo\"],\"is_text\":false},\"pythonpar\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"par\"],\"is_text\":false},\"qoi\":{\"mime_type\":\"image/x-qoi\",\"group\":\"image\",\"description\":\"Quite Ok Image\",\"extensions\":[\"qoi\"],\"is_text\":false},\"qt\":{\"mime_type\":\"video/quicktime\",\"group\":\"video\",\"description\":\"QuickTime\",\"extensions\":[\"mov\"],\"is_text\":false},\"r\":{\"mime_type\":\"text/x-R\",\"group\":\"code\",\"description\":\"R (language)\",\"extensions\":[\"R\"],\"is_text\":true},\"randomascii\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Random ASCII characters\",\"extensions\":[],\"is_text\":true},\"randombytes\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Random bytes\",\"extensions\":[],\"is_text\":false},\"randomtxt\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Random text\",\"extensions\":[],\"is_text\":true},\"rar\":{\"mime_type\":\"application/x-rar\",\"group\":\"archive\",\"description\":\"RAR archive data\",\"extensions\":[\"rar\"],\"is_text\":false},\"rdf\":{\"mime_type\":\"application/rdf+xml\",\"group\":\"text\",\"description\":\"Resource Description Framework document (RDF)\",\"extensions\":[\"rdf\"],\"is_text\":true},\"rdp\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"riff\":{\"mime_type\":\"application/x-riff\",\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"rlib\":{\"mime_type\":\"application/x-archive\",\"group\":\"archive\",\"description\":\"rust library\",\"extensions\":[\"rlib\"],\"is_text\":false},\"rll\":{\"mime_type\":null,\"group\":\"executable\",\"description\":\"Resource Library\",\"extensions\":[\"rll\"],\"is_text\":false},\"rpm\":{\"mime_type\":\"application/x-rpm\",\"group\":\"archive\",\"description\":\"RedHat Package Manager archive (RPM)\",\"extensions\":[\"rpm\"],\"is_text\":false},\"rst\":{\"mime_type\":\"text/x-rst\",\"group\":\"text\",\"description\":\"ReStructuredText document\",\"extensions\":[\"rst\"],\"is_text\":true},\"rtf\":{\"mime_type\":\"text/rtf\",\"group\":\"text\",\"description\":\"Rich Text Format document\",\"extensions\":[\"rtf\"],\"is_text\":true},\"ruby\":{\"mime_type\":\"application/x-ruby\",\"group\":\"code\",\"description\":\"Ruby source\",\"extensions\":[\"rb\"],\"is_text\":true},\"rust\":{\"mime_type\":\"application/x-rust\",\"group\":\"code\",\"description\":\"Rust source\",\"extensions\":[\"rs\"],\"is_text\":true},\"rzip\":{\"mime_type\":null,\"group\":null,\"description\":\"Rzip\",\"extensions\":[\"rz\"],\"is_text\":false},\"scala\":{\"mime_type\":\"application/x-scala\",\"group\":\"code\",\"description\":\"Scala source\",\"extensions\":[\"scala\"],\"is_text\":true},\"scheme\":{\"mime_type\":\"text/x-scheme\",\"group\":\"code\",\"description\":null,\"extensions\":[\"scm\",\"ss\"],\"is_text\":false},\"scr\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"scr\"],\"is_text\":false},\"scriptwsf\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"scss\":{\"mime_type\":\"text/x-scss\",\"group\":\"code\",\"description\":\"SCSS source\",\"extensions\":[\"scss\"],\"is_text\":true},\"sevenzip\":{\"mime_type\":\"application/x-7z-compressed\",\"group\":\"archive\",\"description\":\"7-zip archive data\",\"extensions\":[\"7z\"],\"is_text\":false},\"sgml\":{\"mime_type\":\"application/sgml\",\"group\":\"text\",\"description\":\"sgml\",\"extensions\":[\"sgml\"],\"is_text\":true},\"sh3d\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"sh3d\"],\"is_text\":false},\"shell\":{\"mime_type\":\"text/x-shellscript\",\"group\":\"code\",\"description\":\"Shell script\",\"extensions\":[\"sh\"],\"is_text\":true},\"smali\":{\"mime_type\":\"application/x-smali\",\"group\":\"code\",\"description\":\"Smali source\",\"extensions\":[\"smali\"],\"is_text\":true},\"snap\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Snap archive\",\"extensions\":[\"snap\"],\"is_text\":false},\"so\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ELF executable, shared library\",\"extensions\":[\"so\"],\"is_text\":false},\"solidity\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"Solidity source\",\"extensions\":[\"sol\"],\"is_text\":true},\"sql\":{\"mime_type\":\"application/x-sql\",\"group\":\"code\",\"description\":\"SQL source\",\"extensions\":[\"sql\"],\"is_text\":true},\"sqlite\":{\"mime_type\":null,\"group\":\"application\",\"description\":\"SQLITE database\",\"extensions\":[\"sqlite\",\"sqlite3\"],\"is_text\":false},\"squashfs\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Squash filesystem\",\"extensions\":[],\"is_text\":false},\"srt\":{\"mime_type\":\"text/srt\",\"group\":\"application\",\"description\":\"SubRip Text Format\",\"extensions\":[\"srt\"],\"is_text\":true},\"stlbinary\":{\"mime_type\":\"application/sla\",\"group\":\"image\",\"description\":\"Stereolithography CAD (binary)\",\"extensions\":[\"stl\"],\"is_text\":false},\"stltext\":{\"mime_type\":\"application/sla\",\"group\":\"image\",\"description\":\"Stereolithography CAD (text)\",\"extensions\":[\"stl\"],\"is_text\":true},\"sum\":{\"mime_type\":null,\"group\":\"application\",\"description\":\"Checksum file\",\"extensions\":[\"sum\"],\"is_text\":true},\"svd\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"svg\":{\"mime_type\":\"image/svg+xml\",\"group\":\"image\",\"description\":\"SVG Scalable Vector Graphics image data\",\"extensions\":[\"svg\"],\"is_text\":true},\"swf\":{\"mime_type\":\"application/x-shockwave-flash\",\"group\":\"executable\",\"description\":\"Small Web File\",\"extensions\":[\"swf\"],\"is_text\":false},\"swift\":{\"mime_type\":\"text/x-swift\",\"group\":\"code\",\"description\":\"Swift\",\"extensions\":[\"swift\"],\"is_text\":true},\"symlink\":{\"mime_type\":\"inode/symlink\",\"group\":\"inode\",\"description\":\"Symbolic link\",\"extensions\":[],\"is_text\":false},\"symlinktext\":{\"mime_type\":\"text/plain\",\"group\":\"application\",\"description\":\"Symbolic link (textual representation)\",\"extensions\":[],\"is_text\":true},\"sys\":{\"mime_type\":\"application/x-windows-driver\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"sys\"],\"is_text\":false},\"tar\":{\"mime_type\":\"application/x-tar\",\"group\":\"archive\",\"description\":\"POSIX tar archive\",\"extensions\":[\"tar\"],\"is_text\":false},\"tcl\":{\"mime_type\":\"application/x-tcl\",\"group\":\"code\",\"description\":\"Tickle\",\"extensions\":[\"tcl\"],\"is_text\":true},\"textproto\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Text protocol buffer\",\"extensions\":[\"textproto\",\"textpb\",\"pbtxt\"],\"is_text\":true},\"tga\":{\"mime_type\":\"image/x-tga\",\"group\":\"image\",\"description\":\"Targa image data\",\"extensions\":[\"tga\"],\"is_text\":false},\"thumbsdb\":{\"mime_type\":\"image/vnd.ms-thumb\",\"group\":\"application\",\"description\":\"Windows thumbnail cache\",\"extensions\":[],\"is_text\":false},\"tiff\":{\"mime_type\":\"image/tiff\",\"group\":\"image\",\"description\":\"TIFF image data\",\"extensions\":[\"tiff\",\"tif\"],\"is_text\":false},\"tmdx\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"tmdx\",\"tmvx\"],\"is_text\":false},\"toml\":{\"mime_type\":\"application/toml\",\"group\":\"text\",\"description\":\"Tom's obvious, minimal language\",\"extensions\":[\"toml\"],\"is_text\":true},\"torrent\":{\"mime_type\":\"application/x-bittorrent\",\"group\":\"application\",\"description\":\"BitTorrent file\",\"extensions\":[\"torrent\"],\"is_text\":false},\"troff\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"tsv\":{\"mime_type\":\"text/tsv\",\"group\":\"code\",\"description\":\"TSV document\",\"extensions\":[\"tsv\"],\"is_text\":true},\"tsx\":{\"mime_type\":\"text/x-typescript\",\"group\":\"code\",\"description\":\"TSX source\",\"extensions\":[\"tsx\",\"mtsx\",\"ctsx\"],\"is_text\":true},\"ttf\":{\"mime_type\":\"font/sfnt\",\"group\":\"font\",\"description\":\"TrueType Font data\",\"extensions\":[\"ttf\",\"ttc\"],\"is_text\":false},\"twig\":{\"mime_type\":\"text/x-twig\",\"group\":\"code\",\"description\":\"Twig template\",\"extensions\":[\"twig\"],\"is_text\":true},\"txt\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document\",\"extensions\":[\"txt\"],\"is_text\":true},\"txtascii\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document encoded in ASCII\",\"extensions\":[\"txt\"],\"is_text\":true},\"txtutf16\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document encoded in UTF-16\",\"extensions\":[\"txt\"],\"is_text\":true},\"txtutf8\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document encoded in UTF-8\",\"extensions\":[\"txt\"],\"is_text\":true},\"typescript\":{\"mime_type\":\"application/typescript\",\"group\":\"code\",\"description\":\"TypeScript source\",\"extensions\":[\"ts\",\"mts\",\"cts\"],\"is_text\":true},\"udf\":{\"mime_type\":\"application/x-udf-image\",\"group\":null,\"description\":\"Universal Disc Format\",\"extensions\":[],\"is_text\":false},\"undefined\":{\"mime_type\":\"application/undefined\",\"group\":\"undefined\",\"description\":\"Undefined\",\"extensions\":[],\"is_text\":false},\"unixcompress\":{\"mime_type\":\"application/x-compress\",\"group\":null,\"description\":null,\"extensions\":[\"z\"],\"is_text\":false},\"unknown\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Unknown binary data\",\"extensions\":[],\"is_text\":false},\"vba\":{\"mime_type\":\"text/vbscript\",\"group\":\"code\",\"description\":\"MS Visual Basic source (VBA)\",\"extensions\":[\"vbs\",\"vba\",\"vb\"],\"is_text\":true},\"vbe\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"EncryptedVBS\",\"extensions\":[\"vbe\"],\"is_text\":false},\"vcard\":{\"mime_type\":\"text/vcard\",\"group\":null,\"description\":null,\"extensions\":[\"vcard\"],\"is_text\":false},\"vcs\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"vcxproj\":{\"mime_type\":\"application/xml\",\"group\":\"code\",\"description\":\"Visual Studio MSBuild project\",\"extensions\":[\"vcxproj\"],\"is_text\":true},\"verilog\":{\"mime_type\":\"text/x-verilog\",\"group\":\"code\",\"description\":\"Verilog source\",\"extensions\":[\"v\",\"verilog\",\"vlg\",\"vh\"],\"is_text\":true},\"vhd\":{\"mime_type\":\"application/x-vhd\",\"group\":null,\"description\":\"Virtual Hard Disk\",\"extensions\":[],\"is_text\":false},\"vhdl\":{\"mime_type\":\"text/x-vhdl\",\"group\":\"code\",\"description\":\"VHDL source\",\"extensions\":[\"vhd\"],\"is_text\":true},\"visio\":{\"mime_type\":\"application/vnd.ms-visio.drawing.main+xml\",\"group\":\"document\",\"description\":\"Microsoft Visio\",\"extensions\":[\"vsd\",\"vsdm\",\"vsdx\",\"vdw\"],\"is_text\":false},\"vtt\":{\"mime_type\":\"text/vtt\",\"group\":\"text\",\"description\":\"Web Video Text Tracks\",\"extensions\":[\"vtt\",\"webvtt\"],\"is_text\":true},\"vue\":{\"mime_type\":\"application/javascript\",\"group\":\"code\",\"description\":\"Vue source\",\"extensions\":[\"vue\"],\"is_text\":true},\"wad\":{\"mime_type\":\"application/wad\",\"group\":\"archive\",\"description\":\"WAD\",\"extensions\":[\"wad\"],\"is_text\":false},\"wasm\":{\"mime_type\":\"application/wasm\",\"group\":\"executable\",\"description\":\"Web Assembly\",\"extensions\":[\"wasm\"],\"is_text\":false},\"wav\":{\"mime_type\":\"audio/x-wav\",\"group\":\"audio\",\"description\":\"Waveform Audio file (WAV)\",\"extensions\":[\"wav\"],\"is_text\":false},\"webm\":{\"mime_type\":\"video/webm\",\"group\":\"video\",\"description\":\"WebM media file\",\"extensions\":[\"webm\"],\"is_text\":false},\"webp\":{\"mime_type\":\"image/webp\",\"group\":\"image\",\"description\":\"WebP media file\",\"extensions\":[\"webp\"],\"is_text\":false},\"webtemplate\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Web templating language\",\"extensions\":[],\"is_text\":true},\"wim\":{\"mime_type\":\"application/x-ms-wim\",\"group\":\"unknown\",\"description\":\"Windows Imaging Format\",\"extensions\":[\"wim\",\"swm\",\"esd\"],\"is_text\":false},\"winregistry\":{\"mime_type\":\"text/x-ms-regedit\",\"group\":\"application\",\"description\":\"Windows Registry text\",\"extensions\":[\"reg\"],\"is_text\":true},\"wma\":{\"mime_type\":\"audio/x-ms-wma\",\"group\":\"audio\",\"description\":\"Windows Media Audio\",\"extensions\":[\"wma\"],\"is_text\":false},\"wmf\":{\"mime_type\":\"image/wmf\",\"group\":\"image\",\"description\":\"Windows metafile\",\"extensions\":[\"wmf\"],\"is_text\":false},\"wmv\":{\"mime_type\":\"video/x-ms-wmv\",\"group\":\"video\",\"description\":\"Windows Media Video\",\"extensions\":[\"wmv\"],\"is_text\":false},\"woff\":{\"mime_type\":\"font/woff\",\"group\":\"font\",\"description\":\"Web Open Font Format\",\"extensions\":[\"woff\"],\"is_text\":false},\"woff2\":{\"mime_type\":\"font/woff2\",\"group\":\"font\",\"description\":\"Web Open Font Format v2\",\"extensions\":[\"woff2\"],\"is_text\":false},\"xar\":{\"mime_type\":\"application/x-xar\",\"group\":\"archive\",\"description\":\"XAR archive compressed data\",\"extensions\":[\"pkg\",\"xar\"],\"is_text\":false},\"xcf\":{\"mime_type\":\"image/x-xcf\",\"group\":\"image\",\"description\":\"Gimp image\",\"extensions\":[\"xcf\"],\"is_text\":false},\"xls\":{\"mime_type\":\"application/vnd.ms-excel\",\"group\":\"document\",\"description\":\"Microsoft Excel CDF document\",\"extensions\":[\"xls\"],\"is_text\":false},\"xlsb\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\"group\":\"document\",\"description\":\"Microsoft Excel 2007+ document (binary format)\",\"extensions\":[\"xlsb\"],\"is_text\":false},\"xlsx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\"group\":\"document\",\"description\":\"Microsoft Excel 2007+ document\",\"extensions\":[\"xlsx\",\"xlsm\"],\"is_text\":false},\"xml\":{\"mime_type\":\"text/xml\",\"group\":\"code\",\"description\":\"XML document\",\"extensions\":[\"xml\"],\"is_text\":true},\"xpi\":{\"mime_type\":\"application/zip\",\"group\":\"archive\",\"description\":\"Compressed installation archive (XPI)\",\"extensions\":[\"xpi\"],\"is_text\":false},\"xsd\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"xsd\"],\"is_text\":false},\"xz\":{\"mime_type\":\"application/x-xz\",\"group\":\"archive\",\"description\":\"XZ compressed data\",\"extensions\":[\"xz\"],\"is_text\":false},\"yaml\":{\"mime_type\":\"application/x-yaml\",\"group\":\"code\",\"description\":\"YAML source\",\"extensions\":[\"yml\",\"yaml\"],\"is_text\":true},\"yara\":{\"mime_type\":\"text/x-yara\",\"group\":\"code\",\"description\":\"YARA rule\",\"extensions\":[\"yar\",\"yara\"],\"is_text\":true},\"zig\":{\"mime_type\":\"text/zig\",\"group\":\"code\",\"description\":\"Zig source\",\"extensions\":[\"zig\"],\"is_text\":true},\"zip\":{\"mime_type\":\"application/zip\",\"group\":\"archive\",\"description\":\"Zip archive data\",\"extensions\":[\"zip\"],\"is_text\":false},\"zlibstream\":{\"mime_type\":\"application/zlib\",\"group\":\"application\",\"description\":\"zlib compressed data\",\"extensions\":[],\"is_text\":false},\"zst\":{\"mime_type\":\"application/zstd\",\"group\":\"archive\",\"description\":\"Zstandard\",\"extensions\":[\"zst\"],\"is_text\":false}}" }, { "path": "assets/models/CHANGELOG.md", "content": "# Changelog\n\nHere we document the main changes of the various models.\n\nIndicated inference speed calculated by averaging 100 inferences (within one invocation) on an AMD Ryzen 9 7950X 16-Core Processor CPU.\n\n## `standard_v3_3` - 2025-04-11\n\n- [216 possible tool's outputs](./standard_v3_3/README.md), ~99% average accuracy, ~2ms inference speed.\n- Better dataset balance between javascript vs. typescript (leading to an increased accuracy for typescript, 85% => 95%).\n- New synthetic datasets with utf8-encoded, non-ascii characters for simple text and JSON.\n- More thresholds tuning.\n\n## `standard_v3_2` - 2025-03-17\n\n- [216 possible tool's outputs](./standard_v3_2/README.md), ~99% average accuracy, ~2ms inference speed.\n- Difference with respect `standard_v3_1`: trained on a new (synthetic) dataset of CSV files to address a regression with CSV files (https://github.com/google/magika/issues/983); model selection now uses minimal test loss instead of other heuristics.\n\n## `standard_v3_1`\n\n- [216 possible tool's outputs](./standard_v3_1/README.md).\n- Overall same average accuracy of `standard_v3_0`, ~99%, but more robust detections of short textual input and improved detection of Javascript.\n- Inference speed: ~2ms (similar to `standard_v3_0`).\n- Augmentation techniques used during training: CutMix, which was used for `v1` but not for `v2_1`; and \"Random Snippet Selection\", with which we train the model with random snippets extracted from samples in our dataset (this is only enabled for key textual content types).\n- Tweaked balance among content types in training dataset.\n\n## `standard_v3_0`\n\n- [216 possible tool's outputs](./standard_v3_0/README.md).\n- Overall same average accuracy of `standard_v2_1`, ~99%.\n- Inference speed: ~2ms (~3x faster than `standard_v2_1`, ~20% faster than `standard_v1`).\n\n## `standard_v2_1`\n\n- [Support for 200+ content types](./standard_v2_1/README.md), almost double what supported in `standard_v1`.\n- Overall average accuracy of ~99%.\n- Inference speed: ~6.2ms, which is slower than `standard_v1`; See `fast_v2_1` in case you need something faster (at the price of less accuracy).\n\n## `fast_v2_1`\n\n- Similar to `standard_v2_1`, but significantly faster (about 4x faster).\n- Overall average accuracy of ~98.5%.\n\n## `standard_v1`\n\n- Initial release.\n- Support for about 100 content types.\n- Average accuracy 99%+.\n- Inference speed: ~2.6ms.\n" }, { "path": "assets/models/begonly_v2_1/config.min.json", "content": "{\"beg_size\": 2048, \"mid_size\": 0, \"end_size\": 0, \"use_inputs_at_offsets\": false, \"medium_confidence_threshold\": 0.5, \"min_file_size_for_dl\": 8, \"padding_token\": 256, \"block_size\": 4096, \"target_labels_space\": [\"3gp\", \"ace\", \"ai\", \"aidl\", \"apk\", \"applebplist\", \"appleplist\", \"asm\", \"asp\", \"autohotkey\", \"autoit\", \"awk\", \"batch\", \"bazel\", \"bib\", \"bmp\", \"bzip\", \"c\", \"cab\", \"cat\", \"chm\", \"clojure\", \"cmake\", \"cobol\", \"coff\", \"coffeescript\", \"cpp\", \"crt\", \"crx\", \"cs\", \"csproj\", \"css\", \"csv\", \"dart\", \"deb\", \"dex\", \"dicom\", \"diff\", \"dm\", \"dmg\", \"doc\", \"dockerfile\", \"docx\", \"dsstore\", \"dwg\", \"dxf\", \"elf\", \"elixir\", \"emf\", \"eml\", \"epub\", \"erb\", \"erlang\", \"flac\", \"flv\", \"fortran\", \"gemfile\", \"gemspec\", \"gif\", \"gitattributes\", \"gitmodules\", \"go\", \"gradle\", \"groovy\", \"gzip\", \"h5\", \"handlebars\", \"haskell\", \"hcl\", \"hlp\", \"htaccess\", \"html\", \"icns\", \"ico\", \"ics\", \"ignorefile\", \"ini\", \"internetshortcut\", \"ipynb\", \"iso\", \"jar\", \"java\", \"javabytecode\", \"javascript\", \"jinja\", \"jp2\", \"jpeg\", \"json\", \"jsonl\", \"julia\", \"kotlin\", \"latex\", \"lha\", \"lisp\", \"lnk\", \"lua\", \"m3u\", \"m4\", \"macho\", \"makefile\", \"markdown\", \"matlab\", \"mht\", \"midi\", \"mkv\", \"mp3\", \"mp4\", \"mscompress\", \"msi\", \"mum\", \"npy\", \"npz\", \"nupkg\", \"objectivec\", \"ocaml\", \"odp\", \"ods\", \"odt\", \"ogg\", \"one\", \"onnx\", \"otf\", \"outlook\", \"parquet\", \"pascal\", \"pcap\", \"pdb\", \"pdf\", \"pebin\", \"pem\", \"perl\", \"php\", \"pickle\", \"png\", \"po\", \"postscript\", \"powershell\", \"ppt\", \"pptx\", \"prolog\", \"proteindb\", \"proto\", \"psd\", \"python\", \"pythonbytecode\", \"pytorch\", \"qt\", \"r\", \"rar\", \"rdf\", \"rpm\", \"rst\", \"rtf\", \"ruby\", \"rust\", \"scala\", \"scss\", \"sevenzip\", \"sgml\", \"shell\", \"smali\", \"snap\", \"solidity\", \"sql\", \"sqlite\", \"squashfs\", \"srt\", \"stlbinary\", \"stltext\", \"sum\", \"svg\", \"swf\", \"swift\", \"tar\", \"tcl\", \"textproto\", \"tga\", \"thumbsdb\", \"tiff\", \"toml\", \"torrent\", \"tsv\", \"ttf\", \"twig\", \"txt\", \"typescript\", \"unknown\", \"vba\", \"vcxproj\", \"verilog\", \"vhdl\", \"vtt\", \"vue\", \"wasm\", \"wav\", \"webm\", \"webp\", \"winregistry\", \"wmf\", \"woff\", \"woff2\", \"xar\", \"xls\", \"xlsb\", \"xlsx\", \"xml\", \"xpi\", \"xz\", \"yaml\", \"yara\", \"zig\", \"zip\", \"zlibstream\"], \"thresholds\": {\"latex\": 0.95, \"pascal\": 0.95}, \"overwrite_map\": {}}\n" }, { "path": "assets/models/begonly_v2_1/metadata.json", "content": "{\"model_name_hash\": \"e66844a04ae7a03bd9f228d9b778ec8429d361d0dca09b951b327ffad5beb07a\", \"namespace_hash\": \"ce3c9130af6416f40d71c5934f927acbd174f904a550fca2185aa3cd3528ca35\"}\n" }, { "path": "assets/models/fast_v2_1/config.min.json", "content": "{\"beg_size\": 512, \"mid_size\": 0, \"end_size\": 512, \"use_inputs_at_offsets\": false, \"medium_confidence_threshold\": 0.5, \"min_file_size_for_dl\": 8, \"padding_token\": 256, \"block_size\": 4096, \"target_labels_space\": [\"3gp\", \"ace\", \"ai\", \"aidl\", \"apk\", \"applebplist\", \"appleplist\", \"asm\", \"asp\", \"autohotkey\", \"autoit\", \"awk\", \"batch\", \"bazel\", \"bib\", \"bmp\", \"bzip\", \"c\", \"cab\", \"cat\", \"chm\", \"clojure\", \"cmake\", \"cobol\", \"coff\", \"coffeescript\", \"cpp\", \"crt\", \"crx\", \"cs\", \"csproj\", \"css\", \"csv\", \"dart\", \"deb\", \"dex\", \"dicom\", \"diff\", \"dm\", \"dmg\", \"doc\", \"dockerfile\", \"docx\", \"dsstore\", \"dwg\", \"dxf\", \"elf\", \"elixir\", \"emf\", \"eml\", \"epub\", \"erb\", \"erlang\", \"flac\", \"flv\", \"fortran\", \"gemfile\", \"gemspec\", \"gif\", \"gitattributes\", \"gitmodules\", \"go\", \"gradle\", \"groovy\", \"gzip\", \"h5\", \"handlebars\", \"haskell\", \"hcl\", \"hlp\", \"htaccess\", \"html\", \"icns\", \"ico\", \"ics\", \"ignorefile\", \"ini\", \"internetshortcut\", \"ipynb\", \"iso\", \"jar\", \"java\", \"javabytecode\", \"javascript\", \"jinja\", \"jp2\", \"jpeg\", \"json\", \"jsonl\", \"julia\", \"kotlin\", \"latex\", \"lha\", \"lisp\", \"lnk\", \"lua\", \"m3u\", \"m4\", \"macho\", \"makefile\", \"markdown\", \"matlab\", \"mht\", \"midi\", \"mkv\", \"mp3\", \"mp4\", \"mscompress\", \"msi\", \"mum\", \"npy\", \"npz\", \"nupkg\", \"objectivec\", \"ocaml\", \"odp\", \"ods\", \"odt\", \"ogg\", \"one\", \"onnx\", \"otf\", \"outlook\", \"parquet\", \"pascal\", \"pcap\", \"pdb\", \"pdf\", \"pebin\", \"pem\", \"perl\", \"php\", \"pickle\", \"png\", \"po\", \"postscript\", \"powershell\", \"ppt\", \"pptx\", \"prolog\", \"proteindb\", \"proto\", \"psd\", \"python\", \"pythonbytecode\", \"pytorch\", \"qt\", \"r\", \"rar\", \"rdf\", \"rpm\", \"rst\", \"rtf\", \"ruby\", \"rust\", \"scala\", \"scss\", \"sevenzip\", \"sgml\", \"shell\", \"smali\", \"snap\", \"solidity\", \"sql\", \"sqlite\", \"squashfs\", \"srt\", \"stlbinary\", \"stltext\", \"sum\", \"svg\", \"swf\", \"swift\", \"tar\", \"tcl\", \"textproto\", \"tga\", \"thumbsdb\", \"tiff\", \"toml\", \"torrent\", \"tsv\", \"ttf\", \"twig\", \"txt\", \"typescript\", \"unknown\", \"vba\", \"vcxproj\", \"verilog\", \"vhdl\", \"vtt\", \"vue\", \"wasm\", \"wav\", \"webm\", \"webp\", \"winregistry\", \"wmf\", \"woff\", \"woff2\", \"xar\", \"xls\", \"xlsb\", \"xlsx\", \"xml\", \"xpi\", \"xz\", \"yaml\", \"yara\", \"zig\", \"zip\", \"zlibstream\"], \"thresholds\": {\"latex\": 0.95, \"pascal\": 0.95}, \"overwrite_map\": {}}\n" }, { "path": "assets/models/fast_v2_1/metadata.json", "content": "{\"model_name_hash\": \"83b9d2bd0c450deffc70624554c99fa63e1830db852cdce860b7e215fa176f9f\", \"namespace_hash\": \"ce3c9130af6416f40d71c5934f927acbd174f904a550fca2185aa3cd3528ca35\"}\n" }, { "path": "assets/models/standard_v1/README.md", "content": "# Model documentation\n\nThe list of supported content types is indicated by the `target_labels_space` list in the model config, which you can find [here](https://github.com/google/magika/blob/7f947319f1ebe09626368a3f989a0863fcd7c52a/assets/models/standard_v1/model_config.json#L440-L554).\n\nNote: This model will be deprecated soon, in favor of more recent models (e.g., `standard_v3_x`)." }, { "path": "assets/models/standard_v1/content_types_config.json", "content": "{\n \"3gp\": {\n \"name\": \"3gp\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ace\": {\n \"name\": \"ace\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"aff\": {\n \"name\": \"aff\",\n \"extensions\": [\n \"aff\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ai\": {\n \"name\": \"ai\",\n \"extensions\": [\n \"ai\"\n ],\n \"mime_type\": \"application/pdf\",\n \"group\": \"document\",\n \"magic\": \"PDF document\",\n \"description\": \"Adobe Illustrator Artwork\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"ai\",\n \"target_label\": \"ai\",\n \"correct_labels\": [\n \"ai\",\n \"pdf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"algol68\": {\n \"name\": \"algol68\",\n \"extensions\": [\n \"a68\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"apk\": {\n \"name\": \"apk\",\n \"extensions\": [\n \"apk\"\n ],\n \"mime_type\": \"application/vnd.android.package-archive\",\n \"group\": \"executable\",\n \"magic\": \"Java archive data\",\n \"description\": \"Android package\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"apk\",\n \"correct_labels\": [\n \"apk\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"appleplist\": {\n \"name\": \"appleplist\",\n \"extensions\": [\n \"plist\"\n ],\n \"mime_type\": \"application/x-plist\",\n \"group\": \"application\",\n \"magic\": \"Apple binary property list\",\n \"description\": \"Android property list\",\n \"vt_type\": \"appleplist\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"text\"\n ],\n \"model_target_label\": \"appleplist\",\n \"target_label\": \"appleplist\",\n \"correct_labels\": [\n \"appleplist\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"arj\": {\n \"name\": \"arj\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"asm\": {\n \"name\": \"asm\",\n \"extensions\": [\n \"S\",\n \"asm\"\n ],\n \"mime_type\": \"text/x-asm\",\n \"group\": \"code\",\n \"magic\": \"assembler source\",\n \"description\": \"Assembly\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"asm\",\n \"target_label\": \"asm\",\n \"correct_labels\": [\n \"asm\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"asp\": {\n \"name\": \"asp\",\n \"extensions\": [\n \"aspx\",\n \"asp\"\n ],\n \"mime_type\": \"text/html\",\n \"group\": \"code\",\n \"magic\": \"HTML document\",\n \"description\": \"ASP source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"asp\",\n \"target_label\": \"asp\",\n \"correct_labels\": [\n \"asp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"avi\": {\n \"name\": \"avi\",\n \"extensions\": [\n \"avi\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ax\": {\n \"name\": \"ax\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"batch\": {\n \"name\": \"batch\",\n \"extensions\": [\n \"bat\"\n ],\n \"mime_type\": \"text/x-msdos-batch\",\n \"group\": \"code\",\n \"magic\": \"DOS batch file\",\n \"description\": \"DOS batch file\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"batch\",\n \"target_label\": \"batch\",\n \"correct_labels\": [\n \"batch\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"bcad\": {\n \"name\": \"bcad\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"bib\": {\n \"name\": \"bib\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"bmp\": {\n \"name\": \"bmp\",\n \"extensions\": [\n \"bmp\"\n ],\n \"mime_type\": \"image/bmp\",\n \"group\": \"image\",\n \"magic\": \"PC bitmap\",\n \"description\": \"BMP image data\",\n \"vt_type\": \"bmp\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"bmp\",\n \"target_label\": \"bmp\",\n \"correct_labels\": [\n \"bmp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"bpl\": {\n \"name\": \"bpl\",\n \"extensions\": [\n \"bpl\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"brainfuck\": {\n \"name\": \"brainfuck\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"bzip\": {\n \"name\": \"bzip\",\n \"extensions\": [\n \"bz2\",\n \"tbz2\",\n \"tar.bz2\"\n ],\n \"mime_type\": \"application/x-bzip2\",\n \"group\": \"archive\",\n \"magic\": \"bzip2 compressed data\",\n \"description\": \"bzip2 compressed data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"bzip\",\n \"target_label\": \"bzip\",\n \"correct_labels\": [\n \"bzip\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"c\": {\n \"name\": \"c\",\n \"extensions\": [\n \"c\",\n \"cpp\",\n \"h\",\n \"hpp\",\n \"cc\"\n ],\n \"mime_type\": \"text/x-c\",\n \"group\": \"code\",\n \"magic\": \"C source\",\n \"description\": \"C source\",\n \"vt_type\": \"c,cpp\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"c\",\n \"target_label\": \"c\",\n \"correct_labels\": [\n \"c\",\n \"cpp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"cab\": {\n \"name\": \"cab\",\n \"extensions\": [\n \"cab\"\n ],\n \"mime_type\": \"application/vnd.ms-cab-compressed\",\n \"group\": \"archive\",\n \"magic\": \"Microsoft Cabinet archive data\",\n \"description\": \"Microsoft Cabinet archive data\",\n \"vt_type\": \"cab\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"cab\",\n \"target_label\": \"cab\",\n \"correct_labels\": [\n \"cab\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"cad\": {\n \"name\": \"cad\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"cat\": {\n \"name\": \"cat\",\n \"extensions\": [\n \"cat\"\n ],\n \"mime_type\": \"application/octet-stream\",\n \"group\": \"application\",\n \"magic\": \"data\",\n \"description\": \"Windows Catalog file\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"cat\",\n \"target_label\": \"cat\",\n \"correct_labels\": [\n \"cat\",\n \"ctl\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"cdf\": {\n \"name\": \"cdf\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"chm\": {\n \"name\": \"chm\",\n \"extensions\": [\n \"chm\"\n ],\n \"mime_type\": \"application/chm\",\n \"group\": \"application\",\n \"magic\": \"MS Windows HtmlHelp Data\",\n \"description\": \"MS Windows HtmlHelp Data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"chm\",\n \"target_label\": \"chm\",\n \"correct_labels\": [\n \"chm\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"clojure\": {\n \"name\": \"clojure\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"cmake\": {\n \"name\": \"cmake\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"cobol\": {\n \"name\": \"cobol\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"coff\": {\n \"name\": \"coff\",\n \"extensions\": [],\n \"mime_type\": \"application/x-coff\",\n \"group\": \"executable\",\n \"magic\": \"Intel 80386 COFF\",\n \"description\": \"Intel 80386 COFF\",\n \"vt_type\": \"coff\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"coff\",\n \"target_label\": \"coff\",\n \"correct_labels\": [\n \"coff\",\n \"exp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"coffee\": {\n \"name\": \"coffee\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"com\": {\n \"name\": \"com\",\n \"extensions\": [],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": \"com\",\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"cpl\": {\n \"name\": \"cpl\",\n \"extensions\": [\n \"cpl\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"executable\",\n \"magic\": \"PE32 executable\",\n \"description\": \"PE Windows executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"cpl\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"cpp\": {\n \"name\": \"cpp\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"crx\": {\n \"name\": \"crx\",\n \"extensions\": [\n \"crx\"\n ],\n \"mime_type\": \"application/x-chrome-extension\",\n \"group\": \"executable\",\n \"magic\": \"Google Chrome extension\",\n \"description\": \"Google Chrome extension\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"crx\",\n \"target_label\": \"crx\",\n \"correct_labels\": [\n \"crx\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"cs\": {\n \"name\": \"cs\",\n \"extensions\": [\n \"cs\"\n ],\n \"mime_type\": \"text/plain\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"C# source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"cs\",\n \"target_label\": \"cs\",\n \"correct_labels\": [\n \"cs\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"css\": {\n \"name\": \"css\",\n \"extensions\": [\n \"css\"\n ],\n \"mime_type\": \"text/css\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"CSS source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"css\",\n \"target_label\": \"css\",\n \"correct_labels\": [\n \"css\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"csv\": {\n \"name\": \"csv\",\n \"extensions\": [\n \"csv\"\n ],\n \"mime_type\": \"text/csv\",\n \"group\": \"code\",\n \"magic\": \"CSV text\",\n \"description\": \"CSV document\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"csv\",\n \"target_label\": \"csv\",\n \"correct_labels\": [\n \"csv\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"ctl\": {\n \"name\": \"ctl\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"dart\": {\n \"name\": \"dart\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"deb\": {\n \"name\": \"deb\",\n \"extensions\": [\n \"deb\"\n ],\n \"mime_type\": \"application/vnd.debian.binary-package\",\n \"group\": \"archive\",\n \"magic\": \"Debian binary package\",\n \"description\": \"Debian binary package\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"deb\",\n \"target_label\": \"deb\",\n \"correct_labels\": [\n \"deb\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"dex\": {\n \"name\": \"dex\",\n \"extensions\": [\n \"dex\"\n ],\n \"mime_type\": \"application/x-android-dex\",\n \"group\": \"executable\",\n \"magic\": \"Dalvik dex file\",\n \"description\": \"Dalvik dex file\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"dex\",\n \"target_label\": \"dex\",\n \"correct_labels\": [\n \"dex\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"dey\": {\n \"name\": \"dey\",\n \"extensions\": [],\n \"mime_type\": \"application/x-android-dey\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"diff\": {\n \"name\": \"diff\",\n \"extensions\": [\n \"diff\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"directory\": {\n \"name\": \"directory\",\n \"extensions\": [],\n \"mime_type\": \"inode/directory\",\n \"group\": \"inode\",\n \"magic\": \"directory\",\n \"description\": \"A directory\",\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": \"directory\",\n \"correct_labels\": [\n \"directory\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": false\n },\n \"dll\": {\n \"name\": \"dll\",\n \"extensions\": [\n \"dll\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"executable\",\n \"magic\": \"PE Windows executable\",\n \"description\": \"PE Windows executable\",\n \"vt_type\": \"pedll\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"dll\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"dm\": {\n \"name\": \"dm\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"dmg\": {\n \"name\": \"dmg\",\n \"extensions\": [\n \"dmg\"\n ],\n \"mime_type\": \"application/x-apple-diskimage\",\n \"group\": \"archive\",\n \"magic\": \"Apple disk image\",\n \"description\": \"Apple disk image\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"dmg\",\n \"target_label\": \"dmg\",\n \"correct_labels\": [\n \"dmg\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"doc\": {\n \"name\": \"doc\",\n \"extensions\": [\n \"doc\"\n ],\n \"mime_type\": \"application/msword\",\n \"group\": \"document\",\n \"magic\": \"Composite Document File\",\n \"description\": \"Microsoft Word CDF document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"cdf\"\n ],\n \"model_target_label\": \"cdf\",\n \"target_label\": \"doc\",\n \"correct_labels\": [\n \"doc\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"dockerfile\": {\n \"name\": \"dockerfile\",\n \"extensions\": [\n \"=Dockerfile\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"docx\": {\n \"name\": \"docx\",\n \"extensions\": [\n \"docx\",\n \"docm\"\n ],\n \"mime_type\": \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\",\n \"group\": \"document\",\n \"magic\": \"Microsoft Word 2007+\",\n \"description\": \"Microsoft Word 2007+ document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"ooxml\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"docx\",\n \"correct_labels\": [\n \"docx\",\n \"tmdx\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"dosmbr\": {\n \"name\": \"dosmbr\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"dylib\": {\n \"name\": \"dylib\",\n \"extensions\": [\n \"dylib\"\n ],\n \"mime_type\": \"application/x-mach-o\",\n \"group\": \"executable\",\n \"magic\": \"Mach-O executable\",\n \"description\": \"Mach-O executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"macho\"\n ],\n \"model_target_label\": \"macho\",\n \"target_label\": \"macho\",\n \"correct_labels\": [\n \"macho\",\n \"dylib\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"elf\": {\n \"name\": \"elf\",\n \"extensions\": [\n \"elf\",\n \"so\"\n ],\n \"mime_type\": \"application/x-executable-elf\",\n \"group\": \"executable\",\n \"magic\": \"ELF executable\",\n \"description\": \"ELF executable\",\n \"vt_type\": \"elf\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"elf\"\n ],\n \"model_target_label\": \"elf\",\n \"target_label\": \"elf\",\n \"correct_labels\": [\n \"elf\",\n \"so\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"elixir\": {\n \"name\": \"elixir\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"emf\": {\n \"name\": \"emf\",\n \"extensions\": [\n \"emf\"\n ],\n \"mime_type\": \"application/octet-stream\",\n \"group\": \"application\",\n \"magic\": \"Windows Enhanced Metafile\",\n \"description\": \"Windows Enhanced Metafile image data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"emf\",\n \"target_label\": \"emf\",\n \"correct_labels\": [\n \"emf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"eml\": {\n \"name\": \"eml\",\n \"extensions\": [\n \"eml\"\n ],\n \"mime_type\": \"message/rfc822\",\n \"group\": \"text\",\n \"magic\": \"RFC 822 mail\",\n \"description\": \"RFC 822 mail\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"eml\",\n \"target_label\": \"eml\",\n \"correct_labels\": [\n \"eml\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"empty\": {\n \"name\": \"empty\",\n \"extensions\": [],\n \"mime_type\": \"inode/x-empty\",\n \"group\": \"inode\",\n \"magic\": \"empty\",\n \"description\": \"Empty file\",\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": \"empty\",\n \"correct_labels\": [\n \"empty\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": false\n },\n \"epub\": {\n \"name\": \"epub\",\n \"extensions\": [\n \"epub\"\n ],\n \"mime_type\": \"application/epub+zip\",\n \"group\": \"document\",\n \"magic\": \"EPUB document\",\n \"description\": \"EPUB document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"epub\",\n \"correct_labels\": [\n \"epub\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"erlang\": {\n \"name\": \"erlang\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ese\": {\n \"name\": \"ese\",\n \"extensions\": [],\n \"mime_type\": \"application/x-ms-ese\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"exe\": {\n \"name\": \"exe\",\n \"extensions\": [\n \"exe\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"executable\",\n \"magic\": \"ELF executable\",\n \"description\": \"ELF executable\",\n \"vt_type\": \"peexe\",\n \"datasets\": [\n \"vt-ext\",\n \"vt-ext-malicious\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"exe\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"exp\": {\n \"name\": \"exp\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"flac\": {\n \"name\": \"flac\",\n \"extensions\": [\n \"flac\"\n ],\n \"mime_type\": \"audio/flac\",\n \"group\": \"audio\",\n \"magic\": \"FLAC audio bitstream data\",\n \"description\": \"FLAC audio bitstream data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"flac\",\n \"target_label\": \"flac\",\n \"correct_labels\": [\n \"flac\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"fortran\": {\n \"name\": \"fortran\",\n \"extensions\": [\n \"f90\",\n \"f95\",\n \"f03\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"fpx\": {\n \"name\": \"fpx\",\n \"extensions\": [\n \"fpx\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": \"fpx\",\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"gif\": {\n \"name\": \"gif\",\n \"extensions\": [\n \"gif\"\n ],\n \"mime_type\": \"image/gif\",\n \"group\": \"image\",\n \"magic\": \"GIF image data\",\n \"description\": \"GIF image data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"gif\",\n \"target_label\": \"gif\",\n \"correct_labels\": [\n \"gif\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"go\": {\n \"name\": \"go\",\n \"extensions\": [\n \"go\"\n ],\n \"mime_type\": \"text/x-golang\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"Golang source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"go\",\n \"target_label\": \"go\",\n \"correct_labels\": [\n \"go\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"gpx\": {\n \"name\": \"gpx\",\n \"extensions\": [\n \"gpx\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"groovy\": {\n \"name\": \"groovy\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"gzip\": {\n \"name\": \"gzip\",\n \"extensions\": [\n \"gz\",\n \"gzip\",\n \"tgz\",\n \"tar.gz\"\n ],\n \"mime_type\": \"application/gzip\",\n \"group\": \"archive\",\n \"magic\": \"gzip compressed data\",\n \"description\": \"gzip compressed data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"gzip\",\n \"target_label\": \"gzip\",\n \"correct_labels\": [\n \"gzip\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"haskell\": {\n \"name\": \"haskell\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"hfs\": {\n \"name\": \"hfs\",\n \"extensions\": [\n \"hfs\"\n ],\n \"mime_type\": \"application/x-hfs\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"hlp\": {\n \"name\": \"hlp\",\n \"extensions\": [\n \"hlp\"\n ],\n \"mime_type\": \"application/winhlp\",\n \"group\": \"application\",\n \"magic\": \"MS Windows help\",\n \"description\": \"MS Windows help\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"hlp\",\n \"target_label\": \"hlp\",\n \"correct_labels\": [\n \"hlp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"hta\": {\n \"name\": \"hta\",\n \"extensions\": [\n \"hta\"\n ],\n \"mime_type\": \"application/hta\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"html\": {\n \"name\": \"html\",\n \"extensions\": [\n \"html\",\n \"htm\",\n \"xhtml\",\n \"xht\"\n ],\n \"mime_type\": \"text/html\",\n \"group\": \"code\",\n \"magic\": \"HTML document\",\n \"description\": \"HTML document\",\n \"vt_type\": \"html\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"html\",\n \"target_label\": \"html\",\n \"correct_labels\": [\n \"html\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"hve\": {\n \"name\": \"hve\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ico\": {\n \"name\": \"ico\",\n \"extensions\": [\n \"ico\"\n ],\n \"mime_type\": \"image/vnd.microsoft.icon\",\n \"group\": \"image\",\n \"magic\": \"MS Windows icon resource\",\n \"description\": \"MS Windows icon resource\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"ico\",\n \"target_label\": \"ico\",\n \"correct_labels\": [\n \"ico\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"img\": {\n \"name\": \"img\",\n \"extensions\": [\n \"img\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ini\": {\n \"name\": \"ini\",\n \"extensions\": [\n \"ini\"\n ],\n \"mime_type\": \"text/plain\",\n \"group\": \"text\",\n \"magic\": \"Generic INItialization configuration\",\n \"description\": \"INI configuration file\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"ini\",\n \"target_label\": \"ini\",\n \"correct_labels\": [\n \"ini\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"internetshortcut\": {\n \"name\": \"internetshortcut\",\n \"extensions\": [\n \"url\"\n ],\n \"mime_type\": \"application/x-mswinurl\",\n \"group\": \"application\",\n \"magic\": \"MS Windows 95 Internet shortcut\",\n \"description\": \"MS Windows Internet shortcut\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"internetshortcut\",\n \"target_label\": \"internetshortcut\",\n \"correct_labels\": [\n \"internetshortcut\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"iosapp\": {\n \"name\": \"iosapp\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"iso\": {\n \"name\": \"iso\",\n \"extensions\": [\n \"iso\"\n ],\n \"mime_type\": \"application/x-iso9660-image\",\n \"group\": \"archive\",\n \"magic\": \"ISO 9660 CD-ROM filesystem data\",\n \"description\": \"ISO 9660 CD-ROM filesystem data\",\n \"vt_type\": \"isoimage\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"iso\",\n \"target_label\": \"iso\",\n \"correct_labels\": [\n \"iso\",\n \"udf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"jar\": {\n \"name\": \"jar\",\n \"extensions\": [\n \"jar\"\n ],\n \"mime_type\": \"application/java-archive\",\n \"group\": \"archive\",\n \"magic\": \"Java archive data (JAR)\",\n \"description\": \"Java archive data (JAR)\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"jar\",\n \"target_label\": \"jar\",\n \"correct_labels\": [\n \"jar\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"java\": {\n \"name\": \"java\",\n \"extensions\": [\n \"java\"\n ],\n \"mime_type\": \"text/x-java\",\n \"group\": \"code\",\n \"magic\": \"Java source\",\n \"description\": \"Java source\",\n \"vt_type\": \"java\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"java\",\n \"target_label\": \"java\",\n \"correct_labels\": [\n \"java\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"javabytecode\": {\n \"name\": \"javabytecode\",\n \"extensions\": [\n \"class\"\n ],\n \"mime_type\": \"application/x-java-applet\",\n \"group\": \"executable\",\n \"magic\": \"compiled Java class data\",\n \"description\": \"Java compiled bytecode\",\n \"vt_type\": \"class\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"javabytecode\",\n \"target_label\": \"javabytecode\",\n \"correct_labels\": [\n \"javabytecode\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"javascript\": {\n \"name\": \"javascript\",\n \"extensions\": [\n \"js\"\n ],\n \"mime_type\": \"application/javascript\",\n \"group\": \"code\",\n \"magic\": \"JavaScript source\",\n \"description\": \"JavaScript source\",\n \"vt_type\": \"javascript\",\n \"datasets\": [\n \"github\",\n \"vt-ext\",\n \"vt-ext-malicious\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"javascript\",\n \"target_label\": \"javascript\",\n \"correct_labels\": [\n \"javascript\",\n \"typescript\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"jpeg\": {\n \"name\": \"jpeg\",\n \"extensions\": [\n \"jpg\",\n \"jpeg\"\n ],\n \"mime_type\": \"image/jpeg\",\n \"group\": \"image\",\n \"magic\": \"JPEG image data\",\n \"description\": \"JPEG image data\",\n \"vt_type\": \"jpeg\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"jpeg\",\n \"target_label\": \"jpeg\",\n \"correct_labels\": [\n \"jpeg\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"json\": {\n \"name\": \"json\",\n \"extensions\": [\n \"json\"\n ],\n \"mime_type\": \"application/json\",\n \"group\": \"code\",\n \"magic\": \"JSON data\",\n \"description\": \"JSON document\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"json\",\n \"target_label\": \"json\",\n \"correct_labels\": [\n \"json\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"julia\": {\n \"name\": \"julia\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ko\": {\n \"name\": \"ko\",\n \"extensions\": [\n \"ko\"\n ],\n \"mime_type\": \"application/x-executable-elf\",\n \"group\": \"executable\",\n \"magic\": \"ELF executable\",\n \"description\": \"ELF executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"elf\"\n ],\n \"model_target_label\": \"elf\",\n \"target_label\": \"elf\",\n \"correct_labels\": [\n \"elf\",\n \"ko\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"kotlin\": {\n \"name\": \"kotlin\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"latex\": {\n \"name\": \"latex\",\n \"extensions\": [\n \"tex\"\n ],\n \"mime_type\": \"text/x-tex\",\n \"group\": \"text\",\n \"magic\": \"LaTeX document\",\n \"description\": \"LaTeX document\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"latex\",\n \"target_label\": \"latex\",\n \"correct_labels\": [\n \"latex\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"lisp\": {\n \"name\": \"lisp\",\n \"extensions\": [\n \"lisp\"\n ],\n \"mime_type\": \"text/x-lisp\",\n \"group\": \"code\",\n \"magic\": \"Lisp/Scheme program\",\n \"description\": \"Lisp source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"lisp\",\n \"target_label\": \"lisp\",\n \"correct_labels\": [\n \"lisp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"lnk\": {\n \"name\": \"lnk\",\n \"extensions\": [\n \"lnk\"\n ],\n \"mime_type\": \"application/x-ms-shortcut\",\n \"group\": \"application\",\n \"magic\": \"MS Windows shortcut\",\n \"description\": \"MS Windows shortcut\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"lnk\",\n \"target_label\": \"lnk\",\n \"correct_labels\": [\n \"lnk\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"lua\": {\n \"name\": \"lua\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"m3u\": {\n \"name\": \"m3u\",\n \"extensions\": [\n \"m3u8\",\n \"m3u\"\n ],\n \"mime_type\": \"text/plain\",\n \"group\": \"application\",\n \"magic\": \"M3U playlist\",\n \"description\": \"M3U playlist\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"m3u\",\n \"target_label\": \"m3u\",\n \"correct_labels\": [\n \"m3u\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"macho\": {\n \"name\": \"macho\",\n \"extensions\": [],\n \"mime_type\": \"application/x-mach-o\",\n \"group\": \"executable\",\n \"magic\": \"Mach-O executable\",\n \"description\": \"Mach-O executable\",\n \"vt_type\": \"macho\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"macho\"\n ],\n \"model_target_label\": \"macho\",\n \"target_label\": \"macho\",\n \"correct_labels\": [\n \"macho\",\n \"dylib\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"maff\": {\n \"name\": \"maff\",\n \"extensions\": [\n \"maff\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"makefile\": {\n \"name\": \"makefile\",\n \"extensions\": [\n \"=Makefile\"\n ],\n \"mime_type\": \"text/x-makefile\",\n \"group\": \"code\",\n \"magic\": \"makefile script\",\n \"description\": \"Makefile source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"makefile\",\n \"target_label\": \"makefile\",\n \"correct_labels\": [\n \"makefile\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"markdown\": {\n \"name\": \"markdown\",\n \"extensions\": [\n \"md\"\n ],\n \"mime_type\": \"text/markdown\",\n \"group\": \"text\",\n \"magic\": \"ASCII text\",\n \"description\": \"Markdown document\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"markdown\",\n \"target_label\": \"markdown\",\n \"correct_labels\": [\n \"markdown\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"matlab\": {\n \"name\": \"matlab\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"mht\": {\n \"name\": \"mht\",\n \"extensions\": [\n \"mht\"\n ],\n \"mime_type\": \"application/x-mimearchive\",\n \"group\": \"code\",\n \"magic\": \"HTML document\",\n \"description\": \"MHTML document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"mht\",\n \"target_label\": \"mht\",\n \"correct_labels\": [\n \"mht\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"mkv\": {\n \"name\": \"mkv\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"mov\": {\n \"name\": \"mov\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"mp3\": {\n \"name\": \"mp3\",\n \"extensions\": [\n \"mp3\"\n ],\n \"mime_type\": \"audio/mpeg\",\n \"group\": \"audio\",\n \"magic\": \"Audio file with ID3\",\n \"description\": \"MP3 media file\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"mp3\",\n \"target_label\": \"mp3\",\n \"correct_labels\": [\n \"mp3\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"mp4\": {\n \"name\": \"mp4\",\n \"extensions\": [\n \"mp4\"\n ],\n \"mime_type\": \"video/mp4\",\n \"group\": \"video\",\n \"magic\": \"ISO Media\",\n \"description\": \"MP4 medial\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"mp4\",\n \"target_label\": \"mp4\",\n \"correct_labels\": [\n \"mp4\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"mscompress\": {\n \"name\": \"mscompress\",\n \"extensions\": [],\n \"mime_type\": \"application/x-ms-compress-szdd\",\n \"group\": \"archive\",\n \"magic\": \"MS Compress archive data\",\n \"description\": \"MS Compress archive data\",\n \"vt_type\": \"mscompress\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"mscompress\",\n \"target_label\": \"mscompress\",\n \"correct_labels\": [\n \"mscompress\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"msi\": {\n \"name\": \"msi\",\n \"extensions\": [\n \"msi\"\n ],\n \"mime_type\": \"application/x-msi\",\n \"group\": \"archive\",\n \"magic\": \"Composite Document File\",\n \"description\": \"Microsoft Installer file\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"cdf\"\n ],\n \"model_target_label\": \"msi\",\n \"target_label\": \"msi\",\n \"correct_labels\": [\n \"msi\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"mst\": {\n \"name\": \"mst\",\n \"extensions\": [\n \"mst\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"msvisio\": {\n \"name\": \"msvisio\",\n \"extensions\": [],\n \"mime_type\": \"application/vnd.ms-visio.drawing.main+xml\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"mui\": {\n \"name\": \"mui\",\n \"extensions\": [\n \"mui\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"application\",\n \"magic\": \"PE Windows executable\",\n \"description\": \"PE Windows executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"mui\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"mum\": {\n \"name\": \"mum\",\n \"extensions\": [\n \"mum\"\n ],\n \"mime_type\": \"text/xml\",\n \"group\": \"application\",\n \"magic\": \"XML document\",\n \"description\": \"Windows Update Package file\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"mum\",\n \"target_label\": \"mum\",\n \"correct_labels\": [\n \"mum\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"mun\": {\n \"name\": \"mun\",\n \"extensions\": [\n \"mun\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"nim\": {\n \"name\": \"nim\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"null\": {\n \"name\": \"null\",\n \"extensions\": [\n \"null\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"object\": {\n \"name\": \"object\",\n \"extensions\": [\n \"o\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"objectivec\": {\n \"name\": \"objectivec\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ocaml\": {\n \"name\": \"ocaml\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ocx\": {\n \"name\": \"ocx\",\n \"extensions\": [\n \"ocx\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"executable\",\n \"magic\": \"PE Windows executable\",\n \"description\": \"PE Windows executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"ax\",\n \"ocx\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"odex\": {\n \"name\": \"odex\",\n \"extensions\": [\n \"odex\"\n ],\n \"mime_type\": \"application/x-executable-elf\",\n \"group\": \"executable\",\n \"magic\": \"ELF executable\",\n \"description\": \"ODEX ELF executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"elf\"\n ],\n \"model_target_label\": \"odex\",\n \"target_label\": \"odex\",\n \"correct_labels\": [\n \"odex\",\n \"elf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"odp\": {\n \"name\": \"odp\",\n \"extensions\": [\n \"odp\"\n ],\n \"mime_type\": \"application/vnd.oasis.opendocument.presentation\",\n \"group\": \"document\",\n \"magic\": \"OpenDocument Presentation\",\n \"description\": \"OpenDocument Presentation\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\"\n ],\n \"model_target_label\": \"odp\",\n \"target_label\": \"odp\",\n \"correct_labels\": [\n \"odp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"ods\": {\n \"name\": \"ods\",\n \"extensions\": [\n \"ods\"\n ],\n \"mime_type\": \"application/vnd.oasis.opendocument.spreadsheet\",\n \"group\": \"document\",\n \"magic\": \"OpenDocument Spreadsheet\",\n \"description\": \"OpenDocument Spreadsheet\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\"\n ],\n \"model_target_label\": \"ods\",\n \"target_label\": \"ods\",\n \"correct_labels\": [\n \"ods\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"odt\": {\n \"name\": \"odt\",\n \"extensions\": [\n \"odt\"\n ],\n \"mime_type\": \"application/vnd.oasis.opendocument.text\",\n \"group\": \"document\",\n \"magic\": \"OpenDocument Text\",\n \"description\": \"OpenDocument Text\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\"\n ],\n \"model_target_label\": \"odt\",\n \"target_label\": \"odt\",\n \"correct_labels\": [\n \"odt\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"ogg\": {\n \"name\": \"ogg\",\n \"extensions\": [\n \"ogg\"\n ],\n \"mime_type\": \"audio/ogg\",\n \"group\": \"audio\",\n \"magic\": \"Ogg data\",\n \"description\": \"Ogg data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"ogg\",\n \"target_label\": \"ogg\",\n \"correct_labels\": [\n \"ogg\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"ole\": {\n \"name\": \"ole\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ooxml\": {\n \"name\": \"ooxml\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"outlook\": {\n \"name\": \"outlook\",\n \"extensions\": [],\n \"mime_type\": \"application/vnd.ms-outlook\",\n \"group\": \"application\",\n \"magic\": \"CDFV2 Microsoft Outlook Message\",\n \"description\": \"MS Outlook Message\",\n \"vt_type\": \"outlook\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"cdf\"\n ],\n \"model_target_label\": \"cdf\",\n \"target_label\": \"outlook\",\n \"correct_labels\": [\n \"outlook\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"palmos\": {\n \"name\": \"palmos\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": \"palmos\",\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"pascal\": {\n \"name\": \"pascal\",\n \"extensions\": [\n \"pascal\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"pbm\": {\n \"name\": \"pbm\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"pcap\": {\n \"name\": \"pcap\",\n \"extensions\": [\n \"pcap\",\n \"pcapng\"\n ],\n \"mime_type\": \"application/vnd.tcpdump.pcap\",\n \"group\": \"application\",\n \"magic\": \"pcap capture file\",\n \"description\": \"pcap capture file\",\n \"vt_type\": \"pcap\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"pcap\",\n \"target_label\": \"pcap\",\n \"correct_labels\": [\n \"pcap\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"pdf\": {\n \"name\": \"pdf\",\n \"extensions\": [\n \"pdf\"\n ],\n \"mime_type\": \"application/pdf\",\n \"group\": \"document\",\n \"magic\": \"PDF document\",\n \"description\": \"PDF document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"pdf\",\n \"target_label\": \"pdf\",\n \"correct_labels\": [\n \"pdf\",\n \"ai\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"pebin\": {\n \"name\": \"pebin\",\n \"extensions\": [\n \"exe\",\n \"dll\",\n \"sys\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"executable\",\n \"magic\": \"PE executable\",\n \"description\": \"PE executable\",\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": false\n },\n \"pem\": {\n \"name\": \"pem\",\n \"extensions\": [\n \"pem\",\n \"pub\"\n ],\n \"mime_type\": \"application/x-pem-file\",\n \"group\": \"application\",\n \"magic\": \"PEM certificate\",\n \"description\": \"PEM certificate\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"pem\",\n \"target_label\": \"pem\",\n \"correct_labels\": [\n \"pem\",\n \"pgpkey\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"perl\": {\n \"name\": \"perl\",\n \"extensions\": [\n \"pl\"\n ],\n \"mime_type\": \"text/x-perl\",\n \"group\": \"code\",\n \"magic\": \"Perl script text executable\",\n \"description\": \"Perl source\",\n \"vt_type\": \"perl\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"perl\",\n \"target_label\": \"perl\",\n \"correct_labels\": [\n \"perl\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"pgpkey\": {\n \"name\": \"pgpkey\",\n \"extensions\": [],\n \"mime_type\": \"application/pgp-keys\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"php\": {\n \"name\": \"php\",\n \"extensions\": [\n \"php\"\n ],\n \"mime_type\": \"text/x-php\",\n \"group\": \"code\",\n \"magic\": \"PHP script\",\n \"description\": \"PHP source\",\n \"vt_type\": \"php\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"php\",\n \"target_label\": \"php\",\n \"correct_labels\": [\n \"php\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"png\": {\n \"name\": \"png\",\n \"extensions\": [\n \"png\"\n ],\n \"mime_type\": \"image/png\",\n \"group\": \"image\",\n \"magic\": \"PNG image data\",\n \"description\": \"PNG image data\",\n \"vt_type\": \"png\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"png\",\n \"target_label\": \"png\",\n \"correct_labels\": [\n \"png\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"postscript\": {\n \"name\": \"postscript\",\n \"extensions\": [\n \"ps\"\n ],\n \"mime_type\": \"application/postscript\",\n \"group\": \"document\",\n \"magic\": \"PostScript document text\",\n \"description\": \"PostScript document\",\n \"vt_type\": \"postscript\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"postscript\",\n \"target_label\": \"postscript\",\n \"correct_labels\": [\n \"postscript\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"powershell\": {\n \"name\": \"powershell\",\n \"extensions\": [\n \"ps1\"\n ],\n \"mime_type\": \"application/x-powershell\",\n \"group\": \"code\",\n \"magic\": \"a powershell script\",\n \"description\": \"Powershell source\",\n \"vt_type\": \"ps\",\n \"datasets\": [\n \"github\",\n \"vt-ext\",\n \"vt-ext-malicious\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"powershell\",\n \"target_label\": \"powershell\",\n \"correct_labels\": [\n \"powershell\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"ppt\": {\n \"name\": \"ppt\",\n \"extensions\": [\n \"ppt\"\n ],\n \"mime_type\": \"application/vnd.ms-powerpoint\",\n \"group\": \"document\",\n \"magic\": \"Composite Document File\",\n \"description\": \"Microsoft PowerPoint CDF document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"cdf\"\n ],\n \"model_target_label\": \"cdf\",\n \"target_label\": \"ppt\",\n \"correct_labels\": [\n \"ppt\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"pptx\": {\n \"name\": \"pptx\",\n \"extensions\": [\n \"pptx\",\n \"pptm\"\n ],\n \"mime_type\": \"application/vnd.openxmlformats-officedocument.presentationml.presentation\",\n \"group\": \"document\",\n \"magic\": \"Microsoft PowerPoint 2007+\",\n \"description\": \"Microsoft PowerPoint 2007+ document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"ooxml\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"pptx\",\n \"correct_labels\": [\n \"pptx\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"printfox\": {\n \"name\": \"printfox\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"prolog\": {\n \"name\": \"prolog\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"pub\": {\n \"name\": \"pub\",\n \"extensions\": [\n \"pub\"\n ],\n \"mime_type\": \"application/x-mspublisher\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"cdf\"\n ],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"python\": {\n \"name\": \"python\",\n \"extensions\": [\n \"py\"\n ],\n \"mime_type\": \"text/x-python\",\n \"group\": \"code\",\n \"magic\": \"Python script\",\n \"description\": \"Python source\",\n \"vt_type\": \"python\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"python\",\n \"target_label\": \"python\",\n \"correct_labels\": [\n \"python\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"pythonbytecode\": {\n \"name\": \"pythonbytecode\",\n \"extensions\": [\n \"pyc\",\n \"pyo\"\n ],\n \"mime_type\": \"application/x-bytecode.python\",\n \"group\": \"executable\",\n \"magic\": \"python byte-compiled\",\n \"description\": \"Python compiled bytecode\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"pythonbytecode\",\n \"target_label\": \"pythonbytecode\",\n \"correct_labels\": [\n \"pythonbytecode\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"pythonpar\": {\n \"name\": \"pythonpar\",\n \"extensions\": [\n \"par\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"r\": {\n \"name\": \"r\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"randombytes\": {\n \"name\": \"randombytes\",\n \"extensions\": [],\n \"mime_type\": \"application/octet-stream\",\n \"group\": \"unknown\",\n \"magic\": \"data\",\n \"description\": \"Random bytes\",\n \"vt_type\": null,\n \"datasets\": [\n \"synthetic\"\n ],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": \"unknown\",\n \"target_label\": \"unknown\",\n \"correct_labels\": [\n \"unknown\"\n ],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": true\n },\n \"rar\": {\n \"name\": \"rar\",\n \"extensions\": [\n \"rar\"\n ],\n \"mime_type\": \"application/x-rar\",\n \"group\": \"archive\",\n \"magic\": \"RAR archive data\",\n \"description\": \"RAR archive data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"rar\",\n \"target_label\": \"rar\",\n \"correct_labels\": [\n \"rar\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"rdf\": {\n \"name\": \"rdf\",\n \"extensions\": [\n \"rdf\"\n ],\n \"mime_type\": \"application/rdf+xml\",\n \"group\": \"text\",\n \"magic\": \"XML document\",\n \"description\": \"Resource Description Framework document (RDF)\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"rdf\",\n \"target_label\": \"rdf\",\n \"correct_labels\": [\n \"rdf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"rll\": {\n \"name\": \"rll\",\n \"extensions\": [\n \"rll\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"rpm\": {\n \"name\": \"rpm\",\n \"extensions\": [\n \"rpm\"\n ],\n \"mime_type\": \"application/x-rpm\",\n \"group\": \"archive\",\n \"magic\": \"RPM\",\n \"description\": \"RedHat Package Manager archive (RPM)\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"rpm\",\n \"target_label\": \"rpm\",\n \"correct_labels\": [\n \"rpm\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"rst\": {\n \"name\": \"rst\",\n \"extensions\": [\n \"rst\"\n ],\n \"mime_type\": \"text/x-rst\",\n \"group\": \"text\",\n \"magic\": \"ReStructuredText file\",\n \"description\": \"ReStructuredText document\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"rst\",\n \"target_label\": \"rst\",\n \"correct_labels\": [\n \"rst\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"rtf\": {\n \"name\": \"rtf\",\n \"extensions\": [\n \"rtf\"\n ],\n \"mime_type\": \"text/rtf\",\n \"group\": \"text\",\n \"magic\": \"Rich Text Format data\",\n \"description\": \"Rich Text Format document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"rtf\",\n \"target_label\": \"rtf\",\n \"correct_labels\": [\n \"rtf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"ruby\": {\n \"name\": \"ruby\",\n \"extensions\": [\n \"rb\"\n ],\n \"mime_type\": \"application/x-ruby\",\n \"group\": \"code\",\n \"magic\": \"Ruby script\",\n \"description\": \"Ruby source\",\n \"vt_type\": \"ruby\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"ruby\",\n \"target_label\": \"ruby\",\n \"correct_labels\": [\n \"ruby\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"rust\": {\n \"name\": \"rust\",\n \"extensions\": [\n \"rs\"\n ],\n \"mime_type\": \"application/x-rust\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"Rust source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"rust\",\n \"target_label\": \"rust\",\n \"correct_labels\": [\n \"rust\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"s\": {\n \"name\": \"s\",\n \"extensions\": [\n \"s\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"scala\": {\n \"name\": \"scala\",\n \"extensions\": [\n \"scala\"\n ],\n \"mime_type\": \"application/x-scala\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"Scala source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"scala\",\n \"target_label\": \"scala\",\n \"correct_labels\": [\n \"scala\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"scr\": {\n \"name\": \"scr\",\n \"extensions\": [\n \"scr\"\n ],\n \"mime_type\": \"application/x-dosexec\",\n \"group\": \"executable\",\n \"magic\": \"PE Windows executable\",\n \"description\": \"PE Windows executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"scr\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"scriptwsf\": {\n \"name\": \"scriptwsf\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"sevenzip\": {\n \"name\": \"sevenzip\",\n \"extensions\": [\n \"7z\"\n ],\n \"mime_type\": \"application/x-7z-compressed\",\n \"group\": \"archive\",\n \"magic\": \"7-zip archive data\",\n \"description\": \"7-zip archive data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"sevenzip\",\n \"target_label\": \"sevenzip\",\n \"correct_labels\": [\n \"sevenzip\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"sgml\": {\n \"name\": \"sgml\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"sh3d\": {\n \"name\": \"sh3d\",\n \"extensions\": [\n \"sh3d\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"shell\": {\n \"name\": \"shell\",\n \"extensions\": [\n \"sh\"\n ],\n \"mime_type\": \"text/x-shellscript\",\n \"group\": \"code\",\n \"magic\": \"shell script\",\n \"description\": \"Shell script\",\n \"vt_type\": \"shell\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"shell\",\n \"target_label\": \"shell\",\n \"correct_labels\": [\n \"shell\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"smali\": {\n \"name\": \"smali\",\n \"extensions\": [\n \"smali\"\n ],\n \"mime_type\": \"application/x-smali\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"Smali source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"smali\",\n \"target_label\": \"smali\",\n \"correct_labels\": [\n \"smali\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"so\": {\n \"name\": \"so\",\n \"extensions\": [\n \"so\"\n ],\n \"mime_type\": \"application/x-executable-elf\",\n \"group\": \"executable\",\n \"magic\": \"ELF executable\",\n \"description\": \"ELF executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"elf\"\n ],\n \"model_target_label\": \"elf\",\n \"target_label\": \"elf\",\n \"correct_labels\": [\n \"elf\",\n \"so\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"sql\": {\n \"name\": \"sql\",\n \"extensions\": [\n \"sql\"\n ],\n \"mime_type\": \"application/x-sql\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"SQL source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"sql\",\n \"target_label\": \"sql\",\n \"correct_labels\": [\n \"sql\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"squashfs\": {\n \"name\": \"squashfs\",\n \"extensions\": [],\n \"mime_type\": \"application/octet-stream\",\n \"group\": \"archive\",\n \"magic\": \"Squashfs filesystem\",\n \"description\": \"Squash filesystem\",\n \"vt_type\": \"squashfs\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"squashfs\",\n \"target_label\": \"squashfs\",\n \"correct_labels\": [\n \"squashfs\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"svd\": {\n \"name\": \"svd\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"svg\": {\n \"name\": \"svg\",\n \"extensions\": [\n \"svg\"\n ],\n \"mime_type\": \"image/svg+xml\",\n \"group\": \"image\",\n \"magic\": \"SVG Scalable Vector Graphics image\",\n \"description\": \"SVG Scalable Vector Graphics image data\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"svg\",\n \"target_label\": \"svg\",\n \"correct_labels\": [\n \"svg\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"swf\": {\n \"name\": \"swf\",\n \"extensions\": [\n \"swf\"\n ],\n \"mime_type\": \"application/x-shockwave-flash\",\n \"group\": \"executable\",\n \"magic\": \"Macromedia Flash data\",\n \"description\": \"Macromedia Flash data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"swf\",\n \"target_label\": \"swf\",\n \"correct_labels\": [\n \"swf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"swift\": {\n \"name\": \"swift\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"symlinktext\": {\n \"name\": \"symlinktext\",\n \"extensions\": [],\n \"mime_type\": \"text/plain\",\n \"group\": \"application\",\n \"magic\": \"ASCII text\",\n \"description\": \"Symbolic link (textual representation)\",\n \"vt_type\": null,\n \"datasets\": [\n \"synthetic\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"symlinktext\",\n \"target_label\": \"symlinktext\",\n \"correct_labels\": [\n \"symlinktext\",\n \"txt\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"symlink\": {\n \"name\": \"symlink\",\n \"extensions\": [],\n \"mime_type\": \"inode/symlink\",\n \"group\": \"inode\",\n \"magic\": \"symbolic link to \",\n \"description\": \"Symbolic link to \",\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": \"symlink\",\n \"correct_labels\": [\n \"symlink\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": false\n },\n \"sys\": {\n \"name\": \"sys\",\n \"extensions\": [\n \"sys\"\n ],\n \"mime_type\": \"application/x-windows-driver\",\n \"group\": \"executable\",\n \"magic\": \"PE Windows executable\",\n \"description\": \"PE Windows executable\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"pebin\"\n ],\n \"model_target_label\": \"pebin\",\n \"target_label\": \"pebin\",\n \"correct_labels\": [\n \"pebin\",\n \"sys\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"tar\": {\n \"name\": \"tar\",\n \"extensions\": [\n \"tar\"\n ],\n \"mime_type\": \"application/x-tar\",\n \"group\": \"archive\",\n \"magic\": \"POSIX tar archive\",\n \"description\": \"POSIX tar archive\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"tar\",\n \"target_label\": \"tar\",\n \"correct_labels\": [\n \"tar\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"tga\": {\n \"name\": \"tga\",\n \"extensions\": [\n \"tga\"\n ],\n \"mime_type\": \"image/x-tga\",\n \"group\": \"image\",\n \"magic\": \"Targa image data\",\n \"description\": \"Targa image data\",\n \"vt_type\": \"targa\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"tga\",\n \"target_label\": \"tga\",\n \"correct_labels\": [\n \"tga\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"tiff\": {\n \"name\": \"tiff\",\n \"extensions\": [\n \"tiff\",\n \"tif\"\n ],\n \"mime_type\": \"image/tiff\",\n \"group\": \"image\",\n \"magic\": \"TIFF image data\",\n \"description\": \"TIFF image data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"tiff\",\n \"target_label\": \"tiff\",\n \"correct_labels\": [\n \"tiff\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"tmdx\": {\n \"name\": \"tmdx\",\n \"extensions\": [\n \"tmdx\",\n \"tmvx\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"toml\": {\n \"name\": \"toml\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"torrent\": {\n \"name\": \"torrent\",\n \"extensions\": [\n \"torrent\"\n ],\n \"mime_type\": \"application/x-bittorrent\",\n \"group\": \"application\",\n \"magic\": \"BitTorrent file\",\n \"description\": \"BitTorrent file\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"torrent\",\n \"target_label\": \"torrent\",\n \"correct_labels\": [\n \"torrent\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"troff\": {\n \"name\": \"troff\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"ttf\": {\n \"name\": \"ttf\",\n \"extensions\": [\n \"ttf\"\n ],\n \"mime_type\": \"font/sfnt\",\n \"group\": \"font\",\n \"magic\": \"TrueType Font data\",\n \"description\": \"TrueType Font data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"ttf\",\n \"target_label\": \"ttf\",\n \"correct_labels\": [\n \"ttf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"txt\": {\n \"name\": \"txt\",\n \"extensions\": [\n \"txt\"\n ],\n \"mime_type\": \"text/plain\",\n \"group\": \"text\",\n \"magic\": \"ASCII text\",\n \"description\": \"Generic text document\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\",\n \"synthetic\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"txt\",\n \"target_label\": \"txt\",\n \"correct_labels\": [\n \"txt\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"typescript\": {\n \"name\": \"typescript\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"udf\": {\n \"name\": \"udf\",\n \"extensions\": [],\n \"mime_type\": \"application/x-udf-image\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"unixcompress\": {\n \"name\": \"unixcompress\",\n \"extensions\": [\n \"z\"\n ],\n \"mime_type\": \"application/x-compress\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"unknown\": {\n \"name\": \"unknown\",\n \"extensions\": [],\n \"mime_type\": \"application/octet-stream\",\n \"group\": \"unknown\",\n \"magic\": \"data\",\n \"description\": \"Unknown binary data\",\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"unknown\",\n \"target_label\": \"unknown\",\n \"correct_labels\": [\n \"unknown\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": false\n },\n \"vba\": {\n \"name\": \"vba\",\n \"extensions\": [\n \"vbs\"\n ],\n \"mime_type\": \"text/vbscript\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"MS Visual Basic source (VBA)\",\n \"vt_type\": \"vba\",\n \"datasets\": [\n \"vt-ext\",\n \"vt-ext-malicious\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\"\n ],\n \"model_target_label\": \"vba\",\n \"target_label\": \"vba\",\n \"correct_labels\": [\n \"vba\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"verilog\": {\n \"name\": \"verilog\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"vhd\": {\n \"name\": \"vhd\",\n \"extensions\": [],\n \"mime_type\": \"application/x-vhd\",\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"wasm\": {\n \"name\": \"wasm\",\n \"extensions\": [\n \"wasm\"\n ],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"wav\": {\n \"name\": \"wav\",\n \"extensions\": [\n \"wav\"\n ],\n \"mime_type\": \"audio/x-wav\",\n \"group\": \"audio\",\n \"magic\": \"RIFF data\",\n \"description\": \"Waveform Audio file (WAV)\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"wav\",\n \"target_label\": \"wav\",\n \"correct_labels\": [\n \"wav\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"webm\": {\n \"name\": \"webm\",\n \"extensions\": [\n \"webm\"\n ],\n \"mime_type\": \"video/webm\",\n \"group\": \"video\",\n \"magic\": \"WebM\",\n \"description\": \"WebM data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"webm\",\n \"target_label\": \"webm\",\n \"correct_labels\": [\n \"webm\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"webp\": {\n \"name\": \"webp\",\n \"extensions\": [\n \"webp\"\n ],\n \"mime_type\": \"image/webp\",\n \"group\": \"image\",\n \"magic\": \"RIFF data\",\n \"description\": \"WebP data\",\n \"vt_type\": \"webp\",\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"media\"\n ],\n \"model_target_label\": \"webp\",\n \"target_label\": \"webp\",\n \"correct_labels\": [\n \"webp\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"winregistry\": {\n \"name\": \"winregistry\",\n \"extensions\": [\n \"reg\"\n ],\n \"mime_type\": \"text/x-ms-regedit\",\n \"group\": \"application\",\n \"magic\": \"Windows Registry text\",\n \"description\": \"Windows Registry text\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"winregistry\",\n \"target_label\": \"winregistry\",\n \"correct_labels\": [\n \"winregistry\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"wmf\": {\n \"name\": \"wmf\",\n \"extensions\": [\n \"wmf\"\n ],\n \"mime_type\": \"image/wmf\",\n \"group\": \"image\",\n \"magic\": \"Windows metafile\",\n \"description\": \"Windows metafile\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"wmf\",\n \"target_label\": \"wmf\",\n \"correct_labels\": [\n \"wmf\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"woff\": {\n \"name\": \"woff\",\n \"extensions\": [],\n \"mime_type\": null,\n \"group\": null,\n \"magic\": null,\n \"description\": null,\n \"vt_type\": null,\n \"datasets\": [],\n \"parent\": null,\n \"tags\": [],\n \"model_target_label\": null,\n \"target_label\": null,\n \"correct_labels\": [],\n \"in_scope_for_output_content_type\": false,\n \"in_scope_for_training\": false\n },\n \"xar\": {\n \"name\": \"xar\",\n \"extensions\": [\n \"pkg\"\n ],\n \"mime_type\": \"application/x-xar\",\n \"group\": \"archive\",\n \"magic\": \"xar archive compressed\",\n \"description\": \"XAR archive compressed data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"xar\",\n \"target_label\": \"xar\",\n \"correct_labels\": [\n \"xar\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"xls\": {\n \"name\": \"xls\",\n \"extensions\": [\n \"xls\"\n ],\n \"mime_type\": \"application/vnd.ms-excel\",\n \"group\": \"document\",\n \"magic\": \"Composite Document File\",\n \"description\": \"Microsoft Excel CDF document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"cdf\"\n ],\n \"model_target_label\": \"cdf\",\n \"target_label\": \"xls\",\n \"correct_labels\": [\n \"xls\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"xlsb\": {\n \"name\": \"xlsb\",\n \"extensions\": [\n \"xlsb\"\n ],\n \"mime_type\": \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\n \"group\": \"document\",\n \"magic\": \"Microsoft Excel 2007+\",\n \"description\": \"Microsoft Excel 2007+ document (binary format)\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"ooxml\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"xlsb\",\n \"correct_labels\": [\n \"xlsb\",\n \"xlsx\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"xlsx\": {\n \"name\": \"xlsx\",\n \"extensions\": [\n \"xlsx\",\n \"xlsm\"\n ],\n \"mime_type\": \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\n \"group\": \"document\",\n \"magic\": \"Microsoft Excel 2007+\",\n \"description\": \"Microsoft Excel 2007+ document\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"ooxml\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"xlsx\",\n \"correct_labels\": [\n \"xlsx\",\n \"xlsb\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"xml\": {\n \"name\": \"xml\",\n \"extensions\": [\n \"xml\"\n ],\n \"mime_type\": \"text/xml\",\n \"group\": \"code\",\n \"magic\": \"XML document\",\n \"description\": \"XML document\",\n \"vt_type\": \"xml\",\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"xml\",\n \"target_label\": \"xml\",\n \"correct_labels\": [\n \"xml\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"xpi\": {\n \"name\": \"xpi\",\n \"extensions\": [\n \"xpi\"\n ],\n \"mime_type\": \"application/zip\",\n \"group\": \"archive\",\n \"magic\": \"Zip archive data\",\n \"description\": \"Compressed installation archive (XPI)\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"xpi\",\n \"correct_labels\": [\n \"xpi\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"xz\": {\n \"name\": \"xz\",\n \"extensions\": [\n \"xz\"\n ],\n \"mime_type\": \"application/x-xz\",\n \"group\": \"archive\",\n \"magic\": \"XZ compressed data\",\n \"description\": \"XZ compressed data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"archive\"\n ],\n \"model_target_label\": \"xz\",\n \"target_label\": \"xz\",\n \"correct_labels\": [\n \"xz\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"yaml\": {\n \"name\": \"yaml\",\n \"extensions\": [\n \"yml\",\n \"yaml\"\n ],\n \"mime_type\": \"application/x-yaml\",\n \"group\": \"code\",\n \"magic\": \"ASCII text\",\n \"description\": \"YAML source\",\n \"vt_type\": null,\n \"datasets\": [\n \"github\"\n ],\n \"parent\": null,\n \"tags\": [\n \"text\",\n \"dl_target\"\n ],\n \"model_target_label\": \"yaml\",\n \"target_label\": \"yaml\",\n \"correct_labels\": [\n \"yaml\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"zip\": {\n \"name\": \"zip\",\n \"extensions\": [\n \"zip\"\n ],\n \"mime_type\": \"application/zip\",\n \"group\": \"archive\",\n \"magic\": \"Zip archive data\",\n \"description\": \"Zip archive data\",\n \"vt_type\": null,\n \"datasets\": [\n \"vt-ext\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\",\n \"zip_archive\",\n \"archive\"\n ],\n \"model_target_label\": \"zip\",\n \"target_label\": \"zip\",\n \"correct_labels\": [\n \"zip\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n },\n \"zlibstream\": {\n \"name\": \"zlibstream\",\n \"extensions\": [],\n \"mime_type\": \"application/zlib\",\n \"group\": \"application\",\n \"magic\": \"zlib compressed data\",\n \"description\": \"zlib compressed data\",\n \"vt_type\": \"zlib\",\n \"datasets\": [\n \"vt-type\"\n ],\n \"parent\": null,\n \"tags\": [\n \"binary\"\n ],\n \"model_target_label\": \"zlibstream\",\n \"target_label\": \"zlibstream\",\n \"correct_labels\": [\n \"zlibstream\"\n ],\n \"in_scope_for_output_content_type\": true,\n \"in_scope_for_training\": true\n }\n}" }, { "path": "assets/models/standard_v1/magika_config.json", "content": "{\n \"default_model_name\": \"standard_v1\",\n \"medium_confidence_threshold\": 0.5,\n \"min_file_size_for_dl\": 16,\n \"padding_token\": 256\n}" }, { "path": "assets/models/standard_v1/model_config.json", "content": "{\n \"cfg\": {\n \"model_class\": \"dense_v4\",\n \"shuffle_size\": 10000,\n \"input_sizes\": {\n \"beg\": 512,\n \"mid\": 512,\n \"end\": 512\n },\n \"batch_size\": 256,\n \"namespace\": \"2023-08-dense_v4\",\n \"max_train_samples_num_per_ds_per_ct\": 1000000,\n \"epochs\": 30,\n \"dense_v4.input_embedding_dim\": 128,\n \"dense_v4.step_size\": 4,\n \"dense_v4.use_first_layernorm\": true,\n \"dense_v4.premaxpool_dense_dim\": 256,\n \"dense_v4.premaxpool_dense2_dim\": 256,\n \"dense_v4.use_second_layernorm\": true,\n \"dense_v4.spatial_dropout_rate\": 0.1,\n \"dense_v4.dropout_rate\": 0.1,\n \"dataset_format\": \"int-concat/one-hot\",\n \"dense_v4.dense_activation\": \"gelu\",\n \"cutmix_t\": 0.05,\n \"dense_v4.augmentation_ratio\": 0.0,\n \"wandb_mode\": \"online\",\n \"max_test_samples_num_per_ds_per_ct\": 10000,\n \"target_labels_spec\": \"target-label\",\n \"ds_query\": \"all\",\n \"ct_query\": \"all\",\n \"tags\": [\n \"namespace:2023-08-dense_v4\"\n ],\n \"metrics\": [\n \"acc\"\n ],\n \"loss_function\": \"categorical_crossentropy\",\n \"optimizer\": \"adam\",\n \"augmentations_num\": 0,\n \"return_after_summary\": false,\n \"xgboost_model_dir\": null,\n \"dense_v4.use_meanpool\": false,\n \"dense_v4.use_file_size\": false,\n \"dense_v4.padding_byte\": 256\n },\n \"train_dataset_info\": {\n \"split_name\": \"train\",\n \"ds_query\": \"all\",\n \"ct_query\": \"all\",\n \"batch_size\": 256,\n \"shuffle\": 10000,\n \"block_length\": 1000,\n \"repeat_and_randomize\": true,\n \"include_sha256s\": false,\n \"input_sizes\": {\n \"beg\": 512,\n \"mid\": 512,\n \"end\": 512\n },\n \"max_samples_num_per_ds_per_ct\": 1000000,\n \"dataset_format\": \"int-concat/one-hot\",\n \"cutmix_t\": 0.05,\n \"target_labels_spec\": \"target-label\",\n \"total_samples_num_upper_bound\": 129000000,\n \"target_labels_info\": {\n \"target_labels_spec\": \"target-label\",\n \"content_types\": [\n \"ai\",\n \"apk\",\n \"appleplist\",\n \"asm\",\n \"asp\",\n \"batch\",\n \"bmp\",\n \"bzip\",\n \"c\",\n \"cab\",\n \"cat\",\n \"chm\",\n \"coff\",\n \"cpl\",\n \"crx\",\n \"cs\",\n \"css\",\n \"csv\",\n \"deb\",\n \"dex\",\n \"dll\",\n \"dmg\",\n \"doc\",\n \"docx\",\n \"dylib\",\n \"elf\",\n \"emf\",\n \"eml\",\n \"epub\",\n \"exe\",\n \"flac\",\n \"gif\",\n \"go\",\n \"gzip\",\n \"hlp\",\n \"html\",\n \"ico\",\n \"ini\",\n \"internetshortcut\",\n \"iso\",\n \"jar\",\n \"java\",\n \"javabytecode\",\n \"javascript\",\n \"jpeg\",\n \"json\",\n \"ko\",\n \"latex\",\n \"lisp\",\n \"lnk\",\n \"m3u\",\n \"macho\",\n \"makefile\",\n \"markdown\",\n \"mht\",\n \"mp3\",\n \"mp4\",\n \"mscompress\",\n \"msi\",\n \"mui\",\n \"mum\",\n \"ocx\",\n \"odex\",\n \"odp\",\n \"ods\",\n \"odt\",\n \"ogg\",\n \"outlook\",\n \"pcap\",\n \"pdf\",\n \"pem\",\n \"perl\",\n \"php\",\n \"png\",\n \"postscript\",\n \"powershell\",\n \"ppt\",\n \"pptx\",\n \"python\",\n \"pythonbytecode\",\n \"randombytes\",\n \"rar\",\n \"rdf\",\n \"rpm\",\n \"rst\",\n \"rtf\",\n \"ruby\",\n \"rust\",\n \"scala\",\n \"scr\",\n \"sevenzip\",\n \"shell\",\n \"smali\",\n \"so\",\n \"sql\",\n \"squashfs\",\n \"svg\",\n \"swf\",\n \"symlinktext\",\n \"sys\",\n \"tar\",\n \"tga\",\n \"tiff\",\n \"torrent\",\n \"ttf\",\n \"txt\",\n \"vba\",\n \"wav\",\n \"webm\",\n \"webp\",\n \"winregistry\",\n \"wmf\",\n \"xar\",\n \"xls\",\n \"xlsb\",\n \"xlsx\",\n \"xml\",\n \"xpi\",\n \"xz\",\n \"yaml\",\n \"zip\",\n \"zlibstream\"\n ],\n \"content_types_to_ids\": {\n \"ai\": 0,\n \"apk\": 1,\n \"appleplist\": 2,\n \"asm\": 3,\n \"asp\": 4,\n \"batch\": 5,\n \"bmp\": 6,\n \"bzip\": 7,\n \"c\": 8,\n \"cab\": 9,\n \"cat\": 10,\n \"chm\": 11,\n \"coff\": 12,\n \"cpl\": 13,\n \"crx\": 14,\n \"cs\": 15,\n \"css\": 16,\n \"csv\": 17,\n \"deb\": 18,\n \"dex\": 19,\n \"dll\": 20,\n \"dmg\": 21,\n \"doc\": 22,\n \"docx\": 23,\n \"dylib\": 24,\n \"elf\": 25,\n \"emf\": 26,\n \"eml\": 27,\n \"epub\": 28,\n \"exe\": 29,\n \"flac\": 30,\n \"gif\": 31,\n \"go\": 32,\n \"gzip\": 33,\n \"hlp\": 34,\n \"html\": 35,\n \"ico\": 36,\n \"ini\": 37,\n \"internetshortcut\": 38,\n \"iso\": 39,\n \"jar\": 40,\n \"java\": 41,\n \"javabytecode\": 42,\n \"javascript\": 43,\n \"jpeg\": 44,\n \"json\": 45,\n \"ko\": 46,\n \"latex\": 47,\n \"lisp\": 48,\n \"lnk\": 49,\n \"m3u\": 50,\n \"macho\": 51,\n \"makefile\": 52,\n \"markdown\": 53,\n \"mht\": 54,\n \"mp3\": 55,\n \"mp4\": 56,\n \"mscompress\": 57,\n \"msi\": 58,\n \"mui\": 59,\n \"mum\": 60,\n \"ocx\": 61,\n \"odex\": 62,\n \"odp\": 63,\n \"ods\": 64,\n \"odt\": 65,\n \"ogg\": 66,\n \"outlook\": 67,\n \"pcap\": 68,\n \"pdf\": 69,\n \"pem\": 70,\n \"perl\": 71,\n \"php\": 72,\n \"png\": 73,\n \"postscript\": 74,\n \"powershell\": 75,\n \"ppt\": 76,\n \"pptx\": 77,\n \"python\": 78,\n \"pythonbytecode\": 79,\n \"randombytes\": 80,\n \"rar\": 81,\n \"rdf\": 82,\n \"rpm\": 83,\n \"rst\": 84,\n \"rtf\": 85,\n \"ruby\": 86,\n \"rust\": 87,\n \"scala\": 88,\n \"scr\": 89,\n \"sevenzip\": 90,\n \"shell\": 91,\n \"smali\": 92,\n \"so\": 93,\n \"sql\": 94,\n \"squashfs\": 95,\n \"svg\": 96,\n \"swf\": 97,\n \"symlinktext\": 98,\n \"sys\": 99,\n \"tar\": 100,\n \"tga\": 101,\n \"tiff\": 102,\n \"torrent\": 103,\n \"ttf\": 104,\n \"txt\": 105,\n \"vba\": 106,\n \"wav\": 107,\n \"webm\": 108,\n \"webp\": 109,\n \"winregistry\": 110,\n \"wmf\": 111,\n \"xar\": 112,\n \"xls\": 113,\n \"xlsb\": 114,\n \"xlsx\": 115,\n \"xml\": 116,\n \"xpi\": 117,\n \"xz\": 118,\n \"yaml\": 119,\n \"zip\": 120,\n \"zlibstream\": 121\n },\n \"content_types_num\": 122,\n \"target_labels\": [\n \"ai\",\n \"apk\",\n \"appleplist\",\n \"asm\",\n \"asp\",\n \"batch\",\n \"bmp\",\n \"bzip\",\n \"c\",\n \"cab\",\n \"cat\",\n \"chm\",\n \"coff\",\n \"pebin\",\n \"crx\",\n \"cs\",\n \"css\",\n \"csv\",\n \"deb\",\n \"dex\",\n \"pebin\",\n \"dmg\",\n \"doc\",\n \"docx\",\n \"macho\",\n \"elf\",\n \"emf\",\n \"eml\",\n \"epub\",\n \"pebin\",\n \"flac\",\n \"gif\",\n \"go\",\n \"gzip\",\n \"hlp\",\n \"html\",\n \"ico\",\n \"ini\",\n \"internetshortcut\",\n \"iso\",\n \"jar\",\n \"java\",\n \"javabytecode\",\n \"javascript\",\n \"jpeg\",\n \"json\",\n \"elf\",\n \"latex\",\n \"lisp\",\n \"lnk\",\n \"m3u\",\n \"macho\",\n \"makefile\",\n \"markdown\",\n \"mht\",\n \"mp3\",\n \"mp4\",\n \"mscompress\",\n \"msi\",\n \"pebin\",\n \"mum\",\n \"pebin\",\n \"odex\",\n \"odp\",\n \"ods\",\n \"odt\",\n \"ogg\",\n \"outlook\",\n \"pcap\",\n \"pdf\",\n \"pem\",\n \"perl\",\n \"php\",\n \"png\",\n \"postscript\",\n \"powershell\",\n \"ppt\",\n \"pptx\",\n \"python\",\n \"pythonbytecode\",\n \"unknown\",\n \"rar\",\n \"rdf\",\n \"rpm\",\n \"rst\",\n \"rtf\",\n \"ruby\",\n \"rust\",\n \"scala\",\n \"pebin\",\n \"sevenzip\",\n \"shell\",\n \"smali\",\n \"elf\",\n \"sql\",\n \"squashfs\",\n \"svg\",\n \"swf\",\n \"symlinktext\",\n \"pebin\",\n \"tar\",\n \"tga\",\n \"tiff\",\n \"torrent\",\n \"ttf\",\n \"txt\",\n \"vba\",\n \"wav\",\n \"webm\",\n \"webp\",\n \"winregistry\",\n \"wmf\",\n \"xar\",\n \"xls\",\n \"xlsb\",\n \"xlsx\",\n \"xml\",\n \"xpi\",\n \"xz\",\n \"yaml\",\n \"zip\",\n \"zlibstream\"\n ],\n \"target_labels_space\": [\n \"ai\",\n \"apk\",\n \"appleplist\",\n \"asm\",\n \"asp\",\n \"batch\",\n \"bmp\",\n \"bzip\",\n \"c\",\n \"cab\",\n \"cat\",\n \"chm\",\n \"coff\",\n \"crx\",\n \"cs\",\n \"css\",\n \"csv\",\n \"deb\",\n \"dex\",\n \"dmg\",\n \"doc\",\n \"docx\",\n \"elf\",\n \"emf\",\n \"eml\",\n \"epub\",\n \"flac\",\n \"gif\",\n \"go\",\n \"gzip\",\n \"hlp\",\n \"html\",\n \"ico\",\n \"ini\",\n \"internetshortcut\",\n \"iso\",\n \"jar\",\n \"java\",\n \"javabytecode\",\n \"javascript\",\n \"jpeg\",\n \"json\",\n \"latex\",\n \"lisp\",\n \"lnk\",\n \"m3u\",\n \"macho\",\n \"makefile\",\n \"markdown\",\n \"mht\",\n \"mp3\",\n \"mp4\",\n \"mscompress\",\n \"msi\",\n \"mum\",\n \"odex\",\n \"odp\",\n \"ods\",\n \"odt\",\n \"ogg\",\n \"outlook\",\n \"pcap\",\n \"pdf\",\n \"pebin\",\n \"pem\",\n \"perl\",\n \"php\",\n \"png\",\n \"postscript\",\n \"powershell\",\n \"ppt\",\n \"pptx\",\n \"python\",\n \"pythonbytecode\",\n \"rar\",\n \"rdf\",\n \"rpm\",\n \"rst\",\n \"rtf\",\n \"ruby\",\n \"rust\",\n \"scala\",\n \"sevenzip\",\n \"shell\",\n \"smali\",\n \"sql\",\n \"squashfs\",\n \"svg\",\n \"swf\",\n \"symlinktext\",\n \"tar\",\n \"tga\",\n \"tiff\",\n \"torrent\",\n \"ttf\",\n \"txt\",\n \"unknown\",\n \"vba\",\n \"wav\",\n \"webm\",\n \"webp\",\n \"winregistry\",\n \"wmf\",\n \"xar\",\n \"xls\",\n \"xlsb\",\n \"xlsx\",\n \"xml\",\n \"xpi\",\n \"xz\",\n \"yaml\",\n \"zip\",\n \"zlibstream\"\n ],\n \"target_labels_to_ids\": {\n \"ai\": 0,\n \"apk\": 1,\n \"appleplist\": 2,\n \"asm\": 3,\n \"asp\": 4,\n \"batch\": 5,\n \"bmp\": 6,\n \"bzip\": 7,\n \"c\": 8,\n \"cab\": 9,\n \"cat\": 10,\n \"chm\": 11,\n \"coff\": 12,\n \"crx\": 13,\n \"cs\": 14,\n \"css\": 15,\n \"csv\": 16,\n \"deb\": 17,\n \"dex\": 18,\n \"dmg\": 19,\n \"doc\": 20,\n \"docx\": 21,\n \"elf\": 22,\n \"emf\": 23,\n \"eml\": 24,\n \"epub\": 25,\n \"flac\": 26,\n \"gif\": 27,\n \"go\": 28,\n \"gzip\": 29,\n \"hlp\": 30,\n \"html\": 31,\n \"ico\": 32,\n \"ini\": 33,\n \"internetshortcut\": 34,\n \"iso\": 35,\n \"jar\": 36,\n \"java\": 37,\n \"javabytecode\": 38,\n \"javascript\": 39,\n \"jpeg\": 40,\n \"json\": 41,\n \"latex\": 42,\n \"lisp\": 43,\n \"lnk\": 44,\n \"m3u\": 45,\n \"macho\": 46,\n \"makefile\": 47,\n \"markdown\": 48,\n \"mht\": 49,\n \"mp3\": 50,\n \"mp4\": 51,\n \"mscompress\": 52,\n \"msi\": 53,\n \"mum\": 54,\n \"odex\": 55,\n \"odp\": 56,\n \"ods\": 57,\n \"odt\": 58,\n \"ogg\": 59,\n \"outlook\": 60,\n \"pcap\": 61,\n \"pdf\": 62,\n \"pebin\": 63,\n \"pem\": 64,\n \"perl\": 65,\n \"php\": 66,\n \"png\": 67,\n \"postscript\": 68,\n \"powershell\": 69,\n \"ppt\": 70,\n \"pptx\": 71,\n \"python\": 72,\n \"pythonbytecode\": 73,\n \"rar\": 74,\n \"rdf\": 75,\n \"rpm\": 76,\n \"rst\": 77,\n \"rtf\": 78,\n \"ruby\": 79,\n \"rust\": 80,\n \"scala\": 81,\n \"sevenzip\": 82,\n \"shell\": 83,\n \"smali\": 84,\n \"sql\": 85,\n \"squashfs\": 86,\n \"svg\": 87,\n \"swf\": 88,\n \"symlinktext\": 89,\n \"tar\": 90,\n \"tga\": 91,\n \"tiff\": 92,\n \"torrent\": 93,\n \"ttf\": 94,\n \"txt\": 95,\n \"unknown\": 96,\n \"vba\": 97,\n \"wav\": 98,\n \"webm\": 99,\n \"webp\": 100,\n \"winregistry\": 101,\n \"wmf\": 102,\n \"xar\": 103,\n \"xls\": 104,\n \"xlsb\": 105,\n \"xlsx\": 106,\n \"xml\": 107,\n \"xpi\": 108,\n \"xz\": 109,\n \"yaml\": 110,\n \"zip\": 111,\n \"zlibstream\": 112\n },\n \"target_labels_num\": 113,\n \"content_types_to_target_labels_ids\": {\n \"ai\": 0,\n \"apk\": 1,\n \"appleplist\": 2,\n \"asm\": 3,\n \"asp\": 4,\n \"batch\": 5,\n \"bmp\": 6,\n \"bzip\": 7,\n \"c\": 8,\n \"cab\": 9,\n \"cat\": 10,\n \"chm\": 11,\n \"coff\": 12,\n \"cpl\": 63,\n \"crx\": 13,\n \"cs\": 14,\n \"css\": 15,\n \"csv\": 16,\n \"deb\": 17,\n \"dex\": 18,\n \"dll\": 63,\n \"dmg\": 19,\n \"doc\": 20,\n \"docx\": 21,\n \"dylib\": 46,\n \"elf\": 22,\n \"emf\": 23,\n \"eml\": 24,\n \"epub\": 25,\n \"exe\": 63,\n \"flac\": 26,\n \"gif\": 27,\n \"go\": 28,\n \"gzip\": 29,\n \"hlp\": 30,\n \"html\": 31,\n \"ico\": 32,\n \"ini\": 33,\n \"internetshortcut\": 34,\n \"iso\": 35,\n \"jar\": 36,\n \"java\": 37,\n \"javabytecode\": 38,\n \"javascript\": 39,\n \"jpeg\": 40,\n \"json\": 41,\n \"ko\": 22,\n \"latex\": 42,\n \"lisp\": 43,\n \"lnk\": 44,\n \"m3u\": 45,\n \"macho\": 46,\n \"makefile\": 47,\n \"markdown\": 48,\n \"mht\": 49,\n \"mp3\": 50,\n \"mp4\": 51,\n \"mscompress\": 52,\n \"msi\": 53,\n \"mui\": 63,\n \"mum\": 54,\n \"ocx\": 63,\n \"odex\": 55,\n \"odp\": 56,\n \"ods\": 57,\n \"odt\": 58,\n \"ogg\": 59,\n \"outlook\": 60,\n \"pcap\": 61,\n \"pdf\": 62,\n \"pem\": 64,\n \"perl\": 65,\n \"php\": 66,\n \"png\": 67,\n \"postscript\": 68,\n \"powershell\": 69,\n \"ppt\": 70,\n \"pptx\": 71,\n \"python\": 72,\n \"pythonbytecode\": 73,\n \"randombytes\": 96,\n \"rar\": 74,\n \"rdf\": 75,\n \"rpm\": 76,\n \"rst\": 77,\n \"rtf\": 78,\n \"ruby\": 79,\n \"rust\": 80,\n \"scala\": 81,\n \"scr\": 63,\n \"sevenzip\": 82,\n \"shell\": 83,\n \"smali\": 84,\n \"so\": 22,\n \"sql\": 85,\n \"squashfs\": 86,\n \"svg\": 87,\n \"swf\": 88,\n \"symlinktext\": 89,\n \"sys\": 63,\n \"tar\": 90,\n \"tga\": 91,\n \"tiff\": 92,\n \"torrent\": 93,\n \"ttf\": 94,\n \"txt\": 95,\n \"vba\": 97,\n \"wav\": 98,\n \"webm\": 99,\n \"webp\": 100,\n \"winregistry\": 101,\n \"wmf\": 102,\n \"xar\": 103,\n \"xls\": 104,\n \"xlsb\": 105,\n \"xlsx\": 106,\n \"xml\": 107,\n \"xpi\": 108,\n \"xz\": 109,\n \"yaml\": 110,\n \"zip\": 111,\n \"zlibstream\": 112\n }\n }\n },\n \"test_dataset_info\": {\n \"split_name\": \"test\",\n \"ds_query\": \"all\",\n \"ct_query\": \"all\",\n \"batch_size\": 256,\n \"shuffle\": 0,\n \"block_length\": 1000,\n \"repeat_and_randomize\": false,\n \"include_sha256s\": false,\n \"input_sizes\": {\n \"beg\": 512,\n \"mid\": 512,\n \"end\": 512\n },\n \"max_samples_num_per_ds_per_ct\": 10000,\n \"dataset_format\": \"int-concat/one-hot\",\n \"cutmix_t\": null,\n \"target_labels_spec\": \"target-label\",\n \"total_samples_num_upper_bound\": 1290000,\n \"target_labels_info\": {\n \"target_labels_spec\": \"target-label\",\n \"content_types\": [\n \"ai\",\n \"apk\",\n \"appleplist\",\n \"asm\",\n \"asp\",\n \"batch\",\n \"bmp\",\n \"bzip\",\n \"c\",\n \"cab\",\n \"cat\",\n \"chm\",\n \"coff\",\n \"cpl\",\n \"crx\",\n \"cs\",\n \"css\",\n \"csv\",\n \"deb\",\n \"dex\",\n \"dll\",\n \"dmg\",\n \"doc\",\n \"docx\",\n \"dylib\",\n \"elf\",\n \"emf\",\n \"eml\",\n \"epub\",\n \"exe\",\n \"flac\",\n \"gif\",\n \"go\",\n \"gzip\",\n \"hlp\",\n \"html\",\n \"ico\",\n \"ini\",\n \"internetshortcut\",\n \"iso\",\n \"jar\",\n \"java\",\n \"javabytecode\",\n \"javascript\",\n \"jpeg\",\n \"json\",\n \"ko\",\n \"latex\",\n \"lisp\",\n \"lnk\",\n \"m3u\",\n \"macho\",\n \"makefile\",\n \"markdown\",\n \"mht\",\n \"mp3\",\n \"mp4\",\n \"mscompress\",\n \"msi\",\n \"mui\",\n \"mum\",\n \"ocx\",\n \"odex\",\n \"odp\",\n \"ods\",\n \"odt\",\n \"ogg\",\n \"outlook\",\n \"pcap\",\n \"pdf\",\n \"pem\",\n \"perl\",\n \"php\",\n \"png\",\n \"postscript\",\n \"powershell\",\n \"ppt\",\n \"pptx\",\n \"python\",\n \"pythonbytecode\",\n \"randombytes\",\n \"rar\",\n \"rdf\",\n \"rpm\",\n \"rst\",\n \"rtf\",\n \"ruby\",\n \"rust\",\n \"scala\",\n \"scr\",\n \"sevenzip\",\n \"shell\",\n \"smali\",\n \"so\",\n \"sql\",\n \"squashfs\",\n \"svg\",\n \"swf\",\n \"symlinktext\",\n \"sys\",\n \"tar\",\n \"tga\",\n \"tiff\",\n \"torrent\",\n \"ttf\",\n \"txt\",\n \"vba\",\n \"wav\",\n \"webm\",\n \"webp\",\n \"winregistry\",\n \"wmf\",\n \"xar\",\n \"xls\",\n \"xlsb\",\n \"xlsx\",\n \"xml\",\n \"xpi\",\n \"xz\",\n \"yaml\",\n \"zip\",\n \"zlibstream\"\n ],\n \"content_types_to_ids\": {\n \"ai\": 0,\n \"apk\": 1,\n \"appleplist\": 2,\n \"asm\": 3,\n \"asp\": 4,\n \"batch\": 5,\n \"bmp\": 6,\n \"bzip\": 7,\n \"c\": 8,\n \"cab\": 9,\n \"cat\": 10,\n \"chm\": 11,\n \"coff\": 12,\n \"cpl\": 13,\n \"crx\": 14,\n \"cs\": 15,\n \"css\": 16,\n \"csv\": 17,\n \"deb\": 18,\n \"dex\": 19,\n \"dll\": 20,\n \"dmg\": 21,\n \"doc\": 22,\n \"docx\": 23,\n \"dylib\": 24,\n \"elf\": 25,\n \"emf\": 26,\n \"eml\": 27,\n \"epub\": 28,\n \"exe\": 29,\n \"flac\": 30,\n \"gif\": 31,\n \"go\": 32,\n \"gzip\": 33,\n \"hlp\": 34,\n \"html\": 35,\n \"ico\": 36,\n \"ini\": 37,\n \"internetshortcut\": 38,\n \"iso\": 39,\n \"jar\": 40,\n \"java\": 41,\n \"javabytecode\": 42,\n \"javascript\": 43,\n \"jpeg\": 44,\n \"json\": 45,\n \"ko\": 46,\n \"latex\": 47,\n \"lisp\": 48,\n \"lnk\": 49,\n \"m3u\": 50,\n \"macho\": 51,\n \"makefile\": 52,\n \"markdown\": 53,\n \"mht\": 54,\n \"mp3\": 55,\n \"mp4\": 56,\n \"mscompress\": 57,\n \"msi\": 58,\n \"mui\": 59,\n \"mum\": 60,\n \"ocx\": 61,\n \"odex\": 62,\n \"odp\": 63,\n \"ods\": 64,\n \"odt\": 65,\n \"ogg\": 66,\n \"outlook\": 67,\n \"pcap\": 68,\n \"pdf\": 69,\n \"pem\": 70,\n \"perl\": 71,\n \"php\": 72,\n \"png\": 73,\n \"postscript\": 74,\n \"powershell\": 75,\n \"ppt\": 76,\n \"pptx\": 77,\n \"python\": 78,\n \"pythonbytecode\": 79,\n \"randombytes\": 80,\n \"rar\": 81,\n \"rdf\": 82,\n \"rpm\": 83,\n \"rst\": 84,\n \"rtf\": 85,\n \"ruby\": 86,\n \"rust\": 87,\n \"scala\": 88,\n \"scr\": 89,\n \"sevenzip\": 90,\n \"shell\": 91,\n \"smali\": 92,\n \"so\": 93,\n \"sql\": 94,\n \"squashfs\": 95,\n \"svg\": 96,\n \"swf\": 97,\n \"symlinktext\": 98,\n \"sys\": 99,\n \"tar\": 100,\n \"tga\": 101,\n \"tiff\": 102,\n \"torrent\": 103,\n \"ttf\": 104,\n \"txt\": 105,\n \"vba\": 106,\n \"wav\": 107,\n \"webm\": 108,\n \"webp\": 109,\n \"winregistry\": 110,\n \"wmf\": 111,\n \"xar\": 112,\n \"xls\": 113,\n \"xlsb\": 114,\n \"xlsx\": 115,\n \"xml\": 116,\n \"xpi\": 117,\n \"xz\": 118,\n \"yaml\": 119,\n \"zip\": 120,\n \"zlibstream\": 121\n },\n \"content_types_num\": 122,\n \"target_labels\": [\n \"ai\",\n \"apk\",\n \"appleplist\",\n \"asm\",\n \"asp\",\n \"batch\",\n \"bmp\",\n \"bzip\",\n \"c\",\n \"cab\",\n \"cat\",\n \"chm\",\n \"coff\",\n \"pebin\",\n \"crx\",\n \"cs\",\n \"css\",\n \"csv\",\n \"deb\",\n \"dex\",\n \"pebin\",\n \"dmg\",\n \"doc\",\n \"docx\",\n \"macho\",\n \"elf\",\n \"emf\",\n \"eml\",\n \"epub\",\n \"pebin\",\n \"flac\",\n \"gif\",\n \"go\",\n \"gzip\",\n \"hlp\",\n \"html\",\n \"ico\",\n \"ini\",\n \"internetshortcut\",\n \"iso\",\n \"jar\",\n \"java\",\n \"javabytecode\",\n \"javascript\",\n \"jpeg\",\n \"json\",\n \"elf\",\n \"latex\",\n \"lisp\",\n \"lnk\",\n \"m3u\",\n \"macho\",\n \"makefile\",\n \"markdown\",\n \"mht\",\n \"mp3\",\n \"mp4\",\n \"mscompress\",\n \"msi\",\n \"pebin\",\n \"mum\",\n \"pebin\",\n \"odex\",\n \"odp\",\n \"ods\",\n \"odt\",\n \"ogg\",\n \"outlook\",\n \"pcap\",\n \"pdf\",\n \"pem\",\n \"perl\",\n \"php\",\n \"png\",\n \"postscript\",\n \"powershell\",\n \"ppt\",\n \"pptx\",\n \"python\",\n \"pythonbytecode\",\n \"unknown\",\n \"rar\",\n \"rdf\",\n \"rpm\",\n \"rst\",\n \"rtf\",\n \"ruby\",\n \"rust\",\n \"scala\",\n \"pebin\",\n \"sevenzip\",\n \"shell\",\n \"smali\",\n \"elf\",\n \"sql\",\n \"squashfs\",\n \"svg\",\n \"swf\",\n \"symlinktext\",\n \"pebin\",\n \"tar\",\n \"tga\",\n \"tiff\",\n \"torrent\",\n \"ttf\",\n \"txt\",\n \"vba\",\n \"wav\",\n \"webm\",\n \"webp\",\n \"winregistry\",\n \"wmf\",\n \"xar\",\n \"xls\",\n \"xlsb\",\n \"xlsx\",\n \"xml\",\n \"xpi\",\n \"xz\",\n \"yaml\",\n \"zip\",\n \"zlibstream\"\n ],\n \"target_labels_space\": [\n \"ai\",\n \"apk\",\n \"appleplist\",\n \"asm\",\n \"asp\",\n \"batch\",\n \"bmp\",\n \"bzip\",\n \"c\",\n \"cab\",\n \"cat\",\n \"chm\",\n \"coff\",\n \"crx\",\n \"cs\",\n \"css\",\n \"csv\",\n \"deb\",\n \"dex\",\n \"dmg\",\n \"doc\",\n \"docx\",\n \"elf\",\n \"emf\",\n \"eml\",\n \"epub\",\n \"flac\",\n \"gif\",\n \"go\",\n \"gzip\",\n \"hlp\",\n \"html\",\n \"ico\",\n \"ini\",\n \"internetshortcut\",\n \"iso\",\n \"jar\",\n \"java\",\n \"javabytecode\",\n \"javascript\",\n \"jpeg\",\n \"json\",\n \"latex\",\n \"lisp\",\n \"lnk\",\n \"m3u\",\n \"macho\",\n \"makefile\",\n \"markdown\",\n \"mht\",\n \"mp3\",\n \"mp4\",\n \"mscompress\",\n \"msi\",\n \"mum\",\n \"odex\",\n \"odp\",\n \"ods\",\n \"odt\",\n \"ogg\",\n \"outlook\",\n \"pcap\",\n \"pdf\",\n \"pebin\",\n \"pem\",\n \"perl\",\n \"php\",\n \"png\",\n \"postscript\",\n \"powershell\",\n \"ppt\",\n \"pptx\",\n \"python\",\n \"pythonbytecode\",\n \"rar\",\n \"rdf\",\n \"rpm\",\n \"rst\",\n \"rtf\",\n \"ruby\",\n \"rust\",\n \"scala\",\n \"sevenzip\",\n \"shell\",\n \"smali\",\n \"sql\",\n \"squashfs\",\n \"svg\",\n \"swf\",\n \"symlinktext\",\n \"tar\",\n \"tga\",\n \"tiff\",\n \"torrent\",\n \"ttf\",\n \"txt\",\n \"unknown\",\n \"vba\",\n \"wav\",\n \"webm\",\n \"webp\",\n \"winregistry\",\n \"wmf\",\n \"xar\",\n \"xls\",\n \"xlsb\",\n \"xlsx\",\n \"xml\",\n \"xpi\",\n \"xz\",\n \"yaml\",\n \"zip\",\n \"zlibstream\"\n ],\n \"target_labels_to_ids\": {\n \"ai\": 0,\n \"apk\": 1,\n \"appleplist\": 2,\n \"asm\": 3,\n \"asp\": 4,\n \"batch\": 5,\n \"bmp\": 6,\n \"bzip\": 7,\n \"c\": 8,\n \"cab\": 9,\n \"cat\": 10,\n \"chm\": 11,\n \"coff\": 12,\n \"crx\": 13,\n \"cs\": 14,\n \"css\": 15,\n \"csv\": 16,\n \"deb\": 17,\n \"dex\": 18,\n \"dmg\": 19,\n \"doc\": 20,\n \"docx\": 21,\n \"elf\": 22,\n \"emf\": 23,\n \"eml\": 24,\n \"epub\": 25,\n \"flac\": 26,\n \"gif\": 27,\n \"go\": 28,\n \"gzip\": 29,\n \"hlp\": 30,\n \"html\": 31,\n \"ico\": 32,\n \"ini\": 33,\n \"internetshortcut\": 34,\n \"iso\": 35,\n \"jar\": 36,\n \"java\": 37,\n \"javabytecode\": 38,\n \"javascript\": 39,\n \"jpeg\": 40,\n \"json\": 41,\n \"latex\": 42,\n \"lisp\": 43,\n \"lnk\": 44,\n \"m3u\": 45,\n \"macho\": 46,\n \"makefile\": 47,\n \"markdown\": 48,\n \"mht\": 49,\n \"mp3\": 50,\n \"mp4\": 51,\n \"mscompress\": 52,\n \"msi\": 53,\n \"mum\": 54,\n \"odex\": 55,\n \"odp\": 56,\n \"ods\": 57,\n \"odt\": 58,\n \"ogg\": 59,\n \"outlook\": 60,\n \"pcap\": 61,\n \"pdf\": 62,\n \"pebin\": 63,\n \"pem\": 64,\n \"perl\": 65,\n \"php\": 66,\n \"png\": 67,\n \"postscript\": 68,\n \"powershell\": 69,\n \"ppt\": 70,\n \"pptx\": 71,\n \"python\": 72,\n \"pythonbytecode\": 73,\n \"rar\": 74,\n \"rdf\": 75,\n \"rpm\": 76,\n \"rst\": 77,\n \"rtf\": 78,\n \"ruby\": 79,\n \"rust\": 80,\n \"scala\": 81,\n \"sevenzip\": 82,\n \"shell\": 83,\n \"smali\": 84,\n \"sql\": 85,\n \"squashfs\": 86,\n \"svg\": 87,\n \"swf\": 88,\n \"symlinktext\": 89,\n \"tar\": 90,\n \"tga\": 91,\n \"tiff\": 92,\n \"torrent\": 93,\n \"ttf\": 94,\n \"txt\": 95,\n \"unknown\": 96,\n \"vba\": 97,\n \"wav\": 98,\n \"webm\": 99,\n \"webp\": 100,\n \"winregistry\": 101,\n \"wmf\": 102,\n \"xar\": 103,\n \"xls\": 104,\n \"xlsb\": 105,\n \"xlsx\": 106,\n \"xml\": 107,\n \"xpi\": 108,\n \"xz\": 109,\n \"yaml\": 110,\n \"zip\": 111,\n \"zlibstream\": 112\n },\n \"target_labels_num\": 113,\n \"content_types_to_target_labels_ids\": {\n \"ai\": 0,\n \"apk\": 1,\n \"appleplist\": 2,\n \"asm\": 3,\n \"asp\": 4,\n \"batch\": 5,\n \"bmp\": 6,\n \"bzip\": 7,\n \"c\": 8,\n \"cab\": 9,\n \"cat\": 10,\n \"chm\": 11,\n \"coff\": 12,\n \"cpl\": 63,\n \"crx\": 13,\n \"cs\": 14,\n \"css\": 15,\n \"csv\": 16,\n \"deb\": 17,\n \"dex\": 18,\n \"dll\": 63,\n \"dmg\": 19,\n \"doc\": 20,\n \"docx\": 21,\n \"dylib\": 46,\n \"elf\": 22,\n \"emf\": 23,\n \"eml\": 24,\n \"epub\": 25,\n \"exe\": 63,\n \"flac\": 26,\n \"gif\": 27,\n \"go\": 28,\n \"gzip\": 29,\n \"hlp\": 30,\n \"html\": 31,\n \"ico\": 32,\n \"ini\": 33,\n \"internetshortcut\": 34,\n \"iso\": 35,\n \"jar\": 36,\n \"java\": 37,\n \"javabytecode\": 38,\n \"javascript\": 39,\n \"jpeg\": 40,\n \"json\": 41,\n \"ko\": 22,\n \"latex\": 42,\n \"lisp\": 43,\n \"lnk\": 44,\n \"m3u\": 45,\n \"macho\": 46,\n \"makefile\": 47,\n \"markdown\": 48,\n \"mht\": 49,\n \"mp3\": 50,\n \"mp4\": 51,\n \"mscompress\": 52,\n \"msi\": 53,\n \"mui\": 63,\n \"mum\": 54,\n \"ocx\": 63,\n \"odex\": 55,\n \"odp\": 56,\n \"ods\": 57,\n \"odt\": 58,\n \"ogg\": 59,\n \"outlook\": 60,\n \"pcap\": 61,\n \"pdf\": 62,\n \"pem\": 64,\n \"perl\": 65,\n \"php\": 66,\n \"png\": 67,\n \"postscript\": 68,\n \"powershell\": 69,\n \"ppt\": 70,\n \"pptx\": 71,\n \"python\": 72,\n \"pythonbytecode\": 73,\n \"randombytes\": 96,\n \"rar\": 74,\n \"rdf\": 75,\n \"rpm\": 76,\n \"rst\": 77,\n \"rtf\": 78,\n \"ruby\": 79,\n \"rust\": 80,\n \"scala\": 81,\n \"scr\": 63,\n \"sevenzip\": 82,\n \"shell\": 83,\n \"smali\": 84,\n \"so\": 22,\n \"sql\": 85,\n \"squashfs\": 86,\n \"svg\": 87,\n \"swf\": 88,\n \"symlinktext\": 89,\n \"sys\": 63,\n \"tar\": 90,\n \"tga\": 91,\n \"tiff\": 92,\n \"torrent\": 93,\n \"ttf\": 94,\n \"txt\": 95,\n \"vba\": 97,\n \"wav\": 98,\n \"webm\": 99,\n \"webp\": 100,\n \"winregistry\": 101,\n \"wmf\": 102,\n \"xar\": 103,\n \"xls\": 104,\n \"xlsb\": 105,\n \"xlsx\": 106,\n \"xml\": 107,\n \"xpi\": 108,\n \"xz\": 109,\n \"yaml\": 110,\n \"zip\": 111,\n \"zlibstream\": 112\n }\n }\n },\n \"host_id\": \"zrh-rtx4090x1-d4\",\n \"git_last_commit\": \"a67d2c64cfbfc0e03898901899947a16b87b4dac\"\n}" }, { "path": "assets/models/standard_v1/thresholds.json", "content": "{\n \"metadata\": {\n \"min_precision\": 0.995,\n \"min_recall\": 0.99\n },\n \"thresholds\": {\n \"ai\": 0.95,\n \"apk\": 0.95,\n \"appleplist\": 0.95,\n \"asm\": 0.85,\n \"asp\": 0.5,\n \"batch\": 0.95,\n \"bmp\": 0.95,\n \"bzip\": 0.95,\n \"c\": 0.7,\n \"cab\": 0.95,\n \"cat\": 0.95,\n \"chm\": 0.95,\n \"coff\": 0.95,\n \"crx\": 0.95,\n \"cs\": 0.95,\n \"css\": 0.5,\n \"csv\": 0.85,\n \"deb\": 0.95,\n \"dex\": 0.95,\n \"dmg\": 0.95,\n \"doc\": 0.5,\n \"docx\": 0.91,\n \"elf\": 0.95,\n \"emf\": 0.95,\n \"eml\": 0.95,\n \"epub\": 0.95,\n \"flac\": 0.95,\n \"gif\": 0.95,\n \"go\": 0.95,\n \"gzip\": 0.95,\n \"hlp\": 0.95,\n \"html\": 0.95,\n \"ico\": 0.95,\n \"ini\": 0.85,\n \"internetshortcut\": 0.95,\n \"iso\": 0.95,\n \"jar\": 0.95,\n \"java\": 0.91,\n \"javabytecode\": 0.95,\n \"javascript\": 0.8,\n \"jpeg\": 0.95,\n \"json\": 0.75,\n \"latex\": 0.5,\n \"lisp\": 0.95,\n \"lnk\": 0.95,\n \"m3u\": 0.95,\n \"macho\": 0.95,\n \"makefile\": 0.95,\n \"markdown\": 0.95,\n \"mht\": 0.95,\n \"mp3\": 0.95,\n \"mp4\": 0.95,\n \"mscompress\": 0.95,\n \"msi\": 0.7,\n \"mum\": 0.95,\n \"odex\": 0.95,\n \"odp\": 0.95,\n \"ods\": 0.95,\n \"odt\": 0.95,\n \"ogg\": 0.95,\n \"outlook\": 0.95,\n \"pcap\": 0.95,\n \"pdf\": 0.95,\n \"pebin\": 0.95,\n \"pem\": 0.95,\n \"perl\": 0.85,\n \"php\": 0.95,\n \"png\": 0.95,\n \"postscript\": 0.95,\n \"powershell\": 0.95,\n \"ppt\": 0.95,\n \"pptx\": 0.95,\n \"python\": 0.85,\n \"pythonbytecode\": 0.95,\n \"rar\": 0.95,\n \"rdf\": 0.95,\n \"rpm\": 0.95,\n \"rst\": 0.95,\n \"rtf\": 0.95,\n \"ruby\": 0.93,\n \"rust\": 0.95,\n \"scala\": 0.95,\n \"sevenzip\": 0.95,\n \"shell\": 0.95,\n \"smali\": 0.95,\n \"sql\": 0.7,\n \"squashfs\": 0.95,\n \"svg\": 0.95,\n \"swf\": 0.95,\n \"symlinktext\": 0.95,\n \"tar\": 0.95,\n \"tga\": 0.95,\n \"tiff\": 0.95,\n \"torrent\": 0.95,\n \"ttf\": 0.95,\n \"txt\": 0.95,\n \"unknown\": 0.95,\n \"vba\": 0.5,\n \"wav\": 0.95,\n \"webm\": 0.95,\n \"webp\": 0.95,\n \"winregistry\": 0.95,\n \"wmf\": 0.95,\n \"xar\": 0.95,\n \"xls\": 0.8,\n \"xlsb\": 0.55,\n \"xlsx\": 0.95,\n \"xml\": 0.9,\n \"xpi\": 0.93,\n \"xz\": 0.95,\n \"yaml\": 0.75,\n \"zip\": 0.95,\n \"zlibstream\": 0.95\n }\n}" }, { "path": "assets/models/standard_v2_0/README.md", "content": "# Content types supported by model \"standard_v2_0\"\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | qt | QuickTime |\n| 147 | r | R (language) |\n| 148 | rar | RAR archive data |\n| 149 | rdf | Resource Description Framework document (RDF) |\n| 150 | rpm | RedHat Package Manager archive (RPM) |\n| 151 | rst | ReStructuredText document |\n| 152 | rtf | Rich Text Format document |\n| 153 | ruby | Ruby source |\n| 154 | rust | Rust source |\n| 155 | scala | Scala source |\n| 156 | scss | SCSS source |\n| 157 | sevenzip | 7-zip archive data |\n| 158 | sgml | sgml |\n| 159 | shell | Shell script |\n| 160 | smali | Smali source |\n| 161 | snap | Snap archive |\n| 162 | solidity | Solidity source |\n| 163 | sql | SQL source |\n| 164 | sqlite | SQLITE database |\n| 165 | squashfs | Squash filesystem |\n| 166 | srt | SubRip Text Format |\n| 167 | stlbinary | Stereolithography CAD (binary) |\n| 168 | stltext | Stereolithography CAD (text) |\n| 169 | sum | Checksum file |\n| 170 | svg | SVG Scalable Vector Graphics image data |\n| 171 | swf | Small Web File |\n| 172 | swift | Swift |\n| 173 | tar | POSIX tar archive |\n| 174 | tcl | Tickle |\n| 175 | textproto | Text protocol buffer |\n| 176 | tga | Targa image data |\n| 177 | thumbsdb | Windows thumbnail cache |\n| 178 | tiff | TIFF image data |\n| 179 | toml | Tom's obvious, minimal language |\n| 180 | torrent | BitTorrent file |\n| 181 | tsv | TSV document |\n| 182 | ttf | TrueType Font data |\n| 183 | twig | Twig template |\n| 184 | txt | Generic text document |\n| 185 | typescript | Typescript |\n| 186 | unknown | Unknown binary data |\n| 187 | vba | MS Visual Basic source (VBA) |\n| 188 | vcxproj | Visual Studio MSBuild project |\n| 189 | verilog | Verilog source |\n| 190 | vhdl | VHDL source |\n| 191 | vtt | Web Video Text Tracks |\n| 192 | vue | Vue source |\n| 193 | wasm | Web Assembly |\n| 194 | wav | Waveform Audio file (WAV) |\n| 195 | webm | WebM media file |\n| 196 | webp | WebP media file |\n| 197 | winregistry | Windows Registry text |\n| 198 | wmf | Windows metafile |\n| 199 | woff | Web Open Font Format |\n| 200 | woff2 | Web Open Font Format v2 |\n| 201 | xar | XAR archive compressed data |\n| 202 | xls | Microsoft Excel CDF document |\n| 203 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 204 | xlsx | Microsoft Excel 2007+ document |\n| 205 | xml | XML document |\n| 206 | xpi | Compressed installation archive (XPI) |\n| 207 | xz | XZ compressed data |\n| 208 | yaml | YAML source |\n| 209 | yara | YARA rule |\n| 210 | zig | Zig source |\n| 211 | zip | Zip archive data |\n| 212 | zlibstream | zlib compressed data |\n" }, { "path": "assets/models/standard_v2_0/config.min.json", "content": "{\"beg_size\": 2048, \"mid_size\": 0, \"end_size\": 2048, \"use_inputs_at_offsets\": false, \"medium_confidence_threshold\": 0.5, \"min_file_size_for_dl\": 8, \"padding_token\": 256, \"block_size\": 4096, \"target_labels_space\": [\"3gp\", \"ace\", \"ai\", \"aidl\", \"apk\", \"applebplist\", \"appleplist\", \"asm\", \"asp\", \"autohotkey\", \"autoit\", \"awk\", \"batch\", \"bazel\", \"bib\", \"bmp\", \"bzip\", \"c\", \"cab\", \"cat\", \"chm\", \"clojure\", \"cmake\", \"cobol\", \"coff\", \"coffeescript\", \"cpp\", \"crt\", \"crx\", \"cs\", \"csproj\", \"css\", \"csv\", \"dart\", \"deb\", \"dex\", \"dicom\", \"diff\", \"dm\", \"dmg\", \"doc\", \"dockerfile\", \"docx\", \"dsstore\", \"dwg\", \"dxf\", \"elf\", \"elixir\", \"emf\", \"eml\", \"epub\", \"erb\", \"erlang\", \"flac\", \"flv\", \"fortran\", \"gemfile\", \"gemspec\", \"gif\", \"gitattributes\", \"gitmodules\", \"go\", \"gradle\", \"groovy\", \"gzip\", \"h5\", \"handlebars\", \"haskell\", \"hcl\", \"hlp\", \"htaccess\", \"html\", \"icns\", \"ico\", \"ics\", \"ignorefile\", \"ini\", \"internetshortcut\", \"ipynb\", \"iso\", \"jar\", \"java\", \"javabytecode\", \"javascript\", \"jinja\", \"jp2\", \"jpeg\", \"json\", \"jsonl\", \"julia\", \"kotlin\", \"latex\", \"lha\", \"lisp\", \"lnk\", \"lua\", \"m3u\", \"m4\", \"macho\", \"makefile\", \"markdown\", \"matlab\", \"mht\", \"midi\", \"mkv\", \"mp3\", \"mp4\", \"mscompress\", \"msi\", \"mum\", \"npy\", \"npz\", \"nupkg\", \"objectivec\", \"ocaml\", \"odp\", \"ods\", \"odt\", \"ogg\", \"one\", \"onnx\", \"otf\", \"outlook\", \"parquet\", \"pascal\", \"pcap\", \"pdb\", \"pdf\", \"pebin\", \"pem\", \"perl\", \"php\", \"pickle\", \"png\", \"po\", \"postscript\", \"powershell\", \"ppt\", \"pptx\", \"prolog\", \"proteindb\", \"proto\", \"psd\", \"python\", \"pythonbytecode\", \"qt\", \"r\", \"rar\", \"rdf\", \"rpm\", \"rst\", \"rtf\", \"ruby\", \"rust\", \"scala\", \"scss\", \"sevenzip\", \"sgml\", \"shell\", \"smali\", \"snap\", \"solidity\", \"sql\", \"sqlite\", \"squashfs\", \"srt\", \"stlbinary\", \"stltext\", \"sum\", \"svg\", \"swf\", \"swift\", \"tar\", \"tcl\", \"textproto\", \"tga\", \"thumbsdb\", \"tiff\", \"toml\", \"torrent\", \"tsv\", \"ttf\", \"twig\", \"txt\", \"typescript\", \"unknown\", \"vba\", \"vcxproj\", \"verilog\", \"vhdl\", \"vtt\", \"vue\", \"wasm\", \"wav\", \"webm\", \"webp\", \"winregistry\", \"wmf\", \"woff\", \"woff2\", \"xar\", \"xls\", \"xlsb\", \"xlsx\", \"xml\", \"xpi\", \"xz\", \"yaml\", \"yara\", \"zig\", \"zip\", \"zlibstream\"], \"thresholds\": {\"latex\": 0.95}, \"overwrite_map\": {}}\n" }, { "path": "assets/models/standard_v2_0/metadata.json", "content": "{\"model_name_hash\": \"5ae665b58305628b173e97edf9d3043a6021dd40a434b1f72bd88087713d8209\", \"namespace_hash\": \"c3a09b7885a7151502927c0380d2bf9a19c2999e54e8ffd282b21ad7b2aeec62\"}" }, { "path": "assets/models/standard_v2_1/README.md", "content": "# Content types supported by model \"standard_v2_1\"\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | pytorch | Pytorch storage file |\n| 147 | qt | QuickTime |\n| 148 | r | R (language) |\n| 149 | rar | RAR archive data |\n| 150 | rdf | Resource Description Framework document (RDF) |\n| 151 | rpm | RedHat Package Manager archive (RPM) |\n| 152 | rst | ReStructuredText document |\n| 153 | rtf | Rich Text Format document |\n| 154 | ruby | Ruby source |\n| 155 | rust | Rust source |\n| 156 | scala | Scala source |\n| 157 | scss | SCSS source |\n| 158 | sevenzip | 7-zip archive data |\n| 159 | sgml | sgml |\n| 160 | shell | Shell script |\n| 161 | smali | Smali source |\n| 162 | snap | Snap archive |\n| 163 | solidity | Solidity source |\n| 164 | sql | SQL source |\n| 165 | sqlite | SQLITE database |\n| 166 | squashfs | Squash filesystem |\n| 167 | srt | SubRip Text Format |\n| 168 | stlbinary | Stereolithography CAD (binary) |\n| 169 | stltext | Stereolithography CAD (text) |\n| 170 | sum | Checksum file |\n| 171 | svg | SVG Scalable Vector Graphics image data |\n| 172 | swf | Small Web File |\n| 173 | swift | Swift |\n| 174 | tar | POSIX tar archive |\n| 175 | tcl | Tickle |\n| 176 | textproto | Text protocol buffer |\n| 177 | tga | Targa image data |\n| 178 | thumbsdb | Windows thumbnail cache |\n| 179 | tiff | TIFF image data |\n| 180 | toml | Tom's obvious, minimal language |\n| 181 | torrent | BitTorrent file |\n| 182 | tsv | TSV document |\n| 183 | ttf | TrueType Font data |\n| 184 | twig | Twig template |\n| 185 | txt | Generic text document |\n| 186 | typescript | Typescript |\n| 187 | unknown | Unknown binary data |\n| 188 | vba | MS Visual Basic source (VBA) |\n| 189 | vcxproj | Visual Studio MSBuild project |\n| 190 | verilog | Verilog source |\n| 191 | vhdl | VHDL source |\n| 192 | vtt | Web Video Text Tracks |\n| 193 | vue | Vue source |\n| 194 | wasm | Web Assembly |\n| 195 | wav | Waveform Audio file (WAV) |\n| 196 | webm | WebM media file |\n| 197 | webp | WebP media file |\n| 198 | winregistry | Windows Registry text |\n| 199 | wmf | Windows metafile |\n| 200 | woff | Web Open Font Format |\n| 201 | woff2 | Web Open Font Format v2 |\n| 202 | xar | XAR archive compressed data |\n| 203 | xls | Microsoft Excel CDF document |\n| 204 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 205 | xlsx | Microsoft Excel 2007+ document |\n| 206 | xml | XML document |\n| 207 | xpi | Compressed installation archive (XPI) |\n| 208 | xz | XZ compressed data |\n| 209 | yaml | YAML source |\n| 210 | yara | YARA rule |\n| 211 | zig | Zig source |\n| 212 | zip | Zip archive data |\n| 213 | zlibstream | zlib compressed data |\n" }, { "path": "assets/models/standard_v2_1/config.min.json", "content": "{\"beg_size\": 2048, \"mid_size\": 0, \"end_size\": 2048, \"use_inputs_at_offsets\": false, \"medium_confidence_threshold\": 0.5, \"min_file_size_for_dl\": 8, \"padding_token\": 256, \"block_size\": 4096, \"target_labels_space\": [\"3gp\", \"ace\", \"ai\", \"aidl\", \"apk\", \"applebplist\", \"appleplist\", \"asm\", \"asp\", \"autohotkey\", \"autoit\", \"awk\", \"batch\", \"bazel\", \"bib\", \"bmp\", \"bzip\", \"c\", \"cab\", \"cat\", \"chm\", \"clojure\", \"cmake\", \"cobol\", \"coff\", \"coffeescript\", \"cpp\", \"crt\", \"crx\", \"cs\", \"csproj\", \"css\", \"csv\", \"dart\", \"deb\", \"dex\", \"dicom\", \"diff\", \"dm\", \"dmg\", \"doc\", \"dockerfile\", \"docx\", \"dsstore\", \"dwg\", \"dxf\", \"elf\", \"elixir\", \"emf\", \"eml\", \"epub\", \"erb\", \"erlang\", \"flac\", \"flv\", \"fortran\", \"gemfile\", \"gemspec\", \"gif\", \"gitattributes\", \"gitmodules\", \"go\", \"gradle\", \"groovy\", \"gzip\", \"h5\", \"handlebars\", \"haskell\", \"hcl\", \"hlp\", \"htaccess\", \"html\", \"icns\", \"ico\", \"ics\", \"ignorefile\", \"ini\", \"internetshortcut\", \"ipynb\", \"iso\", \"jar\", \"java\", \"javabytecode\", \"javascript\", \"jinja\", \"jp2\", \"jpeg\", \"json\", \"jsonl\", \"julia\", \"kotlin\", \"latex\", \"lha\", \"lisp\", \"lnk\", \"lua\", \"m3u\", \"m4\", \"macho\", \"makefile\", \"markdown\", \"matlab\", \"mht\", \"midi\", \"mkv\", \"mp3\", \"mp4\", \"mscompress\", \"msi\", \"mum\", \"npy\", \"npz\", \"nupkg\", \"objectivec\", \"ocaml\", \"odp\", \"ods\", \"odt\", \"ogg\", \"one\", \"onnx\", \"otf\", \"outlook\", \"parquet\", \"pascal\", \"pcap\", \"pdb\", \"pdf\", \"pebin\", \"pem\", \"perl\", \"php\", \"pickle\", \"png\", \"po\", \"postscript\", \"powershell\", \"ppt\", \"pptx\", \"prolog\", \"proteindb\", \"proto\", \"psd\", \"python\", \"pythonbytecode\", \"pytorch\", \"qt\", \"r\", \"rar\", \"rdf\", \"rpm\", \"rst\", \"rtf\", \"ruby\", \"rust\", \"scala\", \"scss\", \"sevenzip\", \"sgml\", \"shell\", \"smali\", \"snap\", \"solidity\", \"sql\", \"sqlite\", \"squashfs\", \"srt\", \"stlbinary\", \"stltext\", \"sum\", \"svg\", \"swf\", \"swift\", \"tar\", \"tcl\", \"textproto\", \"tga\", \"thumbsdb\", \"tiff\", \"toml\", \"torrent\", \"tsv\", \"ttf\", \"twig\", \"txt\", \"typescript\", \"unknown\", \"vba\", \"vcxproj\", \"verilog\", \"vhdl\", \"vtt\", \"vue\", \"wasm\", \"wav\", \"webm\", \"webp\", \"winregistry\", \"wmf\", \"woff\", \"woff2\", \"xar\", \"xls\", \"xlsb\", \"xlsx\", \"xml\", \"xpi\", \"xz\", \"yaml\", \"yara\", \"zig\", \"zip\", \"zlibstream\"], \"thresholds\": {\"latex\": 0.95, \"pascal\": 0.95}, \"overwrite_map\": {}}\n" }, { "path": "assets/models/standard_v2_1/metadata.json", "content": "{\"model_name_hash\": \"5ae665b58305628b173e97edf9d3043a6021dd40a434b1f72bd88087713d8209\", \"namespace_hash\": \"ce3c9130af6416f40d71c5934f927acbd174f904a550fca2185aa3cd3528ca35\"}\n" }, { "path": "assets/models/standard_v3_0/README.md", "content": "# Model documentation\n\n## Table of Contents\n\n1. [List of possible outputs](#list-of-possible-outputs)\n1. [List of possible model's outputs](#list-of-possible-models-outputs)\n\n## List of possible outputs\n\nThis is the full list of all possible tool's outputs (which are different than the possible raw output of the model, see table below). E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.output.label`.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | directory | A directory |\n| 40 | dm | Dream Maker |\n| 41 | dmg | Apple disk image |\n| 42 | doc | Microsoft Word CDF document |\n| 43 | dockerfile | Dockerfile |\n| 44 | docx | Microsoft Word 2007+ document |\n| 45 | dsstore | Application Desktop Services Store |\n| 46 | dwg | Autocad Drawing |\n| 47 | dxf | Audocad Drawing Exchange Format |\n| 48 | elf | ELF executable |\n| 49 | elixir | Elixir script |\n| 50 | emf | Windows Enhanced Metafile image data |\n| 51 | eml | RFC 822 mail |\n| 52 | empty | Empty file |\n| 53 | epub | EPUB document |\n| 54 | erb | Embedded Ruby source |\n| 55 | erlang | Erlang source |\n| 56 | flac | FLAC audio bitstream data |\n| 57 | flv | Flash Video |\n| 58 | fortran | Fortran |\n| 59 | gemfile | Gemfile file |\n| 60 | gemspec | Gemspec file |\n| 61 | gif | GIF image data |\n| 62 | gitattributes | Gitattributes file |\n| 63 | gitmodules | Gitmodules file |\n| 64 | go | Golang source |\n| 65 | gradle | Gradle source |\n| 66 | groovy | Groovy source |\n| 67 | gzip | gzip compressed data |\n| 68 | h5 | Hierarchical Data Format v5 |\n| 69 | handlebars | Handlebars source |\n| 70 | haskell | Haskell source |\n| 71 | hcl | HashiCorp configuration language |\n| 72 | hlp | MS Windows help |\n| 73 | htaccess | Apache access configuration |\n| 74 | html | HTML document |\n| 75 | icns | Mac OS X icon |\n| 76 | ico | MS Windows icon resource |\n| 77 | ics | Internet Calendaring and Scheduling |\n| 78 | ignorefile | Ignorefile |\n| 79 | ini | INI configuration file |\n| 80 | internetshortcut | MS Windows Internet shortcut |\n| 81 | ipynb | Jupyter notebook |\n| 82 | iso | ISO 9660 CD-ROM filesystem data |\n| 83 | jar | Java archive data (JAR) |\n| 84 | java | Java source |\n| 85 | javabytecode | Java compiled bytecode |\n| 86 | javascript | JavaScript source |\n| 87 | jinja | Jinja template |\n| 88 | jp2 | jpeg2000 |\n| 89 | jpeg | JPEG image data |\n| 90 | json | JSON document |\n| 91 | jsonl | JSONL document |\n| 92 | julia | Julia source |\n| 93 | kotlin | Kotlin source |\n| 94 | latex | LaTeX document |\n| 95 | lha | LHarc archive |\n| 96 | lisp | Lisp source |\n| 97 | lnk | MS Windows shortcut |\n| 98 | lua | Lua |\n| 99 | m3u | M3U playlist |\n| 100 | m4 | GNU Macro |\n| 101 | macho | Mach-O executable |\n| 102 | makefile | Makefile source |\n| 103 | markdown | Markdown document |\n| 104 | matlab | Matlab Source |\n| 105 | mht | MHTML document |\n| 106 | midi | Midi |\n| 107 | mkv | Matroska |\n| 108 | mp3 | MP3 media file |\n| 109 | mp4 | MP4 media file |\n| 110 | mscompress | MS Compress archive data |\n| 111 | msi | Microsoft Installer file |\n| 112 | mum | Windows Update Package file |\n| 113 | npy | Numpy Array |\n| 114 | npz | Numpy Arrays Archive |\n| 115 | nupkg | NuGet Package |\n| 116 | objectivec | ObjectiveC source |\n| 117 | ocaml | OCaml |\n| 118 | odp | OpenDocument Presentation |\n| 119 | ods | OpenDocument Spreadsheet |\n| 120 | odt | OpenDocument Text |\n| 121 | ogg | Ogg data |\n| 122 | one | One Note |\n| 123 | onnx | Open Neural Network Exchange |\n| 124 | otf | OpenType font |\n| 125 | outlook | MS Outlook Message |\n| 126 | parquet | Apache Parquet |\n| 127 | pascal | Pascal source |\n| 128 | pcap | pcap capture file |\n| 129 | pdb | Windows Program Database |\n| 130 | pdf | PDF document |\n| 131 | pebin | PE Windows executable |\n| 132 | pem | PEM certificate |\n| 133 | perl | Perl source |\n| 134 | php | PHP source |\n| 135 | pickle | Python pickle |\n| 136 | png | PNG image |\n| 137 | po | Portable Object (PO) for i18n |\n| 138 | postscript | PostScript document |\n| 139 | powershell | Powershell source |\n| 140 | ppt | Microsoft PowerPoint CDF document |\n| 141 | pptx | Microsoft PowerPoint 2007+ document |\n| 142 | prolog | Prolog source |\n| 143 | proteindb | Protein DB |\n| 144 | proto | Protocol buffer definition |\n| 145 | psd | Adobe Photoshop |\n| 146 | python | Python source |\n| 147 | pythonbytecode | Python compiled bytecode |\n| 148 | pytorch | Pytorch storage file |\n| 149 | qt | QuickTime |\n| 150 | r | R (language) |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | symlink | Symbolic link |\n| 177 | tar | POSIX tar archive |\n| 178 | tcl | Tickle |\n| 179 | textproto | Text protocol buffer |\n| 180 | tga | Targa image data |\n| 181 | thumbsdb | Windows thumbnail cache |\n| 182 | tiff | TIFF image data |\n| 183 | toml | Tom's obvious, minimal language |\n| 184 | torrent | BitTorrent file |\n| 185 | tsv | TSV document |\n| 186 | ttf | TrueType Font data |\n| 187 | twig | Twig template |\n| 188 | txt | Generic text document |\n| 189 | typescript | Typescript |\n| 190 | unknown | Unknown binary data |\n| 191 | vba | MS Visual Basic source (VBA) |\n| 192 | vcxproj | Visual Studio MSBuild project |\n| 193 | verilog | Verilog source |\n| 194 | vhdl | VHDL source |\n| 195 | vtt | Web Video Text Tracks |\n| 196 | vue | Vue source |\n| 197 | wasm | Web Assembly |\n| 198 | wav | Waveform Audio file (WAV) |\n| 199 | webm | WebM media file |\n| 200 | webp | WebP media file |\n| 201 | winregistry | Windows Registry text |\n| 202 | wmf | Windows metafile |\n| 203 | woff | Web Open Font Format |\n| 204 | woff2 | Web Open Font Format v2 |\n| 205 | xar | XAR archive compressed data |\n| 206 | xls | Microsoft Excel CDF document |\n| 207 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 208 | xlsx | Microsoft Excel 2007+ document |\n| 209 | xml | XML document |\n| 210 | xpi | Compressed installation archive (XPI) |\n| 211 | xz | XZ compressed data |\n| 212 | yaml | YAML source |\n| 213 | yara | YARA rule |\n| 214 | zig | Zig source |\n| 215 | zip | Zip archive data |\n| 216 | zlibstream | zlib compressed data |\n\n\n## List of possible model's outputs\n\nThis is the full list of all possible model's output. E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.dl.label`. Note that, in general, the list of \"model outputs\" is different than the \"tool outputs\" as in some cases the model is not even used, or the model's output is overwritten due to a low-confidence score or other reasons. This list is useful mostly for debugging purposes; the vast majority of client should just consult the table above.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | pytorch | Pytorch storage file |\n| 147 | qt | QuickTime |\n| 148 | r | R (language) |\n| 149 | randombytes | Random bytes |\n| 150 | randomtxt | Random text |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | tar | POSIX tar archive |\n| 177 | tcl | Tickle |\n| 178 | textproto | Text protocol buffer |\n| 179 | tga | Targa image data |\n| 180 | thumbsdb | Windows thumbnail cache |\n| 181 | tiff | TIFF image data |\n| 182 | toml | Tom's obvious, minimal language |\n| 183 | torrent | BitTorrent file |\n| 184 | tsv | TSV document |\n| 185 | ttf | TrueType Font data |\n| 186 | twig | Twig template |\n| 187 | txt | Generic text document |\n| 188 | typescript | Typescript |\n| 189 | undefined | Undefined |\n| 190 | vba | MS Visual Basic source (VBA) |\n| 191 | vcxproj | Visual Studio MSBuild project |\n| 192 | verilog | Verilog source |\n| 193 | vhdl | VHDL source |\n| 194 | vtt | Web Video Text Tracks |\n| 195 | vue | Vue source |\n| 196 | wasm | Web Assembly |\n| 197 | wav | Waveform Audio file (WAV) |\n| 198 | webm | WebM media file |\n| 199 | webp | WebP media file |\n| 200 | winregistry | Windows Registry text |\n| 201 | wmf | Windows metafile |\n| 202 | woff | Web Open Font Format |\n| 203 | woff2 | Web Open Font Format v2 |\n| 204 | xar | XAR archive compressed data |\n| 205 | xls | Microsoft Excel CDF document |\n| 206 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 207 | xlsx | Microsoft Excel 2007+ document |\n| 208 | xml | XML document |\n| 209 | xpi | Compressed installation archive (XPI) |\n| 210 | xz | XZ compressed data |\n| 211 | yaml | YAML source |\n| 212 | yara | YARA rule |\n| 213 | zig | Zig source |\n| 214 | zip | Zip archive data |\n| 215 | zlibstream | zlib compressed data |>" }, { "path": "assets/models/standard_v3_0/config.min.json", "content": "{\"beg_size\":1024,\"mid_size\":0,\"end_size\":1024,\"use_inputs_at_offsets\":false,\"medium_confidence_threshold\":0.5,\"min_file_size_for_dl\":8,\"padding_token\":256,\"block_size\":4096,\"target_labels_space\":[\"3gp\",\"ace\",\"ai\",\"aidl\",\"apk\",\"applebplist\",\"appleplist\",\"asm\",\"asp\",\"autohotkey\",\"autoit\",\"awk\",\"batch\",\"bazel\",\"bib\",\"bmp\",\"bzip\",\"c\",\"cab\",\"cat\",\"chm\",\"clojure\",\"cmake\",\"cobol\",\"coff\",\"coffeescript\",\"cpp\",\"crt\",\"crx\",\"cs\",\"csproj\",\"css\",\"csv\",\"dart\",\"deb\",\"dex\",\"dicom\",\"diff\",\"dm\",\"dmg\",\"doc\",\"dockerfile\",\"docx\",\"dsstore\",\"dwg\",\"dxf\",\"elf\",\"elixir\",\"emf\",\"eml\",\"epub\",\"erb\",\"erlang\",\"flac\",\"flv\",\"fortran\",\"gemfile\",\"gemspec\",\"gif\",\"gitattributes\",\"gitmodules\",\"go\",\"gradle\",\"groovy\",\"gzip\",\"h5\",\"handlebars\",\"haskell\",\"hcl\",\"hlp\",\"htaccess\",\"html\",\"icns\",\"ico\",\"ics\",\"ignorefile\",\"ini\",\"internetshortcut\",\"ipynb\",\"iso\",\"jar\",\"java\",\"javabytecode\",\"javascript\",\"jinja\",\"jp2\",\"jpeg\",\"json\",\"jsonl\",\"julia\",\"kotlin\",\"latex\",\"lha\",\"lisp\",\"lnk\",\"lua\",\"m3u\",\"m4\",\"macho\",\"makefile\",\"markdown\",\"matlab\",\"mht\",\"midi\",\"mkv\",\"mp3\",\"mp4\",\"mscompress\",\"msi\",\"mum\",\"npy\",\"npz\",\"nupkg\",\"objectivec\",\"ocaml\",\"odp\",\"ods\",\"odt\",\"ogg\",\"one\",\"onnx\",\"otf\",\"outlook\",\"parquet\",\"pascal\",\"pcap\",\"pdb\",\"pdf\",\"pebin\",\"pem\",\"perl\",\"php\",\"pickle\",\"png\",\"po\",\"postscript\",\"powershell\",\"ppt\",\"pptx\",\"prolog\",\"proteindb\",\"proto\",\"psd\",\"python\",\"pythonbytecode\",\"pytorch\",\"qt\",\"r\",\"randombytes\",\"randomtxt\",\"rar\",\"rdf\",\"rpm\",\"rst\",\"rtf\",\"ruby\",\"rust\",\"scala\",\"scss\",\"sevenzip\",\"sgml\",\"shell\",\"smali\",\"snap\",\"solidity\",\"sql\",\"sqlite\",\"squashfs\",\"srt\",\"stlbinary\",\"stltext\",\"sum\",\"svg\",\"swf\",\"swift\",\"tar\",\"tcl\",\"textproto\",\"tga\",\"thumbsdb\",\"tiff\",\"toml\",\"torrent\",\"tsv\",\"ttf\",\"twig\",\"txt\",\"typescript\",\"vba\",\"vcxproj\",\"verilog\",\"vhdl\",\"vtt\",\"vue\",\"wasm\",\"wav\",\"webm\",\"webp\",\"winregistry\",\"wmf\",\"woff\",\"woff2\",\"xar\",\"xls\",\"xlsb\",\"xlsx\",\"xml\",\"xpi\",\"xz\",\"yaml\",\"yara\",\"zig\",\"zip\",\"zlibstream\"],\"thresholds\":{\"handlebars\":0.9,\"ignorefile\":0.95,\"latex\":0.95,\"markdown\":0.9,\"ocaml\":0.9,\"pascal\":0.95,\"rst\":0.9,\"sql\":0.9,\"tsv\":0.9},\"overwrite_map\":{\"randombytes\":\"unknown\",\"randomtxt\":\"txt\"},\"version_major\":3}" }, { "path": "assets/models/standard_v3_0/metadata.json", "content": "{\"namespace_hash\":\"7ca577b96738951c36df428f8435c81780f92c6f9ef3a73d796a792ffc817703\",\"model_name_hash\":\"e5368af178b89eb94cb1927b8481d5a9dedf5d1ee810d335b018d19eb5195b45\",\"epoch_num\":\"100\"}" }, { "path": "assets/models/standard_v3_1/README.md", "content": "# Model documentation\n\n## Table of Contents\n\n1. [List of possible outputs](#list-of-possible-outputs)\n1. [List of possible model's outputs](#list-of-possible-models-outputs)\n\n## List of possible outputs\n\nThis is the full list of all possible tool's outputs (which are different than the possible raw output of the model, see table below). E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.output.label`.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | directory | A directory |\n| 40 | dm | Dream Maker |\n| 41 | dmg | Apple disk image |\n| 42 | doc | Microsoft Word CDF document |\n| 43 | dockerfile | Dockerfile |\n| 44 | docx | Microsoft Word 2007+ document |\n| 45 | dsstore | Application Desktop Services Store |\n| 46 | dwg | Autocad Drawing |\n| 47 | dxf | Audocad Drawing Exchange Format |\n| 48 | elf | ELF executable |\n| 49 | elixir | Elixir script |\n| 50 | emf | Windows Enhanced Metafile image data |\n| 51 | eml | RFC 822 mail |\n| 52 | empty | Empty file |\n| 53 | epub | EPUB document |\n| 54 | erb | Embedded Ruby source |\n| 55 | erlang | Erlang source |\n| 56 | flac | FLAC audio bitstream data |\n| 57 | flv | Flash Video |\n| 58 | fortran | Fortran |\n| 59 | gemfile | Gemfile file |\n| 60 | gemspec | Gemspec file |\n| 61 | gif | GIF image data |\n| 62 | gitattributes | Gitattributes file |\n| 63 | gitmodules | Gitmodules file |\n| 64 | go | Golang source |\n| 65 | gradle | Gradle source |\n| 66 | groovy | Groovy source |\n| 67 | gzip | gzip compressed data |\n| 68 | h5 | Hierarchical Data Format v5 |\n| 69 | handlebars | Handlebars source |\n| 70 | haskell | Haskell source |\n| 71 | hcl | HashiCorp configuration language |\n| 72 | hlp | MS Windows help |\n| 73 | htaccess | Apache access configuration |\n| 74 | html | HTML document |\n| 75 | icns | Mac OS X icon |\n| 76 | ico | MS Windows icon resource |\n| 77 | ics | Internet Calendaring and Scheduling |\n| 78 | ignorefile | Ignorefile |\n| 79 | ini | INI configuration file |\n| 80 | internetshortcut | MS Windows Internet shortcut |\n| 81 | ipynb | Jupyter notebook |\n| 82 | iso | ISO 9660 CD-ROM filesystem data |\n| 83 | jar | Java archive data (JAR) |\n| 84 | java | Java source |\n| 85 | javabytecode | Java compiled bytecode |\n| 86 | javascript | JavaScript source |\n| 87 | jinja | Jinja template |\n| 88 | jp2 | jpeg2000 |\n| 89 | jpeg | JPEG image data |\n| 90 | json | JSON document |\n| 91 | jsonl | JSONL document |\n| 92 | julia | Julia source |\n| 93 | kotlin | Kotlin source |\n| 94 | latex | LaTeX document |\n| 95 | lha | LHarc archive |\n| 96 | lisp | Lisp source |\n| 97 | lnk | MS Windows shortcut |\n| 98 | lua | Lua |\n| 99 | m3u | M3U playlist |\n| 100 | m4 | GNU Macro |\n| 101 | macho | Mach-O executable |\n| 102 | makefile | Makefile source |\n| 103 | markdown | Markdown document |\n| 104 | matlab | Matlab Source |\n| 105 | mht | MHTML document |\n| 106 | midi | Midi |\n| 107 | mkv | Matroska |\n| 108 | mp3 | MP3 media file |\n| 109 | mp4 | MP4 media file |\n| 110 | mscompress | MS Compress archive data |\n| 111 | msi | Microsoft Installer file |\n| 112 | mum | Windows Update Package file |\n| 113 | npy | Numpy Array |\n| 114 | npz | Numpy Arrays Archive |\n| 115 | nupkg | NuGet Package |\n| 116 | objectivec | ObjectiveC source |\n| 117 | ocaml | OCaml |\n| 118 | odp | OpenDocument Presentation |\n| 119 | ods | OpenDocument Spreadsheet |\n| 120 | odt | OpenDocument Text |\n| 121 | ogg | Ogg data |\n| 122 | one | One Note |\n| 123 | onnx | Open Neural Network Exchange |\n| 124 | otf | OpenType font |\n| 125 | outlook | MS Outlook Message |\n| 126 | parquet | Apache Parquet |\n| 127 | pascal | Pascal source |\n| 128 | pcap | pcap capture file |\n| 129 | pdb | Windows Program Database |\n| 130 | pdf | PDF document |\n| 131 | pebin | PE Windows executable |\n| 132 | pem | PEM certificate |\n| 133 | perl | Perl source |\n| 134 | php | PHP source |\n| 135 | pickle | Python pickle |\n| 136 | png | PNG image |\n| 137 | po | Portable Object (PO) for i18n |\n| 138 | postscript | PostScript document |\n| 139 | powershell | Powershell source |\n| 140 | ppt | Microsoft PowerPoint CDF document |\n| 141 | pptx | Microsoft PowerPoint 2007+ document |\n| 142 | prolog | Prolog source |\n| 143 | proteindb | Protein DB |\n| 144 | proto | Protocol buffer definition |\n| 145 | psd | Adobe Photoshop |\n| 146 | python | Python source |\n| 147 | pythonbytecode | Python compiled bytecode |\n| 148 | pytorch | Pytorch storage file |\n| 149 | qt | QuickTime |\n| 150 | r | R (language) |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | symlink | Symbolic link |\n| 177 | tar | POSIX tar archive |\n| 178 | tcl | Tickle |\n| 179 | textproto | Text protocol buffer |\n| 180 | tga | Targa image data |\n| 181 | thumbsdb | Windows thumbnail cache |\n| 182 | tiff | TIFF image data |\n| 183 | toml | Tom's obvious, minimal language |\n| 184 | torrent | BitTorrent file |\n| 185 | tsv | TSV document |\n| 186 | ttf | TrueType Font data |\n| 187 | twig | Twig template |\n| 188 | txt | Generic text document |\n| 189 | typescript | Typescript |\n| 190 | unknown | Unknown binary data |\n| 191 | vba | MS Visual Basic source (VBA) |\n| 192 | vcxproj | Visual Studio MSBuild project |\n| 193 | verilog | Verilog source |\n| 194 | vhdl | VHDL source |\n| 195 | vtt | Web Video Text Tracks |\n| 196 | vue | Vue source |\n| 197 | wasm | Web Assembly |\n| 198 | wav | Waveform Audio file (WAV) |\n| 199 | webm | WebM media file |\n| 200 | webp | WebP media file |\n| 201 | winregistry | Windows Registry text |\n| 202 | wmf | Windows metafile |\n| 203 | woff | Web Open Font Format |\n| 204 | woff2 | Web Open Font Format v2 |\n| 205 | xar | XAR archive compressed data |\n| 206 | xls | Microsoft Excel CDF document |\n| 207 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 208 | xlsx | Microsoft Excel 2007+ document |\n| 209 | xml | XML document |\n| 210 | xpi | Compressed installation archive (XPI) |\n| 211 | xz | XZ compressed data |\n| 212 | yaml | YAML source |\n| 213 | yara | YARA rule |\n| 214 | zig | Zig source |\n| 215 | zip | Zip archive data |\n| 216 | zlibstream | zlib compressed data |\n\n\n## List of possible model's outputs\n\nThis is the full list of all possible model's output. E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.dl.label`. Note that, in general, the list of \"model outputs\" is different than the \"tool outputs\" as in some cases the model is not even used, or the model's output is overwritten due to a low-confidence score or other reasons. This list is useful mostly for debugging purposes; the vast majority of client should just consult the table above.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | pytorch | Pytorch storage file |\n| 147 | qt | QuickTime |\n| 148 | r | R (language) |\n| 149 | randombytes | Random bytes |\n| 150 | randomtxt | Random text |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | tar | POSIX tar archive |\n| 177 | tcl | Tickle |\n| 178 | textproto | Text protocol buffer |\n| 179 | tga | Targa image data |\n| 180 | thumbsdb | Windows thumbnail cache |\n| 181 | tiff | TIFF image data |\n| 182 | toml | Tom's obvious, minimal language |\n| 183 | torrent | BitTorrent file |\n| 184 | tsv | TSV document |\n| 185 | ttf | TrueType Font data |\n| 186 | twig | Twig template |\n| 187 | txt | Generic text document |\n| 188 | typescript | Typescript |\n| 189 | undefined | Undefined |\n| 190 | vba | MS Visual Basic source (VBA) |\n| 191 | vcxproj | Visual Studio MSBuild project |\n| 192 | verilog | Verilog source |\n| 193 | vhdl | VHDL source |\n| 194 | vtt | Web Video Text Tracks |\n| 195 | vue | Vue source |\n| 196 | wasm | Web Assembly |\n| 197 | wav | Waveform Audio file (WAV) |\n| 198 | webm | WebM media file |\n| 199 | webp | WebP media file |\n| 200 | winregistry | Windows Registry text |\n| 201 | wmf | Windows metafile |\n| 202 | woff | Web Open Font Format |\n| 203 | woff2 | Web Open Font Format v2 |\n| 204 | xar | XAR archive compressed data |\n| 205 | xls | Microsoft Excel CDF document |\n| 206 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 207 | xlsx | Microsoft Excel 2007+ document |\n| 208 | xml | XML document |\n| 209 | xpi | Compressed installation archive (XPI) |\n| 210 | xz | XZ compressed data |\n| 211 | yaml | YAML source |\n| 212 | yara | YARA rule |\n| 213 | zig | Zig source |\n| 214 | zip | Zip archive data |\n| 215 | zlibstream | zlib compressed data |>" }, { "path": "assets/models/standard_v3_1/config.min.json", "content": "{\"beg_size\":1024,\"mid_size\":0,\"end_size\":1024,\"use_inputs_at_offsets\":false,\"medium_confidence_threshold\":0.5,\"min_file_size_for_dl\":8,\"padding_token\":256,\"block_size\":4096,\"target_labels_space\":[\"3gp\",\"ace\",\"ai\",\"aidl\",\"apk\",\"applebplist\",\"appleplist\",\"asm\",\"asp\",\"autohotkey\",\"autoit\",\"awk\",\"batch\",\"bazel\",\"bib\",\"bmp\",\"bzip\",\"c\",\"cab\",\"cat\",\"chm\",\"clojure\",\"cmake\",\"cobol\",\"coff\",\"coffeescript\",\"cpp\",\"crt\",\"crx\",\"cs\",\"csproj\",\"css\",\"csv\",\"dart\",\"deb\",\"dex\",\"dicom\",\"diff\",\"dm\",\"dmg\",\"doc\",\"dockerfile\",\"docx\",\"dsstore\",\"dwg\",\"dxf\",\"elf\",\"elixir\",\"emf\",\"eml\",\"epub\",\"erb\",\"erlang\",\"flac\",\"flv\",\"fortran\",\"gemfile\",\"gemspec\",\"gif\",\"gitattributes\",\"gitmodules\",\"go\",\"gradle\",\"groovy\",\"gzip\",\"h5\",\"handlebars\",\"haskell\",\"hcl\",\"hlp\",\"htaccess\",\"html\",\"icns\",\"ico\",\"ics\",\"ignorefile\",\"ini\",\"internetshortcut\",\"ipynb\",\"iso\",\"jar\",\"java\",\"javabytecode\",\"javascript\",\"jinja\",\"jp2\",\"jpeg\",\"json\",\"jsonl\",\"julia\",\"kotlin\",\"latex\",\"lha\",\"lisp\",\"lnk\",\"lua\",\"m3u\",\"m4\",\"macho\",\"makefile\",\"markdown\",\"matlab\",\"mht\",\"midi\",\"mkv\",\"mp3\",\"mp4\",\"mscompress\",\"msi\",\"mum\",\"npy\",\"npz\",\"nupkg\",\"objectivec\",\"ocaml\",\"odp\",\"ods\",\"odt\",\"ogg\",\"one\",\"onnx\",\"otf\",\"outlook\",\"parquet\",\"pascal\",\"pcap\",\"pdb\",\"pdf\",\"pebin\",\"pem\",\"perl\",\"php\",\"pickle\",\"png\",\"po\",\"postscript\",\"powershell\",\"ppt\",\"pptx\",\"prolog\",\"proteindb\",\"proto\",\"psd\",\"python\",\"pythonbytecode\",\"pytorch\",\"qt\",\"r\",\"randombytes\",\"randomtxt\",\"rar\",\"rdf\",\"rpm\",\"rst\",\"rtf\",\"ruby\",\"rust\",\"scala\",\"scss\",\"sevenzip\",\"sgml\",\"shell\",\"smali\",\"snap\",\"solidity\",\"sql\",\"sqlite\",\"squashfs\",\"srt\",\"stlbinary\",\"stltext\",\"sum\",\"svg\",\"swf\",\"swift\",\"tar\",\"tcl\",\"textproto\",\"tga\",\"thumbsdb\",\"tiff\",\"toml\",\"torrent\",\"tsv\",\"ttf\",\"twig\",\"txt\",\"typescript\",\"vba\",\"vcxproj\",\"verilog\",\"vhdl\",\"vtt\",\"vue\",\"wasm\",\"wav\",\"webm\",\"webp\",\"winregistry\",\"wmf\",\"woff\",\"woff2\",\"xar\",\"xls\",\"xlsb\",\"xlsx\",\"xml\",\"xpi\",\"xz\",\"yaml\",\"yara\",\"zig\",\"zip\",\"zlibstream\"],\"thresholds\":{\"crt\":0.9,\"handlebars\":0.9,\"ignorefile\":0.95,\"latex\":0.95,\"markdown\":0.9,\"ocaml\":0.9,\"pascal\":0.95,\"rst\":0.9,\"sql\":0.9,\"tsv\":0.9},\"overwrite_map\":{\"randombytes\":\"unknown\",\"randomtxt\":\"txt\"},\"protection\":\"none\",\"aes_key_hex\":\"\",\"version_major\":3}\n" }, { "path": "assets/models/standard_v3_1/metadata.json", "content": "{\"epoch_num\":\"200\"}\n" }, { "path": "assets/models/standard_v3_2/README.md", "content": "# Model documentation\n\n## Table of Contents\n\n1. [List of possible outputs](#list-of-possible-outputs)\n1. [List of possible model's outputs](#list-of-possible-models-outputs)\n\n## List of possible outputs\n\nThis is the full list of all possible tool's outputs (which are different than the possible raw output of the model, see table below). E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.output.label`.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | directory | A directory |\n| 40 | dm | Dream Maker |\n| 41 | dmg | Apple disk image |\n| 42 | doc | Microsoft Word CDF document |\n| 43 | dockerfile | Dockerfile |\n| 44 | docx | Microsoft Word 2007+ document |\n| 45 | dsstore | Application Desktop Services Store |\n| 46 | dwg | Autocad Drawing |\n| 47 | dxf | Audocad Drawing Exchange Format |\n| 48 | elf | ELF executable |\n| 49 | elixir | Elixir script |\n| 50 | emf | Windows Enhanced Metafile image data |\n| 51 | eml | RFC 822 mail |\n| 52 | empty | Empty file |\n| 53 | epub | EPUB document |\n| 54 | erb | Embedded Ruby source |\n| 55 | erlang | Erlang source |\n| 56 | flac | FLAC audio bitstream data |\n| 57 | flv | Flash Video |\n| 58 | fortran | Fortran |\n| 59 | gemfile | Gemfile file |\n| 60 | gemspec | Gemspec file |\n| 61 | gif | GIF image data |\n| 62 | gitattributes | Gitattributes file |\n| 63 | gitmodules | Gitmodules file |\n| 64 | go | Golang source |\n| 65 | gradle | Gradle source |\n| 66 | groovy | Groovy source |\n| 67 | gzip | gzip compressed data |\n| 68 | h5 | Hierarchical Data Format v5 |\n| 69 | handlebars | Handlebars source |\n| 70 | haskell | Haskell source |\n| 71 | hcl | HashiCorp configuration language |\n| 72 | hlp | MS Windows help |\n| 73 | htaccess | Apache access configuration |\n| 74 | html | HTML document |\n| 75 | icns | Mac OS X icon |\n| 76 | ico | MS Windows icon resource |\n| 77 | ics | Internet Calendaring and Scheduling |\n| 78 | ignorefile | Ignorefile |\n| 79 | ini | INI configuration file |\n| 80 | internetshortcut | MS Windows Internet shortcut |\n| 81 | ipynb | Jupyter notebook |\n| 82 | iso | ISO 9660 CD-ROM filesystem data |\n| 83 | jar | Java archive data (JAR) |\n| 84 | java | Java source |\n| 85 | javabytecode | Java compiled bytecode |\n| 86 | javascript | JavaScript source |\n| 87 | jinja | Jinja template |\n| 88 | jp2 | jpeg2000 |\n| 89 | jpeg | JPEG image data |\n| 90 | json | JSON document |\n| 91 | jsonl | JSONL document |\n| 92 | julia | Julia source |\n| 93 | kotlin | Kotlin source |\n| 94 | latex | LaTeX document |\n| 95 | lha | LHarc archive |\n| 96 | lisp | Lisp source |\n| 97 | lnk | MS Windows shortcut |\n| 98 | lua | Lua |\n| 99 | m3u | M3U playlist |\n| 100 | m4 | GNU Macro |\n| 101 | macho | Mach-O executable |\n| 102 | makefile | Makefile source |\n| 103 | markdown | Markdown document |\n| 104 | matlab | Matlab Source |\n| 105 | mht | MHTML document |\n| 106 | midi | Midi |\n| 107 | mkv | Matroska |\n| 108 | mp3 | MP3 media file |\n| 109 | mp4 | MP4 media file |\n| 110 | mscompress | MS Compress archive data |\n| 111 | msi | Microsoft Installer file |\n| 112 | mum | Windows Update Package file |\n| 113 | npy | Numpy Array |\n| 114 | npz | Numpy Arrays Archive |\n| 115 | nupkg | NuGet Package |\n| 116 | objectivec | ObjectiveC source |\n| 117 | ocaml | OCaml |\n| 118 | odp | OpenDocument Presentation |\n| 119 | ods | OpenDocument Spreadsheet |\n| 120 | odt | OpenDocument Text |\n| 121 | ogg | Ogg data |\n| 122 | one | One Note |\n| 123 | onnx | Open Neural Network Exchange |\n| 124 | otf | OpenType font |\n| 125 | outlook | MS Outlook Message |\n| 126 | parquet | Apache Parquet |\n| 127 | pascal | Pascal source |\n| 128 | pcap | pcap capture file |\n| 129 | pdb | Windows Program Database |\n| 130 | pdf | PDF document |\n| 131 | pebin | PE Windows executable |\n| 132 | pem | PEM certificate |\n| 133 | perl | Perl source |\n| 134 | php | PHP source |\n| 135 | pickle | Python pickle |\n| 136 | png | PNG image |\n| 137 | po | Portable Object (PO) for i18n |\n| 138 | postscript | PostScript document |\n| 139 | powershell | Powershell source |\n| 140 | ppt | Microsoft PowerPoint CDF document |\n| 141 | pptx | Microsoft PowerPoint 2007+ document |\n| 142 | prolog | Prolog source |\n| 143 | proteindb | Protein DB |\n| 144 | proto | Protocol buffer definition |\n| 145 | psd | Adobe Photoshop |\n| 146 | python | Python source |\n| 147 | pythonbytecode | Python compiled bytecode |\n| 148 | pytorch | Pytorch storage file |\n| 149 | qt | QuickTime |\n| 150 | r | R (language) |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | symlink | Symbolic link |\n| 177 | tar | POSIX tar archive |\n| 178 | tcl | Tickle |\n| 179 | textproto | Text protocol buffer |\n| 180 | tga | Targa image data |\n| 181 | thumbsdb | Windows thumbnail cache |\n| 182 | tiff | TIFF image data |\n| 183 | toml | Tom's obvious, minimal language |\n| 184 | torrent | BitTorrent file |\n| 185 | tsv | TSV document |\n| 186 | ttf | TrueType Font data |\n| 187 | twig | Twig template |\n| 188 | txt | Generic text document |\n| 189 | typescript | TypeScript source |\n| 190 | unknown | Unknown binary data |\n| 191 | vba | MS Visual Basic source (VBA) |\n| 192 | vcxproj | Visual Studio MSBuild project |\n| 193 | verilog | Verilog source |\n| 194 | vhdl | VHDL source |\n| 195 | vtt | Web Video Text Tracks |\n| 196 | vue | Vue source |\n| 197 | wasm | Web Assembly |\n| 198 | wav | Waveform Audio file (WAV) |\n| 199 | webm | WebM media file |\n| 200 | webp | WebP media file |\n| 201 | winregistry | Windows Registry text |\n| 202 | wmf | Windows metafile |\n| 203 | woff | Web Open Font Format |\n| 204 | woff2 | Web Open Font Format v2 |\n| 205 | xar | XAR archive compressed data |\n| 206 | xls | Microsoft Excel CDF document |\n| 207 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 208 | xlsx | Microsoft Excel 2007+ document |\n| 209 | xml | XML document |\n| 210 | xpi | Compressed installation archive (XPI) |\n| 211 | xz | XZ compressed data |\n| 212 | yaml | YAML source |\n| 213 | yara | YARA rule |\n| 214 | zig | Zig source |\n| 215 | zip | Zip archive data |\n| 216 | zlibstream | zlib compressed data |\n\n\n## List of possible model's outputs\n\nThis is the full list of all possible model's output. E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.dl.label`. Note that, in general, the list of \"model outputs\" is different than the \"tool outputs\" as in some cases the model is not even used, or the model's output is overwritten due to a low-confidence score or other reasons. This list is useful mostly for debugging purposes; the vast majority of client should just consult the table above.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | pytorch | Pytorch storage file |\n| 147 | qt | QuickTime |\n| 148 | r | R (language) |\n| 149 | randombytes | Random bytes |\n| 150 | randomtxt | Random text |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | tar | POSIX tar archive |\n| 177 | tcl | Tickle |\n| 178 | textproto | Text protocol buffer |\n| 179 | tga | Targa image data |\n| 180 | thumbsdb | Windows thumbnail cache |\n| 181 | tiff | TIFF image data |\n| 182 | toml | Tom's obvious, minimal language |\n| 183 | torrent | BitTorrent file |\n| 184 | tsv | TSV document |\n| 185 | ttf | TrueType Font data |\n| 186 | twig | Twig template |\n| 187 | txt | Generic text document |\n| 188 | typescript | TypeScript source |\n| 189 | undefined | Undefined |\n| 190 | vba | MS Visual Basic source (VBA) |\n| 191 | vcxproj | Visual Studio MSBuild project |\n| 192 | verilog | Verilog source |\n| 193 | vhdl | VHDL source |\n| 194 | vtt | Web Video Text Tracks |\n| 195 | vue | Vue source |\n| 196 | wasm | Web Assembly |\n| 197 | wav | Waveform Audio file (WAV) |\n| 198 | webm | WebM media file |\n| 199 | webp | WebP media file |\n| 200 | winregistry | Windows Registry text |\n| 201 | wmf | Windows metafile |\n| 202 | woff | Web Open Font Format |\n| 203 | woff2 | Web Open Font Format v2 |\n| 204 | xar | XAR archive compressed data |\n| 205 | xls | Microsoft Excel CDF document |\n| 206 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 207 | xlsx | Microsoft Excel 2007+ document |\n| 208 | xml | XML document |\n| 209 | xpi | Compressed installation archive (XPI) |\n| 210 | xz | XZ compressed data |\n| 211 | yaml | YAML source |\n| 212 | yara | YARA rule |\n| 213 | zig | Zig source |\n| 214 | zip | Zip archive data |\n| 215 | zlibstream | zlib compressed data |>" }, { "path": "assets/models/standard_v3_2/config.min.json", "content": "{\"beg_size\":1024,\"mid_size\":0,\"end_size\":1024,\"use_inputs_at_offsets\":false,\"medium_confidence_threshold\":0.5,\"min_file_size_for_dl\":8,\"padding_token\":256,\"block_size\":4096,\"target_labels_space\":[\"3gp\",\"ace\",\"ai\",\"aidl\",\"apk\",\"applebplist\",\"appleplist\",\"asm\",\"asp\",\"autohotkey\",\"autoit\",\"awk\",\"batch\",\"bazel\",\"bib\",\"bmp\",\"bzip\",\"c\",\"cab\",\"cat\",\"chm\",\"clojure\",\"cmake\",\"cobol\",\"coff\",\"coffeescript\",\"cpp\",\"crt\",\"crx\",\"cs\",\"csproj\",\"css\",\"csv\",\"dart\",\"deb\",\"dex\",\"dicom\",\"diff\",\"dm\",\"dmg\",\"doc\",\"dockerfile\",\"docx\",\"dsstore\",\"dwg\",\"dxf\",\"elf\",\"elixir\",\"emf\",\"eml\",\"epub\",\"erb\",\"erlang\",\"flac\",\"flv\",\"fortran\",\"gemfile\",\"gemspec\",\"gif\",\"gitattributes\",\"gitmodules\",\"go\",\"gradle\",\"groovy\",\"gzip\",\"h5\",\"handlebars\",\"haskell\",\"hcl\",\"hlp\",\"htaccess\",\"html\",\"icns\",\"ico\",\"ics\",\"ignorefile\",\"ini\",\"internetshortcut\",\"ipynb\",\"iso\",\"jar\",\"java\",\"javabytecode\",\"javascript\",\"jinja\",\"jp2\",\"jpeg\",\"json\",\"jsonl\",\"julia\",\"kotlin\",\"latex\",\"lha\",\"lisp\",\"lnk\",\"lua\",\"m3u\",\"m4\",\"macho\",\"makefile\",\"markdown\",\"matlab\",\"mht\",\"midi\",\"mkv\",\"mp3\",\"mp4\",\"mscompress\",\"msi\",\"mum\",\"npy\",\"npz\",\"nupkg\",\"objectivec\",\"ocaml\",\"odp\",\"ods\",\"odt\",\"ogg\",\"one\",\"onnx\",\"otf\",\"outlook\",\"parquet\",\"pascal\",\"pcap\",\"pdb\",\"pdf\",\"pebin\",\"pem\",\"perl\",\"php\",\"pickle\",\"png\",\"po\",\"postscript\",\"powershell\",\"ppt\",\"pptx\",\"prolog\",\"proteindb\",\"proto\",\"psd\",\"python\",\"pythonbytecode\",\"pytorch\",\"qt\",\"r\",\"randombytes\",\"randomtxt\",\"rar\",\"rdf\",\"rpm\",\"rst\",\"rtf\",\"ruby\",\"rust\",\"scala\",\"scss\",\"sevenzip\",\"sgml\",\"shell\",\"smali\",\"snap\",\"solidity\",\"sql\",\"sqlite\",\"squashfs\",\"srt\",\"stlbinary\",\"stltext\",\"sum\",\"svg\",\"swf\",\"swift\",\"tar\",\"tcl\",\"textproto\",\"tga\",\"thumbsdb\",\"tiff\",\"toml\",\"torrent\",\"tsv\",\"ttf\",\"twig\",\"txt\",\"typescript\",\"vba\",\"vcxproj\",\"verilog\",\"vhdl\",\"vtt\",\"vue\",\"wasm\",\"wav\",\"webm\",\"webp\",\"winregistry\",\"wmf\",\"woff\",\"woff2\",\"xar\",\"xls\",\"xlsb\",\"xlsx\",\"xml\",\"xpi\",\"xz\",\"yaml\",\"yara\",\"zig\",\"zip\",\"zlibstream\"],\"thresholds\":{\"crt\":0.9,\"handlebars\":0.9,\"ignorefile\":0.95,\"latex\":0.95,\"markdown\":0.75,\"ocaml\":0.9,\"pascal\":0.95,\"rst\":0.9,\"sql\":0.9,\"tsv\":0.9},\"overwrite_map\":{\"randombytes\":\"unknown\",\"randomtxt\":\"txt\"},\"protection\":\"none\",\"aes_key_hex\":\"\",\"version_major\":3}\n" }, { "path": "assets/models/standard_v3_2/metadata.json", "content": "{\"epoch_num\":\"190\"}\n" }, { "path": "assets/models/standard_v3_3/README.md", "content": "# Model documentation\n\n## Table of Contents\n\n1. [List of possible outputs](#list-of-possible-outputs)\n1. [List of possible model's outputs](#list-of-possible-models-outputs)\n\n## List of possible outputs\n\nThis is the full list of all possible tool's outputs (which are different than the possible raw output of the model, see table below). E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.output.label`.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | directory | A directory |\n| 40 | dm | Dream Maker |\n| 41 | dmg | Apple disk image |\n| 42 | doc | Microsoft Word CDF document |\n| 43 | dockerfile | Dockerfile |\n| 44 | docx | Microsoft Word 2007+ document |\n| 45 | dsstore | Application Desktop Services Store |\n| 46 | dwg | Autocad Drawing |\n| 47 | dxf | Audocad Drawing Exchange Format |\n| 48 | elf | ELF executable |\n| 49 | elixir | Elixir script |\n| 50 | emf | Windows Enhanced Metafile image data |\n| 51 | eml | RFC 822 mail |\n| 52 | empty | Empty file |\n| 53 | epub | EPUB document |\n| 54 | erb | Embedded Ruby source |\n| 55 | erlang | Erlang source |\n| 56 | flac | FLAC audio bitstream data |\n| 57 | flv | Flash Video |\n| 58 | fortran | Fortran |\n| 59 | gemfile | Gemfile file |\n| 60 | gemspec | Gemspec file |\n| 61 | gif | GIF image data |\n| 62 | gitattributes | Gitattributes file |\n| 63 | gitmodules | Gitmodules file |\n| 64 | go | Golang source |\n| 65 | gradle | Gradle source |\n| 66 | groovy | Groovy source |\n| 67 | gzip | gzip compressed data |\n| 68 | h5 | Hierarchical Data Format v5 |\n| 69 | handlebars | Handlebars source |\n| 70 | haskell | Haskell source |\n| 71 | hcl | HashiCorp configuration language |\n| 72 | hlp | MS Windows help |\n| 73 | htaccess | Apache access configuration |\n| 74 | html | HTML document |\n| 75 | icns | Mac OS X icon |\n| 76 | ico | MS Windows icon resource |\n| 77 | ics | Internet Calendaring and Scheduling |\n| 78 | ignorefile | Ignorefile |\n| 79 | ini | INI configuration file |\n| 80 | internetshortcut | MS Windows Internet shortcut |\n| 81 | ipynb | Jupyter notebook |\n| 82 | iso | ISO 9660 CD-ROM filesystem data |\n| 83 | jar | Java archive data (JAR) |\n| 84 | java | Java source |\n| 85 | javabytecode | Java compiled bytecode |\n| 86 | javascript | JavaScript source |\n| 87 | jinja | Jinja template |\n| 88 | jp2 | jpeg2000 |\n| 89 | jpeg | JPEG image data |\n| 90 | json | JSON document |\n| 91 | jsonl | JSONL document |\n| 92 | julia | Julia source |\n| 93 | kotlin | Kotlin source |\n| 94 | latex | LaTeX document |\n| 95 | lha | LHarc archive |\n| 96 | lisp | Lisp source |\n| 97 | lnk | MS Windows shortcut |\n| 98 | lua | Lua |\n| 99 | m3u | M3U playlist |\n| 100 | m4 | GNU Macro |\n| 101 | macho | Mach-O executable |\n| 102 | makefile | Makefile source |\n| 103 | markdown | Markdown document |\n| 104 | matlab | Matlab Source |\n| 105 | mht | MHTML document |\n| 106 | midi | Midi |\n| 107 | mkv | Matroska |\n| 108 | mp3 | MP3 media file |\n| 109 | mp4 | MP4 media file |\n| 110 | mscompress | MS Compress archive data |\n| 111 | msi | Microsoft Installer file |\n| 112 | mum | Windows Update Package file |\n| 113 | npy | Numpy Array |\n| 114 | npz | Numpy Arrays Archive |\n| 115 | nupkg | NuGet Package |\n| 116 | objectivec | ObjectiveC source |\n| 117 | ocaml | OCaml |\n| 118 | odp | OpenDocument Presentation |\n| 119 | ods | OpenDocument Spreadsheet |\n| 120 | odt | OpenDocument Text |\n| 121 | ogg | Ogg data |\n| 122 | one | One Note |\n| 123 | onnx | Open Neural Network Exchange |\n| 124 | otf | OpenType font |\n| 125 | outlook | MS Outlook Message |\n| 126 | parquet | Apache Parquet |\n| 127 | pascal | Pascal source |\n| 128 | pcap | pcap capture file |\n| 129 | pdb | Windows Program Database |\n| 130 | pdf | PDF document |\n| 131 | pebin | PE Windows executable |\n| 132 | pem | PEM certificate |\n| 133 | perl | Perl source |\n| 134 | php | PHP source |\n| 135 | pickle | Python pickle |\n| 136 | png | PNG image |\n| 137 | po | Portable Object (PO) for i18n |\n| 138 | postscript | PostScript document |\n| 139 | powershell | Powershell source |\n| 140 | ppt | Microsoft PowerPoint CDF document |\n| 141 | pptx | Microsoft PowerPoint 2007+ document |\n| 142 | prolog | Prolog source |\n| 143 | proteindb | Protein DB |\n| 144 | proto | Protocol buffer definition |\n| 145 | psd | Adobe Photoshop |\n| 146 | python | Python source |\n| 147 | pythonbytecode | Python compiled bytecode |\n| 148 | pytorch | Pytorch storage file |\n| 149 | qt | QuickTime |\n| 150 | r | R (language) |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | symlink | Symbolic link |\n| 177 | tar | POSIX tar archive |\n| 178 | tcl | Tickle |\n| 179 | textproto | Text protocol buffer |\n| 180 | tga | Targa image data |\n| 181 | thumbsdb | Windows thumbnail cache |\n| 182 | tiff | TIFF image data |\n| 183 | toml | Tom's obvious, minimal language |\n| 184 | torrent | BitTorrent file |\n| 185 | tsv | TSV document |\n| 186 | ttf | TrueType Font data |\n| 187 | twig | Twig template |\n| 188 | txt | Generic text document |\n| 189 | typescript | TypeScript source |\n| 190 | unknown | Unknown binary data |\n| 191 | vba | MS Visual Basic source (VBA) |\n| 192 | vcxproj | Visual Studio MSBuild project |\n| 193 | verilog | Verilog source |\n| 194 | vhdl | VHDL source |\n| 195 | vtt | Web Video Text Tracks |\n| 196 | vue | Vue source |\n| 197 | wasm | Web Assembly |\n| 198 | wav | Waveform Audio file (WAV) |\n| 199 | webm | WebM media file |\n| 200 | webp | WebP media file |\n| 201 | winregistry | Windows Registry text |\n| 202 | wmf | Windows metafile |\n| 203 | woff | Web Open Font Format |\n| 204 | woff2 | Web Open Font Format v2 |\n| 205 | xar | XAR archive compressed data |\n| 206 | xls | Microsoft Excel CDF document |\n| 207 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 208 | xlsx | Microsoft Excel 2007+ document |\n| 209 | xml | XML document |\n| 210 | xpi | Compressed installation archive (XPI) |\n| 211 | xz | XZ compressed data |\n| 212 | yaml | YAML source |\n| 213 | yara | YARA rule |\n| 214 | zig | Zig source |\n| 215 | zip | Zip archive data |\n| 216 | zlibstream | zlib compressed data |\n\n\n## List of possible model's outputs\n\nThis is the full list of all possible model's output. E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.dl.label`. Note that, in general, the list of \"model outputs\" is different than the \"tool outputs\" as in some cases the model is not even used, or the model's output is overwritten due to a low-confidence score or other reasons. This list is useful mostly for debugging purposes; the vast majority of client should just consult the table above.\n\n| Index | Content Type Label | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | pytorch | Pytorch storage file |\n| 147 | qt | QuickTime |\n| 148 | r | R (language) |\n| 149 | randombytes | Random bytes |\n| 150 | randomtxt | Random text |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | tar | POSIX tar archive |\n| 177 | tcl | Tickle |\n| 178 | textproto | Text protocol buffer |\n| 179 | tga | Targa image data |\n| 180 | thumbsdb | Windows thumbnail cache |\n| 181 | tiff | TIFF image data |\n| 182 | toml | Tom's obvious, minimal language |\n| 183 | torrent | BitTorrent file |\n| 184 | tsv | TSV document |\n| 185 | ttf | TrueType Font data |\n| 186 | twig | Twig template |\n| 187 | txt | Generic text document |\n| 188 | typescript | TypeScript source |\n| 189 | undefined | Undefined |\n| 190 | vba | MS Visual Basic source (VBA) |\n| 191 | vcxproj | Visual Studio MSBuild project |\n| 192 | verilog | Verilog source |\n| 193 | vhdl | VHDL source |\n| 194 | vtt | Web Video Text Tracks |\n| 195 | vue | Vue source |\n| 196 | wasm | Web Assembly |\n| 197 | wav | Waveform Audio file (WAV) |\n| 198 | webm | WebM media file |\n| 199 | webp | WebP media file |\n| 200 | winregistry | Windows Registry text |\n| 201 | wmf | Windows metafile |\n| 202 | woff | Web Open Font Format |\n| 203 | woff2 | Web Open Font Format v2 |\n| 204 | xar | XAR archive compressed data |\n| 205 | xls | Microsoft Excel CDF document |\n| 206 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 207 | xlsx | Microsoft Excel 2007+ document |\n| 208 | xml | XML document |\n| 209 | xpi | Compressed installation archive (XPI) |\n| 210 | xz | XZ compressed data |\n| 211 | yaml | YAML source |\n| 212 | yara | YARA rule |\n| 213 | zig | Zig source |\n| 214 | zip | Zip archive data |\n| 215 | zlibstream | zlib compressed data |>" }, { "path": "assets/models/standard_v3_3/config.min.json", "content": "{\"beg_size\":1024,\"mid_size\":0,\"end_size\":1024,\"use_inputs_at_offsets\":false,\"medium_confidence_threshold\":0.5,\"min_file_size_for_dl\":8,\"padding_token\":256,\"block_size\":4096,\"target_labels_space\":[\"3gp\",\"ace\",\"ai\",\"aidl\",\"apk\",\"applebplist\",\"appleplist\",\"asm\",\"asp\",\"autohotkey\",\"autoit\",\"awk\",\"batch\",\"bazel\",\"bib\",\"bmp\",\"bzip\",\"c\",\"cab\",\"cat\",\"chm\",\"clojure\",\"cmake\",\"cobol\",\"coff\",\"coffeescript\",\"cpp\",\"crt\",\"crx\",\"cs\",\"csproj\",\"css\",\"csv\",\"dart\",\"deb\",\"dex\",\"dicom\",\"diff\",\"dm\",\"dmg\",\"doc\",\"dockerfile\",\"docx\",\"dsstore\",\"dwg\",\"dxf\",\"elf\",\"elixir\",\"emf\",\"eml\",\"epub\",\"erb\",\"erlang\",\"flac\",\"flv\",\"fortran\",\"gemfile\",\"gemspec\",\"gif\",\"gitattributes\",\"gitmodules\",\"go\",\"gradle\",\"groovy\",\"gzip\",\"h5\",\"handlebars\",\"haskell\",\"hcl\",\"hlp\",\"htaccess\",\"html\",\"icns\",\"ico\",\"ics\",\"ignorefile\",\"ini\",\"internetshortcut\",\"ipynb\",\"iso\",\"jar\",\"java\",\"javabytecode\",\"javascript\",\"jinja\",\"jp2\",\"jpeg\",\"json\",\"jsonl\",\"julia\",\"kotlin\",\"latex\",\"lha\",\"lisp\",\"lnk\",\"lua\",\"m3u\",\"m4\",\"macho\",\"makefile\",\"markdown\",\"matlab\",\"mht\",\"midi\",\"mkv\",\"mp3\",\"mp4\",\"mscompress\",\"msi\",\"mum\",\"npy\",\"npz\",\"nupkg\",\"objectivec\",\"ocaml\",\"odp\",\"ods\",\"odt\",\"ogg\",\"one\",\"onnx\",\"otf\",\"outlook\",\"parquet\",\"pascal\",\"pcap\",\"pdb\",\"pdf\",\"pebin\",\"pem\",\"perl\",\"php\",\"pickle\",\"png\",\"po\",\"postscript\",\"powershell\",\"ppt\",\"pptx\",\"prolog\",\"proteindb\",\"proto\",\"psd\",\"python\",\"pythonbytecode\",\"pytorch\",\"qt\",\"r\",\"randombytes\",\"randomtxt\",\"rar\",\"rdf\",\"rpm\",\"rst\",\"rtf\",\"ruby\",\"rust\",\"scala\",\"scss\",\"sevenzip\",\"sgml\",\"shell\",\"smali\",\"snap\",\"solidity\",\"sql\",\"sqlite\",\"squashfs\",\"srt\",\"stlbinary\",\"stltext\",\"sum\",\"svg\",\"swf\",\"swift\",\"tar\",\"tcl\",\"textproto\",\"tga\",\"thumbsdb\",\"tiff\",\"toml\",\"torrent\",\"tsv\",\"ttf\",\"twig\",\"txt\",\"typescript\",\"vba\",\"vcxproj\",\"verilog\",\"vhdl\",\"vtt\",\"vue\",\"wasm\",\"wav\",\"webm\",\"webp\",\"winregistry\",\"wmf\",\"woff\",\"woff2\",\"xar\",\"xls\",\"xlsb\",\"xlsx\",\"xml\",\"xpi\",\"xz\",\"yaml\",\"yara\",\"zig\",\"zip\",\"zlibstream\"],\"thresholds\":{\"crt\":0.9,\"handlebars\":0.9,\"ignorefile\":0.95,\"latex\":0.95,\"markdown\":0.75,\"ocaml\":0.9,\"pascal\":0.95,\"r\":0.9,\"rst\":0.9,\"sql\":0.9,\"tsv\":0.9,\"zig\":0.9},\"overwrite_map\":{\"randombytes\":\"unknown\",\"randomtxt\":\"txt\"},\"protection\":\"none\",\"aes_key_hex\":\"\",\"version_major\":3}\n" }, { "path": "assets/models/standard_v3_3/metadata.json", "content": "{\"epoch_num\":\"91\"}\n" }, { "path": "dist-workspace.toml", "content": "[workspace]\nmembers = [\"cargo:rust/cli\"]\n\n# Config for 'dist'\n[dist]\n# Skip checking whether the specified configuration files are up to date\nallow-dirty = [\"ci\"]\n# The preferred dist version to use in CI (Cargo.toml SemVer syntax)\ncargo-dist-version = \"0.31.0\"\n# CI backends to support\nci = \"github\"\n# The installers to generate for each app\ninstallers = [\"shell\", \"powershell\"]\n# Target platforms to build apps for (Rust target-triple syntax)\ntargets = [\"aarch64-apple-darwin\", \"aarch64-unknown-linux-gnu\", \"x86_64-unknown-linux-gnu\", \"x86_64-pc-windows-msvc\"]\n# Path that installers should place binaries in\ninstall-path = \"CARGO_HOME\"\n# Whether to install an updater program\ninstall-updater = true\n# Generate and dist a source tarball\nsource-tarball = false\n# Whether to auto-include files like READMEs, LICENSEs, and CHANGELOGs (default true)\nauto-includes = false\n# A prefix git tags must include for dist to care about them\ntag-namespace = \"cli\"\n# Whether +crt-static should be used on msvc\nmsvc-crt-static = false\n# Whether to enable GitHub Attestations\ngithub-attestations = true\n# Post-announce jobs to run in CI\npost-announce-jobs = [\"./cli-latest\"]\n\n[dist.github-custom-runners]\nglobal = \"ubuntu-latest\"\naarch64-unknown-linux-gnu = \"ubuntu-24.04-arm\"\nx86_64-unknown-linux-gnu = \"ubuntu-latest\"\naarch64-apple-darwin = \"macos-latest\"\nx86_64-pc-windows-msvc = \"windows-latest\"\n" }, { "path": "docs/concepts.md", "content": "# Magika Concepts\n\nThe documentation has moved, see the [Core Concepts](https://securityresearch.google/magika/core-concepts/how-magika-works/) section of the new website.\n" }, { "path": "docs/js.md", "content": "# JavaScript Documentation\n\nThe documentation has moved, see the [JavaScript bindings](https://securityresearch.google/magika/cli-and-bindings/js/) section of the new website.\n" }, { "path": "go/README.md", "content": "# Go library\n\nThis directory contains the Go library for Magika.\n\nThe inference relies on the [ONNX Runtime](https://onnxruntime.ai/), and it\nrequires [cgo](https://go.dev/blog/cgo) for interfacing with the ONNX Runtime\n[C API](https://onnxruntime.ai/docs/api/c/).\n\n## Usage\nAs illustrated in [`example/main.go`](./example/main.go), calling magika from go boils down\nto creating a scanner associated with a given model, and scanning the content.\n\n```golang\n//go:build cgo && onnxruntime\n\n// This package illustrates the usage of the Magika go binding.\n//\n// It requires the onnxruntime and the Magika assets to be accessible.\n// onnxruntime is available on https://github.com/microsoft/onnxruntime/releases\n// Magika asserts are available on https://github.com/google/magika/tree/main/assets\n//\n// Tag and link directives must be provided a build or run time:\n// go run -tags onnxruntime -ldflags=\"-linkmode=external -extldflags=-L/opt/onnxruntime/lib\" .\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/google/magika/go/magika\"\n)\n\nconst (\n\t// assetsDir holds where the Magika assets have been installed.\n\tassetsDir = \"/opt/magika/assets\"\n\t// modelName holds the Magika model to use.\n\tmodelName = \"standard_v3_3\"\n)\n\nfunc main() {\n\t// Create a scanner.\n\ts, err := magika.NewScanner(assetsDir, modelName)\n\tif err != nil {\n\t\tlog.Fatalf(\"NewScanner failed: %v\", err)\n\t}\n\t// Scan\n\tct, err := s.Scan(strings.NewReader(\"Hello world\"), 11)\n\tif err != nil {\n\t\tlog.Fatalf(\"Scan failed: %v\", err)\n\t}\n\tfmt.Printf(\"%+v\\n\", ct)\n}\n```\n\nInspiration on how to download and install onnxruntime and magika assets can be found in [`docker/Dockerfile`](docker/Dockerfile),\nand [`cli/cli.go`](cli/cli.go) provides a somewhat more elaborate usage of the go binding.\n\n## Content\n- [`docker`](./docker) contains a sample docker file that builds a\ncontainer image that ties together a Magika CLI, an ONNX Runtime,\nand a [model](../assets/models/standard_v3_3).\n- [`cli`](./cli) contains a basic CLI that illustrates how \nthe Magika go library may be called from within an application.\n- [`magika`](./magika) contains the library, that extracts\nfeatures from a sequence of bytes, and provides a scanner to infer\ncontent types.\n- [`onnx`](./onnx) wraps the C API of the ONNX Runtime to\nprovide an inference engine.\n- [`example`](./example) contains a rudimentary example for creating\nand using a content type scanner.\n" }, { "path": "go/cli/cli.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/google/magika/go/magika\"\n)\n\nconst (\n\tassetsDirEnv = \"MAGIKA_ASSETS_DIR\"\n\tmodelNameEnv = \"MAGIKA_MODEL\"\n)\n\n// cli is a basic CLI that infers the content type of the files listed on\n// the command line. The assets dir and the model name are given via the\n// environment variable MAGIKA_ASSETS_DIR and MAGIKA_MODEL respectively.\nfunc cli(w io.Writer, args ...string) error {\n\tassetsDir := os.Getenv(assetsDirEnv)\n\tif assetsDir == \"\" {\n\t\treturn fmt.Errorf(\"%s environment variable not set or empty\", assetsDirEnv)\n\t}\n\tmodelName := os.Getenv(modelNameEnv)\n\tif modelName == \"\" {\n\t\treturn fmt.Errorf(\"%s environment variable not set or empty\", modelNameEnv)\n\t}\n\ts, err := magika.NewScanner(assetsDir, modelName)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create scanner: %w\", err)\n\t}\n\n\t// For each filename given as argument, read the file and scan its content.\n\tfor _, a := range args {\n\t\tfmt.Fprintf(w, \"%s: \", a)\n\t\tb, err := os.ReadFile(a)\n\t\tif err != nil {\n\t\t\tfmt.Fprintf(w, \"%v\\n\", err)\n\t\t\tcontinue\n\t\t}\n\t\tct, err := s.Scan(bytes.NewReader(b), len(b))\n\t\tif err != nil {\n\t\t\tfmt.Fprintf(w, \"scan: %v\\n\", err)\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Fprintf(w, \"%s\\n\", ct.Label)\n\t}\n\treturn nil\n}\n" }, { "path": "go/cli/cli_test.go", "content": "//go:build cgo && onnxruntime\n\npackage main\n\nimport (\n\t\"path\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/google/go-cmp/cmp\"\n)\n\nfunc TestCLI(t *testing.T) {\n\tconst basicDir = \"../../tests_data/basic\"\n\tvar (\n\t\tfiles = []string{\n\t\t\tpath.Join(basicDir, \"python/code.py\"),\n\t\t\tpath.Join(basicDir, \"zip/magika_test.zip\"),\n\t\t}\n\t\tb strings.Builder\n\t)\n\tif err := cli(&b, files...); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif d := cmp.Diff(strings.Join([]string{\n\t\t\"../../tests_data/basic/python/code.py: python\",\n\t\t\"../../tests_data/basic/zip/magika_test.zip: zip\",\n\t}, \"\\n\"), strings.TrimSpace(b.String())); d != \"\" {\n\t\tt.Errorf(\"mismatch (-want +got):\\n%s\", d)\n\t}\n}\n" }, { "path": "go/cli/main.go", "content": "/*\nCLI is a simple command line interface for magika.\n\nIt takes a list of files as argument, and infers their types in sequence.\nFor example:\n\n\t$ magika test.go readme.md\n\ttest.go: go\n\treadme.md: markdown\n\nThe primary intent is to illustrate how the magika go library can be used\nand compiled, using cgo and the ONNX Runtime library.\n*/\npackage main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n)\n\nfunc main() {\n\tif err := cli(os.Stdout, os.Args[1:]...); err != nil {\n\t\tfmt.Printf(\"Error: %v\\n\", err)\n\t\tos.Exit(1)\n\t}\n}\n" }, { "path": "go/cli/tests_data/magika_test_pptx.txt", "content": "This is a test for Magika!\n\nVery cool if this can be detected correctly!\n" }, { "path": "go/docker/Dockerfile", "content": "# Sample Dockerfile to build an image that ties together an ONNX Runtime,\n# a Magika model, and a Magika CLI.\n#\n# It expects the root of the repository as build context:\n# $ docker build -f go/docker/Dockerfile -t magika-go:latest .\n#\n# Then, to list the content type of the files in the current directory:\n# docker run --rm --name magika-go -v $PWD:$PWD:ro -w $PWD magika-go:latest *\n\n# Build stage for ONNX Runtime and magika.\nFROM golang:latest AS build\n\n# Work in a clean temp directory.\nWORKDIR /tmp\n\n# Download, check, and install ONNX Runtime (https://onnxruntime.ai/) in\n# /opt/onnxruntime.\n# Releases are located at https://github.com/microsoft/onnxruntime/releases.\n# We need the SDK (/include) for compiling, and the library (/lib) for inference.\nARG ONNX_NAME=onnxruntime\nARG ONNX_ARCH=linux-x64\nARG ONNX_VERSION=1.19.2\nARG ONNX_FULLNAME=${ONNX_NAME}-${ONNX_ARCH}-${ONNX_VERSION}\nARG ONNX_TARBALL=${ONNX_FULLNAME}.tgz\nARG ONNX_DIGEST=eb00c64e0041f719913c4080e0fed7d9963dc3aa9b54664df6036d8308dbcd33\n\nRUN curl -sL -O https://github.com/microsoft/${ONNX_NAME}/releases/download/v${ONNX_VERSION}/${ONNX_TARBALL} \\\n && echo \"${ONNX_DIGEST} ${ONNX_TARBALL}\" > checksum.txt \\\n && sha256sum -c checksum.txt \\\n && tar -xzf ${ONNX_TARBALL} -C /opt \\\n && ln -s /opt/${ONNX_FULLNAME} /opt/onnxruntime\n\n# Retrieve the magika go code from the build context, test, and build the cli.\nCOPY go go/\nCOPY tests_data tests_data/\nCOPY assets/content_types_kb.min.json assets/content_types_kb.min.json\nCOPY assets/models/standard_v3_3 assets/models/standard_v3_3/\n\nARG CGO_ENABLED=1\nARG CGO_CFLAGS=-I/opt/onnxruntime/include\nARG LD_LIBRARY_PATH=/opt/onnxruntime/lib\n\n# Run the tests.\nWORKDIR go\nRUN MAGIKA_ASSETS_DIR=../../assets \\\n MAGIKA_MODEL=standard_v3_3 \\\n go test -v -tags onnxruntime -ldflags=\"-linkmode=external -extldflags=-L/opt/onnxruntime/lib\" ./...\n\n# Build the CLI.\nWORKDIR cli\nRUN go build -tags onnxruntime -ldflags=\"-linkmode=external -extldflags=-L/opt/onnxruntime/lib\" .\n\n\n# Final stage: copy resources from the build and set environment variables.\nFROM debian:latest\n\n# Add the ONNX Runtime.\nENV LD_LIBRARY_PATH=/opt/onnxruntime/lib\nCOPY --from=build /opt/onnxruntime/lib ${LD_LIBRARY_PATH}\n\n# Magika model.\nENV MAGIKA_ASSETS_DIR=/opt/magika/assets\nENV MAGIKA_MODEL=standard_v3_3\nCOPY assets/models/${MAGIKA_MODEL} ${MAGIKA_ASSETS_DIR}/models/${MAGIKA_MODEL}/\nCOPY assets/content_types_kb.min.json ${MAGIKA_ASSETS_DIR}/content_types_kb.min.json\n\n# Magika CLI.\nCOPY --from=build /tmp/go/cli/cli /usr/local/bin/magika\nENTRYPOINT [\"magika\"]\n" }, { "path": "go/example/main.go", "content": "//go:build cgo && onnxruntime\n\n// This package illustrates the usage of the Magika go binding.\n//\n// It requires the onnxruntime and the Magika assets to be accessible.\n// onnxruntime is available on https://github.com/microsoft/onnxruntime/releases\n// Magika asserts are available on https://github.com/google/magika/tree/main/assets\n//\n// Tag and link directives must be provided a build or run time:\n// go run -tags onnxruntime -ldflags=\"-linkmode=external -extldflags=-L/opt/onnxruntime/lib\" .\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"log\"\n\t\"strings\"\n\n\t\"github.com/google/magika/go/magika\"\n)\n\nconst (\n\t// assetsDir holds where the Magika assets have been installed.\n\tassetsDir = \"/opt/magika/assets\"\n\t// modelName holds the Magika model to use.\n\tmodelName = \"standard_v3_3\"\n)\n\nfunc main() {\n\t// Create a scanner.\n\ts, err := magika.NewScanner(assetsDir, modelName)\n\tif err != nil {\n\t\tlog.Fatalf(\"NewScanner failed: %v\", err)\n\t}\n\t// Scan\n\tct, err := s.Scan(strings.NewReader(\"Hello world\"), 11)\n\tif err != nil {\n\t\tlog.Fatalf(\"Scan failed: %v\", err)\n\t}\n\tfmt.Printf(\"%+v\\n\", ct)\n}\n" }, { "path": "go/go.mod", "content": "module github.com/google/magika/go\n\ngo 1.22.3\n\nrequire github.com/google/go-cmp v0.6.0 // indirect\n" }, { "path": "go/go.sum", "content": "github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=\ngithub.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=\n" }, { "path": "go/magika/config.go", "content": "package magika\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path\"\n)\n\nconst (\n\tconfigFile = \"config.min.json\"\n\tcontentTypesKBFile = \"content_types_kb.min.json\"\n\tmodelFile = \"model.onnx\"\n\tmodelsDir = \"models\"\n)\n\n// Config holds the portion of Magika's model configuration that is relevant\n// for inference.\ntype Config struct {\n\tBegSize int `json:\"beg_size\"`\n\tMidSize int `json:\"mid_size\"`\n\tEndSize int `json:\"end_size\"`\n\tUseInputsAtOffsets bool `json:\"use_inputs_at_offsets\"`\n\tMediumConfidenceThreshold float32 `json:\"medium_confidence_threshold\"`\n\tMinFileSizeForDl int64 `json:\"min_file_size_for_dl\"`\n\tPaddingToken int `json:\"padding_token\"`\n\tBlockSize int `json:\"block_size\"`\n\tTargetLabelsSpace []string `json:\"target_labels_space\"`\n\tThresholds map[string]float32 `json:\"thresholds\"`\n\tOverwrite map[string]string `json:\"overwrite_map\"`\n}\n\n// ReadConfig is a helper that reads and unmarshal a Config, given an assets\n// dir and a model name.\nfunc ReadConfig(assetsDir, name string) (Config, error) {\n\tvar cfg Config\n\tp := configPath(assetsDir, name)\n\tb, err := os.ReadFile(p)\n\tif err != nil {\n\t\treturn Config{}, fmt.Errorf(\"read %q: %w\", p, err)\n\t}\n\tif err := json.Unmarshal(b, &cfg); err != nil {\n\t\treturn Config{}, fmt.Errorf(\"unmarshal: %w\", err)\n\t}\n\treturn cfg, nil\n}\n\n// contentTypesKBPath returns the content types KB path for the given\n// asset folder.\nfunc contentTypesKBPath(assetDir string) string {\n\treturn path.Join(assetDir, contentTypesKBFile)\n}\n\n// configPath returns the model config for the given asset folder and model\n// name.\nfunc configPath(assetDir, name string) string {\n\treturn path.Join(assetDir, modelsDir, name, configFile)\n}\n\n// modelPath returns the Onnx model for the given asset folder and model name.\nfunc modelPath(assetDir, name string) string {\n\treturn path.Join(assetDir, modelsDir, name, modelFile)\n}\n" }, { "path": "go/magika/content.go", "content": "package magika\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n)\n\nconst (\n\tcontentTypeLabelEmpty = \"empty\"\n\tcontentTypeLabelTxt = \"txt\"\n\tcontentTypeLabelUnknown = \"unknown\"\n)\n\n// ContentType holds the definition of a content type.\ntype ContentType struct {\n\tLabel string // As keyed in the content types KB.\n\tMimeType string `json:\"mime_type\"`\n\tGroup string `json:\"group\"`\n\tDescription string `json:\"description\"`\n\tExtensions []string `json:\"extensions\"`\n\tIsText bool `json:\"is_text\"`\n}\n\n// readContentTypesKB is a helper that reads and unmarshal a content types KB,\n// given the assets dir.\n// It returns a dictionary that maps a label as defined in the model config\n// target label space to a content type.\nfunc readContentTypesKB(assetsDir string) (map[string]ContentType, error) {\n\tvar ckb map[string]ContentType\n\tp := contentTypesKBPath(assetsDir)\n\tb, err := os.ReadFile(p)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read %q: %w\", p, err)\n\t}\n\tif err := json.Unmarshal(b, &ckb); err != nil {\n\t\treturn nil, fmt.Errorf(\"unmarshal: %w\", err)\n\t}\n\tfor label, ct := range ckb {\n\t\tct.Label = label\n\t\tckb[label] = ct\n\t}\n\treturn ckb, nil\n}\n" }, { "path": "go/magika/features.go", "content": "package magika\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n)\n\n// Features holds the features of a give slice of bytes.\ntype Features struct {\n\tfirstBlock []byte\n\tBeg []int32 `json:\"beg\"`\n\tMid []int32 `json:\"mid\"`\n\tEnd []int32 `json:\"end\"`\n\tOffset8000 []int32 `json:\"offset_0x8000_0x8007\"`\n\tOffset8800 []int32 `json:\"offset_0x8800_0x8807\"`\n\tOffset9000 []int32 `json:\"offset_0x9000_0x9007\"`\n\tOffset9800 []int32 `json:\"offset_0x9800_0x9807\"`\n}\n\n// ExtractFeatures extract the features from the given reader.\n// The number of bytes that can be read from the reader is given by size.\nfunc ExtractFeatures(cfg Config, r io.ReaderAt, size int) (Features, error) {\n\tvar (\n\t\ter = errReader{r: r, sz: size}\n\t\tbeg = er.readAt(0, cfg.BlockSize)\n\t\tmid = er.readAt((size-cfg.MidSize)/2, cfg.MidSize)\n\t\tend = er.readAt(size-cfg.BlockSize, cfg.BlockSize)\n\t)\n\tf := buildFeatures(cfg, beg, mid, end)\n\n\tpeek := func(off int) []int32 {\n\t\tb := er.readAt(off, 8)\n\t\tif len(b) < 8 {\n\t\t\tb = nil\n\t\t}\n\t\treturn padInt32(cfg, b, 0, 8)\n\t}\n\tf.Offset8000 = peek(0x8000)\n\tf.Offset8800 = peek(0x8800)\n\tf.Offset9000 = peek(0x9000)\n\tf.Offset9800 = peek(0x9800)\n\n\tif er.err != nil {\n\t\treturn Features{}, er.err\n\t}\n\treturn f, nil\n}\n\n// Flatten returns a flattened array of the given features.\nfunc (f Features) Flatten() []int32 {\n\tres := make([]int32, 0, len(f.Beg)+len(f.Mid)+len(f.End))\n\tres = append(res, f.Beg...)\n\tres = append(res, f.Mid...)\n\tres = append(res, f.End...)\n\treturn res\n}\n\n// errReader wraps an io.ReaderAt and accumulates errors that may arise during\n// reading. It also silently protects against out of range read.\n// This allows for a simpler parsing code flow with a unique error check at\n// the end of parsing.\ntype errReader struct {\n\tr io.ReaderAt\n\tsz int\n\terr error\n}\n\nfunc (e *errReader) readAt(off, n int) []byte {\n\tif e.err != nil || off >= e.sz {\n\t\treturn nil\n\t}\n\tif off < 0 {\n\t\tn += off\n\t\toff = 0\n\t}\n\tn = min(n, e.sz-off)\n\tb := make([]byte, n)\n\tp, err := e.r.ReadAt(b, int64(max(off, 0)))\n\tif err != nil && err != io.EOF {\n\t\te.err = fmt.Errorf(\"read %d bytes at %d: %w\", n, max(off, 0), err)\n\t\treturn nil\n\t}\n\treturn b[:p]\n}\n\n// buildFeatures builds features from the beg, mid, and end bytes.\nfunc buildFeatures(cfg Config, beg, mid, end []byte) Features {\n\tfirstBlock := beg\n\n\tspaces := string([]rune{'\\t', '\\n', '\\v', '\\f', '\\r', ' '})\n\t// Trim beg and end, and truncate to BegSize and EndSize.\n\tbeg = bytes.TrimLeft(beg, spaces)\n\tend = bytes.TrimRight(end, spaces)\n\tbeg = safeSlice(beg, 0, cfg.BegSize)\n\tend = safeSlice(end, len(end)-cfg.EndSize, len(end))\n\n\treturn Features{\n\t\tfirstBlock: firstBlock,\n\t\tBeg: padInt32(cfg, beg, 0, cfg.BegSize),\n\t\tMid: padInt32(cfg, mid, (cfg.MidSize-len(mid))/2, cfg.MidSize),\n\t\tEnd: padInt32(cfg, end, cfg.EndSize-len(end), cfg.EndSize),\n\t}\n}\n\n// padInt32 pads and convert the given bytes into int32.\n// The len of the returned is the given size.\n// if prefix is non-zero, that many padding is add as prefix.\n// then the given bytes are converted into int32\n// finally, padding occurs until the returned slice is of the given size.\nfunc padInt32(cfg Config, b []byte, prefix, size int) []int32 {\n\tr := make([]int32, 0, size)\n\tfor len(r) < prefix {\n\t\tr = append(r, int32(cfg.PaddingToken))\n\t}\n\tfor _, bb := range b {\n\t\tr = append(r, int32(bb))\n\t}\n\tfor len(r) < size {\n\t\tr = append(r, int32(cfg.PaddingToken))\n\t}\n\treturn r\n}\n\n// safeSlice returns a slice from the given array, silently clipping\n// out-of-bound indices. This happens when the given input data contains\n// fewer bytes than the sampling size.\nfunc safeSlice(b []byte, from, to int) []byte {\n\treturn b[max(from, 0):min(to, len(b))]\n}\n" }, { "path": "go/magika/features_test.go", "content": "package magika\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n)\n\nfunc TestExtractFeatures(t *testing.T) {\n\tconst artifacts = \"../../tests_data/features_extraction/reference.json.gz\"\n\tb, err := loadArtifacts(t, artifacts)\n\tif err != nil {\n\t\tt.Fatalf(\"load artifacts: %s\", err)\n\t}\n\n\tvar cases []struct {\n\t\tTestInfo Config `json:\"test_info\"`\n\t\tContent []byte `json:\"content\"`\n\t\tFeaturesV2 Features `json:\"features_v2\"`\n\t}\n\tif err := json.Unmarshal(b, &cases); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor i, c := range cases {\n\t\tft, err := ExtractFeatures(c.TestInfo, bytes.NewReader(c.Content), len(c.Content))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif d := cmp.Diff(c.FeaturesV2, ft, cmpopts.IgnoreUnexported(Features{})); d != \"\" {\n\t\t\tt.Errorf(\"Feature [%d]: mismatch (-want +got):\\n%s\", i, d)\n\t\t}\n\t}\n}\n\nfunc TestReferenceExtractFeatures(t *testing.T) {\n\tconst artifacts = \"../../tests_data/reference/features_extraction_examples.json.gz\"\n\tb, err := loadArtifacts(t, artifacts)\n\tif err != nil {\n\t\tt.Fatalf(\"load artifacts: %s\", err)\n\t}\n\n\tvar cases []struct {\n\t\tTestInfo Config `json:\"args\"`\n\t\tContent []byte `json:\"content_base64\"`\n\t\tFeaturesV2 Features `json:\"features\"`\n\t}\n\tif err := json.Unmarshal(b, &cases); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor i, c := range cases {\n\t\tft, err := ExtractFeatures(c.TestInfo, bytes.NewReader(c.Content), len(c.Content))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif d := cmp.Diff(c.FeaturesV2, ft,\n\t\t\tcmpopts.IgnoreUnexported(Features{}),\n\t\t\tcmpopts.IgnoreFields(Features{}, \"Offset8000\", \"Offset8800\", \"Offset9000\", \"Offset9800\"),\n\t\t); d != \"\" {\n\t\t\tt.Errorf(\"Feature [%d]: mismatch (-want +got):\\n%s\", i, d)\n\t\t}\n\t}\n}\n\nfunc loadArtifacts(t *testing.T, path string) ([]byte, error) {\n\tt.Helper()\n\tf, err := os.Open(path)\n\tif err != nil {\n\t\tt.Fatalf(\"Open %s: %v\", path, err)\n\t}\n\tr, err := gzip.NewReader(f)\n\tif err != nil {\n\t\tt.Fatalf(\"could not uncompress test data: %s\", err)\n\t}\n\tb, err := io.ReadAll(r)\n\tif err != nil {\n\t\tt.Fatalf(\"could not read uncompress test data: %s\", err)\n\t}\n\treturn b, nil\n}\n" }, { "path": "go/magika/scanner.go", "content": "package magika\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"unicode/utf8\"\n\n\t\"github.com/google/magika/go/onnx\"\n)\n\n// Scanner represents a Magika scanner that returns the content type\n// of the scanned content running the Magika model using an ONNX Runtime.\n// This is a similar scanner interface to licensecheck, that scans\n// content to identify licenses.\ntype Scanner struct {\n\tonnx onnx.Onnx\n\tcfg Config\n\tckb map[string]ContentType\n}\n\n// NewScanner returns a scanner based on the model of the given name defined\n// in the given the assets dir.\nfunc NewScanner(assetsDir, name string) (*Scanner, error) {\n\tcfg, err := ReadConfig(assetsDir, name)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read config: %w\", err)\n\t}\n\tp := modelPath(assetsDir, name)\n\tob, err := onnx.NewOnnx(p, len(cfg.TargetLabelsSpace))\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"new onnx: %w\", err)\n\t}\n\tif ob == nil {\n\t\treturn nil, errors.New(\"new onnx: nil onnx object\")\n\t}\n\tckb, err := readContentTypesKB(assetsDir)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read content types KB: %w\", err)\n\t}\n\treturn &Scanner{\n\t\tonnx: ob,\n\t\tcfg: cfg,\n\t\tckb: ckb,\n\t}, nil\n}\n\n// Scan scans the given reader containing the given size of bytes, and\n// returns the inferred content type.\n// It is safe for concurrent use.\nfunc (s *Scanner) Scan(r io.ReaderAt, size int) (ContentType, error) {\n\tct, _, err := s.scanScore(r, size)\n\treturn ct, err\n}\n\n// scanScore scans the given reader containing the given size of bytes, and\n// returns the inferred content type and its score.\n// It is safe for concurrent use.\nfunc (s *Scanner) scanScore(r io.ReaderAt, size int) (ContentType, float32, error) {\n\tif size == 0 {\n\t\treturn s.ckb[contentTypeLabelEmpty], 1, nil\n\t}\n\tft, err := ExtractFeatures(s.cfg, r, size)\n\tif err != nil {\n\t\treturn ContentType{}, 0, fmt.Errorf(\"extract features: %w\", err)\n\t}\n\t// Do not use the model for small files.\n\tif ft.Beg[s.cfg.MinFileSizeForDl-1] == int32(s.cfg.PaddingToken) {\n\t\tif utf8.Valid(ft.firstBlock) {\n\t\t\treturn s.ckb[contentTypeLabelTxt], 1, nil\n\t\t} else {\n\t\t\treturn s.ckb[contentTypeLabelUnknown], 1, nil\n\t\t}\n\t}\n\tscores, err := s.onnx.Run(ft.Flatten())\n\tif err != nil {\n\t\treturn ContentType{}, 0, fmt.Errorf(\"run onnx: %w\", err)\n\t}\n\tif len(scores) == 0 {\n\t\treturn ContentType{}, 0, errors.New(\"run onnx: empty result\")\n\t}\n\tbest := 0\n\tfor i, v := range scores {\n\t\tif v > scores[best] {\n\t\t\tbest = i\n\t\t}\n\t}\n\tct, err := s.contentType(best, scores[best])\n\tif err != nil {\n\t\treturn ContentType{}, 0, fmt.Errorf(\"get content type: %w\", err)\n\t}\n\treturn ct, scores[best], nil\n}\n\nfunc (s *Scanner) contentType(best int, score float32) (ContentType, error) {\n\tl := s.cfg.TargetLabelsSpace[best]\n\tct, ok := s.ckb[l]\n\tif !ok {\n\t\treturn ContentType{}, fmt.Errorf(\"no content type found for %q\", l)\n\t}\n\tth := s.cfg.MediumConfidenceThreshold\n\tif t, ok := s.cfg.Thresholds[l]; ok {\n\t\tth = t\n\t}\n\t// Return the inferred content type if the threshold is met, otherwise\n\t// falls back to a relevant default.\n\tswitch {\n\tcase score >= th:\n\tcase ct.IsText:\n\t\tl = contentTypeLabelTxt\n\tdefault:\n\t\tl = contentTypeLabelUnknown\n\t}\n\tct, ok = s.ckb[l]\n\tif !ok {\n\t\treturn ContentType{}, fmt.Errorf(\"no content type found for %q\", l)\n\t}\n\tif l, ok = s.cfg.Overwrite[l]; ok {\n\t\tif ct, ok = s.ckb[l]; !ok {\n\t\t\treturn ContentType{}, fmt.Errorf(\"no content type found for %q\", l)\n\t\t}\n\t}\n\treturn ct, nil\n}\n" }, { "path": "go/magika/scanner_test.go", "content": "//go:build cgo && onnxruntime\n\npackage magika\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path\"\n\t\"testing\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n)\n\nfunc TestScannerBasic(t *testing.T) {\n\tconst basicDir = \"../../tests_data/basic\"\n\tes, err := os.ReadDir(basicDir)\n\tif err != nil {\n\t\tt.Fatalf(\"read tests data: %v\", err)\n\t}\n\ts := newTestScanner(t)\n\tfor _, e := range es {\n\t\tt.Run(e.Name(), func(t *testing.T) {\n\t\t\tdir := path.Join(basicDir, e.Name())\n\t\t\tes, err := os.ReadDir(dir)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"read tests data: %v\", err)\n\t\t\t}\n\t\t\tfor _, ee := range es {\n\t\t\t\tp := path.Join(dir, ee.Name())\n\t\t\t\tb, err := os.ReadFile(p)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"read %s: %v\", p, err)\n\t\t\t\t}\n\t\t\t\tct, err := s.Scan(bytes.NewReader(b), len(b))\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"scan %s: %v\", p, err)\n\t\t\t\t}\n\t\t\t\tif d := cmp.Diff(e.Name(), ct.Label); d != \"\" {\n\t\t\t\t\tt.Errorf(\"unexpected content type for %s (-want +got):\\n%s\", ee.Name(), d)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestScannerSmall(t *testing.T) {\n\ts := newTestScanner(t)\n\tfor _, c := range []struct {\n\t\tname string\n\t\tdata []byte\n\t\twant string\n\t}{{\n\t\tname: \"empty\",\n\t\tdata: []byte{},\n\t\twant: contentTypeLabelEmpty,\n\t}, {\n\t\tname: \"small txt\",\n\t\tdata: []byte(\"small\"),\n\t\twant: contentTypeLabelTxt,\n\t}, {\n\t\tname: \"small bin\",\n\t\tdata: []byte{0x80, 0x80, 0x80, 0x80},\n\t\twant: contentTypeLabelUnknown,\n\t}} {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tct, err := s.Scan(bytes.NewReader(c.data), len(c.data))\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"scan: %v\", err)\n\t\t\t}\n\t\t\tif d := cmp.Diff(s.ckb[c.want], ct); d != \"\" {\n\t\t\t\tt.Errorf(\"unexpected content type (-want +got):\\n%s\", d)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestScannerReference(t *testing.T) {\n\ttype prediction struct {\n\t\tDl string `json:\"dl\"`\n\t\tOutput string `json:\"output\"`\n\t\tScore float32 `json:\"score\"`\n\t\tOverwriteReason string `json:\"overwrite_reason\"`\n\t}\n\ttype tcase struct {\n\t\tPredictionMode string `json:\"prediction_mode\"`\n\t\tPath string `json:\"path\"`\n\t\tContent []byte `json:\"content_base64\"`\n\t\tStatus string `json:\"status\"`\n\t\tPrediction prediction `json:\"prediction\"`\n\t}\n\n\tfor _, artifacts := range []string{\n\t\t\"standard_v3_3-inference_examples_by_content.json.gz\",\n\t\t\"standard_v3_3-inference_examples_by_path.json.gz\",\n\t} {\n\t\tb, err := loadArtifacts(t, path.Join(\"../../tests_data/reference\", artifacts))\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"load artifacts: %v\", err)\n\t\t}\n\n\t\tvar tcases []*tcase\n\t\tif err := json.Unmarshal(b, &tcases); err != nil {\n\t\t\tt.Fatalf(\"unmarshal: %s\", err)\n\t\t}\n\n\t\ts := newTestScanner(t)\n\t\tfor _, pm := range []string{\"high_confidence\"} {\n\t\t\tt.Run(fmt.Sprintf(\"%s-%s\", artifacts, pm), func(t *testing.T) {\n\t\t\t\tvar count int\n\t\t\t\tfor i, c := range tcases {\n\t\t\t\t\tif c.PredictionMode != pm {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tcount++\n\t\t\t\t\tif c.Path != \"\" {\n\t\t\t\t\t\tp := path.Join(\"../..\", c.Path)\n\t\t\t\t\t\tb, err := os.ReadFile(p)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tt.Errorf(\"read %s [%d]: %v\", p, i, err)\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\tc.Content = b\n\t\t\t\t\t}\n\t\t\t\t\tct, score, err := s.scanScore(bytes.NewReader(c.Content), len(c.Content))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tt.Errorf(\"scan [%d]: %v\", i, err)\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tgot := prediction{\n\t\t\t\t\t\tOutput: ct.Label,\n\t\t\t\t\t\tScore: score,\n\t\t\t\t\t}\n\t\t\t\t\tif d := cmp.Diff(c.Prediction, got,\n\t\t\t\t\t\tcmpopts.EquateApprox(0, 1e-5),\n\t\t\t\t\t\tcmpopts.IgnoreFields(prediction{}, \"Dl\", \"OverwriteReason\"),\n\t\t\t\t\t); d != \"\" {\n\t\t\t\t\t\tt.Errorf(\"unexpected score [%d] (-want +got):\\n%s\", i, d)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif count == 0 {\n\t\t\t\t\tt.Errorf(\"no test cases found\")\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t}\n}\n\nfunc newTestScanner(t *testing.T) *Scanner {\n\tt.Helper()\n\tconst (\n\t\tassetsDir = \"../../assets\"\n\t\tmodelName = \"standard_v3_3\"\n\t)\n\ts, err := NewScanner(assetsDir, modelName)\n\tif err != nil {\n\t\tt.Fatalf(\"new scanner: %v\", err)\n\t}\n\treturn s\n}\n" }, { "path": "go/onnx/onnx.go", "content": "package onnx\n\n// Onnx represents something that can run inferences on features.\ntype Onnx interface {\n\t// Run returns the result of the inference on the given features.\n\tRun(features []int32) ([]float32, error)\n}\n" }, { "path": "go/onnx/onnx_runtime.go", "content": "//go:build cgo && onnxruntime\n\npackage onnx\n\n// #cgo LDFLAGS: -lonnxruntime\n// #include \"onnx_runtime.h\"\nimport \"C\"\n\nimport (\n\t\"fmt\"\n)\n\n// NewOnnx returns an onnx that can perform inferences using an ONNX Runtime\n// (https://onnxruntime.ai/) and the given model.\n// It wraps the C calls to the ONNX Runtime API https://onnxruntime.ai/docs/api/c.\nfunc NewOnnx(modelPath string, sizeTarget int) (Onnx, error) {\n\tort := &onnxRuntime{\n\t\tapi: C.GetApiBase(),\n\t\tsizeTarget: sizeTarget,\n\t}\n\tif err := C.CreateSession(ort.api, C.CString(modelPath), &ort.session, &ort.memory); err != nil {\n\t\treturn nil, fmt.Errorf(\"create session: %v\", C.GoString(C.GetErrorMessage(err)))\n\t}\n\treturn ort, nil\n}\n\n// onnxRuntime implements the Onnx interface relying on a cgo call\n// to a C ONNX Runtime library.\ntype onnxRuntime struct {\n\tapi *C.OrtApi\n\tsession *C.OrtSession\n\tmemory *C.OrtMemoryInfo\n\tsizeTarget int\n}\n\nfunc (ort *onnxRuntime) Run(features []int32) ([]float32, error) {\n\ttarget := make([]float32, ort.sizeTarget)\n\tif err := C.Run(ort.api, ort.session, ort.memory, (*C.int32_t)(&features[0]), C.int64_t(len(features)), (*C.float)(&target[0]), C.int64_t(len(target))); err != nil {\n\t\treturn nil, fmt.Errorf(\"run: %v\", C.GoString(C.GetErrorMessage(err)))\n\t}\n\treturn target, nil\n}\n" }, { "path": "go/onnx/onnx_runtime.h", "content": "#include \n#include \n\n#define RETURN_ON_ERROR(expr) { \\\n\tOrtStatus* onnx_status = (expr); \\\n\tif (onnx_status != NULL) { \\\n\t\treturn onnx_status; \\\n\t} \\\n}\n\nconst OrtApi *GetApiBase() {\n\treturn OrtGetApiBase()->GetApi(ORT_API_VERSION);\n}\n\nOrtStatus *CreateSession(const OrtApi *ort, const char *model, OrtSession **session, OrtMemoryInfo **memory_info) {\n\tOrtEnv *env;\n\tRETURN_ON_ERROR(ort->CreateEnv(ORT_LOGGING_LEVEL_ERROR, \"onnx\", &env));\n\tRETURN_ON_ERROR(ort->DisableTelemetryEvents(env));\n\tOrtSessionOptions *options;\n\tRETURN_ON_ERROR(ort->CreateSessionOptions(&options));\n\tRETURN_ON_ERROR(ort->EnableCpuMemArena(options));\n\tRETURN_ON_ERROR(ort->CreateSession(env, model, options, session));\n\tRETURN_ON_ERROR(ort->CreateCpuMemoryInfo(OrtArenaAllocator, OrtMemTypeDefault, memory_info));\n\treturn NULL;\n}\n\nOrtStatus *Run(const OrtApi *ort, OrtSession *session, OrtMemoryInfo *memory_info, int32_t features[], int64_t sizeFeatures, float target[], int64_t sizeTarget) {\n\tconst char *input_names[] = {\"bytes\"};\n\tconst char *output_names[] = {\"target_label\"};\n\tconst int64_t input_shape[] = {1, sizeFeatures};\n\tOrtValue *input_tensor = NULL;\n\tRETURN_ON_ERROR(ort->CreateTensorWithDataAsOrtValue(memory_info, features, sizeFeatures * sizeof(int32_t), input_shape, 2, ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32, &input_tensor));\n\tOrtValue *output_tensor = NULL;\n\tRETURN_ON_ERROR(ort->Run(session, NULL, input_names, (const OrtValue *const *) &input_tensor, 1, output_names, 1, &output_tensor));\n\tfloat *out = NULL;\n\tRETURN_ON_ERROR(ort->GetTensorMutableData(output_tensor, (void **) &out));\n\tmemcpy(target, out, sizeTarget * sizeof(float));\n\tort->ReleaseValue(input_tensor);\n\tort->ReleaseValue(output_tensor);\n\treturn NULL;\n}\n\nconst char *GetErrorMessage(const OrtStatus* onnx_status) {\n\tif (onnx_status == NULL) {\n\t\treturn \"\";\n\t}\n\treturn OrtGetApiBase()->GetApi(ORT_API_VERSION)->GetErrorMessage(onnx_status);\n}\n" }, { "path": "go/onnx/onnx_runtime_test.go", "content": "//go:build cgo && onnxruntime\n\npackage onnx_test\n\nimport (\n\t\"math/rand/v2\"\n\t\"testing\"\n\n\t\"github.com/google/magika/go/magika\"\n\t\"github.com/google/magika/go/onnx\"\n)\n\nfunc TestONNXRuntime(t *testing.T) {\n\tconst (\n\t\tassetsDir = \"../../assets\"\n\t\tmodelName = \"standard_v3_3\"\n\t\tmodelPath = \"../../assets/models/\" + modelName + \"/model.onnx\"\n\t)\n\n\tcfg, err := magika.ReadConfig(assetsDir, modelName)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\trt, err := onnx.NewOnnx(modelPath, len(cfg.TargetLabelsSpace))\n\tif err != nil {\n\t\tt.Fatalf(\"Create onnx: %v\", err)\n\t}\n\n\t// Initialize a random features tensor.\n\tfeatures := make([]int32, cfg.BegSize+cfg.MidSize+cfg.EndSize)\n\tfor i := range features {\n\t\tfeatures[i] = rand.Int32()\n\t}\n\n\t// Get the scores and check its size.\n\tscores, err := rt.Run(features)\n\tif err != nil {\n\t\tt.Fatalf(\"Run onnx: %v\", err)\n\t}\n\tif n, m := len(scores), len(cfg.TargetLabelsSpace); n != m {\n\t\tt.Fatalf(\"Unexpected scores len: got %d, want %d\", n, m)\n\t}\n}\n" }, { "path": "go/onnx/onnx_zero.go", "content": "//go:build !(cgo && onnxruntime)\n\npackage onnx\n\n// NewOnnx returns a nil Onnx runtime.\n// This allows for building and unit testing in a non-cgo context.\nfunc NewOnnx(string, int) (Onnx, error) {\n\treturn nil, nil\n}\n" }, { "path": "js/.gitignore", "content": "package-lock.json\nnode_modules\n" }, { "path": "js/CHANGELOG.md", "content": "# CHANGELOG\n## [1.0.0]\n - Mark end of experimental mode. No major changes.\n\n## [0.3.2]\n\n- Upgrade to `standard_v3_3` model.\n\n## [0.3.1]\n\n- Overhaul of the API to use much cleaner abstractions.\n- Removed identifyBytesFull and identifyStreamFull: identifyBytes and\n identifyStream now return all the scores as well (accessible with\n `result.prediction.scores_map`).\n- Restrict the input types to `Uint8Array` and `Buffer`.\n\n## [0.2.13] - 2024-03-26\n\n- This is the first working (but still very experimental) version.\n" }, { "path": "js/README.md", "content": "# Magika TypeScript/JavaScript library\n\nMagika is a novel AI-powered file type detection tool that relies on the recent advance of deep learning to provide accurate detection. Under the hood, Magika employs a custom, highly optimized model that only weighs about a few MBs, and enables precise file identification within milliseconds, even when running on a single CPU. Magika has been trained and evaluated on a dataset of ~100M samples across 200+ content types (covering both binary and textual file formats), and it achieves an average ~99% accuracy on our test set.\n\nThis npm package allows you to run Magika in the browser or in Node!\n\nMagika's website: [https://securityresearch.google/magika/](https://securityresearch.google/magika/).\nMagika on GitHub: [https://github.com/google/magika](https://github.com/google/magika).\n\n\n# Installing MagikaJS\n\n```bash\nnpm install magika\n```\n\n# Using MagikaJS\n\nSimple usage in Node:\n\n```js\nimport { readFile } from \"fs/promises\";\nimport { MagikaNode as Magika } from \"magika/node\";\n\nconst data = await readFile(\"some file\");\nconst magika = await Magika().create();\nconst prediction = await magika.identifyBytes(data);\nconsole.log(prediction);\n```\n\nSimple usage in the browser:\n\n```js\nimport { Magika } from \"magika\";\n\nconst file = new File([\"# Hello I am a markdown file\"], \"hello.md\");\nconst fileBytes = new Uint8Array(await file.arrayBuffer());\nconst magika = await Magika.create();\nconst prediction = await magika.identifyBytes(fileBytes);\nconsole.log(prediction);\n```\n\nFor more, see our [documentation](https://securityresearch.google/magika/cli-and-bindings/js/).\n\n# Command-line tool\n\nPlease use the official CLI (with `pip install magika`) as it can perform batch processing and search for files recursively.\nRead more about that in the main [README](https://github.com/google/magika/blob/main/README.md).\nThis one is useful to load the TensorflowJS model and see that it works as expected.\n\nInstall it with `npm install -g magika`. You can then run it by executing `magika-js `\n\n```help\nUsage: magika-js [options] \n\nMagika JS - file type detection with ML. https://securityresearch.google/magika/\n\nArguments:\n paths Paths of the files to detect\n\nOptions:\n --json-output Format output in JSON\n --model-url Model URL\n --model-path Model file path\n --model-config-url Model config URL\n --model-config-path Model config file path\n --by-stream Identify file via stream, not via bytes\n --debug Output debug information\n -h, --help display help for command\n```\n\n# Reporting errors in detections\n\nPlease open an issue on [Github](https://github.com/google/magika/issues).\n\n# Citation\n\nIf you use this software for your research, please cite it as:\n\n```bibtex\n@InProceedings{fratantonio25:magika,\n author = {Yanick Fratantonio and Luca Invernizzi and Loua Farah and Kurt Thomas and Marina Zhang and Ange Albertini and Francois Galilee and Giancarlo Metitieri and Julien Cretin and Alexandre Petit-Bianco and David Tao and Elie Bursztein},\n title = {{Magika: AI-Powered Content-Type Detection}},\n booktitle = {Proceedings of the International Conference on Software Engineering (ICSE)},\n month = {April},\n year = {2025}\n}\n```\n\n# Loading the model and configuration\n\nMagikaJS is designed to be flexible in how you provide the model and configuration file to it.\n\nBoth the Node and browser versions accept URLs to asynchronously load these two assets.\n\n```js\nconst magika = await magika.create({\n modelURL: \"https://...\",\n configURL: \"https://...\",\n});\n```\n\nThe Node version also allows to load local files.\n\n```js\nconst magika = await magika.create({\n modelPath: \"./assets/...\",\n configPath: \"./assets/...\",\n});\n```\n\n# Development\n\nUsing the model hosted On Github:\n\n```bash\nyarn install\nyarn run build\nyarn run bin -- README.md\n```\n\nUsing the local model:\n\n```bash\nyarn install\nyarn run build\n(cd ../website; yarn install; yarn run dev) &\nyarn run bin --model-url http://localhost:5173/magika/model/model.json --config-url http://localhost:5173/magika/model/config.json ../tests_data/basic/*\n```\n\nUsing the local `magika` package when developing the website:\n\n```bash\nyarn install\nyarn run build\nyarn link\n(cd ../website; yarn link magika; yarn install; yarn run dev) &\n```\n\n## Testing\n\nExecute:\n\n```bash\nyarn install\nyarn run build\nyarn run test\n```\n" }, { "path": "js/magika-cli.ts", "content": "#! /usr/bin/env node\n// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// Command line tool to test the JavaScript implementation of Magika. Please use\n// the official command line tool (`pip install magika`) for normal use.\n\n// To run this, you need to install the optional dependencies too.\nimport chalk from \"chalk\";\nimport { CommanderError, program } from \"commander\";\nimport * as fs from \"fs\";\nimport { readFile } from \"fs/promises\";\nimport { MagikaNode as Magika } from \"./magika-node.js\";\n\nprogram\n .description(\n \"Magika JS - file type detection with ML. https://google.github.io/magika\",\n )\n .option(\"--json-output\", \"Format output in JSON\")\n .option(\"--model-url \", \"Model URL\", Magika.MODEL_URL)\n .option(\"--model-path \", \"Model file path\")\n .option(\n \"--model-config-url \",\n \"Model config URL\",\n Magika.MODEL_CONFIG_URL,\n )\n .option(\"--model-config-path \", \"Model config file path\")\n .option(\"--by-stream\", \"Identify file via stream, not via bytes\")\n .option(\"--debug\", \"Output debug information\")\n .argument(\"\", \"Paths of the files to detect\");\n\nprogram.exitOverride();\n\ntry {\n program.parse(process.argv);\n} catch (error: any) {\n if (\n error instanceof CommanderError &&\n error.code === \"commander.helpDisplayed\"\n ) {\n // Help was already displayed by commander, so just exit cleanly.\n process.exit(0);\n } else {\n // There was an error parsing the arguments, let's print the help.\n try {\n program.help();\n } catch (error: any) {\n // Avoid that commander shows some weird exception.\n process.exit(1);\n }\n }\n}\n\nconst flags = program.opts();\n\n(async () => {\n const magika = await Magika.create({\n modelURL: flags.modelUrl,\n modelPath: flags.modelPath,\n modelConfigURL: flags.configUrl,\n modelConfigPath: flags.configPath,\n });\n await Promise.all(\n program.args.map(async (path) => {\n let data = null;\n try {\n data = await readFile(path);\n } catch (error) {\n console.error(\"Skipping file\", path, error);\n }\n\n if (data != null) {\n if (flags.byStream) {\n const magika_result_by_stream = await magika.identifyStream(\n fs.createReadStream(path),\n data.length,\n );\n if (flags.jsonOutput) {\n console.log(path, magika_result_by_stream);\n } else if (flags.debug) {\n console.log(\n chalk.yellow(path),\n \"by_stream\",\n chalk.green(\n magika_result_by_stream.prediction.dl.label,\n magika_result_by_stream.prediction.output.label,\n ),\n chalk.white(magika_result_by_stream.prediction.score),\n );\n } else {\n console.log(\n chalk.yellow(path),\n chalk.green(magika_result_by_stream.prediction.output.label),\n chalk.white(magika_result_by_stream.prediction.score.toFixed(3)),\n );\n }\n } else {\n const magika_result_by_path = await magika.identifyBytes(data);\n if (flags.jsonOutput) {\n console.log(path, magika_result_by_path);\n } else if (flags.debug) {\n console.log(\n chalk.yellow(path),\n \"by_path\",\n chalk.green(\n magika_result_by_path.prediction.dl.label,\n magika_result_by_path.prediction.output.label,\n ),\n chalk.white(magika_result_by_path.prediction.score),\n );\n } else {\n console.log(\n chalk.yellow(path),\n chalk.green(magika_result_by_path.prediction.output.label),\n chalk.white(magika_result_by_path.prediction.score.toFixed(3)),\n );\n }\n }\n }\n }),\n );\n})();\n" }, { "path": "js/magika-node.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { ReadStream } from \"fs\";\nimport { finished } from \"stream/promises\";\nimport { Magika } from \"./magika.js\";\nimport { MagikaOptions } from \"./src/magika-options.js\";\nimport { MagikaResult } from \"./src/magika-result.js\";\nimport { ModelNode } from \"./src/model-node.js\";\nimport { ModelConfigNode } from \"./src/model-config-node.js\";\n\n/**\n * The main Magika object for Node use (`MagikaNode`).\n *\n * Example usage:\n * ```js\n * import { readFile } from \"fs/promises\";\n * import { MagikaNode as Magika } from \"magika/node\";\n * const data = await readFile(\"some file\");\n * const magika = await Magika.create();\n * const result = await magika.identifyBytes(data);\n * console.log(result.prediction.output.label);\n * ```\n * For a client-side implementation, please import `Magika` instead.\n *\n * Note that this `MagikaNode` class extends `Magika`, which means that all\n * public `Magika` APIs (e.g., `identifyBytes`) are available for `MagikaNode`\n * as well.\n *\n * Demos:\n * - Node: `/js/magika-cli.js`, which you can run with `yarn run bin -h`.\n * - Client-side: see `/website/src/components/FileClassifierDemo.vue`\n */\nexport class MagikaNode extends Magika {\n model_config: ModelConfigNode;\n model: ModelNode;\n\n protected constructor() {\n super();\n // We load the version of the model that uses tfjs/node.\n this.model_config = new ModelConfigNode();\n this.model = new ModelNode(this.model_config);\n }\n\n /**\n * Factory method to create a Magika instance.\n *\n * @param {MagikaOptions} options The urls or file paths where the model and\n * its config are stored.\n *\n * Parameters are optional. If not provided, the model will be loaded from GitHub.\n */\n public static async create(options?: MagikaOptions): Promise {\n const magika = new MagikaNode();\n await magika.load(options);\n return magika;\n }\n\n protected async load(options?: MagikaOptions): Promise {\n const promises: Promise[] = [];\n if (options?.modelConfigPath != null) {\n promises.push(this.model_config.loadFile(options?.modelConfigPath));\n } else {\n promises.push(\n this.model_config.loadUrl(\n options?.modelConfigURL || Magika.MODEL_CONFIG_URL,\n ),\n );\n }\n if (options?.modelPath != null) {\n promises.push(this.model.loadFile(options?.modelPath));\n } else {\n promises.push(this.model.loadUrl(options?.modelURL || Magika.MODEL_URL));\n }\n await Promise.all(promises);\n }\n\n /**\n * Identifies the content type from a read stream\n *\n * @param {ReadStream} stream A read stream.\n * @param {number} length Total length of stream data.\n * @returns {MagikaResult} An object containing the result of the content type\n * prediction.\n */\n public async identifyStream(\n stream: ReadStream,\n length: number,\n ): Promise {\n let result = await this._identifyFromStream(stream, length);\n return result;\n }\n\n private async _identifyFromStream(\n stream: ReadStream,\n length: number,\n ): Promise {\n let accData: Buffer = Buffer.from(\"\");\n let fileData: Buffer = Buffer.from(\"\");\n stream.on(\"data\", (data: string | Buffer) => {\n if (typeof data === \"string\") {\n throw new Error(\"Stream data should be a Buffer, not a string\");\n }\n\n // ReadStream allows us to read a file chunk by chunk, sequentially.\n // It does not allow to seek around. So, the optimization we do here\n // is to avoid to store the full file in memory; but we are indeed\n // traversing the full file.\n\n // Here we collect the file bytes. For small files, we collect the full\n // stream of bytes. For large files, we collect only the first and last\n // `block_size` bytes.\n\n if (length <= 4 * this.model_config.block_size) {\n // The file is small; we read the full file in memory.\n fileData = Buffer.concat([fileData, data]);\n } else {\n accData = Buffer.concat([accData, data]);\n if (fileData.length === 0) {\n if (accData.length >= this.model_config.block_size) {\n // We have at least block_size bytes, let's keep them as the first\n // block.\n fileData = Buffer.concat([\n fileData,\n accData.subarray(0, this.model_config.block_size),\n ]);\n }\n }\n if (fileData.length > 0) {\n // If we are here, it means we have already collected block_size bytes\n // and kept it as the \"beg block\". Now, we keep processing bytes, and\n // we just store the last block_size bytes. Then, once we are at the\n // very end of the stream, we take these last block_size bytes as the\n // \"end block\".\n accData = accData.subarray(\n accData.length - this.model_config.block_size,\n );\n if (stream.bytesRead === length) {\n // We have just read the last chunk. We now these last block_size\n // bytes as \"the end block\", which together with the \"beg block\"\n // form the file's bytes that we can pass to the features\n // extraction.\n fileData = Buffer.concat([fileData, accData]);\n }\n }\n }\n });\n await finished(stream);\n return await this._identifyFromBytes(fileData);\n }\n}\n" }, { "path": "js/magika.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { ContentTypeInfo } from \"./src/content-type-info.js\";\nimport { ContentTypeLabel } from \"./src/content-type-label.js\";\nimport { ContentTypesInfos } from \"./src/content-types-infos.js\";\nimport { MagikaOptions } from \"./src/magika-options.js\";\nimport { MagikaResult } from \"./src/magika-result.js\";\nimport { ModelConfig } from \"./src/model-config.js\";\nimport { ModelFeatures } from \"./src/model-features.js\";\nimport { ModelPrediction } from \"./src/model-prediction.js\";\nimport { Model } from \"./src/model.js\";\nimport { OverwriteReason } from \"./src/overwrite-reason.js\";\nimport { Status } from \"./src/status.js\";\n\n/**\n * The main Magika object for client-side use.\n *\n * Example usage:\n * ```js\n * const file = new File([\"# Hello I am a markdown file\"], \"hello.md\");\n * const fileBytes = new Uint8Array(await file.arrayBuffer());\n * const magika = await Magika.create();\n * const result = await magika.identifyBytes(fileBytes);\n * console.log(result.prediction.output.label);\n * ```\n * For a Node implementation, please import `MagikaNode` instead.\n *\n * Demos:\n * - Node: `/js/magika-cli.js`, which you can run with `yarn run bin -h`.\n * - Client-side: see `/website/src/components/FileClassifierDemo.vue`\n */\nexport class Magika {\n model_config: ModelConfig;\n model: Model;\n model_name: string;\n cts_infos: ContentTypesInfos;\n\n static MODEL_VERSION = \"standard_v3_3\";\n static MODEL_CONFIG_URL = `https://google.github.io/magika/models/${this.MODEL_VERSION}/config.min.json`;\n static MODEL_URL = `https://google.github.io/magika/models/${this.MODEL_VERSION}/model.json`;\n static WHITESPACE_CHARS = [...\" \\t\\n\\r\\v\\f\"].map((c) => c.charCodeAt(0));\n\n protected constructor() {\n this.model_config = new ModelConfig();\n this.model = new Model(this.model_config);\n this.model_name = \"unknown\";\n this.cts_infos = ContentTypesInfos.get();\n }\n\n /**\n * Factory method to create a Magika instance.\n *\n * @param {MagikaOptions} options The urls or file paths where the model and\n * its config are stored.\n *\n * Parameters are optional. If not provided, the model will be loaded from GitHub.\n */\n public static async create(options?: MagikaOptions): Promise {\n const magika = new Magika();\n await magika.load(options);\n return magika;\n }\n\n protected async load(options?: MagikaOptions): Promise {\n const modelURL = options?.modelURL || Magika.MODEL_URL;\n const modelConfigURL = options?.modelConfigURL || Magika.MODEL_CONFIG_URL;\n this.model_name = this._getModelName(modelURL);\n await Promise.all([\n this.model.loadUrl(modelURL),\n this.model_config.loadUrl(modelConfigURL),\n ]);\n }\n\n /**\n * Identifies the content type of a byte array.\n *\n * @param {Uint8Array} fileBytes A fixed-length sequence of bytes.\n * @returns {MagikaResult} An object containing the result of the content type\n * prediction.\n */\n public async identifyBytes(fileBytes: Uint8Array): Promise {\n const result = await this._identifyFromBytes(fileBytes);\n return result;\n }\n\n public getModelName(): string {\n return this.model_name;\n }\n\n private _getResultFromFewBytes(\n fileBytes: Uint8Array,\n path: string = \"-\",\n ): MagikaResult {\n if (fileBytes.length > 4 * this.model_config.block_size) {\n throw new Error(\"fileBytes is unexpectedly long for this function.\");\n }\n const decoder = new TextDecoder(\"utf-8\", { fatal: true });\n try {\n decoder.decode(fileBytes);\n\n return this._getResultFromLabelsAndScore(\n path,\n Status.OK,\n ContentTypeLabel.UNDEFINED,\n ContentTypeLabel.TXT,\n 1.0,\n );\n } catch (error) {\n return this._getResultFromLabelsAndScore(\n path,\n Status.OK,\n ContentTypeLabel.UNDEFINED,\n ContentTypeLabel.UNKNOWN,\n 1.0,\n );\n }\n }\n\n private static _lstrip(fileBytes: Uint8Array): Uint8Array {\n let startIndex = 0;\n while (\n startIndex < fileBytes.length &&\n Magika.WHITESPACE_CHARS.includes(fileBytes[startIndex])\n ) {\n startIndex++;\n }\n return fileBytes.subarray(startIndex);\n }\n\n private static _rstrip(fileBytes: Uint8Array): Uint8Array {\n let endIndex = fileBytes.length - 1;\n while (\n endIndex >= 0 &&\n Magika.WHITESPACE_CHARS.includes(fileBytes[endIndex])\n ) {\n endIndex--;\n }\n return fileBytes.subarray(0, endIndex + 1);\n }\n\n protected async _identifyFromBytes(\n fileBytes: Uint8Array,\n ): Promise {\n if (fileBytes.length === 0) {\n return this._getResultFromLabelsAndScore(\n \"-\",\n Status.OK,\n ContentTypeLabel.UNDEFINED,\n ContentTypeLabel.EMPTY,\n 1.0,\n );\n }\n\n if (fileBytes.length < this.model_config.min_file_size_for_dl) {\n return this._getResultFromFewBytes(fileBytes);\n }\n\n const features = Magika._extractFeaturesFromBytes(\n fileBytes,\n this.model_config.beg_size,\n this.model_config.mid_size,\n this.model_config.end_size,\n this.model_config.padding_token,\n this.model_config.block_size,\n this.model_config.use_inputs_at_offsets,\n );\n return await this._getResultFromFeatures(features);\n }\n\n private _getOutputLabelFromModelPrediction(\n model_prediction: ModelPrediction,\n ): [ContentTypeLabel, OverwriteReason] {\n let overwrite_reason = OverwriteReason.NONE;\n\n // Overwrite model_prediction.label if specified in the overwrite_map.\n let output_label =\n this.model_config.overwrite_map[model_prediction.label] ??\n model_prediction.label;\n if (output_label != model_prediction.label) {\n overwrite_reason = OverwriteReason.OVERWRITE_MAP;\n }\n\n // The following code checks whether the score is \"high enough\" according to\n // HIGH_CONFIDENCE prediction mode (the only one we currently support in\n // this implementation). If it's not, it means we can't trust the model, and\n // we return a generic content type.\n if (\n model_prediction.score <\n (this.model_config.thresholds[model_prediction.label] ??\n this.model_config.medium_confidence_threshold)\n ) {\n overwrite_reason = OverwriteReason.LOW_CONFIDENCE;\n if (this.cts_infos[model_prediction.label].is_text) {\n output_label = ContentTypeLabel.TXT;\n } else {\n output_label = ContentTypeLabel.UNKNOWN;\n }\n if (model_prediction.label === output_label) {\n // overwrite_reason is useful to convey to clients why the output\n // predicted is different than the model predicted type; if those two\n // are the same, the model predicted type has not actually been\n // overwritten, so we set this to NONE.\n overwrite_reason = OverwriteReason.NONE;\n }\n }\n\n return [output_label, overwrite_reason];\n }\n\n protected static _extractFeaturesFromBytes(\n fileBytes: Uint8Array,\n beg_size: number,\n mid_size: number,\n end_size: number,\n padding_token: number,\n block_size: number,\n use_inputs_at_offsets: boolean,\n ): ModelFeatures {\n const begChunk = this._lstrip(\n fileBytes.slice(0, Math.min(block_size, fileBytes.length)),\n );\n const begBytes = begChunk.slice(0, Math.min(begChunk.length, beg_size));\n\n const endChunk = this._rstrip(\n fileBytes.slice(Math.max(0, fileBytes.length - block_size)),\n );\n const endBytes = endChunk.slice(Math.max(0, endChunk.length - end_size));\n const endOffset = Math.max(0, end_size - endBytes.length);\n\n return new ModelFeatures(\n beg_size,\n mid_size,\n end_size,\n padding_token,\n use_inputs_at_offsets,\n )\n .withStart(begBytes, 0)\n .withEnd(endBytes, endOffset);\n }\n\n private _getContentTypeInfo(label: ContentTypeLabel): ContentTypeInfo {\n return this.cts_infos[label];\n }\n\n private _getResultFromLabelsAndScore(\n path: string,\n status: Status = Status.OK,\n dl_label: ContentTypeLabel,\n output: ContentTypeLabel,\n score: number,\n overwrite_reason: OverwriteReason = OverwriteReason.NONE,\n scores_map?: Partial>,\n ): MagikaResult {\n return {\n path: path,\n status: status,\n prediction: {\n dl: this._getContentTypeInfo(dl_label),\n output: this._getContentTypeInfo(output),\n score: score,\n overwrite_reason: overwrite_reason,\n scores_map: scores_map,\n },\n };\n }\n\n private async _getResultFromFeatures(\n features: ModelFeatures,\n ): Promise {\n let model_prediction = await this.model.predict(features);\n let [output_label, overwrite_reason] =\n this._getOutputLabelFromModelPrediction(model_prediction);\n return this._getResultFromLabelsAndScore(\n \"-\",\n Status.OK,\n model_prediction.label,\n output_label,\n model_prediction.score,\n overwrite_reason,\n model_prediction.scores_map,\n );\n }\n\n protected _getModelName(pathOrUrl: string): string {\n const UNKNOWN_MODEL_NAME = \"unknown\";\n try {\n const parts = pathOrUrl.split(\"/\");\n // Filter out empty strings that can occur due to leading/trailing slashes\n const nonEmptyParts = parts.filter((part) => part !== \"\");\n\n if (nonEmptyParts.length >= 2) {\n return nonEmptyParts[nonEmptyParts.length - 2];\n } else {\n return UNKNOWN_MODEL_NAME;\n }\n } catch (error) {\n console.error(\"Error processing path or URL to get model name:\", error);\n return UNKNOWN_MODEL_NAME;\n }\n }\n}\n" }, { "path": "js/package.json", "content": "{\n \"name\": \"magika\",\n \"version\": \"1.0.0\",\n \"description\": \"A tool to detect content types with deep learning.\",\n \"type\": \"module\",\n \"main\": \"./dist/cjs/magika.js\",\n \"types\": \"./dist/cjs/magika.d.ts\",\n \"module\": \"./dist/mjs/magika.js\",\n \"exports\": {\n \"./node\": {\n \"import\": \"./dist/mjs/magika-node.js\",\n \"require\": \"./dist/cjs/magika-node.js\",\n \"default\": \"./dist/cjs/magika-node.js\"\n },\n \".\": {\n \"import\": \"./dist/mjs/magika.js\",\n \"require\": \"./dist/cjs/magika.js\",\n \"default\": \"./dist/cjs/magika.js\"\n }\n },\n \"repository\": \"https://github.com/google/magika\",\n \"author\": \"Luca Invernizzi , Yanick Fratantonio \",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"content type\",\n \"file type\",\n \"magic\"\n ],\n \"scripts\": {\n \"bin\": \"TF_CPP_MIN_LOG_LEVEL=1 node ./dist/mjs/magika-cli.js\",\n \"make-docs\": \"documentation build *.ts --parse-extension ts -f md --github -o ../docs/js.md\",\n \"test\": \"TF_CPP_MIN_LOG_LEVEL=1 yarn build && node node_modules/jest/bin/jest.js\",\n \"build\": \"rm -fr dist/* && tsc -p tsconfig.esm.json && tsc -p tsconfig.cjs.json && node postBuild.js\",\n \"format\": \"prettier --write *.ts src/*.ts\"\n },\n \"bin\": {\n \"magika-js\": \"./dist/cjs/magika-cli.js\"\n },\n \"dependencies\": {\n \"@tensorflow/tfjs\": \"^4.22.0\"\n },\n \"optionalDependencies\": {\n \"@tensorflow/tfjs-node\": \"^4.22.0\",\n \"chalk\": \"^5.3.0\",\n \"commander\": \"^13.1.0\"\n },\n \"jest\": {\n \"moduleFileExtensions\": [\n \"js\",\n \"json\",\n \"ts\"\n ],\n \"transformIgnorePatterns\": [\n \"/node_modules/\"\n ],\n \"testPathIgnorePatterns\": [\n \"/node_modules/\"\n ],\n \"rootDir\": \"./\",\n \"testRegex\": \".test.ts$\",\n \"transform\": {\n \"^.+\\\\.ts$\": [\n \"ts-jest\",\n {\n \"tsconfig\": \"./tsconfig.cjs.json\"\n }\n ]\n },\n \"moduleNameMapper\": {\n \"(.+)\\\\.js\": \"$1\"\n },\n \"testEnvironment\": \"node\",\n \"collectCoverage\": false\n },\n \"devDependencies\": {\n \"@types/jest\": \"^29.5.12\",\n \"@types/node\": \"^24.3.1\",\n \"documentation\": \"^14.0.3\",\n \"fast-check\": \"^4.1.0\",\n \"jest\": \"^29.5.0\",\n \"prettier\": \"^3.5.3\",\n \"ts-jest\": \"^29.0.5\",\n \"typescript\": \"^5.0.2\"\n }\n}\n" }, { "path": "js/postBuild.js", "content": "\n// format sub package.json for dual cjs and esm support\nimport fs from 'fs';\n\nconst formatPackage = (source, output, type) => {\n\tconst remove = {main: true, module: true, browser: true, types: true, exports: true};\n\tconst json = JSON.parse(fs.readFileSync(source, 'utf-8'));\n\tfs.writeFileSync(output, JSON.stringify({\n\t\t...Object.fromEntries(Object.entries(json).filter(([key]) => !remove[key])),\n\t\ttype:type\n\t}, null, 4));\n}\nformatPackage('./package.json', './dist/cjs/package.json', 'commonjs');\nformatPackage('./package.json', './dist/mjs/package.json', 'module');\n" }, { "path": "js/simple_examples/browser-esmodule-example/index.html", "content": "\n\n\n\n \n \n Playwright Test\n\n\n\n \n\n\n" }, { "path": "js/simple_examples/browser-esmodule-example/index.js", "content": "import { Magika } from \"magika\";\n\nasync function main() {\n const magika = await Magika.create();\n const data = new TextEncoder().encode(\"import os; print(os.uname())\");\n const prediction = await magika.identifyBytes(data);\n const statusDiv = document.createElement(\"div\");\n statusDiv.className = \"status\";\n statusDiv.textContent = prediction.status;\n document.body.appendChild(statusDiv);\n\n const labelDiv = document.createElement(\"div\");\n labelDiv.className = \"label\";\n labelDiv.textContent = prediction.prediction.output.label;\n document.body.appendChild(labelDiv);\n}\n\ndocument.addEventListener(\"DOMContentLoaded\", main);\n" }, { "path": "js/simple_examples/browser-esmodule-example/package.json", "content": "{\n \"name\": \"browser-esmodule-example\",\n \"version\": \"1.0.0\",\n \"description\": \"Magika browser esmodule example\",\n \"scripts\": {\n \"server\": \"vite --port 8000\",\n \"start\": \"npx playwright test\"\n },\n \"author\": \"Luca Invernizzi \",\n \"license\": \"ISC\",\n \"dependencies\": {\n \"magika\": \"file:../../\",\n \"vite\": \"^7.1.7\"\n },\n \"devDependencies\": {\n \"@playwright/test\": \"^1.55.1\"\n }\n}\n" }, { "path": "js/simple_examples/browser-esmodule-example/playwright.config.ts", "content": "import { defineConfig } from \"@playwright/test\";\n\nexport default defineConfig({\n // Run your local dev server before starting the tests\n webServer: {\n command: \"npm run server\",\n url: \"http://localhost:8000\",\n reuseExistingServer: !process.env.CI,\n stdout: \"ignore\",\n stderr: \"pipe\",\n },\n});\n" }, { "path": "js/simple_examples/browser-esmodule-example/test/simple.spec.ts", "content": "import { test, expect } from \"@playwright/test\";\n\ntest(\"can run Magika\", async ({ page }) => {\n await page.goto(\"http://localhost:8000\");\n await page.waitForSelector(\"div.status\");\n await expect(page.locator(\"div.status\")).toHaveText(\"ok\");\n await page.waitForSelector(\"div.label\");\n await expect(page.locator(\"div.label\")).toHaveText(\"python\");\n});\n" }, { "path": "js/simple_examples/node-commonjs-example/index.js", "content": "const { MagikaNode: Magika } = require(\"magika/node\");\n\nasync function main() {\n const magika = await Magika.create();\n const prediction = await magika.identifyBytes(\n Buffer.from(\"import os; print(os.uname())\")\n );\n console.log(prediction.status, prediction.prediction.output.label);\n}\n\nmain();\n" }, { "path": "js/simple_examples/node-commonjs-example/package.json", "content": "{\n \"name\": \"magika-node-commonjs-example\",\n \"version\": \"1.0.0\",\n \"main\": \"index.js\",\n \"scripts\": {\n \"start\": \"node index.js\"\n },\n \"author\": \"Luca Invernizzi \",\n \"license\": \"ISC\",\n \"description\": \"Magika node commonJS example\",\n \"dependencies\": {\n \"magika\": \"file:../../\"\n }\n}\n" }, { "path": "js/simple_examples/node-esmodule-example/index.js", "content": "import { MagikaNode as Magika } from \"magika/node\";\nimport { TextEncoder } from \"util\";\n\nconst magika = await Magika.create();\nconst text = \"import os; print(os.uname())\";\nconst bytes = new TextEncoder().encode(text);\nconst prediction = await magika.identifyBytes(bytes);\nconsole.log(prediction.status, prediction.prediction.output.label);\n" }, { "path": "js/simple_examples/node-esmodule-example/package.json", "content": "{\n \"name\": \"magika-node-esmodule-example\",\n \"version\": \"1.0.0\",\n \"main\": \"index.js\",\n \"type\": \"module\",\n \"scripts\": {\n \"start\": \"node index.js\"\n },\n \"author\": \"Luca Invernizzi \",\n \"license\": \"ISC\",\n \"description\": \"Using Magika in Node via imports\",\n \"dependencies\": {\n \"magika\": \"file:../../\"\n }\n}\n" }, { "path": "js/simple_examples/run_examples.sh", "content": "#!/bin/bash\n\n# Exit on error\nset -e\n\nROOT_DIR=$(pwd)\nexport TF_CPP_MIN_LOG_LEVEL=2\nexport NODE_OPTIONS='--no-warnings'\n\nEXAMPLES=(\"node-commonjs-example\" \"node-esmodule-example\" \"browser-esmodule-example\" \"typescript-esmodule-example\")\n\nfor example in \"${EXAMPLES[@]}\"\ndo\n echo \"--- Running Example: $example ---\"\n if [ -d \"$example\" ]; then\n cd \"$example\"\n if [ -f \"package.json\" ]; then\n npm run --silent start\n else\n echo \"No package.json found in $(pwd)\"\n fi\n cd \"$ROOT_DIR\"\n else\n echo \"Directory $example not found\"\n fi\ndone\n\n" }, { "path": "js/simple_examples/typescript-esmodule-example/index.ts", "content": "import { MagikaNode as Magika } from \"magika/node\";\n\nasync function main(): Promise {\n const magika = await Magika.create();\n const data = Buffer.from(\"import os; print(os.uname())\");\n const prediction = await magika.identifyBytes(data);\n console.log(prediction.status, prediction.prediction.output.label);\n}\n\nmain();\n" }, { "path": "js/simple_examples/typescript-esmodule-example/package.json", "content": "{\n \"name\": \"magika-typescript-esmodule-example\",\n \"version\": \"1.0.0\",\n \"main\": \"index.js\",\n \"type\": \"module\",\n \"scripts\": {\n \"start\": \"npx tsx index.ts\"\n },\n \"author\": \"Luca Invernizzi \",\n \"license\": \"ISC\",\n \"description\": \"Using magika in node & typescript via imports\",\n \"dependencies\": {\n \"magika\": \"file:../../\"\n },\n \"devDependencies\": {\n \"ts-node\": \"^10.9.2\",\n \"tsx\": \"^4.20.6\",\n \"typescript\": \"^5.4.5\"\n }\n}\n" }, { "path": "js/src/.npmignore", "content": "" }, { "path": "js/src/content-type-info.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { ContentTypeLabel } from \"./content-type-label.js\";\n\nexport interface ContentTypeInfo {\n label: ContentTypeLabel;\n is_text: boolean;\n}\n" }, { "path": "js/src/content-type-label.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// NOTE: DO NOT EDIT --- This file is automatically generated by sync.py.\n\n// This is the list of all possible content types we know about; however, models\n// support a smaller subset of them. See model's README.md for details.\nexport enum ContentTypeLabel {\n _3DS = \"3ds\",\n _3DSM = \"3dsm\",\n _3DSX = \"3dsx\",\n _3GP = \"3gp\",\n _3MF = \"3mf\",\n ABNF = \"abnf\",\n ACE = \"ace\",\n ADA = \"ada\",\n AFF = \"aff\",\n AI = \"ai\",\n AIDL = \"aidl\",\n ALGOL68 = \"algol68\",\n ANI = \"ani\",\n APK = \"apk\",\n APPLEBPLIST = \"applebplist\",\n APPLEDOUBLE = \"appledouble\",\n APPLEPLIST = \"appleplist\",\n APPLESINGLE = \"applesingle\",\n AR = \"ar\",\n ARC = \"arc\",\n ARJ = \"arj\",\n ARROW = \"arrow\",\n ASC = \"asc\",\n ASD = \"asd\",\n ASF = \"asf\",\n ASM = \"asm\",\n ASP = \"asp\",\n AU = \"au\",\n AUTOHOTKEY = \"autohotkey\",\n AUTOIT = \"autoit\",\n AVI = \"avi\",\n AVIF = \"avif\",\n AVRO = \"avro\",\n AWK = \"awk\",\n AX = \"ax\",\n BATCH = \"batch\",\n BAZEL = \"bazel\",\n BCAD = \"bcad\",\n BIB = \"bib\",\n BMP = \"bmp\",\n BPG = \"bpg\",\n BPL = \"bpl\",\n BRAINFUCK = \"brainfuck\",\n BRF = \"brf\",\n BZIP = \"bzip\",\n BZIP3 = \"bzip3\",\n C = \"c\",\n CAB = \"cab\",\n CAD = \"cad\",\n CAT = \"cat\",\n CDF = \"cdf\",\n CHM = \"chm\",\n CLOJURE = \"clojure\",\n CMAKE = \"cmake\",\n COBOL = \"cobol\",\n COFF = \"coff\",\n COFFEESCRIPT = \"coffeescript\",\n COM = \"com\",\n CPL = \"cpl\",\n CPP = \"cpp\",\n CRT = \"crt\",\n CRX = \"crx\",\n CS = \"cs\",\n CSPROJ = \"csproj\",\n CSS = \"css\",\n CSV = \"csv\",\n CTL = \"ctl\",\n DART = \"dart\",\n DEB = \"deb\",\n DEX = \"dex\",\n DEY = \"dey\",\n DICOM = \"dicom\",\n DIFF = \"diff\",\n DIRECTORY = \"directory\",\n DJANGO = \"django\",\n DLL = \"dll\",\n DM = \"dm\",\n DMG = \"dmg\",\n DMIGD = \"dmigd\",\n DMSCRIPT = \"dmscript\",\n DOC = \"doc\",\n DOCKERFILE = \"dockerfile\",\n DOCX = \"docx\",\n DOSMBR = \"dosmbr\",\n DOTX = \"dotx\",\n DSSTORE = \"dsstore\",\n DWG = \"dwg\",\n DXF = \"dxf\",\n DYLIB = \"dylib\",\n EBML = \"ebml\",\n ELF = \"elf\",\n ELIXIR = \"elixir\",\n EMF = \"emf\",\n EML = \"eml\",\n EMPTY = \"empty\",\n EPUB = \"epub\",\n ERB = \"erb\",\n ERLANG = \"erlang\",\n ESE = \"ese\",\n EXE = \"exe\",\n EXP = \"exp\",\n FLAC = \"flac\",\n FLUTTER = \"flutter\",\n FLV = \"flv\",\n FORTRAN = \"fortran\",\n FPX = \"fpx\",\n GEMFILE = \"gemfile\",\n GEMSPEC = \"gemspec\",\n GIF = \"gif\",\n GITATTRIBUTES = \"gitattributes\",\n GITMODULES = \"gitmodules\",\n GLEAM = \"gleam\",\n GO = \"go\",\n GPX = \"gpx\",\n GRADLE = \"gradle\",\n GROOVY = \"groovy\",\n GZIP = \"gzip\",\n H = \"h\",\n H5 = \"h5\",\n HANDLEBARS = \"handlebars\",\n HASKELL = \"haskell\",\n HCL = \"hcl\",\n HEIF = \"heif\",\n HFS = \"hfs\",\n HLP = \"hlp\",\n HPP = \"hpp\",\n HTA = \"hta\",\n HTACCESS = \"htaccess\",\n HTML = \"html\",\n HVE = \"hve\",\n HWP = \"hwp\",\n ICC = \"icc\",\n ICNS = \"icns\",\n ICO = \"ico\",\n ICS = \"ics\",\n IGNOREFILE = \"ignorefile\",\n IMG = \"img\",\n INI = \"ini\",\n INTERNETSHORTCUT = \"internetshortcut\",\n IOSAPP = \"iosapp\",\n IPYNB = \"ipynb\",\n ISO = \"iso\",\n JAR = \"jar\",\n JAVA = \"java\",\n JAVABYTECODE = \"javabytecode\",\n JAVASCRIPT = \"javascript\",\n JINJA = \"jinja\",\n JNG = \"jng\",\n JNLP = \"jnlp\",\n JP2 = \"jp2\",\n JPEG = \"jpeg\",\n JSON = \"json\",\n JSONC = \"jsonc\",\n JSONL = \"jsonl\",\n JSX = \"jsx\",\n JULIA = \"julia\",\n JXL = \"jxl\",\n KO = \"ko\",\n KOTLIN = \"kotlin\",\n KS = \"ks\",\n LATEX = \"latex\",\n LATEXAUX = \"latexaux\",\n LESS = \"less\",\n LHA = \"lha\",\n LICENSE = \"license\",\n LISP = \"lisp\",\n LITCS = \"litcs\",\n LNK = \"lnk\",\n LOCK = \"lock\",\n LRZ = \"lrz\",\n LUA = \"lua\",\n LZ = \"lz\",\n LZ4 = \"lz4\",\n LZX = \"lzx\",\n M3U = \"m3u\",\n M4 = \"m4\",\n MACHO = \"macho\",\n MAFF = \"maff\",\n MAKEFILE = \"makefile\",\n MARKDOWN = \"markdown\",\n MATLAB = \"matlab\",\n MHT = \"mht\",\n MIDI = \"midi\",\n MKV = \"mkv\",\n MP2 = \"mp2\",\n MP3 = \"mp3\",\n MP4 = \"mp4\",\n MPEGTS = \"mpegts\",\n MSCOMPRESS = \"mscompress\",\n MSI = \"msi\",\n MSIX = \"msix\",\n MST = \"mst\",\n MUI = \"mui\",\n MUM = \"mum\",\n MUN = \"mun\",\n NIM = \"nim\",\n NPY = \"npy\",\n NPZ = \"npz\",\n NULL = \"null\",\n NUPKG = \"nupkg\",\n OBJECT = \"object\",\n OBJECTIVEC = \"objectivec\",\n OCAML = \"ocaml\",\n OCX = \"ocx\",\n ODEX = \"odex\",\n ODIN = \"odin\",\n ODP = \"odp\",\n ODS = \"ods\",\n ODT = \"odt\",\n OGG = \"ogg\",\n OLE = \"ole\",\n ONE = \"one\",\n ONNX = \"onnx\",\n OOXML = \"ooxml\",\n OTF = \"otf\",\n OUTLOOK = \"outlook\",\n PALMOS = \"palmos\",\n PARQUET = \"parquet\",\n PASCAL = \"pascal\",\n PBM = \"pbm\",\n PCAP = \"pcap\",\n PDB = \"pdb\",\n PDF = \"pdf\",\n PEBIN = \"pebin\",\n PEM = \"pem\",\n PERL = \"perl\",\n PGP = \"pgp\",\n PHP = \"php\",\n PICKLE = \"pickle\",\n PNG = \"png\",\n PO = \"po\",\n POSTSCRIPT = \"postscript\",\n POWERSHELL = \"powershell\",\n PPT = \"ppt\",\n PPTX = \"pptx\",\n PRINTFOX = \"printfox\",\n PROLOG = \"prolog\",\n PROTEINDB = \"proteindb\",\n PROTO = \"proto\",\n PROTOBUF = \"protobuf\",\n PSD = \"psd\",\n PUB = \"pub\",\n PYTHON = \"python\",\n PYTHONBYTECODE = \"pythonbytecode\",\n PYTHONPAR = \"pythonpar\",\n PYTORCH = \"pytorch\",\n QOI = \"qoi\",\n QT = \"qt\",\n R = \"r\",\n RANDOMASCII = \"randomascii\",\n RANDOMBYTES = \"randombytes\",\n RANDOMTXT = \"randomtxt\",\n RAR = \"rar\",\n RDF = \"rdf\",\n RDP = \"rdp\",\n RIFF = \"riff\",\n RLIB = \"rlib\",\n RLL = \"rll\",\n RPM = \"rpm\",\n RST = \"rst\",\n RTF = \"rtf\",\n RUBY = \"ruby\",\n RUST = \"rust\",\n RZIP = \"rzip\",\n SCALA = \"scala\",\n SCHEME = \"scheme\",\n SCR = \"scr\",\n SCRIPTWSF = \"scriptwsf\",\n SCSS = \"scss\",\n SEVENZIP = \"sevenzip\",\n SGML = \"sgml\",\n SH3D = \"sh3d\",\n SHELL = \"shell\",\n SMALI = \"smali\",\n SNAP = \"snap\",\n SO = \"so\",\n SOLIDITY = \"solidity\",\n SQL = \"sql\",\n SQLITE = \"sqlite\",\n SQUASHFS = \"squashfs\",\n SRT = \"srt\",\n STLBINARY = \"stlbinary\",\n STLTEXT = \"stltext\",\n SUM = \"sum\",\n SVD = \"svd\",\n SVG = \"svg\",\n SWF = \"swf\",\n SWIFT = \"swift\",\n SYMLINK = \"symlink\",\n SYMLINKTEXT = \"symlinktext\",\n SYS = \"sys\",\n TAR = \"tar\",\n TCL = \"tcl\",\n TEXTPROTO = \"textproto\",\n TGA = \"tga\",\n THUMBSDB = \"thumbsdb\",\n TIFF = \"tiff\",\n TMDX = \"tmdx\",\n TOML = \"toml\",\n TORRENT = \"torrent\",\n TROFF = \"troff\",\n TSV = \"tsv\",\n TSX = \"tsx\",\n TTF = \"ttf\",\n TWIG = \"twig\",\n TXT = \"txt\",\n TXTASCII = \"txtascii\",\n TXTUTF16 = \"txtutf16\",\n TXTUTF8 = \"txtutf8\",\n TYPESCRIPT = \"typescript\",\n UDF = \"udf\",\n UNDEFINED = \"undefined\",\n UNIXCOMPRESS = \"unixcompress\",\n UNKNOWN = \"unknown\",\n VBA = \"vba\",\n VBE = \"vbe\",\n VCARD = \"vcard\",\n VCS = \"vcs\",\n VCXPROJ = \"vcxproj\",\n VERILOG = \"verilog\",\n VHD = \"vhd\",\n VHDL = \"vhdl\",\n VISIO = \"visio\",\n VTT = \"vtt\",\n VUE = \"vue\",\n WAD = \"wad\",\n WASM = \"wasm\",\n WAV = \"wav\",\n WEBM = \"webm\",\n WEBP = \"webp\",\n WEBTEMPLATE = \"webtemplate\",\n WIM = \"wim\",\n WINREGISTRY = \"winregistry\",\n WMA = \"wma\",\n WMF = \"wmf\",\n WMV = \"wmv\",\n WOFF = \"woff\",\n WOFF2 = \"woff2\",\n XAR = \"xar\",\n XCF = \"xcf\",\n XLS = \"xls\",\n XLSB = \"xlsb\",\n XLSX = \"xlsx\",\n XML = \"xml\",\n XPI = \"xpi\",\n XSD = \"xsd\",\n XZ = \"xz\",\n YAML = \"yaml\",\n YARA = \"yara\",\n ZIG = \"zig\",\n ZIP = \"zip\",\n ZLIBSTREAM = \"zlibstream\",\n ZST = \"zst\",\n}\n" }, { "path": "js/src/content-types-infos.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// NOTE: DO NOT EDIT --- This file is automatically generated by sync.py.\n\nimport { ContentTypeInfo } from \"./content-type-info.js\";\nimport { ContentTypeLabel } from \"./content-type-label.js\";\n\nexport type ContentTypesInfos = Record;\n\nexport const ContentTypesInfos = {\n get: (): ContentTypesInfos => ({\n [ContentTypeLabel._3DS]: {\n label: ContentTypeLabel._3DS,\n is_text: false,\n },\n [ContentTypeLabel._3DSM]: {\n label: ContentTypeLabel._3DSM,\n is_text: false,\n },\n [ContentTypeLabel._3DSX]: {\n label: ContentTypeLabel._3DSX,\n is_text: false,\n },\n [ContentTypeLabel._3GP]: {\n label: ContentTypeLabel._3GP,\n is_text: false,\n },\n [ContentTypeLabel._3MF]: {\n label: ContentTypeLabel._3MF,\n is_text: false,\n },\n [ContentTypeLabel.ABNF]: {\n label: ContentTypeLabel.ABNF,\n is_text: false,\n },\n [ContentTypeLabel.ACE]: {\n label: ContentTypeLabel.ACE,\n is_text: false,\n },\n [ContentTypeLabel.ADA]: {\n label: ContentTypeLabel.ADA,\n is_text: false,\n },\n [ContentTypeLabel.AFF]: {\n label: ContentTypeLabel.AFF,\n is_text: true,\n },\n [ContentTypeLabel.AI]: {\n label: ContentTypeLabel.AI,\n is_text: false,\n },\n [ContentTypeLabel.AIDL]: {\n label: ContentTypeLabel.AIDL,\n is_text: true,\n },\n [ContentTypeLabel.ALGOL68]: {\n label: ContentTypeLabel.ALGOL68,\n is_text: false,\n },\n [ContentTypeLabel.ANI]: {\n label: ContentTypeLabel.ANI,\n is_text: false,\n },\n [ContentTypeLabel.APK]: {\n label: ContentTypeLabel.APK,\n is_text: false,\n },\n [ContentTypeLabel.APPLEBPLIST]: {\n label: ContentTypeLabel.APPLEBPLIST,\n is_text: false,\n },\n [ContentTypeLabel.APPLEDOUBLE]: {\n label: ContentTypeLabel.APPLEDOUBLE,\n is_text: false,\n },\n [ContentTypeLabel.APPLEPLIST]: {\n label: ContentTypeLabel.APPLEPLIST,\n is_text: true,\n },\n [ContentTypeLabel.APPLESINGLE]: {\n label: ContentTypeLabel.APPLESINGLE,\n is_text: false,\n },\n [ContentTypeLabel.AR]: {\n label: ContentTypeLabel.AR,\n is_text: false,\n },\n [ContentTypeLabel.ARC]: {\n label: ContentTypeLabel.ARC,\n is_text: false,\n },\n [ContentTypeLabel.ARJ]: {\n label: ContentTypeLabel.ARJ,\n is_text: false,\n },\n [ContentTypeLabel.ARROW]: {\n label: ContentTypeLabel.ARROW,\n is_text: false,\n },\n [ContentTypeLabel.ASC]: {\n label: ContentTypeLabel.ASC,\n is_text: true,\n },\n [ContentTypeLabel.ASD]: {\n label: ContentTypeLabel.ASD,\n is_text: false,\n },\n [ContentTypeLabel.ASF]: {\n label: ContentTypeLabel.ASF,\n is_text: false,\n },\n [ContentTypeLabel.ASM]: {\n label: ContentTypeLabel.ASM,\n is_text: true,\n },\n [ContentTypeLabel.ASP]: {\n label: ContentTypeLabel.ASP,\n is_text: true,\n },\n [ContentTypeLabel.AU]: {\n label: ContentTypeLabel.AU,\n is_text: false,\n },\n [ContentTypeLabel.AUTOHOTKEY]: {\n label: ContentTypeLabel.AUTOHOTKEY,\n is_text: true,\n },\n [ContentTypeLabel.AUTOIT]: {\n label: ContentTypeLabel.AUTOIT,\n is_text: true,\n },\n [ContentTypeLabel.AVI]: {\n label: ContentTypeLabel.AVI,\n is_text: false,\n },\n [ContentTypeLabel.AVIF]: {\n label: ContentTypeLabel.AVIF,\n is_text: false,\n },\n [ContentTypeLabel.AVRO]: {\n label: ContentTypeLabel.AVRO,\n is_text: false,\n },\n [ContentTypeLabel.AWK]: {\n label: ContentTypeLabel.AWK,\n is_text: true,\n },\n [ContentTypeLabel.AX]: {\n label: ContentTypeLabel.AX,\n is_text: false,\n },\n [ContentTypeLabel.BATCH]: {\n label: ContentTypeLabel.BATCH,\n is_text: true,\n },\n [ContentTypeLabel.BAZEL]: {\n label: ContentTypeLabel.BAZEL,\n is_text: true,\n },\n [ContentTypeLabel.BCAD]: {\n label: ContentTypeLabel.BCAD,\n is_text: false,\n },\n [ContentTypeLabel.BIB]: {\n label: ContentTypeLabel.BIB,\n is_text: true,\n },\n [ContentTypeLabel.BMP]: {\n label: ContentTypeLabel.BMP,\n is_text: false,\n },\n [ContentTypeLabel.BPG]: {\n label: ContentTypeLabel.BPG,\n is_text: false,\n },\n [ContentTypeLabel.BPL]: {\n label: ContentTypeLabel.BPL,\n is_text: false,\n },\n [ContentTypeLabel.BRAINFUCK]: {\n label: ContentTypeLabel.BRAINFUCK,\n is_text: true,\n },\n [ContentTypeLabel.BRF]: {\n label: ContentTypeLabel.BRF,\n is_text: false,\n },\n [ContentTypeLabel.BZIP]: {\n label: ContentTypeLabel.BZIP,\n is_text: false,\n },\n [ContentTypeLabel.BZIP3]: {\n label: ContentTypeLabel.BZIP3,\n is_text: false,\n },\n [ContentTypeLabel.C]: {\n label: ContentTypeLabel.C,\n is_text: true,\n },\n [ContentTypeLabel.CAB]: {\n label: ContentTypeLabel.CAB,\n is_text: false,\n },\n [ContentTypeLabel.CAD]: {\n label: ContentTypeLabel.CAD,\n is_text: false,\n },\n [ContentTypeLabel.CAT]: {\n label: ContentTypeLabel.CAT,\n is_text: false,\n },\n [ContentTypeLabel.CDF]: {\n label: ContentTypeLabel.CDF,\n is_text: false,\n },\n [ContentTypeLabel.CHM]: {\n label: ContentTypeLabel.CHM,\n is_text: false,\n },\n [ContentTypeLabel.CLOJURE]: {\n label: ContentTypeLabel.CLOJURE,\n is_text: true,\n },\n [ContentTypeLabel.CMAKE]: {\n label: ContentTypeLabel.CMAKE,\n is_text: true,\n },\n [ContentTypeLabel.COBOL]: {\n label: ContentTypeLabel.COBOL,\n is_text: true,\n },\n [ContentTypeLabel.COFF]: {\n label: ContentTypeLabel.COFF,\n is_text: false,\n },\n [ContentTypeLabel.COFFEESCRIPT]: {\n label: ContentTypeLabel.COFFEESCRIPT,\n is_text: true,\n },\n [ContentTypeLabel.COM]: {\n label: ContentTypeLabel.COM,\n is_text: false,\n },\n [ContentTypeLabel.CPL]: {\n label: ContentTypeLabel.CPL,\n is_text: false,\n },\n [ContentTypeLabel.CPP]: {\n label: ContentTypeLabel.CPP,\n is_text: true,\n },\n [ContentTypeLabel.CRT]: {\n label: ContentTypeLabel.CRT,\n is_text: false,\n },\n [ContentTypeLabel.CRX]: {\n label: ContentTypeLabel.CRX,\n is_text: false,\n },\n [ContentTypeLabel.CS]: {\n label: ContentTypeLabel.CS,\n is_text: true,\n },\n [ContentTypeLabel.CSPROJ]: {\n label: ContentTypeLabel.CSPROJ,\n is_text: true,\n },\n [ContentTypeLabel.CSS]: {\n label: ContentTypeLabel.CSS,\n is_text: true,\n },\n [ContentTypeLabel.CSV]: {\n label: ContentTypeLabel.CSV,\n is_text: true,\n },\n [ContentTypeLabel.CTL]: {\n label: ContentTypeLabel.CTL,\n is_text: false,\n },\n [ContentTypeLabel.DART]: {\n label: ContentTypeLabel.DART,\n is_text: true,\n },\n [ContentTypeLabel.DEB]: {\n label: ContentTypeLabel.DEB,\n is_text: false,\n },\n [ContentTypeLabel.DEX]: {\n label: ContentTypeLabel.DEX,\n is_text: false,\n },\n [ContentTypeLabel.DEY]: {\n label: ContentTypeLabel.DEY,\n is_text: false,\n },\n [ContentTypeLabel.DICOM]: {\n label: ContentTypeLabel.DICOM,\n is_text: false,\n },\n [ContentTypeLabel.DIFF]: {\n label: ContentTypeLabel.DIFF,\n is_text: true,\n },\n [ContentTypeLabel.DIRECTORY]: {\n label: ContentTypeLabel.DIRECTORY,\n is_text: false,\n },\n [ContentTypeLabel.DJANGO]: {\n label: ContentTypeLabel.DJANGO,\n is_text: false,\n },\n [ContentTypeLabel.DLL]: {\n label: ContentTypeLabel.DLL,\n is_text: false,\n },\n [ContentTypeLabel.DM]: {\n label: ContentTypeLabel.DM,\n is_text: true,\n },\n [ContentTypeLabel.DMG]: {\n label: ContentTypeLabel.DMG,\n is_text: false,\n },\n [ContentTypeLabel.DMIGD]: {\n label: ContentTypeLabel.DMIGD,\n is_text: true,\n },\n [ContentTypeLabel.DMSCRIPT]: {\n label: ContentTypeLabel.DMSCRIPT,\n is_text: true,\n },\n [ContentTypeLabel.DOC]: {\n label: ContentTypeLabel.DOC,\n is_text: false,\n },\n [ContentTypeLabel.DOCKERFILE]: {\n label: ContentTypeLabel.DOCKERFILE,\n is_text: true,\n },\n [ContentTypeLabel.DOCX]: {\n label: ContentTypeLabel.DOCX,\n is_text: false,\n },\n [ContentTypeLabel.DOSMBR]: {\n label: ContentTypeLabel.DOSMBR,\n is_text: false,\n },\n [ContentTypeLabel.DOTX]: {\n label: ContentTypeLabel.DOTX,\n is_text: false,\n },\n [ContentTypeLabel.DSSTORE]: {\n label: ContentTypeLabel.DSSTORE,\n is_text: false,\n },\n [ContentTypeLabel.DWG]: {\n label: ContentTypeLabel.DWG,\n is_text: false,\n },\n [ContentTypeLabel.DXF]: {\n label: ContentTypeLabel.DXF,\n is_text: true,\n },\n [ContentTypeLabel.DYLIB]: {\n label: ContentTypeLabel.DYLIB,\n is_text: false,\n },\n [ContentTypeLabel.EBML]: {\n label: ContentTypeLabel.EBML,\n is_text: false,\n },\n [ContentTypeLabel.ELF]: {\n label: ContentTypeLabel.ELF,\n is_text: false,\n },\n [ContentTypeLabel.ELIXIR]: {\n label: ContentTypeLabel.ELIXIR,\n is_text: true,\n },\n [ContentTypeLabel.EMF]: {\n label: ContentTypeLabel.EMF,\n is_text: false,\n },\n [ContentTypeLabel.EML]: {\n label: ContentTypeLabel.EML,\n is_text: true,\n },\n [ContentTypeLabel.EMPTY]: {\n label: ContentTypeLabel.EMPTY,\n is_text: false,\n },\n [ContentTypeLabel.EPUB]: {\n label: ContentTypeLabel.EPUB,\n is_text: false,\n },\n [ContentTypeLabel.ERB]: {\n label: ContentTypeLabel.ERB,\n is_text: true,\n },\n [ContentTypeLabel.ERLANG]: {\n label: ContentTypeLabel.ERLANG,\n is_text: true,\n },\n [ContentTypeLabel.ESE]: {\n label: ContentTypeLabel.ESE,\n is_text: false,\n },\n [ContentTypeLabel.EXE]: {\n label: ContentTypeLabel.EXE,\n is_text: false,\n },\n [ContentTypeLabel.EXP]: {\n label: ContentTypeLabel.EXP,\n is_text: false,\n },\n [ContentTypeLabel.FLAC]: {\n label: ContentTypeLabel.FLAC,\n is_text: false,\n },\n [ContentTypeLabel.FLUTTER]: {\n label: ContentTypeLabel.FLUTTER,\n is_text: false,\n },\n [ContentTypeLabel.FLV]: {\n label: ContentTypeLabel.FLV,\n is_text: false,\n },\n [ContentTypeLabel.FORTRAN]: {\n label: ContentTypeLabel.FORTRAN,\n is_text: true,\n },\n [ContentTypeLabel.FPX]: {\n label: ContentTypeLabel.FPX,\n is_text: false,\n },\n [ContentTypeLabel.GEMFILE]: {\n label: ContentTypeLabel.GEMFILE,\n is_text: true,\n },\n [ContentTypeLabel.GEMSPEC]: {\n label: ContentTypeLabel.GEMSPEC,\n is_text: true,\n },\n [ContentTypeLabel.GIF]: {\n label: ContentTypeLabel.GIF,\n is_text: false,\n },\n [ContentTypeLabel.GITATTRIBUTES]: {\n label: ContentTypeLabel.GITATTRIBUTES,\n is_text: true,\n },\n [ContentTypeLabel.GITMODULES]: {\n label: ContentTypeLabel.GITMODULES,\n is_text: true,\n },\n [ContentTypeLabel.GLEAM]: {\n label: ContentTypeLabel.GLEAM,\n is_text: true,\n },\n [ContentTypeLabel.GO]: {\n label: ContentTypeLabel.GO,\n is_text: true,\n },\n [ContentTypeLabel.GPX]: {\n label: ContentTypeLabel.GPX,\n is_text: false,\n },\n [ContentTypeLabel.GRADLE]: {\n label: ContentTypeLabel.GRADLE,\n is_text: true,\n },\n [ContentTypeLabel.GROOVY]: {\n label: ContentTypeLabel.GROOVY,\n is_text: true,\n },\n [ContentTypeLabel.GZIP]: {\n label: ContentTypeLabel.GZIP,\n is_text: false,\n },\n [ContentTypeLabel.H]: {\n label: ContentTypeLabel.H,\n is_text: true,\n },\n [ContentTypeLabel.H5]: {\n label: ContentTypeLabel.H5,\n is_text: false,\n },\n [ContentTypeLabel.HANDLEBARS]: {\n label: ContentTypeLabel.HANDLEBARS,\n is_text: true,\n },\n [ContentTypeLabel.HASKELL]: {\n label: ContentTypeLabel.HASKELL,\n is_text: true,\n },\n [ContentTypeLabel.HCL]: {\n label: ContentTypeLabel.HCL,\n is_text: true,\n },\n [ContentTypeLabel.HEIF]: {\n label: ContentTypeLabel.HEIF,\n is_text: false,\n },\n [ContentTypeLabel.HFS]: {\n label: ContentTypeLabel.HFS,\n is_text: false,\n },\n [ContentTypeLabel.HLP]: {\n label: ContentTypeLabel.HLP,\n is_text: false,\n },\n [ContentTypeLabel.HPP]: {\n label: ContentTypeLabel.HPP,\n is_text: true,\n },\n [ContentTypeLabel.HTA]: {\n label: ContentTypeLabel.HTA,\n is_text: false,\n },\n [ContentTypeLabel.HTACCESS]: {\n label: ContentTypeLabel.HTACCESS,\n is_text: true,\n },\n [ContentTypeLabel.HTML]: {\n label: ContentTypeLabel.HTML,\n is_text: true,\n },\n [ContentTypeLabel.HVE]: {\n label: ContentTypeLabel.HVE,\n is_text: false,\n },\n [ContentTypeLabel.HWP]: {\n label: ContentTypeLabel.HWP,\n is_text: false,\n },\n [ContentTypeLabel.ICC]: {\n label: ContentTypeLabel.ICC,\n is_text: false,\n },\n [ContentTypeLabel.ICNS]: {\n label: ContentTypeLabel.ICNS,\n is_text: false,\n },\n [ContentTypeLabel.ICO]: {\n label: ContentTypeLabel.ICO,\n is_text: false,\n },\n [ContentTypeLabel.ICS]: {\n label: ContentTypeLabel.ICS,\n is_text: true,\n },\n [ContentTypeLabel.IGNOREFILE]: {\n label: ContentTypeLabel.IGNOREFILE,\n is_text: true,\n },\n [ContentTypeLabel.IMG]: {\n label: ContentTypeLabel.IMG,\n is_text: false,\n },\n [ContentTypeLabel.INI]: {\n label: ContentTypeLabel.INI,\n is_text: true,\n },\n [ContentTypeLabel.INTERNETSHORTCUT]: {\n label: ContentTypeLabel.INTERNETSHORTCUT,\n is_text: true,\n },\n [ContentTypeLabel.IOSAPP]: {\n label: ContentTypeLabel.IOSAPP,\n is_text: false,\n },\n [ContentTypeLabel.IPYNB]: {\n label: ContentTypeLabel.IPYNB,\n is_text: true,\n },\n [ContentTypeLabel.ISO]: {\n label: ContentTypeLabel.ISO,\n is_text: false,\n },\n [ContentTypeLabel.JAR]: {\n label: ContentTypeLabel.JAR,\n is_text: false,\n },\n [ContentTypeLabel.JAVA]: {\n label: ContentTypeLabel.JAVA,\n is_text: true,\n },\n [ContentTypeLabel.JAVABYTECODE]: {\n label: ContentTypeLabel.JAVABYTECODE,\n is_text: false,\n },\n [ContentTypeLabel.JAVASCRIPT]: {\n label: ContentTypeLabel.JAVASCRIPT,\n is_text: true,\n },\n [ContentTypeLabel.JINJA]: {\n label: ContentTypeLabel.JINJA,\n is_text: true,\n },\n [ContentTypeLabel.JNG]: {\n label: ContentTypeLabel.JNG,\n is_text: false,\n },\n [ContentTypeLabel.JNLP]: {\n label: ContentTypeLabel.JNLP,\n is_text: true,\n },\n [ContentTypeLabel.JP2]: {\n label: ContentTypeLabel.JP2,\n is_text: false,\n },\n [ContentTypeLabel.JPEG]: {\n label: ContentTypeLabel.JPEG,\n is_text: false,\n },\n [ContentTypeLabel.JSON]: {\n label: ContentTypeLabel.JSON,\n is_text: true,\n },\n [ContentTypeLabel.JSONC]: {\n label: ContentTypeLabel.JSONC,\n is_text: false,\n },\n [ContentTypeLabel.JSONL]: {\n label: ContentTypeLabel.JSONL,\n is_text: true,\n },\n [ContentTypeLabel.JSX]: {\n label: ContentTypeLabel.JSX,\n is_text: true,\n },\n [ContentTypeLabel.JULIA]: {\n label: ContentTypeLabel.JULIA,\n is_text: true,\n },\n [ContentTypeLabel.JXL]: {\n label: ContentTypeLabel.JXL,\n is_text: false,\n },\n [ContentTypeLabel.KO]: {\n label: ContentTypeLabel.KO,\n is_text: false,\n },\n [ContentTypeLabel.KOTLIN]: {\n label: ContentTypeLabel.KOTLIN,\n is_text: true,\n },\n [ContentTypeLabel.KS]: {\n label: ContentTypeLabel.KS,\n is_text: true,\n },\n [ContentTypeLabel.LATEX]: {\n label: ContentTypeLabel.LATEX,\n is_text: true,\n },\n [ContentTypeLabel.LATEXAUX]: {\n label: ContentTypeLabel.LATEXAUX,\n is_text: false,\n },\n [ContentTypeLabel.LESS]: {\n label: ContentTypeLabel.LESS,\n is_text: false,\n },\n [ContentTypeLabel.LHA]: {\n label: ContentTypeLabel.LHA,\n is_text: false,\n },\n [ContentTypeLabel.LICENSE]: {\n label: ContentTypeLabel.LICENSE,\n is_text: true,\n },\n [ContentTypeLabel.LISP]: {\n label: ContentTypeLabel.LISP,\n is_text: true,\n },\n [ContentTypeLabel.LITCS]: {\n label: ContentTypeLabel.LITCS,\n is_text: false,\n },\n [ContentTypeLabel.LNK]: {\n label: ContentTypeLabel.LNK,\n is_text: false,\n },\n [ContentTypeLabel.LOCK]: {\n label: ContentTypeLabel.LOCK,\n is_text: true,\n },\n [ContentTypeLabel.LRZ]: {\n label: ContentTypeLabel.LRZ,\n is_text: false,\n },\n [ContentTypeLabel.LUA]: {\n label: ContentTypeLabel.LUA,\n is_text: true,\n },\n [ContentTypeLabel.LZ]: {\n label: ContentTypeLabel.LZ,\n is_text: false,\n },\n [ContentTypeLabel.LZ4]: {\n label: ContentTypeLabel.LZ4,\n is_text: false,\n },\n [ContentTypeLabel.LZX]: {\n label: ContentTypeLabel.LZX,\n is_text: false,\n },\n [ContentTypeLabel.M3U]: {\n label: ContentTypeLabel.M3U,\n is_text: true,\n },\n [ContentTypeLabel.M4]: {\n label: ContentTypeLabel.M4,\n is_text: true,\n },\n [ContentTypeLabel.MACHO]: {\n label: ContentTypeLabel.MACHO,\n is_text: false,\n },\n [ContentTypeLabel.MAFF]: {\n label: ContentTypeLabel.MAFF,\n is_text: false,\n },\n [ContentTypeLabel.MAKEFILE]: {\n label: ContentTypeLabel.MAKEFILE,\n is_text: true,\n },\n [ContentTypeLabel.MARKDOWN]: {\n label: ContentTypeLabel.MARKDOWN,\n is_text: true,\n },\n [ContentTypeLabel.MATLAB]: {\n label: ContentTypeLabel.MATLAB,\n is_text: true,\n },\n [ContentTypeLabel.MHT]: {\n label: ContentTypeLabel.MHT,\n is_text: true,\n },\n [ContentTypeLabel.MIDI]: {\n label: ContentTypeLabel.MIDI,\n is_text: false,\n },\n [ContentTypeLabel.MKV]: {\n label: ContentTypeLabel.MKV,\n is_text: false,\n },\n [ContentTypeLabel.MP2]: {\n label: ContentTypeLabel.MP2,\n is_text: false,\n },\n [ContentTypeLabel.MP3]: {\n label: ContentTypeLabel.MP3,\n is_text: false,\n },\n [ContentTypeLabel.MP4]: {\n label: ContentTypeLabel.MP4,\n is_text: false,\n },\n [ContentTypeLabel.MPEGTS]: {\n label: ContentTypeLabel.MPEGTS,\n is_text: false,\n },\n [ContentTypeLabel.MSCOMPRESS]: {\n label: ContentTypeLabel.MSCOMPRESS,\n is_text: false,\n },\n [ContentTypeLabel.MSI]: {\n label: ContentTypeLabel.MSI,\n is_text: false,\n },\n [ContentTypeLabel.MSIX]: {\n label: ContentTypeLabel.MSIX,\n is_text: false,\n },\n [ContentTypeLabel.MST]: {\n label: ContentTypeLabel.MST,\n is_text: false,\n },\n [ContentTypeLabel.MUI]: {\n label: ContentTypeLabel.MUI,\n is_text: false,\n },\n [ContentTypeLabel.MUM]: {\n label: ContentTypeLabel.MUM,\n is_text: true,\n },\n [ContentTypeLabel.MUN]: {\n label: ContentTypeLabel.MUN,\n is_text: false,\n },\n [ContentTypeLabel.NIM]: {\n label: ContentTypeLabel.NIM,\n is_text: false,\n },\n [ContentTypeLabel.NPY]: {\n label: ContentTypeLabel.NPY,\n is_text: false,\n },\n [ContentTypeLabel.NPZ]: {\n label: ContentTypeLabel.NPZ,\n is_text: false,\n },\n [ContentTypeLabel.NULL]: {\n label: ContentTypeLabel.NULL,\n is_text: false,\n },\n [ContentTypeLabel.NUPKG]: {\n label: ContentTypeLabel.NUPKG,\n is_text: false,\n },\n [ContentTypeLabel.OBJECT]: {\n label: ContentTypeLabel.OBJECT,\n is_text: false,\n },\n [ContentTypeLabel.OBJECTIVEC]: {\n label: ContentTypeLabel.OBJECTIVEC,\n is_text: true,\n },\n [ContentTypeLabel.OCAML]: {\n label: ContentTypeLabel.OCAML,\n is_text: true,\n },\n [ContentTypeLabel.OCX]: {\n label: ContentTypeLabel.OCX,\n is_text: false,\n },\n [ContentTypeLabel.ODEX]: {\n label: ContentTypeLabel.ODEX,\n is_text: false,\n },\n [ContentTypeLabel.ODIN]: {\n label: ContentTypeLabel.ODIN,\n is_text: true,\n },\n [ContentTypeLabel.ODP]: {\n label: ContentTypeLabel.ODP,\n is_text: false,\n },\n [ContentTypeLabel.ODS]: {\n label: ContentTypeLabel.ODS,\n is_text: false,\n },\n [ContentTypeLabel.ODT]: {\n label: ContentTypeLabel.ODT,\n is_text: false,\n },\n [ContentTypeLabel.OGG]: {\n label: ContentTypeLabel.OGG,\n is_text: false,\n },\n [ContentTypeLabel.OLE]: {\n label: ContentTypeLabel.OLE,\n is_text: false,\n },\n [ContentTypeLabel.ONE]: {\n label: ContentTypeLabel.ONE,\n is_text: false,\n },\n [ContentTypeLabel.ONNX]: {\n label: ContentTypeLabel.ONNX,\n is_text: false,\n },\n [ContentTypeLabel.OOXML]: {\n label: ContentTypeLabel.OOXML,\n is_text: false,\n },\n [ContentTypeLabel.OTF]: {\n label: ContentTypeLabel.OTF,\n is_text: false,\n },\n [ContentTypeLabel.OUTLOOK]: {\n label: ContentTypeLabel.OUTLOOK,\n is_text: false,\n },\n [ContentTypeLabel.PALMOS]: {\n label: ContentTypeLabel.PALMOS,\n is_text: false,\n },\n [ContentTypeLabel.PARQUET]: {\n label: ContentTypeLabel.PARQUET,\n is_text: false,\n },\n [ContentTypeLabel.PASCAL]: {\n label: ContentTypeLabel.PASCAL,\n is_text: true,\n },\n [ContentTypeLabel.PBM]: {\n label: ContentTypeLabel.PBM,\n is_text: false,\n },\n [ContentTypeLabel.PCAP]: {\n label: ContentTypeLabel.PCAP,\n is_text: false,\n },\n [ContentTypeLabel.PDB]: {\n label: ContentTypeLabel.PDB,\n is_text: false,\n },\n [ContentTypeLabel.PDF]: {\n label: ContentTypeLabel.PDF,\n is_text: false,\n },\n [ContentTypeLabel.PEBIN]: {\n label: ContentTypeLabel.PEBIN,\n is_text: false,\n },\n [ContentTypeLabel.PEM]: {\n label: ContentTypeLabel.PEM,\n is_text: true,\n },\n [ContentTypeLabel.PERL]: {\n label: ContentTypeLabel.PERL,\n is_text: true,\n },\n [ContentTypeLabel.PGP]: {\n label: ContentTypeLabel.PGP,\n is_text: false,\n },\n [ContentTypeLabel.PHP]: {\n label: ContentTypeLabel.PHP,\n is_text: true,\n },\n [ContentTypeLabel.PICKLE]: {\n label: ContentTypeLabel.PICKLE,\n is_text: false,\n },\n [ContentTypeLabel.PNG]: {\n label: ContentTypeLabel.PNG,\n is_text: false,\n },\n [ContentTypeLabel.PO]: {\n label: ContentTypeLabel.PO,\n is_text: true,\n },\n [ContentTypeLabel.POSTSCRIPT]: {\n label: ContentTypeLabel.POSTSCRIPT,\n is_text: false,\n },\n [ContentTypeLabel.POWERSHELL]: {\n label: ContentTypeLabel.POWERSHELL,\n is_text: true,\n },\n [ContentTypeLabel.PPT]: {\n label: ContentTypeLabel.PPT,\n is_text: false,\n },\n [ContentTypeLabel.PPTX]: {\n label: ContentTypeLabel.PPTX,\n is_text: false,\n },\n [ContentTypeLabel.PRINTFOX]: {\n label: ContentTypeLabel.PRINTFOX,\n is_text: false,\n },\n [ContentTypeLabel.PROLOG]: {\n label: ContentTypeLabel.PROLOG,\n is_text: true,\n },\n [ContentTypeLabel.PROTEINDB]: {\n label: ContentTypeLabel.PROTEINDB,\n is_text: true,\n },\n [ContentTypeLabel.PROTO]: {\n label: ContentTypeLabel.PROTO,\n is_text: true,\n },\n [ContentTypeLabel.PROTOBUF]: {\n label: ContentTypeLabel.PROTOBUF,\n is_text: false,\n },\n [ContentTypeLabel.PSD]: {\n label: ContentTypeLabel.PSD,\n is_text: false,\n },\n [ContentTypeLabel.PUB]: {\n label: ContentTypeLabel.PUB,\n is_text: false,\n },\n [ContentTypeLabel.PYTHON]: {\n label: ContentTypeLabel.PYTHON,\n is_text: true,\n },\n [ContentTypeLabel.PYTHONBYTECODE]: {\n label: ContentTypeLabel.PYTHONBYTECODE,\n is_text: false,\n },\n [ContentTypeLabel.PYTHONPAR]: {\n label: ContentTypeLabel.PYTHONPAR,\n is_text: false,\n },\n [ContentTypeLabel.PYTORCH]: {\n label: ContentTypeLabel.PYTORCH,\n is_text: false,\n },\n [ContentTypeLabel.QOI]: {\n label: ContentTypeLabel.QOI,\n is_text: false,\n },\n [ContentTypeLabel.QT]: {\n label: ContentTypeLabel.QT,\n is_text: false,\n },\n [ContentTypeLabel.R]: {\n label: ContentTypeLabel.R,\n is_text: true,\n },\n [ContentTypeLabel.RANDOMASCII]: {\n label: ContentTypeLabel.RANDOMASCII,\n is_text: true,\n },\n [ContentTypeLabel.RANDOMBYTES]: {\n label: ContentTypeLabel.RANDOMBYTES,\n is_text: false,\n },\n [ContentTypeLabel.RANDOMTXT]: {\n label: ContentTypeLabel.RANDOMTXT,\n is_text: true,\n },\n [ContentTypeLabel.RAR]: {\n label: ContentTypeLabel.RAR,\n is_text: false,\n },\n [ContentTypeLabel.RDF]: {\n label: ContentTypeLabel.RDF,\n is_text: true,\n },\n [ContentTypeLabel.RDP]: {\n label: ContentTypeLabel.RDP,\n is_text: false,\n },\n [ContentTypeLabel.RIFF]: {\n label: ContentTypeLabel.RIFF,\n is_text: false,\n },\n [ContentTypeLabel.RLIB]: {\n label: ContentTypeLabel.RLIB,\n is_text: false,\n },\n [ContentTypeLabel.RLL]: {\n label: ContentTypeLabel.RLL,\n is_text: false,\n },\n [ContentTypeLabel.RPM]: {\n label: ContentTypeLabel.RPM,\n is_text: false,\n },\n [ContentTypeLabel.RST]: {\n label: ContentTypeLabel.RST,\n is_text: true,\n },\n [ContentTypeLabel.RTF]: {\n label: ContentTypeLabel.RTF,\n is_text: true,\n },\n [ContentTypeLabel.RUBY]: {\n label: ContentTypeLabel.RUBY,\n is_text: true,\n },\n [ContentTypeLabel.RUST]: {\n label: ContentTypeLabel.RUST,\n is_text: true,\n },\n [ContentTypeLabel.RZIP]: {\n label: ContentTypeLabel.RZIP,\n is_text: false,\n },\n [ContentTypeLabel.SCALA]: {\n label: ContentTypeLabel.SCALA,\n is_text: true,\n },\n [ContentTypeLabel.SCHEME]: {\n label: ContentTypeLabel.SCHEME,\n is_text: false,\n },\n [ContentTypeLabel.SCR]: {\n label: ContentTypeLabel.SCR,\n is_text: false,\n },\n [ContentTypeLabel.SCRIPTWSF]: {\n label: ContentTypeLabel.SCRIPTWSF,\n is_text: false,\n },\n [ContentTypeLabel.SCSS]: {\n label: ContentTypeLabel.SCSS,\n is_text: true,\n },\n [ContentTypeLabel.SEVENZIP]: {\n label: ContentTypeLabel.SEVENZIP,\n is_text: false,\n },\n [ContentTypeLabel.SGML]: {\n label: ContentTypeLabel.SGML,\n is_text: true,\n },\n [ContentTypeLabel.SH3D]: {\n label: ContentTypeLabel.SH3D,\n is_text: false,\n },\n [ContentTypeLabel.SHELL]: {\n label: ContentTypeLabel.SHELL,\n is_text: true,\n },\n [ContentTypeLabel.SMALI]: {\n label: ContentTypeLabel.SMALI,\n is_text: true,\n },\n [ContentTypeLabel.SNAP]: {\n label: ContentTypeLabel.SNAP,\n is_text: false,\n },\n [ContentTypeLabel.SO]: {\n label: ContentTypeLabel.SO,\n is_text: false,\n },\n [ContentTypeLabel.SOLIDITY]: {\n label: ContentTypeLabel.SOLIDITY,\n is_text: true,\n },\n [ContentTypeLabel.SQL]: {\n label: ContentTypeLabel.SQL,\n is_text: true,\n },\n [ContentTypeLabel.SQLITE]: {\n label: ContentTypeLabel.SQLITE,\n is_text: false,\n },\n [ContentTypeLabel.SQUASHFS]: {\n label: ContentTypeLabel.SQUASHFS,\n is_text: false,\n },\n [ContentTypeLabel.SRT]: {\n label: ContentTypeLabel.SRT,\n is_text: true,\n },\n [ContentTypeLabel.STLBINARY]: {\n label: ContentTypeLabel.STLBINARY,\n is_text: false,\n },\n [ContentTypeLabel.STLTEXT]: {\n label: ContentTypeLabel.STLTEXT,\n is_text: true,\n },\n [ContentTypeLabel.SUM]: {\n label: ContentTypeLabel.SUM,\n is_text: true,\n },\n [ContentTypeLabel.SVD]: {\n label: ContentTypeLabel.SVD,\n is_text: false,\n },\n [ContentTypeLabel.SVG]: {\n label: ContentTypeLabel.SVG,\n is_text: true,\n },\n [ContentTypeLabel.SWF]: {\n label: ContentTypeLabel.SWF,\n is_text: false,\n },\n [ContentTypeLabel.SWIFT]: {\n label: ContentTypeLabel.SWIFT,\n is_text: true,\n },\n [ContentTypeLabel.SYMLINK]: {\n label: ContentTypeLabel.SYMLINK,\n is_text: false,\n },\n [ContentTypeLabel.SYMLINKTEXT]: {\n label: ContentTypeLabel.SYMLINKTEXT,\n is_text: true,\n },\n [ContentTypeLabel.SYS]: {\n label: ContentTypeLabel.SYS,\n is_text: false,\n },\n [ContentTypeLabel.TAR]: {\n label: ContentTypeLabel.TAR,\n is_text: false,\n },\n [ContentTypeLabel.TCL]: {\n label: ContentTypeLabel.TCL,\n is_text: true,\n },\n [ContentTypeLabel.TEXTPROTO]: {\n label: ContentTypeLabel.TEXTPROTO,\n is_text: true,\n },\n [ContentTypeLabel.TGA]: {\n label: ContentTypeLabel.TGA,\n is_text: false,\n },\n [ContentTypeLabel.THUMBSDB]: {\n label: ContentTypeLabel.THUMBSDB,\n is_text: false,\n },\n [ContentTypeLabel.TIFF]: {\n label: ContentTypeLabel.TIFF,\n is_text: false,\n },\n [ContentTypeLabel.TMDX]: {\n label: ContentTypeLabel.TMDX,\n is_text: false,\n },\n [ContentTypeLabel.TOML]: {\n label: ContentTypeLabel.TOML,\n is_text: true,\n },\n [ContentTypeLabel.TORRENT]: {\n label: ContentTypeLabel.TORRENT,\n is_text: false,\n },\n [ContentTypeLabel.TROFF]: {\n label: ContentTypeLabel.TROFF,\n is_text: false,\n },\n [ContentTypeLabel.TSV]: {\n label: ContentTypeLabel.TSV,\n is_text: true,\n },\n [ContentTypeLabel.TSX]: {\n label: ContentTypeLabel.TSX,\n is_text: true,\n },\n [ContentTypeLabel.TTF]: {\n label: ContentTypeLabel.TTF,\n is_text: false,\n },\n [ContentTypeLabel.TWIG]: {\n label: ContentTypeLabel.TWIG,\n is_text: true,\n },\n [ContentTypeLabel.TXT]: {\n label: ContentTypeLabel.TXT,\n is_text: true,\n },\n [ContentTypeLabel.TXTASCII]: {\n label: ContentTypeLabel.TXTASCII,\n is_text: true,\n },\n [ContentTypeLabel.TXTUTF16]: {\n label: ContentTypeLabel.TXTUTF16,\n is_text: true,\n },\n [ContentTypeLabel.TXTUTF8]: {\n label: ContentTypeLabel.TXTUTF8,\n is_text: true,\n },\n [ContentTypeLabel.TYPESCRIPT]: {\n label: ContentTypeLabel.TYPESCRIPT,\n is_text: true,\n },\n [ContentTypeLabel.UDF]: {\n label: ContentTypeLabel.UDF,\n is_text: false,\n },\n [ContentTypeLabel.UNDEFINED]: {\n label: ContentTypeLabel.UNDEFINED,\n is_text: false,\n },\n [ContentTypeLabel.UNIXCOMPRESS]: {\n label: ContentTypeLabel.UNIXCOMPRESS,\n is_text: false,\n },\n [ContentTypeLabel.UNKNOWN]: {\n label: ContentTypeLabel.UNKNOWN,\n is_text: false,\n },\n [ContentTypeLabel.VBA]: {\n label: ContentTypeLabel.VBA,\n is_text: true,\n },\n [ContentTypeLabel.VBE]: {\n label: ContentTypeLabel.VBE,\n is_text: false,\n },\n [ContentTypeLabel.VCARD]: {\n label: ContentTypeLabel.VCARD,\n is_text: false,\n },\n [ContentTypeLabel.VCS]: {\n label: ContentTypeLabel.VCS,\n is_text: false,\n },\n [ContentTypeLabel.VCXPROJ]: {\n label: ContentTypeLabel.VCXPROJ,\n is_text: true,\n },\n [ContentTypeLabel.VERILOG]: {\n label: ContentTypeLabel.VERILOG,\n is_text: true,\n },\n [ContentTypeLabel.VHD]: {\n label: ContentTypeLabel.VHD,\n is_text: false,\n },\n [ContentTypeLabel.VHDL]: {\n label: ContentTypeLabel.VHDL,\n is_text: true,\n },\n [ContentTypeLabel.VISIO]: {\n label: ContentTypeLabel.VISIO,\n is_text: false,\n },\n [ContentTypeLabel.VTT]: {\n label: ContentTypeLabel.VTT,\n is_text: true,\n },\n [ContentTypeLabel.VUE]: {\n label: ContentTypeLabel.VUE,\n is_text: true,\n },\n [ContentTypeLabel.WAD]: {\n label: ContentTypeLabel.WAD,\n is_text: false,\n },\n [ContentTypeLabel.WASM]: {\n label: ContentTypeLabel.WASM,\n is_text: false,\n },\n [ContentTypeLabel.WAV]: {\n label: ContentTypeLabel.WAV,\n is_text: false,\n },\n [ContentTypeLabel.WEBM]: {\n label: ContentTypeLabel.WEBM,\n is_text: false,\n },\n [ContentTypeLabel.WEBP]: {\n label: ContentTypeLabel.WEBP,\n is_text: false,\n },\n [ContentTypeLabel.WEBTEMPLATE]: {\n label: ContentTypeLabel.WEBTEMPLATE,\n is_text: true,\n },\n [ContentTypeLabel.WIM]: {\n label: ContentTypeLabel.WIM,\n is_text: false,\n },\n [ContentTypeLabel.WINREGISTRY]: {\n label: ContentTypeLabel.WINREGISTRY,\n is_text: true,\n },\n [ContentTypeLabel.WMA]: {\n label: ContentTypeLabel.WMA,\n is_text: false,\n },\n [ContentTypeLabel.WMF]: {\n label: ContentTypeLabel.WMF,\n is_text: false,\n },\n [ContentTypeLabel.WMV]: {\n label: ContentTypeLabel.WMV,\n is_text: false,\n },\n [ContentTypeLabel.WOFF]: {\n label: ContentTypeLabel.WOFF,\n is_text: false,\n },\n [ContentTypeLabel.WOFF2]: {\n label: ContentTypeLabel.WOFF2,\n is_text: false,\n },\n [ContentTypeLabel.XAR]: {\n label: ContentTypeLabel.XAR,\n is_text: false,\n },\n [ContentTypeLabel.XCF]: {\n label: ContentTypeLabel.XCF,\n is_text: false,\n },\n [ContentTypeLabel.XLS]: {\n label: ContentTypeLabel.XLS,\n is_text: false,\n },\n [ContentTypeLabel.XLSB]: {\n label: ContentTypeLabel.XLSB,\n is_text: false,\n },\n [ContentTypeLabel.XLSX]: {\n label: ContentTypeLabel.XLSX,\n is_text: false,\n },\n [ContentTypeLabel.XML]: {\n label: ContentTypeLabel.XML,\n is_text: true,\n },\n [ContentTypeLabel.XPI]: {\n label: ContentTypeLabel.XPI,\n is_text: false,\n },\n [ContentTypeLabel.XSD]: {\n label: ContentTypeLabel.XSD,\n is_text: false,\n },\n [ContentTypeLabel.XZ]: {\n label: ContentTypeLabel.XZ,\n is_text: false,\n },\n [ContentTypeLabel.YAML]: {\n label: ContentTypeLabel.YAML,\n is_text: true,\n },\n [ContentTypeLabel.YARA]: {\n label: ContentTypeLabel.YARA,\n is_text: true,\n },\n [ContentTypeLabel.ZIG]: {\n label: ContentTypeLabel.ZIG,\n is_text: true,\n },\n [ContentTypeLabel.ZIP]: {\n label: ContentTypeLabel.ZIP,\n is_text: false,\n },\n [ContentTypeLabel.ZLIBSTREAM]: {\n label: ContentTypeLabel.ZLIBSTREAM,\n is_text: false,\n },\n [ContentTypeLabel.ZST]: {\n label: ContentTypeLabel.ZST,\n is_text: false,\n },\n }),\n};\n" }, { "path": "js/src/magika-options.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nexport interface MagikaOptions {\n modelURL?: string;\n modelPath?: string;\n modelConfigURL?: string;\n modelConfigPath?: string;\n}\n" }, { "path": "js/src/magika-prediction.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { ContentTypeInfo } from \"./content-type-info.js\";\nimport { ContentTypeLabel } from \"./content-type-label.js\";\nimport { OverwriteReason } from \"./overwrite-reason.js\";\n\nexport interface MagikaPrediction {\n dl: ContentTypeInfo;\n output: ContentTypeInfo;\n score: number;\n overwrite_reason: OverwriteReason;\n scores_map?: Partial>;\n}\n" }, { "path": "js/src/magika-result.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { MagikaPrediction } from \"./magika-prediction.js\";\nimport { Status } from \"./status.js\";\n\nexport interface MagikaResult {\n path: string;\n status: Status;\n prediction: MagikaPrediction;\n}\n" }, { "path": "js/src/model-config-node.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport * as fs from \"fs/promises\";\nimport { ModelConfig } from \"./model-config.js\";\n\nexport class ModelConfigNode extends ModelConfig {\n async loadFile(configPath: string): Promise {\n if (this.loaded) {\n return;\n }\n const config = JSON.parse((await fs.readFile(configPath)).toString());\n this.setConfig(config);\n this.loaded = true;\n }\n}\n" }, { "path": "js/src/model-config.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { ContentTypeLabel } from \"./content-type-label.js\";\n\nexport class ModelConfig {\n beg_size: number = 0;\n mid_size: number = 0;\n end_size: number = 0;\n use_inputs_at_offsets: boolean = false;\n medium_confidence_threshold: number = 0;\n min_file_size_for_dl: number = 0;\n padding_token: number = -1;\n block_size: number = 0;\n target_labels_space: ContentTypeLabel[] = [];\n thresholds: Partial> = {};\n overwrite_map: Partial> = {};\n loaded: boolean = false;\n\n async loadUrl(configURL: string): Promise {\n if (this.loaded) {\n return;\n }\n const config = (await (await fetch(configURL)).json()) as Record<\n string,\n any\n >;\n this.setConfig(config);\n this.loaded = true;\n }\n\n protected setConfig(config: Record): void {\n this.beg_size = config.beg_size;\n this.mid_size = config.mid_size;\n this.end_size = config.end_size;\n this.use_inputs_at_offsets = config.use_inputs_at_offsets;\n this.medium_confidence_threshold = config.medium_confidence_threshold;\n this.min_file_size_for_dl = config.min_file_size_for_dl;\n this.padding_token = config.padding_token;\n this.block_size = config.block_size;\n this.target_labels_space = [];\n for (const label of config.target_labels_space as string[]) {\n this.target_labels_space.push(label as ContentTypeLabel);\n }\n for (const [label, th] of Object.entries(\n config.thresholds as Record,\n )) {\n this.thresholds[label as ContentTypeLabel] = th;\n }\n for (const [label, target_label] of Object.entries(\n config.overwrite_map as Record,\n )) {\n this.overwrite_map[label as ContentTypeLabel] =\n target_label as ContentTypeLabel;\n }\n\n if (\n !(\n this.beg_size > 0 &&\n this.mid_size === 0 &&\n this.end_size > 0 &&\n !this.use_inputs_at_offsets &&\n this.medium_confidence_threshold > 0 &&\n this.min_file_size_for_dl > 0 &&\n this.padding_token != -1 &&\n this.block_size > 0\n )\n ) {\n throw new Error(\"Invalid config\");\n }\n }\n}\n" }, { "path": "js/src/model-features.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nexport class ModelFeatures {\n beg_ints: Uint16Array;\n end_ints: Uint16Array;\n locked: { beg: boolean; end: boolean };\n\n constructor(\n beg_size: number,\n mid_size: number,\n end_size: number,\n padding_token: number,\n use_inputs_at_offsets: boolean,\n ) {\n if (mid_size != 0) {\n throw new Error(\n `Assertion failed: This implementation does not support mid_size (${mid_size}) != 0 model config.`,\n );\n }\n if (use_inputs_at_offsets) {\n throw new Error(\n `Assertion failed: This implementation does not support use_inputs_at_offsets = true model config.`,\n );\n }\n\n this.beg_ints = new Uint16Array(beg_size).fill(padding_token);\n this.end_ints = new Uint16Array(end_size).fill(padding_token);\n this.locked = { beg: false, end: false };\n }\n\n withStart(data: Uint8Array, offset: number): this {\n if (!this.locked.beg) {\n this.locked.beg = true;\n this.beg_ints.set(data, offset);\n }\n return this;\n }\n\n withEnd(data: Uint8Array, offset: number): this {\n if (!this.locked.end) {\n this.locked.end = true;\n this.end_ints.set(data, offset);\n }\n return this;\n }\n\n toArray(): number[] {\n return [...this.beg_ints, ...this.end_ints];\n }\n}\n" }, { "path": "js/src/model-node.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport * as tf from \"@tensorflow/tfjs\";\nimport * as tfn from \"@tensorflow/tfjs-node\";\nimport { Model } from \"./model.js\";\n\nexport class ModelNode extends Model {\n async loadFile(modelPath: string): Promise {\n if (!this.model) {\n const handler = tfn.io.fileSystem(modelPath);\n this.model = await tf.loadGraphModel(handler);\n }\n }\n}\n" }, { "path": "js/src/model-prediction.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { ContentTypeLabel } from \"./content-type-label.js\";\n\nexport interface ModelPrediction {\n label: ContentTypeLabel;\n score: number;\n scores_map: Partial>;\n}\n" }, { "path": "js/src/model.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport * as tf from \"@tensorflow/tfjs\";\nimport { GraphModel } from \"@tensorflow/tfjs\";\nimport { ModelConfig } from \"./model-config.js\";\nimport { ModelPrediction } from \"./model-prediction.js\";\nimport { ModelFeatures } from \"./model-features.js\";\nimport { ContentTypeLabel } from \"./content-type-label.js\";\n\nexport class Model {\n model?: GraphModel;\n\n constructor(public model_config: ModelConfig) {}\n\n async loadUrl(modelURL: string): Promise {\n if (!this.model) {\n this.model = await tf.loadGraphModel(modelURL);\n }\n }\n\n async predict(features: ModelFeatures): Promise {\n if (!this.model) {\n throw new Error(\"model has not been loaded\");\n }\n let features_array = features.toArray();\n const modelInput = tf.tensor(\n [features_array],\n [1, features_array.length],\n \"int32\",\n );\n const modelOutput = tf.squeeze(\n (await this.model.executeAsync(modelInput)) as any,\n );\n const maxScoreIndexTensor = tf.argMax(modelOutput);\n const maxScoreIndex = maxScoreIndexTensor.dataSync()[0];\n const rawScores = modelOutput.dataSync();\n maxScoreIndexTensor.dispose();\n modelInput.dispose();\n modelOutput.dispose();\n\n const maxScoreLabel = this.model_config.target_labels_space[maxScoreIndex];\n const maxScore = rawScores[maxScoreIndex];\n\n if (rawScores.length != this.model_config.target_labels_space.length) {\n throw new Error(\n `Assertion failed: Expected rawScores.length (${rawScores.length}) to have the same length of the targets_label_space (${this.model_config.target_labels_space.length})`,\n );\n }\n\n let scores_map: Partial> = {};\n for (let i = 0; i < rawScores.length; i++) {\n const label: ContentTypeLabel = this.model_config.target_labels_space[i];\n const score: number = rawScores[i];\n scores_map[label] = score;\n }\n\n return { label: maxScoreLabel, score: maxScore, scores_map: scores_map };\n }\n}\n" }, { "path": "js/src/overwrite-reason.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nexport enum OverwriteReason {\n NONE = \"none\",\n LOW_CONFIDENCE = \"low_confidence\",\n OVERWRITE_MAP = \"overwrite_map\",\n}\n" }, { "path": "js/src/prediction-mode.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nexport enum PredictionMode {\n BEST_GUESS = \"best_guess\",\n MEDIUM_CONFIDENCE = \"medium_confidence\",\n HIGH_CONFIDENCE = \"high_confidence\",\n}\n" }, { "path": "js/src/status.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nexport enum Status {\n // No errors.\n OK = \"ok\",\n\n // Used when a file path does not exist.\n FILE_NOT_FOUND_ERROR = \"file-not-found-error\",\n\n //Used when a file path exists, but there are permission issues, e.g., can't read file.\n PERMISSION_ERROR = \"permission-error\",\n\n // Represents a generic error-like unknown status.\n UNKNOWN = \"unknown\",\n}\n" }, { "path": "js/test/features-extraction-vs-reference.test.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { beforeAll, describe, expect, it } from \"@jest/globals\";\nimport { MagikaNode as Magika } from \"../magika-node.js\";\nimport * as utils from \"./utils.js\";\nimport { ModelFeatures } from \"../src/model-features.js\";\n\nconst FEATURES_EXTRACTION_EXAMPLES: FeaturesExtractionExamples = [\n ...parseGzippedFeaturesExtractionExamples(),\n];\n\nclass TestableMagika extends Magika {\n public static extractFeaturesFromBytes(\n fileBytes: Uint8Array,\n beg_size: number,\n mid_size: number,\n end_size: number,\n padding_token: number,\n block_size: number,\n use_inputs_at_offsets: boolean,\n ): ModelFeatures {\n return Magika._extractFeaturesFromBytes(\n fileBytes,\n beg_size,\n mid_size,\n end_size,\n padding_token,\n block_size,\n use_inputs_at_offsets,\n );\n }\n}\n\ndescribe(\"Magika -- features extraction vs. reference\", () => {\n let magika: Magika;\n const repoRootDir = \"../\";\n\n beforeAll(async () => {\n magika = await Magika.create();\n });\n\n it.each(FEATURES_EXTRACTION_EXAMPLES)(\n \"check features extraction vs. reference\",\n async (example) => {\n if (example.args.mid_size != 0 || example.args.use_inputs_at_offsets) {\n // We do not support these settings at the moment.\n return;\n }\n\n const fileBytes = Buffer.from(example.content_base64, \"base64\");\n const features = TestableMagika.extractFeaturesFromBytes(\n fileBytes,\n example.args.beg_size,\n example.args.mid_size,\n example.args.end_size,\n example.args.padding_token,\n example.args.block_size,\n example.args.use_inputs_at_offsets,\n );\n\n expect(features.beg_ints).toEqual(new Uint16Array(example.features.beg));\n expect(features.end_ints).toEqual(new Uint16Array(example.features.end));\n },\n );\n});\n\ninterface FeaturesExtractionExample {\n args: FeaturesExtractionExampleArgs;\n metadata: FeaturesExtractionExampleMetadata;\n content_base64: string;\n features: ExampleModelFeatures;\n}\n\ninterface FeaturesExtractionExampleArgs {\n beg_size: number;\n mid_size: number;\n end_size: number;\n block_size: number;\n padding_token: number;\n use_inputs_at_offsets: boolean;\n}\n\ninterface FeaturesExtractionExampleMetadata {\n core_content_size: number;\n left_ws_num: number;\n right_ws_num: number;\n}\n\ninterface ExampleModelFeatures {\n beg: number[];\n mid: number[];\n end: number[];\n offset_0x8000_0x8007: number[];\n offset_0x8800_0x8807: number[];\n offset_0x9000_0x9007: number[];\n offset_0x9800_0x9807: number[];\n}\n\ntype FeaturesExtractionExamples = FeaturesExtractionExample[];\n\nfunction parseGzippedFeaturesExtractionExamples(): FeaturesExtractionExamples {\n const parsedData = utils.parseGzippedJSON(\n \"../tests_data/reference/features_extraction_examples.json.gz\",\n );\n const featuresExtractionExamples = parsedData as FeaturesExtractionExamples;\n for (const example of featuresExtractionExamples) {\n if (\n example.features.beg.length != example.args.beg_size ||\n example.features.end.length != example.args.end_size\n ) {\n const error_msg = `Error parsing: ${JSON.stringify(example)}`;\n throw new Error(error_msg);\n }\n }\n return featuresExtractionExamples;\n}\n" }, { "path": "js/test/inference-vs-reference.test.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { beforeAll, afterAll, describe, expect, it } from \"@jest/globals\";\nimport * as fs from \"fs\";\nimport { mkdtemp, rm, writeFile } from \"fs/promises\";\nimport path from \"path\";\nimport * as os from \"os\";\nimport { MagikaNode as Magika } from \"../magika-node\";\nimport { ContentTypeLabel } from \"../src/content-type-label\";\nimport { OverwriteReason } from \"../src/overwrite-reason\";\nimport { PredictionMode } from \"../src/prediction-mode\";\nimport { Status } from \"../src/status\";\nimport * as utils from \"./utils\";\n\nconst EXAMPLES_BY_PATH_INFO: Array<[string, ExampleByPath]> = (() => {\n const result: Array<[string, ExampleByPath]> = [];\n for (const example of parseGzippedExamplesByPath()) {\n result.push([example.path, example]);\n }\n return result;\n})();\n\nconst EXAMPLES_BY_CONTENT: ExamplesByContent = [\n ...parseGzippedExamplesByContent(),\n];\n\ndescribe(\"Magika -- inference vs. reference\", () => {\n let magika: Magika;\n const repoRootDir = \"../\";\n let workdir = \"\";\n\n beforeAll(async () => {\n magika = await Magika.create();\n workdir = await mkdtemp(path.join(os.tmpdir(), \"magika-\"));\n });\n\n afterAll(async () => {\n // Make sure we would only delete a tmp dir.\n if (workdir && workdir.startsWith(\"/tmp/\")) {\n await rm(workdir, { recursive: true, force: true });\n }\n });\n\n it.each(EXAMPLES_BY_PATH_INFO)(\n 'check inference vs. reference - examples_by_path for \"%s\"',\n async (examplePath, exampleByPath) => {\n if (exampleByPath.prediction_mode != PredictionMode.HIGH_CONFIDENCE) {\n // We only support HIGH_CONFIDENCE mode for now.\n return;\n }\n\n const fileBytes = fs.readFileSync(repoRootDir + exampleByPath.path);\n let tempFilePath = path.join(workdir, \"file.bin\");\n await writeFile(tempFilePath, fileBytes);\n\n const result = await magika.identifyBytes(fileBytes);\n const resultByStream = await magika.identifyStream(\n fs.createReadStream(tempFilePath),\n fileBytes.length,\n );\n expect(result).toStrictEqual(resultByStream);\n\n expect(result.path).toBe(\"-\");\n expect(result.status).toBe(exampleByPath.status);\n expect(result.prediction.dl.label).toBe(exampleByPath.prediction?.dl);\n expect(result.prediction.output.label).toBe(\n exampleByPath.prediction?.output,\n );\n expect(result.prediction.score).toBeCloseTo(\n exampleByPath.prediction!.score,\n );\n expect(result.prediction.overwrite_reason).toBe(\n exampleByPath.prediction?.overwrite_reason,\n );\n },\n );\n\n it.each(EXAMPLES_BY_CONTENT)(\n \"check inference vs. reference - examples_by_content\",\n async (exampleByContent) => {\n if (exampleByContent.prediction_mode != PredictionMode.HIGH_CONFIDENCE) {\n // We only support HIGH_CONFIDENCE mode for now.\n return;\n }\n\n const fileBytes = Buffer.from(exampleByContent.content_base64, \"base64\");\n let tempFilePath = path.join(workdir, \"fileBytes.bin\");\n await writeFile(tempFilePath, fileBytes);\n\n const result = await magika.identifyBytes(fileBytes);\n const resultByStream = await magika.identifyStream(\n fs.createReadStream(tempFilePath),\n fileBytes.length,\n );\n expect(result).toStrictEqual(resultByStream);\n\n expect(result.status).toBe(exampleByContent.status);\n expect(result.prediction.score).toBeCloseTo(\n exampleByContent.prediction!.score,\n 1,\n );\n expect(result.prediction.dl.label).toBe(exampleByContent.prediction?.dl);\n expect(result.prediction.output.label).toBe(\n exampleByContent.prediction?.output,\n );\n expect(result.prediction.overwrite_reason).toBe(\n exampleByContent.prediction?.overwrite_reason,\n );\n },\n );\n});\n\ninterface Prediction {\n dl: ContentTypeLabel;\n output: ContentTypeLabel;\n score: number; // Python float maps to TypeScript number\n overwrite_reason: OverwriteReason; // Keep snake_case to match JSON\n}\n\ninterface ExampleByPath {\n prediction_mode: PredictionMode;\n path: string;\n status: Status;\n prediction: Prediction | null;\n}\n\ntype ExamplesByPath = ExampleByPath[];\n\ninterface ExampleByContent {\n prediction_mode: PredictionMode;\n content_base64: string;\n status: Status;\n prediction: Prediction | null;\n}\n\ntype ExamplesByContent = ExampleByContent[];\n\nfunction parseGzippedExamplesByPath(): ExamplesByPath {\n const parsedData = utils.parseGzippedJSON(\n \"../tests_data/reference/standard_v3_3-inference_examples_by_path.json.gz\",\n );\n const examplesByPath = parsedData as ExamplesByPath;\n for (const exampleByPath of examplesByPath) {\n if (\n !validatePredictionMode(exampleByPath.prediction_mode) ||\n !validatePrediction(exampleByPath.prediction ?? undefined)\n ) {\n const error_msg = `Error parsing: ${JSON.stringify(exampleByPath)}`;\n throw new Error(error_msg);\n }\n }\n return examplesByPath;\n}\n\nfunction parseGzippedExamplesByContent(): ExamplesByContent {\n const parsedData = utils.parseGzippedJSON(\n \"../tests_data/reference/standard_v3_3-inference_examples_by_content.json.gz\",\n );\n const examplesByContent = parsedData as ExamplesByContent;\n for (const exampleByContent of examplesByContent) {\n if (\n !validatePredictionMode(exampleByContent.prediction_mode) ||\n !validatePrediction(exampleByContent.prediction ?? undefined)\n ) {\n const error_msg = `Error parsing: ${JSON.stringify(exampleByContent)}`;\n throw new Error(error_msg);\n }\n }\n return examplesByContent;\n}\n\nfunction validatePredictionMode(prediction_mode: PredictionMode): boolean {\n return Object.values(PredictionMode).includes(prediction_mode);\n}\n\nfunction validatePrediction(prediction?: Prediction): boolean {\n if (prediction === undefined) {\n return true;\n }\n\n if (!Object.values(ContentTypeLabel).includes(prediction.dl)) {\n return false;\n }\n if (!Object.values(ContentTypeLabel).includes(prediction.output)) {\n return false;\n }\n if (!Object.values(OverwriteReason).includes(prediction.overwrite_reason)) {\n return false;\n }\n return true;\n}\n" }, { "path": "js/test/magika-cli.test.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { describe, expect, it } from \"@jest/globals\";\nimport { spawn } from \"child_process\";\nimport path from \"path\";\n\ndescribe(\"magika-cli.ts CLI Tests\", () => {\n const scriptPath = path.resolve(__dirname, \"../dist/mjs/magika-cli.js\");\n const nodeExecutable = process.execPath;\n\n async function executeCli(\n args: string[] = [],\n ): Promise<{ stdout: string; stderr: string; exitCode: number | null }> {\n return new Promise((resolve, reject) => {\n const process = spawn(nodeExecutable, [scriptPath, ...args]);\n let stdout = \"\";\n let stderr = \"\";\n\n process.stdout.on(\"data\", (data) => {\n stdout += data.toString();\n });\n\n process.stderr.on(\"data\", (data) => {\n stderr += data.toString();\n });\n\n process.on(\"close\", (code) => {\n resolve({ stdout, stderr, exitCode: code });\n });\n\n process.on(\"error\", (err) => {\n reject(err);\n });\n });\n }\n\n it(\"should display help information when no arguments are provided\", async () => {\n const { stdout, stderr, exitCode } = await executeCli();\n expect(exitCode).toBe(1);\n expect(stdout).toContain(\"Usage: \");\n expect(stdout).toContain(\"Options:\");\n expect(stderr).toContain(\"error: missing required argument\");\n\n // Check that the help is printed only once.\n const usageOccurrences = (stdout.match(/Usage:/g) || []).length;\n expect(usageOccurrences).toBe(1);\n const optionsOccurrences = (stdout.match(/Options:/g) || []).length;\n expect(optionsOccurrences).toBe(1);\n });\n\n it(\"should display help information when --help is passed\", async () => {\n const { stdout, stderr, exitCode } = await executeCli([\"--help\"]);\n expect(exitCode).toBe(0);\n expect(stdout).toContain(\"Usage: \");\n expect(stdout).toContain(\"Options:\");\n });\n\n it(\"should process (by path) a specific file and output the expected result\", async () => {\n const filePath = \"../README.md\";\n const expectedLabel = \"markdown\";\n const { stdout, exitCode } = await executeCli([filePath]);\n expect(exitCode).toBe(0);\n expect(stdout).toContain(filePath);\n expect(stdout).toContain(expectedLabel);\n });\n\n it(\"should process (by stream) a specific file and output the expected result\", async () => {\n const filePath = \"../README.md\";\n const expectedLabel = \"markdown\";\n const { stdout, exitCode } = await executeCli([\"--by-stream\", filePath]);\n expect(exitCode).toBe(0);\n expect(stdout).toContain(filePath);\n expect(stdout).toContain(expectedLabel);\n });\n});\n" }, { "path": "js/test/magika.test.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { TfnMock } from \"./tfnHook\";\n// TfnMock must be imported first; leave this line here to avoid imports\n// sorting.\nimport {\n afterAll,\n beforeAll,\n beforeEach,\n describe,\n expect,\n it,\n jest,\n} from \"@jest/globals\";\nimport * as fc from \"fast-check\";\nimport * as fs from \"fs\";\nimport { Dirent, readdirSync } from \"fs\";\nimport { mkdtemp, readFile, rm } from \"fs/promises\";\nimport * as os from \"os\";\nimport * as path from \"path\";\nimport { Readable } from \"stream\";\nimport { finished } from \"stream/promises\";\nimport { ReadableStream } from \"stream/web\";\nimport { MagikaNode as Magika } from \"../magika-node\";\nimport { ContentTypeLabel } from \"../src/content-type-label\";\n\n/**\n * Returns a list of test files and their correct labels.\n *\n * @param directory the directory to recursively scan for test files.\n * @returns the list of file paths and labels.\n */\nconst getTestFilesWithLabels = (\n directory: string,\n): Array<[string, string, Dirent]> =>\n readdirSync(directory, { recursive: true, withFileTypes: true })\n .filter((dirent) => dirent.isFile())\n .map<[string, string, Dirent]>((dirent) => {\n const label = dirent.parentPath.split(\"/\").pop() || \"UNKNOWN\";\n const filePath = path.join(dirent.parentPath, dirent.name);\n return [label, filePath, dirent];\n });\n\n/**\n * Array of all our test files and their labels.\n */\nconst BASIC_TEST_FILES: Array<[string, string, Dirent]> = [\n ...getTestFilesWithLabels(\"../tests_data/basic\"),\n];\n\ndescribe(\"Magika class\", () => {\n const workdir = {\n root: \"\",\n model_config: \"\",\n model: \"\",\n };\n beforeAll(async () => {\n workdir.root = await mkdtemp(path.join(os.tmpdir(), \"magika-\"));\n workdir.model_config = path.join(workdir.root, \"config.json\");\n workdir.model = path.join(workdir.root, \"model.json\");\n\n const model_config = Readable.fromWeb(\n (await fetch(Magika.MODEL_CONFIG_URL)).body as ReadableStream,\n );\n const model = Readable.fromWeb(\n (await fetch(Magika.MODEL_URL)).body as ReadableStream,\n );\n await Promise.all([\n await finished(\n model_config.pipe(fs.createWriteStream(workdir.model_config)),\n ),\n await finished(model.pipe(fs.createWriteStream(workdir.model))),\n ]);\n const weights = JSON.parse((await readFile(workdir.model)).toString())\n .weightsManifest.filter(\n (weights: { paths?: string[] }) => weights?.paths != null,\n )\n .map((weights: { paths: string[] }) => {\n return weights.paths.map((path) => {\n return {\n name: path,\n url: Magika.MODEL_URL.replace(/model\\.json$/, path),\n };\n });\n })\n .flat();\n await Promise.all(\n weights.map(async (weight: { name: string; url: string }) => {\n const model_config = Readable.fromWeb(\n (await fetch(weight.url)).body as ReadableStream,\n );\n await finished(\n model_config.pipe(\n fs.createWriteStream(path.join(workdir.root, weight.name)),\n ),\n );\n }),\n );\n });\n\n beforeEach(async () => {\n TfnMock.reset();\n });\n\n afterAll(async () => {\n if (workdir.root) {\n await rm(workdir.root, { recursive: true, force: true });\n }\n });\n\n it(\"should load default model from url\", async () => {\n const magika = await Magika.create();\n expect(magika.model.model).toBeDefined();\n expect(magika.model_config.target_labels_space.length).toBeGreaterThan(0);\n expect(Object.values(TfnMock.accessed).reduce((a, b) => a + b, 0)).toBe(0);\n });\n\n it(\"should load model from file path\", async () => {\n const magika = await Magika.create({\n modelConfigPath: workdir.model_config,\n modelPath: workdir.model,\n });\n expect(magika.model.model).toBeDefined();\n expect(magika.model_config.target_labels_space.length).toBeGreaterThan(0);\n expect(TfnMock.accessed.io).toBe(1);\n expect(Object.values(TfnMock.accessed).reduce((a, b) => a + b, 0)).toBe(1);\n });\n\n it(\"scores should be in the expected range\", async () => {\n const magika = await Magika.create();\n fc.assert(\n fc.asyncProperty(\n fc.array(fc.integer({ min: 0, max: 255 }), {\n minLength: 0,\n maxLength: 10,\n }),\n async (bytesContent) => {\n const result = await magika.identifyBytes(\n Uint8Array.from(bytesContent),\n );\n expect(result.prediction.score).toBeGreaterThanOrEqual(0);\n expect(result.prediction.score).toBeLessThanOrEqual(1);\n },\n ),\n );\n });\n\n it.each(BASIC_TEST_FILES)(\n 'by_stream vs by_byte should return the same (correct) features/label for \"%s\" \"%s\"',\n async (label, testPath, testFile) => {\n const magika = await Magika.create({\n modelConfigPath: workdir.model_config,\n modelPath: workdir.model,\n });\n const featuresMock = jest.spyOn(magika.model, \"predict\");\n\n // Do predictions by stream and by path.\n const filePath = path.join(testFile.parentPath, testFile.name);\n const streamResult = await magika.identifyStream(\n fs.createReadStream(filePath),\n (await fs.promises.stat(filePath)).size,\n );\n const fileBytes = await fs.promises.readFile(filePath);\n const byteResult = await magika.identifyBytes(fileBytes);\n\n // Compare the results; they should match between them\n expect(streamResult).toStrictEqual(byteResult);\n if (streamResult.prediction.dl.label != ContentTypeLabel.UNDEFINED) {\n expect(featuresMock.mock.calls[0][0]).toStrictEqual(\n featuresMock.mock.calls[1][0],\n );\n }\n // Check that the predictions make the expectations.\n expect(streamResult.prediction.output.label).toBe(label);\n\n // Check properties on the TfnMock object.\n expect(TfnMock.accessed.io).toBe(1);\n expect(Object.values(TfnMock.accessed).reduce((a, b) => a + b, 0)).toBe(\n 1,\n );\n\n // The predictions are the same via bytes and via stream, let's just take one.\n const prediction = byteResult.prediction;\n expect(prediction).not.toBeUndefined();\n expect(prediction.dl).not.toBeUndefined();\n expect(prediction.output).not.toBeUndefined();\n expect(prediction.score).not.toBeUndefined();\n\n if (prediction.dl.label == ContentTypeLabel.UNDEFINED) {\n // If dl.label == UNDEFINED, scores_map should not be set.\n expect(prediction.scores_map).toBeUndefined();\n } else {\n // If dl.label is not UNDEFINED, scores_map should be set.\n expect(prediction.scores_map).not.toBeUndefined();\n // Check that the max score and label associated to max score matches\n // what's returned in the prediction.\n const scores = Object.values(prediction?.scores_map ?? {});\n let curr_max_score = scores[0];\n let argmax_idx = 0;\n for (let i = 1; i < scores.length; i++) {\n if (scores[i] > curr_max_score) {\n curr_max_score = scores[i];\n argmax_idx = i;\n }\n }\n const predicted_label =\n magika.model_config.target_labels_space[argmax_idx];\n expect(predicted_label).toBe(prediction.dl.label);\n expect(curr_max_score).toBe(prediction.score);\n }\n },\n );\n\n it.each(BASIC_TEST_FILES)(\n 'Magika is agnostic to the format of the input bytes for \"%s\" \"%s\"',\n async (label, testPath, testFile) => {\n const magika = await Magika.create({\n modelConfigPath: workdir.model_config,\n modelPath: workdir.model,\n });\n const featuresMock = jest.spyOn(magika.model, \"predict\");\n const filePath = path.join(testFile.parentPath, testFile.name);\n const inputBuffer = await fs.promises.readFile(filePath);\n const inputUint8 = new Uint8Array(inputBuffer);\n const resultFromBuffer = await magika.identifyBytes(inputBuffer);\n const resultFromUint8 = await magika.identifyBytes(inputUint8);\n expect(resultFromBuffer).toStrictEqual(resultFromUint8);\n\n if (resultFromBuffer.prediction.dl.label != ContentTypeLabel.UNDEFINED) {\n expect(featuresMock.mock.calls[0][0]).toStrictEqual(\n featuresMock.mock.calls[1][0],\n );\n }\n },\n );\n});\n" }, { "path": "js/test/tfnHook.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport { jest } from \"@jest/globals\";\n\nexport class TfnMock {\n static accessed: Record = {};\n\n static mock = jest.mock(\n \"@tensorflow/tfjs-node\",\n () => {\n const hook = {};\n const original = jest.requireActual(\"@tensorflow/tfjs-node\") as any;\n Object.keys(original as object).forEach((key) => {\n TfnMock.accessed[key] = 0;\n Object.defineProperty(hook, key, {\n configurable: true, // allow spyOn to work\n enumerable: true, // so the key shows up\n get(): any {\n TfnMock.accessed[key] = (TfnMock.accessed[key] || 0) + 1;\n return original[key];\n },\n });\n });\n return hook;\n },\n { virtual: true },\n );\n\n static reset() {\n for (const i in TfnMock.accessed) {\n TfnMock.accessed[i] = 0;\n }\n }\n}\n" }, { "path": "js/test/utils.ts", "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nimport * as fs from \"fs\";\nimport * as zlib from \"zlib\";\n\nexport function parseGzippedJSON(filePath: string): Array {\n const gzippedBuffer = fs.readFileSync(filePath);\n const jsonBuffer = zlib.gunzipSync(gzippedBuffer);\n const jsonString = jsonBuffer.toString(\"utf-8\");\n const parsedData = JSON.parse(jsonString);\n if (!Array.isArray(parsedData)) {\n throw new Error(\"Parsed JSON is not an array as expected for ExampleList.\");\n }\n return parsedData as Array;\n}\n" }, { "path": "js/tsconfig.cjs.json", "content": "{\n \"extends\": \"./tsconfig.json\",\n \"compilerOptions\": {\n \"module\": \"commonjs\",\n \"moduleResolution\": \"node\",\n \"outDir\": \"dist/cjs\",\n \"target\": \"es2015\"\n }\n}" }, { "path": "js/tsconfig.esm.json", "content": "{\n \"extends\": \"./tsconfig.json\",\n \"compilerOptions\": {\n \"module\": \"nodenext\",\n \"moduleResolution\": \"nodenext\",\n \"outDir\": \"dist/mjs\",\n \"target\": \"esnext\"\n }\n}" }, { "path": "js/tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"allowSyntheticDefaultImports\": true,\n \"baseUrl\": \".\",\n \"declaration\": true,\n \"downlevelIteration\": true,\n \"esModuleInterop\": true,\n \"inlineSourceMap\": false,\n \"lib\": [\"esnext\"],\n \"listEmittedFiles\": false,\n \"listFiles\": false,\n \"noFallthroughCasesInSwitch\": true,\n \"pretty\": true,\n \"resolveJsonModule\": true,\n \"rootDir\": \".\",\n \"skipLibCheck\": true,\n \"strict\": true,\n \"traceResolution\": false,\n \"types\": [\"node\", \"jest\"]\n },\n \"compileOnSave\": false,\n \"exclude\": [\"node_modules\", \"dist\"],\n \"include\": [\"src\", \"magika-cli.ts\", \"magika.ts\", \"magika-node.ts\"]\n}" }, { "path": "python/.gitignore", "content": "*.pyc\n__pycache__/\n**/.ruff_cache/\n.ipynb_checkpoints\nvenv/\n.env\n*.swp\n*.h5\n*.egg-info\ndist/*\n*.pickle\n.s.yml\n" }, { "path": "python/.python-version", "content": "3.12\n" }, { "path": "python/CHANGELOG.md", "content": "# Changelog\n\nAll notable changes to this project will be documented in this file.\n\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),\nand this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).\n\nNote that for version number starting with a `0`, i.e., `0.x.y`, a bump of `x`\nshould be considered as a major (and thus potentially breaking) change. See\nsemver guidelines for more details about this.\n\n## [1.0.2] - 2026-02-25\n- Mark python 3.14 as supported.\n- Remove direct dependency on numpy.\n- Remove dependency on python-dotenv (note: .env files are no longer loaded automatically).\n- Remove onnxruntime<=1.20.1 Windows pin.\n\n## [1.0.1] - 2025-10-31\n- Mark end of experimental phase. No changes from last version.\n\n## [0.6.3] - 2025-10-30\n- Pin onnxruntime on Windows ([#1099](https://github.com/google/magika/pull/1099)).\n- Add docstrings for all relevant modules, classes, and methods.\n- Improved READMEs and overall [documentation](https://securityresearch.google/magika/cli-and-bindings/python/).\n\n## [0.6.2] - 2025-05-02\n\n- Mark python 3.13 as supported.\n- New model `standard_v3_3` model, with better support for TypeScript and non-ascii characters in textual files. See [models' CHANGELOG](../assets/models/CHANGELOG.md) for more information.\n- `identify_stream()` now restores the stream's original position after reading from it, preventing side effects on subsequent stream operations. ([#1020](https://github.com/google/magika/pull/1020))\n- Add `asdict()` utility method to `MagikaResult`.\n- Set `prediction.overwrite_reason` to `Overwrite.NONE` if `output.label` is the same as `dl.label`. ([#1023](https://github.com/google/magika/pull/1023))\n- Bugfix: limit the number of bytes we read in case of an input with just many whitespaces. ([#1015](https://github.com/google/magika/pull/1015))\n- Bugfix: do not alter warnings' simplefilter as this has visible side effects for other modules. ([#1017](https://github.com/google/magika/pull/1017))\n- Bugfix: magika's python client now properly warns for low-confidence predictions.\n- Bugfix: update Magika's StrEnum string representation to be compatible with standard library.\n\n## [0.6.1] - 2025-03-19\n\n### Overview\n\nMagika v0.6.1 is a significant update featuring a new model with 2x supported content types, a new command line client in Rust, performance improvements, API enhancements, and a few breaking changes. This changelog entry rolls up all changes from v0.5.1, the last stable release.\n\n> [!IMPORTANT]\n> There are a few breaking changes! After reading about the new key features and improvements, we suggest to consult the migration guide below and the [updated documentation](./README.md).\n\n### Key Features and Improvements\n\n- **New deep learning model:** We introduce a new model, `standard_v3_2`, which supports 2x content types (200+ in total, see full list [here](../assets/models/standard_v3_2/README.md)), has a similar ~99% average accuracy, and is ~20% faster, with an inference speed of about ~2ms on CPUs (YMMV depending on your testing setup). See [models' CHANGELOG](../assets/models/CHANGELOG.md) for more information.\n- **New command line client, written in Rust:** We developed a new command line client, written in Rust, which is not affected by the one-time bootstrap overhead caused by the python's interpreter itself. This new client is packaged, pre-compiled, into the `magika` python package. This new client replaces the old client written in Python (but the old Python one is still available as a fallback for those platforms for which we don't have precompiled rust binaries).\n- **New stream-based identification:** Added `identify_stream(stream: typing.BinaryIO)` API to infer content types from open binary streams. ([#970](https://github.com/google/magika/issues/970))\n- **Improved path handling:** `identify_path` and `identify_paths` now accept `Union[str, os.PathLike]` objects. You no longer need to explicitly use `pathlib.Path`. ([#935](https://github.com/google/magika/issues/935))\n- **Improved python API:** The new Python APIs offer a number of improvements, including: the inference APIs now return a `MagikaResult`, which is a [`absl::StatusOr`](https://abseil.io/docs/cpp/guides/status)-like object that wraps `MagikaPrediction`, with a clear separation between valid predictions and error situations; the output content types (`label`) are not just `str` anymore, but of type `ContentTypeLabel`, making integrations more robust (`ContentTypeLabel` extends `StrEnum`: thus, they are not just `str`, but you can treat them as such). The `MagikaPrediction` object now has additional `is_text` and `extensions` fields (in addition to the existing `label`, `mime_type`, `group`, and `description`).\n- **New debugging APIs**: Added new APIs to ease debugging and introspection, such as `get_output_content_types()`, `get_model_content_types()`, `get_module_version()`, and `get_model_name()`.\n\n### Breaking Changes and Migration Guide\n\nThis release introduces several breaking changes. Please review this guide carefully to update your code:\n\n1. **New `identify_*` API output format:** The inference Python APIs now return a `MagikaResult` object, which is similar to `absl::StatusOr`; This provides a cleaner way to handle errors. `dl.ct_label` and `output.ct_label` are renamed to `dl.label` and `output.label`. `label`s are now of type `ContentTypeLabel`, which extends `StrEnum` (thus, they are not just `str`, but you can treat them as such). The `score` field is now at the top level, alongside `dl` and `output`. The `magic` field has been removed as it was often either incorrect or redundant; use `description` instead.\n\n- **Before (v0.5.x and earlier):**\n\n ```python\n import magika\n m = magika.Magika()\n result = m.identify_path(\"my_file.py\")\n print(result.output.ct_label) # Assumed success\n ```\n\n- **After (v0.6.1):**\n ```python\n import magika\n m = magika.Magika()\n result = m.identify_path(\"my_file.py\")\n if result.ok():\n print(result.output.label)\n else:\n print(f\"Error: {result.status}\")\n ```\n\n2. **CLI Output Format Change (v0.6.0):** The JSON output format of the CLI has changed. Those changes are analogous to the changes to the Python APIs. The `score` field is now at the top level, alongside `dl` and `output`, and is no longer nested within `dl` or `output`. The output also includes `is_text` and `extensions` fields. The `magic` metadata has been removed as it was often either incorrect or redundant; use `description` instead. Moreover, similarly to what happens under the hood with the `StatusOr` pattern, `result.status` indicates whether the prediction was successful, and the prediction results are available under the `result.value` key.\n\n- **Before (v0.5.x and earlier):** (Illustrative example - adapt to your specific output)\n\n ```json\n {\n \"path\": \"code.py\",\n \"dl\": {\n \"ct_label\": \"python\",\n \"score\": 0.9940916895866394,\n \"group\": \"code\",\n \"mime_type\": \"text/x-python\",\n \"magic\": \"Python script\",\n \"description\": \"Python source\"\n },\n \"output\": {\n \"ct_label\": \"python\",\n \"score\": 0.9940916895866394,\n \"group\": \"code\",\n \"mime_type\": \"text/x-python\",\n \"magic\": \"Python script\",\n \"description\": \"Python source\"\n }\n }\n ```\n\n- **After (v0.6.1):**\n ```json\n {\n \"path\": \"code.py\",\n \"result\": {\n \"status\": \"ok\",\n \"value\": {\n \"dl\": {\n \"description\": \"Python source\",\n \"extensions\": [\"py\", \"pyi\"],\n \"group\": \"code\",\n \"is_text\": true,\n \"label\": \"python\",\n \"mime_type\": \"text/x-python\"\n },\n \"output\": {\n \"description\": \"Python source\",\n \"extensions\": [\"py\", \"pyi\"],\n \"group\": \"code\",\n \"is_text\": true,\n \"label\": \"python\",\n \"mime_type\": \"text/x-python\"\n },\n \"score\": 0.9890000224113464\n }\n }\n }\n ```\n\n3. **`dl.label == ContentTypeLabel.UNDEFINED` when the model is not used:** There are situations in which the deep learning model is not used, for example when the file is too small or empty. In these cases, `dl.label` is now set to `ContentTypeLabel.UNDEFINED` instead of having the full `dl` block being set to `None`.\n\n- **Before (v0.5.x and earlier):**\n\n ```python\n # ... (assuming successful result)\n if prediction.dl is not None:\n print(prediction.dl.ct_label)\n ```\n\n- **After (v0.6.1):**\n ```python\n # ... (assuming successful result)\n if prediction.dl.label != magika.ContentTypeLabel.UNDEFINED:\n print(prediction.dl.label)\n ```\n\n4. **Expanded List of Content Types:** The model now supports over 200 content types.\n\n- **Migration:** Review the [updated list of supported content types](../assets/models/standard_v3_2/README.md) and adjust any code that relies on specific content type labels returned by previous versions. Labels have _not_ changed, but a file previously detected as `javascript` may not be detected as `typescript`. Consider using `get_output_content_types()` to dynamically retrieve the supported labels.\n\n5. **Pure Python Wheel and Rust Client Fallback:** If you are installing Magika on a platform _without_ pre-built wheels (e.g., Windows on ARM), you will automatically get the pure-python wheel. In this case, the package does _not_ include the Rust binary client, but it does include the old python client as fallback; you can use such old python client with `$ magika-python-client`.\n\n### Full Changelog\n\nFor a detailed list of all changes, including those from the -rc releases, please refer to the individual changelog entries for each release candidate:\n\n- [0.6.1-rc3](https://github.com/google/magika/blob/python-v0.6.1-rc3/python/CHANGELOG.md#061-rc3---2025-03-17)\n- [0.6.1-rc2](https://github.com/google/magika/blob/python-v0.6.1-rc2/python/CHANGELOG.md#061-rc2---2025-03-11)\n- [0.6.1-rc1](https://github.com/google/magika/blob/python-v0.6.1-rc1/python/CHANGELOG.md#061-rc1---2025-02-04)\n- [0.6.1-rc0](https://github.com/google/magika/blob/python-v0.6.1-rc0/python/CHANGELOG.md#061-rc0---2025-01-23)\n- [0.6.0-rc3](https://github.com/google/magika/blob/python-v0.6.0-rc3/python/CHANGELOG.md#060-rc3---2024-11-20)\n- [0.6.0-rc2](https://github.com/google/magika/blob/python-v0.6.0-rc2/python/CHANGELOG.md#060-rc2---2024-11-19)\n- [0.6.0-rc1](https://github.com/google/magika/blob/python-v0.6.0-rc2/python/CHANGELOG.md#060-rc1---2024-10-07)\n\n## [0.5.1] - 2024-03-06\n\n- Add support for python 3.12. Magika now supports python >=3.8 and <3.13.\n- Fix bugs for features extraction to cover more corner cases.\n- Remove MIME types from table of supported content types (Relevant for `--list-output-content-types`; see FAQs for context).\n- Refactor features extraction around a Seekable abstraction; we now have only one reference implementation.\n- Start groundwork for v2 of features extraction.\n- Various clean ups and internal refactors.\n\n## [0.5.0] - 2024-02-15\n\n- New public python APIs: `identify_paths`, `identify_path`, `identify_bytes`.\n- The APIs now return a `MagikaResult` object.\n- When the model's prediction has low confidence and we return a generic content\n type, print anyways (with a disclaimer) the model's best guess.\n- Updated description for \"unknown\" to \"Unknown binary data\".\n- Magika CLI now defaults to \"high-confidence\" mode. \"default\" mode is now called \"medium-confidence\".\n- Magika CLI `-p/--output-probability` has been renamed to `-s/--output-score` for consistency.\n- Default model is now called `standard_v1`.\n- Major refactoring and clean up.\n\n## [0.4.1] - 2024-02-07\n\n- Various improvements and clean ups.\n\n## [0.4.0] - 2023-12-22\n\n### Changed\n\n- Update model to dense_v4_top_20230910.\n- Package now contains the model itself.\n- Support reading from stdin:\n - `$ cat | magika -`\n - `$ curl | magika -`\n- Change how we deal with padding, using 256 instead of 0. This boosts precision.\n- \"symlink\" output label has been renamed to \"symlinktext\" to better reflect its nature.\n- New `--prediction-mode` CLI option to indicate which confidence is required\n for the predictions. We support three modes: `best-guess`, `default`,\n `high-confidence`.\n- Support for directories and symlinks similarly to `file`.\n- Adapt `-r` / `--recursive` CLI option to be compatible with the new way magika\n handles directories.\n- Add special handling for small files.\n- Magika does not crash anymore when scanning files with permission\n issues. It now returns \"permission_error\".\n- Do not resolve file paths (i.e., relative paths remain relative).\n- Add --no-dereference CLI option: by default symlinks are dereferenced.\n this option makes magika not dereferencing symlinks. This is what `file` does.\n- Clean up and many bug fixes.\n\n## [0.3.1] - 2023-08-23\n\n### Changed\n\n- Removed warnings when using MIME type and compatibility mode.\n\n## [0.3.0] - 2023-08-23\n\n### Changed\n\n- By default, magika now outputs a human-readable output.\n- Add `-l` / `--label` CLI option to output a stable, content type label.\n- JSON/JSONL output now shows all metadata about a given content type.\n- Add metadata about magic and description for each relevant content type.\n- Logs are now printed to stderr, not stdout.\n- Add `--generate-report` CLI option to output a JSON report that can be useful for debugging and reporting feedback.\n- Be more flexible with the required python version (now we require \"^3.8\" instead of \"^3.8,<3.11\")\n- Show a descriptive error in case magika can't find any file to scan (instead of silently exiting).\n\n## [0.2.2] - 2023-08-11\n\n### Changed\n\n- If the prediction score is higher than a given threshold (0.95), consider it\n regardless of the per-content-type threshold.\n- Output format is back being just ``; group is displayed only\n when showing metadata.\n- Update metadata of some content types.\n\n## [0.2.1] - 2023-08-10\n\n### Changed\n\n- Several small bug fixes.\n\n## [0.2.0] - 2023-08-09\n\n### Added\n\n- Input files are now processed in multiple small batches, instead of one big batch.\n- Per-content-type threshold based on the 0.005 quantile for recall.\n- MIME type and \"group\" metadata for all content types.\n- Introduce basic support for compatibility mode.\n- `-c` / `--compatibility-mode` CLI option to enable compatibility mode.\n- `--no-colors` CLI option to disable colors.\n- `-b` / `--batch-size` CLI option to specify the batch size.\n- `--guess` / `--output-highest-probability` CLI option to output the content type with the highest probability regardless of its probability score.\n- `--version` CLI option to print Magika's version.\n\n### Changed\n\n- Output follows the `::` format.\n- Probability score is not shown by default; enable with `-p`.\n- Output is colored according to the file content type's group.\n- Remove dependency from richlogger, add a much simpler logger.\n\n## [0.1.0] - 2023-07-28\n\n- First release.\n" }, { "path": "python/README.md", "content": "# Magika Python Package\n\n[![image](https://img.shields.io/pypi/v/magika.svg)](https://pypi.python.org/pypi/magika)\n[![NPM Version](https://img.shields.io/npm/v/magika)](https://npmjs.com/package/magika)\n[![image](https://img.shields.io/pypi/l/magika.svg)](https://pypi.python.org/pypi/magika)\n[![image](https://img.shields.io/pypi/pyversions/magika.svg)](https://pypi.python.org/pypi/magika)\n[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8706/badge)](https://www.bestpractices.dev/en/projects/8706)\n![CodeQL](https://github.com/google/magika/workflows/CodeQL/badge.svg)\n[![Actions status](https://github.com/google/magika/actions/workflows/python-build-and-release-package.yml/badge.svg)](https://github.com/google/magika/actions)\n[![PyPI Monthly Downloads](https://static.pepy.tech/badge/magika/month)](https://pepy.tech/projects/magika)\n[![PyPI Downloads](https://static.pepy.tech/badge/magika)](https://pepy.tech/projects/magika)\n\n\n\nMagika is a novel AI-powered file type detection tool that relies on the recent advance of deep learning to provide accurate detection. Under the hood, Magika employs a custom, highly optimized model that only weighs about a few MBs, and enables precise file identification within milliseconds, even when running on a single CPU. Magika has been trained and evaluated on a dataset of ~100M samples across 200+ content types (covering both binary and textual file formats), and it achieves an average ~99% accuracy on our test set.\n\nUse Magika as a command line client or in your Python code!\n\nYou can find more information on which content types are supported, extended documentation, and bindings for other languages on Magika's website at [https://securityresearch.google/magika/](https://securityresearch.google/magika/).\n\n\n## Installing Magika\n\nMagika is available as `magika` on [PyPI](https://pypi.org/project/magika):\n\nTo install the most recent stable version:\n\n```shell\npip install magika\n```\n\nIf you intend to use Magika only as a command line, you may want to use `pipx install magika` instead.\n\nIf you want to test out the latest release candidate, you can install it with `pip install --pre magika`.\n\n## Using Magika as a command-line tool\n\n> Beginning with version `0.6.0`, the magika Python package includes a pre-compiled Rust-based command-line tool, replacing the previous Python version. This binary is distributed as platform-specific wheels for most common architectures. For unsupported platforms, a pure-Python wheel is also available, providing the legacy Python client as a fallback.\n\n```shell\n$ cd tests_data/basic && magika -r *\nasm/code.asm: Assembly (code)\nbatch/simple.bat: DOS batch file (code)\nc/code.c: C source (code)\ncss/code.css: CSS source (code)\ncsv/magika_test.csv: CSV document (code)\ndockerfile/Dockerfile: Dockerfile (code)\ndocx/doc.docx: Microsoft Word 2007+ document (document)\nepub/doc.epub: EPUB document (document)\nepub/magika_test.epub: EPUB document (document)\nflac/test.flac: FLAC audio bitstream data (audio)\nhandlebars/example.handlebars: Handlebars source (code)\nhtml/doc.html: HTML document (code)\nini/doc.ini: INI configuration file (text)\njavascript/code.js: JavaScript source (code)\njinja/example.j2: Jinja template (code)\njpeg/magika_test.jpg: JPEG image data (image)\njson/doc.json: JSON document (code)\nlatex/sample.tex: LaTeX document (text)\nmakefile/simple.Makefile: Makefile source (code)\nmarkdown/README.md: Markdown document (text)\n[...]\n```\n\n```shell\n$ magika ./tests_data/basic/python/code.py --json\n[\n {\n \"path\": \"./tests_data/basic/python/code.py\",\n \"result\": {\n \"status\": \"ok\",\n \"value\": {\n \"dl\": {\n \"description\": \"Python source\",\n \"extensions\": [\n \"py\",\n \"pyi\"\n ],\n \"group\": \"code\",\n \"is_text\": true,\n \"label\": \"python\",\n \"mime_type\": \"text/x-python\"\n },\n \"output\": {\n \"description\": \"Python source\",\n \"extensions\": [\n \"py\",\n \"pyi\"\n ],\n \"group\": \"code\",\n \"is_text\": true,\n \"label\": \"python\",\n \"mime_type\": \"text/x-python\"\n },\n \"score\": 0.753000020980835\n }\n }\n }\n]\n```\n\n```shell\n$ cat doc.ini | magika -\n-: INI configuration file (text)\n```\n\n```help\n$ magika --help\nDetermines the content type of files with deep-learning\n\nUsage: magika [OPTIONS] [PATH]...\n\nArguments:\n [PATH]...\n List of paths to the files to analyze.\n\n Use a dash (-) to read from standard input (can only be used once).\n\nOptions:\n -r, --recursive\n Identifies files within directories instead of identifying the directory itself\n\n --no-dereference\n Identifies symbolic links as is instead of identifying their content by following them\n\n --colors\n Prints with colors regardless of terminal support\n\n --no-colors\n Prints without colors regardless of terminal support\n\n -s, --output-score\n Prints the prediction score in addition to the content type\n\n -i, --mime-type\n Prints the MIME type instead of the content type description\n\n -l, --label\n Prints a simple label instead of the content type description\n\n --json\n Prints in JSON format\n\n --jsonl\n Prints in JSONL format\n\n --format \n Prints using a custom format (use --help for details).\n\n The following placeholders are supported:\n\n %p The file path\n %l The unique label identifying the content type\n %d The description of the content type\n %g The group of the content type\n %m The MIME type of the content type\n %e Possible file extensions for the content type\n %s The score of the content type for the file\n %S The score of the content type for the file in percent\n %b The model output if overruled (empty otherwise)\n %% A literal %\n\n -h, --help\n Print help (see a summary with '-h')\n\n -V, --version\n Print version\n```\n\nCheck the [Rust CLI docs](https://securityresearch.google/magika/cli-and-bindings/cli/) for more information.\n\n## Using Magika as a Python module\n\n\nHere is a few examples on how to use the `Magika` Python module:\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_bytes(b'function log(msg) {console.log(msg);}')\n>>> print(res.output.label)\njavascript\n```\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_path('./tests_data/basic/ini/doc.ini')\n>>> print(res.output.label)\nini\n```\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f:\n>>> res = m.identify_stream(f)\n>>> print(res.output.label)\nini\n```\n\n\n## Core Concepts\n\nTo get the most out of Magika, it's worth learning about its core concepts. You can read about how Magika works, the models, the supported content types, the prediction modes, and notes on how to understand all parts of the output in the [Core Concepts](https://securityresearch.google/magika/core-concepts) section of Magika's website.\n\n\n## API documentation\n\nThe Python API is documented in the [Python bindings](https://securityresearch.google/magika/cli-and-bindings/python/) section of Magika's website, and via docstrings in the `Magika` module source code.\n\n\n## Research Paper and Citation\n\nWe describe how we developed Magika and the choices we made in our research paper, which was accepted at the International Conference on Software Engineering (ICSE) 2025. You can find a copy of the paper [here](https://securityresearch.google/magika/2025_icse_magika.pdf).\n\nIf you use this software for your research, please cite it as:\n\n```bibtex\n@InProceedings{fratantonio25:magika,\n author = {Yanick Fratantonio and Luca Invernizzi and Loua Farah and Kurt Thomas and Marina Zhang and Ange Albertini and Francois Galilee and Giancarlo Metitieri and Julien Cretin and Alexandre Petit-Bianco and David Tao and Elie Bursztein},\n title = {{Magika: AI-Powered Content-Type Detection}},\n booktitle = {Proceedings of the International Conference on Software Engineering (ICSE)},\n month = {April},\n year = {2025}\n}\n```\n" }, { "path": "python/mypy.ini", "content": "[mypy]\nshow_error_codes = True\nfollow_imports = silent\nlocal_partial_types = true\nstrict_equality = true\nno_implicit_optional = true\nwarn_incomplete_stub = true\nwarn_redundant_casts = true\nwarn_unused_configs = true\nwarn_unused_ignores = true\nenable_error_code = ignore-without-code, redundant-self, truthy-iterable\ndisable_error_code = annotation-unchecked, import-not-found, import-untyped, type-arg, no-any-unimported\nextra_checks = false\ncheck_untyped_defs = true\ndisallow_incomplete_defs = true\ndisallow_subclassing_any = true\ndisallow_untyped_calls = true\ndisallow_untyped_decorators = true\ndisallow_untyped_defs = true\nwarn_return_any = true\nwarn_unreachable = true\nallow_redefinition = false\nstrict_optional = true\n\n[mypy-magika.*]\nignore_missing_imports = true\nno_implicit_reexport = true\ndisallow_untyped_calls = true\ndisallow_any_unimported = true\ndisallow_untyped_decorators = true\nstrict = true\nenable_error_code = ignore-without-code, redundant-self, truthy-iterable, possibly-undefined, truthy-bool, truthy-iterable, unused-ignore, mutable-override\n\n[mypy-magika.strenum.*]\nignore_errors = True\n\n[mypy-tests.*]\ndisallow_untyped_defs = false\ndisallow_untyped_calls = false\ndisallow_untyped_decorators = false\n" }, { "path": "python/pyproject.toml", "content": "[project]\nname = \"magika\"\ndescription = \"A tool to determine the content type of a file with deep learning\"\nauthors = [\n {name = \"Magika Developers\", email = \"magika-dev@google.com\"},\n]\nreadme = \"README.md\"\nlicense = {\"text\" = \"Apache-2.0\"}\nrequires-python = \">=3.8\"\nkeywords = [\"content type detection\", \"machine learning\"]\nclassifiers = [\n \"Development Status :: 5 - Production/Stable\",\n \"Environment :: Console\",\n \"License :: OSI Approved :: Apache Software License\",\n \"Intended Audience :: Developers\",\n \"Intended Audience :: Science/Research\",\n \"Intended Audience :: System Administrators\",\n \"Programming Language :: Python\",\n \"Programming Language :: Python :: 3\",\n \"Programming Language :: Python :: 3.8\",\n \"Programming Language :: Python :: 3.9\",\n \"Programming Language :: Python :: 3.10\",\n \"Programming Language :: Python :: 3.11\",\n \"Programming Language :: Python :: 3.12\",\n \"Programming Language :: Python :: 3.13\",\n \"Programming Language :: Python :: 3.14\",\n \"Operating System :: MacOS\",\n \"Operating System :: Microsoft :: Windows\",\n \"Operating System :: Unix\",\n \"Topic :: Scientific/Engineering :: Artificial Intelligence\",\n \"Topic :: Security\",\n \"Topic :: Software Development\",\n \"Typing :: Typed\",\n]\nversion = \"1.0.2\"\n\ndependencies = [\n \"click>=8.1.7\",\n\n # Python 3.14+: Min version 1.24.1\n \"onnxruntime>=1.24.1 ; python_version >= '3.14'\",\n # Python 3.13: Min version 1.21.0\n \"onnxruntime>=1.21.0 ; python_version == '3.13'\",\n # Python 3.11 - 3.12: Min version 1.17.0\n \"onnxruntime>=1.17.0 ; python_version >= '3.11' and python_version < '3.13'\",\n # Python 3.10: Support dropped in 1.24.1 (use latest compatible)\n \"onnxruntime>=1.17.0, <1.24.1 ; python_version == '3.10'\",\n # Python 3.9: Support dropped in 1.20.0\n \"onnxruntime>=1.17.0, <1.20.0 ; python_version <= '3.9'\",\n]\n\n\n[project.urls]\nHomepage = \"https://github.com/google/magika\"\nDocumentation = \"https://github.com/google/magika/blob/main/python/README.md\"\nRepository = \"https://github.com/google/magika/\"\nIssues = \"https://github.com/google/magika/issues\"\nChangelog = \"https://github.com/google/magika/blob/main/python/CHANGELOG.md\"\n\n\n[dependency-groups]\ndev = [\n \"mypy>=1.11.2\",\n \"ipython>=8.12.3\",\n \"pytest>=8.3.2\",\n \"ruff>=0.14.0\",\n \"tomli-w>=1.0.0\",\n \"tomli>=2.0.1\",\n \"tqdm>=4.67.1\",\n \"dacite>=1.9.2\",\n \"requests>=2.32.4\",\n]\n\n[build-system]\nrequires = [\"maturin>=1.12.2\"]\nbuild-backend = \"maturin\"\n\n[[tool.uv.index]]\nurl = \"https://pypi.org/simple\"\ndefault = true\n\n[tool.maturin]\npython-source = \"src\"\nmodule-name = \"magika\"\nbindings = \"bin\"\nmanifest-path = \"../rust/cli/Cargo.toml\"\nlocked = true\n\n[tool.ruff.lint]\n# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.\n# Unlike Flake8, Ruff doesn't enable pycodestyle warnings (`W`) or\n# McCabe complexity (`C901`) by default.\nselect = [\"D\", \"E4\", \"E7\", \"E9\", \"F\", \"I001\"]\nignore = [\"D105\"]\n\n[tool.ruff.lint.per-file-ignores]\n\"scripts/*\" = [\"D\"]\n\"tests/*\" = [\"D\"]\n\n[tool.ruff.lint.pydocstyle]\nconvention = \"google\"\n\n[tool.ruff.format]\ndocstring-code-format = true\n" }, { "path": "python/pytest.ini", "content": "[pytest]\nlog_cli = 1\nlog_level = WARNING\n\nmarkers =\n smoketest\n slow\n\n" }, { "path": "python/scripts/check_changelog.sh", "content": "#!/bin/bash\n# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nset -euo pipefail\n\nCHANGED_FILES=$(git diff --name-only \"origin/${1:-main}\"...HEAD)\n\necho \"Checking python/* for undocumented changes...\"\n\nif echo \"$CHANGED_FILES\" | grep -qE '^python/.*$'; then\n if ! echo \"$CHANGED_FILES\" | grep -qE '^python/CHANGELOG\\.md$'; then\n echo \"::warning title=Changelog Missing::Some changes in the Python package are not documented in python/CHANGELOG.md\"\n fi\nfi\n" }, { "path": "python/scripts/check_copyright.py", "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# https://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport re\nimport subprocess\nimport sys\nfrom pathlib import Path\n\nimport click\n\nEXTENSIONS = (\".py\", \".sh\", \".ts\", \".rs\")\nRELEVANT_DIRS = (\"python/\", \"js/\", \"rust/\")\nEXCLUDE_DIRS = (\"js/simple_examples\",)\nCOPYRIGHT_PATTERN = re.compile(r\"Copyright\", re.IGNORECASE)\n\n\n@click.command()\ndef main():\n \"\"\"\n Check for missing copyright headers in the target files.\n\n The command performs the following steps:\n - Retrieves the list of git-tracked files meeting the criteria.\n - Checks each file for a valid copyright header.\n - Prints any files that are missing the header.\n - Exits with status code 1 if any file is missing the header; otherwise exits successfully.\n \"\"\"\n\n error_files = []\n for file_path in get_relevant_files_paths():\n if not has_copyright(file_path):\n error_files.append(file_path)\n\n if error_files:\n click.secho(\"Missing copyright in:\", fg=\"red\", bold=True)\n for file in error_files:\n click.echo(f\"- {file}\")\n sys.exit(1)\n else:\n click.secho(\"All files have valid copyright.\", fg=\"green\")\n\n\ndef get_relevant_files_paths() -> list[Path]:\n \"\"\"Finds relevant, tracked files using Git.\n\n Filters the output of `git ls-files` based on three criteria defined\n in global constants:\n - File must have an extension in `EXTENSIONS`.\n - File path must be within a directory in `DIRECTORIES`.\n - File path must NOT contain the string in `EXCLUDED_PATH`.\n \"\"\"\n\n repo_root_dir = Path(__file__).parent.parent.parent.resolve()\n assert (repo_root_dir / \".git\").is_dir()\n\n paths = []\n try:\n result = subprocess.run(\n [\"git\", \"ls-files\", str(repo_root_dir)],\n capture_output=True,\n text=True,\n check=True,\n cwd=str(repo_root_dir),\n )\n for rel_path_str in result.stdout.strip().splitlines():\n path = repo_root_dir / rel_path_str\n if (\n path.is_file()\n and path.stat().st_size > 0\n and path.suffix in EXTENSIONS\n and rel_path_str.startswith(RELEVANT_DIRS)\n and not rel_path_str.startswith(EXCLUDE_DIRS)\n ):\n paths.append(path)\n except subprocess.CalledProcessError as e:\n click.secho(f\"Git command failed: {e}\", fg=\"red\", bold=True)\n sys.exit(2)\n\n return paths\n\n\ndef has_copyright(path: Path) -> bool:\n \"\"\"Checks if a file contains a copyright notice within the first N lines.\n\n Returns True if found, False otherwise.\n \"\"\"\n\n with path.open(\"r\", encoding=\"utf-8\") as f:\n for _ in range(5):\n line = f.readline()\n if COPYRIGHT_PATTERN.search(line):\n return True\n return False\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "python/scripts/check_documentation.py", "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\n\"\"\"Performs a number of sanity checks on Magika's documentation.\"\"\"\n\nfrom __future__ import annotations\n\nimport re\nimport sys\nfrom dataclasses import dataclass\nfrom pathlib import Path\n\nimport click\nimport requests\n\nREPO_ROOT_DIR = Path(__file__).parent.parent.parent\nassert REPO_ROOT_DIR.is_dir() and (REPO_ROOT_DIR / \".git\").is_dir()\n\n\nIGNORE_PREFIX_PATTERNS = [\n \".mypy_cache\",\n \".pytest_cache\",\n \".ruff_cache\",\n \"python/.venv\",\n \"python/dist\",\n \"website/node_modules\",\n \"website/dist\",\n \"js/node_modules\",\n \"js/dist\",\n \"website-ng/node_modules\",\n \"website-ng/dist\",\n]\n\n\n@click.command()\n@click.option(\"-v\", \"--verbose\", is_flag=True)\ndef main(verbose: bool) -> None:\n with_errors = False\n\n success = check_versions_are_up_to_date()\n if not success:\n with_errors = True\n\n success = check_markdown_links(verbose)\n if not success:\n with_errors = True\n\n if with_errors:\n print(\"There was at least one error.\")\n sys.exit(1)\n\n print(\"Everything looks good.\")\n\n\ndef check_versions_are_up_to_date() -> bool:\n \"\"\"Checks that the mentioned latest versions and models are up to date.\n Returns True if everything is good, False otherwise.\"\"\"\n\n # Actual last versions and models\n rust_cli_latest_stable_version = get_max_stable_version_for_crate(\"magika-cli\")\n rust_lib_latest_stable_version = get_max_stable_version_for_crate(\"magika\")\n rust_default_model_name = get_rust_default_model_name()\n python_latest_stable_version = get_python_latest_stable_version()\n python_default_model_name = get_python_default_model_name()\n javascript_latest_stable_version = get_latest_version_for_npm_package(\"magika\")\n javascript_default_model_name = get_javascript_default_model_name()\n demo_model_name = get_demo_model_name()\n\n expected_table = [\n (rust_cli_latest_stable_version, rust_default_model_name),\n (python_latest_stable_version, python_default_model_name),\n (javascript_latest_stable_version, javascript_default_model_name),\n (rust_lib_latest_stable_version, rust_default_model_name),\n (\"-\", demo_model_name),\n (\"-\", \"-\"),\n ]\n\n # Extract documented last versions and models\n bindings_overview_path = (\n REPO_ROOT_DIR\n / \"website-ng\"\n / \"src\"\n / \"content\"\n / \"docs\"\n / \"cli-and-bindings\"\n / \"overview.md\"\n )\n assert bindings_overview_path.is_file()\n lines = bindings_overview_path.read_text().splitlines()\n parsed_table = []\n for line in lines:\n # This is a hack to parse the table in the binding's overview, but it is\n # simple and self-contained enough to not cause problems. And we'll\n # notice immediately if things break.\n if line.startswith(\"| [\"):\n cols = line.split(\"|\")\n latest_version = cols[3].strip(\" `\")\n default_model = cols[4].strip()\n if default_model != \"-\":\n default_model = default_model.split(\"]\")[0].split(\"[\")[1].strip(\" `\")\n parsed_table.append((latest_version, default_model))\n\n if expected_table == parsed_table:\n return True\n else:\n print(\n f\"ERROR: Found stale information in binding's overview table:\\n{expected_table=}\\n{parsed_table=}\"\n )\n return False\n\n\ndef get_python_latest_stable_version() -> str:\n res = requests.get(\"https://pypi.org/pypi/magika/json\")\n assert res.status_code == 200\n latest_stable_version = res.json().get(\"info\", {}).get(\"version\", None)\n assert latest_stable_version is not None\n return latest_stable_version\n\n\ndef get_python_default_model_name() -> str:\n magika_path = REPO_ROOT_DIR / \"python\" / \"src\" / \"magika\" / \"magika.py\"\n return extract_one_match_with_regex_from_file(\n magika_path, '_DEFAULT_MODEL_NAME = \"([a-zA-Z0-9_]+)\"'\n )\n\n\ndef get_javascript_default_model_name() -> str:\n magika_path = REPO_ROOT_DIR / \"js\" / \"magika.ts\"\n return extract_one_match_with_regex_from_file(\n magika_path, 'static MODEL_VERSION = \"([a-zA-Z0-9_]+)\";'\n )\n\n\ndef get_demo_model_name() -> str:\n \"\"\"Get the model name used by the demo.\"\"\"\n\n demo_path = (\n REPO_ROOT_DIR / \"website-ng\" / \"src\" / \"components\" / \"MagikaDemo.svelte\"\n )\n return extract_one_match_with_regex_from_file(\n demo_path, 'const MAGIKA_MODEL_VERSION = \"([a-zA-Z0-9_]+)\";'\n )\n\n\ndef get_rust_default_model_name() -> str:\n model_symlink_path = REPO_ROOT_DIR / \"rust\" / \"gen\" / \"model\"\n assert model_symlink_path.is_symlink()\n return model_symlink_path.readlink().name\n\n\ndef check_markdown_links(verbose: bool) -> bool:\n \"\"\"Checks that links in Markdown files are OK. Returns True if everything is\n good, False otherwise.\"\"\"\n\n with_errors = False\n for path in enumerate_markdown_files_in_dir(Path(\".\")):\n if verbose:\n print(f\"Analyzing file: {path}\")\n for ui in extract_uris_infos_from_file(\n path,\n verbose=verbose,\n ):\n if not ui.is_valid:\n with_errors = True\n print(\n f\"ERROR: {path.relative_to(REPO_ROOT_DIR)} has non-valid uri: {ui.uri}\"\n )\n\n # For python/README.md (which is used on pypi), we also check that\n # the URIs are either pointing to an external resource or are pure\n # anchors.\n if str(path.relative_to(REPO_ROOT_DIR)) == \"python/README.md\":\n if not ui.is_external and not ui.is_pure_anchor:\n with_errors = True\n print(\n f\"ERROR: {path.relative_to(REPO_ROOT_DIR)}, in python/, has a non-external uri: {ui.uri}\"\n )\n # Same for js/README.md, which ends up on npm.\n if str(path.relative_to(REPO_ROOT_DIR)) == \"js/README.md\":\n if not ui.is_external and not ui.is_pure_anchor:\n with_errors = True\n print(\n f\"ERROR: {path.relative_to(REPO_ROOT_DIR)}, in python/, has a non-external uri: {ui.uri}\"\n )\n\n success = with_errors is False\n return success\n\n\ndef enumerate_markdown_files_in_dir(rel_dir: Path) -> list[Path]:\n if rel_dir.is_absolute():\n print(f\"{rel_dir} is not relative\")\n sys.exit(1)\n a_dir = REPO_ROOT_DIR / rel_dir\n assert a_dir.is_dir()\n paths: list[Path] = []\n for path in sorted(a_dir.rglob(\"*.md\")):\n should_ignore = False\n for exclude_prefix_pattern in IGNORE_PREFIX_PATTERNS:\n if str(path.relative_to(REPO_ROOT_DIR)).startswith(exclude_prefix_pattern):\n should_ignore = True\n break\n if not should_ignore:\n paths.append(path)\n return paths\n\n\ndef extract_uris_infos_from_file(path: Path, verbose: bool) -> list[UriInfo]:\n uri_regex = r\"\\[.*?\\]\\((.*?)\\)\"\n uris: list[str] = re.findall(uri_regex, path.read_text())\n\n uris_infos: list[UriInfo] = []\n for uri in uris:\n if verbose:\n print(f\"Analyzing uri: {uri}\")\n\n is_external = uri.startswith(\"http://\") or uri.startswith(\"https://\")\n is_valid = None\n is_pure_anchor = None\n is_insecure = None\n\n if is_external:\n # We treat links pointing to our own repo in a special way. For\n # simplicity, we only deal with links pointing to the main branch.\n repo_main_prefix_url = \"https://github.com/google/magika/blob/main/\"\n if uri.startswith(repo_main_prefix_url):\n rel_path = uri.removeprefix(repo_main_prefix_url)\n assert rel_path.find(\"#\") == -1, (\n \"Local links with anchors not supported yet\"\n )\n abs_path = REPO_ROOT_DIR / rel_path\n is_valid = abs_path.is_file()\n else:\n # We mark any other external link as valid, as actually checking\n # it's too much of a pain.\n is_valid = True\n is_pure_anchor = False\n if uri.startswith(\"http://\"):\n is_insecure = True\n print(f\"WARNING: {uri} is not using https\")\n else:\n is_insecure = False\n else:\n is_insecure = False\n if uri.startswith(\"#\"):\n is_valid = True\n is_pure_anchor = True\n else:\n is_pure_anchor = False\n if Path(uri).is_absolute():\n website_files_dir = (\n REPO_ROOT_DIR\n / \"website-ng\"\n / \"src\"\n / \"content\"\n / \"docs\"\n / uri.removeprefix(\"/magika/\")\n )\n md_path = website_files_dir.with_suffix(\".md\")\n mdx_path = website_files_dir.with_suffix(\".mdx\")\n public_path = (\n REPO_ROOT_DIR / \"website-ng\" / \"public\" / Path(uri).name\n )\n if (\n website_files_dir.is_dir()\n or md_path.is_file()\n or mdx_path.is_file()\n or public_path.is_file()\n ):\n is_valid = True\n else:\n is_valid = False\n else:\n if uri.find(\"#\") >= 0:\n # This URI is not a pure anchor, but it does have an\n # anchor. We remove it so that we can check whether the\n # file exists or not.\n rel_file_path = uri.split(\"#\")[0]\n else:\n rel_file_path = uri\n abs_path = path.parent / rel_file_path\n is_valid = abs_path.is_file() or abs_path.is_dir()\n\n assert is_valid is not None\n assert is_pure_anchor is not None\n assert is_insecure is not None\n\n uris_infos.append(\n UriInfo(\n uri=uri,\n is_external=is_external,\n is_valid=is_valid,\n is_pure_anchor=is_pure_anchor,\n is_insecure=is_insecure,\n )\n )\n\n return uris_infos\n\n\ndef get_max_stable_version_for_crate(crate_name: str) -> str:\n url = f\"https://crates.io/api/v1/crates/{crate_name}\"\n r = requests.get(url)\n crate_info = r.json()\n return crate_info[\"crate\"][\"max_stable_version\"]\n\n\ndef get_latest_version_for_npm_package(package_name: str) -> str:\n url = f\"https://registry.npmjs.org/{package_name}/latest\"\n r = requests.get(url)\n crate_info = r.json()\n return crate_info[\"version\"]\n\n\ndef extract_one_match_with_regex_from_file(path: Path, regex: str) -> str:\n \"\"\"Extract one (and only one!) match with a regex from a file.\n\n Raises an error in case of zero or more than one hits.\n \"\"\"\n\n assert path.is_file()\n matching_str = None\n for line in path.read_text().splitlines():\n m = re.fullmatch(regex, line.strip())\n if m:\n # If we already found something, there is a bug somewhere\n assert matching_str is None\n matching_str = m.group(1)\n\n assert matching_str is not None\n return matching_str\n\n\n@dataclass(kw_only=True)\nclass UriInfo:\n uri: str\n is_external: bool\n is_valid: bool\n is_pure_anchor: bool\n is_insecure: bool\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "python/scripts/check_source.sh", "content": "#!/bin/bash\n# Copyright 2023-2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# https://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# From https://stackoverflow.com/a/246128\nSCRIPT_DIR=$( cd -- \"$( dirname -- \"${BASH_SOURCE[0]}\" )\" &> /dev/null && pwd )\n\nPYTHON_ROOT_DIR=$SCRIPT_DIR/../\n\npushd $PYTHON_ROOT_DIR > /dev/null\n\necho \"Running ruff...\"\nruff check\n\necho \"Running mypy...\"\nmypy src/magika tests\n\npopd > /dev/null\n" }, { "path": "python/scripts/generate_reference.py", "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport subprocess\nfrom pathlib import Path\n\nimport click\n\npython_root_dir = Path(__file__).parent.parent\n\n\n@click.command()\ndef main():\n test_scripts_paths = [\n python_root_dir / \"tests\" / \"test_features_extraction_vs_reference.py\",\n python_root_dir / \"tests\" / \"test_inference_vs_reference.py\",\n ]\n\n for test_script_path in test_scripts_paths:\n assert test_script_path.is_file()\n cmd = [\n \"uv\",\n \"run\",\n str(test_script_path),\n \"generate-tests\",\n ]\n\n print(f\"Running CMD: {' '.join(cmd)}\")\n subprocess.run(\n cmd,\n cwd=python_root_dir,\n check=True,\n )\n\n print(\"Everything went good.\")\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "python/scripts/pre_release_check.py", "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\n\"\"\"\nIt performs a number of checks to determine whether the package is ready for release.\n\"\"\"\n\nimport re\nimport subprocess\nimport sys\n\nimport click\n\nimport magika\n\n\n@click.command()\n@click.option(\n \"--expected-version\",\n default=\"\",\n help=\"Expected version string (e.g., '1.2.3'). If provided, checks will be validated against this value.\",\n)\n@click.option(\n \"--report-only\",\n is_flag=True,\n help=\"Print errors without failing. (Default: Fails on errors)\",\n)\n@click.option(\n \"--check-pip-show-package-version/--no-check-pip-show-package-version\",\n is_flag=True,\n default=True,\n help=\"Enable/disable version check via 'pip show'. (Default: Enabled)\",\n)\n@click.option(\n \"--use-python-client\",\n is_flag=True,\n help=\"Use the Python client instead of Rust client. (Default: False)\",\n)\ndef main(\n expected_version: str,\n report_only: bool,\n check_pip_show_package_version: bool,\n use_python_client: bool,\n) -> None:\n \"\"\"Checks versions consistency for the `magika` package.\"\"\"\n\n if report_only:\n click.echo('Running in \"report only\" mode.')\n if not check_pip_show_package_version:\n click.echo(\"Skipping checking package version via pip show.\")\n if use_python_client:\n click.echo(\"Using python client instead of Rust client.\")\n\n strict_mode = not report_only\n if strict_mode:\n if expected_version == \"\":\n click.secho(\n \"ERROR: when not using --report-only, --expected-version is required.\"\n )\n sys.exit(1)\n\n with_errors = False\n\n # Get the versions\n module_version = getattr(magika, \"__version__\", \"\")\n try:\n instance_version = magika.Magika().get_module_version()\n except Exception:\n instance_version = \"\"\n\n if check_pip_show_package_version:\n pip_show_package_version = get_magika_package_version_via_pip_show()\n else:\n pip_show_package_version = \"\"\n\n if use_python_client:\n cli_version = instance_version\n else:\n cli_version = get_rust_cli_version()\n\n if module_version == \"\":\n click.echo(\"ERROR: failed to get module_version.\")\n with_errors = True\n if instance_version == \"\":\n click.echo(\"ERROR: failed to get instance_version.\")\n with_errors = True\n if check_pip_show_package_version and pip_show_package_version == \"\":\n click.echo(\"ERROR: failed to get pip_show_package_version.\")\n with_errors = True\n if cli_version == \"\":\n click.echo(\"ERROR: failed to get cli_version.\")\n with_errors = True\n\n click.echo(\n f\"Extracted versions: {expected_version=}, {module_version=}, {instance_version=}, {pip_show_package_version=}, {cli_version=}.\"\n )\n\n if expected_version != \"\" and module_version != expected_version:\n click.echo(f\"ERROR: {module_version=} != {expected_version=}\")\n with_errors = True\n\n if module_version != instance_version:\n click.echo(f\"ERROR: {instance_version=} != {module_version=}\")\n with_errors = True\n\n if check_pip_show_package_version:\n if module_version != pip_show_package_version:\n click.echo(f\"ERROR: {module_version=} != {pip_show_package_version=}\")\n with_errors = True\n\n # From now on, we assume all the python-related versions are the same. If\n # they are not, we would have at least one error above.\n\n if not is_valid_python_version(module_version):\n click.echo(f\"ERROR: {module_version=} is not a valid python version.\")\n with_errors = True\n\n if module_version.endswith(\"-dev\") or cli_version.endswith(\"-dev\"):\n click.echo(\"ERROR: One of the versions is a -dev version.\")\n with_errors = True\n\n if cli_version.endswith(\"-rc\") and not module_version.endswith(\"-rc\"):\n click.echo(\"ERROR: The CLI has an -rc version, but the python module does not.\")\n with_errors = True\n\n if with_errors:\n click.secho(\"There was at least one error.\", fg=\"red\")\n if strict_mode:\n sys.exit(1)\n else:\n click.secho(\"All tests pass!\", fg=\"green\")\n\n\ndef get_rust_cli_version() -> str:\n \"\"\"Get the version of the Rust CLI `magika`.\n\n Returns an empty string (\"\") if an error is encountered.\n \"\"\"\n try:\n result = subprocess.run(\n [\"magika\", \"--version\"], capture_output=True, text=True, check=True\n )\n parts = result.stdout.strip().split()\n if len(parts) < 2:\n click.echo(\"ERROR: Could not parse CLI version output.\")\n return \"\"\n cli_version = parts[1]\n return cli_version\n except subprocess.CalledProcessError as e:\n click.echo(f\"ERROR: Could not retrieve CLI version: {e}\")\n return \"\"\n\n\ndef get_magika_package_version_via_pip_show() -> str:\n try:\n r = subprocess.run(\n [\"python3\", \"-m\", \"pip\", \"show\", \"magika\"], capture_output=True, text=True\n )\n lines = r.stdout.strip().split(\"\\n\")\n for line in lines:\n if line.startswith(\"Version: \"):\n return line.split(\": \", 1)[1]\n click.echo(\n f\"ERROR: Could not extract the package version via pip show. Output from pip show:\\nstdout={r.stdout}\\n\\nstderr={r.stderr}\"\n )\n return \"\"\n except subprocess.CalledProcessError as e:\n click.echo(f\"ERROR: Could not retrieve package version via pip show: {e}\")\n return \"\"\n\n\ndef is_valid_python_version(version: str) -> bool:\n # Regex from PEP440: '[N!]N(.N)*[{a|b|rc}N][.postN][.devN]'\n PEP440_CANONICAL_REGEX = re.compile(\n r\"\"\"\n^\n# Optional Epoch segment (e.g., 1!)\n(?P\\d+!)?\n\n# Required Release segment (e.g., 1.2.3)\n(?P[0-9]+(?:\\.[0-9]+)*)\n\n# Optional Pre-release segment (e.g., a1, b2, rc3)\n(?P
\n    (?:a|b|rc)\n    [0-9]+\n)?\n\n# Optional Post-release segment (e.g., .post4)\n(?P\n    (?:\\.post[0-9]+)\n)?\n\n# Optional Development release segment (e.g., .dev5)\n(?P\n    (?:\\.dev[0-9]+)\n)?\n$\n\"\"\",\n        re.VERBOSE | re.IGNORECASE,\n    )\n    return PEP440_CANONICAL_REGEX.fullmatch(version) is not None\n\n\ndef test_is_valid_python_version() -> None:\n    assert is_valid_python_version(\"1.2.3\") is True\n    assert is_valid_python_version(\"1.2.3.rc\") is False\n    assert is_valid_python_version(\"1.2.3.rc0\") is False\n    assert is_valid_python_version(\"1.2.3rc0\") is True\n    assert is_valid_python_version(\"1.2.3rc1\") is True\n    assert is_valid_python_version(\"1.2.3-dev\") is False\n    assert is_valid_python_version(\"1.2.3.dev0\") is True\n    assert is_valid_python_version(\"1.2.3-dev0\") is False\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/scripts/prepare_pyproject_for_pure_python_wheel.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nfrom pathlib import Path\n\nimport tomli\nimport tomli_w\n\n\ndef main() -> None:\n    pyproject_toml_path = Path(__file__).parent.parent / \"pyproject.toml\"\n\n    pyproject_content = tomli.loads(pyproject_toml_path.read_text())\n\n    # Remove entry about maturin, we don't need it\n    _ = pyproject_content[\"tool\"].pop(\"maturin\")\n\n    # Tell uv we want to use the hatchling build system\n    pyproject_content[\"build-system\"] = {\n        \"requires\": [\"hatchling\"],\n        \"build-backend\": \"hatchling.build\",\n    }\n\n    # Make the python's magika client available as a script\n    pyproject_content[\"project\"][\"scripts\"] = {\n        \"magika-python-client\": \"magika.cli.magika_client:main\",\n        \"magika\": \"magika.cli.magika_rust_client_not_found_warning:main\",\n    }\n\n    pyproject_toml_path.write_text(tomli_w.dumps(pyproject_content))\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/scripts/run_quick_test_magika_cli.py",
    "content": "#!/usr/bin/env python\n# Copyright 2023-2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     https://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"\nThis script assumes that the `magika` python package has been already installed,\nand that the `magika` client is available in the PATH. The script tests that the\n`magika` client appears functional.\n\nThis script should only rely on dependencies installed with `pip install\nmagika`; this script is used as part of \"build & test package\" github action,\nand the dev dependencies are not available.\n\"\"\"\n\nimport subprocess\nimport sys\nfrom pathlib import Path\nfrom typing import Optional\n\nimport click\n\n\n@click.command()\n@click.option(\n    \"--client-path\",\n    type=click.Path(exists=False, resolve_path=False),\n)\ndef main(client_path: Optional[Path]) -> None:\n    \"\"\"Tests the Rust or Python Magika client. By default, it runs \"magika\"\n    (expected in PATH). Use --client-path to specify a different client\n    executable. Note that \"client_path\" may not point to a valid file, but still\n    be a valid target to run as it may be found in the PATH.\n    \"\"\"\n\n    basic_tests_dir = (\n        Path(__file__).resolve().parent.parent.parent / \"tests_data\" / \"basic\"\n    )\n    assert basic_tests_dir.is_dir()\n\n    if client_path is None:\n        client_path = Path(\"magika\")\n    print(f'Testing client: \"{client_path}\"')\n\n    p = subprocess.run([str(client_path), \"--version\"], capture_output=True, text=True)\n    print(f'Output of \"magika --version\": {p.stdout.strip()}')\n\n    p = subprocess.run(\n        [str(client_path), \"-r\", \"--label\", \"--no-colors\", str(basic_tests_dir)],\n        capture_output=True,\n        text=True,\n    )\n\n    if p.returncode != 0:\n        print(\"ERROR: magika CLI exited with non-zero status.\")\n        print(f\"stdout:\\n{p.stdout}\\n\" + \"-\" * 40)\n        print(f\"stderr:\\n{p.stderr}\\n\" + \"-\" * 40)\n        sys.exit(1)\n\n    if p.stderr != \"\":\n        print(f\"WARNING: p.stderr not empty: {p.stderr}\")\n\n    with_error = False\n    lines = p.stdout.split(\"\\n\")\n    for line in lines:\n        line = line.strip()\n        if line == \"\":\n            continue\n        file_path_str, file_output_str = line.split(\": \", 1)\n        file_path = Path(file_path_str)\n        output_label = file_output_str.strip().split(\" \", 1)[0]\n        expected_label = file_path.parent.name\n        if expected_label != output_label:\n            with_error = True\n            print(\n                f\"ERROR: Misprediction for {file_path}: expected_label={expected_label}, output_label={output_label}\"\n            )\n\n    if with_error:\n        print(\"ERROR: There was at least one misprediction\")\n        sys.exit(1)\n\n    print(\"All examples were predicted correctly\")\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/scripts/run_quick_test_magika_module.py",
    "content": "#!/usr/bin/env python\n# Copyright 2023-2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     https://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"\nThis script should only rely on dependencies installed with `pip install\nmagika`; this script is used as part of \"build & test package\" github action,\nand the dev dependencies are not available.\n\"\"\"\n\nimport statistics\nimport sys\nimport time\nfrom pathlib import Path\n\nimport click\n\nfrom magika import ContentTypeLabel, Magika, PredictionMode\n\n\n@click.command()\n@click.option(\"--print-inference-stats\", is_flag=True, help=\"Print inference stats.\")\n@click.option(\"--repeat\", default=1, help=\"Number of times to run the test set.\")\ndef main(print_inference_stats: bool, repeat: int) -> None:\n    m = Magika(prediction_mode=PredictionMode.HIGH_CONFIDENCE)\n\n    print(f\"Magika instance details: {m}\")\n\n    res = m.identify_bytes(b\"text\")\n    assert res.dl.label == ContentTypeLabel.UNDEFINED\n    assert res.output.label == ContentTypeLabel.TXT\n    assert res.score == 1.0\n\n    res = m.identify_bytes(b\"\\xff\\xff\\xff\")\n    assert res.dl.label == ContentTypeLabel.UNDEFINED\n    assert res.output.label == ContentTypeLabel.UNKNOWN\n    assert res.score == 1.0\n\n    basic_tests_dir = (\n        Path(__file__).parent.parent.parent / \"tests_data\" / \"basic\"\n    ).resolve()\n\n    files_paths = sorted(filter(lambda p: p.is_file(), basic_tests_dir.rglob(\"*\")))\n\n    latencies = []\n\n    with_error = False\n    for i in range(repeat):\n        for file_path in files_paths:\n            start_time = time.perf_counter()\n            res = m.identify_path(file_path)\n            end_time = time.perf_counter()\n            latencies.append((end_time - start_time) * 1000)\n\n            # Check for misprediction only on the first run.\n            if i == 0:\n                output_label = res.output.label\n                expected_label = file_path.parent.name\n                if expected_label != output_label:\n                    with_error = True\n                    print(\n                        f\"ERROR: Misprediction for {file_path}: expected_label={expected_label}, output_label={output_label}\"\n                    )\n\n    if with_error:\n        print(\"ERROR: There was at least one misprediction\")\n        sys.exit(1)\n\n    print(\"All examples were predicted correctly\")\n\n    if print_inference_stats and latencies:\n        print(f\"Inference stats over {len(latencies)} files (repeat={repeat}):\")\n        print(f\"  Min: {min(latencies):.4f} ms\")\n        print(f\"  Max: {max(latencies):.4f} ms\")\n        print(f\"  Mean: {statistics.mean(latencies):.4f} ms\")\n        print(f\"  Median: {statistics.median(latencies):.4f} ms\")\n        print(f\"  Total: {sum(latencies):.4f} ms\")\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/scripts/sync.py",
    "content": "#!/usr/bin/env python3\n# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport enum\nimport json\nimport shutil\nimport sys\nfrom pathlib import Path\nfrom typing import Optional\n\nimport click\n\nMODELS_NAMES_TO_INCLUDE_IN_PYTHON_PACKAGE = [\n    \"standard_v3_3\",\n]\n\nREPO_ROOT_DIR = Path(__file__).parent.parent.parent\nassert REPO_ROOT_DIR.is_dir() and (REPO_ROOT_DIR / \".git\").is_dir()\n\nASSETS_DIR = REPO_ROOT_DIR / \"assets\"\nassert ASSETS_DIR.is_dir()\n\nCONTENT_TYPES_KB_PATH = ASSETS_DIR / \"content_types_kb.min.json\"\nassert CONTENT_TYPES_KB_PATH.is_file()\n\nASSETS_MODELS_DIR = ASSETS_DIR / \"models\"\nassert ASSETS_MODELS_DIR.is_dir()\n\nPYTHON_ROOT_DIR = REPO_ROOT_DIR / \"python\"\nassert PYTHON_ROOT_DIR.is_dir()\n\nPYTHON_CONTENT_TYPES_KB_PATH = (\n    PYTHON_ROOT_DIR / \"src\" / \"magika\" / \"config\" / \"content_types_kb.min.json\"\n)\n\nPYTHON_MODELS_DIR = PYTHON_ROOT_DIR / \"src\" / \"magika\" / \"models\"\nassert PYTHON_MODELS_DIR.is_dir()\n\nPYTHON_CONTENT_TYPES_LABELS_PY_PATH = (\n    PYTHON_ROOT_DIR / \"src\" / \"magika\" / \"types\" / \"content_type_label.py\"\n)\n\nJS_ROOT_DIR = REPO_ROOT_DIR / \"js\"\nassert PYTHON_ROOT_DIR.is_dir()\n\nPYTHON_CONTENT_TYPES_KB_PATH = (\n    PYTHON_ROOT_DIR / \"src\" / \"magika\" / \"config\" / \"content_types_kb.min.json\"\n)\n\nPYTHON_MODELS_DIR = PYTHON_ROOT_DIR / \"src\" / \"magika\" / \"models\"\nassert PYTHON_MODELS_DIR.is_dir()\n\nPYTHON_CONTENT_TYPES_LABELS_PY_PATH = (\n    PYTHON_ROOT_DIR / \"src\" / \"magika\" / \"types\" / \"content_type_label.py\"\n)\n\n\nclass Target(enum.StrEnum):\n    JS = \"js\"\n    PYTHON = \"python\"\n\n\n@click.command()\n@click.argument(\"target\", type=Target)\n@click.option(\n    \"--models-names\",\n    \"models_names_str\",\n    help=\"Comma-separated list of models names to import in the package\",\n)\ndef main(target: Target, models_names_str: Optional[str]) -> None:\n    if target == Target.PYTHON:\n        if models_names_str is None:\n            models_names = MODELS_NAMES_TO_INCLUDE_IN_PYTHON_PACKAGE\n        else:\n            models_names = list(map(lambda s: s.strip(), models_names_str.split(\",\")))\n\n        print(f\"Including these models in the python package: {models_names}\")\n\n        update_python_content_type_kb()\n        update_python_content_type_label_py()\n\n        print(f\"Deleting {PYTHON_MODELS_DIR}\")\n        shutil.rmtree(PYTHON_MODELS_DIR)\n        for model_name in models_names:\n            add_model_to_python_package(model_name)\n\n    elif target == Target.JS:\n        update_js_content_type_files()\n\n        # FIXME: the model is currently copied manually\n        print(\"WARNING: copying the model is currently NOT supported by this script\")\n\n\ndef update_python_content_type_kb() -> None:\n    print(\n        f\"Syncing python's content types KB: {CONTENT_TYPES_KB_PATH} => {PYTHON_CONTENT_TYPES_KB_PATH}\"\n    )\n    PYTHON_CONTENT_TYPES_KB_PATH.parent.mkdir(parents=True, exist_ok=True)\n    shutil.copy(CONTENT_TYPES_KB_PATH, PYTHON_CONTENT_TYPES_KB_PATH)\n\n\ndef add_model_to_python_package(model_name: str) -> None:\n    assets_model_dir = ASSETS_MODELS_DIR / model_name\n    if not assets_model_dir.is_dir():\n        print(f'ERROR: model \"{model_name} not found')\n        sys.exit(1)\n\n    python_model_dir = PYTHON_MODELS_DIR / model_name\n\n    print(f\"Adding model {assets_model_dir} => {python_model_dir}\")\n    shutil.copytree(assets_model_dir, python_model_dir)\n\n\nCONTENT_TYPE_LABEL_PY_SOURCE_PREFIX = \"\"\"\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nfrom magika.types.strenum import StrEnum\n\n# NOTE: DO NOT EDIT --- This file is automatically generated.\n\n\n# This is the list of all possible content types we know about; however, models\n# support a smaller subset of them. See model's README.md for details.\nclass ContentTypeLabel(StrEnum):\n\"\"\"\n\n\ndef update_python_content_type_label_py() -> None:\n    print(f\"Updating {PYTHON_CONTENT_TYPES_LABELS_PY_PATH}\")\n\n    kb = json.loads(CONTENT_TYPES_KB_PATH.read_text())\n\n    enum_body_lines = []\n    for ct_label_str in sorted(kb.keys()):\n        if ct_label_str[0].isdigit():\n            line = (\" \" * 4) + f'_{ct_label_str.upper()} = \"{ct_label_str}\"'\n        else:\n            line = (\" \" * 4) + f'{ct_label_str.upper()} = \"{ct_label_str}\"'\n        enum_body_lines.append(line)\n\n    out = (\n        CONTENT_TYPE_LABEL_PY_SOURCE_PREFIX.strip()\n        + \"\\n\"\n        + \"\\n\".join(enum_body_lines)\n        + \"\\n\"\n    )\n    out += (\n        \"\\n\"\n        + (\" \" * 4)\n        + (\n            \"\"\"\n    def __repr__(self) -> str:\n        return str(self)\n    \"\"\".strip()\n            + \"\\n\"\n        )\n    )\n\n    PYTHON_CONTENT_TYPES_LABELS_PY_PATH.write_text(out)\n\n\nCOPYRIGHT_AND_DONOT_EDIT_PREFIX = \"\"\"\n// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// NOTE: DO NOT EDIT --- This file is automatically generated by sync.py.\n\"\"\"\n\n\nCONTENT_TYPE_LABEL_TS_SOURCE_PREFIX = (\n    COPYRIGHT_AND_DONOT_EDIT_PREFIX.strip()\n    + \"\\n\\n\"\n    + \"\"\"\n// This is the list of all possible content types we know about; however, models\n// support a smaller subset of them. See model's README.md for details.\nexport enum ContentTypeLabel {\n\"\"\".strip()\n)\n\n\ndef update_js_content_type_files() -> None:\n    # Update content type labels enum\n    content_type_label_ts_path = JS_ROOT_DIR / \"src\" / \"content-type-label.ts\"\n\n    kb = json.loads(CONTENT_TYPES_KB_PATH.read_text())\n\n    enum_body_lines = []\n    for ct_label_str in sorted(kb.keys()):\n        if ct_label_str[0].isdigit():\n            line = (\" \" * 2) + f'_{ct_label_str.upper()} = \"{ct_label_str}\",'\n        else:\n            line = (\" \" * 2) + f'{ct_label_str.upper()} = \"{ct_label_str}\",'\n        enum_body_lines.append(line)\n\n    out = (\n        CONTENT_TYPE_LABEL_TS_SOURCE_PREFIX.strip()\n        + \"\\n\"\n        + \"\\n\".join(enum_body_lines)\n        + \"\\n\"\n        + \"}\\n\"\n    ).strip() + \"\\n\"\n\n    content_type_label_ts_path.write_text(out)\n    print(f\"Updated {content_type_label_ts_path}\")\n\n    # Update content types info\n    content_types_infos_ts_path = JS_ROOT_DIR / \"src\" / \"content-types-infos.ts\"\n    content_types_info_content = COPYRIGHT_AND_DONOT_EDIT_PREFIX.strip() + \"\\n\\n\"\n\n    content_types_info_content += (\n        \"\"\"\nimport { ContentTypeInfo } from \"./content-type-info\";\nimport { ContentTypeLabel } from \"./content-type-label\";\n\nexport type ContentTypesInfos = Record;\n\nexport const ContentTypesInfos = {\n  get: (): ContentTypesInfos => ({\n\"\"\".strip()\n        + \"\\n\"\n    )\n    for ct_label_str, ct_info in sorted(kb.items()):\n        if ct_label_str[0].isdigit():\n            ct_label_enum = f\"_{ct_label_str.upper()}\"\n        else:\n            ct_label_enum = ct_label_str.upper()\n        is_text = ct_info[\"is_text\"]\n        content_types_info_content += (\n            \"    \"\n            + f\"\"\"\n    [ContentTypeLabel.{ct_label_enum}]: {{\n      label: ContentTypeLabel.{ct_label_enum},\n      is_text: {\"true\" if is_text else \"false\"},\n    }},\n\"\"\".strip()\n            + \"\\n\"\n        )\n\n    content_types_info_content += \"  })\\n};\\n\"\n\n    content_types_infos_ts_path.write_text(content_types_info_content)\n    print(f\"Updated {content_types_infos_ts_path}\")\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/scripts/test_magika_model.py",
    "content": "#!/usr/bin/env python3\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\n\"\"\"\nThis script tests a given model against the tests cases, check whether the\npredictions are correct, and raise an error otherwise.\n\nThis is useful when evaluating new models.\n\"\"\"\n\nimport sys\nfrom pathlib import Path\n\nimport click\n\nimport magika\nfrom magika import colors\n\n\n@click.command()\n@click.argument(\"model_dir_or_name\")\ndef main(model_dir_or_name: str) -> None:\n    if Path(model_dir_or_name).is_dir():\n        model_dir = Path(model_dir_or_name)\n    else:\n        models_dir = Path(magika.__file__).parent / \"models\"\n        model_dir = models_dir / model_dir_or_name\n\n    if not model_dir.is_dir():\n        log_error(f\"{model_dir_or_name} is not a dir nor a model name\")\n        sys.exit(1)\n\n    m = magika.Magika(model_dir=model_dir)\n\n    with_error = False\n\n    tests_data_dir = Path(__file__).parent.parent.parent / \"tests_data\"\n    tests_dirs_names = [\"basic\", \"previous_missdetections\"]\n\n    for tests_dir_name in tests_dirs_names:\n        tests_dir = tests_data_dir / tests_dir_name\n        for test_path in tests_dir.rglob(\"*\"):\n            if not test_path.is_file():\n                continue\n\n            mr = m.identify_path(test_path)\n            assert mr.ok\n\n            predicted_content_type = mr.prediction.output.label\n            expected_content_type = test_path.parent.name\n            if predicted_content_type != expected_content_type:\n                with_error = True\n                log_error(\n                    f'{test_path} predicted as \"{predicted_content_type}\" (score: {mr.prediction.score:.4f}), expected \"{expected_content_type}\".'\n                )\n\n    if with_error:\n        log_error(\"There was at least one error.\")\n    else:\n        log_ok(\"All tests examples were predicted correctly.\")\n\n\ndef log_ok(msg: str) -> None:\n    print(f\"{colors.GREEN}{msg}{colors.RESET}\")\n\n\ndef log_error(msg: str) -> None:\n    print(f\"{colors.RED}ERROR: {msg}{colors.RESET}\")\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/src/magika/__init__.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D104\n\n\nfrom importlib.metadata import PackageNotFoundError, version\n\nfrom magika.magika import Magika\nfrom magika.types.content_type_info import ContentTypeInfo\nfrom magika.types.content_type_label import ContentTypeLabel\nfrom magika.types.magika_error import MagikaError\nfrom magika.types.magika_prediction import MagikaPrediction\nfrom magika.types.magika_result import MagikaResult\nfrom magika.types.overwrite_reason import OverwriteReason\nfrom magika.types.prediction_mode import PredictionMode\nfrom magika.types.status import Status\n\ntry:\n    __version__ = version(__name__)\nexcept PackageNotFoundError:\n    # Package is not installed (e.g., during development)\n    __version__ = \"unknown\"\n\n__all__ = [\n    \"ContentTypeInfo\",\n    \"ContentTypeLabel\",\n    \"Magika\",\n    \"MagikaError\",\n    \"MagikaPrediction\",\n    \"MagikaResult\",\n    \"OverwriteReason\",\n    \"PredictionMode\",\n    \"Status\",\n]\n"
  },
  {
    "path": "python/src/magika/cli/magika_client.py",
    "content": "#!/usr/bin/env python3\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Python-based command-line client for Magika.\n\nThis client serves as a fallback and is maintained for backward compatibility.\nIt is primarily useful for installations from the pure-Python wheel, especially\non platforms where the Rust-based binary client is not yet available.\n\nFor optimal performance, the Rust-based client is recommended.\n\"\"\"\n\nimport importlib.metadata\nimport json\nimport logging\nimport os\nimport sys\nfrom pathlib import Path\nfrom typing import List, Optional, Tuple\n\nimport click\n\nfrom magika import Magika, MagikaError, PredictionMode, colors\nfrom magika.logger import get_logger\nfrom magika.types import ContentTypeLabel, MagikaResult\nfrom magika.types.overwrite_reason import OverwriteReason\n\nVERSION = importlib.metadata.version(\"magika\")\n\nCONTACT_EMAIL = \"magika-dev@google.com\"\n\nCONTEXT_SETTINGS = dict(help_option_names=[\"-h\", \"--help\"])\n\nHELP_EPILOG = f\"\"\"\nMagika version: \"{VERSION}\"\\f\nDefault model: \"{Magika._get_default_model_name()}\"\n\nSend any feedback to {CONTACT_EMAIL} or via GitHub issues.\n\"\"\"\n\n\n@click.command(\n    context_settings=CONTEXT_SETTINGS,\n    epilog=HELP_EPILOG,\n)\n@click.argument(\n    \"file\",\n    type=click.Path(exists=False, readable=False, path_type=Path),\n    required=False,\n    nargs=-1,\n)\n@click.option(\n    \"-r\",\n    \"--recursive\",\n    is_flag=True,\n    help='When passing this option, magika scans every file within directories, instead of outputting \"directory\"',\n)\n@click.option(\"--json\", \"json_output\", is_flag=True, help=\"Output in JSON format.\")\n@click.option(\"--jsonl\", \"jsonl_output\", is_flag=True, help=\"Output in JSONL format.\")\n@click.option(\n    \"-i\",\n    \"--mime-type\",\n    \"mime_output\",\n    is_flag=True,\n    help=\"Output the MIME type instead of a verbose content type description.\",\n)\n@click.option(\n    \"-l\",\n    \"--label\",\n    \"label_output\",\n    is_flag=True,\n    help=\"Output a simple label instead of a verbose content type description. Use --list-output-content-types for the list of supported output.\",\n)\n@click.option(\n    \"-c\",\n    \"--compatibility-mode\",\n    \"magic_compatibility_mode\",\n    is_flag=True,\n    help=\"Compatibility mode: output is as close as possible to `file` and colors are disabled.\",\n)\n@click.option(\n    \"-s\",\n    \"--output-score\",\n    is_flag=True,\n    help=\"Output the prediction's score in addition to the content type.\",\n)\n@click.option(\n    \"-m\",\n    \"--prediction-mode\",\n    \"prediction_mode_str\",\n    type=click.Choice(PredictionMode.get_valid_prediction_modes(), case_sensitive=True),\n    default=PredictionMode.HIGH_CONFIDENCE,\n)\n@click.option(\n    \"--batch-size\", default=32, help=\"How many files to process in one batch.\"\n)\n@click.option(\n    \"--no-dereference\",\n    is_flag=True,\n    help=\"This option causes symlinks not to be followed. By default, symlinks are dereferenced.\",\n)\n@click.option(\n    \"--colors/--no-colors\",\n    \"with_colors\",\n    is_flag=True,\n    default=True,\n    help=\"Enable/disable use of colors.\",\n)\n@click.option(\"-v\", \"--verbose\", is_flag=True, help=\"Enable more verbose output.\")\n@click.option(\"-vv\", \"--debug\", is_flag=True, help=\"Enable debug logging.\")\n@click.option(\n    \"--version\", \"output_version\", is_flag=True, help=\"Print the version and exit.\"\n)\n@click.option(\n    \"--model-dir\",\n    type=click.Path(\n        exists=True, file_okay=False, dir_okay=True, resolve_path=True, path_type=Path\n    ),\n    help=\"Use a custom model.\",\n)\ndef main(\n    file: List[Path],\n    recursive: bool,\n    json_output: bool,\n    jsonl_output: bool,\n    mime_output: bool,\n    label_output: bool,\n    magic_compatibility_mode: bool,\n    output_score: bool,\n    prediction_mode_str: str,\n    batch_size: int,\n    no_dereference: bool,\n    with_colors: bool,\n    verbose: bool,\n    debug: bool,\n    output_version: bool,\n    model_dir: Optional[Path],\n) -> None:\n    \"\"\"Magika - Determine type of FILEs with deep-learning.\"\"\"\n    # click uses the name of the variable to determine how it will show up in\n    # the --help. Since we don't like to see \"file_paths\" in the help, we name\n    # the argument \"file\" (which is ugly) and we re-assign it as soon as we can.\n    files_paths = file\n\n    if magic_compatibility_mode:\n        # In compatibility mode we disable colors.\n        with_colors = False\n\n    _l = get_logger(use_colors=with_colors)\n\n    if verbose:\n        _l.setLevel(logging.INFO)\n    if debug:\n        _l.setLevel(logging.DEBUG)\n\n    if output_version:\n        _l.raw_print_to_stdout(\"Magika python client\")\n        _l.raw_print_to_stdout(f\"Magika version: {VERSION}\")\n        _l.raw_print_to_stdout(f\"Default model: {Magika._get_default_model_name()}\")\n        sys.exit(0)\n\n    if len(files_paths) == 0:\n        _l.error(\"You need to pass at least one path, or - to read from stdin.\")\n        sys.exit(1)\n\n    read_from_stdin = False\n    for p in files_paths:\n        if str(p) == \"-\":\n            read_from_stdin = True\n        elif not p.exists():\n            _l.error(f'File or directory \"{str(p)}\" does not exist.')\n            sys.exit(1)\n    if read_from_stdin:\n        if len(files_paths) > 1:\n            _l.error('If you pass \"-\", you cannot pass anything else.')\n            sys.exit(1)\n        if recursive:\n            _l.error('If you pass \"-\", recursive scan is not meaningful.')\n            sys.exit(1)\n\n    if batch_size <= 0 or batch_size > 512:\n        _l.error(\"Batch size needs to be greater than 0 and less or equal than 512.\")\n        sys.exit(1)\n\n    if json_output and jsonl_output:\n        _l.error(\"You should use either --json or --jsonl, not both.\")\n        sys.exit(1)\n\n    if int(mime_output) + int(label_output) + int(magic_compatibility_mode) > 1:\n        _l.error(\"You should use only one of --mime, --label, --compatibility-mode.\")\n        sys.exit(1)\n\n    if recursive:\n        # recursively enumerate files within directories\n        expanded_paths = []\n        for p in files_paths:\n            if p.exists():\n                if p.is_file():\n                    expanded_paths.append(p)\n                elif p.is_dir():\n                    expanded_paths.extend(sorted(p.rglob(\"*\")))\n            elif str(p) == \"-\":\n                # this is \"read from stdin\", that's OK\n                pass\n            else:\n                _l.error(f'File or directory \"{str(p)}\" does not exist.')\n                sys.exit(1)\n        # the resulting list may still include some directories; thus, we filter them out.\n        files_paths: List[Path] = list(filter(lambda x: not x.is_dir(), expanded_paths))  # type: ignore[no-redef]\n\n    _l.info(f\"Considering {len(files_paths)} files\")\n    _l.debug(f\"Files: {files_paths}\")\n\n    # Select an alternative model checking: 1) CLI option, 2) env variable.\n    # If none of these is set, model_dir is left to None, and the Magika module\n    # will use the default model.\n    if model_dir is None:\n        model_dir_str = os.environ.get(\"MAGIKA_MODEL_DIR\")\n        if model_dir_str is not None and model_dir_str.strip() != \"\":\n            model_dir = Path(model_dir_str)\n\n    try:\n        magika = Magika(\n            model_dir=model_dir,\n            prediction_mode=PredictionMode(prediction_mode_str),\n            no_dereference=no_dereference,\n            verbose=verbose,\n            debug=debug,\n            use_colors=with_colors,\n        )\n    except MagikaError as mr:\n        _l.error(str(mr))\n        sys.exit(1)\n\n    start_color = \"\"\n    end_color = \"\"\n\n    color_by_group = {\n        \"document\": colors.LIGHT_PURPLE,\n        \"executable\": colors.LIGHT_GREEN,\n        \"archive\": colors.LIGHT_RED,\n        \"audio\": colors.YELLOW,\n        \"image\": colors.YELLOW,\n        \"video\": colors.YELLOW,\n        \"code\": colors.LIGHT_BLUE,\n    }\n\n    # updated only when we need to output in JSON format\n    all_predictions: List[Tuple[Path, MagikaResult]] = []\n\n    batches_num = len(files_paths) // batch_size\n    if len(files_paths) % batch_size != 0:\n        batches_num += 1\n    for batch_idx in range(batches_num):\n        batch_files_paths = files_paths[\n            batch_idx * batch_size : (batch_idx + 1) * batch_size\n        ]\n\n        if _should_read_from_stdin(files_paths):\n            batch_predictions = [_get_magika_result_from_stdin(magika)]\n        else:\n            batch_predictions = magika.identify_paths(batch_files_paths)\n\n        if json_output:\n            # we do not stream the output for JSON output\n            all_predictions.extend(zip(batch_files_paths, batch_predictions))\n        elif jsonl_output:\n            for file_path, result in zip(batch_files_paths, batch_predictions):\n                _l.raw_print_to_stdout(json.dumps(result.asdict()))\n        else:\n            for file_path, result in zip(batch_files_paths, batch_predictions):\n                if result.ok:\n                    if mime_output:\n                        # If the user requested the MIME type, we use the mime type\n                        # regardless of the compatibility mode.\n                        output = result.prediction.output.mime_type\n                    elif label_output:\n                        output = str(result.prediction.output.label)\n                    else:  # human-readable description\n                        output = f\"{result.prediction.output.description} ({result.prediction.output.group})\"\n\n                        if (\n                            result.prediction.dl.label != ContentTypeLabel.UNDEFINED\n                            and result.prediction.dl.label\n                            != result.prediction.output.label\n                            and result.prediction.overwrite_reason\n                            == OverwriteReason.LOW_CONFIDENCE\n                        ):\n                            # It seems that we had a low-confidence prediction\n                            # from the model. Let's warn the user about our best\n                            # bet.\n                            output += (\n                                \" [Low-confidence model best-guess: \"\n                                f\"{result.prediction.dl.description} ({result.prediction.dl.group}), \"\n                                f\"score={result.prediction.score}]\"\n                            )\n\n                    if with_colors:\n                        start_color = color_by_group.get(\n                            result.prediction.output.group, colors.WHITE\n                        )\n                        end_color = colors.RESET\n                else:\n                    output = result.status\n                    start_color = \"\"\n                    end_color = \"\"\n\n                if output_score and result.ok:\n                    score = int(result.prediction.score * 100)\n                    _l.raw_print_to_stdout(\n                        f\"{start_color}{file_path}: {output} {score}%{end_color}\"\n                    )\n                else:\n                    _l.raw_print_to_stdout(\n                        f\"{start_color}{file_path}: {output}{end_color}\"\n                    )\n\n    if json_output:\n        _l.raw_print_to_stdout(\n            json.dumps(\n                [result.asdict() for (_, result) in all_predictions],\n                indent=4,\n            )\n        )\n\n\ndef _should_read_from_stdin(files_paths: List[Path]) -> bool:\n    return len(files_paths) == 1 and str(files_paths[0]) == \"-\"\n\n\ndef _get_magika_result_from_stdin(magika: Magika) -> MagikaResult:\n    content = sys.stdin.buffer.read()\n    result = magika.identify_bytes(content)\n    return result\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/src/magika/cli/magika_rust_client_not_found_warning.py",
    "content": "#!/usr/bin/env python\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Placeholder script for the primary 'magika' command-line interface.\n\nThis module serves as a fallback entry point for the 'magika' command. **It is\nincluded only in the pure-Python package.** If this script is executed, it\nindicates that the user has installed the pure-Python package and not the\npackage that contains the native binary.\n\nThe script explicitly notifies the user that they are not using the binary\nclient and guides them to use the alternative Python client (`$\nmagika-python-client`) or to seek support.\n\"\"\"\n\nimport sys\n\n\ndef main() -> None:  # noqa:  D103\n    message = \"\"\"\nWARNING: you have attempted to run `$ magika` (the Rust client), but this is not\navailable in the python package you installed, likely because magika pipeline\ndoes not currently build binary wheels compatible with your platform settings.\n\nIf you think this is a problem worth solving, please open an issue at\nhttps://github.com/google/magika.\n\nIn the meantime, you can use the old python magika client with `$ magika-python-client`.\n\"\"\"\n\n    print(message.strip())\n    sys.exit(1)\n\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "python/src/magika/colors.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100, D101, D102, D103, D107\n\n\n# Taken from https://en.wikipedia.org/wiki/ANSI_escape_code\n\nBLACK = \"\\033[0;30m\"\nRED = \"\\033[0;31m\"\nGREEN = \"\\033[0;32m\"\nYELLOW = \"\\033[0;33m\"\nBLUE = \"\\033[0;34m\"\nPURPLE = \"\\033[0;35m\"\nCYAN = \"\\033[0;36m\"\nLIGHT_GRAY = \"\\033[0;37m\"\n\nDARK_GRAY = \"\\033[1;30m\"\nLIGHT_RED = \"\\033[1;31m\"\nLIGHT_GREEN = \"\\033[1;32m\"\nLIGHT_YELLOW = \"\\033[1;33m\"\nLIGHT_BLUE = \"\\033[1;34m\"\nLIGHT_PURPLE = \"\\033[1;35m\"\nLIGHT_CYAN = \"\\033[1;36m\"\nWHITE = \"\\033[1;37m\"\n\nRESET = \"\\033[0;39m\"\n"
  },
  {
    "path": "python/src/magika/config/content_types_kb.min.json",
    "content": "{\"3gp\":{\"mime_type\":\"video/3gpp\",\"group\":\"video\",\"description\":\"3GPP multimedia file\",\"extensions\":[\"3gp\"],\"is_text\":false},\"3ds\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Nintendo 3DS roms\",\"extensions\":[\"3ds\"],\"is_text\":false},\"3dsx\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Nintendo 3DS homebrew\",\"extensions\":[\"3dsx\"],\"is_text\":false},\"3dsm\":{\"mime_type\":\"application/x-3ds\",\"group\":\"image\",\"description\":\"3D studio Max\",\"extensions\":[\"3ds\"],\"is_text\":false},\"3mf\":{\"mime_type\":\"application/vnd.ms-package.3dmanufacturing-3dmodel+xml\",\"group\":\"image\",\"description\":\"3D Manufacturing Format\",\"extensions\":[\"3mf\"],\"is_text\":false},\"abnf\":{\"mime_type\":\"text/plain\",\"group\":null,\"description\":\"augmented Backus\\u2013Naur form\",\"extensions\":[\"abnf\"],\"is_text\":false},\"ace\":{\"mime_type\":\"application/x-ace-compressed\",\"group\":\"archive\",\"description\":\"ACE archive\",\"extensions\":[\"ace\"],\"is_text\":false},\"ada\":{\"mime_type\":\"text/x-ada\",\"group\":\"code\",\"description\":\"ADA source\",\"extensions\":[],\"is_text\":false},\"aff\":{\"mime_type\":\"text/plain\",\"group\":null,\"description\":\"Hunspell Affix\",\"extensions\":[\"aff\"],\"is_text\":true},\"ai\":{\"mime_type\":\"application/pdf\",\"group\":\"document\",\"description\":\"Adobe Illustrator Artwork\",\"extensions\":[\"ai\"],\"is_text\":false},\"aidl\":{\"mime_type\":\"text/plain\",\"group\":null,\"description\":\"Android Interface Definition Language\",\"extensions\":[\"aidl\"],\"is_text\":true},\"algol68\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"a68\"],\"is_text\":false},\"ani\":{\"mime_type\":\"application/x-navi-animation\",\"group\":null,\"description\":\"Animated cursor\",\"extensions\":[\"ani\"],\"is_text\":false},\"apk\":{\"mime_type\":\"application/vnd.android.package-archive\",\"group\":\"executable\",\"description\":\"Android package\",\"extensions\":[\"apk\"],\"is_text\":false},\"applebplist\":{\"mime_type\":\"application/x-bplist\",\"group\":\"application\",\"description\":\"Apple binary property list\",\"extensions\":[\"bplist\",\"plist\"],\"is_text\":false},\"appledouble\":{\"mime_type\":\"multipart/appledouble\",\"group\":\"unknown\",\"description\":\"AppleDouble\",\"extensions\":[],\"is_text\":false},\"appleplist\":{\"mime_type\":\"application/x-plist\",\"group\":\"application\",\"description\":\"Apple property list\",\"extensions\":[\"plist\"],\"is_text\":true},\"applesingle\":{\"mime_type\":\"application/applefile\",\"group\":\"unknown\",\"description\":\"AppleSingle\",\"extensions\":[],\"is_text\":false},\"ar\":{\"mime_type\":\"application/x-archive\",\"group\":\"archive\",\"description\":\"AR Archive\",\"extensions\":[],\"is_text\":false},\"arc\":{\"mime_type\":\"application/x-arc\",\"group\":\"archive\",\"description\":\"Arc\",\"extensions\":[\"arc\"],\"is_text\":false},\"arj\":{\"mime_type\":\"application/arj\",\"group\":\"archive\",\"description\":\"Arj\",\"extensions\":[],\"is_text\":false},\"arrow\":{\"mime_type\":\"vnd.apache.arrow.file\",\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"asc\":{\"mime_type\":\"application/pgp-signature\",\"group\":\"text\",\"description\":\"PGP\",\"extensions\":[\"asc\"],\"is_text\":true},\"asd\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"au\":{\"mime_type\":\"audio/basic\",\"group\":\"audio\",\"description\":\"NeXT/Sun AU\",\"extensions\":[\"au\"],\"is_text\":false},\"asf\":{\"mime_type\":\"video/x-ms-wma\",\"group\":\"application\",\"description\":\"Microsoft Advanced Systems Format\",\"extensions\":[\"asf\"],\"is_text\":false},\"asm\":{\"mime_type\":\"text/x-asm\",\"group\":\"code\",\"description\":\"Assembly\",\"extensions\":[\"s\",\"S\",\"asm\"],\"is_text\":true},\"asp\":{\"mime_type\":\"text/html\",\"group\":\"code\",\"description\":\"ASP source\",\"extensions\":[\"aspx\",\"asp\"],\"is_text\":true},\"autohotkey\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"AutoHotKey script\",\"extensions\":[],\"is_text\":true},\"autoit\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"AutoIt script\",\"extensions\":[\"au3\"],\"is_text\":true},\"avi\":{\"mime_type\":\"video/x-msvideo\",\"group\":\"video\",\"description\":\"Audio Video Interleave\",\"extensions\":[\"avi\"],\"is_text\":false},\"avif\":{\"mime_type\":\"image/avif\",\"group\":\"video\",\"description\":\"AV1 Image File Format\",\"extensions\":[\"avif\",\"avifs\"],\"is_text\":false},\"avro\":{\"mime_type\":\"application/x-avro-binary\",\"group\":null,\"description\":\"Apache Avro binary\",\"extensions\":[\"avro\"],\"is_text\":false},\"awk\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Awk\",\"extensions\":[\"awk\"],\"is_text\":true},\"ax\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"Directshow filter\",\"extensions\":[\"ax\"],\"is_text\":false},\"batch\":{\"mime_type\":\"text/x-msdos-batch\",\"group\":\"code\",\"description\":\"DOS batch file\",\"extensions\":[\"bat\"],\"is_text\":true},\"bazel\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Bazel build file\",\"extensions\":[\"bzl\"],\"is_text\":true},\"bcad\":{\"mime_type\":\"application/octet-stream\",\"group\":\"document\",\"description\":\"bCAD Drawing\",\"extensions\":[\"bdf\"],\"is_text\":false},\"bib\":{\"mime_type\":\"text/x-bibtex\",\"group\":\"text\",\"description\":\"BibTeX\",\"extensions\":[\"bib\"],\"is_text\":true},\"bmp\":{\"mime_type\":\"image/bmp\",\"group\":\"image\",\"description\":\"BMP image data\",\"extensions\":[\"bmp\"],\"is_text\":false},\"bpg\":{\"mime_type\":\"image/bpg\",\"group\":\"image\",\"description\":\"BPG\",\"extensions\":[\"bpg\"],\"is_text\":false},\"bpl\":{\"mime_type\":null,\"group\":\"unknown\",\"description\":null,\"extensions\":[\"bpl\"],\"is_text\":false},\"brainfuck\":{\"mime_type\":\"text/x-brainfuck\",\"group\":\"code\",\"description\":\"Brainfuck source\",\"extensions\":[\"b\",\"bf\"],\"is_text\":true},\"brf\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Braille Ready Format\",\"extensions\":[\"brf\",\"bfm\"],\"is_text\":false},\"bzip\":{\"mime_type\":\"application/x-bzip2\",\"group\":\"archive\",\"description\":\"bzip2 compressed data\",\"extensions\":[\"bz2\",\"tbz2\",\"tar.bz2\"],\"is_text\":false},\"bzip3\":{\"mime_type\":\"application/x-bzip3\",\"group\":\"archive\",\"description\":\"bzip3 compressed data\",\"extensions\":[\"bz3\"],\"is_text\":false},\"c\":{\"mime_type\":\"text/x-c\",\"group\":\"code\",\"description\":\"C source\",\"extensions\":[\"c\"],\"is_text\":true},\"cab\":{\"mime_type\":\"application/vnd.ms-cab-compressed\",\"group\":\"archive\",\"description\":\"Microsoft Cabinet archive data\",\"extensions\":[\"cab\"],\"is_text\":false},\"cad\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"cat\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Windows Catalog file\",\"extensions\":[\"cat\"],\"is_text\":false},\"cdf\":{\"mime_type\":null,\"group\":\"archive\",\"description\":null,\"extensions\":[],\"is_text\":false},\"chm\":{\"mime_type\":\"application/chm\",\"group\":\"application\",\"description\":\"MS Windows HtmlHelp Data\",\"extensions\":[\"chm\"],\"is_text\":false},\"clojure\":{\"mime_type\":\"text/x-clojure\",\"group\":\"code\",\"description\":\"Clojure\",\"extensions\":[\"clj\",\"cljs\",\"cljc\",\"cljr\"],\"is_text\":true},\"cmake\":{\"mime_type\":\"text/x-cmake\",\"group\":\"code\",\"description\":\"CMake build file\",\"extensions\":[\"cmake\"],\"is_text\":true},\"cobol\":{\"mime_type\":\"text/x-cobol\",\"group\":\"code\",\"description\":\"Cobol\",\"extensions\":[\"cbl\",\"cob\",\"cpy\",\"CBL\",\"COB\",\"CPY\"],\"is_text\":true},\"coff\":{\"mime_type\":\"application/x-coff\",\"group\":\"executable\",\"description\":\"Intel 80386 COFF\",\"extensions\":[\"obj\",\"o\"],\"is_text\":false},\"coffeescript\":{\"mime_type\":\"text/coffeescript\",\"group\":\"code\",\"description\":\"CoffeeScript\",\"extensions\":[\"coffee\"],\"is_text\":true},\"com\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":null,\"extensions\":[],\"is_text\":false},\"cpl\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"cpl\"],\"is_text\":false},\"cpp\":{\"mime_type\":\"text/x-c\",\"group\":\"code\",\"description\":\"C++ source\",\"extensions\":[\"cc\",\"cpp\",\"cxx\",\"c++\",\"cppm\",\"ixx\"],\"is_text\":true},\"crt\":{\"mime_type\":\"application/x-x509-ca-cert\",\"group\":\"text\",\"description\":\"Certificates (binary format)\",\"extensions\":[\"der\",\"cer\",\"crt\"],\"is_text\":false},\"crx\":{\"mime_type\":\"application/x-chrome-extension\",\"group\":\"executable\",\"description\":\"Google Chrome extension\",\"extensions\":[\"crx\"],\"is_text\":false},\"cs\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"C# source\",\"extensions\":[\"cs\",\"csx\"],\"is_text\":true},\"csproj\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\".NET project config\",\"extensions\":[\"csproj\"],\"is_text\":true},\"css\":{\"mime_type\":\"text/css\",\"group\":\"code\",\"description\":\"CSS source\",\"extensions\":[\"css\"],\"is_text\":true},\"csv\":{\"mime_type\":\"text/csv\",\"group\":\"code\",\"description\":\"CSV document\",\"extensions\":[\"csv\"],\"is_text\":true},\"ctl\":{\"mime_type\":\"application/octet-stream\",\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"dart\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Dart source\",\"extensions\":[\"dart\"],\"is_text\":true},\"deb\":{\"mime_type\":\"application/vnd.debian.binary-package\",\"group\":\"archive\",\"description\":\"Debian binary package\",\"extensions\":[\"deb\"],\"is_text\":false},\"dex\":{\"mime_type\":\"application/x-android-dex\",\"group\":\"executable\",\"description\":\"Dalvik dex file\",\"extensions\":[\"dex\"],\"is_text\":false},\"dey\":{\"mime_type\":\"application/x-android-dey\",\"group\":\"executable\",\"description\":\"Dalvik dex file\",\"extensions\":[],\"is_text\":false},\"dicom\":{\"mime_type\":\"application/dicom\",\"group\":\"image\",\"description\":\"DICOM\",\"extensions\":[\"dcm\"],\"is_text\":false},\"diff\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Diff file\",\"extensions\":[\"diff\",\"patch\"],\"is_text\":true},\"directory\":{\"mime_type\":\"inode/directory\",\"group\":\"inode\",\"description\":\"A directory\",\"extensions\":[],\"is_text\":false},\"django\":{\"mime_type\":\"text/x-django\",\"group\":\"code\",\"description\":\"Django source\",\"extensions\":[],\"is_text\":false},\"dll\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"dll\"],\"is_text\":false},\"dm\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Dream Maker\",\"extensions\":[\"dm\"],\"is_text\":true},\"dmigd\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Dominion Mods\",\"extensions\":[\"dm\"],\"is_text\":true},\"dmg\":{\"mime_type\":\"application/x-apple-diskimage\",\"group\":\"archive\",\"description\":\"Apple disk image\",\"extensions\":[\"dmg\"],\"is_text\":false},\"dmscript\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Digital Micrograph Script\",\"extensions\":[\"s\"],\"is_text\":true},\"doc\":{\"mime_type\":\"application/msword\",\"group\":\"document\",\"description\":\"Microsoft Word CDF document\",\"extensions\":[\"doc\"],\"is_text\":false},\"dockerfile\":{\"mime_type\":\"text/x-dockerfile\",\"group\":\"code\",\"description\":\"Dockerfile\",\"extensions\":[],\"is_text\":true},\"docx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.wordprocessingml.document\",\"group\":\"document\",\"description\":\"Microsoft Word 2007+ document\",\"extensions\":[\"docx\",\"docm\"],\"is_text\":false},\"dosmbr\":{\"mime_type\":\"application/octet-stream\",\"group\":null,\"description\":\"Master boot record\",\"extensions\":[],\"is_text\":false},\"dotx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.wordprocessingml.template\",\"group\":\"document\",\"description\":\"Office Word 2007 template\",\"extensions\":[\"dotx\"],\"is_text\":false},\"dsstore\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Application Desktop Services Store\",\"extensions\":[],\"is_text\":false},\"dwg\":{\"mime_type\":\"image/x-dwg\",\"group\":\"image\",\"description\":\"Autocad Drawing\",\"extensions\":[\"dwg\"],\"is_text\":false},\"dxf\":{\"mime_type\":\"image/vnd.dxf\",\"group\":\"image\",\"description\":\"Audocad Drawing Exchange Format\",\"extensions\":[\"dxf\"],\"is_text\":true},\"dylib\":{\"mime_type\":\"application/x-mach-o\",\"group\":\"executable\",\"description\":\"Mach-O executable\",\"extensions\":[\"dylib\"],\"is_text\":false},\"ebml\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Extensible Binary Meta Language\",\"extensions\":[],\"is_text\":false},\"elf\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ELF executable\",\"extensions\":[\"elf\"],\"is_text\":false},\"elixir\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Elixir script\",\"extensions\":[\"exs\"],\"is_text\":true},\"emf\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Windows Enhanced Metafile image data\",\"extensions\":[\"emf\"],\"is_text\":false},\"eml\":{\"mime_type\":\"message/rfc822\",\"group\":\"text\",\"description\":\"RFC 822 mail\",\"extensions\":[\"eml\"],\"is_text\":true},\"empty\":{\"mime_type\":\"inode/x-empty\",\"group\":\"inode\",\"description\":\"Empty file\",\"extensions\":[],\"is_text\":false},\"epub\":{\"mime_type\":\"application/epub+zip\",\"group\":\"document\",\"description\":\"EPUB document\",\"extensions\":[\"epub\"],\"is_text\":false},\"erb\":{\"mime_type\":\"text/x-ruby\",\"group\":\"code\",\"description\":\"Embedded Ruby source\",\"extensions\":[\"erb\"],\"is_text\":true},\"erlang\":{\"mime_type\":\"text/x-erlang\",\"group\":\"code\",\"description\":\"Erlang source\",\"extensions\":[\"erl\",\"hrl\"],\"is_text\":true},\"ese\":{\"mime_type\":\"application/x-ms-ese\",\"group\":null,\"description\":\"ESE Db\",\"extensions\":[\"dat\"],\"is_text\":false},\"exe\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE executable\",\"extensions\":[\"exe\"],\"is_text\":false},\"exp\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"flac\":{\"mime_type\":\"audio/flac\",\"group\":\"audio\",\"description\":\"FLAC audio bitstream data\",\"extensions\":[\"flac\"],\"is_text\":false},\"flutter\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"flv\":{\"mime_type\":\"video/x-flv\",\"group\":\"video\",\"description\":\"Flash Video\",\"extensions\":[\"flv\"],\"is_text\":false},\"fortran\":{\"mime_type\":\"text/x-fortran\",\"group\":\"document\",\"description\":\"Fortran\",\"extensions\":[\"f90\",\"f95\",\"f03\",\"F90\"],\"is_text\":true},\"fpx\":{\"mime_type\":null,\"group\":\"image\",\"description\":\"Flashpix\",\"extensions\":[\"fpx\"],\"is_text\":false},\"gemfile\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gemfile file\",\"extensions\":[],\"is_text\":true},\"gemspec\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gemspec file\",\"extensions\":[\"gemspec\"],\"is_text\":true},\"gif\":{\"mime_type\":\"image/gif\",\"group\":\"image\",\"description\":\"GIF image data\",\"extensions\":[\"gif\"],\"is_text\":false},\"gitattributes\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gitattributes file\",\"extensions\":[],\"is_text\":true},\"gitmodules\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Gitmodules file\",\"extensions\":[],\"is_text\":true},\"gleam\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"Gleam source\",\"extensions\":[\"gleam\"],\"is_text\":true},\"go\":{\"mime_type\":\"text/x-golang\",\"group\":\"code\",\"description\":\"Golang source\",\"extensions\":[\"go\"],\"is_text\":true},\"gpx\":{\"mime_type\":null,\"group\":null,\"description\":\"XML document\",\"extensions\":[\"gpx\"],\"is_text\":false},\"gradle\":{\"mime_type\":\"text/x-groovy\",\"group\":\"code\",\"description\":\"Gradle source\",\"extensions\":[\"gradle\"],\"is_text\":true},\"groovy\":{\"mime_type\":\"text/x-groovy\",\"group\":\"code\",\"description\":\"Groovy source\",\"extensions\":[\"groovy\"],\"is_text\":true},\"gzip\":{\"mime_type\":\"application/gzip\",\"group\":\"archive\",\"description\":\"gzip compressed data\",\"extensions\":[\"gz\",\"gzip\",\"tgz\",\"tar.gz\"],\"is_text\":false},\"h\":{\"mime_type\":\"text/x-c\",\"group\":\"code\",\"description\":\"C header source\",\"extensions\":[\"h\"],\"is_text\":true},\"h5\":{\"mime_type\":\"application/x-hdf5\",\"group\":\"archive\",\"description\":\"Hierarchical Data Format v5\",\"extensions\":[\"h5\",\"hdf5\"],\"is_text\":false},\"handlebars\":{\"mime_type\":\"text/x-handlebars-template\",\"group\":\"code\",\"description\":\"Handlebars source\",\"extensions\":[\"hbs\",\"handlebars\"],\"is_text\":true},\"haskell\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Haskell source\",\"extensions\":[\"hs\",\"lhs\"],\"is_text\":true},\"hcl\":{\"mime_type\":\"text/x-hcl\",\"group\":\"code\",\"description\":\"HashiCorp configuration language\",\"extensions\":[\"hcl\"],\"is_text\":true},\"heif\":{\"mime_type\":\"image/heic\",\"group\":\"image\",\"description\":\"High Efficiency Image File\",\"extensions\":[\"heif\",\"heifs\",\"heic\",\"heics\"],\"is_text\":false},\"hfs\":{\"mime_type\":\"application/x-hfs\",\"group\":null,\"description\":null,\"extensions\":[\"hfs\"],\"is_text\":false},\"hlp\":{\"mime_type\":\"application/winhlp\",\"group\":\"application\",\"description\":\"MS Windows help\",\"extensions\":[\"hlp\"],\"is_text\":false},\"hpp\":{\"mime_type\":\"text/x-h\",\"group\":\"code\",\"description\":null,\"extensions\":[\"hh\",\"hpp\",\"hxx\",\"h++\"],\"is_text\":true},\"hta\":{\"mime_type\":\"application/hta\",\"group\":\"code\",\"description\":\"HTML Application\",\"extensions\":[\"hta\"],\"is_text\":false},\"htaccess\":{\"mime_type\":\"text/x-apache-conf\",\"group\":\"code\",\"description\":\"Apache access configuration\",\"extensions\":[],\"is_text\":true},\"html\":{\"mime_type\":\"text/html\",\"group\":\"code\",\"description\":\"HTML document\",\"extensions\":[\"html\",\"htm\",\"xhtml\",\"xht\"],\"is_text\":true},\"hve\":{\"mime_type\":null,\"group\":\"unknown\",\"description\":null,\"extensions\":[],\"is_text\":false},\"hwp\":{\"mime_type\":\"application/x-hwp\",\"group\":\"document\",\"description\":\"Hangul Word Processor\",\"extensions\":[\"hwp\"],\"is_text\":false},\"icc\":{\"mime_type\":\"application/vnd.iccprofile\",\"group\":null,\"description\":\"ICC profile\",\"extensions\":[\"icc\"],\"is_text\":false},\"icns\":{\"mime_type\":\"image/x-icns\",\"group\":\"image\",\"description\":\"Mac OS X icon\",\"extensions\":[\"icns\"],\"is_text\":false},\"ico\":{\"mime_type\":\"image/vnd.microsoft.icon\",\"group\":\"image\",\"description\":\"MS Windows icon resource\",\"extensions\":[\"ico\"],\"is_text\":false},\"ics\":{\"mime_type\":\"text/calendar\",\"group\":\"application\",\"description\":\"Internet Calendaring and Scheduling\",\"extensions\":[\"ics\"],\"is_text\":true},\"ignorefile\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Ignorefile\",\"extensions\":[],\"is_text\":true},\"img\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"img\"],\"is_text\":false},\"ini\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"INI configuration file\",\"extensions\":[\"ini\"],\"is_text\":true},\"internetshortcut\":{\"mime_type\":\"application/x-mswinurl\",\"group\":\"application\",\"description\":\"MS Windows Internet shortcut\",\"extensions\":[\"url\"],\"is_text\":true},\"iosapp\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"ipynb\":{\"mime_type\":\"application/json\",\"group\":\"code\",\"description\":\"Jupyter notebook\",\"extensions\":[\"ipynb\"],\"is_text\":true},\"iso\":{\"mime_type\":\"application/x-iso9660-image\",\"group\":\"archive\",\"description\":\"ISO 9660 CD-ROM filesystem data\",\"extensions\":[\"iso\"],\"is_text\":false},\"jar\":{\"mime_type\":\"application/java-archive\",\"group\":\"archive\",\"description\":\"Java archive data (JAR)\",\"extensions\":[\"jar\",\"klib\"],\"is_text\":false},\"java\":{\"mime_type\":\"text/x-java\",\"group\":\"code\",\"description\":\"Java source\",\"extensions\":[\"java\"],\"is_text\":true},\"javabytecode\":{\"mime_type\":\"application/x-java-applet\",\"group\":\"executable\",\"description\":\"Java compiled bytecode\",\"extensions\":[\"class\"],\"is_text\":false},\"javascript\":{\"mime_type\":\"application/javascript\",\"group\":\"code\",\"description\":\"JavaScript source\",\"extensions\":[\"js\",\"mjs\",\"cjs\"],\"is_text\":true},\"jinja\":{\"mime_type\":\"text/x-jinja2-template\",\"group\":\"code\",\"description\":\"Jinja template\",\"extensions\":[\"jinja\",\"jinja2\",\"j2\"],\"is_text\":true},\"jng\":{\"mime_type\":\"image/jng\",\"group\":\"image\",\"description\":\"JPEG network graphics\",\"extensions\":[\"jng\"],\"is_text\":false},\"jnlp\":{\"mime_type\":\"application/x-java-jnlp-file\",\"group\":\"code\",\"description\":\"Java Network Launch Protocol\",\"extensions\":[\"jnlp\"],\"is_text\":true},\"jp2\":{\"mime_type\":\"image/jpeg2000\",\"group\":\"image\",\"description\":\"jpeg2000\",\"extensions\":[\"jp2\"],\"is_text\":false},\"jpeg\":{\"mime_type\":\"image/jpeg\",\"group\":\"image\",\"description\":\"JPEG image data\",\"extensions\":[\"jpg\",\"jpeg\"],\"is_text\":false},\"json\":{\"mime_type\":\"application/json\",\"group\":\"code\",\"description\":\"JSON document\",\"extensions\":[\"json\"],\"is_text\":true},\"jsonc\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"jsonl\":{\"mime_type\":\"application/json\",\"group\":\"code\",\"description\":\"JSONL document\",\"extensions\":[\"jsonl\",\"jsonld\"],\"is_text\":true},\"jsx\":{\"mime_type\":\"application/javascript\",\"group\":\"code\",\"description\":\"JSX source\",\"extensions\":[\"jsx\",\"mjsx\",\"cjsx\"],\"is_text\":true},\"julia\":{\"mime_type\":\"text/x-julia\",\"group\":\"code\",\"description\":\"Julia source\",\"extensions\":[\"jl\"],\"is_text\":true},\"jxl\":{\"mime_type\":\"image/jxl\",\"group\":\"image\",\"description\":\"JPEG XL\",\"extensions\":[\"jxl\"],\"is_text\":false},\"ko\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ELF executable, kernel object\",\"extensions\":[\"ko\"],\"is_text\":false},\"kotlin\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Kotlin source\",\"extensions\":[\"kt\",\"kts\"],\"is_text\":true},\"ks\":{\"mime_type\":null,\"group\":null,\"description\":\"Tyrano\",\"extensions\":[\"ks\"],\"is_text\":true},\"latex\":{\"mime_type\":\"text/x-tex\",\"group\":\"text\",\"description\":\"LaTeX document\",\"extensions\":[\"tex\",\"sty\"],\"is_text\":true},\"latexaux\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"aux\"],\"is_text\":false},\"less\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"lha\":{\"mime_type\":\"application/x-lha\",\"group\":\"archive\",\"description\":\"LHarc archive\",\"extensions\":[\"lha\",\"lzh\"],\"is_text\":false},\"license\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"License file\",\"extensions\":[],\"is_text\":true},\"lisp\":{\"mime_type\":\"text/x-lisp\",\"group\":\"code\",\"description\":\"Lisp source\",\"extensions\":[\"lisp\",\"lsp\",\"l\",\"cl\"],\"is_text\":true},\"litcs\":{\"mime_type\":null,\"group\":null,\"description\":\"Literate CS\",\"extensions\":[\"litcoffee\"],\"is_text\":false},\"lnk\":{\"mime_type\":\"application/x-ms-shortcut\",\"group\":\"application\",\"description\":\"MS Windows shortcut\",\"extensions\":[\"lnk\"],\"is_text\":false},\"lock\":{\"mime_type\":\"text/plain\",\"group\":\"application\",\"description\":\"Lock file\",\"extensions\":[\"lock\"],\"is_text\":true},\"lrz\":{\"mime_type\":\"application/x-lrzip\",\"group\":null,\"description\":\"LRZip\",\"extensions\":[\"lrz\"],\"is_text\":false},\"lua\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Lua\",\"extensions\":[\"lua\"],\"is_text\":true},\"lz\":{\"mime_type\":\"application/x-lzip\",\"group\":\"archive\",\"description\":\"LZip\",\"extensions\":[\"lz\"],\"is_text\":false},\"lz4\":{\"mime_type\":\"application/x-lz4\",\"group\":\"archive\",\"description\":\"LZ4\",\"extensions\":[\"lz4\"],\"is_text\":false},\"lzx\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"m3u\":{\"mime_type\":\"text/plain\",\"group\":\"application\",\"description\":\"M3U playlist\",\"extensions\":[\"m3u8\",\"m3u\"],\"is_text\":true},\"m4\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"GNU Macro\",\"extensions\":[\"m4\"],\"is_text\":true},\"macho\":{\"mime_type\":\"application/x-mach-o\",\"group\":\"executable\",\"description\":\"Mach-O executable\",\"extensions\":[],\"is_text\":false},\"maff\":{\"mime_type\":\"application/x-maff\",\"group\":null,\"description\":null,\"extensions\":[\"maff\"],\"is_text\":false},\"makefile\":{\"mime_type\":\"text/x-makefile\",\"group\":\"code\",\"description\":\"Makefile source\",\"extensions\":[],\"is_text\":true},\"markdown\":{\"mime_type\":\"text/markdown\",\"group\":\"text\",\"description\":\"Markdown document\",\"extensions\":[\"md\",\"markdown\"],\"is_text\":true},\"matlab\":{\"mime_type\":\"text/x-matlab\",\"group\":\"code\",\"description\":\"Matlab Source\",\"extensions\":[\"m\",\"matlab\"],\"is_text\":true},\"mht\":{\"mime_type\":\"application/x-mimearchive\",\"group\":\"code\",\"description\":\"MHTML document\",\"extensions\":[\"mht\"],\"is_text\":true},\"midi\":{\"mime_type\":\"audio/midi\",\"group\":\"audio\",\"description\":\"Midi\",\"extensions\":[\"mid\"],\"is_text\":false},\"mkv\":{\"mime_type\":\"video/x-matroska\",\"group\":\"video\",\"description\":\"Matroska\",\"extensions\":[\"mkv\"],\"is_text\":false},\"mp2\":{\"mime_type\":null,\"group\":null,\"description\":\"MP2 stream\",\"extensions\":[\"mp2\"],\"is_text\":false},\"mp3\":{\"mime_type\":\"audio/mpeg\",\"group\":\"audio\",\"description\":\"MP3 media file\",\"extensions\":[\"mp3\"],\"is_text\":false},\"mp4\":{\"mime_type\":\"video/mp4\",\"group\":\"video\",\"description\":\"MP4 media file\",\"extensions\":[\"mp4\"],\"is_text\":false},\"mpegts\":{\"mime_type\":\"video/MP2T\",\"group\":\"video\",\"description\":\"MPEG Transport stream\",\"extensions\":[\"ts\",\"tsv\",\"tsa\",\"m2t\"],\"is_text\":false},\"mscompress\":{\"mime_type\":\"application/x-ms-compress-szdd\",\"group\":\"archive\",\"description\":\"MS Compress archive data\",\"extensions\":[],\"is_text\":false},\"msi\":{\"mime_type\":\"application/x-msi\",\"group\":\"archive\",\"description\":\"Microsoft Installer file\",\"extensions\":[\"msi\"],\"is_text\":false},\"msix\":{\"mime_type\":\"application/msix\",\"group\":\"application\",\"description\":\"Windows app package\",\"extensions\":[\"msix\"],\"is_text\":false},\"mst\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"mst\"],\"is_text\":false},\"mui\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"application\",\"description\":\"PE Windows executable\",\"extensions\":[\"mui\"],\"is_text\":false},\"mum\":{\"mime_type\":\"text/xml\",\"group\":\"application\",\"description\":\"Windows Update Package file\",\"extensions\":[\"mum\"],\"is_text\":true},\"mun\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"mun\"],\"is_text\":false},\"nim\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"npy\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Numpy Array\",\"extensions\":[\"npy\"],\"is_text\":false},\"npz\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Numpy Arrays Archive\",\"extensions\":[\"npz\"],\"is_text\":false},\"null\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"null\"],\"is_text\":false},\"nupkg\":{\"mime_type\":\"application/octet-stream\",\"group\":null,\"description\":\"NuGet Package\",\"extensions\":[\"nupkg\"],\"is_text\":false},\"object\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"o\"],\"is_text\":false},\"objectivec\":{\"mime_type\":\"text/x-objcsrc\",\"group\":\"code\",\"description\":\"ObjectiveC source\",\"extensions\":[\"m\",\"mm\"],\"is_text\":true},\"ocaml\":{\"mime_type\":\"text-ocaml\",\"group\":\"code\",\"description\":\"OCaml\",\"extensions\":[\"ml\",\"mli\"],\"is_text\":true},\"ocx\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"ocx\"],\"is_text\":false},\"odex\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ODEX ELF executable\",\"extensions\":[\"odex\"],\"is_text\":false},\"odin\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"Odin\",\"extensions\":[\"odin\"],\"is_text\":true},\"odp\":{\"mime_type\":\"application/vnd.oasis.opendocument.presentation\",\"group\":\"document\",\"description\":\"OpenDocument Presentation\",\"extensions\":[\"odp\"],\"is_text\":false},\"ods\":{\"mime_type\":\"application/vnd.oasis.opendocument.spreadsheet\",\"group\":\"document\",\"description\":\"OpenDocument Spreadsheet\",\"extensions\":[\"ods\"],\"is_text\":false},\"odt\":{\"mime_type\":\"application/vnd.oasis.opendocument.text\",\"group\":\"document\",\"description\":\"OpenDocument Text\",\"extensions\":[\"odt\"],\"is_text\":false},\"ogg\":{\"mime_type\":\"audio/ogg\",\"group\":\"audio\",\"description\":\"Ogg data\",\"extensions\":[\"ogg\"],\"is_text\":false},\"ole\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"one\":{\"mime_type\":\"application/msonenote\",\"group\":\"document\",\"description\":\"One Note\",\"extensions\":[\"one\"],\"is_text\":false},\"onnx\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Open Neural Network Exchange\",\"extensions\":[\"onnx\"],\"is_text\":false},\"ooxml\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"otf\":{\"mime_type\":\"font/otf\",\"group\":\"font\",\"description\":\"OpenType font\",\"extensions\":[\"otf\"],\"is_text\":false},\"outlook\":{\"mime_type\":\"application/vnd.ms-outlook\",\"group\":\"application\",\"description\":\"MS Outlook Message\",\"extensions\":[],\"is_text\":false},\"palmos\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"parquet\":{\"mime_type\":\"application/vnd.apache.parquet\",\"group\":\"unknown\",\"description\":\"Apache Parquet\",\"extensions\":[\"pqt\",\"parquet\"],\"is_text\":false},\"pascal\":{\"mime_type\":\"text/x-pascal\",\"group\":\"code\",\"description\":\"Pascal source\",\"extensions\":[\"pas\",\"pp\"],\"is_text\":true},\"pbm\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"pcap\":{\"mime_type\":\"application/vnd.tcpdump.pcap\",\"group\":\"application\",\"description\":\"pcap capture file\",\"extensions\":[\"pcap\",\"pcapng\"],\"is_text\":false},\"pdb\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Windows Program Database\",\"extensions\":[\"pdb\"],\"is_text\":false},\"pdf\":{\"mime_type\":\"application/pdf\",\"group\":\"document\",\"description\":\"PDF document\",\"extensions\":[\"pdf\"],\"is_text\":false},\"pebin\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"exe\",\"dll\"],\"is_text\":false},\"pem\":{\"mime_type\":\"application/x-pem-file\",\"group\":\"application\",\"description\":\"PEM certificate\",\"extensions\":[\"pem\",\"pub\",\"gpg\"],\"is_text\":true},\"perl\":{\"mime_type\":\"text/x-perl\",\"group\":\"code\",\"description\":\"Perl source\",\"extensions\":[\"pl\"],\"is_text\":true},\"pgp\":{\"mime_type\":\"application/pgp-keys\",\"group\":null,\"description\":\"PGP\",\"extensions\":[\"gpg\",\"pgp\"],\"is_text\":false},\"php\":{\"mime_type\":\"text/x-php\",\"group\":\"code\",\"description\":\"PHP source\",\"extensions\":[\"php\"],\"is_text\":true},\"pickle\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Python pickle\",\"extensions\":[\"pickle\",\"pkl\"],\"is_text\":false},\"png\":{\"mime_type\":\"image/png\",\"group\":\"image\",\"description\":\"PNG image\",\"extensions\":[\"png\"],\"is_text\":false},\"po\":{\"mime_type\":\"text/gettext-translation\",\"group\":\"application\",\"description\":\"Portable Object (PO) for i18n\",\"extensions\":[\"po\"],\"is_text\":true},\"postscript\":{\"mime_type\":\"application/postscript\",\"group\":\"document\",\"description\":\"PostScript document\",\"extensions\":[\"ps\"],\"is_text\":false},\"powershell\":{\"mime_type\":\"application/x-powershell\",\"group\":\"code\",\"description\":\"Powershell source\",\"extensions\":[\"ps1\"],\"is_text\":true},\"ppt\":{\"mime_type\":\"application/vnd.ms-powerpoint\",\"group\":\"document\",\"description\":\"Microsoft PowerPoint CDF document\",\"extensions\":[\"ppt\"],\"is_text\":false},\"pptx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.presentationml.presentation\",\"group\":\"document\",\"description\":\"Microsoft PowerPoint 2007+ document\",\"extensions\":[\"pptx\",\"pptm\"],\"is_text\":false},\"printfox\":{\"mime_type\":null,\"group\":null,\"description\":\"c64\",\"extensions\":[],\"is_text\":false},\"prolog\":{\"mime_type\":\"text/x-prolog\",\"group\":\"code\",\"description\":\"Prolog source\",\"extensions\":[\"pl\",\"pro\",\"P\"],\"is_text\":true},\"proteindb\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Protein DB\",\"extensions\":[\"pdb\"],\"is_text\":true},\"proto\":{\"mime_type\":\"text/x-proto\",\"group\":\"code\",\"description\":\"Protocol buffer definition\",\"extensions\":[\"proto\"],\"is_text\":true},\"protobuf\":{\"mime_type\":\"application/protobuf\",\"group\":\"unknown\",\"description\":\"Protocol buffers\",\"extensions\":[\"protobuf\",\"pb\"],\"is_text\":false},\"psd\":{\"mime_type\":\"image/vnd.adobe.photoshop\",\"group\":\"image\",\"description\":\"Adobe Photoshop\",\"extensions\":[\"psd\"],\"is_text\":false},\"pytorch\":{\"mime_type\":\"application/octet-stream\",\"group\":\"application\",\"description\":\"Pytorch storage file\",\"extensions\":[\"pt\",\"pth\"],\"is_text\":false},\"pub\":{\"mime_type\":\"application/x-mspublisher\",\"group\":null,\"description\":null,\"extensions\":[\"pub\"],\"is_text\":false},\"python\":{\"mime_type\":\"text/x-python\",\"group\":\"code\",\"description\":\"Python source\",\"extensions\":[\"py\",\"pyi\"],\"is_text\":true},\"pythonbytecode\":{\"mime_type\":\"application/x-bytecode.python\",\"group\":\"executable\",\"description\":\"Python compiled bytecode\",\"extensions\":[\"pyc\",\"pyo\"],\"is_text\":false},\"pythonpar\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"par\"],\"is_text\":false},\"qoi\":{\"mime_type\":\"image/x-qoi\",\"group\":\"image\",\"description\":\"Quite Ok Image\",\"extensions\":[\"qoi\"],\"is_text\":false},\"qt\":{\"mime_type\":\"video/quicktime\",\"group\":\"video\",\"description\":\"QuickTime\",\"extensions\":[\"mov\"],\"is_text\":false},\"r\":{\"mime_type\":\"text/x-R\",\"group\":\"code\",\"description\":\"R (language)\",\"extensions\":[\"R\"],\"is_text\":true},\"randomascii\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Random ASCII characters\",\"extensions\":[],\"is_text\":true},\"randombytes\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Random bytes\",\"extensions\":[],\"is_text\":false},\"randomtxt\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Random text\",\"extensions\":[],\"is_text\":true},\"rar\":{\"mime_type\":\"application/x-rar\",\"group\":\"archive\",\"description\":\"RAR archive data\",\"extensions\":[\"rar\"],\"is_text\":false},\"rdf\":{\"mime_type\":\"application/rdf+xml\",\"group\":\"text\",\"description\":\"Resource Description Framework document (RDF)\",\"extensions\":[\"rdf\"],\"is_text\":true},\"rdp\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"riff\":{\"mime_type\":\"application/x-riff\",\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"rlib\":{\"mime_type\":\"application/x-archive\",\"group\":\"archive\",\"description\":\"rust library\",\"extensions\":[\"rlib\"],\"is_text\":false},\"rll\":{\"mime_type\":null,\"group\":\"executable\",\"description\":\"Resource Library\",\"extensions\":[\"rll\"],\"is_text\":false},\"rpm\":{\"mime_type\":\"application/x-rpm\",\"group\":\"archive\",\"description\":\"RedHat Package Manager archive (RPM)\",\"extensions\":[\"rpm\"],\"is_text\":false},\"rst\":{\"mime_type\":\"text/x-rst\",\"group\":\"text\",\"description\":\"ReStructuredText document\",\"extensions\":[\"rst\"],\"is_text\":true},\"rtf\":{\"mime_type\":\"text/rtf\",\"group\":\"text\",\"description\":\"Rich Text Format document\",\"extensions\":[\"rtf\"],\"is_text\":true},\"ruby\":{\"mime_type\":\"application/x-ruby\",\"group\":\"code\",\"description\":\"Ruby source\",\"extensions\":[\"rb\"],\"is_text\":true},\"rust\":{\"mime_type\":\"application/x-rust\",\"group\":\"code\",\"description\":\"Rust source\",\"extensions\":[\"rs\"],\"is_text\":true},\"rzip\":{\"mime_type\":null,\"group\":null,\"description\":\"Rzip\",\"extensions\":[\"rz\"],\"is_text\":false},\"scala\":{\"mime_type\":\"application/x-scala\",\"group\":\"code\",\"description\":\"Scala source\",\"extensions\":[\"scala\"],\"is_text\":true},\"scheme\":{\"mime_type\":\"text/x-scheme\",\"group\":\"code\",\"description\":null,\"extensions\":[\"scm\",\"ss\"],\"is_text\":false},\"scr\":{\"mime_type\":\"application/x-dosexec\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"scr\"],\"is_text\":false},\"scriptwsf\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"scss\":{\"mime_type\":\"text/x-scss\",\"group\":\"code\",\"description\":\"SCSS source\",\"extensions\":[\"scss\"],\"is_text\":true},\"sevenzip\":{\"mime_type\":\"application/x-7z-compressed\",\"group\":\"archive\",\"description\":\"7-zip archive data\",\"extensions\":[\"7z\"],\"is_text\":false},\"sgml\":{\"mime_type\":\"application/sgml\",\"group\":\"text\",\"description\":\"sgml\",\"extensions\":[\"sgml\"],\"is_text\":true},\"sh3d\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"sh3d\"],\"is_text\":false},\"shell\":{\"mime_type\":\"text/x-shellscript\",\"group\":\"code\",\"description\":\"Shell script\",\"extensions\":[\"sh\"],\"is_text\":true},\"smali\":{\"mime_type\":\"application/x-smali\",\"group\":\"code\",\"description\":\"Smali source\",\"extensions\":[\"smali\"],\"is_text\":true},\"snap\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Snap archive\",\"extensions\":[\"snap\"],\"is_text\":false},\"so\":{\"mime_type\":\"application/x-executable-elf\",\"group\":\"executable\",\"description\":\"ELF executable, shared library\",\"extensions\":[\"so\"],\"is_text\":false},\"solidity\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"Solidity source\",\"extensions\":[\"sol\"],\"is_text\":true},\"sql\":{\"mime_type\":\"application/x-sql\",\"group\":\"code\",\"description\":\"SQL source\",\"extensions\":[\"sql\"],\"is_text\":true},\"sqlite\":{\"mime_type\":null,\"group\":\"application\",\"description\":\"SQLITE database\",\"extensions\":[\"sqlite\",\"sqlite3\"],\"is_text\":false},\"squashfs\":{\"mime_type\":\"application/octet-stream\",\"group\":\"archive\",\"description\":\"Squash filesystem\",\"extensions\":[],\"is_text\":false},\"srt\":{\"mime_type\":\"text/srt\",\"group\":\"application\",\"description\":\"SubRip Text Format\",\"extensions\":[\"srt\"],\"is_text\":true},\"stlbinary\":{\"mime_type\":\"application/sla\",\"group\":\"image\",\"description\":\"Stereolithography CAD (binary)\",\"extensions\":[\"stl\"],\"is_text\":false},\"stltext\":{\"mime_type\":\"application/sla\",\"group\":\"image\",\"description\":\"Stereolithography CAD (text)\",\"extensions\":[\"stl\"],\"is_text\":true},\"sum\":{\"mime_type\":null,\"group\":\"application\",\"description\":\"Checksum file\",\"extensions\":[\"sum\"],\"is_text\":true},\"svd\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"svg\":{\"mime_type\":\"image/svg+xml\",\"group\":\"image\",\"description\":\"SVG Scalable Vector Graphics image data\",\"extensions\":[\"svg\"],\"is_text\":true},\"swf\":{\"mime_type\":\"application/x-shockwave-flash\",\"group\":\"executable\",\"description\":\"Small Web File\",\"extensions\":[\"swf\"],\"is_text\":false},\"swift\":{\"mime_type\":\"text/x-swift\",\"group\":\"code\",\"description\":\"Swift\",\"extensions\":[\"swift\"],\"is_text\":true},\"symlink\":{\"mime_type\":\"inode/symlink\",\"group\":\"inode\",\"description\":\"Symbolic link\",\"extensions\":[],\"is_text\":false},\"symlinktext\":{\"mime_type\":\"text/plain\",\"group\":\"application\",\"description\":\"Symbolic link (textual representation)\",\"extensions\":[],\"is_text\":true},\"sys\":{\"mime_type\":\"application/x-windows-driver\",\"group\":\"executable\",\"description\":\"PE Windows executable\",\"extensions\":[\"sys\"],\"is_text\":false},\"tar\":{\"mime_type\":\"application/x-tar\",\"group\":\"archive\",\"description\":\"POSIX tar archive\",\"extensions\":[\"tar\"],\"is_text\":false},\"tcl\":{\"mime_type\":\"application/x-tcl\",\"group\":\"code\",\"description\":\"Tickle\",\"extensions\":[\"tcl\"],\"is_text\":true},\"textproto\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Text protocol buffer\",\"extensions\":[\"textproto\",\"textpb\",\"pbtxt\"],\"is_text\":true},\"tga\":{\"mime_type\":\"image/x-tga\",\"group\":\"image\",\"description\":\"Targa image data\",\"extensions\":[\"tga\"],\"is_text\":false},\"thumbsdb\":{\"mime_type\":\"image/vnd.ms-thumb\",\"group\":\"application\",\"description\":\"Windows thumbnail cache\",\"extensions\":[],\"is_text\":false},\"tiff\":{\"mime_type\":\"image/tiff\",\"group\":\"image\",\"description\":\"TIFF image data\",\"extensions\":[\"tiff\",\"tif\"],\"is_text\":false},\"tmdx\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"tmdx\",\"tmvx\"],\"is_text\":false},\"toml\":{\"mime_type\":\"application/toml\",\"group\":\"text\",\"description\":\"Tom's obvious, minimal language\",\"extensions\":[\"toml\"],\"is_text\":true},\"torrent\":{\"mime_type\":\"application/x-bittorrent\",\"group\":\"application\",\"description\":\"BitTorrent file\",\"extensions\":[\"torrent\"],\"is_text\":false},\"troff\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"tsv\":{\"mime_type\":\"text/tsv\",\"group\":\"code\",\"description\":\"TSV document\",\"extensions\":[\"tsv\"],\"is_text\":true},\"tsx\":{\"mime_type\":\"text/x-typescript\",\"group\":\"code\",\"description\":\"TSX source\",\"extensions\":[\"tsx\",\"mtsx\",\"ctsx\"],\"is_text\":true},\"ttf\":{\"mime_type\":\"font/sfnt\",\"group\":\"font\",\"description\":\"TrueType Font data\",\"extensions\":[\"ttf\",\"ttc\"],\"is_text\":false},\"twig\":{\"mime_type\":\"text/x-twig\",\"group\":\"code\",\"description\":\"Twig template\",\"extensions\":[\"twig\"],\"is_text\":true},\"txt\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document\",\"extensions\":[\"txt\"],\"is_text\":true},\"txtascii\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document encoded in ASCII\",\"extensions\":[\"txt\"],\"is_text\":true},\"txtutf16\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document encoded in UTF-16\",\"extensions\":[\"txt\"],\"is_text\":true},\"txtutf8\":{\"mime_type\":\"text/plain\",\"group\":\"text\",\"description\":\"Generic text document encoded in UTF-8\",\"extensions\":[\"txt\"],\"is_text\":true},\"typescript\":{\"mime_type\":\"application/typescript\",\"group\":\"code\",\"description\":\"TypeScript source\",\"extensions\":[\"ts\",\"mts\",\"cts\"],\"is_text\":true},\"udf\":{\"mime_type\":\"application/x-udf-image\",\"group\":null,\"description\":\"Universal Disc Format\",\"extensions\":[],\"is_text\":false},\"undefined\":{\"mime_type\":\"application/undefined\",\"group\":\"undefined\",\"description\":\"Undefined\",\"extensions\":[],\"is_text\":false},\"unixcompress\":{\"mime_type\":\"application/x-compress\",\"group\":null,\"description\":null,\"extensions\":[\"z\"],\"is_text\":false},\"unknown\":{\"mime_type\":\"application/octet-stream\",\"group\":\"unknown\",\"description\":\"Unknown binary data\",\"extensions\":[],\"is_text\":false},\"vba\":{\"mime_type\":\"text/vbscript\",\"group\":\"code\",\"description\":\"MS Visual Basic source (VBA)\",\"extensions\":[\"vbs\",\"vba\",\"vb\"],\"is_text\":true},\"vbe\":{\"mime_type\":null,\"group\":\"code\",\"description\":\"EncryptedVBS\",\"extensions\":[\"vbe\"],\"is_text\":false},\"vcard\":{\"mime_type\":\"text/vcard\",\"group\":null,\"description\":null,\"extensions\":[\"vcard\"],\"is_text\":false},\"vcs\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[],\"is_text\":false},\"vcxproj\":{\"mime_type\":\"application/xml\",\"group\":\"code\",\"description\":\"Visual Studio MSBuild project\",\"extensions\":[\"vcxproj\"],\"is_text\":true},\"verilog\":{\"mime_type\":\"text/x-verilog\",\"group\":\"code\",\"description\":\"Verilog source\",\"extensions\":[\"v\",\"verilog\",\"vlg\",\"vh\"],\"is_text\":true},\"vhd\":{\"mime_type\":\"application/x-vhd\",\"group\":null,\"description\":\"Virtual Hard Disk\",\"extensions\":[],\"is_text\":false},\"vhdl\":{\"mime_type\":\"text/x-vhdl\",\"group\":\"code\",\"description\":\"VHDL source\",\"extensions\":[\"vhd\"],\"is_text\":true},\"visio\":{\"mime_type\":\"application/vnd.ms-visio.drawing.main+xml\",\"group\":\"document\",\"description\":\"Microsoft Visio\",\"extensions\":[\"vsd\",\"vsdm\",\"vsdx\",\"vdw\"],\"is_text\":false},\"vtt\":{\"mime_type\":\"text/vtt\",\"group\":\"text\",\"description\":\"Web Video Text Tracks\",\"extensions\":[\"vtt\",\"webvtt\"],\"is_text\":true},\"vue\":{\"mime_type\":\"application/javascript\",\"group\":\"code\",\"description\":\"Vue source\",\"extensions\":[\"vue\"],\"is_text\":true},\"wad\":{\"mime_type\":\"application/wad\",\"group\":\"archive\",\"description\":\"WAD\",\"extensions\":[\"wad\"],\"is_text\":false},\"wasm\":{\"mime_type\":\"application/wasm\",\"group\":\"executable\",\"description\":\"Web Assembly\",\"extensions\":[\"wasm\"],\"is_text\":false},\"wav\":{\"mime_type\":\"audio/x-wav\",\"group\":\"audio\",\"description\":\"Waveform Audio file (WAV)\",\"extensions\":[\"wav\"],\"is_text\":false},\"webm\":{\"mime_type\":\"video/webm\",\"group\":\"video\",\"description\":\"WebM media file\",\"extensions\":[\"webm\"],\"is_text\":false},\"webp\":{\"mime_type\":\"image/webp\",\"group\":\"image\",\"description\":\"WebP media file\",\"extensions\":[\"webp\"],\"is_text\":false},\"webtemplate\":{\"mime_type\":\"text/plain\",\"group\":\"code\",\"description\":\"Web templating language\",\"extensions\":[],\"is_text\":true},\"wim\":{\"mime_type\":\"application/x-ms-wim\",\"group\":\"unknown\",\"description\":\"Windows Imaging Format\",\"extensions\":[\"wim\",\"swm\",\"esd\"],\"is_text\":false},\"winregistry\":{\"mime_type\":\"text/x-ms-regedit\",\"group\":\"application\",\"description\":\"Windows Registry text\",\"extensions\":[\"reg\"],\"is_text\":true},\"wma\":{\"mime_type\":\"audio/x-ms-wma\",\"group\":\"audio\",\"description\":\"Windows Media Audio\",\"extensions\":[\"wma\"],\"is_text\":false},\"wmf\":{\"mime_type\":\"image/wmf\",\"group\":\"image\",\"description\":\"Windows metafile\",\"extensions\":[\"wmf\"],\"is_text\":false},\"wmv\":{\"mime_type\":\"video/x-ms-wmv\",\"group\":\"video\",\"description\":\"Windows Media Video\",\"extensions\":[\"wmv\"],\"is_text\":false},\"woff\":{\"mime_type\":\"font/woff\",\"group\":\"font\",\"description\":\"Web Open Font Format\",\"extensions\":[\"woff\"],\"is_text\":false},\"woff2\":{\"mime_type\":\"font/woff2\",\"group\":\"font\",\"description\":\"Web Open Font Format v2\",\"extensions\":[\"woff2\"],\"is_text\":false},\"xar\":{\"mime_type\":\"application/x-xar\",\"group\":\"archive\",\"description\":\"XAR archive compressed data\",\"extensions\":[\"pkg\",\"xar\"],\"is_text\":false},\"xcf\":{\"mime_type\":\"image/x-xcf\",\"group\":\"image\",\"description\":\"Gimp image\",\"extensions\":[\"xcf\"],\"is_text\":false},\"xls\":{\"mime_type\":\"application/vnd.ms-excel\",\"group\":\"document\",\"description\":\"Microsoft Excel CDF document\",\"extensions\":[\"xls\"],\"is_text\":false},\"xlsb\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\"group\":\"document\",\"description\":\"Microsoft Excel 2007+ document (binary format)\",\"extensions\":[\"xlsb\"],\"is_text\":false},\"xlsx\":{\"mime_type\":\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\"group\":\"document\",\"description\":\"Microsoft Excel 2007+ document\",\"extensions\":[\"xlsx\",\"xlsm\"],\"is_text\":false},\"xml\":{\"mime_type\":\"text/xml\",\"group\":\"code\",\"description\":\"XML document\",\"extensions\":[\"xml\"],\"is_text\":true},\"xpi\":{\"mime_type\":\"application/zip\",\"group\":\"archive\",\"description\":\"Compressed installation archive (XPI)\",\"extensions\":[\"xpi\"],\"is_text\":false},\"xsd\":{\"mime_type\":null,\"group\":null,\"description\":null,\"extensions\":[\"xsd\"],\"is_text\":false},\"xz\":{\"mime_type\":\"application/x-xz\",\"group\":\"archive\",\"description\":\"XZ compressed data\",\"extensions\":[\"xz\"],\"is_text\":false},\"yaml\":{\"mime_type\":\"application/x-yaml\",\"group\":\"code\",\"description\":\"YAML source\",\"extensions\":[\"yml\",\"yaml\"],\"is_text\":true},\"yara\":{\"mime_type\":\"text/x-yara\",\"group\":\"code\",\"description\":\"YARA rule\",\"extensions\":[\"yar\",\"yara\"],\"is_text\":true},\"zig\":{\"mime_type\":\"text/zig\",\"group\":\"code\",\"description\":\"Zig source\",\"extensions\":[\"zig\"],\"is_text\":true},\"zip\":{\"mime_type\":\"application/zip\",\"group\":\"archive\",\"description\":\"Zip archive data\",\"extensions\":[\"zip\"],\"is_text\":false},\"zlibstream\":{\"mime_type\":\"application/zlib\",\"group\":\"application\",\"description\":\"zlib compressed data\",\"extensions\":[],\"is_text\":false},\"zst\":{\"mime_type\":\"application/zstd\",\"group\":\"archive\",\"description\":\"Zstandard\",\"extensions\":[\"zst\"],\"is_text\":false}}"
  },
  {
    "path": "python/src/magika/logger.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100, D101, D102, D103, D107\n\nfrom __future__ import annotations\n\nimport logging\nimport sys\nfrom typing import Optional, TextIO\n\nfrom magika import colors\n\n_logger: Optional[SimpleLogger] = None\n\n\nclass SimpleLogger:\n    \"\"\"A simple, self-contained logger that writes output to stderr.\n\n    This implementation avoids reliance on external dependencies (e.g., `rich`)\n    by sending log messages (debug, info, warning, etc.) directly to standard\n    error (`sys.stderr`) by default.\n    \"\"\"\n\n    def __init__(self, use_colors: bool = False):\n        self.level = logging.WARNING\n        self.use_colors = use_colors\n\n    def setLevel(self, level: int) -> None:\n        self.level = level\n\n    def raw_print_to_stdout(self, msg: str) -> None:\n        self.raw_print(msg, file=sys.stdout)\n\n    def raw_print(\n        self, msg: str, file: Optional[TextIO] = None, flush: bool = True\n    ) -> None:\n        if file is None:\n            # We avoid using a default value for the `file` argument because we\n            # need to get the reference to the \"current\" stderr; if we used a\n            # default argument, we would just store the \"current at\n            # instantiation time\" stderr, which may not be the current one.\n            # This, in turn, could create problems for testing.\n            file = sys.stderr\n        print(msg, file=file, flush=flush)\n\n    def debug(self, msg: str) -> None:\n        if logging.DEBUG >= self.level:\n            if self.use_colors:\n                self.raw_print(f\"{colors.GREEN}DEBUG: {msg}{colors.RESET}\")\n            else:\n                self.raw_print(f\"DEBUG: {msg}\")\n\n    def info(self, msg: str) -> None:\n        if logging.INFO >= self.level:\n            self.raw_print(f\"INFO: {msg}\")\n\n    def warning(self, msg: str) -> None:\n        if logging.WARNING >= self.level:\n            if self.use_colors:\n                self.raw_print(f\"{colors.YELLOW}WARNING: {msg}{colors.RESET}\")\n            else:\n                self.raw_print(f\"WARNING: {msg}\")\n\n    def error(self, msg: str) -> None:\n        if logging.ERROR >= self.level:\n            if self.use_colors:\n                self.raw_print(f\"{colors.RED}ERROR: {msg}{colors.RESET}\")\n            else:\n                self.raw_print(f\"ERROR: {msg}\")\n\n\ndef get_logger(use_colors: bool = False) -> SimpleLogger:\n    global _logger\n\n    if _logger is None:\n        _logger = SimpleLogger(use_colors=use_colors)\n\n    return _logger\n"
  },
  {
    "path": "python/src/magika/magika.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Magika (the Python library).\n\nThis module provides the `Magika` class, the main entry point for using Magika\nto identify file content types.\n\"\"\"\n\nimport io\nimport json\nimport logging\nimport os\nimport time\nfrom pathlib import Path\nfrom typing import BinaryIO, Dict, List, Optional, Sequence, Set, Tuple, Union\n\nimport onnxruntime as rt\n\nfrom magika.logger import get_logger\nfrom magika.types import (\n    ContentTypeInfo,\n    ContentTypeLabel,\n    MagikaError,\n    MagikaPrediction,\n    MagikaResult,\n    ModelConfig,\n    ModelFeatures,\n    ModelOutput,\n    OverwriteReason,\n    PredictionMode,\n    Seekable,\n    Status,\n)\n\n_DEFAULT_MODEL_NAME = \"standard_v3_3\"\n\n\nclass Magika:\n    \"\"\"Main Magika class for content type identification.\n\n    This class provides methods to identify the content type of files, bytes,\n    and streams.\n    \"\"\"\n\n    def __init__(\n        self,\n        model_dir: Optional[Path] = None,\n        prediction_mode: PredictionMode = PredictionMode.HIGH_CONFIDENCE,\n        no_dereference: bool = False,\n        verbose: bool = False,\n        debug: bool = False,\n        use_colors: bool = False,\n    ) -> None:\n        \"\"\"Initializes the Magika instance.\n\n        Args:\n            model_dir: Path to the directory containing the model and its\n                configuration. If None, the default model is used.\n            prediction_mode: The prediction mode to use.  Defaults to\n                PredictionMode.HIGH_CONFIDENCE.\n            no_dereference: If True, do not follow symlinks.  Defaults to False.\n            verbose: If True, enable verbose logging. Defaults to False.\n            debug: If True, enable debug logging. Defaults to False.\n            use_colors: If True, use colors in the logger.  Defaults to False.\n        \"\"\"\n        self._log = get_logger(use_colors=use_colors)\n\n        if verbose:\n            self._log.setLevel(logging.INFO)\n\n        if debug:\n            self._log.setLevel(logging.DEBUG)\n\n        if model_dir is not None:\n            self._model_dir = model_dir\n        else:\n            # use default model\n            self._model_dir = (\n                Path(__file__).parent / \"models\" / self._get_default_model_name()\n            )\n\n        self._model_path = self._model_dir / \"model.onnx\"\n        self._model_config_path = self._model_dir / \"config.min.json\"\n\n        if not self._model_dir.is_dir():\n            raise MagikaError(f\"model dir not found at {str(self._model_dir)}\")\n        if not self._model_path.is_file():\n            raise MagikaError(f\"model not found at {str(self._model_path)}\")\n        if not self._model_config_path.is_file():\n            raise MagikaError(\n                f\"model config not found at {str(self._model_config_path)}\"\n            )\n\n        self._model_config: ModelConfig = Magika._load_model_config(\n            self._model_config_path\n        )\n\n        self._target_labels_space = list(\n            map(str, self._model_config.target_labels_space)\n        )\n\n        self._prediction_mode = prediction_mode\n\n        self._no_dereference = no_dereference\n\n        content_types_kb_path = (\n            Path(__file__).parent / \"config\" / \"content_types_kb.min.json\"\n        )\n        self._cts_infos = Magika._load_content_types_kb(content_types_kb_path)\n\n        self._onnx_session = self._init_onnx_session()\n\n    def __repr__(self) -> str:\n        return str(self)\n\n    def __str__(self) -> str:\n        return f'Magika(module_version=\"{self.get_module_version()}\", model_name=\"{self.get_model_name()}\")'\n\n    def get_module_version(self) -> str:\n        \"\"\"Gets the version of the Magika Python module.\"\"\"\n        return str(__import__(self.__module__).__version__)\n\n    def get_model_name(self) -> str:\n        \"\"\"Gets the name of the loaded model.\"\"\"\n        return self._model_dir.name\n\n    def identify_path(self, path: Union[str, os.PathLike]) -> MagikaResult:\n        \"\"\"Identify the content type of a file given its path.\"\"\"\n        if isinstance(path, str) or isinstance(path, os.PathLike):\n            path = Path(path)\n        else:\n            raise TypeError(\n                f\"Path '{path}' is invalid: input path should be of type `Union[str, os.PathLike]`\"\n            )\n\n        return self._get_result_from_path(path)\n\n    def identify_paths(\n        self, paths: Sequence[Union[str, os.PathLike]]\n    ) -> List[MagikaResult]:\n        \"\"\"Identify the content type of a list of files given their paths.\"\"\"\n        if not isinstance(paths, Sequence):\n            raise TypeError(\"Input paths should be of type Sequence[Path]\")\n\n        paths_ = []\n        for path in paths:\n            if isinstance(path, str) or isinstance(path, os.PathLike):\n                paths_.append(Path(path))\n            else:\n                raise TypeError(\n                    f\"Input '{path}' is invalid: input path should be of type `Union[str, os.PathLike]`\"\n                )\n\n        return self._get_results_from_paths(paths_)\n\n    def identify_bytes(self, content: bytes) -> MagikaResult:\n        \"\"\"Identify the content type of raw bytes.\"\"\"\n        if not isinstance(content, bytes):\n            raise TypeError(\n                f\"Input content should be of type 'bytes', not {type(content)}.\"\n            )\n\n        return self._get_result_from_seekable(Seekable(io.BytesIO(content)))\n\n    def identify_stream(self, stream: BinaryIO) -> MagikaResult:\n        \"\"\"Identify the content type of a BinaryIO stream.\n\n        Identifies the content type from an already-open binary file-like object\n        (e.g., the output of `open(file_path, 'rb')`). Note: 1) Magika will\n        `seek()` around the stream; 2) the stream _is not closed_ (closing it is\n        the responsibility of the caller).\n        \"\"\"\n        if not isinstance(stream, io.IOBase) or not stream.readable():  # type: ignore[unreachable]\n            raise TypeError(\"Input stream must be a readable BinaryIO object.\")\n\n        # Explicitly test for the most common error so that we can return an\n        # helpful error message.\n        if isinstance(stream, io.TextIOBase):  # type: ignore[unreachable]\n            raise TypeError(\n                \"Input stream must be opened in bytes mode, not in text mode.\"\n            )\n\n        if not isinstance(stream, io.BufferedIOBase):\n            raise TypeError(\"Input stream must be a readable BinaryIO object.\")\n\n        if (\n            not hasattr(stream, \"seek\")\n            or not hasattr(stream, \"read\")\n            or not hasattr(stream, \"tell\")\n        ):\n            raise TypeError(\"Input stream must have seek, read, and tell methods.\")\n\n        try:\n            current_position = stream.tell()\n            result = self._get_result_from_seekable(Seekable(stream))\n        finally:\n            # seek to the previous position even in case of exceptions\n            stream.seek(current_position)\n        return result\n\n    def get_output_content_types(self) -> List[ContentTypeLabel]:\n        \"\"\"This method returns the list of all possible output content types.\n\n        I.e., all possible values for `MagikaResult.prediction.output.label`.\n        This considers the list of possible outputs from the model itself, but\n        also keeps into account additional configuration such as `override_map`\n        and special content types such as `empty` or `symlink`.\n\n        Consult the documentation for more details.\n        \"\"\"\n        target_labels_space = self._model_config.target_labels_space\n        overwrite_map = self._model_config.overwrite_map\n\n        output_content_types: Set[ContentTypeLabel] = {\n            ContentTypeLabel.DIRECTORY,\n            ContentTypeLabel.EMPTY,\n            ContentTypeLabel.SYMLINK,\n            ContentTypeLabel.TXT,\n            ContentTypeLabel.UNKNOWN,\n        }\n        for ct in target_labels_space:\n            # Check if we would overwrite this target label; if not, use the\n            # target label itself.\n            output_ct = overwrite_map.get(ct, ct)\n            output_content_types.add(output_ct)\n\n        return sorted(output_content_types)\n\n    def get_model_content_types(self) -> List[ContentTypeLabel]:\n        \"\"\"This method returns the list of all possible output of the model.\n\n        I.e., all possible values for `MagikaResult.prediction.dl.label`. Note\n        that, in general, the list of \"model outputs\" is different than the\n        \"tool outputs\" as in some cases the model is not even used, or the\n        model's output is overwritten due to a low-confidence score, or other\n        reasons. This API is useful mostly for debugging purposes; the vast\n        majority of client should use `get_output_content_types()`.\n\n        Consult the documentation for more details.\n        \"\"\"\n        model_content_types: Set[ContentTypeLabel] = {\n            ContentTypeLabel.UNDEFINED,\n        }\n        model_content_types.update(self._model_config.target_labels_space)\n        return sorted(model_content_types)\n\n    @staticmethod\n    def _get_default_model_name() -> str:\n        \"\"\"Returns the default model name.\n\n        This method is static so that it can be used by external clients/tests\n        without the need to instantiate a Magika object.\n        \"\"\"\n        return _DEFAULT_MODEL_NAME\n\n    @staticmethod\n    def _load_content_types_kb(\n        content_types_kb_json_path: Path,\n    ) -> Dict[ContentTypeLabel, ContentTypeInfo]:\n        TXT_MIME_TYPE = \"text/plain\"\n        UNKNOWN_MIME_TYPE = \"application/octet-stream\"\n        UNKNOWN_GROUP = \"unknown\"\n\n        out = {}\n        for ct_name, ct_info in json.loads(\n            content_types_kb_json_path.read_text()\n        ).items():\n            is_text = ct_info[\"is_text\"]\n            if is_text:\n                default_mime_type = TXT_MIME_TYPE\n            else:\n                default_mime_type = UNKNOWN_MIME_TYPE\n            mime_type = (\n                default_mime_type\n                if ct_info[\"mime_type\"] is None\n                else ct_info[\"mime_type\"]\n            )\n            group = UNKNOWN_GROUP if ct_info[\"group\"] is None else ct_info[\"group\"]\n            description = (\n                ct_name if ct_info[\"description\"] is None else ct_info[\"description\"]\n            )\n            extensions = ct_info[\"extensions\"]\n            out[ContentTypeLabel(ct_name)] = ContentTypeInfo(\n                label=ContentTypeLabel(ct_name),\n                mime_type=mime_type,\n                group=group,\n                description=description,\n                extensions=extensions,\n                is_text=is_text,\n            )\n        return out\n\n    @staticmethod\n    def _load_model_config(model_config_path: Path) -> ModelConfig:\n        config = json.loads(model_config_path.read_text())\n\n        return ModelConfig(\n            beg_size=config[\"beg_size\"],\n            mid_size=config[\"mid_size\"],\n            end_size=config[\"end_size\"],\n            use_inputs_at_offsets=config[\"use_inputs_at_offsets\"],\n            medium_confidence_threshold=config[\"medium_confidence_threshold\"],\n            min_file_size_for_dl=config[\"min_file_size_for_dl\"],\n            padding_token=config[\"padding_token\"],\n            block_size=config[\"block_size\"],\n            target_labels_space=[\n                ContentTypeLabel(ct_str) for ct_str in config[\"target_labels_space\"]\n            ],\n            thresholds={\n                ContentTypeLabel(k): v for k, v in config[\"thresholds\"].items()\n            },\n            overwrite_map={\n                ContentTypeLabel(k): ContentTypeLabel(v)\n                for k, v in config[\"overwrite_map\"].items()\n            },\n        )\n\n    def _init_onnx_session(self) -> rt.InferenceSession:\n        start_time = time.time()\n        rt.disable_telemetry_events()\n\n        onnx_session = rt.InferenceSession(\n            self._model_path,\n            providers=[\"CPUExecutionProvider\"],\n        )\n        elapsed_time = 1000 * (time.time() - start_time)\n        self._log.debug(\n            f'ONNX DL model \"{self._model_path}\" loaded in {elapsed_time:.03f} ms'\n        )\n        return onnx_session\n\n    def _get_ct_info(self, content_type: ContentTypeLabel) -> ContentTypeInfo:\n        return self._cts_infos[content_type]\n\n    def _get_results_from_paths(self, paths: List[Path]) -> List[MagikaResult]:\n        \"\"\"Get results for a list of paths.\n\n        Given a list of paths, returns a list of MagikaResult objects, which\n        contain relevant information, such as: file path, the output of the DL\n        model, the confidence score, the output of the tool, and associated\n        metadata. The order of the predictions matches the order of the input\n        paths.\n        \"\"\"\n        # We do a first pass on all files: we collect features for the files\n        # that need to be analyzed with the DL model, and we already determine\n        # the output for the remaining ones.\n\n        # We use a \"str\" instead of Path because it makes it easier later on to\n        # serialize.\n        all_outputs: Dict[str, MagikaResult] = {}  # {path: , ...}\n\n        # We use a list and not the dict because that's what we need later on\n        # for inference.\n        all_features: List[Tuple[Path, ModelFeatures]] = []\n\n        self._log.debug(\n            f\"Processing input files and extracting features for {len(paths)} samples\"\n        )\n        start_time = time.time()\n        for path in paths:\n            output, features = self._get_result_or_features_from_path(path)\n            if output is not None:\n                all_outputs[str(path)] = output\n            else:\n                assert features is not None\n                all_features.append((path, features))\n        elapsed_time = 1000 * (time.time() - start_time)\n        self._log.debug(f\"First pass and features extracted in {elapsed_time:.03f} ms\")\n\n        # Get the outputs via DL for the files that need it.\n        for path_str, result in self._get_results_from_features(all_features).items():\n            all_outputs[path_str] = result\n\n        # Finally, we collect the predictions in a final list, sorted by the\n        # initial paths list (and not by insertion order).\n        sorted_outputs = []\n        for path in paths:\n            sorted_outputs.append(all_outputs[str(path)])\n        return sorted_outputs\n\n    def _get_result_from_path(self, path: Path) -> MagikaResult:\n        return self._get_results_from_paths([path])[0]\n\n    def _get_result_from_seekable(self, seekable: Seekable) -> MagikaResult:\n        result, features = self._get_result_or_features_from_seekable(seekable)\n        if result is not None:\n            return result\n        assert features is not None\n        return self._get_results_from_features([(Path(\"-\"), features)])[\"-\"]\n\n    @staticmethod\n    def _extract_features_from_seekable(\n        seekable: Seekable,\n        beg_size: int,\n        mid_size: int,\n        end_size: int,\n        padding_token: int,\n        block_size: int,\n        use_inputs_at_offsets: bool,\n    ) -> ModelFeatures:\n        \"\"\"Extract features from an input seekable.\n\n        This implements features extraction v2 from a seekable, which is an\n        abstraction about anything that has a size and that can be \"read_at\" a\n        specific offset, such as a file or a buffer. This is implemented so that\n        we do not need to load the entire file in memory or scan the entire\n        buffer.\n\n        High-level overview on what we do:\n        - We read (at most) `block_size` bytes from the beginning and from the\n        end.\n        - We normalize these bytes by stripping whitespaces.\n        - We consider `beg_size` and `end_size` bytes as `beg` and `end`\n        features. If we don't have enough bytes, we use `padding_token` as\n        padding.\n\n        See comments below for the specifics and handling of corner cases.\n\n        NOTE: This implementation does not support extraction of `mid` features\n        and `use_inputs_at_offsets`.\n        \"\"\"\n        assert beg_size < block_size\n        assert mid_size == 0\n        assert end_size < block_size\n        assert not use_inputs_at_offsets\n\n        # we read at most block_size bytes\n        bytes_num_to_read = min(block_size, seekable.size)\n\n        if beg_size > 0:\n            # Read at most `block_size` bytes from the beginning; `lstrip()``\n            # them (or `strip()` them if the file size is less or equal than\n            # `block_size`); take at most `beg_size` bytes, and optionally pad\n            # them with `padding_token` to get to a list of `beg_size` integers.\n            beg_content = seekable.read_at(0, bytes_num_to_read)\n            beg_content = beg_content.lstrip()\n            beg_ints = Magika._get_beg_ints_with_padding(\n                beg_content, beg_size, padding_token\n            )\n        else:\n            beg_ints = []\n\n        if end_size > 0:\n            # Read at most `block_size` bytes from the end; `rstrip()`` them (or\n            # `strip()` them if the file size is less or equal than\n            # `block_size`); take at most `end_size` bytes (from the end), and\n            # optionally pad them (at the beginning) with `padding_token` to get\n            # to a list of `end_size` integers.\n            end_content = seekable.read_at(\n                seekable.size - bytes_num_to_read, bytes_num_to_read\n            )\n            end_content = end_content.rstrip()\n            end_ints = Magika._get_end_ints_with_padding(\n                end_content, end_size, padding_token\n            )\n        else:\n            end_ints = []\n\n        return ModelFeatures(\n            beg=beg_ints,\n            mid=[],\n            end=end_ints,\n            offset_0x8000_0x8007=[],\n            offset_0x8800_0x8807=[],\n            offset_0x9000_0x9007=[],\n            offset_0x9800_0x9807=[],\n        )\n\n    @staticmethod\n    def _get_beg_ints_with_padding(\n        beg_content: bytes, beg_size: int, padding_token: int\n    ) -> List[int]:\n        \"\"\"Take an (already-stripped) buffer as input and extract beg ints.\n\n        This returns a list of integers whose length is exactly beg_size. If\n        the buffer is bigger than required, take only the initial portion. If\n        the buffer is shorter, add padding at the end.\n        \"\"\"\n        if beg_size < len(beg_content):\n            # we don't need so many bytes\n            beg_content = beg_content[0:beg_size]\n\n        beg_ints = list(map(int, beg_content))\n\n        if len(beg_ints) < beg_size:\n            # we don't have enough ints, add padding\n            beg_ints = beg_ints + ([padding_token] * (beg_size - len(beg_ints)))\n\n        assert len(beg_ints) == beg_size\n\n        return beg_ints\n\n    @staticmethod\n    def _get_end_ints_with_padding(\n        end_content: bytes, end_size: int, padding_token: int\n    ) -> List[int]:\n        \"\"\"Take an (already-stripped) buffer as input and extract end ints.\n\n        This returns a list of integers whose length is exactly end_size. If the\n        buffer is bigger than required, take only the last portion. If the\n        buffer is shorter, add padding at the beginning.\n        \"\"\"\n        if end_size < len(end_content):\n            # we don't need so many bytes\n            end_content = end_content[len(end_content) - end_size : len(end_content)]\n\n        end_ints = list(map(int, end_content))\n\n        if len(end_ints) < end_size:\n            # we don't have enough ints, add padding\n            end_ints = ([padding_token] * (end_size - len(end_ints))) + end_ints\n\n        assert len(end_ints) == end_size\n\n        return end_ints\n\n    def _get_model_outputs_from_features(\n        self, all_features: List[Tuple[Path, ModelFeatures]]\n    ) -> List[Tuple[Path, ModelOutput]]:\n        raw_preds = self._get_raw_predictions(all_features)\n\n        outputs = []\n        for (path, _), preds in zip(all_features, raw_preds):\n            target_label_idx = max(range(len(preds)), key=preds.__getitem__)\n            score = preds[target_label_idx]\n            label = self._target_labels_space[target_label_idx]\n            outputs.append(\n                (path, ModelOutput(label=ContentTypeLabel(label), score=score))\n            )\n        return outputs\n\n    def _get_results_from_features(\n        self, all_features: List[Tuple[Path, ModelFeatures]]\n    ) -> Dict[str, MagikaResult]:\n        # We now do inference for those files that need it.\n\n        if len(all_features) == 0:\n            # nothing to be done\n            return {}\n\n        results: Dict[str, MagikaResult] = {}\n\n        for path, model_output in self._get_model_outputs_from_features(all_features):\n            # In additional to the content type label from the DL model, we\n            # also allow for other logic to overwrite such result. For\n            # debugging and information purposes, the JSON output stores\n            # both the raw DL model output and the final output we return to\n            # the user.\n\n            output_label, overwrite_reason = (\n                self._get_output_label_from_dl_label_and_score(\n                    model_output.label, model_output.score\n                )\n            )\n\n            results[str(path)] = self._get_result_from_labels_and_score(\n                path=path,\n                dl_label=model_output.label,\n                output_label=output_label,\n                score=model_output.score,\n                overwrite_reason=overwrite_reason,\n            )\n\n        return results\n\n    def _get_output_label_from_dl_label_and_score(\n        self, dl_label: ContentTypeLabel, score: float\n    ) -> Tuple[ContentTypeLabel, OverwriteReason]:\n        overwrite_reason = OverwriteReason.NONE\n\n        # Overwrite dl_label if specified in the overwrite_map model config.\n        output_label = self._model_config.overwrite_map.get(dl_label, dl_label)\n        if output_label != dl_label:\n            overwrite_reason = OverwriteReason.OVERWRITE_MAP\n\n        # The following code checks whether the score is \"high enough\", where\n        # \"high enough\" depends on the selected prediction mode. If the score is\n        # high enough, we return the (potentially overwritten) model prediction;\n        # if it is not, we return a generic content type, such as TXT or\n        # UNKNOWN.\n        if self._prediction_mode == PredictionMode.BEST_GUESS:\n            # We take the (potentially overwritten) model prediction, no matter\n            # what the score is.\n            pass\n        elif (\n            self._prediction_mode == PredictionMode.HIGH_CONFIDENCE\n            and score\n            >= self._model_config.thresholds.get(\n                dl_label, self._model_config.medium_confidence_threshold\n            )\n        ):\n            # The model score is higher than the per-content-type\n            # high-confidence threshold, so we keep it (note that the model\n            # prediction may have been overwritten).\n            pass\n        elif (\n            self._prediction_mode == PredictionMode.MEDIUM_CONFIDENCE\n            and score >= self._model_config.medium_confidence_threshold\n        ):\n            # The model score is higher than the generic medium-confidence\n            # threshold, so we keep it (note that the model prediction may have\n            # been overwritten).\n            pass\n        else:\n            # We are not in a condition to trust the model, we opt to return\n            # generic labels. Note that here we use an implicit assumption that\n            # the model has, at the very least, got the binary vs. text category\n            # right. This allows us to pick between unknown and txt without the\n            # need to read or scan the file bytes once again.\n            overwrite_reason = OverwriteReason.LOW_CONFIDENCE\n            if self._get_ct_info(output_label).is_text:\n                output_label = ContentTypeLabel.TXT\n            else:\n                output_label = ContentTypeLabel.UNKNOWN\n            if dl_label == output_label:\n                # overwrite_reason is useful to convey to clients why the output\n                # predicted is different than the model predicted type; if those\n                # two are the same, the model predicted type has not actually\n                # been overwritten, so we set this to NONE.\n                overwrite_reason = OverwriteReason.NONE\n\n        return output_label, overwrite_reason\n\n    def _get_result_from_labels_and_score(\n        self,\n        path: Path,\n        dl_label: ContentTypeLabel,\n        output_label: ContentTypeLabel,\n        score: float,\n        overwrite_reason: OverwriteReason = OverwriteReason.NONE,\n    ) -> MagikaResult:\n        return MagikaResult(\n            path=path,\n            prediction=MagikaPrediction(\n                dl=self._get_ct_info(dl_label),\n                output=self._get_ct_info(output_label),\n                score=score,\n                overwrite_reason=overwrite_reason,\n            ),\n        )\n\n    def _get_result_or_features_from_path(\n        self, path: Path\n    ) -> Tuple[Optional[MagikaResult], Optional[ModelFeatures]]:\n        \"\"\"Given a path, we return either a MagikaOutput or a MagikaFeatures.\n\n        There are some files and corner cases for which we do not need to use\n        deep learning to get the output; in these cases, we already return a\n        MagikaOutput object.\n\n        For some other files, we do need to use deep learning, in which case we\n        return a MagikaFeatures object. Note that for now we just collect the\n        features instead of already performing inference because we want to use\n        batching.\n        \"\"\"\n        if self._no_dereference and path.is_symlink():\n            result = self._get_result_from_labels_and_score(\n                path=path,\n                dl_label=ContentTypeLabel.UNDEFINED,\n                output_label=ContentTypeLabel.SYMLINK,\n                score=1.0,\n            )\n            return result, None\n\n        if not path.exists():\n            return MagikaResult(path=path, status=Status.FILE_NOT_FOUND_ERROR), None\n\n        if path.is_file():\n            if not os.access(path, os.R_OK):\n                return MagikaResult(path=path, status=Status.PERMISSION_ERROR), None\n\n            else:\n                # There are no additional path-specific corner cases, we can\n                # treat the input path as a stream.\n                with open(path, \"rb\") as stream:\n                    return self._get_result_or_features_from_seekable(\n                        Seekable(stream), path\n                    )\n\n        elif path.is_dir():\n            result = self._get_result_from_labels_and_score(\n                path=path,\n                dl_label=ContentTypeLabel.UNDEFINED,\n                output_label=ContentTypeLabel.DIRECTORY,\n                score=1.0,\n            )\n            return result, None\n\n        else:\n            result = self._get_result_from_labels_and_score(\n                path=path,\n                dl_label=ContentTypeLabel.UNDEFINED,\n                output_label=ContentTypeLabel.UNKNOWN,\n                score=1.0,\n            )\n            return result, None\n\n        raise Exception(\"unreachable\")\n\n    def _get_result_or_features_from_seekable(\n        self, seekable: Seekable, path: Path = Path(\"-\")\n    ) -> Tuple[Optional[MagikaResult], Optional[ModelFeatures]]:\n        \"\"\"Get result or features from a seekable object.\n\n        Given a Seekable object (which is a wrapper of BinaryIO), we return\n        either a MagikaOutput or a MagikaFeatures.\n\n        There are some corner cases for which we do not need to use deep\n        learning to get the output; in these cases, we return directly a\n        MagikaOutput object.\n\n        For all other cases, we do need to use deep learning, in which case we\n        return a MagikaFeatures object. Note that for now we just collect the\n        features instead of already performing inference because we want to use\n        batching.\n        \"\"\"\n        if seekable.size == 0:\n            result = self._get_result_from_labels_and_score(\n                path=path,\n                dl_label=ContentTypeLabel.UNDEFINED,\n                output_label=ContentTypeLabel.EMPTY,\n                score=1.0,\n            )\n            return result, None\n\n        elif seekable.size < self._model_config.min_file_size_for_dl:\n            content = seekable.read_at(0, seekable.size)\n            result = self._get_result_from_few_bytes(content, path=path)\n            return result, None\n\n        else:\n            file_features = Magika._extract_features_from_seekable(\n                seekable,\n                self._model_config.beg_size,\n                self._model_config.mid_size,\n                self._model_config.end_size,\n                self._model_config.padding_token,\n                self._model_config.block_size,\n                self._model_config.use_inputs_at_offsets,\n            )\n            # Check whether we have enough bytes for a meaningful\n            # detection, and not just padding.\n            if (\n                file_features.beg[self._model_config.min_file_size_for_dl - 1]\n                == self._model_config.padding_token\n            ):\n                # If the n-th token is padding, then it means that,\n                # post-stripping, we do not have enough meaningful\n                # bytes.\n                bytes_to_read = min(seekable.size, self._model_config.block_size)\n                content = seekable.read_at(0, bytes_to_read)\n                result = self._get_result_from_few_bytes(content, path=path)\n                return result, None\n\n            else:\n                # We have enough bytes, return the features for a model\n                # prediction.\n                return None, file_features\n\n        raise Exception(\"unreachable\")\n\n    def _get_result_from_few_bytes(\n        self, content: bytes, path: Path = Path(\"-\")\n    ) -> MagikaResult:\n        assert len(content) <= 4 * self._model_config.block_size\n        label = self._get_label_from_few_bytes(content)\n        return self._get_result_from_labels_and_score(\n            path=path,\n            dl_label=ContentTypeLabel.UNDEFINED,\n            output_label=label,\n            score=1.0,\n        )\n\n    def _get_label_from_few_bytes(self, content: bytes) -> ContentTypeLabel:\n        try:\n            label = ContentTypeLabel.TXT\n            _ = content.decode(\"utf-8\")\n        except UnicodeDecodeError:\n            label = ContentTypeLabel.UNKNOWN\n        return label\n\n    def _get_raw_predictions(\n        self, features: List[Tuple[Path, ModelFeatures]]\n    ) -> List[List[float]]:\n        \"\"\"Get raw predictions from features.\n\n        Given a list of (path, features), return a (files_num, features_size)\n        matrix encoding the predictions.\n        \"\"\"\n        start_time = time.time()\n        X_bytes = []\n        for _, fs in features:\n            sample_bytes = []\n            if self._model_config.beg_size > 0:\n                sample_bytes.extend(fs.beg[: self._model_config.beg_size])\n            if self._model_config.mid_size > 0:\n                sample_bytes.extend(fs.mid[: self._model_config.mid_size])\n            if self._model_config.end_size > 0:\n                sample_bytes.extend(fs.end[-self._model_config.end_size :])\n            X_bytes.append(sample_bytes)\n\n        elapsed_time = 1000 * (time.time() - start_time)\n        self._log.debug(f\"DL input prepared in {elapsed_time:.03f} ms\")\n\n        raw_predictions_list = []\n        samples_num = len(X_bytes)\n\n        max_internal_batch_size = 1000\n        batches_num = samples_num // max_internal_batch_size\n        if samples_num % max_internal_batch_size != 0:\n            batches_num += 1\n\n        for batch_idx in range(batches_num):\n            self._log.debug(\n                f\"Getting raw predictions for (internal) batch {batch_idx + 1}/{batches_num}\"\n            )\n            start_idx = batch_idx * max_internal_batch_size\n            end_idx = min((batch_idx + 1) * max_internal_batch_size, samples_num)\n\n            batch_features = X_bytes[start_idx:end_idx]\n\n            start_time = time.time()\n            # onnxruntime accepts simple list of lists of ints/floats for input \"bytes\"\n            # It returns a list of numpy arrays (usually one per output node).\n            batch_raw_predictions_np = self._onnx_session.run(\n                [\"target_label\"], {\"bytes\": batch_features}\n            )[0]\n            # Convert numpy array to list of lists of floats immediately\n            batch_raw_predictions = batch_raw_predictions_np.tolist()\n\n            elapsed_time = 1000 * (time.time() - start_time)\n            self._log.debug(f\"DL raw prediction in {elapsed_time:.03f} ms\")\n\n            raw_predictions_list.extend(batch_raw_predictions)\n        return raw_predictions_list\n"
  },
  {
    "path": "python/src/magika/models/standard_v3_3/README.md",
    "content": "# Model documentation\n\n## Table of Contents\n\n1. [List of possible outputs](#list-of-possible-outputs)\n1. [List of possible model's outputs](#list-of-possible-models-outputs)\n\n## List of possible outputs\n\nThis is the full list of all possible tool's outputs (which are different than the possible raw output of the model, see table below). E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.output.label`.\n\n| Index   |      Content Type Label      | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | directory | A directory |\n| 40 | dm | Dream Maker |\n| 41 | dmg | Apple disk image |\n| 42 | doc | Microsoft Word CDF document |\n| 43 | dockerfile | Dockerfile |\n| 44 | docx | Microsoft Word 2007+ document |\n| 45 | dsstore | Application Desktop Services Store |\n| 46 | dwg | Autocad Drawing |\n| 47 | dxf | Audocad Drawing Exchange Format |\n| 48 | elf | ELF executable |\n| 49 | elixir | Elixir script |\n| 50 | emf | Windows Enhanced Metafile image data |\n| 51 | eml | RFC 822 mail |\n| 52 | empty | Empty file |\n| 53 | epub | EPUB document |\n| 54 | erb | Embedded Ruby source |\n| 55 | erlang | Erlang source |\n| 56 | flac | FLAC audio bitstream data |\n| 57 | flv | Flash Video |\n| 58 | fortran | Fortran |\n| 59 | gemfile | Gemfile file |\n| 60 | gemspec | Gemspec file |\n| 61 | gif | GIF image data |\n| 62 | gitattributes | Gitattributes file |\n| 63 | gitmodules | Gitmodules file |\n| 64 | go | Golang source |\n| 65 | gradle | Gradle source |\n| 66 | groovy | Groovy source |\n| 67 | gzip | gzip compressed data |\n| 68 | h5 | Hierarchical Data Format v5 |\n| 69 | handlebars | Handlebars source |\n| 70 | haskell | Haskell source |\n| 71 | hcl | HashiCorp configuration language |\n| 72 | hlp | MS Windows help |\n| 73 | htaccess | Apache access configuration |\n| 74 | html | HTML document |\n| 75 | icns | Mac OS X icon |\n| 76 | ico | MS Windows icon resource |\n| 77 | ics | Internet Calendaring and Scheduling |\n| 78 | ignorefile | Ignorefile |\n| 79 | ini | INI configuration file |\n| 80 | internetshortcut | MS Windows Internet shortcut |\n| 81 | ipynb | Jupyter notebook |\n| 82 | iso | ISO 9660 CD-ROM filesystem data |\n| 83 | jar | Java archive data (JAR) |\n| 84 | java | Java source |\n| 85 | javabytecode | Java compiled bytecode |\n| 86 | javascript | JavaScript source |\n| 87 | jinja | Jinja template |\n| 88 | jp2 | jpeg2000 |\n| 89 | jpeg | JPEG image data |\n| 90 | json | JSON document |\n| 91 | jsonl | JSONL document |\n| 92 | julia | Julia source |\n| 93 | kotlin | Kotlin source |\n| 94 | latex | LaTeX document |\n| 95 | lha | LHarc archive |\n| 96 | lisp | Lisp source |\n| 97 | lnk | MS Windows shortcut |\n| 98 | lua | Lua |\n| 99 | m3u | M3U playlist |\n| 100 | m4 | GNU Macro |\n| 101 | macho | Mach-O executable |\n| 102 | makefile | Makefile source |\n| 103 | markdown | Markdown document |\n| 104 | matlab | Matlab Source |\n| 105 | mht | MHTML document |\n| 106 | midi | Midi |\n| 107 | mkv | Matroska |\n| 108 | mp3 | MP3 media file |\n| 109 | mp4 | MP4 media file |\n| 110 | mscompress | MS Compress archive data |\n| 111 | msi | Microsoft Installer file |\n| 112 | mum | Windows Update Package file |\n| 113 | npy | Numpy Array |\n| 114 | npz | Numpy Arrays Archive |\n| 115 | nupkg | NuGet Package |\n| 116 | objectivec | ObjectiveC source |\n| 117 | ocaml | OCaml |\n| 118 | odp | OpenDocument Presentation |\n| 119 | ods | OpenDocument Spreadsheet |\n| 120 | odt | OpenDocument Text |\n| 121 | ogg | Ogg data |\n| 122 | one | One Note |\n| 123 | onnx | Open Neural Network Exchange |\n| 124 | otf | OpenType font |\n| 125 | outlook | MS Outlook Message |\n| 126 | parquet | Apache Parquet |\n| 127 | pascal | Pascal source |\n| 128 | pcap | pcap capture file |\n| 129 | pdb | Windows Program Database |\n| 130 | pdf | PDF document |\n| 131 | pebin | PE Windows executable |\n| 132 | pem | PEM certificate |\n| 133 | perl | Perl source |\n| 134 | php | PHP source |\n| 135 | pickle | Python pickle |\n| 136 | png | PNG image |\n| 137 | po | Portable Object (PO) for i18n |\n| 138 | postscript | PostScript document |\n| 139 | powershell | Powershell source |\n| 140 | ppt | Microsoft PowerPoint CDF document |\n| 141 | pptx | Microsoft PowerPoint 2007+ document |\n| 142 | prolog | Prolog source |\n| 143 | proteindb | Protein DB |\n| 144 | proto | Protocol buffer definition |\n| 145 | psd | Adobe Photoshop |\n| 146 | python | Python source |\n| 147 | pythonbytecode | Python compiled bytecode |\n| 148 | pytorch | Pytorch storage file |\n| 149 | qt | QuickTime |\n| 150 | r | R (language) |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | symlink | Symbolic link |\n| 177 | tar | POSIX tar archive |\n| 178 | tcl | Tickle |\n| 179 | textproto | Text protocol buffer |\n| 180 | tga | Targa image data |\n| 181 | thumbsdb | Windows thumbnail cache |\n| 182 | tiff | TIFF image data |\n| 183 | toml | Tom's obvious, minimal language |\n| 184 | torrent | BitTorrent file |\n| 185 | tsv | TSV document |\n| 186 | ttf | TrueType Font data |\n| 187 | twig | Twig template |\n| 188 | txt | Generic text document |\n| 189 | typescript | TypeScript source |\n| 190 | unknown | Unknown binary data |\n| 191 | vba | MS Visual Basic source (VBA) |\n| 192 | vcxproj | Visual Studio MSBuild project |\n| 193 | verilog | Verilog source |\n| 194 | vhdl | VHDL source |\n| 195 | vtt | Web Video Text Tracks |\n| 196 | vue | Vue source |\n| 197 | wasm | Web Assembly |\n| 198 | wav | Waveform Audio file (WAV) |\n| 199 | webm | WebM media file |\n| 200 | webp | WebP media file |\n| 201 | winregistry | Windows Registry text |\n| 202 | wmf | Windows metafile |\n| 203 | woff | Web Open Font Format |\n| 204 | woff2 | Web Open Font Format v2 |\n| 205 | xar | XAR archive compressed data |\n| 206 | xls | Microsoft Excel CDF document |\n| 207 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 208 | xlsx | Microsoft Excel 2007+ document |\n| 209 | xml | XML document |\n| 210 | xpi | Compressed installation archive (XPI) |\n| 211 | xz | XZ compressed data |\n| 212 | yaml | YAML source |\n| 213 | yara | YARA rule |\n| 214 | zig | Zig source |\n| 215 | zip | Zip archive data |\n| 216 | zlibstream | zlib compressed data |\n\n\n## List of possible model's outputs\n\nThis is the full list of all possible model's output. E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.dl.label`. Note that, in general, the list of \"model outputs\" is different than the \"tool outputs\" as in some cases the model is not even used, or the model's output is overwritten due to a low-confidence score or other reasons. This list is useful mostly for debugging purposes; the vast majority of client should just consult the table above.\n\n| Index   |      Content Type Label      | Description |\n|----------|:-------------:|------|\n| 1 | 3gp | 3GPP multimedia file |\n| 2 | ace | ACE archive |\n| 3 | ai | Adobe Illustrator Artwork |\n| 4 | aidl | Android Interface Definition Language |\n| 5 | apk | Android package |\n| 6 | applebplist | Apple binary property list |\n| 7 | appleplist | Apple property list |\n| 8 | asm | Assembly |\n| 9 | asp | ASP source |\n| 10 | autohotkey | AutoHotKey script |\n| 11 | autoit | AutoIt script |\n| 12 | awk | Awk |\n| 13 | batch | DOS batch file |\n| 14 | bazel | Bazel build file |\n| 15 | bib | BibTeX |\n| 16 | bmp | BMP image data |\n| 17 | bzip | bzip2 compressed data |\n| 18 | c | C source |\n| 19 | cab | Microsoft Cabinet archive data |\n| 20 | cat | Windows Catalog file |\n| 21 | chm | MS Windows HtmlHelp Data |\n| 22 | clojure | Clojure |\n| 23 | cmake | CMake build file |\n| 24 | cobol | Cobol |\n| 25 | coff | Intel 80386 COFF |\n| 26 | coffeescript | CoffeeScript |\n| 27 | cpp | C++ source |\n| 28 | crt | Certificates (binary format) |\n| 29 | crx | Google Chrome extension |\n| 30 | cs | C# source |\n| 31 | csproj | .NET project config |\n| 32 | css | CSS source |\n| 33 | csv | CSV document |\n| 34 | dart | Dart source |\n| 35 | deb | Debian binary package |\n| 36 | dex | Dalvik dex file |\n| 37 | dicom | DICOM |\n| 38 | diff | Diff file |\n| 39 | dm | Dream Maker |\n| 40 | dmg | Apple disk image |\n| 41 | doc | Microsoft Word CDF document |\n| 42 | dockerfile | Dockerfile |\n| 43 | docx | Microsoft Word 2007+ document |\n| 44 | dsstore | Application Desktop Services Store |\n| 45 | dwg | Autocad Drawing |\n| 46 | dxf | Audocad Drawing Exchange Format |\n| 47 | elf | ELF executable |\n| 48 | elixir | Elixir script |\n| 49 | emf | Windows Enhanced Metafile image data |\n| 50 | eml | RFC 822 mail |\n| 51 | epub | EPUB document |\n| 52 | erb | Embedded Ruby source |\n| 53 | erlang | Erlang source |\n| 54 | flac | FLAC audio bitstream data |\n| 55 | flv | Flash Video |\n| 56 | fortran | Fortran |\n| 57 | gemfile | Gemfile file |\n| 58 | gemspec | Gemspec file |\n| 59 | gif | GIF image data |\n| 60 | gitattributes | Gitattributes file |\n| 61 | gitmodules | Gitmodules file |\n| 62 | go | Golang source |\n| 63 | gradle | Gradle source |\n| 64 | groovy | Groovy source |\n| 65 | gzip | gzip compressed data |\n| 66 | h5 | Hierarchical Data Format v5 |\n| 67 | handlebars | Handlebars source |\n| 68 | haskell | Haskell source |\n| 69 | hcl | HashiCorp configuration language |\n| 70 | hlp | MS Windows help |\n| 71 | htaccess | Apache access configuration |\n| 72 | html | HTML document |\n| 73 | icns | Mac OS X icon |\n| 74 | ico | MS Windows icon resource |\n| 75 | ics | Internet Calendaring and Scheduling |\n| 76 | ignorefile | Ignorefile |\n| 77 | ini | INI configuration file |\n| 78 | internetshortcut | MS Windows Internet shortcut |\n| 79 | ipynb | Jupyter notebook |\n| 80 | iso | ISO 9660 CD-ROM filesystem data |\n| 81 | jar | Java archive data (JAR) |\n| 82 | java | Java source |\n| 83 | javabytecode | Java compiled bytecode |\n| 84 | javascript | JavaScript source |\n| 85 | jinja | Jinja template |\n| 86 | jp2 | jpeg2000 |\n| 87 | jpeg | JPEG image data |\n| 88 | json | JSON document |\n| 89 | jsonl | JSONL document |\n| 90 | julia | Julia source |\n| 91 | kotlin | Kotlin source |\n| 92 | latex | LaTeX document |\n| 93 | lha | LHarc archive |\n| 94 | lisp | Lisp source |\n| 95 | lnk | MS Windows shortcut |\n| 96 | lua | Lua |\n| 97 | m3u | M3U playlist |\n| 98 | m4 | GNU Macro |\n| 99 | macho | Mach-O executable |\n| 100 | makefile | Makefile source |\n| 101 | markdown | Markdown document |\n| 102 | matlab | Matlab Source |\n| 103 | mht | MHTML document |\n| 104 | midi | Midi |\n| 105 | mkv | Matroska |\n| 106 | mp3 | MP3 media file |\n| 107 | mp4 | MP4 media file |\n| 108 | mscompress | MS Compress archive data |\n| 109 | msi | Microsoft Installer file |\n| 110 | mum | Windows Update Package file |\n| 111 | npy | Numpy Array |\n| 112 | npz | Numpy Arrays Archive |\n| 113 | nupkg | NuGet Package |\n| 114 | objectivec | ObjectiveC source |\n| 115 | ocaml | OCaml |\n| 116 | odp | OpenDocument Presentation |\n| 117 | ods | OpenDocument Spreadsheet |\n| 118 | odt | OpenDocument Text |\n| 119 | ogg | Ogg data |\n| 120 | one | One Note |\n| 121 | onnx | Open Neural Network Exchange |\n| 122 | otf | OpenType font |\n| 123 | outlook | MS Outlook Message |\n| 124 | parquet | Apache Parquet |\n| 125 | pascal | Pascal source |\n| 126 | pcap | pcap capture file |\n| 127 | pdb | Windows Program Database |\n| 128 | pdf | PDF document |\n| 129 | pebin | PE Windows executable |\n| 130 | pem | PEM certificate |\n| 131 | perl | Perl source |\n| 132 | php | PHP source |\n| 133 | pickle | Python pickle |\n| 134 | png | PNG image |\n| 135 | po | Portable Object (PO) for i18n |\n| 136 | postscript | PostScript document |\n| 137 | powershell | Powershell source |\n| 138 | ppt | Microsoft PowerPoint CDF document |\n| 139 | pptx | Microsoft PowerPoint 2007+ document |\n| 140 | prolog | Prolog source |\n| 141 | proteindb | Protein DB |\n| 142 | proto | Protocol buffer definition |\n| 143 | psd | Adobe Photoshop |\n| 144 | python | Python source |\n| 145 | pythonbytecode | Python compiled bytecode |\n| 146 | pytorch | Pytorch storage file |\n| 147 | qt | QuickTime |\n| 148 | r | R (language) |\n| 149 | randombytes | Random bytes |\n| 150 | randomtxt | Random text |\n| 151 | rar | RAR archive data |\n| 152 | rdf | Resource Description Framework document (RDF) |\n| 153 | rpm | RedHat Package Manager archive (RPM) |\n| 154 | rst | ReStructuredText document |\n| 155 | rtf | Rich Text Format document |\n| 156 | ruby | Ruby source |\n| 157 | rust | Rust source |\n| 158 | scala | Scala source |\n| 159 | scss | SCSS source |\n| 160 | sevenzip | 7-zip archive data |\n| 161 | sgml | sgml |\n| 162 | shell | Shell script |\n| 163 | smali | Smali source |\n| 164 | snap | Snap archive |\n| 165 | solidity | Solidity source |\n| 166 | sql | SQL source |\n| 167 | sqlite | SQLITE database |\n| 168 | squashfs | Squash filesystem |\n| 169 | srt | SubRip Text Format |\n| 170 | stlbinary | Stereolithography CAD (binary) |\n| 171 | stltext | Stereolithography CAD (text) |\n| 172 | sum | Checksum file |\n| 173 | svg | SVG Scalable Vector Graphics image data |\n| 174 | swf | Small Web File |\n| 175 | swift | Swift |\n| 176 | tar | POSIX tar archive |\n| 177 | tcl | Tickle |\n| 178 | textproto | Text protocol buffer |\n| 179 | tga | Targa image data |\n| 180 | thumbsdb | Windows thumbnail cache |\n| 181 | tiff | TIFF image data |\n| 182 | toml | Tom's obvious, minimal language |\n| 183 | torrent | BitTorrent file |\n| 184 | tsv | TSV document |\n| 185 | ttf | TrueType Font data |\n| 186 | twig | Twig template |\n| 187 | txt | Generic text document |\n| 188 | typescript | TypeScript source |\n| 189 | undefined | Undefined |\n| 190 | vba | MS Visual Basic source (VBA) |\n| 191 | vcxproj | Visual Studio MSBuild project |\n| 192 | verilog | Verilog source |\n| 193 | vhdl | VHDL source |\n| 194 | vtt | Web Video Text Tracks |\n| 195 | vue | Vue source |\n| 196 | wasm | Web Assembly |\n| 197 | wav | Waveform Audio file (WAV) |\n| 198 | webm | WebM media file |\n| 199 | webp | WebP media file |\n| 200 | winregistry | Windows Registry text |\n| 201 | wmf | Windows metafile |\n| 202 | woff | Web Open Font Format |\n| 203 | woff2 | Web Open Font Format v2 |\n| 204 | xar | XAR archive compressed data |\n| 205 | xls | Microsoft Excel CDF document |\n| 206 | xlsb | Microsoft Excel 2007+ document (binary format) |\n| 207 | xlsx | Microsoft Excel 2007+ document |\n| 208 | xml | XML document |\n| 209 | xpi | Compressed installation archive (XPI) |\n| 210 | xz | XZ compressed data |\n| 211 | yaml | YAML source |\n| 212 | yara | YARA rule |\n| 213 | zig | Zig source |\n| 214 | zip | Zip archive data |\n| 215 | zlibstream | zlib compressed data |>"
  },
  {
    "path": "python/src/magika/models/standard_v3_3/config.min.json",
    "content": "{\"beg_size\":1024,\"mid_size\":0,\"end_size\":1024,\"use_inputs_at_offsets\":false,\"medium_confidence_threshold\":0.5,\"min_file_size_for_dl\":8,\"padding_token\":256,\"block_size\":4096,\"target_labels_space\":[\"3gp\",\"ace\",\"ai\",\"aidl\",\"apk\",\"applebplist\",\"appleplist\",\"asm\",\"asp\",\"autohotkey\",\"autoit\",\"awk\",\"batch\",\"bazel\",\"bib\",\"bmp\",\"bzip\",\"c\",\"cab\",\"cat\",\"chm\",\"clojure\",\"cmake\",\"cobol\",\"coff\",\"coffeescript\",\"cpp\",\"crt\",\"crx\",\"cs\",\"csproj\",\"css\",\"csv\",\"dart\",\"deb\",\"dex\",\"dicom\",\"diff\",\"dm\",\"dmg\",\"doc\",\"dockerfile\",\"docx\",\"dsstore\",\"dwg\",\"dxf\",\"elf\",\"elixir\",\"emf\",\"eml\",\"epub\",\"erb\",\"erlang\",\"flac\",\"flv\",\"fortran\",\"gemfile\",\"gemspec\",\"gif\",\"gitattributes\",\"gitmodules\",\"go\",\"gradle\",\"groovy\",\"gzip\",\"h5\",\"handlebars\",\"haskell\",\"hcl\",\"hlp\",\"htaccess\",\"html\",\"icns\",\"ico\",\"ics\",\"ignorefile\",\"ini\",\"internetshortcut\",\"ipynb\",\"iso\",\"jar\",\"java\",\"javabytecode\",\"javascript\",\"jinja\",\"jp2\",\"jpeg\",\"json\",\"jsonl\",\"julia\",\"kotlin\",\"latex\",\"lha\",\"lisp\",\"lnk\",\"lua\",\"m3u\",\"m4\",\"macho\",\"makefile\",\"markdown\",\"matlab\",\"mht\",\"midi\",\"mkv\",\"mp3\",\"mp4\",\"mscompress\",\"msi\",\"mum\",\"npy\",\"npz\",\"nupkg\",\"objectivec\",\"ocaml\",\"odp\",\"ods\",\"odt\",\"ogg\",\"one\",\"onnx\",\"otf\",\"outlook\",\"parquet\",\"pascal\",\"pcap\",\"pdb\",\"pdf\",\"pebin\",\"pem\",\"perl\",\"php\",\"pickle\",\"png\",\"po\",\"postscript\",\"powershell\",\"ppt\",\"pptx\",\"prolog\",\"proteindb\",\"proto\",\"psd\",\"python\",\"pythonbytecode\",\"pytorch\",\"qt\",\"r\",\"randombytes\",\"randomtxt\",\"rar\",\"rdf\",\"rpm\",\"rst\",\"rtf\",\"ruby\",\"rust\",\"scala\",\"scss\",\"sevenzip\",\"sgml\",\"shell\",\"smali\",\"snap\",\"solidity\",\"sql\",\"sqlite\",\"squashfs\",\"srt\",\"stlbinary\",\"stltext\",\"sum\",\"svg\",\"swf\",\"swift\",\"tar\",\"tcl\",\"textproto\",\"tga\",\"thumbsdb\",\"tiff\",\"toml\",\"torrent\",\"tsv\",\"ttf\",\"twig\",\"txt\",\"typescript\",\"vba\",\"vcxproj\",\"verilog\",\"vhdl\",\"vtt\",\"vue\",\"wasm\",\"wav\",\"webm\",\"webp\",\"winregistry\",\"wmf\",\"woff\",\"woff2\",\"xar\",\"xls\",\"xlsb\",\"xlsx\",\"xml\",\"xpi\",\"xz\",\"yaml\",\"yara\",\"zig\",\"zip\",\"zlibstream\"],\"thresholds\":{\"crt\":0.9,\"handlebars\":0.9,\"ignorefile\":0.95,\"latex\":0.95,\"markdown\":0.75,\"ocaml\":0.9,\"pascal\":0.95,\"r\":0.9,\"rst\":0.9,\"sql\":0.9,\"tsv\":0.9,\"zig\":0.9},\"overwrite_map\":{\"randombytes\":\"unknown\",\"randomtxt\":\"txt\"},\"protection\":\"none\",\"aes_key_hex\":\"\",\"version_major\":3}\n"
  },
  {
    "path": "python/src/magika/models/standard_v3_3/metadata.json",
    "content": "{\"epoch_num\":\"91\"}\n"
  },
  {
    "path": "python/src/magika/py.typed",
    "content": ""
  },
  {
    "path": "python/src/magika/types/__init__.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D104\n\n\nfrom magika.types.content_type_info import ContentTypeInfo  # noqa: F401\nfrom magika.types.content_type_label import ContentTypeLabel  # noqa: F401\nfrom magika.types.magika_error import MagikaError  # noqa: F401\nfrom magika.types.magika_prediction import MagikaPrediction  # noqa: F401\nfrom magika.types.magika_result import MagikaResult  # noqa: F401\nfrom magika.types.model import (  # noqa: F401\n    ModelConfig,\n    ModelFeatures,\n    ModelOutput,\n)\nfrom magika.types.overwrite_reason import OverwriteReason  # noqa: F401\nfrom magika.types.prediction_mode import PredictionMode  # noqa: F401\nfrom magika.types.seekable import Seekable  # noqa: F401\nfrom magika.types.status import Status  # noqa: F401\n\n__all__ = [\n    \"ContentTypeInfo\",\n    \"ContentTypeLabel\",\n    \"MagikaError\",\n    \"MagikaPrediction\",\n    \"MagikaResult\",\n    \"ModelConfig\",\n    \"ModelFeatures\",\n    \"ModelOutput\",\n    \"OverwriteReason\",\n    \"PredictionMode\",\n    \"Seekable\",\n    \"Status\",\n]\n"
  },
  {
    "path": "python/src/magika/types/content_type_info.py",
    "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Module defining the ContentTypeInfo dataclass.\"\"\"\n\nimport warnings\nfrom dataclasses import dataclass\nfrom typing import List\n\nfrom magika.logger import get_logger\nfrom magika.types.content_type_label import ContentTypeLabel\n\n\n@dataclass(frozen=True)\nclass ContentTypeInfo:\n    \"\"\"Dataclass holding information about a content type.\n\n    Attributes:\n        label: The ContentTypeLabel enum value.\n        mime_type: The mime type associated to the content type.\n        group: A high-level category for the content type (e.g., \"document\",\n            \"image\").\n        description: A human-readable description.\n        extensions: A list of common file extensions.\n        is_text: A boolean indicating if the content type is text-based.\n    \"\"\"\n\n    label: ContentTypeLabel\n    mime_type: str\n    group: str\n    description: str\n    extensions: List[str]\n    is_text: bool\n\n    @property\n    def ct_label(self) -> str:\n        \"\"\"DEPRECATED: Returns the string value of the content type label.\n\n        Warns:\n            DeprecationWarning: This property is deprecated. Use `.label`\n                instead.\n        \"\"\"\n        warnings.warn(\n            \"`.ct_label` is deprecated and will be removed in a future version. Use `.label` instead. Consult the documentation for more information.\",\n            category=DeprecationWarning,\n            stacklevel=2,\n        )\n        return str(self.label)\n\n    @property\n    def score(self) -> float:\n        \"\"\"UNSUPPORTED: This property is no longer supported and raises an error.\n\n        Raises:\n            AttributeError: This property is unsupported. The score is now on\n                the MagikaResult object.\n        \"\"\"\n        error_msg = \"Unsupported field error: `.score.` is not stored anymore in the `dl` or `output` objects; it is now stored in `MagikaResult`. Consult the documentation for more information.\"\n        log = get_logger()\n        log.error(error_msg)\n        raise AttributeError(error_msg)\n\n    @property\n    def magic(self) -> str:\n        \"\"\"DEPRECATED: Returns the description of the content type.\n\n        Warns:\n            DeprecationWarning: This property is deprecated. Use\n                `.description` instead.\n        \"\"\"\n        warnings.warn(\n            \"`.magic` is deprecated and will be removed in a future version. Use `.description` instead. Consult the documentation for more information.\",\n            category=DeprecationWarning,\n            stacklevel=2,\n        )\n        return self.description\n"
  },
  {
    "path": "python/src/magika/types/content_type_label.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Enumeration of all known content type labels.\"\"\"\n\nfrom magika.types.strenum import StrEnum\n\n# NOTE: DO NOT EDIT --- This file is automatically generated.\n\n\n# This is the list of all possible content types we know about; however, models\n# support a smaller subset of them. See model's README.md for details.\nclass ContentTypeLabel(StrEnum):\n    \"\"\"A string-based enumeration of all possible content type labels.\n\n    This enum provides a standardized set of identifiers for content types\n    recognized by Magika.\n    \"\"\"\n\n    _3DS = \"3ds\"\n    _3DSM = \"3dsm\"\n    _3DSX = \"3dsx\"\n    _3GP = \"3gp\"\n    _3MF = \"3mf\"\n    ABNF = \"abnf\"\n    ACE = \"ace\"\n    ADA = \"ada\"\n    AFF = \"aff\"\n    AI = \"ai\"\n    AIDL = \"aidl\"\n    ALGOL68 = \"algol68\"\n    ANI = \"ani\"\n    APK = \"apk\"\n    APPLEBPLIST = \"applebplist\"\n    APPLEDOUBLE = \"appledouble\"\n    APPLEPLIST = \"appleplist\"\n    APPLESINGLE = \"applesingle\"\n    AR = \"ar\"\n    ARC = \"arc\"\n    ARJ = \"arj\"\n    ARROW = \"arrow\"\n    ASC = \"asc\"\n    ASD = \"asd\"\n    ASF = \"asf\"\n    ASM = \"asm\"\n    ASP = \"asp\"\n    AU = \"au\"\n    AUTOHOTKEY = \"autohotkey\"\n    AUTOIT = \"autoit\"\n    AVI = \"avi\"\n    AVIF = \"avif\"\n    AVRO = \"avro\"\n    AWK = \"awk\"\n    AX = \"ax\"\n    BATCH = \"batch\"\n    BAZEL = \"bazel\"\n    BCAD = \"bcad\"\n    BIB = \"bib\"\n    BMP = \"bmp\"\n    BPG = \"bpg\"\n    BPL = \"bpl\"\n    BRAINFUCK = \"brainfuck\"\n    BRF = \"brf\"\n    BZIP = \"bzip\"\n    BZIP3 = \"bzip3\"\n    C = \"c\"\n    CAB = \"cab\"\n    CAD = \"cad\"\n    CAT = \"cat\"\n    CDF = \"cdf\"\n    CHM = \"chm\"\n    CLOJURE = \"clojure\"\n    CMAKE = \"cmake\"\n    COBOL = \"cobol\"\n    COFF = \"coff\"\n    COFFEESCRIPT = \"coffeescript\"\n    COM = \"com\"\n    CPL = \"cpl\"\n    CPP = \"cpp\"\n    CRT = \"crt\"\n    CRX = \"crx\"\n    CS = \"cs\"\n    CSPROJ = \"csproj\"\n    CSS = \"css\"\n    CSV = \"csv\"\n    CTL = \"ctl\"\n    DART = \"dart\"\n    DEB = \"deb\"\n    DEX = \"dex\"\n    DEY = \"dey\"\n    DICOM = \"dicom\"\n    DIFF = \"diff\"\n    DIRECTORY = \"directory\"\n    DJANGO = \"django\"\n    DLL = \"dll\"\n    DM = \"dm\"\n    DMG = \"dmg\"\n    DMIGD = \"dmigd\"\n    DMSCRIPT = \"dmscript\"\n    DOC = \"doc\"\n    DOCKERFILE = \"dockerfile\"\n    DOCX = \"docx\"\n    DOSMBR = \"dosmbr\"\n    DOTX = \"dotx\"\n    DSSTORE = \"dsstore\"\n    DWG = \"dwg\"\n    DXF = \"dxf\"\n    DYLIB = \"dylib\"\n    EBML = \"ebml\"\n    ELF = \"elf\"\n    ELIXIR = \"elixir\"\n    EMF = \"emf\"\n    EML = \"eml\"\n    EMPTY = \"empty\"\n    EPUB = \"epub\"\n    ERB = \"erb\"\n    ERLANG = \"erlang\"\n    ESE = \"ese\"\n    EXE = \"exe\"\n    EXP = \"exp\"\n    FLAC = \"flac\"\n    FLUTTER = \"flutter\"\n    FLV = \"flv\"\n    FORTRAN = \"fortran\"\n    FPX = \"fpx\"\n    GEMFILE = \"gemfile\"\n    GEMSPEC = \"gemspec\"\n    GIF = \"gif\"\n    GITATTRIBUTES = \"gitattributes\"\n    GITMODULES = \"gitmodules\"\n    GLEAM = \"gleam\"\n    GO = \"go\"\n    GPX = \"gpx\"\n    GRADLE = \"gradle\"\n    GROOVY = \"groovy\"\n    GZIP = \"gzip\"\n    H = \"h\"\n    H5 = \"h5\"\n    HANDLEBARS = \"handlebars\"\n    HASKELL = \"haskell\"\n    HCL = \"hcl\"\n    HEIF = \"heif\"\n    HFS = \"hfs\"\n    HLP = \"hlp\"\n    HPP = \"hpp\"\n    HTA = \"hta\"\n    HTACCESS = \"htaccess\"\n    HTML = \"html\"\n    HVE = \"hve\"\n    HWP = \"hwp\"\n    ICC = \"icc\"\n    ICNS = \"icns\"\n    ICO = \"ico\"\n    ICS = \"ics\"\n    IGNOREFILE = \"ignorefile\"\n    IMG = \"img\"\n    INI = \"ini\"\n    INTERNETSHORTCUT = \"internetshortcut\"\n    IOSAPP = \"iosapp\"\n    IPYNB = \"ipynb\"\n    ISO = \"iso\"\n    JAR = \"jar\"\n    JAVA = \"java\"\n    JAVABYTECODE = \"javabytecode\"\n    JAVASCRIPT = \"javascript\"\n    JINJA = \"jinja\"\n    JNG = \"jng\"\n    JNLP = \"jnlp\"\n    JP2 = \"jp2\"\n    JPEG = \"jpeg\"\n    JSON = \"json\"\n    JSONC = \"jsonc\"\n    JSONL = \"jsonl\"\n    JSX = \"jsx\"\n    JULIA = \"julia\"\n    JXL = \"jxl\"\n    KO = \"ko\"\n    KOTLIN = \"kotlin\"\n    KS = \"ks\"\n    LATEX = \"latex\"\n    LATEXAUX = \"latexaux\"\n    LESS = \"less\"\n    LHA = \"lha\"\n    LICENSE = \"license\"\n    LISP = \"lisp\"\n    LITCS = \"litcs\"\n    LNK = \"lnk\"\n    LOCK = \"lock\"\n    LRZ = \"lrz\"\n    LUA = \"lua\"\n    LZ = \"lz\"\n    LZ4 = \"lz4\"\n    LZX = \"lzx\"\n    M3U = \"m3u\"\n    M4 = \"m4\"\n    MACHO = \"macho\"\n    MAFF = \"maff\"\n    MAKEFILE = \"makefile\"\n    MARKDOWN = \"markdown\"\n    MATLAB = \"matlab\"\n    MHT = \"mht\"\n    MIDI = \"midi\"\n    MKV = \"mkv\"\n    MP2 = \"mp2\"\n    MP3 = \"mp3\"\n    MP4 = \"mp4\"\n    MPEGTS = \"mpegts\"\n    MSCOMPRESS = \"mscompress\"\n    MSI = \"msi\"\n    MSIX = \"msix\"\n    MST = \"mst\"\n    MUI = \"mui\"\n    MUM = \"mum\"\n    MUN = \"mun\"\n    NIM = \"nim\"\n    NPY = \"npy\"\n    NPZ = \"npz\"\n    NULL = \"null\"\n    NUPKG = \"nupkg\"\n    OBJECT = \"object\"\n    OBJECTIVEC = \"objectivec\"\n    OCAML = \"ocaml\"\n    OCX = \"ocx\"\n    ODEX = \"odex\"\n    ODIN = \"odin\"\n    ODP = \"odp\"\n    ODS = \"ods\"\n    ODT = \"odt\"\n    OGG = \"ogg\"\n    OLE = \"ole\"\n    ONE = \"one\"\n    ONNX = \"onnx\"\n    OOXML = \"ooxml\"\n    OTF = \"otf\"\n    OUTLOOK = \"outlook\"\n    PALMOS = \"palmos\"\n    PARQUET = \"parquet\"\n    PASCAL = \"pascal\"\n    PBM = \"pbm\"\n    PCAP = \"pcap\"\n    PDB = \"pdb\"\n    PDF = \"pdf\"\n    PEBIN = \"pebin\"\n    PEM = \"pem\"\n    PERL = \"perl\"\n    PGP = \"pgp\"\n    PHP = \"php\"\n    PICKLE = \"pickle\"\n    PNG = \"png\"\n    PO = \"po\"\n    POSTSCRIPT = \"postscript\"\n    POWERSHELL = \"powershell\"\n    PPT = \"ppt\"\n    PPTX = \"pptx\"\n    PRINTFOX = \"printfox\"\n    PROLOG = \"prolog\"\n    PROTEINDB = \"proteindb\"\n    PROTO = \"proto\"\n    PROTOBUF = \"protobuf\"\n    PSD = \"psd\"\n    PUB = \"pub\"\n    PYTHON = \"python\"\n    PYTHONBYTECODE = \"pythonbytecode\"\n    PYTHONPAR = \"pythonpar\"\n    PYTORCH = \"pytorch\"\n    QOI = \"qoi\"\n    QT = \"qt\"\n    R = \"r\"\n    RANDOMASCII = \"randomascii\"\n    RANDOMBYTES = \"randombytes\"\n    RANDOMTXT = \"randomtxt\"\n    RAR = \"rar\"\n    RDF = \"rdf\"\n    RDP = \"rdp\"\n    RIFF = \"riff\"\n    RLIB = \"rlib\"\n    RLL = \"rll\"\n    RPM = \"rpm\"\n    RST = \"rst\"\n    RTF = \"rtf\"\n    RUBY = \"ruby\"\n    RUST = \"rust\"\n    RZIP = \"rzip\"\n    SCALA = \"scala\"\n    SCHEME = \"scheme\"\n    SCR = \"scr\"\n    SCRIPTWSF = \"scriptwsf\"\n    SCSS = \"scss\"\n    SEVENZIP = \"sevenzip\"\n    SGML = \"sgml\"\n    SH3D = \"sh3d\"\n    SHELL = \"shell\"\n    SMALI = \"smali\"\n    SNAP = \"snap\"\n    SO = \"so\"\n    SOLIDITY = \"solidity\"\n    SQL = \"sql\"\n    SQLITE = \"sqlite\"\n    SQUASHFS = \"squashfs\"\n    SRT = \"srt\"\n    STLBINARY = \"stlbinary\"\n    STLTEXT = \"stltext\"\n    SUM = \"sum\"\n    SVD = \"svd\"\n    SVG = \"svg\"\n    SWF = \"swf\"\n    SWIFT = \"swift\"\n    SYMLINK = \"symlink\"\n    SYMLINKTEXT = \"symlinktext\"\n    SYS = \"sys\"\n    TAR = \"tar\"\n    TCL = \"tcl\"\n    TEXTPROTO = \"textproto\"\n    TGA = \"tga\"\n    THUMBSDB = \"thumbsdb\"\n    TIFF = \"tiff\"\n    TMDX = \"tmdx\"\n    TOML = \"toml\"\n    TORRENT = \"torrent\"\n    TROFF = \"troff\"\n    TSV = \"tsv\"\n    TSX = \"tsx\"\n    TTF = \"ttf\"\n    TWIG = \"twig\"\n    TXT = \"txt\"\n    TXTASCII = \"txtascii\"\n    TXTUTF16 = \"txtutf16\"\n    TXTUTF8 = \"txtutf8\"\n    TYPESCRIPT = \"typescript\"\n    UDF = \"udf\"\n    UNDEFINED = \"undefined\"\n    UNIXCOMPRESS = \"unixcompress\"\n    UNKNOWN = \"unknown\"\n    VBA = \"vba\"\n    VBE = \"vbe\"\n    VCARD = \"vcard\"\n    VCS = \"vcs\"\n    VCXPROJ = \"vcxproj\"\n    VERILOG = \"verilog\"\n    VHD = \"vhd\"\n    VHDL = \"vhdl\"\n    VISIO = \"visio\"\n    VTT = \"vtt\"\n    VUE = \"vue\"\n    WAD = \"wad\"\n    WASM = \"wasm\"\n    WAV = \"wav\"\n    WEBM = \"webm\"\n    WEBP = \"webp\"\n    WEBTEMPLATE = \"webtemplate\"\n    WIM = \"wim\"\n    WINREGISTRY = \"winregistry\"\n    WMA = \"wma\"\n    WMF = \"wmf\"\n    WMV = \"wmv\"\n    WOFF = \"woff\"\n    WOFF2 = \"woff2\"\n    XAR = \"xar\"\n    XCF = \"xcf\"\n    XLS = \"xls\"\n    XLSB = \"xlsb\"\n    XLSX = \"xlsx\"\n    XML = \"xml\"\n    XPI = \"xpi\"\n    XSD = \"xsd\"\n    XZ = \"xz\"\n    YAML = \"yaml\"\n    YARA = \"yara\"\n    ZIG = \"zig\"\n    ZIP = \"zip\"\n    ZLIBSTREAM = \"zlibstream\"\n    ZST = \"zst\"\n\n    def __repr__(self) -> str:\n        return str(self)\n"
  },
  {
    "path": "python/src/magika/types/magika_error.py",
    "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100, D101\n\n\nclass MagikaError(Exception):\n    pass\n"
  },
  {
    "path": "python/src/magika/types/magika_prediction.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Module defining the MagikaPrediction dataclass.\"\"\"\n\nfrom __future__ import annotations\n\nfrom dataclasses import dataclass\n\nfrom magika.types.content_type_info import ContentTypeInfo\nfrom magika.types.overwrite_reason import OverwriteReason\n\n\n@dataclass(frozen=True)\nclass MagikaPrediction:\n    \"\"\"Encodes the detailed result of Magika's content type inference.\n\n    This dataclass holds both the raw Deep Learning model's prediction and the\n    final, potentially modified, output prediction.\n\n    Attributes:\n        dl: The raw prediction from the Deep Learning (DL) model.\n        output: The final, consolidated content type prediction, which may\n            differ from `dl` due to heuristics or post-processing.\n        score: The confidence score (0.0 to 1.0) associated with the final\n            prediction.\n        overwrite_reason: The reason the `output` might have overridden the\n            raw `dl` prediction.\n    \"\"\"\n\n    dl: ContentTypeInfo\n    output: ContentTypeInfo\n    score: float\n    overwrite_reason: OverwriteReason\n"
  },
  {
    "path": "python/src/magika/types/magika_result.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Defines MagikaResult, which encodes the result of a scan.\"\"\"\n\nimport dataclasses\nfrom pathlib import Path\nfrom typing import Any, Dict, Optional\n\nfrom magika.types.content_type_info import ContentTypeInfo\nfrom magika.types.magika_prediction import MagikaPrediction\nfrom magika.types.status import Status\n\n\nclass MagikaResult:\n    \"\"\"Encodes the result of a content type inference scan.\"\"\"\n\n    def __init__(\n        self,\n        *,\n        path: Path,\n        status: Status = Status.OK,\n        prediction: Optional[MagikaPrediction] = None,\n    ):\n        \"\"\"Initializes a new MagikaResult object.\n\n        Args:\n            path: The file path that was analyzed.\n            status: The status of the analysis operation (e.g., OK, FILE_NOT_FOUND_ERROR).\n            prediction: The prediction details if the analysis was successful; None otherwise.\n        \"\"\"\n        self._path = path\n        self._status = status\n        self._prediction = prediction\n\n    def __post_init__(self) -> None:\n        assert self._path is not None\n        if self._status == Status.OK:\n            if self._prediction is None:\n                raise ValueError(\"prediction must be set when status == OK\")\n        else:\n            if self._prediction is not None:\n                raise ValueError(\"prediction cannot be set when status != OK\")\n\n    @property\n    def path(self) -> Path:\n        \"\"\"The file path that was analyzed.\n\n        Returns:\n            The Path object representing the analyzed file.\n        \"\"\"\n        return self._path\n\n    @property\n    def ok(self) -> bool:\n        \"\"\"Checks if the analysis was successful.\n\n        Returns:\n            True if status is OK, False otherwise.\n        \"\"\"\n        return self._status == Status.OK\n\n    @property\n    def status(self) -> Status:\n        \"\"\"The operational status of the analysis.\n\n        Returns:\n            The Status enumeration value (e.g., Status.OK, Status.FILE_NOT_FOUND_ERROR).\n        \"\"\"\n        return self._status\n\n    @property\n    def prediction(self) -> MagikaPrediction:\n        \"\"\"The detailed content type prediction result.\n\n        Returns:\n            The prediction object containing content type information.\n\n        Raises:\n            ValueError: If the status is not OK.\n        \"\"\"\n        if self.ok:\n            assert self._prediction is not None\n            return self._prediction\n        raise ValueError(\"prediction is not set when status != OK\")\n\n    # In the vast majority of cases, Magika would return with status == OK ( for\n    # `identify_bytes()` there is not even a code path that would return an\n    # error). To optimize for such frequent scenario, we add the following\n    # properties to forward the underlying value. Clients that want to make use\n    # of the full power of the absl-like StatusOr pattern can still do so, but\n    # we do not force all clients, regardless of their complexity or\n    # criticality, to use the more verbose `mr.prediction.output`.\n    @property\n    def dl(self) -> ContentTypeInfo:\n        \"\"\"The predicted content type from the Deep Learning (dl) model.\n\n        Note: This is a convenience property, equivalent to `self.prediction.dl`.\n\n        Returns:\n            The ContentTypeInfo for the dl model's prediction.\n        \"\"\"\n        return self.prediction.dl\n\n    @property\n    def output(self) -> ContentTypeInfo:\n        \"\"\"The final, consolidated output content type prediction.\n\n        Note: This is a convenience property, equivalent to `self.prediction.output`.\n\n        Returns:\n            The ContentTypeInfo for the final prediction.\n        \"\"\"\n        return self.prediction.output\n\n    @property\n    def score(self) -> float:\n        \"\"\"The confidence score of the final prediction.\n\n        Note: This is a convenience property, equivalent to `self.prediction.score`.\n\n        Returns:\n            The confidence score as a float (0.0 to 1.0).\n        \"\"\"\n        return self.prediction.score\n\n    def asdict(self) -> Dict:\n        \"\"\"Serializes the MagikaResult object into a dictionary.\n\n        The dictionary includes the file path, status, and the full prediction\n        details if the status is OK.\n\n        Returns:\n            A dictionary representation of the result.\n        \"\"\"\n        out: Dict[str, Any] = {\n            \"path\": str(self.path),\n            \"status\": self.status,\n        }\n        if self.ok:\n            out[\"prediction\"] = dataclasses.asdict(self.prediction)\n        return out\n\n    def __repr__(self) -> str:\n        return str(self)\n\n    def __str__(self) -> str:\n        if self.ok:\n            return f\"MagikaResult(path={self.path}, status={self.status}, prediction={self.prediction})\"\n        else:\n            return f\"MagikaResult(path={self.path}, status={self.status})\"\n"
  },
  {
    "path": "python/src/magika/types/model.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100, D101, D102, D103, D107\n\n\nfrom dataclasses import dataclass\nfrom typing import Dict, List\n\nfrom magika.types.content_type_label import ContentTypeLabel\n\n\n@dataclass(frozen=True)\nclass ModelFeatures:\n    beg: List[int]\n    mid: List[int]\n    end: List[int]\n    # for ISO\n    offset_0x8000_0x8007: List[int]\n    offset_0x8800_0x8807: List[int]\n    offset_0x9000_0x9007: List[int]\n    # for UDF\n    offset_0x9800_0x9807: List[int]\n\n\n@dataclass(frozen=True)\nclass ModelOutput:\n    label: ContentTypeLabel\n    score: float\n\n\n@dataclass(frozen=True)\nclass ModelConfig:\n    beg_size: int\n    mid_size: int\n    end_size: int\n    use_inputs_at_offsets: bool\n    medium_confidence_threshold: float\n    min_file_size_for_dl: int\n    padding_token: int\n    block_size: int\n    target_labels_space: List[ContentTypeLabel]\n    thresholds: Dict[ContentTypeLabel, float]\n    overwrite_map: Dict[ContentTypeLabel, ContentTypeLabel]\n"
  },
  {
    "path": "python/src/magika/types/overwrite_reason.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100\n\n\nimport enum\n\nfrom magika.types.strenum import LowerCaseStrEnum\n\n\nclass OverwriteReason(LowerCaseStrEnum):\n    \"\"\"Enum to represent possible reasons for overriding the model's prediction.\n\n    Consult the documentation for additional context.\n    \"\"\"\n\n    NONE = enum.auto()\n    LOW_CONFIDENCE = enum.auto()\n    OVERWRITE_MAP = enum.auto()\n"
  },
  {
    "path": "python/src/magika/types/prediction_mode.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100\n\nfrom __future__ import annotations\n\nimport enum\nfrom typing import List\n\nfrom magika.types.strenum import LowerCaseStrEnum\n\n\nclass PredictionMode(LowerCaseStrEnum):\n    \"\"\"Enum to represent the possible prediction modes.\n\n    Consult the documentation for additional context.\n    \"\"\"\n\n    BEST_GUESS = enum.auto()\n    MEDIUM_CONFIDENCE = enum.auto()\n    HIGH_CONFIDENCE = enum.auto()\n\n    @staticmethod\n    def get_valid_prediction_modes() -> List[str]:\n        \"\"\"Get a list of valid prediction modes.\"\"\"\n        return [pm for pm in PredictionMode]\n"
  },
  {
    "path": "python/src/magika/types/seekable.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100, D101, D102, D103, D107\n\nimport io\nfrom typing import BinaryIO\n\n\nclass Seekable:\n    def __init__(self, stream: BinaryIO) -> None:\n        self._stream = stream\n        stream.seek(0, io.SEEK_END)\n        self._size = stream.tell()\n\n    @property\n    def size(self) -> int:\n        return self._size\n\n    def read_at(self, offset: int, size: int) -> bytes:\n        if size == 0:\n            return b\"\"\n\n        assert offset + size <= self.size\n        self._stream.seek(offset)\n        return self._stream.read(size)\n"
  },
  {
    "path": "python/src/magika/types/status.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100\n\nimport enum\n\nfrom magika.types.strenum import LowerCaseStrEnum\n\n\nclass Status(LowerCaseStrEnum):\n    \"\"\"Enum to represent the possible status outcomes of a content type inference result.\"\"\"\n\n    OK = enum.auto()\n\n    # Used when a file path does not exist.\n    FILE_NOT_FOUND_ERROR = enum.auto()\n\n    # Used when a file path exists, but there are permission issues, e.g., can't\n    # read file.\n    PERMISSION_ERROR = enum.auto()\n\n    # Represents a generic error-like unknown status.\n    UNKNOWN = enum.auto()\n"
  },
  {
    "path": "python/src/magika/types/strenum.py",
    "content": "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ruff: noqa: D100, D101, D102, D103, D107\n\n\"\"\"Backport of StrEnum.\n\nStrEnum was introduced in python 3.11, but we do not rely on it because we aim\nat supporting python since version 3.8.\n\nThe following code has been taken (and adapted) from:\nhttps://github.com/irgeek/StrEnum/blob/master/strenum/__init__.py#L21\n\"\"\"\n\nfrom __future__ import annotations\n\nimport enum\nfrom typing import Union\n\n\nclass StrEnum(str, enum.Enum):\n    \"\"\"StrEnum is a Python ``enum.Enum`` that inherits from ``str``.\n\n    The default ``auto()`` behavior uses the lower-case version of the name.\n    This is meant to reflect the behavior of `enum.StrEnum`, available from\n    Python 3.11.\n    \"\"\"\n\n    def __new__(cls, value: Union[str, StrEnum], *args, **kwargs):  # type: ignore[no-untyped-def]\n        if not isinstance(value, (str, enum.auto)):\n            raise TypeError(\n                f\"Values of StrEnums must be strings: {value!r} is a {type(value)}\"\n            )\n        return super().__new__(cls, value, *args, **kwargs)\n\n    def __str__(self) -> str:\n        return str(self.value)\n\n    def _generate_next_value_(name, *_):  # type: ignore[no-untyped-def,override]\n        return name\n\n\nclass LowerCaseStrEnum(StrEnum):\n    def _generate_next_value_(name, *_):  # type: ignore[no-untyped-def,override]\n        return name.lower()\n"
  },
  {
    "path": "python/tests/__init__.py",
    "content": ""
  },
  {
    "path": "python/tests/test_features_extraction_vs_reference.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom __future__ import annotations\n\nimport base64\nimport io\nimport json\nfrom dataclasses import asdict, dataclass\nfrom pathlib import Path\nfrom typing import List, Tuple\n\nimport click\nimport dacite\nfrom tqdm import tqdm\n\nfrom magika import Magika\nfrom magika.types import ModelFeatures, Seekable\n\ntry:\n    from tests import utils as test_utils\nexcept ImportError:\n    # Hack to support both `uv run pytest tests/` and `uv run ./tests/test_...\n    # `\n    import sys\n\n    sys.path.append(str(Path(__file__).parent.parent))\n    from tests import utils as test_utils\n\n\n@click.group()\ndef cli():\n    pass\n\n\n@cli.command()\n@click.option(\"--debug/--no-debug\", is_flag=True, default=True)\ndef run_tests(debug: bool) -> None:\n    test_features_extraction_vs_reference(debug=debug)\n\n\n@cli.command()\n@click.option(\"--test-mode\", is_flag=True)\ndef generate_tests(test_mode: bool) -> None:\n    _generate_reference_features_extraction(test_mode=test_mode)\n\n\ndef test_features_extraction_vs_reference(debug: bool = False) -> None:\n    examples = _get_examples_from_reference()\n    if debug:\n        print(f\"Loaded {len(examples)} tests cases\")\n\n    for example in tqdm(examples, disable=not debug):\n        example_content = base64.b64decode(example.content_base64)\n\n        features = Magika._extract_features_from_seekable(\n            Seekable(io.BytesIO(example_content)),\n            beg_size=example.args.beg_size,\n            mid_size=example.args.mid_size,\n            end_size=example.args.end_size,\n            padding_token=example.args.padding_token,\n            block_size=example.args.block_size,\n            use_inputs_at_offsets=example.args.use_inputs_at_offsets,\n        )\n        _check_features_vs_reference_example_features(\n            features, example.features, debug=debug\n        )\n\n\ndef test_reference_generation() -> None:\n    _generate_reference_features_extraction(test_mode=True)\n\n\ndef _generate_reference_features_extraction(test_mode: bool) -> None:\n    print(\"Generating reference features extraction tests cases...\")\n    tests_cases = _generate_reference_features_extraction_tests_cases()\n    print(f\"Generated {len(tests_cases)} tests cases\")\n    _dump_reference_features_extraction_examples(tests_cases, test_mode=test_mode)\n\n\ndef _dump_reference_features_extraction_examples(\n    examples: List[FeaturesExtractionExample],\n    test_mode: bool,\n) -> None:\n    reference_features_extraction_examples_path = (\n        test_utils.get_reference_features_extraction_examples_path()\n    )\n\n    if test_mode:\n        print('WARNING: running in \"test_mode\", not writing examples to file')\n    else:\n        reference_features_extraction_examples_path.parent.mkdir(\n            parents=True, exist_ok=True\n        )\n        reference_features_extraction_examples_path.write_bytes(\n            test_utils.gzip_compress(\n                json.dumps([asdict(example) for example in examples]).encode(\"ascii\")\n            )\n        )\n        print(f\"Wrote tests cases to {reference_features_extraction_examples_path}\")\n\n\ndef _generate_reference_features_extraction_tests_cases() -> List[\n    FeaturesExtractionExample\n]:\n    tests_cases_inputs: List[\n        Tuple[FeaturesExtractionExampleArgs, FeaturesExtractionExampleMetadata, bytes]\n    ] = _generate_reference_features_extraction_tests_cases_inputs()\n\n    tests_cases = []\n    for test_args, test_metadata, test_content in tests_cases_inputs:\n        features = Magika._extract_features_from_seekable(\n            Seekable(io.BytesIO(test_content)),\n            test_args.beg_size,\n            test_args.mid_size,\n            test_args.end_size,\n            test_args.padding_token,\n            test_args.block_size,\n            test_args.use_inputs_at_offsets,\n        )\n\n        example = FeaturesExtractionExample(\n            args=test_args,\n            metadata=test_metadata,\n            content_base64=base64.b64encode(test_content).decode(\"ascii\"),\n            features=features,\n        )\n\n        tests_cases.append(example)\n\n    return tests_cases\n\n\ndef _generate_reference_features_extraction_tests_cases_inputs() -> List[\n    Tuple[FeaturesExtractionExampleArgs, FeaturesExtractionExampleMetadata, bytes]\n]:\n    beg_size = 128\n    mid_size = 0\n    end_size = 64\n    block_size = 512\n    padding_token = 256\n    use_inputs_at_offsets = False\n\n    assert mid_size == 0\n    assert use_inputs_at_offsets is False\n    assert beg_size < block_size\n    assert mid_size < block_size\n    assert end_size < block_size\n\n    ws_num_options = [\n        0,\n        1,\n        10,\n        beg_size - 1,\n        beg_size,\n        beg_size + 1,\n        end_size - 1,\n        end_size,\n        end_size + 1,\n        beg_size + end_size - 1,\n        beg_size + end_size,\n        beg_size + end_size + 1,\n        block_size - 1,\n        block_size,\n        block_size + 1,\n        2 * block_size - 1,\n        2 * block_size,\n        2 * block_size + 1,\n        4 * block_size - 1,\n        4 * block_size,\n        4 * block_size + 1,\n    ]\n\n    content_size_options = list(ws_num_options)\n\n    tests_cases_inputs = []\n    for core_content_size in content_size_options:\n        for left_ws_num in ws_num_options:\n            for right_ws_num in ws_num_options:\n                test_args = FeaturesExtractionExampleArgs(\n                    beg_size=beg_size,\n                    mid_size=mid_size,\n                    end_size=end_size,\n                    block_size=block_size,\n                    padding_token=padding_token,\n                    use_inputs_at_offsets=use_inputs_at_offsets,\n                )\n                test_metadata = FeaturesExtractionExampleMetadata(\n                    core_content_size=core_content_size,\n                    left_ws_num=left_ws_num,\n                    right_ws_num=right_ws_num,\n                )\n\n                content = _generate_content_from_metadata(test_metadata)\n                tests_cases_inputs.append((test_args, test_metadata, content))\n\n    return tests_cases_inputs\n\n\ndef _generate_content_from_metadata(\n    test_info: FeaturesExtractionExampleMetadata,\n) -> bytes:\n    \"\"\"Generate content with a given \"core size\", with n left and right\n    whitespaces, and the core content. with_ws_near_beg and with_ws_near_end\n    specify if we need to put spaces near the beg/end, e.g., \"A AAA\". This is\n    useful to test that we don't strip whitespaces that we are not supposed to\n    strip.\"\"\"\n\n    content = bytearray(\n        test_utils.generate_pattern(test_info.core_content_size, only_printable=True)\n    )\n\n    if test_info.core_content_size >= 5:\n        # inject characters that other implementations may mistakenly strip\n        content[0] = ord(\"\\x00\")\n        content[1] = ord(\" \")\n        content[-2] = ord(\" \")\n        content[-1] = ord(\"\\x00\")\n\n    return (\n        test_utils.generate_whitespaces(test_info.left_ws_num)\n        + bytes(content)\n        + test_utils.generate_whitespaces(test_info.right_ws_num)\n    )\n\n\ndef _get_examples_from_reference() -> List[FeaturesExtractionExample]:\n    ref_features_extraction_examples_path = (\n        test_utils.get_reference_features_extraction_examples_path()\n    )\n\n    return [\n        dacite.from_dict(FeaturesExtractionExample, example)\n        for example in json.loads(\n            test_utils.gzip_decompress(\n                ref_features_extraction_examples_path.read_bytes()\n            )\n        )\n    ]\n\n\ndef _check_features_vs_reference_example_features(\n    features: ModelFeatures, example_features: ModelFeatures, debug: bool = False\n) -> None:\n    with_error = False\n    if features.beg != example_features.beg:\n        with_error = True\n        if debug:\n            print(\"beg does not match\")\n    if features.mid != example_features.mid:\n        with_error = True\n        if debug:\n            print(\"mid does not match\")\n    if features.end != example_features.end:\n        with_error = True\n        if debug:\n            print(\"end does not match\")\n    try:\n        assert features == example_features\n    except AssertionError:\n        with_error = True\n        if debug:\n            print(\"other fields do not match\")\n\n    if with_error:\n        raise Exception\n\n\n@dataclass\nclass FeaturesExtractionExample:\n    \"\"\"Data model for features_extraction_examples.json.gz.\"\"\"\n\n    args: FeaturesExtractionExampleArgs\n    metadata: FeaturesExtractionExampleMetadata\n    content_base64: str\n    features: ModelFeatures\n\n\n@dataclass\nclass FeaturesExtractionExampleArgs:\n    beg_size: int\n    mid_size: int\n    end_size: int\n    block_size: int\n    padding_token: int\n    use_inputs_at_offsets: bool\n\n\n@dataclass\nclass FeaturesExtractionExampleMetadata:\n    core_content_size: int\n    left_ws_num: int\n    right_ws_num: int\n\n\nif __name__ == \"__main__\":\n    cli()\n"
  },
  {
    "path": "python/tests/test_inference_vs_reference.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom __future__ import annotations\n\nimport base64\nimport enum\nimport json\nimport random\nimport tempfile\nfrom dataclasses import asdict, dataclass\nfrom pathlib import Path\nfrom typing import Dict, Generator, List, Optional, Set, Tuple\n\nimport click\nimport dacite\nimport pytest\nfrom tqdm import tqdm\n\nfrom magika import ContentTypeLabel, Magika, PredictionMode\nfrom magika.types import MagikaResult, OverwriteReason\nfrom magika.types.status import Status\n\ntry:\n    from tests import utils as test_utils\nexcept ImportError:\n    # Hack to support both `uv run pytest tests/` and `uv run ./tests/test_...\n    # `\n    import sys\n\n    sys.path.append(str(Path(__file__).parent.parent))\n    from tests import utils as test_utils\n\n\n@click.group()\ndef cli():\n    pass\n\n\n@cli.command()\n@click.option(\"--debug/--no-debug\", is_flag=True, default=True)\ndef run_tests(debug: bool) -> None:\n    test_inference_vs_reference(debug=debug)\n\n\n@cli.command()\n@click.option(\"--test-mode\", is_flag=True)\ndef generate_tests(test_mode: bool) -> None:\n    _generate_reference_for_inference(test_mode=test_mode)\n\n\ndef test_inference_vs_reference(debug: bool = False) -> None:\n    repo_root_dir = test_utils.get_repo_root_dir()\n\n    magika_by_prediction_mode: Dict[PredictionMode, Magika] = {}\n    for prediction_mode in [\n        PredictionMode.HIGH_CONFIDENCE,\n        PredictionMode.MEDIUM_CONFIDENCE,\n        PredictionMode.BEST_GUESS,\n    ]:\n        magika_by_prediction_mode[prediction_mode] = Magika(\n            prediction_mode=prediction_mode\n        )\n\n    model_name = magika_by_prediction_mode[\n        PredictionMode.HIGH_CONFIDENCE\n    ].get_model_name()\n\n    examples_by_path = _get_examples_by_path(model_name)\n    if debug:\n        print(f\"Loaded {len(examples_by_path)} examples by path\")\n    for example in tqdm(examples_by_path, disable=not debug):\n        m = magika_by_prediction_mode[example.prediction_mode]\n        abs_path = repo_root_dir / example.path\n        result = m.identify_path(abs_path)\n        _check_result_vs_reference_example(\n            result, abs_path, example.status, example.prediction\n        )\n\n        result = m.identify_bytes(abs_path.read_bytes())\n        _check_result_vs_reference_example(\n            result, Path(\"-\"), example.status, example.prediction\n        )\n\n        with open(abs_path, \"rb\") as f:\n            result = m.identify_stream(f)\n            _check_result_vs_reference_example(\n                result, Path(\"-\"), example.status, example.prediction\n            )\n\n    examples_by_content = _get_examples_by_content(model_name)\n    if debug:\n        print(f\"Loaded {len(examples_by_content)} examples by content\")\n    for example in tqdm(examples_by_content, disable=not debug):\n        m = magika_by_prediction_mode[example.prediction_mode]\n        example_content = base64.b64decode(example.content_base64)\n        result = m.identify_bytes(example_content)\n        _check_result_vs_reference_example(\n            result, Path(\"-\"), example.status, example.prediction\n        )\n\n        with tempfile.TemporaryDirectory() as td:\n            tf_path = Path(td) / \"file.bin\"\n            tf_path.write_bytes(example_content)\n\n            result = m.identify_path(tf_path)\n            _check_result_vs_reference_example(\n                result, tf_path, example.status, example.prediction\n            )\n\n            with open(tf_path, \"rb\") as f:\n                result = m.identify_stream(f)\n                _check_result_vs_reference_example(\n                    result, Path(\"-\"), example.status, example.prediction\n                )\n\n\ndef test_reference_generation() -> None:\n    # This is useful to exercise the various paths to make sure the reference\n    # generation stays up to date.\n    _generate_reference_for_inference(test_mode=True)\n\n\ndef _get_examples_by_path(model_name: str) -> List[ExampleByPath]:\n    reference_for_inference_examples_by_path = (\n        test_utils.get_reference_for_inference_examples_by_path_path(model_name)\n    )\n    return [\n        dacite.from_dict(\n            ExampleByPath,\n            entry,\n            config=dacite.Config(\n                cast=[ContentTypeLabel, OverwriteReason, PredictionMode, Status]\n            ),\n        )\n        for entry in json.loads(\n            test_utils.gzip_decompress(\n                reference_for_inference_examples_by_path.read_bytes()\n            )\n        )\n    ]\n\n\ndef _get_examples_by_content(model_name: str) -> List[ExampleByContent]:\n    reference_for_inference_examples_by_content = (\n        test_utils.get_reference_for_inference_examples_by_content_path(model_name)\n    )\n    return [\n        dacite.from_dict(\n            ExampleByContent,\n            entry,\n            config=dacite.Config(\n                cast=[ContentTypeLabel, OverwriteReason, PredictionMode, Status]\n            ),\n        )\n        for entry in json.loads(\n            test_utils.gzip_decompress(\n                reference_for_inference_examples_by_content.read_bytes()\n            )\n        )\n    ]\n\n\ndef _generate_reference_for_inference(test_mode: bool) -> None:\n    model_name = Magika._get_default_model_name()\n    examples_by_path = _generate_examples_by_path(model_name)\n    _dump_examples_by_path(model_name, examples_by_path, test_mode=test_mode)\n    examples_by_content = _generate_examples_by_content(model_name, test_mode=test_mode)\n    _dump_examples_by_content(model_name, examples_by_content, test_mode=test_mode)\n\n\ndef _generate_examples_by_path(\n    model_name: str,\n) -> List[ExampleByPath]:\n    print(f'Generating examples by path for model \"{model_name}\"...')\n\n    repo_root_dir = test_utils.get_repo_root_dir()\n    tests_paths = test_utils.get_basic_test_files_paths()\n    examples_by_path = []\n\n    for prediction_mode in [\n        PredictionMode.HIGH_CONFIDENCE,\n        PredictionMode.MEDIUM_CONFIDENCE,\n        PredictionMode.BEST_GUESS,\n    ]:\n        m = Magika(prediction_mode=prediction_mode)\n        assert m.get_model_name() == model_name\n\n        for test_path in tqdm(tests_paths):\n            result = m.identify_path(test_path)\n            if result.ok:\n                example = ExampleByPath(\n                    prediction_mode=prediction_mode,\n                    path=str(test_path.resolve().relative_to(repo_root_dir)),\n                    status=result.status,\n                    prediction=Prediction(\n                        dl=result.prediction.dl.label,\n                        output=result.prediction.output.label,\n                        score=result.prediction.score,\n                        overwrite_reason=result.prediction.overwrite_reason,\n                    ),\n                )\n            else:\n                example = ExampleByPath(\n                    prediction_mode=prediction_mode,\n                    path=str(test_path),\n                    status=result.status,\n                    prediction=None,\n                )\n            examples_by_path.append(example)\n\n    return examples_by_path\n\n\ndef _generate_examples_by_content(\n    model_name: str, test_mode: bool\n) -> List[ExampleByContent]:\n    random.seed(42)\n\n    print(f'Generating examples by content for model \"{model_name}\"...')\n\n    # First we generate corner cases examples by content, without caring about\n    # the prediction mode. In fact, at the example generation phase, we only\n    # care about the model prediction, which is not affected by the prediction\n    # mode. Then, we generate the reference by looping over possible prediction\n    # modes and all the cornern case examples.\n\n    magika = Magika()\n    assert magika.get_model_name() == model_name\n\n    content_list = []\n    content_list.append(b\"\")\n    for size in [\n        magika._model_config.min_file_size_for_dl - 1,\n        magika._model_config.min_file_size_for_dl,\n        magika._model_config.min_file_size_for_dl + 1,\n        magika._model_config.beg_size - 1,\n        magika._model_config.beg_size,\n        magika._model_config.beg_size + 1,\n        magika._model_config.end_size - 1,\n        magika._model_config.end_size,\n        magika._model_config.end_size + 1,\n        magika._model_config.beg_size + magika._model_config.end_size - 1,\n        magika._model_config.beg_size + magika._model_config.end_size,\n        magika._model_config.beg_size + magika._model_config.end_size + 1,\n        magika._model_config.block_size - 1,\n        magika._model_config.block_size,\n        magika._model_config.block_size + 1,\n    ]:\n        content_list.append(test_utils.generate_pattern(size, only_printable=True))\n        content_list.append(test_utils.generate_pattern(size, only_printable=False))\n\n    # We now generate additional examples to check for additional corner cases,\n    # related to prediction mode, thresholds, and overwrite map. We use a\n    # fuzzing-like approach to generate weird samples at random, we then check\n    # each of them to fill what we need for the test suite.\n    collector = CornerCaseCollector(magika)\n    generator = collector.get_corner_case_candidates_generator()\n    for candidate_idx, (source_info, content) in enumerate(generator):\n        is_useful, result, cc_info = collector.inspect_content(content)\n        if is_useful:\n            print(\n                source_info,\n                result.dl.label,\n                result.score,\n                result.output.label,\n                cc_info,\n                collector.get_missing_examples_num(),\n            )\n\n            content_list.append(content)\n\n            if collector.is_complete():\n                break\n\n        if test_mode:\n            if candidate_idx >= 100:\n                # In \"test_mode\", we exit after evaluating 100 samples, even if\n                # we are not done\n                break\n\n    if not collector.is_complete():\n        if test_mode:\n            print(\n                'WARNING: running in \"test_mode\", exiting corner cases generation early'\n            )\n        else:\n            print(\n                f\"ERROR: Missing {collector.get_missing_examples_num()} corner cases:\"\n            )\n            for corner_case_info in collector._missing_corner_cases:\n                print(f\"\\t{corner_case_info}\")\n            sys.exit(1)\n\n    examples_by_content = []\n    for prediction_mode in [\n        PredictionMode.HIGH_CONFIDENCE,\n        PredictionMode.MEDIUM_CONFIDENCE,\n        PredictionMode.BEST_GUESS,\n    ]:\n        magika = Magika(prediction_mode=prediction_mode)\n        for content in content_list:\n            result = magika.identify_bytes(content)\n            if result.ok:\n                example = ExampleByContent(\n                    prediction_mode=prediction_mode,\n                    content_base64=base64.b64encode(content).decode(\"ascii\"),\n                    status=result.status,\n                    prediction=Prediction(\n                        dl=result.prediction.dl.label,\n                        output=result.prediction.output.label,\n                        score=result.prediction.score,\n                        overwrite_reason=result.prediction.overwrite_reason,\n                    ),\n                )\n            else:\n                example = ExampleByContent(\n                    prediction_mode=prediction_mode,\n                    content_base64=base64.b64encode(content).decode(\"ascii\"),\n                    status=result.status,\n                    prediction=None,\n                )\n            examples_by_content.append(example)\n\n    return examples_by_content\n\n\ndef _dump_examples_by_path(\n    model_name: str,\n    examples_by_path: List[ExampleByPath],\n    test_mode: bool,\n) -> None:\n    examples_by_path_path = (\n        test_utils.get_reference_for_inference_examples_by_path_path(model_name)\n    )\n\n    if test_mode:\n        print(\n            f'WARNING: running in \"test_mode\", not writing examples by path to {examples_by_path_path}'\n        )\n    else:\n        examples_by_path_path.parent.mkdir(parents=True, exist_ok=True)\n        examples_by_path_path.write_bytes(\n            test_utils.gzip_compress(\n                json.dumps(\n                    [asdict(example) for example in examples_by_path],\n                    separators=(\",\", \":\"),\n                ).encode(\"ascii\")\n            )\n        )\n        print(\n            f\"Wrote {len(examples_by_path)} examples by path to {examples_by_path_path}\"\n        )\n\n\ndef _dump_examples_by_content(\n    model_name: str,\n    examples_by_content: List[ExampleByContent],\n    test_mode: bool,\n) -> None:\n    examples_by_content_path = (\n        test_utils.get_reference_for_inference_examples_by_content_path(model_name)\n    )\n\n    if test_mode:\n        print(\n            f'WARNING: running in \"test_mode\", not writing examples by content to {examples_by_content_path}'\n        )\n    else:\n        examples_by_content_path.parent.mkdir(parents=True, exist_ok=True)\n\n        examples_by_content_path.write_bytes(\n            test_utils.gzip_compress(\n                json.dumps(\n                    [asdict(example) for example in examples_by_content],\n                    separators=(\",\", \":\"),\n                ).encode(\"ascii\"),\n            )\n        )\n        print(\n            f\"Wrote {len(examples_by_content)} examples by content to {examples_by_content_path}\"\n        )\n\n\n@dataclass(frozen=True)\nclass CornerCaseInfo:\n    label_category: LabelCategory\n    with_threshold: bool\n    with_overwrite: bool\n    score_range: ScoreRange\n\n    def __repr__(self) -> str:\n        return (\n            f\"{self.__class__.__name__}(\"\n            f\"{self.label_category},\"\n            f\"{'TH' if self.with_threshold else 'NO_TH'},\"\n            f\"{'OW' if self.with_overwrite else 'NO_OW'},\"\n            f\"{self.score_range})\"\n        )\n\n\nclass LabelCategory(enum.Enum):\n    GENERIC_TEXT = enum.auto()\n    GENERIC_BINARY = enum.auto()\n    NON_GENERIC_TEXT = enum.auto()\n    NON_GENERIC_BINARY = enum.auto()\n\n\nclass ScoreRange(enum.Enum):\n    LT_050 = enum.auto()\n    GE_050 = enum.auto()\n    GE_050_LT_T = enum.auto()\n    GE_T = enum.auto()\n\n\nclass CornerCaseCollector:\n    def __init__(self, magika: Magika):\n        self._magika = magika\n        self._missing_corner_cases: Set[CornerCaseInfo] = set()\n        # fmt: off\n        self._missing_corner_cases.update({\n            # NON_GENERIC_TEXT\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, False, False, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, False, False, ScoreRange.GE_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, True, False, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, True, False, ScoreRange.GE_050_LT_T),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, True, False, ScoreRange.GE_T),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, False, True, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_TEXT, False, True, ScoreRange.GE_050),\n            # NON_GENERIC_BINARY\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, False, False, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, False, False, ScoreRange.GE_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, True, False, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, True, False, ScoreRange.GE_050_LT_T),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, True, False, ScoreRange.GE_T),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, False, True, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.NON_GENERIC_BINARY, False, True, ScoreRange.GE_050),\n        })\n        self._missing_corner_cases.update({\n            CornerCaseInfo(LabelCategory.GENERIC_TEXT, False, False, ScoreRange.LT_050),\n            CornerCaseInfo(LabelCategory.GENERIC_TEXT, False, False, ScoreRange.GE_050),\n            # No GENERIC_BINARY (aka UNKNOWN) because the model would never output that\n        })\n        # fmt: on\n\n    def inspect_content(\n        self, content: bytes\n    ) -> Tuple[bool, MagikaResult, CornerCaseInfo]:\n        res = self._magika.identify_bytes(content)\n        cce = self._get_cornern_case_example(res.dl.label, res.score)\n        if cce in self._missing_corner_cases:\n            self._missing_corner_cases.remove(cce)\n            return True, res, cce\n        return False, res, cce\n\n    def is_complete(self) -> bool:\n        return self.get_missing_examples_num() == 0\n\n    def get_missing_examples(self) -> Set[CornerCaseInfo]:\n        return self._missing_corner_cases\n\n    def get_missing_examples_num(self) -> int:\n        return len(self._missing_corner_cases)\n\n    def _get_cornern_case_example(\n        self, dl_label: ContentTypeLabel, score: float\n    ) -> CornerCaseInfo:\n        return CornerCaseInfo(\n            label_category=self._get_label_category(dl_label),\n            with_threshold=self._has_threshold(dl_label),\n            with_overwrite=self._has_overwrite(dl_label),\n            score_range=self._get_score_range(dl_label, score),\n        )\n\n    def _get_label_category(self, dl_label: ContentTypeLabel) -> LabelCategory:\n        m = {\n            # is_generic, is_text\n            (True, True): LabelCategory.GENERIC_TEXT,\n            (True, False): LabelCategory.GENERIC_BINARY,\n            (False, True): LabelCategory.NON_GENERIC_TEXT,\n            (False, False): LabelCategory.NON_GENERIC_BINARY,\n        }\n        return m[\n            self._is_generic(dl_label),\n            self._is_text(dl_label),\n        ]\n\n    def _is_generic(self, dl_label: ContentTypeLabel) -> bool:\n        return dl_label in [ContentTypeLabel.TXT, ContentTypeLabel.UNKNOWN]\n\n    def _is_text(self, dl_label: ContentTypeLabel) -> bool:\n        return self._magika._cts_infos[dl_label].is_text\n\n    def _has_threshold(self, dl_label: ContentTypeLabel) -> bool:\n        return dl_label in self._magika._model_config.thresholds.keys()\n\n    def _get_threshold(self, dl_label: ContentTypeLabel) -> float:\n        return self._magika._model_config.thresholds[dl_label]\n\n    def _has_overwrite(self, dl_label: ContentTypeLabel) -> bool:\n        return dl_label in self._magika._model_config.overwrite_map.keys()\n\n    def _get_score_range(self, dl_label: ContentTypeLabel, score: float) -> ScoreRange:\n        if score < 0.50:\n            return ScoreRange.LT_050\n        else:\n            if self._has_threshold(dl_label):\n                if score < self._get_threshold(dl_label):\n                    return ScoreRange.GE_050_LT_T\n                else:\n                    return ScoreRange.GE_T\n            else:\n                return ScoreRange.GE_050\n\n    def get_corner_case_candidates_generator(\n        self,\n    ) -> Generator[Tuple[str, bytes], None, None]:\n        beg_size = self._magika._model_config.beg_size\n        end_size = self._magika._model_config.end_size\n\n        print(\"Using random bytes\")\n        for n in range(1_000):\n            if random.random() < 0.5:\n                yield (\n                    \"randomtxt\",\n                    test_utils.get_random_ascii_bytes(\n                        random.randrange(8, beg_size + end_size)\n                    ),\n                )\n            else:\n                yield (\n                    \"randombytes\",\n                    test_utils.get_random_bytes(\n                        random.randrange(8, beg_size + end_size)\n                    ),\n                )\n\n        base_examples = []\n        base_examples.append(\n            (\"randomtxt\", test_utils.get_random_ascii_bytes(beg_size + end_size))\n        )\n        base_examples.append(\n            (\"randombytes\", test_utils.get_random_bytes(beg_size + end_size))\n        )\n        for example_path in test_utils.get_basic_test_files_paths():\n            example_content = example_path.read_bytes()\n            if len(example_content) < beg_size + end_size:\n                base_content = example_content\n            else:\n                base_content = b\"\"\n                if beg_size > 0:\n                    example_content += base_content[:beg_size]\n                if end_size > 0:\n                    example_content += base_content[-end_size:]\n            base_example = (str(example_path), base_content)\n            yield base_example\n            base_examples.append(base_example)\n\n        for base_source, base_content in base_examples:\n            print(f\"Using {base_source} as base\")\n            for only_printable in [True, False]:\n                for n in range(\n                    0,\n                    min(\n                        beg_size,\n                        end_size,\n                        len(base_content),\n                    ),\n                ):\n                    patched_content = bytearray(base_content[:])\n                    patched_content[0:n] = test_utils.generate_pattern(\n                        n, only_printable=only_printable\n                    )\n                    yield (f\"base_{base_source}_beg_{n}\", bytes(patched_content))\n\n                    patched_content[len(base_content) - n : len(base_content)] = (\n                        test_utils.generate_pattern(n, only_printable=only_printable)\n                    )\n                    yield (f\"base_{base_source}_end_{n}\", bytes(patched_content))\n\n\ndef _check_result_vs_reference_example(\n    result: MagikaResult,\n    expected_path: Path,\n    expected_status: Status,\n    expected_prediction: Prediction,\n) -> None:\n    assert result.path == expected_path\n    assert result.status == expected_status\n    if result.ok:\n        assert result.prediction.dl.label == expected_prediction.dl\n        assert result.prediction.output.label == expected_prediction.output\n        assert result.prediction.score == pytest.approx(\n            expected_prediction.score, abs=1e-5\n        )\n        assert (\n            result.prediction.overwrite_reason == expected_prediction.overwrite_reason\n        )\n\n\n@dataclass\nclass ExampleByPath:\n    \"\"\"Data model for -inference_examples_by_path.json.gz.\"\"\"\n\n    prediction_mode: PredictionMode\n    path: str\n    status: Status\n    prediction: Optional[Prediction]\n\n\n@dataclass\nclass ExampleByContent:\n    \"\"\"Data model for -inference_examples_by_content.json.gz.\"\"\"\n\n    prediction_mode: PredictionMode\n    content_base64: str\n    status: Status\n    prediction: Optional[Prediction]\n\n\n@dataclass\nclass Prediction:\n    dl: ContentTypeLabel\n    output: ContentTypeLabel\n    score: float\n    overwrite_reason: OverwriteReason\n\n\nif __name__ == \"__main__\":\n    cli()\n"
  },
  {
    "path": "python/tests/test_magika_python_module.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport io\nimport signal\nimport tempfile\nfrom pathlib import Path\nfrom typing import Any, List, Optional\n\nimport pytest\n\nfrom magika import Magika, PredictionMode\nfrom magika.types import (\n    ContentTypeInfo,\n    ContentTypeLabel,\n    MagikaPrediction,\n    MagikaResult,\n    Status,\n)\nfrom magika.types.overwrite_reason import OverwriteReason\nfrom tests import utils\n\n\n@pytest.mark.smoketest\ndef test_magika_module_check_version() -> None:\n    import magika as magika_module\n\n    assert isinstance(magika_module.__version__, str)\n\n    m = Magika()\n    assert m.get_module_version() == magika_module.__version__\n\n    # Check that, when we don't specify `model_dir`, Magika uses the default\n    # model.\n    assert m.get_model_name() == m._get_default_model_name()\n\n\n@pytest.mark.smoketest\ndef test_magika_module_with_one_test_file() -> None:\n    test_path = utils.get_one_basic_test_file_path()\n\n    m = Magika()\n\n    _ = m.identify_path(test_path)\n    _ = m.identify_paths([test_path])\n    _ = m.identify_bytes(test_path.read_bytes())\n    with open(test_path, \"rb\") as f:\n        _ = m.identify_stream(f)\n\n\n@pytest.mark.smoketest\ndef test_magika_module_with_explicit_model_dir() -> None:\n    model_dir = utils.get_default_model_dir()\n    test_path = utils.get_one_basic_test_file_path()\n\n    m = Magika(model_dir=model_dir)\n\n    _ = m.identify_path(test_path)\n    _ = m.identify_paths([test_path])\n    _ = m.identify_bytes(test_path.read_bytes())\n    with open(test_path, \"rb\") as f:\n        _ = m.identify_stream(f)\n\n\ndef test_magika_module_with_basic_tests_by_paths() -> None:\n    tests_paths = utils.get_basic_test_files_paths()\n\n    m = Magika()\n    results = m.identify_paths(tests_paths)\n    check_results_vs_expected_results(tests_paths, results)\n\n\ndef test_magika_module_with_basic_tests_by_path() -> None:\n    tests_paths = utils.get_basic_test_files_paths()\n\n    m = Magika()\n\n    for test_path in tests_paths:\n        result = m.identify_path(test_path)\n        check_result_vs_expected_result(test_path, result)\n\n\ndef test_magika_module_with_basic_tests_by_bytes() -> None:\n    tests_paths = utils.get_basic_test_files_paths()\n\n    m = Magika()\n\n    for test_path in tests_paths:\n        content = test_path.read_bytes()\n        result = m.identify_bytes(content)\n        check_result_vs_expected_result(\n            test_path, result, expected_result_path=Path(\"-\")\n        )\n\n\ndef test_magika_module_with_basic_tests_by_stream() -> None:\n    tests_paths = utils.get_basic_test_files_paths()\n\n    m = Magika()\n\n    for test_path in tests_paths:\n        with open(test_path, \"rb\") as f:\n            result = m.identify_stream(f)\n        check_result_vs_expected_result(\n            test_path, result, expected_result_path=Path(\"-\")\n        )\n\n\ndef test_magika_module_with_all_models() -> None:\n    tests_paths = utils.get_basic_test_files_paths()\n\n    models_dir = utils.get_models_dir()\n    for model_dir in models_dir.iterdir():\n        m = Magika(model_dir=model_dir)\n        for test_path in tests_paths:\n            result = m.identify_path(test_path)\n            check_result_vs_expected_result(test_path, result)\n\n\ndef test_magika_module_with_previously_missdetected_samples() -> None:\n    model_dir = utils.get_default_model_dir()\n    tests_paths = utils.get_previously_missdetected_files_paths()\n\n    m = Magika(model_dir=model_dir)\n    results = m.identify_paths(tests_paths)\n    check_results_vs_expected_results(tests_paths, results)\n\n\ndef test_magika_module_with_empty_content() -> None:\n    m = Magika()\n\n    empty_content = b\"\"\n\n    res = m.identify_bytes(empty_content)\n    assert res.ok\n    assert res.path == Path(\"-\")\n    assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n    assert res.prediction.output.label == ContentTypeLabel.EMPTY\n    assert res.prediction.score == 1.0\n\n    with tempfile.TemporaryDirectory() as td:\n        tf_path = Path(td) / \"empty.dat\"\n        tf_path.write_bytes(empty_content)\n        res = m.identify_path(tf_path)\n        assert res.path == tf_path\n        assert res.ok\n        assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n        assert res.prediction.output.label == ContentTypeLabel.EMPTY\n        assert res.prediction.score == 1.0\n\n    res = m.identify_stream(io.BytesIO(b\"\"))\n    assert res.path == Path(\"-\")\n    assert res.ok\n    assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n    assert res.prediction.output.label == ContentTypeLabel.EMPTY\n    assert res.prediction.score == 1.0\n\n\ndef test_magika_module_with_short_content() -> None:\n    m = Magika()\n\n    text_content = b\"asd\"\n    binary_content = b\"\\x80\\x80\\x80\"\n\n    for content, expected_ct_label in zip(\n        [text_content, binary_content],\n        [ContentTypeLabel.TXT, ContentTypeLabel.UNKNOWN],\n    ):\n        with tempfile.TemporaryDirectory() as td:\n            # prediction via path\n            tf_path = Path(td) / \"file.txt\"\n            tf_path.write_bytes(content)\n            res = m.identify_path(tf_path)\n            assert res.path == tf_path\n            assert res.ok\n            assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n            assert res.prediction.output.label == expected_ct_label\n            assert res.prediction.score == 1.0\n\n            # prediction via bytes\n            res = m.identify_bytes(content)\n            assert res.path == Path(\"-\")\n            assert res.ok\n            assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n            assert res.prediction.output.label == expected_ct_label\n            assert res.prediction.score == 1.0\n\n            # prediction via stream\n            res = m.identify_stream(io.BytesIO(content))\n            assert res.path == Path(\"-\")\n            assert res.ok\n            assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n            assert res.prediction.output.label == expected_ct_label\n            assert res.prediction.score == 1.0\n\n\ndef test_magika_module_with_python_and_non_python_content() -> None:\n    python_content = (\n        b\"import flask\\nimport requests\\n\\ndef foo(a):\\n    print(f'Test {a}')\\n\"\n    )\n    non_python_content = b\"clearly not python\"\n\n    m = Magika()\n\n    res = m.identify_bytes(python_content)\n    assert res.ok\n    assert res.prediction.output.label == ContentTypeLabel.PYTHON\n\n    res = m.identify_bytes(non_python_content)\n    assert res.ok\n    assert res.prediction.output.label == ContentTypeLabel.TXT\n\n\ndef test_magika_module_identify_stream_does_not_alter_position() -> None:\n    m = Magika()\n\n    contents = [\n        b\"\",\n        b\"short\",\n        b\"A\" * 100,\n        b\"A\" * 1000,\n        b\"A\" * 10000,\n    ]\n    for content in contents:\n        stream = io.BytesIO(content)\n        # seek to a specific non-special position\n        pos = min(2, len(content))\n        stream.seek(pos)\n        res = m.identify_stream(stream)\n        assert res.ok\n        assert stream.tell() == pos\n\n\ndef test_magika_module_with_whitespaces() -> None:\n    m = Magika()\n\n    ws_nums = sorted(\n        {\n            1,\n            m._model_config.min_file_size_for_dl - 1,\n            m._model_config.min_file_size_for_dl,\n            m._model_config.min_file_size_for_dl + 1,\n            m._model_config.beg_size - 1,\n            m._model_config.beg_size,\n            m._model_config.beg_size + 1,\n            m._model_config.end_size - 1,\n            m._model_config.end_size,\n            m._model_config.end_size + 1,\n            m._model_config.beg_size + m._model_config.end_size - 1,\n            m._model_config.beg_size + m._model_config.end_size,\n            m._model_config.beg_size + m._model_config.end_size + 1,\n            m._model_config.beg_size + m._model_config.end_size + 1,\n            m._model_config.block_size - 1,\n            m._model_config.block_size,\n            m._model_config.block_size + 1,\n            2 * m._model_config.block_size - 1,\n            2 * m._model_config.block_size,\n            2 * m._model_config.block_size + 1,\n            4 * m._model_config.block_size - 1,\n            4 * m._model_config.block_size,\n            4 * m._model_config.block_size + 1,\n        }\n    )\n\n    for ws_num in ws_nums:\n        print(f\"Calling indentify_bytes with {ws_num} whitespaces\")\n        content = b\" \" * ws_num\n        res = m.identify_bytes(content)\n        assert (\n            res.ok\n            and res.dl.label == ContentTypeLabel.UNDEFINED\n            and res.output.label == ContentTypeLabel.TXT\n        )\n        res = m.identify_stream(io.BytesIO(content))\n        assert (\n            res.ok\n            and res.dl.label == ContentTypeLabel.UNDEFINED\n            and res.output.label == ContentTypeLabel.TXT\n        )\n        with tempfile.TemporaryDirectory() as td:\n            tf_path = Path(td) / \"test.bin\"\n            tf_path.write_bytes(content)\n            res = m.identify_path(tf_path)\n            assert (\n                res.ok\n                and res.dl.label == ContentTypeLabel.UNDEFINED\n                and res.output.label == ContentTypeLabel.TXT\n            )\n\n\ndef test_magika_module_with_different_prediction_modes() -> None:\n    model_dir = utils.get_default_model_dir()\n    m = Magika(model_dir=model_dir, prediction_mode=PredictionMode.BEST_GUESS)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.01\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.40\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.60\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.99\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n\n    m = Magika(model_dir=model_dir, prediction_mode=PredictionMode.MEDIUM_CONFIDENCE)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.01\n    ) == (\n        ContentTypeLabel.TXT,\n        OverwriteReason.LOW_CONFIDENCE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, m._model_config.medium_confidence_threshold - 0.01\n    ) == (ContentTypeLabel.TXT, OverwriteReason.LOW_CONFIDENCE)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.60\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.99\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n\n    m = Magika(model_dir=model_dir, prediction_mode=PredictionMode.HIGH_CONFIDENCE)\n    high_confidence_threshold = m._model_config.thresholds.get(\n        ContentTypeLabel.PYTHON, m._model_config.medium_confidence_threshold\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.01\n    ) == (\n        ContentTypeLabel.TXT,\n        OverwriteReason.LOW_CONFIDENCE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, high_confidence_threshold - 0.01\n    ) == (ContentTypeLabel.TXT, OverwriteReason.LOW_CONFIDENCE)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, high_confidence_threshold + 0.01\n    ) == (ContentTypeLabel.PYTHON, OverwriteReason.NONE)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.99\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n\n    # test that the default is HIGH_CONFIDENCE\n    m = Magika(model_dir=model_dir)\n    high_confidence_threshold = m._model_config.thresholds.get(\n        ContentTypeLabel.PYTHON, m._model_config.medium_confidence_threshold\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.01\n    ) == (\n        ContentTypeLabel.TXT,\n        OverwriteReason.LOW_CONFIDENCE,\n    )\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, high_confidence_threshold - 0.01\n    ) == (ContentTypeLabel.TXT, OverwriteReason.LOW_CONFIDENCE)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, high_confidence_threshold + 0.01\n    ) == (ContentTypeLabel.PYTHON, OverwriteReason.NONE)\n    assert m._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, 0.99\n    ) == (\n        ContentTypeLabel.PYTHON,\n        OverwriteReason.NONE,\n    )\n\n\ndef test_magika_module_overwrite_reason() -> None:\n    m_high = Magika(prediction_mode=PredictionMode.HIGH_CONFIDENCE)\n    m_medium = Magika(prediction_mode=PredictionMode.MEDIUM_CONFIDENCE)\n    m_best = Magika(prediction_mode=PredictionMode.BEST_GUESS)\n\n    python_high_confidence_threshold = m_high._model_config.thresholds.get(\n        ContentTypeLabel.PYTHON, m_high._model_config.medium_confidence_threshold\n    )\n    medium_confidence_threshold = m_medium._model_config.medium_confidence_threshold\n\n    assert m_high._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, python_high_confidence_threshold + 0.01\n    ) == (ContentTypeLabel.PYTHON, OverwriteReason.NONE)\n    assert m_high._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, python_high_confidence_threshold - 0.01\n    ) == (ContentTypeLabel.TXT, OverwriteReason.LOW_CONFIDENCE)\n\n    assert m_medium._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, medium_confidence_threshold + 0.01\n    ) == (ContentTypeLabel.PYTHON, OverwriteReason.NONE)\n    assert m_medium._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, medium_confidence_threshold - 0.01\n    ) == (ContentTypeLabel.TXT, OverwriteReason.LOW_CONFIDENCE)\n\n    assert m_best._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, medium_confidence_threshold + 0.01\n    ) == (ContentTypeLabel.PYTHON, OverwriteReason.NONE)\n    assert m_best._get_output_label_from_dl_label_and_score(\n        ContentTypeLabel.PYTHON, medium_confidence_threshold - 0.01\n    ) == (ContentTypeLabel.PYTHON, OverwriteReason.NONE)\n\n    for overwrite_map_ct_key in sorted(m_high._model_config.overwrite_map.keys()):\n        overwrite_map_ct_value = m_high._model_config.overwrite_map[\n            overwrite_map_ct_key\n        ]\n        is_overwrite_map_ct_target_text = m_high._cts_infos[\n            overwrite_map_ct_value\n        ].is_text\n        overwrite_map_ct_high_confidence_threshold = (\n            m_high._model_config.thresholds.get(\n                overwrite_map_ct_key, m_high._model_config.medium_confidence_threshold\n            )\n        )\n        assert m_high._get_output_label_from_dl_label_and_score(\n            overwrite_map_ct_key, overwrite_map_ct_high_confidence_threshold + 0.01\n        ) == (overwrite_map_ct_value, OverwriteReason.OVERWRITE_MAP)\n        assert m_high._get_output_label_from_dl_label_and_score(\n            overwrite_map_ct_key, overwrite_map_ct_high_confidence_threshold - 0.01\n        ) == (\n            ContentTypeLabel.TXT\n            if is_overwrite_map_ct_target_text\n            else ContentTypeLabel.UNKNOWN,\n            OverwriteReason.LOW_CONFIDENCE,\n        )\n\n    for generic_ct in [ContentTypeLabel.TXT, ContentTypeLabel.UNKNOWN]:\n        generic_type_high_confidence_threshold = m_high._model_config.thresholds.get(\n            generic_ct,\n            m_high._model_config.medium_confidence_threshold,\n        )\n        assert m_high._get_output_label_from_dl_label_and_score(\n            generic_ct,\n            generic_type_high_confidence_threshold - 0.01,\n        ) == (generic_ct, OverwriteReason.NONE)\n        assert m_medium._get_output_label_from_dl_label_and_score(\n            generic_ct, medium_confidence_threshold - 0.01\n        ) == (generic_ct, OverwriteReason.NONE)\n\n\ndef test_magika_module_with_directory() -> None:\n    m = Magika()\n\n    with tempfile.TemporaryDirectory() as td:\n        td_path = Path(td)\n        res = m.identify_path(td_path)\n        assert res.path == td_path\n        assert res.ok\n        assert res.prediction.dl.label == ContentTypeLabel.UNDEFINED\n        assert res.prediction.output.label == ContentTypeLabel.DIRECTORY\n        assert res.prediction.score == 1.0\n\n\ndef test_magika_module_multiple_copies_of_the_same_file() -> None:\n    with tempfile.TemporaryDirectory() as td:\n        test_path = Path(td) / \"test.txt\"\n        test_path.write_text(\"test\")\n\n        test_paths = [test_path] * 3\n\n        m = Magika()\n        results = m.identify_paths(test_paths)\n        assert len(results) == len(test_paths)\n        for result in results:\n            assert result.path == test_path\n            assert result.ok\n            assert result.prediction.output.label == ContentTypeLabel.TXT\n\n\ndef test_magika_module_with_symlink() -> None:\n    with tempfile.TemporaryDirectory() as td:\n        test_path = Path(td) / \"test.txt\"\n        test_path.write_text(\"test\")\n\n        symlink_path = Path(td) / \"symlink-test.txt\"\n        symlink_path.symlink_to(test_path)\n\n        m = Magika()\n        res = m.identify_path(test_path)\n        assert res.path == test_path\n        assert res.ok\n        assert res.prediction.output.label == ContentTypeLabel.TXT\n        res = m.identify_path(symlink_path)\n        assert res.path == symlink_path\n        assert res.ok\n        assert res.prediction.output.label == ContentTypeLabel.TXT\n\n        m = Magika(no_dereference=True)\n        res = m.identify_path(test_path)\n        assert res.path == test_path\n        assert res.ok\n        assert res.prediction.output.label == ContentTypeLabel.TXT\n        res = m.identify_path(symlink_path)\n        assert res.path == symlink_path\n        assert res.ok\n        assert res.prediction.output.label == ContentTypeLabel.SYMLINK\n\n\ndef test_magika_module_with_non_existing_file() -> None:\n    m = Magika()\n\n    with tempfile.TemporaryDirectory() as td:\n        non_existing_path = Path(td) / \"non_existing.txt\"\n\n        res = m.identify_path(non_existing_path)\n        assert res.path == non_existing_path\n        assert not res.ok\n        assert res.status == Status.FILE_NOT_FOUND_ERROR\n\n\ndef test_magika_module_with_permission_error() -> None:\n    m = Magika()\n\n    with tempfile.TemporaryDirectory() as td:\n        unreadable_test_path = Path(td) / \"test.txt\"\n        unreadable_test_path.write_text(\"text\")\n\n        unreadable_test_path.chmod(0o000)\n\n        res = m.identify_path(unreadable_test_path)\n        assert res.path == unreadable_test_path\n        assert not res.ok\n        assert res.status == Status.PERMISSION_ERROR\n\n    # Check that an empty, non-accessible file is marked as \"permission error\".\n    # Note that on some file-systems, one can read the file size even without\n    # read permission, and it would thus be possible to return \"empty\" (this is\n    # what we were actually doing in the past). However, returning\n    # \"permission_error\" makes the expected behavior consistent across file\n    # systems and it simplifies the implementation.\n    with tempfile.TemporaryDirectory() as td:\n        unreadable_test_path = Path(td) / \"test.txt\"\n        unreadable_test_path.write_text(\"\")\n\n        unreadable_test_path.chmod(0o000)\n\n        res = m.identify_path(unreadable_test_path)\n        assert res.path == unreadable_test_path\n        assert not res.ok\n        assert res.status == Status.PERMISSION_ERROR\n\n\n@pytest.mark.skip\ndef test_magika_module_with_really_many_files() -> None:\n    test_file_path = utils.get_one_basic_test_file_path()\n\n    m = Magika()\n\n    for n in [10000]:\n        test_files_paths = [test_file_path] * n\n\n        results = m.identify_paths(test_files_paths)\n        for result in results:\n            assert result.ok\n            # TODO: add more checks\n\n\n@pytest.mark.slow\ndef test_magika_module_with_big_file() -> None:\n    def signal_handler(signum: int, frame: Any) -> None:\n        raise Exception(\"Timeout\")\n\n    signal.signal(signal.SIGALRM, signal_handler)\n\n    # It should take much less than this, but pytest weird scheduling sometimes\n    # creates unexpected slow downs.\n    timeout = 2\n\n    m = Magika()\n\n    for sample_size in [1000, 10000, 1_000_000, 1_000_000_000, 10_000_000_000]:\n        with tempfile.TemporaryDirectory() as td:\n            sample_path = Path(td) / \"sample.dat\"\n            utils.write_random_file_with_size(sample_path, sample_size)\n            print(f\"Starting running Magika with a timeout of {timeout}\")\n            signal.alarm(timeout)\n            res = m.identify_path(sample_path)\n            assert res.ok\n            signal.alarm(0)\n            print(\"Done running Magika\")\n\n\ndef test_api_call_with_bad_types() -> None:\n    m = Magika()\n\n    _ = m.identify_path(Path(\"/non_existing.txt\"))\n    _ = m.identify_path(\"/non_existing.txt\")\n    with pytest.raises(TypeError):\n        _ = m.identify_path(b\"/non_existing.txt\")  # type: ignore[arg-type]\n\n    _ = m.identify_paths([Path(\"/non_existing.txt\")])\n    _ = m.identify_paths([\"/non_existing.txt\"])\n    _ = m.identify_paths([Path(\"/non_existing.txt\"), Path(\"/not_existing2.txt\")])\n    _ = m.identify_paths([Path(\"/non_existing.txt\"), \"/not_existing2.txt\"])\n    _ = m.identify_paths([\"/non_existing.txt\", \"/not_existing2.txt\"])\n    with pytest.raises(TypeError):\n        _ = m.identify_paths(Path(\"/non_existing.txt\"))  # type: ignore[arg-type]\n    with pytest.raises(TypeError):\n        _ = m.identify_paths([b\"/non_existing.txt\"])  # type: ignore[list-item]\n    with pytest.raises(TypeError):\n        _ = m.identify_paths([Path(\"/non_existing.txt\"), b\"/not_existing2.txt\"])  # type: ignore[list-item]\n\n    _ = m.identify_bytes(b\"bytes content\")\n    with pytest.raises(TypeError):\n        _ = m.identify_bytes(\"str content\")  # type: ignore[arg-type]\n\n    _ = m.identify_stream(io.BytesIO(b\"bytes stream content\"))\n    with pytest.raises(TypeError):\n        _ = m.identify_stream(io.StringIO(\"str stream content\"))  # type: ignore[arg-type]\n    with pytest.raises(TypeError):\n        _ = m.identify_stream(b\"bytes content\")  # type: ignore[arg-type]\n    with pytest.raises(TypeError):\n        _ = m.identify_stream(\"str content\")  # type: ignore[arg-type]\n\n\ndef test_access_magika_result_and_prediction():\n    m = Magika()\n\n    res = m.identify_bytes(b\"text\")\n    assert isinstance(res, MagikaResult)\n    assert isinstance(res.path, Path)\n    assert isinstance(res.ok, bool)\n    assert isinstance(res.status, Status)\n    assert isinstance(res.prediction, MagikaPrediction)\n    assert isinstance(res.prediction.dl, ContentTypeInfo)\n    assert isinstance(res.prediction.output, ContentTypeInfo)\n    assert isinstance(res.prediction.score, float)\n    # test access to forwarded properties\n    assert isinstance(res.dl, ContentTypeInfo)\n    assert isinstance(res.output, ContentTypeInfo)\n    assert isinstance(res.score, float)\n    # test access to non-existing properties\n    with pytest.raises(AttributeError):\n        _ = res.foo  # type: ignore[attr-defined]\n    with pytest.raises(AttributeError):\n        _ = res.prediction.foo  # type: ignore[attr-defined]\n\n    res = m.identify_path(Path(\"/non_existing.txt\"))\n    assert isinstance(res, MagikaResult)\n    assert isinstance(res.path, Path)\n    assert isinstance(res.ok, bool)\n    assert isinstance(res.status, Status)\n    with pytest.raises(ValueError):\n        _ = res.prediction\n    with pytest.raises(ValueError):\n        _ = res.prediction.dl\n    with pytest.raises(ValueError):\n        _ = res.prediction.output\n    with pytest.raises(ValueError):\n        _ = res.prediction.score\n    with pytest.raises(ValueError):\n        _ = res.dl\n    with pytest.raises(ValueError):\n        _ = res.output\n    with pytest.raises(ValueError):\n        _ = res.score\n    with pytest.raises(AttributeError):\n        _ = res.foo  # type: ignore[attr-defined]\n    with pytest.raises(ValueError):\n        _ = res.prediction.foo  # type: ignore[attr-defined]\n\n\ndef test_access_backward_compatibility_layer() -> None:\n    m = Magika()\n\n    res = m.identify_bytes(b\"text\")\n    assert isinstance(res, MagikaResult)\n    assert isinstance(res.path, Path)\n    assert isinstance(res.ok, bool)\n    assert isinstance(res.status, Status)\n    assert isinstance(res.prediction, MagikaPrediction)\n    assert isinstance(res.prediction.dl, ContentTypeInfo)\n    assert isinstance(res.prediction.output, ContentTypeInfo)\n    assert isinstance(res.prediction.score, float)\n\n    with pytest.warns(DeprecationWarning):\n        assert res.dl.ct_label == res.prediction.dl.label\n    with pytest.warns(DeprecationWarning):\n        assert res.output.ct_label == res.prediction.output.label\n\n    with pytest.raises(AttributeError):\n        _ = res.dl.score\n    with pytest.raises(AttributeError):\n        _ = res.output.score\n\n    with pytest.warns(DeprecationWarning):\n        assert res.dl.magic == res.prediction.dl.description\n    with pytest.warns(DeprecationWarning):\n        assert res.output.magic == res.prediction.output.description\n\n\ndef test_get_model_and_output_content_types() -> None:\n    m = Magika()\n    output_content_types = m.get_output_content_types()\n    output_content_types_set = set(output_content_types)\n    model_content_types = m.get_model_content_types()\n    model_content_types_set = set(model_content_types)\n\n    assert isinstance(output_content_types, List)\n    assert len(output_content_types) > 0\n    assert isinstance(model_content_types, List)\n    assert len(model_content_types) > 0\n\n    for ct in output_content_types:\n        assert isinstance(ct, ContentTypeLabel)\n\n    # Check for no duplicates\n    assert len(output_content_types) == len(output_content_types_set)\n\n    # Check basic properties about special ContentTypeLabel entries\n    special_output_content_types = {\n        ContentTypeLabel.DIRECTORY,\n        ContentTypeLabel.EMPTY,\n        ContentTypeLabel.SYMLINK,\n        ContentTypeLabel.TXT,\n        ContentTypeLabel.UNKNOWN,\n    }\n    special_model_content_types = {ContentTypeLabel.UNDEFINED}\n    assert special_output_content_types.issubset(output_content_types_set)\n    assert not special_model_content_types.issubset(output_content_types_set)\n    assert special_model_content_types.issubset(model_content_types_set)\n    assert not special_output_content_types.issubset(model_content_types_set)\n\n    # Spot check for popular content types\n    assert {\n        ContentTypeLabel.ELF,\n        ContentTypeLabel.PDF,\n    }.issubset(output_content_types_set)\n    assert {\n        ContentTypeLabel.ELF,\n        ContentTypeLabel.PDF,\n    }.issubset(model_content_types_set)\n\n\ndef test_magika_imports():\n    imported_modules = utils.get_imported_objects_after_wildcard()\n\n    # Check that Magika and other public classes are correctly imported\n    from magika import (\n        ContentTypeInfo,\n        ContentTypeLabel,\n        Magika,\n        MagikaError,\n        MagikaPrediction,\n        MagikaResult,\n        OverwriteReason,\n        PredictionMode,\n        Status,\n    )\n\n    assert imported_modules.get(\"ContentTypeInfo\") == ContentTypeInfo\n    assert imported_modules.get(\"ContentTypeLabel\") == ContentTypeLabel\n    assert imported_modules.get(\"Magika\") == Magika\n    assert imported_modules.get(\"MagikaError\") == MagikaError\n    assert imported_modules.get(\"MagikaPrediction\") == MagikaPrediction\n    assert imported_modules.get(\"MagikaResult\") == MagikaResult\n    assert imported_modules.get(\"OverwriteReason\") == OverwriteReason\n    assert imported_modules.get(\"PredictionMode\") == PredictionMode\n    assert imported_modules.get(\"Status\") == Status\n\n    # Check that internal classes are not imported\n    assert imported_modules.get(\"ModelFeatures\") is None\n    assert imported_modules.get(\"ModelOutput\") is None\n\n\ndef get_expected_content_type_label_from_test_file_path(\n    test_path: Path,\n) -> ContentTypeLabel:\n    return ContentTypeLabel(test_path.parent.name)\n\n\ndef check_result_vs_expected_result(\n    file_path: Path, result: MagikaResult, expected_result_path: Optional[Path] = None\n) -> None:\n    if expected_result_path is None:\n        expected_result_path = file_path\n    assert result.path == expected_result_path\n    assert result.ok\n    expected_ct_label = get_expected_content_type_label_from_test_file_path(file_path)\n    assert result.prediction.output.label == expected_ct_label\n\n\ndef check_results_vs_expected_results(\n    files_paths: List[Path], results: List[MagikaResult]\n) -> None:\n    for file_path, result in zip(files_paths, results):\n        check_result_vs_expected_result(file_path, result)\n"
  },
  {
    "path": "python/tests/test_python_magika_client.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport subprocess\nfrom pathlib import Path\n\n\ndef test_python_magika_client() -> None:\n    python_root_dir = Path(__file__).parent.parent\n    python_magika_client_path = (\n        python_root_dir / \"src\" / \"magika\" / \"cli\" / \"magika_client.py\"\n    ).resolve()\n\n    # quick test to check there are no obvious problems\n    cmd = [str(python_magika_client_path), \"--help\"]\n    subprocess.run(cmd, capture_output=True, check=True)\n\n    # quick test to check there are no crashes\n    cmd = [str(python_magika_client_path), str(python_magika_client_path)]\n    subprocess.run(cmd, capture_output=True, check=True)\n"
  },
  {
    "path": "python/tests/utils.py",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport gzip\nimport math\nimport random\nimport string\nfrom pathlib import Path\nfrom typing import Any, Dict, List\n\nimport pytest\n\n\ndef get_repo_root_dir() -> Path:\n    return Path(__file__).parent.parent.parent.resolve()\n\n\ndef get_tests_data_dir() -> Path:\n    repo_root_dir = get_repo_root_dir()\n    tests_data_dir = repo_root_dir / \"tests_data\"\n    if tests_data_dir.is_dir():\n        return tests_data_dir\n    tests_data_dir = repo_root_dir.parent / \"tests_data\"\n    assert tests_data_dir.is_dir()\n    return tests_data_dir\n\n\ndef get_basic_tests_files_dir() -> Path:\n    tests_files_dir = get_tests_data_dir() / \"basic\"\n    assert tests_files_dir.is_dir()\n    return tests_files_dir\n\n\ndef get_mitra_tests_files_dir() -> Path:\n    tests_files_dir = get_tests_data_dir() / \"mitra\"\n    assert tests_files_dir.is_dir()\n    return tests_files_dir\n\n\ndef get_previously_missdetected_files_dir() -> Path:\n    tests_files_dir = get_tests_data_dir() / \"previous_missdetections\"\n    assert tests_files_dir.is_dir()\n    return tests_files_dir\n\n\ndef get_basic_test_files_paths() -> List[Path]:\n    tests_files_dir = get_basic_tests_files_dir()\n    test_files_paths = sorted(filter(lambda p: p.is_file(), tests_files_dir.rglob(\"*\")))\n    return test_files_paths\n\n\ndef get_mitra_test_files_paths() -> List[Path]:\n    tests_files_dir = get_mitra_tests_files_dir()\n    test_files_paths = sorted(filter(lambda p: p.is_file(), tests_files_dir.rglob(\"*\")))\n    return test_files_paths\n\n\ndef get_previously_missdetected_files_paths() -> List[Path]:\n    tests_files_dir = get_previously_missdetected_files_dir()\n    test_files_paths = sorted(filter(lambda p: p.is_file(), tests_files_dir.rglob(\"*\")))\n    return test_files_paths\n\n\ndef get_reference_features_extraction_examples_path() -> Path:\n    return get_tests_data_dir() / \"reference\" / \"features_extraction_examples.json.gz\"\n\n\ndef get_reference_for_inference_examples_by_path_path(model_name: str) -> Path:\n    return (\n        get_tests_data_dir()\n        / \"reference\"\n        / f\"{model_name}-inference_examples_by_path.json.gz\"\n    )\n\n\ndef get_reference_for_inference_examples_by_content_path(model_name: str) -> Path:\n    return (\n        get_tests_data_dir()\n        / \"reference\"\n        / f\"{model_name}-inference_examples_by_content.json.gz\"\n    )\n\n\ndef get_one_basic_test_file_path() -> Path:\n    return get_basic_test_files_paths()[0]\n\n\ndef get_random_ascii_bytes(size: int) -> bytes:\n    return bytes(\n        [\n            random.choice(bytes(string.printable[:62].encode(\"ascii\")))\n            for _ in range(size)\n        ]\n    )\n\n\ndef get_random_bytes(size: int) -> bytes:\n    return bytes([random.choice(range(256)) for _ in range(size)])\n\n\ndef get_lines_from_stream(stream: str) -> List[str]:\n    candidates = stream.split(\"\\n\")\n    lines = []\n    for line in candidates:\n        line = line.strip()\n        if line == \"\":\n            continue\n        lines.append(line)\n    return lines\n\n\ndef write_random_file_with_size(sample_path: Path, sample_size: int) -> None:\n    print(f\"Writing random file at {str(sample_path)} with size {sample_size}\")\n    assert not sample_path.is_file()\n    block_size = 1024 * 1024 * 1024  # 1GB\n    with open(sample_path, \"wb\") as f:\n        if sample_size > block_size:\n            for _ in range(sample_size // block_size):\n                f.write(b\"A\" * block_size)\n            if sample_size % block_size != 0:\n                f.write(b\"A\" * (sample_size % block_size))\n        else:\n            f.write(b\"A\" * sample_size)\n    print(\"Random file created\")\n\n\ndef get_models_dir() -> Path:\n    return Path(__file__).parent.parent / \"src\" / \"magika\" / \"models\"\n\n\ndef get_default_model_dir() -> Path:\n    from magika.magika import Magika\n\n    return get_models_dir() / Magika._get_default_model_name()\n\n\ndef generate_whitespaces(size: int) -> bytes:\n    whitespaces = string.whitespace\n    ws_len = len(whitespaces)\n    return bytes([ord(whitespaces[idx % ws_len]) for idx in range(size)])\n\n\ndef generate_pattern(size: int, only_printable: bool) -> bytes:\n    \"\"\"Generate a pattern we can use to test features extraction\"\"\"\n\n    if only_printable:\n        chars = string.printable[: 10 + 26 * 2]\n        base_pattern = chars.encode(\"ascii\")\n    else:\n        base_pattern = bytes(range(256))\n\n    base_pattern_len = len(base_pattern)\n    pattern = (base_pattern * int(math.ceil(size / base_pattern_len)))[:size]\n    assert len(pattern) == size\n    return pattern\n\n\ndef gzip_compress(content: bytes) -> bytes:\n    return gzip.compress(content, mtime=0.0)\n\n\ndef gzip_decompress(content: bytes) -> bytes:\n    return gzip.decompress(content)\n\n\ndef get_imported_objects_after_wildcard() -> Dict[str, Any]:\n    # Execute 'from magika import *' in a new, clean namespace\n    namespace: Dict[str, Any] = {}\n    try:\n        exec(\"from magika import *\", globals(), namespace)\n    except ImportError as e:\n        pytest.fail(f\"Could not import module 'magika': {e}\")\n    return namespace\n"
  },
  {
    "path": "rust/.gitignore",
    "content": "/target/\n"
  },
  {
    "path": "rust/README.md",
    "content": "This directory contains the Rust crates and their tools. It has the following structure:\n- The `cli` directory contains the Magika Rust CLI. It is published on crates.io as `magika-cli`. It\n  can be compiled with `cargo build --release` from the `cli` directory. The output binary will be\n  `target/release/magika`.\n- The `lib` directory contains the Magika Rust library. It is published on crates.io as `magika`.\n- The `gen` directory is for maintainers when a new model is available.\n- The `test.sh` script tests the crates listed above. It runs as part of the Github continuous\n  integration.\n- The `sync.sh` script updates the library when a new model is available using the `gen` crate.\n- The `publish.sh` script prepares the library and the CLI for publishing to crates.io. It generates\n  a commit that must be merged first.\n- The `color.sh` is a shell library for the scripts above.\n- The remaining files have the usual meaning associated to their name.\n"
  },
  {
    "path": "rust/changelog.sh",
    "content": "#!/bin/sh\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ./color.sh\n\nfail() {\n  kind=$1\n  dir=$2\n  case $kind in\n    stale) message=\"Some changes have not been logged.\" ;;\n    format) message=\"This line should be an H2 version.\" ;;\n    diff) message=\"This version differs from the Cargo.toml file.\" ;;\n    *) error \"Unsupported kind '$kind'\" ;;\n  esac\n  if [ -z \"$CI\" ]\n  then error \"$message\"\n  else echo \"::warning file=rust/$dir/CHANGELOG.md,line=3::$message\"\n  fi\n}\n\nfor dir in lib cli; do\n  ( cd $dir\n    info \"Checking $dir\"\n    ref=$(git log -n1 --pretty=format:%H origin/main.. -- CHANGELOG.md)\n    [ -n \"$ref\" ] || ref=origin/main\n    git diff --quiet $ref -- Cargo.toml src || fail stale $dir\n    cver=\"$(sed -n '3s/^## //p' CHANGELOG.md)\"\n    [ -n \"$cver\" ] || fail format $dir\n    pver=\"$(sed -n '/^\\[package]$/,/^$/{s/^version = \"\\(.*\\)\"$/\\1/p}' Cargo.toml)\"\n    [ \"$pver\" = \"$cver\" ] || fail diff $dir\n  )\ndone\n"
  },
  {
    "path": "rust/cli/CHANGELOG.md",
    "content": "# Changelog\n\n## 1.0.3-dev\n\n### Patch\n\n- Update dependencies\n- Fix new clippy lints\n\n## 1.0.2\n\n### Patch\n\n- Update dependencies\n- Enable full LTO for the release profile\n\n## 1.0.1\n\n### Patch\n\n- Update dependencies\n\n## 1.0.0\n\n### Patch\n\n- Change description\n- Update dependencies\n\n## 0.1.4\n\n### Minor\n\n- Use true colors when available\n\n### Patch\n\n- Dissociate repository from published content (see `publish.sh` script)\n- Remove `package.metadata.deb`\n\n## 0.1.3\n\n### Minor\n\n- Change performance tuning configuration\n\n### Patch\n\n- Add `package.metadata.deb` for `cargo-deb` customization\n- Update dependencies\n\n## 0.1.2\n\n### Minor\n\n- Use the `standard_v3_3` model instead of `standard_v3_2` (see [model changelog])\n- Do not print the low-confidence warning if the content type was simply overwritten\n\n### Patch\n\n- Update dependencies\n\n## 0.1.1\n\n### Minor\n\n- Use the `standard_v3_2` model instead of `standard_v3_1` (see [model changelog])\n\n## 0.1.0\n\nNo changes.\n\n## 0.1.0-rc.5\n\n### Minor\n\n- Use the `standard_v3_1` model instead of `standard_v3_0` (see [model changelog])\n\n## 0.1.0-rc.4\n\n### Minor\n\n- Update the model thresholds\n\n## 0.1.0-rc.3\n\n### Minor\n\n- Use the `standard_v3_0` model instead of `standard_v2_1` (see [model changelog])\n\n### Patch\n\n- Update dependencies\n\n## 0.1.0-rc.2\n\n### Minor\n\n- Exit with non-zero code if at least one error was encountered (fixes #780)\n\n### Patch\n\n- Update dependencies\n\n## 0.1.0-rc.1\n\n### Minor\n\n- Print model version with `--version`\n- Change model from `standard_v2_0` to `standard_v2_1`\n\n### Patch\n\n- Fix running on MacOS by defaulting `--intra-threads` to 4\n- Fix the `--version` binary name from `magika-cli` to `magika`\n- Make sure ONNX Runtime telemetry is disabled\n- Change the default of the hidden flag `--num-tasks` from 1 to the number of CPUs\n\n## 0.1.0-rc.0\n\nThis version is the initial implementation and should be considered unstable. In particular, it\nships a new model in comparison to the Python binary and we would love feedback on\n[GitHub](https://github.com/google/magika/issues).\n\n## 0.0.0\n\nThis version is a placeholder and does not expose anything.\n\n[model changelog]: https://github.com/google/magika/blob/main/assets/models/CHANGELOG.md\n"
  },
  {
    "path": "rust/cli/Cargo.toml",
    "content": "[package]\nname = \"magika\"\nversion = \"1.0.3-dev\"\nauthors = [\"Magika Developers \"]\nlicense = \"Apache-2.0\"\nedition = \"2021\"\ndescription = \"Determines file content types using AI\"\nrepository = \"https://github.com/google/magika\"\nhomepage = \"https://securityresearch.google/magika\"\nkeywords = [\"cli\", \"file\", \"magic\"]\ncategories = [\"command-line-utilities\", \"filesystem\", \"parser-implementations\"]\ninclude = [\"/LICENSE\", \"/src\"]\n\n[dependencies]\nanyhow = \"1.0.86\"\nasync-channel = \"2.3.1\"\nclap = { version = \"4.5.9\", features = [\"cargo\", \"derive\", \"string\"] }\ncolored = \"3.0.0\"\nmagika_lib = { package = \"magika\", version = \"=1.1.0-dev\", path = \"../lib\", features = [\"serde\"] }\nnum_cpus = \"1.16.0\"\nort = \"=2.0.0-rc.12\"\nserde = { version = \"1.0.204\", features = [\"derive\"] }\nserde_json = \"1.0.120\"\ntokio = { version = \"1.43.1\", features = [\"full\"] }\n\n[profile.release]\ncodegen-units = 1\nlto = true\n\n[profile.dist]\ninherits = \"release\"\n"
  },
  {
    "path": "rust/cli/README.md",
    "content": "# Magika CLI\n\nThis binary crate implements a command-line interface (CLI) to the library crate\n[magika](https://crates.io/crates/magika) which provides file content type detection using AI.\n\n## Disclaimer\n\nThis project is not an official Google project. It is not supported by Google and Google\nspecifically disclaims all warranties as to its quality, merchantability, or fitness for a\nparticular purpose.\n\nThe `magika` library and this `magika-cli` binary are still unstable (as indicated by the major\nversion of zero) and new versions might introduce breaking changes (all changes will follow [cargo\nsemver compatibility](https://doc.rust-lang.org/cargo/reference/semver.html)). In particular,\nversion 0.1.0-rc.0 ships a new model in comparison to the Python binary and we would love feedback\non [GitHub](https://github.com/google/magika/issues).\n\n## Installation\n\nYou can install the latest version from the `magika` python package:\n\n```shell\npipx install magika\n```\n\nYou can install the latest version from a shell using `curl`:\n\n```shell\ncurl -LsSf https://securityresearch.google/magika/install.sh | sh\n```\n\nYou can install the latest version from a shell using `wget`:\n\n```shell\nwget -qO- https://securityresearch.google/magika/install.sh | sh\n```\n\nYou can install the latest version from a powershell:\n\n```shell\npowershell -ExecutionPolicy Bypass -c \"irm https://securityresearch.google/magika/install.ps1 | iex\"\n```\n\nYou can install the latest version from crates.io:\n\n```shell\ncargo install --locked magika-cli\n```\n\nIt is also possible to install from the git repository, in which case the version (accessible with\n`magika --version`) will be suffixed by `-dev` (e.g. `0.1.0-dev`) to indicate that the binary is the\ndevelopment version of the version prefix (e.g. `0.1.0` for the previous example).\n\nTo install the latest version from the git repository:\n\n```shell\ncargo install --locked --git=https://github.com/google/magika.git magika-cli\n```\n\nTo install from a local clone of the git repository (possibly with custom changes):\n\n```shell\ngit clone https://github.com/google/magika.git\ncd magika\ncargo install --locked --path=rust/cli\n```\n\n## Examples\n\n```shell\n% cd tests_data/basic && magika -r * | head\nasm/code.asm: Assembly (code)\nbatch/simple.bat: DOS batch file (code)\nc/code.c: C source (code)\ncss/code.css: CSS source (code)\ncsv/magika_test.csv: CSV document (code)\ndockerfile/Dockerfile: Dockerfile (code)\ndocx/doc.docx: Microsoft Word 2007+ document (document)\ndocx/magika_test.docx: Microsoft Word 2007+ document (document)\neml/sample.eml: RFC 822 mail (text)\nempty/empty_file: Empty file (inode)\n```\n\n```shell\n% magika ./tests_data/basic/python/code.py --json\n[\n  {\n    \"path\": \"./tests_data/basic/python/code.py\",\n    \"result\": {\n      \"status\": \"ok\",\n      \"value\": {\n        \"dl\": {\n          \"description\": \"Python source\",\n          \"extensions\": [\n            \"py\",\n            \"pyi\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"python\",\n          \"mime_type\": \"text/x-python\"\n        },\n        \"output\": {\n          \"description\": \"Python source\",\n          \"extensions\": [\n            \"py\",\n            \"pyi\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"python\",\n          \"mime_type\": \"text/x-python\"\n        },\n        \"score\": 0.996999979019165\n      }\n    }\n  }\n]\n```\n\n```shell\n% cat tests_data/basic/ini/doc.ini | magika -\n-: INI configuration file (text)\n```\n\n```shell\n% magika --help\nDetermines file content types using AI\n\nUsage: magika [OPTIONS] [PATH]...\n\nArguments:\n  [PATH]...\n          List of paths to the files to analyze.\n\n          Use a dash (-) to read from standard input (can only be used once).\n\nOptions:\n  -r, --recursive\n          Identifies files within directories instead of identifying the directory itself\n\n      --no-dereference\n          Identifies symbolic links as is instead of identifying their content by following them\n\n      --colors\n          Prints with colors regardless of terminal support\n\n      --no-colors\n          Prints without colors regardless of terminal support\n\n  -s, --output-score\n          Prints the prediction score in addition to the content type\n\n  -i, --mime-type\n          Prints the MIME type instead of the content type description\n\n  -l, --label\n          Prints a simple label instead of the content type description\n\n      --json\n          Prints in JSON format\n\n      --jsonl\n          Prints in JSONL format\n\n      --format \n          Prints using a custom format (use --help for details).\n\n          The following placeholders are supported:\n\n            %p  The file path\n            %l  The unique label identifying the content type\n            %d  The description of the content type\n            %g  The group of the content type\n            %m  The MIME type of the content type\n            %e  Possible file extensions for the content type\n            %s  The score of the content type for the file\n            %S  The score of the content type for the file in percent\n            %b  The model output if overruled (empty otherwise)\n            %%  A literal %\n\n  -h, --help\n          Print help (see a summary with '-h')\n\n  -V, --version\n          Print version\n```\n\n\nSee the [docs on Magika's core\nconcepts](https://securityresearch.google/magika/core-concepts/how-magika-works/) for more details\nabout the output format and other important aspects.\n"
  },
  {
    "path": "rust/cli/output",
    "content": "+ magika rust/code.rs\nrust/code.rs: Rust source (code)\n+ env -u COLORTERM magika rust/code.rs --colors\n\u001b[94mrust/code.rs: Rust source (code)\u001b[0m\n+ magika rust/code.rs --output-score\nrust/code.rs: Rust source (code) 57%\n+ magika rust/code.rs --json\n[\n  {\n    \"path\": \"rust/code.rs\",\n    \"result\": {\n      \"status\": \"ok\",\n      \"value\": {\n        \"dl\": {\n          \"description\": \"Rust source\",\n          \"extensions\": [\n            \"rs\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"rust\",\n          \"mime_type\": \"application/x-rust\"\n        },\n        \"output\": {\n          \"description\": \"Rust source\",\n          \"extensions\": [\n            \"rs\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"rust\",\n          \"mime_type\": \"application/x-rust\"\n        },\n        \"score\": 0.5759999752044678\n      }\n    }\n  }\n]\n+ magika rust/code.rs python/code.py --json\n[\n  {\n    \"path\": \"rust/code.rs\",\n    \"result\": {\n      \"status\": \"ok\",\n      \"value\": {\n        \"dl\": {\n          \"description\": \"Rust source\",\n          \"extensions\": [\n            \"rs\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"rust\",\n          \"mime_type\": \"application/x-rust\"\n        },\n        \"output\": {\n          \"description\": \"Rust source\",\n          \"extensions\": [\n            \"rs\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"rust\",\n          \"mime_type\": \"application/x-rust\"\n        },\n        \"score\": 0.5759999752044678\n      }\n    }\n  },\n  {\n    \"path\": \"python/code.py\",\n    \"result\": {\n      \"status\": \"ok\",\n      \"value\": {\n        \"dl\": {\n          \"description\": \"Python source\",\n          \"extensions\": [\n            \"py\",\n            \"pyi\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"python\",\n          \"mime_type\": \"text/x-python\"\n        },\n        \"output\": {\n          \"description\": \"Python source\",\n          \"extensions\": [\n            \"py\",\n            \"pyi\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"python\",\n          \"mime_type\": \"text/x-python\"\n        },\n        \"score\": 0.996999979019165\n      }\n    }\n  }\n]\n+ magika rust/code.rs --jsonl\n{\"path\":\"rust/code.rs\",\"result\":{\"status\":\"ok\",\"value\":{\"dl\":{\"description\":\"Rust source\",\"extensions\":[\"rs\"],\"group\":\"code\",\"is_text\":true,\"label\":\"rust\",\"mime_type\":\"application/x-rust\"},\"output\":{\"description\":\"Rust source\",\"extensions\":[\"rs\"],\"group\":\"code\",\"is_text\":true,\"label\":\"rust\",\"mime_type\":\"application/x-rust\"},\"score\":0.5759999752044678}}}\n+ magika rust/code.rs python/code.py --jsonl\n{\"path\":\"rust/code.rs\",\"result\":{\"status\":\"ok\",\"value\":{\"dl\":{\"description\":\"Rust source\",\"extensions\":[\"rs\"],\"group\":\"code\",\"is_text\":true,\"label\":\"rust\",\"mime_type\":\"application/x-rust\"},\"output\":{\"description\":\"Rust source\",\"extensions\":[\"rs\"],\"group\":\"code\",\"is_text\":true,\"label\":\"rust\",\"mime_type\":\"application/x-rust\"},\"score\":0.5759999752044678}}}\n{\"path\":\"python/code.py\",\"result\":{\"status\":\"ok\",\"value\":{\"dl\":{\"description\":\"Python source\",\"extensions\":[\"py\",\"pyi\"],\"group\":\"code\",\"is_text\":true,\"label\":\"python\",\"mime_type\":\"text/x-python\"},\"output\":{\"description\":\"Python source\",\"extensions\":[\"py\",\"pyi\"],\"group\":\"code\",\"is_text\":true,\"label\":\"python\",\"mime_type\":\"text/x-python\"},\"score\":0.996999979019165}}}\n+ magika rust/code.rs --mime-type\nrust/code.rs: application/x-rust\n"
  },
  {
    "path": "rust/cli/publish.sh",
    "content": "#!/bin/sh\n# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ../color.sh\n\n# This script renames the package to magika-cli and publishes to crates.io.\n\n[ -z \"$(git status -s)\" ] || error \"Repository is not clean\"\n\ninfo \"Patch Cargo.toml\"\nsed -i '2s/\"$/-cli\"/;s/^magika_lib = { package = \"magika\",/magika = {/' Cargo.toml\necho >> Cargo.toml\necho '[[bin]]' >> Cargo.toml\necho 'name = \"magika\"' >> Cargo.toml\necho 'path = \"src/main.rs\"' >> Cargo.toml\n\ninfo \"Patch src/main.rs\"\nsed -i 's/^use magika_lib/use magika/;s/self as magika,//' src/main.rs\ncargo fmt -- src/main.rs\n\ninfo \"Run tests\"\n./test.sh\n\ninfo \"Create a temporary commit\"\ngit commit -aqm'Temporary commit to publish magika-cli'\n\ninfo \"Publish to crates.io\"\ncargo publish\n\ninfo \"Revert the temporary commit\"\ngit reset --hard HEAD~\n"
  },
  {
    "path": "rust/cli/src/main.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse std::borrow::Cow;\nuse std::collections::HashMap;\nuse std::fmt::Write;\nuse std::io::ErrorKind;\nuse std::path::{Path, PathBuf};\nuse std::sync::Arc;\n\nuse anyhow::{bail, ensure, Result};\nuse clap::{Args, Parser};\nuse colored::ColoredString;\nuse magika_lib::{\n    self as magika, ContentType, Features, FeaturesOrRuled, FileType, InferredType,\n    OverwriteReason, Session, TypeInfo,\n};\nuse ort::session::builder::GraphOptimizationLevel;\nuse serde::Serialize;\nuse tokio::fs::File;\nuse tokio::io::AsyncReadExt;\n\n/// Determines file content types using AI.\n#[derive(Parser)]\n#[command(name = \"magika\", version = Version, arg_required_else_help = true)]\nstruct Flags {\n    /// List of paths to the files to analyze.\n    ///\n    /// Use a dash (-) to read from standard input (can only be used once).\n    path: Vec,\n\n    /// Identifies files within directories instead of identifying the directory itself.\n    #[arg(short, long)]\n    recursive: bool,\n\n    /// Identifies symbolic links as is instead of identifying their content by following them.\n    #[arg(long)]\n    no_dereference: bool,\n\n    #[clap(flatten)]\n    colors: Colors,\n\n    #[clap(flatten)]\n    modifiers: Modifiers,\n\n    #[clap(flatten)]\n    format: Format,\n\n    #[clap(flatten)]\n    experimental: Experimental,\n}\n\nstruct Version;\nimpl clap::builder::IntoResettable for Version {\n    fn into_resettable(self) -> clap::builder::Resettable {\n        let binary = clap::crate_version!();\n        let model = magika::MODEL_NAME;\n        clap::builder::Resettable::Value(format!(\"{binary} {model}\").into())\n    }\n}\n\n#[derive(Args)]\n#[group(multiple = false)]\nstruct Colors {\n    /// Prints with colors regardless of terminal support.\n    #[arg(long = \"colors\")]\n    enable: bool,\n\n    /// Prints without colors regardless of terminal support.\n    #[arg(long = \"no-colors\")]\n    disable: bool,\n}\n\n#[derive(Args)]\n#[group(conflicts_with = \"format\")]\nstruct Modifiers {\n    /// Prints the prediction score in addition to the content type.\n    #[arg(short = 's', long)]\n    output_score: bool,\n\n    /// Prints the MIME type instead of the content type description.\n    #[arg(short = 'i', long)]\n    mime_type: bool,\n\n    /// Prints a simple label instead of the content type description.\n    #[arg(short, long, conflicts_with = \"mime_type\")]\n    label: bool,\n}\n\n#[derive(Args)]\n#[group(id = \"format\", multiple = false)]\nstruct Format {\n    /// Prints in JSON format.\n    #[arg(long)]\n    json: bool,\n\n    /// Prints in JSONL format.\n    #[arg(long)]\n    jsonl: bool,\n\n    /// Prints using a custom format (use --help for details).\n    ///\n    /// The following placeholders are supported:\n    ///\n    ///   %p  The file path\n    ///   %l  The unique label identifying the content type\n    ///   %d  The description of the content type\n    ///   %g  The group of the content type\n    ///   %m  The MIME type of the content type\n    ///   %e  Possible file extensions for the content type\n    ///   %s  The score of the content type for the file\n    ///   %S  The score of the content type for the file in percent\n    ///   %b  The model output if overruled (empty otherwise)\n    ///   %%  A literal %\n    #[arg(long = \"format\", verbatim_doc_comment)]\n    custom: Option,\n}\n\n#[derive(Args)]\nstruct Experimental {\n    /// Number of files to identify in a single inference.\n    #[arg(hide = true, long, default_value = \"1\")]\n    batch_size: usize,\n\n    /// Number of tasks for batch parallelism.\n    #[arg(hide = true, long)]\n    num_tasks: Option,\n\n    /// Number of threads for graph parallelism (ONNX Runtime configuration).\n    ///\n    /// This has no effect if --parallel-execution is false or unset.\n    #[arg(hide = true, long)]\n    inter_threads: Option,\n\n    /// Number of threads for node parallelism (ONNX Runtime configuration).\n    #[arg(hide = true, long)]\n    intra_threads: Option,\n\n    /// Graph optimization level, from 0 to 3 (ONNX Runtime configuration).\n    #[arg(hide = true, long)]\n    optimization_level: Option,\n\n    /// Whether to enable parallel execution (ONNX Runtime configuration).\n    #[arg(hide = true, long)]\n    parallel_execution: Option,\n}\n\n#[tokio::main]\nasync fn main() -> Result<()> {\n    let mut flags = Flags::parse();\n    ensure!(0 < flags.experimental.batch_size, \"--batch-size cannot be zero\");\n    // If --num-tasks is set, we don't do any guessing.\n    let num_tasks = flags.experimental.num_tasks.unwrap_or_else(|| {\n        // Otherwise, if --intra-thread is set, we use a single task.\n        if flags.experimental.intra_threads.is_some() {\n            return 1;\n        }\n        // Otherwise, we use the minimum number of intra threads (which is 2).\n        flags.experimental.intra_threads = Some(2);\n        // And as many tasks as physical CPUs with a minimum of 2.\n        std::cmp::max(2, num_cpus::get_physical())\n    });\n    ensure!(0 < num_tasks, \"--num-tasks cannot be zero\");\n    ensure!(\n        flags.path.iter().filter(|x| x.to_str() == Some(\"-\")).count() <= 1,\n        \"only one path can be the standard input\"\n    );\n    let flags = Arc::new(flags);\n    if flags.colors.enable {\n        colored::control::set_override(true);\n    }\n    if flags.colors.disable {\n        colored::control::set_override(false);\n    }\n    let (result_sender, mut result_receiver) =\n        tokio::sync::mpsc::channel::>(num_tasks * flags.experimental.batch_size);\n    let (batch_sender, batch_receiver) = async_channel::bounded::(num_tasks);\n    tokio::spawn({\n        let flags = flags.clone();\n        let result_sender = result_sender.clone();\n        async move {\n            if let Err(e) = extract_features(&flags, &batch_sender, &result_sender).await {\n                result_sender.send(Err(e)).await.unwrap();\n            }\n        }\n    });\n    for _ in 0..num_tasks {\n        let mut magika = build_session(&flags)?;\n        tokio::spawn({\n            let batch_receiver = batch_receiver.clone();\n            let result_sender = result_sender.clone();\n            async move {\n                if let Err(e) = infer_batch(&mut magika, &batch_receiver, &result_sender).await {\n                    result_sender.send(Err(e)).await.unwrap();\n                }\n            }\n        });\n    }\n    drop(result_sender);\n    if flags.format.json {\n        print!(\"[\");\n    }\n    let mut reorder = Reorder::default();\n    let mut errors = false;\n    while let Some(response) = result_receiver.recv().await {\n        reorder.push(response?);\n        while let Some(response) = reorder.pop() {\n            errors |= response.result.is_err();\n            if flags.format.json {\n                if reorder.next != 1 {\n                    print!(\",\");\n                }\n                for line in serde_json::to_string_pretty(&response.json()?)?.lines() {\n                    print!(\"\\n  {line}\");\n                }\n            } else {\n                println!(\"{}\", response.format(&flags)?);\n            }\n        }\n    }\n    debug_assert!(reorder.is_empty());\n    if flags.format.json {\n        if reorder.next != 0 {\n            println!();\n        }\n        println!(\"]\");\n    }\n    if errors {\n        std::process::exit(1);\n    }\n    Ok(())\n}\n\nasync fn extract_features(\n    flags: &Flags, batch_sender: &async_channel::Sender,\n    result_sender: &tokio::sync::mpsc::Sender>,\n) -> Result<()> {\n    let mut paths = Vec::new();\n    let mut features = Vec::new();\n    let mut flags_paths = flags.path.clone();\n    flags_paths.reverse();\n    let mut order = 0;\n    while let Some(path) = flags_paths.pop() {\n        let mut result = None;\n        match process_path(flags, &mut flags_paths, &path).await {\n            Err(x) => result = Some(Err(x)),\n            Ok(ProcessPath::Recursive) => continue,\n            Ok(ProcessPath::Ruled(x)) => result = Some(Ok(x)),\n            Ok(ProcessPath::Features(x)) => features.push(x),\n        };\n        match result {\n            Some(result) => result_sender.send(Ok(Response { order, path, result })).await?,\n            None => paths.push((order, path)),\n        }\n        order += 1;\n        if features.len() == flags.experimental.batch_size {\n            batch_sender.send(Batch { paths, features }).await?;\n            paths = Vec::new();\n            features = Vec::new();\n        }\n    }\n    if !paths.is_empty() {\n        batch_sender.send(Batch { paths, features }).await?;\n    }\n    Ok(())\n}\n\nenum ProcessPath {\n    Recursive,\n    Features(Features),\n    Ruled(FileType),\n}\n\nimpl From for ProcessPath {\n    fn from(value: FeaturesOrRuled) -> Self {\n        match value {\n            FeaturesOrRuled::Features(x) => ProcessPath::Features(x),\n            FeaturesOrRuled::Ruled(x) => ProcessPath::Ruled(FileType::Ruled(x)),\n        }\n    }\n}\n\nasync fn process_path(\n    flags: &Flags, paths: &mut Vec, path: &Path,\n) -> magika::Result {\n    if path.to_str() == Some(\"-\") {\n        let mut stdin = Vec::new();\n        tokio::io::stdin().read_to_end(&mut stdin).await?;\n        return Ok(FeaturesOrRuled::extract_sync(&stdin[..])?.into());\n    }\n    let metadata = if flags.no_dereference {\n        tokio::fs::symlink_metadata(&path).await?\n    } else {\n        tokio::fs::metadata(&path).await?\n    };\n    if metadata.is_dir() {\n        return Ok(if flags.recursive {\n            let mut entries = tokio::fs::read_dir(&path).await?;\n            let mut dir_paths = Vec::new();\n            while let Some(entry) = entries.next_entry().await? {\n                dir_paths.push(entry.path());\n            }\n            dir_paths.sort();\n            while let Some(path) = dir_paths.pop() {\n                paths.push(path);\n            }\n            ProcessPath::Recursive\n        } else {\n            ProcessPath::Ruled(FileType::Directory)\n        });\n    }\n    if metadata.is_symlink() {\n        return Ok(ProcessPath::Ruled(FileType::Symlink));\n    }\n    let file = File::open(&path).await?;\n    Ok(FeaturesOrRuled::extract_async(file).await?.into())\n}\n\nfn build_session(flags: &Flags) -> Result {\n    ort::init().with_telemetry(false).commit();\n    let mut magika = Session::builder();\n    if let Some(inter_threads) = flags.experimental.inter_threads {\n        magika = magika.with_inter_threads(inter_threads);\n    }\n    // Apparently, SetIntraOpNumThreads must be called on MacOS, otherwise we get the following\n    // error: intra op thread pool must have at least one thread for RunAsync.\n    let intra_threads_default = cfg!(target_os = \"macos\").then_some(4);\n    if let Some(intra_threads) = flags.experimental.intra_threads.or(intra_threads_default) {\n        magika = magika.with_intra_threads(intra_threads);\n    }\n    if let Some(opt_level) = flags.experimental.optimization_level {\n        let opt_level = match opt_level {\n            0 => GraphOptimizationLevel::Disable,\n            1 => GraphOptimizationLevel::Level1,\n            2 => GraphOptimizationLevel::Level2,\n            3 => GraphOptimizationLevel::Level3,\n            _ => bail!(\"--optimization-level must be 0, 1, 2, or 3\"),\n        };\n        magika = magika.with_optimization_level(opt_level);\n    }\n    if let Some(parallel_execution) = flags.experimental.parallel_execution {\n        magika = magika.with_parallel_execution(parallel_execution);\n    }\n    Ok(magika.build()?)\n}\n\nasync fn infer_batch(\n    magika: &mut Session, receiver: &async_channel::Receiver,\n    sender: &tokio::sync::mpsc::Sender>,\n) -> Result<()> {\n    while let Ok(Batch { paths, features }) = receiver.recv().await {\n        let batch = magika.identify_features_batch_async(&features).await?;\n        assert_eq!(batch.len(), paths.len());\n        for ((order, path), output) in paths.into_iter().zip(batch) {\n            let result = Ok(output);\n            sender.send(Ok(Response { order, path, result })).await?;\n        }\n    }\n    Ok(())\n}\n\n#[derive(Debug, Default)]\nstruct Reorder {\n    next: usize,\n    todo: HashMap,\n}\n\nimpl Reorder {\n    fn is_empty(&self) -> bool {\n        self.todo.is_empty()\n    }\n\n    fn push(&mut self, response: Response) {\n        debug_assert!(self.next <= response.order);\n        let prev = self.todo.insert(response.order, response);\n        debug_assert!(prev.is_none());\n    }\n\n    fn pop(&mut self) -> Option {\n        let result = self.todo.remove(&self.next)?;\n        self.next += 1;\n        Some(result)\n    }\n}\n\nstruct Batch {\n    paths: Vec<(usize, PathBuf)>,\n    features: Vec,\n}\n\n#[derive(Debug)]\nstruct Response {\n    order: usize,\n    path: PathBuf,\n    result: Result,\n}\n\n#[derive(Serialize)]\n#[serde(rename_all = \"snake_case\")]\nenum JsonError {\n    Unknown,\n    FileDoesNotExist,\n    PermissionError,\n}\n\n#[derive(Serialize)]\nstruct JsonResult<'a> {\n    dl: &'a TypeInfo,\n    output: &'a TypeInfo,\n    score: f32,\n}\n\nimpl From for JsonError {\n    fn from(value: magika::Error) -> Self {\n        match value {\n            magika::Error::IOError(x) => match x.kind() {\n                ErrorKind::NotFound => JsonError::FileDoesNotExist,\n                ErrorKind::PermissionDenied => JsonError::PermissionError,\n                _ => JsonError::Unknown,\n            },\n            _ => JsonError::Unknown,\n        }\n    }\n}\n\nimpl Response {\n    fn format(self, flags: &Flags) -> Result {\n        let mut result = String::new();\n        let format = match &flags.format.custom {\n            Some(x) => x.clone(),\n            None if flags.format.json => unreachable!(),\n            None if flags.format.jsonl => {\n                return Ok(serde_json::to_string(&self.json()?)?.into());\n            }\n            None => {\n                let mut format = \"%p: \".to_string();\n                format.push_str(match () {\n                    () if flags.modifiers.mime_type => \"%m\",\n                    () if flags.modifiers.label => \"%l\",\n                    () => \"%d (%g)\",\n                });\n                format.push_str(\"%b\");\n                format.push_str(if flags.modifiers.output_score { \" %S\" } else { \"\" });\n                format\n            }\n        };\n        let mut format = format.chars();\n        loop {\n            match format.next() {\n                Some('%') => match format.next() {\n                    Some('p') => write!(&mut result, \"{}\", self.path.display())?,\n                    Some('l') => write!(&mut result, \"{}\", self.label())?,\n                    Some('d') => write!(&mut result, \"{}\", self.description())?,\n                    Some('g') => write!(&mut result, \"{}\", self.group())?,\n                    Some('m') => write!(&mut result, \"{}\", self.mime_type())?,\n                    Some('e') => write!(&mut result, \"{}\", join(self.extensions()))?,\n                    Some('s') => write!(&mut result, \"{:.2}\", self.score())?,\n                    Some('S') => write!(&mut result, \"{}%\", (100. * self.score()).trunc())?,\n                    Some('b') => {\n                        if let Ok(FileType::Inferred(InferredType {\n                            content_type: Some((_, OverwriteReason::LowConfidence)),\n                            inferred_type,\n                            score,\n                        })) = &self.result\n                        {\n                            write!(\n                                &mut result,\n                                \" [Low-confidence model best-guess: {} ({}), score={:.3}]\",\n                                inferred_type.info().description,\n                                inferred_type.info().group,\n                                score,\n                            )?;\n                        }\n                    }\n                    Some(c) => result.push(c),\n                    None => break,\n                },\n                Some(c) => result.push(c),\n                None => break,\n            }\n        }\n        Ok(self.color(result.into()))\n    }\n\n    fn json(self) -> Result {\n        let path = self.path.to_path_buf();\n        let result = match self.result {\n            Ok(x) => {\n                let dl = match &x {\n                    FileType::Inferred(x) => x.inferred_type.info(),\n                    _ => ContentType::Undefined.info(),\n                };\n                let output = x.info();\n                let score = (x.score() * 1000.).trunc() / 1000.;\n                let value = serde_json::to_value(JsonResult { dl, output, score })?;\n                serde_json::json!({ \"status\": \"ok\", \"value\": value })\n            }\n            Err(error) => serde_json::json!({ \"status\": JsonError::from(error) }),\n        };\n        Ok(serde_json::json!({ \"path\": path, \"result\": result }))\n    }\n\n    fn label(&self) -> &str {\n        match &self.result {\n            Err(_) => \"error\",\n            Ok(x) => x.info().label,\n        }\n    }\n\n    fn description(&self) -> Cow<'_, str> {\n        match &self.result {\n            Err(e) => e.to_string().into(),\n            Ok(x) => x.info().description.into(),\n        }\n    }\n\n    fn group(&self) -> &str {\n        match &self.result {\n            Err(_) => \"error\",\n            Ok(x) => x.info().group,\n        }\n    }\n\n    fn mime_type(&self) -> &str {\n        match &self.result {\n            Err(_) => \"error\",\n            Ok(x) => x.info().mime_type,\n        }\n    }\n\n    fn extensions(&self) -> &[&str] {\n        match &self.result {\n            Err(_) => &[],\n            Ok(x) => x.info().extensions,\n        }\n    }\n\n    fn score(&self) -> f32 {\n        match &self.result {\n            Err(_) => 1.0,\n            Ok(x) => x.score(),\n        }\n    }\n\n    fn color(&self, result: ColoredString) -> ColoredString {\n        use colored::Colorize as _;\n        // We only use true colors (except for errors). If the terminal doesn't support true colors,\n        // the colored crate will automatically choose the closest one.\n        match &self.result {\n            Err(_) => result.bold().red(),\n            Ok(x) => match x.info().group {\n                // Tailwind Colors\n                \"application\" => result.truecolor(0xf4, 0x3f, 0x5e), // Rose 500\n                \"archive\" => result.truecolor(0xf5, 0x9e, 0x0b),     // Amber 500\n                \"audio\" => result.truecolor(0x84, 0xcc, 0x16),       // Lime 500\n                \"code\" => result.truecolor(0x8b, 0x5c, 0xf6),        // Violet 500\n                \"document\" => result.truecolor(0x3b, 0x82, 0xf6),    // Blue 500\n                \"executable\" => result.truecolor(0xec, 0x48, 0x99),  // Pink 500\n                \"image\" => result.truecolor(0x06, 0xb6, 0xd4),       // Cyan 500\n                \"video\" => result.truecolor(0x10, 0xb9, 0x81),       // Emerald 500\n                _ => result.bold().truecolor(0xcc, 0xcc, 0xcc),\n            },\n        }\n    }\n}\n\nfn join>(xs: impl IntoIterator) -> String {\n    let mut result = String::new();\n    result.push('[');\n    for (i, x) in xs.into_iter().enumerate() {\n        if i != 0 {\n            result.push(',');\n        }\n        result.push_str(x.as_ref());\n    }\n    result.push(']');\n    result\n}\n"
  },
  {
    "path": "rust/cli/test.sh",
    "content": "#!/bin/sh\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ../color.sh\n\nx cargo check\nx cargo build --release\nx cargo fmt -- --check\nx cargo clippy -- --deny=warnings\n\nPATH=$(dirname $PWD)/target/release:$PATH\n\nTEST_SUITES='basic previous_missdetections'\ninfo \"Test against the test suites: $TEST_SUITES\"\n( cd ../../tests_data\n  magika --format='%p: %l' --recursive $TEST_SUITES | while read line; do\n    file=${line%: *}\n    directory=${file%/*}\n    expected=${directory##*/}\n    actual=${line#*: }\n    [ \"$expected\" = \"$actual\" ] || error \"$file is detected as $actual\"\n  done\n)\n\n# We rely below on the fact that we don't have permission on /etc/shadow.\n[ $(id -u) -eq 0 ] && success \"No more tests in Docker\"\n\ninfo \"Test exit code with at least one error\"\ntest_error() {\n  files=\"$1\"\n  expected=\"$2\"\n  ( set +e\n    actual=\"$(magika $files)\"\n    code=$?\n    [ $code -eq 1 ] || error \"invalid exit code for magika $files\"\n    [ \"$actual\" = \"$expected\" ] || error \"invalid output for magika $files\"\n  )\n}\ntest_error '/etc/shadow' \"\\\n/etc/shadow: Permission denied (os error 13) (error)\"\ntest_error 'non_existent src/main.rs' \"\\\nnon_existent: No such file or directory (os error 2) (error)\nsrc/main.rs: Rust source (code)\"\n"
  },
  {
    "path": "rust/color.sh",
    "content": "# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nx() { ( set -x; \"$@\"; ); }\n\ncolor() { echo \"\u001b[$1m$2:\u001b[m $3\"; }\ninfo() { color '1;36' Info \"$*\"; }\ntodo() { color '1;33' Todo \"$*\"; }\nsuccess() { color '1;32' Done \"$*\"; exit 0; }\nerror() { color '1;31' Error \"$*\"; exit 1; }\n"
  },
  {
    "path": "rust/gen/Cargo.toml",
    "content": "[package]\nname = \"gen\"\nversion = \"0.0.0\"\nedition = \"2021\"\npublish = false\n\n[dependencies]\nanyhow = \"1.0.82\"\nserde = { version = \"1.0.197\", features = [\"derive\"] }\nserde_json = \"1.0.114\"\n"
  },
  {
    "path": "rust/gen/README.md",
    "content": "This crate is for maintenance purposes only. It is used to update the Rust library to a new model.\nThere are 3 files in the Rust library that depend on the model:\n\n- The model itself, `rust/lib/src/model.onnx`, which is a symbolic link to some model under\n  `assets/models`, controlled by the `rust/gen/model` symbolic link. Publishing the crate will\n  dereference this symbolic link.\n- The labels describing the model output, `rust/lib/src/model.rs`, which is generated from the model\n  configuration, `rust/gen/model/config.min.json`.\n- The list of possible file types, `rust/lib/src/content.rs`, which is generated from the knowledge\n  base of content types, `assets/content_types_kb.min.json`.\n\nThe purpose of this crate is to generate the last two files. There is a test to make sure that they\nare up-to-date. If the test fails, one simply needs to run `./sync.sh` from the `rust` directory to\nregenerate them.\n\nAn alternative design to generating the files before publishing the crate, would be to publish the\nmodel and Magika configurations and use a build script to generate the files during compilation.\nThis has a few disadvantages:\n\n- We need to publish the model and Magika configurations which contain more information than needed\n  to use the library (and the CLI).\n- We need to use a build script, which is frown upon for security reasons, as the entity compiling\n  the library or CLI now needs to trust the build script, which can run arbitrary code. This only\n  matters when the entity compiling the library or CLI is not the same as the one running the\n  library or CLI (e.g. Debian maintainers), since the library and CLI too can run arbitrary code.\n- Using a build script also increases compilation time (and compilation complexity) instead of\n  having it factored before publishing.\n"
  },
  {
    "path": "rust/gen/content_types",
    "content": "3gp\nace\nai\naidl\napk\napplebplist\nappleplist\nasm\nasp\nautohotkey\nautoit\nawk\nbatch\nbazel\nbib\nbmp\nbzip\nc\ncab\ncat\nchm\nclojure\ncmake\ncobol\ncoff\ncoffeescript\ncpp\ncrt\ncrx\ncs\ncsproj\ncss\ncsv\ndart\ndeb\ndex\ndicom\ndiff\ndirectory\ndm\ndmg\ndoc\ndockerfile\ndocx\ndsstore\ndwg\ndxf\nelf\nelixir\nemf\neml\nempty\nepub\nerb\nerlang\nflac\nflv\nfortran\ngemfile\ngemspec\ngif\ngitattributes\ngitmodules\ngo\ngradle\ngroovy\ngzip\nh5\nhandlebars\nhaskell\nhcl\nhlp\nhtaccess\nhtml\nicns\nico\nics\nignorefile\nini\ninternetshortcut\nipynb\niso\njar\njava\njavabytecode\njavascript\njinja\njp2\njpeg\njson\njsonl\njulia\nkotlin\nlatex\nlha\nlisp\nlnk\nlua\nm3u\nm4\nmacho\nmakefile\nmarkdown\nmatlab\nmht\nmidi\nmkv\nmp3\nmp4\nmscompress\nmsi\nmum\nnpy\nnpz\nnupkg\nobjectivec\nocaml\nodp\nods\nodt\nogg\none\nonnx\notf\noutlook\nparquet\npascal\npcap\npdb\npdf\npebin\npem\nperl\nphp\npickle\npng\npo\npostscript\npowershell\nppt\npptx\nprolog\nproteindb\nproto\npsd\npython\npythonbytecode\npytorch\nqt\nr\nrandombytes\nrandomtxt\nrar\nrdf\nrpm\nrst\nrtf\nruby\nrust\nscala\nscss\nsevenzip\nsgml\nshell\nsmali\nsnap\nsolidity\nsql\nsqlite\nsquashfs\nsrt\nstlbinary\nstltext\nsum\nsvg\nswf\nswift\nsymlink\ntar\ntcl\ntextproto\ntga\nthumbsdb\ntiff\ntoml\ntorrent\ntsv\nttf\ntwig\ntxt\ntypescript\nundefined\nunknown\nvba\nvcxproj\nverilog\nvhdl\nvtt\nvue\nwasm\nwav\nwebm\nwebp\nwinregistry\nwmf\nwoff\nwoff2\nxar\nxls\nxlsb\nxlsx\nxml\nxpi\nxz\nyaml\nyara\nzig\nzip\nzlibstream\n"
  },
  {
    "path": "rust/gen/src/main.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse std::collections::{BTreeMap, BTreeSet};\nuse std::fs::File;\nuse std::io::Write;\nuse std::path::Path;\n\nuse anyhow::{ensure, Context, Result};\nuse serde::Deserialize;\n\nfn main() -> Result<()> {\n    let content_types: BTreeMap =\n        serde_json::from_reader(File::open(\"../../assets/content_types_kb.min.json\")?)?;\n    let model_name = std::fs::read_link(\"model\")?;\n    let model_name = model_name\n        .components()\n        .next_back()\n        .context(\"model link\")?\n        .as_os_str()\n        .to_str()\n        .context(\"model name\")?;\n    let model_config = serde_json::from_reader(File::open(\"model/config.min.json\")?)?;\n    let content_types = generate_content_types(content_types, model_name, &model_config)?;\n    generate_model_config(&content_types, model_config)?;\n    Ok(())\n}\n\nfn generate_content_types(\n    mut content_types: BTreeMap, model_name: &str, model_config: &ModelConfig,\n) -> Result> {\n    // We only want to generate content types that are already exposed or that are model labels.\n    // This is a conservative approach to avoid exposing the whole knowledge base if it contains\n    // experimental content types that won't ever be exposed in the future.\n    let content_types_content = std::fs::read_to_string(\"content_types\")?;\n    let mut labels = content_types_content.lines().collect::>();\n    labels.extend(model_config.target_labels_space.iter().map(|x| x.as_str()));\n    let mut content_types_file = File::create(\"content_types\")?;\n    for label in &labels {\n        writeln!(&mut content_types_file, \"{label}\")?;\n    }\n    content_types.retain(|x, _| labels.contains(x.as_str()));\n    let mut output = create_generated_file(\"../lib/src/content.rs\")?;\n    writeln!(output, \"use crate::file::TypeInfo;\\n\")?;\n    writeln!(output, \"/// Model name (only comparable with equality).\")?;\n    writeln!(output, \"pub const MODEL_NAME: &str = {model_name:?};\\n\")?;\n    writeln!(output, \"/// Model major version.\")?;\n    writeln!(output, \"pub const MODEL_MAJOR_VERSION: u32 = {};\\n\", model_config.version_major)?;\n    struct Variant {\n        label: String,\n        doc: String,\n    }\n    let mut variants = Vec::new();\n    for (label, info) in content_types {\n        let ContentType { mime_type, group, description, extensions, is_text } = info.clone();\n        let mime_type = mime_type.unwrap_or_else(|| {\n            if is_text { \"text/plain\" } else { \"application/octet-stream\" }.to_string()\n        });\n        let group = group.unwrap_or_else(|| \"unknown\".to_string());\n        let description = description.unwrap_or_else(|| label.clone());\n        if !matches!(label.as_str(), \"directory\" | \"symlink\") {\n            variants.push(Variant { label: label.clone(), doc: description.clone() });\n        }\n        writeln!(output, \"pub(crate) static {}: TypeInfo = TypeInfo {{\", const_name(&label))?;\n        writeln!(output, \"    label: {label:?},\")?;\n        writeln!(output, \"    mime_type: {mime_type:?},\")?;\n        writeln!(output, \"    group: {group:?},\")?;\n        writeln!(output, \"    description: {description:?},\")?;\n        writeln!(output, \"    extensions: &{extensions:?},\")?;\n        writeln!(output, \"    is_text: {is_text:?},\")?;\n        writeln!(output, \"}};\\n\")?;\n    }\n    writeln!(output, \"/// Content types for regular files.\")?;\n    writeln!(output, \"#[derive(Debug, Copy, Clone, PartialEq, Eq)]\")?;\n    writeln!(output, \"#[non_exhaustive]\")?;\n    writeln!(output, \"pub enum ContentType {{\")?;\n    for Variant { label, doc } in &variants {\n        writeln!(output, \"    /// {doc}\")?;\n        writeln!(output, \"    {},\", enum_name(label))?;\n    }\n    writeln!(output, \"}}\\n\")?;\n    writeln!(output, \"impl ContentType {{\")?;\n    writeln!(output, \"    pub(crate) const SIZE: usize = {};\\n\", variants.len())?;\n    writeln!(output, \"    /// Returns the content type information.\")?;\n    writeln!(output, \"    pub fn info(self) -> &'static TypeInfo {{\")?;\n    writeln!(output, \"        match self {{\")?;\n    for Variant { label, .. } in &variants {\n        writeln!(\n            output,\n            \"            ContentType::{} => &{},\",\n            enum_name(label),\n            const_name(label),\n        )?;\n    }\n    writeln!(output, \"        }}\")?;\n    writeln!(output, \"    }}\")?;\n    writeln!(output, \"}}\")?;\n    Ok(variants.into_iter().map(|x| x.label).collect())\n}\n\nfn generate_model_config(content_types: &[String], model_config: ModelConfig) -> Result<()> {\n    let ModelConfig {\n        beg_size,\n        mid_size,\n        end_size,\n        use_inputs_at_offsets,\n        medium_confidence_threshold,\n        min_file_size_for_dl,\n        padding_token,\n        block_size,\n        target_labels_space,\n        thresholds,\n        overwrite_map,\n        protection: _,\n        aes_key_hex: _,\n        version_major: _,\n    } = model_config;\n    let mut output = create_generated_file(\"../lib/src/model.rs\")?;\n    writeln!(output, \"use std::borrow::Cow;\\n\")?;\n    writeln!(output, \"use crate::config::ModelConfig;\")?;\n    writeln!(output, \"use crate::ContentType;\\n\")?;\n    writeln!(output, \"pub(crate) const CONFIG: ModelConfig = ModelConfig {{\")?;\n    writeln!(output, \"    beg_size: {beg_size},\")?;\n    ensure!(mid_size == 0, \"unsupported mid_size\");\n    writeln!(output, \"    end_size: {end_size},\")?;\n    ensure!(!use_inputs_at_offsets, \"unsupported use_inputs_at_offsets\");\n    writeln!(output, \"    min_file_size_for_dl: {min_file_size_for_dl},\")?;\n    writeln!(output, \"    padding_token: {padding_token},\")?;\n    writeln!(output, \"    block_size: {block_size},\")?;\n    writeln!(output, \"    thresholds: Cow::Borrowed(&THRESHOLDS),\")?;\n    writeln!(output, \"    overwrite_map: Cow::Borrowed(&OVERWRITE_MAP),\")?;\n    writeln!(output, \"}};\\n\")?;\n    let mut thresholds_array = vec![medium_confidence_threshold; content_types.len()];\n    for (label, threshold) in thresholds {\n        let pos = content_types.iter().position(|x| *x == label).unwrap();\n        thresholds_array[pos] = threshold;\n    }\n    writeln!(output, \"#[rustfmt::skip]\")?;\n    writeln!(output, \"const THRESHOLDS: [f32; ContentType::SIZE] = {thresholds_array:?};\")?;\n    writeln!(output, \"const OVERWRITE_MAP: [ContentType; ContentType::SIZE] = [\")?;\n    let mut overwrite_array = content_types.to_vec();\n    for (src, dst) in overwrite_map {\n        let pos = content_types.iter().position(|x| *x == src).unwrap();\n        overwrite_array[pos] = dst;\n    }\n    for label in overwrite_array {\n        writeln!(output, \"    ContentType::{},\", enum_name(&label))?;\n    }\n    writeln!(output, \"];\\n\")?;\n    writeln!(output, \"#[derive(Debug, Copy, Clone, PartialEq, Eq)]\\n#[repr(u32)]\")?;\n    writeln!(output, \"#[allow(dead_code)] // only constructed through transmute\")?;\n    writeln!(output, \"pub(crate) enum Label {{\")?;\n    for label in &target_labels_space {\n        writeln!(output, \"    {},\", enum_name(label))?;\n    }\n    writeln!(output, \"}}\\n\")?;\n    writeln!(output, \"pub(crate) const NUM_LABELS: usize = {};\", target_labels_space.len())?;\n    writeln!(output, \"impl Label {{\")?;\n    writeln!(output, \"    pub(crate) fn content_type(self) -> ContentType {{\")?;\n    writeln!(output, \"        match self {{\")?;\n    for label in &target_labels_space {\n        let name = enum_name(label);\n        writeln!(output, \"            Label::{name} => ContentType::{name},\")?;\n    }\n    writeln!(output, \"        }}\")?;\n    writeln!(output, \"    }}\")?;\n    writeln!(output, \"}}\")?;\n    Ok(())\n}\n\nfn create_generated_file(path: impl AsRef) -> Result {\n    let header = std::fs::read_to_string(file!())?;\n    let header = header.split(\"\\n\\n\").next().context(\"main.rs does not contain an empty line\")?;\n    let mut output = File::create(path)?;\n    writeln!(output, \"{header}\\n\")?;\n    writeln!(output, \"// DO NOT EDIT, see link below for more information:\")?;\n    writeln!(output, \"// https://github.com/google/magika/tree/main/rust/gen\\n\")?;\n    Ok(output)\n}\n\n#[derive(Clone, Deserialize)]\n#[serde(deny_unknown_fields)]\nstruct ContentType {\n    mime_type: Option,\n    group: Option,\n    description: Option,\n    extensions: Vec,\n    is_text: bool,\n}\n\n#[derive(Deserialize)]\n#[serde(deny_unknown_fields)]\nstruct ModelConfig {\n    beg_size: usize,\n    mid_size: usize,\n    end_size: usize,\n    use_inputs_at_offsets: bool,\n    medium_confidence_threshold: f32,\n    min_file_size_for_dl: usize,\n    padding_token: i32,\n    block_size: usize,\n    target_labels_space: Vec,\n    thresholds: BTreeMap,\n    overwrite_map: BTreeMap,\n    #[allow(dead_code)]\n    protection: String,\n    #[allow(dead_code)]\n    aes_key_hex: String,\n    version_major: u32,\n}\n\nfn enum_name(xs: &str) -> String {\n    assert!(xs.is_ascii());\n    let mut xs = xs.as_bytes().to_vec();\n    match xs[0] {\n        b'A'..=b'Z' => (),\n        b'a'..=b'z' => xs[0] = xs[0].to_ascii_uppercase(),\n        _ => xs.insert(0, b'_'),\n    }\n    String::from_utf8(xs).unwrap()\n}\n\nfn const_name(xs: &str) -> String {\n    assert!(xs.is_ascii());\n    let mut xs = xs.as_bytes().to_ascii_uppercase();\n    if !xs[0].is_ascii_uppercase() {\n        xs.insert(0, b'_');\n    }\n    String::from_utf8(xs).unwrap()\n}\n"
  },
  {
    "path": "rust/gen/test.sh",
    "content": "#!/bin/sh\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ../color.sh\n\nx cargo check\nx cargo fmt -- --check\nx cargo clippy -- --deny=warnings\n"
  },
  {
    "path": "rust/latest.sh",
    "content": "#!/bin/sh\n# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ./color.sh\n\n# This script updates (or creates if it does not exist) the trampoline release for the website. The\n# trampoline release is a \"fake\" release (which commits doesn't matter) containing a copy of the\n# bash and powershell scripts from the latest cli/vX.Y.Z release.\n\nREPO=google/magika\nRELEASES=$(gh release list --repo=$REPO --json=tagName |\n             tr -d '[{}]' | tr , '\\n' | sed 's/^.*\"\\([^\"]*\\)\"$/\\1/')\ninfo \"Found the following releases\"\necho \"$RELEASES\"\n\nTAG=$(echo \"$RELEASES\" | grep cli/ | head -n1)\nLATEST=cli-latest\n[ -n \"$TAG\" ] || error \"No CLI release found\"\ninfo \"The latest CLI release is $TAG\"\n\nx gh release download --repo=$REPO $TAG --pattern='magika-installer.*'\n\ninfo \"Delete the previous trampoline release, if any\"\necho \"$RELEASES\" | grep -q $LATEST && x gh release delete --repo=$REPO --yes $LATEST\n\ncat >> notes.txt <\"]\nlicense = \"Apache-2.0\"\nedition = \"2021\"\ndescription = \"Determines file content types using AI\"\nrepository = \"https://github.com/google/magika\"\nhomepage = \"https://securityresearch.google/magika\"\nkeywords = [\"file\", \"magic\"]\ncategories = [\"command-line-utilities\", \"filesystem\", \"parser-implementations\"]\ninclude = [\"/LICENSE\", \"/src\"]\n\n[package.metadata.docs.rs]\nfeatures = [\"_doc\"]\n\n[features]\nserde = [\"dep:serde\"]\n# Internal features.\n_doc = [\"serde\"]\n_test = [\"ort/download-binaries\", \"ort/tls-native\"]\n\n[dependencies]\nndarray = \"0.17.1\"\nserde = { version = \"1.0.204\", features = [\"derive\"], optional = true }\nthiserror = \"1.0.63\"\ntokio = { version = \"1.43.1\", features = [\"fs\", \"io-util\"] }\n\n[dependencies.ort]\nversion = \"=2.0.0-rc.12\"\ndefault-features = false\nfeatures = [\"ndarray\", \"std\"]\n\n[dev-dependencies]\ndata-encoding = \"2.6.0\"\nflate2 = \"1.0.30\"\nserde = { version = \"1.0.204\", features = [\"derive\"] }\nserde_json = \"1.0.120\"\n\n[lints.rust]\nmissing_docs = \"warn\"\nunreachable_pub = \"warn\"\nunused = { level = \"warn\", priority = -1 }\n"
  },
  {
    "path": "rust/lib/README.md",
    "content": "# Magika\n\nThis library crate provides file content type detection using AI. A command-line interface (CLI) for\nthis library is provided by the [magika-cli](https://crates.io/crates/magika-cli) binary crate.\n\n## Disclaimer\n\nThis project is not an official Google project. It is not supported by Google and Google\nspecifically disclaims all warranties as to its quality, merchantability, or fitness for a\nparticular purpose.\n\nThis `magika` library and the `magika-cli` binary are still unstable (as indicated by the major\nversion of zero) and new versions might introduce breaking changes (all changes will follow [cargo\nsemver compatibility](https://doc.rust-lang.org/cargo/reference/semver.html)). In particular,\nversion 0.1.0-rc.0 ships a new model in comparison to the Python binary and we would love feedback\non [GitHub](https://github.com/google/magika/issues).\n"
  },
  {
    "path": "rust/lib/src/builder.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse ort::session::builder::GraphOptimizationLevel;\n\nuse crate::{Result, Session};\n\n/// Configures and creates a Magika session.\n#[derive(Debug, Default)]\npub struct Builder {\n    inter_threads: Option,\n    intra_threads: Option,\n    optimization_level: Option,\n    parallel_execution: Option,\n}\n\nimpl Builder {\n    /// Configures the number of threads to parallelize the execution of the graph.\n    pub fn with_inter_threads(mut self, num_threads: usize) -> Self {\n        self.inter_threads = Some(num_threads);\n        self\n    }\n\n    /// Configures the number of threads to parallelize the execution within nodes.\n    pub fn with_intra_threads(mut self, num_threads: usize) -> Self {\n        self.intra_threads = Some(num_threads);\n        self\n    }\n\n    /// Configures the session optimization level.\n    pub fn with_optimization_level(mut self, opt_level: GraphOptimizationLevel) -> Self {\n        self.optimization_level = Some(opt_level);\n        self\n    }\n\n    /// Configures the session parallel execution.\n    pub fn with_parallel_execution(mut self, parallel_execution: bool) -> Self {\n        self.parallel_execution = Some(parallel_execution);\n        self\n    }\n\n    /// Consumes the builder to create a Magika session.\n    pub fn build(self) -> Result {\n        Ok(self.build_()?)\n    }\n\n    fn build_(self) -> ort::Result {\n        let mut session = ort::session::Session::builder()?;\n        let Builder { inter_threads, intra_threads, optimization_level, parallel_execution } = self;\n        if let Some(num_threads) = inter_threads {\n            session = session.with_inter_threads(num_threads)?;\n        }\n        if let Some(num_threads) = intra_threads {\n            session = session.with_intra_threads(num_threads)?;\n        }\n        if let Some(opt_level) = optimization_level {\n            session = session.with_optimization_level(opt_level)?;\n        }\n        if let Some(parallel_execution) = parallel_execution {\n            session = session.with_parallel_execution(parallel_execution)?;\n        }\n        let session = session.commit_from_memory(include_bytes!(\"model.onnx\"))?;\n        Ok(Session { session })\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/config.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse std::borrow::Cow;\n\nuse crate::ContentType;\n\n#[derive(Debug)]\npub(crate) struct ModelConfig {\n    pub(crate) beg_size: usize,\n    pub(crate) end_size: usize,\n    pub(crate) min_file_size_for_dl: usize,\n    pub(crate) padding_token: i32,\n    pub(crate) block_size: usize,\n    pub(crate) thresholds: Cow<'static, [f32; ContentType::SIZE]>,\n    pub(crate) overwrite_map: Cow<'static, [ContentType; ContentType::SIZE]>,\n}\n\npub(crate) struct SplitFeatures<'a> {\n    pub(crate) beg: &'a mut [i32],\n    pub(crate) end: &'a mut [i32],\n}\n\nimpl ModelConfig {\n    pub(crate) fn features_size(&self) -> usize {\n        self.beg_size + self.end_size\n    }\n\n    pub(crate) fn split_features<'a>(&self, features: &'a mut [i32]) -> SplitFeatures<'a> {\n        let (beg, features) = features.split_at_mut(self.beg_size);\n        let (end, features) = features.split_at_mut(self.end_size);\n        debug_assert!(features.is_empty());\n        SplitFeatures { beg, end }\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/content.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// DO NOT EDIT, see link below for more information:\n// https://github.com/google/magika/tree/main/rust/gen\n\nuse crate::file::TypeInfo;\n\n/// Model name (only comparable with equality).\npub const MODEL_NAME: &str = \"standard_v3_3\";\n\n/// Model major version.\npub const MODEL_MAJOR_VERSION: u32 = 3;\n\npub(crate) static _3GP: TypeInfo = TypeInfo {\n    label: \"3gp\",\n    mime_type: \"video/3gpp\",\n    group: \"video\",\n    description: \"3GPP multimedia file\",\n    extensions: &[\"3gp\"],\n    is_text: false,\n};\n\npub(crate) static ACE: TypeInfo = TypeInfo {\n    label: \"ace\",\n    mime_type: \"application/x-ace-compressed\",\n    group: \"archive\",\n    description: \"ACE archive\",\n    extensions: &[\"ace\"],\n    is_text: false,\n};\n\npub(crate) static AI: TypeInfo = TypeInfo {\n    label: \"ai\",\n    mime_type: \"application/pdf\",\n    group: \"document\",\n    description: \"Adobe Illustrator Artwork\",\n    extensions: &[\"ai\"],\n    is_text: false,\n};\n\npub(crate) static AIDL: TypeInfo = TypeInfo {\n    label: \"aidl\",\n    mime_type: \"text/plain\",\n    group: \"unknown\",\n    description: \"Android Interface Definition Language\",\n    extensions: &[\"aidl\"],\n    is_text: true,\n};\n\npub(crate) static APK: TypeInfo = TypeInfo {\n    label: \"apk\",\n    mime_type: \"application/vnd.android.package-archive\",\n    group: \"executable\",\n    description: \"Android package\",\n    extensions: &[\"apk\"],\n    is_text: false,\n};\n\npub(crate) static APPLEBPLIST: TypeInfo = TypeInfo {\n    label: \"applebplist\",\n    mime_type: \"application/x-bplist\",\n    group: \"application\",\n    description: \"Apple binary property list\",\n    extensions: &[\"bplist\", \"plist\"],\n    is_text: false,\n};\n\npub(crate) static APPLEPLIST: TypeInfo = TypeInfo {\n    label: \"appleplist\",\n    mime_type: \"application/x-plist\",\n    group: \"application\",\n    description: \"Apple property list\",\n    extensions: &[\"plist\"],\n    is_text: true,\n};\n\npub(crate) static ASM: TypeInfo = TypeInfo {\n    label: \"asm\",\n    mime_type: \"text/x-asm\",\n    group: \"code\",\n    description: \"Assembly\",\n    extensions: &[\"s\", \"S\", \"asm\"],\n    is_text: true,\n};\n\npub(crate) static ASP: TypeInfo = TypeInfo {\n    label: \"asp\",\n    mime_type: \"text/html\",\n    group: \"code\",\n    description: \"ASP source\",\n    extensions: &[\"aspx\", \"asp\"],\n    is_text: true,\n};\n\npub(crate) static AUTOHOTKEY: TypeInfo = TypeInfo {\n    label: \"autohotkey\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"AutoHotKey script\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static AUTOIT: TypeInfo = TypeInfo {\n    label: \"autoit\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"AutoIt script\",\n    extensions: &[\"au3\"],\n    is_text: true,\n};\n\npub(crate) static AWK: TypeInfo = TypeInfo {\n    label: \"awk\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Awk\",\n    extensions: &[\"awk\"],\n    is_text: true,\n};\n\npub(crate) static BATCH: TypeInfo = TypeInfo {\n    label: \"batch\",\n    mime_type: \"text/x-msdos-batch\",\n    group: \"code\",\n    description: \"DOS batch file\",\n    extensions: &[\"bat\"],\n    is_text: true,\n};\n\npub(crate) static BAZEL: TypeInfo = TypeInfo {\n    label: \"bazel\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Bazel build file\",\n    extensions: &[\"bzl\"],\n    is_text: true,\n};\n\npub(crate) static BIB: TypeInfo = TypeInfo {\n    label: \"bib\",\n    mime_type: \"text/x-bibtex\",\n    group: \"text\",\n    description: \"BibTeX\",\n    extensions: &[\"bib\"],\n    is_text: true,\n};\n\npub(crate) static BMP: TypeInfo = TypeInfo {\n    label: \"bmp\",\n    mime_type: \"image/bmp\",\n    group: \"image\",\n    description: \"BMP image data\",\n    extensions: &[\"bmp\"],\n    is_text: false,\n};\n\npub(crate) static BZIP: TypeInfo = TypeInfo {\n    label: \"bzip\",\n    mime_type: \"application/x-bzip2\",\n    group: \"archive\",\n    description: \"bzip2 compressed data\",\n    extensions: &[\"bz2\", \"tbz2\", \"tar.bz2\"],\n    is_text: false,\n};\n\npub(crate) static C: TypeInfo = TypeInfo {\n    label: \"c\",\n    mime_type: \"text/x-c\",\n    group: \"code\",\n    description: \"C source\",\n    extensions: &[\"c\"],\n    is_text: true,\n};\n\npub(crate) static CAB: TypeInfo = TypeInfo {\n    label: \"cab\",\n    mime_type: \"application/vnd.ms-cab-compressed\",\n    group: \"archive\",\n    description: \"Microsoft Cabinet archive data\",\n    extensions: &[\"cab\"],\n    is_text: false,\n};\n\npub(crate) static CAT: TypeInfo = TypeInfo {\n    label: \"cat\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"Windows Catalog file\",\n    extensions: &[\"cat\"],\n    is_text: false,\n};\n\npub(crate) static CHM: TypeInfo = TypeInfo {\n    label: \"chm\",\n    mime_type: \"application/chm\",\n    group: \"application\",\n    description: \"MS Windows HtmlHelp Data\",\n    extensions: &[\"chm\"],\n    is_text: false,\n};\n\npub(crate) static CLOJURE: TypeInfo = TypeInfo {\n    label: \"clojure\",\n    mime_type: \"text/x-clojure\",\n    group: \"code\",\n    description: \"Clojure\",\n    extensions: &[\"clj\", \"cljs\", \"cljc\", \"cljr\"],\n    is_text: true,\n};\n\npub(crate) static CMAKE: TypeInfo = TypeInfo {\n    label: \"cmake\",\n    mime_type: \"text/x-cmake\",\n    group: \"code\",\n    description: \"CMake build file\",\n    extensions: &[\"cmake\"],\n    is_text: true,\n};\n\npub(crate) static COBOL: TypeInfo = TypeInfo {\n    label: \"cobol\",\n    mime_type: \"text/x-cobol\",\n    group: \"code\",\n    description: \"Cobol\",\n    extensions: &[\"cbl\", \"cob\", \"cpy\", \"CBL\", \"COB\", \"CPY\"],\n    is_text: true,\n};\n\npub(crate) static COFF: TypeInfo = TypeInfo {\n    label: \"coff\",\n    mime_type: \"application/x-coff\",\n    group: \"executable\",\n    description: \"Intel 80386 COFF\",\n    extensions: &[\"obj\", \"o\"],\n    is_text: false,\n};\n\npub(crate) static COFFEESCRIPT: TypeInfo = TypeInfo {\n    label: \"coffeescript\",\n    mime_type: \"text/coffeescript\",\n    group: \"code\",\n    description: \"CoffeeScript\",\n    extensions: &[\"coffee\"],\n    is_text: true,\n};\n\npub(crate) static CPP: TypeInfo = TypeInfo {\n    label: \"cpp\",\n    mime_type: \"text/x-c\",\n    group: \"code\",\n    description: \"C++ source\",\n    extensions: &[\"cc\", \"cpp\", \"cxx\", \"c++\", \"cppm\", \"ixx\"],\n    is_text: true,\n};\n\npub(crate) static CRT: TypeInfo = TypeInfo {\n    label: \"crt\",\n    mime_type: \"application/x-x509-ca-cert\",\n    group: \"text\",\n    description: \"Certificates (binary format)\",\n    extensions: &[\"der\", \"cer\", \"crt\"],\n    is_text: false,\n};\n\npub(crate) static CRX: TypeInfo = TypeInfo {\n    label: \"crx\",\n    mime_type: \"application/x-chrome-extension\",\n    group: \"executable\",\n    description: \"Google Chrome extension\",\n    extensions: &[\"crx\"],\n    is_text: false,\n};\n\npub(crate) static CS: TypeInfo = TypeInfo {\n    label: \"cs\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"C# source\",\n    extensions: &[\"cs\", \"csx\"],\n    is_text: true,\n};\n\npub(crate) static CSPROJ: TypeInfo = TypeInfo {\n    label: \"csproj\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \".NET project config\",\n    extensions: &[\"csproj\"],\n    is_text: true,\n};\n\npub(crate) static CSS: TypeInfo = TypeInfo {\n    label: \"css\",\n    mime_type: \"text/css\",\n    group: \"code\",\n    description: \"CSS source\",\n    extensions: &[\"css\"],\n    is_text: true,\n};\n\npub(crate) static CSV: TypeInfo = TypeInfo {\n    label: \"csv\",\n    mime_type: \"text/csv\",\n    group: \"code\",\n    description: \"CSV document\",\n    extensions: &[\"csv\"],\n    is_text: true,\n};\n\npub(crate) static DART: TypeInfo = TypeInfo {\n    label: \"dart\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Dart source\",\n    extensions: &[\"dart\"],\n    is_text: true,\n};\n\npub(crate) static DEB: TypeInfo = TypeInfo {\n    label: \"deb\",\n    mime_type: \"application/vnd.debian.binary-package\",\n    group: \"archive\",\n    description: \"Debian binary package\",\n    extensions: &[\"deb\"],\n    is_text: false,\n};\n\npub(crate) static DEX: TypeInfo = TypeInfo {\n    label: \"dex\",\n    mime_type: \"application/x-android-dex\",\n    group: \"executable\",\n    description: \"Dalvik dex file\",\n    extensions: &[\"dex\"],\n    is_text: false,\n};\n\npub(crate) static DICOM: TypeInfo = TypeInfo {\n    label: \"dicom\",\n    mime_type: \"application/dicom\",\n    group: \"image\",\n    description: \"DICOM\",\n    extensions: &[\"dcm\"],\n    is_text: false,\n};\n\npub(crate) static DIFF: TypeInfo = TypeInfo {\n    label: \"diff\",\n    mime_type: \"text/plain\",\n    group: \"text\",\n    description: \"Diff file\",\n    extensions: &[\"diff\", \"patch\"],\n    is_text: true,\n};\n\npub(crate) static DIRECTORY: TypeInfo = TypeInfo {\n    label: \"directory\",\n    mime_type: \"inode/directory\",\n    group: \"inode\",\n    description: \"A directory\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static DM: TypeInfo = TypeInfo {\n    label: \"dm\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Dream Maker\",\n    extensions: &[\"dm\"],\n    is_text: true,\n};\n\npub(crate) static DMG: TypeInfo = TypeInfo {\n    label: \"dmg\",\n    mime_type: \"application/x-apple-diskimage\",\n    group: \"archive\",\n    description: \"Apple disk image\",\n    extensions: &[\"dmg\"],\n    is_text: false,\n};\n\npub(crate) static DOC: TypeInfo = TypeInfo {\n    label: \"doc\",\n    mime_type: \"application/msword\",\n    group: \"document\",\n    description: \"Microsoft Word CDF document\",\n    extensions: &[\"doc\"],\n    is_text: false,\n};\n\npub(crate) static DOCKERFILE: TypeInfo = TypeInfo {\n    label: \"dockerfile\",\n    mime_type: \"text/x-dockerfile\",\n    group: \"code\",\n    description: \"Dockerfile\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static DOCX: TypeInfo = TypeInfo {\n    label: \"docx\",\n    mime_type: \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\",\n    group: \"document\",\n    description: \"Microsoft Word 2007+ document\",\n    extensions: &[\"docx\", \"docm\"],\n    is_text: false,\n};\n\npub(crate) static DSSTORE: TypeInfo = TypeInfo {\n    label: \"dsstore\",\n    mime_type: \"application/octet-stream\",\n    group: \"unknown\",\n    description: \"Application Desktop Services Store\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static DWG: TypeInfo = TypeInfo {\n    label: \"dwg\",\n    mime_type: \"image/x-dwg\",\n    group: \"image\",\n    description: \"Autocad Drawing\",\n    extensions: &[\"dwg\"],\n    is_text: false,\n};\n\npub(crate) static DXF: TypeInfo = TypeInfo {\n    label: \"dxf\",\n    mime_type: \"image/vnd.dxf\",\n    group: \"image\",\n    description: \"Audocad Drawing Exchange Format\",\n    extensions: &[\"dxf\"],\n    is_text: true,\n};\n\npub(crate) static ELF: TypeInfo = TypeInfo {\n    label: \"elf\",\n    mime_type: \"application/x-executable-elf\",\n    group: \"executable\",\n    description: \"ELF executable\",\n    extensions: &[\"elf\"],\n    is_text: false,\n};\n\npub(crate) static ELIXIR: TypeInfo = TypeInfo {\n    label: \"elixir\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Elixir script\",\n    extensions: &[\"exs\"],\n    is_text: true,\n};\n\npub(crate) static EMF: TypeInfo = TypeInfo {\n    label: \"emf\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"Windows Enhanced Metafile image data\",\n    extensions: &[\"emf\"],\n    is_text: false,\n};\n\npub(crate) static EML: TypeInfo = TypeInfo {\n    label: \"eml\",\n    mime_type: \"message/rfc822\",\n    group: \"text\",\n    description: \"RFC 822 mail\",\n    extensions: &[\"eml\"],\n    is_text: true,\n};\n\npub(crate) static EMPTY: TypeInfo = TypeInfo {\n    label: \"empty\",\n    mime_type: \"inode/x-empty\",\n    group: \"inode\",\n    description: \"Empty file\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static EPUB: TypeInfo = TypeInfo {\n    label: \"epub\",\n    mime_type: \"application/epub+zip\",\n    group: \"document\",\n    description: \"EPUB document\",\n    extensions: &[\"epub\"],\n    is_text: false,\n};\n\npub(crate) static ERB: TypeInfo = TypeInfo {\n    label: \"erb\",\n    mime_type: \"text/x-ruby\",\n    group: \"code\",\n    description: \"Embedded Ruby source\",\n    extensions: &[\"erb\"],\n    is_text: true,\n};\n\npub(crate) static ERLANG: TypeInfo = TypeInfo {\n    label: \"erlang\",\n    mime_type: \"text/x-erlang\",\n    group: \"code\",\n    description: \"Erlang source\",\n    extensions: &[\"erl\", \"hrl\"],\n    is_text: true,\n};\n\npub(crate) static FLAC: TypeInfo = TypeInfo {\n    label: \"flac\",\n    mime_type: \"audio/flac\",\n    group: \"audio\",\n    description: \"FLAC audio bitstream data\",\n    extensions: &[\"flac\"],\n    is_text: false,\n};\n\npub(crate) static FLV: TypeInfo = TypeInfo {\n    label: \"flv\",\n    mime_type: \"video/x-flv\",\n    group: \"video\",\n    description: \"Flash Video\",\n    extensions: &[\"flv\"],\n    is_text: false,\n};\n\npub(crate) static FORTRAN: TypeInfo = TypeInfo {\n    label: \"fortran\",\n    mime_type: \"text/x-fortran\",\n    group: \"document\",\n    description: \"Fortran\",\n    extensions: &[\"f90\", \"f95\", \"f03\", \"F90\"],\n    is_text: true,\n};\n\npub(crate) static GEMFILE: TypeInfo = TypeInfo {\n    label: \"gemfile\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Gemfile file\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static GEMSPEC: TypeInfo = TypeInfo {\n    label: \"gemspec\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Gemspec file\",\n    extensions: &[\"gemspec\"],\n    is_text: true,\n};\n\npub(crate) static GIF: TypeInfo = TypeInfo {\n    label: \"gif\",\n    mime_type: \"image/gif\",\n    group: \"image\",\n    description: \"GIF image data\",\n    extensions: &[\"gif\"],\n    is_text: false,\n};\n\npub(crate) static GITATTRIBUTES: TypeInfo = TypeInfo {\n    label: \"gitattributes\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Gitattributes file\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static GITMODULES: TypeInfo = TypeInfo {\n    label: \"gitmodules\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Gitmodules file\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static GO: TypeInfo = TypeInfo {\n    label: \"go\",\n    mime_type: \"text/x-golang\",\n    group: \"code\",\n    description: \"Golang source\",\n    extensions: &[\"go\"],\n    is_text: true,\n};\n\npub(crate) static GRADLE: TypeInfo = TypeInfo {\n    label: \"gradle\",\n    mime_type: \"text/x-groovy\",\n    group: \"code\",\n    description: \"Gradle source\",\n    extensions: &[\"gradle\"],\n    is_text: true,\n};\n\npub(crate) static GROOVY: TypeInfo = TypeInfo {\n    label: \"groovy\",\n    mime_type: \"text/x-groovy\",\n    group: \"code\",\n    description: \"Groovy source\",\n    extensions: &[\"groovy\"],\n    is_text: true,\n};\n\npub(crate) static GZIP: TypeInfo = TypeInfo {\n    label: \"gzip\",\n    mime_type: \"application/gzip\",\n    group: \"archive\",\n    description: \"gzip compressed data\",\n    extensions: &[\"gz\", \"gzip\", \"tgz\", \"tar.gz\"],\n    is_text: false,\n};\n\npub(crate) static H5: TypeInfo = TypeInfo {\n    label: \"h5\",\n    mime_type: \"application/x-hdf5\",\n    group: \"archive\",\n    description: \"Hierarchical Data Format v5\",\n    extensions: &[\"h5\", \"hdf5\"],\n    is_text: false,\n};\n\npub(crate) static HANDLEBARS: TypeInfo = TypeInfo {\n    label: \"handlebars\",\n    mime_type: \"text/x-handlebars-template\",\n    group: \"code\",\n    description: \"Handlebars source\",\n    extensions: &[\"hbs\", \"handlebars\"],\n    is_text: true,\n};\n\npub(crate) static HASKELL: TypeInfo = TypeInfo {\n    label: \"haskell\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Haskell source\",\n    extensions: &[\"hs\", \"lhs\"],\n    is_text: true,\n};\n\npub(crate) static HCL: TypeInfo = TypeInfo {\n    label: \"hcl\",\n    mime_type: \"text/x-hcl\",\n    group: \"code\",\n    description: \"HashiCorp configuration language\",\n    extensions: &[\"hcl\"],\n    is_text: true,\n};\n\npub(crate) static HLP: TypeInfo = TypeInfo {\n    label: \"hlp\",\n    mime_type: \"application/winhlp\",\n    group: \"application\",\n    description: \"MS Windows help\",\n    extensions: &[\"hlp\"],\n    is_text: false,\n};\n\npub(crate) static HTACCESS: TypeInfo = TypeInfo {\n    label: \"htaccess\",\n    mime_type: \"text/x-apache-conf\",\n    group: \"code\",\n    description: \"Apache access configuration\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static HTML: TypeInfo = TypeInfo {\n    label: \"html\",\n    mime_type: \"text/html\",\n    group: \"code\",\n    description: \"HTML document\",\n    extensions: &[\"html\", \"htm\", \"xhtml\", \"xht\"],\n    is_text: true,\n};\n\npub(crate) static ICNS: TypeInfo = TypeInfo {\n    label: \"icns\",\n    mime_type: \"image/x-icns\",\n    group: \"image\",\n    description: \"Mac OS X icon\",\n    extensions: &[\"icns\"],\n    is_text: false,\n};\n\npub(crate) static ICO: TypeInfo = TypeInfo {\n    label: \"ico\",\n    mime_type: \"image/vnd.microsoft.icon\",\n    group: \"image\",\n    description: \"MS Windows icon resource\",\n    extensions: &[\"ico\"],\n    is_text: false,\n};\n\npub(crate) static ICS: TypeInfo = TypeInfo {\n    label: \"ics\",\n    mime_type: \"text/calendar\",\n    group: \"application\",\n    description: \"Internet Calendaring and Scheduling\",\n    extensions: &[\"ics\"],\n    is_text: true,\n};\n\npub(crate) static IGNOREFILE: TypeInfo = TypeInfo {\n    label: \"ignorefile\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Ignorefile\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static INI: TypeInfo = TypeInfo {\n    label: \"ini\",\n    mime_type: \"text/plain\",\n    group: \"text\",\n    description: \"INI configuration file\",\n    extensions: &[\"ini\"],\n    is_text: true,\n};\n\npub(crate) static INTERNETSHORTCUT: TypeInfo = TypeInfo {\n    label: \"internetshortcut\",\n    mime_type: \"application/x-mswinurl\",\n    group: \"application\",\n    description: \"MS Windows Internet shortcut\",\n    extensions: &[\"url\"],\n    is_text: true,\n};\n\npub(crate) static IPYNB: TypeInfo = TypeInfo {\n    label: \"ipynb\",\n    mime_type: \"application/json\",\n    group: \"code\",\n    description: \"Jupyter notebook\",\n    extensions: &[\"ipynb\"],\n    is_text: true,\n};\n\npub(crate) static ISO: TypeInfo = TypeInfo {\n    label: \"iso\",\n    mime_type: \"application/x-iso9660-image\",\n    group: \"archive\",\n    description: \"ISO 9660 CD-ROM filesystem data\",\n    extensions: &[\"iso\"],\n    is_text: false,\n};\n\npub(crate) static JAR: TypeInfo = TypeInfo {\n    label: \"jar\",\n    mime_type: \"application/java-archive\",\n    group: \"archive\",\n    description: \"Java archive data (JAR)\",\n    extensions: &[\"jar\", \"klib\"],\n    is_text: false,\n};\n\npub(crate) static JAVA: TypeInfo = TypeInfo {\n    label: \"java\",\n    mime_type: \"text/x-java\",\n    group: \"code\",\n    description: \"Java source\",\n    extensions: &[\"java\"],\n    is_text: true,\n};\n\npub(crate) static JAVABYTECODE: TypeInfo = TypeInfo {\n    label: \"javabytecode\",\n    mime_type: \"application/x-java-applet\",\n    group: \"executable\",\n    description: \"Java compiled bytecode\",\n    extensions: &[\"class\"],\n    is_text: false,\n};\n\npub(crate) static JAVASCRIPT: TypeInfo = TypeInfo {\n    label: \"javascript\",\n    mime_type: \"application/javascript\",\n    group: \"code\",\n    description: \"JavaScript source\",\n    extensions: &[\"js\", \"mjs\", \"cjs\"],\n    is_text: true,\n};\n\npub(crate) static JINJA: TypeInfo = TypeInfo {\n    label: \"jinja\",\n    mime_type: \"text/x-jinja2-template\",\n    group: \"code\",\n    description: \"Jinja template\",\n    extensions: &[\"jinja\", \"jinja2\", \"j2\"],\n    is_text: true,\n};\n\npub(crate) static JP2: TypeInfo = TypeInfo {\n    label: \"jp2\",\n    mime_type: \"image/jpeg2000\",\n    group: \"image\",\n    description: \"jpeg2000\",\n    extensions: &[\"jp2\"],\n    is_text: false,\n};\n\npub(crate) static JPEG: TypeInfo = TypeInfo {\n    label: \"jpeg\",\n    mime_type: \"image/jpeg\",\n    group: \"image\",\n    description: \"JPEG image data\",\n    extensions: &[\"jpg\", \"jpeg\"],\n    is_text: false,\n};\n\npub(crate) static JSON: TypeInfo = TypeInfo {\n    label: \"json\",\n    mime_type: \"application/json\",\n    group: \"code\",\n    description: \"JSON document\",\n    extensions: &[\"json\"],\n    is_text: true,\n};\n\npub(crate) static JSONL: TypeInfo = TypeInfo {\n    label: \"jsonl\",\n    mime_type: \"application/json\",\n    group: \"code\",\n    description: \"JSONL document\",\n    extensions: &[\"jsonl\", \"jsonld\"],\n    is_text: true,\n};\n\npub(crate) static JULIA: TypeInfo = TypeInfo {\n    label: \"julia\",\n    mime_type: \"text/x-julia\",\n    group: \"code\",\n    description: \"Julia source\",\n    extensions: &[\"jl\"],\n    is_text: true,\n};\n\npub(crate) static KOTLIN: TypeInfo = TypeInfo {\n    label: \"kotlin\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Kotlin source\",\n    extensions: &[\"kt\", \"kts\"],\n    is_text: true,\n};\n\npub(crate) static LATEX: TypeInfo = TypeInfo {\n    label: \"latex\",\n    mime_type: \"text/x-tex\",\n    group: \"text\",\n    description: \"LaTeX document\",\n    extensions: &[\"tex\", \"sty\"],\n    is_text: true,\n};\n\npub(crate) static LHA: TypeInfo = TypeInfo {\n    label: \"lha\",\n    mime_type: \"application/x-lha\",\n    group: \"archive\",\n    description: \"LHarc archive\",\n    extensions: &[\"lha\", \"lzh\"],\n    is_text: false,\n};\n\npub(crate) static LISP: TypeInfo = TypeInfo {\n    label: \"lisp\",\n    mime_type: \"text/x-lisp\",\n    group: \"code\",\n    description: \"Lisp source\",\n    extensions: &[\"lisp\", \"lsp\", \"l\", \"cl\"],\n    is_text: true,\n};\n\npub(crate) static LNK: TypeInfo = TypeInfo {\n    label: \"lnk\",\n    mime_type: \"application/x-ms-shortcut\",\n    group: \"application\",\n    description: \"MS Windows shortcut\",\n    extensions: &[\"lnk\"],\n    is_text: false,\n};\n\npub(crate) static LUA: TypeInfo = TypeInfo {\n    label: \"lua\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Lua\",\n    extensions: &[\"lua\"],\n    is_text: true,\n};\n\npub(crate) static M3U: TypeInfo = TypeInfo {\n    label: \"m3u\",\n    mime_type: \"text/plain\",\n    group: \"application\",\n    description: \"M3U playlist\",\n    extensions: &[\"m3u8\", \"m3u\"],\n    is_text: true,\n};\n\npub(crate) static M4: TypeInfo = TypeInfo {\n    label: \"m4\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"GNU Macro\",\n    extensions: &[\"m4\"],\n    is_text: true,\n};\n\npub(crate) static MACHO: TypeInfo = TypeInfo {\n    label: \"macho\",\n    mime_type: \"application/x-mach-o\",\n    group: \"executable\",\n    description: \"Mach-O executable\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static MAKEFILE: TypeInfo = TypeInfo {\n    label: \"makefile\",\n    mime_type: \"text/x-makefile\",\n    group: \"code\",\n    description: \"Makefile source\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static MARKDOWN: TypeInfo = TypeInfo {\n    label: \"markdown\",\n    mime_type: \"text/markdown\",\n    group: \"text\",\n    description: \"Markdown document\",\n    extensions: &[\"md\", \"markdown\"],\n    is_text: true,\n};\n\npub(crate) static MATLAB: TypeInfo = TypeInfo {\n    label: \"matlab\",\n    mime_type: \"text/x-matlab\",\n    group: \"code\",\n    description: \"Matlab Source\",\n    extensions: &[\"m\", \"matlab\"],\n    is_text: true,\n};\n\npub(crate) static MHT: TypeInfo = TypeInfo {\n    label: \"mht\",\n    mime_type: \"application/x-mimearchive\",\n    group: \"code\",\n    description: \"MHTML document\",\n    extensions: &[\"mht\"],\n    is_text: true,\n};\n\npub(crate) static MIDI: TypeInfo = TypeInfo {\n    label: \"midi\",\n    mime_type: \"audio/midi\",\n    group: \"audio\",\n    description: \"Midi\",\n    extensions: &[\"mid\"],\n    is_text: false,\n};\n\npub(crate) static MKV: TypeInfo = TypeInfo {\n    label: \"mkv\",\n    mime_type: \"video/x-matroska\",\n    group: \"video\",\n    description: \"Matroska\",\n    extensions: &[\"mkv\"],\n    is_text: false,\n};\n\npub(crate) static MP3: TypeInfo = TypeInfo {\n    label: \"mp3\",\n    mime_type: \"audio/mpeg\",\n    group: \"audio\",\n    description: \"MP3 media file\",\n    extensions: &[\"mp3\"],\n    is_text: false,\n};\n\npub(crate) static MP4: TypeInfo = TypeInfo {\n    label: \"mp4\",\n    mime_type: \"video/mp4\",\n    group: \"video\",\n    description: \"MP4 media file\",\n    extensions: &[\"mp4\"],\n    is_text: false,\n};\n\npub(crate) static MSCOMPRESS: TypeInfo = TypeInfo {\n    label: \"mscompress\",\n    mime_type: \"application/x-ms-compress-szdd\",\n    group: \"archive\",\n    description: \"MS Compress archive data\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static MSI: TypeInfo = TypeInfo {\n    label: \"msi\",\n    mime_type: \"application/x-msi\",\n    group: \"archive\",\n    description: \"Microsoft Installer file\",\n    extensions: &[\"msi\"],\n    is_text: false,\n};\n\npub(crate) static MUM: TypeInfo = TypeInfo {\n    label: \"mum\",\n    mime_type: \"text/xml\",\n    group: \"application\",\n    description: \"Windows Update Package file\",\n    extensions: &[\"mum\"],\n    is_text: true,\n};\n\npub(crate) static NPY: TypeInfo = TypeInfo {\n    label: \"npy\",\n    mime_type: \"application/octet-stream\",\n    group: \"archive\",\n    description: \"Numpy Array\",\n    extensions: &[\"npy\"],\n    is_text: false,\n};\n\npub(crate) static NPZ: TypeInfo = TypeInfo {\n    label: \"npz\",\n    mime_type: \"application/octet-stream\",\n    group: \"archive\",\n    description: \"Numpy Arrays Archive\",\n    extensions: &[\"npz\"],\n    is_text: false,\n};\n\npub(crate) static NUPKG: TypeInfo = TypeInfo {\n    label: \"nupkg\",\n    mime_type: \"application/octet-stream\",\n    group: \"unknown\",\n    description: \"NuGet Package\",\n    extensions: &[\"nupkg\"],\n    is_text: false,\n};\n\npub(crate) static OBJECTIVEC: TypeInfo = TypeInfo {\n    label: \"objectivec\",\n    mime_type: \"text/x-objcsrc\",\n    group: \"code\",\n    description: \"ObjectiveC source\",\n    extensions: &[\"m\", \"mm\"],\n    is_text: true,\n};\n\npub(crate) static OCAML: TypeInfo = TypeInfo {\n    label: \"ocaml\",\n    mime_type: \"text-ocaml\",\n    group: \"code\",\n    description: \"OCaml\",\n    extensions: &[\"ml\", \"mli\"],\n    is_text: true,\n};\n\npub(crate) static ODP: TypeInfo = TypeInfo {\n    label: \"odp\",\n    mime_type: \"application/vnd.oasis.opendocument.presentation\",\n    group: \"document\",\n    description: \"OpenDocument Presentation\",\n    extensions: &[\"odp\"],\n    is_text: false,\n};\n\npub(crate) static ODS: TypeInfo = TypeInfo {\n    label: \"ods\",\n    mime_type: \"application/vnd.oasis.opendocument.spreadsheet\",\n    group: \"document\",\n    description: \"OpenDocument Spreadsheet\",\n    extensions: &[\"ods\"],\n    is_text: false,\n};\n\npub(crate) static ODT: TypeInfo = TypeInfo {\n    label: \"odt\",\n    mime_type: \"application/vnd.oasis.opendocument.text\",\n    group: \"document\",\n    description: \"OpenDocument Text\",\n    extensions: &[\"odt\"],\n    is_text: false,\n};\n\npub(crate) static OGG: TypeInfo = TypeInfo {\n    label: \"ogg\",\n    mime_type: \"audio/ogg\",\n    group: \"audio\",\n    description: \"Ogg data\",\n    extensions: &[\"ogg\"],\n    is_text: false,\n};\n\npub(crate) static ONE: TypeInfo = TypeInfo {\n    label: \"one\",\n    mime_type: \"application/msonenote\",\n    group: \"document\",\n    description: \"One Note\",\n    extensions: &[\"one\"],\n    is_text: false,\n};\n\npub(crate) static ONNX: TypeInfo = TypeInfo {\n    label: \"onnx\",\n    mime_type: \"application/octet-stream\",\n    group: \"archive\",\n    description: \"Open Neural Network Exchange\",\n    extensions: &[\"onnx\"],\n    is_text: false,\n};\n\npub(crate) static OTF: TypeInfo = TypeInfo {\n    label: \"otf\",\n    mime_type: \"font/otf\",\n    group: \"font\",\n    description: \"OpenType font\",\n    extensions: &[\"otf\"],\n    is_text: false,\n};\n\npub(crate) static OUTLOOK: TypeInfo = TypeInfo {\n    label: \"outlook\",\n    mime_type: \"application/vnd.ms-outlook\",\n    group: \"application\",\n    description: \"MS Outlook Message\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static PARQUET: TypeInfo = TypeInfo {\n    label: \"parquet\",\n    mime_type: \"application/vnd.apache.parquet\",\n    group: \"unknown\",\n    description: \"Apache Parquet\",\n    extensions: &[\"pqt\", \"parquet\"],\n    is_text: false,\n};\n\npub(crate) static PASCAL: TypeInfo = TypeInfo {\n    label: \"pascal\",\n    mime_type: \"text/x-pascal\",\n    group: \"code\",\n    description: \"Pascal source\",\n    extensions: &[\"pas\", \"pp\"],\n    is_text: true,\n};\n\npub(crate) static PCAP: TypeInfo = TypeInfo {\n    label: \"pcap\",\n    mime_type: \"application/vnd.tcpdump.pcap\",\n    group: \"application\",\n    description: \"pcap capture file\",\n    extensions: &[\"pcap\", \"pcapng\"],\n    is_text: false,\n};\n\npub(crate) static PDB: TypeInfo = TypeInfo {\n    label: \"pdb\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"Windows Program Database\",\n    extensions: &[\"pdb\"],\n    is_text: false,\n};\n\npub(crate) static PDF: TypeInfo = TypeInfo {\n    label: \"pdf\",\n    mime_type: \"application/pdf\",\n    group: \"document\",\n    description: \"PDF document\",\n    extensions: &[\"pdf\"],\n    is_text: false,\n};\n\npub(crate) static PEBIN: TypeInfo = TypeInfo {\n    label: \"pebin\",\n    mime_type: \"application/x-dosexec\",\n    group: \"executable\",\n    description: \"PE Windows executable\",\n    extensions: &[\"exe\", \"dll\"],\n    is_text: false,\n};\n\npub(crate) static PEM: TypeInfo = TypeInfo {\n    label: \"pem\",\n    mime_type: \"application/x-pem-file\",\n    group: \"application\",\n    description: \"PEM certificate\",\n    extensions: &[\"pem\", \"pub\", \"gpg\"],\n    is_text: true,\n};\n\npub(crate) static PERL: TypeInfo = TypeInfo {\n    label: \"perl\",\n    mime_type: \"text/x-perl\",\n    group: \"code\",\n    description: \"Perl source\",\n    extensions: &[\"pl\"],\n    is_text: true,\n};\n\npub(crate) static PHP: TypeInfo = TypeInfo {\n    label: \"php\",\n    mime_type: \"text/x-php\",\n    group: \"code\",\n    description: \"PHP source\",\n    extensions: &[\"php\"],\n    is_text: true,\n};\n\npub(crate) static PICKLE: TypeInfo = TypeInfo {\n    label: \"pickle\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"Python pickle\",\n    extensions: &[\"pickle\", \"pkl\"],\n    is_text: false,\n};\n\npub(crate) static PNG: TypeInfo = TypeInfo {\n    label: \"png\",\n    mime_type: \"image/png\",\n    group: \"image\",\n    description: \"PNG image\",\n    extensions: &[\"png\"],\n    is_text: false,\n};\n\npub(crate) static PO: TypeInfo = TypeInfo {\n    label: \"po\",\n    mime_type: \"text/gettext-translation\",\n    group: \"application\",\n    description: \"Portable Object (PO) for i18n\",\n    extensions: &[\"po\"],\n    is_text: true,\n};\n\npub(crate) static POSTSCRIPT: TypeInfo = TypeInfo {\n    label: \"postscript\",\n    mime_type: \"application/postscript\",\n    group: \"document\",\n    description: \"PostScript document\",\n    extensions: &[\"ps\"],\n    is_text: false,\n};\n\npub(crate) static POWERSHELL: TypeInfo = TypeInfo {\n    label: \"powershell\",\n    mime_type: \"application/x-powershell\",\n    group: \"code\",\n    description: \"Powershell source\",\n    extensions: &[\"ps1\"],\n    is_text: true,\n};\n\npub(crate) static PPT: TypeInfo = TypeInfo {\n    label: \"ppt\",\n    mime_type: \"application/vnd.ms-powerpoint\",\n    group: \"document\",\n    description: \"Microsoft PowerPoint CDF document\",\n    extensions: &[\"ppt\"],\n    is_text: false,\n};\n\npub(crate) static PPTX: TypeInfo = TypeInfo {\n    label: \"pptx\",\n    mime_type: \"application/vnd.openxmlformats-officedocument.presentationml.presentation\",\n    group: \"document\",\n    description: \"Microsoft PowerPoint 2007+ document\",\n    extensions: &[\"pptx\", \"pptm\"],\n    is_text: false,\n};\n\npub(crate) static PROLOG: TypeInfo = TypeInfo {\n    label: \"prolog\",\n    mime_type: \"text/x-prolog\",\n    group: \"code\",\n    description: \"Prolog source\",\n    extensions: &[\"pl\", \"pro\", \"P\"],\n    is_text: true,\n};\n\npub(crate) static PROTEINDB: TypeInfo = TypeInfo {\n    label: \"proteindb\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"Protein DB\",\n    extensions: &[\"pdb\"],\n    is_text: true,\n};\n\npub(crate) static PROTO: TypeInfo = TypeInfo {\n    label: \"proto\",\n    mime_type: \"text/x-proto\",\n    group: \"code\",\n    description: \"Protocol buffer definition\",\n    extensions: &[\"proto\"],\n    is_text: true,\n};\n\npub(crate) static PSD: TypeInfo = TypeInfo {\n    label: \"psd\",\n    mime_type: \"image/vnd.adobe.photoshop\",\n    group: \"image\",\n    description: \"Adobe Photoshop\",\n    extensions: &[\"psd\"],\n    is_text: false,\n};\n\npub(crate) static PYTHON: TypeInfo = TypeInfo {\n    label: \"python\",\n    mime_type: \"text/x-python\",\n    group: \"code\",\n    description: \"Python source\",\n    extensions: &[\"py\", \"pyi\"],\n    is_text: true,\n};\n\npub(crate) static PYTHONBYTECODE: TypeInfo = TypeInfo {\n    label: \"pythonbytecode\",\n    mime_type: \"application/x-bytecode.python\",\n    group: \"executable\",\n    description: \"Python compiled bytecode\",\n    extensions: &[\"pyc\", \"pyo\"],\n    is_text: false,\n};\n\npub(crate) static PYTORCH: TypeInfo = TypeInfo {\n    label: \"pytorch\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"Pytorch storage file\",\n    extensions: &[\"pt\", \"pth\"],\n    is_text: false,\n};\n\npub(crate) static QT: TypeInfo = TypeInfo {\n    label: \"qt\",\n    mime_type: \"video/quicktime\",\n    group: \"video\",\n    description: \"QuickTime\",\n    extensions: &[\"mov\"],\n    is_text: false,\n};\n\npub(crate) static R: TypeInfo = TypeInfo {\n    label: \"r\",\n    mime_type: \"text/x-R\",\n    group: \"code\",\n    description: \"R (language)\",\n    extensions: &[\"R\"],\n    is_text: true,\n};\n\npub(crate) static RANDOMBYTES: TypeInfo = TypeInfo {\n    label: \"randombytes\",\n    mime_type: \"application/octet-stream\",\n    group: \"unknown\",\n    description: \"Random bytes\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static RANDOMTXT: TypeInfo = TypeInfo {\n    label: \"randomtxt\",\n    mime_type: \"text/plain\",\n    group: \"text\",\n    description: \"Random text\",\n    extensions: &[],\n    is_text: true,\n};\n\npub(crate) static RAR: TypeInfo = TypeInfo {\n    label: \"rar\",\n    mime_type: \"application/x-rar\",\n    group: \"archive\",\n    description: \"RAR archive data\",\n    extensions: &[\"rar\"],\n    is_text: false,\n};\n\npub(crate) static RDF: TypeInfo = TypeInfo {\n    label: \"rdf\",\n    mime_type: \"application/rdf+xml\",\n    group: \"text\",\n    description: \"Resource Description Framework document (RDF)\",\n    extensions: &[\"rdf\"],\n    is_text: true,\n};\n\npub(crate) static RPM: TypeInfo = TypeInfo {\n    label: \"rpm\",\n    mime_type: \"application/x-rpm\",\n    group: \"archive\",\n    description: \"RedHat Package Manager archive (RPM)\",\n    extensions: &[\"rpm\"],\n    is_text: false,\n};\n\npub(crate) static RST: TypeInfo = TypeInfo {\n    label: \"rst\",\n    mime_type: \"text/x-rst\",\n    group: \"text\",\n    description: \"ReStructuredText document\",\n    extensions: &[\"rst\"],\n    is_text: true,\n};\n\npub(crate) static RTF: TypeInfo = TypeInfo {\n    label: \"rtf\",\n    mime_type: \"text/rtf\",\n    group: \"text\",\n    description: \"Rich Text Format document\",\n    extensions: &[\"rtf\"],\n    is_text: true,\n};\n\npub(crate) static RUBY: TypeInfo = TypeInfo {\n    label: \"ruby\",\n    mime_type: \"application/x-ruby\",\n    group: \"code\",\n    description: \"Ruby source\",\n    extensions: &[\"rb\"],\n    is_text: true,\n};\n\npub(crate) static RUST: TypeInfo = TypeInfo {\n    label: \"rust\",\n    mime_type: \"application/x-rust\",\n    group: \"code\",\n    description: \"Rust source\",\n    extensions: &[\"rs\"],\n    is_text: true,\n};\n\npub(crate) static SCALA: TypeInfo = TypeInfo {\n    label: \"scala\",\n    mime_type: \"application/x-scala\",\n    group: \"code\",\n    description: \"Scala source\",\n    extensions: &[\"scala\"],\n    is_text: true,\n};\n\npub(crate) static SCSS: TypeInfo = TypeInfo {\n    label: \"scss\",\n    mime_type: \"text/x-scss\",\n    group: \"code\",\n    description: \"SCSS source\",\n    extensions: &[\"scss\"],\n    is_text: true,\n};\n\npub(crate) static SEVENZIP: TypeInfo = TypeInfo {\n    label: \"sevenzip\",\n    mime_type: \"application/x-7z-compressed\",\n    group: \"archive\",\n    description: \"7-zip archive data\",\n    extensions: &[\"7z\"],\n    is_text: false,\n};\n\npub(crate) static SGML: TypeInfo = TypeInfo {\n    label: \"sgml\",\n    mime_type: \"application/sgml\",\n    group: \"text\",\n    description: \"sgml\",\n    extensions: &[\"sgml\"],\n    is_text: true,\n};\n\npub(crate) static SHELL: TypeInfo = TypeInfo {\n    label: \"shell\",\n    mime_type: \"text/x-shellscript\",\n    group: \"code\",\n    description: \"Shell script\",\n    extensions: &[\"sh\"],\n    is_text: true,\n};\n\npub(crate) static SMALI: TypeInfo = TypeInfo {\n    label: \"smali\",\n    mime_type: \"application/x-smali\",\n    group: \"code\",\n    description: \"Smali source\",\n    extensions: &[\"smali\"],\n    is_text: true,\n};\n\npub(crate) static SNAP: TypeInfo = TypeInfo {\n    label: \"snap\",\n    mime_type: \"application/octet-stream\",\n    group: \"archive\",\n    description: \"Snap archive\",\n    extensions: &[\"snap\"],\n    is_text: false,\n};\n\npub(crate) static SOLIDITY: TypeInfo = TypeInfo {\n    label: \"solidity\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Solidity source\",\n    extensions: &[\"sol\"],\n    is_text: true,\n};\n\npub(crate) static SQL: TypeInfo = TypeInfo {\n    label: \"sql\",\n    mime_type: \"application/x-sql\",\n    group: \"code\",\n    description: \"SQL source\",\n    extensions: &[\"sql\"],\n    is_text: true,\n};\n\npub(crate) static SQLITE: TypeInfo = TypeInfo {\n    label: \"sqlite\",\n    mime_type: \"application/octet-stream\",\n    group: \"application\",\n    description: \"SQLITE database\",\n    extensions: &[\"sqlite\", \"sqlite3\"],\n    is_text: false,\n};\n\npub(crate) static SQUASHFS: TypeInfo = TypeInfo {\n    label: \"squashfs\",\n    mime_type: \"application/octet-stream\",\n    group: \"archive\",\n    description: \"Squash filesystem\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static SRT: TypeInfo = TypeInfo {\n    label: \"srt\",\n    mime_type: \"text/srt\",\n    group: \"application\",\n    description: \"SubRip Text Format\",\n    extensions: &[\"srt\"],\n    is_text: true,\n};\n\npub(crate) static STLBINARY: TypeInfo = TypeInfo {\n    label: \"stlbinary\",\n    mime_type: \"application/sla\",\n    group: \"image\",\n    description: \"Stereolithography CAD (binary)\",\n    extensions: &[\"stl\"],\n    is_text: false,\n};\n\npub(crate) static STLTEXT: TypeInfo = TypeInfo {\n    label: \"stltext\",\n    mime_type: \"application/sla\",\n    group: \"image\",\n    description: \"Stereolithography CAD (text)\",\n    extensions: &[\"stl\"],\n    is_text: true,\n};\n\npub(crate) static SUM: TypeInfo = TypeInfo {\n    label: \"sum\",\n    mime_type: \"text/plain\",\n    group: \"application\",\n    description: \"Checksum file\",\n    extensions: &[\"sum\"],\n    is_text: true,\n};\n\npub(crate) static SVG: TypeInfo = TypeInfo {\n    label: \"svg\",\n    mime_type: \"image/svg+xml\",\n    group: \"image\",\n    description: \"SVG Scalable Vector Graphics image data\",\n    extensions: &[\"svg\"],\n    is_text: true,\n};\n\npub(crate) static SWF: TypeInfo = TypeInfo {\n    label: \"swf\",\n    mime_type: \"application/x-shockwave-flash\",\n    group: \"executable\",\n    description: \"Small Web File\",\n    extensions: &[\"swf\"],\n    is_text: false,\n};\n\npub(crate) static SWIFT: TypeInfo = TypeInfo {\n    label: \"swift\",\n    mime_type: \"text/x-swift\",\n    group: \"code\",\n    description: \"Swift\",\n    extensions: &[\"swift\"],\n    is_text: true,\n};\n\npub(crate) static SYMLINK: TypeInfo = TypeInfo {\n    label: \"symlink\",\n    mime_type: \"inode/symlink\",\n    group: \"inode\",\n    description: \"Symbolic link\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static TAR: TypeInfo = TypeInfo {\n    label: \"tar\",\n    mime_type: \"application/x-tar\",\n    group: \"archive\",\n    description: \"POSIX tar archive\",\n    extensions: &[\"tar\"],\n    is_text: false,\n};\n\npub(crate) static TCL: TypeInfo = TypeInfo {\n    label: \"tcl\",\n    mime_type: \"application/x-tcl\",\n    group: \"code\",\n    description: \"Tickle\",\n    extensions: &[\"tcl\"],\n    is_text: true,\n};\n\npub(crate) static TEXTPROTO: TypeInfo = TypeInfo {\n    label: \"textproto\",\n    mime_type: \"text/plain\",\n    group: \"code\",\n    description: \"Text protocol buffer\",\n    extensions: &[\"textproto\", \"textpb\", \"pbtxt\"],\n    is_text: true,\n};\n\npub(crate) static TGA: TypeInfo = TypeInfo {\n    label: \"tga\",\n    mime_type: \"image/x-tga\",\n    group: \"image\",\n    description: \"Targa image data\",\n    extensions: &[\"tga\"],\n    is_text: false,\n};\n\npub(crate) static THUMBSDB: TypeInfo = TypeInfo {\n    label: \"thumbsdb\",\n    mime_type: \"image/vnd.ms-thumb\",\n    group: \"application\",\n    description: \"Windows thumbnail cache\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static TIFF: TypeInfo = TypeInfo {\n    label: \"tiff\",\n    mime_type: \"image/tiff\",\n    group: \"image\",\n    description: \"TIFF image data\",\n    extensions: &[\"tiff\", \"tif\"],\n    is_text: false,\n};\n\npub(crate) static TOML: TypeInfo = TypeInfo {\n    label: \"toml\",\n    mime_type: \"application/toml\",\n    group: \"text\",\n    description: \"Tom's obvious, minimal language\",\n    extensions: &[\"toml\"],\n    is_text: true,\n};\n\npub(crate) static TORRENT: TypeInfo = TypeInfo {\n    label: \"torrent\",\n    mime_type: \"application/x-bittorrent\",\n    group: \"application\",\n    description: \"BitTorrent file\",\n    extensions: &[\"torrent\"],\n    is_text: false,\n};\n\npub(crate) static TSV: TypeInfo = TypeInfo {\n    label: \"tsv\",\n    mime_type: \"text/tsv\",\n    group: \"code\",\n    description: \"TSV document\",\n    extensions: &[\"tsv\"],\n    is_text: true,\n};\n\npub(crate) static TTF: TypeInfo = TypeInfo {\n    label: \"ttf\",\n    mime_type: \"font/sfnt\",\n    group: \"font\",\n    description: \"TrueType Font data\",\n    extensions: &[\"ttf\", \"ttc\"],\n    is_text: false,\n};\n\npub(crate) static TWIG: TypeInfo = TypeInfo {\n    label: \"twig\",\n    mime_type: \"text/x-twig\",\n    group: \"code\",\n    description: \"Twig template\",\n    extensions: &[\"twig\"],\n    is_text: true,\n};\n\npub(crate) static TXT: TypeInfo = TypeInfo {\n    label: \"txt\",\n    mime_type: \"text/plain\",\n    group: \"text\",\n    description: \"Generic text document\",\n    extensions: &[\"txt\"],\n    is_text: true,\n};\n\npub(crate) static TYPESCRIPT: TypeInfo = TypeInfo {\n    label: \"typescript\",\n    mime_type: \"application/typescript\",\n    group: \"code\",\n    description: \"TypeScript source\",\n    extensions: &[\"ts\", \"mts\", \"cts\"],\n    is_text: true,\n};\n\npub(crate) static UNDEFINED: TypeInfo = TypeInfo {\n    label: \"undefined\",\n    mime_type: \"application/undefined\",\n    group: \"undefined\",\n    description: \"Undefined\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static UNKNOWN: TypeInfo = TypeInfo {\n    label: \"unknown\",\n    mime_type: \"application/octet-stream\",\n    group: \"unknown\",\n    description: \"Unknown binary data\",\n    extensions: &[],\n    is_text: false,\n};\n\npub(crate) static VBA: TypeInfo = TypeInfo {\n    label: \"vba\",\n    mime_type: \"text/vbscript\",\n    group: \"code\",\n    description: \"MS Visual Basic source (VBA)\",\n    extensions: &[\"vbs\", \"vba\", \"vb\"],\n    is_text: true,\n};\n\npub(crate) static VCXPROJ: TypeInfo = TypeInfo {\n    label: \"vcxproj\",\n    mime_type: \"application/xml\",\n    group: \"code\",\n    description: \"Visual Studio MSBuild project\",\n    extensions: &[\"vcxproj\"],\n    is_text: true,\n};\n\npub(crate) static VERILOG: TypeInfo = TypeInfo {\n    label: \"verilog\",\n    mime_type: \"text/x-verilog\",\n    group: \"code\",\n    description: \"Verilog source\",\n    extensions: &[\"v\", \"verilog\", \"vlg\", \"vh\"],\n    is_text: true,\n};\n\npub(crate) static VHDL: TypeInfo = TypeInfo {\n    label: \"vhdl\",\n    mime_type: \"text/x-vhdl\",\n    group: \"code\",\n    description: \"VHDL source\",\n    extensions: &[\"vhd\"],\n    is_text: true,\n};\n\npub(crate) static VTT: TypeInfo = TypeInfo {\n    label: \"vtt\",\n    mime_type: \"text/vtt\",\n    group: \"text\",\n    description: \"Web Video Text Tracks\",\n    extensions: &[\"vtt\", \"webvtt\"],\n    is_text: true,\n};\n\npub(crate) static VUE: TypeInfo = TypeInfo {\n    label: \"vue\",\n    mime_type: \"application/javascript\",\n    group: \"code\",\n    description: \"Vue source\",\n    extensions: &[\"vue\"],\n    is_text: true,\n};\n\npub(crate) static WASM: TypeInfo = TypeInfo {\n    label: \"wasm\",\n    mime_type: \"application/wasm\",\n    group: \"executable\",\n    description: \"Web Assembly\",\n    extensions: &[\"wasm\"],\n    is_text: false,\n};\n\npub(crate) static WAV: TypeInfo = TypeInfo {\n    label: \"wav\",\n    mime_type: \"audio/x-wav\",\n    group: \"audio\",\n    description: \"Waveform Audio file (WAV)\",\n    extensions: &[\"wav\"],\n    is_text: false,\n};\n\npub(crate) static WEBM: TypeInfo = TypeInfo {\n    label: \"webm\",\n    mime_type: \"video/webm\",\n    group: \"video\",\n    description: \"WebM media file\",\n    extensions: &[\"webm\"],\n    is_text: false,\n};\n\npub(crate) static WEBP: TypeInfo = TypeInfo {\n    label: \"webp\",\n    mime_type: \"image/webp\",\n    group: \"image\",\n    description: \"WebP media file\",\n    extensions: &[\"webp\"],\n    is_text: false,\n};\n\npub(crate) static WINREGISTRY: TypeInfo = TypeInfo {\n    label: \"winregistry\",\n    mime_type: \"text/x-ms-regedit\",\n    group: \"application\",\n    description: \"Windows Registry text\",\n    extensions: &[\"reg\"],\n    is_text: true,\n};\n\npub(crate) static WMF: TypeInfo = TypeInfo {\n    label: \"wmf\",\n    mime_type: \"image/wmf\",\n    group: \"image\",\n    description: \"Windows metafile\",\n    extensions: &[\"wmf\"],\n    is_text: false,\n};\n\npub(crate) static WOFF: TypeInfo = TypeInfo {\n    label: \"woff\",\n    mime_type: \"font/woff\",\n    group: \"font\",\n    description: \"Web Open Font Format\",\n    extensions: &[\"woff\"],\n    is_text: false,\n};\n\npub(crate) static WOFF2: TypeInfo = TypeInfo {\n    label: \"woff2\",\n    mime_type: \"font/woff2\",\n    group: \"font\",\n    description: \"Web Open Font Format v2\",\n    extensions: &[\"woff2\"],\n    is_text: false,\n};\n\npub(crate) static XAR: TypeInfo = TypeInfo {\n    label: \"xar\",\n    mime_type: \"application/x-xar\",\n    group: \"archive\",\n    description: \"XAR archive compressed data\",\n    extensions: &[\"pkg\", \"xar\"],\n    is_text: false,\n};\n\npub(crate) static XLS: TypeInfo = TypeInfo {\n    label: \"xls\",\n    mime_type: \"application/vnd.ms-excel\",\n    group: \"document\",\n    description: \"Microsoft Excel CDF document\",\n    extensions: &[\"xls\"],\n    is_text: false,\n};\n\npub(crate) static XLSB: TypeInfo = TypeInfo {\n    label: \"xlsb\",\n    mime_type: \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\n    group: \"document\",\n    description: \"Microsoft Excel 2007+ document (binary format)\",\n    extensions: &[\"xlsb\"],\n    is_text: false,\n};\n\npub(crate) static XLSX: TypeInfo = TypeInfo {\n    label: \"xlsx\",\n    mime_type: \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\n    group: \"document\",\n    description: \"Microsoft Excel 2007+ document\",\n    extensions: &[\"xlsx\", \"xlsm\"],\n    is_text: false,\n};\n\npub(crate) static XML: TypeInfo = TypeInfo {\n    label: \"xml\",\n    mime_type: \"text/xml\",\n    group: \"code\",\n    description: \"XML document\",\n    extensions: &[\"xml\"],\n    is_text: true,\n};\n\npub(crate) static XPI: TypeInfo = TypeInfo {\n    label: \"xpi\",\n    mime_type: \"application/zip\",\n    group: \"archive\",\n    description: \"Compressed installation archive (XPI)\",\n    extensions: &[\"xpi\"],\n    is_text: false,\n};\n\npub(crate) static XZ: TypeInfo = TypeInfo {\n    label: \"xz\",\n    mime_type: \"application/x-xz\",\n    group: \"archive\",\n    description: \"XZ compressed data\",\n    extensions: &[\"xz\"],\n    is_text: false,\n};\n\npub(crate) static YAML: TypeInfo = TypeInfo {\n    label: \"yaml\",\n    mime_type: \"application/x-yaml\",\n    group: \"code\",\n    description: \"YAML source\",\n    extensions: &[\"yml\", \"yaml\"],\n    is_text: true,\n};\n\npub(crate) static YARA: TypeInfo = TypeInfo {\n    label: \"yara\",\n    mime_type: \"text/x-yara\",\n    group: \"code\",\n    description: \"YARA rule\",\n    extensions: &[\"yar\", \"yara\"],\n    is_text: true,\n};\n\npub(crate) static ZIG: TypeInfo = TypeInfo {\n    label: \"zig\",\n    mime_type: \"text/zig\",\n    group: \"code\",\n    description: \"Zig source\",\n    extensions: &[\"zig\"],\n    is_text: true,\n};\n\npub(crate) static ZIP: TypeInfo = TypeInfo {\n    label: \"zip\",\n    mime_type: \"application/zip\",\n    group: \"archive\",\n    description: \"Zip archive data\",\n    extensions: &[\"zip\"],\n    is_text: false,\n};\n\npub(crate) static ZLIBSTREAM: TypeInfo = TypeInfo {\n    label: \"zlibstream\",\n    mime_type: \"application/zlib\",\n    group: \"application\",\n    description: \"zlib compressed data\",\n    extensions: &[],\n    is_text: false,\n};\n\n/// Content types for regular files.\n#[derive(Debug, Copy, Clone, PartialEq, Eq)]\n#[non_exhaustive]\npub enum ContentType {\n    /// 3GPP multimedia file\n    _3gp,\n    /// ACE archive\n    Ace,\n    /// Adobe Illustrator Artwork\n    Ai,\n    /// Android Interface Definition Language\n    Aidl,\n    /// Android package\n    Apk,\n    /// Apple binary property list\n    Applebplist,\n    /// Apple property list\n    Appleplist,\n    /// Assembly\n    Asm,\n    /// ASP source\n    Asp,\n    /// AutoHotKey script\n    Autohotkey,\n    /// AutoIt script\n    Autoit,\n    /// Awk\n    Awk,\n    /// DOS batch file\n    Batch,\n    /// Bazel build file\n    Bazel,\n    /// BibTeX\n    Bib,\n    /// BMP image data\n    Bmp,\n    /// bzip2 compressed data\n    Bzip,\n    /// C source\n    C,\n    /// Microsoft Cabinet archive data\n    Cab,\n    /// Windows Catalog file\n    Cat,\n    /// MS Windows HtmlHelp Data\n    Chm,\n    /// Clojure\n    Clojure,\n    /// CMake build file\n    Cmake,\n    /// Cobol\n    Cobol,\n    /// Intel 80386 COFF\n    Coff,\n    /// CoffeeScript\n    Coffeescript,\n    /// C++ source\n    Cpp,\n    /// Certificates (binary format)\n    Crt,\n    /// Google Chrome extension\n    Crx,\n    /// C# source\n    Cs,\n    /// .NET project config\n    Csproj,\n    /// CSS source\n    Css,\n    /// CSV document\n    Csv,\n    /// Dart source\n    Dart,\n    /// Debian binary package\n    Deb,\n    /// Dalvik dex file\n    Dex,\n    /// DICOM\n    Dicom,\n    /// Diff file\n    Diff,\n    /// Dream Maker\n    Dm,\n    /// Apple disk image\n    Dmg,\n    /// Microsoft Word CDF document\n    Doc,\n    /// Dockerfile\n    Dockerfile,\n    /// Microsoft Word 2007+ document\n    Docx,\n    /// Application Desktop Services Store\n    Dsstore,\n    /// Autocad Drawing\n    Dwg,\n    /// Audocad Drawing Exchange Format\n    Dxf,\n    /// ELF executable\n    Elf,\n    /// Elixir script\n    Elixir,\n    /// Windows Enhanced Metafile image data\n    Emf,\n    /// RFC 822 mail\n    Eml,\n    /// Empty file\n    Empty,\n    /// EPUB document\n    Epub,\n    /// Embedded Ruby source\n    Erb,\n    /// Erlang source\n    Erlang,\n    /// FLAC audio bitstream data\n    Flac,\n    /// Flash Video\n    Flv,\n    /// Fortran\n    Fortran,\n    /// Gemfile file\n    Gemfile,\n    /// Gemspec file\n    Gemspec,\n    /// GIF image data\n    Gif,\n    /// Gitattributes file\n    Gitattributes,\n    /// Gitmodules file\n    Gitmodules,\n    /// Golang source\n    Go,\n    /// Gradle source\n    Gradle,\n    /// Groovy source\n    Groovy,\n    /// gzip compressed data\n    Gzip,\n    /// Hierarchical Data Format v5\n    H5,\n    /// Handlebars source\n    Handlebars,\n    /// Haskell source\n    Haskell,\n    /// HashiCorp configuration language\n    Hcl,\n    /// MS Windows help\n    Hlp,\n    /// Apache access configuration\n    Htaccess,\n    /// HTML document\n    Html,\n    /// Mac OS X icon\n    Icns,\n    /// MS Windows icon resource\n    Ico,\n    /// Internet Calendaring and Scheduling\n    Ics,\n    /// Ignorefile\n    Ignorefile,\n    /// INI configuration file\n    Ini,\n    /// MS Windows Internet shortcut\n    Internetshortcut,\n    /// Jupyter notebook\n    Ipynb,\n    /// ISO 9660 CD-ROM filesystem data\n    Iso,\n    /// Java archive data (JAR)\n    Jar,\n    /// Java source\n    Java,\n    /// Java compiled bytecode\n    Javabytecode,\n    /// JavaScript source\n    Javascript,\n    /// Jinja template\n    Jinja,\n    /// jpeg2000\n    Jp2,\n    /// JPEG image data\n    Jpeg,\n    /// JSON document\n    Json,\n    /// JSONL document\n    Jsonl,\n    /// Julia source\n    Julia,\n    /// Kotlin source\n    Kotlin,\n    /// LaTeX document\n    Latex,\n    /// LHarc archive\n    Lha,\n    /// Lisp source\n    Lisp,\n    /// MS Windows shortcut\n    Lnk,\n    /// Lua\n    Lua,\n    /// M3U playlist\n    M3u,\n    /// GNU Macro\n    M4,\n    /// Mach-O executable\n    Macho,\n    /// Makefile source\n    Makefile,\n    /// Markdown document\n    Markdown,\n    /// Matlab Source\n    Matlab,\n    /// MHTML document\n    Mht,\n    /// Midi\n    Midi,\n    /// Matroska\n    Mkv,\n    /// MP3 media file\n    Mp3,\n    /// MP4 media file\n    Mp4,\n    /// MS Compress archive data\n    Mscompress,\n    /// Microsoft Installer file\n    Msi,\n    /// Windows Update Package file\n    Mum,\n    /// Numpy Array\n    Npy,\n    /// Numpy Arrays Archive\n    Npz,\n    /// NuGet Package\n    Nupkg,\n    /// ObjectiveC source\n    Objectivec,\n    /// OCaml\n    Ocaml,\n    /// OpenDocument Presentation\n    Odp,\n    /// OpenDocument Spreadsheet\n    Ods,\n    /// OpenDocument Text\n    Odt,\n    /// Ogg data\n    Ogg,\n    /// One Note\n    One,\n    /// Open Neural Network Exchange\n    Onnx,\n    /// OpenType font\n    Otf,\n    /// MS Outlook Message\n    Outlook,\n    /// Apache Parquet\n    Parquet,\n    /// Pascal source\n    Pascal,\n    /// pcap capture file\n    Pcap,\n    /// Windows Program Database\n    Pdb,\n    /// PDF document\n    Pdf,\n    /// PE Windows executable\n    Pebin,\n    /// PEM certificate\n    Pem,\n    /// Perl source\n    Perl,\n    /// PHP source\n    Php,\n    /// Python pickle\n    Pickle,\n    /// PNG image\n    Png,\n    /// Portable Object (PO) for i18n\n    Po,\n    /// PostScript document\n    Postscript,\n    /// Powershell source\n    Powershell,\n    /// Microsoft PowerPoint CDF document\n    Ppt,\n    /// Microsoft PowerPoint 2007+ document\n    Pptx,\n    /// Prolog source\n    Prolog,\n    /// Protein DB\n    Proteindb,\n    /// Protocol buffer definition\n    Proto,\n    /// Adobe Photoshop\n    Psd,\n    /// Python source\n    Python,\n    /// Python compiled bytecode\n    Pythonbytecode,\n    /// Pytorch storage file\n    Pytorch,\n    /// QuickTime\n    Qt,\n    /// R (language)\n    R,\n    /// Random bytes\n    Randombytes,\n    /// Random text\n    Randomtxt,\n    /// RAR archive data\n    Rar,\n    /// Resource Description Framework document (RDF)\n    Rdf,\n    /// RedHat Package Manager archive (RPM)\n    Rpm,\n    /// ReStructuredText document\n    Rst,\n    /// Rich Text Format document\n    Rtf,\n    /// Ruby source\n    Ruby,\n    /// Rust source\n    Rust,\n    /// Scala source\n    Scala,\n    /// SCSS source\n    Scss,\n    /// 7-zip archive data\n    Sevenzip,\n    /// sgml\n    Sgml,\n    /// Shell script\n    Shell,\n    /// Smali source\n    Smali,\n    /// Snap archive\n    Snap,\n    /// Solidity source\n    Solidity,\n    /// SQL source\n    Sql,\n    /// SQLITE database\n    Sqlite,\n    /// Squash filesystem\n    Squashfs,\n    /// SubRip Text Format\n    Srt,\n    /// Stereolithography CAD (binary)\n    Stlbinary,\n    /// Stereolithography CAD (text)\n    Stltext,\n    /// Checksum file\n    Sum,\n    /// SVG Scalable Vector Graphics image data\n    Svg,\n    /// Small Web File\n    Swf,\n    /// Swift\n    Swift,\n    /// POSIX tar archive\n    Tar,\n    /// Tickle\n    Tcl,\n    /// Text protocol buffer\n    Textproto,\n    /// Targa image data\n    Tga,\n    /// Windows thumbnail cache\n    Thumbsdb,\n    /// TIFF image data\n    Tiff,\n    /// Tom's obvious, minimal language\n    Toml,\n    /// BitTorrent file\n    Torrent,\n    /// TSV document\n    Tsv,\n    /// TrueType Font data\n    Ttf,\n    /// Twig template\n    Twig,\n    /// Generic text document\n    Txt,\n    /// TypeScript source\n    Typescript,\n    /// Undefined\n    Undefined,\n    /// Unknown binary data\n    Unknown,\n    /// MS Visual Basic source (VBA)\n    Vba,\n    /// Visual Studio MSBuild project\n    Vcxproj,\n    /// Verilog source\n    Verilog,\n    /// VHDL source\n    Vhdl,\n    /// Web Video Text Tracks\n    Vtt,\n    /// Vue source\n    Vue,\n    /// Web Assembly\n    Wasm,\n    /// Waveform Audio file (WAV)\n    Wav,\n    /// WebM media file\n    Webm,\n    /// WebP media file\n    Webp,\n    /// Windows Registry text\n    Winregistry,\n    /// Windows metafile\n    Wmf,\n    /// Web Open Font Format\n    Woff,\n    /// Web Open Font Format v2\n    Woff2,\n    /// XAR archive compressed data\n    Xar,\n    /// Microsoft Excel CDF document\n    Xls,\n    /// Microsoft Excel 2007+ document (binary format)\n    Xlsb,\n    /// Microsoft Excel 2007+ document\n    Xlsx,\n    /// XML document\n    Xml,\n    /// Compressed installation archive (XPI)\n    Xpi,\n    /// XZ compressed data\n    Xz,\n    /// YAML source\n    Yaml,\n    /// YARA rule\n    Yara,\n    /// Zig source\n    Zig,\n    /// Zip archive data\n    Zip,\n    /// zlib compressed data\n    Zlibstream,\n}\n\nimpl ContentType {\n    pub(crate) const SIZE: usize = 217;\n\n    /// Returns the content type information.\n    pub fn info(self) -> &'static TypeInfo {\n        match self {\n            ContentType::_3gp => &_3GP,\n            ContentType::Ace => &ACE,\n            ContentType::Ai => &AI,\n            ContentType::Aidl => &AIDL,\n            ContentType::Apk => &APK,\n            ContentType::Applebplist => &APPLEBPLIST,\n            ContentType::Appleplist => &APPLEPLIST,\n            ContentType::Asm => &ASM,\n            ContentType::Asp => &ASP,\n            ContentType::Autohotkey => &AUTOHOTKEY,\n            ContentType::Autoit => &AUTOIT,\n            ContentType::Awk => &AWK,\n            ContentType::Batch => &BATCH,\n            ContentType::Bazel => &BAZEL,\n            ContentType::Bib => &BIB,\n            ContentType::Bmp => &BMP,\n            ContentType::Bzip => &BZIP,\n            ContentType::C => &C,\n            ContentType::Cab => &CAB,\n            ContentType::Cat => &CAT,\n            ContentType::Chm => &CHM,\n            ContentType::Clojure => &CLOJURE,\n            ContentType::Cmake => &CMAKE,\n            ContentType::Cobol => &COBOL,\n            ContentType::Coff => &COFF,\n            ContentType::Coffeescript => &COFFEESCRIPT,\n            ContentType::Cpp => &CPP,\n            ContentType::Crt => &CRT,\n            ContentType::Crx => &CRX,\n            ContentType::Cs => &CS,\n            ContentType::Csproj => &CSPROJ,\n            ContentType::Css => &CSS,\n            ContentType::Csv => &CSV,\n            ContentType::Dart => &DART,\n            ContentType::Deb => &DEB,\n            ContentType::Dex => &DEX,\n            ContentType::Dicom => &DICOM,\n            ContentType::Diff => &DIFF,\n            ContentType::Dm => &DM,\n            ContentType::Dmg => &DMG,\n            ContentType::Doc => &DOC,\n            ContentType::Dockerfile => &DOCKERFILE,\n            ContentType::Docx => &DOCX,\n            ContentType::Dsstore => &DSSTORE,\n            ContentType::Dwg => &DWG,\n            ContentType::Dxf => &DXF,\n            ContentType::Elf => &ELF,\n            ContentType::Elixir => &ELIXIR,\n            ContentType::Emf => &EMF,\n            ContentType::Eml => &EML,\n            ContentType::Empty => &EMPTY,\n            ContentType::Epub => &EPUB,\n            ContentType::Erb => &ERB,\n            ContentType::Erlang => &ERLANG,\n            ContentType::Flac => &FLAC,\n            ContentType::Flv => &FLV,\n            ContentType::Fortran => &FORTRAN,\n            ContentType::Gemfile => &GEMFILE,\n            ContentType::Gemspec => &GEMSPEC,\n            ContentType::Gif => &GIF,\n            ContentType::Gitattributes => &GITATTRIBUTES,\n            ContentType::Gitmodules => &GITMODULES,\n            ContentType::Go => &GO,\n            ContentType::Gradle => &GRADLE,\n            ContentType::Groovy => &GROOVY,\n            ContentType::Gzip => &GZIP,\n            ContentType::H5 => &H5,\n            ContentType::Handlebars => &HANDLEBARS,\n            ContentType::Haskell => &HASKELL,\n            ContentType::Hcl => &HCL,\n            ContentType::Hlp => &HLP,\n            ContentType::Htaccess => &HTACCESS,\n            ContentType::Html => &HTML,\n            ContentType::Icns => &ICNS,\n            ContentType::Ico => &ICO,\n            ContentType::Ics => &ICS,\n            ContentType::Ignorefile => &IGNOREFILE,\n            ContentType::Ini => &INI,\n            ContentType::Internetshortcut => &INTERNETSHORTCUT,\n            ContentType::Ipynb => &IPYNB,\n            ContentType::Iso => &ISO,\n            ContentType::Jar => &JAR,\n            ContentType::Java => &JAVA,\n            ContentType::Javabytecode => &JAVABYTECODE,\n            ContentType::Javascript => &JAVASCRIPT,\n            ContentType::Jinja => &JINJA,\n            ContentType::Jp2 => &JP2,\n            ContentType::Jpeg => &JPEG,\n            ContentType::Json => &JSON,\n            ContentType::Jsonl => &JSONL,\n            ContentType::Julia => &JULIA,\n            ContentType::Kotlin => &KOTLIN,\n            ContentType::Latex => &LATEX,\n            ContentType::Lha => &LHA,\n            ContentType::Lisp => &LISP,\n            ContentType::Lnk => &LNK,\n            ContentType::Lua => &LUA,\n            ContentType::M3u => &M3U,\n            ContentType::M4 => &M4,\n            ContentType::Macho => &MACHO,\n            ContentType::Makefile => &MAKEFILE,\n            ContentType::Markdown => &MARKDOWN,\n            ContentType::Matlab => &MATLAB,\n            ContentType::Mht => &MHT,\n            ContentType::Midi => &MIDI,\n            ContentType::Mkv => &MKV,\n            ContentType::Mp3 => &MP3,\n            ContentType::Mp4 => &MP4,\n            ContentType::Mscompress => &MSCOMPRESS,\n            ContentType::Msi => &MSI,\n            ContentType::Mum => &MUM,\n            ContentType::Npy => &NPY,\n            ContentType::Npz => &NPZ,\n            ContentType::Nupkg => &NUPKG,\n            ContentType::Objectivec => &OBJECTIVEC,\n            ContentType::Ocaml => &OCAML,\n            ContentType::Odp => &ODP,\n            ContentType::Ods => &ODS,\n            ContentType::Odt => &ODT,\n            ContentType::Ogg => &OGG,\n            ContentType::One => &ONE,\n            ContentType::Onnx => &ONNX,\n            ContentType::Otf => &OTF,\n            ContentType::Outlook => &OUTLOOK,\n            ContentType::Parquet => &PARQUET,\n            ContentType::Pascal => &PASCAL,\n            ContentType::Pcap => &PCAP,\n            ContentType::Pdb => &PDB,\n            ContentType::Pdf => &PDF,\n            ContentType::Pebin => &PEBIN,\n            ContentType::Pem => &PEM,\n            ContentType::Perl => &PERL,\n            ContentType::Php => &PHP,\n            ContentType::Pickle => &PICKLE,\n            ContentType::Png => &PNG,\n            ContentType::Po => &PO,\n            ContentType::Postscript => &POSTSCRIPT,\n            ContentType::Powershell => &POWERSHELL,\n            ContentType::Ppt => &PPT,\n            ContentType::Pptx => &PPTX,\n            ContentType::Prolog => &PROLOG,\n            ContentType::Proteindb => &PROTEINDB,\n            ContentType::Proto => &PROTO,\n            ContentType::Psd => &PSD,\n            ContentType::Python => &PYTHON,\n            ContentType::Pythonbytecode => &PYTHONBYTECODE,\n            ContentType::Pytorch => &PYTORCH,\n            ContentType::Qt => &QT,\n            ContentType::R => &R,\n            ContentType::Randombytes => &RANDOMBYTES,\n            ContentType::Randomtxt => &RANDOMTXT,\n            ContentType::Rar => &RAR,\n            ContentType::Rdf => &RDF,\n            ContentType::Rpm => &RPM,\n            ContentType::Rst => &RST,\n            ContentType::Rtf => &RTF,\n            ContentType::Ruby => &RUBY,\n            ContentType::Rust => &RUST,\n            ContentType::Scala => &SCALA,\n            ContentType::Scss => &SCSS,\n            ContentType::Sevenzip => &SEVENZIP,\n            ContentType::Sgml => &SGML,\n            ContentType::Shell => &SHELL,\n            ContentType::Smali => &SMALI,\n            ContentType::Snap => &SNAP,\n            ContentType::Solidity => &SOLIDITY,\n            ContentType::Sql => &SQL,\n            ContentType::Sqlite => &SQLITE,\n            ContentType::Squashfs => &SQUASHFS,\n            ContentType::Srt => &SRT,\n            ContentType::Stlbinary => &STLBINARY,\n            ContentType::Stltext => &STLTEXT,\n            ContentType::Sum => &SUM,\n            ContentType::Svg => &SVG,\n            ContentType::Swf => &SWF,\n            ContentType::Swift => &SWIFT,\n            ContentType::Tar => &TAR,\n            ContentType::Tcl => &TCL,\n            ContentType::Textproto => &TEXTPROTO,\n            ContentType::Tga => &TGA,\n            ContentType::Thumbsdb => &THUMBSDB,\n            ContentType::Tiff => &TIFF,\n            ContentType::Toml => &TOML,\n            ContentType::Torrent => &TORRENT,\n            ContentType::Tsv => &TSV,\n            ContentType::Ttf => &TTF,\n            ContentType::Twig => &TWIG,\n            ContentType::Txt => &TXT,\n            ContentType::Typescript => &TYPESCRIPT,\n            ContentType::Undefined => &UNDEFINED,\n            ContentType::Unknown => &UNKNOWN,\n            ContentType::Vba => &VBA,\n            ContentType::Vcxproj => &VCXPROJ,\n            ContentType::Verilog => &VERILOG,\n            ContentType::Vhdl => &VHDL,\n            ContentType::Vtt => &VTT,\n            ContentType::Vue => &VUE,\n            ContentType::Wasm => &WASM,\n            ContentType::Wav => &WAV,\n            ContentType::Webm => &WEBM,\n            ContentType::Webp => &WEBP,\n            ContentType::Winregistry => &WINREGISTRY,\n            ContentType::Wmf => &WMF,\n            ContentType::Woff => &WOFF,\n            ContentType::Woff2 => &WOFF2,\n            ContentType::Xar => &XAR,\n            ContentType::Xls => &XLS,\n            ContentType::Xlsb => &XLSB,\n            ContentType::Xlsx => &XLSX,\n            ContentType::Xml => &XML,\n            ContentType::Xpi => &XPI,\n            ContentType::Xz => &XZ,\n            ContentType::Yaml => &YAML,\n            ContentType::Yara => &YARA,\n            ContentType::Zig => &ZIG,\n            ContentType::Zip => &ZIP,\n            ContentType::Zlibstream => &ZLIBSTREAM,\n        }\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/error.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n/// Result type of Magika functions.\npub type Result = core::result::Result;\n\n/// Errors returned by Magika functions.\n#[derive(Debug, thiserror::Error)]\npub enum Error {\n    /// Input/output errors reported by the standard library.\n    #[error(\"{0}\")]\n    IOError(#[from] std::io::Error),\n\n    /// Errors reported by the ONNX Runtime.\n    #[error(\"{0}\")]\n    OrtError(#[from] ort::Error),\n\n    /// Shape errors reported by the ndarray library.\n    #[error(\"{0}\")]\n    ShapeError(#[from] ndarray::ShapeError),\n}\n"
  },
  {
    "path": "rust/lib/src/file.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse ndarray::ArrayViewD;\n\nuse crate::model::Label;\nuse crate::ContentType;\n\n/// File types.\n///\n/// The word file is used in the Linux sense where everything is a file. This could be equivalently\n/// understood as a path.\n#[derive(Debug, Clone)]\npub enum FileType {\n    /// The file is a directory.\n    Directory,\n\n    /// The file is a symbolic link.\n    Symlink,\n\n    /// The file is a regular file and was identified using AI.\n    Inferred(InferredType),\n\n    /// The file is a regular file and was identified using rules.\n    Ruled(ContentType),\n}\n\n/// Content type identified using AI.\n#[derive(Debug, Clone)]\npub struct InferredType {\n    /// The content type.\n    ///\n    /// The inferred content type may be overwritten for a variety of reasons. Use\n    /// [`Self::content_type()`] to access the final content type (after possible overwrite).\n    pub content_type: Option<(ContentType, OverwriteReason)>,\n\n    /// The inferred content type.\n    pub inferred_type: ContentType,\n\n    /// The inference score between 0 and 1.\n    pub score: f32,\n}\n\n/// Reason to overwrite an inferred content type.\n#[derive(Debug, Clone)]\npub enum OverwriteReason {\n    /// The inference score is too low for the inferred content type.\n    LowConfidence,\n\n    /// The inferred content type is not canonical.\n    OverwriteMap,\n}\n\nimpl FileType {\n    /// Returns the content type for regular files.\n    pub fn content_type(&self) -> Option {\n        match self {\n            FileType::Directory => None,\n            FileType::Symlink => None,\n            FileType::Inferred(x) => Some(x.content_type()),\n            FileType::Ruled(x) => Some(*x),\n        }\n    }\n\n    /// Returns the file type information.\n    pub fn info(&self) -> &'static TypeInfo {\n        match self {\n            FileType::Directory => &crate::content::DIRECTORY,\n            FileType::Symlink => &crate::content::SYMLINK,\n            FileType::Inferred(x) => x.content_type().info(),\n            FileType::Ruled(x) => x.info(),\n        }\n    }\n\n    /// Returns the score of the identification, between 0 and 1.\n    ///\n    /// If the model was run, this is the model score. Otherwise this is 1.\n    pub fn score(&self) -> f32 {\n        match self {\n            FileType::Directory => 1.0,\n            FileType::Symlink => 1.0,\n            FileType::Inferred(x) => x.score,\n            FileType::Ruled(_) => 1.0,\n        }\n    }\n}\n\nimpl InferredType {\n    /// Returns the content type.\n    pub fn content_type(&self) -> ContentType {\n        match self.content_type {\n            Some((x, _)) => x,\n            None => self.inferred_type,\n        }\n    }\n}\n\n/// File type information.\n#[cfg_attr(feature = \"serde\", derive(serde::Serialize))]\npub struct TypeInfo {\n    /// The unique label identifying this file type.\n    pub label: &'static str,\n\n    /// The MIME type of the file type.\n    pub mime_type: &'static str,\n\n    /// The group of the file type.\n    pub group: &'static str,\n\n    /// The description of the file type.\n    pub description: &'static str,\n\n    /// Possible extensions for the file type.\n    pub extensions: &'static [&'static str],\n\n    /// Whether the file type is text.\n    pub is_text: bool,\n}\n\nimpl FileType {\n    pub(crate) fn convert(tensor: ArrayViewD) -> Vec {\n        let mut results = Vec::new();\n        for view in tensor.view().axis_iter(ndarray::Axis(0)) {\n            let scores = view.to_slice().unwrap();\n            let mut best = 0;\n            for (i, &x) in scores.iter().enumerate() {\n                if scores[best].max(x) == x {\n                    best = i;\n                }\n            }\n            assert!(best < crate::model::NUM_LABELS);\n            let score = scores[best];\n            // SAFETY: Labels are u32 smaller than NUM_LABELS.\n            let label = unsafe { std::mem::transmute::(best as u32) };\n            let inferred_type = label.content_type();\n            let config = &crate::model::CONFIG;\n            let mut content_type = if score < config.thresholds[inferred_type as usize] {\n                let is_text = inferred_type.info().is_text;\n                Some((\n                    if is_text { ContentType::Txt } else { ContentType::Unknown },\n                    OverwriteReason::LowConfidence,\n                ))\n            } else {\n                let overwrite = config.overwrite_map[inferred_type as usize];\n                (overwrite != inferred_type).then_some((overwrite, OverwriteReason::OverwriteMap))\n            };\n            if content_type.as_ref().is_some_and(|(x, _)| *x == inferred_type) {\n                content_type = None;\n            }\n            results.push(FileType::Inferred(InferredType { content_type, inferred_type, score }));\n        }\n        results\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/future.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse std::fs::Metadata;\nuse std::future::Future;\nuse std::path::Path;\nuse std::pin::Pin;\nuse std::sync::OnceLock;\nuse std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker};\n\nuse ndarray::Array2;\nuse ort::session::{NoSelectedOutputs, RunOptions};\nuse ort::value::Tensor;\n\nuse crate::{AsyncInput, Result};\n\npub(crate) fn exec(mut future: impl Future) -> T {\n    let future = unsafe { Pin::new_unchecked(&mut future) };\n    let waker = panic_waker();\n    let mut context = Context::from_waker(&waker);\n    match future.poll(&mut context) {\n        Poll::Ready(x) => x,\n        Poll::Pending => unreachable!(),\n    }\n}\n\npub(crate) trait Env {\n    type File: AsyncInput;\n    async fn symlink_metadata(path: &Path) -> Result;\n    async fn open(path: &Path) -> Result;\n    async fn ort_session_run(\n        session: &mut ort::session::Session, input: Array2,\n    ) -> Result>;\n}\n\npub(crate) enum SyncEnv {}\nimpl Env for SyncEnv {\n    type File = std::fs::File;\n\n    async fn symlink_metadata(path: &Path) -> Result {\n        Ok(std::fs::symlink_metadata(path)?)\n    }\n\n    async fn open(path: &Path) -> Result {\n        Ok(std::fs::File::open(path)?)\n    }\n\n    async fn ort_session_run(\n        session: &mut ort::session::Session, input: Array2,\n    ) -> Result> {\n        Ok(session.run(ort::inputs!(\"bytes\" => Tensor::from_array(input)?))?)\n    }\n}\n\npub(crate) enum AsyncEnv {}\nimpl Env for AsyncEnv {\n    type File = tokio::fs::File;\n\n    async fn symlink_metadata(path: &Path) -> Result {\n        Ok(tokio::fs::symlink_metadata(path).await?)\n    }\n\n    async fn open(path: &Path) -> Result {\n        Ok(tokio::fs::File::open(path).await?)\n    }\n\n    async fn ort_session_run(\n        session: &mut ort::session::Session, input: Array2,\n    ) -> Result> {\n        static OPTIONS: OnceLock> = OnceLock::new();\n        // TODO(https://github.com/rust-lang/rust/issues/109737): Use get_or_try_init.\n        let options = match OPTIONS.get() {\n            Some(x) => x,\n            None => {\n                let _ = OPTIONS.set(RunOptions::new()?);\n                OPTIONS.get().unwrap()\n            }\n        };\n        Ok(session.run_async(ort::inputs!(\"bytes\" => Tensor::from_array(input)?), options)?.await?)\n    }\n}\n\nfn panic_waker() -> Waker {\n    const PANIC_WAKER: RawWakerVTable = RawWakerVTable::new(clone, wake, wake, drop);\n    fn clone(p: *const ()) -> RawWaker {\n        RawWaker::new(p, &PANIC_WAKER)\n    }\n    fn wake(_: *const ()) {\n        unreachable!()\n    }\n    fn drop(_: *const ()) {}\n    let raw = clone(std::ptr::null());\n    unsafe { Waker::from_raw(raw) }\n}\n"
  },
  {
    "path": "rust/lib/src/input.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse std::future::Future;\nuse std::io::{Read, Seek, SeekFrom};\n\nuse tokio::io::{AsyncReadExt as _, AsyncSeekExt as _};\n\nuse crate::config::ModelConfig;\nuse crate::future::exec;\nuse crate::{ContentType, Result};\n\n/// Features to identify a file using AI.\npub struct Features(pub(crate) Vec);\n\n/// Synchronous abstraction over file content.\npub trait SyncInput {\n    /// Returns the size of the input.\n    fn length(&self) -> Result;\n\n    /// Reads from the input at the given offset to fill the buffer.\n    fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> Result<()>;\n}\n\n/// Asynchronous abstraction over file content.\npub trait AsyncInput {\n    /// Returns the size of the input.\n    fn length(&self) -> impl Future>;\n\n    /// Reads from the input at the given offset to fill the buffer.\n    fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> impl Future>;\n}\n\nconst _: () = const {\n    // We assume in the rest of the file, that u64 holds any usize.\n    assert!(std::mem::size_of::() <= std::mem::size_of::());\n};\n\nimpl SyncInput for &[u8] {\n    fn length(&self) -> Result {\n        Ok(self.len() as u64)\n    }\n\n    fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> Result<()> {\n        let offset = offset.try_into().unwrap();\n        buffer.copy_from_slice(&self[offset..][..buffer.len()]);\n        Ok(())\n    }\n}\n\nimpl SyncInput for std::fs::File {\n    fn length(&self) -> Result {\n        Ok(self.metadata()?.len())\n    }\n\n    fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> Result<()> {\n        self.seek(SeekFrom::Start(offset))?;\n        Ok(self.read_exact(buffer)?)\n    }\n}\n\nimpl SyncInput for &mut T {\n    fn length(&self) -> Result {\n        ::length(self)\n    }\n\n    fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> Result<()> {\n        ::read_at(self, buffer, offset)\n    }\n}\n\nimpl AsyncInput for T {\n    fn length(&self) -> impl Future> {\n        std::future::ready(self.length())\n    }\n\n    fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> impl Future> {\n        std::future::ready(self.read_at(buffer, offset))\n    }\n}\n\nimpl AsyncInput for tokio::fs::File {\n    async fn length(&self) -> Result {\n        Ok(self.metadata().await?.len())\n    }\n\n    async fn read_at(&mut self, buffer: &mut [u8], offset: u64) -> Result<()> {\n        self.seek(SeekFrom::Start(offset)).await?;\n        self.read_exact(buffer).await?;\n        Ok(())\n    }\n}\n\n/// Result of features extraction.\npub enum FeaturesOrRuled {\n    /// Features extracted for identification using AI.\n    Features(Features),\n\n    /// Content identified with rules.\n    Ruled(ContentType),\n}\n\nimpl FeaturesOrRuled {\n    /// Extracts the features from a file (synchronously).\n    ///\n    /// Returns the content type directly if the file cannot be identified using AI.\n    pub fn extract_sync(file: impl SyncInput) -> Result {\n        exec(Self::extract(file))\n    }\n\n    /// Extracts the features from a file (asynchronously).\n    ///\n    /// Returns the content type directly if the file cannot be identified using AI.\n    pub async fn extract_async(file: impl AsyncInput) -> Result {\n        Self::extract(file).await\n    }\n\n    pub(crate) async fn extract(file: impl AsyncInput) -> Result {\n        let config = &crate::model::CONFIG;\n        let file_len = file.length().await?;\n        if file_len == 0 {\n            return Ok(FeaturesOrRuled::Ruled(ContentType::Empty));\n        }\n        let (first_block, features) = extract_features_async(config, file, file_len).await?;\n        if features[config.min_file_size_for_dl - 1] != config.padding_token {\n            return Ok(FeaturesOrRuled::Features(Features(features)));\n        }\n        debug_assert!(first_block.len() <= config.block_size);\n        let content_type = match std::str::from_utf8(&first_block) {\n            Ok(_) => ContentType::Txt,\n            Err(_) => ContentType::Unknown,\n        };\n        Ok(FeaturesOrRuled::Ruled(content_type))\n    }\n}\n\nasync fn extract_features_async(\n    config: &ModelConfig, mut file: impl AsyncInput, file_len: u64,\n) -> Result<(Vec, Vec)> {\n    debug_assert!(config.beg_size < config.block_size);\n    debug_assert!(config.end_size < config.block_size);\n    let buffer_size = std::cmp::min(config.block_size as u64, file_len) as usize;\n    let mut content_beg = vec![0; buffer_size];\n    file.read_at(&mut content_beg, 0).await?;\n    let beg = strip_prefix(&content_beg);\n    let mut end = vec![0; buffer_size];\n    file.read_at(&mut end, file_len - buffer_size as u64).await?;\n    let end = strip_suffix(&end);\n    let mut features = vec![config.padding_token; config.features_size()];\n    let split_features = config.split_features(&mut features);\n    copy_features(split_features.beg, beg, 0);\n    copy_features(split_features.end, end, 1);\n    Ok((content_beg, features))\n}\n\nfn copy_features(dst: &mut [i32], src: &[u8], align: usize) {\n    let len = std::cmp::min(dst.len(), src.len());\n    let dst_len = dst.len(); // borrowing issue: cannot inline below\n    let dst = &mut dst[(dst_len - len) * align..][..len];\n    let src = &src[(src.len() - len) * align..][..len];\n    for (dst, src) in dst.iter_mut().zip(src.iter()) {\n        *dst = *src as i32;\n    }\n}\n\nfn strip_prefix(xs: &[u8]) -> &[u8] {\n    strip(xs, |xs| xs.split_first())\n}\n\nfn strip_suffix(xs: &[u8]) -> &[u8] {\n    strip(xs, |xs| xs.split_last())\n}\n\nfn strip(mut xs: &[u8], mut split: impl FnMut(&[u8]) -> Option<(&u8, &[u8])>) -> &[u8] {\n    while let Some((&x, ys)) = split(xs) {\n        if !is_whitespace(x) {\n            break;\n        }\n        xs = ys;\n    }\n    xs\n}\n\nfn is_whitespace(x: u8) -> bool {\n    x.is_ascii_whitespace() || x == 0x0b\n}\n\n#[cfg(test)]\nmod tests {\n    use std::fs::File;\n    use std::io::Read;\n\n    use data_encoding::BASE64;\n    use flate2::read::GzDecoder;\n    use serde::Deserialize;\n\n    use super::*;\n\n    #[test]\n    fn features_extraction_reference() {\n        // We deny unknown fields to be sure we don't pass the tests by accident when the JSON\n        // format is modified. Fields that are not used are simply marked as dead-code.\n        #[derive(Debug, Deserialize)]\n        #[serde(deny_unknown_fields)]\n        struct Args {\n            beg_size: usize,\n            mid_size: usize,\n            end_size: usize,\n            block_size: usize,\n            padding_token: i32,\n            use_inputs_at_offsets: bool,\n        }\n        #[derive(Debug, Deserialize)]\n        #[serde(deny_unknown_fields)]\n        struct Metadata {\n            #[allow(dead_code)] // debugging only\n            core_content_size: usize,\n            #[allow(dead_code)] // debugging only\n            left_ws_num: usize,\n            #[allow(dead_code)] // debugging only\n            right_ws_num: usize,\n        }\n        #[derive(Debug, Deserialize)]\n        #[serde(deny_unknown_fields)]\n        struct Features {\n            beg: Vec,\n            mid: Vec,\n            end: Vec,\n            offset_0x8000_0x8007: Vec,\n            offset_0x8800_0x8807: Vec,\n            offset_0x9000_0x9007: Vec,\n            offset_0x9800_0x9807: Vec,\n        }\n        #[derive(Debug, Deserialize)]\n        #[serde(deny_unknown_fields)]\n        struct Test {\n            args: Args,\n            #[allow(dead_code)] // debugging only\n            metadata: Metadata,\n            content_base64: String,\n            features: Features,\n        }\n        const PATH: &str = \"../../tests_data/reference/features_extraction_examples.json.gz\";\n        let mut tests = String::new();\n        GzDecoder::new(File::open(PATH).unwrap()).read_to_string(&mut tests).unwrap();\n        let tests: Vec = serde_json::from_str(&tests).unwrap();\n        for test in tests {\n            assert_eq!(test.args.mid_size, 0, \"unsupported mid_size\");\n            assert!(!test.args.use_inputs_at_offsets, \"unsupported use_inputs_at_offsets\");\n            assert!(test.features.mid.is_empty(), \"unsupported mid\");\n            assert!(test.features.offset_0x8000_0x8007.is_empty(), \"unsupported offset\");\n            assert!(test.features.offset_0x8800_0x8807.is_empty(), \"unsupported offset\");\n            assert!(test.features.offset_0x9000_0x9007.is_empty(), \"unsupported offset\");\n            assert!(test.features.offset_0x9800_0x9807.is_empty(), \"unsupported offset\");\n            let config = ModelConfig {\n                beg_size: test.args.beg_size,\n                end_size: test.args.end_size,\n                padding_token: test.args.padding_token,\n                block_size: test.args.block_size,\n                ..crate::model::CONFIG\n            };\n            let mut expected = Vec::new();\n            expected.extend_from_slice(&test.features.beg);\n            expected.extend_from_slice(&test.features.end);\n            let content = BASE64.decode(test.content_base64.as_bytes()).unwrap();\n            let actual = extract_features_async(&config, content.as_slice(), content.len() as u64);\n            let actual = exec(actual).unwrap().1;\n            let actual: Vec<_> = actual.into_iter().map(|x| x as usize).collect();\n            assert_eq!(actual, expected, \"{test:?}\");\n        }\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/lib.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n//! Determines file content types using AI.\n//!\n//! This library indirectly depends on the ONNX Runtime through the `ort` [crate][ort-crate]. The\n//! final user is responsible to make sure the ONNX Runtime is available. The `ort` crate provides\n//! [many options][ort-linking] in this regard. The simplest option is to enable the default [cargo\n//! features][ort-features] of the `ort` crate by adding the following dependency to the\n//! `Cargo.toml` file of the final binary:\n//!\n//! ```toml\n//! [dependencies]\n//! ort = \"=2.0.0-rc.12\"\n//! ```\n//!\n//! # Examples\n//!\n//! ```rust\n//! # fn main() -> magika::Result<()> {\n//! // A Magika session can be used multiple times across multiple threads.\n//! let mut magika = magika::Session::new()?;\n//!\n//! // Files can be identified from their path.\n//! assert_eq!(magika.identify_file_sync(\"src/lib.rs\")?.info().label, \"rust\");\n//!\n//! // Contents can also be identified directly from memory.\n//! let result = magika.identify_content_sync(&b\"#!/bin/sh\\necho hello\"[..])?;\n//! assert_eq!(result.info().label, \"shell\");\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! [ort-crate]: https://crates.io/crates/ort\n//! [ort-linking]: https://ort.pyke.io/setup/linking\n//! [ort-features]: https://ort.pyke.io/setup/cargo-features\n\n#![cfg_attr(feature = \"_doc\", feature(doc_cfg))]\n\npub use crate::builder::Builder;\npub use crate::content::{ContentType, MODEL_MAJOR_VERSION, MODEL_NAME};\npub use crate::error::{Error, Result};\npub use crate::file::{FileType, InferredType, OverwriteReason, TypeInfo};\npub use crate::input::{AsyncInput, Features, FeaturesOrRuled, SyncInput};\npub use crate::session::Session;\n\nmod builder;\nmod config;\nmod content;\nmod error;\nmod file;\nmod future;\nmod input;\nmod model;\nmod session;\n\n#[cfg(test)]\nmod tests {\n    use std::fs::File;\n    use std::io::Read;\n\n    use data_encoding::BASE64;\n    use flate2::read::GzDecoder;\n    use serde::Deserialize;\n\n    use super::*;\n\n    #[derive(Debug, Deserialize)]\n    #[serde(deny_unknown_fields)]\n    struct Prediction {\n        dl: String,\n        output: String,\n        score: f32,\n        overwrite_reason: String,\n    }\n\n    fn assert_float(actual: f32, expected: f32, debug: &str) {\n        const PRECISION: f32 = 10000.;\n        let actual = (actual * PRECISION).trunc() / PRECISION;\n        let expected = (expected * PRECISION).trunc() / PRECISION;\n        assert_eq!(actual, expected, \"{debug}\");\n    }\n\n    fn assert_prediction(actual: FileType, expected: Prediction, debug: &str) {\n        let actual = match actual {\n            FileType::Inferred(x) => x,\n            FileType::Ruled(content_type) => {\n                assert_eq!(content_type.info().label, expected.output, \"{debug}\");\n                assert_eq!(1.0, expected.score, \"{debug}\");\n                assert_eq!(\"none\", expected.overwrite_reason, \"{debug}\");\n                assert_eq!(\"undefined\", expected.dl, \"{debug}\");\n                return;\n            }\n            _ => unreachable!(),\n        };\n        assert_eq!(actual.content_type().info().label, expected.output, \"{debug}\");\n        assert_float(actual.score, expected.score, debug);\n        let overwrite_reason = match actual.content_type {\n            None => \"none\",\n            Some((_, OverwriteReason::LowConfidence)) => \"low-confidence\",\n            Some((_, OverwriteReason::OverwriteMap)) => \"overwrite-map\",\n        };\n        assert_eq!(overwrite_reason, expected.overwrite_reason);\n        assert_eq!(actual.inferred_type.info().label, expected.dl, \"{debug}\");\n    }\n\n    #[test]\n    fn identify_by_path_reference() {\n        #[derive(Debug, Deserialize)]\n        #[serde(deny_unknown_fields)]\n        struct Test {\n            prediction_mode: String,\n            path: String,\n            status: String,\n            prediction: Option,\n        }\n        let path =\n            format!(\"../../tests_data/reference/{MODEL_NAME}-inference_examples_by_path.json.gz\");\n        let mut tests = String::new();\n        GzDecoder::new(File::open(path).unwrap()).read_to_string(&mut tests).unwrap();\n        let tests: Vec = serde_json::from_str(&tests).unwrap();\n        let mut session = Session::new().unwrap();\n        for test in tests {\n            if test.prediction_mode != \"high-confidence\" {\n                continue; // we only support high-confidence\n            }\n            assert_eq!(test.status, \"ok\"); // only scenario tested so far\n            let expected = test.prediction.unwrap();\n            let actual = session.identify_file_sync(format!(\"../../{}\", test.path)).unwrap();\n            assert_prediction(actual, expected, &test.path);\n        }\n    }\n\n    #[test]\n    fn identify_by_content_reference() {\n        #[derive(Debug, Deserialize)]\n        #[serde(deny_unknown_fields)]\n        struct Test {\n            prediction_mode: String,\n            content_base64: String,\n            status: String,\n            prediction: Option,\n        }\n        let path = format!(\n            \"../../tests_data/reference/{MODEL_NAME}-inference_examples_by_content.json.gz\"\n        );\n        let mut tests = String::new();\n        GzDecoder::new(File::open(path).unwrap()).read_to_string(&mut tests).unwrap();\n        let tests: Vec = serde_json::from_str(&tests).unwrap();\n        let mut session = Session::new().unwrap();\n        for test in tests {\n            if test.prediction_mode != \"high-confidence\" {\n                continue; // we only support high-confidence\n            }\n            assert_eq!(test.status, \"ok\"); // only scenario tested so far\n            let expected = test.prediction.unwrap();\n            let content = BASE64.decode(test.content_base64.as_bytes()).unwrap();\n            let actual = session.identify_content_sync(content.as_slice()).unwrap();\n            assert_prediction(actual, expected, &test.content_base64);\n        }\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/model.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// DO NOT EDIT, see link below for more information:\n// https://github.com/google/magika/tree/main/rust/gen\n\nuse std::borrow::Cow;\n\nuse crate::config::ModelConfig;\nuse crate::ContentType;\n\npub(crate) const CONFIG: ModelConfig = ModelConfig {\n    beg_size: 1024,\n    end_size: 1024,\n    min_file_size_for_dl: 8,\n    padding_token: 256,\n    block_size: 4096,\n    thresholds: Cow::Borrowed(&THRESHOLDS),\n    overwrite_map: Cow::Borrowed(&OVERWRITE_MAP),\n};\n\n#[rustfmt::skip]\nconst THRESHOLDS: [f32; ContentType::SIZE] = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.95, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.95, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.75, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.95, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.9, 0.5, 0.5];\nconst OVERWRITE_MAP: [ContentType; ContentType::SIZE] = [\n    ContentType::_3gp,\n    ContentType::Ace,\n    ContentType::Ai,\n    ContentType::Aidl,\n    ContentType::Apk,\n    ContentType::Applebplist,\n    ContentType::Appleplist,\n    ContentType::Asm,\n    ContentType::Asp,\n    ContentType::Autohotkey,\n    ContentType::Autoit,\n    ContentType::Awk,\n    ContentType::Batch,\n    ContentType::Bazel,\n    ContentType::Bib,\n    ContentType::Bmp,\n    ContentType::Bzip,\n    ContentType::C,\n    ContentType::Cab,\n    ContentType::Cat,\n    ContentType::Chm,\n    ContentType::Clojure,\n    ContentType::Cmake,\n    ContentType::Cobol,\n    ContentType::Coff,\n    ContentType::Coffeescript,\n    ContentType::Cpp,\n    ContentType::Crt,\n    ContentType::Crx,\n    ContentType::Cs,\n    ContentType::Csproj,\n    ContentType::Css,\n    ContentType::Csv,\n    ContentType::Dart,\n    ContentType::Deb,\n    ContentType::Dex,\n    ContentType::Dicom,\n    ContentType::Diff,\n    ContentType::Dm,\n    ContentType::Dmg,\n    ContentType::Doc,\n    ContentType::Dockerfile,\n    ContentType::Docx,\n    ContentType::Dsstore,\n    ContentType::Dwg,\n    ContentType::Dxf,\n    ContentType::Elf,\n    ContentType::Elixir,\n    ContentType::Emf,\n    ContentType::Eml,\n    ContentType::Empty,\n    ContentType::Epub,\n    ContentType::Erb,\n    ContentType::Erlang,\n    ContentType::Flac,\n    ContentType::Flv,\n    ContentType::Fortran,\n    ContentType::Gemfile,\n    ContentType::Gemspec,\n    ContentType::Gif,\n    ContentType::Gitattributes,\n    ContentType::Gitmodules,\n    ContentType::Go,\n    ContentType::Gradle,\n    ContentType::Groovy,\n    ContentType::Gzip,\n    ContentType::H5,\n    ContentType::Handlebars,\n    ContentType::Haskell,\n    ContentType::Hcl,\n    ContentType::Hlp,\n    ContentType::Htaccess,\n    ContentType::Html,\n    ContentType::Icns,\n    ContentType::Ico,\n    ContentType::Ics,\n    ContentType::Ignorefile,\n    ContentType::Ini,\n    ContentType::Internetshortcut,\n    ContentType::Ipynb,\n    ContentType::Iso,\n    ContentType::Jar,\n    ContentType::Java,\n    ContentType::Javabytecode,\n    ContentType::Javascript,\n    ContentType::Jinja,\n    ContentType::Jp2,\n    ContentType::Jpeg,\n    ContentType::Json,\n    ContentType::Jsonl,\n    ContentType::Julia,\n    ContentType::Kotlin,\n    ContentType::Latex,\n    ContentType::Lha,\n    ContentType::Lisp,\n    ContentType::Lnk,\n    ContentType::Lua,\n    ContentType::M3u,\n    ContentType::M4,\n    ContentType::Macho,\n    ContentType::Makefile,\n    ContentType::Markdown,\n    ContentType::Matlab,\n    ContentType::Mht,\n    ContentType::Midi,\n    ContentType::Mkv,\n    ContentType::Mp3,\n    ContentType::Mp4,\n    ContentType::Mscompress,\n    ContentType::Msi,\n    ContentType::Mum,\n    ContentType::Npy,\n    ContentType::Npz,\n    ContentType::Nupkg,\n    ContentType::Objectivec,\n    ContentType::Ocaml,\n    ContentType::Odp,\n    ContentType::Ods,\n    ContentType::Odt,\n    ContentType::Ogg,\n    ContentType::One,\n    ContentType::Onnx,\n    ContentType::Otf,\n    ContentType::Outlook,\n    ContentType::Parquet,\n    ContentType::Pascal,\n    ContentType::Pcap,\n    ContentType::Pdb,\n    ContentType::Pdf,\n    ContentType::Pebin,\n    ContentType::Pem,\n    ContentType::Perl,\n    ContentType::Php,\n    ContentType::Pickle,\n    ContentType::Png,\n    ContentType::Po,\n    ContentType::Postscript,\n    ContentType::Powershell,\n    ContentType::Ppt,\n    ContentType::Pptx,\n    ContentType::Prolog,\n    ContentType::Proteindb,\n    ContentType::Proto,\n    ContentType::Psd,\n    ContentType::Python,\n    ContentType::Pythonbytecode,\n    ContentType::Pytorch,\n    ContentType::Qt,\n    ContentType::R,\n    ContentType::Unknown,\n    ContentType::Txt,\n    ContentType::Rar,\n    ContentType::Rdf,\n    ContentType::Rpm,\n    ContentType::Rst,\n    ContentType::Rtf,\n    ContentType::Ruby,\n    ContentType::Rust,\n    ContentType::Scala,\n    ContentType::Scss,\n    ContentType::Sevenzip,\n    ContentType::Sgml,\n    ContentType::Shell,\n    ContentType::Smali,\n    ContentType::Snap,\n    ContentType::Solidity,\n    ContentType::Sql,\n    ContentType::Sqlite,\n    ContentType::Squashfs,\n    ContentType::Srt,\n    ContentType::Stlbinary,\n    ContentType::Stltext,\n    ContentType::Sum,\n    ContentType::Svg,\n    ContentType::Swf,\n    ContentType::Swift,\n    ContentType::Tar,\n    ContentType::Tcl,\n    ContentType::Textproto,\n    ContentType::Tga,\n    ContentType::Thumbsdb,\n    ContentType::Tiff,\n    ContentType::Toml,\n    ContentType::Torrent,\n    ContentType::Tsv,\n    ContentType::Ttf,\n    ContentType::Twig,\n    ContentType::Txt,\n    ContentType::Typescript,\n    ContentType::Undefined,\n    ContentType::Unknown,\n    ContentType::Vba,\n    ContentType::Vcxproj,\n    ContentType::Verilog,\n    ContentType::Vhdl,\n    ContentType::Vtt,\n    ContentType::Vue,\n    ContentType::Wasm,\n    ContentType::Wav,\n    ContentType::Webm,\n    ContentType::Webp,\n    ContentType::Winregistry,\n    ContentType::Wmf,\n    ContentType::Woff,\n    ContentType::Woff2,\n    ContentType::Xar,\n    ContentType::Xls,\n    ContentType::Xlsb,\n    ContentType::Xlsx,\n    ContentType::Xml,\n    ContentType::Xpi,\n    ContentType::Xz,\n    ContentType::Yaml,\n    ContentType::Yara,\n    ContentType::Zig,\n    ContentType::Zip,\n    ContentType::Zlibstream,\n];\n\n#[derive(Debug, Copy, Clone, PartialEq, Eq)]\n#[repr(u32)]\n#[allow(dead_code)] // only constructed through transmute\npub(crate) enum Label {\n    _3gp,\n    Ace,\n    Ai,\n    Aidl,\n    Apk,\n    Applebplist,\n    Appleplist,\n    Asm,\n    Asp,\n    Autohotkey,\n    Autoit,\n    Awk,\n    Batch,\n    Bazel,\n    Bib,\n    Bmp,\n    Bzip,\n    C,\n    Cab,\n    Cat,\n    Chm,\n    Clojure,\n    Cmake,\n    Cobol,\n    Coff,\n    Coffeescript,\n    Cpp,\n    Crt,\n    Crx,\n    Cs,\n    Csproj,\n    Css,\n    Csv,\n    Dart,\n    Deb,\n    Dex,\n    Dicom,\n    Diff,\n    Dm,\n    Dmg,\n    Doc,\n    Dockerfile,\n    Docx,\n    Dsstore,\n    Dwg,\n    Dxf,\n    Elf,\n    Elixir,\n    Emf,\n    Eml,\n    Epub,\n    Erb,\n    Erlang,\n    Flac,\n    Flv,\n    Fortran,\n    Gemfile,\n    Gemspec,\n    Gif,\n    Gitattributes,\n    Gitmodules,\n    Go,\n    Gradle,\n    Groovy,\n    Gzip,\n    H5,\n    Handlebars,\n    Haskell,\n    Hcl,\n    Hlp,\n    Htaccess,\n    Html,\n    Icns,\n    Ico,\n    Ics,\n    Ignorefile,\n    Ini,\n    Internetshortcut,\n    Ipynb,\n    Iso,\n    Jar,\n    Java,\n    Javabytecode,\n    Javascript,\n    Jinja,\n    Jp2,\n    Jpeg,\n    Json,\n    Jsonl,\n    Julia,\n    Kotlin,\n    Latex,\n    Lha,\n    Lisp,\n    Lnk,\n    Lua,\n    M3u,\n    M4,\n    Macho,\n    Makefile,\n    Markdown,\n    Matlab,\n    Mht,\n    Midi,\n    Mkv,\n    Mp3,\n    Mp4,\n    Mscompress,\n    Msi,\n    Mum,\n    Npy,\n    Npz,\n    Nupkg,\n    Objectivec,\n    Ocaml,\n    Odp,\n    Ods,\n    Odt,\n    Ogg,\n    One,\n    Onnx,\n    Otf,\n    Outlook,\n    Parquet,\n    Pascal,\n    Pcap,\n    Pdb,\n    Pdf,\n    Pebin,\n    Pem,\n    Perl,\n    Php,\n    Pickle,\n    Png,\n    Po,\n    Postscript,\n    Powershell,\n    Ppt,\n    Pptx,\n    Prolog,\n    Proteindb,\n    Proto,\n    Psd,\n    Python,\n    Pythonbytecode,\n    Pytorch,\n    Qt,\n    R,\n    Randombytes,\n    Randomtxt,\n    Rar,\n    Rdf,\n    Rpm,\n    Rst,\n    Rtf,\n    Ruby,\n    Rust,\n    Scala,\n    Scss,\n    Sevenzip,\n    Sgml,\n    Shell,\n    Smali,\n    Snap,\n    Solidity,\n    Sql,\n    Sqlite,\n    Squashfs,\n    Srt,\n    Stlbinary,\n    Stltext,\n    Sum,\n    Svg,\n    Swf,\n    Swift,\n    Tar,\n    Tcl,\n    Textproto,\n    Tga,\n    Thumbsdb,\n    Tiff,\n    Toml,\n    Torrent,\n    Tsv,\n    Ttf,\n    Twig,\n    Txt,\n    Typescript,\n    Vba,\n    Vcxproj,\n    Verilog,\n    Vhdl,\n    Vtt,\n    Vue,\n    Wasm,\n    Wav,\n    Webm,\n    Webp,\n    Winregistry,\n    Wmf,\n    Woff,\n    Woff2,\n    Xar,\n    Xls,\n    Xlsb,\n    Xlsx,\n    Xml,\n    Xpi,\n    Xz,\n    Yaml,\n    Yara,\n    Zig,\n    Zip,\n    Zlibstream,\n}\n\npub(crate) const NUM_LABELS: usize = 214;\nimpl Label {\n    pub(crate) fn content_type(self) -> ContentType {\n        match self {\n            Label::_3gp => ContentType::_3gp,\n            Label::Ace => ContentType::Ace,\n            Label::Ai => ContentType::Ai,\n            Label::Aidl => ContentType::Aidl,\n            Label::Apk => ContentType::Apk,\n            Label::Applebplist => ContentType::Applebplist,\n            Label::Appleplist => ContentType::Appleplist,\n            Label::Asm => ContentType::Asm,\n            Label::Asp => ContentType::Asp,\n            Label::Autohotkey => ContentType::Autohotkey,\n            Label::Autoit => ContentType::Autoit,\n            Label::Awk => ContentType::Awk,\n            Label::Batch => ContentType::Batch,\n            Label::Bazel => ContentType::Bazel,\n            Label::Bib => ContentType::Bib,\n            Label::Bmp => ContentType::Bmp,\n            Label::Bzip => ContentType::Bzip,\n            Label::C => ContentType::C,\n            Label::Cab => ContentType::Cab,\n            Label::Cat => ContentType::Cat,\n            Label::Chm => ContentType::Chm,\n            Label::Clojure => ContentType::Clojure,\n            Label::Cmake => ContentType::Cmake,\n            Label::Cobol => ContentType::Cobol,\n            Label::Coff => ContentType::Coff,\n            Label::Coffeescript => ContentType::Coffeescript,\n            Label::Cpp => ContentType::Cpp,\n            Label::Crt => ContentType::Crt,\n            Label::Crx => ContentType::Crx,\n            Label::Cs => ContentType::Cs,\n            Label::Csproj => ContentType::Csproj,\n            Label::Css => ContentType::Css,\n            Label::Csv => ContentType::Csv,\n            Label::Dart => ContentType::Dart,\n            Label::Deb => ContentType::Deb,\n            Label::Dex => ContentType::Dex,\n            Label::Dicom => ContentType::Dicom,\n            Label::Diff => ContentType::Diff,\n            Label::Dm => ContentType::Dm,\n            Label::Dmg => ContentType::Dmg,\n            Label::Doc => ContentType::Doc,\n            Label::Dockerfile => ContentType::Dockerfile,\n            Label::Docx => ContentType::Docx,\n            Label::Dsstore => ContentType::Dsstore,\n            Label::Dwg => ContentType::Dwg,\n            Label::Dxf => ContentType::Dxf,\n            Label::Elf => ContentType::Elf,\n            Label::Elixir => ContentType::Elixir,\n            Label::Emf => ContentType::Emf,\n            Label::Eml => ContentType::Eml,\n            Label::Epub => ContentType::Epub,\n            Label::Erb => ContentType::Erb,\n            Label::Erlang => ContentType::Erlang,\n            Label::Flac => ContentType::Flac,\n            Label::Flv => ContentType::Flv,\n            Label::Fortran => ContentType::Fortran,\n            Label::Gemfile => ContentType::Gemfile,\n            Label::Gemspec => ContentType::Gemspec,\n            Label::Gif => ContentType::Gif,\n            Label::Gitattributes => ContentType::Gitattributes,\n            Label::Gitmodules => ContentType::Gitmodules,\n            Label::Go => ContentType::Go,\n            Label::Gradle => ContentType::Gradle,\n            Label::Groovy => ContentType::Groovy,\n            Label::Gzip => ContentType::Gzip,\n            Label::H5 => ContentType::H5,\n            Label::Handlebars => ContentType::Handlebars,\n            Label::Haskell => ContentType::Haskell,\n            Label::Hcl => ContentType::Hcl,\n            Label::Hlp => ContentType::Hlp,\n            Label::Htaccess => ContentType::Htaccess,\n            Label::Html => ContentType::Html,\n            Label::Icns => ContentType::Icns,\n            Label::Ico => ContentType::Ico,\n            Label::Ics => ContentType::Ics,\n            Label::Ignorefile => ContentType::Ignorefile,\n            Label::Ini => ContentType::Ini,\n            Label::Internetshortcut => ContentType::Internetshortcut,\n            Label::Ipynb => ContentType::Ipynb,\n            Label::Iso => ContentType::Iso,\n            Label::Jar => ContentType::Jar,\n            Label::Java => ContentType::Java,\n            Label::Javabytecode => ContentType::Javabytecode,\n            Label::Javascript => ContentType::Javascript,\n            Label::Jinja => ContentType::Jinja,\n            Label::Jp2 => ContentType::Jp2,\n            Label::Jpeg => ContentType::Jpeg,\n            Label::Json => ContentType::Json,\n            Label::Jsonl => ContentType::Jsonl,\n            Label::Julia => ContentType::Julia,\n            Label::Kotlin => ContentType::Kotlin,\n            Label::Latex => ContentType::Latex,\n            Label::Lha => ContentType::Lha,\n            Label::Lisp => ContentType::Lisp,\n            Label::Lnk => ContentType::Lnk,\n            Label::Lua => ContentType::Lua,\n            Label::M3u => ContentType::M3u,\n            Label::M4 => ContentType::M4,\n            Label::Macho => ContentType::Macho,\n            Label::Makefile => ContentType::Makefile,\n            Label::Markdown => ContentType::Markdown,\n            Label::Matlab => ContentType::Matlab,\n            Label::Mht => ContentType::Mht,\n            Label::Midi => ContentType::Midi,\n            Label::Mkv => ContentType::Mkv,\n            Label::Mp3 => ContentType::Mp3,\n            Label::Mp4 => ContentType::Mp4,\n            Label::Mscompress => ContentType::Mscompress,\n            Label::Msi => ContentType::Msi,\n            Label::Mum => ContentType::Mum,\n            Label::Npy => ContentType::Npy,\n            Label::Npz => ContentType::Npz,\n            Label::Nupkg => ContentType::Nupkg,\n            Label::Objectivec => ContentType::Objectivec,\n            Label::Ocaml => ContentType::Ocaml,\n            Label::Odp => ContentType::Odp,\n            Label::Ods => ContentType::Ods,\n            Label::Odt => ContentType::Odt,\n            Label::Ogg => ContentType::Ogg,\n            Label::One => ContentType::One,\n            Label::Onnx => ContentType::Onnx,\n            Label::Otf => ContentType::Otf,\n            Label::Outlook => ContentType::Outlook,\n            Label::Parquet => ContentType::Parquet,\n            Label::Pascal => ContentType::Pascal,\n            Label::Pcap => ContentType::Pcap,\n            Label::Pdb => ContentType::Pdb,\n            Label::Pdf => ContentType::Pdf,\n            Label::Pebin => ContentType::Pebin,\n            Label::Pem => ContentType::Pem,\n            Label::Perl => ContentType::Perl,\n            Label::Php => ContentType::Php,\n            Label::Pickle => ContentType::Pickle,\n            Label::Png => ContentType::Png,\n            Label::Po => ContentType::Po,\n            Label::Postscript => ContentType::Postscript,\n            Label::Powershell => ContentType::Powershell,\n            Label::Ppt => ContentType::Ppt,\n            Label::Pptx => ContentType::Pptx,\n            Label::Prolog => ContentType::Prolog,\n            Label::Proteindb => ContentType::Proteindb,\n            Label::Proto => ContentType::Proto,\n            Label::Psd => ContentType::Psd,\n            Label::Python => ContentType::Python,\n            Label::Pythonbytecode => ContentType::Pythonbytecode,\n            Label::Pytorch => ContentType::Pytorch,\n            Label::Qt => ContentType::Qt,\n            Label::R => ContentType::R,\n            Label::Randombytes => ContentType::Randombytes,\n            Label::Randomtxt => ContentType::Randomtxt,\n            Label::Rar => ContentType::Rar,\n            Label::Rdf => ContentType::Rdf,\n            Label::Rpm => ContentType::Rpm,\n            Label::Rst => ContentType::Rst,\n            Label::Rtf => ContentType::Rtf,\n            Label::Ruby => ContentType::Ruby,\n            Label::Rust => ContentType::Rust,\n            Label::Scala => ContentType::Scala,\n            Label::Scss => ContentType::Scss,\n            Label::Sevenzip => ContentType::Sevenzip,\n            Label::Sgml => ContentType::Sgml,\n            Label::Shell => ContentType::Shell,\n            Label::Smali => ContentType::Smali,\n            Label::Snap => ContentType::Snap,\n            Label::Solidity => ContentType::Solidity,\n            Label::Sql => ContentType::Sql,\n            Label::Sqlite => ContentType::Sqlite,\n            Label::Squashfs => ContentType::Squashfs,\n            Label::Srt => ContentType::Srt,\n            Label::Stlbinary => ContentType::Stlbinary,\n            Label::Stltext => ContentType::Stltext,\n            Label::Sum => ContentType::Sum,\n            Label::Svg => ContentType::Svg,\n            Label::Swf => ContentType::Swf,\n            Label::Swift => ContentType::Swift,\n            Label::Tar => ContentType::Tar,\n            Label::Tcl => ContentType::Tcl,\n            Label::Textproto => ContentType::Textproto,\n            Label::Tga => ContentType::Tga,\n            Label::Thumbsdb => ContentType::Thumbsdb,\n            Label::Tiff => ContentType::Tiff,\n            Label::Toml => ContentType::Toml,\n            Label::Torrent => ContentType::Torrent,\n            Label::Tsv => ContentType::Tsv,\n            Label::Ttf => ContentType::Ttf,\n            Label::Twig => ContentType::Twig,\n            Label::Txt => ContentType::Txt,\n            Label::Typescript => ContentType::Typescript,\n            Label::Vba => ContentType::Vba,\n            Label::Vcxproj => ContentType::Vcxproj,\n            Label::Verilog => ContentType::Verilog,\n            Label::Vhdl => ContentType::Vhdl,\n            Label::Vtt => ContentType::Vtt,\n            Label::Vue => ContentType::Vue,\n            Label::Wasm => ContentType::Wasm,\n            Label::Wav => ContentType::Wav,\n            Label::Webm => ContentType::Webm,\n            Label::Webp => ContentType::Webp,\n            Label::Winregistry => ContentType::Winregistry,\n            Label::Wmf => ContentType::Wmf,\n            Label::Woff => ContentType::Woff,\n            Label::Woff2 => ContentType::Woff2,\n            Label::Xar => ContentType::Xar,\n            Label::Xls => ContentType::Xls,\n            Label::Xlsb => ContentType::Xlsb,\n            Label::Xlsx => ContentType::Xlsx,\n            Label::Xml => ContentType::Xml,\n            Label::Xpi => ContentType::Xpi,\n            Label::Xz => ContentType::Xz,\n            Label::Yaml => ContentType::Yaml,\n            Label::Yara => ContentType::Yara,\n            Label::Zig => ContentType::Zig,\n            Label::Zip => ContentType::Zip,\n            Label::Zlibstream => ContentType::Zlibstream,\n        }\n    }\n}\n"
  },
  {
    "path": "rust/lib/src/session.rs",
    "content": "// Copyright 2024 Google LLC\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse std::path::Path;\n\nuse ndarray::Array2;\n\nuse crate::future::{exec, AsyncEnv, Env, SyncEnv};\nuse crate::{AsyncInput, Builder, Features, FeaturesOrRuled, FileType, Result, SyncInput};\n\n/// A Magika session to identify files.\n#[derive(Debug)]\npub struct Session {\n    pub(crate) session: ort::session::Session,\n}\n\nimpl Session {\n    /// Creates a default session.\n    pub fn new() -> Result {\n        Session::builder().build()\n    }\n\n    /// Initializes a new Magika session builder with default values.\n    pub fn builder() -> Builder {\n        Builder::default()\n    }\n\n    /// Identifies a single file (synchronously).\n    pub fn identify_file_sync(&mut self, file: impl AsRef) -> Result {\n        exec(self.identify_file::(file.as_ref()))\n    }\n\n    /// Identifies a single file (asynchronously).\n    pub async fn identify_file_async(&mut self, file: impl AsRef) -> Result {\n        self.identify_file::(file.as_ref()).await\n    }\n\n    async fn identify_file(&mut self, file: &Path) -> Result {\n        let metadata = E::symlink_metadata(file).await?;\n        if metadata.is_dir() {\n            Ok(FileType::Directory)\n        } else if metadata.is_symlink() {\n            Ok(FileType::Symlink)\n        } else {\n            debug_assert!(metadata.is_file());\n            self.identify_content::(E::open(file).await?).await\n        }\n    }\n\n    /// Identifies a single file from its content (synchronously).\n    pub fn identify_content_sync(&mut self, file: impl SyncInput) -> Result {\n        exec(self.identify_content::(file))\n    }\n\n    /// Identifies a single file from its content (asynchronously).\n    pub async fn identify_content_async(&mut self, file: impl AsyncInput) -> Result {\n        self.identify_content::(file).await\n    }\n\n    async fn identify_content(&mut self, file: impl AsyncInput) -> Result {\n        match FeaturesOrRuled::extract(file).await? {\n            FeaturesOrRuled::Ruled(content_type) => Ok(FileType::Ruled(content_type)),\n            FeaturesOrRuled::Features(features) => self.identify_features::(&features).await,\n        }\n    }\n\n    /// Identifies a single file from its features (synchronously).\n    pub fn identify_features_sync(&mut self, features: &Features) -> Result {\n        exec(self.identify_features::(features))\n    }\n\n    /// Identifies a single file from its features (asynchronously).\n    pub async fn identify_features_async(&mut self, features: &Features) -> Result {\n        self.identify_features::(features).await\n    }\n\n    async fn identify_features(&mut self, features: &Features) -> Result {\n        let results = self.identify_features_batch::(std::slice::from_ref(features)).await?;\n        let [result] = results.try_into().ok().unwrap();\n        Ok(result)\n    }\n\n    /// Identifies multiple files in parallel from their features (synchronously).\n    pub fn identify_features_batch_sync(&mut self, features: &[Features]) -> Result> {\n        exec(self.identify_features_batch::(features))\n    }\n\n    /// Identifies multiple files in parallel from their features (asynchronously).\n    pub async fn identify_features_batch_async(\n        &mut self, features: &[Features],\n    ) -> Result> {\n        self.identify_features_batch::(features).await\n    }\n\n    async fn identify_features_batch(\n        &mut self, features: &[Features],\n    ) -> Result> {\n        if features.is_empty() {\n            return Ok(Vec::new());\n        }\n        let features_size = crate::model::CONFIG.features_size();\n        let input = Array2::from_shape_vec(\n            [features.len(), features_size],\n            features.iter().flat_map(|x| &x.0).cloned().collect(),\n        )?;\n        let mut output = E::ort_session_run(&mut self.session, input).await?;\n        let output = output.remove(\"target_label\").unwrap();\n        let output = output.try_extract_array()?;\n        Ok(FileType::convert(output))\n    }\n}\n"
  },
  {
    "path": "rust/lib/test.sh",
    "content": "#!/bin/sh\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ../color.sh\n\nx cargo check\nx cargo check --features=serde\nx cargo test --features=_test\nx cargo fmt -- --check\nx cargo clippy -- --deny=warnings\nif cargo --version | grep -q nightly; then\n  x env RUSTDOCFLAGS=--deny=warnings cargo doc --features=_doc\nfi\n"
  },
  {
    "path": "rust/onnx/build.sh",
    "content": "#!/bin/bash\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ../color.sh\n\n# This script builds ONNX Runtime as a static library to be linked in the Magika CLI.\n#\n# This is needed when building for manylinux since the prebuilt binaries provided by the ort crate\n# have too recent dependency requirements.\n\nif [ -e runtime ]; then\n  info \"Using cached static libraries.\"\nelse\n  info \"Make sure we have Python 3.x and cmake-3.27 or higher.\"\n  python3 -m venv venv\n  source venv/bin/activate\n  python3 -m pip install cmake==3.31.6\n\n  info \"Clone ONNX Runtime repository (recursively).\"\n  git clone --recursive https://github.com/Microsoft/onnxruntime.git runtime\n  cd runtime\n\n  info \"Checkout v1.24.2 because that's what ort v2.0.0-rc.12 supports.\"\n  git checkout v1.24.2\n\n  # The build fails with GCC 14 due to warnings as errors.\n  sed -i '/function(onnxruntime_set_compile_flags/a\\\n    target_compile_options(${target_name} PRIVATE \"$<$:-Wno-array-bounds>\")\\\n    target_compile_options(${target_name} PRIVATE \"$<$:-Wno-maybe-uninitialized>\")\\\n    target_compile_options(${target_name} PRIVATE \"$<$:-Wno-uninitialized>\")' \\\n    cmake/CMakeLists.txt\n\n  info \"Build the static libraries.\"\n  x ./build.sh --config=Release --parallel $ONNX_RUNTIME_BUILD_FLAGS\n\n  info \"Only keep the static libraries to save cache space.\"\n  find build/Linux -not -name '*.a' \\( -not -type d -or -empty \\) -delete\n  cd ..\nfi\n\ninfo \"Point the ort crate to the locally built static library.\"\ncd ../..\ncat >> .cargo/config.toml < cli/output 2>&1\n\ninfo \"Updating CLI output in README.md\"\n( cd cli\n  for i in $(seq 1 $(grep '^% ' README.md | wc -l)); do\n    grep -n '^% ' README.md | cut -f1 -d: | head -n$i | tail -n1 | while read line; do\n      sed -i $line',/```/{'$line'p;/```/!d}' README.md\n      cmd=\"$(head -n$line README.md | tail -n1 | sed 's/^% //')\"\n      ( cd ../..; eval \"$cmd\"; ) 2>/dev/null > tmp\n      sed -i $line'r tmp' README.md\n    done\n  done\n  rm tmp\n  sed -i 's/ \\+$//' README.md\n)\n\ninfo \"Updating ort version in library documentation\"\n( cd lib\n  VERSION=$(sed -n '/^\\[dependencies.ort\\]$/,/^$/{s/^version = //p}' Cargo.toml)\n  sed -Ei 's#^(//! ort =) .*$#\\1 '\"$VERSION\"'#' src/lib.rs\n)\n\nif [ \"$1\" = --check ]; then\n  if ! git diff --exit-code; then\n    [ -n \"$CI\" ] && todo 'Execute ./sync.sh from the rust directory'\n    error 'Generated files are not in sync'\n  fi\nfi\nsuccess \"Generated files are synced\"\n"
  },
  {
    "path": "rust/taplo.toml",
    "content": "[formatting]\ncolumn_width = 100\nreorder_arrays = true\nreorder_keys = true\n\n[[rule]]\nformatting = { reorder_keys = false }\nkeys = [\"build-dependencies.*\", \"dependencies.*\", \"dev-dependencies.*\", \"package\"]\n\n[[rule]]\nformatting = { reorder_arrays = false }\nkeys = [\"package.metadata.deb\"]\n"
  },
  {
    "path": "rust/test.sh",
    "content": "#!/bin/sh\n# Copyright 2024 Google LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -e\n. ./color.sh\n\nTOOLCHAINS='stable nightly'\n[ -z \"$CI\" ] || TOOLCHAINS=$(rustup show active-toolchain | sed 's/-.*//')\n\nfor toolchain in $TOOLCHAINS; do\n  for dir in gen lib cli; do\n    info \"Running tests from $dir with $toolchain\"\n    ( cd $dir && rustup run $toolchain ./test.sh; )\n  done\ndone\n\n./sync.sh --check\n"
  },
  {
    "path": "tests_data/README.md",
    "content": "# Tests Data\n\nWe use these files for regressions testing.\n\nThese files were not (and should not) be used for training purposes.\n\nThey are organized by directory:\n- `basic/`: a number of simple files of various content types.\n- `mitra/`: a selection of the files available at [https://github.com/corkami/mitra](https://github.com/corkami/mitra/tree/master/input).\n"
  },
  {
    "path": "tests_data/basic/asm/code.asm",
    "content": ".section .text\n\txor    %eax,%eax\n\tpush   %eax\n\tpush   $0x68732f2f\n\tpush   $0x6e69622f\n\tmov    %esp,%ebx\n\tpush   %eax\n\tpush   %ebx\n\tmov    %esp,%ecx\n\tmov    $0xb,%al\n\tint    $0x80\n"
  },
  {
    "path": "tests_data/basic/batch/simple.bat",
    "content": "@echo off\necho %1\necho %2\necho %3"
  },
  {
    "path": "tests_data/basic/c/code.c",
    "content": "#include \n\nint main() {\n    char c;\n    printf(\"Enter a character: \");\n    scanf(\"%c\", &c);\n\n    if ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z'))\n        printf(\"%c is a letter\");\n    else\n        printf(\"%c is not a letter\", c);\n\n    return 0;\n}"
  },
  {
    "path": "tests_data/basic/css/code.css",
    "content": "/* table.mycv-entry { */\n  /* border: 3px; */\n  /* background-color: red; */\n/* } */\n\ntable.cv-entry {\n  border: 0px;\n}\n\n.cv-entry tbody tr td,\ntbody tr th {\n  background-color: #ffffff;\n  border: 0px;\n}\n\ndiv.cv-entry {\n  margin: 0px 10px 10px 10px;\n  text-align: left\n}\n\nspan.doctitle {\n  font-weight: bold;\n  display: block;\n  /*color:#dcb975;*/\n  color:#000000;\n  text-align: left\n}\n\nspan.docauthors {\n  display: block;\n}\n\nspan.docproc {\n  font-style: italic;\n}\n\nspan.doclink {\n  display:block;\n}\n\nimg.social-icon {\n  display:inline;\n  margin: 0px;\n  vertical-align:middle;\n}\n\ntable.contact-info {\n  border: 0px;\n}\n\n.contact-info tbody tr td,\ntbody tr th {\n  background-color: #ffffff;\n  border: 0px;\n}\n"
  },
  {
    "path": "tests_data/basic/csv/magika_test.csv",
    "content": "Name,Value1,Value2,Value3\r\nTest1,1,10,100\r\nTest2,2,20,200\r\nTest3,3,30,300\r\nTest4,4,40,400\r\nTest5,5,50,500"
  },
  {
    "path": "tests_data/basic/dockerfile/Dockerfile",
    "content": "# syntax=docker/dockerfile:1\n\nARG PYTHON_VERSION=3.11\nFROM python:${PYTHON_VERSION}-slim as base\n\nWORKDIR /magika\n\n# This requires buildx\n# RUN --mount=type=cache,target=/root/.cache/pip \\\n#     pip install magika\n\nRUN pip install magika\n\nENTRYPOINT [\"magika\"]\n"
  },
  {
    "path": "tests_data/basic/eml/sample.eml",
    "content": "Delivered-To: samplelefay@gmail.com\r\nReceived: by 2002:a05:6e04:60c8:b0:358:f759:ca6 with SMTP id do8csp3407339imd;\r\n        Sat, 24 May 2025 13:12:33 -0700 (PDT)\r\nX-Google-Smtp-Source: AGHT+IE2JLkhE/TNDdtqP0aHL2ofqoWkUhPk4z8fSmHFWHbViXC85kImFXv+Ve6tpwRtChavAQEY\r\nX-Received: by 2002:a05:6402:28a5:b0:602:b6fd:150a with SMTP id 4fb4d7f45d1cf-602da9d8a34mr2169618a12.33.1748117553220;\r\n        Sat, 24 May 2025 13:12:33 -0700 (PDT)\r\nARC-Seal: i=1; a=rsa-sha256; t=1748117553; cv=none;\r\n        d=google.com; s=arc-20240605;\r\n        b=SddPvSqiWCZ3hcLsPoKY5mgF8G90kpPEboy8YkxV4AqqgrNfPla4NETejNpnuJvLJI\r\n         byt+SaYq7Ve2RoJoDkZSBaT4J2RnU0MA4QW3F7YtWHrCgyo+6O/RvB9/ZTdrrdG58Z54\r\n         p0jFd0pWF+pTEhh71+Y2rDrbDwq9D2omwUp22IaFUZsL7KiLOlKQAMdaIF1GlmAPwEKM\r\n         epNVkNFNIzE/T+051fMcrUm6dIVe9xBIIMv5HJ4px+beQM/cHy7Jgj9Ryxh7l7CYpwmJ\r\n         dJwvYqQRrZ3XAv65p/A55YDtv5OFk82723B9MeW/m5G1XB8T4rFsBbcpcJQxrC5OfXL5\r\n         P6lw==\r\nARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605;\r\n        h=content-transfer-encoding:mime-version:list-unsubscribe\r\n         :list-unsubscribe-post:message-id:subject:from:to:date\r\n         :dkim-signature;\r\n        bh=GBTxS/zUYzBpSA9sl7FQde1QfK2F+C1PfO1VwyLNlxU=;\r\n        fh=GMaPNuYHJRKUGsaZzBcjeSmH+fuDvdR3OD4VRmo8LKk=;\r\n        b=Rrv9c6Y6Z0aAUmlV7kkSgHmjexkqZaryb+ewpcGXZAJLjwG1xT36eEr+xNSza5JUl9\r\n         C+bPB/Roa4aLghXgOUie709Unq91eI6EoMKxltvYryLMShqQf2lUPZLJBH324ARNB0rZ\r\n         G+IjcJwQb8m1ygBkN9f72s6QW9a2L1uWoNZ9FxIEEsP2RZMiLcORzEhC4mNFdydQm7gA\r\n         90EKqelXscw7mii3yfDfmoCb5fnbodG4SDhvQxpRLmv1RosBpDDh92jbKcH8MkYkbzfX\r\n         Bnylixa3LtK0On61xFeg/WyEIMBRgaBP2pBzy9nJhwsx8+1nfGR/ZhmPqUrlz2WJ1FOo\r\n         46ag==;\r\n        dara=google.com\r\nARC-Authentication-Results: i=1; mx.google.com;\r\n       dkim=pass header.i=@anonymousemail.eu header.s=mail header.b=qPEZ9skF;\r\n       spf=pass (google.com: domain of noreply@anonymousemail.eu designates 5.45.184.142 as permitted sender) smtp.mailfrom=noreply@anonymousemail.eu;\r\n       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=anonymousemail.eu\r\nReturn-Path: \r\nReceived: from s9.inboxpress.eu (s9.inboxpress.eu. [5.45.184.142])\r\n        by mx.google.com with ESMTPS id 4fb4d7f45d1cf-6045eb06b43si298940a12.544.2025.05.24.13.12.31\r\n        for \r\n        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\r\n        Sat, 24 May 2025 13:12:33 -0700 (PDT)\r\nReceived-SPF: pass (google.com: domain of noreply@anonymousemail.eu designates 5.45.184.142 as permitted sender) client-ip=5.45.184.142;\r\nAuthentication-Results: mx.google.com;\r\n       dkim=pass header.i=@anonymousemail.eu header.s=mail header.b=qPEZ9skF;\r\n       spf=pass (google.com: domain of noreply@anonymousemail.eu designates 5.45.184.142 as permitted sender) smtp.mailfrom=noreply@anonymousemail.eu;\r\n       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=anonymousemail.eu\r\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anonymousemail.eu;\r\n\ts=mail; t=1748117551;\r\n\tbh=GBTxS/zUYzBpSA9sl7FQde1QfK2F+C1PfO1VwyLNlxU=;\r\n\th=Date:To:From:Subject:List-Unsubscribe:From;\r\n\tb=qPEZ9skFQk06ORoxmFFYnqX188C7IfQLRyph7CHfLArHqw0r8RSzpQfpUjpnmRKED\r\n\t wrm2jqxdj6L+2sNUEZGvs7HznrOrhYY9/RSzVOvGBQ0PMl7Fk0cvU32rm6Tu8+eGYp\r\n\t 7KNpMO4TCJh6oOZ4x33cU+3HHeEZSjPCYO8zBujJherHw5fTEx4VyfuBQpAioVbBkf\r\n\t B5CGscCax1bnoQJBbpFENs6vHZgXvjJcgSxD01g2Hmv5a9mV/7c4VD6WsrvePcEsW+\r\n\t tKTHJBXw1b+xyh4Jv3JjP4KGO2SWTlD+F5bIQxDRGiUtg8KXSHGBaKaGTEhfEduoXX\r\n\t PTxWm9Kg9SC6w==\r\nReceived: from authenticated-user (s9.inboxpress.eu [5.45.184.142])\r\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\r\n\t key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)\r\n\t(No client certificate requested)\r\n\tby s9.inboxpress.eu (Postfix) with ESMTPSA id A66156601C\r\n\tfor ; Sat, 24 May 2025 20:12:31 +0000 (UTC)\r\nDate: Sat, 24 May 2025 20:12:31 +0000\r\nTo: samplelefay@gmail.com\r\nFrom: Anonymousemail \r\nSubject: =?UTF-8?Q?Welcome_to_your_sample_email._=F0=9F=99=83?=\r\nMessage-ID: <1182da168e8347704d33d53e86784b69@anonymousemail.eu>\r\nList-Unsubscribe-Post: List-Unsubscribe=One-Click\r\nList-Unsubscribe: \r\nMIME-Version: 1.0\r\nContent-Type: multipart/alternative;\r\n boundary=\"b1=_jkjvSBcyLEOb6jiwQT8Nx1r1IwoULznyMtSrrJRQO0A\"\r\nContent-Transfer-Encoding: 8bit\r\n\r\n--b1=_jkjvSBcyLEOb6jiwQT8Nx1r1IwoULznyMtSrrJRQO0A\r\nContent-Type: text/plain; charset=UTF-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\nPowered by Anonymousemail\r\n\r\nYou've been sent a sample email! 🎉\r\n\r\n--b1=_jkjvSBcyLEOb6jiwQT8Nx1r1IwoULznyMtSrrJRQO0A\r\nContent-Type: text/html; charset=UTF-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n
Powered by Anonymousemail
You've been sent a sample email! 🎉
\r\n\r\n\r\n--b1=_jkjvSBcyLEOb6jiwQT8Nx1r1IwoULznyMtSrrJRQO0A--\r\n\r\n"
  },
  {
    "path": "tests_data/basic/empty/empty_file",
    "content": ""
  },
  {
    "path": "tests_data/basic/handlebars/example.handlebars",
    "content": "{{#if isUserLoggedIn}}\n  Welcome, {{username}}!\n  {{#each notifications}}\n    
{{this}}
\n  {{/each}}\n{{else}}\n  Please log in.\n{{/if}}\n"
  },
  {
    "path": "tests_data/basic/html/doc.html",
    "content": "\r\n301 Moved Permanently\r\n\r\n
301 Moved Permanently
\r\n
nginx/1.18.0 (Ubuntu)
\r\n\r\n\r\n"
  },
  {
    "path": "tests_data/basic/ignorefile/example.ignorefile",
    "content": ".DS_Store\n.classpath\n.project\n.target/\n.settings/\n"
  },
  {
    "path": "tests_data/basic/ignorefile/other.ignorefile",
    "content": "# Standard build outputs\n/dist\n/build\n*.log\n*.tmp\n.DS_Store\nThumbs.db\nnode_modules/\n!package.json\n**/*.min.js\ncache/?lockfile\ntemp-*/\n/log/*\nsecret\n"
  },
  {
    "path": "tests_data/basic/ini/doc.ini",
    "content": "[pytest]\nlog_cli = 1\nlog_level = WARNING\nfilterwarnings =\n    ignore::DeprecationWarning\n\nmarkers =\n    smoketest\n    slow\n\n"
  },
  {
    "path": "tests_data/basic/javascript/code.js",
    "content": "(function(){/*\n\n Copyright The Closure Library Authors.\n SPDX-License-Identifier: Apache-2.0\n*/\nvar n=this||self,p=function(a,b){a=a.split(\".\");var c=n;a[0]in c||\"undefined\"==typeof c.execScript||c.execScript(\"var \"+a[0]);for(var d;a.length&&(d=a.shift());)a.length||void 0===b?c=c[d]&&c[d]!==Object.prototype[d]?c[d]:c[d]={}:c[d]=b};function q(){for(var a=r,b={},c=0;c>4);64!=h&&(c+=String.fromCharCode(f<<4&240|h>>2),64!=g&&(c+=String.fromCharCode(h<<6&192|g)))}};var w={},y=function(a){w.TAGGING=w.TAGGING||[];w.TAGGING[a]=!0};var ba=Array.isArray,ca=function(a,b){if(a&&ba(a))for(var c=0;cc?a.href:a.href.substr(0,c));a=c;break;case \"protocol\":a=d;break;case \"host\":a=a.hostname.replace(N,\"\").toLowerCase();c&&(c=/^www\\d*\\./.exec(a))&&c[0]&&(a=a.substr(c[0].length));break;case \"port\":a=String(Number(a.port)||(\"http\"===d?80:\"https\"===d?443:\"\"));break;case \"path\":a.pathname||a.hostname||y(1);a=\"/\"===a.pathname.substr(0,1)?a.pathname:\"/\"+a.pathname;a=a.split(\"/\");0<=[].indexOf(a[a.length-1])&&(a[a.length-1]=\"\");a=a.join(\"/\");break;case \"query\":a=a.search.replace(\"?\",\"\");break;case \"extension\":a=\na.pathname.split(\".\");a=1>2;l=(l&3)<<4|t>>4;t=(t&15)<<2|x>>6;x&=63;m||(x=64,k||(t=64));h.push(r[ra],r[l],r[t],r[x])}f.call(e,h.join(\"\"))}}a=b.join(\"*\");return[\"1\",la(a),a].join(\"*\")};\nfunction la(a,b){a=[J.userAgent,(new Date).getTimezoneOffset(),J.userLanguage||J.language,Math.floor((new Date(Date.now())).getTime()/60/1E3)-(void 0===b?0:b),a].join(\"*\");if(!(b=S)){b=Array(256);for(var c=0;256>c;c++){for(var d=c,e=0;8>e;e++)d=d&1?d>>>1^3988292384:d>>>1;b[c]=d}}S=b;b=4294967295;for(c=0;c>>8^S[(b^a.charCodeAt(c))&255];return((b^-1)>>>0).toString(36)}\nfunction ma(a){return function(b){var c=R(G.location.href),d=c.search.replace(\"?\",\"\");a:{var e=d.split(\"&\");for(var f=0;fb;++b){var c=ja.exec(a);if(c){var d=c;break b}a=decodeURIComponent(a)}d=void 0}if(d&&\"1\"===d[1]){var e=d[2],f=d[3];b:{for(d=0;3>d;++d)if(e===la(f,d)){var h=!0;break b}h=!1}if(h){var g=f;break a}y(7)}}g=void 0}e=g;if(void 0!==e){g={};var k=e?e.split(\"*\"):[];for(e=0;e+1>21:b}return b};var $c=function(a){this.C=a||[]};$c.prototype.set=function(a){this.C[a]=!0};$c.prototype.get=function(a){return this.C[a]};$c.prototype.encode=function(){for(var a=[],b=0;b\\x3c/script>'))):(c=M.createElement(\"script\"),c.type=\"text/javascript\",c.async=!0,c.src=ff.createScriptURL(a),d&&(c.onload=d),e&&(c.onerror=e),b&&(c.id=b),g&&c.setAttribute(\"nonce\",\ng),a=M.getElementsByTagName(\"script\")[0],a.parentNode.insertBefore(c,a))}},be=function(a,b){return E(M.location[b?\"href\":\"search\"],a)},E=function(a,b){return(a=a.match(\"(?:&|#|\\\\?)\"+K(b).replace(/([.*+?^=!:${}()|\\[\\]\\/\\\\])/g,\"\\\\$1\")+\"=([^&#]*)\"))&&2==a.length?a[1]:\"\"},xa=function(){var a=\"\"+M.location.hostname;return 0==a.indexOf(\"www.\")?a.substring(4):a},de=function(a,b){var c=a.indexOf(b);if(5==c||6==c)if(a=a.charAt(c+b.length),\"/\"==a||\"?\"==a||\"\"==a||\":\"==a)return!0;return!1},of=function(a,b){var c=\nM.referrer;if(/^(https?|android-app):\\/\\//i.test(c)){if(a)return c;a=\"//\"+M.location.hostname;if(!de(c,a))return b&&(b=a.replace(/\\./g,\"-\")+\".cdn.ampproject.org\",de(c,b))?void 0:c}},za=function(a,b){if(1==b.length&&null!=b[0]&&\"object\"===typeof b[0])return b[0];for(var c={},d=Math.min(a.length+1,b.length),e=0;eg.length||!c&&3!==g.length)a&&(a.na=!0);else if(Number(g[1])){d[b[e].ja]?a&&(a.pa=!0):d[b[e].ja]=[];var ca={version:g[0],timestamp:1E3*Number(g[1]),qa:g[2]};c&&3=b.length)wc(a,b,c);else if(8192>=b.length)x(a,b,c)||wd(a,b,c)||wc(a,b,c);else throw ge(\"len\",b.length),new Da(b.length);},pe=function(a,b,\nc,d){d=d||ua;wd(a+\"?\"+b,\"\",d,c)},wc=function(a,b,c){var d=ta(a+\"?\"+b);d.onload=d.onerror=function(){d.onload=null;d.onerror=null;c()}},wd=function(a,b,c,d){var e=O.XMLHttpRequest;if(!e)return!1;var g=new e;if(!(\"withCredentials\"in g))return!1;a=a.replace(/^http:/,\"https:\");g.open(\"POST\",a,!0);g.withCredentials=!0;g.setRequestHeader(\"Content-Type\",\"text/plain\");g.onreadystatechange=function(){if(4==g.readyState){if(d&&\"text/plain\"===g.getResponseHeader(\"Content-Type\"))try{Ea(d,g.responseText,c)}catch(ca){ge(\"xhr\",\n\"rsp\"),c()}else c();g=null}};g.send(b);return!0},Ea=function(a,b,c){if(1>b.length)ge(\"xhr\",\"ver\",\"0\"),c();else if(3=100*R(a,Ka))throw\"abort\";}function Ma(a){if(G(P(a,Na)))throw\"abort\";}function Oa(){var a=M.location.protocol;if(\"http:\"!=a&&\"https:\"!=a)throw\"abort\";}\nfunction pf(a){var b=!1,c=!1;if(vd.get(89)){c=!0;var d=a.get(kb),e=M.location;if(e){var g=e.pathname||\"\";\"/\"!=g.charAt(0)&&(g=\"/\"+g);e=e.protocol+\"//\"+e.hostname+g+e.search;d&&0===d.indexOf(e)||(b=!0)}}!c&&vd.get(90)&&(c=!0,d=a.get(lb),e=of(!!a.get(ec),!!a.get(Kd)),d!==e&&(b=!0));!c&&vd.get(91)&&(c=!0,a.get(qf)!==M.title&&(b=!0));return c&&!b}\nfunction Pa(a){try{O.navigator.sendBeacon?J(42):O.XMLHttpRequest&&\"withCredentials\"in new O.XMLHttpRequest&&J(40)}catch(c){}a.set(ld,Td(a),!0);a.set(Ac,R(a,Ac)+1);var b=[];ue.map(function(c,d){d.F&&(c=a.get(c),void 0!=c&&c!=d.defaultValue&&(\"boolean\"==typeof c&&(c*=1),b.push(d.F+\"=\"+K(\"\"+c))))});!1===a.get(xe)&&b.push(\"npa=1\");b.push(\"z=\"+Bd());pf(a)&&J(109);a.set(Ra,b.join(\"&\"),!0)}\nfunction Sa(a){var b=P(a,fa);!b&&a.get(Vd)&&(b=\"beacon\");var c=P(a,gd),d=P(a,oe),e=c||(d||bd(!1)+\"\")+\"/collect\",g=a.Z(Ia),ca=P(a,Ra),l=P(a,Na);switch(P(a,ad)){case \"d\":e=c||(d||bd(!1)+\"\")+\"/j/collect\";b=a.get(qe)||void 0;pe(e,ca,b,g);break;default:b?(g=g||ua,\"image\"==b?wc(e,ca,g):\"xhr\"==b&&wd(e,ca,g)||\"beacon\"==b&&x(e,ca,g)||ba(e,ca,g)):ba(e,ca,g)}ca=h(l);g=ca.hitcount;ca.hitcount=g?g+1:1;ca.first_hit||(ca.first_hit=(new Date).getTime());delete h(l).pending_experiments;a.set(Ia,ua,!0);if(rf(a))if(ca=\nP(a,Na),l=sf[ca])for(ca=0;cag.length&&g.push&&g.push(uf(a));else vf[ca]=vf[ca]||[],30>vf[ca].length&&vf[ca].push(uf(a))}function Hc(a){qc().expId&&a.set(Nc,qc().expId);qc().expVar&&a.set(Oc,qc().expVar);var b=P(a,Na);if(b=h(b).pending_experiments){var c=[];for(d in b)b.hasOwnProperty(d)&&b[d]&&c.push(encodeURIComponent(d)+\".\"+encodeURIComponent(b[d]));var d=c.join(\"!\")}else d=void 0;d&&((b=a.get(m))&&(d=b+\"!\"+d),a.set(m,d,!0))}\nfunction cd(){if(O.navigator&&\"preview\"==O.navigator.loadPurpose)throw\"abort\";}function yd(a){var b=O.gaDevIds||[];if(ka(b)){var c=a.get(\"&did\");qa(c)&&0b-9E5};var hd=function(){return Math.round(2147483647*Math.random())},Bd=function(){try{var a=new Uint32Array(1);O.crypto.getRandomValues(a);return a[0]&2147483647}catch(b){return hd()}};function Ta(a){var b=R(a,Ua);500<=b&&J(15);var c=P(a,Va);if(\"transaction\"!=c&&\"item\"!=c){c=R(a,Wa);var d=(new Date).getTime(),e=R(a,Xa);0==e&&a.set(Xa,d);e=Math.round(2*(d-e)/1E3);0=c)throw\"abort\";a.set(Wa,--c)}a.set(Ua,++b)};var Ya=function(){this.data=new ee};Ya.prototype.get=function(a){var b=$a(a),c=this.data.get(a);b&&void 0==c&&(c=ea(b.defaultValue)?b.defaultValue():b.defaultValue);return b&&b.Z?b.Z(this,a,c):c};var P=function(a,b){a=a.get(b);return void 0==a?\"\":\"\"+a},R=function(a,b){a=a.get(b);return void 0==a||\"\"===a?0:Number(a)};Ya.prototype.Z=function(a){return(a=this.get(a))&&ea(a)?a:ua};\nYa.prototype.set=function(a,b,c){if(a)if(\"object\"===typeof a)for(var d in a)a.hasOwnProperty(d)&&ab(this,d,a[d],c);else ab(this,a,b,c)};\nvar ab=function(a,b,c,d){if(void 0!=c)switch(b){case Na:wb.test(c)}var e=$a(b);e&&e.o?e.o(a,b,c,d):a.data.set(b,c,d)},gf={hitPayload:88,location:89,referrer:90,title:91,buildHitTask:93,sendHitTask:94,displayFeaturesTask:95,customTask:97,cookieName:98,cookieDomain:99,cookiePath:100,cookieExpires:101,cookieUpdate:102,cookieFlags:103,storage:104,_x_19:105,transportUrl:106,allowAdFeatures:107,sampleRate:108};\nfunction hf(a,b){var c=gf[a];c&&J(c);\"displayFeaturesTask\"===a&&void 0==b&&J(96);/.*Task$/.test(a)&&J(92)}function mf(a,b){if(a)if(\"object\"===typeof a)for(var c in a)a.hasOwnProperty(c)&&hf(c,b);else hf(a,b)};var ue=new ee,ve=[],bb=function(a,b,c,d,e){this.name=a;this.F=b;this.Z=d;this.o=e;this.defaultValue=c};function $a(a){var b=ue.get(a);if(!b)for(var c=0;c=b?!1:!0},gc=function(a){var b={};if(Ec(b)||Fc(b)){var c=b[Eb];void 0==c||Infinity==c||isNaN(c)||(0c)a[b]=void 0},Fd=function(a){return function(b){if(\"pageview\"==b.get(Va)&&!a.I){a.I=!0;var c=aa(b),d=0a.length)J(12);else{for(var d=[],e=0;e=a&&d.push({hash:ca[0],R:e[g],O:ca})}if(0!=d.length)return 1==d.length?d[0]:Zc(b,d)||Zc(c,d)||Zc(null,d)||d[0]}function Zc(a,b){if(null==a)var c=a=1;else c=La(a),a=La(D(a,\".\")?a.substring(1):\".\"+a);for(var d=0;de?\"?\":\"&\")+a+b.substring(e);b=b.replace(/&+_ga=/,\"&_ga=\");return b=b.replace(RegExp(\"&+_gac=\"),\"&_gac=\")},rd=function(a,b){if(b&&b.action)if(\"get\"==b.method.toLowerCase()){a=a.target.get(\"linkerParam\").split(\"&\");\nfor(var c=0;carguments.length)){if(\"string\"===typeof arguments[0]){var b=arguments[0];var c=[].slice.call(arguments,1)}else b=arguments[0]&&arguments[0][Va],c=arguments;b&&(c=za(me[b]||[],c),c[Va]=b,mf(c),this.model.set(c,void 0,!0),this.filters.D(this.model),this.model.data.m={})}};pc.prototype.ma=function(a,b){var c=this;u(a,c,b)||(v(a,function(){u(a,c,b)}),y(String(c.get(V)),a,void 0,b,!0))};\nfunction td(a,b){var c=P(a,U);a.data.set(la,\"_ga\"==c?\"_gid\":c+\"_gid\");if(\"cookie\"==P(a,ac)){hc=!1;c=Ca(P(a,U));c=Xd(a,c);if(!c){c=P(a,W);var d=P(a,$b)||xa();c=Xc(\"__utma\",d,c);void 0!=c?(J(10),c=c.O[1]+\".\"+c.O[2]):c=void 0}c&&(hc=!0);if(d=c&&!a.get(Hd))if(d=c.split(\".\"),2!=d.length)d=!1;else if(d=Number(d[1])){var e=R(a,Zb);d=d+e<(new Date).getTime()/1E3}else d=!1;d&&(c=void 0);c&&(a.data.set(xd,c),a.data.set(Q,c),(c=uc(a))&&a.data.set(I,c));a.get(je)&&(c=a.get(ce),d=a.get(ie),!c||d&&\"aw.ds\"!=d)&&\n(c={},d=(M?df(c):{})[P(a,Na)],le(c),d&&0!=d.length&&(c=d[0],a.data.set(fe,c.timestamp/1E3),a.data.set(ce,c.qa)));a.get(je)&&(c=a.get(Se),d={},e=(M?df(d,\"_gac_gb\",!0):{})[P(a,Na)],ef(d),e&&0!=e.length&&(d=e[0],e=d.qa,c&&c!==e||(d.labels&&d.labels.length&&(e+=\".\"+d.labels.join(\".\")),a.data.set(Te,d.timestamp/1E3),a.data.set(Se,e))))}if(a.get(Hd)){c=be(\"_ga\",!!a.get(cc));var g=be(\"_gl\",!!a.get(cc));d=De.get(a.get(cc));e=d._ga;g&&0=ca[0]||0>=\nca[1]?\"\":ca.join(\"x\");a.set(rb,c);c=a.set;var l;if((e=(e=O.navigator)?e.plugins:null)&&e.length)for(g=0;ga.split(\"/\")[0].indexOf(\":\")&&(a=g+d[2].substring(0,d[2].lastIndexOf(\"/\"))+\"/\"+a);b.href=a;c=kf(b);return{protocol:(b.protocol||\"\").toLowerCase(),host:c[0],port:c[1],path:c[2],query:b.search||\"\",url:a||\"\"}},cf=function(a,b){return a&&\n0<=a.indexOf(\"/\")?a:(b||bd(!1))+\"/plugins/ua/\"+a};var Z={ga:function(){Z.fa=[]}};Z.ga();Z.D=function(a){var b=Z.J.apply(Z,arguments);b=Z.fa.concat(b);for(Z.fa=[];0c;c++){var d=b[c].src;if(d&&0==d.indexOf(bd(!0)+\"/analytics\")){b=!0;break a}}b=!1}b&&(Ba=!0)}(O.gaplugins=O.gaplugins||{}).Linker=Dc;b=Dc.prototype;C(\"linker\",Dc);X(\"decorate\",b,b.ca,20);X(\"autoLink\",b,b.S,25);X(\"passthrough\",b,b.$,25);C(\"displayfeatures\",fd);C(\"adfeatures\",fd);Z.D.apply(N,a)}};var xf=N.N,yf=O[gb];yf&&yf.r?xf():z(xf);z(function(){Z.D([\"provide\",\"render\",ua])});})(window);\n"
  },
  {
    "path": "tests_data/basic/jinja/example.j2",
    "content": "{% extends \"base.html\" %}\n\n{% block content %}\n  
Welcome, {{ user.username }}
\n  \n  {% if user.is_admin %}\n    
Admin privileges granted. You can go to the admin dashboard.
\n  {% else %}\n    
Regular user access. You can view your profile.
\n  {% endif %}\n\n  
    \n  {% for item in items %}\n    
  • {{ item.name }} - {{ item.price | round(2) }} {{ currency_symbol }}
    • \n  {% else %}\n    
  • No items available.
    • \n  {% endfor %}\n  
\n\n  {% set now = current_time() %}\n  
Page generated at: {{ now.strftime('%Y-%m-%d %H:%M:%S') }}
\n{% endblock %}\n"
  },
  {
    "path": "tests_data/basic/json/doc.json",
    "content": "{\n    \"3gp\": {\n        \"name\": \"3gp\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ace\": {\n        \"name\": \"ace\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"aff\": {\n        \"name\": \"aff\",\n        \"extensions\": [\n            \"aff\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ai\": {\n        \"name\": \"ai\",\n        \"extensions\": [\n            \"ai\"\n        ],\n        \"mime_type\": \"application/pdf\",\n        \"group\": \"document\",\n        \"magic\": \"PDF document\",\n        \"description\": \"Adobe Illustrator Artwork\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"ai\",\n        \"target_label\": \"ai\",\n        \"correct_labels\": [\n            \"ai\",\n            \"pdf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"algol68\": {\n        \"name\": \"algol68\",\n        \"extensions\": [\n            \"a68\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"apk\": {\n        \"name\": \"apk\",\n        \"extensions\": [\n            \"apk\"\n        ],\n        \"mime_type\": \"application/vnd.android.package-archive\",\n        \"group\": \"executable\",\n        \"magic\": \"Java archive data\",\n        \"description\": \"Android package\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"apk\",\n        \"correct_labels\": [\n            \"apk\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"appleplist\": {\n        \"name\": \"appleplist\",\n        \"extensions\": [\n            \"plist\"\n        ],\n        \"mime_type\": \"application/x-plist\",\n        \"group\": \"application\",\n        \"magic\": \"Apple binary property list\",\n        \"description\": \"Android property list\",\n        \"vt_type\": \"appleplist\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"text\"\n        ],\n        \"model_target_label\": \"appleplist\",\n        \"target_label\": \"appleplist\",\n        \"correct_labels\": [\n            \"appleplist\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"arj\": {\n        \"name\": \"arj\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"asm\": {\n        \"name\": \"asm\",\n        \"extensions\": [\n            \"S\",\n            \"asm\"\n        ],\n        \"mime_type\": \"text/x-asm\",\n        \"group\": \"code\",\n        \"magic\": \"assembler source\",\n        \"description\": \"Assembly\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"asm\",\n        \"target_label\": \"asm\",\n        \"correct_labels\": [\n            \"asm\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"asp\": {\n        \"name\": \"asp\",\n        \"extensions\": [\n            \"aspx\",\n            \"asp\"\n        ],\n        \"mime_type\": \"text/html\",\n        \"group\": \"code\",\n        \"magic\": \"HTML document\",\n        \"description\": \"ASP source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"asp\",\n        \"target_label\": \"asp\",\n        \"correct_labels\": [\n            \"asp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"avi\": {\n        \"name\": \"avi\",\n        \"extensions\": [\n            \"avi\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ax\": {\n        \"name\": \"ax\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"batch\": {\n        \"name\": \"batch\",\n        \"extensions\": [\n            \"bat\"\n        ],\n        \"mime_type\": \"text/x-msdos-batch\",\n        \"group\": \"code\",\n        \"magic\": \"DOS batch file\",\n        \"description\": \"DOS batch file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"batch\",\n        \"target_label\": \"batch\",\n        \"correct_labels\": [\n            \"batch\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"bcad\": {\n        \"name\": \"bcad\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"bib\": {\n        \"name\": \"bib\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"bmp\": {\n        \"name\": \"bmp\",\n        \"extensions\": [\n            \"bmp\"\n        ],\n        \"mime_type\": \"image/bmp\",\n        \"group\": \"image\",\n        \"magic\": \"PC bitmap\",\n        \"description\": \"BMP image data\",\n        \"vt_type\": \"bmp\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"bmp\",\n        \"target_label\": \"bmp\",\n        \"correct_labels\": [\n            \"bmp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"bpl\": {\n        \"name\": \"bpl\",\n        \"extensions\": [\n            \"bpl\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"brainfuck\": {\n        \"name\": \"brainfuck\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"bzip\": {\n        \"name\": \"bzip\",\n        \"extensions\": [\n            \"bz2\",\n            \"tbz2\",\n            \"tar.bz2\"\n        ],\n        \"mime_type\": \"application/x-bzip2\",\n        \"group\": \"archive\",\n        \"magic\": \"bzip2 compressed data\",\n        \"description\": \"bzip2 compressed data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"bzip\",\n        \"target_label\": \"bzip\",\n        \"correct_labels\": [\n            \"bzip\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"c\": {\n        \"name\": \"c\",\n        \"extensions\": [\n            \"c\",\n            \"cpp\",\n            \"h\",\n            \"hpp\",\n            \"cc\"\n        ],\n        \"mime_type\": \"text/x-c\",\n        \"group\": \"code\",\n        \"magic\": \"C source\",\n        \"description\": \"C source\",\n        \"vt_type\": \"c,cpp\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"c\",\n        \"target_label\": \"c\",\n        \"correct_labels\": [\n            \"c\",\n            \"cpp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"cab\": {\n        \"name\": \"cab\",\n        \"extensions\": [\n            \"cab\"\n        ],\n        \"mime_type\": \"application/vnd.ms-cab-compressed\",\n        \"group\": \"archive\",\n        \"magic\": \"Microsoft Cabinet archive data\",\n        \"description\": \"Microsoft Cabinet archive data\",\n        \"vt_type\": \"cab\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"cab\",\n        \"target_label\": \"cab\",\n        \"correct_labels\": [\n            \"cab\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"cad\": {\n        \"name\": \"cad\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"cat\": {\n        \"name\": \"cat\",\n        \"extensions\": [\n            \"cat\"\n        ],\n        \"mime_type\": \"application/octet-stream\",\n        \"group\": \"application\",\n        \"magic\": \"data\",\n        \"description\": \"Windows Catalog file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"cat\",\n        \"target_label\": \"cat\",\n        \"correct_labels\": [\n            \"cat\",\n            \"ctl\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"cdf\": {\n        \"name\": \"cdf\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"chm\": {\n        \"name\": \"chm\",\n        \"extensions\": [\n            \"chm\"\n        ],\n        \"mime_type\": \"application/chm\",\n        \"group\": \"application\",\n        \"magic\": \"MS Windows HtmlHelp Data\",\n        \"description\": \"MS Windows HtmlHelp Data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"chm\",\n        \"target_label\": \"chm\",\n        \"correct_labels\": [\n            \"chm\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"clojure\": {\n        \"name\": \"clojure\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"cmake\": {\n        \"name\": \"cmake\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"cobol\": {\n        \"name\": \"cobol\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"coff\": {\n        \"name\": \"coff\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-coff\",\n        \"group\": \"executable\",\n        \"magic\": \"Intel 80386 COFF\",\n        \"description\": \"Intel 80386 COFF\",\n        \"vt_type\": \"coff\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"coff\",\n        \"target_label\": \"coff\",\n        \"correct_labels\": [\n            \"coff\",\n            \"exp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"coffee\": {\n        \"name\": \"coffee\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"com\": {\n        \"name\": \"com\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": \"com\",\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"cpl\": {\n        \"name\": \"cpl\",\n        \"extensions\": [\n            \"cpl\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"executable\",\n        \"magic\": \"PE32 executable\",\n        \"description\": \"PE Windows executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"cpl\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"cpp\": {\n        \"name\": \"cpp\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"crx\": {\n        \"name\": \"crx\",\n        \"extensions\": [\n            \"crx\"\n        ],\n        \"mime_type\": \"application/x-chrome-extension\",\n        \"group\": \"executable\",\n        \"magic\": \"Google Chrome extension\",\n        \"description\": \"Google Chrome extension\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"crx\",\n        \"target_label\": \"crx\",\n        \"correct_labels\": [\n            \"crx\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"cs\": {\n        \"name\": \"cs\",\n        \"extensions\": [\n            \"cs\"\n        ],\n        \"mime_type\": \"text/plain\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"C# source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"cs\",\n        \"target_label\": \"cs\",\n        \"correct_labels\": [\n            \"cs\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"css\": {\n        \"name\": \"css\",\n        \"extensions\": [\n            \"css\"\n        ],\n        \"mime_type\": \"text/css\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"CSS source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"css\",\n        \"target_label\": \"css\",\n        \"correct_labels\": [\n            \"css\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"csv\": {\n        \"name\": \"csv\",\n        \"extensions\": [\n            \"csv\"\n        ],\n        \"mime_type\": \"text/csv\",\n        \"group\": \"code\",\n        \"magic\": \"CSV text\",\n        \"description\": \"CSV document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"csv\",\n        \"target_label\": \"csv\",\n        \"correct_labels\": [\n            \"csv\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"ctl\": {\n        \"name\": \"ctl\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"dart\": {\n        \"name\": \"dart\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"deb\": {\n        \"name\": \"deb\",\n        \"extensions\": [\n            \"deb\"\n        ],\n        \"mime_type\": \"application/vnd.debian.binary-package\",\n        \"group\": \"archive\",\n        \"magic\": \"Debian binary package\",\n        \"description\": \"Debian binary package\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"deb\",\n        \"target_label\": \"deb\",\n        \"correct_labels\": [\n            \"deb\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"dex\": {\n        \"name\": \"dex\",\n        \"extensions\": [\n            \"dex\"\n        ],\n        \"mime_type\": \"application/x-android-dex\",\n        \"group\": \"executable\",\n        \"magic\": \"Dalvik dex file\",\n        \"description\": \"Dalvik dex file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"dex\",\n        \"target_label\": \"dex\",\n        \"correct_labels\": [\n            \"dex\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"dey\": {\n        \"name\": \"dey\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-android-dey\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"diff\": {\n        \"name\": \"diff\",\n        \"extensions\": [\n            \"diff\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"directory\": {\n        \"name\": \"directory\",\n        \"extensions\": [],\n        \"mime_type\": \"inode/directory\",\n        \"group\": \"inode\",\n        \"magic\": \"directory\",\n        \"description\": \"A directory\",\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": \"directory\",\n        \"correct_labels\": [\n            \"directory\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": false\n    },\n    \"dll\": {\n        \"name\": \"dll\",\n        \"extensions\": [\n            \"dll\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"executable\",\n        \"magic\": \"PE Windows executable\",\n        \"description\": \"PE Windows executable\",\n        \"vt_type\": \"pedll\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"dll\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"dm\": {\n        \"name\": \"dm\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"dmg\": {\n        \"name\": \"dmg\",\n        \"extensions\": [\n            \"dmg\"\n        ],\n        \"mime_type\": \"application/x-apple-diskimage\",\n        \"group\": \"archive\",\n        \"magic\": \"Apple disk image\",\n        \"description\": \"Apple disk image\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"dmg\",\n        \"target_label\": \"dmg\",\n        \"correct_labels\": [\n            \"dmg\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"doc\": {\n        \"name\": \"doc\",\n        \"extensions\": [\n            \"doc\"\n        ],\n        \"mime_type\": \"application/msword\",\n        \"group\": \"document\",\n        \"magic\": \"Composite Document File\",\n        \"description\": \"Microsoft Word CDF document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"cdf\"\n        ],\n        \"model_target_label\": \"cdf\",\n        \"target_label\": \"doc\",\n        \"correct_labels\": [\n            \"doc\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"dockerfile\": {\n        \"name\": \"dockerfile\",\n        \"extensions\": [\n            \"=Dockerfile\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"docx\": {\n        \"name\": \"docx\",\n        \"extensions\": [\n            \"docx\",\n            \"docm\"\n        ],\n        \"mime_type\": \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\",\n        \"group\": \"document\",\n        \"magic\": \"Microsoft Word 2007+\",\n        \"description\": \"Microsoft Word 2007+ document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"ooxml\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"docx\",\n        \"correct_labels\": [\n            \"docx\",\n            \"tmdx\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"dosmbr\": {\n        \"name\": \"dosmbr\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"dylib\": {\n        \"name\": \"dylib\",\n        \"extensions\": [\n            \"dylib\"\n        ],\n        \"mime_type\": \"application/x-mach-o\",\n        \"group\": \"executable\",\n        \"magic\": \"Mach-O executable\",\n        \"description\": \"Mach-O executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"macho\"\n        ],\n        \"model_target_label\": \"macho\",\n        \"target_label\": \"macho\",\n        \"correct_labels\": [\n            \"macho\",\n            \"dylib\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"elf\": {\n        \"name\": \"elf\",\n        \"extensions\": [\n            \"elf\",\n            \"so\"\n        ],\n        \"mime_type\": \"application/x-executable-elf\",\n        \"group\": \"executable\",\n        \"magic\": \"ELF executable\",\n        \"description\": \"ELF executable\",\n        \"vt_type\": \"elf\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"elf\"\n        ],\n        \"model_target_label\": \"elf\",\n        \"target_label\": \"elf\",\n        \"correct_labels\": [\n            \"elf\",\n            \"so\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"elixir\": {\n        \"name\": \"elixir\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"emf\": {\n        \"name\": \"emf\",\n        \"extensions\": [\n            \"emf\"\n        ],\n        \"mime_type\": \"application/octet-stream\",\n        \"group\": \"application\",\n        \"magic\": \"Windows Enhanced Metafile\",\n        \"description\": \"Windows Enhanced Metafile image data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"emf\",\n        \"target_label\": \"emf\",\n        \"correct_labels\": [\n            \"emf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"eml\": {\n        \"name\": \"eml\",\n        \"extensions\": [\n            \"eml\"\n        ],\n        \"mime_type\": \"message/rfc822\",\n        \"group\": \"text\",\n        \"magic\": \"RFC 822 mail\",\n        \"description\": \"RFC 822 mail\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"eml\",\n        \"target_label\": \"eml\",\n        \"correct_labels\": [\n            \"eml\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"empty\": {\n        \"name\": \"empty\",\n        \"extensions\": [],\n        \"mime_type\": \"inode/x-empty\",\n        \"group\": \"inode\",\n        \"magic\": \"empty\",\n        \"description\": \"Empty file\",\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": \"empty\",\n        \"correct_labels\": [\n            \"empty\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": false\n    },\n    \"epub\": {\n        \"name\": \"epub\",\n        \"extensions\": [\n            \"epub\"\n        ],\n        \"mime_type\": \"application/epub+zip\",\n        \"group\": \"document\",\n        \"magic\": \"EPUB document\",\n        \"description\": \"EPUB document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"epub\",\n        \"correct_labels\": [\n            \"epub\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"erlang\": {\n        \"name\": \"erlang\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ese\": {\n        \"name\": \"ese\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-ms-ese\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"exe\": {\n        \"name\": \"exe\",\n        \"extensions\": [\n            \"exe\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"executable\",\n        \"magic\": \"ELF executable\",\n        \"description\": \"ELF executable\",\n        \"vt_type\": \"peexe\",\n        \"datasets\": [\n            \"vt-ext\",\n            \"vt-ext-malicious\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"exe\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"exp\": {\n        \"name\": \"exp\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"flac\": {\n        \"name\": \"flac\",\n        \"extensions\": [\n            \"flac\"\n        ],\n        \"mime_type\": \"audio/flac\",\n        \"group\": \"audio\",\n        \"magic\": \"FLAC audio bitstream data\",\n        \"description\": \"FLAC audio bitstream data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"flac\",\n        \"target_label\": \"flac\",\n        \"correct_labels\": [\n            \"flac\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"fortran\": {\n        \"name\": \"fortran\",\n        \"extensions\": [\n            \"f90\",\n            \"f95\",\n            \"f03\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"fpx\": {\n        \"name\": \"fpx\",\n        \"extensions\": [\n            \"fpx\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": \"fpx\",\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"gif\": {\n        \"name\": \"gif\",\n        \"extensions\": [\n            \"gif\"\n        ],\n        \"mime_type\": \"image/gif\",\n        \"group\": \"image\",\n        \"magic\": \"GIF image data\",\n        \"description\": \"GIF image data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"gif\",\n        \"target_label\": \"gif\",\n        \"correct_labels\": [\n            \"gif\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"go\": {\n        \"name\": \"go\",\n        \"extensions\": [\n            \"go\"\n        ],\n        \"mime_type\": \"text/x-golang\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Golang source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"go\",\n        \"target_label\": \"go\",\n        \"correct_labels\": [\n            \"go\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"gpx\": {\n        \"name\": \"gpx\",\n        \"extensions\": [\n            \"gpx\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"groovy\": {\n        \"name\": \"groovy\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"gzip\": {\n        \"name\": \"gzip\",\n        \"extensions\": [\n            \"gz\",\n            \"gzip\",\n            \"tgz\",\n            \"tar.gz\"\n        ],\n        \"mime_type\": \"application/gzip\",\n        \"group\": \"archive\",\n        \"magic\": \"gzip compressed data\",\n        \"description\": \"gzip compressed data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"gzip\",\n        \"target_label\": \"gzip\",\n        \"correct_labels\": [\n            \"gzip\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"haskell\": {\n        \"name\": \"haskell\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"hfs\": {\n        \"name\": \"hfs\",\n        \"extensions\": [\n            \"hfs\"\n        ],\n        \"mime_type\": \"application/x-hfs\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"hlp\": {\n        \"name\": \"hlp\",\n        \"extensions\": [\n            \"hlp\"\n        ],\n        \"mime_type\": \"application/winhlp\",\n        \"group\": \"application\",\n        \"magic\": \"MS Windows help\",\n        \"description\": \"MS Windows help\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"hlp\",\n        \"target_label\": \"hlp\",\n        \"correct_labels\": [\n            \"hlp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"hta\": {\n        \"name\": \"hta\",\n        \"extensions\": [\n            \"hta\"\n        ],\n        \"mime_type\": \"application/hta\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"html\": {\n        \"name\": \"html\",\n        \"extensions\": [\n            \"html\",\n            \"htm\",\n            \"xhtml\",\n            \"xht\"\n        ],\n        \"mime_type\": \"text/html\",\n        \"group\": \"code\",\n        \"magic\": \"HTML document\",\n        \"description\": \"HTML document\",\n        \"vt_type\": \"html\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"html\",\n        \"target_label\": \"html\",\n        \"correct_labels\": [\n            \"html\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"hve\": {\n        \"name\": \"hve\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ico\": {\n        \"name\": \"ico\",\n        \"extensions\": [\n            \"ico\"\n        ],\n        \"mime_type\": \"image/vnd.microsoft.icon\",\n        \"group\": \"image\",\n        \"magic\": \"MS Windows icon resource\",\n        \"description\": \"MS Windows icon resource\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"ico\",\n        \"target_label\": \"ico\",\n        \"correct_labels\": [\n            \"ico\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"img\": {\n        \"name\": \"img\",\n        \"extensions\": [\n            \"img\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ini\": {\n        \"name\": \"ini\",\n        \"extensions\": [\n            \"ini\"\n        ],\n        \"mime_type\": \"text/plain\",\n        \"group\": \"text\",\n        \"magic\": \"Generic INItialization configuration\",\n        \"description\": \"INI configuration file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"ini\",\n        \"target_label\": \"ini\",\n        \"correct_labels\": [\n            \"ini\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"internetshortcut\": {\n        \"name\": \"internetshortcut\",\n        \"extensions\": [\n            \"url\"\n        ],\n        \"mime_type\": \"application/x-mswinurl\",\n        \"group\": \"application\",\n        \"magic\": \"MS Windows 95 Internet shortcut\",\n        \"description\": \"MS Windows Internet shortcut\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"internetshortcut\",\n        \"target_label\": \"internetshortcut\",\n        \"correct_labels\": [\n            \"internetshortcut\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"iosapp\": {\n        \"name\": \"iosapp\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"iso\": {\n        \"name\": \"iso\",\n        \"extensions\": [\n            \"iso\"\n        ],\n        \"mime_type\": \"application/x-iso9660-image\",\n        \"group\": \"archive\",\n        \"magic\": \"ISO 9660 CD-ROM filesystem data\",\n        \"description\": \"ISO 9660 CD-ROM filesystem data\",\n        \"vt_type\": \"isoimage\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"iso\",\n        \"target_label\": \"iso\",\n        \"correct_labels\": [\n            \"iso\",\n            \"udf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"jar\": {\n        \"name\": \"jar\",\n        \"extensions\": [\n            \"jar\"\n        ],\n        \"mime_type\": \"application/java-archive\",\n        \"group\": \"archive\",\n        \"magic\": \"Java archive data (JAR)\",\n        \"description\": \"Java archive data (JAR)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"jar\",\n        \"target_label\": \"jar\",\n        \"correct_labels\": [\n            \"jar\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"java\": {\n        \"name\": \"java\",\n        \"extensions\": [\n            \"java\"\n        ],\n        \"mime_type\": \"text/x-java\",\n        \"group\": \"code\",\n        \"magic\": \"Java source\",\n        \"description\": \"Java source\",\n        \"vt_type\": \"java\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"java\",\n        \"target_label\": \"java\",\n        \"correct_labels\": [\n            \"java\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"javabytecode\": {\n        \"name\": \"javabytecode\",\n        \"extensions\": [\n            \"class\"\n        ],\n        \"mime_type\": \"application/x-java-applet\",\n        \"group\": \"executable\",\n        \"magic\": \"compiled Java class data\",\n        \"description\": \"Java compiled bytecode\",\n        \"vt_type\": \"class\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"javabytecode\",\n        \"target_label\": \"javabytecode\",\n        \"correct_labels\": [\n            \"javabytecode\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"javascript\": {\n        \"name\": \"javascript\",\n        \"extensions\": [\n            \"js\"\n        ],\n        \"mime_type\": \"application/javascript\",\n        \"group\": \"code\",\n        \"magic\": \"Node.js script text executable\",\n        \"description\": \"Javascript source\",\n        \"vt_type\": \"javascript\",\n        \"datasets\": [\n            \"github\",\n            \"vt-ext\",\n            \"vt-ext-malicious\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"javascript\",\n        \"target_label\": \"javascript\",\n        \"correct_labels\": [\n            \"javascript\",\n            \"typescript\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"jpeg\": {\n        \"name\": \"jpeg\",\n        \"extensions\": [\n            \"jpg\",\n            \"jpeg\"\n        ],\n        \"mime_type\": \"image/jpeg\",\n        \"group\": \"image\",\n        \"magic\": \"JPEG image data\",\n        \"description\": \"JPEG image data\",\n        \"vt_type\": \"jpeg\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"jpeg\",\n        \"target_label\": \"jpeg\",\n        \"correct_labels\": [\n            \"jpeg\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"json\": {\n        \"name\": \"json\",\n        \"extensions\": [\n            \"json\"\n        ],\n        \"mime_type\": \"application/json\",\n        \"group\": \"code\",\n        \"magic\": \"JSON data\",\n        \"description\": \"JSON document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"json\",\n        \"target_label\": \"json\",\n        \"correct_labels\": [\n            \"json\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"julia\": {\n        \"name\": \"julia\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ko\": {\n        \"name\": \"ko\",\n        \"extensions\": [\n            \"ko\"\n        ],\n        \"mime_type\": \"application/x-executable-elf\",\n        \"group\": \"executable\",\n        \"magic\": \"ELF executable\",\n        \"description\": \"ELF executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"elf\"\n        ],\n        \"model_target_label\": \"elf\",\n        \"target_label\": \"elf\",\n        \"correct_labels\": [\n            \"elf\",\n            \"ko\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"kotlin\": {\n        \"name\": \"kotlin\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"latex\": {\n        \"name\": \"latex\",\n        \"extensions\": [\n            \"tex\"\n        ],\n        \"mime_type\": \"text/x-tex\",\n        \"group\": \"text\",\n        \"magic\": \"LaTeX document\",\n        \"description\": \"LaTeX document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"latex\",\n        \"target_label\": \"latex\",\n        \"correct_labels\": [\n            \"latex\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"lisp\": {\n        \"name\": \"lisp\",\n        \"extensions\": [\n            \"lisp\"\n        ],\n        \"mime_type\": \"text/x-lisp\",\n        \"group\": \"code\",\n        \"magic\": \"Lisp/Scheme program\",\n        \"description\": \"Lisp source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"lisp\",\n        \"target_label\": \"lisp\",\n        \"correct_labels\": [\n            \"lisp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"lnk\": {\n        \"name\": \"lnk\",\n        \"extensions\": [\n            \"lnk\"\n        ],\n        \"mime_type\": \"application/x-ms-shortcut\",\n        \"group\": \"application\",\n        \"magic\": \"MS Windows shortcut\",\n        \"description\": \"MS Windows shortcut\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"lnk\",\n        \"target_label\": \"lnk\",\n        \"correct_labels\": [\n            \"lnk\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"lua\": {\n        \"name\": \"lua\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"m3u\": {\n        \"name\": \"m3u\",\n        \"extensions\": [\n            \"m3u8\",\n            \"m3u\"\n        ],\n        \"mime_type\": \"text/plain\",\n        \"group\": \"application\",\n        \"magic\": \"M3U playlist\",\n        \"description\": \"M3U playlist\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"m3u\",\n        \"target_label\": \"m3u\",\n        \"correct_labels\": [\n            \"m3u\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"macho\": {\n        \"name\": \"macho\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-mach-o\",\n        \"group\": \"executable\",\n        \"magic\": \"Mach-O executable\",\n        \"description\": \"Mach-O executable\",\n        \"vt_type\": \"macho\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"macho\"\n        ],\n        \"model_target_label\": \"macho\",\n        \"target_label\": \"macho\",\n        \"correct_labels\": [\n            \"macho\",\n            \"dylib\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"maff\": {\n        \"name\": \"maff\",\n        \"extensions\": [\n            \"maff\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"makefile\": {\n        \"name\": \"makefile\",\n        \"extensions\": [\n            \"=Makefile\"\n        ],\n        \"mime_type\": \"text/x-makefile\",\n        \"group\": \"code\",\n        \"magic\": \"makefile script\",\n        \"description\": \"Makefile source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"makefile\",\n        \"target_label\": \"makefile\",\n        \"correct_labels\": [\n            \"makefile\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"markdown\": {\n        \"name\": \"markdown\",\n        \"extensions\": [\n            \"md\"\n        ],\n        \"mime_type\": \"text/markdown\",\n        \"group\": \"text\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Markdown document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"markdown\",\n        \"target_label\": \"markdown\",\n        \"correct_labels\": [\n            \"markdown\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"matlab\": {\n        \"name\": \"matlab\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"mht\": {\n        \"name\": \"mht\",\n        \"extensions\": [\n            \"mht\"\n        ],\n        \"mime_type\": \"application/x-mimearchive\",\n        \"group\": \"code\",\n        \"magic\": \"HTML document\",\n        \"description\": \"MHTML document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"mht\",\n        \"target_label\": \"mht\",\n        \"correct_labels\": [\n            \"mht\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"mkv\": {\n        \"name\": \"mkv\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"mov\": {\n        \"name\": \"mov\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"mp3\": {\n        \"name\": \"mp3\",\n        \"extensions\": [\n            \"mp3\"\n        ],\n        \"mime_type\": \"audio/mpeg\",\n        \"group\": \"audio\",\n        \"magic\": \"Audio file with ID3\",\n        \"description\": \"MP3 media file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"mp3\",\n        \"target_label\": \"mp3\",\n        \"correct_labels\": [\n            \"mp3\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"mp4\": {\n        \"name\": \"mp4\",\n        \"extensions\": [\n            \"mp4\"\n        ],\n        \"mime_type\": \"video/mp4\",\n        \"group\": \"video\",\n        \"magic\": \"ISO Media\",\n        \"description\": \"MP4 medial\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"mp4\",\n        \"target_label\": \"mp4\",\n        \"correct_labels\": [\n            \"mp4\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"mscompress\": {\n        \"name\": \"mscompress\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-ms-compress-szdd\",\n        \"group\": \"archive\",\n        \"magic\": \"MS Compress archive data\",\n        \"description\": \"MS Compress archive data\",\n        \"vt_type\": \"mscompress\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"mscompress\",\n        \"target_label\": \"mscompress\",\n        \"correct_labels\": [\n            \"mscompress\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"msi\": {\n        \"name\": \"msi\",\n        \"extensions\": [\n            \"msi\"\n        ],\n        \"mime_type\": \"application/x-msi\",\n        \"group\": \"archive\",\n        \"magic\": \"Composite Document File\",\n        \"description\": \"Microsoft Installer file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"cdf\"\n        ],\n        \"model_target_label\": \"msi\",\n        \"target_label\": \"msi\",\n        \"correct_labels\": [\n            \"msi\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"mst\": {\n        \"name\": \"mst\",\n        \"extensions\": [\n            \"mst\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"msvisio\": {\n        \"name\": \"msvisio\",\n        \"extensions\": [],\n        \"mime_type\": \"application/vnd.ms-visio.drawing.main+xml\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"mui\": {\n        \"name\": \"mui\",\n        \"extensions\": [\n            \"mui\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"application\",\n        \"magic\": \"PE Windows executable\",\n        \"description\": \"PE Windows executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"mui\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"mum\": {\n        \"name\": \"mum\",\n        \"extensions\": [\n            \"mum\"\n        ],\n        \"mime_type\": \"text/xml\",\n        \"group\": \"application\",\n        \"magic\": \"XML document\",\n        \"description\": \"Windows Update Package file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"mum\",\n        \"target_label\": \"mum\",\n        \"correct_labels\": [\n            \"mum\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"mun\": {\n        \"name\": \"mun\",\n        \"extensions\": [\n            \"mun\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"nim\": {\n        \"name\": \"nim\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"null\": {\n        \"name\": \"null\",\n        \"extensions\": [\n            \"null\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"object\": {\n        \"name\": \"object\",\n        \"extensions\": [\n            \"o\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"objectivec\": {\n        \"name\": \"objectivec\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ocaml\": {\n        \"name\": \"ocaml\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ocx\": {\n        \"name\": \"ocx\",\n        \"extensions\": [\n            \"ocx\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"executable\",\n        \"magic\": \"PE Windows executable\",\n        \"description\": \"PE Windows executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"ax\",\n            \"ocx\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"odex\": {\n        \"name\": \"odex\",\n        \"extensions\": [\n            \"odex\"\n        ],\n        \"mime_type\": \"application/x-executable-elf\",\n        \"group\": \"executable\",\n        \"magic\": \"ELF executable\",\n        \"description\": \"ODEX ELF executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"elf\"\n        ],\n        \"model_target_label\": \"odex\",\n        \"target_label\": \"odex\",\n        \"correct_labels\": [\n            \"odex\",\n            \"elf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"odp\": {\n        \"name\": \"odp\",\n        \"extensions\": [\n            \"odp\"\n        ],\n        \"mime_type\": \"application/vnd.oasis.opendocument.presentation\",\n        \"group\": \"document\",\n        \"magic\": \"OpenDocument Presentation\",\n        \"description\": \"OpenDocument Presentation\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\"\n        ],\n        \"model_target_label\": \"odp\",\n        \"target_label\": \"odp\",\n        \"correct_labels\": [\n            \"odp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"ods\": {\n        \"name\": \"ods\",\n        \"extensions\": [\n            \"ods\"\n        ],\n        \"mime_type\": \"application/vnd.oasis.opendocument.spreadsheet\",\n        \"group\": \"document\",\n        \"magic\": \"OpenDocument Spreadsheet\",\n        \"description\": \"OpenDocument Spreadsheet\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\"\n        ],\n        \"model_target_label\": \"ods\",\n        \"target_label\": \"ods\",\n        \"correct_labels\": [\n            \"ods\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"odt\": {\n        \"name\": \"odt\",\n        \"extensions\": [\n            \"odt\"\n        ],\n        \"mime_type\": \"application/vnd.oasis.opendocument.text\",\n        \"group\": \"document\",\n        \"magic\": \"OpenDocument Text\",\n        \"description\": \"OpenDocument Text\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\"\n        ],\n        \"model_target_label\": \"odt\",\n        \"target_label\": \"odt\",\n        \"correct_labels\": [\n            \"odt\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"ogg\": {\n        \"name\": \"ogg\",\n        \"extensions\": [\n            \"ogg\"\n        ],\n        \"mime_type\": \"audio/ogg\",\n        \"group\": \"audio\",\n        \"magic\": \"Ogg data\",\n        \"description\": \"Ogg data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"ogg\",\n        \"target_label\": \"ogg\",\n        \"correct_labels\": [\n            \"ogg\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"ole\": {\n        \"name\": \"ole\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ooxml\": {\n        \"name\": \"ooxml\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"outlook\": {\n        \"name\": \"outlook\",\n        \"extensions\": [],\n        \"mime_type\": \"application/vnd.ms-outlook\",\n        \"group\": \"application\",\n        \"magic\": \"CDFV2 Microsoft Outlook Message\",\n        \"description\": \"MS Outlook Message\",\n        \"vt_type\": \"outlook\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"cdf\"\n        ],\n        \"model_target_label\": \"cdf\",\n        \"target_label\": \"outlook\",\n        \"correct_labels\": [\n            \"outlook\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"palmos\": {\n        \"name\": \"palmos\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": \"palmos\",\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"pascal\": {\n        \"name\": \"pascal\",\n        \"extensions\": [\n            \"pascal\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"pbm\": {\n        \"name\": \"pbm\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"pcap\": {\n        \"name\": \"pcap\",\n        \"extensions\": [\n            \"pcap\",\n            \"pcapng\"\n        ],\n        \"mime_type\": \"application/vnd.tcpdump.pcap\",\n        \"group\": \"application\",\n        \"magic\": \"pcap capture file\",\n        \"description\": \"pcap capture file\",\n        \"vt_type\": \"pcap\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"pcap\",\n        \"target_label\": \"pcap\",\n        \"correct_labels\": [\n            \"pcap\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"pdf\": {\n        \"name\": \"pdf\",\n        \"extensions\": [\n            \"pdf\"\n        ],\n        \"mime_type\": \"application/pdf\",\n        \"group\": \"document\",\n        \"magic\": \"PDF document\",\n        \"description\": \"PDF document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"pdf\",\n        \"target_label\": \"pdf\",\n        \"correct_labels\": [\n            \"pdf\",\n            \"ai\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"pebin\": {\n        \"name\": \"pebin\",\n        \"extensions\": [\n            \"exe\",\n            \"dll\",\n            \"sys\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"executable\",\n        \"magic\": \"PE executable\",\n        \"description\": \"PE executable\",\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": false\n    },\n    \"pem\": {\n        \"name\": \"pem\",\n        \"extensions\": [\n            \"pem\"\n        ],\n        \"mime_type\": \"application/x-pem-file\",\n        \"group\": \"application\",\n        \"magic\": \"PEM certificate\",\n        \"description\": \"PEM certificate\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"pem\",\n        \"target_label\": \"pem\",\n        \"correct_labels\": [\n            \"pem\",\n            \"pgpkey\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"perl\": {\n        \"name\": \"perl\",\n        \"extensions\": [\n            \"pl\"\n        ],\n        \"mime_type\": \"text/x-perl\",\n        \"group\": \"code\",\n        \"magic\": \"Perl script text executable\",\n        \"description\": \"Perl source\",\n        \"vt_type\": \"perl\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"perl\",\n        \"target_label\": \"perl\",\n        \"correct_labels\": [\n            \"perl\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"pgpkey\": {\n        \"name\": \"pgpkey\",\n        \"extensions\": [],\n        \"mime_type\": \"application/pgp-keys\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"php\": {\n        \"name\": \"php\",\n        \"extensions\": [\n            \"php\"\n        ],\n        \"mime_type\": \"text/x-php\",\n        \"group\": \"code\",\n        \"magic\": \"PHP script\",\n        \"description\": \"PHP source\",\n        \"vt_type\": \"php\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"php\",\n        \"target_label\": \"php\",\n        \"correct_labels\": [\n            \"php\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"png\": {\n        \"name\": \"png\",\n        \"extensions\": [\n            \"png\"\n        ],\n        \"mime_type\": \"image/png\",\n        \"group\": \"image\",\n        \"magic\": \"PNG image data\",\n        \"description\": \"PNG image data\",\n        \"vt_type\": \"png\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"png\",\n        \"target_label\": \"png\",\n        \"correct_labels\": [\n            \"png\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"postscript\": {\n        \"name\": \"postscript\",\n        \"extensions\": [\n            \"ps\"\n        ],\n        \"mime_type\": \"application/postscript\",\n        \"group\": \"document\",\n        \"magic\": \"PostScript document text\",\n        \"description\": \"PostScript document\",\n        \"vt_type\": \"postscript\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"postscript\",\n        \"target_label\": \"postscript\",\n        \"correct_labels\": [\n            \"postscript\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"powershell\": {\n        \"name\": \"powershell\",\n        \"extensions\": [\n            \"ps1\"\n        ],\n        \"mime_type\": \"application/x-powershell\",\n        \"group\": \"code\",\n        \"magic\": \"a powershell script\",\n        \"description\": \"Powershell source\",\n        \"vt_type\": \"ps\",\n        \"datasets\": [\n            \"github\",\n            \"vt-ext\",\n            \"vt-ext-malicious\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"powershell\",\n        \"target_label\": \"powershell\",\n        \"correct_labels\": [\n            \"powershell\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"ppt\": {\n        \"name\": \"ppt\",\n        \"extensions\": [\n            \"ppt\"\n        ],\n        \"mime_type\": \"application/vnd.ms-powerpoint\",\n        \"group\": \"document\",\n        \"magic\": \"Composite Document File\",\n        \"description\": \"Microsoft PowerPoint CDF document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"cdf\"\n        ],\n        \"model_target_label\": \"cdf\",\n        \"target_label\": \"ppt\",\n        \"correct_labels\": [\n            \"ppt\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"pptx\": {\n        \"name\": \"pptx\",\n        \"extensions\": [\n            \"pptx\",\n            \"pptm\"\n        ],\n        \"mime_type\": \"application/vnd.openxmlformats-officedocument.presentationml.presentation\",\n        \"group\": \"document\",\n        \"magic\": \"Microsoft PowerPoint 2007+\",\n        \"description\": \"Microsoft PowerPoint 2007+ document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"ooxml\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"pptx\",\n        \"correct_labels\": [\n            \"pptx\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"printfox\": {\n        \"name\": \"printfox\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"prolog\": {\n        \"name\": \"prolog\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"pub\": {\n        \"name\": \"pub\",\n        \"extensions\": [\n            \"pub\"\n        ],\n        \"mime_type\": \"application/x-mspublisher\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"cdf\"\n        ],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"python\": {\n        \"name\": \"python\",\n        \"extensions\": [\n            \"py\"\n        ],\n        \"mime_type\": \"text/x-python\",\n        \"group\": \"code\",\n        \"magic\": \"Python script\",\n        \"description\": \"Python source\",\n        \"vt_type\": \"python\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"python\",\n        \"target_label\": \"python\",\n        \"correct_labels\": [\n            \"python\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"pythonbytecode\": {\n        \"name\": \"pythonbytecode\",\n        \"extensions\": [\n            \"pyc\",\n            \"pyo\"\n        ],\n        \"mime_type\": \"application/x-bytecode.python\",\n        \"group\": \"executable\",\n        \"magic\": \"python byte-compiled\",\n        \"description\": \"Python compiled bytecode\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"pythonbytecode\",\n        \"target_label\": \"pythonbytecode\",\n        \"correct_labels\": [\n            \"pythonbytecode\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"pythonpar\": {\n        \"name\": \"pythonpar\",\n        \"extensions\": [\n            \"par\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"r\": {\n        \"name\": \"r\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"randombytes\": {\n        \"name\": \"randombytes\",\n        \"extensions\": [],\n        \"mime_type\": \"application/octet-stream\",\n        \"group\": \"unknown\",\n        \"magic\": \"data\",\n        \"description\": \"Random bytes\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"synthetic\"\n        ],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": \"unknown\",\n        \"target_label\": \"unknown\",\n        \"correct_labels\": [\n            \"unknown\"\n        ],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": true\n    },\n    \"rar\": {\n        \"name\": \"rar\",\n        \"extensions\": [\n            \"rar\"\n        ],\n        \"mime_type\": \"application/x-rar\",\n        \"group\": \"archive\",\n        \"magic\": \"RAR archive data\",\n        \"description\": \"RAR archive data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"rar\",\n        \"target_label\": \"rar\",\n        \"correct_labels\": [\n            \"rar\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"rdf\": {\n        \"name\": \"rdf\",\n        \"extensions\": [\n            \"rdf\"\n        ],\n        \"mime_type\": \"application/rdf+xml\",\n        \"group\": \"text\",\n        \"magic\": \"XML document\",\n        \"description\": \"Resource Description Framework document (RDF)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"rdf\",\n        \"target_label\": \"rdf\",\n        \"correct_labels\": [\n            \"rdf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"rll\": {\n        \"name\": \"rll\",\n        \"extensions\": [\n            \"rll\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"rpm\": {\n        \"name\": \"rpm\",\n        \"extensions\": [\n            \"rpm\"\n        ],\n        \"mime_type\": \"application/x-rpm\",\n        \"group\": \"archive\",\n        \"magic\": \"RPM\",\n        \"description\": \"RedHat Package Manager archive (RPM)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"rpm\",\n        \"target_label\": \"rpm\",\n        \"correct_labels\": [\n            \"rpm\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"rst\": {\n        \"name\": \"rst\",\n        \"extensions\": [\n            \"rst\"\n        ],\n        \"mime_type\": \"text/x-rst\",\n        \"group\": \"text\",\n        \"magic\": \"ReStructuredText file\",\n        \"description\": \"ReStructuredText document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"rst\",\n        \"target_label\": \"rst\",\n        \"correct_labels\": [\n            \"rst\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"rtf\": {\n        \"name\": \"rtf\",\n        \"extensions\": [\n            \"rtf\"\n        ],\n        \"mime_type\": \"text/rtf\",\n        \"group\": \"text\",\n        \"magic\": \"Rich Text Format data\",\n        \"description\": \"Rich Text Format document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"rtf\",\n        \"target_label\": \"rtf\",\n        \"correct_labels\": [\n            \"rtf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"ruby\": {\n        \"name\": \"ruby\",\n        \"extensions\": [\n            \"rb\"\n        ],\n        \"mime_type\": \"application/x-ruby\",\n        \"group\": \"code\",\n        \"magic\": \"Ruby script\",\n        \"description\": \"Ruby source\",\n        \"vt_type\": \"ruby\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"ruby\",\n        \"target_label\": \"ruby\",\n        \"correct_labels\": [\n            \"ruby\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"rust\": {\n        \"name\": \"rust\",\n        \"extensions\": [\n            \"rs\"\n        ],\n        \"mime_type\": \"application/x-rust\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Rust source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"rust\",\n        \"target_label\": \"rust\",\n        \"correct_labels\": [\n            \"rust\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"s\": {\n        \"name\": \"s\",\n        \"extensions\": [\n            \"s\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"scala\": {\n        \"name\": \"scala\",\n        \"extensions\": [\n            \"scala\"\n        ],\n        \"mime_type\": \"application/x-scala\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Scala source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"scala\",\n        \"target_label\": \"scala\",\n        \"correct_labels\": [\n            \"scala\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"scr\": {\n        \"name\": \"scr\",\n        \"extensions\": [\n            \"scr\"\n        ],\n        \"mime_type\": \"application/x-dosexec\",\n        \"group\": \"executable\",\n        \"magic\": \"PE Windows executable\",\n        \"description\": \"PE Windows executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"scr\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"scriptwsf\": {\n        \"name\": \"scriptwsf\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"sevenzip\": {\n        \"name\": \"sevenzip\",\n        \"extensions\": [\n            \"7z\"\n        ],\n        \"mime_type\": \"application/x-7z-compressed\",\n        \"group\": \"archive\",\n        \"magic\": \"7-zip archive data\",\n        \"description\": \"7-zip archive data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"sevenzip\",\n        \"target_label\": \"sevenzip\",\n        \"correct_labels\": [\n            \"sevenzip\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"sgml\": {\n        \"name\": \"sgml\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"sh3d\": {\n        \"name\": \"sh3d\",\n        \"extensions\": [\n            \"sh3d\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"shell\": {\n        \"name\": \"shell\",\n        \"extensions\": [\n            \"sh\"\n        ],\n        \"mime_type\": \"text/x-shellscript\",\n        \"group\": \"code\",\n        \"magic\": \"shell script\",\n        \"description\": \"Shell script\",\n        \"vt_type\": \"shell\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"shell\",\n        \"target_label\": \"shell\",\n        \"correct_labels\": [\n            \"shell\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"smali\": {\n        \"name\": \"smali\",\n        \"extensions\": [\n            \"smali\"\n        ],\n        \"mime_type\": \"application/x-smali\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Smali source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"smali\",\n        \"target_label\": \"smali\",\n        \"correct_labels\": [\n            \"smali\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"so\": {\n        \"name\": \"so\",\n        \"extensions\": [\n            \"so\"\n        ],\n        \"mime_type\": \"application/x-executable-elf\",\n        \"group\": \"executable\",\n        \"magic\": \"ELF executable\",\n        \"description\": \"ELF executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"elf\"\n        ],\n        \"model_target_label\": \"elf\",\n        \"target_label\": \"elf\",\n        \"correct_labels\": [\n            \"elf\",\n            \"so\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"sql\": {\n        \"name\": \"sql\",\n        \"extensions\": [\n            \"sql\"\n        ],\n        \"mime_type\": \"application/x-sql\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"SQL source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"sql\",\n        \"target_label\": \"sql\",\n        \"correct_labels\": [\n            \"sql\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"squashfs\": {\n        \"name\": \"squashfs\",\n        \"extensions\": [],\n        \"mime_type\": \"application/octet-stream\",\n        \"group\": \"archive\",\n        \"magic\": \"Squashfs filesystem\",\n        \"description\": \"Squash filesystem\",\n        \"vt_type\": \"squashfs\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"squashfs\",\n        \"target_label\": \"squashfs\",\n        \"correct_labels\": [\n            \"squashfs\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"svd\": {\n        \"name\": \"svd\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"svg\": {\n        \"name\": \"svg\",\n        \"extensions\": [\n            \"svg\"\n        ],\n        \"mime_type\": \"image/svg+xml\",\n        \"group\": \"image\",\n        \"magic\": \"SVG Scalable Vector Graphics image\",\n        \"description\": \"SVG Scalable Vector Graphics image data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"svg\",\n        \"target_label\": \"svg\",\n        \"correct_labels\": [\n            \"svg\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"swf\": {\n        \"name\": \"swf\",\n        \"extensions\": [\n            \"swf\"\n        ],\n        \"mime_type\": \"application/x-shockwave-flash\",\n        \"group\": \"executable\",\n        \"magic\": \"Macromedia Flash data\",\n        \"description\": \"Macromedia Flash data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"swf\",\n        \"target_label\": \"swf\",\n        \"correct_labels\": [\n            \"swf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"swift\": {\n        \"name\": \"swift\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"symlinktext\": {\n        \"name\": \"symlinktext\",\n        \"extensions\": [],\n        \"mime_type\": \"text/plain\",\n        \"group\": \"application\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Symbolic link (textual representation)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"synthetic\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"symlinktext\",\n        \"target_label\": \"symlinktext\",\n        \"correct_labels\": [\n            \"symlinktext\",\n            \"txt\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"symlink\": {\n        \"name\": \"symlink\",\n        \"extensions\": [],\n        \"mime_type\": \"inode/symlink\",\n        \"group\": \"inode\",\n        \"magic\": \"symbolic link to \",\n        \"description\": \"Symbolic link to \",\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": \"symlink\",\n        \"correct_labels\": [\n            \"symlink\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": false\n    },\n    \"sys\": {\n        \"name\": \"sys\",\n        \"extensions\": [\n            \"sys\"\n        ],\n        \"mime_type\": \"application/x-windows-driver\",\n        \"group\": \"executable\",\n        \"magic\": \"PE Windows executable\",\n        \"description\": \"PE Windows executable\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"pebin\"\n        ],\n        \"model_target_label\": \"pebin\",\n        \"target_label\": \"pebin\",\n        \"correct_labels\": [\n            \"pebin\",\n            \"sys\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"tar\": {\n        \"name\": \"tar\",\n        \"extensions\": [\n            \"tar\"\n        ],\n        \"mime_type\": \"application/x-tar\",\n        \"group\": \"archive\",\n        \"magic\": \"POSIX tar archive\",\n        \"description\": \"POSIX tar archive\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"tar\",\n        \"target_label\": \"tar\",\n        \"correct_labels\": [\n            \"tar\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"tga\": {\n        \"name\": \"tga\",\n        \"extensions\": [\n            \"tga\"\n        ],\n        \"mime_type\": \"image/x-tga\",\n        \"group\": \"image\",\n        \"magic\": \"Targa image data\",\n        \"description\": \"Targa image data\",\n        \"vt_type\": \"targa\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"tga\",\n        \"target_label\": \"tga\",\n        \"correct_labels\": [\n            \"tga\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"tiff\": {\n        \"name\": \"tiff\",\n        \"extensions\": [\n            \"tiff\",\n            \"tif\"\n        ],\n        \"mime_type\": \"image/tiff\",\n        \"group\": \"image\",\n        \"magic\": \"TIFF image data\",\n        \"description\": \"TIFF image data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"tiff\",\n        \"target_label\": \"tiff\",\n        \"correct_labels\": [\n            \"tiff\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"tmdx\": {\n        \"name\": \"tmdx\",\n        \"extensions\": [\n            \"tmdx\",\n            \"tmvx\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"toml\": {\n        \"name\": \"toml\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"torrent\": {\n        \"name\": \"torrent\",\n        \"extensions\": [\n            \"torrent\"\n        ],\n        \"mime_type\": \"application/x-bittorrent\",\n        \"group\": \"application\",\n        \"magic\": \"BitTorrent file\",\n        \"description\": \"BitTorrent file\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"torrent\",\n        \"target_label\": \"torrent\",\n        \"correct_labels\": [\n            \"torrent\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"troff\": {\n        \"name\": \"troff\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"ttf\": {\n        \"name\": \"ttf\",\n        \"extensions\": [\n            \"ttf\"\n        ],\n        \"mime_type\": \"font/sfnt\",\n        \"group\": \"font\",\n        \"magic\": \"TrueType Font data\",\n        \"description\": \"TrueType Font data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"ttf\",\n        \"target_label\": \"ttf\",\n        \"correct_labels\": [\n            \"ttf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"txt\": {\n        \"name\": \"txt\",\n        \"extensions\": [\n            \"txt\"\n        ],\n        \"mime_type\": \"text/plain\",\n        \"group\": \"text\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"Generic text document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\",\n            \"synthetic\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"txt\",\n        \"target_label\": \"txt\",\n        \"correct_labels\": [\n            \"txt\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"typescript\": {\n        \"name\": \"typescript\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"udf\": {\n        \"name\": \"udf\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-udf-image\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"unixcompress\": {\n        \"name\": \"unixcompress\",\n        \"extensions\": [\n            \"z\"\n        ],\n        \"mime_type\": \"application/x-compress\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"unknown\": {\n        \"name\": \"unknown\",\n        \"extensions\": [],\n        \"mime_type\": \"application/octet-stream\",\n        \"group\": \"unknown\",\n        \"magic\": \"data\",\n        \"description\": \"Unknown type\",\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"unknown\",\n        \"target_label\": \"unknown\",\n        \"correct_labels\": [\n            \"unknown\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": false\n    },\n    \"vba\": {\n        \"name\": \"vba\",\n        \"extensions\": [\n            \"vbs\"\n        ],\n        \"mime_type\": \"text/vbscript\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"MS Visual Basic source (VBA)\",\n        \"vt_type\": \"vba\",\n        \"datasets\": [\n            \"vt-ext\",\n            \"vt-ext-malicious\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\"\n        ],\n        \"model_target_label\": \"vba\",\n        \"target_label\": \"vba\",\n        \"correct_labels\": [\n            \"vba\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"verilog\": {\n        \"name\": \"verilog\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"vhd\": {\n        \"name\": \"vhd\",\n        \"extensions\": [],\n        \"mime_type\": \"application/x-vhd\",\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"wasm\": {\n        \"name\": \"wasm\",\n        \"extensions\": [\n            \"wasm\"\n        ],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"wav\": {\n        \"name\": \"wav\",\n        \"extensions\": [\n            \"wav\"\n        ],\n        \"mime_type\": \"audio/x-wav\",\n        \"group\": \"audio\",\n        \"magic\": \"RIFF data\",\n        \"description\": \"Waveform Audio file (WAV)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"wav\",\n        \"target_label\": \"wav\",\n        \"correct_labels\": [\n            \"wav\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"webm\": {\n        \"name\": \"webm\",\n        \"extensions\": [\n            \"webm\"\n        ],\n        \"mime_type\": \"video/webm\",\n        \"group\": \"video\",\n        \"magic\": \"WebM\",\n        \"description\": \"WebM data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"webm\",\n        \"target_label\": \"webm\",\n        \"correct_labels\": [\n            \"webm\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"webp\": {\n        \"name\": \"webp\",\n        \"extensions\": [\n            \"webp\"\n        ],\n        \"mime_type\": \"image/webp\",\n        \"group\": \"image\",\n        \"magic\": \"RIFF data\",\n        \"description\": \"WebP data\",\n        \"vt_type\": \"webp\",\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"media\"\n        ],\n        \"model_target_label\": \"webp\",\n        \"target_label\": \"webp\",\n        \"correct_labels\": [\n            \"webp\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"winregistry\": {\n        \"name\": \"winregistry\",\n        \"extensions\": [\n            \"reg\"\n        ],\n        \"mime_type\": \"text/x-ms-regedit\",\n        \"group\": \"application\",\n        \"magic\": \"Windows Registry text\",\n        \"description\": \"Windows Registry text\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"winregistry\",\n        \"target_label\": \"winregistry\",\n        \"correct_labels\": [\n            \"winregistry\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"wmf\": {\n        \"name\": \"wmf\",\n        \"extensions\": [\n            \"wmf\"\n        ],\n        \"mime_type\": \"image/wmf\",\n        \"group\": \"image\",\n        \"magic\": \"Windows metafile\",\n        \"description\": \"Windows metafile\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"wmf\",\n        \"target_label\": \"wmf\",\n        \"correct_labels\": [\n            \"wmf\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"woff\": {\n        \"name\": \"woff\",\n        \"extensions\": [],\n        \"mime_type\": null,\n        \"group\": null,\n        \"magic\": null,\n        \"description\": null,\n        \"vt_type\": null,\n        \"datasets\": [],\n        \"parent\": null,\n        \"tags\": [],\n        \"model_target_label\": null,\n        \"target_label\": null,\n        \"correct_labels\": [],\n        \"in_scope_for_output_content_type\": false,\n        \"in_scope_for_training\": false\n    },\n    \"xar\": {\n        \"name\": \"xar\",\n        \"extensions\": [\n            \"pkg\"\n        ],\n        \"mime_type\": \"application/x-xar\",\n        \"group\": \"archive\",\n        \"magic\": \"xar archive compressed\",\n        \"description\": \"XAR archive compressed data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"xar\",\n        \"target_label\": \"xar\",\n        \"correct_labels\": [\n            \"xar\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"xls\": {\n        \"name\": \"xls\",\n        \"extensions\": [\n            \"xls\"\n        ],\n        \"mime_type\": \"application/vnd.ms-excel\",\n        \"group\": \"document\",\n        \"magic\": \"Composite Document File\",\n        \"description\": \"Microsoft Excel CDF document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"cdf\"\n        ],\n        \"model_target_label\": \"cdf\",\n        \"target_label\": \"xls\",\n        \"correct_labels\": [\n            \"xls\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"xlsb\": {\n        \"name\": \"xlsb\",\n        \"extensions\": [\n            \"xlsb\"\n        ],\n        \"mime_type\": \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\n        \"group\": \"document\",\n        \"magic\": \"Microsoft Excel 2007+\",\n        \"description\": \"Microsoft Excel 2007+ document (binary format)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"ooxml\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"xlsb\",\n        \"correct_labels\": [\n            \"xlsb\",\n            \"xlsx\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"xlsx\": {\n        \"name\": \"xlsx\",\n        \"extensions\": [\n            \"xlsx\",\n            \"xlsm\"\n        ],\n        \"mime_type\": \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\",\n        \"group\": \"document\",\n        \"magic\": \"Microsoft Excel 2007+\",\n        \"description\": \"Microsoft Excel 2007+ document\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"ooxml\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"xlsx\",\n        \"correct_labels\": [\n            \"xlsx\",\n            \"xlsb\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"xml\": {\n        \"name\": \"xml\",\n        \"extensions\": [\n            \"xml\"\n        ],\n        \"mime_type\": \"text/xml\",\n        \"group\": \"code\",\n        \"magic\": \"XML document\",\n        \"description\": \"XML document\",\n        \"vt_type\": \"xml\",\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"xml\",\n        \"target_label\": \"xml\",\n        \"correct_labels\": [\n            \"xml\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"xpi\": {\n        \"name\": \"xpi\",\n        \"extensions\": [\n            \"xpi\"\n        ],\n        \"mime_type\": \"application/zip\",\n        \"group\": \"archive\",\n        \"magic\": \"Zip archive data\",\n        \"description\": \"Compressed installation archive (XPI)\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"xpi\",\n        \"correct_labels\": [\n            \"xpi\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"xz\": {\n        \"name\": \"xz\",\n        \"extensions\": [\n            \"xz\"\n        ],\n        \"mime_type\": \"application/x-xz\",\n        \"group\": \"archive\",\n        \"magic\": \"XZ compressed data\",\n        \"description\": \"XZ compressed data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"xz\",\n        \"target_label\": \"xz\",\n        \"correct_labels\": [\n            \"xz\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"yaml\": {\n        \"name\": \"yaml\",\n        \"extensions\": [\n            \"yml\",\n            \"yaml\"\n        ],\n        \"mime_type\": \"application/x-yaml\",\n        \"group\": \"code\",\n        \"magic\": \"ASCII text\",\n        \"description\": \"YAML source\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"github\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"text\",\n            \"dl_target\"\n        ],\n        \"model_target_label\": \"yaml\",\n        \"target_label\": \"yaml\",\n        \"correct_labels\": [\n            \"yaml\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"zip\": {\n        \"name\": \"zip\",\n        \"extensions\": [\n            \"zip\"\n        ],\n        \"mime_type\": \"application/zip\",\n        \"group\": \"archive\",\n        \"magic\": \"Zip archive data\",\n        \"description\": \"Zip archive data\",\n        \"vt_type\": null,\n        \"datasets\": [\n            \"vt-ext\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\",\n            \"zip_archive\",\n            \"archive\"\n        ],\n        \"model_target_label\": \"zip\",\n        \"target_label\": \"zip\",\n        \"correct_labels\": [\n            \"zip\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    },\n    \"zlibstream\": {\n        \"name\": \"zlibstream\",\n        \"extensions\": [],\n        \"mime_type\": \"application/zlib\",\n        \"group\": \"application\",\n        \"magic\": \"zlib compressed data\",\n        \"description\": \"zlib compressed data\",\n        \"vt_type\": \"zlib\",\n        \"datasets\": [\n            \"vt-type\"\n        ],\n        \"parent\": null,\n        \"tags\": [\n            \"binary\"\n        ],\n        \"model_target_label\": \"zlibstream\",\n        \"target_label\": \"zlibstream\",\n        \"correct_labels\": [\n            \"zlibstream\"\n        ],\n        \"in_scope_for_output_content_type\": true,\n        \"in_scope_for_training\": true\n    }\n}"
  },
  {
    "path": "tests_data/basic/latex/sample.tex",
    "content": "\\documentclass{article}\n\\usepackage{todonotes}\n\n\\title{Example of a LaTeX document}\n\n\\begin{document}\n  \\maketitle\n\n  This is just an example used for testing Magika.\n\n\\end{document}\n"
  },
  {
    "path": "tests_data/basic/makefile/simple.Makefile",
    "content": "prog: prog.o\n\tgcc -o prog prog.o\n\nprog.o: prog.c lib.c\n\tgcc -c prog.c lib.c"
  },
  {
    "path": "tests_data/basic/markdown/README.md",
    "content": "# Magika Python Package\n\nMagika is a novel AI powered file type detection tool that rely on the recent advance of deep learning to provide accurate detection. Under the hood, Magika employs a custom, highly optimized Keras model that only weighs about 1MB, and enables precise file identification within milliseconds, even when running on a single CPU.\n\nUse Magika as a command line client or in your Python code!\n\nPlease check out Magika on GitHub for more information and documentation: [https://github.com/google/magika](https://github.com/google/magika).\n\n\n## Installing Magika\n\n```shell\n$ pip install magika\n```\n\nIf you intend to use Magika only as a command line, you may want to use `$ pipx install magika` instead.\n\n\n## Using Magika as a command-line tool\n\n```shell\n$ magika examples/*\ncode.asm: Assembly (code)\ncode.py: Python source (code)\ndoc.docx: Microsoft Word 2007+ document (document)\ndoc.ini: INI configuration file (text)\nelf64.elf: ELF executable (executable)\nflac.flac: FLAC audio bitstream data (audio)\nimage.bmp: BMP image data (image)\njava.class: Java compiled bytecode (executable)\njpg.jpg: JPEG image data (image)\npdf.pdf: PDF document (document)\npe32.exe: PE executable (executable)\npng.png: PNG image data (image)\nREADME.md: Markdown document (text)\ntar.tar: POSIX tar archive (archive)\nwebm.webm: WebM data (video)\n```\n\n```help\n$ magika --help\nUsage: magika [OPTIONS] [FILE]...\n\n  Magika - Determine type of FILEs with deep-learning.\n\nOptions:\n  -r, --recursive                 When passing this option, magika scans every\n                                  file within directories, instead of\n                                  outputting \"directory\"\n  --json                          Output in JSON format.\n  --jsonl                         Output in JSONL format.\n  -i, --mime-type                 Output the MIME type instead of a verbose\n                                  content type description.\n  -l, --label                     Output a simple label instead of a verbose\n                                  content type description. Use --list-output-\n                                  content-types for the list of supported\n                                  output.\n  -c, --compatibility-mode        Compatibility mode: output is as close as\n                                  possible to `file` and colors are disabled.\n  -s, --output-score              Output the prediction score in addition to\n                                  the content type.\n  -m, --prediction-mode [best-guess|medium-confidence|high-confidence]\n  --batch-size INTEGER            How many files to process in one batch.\n  --no-dereference                This option causes symlinks not to be\n                                  followed. By default, symlinks are\n                                  dereferenced.\n  --colors / --no-colors          Enable/disable use of colors.\n  -v, --verbose                   Enable more verbose output.\n  -vv, --debug                    Enable debug logging.\n  --generate-report               Generate report useful when reporting\n                                  feedback.\n  --version                       Print the version and exit.\n  --list-output-content-types     Show a list of supported content types.\n  --model-dir DIRECTORY           Use a custom model.\n  -h, --help                      Show this message and exit.\n\n  Magika version: \"0.5.0\"\n\n  Default model: \"standard_v1\"\n\n  Send any feedback to magika-dev@google.com or via GitHub issues.\n```\n\n\n## Using Magika as a Python module\n\n```python\nfrom magika import Magika\nmagika = Magika()\nresult = magika.identify_bytes(b\"# Example\\nThis is an example of markdown!\")\nprint(result.output.ct_label)  # Output: \"markdown\"\n```\n\n\n## Citation\nIf you use this software for your research, please cite it as:\n```bibtex\n@software{magika,\nauthor = {Fratantonio, Yanick and Bursztein, Elie and Invernizzi, Luca and Zhang, Marina and Metitieri, Giancarlo and Kurt, Thomas and Galilee, Francois and Petit-Bianco, Alexandre and Farah, Loua and Albertini, Ange},\ntitle = {{Magika content-type scanner}},\nurl = {https://github.com/google/magika}\n}\n```\n"
  },
  {
    "path": "tests_data/basic/markdown/magika_test.md",
    "content": "\n\n# Introduction\n\nThis is a test document for Magika, yay\\! We are going to take this file and convert it in a number of other formats."
  },
  {
    "path": "tests_data/basic/markdown/simple.md",
    "content": "# This is the greatest markdown test sample\n\n## Introduction\n\nTest!\n\n## Main content\n\nLorem ipsum?\n\n## Conclusions\n\nNothing much to say!\n\n"
  },
  {
    "path": "tests_data/basic/mht/sample.mht",
    "content": "MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=\"----=_NextPart_01DBCCE0.CD23D5E0\"\r\n\r\nThis document is a Single File Web Page, also known as a Web Archive file.  If you are seeing this message, your browser or editor doesn't support Web Archive files.  Please download a browser that supports Web Archive.\r\n\r\n------=_NextPart_01DBCCE0.CD23D5E0\r\nContent-Location: file:///C:/790962C5/sample-anonymouse.htm\r\nContent-Transfer-Encoding: base64\r\nContent-Type: text/html; charset=\"unicode\"\r\n\r\n//48AGgAdABtAGwAIAB4AG0AbABuAHMAOgBvAD0AIgB1AHIAbgA6AHMAYwBoAGUAbQBhAHMALQBt\r\nAGkAYwByAG8AcwBvAGYAdAAtAGMAbwBtADoAbwBmAGYAaQBjAGUAOgBvAGYAZgBpAGMAZQAiAA0A\r\nCgB4AG0AbABuAHMAOgB3AD0AIgB1AHIAbgA6AHMAYwBoAGUAbQBhAHMALQBtAGkAYwByAG8AcwBv\r\nAGYAdAAtAGMAbwBtADoAbwBmAGYAaQBjAGUAOgB3AG8AcgBkACIADQAKAHgAbQBsAG4AcwA6AG0A\r\nPQAiAGgAdAB0AHAAOgAvAC8AcwBjAGgAZQBtAGEAcwAuAG0AaQBjAHIAbwBzAG8AZgB0AC4AYwBv\r\nAG0ALwBvAGYAZgBpAGMAZQAvADIAMAAwADQALwAxADIALwBvAG0AbQBsACIADQAKAHgAbQBsAG4A\r\ncwA9ACIAaAB0AHQAcAA6AC8ALwB3AHcAdwAuAHcAMwAuAG8AcgBnAC8AVABSAC8AUgBFAEMALQBo\r\nAHQAbQBsADQAMAAiAD4ADQAKAA0ACgA8AGgAZQBhAGQAPgANAAoAPABtAGUAdABhACAAaAB0AHQA\r\ncAAtAGUAcQB1AGkAdgA9AEMAbwBuAHQAZQBuAHQALQBUAHkAcABlACAAYwBvAG4AdABlAG4AdAA9\r\nACIAdABlAHgAdAAvAGgAdABtAGwAOwAgAGMAaABhAHIAcwBlAHQAPQB1AG4AaQBjAG8AZABlACIA\r\nPgANAAoAPABtAGUAdABhACAAbgBhAG0AZQA9AFAAcgBvAGcASQBkACAAYwBvAG4AdABlAG4AdAA9\r\nAFcAbwByAGQALgBEAG8AYwB1AG0AZQBuAHQAPgANAAoAPABtAGUAdABhACAAbgBhAG0AZQA9AEcA\r\nZQBuAGUAcgBhAHQAbwByACAAYwBvAG4AdABlAG4AdAA9ACIATQBpAGMAcgBvAHMAbwBmAHQAIABX\r\nAG8AcgBkACAAMQA1ACIAPgANAAoAPABtAGUAdABhACAAbgBhAG0AZQA9AE8AcgBpAGcAaQBuAGEA\r\ndABvAHIAIABjAG8AbgB0AGUAbgB0AD0AIgBNAGkAYwByAG8AcwBvAGYAdAAgAFcAbwByAGQAIAAx\r\nADUAIgA+AA0ACgA8AGwAaQBuAGsAIAByAGUAbAA9AEYAaQBsAGUALQBMAGkAcwB0ACAAaAByAGUA\r\nZgA9ACIAcwBhAG0AcABsAGUALQBhAG4AbwBuAHkAbQBvAHUAcwBlAF8AZgBpAGwAZQBzAC8AZgBp\r\nAGwAZQBsAGkAcwB0AC4AeABtAGwAIgA+AA0ACgA8AGwAaQBuAGsAIAByAGUAbAA9AEUAZABpAHQA\r\nLQBUAGkAbQBlAC0ARABhAHQAYQAgAGgAcgBlAGYAPQAiAHMAYQBtAHAAbABlAC0AYQBuAG8AbgB5\r\nAG0AbwB1AHMAZQBfAGYAaQBsAGUAcwAvAGUAZABpAHQAZABhAHQAYQAuAG0AcwBvACIAPgANAAoA\r\nPAAhAC0ALQBbAGkAZgAgAGcAdABlACAAbQBzAG8AIAA5AF0APgA8AHgAbQBsAD4ADQAKACAAPABv\r\nADoATwBmAGYAaQBjAGUARABvAGMAdQBtAGUAbgB0AFMAZQB0AHQAaQBuAGcAcwA+AA0ACgAgACAA\r\nPABvADoAQQBsAGwAbwB3AFAATgBHAC8APgANAAoAIAA8AC8AbwA6AE8AZgBmAGkAYwBlAEQAbwBj\r\nAHUAbQBlAG4AdABTAGUAdAB0AGkAbgBnAHMAPgANAAoAPAAvAHgAbQBsAD4APAAhAFsAZQBuAGQA\r\naQBmAF0ALQAtAD4ADQAKADwAbABpAG4AawAgAHIAZQBsAD0AdABoAGUAbQBlAEQAYQB0AGEAIABo\r\nAHIAZQBmAD0AIgBzAGEAbQBwAGwAZQAtAGEAbgBvAG4AeQBtAG8AdQBzAGUAXwBmAGkAbABlAHMA\r\nLwB0AGgAZQBtAGUAZABhAHQAYQAuAHQAaABtAHgAIgA+AA0ACgA8AGwAaQBuAGsAIAByAGUAbAA9\r\nAGMAbwBsAG8AcgBTAGMAaABlAG0AZQBNAGEAcABwAGkAbgBnAA0ACgBoAHIAZQBmAD0AIgBzAGEA\r\nbQBwAGwAZQAtAGEAbgBvAG4AeQBtAG8AdQBzAGUAXwBmAGkAbABlAHMALwBjAG8AbABvAHIAcwBj\r\nAGgAZQBtAGUAbQBhAHAAcABpAG4AZwAuAHgAbQBsACIAPgANAAoAPAAhAC0ALQBbAGkAZgAgAGcA\r\ndABlACAAbQBzAG8AIAA5AF0APgA8AHgAbQBsAD4ADQAKACAAPAB3ADoAVwBvAHIAZABEAG8AYwB1\r\nAG0AZQBuAHQAPgANAAoAIAAgADwAdwA6AFoAbwBvAG0APgAwADwALwB3ADoAWgBvAG8AbQA+AA0A\r\nCgAgACAAPAB3ADoARABvAGMAdQBtAGUAbgB0AEsAaQBuAGQAPgBEAG8AYwB1AG0AZQBuAHQARQBt\r\nAGEAaQBsADwALwB3ADoARABvAGMAdQBtAGUAbgB0AEsAaQBuAGQAPgANAAoAIAAgADwAdwA6AFQA\r\ncgBhAGMAawBNAG8AdgBlAHMALwA+AA0ACgAgACAAPAB3ADoAVAByAGEAYwBrAEYAbwByAG0AYQB0\r\nAHQAaQBuAGcALwA+AA0ACgAgACAAPAB3ADoAVgBhAGwAaQBkAGEAdABlAEEAZwBhAGkAbgBzAHQA\r\nUwBjAGgAZQBtAGEAcwAvAD4ADQAKACAAIAA8AHcAOgBTAGEAdgBlAEkAZgBYAE0ATABJAG4AdgBh\r\nAGwAaQBkAD4AZgBhAGwAcwBlADwALwB3ADoAUwBhAHYAZQBJAGYAWABNAEwASQBuAHYAYQBsAGkA\r\nZAA+AA0ACgAgACAAPAB3ADoASQBnAG4AbwByAGUATQBpAHgAZQBkAEMAbwBuAHQAZQBuAHQAPgBm\r\nAGEAbABzAGUAPAAvAHcAOgBJAGcAbgBvAHIAZQBNAGkAeABlAGQAQwBvAG4AdABlAG4AdAA+AA0A\r\nCgAgACAAPAB3ADoAQQBsAHcAYQB5AHMAUwBoAG8AdwBQAGwAYQBjAGUAaABvAGwAZABlAHIAVABl\r\nAHgAdAA+AGYAYQBsAHMAZQA8AC8AdwA6AEEAbAB3AGEAeQBzAFMAaABvAHcAUABsAGEAYwBlAGgA\r\nbwBsAGQAZQByAFQAZQB4AHQAPgANAAoAIAAgADwAdwA6AEQAbwBOAG8AdABQAHIAbwBtAG8AdABl\r\nAFEARgAvAD4ADQAKACAAIAA8AHcAOgBMAGkAZABUAGgAZQBtAGUATwB0AGgAZQByAD4ARQBOAC0A\r\nVQBTADwALwB3ADoATABpAGQAVABoAGUAbQBlAE8AdABoAGUAcgA+AA0ACgAgACAAPAB3ADoATABp\r\nAGQAVABoAGUAbQBlAEEAcwBpAGEAbgA+AFgALQBOAE8ATgBFADwALwB3ADoATABpAGQAVABoAGUA\r\nbQBlAEEAcwBpAGEAbgA+AA0ACgAgACAAPAB3ADoATABpAGQAVABoAGUAbQBlAEMAbwBtAHAAbABl\r\nAHgAUwBjAHIAaQBwAHQAPgBYAC0ATgBPAE4ARQA8AC8AdwA6AEwAaQBkAFQAaABlAG0AZQBDAG8A\r\nbQBwAGwAZQB4AFMAYwByAGkAcAB0AD4ADQAKACAAIAA8AHcAOgBDAG8AbQBwAGEAdABpAGIAaQBs\r\nAGkAdAB5AD4ADQAKACAAIAAgADwAdwA6AEQAbwBOAG8AdABFAHgAcABhAG4AZABTAGgAaQBmAHQA\r\nUgBlAHQAdQByAG4ALwA+AA0ACgAgACAAIAA8AHcAOgBCAHIAZQBhAGsAVwByAGEAcABwAGUAZABU\r\nAGEAYgBsAGUAcwAvAD4ADQAKACAAIAAgADwAdwA6AFMAbgBhAHAAVABvAEcAcgBpAGQASQBuAEMA\r\nZQBsAGwALwA+AA0ACgAgACAAIAA8AHcAOgBXAHIAYQBwAFQAZQB4AHQAVwBpAHQAaABQAHUAbgBj\r\nAHQALwA+AA0ACgAgACAAIAA8AHcAOgBVAHMAZQBBAHMAaQBhAG4AQgByAGUAYQBrAFIAdQBsAGUA\r\ncwAvAD4ADQAKACAAIAAgADwAdwA6AEQAbwBuAHQARwByAG8AdwBBAHUAdABvAGYAaQB0AC8APgAN\r\nAAoAIAAgACAAPAB3ADoAUwBwAGwAaQB0AFAAZwBCAHIAZQBhAGsAQQBuAGQAUABhAHIAYQBNAGEA\r\ncgBrAC8APgANAAoAIAAgACAAPAB3ADoARQBuAGEAYgBsAGUATwBwAGUAbgBUAHkAcABlAEsAZQBy\r\nAG4AaQBuAGcALwA+AA0ACgAgACAAIAA8AHcAOgBEAG8AbgB0AEYAbABpAHAATQBpAHIAcgBvAHIA\r\nSQBuAGQAZQBuAHQAcwAvAD4ADQAKACAAIAAgADwAdwA6AE8AdgBlAHIAcgBpAGQAZQBUAGEAYgBs\r\nAGUAUwB0AHkAbABlAEgAcABzAC8APgANAAoAIAAgADwALwB3ADoAQwBvAG0AcABhAHQAaQBiAGkA\r\nbABpAHQAeQA+AA0ACgAgACAAPABtADoAbQBhAHQAaABQAHIAPgANAAoAIAAgACAAPABtADoAbQBh\r\nAHQAaABGAG8AbgB0ACAAbQA6AHYAYQBsAD0AIgBDAGEAbQBiAHIAaQBhACAATQBhAHQAaAAiAC8A\r\nPgANAAoAIAAgACAAPABtADoAYgByAGsAQgBpAG4AIABtADoAdgBhAGwAPQAiAGIAZQBmAG8AcgBl\r\nACIALwA+AA0ACgAgACAAIAA8AG0AOgBiAHIAawBCAGkAbgBTAHUAYgAgAG0AOgB2AGEAbAA9ACIA\r\nJgAjADQANQA7AC0AIgAvAD4ADQAKACAAIAAgADwAbQA6AHMAbQBhAGwAbABGAHIAYQBjACAAbQA6\r\nAHYAYQBsAD0AIgBvAGYAZgAiAC8APgANAAoAIAAgACAAPABtADoAZABpAHMAcABEAGUAZgAvAD4A\r\nDQAKACAAIAAgADwAbQA6AGwATQBhAHIAZwBpAG4AIABtADoAdgBhAGwAPQAiADAAIgAvAD4ADQAK\r\nACAAIAAgADwAbQA6AHIATQBhAHIAZwBpAG4AIABtADoAdgBhAGwAPQAiADAAIgAvAD4ADQAKACAA\r\nIAAgADwAbQA6AGQAZQBmAEoAYwAgAG0AOgB2AGEAbAA9ACIAYwBlAG4AdABlAHIARwByAG8AdQBw\r\nACIALwA+AA0ACgAgACAAIAA8AG0AOgB3AHIAYQBwAEkAbgBkAGUAbgB0ACAAbQA6AHYAYQBsAD0A\r\nIgAxADQANAAwACIALwA+AA0ACgAgACAAIAA8AG0AOgBpAG4AdABMAGkAbQAgAG0AOgB2AGEAbAA9\r\nACIAcwB1AGIAUwB1AHAAIgAvAD4ADQAKACAAIAAgADwAbQA6AG4AYQByAHkATABpAG0AIABtADoA\r\ndgBhAGwAPQAiAHUAbgBkAE8AdgByACIALwA+AA0ACgAgACAAPAAvAG0AOgBtAGEAdABoAFAAcgA+\r\nADwALwB3ADoAVwBvAHIAZABEAG8AYwB1AG0AZQBuAHQAPgANAAoAPAAvAHgAbQBsAD4APAAhAFsA\r\nZQBuAGQAaQBmAF0ALQAtAD4APAAhAC0ALQBbAGkAZgAgAGcAdABlACAAbQBzAG8AIAA5AF0APgA8\r\nAHgAbQBsAD4ADQAKACAAPAB3ADoATABhAHQAZQBuAHQAUwB0AHkAbABlAHMAIABEAGUAZgBMAG8A\r\nYwBrAGUAZABTAHQAYQB0AGUAPQAiAGYAYQBsAHMAZQAiACAARABlAGYAVQBuAGgAaQBkAGUAVwBo\r\nAGUAbgBVAHMAZQBkAD0AIgBmAGEAbABzAGUAIgANAAoAIAAgAEQAZQBmAFMAZQBtAGkASABpAGQA\r\nZABlAG4APQAiAGYAYQBsAHMAZQAiACAARABlAGYAUQBGAG8AcgBtAGEAdAA9ACIAZgBhAGwAcwBl\r\nACIAIABEAGUAZgBQAHIAaQBvAHIAaQB0AHkAPQAiADkAOQAiAA0ACgAgACAATABhAHQAZQBuAHQA\r\nUwB0AHkAbABlAEMAbwB1AG4AdAA9ACIAMwA3ADYAIgA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIA\r\naQB0AHkAPQAiADAAIgAgAFEARgBvAHIAbQBhAHQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAi\r\nAE4AbwByAG0AYQBsACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADkAIgAg\r\nAFEARgBvAHIAbQBhAHQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAGgAZQBhAGQAaQBuAGcA\r\nIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADkAIgAgAFMAZQBtAGkA\r\nSABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgANAAoAIAAgACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBV\r\nAHMAZQBkAD0AIgB0AHIAdQBlACIAIABRAEYAbwByAG0AYQB0AD0AIgB0AHIAdQBlACIAIABOAGEA\r\nbQBlAD0AIgBoAGUAYQBkAGkAbgBnACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkA\r\ndAB5AD0AIgA5ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiACAAUQBGAG8AcgBtAGEA\r\ndAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIAaABlAGEAZABpAG4AZwAgADMAIgAvAD4ADQAK\r\nACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEA\r\nbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAOQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9\r\nACIAdAByAHUAZQAiAA0ACgAgACAAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQA\r\ncgB1AGUAIgAgAFEARgBvAHIAbQBhAHQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAGgAZQBh\r\nAGQAaQBuAGcAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADkAIgAg\r\nAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgANAAoAIAAgACAAVQBuAGgAaQBkAGUA\r\nVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIAIABRAEYAbwByAG0AYQB0AD0AIgB0AHIAdQBl\r\nACIAIABOAGEAbQBlAD0AIgBoAGUAYQBkAGkAbgBnACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwA\r\ncwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUABy\r\nAGkAbwByAGkAdAB5AD0AIgA5ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIA\r\nDQAKACAAIAAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiACAAUQBG\r\nAG8AcgBtAGEAdAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIAaABlAGEAZABpAG4AZwAgADYA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAOQAiACAAUwBlAG0AaQBIAGkA\r\nZABkAGUAbgA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBl\r\nAGQAPQAiAHQAcgB1AGUAIgAgAFEARgBvAHIAbQBhAHQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUA\r\nPQAiAGgAZQBhAGQAaQBuAGcAIAA3ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBw\r\nAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkA\r\nPQAiADkAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgANAAoAIAAgACAAVQBu\r\nAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIAIABRAEYAbwByAG0AYQB0AD0A\r\nIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgBoAGUAYQBkAGkAbgBnACAAOAAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA5ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0\r\nAHIAdQBlACIADQAKACAAIAAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUA\r\nZQAiACAAUQBGAG8AcgBtAGEAdAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIAaABlAGEAZABp\r\nAG4AZwAgADkAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1\r\nAGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAA\r\nIABOAGEAbQBlAD0AIgBpAG4AZABlAHgAIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgA\r\naQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAi\r\nAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAaQBuAGQAZQB4ACAAMgAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUA\r\nVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAGkAbgBk\r\nAGUAeAAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1\r\nAGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAA\r\nIABOAGEAbQBlAD0AIgBpAG4AZABlAHgAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgA\r\naQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAi\r\nAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAaQBuAGQAZQB4ACAANQAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUA\r\nVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAGkAbgBk\r\nAGUAeAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1\r\nAGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAA\r\nIABOAGEAbQBlAD0AIgBpAG4AZABlAHgAIAA3ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgA\r\naQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAi\r\nAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAaQBuAGQAZQB4ACAAOAAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUA\r\nVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAGkAbgBk\r\nAGUAeAAgADkAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMwA5ACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAFUAbgBoAGkAZABlAFcA\r\naABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIAdABvAGMAIAAxACIALwA+\r\nAA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIA\r\nZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADMAOQAiACAAUwBlAG0AaQBIAGkAZABk\r\nAGUAbgA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQA\r\nPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAHQAbwBjACAAMgAiAC8APgANAAoAIAAgADwAdwA6\r\nAEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAA\r\nUAByAGkAbwByAGkAdAB5AD0AIgAzADkAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1\r\nAGUAIgANAAoAIAAgACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIA\r\nIABOAGEAbQBlAD0AIgB0AG8AYwAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIAMwA5ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIA\r\ndABvAGMAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABM\r\nAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADMAOQAiACAA\r\nUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABVAG4AaABpAGQAZQBX\r\nAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAHQAbwBjACAANQAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAzADkAIgAgAFMAZQBtAGkASABpAGQA\r\nZABlAG4APQAiAHQAcgB1AGUAIgANAAoAIAAgACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBk\r\nAD0AIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgB0AG8AYwAgADYAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFAAcgBpAG8AcgBpAHQAeQA9ACIAMwA5ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIA\r\ndQBlACIADQAKACAAIAAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAi\r\nACAATgBhAG0AZQA9ACIAdABvAGMAIAA3ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMA\r\nZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0\r\nAHkAPQAiADMAOQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiAA0ACgAgACAA\r\nIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAi\r\nAHQAbwBjACAAOAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAzADkAIgAg\r\nAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgANAAoAIAAgACAAVQBuAGgAaQBkAGUA\r\nVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgB0AG8AYwAgADkAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBo\r\nAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0A\r\nIgBOAG8AcgBtAGEAbAAgAEkAbgBkAGUAbgB0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgA\r\naQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAi\r\nAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAZgBvAG8AdABuAG8AdABlACAAdABlAHgA\r\ndAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAA\r\nVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBt\r\nAGUAPQAiAGEAbgBuAG8AdABhAHQAaQBvAG4AIAB0AGUAeAB0ACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAaABlAGEAZABlAHIAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBo\r\nAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0A\r\nIgBmAG8AbwB0AGUAcgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBu\r\nACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIA\r\ndAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAK\r\nACAAIAAgAE4AYQBtAGUAPQAiAGkAbgBkAGUAeAAgAGgAZQBhAGQAaQBuAGcAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMwA1ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0A\r\nIgB0AHIAdQBlACIADQAKACAAIAAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdABy\r\nAHUAZQAiACAAUQBGAG8AcgBtAGEAdAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIAYwBhAHAA\r\ndABpAG8AbgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUA\r\nZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAE4AYQBtAGUAPQAiAHQAYQBiAGwAZQAgAG8AZgAgAGYAaQBnAHUAcgBlAHMAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcA\r\naABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBlAG4AdgBl\r\nAGwAbwBwAGUAIABhAGQAZAByAGUAcwBzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMA\r\nZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBk\r\nAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQA\r\ncgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAZQBuAHYAZQBsAG8AcABlACAAcgBlAHQAdQBy\r\nAG4AIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsA\r\nZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAg\r\nAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEA\r\nbQBlAD0AIgBmAG8AbwB0AG4AbwB0AGUAIAByAGUAZgBlAHIAZQBuAGMAZQAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBo\r\nAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAGEAbgBuAG8A\r\ndABhAHQAaQBvAG4AIAByAGUAZgBlAHIAZQBuAGMAZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0A\r\naQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBk\r\nAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAGwAaQBuAGUAIABuAHUAbQBiAGUA\r\ncgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAA\r\nVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBt\r\nAGUAPQAiAHAAYQBnAGUAIABuAHUAbQBiAGUAcgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBI\r\nAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0A\r\nIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAGUAbgBkAG4AbwB0AGUAIAByAGUAZgBl\r\nAHIAZQBuAGMAZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdABy\r\nAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAA\r\nIAAgAE4AYQBtAGUAPQAiAGUAbgBkAG4AbwB0AGUAIAB0AGUAeAB0ACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBu\r\nAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAdABhAGIAbABlACAA\r\nbwBmACAAYQB1AHQAaABvAHIAaQB0AGkAZQBzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgA\r\naQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAi\r\nAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAbQBhAGMAcgBvACIALwA+AA0ACgAgACAA\r\nPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBl\r\nACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgA\r\nZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAdABvAGEAIABo\r\nAGUAYQBkAGkAbgBnACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0\r\nAHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoA\r\nIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkA\r\nZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0\r\nAHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABCAHUAbABsAGUAdAAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgA\r\naQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAi\r\nAEwAaQBzAHQAIABOAHUAbQBiAGUAcgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABk\r\nAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIA\r\ndQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIAAyACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBu\r\nAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgADMA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUA\r\nbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBl\r\nAD0AIgBMAGkAcwB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9\r\nACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIA\r\nDQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIAA1ACIALwA+AA0ACgAgACAAPAB3ADoATABz\r\nAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUA\r\nbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBl\r\nAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgAEIAdQBsAGwA\r\nZQB0ACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUA\r\nZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABCAHUAbABsAGUAdAAgADMAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4A\r\nVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAQgB1\r\nAGwAbABlAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0\r\nAHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoA\r\nIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgAEIAdQBsAGwAZQB0ACAANQAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBo\r\nAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQA\r\nIABOAHUAbQBiAGUAcgAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4A\r\nPQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAi\r\nAA0ACgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAATgB1AG0AYgBlAHIAIAAzACIALwA+AA0A\r\nCgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBh\r\nAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQA\r\nZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABp\r\nAHMAdAAgAE4AdQBtAGIAZQByACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABk\r\nAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIA\r\ndQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABOAHUAbQBiAGUAcgAgADUAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMQAwACIAIABRAEYAbwByAG0AYQB0\r\nAD0AIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgBUAGkAdABsAGUAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4A\r\nVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBDAGwAbwBzAGkAbgBn\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABV\r\nAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0A\r\nZQA9ACIAUwBpAGcAbgBhAHQAdQByAGUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIAMQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABV\r\nAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAEQA\r\nZQBmAGEAdQBsAHQAIABQAGEAcgBhAGcAcgBhAHAAaAAgAEYAbwBuAHQAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABl\r\nAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBCAG8AZAB5ACAA\r\nVABlAHgAdAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUA\r\nZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAE4AYQBtAGUAPQAiAEIAbwBkAHkAIABUAGUAeAB0ACAASQBuAGQAZQBuAHQAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcA\r\naABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0\r\nACAAQwBvAG4AdABpAG4AdQBlACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQA\r\naQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBu\r\nAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUA\r\nIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgAEMAbwBuAHQAaQBuAHUAZQAgADIAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBo\r\nAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0A\r\nIgBMAGkAcwB0ACAAQwBvAG4AdABpAG4AdQBlACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0A\r\naQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBk\r\nAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABDAG8AbgB0AGkA\r\nbgB1AGUAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABM\r\nAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIA\r\ndQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAg\r\nACAATgBhAG0AZQA9ACIATABpAHMAdAAgAEMAbwBuAHQAaQBuAHUAZQAgADUAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcA\r\naABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBNAGUAcwBz\r\nAGEAZwBlACAASABlAGEAZABlAHIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAA\r\ndABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9\r\nACIAMQAxACIAIABRAEYAbwByAG0AYQB0AD0AIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgBTAHUA\r\nYgB0AGkAdABsAGUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQA\r\ncgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAg\r\nACAAIABOAGEAbQBlAD0AIgBTAGEAbAB1AHQAYQB0AGkAbwBuACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARABhAHQAZQAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBk\r\nAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEIA\r\nbwBkAHkAIABUAGUAeAB0ACAARgBpAHIAcwB0ACAASQBuAGQAZQBuAHQAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABl\r\nAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBCAG8AZAB5ACAA\r\nVABlAHgAdAAgAEYAaQByAHMAdAAgAEkAbgBkAGUAbgB0ACAAMgAiAC8APgANAAoAIAAgADwAdwA6\r\nAEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAA\r\nUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBV\r\nAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAE4AbwB0AGUAIABIAGUA\r\nYQBkAGkAbgBnACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABM\r\nAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIA\r\ndQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAg\r\nACAATgBhAG0AZQA9ACIAQgBvAGQAeQAgAFQAZQB4AHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAQgBvAGQAeQAgAFQAZQB4\r\nAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8A\r\nYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBl\r\nACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAA\r\nTgBhAG0AZQA9ACIAQgBvAGQAeQAgAFQAZQB4AHQAIABJAG4AZABlAG4AdAAgADIAIgAvAD4ADQAK\r\nACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEA\r\nbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABl\r\nAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBCAG8A\r\nZAB5ACAAVABlAHgAdAAgAEkAbgBkAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0A\r\naQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBk\r\nAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEIAbABvAGMAawAgAFQAZQB4AHQA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUA\r\nbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBl\r\nAD0AIgBIAHkAcABlAHIAbABpAG4AawAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABk\r\nAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIA\r\ndQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEYAbwBsAGwAbwB3AGUAZABIAHkAcABlAHIAbABp\r\nAG4AawAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMA\r\nawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAyADIAIgAgAFEARgBv\r\nAHIAbQBhAHQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAFMAdAByAG8AbgBnACIALwA+AA0A\r\nCgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBh\r\nAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADIAMAAiACAAUQBGAG8AcgBtAGEAdAA9ACIA\r\ndAByAHUAZQAiACAATgBhAG0AZQA9ACIARQBtAHAAaABhAHMAaQBzACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBu\r\nAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARABvAGMAdQBtAGUA\r\nbgB0ACAATQBhAHAAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQA\r\ncgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAg\r\nACAAIABOAGEAbQBlAD0AIgBQAGwAYQBpAG4AIABUAGUAeAB0ACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARQAtAG0AYQBpAGwAIABT\r\nAGkAZwBuAGEAdAB1AHIAZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9\r\nACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIA\r\nDQAKACAAIAAgAE4AYQBtAGUAPQAiAEgAVABNAEwAIABUAG8AcAAgAG8AZgAgAEYAbwByAG0AIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBo\r\nAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0A\r\nIgBIAFQATQBMACAAQgBvAHQAdABvAG0AIABvAGYAIABGAG8AcgBtACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBu\r\nAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATgBvAHIAbQBhAGwA\r\nIAAoAFcAZQBiACkAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQA\r\ncgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAg\r\nACAAIABOAGEAbQBlAD0AIgBIAFQATQBMACAAQQBjAHIAbwBuAHkAbQAiAC8APgANAAoAIAAgADwA\r\ndwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAi\r\nACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUA\r\nbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEgAVABNAEwAIABB\r\nAGQAZAByAGUAcwBzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0\r\nAHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoA\r\nIAAgACAATgBhAG0AZQA9ACIASABUAE0ATAAgAEMAaQB0AGUAIgAvAD4ADQAKACAAIAA8AHcAOgBM\r\nAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMA\r\nZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBz\r\nAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBIAFQATQBMACAAQwBvAGQA\r\nZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAA\r\nVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBt\r\nAGUAPQAiAEgAVABNAEwAIABEAGUAZgBpAG4AaQB0AGkAbwBuACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIASABUAE0ATAAgAEsAZQB5\r\nAGIAbwBhAHIAZAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdABy\r\nAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAA\r\nIAAgAE4AYQBtAGUAPQAiAEgAVABNAEwAIABQAHIAZQBmAG8AcgBtAGEAdAB0AGUAZAAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBk\r\nAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEgA\r\nVABNAEwAIABTAGEAbQBwAGwAZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0\r\nAGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUA\r\nbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBl\r\nACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEgAVABNAEwAIABUAHkAcABlAHcAcgBpAHQAZQByACIA\r\nLwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9\r\nACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4A\r\naABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9\r\nACIASABUAE0ATAAgAFYAYQByAGkAYQBiAGwAZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBI\r\nAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0A\r\nIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAE4AbwByAG0AYQBsACAAVABhAGIAbABl\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABV\r\nAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0A\r\nZQA9ACIAYQBuAG4AbwB0AGEAdABpAG8AbgAgAHMAdQBiAGoAZQBjAHQAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABl\r\nAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBOAG8AIABMAGkA\r\ncwB0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIA\r\nIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBh\r\nAG0AZQA9ACIATwB1AHQAbABpAG4AZQAgAEwAaQBzAHQAIAAxACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATwB1AHQAbABpAG4AZQAg\r\nAEwAaQBzAHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0\r\nAHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoA\r\nIAAgACAATgBhAG0AZQA9ACIATwB1AHQAbABpAG4AZQAgAEwAaQBzAHQAIAAzACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBX\r\nAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIA\r\nbABlACAAUwBpAG0AcABsAGUAIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBw\r\nAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQA\r\nZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1\r\nAGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAAUwBpAG0AcABsAGUAIAAyACIA\r\nLwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9\r\nACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4A\r\naABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9\r\nACIAVABhAGIAbABlACAAUwBpAG0AcABsAGUAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBp\r\nAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQA\r\nPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAAQwBsAGEAcwBz\r\nAGkAYwAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1\r\nAGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAA\r\nIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABDAGwAYQBzAHMAaQBjACAAMgAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBo\r\nAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwA\r\nZQAgAEMAbABhAHMAcwBpAGMAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBw\r\nAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQA\r\nZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1\r\nAGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAAQwBsAGEAcwBzAGkAYwAgADQA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUA\r\nbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBl\r\nAD0AIgBUAGEAYgBsAGUAIABDAG8AbABvAHIAZgB1AGwAIAAxACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAAQwBv\r\nAGwAbwByAGYAdQBsACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9\r\nACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIA\r\nDQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEMAbwBsAG8AcgBmAHUAbAAgADMAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBo\r\nAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0A\r\nIgBUAGEAYgBsAGUAIABDAG8AbAB1AG0AbgBzACAAMQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0A\r\naQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBk\r\nAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEMAbwBsAHUA\r\nbQBuAHMAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABM\r\nAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIA\r\ndQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAg\r\nACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAAQwBvAGwAdQBtAG4AcwAgADMAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcA\r\naABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBUAGEAYgBs\r\nAGUAIABDAG8AbAB1AG0AbgBzACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABk\r\nAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIA\r\ndQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEMAbwBsAHUAbQBuAHMAIAA1\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABV\r\nAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0A\r\nZQA9ACIAVABhAGIAbABlACAARwByAGkAZAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABF\r\nAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkA\r\nSABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9\r\nACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABHAHIAaQBkACAA\r\nMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAA\r\nVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBt\r\nAGUAPQAiAFQAYQBiAGwAZQAgAEcAcgBpAGQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBp\r\nAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQA\r\nPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAARwByAGkAZAAg\r\nADQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsA\r\nZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAg\r\nAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEA\r\nbQBlAD0AIgBUAGEAYgBsAGUAIABHAHIAaQBkACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0A\r\naQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBk\r\nAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEcAcgBpAGQA\r\nIAA2ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIA\r\nIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBh\r\nAG0AZQA9ACIAVABhAGIAbABlACAARwByAGkAZAAgADcAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMA\r\nZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBt\r\nAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUA\r\nZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABHAHIAaQBk\r\nACAAOAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMA\r\nawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAi\r\nACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4A\r\nYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEwAaQBzAHQAIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABz\r\nAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUA\r\nbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBl\r\nAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAATABpAHMA\r\ndAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBj\r\nAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUA\r\nIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABO\r\nAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABMAGkAcwB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwA\r\ncwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBl\r\nAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMA\r\nZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEwAaQBz\r\nAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8A\r\nYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBl\r\nACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAA\r\nTgBhAG0AZQA9ACIAVABhAGIAbABlACAATABpAHMAdAAgADUAIgAvAD4ADQAKACAAIAA8AHcAOgBM\r\nAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMA\r\nZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBz\r\nAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABMAGkA\r\ncwB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUA\r\nZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAEwAaQBzAHQAIAA3ACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABT\r\nAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUA\r\ncwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAATABp\r\nAHMAdAAgADgAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4APQAiAHQAcgB1\r\nAGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAiAA0ACgAgACAA\r\nIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIAAzAEQAIABlAGYAZgBlAGMAdABzACAAMQAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBk\r\nAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQA\r\nYQBiAGwAZQAgADMARAAgAGUAZgBmAGUAYwB0AHMAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABz\r\nAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUA\r\nbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBXAGgAZQBuAFUAcwBl\r\nAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAAMwBEACAA\r\nZQBmAGYAZQBjAHQAcwAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBtAGkASABpAGQAZABlAG4A\r\nPQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdAByAHUAZQAi\r\nAA0ACgAgACAAIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABDAG8AbgB0AGUAbQBwAG8AcgBhAHIA\r\neQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAA\r\nVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBt\r\nAGUAPQAiAFQAYQBiAGwAZQAgAEUAbABlAGcAYQBuAHQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMA\r\nZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFMAZQBt\r\nAGkASABpAGQAZABlAG4APQAiAHQAcgB1AGUAIgAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUA\r\nZAA9ACIAdAByAHUAZQAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBUAGEAYgBsAGUAIABQAHIAbwBm\r\nAGUAcwBzAGkAbwBuAGEAbAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9\r\nACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIA\r\nDQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAFMAdQBiAHQAbABlACAAMQAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBk\r\nAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQA\r\nYQBiAGwAZQAgAFMAdQBiAHQAbABlACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkA\r\nZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0\r\nAHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAFcAZQBiACAAMQAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgA\r\naQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAi\r\nAFQAYQBiAGwAZQAgAFcAZQBiACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABk\r\nAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIA\r\ndQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwAZQAgAFcAZQBiACAAMwAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBk\r\nAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEIA\r\nYQBsAGwAbwBvAG4AIABUAGUAeAB0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBw\r\nAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkA\r\nPQAiADMAOQAiACAATgBhAG0AZQA9ACIAVABhAGIAbABlACAARwByAGkAZAAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBo\r\nAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAFQAYQBiAGwA\r\nZQAgAFQAaABlAG0AZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBu\r\nACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIA\r\ndAByAHUAZQAiACAATgBhAG0AZQA9ACIAUABsAGEAYwBlAGgAbwBsAGQAZQByACAAVABlAHgAdAAi\r\nAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQA\r\nPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAxACIAIABRAEYAbwByAG0AYQB0\r\nAD0AIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgBOAG8AIABTAHAAYQBjAGkAbgBnACIALwA+AA0A\r\nCgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBh\r\nAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMAAiACAATgBhAG0AZQA9ACIATABpAGcA\r\naAB0ACAAUwBoAGEAZABpAG4AZwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0\r\nAGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0A\r\nIgA2ADEAIgAgAE4AYQBtAGUAPQAiAEwAaQBnAGgAdAAgAEwAaQBzAHQAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgAyACIAIABOAGEAbQBlAD0AIgBMAGkAZwBoAHQAIABH\r\nAHIAaQBkACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8A\r\nYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMwAiACAATgBh\r\nAG0AZQA9ACIATQBlAGQAaQB1AG0AIABTAGgAYQBkAGkAbgBnACAAMQAiAC8APgANAAoAIAAgADwA\r\ndwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAi\r\nACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADQAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAA\r\nUwBoAGEAZABpAG4AZwAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIA\r\nNgA1ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAEwAaQBzAHQAIAAxACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYANgAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1\r\nAG0AIABMAGkAcwB0ACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2\r\nADcAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAARwByAGkAZAAgADEAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA4ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUA\r\nbQAgAEcAcgBpAGQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBv\r\nAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYA\r\nOQAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABHAHIAaQBkACAAMwAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA3ADAAIgAgAE4AYQBtAGUAPQAiAEQAYQByAGsAIABM\r\nAGkAcwB0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8A\r\nYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADcAMQAiACAATgBh\r\nAG0AZQA9ACIAQwBvAGwAbwByAGYAdQBsACAAUwBoAGEAZABpAG4AZwAiAC8APgANAAoAIAAgADwA\r\ndwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAi\r\nACAAUAByAGkAbwByAGkAdAB5AD0AIgA3ADIAIgAgAE4AYQBtAGUAPQAiAEMAbwBsAG8AcgBmAHUA\r\nbAAgAEwAaQBzAHQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAzACIA\r\nIABOAGEAbQBlAD0AIgBDAG8AbABvAHIAZgB1AGwAIABHAHIAaQBkACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMAAiACAATgBhAG0AZQA9ACIATABpAGcAaAB0ACAAUwBo\r\nAGEAZABpAG4AZwAgAEEAYwBjAGUAbgB0ACAAMQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA2ADEAIgAgAE4AYQBtAGUAPQAiAEwAaQBnAGgAdAAgAEwAaQBzAHQAIABBAGMA\r\nYwBlAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgAyACIA\r\nIABOAGEAbQBlAD0AIgBMAGkAZwBoAHQAIABHAHIAaQBkACAAQQBjAGMAZQBuAHQAIAAxACIALwA+\r\nAA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIA\r\nZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMwAiACAATgBhAG0AZQA9ACIATQBl\r\nAGQAaQB1AG0AIABTAGgAYQBkAGkAbgBnACAAMQAgAEEAYwBjAGUAbgB0ACAAMQAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADQAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkA\r\ndQBtACAAUwBoAGEAZABpAG4AZwAgADIAIABBAGMAYwBlAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA1ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAg\r\nAEwAaQBzAHQAIAAxACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABTAGUAbQBp\r\nAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABOAGEAbQBlAD0AIgBSAGUAdgBpAHMAaQBvAG4A\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMwA0ACIAIABRAEYAbwByAG0A\r\nYQB0AD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABQAGEAcgBh\r\nAGcAcgBhAHAAaAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAyADkAIgAg\r\nAFEARgBvAHIAbQBhAHQAPQAiAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAFEAdQBvAHQAZQAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAzADAAIgAgAFEARgBvAHIAbQBhAHQA\r\nPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIASQBuAHQAZQBuAHMAZQAgAFEAdQBv\r\nAHQAZQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMA\r\nawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADYAIgAgAE4AYQBt\r\nAGUAPQAiAE0AZQBkAGkAdQBtACAATABpAHMAdAAgADIAIABBAGMAYwBlAG4AdAAgADEAIgAvAD4A\r\nDQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBm\r\nAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA3ACIAIABOAGEAbQBlAD0AIgBNAGUA\r\nZABpAHUAbQAgAEcAcgBpAGQAIAAxACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAOAAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABH\r\nAHIAaQBkACAAMgAgAEEAYwBjAGUAbgB0ACAAMQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA2ADkAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAARwByAGkAZAAgADMA\r\nIABBAGMAYwBlAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIA\r\nNwAwACIAIABOAGEAbQBlAD0AIgBEAGEAcgBrACAATABpAHMAdAAgAEEAYwBjAGUAbgB0ACAAMQAi\r\nAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQA\r\nPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA3ADEAIgAgAE4AYQBtAGUAPQAi\r\nAEMAbwBsAG8AcgBmAHUAbAAgAFMAaABhAGQAaQBuAGcAIABBAGMAYwBlAG4AdAAgADEAIgAvAD4A\r\nDQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBm\r\nAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAyACIAIABOAGEAbQBlAD0AIgBDAG8A\r\nbABvAHIAZgB1AGwAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABQAHIAaQBvAHIAaQB0AHkAPQAiADcAMwAiACAATgBhAG0AZQA9ACIAQwBvAGwAbwByAGYAdQBs\r\nACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAAMQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA2ADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBnAGgAdAAgAFMAaABhAGQAaQBuAGcA\r\nIABBAGMAYwBlAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIA\r\nNgAxACIAIABOAGEAbQBlAD0AIgBMAGkAZwBoAHQAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAAy\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMgAiACAATgBhAG0AZQA9\r\nACIATABpAGcAaAB0ACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAgADwA\r\ndwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAi\r\nACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADMAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAA\r\nUwBoAGEAZABpAG4AZwAgADEAIABBAGMAYwBlAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBM\r\nAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAA\r\ncgBpAG8AcgBpAHQAeQA9ACIANgA0ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAFMAaABh\r\nAGQAaQBuAGcAIAAyACAAQQBjAGMAZQBuAHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBv\r\nAHIAaQB0AHkAPQAiADYANQAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABMAGkAcwB0ACAA\r\nMQAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0\r\nAGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0A\r\nIgA2ADYAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAATABpAHMAdAAgADIAIABBAGMAYwBl\r\nAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA3ACIAIABO\r\nAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAEcAcgBpAGQAIAAxACAAQQBjAGMAZQBuAHQAIAAyACIA\r\nLwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9\r\nACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAOAAiACAATgBhAG0AZQA9ACIA\r\nTQBlAGQAaQB1AG0AIABHAHIAaQBkACAAMgAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADkAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBt\r\nACAARwByAGkAZAAgADMAIABBAGMAYwBlAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMA\r\nZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBp\r\nAG8AcgBpAHQAeQA9ACIANwAwACIAIABOAGEAbQBlAD0AIgBEAGEAcgBrACAATABpAHMAdAAgAEEA\r\nYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBu\r\nACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA3ADEA\r\nIgAgAE4AYQBtAGUAPQAiAEMAbwBsAG8AcgBmAHUAbAAgAFMAaABhAGQAaQBuAGcAIABBAGMAYwBl\r\nAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAyACIAIABO\r\nAGEAbQBlAD0AIgBDAG8AbABvAHIAZgB1AGwAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAAyACIA\r\nLwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9\r\nACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADcAMwAiACAATgBhAG0AZQA9ACIA\r\nQwBvAGwAbwByAGYAdQBsACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBnAGgAdAAg\r\nAFMAaABhAGQAaQBuAGcAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMA\r\nZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBp\r\nAG8AcgBpAHQAeQA9ACIANgAxACIAIABOAGEAbQBlAD0AIgBMAGkAZwBoAHQAIABMAGkAcwB0ACAA\r\nQQBjAGMAZQBuAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBv\r\nAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYA\r\nMgAiACAATgBhAG0AZQA9ACIATABpAGcAaAB0ACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAAMwAi\r\nAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQA\r\nPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADMAIgAgAE4AYQBtAGUAPQAi\r\nAE0AZQBkAGkAdQBtACAAUwBoAGEAZABpAG4AZwAgADEAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4A\r\nDQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBm\r\nAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA0ACIAIABOAGEAbQBlAD0AIgBNAGUA\r\nZABpAHUAbQAgAFMAaABhAGQAaQBuAGcAIAAyACAAQQBjAGMAZQBuAHQAIAAzACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYANQAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1\r\nAG0AIABMAGkAcwB0ACAAMQAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwA\r\ncwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUABy\r\nAGkAbwByAGkAdAB5AD0AIgA2ADYAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAATABpAHMA\r\ndAAgADIAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIANgA3ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAEcAcgBpAGQAIAAxACAAQQBj\r\nAGMAZQBuAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAOAAi\r\nACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABHAHIAaQBkACAAMgAgAEEAYwBjAGUAbgB0ACAA\r\nMwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADkAIgAgAE4AYQBtAGUA\r\nPQAiAE0AZQBkAGkAdQBtACAARwByAGkAZAAgADMAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4ADQAK\r\nACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEA\r\nbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAwACIAIABOAGEAbQBlAD0AIgBEAGEAcgBr\r\nACAATABpAHMAdAAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA3ADEAIgAgAE4AYQBtAGUAPQAiAEMAbwBsAG8AcgBmAHUAbAAgAFMAaABhAGQA\r\naQBuAGcAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIANwAyACIAIABOAGEAbQBlAD0AIgBDAG8AbABvAHIAZgB1AGwAIABMAGkAcwB0ACAAQQBj\r\nAGMAZQBuAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADcAMwAi\r\nACAATgBhAG0AZQA9ACIAQwBvAGwAbwByAGYAdQBsACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAA\r\nMwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADAAIgAgAE4AYQBtAGUA\r\nPQAiAEwAaQBnAGgAdAAgAFMAaABhAGQAaQBuAGcAIABBAGMAYwBlAG4AdAAgADQAIgAvAD4ADQAK\r\nACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEA\r\nbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgAxACIAIABOAGEAbQBlAD0AIgBMAGkAZwBo\r\nAHQAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBv\r\nAHIAaQB0AHkAPQAiADYAMgAiACAATgBhAG0AZQA9ACIATABpAGcAaAB0ACAARwByAGkAZAAgAEEA\r\nYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBu\r\nACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADMA\r\nIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAAUwBoAGEAZABpAG4AZwAgADEAIABBAGMAYwBl\r\nAG4AdAAgADQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA0ACIAIABO\r\nAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAFMAaABhAGQAaQBuAGcAIAAyACAAQQBjAGMAZQBuAHQA\r\nIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYANQAiACAATgBhAG0A\r\nZQA9ACIATQBlAGQAaQB1AG0AIABMAGkAcwB0ACAAMQAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADYAIgAgAE4AYQBtAGUAPQAiAE0AZQBk\r\nAGkAdQBtACAATABpAHMAdAAgADIAIABBAGMAYwBlAG4AdAAgADQAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA3ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAEcA\r\ncgBpAGQAIAAxACAAQQBjAGMAZQBuAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIA\r\naQB0AHkAPQAiADYAOAAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABHAHIAaQBkACAAMgAg\r\nAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2\r\nADkAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAARwByAGkAZAAgADMAIABBAGMAYwBlAG4A\r\ndAAgADQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBj\r\nAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAwACIAIABOAGEA\r\nbQBlAD0AIgBEAGEAcgBrACAATABpAHMAdAAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA3ADEAIgAgAE4AYQBtAGUAPQAiAEMAbwBsAG8AcgBm\r\nAHUAbAAgAFMAaABhAGQAaQBuAGcAIABBAGMAYwBlAG4AdAAgADQAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAyACIAIABOAGEAbQBlAD0AIgBDAG8AbABvAHIAZgB1AGwA\r\nIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIA\r\naQB0AHkAPQAiADcAMwAiACAATgBhAG0AZQA9ACIAQwBvAGwAbwByAGYAdQBsACAARwByAGkAZAAg\r\nAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2\r\nADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBnAGgAdAAgAFMAaABhAGQAaQBuAGcAIABBAGMAYwBlAG4A\r\ndAAgADUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBj\r\nAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgAxACIAIABOAGEA\r\nbQBlAD0AIgBMAGkAZwBoAHQAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAA1ACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMgAiACAATgBhAG0AZQA9ACIATABpAGcAaAB0\r\nACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA2ADMAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAAUwBoAGEAZABpAG4A\r\nZwAgADEAIABBAGMAYwBlAG4AdAAgADUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIANgA0ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAFMAaABhAGQAaQBuAGcAIAAy\r\nACAAQQBjAGMAZQBuAHQAIAA1ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQA\r\naQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAi\r\nADYANQAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABMAGkAcwB0ACAAMQAgAEEAYwBjAGUA\r\nbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADYAIgAgAE4A\r\nYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAATABpAHMAdAAgADIAIABBAGMAYwBlAG4AdAAgADUAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA3ACIAIABOAGEAbQBlAD0AIgBN\r\nAGUAZABpAHUAbQAgAEcAcgBpAGQAIAAxACAAQQBjAGMAZQBuAHQAIAA1ACIALwA+AA0ACgAgACAA\r\nPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBl\r\nACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAOAAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0A\r\nIABHAHIAaQBkACAAMgAgAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkA\r\nbwByAGkAdAB5AD0AIgA2ADkAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAARwByAGkAZAAg\r\nADMAIABBAGMAYwBlAG4AdAAgADUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAA\r\ndABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9\r\nACIANwAwACIAIABOAGEAbQBlAD0AIgBEAGEAcgBrACAATABpAHMAdAAgAEEAYwBjAGUAbgB0ACAA\r\nNQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA3ADEAIgAgAE4AYQBtAGUA\r\nPQAiAEMAbwBsAG8AcgBmAHUAbAAgAFMAaABhAGQAaQBuAGcAIABBAGMAYwBlAG4AdAAgADUAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAyACIAIABOAGEAbQBlAD0AIgBD\r\nAG8AbABvAHIAZgB1AGwAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAA1ACIALwA+AA0ACgAgACAA\r\nPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBl\r\nACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADcAMwAiACAATgBhAG0AZQA9ACIAQwBvAGwAbwByAGYA\r\ndQBsACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkA\r\nbwByAGkAdAB5AD0AIgA2ADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBnAGgAdAAgAFMAaABhAGQAaQBu\r\nAGcAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAA\r\ndABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9\r\nACIANgAxACIAIABOAGEAbQBlAD0AIgBMAGkAZwBoAHQAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQA\r\nIAA2ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAMgAiACAATgBhAG0A\r\nZQA9ACIATABpAGcAaAB0ACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADMAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBt\r\nACAAUwBoAGEAZABpAG4AZwAgADEAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA0ACIAIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAFMA\r\naABhAGQAaQBuAGcAIAAyACAAQQBjAGMAZQBuAHQAIAA2ACIALwA+AA0ACgAgACAAPAB3ADoATABz\r\nAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIA\r\naQBvAHIAaQB0AHkAPQAiADYANQAiACAATgBhAG0AZQA9ACIATQBlAGQAaQB1AG0AIABMAGkAcwB0\r\nACAAMQAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5\r\nAD0AIgA2ADYAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkAdQBtACAATABpAHMAdAAgADIAIABBAGMA\r\nYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANgA3ACIA\r\nIABOAGEAbQBlAD0AIgBNAGUAZABpAHUAbQAgAEcAcgBpAGQAIAAxACAAQQBjAGMAZQBuAHQAIAA2\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADYAOAAiACAATgBhAG0AZQA9\r\nACIATQBlAGQAaQB1AG0AIABHAHIAaQBkACAAMgAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA2ADkAIgAgAE4AYQBtAGUAPQAiAE0AZQBkAGkA\r\ndQBtACAARwByAGkAZAAgADMAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBM\r\nAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAA\r\ncgBpAG8AcgBpAHQAeQA9ACIANwAwACIAIABOAGEAbQBlAD0AIgBEAGEAcgBrACAATABpAHMAdAAg\r\nAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA3\r\nADEAIgAgAE4AYQBtAGUAPQAiAEMAbwBsAG8AcgBmAHUAbAAgAFMAaABhAGQAaQBuAGcAIABBAGMA\r\nYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANwAyACIA\r\nIABOAGEAbQBlAD0AIgBDAG8AbABvAHIAZgB1AGwAIABMAGkAcwB0ACAAQQBjAGMAZQBuAHQAIAA2\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADcAMwAiACAATgBhAG0AZQA9\r\nACIAQwBvAGwAbwByAGYAdQBsACAARwByAGkAZAAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAxADkAIgAgAFEARgBvAHIAbQBhAHQAPQAiAHQA\r\ncgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAUwB1AGIAdABsAGUAIABFAG0AcABoAGEAcwBp\r\nAHMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsA\r\nZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMgAxACIAIABRAEYAbwBy\r\nAG0AYQB0AD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEkAbgB0AGUAbgBzAGUA\r\nIABFAG0AcABoAGEAcwBpAHMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIA\r\nMwAxACIAIABRAEYAbwByAG0AYQB0AD0AIgB0AHIAdQBlACIADQAKACAAIAAgAE4AYQBtAGUAPQAi\r\nAFMAdQBiAHQAbABlACAAUgBlAGYAZQByAGUAbgBjAGUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMA\r\nZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBp\r\nAG8AcgBpAHQAeQA9ACIAMwAyACIAIABRAEYAbwByAG0AYQB0AD0AIgB0AHIAdQBlACIADQAKACAA\r\nIAAgAE4AYQBtAGUAPQAiAEkAbgB0AGUAbgBzAGUAIABSAGUAZgBlAHIAZQBuAGMAZQAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgAzADMAIgAgAFEARgBvAHIAbQBhAHQAPQAi\r\nAHQAcgB1AGUAIgAgAE4AYQBtAGUAPQAiAEIAbwBvAGsAIABUAGkAdABsAGUAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMwA3ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0A\r\nIgB0AHIAdQBlACIADQAKACAAIAAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdABy\r\nAHUAZQAiACAATgBhAG0AZQA9ACIAQgBpAGIAbABpAG8AZwByAGEAcABoAHkAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIAMwA5ACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0A\r\nIgB0AHIAdQBlACIADQAKACAAIAAgAFUAbgBoAGkAZABlAFcAaABlAG4AVQBzAGUAZAA9ACIAdABy\r\nAHUAZQAiACAAUQBGAG8AcgBtAGEAdAA9ACIAdAByAHUAZQAiACAATgBhAG0AZQA9ACIAVABPAEMA\r\nIABIAGUAYQBkAGkAbgBnACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBv\r\nAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQA\r\nMQAiACAATgBhAG0AZQA9ACIAUABsAGEAaQBuACAAVABhAGIAbABlACAAMQAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADIAIgAgAE4AYQBtAGUAPQAiAFAAbABhAGkAbgAg\r\nAFQAYQBiAGwAZQAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8A\r\nbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAAz\r\nACIAIABOAGEAbQBlAD0AIgBQAGwAYQBpAG4AIABUAGEAYgBsAGUAIAAzACIALwA+AA0ACgAgACAA\r\nPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBl\r\nACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANAAiACAATgBhAG0AZQA9ACIAUABsAGEAaQBuACAA\r\nVABhAGIAbABlACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBu\r\nACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADUA\r\nIgAgAE4AYQBtAGUAPQAiAFAAbABhAGkAbgAgAFQAYQBiAGwAZQAgADUAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAAwACIAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABh\r\nAGIAbABlACAATABpAGcAaAB0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQA\r\naQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAi\r\nADQANgAiACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADEAIABMAGkAZwBoAHQA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA3ACIAIABOAGEAbQBlAD0A\r\nIgBHAHIAaQBkACAAVABhAGIAbABlACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkA\r\ndAB5AD0AIgA0ADgAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAAzACIALwA+\r\nAA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIA\r\nZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOQAiACAATgBhAG0AZQA9ACIARwBy\r\nAGkAZAAgAFQAYQBiAGwAZQAgADQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAA\r\ndABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9\r\nACIANQAwACIAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAANQAgAEQAYQByAGsA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAxACIAIABOAGEAbQBlAD0A\r\nIgBHAHIAaQBkACAAVABhAGIAbABlACAANgAgAEMAbwBsAG8AcgBmAHUAbAAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADIAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABU\r\nAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBv\r\nAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIA\r\nbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAAMQAiAC8APgANAAoAIAAgADwAdwA6\r\nAEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAA\r\nUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBs\r\nAGUAIAAyACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMA\r\nZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0\r\nAHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADMAIABBAGMA\r\nYwBlAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAg\r\nAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIA\r\nIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAAMQAi\r\nAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQA\r\nPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAi\r\nAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAAMQAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0A\r\nZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBl\r\nAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAK\r\nACAAIAAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYA\r\ndQBsACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBw\r\nAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkA\r\nPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAAMQAg\r\nAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAAyACAA\r\nQQBjAGMAZQBuAHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBv\r\nAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQA\r\nOAAiACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAg\r\nADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsA\r\nZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBl\r\nAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoA\r\nIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBs\r\nAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQA\r\nIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARwBy\r\nAGkAZAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADIA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4A\r\nYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBj\r\nAGMAZQBuAHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAi\r\nAA0ACgAgACAAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgA\r\ndAAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0\r\nAGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0A\r\nIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBu\r\nAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8A\r\nYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBh\r\nAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4A\r\nDQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBm\r\nAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBHAHIA\r\naQBkACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6\r\nAEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAA\r\nUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBs\r\nAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwA\r\ncwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUABy\r\nAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQA\r\nYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADMAIgAvAD4ADQAK\r\nACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEA\r\nbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAi\r\nAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQA\r\nIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAA\r\nIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBj\r\nAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAg\r\nAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQAIAA0ACIA\r\nLwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9\r\nACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIA\r\nRwByAGkAZAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADQAIgAvAD4ADQAKACAAIAA8\r\nAHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUA\r\nIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABh\r\nAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUA\r\neABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwBy\r\nAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA1ACAA\r\nRABhAHIAawAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkA\r\ndAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAg\r\nADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADQAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEcAcgBpAGQA\r\nIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAA0ACIALwA+\r\nAA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIA\r\nZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBl\r\nAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAA\r\nNQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUA\r\nPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQAIAA1ACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIARwByAGkAZAAg\r\nAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMA\r\nZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBp\r\nAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAA\r\nNAAgAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0\r\nAGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0A\r\nIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAg\r\nAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1\r\nADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADYAIABDAG8A\r\nbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADUAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABF\r\nAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8A\r\ncgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBs\r\nAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAA1ACIALwA+AA0ACgAgACAA\r\nPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBl\r\nACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBHAHIA\r\naQBkACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEcAcgBp\r\nAGQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQAIAA2ACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQ\r\nAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwA\r\nZQAgADMAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBHAHIAaQBkACAAVABhAGIAbABlACAANAAgAEEAYwBj\r\nAGUAbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAg\r\nAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUA\r\nbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoA\r\nIAAgACAATgBhAG0AZQA9ACIARwByAGkAZAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1\r\nAGwAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAA\r\ndABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9\r\nACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEcAcgBpAGQAIABUAGEAYgBsAGUAIAA3ACAA\r\nQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAA2ACIALwA+AA0ACgAgACAAPAB3ADoATABz\r\nAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIA\r\naQBvAHIAaQB0AHkAPQAiADQANgAiACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAg\r\nADEAIABMAGkAZwBoAHQAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8A\r\nbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA3\r\nACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAAMgAiAC8APgANAAoAIAAgADwA\r\ndwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAi\r\nACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADgAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEA\r\nYgBsAGUAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABM\r\nAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOQAiACAA\r\nTgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADQAIgAvAD4ADQAKACAAIAA8AHcAOgBM\r\nAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAA\r\ncgBpAG8AcgBpAHQAeQA9ACIANQAwACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABl\r\nACAANQAgAEQAYQByAGsAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8A\r\nbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAx\r\nACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAANgAgAEMAbwBsAG8AcgBmAHUA\r\nbAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADIAIgAgAE4AYQBtAGUA\r\nPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBM\r\nAGkAcwB0ACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAAMQAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEwA\r\naQBzAHQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3\r\nADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIA\r\nIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBi\r\nAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgA\r\nYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBp\r\nAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAANAAgAEEA\r\nYwBjAGUAbgB0ACAAMQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBu\r\nACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAA\r\nIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBj\r\nAGUAbgB0ACAAMQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgAN\r\nAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIA\r\nZgB1AGwAIABBAGMAYwBlAG4AdAAgADEAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBl\r\nAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQA\r\neQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA3\r\nACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAAxACIALwA+AA0ACgAgACAAPAB3ADoA\r\nTABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQ\r\nAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAA\r\nVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAAMgAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABU\r\nAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQA\r\nRQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBv\r\nAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADMA\r\nIABBAGMAYwBlAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABp\r\nAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIA\r\nNAA5ACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0\r\nACAAMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMA\r\nawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBt\r\nAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAA\r\nMgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBl\r\nAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAA\r\nTgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABB\r\nAGMAYwBlAG4AdAAgADIAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8A\r\nbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAy\r\nACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwA\r\nbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAAyACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4\r\nAGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIA\r\naQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABl\r\nACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgANAAoAIAAgADwAdwA6AEwA\r\ncwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUABy\r\nAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUA\r\nIAAyACAAQQBjAGMAZQBuAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBw\r\nAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkA\r\nPQAiADQAOAAiACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBl\r\nAG4AdAAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwA\r\nbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABO\r\nAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAAMwAiAC8A\r\nPgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAi\r\nAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEwA\r\naQBzAHQAIABUAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAAMwAiAC8APgAN\r\nAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYA\r\nYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9\r\nACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4A\r\ndAAgADMAIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBj\r\nAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAA\r\nIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBs\r\nACAAQQBjAGMAZQBuAHQAIAAzACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQA\r\naQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAi\r\nADQANgAiAA0ACgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAAMQAgAEwA\r\naQBnAGgAdAAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkA\r\ndAB5AD0AIgA0ADcAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAAyACAAQQBj\r\nAGMAZQBuAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4A\r\nIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAi\r\nACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADQA\r\nIgAvAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBk\r\nAD0AIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0A\r\nIgBMAGkAcwB0ACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAg\r\nADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMA\r\nZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABU\r\nAGEAYgBsAGUAIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAANAAiAC8APgANAAoAIAAgADwA\r\ndwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAi\r\nACAAUAByAGkAbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMA\r\ndAAgAFQAYQBiAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADQAIgAv\r\nAD4ADQAKACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0A\r\nIgBmAGEAbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBt\r\nAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMA\r\nZQBuAHQAIAA0ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABM\r\nAG8AYwBrAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0A\r\nCgAgACAAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAg\r\nAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkA\r\nbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0\r\nADcAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQA\r\nIAA1ACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBr\r\nAGUAZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0A\r\nZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADUAIgAvAD4ADQAK\r\nACAAIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEA\r\nbABzAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0\r\nACAAVABhAGIAbABlACAANAAgAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwA\r\ncwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUABy\r\nAGkAbwByAGkAdAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUA\r\nIAA1ACAARABhAHIAawAgAEEAYwBjAGUAbgB0ACAANQAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBk\r\nAEUAeABjAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkA\r\nbwByAGkAdAB5AD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBi\r\nAGwAZQAgADYAIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADUAIgAvAD4ADQAKACAA\r\nIAA8AHcAOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABz\r\nAGUAIgAgAFAAcgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwA\r\naQBzAHQAIABUAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAA1\r\nACIALwA+AA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUA\r\nZAA9ACIAZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQANgAiAA0ACgAgACAAIABO\r\nAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIAbABlACAAMQAgAEwAaQBnAGgAdAAgAEEAYwBjAGUA\r\nbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5AD0AIgA0ADcAIgAgAE4A\r\nYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAAyACAAQQBjAGMAZQBuAHQAIAA2ACIALwA+\r\nAA0ACgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIA\r\nZgBhAGwAcwBlACIAIABQAHIAaQBvAHIAaQB0AHkAPQAiADQAOAAiACAATgBhAG0AZQA9ACIATABp\r\nAHMAdAAgAFQAYQBiAGwAZQAgADMAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcA\r\nOgBMAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAg\r\nAFAAcgBpAG8AcgBpAHQAeQA9ACIANAA5ACIAIABOAGEAbQBlAD0AIgBMAGkAcwB0ACAAVABhAGIA\r\nbABlACAANAAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABj\r\nAGUAcAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkA\r\ndAB5AD0AIgA1ADAAIgAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABUAGEAYgBsAGUAIAA1ACAARABh\r\nAHIAawAgAEEAYwBjAGUAbgB0ACAANgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUA\r\ncAB0AGkAbwBuACAATABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUAByAGkAbwByAGkAdAB5\r\nAD0AIgA1ADEAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATABpAHMAdAAgAFQAYQBiAGwAZQAgADYA\r\nIABDAG8AbABvAHIAZgB1AGwAIABBAGMAYwBlAG4AdAAgADYAIgAvAD4ADQAKACAAIAA8AHcAOgBM\r\nAHMAZABFAHgAYwBlAHAAdABpAG8AbgAgAEwAbwBjAGsAZQBkAD0AIgBmAGEAbABzAGUAIgAgAFAA\r\ncgBpAG8AcgBpAHQAeQA9ACIANQAyACIADQAKACAAIAAgAE4AYQBtAGUAPQAiAEwAaQBzAHQAIABU\r\nAGEAYgBsAGUAIAA3ACAAQwBvAGwAbwByAGYAdQBsACAAQQBjAGMAZQBuAHQAIAA2ACIALwA+AA0A\r\nCgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBh\r\nAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQA\r\nZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIATQBl\r\nAG4AdABpAG8AbgAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAA\r\nTABvAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdABy\r\nAHUAZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAA\r\nIAAgAE4AYQBtAGUAPQAiAFMAbQBhAHIAdAAgAEgAeQBwAGUAcgBsAGkAbgBrACIALwA+AA0ACgAg\r\nACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBhAGwA\r\ncwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQAZQBX\r\nAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIASABhAHMA\r\naAB0AGEAZwAiAC8APgANAAoAIAAgADwAdwA6AEwAcwBkAEUAeABjAGUAcAB0AGkAbwBuACAATABv\r\nAGMAawBlAGQAPQAiAGYAYQBsAHMAZQAiACAAUwBlAG0AaQBIAGkAZABkAGUAbgA9ACIAdAByAHUA\r\nZQAiACAAVQBuAGgAaQBkAGUAVwBoAGUAbgBVAHMAZQBkAD0AIgB0AHIAdQBlACIADQAKACAAIAAg\r\nAE4AYQBtAGUAPQAiAFUAbgByAGUAcwBvAGwAdgBlAGQAIABNAGUAbgB0AGkAbwBuACIALwA+AA0A\r\nCgAgACAAPAB3ADoATABzAGQARQB4AGMAZQBwAHQAaQBvAG4AIABMAG8AYwBrAGUAZAA9ACIAZgBh\r\nAGwAcwBlACIAIABTAGUAbQBpAEgAaQBkAGQAZQBuAD0AIgB0AHIAdQBlACIAIABVAG4AaABpAGQA\r\nZQBXAGgAZQBuAFUAcwBlAGQAPQAiAHQAcgB1AGUAIgANAAoAIAAgACAATgBhAG0AZQA9ACIAUwBt\r\nAGEAcgB0ACAATABpAG4AawAiAC8APgANAAoAIAA8AC8AdwA6AEwAYQB0AGUAbgB0AFMAdAB5AGwA\r\nZQBzAD4ADQAKADwALwB4AG0AbAA+ADwAIQBbAGUAbgBkAGkAZgBdAC0ALQA+AA0ACgA8AHMAdAB5\r\nAGwAZQA+AA0ACgA8ACEALQAtAA0ACgAgAC8AKgAgAEYAbwBuAHQAIABEAGUAZgBpAG4AaQB0AGkA\r\nbwBuAHMAIAAqAC8ADQAKACAAQABmAG8AbgB0AC0AZgBhAGMAZQANAAoACQB7AGYAbwBuAHQALQBm\r\nAGEAbQBpAGwAeQA6ACIAQwBhAG0AYgByAGkAYQAgAE0AYQB0AGgAIgA7AA0ACgAJAHAAYQBuAG8A\r\ncwBlAC0AMQA6ADIAIAA0ACAANQAgADMAIAA1ACAANAAgADYAIAAzACAAMgAgADQAOwANAAoACQBt\r\nAHMAbwAtAGYAbwBuAHQALQBjAGgAYQByAHMAZQB0ADoAMAA7AA0ACgAJAG0AcwBvAC0AZwBlAG4A\r\nZQByAGkAYwAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AHIAbwBtAGEAbgA7AA0ACgAJAG0AcwBv\r\nAC0AZgBvAG4AdAAtAHAAaQB0AGMAaAA6AHYAYQByAGkAYQBiAGwAZQA7AA0ACgAJAG0AcwBvAC0A\r\nZgBvAG4AdAAtAHMAaQBnAG4AYQB0AHUAcgBlADoALQA1ADMANgA4ADYAOQAxADIAMQAgADEAMQAw\r\nADcAMwAwADUANwAyADcAIAAzADMANQA1ADQANAAzADIAIAAwACAANAAxADUAIAAwADsAfQANAAoA\r\nQABmAG8AbgB0AC0AZgBhAGMAZQANAAoACQB7AGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AEMAYQBs\r\nAGkAYgByAGkAOwANAAoACQBwAGEAbgBvAHMAZQAtADEAOgAyACAAMQA1ACAANQAgADIAIAAyACAA\r\nMgAgADQAIAAzACAAMgAgADQAOwANAAoACQBtAHMAbwAtAGYAbwBuAHQALQBjAGgAYQByAHMAZQB0\r\nADoAMAA7AA0ACgAJAG0AcwBvAC0AZwBlAG4AZQByAGkAYwAtAGYAbwBuAHQALQBmAGEAbQBpAGwA\r\neQA6AHMAdwBpAHMAcwA7AA0ACgAJAG0AcwBvAC0AZgBvAG4AdAAtAHAAaQB0AGMAaAA6AHYAYQBy\r\nAGkAYQBiAGwAZQA7AA0ACgAJAG0AcwBvAC0AZgBvAG4AdAAtAHMAaQBnAG4AYQB0AHUAcgBlADoA\r\nLQA0ADYAOQA3ADUAMAAwADEANwAgAC0AMQAwADQAMAAxADcAOAAwADUAMwAgADkAIAAwACAANQAx\r\nADEAIAAwADsAfQANAAoAQABmAG8AbgB0AC0AZgBhAGMAZQANAAoACQB7AGYAbwBuAHQALQBmAGEA\r\nbQBpAGwAeQA6AEEAcAB0AG8AcwA7AA0ACgAJAG0AcwBvAC0AZgBvAG4AdAAtAGMAaABhAHIAcwBl\r\nAHQAOgAwADsADQAKAAkAbQBzAG8ALQBnAGUAbgBlAHIAaQBjAC0AZgBvAG4AdAAtAGYAYQBtAGkA\r\nbAB5ADoAcwB3AGkAcwBzADsADQAKAAkAbQBzAG8ALQBmAG8AbgB0AC0AcABpAHQAYwBoADoAdgBh\r\nAHIAaQBhAGIAbABlADsADQAKAAkAbQBzAG8ALQBmAG8AbgB0AC0AcwBpAGcAbgBhAHQAdQByAGUA\r\nOgA1ADMANgA4ADcAMQA1ADUAOQAgADMAIAAwACAAMAAgADQAMQA1ACAAMAA7AH0ADQAKAEAAZgBv\r\nAG4AdAAtAGYAYQBjAGUADQAKAAkAewBmAG8AbgB0AC0AZgBhAG0AaQBsAHkAOgAiAFMAZQBnAG8A\r\nZQAgAFUASQAgAEUAbQBvAGoAaQAiADsADQAKAAkAcABhAG4AbwBzAGUALQAxADoAMgAgADEAMQAg\r\nADUAIAAyACAANAAgADIAIAA0ACAAMgAgADIAIAAzADsADQAKAAkAbQBzAG8ALQBmAG8AbgB0AC0A\r\nYwBoAGEAcgBzAGUAdAA6ADAAOwANAAoACQBtAHMAbwAtAGcAZQBuAGUAcgBpAGMALQBmAG8AbgB0\r\nAC0AZgBhAG0AaQBsAHkAOgBzAHcAaQBzAHMAOwANAAoACQBtAHMAbwAtAGYAbwBuAHQALQBwAGkA\r\ndABjAGgAOgB2AGEAcgBpAGEAYgBsAGUAOwANAAoACQBtAHMAbwAtAGYAbwBuAHQALQBzAGkAZwBu\r\nAGEAdAB1AHIAZQA6ADMAIAAzADMANQA1ADQANAAzADIAIAAxADMANAAyADEANwA3ADIAOAAgADAA\r\nIAAxACAAMAA7AH0ADQAKACAALwAqACAAUwB0AHkAbABlACAARABlAGYAaQBuAGkAdABpAG8AbgBz\r\nACAAKgAvAA0ACgAgAHAALgBNAHMAbwBOAG8AcgBtAGEAbAAsACAAbABpAC4ATQBzAG8ATgBvAHIA\r\nbQBhAGwALAAgAGQAaQB2AC4ATQBzAG8ATgBvAHIAbQBhAGwADQAKAAkAewBtAHMAbwAtAHMAdAB5\r\nAGwAZQAtAHUAbgBoAGkAZABlADoAbgBvADsADQAKAAkAbQBzAG8ALQBzAHQAeQBsAGUALQBxAGYA\r\nbwByAG0AYQB0ADoAeQBlAHMAOwANAAoACQBtAHMAbwAtAHMAdAB5AGwAZQAtAHAAYQByAGUAbgB0\r\nADoAIgAiADsADQAKAAkAbQBhAHIAZwBpAG4AOgAwAGkAbgA7AA0ACgAJAG0AcwBvAC0AcABhAGcA\r\naQBuAGEAdABpAG8AbgA6AHcAaQBkAG8AdwAtAG8AcgBwAGgAYQBuADsADQAKAAkAZgBvAG4AdAAt\r\nAHMAaQB6AGUAOgAxADIALgAwAHAAdAA7AA0ACgAJAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6ACIA\r\nQQBwAHQAbwBzACIALABzAGEAbgBzAC0AcwBlAHIAaQBmADsADQAKAAkAbQBzAG8ALQBmAGEAcgBl\r\nAGEAcwB0AC0AZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAQQBwAHQAbwBzADsADQAKAAkAbQBzAG8A\r\nLQBmAGEAcgBlAGEAcwB0AC0AdABoAGUAbQBlAC0AZgBvAG4AdAA6AG0AaQBuAG8AcgAtAGwAYQB0\r\nAGkAbgA7AA0ACgAJAG0AcwBvAC0AYgBpAGQAaQAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AEEA\r\ncAB0AG8AcwA7AH0ADQAKAHAADQAKAAkAewBtAHMAbwAtAHMAdAB5AGwAZQAtAG4AbwBzAGgAbwB3\r\nADoAeQBlAHMAOwANAAoACQBtAHMAbwAtAHMAdAB5AGwAZQAtAHAAcgBpAG8AcgBpAHQAeQA6ADkA\r\nOQA7AA0ACgAJAG0AcwBvAC0AbQBhAHIAZwBpAG4ALQB0AG8AcAAtAGEAbAB0ADoAYQB1AHQAbwA7\r\nAA0ACgAJAG0AYQByAGcAaQBuAC0AcgBpAGcAaAB0ADoAMABpAG4AOwANAAoACQBtAHMAbwAtAG0A\r\nYQByAGcAaQBuAC0AYgBvAHQAdABvAG0ALQBhAGwAdAA6AGEAdQB0AG8AOwANAAoACQBtAGEAcgBn\r\nAGkAbgAtAGwAZQBmAHQAOgAwAGkAbgA7AA0ACgAJAG0AcwBvAC0AcABhAGcAaQBuAGEAdABpAG8A\r\nbgA6AHcAaQBkAG8AdwAtAG8AcgBwAGgAYQBuADsADQAKAAkAZgBvAG4AdAAtAHMAaQB6AGUAOgAx\r\nADIALgAwAHAAdAA7AA0ACgAJAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6ACIAQQBwAHQAbwBzACIA\r\nLABzAGEAbgBzAC0AcwBlAHIAaQBmADsADQAKAAkAbQBzAG8ALQBmAGEAcgBlAGEAcwB0AC0AZgBv\r\nAG4AdAAtAGYAYQBtAGkAbAB5ADoAQQBwAHQAbwBzADsADQAKAAkAbQBzAG8ALQBmAGEAcgBlAGEA\r\ncwB0AC0AdABoAGUAbQBlAC0AZgBvAG4AdAA6AG0AaQBuAG8AcgAtAGwAYQB0AGkAbgA7AA0ACgAJ\r\nAG0AcwBvAC0AYgBpAGQAaQAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AEEAcAB0AG8AcwA7AH0A\r\nDQAKAHAALgBtAHMAbwBuAG8AcgBtAGEAbAAwACwAIABsAGkALgBtAHMAbwBuAG8AcgBtAGEAbAAw\r\nACwAIABkAGkAdgAuAG0AcwBvAG4AbwByAG0AYQBsADAADQAKAAkAewBtAHMAbwAtAHMAdAB5AGwA\r\nZQAtAG4AYQBtAGUAOgBtAHMAbwBuAG8AcgBtAGEAbAA7AA0ACgAJAG0AcwBvAC0AcwB0AHkAbABl\r\nAC0AdQBuAGgAaQBkAGUAOgBuAG8AOwANAAoACQBtAHMAbwAtAG0AYQByAGcAaQBuAC0AdABvAHAA\r\nLQBhAGwAdAA6AGEAdQB0AG8AOwANAAoACQBtAGEAcgBnAGkAbgAtAHIAaQBnAGgAdAA6ADAAaQBu\r\nADsADQAKAAkAbQBzAG8ALQBtAGEAcgBnAGkAbgAtAGIAbwB0AHQAbwBtAC0AYQBsAHQAOgBhAHUA\r\ndABvADsADQAKAAkAbQBhAHIAZwBpAG4ALQBsAGUAZgB0ADoAMABpAG4AOwANAAoACQBtAHMAbwAt\r\nAHAAYQBnAGkAbgBhAHQAaQBvAG4AOgB3AGkAZABvAHcALQBvAHIAcABoAGEAbgA7AA0ACgAJAGYA\r\nbwBuAHQALQBzAGkAegBlADoAMQAyAC4AMABwAHQAOwANAAoACQBmAG8AbgB0AC0AZgBhAG0AaQBs\r\nAHkAOgAiAEEAcAB0AG8AcwAiACwAcwBhAG4AcwAtAHMAZQByAGkAZgA7AA0ACgAJAG0AcwBvAC0A\r\nZgBhAHIAZQBhAHMAdAAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AEEAcAB0AG8AcwA7AA0ACgAJ\r\nAG0AcwBvAC0AZgBhAHIAZQBhAHMAdAAtAHQAaABlAG0AZQAtAGYAbwBuAHQAOgBtAGkAbgBvAHIA\r\nLQBsAGEAdABpAG4AOwANAAoACQBtAHMAbwAtAGIAaQBkAGkALQBmAG8AbgB0AC0AZgBhAG0AaQBs\r\nAHkAOgBBAHAAdABvAHMAOwB9AA0ACgAuAE0AcwBvAEMAaABwAEQAZQBmAGEAdQBsAHQADQAKAAkA\r\newBtAHMAbwAtAHMAdAB5AGwAZQAtAHQAeQBwAGUAOgBlAHgAcABvAHIAdAAtAG8AbgBsAHkAOwAN\r\nAAoACQBtAHMAbwAtAGQAZQBmAGEAdQBsAHQALQBwAHIAbwBwAHMAOgB5AGUAcwA7AA0ACgAJAG0A\r\ncwBvAC0AYQBzAGMAaQBpAC0AZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAQQBwAHQAbwBzADsADQAK\r\nAAkAbQBzAG8ALQBhAHMAYwBpAGkALQB0AGgAZQBtAGUALQBmAG8AbgB0ADoAbQBpAG4AbwByAC0A\r\nbABhAHQAaQBuADsADQAKAAkAbQBzAG8ALQBmAGEAcgBlAGEAcwB0AC0AZgBvAG4AdAAtAGYAYQBt\r\nAGkAbAB5ADoAQQBwAHQAbwBzADsADQAKAAkAbQBzAG8ALQBmAGEAcgBlAGEAcwB0AC0AdABoAGUA\r\nbQBlAC0AZgBvAG4AdAA6AG0AaQBuAG8AcgAtAGwAYQB0AGkAbgA7AA0ACgAJAG0AcwBvAC0AaABh\r\nAG4AcwBpAC0AZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAQQBwAHQAbwBzADsADQAKAAkAbQBzAG8A\r\nLQBoAGEAbgBzAGkALQB0AGgAZQBtAGUALQBmAG8AbgB0ADoAbQBpAG4AbwByAC0AbABhAHQAaQBu\r\nADsADQAKAAkAbQBzAG8ALQBiAGkAZABpAC0AZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAIgBUAGkA\r\nbQBlAHMAIABOAGUAdwAgAFIAbwBtAGEAbgAiADsADQAKAAkAbQBzAG8ALQBiAGkAZABpAC0AdABo\r\nAGUAbQBlAC0AZgBvAG4AdAA6AG0AaQBuAG8AcgAtAGIAaQBkAGkAOwB9AA0ACgBAAHAAYQBnAGUA\r\nIABXAG8AcgBkAFMAZQBjAHQAaQBvAG4AMQANAAoACQB7AHMAaQB6AGUAOgA4AC4ANQBpAG4AIAAx\r\nADEALgAwAGkAbgA7AA0ACgAJAG0AYQByAGcAaQBuADoAMQAuADAAaQBuACAAMQAuADAAaQBuACAA\r\nMQAuADAAaQBuACAAMQAuADAAaQBuADsADQAKAAkAbQBzAG8ALQBoAGUAYQBkAGUAcgAtAG0AYQBy\r\nAGcAaQBuADoALgA1AGkAbgA7AA0ACgAJAG0AcwBvAC0AZgBvAG8AdABlAHIALQBtAGEAcgBnAGkA\r\nbgA6AC4ANQBpAG4AOwANAAoACQBtAHMAbwAtAHAAYQBwAGUAcgAtAHMAbwB1AHIAYwBlADoAMAA7\r\nAH0ADQAKAGQAaQB2AC4AVwBvAHIAZABTAGUAYwB0AGkAbwBuADEADQAKAAkAewBwAGEAZwBlADoA\r\nVwBvAHIAZABTAGUAYwB0AGkAbwBuADEAOwB9AA0ACgAtAC0APgANAAoAPAAvAHMAdAB5AGwAZQA+\r\nAA0ACgA8ACEALQAtAFsAaQBmACAAZwB0AGUAIABtAHMAbwAgADEAMABdAD4ADQAKADwAcwB0AHkA\r\nbABlAD4ADQAKACAALwAqACAAUwB0AHkAbABlACAARABlAGYAaQBuAGkAdABpAG8AbgBzACAAKgAv\r\nAA0ACgAgAHQAYQBiAGwAZQAuAE0AcwBvAE4AbwByAG0AYQBsAFQAYQBiAGwAZQANAAoACQB7AG0A\r\ncwBvAC0AcwB0AHkAbABlAC0AbgBhAG0AZQA6ACIAVABhAGIAbABlACAATgBvAHIAbQBhAGwAIgA7\r\nAA0ACgAJAG0AcwBvAC0AdABzAHQAeQBsAGUALQByAG8AdwBiAGEAbgBkAC0AcwBpAHoAZQA6ADAA\r\nOwANAAoACQBtAHMAbwAtAHQAcwB0AHkAbABlAC0AYwBvAGwAYgBhAG4AZAAtAHMAaQB6AGUAOgAw\r\nADsADQAKAAkAbQBzAG8ALQBzAHQAeQBsAGUALQBuAG8AcwBoAG8AdwA6AHkAZQBzADsADQAKAAkA\r\nbQBzAG8ALQBzAHQAeQBsAGUALQBwAHIAaQBvAHIAaQB0AHkAOgA5ADkAOwANAAoACQBtAHMAbwAt\r\nAHMAdAB5AGwAZQAtAHAAYQByAGUAbgB0ADoAIgAiADsADQAKAAkAbQBzAG8ALQBwAGEAZABkAGkA\r\nbgBnAC0AYQBsAHQAOgAwAGkAbgAgADUALgA0AHAAdAAgADAAaQBuACAANQAuADQAcAB0ADsADQAK\r\nAAkAbQBzAG8ALQBwAGEAcgBhAC0AbQBhAHIAZwBpAG4AOgAwAGkAbgA7AA0ACgAJAG0AcwBvAC0A\r\ncABhAGcAaQBuAGEAdABpAG8AbgA6AHcAaQBkAG8AdwAtAG8AcgBwAGgAYQBuADsADQAKAAkAZgBv\r\nAG4AdAAtAHMAaQB6AGUAOgAxADIALgAwAHAAdAA7AA0ACgAJAGYAbwBuAHQALQBmAGEAbQBpAGwA\r\neQA6ACIAQQBwAHQAbwBzACIALABzAGEAbgBzAC0AcwBlAHIAaQBmADsADQAKAAkAbQBzAG8ALQBh\r\nAHMAYwBpAGkALQBmAG8AbgB0AC0AZgBhAG0AaQBsAHkAOgBBAHAAdABvAHMAOwANAAoACQBtAHMA\r\nbwAtAGEAcwBjAGkAaQAtAHQAaABlAG0AZQAtAGYAbwBuAHQAOgBtAGkAbgBvAHIALQBsAGEAdABp\r\nAG4AOwANAAoACQBtAHMAbwAtAGgAYQBuAHMAaQAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AEEA\r\ncAB0AG8AcwA7AA0ACgAJAG0AcwBvAC0AaABhAG4AcwBpAC0AdABoAGUAbQBlAC0AZgBvAG4AdAA6\r\nAG0AaQBuAG8AcgAtAGwAYQB0AGkAbgA7AA0ACgAJAG0AcwBvAC0AZgBvAG4AdAAtAGsAZQByAG4A\r\naQBuAGcAOgAxAC4AMABwAHQAOwANAAoACQBtAHMAbwAtAGwAaQBnAGEAdAB1AHIAZQBzADoAcwB0\r\nAGEAbgBkAGEAcgBkAGMAbwBuAHQAZQB4AHQAdQBhAGwAOwB9AA0ACgA8AC8AcwB0AHkAbABlAD4A\r\nDQAKADwAIQBbAGUAbgBkAGkAZgBdAC0ALQA+AA0ACgA8AC8AaABlAGEAZAA+AA0ACgANAAoAPABi\r\nAG8AZAB5ACAAbABhAG4AZwA9AEUATgAtAFUAUwAgAHMAdAB5AGwAZQA9ACcAdABhAGIALQBpAG4A\r\ndABlAHIAdgBhAGwAOgAuADUAaQBuADsAdwBvAHIAZAAtAHcAcgBhAHAAOgBiAHIAZQBhAGsALQB3\r\nAG8AcgBkACcAPgANAAoADQAKADwAZABpAHYAIABjAGwAYQBzAHMAPQBXAG8AcgBkAFMAZQBjAHQA\r\naQBvAG4AMQA+AA0ACgANAAoAPABwACAAYwBsAGEAcwBzAD0ATQBzAG8ATgBvAHIAbQBhAGwAIABz\r\nAHQAeQBsAGUAPQAnAG0AYQByAGcAaQBuAC0AbABlAGYAdAA6ADIAMgA1AC4AMABwAHQAOwB0AGUA\r\neAB0AC0AaQBuAGQAZQBuAHQAOgAtADIAMgA1AC4AMABwAHQAOwB0AGEAYgAtAHMAdABvAHAAcwA6\r\nAA0ACgAyADIANQAuADAAcAB0ADsAbQBzAG8ALQBsAGEAeQBvAHUAdAAtAGcAcgBpAGQALQBhAGwA\r\naQBnAG4AOgBuAG8AbgBlADsAdABlAHgAdAAtAGEAdQB0AG8AcwBwAGEAYwBlADoAbgBvAG4AZQAn\r\nAD4APABiAD4APABzAHAAYQBuAA0ACgBzAHQAeQBsAGUAPQAnAGYAbwBuAHQALQBzAGkAegBlADoA\r\nMQAxAC4AMABwAHQAOwBmAG8AbgB0AC0AZgBhAG0AaQBsAHkAOgAiAEMAYQBsAGkAYgByAGkAIgAs\r\nAHMAYQBuAHMALQBzAGUAcgBpAGYAOwBjAG8AbABvAHIAOgBiAGwAYQBjAGsAJwA+AEYAcgBvAG0A\r\nOgA8AHMAcABhAG4ADQAKAHMAdAB5AGwAZQA9ACcAbQBzAG8ALQB0AGEAYgAtAGMAbwB1AG4AdAA6\r\nADEAJwA+ACAAPAAvAHMAcABhAG4APgA8AC8AcwBwAGEAbgA+ADwALwBiAD4APABzAHAAYQBuACAA\r\ncwB0AHkAbABlAD0AJwBmAG8AbgB0AC0AcwBpAHoAZQA6ADEAMQAuADAAcAB0ADsADQAKAGYAbwBu\r\nAHQALQBmAGEAbQBpAGwAeQA6ACIAQwBhAGwAaQBiAHIAaQAiACwAcwBhAG4AcwAtAHMAZQByAGkA\r\nZgA7AGMAbwBsAG8AcgA6AGIAbABhAGMAawAnAD4AQQBuAG8AbgB5AG0AbwB1AHMAZQBtAGEAaQBs\r\nAA0ACgAmAGwAdAA7AG4AbwByAGUAcABsAHkAQABhAG4AbwBuAHkAbQBvAHUAcwBlAG0AYQBpAGwA\r\nLgBlAHUAJgBnAHQAOwA8AG8AOgBwAD4APAAvAG8AOgBwAD4APAAvAHMAcABhAG4APgA8AC8AcAA+\r\nAA0ACgANAAoAPABwACAAYwBsAGEAcwBzAD0ATQBzAG8ATgBvAHIAbQBhAGwAIABzAHQAeQBsAGUA\r\nPQAnAG0AYQByAGcAaQBuAC0AbABlAGYAdAA6ADIAMgA1AC4AMABwAHQAOwB0AGUAeAB0AC0AaQBu\r\nAGQAZQBuAHQAOgAtADIAMgA1AC4AMABwAHQAOwB0AGEAYgAtAHMAdABvAHAAcwA6AA0ACgAyADIA\r\nNQAuADAAcAB0ADsAbQBzAG8ALQBsAGEAeQBvAHUAdAAtAGcAcgBpAGQALQBhAGwAaQBnAG4AOgBu\r\nAG8AbgBlADsAdABlAHgAdAAtAGEAdQB0AG8AcwBwAGEAYwBlADoAbgBvAG4AZQAnAD4APABiAD4A\r\nPABzAHAAYQBuAA0ACgBzAHQAeQBsAGUAPQAnAGYAbwBuAHQALQBzAGkAegBlADoAMQAxAC4AMABw\r\nAHQAOwBmAG8AbgB0AC0AZgBhAG0AaQBsAHkAOgAiAEMAYQBsAGkAYgByAGkAIgAsAHMAYQBuAHMA\r\nLQBzAGUAcgBpAGYAOwBjAG8AbABvAHIAOgBiAGwAYQBjAGsAJwA+AFMAZQBuAHQAOgA8AHMAcABh\r\nAG4ADQAKAHMAdAB5AGwAZQA9ACcAbQBzAG8ALQB0AGEAYgAtAGMAbwB1AG4AdAA6ADEAJwA+ACAA\r\nPAAvAHMAcABhAG4APgA8AC8AcwBwAGEAbgA+ADwALwBiAD4APABzAHAAYQBuACAAcwB0AHkAbABl\r\nAD0AJwBmAG8AbgB0AC0AcwBpAHoAZQA6ADEAMQAuADAAcAB0ADsADQAKAGYAbwBuAHQALQBmAGEA\r\nbQBpAGwAeQA6ACIAQwBhAGwAaQBiAHIAaQAiACwAcwBhAG4AcwAtAHMAZQByAGkAZgA7AGMAbwBs\r\nAG8AcgA6AGIAbABhAGMAawAnAD4AUwBhAHQAdQByAGQAYQB5ACwAIABNAGEAeQAgADIANAAsACAA\r\nMgAwADIANQAgADQAOgAxADMAIABQAE0APABvADoAcAA+ADwALwBvADoAcAA+ADwALwBzAHAAYQBu\r\nAD4APAAvAHAAPgANAAoADQAKADwAcAAgAGMAbABhAHMAcwA9AE0AcwBvAE4AbwByAG0AYQBsACAA\r\ncwB0AHkAbABlAD0AJwBtAGEAcgBnAGkAbgAtAGwAZQBmAHQAOgAyADIANQAuADAAcAB0ADsAdABl\r\nAHgAdAAtAGkAbgBkAGUAbgB0ADoALQAyADIANQAuADAAcAB0ADsAdABhAGIALQBzAHQAbwBwAHMA\r\nOgANAAoAMgAyADUALgAwAHAAdAA7AG0AcwBvAC0AbABhAHkAbwB1AHQALQBnAHIAaQBkAC0AYQBs\r\nAGkAZwBuADoAbgBvAG4AZQA7AHQAZQB4AHQALQBhAHUAdABvAHMAcABhAGMAZQA6AG4AbwBuAGUA\r\nJwA+ADwAYgA+ADwAcwBwAGEAbgANAAoAcwB0AHkAbABlAD0AJwBmAG8AbgB0AC0AcwBpAHoAZQA6\r\nADEAMQAuADAAcAB0ADsAZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAIgBDAGEAbABpAGIAcgBpACIA\r\nLABzAGEAbgBzAC0AcwBlAHIAaQBmADsAYwBvAGwAbwByADoAYgBsAGEAYwBrACcAPgBUAG8AOgA8\r\nAHMAcABhAG4ADQAKAHMAdAB5AGwAZQA9ACcAbQBzAG8ALQB0AGEAYgAtAGMAbwB1AG4AdAA6ADEA\r\nJwA+ACAAPAAvAHMAcABhAG4APgA8AC8AcwBwAGEAbgA+ADwALwBiAD4APABzAHAAYQBuACAAcwB0\r\nAHkAbABlAD0AJwBmAG8AbgB0AC0AcwBpAHoAZQA6ADEAMQAuADAAcAB0ADsADQAKAGYAbwBuAHQA\r\nLQBmAGEAbQBpAGwAeQA6ACIAQwBhAGwAaQBiAHIAaQAiACwAcwBhAG4AcwAtAHMAZQByAGkAZgA7\r\nAGMAbwBsAG8AcgA6AGIAbABhAGMAawAnAD4AcwBhAG0AcABsAGUAbABlAGYAYQB5AEAAZwBtAGEA\r\naQBsAC4AYwBvAG0APABvADoAcAA+ADwALwBvADoAcAA+ADwALwBzAHAAYQBuAD4APAAvAHAAPgAN\r\nAAoADQAKADwAcAAgAGMAbABhAHMAcwA9AE0AcwBvAE4AbwByAG0AYQBsACAAcwB0AHkAbABlAD0A\r\nJwBtAGEAcgBnAGkAbgAtAGwAZQBmAHQAOgAyADIANQAuADAAcAB0ADsAdABlAHgAdAAtAGkAbgBk\r\nAGUAbgB0ADoALQAyADIANQAuADAAcAB0ADsAdABhAGIALQBzAHQAbwBwAHMAOgANAAoAMgAyADUA\r\nLgAwAHAAdAA7AG0AcwBvAC0AbABhAHkAbwB1AHQALQBnAHIAaQBkAC0AYQBsAGkAZwBuADoAbgBv\r\nAG4AZQA7AHQAZQB4AHQALQBhAHUAdABvAHMAcABhAGMAZQA6AG4AbwBuAGUAJwA+ADwAYgA+ADwA\r\ncwBwAGEAbgANAAoAcwB0AHkAbABlAD0AJwBmAG8AbgB0AC0AcwBpAHoAZQA6ADEAMQAuADAAcAB0\r\nADsAZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAIgBDAGEAbABpAGIAcgBpACIALABzAGEAbgBzAC0A\r\ncwBlAHIAaQBmADsAYwBvAGwAbwByADoAYgBsAGEAYwBrACcAPgBTAHUAYgBqAGUAYwB0ADoAPABz\r\nAHAAYQBuAA0ACgBzAHQAeQBsAGUAPQAnAG0AcwBvAC0AdABhAGIALQBjAG8AdQBuAHQAOgAxACcA\r\nPgAgADwALwBzAHAAYQBuAD4APAAvAHMAcABhAG4APgA8AC8AYgA+ADwAcwBwAGEAbgAgAHMAdAB5\r\nAGwAZQA9ACcAZgBvAG4AdAAtAHMAaQB6AGUAOgAxADEALgAwAHAAdAA7AA0ACgBmAG8AbgB0AC0A\r\nZgBhAG0AaQBsAHkAOgAiAEMAYQBsAGkAYgByAGkAIgAsAHMAYQBuAHMALQBzAGUAcgBpAGYAOwBj\r\nAG8AbABvAHIAOgBiAGwAYQBjAGsAJwA+AFcAZQBsAGMAbwBtAGUAIAB0AG8AIAB5AG8AdQByACAA\r\ncwBhAG0AcABsAGUAIABlAG0AYQBpAGwALgAgADwALwBzAHAAYQBuAD4APABzAHAAYQBuAA0ACgBz\r\nAHQAeQBsAGUAPQAnAGYAbwBuAHQALQBzAGkAegBlADoAMQAxAC4AMABwAHQAOwBmAG8AbgB0AC0A\r\nZgBhAG0AaQBsAHkAOgAiAFMAZQBnAG8AZQAgAFUASQAgAEUAbQBvAGoAaQAiACwAcwBhAG4AcwAt\r\nAHMAZQByAGkAZgA7AG0AcwBvAC0AYgBpAGQAaQAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6AA0A\r\nCgAiAFMAZQBnAG8AZQAgAFUASQAgAEUAbQBvAGoAaQAiADsAYwBvAGwAbwByADoAYgBsAGEAYwBr\r\nACcAPgAmACMAMQAyADgANQA3ADkAOwA8AC8AcwBwAGEAbgA+ADwAcwBwAGEAbgAgAHMAdAB5AGwA\r\nZQA9ACcAZgBvAG4AdAAtAHMAaQB6AGUAOgAxADEALgAwAHAAdAA7AA0ACgBmAG8AbgB0AC0AZgBh\r\nAG0AaQBsAHkAOgAiAEMAYQBsAGkAYgByAGkAIgAsAHMAYQBuAHMALQBzAGUAcgBpAGYAOwBjAG8A\r\nbABvAHIAOgBiAGwAYQBjAGsAJwA+ADwAbwA6AHAAPgA8AC8AbwA6AHAAPgA8AC8AcwBwAGEAbgA+\r\nADwALwBwAD4ADQAKAA0ACgA8AHAAIABjAGwAYQBzAHMAPQBNAHMAbwBOAG8AcgBtAGEAbAA+ADwA\r\nbwA6AHAAPgAmAG4AYgBzAHAAOwA8AC8AbwA6AHAAPgA8AC8AcAA+AA0ACgANAAoAPABwAD4APABz\r\nAHAAYQBuACAAcwB0AHkAbABlAD0AJwBjAG8AbABvAHIAOgAjAEMAMAAzADkAMgBCACcAPgBQAG8A\r\ndwBlAHIAZQBkACAAYgB5ACAAPABzAHQAcgBvAG4AZwA+ADwAcwBwAGEAbgAgAHMAdAB5AGwAZQA9\r\nACcAZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoADQAKACIAQQBwAHQAbwBzACIALABzAGEAbgBzAC0A\r\ncwBlAHIAaQBmADsAbQBzAG8ALQBiAGkAZABpAC0AZgBvAG4AdAAtAGYAYQBtAGkAbAB5ADoAQQBw\r\nAHQAbwBzACcAPgBBAG4AbwBuAHkAbQBvAHUAcwBlAG0AYQBpAGwAPAAvAHMAcABhAG4APgA8AC8A\r\ncwB0AHIAbwBuAGcAPgA8AC8AcwBwAGEAbgA+ADwALwBwAD4ADQAKAA0ACgA8AHAAPgBZAG8AdQAn\r\nAHYAZQAgAGIAZQBlAG4AIABzAGUAbgB0ACAAYQAgAHMAYQBtAHAAbABlACAAZQBtAGEAaQBsACEA\r\nIAA8AHMAcABhAG4AIABzAHQAeQBsAGUAPQAnAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6ACIAUwBl\r\nAGcAbwBlACAAVQBJACAARQBtAG8AagBpACIALABzAGEAbgBzAC0AcwBlAHIAaQBmADsADQAKAG0A\r\ncwBvAC0AYgBpAGQAaQAtAGYAbwBuAHQALQBmAGEAbQBpAGwAeQA6ACIAUwBlAGcAbwBlACAAVQBJ\r\nACAARQBtAG8AagBpACIAJwA+ACYAIwAxADIANwA4ADgAMQA7ADwALwBzAHAAYQBuAD4APAAvAHAA\r\nPgANAAoADQAKADwALwBkAGkAdgA+AA0ACgANAAoAPAAvAGIAbwBkAHkAPgANAAoADQAKADwALwBo\r\nAHQAbQBsAD4ADQAKAA==\r\n\r\n------=_NextPart_01DBCCE0.CD23D5E0\r\nContent-Location: file:///C:/790962C5/sample-anonymouse_files/themedata.thmx\r\nContent-Transfer-Encoding: base64\r\nContent-Type: application/vnd.ms-officetheme\r\n\r\nUEsDBBQABgAIAAAAIQDp3g+//wAAABwCAAATAAAAW0NvbnRlbnRfVHlwZXNdLnhtbKyRy07DMBBF\r\n90j8g+UtSpyyQAgl6YLHjseifMDImSQWydiyp1X790zSVEKoIBZsLNkz954743K9Hwe1w5icp0qv\r\n8kIrJOsbR12l3zdP2a1WiYEaGDxhpQ+Y9Lq+vCg3h4BJiZpSpXvmcGdMsj2OkHIfkKTS+jgCyzV2\r\nJoD9gA7NdVHcGOuJkTjjyUPX5QO2sB1YPe7l+Zgk4pC0uj82TqxKQwiDs8CS1Oyo+UbJFkIuyrkn\r\n9S6kK4mhzVnCVPkZsOheZTXRNajeIPILjBLDsAyJX89nIBkt5r87nons29ZZbLzdjrKOfDZezE7B\r\n/xRg9T/oE9PMf1t/AgAA//8DAFBLAwQUAAYACAAAACEApdan58AAAAA2AQAACwAAAF9yZWxzLy5y\r\nZWxzhI/PasMwDIfvhb2D0X1R0sMYJXYvpZBDL6N9AOEof2giG9sb69tPxwYKuwiEpO/3qT3+rov5\r\n4ZTnIBaaqgbD4kM/y2jhdj2/f4LJhaSnJQhbeHCGo3vbtV+8UNGjPM0xG6VItjCVEg+I2U+8Uq5C\r\nZNHJENJKRds0YiR/p5FxX9cfmJ4Z4DZM0/UWUtc3YK6PqMn/s8MwzJ5PwX+vLOVFBG43lExp5GKh\r\nqC/jU72QqGWq1B7Qtbj51v0BAAD//wMAUEsDBBQABgAIAAAAIQBreZYWgwAAAIoAAAAcAAAAdGhl\r\nbWUvdGhlbWUvdGhlbWVNYW5hZ2VyLnhtbAzMTQrDIBBA4X2hd5DZN2O7KEVissuuu/YAQ5waQceg\r\n0p/b1+XjgzfO3xTVm0sNWSycBw2KZc0uiLfwfCynG6jaSBzFLGzhxxXm6XgYybSNE99JyHNRfSPV\r\nkIWttd0g1rUr1SHvLN1euSRqPYtHV+jT9yniResrJgoCOP0BAAD//wMAUEsDBBQABgAIAAAAIQBb\r\nlZ6x9AcAABMiAAAWAAAAdGhlbWUvdGhlbWUvdGhlbWUxLnhtbOxaS48buRG+B8h/aPRdVnfrPbC8\r\n0NOz9ow9sGQHe6QkSk0Puyk0qZkRFgsE3lMuAQJsgr0skFsOi0UWyAJZ5JIfY8BGsvkRKZKtFilR\r\nngcMxAhm5tJNfVX8WFWsqmb3w8+uEupd4IwTlrb98EHgezidshlJF23/5XhYavoeFyidIcpS3PbX\r\nmPufPfr1rx6iIxHjBHsgn/Ij1PZjIZZH5TKfwjDiD9gSp/DbnGUJEnCbLcqzDF2C3oSWoyColxNE\r\nUt9LUQJqn8/nZIq9sVTpP9ooH1C4TQWXA1OajaRqbEko7Ow8lAi+5j2aeReItn2YZ8Yux/hK+B5F\r\nXMAPbT9Qf3750cMyOsqFqDgga8gN1V8ulwvMziM1Z7aYFJMGg6hZDQv9CkDFPm7QlP+FPgVA0yms\r\nVHMxdYa1etCMcqwB0pcO3a1GWLHxhv7KHuewVe9GVUu/Amn91T18MGwN+jULr0AaX9vDd4Ko26pY\r\neAXS+PoevjroNKKBhVegmJL0fB9dbzSb9RxdQOaMHjvhrXo9aPRz+BYF0VBEl5xizlJxKNYS9Jpl\r\nQwBIIEWCpJ5YL/EcTSGKO0vBuNcnfEnR2veWKGUchoMoDCH0qkFU/CuLoyOMDGnJC5jwvSHJx+PT\r\njCxF238CWn0D8u7nn9+++entm7+//frrt2/+6p2QRSy0KkvuGKULU+6Xv/zhP9/91vv33/78yzd/\r\ndOO5iX//w+/e/+OfH1IPW21rind/+vH9Tz+++/b3//r+G4f2ToYmJnxMEsy9Z/jSe8ESWKAyhc0f\r\nT7LbSYxjREyJTrrgKEVyFof+gYgt9LM1osiB62Lbjq8ySDUu4OPVa4vwKM5Wgjg0Po0TC3jKGO2y\r\nzGmFp3Iuw8zjVbpwT56tTNwLhC5cc/dQanl5sFpCjiUulb0YWzTPKEoFWuAUC0/+xs4xdqzuC0Is\r\nu56SacY4mwvvC+J1EXGaZEwmVjRthY5JAn5ZuwiCvy3bnL7yuoy6Vt3HFzYS9gaiDvJjTC0zPkYr\r\ngRKXyjFKqGnwEyRiF8nROpuauAEX4OkFpswbzDDnLpnnGazXcPpTBNnN6fZTuk5sZCbIuUvnCWLM\r\nRPbZeS9GydKFHZE0NrGf83MIUeSdMeGCnzJ7h8h78ANKD7r7FcGWu6/PBi8hy5mUtgEif1llDl8+\r\nxsyK39GazhF2pZpOllgptpMRZ3R0VwsrtE8wpugSzTD2Xn7uYNBlS8vmW9JPYsgqx9gVWE+QHavy\r\nPsUceiXZ3OznyRPCrZAd4QU7wOd0vZN41ihNUHZI8zPwumnzwSSDzeig8JxOz03gMwI9IMSL0yjP\r\nOegwgvug1rMYWQVM3nN3vK4zy3832WOwL19bNG6wL0EG31oGErsp80HbjBG1JtgGzBgR78SVbkHE\r\ncv9WRBZXJbZyys3tTbt1A3RHVtOTkPSaDuh/0/k4AvHj9DxuxVbCumW3cyihHO/0OIdwu51Nj2Uz\r\n8uk3Nn20Ss8w1JL9rHXf19z3Nf7/fV9zaD/fdzOHeo77bsaHLuO+m8kPWD5ON7NtYKC3kYcM+rBH\r\nHf0kB09+5oTSkVhTfMLV4Q+HZ5rZEAalnDr1xMVJ4DKGS1nmYAILt8iQkvEyJn5DRDyK0RJOiEJf\r\nKlnwXPWCe0vG4eBIDTt1SzxdJadspg881QlToCsrR2I7HtTg6EmPw2GV0Oh6Ix+U/NSpKvBVbBfq\r\nsHVDQMrehoQxmU2i4iDR2AxeQ0KenX0cFi0Hi6ZUv3HVnimAWuEVeOj24FG97deqkhCclPMpNOgz\r\n6Sft6o13lTM/pqcPGdOKADhc1CuBo/nC0y3J9eDy5Op0qN3A0xYJ5RQdVjYJZRnV4PEYHoXz6JSj\r\nN6FxW1+3ti616ElTqPkgvrc0Gs0Psbirr0FuNzfQ1MwUNPUuYY9HsOl8b4qWbX8OJ8dwmSwheLh8\r\n8EJ0Aa9fpiLTO/4uqWWZcdFHPNYWV1lH+ychAmceJUnbl+sv/EBTlUQ0uRZs3U+VXCQ33KdGDrxu\r\nexnP53gqTL8bI9LS+hZSvE4Wzl+V+N3BUpKtwN2jeHbpTegqe4EgxGqNUHp3Rji8QAi1q2cE3ogV\r\nmWwbfzuVKc/+5ispFUN6HNFljPKSYmZzDVcFpaCj7gobGHf5msGghknySjhZyAprGtUqp0Xt0hwO\r\nlt3rhaTljKy5LZpWWpFl053GrBk2dWDHlner8garjYkhqZklXufu3Zzb2iS7nUahKBNg8MJ+d6v9\r\nBrXtZBY1yXg/D8uknY/axWOzwGuo3aRKGGm/vlG7Y7eiSDing8E7lX6Q241aGJpvGktlafXq3Hy7\r\nzSavIXn0oc1dUf2+m6ZwJ6OSL88y5dsJm63zS8p1otE+l02pRNL0BZ57ZHbV9iNX56jfuYZ5N6DQ\r\nUkwWr0LQ2e3ZgjleiuoNWwjrAC+CSm9KW7iQUDND710Iq3NFF21xtaEse3XAKxNyvWowbW4puNq3\r\nIpyRZwh625Hq7HTuBdpXIs8vcOWtMtL2vwxqnWovqvVKQbM2KFUr1aDUrHUqpU6tVgkHtTDod6Ov\r\ngJ6Ik7Cmv30Ywqsgus6/gFDje19BJJu3XQ+mLCkz9ZVDWXlffQURRoe/ggBHAq1oEFajTtQr9fph\r\nvVSN+vVSs1HplHpRvR91oGjXh52vfO9CgcNuvz8c1qJSvQe4atCplTrdSq9Ubw660TAcVPsBgPPy\r\ncwVPMWCzjS3gUvF69F8AAAD//wMAUEsDBBQABgAIAAAAIQAN0ZCftgAAABsBAAAnAAAAdGhlbWUv\r\ndGhlbWUvX3JlbHMvdGhlbWVNYW5hZ2VyLnhtbC5yZWxzhI9NCsIwFIT3gncIb2/TuhCRJt2I0K3U\r\nA4TkNQ02PyRR7O0NriwILodhvplpu5edyRNjMt4xaKoaCDrplXGawW247I5AUhZOidk7ZLBggo5v\r\nN+0VZ5FLKE0mJFIoLjGYcg4nSpOc0IpU+YCuOKOPVuQio6ZByLvQSPd1faDxmwF8xSS9YhB71QAZ\r\nllCa/7P9OBqJZy8fFl3+UUFz2YUFKKLGzOAjm6pMBMpburrE3wAAAP//AwBQSwECLQAUAAYACAAA\r\nACEA6d4Pv/8AAAAcAgAAEwAAAAAAAAAAAAAAAAAAAAAAW0NvbnRlbnRfVHlwZXNdLnhtbFBLAQIt\r\nABQABgAIAAAAIQCl1qfnwAAAADYBAAALAAAAAAAAAAAAAAAAADABAABfcmVscy8ucmVsc1BLAQIt\r\nABQABgAIAAAAIQBreZYWgwAAAIoAAAAcAAAAAAAAAAAAAAAAABkCAAB0aGVtZS90aGVtZS90aGVt\r\nZU1hbmFnZXIueG1sUEsBAi0AFAAGAAgAAAAhAFuVnrH0BwAAEyIAABYAAAAAAAAAAAAAAAAA1gIA\r\nAHRoZW1lL3RoZW1lL3RoZW1lMS54bWxQSwECLQAUAAYACAAAACEADdGQn7YAAAAbAQAAJwAAAAAA\r\nAAAAAAAAAAD+CgAAdGhlbWUvdGhlbWUvX3JlbHMvdGhlbWVNYW5hZ2VyLnhtbC5yZWxzUEsFBgAA\r\nAAAFAAUAXQEAAPkLAAAAAA==\r\n\r\n------=_NextPart_01DBCCE0.CD23D5E0\r\nContent-Location: file:///C:/790962C5/sample-anonymouse_files/colorschememapping.xml\r\nContent-Transfer-Encoding: quoted-printable\r\nContent-Type: text/xml\r\n\r\n\r\n\r\n------=_NextPart_01DBCCE0.CD23D5E0\r\nContent-Location: file:///C:/790962C5/sample-anonymouse_files/filelist.xml\r\nContent-Transfer-Encoding: quoted-printable\r\nContent-Type: text/xml; charset=\"utf-8\"\r\n\r\n\r\n \r\n \r\n \r\n \r\n\r\n------=_NextPart_01DBCCE0.CD23D5E0--\r\n"
  },
  {
    "path": "tests_data/basic/pem/doc.pem",
    "content": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nmFIEWhdc7RMIKoZIzj0DAQcCAwTCrR7Da5QHqFi/CtJJ6egFb48zR9bn48epqb92\nkfLIN/sjBc6iqvjcXQM8pfhFZnf5Bhk0ZzwvuAHzCZSJgNgNtCFFbGllIEJ1cnN6\ndGVpbiA8Y29udGFjdEBlbGllLm5ldD6IgAQTEwgAHAUCWhdc7QILCQIbAwQVCAkK\nBBYCAwECF4ACHgEAFgkQtc4ql0fc7HoLGlRSRVpPUi1HUEd3nQD/bqujXolVmt7n\nGmY/kIRWEro2oUp2rXL5sAbABMC/SrIA/ihJa5nfZz8wAe7IzD76cRHW0qGpUHSM\nehJzdDXXsEhruFYEWhdc7RIIKoZIzj0DAQcCAwSF6kdXcDKXmK5UYjfoRV07yxQo\nxapjucsZcXytjdLqbPDJr+Sw7Rlz41XIM3QQzOksFdNzlNemBuXBUE/K2522AwEI\nB4htBBgTCAAJBQJaF1ztAhsMABYJELXOKpdH3Ox6CxpUUkVaT1ItR1BHl9QBAI55\n7DxLdB2WMXemGZ0U07vqGt2jSzTtUdYhqk4DkXUeAP98LFF4syoKrxD2pcArpKzI\nOwBiyuQgLZqQr2mtIPFWCw==\n=qq2a\n-----END PGP PUBLIC KEY BLOCK-----\n"
  },
  {
    "path": "tests_data/basic/pem/doc.pub",
    "content": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nmFIEWhdc7RMIKoZIzj0DAQcCAwTCrR7Da5QHqFi/CtJJ6egFb48zR9bn48epqb92\nkfLIN/sjBc6iqvjcXQM8pfhFZnf5Bhk0ZzwvuAHzCZSJgNgNtCFFbGllIEJ1cnN6\ndGVpbiA8Y29udGFjdEBlbGllLm5ldD6IgAQTEwgAHAUCWhdc7QILCQIbAwQVCAkK\nBBYCAwECF4ACHgEAFgkQtc4ql0fc7HoLGlRSRVpPUi1HUEd3nQD/bqujXolVmt7n\nGmY/kIRWEro2oUp2rXL5sAbABMC/SrIA/ihJa5nfZz8wAe7IzD76cRHW0qGpUHSM\nehJzdDXXsEhruFYEWhdc7RIIKoZIzj0DAQcCAwSF6kdXcDKXmK5UYjfoRV07yxQo\nxapjucsZcXytjdLqbPDJr+Sw7Rlz41XIM3QQzOksFdNzlNemBuXBUE/K2522AwEI\nB4htBBgTCAAJBQJaF1ztAhsMABYJELXOKpdH3Ox6CxpUUkVaT1ItR1BHl9QBAI55\n7DxLdB2WMXemGZ0U07vqGt2jSzTtUdYhqk4DkXUeAP98LFF4syoKrxD2pcArpKzI\nOwBiyuQgLZqQr2mtIPFWCw==\n=qq2a\n-----END PGP PUBLIC KEY BLOCK-----\n"
  },
  {
    "path": "tests_data/basic/python/code.py",
    "content": "def print_primes(max_n: int) -> None:\n    for i in range(2, max_n + 1):\n        if is_prime(i):\n            print(i)\n\n\ndef is_prime(n: int) -> bool:\n    for i in range(2, n // 2 + 1):\n        if n % i == 0:\n            return False\n    return True\n"
  },
  {
    "path": "tests_data/basic/rtf/doc.rtf",
    "content": "{\\rtf1\\ansi\\ansicpg1252\\uc0\\stshfdbch0\\stshfloch0\\stshfhich0\\stshfbi0\\deff0\\adeff0{\\fonttbl{\\f0\\froman\\fcharset0\\fprq2{\\*\\panose 02020603050405020304}Times New Roman;}{\\f1\\froman\\fcharset2\\fprq2{\\*\\panose 05050102010706020507}Symbol;}{\\f2\\fswiss\\fcharset0\\fprq2{\\*\\panose 020b0604020202020204}Arial;}{\\f3\\fnil\\fcharset0 Roboto;}}{\r\n\\colortbl;\\red109\\green158\\blue235;\\red0\\green0\\blue0;\\red106\\green168\\blue79;\\red230\\green145\\blue56;\\red102\\green102\\blue102;}{\\stylesheet{\\s0\\snext0\\sqformat\\spriority0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 Normal;}{\\s1\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb400\\sa120\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs40\\ltrch\\b0\\i0\\fs40\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 heading 1;}{\\s2\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb360\\sa120\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs32\\ltrch\\b0\\i0\\fs32\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 heading 2;}{\\s3\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb320\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs28\\ltrch\\b0\\i0\\fs28\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf4 heading 3;}{\\s4\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb280\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs24\\ltrch\\b0\\i0\\fs24\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 heading 4;}{\\s5\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb240\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 heading 5;}{\\s6\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb240\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai\\af3\\afs22\\ltrch\\b0\\i\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 heading 6;}{\\*\\cs10\\additive\\ssemihidden\\spriority0 Default Paragraph Font;\r\n}{\\*\\ts11\\tsrowd\\snext11\\ssemihidden\\spriority0\\aspalpha\\aspnum\\adjustright\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\tsvertalt\\tsbrdrl\\tsbrdrr\\tsbrdrt\\tsbrdrb\\tsbrdrdgr\\tsbrdrdgl\\tsbrdrh\\tsbrdrv\\trpaddl108\\trpaddfl3\\trwWidthB0\\trftsWidthB3\\trpaddt0\\trpaddft3\\trpaddb0\r\n\\trpaddfb3\\trpaddr108\\trpaddfr3 Normal Table;}{\\s15\\sbasedon0\\snext15\\styrsid15694742\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb0\\sa60\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\qc\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs52\\ltrch\\b0\\i0\\fs52\\loch\\af3\r\n\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 Title;}{\\s16\\sbasedon0\\snext16\\styrsid15694742\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb0\\sa320\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\qc\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs30\\ltrch\\b0\\i0\\fs30\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 Subtitle;}}{\\*\\rsidtbl\\rsid10976062\\rsid13249109}{\\*\\generator Aspose.Words for Java 23.4.0;}{\\info\\version1\\edmins0\\nofpages1\\nofwords0\\nofchars0\\nofcharsws0}\\paperw12240\\paperh15840\\margl1440\\margr1440\\margt1440\\margb1440\\gutter0\r\n{\\mmathPr\\mbrkBin0\\mbrkBinSub0\\mdefJc1\\mdispDef1\\minterSp0\\mintLim0\\mintraSp0\\mlMargin0\\mmathFont0\\mnaryLim1\\mpostSp0\\mpreSp0\\mrMargin0\\msmallFrac0\\mwrapIndent1440\\mwrapRight0}\\deflang1033\\deflangfe2052\\adeflang1025\\jexpand\\showxmlerrors1\\validatexml1{\r\n\\*\\wgrffmtfilter 013f}\\viewkind1\\viewscale100\\fet0\\ftnbj\\aenddoc\\ftnrstcont\\aftnrstcont\\ftnnar\\aftnnrlc\\widowctrl\\nospaceforul\\nolnhtadjtbl\\alntblind\\lyttblrtgr\\dntblnsbdb\\noxlattoyen\\wrppunct\\nobrkwrptbl\\expshrtn\\snaptogridincell\\asianbrkrule\\htmautsp\\noultrlspc\r\n\\useltbaln\\splytwnine\\ftnlytwnine\\lytcalctblwd\\allowfieldendsel\\lnbrkrule\\nouicompat\\nofeaturethrottle1\\utinl\\formshade\\nojkernpunct\\dghspace180\\dgvspace180\\dghorigin1800\\dgvorigin1440\\dghshow1\\dgvshow1\\dgmargin\\pgbrdrhead\\pgbrdrfoot\\rsidroot10976062\\sectd\\sectlinegrid360\\pgwsxn12240\\pghsxn15840\\marglsxn1440\\margrsxn1440\\margtsxn1440\\margbsxn1440\\guttersxn0\\headery720\\footery720\\colsx720\\ltrsect\\pard\\plain\\itap0\\s1\\keep\\keepn\\ilvl0\\fi0\\sb400\\sa120\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\r\n\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs40\\ltrch\\b0\\i0\\fs40\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1{\\*\\bkmkstart h.z8khh864y3wu}{\\*\\bkmkend h.z8khh864y3wu}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs40\\ltrch\\b0\\i0\\fs40\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 Hello}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs40\\ltrch\\b0\\i0\\fs40\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs40\\ltrch\\b0\\i0\\fs40\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 world}{\\rtlch\\ab0\\ai0\\af3\\afs40\\ltrch\\b0\\i0\\fs40\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf1\\par}\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\r\n\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf2\\par}\r\n\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\r\n\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 hello}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2  }{\r\n\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 world}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\r\n\\hich\\f3\\strike0\\ulnone\\cf2 !}{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf2\\par}\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\\widctlpar\r\n\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf2\\par}\\pard\\plain\\itap0\\s2\r\n\\keep\\keepn\\ilvl0\\fi0\\sb360\\sa120\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs32\\ltrch\\b0\\i0\\fs32\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3{\r\n\\*\\bkmkstart h.oe6g4mjyfx6j}{\\*\\bkmkend h.oe6g4mjyfx6j}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 Yes}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 , }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 another}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 hello}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs32\\ltrch\\b0\\i0\\fs32\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 world}{\\rtlch\\ab0\\ai0\\af3\\afs32\\ltrch\\b0\\i0\\fs32\\loch\\af3\\dbch\\af3\r\n\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf3\\par}\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf2\\par}\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\r\n\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\r\n\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 hello}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 !}{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\r\n\\ulnone\\cf2\\par}{\r\n\\*\\latentstyles\\lsdstimax267\\lsdlockeddef0\\lsdsemihiddendef0\\lsdunhideuseddef0\\lsdqformatdef0\\lsdprioritydef0{\\lsdlockedexcept\\lsdqformat1 Normal;\\lsdqformat1 heading 1;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 2;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 3;\r\n\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 4;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 5;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 6;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 7;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 8;\r\n\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 9;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 caption;\\lsdqformat1 Title;\\lsdqformat1 Subtitle;\\lsdqformat1 Strong;\\lsdqformat1 Emphasis;\\lsdsemihidden1\\lsdpriority99 Placeholder Text;\\lsdqformat1\\lsdpriority1 No Spacing;\r\n\\lsdpriority60 Light Shading;\\lsdpriority61 Light List;\\lsdpriority62 Light Grid;\\lsdpriority63 Medium Shading 1;\\lsdpriority64 Medium Shading 2;\\lsdpriority65 Medium List 1;\\lsdpriority66 Medium List 2;\\lsdpriority67 Medium Grid 1;\\lsdpriority68 Medium Grid 2;\r\n\\lsdpriority69 Medium Grid 3;\\lsdpriority70 Dark List;\\lsdpriority71 Colorful Shading;\\lsdpriority72 Colorful List;\\lsdpriority73 Colorful Grid;\\lsdpriority60 Light Shading Accent 1;\\lsdpriority61 Light List Accent 1;\\lsdpriority62 Light Grid Accent 1;\\lsdpriority63 Medium Shading 1 Accent 1;\r\n\\lsdpriority64 Medium Shading 2 Accent 1;\\lsdpriority65 Medium List 1 Accent 1;\\lsdsemihidden1\\lsdpriority99 Revision;\\lsdqformat1\\lsdpriority34 List Paragraph;\\lsdqformat1\\lsdpriority29 Quote;\\lsdqformat1\\lsdpriority30 Intense Quote;\\lsdpriority66 Medium List 2 Accent 1;\r\n\\lsdpriority67 Medium Grid 1 Accent 1;\\lsdpriority68 Medium Grid 2 Accent 1;\\lsdpriority69 Medium Grid 3 Accent 1;\\lsdpriority70 Dark List Accent 1;\\lsdpriority71 Colorful Shading Accent 1;\\lsdpriority72 Colorful List Accent 1;\\lsdpriority73 Colorful Grid Accent 1;\r\n\\lsdpriority60 Light Shading Accent 2;\\lsdpriority61 Light List Accent 2;\\lsdpriority62 Light Grid Accent 2;\\lsdpriority63 Medium Shading 1 Accent 2;\\lsdpriority64 Medium Shading 2 Accent 2;\\lsdpriority65 Medium List 1 Accent 2;\\lsdpriority66 Medium List 2 Accent 2;\r\n\\lsdpriority67 Medium Grid 1 Accent 2;\\lsdpriority68 Medium Grid 2 Accent 2;\\lsdpriority69 Medium Grid 3 Accent 2;\\lsdpriority70 Dark List Accent 2;\\lsdpriority71 Colorful Shading Accent 2;\\lsdpriority72 Colorful List Accent 2;\\lsdpriority73 Colorful Grid Accent 2;\r\n\\lsdpriority60 Light Shading Accent 3;\\lsdpriority61 Light List Accent 3;\\lsdpriority62 Light Grid Accent 3;\\lsdpriority63 Medium Shading 1 Accent 3;\\lsdpriority64 Medium Shading 2 Accent 3;\\lsdpriority65 Medium List 1 Accent 3;\\lsdpriority66 Medium List 2 Accent 3;\r\n\\lsdpriority67 Medium Grid 1 Accent 3;\\lsdpriority68 Medium Grid 2 Accent 3;\\lsdpriority69 Medium Grid 3 Accent 3;\\lsdpriority70 Dark List Accent 3;\\lsdpriority71 Colorful Shading Accent 3;\\lsdpriority72 Colorful List Accent 3;\\lsdpriority73 Colorful Grid Accent 3;\r\n\\lsdpriority60 Light Shading Accent 4;\\lsdpriority61 Light List Accent 4;\\lsdpriority62 Light Grid Accent 4;\\lsdpriority63 Medium Shading 1 Accent 4;\\lsdpriority64 Medium Shading 2 Accent 4;\\lsdpriority65 Medium List 1 Accent 4;\\lsdpriority66 Medium List 2 Accent 4;\r\n\\lsdpriority67 Medium Grid 1 Accent 4;\\lsdpriority68 Medium Grid 2 Accent 4;\\lsdpriority69 Medium Grid 3 Accent 4;\\lsdpriority70 Dark List Accent 4;\\lsdpriority71 Colorful Shading Accent 4;\\lsdpriority72 Colorful List Accent 4;\\lsdpriority73 Colorful Grid Accent 4;\r\n\\lsdpriority60 Light Shading Accent 5;\\lsdpriority61 Light List Accent 5;\\lsdpriority62 Light Grid Accent 5;\\lsdpriority63 Medium Shading 1 Accent 5;\\lsdpriority64 Medium Shading 2 Accent 5;\\lsdpriority65 Medium List 1 Accent 5;\\lsdpriority66 Medium List 2 Accent 5;\r\n\\lsdpriority67 Medium Grid 1 Accent 5;\\lsdpriority68 Medium Grid 2 Accent 5;\\lsdpriority69 Medium Grid 3 Accent 5;\\lsdpriority70 Dark List Accent 5;\\lsdpriority71 Colorful Shading Accent 5;\\lsdpriority72 Colorful List Accent 5;\\lsdpriority73 Colorful Grid Accent 5;\r\n\\lsdpriority60 Light Shading Accent 6;\\lsdpriority61 Light List Accent 6;\\lsdpriority62 Light Grid Accent 6;\\lsdpriority63 Medium Shading 1 Accent 6;\\lsdpriority64 Medium Shading 2 Accent 6;\\lsdpriority65 Medium List 1 Accent 6;\\lsdpriority66 Medium List 2 Accent 6;\r\n\\lsdpriority67 Medium Grid 1 Accent 6;\\lsdpriority68 Medium Grid 2 Accent 6;\\lsdpriority69 Medium Grid 3 Accent 6;\\lsdpriority70 Dark List Accent 6;\\lsdpriority71 Colorful Shading Accent 6;\\lsdpriority72 Colorful List Accent 6;\\lsdpriority73 Colorful Grid Accent 6;\r\n\\lsdqformat1\\lsdpriority19 Subtle Emphasis;\\lsdqformat1\\lsdpriority21 Intense Emphasis;\\lsdqformat1\\lsdpriority31 Subtle Reference;\\lsdqformat1\\lsdpriority32 Intense Reference;\\lsdqformat1\\lsdpriority33 Book Title;\\lsdsemihidden1\\lsdunhideused1\\lsdpriority37 Bibliography;\r\n\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1\\lsdpriority39 TOC Heading;}}}"
  },
  {
    "path": "tests_data/basic/rtf/magika_test.rtf",
    "content": "{\\rtf1\\ansi\\ansicpg1252\\uc0\\stshfdbch0\\stshfloch0\\stshfhich0\\stshfbi0\\deff0\\adeff0{\\fonttbl{\\f0\\froman\\fcharset0\\fprq2{\\*\\panose 02020603050405020304}Times New Roman;}{\\f1\\froman\\fcharset2\\fprq2{\\*\\panose 05050102010706020507}Symbol;}{\\f2\\fswiss\\fcharset0\\fprq2{\\*\\panose 020b0604020202020204}Arial;}{\\f3\\fnil\\fcharset0 Roboto;}}{\r\n\\colortbl;\\red0\\green0\\blue0;\\red109\\green158\\blue235;\\red106\\green168\\blue79;\\red230\\green145\\blue56;\\red102\\green102\\blue102;}{\\stylesheet{\\s0\\snext0\\sqformat\\spriority0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 Normal;}{\\s1\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb400\\sa120\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs40\\ltrch\\b0\\i0\\fs40\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 heading 1;}{\\s2\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb360\\sa120\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs32\\ltrch\\b0\\i0\\fs32\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf3 heading 2;}{\\s3\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb320\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs28\\ltrch\\b0\\i0\\fs28\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf4 heading 3;}{\\s4\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb280\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs24\\ltrch\\b0\\i0\\fs24\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 heading 4;}{\\s5\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb240\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 heading 5;}{\\s6\\sbasedon0\\snext0\\styrsid15694742\r\n\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb240\\sa80\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai\\af3\\afs22\\ltrch\\b0\\i\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 heading 6;}{\\*\\cs10\\additive\\ssemihidden\\spriority0 Default Paragraph Font;\r\n}{\\*\\ts11\\tsrowd\\snext11\\ssemihidden\\spriority0\\aspalpha\\aspnum\\adjustright\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\tsvertalt\\tsbrdrl\\tsbrdrr\\tsbrdrt\\tsbrdrb\\tsbrdrdgr\\tsbrdrdgl\\tsbrdrh\\tsbrdrv\\trpaddl108\\trpaddfl3\\trwWidthB0\\trftsWidthB3\\trpaddt0\\trpaddft3\\trpaddb0\r\n\\trpaddfb3\\trpaddr108\\trpaddfr3 Normal Table;}{\\s15\\sbasedon0\\snext15\\styrsid15694742\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb0\\sa60\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\qc\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs52\\ltrch\\b0\\i0\\fs52\\loch\\af3\r\n\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 Title;}{\\s16\\sbasedon0\\snext16\\styrsid15694742\\sqformat\\spriority0\\keep\\keepn\\fi0\\sb0\\sa320\\aspalpha\\aspnum\\adjustright\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\qc\\faauto\\sl240\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs30\\ltrch\\b0\\i0\\fs30\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf5 Subtitle;}}{\\*\\rsidtbl\\rsid10976062\\rsid13249109}{\\*\\generator Aspose.Words for Java 23.4.0;}{\\info\\version1\\edmins0\\nofpages1\\nofwords0\\nofchars0\\nofcharsws0}\\paperw12240\\paperh15840\\margl1440\\margr1440\\margt1440\\margb1440\\gutter0\r\n{\\mmathPr\\mbrkBin0\\mbrkBinSub0\\mdefJc1\\mdispDef1\\minterSp0\\mintLim0\\mintraSp0\\mlMargin0\\mmathFont0\\mnaryLim1\\mpostSp0\\mpreSp0\\mrMargin0\\msmallFrac0\\mwrapIndent1440\\mwrapRight0}\\deflang1033\\deflangfe2052\\adeflang1025\\jexpand\\showxmlerrors1\\validatexml1{\r\n\\*\\wgrffmtfilter 013f}\\viewkind1\\viewscale100\\fet0\\ftnbj\\aenddoc\\ftnrstcont\\aftnrstcont\\ftnnar\\aftnnrlc\\widowctrl\\nospaceforul\\nolnhtadjtbl\\alntblind\\lyttblrtgr\\dntblnsbdb\\noxlattoyen\\wrppunct\\nobrkwrptbl\\expshrtn\\snaptogridincell\\asianbrkrule\\htmautsp\\noultrlspc\r\n\\useltbaln\\splytwnine\\ftnlytwnine\\lytcalctblwd\\allowfieldendsel\\lnbrkrule\\nouicompat\\nofeaturethrottle1\\utinl\\formshade\\nojkernpunct\\dghspace180\\dgvspace180\\dghorigin1800\\dgvorigin1440\\dghshow1\\dgvshow1\\dgmargin\\pgbrdrhead\\pgbrdrfoot\\rsidroot10976062\\sectd\\sectlinegrid360\\pgwsxn12240\\pghsxn15840\\marglsxn1440\\margrsxn1440\\margtsxn1440\\margbsxn1440\\guttersxn0\\headery720\\footery720\\colsx720\\ltrsect\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\r\n\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf1\\par}\\pard\r\n\\plain\\itap0\\s1\\keep\\keepn\\ilvl0\\fi0\\sb400\\sa120\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs40\\ltrch\\b0\\i0\\fs40\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2\r\n{\\*\\bkmkstart h.9ddtw4s9rgoq}{\\*\\bkmkend h.9ddtw4s9rgoq}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs40\\ltrch\\b0\\i0\\fs40\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf2 Introduction}{\\rtlch\\ab0\\ai0\\af3\\afs40\\ltrch\\b0\\i0\\fs40\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf2\\par}\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\\brdrr\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\r\n\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf1\\par}\\pard\\plain\\itap0\\s0\\ilvl0\\fi0\\sb0\\sa0\\aspalpha\\aspnum\\adjustright\\brdrt\\brdrl\\brdrb\r\n\\brdrr\\brdrbtw\\brdrbar\\widctlpar\\ltrpar\\li0\\lin0\\ri0\\rin0\\ql\\faauto\\sl276\\slmult1\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 This}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 is}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 a}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 test}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 document}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 for}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 Magika}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 , }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 yay}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 ! }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 We}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 are}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 going}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 to}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 take}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 this}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 file}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 and}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 convert}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 it}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 in}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 a}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 number}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 of}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 other}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1  }{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\r\n\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 formats}{\\rtlch\\ab0\\ai0\\af3\\alang1025\\afs22\\ltrch\\b0\\i0\\fs22\\lang1033\\langnp1033\\langfe1033\\langfenp1033\\loch\\af3\\dbch\\af3\\hich\\f3\\strike0\\ulnone\\cf1 .}{\\rtlch\\ab0\\ai0\\af3\\afs22\\ltrch\\b0\\i0\\fs22\\loch\\af3\\dbch\\af3\r\n\\hich\\f3\\insrsid10976062\\strike0\\ulnone\\cf1\\par}{\r\n\\*\\latentstyles\\lsdstimax267\\lsdlockeddef0\\lsdsemihiddendef0\\lsdunhideuseddef0\\lsdqformatdef0\\lsdprioritydef0{\\lsdlockedexcept\\lsdqformat1 Normal;\\lsdqformat1 heading 1;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 2;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 3;\r\n\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 4;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 5;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 6;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 7;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 8;\r\n\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 heading 9;\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1 caption;\\lsdqformat1 Title;\\lsdqformat1 Subtitle;\\lsdqformat1 Strong;\\lsdqformat1 Emphasis;\\lsdsemihidden1\\lsdpriority99 Placeholder Text;\\lsdqformat1\\lsdpriority1 No Spacing;\r\n\\lsdpriority60 Light Shading;\\lsdpriority61 Light List;\\lsdpriority62 Light Grid;\\lsdpriority63 Medium Shading 1;\\lsdpriority64 Medium Shading 2;\\lsdpriority65 Medium List 1;\\lsdpriority66 Medium List 2;\\lsdpriority67 Medium Grid 1;\\lsdpriority68 Medium Grid 2;\r\n\\lsdpriority69 Medium Grid 3;\\lsdpriority70 Dark List;\\lsdpriority71 Colorful Shading;\\lsdpriority72 Colorful List;\\lsdpriority73 Colorful Grid;\\lsdpriority60 Light Shading Accent 1;\\lsdpriority61 Light List Accent 1;\\lsdpriority62 Light Grid Accent 1;\\lsdpriority63 Medium Shading 1 Accent 1;\r\n\\lsdpriority64 Medium Shading 2 Accent 1;\\lsdpriority65 Medium List 1 Accent 1;\\lsdsemihidden1\\lsdpriority99 Revision;\\lsdqformat1\\lsdpriority34 List Paragraph;\\lsdqformat1\\lsdpriority29 Quote;\\lsdqformat1\\lsdpriority30 Intense Quote;\\lsdpriority66 Medium List 2 Accent 1;\r\n\\lsdpriority67 Medium Grid 1 Accent 1;\\lsdpriority68 Medium Grid 2 Accent 1;\\lsdpriority69 Medium Grid 3 Accent 1;\\lsdpriority70 Dark List Accent 1;\\lsdpriority71 Colorful Shading Accent 1;\\lsdpriority72 Colorful List Accent 1;\\lsdpriority73 Colorful Grid Accent 1;\r\n\\lsdpriority60 Light Shading Accent 2;\\lsdpriority61 Light List Accent 2;\\lsdpriority62 Light Grid Accent 2;\\lsdpriority63 Medium Shading 1 Accent 2;\\lsdpriority64 Medium Shading 2 Accent 2;\\lsdpriority65 Medium List 1 Accent 2;\\lsdpriority66 Medium List 2 Accent 2;\r\n\\lsdpriority67 Medium Grid 1 Accent 2;\\lsdpriority68 Medium Grid 2 Accent 2;\\lsdpriority69 Medium Grid 3 Accent 2;\\lsdpriority70 Dark List Accent 2;\\lsdpriority71 Colorful Shading Accent 2;\\lsdpriority72 Colorful List Accent 2;\\lsdpriority73 Colorful Grid Accent 2;\r\n\\lsdpriority60 Light Shading Accent 3;\\lsdpriority61 Light List Accent 3;\\lsdpriority62 Light Grid Accent 3;\\lsdpriority63 Medium Shading 1 Accent 3;\\lsdpriority64 Medium Shading 2 Accent 3;\\lsdpriority65 Medium List 1 Accent 3;\\lsdpriority66 Medium List 2 Accent 3;\r\n\\lsdpriority67 Medium Grid 1 Accent 3;\\lsdpriority68 Medium Grid 2 Accent 3;\\lsdpriority69 Medium Grid 3 Accent 3;\\lsdpriority70 Dark List Accent 3;\\lsdpriority71 Colorful Shading Accent 3;\\lsdpriority72 Colorful List Accent 3;\\lsdpriority73 Colorful Grid Accent 3;\r\n\\lsdpriority60 Light Shading Accent 4;\\lsdpriority61 Light List Accent 4;\\lsdpriority62 Light Grid Accent 4;\\lsdpriority63 Medium Shading 1 Accent 4;\\lsdpriority64 Medium Shading 2 Accent 4;\\lsdpriority65 Medium List 1 Accent 4;\\lsdpriority66 Medium List 2 Accent 4;\r\n\\lsdpriority67 Medium Grid 1 Accent 4;\\lsdpriority68 Medium Grid 2 Accent 4;\\lsdpriority69 Medium Grid 3 Accent 4;\\lsdpriority70 Dark List Accent 4;\\lsdpriority71 Colorful Shading Accent 4;\\lsdpriority72 Colorful List Accent 4;\\lsdpriority73 Colorful Grid Accent 4;\r\n\\lsdpriority60 Light Shading Accent 5;\\lsdpriority61 Light List Accent 5;\\lsdpriority62 Light Grid Accent 5;\\lsdpriority63 Medium Shading 1 Accent 5;\\lsdpriority64 Medium Shading 2 Accent 5;\\lsdpriority65 Medium List 1 Accent 5;\\lsdpriority66 Medium List 2 Accent 5;\r\n\\lsdpriority67 Medium Grid 1 Accent 5;\\lsdpriority68 Medium Grid 2 Accent 5;\\lsdpriority69 Medium Grid 3 Accent 5;\\lsdpriority70 Dark List Accent 5;\\lsdpriority71 Colorful Shading Accent 5;\\lsdpriority72 Colorful List Accent 5;\\lsdpriority73 Colorful Grid Accent 5;\r\n\\lsdpriority60 Light Shading Accent 6;\\lsdpriority61 Light List Accent 6;\\lsdpriority62 Light Grid Accent 6;\\lsdpriority63 Medium Shading 1 Accent 6;\\lsdpriority64 Medium Shading 2 Accent 6;\\lsdpriority65 Medium List 1 Accent 6;\\lsdpriority66 Medium List 2 Accent 6;\r\n\\lsdpriority67 Medium Grid 1 Accent 6;\\lsdpriority68 Medium Grid 2 Accent 6;\\lsdpriority69 Medium Grid 3 Accent 6;\\lsdpriority70 Dark List Accent 6;\\lsdpriority71 Colorful Shading Accent 6;\\lsdpriority72 Colorful List Accent 6;\\lsdpriority73 Colorful Grid Accent 6;\r\n\\lsdqformat1\\lsdpriority19 Subtle Emphasis;\\lsdqformat1\\lsdpriority21 Intense Emphasis;\\lsdqformat1\\lsdpriority31 Subtle Reference;\\lsdqformat1\\lsdpriority32 Intense Reference;\\lsdqformat1\\lsdpriority33 Book Title;\\lsdsemihidden1\\lsdunhideused1\\lsdpriority37 Bibliography;\r\n\\lsdsemihidden1\\lsdunhideused1\\lsdqformat1\\lsdpriority39 TOC Heading;}}}"
  },
  {
    "path": "tests_data/basic/ruby/code.rb",
    "content": "class Kalimat\n  def initialize(nama = \"Dunia\")\n    @nama = nama\n  end\n  def sapaan\n    puts \"Hai #{@nama}.\"\n  end\n  def perpisahan\n    puts \"Sampai jumpa #{@nama}.\"\n  end\nend\n"
  },
  {
    "path": "tests_data/basic/rust/asm.rs",
    "content": "use std::arch::asm;\n\nfn main() {\n \n    let mut x: u64 = 5;\n\n    println!(\"Original value of x: {}\", x);\n\n    unsafe {\n        asm!(\n            \"mov rax, {x}\",      \n            \"mul rax\",            \n            \"mov {x}, rax\",       \n            x = inout(reg) x,    \n        );\n    }\n\n    println!(\"Squared value of x: {}\", x);\n\n    assert_eq!(x, 5 * 5);\n}\n"
  },
  {
    "path": "tests_data/basic/rust/code.rs",
    "content": "fn main() {\n    println!(\"Hello World!\");\n}\n"
  },
  {
    "path": "tests_data/basic/rust/test_case1.rs",
    "content": "/// Sample function to load a file\nfn load_model() {\n    println!(\"Magika model ⏳\");\n    // Simulating a delay for loading\n    std::thread::sleep(std::time::Duration::from_millis(200));\n    println!(\"Model loaded successfully!\");\n}\n\nfn main() {\n    // Load the model\n    load_model();\n}"
  },
  {
    "path": "tests_data/basic/rust/test_case2.rs",
    "content": "use std::fs::File;\nuse std::io::{Write, BufWriter}; \n\n// Function to create a  CSV file\nfn create_csv() -> std::io::Result<()> {\n    let mut file = File::create(\"sample.csv\")?;\n    writeln!(file, \"Name,Age,City\")?;\n    writeln!(file, \"Alice,30,New York\")?;\n    writeln!(file, \"Bob,25,Los Angeles\")?;\n    writeln!(file, \"Charlie,35,Chicago\")?;\n    Ok(())\n}\n\n// Function to create a  JSON file\nfn create_json() -> std::io::Result<()> {\n    let mut file = File::create(\"sample.json\")?;\n    writeln!(file, \"{{\\\"name\\\": \\\"Alice\\\", \\\"age\\\": 30, \\\"city\\\": \\\"New York\\\"}}\")?;\n    Ok(())\n}\n\n// Main function \nfn main() -> std::io::Result<()> {\n    // Function calls\n    create_csv()?;\n    create_json()?;\n    println!(\"Sample files created successfully.\");\n    Ok(())\n}"
  },
  {
    "path": "tests_data/basic/smali/code.smali",
    "content": ".class public LHelloWorld;\n\n.super Ljava/lang/Object;\n\n.method public static main([Ljava/lang/String;)V\n    .registers 2\n\n    sget-object v0, Ljava/lang/System;->out:Ljava/io/PrintStream;\n\n    const-string\tv1, \"Hello World!\"\n\n    invoke-virtual {v0, v1}, Ljava/io/PrintStream;->println(Ljava/lang/String;)V\n\n    return-void\n.end method\n"
  },
  {
    "path": "tests_data/basic/srt/code.srt",
    "content": "1\n00:00:00,000 --> 00:00:02,000\nHello, World!\n"
  },
  {
    "path": "tests_data/basic/swift/code.swift",
    "content": "import Foundation\r\n\r\nlet string = \"Hello, World!\"\r\nprint(string)\r\n"
  },
  {
    "path": "tests_data/basic/toml/doc.toml",
    "content": "[tool.poetry]\nname = \"magika\"\nversion = \"0.6.0-dev\"\ndescription = \"A tool to determine the content type of a file with deep-learning\"\nauthors = [\"Yanick Fratantonio \"]\nreadme = \"README.md\"\npackages = [{include = \"magika\"}]\n\n[tool.poetry.dependencies]\npython = \"^3.8,<3.13\"\nclick = \"^8.1.3\"\ntqdm = \"^4.66.2\"\nonnxruntime = \"^1.17.0\"\nnumpy = [\n    {version = \"^1.24\", python = \">=3.8,<3.9\"},\n    {version = \"^1.26\", python = \">=3.9,<3.13\"}\n]\ntabulate = \"^0.9.0\"\npython-dotenv = \"^1.0.1\"\n\n[tool.poetry.group.dev.dependencies]\npytest = \"^8.0.1\"\nipython = [\n    {version = \"^8.12.3\", python = \">=3.8,<3.9\"},\n    {version = \"^8.18.1\", python = \">=3.9,<3.10\"},\n    {version = \"^8.21.0\", python = \">=3.10,<3.13\"}\n]\nruff = \">=0.2.2,<0.4.0\"\nmypy = \"^1.8.0\"\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n\n[tool.ruff.lint]\n# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`)  codes by default.\n# Unlike Flake8, Ruff doesn't enable pycodestyle warnings (`W`) or\n# McCabe complexity (`C901`) by default.\nselect = [\"E4\", \"E7\", \"E9\", \"F\", \"I001\"]\nignore = []\n"
  },
  {
    "path": "tests_data/basic/tsv/magika_test.tsv",
    "content": "Name\tValue1\tValue2\tValue3\r\nTest1\t1\t10\t100\r\nTest2\t2\t20\t200\r\nTest3\t3\t30\t300\r\nTest4\t4\t40\t400\r\nTest5\t5\t50\t500"
  },
  {
    "path": "tests_data/basic/twig/example.twig",
    "content": "{% set items = ['apple', 'banana', 'cherry'] %}\n\n
    \n  {% for item in items %}\n    
  • {{ loop.index }} - {{ item|title }}
    • \n  {% endfor %}\n
\n\n{% block content %}\n  
This is content from a block definition.
\n{% endblock %}\n\n{{ dump(items) }}\n\n{{ 'hello world'|title }}\n\n{% macro input(name, value = '', type = 'text') %}\n  \n{% endmacro %}\n\n"
  },
  {
    "path": "tests_data/basic/txt/complex-sentence.txt",
    "content": "This is yet another simple test, it includes one simple sentence, but it is not as trivial as other simpler tests."
  },
  {
    "path": "tests_data/basic/txt/few-words.txt",
    "content": "this is just a test"
  },
  {
    "path": "tests_data/basic/txt/lorem-big.txt",
    "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer aliquam dui sit amet justo eleifend ullamcorper. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Mauris ut nunc feugiat, rutrum sem quis, sodales velit. Etiam gravida, nisi ut facilisis laoreet, metus odio laoreet dolor, in molestie leo nisl scelerisque justo. Nulla blandit massa a metus sagittis, vel blandit dui luctus. Donec arcu arcu, egestas convallis ultricies id, viverra a ante. Donec maximus turpis metus, eget mattis massa scelerisque in. Nam mattis, mi vitae iaculis dapibus, tellus eros cursus dui, sed sagittis elit augue ornare lacus. In massa massa, ornare in malesuada quis, elementum ut magna. Nullam rhoncus volutpat nibh, eu aliquet lectus. Praesent scelerisque velit sem, quis sodales augue venenatis eu.\n\nUt facilisis erat erat, at finibus purus elementum et. Quisque dignissim nunc et magna blandit tincidunt. Sed in libero eget ligula molestie aliquet quis a enim. Curabitur ac mauris nec sapien sollicitudin consectetur vitae luctus nisi. Ut augue nisi, facilisis eget ullamcorper in, eleifend at augue. Sed lacinia est eget porta aliquam. Aliquam erat volutpat. Aliquam congue, mi sed laoreet vestibulum, lectus purus condimentum sem, sit amet elementum felis lectus porttitor mauris.\n\nNam lobortis commodo ex, in lacinia augue congue nec. Fusce fermentum, sem vel hendrerit viverra, nulla elit molestie velit, a iaculis ex mi vitae ipsum. Mauris eleifend leo quis mi venenatis, gravida venenatis lacus commodo. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Integer laoreet vel leo quis suscipit. Nam consectetur ligula quis interdum volutpat. Donec at libero semper libero faucibus placerat in ut quam. Aliquam ipsum urna, posuere quis orci sit amet, vulputate finibus orci. Sed tortor metus, scelerisque eu magna sed, convallis aliquam erat. Sed pretium elit fermentum sodales hendrerit.\n\nUt eleifend lectus et ullamcorper interdum. Sed laoreet, quam at placerat viverra, ipsum erat rhoncus dolor, sit amet elementum nisl elit at lectus. Praesent volutpat nulla at diam placerat, ut interdum diam vehicula. Etiam in mattis metus, sed pharetra urna. Aenean vitae urna sit amet metus malesuada sodales. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean varius est quam, a rutrum mi luctus et. Quisque fermentum, augue nec lobortis accumsan, enim arcu ultricies orci, at vestibulum magna ex id dui. Vestibulum vestibulum efficitur elementum. Mauris in congue nunc, vitae pulvinar ipsum. Pellentesque at volutpat lectus, quis feugiat justo. Sed sed malesuada justo. Quisque fermentum molestie porta. Aenean volutpat tellus a viverra ornare. Nulla semper mauris id orci efficitur, eu pharetra velit porta.\n\nProin velit turpis, rutrum ut pretium molestie, pretium ut eros. Vivamus porttitor elementum varius. Proin a erat et lectus dapibus mollis at ut sapien. Cras pellentesque neque massa, non congue lorem bibendum at. Quisque accumsan elit at leo commodo molestie. Phasellus nibh enim, vehicula vitae massa non, scelerisque semper ligula. Aliquam facilisis, dolor non pulvinar consequat, dolor nisl porttitor dui, efficitur ultrices urna massa quis libero.\n\nUt vel tincidunt odio. Quisque semper et est sed pharetra. Cras enim orci, convallis sit amet enim sed, varius mattis velit. Ut ac nisi in dui ultrices vehicula vel et velit. Quisque dictum feugiat velit. Etiam pellentesque turpis ullamcorper urna finibus mattis. Suspendisse convallis vulputate justo, vel fringilla tortor interdum ut. Nullam placerat, magna nec blandit interdum, nisl turpis pharetra dolor, quis laoreet neque lacus a ipsum.\n\nNam blandit magna nisl, a mattis felis cursus sit amet. Fusce sit amet arcu feugiat, volutpat felis sed, tincidunt dolor. Praesent vulputate ex diam, ut congue sem pretium ut. Morbi pretium nunc sit amet turpis tincidunt, a ultrices enim tristique. Praesent lacinia id nisl nec hendrerit. Cras fermentum dui sit amet elementum tristique. Curabitur vel suscipit dui. Aenean at dolor dui. Quisque eget ex at tellus aliquet venenatis quis sit amet urna. Pellentesque gravida, ipsum at pharetra laoreet, ante mi sollicitudin ligula, a congue felis purus consectetur lectus. Interdum et malesuada fames ac ante ipsum primis in faucibus. Duis mi diam, egestas a posuere non, ornare a lectus.\n\nOrci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Sed lacinia porttitor arcu ac pulvinar. Etiam tincidunt porta lacus, eu lacinia quam venenatis sed. Suspendisse posuere metus sit amet massa congue, non pretium lectus vulputate. Donec in odio sit amet metus imperdiet accumsan. Praesent posuere venenatis ipsum in rutrum. Curabitur vulputate sapien sit amet diam sodales, non aliquet nisi blandit. Sed et lectus hendrerit, porttitor velit sed, tincidunt nulla. Donec aliquam dictum enim et porttitor. Etiam et mi sit amet justo vehicula posuere ac ut quam. Vestibulum malesuada, risus in placerat feugiat, mauris diam mollis ligula, nec mollis ligula lectus eu ligula. Ut bibendum fermentum justo, eget convallis nunc egestas eu. Nam a tellus sit amet sapien viverra tempor ac nec lorem. Maecenas ultricies neque ut odio pulvinar mattis. Vivamus aliquam nisl in ultrices suscipit.\n\nPraesent eleifend nisi eget nisl elementum, suscipit accumsan mauris ullamcorper. Cras placerat, purus non pulvinar volutpat, nisi mauris tincidunt justo, dictum luctus dui purus non erat. Ut et arcu lorem. Fusce in nisi vitae mauris pharetra congue. Ut nec ex lacinia, molestie elit vitae, hendrerit neque. Curabitur lobortis justo non libero scelerisque eleifend. Vivamus augue augue, fringilla ut ante nec, hendrerit efficitur dolor.\n\nDonec molestie elit congue tristique dictum. Cras finibus sed metus vitae placerat. Nunc vel cursus enim. Duis condimentum, massa et commodo sagittis, eros risus tempor odio, id ornare mauris leo a velit. In pretium risus vitae lectus pulvinar, in dignissim risus pharetra. Interdum et malesuada fames ac ante ipsum primis in faucibus. Etiam faucibus eleifend ante eu mattis."
  },
  {
    "path": "tests_data/basic/txt/lorem-small.txt",
    "content": "Utilitatis causa amicitia est quaesita.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Collatio igitur ista te nihil iuvat. Honesta oratio, Socratica, Platonis etiam. Primum in nostrane potestate est, quid meminerimus? Duo Reges: constructio interrete. Quid, si etiam iucunda memoria est praeteritorum malorum? Si quidem, inquit, tollerem, sed relinquo. An nisi populari fama?\n\nQuamquam id quidem licebit iis existimare, qui legerint. Summum a vobis bonum voluptas dicitur. At hoc in eo M. Refert tamen, quo modo. Quid sequatur, quid repugnet, vident. Iam id ipsum absurdum, maximum malum neglegi."
  },
  {
    "path": "tests_data/basic/txt/magika_test_pptx.txt",
    "content": "This is a test for Magika!\n\nVery cool if this can be detected correctly!\n"
  },
  {
    "path": "tests_data/basic/txt/many-words.txt",
    "content": "this is just a test but it contains more words than the simple test"
  },
  {
    "path": "tests_data/basic/txt/one-sentence-with-newline.txt",
    "content": "This is just a test that includes a simple sentence.\n"
  },
  {
    "path": "tests_data/basic/txt/one-sentence.txt",
    "content": "This is just a test that includes a simple sentence."
  },
  {
    "path": "tests_data/basic/txt/random-ascii.txt",
    "content": "faslkdfjhasdfkljhasdfklajshdfaklsjdfhaluehzsdjvnmcnbxzcv"
  },
  {
    "path": "tests_data/basic/typescript/code.ts",
    "content": "// This is typescript, and not valid javascript.\ninterface Person {\n  name: string;\n  age: number;\n}\n\nfunction greet(person: Person): string {\n  return `Hello, ${person.name}. You are ${person.age} years old.`;\n}\n\nconst user: Person = {\n  name: \"Bob\",\n  age: 42,\n};\n\nconsole.log(greet(user));\n"
  },
  {
    "path": "tests_data/basic/yaml/dependabot.yml",
    "content": "version: 2\r\n\r\nupdates:\r\n  - package-ecosystem: \"github-actions\"\r\n    directory: \"/\"\r\n    schedule:\r\n      interval: \"monthly\"\r\n  - package-ecosystem: \"docker\"\r\n    directory: \"/\"\r\n    schedule:\r\n      interval: \"daily\"\r\n  - package-ecosystem: \"pip\"\r\n    directory: \"/python\"\r\n    schedule:\r\n      interval: \"daily\"\r\n  - package-ecosystem: \"npm\"\r\n    directory: \"/js\"\r\n    schedule:\r\n      interval: \"weekly\"\r\n  - package-ecosystem: \"cargo\"\r\n    directory: \"/rust\"\r\n    schedule:\r\n      interval: \"weekly\"\r\n"
  },
  {
    "path": "tests_data/basic/yaml/python-test.yml",
    "content": "name: Python - test\n\non:\n  workflow_dispatch:\n  push:\n    branches:\n      - 'main'\n    paths:\n      - 'python/**'\n      - 'tests_data/**'\n      - '.github/workflows/**'\n  pull_request:\n    paths:\n      - 'python/**'\n      - 'tests_data/**'\n      - '.github/workflows/**'\n\npermissions:\n  contents: read\n\njobs:\n  unit-testing:\n    strategy:\n      matrix:\n        python-version: [ \"3.8.x\", \"3.9.x\", \"3.10.x\", \"3.11.x\", \"3.12.x\" ]\n        os: [ \"ubuntu-latest\", \"macos-latest\" ]\n    runs-on: ${{ matrix.os }}\n    steps:\n      - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # pin@v4\n\n      - name: Setup Python\n        uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f # pin@v5\n        with:\n          python-version: '${{ matrix.python-version }}'\n\n      - name: Install poetry\n        uses: abatilo/actions-poetry@7b6d33e44b4f08d7021a1dee3c044e9c253d6439 # pin@v3\n        with:\n          poetry-version: \"1.7.1\"\n\n      - name: Install the project dependencies\n        working-directory: python\n        run: poetry install\n\n      - name: Run ruff check\n        working-directory: python\n        run: poetry run ruff check --verbose\n\n      - name: Run ruff format check\n        working-directory: python\n        run: poetry run ruff format --check --verbose\n\n      - name: Run mypy\n        working-directory: python\n        run: poetry run mypy magika tests\n\n      - name: Run pytest\n        working-directory: python\n        run: poetry run pytest tests -m \"not slow\"\n"
  },
  {
    "path": "tests_data/basic/yara/rule.yar",
    "content": "rule Rule_485729_77379 {\n  strings:\n    $s1 = \"HKEY_LOCAL_MACHINE\\\\SOFTWARE\\\\Microsoft\\\\Windows\\\\CurrentVersion\\\\Run\"\n    $s2 = \"Win32_Process\"\n    $s3 = \"Create\" wide\n  condition:\n    $s1 and ($s2 and $s3)\n  meta:\n    author = \"CyberThreatResearch\"\n    date = \"2019-09-23\"\n    tags = \"malware, persistence, registry\"\n}\n"
  },
  {
    "path": "tests_data/basic/zig/code.zig",
    "content": "const std = @import(\"std\");\n\npub fn main() void {\n    std.debug.print(\"Hello, World!\\n\", .{});\n}\n"
  },
  {
    "path": "tests_data/mitra/php/php.php",
    "content": "PHP: Hypertext Preprocessor
\"; ?>"
  },
  {
    "path": "tests_data/mitra_candidates/html.htm",
    "content": "-->\n
\n\n
HTML page
\n\n\n\n
\n\n\n### Table of Contents\n\n*   [MagikaNode][1]\n    *   [identifyStream][2]\n        *   [Parameters][3]\n    *   [create][4]\n        *   [Parameters][5]\n*   [Magika][6]\n    *   [identifyBytes][7]\n        *   [Parameters][8]\n    *   [create][9]\n        *   [Parameters][10]\n\n## MagikaNode\n\n**Extends Magika**\n\nThe main Magika object for Node use (`MagikaNode`).\n\nExample usage:\n\n```js\nimport { readFile } from \"fs/promises\";\nimport { MagikaNode as Magika } from \"magika/node\";\nconst data = await readFile(\"some file\");\nconst magika = await Magika.create();\nconst result = await magika.identifyBytes(data);\nconsole.log(result.prediction.output.label);\n```\n\nFor a client-side implementation, please import `Magika` instead.\n\nNote that this `MagikaNode` class extends `Magika`, which means that all\npublic `Magika` APIs (e.g., `identifyBytes`) are available for `MagikaNode`\nas well.\n\nDemos:\n\n*   Node: `/js/magika-cli.js`, which you can run with `yarn run bin -h`.\n*   Client-side: see `/website/src/components/FileClassifierDemo.vue`\n\n### identifyStream\n\nIdentifies the content type from a read stream\n\n#### Parameters\n\n*   `stream` **ReadStream** A read stream.\n*   `length` **[number][11]** Total length of stream data.\n\nReturns **MagikaResult** An object containing the result of the content type\nprediction.\n\n### create\n\nFactory method to create a Magika instance.\n\n#### Parameters\n\n*   `options` **MagikaOptions** The urls or file paths where the model and\n    its config are stored.Parameters are optional. If not provided, the model will be loaded from GitHub.\n\nReturns **[Promise][12]<[MagikaNode][1]>** \n\n## Magika\n\nThe main Magika object for client-side use.\n\nExample usage:\n\n```js\nconst file = new File([\"# Hello I am a markdown file\"], \"hello.md\");\nconst fileBytes = new Uint8Array(await file.arrayBuffer());\nconst magika = await Magika.create();\nconst result = await magika.identifyBytes(fileBytes);\nconsole.log(result.prediction.output.label);\n```\n\nFor a Node implementation, please import `MagikaNode` instead.\n\nDemos:\n\n*   Node: `/js/magika-cli.js`, which you can run with `yarn run bin -h`.\n*   Client-side: see `/website/src/components/FileClassifierDemo.vue`\n\n### identifyBytes\n\nIdentifies the content type of a byte array.\n\n#### Parameters\n\n*   `fileBytes` **[Uint8Array][13]** A fixed-length sequence of bytes.\n\nReturns **MagikaResult** An object containing the result of the content type\nprediction.\n\n### create\n\nFactory method to create a Magika instance.\n\n#### Parameters\n\n*   `options` **MagikaOptions** The urls or file paths where the model and\n    its config are stored.Parameters are optional. If not provided, the model will be loaded from GitHub.\n\nReturns **[Promise][12]<[Magika][6]>** \n\n[1]: #magikanode\n\n[2]: #identifystream\n\n[3]: #parameters\n\n[4]: #create\n\n[5]: #parameters-1\n\n[6]: #magika\n\n[7]: #identifybytes\n\n[8]: #parameters-2\n\n[9]: #create-1\n\n[10]: #parameters-3\n\n[11]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number\n\n[12]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise\n\n[13]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array\n"
  },
  {
    "path": "website-ng/src/content/docs/cli-and-bindings/js.md",
    "content": "---\ntitle: \"JavaScript / TypeScript Library\"\n---\n\nUse Magika in the browser or in Node!\n\n## Installing MagikaJS\n\n```bash\nnpm install magika\n```\n\n## Using Magika in JavaScript\n\nSimple usage in Node:\n\n```js\nimport { readFile } from \"fs/promises\";\nimport { MagikaNode as Magika } from \"magika/node\";\n\nconst data = await readFile(\"some file\");\nconst magika = await Magika().create();\nconst prediction = await magika.identifyBytes(data);\nconsole.log(prediction);\n```\n\nSimple usage in the browser:\n\n```js\nimport { Magika } from \"magika\";\n\nconst file = new File([\"# Hello I am a markdown file\"], \"hello.md\");\nconst fileBytes = new Uint8Array(await file.arrayBuffer());\nconst magika = await Magika.create();\nconst prediction = await magika.identifyBytes(fileBytes);\nconsole.log(prediction);\n```\n\nFor more, see the API reference below.\n\n## Command-line tool\n\nPlease use the official CLI as it can perform batch processing and search for files recursively.\nRead more about that in the main the [Command Line Interface (CLI)](/magika/cli-and-bindings/cli/) section.\nThis one is useful to load the TensorflowJS model and see that it works as expected.\n\nInstall it with `npm install -g magika`. You can then run it by executing `magika-js `\n\n```\nUsage: magika-js [options] \n\nMagika JS - file type detection with ML. https://google.github.io/magika\n\nArguments:\n  paths                                    Paths of the files to detect\n\nOptions:\n  --json-output                            Format output in JSON\n  --model-url                   Model URL (default: \"https://google.github.io/magika/models/standard_v3_2/model.json\")\n  --model-path                 Model file path\n  --model-config-url     Model config URL (default: \"https://google.github.io/magika/models/standard_v3_2/config.min.json\")\n  --model-config-path   Model config file path\n  --by-stream                              Identify file via stream, not via bytes\n  --debug                                  Output debug information\n  -h, --help                               display help for command\n```\n\n\n## Loading the model and configuration\n\nMagikaJS is designed to be flexible in how you provide the model and configuration file to it.\n\nBoth the Node and browser versions accept URLs to asynchronously load these two assets.\n\n```js\nconst magika = await magika.create({\n  modelURL: \"https://...\",\n  configURL: \"https://...\",\n});\n```\n\nThe Node version also allows to load local files.\n\n```js\nconst magika = await magika.create({\n  modelPath: \"./assets/...\",\n  configPath: \"./assets/...\",\n});\n```\n\n## Development\n\nUsing the model hosted On Github:\n\n```bash\nyarn install\nyarn run build\nyarn run bin -- README.md\n```\n\nUsing the local model:\n\n```bash\nyarn install\nyarn run build\n(cd ../website; yarn install; yarn run dev) &\nyarn run bin --model-url http://localhost:5173/magika/model/model.json --config-url http://localhost:5173/magika/model/config.json ../tests_data/basic/*\n```\n\nUsing the local `magika` package when developing the website:\n\n```bash\nyarn install\nyarn run build\nyarn link\n(cd ../website; yarn link magika; yarn install; yarn run dev) &\n```\n\n## Testing\n\nExecute:\n\n```bash\nyarn install\nyarn run build\nyarn run test\n```\n\n## API Reference\n\nSee the [JavaScript API Reference](/magika/cli-and-bindings/js-api) section.\n"
  },
  {
    "path": "website-ng/src/content/docs/cli-and-bindings/other-bindings.md",
    "content": "---\ntitle: \"Other bindings\"\n---\n\n### Go (In Progress)\nA Go port of Magika is currently under development. While the implementation is largely complete, a few final steps are needed before it can be published as a package. You can explore the source code and track its progress on GitHub.\n\nSource Code: github.com/google/magika/tree/main/go\n\n### Other Languages\n\nOfficial bindings for other languages are not yet available. However, since Magika's core is built in Rust, it can be integrated into many programming environments using a Foreign Function Interface (FFI). For example:\n\n- Java (JVM): Integration should be possible using the Java Native Interface (JNI).\n\n- .NET: Integration can be achieved using P/Invoke.\n\nWe encourage community contributions to create and maintain new bindings. If you are interested in developing one, please feel free to contact us to discuss it."
  },
  {
    "path": "website-ng/src/content/docs/cli-and-bindings/overview.md",
    "content": "---\ntitle: CLI & Bindings Overview\n---\n\nMagika provides a native CLI for command-line use and official language bindings—including Python, JavaScript, and Rust—to enable seamless integration into diverse projects.\n\n\n| Artifact                                                       | Status         | Latest version | Default model                                              |\n| -------------------------------------------------------------- | -------------- | -------------- | ---------------------------------------------------------- |\n| [`magika` CLI](/magika/cli-and-bindings/cli)                          | Stable         | `1.0.2`        | [`standard_v3_3`](https://github.com/google/magika/blob/main/assets/models/standard_v3_3/README.md) |\n| [Python `Magika` module](/magika/cli-and-bindings/python)             | Stable         | `1.0.2`        | [`standard_v3_3`](https://github.com/google/magika/blob/main/assets/models/standard_v3_3/README.md) |\n| [JavaScript / TypeScript package](/magika/cli-and-bindings/js)        | Stable         | `1.0.0`        | [`standard_v3_3`](https://github.com/google/magika/blob/main/assets/models/standard_v3_3/README.md) |\n| [Rust `magika` library](/magika/cli-and-bindings/rust)                | Stable         | `1.0.1`        | [`standard_v3_3`](https://github.com/google/magika/blob/main/assets/models/standard_v3_3/README.md) |\n| [Demo Website](/magika/demo/magika-demo/)                             | Stable         | -              | [`standard_v3_3`](https://github.com/google/magika/blob/main/assets/models/standard_v3_3/README.md) |\n| [Other bindings](/magika/cli-and-bindings/other-bindings)             | WIP            | -              | -                                                          |\n"
  },
  {
    "path": "website-ng/src/content/docs/cli-and-bindings/python.md",
    "content": "---\ntitle: \"Python `Magika` Module\"\n---\n\nThis guide provides documentation on how to use the `magika` Python module to identify file types from your code.\n\n:::tip\nThis section assumes you are familiar with the topics discussed in the [Core Concepts](/magika/core-concepts/) section.\n:::\n\n## Quick Examples\n\nThe `magika` API is designed to be simple and intuitive. The following examples cover the most common use cases for identifying content from bytes, paths, and streams.\n\n**From bytes:**\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_bytes(b'function log(msg) {console.log(msg);}')\n>>> print(res.output.label)\njavascript\n```\n\n**From a file path:**\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_path('./tests_data/basic/ini/doc.ini')\n>>> print(res.output.label)\nini\n```\n\n**From an open file stream:**\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f:\n>>>     res = m.identify_stream(f)\n>>> print(res.output.label)\nini\n```\n\n## API Reference\n\n### Instantiating `Magika`\n\nFirst, create an instance of the `Magika` class. The constructor accepts several optional arguments to customize its behavior.\n\n```python\nfrom magika import Magika, PredictionMode\n\n# Default instantiation\nmagika = Magika()\n\n# Custom instantiation\nmagika_custom = Magika(\n    model_dir=\"/path/to/custom/model\",\n    prediction_mode=PredictionMode.BEST_GUESS,\n    no_dereference=True,\n)\n```\n\n**Constructor Arguments:**\n- `model_dir` (`Path`, optional): Path to a directory containing a custom model. If not provided, defaults to the latest bundled model.\n- `prediction_mode` (`PredictionMode`, optional): The prediction mode to use. Defaults to `PredictionMode.HIGH_CONFIDENCE`.\n- `no_dereference` (`bool`, optional): If `True`, symbolic links will not be followed; their content type will be reported as `symlink`. Defaults to `False`.\n\n**Identifying Content**\n\nOnce instantiated, the `Magika` object provides several methods for identifying content from different sources.\n- `magika.identify_bytes(bytes)`: Identifies the content type of an in-memory bytes object.\n- `magika.identify_path(path)`: Identifies the content type of a single file from its path (`str | os.PathLike`).\n- `magika.identify_paths(paths)`: Identifies the content type for a list of file paths.\n- `magika.identify_stream(stream)`: Identifies the content type from an already-open binary file-like object (e.g., the output of `open(file_path, 'rb')`). Note: 1) Magika will `seek()` around the stream; 2) the stream _is not closed_ (closing is the responsibility of the caller).\n\nIf you are dealing with large files, the `identify_path`, `identify_paths`, and `identify_stream` variants are generally better: their implementation `seek()`s around the file/stream to extract the needed features, without loading the entire content in memory.\n\n:::tip[Performance with Large Files]\nFor large files, the `identify_path`, `identify_paths`, and `identify_stream` methods are highly recommended. They are optimized to read only the necessary portions of the file by seeking within the file/stream, which avoids loading the entire content into memory. If your content is already loaded into a bytes object, `identify_bytes` is the most direct and efficient option.\n:::\n\n**Understanding the Result**\n\nAll `identify_*` methods return a `MagikaResult` object. This object acts as a wrapper that contains the prediction details and the status of the operation. **You should always check if the operation was successful before accessing the prediction.**\n\n```python\n>>> result = m.identify_path(\"path/to/file\")\n>>> if result.ok:\n...     print(f\"File is a {result.output.description}\")\n...     print(f\"MIME Type: {result.output.mime_type}\")\n... else:\n...     print(f\"Error: {result.status.message}\")\n```\n\n\n### Data Models\n\nThe `MagikaResult` object and its nested data classes provide detailed information about the scan.\n\nConsult the [Understanding the Output](/magika/core-concepts/understanding-the-output) section for more context.\n\n`MagikaResult`\n\n```python\nclass MagikaResult:\n    path: Path\n    ok: bool\n    status: Status\n    prediction: MagikaPrediction\n    # Shortcuts available only when result.ok is True\n    dl: ContentTypeInfo\n    output: ContentTypeInfo\n    score: float\n```\n\n- `ok` (bool): `True` if the identification was successful, `False` otherwise.\n- `status` (Status): Provides details on an error if `ok` is `False`.\n- `prediction` (`MagikaPrediction`): The core prediction object, available only if `ok` is `True`.\n- `dl`, `output`, `score`: For convenience, these are direct shortcuts to the corresponding fields within the `prediction` object.\n\n\n`MagikaPrediction`\n\nContains the core deep learning model prediction and the final Magika output.\n\n```python\nclass MagikaPrediction:\n    dl: ContentTypeInfo\n    output: ContentTypeInfo\n    score: float\n    overwrite_reason: OverwriteReason\n```\n\n- `dl` (`ContentTypeInfo`): The raw prediction from the deep learning model.\n- `output` (`ContentTypeInfo`): The final prediction from \"Magika the tool,\" which considers the model's prediction, its confidence score, and the selected prediction mode. **This is the result most users should rely on.**\n- `score` (`float`): The model's confidence score (from 0.0 to 1.0).\n- `overwrite_reason` (`OverwriteReason`): It indicates why the deep learning model's prediction was overwritten (e.g., low confidence).\n\n`ContentTypeInfo`\n\nContains detailed metadata about a predicted content type.\n\n```python\nclass ContentTypeInfo:\n    label: ContentTypeLabel # e.g., \"python\"\n    mime_type: str          # e.g., \"text/x-python\"\n    group: str              # e.g., \"code\"\n    description: str        # e.g., \"Python source\"\n    extensions: List[str]   # e.g., [\"py\", \"pyc\"]\n    is_text: bool           # e.g., True\n```\n\n`ContentTypeLabel`\n\nA string enum (`StrEnum`) of all possible content type labels. Because it's a `StrEnum`, its members can be used and compared just like regular strings.\n\n```python\nclass ContentTypeLabel(StrEnum):\n    APK = \"apk\"\n    BMP = \"bmp\"\n    # ... and many more\n```\n\n:::caution\n**`ContentTypeLabel` is a superset of supported types.**\n\nThis enum is generated from our internal Content Types Knowledge Base and includes many types that the default model may not be trained to detect.\n\nThe presence of a label in this enum **does not guarantee** it can be a prediction result. To get the definitive list of possible output labels that \"Magika the tool\" can return, use the `magika.get_output_content_types()` method, discussed next.\n:::\n\n\n### Additional APIs\n\nThe `Magika` class also exposes a few helper methods:\n- `get_output_content_types()`: Returns a list of all possible content type labels that Magika can return in the `output.label` field. This is the recommended way to get a definitive list of Magika's possible outputs.\n- `get_model_content_types()`: Returns a list of all possible content type labels the _deep learning model_ can return (i.e., the possible values for `dl.label`, in addition to `undefined`). This is useful for debugging.\n- `get_module_version()`: Returns the `magika` Python package version as a string.\n- `get_model_version()`: Returns the name of the model being used as a string.\n\n\n## Development setup\n\nThis section is for contributors to the `magika` Python package.\n\n- **Project Management:** `magika` uses `uv` for dependency management. To install all development dependencies, run: `cd python; uv sync`.\n\n- **Testing:** To run the test suite, use `pytest`. You can exclude slow tests for faster runs: `cd python; uv run pytest tests -m \"not slow\"`. Refer to the GitHub Actions workflows for more testing examples.\n\n- **Packaging:** We use `maturin` to build the Python package, which combines the Rust-based CLI with the Python source code. This process is automated in our [Build and Release Python Package GitHub Action](https://github.com/google/magika/blob/main/.github/workflows/python-build-and-release-package.yml).\n\n- **Publishing:** We publish to PyPI via GitHub Trusted Publishing. This is automated by the [Build and Release Python Package GitHub Action](https://github.com/google/magika/blob/main/.github/workflows/python-build-and-release-package.yml), which publishes packages (binary wheels, pure-python wheels, and source distribution) to PyPI (or TestPyPI) after pushing a tag with `python-v*` (or `python-test-v*`) as prefix. This also takes care of attestation.\n"
  },
  {
    "path": "website-ng/src/content/docs/cli-and-bindings/rust.md",
    "content": "---\ntitle: \"Rust Library\"\n---\n\nThe documentation for the Rust's `magika` library is on docs.rs: https://docs.rs/magika."
  },
  {
    "path": "website-ng/src/content/docs/contributing/creating-new-bindings.md",
    "content": "---\ntitle: Creating New Bindings\n---\n\nThese notes aim at helping bindings developers.\n\n### Reference implementation\n\nThe reference implementation is the python's `Magika` module, at `python/src/magika.py`.\n\nThe input vs. expected output examples are stored in `tests_data/reference`. See below about information on the format.\n\nThe reference tests are generated with `cd python && uv run ./scripts/generate_reference.py`.\n\n\n### Aspects to implement\n\nThere are three aspects that need to be implemented:\n- Logic that handles \"should we even use the model\"? See `_get_result_or_features_from_path`.\n- Features extraction. See `_extract_features_from_seekable`.\n- How to obtain \"Magika's output\" from the model's prediction, the score (which depends on the prediction mode, thresholds, and overwrite_map). See `_get_output_ct_label_from_dl_result`.\n\n\n### Testing\n\nWe have a number of test cases that one can use to check that a new implementation matches the reference implementation.\n\nTesting that the output (e.g., model prediction, tool overall prediction, score) of the tool matches the expectations:\n- We have a number of test cases that cover normal situations as well as corner cases related to small files, content types with custom thresholds and overwrite maps, and prediction modes. Note that these corner cases are model-specific (the actual weights). We use a fuzzing-like approach to generate them.\n- These examples are stored in two formats, \"examples by path\" and \"examples by content\". They are stored at `tests_data/reference/-inference_examples_by_content.json.gz` and `tests_data/reference/-inference_examples_by_content.json.gz`. These store a list of `ExampleByPath` and `ExampleByContent` (defined in `python/tests/test_inference_vs_reference.py`), respectively.\n\nTesting the features extraction:\n- Input and expected output of the features extraction: `tests_data/reference/features_extraction_examples.json.gz`.\n- The JSON contains a list of `FeaturesExtractionExample` (defined in `python/tests/test_features_extraction_vs_reference.py`).\n- Suggestion: having a testable \"extract features\" function makes your life much easier.\n- Note that end-to-end tests would not be enough to be confident the features extraction is correctly implemented, as small bugs may require VERY specific input to show differences.\n\nWhat is *not* covered by the existing tests:\n- How to deal with special files (e.g., symlinks, directory).\n- How to deal with `permission error`.\n- How to deal with `file_not_found_error`.\n"
  },
  {
    "path": "website-ng/src/content/docs/contributing/how-to-contribute.md",
    "content": "---\ntitle: How to Contribute\n---\n\nWe would love to accept your patches and contributions to this project!\n\n### Before you begin\n\n#### Sign our Contributor License Agreement\n\nContributions to this project must be accompanied by a\n[Contributor License Agreement](https://cla.developers.google.com/about) (CLA).\nYou (or your employer) retain the copyright to your contribution; this simply\ngives us permission to use and redistribute your contributions as part of the\nproject.\n\nIf you or your current employer have already signed the Google CLA (even if it\nwas for a different project), you probably don't need to do it again.\n\nVisit  to see your current agreements or to\nsign a new one.\n\n#### Review our Community Guidelines\n\nThis project follows [Google's Open Source Community\nGuidelines](https://opensource.google/conduct/).\n\n### Contribution process\n\n#### Code Reviews\n\nAll submissions, including submissions by project members, require review. We\nuse [GitHub pull requests](https://docs.github.com/articles/about-pull-requests)\nfor this purpose.\n"
  },
  {
    "path": "website-ng/src/content/docs/contributing/known-limitations.md",
    "content": "---\ntitle: Known Limitations\n---\n\nMagika significantly improves over the state of the art, but there's always room for improvement. More work can be done to increase detection accuracy, support for additional content types, bindings for more languages, etc.\n\nThis initial release is not targeting polyglot detection, and we're looking forward to seeing adversarial examples from the community.\n\nWe would also love to hear from the community about encountered problems, misdetections, features requests, need for support for additional content types, etc.\n\nCheck our open GitHub issues to see what is on our roadmap and please report misdetections or feature requests by either opening GitHub issues (preferred) or by emailing us at magika-dev@google.com.\n\nCheck [GitHub's open issues labeled as \"help wanted\"](https://github.com/google/magika/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) as a starting point.\n\n:::caution\nDo **NOT** send reports about files that may contain PII!\n:::\n"
  },
  {
    "path": "website-ng/src/content/docs/contributing/reporting-security-vulnerabilities.md",
    "content": "---\ntitle: Reporting Security Vulnerabilities\n---\n\nPlease contact us directly at magika-dev@google.com.\n"
  },
  {
    "path": "website-ng/src/content/docs/core-concepts/how-magika-works.md",
    "content": "---\ntitle: How Magika Works\n---\n\nMagika's command-line tool and language bindings are essentially wrappers around a compact deep learning model, optimized for efficient inference on standard CPUs.\n\nThe identification process is highly efficient because Magika primarily inspects a few hundred bytes of a file (depending on the model, usually up to 2K bytes). This approach ensures a fast, constant-time inference that is independent of the overall file size.\n\nThe core process works as follows:\n\n1. Magika reads a few chunks of the input file (or byte stream). This is fast and memory efficient even for big files, as they are never fully read in memory.\n2. It extracts \"features\" from these initial bytes, which are then processed by the deep learning model to predict the content type.\n3. After the model makes a prediction, Magika evaluates its confidence score.\n4. If this score exceeds a predefined threshold for the predicted type, Magika accepts the model's prediction. If the confidence is too low, Magika returns a more generic label, such as `txt` (for text files) or `unknown` (for binary files).\n\nThis distinction is important: Magika internally manages two content type labels—one from the deep learning model and one from **\"Magika the tool.\"** While they are often the same, they can differ when the model's confidence is low or in certain edge cases.\n\nThe model is not used in all situations. Specifically:\n- If the input file is **empty**, Magika returns `empty`.\n- If the input is not a regular file, such as a **directory** or a **symlink**, Magika returns `directory` or `symlink`.\n- If the file is **too small** for the model (e.g., under ~8 bytes), Magika uses simple heuristics to return a generic answer like `txt` or `unknown`.\n\nIn these cases, the model is not run, and its internal content type label is set to `undefined`. By default, users only see the final, processed prediction, but the model's raw output can be inspected for debugging. See the [Understanding the Output](/magika/core-concepts/understanding-the-output/) section for details."
  },
  {
    "path": "website-ng/src/content/docs/core-concepts/models-and-content-types.md",
    "content": "---\ntitle: Models & Supported Content Types\n---\n\nEach Magika model is trained to detect a specific set of content types. Newer models typically expand this set, supporting a superset of the content types from previous versions, unless specified otherwise.\n\nFor instance, our initial `standard_v1` model supported approximately 100 content types. The latest model, `standard_v3_3`, supports over 200 content types, while maintaining similar accuracy and inference speed.\n\nThe list of supported content types is documented in the [model's page](/magika/models/standard_v3_3).\n\n:::tip\nThe models' READMEs contain two lists: \"the output space of the model\" and \"the output space of Magika the tool.\" The second list is a superset of the first, including additional labels like `empty`, `directory`, and so on.\n:::\n\nDetails on the improvements and tradeoffs for each model can be found in the models' CHANGELOG on GitHub: [models/CHANGELOG.md](https://github.com/google/magika/blob/main/assets/models/CHANGELOG.md).\n\nClients and bindings usually integrate the latest available model, but this may not always be the case. For more information, check the [bindings section](/magika/cli-and-bindings/overview).\n\n:::caution\nYou may see a \"content types knowledge base\" (KB) in the GitHub source code. This is a comprehensive list of all content types we track internally for research and development. It should **not** be confused with the content types Magika currently supports. The KB is a superset of what any single model supports. To be certain which content types a specific model supports, always refer to its README file.\n:::\n"
  },
  {
    "path": "website-ng/src/content/docs/core-concepts/prediction-modes.md",
    "content": "---\ntitle: Prediction Modes\n---\n\nMagika's deep learning model returns each prediction with a confidence score (from 0.0 to 1.0). A common challenge with classification models is determining the minimum score required to trust a result.\n\nInstead of a single, global threshold, Magika uses **per-content-type thresholds**. The rationale is that the model is naturally more confident about some types than others. For example, our experiments show that most valid PDFs are detected with over 99% confidence, so a prediction with an 80% score might be questionable. In contrast, an 80% score for a JavaScript file is often a very reliable prediction.\n\nMagika manages these confidence levels in two ways:\n- **Pre-tuned Thresholds:** Each model ships with carefully tuned, per-content-type thresholds derived from evaluating the model on our large validation dataset.\n- **Prediction Modes:** Because the impact of a misidentification varies by use case, Magika allows you to select a prediction mode. This lets you balance precision (accuracy of predictions) and recall (number of identified files). The available modes are `high-confidence`, `medium-confidence`, and `best-guess`.\n\nThe `high-confidence` mode offers higher precision at the cost of lower recall. In contrast, `best-guess` provides the highest recall—potentially with lower precision—as it returns the model's prediction regardless of its confidence score. This can be selected via a command-line flag or as an option in the language bindings."
  },
  {
    "path": "website-ng/src/content/docs/core-concepts/understanding-the-output.md",
    "content": "---\ntitle: Understanding the Output\n---\n\nWhether you use the CLI or one of the language bindings (Python, Rust, JavaScript), Magika provides the same core prediction data. While many users only need the final content type label, detailed information is always available. The CLI offers flexible output formats like JSON, and the APIs provide dedicated result objects (e.g., the Python `MagikaResult` object).\n\nThe meaning of each field is best understood through an example.\n\n```shell\n$ magika tests_data/basic/javascript/code.js --json\n[\n  {\n    \"path\": \"tests_data/basic/javascript/code.js\",\n    \"result\": {\n      \"status\": \"ok\",\n      \"value\": {\n        \"dl\": {\n          \"description\": \"JavaScript source\",\n          \"extensions\": [\n            \"js\",\n            \"mjs\",\n            \"cjs\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"javascript\",\n          \"mime_type\": \"application/javascript\"\n        },\n        \"output\": {\n          \"description\": \"JavaScript source\",\n          \"extensions\": [\n            \"js\",\n            \"mjs\",\n            \"cjs\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"javascript\",\n          \"mime_type\": \"application/javascript\"\n        },\n        \"score\": 0.9710000157356262\n      }\n    }\n  }\n]\n```\n\nThis is how to interpret the output:\n- `path` is simply the file path this prediction is referring to (relevant when scanning multiple files at the same time).\n- `result.status` indicates whether magika was able to scan the sample. `ok` means all was good, in which case a `value` field is present with the details about the output.\n- `score` indicates the confidence of the prediction.\n- the `dl` block returns information about the prediction with the deep learning model. In this case, the model predicted `javascript`.\n- the `output` block returns information about the prediction of \"Magika the tool\", which, as discuss in previous sections, considers a number of aspects such as the prediction of the deep learning model, its confidence score, and the selected prediction mode. In the example above, the model's confidence was high enough to be trustworthy, and thus the output of the \"Magika the tool\" matches the content type inferred by the deep learning model.\n- the `dl` and `output` blocks contain a number of metadata about the predicted content type, such as a simple textual label suitable for automated processing (`label`), a human-readable description (`description`), MIME Type (`mime_type`), a list of extensions usually associated with the predicted content type (`extensions`), a high-level group (`group`), and a boolean that indicates whether the type is textual or not (`is_text`).\n\nHere is how to interpret the output:\n- `path`: The file path corresponding to this prediction.\n- `result.status`: `ok` indicates a successful scan. If the status is not `ok`, the `value` field will be absent.\n\nThe `value` field is present on successful scans and contains the following details:\n- `score`: The model's confidence in this prediction.\n- `dl`: Contains the raw prediction from the deep learning model.\n- `output`: Contains the final prediction from \"Magika the tool.\" This result considers the model's prediction, its confidence score, and the selected prediction mode. In this example, the model's confidence was high, so the final output matches the model's prediction.\n\nWithin both `dl` and `output`, you will find:\n- `label`: A simple, machine-readable content type label (e.g., `javascript`). The possible values for `dl.label` and `output.label` are documented in each model's README.\n- `description`: A human-readable description.\n- `mime_type`: The corresponding MIME type.\n- `group`: A high-level category (e.g., code, document, media).\n- `is_text`: A boolean indicating if the content is textual.\n- `extensions`: A list of common file extensions for this content type.\n\nAs mentioned previously, when the model is not used (e.g., for empty files), `dl.label` is set to `undefined`, and the output block will contain a generic content type like `txt` or `unknown`.\n\nFor most applications, you should use the `output.label` field, which is the default output of the CLI. The raw `dl` block is provided primarily for debugging and advanced use cases.\n\nSee also the [FAQ](/magika/additional-resources/faq) for why it is best to integrate Magika's results by focusing on label rather than other fields like `mime_type`.\n"
  },
  {
    "path": "website-ng/src/content/docs/demo/magika-demo.mdx",
    "content": "---\ntitle: \"Magika Demo\"\n---\n\nimport { Card } from '@astrojs/starlight/components';\nimport MagikaDemo from '@/components/MagikaDemo.svelte';\n\n\n"
  },
  {
    "path": "website-ng/src/content/docs/getting-started/installation.mdx",
    "content": "---\ntitle: Installation\n---\n\nimport { Tabs, TabItem } from '@astrojs/starlight/components';\n\n### Command Line Tool\n\nIf you intend to use Magika only as a command line, you can install it in a number of ways:\n\n\n  \n  ```shell\n  pipx install magika\n  ```\n  \n  \n  ```shell\n  curl -LsSf https://securityresearch.google/magika/install.sh | sh\n  ```\n  \n  \n  ```shell\n  wget -qO- https://securityresearch.google/magika/install.sh | sh\n  ```\n  \n  \n  ```shell\n  powershell -ExecutionPolicy Bypass -c \"irm https://securityresearch.google/magika/install.ps1 | iex\"\n  ```\n  \n  \n  ```shell\n  brew install magika\n  ```\n  \n\n\n\n### Python package\n\nMagika python package is available as `magika` on PyPI:\n\n```shell\npip install magika\n```\n\nIf you want to test out the latest release candidate from pypi, you can install it with\n\n```shell\npip install --pre magika\n```\n\n### JavaScript Package\n\nJavaScript / TypeScript bindings can be installed with npm:\n\n```shell\nnpm install magika\n```\n\n### Docker\n\nIf you want to test Magika's latest version (from git) within a Docker container, you can run:\n\n```shell\ngit clone https://github.com/google/magika\ncd magika/\ndocker build -t magika .\ndocker run -it --rm -v $(pwd):/magika magika -r /magika/tests_data/basic\n```"
  },
  {
    "path": "website-ng/src/content/docs/getting-started/quick-start.md",
    "content": "---\ntitle: Quick Start\n---\n\nHere you can find a number of quick examples just to get you started.\n\nTo learn about Magika's inner workings, see the [Core Concepts](/magika/core-concepts/how-magika-works) section.\n\n### Command Line Tool Examples\n\n```shell\n% cd tests_data/basic && magika -r * | head\nasm/code.asm: Assembly (code)\nbatch/simple.bat: DOS batch file (code)\nc/code.c: C source (code)\ncss/code.css: CSS source (code)\ncsv/magika_test.csv: CSV document (code)\ndockerfile/Dockerfile: Dockerfile (code)\ndocx/doc.docx: Microsoft Word 2007+ document (document)\ndocx/magika_test.docx: Microsoft Word 2007+ document (document)\neml/sample.eml: RFC 822 mail (text)\nempty/empty_file: Empty file (inode)\n```\n\n```shell\n% magika ./tests_data/basic/python/code.py --json\n[\n  {\n    \"path\": \"./tests_data/basic/python/code.py\",\n    \"result\": {\n      \"status\": \"ok\",\n      \"value\": {\n        \"dl\": {\n          \"description\": \"Python source\",\n          \"extensions\": [\n            \"py\",\n            \"pyi\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"python\",\n          \"mime_type\": \"text/x-python\"\n        },\n        \"output\": {\n          \"description\": \"Python source\",\n          \"extensions\": [\n            \"py\",\n            \"pyi\"\n          ],\n          \"group\": \"code\",\n          \"is_text\": true,\n          \"label\": \"python\",\n          \"mime_type\": \"text/x-python\"\n        },\n        \"score\": 0.996999979019165\n      }\n    }\n  }\n]\n```\n\n```shell\n% cat tests_data/basic/ini/doc.ini | magika -\n-: INI configuration file (text)\n```\n\n```shell\n% magika --help\nDetermines file content types using AI\n\nUsage: magika [OPTIONS] [PATH]...\n\nArguments:\n  [PATH]...\n          List of paths to the files to analyze.\n\n          Use a dash (-) to read from standard input (can only be used once).\n\nOptions:\n  -r, --recursive\n          Identifies files within directories instead of identifying the directory itself\n\n      --no-dereference\n          Identifies symbolic links as is instead of identifying their content by following them\n\n      --colors\n          Prints with colors regardless of terminal support\n\n      --no-colors\n          Prints without colors regardless of terminal support\n\n  -s, --output-score\n          Prints the prediction score in addition to the content type\n\n  -i, --mime-type\n          Prints the MIME type instead of the content type description\n\n  -l, --label\n          Prints a simple label instead of the content type description\n\n      --json\n          Prints in JSON format\n\n      --jsonl\n          Prints in JSONL format\n\n      --format \n          Prints using a custom format (use --help for details).\n\n          The following placeholders are supported:\n\n            %p  The file path\n            %l  The unique label identifying the content type\n            %d  The description of the content type\n            %g  The group of the content type\n            %m  The MIME type of the content type\n            %e  Possible file extensions for the content type\n            %s  The score of the content type for the file\n            %S  The score of the content type for the file in percent\n            %b  The model output if overruled (empty otherwise)\n            %%  A literal %\n\n  -h, --help\n          Print help (see a summary with '-h')\n\n  -V, --version\n          Print version\n```\n\nFor more examples and documentation about the CLI, see https://crates.io/crates/magika-cli.\n\n\n### Python Examples\n\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_bytes(b'function log(msg) {console.log(msg);}')\n>>> print(res.output.label)\njavascript\n```\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> res = m.identify_path('./tests_data/basic/ini/doc.ini')\n>>> print(res.output.label)\nini\n```\n\n```python\n>>> from magika import Magika\n>>> m = Magika()\n>>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f:\n>>>     res = m.identify_stream(f)\n>>> print(res.output.label)\nini\n```\n\nFor more examples and documentation about the Python module, see the [Python `Magika` module](/magika/cli-and-bindings/python) section."
  },
  {
    "path": "website-ng/src/content/docs/index.mdx",
    "content": "---\ntitle: Magika\ndescription: Magika landing page\ntemplate: splash\n---\n"
  },
  {
    "path": "website-ng/src/content/docs/introduction/overview.md",
    "content": "---\ntitle: Magika\n---\n\nMagika is a novel AI-powered file type detection tool that relies on the recent advance of deep learning to provide accurate detection. Under the hood, Magika employs a custom, highly optimized model that only weighs about a few MBs, and enables precise file identification within milliseconds, even when running on a single CPU. Magika has been trained and evaluated on a dataset of ~100M samples across 200+ content types (covering both binary and textual file formats), and it achieves an average ~99% accuracy on our test set.\n\nHere is an example of what Magika command line output looks like:\n\n
\n    \n
\n\nMagika is used at scale to help improve Google users' safety by routing Gmail, Drive, and Safe Browsing files to the proper security and content policy scanners, processing hundreds billions samples on a weekly basis. Magika has also been integrated with [VirusTotal](https://www.virustotal.com/) ([example](/magika/magika-vt.png)) and [abuse.ch](https://bazaar.abuse.ch/) ([example](/magika/magika-abusech.png)).\n\nFor more context you can read our initial [announcement post on Google's OSS blog](https://opensource.googleblog.com/2024/02/magika-ai-powered-fast-and-efficient-file-type-identification.html), and you can read more in our [research paper](/magika/additional-resources/research-papers-and-citation), published at the IEEE/ACM International Conference on Software Engineering (ICSE) 2025.\n\nYou can try Magika without installing anything by using our [web demo](/magika/demo/magika-demo), which runs locally in your browser!\n\n### Highlights\n\n- Available as a command line tool written in Rust, a Python API, and additional bindings for Rust, JavaScript/TypeScript (with an experimental npm package (which powers the [web demo](/magika/demo/magika-demo)), and GoLang (WIP).\n- Trained and evaluated on a dataset of ~100M files across [200+ content types](/magika/models/standard_v3_3).\n- On our test set, Magika achieves ~99% average precision and recall, outperforming existing approaches -- especially on textual content types.\n- After the model is loaded (which is a one-off overhead), the inference time is about 5ms per file, even when run on a single CPU.\n- You can invoke Magika with even thousands of files at the same time. You can also use `-r` for recursively scanning a directory.\n- Near-constant inference time, independently from the file size; Magika only uses a limited subset of the file's content.\n- Magika uses a pre-tuned, per-content-type threshold system that determines whether to \"trust\" the prediction for the model, or whether to return a generic label, such as \"Generic text document\" or \"Unknown binary data\".\n- The tolerance to errors can be controlled via different [prediction modes](/magika/core-concepts/prediction-modes).\n- Support for all major operating systems.\n- The client and the bindings are already open source, and more is coming soon!\n"
  },
  {
    "path": "website-ng/src/content/docs/models/standard_v3_3.md",
    "content": "---\ntitle: standard_v3_3 model\n---\n\nThis documents the `standard_v3_3` model.\n\n\n## List of possible outputs\n\nThis is the full list of all possible Magika's outputs (which is different than the list of possible outputs of the underlying model; see the following section for such list, and consult the [Core Concepts](/magika/core-concepts/how-magika-works/) section for more context). E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.output.label`.\n\n|      Content Type Label      | Description |\n|:-------------:|------|\n| 3gp | 3GPP multimedia file |\n| ace | ACE archive |\n| ai | Adobe Illustrator Artwork |\n| aidl | Android Interface Definition Language |\n| apk | Android package |\n| applebplist | Apple binary property list |\n| appleplist | Apple property list |\n| asm | Assembly |\n| asp | ASP source |\n| autohotkey | AutoHotKey script |\n| autoit | AutoIt script |\n| awk | Awk |\n| batch | DOS batch file |\n| bazel | Bazel build file |\n| bib | BibTeX |\n| bmp | BMP image data |\n| bzip | bzip2 compressed data |\n| c | C source |\n| cab | Microsoft Cabinet archive data |\n| cat | Windows Catalog file |\n| chm | MS Windows HtmlHelp Data |\n| clojure | Clojure |\n| cmake | CMake build file |\n| cobol | Cobol |\n| coff | Intel 80386 COFF |\n| coffeescript | CoffeeScript |\n| cpp | C++ source |\n| crt | Certificates (binary format) |\n| crx | Google Chrome extension |\n| cs | C# source |\n| csproj | .NET project config |\n| css | CSS source |\n| csv | CSV document |\n| dart | Dart source |\n| deb | Debian binary package |\n| dex | Dalvik dex file |\n| dicom | DICOM |\n| diff | Diff file |\n| directory | A directory |\n| dm | Dream Maker |\n| dmg | Apple disk image |\n| doc | Microsoft Word CDF document |\n| dockerfile | Dockerfile |\n| docx | Microsoft Word 2007+ document |\n| dsstore | Application Desktop Services Store |\n| dwg | Autocad Drawing |\n| dxf | Audocad Drawing Exchange Format |\n| elf | ELF executable |\n| elixir | Elixir script |\n| emf | Windows Enhanced Metafile image data |\n| eml | RFC 822 mail |\n| empty | Empty file |\n| epub | EPUB document |\n| erb | Embedded Ruby source |\n| erlang | Erlang source |\n| flac | FLAC audio bitstream data |\n| flv | Flash Video |\n| fortran | Fortran |\n| gemfile | Gemfile file |\n| gemspec | Gemspec file |\n| gif | GIF image data |\n| gitattributes | Gitattributes file |\n| gitmodules | Gitmodules file |\n| go | Golang source |\n| gradle | Gradle source |\n| groovy | Groovy source |\n| gzip | gzip compressed data |\n| h5 | Hierarchical Data Format v5 |\n| handlebars | Handlebars source |\n| haskell | Haskell source |\n| hcl | HashiCorp configuration language |\n| hlp | MS Windows help |\n| htaccess | Apache access configuration |\n| html | HTML document |\n| icns | Mac OS X icon |\n| ico | MS Windows icon resource |\n| ics | Internet Calendaring and Scheduling |\n| ignorefile | Ignorefile |\n| ini | INI configuration file |\n| internetshortcut | MS Windows Internet shortcut |\n| ipynb | Jupyter notebook |\n| iso | ISO 9660 CD-ROM filesystem data |\n| jar | Java archive data (JAR) |\n| java | Java source |\n| javabytecode | Java compiled bytecode |\n| javascript | JavaScript source |\n| jinja | Jinja template |\n| jp2 | jpeg2000 |\n| jpeg | JPEG image data |\n| json | JSON document |\n| jsonl | JSONL document |\n| julia | Julia source |\n| kotlin | Kotlin source |\n| latex | LaTeX document |\n| lha | LHarc archive |\n| lisp | Lisp source |\n| lnk | MS Windows shortcut |\n| lua | Lua |\n| m3u | M3U playlist |\n| m4 | GNU Macro |\n| macho | Mach-O executable |\n| makefile | Makefile source |\n| markdown | Markdown document |\n| matlab | Matlab Source |\n| mht | MHTML document |\n| midi | Midi |\n| mkv | Matroska |\n| mp3 | MP3 media file |\n| mp4 | MP4 media file |\n| mscompress | MS Compress archive data |\n| msi | Microsoft Installer file |\n| mum | Windows Update Package file |\n| npy | Numpy Array |\n| npz | Numpy Arrays Archive |\n| nupkg | NuGet Package |\n| objectivec | ObjectiveC source |\n| ocaml | OCaml |\n| odp | OpenDocument Presentation |\n| ods | OpenDocument Spreadsheet |\n| odt | OpenDocument Text |\n| ogg | Ogg data |\n| one | One Note |\n| onnx | Open Neural Network Exchange |\n| otf | OpenType font |\n| outlook | MS Outlook Message |\n| parquet | Apache Parquet |\n| pascal | Pascal source |\n| pcap | pcap capture file |\n| pdb | Windows Program Database |\n| pdf | PDF document |\n| pebin | PE Windows executable |\n| pem | PEM certificate |\n| perl | Perl source |\n| php | PHP source |\n| pickle | Python pickle |\n| png | PNG image |\n| po | Portable Object (PO) for i18n |\n| postscript | PostScript document |\n| powershell | Powershell source |\n| ppt | Microsoft PowerPoint CDF document |\n| pptx | Microsoft PowerPoint 2007+ document |\n| prolog | Prolog source |\n| proteindb | Protein DB |\n| proto | Protocol buffer definition |\n| psd | Adobe Photoshop |\n| python | Python source |\n| pythonbytecode | Python compiled bytecode |\n| pytorch | Pytorch storage file |\n| qt | QuickTime |\n| r | R (language) |\n| rar | RAR archive data |\n| rdf | Resource Description Framework document (RDF) |\n| rpm | RedHat Package Manager archive (RPM) |\n| rst | ReStructuredText document |\n| rtf | Rich Text Format document |\n| ruby | Ruby source |\n| rust | Rust source |\n| scala | Scala source |\n| scss | SCSS source |\n| sevenzip | 7-zip archive data |\n| sgml | sgml |\n| shell | Shell script |\n| smali | Smali source |\n| snap | Snap archive |\n| solidity | Solidity source |\n| sql | SQL source |\n| sqlite | SQLITE database |\n| squashfs | Squash filesystem |\n| srt | SubRip Text Format |\n| stlbinary | Stereolithography CAD (binary) |\n| stltext | Stereolithography CAD (text) |\n| sum | Checksum file |\n| svg | SVG Scalable Vector Graphics image data |\n| swf | Small Web File |\n| swift | Swift |\n| symlink | Symbolic link |\n| tar | POSIX tar archive |\n| tcl | Tickle |\n| textproto | Text protocol buffer |\n| tga | Targa image data |\n| thumbsdb | Windows thumbnail cache |\n| tiff | TIFF image data |\n| toml | Tom's obvious, minimal language |\n| torrent | BitTorrent file |\n| tsv | TSV document |\n| ttf | TrueType Font data |\n| twig | Twig template |\n| txt | Generic text document |\n| typescript | TypeScript source |\n| unknown | Unknown binary data |\n| vba | MS Visual Basic source (VBA) |\n| vcxproj | Visual Studio MSBuild project |\n| verilog | Verilog source |\n| vhdl | VHDL source |\n| vtt | Web Video Text Tracks |\n| vue | Vue source |\n| wasm | Web Assembly |\n| wav | Waveform Audio file (WAV) |\n| webm | WebM media file |\n| webp | WebP media file |\n| winregistry | Windows Registry text |\n| wmf | Windows metafile |\n| woff | Web Open Font Format |\n| woff2 | Web Open Font Format v2 |\n| xar | XAR archive compressed data |\n| xls | Microsoft Excel CDF document |\n| xlsb | Microsoft Excel 2007+ document (binary format) |\n| xlsx | Microsoft Excel 2007+ document |\n| xml | XML document |\n| xpi | Compressed installation archive (XPI) |\n| xz | XZ compressed data |\n| yaml | YAML source |\n| yara | YARA rule |\n| zig | Zig source |\n| zip | Zip archive data |\n| zlibstream | zlib compressed data |\n\n\n## List of possible model's outputs\n\nThis is the full list of all possible model's output. E.g., this is the list of all possible values for Magika python module's `MagikaResult.prediction.dl.label`. Note that, in general, the list of \"model outputs\" is different than the \"tool outputs\" as in some cases the model is not even used. Consult the [Core Concepts](/magika/core-concepts/how-magika-works/) section for more context. This list is useful mostly for debugging purposes; the vast majority of clients should just consult the table in the section above.\n\n|      Content Type Label      | Description |\n|:-------------:|------|\n| 3gp | 3GPP multimedia file |\n| ace | ACE archive |\n| ai | Adobe Illustrator Artwork |\n| aidl | Android Interface Definition Language |\n| apk | Android package |\n| applebplist | Apple binary property list |\n| appleplist | Apple property list |\n| asm | Assembly |\n| asp | ASP source |\n| autohotkey | AutoHotKey script |\n| autoit | AutoIt script |\n| awk | Awk |\n| batch | DOS batch file |\n| bazel | Bazel build file |\n| bib | BibTeX |\n| bmp | BMP image data |\n| bzip | bzip2 compressed data |\n| c | C source |\n| cab | Microsoft Cabinet archive data |\n| cat | Windows Catalog file |\n| chm | MS Windows HtmlHelp Data |\n| clojure | Clojure |\n| cmake | CMake build file |\n| cobol | Cobol |\n| coff | Intel 80386 COFF |\n| coffeescript | CoffeeScript |\n| cpp | C++ source |\n| crt | Certificates (binary format) |\n| crx | Google Chrome extension |\n| cs | C# source |\n| csproj | .NET project config |\n| css | CSS source |\n| csv | CSV document |\n| dart | Dart source |\n| deb | Debian binary package |\n| dex | Dalvik dex file |\n| dicom | DICOM |\n| diff | Diff file |\n| dm | Dream Maker |\n| dmg | Apple disk image |\n| doc | Microsoft Word CDF document |\n| dockerfile | Dockerfile |\n| docx | Microsoft Word 2007+ document |\n| dsstore | Application Desktop Services Store |\n| dwg | Autocad Drawing |\n| dxf | Audocad Drawing Exchange Format |\n| elf | ELF executable |\n| elixir | Elixir script |\n| emf | Windows Enhanced Metafile image data |\n| eml | RFC 822 mail |\n| epub | EPUB document |\n| erb | Embedded Ruby source |\n| erlang | Erlang source |\n| flac | FLAC audio bitstream data |\n| flv | Flash Video |\n| fortran | Fortran |\n| gemfile | Gemfile file |\n| gemspec | Gemspec file |\n| gif | GIF image data |\n| gitattributes | Gitattributes file |\n| gitmodules | Gitmodules file |\n| go | Golang source |\n| gradle | Gradle source |\n| groovy | Groovy source |\n| gzip | gzip compressed data |\n| h5 | Hierarchical Data Format v5 |\n| handlebars | Handlebars source |\n| haskell | Haskell source |\n| hcl | HashiCorp configuration language |\n| hlp | MS Windows help |\n| htaccess | Apache access configuration |\n| html | HTML document |\n| icns | Mac OS X icon |\n| ico | MS Windows icon resource |\n| ics | Internet Calendaring and Scheduling |\n| ignorefile | Ignorefile |\n| ini | INI configuration file |\n| internetshortcut | MS Windows Internet shortcut |\n| ipynb | Jupyter notebook |\n| iso | ISO 9660 CD-ROM filesystem data |\n| jar | Java archive data (JAR) |\n| java | Java source |\n| javabytecode | Java compiled bytecode |\n| javascript | JavaScript source |\n| jinja | Jinja template |\n| jp2 | jpeg2000 |\n| jpeg | JPEG image data |\n| json | JSON document |\n| jsonl | JSONL document |\n| julia | Julia source |\n| kotlin | Kotlin source |\n| latex | LaTeX document |\n| lha | LHarc archive |\n| lisp | Lisp source |\n| lnk | MS Windows shortcut |\n| lua | Lua |\n| m3u | M3U playlist |\n| m4 | GNU Macro |\n| macho | Mach-O executable |\n| makefile | Makefile source |\n| markdown | Markdown document |\n| matlab | Matlab Source |\n| mht | MHTML document |\n| midi | Midi |\n| mkv | Matroska |\n| mp3 | MP3 media file |\n| mp4 | MP4 media file |\n| mscompress | MS Compress archive data |\n| msi | Microsoft Installer file |\n| mum | Windows Update Package file |\n| npy | Numpy Array |\n| npz | Numpy Arrays Archive |\n| nupkg | NuGet Package |\n| objectivec | ObjectiveC source |\n| ocaml | OCaml |\n| odp | OpenDocument Presentation |\n| ods | OpenDocument Spreadsheet |\n| odt | OpenDocument Text |\n| ogg | Ogg data |\n| one | One Note |\n| onnx | Open Neural Network Exchange |\n| otf | OpenType font |\n| outlook | MS Outlook Message |\n| parquet | Apache Parquet |\n| pascal | Pascal source |\n| pcap | pcap capture file |\n| pdb | Windows Program Database |\n| pdf | PDF document |\n| pebin | PE Windows executable |\n| pem | PEM certificate |\n| perl | Perl source |\n| php | PHP source |\n| pickle | Python pickle |\n| png | PNG image |\n| po | Portable Object (PO) for i18n |\n| postscript | PostScript document |\n| powershell | Powershell source |\n| ppt | Microsoft PowerPoint CDF document |\n| pptx | Microsoft PowerPoint 2007+ document |\n| prolog | Prolog source |\n| proteindb | Protein DB |\n| proto | Protocol buffer definition |\n| psd | Adobe Photoshop |\n| python | Python source |\n| pythonbytecode | Python compiled bytecode |\n| pytorch | Pytorch storage file |\n| qt | QuickTime |\n| r | R (language) |\n| randombytes | Random bytes |\n| randomtxt | Random text |\n| rar | RAR archive data |\n| rdf | Resource Description Framework document (RDF) |\n| rpm | RedHat Package Manager archive (RPM) |\n| rst | ReStructuredText document |\n| rtf | Rich Text Format document |\n| ruby | Ruby source |\n| rust | Rust source |\n| scala | Scala source |\n| scss | SCSS source |\n| sevenzip | 7-zip archive data |\n| sgml | sgml |\n| shell | Shell script |\n| smali | Smali source |\n| snap | Snap archive |\n| solidity | Solidity source |\n| sql | SQL source |\n| sqlite | SQLITE database |\n| squashfs | Squash filesystem |\n| srt | SubRip Text Format |\n| stlbinary | Stereolithography CAD (binary) |\n| stltext | Stereolithography CAD (text) |\n| sum | Checksum file |\n| svg | SVG Scalable Vector Graphics image data |\n| swf | Small Web File |\n| swift | Swift |\n| tar | POSIX tar archive |\n| tcl | Tickle |\n| textproto | Text protocol buffer |\n| tga | Targa image data |\n| thumbsdb | Windows thumbnail cache |\n| tiff | TIFF image data |\n| toml | Tom's obvious, minimal language |\n| torrent | BitTorrent file |\n| tsv | TSV document |\n| ttf | TrueType Font data |\n| twig | Twig template |\n| txt | Generic text document |\n| typescript | TypeScript source |\n| undefined | Undefined |\n| vba | MS Visual Basic source (VBA) |\n| vcxproj | Visual Studio MSBuild project |\n| verilog | Verilog source |\n| vhdl | VHDL source |\n| vtt | Web Video Text Tracks |\n| vue | Vue source |\n| wasm | Web Assembly |\n| wav | Waveform Audio file (WAV) |\n| webm | WebM media file |\n| webp | WebP media file |\n| winregistry | Windows Registry text |\n| wmf | Windows metafile |\n| woff | Web Open Font Format |\n| woff2 | Web Open Font Format v2 |\n| xar | XAR archive compressed data |\n| xls | Microsoft Excel CDF document |\n| xlsb | Microsoft Excel 2007+ document (binary format) |\n| xlsx | Microsoft Excel 2007+ document |\n| xml | XML document |\n| xpi | Compressed installation archive (XPI) |\n| xz | XZ compressed data |\n| yaml | YAML source |\n| yara | YARA rule |\n| zig | Zig source |\n| zip | Zip archive data |\n| zlibstream | zlib compressed data |"
  },
  {
    "path": "website-ng/src/content.config.ts",
    "content": "import { defineCollection } from 'astro:content';\nimport { docsLoader } from '@astrojs/starlight/loaders';\nimport { docsSchema } from '@astrojs/starlight/schema';\n\nexport const collections = {\n\tdocs: defineCollection({ loader: docsLoader(), schema: docsSchema() }),\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/button/button.svelte",
    "content": "\n\n\n\n{#if href}\n\t\n\t\t{@render children?.()}\n\t\n{:else}\n\t\n\t\t{@render children?.()}\n\t\n{/if}\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/button/index.ts",
    "content": "import Root, {\n\ttype ButtonProps,\n\ttype ButtonSize,\n\ttype ButtonVariant,\n\tbuttonVariants,\n} from \"./button.svelte\";\n\nexport {\n\tRoot,\n\ttype ButtonProps as Props,\n\t//\n\tRoot as Button,\n\tbuttonVariants,\n\ttype ButtonProps,\n\ttype ButtonSize,\n\ttype ButtonVariant,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card-action.svelte",
    "content": "\n\n\n\t{@render children?.()}\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card-content.svelte",
    "content": "\n\n
\n\t{@render children?.()}\n
\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card-description.svelte",
    "content": "\n\n\n\t{@render children?.()}\n
\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card-footer.svelte",
    "content": "\n\n\n\t{@render children?.()}\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card-header.svelte",
    "content": "\n\n\n\t{@render children?.()}\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card-title.svelte",
    "content": "\n\n\n\t{@render children?.()}\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/card.svelte",
    "content": "\n\n\n\t{@render children?.()}\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/card/index.ts",
    "content": "import Root from \"./card.svelte\";\nimport Content from \"./card-content.svelte\";\nimport Description from \"./card-description.svelte\";\nimport Footer from \"./card-footer.svelte\";\nimport Header from \"./card-header.svelte\";\nimport Title from \"./card-title.svelte\";\nimport Action from \"./card-action.svelte\";\n\nexport {\n\tRoot,\n\tContent,\n\tDescription,\n\tFooter,\n\tHeader,\n\tTitle,\n\tAction,\n\t//\n\tRoot as Card,\n\tContent as CardContent,\n\tDescription as CardDescription,\n\tFooter as CardFooter,\n\tHeader as CardHeader,\n\tTitle as CardTitle,\n\tAction as CardAction,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/file-drop-zone/file-drop-zone.svelte",
    "content": "\n\n\n\n e.preventDefault()}\n\tondrop={drop}\n\tfor={id}\n\taria-disabled={!canUploadFiles}\n\tclass={cn(\n\t\t'border-border hover:bg-accent/25 flex h-48 w-full place-items-center justify-center rounded-lg border-2 border-dashed p-6 transition-all hover:cursor-pointer aria-disabled:opacity-50 aria-disabled:hover:cursor-not-allowed',\n\t\tclassName\n\t)}\n>\n\t{#if children}\n\t\t{@render children()}\n\t{:else}\n\t\t
\n\t\t\t\n\t\t\t\t\n\t\t\t
\n\t\t\t
\n\t\t\t\t\n\t\t\t\t\tDrag 'n' drop files here, or click to select files\n\t\t\t\t\n\t\t\t\t{#if maxFiles || maxFileSize}\n\t\t\t\t\t\n\t\t\t\t\t\t{#if maxFiles}\n\t\t\t\t\t\t\tYou can upload {maxFiles} files\n\t\t\t\t\t\t{/if}\n\t\t\t\t\t\t{#if maxFiles && maxFileSize}\n\t\t\t\t\t\t\t(up to {displaySize(maxFileSize)} each)\n\t\t\t\t\t\t{/if}\n\t\t\t\t\t\t{#if maxFileSize && !maxFiles}\n\t\t\t\t\t\t\tMaximum size {displaySize(maxFileSize)}\n\t\t\t\t\t\t{/if}\n\t\t\t\t\t\n\t\t\t\t{/if}\n\t\t\t
\n\t\t\n\t{/if}\n\t 1}\n\t\ttype=\"file\"\n\t\tonchange={change}\n\t\tclass=\"hidden\"\n\t/>\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/file-drop-zone/index.ts",
    "content": "/*\n\tInstalled from @ieedan/shadcn-svelte-extras\n*/\n\nimport FileDropZone from \"./file-drop-zone.svelte\";\nimport { type FileRejectedReason, type FileDropZoneProps } from \"./types\";\n\nexport const displaySize = (bytes: number): string => {\n  if (bytes < KILOBYTE) return `${bytes.toFixed(0)} B`;\n\n  if (bytes < MEGABYTE) return `${(bytes / KILOBYTE).toFixed(0)} KB`;\n\n  if (bytes < GIGABYTE) return `${(bytes / MEGABYTE).toFixed(0)} MB`;\n\n  return `${(bytes / GIGABYTE).toFixed(0)} GB`;\n};\n\n// Utilities for working with file sizes\nexport const BYTE = 1;\nexport const KILOBYTE = 1024;\nexport const MEGABYTE = 1024 * KILOBYTE;\nexport const GIGABYTE = 1024 * MEGABYTE;\n\n// utilities for limiting accepted files\nexport const ACCEPT_IMAGE = \"image/*\";\nexport const ACCEPT_VIDEO = \"video/*\";\nexport const ACCEPT_AUDIO = \"audio/*\";\n\nexport { FileDropZone, type FileRejectedReason, type FileDropZoneProps };\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/file-drop-zone/types.ts",
    "content": "/*\n\tInstalled from @ieedan/shadcn-svelte-extras\n*/\n\nimport type { WithChildren } from \"bits-ui\";\nimport type { HTMLInputAttributes } from \"svelte/elements\";\n\nexport type FileRejectedReason =\n  | \"Maximum file size exceeded\"\n  | \"File type not allowed\"\n  | \"Maximum files uploaded\";\n\nexport type FileDropZonePropsWithoutHTML = WithChildren<{\n  ref?: HTMLInputElement | null;\n  /** Called with the uploaded files when the user drops or clicks and selects their files.\n   *\n   * @param files\n   */\n  onUpload: (files: File[]) => Promise;\n  /** The maximum amount files allowed to be uploaded */\n  maxFiles?: number;\n  fileCount?: number;\n  /** The maximum size of a file in bytes */\n  maxFileSize?: number;\n  /** Called when a file does not meet the upload criteria (size, or type) */\n  onFileRejected?: (opts: { reason: FileRejectedReason; file: File }) => void;\n\n  // just for extra documentation\n  /** Takes a comma separated list of one or more file types.\n   *\n   *  [MDN Reference](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/accept)\n   *\n   * ### Usage\n   * ```svelte\n   * \n   * ```\n   *\n   * ### Common Values\n   * ```svelte\n   * \n   * \n   * \n   * ```\n   */\n  accept?: string;\n}>;\n\nexport type FileDropZoneProps = FileDropZonePropsWithoutHTML &\n  Omit;\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/input/index.ts",
    "content": "import Root from \"./input.svelte\";\n\nexport {\n\tRoot,\n\t//\n\tRoot as Input,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/input/input.svelte",
    "content": "\n\n{#if type === \"file\"}\n\t\n{:else}\n\t\n{/if}\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/label/index.ts",
    "content": "import Root from \"./label.svelte\";\n\nexport {\n\tRoot,\n\t//\n\tRoot as Label,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/label/label.svelte",
    "content": "\n\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/progress/index.ts",
    "content": "import Root from \"./progress.svelte\";\n\nexport {\n\tRoot,\n\t//\n\tRoot as Progress,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/progress/progress.svelte",
    "content": "\n\n\n  \n  {#if showPercentage}\n    \n      {percentage}%\n    \n  {/if}\n\n\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/tabs/index.ts",
    "content": "import Root from \"./tabs.svelte\";\nimport Content from \"./tabs-content.svelte\";\nimport List from \"./tabs-list.svelte\";\nimport Trigger from \"./tabs-trigger.svelte\";\n\nexport {\n\tRoot,\n\tContent,\n\tList,\n\tTrigger,\n\t//\n\tRoot as Tabs,\n\tContent as TabsContent,\n\tList as TabsList,\n\tTrigger as TabsTrigger,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/tabs/tabs-content.svelte",
    "content": "\n\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/tabs/tabs-list.svelte",
    "content": "\n\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/tabs/tabs-trigger.svelte",
    "content": "\n\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/tabs/tabs.svelte",
    "content": "\n\n\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/textarea/index.ts",
    "content": "import Root from \"./textarea.svelte\";\n\nexport {\n\tRoot,\n\t//\n\tRoot as Textarea,\n};\n"
  },
  {
    "path": "website-ng/src/lib/components/ui/textarea/textarea.svelte",
    "content": "\n\n\n"
  },
  {
    "path": "website-ng/src/lib/utils/utils.ts",
    "content": "/*\n\tInstalled from @ieedan/shadcn-svelte-extras\n*/\n\nimport { type ClassValue, clsx } from \"clsx\";\nimport { twMerge } from \"tailwind-merge\";\n\nexport function cn(...inputs: ClassValue[]) {\n  return twMerge(clsx(inputs));\n}\n\n// eslint-disable-next-line @typescript-eslint/no-explicit-any\nexport type WithoutChild = T extends { child?: any } ? Omit : T;\n// eslint-disable-next-line @typescript-eslint/no-explicit-any\nexport type WithoutChildren = T extends { children?: any }\n  ? Omit\n  : T;\nexport type WithoutChildrenOrChild = WithoutChildren>;\nexport type WithElementRef = T & {\n  ref?: U | null;\n};\n"
  },
  {
    "path": "website-ng/src/lib/utils.ts",
    "content": "import { clsx, type ClassValue } from \"clsx\";\nimport { twMerge } from \"tailwind-merge\";\n\nexport function cn(...inputs: ClassValue[]) {\n\treturn twMerge(clsx(inputs));\n}\n\n// eslint-disable-next-line @typescript-eslint/no-explicit-any\nexport type WithoutChild = T extends { child?: any } ? Omit : T;\n// eslint-disable-next-line @typescript-eslint/no-explicit-any\nexport type WithoutChildren = T extends { children?: any } ? Omit : T;\nexport type WithoutChildrenOrChild = WithoutChildren>;\nexport type WithElementRef = T & { ref?: U | null };\n"
  },
  {
    "path": "website-ng/src/pages/install.ps1.ts",
    "content": "import type { APIContext } from \"astro\";\n// We don't want a  redirection page, so we disable prerendering.\nexport const prerender = false;\n\nexport async function GET({ redirect }: APIContext) {\n  console.log(\"Redirecting to latest installer script\");\n  return redirect(\n    \"https://github.com/google/magika/releases/download/cli-latest/magika-installer.ps1\",\n    302\n  );\n}\n"
  },
  {
    "path": "website-ng/src/pages/install.sh.ts",
    "content": "import type { APIContext } from \"astro\";\n// We don't want a  redirection page, so we disable prerendering.\nexport const prerender = false;\n\nexport async function GET({ redirect }: APIContext) {\n  console.log(\"Redirecting to latest installer script\");\n  return redirect(\n    \"https://github.com/google/magika/releases/download/cli-latest/magika-installer.sh\",\n    302\n  );\n}\n"
  },
  {
    "path": "website-ng/src/styles/global.css",
    "content": "/* Custom fonts */\n@import url(\"https://fonts.googleapis.com/css?family=Google+Sans+Text:300,400,700&display=swap\");\n@import url(\"https://fonts.googleapis.com/css?family=Google+Sans&display=swap\");\n\n/* Starlight Global Styles */\n@import \"@astrojs/starlight-tailwind\";\n@import \"tailwindcss\";\n@import \"tailwindcss/theme.css\" layer(theme);\n@import \"tailwindcss/utilities.css\" layer(utilities);\n\n/* Shadcn UI + Tailwind CSS + Animate.css */\n@import \"tw-animate-css\";\n\n/* Shadcn UI Theming */\n\n@custom-variant dark (&:is(.dark *));\n\n:root {\n  --radius: 0.625rem;\n  --background: oklch(1 0 0);\n  --foreground: oklch(0.145 0 0);\n  --card: oklch(1 0 0);\n  --card-foreground: oklch(0.145 0 0);\n  --popover: oklch(1 0 0);\n  --popover-foreground: oklch(0.145 0 0);\n  --primary: oklch(0.205 0 0);\n  --primary-foreground: oklch(0.985 0 0);\n  --secondary: oklch(0.97 0 0);\n  --secondary-foreground: oklch(0.205 0 0);\n  --muted: oklch(0.97 0 0);\n  --muted-foreground: oklch(0.556 0 0);\n  --accent: oklch(0.97 0 0);\n  --accent-foreground: oklch(0.205 0 0);\n  --destructive: oklch(0.577 0.245 27.325);\n  --border: oklch(0.922 0 0);\n  --input: oklch(0.922 0 0);\n  --ring: oklch(0.708 0 0);\n  --chart-1: oklch(0.646 0.222 41.116);\n  --chart-2: oklch(0.6 0.118 184.704);\n  --chart-3: oklch(0.398 0.07 227.392);\n  --chart-4: oklch(0.828 0.189 84.429);\n  --chart-5: oklch(0.769 0.188 70.08);\n  --sidebar: oklch(0.985 0 0);\n  --sidebar-foreground: oklch(0.145 0 0);\n  --sidebar-primary: oklch(0.205 0 0);\n  --sidebar-primary-foreground: oklch(0.985 0 0);\n  --sidebar-accent: oklch(0.97 0 0);\n  --sidebar-accent-foreground: oklch(0.205 0 0);\n  --sidebar-border: oklch(0.922 0 0);\n  --sidebar-ring: oklch(0.708 0 0);\n}\n\n.dark {\n  --background: oklch(0.145 0 0);\n  --foreground: oklch(0.985 0 0);\n  --card: oklch(0.205 0 0);\n  --card-foreground: oklch(0.985 0 0);\n  --popover: oklch(0.205 0 0);\n  --popover-foreground: oklch(0.985 0 0);\n  --primary: oklch(0.922 0 0);\n  --primary-foreground: oklch(0.205 0 0);\n  --secondary: oklch(0.269 0 0);\n  --secondary-foreground: oklch(0.985 0 0);\n  --muted: oklch(0.269 0 0);\n  --muted-foreground: oklch(0.708 0 0);\n  --accent: oklch(0.269 0 0);\n  --accent-foreground: oklch(0.985 0 0);\n  --destructive: oklch(0.704 0.191 22.216);\n  --border: oklch(1 0 0 / 10%);\n  --input: oklch(1 0 0 / 15%);\n  --ring: oklch(0.556 0 0);\n  --chart-1: oklch(0.488 0.243 264.376);\n  --chart-2: oklch(0.696 0.17 162.48);\n  --chart-3: oklch(0.769 0.188 70.08);\n  --chart-4: oklch(0.627 0.265 303.9);\n  --chart-5: oklch(0.645 0.246 16.439);\n  --sidebar: oklch(0.205 0 0);\n  --sidebar-foreground: oklch(0.985 0 0);\n  --sidebar-primary: oklch(0.488 0.243 264.376);\n  --sidebar-primary-foreground: oklch(0.985 0 0);\n  --sidebar-accent: oklch(0.269 0 0);\n  --sidebar-accent-foreground: oklch(0.985 0 0);\n  --sidebar-border: oklch(1 0 0 / 10%);\n  --sidebar-ring: oklch(0.556 0 0);\n}\n\n@theme inline {\n  --radius-sm: calc(var(--radius) - 4px);\n  --radius-md: calc(var(--radius) - 2px);\n  --radius-lg: var(--radius);\n  --radius-xl: calc(var(--radius) + 4px);\n  --color-background: var(--background);\n  --color-foreground: var(--foreground);\n  --color-card: var(--card);\n  --color-card-foreground: var(--card-foreground);\n  --color-popover: var(--popover);\n  --color-popover-foreground: var(--popover-foreground);\n  --color-primary: var(--primary);\n  --color-primary-foreground: var(--primary-foreground);\n  --color-secondary: var(--secondary);\n  --color-secondary-foreground: var(--secondary-foreground);\n  --color-muted: var(--muted);\n  --color-muted-foreground: var(--muted-foreground);\n  --color-accent: var(--accent);\n  --color-accent-foreground: var(--accent-foreground);\n  --color-destructive: var(--destructive);\n  --color-border: var(--border);\n  --color-input: var(--input);\n  --color-ring: var(--ring);\n  --color-chart-1: var(--chart-1);\n  --color-chart-2: var(--chart-2);\n  --color-chart-3: var(--chart-3);\n  --color-chart-4: var(--chart-4);\n  --color-chart-5: var(--chart-5);\n  --color-sidebar: var(--sidebar);\n  --color-sidebar-foreground: var(--sidebar-foreground);\n  --color-sidebar-primary: var(--sidebar-primary);\n  --color-sidebar-primary-foreground: var(--sidebar-primary-foreground);\n  --color-sidebar-accent: var(--sidebar-accent);\n  --color-sidebar-accent-foreground: var(--sidebar-accent-foreground);\n  --color-sidebar-border: var(--sidebar-border);\n  --color-sidebar-ring: var(--sidebar-ring);\n}\n\n@layer base {\n  * {\n    @apply border-border outline-ring/50;\n  }\n  body {\n    @apply bg-background text-foreground;\n  }\n}\n\n@layer base, starlight, theme, components, utilities;\n\n/* Show lists correctly */\n.sl-container ul {\n  list-style: disc;\n}\n\n.sl-container ol {\n  list-style: decimal;\n}\n\n.sl-container starlight-toc nav ul {\n  list-style: none;\n}\n"
  },
  {
    "path": "website-ng/svelte.config.js",
    "content": "export default {\n\tcompilerOptions: {\n\t\texperimental: {\n\t\t\tasync: true\n\t\t}\n\t}\n};"
  },
  {
    "path": "website-ng/tsconfig.json",
    "content": "{\n  \"extends\": \"astro/tsconfigs/strict\",\n  \"include\": [\n    \".astro/types.d.ts\",\n    \"**/*\"\n  ],\n  \"exclude\": [\n    \"dist\"\n  ],\n  \"compilerOptions\": {\n    \"jsx\": \"preserve\",\n    \"baseUrl\": \"./\",\n    \"paths\": {\n      \"@/*\": [\"src/*\"]\n    }\n  }\n}"
  }
]