[ { "path": ".codeclimate.yml", "content": "ratings:\n paths: \n - \"**/*.go\"\n\nexclude_paths:\n- test/\n- Godeps/\n- thirdparty/\n- \"**/*.pb.go\"\n\nengines:\n fixme:\n enabled: true\n config:\n strings:\n - FIXME\n - HACK\n - XXX\n - BUG\n golint:\n enabled: true\n govet:\n enabled: true\n gofmt:\n enabled: true\n\nversion: \"2\"\nchecks:\n argument-count:\n enabled: false\n complex-logic:\n enabled: false\n file-lines:\n enabled: false\n method-complexity:\n enabled: false\n method-count:\n enabled: false\n method-lines:\n enabled: false\n nested-control-flow:\n enabled: false\n return-statements:\n enabled: false\n similar-code:\n enabled: false\n" }, { "path": ".cspell.yml", "content": "ignoreWords:\n - childs # This spelling is used in the files command\n - NodeCreater # This spelling is used in the fuse dependency\n - Boddy # One of the contributors to the project - Chris Boddy\n - Botto # One of the contributors to the project - Santiago Botto\n - cose # dag-cose" }, { "path": ".dockerignore", "content": "Dockerfile\nDockerfile.fast\n.git/\n!.git/HEAD\n!.git/refs/\n!.git/packed-refs\ntest/sharness/lib/sharness/\ntest/sharness/trash*\nrb-pinning-service-api/\n\n# The Docker client might not be running on Linux\n# so delete any compiled binaries\nbin/gx\nbin/gx*\nbin/tmp\n" }, { "path": ".gitattributes", "content": "# Default to text\n* text eol=lf\n\n# True text\n*.md text eol=auto\nLICENSE text eol=auto\n\n# Known binary types\n*.png binary\n*.tar binary\n*.gz binary\n*.xz binary\n*.car binary\n\n# Binary assets\nassets/init-doc/* binary\ncore/coreunix/test_data/** binary\ntest/cli/migrations/testdata/** binary\n\n# Generated test data\ntest/cli/migrations/testdata/** linguist-generated=true\ntest/cli/autoconf/testdata/** linguist-generated=true\ntest/cli/fixtures/** linguist-generated=true\ntest/sharness/t0054-dag-car-import-export-data/** linguist-generated=true\ntest/sharness/t0109-gateway-web-_redirects-data/** linguist-generated=true\ntest/sharness/t0114-gateway-subdomains/** linguist-generated=true\ntest/sharness/t0115-gateway-dir-listing/** linguist-generated=true\ntest/sharness/t0116-gateway-cache/** linguist-generated=true\ntest/sharness/t0119-prometheus-data/** linguist-generated=true\ntest/sharness/t0165-keystore-data/** linguist-generated=true\ntest/sharness/t0275-cid-security-data/** linguist-generated=true\ntest/sharness/t0280-plugin-dag-jose-data/** linguist-generated=true\ntest/sharness/t0280-plugin-data/** linguist-generated=true\ntest/sharness/t0280-plugin-git-data/** linguist-generated=true\ntest/sharness/t0400-api-no-gateway/** linguist-generated=true\ntest/sharness/t0701-delegated-routing-reframe/** linguist-generated=true\ntest/sharness/t0702-delegated-routing-http/** linguist-generated=true\n" }, { "path": ".github/CODEOWNERS", "content": "# Code owners are automatically requested for review when someone opens a pull\n# request that modifies code that they own. Code owners are not automatically\n# requested to review draft pull requests.\n\n# Default\n* @ipfs/kubo-maintainers \n\n# HTTP Gateway\ncore/corehttp/ @lidel\ntest/sharness/*gateway*.sh @lidel\n" }, { "path": ".github/FUNDING.yml", "content": "custom: [ipshipyard.gitwallet.co]\n" }, { "path": ".github/ISSUE_TEMPLATE/bug-report.yml", "content": "name: Bug Report\ndescription: Report a bug in Kubo.\nlabels:\n - kind/bug\n - need/triage\nbody:\n - type: markdown\n attributes:\n value: |\n - Make sure you are running the [latest version of Kubo][releases] before reporting an issue.\n - If you have an enhancement or feature request for Kubo, please select [a different option][issues].\n - Please report possible security issues by email to security@ipfs.io\n\n [issues]: https://github.com/ipfs/kubo/issues/new/choose\n [releases]: https://github.com/ipfs/kubo/releases\n - type: checkboxes\n attributes:\n label: Checklist\n description: Please verify that you've followed these steps\n options:\n - label: This is a bug report, not a question. Ask questions on [discuss.ipfs.tech](https://discuss.ipfs.tech/c/help/13).\n required: true\n - label: I have searched on the [issue tracker](https://github.com/ipfs/kubo/issues?q=is%3Aissue) for my bug.\n required: true\n - label: I am running the latest [kubo version](https://dist.ipfs.tech/#kubo) or have an issue updating.\n required: true\n - type: dropdown\n id: install\n validations:\n required: true\n attributes:\n label: Installation method\n description: Please select your installation method\n options:\n - dist.ipfs.tech or ipfs-update\n - docker image\n - ipfs-desktop\n - third-party binary\n - built from source\n - type: textarea\n id: version\n attributes:\n label: Version\n render: Text\n description: |\n Enter the output of `ipfs version --all`. If you can't run that command, please include a copy of your [gateway's version page](http://localhost:8080/api/v0/version?enc=text&all=true).\n - type: textarea\n id: config\n attributes:\n label: Config\n render: json\n description: |\n Enter the output of `ipfs config show`.\n - type: textarea\n attributes:\n label: Description\n description: |\n This is where you get to tell us what went wrong. When doing so, please make sure to include *all* relevant information.\n\n Please try to include:\n * What you were doing when you experienced the bug.\n * Any error messages you saw, *where* you saw them, and what you believe may have caused them (if you have any ideas).\n * When possible, steps to reliably produce the bug.\n" }, { "path": ".github/ISSUE_TEMPLATE/config.yml", "content": "blank_issues_enabled: false\ncontact_links:\n - name: Getting Help on IPFS\n url: https://ipfs.tech/help\n about: All information about how and where to get help on IPFS.\n - name: Kubo configuration reference\n url: https://github.com/ipfs/kubo/blob/master/docs/config.md#readme\n about: Documentation on the different configuration settings\n - name: Kubo experimental features docs\n url: https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#readme\n about: Documentation on Private Networks, Filestore and other experimental features.\n - name: Kubo RPC API Reference\n url: https://docs.ipfs.tech/reference/kubo/rpc/\n about: Documentation of all Kubo RPC API endpoints.\n - name: IPFS Official Discussion Forum\n url: https://discuss.ipfs.tech\n about: Please post general questions, support requests, and discussions here.\n" }, { "path": ".github/ISSUE_TEMPLATE/doc.yml", "content": "name: Documentation Issue\ndescription: Report missing, erroneous docs, broken links or propose new Kubo docs.\nlabels:\n - topic/docs-ipfs\n - need/triage\nbody:\n - type: markdown\n attributes:\n value: |\n Problems with documentation on https://docs.ipfs.tech should be reported to https://github.com/ipfs/ipfs-docs\n - type: checkboxes\n attributes:\n label: Checklist\n description: Please verify the following.\n options:\n - label: I am reporting a documentation issue in this repo, not https://docs.ipfs.tech.\n required: true\n - label: I have searched on the [issue tracker](https://github.com/ipfs/kubo/issues?q=is%3Aissue) for my issue.\n required: true\n - type: input\n attributes:\n label: Location\n description: |\n If possible, please provide a link to the documentation issue.\n - type: textarea\n attributes:\n label: Description\n description: |\n Please describe your issue.\n" }, { "path": ".github/ISSUE_TEMPLATE/enhancement.yml", "content": "name: Enhancement\ndescription: Suggest an improvement to an existing kubo feature.\nlabels:\n - kind/enhancement\n - need/triage\nbody:\n - type: markdown\n attributes:\n value: |\n Suggest an enhancement to Kubo (the program). If you'd like to suggest an improvement to the IPFS protocol, please discuss it on [the forum](https://discuss.ipfs.tech).\n\n Issues in this repo must be specific, actionable, and well motivated. They should be starting points for _building_ new features, not brainstorming ideas.\n\n If you have an idea you'd like to discuss, please open a new thread on [the forum](https://discuss.ipfs.tech).\n\n **Example:**\n\n > Reduce memory usage of `ipfs cat` (specific) by buffering less in ... (actionable). This would let me run Kubo on my Raspberry Pi (motivated).\n - type: checkboxes\n attributes:\n label: Checklist\n description: Please verify the following.\n options:\n - label: My issue is specific & actionable.\n required: true\n - label: I am not suggesting a protocol enhancement.\n required: true\n - label: I have searched on the [issue tracker](https://github.com/ipfs/kubo/issues?q=is%3Aissue) for my issue.\n required: true\n - type: textarea\n attributes:\n label: Description\n description: |\n Please describe your idea. When requesting an enhancement, please be sure to include your motivation and try to be as specific as possible.\n" }, { "path": ".github/ISSUE_TEMPLATE/feature.yml", "content": "name: Feature\ndescription: Suggest a new feature in Kubo.\nlabels:\n - kind/feature\n - need/triage\nbody:\n - type: markdown\n attributes:\n value: |\n Suggest a new feature in Kubo (the program). If you'd like to suggest an improvement to the IPFS protocol, please discuss it on [the forum](https://discuss.ipfs.tech).\n\n Issues in this repo must be specific, actionable, and well motivated. They should be starting points for _building_ new features, not brainstorming ideas.\n\n If you have an idea you'd like to discuss, please open a new thread on [the forum](https://discuss.ipfs.tech).\n\n **Example:**\n\n > Add deduplication-optimized chunking of tar files in `ipfs add` (specific) by examining tar headers ... (actionable). This would let me efficiently store and update many versions of code archives (motivated).\n\n - type: checkboxes\n attributes:\n label: Checklist\n description: Please verify the following.\n options:\n - label: My issue is specific & actionable.\n required: true\n - label: I am not suggesting a protocol enhancement.\n required: true\n - label: I have searched on the [issue tracker](https://github.com/ipfs/kubo/issues?q=is%3Aissue) for my issue.\n required: true\n - type: textarea\n attributes:\n label: Description\n description: |\n Please describe your idea. When requesting a feature, please be sure to include your motivation and and a concrete description of how the feature should work.\n" }, { "path": ".github/auto-comment.yml", "content": "# Comment to a new issue.\n# Disabled\n# issueOpened: \"\"\n\n# Disabled\n# pullRequestOpened: \"\"\n" }, { "path": ".github/build-platforms.yml", "content": "# Build platforms configuration for Kubo\n# Matches https://github.com/ipfs/distributions/blob/master/dists/kubo/build_matrix\n# plus linux-riscv64 for emerging architecture support\n#\n# The Go compiler handles FUSE support automatically via build tags.\n# Platforms are simply listed - no need to specify FUSE capability.\n\nplatforms:\n - darwin-amd64\n - darwin-arm64\n - freebsd-amd64\n - linux-amd64\n - linux-arm64\n - linux-riscv64\n - openbsd-amd64\n - windows-amd64\n - windows-arm64" }, { "path": ".github/dependabot.yml", "content": "# Dependabot PRs are auto-tidied by .github/workflows/dependabot-tidy.yml\nversion: 2\nupdates:\n - package-ecosystem: \"github-actions\"\n directory: \"/\"\n schedule:\n interval: \"weekly\"\n\n - package-ecosystem: \"gomod\"\n directory: \"/\"\n schedule:\n interval: \"monthly\"\n open-pull-requests-limit: 10\n labels:\n - \"dependencies\"\n ignore:\n # Updated via go-ds-* wrappers in ipfs-ecosystem group\n - dependency-name: \"github.com/cockroachdb/pebble*\"\n - dependency-name: \"github.com/syndtr/goleveldb\"\n - dependency-name: \"github.com/dgraph-io/badger*\"\n groups:\n ipfs-ecosystem:\n patterns:\n - \"github.com/ipfs/*\"\n - \"github.com/ipfs-shipyard/*\"\n - \"github.com/ipshipyard/*\"\n - \"github.com/multiformats/*\"\n - \"github.com/ipld/*\"\n libp2p-ecosystem:\n patterns:\n - \"github.com/libp2p/*\"\n golang-x:\n patterns:\n - \"golang.org/x/*\"\n opentelemetry:\n patterns:\n - \"go.opentelemetry.io/*\"\n prometheus:\n patterns:\n - \"github.com/prometheus/*\"\n - \"contrib.go.opencensus.io/*\"\n - \"go.opencensus.io\"\n uber:\n patterns:\n - \"go.uber.org/*\"\n" }, { "path": ".github/pull_request_template.md", "content": "\n" }, { "path": ".github/workflows/changelog.yml", "content": "name: Changelog\n\non:\n pull_request:\n types:\n - opened\n - edited\n - synchronize\n - reopened\n - labeled\n - unlabeled\n paths:\n - '**.go'\n - '**/go.mod'\n - '**/go.sum'\n\njobs:\n changelog:\n if: contains(github.event.pull_request.title, '[skip changelog]') == false &&\n contains(github.event.pull_request.labels.*.name, 'skip/changelog') == false\n runs-on: ubuntu-latest\n name: Changelog\n steps:\n - id: changelog\n env:\n GITHUB_TOKEN: ${{ github.token }}\n ENDPOINT: repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files\n SELECTOR: 'map(select(.filename | startswith(\"docs/changelogs/\"))) | length'\n run: gh api \"$ENDPOINT\" --jq \"$SELECTOR\" | xargs -I{} echo \"modified={}\" | tee -a $GITHUB_OUTPUT\n - if: steps.changelog.outputs.modified == '0'\n env:\n MESSAGE: |\n docs/changelogs/ was not modified in this PR. Please do one of the following:\n - add a changelog entry\n - add `[skip changelog]` to the PR title\n - label the PR with `skip/changelog`\n run: |\n echo \"::error::${MESSAGE//$'\\n'/%0A}\"\n exit 1\n" }, { "path": ".github/workflows/codeql-analysis.yml", "content": "# https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed\nname: CodeQL\n\non:\n workflow_dispatch:\n push:\n branches: [ master ]\n pull_request:\n # The branches below must be a subset of the branches above\n branches: [ master ]\n paths-ignore:\n - '**/*.md'\n schedule:\n - cron: '30 12 * * 2'\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n security-events: write # (github/codeql-action/autobuild)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n codeql:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 20\n\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n\n - name: Setup Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n\n # Initializes the CodeQL tools for scanning.\n - name: Initialize CodeQL\n uses: github/codeql-action/init@v4\n with:\n languages: go\n\n - name: Autobuild\n uses: github/codeql-action/autobuild@v4\n\n - name: Perform CodeQL Analysis\n uses: github/codeql-action/analyze@v4\n" }, { "path": ".github/workflows/dependabot-tidy.yml", "content": "# Dependabot only updates go.mod/go.sum in the root module, but this repo has\n# multiple Go modules (see docs/examples/). This workflow runs `make mod_tidy`\n# on Dependabot PRs to keep all go.sum files in sync, preventing go-check CI\n# failures.\nname: Dependabot Tidy\n\non:\n pull_request_target:\n types: [opened, synchronize]\n workflow_dispatch:\n inputs:\n pr_number:\n description: 'PR number to run mod_tidy on'\n required: true\n type: number\n\npermissions:\n contents: write\n pull-requests: write\n\njobs:\n tidy:\n if: github.actor == 'dependabot[bot]' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n steps:\n - name: Get PR info\n id: pr\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n run: |\n if [[ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]]; then\n pr_number=\"${{ inputs.pr_number }}\"\n else\n pr_number=\"${{ github.event.pull_request.number }}\"\n fi\n echo \"number=$pr_number\" >> $GITHUB_OUTPUT\n branch=$(gh pr view \"$pr_number\" --repo \"${{ github.repository }}\" --json headRefName -q '.headRefName')\n echo \"branch=$branch\" >> $GITHUB_OUTPUT\n - uses: actions/checkout@v6\n with:\n ref: ${{ steps.pr.outputs.branch }}\n token: ${{ secrets.GITHUB_TOKEN }}\n - uses: actions/setup-go@v6\n with:\n go-version-file: go.mod\n - name: Run make mod_tidy\n run: make mod_tidy\n - name: Check for changes\n id: git-check\n run: |\n if [[ -n $(git status --porcelain) ]]; then\n echo \"modified=true\" >> $GITHUB_OUTPUT\n fi\n - name: Commit changes\n if: steps.git-check.outputs.modified == 'true'\n run: |\n git config user.name \"github-actions[bot]\"\n git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n git add -A\n git commit -m \"chore: run make mod_tidy\"\n git push\n" }, { "path": ".github/workflows/docker-check.yml", "content": "# This workflow performs a quick Docker build check on PRs and pushes to master.\n# It builds the Docker image and runs a basic smoke test to ensure the image works.\n# This is a lightweight check - for full multi-platform builds and publishing, see docker-image.yml\nname: Docker Check\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - '**/*.md'\n push:\n branches:\n - 'master'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n lint:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 5\n steps:\n - uses: actions/checkout@v6\n - uses: hadolint/hadolint-action@v3.3.0\n with:\n dockerfile: Dockerfile\n failure-threshold: warning\n verbose: true\n format: tty\n\n build:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 10\n env:\n IMAGE_NAME: ipfs/kubo\n WIP_IMAGE_TAG: wip\n defaults:\n run:\n shell: bash\n steps:\n - uses: actions/checkout@v6\n \n - name: Set up Docker Buildx\n uses: docker/setup-buildx-action@v4\n \n - name: Build Docker image with BuildKit\n uses: docker/build-push-action@v7\n with:\n context: .\n push: false\n load: true\n tags: ${{ env.IMAGE_NAME }}:${{ env.WIP_IMAGE_TAG }}\n cache-from: |\n type=gha\n type=registry,ref=${{ env.IMAGE_NAME }}:buildcache\n cache-to: type=gha,mode=max\n \n - name: Test Docker image\n run: docker run --rm $IMAGE_NAME:$WIP_IMAGE_TAG --version\n" }, { "path": ".github/workflows/docker-image.yml", "content": "# This workflow builds and publishes official Docker images to Docker Hub.\n# It handles multi-platform builds (amd64, arm/v7, arm64/v8) and pushes tagged releases.\n# This workflow is triggered on tags, specific branches, and can be manually dispatched.\n# For quick build checks during development, see docker-check.yml\nname: Docker Push\n\non:\n workflow_dispatch:\n inputs:\n push:\n description: 'Push to Docker Hub'\n required: true\n default: 'false'\n tags:\n description: 'Custom tags to use for the push'\n required: false\n default: ''\n # # If we decide to build all images on every PR, we should make sure that\n # # they are NOT pushed to Docker Hub.\n # pull_request:\n # paths-ignore:\n # - '**/*.md'\n push:\n branches:\n - 'master'\n - 'staging'\n - 'bifrost-*'\n tags:\n - 'v*'\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\njobs:\n docker-hub:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n name: Push Docker image to Docker Hub\n runs-on: ubuntu-latest\n timeout-minutes: 15\n env:\n IMAGE_NAME: ipfs/kubo\n outputs:\n tags: ${{ steps.tags.outputs.value }}\n steps:\n - name: Check out the repo\n uses: actions/checkout@v6\n\n - name: Set up QEMU\n uses: docker/setup-qemu-action@v4\n\n - name: Set up Docker Buildx\n uses: docker/setup-buildx-action@v4\n\n - name: Log in to Docker Hub\n uses: docker/login-action@v4\n with:\n username: ${{ vars.DOCKER_USERNAME }}\n password: ${{ secrets.DOCKER_PASSWORD }}\n\n - name: Get tags\n id: tags\n if: github.event.inputs.tags == ''\n run: |\n echo \"value<> $GITHUB_OUTPUT\n ./bin/get-docker-tags.sh \"$(date -u +%F)\" >> $GITHUB_OUTPUT\n echo \"EOF\" >> $GITHUB_OUTPUT\n shell: bash\n\n # We have to build each platform separately because when using multi-arch\n # builds, only one platform is being loaded into the cache. This would\n # prevent us from testing the other platforms.\n - name: Build Docker image (linux/amd64)\n uses: docker/build-push-action@v7\n with:\n platforms: linux/amd64\n context: .\n push: false\n load: true\n file: ./Dockerfile\n tags: ${{ env.IMAGE_NAME }}:linux-amd64\n cache-from: |\n type=gha\n type=registry,ref=${{ env.IMAGE_NAME }}:buildcache\n cache-to: type=gha,mode=max\n\n - name: Build Docker image (linux/arm/v7)\n uses: docker/build-push-action@v7\n with:\n platforms: linux/arm/v7\n context: .\n push: false\n load: true\n file: ./Dockerfile\n tags: ${{ env.IMAGE_NAME }}:linux-arm-v7\n cache-from: |\n type=gha\n type=registry,ref=${{ env.IMAGE_NAME }}:buildcache\n cache-to: type=gha,mode=max\n\n - name: Build Docker image (linux/arm64/v8)\n uses: docker/build-push-action@v7\n with:\n platforms: linux/arm64/v8\n context: .\n push: false\n load: true\n file: ./Dockerfile\n tags: ${{ env.IMAGE_NAME }}:linux-arm64-v8\n cache-from: |\n type=gha\n type=registry,ref=${{ env.IMAGE_NAME }}:buildcache\n cache-to: type=gha,mode=max\n\n # We test all the images on amd64 host here. This uses QEMU to emulate\n # the other platforms.\n # NOTE: --version should finish instantly, but sometimes\n # it hangs on github CI (could be qemu issue), so we retry to remove false negatives\n - name: Smoke-test linux-amd64\n run: for i in {1..3}; do timeout 15s docker run --rm $IMAGE_NAME:linux-amd64 version --all && break || [ $i = 3 ] && exit 1; done\n timeout-minutes: 1\n - name: Smoke-test linux-arm-v7\n run: for i in {1..3}; do timeout 15s docker run --rm $IMAGE_NAME:linux-arm-v7 version --all && break || [ $i = 3 ] && exit 1; done\n timeout-minutes: 1\n - name: Smoke-test linux-arm64-v8\n run: for i in {1..3}; do timeout 15s docker run --rm $IMAGE_NAME:linux-arm64-v8 version --all && break || [ $i = 3 ] && exit 1; done\n timeout-minutes: 1\n\n # This will only push the previously built images.\n - if: github.event_name != 'workflow_dispatch' || github.event.inputs.push == 'true'\n name: Publish to Docker Hub\n uses: docker/build-push-action@v7\n with:\n platforms: linux/amd64,linux/arm/v7,linux/arm64/v8\n context: .\n push: true\n file: ./Dockerfile\n tags: \"${{ github.event.inputs.tags || steps.tags.outputs.value }}\"\n cache-from: |\n type=gha\n type=registry,ref=${{ env.IMAGE_NAME }}:buildcache\n cache-to: |\n type=gha,mode=max\n type=registry,ref=${{ env.IMAGE_NAME }}:buildcache,mode=max\n" }, { "path": ".github/workflows/gateway-conformance.yml", "content": "name: Gateway Conformance\n\non:\n workflow_dispatch:\n push:\n branches:\n - master\n pull_request:\n paths-ignore:\n - '**/*.md'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\ndefaults:\n run:\n shell: bash\n\nenv:\n # hostnames expected by https://github.com/ipfs/gateway-conformance\n GATEWAY_PUBLIC_GATEWAYS: |\n {\n \"example.com\": {\n \"UseSubdomains\": true,\n \"InlineDNSLink\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\"]\n },\n \"localhost\": {\n \"UseSubdomains\": true,\n \"InlineDNSLink\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\"]\n }\n }\n\njobs:\n # Testing all gateway features via TCP port specified in Addresses.Gateway\n gateway-conformance:\n runs-on: ubuntu-latest\n timeout-minutes: 10\n steps:\n # 1. Download the gateway-conformance fixtures\n - name: Download gateway-conformance fixtures\n uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.12\n with:\n output: fixtures\n\n # 2. Build the kubo-gateway\n - name: Checkout kubo-gateway\n uses: actions/checkout@v6\n with:\n path: kubo-gateway\n - name: Setup Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'kubo-gateway/go.mod'\n cache: true\n cache-dependency-path: kubo-gateway/go.sum\n - name: Build kubo-gateway\n run: make build\n working-directory: kubo-gateway\n\n # 3. Init the kubo-gateway\n - name: Init kubo-gateway\n run: |\n ./ipfs init -e\n ./ipfs config --json Gateway.PublicGateways \"$GATEWAY_PUBLIC_GATEWAYS\"\n working-directory: kubo-gateway/cmd/ipfs\n\n # 4. Populate the Kubo gateway with the gateway-conformance fixtures\n - name: Import fixtures\n run: |\n # Import car files\n find ./fixtures -name '*.car' -exec kubo-gateway/cmd/ipfs/ipfs dag import --pin-roots=false {} \\;\n\n # Import ipns records\n records=$(find ./fixtures -name '*.ipns-record')\n for record in $records\n do\n key=$(basename -s .ipns-record \"$record\" | cut -d'_' -f1)\n kubo-gateway/cmd/ipfs/ipfs routing put --allow-offline \"/ipns/$key\" \"$record\"\n done\n\n # Import dnslink records\n # the IPFS_NS_MAP env will be used by the daemon\n echo \"IPFS_NS_MAP=$(cat ./fixtures/dnslinks.IPFS_NS_MAP)\" >> $GITHUB_ENV\n\n # 5. Start the kubo-gateway\n - name: Start kubo-gateway\n run: |\n ./ipfs daemon --offline &\n working-directory: kubo-gateway/cmd/ipfs\n\n # 6. Run the gateway-conformance tests\n - name: Run gateway-conformance tests\n uses: ipfs/gateway-conformance/.github/actions/test@v0.12\n with:\n gateway-url: http://127.0.0.1:8080\n subdomain-url: http://localhost:8080\n args: -skip 'TestGatewayCar/GET_response_for_application/vnd.ipld.car/Header_Content-Length'\n json: output.json\n xml: output.xml\n html: output.html\n markdown: output.md\n\n # 7. Upload the results\n - name: Upload MD summary\n if: failure() || success()\n run: cat output.md >> $GITHUB_STEP_SUMMARY\n - name: Upload HTML report\n if: failure() || success()\n uses: actions/upload-artifact@v7\n with:\n name: gateway-conformance.html\n path: output.html\n - name: Upload JSON report\n if: failure() || success()\n uses: actions/upload-artifact@v7\n with:\n name: gateway-conformance.json\n path: output.json\n\n # Testing trustless gateway feature subset exposed as libp2p protocol\n gateway-conformance-libp2p-experiment:\n runs-on: ubuntu-latest\n timeout-minutes: 10\n steps:\n # 1. Download the gateway-conformance fixtures\n - name: Download gateway-conformance fixtures\n uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.12\n with:\n output: fixtures\n\n # 2. Build the kubo-gateway\n - name: Checkout kubo-gateway\n uses: actions/checkout@v6\n with:\n path: kubo-gateway\n - name: Setup Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'kubo-gateway/go.mod'\n cache: true\n cache-dependency-path: kubo-gateway/go.sum\n - name: Build kubo-gateway\n run: make build\n working-directory: kubo-gateway\n\n # 3. Init the kubo-gateway\n - name: Init kubo-gateway\n run: |\n ./ipfs init --profile=test\n ./ipfs config --json Gateway.PublicGateways \"$GATEWAY_PUBLIC_GATEWAYS\"\n ./ipfs config --json Experimental.GatewayOverLibp2p true\n ./ipfs config Addresses.Gateway \"/ip4/127.0.0.1/tcp/8080\"\n ./ipfs config Addresses.API \"/ip4/127.0.0.1/tcp/5001\"\n working-directory: kubo-gateway/cmd/ipfs\n\n # 4. Populate the Kubo gateway with the gateway-conformance fixtures\n - name: Import fixtures\n run: |\n # Import car files\n find ./fixtures -name '*.car' -exec kubo-gateway/cmd/ipfs/ipfs dag import --pin-roots=false {} \\;\n\n # 5. Start the kubo-gateway\n - name: Start kubo-gateway\n run: |\n ( ./ipfs daemon & ) | sed '/Daemon is ready/q'\n while [[ \"$(./ipfs id | jq '.Addresses | length')\" == '0' ]]; do sleep 1; done\n working-directory: kubo-gateway/cmd/ipfs\n\n # 6. Setup a kubo http-p2p-proxy to expose libp2p protocol as a regular HTTP port for gateway conformance tests\n - name: Init p2p-proxy kubo node\n env:\n IPFS_PATH: \"~/.kubo-p2p-proxy\"\n run: |\n ./ipfs init --profile=test -e\n ./ipfs config --json Experimental.Libp2pStreamMounting true\n ./ipfs config Addresses.Gateway \"/ip4/127.0.0.1/tcp/8081\"\n ./ipfs config Addresses.API \"/ip4/127.0.0.1/tcp/5002\"\n working-directory: kubo-gateway/cmd/ipfs\n\n # 7. Start the kubo http-p2p-proxy\n - name: Start kubo http-p2p-proxy\n env:\n IPFS_PATH: \"~/.kubo-p2p-proxy\"\n run: |\n ( ./ipfs daemon & ) | sed '/Daemon is ready/q'\n while [[ \"$(./ipfs id | jq '.Addresses | length')\" == '0' ]]; do sleep 1; done\n working-directory: kubo-gateway/cmd/ipfs\n\n # 8. Start forwarding data from the http-p2p-proxy to the node serving the Gateway API over libp2p\n - name: Start http-over-libp2p forwarding proxy\n run: |\n gatewayNodeId=$(./ipfs --api=/ip4/127.0.0.1/tcp/5001 id -f=\"\")\n ./ipfs --api=/ip4/127.0.0.1/tcp/5002 swarm connect $(./ipfs --api=/ip4/127.0.0.1/tcp/5001 swarm addrs local --id | head -n 1)\n ./ipfs --api=/ip4/127.0.0.1/tcp/5002 p2p forward --allow-custom-protocol /http/1.1 /ip4/127.0.0.1/tcp/8092 /p2p/$gatewayNodeId\n working-directory: kubo-gateway/cmd/ipfs\n\n # 9. Run the gateway-conformance tests over libp2p\n - name: Run gateway-conformance tests over libp2p\n uses: ipfs/gateway-conformance/.github/actions/test@v0.12\n with:\n gateway-url: http://127.0.0.1:8092\n args: --specs \"trustless-gateway,-trustless-ipns-gateway\" -skip 'TestGatewayCar/GET_response_for_application/vnd.ipld.car/Header_Content-Length'\n json: output.json\n xml: output.xml\n html: output.html\n markdown: output.md\n\n # 10. Upload the results\n - name: Upload MD summary\n if: failure() || success()\n run: cat output.md >> $GITHUB_STEP_SUMMARY\n - name: Upload HTML report\n if: failure() || success()\n uses: actions/upload-artifact@v7\n with:\n name: gateway-conformance-libp2p.html\n path: output.html\n - name: Upload JSON report\n if: failure() || success()\n uses: actions/upload-artifact@v7\n with:\n name: gateway-conformance-libp2p.json\n path: output.json\n" }, { "path": ".github/workflows/generated-pr.yml", "content": "name: Close Generated PRs\n\non:\n schedule:\n - cron: '0 0 * * *'\n workflow_dispatch:\n\npermissions:\n issues: write\n pull-requests: write\n\njobs:\n stale:\n uses: ipdxco/unified-github-workflows/.github/workflows/reusable-generated-pr.yml@v1\n" }, { "path": ".github/workflows/gobuild.yml", "content": "name: Go Build\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - '**/*.md'\n push:\n branches:\n - 'master'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n go-build:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"4xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: 20\n env:\n TEST_DOCKER: 0\n TEST_VERBOSE: 1\n GIT_PAGER: cat\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - uses: actions/checkout@v6\n - uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n cache: true\n cache-dependency-path: go.sum\n\n - name: Build all platforms\n run: |\n # Read platforms from build-platforms.yml and build each one\n echo \"Building kubo for all platforms...\"\n\n # Read and build each platform\n grep '^ - ' .github/build-platforms.yml | sed 's/^ - //' | while read -r platform; do\n if [ -z \"$platform\" ]; then\n continue\n fi\n\n echo \"::group::Building $platform\"\n GOOS=$(echo \"$platform\" | cut -d- -f1)\n GOARCH=$(echo \"$platform\" | cut -d- -f2)\n\n echo \"Building $platform\"\n echo \" GOOS=$GOOS GOARCH=$GOARCH go build -o /dev/null ./cmd/ipfs\"\n GOOS=$GOOS GOARCH=$GOARCH go build -o /dev/null ./cmd/ipfs\n echo \"::endgroup::\"\n done\n\n echo \"All platforms built successfully\"" }, { "path": ".github/workflows/golang-analysis.yml", "content": "name: Go Check\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - '**/*.md'\n push:\n branches:\n - 'master'\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n go-check:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 10\n steps:\n - uses: actions/checkout@v6\n with:\n submodules: recursive\n - uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n - name: Check that go.mod is tidy\n uses: protocol/multiple-go-modules@v1.4\n with:\n run: |\n go mod tidy\n if [[ -n $(git ls-files --other --exclude-standard --directory -- go.sum) ]]; then\n echo \"go.sum was added by go mod tidy\"\n exit 1\n fi\n git diff --exit-code -- go.sum go.mod\n - name: go fmt\n if: always() # run this step even if the previous one failed\n run: |\n out=$(go fmt ./...)\n if [[ -n \"$out\" ]]; then\n echo \"Files are not go-fmt-ed:\"\n echo \"$out\"\n exit 1\n fi\n - name: go fix\n if: always() # run this step even if the previous one failed\n run: |\n go fix ./...\n if [[ -n $(git diff --name-only) ]]; then\n echo \"go fix produced changes. Run 'go fix ./...' locally and commit the result.\"\n git diff\n exit 1\n fi\n - name: go vet\n if: always() # run this step even if the previous one failed\n uses: protocol/multiple-go-modules@v1.4\n with:\n run: go vet ./...\n" }, { "path": ".github/workflows/golint.yml", "content": "name: Go Lint\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - '**/*.md'\n push:\n branches:\n - 'master'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n go-lint:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 10\n env:\n TEST_DOCKER: 0\n TEST_FUSE: 0\n TEST_VERBOSE: 1\n GIT_PAGER: cat\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - uses: actions/checkout@v6\n - uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n - run: make -O test_go_lint\n" }, { "path": ".github/workflows/gotest.yml", "content": "name: Go Test\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - '**/*.md'\n push:\n branches:\n - 'master'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n # Unit tests with coverage collection (uploaded to Codecov)\n unit-tests:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"2xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: 15\n env:\n GOTRACEBACK: single # reduce noise on test timeout panics\n TEST_DOCKER: 0\n TEST_FUSE: 0\n TEST_VERBOSE: 1\n GIT_PAGER: cat\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - name: Check out Kubo\n uses: actions/checkout@v6\n - name: Set up Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n - name: Install missing tools\n run: sudo apt update && sudo apt install -y zsh\n - name: Run unit tests\n run: |\n make test_unit &&\n [[ ! $(jq -s -c 'map(select(.Action == \"fail\")) | .[]' test/unit/gotest.json) ]]\n - name: Upload coverage to Codecov\n uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2\n if: failure() || success()\n with:\n name: unittests\n files: coverage/unit_tests.coverprofile\n token: ${{ secrets.CODECOV_TOKEN }}\n fail_ci_if_error: false\n - name: Create a proper JUnit XML report\n uses: ipdxco/gotest-json-to-junit-xml@v1\n with:\n input: test/unit/gotest.json\n output: test/unit/gotest.junit.xml\n if: failure() || success()\n - name: Archive the JUnit XML report\n uses: actions/upload-artifact@v7\n with:\n name: unit-tests-junit\n path: test/unit/gotest.junit.xml\n if: failure() || success()\n - name: Create a HTML report\n uses: ipdxco/junit-xml-to-html@v1\n with:\n mode: no-frames\n input: test/unit/gotest.junit.xml\n output: test/unit/gotest.html\n if: failure() || success()\n - name: Archive the HTML report\n uses: actions/upload-artifact@v7\n with:\n name: unit-tests-html\n path: test/unit/gotest.html\n if: failure() || success()\n - name: Create a Markdown report\n uses: ipdxco/junit-xml-to-html@v1\n with:\n mode: summary\n input: test/unit/gotest.junit.xml\n output: test/unit/gotest.md\n if: failure() || success()\n - name: Set the summary\n run: cat test/unit/gotest.md >> $GITHUB_STEP_SUMMARY\n if: failure() || success()\n\n # End-to-end integration/regression tests from test/cli\n # (Go-based replacement for legacy test/sharness shell scripts)\n cli-tests:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"2xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: 15\n env:\n GOTRACEBACK: single # reduce noise on test timeout panics\n TEST_VERBOSE: 1\n GIT_PAGER: cat\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - name: Check out Kubo\n uses: actions/checkout@v6\n - name: Set up Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n - name: Install missing tools\n run: sudo apt update && sudo apt install -y zsh\n - name: Run CLI tests\n env:\n IPFS_PATH: ${{ runner.temp }}/ipfs-test\n run: make test_cli\n - name: Create JUnit XML report\n uses: ipdxco/gotest-json-to-junit-xml@v1\n with:\n input: test/cli/cli-tests.json\n output: test/cli/cli-tests.junit.xml\n if: failure() || success()\n - name: Archive JUnit XML report\n uses: actions/upload-artifact@v7\n with:\n name: cli-tests-junit\n path: test/cli/cli-tests.junit.xml\n if: failure() || success()\n - name: Create HTML report\n uses: ipdxco/junit-xml-to-html@v1\n with:\n mode: no-frames\n input: test/cli/cli-tests.junit.xml\n output: test/cli/cli-tests.html\n if: failure() || success()\n - name: Archive HTML report\n uses: actions/upload-artifact@v7\n with:\n name: cli-tests-html\n path: test/cli/cli-tests.html\n if: failure() || success()\n - name: Create Markdown report\n uses: ipdxco/junit-xml-to-html@v1\n with:\n mode: summary\n input: test/cli/cli-tests.junit.xml\n output: test/cli/cli-tests.md\n if: failure() || success()\n - name: Set summary\n run: cat test/cli/cli-tests.md >> $GITHUB_STEP_SUMMARY\n if: failure() || success()\n\n # Example tests (kubo-as-a-library)\n example-tests:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"2xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: 5\n env:\n GOTRACEBACK: single\n defaults:\n run:\n shell: bash\n steps:\n - name: Check out Kubo\n uses: actions/checkout@v6\n - name: Set up Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n - name: Run example tests\n run: make test_examples\n" }, { "path": ".github/workflows/interop.yml", "content": "# Interoperability Tests\n#\n# This workflow ensures Kubo remains compatible with the broader IPFS ecosystem.\n# It builds Kubo from source, then runs:\n#\n# 1. helia-interop: Tests compatibility with Helia (JavaScript IPFS implementation)\n# using Playwright-based tests from @helia/interop package.\n#\n# 2. ipfs-webui: Runs E2E tests from ipfs/ipfs-webui repository to verify\n# the web interface works correctly with the locally built Kubo binary.\n#\n# Both jobs use caching to speed up repeated runs (npm dependencies, Playwright\n# browsers, and webui build artifacts).\n\nname: Interop\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - '**/*.md'\n push:\n branches:\n - 'master'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\ndefaults:\n run:\n shell: bash\n\njobs:\n interop-prep:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 5\n env:\n TEST_DOCKER: 0\n TEST_FUSE: 0\n TEST_VERBOSE: 1\n GIT_PAGER: cat\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - uses: actions/checkout@v6\n - uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n - run: make build\n - uses: actions/upload-artifact@v7\n with:\n name: kubo\n path: cmd/ipfs/ipfs\n helia-interop:\n needs: [interop-prep]\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"2xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: 20\n defaults:\n run:\n shell: bash\n steps:\n - uses: actions/setup-node@v6\n with:\n node-version: lts/*\n - uses: actions/download-artifact@v8\n with:\n name: kubo\n path: cmd/ipfs\n - run: chmod +x cmd/ipfs/ipfs\n - run: sudo apt update\n - run: sudo apt install -y libxkbcommon0 libxdamage1 libgbm1 libpango-1.0-0 libcairo2 # dependencies for playwright\n # Cache node_modules based on latest @helia/interop version from npm registry.\n # This ensures we always test against the latest release while still benefiting\n # from caching when the version hasn't changed.\n - name: Get latest @helia/interop version\n id: helia-version\n run: echo \"version=$(npm view @helia/interop version)\" >> $GITHUB_OUTPUT\n - name: Cache helia-interop node_modules\n uses: actions/cache@v5\n id: helia-cache\n with:\n path: node_modules\n key: ${{ runner.os }}-helia-interop-${{ steps.helia-version.outputs.version }}\n - name: Install @helia/interop\n if: steps.helia-cache.outputs.cache-hit != 'true'\n run: npm install @helia/interop\n # TODO(IPIP-499): Remove --grep --invert workaround once helia implements IPIP-499\n # Tracking issue: https://github.com/ipfs/helia/issues/941\n #\n # PROVISIONAL HACK: Skip '@helia/mfs - should have the same CID after\n # creating a file' test due to IPIP-499 changes in kubo.\n #\n # WHY IT FAILS: The test creates a 5-byte file in MFS on both kubo and helia,\n # then compares the root directory CID. With kubo PR #11148, `ipfs files write`\n # now produces raw CIDs for single-block files (matching `ipfs add --raw-leaves`),\n # while helia uses `reduceSingleLeafToSelf: false` which keeps the dag-pb wrapper.\n # Different file CIDs lead to different directory CIDs.\n #\n # We run aegir directly (instead of helia-interop binary) because only aegir\n # supports the --grep/--invert flags needed to exclude specific tests.\n - name: Run helia-interop tests (excluding IPIP-499 incompatible test)\n run: npx aegir test -t node --bail -- --grep 'should have the same CID after creating a file' --invert\n env:\n KUBO_BINARY: ${{ github.workspace }}/cmd/ipfs/ipfs\n working-directory: node_modules/@helia/interop\n ipfs-webui:\n needs: [interop-prep]\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"2xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: 20\n env:\n NO_SANDBOX: true\n LIBP2P_TCP_REUSEPORT: false\n LIBP2P_ALLOW_WEAK_RSA_KEYS: 1\n E2E_IPFSD_TYPE: go\n GIT_PAGER: cat\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - uses: actions/download-artifact@v8\n with:\n name: kubo\n path: cmd/ipfs\n - run: chmod +x cmd/ipfs/ipfs\n - uses: actions/checkout@v6\n with:\n repository: ipfs/ipfs-webui\n path: ipfs-webui\n - uses: actions/setup-node@v6\n with:\n node-version-file: 'ipfs-webui/.tool-versions'\n - id: webui-ref\n run: echo \"ref=$(git rev-parse --short HEAD)\" | tee -a $GITHUB_OUTPUT\n working-directory: ipfs-webui\n - id: webui-state\n env:\n GITHUB_TOKEN: ${{ github.token }}\n ENDPOINT: repos/ipfs/ipfs-webui/commits/${{ steps.webui-ref.outputs.ref }}/status\n SELECTOR: .state\n KEY: state\n run: gh api \"$ENDPOINT\" --jq \"$SELECTOR\" | xargs -I{} echo \"$KEY={}\" | tee -a $GITHUB_OUTPUT\n # Cache node_modules based on package-lock.json\n - name: Cache node_modules\n uses: actions/cache@v5\n id: node-modules-cache\n with:\n path: ipfs-webui/node_modules\n key: ${{ runner.os }}-webui-node-modules-${{ hashFiles('ipfs-webui/package-lock.json') }}\n restore-keys: |\n ${{ runner.os }}-webui-node-modules-\n - name: Install dependencies\n if: steps.node-modules-cache.outputs.cache-hit != 'true'\n run: npm ci --prefer-offline --no-audit --progress=false\n working-directory: ipfs-webui\n # Cache Playwright browsers\n - name: Cache Playwright browsers\n uses: actions/cache@v5\n id: playwright-cache\n with:\n path: ~/.cache/ms-playwright\n key: ${{ runner.os }}-playwright-${{ hashFiles('ipfs-webui/package-lock.json') }}\n restore-keys: |\n ${{ runner.os }}-playwright-\n # On cache miss: download browsers and install OS dependencies\n - name: Install Playwright with dependencies\n if: steps.playwright-cache.outputs.cache-hit != 'true'\n run: npx playwright install --with-deps\n working-directory: ipfs-webui\n # On cache hit: only ensure OS dependencies are present (fast, idempotent)\n - name: Install Playwright OS dependencies\n if: steps.playwright-cache.outputs.cache-hit == 'true'\n run: npx playwright install-deps\n working-directory: ipfs-webui\n # Cache test build output\n - name: Cache test build\n uses: actions/cache@v5\n id: test-build-cache\n with:\n path: ipfs-webui/build\n key: ${{ runner.os }}-webui-build-${{ hashFiles('ipfs-webui/package-lock.json', 'ipfs-webui/src/**', 'ipfs-webui/public/**') }}\n restore-keys: |\n ${{ runner.os }}-webui-build-\n - name: Build ipfs-webui@${{ steps.webui-ref.outputs.ref }} (state=${{ steps.webui-state.outputs.state }})\n if: steps.test-build-cache.outputs.cache-hit != 'true'\n run: npm run test:build\n working-directory: ipfs-webui\n - name: Test ipfs-webui@${{ steps.webui-ref.outputs.ref }} (state=${{ steps.webui-state.outputs.state }}) E2E against the locally built Kubo binary\n run: npm run test:e2e\n env:\n IPFS_GO_EXEC: ${{ github.workspace }}/cmd/ipfs/ipfs\n working-directory: ipfs-webui\n - name: Upload test artifacts on failure\n if: failure()\n uses: actions/upload-artifact@v7\n with:\n name: webui-test-results\n path: ipfs-webui/test-results/\n retention-days: 7\n" }, { "path": ".github/workflows/sharness.yml", "content": "name: Sharness\n\non:\n workflow_dispatch:\n pull_request:\n paths-ignore:\n - \"**/*.md\"\n push:\n branches:\n - \"master\"\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n sharness-test:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '[\"self-hosted\", \"linux\", \"x64\", \"4xlarge\"]' || '\"ubuntu-latest\"') }}\n timeout-minutes: ${{ github.repository == 'ipfs/kubo' && 15 || 60 }}\n defaults:\n run:\n shell: bash\n steps:\n - name: Checkout Kubo\n uses: actions/checkout@v6\n with:\n path: kubo\n - name: Setup Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'kubo/go.mod'\n - name: Install missing tools\n run: sudo apt update && sudo apt install -y socat net-tools fish libxml2-utils\n - uses: actions/cache@v5\n with:\n path: test/sharness/lib/dependencies\n key: ${{ runner.os }}-test-generate-junit-html-${{ hashFiles('test/sharness/lib/test-generate-junit-html.sh') }}\n - name: Run Sharness tests\n run: |\n make -O -j \"$PARALLEL\" \\\n test_sharness \\\n coverage/sharness_tests.coverprofile \\\n test/sharness/test-results/sharness.xml\n working-directory: kubo\n env:\n TEST_DOCKER: 1\n TEST_PLUGIN: 0\n TEST_FUSE: 0\n TEST_VERBOSE: 1\n TEST_JUNIT: 1\n TEST_EXPENSIVE: 1\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n CONTINUE_ON_S_FAILURE: 1\n # increasing parallelism beyond 10 doesn't speed up the tests much\n PARALLEL: ${{ github.repository == 'ipfs/kubo' && 10 || 3 }}\n - name: Upload coverage report\n uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2\n if: failure() || success()\n with:\n name: sharness\n files: kubo/coverage/sharness_tests.coverprofile\n token: ${{ secrets.CODECOV_TOKEN }}\n fail_ci_if_error: false\n - name: Aggregate results\n run: find kubo/test/sharness/test-results -name 't*-*.sh.*.counts' | kubo/test/sharness/lib/sharness/aggregate-results.sh > kubo/test/sharness/test-results/summary.txt\n - name: 👉️ If this step failed, go to «Summary» (top left) → «HTML Report» → inspect the «Failures» column\n run: |\n cat kubo/test/sharness/test-results/summary.txt &&\n grep 'failed\\s*0' kubo/test/sharness/test-results/summary.txt\n - name: Add aggregate results to the summary\n if: failure() || success()\n run: |\n echo \"# Summary\" >> $GITHUB_STEP_SUMMARY\n echo >> $GITHUB_STEP_SUMMARY\n cat kubo/test/sharness/test-results/summary.txt >> $GITHUB_STEP_SUMMARY\n - name: Generate one-page HTML report\n uses: ipdxco/junit-xml-to-html@v1\n if: failure() || success()\n with:\n mode: no-frames\n input: kubo/test/sharness/test-results/sharness.xml\n output: kubo/test/sharness/test-results/sharness.html\n - name: Upload one-page HTML report to S3\n id: one-page\n uses: ipdxco/custom-github-runners/.github/actions/upload-artifact@main\n if: github.repository == 'ipfs/kubo' && (failure() || success())\n with:\n source: kubo/test/sharness/test-results/sharness.html\n destination: sharness.html\n - name: Upload one-page HTML report\n if: github.repository != 'ipfs/kubo' && (failure() || success())\n uses: actions/upload-artifact@v7\n with:\n name: sharness.html\n path: kubo/test/sharness/test-results/sharness.html\n - name: Generate full HTML report\n uses: ipdxco/junit-xml-to-html@v1\n if: failure() || success()\n with:\n mode: frames\n input: kubo/test/sharness/test-results/sharness.xml\n output: kubo/test/sharness/test-results/sharness-html\n - name: Upload full HTML report to S3\n id: full\n uses: ipdxco/custom-github-runners/.github/actions/upload-artifact@main\n if: github.repository == 'ipfs/kubo' && (failure() || success())\n with:\n source: kubo/test/sharness/test-results/sharness-html\n destination: sharness-html/\n - name: Upload full HTML report\n if: github.repository != 'ipfs/kubo' && (failure() || success())\n uses: actions/upload-artifact@v7\n with:\n name: sharness-html\n path: kubo/test/sharness/test-results/sharness-html\n - name: Add S3 links to the summary\n if: github.repository == 'ipfs/kubo' && (failure() || success())\n run: echo \"$MD\" >> $GITHUB_STEP_SUMMARY\n env:\n MD: |\n # HTML Reports\n\n - View the [one page HTML report](${{ steps.one-page.outputs.url }})\n - View the [full HTML report](${{ steps.full.outputs.url }}index.html)\n" }, { "path": ".github/workflows/spellcheck.yml", "content": "name: Spell Check\n\non:\n pull_request:\n push:\n branches: [\"master\"]\n workflow_dispatch:\n\npermissions:\n contents: read\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n spellcheck:\n uses: ipdxco/unified-github-workflows/.github/workflows/reusable-spellcheck.yml@v1\n" }, { "path": ".github/workflows/stale.yml", "content": "name: Close Stale Issues\n\non:\n schedule:\n - cron: '0 0 * * *'\n workflow_dispatch:\n\npermissions:\n issues: write\n pull-requests: write\n\njobs:\n stale:\n uses: ipdxco/unified-github-workflows/.github/workflows/reusable-stale-issue.yml@v1\n" }, { "path": ".github/workflows/sync-release-assets.yml", "content": "name: Sync GitHub Release Assets\n\non:\n workflow_dispatch:\n schedule:\n - cron: '0 0 * * *'\n\nconcurrency:\n group: release-assets-dist-sync\n cancel-in-progress: true\n\npermissions:\n contents: write # to upload release asset\n\njobs:\n dist-ipfs-tech:\n if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'\n runs-on: \"ubuntu-latest\"\n timeout-minutes: 15\n steps:\n - uses: ipfs/download-ipfs-distribution-action@v1\n - uses: ipfs/start-ipfs-daemon-action@v1\n with:\n args: --init --init-profile=flatfs,server --enable-gc=false\n - uses: actions/setup-node@v6\n with:\n node-version: 14\n - name: Sync the latest 5 github releases\n uses: actions/github-script@v8\n with:\n script: |\n const fs = require('fs').promises\n const max_synced = 5\n\n // fetch github releases\n resp = await github.rest.repos.listReleases({\n owner: context.repo.owner,\n repo: context.repo.repo,\n page: 1,\n per_page: max_synced\n })\n const release_assets = [];\n num_synced = 0;\n for (const release of resp.data) {\n console.log(\"checking release tagged\", release.tag_name)\n if (num_synced > max_synced) {\n console.log(\"done: synced\", max_synced, \"latest releases\")\n break;\n }\n num_synced += 1\n\n const github_assets = new Set()\n for (const asset of release.assets) {\n github_assets.add(asset.name)\n }\n\n // fetch asset info from dist.ipfs.tech\n p = '/ipns/dist.ipfs.tech/kubo/' + release.tag_name\n let stdout = ''\n const options = {}\n options.listeners = {\n stdout: (data) => {\n stdout += data.toString();\n }\n }\n await exec.exec('ipfs', ['ls', p], options)\n\n const dist_assets = new Set()\n const missing_files = []\n for (const raw_line of stdout.split(\"\\n\")) {\n line = raw_line.trim();\n if (line.length != 0) {\n file = line.split(/(\\s+)/).filter( function(e) { return e.trim().length > 0; } )[2]\n dist_assets.add(file)\n if (!github_assets.has(file)) {\n missing_files.push(file)\n }\n }\n }\n\n // if dist.ipfs.tech has files not found in github, copy them over\n for (const file of missing_files) {\n file_sha = file + \".sha512\"\n file_cid = file + \".cid\"\n\n // skip files that don't have .cid and .sha512 checksum files\n if (!dist_assets.has(file_sha) || !dist_assets.has(file_cid)) {\n if (!file.endsWith('.cid') && !file.endsWith('.sha512')) { // silent skip of .sha512.sha512 :)\n console.log(`skipping \"${file}\" as dist.ipfs.tech does not provide .cid and .sha512 checksum files for it`)\n }\n continue\n }\n\n console.log(\"fetching\", file, \"from dist.ipfs.tech\")\n await exec.exec('ipfs', ['get', p + '/' + file])\n await exec.exec('ipfs', ['get', p + '/' + file_sha])\n await exec.exec('ipfs', ['get', p + '/' + file_cid])\n console.log(\"verifying contents of\", file)\n\n // compute sha512 output for file\n let sha_stdout = ''\n const sha_options = {}\n sha_options.listeners = {\n stdout: (data) => {\n sha_stdout += data.toString();\n }\n }\n await exec.exec('sha512sum', [file], sha_options)\n // read expected sha512 output\n const sha_data = await fs.readFile(file_sha, \"utf8\")\n const digest = (s) => s.split(' ').shift()\n if (digest(sha_data) != digest(sha_stdout)) {\n console.log(`${file}.sha512: ${sha_data}`)\n console.log(`sha512sum ${file}: ${sha_stdout}`)\n throw \"checksum verification failed for \" + file\n }\n\n console.log(\"uploading\", file, \"to github release\", release.tag_name)\n const uploadReleaseAsset = async (file) => github.rest.repos.uploadReleaseAsset({\n owner: context.repo.owner,\n repo: context.repo.repo,\n release_id: release.id,\n headers: {\n \"content-type\": \"application/octet-stream\",\n \"content-length\": `${(await fs.stat(file)).size}`\n },\n name: file,\n data: await fs.readFile(file)\n })\n await uploadReleaseAsset(file)\n await uploadReleaseAsset(file_sha)\n await uploadReleaseAsset(file_cid)\n\n }\n // summary of assets on both sides\n release_assets.push({ tag: release.tag_name, github_assets, dist_assets })\n }\n console.log(release_assets)\n return release_assets\n" }, { "path": ".github/workflows/test-migrations.yml", "content": "name: Migrations\n\non:\n workflow_dispatch:\n pull_request:\n paths:\n # Migration implementation files\n - 'repo/fsrepo/migrations/**'\n - 'test/cli/migrations/**'\n # Config and repo handling\n - 'repo/fsrepo/**'\n # This workflow file itself\n - '.github/workflows/test-migrations.yml'\n push:\n branches:\n - 'master'\n - 'release-*'\n paths:\n - 'repo/fsrepo/migrations/**'\n - 'test/cli/migrations/**'\n - 'repo/fsrepo/**'\n - '.github/workflows/test-migrations.yml'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}\n cancel-in-progress: true\n\njobs:\n test:\n strategy:\n fail-fast: false\n matrix:\n os: [ubuntu-latest, windows-latest, macos-latest]\n runs-on: ${{ matrix.os }}\n timeout-minutes: 20\n env:\n TEST_VERBOSE: 1\n IPFS_CHECK_RCMGR_DEFAULTS: 1\n defaults:\n run:\n shell: bash\n steps:\n - name: Check out Kubo\n uses: actions/checkout@v6\n\n - name: Set up Go\n uses: actions/setup-go@v6\n with:\n go-version-file: 'go.mod'\n\n - name: Build kubo binary\n run: |\n make build\n echo \"Built ipfs binary at $(pwd)/cmd/ipfs/\"\n\n - name: Add kubo to PATH\n run: |\n echo \"$(pwd)/cmd/ipfs\" >> $GITHUB_PATH\n\n - name: Verify ipfs in PATH\n run: |\n which ipfs || echo \"ipfs not in PATH\"\n ipfs version || echo \"Failed to run ipfs version\"\n\n - name: Run migration unit tests\n run: |\n go test ./repo/fsrepo/migrations/...\n\n - name: Run CLI migration tests\n env:\n IPFS_PATH: ${{ runner.temp }}/ipfs-test\n run: |\n export PATH=\"${{ github.workspace }}/cmd/ipfs:$PATH\"\n which ipfs || echo \"ipfs not found in PATH\"\n ipfs version || echo \"Failed to run ipfs version\"\n go test ./test/cli/migrations/...\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v7\n with:\n name: ${{ matrix.os }}-test-results\n path: |\n test/**/*.log\n ${{ runner.temp }}/ipfs-test/\n" }, { "path": ".gitignore", "content": "# ipfs can generate profiling dump files\n*.cpuprof\n*.memprof\n\n*.swp\n.ipfsconfig\n*.out\n*.coverprofile\n*.test\n*.orig\n*~\n\ncoverage.txt\ngx-workspace-update.json\n\n.ipfs\nbin/gx\nbin/protoc-*\nbin/gx*\nbin/tmp\nbin/gocovmerge\nbin/cover\n\n\nvendor\n.tarball\ngo-ipfs-source.tar.gz\ndocs/examples/go-ipfs-as-a-library/example-folder/Qm*\n/test/sharness/t0054-dag-car-import-export-data/*.car\n\n# test artifacts from make test_unit / test_cli\n/test/unit/gotest.json\n/test/unit/gotest.junit.xml\n/test/cli/cli-tests.json\n\n# ignore build output from snapcraft\n/ipfs_*.snap\n/parts\n/stage\n/prime\n" }, { "path": ".golangci.yml", "content": "linters:\n enable:\n - stylecheck\n\nlinters-settings:\n stylecheck:\n checks:\n - all\n - '-ST1003'\n dot-import-whitelist:\n - github.com/ipfs/kubo/test/cli/testutils\n" }, { "path": ".hadolint.yaml", "content": "# Hadolint configuration for Kubo Docker image\n# https://github.com/hadolint/hadolint\n\n# Ignore specific rules\nignored:\n # DL3008: Pin versions in apt-get install\n # We use stable base images and prefer smaller layers over version pinning\n - DL3008\n\n# Trust base images from these registries\ntrustedRegistries:\n - docker.io\n - gcr.io" }, { "path": ".mailmap", "content": "# This file allows re-mapping author names/emails\n# while maintaining the integrity of the repository\n#\n# Spec: https://www.kernel.org/pub/software/scm/git/docs/git-shortlog.html#_mapping_authors\n#\n\n@RubenKelevra \nAaron Hill \nAdam Gashlin \nAdam Uhlir \nAdin Schmahmann \nAdrian Lanzafame \nAdrian Ulrich \nAlan Shaw \nAlec Brickner \nAlex \nAlfie John \nAlfonso Montero \nAli Mirlou \nAndres Buritica \nAndrew Chin \nAndrew Nesbitt \nAndy Leap \nAntti Kaihola \nArtem Andreenko \nArthur Elliott \nBamvor Zhang \nBaptiste Jonglez \nBernhard M. Wiedemann \nBoris Mann \nBrendan Mc \nBrendan McMillion \nBrendan O'Brien \nBrian Tiger Chow \nBrian Tiger Chow \nCaian \nCaio Alonso \nCarlos Cobo \nCasey Chance \nCayman Nava \nChas Leichner \nChris Boddy \nChris Chinchilla \nChris Grimmett \nChris P \nChristopher Sasarak \nChristian Couder \nChristian Kniep \nChristopher Buesser \nCole Brown \nCorbin Page \nCornelius Toole \nDan <35669742+NukeManDan@users.noreply.github.com>\nDaniel Aleksandersen \nDaniel Grossmann-Kavanagh \nDaniel Mack \nDavid \nDavid Braun \nDavid Brennan \nDavid Dias \nDavid Wagner \nDevin \nDimitris Apostolou \nDiogo Silva \nDirk McCormick \nDjalil Dreamski <32184973+dreamski21@users.noreply.github.com>\nDominic Della Valle \nDominic Tarr \nIan Preston \nDylan Powers \nEdison Lee \nElias Gabrielsson \nEmery Hemingway \nEnrique Erne \nEoghan Ó Carragáin \nErik Ingenito \nEsteban \nEthan Buchman \nEtienne Laurin \nForrest Weston \nFrancesco Canessa \nFrank Sachsenheim \nFrederik Riedel \nFriedel Ziegelmayer \nGeorge Antoniadis \nGeorge Masgras \nGiulitti Salvatore \nGiuseppe Bertone \nGowtham G \nHarald Nordgren \nHarlan T Wood \nHector Sanjuan \nHenrique Dias \nHenry \nHerman Junge \nHlib \nHo-Sheng Hsiao \nHucg <41573766+hcg1314@users.noreply.github.com>\nIaroslav Gridin \nIgor Velkov \nIvan \nJP Hastings-Spital \nJack Loughran <30052269+jackloughran@users.noreply.github.com>\nJakub Sztandera \nJakub Kaczmarzyk \nJames Stanley \nJamie Wilkinson \nJan Winkelmann \nJason Carver \nJeff Thompson \nJeromy Johnson \nJesse Weinstein \nJessica Schilling \nJim McDonald \nJohn Reed \nJohnny <9611008+johnnymatthews@users.noreply.github.com>\nJon Choi \nJonathan Dahan \nJordan Danford \nJorropo \nJuan Batiz-Benet \nJustin Drake \nKacper Łukawski \nKarthik Bala \nKejie Zhang <601172892@qq.com>\nKerem \nKevin Atkinson \nKevin Simper \nKevin Wallace \nKirill Goncharov \nKishan Mohanbhai Sagathiya \nKnut Ahlers \nKonstantin Koroviev \nKoushik Roy \nKristoffer Ström \nKuro1 <412681778@qq.com>\nLars Gierth \nLeo Arias \nLi Zheng \nLorenzo Manacorda \nLorenzo Setale \nLouis Thibault \nLucas Garron \nLucas Molas \nMarcin Janczyk \nMarcin Rataj \nMarkus Amalthea Magnuson \nMarten Seemann \nMasashi Salvador Mitsuzawa \nMassino Tinovan \nMat Kelly \nMathijs de Bruin \nMatouš Skála \nMatt Bell \nMatt Joiner \nMax Chechel \nMax Kerp \nMib Kd743naq \nMichael Avila \nMichael Lovci \nMichael Muré \nMichael Pfister \nMichelle Lee \nMiguel Torres \nMikaela Suomalainen \nMildred Ki'Lya \nMolly \nMuneeb Ali \nMykola Nikishov \nNathan Musoke \nNick Hamann \nOli Evans \nOr Rikon \nOverbool \nPatrick Connolly \nPavol Rusnak \nPeter Borzov \nPeter Rabbitson \nPeter Wu \nPhilip Nelson \nPierre-Alain TORET \nPoorPockets McNewHold \nPretty Please Mark Darkly <55382229+pleasemarkdarkly@users.noreply.github.com>\nPéter Szilágyi \nQuantomic \nQuinn Slack \nRaúl Kripalani \nReadmeCritic \nRemco Bloemen \nRichard Littauer \nRideWindX \nRob Brackett \nRobert Carlsen \nRod Vagg \nRoerick Sweeney \nRoman Khafizianov \nRoman Proskuryakov \nRonsor \nRubenKelevra \nRyan Carver \nRyan Morey \nSH \nSag0Sag0 \nSander Pick \nScott Bigelow \nSean Lang \nShanti Bouchez-Mongardé \nShaun Bruce \nSherod Taylor \nSimon Kirkby \nSimon Menke \nSiraj Ravel \nSiva Chandran \nSpartucus \nStephan Kulla \nStephan Seidt \nStephen Sugden \nStephen Whitmore \nSteve Recio \nSteven Allen \nSteven Vandevelde \nSönke Hahn \nTUSF \nTarnay Kálmán \nThomas Gardner \nTiger \nTim Groeneveld \nTim Stahel \nTimothy Hobbs \nTom O'Donnell \nTom Swindell \nTommi Virtanen \nTonis Tiigi \nTor Arne Vestbø \nTravis Person \nTylar \nJohn Reed \nVasil Dimov \nVijayee Kulkaa \nVikram \nVitor Baptista \nW. Trevor King \nWes Morgan \nWill Scott \nWilli Butz \nXiaoyi Wang \nYuval Langer \nZander Mackie \nZenGround0 \nachingbrain \nadamliesko \nanarcat \nbbenshoof \ncamelmasa \nchenminjian <727180553@qq.com>\ndevedge \ndgrisham \ndrathir \nepitron \neric wu \nflowed \nforstmeier \nfyrchik \ngatesvp \nhannahhoward \nhikerpig \nhoenirvili \nhucg \nivan386 \nklauspost \nkpcyrd \nkvm2116 \nmateon1 \nmatrushka \nmichael \nmyself659 \nnmalhotra \npalkeo \nrequilence \nrht \nrob-deutsch \nslothbag \nsroerick \nswedneck <40505480+swedneck@users.noreply.github.com>\ntarekbadr \ntcme \ntg \ntheswitch \nverokarhu \nvitzli \nvyzo \nwzhd \nzramsay \nŁukasz Magiera \nᴍᴀᴛᴛ ʙᴇʟʟ \nᴠɪᴄᴛᴏʀ ʙᴊᴇʟᴋʜᴏʟᴍ \n" }, { "path": "AGENTS.md", "content": "# AI Agent Instructions for Kubo\n\nThis file provides instructions for AI coding agents working on the [Kubo](https://github.com/ipfs/kubo) codebase (the Go implementation of IPFS). Follow the [Developer Guide](docs/developer-guide.md) for full details.\n\n## Quick Reference\n\n| Task | Command |\n|-------------------|----------------------------------------------------------|\n| Tidy deps | `make mod_tidy` (run first if `go.mod` changed) |\n| Build | `make build` |\n| Unit tests | `go test ./... -run TestName -v` |\n| Integration tests | `make build && go test ./test/cli/... -run TestName -v` |\n| Lint | `make -O test_go_lint` |\n| Format | `go fmt ./...` |\n\n## Project Overview\n\nKubo is the reference implementation of IPFS in Go. Most IPFS protocol logic lives in [boxo](https://github.com/ipfs/boxo) (the IPFS SDK); kubo wires it together and exposes it via CLI and HTTP RPC API. If a change belongs in the protocol layer, it likely belongs in boxo, not here.\n\nKey directories:\n\n| Directory | Purpose |\n|--------------------|----------------------------------------------------------|\n| `cmd/ipfs/` | CLI entry point and binary |\n| `core/` | core IPFS node implementation |\n| `core/commands/` | CLI command definitions |\n| `core/coreapi/` | Go API implementation |\n| `client/rpc/` | HTTP RPC client |\n| `plugin/` | plugin system |\n| `repo/` | repository management |\n| `test/cli/` | Go-based CLI integration tests (preferred for new tests) |\n| `test/sharness/` | legacy shell-based integration tests |\n| `docs/` | documentation |\n\nOther key external dependencies: [go-libp2p](https://github.com/libp2p/go-libp2p) (networking), [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) (DHT).\n\n## Go Style\n\nFollow these Go style references:\n\n- [Go Code Review Comments](https://go.dev/wiki/CodeReviewComments)\n- [Google Go Style Decisions](https://google.github.io/styleguide/go/decisions)\n\nSpecific conventions for this project:\n\n- check the Go version in `go.mod` and use idiomatic features available at that version\n- readability over micro-optimization: clear code is more important than saving microseconds\n- prefer standard library functions and utilities over writing your own\n- use early returns and indent the error flow, not the happy path\n- use `slices.Contains`, `slices.DeleteFunc`, and the `maps` package instead of manual loops\n- preallocate slices and maps when the size is known: `make([]T, 0, n)`\n- use `map[K]struct{}` for sets, not `map[K]bool`\n- receiver names: single-letter abbreviations matching the type (e.g., `s *Server`, `c *Client`)\n- run `go fmt` after modifying Go source files, never indent manually\n\n### Error Handling\n\n- wrap errors with `fmt.Errorf(\"context: %w\", err)`, never discard errors silently\n- use `errors.Is` / `errors.As` for error checking, not string comparison\n- never use `panic` in library code; only in `main` or test helpers\n- return `nil` explicitly for the error value on success paths\n\n### Canonical Examples\n\nWhen adding or modifying code, follow the patterns established in these files:\n\n- CLI command structure: `core/commands/dag/dag.go`\n- CLI integration test: `test/cli/dag_test.go`\n- Test harness usage: `test/cli/harness/` package\n\n## Building\n\nAlways run commands from the repository root.\n\n```bash\nmake mod_tidy # update go.mod/go.sum (use this instead of go mod tidy)\nmake build # build the ipfs binary to cmd/ipfs/ipfs\nmake install # install to $GOPATH/bin\nmake -O test_go_lint # run linter (use this instead of golangci-lint directly)\n```\n\nIf you modify `go.mod` (add/remove/update dependencies), you must run `make mod_tidy` first, before building or testing. Use `make mod_tidy` instead of `go mod tidy` directly, as the project has multiple `go.mod` files.\n\nIf you modify any `.go` files outside of `test/`, you must run `make build` before running integration tests.\n\n## Testing\n\nThe full test suite is composed of several targets:\n\n| Make target | What it runs |\n|----------------------|-----------------------------------------------------------------------|\n| `make test` | all tests (`test_go_fmt` + `test_unit` + `test_cli` + `test_sharness`) |\n| `make test_short` | fast subset (`test_go_fmt` + `test_unit`) |\n| `make test_unit` | unit tests with coverage (excludes `test/cli`) |\n| `make test_cli` | CLI integration tests (requires `make build` first) |\n| `make test_sharness` | legacy shell-based integration tests |\n| `make test_go_fmt` | checks Go source formatting |\n| `make -O test_go_lint` | runs `golangci-lint` |\n\nDuring development, prefer running a specific test rather than the full suite:\n\n```bash\n# run a single unit test\ngo test ./core/... -run TestSpecificUnit -v\n\n# run a single CLI integration test (requires make build first)\ngo test ./test/cli/... -run TestSpecificCLI -v\n```\n\n### Environment Setup for Integration Tests\n\nBefore running `test_cli` or `test_sharness`, set these environment variables from the repo root:\n\n```bash\nexport PATH=\"$PWD/cmd/ipfs:$PATH\"\nexport IPFS_PATH=\"$(mktemp -d)\"\n```\n\n- `PATH`: integration tests use the `ipfs` binary from `PATH`, not Go source directly\n- `IPFS_PATH`: isolates test data from `~/.ipfs` or other running nodes\n\nIf you see \"version (N) is lower than repos (M)\", the `ipfs` binary in `PATH` is outdated. Rebuild with `make build` and verify `PATH`.\n\n### Running Sharness Tests\n\nSharness tests are legacy shell-based tests. Run individual tests with a timeout:\n\n```bash\ncd test/sharness && timeout 60s ./t0080-repo.sh\n```\n\nTo investigate a failing test, pass `-v` for verbose output. In this mode, daemons spawned by the test are not shut down automatically and must be killed manually afterwards.\n\n### Cleaning Up Stale Daemons\n\nBefore running `test/cli` or `test/sharness`, stop any stale `ipfs daemon` processes owned by the current user. Leftover daemons hold locks and bind ports, causing test failures:\n\n```bash\npkill -f \"ipfs daemon\"\n```\n\n### Writing Tests\n\n- all new integration tests go in `test/cli/`, not `test/sharness/`\n- if a `test/sharness` test needs significant changes, remove it and add a replacement in `test/cli/`\n- use [testify](https://github.com/stretchr/testify) for assertions (already a dependency)\n- for Go 1.25+, use `testing/synctest` when testing concurrent code (goroutines, channels, timers)\n- reuse existing `.car` fixtures in `test/cli/fixtures/` when possible; only add new fixtures when the test requires data not covered by existing ones\n- always re-run modified tests locally before submitting to confirm they pass\n- avoid emojis in test names and test log output\n\n## Before Submitting\n\nRun these steps in order before considering work complete:\n\n1. `make mod_tidy` (if `go.mod` changed)\n2. `go fmt ./...`\n3. `make build` (if non-test `.go` files changed)\n4. `make -O test_go_lint`\n5. `go test ./...` (or the relevant subset)\n\n## Documentation and Commit Messages\n\n- after editing CLI help text in `core/commands/`, verify width: `go test ./test/cli/... -run TestCommandDocsWidth`\n- config options are documented in `docs/config.md`\n- changelogs in `docs/changelogs/`: only edit the Table of Contents and the Highlights section; the Changelog and Contributors sections are auto-generated and must not be modified\n- follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)\n- keep commit titles short and messages terse\n\n## Writing Style\n\nWhen writing docs, comments, and commit messages:\n\n- avoid emojis in code, comments, and log output\n- keep an empty line before lists in markdown\n- use backticks around CLI commands, paths, environment variables, and config options\n\n## PR Guidelines\n\n- explain what changed and why in the PR description\n- include test coverage for new functionality and bug fixes\n- run `make -O test_go_lint` and fix any lint issues before submitting\n- verify that `go test ./...` passes locally\n- when modifying `test/sharness` tests significantly, migrate them to `test/cli` instead\n- end the PR description with a `## References` section listing related context, one link per line\n- if the PR closes an issue in `ipfs/kubo`, each closing reference should be a bullet starting with `Closes`:\n\n```markdown\n## References\n\n- Closes https://github.com/ipfs/kubo/issues/1234\n- Closes https://github.com/ipfs/kubo/issues/5678\n- https://discuss.ipfs.tech/t/related-topic/999\n```\n\n## Scope and Safety\n\nDo not modify or touch:\n\n- files under `test/sharness/lib/` (third-party sharness test framework)\n- CI workflows in `.github/` unless explicitly asked\n- auto-generated sections in `docs/changelogs/` (Changelog and Contributors are generated; only TOC and Highlights are human-edited)\n\nDo not run without being asked:\n\n- `make test` or `make test_sharness` (full suite is slow; prefer targeted tests)\n- `ipfs daemon` without a timeout\n\n## Running the Daemon\n\nAlways run the daemon with a timeout or shut it down promptly:\n\n```bash\ntimeout 60s ipfs daemon # auto-kill after 60s\nipfs shutdown # graceful shutdown via API\n```\n\nKill dangling daemons before re-running tests: `pkill -f \"ipfs daemon\"`\n" }, { "path": "CHANGELOG.md", "content": "# Kubo Changelogs\n\n- [v0.42](docs/changelogs/v0.42.md)\n- [v0.41](docs/changelogs/v0.41.md)\n- [v0.40](docs/changelogs/v0.40.md)\n- [v0.39](docs/changelogs/v0.39.md)\n- [v0.38](docs/changelogs/v0.38.md)\n- [v0.37](docs/changelogs/v0.37.md)\n- [v0.36](docs/changelogs/v0.36.md)\n- [v0.35](docs/changelogs/v0.35.md)\n- [v0.34](docs/changelogs/v0.34.md)\n- [v0.33](docs/changelogs/v0.33.md)\n- [v0.32](docs/changelogs/v0.32.md)\n- [v0.31](docs/changelogs/v0.31.md)\n- [v0.30](docs/changelogs/v0.30.md)\n- [v0.29](docs/changelogs/v0.29.md)\n- [v0.28](docs/changelogs/v0.28.md)\n- [v0.27](docs/changelogs/v0.27.md)\n- [v0.26](docs/changelogs/v0.26.md)\n- [v0.25](docs/changelogs/v0.25.md)\n- [v0.24](docs/changelogs/v0.24.md)\n- [v0.23](docs/changelogs/v0.23.md)\n- [v0.22](docs/changelogs/v0.22.md)\n- [v0.21](docs/changelogs/v0.21.md)\n- [v0.20](docs/changelogs/v0.20.md)\n- [v0.19](docs/changelogs/v0.19.md)\n- [v0.18](docs/changelogs/v0.18.md)\n- [v0.17](docs/changelogs/v0.17.md)\n- [v0.16](docs/changelogs/v0.16.md)\n- [v0.15](docs/changelogs/v0.15.md)\n- [v0.14](docs/changelogs/v0.14.md)\n- [v0.13](docs/changelogs/v0.13.md)\n- [v0.12](docs/changelogs/v0.12.md)\n- [v0.11](docs/changelogs/v0.11.md)\n- [v0.10](docs/changelogs/v0.10.md)\n- [v0.9](docs/changelogs/v0.9.md)\n- [v0.8](docs/changelogs/v0.8.md)\n- [v0.7](docs/changelogs/v0.7.md)\n- [v0.6](docs/changelogs/v0.6.md)\n- [v0.5](docs/changelogs/v0.5.md)\n- [v0.4](docs/changelogs/v0.4.md)\n- [v0.3](docs/changelogs/v0.3.md)\n- [v0.2](docs/changelogs/v0.2.md)\n" }, { "path": "CONTRIBUTING.md", "content": "# Contributing to Kubo\n\n**For development setup, building, and testing, see the [Developer Guide](docs/developer-guide.md).**\n\nIPFS as a project, including Kubo and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).\n\nWe also adhere to the [Go IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md) which provide additional information on how to collaborate and contribute to the Go implementation of IPFS.\n\nWe appreciate your time and attention for going over these. Please open an issue on ipfs/community if you have any questions.\n\nThank you.\n" }, { "path": "Dockerfile", "content": "# syntax=docker/dockerfile:1\n# Enables BuildKit with cache mounts for faster builds\nFROM --platform=${BUILDPLATFORM:-linux/amd64} golang:1.25 AS builder\n\nARG TARGETOS TARGETARCH\n\nENV SRC_DIR=/kubo\n\n# Cache go module downloads between builds for faster rebuilds\nCOPY go.mod go.sum $SRC_DIR/\nWORKDIR $SRC_DIR\nRUN --mount=type=cache,target=/go/pkg/mod \\\n go mod download\n\nCOPY . $SRC_DIR\n\n# Preload an in-tree but disabled-by-default plugin by adding it to the IPFS_PLUGINS variable\n# e.g. docker build --build-arg IPFS_PLUGINS=\"foo bar baz\"\nARG IPFS_PLUGINS\n\n# Allow for other targets to be built, e.g.: docker build --build-arg MAKE_TARGET=\"nofuse\"\nARG MAKE_TARGET=build\n\n# Build ipfs binary with cached go modules and build cache.\n# mkdir .git/objects allows git rev-parse to read commit hash for version info\nRUN --mount=type=cache,target=/go/pkg/mod \\\n --mount=type=cache,target=/root/.cache/go-build \\\n mkdir -p .git/objects \\\n && GOOS=$TARGETOS GOARCH=$TARGETARCH GOFLAGS=-buildvcs=false make ${MAKE_TARGET} IPFS_PLUGINS=$IPFS_PLUGINS\n\n# Extract required runtime tools from Debian.\n# We use Debian instead of Alpine because we need glibc compatibility\n# for the busybox base image we're using.\nFROM debian:bookworm-slim AS utilities\nRUN set -eux; \\\n\tapt-get update; \\\n\tapt-get install -y --no-install-recommends \\\n\t\ttini \\\n # Using gosu (~2MB) instead of su-exec (~20KB) because it's easier to\n # install on Debian. Useful links:\n # - https://github.com/ncopa/su-exec#why-reinvent-gosu\n # - https://github.com/tianon/gosu/issues/52#issuecomment-441946745\n\t\tgosu \\\n # fusermount enables IPFS mount commands\n fuse \\\n ca-certificates \\\n\t; \\\n\tapt-get clean; \\\n\trm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*\n\n# Final minimal image with shell for debugging (busybox provides sh)\nFROM busybox:stable-glibc\n\n# Copy ipfs binary, startup scripts, and runtime dependencies\nENV SRC_DIR=/kubo\nCOPY --from=utilities /usr/sbin/gosu /sbin/gosu\nCOPY --from=utilities /usr/bin/tini /sbin/tini\nCOPY --from=utilities /bin/fusermount /usr/local/bin/fusermount\nCOPY --from=utilities /etc/ssl/certs /etc/ssl/certs\nCOPY --from=builder $SRC_DIR/cmd/ipfs/ipfs /usr/local/bin/ipfs\nCOPY --from=builder --chmod=755 $SRC_DIR/bin/container_daemon /usr/local/bin/start_ipfs\nCOPY --from=builder $SRC_DIR/bin/container_init_run /usr/local/bin/container_init_run\n\n# Set SUID for fusermount to enable FUSE mounting by non-root user\nRUN chmod 4755 /usr/local/bin/fusermount\n\n# Swarm P2P port (TCP/UDP) - expose publicly for peer connections\nEXPOSE 4001 4001/udp\n# API port - keep private, only for trusted clients\nEXPOSE 5001\n# Gateway port - can be exposed publicly via reverse proxy\nEXPOSE 8080\n# Swarm WebSockets - expose publicly for browser-based peers\nEXPOSE 8081\n\n# Create ipfs user (uid 1000) and required directories with proper ownership\nENV IPFS_PATH=/data/ipfs\nRUN mkdir -p $IPFS_PATH /ipfs /ipns /mfs /container-init.d \\\n && adduser -D -h $IPFS_PATH -u 1000 -G users ipfs \\\n && chown ipfs:users $IPFS_PATH /ipfs /ipns /mfs /container-init.d\n\n# Volume for IPFS repository data persistence\nVOLUME $IPFS_PATH\n\n# The default logging level\nENV GOLOG_LOG_LEVEL=\"\"\n\n# Entrypoint initializes IPFS repo if needed and configures networking.\n# tini ensures proper signal handling and zombie process cleanup\nENTRYPOINT [\"/sbin/tini\", \"--\", \"/usr/local/bin/start_ipfs\"]\n\n# Health check verifies IPFS daemon is responsive.\n# Uses empty directory CID (QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn) as test\nHEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \\\n CMD ipfs --api=/ip4/127.0.0.1/tcp/5001 dag stat /ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn || exit 1\n\n# Default: run IPFS daemon with auto-migration enabled\nCMD [\"daemon\", \"--migrate=true\", \"--agent-version-suffix=docker\"]\n" }, { "path": "FUNDING.json", "content": "{\n \"opRetro\": {\n \"projectId\": \"0x7f330267969cf845a983a9d4e7b7dbcca5c700a5191269af377836d109e0bb69\"\n }\n}\n" }, { "path": "GNUmakefile", "content": "# General tools\n\nSHELL=PATH='$(PATH)' /bin/sh\n\n# enable second expansion\n.SECONDEXPANSION:\n\ninclude Rules.mk\n" }, { "path": "LICENSE", "content": "This project is transitioning from an MIT-only license to a dual MIT/Apache-2.0 license. \nUnless otherwise noted, all code contributed prior to 2019-05-06 and not contributed by \na user listed in [this signoff issue](https://github.com/ipfs/go-ipfs/issues/6302) is \nlicensed under MIT-only. All new contributions (and past contributions since 2019-05-06) \nare licensed under a dual MIT/Apache-2.0 license.\n\nMIT: https://www.opensource.org/licenses/mit\nApache-2.0: https://www.apache.org/licenses/LICENSE-2.0\n" }, { "path": "LICENSE-APACHE", "content": "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n" }, { "path": "LICENSE-MIT", "content": "The MIT License (MIT)\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n" }, { "path": "Makefile", "content": "all:\n\t@gmake $@\n.PHONY: all\n\n.DEFAULT:\n\t@gmake $@\n" }, { "path": "README.md", "content": "

\n
\n \"Kubo\n
\n Kubo: IPFS Implementation in Go\n
\n

\n\n

The first implementation of IPFS.

\n\n

\n \"Official\n \"Discourse\n \"Matrix\"\n \n \"GitHub\n

\n\n
\n\n

\nWhat is Kubo? | Quick Taste | Install | Documentation | Development | Getting Help\n

\n\n## What is Kubo?\n\nKubo was the first [IPFS](https://docs.ipfs.tech/concepts/what-is-ipfs/) implementation and is the [most widely used one today](https://probelab.io/ipfs/topology/#chart-agent-types-avg). It takes an opinionated approach to content-addressing ([CIDs](https://docs.ipfs.tech/concepts/glossary/#cid), [DAGs](https://docs.ipfs.tech/concepts/glossary/#dag)) that maximizes interoperability: [UnixFS](https://docs.ipfs.tech/concepts/glossary/#unixfs) for files and directories, [HTTP Gateways](https://docs.ipfs.tech/concepts/glossary/#gateway) for web browsers, [Bitswap](https://docs.ipfs.tech/concepts/glossary/#bitswap) and [HTTP](https://specs.ipfs.tech/http-gateways/trustless-gateway/) for verifiable data transfer.\n\n**Features:**\n\n- Runs an IPFS node as a network service (LAN [mDNS](https://github.com/libp2p/specs/blob/master/discovery/mdns.md) and WAN [Amino DHT](https://docs.ipfs.tech/concepts/glossary/#dht))\n- [Command-line interface](https://docs.ipfs.tech/reference/kubo/cli/) (`ipfs --help`)\n- [WebUI](https://github.com/ipfs/ipfs-webui/#readme) for node management\n- [HTTP Gateway](https://specs.ipfs.tech/http-gateways/) for trusted and [trustless](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval) content retrieval\n- [HTTP RPC API](https://docs.ipfs.tech/reference/kubo/rpc/) to control the daemon\n- [HTTP Routing V1](https://specs.ipfs.tech/routing/http-routing-v1/) client and server for [delegated routing](./docs/delegated-routing.md)\n- [Content blocking](./docs/content-blocking.md) for public node operators\n\n**Other IPFS implementations:** [Helia](https://github.com/ipfs/helia) (JavaScript), [more...](https://docs.ipfs.tech/concepts/ipfs-implementations/)\n\n## Quick Taste\n\nAfter [installing Kubo](#install), verify it works:\n\n```console\n$ ipfs init\ngenerating ED25519 keypair...done\npeer identity: 12D3KooWGcSLQdLDBi2BvoP8WnpdHvhWPbxpGcqkf93rL2XMZK7R\n\n$ ipfs daemon &\nDaemon is ready\n\n$ echo \"hello IPFS\" | ipfs add -q --cid-version 1\nbafkreicouv3sksjuzxb3rbb6rziy6duakk2aikegsmtqtz5rsuppjorxsa\n\n$ ipfs cat bafkreicouv3sksjuzxb3rbb6rziy6duakk2aikegsmtqtz5rsuppjorxsa\nhello IPFS\n```\n\nVerify this CID is provided by your node to the IPFS network: \n\nSee `ipfs add --help` for all import options. Ready for more? Follow the [command-line quick start](https://docs.ipfs.tech/how-to/command-line-quick-start/).\n\n## Install\n\nFollow the [official installation guide](https://docs.ipfs.tech/install/command-line/), or choose: [prebuilt binary](#official-prebuilt-binaries) | [Docker](#docker) | [package manager](#package-managers) | [from source](#build-from-source).\n\nPrefer a GUI? Try [IPFS Desktop](https://docs.ipfs.tech/install/ipfs-desktop/) and/or [IPFS Companion](https://docs.ipfs.tech/install/ipfs-companion/).\n\n### Minimal System Requirements\n\nKubo runs on most Linux, macOS, and Windows systems. For optimal performance, we recommend at least 6 GB of RAM and 2 CPU cores (more is ideal, as Kubo is highly parallel).\n\n> [!IMPORTANT]\n> Larger pinsets require additional memory, with an estimated ~1 GiB of RAM per 20 million items for reproviding to the Amino DHT.\n\n> [!CAUTION]\n> Systems with less than the recommended memory may experience instability, frequent OOM errors or restarts, and missing data announcement (reprovider window), which can make data fully or partially inaccessible to other peers. Running Kubo on underprovisioned hardware is at your own risk.\n\n### Official Prebuilt Binaries\n\nDownload from https://dist.ipfs.tech#kubo or [GitHub Releases](https://github.com/ipfs/kubo/releases/latest).\n\n### Docker\n\nOfficial images are published at https://hub.docker.com/r/ipfs/kubo/: [![Docker Image Version (latest semver)](https://img.shields.io/docker/v/ipfs/kubo?color=blue&label=kubo%20docker%20image&logo=docker&sort=semver&style=flat-square&cacheSeconds=3600)](https://hub.docker.com/r/ipfs/kubo/)\n\n#### 🟢 Release Images\n\nUse these for production deployments.\n\n- `latest` and [`release`](https://hub.docker.com/r/ipfs/kubo/tags?name=release) always point at [the latest stable release](https://github.com/ipfs/kubo/releases/latest)\n- [`vN.N.N`](https://hub.docker.com/r/ipfs/kubo/tags?name=v) points at a specific [release tag](https://github.com/ipfs/kubo/releases)\n\n```console\n$ docker pull ipfs/kubo:latest\n$ docker run --rm -it --net=host ipfs/kubo:latest\n```\n\nTo [customize your node](https://docs.ipfs.tech/install/run-ipfs-inside-docker/#customizing-your-node), pass config via `-e` or mount scripts in `/container-init.d`.\n\n#### 🟠 Developer Preview Images\n\nFor internal testing, not intended for production.\n\n- [`master-latest`](https://hub.docker.com/r/ipfs/kubo/tags?name=master-latest) points at `HEAD` of [`master`](https://github.com/ipfs/kubo/commits/master/)\n- [`master-YYYY-DD-MM-GITSHA`](https://hub.docker.com/r/ipfs/kubo/tags?name=master-2) points at a specific commit\n\n#### 🔴 Internal Staging Images\n\nFor testing arbitrary commits and experimental patches (force push to `staging` branch).\n\n- [`staging-latest`](https://hub.docker.com/r/ipfs/kubo/tags?name=staging-latest) points at `HEAD` of [`staging`](https://github.com/ipfs/kubo/commits/staging/)\n- [`staging-YYYY-DD-MM-GITSHA`](https://hub.docker.com/r/ipfs/kubo/tags?name=staging-2) points at a specific commit\n\n### Build from Source\n\n![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/ipfs/kubo?label=Requires%20Go&logo=go&style=flat-square&cacheSeconds=3600)\n\n```bash\ngit clone https://github.com/ipfs/kubo.git\ncd kubo\nmake build # creates cmd/ipfs/ipfs\nmake install # installs to $GOPATH/bin/ipfs\n```\n\nSee the [Developer Guide](docs/developer-guide.md) for details, Windows instructions, and troubleshooting.\n\n### Package Managers\n\nKubo is available in community-maintained packages across many operating systems, Linux distributions, and package managers. See [Repology](https://repology.org/project/kubo/versions) for the full list: [![Packaging status](https://repology.org/badge/tiny-repos/kubo.svg)](https://repology.org/project/kubo/versions)\n\n> [!WARNING]\n> These packages are maintained by third-party volunteers. The IPFS Project and Kubo maintainers are not responsible for their contents or supply chain security. For increased security, [build from source](#build-from-source).\n\n#### Linux\n\n| Distribution | Install | Version |\n|--------------|---------|---------|\n| Ubuntu | [PPA](https://launchpad.net/~twdragon/+archive/ubuntu/ipfs): `sudo apt install ipfs-kubo` | [![PPA: twdragon](https://img.shields.io/badge/PPA-twdragon-E95420?logo=ubuntu)](https://launchpad.net/~twdragon/+archive/ubuntu/ipfs) |\n| Arch | `pacman -S kubo` | [![Arch package](https://repology.org/badge/version-for-repo/arch/kubo.svg)](https://archlinux.org/packages/extra/x86_64/kubo/) |\n| Fedora | [COPR](https://copr.fedorainfracloud.org/coprs/taw/ipfs/): `dnf install kubo` | [![COPR: taw](https://img.shields.io/badge/COPR-taw-51A2DA?logo=fedora)](https://copr.fedorainfracloud.org/coprs/taw/ipfs/) |\n| Nix | `nix-env -i kubo` | [![nixpkgs unstable](https://repology.org/badge/version-for-repo/nix_unstable/kubo.svg)](https://search.nixos.org/packages?query=kubo) |\n| Gentoo | `emerge -a net-p2p/kubo` | [![Gentoo package](https://repology.org/badge/version-for-repo/gentoo/kubo.svg)](https://packages.gentoo.org/packages/net-p2p/kubo) |\n| openSUSE | `zypper install kubo` | [![openSUSE Tumbleweed](https://repology.org/badge/version-for-repo/opensuse_tumbleweed/kubo.svg)](https://software.opensuse.org/package/kubo) |\n| Solus | `sudo eopkg install kubo` | [![Solus package](https://repology.org/badge/version-for-repo/solus/kubo.svg)](https://packages.getsol.us/shannon/k/kubo/) |\n| Guix | `guix install kubo` | [![Guix package](https://repology.org/badge/version-for-repo/gnuguix/kubo.svg)](https://packages.guix.gnu.org/packages/kubo/) |\n| _other_ | [See Repology for the full list](https://repology.org/project/kubo/versions) | |\n\n~~Snap~~ no longer supported ([#8688](https://github.com/ipfs/kubo/issues/8688))\n\n#### macOS\n\n| Manager | Install | Version |\n|---------|---------|---------|\n| Homebrew | `brew install ipfs` | [![Homebrew](https://repology.org/badge/version-for-repo/homebrew/kubo.svg)](https://formulae.brew.sh/formula/ipfs) |\n| MacPorts | `sudo port install ipfs` | [![MacPorts](https://repology.org/badge/version-for-repo/macports/kubo.svg)](https://ports.macports.org/port/ipfs/) |\n| Nix | `nix-env -i kubo` | [![nixpkgs unstable](https://repology.org/badge/version-for-repo/nix_unstable/kubo.svg)](https://search.nixos.org/packages?query=kubo) |\n| _other_ | [See Repology for the full list](https://repology.org/project/kubo/versions) | |\n\n#### Windows\n\n| Manager | Install | Version |\n|---------|---------|---------|\n| Scoop | `scoop install kubo` | [![Scoop](https://repology.org/badge/version-for-repo/scoop/kubo.svg)](https://scoop.sh/#/apps?q=kubo) |\n| _other_ | [See Repology for the full list](https://repology.org/project/kubo/versions) | |\n\n~~Chocolatey~~ no longer supported ([#9341](https://github.com/ipfs/kubo/issues/9341))\n\n## Documentation\n\n| Topic | Description |\n|-------|-------------|\n| [Configuration](docs/config.md) | All config options reference |\n| [Environment variables](docs/environment-variables.md) | Runtime settings via env vars |\n| [Experimental features](docs/experimental-features.md) | Opt-in features in development |\n| [HTTP Gateway](docs/gateway.md) | Path, subdomain, and trustless gateway setup |\n| [HTTP RPC clients](docs/http-rpc-clients.md) | Client libraries for Go, JS |\n| [Delegated routing](docs/delegated-routing.md) | Multi-router and HTTP routing |\n| [Metrics & monitoring](docs/metrics.md) | Prometheus metrics |\n| [Content blocking](docs/content-blocking.md) | Denylist for public nodes |\n| [Customizing](docs/customizing.md) | Unsure if use Plugins, Boxo, or fork? |\n| [Debug guide](docs/debug-guide.md) | CPU profiles, memory analysis, tracing |\n| [Changelogs](docs/changelogs/) | Release notes for each version |\n| [All documentation](https://github.com/ipfs/kubo/tree/master/docs) | Full list of docs |\n\n## Development\n\nSee the [Developer Guide](docs/developer-guide.md) for build instructions, testing, and contribution workflow. AI coding agents should follow [AGENTS.md](AGENTS.md).\n\n## Getting Help\n\n- [IPFS Forum](https://discuss.ipfs.tech) - community support, questions, and discussion\n- [Community](https://docs.ipfs.tech/community/) - chat, events, and working groups\n- [GitHub Issues](https://github.com/ipfs/kubo/issues) - bug reports for Kubo specifically\n- [IPFS Docs Issues](https://github.com/ipfs/ipfs-docs/issues) - documentation issues\n\n## Security Issues\n\nSee [`SECURITY.md`](SECURITY.md).\n\n## Contributing\n\n[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)\n\nWe welcome contributions. See [CONTRIBUTING.md](CONTRIBUTING.md) and the [Developer Guide](docs/developer-guide.md).\n\nThis repository follows the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).\n\n## Maintainer Info\n\n\n\n> [!NOTE]\n> Kubo is maintained by the [Shipyard](https://ipshipyard.com/) team.\n>\n> [Release Process](https://ipshipyard.notion.site/Kubo-Release-Process-6dba4f5755c9458ab5685eeb28173778)\n\n## License\n\nDual-licensed under Apache 2.0 and MIT:\n\n- [LICENSE-APACHE](LICENSE-APACHE)\n- [LICENSE-MIT](LICENSE-MIT)\n" }, { "path": "Rules.mk", "content": "TGT_BIN :=\nCLEAN :=\nCOVERAGE :=\nDISTCLEAN :=\nTEST :=\nTEST_SHORT :=\nGOCC ?= go\nPROTOC ?= protoc\n\nall: help # all has to be first defined target\n.PHONY: all\n\ninclude mk/git.mk # has to be before tarball.mk\ninclude mk/tarball.mk\ninclude mk/util.mk\ninclude mk/golang.mk\n\n# -------------------- #\n# extra properties #\n# -------------------- #\n\nifeq ($(TEST_FUSE),0)\n\tGOTAGS += nofuse\nendif\nexport LIBP2P_TCP_REUSEPORT=false\n\n# -------------------- #\n# sub-files #\n# -------------------- #\ndir := bin\ninclude $(dir)/Rules.mk\n\n# tests need access to rules from plugin\ndir := plugin\ninclude $(dir)/Rules.mk\n\ndir := test\ninclude $(dir)/Rules.mk\n\ndir := cmd/ipfs\ninclude $(dir)/Rules.mk\n\n# include this file only if coverage target is executed\n# it is quite expensive\nifneq ($(filter coverage% clean distclean test/unit/gotest.junit.xml,$(MAKECMDGOALS)),)\n\t# has to be after cmd/ipfs due to PATH\n\tdir := coverage\n\tinclude $(dir)/Rules.mk\nendif\n\n# -------------------- #\n# universal rules #\n# -------------------- #\n\n%.pb.go: %.proto bin/protoc-gen-gogofaster\n\t$(PROTOC) --gogofaster_out=. --proto_path=.:$(GOPATH)/src:$(dir $@) $<\n\n# -------------------- #\n# core targets #\n# -------------------- #\n\nbuild: $(TGT_BIN)\n.PHONY: build\n\nclean:\n\trm -rf $(CLEAN)\n.PHONY: clean\n\nmod_tidy:\n\t@find . -name go.mod -execdir $(GOCC) mod tidy \\;\n.PHONY: mod_tidy\n\ncoverage: $(COVERAGE)\n.PHONY: coverage\n\ndistclean: clean\n\trm -rf $(DISTCLEAN)\n\tgit clean -ffxd\n.PHONY: distclean\n\ntest: $(TEST)\n.PHONY: test\n\ntest_short: $(TEST_SHORT)\n.PHONY: test_short\n\ndeps:\n.PHONY: deps\n\nnofuse: GOTAGS += nofuse\nnofuse: build\n.PHONY: nofuse\n\ninstall: cmd/ipfs-install\n.PHONY: install\n\ninstall_unsupported: install\n\t@echo \"/=======================================================================\\\\\"\n\t@echo '| |'\n\t@echo '| `make install_unsupported` is deprecated, use `make install` instead. |'\n\t@echo '| |'\n\t@echo \"\\\\=======================================================================/\"\n.PHONY: install_unsupported\n\nuninstall:\n\t$(GOCC) clean -i ./cmd/ipfs\n.PHONY: uninstall\n\nsupported:\n\t@echo \"Currently supported platforms (from .github/build-platforms.yml):\"\n\t@grep '^ - ' .github/build-platforms.yml | sed 's/^ - //' || (echo \"Error: .github/build-platforms.yml not found\"; exit 1)\n.PHONY: supported\n\nhelp:\n\t@echo 'DEPENDENCY TARGETS:'\n\t@echo ''\n\t@echo ' deps - Download dependencies using bundled gx'\n\t@echo ' test_sharness_deps - Download and build dependencies for sharness'\n\t@echo ''\n\t@echo 'BUILD TARGETS:'\n\t@echo ''\n\t@echo ' all - print this help message'\n\t@echo ' build - Build binary at ./cmd/ipfs/ipfs'\n\t@echo ' nofuse - Build binary with no fuse support'\n\t@echo ' install - Build binary and install into $$GOBIN'\n\t@echo ' mod_tidy - Remove unused dependencies from go.mod files'\n#\t@echo ' dist_install - TODO: c.f. ./cmd/ipfs/dist/README.md'\n\t@echo ''\n\t@echo 'CLEANING TARGETS:'\n\t@echo ''\n\t@echo ' clean - Remove files generated by build'\n\t@echo ' distclean - Remove files that are no part of a repository'\n\t@echo ' uninstall - Remove binary from $$GOPATH/bin'\n\t@echo ''\n\t@echo 'TESTING TARGETS:'\n\t@echo ''\n\t@echo ' test - Run all tests (test_go_fmt, test_unit, test_cli, test_sharness)'\n\t@echo ' test_short - Run fast tests (test_go_fmt, test_unit)'\n\t@echo ' test_unit - Run unit tests with coverage (excludes test/cli)'\n\t@echo ' test_cli - Run CLI integration tests (requires built binary)'\n\t@echo ' test_go_fmt - Check Go source formatting'\n\t@echo ' test_go_build - Build kubo for all platforms from .github/build-platforms.yml'\n\t@echo ' test_go_lint - Run golangci-lint'\n\t@echo ' test_sharness - Run sharness tests'\n\t@echo ' coverage - Collect coverage info from unit tests and sharness'\n\t@echo\n.PHONY: help\n" }, { "path": "SECURITY.md", "content": "# Security Policy\n\nThe IPFS protocol and its implementations are still in heavy development. This\nmeans that there may be problems in our protocols, or there may be mistakes in\nour implementations. We take security\nvulnerabilities very seriously. If you discover a security issue, please bring\nit to our attention right away!\n\n## Reporting a Vulnerability\n\nIf you find a vulnerability that may affect live deployments -- for example, by\nexposing a remote execution exploit -- please **send your report privately** to\nsecurity@ipfs.io. Please **DO NOT file a public issue**.\n\nIf the issue is a protocol weakness that cannot be immediately exploited or\nsomething not yet deployed, just discuss it openly.\n\n## Reporting a non security bug\n\nFor non-security bugs, please simply file a GitHub [issue](https://github.com/ipfs/go-ipfs/issues/new/choose).\n" }, { "path": "assets/README.md", "content": "# Assets loaded in with IPFS\n\nThis directory contains the go-ipfs assets:\n\n* Getting started documentation (`init-doc`).\n" }, { "path": "assets/assets.go", "content": "package assets\n\nimport (\n\t\"embed\"\n\t\"fmt\"\n\tgopath \"path\"\n\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tcid \"github.com/ipfs/go-cid\"\n)\n\n//go:embed init-doc\nvar Asset embed.FS\n\n// initDocPaths lists the paths for the docs we want to seed during --init.\nvar initDocPaths = []string{\n\tgopath.Join(\"init-doc\", \"about\"),\n\tgopath.Join(\"init-doc\", \"readme\"),\n\tgopath.Join(\"init-doc\", \"help\"),\n\tgopath.Join(\"init-doc\", \"contact\"),\n\tgopath.Join(\"init-doc\", \"security-notes\"),\n\tgopath.Join(\"init-doc\", \"quick-start\"),\n\tgopath.Join(\"init-doc\", \"ping\"),\n}\n\n// SeedInitDocs adds the list of embedded init documentation to the passed node, pins it and returns the root key.\nfunc SeedInitDocs(nd *core.IpfsNode) (cid.Cid, error) {\n\treturn addAssetList(nd, initDocPaths)\n}\n\nfunc addAssetList(nd *core.IpfsNode, l []string) (cid.Cid, error) {\n\tapi, err := coreapi.NewCoreAPI(nd)\n\tif err != nil {\n\t\treturn cid.Cid{}, err\n\t}\n\n\tdirMap := map[string]files.Node{}\n\n\tfor _, p := range l {\n\t\td, err := Asset.ReadFile(p)\n\t\tif err != nil {\n\t\t\treturn cid.Cid{}, fmt.Errorf(\"assets: could load Asset '%s': %s\", p, err)\n\t\t}\n\n\t\tdirMap[gopath.Base(p)] = files.NewBytesFile(d)\n\t}\n\n\tbasePath, err := api.Unixfs().Add(nd.Context(), files.NewMapDirectory(dirMap))\n\tif err != nil {\n\t\treturn cid.Cid{}, err\n\t}\n\n\tif err := api.Pin().Add(nd.Context(), basePath); err != nil {\n\t\treturn cid.Cid{}, err\n\t}\n\n\treturn basePath.RootCid(), nil\n}\n" }, { "path": "assets/init-doc/about", "content": "\n IPFS -- Inter-Planetary File system\n\nIPFS is a global, versioned, peer-to-peer filesystem. It combines good ideas\nfrom Git, BitTorrent, Kademlia, SFS, and the Web. It is like a single bit-\ntorrent swarm, exchanging git objects. IPFS provides an interface as simple\nas the HTTP web, but with permanence built-in. You can also mount the world\nat /ipfs.\n\nIPFS is a protocol:\n- defines a content-addressed file system\n- coordinates content delivery\n- combines Kademlia + BitTorrent + Git\n\nIPFS is a filesystem:\n- has directories and files\n- mountable filesystem (via FUSE)\n\nIPFS is a web:\n- can be used to view documents like the web\n- files accessible via HTTP at `http://ipfs.io/`\n- browsers or extensions can learn to use `ipfs://` directly\n- hash-addressed content guarantees the authenticity\n\nIPFS is modular:\n- connection layer over any network protocol\n- routing layer\n- uses a routing layer DHT (kademlia/coral)\n- uses a path-based naming service\n- uses BitTorrent-inspired block exchange\n\nIPFS uses crypto:\n- cryptographic-hash content addressing\n- block-level deduplication\n- file integrity + versioning\n- filesystem-level encryption + signing support\n\nIPFS is p2p:\n- worldwide peer-to-peer file transfers\n- completely decentralized architecture\n- **no** central point of failure\n\nIPFS is a CDN:\n- add a file to the filesystem locally, and it's now available to the world\n- caching-friendly (content-hash naming)\n- BitTorrent-based bandwidth distribution\n\nIPFS has a name service:\n- IPNS, an SFS inspired name system\n- global namespace based on PKI\n- serves to build trust chains\n- compatible with other NSes\n- can map DNS, .onion, .bit, etc to IPNS\n" }, { "path": "assets/init-doc/contact", "content": "Come hang out in our IRC chat room if you have any questions.\n\nContact the ipfs dev team:\n- Bugs: https://github.com/ipfs/go-ipfs/issues\n- Help: irc.freenode.org/#ipfs\n- Email: dev@ipfs.io\n" }, { "path": "assets/init-doc/docs/index", "content": "Index\n" }, { "path": "assets/init-doc/help", "content": "Some helpful resources for finding your way around ipfs:\n\n- quick-start: a quick show of various ipfs features.\n- ipfs commands: a list of all commands\n- ipfs --help: every command describes itself\n- https://github.com/ipfs/go-ipfs -- the src repository\n- #ipfs on irc.freenode.org -- the community IRC channel\n" }, { "path": "assets/init-doc/ping", "content": "ipfs" }, { "path": "assets/init-doc/quick-start", "content": "# 0.1 - Quick Start\n\nThis is a set of short examples with minimal explanation. It is meant as\na \"quick start\".\n\n\nAdd a file to ipfs:\n\n echo \"hello world\" >hello\n ipfs add hello\n\n\nView it:\n\n ipfs cat \n\n\nTry a directory:\n\n mkdir foo\n mkdir foo/bar\n echo \"baz\" > foo/baz\n echo \"baz\" > foo/bar/baz\n ipfs add -r foo\n\n\nView things:\n\n ipfs ls \n ipfs ls /bar\n ipfs cat /baz\n ipfs cat /bar/baz\n ipfs cat /bar\n ipfs ls /baz\n\n\nReferences:\n\n ipfs refs \n ipfs refs -r \n ipfs refs --help\n\n\nGet:\n\n ipfs get -o foo2\n diff foo foo2\n\n\nObjects:\n\n ipfs object get \n ipfs object get /foo2\n ipfs object --help\n\n\nPin + GC:\n\n ipfs pin add \n ipfs repo gc\n ipfs ls \n ipfs pin rm \n ipfs repo gc\n\n\nDaemon:\n\n ipfs daemon (in another terminal)\n ipfs id\n\n\nNetwork:\n\n (must be online)\n ipfs swarm peers\n ipfs id\n ipfs cat \n\n\nMount:\n\n (warning: fuse is finicky!)\n ipfs mount\n cd /ipfs/\n ls\n\n\nTool:\n\n ipfs version\n ipfs update\n ipfs commands\n ipfs config --help\n open http://localhost:5001/webui\n\n\nBrowse:\n\n WebUI:\n\n http://localhost:5001/webui\n\n video:\n\n http://localhost:8080/ipfs/QmVc6zuAneKJzicnJpfrqCH9gSy6bz54JhcypfJYhGUFQu/play#/ipfs/QmTKZgRNwDNZwHtJSjCp6r5FYefzpULfy37JvMt9DwvXse\n\n images:\n\n http://localhost:8080/ipfs/QmZpc3HvfjEXvLWGQPWbHk3AjD5j8NEN4gmFN8Jmrd5g83/cs\n\n markdown renderer app:\n\n http://localhost:8080/ipfs/QmX7M9CiYXjVeFnkfVGf3y5ixTZ2ACeSGyL1vBJY1HvQPp/mdown\n" }, { "path": "assets/init-doc/readme", "content": "Hello and Welcome to IPFS!\n\n██╗██████╗ ███████╗███████╗\n██║██╔══██╗██╔════╝██╔════╝\n██║██████╔╝█████╗ ███████╗\n██║██╔═══╝ ██╔══╝ ╚════██║\n██║██║ ██║ ███████║\n╚═╝╚═╝ ╚═╝ ╚══════╝\n\nIf you're seeing this, you have successfully installed\nIPFS and are now interfacing with the ipfs merkledag!\n\n -------------------------------------------------------\n| Warning: |\n| This is alpha software. Use at your own discretion! |\n| Much is missing or lacking polish. There are bugs. |\n| Not yet secure. Read the security notes for more. |\n -------------------------------------------------------\n\nCheck out some of the other files in this directory:\n\n ./about\n ./help\n ./quick-start <-- usage examples\n ./readme <-- this file\n ./security-notes\n" }, { "path": "assets/init-doc/security-notes", "content": " IPFS Alpha Security Notes\n\nWe try hard to ensure our system is safe and robust, but all software\nhas bugs, especially new software. This distribution is meant to be an\nalpha preview, don't use it for anything mission critical.\n\nPlease note the following:\n\n- This is alpha software and has not been audited. It is our goal\n to conduct a proper security audit once we close in on a 1.0 release.\n\n- ipfs is a networked program, and may have serious undiscovered\n vulnerabilities. It is written in Go, and we do not execute any\n user provided data. But please point any problems out to us in a\n github issue, or email security@ipfs.io privately.\n\n- security@ipfs.io GPG key:\n - 4B9665FB 92636D17 7C7A86D3 50AAE8A9 59B13AF3\n - https://pgp.mit.edu/pks/lookup?op=get&search=0x50AAE8A959B13AF3\n\n- ipfs uses encryption for all communication, but it's NOT PROVEN SECURE\n YET! It may be totally broken. For now, the code is included to make\n sure we benchmark our operations with encryption in mind. In the future,\n there will be an \"unsafe\" mode for high performance intranet apps.\n If this is a blocking feature for you, please contact us.\n" }, { "path": "bin/Rules.mk", "content": "include mk/header.mk\n\ndist_root_$(d)=\"/ipfs/QmPrXH9jRVwvd7r5MC5e6nV4uauQGzLk1i2647Ye9Vbbwe\"\n\nTGTS_$(d) := $(d)/protoc\nDISTCLEAN += $(d)/protoc $(d)/tmp\n\nPATH := $(realpath $(d)):$(PATH)\n\n$(TGTS_$(d)):\n\trm -f $@$(?exe)\nifeq ($(WINDOWS),1)\n\tcp $^$(?exe) $@$(?exe)\nelse\n\tln -s $(notdir $^) $@\nendif\n\nbin/protoc-gen-gogofaster:\n\t$(call go-build,github.com/gogo/protobuf/protoc-gen-gogofaster)\n\nCLEAN += $(TGTS_$(d))\ninclude mk/footer.mk\n" }, { "path": "bin/archive-branches.sh", "content": "#!/usr/bin/env bash\nset -euo pipefail\nIFS=$'\\n\\t'\n\nauth=\"\"\n#auth=\"-u kubuxu:$GH_TOKEN\"\norg=ipfs\nrepo=go-ipfs\narch_repo=go-ipfs-archived\napi_repo=\"repos/$org/$repo\"\n\nexclusions=(\n\t'master'\n\t'release'\n\t'feat/zcash'\n\t'feat/ai-mirror'\n)\n\ngh_api_next() {\n\tlinks=$(grep '^Link:' | sed -e 's/Link: //' -e 's/, /\\n/g')\n\techo \"$links\" | grep '; rel=\"next\"' >/dev/null || return\n\tlink=$(echo \"$links\" | grep '; rel=\"next\"' | sed -e 's/^.*//')\n\n\tcurl $auth -f -sD >(gh_api_next) \"$link\"\n}\n\ngh_api() {\n\tcurl $auth -f -sD >(gh_api_next) \"https://api.github.com/$1\" | jq -s '[.[] | .[]]'\n}\n\npr_branches() {\n\tgh_api \"$api_repo/pulls\" | jq -r '.[].head.label | select(test(\"^ipfs:\"))' \\\n\t\t| sed 's/^ipfs://'\n}\n\norigin_refs() {\n\tformat=${1-'%(refname:short)'}\n\n\tgit for-each-ref --format \"$format\" refs/remotes/origin | sed 's|^origin/||'\n}\n\nactive_branches() {\n\torigin_refs '%(refname:short) %(committerdate:unix)' |awk \\\n'\tBEGIN { monthAgo = systime() - 31*24*60*60 }\n\t{ if ($2 > monthAgo) print $1 }\n'\n}\n\ngit remote add archived \"git@github.com:$org/$arch_repo.git\" || true\n\nbranches_to_move=\"$(cat <(active_branches) <(pr_branches) <((IFS=$'\\n'; echo \"${exclusions[*]}\")) | sort -u | comm - <(origin_refs | sort) -13)\"\n\necho \"================\"\nprintf \"%s\\n\" \"$branches_to_move\"\necho \"================\"\n\necho \"Please confirm move of above branches [y/N]:\"\n\nread line\ncase $line in\n [Yy]|[Yy][Ee][Ss]) ;;\n *) exit 1 ;;\nesac\n\n\nprintf \"%s\\n\" \"$branches_to_move\" | \\\nwhile read -r ref; do\n\t\tgit push archived \"origin/$ref:refs/heads/$ref/$(date --rfc-3339=date)\"\n\t\tgit push origin --delete \"$ref\"\n\tdone\n\n" }, { "path": "bin/container_daemon", "content": "#!/bin/sh\nset -e\n\nuser=ipfs\nrepo=\"$IPFS_PATH\"\n\nif [ \"$(id -u)\" -eq 0 ]; then\n echo \"Changing user to $user\"\n # ensure folder is writable\n gosu \"$user\" test -w \"$repo\" || chown -R -- \"$user\" \"$repo\"\n # restart script with new privileges\n exec gosu \"$user\" \"$0\" \"$@\"\nfi\n\n# 2nd invocation with regular user\nipfs version\n\n\nif [ -e \"$repo/config\" ]; then\n echo \"Found IPFS fs-repo at $repo\"\nelse\n ipfs init ${IPFS_PROFILE:+\"--profile=$IPFS_PROFILE\"}\n ipfs config Addresses.API /ip4/0.0.0.0/tcp/5001\n ipfs config Addresses.Gateway /ip4/0.0.0.0/tcp/8080\n\n # Set up the swarm key, if provided\n\n SWARM_KEY_FILE=\"$repo/swarm.key\"\n SWARM_KEY_PERM=0400\n\n # Create a swarm key from a given environment variable\n if [ -n \"$IPFS_SWARM_KEY\" ] ; then\n echo \"Copying swarm key from variable...\"\n printf \"%s\\n\" \"$IPFS_SWARM_KEY\" >\"$SWARM_KEY_FILE\" || exit 1\n chmod $SWARM_KEY_PERM \"$SWARM_KEY_FILE\"\n fi\n\n # Unset the swarm key variable\n unset IPFS_SWARM_KEY\n\n # Check during initialization if a swarm key was provided and\n # copy it to the ipfs directory with the right permissions\n # WARNING: This will replace the swarm key if it exists\n if [ -n \"$IPFS_SWARM_KEY_FILE\" ] ; then\n echo \"Copying swarm key from file...\"\n install -m $SWARM_KEY_PERM \"$IPFS_SWARM_KEY_FILE\" \"$SWARM_KEY_FILE\" || exit 1\n fi\n\n # Unset the swarm key file variable\n unset IPFS_SWARM_KEY_FILE\nfi\n\nfind /container-init.d -maxdepth 1 \\( -type f -o -type l \\) -iname '*.sh' -print0 | sort -z | xargs -n 1 -0 -r container_init_run\n\nexec ipfs \"$@\"\n" }, { "path": "bin/container_init_run", "content": "#!/bin/sh\n\nset -e\n\n# used by the container startup script for running initialization scripts\n\nscript=\"$1\"\nif [ -x \"$script\" ] ; then\n printf \"Executing '%s'...\\n\" \"$script\"\n \"$script\"\nelse\n printf \"Sourcing '%s'...\\n\" \"$script\"\n . \"$script\"\nfi\n" }, { "path": "bin/dist_get", "content": "#!/bin/sh\n\nGOCC=${GOCC=go}\n\ndie() {\n\techo \"$@\" >&2\n\texit 1\n}\n\nhave_binary() {\n\ttype \"$1\" > /dev/null 2> /dev/null\n}\n\ncheck_writable() {\n\tprintf \"\" > \"$1\" && rm \"$1\"\n}\n\ntry_download() {\n\turl=\"$1\"\n\toutput=\"$2\"\n\tcommand=\"$3\"\n\tutil_name=\"$(set -- $command; echo \"$1\")\"\n\n\tif ! have_binary \"$util_name\"; then\n\t\treturn 1\n\tfi\n\n\tprintf '==> Using %s to download \"%s\" to \"%s\"\\n' \"$util_name\" \"$url\" \"$output\"\n\tif eval \"$command\"; then\n\t\techo \"==> Download complete!\"\n\t\treturn\n\telse\n\t\techo \"error: couldn't download with $util_name ($?)\"\n\t\treturn 1\n\tfi\n}\n\ndownload() {\n\tdl_url=\"$1\"\n\tdl_output=\"$2\"\n\n\ttest \"$#\" -eq \"2\" || die \"download requires exactly two arguments, was given $@\"\n\n\tif ! check_writable \"$dl_output\"; then\n\t\tdie \"download error: cannot write to $dl_output\"\n\tfi\n\n\ttry_download \"$dl_url\" \"$dl_output\" \"wget '$dl_url' -O '$dl_output'\" && return\n\ttry_download \"$dl_url\" \"$dl_output\" \"curl --silent --fail --output '$dl_output' '$dl_url'\" && return\n\ttry_download \"$dl_url\" \"$dl_output\" \"fetch '$dl_url' -o '$dl_output'\" && return\n\ttry_download \"$dl_url\" \"$dl_output\" \"http '$dl_url' > '$dl_output'\" && return\n\ttry_download \"$dl_url\" \"$dl_output\" \"ftp -o '$dl_output' '$dl_url'\" && return\n\n\tdie \"Unable to download $dl_url. exiting.\"\n}\n\nunarchive() {\n\tua_archivetype=\"$1\"\n\tua_infile=\"$2\"\n\tua_outfile=\"$3\"\n\tua_distname=\"$4\"\n\tua_binpostfix=\"\"\n\tua_os=$(uname -o)\n\n\tif [ \"$ua_os\" = \"Msys\" ] || [ \"$ua_os\" = \"Cygwin\" ] ; then\n\t ua_binpostfix=\".exe\"\n\tfi\n\tua_outfile=\"$ua_outfile$ua_binpostfix\"\n\n\tif ! check_writable \"$ua_outfile\"; then\n\t\tdie \"unarchive error: cannot write to $ua_outfile\"\n\tfi\n\n\tcase \"$ua_archivetype\" in\n\t\ttar.gz)\n\t\t\tif have_binary tar; then\n\t\t\t\techo \"==> using 'tar' to extract binary from archive\"\n\t\t\t\t< \"$ua_infile\" tar -Ozxf - \"$ua_distname/$ua_distname$ua_binpostfix\" > \"$ua_outfile\" \\\n\t\t\t\t\t|| die \"tar has failed\"\n\t\t\telse\n\t\t\t\tdie \"no binary on system for extracting tar files\"\n\t\t\tfi\n\t\t\t;;\n\t\tzip)\n\t\t\tif have_binary unzip; then\n\t\t\t\techo \"==> using 'unzip' to extract binary from archive\"\n\t\t\t\tunzip -p \"$ua_infile\" \"$ua_distname/$ua_distname$ua_binpostfix\" > \"$ua_outfile\" \\\n\t\t\t\t\t|| die \"unzip has failed\"\n\t\t\telse\n\t\t\t\tdie \"no installed method for extracting .zip archives\"\n\t\t\tfi\n\t\t\t;;\n\t\t*)\n\t\t\tdie \"unrecognized archive type '$ua_archivetype'\"\n\tesac\n\n\tchmod +x \"$ua_outfile\" || die \"chmod has failed\"\n}\n\nget_go_vars() {\n\tif [ ! -z \"$GOOS\" ] && [ ! -z \"$GOARCH\" ]; then\n\t\tprintf \"%s-%s\" \"$GOOS\" \"$GOARCH\"\n\telif have_binary go; then\n\t\tprintf \"%s-%s\" \"$($GOCC env GOOS)\" \"$($GOCC env GOARCH)\"\n\telse\n\t\tdie \"no way of determining system GOOS and GOARCH\\nPlease manually set GOOS and GOARCH then retry.\"\n\tfi\n}\n\nmkurl() {\n\tm_root=\"$1\"\n\tm_name=\"$2\"\n\tm_vers=\"$3\"\n\tm_archive=\"$4\"\n\tm_govars=$(get_go_vars) || die \"could not get go env vars\"\n\n\techo \"https://ipfs.io$m_root/$m_name/$m_vers/${m_name}_${m_vers}_$m_govars.$m_archive\"\n}\n\ndistroot=\"$1\"\ndistname=\"$2\"\noutpath=\"$3\"\nversion=\"$4\"\n\nif [ -z \"$distroot\" ] || [ -z \"$distname\" ] || [ -z \"$outpath\" ] || [ -z \"$version\" ]; then\n\tdie \"usage: dist_get \"\nfi\n\ncase $version in\n\tv*)\n\t\t# correct input\n\t\t;;\n\t*)\n\t\techo \"invalid version '$version'\" >&2\n\t\tdie \"versions must begin with 'v', for example: v0.4.0\"\n\t\t;;\nesac\n\n# TODO: don't depend on the go tool being installed to detect this\ngoenv=$(get_go_vars) || die \"could not get go env vars\"\n\ncase $goenv in\n\tlinux-*)\n\t\tarchive=\"tar.gz\"\n\t\t;;\n\tdarwin-*)\n\t\tarchive=\"tar.gz\"\n\t\t;;\n\twindows-*)\n\t\tarchive=\"zip\"\n\t\t;;\n\tfreebsd-*)\n\t\tarchive=\"tar.gz\"\n\t\t;;\n\topenbsd-*)\n\t\tarchive=\"tar.gz\"\n\t\t;;\n\t*)\n\t\techo \"unrecognized system environment: $goenv\" >&2\n\t\tdie \"currently only linux, darwin, windows and freebsd are supported by this script\"\nesac\n\n\nmkdir -p bin/tmp\n\nurl=$(mkurl \"$distroot\" \"$distname\" \"$version\" \"$archive\")\ntmpfi=\"bin/tmp/$distname.$archive\"\n\ndownload \"$url\" \"$tmpfi\"\nif [ $? -ne 0 ]; then\n\tdie \"failed to download $url to $tmpfi\"\nfi\n\nunarchive \"$archive\" \"$tmpfi\" \"$outpath\" \"$distname\"\nif [ $? -ne 0 ]; then\n\tdie \"failed to extract archive $tmpfi\"\nfi\n" }, { "path": "bin/gencmdref", "content": "#!/usr/bin/env python\n\nimport os\nimport sys\nimport datetime\n\nfrom subprocess import check_output\n\ndef run(cmd):\n return check_output(cmd)\n\ndef main():\n lines = [l.strip() for l in sys.stdin]\n\n print '# ipfs command reference'\n print ''\n print 'generated on', datetime.datetime.now()\n print ''\n for line in lines:\n print '- [%s](#%s)' % (line, line.replace(' ', '-'))\n print ''\n\n for line in lines:\n print '## %s' % line\n print ''\n print '```'\n print run((line + ' --help').split(' ')).strip()\n print '```'\n print ''\n\nif __name__ == '__main__':\n if '-h' in sys.argv or '--help' in sys.argv:\n print 'usage: ipfs commands | %s >cmdref.md' % sys.argv[0]\n print 'outputs all commands with --help to a markdown file'\n exit(0)\n\n main()\n" }, { "path": "bin/get-docker-tags.sh", "content": "#!/usr/bin/env bash\n\n# get-docker-tags.sh\n#\n# Usage:\n# ./get-docker-tags.sh [git tag name]\n#\n# Example:\n#\n# # get tag for the master branch\n# ./get-docker-tags.sh $(date -u +%F) testingsha master\n#\n# # get tag for a release tag\n# ./get-docker-tags.sh $(date -u +%F) testingsha release v0.5.0\n#\nset -euo pipefail\n\nif [[ $# -lt 1 ]] ; then\n echo 'At least 1 arg required.'\n echo 'Usage:'\n echo './get-docker-tags.sh [git commit sha1] [git branch name] [git tag name]'\n exit 1\nfi\n\nBUILD_NUM=$1\nGIT_SHA1=${2:-$(git rev-parse HEAD)}\nGIT_SHA1_SHORT=$(echo \"$GIT_SHA1\" | cut -c 1-7)\nGIT_BRANCH=${3:-$(git symbolic-ref -q --short HEAD || echo \"unknown\")}\nGIT_TAG=${4:-$(git describe --tags --exact-match 2> /dev/null || echo \"\")}\n\nIMAGE_NAME=${IMAGE_NAME:-ipfs/kubo}\n\nechoImageName () {\n local IMAGE_TAG=$1\n echo \"$IMAGE_NAME:$IMAGE_TAG\"\n}\n\nif [[ $GIT_TAG =~ ^v[0-9]+\\.[0-9]+\\.[0-9]+-rc ]]; then\n echoImageName \"$GIT_TAG\"\n\nelif [[ $GIT_TAG =~ ^v[0-9]+\\.[0-9]+\\.[0-9]+$ ]]; then\n echoImageName \"$GIT_TAG\"\n echoImageName \"latest\"\n echoImageName \"release\" # see: https://github.com/ipfs/kubo/issues/3999#issuecomment-742228981\n\nelif [[ $GIT_BRANCH =~ ^bifrost-.* ]]; then\n # sanitize the branch name since docker tags have stricter char limits than git branch names\n branch=$(echo \"$GIT_BRANCH\" | tr '/' '-' | tr --delete --complement '[:alnum:]-')\n echoImageName \"${branch}-${BUILD_NUM}-${GIT_SHA1_SHORT}\"\n\nelif [ \"$GIT_BRANCH\" = \"master\" ] || [ \"$GIT_BRANCH\" = \"staging\" ]; then\n echoImageName \"${GIT_BRANCH}-${BUILD_NUM}-${GIT_SHA1_SHORT}\"\n echoImageName \"${GIT_BRANCH}-latest\"\n\nelse\n echo \"Nothing to do. No docker tag defined for branch: $GIT_BRANCH, tag: $GIT_TAG\"\n\nfi\n" }, { "path": "bin/graphmd", "content": "#!/bin/sh\n\nif [ \"$#\" -ne 1 ]; then\n echo \"usage: $0 ...\"\n echo \"output merkledag links in graphviz dot\"\n echo \"\"\n echo \"use it with dot:\"\n echo \"\t$0 QmZPAMWUfLD95GsdorXt9hH7aVrarb2SuLDMVVe6gABYmx | dot -Tsvg\"\n echo \"\t$0 QmZPAMWUfLD95GsdorXt9hH7aVrarb2SuLDMVVe6gABYmx | dot -Tpng\"\n echo \"\t$0 QmZPAMWUfLD95GsdorXt9hH7aVrarb2SuLDMVVe6gABYmx | dot -Tpdf\"\n echo \"\"\n exit 1\nfi\n\nsrc=' [fontsize=8 shape=box];'\ndst=' [fontsize=8 shape=box];'\nedge=' -> [label=\"\"];'\nfmt=\"$src\n$dst\n$edge\"\n\necho \"digraph {\"\necho \"\tgraph [rankdir=LR];\"\nipfs refs -r --format=\"$fmt\" \"$@\" | awk '{ print \"\\t\" $0 }'\n# ipfs refs -r --format=\"$fmt\" \"$@\" | awk '{ print \"\\t\" $0 }' | unflatten -l3\necho \"}\"\n" }, { "path": "bin/ipns-republish", "content": "#!/bin/bash\n\nif [ \"$#\" -ne 1 ]; then\n echo \"usage: $0 \"\n echo \"republishes an ipns name every 20 minutes\"\n echo \"(this is an icky stop-gap until ipfs nodes do it for you)\"\n echo \"\"\n echo \"example:\"\n echo \" > $0 QmSYCpuKPbPQ2iFr2swJj2hvz7wQUXfPBXPiuVsQdL5FEs\"\n echo \"\"\n exit 1\nfi\n\n# must be run online.\nipfs swarm peers >/dev/null\nif [ $? -ne 0 ]; then\n echo \"error: ipfs daemon must be online and connected to peers \"\n exit 1\nfi\n\n# check the object is there\nipfs dag stat \"$1\" >/dev/null\nif [ $? -ne 0 ]; then\n echo \"error: ipfs cannot find $1\"\n exit 1\nfi\n\necho \"republishing $1 every 20 minutes\"\nwhile :\ndo\n ipfs name publish $1\n sleep 1200\ndone\n" }, { "path": "bin/maketarball.sh", "content": "#!/usr/bin/env bash\n# vim: set expandtab sw=2 ts=2:\n\n# bash safe mode\nset -euo pipefail\nIFS=$'\\n\\t'\n\n# readlink doesn't work on macos\nOUTPUT=\"${1:-go-ipfs-source.tar.gz}\"\nif ! [[ \"$OUTPUT\" = /* ]]; then\n OUTPUT=\"$PWD/$OUTPUT\"\nfi\n\nGOCC=${GOCC=go}\n\nTEMP=\"$(mktemp -d)\"\ncp -r . \"$TEMP\"\n( cd \"$TEMP\" &&\n echo $PWD &&\n $GOCC mod vendor &&\n (git describe --always --match=NeVeRmAtCh --dirty 2>/dev/null || true) > .tarball &&\n chmod -R u=rwX,go=rX \"$TEMP\" # normalize permissions\n tar -czf \"$OUTPUT\" --exclude=\"./.git\" .\n )\n\nrm -rf \"$TEMP\"\n" }, { "path": "bin/mkreleaselog", "content": "#!/bin/bash\n#\n# Invocation: mkreleaselog [FIRST_REF [LAST_REF]]\n#\n# Generates release notes with contributor statistics, deduplicating by GitHub handle.\n# GitHub handles are resolved from:\n# 1. GitHub noreply emails (user@users.noreply.github.com)\n# 2. Merge commit messages (Merge pull request #N from user/branch)\n# 3. GitHub API via gh CLI (for squash merges)\n#\n# Results are cached in ~/.cache/mkreleaselog/github-handles.json\n\nset -euo pipefail\nexport GO111MODULE=on\nGOPATH=\"$(go env GOPATH)\"\nexport GOPATH\n\n# List of PCRE regular expressions to match \"included\" modules.\nINCLUDE_MODULES=(\n # orgs\n \"^github.com/ipfs/\"\n \"^github.com/ipld/\"\n \"^github.com/libp2p/\"\n \"^github.com/multiformats/\"\n \"^github.com/filecoin-project/\"\n \"^github.com/ipfs-shipyard/\"\n \"^github.com/ipshipyard/\"\n \"^github.com/probe-lab/\"\n\n # Authors of personal modules used by go-ipfs that should be mentioned in the\n # release notes.\n \"^github.com/whyrusleeping/\"\n \"^github.com/gammazero/\"\n \"^github.com/Jorropo/\"\n \"^github.com/guillaumemichel/\"\n \"^github.com/Kubuxu/\"\n \"^github.com/jbenet/\"\n \"^github.com/Stebalien/\"\n \"^github.com/marten-seemann/\"\n \"^github.com/hsanjuan/\"\n \"^github.com/lucas-clemente/\"\n \"^github.com/warpfork/\"\n)\n\n# List of PCRE regular expressions to match \"excluded\" modules. Applied after includes.\nEXCLUDE_MODULES=(\n \"^github.com/marten-seemann/qtls\"\n)\n\n# Ignored files as git pathspecs. These patters will match any full path component.\nIGNORE_FILES=(\n \".gx\"\n \"package.json\"\n \".travis.yml\"\n \"go.mod\"\n \"go.sum\"\n \".github\"\n \"*.pb.go\"\n \"cbor_gen.go\"\n \"ipldsch_*.go\"\n \"*.gen.go\"\n)\n\n##########################################################################################\n# GitHub Handle Resolution Infrastructure\n##########################################################################################\n\n# Cache location following XDG spec\nGITHUB_CACHE_DIR=\"${XDG_CACHE_HOME:-$HOME/.cache}/mkreleaselog\"\nGITHUB_CACHE_FILE=\"$GITHUB_CACHE_DIR/github-handles.json\"\n\n# Timeout for gh CLI commands (seconds)\nGH_TIMEOUT=10\n\n# Associative array for email -> github handle mapping (runtime cache)\ndeclare -A EMAIL_TO_GITHUB\n\n# Check if gh CLI is available and authenticated\ngh_available() {\n command -v gh >/dev/null 2>&1 && gh auth status >/dev/null 2>&1\n}\n\n# Load cached email -> github handle mappings from disk\nload_github_cache() {\n EMAIL_TO_GITHUB=()\n\n if [[ ! -f \"$GITHUB_CACHE_FILE\" ]]; then\n return 0\n fi\n\n # Validate JSON before loading\n if ! jq -e '.' \"$GITHUB_CACHE_FILE\" >/dev/null 2>&1; then\n msg \"Warning: corrupted cache file, ignoring\"\n return 0\n fi\n\n local email handle\n while IFS=$'\\t' read -r email handle; do\n # Validate handle format (alphanumeric, hyphens, max 39 chars)\n if [[ -n \"$email\" && -n \"$handle\" && \"$handle\" =~ ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$ ]]; then\n EMAIL_TO_GITHUB[\"$email\"]=\"$handle\"\n fi\n done < <(jq -r 'to_entries[] | \"\\(.key)\\t\\(.value)\"' \"$GITHUB_CACHE_FILE\" 2>/dev/null)\n\n msg \"Loaded ${#EMAIL_TO_GITHUB[@]} cached GitHub handle mappings\"\n}\n\n# Save email -> github handle mappings to disk (atomic write)\nsave_github_cache() {\n if [[ ${#EMAIL_TO_GITHUB[@]} -eq 0 ]]; then\n return 0\n fi\n\n mkdir -p \"$GITHUB_CACHE_DIR\"\n\n local tmp_file\n tmp_file=\"$(mktemp \"$GITHUB_CACHE_DIR/cache.XXXXXX\")\" || return 1\n\n # Build JSON from associative array\n {\n echo \"{\"\n local first=true\n local key\n for key in \"${!EMAIL_TO_GITHUB[@]}\"; do\n if [[ \"$first\" == \"true\" ]]; then\n first=false\n else\n echo \",\"\n fi\n # Escape special characters in email for JSON\n printf ' %s: %s' \"$(jq -n --arg e \"$key\" '$e')\" \"$(jq -n --arg h \"${EMAIL_TO_GITHUB[$key]}\" '$h')\"\n done\n echo\n echo \"}\"\n } > \"$tmp_file\"\n\n # Validate before replacing\n if jq -e '.' \"$tmp_file\" >/dev/null 2>&1; then\n mv \"$tmp_file\" \"$GITHUB_CACHE_FILE\"\n msg \"Saved ${#EMAIL_TO_GITHUB[@]} GitHub handle mappings to cache\"\n else\n rm -f \"$tmp_file\"\n msg \"Warning: failed to save cache (invalid JSON)\"\n fi\n}\n\n# Extract GitHub handle from email if it's a GitHub noreply address\n# Handles: user@users.noreply.github.com and 12345678+user@users.noreply.github.com\nextract_handle_from_noreply() {\n local email=\"$1\"\n\n if [[ \"$email\" =~ ^([0-9]+\\+)?([a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?)@users\\.noreply\\.github\\.com$ ]]; then\n echo \"${BASH_REMATCH[2]}\"\n return 0\n fi\n return 1\n}\n\n# Extract GitHub handle from merge commit subject\n# Handles: \"Merge pull request #123 from username/branch\"\nextract_handle_from_merge_commit() {\n local subject=\"$1\"\n\n if [[ \"$subject\" =~ ^Merge\\ pull\\ request\\ \\#[0-9]+\\ from\\ ([a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?)/.*$ ]]; then\n echo \"${BASH_REMATCH[1]}\"\n return 0\n fi\n return 1\n}\n\n# Extract PR number from commit subject\n# Handles: \"Subject (#123)\" and \"Merge pull request #123 from\"\nextract_pr_number() {\n local subject=\"$1\"\n\n if [[ \"$subject\" =~ \\(#([0-9]+)\\)$ ]]; then\n echo \"${BASH_REMATCH[1]}\"\n return 0\n elif [[ \"$subject\" =~ ^Merge\\ pull\\ request\\ \\#([0-9]+)\\ from ]]; then\n echo \"${BASH_REMATCH[1]}\"\n return 0\n fi\n return 1\n}\n\n# Query GitHub API for PR author (with timeout and error handling)\nquery_pr_author() {\n local gh_repo=\"$1\" # e.g., \"ipfs/kubo\"\n local pr_num=\"$2\"\n\n if ! gh_available; then\n return 1\n fi\n\n local handle\n handle=\"$(timeout \"$GH_TIMEOUT\" gh pr view \"$pr_num\" --repo \"$gh_repo\" --json author -q '.author.login' 2>/dev/null)\" || return 1\n\n # Validate handle format\n if [[ -n \"$handle\" && \"$handle\" =~ ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$ ]]; then\n echo \"$handle\"\n return 0\n fi\n return 1\n}\n\n# Query GitHub API for commit author (fallback when no PR available)\nquery_commit_author() {\n local gh_repo=\"$1\" # e.g., \"ipfs/kubo\"\n local commit_sha=\"$2\"\n\n if ! gh_available; then\n return 1\n fi\n\n local handle\n handle=\"$(timeout \"$GH_TIMEOUT\" gh api \"/repos/$gh_repo/commits/$commit_sha\" --jq '.author.login // empty' 2>/dev/null)\" || return 1\n\n # Validate handle format\n if [[ -n \"$handle\" && \"$handle\" =~ ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$ ]]; then\n echo \"$handle\"\n return 0\n fi\n return 1\n}\n\n# Resolve email to GitHub handle using all available methods\n# Args: email, commit_hash (optional), repo_dir (optional), gh_repo (optional)\nresolve_github_handle() {\n local email=\"$1\"\n local commit=\"${2:-}\"\n local repo_dir=\"${3:-}\"\n local gh_repo=\"${4:-}\"\n\n # Skip empty emails\n [[ -z \"$email\" ]] && return 1\n\n # Check runtime cache first\n if [[ -n \"${EMAIL_TO_GITHUB[$email]:-}\" ]]; then\n echo \"${EMAIL_TO_GITHUB[$email]}\"\n return 0\n fi\n\n local handle=\"\"\n\n # Method 1: Extract from noreply email\n if handle=\"$(extract_handle_from_noreply \"$email\")\"; then\n EMAIL_TO_GITHUB[\"$email\"]=\"$handle\"\n echo \"$handle\"\n return 0\n fi\n\n # Method 2: Look at commit message for merge commit pattern\n if [[ -n \"$commit\" && -n \"$repo_dir\" ]]; then\n local subject\n subject=\"$(git -C \"$repo_dir\" log -1 --format='%s' \"$commit\" 2>/dev/null)\" || true\n\n if [[ -n \"$subject\" ]]; then\n if handle=\"$(extract_handle_from_merge_commit \"$subject\")\"; then\n EMAIL_TO_GITHUB[\"$email\"]=\"$handle\"\n echo \"$handle\"\n return 0\n fi\n\n # Method 3: Query GitHub API for PR author\n if [[ -n \"$gh_repo\" ]]; then\n local pr_num\n if pr_num=\"$(extract_pr_number \"$subject\")\"; then\n if handle=\"$(query_pr_author \"$gh_repo\" \"$pr_num\")\"; then\n EMAIL_TO_GITHUB[\"$email\"]=\"$handle\"\n echo \"$handle\"\n return 0\n fi\n fi\n fi\n fi\n fi\n\n return 1\n}\n\n# Build GitHub handle mappings for all commits in a range\n# This does a single pass to collect PR numbers, then batch queries them\nbuild_github_mappings() {\n local module=\"$1\"\n local start=\"$2\"\n local end=\"${3:-HEAD}\"\n local repo\n repo=\"$(strip_version \"$module\")\"\n local dir\n local gh_repo=\"\"\n\n if [[ \"$module\" == \"github.com/ipfs/kubo\" ]]; then\n dir=\"$ROOT_DIR\"\n else\n dir=\"$GOPATH/src/$repo\"\n fi\n\n # Extract gh_repo for API calls (e.g., \"ipfs/kubo\" from \"github.com/ipfs/kubo\")\n if [[ \"$repo\" =~ ^github\\.com/(.+)$ ]]; then\n gh_repo=\"${BASH_REMATCH[1]}\"\n fi\n\n msg \"Building GitHub handle mappings for $module...\"\n\n # Collect all unique emails and their commit context\n declare -A email_commits=()\n local hash email subject\n\n while IFS=$'\\t' read -r hash email subject; do\n [[ -z \"$email\" ]] && continue\n\n # Skip if already resolved\n [[ -n \"${EMAIL_TO_GITHUB[$email]:-}\" ]] && continue\n\n # Try to resolve without API first\n local handle=\"\"\n\n # Method 1: noreply email\n if handle=\"$(extract_handle_from_noreply \"$email\")\"; then\n EMAIL_TO_GITHUB[\"$email\"]=\"$handle\"\n continue\n fi\n\n # Method 2: merge commit message\n if handle=\"$(extract_handle_from_merge_commit \"$subject\")\"; then\n EMAIL_TO_GITHUB[\"$email\"]=\"$handle\"\n continue\n fi\n\n # Store for potential API lookup\n if [[ -z \"${email_commits[$email]:-}\" ]]; then\n email_commits[\"$email\"]=\"$hash\"\n fi\n done < <(git -C \"$dir\" log --format='tformat:%H%x09%aE%x09%s' --no-merges \"$start..$end\" 2>/dev/null)\n\n # API batch lookup for remaining emails (if gh is available)\n if gh_available && [[ -n \"$gh_repo\" && ${#email_commits[@]} -gt 0 ]]; then\n msg \"Querying GitHub API for ${#email_commits[@]} unknown contributors...\"\n local key\n for key in \"${!email_commits[@]}\"; do\n # Skip if already resolved\n [[ -n \"${EMAIL_TO_GITHUB[$key]:-}\" ]] && continue\n\n local commit_hash=\"${email_commits[$key]}\"\n local subj handle\n subj=\"$(git -C \"$dir\" log -1 --format='%s' \"$commit_hash\" 2>/dev/null)\" || true\n\n # Try PR author lookup first (cheaper API call)\n local pr_num\n if pr_num=\"$(extract_pr_number \"$subj\")\"; then\n if handle=\"$(query_pr_author \"$gh_repo\" \"$pr_num\")\"; then\n EMAIL_TO_GITHUB[\"$key\"]=\"$handle\"\n continue\n fi\n fi\n\n # Fallback: commit author API (works for any commit)\n if handle=\"$(query_commit_author \"$gh_repo\" \"$commit_hash\")\"; then\n EMAIL_TO_GITHUB[\"$key\"]=\"$handle\"\n fi\n done\n fi\n}\n\n##########################################################################################\n# Original infrastructure with modifications\n##########################################################################################\n\nbuild_include_regex() {\n local result=\"\"\n local mod\n for mod in \"${INCLUDE_MODULES[@]}\"; do\n if [[ -n \"$result\" ]]; then\n result=\"$result|$mod\"\n else\n result=\"$mod\"\n fi\n done\n echo \"($result)\"\n}\n\nbuild_exclude_regex() {\n local result=\"\"\n local mod\n for mod in \"${EXCLUDE_MODULES[@]}\"; do\n if [[ -n \"$result\" ]]; then\n result=\"$result|$mod\"\n else\n result=\"$mod\"\n fi\n done\n if [[ -n \"$result\" ]]; then\n echo \"($result)\"\n else\n echo '$^' # match nothing\n fi\n}\n\nif [[ ${#INCLUDE_MODULES[@]} -gt 0 ]]; then\n INCLUDE_REGEX=\"$(build_include_regex)\"\nelse\n INCLUDE_REGEX=\"\" # \"match anything\"\nfi\n\nif [[ ${#EXCLUDE_MODULES[@]} -gt 0 ]]; then\n EXCLUDE_REGEX=\"$(build_exclude_regex)\"\nelse\n EXCLUDE_REGEX='$^' # \"match nothing\"\nfi\n\nIGNORE_FILES_PATHSPEC=()\nfor f in \"${IGNORE_FILES[@]}\"; do\n IGNORE_FILES_PATHSPEC+=(\":^:**/$f\" \":^:$f\") # Prepend the magic \"ignore this\" sequence.\ndone\n\n\nNL=$'\\n'\n\nROOT_DIR=\"$(git rev-parse --show-toplevel)\"\n\nmsg() {\n echo \"$*\" >&2\n}\n\nstatlog() {\n local module=\"$1\"\n local rpath\n local gh_repo=\"\"\n\n if [[ \"$module\" == \"github.com/ipfs/kubo\" ]]; then\n rpath=\"$ROOT_DIR\"\n else\n rpath=\"$GOPATH/src/$(strip_version \"$module\")\"\n fi\n\n # Extract gh_repo for API calls\n local repo\n repo=\"$(strip_version \"$module\")\"\n if [[ \"$repo\" =~ ^github\\.com/(.+)$ ]]; then\n gh_repo=\"${BASH_REMATCH[1]}\"\n fi\n\n local start=\"${2:-}\"\n local end=\"${3:-HEAD}\"\n local mailmap_file=\"$rpath/.mailmap\"\n if ! [[ -e \"$mailmap_file\" ]]; then\n mailmap_file=\"$ROOT_DIR/.mailmap\"\n fi\n\n local stack=()\n local line\n while read -r line; do\n if [[ -n \"$line\" ]]; then\n stack+=(\"$line\")\n continue\n fi\n\n local changes\n read -r changes\n\n local changed=0\n local insertions=0\n local deletions=0\n local count event\n while read -r count event; do\n if [[ \"$event\" =~ ^file ]]; then\n changed=$count\n elif [[ \"$event\" =~ ^insertion ]]; then\n insertions=$count\n elif [[ \"$event\" =~ ^deletion ]]; then\n deletions=$count\n else\n echo \"unknown event $event\" >&2\n exit 1\n fi\n done<<<\"${changes//,/$NL}\"\n\n local author\n for author in \"${stack[@]}\"; do\n local hash name email\n IFS=$'\\t' read -r hash name email <<<\"$author\"\n\n # Resolve GitHub handle\n local github_handle=\"\"\n github_handle=\"$(resolve_github_handle \"$email\" \"$hash\" \"$rpath\" \"$gh_repo\")\" || true\n\n jq -n \\\n --arg \"hash\" \"$hash\" \\\n --arg \"name\" \"$name\" \\\n --arg \"email\" \"$email\" \\\n --arg \"github\" \"$github_handle\" \\\n --argjson \"changed\" \"$changed\" \\\n --argjson \"insertions\" \"$insertions\" \\\n --argjson \"deletions\" \"$deletions\" \\\n '{Commit: $hash, Author: $name, Email: $email, GitHub: $github, Files: $changed, Insertions: $insertions, Deletions: $deletions}'\n done\n stack=()\n done < <(git -C \"$rpath\" -c mailmap.file=\"$mailmap_file\" log --use-mailmap --shortstat --no-merges --pretty=\"tformat:%H%x09%aN%x09%aE\" \"$start..$end\" -- . \"${IGNORE_FILES_PATHSPEC[@]}\")\n}\n\n# Returns a stream of deps changed between $1 and $2.\ndep_changes() {\n cat \"$1\" \"$2\" | jq -s 'JOIN(INDEX(.[0][]; .Path); .[1][]; .Path; {Path: .[0].Path, Old: (.[1] | del(.Path)), New: (.[0] | del(.Path))}) | select(.New.Version != .Old.Version)'\n}\n\n# resolve_commits resolves a git ref for each version.\nresolve_commits() {\n jq '. + {Ref: (.Version|capture(\"^((?.*)\\\\+incompatible|v.*-(0\\\\.)?[0-9]{14}-(?[a-f0-9]{12})|(?v.*))$\") | .ref1 // .ref2 // .ref3)}'\n}\n\npr_link() {\n local repo=\"$1\"\n local prnum=\"$2\"\n local ghname=\"${repo##github.com/}\"\n printf -- \"[%s#%s](https://%s/pull/%s)\" \"$ghname\" \"$prnum\" \"$repo\" \"$prnum\"\n}\n\nignored_commit() {\n local repo=\"$1\"\n local commit=\"$2\"\n local matches\n\n # Check to see if this commit includes any non-ignored files.\n matches=$(git -C \"$repo\" diff-tree --no-commit-id --name-only -r \"$commit^\" \"$commit\" \\\n -- \"${IGNORE_FILES_PATHSPEC[@]}\" | wc -l)\n [[ \"$matches\" -eq 0 ]]\n}\n\n# Generate a release log for a range of commits in a single repo.\nrelease_log() {\n local module=\"$1\"\n local start=\"$2\"\n local end=\"${3:-HEAD}\"\n local repo\n repo=\"$(strip_version \"$1\")\"\n local dir\n if [[ \"$module\" == \"github.com/ipfs/kubo\" ]]; then\n dir=\"$ROOT_DIR\"\n else\n dir=\"$GOPATH/src/$repo\"\n fi\n\n local commit subject\n while read -r commit subject; do\n # Skip commits that only touch ignored files.\n if ignored_commit \"$dir\" \"$commit\"; then\n continue\n fi\n\n if [[ \"$subject\" =~ ^Merge\\ pull\\ request\\ \\#([0-9]+)\\ from ]]; then\n local prnum=\"${BASH_REMATCH[1]}\"\n local desc\n desc=\"$(git -C \"$dir\" show --summary --format='tformat:%b' \"$commit\" | head -1)\"\n printf -- \"- %s (%s)\\n\" \"$desc\" \"$(pr_link \"$repo\" \"$prnum\")\"\n elif [[ \"$subject\" =~ \\(#([0-9]+)\\)$ ]]; then\n local prnum=\"${BASH_REMATCH[1]}\"\n printf -- \"- %s (%s)\\n\" \"$subject\" \"$(pr_link \"$repo\" \"$prnum\")\"\n else\n printf -- \"- %s\\n\" \"$subject\"\n fi\n done < <(git -C \"$dir\" log --format='tformat:%H %s' --first-parent \"$start..$end\")\n}\n\nindent() {\n sed -e 's/^/ /'\n}\n\nmod_deps() {\n go list -mod=mod -json -m all | jq 'select(.Version != null)'\n}\n\nensure() {\n local repo\n repo=\"$(strip_version \"$1\")\"\n local commit=\"$2\"\n local rpath\n if [[ \"$1\" == \"github.com/ipfs/kubo\" ]]; then\n rpath=\"$ROOT_DIR\"\n else\n rpath=\"$GOPATH/src/$repo\"\n fi\n if [[ \"$1\" != \"github.com/ipfs/kubo\" ]] && [[ ! -d \"$rpath\" ]]; then\n msg \"Cloning $repo...\"\n git clone \"http://$repo\" \"$rpath\" >&2\n fi\n\n if ! git -C \"$rpath\" rev-parse --verify \"$commit\" >/dev/null; then\n msg \"Fetching $repo...\"\n git -C \"$rpath\" fetch --all >&2\n fi\n\n git -C \"$rpath\" rev-parse --verify \"$commit\" >/dev/null || return 1\n}\n\n# Summarize stats, grouping by GitHub handle (with fallback to email for dedup)\nstatsummary() {\n jq -s '\n # Group by GitHub handle if available, otherwise by email\n group_by(if .GitHub != \"\" then .GitHub else .Email end)[] |\n {\n # Use first non-empty GitHub handle, or fall back to Author name\n Author: .[0].Author,\n GitHub: (map(select(.GitHub != \"\")) | .[0].GitHub // \"\"),\n Email: .[0].Email,\n Commits: (. | length),\n Insertions: (map(.Insertions) | add),\n Deletions: (map(.Deletions) | add),\n Files: (map(.Files) | add)\n }\n ' | jq '. + {Lines: (.Deletions + .Insertions)}'\n}\n\nstrip_version() {\n local repo=\"$1\"\n if [[ \"$repo\" =~ .*/v[0-9]+$ ]]; then\n repo=\"$(dirname \"$repo\")\"\n fi\n echo \"$repo\"\n}\n\nrecursive_release_log() {\n local start=\"${1:-$(git tag -l | sort -V | grep -v -- '-rc' | grep 'v'| tail -n1)}\"\n local end=\"${2:-$(git rev-parse HEAD)}\"\n local repo_root\n repo_root=\"$(git rev-parse --show-toplevel)\"\n local module\n module=\"$(go list -m)\"\n local dir\n dir=\"$(go list -m -f '{{.Dir}}')\"\n\n # Load cached GitHub handle mappings\n load_github_cache\n\n # Kubo can be run from any directory, dependencies still use GOPATH\n\n (\n local result=0\n local workspace\n workspace=\"$(mktemp -d)\"\n # shellcheck disable=SC2064\n trap \"rm -rf '$workspace'\" INT TERM EXIT\n cd \"$workspace\"\n\n echo \"Computing old deps...\" >&2\n git -C \"$repo_root\" show \"$start:go.mod\" >go.mod\n mod_deps | resolve_commits | jq -s > old_deps.json\n\n echo \"Computing new deps...\" >&2\n git -C \"$repo_root\" show \"$end:go.mod\" >go.mod\n mod_deps | resolve_commits | jq -s > new_deps.json\n\n rm -f go.mod go.sum\n\n printf -- \"Generating Changelog for %s %s..%s\\n\" \"$module\" \"$start\" \"$end\" >&2\n\n # Pre-build GitHub mappings for main module\n build_github_mappings \"$module\" \"$start\" \"$end\"\n\n echo \"### 📝 Changelog\"\n echo\n echo \"
Full Changelog\"\n echo\n\n printf -- \"- %s:\\n\" \"$module\"\n release_log \"$module\" \"$start\" \"$end\" | indent\n\n statlog \"$module\" \"$start\" \"$end\" > statlog.json\n\n local dep_module new new_ref old old_ref\n while read -r dep_module new new_ref old old_ref; do\n if ! ensure \"$dep_module\" \"$new_ref\"; then\n result=1\n local changelog=\"failed to fetch repo\"\n else\n # Pre-build GitHub mappings for dependency\n build_github_mappings \"$dep_module\" \"$old_ref\" \"$new_ref\"\n statlog \"$dep_module\" \"$old_ref\" \"$new_ref\" >> statlog.json\n local changelog\n changelog=\"$(release_log \"$dep_module\" \"$old_ref\" \"$new_ref\")\"\n fi\n if [[ -n \"$changelog\" ]]; then\n printf -- \"- %s (%s -> %s):\\n\" \"$dep_module\" \"$old\" \"$new\"\n echo \"$changelog\" | indent\n fi\n done < <(dep_changes old_deps.json new_deps.json |\n jq --arg inc \"$INCLUDE_REGEX\" --arg exc \"$EXCLUDE_REGEX\" \\\n 'select(.Path | test($inc)) | select(.Path | test($exc) | not)' |\n jq -r '\"\\(.Path) \\(.New.Version) \\(.New.Ref) \\(.Old.Version) \\(.Old.Ref // \"\")\"')\n\n echo\n echo \"
\"\n echo\n echo \"### 👨‍👩‍👧‍👦 Contributors\"\n echo\n\n echo \"| Contributor | Commits | Lines ± | Files Changed |\"\n echo \"|-------------|---------|---------|---------------|\"\n statsummary [git tag name] [dry run]\n#\n# Example:\n# # dry run. pass a 5th arg to have it print what it would do rather than do it.\n# ./push-docker-tags.sh $(date -u +%F) testingsha master \"\" dryrun\n#\n# # push tag for the master branch\n# ./push-docker-tags.sh $(date -u +%F) testingsha master\n#\n# # push tag for a release tag\n# ./push-docker-tags.sh $(date -u +%F) testingsha release v0.5.0\n#\nset -euo pipefail\n\nif [[ $# -lt 1 ]] ; then\n echo 'At least 1 arg required. Pass 5 args for a dry run.'\n echo 'Usage:'\n echo './push-docker-tags.sh [git commit sha1] [git branch name] [git tag name] [dry run]'\n exit 1\nfi\n\nBUILD_NUM=$1\nGIT_SHA1=${2:-$(git rev-parse HEAD)}\nGIT_SHA1_SHORT=$(echo \"$GIT_SHA1\" | cut -c 1-7)\nGIT_BRANCH=${3:-$(git symbolic-ref -q --short HEAD || echo \"unknown\")}\nGIT_TAG=${4:-$(git describe --tags --exact-match || echo \"\")}\nDRY_RUN=${5:-false}\n\nWIP_IMAGE_TAG=${WIP_IMAGE_TAG:-wip}\nIMAGE_NAME=${IMAGE_NAME:-ipfs/kubo}\n\npushTag () {\n local IMAGE_TAG=$1\n if [ \"$DRY_RUN\" != false ]; then\n echo \"DRY RUN! I would have tagged and pushed the following...\"\n echo docker tag \"$IMAGE_NAME:$WIP_IMAGE_TAG\" \"$IMAGE_NAME:$IMAGE_TAG\"\n echo docker push \"$IMAGE_NAME:$IMAGE_TAG\"\n else\n echo \"Tagging $IMAGE_NAME:$IMAGE_TAG and pushing to dockerhub\"\n docker tag \"$IMAGE_NAME:$WIP_IMAGE_TAG\" \"$IMAGE_NAME:$IMAGE_TAG\"\n docker push \"$IMAGE_NAME:$IMAGE_TAG\"\n fi\n}\n\nif [[ $GIT_TAG =~ ^v[0-9]+\\.[0-9]+\\.[0-9]+-rc ]]; then\n pushTag \"$GIT_TAG\"\n\nelif [[ $GIT_TAG =~ ^v[0-9]+\\.[0-9]+\\.[0-9]+$ ]]; then\n pushTag \"$GIT_TAG\"\n pushTag \"latest\"\n pushTag \"release\" # see: https://github.com/ipfs/kubo/issues/3999#issuecomment-742228981\n\nelif [[ $GIT_BRANCH =~ ^bifrost-.* ]]; then\n # sanitize the branch name since docker tags have stricter char limits than git branch names\n branch=$(echo \"$GIT_BRANCH\" | tr '/' '-' | tr --delete --complement '[:alnum:]-')\n pushTag \"${branch}-${BUILD_NUM}-${GIT_SHA1_SHORT}\"\n\nelif [ \"$GIT_BRANCH\" = \"master\" ] || [ \"$GIT_BRANCH\" = \"staging\" ]; then\n pushTag \"${GIT_BRANCH}-${BUILD_NUM}-${GIT_SHA1_SHORT}\"\n pushTag \"${GIT_BRANCH}-latest\"\n\nelse\n echo \"Nothing to do. No docker tag defined for branch: $GIT_BRANCH, tag: $GIT_TAG\"\n\nfi\n" }, { "path": "bin/test-go-build-platforms", "content": "#!/bin/bash\nset -e\n\necho \"Building kubo for all platforms in .github/build-platforms.yml...\"\n\nif [ ! -f .github/build-platforms.yml ]; then\n echo \"Error: .github/build-platforms.yml not found\"\n exit 1\nfi\n\ngrep '^ - ' .github/build-platforms.yml | sed 's/^ - //' | while read -r platform; do\n if [ -z \"$platform\" ]; then\n continue\n fi\n\n GOOS=$(echo \"$platform\" | cut -d- -f1)\n GOARCH=$(echo \"$platform\" | cut -d- -f2)\n\n echo \"Building $platform...\"\n echo \" GOOS=$GOOS GOARCH=$GOARCH go build -o /dev/null ./cmd/ipfs\"\n GOOS=$GOOS GOARCH=$GOARCH go build -o /dev/null ./cmd/ipfs\ndone\n\necho \"All platforms built successfully\"" }, { "path": "bin/test-go-fmt", "content": "#!/usr/bin/env bash\nset -euo pipefail\nT=\"$(mktemp)\"\nfind . \\\n -path ./test/sharness -prune \\\n -o -path ./plugin/loader/preload.go -prune \\\n -o -name '*.go' -print0 | xargs -0 gofmt -s -l > \"$T\"\n\nif [ -n \"$(cat $T)\" ]; then\n\techo \"Following Go code is not formatted.\"\n\techo \"-----------------------------------\"\n\tcat \"$T\"\n\techo \"-----------------------------------\"\n\techo \"Run 'go fmt ./...' in your source directory\"\n\trm -f \"$T\"\n\texit 1\nfi\nrm -f \"$T\"\n" }, { "path": "blocks/blockstoreutil/remove.go", "content": "// Package blockstoreutil provides utility functions for Blockstores.\npackage blockstoreutil\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\n\tbs \"github.com/ipfs/boxo/blockstore\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\tcid \"github.com/ipfs/go-cid\"\n\tformat \"github.com/ipfs/go-ipld-format\"\n)\n\n// RemovedBlock is used to represent the result of removing a block.\n// If a block was removed successfully, then the Error will be empty.\n// If a block could not be removed, then Error will contain the\n// reason the block could not be removed. If the removal was aborted\n// due to a fatal error, Hash will be empty, Error will contain the\n// reason, and no more results will be sent.\ntype RemovedBlock struct {\n\tHash string\n\tError error\n}\n\n// RmBlocksOpts is used to wrap options for RmBlocks().\ntype RmBlocksOpts struct {\n\tPrefix string\n\tQuiet bool\n\tForce bool\n}\n\n// RmBlocks removes the blocks provided in the cids slice.\n// It returns a channel where objects of type RemovedBlock are placed, when\n// not using the Quiet option. Block removal is asynchronous and will\n// skip any pinned blocks.\nfunc RmBlocks(ctx context.Context, blocks bs.GCBlockstore, pins pin.Pinner, cids []cid.Cid, opts RmBlocksOpts) (<-chan any, error) {\n\t// make the channel large enough to hold any result to avoid\n\t// blocking while holding the GCLock\n\tout := make(chan any, len(cids))\n\tgo func() {\n\t\tdefer close(out)\n\n\t\tunlocker := blocks.GCLock(ctx)\n\t\tdefer unlocker.Unlock(ctx)\n\n\t\tstillOkay := FilterPinned(ctx, pins, out, cids)\n\n\t\tfor _, c := range stillOkay {\n\t\t\t// Kept for backwards compatibility. We may want to\n\t\t\t// remove this sometime in the future.\n\t\t\thas, err := blocks.Has(ctx, c)\n\t\t\tif err != nil {\n\t\t\t\tout <- &RemovedBlock{Hash: c.String(), Error: err}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif !has && !opts.Force {\n\t\t\t\tout <- &RemovedBlock{Hash: c.String(), Error: format.ErrNotFound{Cid: c}}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\terr = blocks.DeleteBlock(ctx, c)\n\t\t\tif err != nil {\n\t\t\t\tout <- &RemovedBlock{Hash: c.String(), Error: err}\n\t\t\t} else if !opts.Quiet {\n\t\t\t\tout <- &RemovedBlock{Hash: c.String()}\n\t\t\t}\n\t\t}\n\t}()\n\treturn out, nil\n}\n\n// FilterPinned takes a slice of Cids and returns it with the pinned Cids\n// removed. If a Cid is pinned, it will place RemovedBlock objects in the given\n// out channel, with an error which indicates that the Cid is pinned.\n// This function is used in RmBlocks to filter out any blocks which are not\n// to be removed (because they are pinned).\nfunc FilterPinned(ctx context.Context, pins pin.Pinner, out chan<- any, cids []cid.Cid) []cid.Cid {\n\tstillOkay := make([]cid.Cid, 0, len(cids))\n\tres, err := pins.CheckIfPinned(ctx, cids...)\n\tif err != nil {\n\t\tout <- &RemovedBlock{Error: fmt.Errorf(\"pin check failed: %w\", err)}\n\t\treturn nil\n\t}\n\tfor _, r := range res {\n\t\tif !r.Pinned() {\n\t\t\tstillOkay = append(stillOkay, r.Key)\n\t\t} else {\n\t\t\tout <- &RemovedBlock{\n\t\t\t\tHash: r.Key.String(),\n\t\t\t\tError: errors.New(r.String()),\n\t\t\t}\n\t\t}\n\t}\n\treturn stillOkay\n}\n" }, { "path": "client/rpc/README.md", "content": "# `coreiface.CoreAPI` over http `rpc`\n\n> IPFS CoreAPI implementation using HTTP API\n\nThis package implements [`coreiface.CoreAPI`](https://pkg.go.dev/github.com/ipfs/kubo/core/coreiface#CoreAPI) over the HTTP API.\n\n## Documentation\n\nhttps://pkg.go.dev/github.com/ipfs/kubo/client/rpc\n\n### Example\n\nPin file on your local IPFS node based on its CID:\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/client/rpc\"\n)\n\nfunc main() {\n\t// \"Connect\" to local node\n\tnode, err := rpc.NewLocalApi()\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\t// Pin a given file by its CID\n\tctx := context.Background()\n\tc, err := cid.Decode(\"bafkreidtuosuw37f5xmn65b3ksdiikajy7pwjjslzj2lxxz2vc4wdy3zku\")\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\tp := path.FromCid(c)\n\terr = node.Pin().Add(ctx, p)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n}\n```\n" }, { "path": "client/rpc/api.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/blang/semver/v4\"\n\t\"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/go-cid\"\n\tlegacy \"github.com/ipfs/go-ipld-legacy\"\n\tipfs \"github.com/ipfs/kubo\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/misc/fsutil\"\n\tdagpb \"github.com/ipld/go-codec-dagpb\"\n\t_ \"github.com/ipld/go-ipld-prime/codec/dagcbor\"\n\t\"github.com/ipld/go-ipld-prime/node/basicnode\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nconst (\n\tDefaultPathName = \".ipfs\"\n\tDefaultPathRoot = \"~/\" + DefaultPathName\n\tDefaultApiFile = \"api\"\n\tEnvDir = \"IPFS_PATH\"\n)\n\n// ErrApiNotFound if we fail to find a running daemon.\nvar ErrApiNotFound = errors.New(\"ipfs api address could not be found\")\n\n// HttpApi implements github.com/ipfs/interface-go-ipfs-core/CoreAPI using\n// IPFS HTTP API.\n//\n// For interface docs see\n// https://godoc.org/github.com/ipfs/interface-go-ipfs-core#CoreAPI\ntype HttpApi struct {\n\turl string\n\thttpcli http.Client\n\tHeaders http.Header\n\tapplyGlobal func(*requestBuilder)\n\tipldDecoder *legacy.Decoder\n\tversionMu sync.Mutex\n\tversion *semver.Version\n}\n\n// NewLocalApi tries to construct new HttpApi instance communicating with local\n// IPFS daemon\n//\n// Daemon api address is pulled from the $IPFS_PATH/api file.\n// If $IPFS_PATH env var is not present, it defaults to ~/.ipfs.\nfunc NewLocalApi() (*HttpApi, error) {\n\tbaseDir := os.Getenv(EnvDir)\n\tif baseDir == \"\" {\n\t\tbaseDir = DefaultPathRoot\n\t}\n\n\treturn NewPathApi(baseDir)\n}\n\n// NewPathApi constructs new HttpApi by pulling api address from specified\n// ipfspath. Api file should be located at $ipfspath/api.\nfunc NewPathApi(ipfspath string) (*HttpApi, error) {\n\ta, err := ApiAddr(ipfspath)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\terr = ErrApiNotFound\n\t\t}\n\t\treturn nil, err\n\t}\n\treturn NewApi(a)\n}\n\n// ApiAddr reads api file in specified ipfs path.\nfunc ApiAddr(ipfspath string) (ma.Multiaddr, error) {\n\tbaseDir, err := fsutil.ExpandHome(ipfspath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tapiFile := filepath.Join(baseDir, DefaultApiFile)\n\n\tapi, err := os.ReadFile(apiFile)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ma.NewMultiaddr(strings.TrimSpace(string(api)))\n}\n\n// NewApi constructs HttpApi with specified endpoint.\nfunc NewApi(a ma.Multiaddr) (*HttpApi, error) {\n\ttransport := &http.Transport{\n\t\tProxy: http.ProxyFromEnvironment,\n\t\tDisableKeepAlives: true,\n\t}\n\n\tnetwork, address, err := manet.DialArgs(a)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif network == \"unix\" {\n\t\ttransport.DialContext = func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\treturn net.Dial(\"unix\", address)\n\t\t}\n\t\tc := &http.Client{\n\t\t\tTransport: transport,\n\t\t}\n\t\t// This will create an API client which\n\t\t// makes requests to `http://unix`.\n\t\treturn NewURLApiWithClient(network, c)\n\t}\n\n\tc := &http.Client{\n\t\tTransport: transport,\n\t}\n\n\treturn NewApiWithClient(a, c)\n}\n\n// NewApiWithClient constructs HttpApi with specified endpoint and custom http client.\nfunc NewApiWithClient(a ma.Multiaddr, c *http.Client) (*HttpApi, error) {\n\t_, url, err := manet.DialArgs(a)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif a, err := ma.NewMultiaddr(url); err == nil {\n\t\t_, host, err := manet.DialArgs(a)\n\t\tif err == nil {\n\t\t\turl = host\n\t\t}\n\t}\n\n\tproto := \"http://\"\n\n\t// By default, DialArgs is going to provide details suitable for connecting\n\t// a socket to, but not really suitable for making an informed choice of http\n\t// protocol. For multiaddresses specifying tls and/or https we want to make\n\t// a https request instead of a http request.\n\tprotocols := a.Protocols()\n\tfor _, p := range protocols {\n\t\tif p.Code == ma.P_HTTPS || p.Code == ma.P_TLS {\n\t\t\tproto = \"https://\"\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn NewURLApiWithClient(proto+url, c)\n}\n\nfunc NewURLApiWithClient(url string, c *http.Client) (*HttpApi, error) {\n\tdecoder := legacy.NewDecoder()\n\t// Add support for these codecs to match what is done in the merkledag library\n\t// Note: to match prior behavior the go-ipld-prime CBOR decoder is manually included\n\t// TODO: allow the codec registry used to be configured by the caller not through a global variable\n\tdecoder.RegisterCodec(cid.DagProtobuf, dagpb.Type.PBNode, merkledag.ProtoNodeConverter)\n\tdecoder.RegisterCodec(cid.Raw, basicnode.Prototype.Bytes, merkledag.RawNodeConverter)\n\n\tapi := &HttpApi{\n\t\turl: url,\n\t\thttpcli: *c,\n\t\tHeaders: make(map[string][]string),\n\t\tapplyGlobal: func(*requestBuilder) {},\n\t\tipldDecoder: decoder,\n\t}\n\n\t// We don't support redirects.\n\tapi.httpcli.CheckRedirect = func(_ *http.Request, _ []*http.Request) error {\n\t\treturn fmt.Errorf(\"unexpected redirect\")\n\t}\n\n\treturn api, nil\n}\n\nfunc (api *HttpApi) WithOptions(opts ...caopts.ApiOption) (iface.CoreAPI, error) {\n\toptions, err := caopts.ApiOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tsubApi := &HttpApi{\n\t\turl: api.url,\n\t\thttpcli: api.httpcli,\n\t\tHeaders: api.Headers,\n\t\tapplyGlobal: func(req *requestBuilder) {\n\t\t\tif options.Offline {\n\t\t\t\treq.Option(\"offline\", options.Offline)\n\t\t\t}\n\t\t},\n\t\tipldDecoder: api.ipldDecoder,\n\t}\n\n\treturn subApi, nil\n}\n\nfunc (api *HttpApi) Request(command string, args ...string) RequestBuilder {\n\theaders := make(map[string]string)\n\tif api.Headers != nil {\n\t\tfor k := range api.Headers {\n\t\t\theaders[k] = api.Headers.Get(k)\n\t\t}\n\t}\n\treturn &requestBuilder{\n\t\tcommand: command,\n\t\targs: args,\n\t\tshell: api,\n\t\theaders: headers,\n\t}\n}\n\nfunc (api *HttpApi) Unixfs() iface.UnixfsAPI {\n\treturn (*UnixfsAPI)(api)\n}\n\nfunc (api *HttpApi) Block() iface.BlockAPI {\n\treturn (*BlockAPI)(api)\n}\n\nfunc (api *HttpApi) Dag() iface.APIDagService {\n\treturn (*HttpDagServ)(api)\n}\n\nfunc (api *HttpApi) Name() iface.NameAPI {\n\treturn (*NameAPI)(api)\n}\n\nfunc (api *HttpApi) Key() iface.KeyAPI {\n\treturn (*KeyAPI)(api)\n}\n\nfunc (api *HttpApi) Pin() iface.PinAPI {\n\treturn (*PinAPI)(api)\n}\n\nfunc (api *HttpApi) Object() iface.ObjectAPI {\n\treturn (*ObjectAPI)(api)\n}\n\nfunc (api *HttpApi) Swarm() iface.SwarmAPI {\n\treturn (*SwarmAPI)(api)\n}\n\nfunc (api *HttpApi) PubSub() iface.PubSubAPI {\n\treturn (*PubsubAPI)(api)\n}\n\nfunc (api *HttpApi) Routing() iface.RoutingAPI {\n\treturn (*RoutingAPI)(api)\n}\n\nfunc (api *HttpApi) loadRemoteVersion() (*semver.Version, error) {\n\tapi.versionMu.Lock()\n\tdefer api.versionMu.Unlock()\n\n\tif api.version == nil {\n\t\tctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(time.Second*30))\n\t\tdefer cancel()\n\n\t\tresp, err := api.Request(\"version\").Send(ctx)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif resp.Error != nil {\n\t\t\treturn nil, resp.Error\n\t\t}\n\t\tdefer resp.Close()\n\t\tvar out ipfs.VersionInfo\n\t\tdec := json.NewDecoder(resp.Output)\n\t\tif err := dec.Decode(&out); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tremoteVersion, err := semver.New(out.Version)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tapi.version = remoteVersion\n\t}\n\n\treturn api.version, nil\n}\n" }, { "path": "client/rpc/api_test.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/config\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/tests\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\ntype NodeProvider struct{}\n\nfunc (np NodeProvider) MakeAPISwarm(t *testing.T, ctx context.Context, fullIdentity, online bool, n int) ([]iface.CoreAPI, error) {\n\th := harness.NewT(t)\n\n\tapis := make([]iface.CoreAPI, n)\n\tnodes := h.NewNodes(n)\n\n\tvar wg, zero sync.WaitGroup\n\tzeroNode := nodes[0]\n\twg.Add(len(apis))\n\tzero.Add(1)\n\n\tvar errs []error\n\tvar errsLk sync.Mutex\n\n\tfor i, n := range nodes {\n\t\tgo func(i int, n *harness.Node) {\n\t\t\tif err := func() error {\n\t\t\t\tdefer wg.Done()\n\t\t\t\tvar err error\n\n\t\t\t\tn.Init(\"--empty-repo\")\n\n\t\t\t\tc := n.ReadConfig()\n\t\t\t\tc.Experimental.FilestoreEnabled = true\n\t\t\t\t// only provide things we pin. Allows to test\n\t\t\t\t// provide operations.\n\t\t\t\tc.Provide.Strategy = config.NewOptionalString(\"roots\")\n\t\t\t\tn.WriteConfig(c)\n\t\t\t\tn.StartDaemon(\"--enable-pubsub-experiment\", \"--offline=\"+strconv.FormatBool(!online))\n\n\t\t\t\tif online {\n\t\t\t\t\tif i > 0 {\n\t\t\t\t\t\tzero.Wait()\n\t\t\t\t\t\tn.Connect(zeroNode)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tzero.Done()\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tapiMaddr, err := n.TryAPIAddr()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tapi, err := NewApi(apiMaddr)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tapis[i] = api\n\n\t\t\t\t// empty node is pinned even with --empty-repo, we don't want that\n\t\t\t\temptyNode, err := path.NewPath(\"/ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\")\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tif err := api.Pin().Rm(ctx, emptyNode); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}(); err != nil {\n\t\t\t\terrsLk.Lock()\n\t\t\t\terrs = append(errs, err)\n\t\t\t\terrsLk.Unlock()\n\t\t\t}\n\t\t}(i, n)\n\t}\n\n\twg.Wait()\n\n\treturn apis, errors.Join(errs...)\n}\n\nfunc TestHttpApi(t *testing.T) {\n\tt.Parallel()\n\n\ttests.TestApi(NodeProvider{})(t)\n}\n\nfunc Test_NewURLApiWithClient_With_Headers(t *testing.T) {\n\tt.Parallel()\n\n\tvar (\n\t\theaderToTest = \"Test-Header\"\n\t\texpectedHeaderValue = \"thisisaheadertest\"\n\t)\n\tts := httptest.NewServer(\n\t\thttp.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tval := r.Header.Get(headerToTest)\n\t\t\tif val != expectedHeaderValue {\n\t\t\t\tw.WriteHeader(400)\n\t\t\t\treturn\n\t\t\t}\n\t\t\thttp.ServeContent(w, r, \"\", time.Now(), strings.NewReader(\"test\"))\n\t\t}),\n\t)\n\tdefer ts.Close()\n\tapi, err := NewURLApiWithClient(ts.URL, &http.Client{\n\t\tTransport: &http.Transport{\n\t\t\tProxy: http.ProxyFromEnvironment,\n\t\t\tDisableKeepAlives: true,\n\t\t},\n\t})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tapi.Headers.Set(headerToTest, expectedHeaderValue)\n\tp, err := path.NewPath(\"/ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := api.Pin().Rm(context.Background(), p); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc Test_NewURLApiWithClient_HTTP_Variant(t *testing.T) {\n\tt.Parallel()\n\n\ttestcases := []struct {\n\t\taddress string\n\t\texpected string\n\t}{\n\t\t{address: \"/ip4/127.0.0.1/tcp/80\", expected: \"http://127.0.0.1:80\"},\n\t\t{address: \"/ip4/127.0.0.1/tcp/443/tls\", expected: \"https://127.0.0.1:443\"},\n\t\t{address: \"/ip4/127.0.0.1/tcp/443/https\", expected: \"https://127.0.0.1:443\"},\n\t\t{address: \"/ip4/127.0.0.1/tcp/443/tls/http\", expected: \"https://127.0.0.1:443\"},\n\t}\n\n\tfor _, tc := range testcases {\n\t\taddress, err := ma.NewMultiaddr(tc.address)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tapi, err := NewApiWithClient(address, &http.Client{})\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif api.url != tc.expected {\n\t\t\tt.Errorf(\"Expected = %s; got %s\", tc.expected, api.url)\n\t\t}\n\t}\n}\n" }, { "path": "client/rpc/apifile.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tunixfs \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n)\n\nconst forwardSeekLimit = 1 << 14 // 16k\n\nfunc (api *UnixfsAPI) Get(ctx context.Context, p path.Path) (files.Node, error) {\n\tif p.Mutable() { // use resolved path in case we are dealing with IPNS / MFS\n\t\tvar err error\n\t\tp, _, err = api.core().ResolvePath(ctx, p)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tvar stat struct {\n\t\tHash string\n\t\tType string\n\t\tSize int64 // unixfs size\n\t\tMode string\n\t\tMtime int64\n\t\tMtimeNsecs int\n\t}\n\terr := api.core().Request(\"files/stat\", p.String()).Exec(ctx, &stat)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmode, err := stringToFileMode(stat.Mode)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar modTime time.Time\n\tif stat.Mtime != 0 {\n\t\tmodTime = time.Unix(stat.Mtime, int64(stat.MtimeNsecs)).UTC()\n\t}\n\n\tswitch stat.Type {\n\tcase \"file\":\n\t\treturn api.getFile(ctx, p, stat.Size, mode, modTime)\n\tcase \"directory\":\n\t\treturn api.getDir(ctx, p, stat.Size, mode, modTime)\n\tcase \"symlink\":\n\t\treturn api.getSymlink(ctx, p, modTime)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unsupported file type '%s'\", stat.Type)\n\t}\n}\n\ntype apiFile struct {\n\tctx context.Context\n\tcore *HttpApi\n\tsize int64\n\tpath path.Path\n\n\tmode os.FileMode\n\tmtime time.Time\n\n\tr *Response\n\tat int64\n}\n\nfunc (f *apiFile) reset() error {\n\tif f.r != nil {\n\t\t_ = f.r.Cancel()\n\t\tf.r = nil\n\t}\n\treq := f.core.Request(\"cat\", f.path.String())\n\tif f.at != 0 {\n\t\treq.Option(\"offset\", f.at)\n\t}\n\tresp, err := req.Send(f.ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif resp.Error != nil {\n\t\treturn resp.Error\n\t}\n\tf.r = resp\n\treturn nil\n}\n\nfunc (f *apiFile) Read(p []byte) (int, error) {\n\tn, err := f.r.Output.Read(p)\n\tif n > 0 {\n\t\tf.at += int64(n)\n\t}\n\treturn n, err\n}\n\nfunc (f *apiFile) ReadAt(p []byte, off int64) (int, error) {\n\t// Always make a new request. This method should be parallel-safe.\n\tresp, err := f.core.Request(\"cat\", f.path.String()).\n\t\tOption(\"offset\", off).Option(\"length\", len(p)).Send(f.ctx)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tif resp.Error != nil {\n\t\treturn 0, resp.Error\n\t}\n\tdefer resp.Output.Close()\n\n\tn, err := io.ReadFull(resp.Output, p)\n\tif err == io.ErrUnexpectedEOF {\n\t\terr = io.EOF\n\t}\n\treturn n, err\n}\n\nfunc (f *apiFile) Seek(offset int64, whence int) (int64, error) {\n\tswitch whence {\n\tcase io.SeekEnd:\n\t\toffset = f.size + offset\n\tcase io.SeekCurrent:\n\t\toffset = f.at + offset\n\t}\n\tif f.at == offset { // noop\n\t\treturn offset, nil\n\t}\n\n\tif f.at < offset && offset-f.at < forwardSeekLimit { // forward skip\n\t\tr, err := io.CopyN(io.Discard, f.r.Output, offset-f.at)\n\n\t\tf.at += r\n\t\treturn f.at, err\n\t}\n\tf.at = offset\n\treturn f.at, f.reset()\n}\n\nfunc (f *apiFile) Close() error {\n\tif f.r != nil {\n\t\treturn f.r.Cancel()\n\t}\n\treturn nil\n}\n\nfunc (f *apiFile) Mode() os.FileMode {\n\treturn f.mode\n}\n\nfunc (f *apiFile) ModTime() time.Time {\n\treturn f.mtime\n}\n\nfunc (f *apiFile) Size() (int64, error) {\n\treturn f.size, nil\n}\n\nfunc stringToFileMode(mode string) (os.FileMode, error) {\n\tif mode == \"\" {\n\t\treturn 0, nil\n\t}\n\tmode64, err := strconv.ParseUint(mode, 8, 32)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"cannot parse mode %s: %s\", mode, err)\n\t}\n\treturn os.FileMode(uint32(mode64)), nil\n}\n\nfunc (api *UnixfsAPI) getFile(ctx context.Context, p path.Path, size int64, mode os.FileMode, mtime time.Time) (files.Node, error) {\n\tf := &apiFile{\n\t\tctx: ctx,\n\t\tcore: api.core(),\n\t\tsize: size,\n\t\tpath: p,\n\t\tmode: mode,\n\t\tmtime: mtime,\n\t}\n\n\treturn f, f.reset()\n}\n\ntype apiIter struct {\n\tctx context.Context\n\tcore *UnixfsAPI\n\n\terr error\n\n\tdec *json.Decoder\n\tcurFile files.Node\n\tcur lsLink\n}\n\nfunc (it *apiIter) Err() error {\n\treturn it.err\n}\n\nfunc (it *apiIter) Name() string {\n\treturn it.cur.Name\n}\n\nfunc (it *apiIter) Next() bool {\n\tif it.ctx.Err() != nil {\n\t\tit.err = it.ctx.Err()\n\t\treturn false\n\t}\n\n\tvar out lsOutput\n\tif err := it.dec.Decode(&out); err != nil {\n\t\tif err != io.EOF {\n\t\t\tit.err = err\n\t\t}\n\t\treturn false\n\t}\n\n\tif len(out.Objects) != 1 {\n\t\tit.err = fmt.Errorf(\"ls returned more objects than expected (%d)\", len(out.Objects))\n\t\treturn false\n\t}\n\n\tif len(out.Objects[0].Links) != 1 {\n\t\tit.err = fmt.Errorf(\"ls returned more links than expected (%d)\", len(out.Objects[0].Links))\n\t\treturn false\n\t}\n\n\tit.cur = out.Objects[0].Links[0]\n\tc, err := cid.Parse(it.cur.Hash)\n\tif err != nil {\n\t\tit.err = err\n\t\treturn false\n\t}\n\n\tswitch it.cur.Type {\n\tcase unixfs.THAMTShard, unixfs.TMetadata, unixfs.TDirectory:\n\t\tit.curFile, err = it.core.getDir(it.ctx, path.FromCid(c), int64(it.cur.Size), it.cur.Mode, it.cur.ModTime)\n\t\tif err != nil {\n\t\t\tit.err = err\n\t\t\treturn false\n\t\t}\n\tcase unixfs.TFile:\n\t\tit.curFile, err = it.core.getFile(it.ctx, path.FromCid(c), int64(it.cur.Size), it.cur.Mode, it.cur.ModTime)\n\t\tif err != nil {\n\t\t\tit.err = err\n\t\t\treturn false\n\t\t}\n\tcase unixfs.TSymlink:\n\t\tit.curFile, err = it.core.getSymlink(it.ctx, path.FromCid(c), it.cur.ModTime)\n\t\tif err != nil {\n\t\t\tit.err = err\n\t\t\treturn false\n\t\t}\n\tdefault:\n\t\tit.err = fmt.Errorf(\"file type %d not supported\", it.cur.Type)\n\t\treturn false\n\t}\n\treturn true\n}\n\nfunc (it *apiIter) Node() files.Node {\n\treturn it.curFile\n}\n\ntype apiDir struct {\n\tctx context.Context\n\tcore *UnixfsAPI\n\tsize int64\n\tpath path.Path\n\n\tmode os.FileMode\n\tmtime time.Time\n\n\tdec *json.Decoder\n}\n\nfunc (d *apiDir) Close() error {\n\treturn nil\n}\n\nfunc (d *apiDir) Mode() os.FileMode {\n\treturn d.mode\n}\n\nfunc (d *apiDir) ModTime() time.Time {\n\treturn d.mtime\n}\n\nfunc (d *apiDir) Size() (int64, error) {\n\treturn d.size, nil\n}\n\nfunc (d *apiDir) Entries() files.DirIterator {\n\treturn &apiIter{\n\t\tctx: d.ctx,\n\t\tcore: d.core,\n\t\tdec: d.dec,\n\t}\n}\n\nfunc (api *UnixfsAPI) getDir(ctx context.Context, p path.Path, size int64, mode os.FileMode, modTime time.Time) (files.Node, error) {\n\tresp, err := api.core().Request(\"ls\", p.String()).\n\t\tOption(\"resolve-size\", true).\n\t\tOption(\"stream\", true).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\tdata, _ := io.ReadAll(resp.Output)\n\trdr := bytes.NewReader(data)\n\n\td := &apiDir{\n\t\tctx: ctx,\n\t\tcore: api,\n\t\tsize: size,\n\t\tpath: p,\n\t\tmode: mode,\n\t\tmtime: modTime,\n\n\t\t//dec: json.NewDecoder(resp.Output),\n\t\tdec: json.NewDecoder(rdr),\n\t}\n\n\treturn d, nil\n}\n\nfunc (api *UnixfsAPI) getSymlink(ctx context.Context, p path.Path, modTime time.Time) (files.Node, error) {\n\tresp, err := api.core().Request(\"cat\", p.String()).\n\t\tOption(\"resolve-size\", true).\n\t\tOption(\"stream\", true).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\ttarget, err := io.ReadAll(resp.Output)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn files.NewSymlinkFile(string(target), modTime), nil\n}\n\nvar (\n\t_ files.File = &apiFile{}\n\t_ files.Directory = &apiDir{}\n)\n" }, { "path": "client/rpc/auth/auth.go", "content": "package auth\n\nimport \"net/http\"\n\nvar _ http.RoundTripper = &AuthorizedRoundTripper{}\n\ntype AuthorizedRoundTripper struct {\n\tauthorization string\n\troundTripper http.RoundTripper\n}\n\n// NewAuthorizedRoundTripper creates a new [http.RoundTripper] that will set the\n// Authorization HTTP header with the value of [authorization]. The given [roundTripper] is\n// the base [http.RoundTripper]. If it is nil, [http.DefaultTransport] is used.\nfunc NewAuthorizedRoundTripper(authorization string, roundTripper http.RoundTripper) http.RoundTripper {\n\tif roundTripper == nil {\n\t\troundTripper = http.DefaultTransport\n\t}\n\n\treturn &AuthorizedRoundTripper{\n\t\tauthorization: authorization,\n\t\troundTripper: roundTripper,\n\t}\n}\n\nfunc (tp *AuthorizedRoundTripper) RoundTrip(r *http.Request) (*http.Response, error) {\n\tr.Header.Set(\"Authorization\", tp.authorization)\n\treturn tp.roundTripper.RoundTrip(r)\n}\n" }, { "path": "client/rpc/block.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\tmc \"github.com/multiformats/go-multicodec\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\ntype BlockAPI HttpApi\n\ntype blockStat struct {\n\tKey string\n\tBSize int `json:\"Size\"`\n\n\tcid cid.Cid\n}\n\nfunc (s *blockStat) Size() int {\n\treturn s.BSize\n}\n\nfunc (s *blockStat) Path() path.ImmutablePath {\n\treturn path.FromCid(s.cid)\n}\n\nfunc (api *BlockAPI) Put(ctx context.Context, r io.Reader, opts ...caopts.BlockPutOption) (iface.BlockStat, error) {\n\toptions, err := caopts.BlockPutOptions(opts...)\n\tpx := options.CidPrefix\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmht, ok := mh.Codes[px.MhType]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"unknowm mhType %d\", px.MhType)\n\t}\n\n\tvar cidOptKey, cidOptVal string\n\tswitch {\n\tcase px.Version == 0 && px.Codec == cid.DagProtobuf:\n\t\t// ensure legacy --format=v0 passes as BlockPutOption still works\n\t\tcidOptKey = \"format\"\n\t\tcidOptVal = \"v0\"\n\tdefault:\n\t\t// pass codec as string\n\t\tcidOptKey = \"cid-codec\"\n\t\tcidOptVal = mc.Code(px.Codec).String()\n\t}\n\n\treq := api.core().Request(\"block/put\").\n\t\tOption(\"mhtype\", mht).\n\t\tOption(\"mhlen\", px.MhLength).\n\t\tOption(cidOptKey, cidOptVal).\n\t\tOption(\"pin\", options.Pin).\n\t\tFileBody(r)\n\n\tvar out blockStat\n\tif err := req.Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\tout.cid, err = cid.Parse(out.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &out, nil\n}\n\nfunc (api *BlockAPI) Get(ctx context.Context, p path.Path) (io.Reader, error) {\n\tresp, err := api.core().Request(\"block/get\", p.String()).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, parseErrNotFoundWithFallbackToError(resp.Error)\n\t}\n\n\t// TODO: make get return ReadCloser to avoid copying\n\tdefer resp.Close()\n\tb := new(bytes.Buffer)\n\tif _, err := io.Copy(b, resp.Output); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn b, nil\n}\n\nfunc (api *BlockAPI) Rm(ctx context.Context, p path.Path, opts ...caopts.BlockRmOption) error {\n\toptions, err := caopts.BlockRmOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tremovedBlock := struct {\n\t\tHash string `json:\",omitempty\"`\n\t\tError string `json:\",omitempty\"`\n\t}{}\n\n\treq := api.core().Request(\"block/rm\").\n\t\tOption(\"force\", options.Force).\n\t\tArguments(p.String())\n\n\tif err := req.Exec(ctx, &removedBlock); err != nil {\n\t\treturn err\n\t}\n\n\treturn parseErrNotFoundWithFallbackToMSG(removedBlock.Error)\n}\n\nfunc (api *BlockAPI) Stat(ctx context.Context, p path.Path) (iface.BlockStat, error) {\n\tvar out blockStat\n\terr := api.core().Request(\"block/stat\", p.String()).Exec(ctx, &out)\n\tif err != nil {\n\t\treturn nil, parseErrNotFoundWithFallbackToError(err)\n\t}\n\tout.cid, err = cid.Parse(out.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &out, nil\n}\n\nfunc (api *BlockAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/dag.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/go-cid\"\n\tformat \"github.com/ipfs/go-ipld-format\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\tmulticodec \"github.com/multiformats/go-multicodec\"\n)\n\ntype (\n\thttpNodeAdder HttpApi\n\tHttpDagServ httpNodeAdder\n\tpinningHttpNodeAdder httpNodeAdder\n)\n\nfunc (api *HttpDagServ) Get(ctx context.Context, c cid.Cid) (format.Node, error) {\n\tr, err := api.core().Block().Get(ctx, path.FromCid(c))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdata, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tblk, err := blocks.NewBlockWithCid(data, c)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api.ipldDecoder.DecodeNode(ctx, blk)\n}\n\nfunc (api *HttpDagServ) GetMany(ctx context.Context, cids []cid.Cid) <-chan *format.NodeOption {\n\tout := make(chan *format.NodeOption)\n\n\tfor _, c := range cids {\n\t\t// TODO: Consider limiting concurrency of this somehow\n\t\tgo func(c cid.Cid) {\n\t\t\tn, err := api.Get(ctx, c)\n\n\t\t\tselect {\n\t\t\tcase out <- &format.NodeOption{Node: n, Err: err}:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t}(c)\n\t}\n\treturn out\n}\n\nfunc (api *httpNodeAdder) add(ctx context.Context, nd format.Node, pin bool) error {\n\tc := nd.Cid()\n\tprefix := c.Prefix()\n\n\t// preserve 'cid-codec' when sent over HTTP\n\tcidCodec := multicodec.Code(prefix.Codec).String()\n\n\t// 'format' got replaced by 'cid-codec' in https://github.com/ipfs/interface-go-ipfs-core/pull/80\n\t// but we still support it here for backward-compatibility with use of CIDv0\n\tformat := \"\"\n\tif prefix.Version == 0 {\n\t\tcidCodec = \"\"\n\t\tformat = \"v0\"\n\t}\n\n\tstat, err := api.core().Block().Put(ctx, bytes.NewReader(nd.RawData()),\n\t\toptions.Block.Hash(prefix.MhType, prefix.MhLength),\n\t\toptions.Block.CidCodec(cidCodec),\n\t\toptions.Block.Format(format),\n\t\toptions.Block.Pin(pin))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif !stat.Path().RootCid().Equals(c) {\n\t\treturn fmt.Errorf(\"cids didn't match - local %s, remote %s\", c.String(), stat.Path().RootCid().String())\n\t}\n\treturn nil\n}\n\nfunc (api *httpNodeAdder) addMany(ctx context.Context, nds []format.Node, pin bool) error {\n\tfor _, nd := range nds {\n\t\t// TODO: optimize\n\t\tif err := api.add(ctx, nd, pin); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (api *HttpDagServ) AddMany(ctx context.Context, nds []format.Node) error {\n\treturn (*httpNodeAdder)(api).addMany(ctx, nds, false)\n}\n\nfunc (api *HttpDagServ) Add(ctx context.Context, nd format.Node) error {\n\treturn (*httpNodeAdder)(api).add(ctx, nd, false)\n}\n\nfunc (api *pinningHttpNodeAdder) Add(ctx context.Context, nd format.Node) error {\n\treturn (*httpNodeAdder)(api).add(ctx, nd, true)\n}\n\nfunc (api *pinningHttpNodeAdder) AddMany(ctx context.Context, nds []format.Node) error {\n\treturn (*httpNodeAdder)(api).addMany(ctx, nds, true)\n}\n\nfunc (api *HttpDagServ) Pinning() format.NodeAdder {\n\treturn (*pinningHttpNodeAdder)(api)\n}\n\nfunc (api *HttpDagServ) Remove(ctx context.Context, c cid.Cid) error {\n\treturn api.core().Block().Rm(ctx, path.FromCid(c)) // TODO: should we force rm?\n}\n\nfunc (api *HttpDagServ) RemoveMany(ctx context.Context, cids []cid.Cid) error {\n\tfor _, c := range cids {\n\t\t// TODO: optimize\n\t\tif err := api.Remove(ctx, c); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (api *httpNodeAdder) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n\nfunc (api *HttpDagServ) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/errors.go", "content": "package rpc\n\nimport (\n\t\"errors\"\n\t\"strings\"\n\t\"unicode/utf8\"\n\n\t\"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\n// This file handle parsing and returning the correct ABI based errors from error messages\n\ntype prePostWrappedNotFoundError struct {\n\tpre string\n\tpost string\n\n\twrapped ipld.ErrNotFound\n}\n\nfunc (e prePostWrappedNotFoundError) String() string {\n\treturn e.Error()\n}\n\nfunc (e prePostWrappedNotFoundError) Error() string {\n\treturn e.pre + e.wrapped.Error() + e.post\n}\n\nfunc (e prePostWrappedNotFoundError) Unwrap() error {\n\treturn e.wrapped\n}\n\nfunc parseErrNotFoundWithFallbackToMSG(msg string) error {\n\terr, handled := parseErrNotFound(msg)\n\tif handled {\n\t\treturn err\n\t}\n\n\treturn errors.New(msg)\n}\n\nfunc parseErrNotFoundWithFallbackToError(msg error) error {\n\terr, handled := parseErrNotFound(msg.Error())\n\tif handled {\n\t\treturn err\n\t}\n\n\treturn msg\n}\n\nfunc parseErrNotFound(msg string) (error, bool) {\n\tif msg == \"\" {\n\t\treturn nil, true // Fast path\n\t}\n\n\tif err, handled := parseIPLDErrNotFound(msg); handled {\n\t\treturn err, true\n\t}\n\n\tif err, handled := parseBlockstoreNotFound(msg); handled {\n\t\treturn err, true\n\t}\n\n\treturn nil, false\n}\n\n// Assume CIDs break on:\n// - Whitespaces: \" \\t\\n\\r\\v\\f\"\n// - Semicolon: \";\" this is to parse ipld.ErrNotFound wrapped in multierr\n// - Double Quotes: \"\\\"\" this is for parsing %q and %#v formatting.\nconst cidBreakSet = \" \\t\\n\\r\\v\\f;\\\"\"\n\nfunc parseIPLDErrNotFound(msg string) (error, bool) {\n\t// The pattern we search for is:\n\tconst ipldErrNotFoundKey = \"ipld: could not find \" /*CID*/\n\t// We try to parse the CID, if it's invalid we give up and return a simple text error.\n\t// We also accept \"node\" in place of the CID because that means it's an Undefined CID.\n\n\tkeyIndex := strings.Index(msg, ipldErrNotFoundKey)\n\n\tif keyIndex < 0 { // Unknown error\n\t\treturn nil, false\n\t}\n\n\tcidStart := keyIndex + len(ipldErrNotFoundKey)\n\n\tmsgPostKey := msg[cidStart:]\n\tvar c cid.Cid\n\tvar postIndex int\n\tif strings.HasPrefix(msgPostKey, \"node\") {\n\t\t// Fallback case\n\t\tc = cid.Undef\n\t\tpostIndex = len(\"node\")\n\t} else {\n\t\tpostIndex = strings.IndexFunc(msgPostKey, func(r rune) bool {\n\t\t\treturn strings.ContainsAny(string(r), cidBreakSet)\n\t\t})\n\t\tif postIndex < 0 {\n\t\t\t// no breakage meaning the string look like this something + \"ipld: could not find bafy\"\n\t\t\tpostIndex = len(msgPostKey)\n\t\t}\n\n\t\tcidStr := msgPostKey[:postIndex]\n\n\t\tvar err error\n\t\tc, err = cid.Decode(cidStr)\n\t\tif err != nil {\n\t\t\t// failed to decode CID give up\n\t\t\treturn nil, false\n\t\t}\n\n\t\t// check that the CID is either a CIDv0 or a base32 multibase\n\t\t// because that what ipld.ErrNotFound.Error() -> cid.Cid.String() do currently\n\t\tif c.Version() != 0 {\n\t\t\tbaseRune, _ := utf8.DecodeRuneInString(cidStr)\n\t\t\tif baseRune == utf8.RuneError || baseRune != mbase.Base32 {\n\t\t\t\t// not a multibase we expect, give up\n\t\t\t\treturn nil, false\n\t\t\t}\n\t\t}\n\t}\n\n\terr := ipld.ErrNotFound{Cid: c}\n\tpre := msg[:keyIndex]\n\tpost := msgPostKey[postIndex:]\n\n\tif len(pre) > 0 || len(post) > 0 {\n\t\treturn prePostWrappedNotFoundError{\n\t\t\tpre: pre,\n\t\t\tpost: post,\n\t\t\twrapped: err,\n\t\t}, true\n\t}\n\n\treturn err, true\n}\n\n// This is a simple error type that just return msg as Error().\n// But that also match ipld.ErrNotFound when called with Is(err).\n// That is needed to keep compatibility with code that use string.Contains(err.Error(), \"blockstore: block not found\")\n// and code using ipld.ErrNotFound.\ntype blockstoreNotFoundMatchingIPLDErrNotFound struct {\n\tmsg string\n}\n\nfunc (e blockstoreNotFoundMatchingIPLDErrNotFound) String() string {\n\treturn e.Error()\n}\n\nfunc (e blockstoreNotFoundMatchingIPLDErrNotFound) Error() string {\n\treturn e.msg\n}\n\nfunc (e blockstoreNotFoundMatchingIPLDErrNotFound) Is(err error) bool {\n\t_, ok := err.(ipld.ErrNotFound)\n\treturn ok\n}\n\nfunc parseBlockstoreNotFound(msg string) (error, bool) {\n\tif !strings.Contains(msg, \"blockstore: block not found\") {\n\t\treturn nil, false\n\t}\n\n\treturn blockstoreNotFoundMatchingIPLDErrNotFound{msg: msg}, true\n}\n" }, { "path": "client/rpc/errors_test.go", "content": "package rpc\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tmbase \"github.com/multiformats/go-multibase\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nvar randomSha256MH = mh.Multihash{0x12, 0x20, 0x88, 0x82, 0x73, 0x37, 0x7c, 0xc1, 0xc9, 0x96, 0xad, 0xee, 0xd, 0x26, 0x84, 0x2, 0xc9, 0xc9, 0x5c, 0xf9, 0x5c, 0x4d, 0x9b, 0xc3, 0x3f, 0xfb, 0x4a, 0xd8, 0xaf, 0x28, 0x6b, 0xca, 0x1a, 0xf2}\n\nfunc doParseIpldNotFoundTest(t *testing.T, original error) {\n\toriginalMsg := original.Error()\n\n\trebuilt := parseErrNotFoundWithFallbackToMSG(originalMsg)\n\n\trebuiltMsg := rebuilt.Error()\n\n\tif originalMsg != rebuiltMsg {\n\t\tt.Errorf(\"expected message to be %q; got %q\", originalMsg, rebuiltMsg)\n\t}\n\n\toriginalNotFound := ipld.IsNotFound(original)\n\trebuiltNotFound := ipld.IsNotFound(rebuilt)\n\tif originalNotFound != rebuiltNotFound {\n\t\tt.Errorf(\"for %q expected Ipld.IsNotFound to be %t; got %t\", originalMsg, originalNotFound, rebuiltNotFound)\n\t}\n}\n\nfunc TestParseIPLDNotFound(t *testing.T) {\n\tt.Parallel()\n\n\tif err := parseErrNotFoundWithFallbackToMSG(\"\"); err != nil {\n\t\tt.Errorf(\"expected empty string to give no error; got %T %q\", err, err.Error())\n\t}\n\n\tcidBreaks := make([]string, len(cidBreakSet))\n\tfor i, v := range cidBreakSet {\n\t\tcidBreaks[i] = \"%w\" + string(v)\n\t}\n\n\tbase58BTCEncoder, err := mbase.NewEncoder(mbase.Base58BTC)\n\tif err != nil {\n\t\tt.Fatalf(\"expected to find Base58BTC encoder; got error %q\", err.Error())\n\t}\n\n\tfor _, wrap := range append(cidBreaks,\n\t\t\"\",\n\t\t\"merkledag: %w\",\n\t\t\"testing: %w the test\",\n\t\t\"%w is wrong\",\n\t) {\n\t\tfor _, err := range [...]error{\n\t\t\terrors.New(\"ipld: could not find \"),\n\t\t\terrors.New(\"ipld: could not find Bad_CID\"),\n\t\t\terrors.New(\"ipld: could not find \" + cid.NewCidV1(cid.Raw, randomSha256MH).Encode(base58BTCEncoder)), // Test that we only accept CIDv0 and base32 CIDs\n\t\t\terrors.New(\"network connection timeout\"),\n\t\t\tipld.ErrNotFound{Cid: cid.Undef},\n\t\t\tipld.ErrNotFound{Cid: cid.NewCidV0(randomSha256MH)},\n\t\t\tipld.ErrNotFound{Cid: cid.NewCidV1(cid.Raw, randomSha256MH)},\n\t\t} {\n\t\t\tif wrap != \"\" {\n\t\t\t\terr = fmt.Errorf(wrap, err)\n\t\t\t}\n\n\t\t\tdoParseIpldNotFoundTest(t, err)\n\t\t}\n\t}\n}\n\nfunc TestBlockstoreNotFoundMatchingIPLDErrNotFound(t *testing.T) {\n\tt.Parallel()\n\n\tif !ipld.IsNotFound(blockstoreNotFoundMatchingIPLDErrNotFound{}) {\n\t\tt.Fatalf(\"expected blockstoreNotFoundMatchingIPLDErrNotFound to match ipld.IsNotFound; got false\")\n\t}\n\n\tfor _, wrap := range [...]string{\n\t\t\"\",\n\t\t\"merkledag: %w\",\n\t\t\"testing: %w the test\",\n\t\t\"%w is wrong\",\n\t} {\n\t\tfor _, err := range [...]error{\n\t\t\terrors.New(\"network connection timeout\"),\n\t\t\tblockstoreNotFoundMatchingIPLDErrNotFound{\"blockstore: block not found\"},\n\t\t} {\n\t\t\tif wrap != \"\" {\n\t\t\t\terr = fmt.Errorf(wrap, err)\n\t\t\t}\n\n\t\t\tdoParseIpldNotFoundTest(t, err)\n\t\t}\n\t}\n}\n" }, { "path": "client/rpc/key.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/path\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/multiformats/go-multibase\"\n)\n\ntype KeyAPI HttpApi\n\ntype key struct {\n\tname string\n\tpid peer.ID\n\tpath path.Path\n}\n\nfunc newKey(name, pidStr string) (*key, error) {\n\tpid, err := peer.Decode(pidStr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpath, err := path.NewPath(\"/ipns/\" + ipns.NameFromPeer(pid).String())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &key{name: name, pid: pid, path: path}, nil\n}\n\nfunc (k *key) Name() string {\n\treturn k.name\n}\n\nfunc (k *key) Path() path.Path {\n\treturn k.path\n}\n\nfunc (k *key) ID() peer.ID {\n\treturn k.pid\n}\n\ntype keyOutput struct {\n\tName string\n\tId string\n}\n\nfunc (api *KeyAPI) Generate(ctx context.Context, name string, opts ...caopts.KeyGenerateOption) (iface.Key, error) {\n\toptions, err := caopts.KeyGenerateOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar out keyOutput\n\terr = api.core().Request(\"key/gen\", name).\n\t\tOption(\"type\", options.Algorithm).\n\t\tOption(\"size\", options.Size).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn newKey(out.Name, out.Id)\n}\n\nfunc (api *KeyAPI) Rename(ctx context.Context, oldName string, newName string, opts ...caopts.KeyRenameOption) (iface.Key, bool, error) {\n\toptions, err := caopts.KeyRenameOptions(opts...)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\tvar out struct {\n\t\tWas string\n\t\tNow string\n\t\tId string\n\t\tOverwrite bool\n\t}\n\terr = api.core().Request(\"key/rename\", oldName, newName).\n\t\tOption(\"force\", options.Force).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\tkey, err := newKey(out.Now, out.Id)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\treturn key, out.Overwrite, err\n}\n\nfunc (api *KeyAPI) List(ctx context.Context) ([]iface.Key, error) {\n\tvar out struct {\n\t\tKeys []keyOutput\n\t}\n\tif err := api.core().Request(\"key/ls\").Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\n\tres := make([]iface.Key, len(out.Keys))\n\tfor i, k := range out.Keys {\n\t\tkey, err := newKey(k.Name, k.Id)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres[i] = key\n\t}\n\n\treturn res, nil\n}\n\nfunc (api *KeyAPI) Self(ctx context.Context) (iface.Key, error) {\n\tvar id struct{ ID string }\n\tif err := api.core().Request(\"id\").Exec(ctx, &id); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn newKey(\"self\", id.ID)\n}\n\nfunc (api *KeyAPI) Remove(ctx context.Context, name string) (iface.Key, error) {\n\tvar out struct {\n\t\tKeys []keyOutput\n\t}\n\tif err := api.core().Request(\"key/rm\", name).Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\tif len(out.Keys) != 1 {\n\t\treturn nil, errors.New(\"got unexpected number of keys back\")\n\t}\n\n\treturn newKey(out.Keys[0].Name, out.Keys[0].Id)\n}\n\nfunc (api *KeyAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n\nfunc (api *KeyAPI) Sign(ctx context.Context, name string, data []byte) (iface.Key, []byte, error) {\n\tvar out struct {\n\t\tKey keyOutput\n\t\tSignature string\n\t}\n\n\terr := api.core().Request(\"key/sign\").\n\t\tOption(\"key\", name).\n\t\tFileBody(bytes.NewReader(data)).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tkey, err := newKey(out.Key.Name, out.Key.Id)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\t_, signature, err := multibase.Decode(out.Signature)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\treturn key, signature, nil\n}\n\nfunc (api *KeyAPI) Verify(ctx context.Context, keyOrName string, signature, data []byte) (iface.Key, bool, error) {\n\tvar out struct {\n\t\tKey keyOutput\n\t\tSignatureValid bool\n\t}\n\n\terr := api.core().Request(\"key/verify\").\n\t\tOption(\"key\", keyOrName).\n\t\tOption(\"signature\", toMultibase(signature)).\n\t\tFileBody(bytes.NewReader(data)).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\tkey, err := newKey(out.Key.Name, out.Key.Id)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\treturn key, out.SignatureValid, nil\n}\n" }, { "path": "client/rpc/name.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/path\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\ntype NameAPI HttpApi\n\ntype ipnsEntry struct {\n\tName string `json:\"Name\"`\n\tValue string `json:\"Value\"`\n}\n\nfunc (api *NameAPI) Publish(ctx context.Context, p path.Path, opts ...caopts.NamePublishOption) (ipns.Name, error) {\n\toptions, err := caopts.NamePublishOptions(opts...)\n\tif err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\n\treq := api.core().Request(\"name/publish\", p.String()).\n\t\tOption(\"key\", options.Key).\n\t\tOption(\"allow-offline\", options.AllowOffline).\n\t\tOption(\"lifetime\", options.ValidTime).\n\t\tOption(\"resolve\", false)\n\n\tif options.TTL != nil {\n\t\treq.Option(\"ttl\", options.TTL)\n\t}\n\n\tvar out ipnsEntry\n\tif err := req.Exec(ctx, &out); err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\treturn ipns.NameFromString(out.Name)\n}\n\nfunc (api *NameAPI) Search(ctx context.Context, name string, opts ...caopts.NameResolveOption) (<-chan iface.IpnsResult, error) {\n\toptions, err := caopts.NameResolveOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tropts := namesys.ProcessResolveOptions(options.ResolveOpts)\n\tif ropts.Depth != namesys.DefaultDepthLimit && ropts.Depth != 1 {\n\t\treturn nil, fmt.Errorf(\"Name.Resolve: depth other than 1 or %d not supported\", namesys.DefaultDepthLimit)\n\t}\n\n\treq := api.core().Request(\"name/resolve\", name).\n\t\tOption(\"nocache\", !options.Cache).\n\t\tOption(\"recursive\", ropts.Depth != 1).\n\t\tOption(\"dht-record-count\", ropts.DhtRecordCount).\n\t\tOption(\"dht-timeout\", ropts.DhtTimeout).\n\t\tOption(\"stream\", true)\n\tresp, err := req.Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\tres := make(chan iface.IpnsResult)\n\n\tgo func() {\n\t\tdefer close(res)\n\t\tdefer resp.Close()\n\n\t\tdec := json.NewDecoder(resp.Output)\n\n\t\tfor {\n\t\t\tvar out struct{ Path string }\n\t\t\terr := dec.Decode(&out)\n\t\t\tif err == io.EOF {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tvar ires iface.IpnsResult\n\t\t\tif err == nil {\n\t\t\t\tp, err := path.NewPath(out.Path)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tires.Path = p\n\t\t\t}\n\n\t\t\tselect {\n\t\t\tcase res <- ires:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn res, nil\n}\n\nfunc (api *NameAPI) Resolve(ctx context.Context, name string, opts ...caopts.NameResolveOption) (path.Path, error) {\n\toptions, err := caopts.NameResolveOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tropts := namesys.ProcessResolveOptions(options.ResolveOpts)\n\tif ropts.Depth != namesys.DefaultDepthLimit && ropts.Depth != 1 {\n\t\treturn nil, fmt.Errorf(\"Name.Resolve: depth other than 1 or %d not supported\", namesys.DefaultDepthLimit)\n\t}\n\n\treq := api.core().Request(\"name/resolve\", name).\n\t\tOption(\"nocache\", !options.Cache).\n\t\tOption(\"recursive\", ropts.Depth != 1).\n\t\tOption(\"dht-record-count\", ropts.DhtRecordCount).\n\t\tOption(\"dht-timeout\", ropts.DhtTimeout)\n\n\tvar out struct{ Path string }\n\tif err := req.Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn path.NewPath(out.Path)\n}\n\nfunc (api *NameAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/object.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\ntype ObjectAPI HttpApi\n\ntype objectOut struct {\n\tHash string\n}\n\nfunc (api *ObjectAPI) AddLink(ctx context.Context, base path.Path, name string, child path.Path, opts ...caopts.ObjectAddLinkOption) (path.ImmutablePath, error) {\n\toptions, err := caopts.ObjectAddLinkOptions(opts...)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tvar out objectOut\n\terr = api.core().Request(\"object/patch/add-link\", base.String(), name, child.String()).\n\t\tOption(\"create\", options.Create).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tc, err := cid.Parse(out.Hash)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn path.FromCid(c), nil\n}\n\nfunc (api *ObjectAPI) RmLink(ctx context.Context, base path.Path, link string) (path.ImmutablePath, error) {\n\tvar out objectOut\n\terr := api.core().Request(\"object/patch/rm-link\", base.String(), link).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tc, err := cid.Parse(out.Hash)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn path.FromCid(c), nil\n}\n\ntype change struct {\n\tType iface.ChangeType\n\tPath string\n\tBefore cid.Cid\n\tAfter cid.Cid\n}\n\nfunc (api *ObjectAPI) Diff(ctx context.Context, a path.Path, b path.Path) ([]iface.ObjectChange, error) {\n\tvar out struct {\n\t\tChanges []change\n\t}\n\tif err := api.core().Request(\"object/diff\", a.String(), b.String()).Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\tres := make([]iface.ObjectChange, len(out.Changes))\n\tfor i, ch := range out.Changes {\n\t\tres[i] = iface.ObjectChange{\n\t\t\tType: ch.Type,\n\t\t\tPath: ch.Path,\n\t\t}\n\t\tif ch.Before != cid.Undef {\n\t\t\tres[i].Before = path.FromCid(ch.Before)\n\t\t}\n\t\tif ch.After != cid.Undef {\n\t\t\tres[i].After = path.FromCid(ch.After)\n\t\t}\n\t}\n\treturn res, nil\n}\n\nfunc (api *ObjectAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/path.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tcid \"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n)\n\nfunc (api *HttpApi) ResolvePath(ctx context.Context, p path.Path) (path.ImmutablePath, []string, error) {\n\tvar out struct {\n\t\tCid cid.Cid\n\t\tRemPath string\n\t}\n\n\tvar err error\n\tif p.Namespace() == path.IPNSNamespace {\n\t\tif p, err = api.Name().Resolve(ctx, p.String()); err != nil {\n\t\t\treturn path.ImmutablePath{}, nil, err\n\t\t}\n\t}\n\n\tif err := api.Request(\"dag/resolve\", p.String()).Exec(ctx, &out); err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\tp, err = path.NewPathFromSegments(p.Namespace(), out.Cid.String(), out.RemPath)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\timPath, err := path.NewImmutablePath(p)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\treturn imPath, path.StringToSegments(out.RemPath), nil\n}\n\nfunc (api *HttpApi) ResolveNode(ctx context.Context, p path.Path) (ipld.Node, error) {\n\trp, _, err := api.ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api.Dag().Get(ctx, rp.RootCid())\n}\n" }, { "path": "client/rpc/pin.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"io\"\n\t\"strings\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\ntype PinAPI HttpApi\n\ntype pinRefKeyObject struct {\n\tType string\n}\n\ntype pinRefKeyList struct {\n\tKeys map[string]pinRefKeyObject\n}\n\ntype pin struct {\n\tpath path.ImmutablePath\n\ttyp string\n\tname string\n\terr error\n}\n\nfunc (p pin) Err() error {\n\treturn p.err\n}\n\nfunc (p pin) Path() path.ImmutablePath {\n\treturn p.path\n}\n\nfunc (p pin) Name() string {\n\treturn p.name\n}\n\nfunc (p pin) Type() string {\n\treturn p.typ\n}\n\nfunc (api *PinAPI) Add(ctx context.Context, p path.Path, opts ...caopts.PinAddOption) error {\n\toptions, err := caopts.PinAddOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treq := api.core().Request(\"pin/add\", p.String()).\n\t\tOption(\"recursive\", options.Recursive)\n\tif options.Name != \"\" {\n\t\treq = req.Option(\"name\", options.Name)\n\t}\n\treturn req.Exec(ctx, nil)\n}\n\ntype pinLsObject struct {\n\tCid string\n\tName string\n\tType string\n}\n\nfunc (api *PinAPI) Ls(ctx context.Context, pins chan<- iface.Pin, opts ...caopts.PinLsOption) error {\n\tdefer close(pins)\n\n\toptions, err := caopts.PinLsOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tres, err := api.core().Request(\"pin/ls\").\n\t\tOption(\"type\", options.Type).\n\t\tOption(\"names\", options.Detailed).\n\t\tOption(\"stream\", true).\n\t\tSend(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer res.Output.Close()\n\n\tdec := json.NewDecoder(res.Output)\n\tfor {\n\t\tvar out pinLsObject\n\t\terr := dec.Decode(&out)\n\t\tif err != nil {\n\t\t\tif err != io.EOF {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tc, err := cid.Parse(out.Cid)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tselect {\n\t\tcase pins <- pin{typ: out.Type, name: out.Name, path: path.FromCid(c)}:\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\t}\n\t}\n}\n\n// IsPinned returns whether or not the given cid is pinned\n// and an explanation of why its pinned.\nfunc (api *PinAPI) IsPinned(ctx context.Context, p path.Path, opts ...caopts.PinIsPinnedOption) (string, bool, error) {\n\toptions, err := caopts.PinIsPinnedOptions(opts...)\n\tif err != nil {\n\t\treturn \"\", false, err\n\t}\n\tvar out pinRefKeyList\n\terr = api.core().Request(\"pin/ls\").\n\t\tOption(\"type\", options.WithType).\n\t\tOption(\"arg\", p.String()).\n\t\tExec(ctx, &out)\n\tif err != nil {\n\t\t// TODO: This error-type discrimination based on sub-string matching is brittle.\n\t\t// It is addressed by this open issue: https://github.com/ipfs/go-ipfs/issues/7563\n\t\tif strings.Contains(err.Error(), \"is not pinned\") {\n\t\t\treturn \"\", false, nil\n\t\t}\n\t\treturn \"\", false, err\n\t}\n\n\tfor _, obj := range out.Keys {\n\t\treturn obj.Type, true, nil\n\t}\n\treturn \"\", false, errors.New(\"http api returned no error and no results\")\n}\n\nfunc (api *PinAPI) Rm(ctx context.Context, p path.Path, opts ...caopts.PinRmOption) error {\n\toptions, err := caopts.PinRmOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn api.core().Request(\"pin/rm\", p.String()).\n\t\tOption(\"recursive\", options.Recursive).\n\t\tExec(ctx, nil)\n}\n\nfunc (api *PinAPI) Update(ctx context.Context, from path.Path, to path.Path, opts ...caopts.PinUpdateOption) error {\n\toptions, err := caopts.PinUpdateOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn api.core().Request(\"pin/update\", from.String(), to.String()).\n\t\tOption(\"unpin\", options.Unpin).Exec(ctx, nil)\n}\n\ntype pinVerifyRes struct {\n\tok bool\n\tbadNodes []iface.BadPinNode\n\terr error\n}\n\nfunc (r pinVerifyRes) Ok() bool {\n\treturn r.ok\n}\n\nfunc (r pinVerifyRes) BadNodes() []iface.BadPinNode {\n\treturn r.badNodes\n}\n\nfunc (r pinVerifyRes) Err() error {\n\treturn r.err\n}\n\ntype badNode struct {\n\terr error\n\tcid cid.Cid\n}\n\nfunc (n badNode) Path() path.ImmutablePath {\n\treturn path.FromCid(n.cid)\n}\n\nfunc (n badNode) Err() error {\n\treturn n.err\n}\n\nfunc (api *PinAPI) Verify(ctx context.Context) (<-chan iface.PinStatus, error) {\n\tresp, err := api.core().Request(\"pin/verify\").Option(\"verbose\", true).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\tres := make(chan iface.PinStatus)\n\n\tgo func() {\n\t\tdefer resp.Close()\n\t\tdefer close(res)\n\t\tdec := json.NewDecoder(resp.Output)\n\t\tfor {\n\t\t\tvar out struct {\n\t\t\t\tCid string\n\t\t\t\tErr string\n\t\t\t\tOk bool\n\n\t\t\t\tBadNodes []struct {\n\t\t\t\t\tCid string\n\t\t\t\t\tErr string\n\t\t\t\t}\n\t\t\t}\n\t\t\tif err := dec.Decode(&out); err != nil {\n\t\t\t\tif err == io.EOF {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tselect {\n\t\t\t\tcase res <- pinVerifyRes{err: err}:\n\t\t\t\t\treturn\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif out.Err != \"\" {\n\t\t\t\tselect {\n\t\t\t\tcase res <- pinVerifyRes{err: errors.New(out.Err)}:\n\t\t\t\t\treturn\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tbadNodes := make([]iface.BadPinNode, len(out.BadNodes))\n\t\t\tfor i, n := range out.BadNodes {\n\t\t\t\tc, err := cid.Decode(n.Cid)\n\t\t\t\tif err != nil {\n\t\t\t\t\tbadNodes[i] = badNode{cid: c, err: err}\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\tif n.Err != \"\" {\n\t\t\t\t\terr = errors.New(n.Err)\n\t\t\t\t}\n\t\t\t\tbadNodes[i] = badNode{cid: c, err: err}\n\t\t\t}\n\n\t\t\tselect {\n\t\t\tcase res <- pinVerifyRes{ok: out.Ok, badNodes: badNodes}:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn res, nil\n}\n\nfunc (api *PinAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/pubsub.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\ntype PubsubAPI HttpApi\n\nfunc (api *PubsubAPI) Ls(ctx context.Context) ([]string, error) {\n\tvar out struct {\n\t\tStrings []string\n\t}\n\n\tif err := api.core().Request(\"pubsub/ls\").Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\ttopics := make([]string, len(out.Strings))\n\tfor n, mb := range out.Strings {\n\t\t_, topic, err := mbase.Decode(mb)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\ttopics[n] = string(topic)\n\t}\n\treturn topics, nil\n}\n\nfunc (api *PubsubAPI) Peers(ctx context.Context, opts ...caopts.PubSubPeersOption) ([]peer.ID, error) {\n\toptions, err := caopts.PubSubPeersOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar out struct {\n\t\tStrings []string\n\t}\n\n\tvar optionalTopic string\n\tif len(options.Topic) > 0 {\n\t\toptionalTopic = toMultibase([]byte(options.Topic))\n\t}\n\tif err := api.core().Request(\"pubsub/peers\", optionalTopic).Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\n\tres := make([]peer.ID, len(out.Strings))\n\tfor i, sid := range out.Strings {\n\t\tid, err := peer.Decode(sid)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres[i] = id\n\t}\n\treturn res, nil\n}\n\nfunc (api *PubsubAPI) Publish(ctx context.Context, topic string, message []byte) error {\n\treturn api.core().Request(\"pubsub/pub\", toMultibase([]byte(topic))).\n\t\tFileBody(bytes.NewReader(message)).\n\t\tExec(ctx, nil)\n}\n\ntype pubsubSub struct {\n\tmessages chan pubsubMessage\n\n\tdone chan struct{}\n\trcloser func() error\n}\n\ntype pubsubMessage struct {\n\tJFrom string `json:\"from,omitempty\"`\n\tJData string `json:\"data,omitempty\"`\n\tJSeqno string `json:\"seqno,omitempty\"`\n\tJTopicIDs []string `json:\"topicIDs,omitempty\"`\n\n\t// real values after unpacking from text/multibase envelopes\n\tfrom peer.ID\n\tdata []byte\n\tseqno []byte\n\ttopics []string\n\n\terr error\n}\n\nfunc (msg *pubsubMessage) From() peer.ID {\n\treturn msg.from\n}\n\nfunc (msg *pubsubMessage) Data() []byte {\n\treturn msg.data\n}\n\nfunc (msg *pubsubMessage) Seq() []byte {\n\treturn msg.seqno\n}\n\n// TODO: do we want to keep this interface as []string,\n// or change to more correct [][]byte?\nfunc (msg *pubsubMessage) Topics() []string {\n\treturn msg.topics\n}\n\nfunc (s *pubsubSub) Next(ctx context.Context) (iface.PubSubMessage, error) {\n\tselect {\n\tcase msg, ok := <-s.messages:\n\t\tif !ok {\n\t\t\treturn nil, io.EOF\n\t\t}\n\t\tif msg.err != nil {\n\t\t\treturn nil, msg.err\n\t\t}\n\t\t// unpack values from text/multibase envelopes\n\t\tvar err error\n\t\tmsg.from, err = peer.Decode(msg.JFrom)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\t_, msg.data, err = mbase.Decode(msg.JData)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\t_, msg.seqno, err = mbase.Decode(msg.JSeqno)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tfor _, mbt := range msg.JTopicIDs {\n\t\t\t_, topic, err := mbase.Decode(mbt)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tmsg.topics = append(msg.topics, string(topic))\n\t\t}\n\t\treturn &msg, nil\n\tcase <-ctx.Done():\n\t\treturn nil, ctx.Err()\n\t}\n}\n\nfunc (api *PubsubAPI) Subscribe(ctx context.Context, topic string, opts ...caopts.PubSubSubscribeOption) (iface.PubSubSubscription, error) {\n\t/* right now we have no options (discover got deprecated)\n\toptions, err := caopts.PubSubSubscribeOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\t*/\n\tresp, err := api.core().Request(\"pubsub/sub\", toMultibase([]byte(topic))).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\tsub := &pubsubSub{\n\t\tmessages: make(chan pubsubMessage),\n\t\tdone: make(chan struct{}),\n\t\trcloser: func() error {\n\t\t\treturn resp.Cancel()\n\t\t},\n\t}\n\n\tdec := json.NewDecoder(resp.Output)\n\n\tgo func() {\n\t\tdefer close(sub.messages)\n\n\t\tfor {\n\t\t\tvar msg pubsubMessage\n\t\t\tif err := dec.Decode(&msg); err != nil {\n\t\t\t\tif err == io.EOF {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tmsg.err = err\n\t\t\t}\n\n\t\t\tselect {\n\t\t\tcase sub.messages <- msg:\n\t\t\tcase <-sub.done:\n\t\t\t\treturn\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn sub, nil\n}\n\nfunc (s *pubsubSub) Close() error {\n\tif s.done != nil {\n\t\tclose(s.done)\n\t\ts.done = nil\n\t}\n\treturn s.rcloser()\n}\n\nfunc (api *PubsubAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n\n// Encodes bytes into URL-safe multibase that can be sent over HTTP RPC (URL or body).\nfunc toMultibase(data []byte) string {\n\tmb, _ := mbase.Encode(mbase.Base64url, data)\n\treturn mb\n}\n" }, { "path": "client/rpc/request.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"strings\"\n)\n\ntype Request struct {\n\tCtx context.Context\n\tApiBase string\n\tCommand string\n\tArgs []string\n\tOpts map[string]string\n\tBody io.Reader\n\tHeaders map[string]string\n}\n\nfunc NewRequest(ctx context.Context, url, command string, args ...string) *Request {\n\tif !strings.HasPrefix(url, \"http\") {\n\t\turl = \"http://\" + url\n\t}\n\n\topts := map[string]string{\n\t\t\"encoding\": \"json\",\n\t\t\"stream-channels\": \"true\",\n\t}\n\treturn &Request{\n\t\tCtx: ctx,\n\t\tApiBase: url + \"/api/v0\",\n\t\tCommand: command,\n\t\tArgs: args,\n\t\tOpts: opts,\n\t\tHeaders: make(map[string]string),\n\t}\n}\n" }, { "path": "client/rpc/requestbuilder.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/blang/semver/v4\"\n\t\"github.com/ipfs/boxo/files\"\n)\n\ntype RequestBuilder interface {\n\tArguments(args ...string) RequestBuilder\n\tBodyString(body string) RequestBuilder\n\tBodyBytes(body []byte) RequestBuilder\n\tBody(body io.Reader) RequestBuilder\n\tFileBody(body io.Reader) RequestBuilder\n\tOption(key string, value any) RequestBuilder\n\tHeader(name, value string) RequestBuilder\n\tSend(ctx context.Context) (*Response, error)\n\tExec(ctx context.Context, res any) error\n}\n\n// encodedAbsolutePathVersion is the version from which the absolute path header in\n// multipart requests is %-encoded. Before this version, its sent raw.\nvar encodedAbsolutePathVersion = semver.MustParse(\"0.23.0-dev\")\n\n// requestBuilder is an IPFS commands request builder.\ntype requestBuilder struct {\n\tcommand string\n\targs []string\n\topts map[string]string\n\theaders map[string]string\n\tbody io.Reader\n\tbuildError error\n\n\tshell *HttpApi\n}\n\n// Arguments adds the arguments to the args.\nfunc (r *requestBuilder) Arguments(args ...string) RequestBuilder {\n\tr.args = append(r.args, args...)\n\treturn r\n}\n\n// BodyString sets the request body to the given string.\nfunc (r *requestBuilder) BodyString(body string) RequestBuilder {\n\treturn r.Body(strings.NewReader(body))\n}\n\n// BodyBytes sets the request body to the given buffer.\nfunc (r *requestBuilder) BodyBytes(body []byte) RequestBuilder {\n\treturn r.Body(bytes.NewReader(body))\n}\n\n// Body sets the request body to the given reader.\nfunc (r *requestBuilder) Body(body io.Reader) RequestBuilder {\n\tr.body = body\n\treturn r\n}\n\n// FileBody sets the request body to the given reader wrapped into multipartreader.\nfunc (r *requestBuilder) FileBody(body io.Reader) RequestBuilder {\n\tpr, _ := files.NewReaderPathFile(\"/dev/stdin\", io.NopCloser(body), nil)\n\td := files.NewMapDirectory(map[string]files.Node{\"\": pr})\n\n\tversion, err := r.shell.loadRemoteVersion()\n\tif err != nil {\n\t\t// Unfortunately, we cannot return an error here. Changing this API is also\n\t\t// not the best since we would otherwise have an inconsistent RequestBuilder.\n\t\t// We save the error and return it when calling [requestBuilder.Send].\n\t\tr.buildError = err\n\t\treturn r\n\t}\n\n\tuseEncodedAbsPaths := version.LT(encodedAbsolutePathVersion)\n\tr.body = files.NewMultiFileReader(d, false, useEncodedAbsPaths)\n\n\treturn r\n}\n\n// Option sets the given option.\nfunc (r *requestBuilder) Option(key string, value any) RequestBuilder {\n\tvar s string\n\tswitch v := value.(type) {\n\tcase bool:\n\t\ts = strconv.FormatBool(v)\n\tcase string:\n\t\ts = v\n\tcase []byte:\n\t\ts = string(v)\n\tdefault:\n\t\t// slow case.\n\t\ts = fmt.Sprint(value)\n\t}\n\tif r.opts == nil {\n\t\tr.opts = make(map[string]string, 1)\n\t}\n\tr.opts[key] = s\n\treturn r\n}\n\n// Header sets the given header.\nfunc (r *requestBuilder) Header(name, value string) RequestBuilder {\n\tif r.headers == nil {\n\t\tr.headers = make(map[string]string, 1)\n\t}\n\tr.headers[name] = value\n\treturn r\n}\n\n// Send sends the request and return the response.\nfunc (r *requestBuilder) Send(ctx context.Context) (*Response, error) {\n\tif r.buildError != nil {\n\t\treturn nil, r.buildError\n\t}\n\n\tr.shell.applyGlobal(r)\n\n\treq := NewRequest(ctx, r.shell.url, r.command, r.args...)\n\treq.Opts = r.opts\n\treq.Headers = r.headers\n\treq.Body = r.body\n\treturn req.Send(&r.shell.httpcli)\n}\n\n// Exec sends the request a request and decodes the response.\nfunc (r *requestBuilder) Exec(ctx context.Context, res any) error {\n\thttpRes, err := r.Send(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif res == nil {\n\t\tlateErr := httpRes.Close()\n\t\tif httpRes.Error != nil {\n\t\t\treturn httpRes.Error\n\t\t}\n\t\treturn lateErr\n\t}\n\n\treturn httpRes.decode(res)\n}\n\nvar _ RequestBuilder = &requestBuilder{}\n" }, { "path": "client/rpc/response.go", "content": "package rpc\n\nimport (\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"mime\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcmdhttp \"github.com/ipfs/go-ipfs-cmds/http\"\n)\n\ntype Error = cmds.Error\n\ntype trailerReader struct {\n\tresp *http.Response\n}\n\nfunc (r *trailerReader) Read(b []byte) (int, error) {\n\tn, err := r.resp.Body.Read(b)\n\tif err != nil {\n\t\tif e := r.resp.Trailer.Get(cmdhttp.StreamErrHeader); e != \"\" {\n\t\t\terr = errors.New(e)\n\t\t}\n\t}\n\treturn n, err\n}\n\nfunc (r *trailerReader) Close() error {\n\treturn r.resp.Body.Close()\n}\n\ntype Response struct {\n\tOutput io.ReadCloser\n\tError *Error\n}\n\nfunc (r *Response) Close() error {\n\tif r.Output != nil {\n\n\t\t// drain output (response body)\n\t\t_, err1 := io.Copy(io.Discard, r.Output)\n\t\terr2 := r.Output.Close()\n\t\tif err1 != nil {\n\t\t\treturn err1\n\t\t}\n\t\treturn err2\n\t}\n\treturn nil\n}\n\n// Cancel aborts running request (without draining request body).\nfunc (r *Response) Cancel() error {\n\tif r.Output != nil {\n\t\treturn r.Output.Close()\n\t}\n\n\treturn nil\n}\n\n// Decode reads request body and decodes it as json.\nfunc (r *Response) decode(dec any) error {\n\tif r.Error != nil {\n\t\treturn r.Error\n\t}\n\n\terr := json.NewDecoder(r.Output).Decode(dec)\n\terr2 := r.Close()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn err2\n}\n\nfunc (r *Request) Send(c *http.Client) (*Response, error) {\n\turl := r.getURL()\n\treq, err := http.NewRequest(\"POST\", url, r.Body)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treq = req.WithContext(r.Ctx)\n\n\t// Add any headers that were supplied via the requestBuilder.\n\tfor k, v := range r.Headers {\n\t\treq.Header.Add(k, v)\n\t}\n\n\tif fr, ok := r.Body.(*files.MultiFileReader); ok {\n\t\treq.Header.Set(\"Content-Type\", \"multipart/form-data; boundary=\"+fr.Boundary())\n\t\treq.Header.Set(\"Content-Disposition\", \"form-data; name=\\\"files\\\"\")\n\t}\n\n\tresp, err := c.Do(req)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcontentType, _, err := mime.ParseMediaType(resp.Header.Get(\"Content-Type\"))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnresp := new(Response)\n\n\tnresp.Output = &trailerReader{resp}\n\tif resp.StatusCode >= http.StatusBadRequest {\n\t\te := new(Error)\n\t\tswitch {\n\t\tcase resp.StatusCode == http.StatusNotFound:\n\t\t\te.Message = \"command not found\"\n\t\tcase contentType == \"text/plain\":\n\t\t\tout, err := io.ReadAll(resp.Body)\n\t\t\tif err != nil {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"ipfs-shell: warning! response (%d) read error: %s\\n\", resp.StatusCode, err)\n\t\t\t}\n\t\t\te.Message = string(out)\n\n\t\t\t// set special status codes.\n\t\t\tswitch resp.StatusCode {\n\t\t\tcase http.StatusNotFound, http.StatusBadRequest:\n\t\t\t\te.Code = cmds.ErrClient\n\t\t\tcase http.StatusTooManyRequests:\n\t\t\t\te.Code = cmds.ErrRateLimited\n\t\t\tcase http.StatusForbidden:\n\t\t\t\te.Code = cmds.ErrForbidden\n\t\t\t}\n\t\tcase contentType == \"application/json\":\n\t\t\tif err = json.NewDecoder(resp.Body).Decode(e); err != nil {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"ipfs-shell: warning! response (%d) unmarshall error: %s\\n\", resp.StatusCode, err)\n\t\t\t}\n\t\tdefault:\n\t\t\t// This is a server-side bug (probably).\n\t\t\te.Code = cmds.ErrImplementation\n\t\t\tfmt.Fprintf(os.Stderr, \"ipfs-shell: warning! unhandled response (%d) encoding: %s\", resp.StatusCode, contentType)\n\t\t\tout, err := io.ReadAll(resp.Body)\n\t\t\tif err != nil {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"ipfs-shell: response (%d) read error: %s\\n\", resp.StatusCode, err)\n\t\t\t}\n\t\t\te.Message = fmt.Sprintf(\"unknown ipfs-shell error encoding: %q - %q\", contentType, out)\n\t\t}\n\t\tnresp.Error = e\n\t\tnresp.Output = nil\n\n\t\t// drain body and close\n\t\t_, _ = io.Copy(io.Discard, resp.Body)\n\t\t_ = resp.Body.Close()\n\t}\n\n\treturn nresp, nil\n}\n\nfunc (r *Request) getURL() string {\n\tvalues := make(url.Values)\n\tfor _, arg := range r.Args {\n\t\tvalues.Add(\"arg\", arg)\n\t}\n\tfor k, v := range r.Opts {\n\t\tvalues.Add(k, v)\n\t}\n\n\treturn fmt.Sprintf(\"%s/%s?%s\", r.ApiBase, r.Command, values.Encode())\n}\n" }, { "path": "client/rpc/routing.go", "content": "package rpc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n)\n\ntype RoutingAPI HttpApi\n\nfunc (api *RoutingAPI) Get(ctx context.Context, key string) ([]byte, error) {\n\tresp, err := api.core().Request(\"routing/get\", key).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\tdefer resp.Close()\n\n\tvar out routing.QueryEvent\n\n\tdec := json.NewDecoder(resp.Output)\n\tif err := dec.Decode(&out); err != nil {\n\t\treturn nil, err\n\t}\n\n\tres, err := base64.StdEncoding.DecodeString(out.Extra)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn res, nil\n}\n\nfunc (api *RoutingAPI) Put(ctx context.Context, key string, value []byte, opts ...options.RoutingPutOption) error {\n\tvar cfg options.RoutingPutSettings\n\tfor _, o := range opts {\n\t\tif err := o(&cfg); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tresp, err := api.core().Request(\"routing/put\", key).\n\t\tOption(\"allow-offline\", cfg.AllowOffline).\n\t\tFileBody(bytes.NewReader(value)).\n\t\tSend(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif resp.Error != nil {\n\t\treturn resp.Error\n\t}\n\treturn nil\n}\n\nfunc (api *RoutingAPI) FindPeer(ctx context.Context, p peer.ID) (peer.AddrInfo, error) {\n\tvar out struct {\n\t\tType routing.QueryEventType\n\t\tResponses []peer.AddrInfo\n\t}\n\tresp, err := api.core().Request(\"routing/findpeer\", p.String()).Send(ctx)\n\tif err != nil {\n\t\treturn peer.AddrInfo{}, err\n\t}\n\tif resp.Error != nil {\n\t\treturn peer.AddrInfo{}, resp.Error\n\t}\n\tdefer resp.Close()\n\tdec := json.NewDecoder(resp.Output)\n\tfor {\n\t\tif err := dec.Decode(&out); err != nil {\n\t\t\treturn peer.AddrInfo{}, err\n\t\t}\n\t\tif out.Type == routing.FinalPeer {\n\t\t\treturn out.Responses[0], nil\n\t\t}\n\t}\n}\n\nfunc (api *RoutingAPI) FindProviders(ctx context.Context, p path.Path, opts ...options.RoutingFindProvidersOption) (<-chan peer.AddrInfo, error) {\n\toptions, err := options.RoutingFindProvidersOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresp, err := api.core().Request(\"routing/findprovs\", rp.RootCid().String()).\n\t\tOption(\"num-providers\", options.NumProviders).\n\t\tSend(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\tres := make(chan peer.AddrInfo)\n\n\tgo func() {\n\t\tdefer resp.Close()\n\t\tdefer close(res)\n\t\tdec := json.NewDecoder(resp.Output)\n\n\t\tfor {\n\t\t\tvar out struct {\n\t\t\t\tExtra string\n\t\t\t\tType routing.QueryEventType\n\t\t\t\tResponses []peer.AddrInfo\n\t\t\t}\n\n\t\t\tif err := dec.Decode(&out); err != nil {\n\t\t\t\treturn // todo: handle this somehow\n\t\t\t}\n\t\t\tif out.Type == routing.QueryError {\n\t\t\t\treturn // usually a 'not found' error\n\t\t\t\t// todo: handle other errors\n\t\t\t}\n\t\t\tif out.Type == routing.Provider {\n\t\t\t\tfor _, pi := range out.Responses {\n\t\t\t\t\tselect {\n\t\t\t\t\tcase res <- pi:\n\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn res, nil\n}\n\nfunc (api *RoutingAPI) Provide(ctx context.Context, p path.Path, opts ...options.RoutingProvideOption) error {\n\toptions, err := options.RoutingProvideOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn api.core().Request(\"routing/provide\", rp.RootCid().String()).\n\t\tOption(\"recursive\", options.Recursive).\n\t\tExec(ctx, nil)\n}\n\nfunc (api *RoutingAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/swarm.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\t\"github.com/multiformats/go-multiaddr\"\n)\n\ntype SwarmAPI HttpApi\n\nfunc (api *SwarmAPI) Connect(ctx context.Context, pi peer.AddrInfo) error {\n\tpidma, err := multiaddr.NewComponent(\"p2p\", pi.ID.String())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tsaddrs := make([]string, len(pi.Addrs))\n\tfor i, addr := range pi.Addrs {\n\t\tsaddrs[i] = addr.Encapsulate(pidma).String()\n\t}\n\n\treturn api.core().Request(\"swarm/connect\", saddrs...).Exec(ctx, nil)\n}\n\nfunc (api *SwarmAPI) Disconnect(ctx context.Context, addr multiaddr.Multiaddr) error {\n\treturn api.core().Request(\"swarm/disconnect\", addr.String()).Exec(ctx, nil)\n}\n\ntype connInfo struct {\n\taddr multiaddr.Multiaddr\n\tpeer peer.ID\n\tlatency time.Duration\n\tmuxer string\n\tdirection network.Direction\n\tstreams []protocol.ID\n}\n\nfunc (c *connInfo) ID() peer.ID {\n\treturn c.peer\n}\n\nfunc (c *connInfo) Address() multiaddr.Multiaddr {\n\treturn c.addr\n}\n\nfunc (c *connInfo) Direction() network.Direction {\n\treturn c.direction\n}\n\nfunc (c *connInfo) Latency() (time.Duration, error) {\n\treturn c.latency, nil\n}\n\nfunc (c *connInfo) Streams() ([]protocol.ID, error) {\n\treturn c.streams, nil\n}\n\nfunc (api *SwarmAPI) Peers(ctx context.Context) ([]iface.ConnectionInfo, error) {\n\tvar resp struct {\n\t\tPeers []struct {\n\t\t\tAddr string\n\t\t\tPeer string\n\t\t\tLatency string\n\t\t\tMuxer string\n\t\t\tDirection network.Direction\n\t\t\tStreams []struct {\n\t\t\t\tProtocol string\n\t\t\t}\n\t\t}\n\t}\n\n\terr := api.core().Request(\"swarm/peers\").\n\t\tOption(\"streams\", true).\n\t\tOption(\"latency\", true).\n\t\tExec(ctx, &resp)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tres := make([]iface.ConnectionInfo, len(resp.Peers))\n\tfor i, conn := range resp.Peers {\n\t\tlatency, _ := time.ParseDuration(conn.Latency)\n\t\tout := &connInfo{\n\t\t\tlatency: latency,\n\t\t\tmuxer: conn.Muxer,\n\t\t\tdirection: conn.Direction,\n\t\t}\n\n\t\tout.peer, err = peer.Decode(conn.Peer)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tout.addr, err = multiaddr.NewMultiaddr(conn.Addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tout.streams = make([]protocol.ID, len(conn.Streams))\n\t\tfor i, p := range conn.Streams {\n\t\t\tout.streams[i] = protocol.ID(p.Protocol)\n\t\t}\n\n\t\tres[i] = out\n\t}\n\n\treturn res, nil\n}\n\nfunc (api *SwarmAPI) KnownAddrs(ctx context.Context) (map[peer.ID][]multiaddr.Multiaddr, error) {\n\tvar out struct {\n\t\tAddrs map[string][]string\n\t}\n\tif err := api.core().Request(\"swarm/addrs\").Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\tres := map[peer.ID][]multiaddr.Multiaddr{}\n\tfor spid, saddrs := range out.Addrs {\n\t\taddrs := make([]multiaddr.Multiaddr, len(saddrs))\n\n\t\tfor i, addr := range saddrs {\n\t\t\ta, err := multiaddr.NewMultiaddr(addr)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\taddrs[i] = a\n\t\t}\n\n\t\tpid, err := peer.Decode(spid)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tres[pid] = addrs\n\t}\n\n\treturn res, nil\n}\n\nfunc (api *SwarmAPI) LocalAddrs(ctx context.Context) ([]multiaddr.Multiaddr, error) {\n\tvar out struct {\n\t\tStrings []string\n\t}\n\n\tif err := api.core().Request(\"swarm/addrs/local\").Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\n\tres := make([]multiaddr.Multiaddr, len(out.Strings))\n\tfor i, addr := range out.Strings {\n\t\tma, err := multiaddr.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres[i] = ma\n\t}\n\treturn res, nil\n}\n\nfunc (api *SwarmAPI) ListenAddrs(ctx context.Context) ([]multiaddr.Multiaddr, error) {\n\tvar out struct {\n\t\tStrings []string\n\t}\n\n\tif err := api.core().Request(\"swarm/addrs/listen\").Exec(ctx, &out); err != nil {\n\t\treturn nil, err\n\t}\n\n\tres := make([]multiaddr.Multiaddr, len(out.Strings))\n\tfor i, addr := range out.Strings {\n\t\tma, err := multiaddr.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres[i] = ma\n\t}\n\treturn res, nil\n}\n\nfunc (api *SwarmAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "client/rpc/unixfs.go", "content": "package rpc\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tunixfs \"github.com/ipfs/boxo/ipld/unixfs\"\n\tunixfs_pb \"github.com/ipfs/boxo/ipld/unixfs/pb\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\ntype addEvent struct {\n\tName string\n\tHash string `json:\",omitempty\"`\n\tBytes int64 `json:\",omitempty\"`\n\tSize string `json:\",omitempty\"`\n}\n\ntype UnixfsAPI HttpApi\n\nfunc (api *UnixfsAPI) Add(ctx context.Context, f files.Node, opts ...caopts.UnixfsAddOption) (path.ImmutablePath, error) {\n\toptions, _, err := caopts.UnixfsAddOptions(opts...)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tmht, ok := mh.Codes[options.MhType]\n\tif !ok {\n\t\treturn path.ImmutablePath{}, fmt.Errorf(\"unknowm mhType %d\", options.MhType)\n\t}\n\n\treq := api.core().Request(\"add\").\n\t\tOption(\"hash\", mht).\n\t\tOption(\"chunker\", options.Chunker).\n\t\tOption(\"cid-version\", options.CidVersion).\n\t\tOption(\"fscache\", options.FsCache).\n\t\tOption(\"inline\", options.Inline).\n\t\tOption(\"inline-limit\", options.InlineLimit).\n\t\tOption(\"nocopy\", options.NoCopy).\n\t\tOption(\"only-hash\", options.OnlyHash).\n\t\tOption(\"pin\", options.Pin).\n\t\tOption(\"silent\", options.Silent).\n\t\tOption(\"progress\", options.Progress)\n\n\tif options.RawLeavesSet {\n\t\treq.Option(\"raw-leaves\", options.RawLeaves)\n\t}\n\n\tswitch options.Layout {\n\tcase caopts.BalancedLayout:\n\t\t// noop, default\n\tcase caopts.TrickleLayout:\n\t\treq.Option(\"trickle\", true)\n\t}\n\n\td := files.NewMapDirectory(map[string]files.Node{\"\": f}) // unwrapped on the other side\n\n\tversion, err := api.core().loadRemoteVersion()\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\tuseEncodedAbsPaths := version.LT(encodedAbsolutePathVersion)\n\treq.Body(files.NewMultiFileReader(d, false, useEncodedAbsPaths))\n\n\tvar out addEvent\n\tresp, err := req.Send(ctx)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\tif resp.Error != nil {\n\t\treturn path.ImmutablePath{}, resp.Error\n\t}\n\tdefer resp.Output.Close()\n\tdec := json.NewDecoder(resp.Output)\n\n\tfor {\n\t\tvar evt addEvent\n\t\tif err := dec.Decode(&evt); err != nil {\n\t\t\tif errors.Is(err, io.EOF) {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn path.ImmutablePath{}, err\n\t\t}\n\t\tout = evt\n\n\t\tif options.Events != nil {\n\t\t\tifevt := &iface.AddEvent{\n\t\t\t\tName: out.Name,\n\t\t\t\tSize: out.Size,\n\t\t\t\tBytes: out.Bytes,\n\t\t\t}\n\n\t\t\tif out.Hash != \"\" {\n\t\t\t\tc, err := cid.Parse(out.Hash)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn path.ImmutablePath{}, err\n\t\t\t\t}\n\n\t\t\t\tifevt.Path = path.FromCid(c)\n\t\t\t}\n\n\t\t\tselect {\n\t\t\tcase options.Events <- ifevt:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn path.ImmutablePath{}, ctx.Err()\n\t\t\t}\n\t\t}\n\t}\n\n\tc, err := cid.Parse(out.Hash)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn path.FromCid(c), nil\n}\n\ntype lsLink struct {\n\tName, Hash string\n\tSize uint64\n\tType unixfs_pb.Data_DataType\n\tTarget string\n\n\tMode os.FileMode\n\tModTime time.Time\n}\n\ntype lsObject struct {\n\tHash string\n\tLinks []lsLink\n}\n\ntype lsOutput struct {\n\tObjects []lsObject\n}\n\nfunc (api *UnixfsAPI) Ls(ctx context.Context, p path.Path, out chan<- iface.DirEntry, opts ...caopts.UnixfsLsOption) error {\n\tdefer close(out)\n\n\toptions, err := caopts.UnixfsLsOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tresp, err := api.core().Request(\"ls\", p.String()).\n\t\tOption(\"resolve-type\", options.ResolveChildren).\n\t\tOption(\"size\", options.ResolveChildren).\n\t\tOption(\"stream\", true).\n\t\tSend(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif resp.Error != nil {\n\t\treturn err\n\t}\n\tdefer resp.Close()\n\n\tdec := json.NewDecoder(resp.Output)\n\n\tfor {\n\t\tvar link lsOutput\n\t\tif err = dec.Decode(&link); err != nil {\n\t\t\tif err != io.EOF {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tif len(link.Objects) != 1 {\n\t\t\treturn errors.New(\"unexpected Objects len\")\n\t\t}\n\n\t\tif len(link.Objects[0].Links) != 1 {\n\t\t\treturn errors.New(\"unexpected Links len\")\n\t\t}\n\n\t\tl0 := link.Objects[0].Links[0]\n\n\t\tc, err := cid.Decode(l0.Hash)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar ftype iface.FileType\n\t\tswitch l0.Type {\n\t\tcase unixfs.TRaw, unixfs.TFile:\n\t\t\tftype = iface.TFile\n\t\tcase unixfs.THAMTShard, unixfs.TDirectory, unixfs.TMetadata:\n\t\t\tftype = iface.TDirectory\n\t\tcase unixfs.TSymlink:\n\t\t\tftype = iface.TSymlink\n\t\t}\n\n\t\tselect {\n\t\tcase out <- iface.DirEntry{\n\t\t\tName: l0.Name,\n\t\t\tCid: c,\n\t\t\tSize: l0.Size,\n\t\t\tType: ftype,\n\t\t\tTarget: l0.Target,\n\n\t\t\tMode: l0.Mode,\n\t\t\tModTime: l0.ModTime,\n\t\t}:\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\t}\n\t}\n}\n\nfunc (api *UnixfsAPI) core() *HttpApi {\n\treturn (*HttpApi)(api)\n}\n" }, { "path": "codecov.yml", "content": "codecov:\n ci:\n - \"!travis-ci.org\"\n - \"!ci.ipfs.team:8111\"\n - \"!ci.ipfs.team\"\n - \"github.com\"\n notify:\n require_ci_to_pass: no\n after_n_builds: 2\n\ncoverage:\n range: \"50...100\"\n\n status:\n project:\n default:\n threshold: 0.2%\n\n patch:\n default:\n threshold: 2%\n\ncomment: off\n" }, { "path": "commands/context.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"strings\"\n\t\"time\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tcoreapi \"github.com/ipfs/kubo/core/coreapi\"\n\tloader \"github.com/ipfs/kubo/plugin/loader\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nvar log = logging.Logger(\"command\")\n\n// Context represents request context.\ntype Context struct {\n\tConfigRoot string\n\tReqLog *ReqLog\n\n\tPlugins *loader.PluginLoader\n\n\tGateway bool\n\tapi coreiface.CoreAPI\n\tnode *core.IpfsNode\n\tConstructNode func() (*core.IpfsNode, error)\n}\n\nfunc (c *Context) GetConfig() (*config.Config, error) {\n\tnode, err := c.GetNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn node.Repo.Config()\n}\n\n// GetNode returns the node of the current Command execution\n// context. It may construct it with the provided function.\nfunc (c *Context) GetNode() (*core.IpfsNode, error) {\n\tvar err error\n\tif c.node == nil {\n\t\tif c.ConstructNode == nil {\n\t\t\treturn nil, errors.New(\"nil ConstructNode function\")\n\t\t}\n\t\tc.node, err = c.ConstructNode()\n\t}\n\treturn c.node, err\n}\n\n// ClearCachedNode clears any cached node, forcing GetNode to construct a new one.\n//\n// This method is critical for mitigating racy FX dependency injection behavior\n// that can occur during daemon startup. The daemon may create multiple IpfsNode\n// instances during initialization - first an offline node during early init, then\n// the proper online daemon node. Without clearing the cache, HTTP RPC handlers may\n// end up using the first (offline) cached node instead of the intended online daemon node.\n//\n// This behavior was likely present forever in go-ipfs, but recent changes made it more\n// prominent and forced us to proactively mitigate FX shortcomings. The daemon calls\n// this method immediately before setting its ConstructNode function to ensure that\n// subsequent GetNode() calls use the correct online daemon node rather than any\n// stale cached offline node from initialization.\nfunc (c *Context) ClearCachedNode() {\n\tc.node = nil\n}\n\n// GetAPI returns CoreAPI instance backed by ipfs node.\n// It may construct the node with the provided function.\nfunc (c *Context) GetAPI() (coreiface.CoreAPI, error) {\n\tif c.api == nil {\n\t\tn, err := c.GetNode()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tfetchBlocks := true\n\t\tif c.Gateway {\n\t\t\tcfg, err := c.GetConfig()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tfetchBlocks = !cfg.Gateway.NoFetch\n\t\t}\n\n\t\tc.api, err = coreapi.NewCoreAPI(n, options.Api.FetchBlocks(fetchBlocks))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn c.api, nil\n}\n\n// Context returns the node's context.\nfunc (c *Context) Context() context.Context {\n\tn, err := c.GetNode()\n\tif err != nil {\n\t\tlog.Debug(\"error getting node: \", err)\n\t\treturn context.Background()\n\t}\n\n\treturn n.Context()\n}\n\n// LogRequest adds the passed request to the request log and\n// returns a function that should be called when the request\n// lifetime is over.\nfunc (c *Context) LogRequest(req *cmds.Request) func() {\n\trle := &ReqLogEntry{\n\t\tStartTime: time.Now(),\n\t\tActive: true,\n\t\tCommand: strings.Join(req.Path, \"/\"),\n\t\tOptions: req.Options,\n\t\tArgs: req.Arguments,\n\t\tlog: c.ReqLog,\n\t}\n\tc.ReqLog.AddEntry(rle)\n\n\treturn func() {\n\t\tc.ReqLog.Finish(rle)\n\t}\n}\n\n// Close cleans up the application state.\nfunc (c *Context) Close() {\n\t// let's not forget teardown. If a node was initialized, we must close it.\n\t// Note that this means the underlying req.Context().Node variable is exposed.\n\t// this is gross, and should be changed when we extract out the exec Context.\n\tif c.node != nil {\n\t\tlog.Info(\"Shutting down node...\")\n\t\tc.node.Close()\n\t}\n}\n" }, { "path": "commands/reqlog.go", "content": "package commands\n\nimport (\n\t\"sync\"\n\t\"time\"\n)\n\n// ReqLogEntry is an entry in the request log.\ntype ReqLogEntry struct {\n\tStartTime time.Time\n\tEndTime time.Time\n\tActive bool\n\tCommand string\n\tOptions map[string]any\n\tArgs []string\n\tID int\n\n\tlog *ReqLog\n}\n\n// Copy returns a copy of the ReqLogEntry.\nfunc (r *ReqLogEntry) Copy() *ReqLogEntry {\n\tout := *r\n\tout.log = nil\n\treturn &out\n}\n\n// ReqLog is a log of requests.\ntype ReqLog struct {\n\tRequests []*ReqLogEntry\n\tnextID int\n\tlock sync.Mutex\n\tkeep time.Duration\n}\n\n// AddEntry adds an entry to the log.\nfunc (rl *ReqLog) AddEntry(rle *ReqLogEntry) {\n\trl.lock.Lock()\n\tdefer rl.lock.Unlock()\n\n\trle.ID = rl.nextID\n\trl.nextID++\n\trl.Requests = append(rl.Requests, rle)\n\n\tif !rle.Active {\n\t\trl.maybeCleanup()\n\t}\n}\n\n// ClearInactive removes stale entries.\nfunc (rl *ReqLog) ClearInactive() {\n\trl.lock.Lock()\n\tdefer rl.lock.Unlock()\n\n\tk := rl.keep\n\trl.keep = 0\n\trl.cleanup()\n\trl.keep = k\n}\n\nfunc (rl *ReqLog) maybeCleanup() {\n\t// only do it every so often or it might\n\t// become a perf issue\n\tif len(rl.Requests)%10 == 0 {\n\t\trl.cleanup()\n\t}\n}\n\nfunc (rl *ReqLog) cleanup() {\n\ti := 0\n\tnow := time.Now()\n\tfor j := 0; j < len(rl.Requests); j++ {\n\t\trj := rl.Requests[j]\n\t\tif rj.Active || rl.Requests[j].EndTime.Add(rl.keep).After(now) {\n\t\t\trl.Requests[i] = rl.Requests[j]\n\t\t\ti++\n\t\t}\n\t}\n\trl.Requests = rl.Requests[:i]\n}\n\n// SetKeepTime sets a duration after which an entry will be considered inactive.\nfunc (rl *ReqLog) SetKeepTime(t time.Duration) {\n\trl.lock.Lock()\n\tdefer rl.lock.Unlock()\n\trl.keep = t\n}\n\n// Report generates a copy of all the entries in the requestlog.\nfunc (rl *ReqLog) Report() []*ReqLogEntry {\n\trl.lock.Lock()\n\tdefer rl.lock.Unlock()\n\tout := make([]*ReqLogEntry, len(rl.Requests))\n\n\tfor i, e := range rl.Requests {\n\t\tout[i] = e.Copy()\n\t}\n\n\treturn out\n}\n\n// Finish marks an entry in the log as finished.\nfunc (rl *ReqLog) Finish(rle *ReqLogEntry) {\n\trl.lock.Lock()\n\tdefer rl.lock.Unlock()\n\n\trle.Active = false\n\trle.EndTime = time.Now()\n\n\trl.maybeCleanup()\n}\n" }, { "path": "config/addresses.go", "content": "package config\n\n// Addresses stores the (string) multiaddr addresses for the node.\ntype Addresses struct {\n\tSwarm []string // addresses for the swarm to listen on\n\tAnnounce []string // swarm addresses to announce to the network, if len > 0 replaces auto detected addresses\n\tAppendAnnounce []string // similar to Announce but doesn't overwrite auto detected addresses, they are just appended\n\tNoAnnounce []string // swarm addresses not to announce to the network\n\tAPI Strings // address for the local API (RPC)\n\tGateway Strings // address to listen on for IPFS HTTP object gateway\n}\n" }, { "path": "config/api.go", "content": "package config\n\nimport (\n\t\"encoding/base64\"\n\t\"strings\"\n)\n\nconst (\n\tAPITag = \"API\"\n\tAuthorizationTag = \"Authorizations\"\n)\n\ntype RPCAuthScope struct {\n\t// AuthSecret is the secret that will be compared to the HTTP \"Authorization\".\n\t// header. A secret is in the format \"type:value\". Check the documentation for\n\t// supported types.\n\tAuthSecret string\n\n\t// AllowedPaths is an explicit list of RPC path prefixes to allow.\n\t// By default, none are allowed. [\"/api/v0\"] exposes all RPCs.\n\tAllowedPaths []string\n}\n\ntype API struct {\n\t// HTTPHeaders are the HTTP headers to return with the API.\n\tHTTPHeaders map[string][]string\n\n\t// Authorization is a map of authorizations used to authenticate in the API.\n\t// If the map is empty, then the RPC API is exposed to everyone. Check the\n\t// documentation for more details.\n\tAuthorizations map[string]*RPCAuthScope `json:\",omitempty\"`\n}\n\n// ConvertAuthSecret converts the given secret in the format \"type:value\" into an\n// HTTP Authorization header value. It can handle 'bearer' and 'basic' as type.\n// If type exists and is not known, an empty string is returned. If type does not\n// exist, 'bearer' type is assumed.\nfunc ConvertAuthSecret(secret string) string {\n\tif secret == \"\" {\n\t\treturn secret\n\t}\n\n\tsplit := strings.SplitN(secret, \":\", 2)\n\tif len(split) < 2 {\n\t\t// No prefix: assume bearer token.\n\t\treturn \"Bearer \" + secret\n\t}\n\n\tif strings.HasPrefix(secret, \"basic:\") {\n\t\tif strings.Contains(split[1], \":\") {\n\t\t\t// Assume basic:user:password\n\t\t\treturn \"Basic \" + base64.StdEncoding.EncodeToString([]byte(split[1]))\n\t\t} else {\n\t\t\t// Assume already base64 encoded.\n\t\t\treturn \"Basic \" + split[1]\n\t\t}\n\t} else if strings.HasPrefix(secret, \"bearer:\") {\n\t\treturn \"Bearer \" + split[1]\n\t}\n\n\t// Unknown. Type is present, but we can't handle it.\n\treturn \"\"\n}\n" }, { "path": "config/api_test.go", "content": "package config\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestConvertAuthSecret(t *testing.T) {\n\tfor _, testCase := range []struct {\n\t\tinput string\n\t\toutput string\n\t}{\n\t\t{\"\", \"\"},\n\t\t{\"someToken\", \"Bearer someToken\"},\n\t\t{\"bearer:someToken\", \"Bearer someToken\"},\n\t\t{\"basic:user:pass\", \"Basic dXNlcjpwYXNz\"},\n\t\t{\"basic:dXNlcjpwYXNz\", \"Basic dXNlcjpwYXNz\"},\n\t} {\n\t\tassert.Equal(t, testCase.output, ConvertAuthSecret(testCase.input))\n\t}\n}\n" }, { "path": "config/autoconf.go", "content": "package config\n\nimport (\n\t\"maps\"\n\t\"math/rand/v2\"\n\t\"strings\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nvar log = logging.Logger(\"config\")\n\n// AutoConf contains the configuration for the autoconf subsystem\ntype AutoConf struct {\n\t// URL is the HTTP(S) URL to fetch the autoconf.json from\n\t// Default: see boxo/autoconf.MainnetAutoConfURL\n\tURL *OptionalString `json:\",omitempty\"`\n\n\t// Enabled determines whether to use autoconf\n\t// Default: true\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// RefreshInterval is how often to refresh autoconf data\n\t// Default: 24h\n\tRefreshInterval *OptionalDuration `json:\",omitempty\"`\n\n\t// TLSInsecureSkipVerify allows skipping TLS verification (for testing only)\n\t// Default: false\n\tTLSInsecureSkipVerify Flag `json:\",omitempty\"`\n}\n\nconst (\n\t// AutoPlaceholder is the string used as a placeholder for autoconf values\n\tAutoPlaceholder = \"auto\"\n\n\t// DefaultAutoConfEnabled is the default value for AutoConf.Enabled\n\tDefaultAutoConfEnabled = true\n\n\t// DefaultAutoConfURL is the default URL for fetching autoconf\n\tDefaultAutoConfURL = autoconf.MainnetAutoConfURL\n\n\t// DefaultAutoConfRefreshInterval is the default interval for refreshing autoconf data\n\tDefaultAutoConfRefreshInterval = autoconf.DefaultRefreshInterval\n\n\t// AutoConf client configuration constants\n\tDefaultAutoConfCacheSize = autoconf.DefaultCacheSize\n\tDefaultAutoConfTimeout = autoconf.DefaultTimeout\n)\n\n// getNativeSystems returns the list of systems that should be used natively based on routing type\nfunc getNativeSystems(routingType string) []string {\n\tswitch routingType {\n\tcase \"dht\", \"dhtclient\", \"dhtserver\":\n\t\treturn []string{autoconf.SystemAminoDHT} // Only native DHT\n\tcase \"auto\", \"autoclient\":\n\t\treturn []string{autoconf.SystemAminoDHT} // Native DHT, delegated others\n\tcase \"delegated\":\n\t\treturn []string{} // Everything delegated\n\tcase \"none\":\n\t\treturn []string{} // No native systems\n\tdefault:\n\t\treturn []string{} // Custom mode\n\t}\n}\n\n// selectRandomResolver picks a random resolver from a list for load balancing\nfunc selectRandomResolver(resolvers []string) string {\n\tif len(resolvers) == 0 {\n\t\treturn \"\"\n\t}\n\treturn resolvers[rand.IntN(len(resolvers))]\n}\n\n// DNSResolversWithAutoConf returns DNS resolvers with \"auto\" values replaced by autoconf values\nfunc (c *Config) DNSResolversWithAutoConf() map[string]string {\n\tif c.DNS.Resolvers == nil {\n\t\treturn nil\n\t}\n\n\tresolved := make(map[string]string)\n\tautoConf := c.getAutoConf()\n\tautoExpanded := 0\n\n\t// Process each configured resolver\n\tfor domain, resolver := range c.DNS.Resolvers {\n\t\tif resolver == AutoPlaceholder {\n\t\t\t// Try to resolve from autoconf\n\t\t\tif autoConf != nil && autoConf.DNSResolvers != nil {\n\t\t\t\tif resolvers, exists := autoConf.DNSResolvers[domain]; exists && len(resolvers) > 0 {\n\t\t\t\t\tresolved[domain] = selectRandomResolver(resolvers)\n\t\t\t\t\tautoExpanded++\n\t\t\t\t}\n\t\t\t}\n\t\t\t// If autoConf is disabled or domain not found, skip this \"auto\" resolver\n\t\t} else {\n\t\t\t// Keep custom resolver as-is\n\t\t\tresolved[domain] = resolver\n\t\t}\n\t}\n\n\t// Add default resolvers from autoconf that aren't already configured\n\tif autoConf != nil && autoConf.DNSResolvers != nil {\n\t\tfor domain, resolvers := range autoConf.DNSResolvers {\n\t\t\tif _, exists := resolved[domain]; !exists && len(resolvers) > 0 {\n\t\t\t\tresolved[domain] = selectRandomResolver(resolvers)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Log expansion statistics\n\tif autoExpanded > 0 {\n\t\tlog.Debugf(\"expanded %d 'auto' DNS.Resolvers from autoconf\", autoExpanded)\n\t}\n\n\treturn resolved\n}\n\n// expandAutoConfSlice is a generic helper for expanding \"auto\" placeholders in string slices\n// It handles the common pattern of: iterate through slice, expand \"auto\" once, keep custom values\nfunc expandAutoConfSlice(sourceSlice []string, autoConfData []string) []string {\n\tvar resolved []string\n\tautoExpanded := false\n\n\tfor _, item := range sourceSlice {\n\t\tif item == AutoPlaceholder {\n\t\t\t// Replace with autoconf data (only once)\n\t\t\tif autoConfData != nil && !autoExpanded {\n\t\t\t\tresolved = append(resolved, autoConfData...)\n\t\t\t\tautoExpanded = true\n\t\t\t}\n\t\t\t// If autoConfData is nil or already expanded, skip redundant \"auto\" entries silently\n\t\t} else {\n\t\t\t// Keep custom item\n\t\t\tresolved = append(resolved, item)\n\t\t}\n\t}\n\n\treturn resolved\n}\n\n// BootstrapWithAutoConf returns bootstrap config with \"auto\" values replaced by autoconf values\nfunc (c *Config) BootstrapWithAutoConf() []string {\n\tautoConf := c.getAutoConf()\n\tvar autoConfData []string\n\n\tif autoConf != nil {\n\t\troutingType := c.Routing.Type.WithDefault(DefaultRoutingType)\n\t\tnativeSystems := getNativeSystems(routingType)\n\t\tautoConfData = autoConf.GetBootstrapPeers(nativeSystems...)\n\t\tlog.Debugf(\"BootstrapWithAutoConf: processing with routing type: %s\", routingType)\n\t} else {\n\t\tlog.Debugf(\"BootstrapWithAutoConf: autoConf disabled, using original config\")\n\t}\n\n\tresult := expandAutoConfSlice(c.Bootstrap, autoConfData)\n\tlog.Debugf(\"BootstrapWithAutoConf: final result contains %d peers\", len(result))\n\treturn result\n}\n\n// getAutoConf is a helper to get autoconf data with fallbacks\nfunc (c *Config) getAutoConf() *autoconf.Config {\n\tif !c.AutoConf.Enabled.WithDefault(DefaultAutoConfEnabled) {\n\t\tlog.Debugf(\"getAutoConf: AutoConf disabled, returning nil\")\n\t\treturn nil\n\t}\n\n\t// Create or get cached client with config\n\tclient, err := GetAutoConfClient(c)\n\tif err != nil {\n\t\tlog.Debugf(\"getAutoConf: client creation failed - %v\", err)\n\t\treturn nil\n\t}\n\n\t// Use GetCached to avoid network I/O during config operations\n\t// This ensures config retrieval doesn't block on network operations\n\tresult := client.GetCached()\n\n\tlog.Debugf(\"getAutoConf: returning autoconf data\")\n\treturn result\n}\n\n// BootstrapPeersWithAutoConf returns bootstrap peers with \"auto\" values replaced by autoconf values\n// and parsed into peer.AddrInfo structures\nfunc (c *Config) BootstrapPeersWithAutoConf() ([]peer.AddrInfo, error) {\n\tbootstrapStrings := c.BootstrapWithAutoConf()\n\treturn ParseBootstrapPeers(bootstrapStrings)\n}\n\n// DelegatedRoutersWithAutoConf returns delegated router URLs without trailing slashes\nfunc (c *Config) DelegatedRoutersWithAutoConf() []string {\n\tautoConf := c.getAutoConf()\n\n\t// Use autoconf to expand the endpoints with supported paths for read operations\n\troutingType := c.Routing.Type.WithDefault(DefaultRoutingType)\n\tnativeSystems := getNativeSystems(routingType)\n\treturn autoconf.ExpandDelegatedEndpoints(\n\t\tc.Routing.DelegatedRouters,\n\t\tautoConf,\n\t\tnativeSystems,\n\t\t// Kubo supports all read paths\n\t\tautoconf.RoutingV1ProvidersPath,\n\t\tautoconf.RoutingV1PeersPath,\n\t\tautoconf.RoutingV1IPNSPath,\n\t)\n}\n\n// DelegatedPublishersWithAutoConf returns delegated publisher URLs without trailing slashes\nfunc (c *Config) DelegatedPublishersWithAutoConf() []string {\n\tautoConf := c.getAutoConf()\n\n\t// Use autoconf to expand the endpoints with IPNS write path\n\troutingType := c.Routing.Type.WithDefault(DefaultRoutingType)\n\tnativeSystems := getNativeSystems(routingType)\n\treturn autoconf.ExpandDelegatedEndpoints(\n\t\tc.Ipns.DelegatedPublishers,\n\t\tautoConf,\n\t\tnativeSystems,\n\t\tautoconf.RoutingV1IPNSPath, // Only IPNS operations (for write)\n\t)\n}\n\n// expandConfigField expands a specific config field with autoconf values\n// Handles both top-level fields (\"Bootstrap\") and nested fields (\"DNS.Resolvers\")\nfunc (c *Config) expandConfigField(expandedCfg map[string]any, fieldPath string) {\n\t// Check if this field supports autoconf expansion\n\texpandFunc, supported := supportedAutoConfFields[fieldPath]\n\tif !supported {\n\t\treturn\n\t}\n\n\t// Handle top-level fields (no dot in path)\n\tif !strings.Contains(fieldPath, \".\") {\n\t\tif _, exists := expandedCfg[fieldPath]; exists {\n\t\t\texpandedCfg[fieldPath] = expandFunc(c)\n\t\t}\n\t\treturn\n\t}\n\n\t// Handle nested fields (section.field format)\n\tparts := strings.SplitN(fieldPath, \".\", 2)\n\tif len(parts) != 2 {\n\t\treturn\n\t}\n\n\tsectionName, fieldName := parts[0], parts[1]\n\tif section, exists := expandedCfg[sectionName]; exists {\n\t\tif sectionMap, ok := section.(map[string]any); ok {\n\t\t\tif _, exists := sectionMap[fieldName]; exists {\n\t\t\t\tsectionMap[fieldName] = expandFunc(c)\n\t\t\t\texpandedCfg[sectionName] = sectionMap\n\t\t\t}\n\t\t}\n\t}\n}\n\n// ExpandAutoConfValues expands \"auto\" placeholders in config with their actual values using the same methods as the daemon\nfunc (c *Config) ExpandAutoConfValues(cfg map[string]any) (map[string]any, error) {\n\t// Create a deep copy of the config map to avoid modifying the original\n\texpandedCfg := maps.Clone(cfg)\n\n\t// Use the same expansion methods that the daemon uses - ensures runtime consistency\n\t// Unified expansion for all supported autoconf fields\n\tc.expandConfigField(expandedCfg, \"Bootstrap\")\n\tc.expandConfigField(expandedCfg, \"DNS.Resolvers\")\n\tc.expandConfigField(expandedCfg, \"Routing.DelegatedRouters\")\n\tc.expandConfigField(expandedCfg, \"Ipns.DelegatedPublishers\")\n\n\treturn expandedCfg, nil\n}\n\n// supportedAutoConfFields maps field keys to their expansion functions\nvar supportedAutoConfFields = map[string]func(*Config) any{\n\t\"Bootstrap\": func(c *Config) any {\n\t\texpanded := c.BootstrapWithAutoConf()\n\t\treturn stringSliceToInterfaceSlice(expanded)\n\t},\n\t\"DNS.Resolvers\": func(c *Config) any {\n\t\texpanded := c.DNSResolversWithAutoConf()\n\t\treturn stringMapToInterfaceMap(expanded)\n\t},\n\t\"Routing.DelegatedRouters\": func(c *Config) any {\n\t\texpanded := c.DelegatedRoutersWithAutoConf()\n\t\treturn stringSliceToInterfaceSlice(expanded)\n\t},\n\t\"Ipns.DelegatedPublishers\": func(c *Config) any {\n\t\texpanded := c.DelegatedPublishersWithAutoConf()\n\t\treturn stringSliceToInterfaceSlice(expanded)\n\t},\n}\n\n// ExpandConfigField expands auto values for a specific config field using the same methods as the daemon\nfunc (c *Config) ExpandConfigField(key string, value any) any {\n\tif expandFunc, supported := supportedAutoConfFields[key]; supported {\n\t\treturn expandFunc(c)\n\t}\n\n\t// Return original value if no expansion needed (not a field that supports auto values)\n\treturn value\n}\n\n// Helper functions for type conversion between string types and any types for JSON compatibility\n\nfunc stringSliceToInterfaceSlice(slice []string) []any {\n\tresult := make([]any, len(slice))\n\tfor i, v := range slice {\n\t\tresult[i] = v\n\t}\n\treturn result\n}\n\nfunc stringMapToInterfaceMap(m map[string]string) map[string]any {\n\tresult := make(map[string]any)\n\tfor k, v := range m {\n\t\tresult[k] = v\n\t}\n\treturn result\n}\n" }, { "path": "config/autoconf_client.go", "content": "package config\n\nimport (\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"slices\"\n\t\"sync\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tversion \"github.com/ipfs/kubo\"\n)\n\nvar autoconfLog = logging.Logger(\"autoconf\")\n\n// Singleton state for autoconf client\nvar (\n\tclientOnce sync.Once\n\tclientCache *autoconf.Client\n\tclientErr error\n)\n\n// GetAutoConfClient returns a cached autoconf client or creates a new one.\n// This is thread-safe and uses a singleton pattern.\nfunc GetAutoConfClient(cfg *Config) (*autoconf.Client, error) {\n\tclientOnce.Do(func() {\n\t\tclientCache, clientErr = newAutoConfClient(cfg)\n\t})\n\treturn clientCache, clientErr\n}\n\n// newAutoConfClient creates a new autoconf client with the given config\nfunc newAutoConfClient(cfg *Config) (*autoconf.Client, error) {\n\t// Get repo path for cache directory\n\trepoPath, err := PathRoot()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to get repo path: %w\", err)\n\t}\n\n\t// Prepare refresh interval with nil check\n\trefreshInterval := cfg.AutoConf.RefreshInterval\n\tif refreshInterval == nil {\n\t\trefreshInterval = &OptionalDuration{}\n\t}\n\n\t// Use default URL if not specified\n\turl := cfg.AutoConf.URL.WithDefault(DefaultAutoConfURL)\n\n\t// Build client options\n\toptions := []autoconf.Option{\n\t\tautoconf.WithCacheDir(filepath.Join(repoPath, \"autoconf\")),\n\t\tautoconf.WithUserAgent(version.GetUserAgentVersion()),\n\t\tautoconf.WithCacheSize(DefaultAutoConfCacheSize),\n\t\tautoconf.WithTimeout(DefaultAutoConfTimeout),\n\t\tautoconf.WithRefreshInterval(refreshInterval.WithDefault(DefaultAutoConfRefreshInterval)),\n\t\tautoconf.WithFallback(autoconf.GetMainnetFallbackConfig),\n\t\tautoconf.WithURL(url),\n\t}\n\n\treturn autoconf.NewClient(options...)\n}\n\n// ValidateAutoConfWithRepo validates that autoconf setup is correct at daemon startup with repo access\nfunc ValidateAutoConfWithRepo(cfg *Config, swarmKeyExists bool) error {\n\tif !cfg.AutoConf.Enabled.WithDefault(DefaultAutoConfEnabled) {\n\t\t// AutoConf is disabled, check for \"auto\" values and warn\n\t\treturn validateAutoConfDisabled(cfg)\n\t}\n\n\t// Check for private network with default mainnet URL\n\turl := cfg.AutoConf.URL.WithDefault(DefaultAutoConfURL)\n\tif swarmKeyExists && url == DefaultAutoConfURL {\n\t\treturn fmt.Errorf(\"AutoConf cannot use the default mainnet URL (%s) on a private network (swarm.key or LIBP2P_FORCE_PNET detected). Either disable AutoConf by setting AutoConf.Enabled=false, or configure AutoConf.URL to point to a configuration service specific to your private swarm\", DefaultAutoConfURL)\n\t}\n\n\t// Further validation will happen lazily when config is accessed\n\treturn nil\n}\n\n// validateAutoConfDisabled checks for \"auto\" values when AutoConf is disabled and logs errors\nfunc validateAutoConfDisabled(cfg *Config) error {\n\thasAutoValues := false\n\tvar errors []string\n\n\t// Check Bootstrap\n\tif slices.Contains(cfg.Bootstrap, AutoPlaceholder) {\n\t\thasAutoValues = true\n\t\terrors = append(errors, \"Bootstrap contains 'auto' but AutoConf.Enabled=false\")\n\t}\n\n\t// Check DNS.Resolvers\n\tif cfg.DNS.Resolvers != nil {\n\t\tfor _, resolver := range cfg.DNS.Resolvers {\n\t\t\tif resolver == AutoPlaceholder {\n\t\t\t\thasAutoValues = true\n\t\t\t\terrors = append(errors, \"DNS.Resolvers contains 'auto' but AutoConf.Enabled=false\")\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t}\n\n\t// Check Routing.DelegatedRouters\n\tif slices.Contains(cfg.Routing.DelegatedRouters, AutoPlaceholder) {\n\t\thasAutoValues = true\n\t\terrors = append(errors, \"Routing.DelegatedRouters contains 'auto' but AutoConf.Enabled=false\")\n\t}\n\n\t// Check Ipns.DelegatedPublishers\n\tif slices.Contains(cfg.Ipns.DelegatedPublishers, AutoPlaceholder) {\n\t\thasAutoValues = true\n\t\terrors = append(errors, \"Ipns.DelegatedPublishers contains 'auto' but AutoConf.Enabled=false\")\n\t}\n\n\t// Log all errors\n\tfor _, errMsg := range errors {\n\t\tautoconfLog.Error(errMsg)\n\t}\n\n\t// If only auto values exist and no static ones, fail to start\n\tif hasAutoValues {\n\t\tif len(cfg.Bootstrap) == 1 && cfg.Bootstrap[0] == AutoPlaceholder {\n\t\t\tautoconfLog.Error(\"Kubo cannot start with only 'auto' Bootstrap values when AutoConf.Enabled=false\")\n\t\t\treturn fmt.Errorf(\"no usable bootstrap peers: AutoConf is disabled (AutoConf.Enabled=false) but 'auto' placeholder is used in Bootstrap config. Either set AutoConf.Enabled=true to enable automatic configuration, or replace 'auto' with specific Bootstrap peer addresses\")\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "config/autoconf_test.go", "content": "package config\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAutoConfDefaults(t *testing.T) {\n\t// Test that AutoConf has the correct default values\n\tcfg := &Config{\n\t\tAutoConf: AutoConf{\n\t\t\tURL: NewOptionalString(DefaultAutoConfURL),\n\t\t\tEnabled: True,\n\t\t},\n\t}\n\n\tassert.Equal(t, DefaultAutoConfURL, cfg.AutoConf.URL.WithDefault(DefaultAutoConfURL))\n\tassert.True(t, cfg.AutoConf.Enabled.WithDefault(DefaultAutoConfEnabled))\n\n\t// Test default refresh interval\n\tif cfg.AutoConf.RefreshInterval == nil {\n\t\t// This is expected - nil means use default\n\t\tduration := (*OptionalDuration)(nil).WithDefault(DefaultAutoConfRefreshInterval)\n\t\tassert.Equal(t, DefaultAutoConfRefreshInterval, duration)\n\t}\n}\n\nfunc TestAutoConfProfile(t *testing.T) {\n\tcfg := &Config{\n\t\tBootstrap: []string{\"some\", \"existing\", \"peers\"},\n\t\tDNS: DNS{\n\t\t\tResolvers: map[string]string{\n\t\t\t\t\"eth.\": \"https://example.com\",\n\t\t\t},\n\t\t},\n\t\tRouting: Routing{\n\t\t\tDelegatedRouters: []string{\"https://existing.router\"},\n\t\t},\n\t\tIpns: Ipns{\n\t\t\tDelegatedPublishers: []string{\"https://existing.publisher\"},\n\t\t},\n\t\tAutoConf: AutoConf{\n\t\t\tEnabled: False,\n\t\t},\n\t}\n\n\t// Apply autoconf profile\n\tprofile, ok := Profiles[\"autoconf-on\"]\n\trequire.True(t, ok, \"autoconf-on profile not found\")\n\n\terr := profile.Transform(cfg)\n\trequire.NoError(t, err)\n\n\t// Check that values were set to \"auto\"\n\tassert.Equal(t, []string{AutoPlaceholder}, cfg.Bootstrap)\n\tassert.Equal(t, AutoPlaceholder, cfg.DNS.Resolvers[\".\"])\n\tassert.Equal(t, []string{AutoPlaceholder}, cfg.Routing.DelegatedRouters)\n\tassert.Equal(t, []string{AutoPlaceholder}, cfg.Ipns.DelegatedPublishers)\n\n\t// Check that AutoConf was enabled\n\tassert.True(t, cfg.AutoConf.Enabled.WithDefault(DefaultAutoConfEnabled))\n\n\t// Check that URL was set\n\tassert.Equal(t, DefaultAutoConfURL, cfg.AutoConf.URL.WithDefault(DefaultAutoConfURL))\n}\n\nfunc TestInitWithAutoValues(t *testing.T) {\n\tidentity := Identity{\n\t\tPeerID: \"QmTest\",\n\t}\n\n\tcfg, err := InitWithIdentity(identity)\n\trequire.NoError(t, err)\n\n\t// Check that Bootstrap is set to \"auto\"\n\tassert.Equal(t, []string{AutoPlaceholder}, cfg.Bootstrap)\n\n\t// Check that DNS resolver is set to \"auto\"\n\tassert.Equal(t, AutoPlaceholder, cfg.DNS.Resolvers[\".\"])\n\n\t// Check that DelegatedRouters is set to \"auto\"\n\tassert.Equal(t, []string{AutoPlaceholder}, cfg.Routing.DelegatedRouters)\n\n\t// Check that DelegatedPublishers is set to \"auto\"\n\tassert.Equal(t, []string{AutoPlaceholder}, cfg.Ipns.DelegatedPublishers)\n\n\t// Check that AutoConf is enabled with correct URL\n\tassert.True(t, cfg.AutoConf.Enabled.WithDefault(DefaultAutoConfEnabled))\n\tassert.Equal(t, DefaultAutoConfURL, cfg.AutoConf.URL.WithDefault(DefaultAutoConfURL))\n}\n" }, { "path": "config/autonat.go", "content": "package config\n\nimport (\n\t\"fmt\"\n)\n\n// AutoNATServiceMode configures the ipfs node's AutoNAT service.\ntype AutoNATServiceMode int\n\nconst (\n\t// AutoNATServiceUnset indicates that the user has not set the\n\t// AutoNATService mode.\n\t//\n\t// When unset, nodes configured to be public DHT nodes will _also_\n\t// perform limited AutoNAT dialbacks.\n\tAutoNATServiceUnset AutoNATServiceMode = iota\n\t// AutoNATServiceEnabled indicates that the user has enabled the\n\t// AutoNATService.\n\tAutoNATServiceEnabled\n\t// AutoNATServiceDisabled indicates that the user has disabled the\n\t// AutoNATService.\n\tAutoNATServiceDisabled\n\t// AutoNATServiceEnabledV1Only forces use of V1 and disables V2\n\t// (used for testing)\n\tAutoNATServiceEnabledV1Only\n)\n\nfunc (m *AutoNATServiceMode) UnmarshalText(text []byte) error {\n\tswitch string(text) {\n\tcase \"\":\n\t\t*m = AutoNATServiceUnset\n\tcase \"enabled\":\n\t\t*m = AutoNATServiceEnabled\n\tcase \"disabled\":\n\t\t*m = AutoNATServiceDisabled\n\tcase \"legacy-v1\":\n\t\t*m = AutoNATServiceEnabledV1Only\n\tdefault:\n\t\treturn fmt.Errorf(\"unknown autonat mode: %s\", string(text))\n\t}\n\treturn nil\n}\n\nfunc (m AutoNATServiceMode) MarshalText() ([]byte, error) {\n\tswitch m {\n\tcase AutoNATServiceUnset:\n\t\treturn nil, nil\n\tcase AutoNATServiceEnabled:\n\t\treturn []byte(\"enabled\"), nil\n\tcase AutoNATServiceDisabled:\n\t\treturn []byte(\"disabled\"), nil\n\tcase AutoNATServiceEnabledV1Only:\n\t\treturn []byte(\"legacy-v1\"), nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unknown autonat mode: %d\", m)\n\t}\n}\n\n// AutoNATConfig configures the node's AutoNAT subsystem.\ntype AutoNATConfig struct {\n\t// ServiceMode configures the node's AutoNAT service mode.\n\tServiceMode AutoNATServiceMode `json:\",omitempty\"`\n\n\t// Throttle configures AutoNAT dialback throttling.\n\t//\n\t// If unset, the conservative libp2p defaults will be unset. To help the\n\t// network, please consider setting this and increasing the limits.\n\t//\n\t// By default, the limits will be a total of 30 dialbacks, with a\n\t// per-peer max of 3 peer, resetting every minute.\n\tThrottle *AutoNATThrottleConfig `json:\",omitempty\"`\n}\n\n// AutoNATThrottleConfig configures the throttle limites.\ntype AutoNATThrottleConfig struct {\n\t// GlobalLimit and PeerLimit sets the global and per-peer dialback\n\t// limits. The AutoNAT service will only perform the specified number of\n\t// dialbacks per interval.\n\t//\n\t// Setting either to 0 will disable the appropriate limit.\n\tGlobalLimit, PeerLimit int\n\n\t// Interval specifies how frequently this node should reset the\n\t// global/peer dialback limits.\n\t//\n\t// When unset, this defaults to 1 minute.\n\tInterval OptionalDuration\n}\n" }, { "path": "config/autotls.go", "content": "package config\n\nimport (\n\t\"time\"\n\n\tp2pforge \"github.com/ipshipyard/p2p-forge/client\"\n)\n\n// AutoTLS includes optional configuration of p2p-forge client of service\n// for obtaining a domain and TLS certificate to improve connectivity for web\n// browser clients. More: https://github.com/ipshipyard/p2p-forge#readme\ntype AutoTLS struct {\n\t// Enables the p2p-forge feature and all related features.\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// Optional, controls if Kubo should add /tls/sni/.../ws listener to every /tcp port if no explicit /ws is defined in Addresses.Swarm\n\tAutoWSS Flag `json:\",omitempty\"`\n\n\t// Optional, controls whether to skip network DNS lookups for p2p-forge domains.\n\t// Applies to resolution via DNS.Resolvers, including /dns* multiaddrs in go-libp2p.\n\t// When enabled (default), A/AAAA queries for *.libp2p.direct are resolved\n\t// locally by parsing the IP directly from the hostname, avoiding network I/O.\n\t// Set to false to always use network DNS (useful for debugging).\n\tSkipDNSLookup Flag `json:\",omitempty\"`\n\n\t// Optional override of the parent domain that will be used\n\tDomainSuffix *OptionalString `json:\",omitempty\"`\n\n\t// Optional override of HTTP API that acts as ACME DNS-01 Challenge broker\n\tRegistrationEndpoint *OptionalString `json:\",omitempty\"`\n\n\t// Optional Authorization token, used with private/test instances of p2p-forge\n\tRegistrationToken *OptionalString `json:\",omitempty\"`\n\n\t// Optional registration delay used when AutoTLS.Enabled is not explicitly set to true in config\n\tRegistrationDelay *OptionalDuration `json:\",omitempty\"`\n\n\t// Optional override of CA ACME API used by p2p-forge system\n\tCAEndpoint *OptionalString `json:\",omitempty\"`\n\n\t// Optional, controls if features like AutoWSS should generate shorter /dnsX instead of /ipX/../sni/..\n\tShortAddrs Flag `json:\",omitempty\"`\n}\n\nconst (\n\tDefaultAutoTLSEnabled = true // with DefaultAutoTLSRegistrationDelay, unless explicitly enabled in config\n\tDefaultDomainSuffix = p2pforge.DefaultForgeDomain\n\tDefaultRegistrationEndpoint = p2pforge.DefaultForgeEndpoint\n\tDefaultCAEndpoint = p2pforge.DefaultCAEndpoint\n\tDefaultAutoWSS = true // requires AutoTLS.Enabled\n\tDefaultAutoTLSShortAddrs = true // requires AutoTLS.Enabled\n\tDefaultAutoTLSSkipDNSLookup = true // skip network DNS for p2p-forge domains\n\tDefaultAutoTLSRegistrationDelay = 1 * time.Hour\n)\n" }, { "path": "config/bitswap.go", "content": "package config\n\n// Bitswap holds Bitswap configuration options\ntype Bitswap struct {\n\t// Libp2pEnabled controls if the node initializes bitswap over libp2p (enabled by default)\n\t// (This can be disabled if HTTPRetrieval.Enabled is set to true)\n\tLibp2pEnabled Flag `json:\",omitempty\"`\n\t// ServerEnabled controls if the node responds to WANTs (depends on Libp2pEnabled, enabled by default)\n\tServerEnabled Flag `json:\",omitempty\"`\n}\n\nconst (\n\tDefaultBitswapLibp2pEnabled = true\n\tDefaultBitswapServerEnabled = true\n)\n" }, { "path": "config/bootstrap_peers.go", "content": "package config\n\nimport (\n\t\"errors\"\n\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\n// ErrInvalidPeerAddr signals an address is not a valid peer address.\nvar ErrInvalidPeerAddr = errors.New(\"invalid peer address\")\n\nfunc (c *Config) BootstrapPeers() ([]peer.AddrInfo, error) {\n\treturn ParseBootstrapPeers(c.Bootstrap)\n}\n\nfunc (c *Config) SetBootstrapPeers(bps []peer.AddrInfo) {\n\tc.Bootstrap = BootstrapPeerStrings(bps)\n}\n\n// ParseBootstrapPeers parses a bootstrap list into a list of AddrInfos.\nfunc ParseBootstrapPeers(addrs []string) ([]peer.AddrInfo, error) {\n\tmaddrs := make([]ma.Multiaddr, len(addrs))\n\tfor i, addr := range addrs {\n\t\tvar err error\n\t\tmaddrs[i], err = ma.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn peer.AddrInfosFromP2pAddrs(maddrs...)\n}\n\n// BootstrapPeerStrings formats a list of AddrInfos as a bootstrap peer list\n// suitable for serialization.\nfunc BootstrapPeerStrings(bps []peer.AddrInfo) []string {\n\tbpss := make([]string, 0, len(bps))\n\tfor _, pi := range bps {\n\t\taddrs, err := peer.AddrInfoToP2pAddrs(&pi)\n\t\tif err != nil {\n\t\t\t// programmer error.\n\t\t\tpanic(err)\n\t\t}\n\t\tfor _, addr := range addrs {\n\t\t\tbpss = append(bpss, addr.String())\n\t\t}\n\t}\n\treturn bpss\n}\n" }, { "path": "config/bootstrap_peers_test.go", "content": "package config\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestBootstrapPeerStrings(t *testing.T) {\n\t// Test round-trip: string -> parse -> format -> string\n\t// This ensures that parsing and formatting are inverse operations\n\n\t// Start with the default bootstrap peer multiaddr strings\n\toriginalStrings := autoconf.FallbackBootstrapPeers\n\n\t// Parse multiaddr strings into structured peer data\n\tparsed, err := ParseBootstrapPeers(originalStrings)\n\trequire.NoError(t, err, \"parsing bootstrap peers should succeed\")\n\n\t// Format the parsed data back into multiaddr strings\n\tformattedStrings := BootstrapPeerStrings(parsed)\n\n\t// Verify round-trip: we should get back exactly what we started with\n\tassert.ElementsMatch(t, originalStrings, formattedStrings,\n\t\t\"round-trip through parse/format should preserve all bootstrap peers\")\n}\n" }, { "path": "config/config.go", "content": "// package config implements the ipfs config file datastructures and utilities.\npackage config\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"reflect\"\n\t\"strings\"\n\n\t\"github.com/ipfs/kubo/misc/fsutil\"\n)\n\n// Config is used to load ipfs config files.\ntype Config struct {\n\tIdentity Identity // local node's peer identity\n\tDatastore Datastore // local node's storage\n\tAddresses Addresses // local node's addresses\n\tMounts Mounts // local node's mount points\n\tDiscovery Discovery // local node's discovery mechanisms\n\tRouting Routing // local node's routing settings\n\tIpns Ipns // Ipns settings\n\tBootstrap []string // local nodes's bootstrap peer addresses\n\tGateway Gateway // local node's gateway server options\n\tAPI API // local node's API settings\n\tSwarm SwarmConfig\n\tAutoNAT AutoNATConfig\n\tAutoTLS AutoTLS\n\tPubsub PubsubConfig\n\tPeering Peering\n\tDNS DNS\n\n\tMigration Migration\n\tAutoConf AutoConf\n\n\tProvide Provide // Merged Provider and Reprovider configuration\n\tProvider Provider // Deprecated: use Provide. Will be removed in a future release.\n\tReprovider Reprovider // Deprecated: use Provide. Will be removed in a future release.\n\tHTTPRetrieval HTTPRetrieval\n\tExperimental Experiments\n\tPlugins Plugins\n\tPinning Pinning\n\tImport Import\n\tVersion Version\n\n\tInternal Internal // experimental/unstable options\n\n\tBitswap Bitswap\n}\n\nconst (\n\t// DefaultPathName is the default config dir name.\n\tDefaultPathName = \".ipfs\"\n\t// DefaultPathRoot is the path to the default config dir location.\n\tDefaultPathRoot = \"~/\" + DefaultPathName\n\t// DefaultConfigFile is the filename of the configuration file.\n\tDefaultConfigFile = \"config\"\n\t// EnvDir is the environment variable used to change the path root.\n\tEnvDir = \"IPFS_PATH\"\n)\n\n// PathRoot returns the default configuration root directory.\nfunc PathRoot() (string, error) {\n\tdir := os.Getenv(EnvDir)\n\tvar err error\n\tif len(dir) == 0 {\n\t\tdir, err = fsutil.ExpandHome(DefaultPathRoot)\n\t}\n\treturn dir, err\n}\n\n// Path returns the path `extension` relative to the configuration root. If an\n// empty string is provided for `configroot`, the default root is used.\nfunc Path(configroot, extension string) (string, error) {\n\tif len(configroot) == 0 {\n\t\tdir, err := PathRoot()\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t\treturn filepath.Join(dir, extension), nil\n\n\t}\n\treturn filepath.Join(configroot, extension), nil\n}\n\n// Filename returns the configuration file path given a configuration root\n// directory and a user-provided configuration file path argument with the\n// following rules:\n// - If the user-provided configuration file path is empty, use the default one.\n// - If the configuration root directory is empty, use the default one.\n// - If the user-provided configuration file path is only a file name, use the\n// configuration root directory, otherwise use only the user-provided path\n// and ignore the configuration root.\nfunc Filename(configroot, userConfigFile string) (string, error) {\n\tif userConfigFile == \"\" {\n\t\treturn Path(configroot, DefaultConfigFile)\n\t}\n\n\tif filepath.Dir(userConfigFile) == \".\" {\n\t\treturn Path(configroot, userConfigFile)\n\t}\n\n\treturn userConfigFile, nil\n}\n\n// HumanOutput gets a config value ready for printing.\nfunc HumanOutput(value any) ([]byte, error) {\n\ts, ok := value.(string)\n\tif ok {\n\t\treturn []byte(strings.Trim(s, \"\\n\")), nil\n\t}\n\treturn Marshal(value)\n}\n\n// Marshal configuration with JSON.\nfunc Marshal(value any) ([]byte, error) {\n\t// need to prettyprint, hence MarshalIndent, instead of Encoder\n\treturn json.MarshalIndent(value, \"\", \" \")\n}\n\nfunc FromMap(v map[string]any) (*Config, error) {\n\tbuf := new(bytes.Buffer)\n\tif err := json.NewEncoder(buf).Encode(v); err != nil {\n\t\treturn nil, err\n\t}\n\tvar conf Config\n\tif err := json.NewDecoder(buf).Decode(&conf); err != nil {\n\t\treturn nil, fmt.Errorf(\"failure to decode config: %w\", err)\n\t}\n\treturn &conf, nil\n}\n\nfunc ToMap(conf *Config) (map[string]any, error) {\n\tbuf := new(bytes.Buffer)\n\tif err := json.NewEncoder(buf).Encode(conf); err != nil {\n\t\treturn nil, err\n\t}\n\tvar m map[string]any\n\tif err := json.NewDecoder(buf).Decode(&m); err != nil {\n\t\treturn nil, fmt.Errorf(\"failure to decode config: %w\", err)\n\t}\n\treturn m, nil\n}\n\n// Convert config to a map, without using encoding/json, since\n// zero/empty/'omitempty' fields are excluded by encoding/json during\n// marshaling.\nfunc ReflectToMap(conf any) any {\n\tv := reflect.ValueOf(conf)\n\tif !v.IsValid() {\n\t\treturn nil\n\t}\n\n\t// Handle pointer type\n\tif v.Kind() == reflect.Pointer {\n\t\tif v.IsNil() {\n\t\t\t// Create a zero value of the pointer's element type\n\t\t\telemType := v.Type().Elem()\n\t\t\tzero := reflect.Zero(elemType)\n\t\t\treturn ReflectToMap(zero.Interface())\n\t\t}\n\t\tv = v.Elem()\n\t}\n\n\tswitch v.Kind() {\n\tcase reflect.Struct:\n\t\tresult := make(map[string]any)\n\t\tt := v.Type()\n\t\tfor i := 0; i < v.NumField(); i++ {\n\t\t\tfield := v.Field(i)\n\t\t\t// Only include exported fields\n\t\t\tif field.CanInterface() {\n\t\t\t\tresult[t.Field(i).Name] = ReflectToMap(field.Interface())\n\t\t\t}\n\t\t}\n\t\treturn result\n\n\tcase reflect.Map:\n\t\tresult := make(map[string]any)\n\t\titer := v.MapRange()\n\t\tfor iter.Next() {\n\t\t\tkey := iter.Key()\n\t\t\t// Convert map keys to strings for consistency\n\t\t\tkeyStr := fmt.Sprint(ReflectToMap(key.Interface()))\n\t\t\tresult[keyStr] = ReflectToMap(iter.Value().Interface())\n\t\t}\n\t\t// Add a sample to differentiate between a map and a struct on validation.\n\t\tsample := reflect.Zero(v.Type().Elem())\n\t\tif sample.CanInterface() {\n\t\t\tresult[\"*\"] = ReflectToMap(sample.Interface())\n\t\t}\n\t\treturn result\n\n\tcase reflect.Slice, reflect.Array:\n\t\tresult := make([]any, v.Len())\n\t\tfor i := 0; i < v.Len(); i++ {\n\t\t\tresult[i] = ReflectToMap(v.Index(i).Interface())\n\t\t}\n\t\treturn result\n\n\tdefault:\n\t\t// For basic types (int, string, etc.), just return the value\n\t\tif v.CanInterface() {\n\t\t\treturn v.Interface()\n\t\t}\n\t\treturn nil\n\t}\n}\n\n// Clone copies the config. Use when updating.\nfunc (c *Config) Clone() (*Config, error) {\n\tvar newConfig Config\n\tvar buf bytes.Buffer\n\n\tif err := json.NewEncoder(&buf).Encode(c); err != nil {\n\t\treturn nil, fmt.Errorf(\"failure to encode config: %w\", err)\n\t}\n\n\tif err := json.NewDecoder(&buf).Decode(&newConfig); err != nil {\n\t\treturn nil, fmt.Errorf(\"failure to decode config: %w\", err)\n\t}\n\n\treturn &newConfig, nil\n}\n\n// Check if the provided key is present in the structure.\nfunc CheckKey(key string) error {\n\tconf := Config{}\n\n\t// Convert an empty config to a map without JSON.\n\tcursor := ReflectToMap(&conf)\n\n\t// Parse the key and verify it's presence in the map.\n\tvar ok bool\n\tvar mapCursor map[string]any\n\n\tparts := strings.Split(key, \".\")\n\tfor i, part := range parts {\n\t\tmapCursor, ok = cursor.(map[string]any)\n\t\tif !ok {\n\t\t\tif cursor == nil {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tpath := strings.Join(parts[:i], \".\")\n\t\t\treturn fmt.Errorf(\"%s key is not a map\", path)\n\t\t}\n\n\t\tcursor, ok = mapCursor[part]\n\t\tif !ok {\n\t\t\t// If the config sections is a map, validate against the default entry.\n\t\t\tif cursor, ok = mapCursor[\"*\"]; ok {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpath := strings.Join(parts[:i+1], \".\")\n\t\t\treturn fmt.Errorf(\"%s not found\", path)\n\t\t}\n\t}\n\treturn nil\n}\n" }, { "path": "config/config_test.go", "content": "package config\n\nimport (\n\t\"testing\"\n)\n\nfunc TestClone(t *testing.T) {\n\tc := new(Config)\n\tc.Identity.PeerID = \"faketest\"\n\tc.API.HTTPHeaders = map[string][]string{\"foo\": {\"bar\"}}\n\n\tnewCfg, err := c.Clone()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif newCfg.Identity.PeerID != c.Identity.PeerID {\n\t\tt.Fatal(\"peer ID not preserved\")\n\t}\n\n\tc.API.HTTPHeaders[\"foo\"] = []string{\"baz\"}\n\tif newCfg.API.HTTPHeaders[\"foo\"][0] != \"bar\" {\n\t\tt.Fatal(\"HTTP headers not preserved\")\n\t}\n\n\tdelete(c.API.HTTPHeaders, \"foo\")\n\tif newCfg.API.HTTPHeaders[\"foo\"][0] != \"bar\" {\n\t\tt.Fatal(\"HTTP headers not preserved\")\n\t}\n}\n\nfunc TestReflectToMap(t *testing.T) {\n\t// Helper function to create a test config with various field types\n\treflectedConfig := ReflectToMap(new(Config))\n\n\tmapConfig, ok := reflectedConfig.(map[string]any)\n\tif !ok {\n\t\tt.Fatal(\"Config didn't convert to map\")\n\t}\n\n\treflectedIdentity, ok := mapConfig[\"Identity\"]\n\tif !ok {\n\t\tt.Fatal(\"Identity field not found\")\n\t}\n\n\tmapIdentity, ok := reflectedIdentity.(map[string]any)\n\tif !ok {\n\t\tt.Fatal(\"Identity field didn't convert to map\")\n\t}\n\n\t// Test string field reflection\n\treflectedPeerID, ok := mapIdentity[\"PeerID\"]\n\tif !ok {\n\t\tt.Fatal(\"PeerID field not found in Identity\")\n\t}\n\tif _, ok := reflectedPeerID.(string); !ok {\n\t\tt.Fatal(\"PeerID field didn't convert to string\")\n\t}\n\n\t// Test omitempty json string field\n\treflectedPrivKey, ok := mapIdentity[\"PrivKey\"]\n\tif !ok {\n\t\tt.Fatal(\"PrivKey omitempty field not found in Identity\")\n\t}\n\tif _, ok := reflectedPrivKey.(string); !ok {\n\t\tt.Fatal(\"PrivKey omitempty field didn't convert to string\")\n\t}\n\n\t// Test slices field\n\treflectedBootstrap, ok := mapConfig[\"Bootstrap\"]\n\tif !ok {\n\t\tt.Fatal(\"Bootstrap field not found in config\")\n\t}\n\tbootstrap, ok := reflectedBootstrap.([]any)\n\tif !ok {\n\t\tt.Fatal(\"Bootstrap field didn't convert to []string\")\n\t}\n\tif len(bootstrap) != 0 {\n\t\tt.Fatal(\"Bootstrap len is incorrect\")\n\t}\n\n\treflectedDatastore, ok := mapConfig[\"Datastore\"]\n\tif !ok {\n\t\tt.Fatal(\"Datastore field not found in config\")\n\t}\n\tdatastore, ok := reflectedDatastore.(map[string]any)\n\tif !ok {\n\t\tt.Fatal(\"Datastore field didn't convert to map\")\n\t}\n\tstorageGCWatermark, ok := datastore[\"StorageGCWatermark\"]\n\tif !ok {\n\t\tt.Fatal(\"StorageGCWatermark field not found in Datastore\")\n\t}\n\t// Test int field\n\tif _, ok := storageGCWatermark.(int64); !ok {\n\t\tt.Fatal(\"StorageGCWatermark field didn't convert to int64\")\n\t}\n\tnoSync, ok := datastore[\"NoSync\"]\n\tif !ok {\n\t\tt.Fatal(\"NoSync field not found in Datastore\")\n\t}\n\t// Test bool field\n\tif _, ok := noSync.(bool); !ok {\n\t\tt.Fatal(\"NoSync field didn't convert to bool\")\n\t}\n\n\treflectedDNS, ok := mapConfig[\"DNS\"]\n\tif !ok {\n\t\tt.Fatal(\"DNS field not found in config\")\n\t}\n\tDNS, ok := reflectedDNS.(map[string]any)\n\tif !ok {\n\t\tt.Fatal(\"DNS field didn't convert to map\")\n\t}\n\treflectedResolvers, ok := DNS[\"Resolvers\"]\n\tif !ok {\n\t\tt.Fatal(\"Resolvers field not found in DNS\")\n\t}\n\t// Test map field\n\tif _, ok := reflectedResolvers.(map[string]any); !ok {\n\t\tt.Fatal(\"Resolvers field didn't convert to map\")\n\t}\n\n\t// Test pointer field\n\tif _, ok := DNS[\"MaxCacheTTL\"].(map[string]any); !ok {\n\t\t// Since OptionalDuration only field is private, we cannot test it\n\t\tt.Fatal(\"MaxCacheTTL field didn't convert to map\")\n\t}\n}\n\n// Test validation of options set through \"ipfs config\"\nfunc TestCheckKey(t *testing.T) {\n\terr := CheckKey(\"Foo.Bar\")\n\tif err == nil {\n\t\tt.Fatal(\"Foo.Bar isn't a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Provide.Strategy\")\n\tif err != nil {\n\t\tt.Fatalf(\"%s: %s\", err, \"Provide.Strategy is a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Provide.DHT.MaxWorkers\")\n\tif err != nil {\n\t\tt.Fatalf(\"%s: %s\", err, \"Provide.DHT.MaxWorkers is a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Provide.DHT.Interval\")\n\tif err != nil {\n\t\tt.Fatalf(\"%s: %s\", err, \"Provide.DHT.Interval is a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Provide.Foo\")\n\tif err == nil {\n\t\tt.Fatal(\"Provide.Foo isn't a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Gateway.PublicGateways.Foo.Paths\")\n\tif err != nil {\n\t\tt.Fatalf(\"%s: %s\", err, \"Gateway.PublicGateways.Foo.Paths is a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Gateway.PublicGateways.Foo.Bar\")\n\tif err == nil {\n\t\tt.Fatal(\"Gateway.PublicGateways.Foo.Bar isn't a valid key in the config\")\n\t}\n\n\terr = CheckKey(\"Plugins.Plugins.peerlog.Config.Enabled\")\n\tif err != nil {\n\t\tt.Fatalf(\"%s: %s\", err, \"Plugins.Plugins.peerlog.Config.Enabled is a valid key in the config\")\n\t}\n}\n" }, { "path": "config/datastore.go", "content": "package config\n\nimport (\n\t\"encoding/json\"\n)\n\nconst (\n\t// DefaultDataStoreDirectory is the directory to store all the local IPFS data.\n\tDefaultDataStoreDirectory = \"datastore\"\n\n\t// DefaultBlockKeyCacheSize is the size for the blockstore two-queue\n\t// cache which caches block keys and sizes.\n\tDefaultBlockKeyCacheSize = 64 << 10\n\n\t// DefaultWriteThrough specifies whether to use a \"write-through\"\n\t// Blockstore and Blockservice. This means that they will write\n\t// without performing any reads to check if the incoming blocks are\n\t// already present in the datastore. Enable for datastores with fast\n\t// writes and slower reads.\n\tDefaultWriteThrough bool = true\n)\n\n// Datastore tracks the configuration of the datastore.\ntype Datastore struct {\n\tStorageMax string // in B, kB, kiB, MB, ...\n\tStorageGCWatermark int64 // in percentage to multiply on StorageMax\n\tGCPeriod string // in ns, us, ms, s, m, h\n\n\t// deprecated fields, use Spec\n\tType string `json:\",omitempty\"`\n\tPath string `json:\",omitempty\"`\n\tNoSync bool `json:\",omitempty\"`\n\tParams *json.RawMessage `json:\",omitempty\"`\n\n\tSpec map[string]any\n\n\tHashOnRead bool\n\tBloomFilterSize int\n\tBlockKeyCacheSize OptionalInteger\n\tWriteThrough Flag `json:\",omitempty\"`\n}\n\n// DataStorePath returns the default data store path given a configuration root\n// (set an empty string to have the default configuration root).\nfunc DataStorePath(configroot string) (string, error) {\n\treturn Path(configroot, DefaultDataStoreDirectory)\n}\n" }, { "path": "config/discovery.go", "content": "package config\n\ntype Discovery struct {\n\tMDNS MDNS\n}\n\ntype MDNS struct {\n\tEnabled bool\n}\n" }, { "path": "config/dns.go", "content": "package config\n\n// DNS specifies DNS resolution rules using custom resolvers.\ntype DNS struct {\n\t// Resolvers is a map of FQDNs to URLs for custom DNS resolution.\n\t// URLs starting with `https://` indicate DoH endpoints.\n\t// Support for other resolver types can be added in the future.\n\t// https://en.wikipedia.org/wiki/Fully_qualified_domain_name\n\t// https://en.wikipedia.org/wiki/DNS_over_HTTPS\n\t//\n\t// Example:\n\t// - Custom resolver for ENS: `eth.` → `https://dns.eth.limo/dns-query`\n\t// - Override the default OS resolver: `.` → `https://1.1.1.1/dns-query`\n\tResolvers map[string]string\n\t// MaxCacheTTL is the maximum duration DNS entries are valid in the cache.\n\tMaxCacheTTL *OptionalDuration `json:\",omitempty\"`\n}\n" }, { "path": "config/experiments.go", "content": "package config\n\ntype Experiments struct {\n\tFilestoreEnabled bool\n\tUrlstoreEnabled bool\n\tShardingEnabled bool `json:\",omitempty\"` // deprecated by autosharding: https://github.com/ipfs/kubo/pull/8527\n\tLibp2pStreamMounting bool\n\tP2pHttpProxy bool //nolint\n\tStrategicProviding bool `json:\",omitempty\"` // removed, use Provider.Enabled instead\n\tOptimisticProvide bool\n\tOptimisticProvideJobsPoolSize int\n\tGatewayOverLibp2p bool `json:\",omitempty\"`\n\n\tGraphsyncEnabled graphsyncEnabled `json:\",omitempty\"`\n\tAcceleratedDHTClient experimentalAcceleratedDHTClient `json:\",omitempty\"`\n}\n" }, { "path": "config/gateway.go", "content": "package config\n\nimport (\n\t\"github.com/ipfs/boxo/gateway\"\n)\n\nconst (\n\tDefaultInlineDNSLink = false\n\tDefaultDeserializedResponses = true\n\tDefaultDisableHTMLErrors = false\n\tDefaultExposeRoutingAPI = true\n\tDefaultDiagnosticServiceURL = \"https://check.ipfs.network\"\n\tDefaultAllowCodecConversion = false\n\n\t// Gateway limit defaults from boxo\n\tDefaultRetrievalTimeout = gateway.DefaultRetrievalTimeout\n\tDefaultMaxRequestDuration = gateway.DefaultMaxRequestDuration\n\tDefaultMaxConcurrentRequests = gateway.DefaultMaxConcurrentRequests\n\tDefaultMaxRangeRequestFileSize = 0 // 0 means no limit\n)\n\ntype GatewaySpec struct {\n\t// Paths is explicit list of path prefixes that should be handled by\n\t// this gateway. Example: `[\"/ipfs\", \"/ipns\"]`\n\tPaths []string\n\n\t// UseSubdomains indicates whether or not this gateway uses subdomains\n\t// for IPFS resources instead of paths. That is: http://CID.ipfs.GATEWAY/...\n\t//\n\t// If this flag is set, any /ipns/$id and/or /ipfs/$id paths in Paths\n\t// will be permanently redirected to http://$id.[ipns|ipfs].$gateway/.\n\t//\n\t// We do not support using both paths and subdomains for a single domain\n\t// for security reasons (Origin isolation).\n\tUseSubdomains bool\n\n\t// NoDNSLink configures this gateway to _not_ resolve DNSLink for the FQDN\n\t// provided in `Host` HTTP header.\n\tNoDNSLink bool\n\n\t// InlineDNSLink configures this gateway to always inline DNSLink names\n\t// (FQDN) into a single DNS label in order to interop with wildcard TLS certs\n\t// and Origin per CID isolation provided by rules like https://publicsuffix.org\n\tInlineDNSLink Flag\n\n\t// DeserializedResponses configures this gateway to respond to deserialized\n\t// responses. Disabling this option enables a Trustless Gateway, as per:\n\t// https://specs.ipfs.tech/http-gateways/trustless-gateway/.\n\tDeserializedResponses Flag\n}\n\n// Gateway contains options for the HTTP gateway server.\ntype Gateway struct {\n\t// HTTPHeaders configures the headers that should be returned by this\n\t// gateway.\n\tHTTPHeaders map[string][]string // HTTP headers to return with the gateway\n\n\t// RootRedirect is the path to which requests to `/` on this gateway\n\t// should be redirected.\n\tRootRedirect string\n\n\t// NoFetch configures the gateway to _not_ fetch blocks in response to\n\t// requests.\n\tNoFetch bool\n\n\t// NoDNSLink configures the gateway to _not_ perform DNS TXT record\n\t// lookups in response to requests with values in `Host` HTTP header.\n\t// This flag can be overridden per FQDN in PublicGateways.\n\tNoDNSLink bool\n\n\t// DeserializedResponses configures this gateway to respond to deserialized\n\t// requests. Disabling this option enables a Trustless only gateway, as per:\n\t// https://specs.ipfs.tech/http-gateways/trustless-gateway/. This can\n\t// be overridden per FQDN in PublicGateways.\n\tDeserializedResponses Flag\n\n\t// AllowCodecConversion enables automatic conversion between codecs when\n\t// the requested format differs from the block's native codec (e.g.,\n\t// converting dag-pb or dag-cbor to dag-json). When disabled, the gateway\n\t// returns 406 Not Acceptable for codec mismatches per IPIP-524.\n\tAllowCodecConversion Flag\n\n\t// DisableHTMLErrors disables pretty HTML pages when an error occurs. Instead, a `text/plain`\n\t// page will be sent with the raw error message.\n\tDisableHTMLErrors Flag\n\n\t// PublicGateways configures behavior of known public gateways.\n\t// Each key is a fully qualified domain name (FQDN).\n\tPublicGateways map[string]*GatewaySpec\n\n\t// ExposeRoutingAPI configures the gateway port to expose\n\t// routing system as HTTP API at /routing/v1 (https://specs.ipfs.tech/routing/http-routing-v1/).\n\tExposeRoutingAPI Flag\n\n\t// RetrievalTimeout enforces a maximum duration for content retrieval:\n\t// - Time to first byte: If the gateway cannot start writing the response within\n\t// this duration (e.g., stuck searching for providers), a 504 Gateway Timeout\n\t// is returned.\n\t// - Time between writes: After the first byte, the timeout resets each time new\n\t// bytes are written to the client. If the gateway cannot write additional data\n\t// within this duration after the last successful write, the response is terminated.\n\t// This helps free resources when the gateway gets stuck looking for providers\n\t// or cannot retrieve the requested content.\n\t// A value of 0 disables this timeout.\n\tRetrievalTimeout *OptionalDuration `json:\",omitempty\"`\n\n\t// MaxRequestDuration is an absolute deadline for the entire request.\n\t// Unlike RetrievalTimeout (which resets on each data write and catches\n\t// stalled transfers), this is a hard limit on the total time a request\n\t// can take. Returns 504 Gateway Timeout when exceeded.\n\t// This protects the gateway from edge cases and slow client attacks.\n\t// A value of 0 uses the default (1 hour).\n\tMaxRequestDuration *OptionalDuration `json:\",omitempty\"`\n\n\t// MaxConcurrentRequests limits concurrent HTTP requests handled by the gateway.\n\t// Requests beyond this limit receive 429 Too Many Requests with Retry-After header.\n\t// A value of 0 disables the limit.\n\tMaxConcurrentRequests *OptionalInteger `json:\",omitempty\"`\n\n\t// MaxRangeRequestFileSize limits the maximum file size for HTTP range requests.\n\t// Range requests for files larger than this limit return 501 Not Implemented.\n\t// This protects against CDN issues with large file range requests and prevents\n\t// excessive bandwidth consumption. A value of 0 disables the limit.\n\tMaxRangeRequestFileSize *OptionalBytes `json:\",omitempty\"`\n\n\t// DiagnosticServiceURL is the URL for a service to diagnose CID retrievability issues.\n\t// When the gateway returns a 504 Gateway Timeout error, an \"Inspect retrievability of CID\"\n\t// button will be shown that links to this service with the CID appended as ?cid=.\n\t// Set to empty string to disable the button.\n\tDiagnosticServiceURL *OptionalString `json:\",omitempty\"`\n}\n" }, { "path": "config/http_retrieval.go", "content": "package config\n\n// HTTPRetrieval is the configuration object for HTTP Retrieval settings.\n// Implicit defaults can be found in core/node/bitswap.go\ntype HTTPRetrieval struct {\n\tEnabled Flag `json:\",omitempty\"`\n\tAllowlist []string `json:\",omitempty\"`\n\tDenylist []string `json:\",omitempty\"`\n\tNumWorkers *OptionalInteger `json:\",omitempty\"`\n\tMaxBlockSize *OptionalString `json:\",omitempty\"`\n\tTLSInsecureSkipVerify Flag `json:\",omitempty\"`\n}\n\nconst (\n\tDefaultHTTPRetrievalEnabled = true\n\tDefaultHTTPRetrievalNumWorkers = 16\n\tDefaultHTTPRetrievalTLSInsecureSkipVerify = false // only for testing with self-signed HTTPS certs\n\tDefaultHTTPRetrievalMaxBlockSize = \"2MiB\" // matching bitswap: https://specs.ipfs.tech/bitswap-protocol/#block-sizes\n)\n" }, { "path": "config/identity.go", "content": "package config\n\nimport (\n\t\"encoding/base64\"\n\n\tic \"github.com/libp2p/go-libp2p/core/crypto\"\n)\n\nconst (\n\tIdentityTag = \"Identity\"\n\tPrivKeyTag = \"PrivKey\"\n\tPrivKeySelector = IdentityTag + \".\" + PrivKeyTag\n)\n\n// Identity tracks the configuration of the local node's identity.\ntype Identity struct {\n\tPeerID string\n\tPrivKey string `json:\",omitempty\"`\n}\n\n// DecodePrivateKey is a helper to decode the users PrivateKey.\nfunc (i *Identity) DecodePrivateKey(passphrase string) (ic.PrivKey, error) {\n\tpkb, err := base64.StdEncoding.DecodeString(i.PrivKey)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// currently storing key unencrypted. in the future we need to encrypt it.\n\t// TODO(security)\n\treturn ic.UnmarshalPrivateKey(pkb)\n}\n" }, { "path": "config/import.go", "content": "package config\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"strconv\"\n\t\"strings\"\n\n\tchunk \"github.com/ipfs/boxo/chunker\"\n\t\"github.com/ipfs/boxo/ipld/unixfs/importer/helpers\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\t\"github.com/ipfs/boxo/verifcid\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nconst (\n\tDefaultCidVersion = 0\n\tDefaultUnixFSRawLeaves = false\n\tDefaultUnixFSChunker = \"size-262144\"\n\tDefaultHashFunction = \"sha2-256\"\n\tDefaultFastProvideRoot = true\n\tDefaultFastProvideWait = false\n\n\tDefaultUnixFSHAMTDirectorySizeThreshold = 262144 // 256KiB - https://github.com/ipfs/boxo/blob/6c5a07602aed248acc86598f30ab61923a54a83e/ipld/unixfs/io/directory.go#L26\n\n\t// DefaultBatchMaxNodes controls the maximum number of nodes in a\n\t// write-batch. The total size of the batch is limited by\n\t// BatchMaxnodes and BatchMaxSize.\n\tDefaultBatchMaxNodes = 128\n\t// DefaultBatchMaxSize controls the maximum size of a single\n\t// write-batch. The total size of the batch is limited by\n\t// BatchMaxnodes and BatchMaxSize.\n\tDefaultBatchMaxSize = 100 << 20 // 20MiB\n\n\t// HAMTSizeEstimation values for Import.UnixFSHAMTDirectorySizeEstimation\n\tHAMTSizeEstimationLinks = \"links\" // legacy: estimate using link names + CID byte lengths (default)\n\tHAMTSizeEstimationBlock = \"block\" // full serialized dag-pb block size\n\tHAMTSizeEstimationDisabled = \"disabled\" // disable HAMT sharding entirely\n\n\t// DAGLayout values for Import.UnixFSDAGLayout\n\tDAGLayoutBalanced = \"balanced\" // balanced DAG layout (default)\n\tDAGLayoutTrickle = \"trickle\" // trickle DAG layout\n\n\tDefaultUnixFSHAMTDirectorySizeEstimation = HAMTSizeEstimationLinks // legacy behavior\n\tDefaultUnixFSDAGLayout = DAGLayoutBalanced // balanced DAG layout\n\tDefaultUnixFSIncludeEmptyDirs = true // include empty directories\n)\n\nvar (\n\tDefaultUnixFSFileMaxLinks = int64(helpers.DefaultLinksPerBlock)\n\tDefaultUnixFSDirectoryMaxLinks = int64(0)\n\tDefaultUnixFSHAMTDirectoryMaxFanout = int64(uio.DefaultShardWidth)\n)\n\n// Import configures the default options for ingesting data. This affects commands\n// that ingest data, such as 'ipfs add', 'ipfs dag put, 'ipfs block put', 'ipfs files write'.\ntype Import struct {\n\tCidVersion OptionalInteger\n\tUnixFSRawLeaves Flag\n\tUnixFSChunker OptionalString\n\tHashFunction OptionalString\n\tUnixFSFileMaxLinks OptionalInteger\n\tUnixFSDirectoryMaxLinks OptionalInteger\n\tUnixFSHAMTDirectoryMaxFanout OptionalInteger\n\tUnixFSHAMTDirectorySizeThreshold OptionalBytes\n\tUnixFSHAMTDirectorySizeEstimation OptionalString // \"links\", \"block\", or \"disabled\"\n\tUnixFSDAGLayout OptionalString // \"balanced\" or \"trickle\"\n\tBatchMaxNodes OptionalInteger\n\tBatchMaxSize OptionalInteger\n\tFastProvideRoot Flag\n\tFastProvideWait Flag\n}\n\n// ValidateImportConfig validates the Import configuration according to UnixFS spec requirements.\n// See: https://specs.ipfs.tech/unixfs/#hamt-structure-and-parameters\nfunc ValidateImportConfig(cfg *Import) error {\n\t// Validate CidVersion\n\tif !cfg.CidVersion.IsDefault() {\n\t\tcidVer := cfg.CidVersion.WithDefault(DefaultCidVersion)\n\t\tif cidVer != 0 && cidVer != 1 {\n\t\t\treturn fmt.Errorf(\"Import.CidVersion must be 0 or 1, got %d\", cidVer)\n\t\t}\n\t}\n\n\t// Validate UnixFSFileMaxLinks\n\tif !cfg.UnixFSFileMaxLinks.IsDefault() {\n\t\tmaxLinks := cfg.UnixFSFileMaxLinks.WithDefault(DefaultUnixFSFileMaxLinks)\n\t\tif maxLinks <= 0 {\n\t\t\treturn fmt.Errorf(\"Import.UnixFSFileMaxLinks must be positive, got %d\", maxLinks)\n\t\t}\n\t}\n\n\t// Validate UnixFSDirectoryMaxLinks\n\tif !cfg.UnixFSDirectoryMaxLinks.IsDefault() {\n\t\tmaxLinks := cfg.UnixFSDirectoryMaxLinks.WithDefault(DefaultUnixFSDirectoryMaxLinks)\n\t\tif maxLinks < 0 {\n\t\t\treturn fmt.Errorf(\"Import.UnixFSDirectoryMaxLinks must be non-negative, got %d\", maxLinks)\n\t\t}\n\t}\n\n\t// Validate UnixFSHAMTDirectoryMaxFanout if set\n\tif !cfg.UnixFSHAMTDirectoryMaxFanout.IsDefault() {\n\t\tfanout := cfg.UnixFSHAMTDirectoryMaxFanout.WithDefault(DefaultUnixFSHAMTDirectoryMaxFanout)\n\n\t\t// Valid values are powers of 2 between 8 and 1024: 8, 16, 32, 64, 128, 256, 512, 1024\n\t\tif fanout < 8 || !isPowerOfTwo(fanout) || fanout > 1024 {\n\t\t\treturn fmt.Errorf(\"Import.UnixFSHAMTDirectoryMaxFanout must be a power of 2, between 8 and 1024 (got %d)\", fanout)\n\t\t}\n\t}\n\n\t// Validate BatchMaxNodes\n\tif !cfg.BatchMaxNodes.IsDefault() {\n\t\tmaxNodes := cfg.BatchMaxNodes.WithDefault(DefaultBatchMaxNodes)\n\t\tif maxNodes <= 0 {\n\t\t\treturn fmt.Errorf(\"Import.BatchMaxNodes must be positive, got %d\", maxNodes)\n\t\t}\n\t}\n\n\t// Validate BatchMaxSize\n\tif !cfg.BatchMaxSize.IsDefault() {\n\t\tmaxSize := cfg.BatchMaxSize.WithDefault(DefaultBatchMaxSize)\n\t\tif maxSize <= 0 {\n\t\t\treturn fmt.Errorf(\"Import.BatchMaxSize must be positive, got %d\", maxSize)\n\t\t}\n\t}\n\n\t// Validate UnixFSChunker format\n\tif !cfg.UnixFSChunker.IsDefault() {\n\t\tchunker := cfg.UnixFSChunker.WithDefault(DefaultUnixFSChunker)\n\t\tif !isValidChunker(chunker) {\n\t\t\treturn fmt.Errorf(\"Import.UnixFSChunker invalid format: %q (expected \\\"size-\\\", \\\"rabin---\\\", or \\\"buzhash\\\")\", chunker)\n\t\t}\n\t}\n\n\t// Validate HashFunction\n\tif !cfg.HashFunction.IsDefault() {\n\t\thashFunc := cfg.HashFunction.WithDefault(DefaultHashFunction)\n\t\thashCode, ok := mh.Names[strings.ToLower(hashFunc)]\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"Import.HashFunction unrecognized: %q\", hashFunc)\n\t\t}\n\t\t// Check if the hash is allowed by verifcid\n\t\tif !verifcid.DefaultAllowlist.IsAllowed(hashCode) {\n\t\t\treturn fmt.Errorf(\"Import.HashFunction %q is not allowed for use in IPFS\", hashFunc)\n\t\t}\n\t}\n\n\t// Validate UnixFSHAMTDirectorySizeEstimation\n\tif !cfg.UnixFSHAMTDirectorySizeEstimation.IsDefault() {\n\t\test := cfg.UnixFSHAMTDirectorySizeEstimation.WithDefault(DefaultUnixFSHAMTDirectorySizeEstimation)\n\t\tswitch est {\n\t\tcase HAMTSizeEstimationLinks, HAMTSizeEstimationBlock, HAMTSizeEstimationDisabled:\n\t\t\t// valid\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"Import.UnixFSHAMTDirectorySizeEstimation must be %q, %q, or %q, got %q\",\n\t\t\t\tHAMTSizeEstimationLinks, HAMTSizeEstimationBlock, HAMTSizeEstimationDisabled, est)\n\t\t}\n\t}\n\n\t// Validate UnixFSDAGLayout\n\tif !cfg.UnixFSDAGLayout.IsDefault() {\n\t\tlayout := cfg.UnixFSDAGLayout.WithDefault(DefaultUnixFSDAGLayout)\n\t\tswitch layout {\n\t\tcase DAGLayoutBalanced, DAGLayoutTrickle:\n\t\t\t// valid\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"Import.UnixFSDAGLayout must be %q or %q, got %q\",\n\t\t\t\tDAGLayoutBalanced, DAGLayoutTrickle, layout)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// isPowerOfTwo checks if a number is a power of 2\nfunc isPowerOfTwo(n int64) bool {\n\treturn n > 0 && (n&(n-1)) == 0\n}\n\n// isValidChunker validates chunker format\nfunc isValidChunker(chunker string) bool {\n\tif chunker == \"buzhash\" {\n\t\treturn true\n\t}\n\n\t// Check for size- format\n\tif sizeStr, ok := strings.CutPrefix(chunker, \"size-\"); ok {\n\t\tif sizeStr == \"\" {\n\t\t\treturn false\n\t\t}\n\t\t// Check if it's a valid positive integer (no negative sign allowed)\n\t\tif sizeStr[0] == '-' {\n\t\t\treturn false\n\t\t}\n\t\tsize, err := strconv.Atoi(sizeStr)\n\t\t// Size must be positive (not zero)\n\t\treturn err == nil && size > 0\n\t}\n\n\t// Check for rabin--- format\n\tif strings.HasPrefix(chunker, \"rabin-\") {\n\t\tparts := strings.Split(chunker, \"-\")\n\t\tif len(parts) != 4 {\n\t\t\treturn false\n\t\t}\n\n\t\t// Parse and validate min, avg, max values\n\t\tvalues := make([]int, 3)\n\t\tfor i := range 3 {\n\t\t\tval, err := strconv.Atoi(parts[i+1])\n\t\t\tif err != nil {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tvalues[i] = val\n\t\t}\n\n\t\t// Validate ordering: min <= avg <= max\n\t\tmin, avg, max := values[0], values[1], values[2]\n\t\treturn min <= avg && avg <= max\n\t}\n\n\treturn false\n}\n\n// HAMTSizeEstimationMode returns the boxo SizeEstimationMode based on the config value.\nfunc (i *Import) HAMTSizeEstimationMode() uio.SizeEstimationMode {\n\tswitch i.UnixFSHAMTDirectorySizeEstimation.WithDefault(DefaultUnixFSHAMTDirectorySizeEstimation) {\n\tcase HAMTSizeEstimationLinks:\n\t\treturn uio.SizeEstimationLinks\n\tcase HAMTSizeEstimationBlock:\n\t\treturn uio.SizeEstimationBlock\n\tcase HAMTSizeEstimationDisabled:\n\t\treturn uio.SizeEstimationDisabled\n\tdefault:\n\t\treturn uio.SizeEstimationLinks\n\t}\n}\n\n// UnixFSSplitterFunc returns a SplitterGen function based on Import.UnixFSChunker.\n// The returned function creates a Splitter for the configured chunking strategy.\n// The chunker string is parsed once when this method is called, not on each use.\nfunc (i *Import) UnixFSSplitterFunc() chunk.SplitterGen {\n\tchunkerStr := i.UnixFSChunker.WithDefault(DefaultUnixFSChunker)\n\n\t// Parse size-based chunker (most common case) and return optimized generator\n\tif sizeStr, ok := strings.CutPrefix(chunkerStr, \"size-\"); ok {\n\t\tif size, err := strconv.ParseInt(sizeStr, 10, 64); err == nil && size > 0 {\n\t\t\treturn chunk.SizeSplitterGen(size)\n\t\t}\n\t}\n\n\t// For other chunker types (rabin, buzhash) or invalid config,\n\t// fall back to parsing per-use (these are rare cases)\n\treturn func(r io.Reader) chunk.Splitter {\n\t\ts, err := chunk.FromString(r, chunkerStr)\n\t\tif err != nil {\n\t\t\treturn chunk.DefaultSplitter(r)\n\t\t}\n\t\treturn s\n\t}\n}\n" }, { "path": "config/import_test.go", "content": "package config\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/ipld/unixfs/io\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nfunc TestValidateImportConfig_HAMTFanout(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfanout int64\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t// Valid values - powers of 2, multiples of 8, and <= 1024\n\t\t{name: \"valid 8\", fanout: 8, wantErr: false},\n\t\t{name: \"valid 16\", fanout: 16, wantErr: false},\n\t\t{name: \"valid 32\", fanout: 32, wantErr: false},\n\t\t{name: \"valid 64\", fanout: 64, wantErr: false},\n\t\t{name: \"valid 128\", fanout: 128, wantErr: false},\n\t\t{name: \"valid 256\", fanout: 256, wantErr: false},\n\t\t{name: \"valid 512\", fanout: 512, wantErr: false},\n\t\t{name: \"valid 1024\", fanout: 1024, wantErr: false},\n\n\t\t// Invalid values - not powers of 2\n\t\t{name: \"invalid 7\", fanout: 7, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 15\", fanout: 15, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 100\", fanout: 100, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 257\", fanout: 257, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 1000\", fanout: 1000, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\n\t\t// Invalid values - powers of 2 but less than 8\n\t\t{name: \"invalid 1\", fanout: 1, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 2\", fanout: 2, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 4\", fanout: 4, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\n\t\t// Invalid values - exceeds 1024\n\t\t{name: \"invalid 2048\", fanout: 2048, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid 4096\", fanout: 4096, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\n\t\t// Invalid values - negative or zero\n\t\t{name: \"invalid 0\", fanout: 0, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid -8\", fanout: -8, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t\t{name: \"invalid -256\", fanout: -256, wantErr: true, errMsg: \"must be a power of 2, between 8 and 1024\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tUnixFSHAMTDirectoryMaxFanout: *NewOptionalInteger(tt.fanout),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error for fanout=%d, got nil\", tt.fanout)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error for fanout=%d: %v\", tt.fanout, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_CidVersion(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcidVer int64\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid 0\", cidVer: 0, wantErr: false},\n\t\t{name: \"valid 1\", cidVer: 1, wantErr: false},\n\t\t{name: \"invalid 2\", cidVer: 2, wantErr: true, errMsg: \"must be 0 or 1\"},\n\t\t{name: \"invalid -1\", cidVer: -1, wantErr: true, errMsg: \"must be 0 or 1\"},\n\t\t{name: \"invalid 100\", cidVer: 100, wantErr: true, errMsg: \"must be 0 or 1\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tCidVersion: *NewOptionalInteger(tt.cidVer),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error for cidVer=%d, got nil\", tt.cidVer)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error for cidVer=%d: %v\", tt.cidVer, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_UnixFSFileMaxLinks(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmaxLinks int64\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid 1\", maxLinks: 1, wantErr: false},\n\t\t{name: \"valid 174\", maxLinks: 174, wantErr: false},\n\t\t{name: \"valid 1000\", maxLinks: 1000, wantErr: false},\n\t\t{name: \"invalid 0\", maxLinks: 0, wantErr: true, errMsg: \"must be positive\"},\n\t\t{name: \"invalid -1\", maxLinks: -1, wantErr: true, errMsg: \"must be positive\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tUnixFSFileMaxLinks: *NewOptionalInteger(tt.maxLinks),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error for maxLinks=%d, got nil\", tt.maxLinks)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error for maxLinks=%d: %v\", tt.maxLinks, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_UnixFSDirectoryMaxLinks(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmaxLinks int64\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid 0\", maxLinks: 0, wantErr: false}, // 0 means no limit\n\t\t{name: \"valid 1\", maxLinks: 1, wantErr: false},\n\t\t{name: \"valid 1000\", maxLinks: 1000, wantErr: false},\n\t\t{name: \"invalid -1\", maxLinks: -1, wantErr: true, errMsg: \"must be non-negative\"},\n\t\t{name: \"invalid -100\", maxLinks: -100, wantErr: true, errMsg: \"must be non-negative\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tUnixFSDirectoryMaxLinks: *NewOptionalInteger(tt.maxLinks),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error for maxLinks=%d, got nil\", tt.maxLinks)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error for maxLinks=%d: %v\", tt.maxLinks, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_BatchMax(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmaxNodes int64\n\t\tmaxSize int64\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid nodes 1\", maxNodes: 1, maxSize: -999, wantErr: false},\n\t\t{name: \"valid nodes 128\", maxNodes: 128, maxSize: -999, wantErr: false},\n\t\t{name: \"valid size 1\", maxNodes: -999, maxSize: 1, wantErr: false},\n\t\t{name: \"valid size 20MB\", maxNodes: -999, maxSize: 20 << 20, wantErr: false},\n\t\t{name: \"invalid nodes 0\", maxNodes: 0, maxSize: -999, wantErr: true, errMsg: \"BatchMaxNodes must be positive\"},\n\t\t{name: \"invalid nodes -1\", maxNodes: -1, maxSize: -999, wantErr: true, errMsg: \"BatchMaxNodes must be positive\"},\n\t\t{name: \"invalid size 0\", maxNodes: -999, maxSize: 0, wantErr: true, errMsg: \"BatchMaxSize must be positive\"},\n\t\t{name: \"invalid size -1\", maxNodes: -999, maxSize: -1, wantErr: true, errMsg: \"BatchMaxSize must be positive\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{}\n\t\t\tif tt.maxNodes != -999 {\n\t\t\t\tcfg.BatchMaxNodes = *NewOptionalInteger(tt.maxNodes)\n\t\t\t}\n\t\t\tif tt.maxSize != -999 {\n\t\t\t\tcfg.BatchMaxSize = *NewOptionalInteger(tt.maxSize)\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error, got nil\")\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error: %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_UnixFSChunker(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tchunker string\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid size-262144\", chunker: \"size-262144\", wantErr: false},\n\t\t{name: \"valid size-1\", chunker: \"size-1\", wantErr: false},\n\t\t{name: \"valid size-1048576\", chunker: \"size-1048576\", wantErr: false},\n\t\t{name: \"valid rabin\", chunker: \"rabin-128-256-512\", wantErr: false},\n\t\t{name: \"valid rabin min\", chunker: \"rabin-16-32-64\", wantErr: false},\n\t\t{name: \"valid buzhash\", chunker: \"buzhash\", wantErr: false},\n\t\t{name: \"invalid size-\", chunker: \"size-\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid size-abc\", chunker: \"size-abc\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid rabin-\", chunker: \"rabin-\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid rabin-128\", chunker: \"rabin-128\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid rabin-128-256\", chunker: \"rabin-128-256\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid rabin-a-b-c\", chunker: \"rabin-a-b-c\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid unknown\", chunker: \"unknown\", wantErr: true, errMsg: \"invalid format\"},\n\t\t{name: \"invalid empty\", chunker: \"\", wantErr: true, errMsg: \"invalid format\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tUnixFSChunker: *NewOptionalString(tt.chunker),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error for chunker=%s, got nil\", tt.chunker)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error for chunker=%s: %v\", tt.chunker, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_HashFunction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\thashFunc string\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid sha2-256\", hashFunc: \"sha2-256\", wantErr: false},\n\t\t{name: \"valid sha2-512\", hashFunc: \"sha2-512\", wantErr: false},\n\t\t{name: \"valid sha3-256\", hashFunc: \"sha3-256\", wantErr: false},\n\t\t{name: \"valid blake2b-256\", hashFunc: \"blake2b-256\", wantErr: false},\n\t\t{name: \"valid blake3\", hashFunc: \"blake3\", wantErr: false},\n\t\t{name: \"invalid unknown\", hashFunc: \"unknown-hash\", wantErr: true, errMsg: \"unrecognized\"},\n\t\t{name: \"invalid empty\", hashFunc: \"\", wantErr: true, errMsg: \"unrecognized\"},\n\t}\n\n\t// Check for hashes that exist but are not allowed\n\t// MD5 should exist but not be allowed\n\tif code, ok := mh.Names[\"md5\"]; ok {\n\t\ttests = append(tests, struct {\n\t\t\tname string\n\t\t\thashFunc string\n\t\t\twantErr bool\n\t\t\terrMsg string\n\t\t}{name: \"md5 not allowed\", hashFunc: \"md5\", wantErr: true, errMsg: \"not allowed\"})\n\t\t_ = code // use the variable\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tHashFunction: *NewOptionalString(tt.hashFunc),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() expected error for hashFunc=%s, got nil\", tt.hashFunc)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"ValidateImportConfig() unexpected error for hashFunc=%s: %v\", tt.hashFunc, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_DefaultValue(t *testing.T) {\n\t// Test that default (unset) value doesn't trigger validation\n\tcfg := &Import{}\n\n\terr := ValidateImportConfig(cfg)\n\tif err != nil {\n\t\tt.Errorf(\"ValidateImportConfig() unexpected error for default config: %v\", err)\n\t}\n}\n\nfunc TestIsValidChunker(t *testing.T) {\n\ttests := []struct {\n\t\tchunker string\n\t\twant bool\n\t}{\n\t\t{\"buzhash\", true},\n\t\t{\"size-262144\", true},\n\t\t{\"size-1\", true},\n\t\t{\"size-0\", false}, // 0 is not valid - must be positive\n\t\t{\"size-9999999\", true},\n\t\t{\"rabin-128-256-512\", true},\n\t\t{\"rabin-16-32-64\", true},\n\t\t{\"rabin-1-2-3\", true},\n\t\t{\"rabin-512-256-128\", false}, // Invalid ordering: min > avg > max\n\t\t{\"rabin-256-128-512\", false}, // Invalid ordering: min > avg\n\t\t{\"rabin-128-512-256\", false}, // Invalid ordering: avg > max\n\n\t\t{\"\", false},\n\t\t{\"size-\", false},\n\t\t{\"size-abc\", false},\n\t\t{\"size--1\", false},\n\t\t{\"rabin-\", false},\n\t\t{\"rabin-128\", false},\n\t\t{\"rabin-128-256\", false},\n\t\t{\"rabin-128-256-512-1024\", false},\n\t\t{\"rabin-a-b-c\", false},\n\t\t{\"unknown\", false},\n\t\t{\"buzzhash\", false}, // typo\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.chunker, func(t *testing.T) {\n\t\t\tif got := isValidChunker(tt.chunker); got != tt.want {\n\t\t\t\tt.Errorf(\"isValidChunker(%q) = %v, want %v\", tt.chunker, got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsPowerOfTwo(t *testing.T) {\n\ttests := []struct {\n\t\tn int64\n\t\twant bool\n\t}{\n\t\t{0, false},\n\t\t{1, true},\n\t\t{2, true},\n\t\t{3, false},\n\t\t{4, true},\n\t\t{5, false},\n\t\t{6, false},\n\t\t{7, false},\n\t\t{8, true},\n\t\t{16, true},\n\t\t{32, true},\n\t\t{64, true},\n\t\t{100, false},\n\t\t{128, true},\n\t\t{256, true},\n\t\t{512, true},\n\t\t{1024, true},\n\t\t{2048, true},\n\t\t{-1, false},\n\t\t{-8, false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(\"\", func(t *testing.T) {\n\t\t\tif got := isPowerOfTwo(tt.n); got != tt.want {\n\t\t\t\tt.Errorf(\"isPowerOfTwo(%d) = %v, want %v\", tt.n, got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_HAMTSizeEstimation(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvalue string\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid links\", value: HAMTSizeEstimationLinks, wantErr: false},\n\t\t{name: \"valid block\", value: HAMTSizeEstimationBlock, wantErr: false},\n\t\t{name: \"valid disabled\", value: HAMTSizeEstimationDisabled, wantErr: false},\n\t\t{name: \"invalid unknown\", value: \"unknown\", wantErr: true, errMsg: \"must be\"},\n\t\t{name: \"invalid empty\", value: \"\", wantErr: true, errMsg: \"must be\"},\n\t\t{name: \"invalid typo\", value: \"link\", wantErr: true, errMsg: \"must be\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tUnixFSHAMTDirectorySizeEstimation: *NewOptionalString(tt.value),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"expected error for value=%q, got nil\", tt.value)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"unexpected error for value=%q: %v\", tt.value, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateImportConfig_DAGLayout(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvalue string\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{name: \"valid balanced\", value: DAGLayoutBalanced, wantErr: false},\n\t\t{name: \"valid trickle\", value: DAGLayoutTrickle, wantErr: false},\n\t\t{name: \"invalid unknown\", value: \"unknown\", wantErr: true, errMsg: \"must be\"},\n\t\t{name: \"invalid empty\", value: \"\", wantErr: true, errMsg: \"must be\"},\n\t\t{name: \"invalid flat\", value: \"flat\", wantErr: true, errMsg: \"must be\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Import{\n\t\t\t\tUnixFSDAGLayout: *NewOptionalString(tt.value),\n\t\t\t}\n\n\t\t\terr := ValidateImportConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"expected error for value=%q, got nil\", tt.value)\n\t\t\t\t} else if tt.errMsg != \"\" && !strings.Contains(err.Error(), tt.errMsg) {\n\t\t\t\t\tt.Errorf(\"error = %v, want error containing %q\", err, tt.errMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"unexpected error for value=%q: %v\", tt.value, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestImport_HAMTSizeEstimationMode(t *testing.T) {\n\ttests := []struct {\n\t\tcfg string\n\t\twant io.SizeEstimationMode\n\t}{\n\t\t{HAMTSizeEstimationLinks, io.SizeEstimationLinks},\n\t\t{HAMTSizeEstimationBlock, io.SizeEstimationBlock},\n\t\t{HAMTSizeEstimationDisabled, io.SizeEstimationDisabled},\n\t\t{\"\", io.SizeEstimationLinks}, // default (unset returns default)\n\t\t{\"unknown\", io.SizeEstimationLinks}, // fallback to default\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.cfg, func(t *testing.T) {\n\t\t\tvar imp Import\n\t\t\tif tt.cfg != \"\" {\n\t\t\t\timp.UnixFSHAMTDirectorySizeEstimation = *NewOptionalString(tt.cfg)\n\t\t\t}\n\t\t\tgot := imp.HAMTSizeEstimationMode()\n\t\t\tif got != tt.want {\n\t\t\t\tt.Errorf(\"Import.HAMTSizeEstimationMode() with %q = %v, want %v\", tt.cfg, got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "config/init.go", "content": "package config\n\nimport (\n\t\"crypto/rand\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"io\"\n\t\"time\"\n\n\t\"github.com/cockroachdb/pebble/v2\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nfunc Init(out io.Writer, nBitsForKeypair int) (*Config, error) {\n\tidentity, err := CreateIdentity(out, []options.KeyGenerateOption{options.Key.Size(nBitsForKeypair)})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn InitWithIdentity(identity)\n}\n\nfunc InitWithIdentity(identity Identity) (*Config, error) {\n\tdatastore := DefaultDatastoreConfig()\n\n\tconf := &Config{\n\t\tAPI: API{\n\t\t\tHTTPHeaders: map[string][]string{},\n\t\t},\n\n\t\t// setup the node's default addresses.\n\t\t// NOTE: two swarm listen addrs, one tcp, one utp.\n\t\tAddresses: addressesConfig(),\n\n\t\tDatastore: datastore,\n\t\tBootstrap: []string{AutoPlaceholder},\n\t\tIdentity: identity,\n\t\tDiscovery: Discovery{\n\t\t\tMDNS: MDNS{\n\t\t\t\tEnabled: true,\n\t\t\t},\n\t\t},\n\n\t\t// setup the node mount points.\n\t\tMounts: Mounts{\n\t\t\tIPFS: \"/ipfs\",\n\t\t\tIPNS: \"/ipns\",\n\t\t\tMFS: \"/mfs\",\n\t\t},\n\n\t\tIpns: Ipns{\n\t\t\tResolveCacheSize: 128,\n\t\t\tDelegatedPublishers: []string{AutoPlaceholder},\n\t\t},\n\n\t\tGateway: Gateway{\n\t\t\tRootRedirect: \"\",\n\t\t\tNoFetch: false,\n\t\t\tHTTPHeaders: map[string][]string{},\n\t\t},\n\t\tPinning: Pinning{\n\t\t\tRemoteServices: map[string]RemotePinningService{},\n\t\t},\n\t\tDNS: DNS{\n\t\t\tResolvers: map[string]string{\n\t\t\t\t\".\": AutoPlaceholder,\n\t\t\t},\n\t\t},\n\t\tRouting: Routing{\n\t\t\tDelegatedRouters: []string{AutoPlaceholder},\n\t\t},\n\t}\n\n\treturn conf, nil\n}\n\n// DefaultConnMgrHighWater is the default value for the connection managers\n// 'high water' mark.\nconst DefaultConnMgrHighWater = 96\n\n// DefaultConnMgrLowWater is the default value for the connection managers 'low\n// water' mark.\nconst DefaultConnMgrLowWater = 32\n\n// DefaultConnMgrGracePeriod is the default value for the connection managers\n// grace period.\nconst DefaultConnMgrGracePeriod = time.Second * 20\n\n// DefaultConnMgrSilencePeriod controls how often the connection manager enforces the limits.\nconst DefaultConnMgrSilencePeriod = time.Second * 10\n\n// DefaultConnMgrType is the default value for the connection managers\n// type.\nconst DefaultConnMgrType = \"basic\"\n\n// DefaultResourceMgrMinInboundConns is a MAGIC number that probably a good\n// enough number of inbound conns to be a good network citizen.\nconst DefaultResourceMgrMinInboundConns = 800\n\nfunc addressesConfig() Addresses {\n\treturn Addresses{\n\t\tSwarm: []string{\n\t\t\t\"/ip4/0.0.0.0/tcp/4001\",\n\t\t\t\"/ip6/::/tcp/4001\",\n\t\t\t\"/ip4/0.0.0.0/udp/4001/webrtc-direct\",\n\t\t\t\"/ip4/0.0.0.0/udp/4001/quic-v1\",\n\t\t\t\"/ip4/0.0.0.0/udp/4001/quic-v1/webtransport\",\n\t\t\t\"/ip6/::/udp/4001/webrtc-direct\",\n\t\t\t\"/ip6/::/udp/4001/quic-v1\",\n\t\t\t\"/ip6/::/udp/4001/quic-v1/webtransport\",\n\t\t},\n\t\tAnnounce: []string{},\n\t\tAppendAnnounce: []string{},\n\t\tNoAnnounce: []string{},\n\t\tAPI: Strings{\"/ip4/127.0.0.1/tcp/5001\"},\n\t\tGateway: Strings{\"/ip4/127.0.0.1/tcp/8080\"},\n\t}\n}\n\n// DefaultDatastoreConfig is an internal function exported to aid in testing.\nfunc DefaultDatastoreConfig() Datastore {\n\treturn Datastore{\n\t\tStorageMax: \"10GB\",\n\t\tStorageGCWatermark: 90, // 90%\n\t\tGCPeriod: \"1h\",\n\t\tBloomFilterSize: 0,\n\t\tSpec: flatfsSpec(),\n\t}\n}\n\nfunc pebbleSpec() map[string]any {\n\treturn map[string]any{\n\t\t\"type\": \"pebbleds\",\n\t\t\"prefix\": \"pebble.datastore\",\n\t\t\"path\": \"pebbleds\",\n\t\t\"formatMajorVersion\": int(pebble.FormatNewest),\n\t}\n}\n\nfunc pebbleSpecMeasure() map[string]any {\n\treturn map[string]any{\n\t\t\"type\": \"measure\",\n\t\t\"prefix\": \"pebble.datastore\",\n\t\t\"child\": map[string]any{\n\t\t\t\"formatMajorVersion\": int(pebble.FormatNewest),\n\t\t\t\"type\": \"pebbleds\",\n\t\t\t\"path\": \"pebbleds\",\n\t\t},\n\t}\n}\n\nfunc badgerSpec() map[string]any {\n\treturn map[string]any{\n\t\t\"type\": \"badgerds\",\n\t\t\"prefix\": \"badger.datastore\",\n\t\t\"path\": \"badgerds\",\n\t\t\"syncWrites\": false,\n\t\t\"truncate\": true,\n\t}\n}\n\nfunc badgerSpecMeasure() map[string]any {\n\treturn map[string]any{\n\t\t\"type\": \"measure\",\n\t\t\"prefix\": \"badger.datastore\",\n\t\t\"child\": map[string]any{\n\t\t\t\"type\": \"badgerds\",\n\t\t\t\"path\": \"badgerds\",\n\t\t\t\"syncWrites\": false,\n\t\t\t\"truncate\": true,\n\t\t},\n\t}\n}\n\nfunc flatfsSpec() map[string]any {\n\treturn map[string]any{\n\t\t\"type\": \"mount\",\n\t\t\"mounts\": []any{\n\t\t\tmap[string]any{\n\t\t\t\t\"mountpoint\": \"/blocks\",\n\t\t\t\t\"type\": \"flatfs\",\n\t\t\t\t\"prefix\": \"flatfs.datastore\",\n\t\t\t\t\"path\": \"blocks\",\n\t\t\t\t\"sync\": false,\n\t\t\t\t\"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n\t\t\t},\n\t\t\tmap[string]any{\n\t\t\t\t\"mountpoint\": \"/\",\n\t\t\t\t\"type\": \"levelds\",\n\t\t\t\t\"prefix\": \"leveldb.datastore\",\n\t\t\t\t\"path\": \"datastore\",\n\t\t\t\t\"compression\": \"none\",\n\t\t\t},\n\t\t},\n\t}\n}\n\nfunc flatfsSpecMeasure() map[string]any {\n\treturn map[string]any{\n\t\t\"type\": \"mount\",\n\t\t\"mounts\": []any{\n\t\t\tmap[string]any{\n\t\t\t\t\"mountpoint\": \"/blocks\",\n\t\t\t\t\"type\": \"measure\",\n\t\t\t\t\"prefix\": \"flatfs.datastore\",\n\t\t\t\t\"child\": map[string]any{\n\t\t\t\t\t\"type\": \"flatfs\",\n\t\t\t\t\t\"path\": \"blocks\",\n\t\t\t\t\t\"sync\": false,\n\t\t\t\t\t\"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tmap[string]any{\n\t\t\t\t\"mountpoint\": \"/\",\n\t\t\t\t\"type\": \"measure\",\n\t\t\t\t\"prefix\": \"leveldb.datastore\",\n\t\t\t\t\"child\": map[string]any{\n\t\t\t\t\t\"type\": \"levelds\",\n\t\t\t\t\t\"path\": \"datastore\",\n\t\t\t\t\t\"compression\": \"none\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n}\n\n// CreateIdentity initializes a new identity.\nfunc CreateIdentity(out io.Writer, opts []options.KeyGenerateOption) (Identity, error) {\n\t// TODO guard higher up\n\tident := Identity{}\n\n\tsettings, err := options.KeyGenerateOptions(opts...)\n\tif err != nil {\n\t\treturn ident, err\n\t}\n\n\tvar sk crypto.PrivKey\n\tvar pk crypto.PubKey\n\n\tswitch settings.Algorithm {\n\tcase \"rsa\":\n\t\tif settings.Size == -1 {\n\t\t\tsettings.Size = options.DefaultRSALen\n\t\t}\n\n\t\tfmt.Fprintf(out, \"generating %d-bit RSA keypair...\", settings.Size)\n\n\t\tpriv, pub, err := crypto.GenerateKeyPair(crypto.RSA, settings.Size)\n\t\tif err != nil {\n\t\t\treturn ident, err\n\t\t}\n\n\t\tsk = priv\n\t\tpk = pub\n\tcase \"ed25519\":\n\t\tif settings.Size != -1 {\n\t\t\treturn ident, fmt.Errorf(\"number of key bits does not apply when using ed25519 keys\")\n\t\t}\n\t\tfmt.Fprintf(out, \"generating ED25519 keypair...\")\n\t\tpriv, pub, err := crypto.GenerateEd25519Key(rand.Reader)\n\t\tif err != nil {\n\t\t\treturn ident, err\n\t\t}\n\n\t\tsk = priv\n\t\tpk = pub\n\tdefault:\n\t\treturn ident, fmt.Errorf(\"unrecognized key type: %s\", settings.Algorithm)\n\t}\n\tfmt.Fprintf(out, \"done\\n\")\n\n\t// currently storing key unencrypted. in the future we need to encrypt it.\n\t// TODO(security)\n\tskbytes, err := crypto.MarshalPrivateKey(sk)\n\tif err != nil {\n\t\treturn ident, err\n\t}\n\tident.PrivKey = base64.StdEncoding.EncodeToString(skbytes)\n\n\tid, err := peer.IDFromPublicKey(pk)\n\tif err != nil {\n\t\treturn ident, err\n\t}\n\tident.PeerID = id.String()\n\tfmt.Fprintf(out, \"peer identity: %s\\n\", ident.PeerID)\n\treturn ident, nil\n}\n" }, { "path": "config/init_test.go", "content": "package config\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\tcrypto_pb \"github.com/libp2p/go-libp2p/core/crypto/pb\"\n)\n\nfunc TestCreateIdentity(t *testing.T) {\n\twriter := bytes.NewBuffer(nil)\n\tid, err := CreateIdentity(writer, []options.KeyGenerateOption{options.Key.Type(options.Ed25519Key)})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tpk, err := id.DecodePrivateKey(\"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif pk.Type() != crypto_pb.KeyType_Ed25519 {\n\t\tt.Fatal(\"unexpected type:\", pk.Type())\n\t}\n\n\tid, err = CreateIdentity(writer, []options.KeyGenerateOption{options.Key.Type(options.RSAKey)})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tpk, err = id.DecodePrivateKey(\"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif pk.Type() != crypto_pb.KeyType_RSA {\n\t\tt.Fatal(\"unexpected type:\", pk.Type())\n\t}\n}\n\nfunc TestCreateIdentityOptions(t *testing.T) {\n\tvar w bytes.Buffer\n\n\t// ed25519 keys with bit size must fail.\n\t_, err := CreateIdentity(&w, []options.KeyGenerateOption{\n\t\toptions.Key.Type(options.Ed25519Key),\n\t\toptions.Key.Size(2048),\n\t})\n\tif err == nil {\n\t\tt.Errorf(\"ed25519 keys cannot have a custom bit size\")\n\t}\n}\n" }, { "path": "config/internal.go", "content": "package config\n\nconst (\n\t// DefaultMFSNoFlushLimit is the default limit for consecutive unflushed MFS operations\n\tDefaultMFSNoFlushLimit = 256\n)\n\ntype Internal struct {\n\t// All marked as omitempty since we are expecting to make changes to all subcomponents of Internal\n\tBitswap *InternalBitswap `json:\",omitempty\"`\n\tUnixFSShardingSizeThreshold *OptionalString `json:\",omitempty\"` // moved to Import.UnixFSHAMTDirectorySizeThreshold\n\tLibp2pForceReachability *OptionalString `json:\",omitempty\"`\n\tBackupBootstrapInterval *OptionalDuration `json:\",omitempty\"`\n\t// MFSNoFlushLimit controls the maximum number of consecutive\n\t// MFS operations allowed with --flush=false before requiring a manual flush.\n\t// This prevents unbounded memory growth and ensures data consistency.\n\t// Set to 0 to disable limiting (old behavior, may cause high memory usage)\n\t// This is an EXPERIMENTAL feature and may change or be removed in future releases.\n\t// See https://github.com/ipfs/kubo/issues/10842\n\tMFSNoFlushLimit *OptionalInteger `json:\",omitempty\"`\n}\n\ntype InternalBitswap struct {\n\tTaskWorkerCount OptionalInteger\n\tEngineBlockstoreWorkerCount OptionalInteger\n\tEngineTaskWorkerCount OptionalInteger\n\tMaxOutstandingBytesPerPeer OptionalInteger\n\tProviderSearchDelay OptionalDuration\n\tProviderSearchMaxResults OptionalInteger\n\tWantHaveReplaceSize OptionalInteger\n\tBroadcastControl *BitswapBroadcastControl\n}\n\ntype BitswapBroadcastControl struct {\n\t// EnableEnables or disables broadcast control functionality. Setting this\n\t// to false disables broadcast control functionality and restores the\n\t// previous broadcast behavior of sending broadcasts to all peers. When\n\t// disabled, all other BroadcastControl configuration items are ignored.\n\t// Default is [DefaultBroadcastControlEnable].\n\tEnable Flag `json:\",omitempty\"`\n\t// MaxPeers sets a hard limit on the number of peers to send broadcasts to.\n\t// A value of 0 means no broadcasts are sent. A value of -1 means there is\n\t// no limit. Default is [DefaultBroadcastControlMaxPeers].\n\tMaxPeers OptionalInteger\n\t// LocalPeers enables or disables broadcast control for peers on the local\n\t// network. If false, than always broadcast to peers on the local network.\n\t// If true, apply broadcast control to local peers. Default is\n\t// [DefaultBroadcastControlLocalPeers].\n\tLocalPeers Flag `json:\",omitempty\"`\n\t// PeeredPeers enables or disables broadcast reduction for peers configured\n\t// for peering. If false, than always broadcast to peers configured for\n\t// peering. If true, apply broadcast reduction to peered peers. Default is\n\t// [DefaultBroadcastControlPeeredPeers].\n\tPeeredPeers Flag `json:\",omitempty\"`\n\t// MaxRandomPeers is the number of peers to broadcast to anyway, even\n\t// though broadcast reduction logic has determined that they are not\n\t// broadcast targets. Setting this to a non-zero value ensures at least\n\t// this number of random peers receives a broadcast. This may be helpful in\n\t// cases where peers that are not receiving broadcasts my have wanted\n\t// blocks. Default is [DefaultBroadcastControlMaxRandomPeers].\n\tMaxRandomPeers OptionalInteger\n\t// SendToPendingPeers enables or disables sending broadcasts to any peers\n\t// to which there is a pending message to send. When enabled, this sends\n\t// broadcasts to many more peers, but does so in a way that does not\n\t// increase the number of separate broadcast messages. There is still the\n\t// increased cost of the recipients having to process and respond to the\n\t// broadcasts. Default is [DefaultBroadcastControlSendToPendingPeers].\n\tSendToPendingPeers Flag `json:\",omitempty\"`\n}\n\nconst (\n\tDefaultBroadcastControlEnable = true // Enabled\n\tDefaultBroadcastControlMaxPeers = -1 // Unlimited\n\tDefaultBroadcastControlLocalPeers = false // No control of local\n\tDefaultBroadcastControlPeeredPeers = false // No control of peered\n\tDefaultBroadcastControlMaxRandomPeers = 0 // No randoms\n\tDefaultBroadcastControlSendToPendingPeers = false // Disabled\n)\n" }, { "path": "config/ipns.go", "content": "package config\n\nimport (\n\t\"math\"\n\t\"time\"\n)\n\nconst (\n\tDefaultIpnsMaxCacheTTL = time.Duration(math.MaxInt64)\n)\n\ntype Ipns struct {\n\tRepublishPeriod string\n\tRecordLifetime string\n\n\tResolveCacheSize int\n\n\t// MaxCacheTTL is the maximum duration IPNS entries are valid in the cache.\n\tMaxCacheTTL *OptionalDuration `json:\",omitempty\"`\n\n\t// Enable namesys pubsub (--enable-namesys-pubsub)\n\tUsePubsub Flag `json:\",omitempty\"`\n\n\t// Simplified configuration for delegated IPNS publishers\n\tDelegatedPublishers []string\n}\n" }, { "path": "config/migration.go", "content": "package config\n\nconst DefaultMigrationKeep = \"cache\"\n\n// DefaultMigrationDownloadSources defines the default download sources for legacy migrations (repo versions <16).\n// Only HTTPS is supported for legacy migrations. IPFS downloads are not supported.\nvar DefaultMigrationDownloadSources = []string{\"HTTPS\"}\n\n// Migration configures how legacy migrations are downloaded (repo versions <16).\n//\n// DEPRECATED: This configuration only applies to legacy external migrations for repository\n// versions below 16. Modern repositories (v16+) use embedded migrations that do not require\n// external downloads. These settings will be ignored for modern repository versions.\ntype Migration struct {\n\t// DEPRECATED: This field is deprecated and ignored for modern repositories (repo versions ≥16).\n\tDownloadSources []string `json:\",omitempty\"`\n\t// DEPRECATED: This field is deprecated and ignored for modern repositories (repo versions ≥16).\n\tKeep string `json:\",omitempty\"`\n}\n" }, { "path": "config/migration_test.go", "content": "package config\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n)\n\nfunc TestMigrationDecode(t *testing.T) {\n\tstr := `\n\t\t{\n\t\t\t\"DownloadSources\": [\"IPFS\", \"HTTP\", \"127.0.0.1\"],\n\t\t\t\"Keep\": \"cache\"\n\t\t}\n\t`\n\n\tvar cfg Migration\n\tif err := json.Unmarshal([]byte(str), &cfg); err != nil {\n\t\tt.Errorf(\"failed while unmarshalling migration struct: %s\", err)\n\t}\n\n\tif len(cfg.DownloadSources) != 3 {\n\t\tt.Fatal(\"wrong number of DownloadSources\")\n\t}\n\texpect := []string{\"IPFS\", \"HTTP\", \"127.0.0.1\"}\n\tfor i := range expect {\n\t\tif cfg.DownloadSources[i] != expect[i] {\n\t\t\tt.Errorf(\"wrong DownloadSource at %d\", i)\n\t\t}\n\t}\n\n\tif cfg.Keep != \"cache\" {\n\t\tt.Error(\"wrong value for Keep\")\n\t}\n}\n" }, { "path": "config/mounts.go", "content": "package config\n\n// Mounts stores the (string) mount points.\ntype Mounts struct {\n\tIPFS string\n\tIPNS string\n\tMFS string\n\tFuseAllowOther bool\n}\n" }, { "path": "config/peering.go", "content": "package config\n\nimport \"github.com/libp2p/go-libp2p/core/peer\"\n\n// Peering configures the peering service.\ntype Peering struct {\n\t// Peers lists the nodes to attempt to stay connected with.\n\tPeers []peer.AddrInfo\n}\n" }, { "path": "config/plugins.go", "content": "package config\n\ntype Plugins struct {\n\tPlugins map[string]Plugin\n\t// TODO: Loader Path? Leaving that out for now due to security concerns.\n}\n\ntype Plugin struct {\n\tDisabled bool\n\tConfig any `json:\",omitempty\"`\n}\n" }, { "path": "config/profile.go", "content": "package config\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"time\"\n)\n\n// Transformer is a function which takes configuration and applies some filter to it.\ntype Transformer func(c *Config) error\n\n// Profile contains the profile transformer the description of the profile.\ntype Profile struct {\n\t// Description briefly describes the functionality of the profile.\n\tDescription string\n\n\t// Transform takes ipfs configuration and applies the profile to it.\n\tTransform Transformer\n\n\t// InitOnly specifies that this profile can only be applied on init.\n\tInitOnly bool\n}\n\n// defaultServerFilters has is a list of IPv4 and IPv6 prefixes that are private, local only, or unrouteable.\n// according to https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml\n// and https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml\nvar defaultServerFilters = []string{\n\t\"/ip4/10.0.0.0/ipcidr/8\",\n\t\"/ip4/100.64.0.0/ipcidr/10\",\n\t\"/ip4/169.254.0.0/ipcidr/16\",\n\t\"/ip4/172.16.0.0/ipcidr/12\",\n\t\"/ip4/192.0.0.0/ipcidr/24\",\n\t\"/ip4/192.0.2.0/ipcidr/24\",\n\t\"/ip4/192.168.0.0/ipcidr/16\",\n\t\"/ip4/198.18.0.0/ipcidr/15\",\n\t\"/ip4/198.51.100.0/ipcidr/24\",\n\t\"/ip4/203.0.113.0/ipcidr/24\",\n\t\"/ip4/240.0.0.0/ipcidr/4\",\n\t\"/ip6/100::/ipcidr/64\",\n\t\"/ip6/2001:2::/ipcidr/48\",\n\t\"/ip6/2001:db8::/ipcidr/32\",\n\t\"/ip6/fc00::/ipcidr/7\",\n\t\"/ip6/fe80::/ipcidr/10\",\n}\n\n// Profiles is a map holding configuration transformers. Docs are in docs/config.md.\nvar Profiles = map[string]Profile{\n\t\"server\": {\n\t\tDescription: `Disables local host discovery, recommended when\nrunning IPFS on machines with public IPv4 addresses.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Addresses.NoAnnounce = appendSingle(c.Addresses.NoAnnounce, defaultServerFilters)\n\t\t\tc.Swarm.AddrFilters = appendSingle(c.Swarm.AddrFilters, defaultServerFilters)\n\t\t\tc.Discovery.MDNS.Enabled = false\n\t\t\tc.Swarm.DisableNatPortMap = true\n\t\t\treturn nil\n\t\t},\n\t},\n\n\t\"local-discovery\": {\n\t\tDescription: `Sets default values to fields affected by the server\nprofile, enables discovery in local networks.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Addresses.NoAnnounce = deleteEntries(c.Addresses.NoAnnounce, defaultServerFilters)\n\t\t\tc.Swarm.AddrFilters = deleteEntries(c.Swarm.AddrFilters, defaultServerFilters)\n\t\t\tc.Discovery.MDNS.Enabled = true\n\t\t\tc.Swarm.DisableNatPortMap = false\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"test\": {\n\t\tDescription: `Reduces external interference of IPFS daemon, this\nis useful when using the daemon in test environments.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Addresses.API = Strings{\"/ip4/127.0.0.1/tcp/0\"}\n\t\t\tc.Addresses.Gateway = Strings{\"/ip4/127.0.0.1/tcp/0\"}\n\t\t\tc.Addresses.Swarm = []string{\n\t\t\t\t\"/ip4/127.0.0.1/tcp/0\",\n\t\t\t}\n\n\t\t\tc.Swarm.DisableNatPortMap = true\n\t\t\tc.Routing.LoopbackAddressesOnLanDHT = True\n\n\t\t\tc.Bootstrap = []string{}\n\t\t\tc.Discovery.MDNS.Enabled = false\n\t\t\tc.AutoTLS.Enabled = False\n\t\t\tc.AutoConf.Enabled = False\n\n\t\t\t// Explicitly set autoconf-controlled fields to empty when autoconf is disabled\n\t\t\tc.DNS.Resolvers = map[string]string{}\n\t\t\tc.Routing.DelegatedRouters = []string{}\n\t\t\tc.Ipns.DelegatedPublishers = []string{}\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"default-networking\": {\n\t\tDescription: `Restores default network settings.\nInverse profile of the test profile.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Addresses = addressesConfig()\n\n\t\t\t// Use AutoConf system for bootstrap peers\n\t\t\tc.Bootstrap = []string{AutoPlaceholder}\n\t\t\tc.AutoConf.Enabled = Default\n\t\t\tc.AutoConf.URL = nil // Clear URL to use implicit default\n\n\t\t\tc.Swarm.DisableNatPortMap = false\n\t\t\tc.Discovery.MDNS.Enabled = true\n\t\t\tc.AutoTLS.Enabled = Default\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"default-datastore\": {\n\t\tDescription: `Configures the node to use the default datastore (flatfs).\n\nRead the \"flatfs\" profile description for more information on this datastore.\n\nThis profile may only be applied when first initializing the node.\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = flatfsSpec()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"flatfs\": {\n\t\tDescription: `Configures the node to use the flatfs datastore.\n\nThis is the most battle-tested and reliable datastore.\nYou should use this datastore if:\n\n* You need a very simple and very reliable datastore, and you trust your\n filesystem. This datastore stores each block as a separate file in the\n underlying filesystem so it's unlikely to loose data unless there's an issue\n with the underlying file system.\n* You need to run garbage collection in a way that reclaims free space as soon as possible.\n* You want to minimize memory usage.\n* You are ok with the default speed of data import, or prefer to use --nocopy.\n\nSee configuration documentation at:\nhttps://github.com/ipfs/kubo/blob/master/docs/datastores.md#flatfs\n\nNOTE: This profile may only be applied when first initializing node at IPFS_PATH\n via 'ipfs init --profile flatfs'\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = flatfsSpec()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"flatfs-measure\": {\n\t\tDescription: `Configures the node to use the flatfs datastore with metrics tracking wrapper.\nAdditional '*_datastore_*' metrics will be exposed on /debug/metrics/prometheus\n\nNOTE: This profile may only be applied when first initializing node at IPFS_PATH\n via 'ipfs init --profile flatfs-measure'\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = flatfsSpecMeasure()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"pebbleds\": {\n\t\tDescription: `Configures the node to use the pebble high-performance datastore.\n\nPebble is a LevelDB/RocksDB inspired key-value store focused on performance\nand internal usage by CockroachDB.\nYou should use this datastore if:\n\n- You need a datastore that is focused on performance.\n- You need reliability by default, but may choose to disable WAL for maximum performance when reliability is not critical.\n- This datastore is good for multi-terabyte data sets.\n- May benefit from tuning depending on read/write patterns and throughput.\n- Performance is helped significantly by running on a system with plenty of memory.\n\nSee configuration documentation at:\nhttps://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds\n\nNOTE: This profile may only be applied when first initializing node at IPFS_PATH\n via 'ipfs init --profile pebbleds'\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = pebbleSpec()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"pebbleds-measure\": {\n\t\tDescription: `Configures the node to use the pebble datastore with metrics tracking wrapper.\nAdditional '*_datastore_*' metrics will be exposed on /debug/metrics/prometheus\n\nNOTE: This profile may only be applied when first initializing node at IPFS_PATH\n via 'ipfs init --profile pebbleds-measure'\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = pebbleSpecMeasure()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"badgerds\": {\n\t\tDescription: `DEPRECATED: Configures the node to use the legacy badgerv1 datastore.\nThis profile will be removed in a future Kubo release.\nNew deployments should use 'flatfs' or 'pebbleds' instead.\n\nNOTE: this is badger 1.x, which has known bugs and is no longer supported by the upstream team.\nIt is provided here only for pre-existing users, allowing them to migrate away to more modern datastore.\n\nOther caveats:\n\n* This datastore will not properly reclaim space when your datastore is\n smaller than several gigabytes. If you run IPFS with --enable-gc, you plan\n on storing very little data in your IPFS node, and disk usage is more\n critical than performance, consider using flatfs.\n* This datastore uses up to several gigabytes of memory.\n* Good for medium-size datastores, but may run into performance issues\n if your dataset is bigger than a terabyte.\n\nTo migrate: create a new IPFS_PATH with 'ipfs init --profile=flatfs',\nmove pinned data via 'ipfs dag export/import' or 'ipfs pin ls -t recursive|add',\nand decommission the old badger-based node.\nWhen it comes to block storage, use experimental 'pebbleds' only if you are sure\nmodern 'flatfs' does not serve your use case (most users will be perfectly fine\nwith flatfs, it is also possible to keep flatfs for blocks and replace leveldb\nwith pebble if preferred over leveldb).\n\nSee configuration documentation at:\nhttps://github.com/ipfs/kubo/blob/master/docs/datastores.md#badgerds\n\nNOTE: This profile may only be applied when first initializing node at IPFS_PATH\n via 'ipfs init --profile badgerds'\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = badgerSpec()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"badgerds-measure\": {\n\t\tDescription: `DEPRECATED: Configures the node to use the legacy badgerv1 datastore with metrics wrapper.\nThis profile will be removed in a future Kubo release.\nNew deployments should use 'flatfs' or 'pebbleds' instead.\n\nNOTE: This profile may only be applied when first initializing node at IPFS_PATH\n via 'ipfs init --profile badgerds-measure'\n`,\n\n\t\tInitOnly: true,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Datastore.Spec = badgerSpecMeasure()\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"lowpower\": {\n\t\tDescription: `Reduces daemon overhead on the system. May affect node\nfunctionality - performance of content discovery and data\nfetching may be degraded.\n`,\n\t\tTransform: func(c *Config) error {\n\t\t\t// Disable \"server\" services (dht, autonat, limited relay)\n\t\t\tc.Routing.Type = NewOptionalString(\"autoclient\")\n\t\t\tc.AutoNAT.ServiceMode = AutoNATServiceDisabled\n\t\t\tc.Swarm.RelayService.Enabled = False\n\n\t\t\t// Keep bare minimum connections around\n\t\t\tlowWater := int64(20)\n\t\t\thighWater := int64(40)\n\t\t\tgracePeriod := time.Minute\n\t\t\tc.Swarm.ConnMgr.Type = NewOptionalString(\"basic\")\n\t\t\tc.Swarm.ConnMgr.LowWater = &OptionalInteger{value: &lowWater}\n\t\t\tc.Swarm.ConnMgr.HighWater = &OptionalInteger{value: &highWater}\n\t\t\tc.Swarm.ConnMgr.GracePeriod = &OptionalDuration{&gracePeriod}\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"announce-off\": {\n\t\tDescription: `Disables Provide system (announcing to Amino DHT).\n\n\t\tUSE WITH CAUTION:\n\t\tThe main use case for this is setups with manual Peering.Peers config.\n\t\tData from this node will not be announced on the DHT. This will make\n\t\tDHT-based routing and data retrieval impossible if this node is the only\n\t\tone hosting it, and other peers are not already connected to it.\n`,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Provide.Enabled = False\n\t\t\tc.Provide.DHT.Interval = NewOptionalDuration(0) // 0 disables periodic reprovide\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"announce-on\": {\n\t\tDescription: `Re-enables Provide system (reverts announce-off profile).`,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Provide.Enabled = True\n\t\t\tc.Provide.DHT.Interval = NewOptionalDuration(DefaultProvideDHTInterval) // have to apply explicit default because nil would be ignored\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"randomports\": {\n\t\tDescription: `Use a random port number for swarm.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tport, err := getAvailablePort()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tc.Addresses.Swarm = []string{\n\t\t\t\tfmt.Sprintf(\"/ip4/0.0.0.0/tcp/%d\", port),\n\t\t\t\tfmt.Sprintf(\"/ip6/::/tcp/%d\", port),\n\t\t\t}\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"unixfs-v0-2015\": {\n\t\tDescription: `Legacy UnixFS import profile for backward-compatible CID generation.\nProduces CIDv0 with no raw leaves, sha2-256, 256 KiB chunks, and\nlink-based HAMT size estimation. Use only when legacy CIDs are required.\nSee https://specs.ipfs.tech/ipips/ipip-0499/. Alias: legacy-cid-v0`,\n\t\tTransform: applyUnixFSv02015,\n\t},\n\t\"legacy-cid-v0\": {\n\t\tDescription: `Alias for unixfs-v0-2015 profile.`,\n\t\tTransform: applyUnixFSv02015,\n\t},\n\t\"unixfs-v1-2025\": {\n\t\tDescription: `Recommended UnixFS import profile for cross-implementation CID determinism.\nUses CIDv1, raw leaves, sha2-256, 1 MiB chunks, 1024 links per file node,\n256 HAMT fanout, and block-based size estimation for HAMT threshold.\nSee https://specs.ipfs.tech/ipips/ipip-0499/`,\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Import.CidVersion = *NewOptionalInteger(1)\n\t\t\tc.Import.UnixFSRawLeaves = True\n\t\t\tc.Import.UnixFSChunker = *NewOptionalString(\"size-1048576\") // 1 MiB\n\t\t\tc.Import.HashFunction = *NewOptionalString(\"sha2-256\")\n\t\t\tc.Import.UnixFSFileMaxLinks = *NewOptionalInteger(1024)\n\t\t\tc.Import.UnixFSDirectoryMaxLinks = *NewOptionalInteger(0)\n\t\t\tc.Import.UnixFSHAMTDirectoryMaxFanout = *NewOptionalInteger(256)\n\t\t\tc.Import.UnixFSHAMTDirectorySizeThreshold = *NewOptionalBytes(\"256KiB\")\n\t\t\tc.Import.UnixFSHAMTDirectorySizeEstimation = *NewOptionalString(HAMTSizeEstimationBlock)\n\t\t\tc.Import.UnixFSDAGLayout = *NewOptionalString(DAGLayoutBalanced)\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"autoconf-on\": {\n\t\tDescription: `Sets configuration to use implicit defaults from remote autoconf service.\nBootstrap peers, DNS resolvers, delegated routers, and IPNS delegated publishers are set to \"auto\".\nThis profile requires AutoConf to be enabled and configured.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Bootstrap = []string{AutoPlaceholder}\n\t\t\tc.DNS.Resolvers = map[string]string{\n\t\t\t\t\".\": AutoPlaceholder,\n\t\t\t}\n\t\t\tc.Routing.DelegatedRouters = []string{AutoPlaceholder}\n\t\t\tc.Ipns.DelegatedPublishers = []string{AutoPlaceholder}\n\t\t\tc.AutoConf.Enabled = True\n\t\t\tif c.AutoConf.URL == nil {\n\t\t\t\tc.AutoConf.URL = NewOptionalString(DefaultAutoConfURL)\n\t\t\t}\n\t\t\treturn nil\n\t\t},\n\t},\n\t\"autoconf-off\": {\n\t\tDescription: `Disables AutoConf and sets networking fields to empty for manual configuration.\nBootstrap peers, DNS resolvers, delegated routers, and IPNS delegated publishers are set to empty.\nUse this when you want normal networking but prefer manual control over all endpoints.`,\n\n\t\tTransform: func(c *Config) error {\n\t\t\tc.Bootstrap = nil\n\t\t\tc.DNS.Resolvers = nil\n\t\t\tc.Routing.DelegatedRouters = nil\n\t\t\tc.Ipns.DelegatedPublishers = nil\n\t\t\tc.AutoConf.Enabled = False\n\t\t\treturn nil\n\t\t},\n\t},\n}\n\nfunc getAvailablePort() (port int, err error) {\n\tln, err := net.Listen(\"tcp\", \"[::]:0\")\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer ln.Close()\n\tport = ln.Addr().(*net.TCPAddr).Port\n\treturn port, nil\n}\n\nfunc appendSingle(a []string, b []string) []string {\n\tout := make([]string, 0, len(a)+len(b))\n\tm := map[string]bool{}\n\tfor _, f := range a {\n\t\tif !m[f] {\n\t\t\tout = append(out, f)\n\t\t}\n\t\tm[f] = true\n\t}\n\tfor _, f := range b {\n\t\tif !m[f] {\n\t\t\tout = append(out, f)\n\t\t}\n\t\tm[f] = true\n\t}\n\treturn out\n}\n\nfunc deleteEntries(arr []string, del []string) []string {\n\tm := map[string]struct{}{}\n\tfor _, f := range arr {\n\t\tm[f] = struct{}{}\n\t}\n\tfor _, f := range del {\n\t\tdelete(m, f)\n\t}\n\treturn mapKeys(m)\n}\n\nfunc mapKeys(m map[string]struct{}) []string {\n\tout := make([]string, 0, len(m))\n\tfor f := range m {\n\t\tout = append(out, f)\n\t}\n\treturn out\n}\n\n// applyUnixFSv02015 applies the legacy UnixFS v0 (2015) import settings.\nfunc applyUnixFSv02015(c *Config) error {\n\tc.Import.CidVersion = *NewOptionalInteger(0)\n\tc.Import.UnixFSRawLeaves = False\n\tc.Import.UnixFSChunker = *NewOptionalString(\"size-262144\") // 256 KiB\n\tc.Import.HashFunction = *NewOptionalString(\"sha2-256\")\n\tc.Import.UnixFSFileMaxLinks = *NewOptionalInteger(174)\n\tc.Import.UnixFSDirectoryMaxLinks = *NewOptionalInteger(0)\n\tc.Import.UnixFSHAMTDirectoryMaxFanout = *NewOptionalInteger(256)\n\tc.Import.UnixFSHAMTDirectorySizeThreshold = *NewOptionalBytes(\"256KiB\")\n\tc.Import.UnixFSHAMTDirectorySizeEstimation = *NewOptionalString(HAMTSizeEstimationLinks)\n\tc.Import.UnixFSDAGLayout = *NewOptionalString(DAGLayoutBalanced)\n\treturn nil\n}\n" }, { "path": "config/provide.go", "content": "package config\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/libp2p/go-libp2p-kad-dht/amino\"\n)\n\nconst (\n\tDefaultProvideEnabled = true\n\tDefaultProvideStrategy = \"all\"\n\n\t// DHT provider defaults\n\tDefaultProvideDHTInterval = 22 * time.Hour // https://github.com/ipfs/kubo/pull/9326\n\tDefaultProvideDHTMaxWorkers = 16 // Unified default for both sweep and legacy providers\n\tDefaultProvideDHTSweepEnabled = true\n\tDefaultProvideDHTResumeEnabled = true\n\tDefaultProvideDHTDedicatedPeriodicWorkers = 2\n\tDefaultProvideDHTDedicatedBurstWorkers = 1\n\tDefaultProvideDHTMaxProvideConnsPerWorker = 20\n\tDefaultProvideDHTKeystoreBatchSize = 1 << 14 // ~544 KiB per batch (1 multihash = 34 bytes)\n\tDefaultProvideDHTOfflineDelay = 2 * time.Hour\n\n\t// DefaultFastProvideTimeout is the maximum time allowed for fast-provide operations.\n\t// Prevents hanging on network issues when providing root CID.\n\t// 10 seconds is sufficient for DHT operations with sweep provider or accelerated client.\n\tDefaultFastProvideTimeout = 10 * time.Second\n)\n\ntype ProvideStrategy int\n\nconst (\n\tProvideStrategyAll ProvideStrategy = 1 << iota\n\tProvideStrategyPinned\n\tProvideStrategyRoots\n\tProvideStrategyMFS\n)\n\n// Provide configures both immediate CID announcements (provide operations) for new content\n// and periodic re-announcements of existing CIDs (reprovide operations).\n// This section combines the functionality previously split between Provider and Reprovider.\ntype Provide struct {\n\t// Enabled controls whether both provide and reprovide systems are enabled.\n\t// When disabled, the node will not announce any content to the routing system.\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// Strategy determines which CIDs are announced to the routing system.\n\t// Default: DefaultProvideStrategy\n\tStrategy *OptionalString `json:\",omitempty\"`\n\n\t// DHT configures DHT-specific provide and reprovide settings.\n\tDHT ProvideDHT\n}\n\n// ProvideDHT configures DHT provider settings for both immediate announcements\n// and periodic reprovides.\ntype ProvideDHT struct {\n\t// Interval sets the time between rounds of reproviding local content\n\t// to the routing system. Set to \"0\" to disable content reproviding.\n\t// Default: DefaultProvideDHTInterval\n\tInterval *OptionalDuration `json:\",omitempty\"`\n\n\t// MaxWorkers sets the maximum number of concurrent workers for provide operations.\n\t// When SweepEnabled is false: controls NEW CID announcements only.\n\t// When SweepEnabled is true: controls total worker pool for all operations.\n\t// Default: DefaultProvideDHTMaxWorkers\n\tMaxWorkers *OptionalInteger `json:\",omitempty\"`\n\n\t// SweepEnabled activates the sweeping reprovider system which spreads\n\t// reprovide operations over time.\n\t// Default: DefaultProvideDHTSweepEnabled\n\tSweepEnabled Flag `json:\",omitempty\"`\n\n\t// DedicatedPeriodicWorkers sets workers dedicated to periodic reprovides (sweep mode only).\n\t// Default: DefaultProvideDHTDedicatedPeriodicWorkers\n\tDedicatedPeriodicWorkers *OptionalInteger `json:\",omitempty\"`\n\n\t// DedicatedBurstWorkers sets workers dedicated to burst provides (sweep mode only).\n\t// Default: DefaultProvideDHTDedicatedBurstWorkers\n\tDedicatedBurstWorkers *OptionalInteger `json:\",omitempty\"`\n\n\t// MaxProvideConnsPerWorker sets concurrent connections per worker for sending provider records (sweep mode only).\n\t// Default: DefaultProvideDHTMaxProvideConnsPerWorker\n\tMaxProvideConnsPerWorker *OptionalInteger `json:\",omitempty\"`\n\n\t// KeystoreBatchSize sets the batch size for keystore operations during reprovide refresh (sweep mode only).\n\t// Default: DefaultProvideDHTKeystoreBatchSize\n\tKeystoreBatchSize *OptionalInteger `json:\",omitempty\"`\n\n\t// OfflineDelay sets the delay after which the provider switches from Disconnected to Offline state (sweep mode only).\n\t// Default: DefaultProvideDHTOfflineDelay\n\tOfflineDelay *OptionalDuration `json:\",omitempty\"`\n\n\t// ResumeEnabled controls whether the provider resumes from its previous state on restart.\n\t// When enabled, the provider persists its reprovide cycle state and provide queue to the datastore,\n\t// and restores them on restart. When disabled, the provider starts fresh on each restart.\n\t// Default: true\n\tResumeEnabled Flag `json:\",omitempty\"`\n}\n\nfunc ParseProvideStrategy(s string) ProvideStrategy {\n\tvar strategy ProvideStrategy\n\tfor part := range strings.SplitSeq(s, \"+\") {\n\t\tswitch part {\n\t\tcase \"all\", \"flat\", \"\": // special case, does not mix with others (\"flat\" is deprecated, maps to \"all\")\n\t\t\treturn ProvideStrategyAll\n\t\tcase \"pinned\":\n\t\t\tstrategy |= ProvideStrategyPinned\n\t\tcase \"roots\":\n\t\t\tstrategy |= ProvideStrategyRoots\n\t\tcase \"mfs\":\n\t\t\tstrategy |= ProvideStrategyMFS\n\t\t}\n\t}\n\treturn strategy\n}\n\n// ValidateProvideConfig validates the Provide configuration according to DHT requirements.\nfunc ValidateProvideConfig(cfg *Provide) error {\n\t// Validate Provide.DHT.Interval\n\tif !cfg.DHT.Interval.IsDefault() {\n\t\tinterval := cfg.DHT.Interval.WithDefault(DefaultProvideDHTInterval)\n\t\tif interval > amino.DefaultProvideValidity {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.Interval (%v) must be less than or equal to DHT provider record validity (%v)\", interval, amino.DefaultProvideValidity)\n\t\t}\n\t\tif interval < 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.Interval must be non-negative, got %v\", interval)\n\t\t}\n\t}\n\n\t// Validate MaxWorkers\n\tif !cfg.DHT.MaxWorkers.IsDefault() {\n\t\tmaxWorkers := cfg.DHT.MaxWorkers.WithDefault(DefaultProvideDHTMaxWorkers)\n\t\tif maxWorkers <= 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.MaxWorkers must be positive, got %d\", maxWorkers)\n\t\t}\n\t}\n\n\t// Validate DedicatedPeriodicWorkers\n\tif !cfg.DHT.DedicatedPeriodicWorkers.IsDefault() {\n\t\tworkers := cfg.DHT.DedicatedPeriodicWorkers.WithDefault(DefaultProvideDHTDedicatedPeriodicWorkers)\n\t\tif workers < 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.DedicatedPeriodicWorkers must be non-negative, got %d\", workers)\n\t\t}\n\t}\n\n\t// Validate DedicatedBurstWorkers\n\tif !cfg.DHT.DedicatedBurstWorkers.IsDefault() {\n\t\tworkers := cfg.DHT.DedicatedBurstWorkers.WithDefault(DefaultProvideDHTDedicatedBurstWorkers)\n\t\tif workers < 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.DedicatedBurstWorkers must be non-negative, got %d\", workers)\n\t\t}\n\t}\n\n\t// Validate MaxProvideConnsPerWorker\n\tif !cfg.DHT.MaxProvideConnsPerWorker.IsDefault() {\n\t\tconns := cfg.DHT.MaxProvideConnsPerWorker.WithDefault(DefaultProvideDHTMaxProvideConnsPerWorker)\n\t\tif conns <= 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.MaxProvideConnsPerWorker must be positive, got %d\", conns)\n\t\t}\n\t}\n\n\t// Validate KeystoreBatchSize\n\tif !cfg.DHT.KeystoreBatchSize.IsDefault() {\n\t\tbatchSize := cfg.DHT.KeystoreBatchSize.WithDefault(DefaultProvideDHTKeystoreBatchSize)\n\t\tif batchSize <= 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.KeystoreBatchSize must be positive, got %d\", batchSize)\n\t\t}\n\t}\n\n\t// Validate OfflineDelay\n\tif !cfg.DHT.OfflineDelay.IsDefault() {\n\t\tdelay := cfg.DHT.OfflineDelay.WithDefault(DefaultProvideDHTOfflineDelay)\n\t\tif delay < 0 {\n\t\t\treturn fmt.Errorf(\"Provide.DHT.OfflineDelay must be non-negative, got %v\", delay)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// ShouldProvideForStrategy determines if content should be provided based on the provide strategy\n// and content characteristics (pinned status, root status, MFS status).\nfunc ShouldProvideForStrategy(strategy ProvideStrategy, isPinned bool, isPinnedRoot bool, isMFS bool) bool {\n\tif strategy == ProvideStrategyAll {\n\t\t// 'all' strategy: always provide\n\t\treturn true\n\t}\n\n\t// For combined strategies, check each component\n\tif strategy&ProvideStrategyPinned != 0 && isPinned {\n\t\treturn true\n\t}\n\tif strategy&ProvideStrategyRoots != 0 && isPinnedRoot {\n\t\treturn true\n\t}\n\tif strategy&ProvideStrategyMFS != 0 && isMFS {\n\t\treturn true\n\t}\n\n\treturn false\n}\n" }, { "path": "config/provide_test.go", "content": "package config\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestParseProvideStrategy(t *testing.T) {\n\ttests := []struct {\n\t\tinput string\n\t\texpect ProvideStrategy\n\t}{\n\t\t{\"all\", ProvideStrategyAll},\n\t\t{\"pinned\", ProvideStrategyPinned},\n\t\t{\"mfs\", ProvideStrategyMFS},\n\t\t{\"pinned+mfs\", ProvideStrategyPinned | ProvideStrategyMFS},\n\t\t{\"invalid\", 0},\n\t\t{\"all+invalid\", ProvideStrategyAll},\n\t\t{\"\", ProvideStrategyAll},\n\t\t{\"flat\", ProvideStrategyAll}, // deprecated, maps to \"all\"\n\t\t{\"flat+all\", ProvideStrategyAll},\n\t}\n\n\tfor _, tt := range tests {\n\t\tresult := ParseProvideStrategy(tt.input)\n\t\tif result != tt.expect {\n\t\t\tt.Errorf(\"ParseProvideStrategy(%q) = %d, want %d\", tt.input, result, tt.expect)\n\t\t}\n\t}\n}\n\nfunc TestValidateProvideConfig_Interval(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinterval time.Duration\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{\"valid default (22h)\", 22 * time.Hour, false, \"\"},\n\t\t{\"valid max (48h)\", 48 * time.Hour, false, \"\"},\n\t\t{\"valid small (1h)\", 1 * time.Hour, false, \"\"},\n\t\t{\"valid zero (disabled)\", 0, false, \"\"},\n\t\t{\"invalid over limit (49h)\", 49 * time.Hour, true, \"must be less than or equal to DHT provider record validity\"},\n\t\t{\"invalid over limit (72h)\", 72 * time.Hour, true, \"must be less than or equal to DHT provider record validity\"},\n\t\t{\"invalid negative\", -1 * time.Hour, true, \"must be non-negative\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Provide{\n\t\t\t\tDHT: ProvideDHT{\n\t\t\t\t\tInterval: NewOptionalDuration(tt.interval),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\terr := ValidateProvideConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\trequire.Error(t, err, \"expected error for interval=%v\", tt.interval)\n\t\t\t\tif tt.errMsg != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tt.errMsg, \"error message mismatch\")\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err, \"unexpected error for interval=%v\", tt.interval)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateProvideConfig_MaxWorkers(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmaxWorkers int64\n\t\twantErr bool\n\t\terrMsg string\n\t}{\n\t\t{\"valid default\", 16, false, \"\"},\n\t\t{\"valid high\", 100, false, \"\"},\n\t\t{\"valid low\", 1, false, \"\"},\n\t\t{\"invalid zero\", 0, true, \"must be positive\"},\n\t\t{\"invalid negative\", -1, true, \"must be positive\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tcfg := &Provide{\n\t\t\t\tDHT: ProvideDHT{\n\t\t\t\t\tMaxWorkers: NewOptionalInteger(tt.maxWorkers),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\terr := ValidateProvideConfig(cfg)\n\n\t\t\tif tt.wantErr {\n\t\t\t\trequire.Error(t, err, \"expected error for maxWorkers=%d\", tt.maxWorkers)\n\t\t\t\tif tt.errMsg != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tt.errMsg, \"error message mismatch\")\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err, \"unexpected error for maxWorkers=%d\", tt.maxWorkers)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestShouldProvideForStrategy(t *testing.T) {\n\tt.Run(\"all strategy always provides\", func(t *testing.T) {\n\t\t// ProvideStrategyAll should return true regardless of flags\n\t\ttestCases := []struct{ pinned, pinnedRoot, mfs bool }{\n\t\t\t{false, false, false},\n\t\t\t{true, true, true},\n\t\t\t{true, false, false},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tassert.True(t, ShouldProvideForStrategy(\n\t\t\t\tProvideStrategyAll, tc.pinned, tc.pinnedRoot, tc.mfs))\n\t\t}\n\t})\n\n\tt.Run(\"single strategies match only their flag\", func(t *testing.T) {\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tstrategy ProvideStrategy\n\t\t\tpinned, pinnedRoot, mfs bool\n\t\t\twant bool\n\t\t}{\n\t\t\t{\"pinned: matches when pinned=true\", ProvideStrategyPinned, true, false, false, true},\n\t\t\t{\"pinned: ignores other flags\", ProvideStrategyPinned, false, true, true, false},\n\n\t\t\t{\"roots: matches when pinnedRoot=true\", ProvideStrategyRoots, false, true, false, true},\n\t\t\t{\"roots: ignores other flags\", ProvideStrategyRoots, true, false, true, false},\n\n\t\t\t{\"mfs: matches when mfs=true\", ProvideStrategyMFS, false, false, true, true},\n\t\t\t{\"mfs: ignores other flags\", ProvideStrategyMFS, true, true, false, false},\n\t\t}\n\n\t\tfor _, tt := range tests {\n\t\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t\tgot := ShouldProvideForStrategy(tt.strategy, tt.pinned, tt.pinnedRoot, tt.mfs)\n\t\t\t\tassert.Equal(t, tt.want, got)\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"combined strategies use OR logic (else-if bug fix)\", func(t *testing.T) {\n\t\t// CRITICAL: Tests the fix where bitflag combinations (pinned+mfs) didn't work\n\t\t// because of else-if instead of separate if statements\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tstrategy ProvideStrategy\n\t\t\tpinned, pinnedRoot, mfs bool\n\t\t\twant bool\n\t\t}{\n\t\t\t// pinned|mfs: provide if EITHER matches\n\t\t\t{\"pinned|mfs when pinned\", ProvideStrategyPinned | ProvideStrategyMFS, true, false, false, true},\n\t\t\t{\"pinned|mfs when mfs\", ProvideStrategyPinned | ProvideStrategyMFS, false, false, true, true},\n\t\t\t{\"pinned|mfs when both\", ProvideStrategyPinned | ProvideStrategyMFS, true, false, true, true},\n\t\t\t{\"pinned|mfs when neither\", ProvideStrategyPinned | ProvideStrategyMFS, false, false, false, false},\n\n\t\t\t// roots|mfs\n\t\t\t{\"roots|mfs when root\", ProvideStrategyRoots | ProvideStrategyMFS, false, true, false, true},\n\t\t\t{\"roots|mfs when mfs\", ProvideStrategyRoots | ProvideStrategyMFS, false, false, true, true},\n\t\t\t{\"roots|mfs when neither\", ProvideStrategyRoots | ProvideStrategyMFS, false, false, false, false},\n\n\t\t\t// pinned|roots\n\t\t\t{\"pinned|roots when pinned\", ProvideStrategyPinned | ProvideStrategyRoots, true, false, false, true},\n\t\t\t{\"pinned|roots when root\", ProvideStrategyPinned | ProvideStrategyRoots, false, true, false, true},\n\t\t\t{\"pinned|roots when neither\", ProvideStrategyPinned | ProvideStrategyRoots, false, false, false, false},\n\n\t\t\t// triple combination\n\t\t\t{\"all-three when any matches\", ProvideStrategyPinned | ProvideStrategyRoots | ProvideStrategyMFS, false, false, true, true},\n\t\t\t{\"all-three when none match\", ProvideStrategyPinned | ProvideStrategyRoots | ProvideStrategyMFS, false, false, false, false},\n\t\t}\n\n\t\tfor _, tt := range tests {\n\t\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t\tgot := ShouldProvideForStrategy(tt.strategy, tt.pinned, tt.pinnedRoot, tt.mfs)\n\t\t\t\tassert.Equal(t, tt.want, got)\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"zero strategy never provides\", func(t *testing.T) {\n\t\tassert.False(t, ShouldProvideForStrategy(ProvideStrategy(0), false, false, false))\n\t\tassert.False(t, ShouldProvideForStrategy(ProvideStrategy(0), true, true, true))\n\t})\n}\n" }, { "path": "config/provider.go", "content": "package config\n\n// Provider configuration describes how NEW CIDs are announced the moment they are created.\n// For periodical reprovide configuration, see Provide.*\n//\n// Deprecated: use Provide instead. This will be removed in a future release.\ntype Provider struct {\n\t// Deprecated: use Provide.Enabled instead. This will be removed in a future release.\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// Deprecated: unused, you are likely looking for Provide.Strategy instead. This will be removed in a future release.\n\tStrategy *OptionalString `json:\",omitempty\"`\n\n\t// Deprecated: use Provide.DHT.MaxWorkers instead. This will be removed in a future release.\n\tWorkerCount *OptionalInteger `json:\",omitempty\"`\n}\n" }, { "path": "config/pubsub.go", "content": "package config\n\nconst (\n\t// LastSeenMessagesStrategy is a strategy that calculates the TTL countdown\n\t// based on the last time a Pubsub message is seen. This means that if a message\n\t// is received and then seen again within the specified TTL window, it\n\t// won't be emitted until the TTL countdown expires from the last time the\n\t// message was seen.\n\tLastSeenMessagesStrategy = \"last-seen\"\n\n\t// FirstSeenMessagesStrategy is a strategy that calculates the TTL\n\t// countdown based on the first time a Pubsub message is seen. This means that if\n\t// a message is received and then seen again within the specified TTL\n\t// window, it won't be emitted.\n\tFirstSeenMessagesStrategy = \"first-seen\"\n\n\t// DefaultSeenMessagesStrategy is the strategy that is used by default if\n\t// no Pubsub.SeenMessagesStrategy is specified.\n\tDefaultSeenMessagesStrategy = LastSeenMessagesStrategy\n)\n\ntype PubsubConfig struct {\n\t// Router can be either floodsub (legacy) or gossipsub (new and\n\t// backwards compatible).\n\tRouter string\n\n\t// DisableSigning disables message signing. Message signing is *enabled*\n\t// by default.\n\tDisableSigning bool\n\n\t// Enable pubsub (--enable-pubsub-experiment)\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// SeenMessagesTTL is a value that controls the time window within which\n\t// duplicate messages will be identified and won't be emitted.\n\tSeenMessagesTTL *OptionalDuration `json:\",omitempty\"`\n\n\t// SeenMessagesStrategy is a setting that determines how the time-to-live\n\t// (TTL) countdown for deduplicating messages is calculated.\n\tSeenMessagesStrategy *OptionalString `json:\",omitempty\"`\n}\n" }, { "path": "config/remotepin.go", "content": "package config\n\nvar (\n\tRemoteServicesPath = \"Pinning.RemoteServices\"\n\tPinningConcealSelector = []string{\"Pinning\", \"RemoteServices\", \"*\", \"API\", \"Key\"}\n)\n\ntype Pinning struct {\n\tRemoteServices map[string]RemotePinningService\n}\n\ntype RemotePinningService struct {\n\tAPI RemotePinningServiceAPI\n\tPolicies RemotePinningServicePolicies\n}\n\ntype RemotePinningServiceAPI struct {\n\tEndpoint string\n\tKey string\n}\n\ntype RemotePinningServicePolicies struct {\n\tMFS RemotePinningServiceMFSPolicy\n}\n\ntype RemotePinningServiceMFSPolicy struct {\n\t// Enable enables watching for changes in MFS and re-pinning the MFS root cid whenever a change occurs.\n\tEnable bool\n\t// Name is the pin name for MFS.\n\tPinName string\n\t// RepinInterval determines the repin interval when the policy is enabled. In ns, us, ms, s, m, h.\n\tRepinInterval string\n}\n" }, { "path": "config/reprovider.go", "content": "package config\n\n// Reprovider configuration describes how CID from local datastore are periodically re-announced to routing systems.\n// For provide behavior of ad-hoc or newly created CIDs and their first-time announcement, see Provide.*\n//\n// Deprecated: use Provide instead. This will be removed in a future release.\ntype Reprovider struct {\n\t// Deprecated: use Provide.DHT.Interval instead. This will be removed in a future release.\n\tInterval *OptionalDuration `json:\",omitempty\"`\n\n\t// Deprecated: use Provide.Strategy instead. This will be removed in a future release.\n\tStrategy *OptionalString `json:\",omitempty\"`\n}\n" }, { "path": "config/routing.go", "content": "package config\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"runtime\"\n\t\"slices\"\n\t\"strings\"\n)\n\nconst (\n\tDefaultAcceleratedDHTClient = false\n\tDefaultLoopbackAddressesOnLanDHT = false\n\tDefaultRoutingType = \"auto\"\n\tCidContactRoutingURL = \"https://cid.contact\"\n\tPublicGoodDelegatedRoutingURL = \"https://delegated-ipfs.dev\" // cid.contact + amino dht (incl. IPNS PUTs)\n\tEnvHTTPRouters = \"IPFS_HTTP_ROUTERS\"\n\tEnvHTTPRoutersFilterProtocols = \"IPFS_HTTP_ROUTERS_FILTER_PROTOCOLS\"\n)\n\nvar (\n\t// Default filter-protocols to pass along with delegated routing requests (as defined in IPIP-484)\n\t// and also filter out locally\n\tDefaultHTTPRoutersFilterProtocols = getEnvOrDefault(EnvHTTPRoutersFilterProtocols, []string{\n\t\t\"unknown\", // allow results without protocol list, we can do libp2p identify to test them\n\t\t\"transport-bitswap\",\n\t\t// http is added dynamically in routing/delegated.go.\n\t\t// 'transport-ipfs-gateway-http'\n\t})\n)\n\n// Routing defines configuration options for libp2p routing.\ntype Routing struct {\n\t// Type sets default daemon routing mode.\n\t//\n\t// Can be one of \"auto\", \"autoclient\", \"dht\", \"dhtclient\", \"dhtserver\", \"none\", \"delegated\", or \"custom\".\n\t// When unset or set to \"auto\", DHT and implicit routers are used.\n\t// When \"delegated\" is set, only HTTP delegated routers and IPNS publishers are used (no DHT).\n\t// When \"custom\" is set, user-provided Routing.Routers is used.\n\tType *OptionalString `json:\",omitempty\"`\n\n\tAcceleratedDHTClient Flag `json:\",omitempty\"`\n\n\tLoopbackAddressesOnLanDHT Flag `json:\",omitempty\"`\n\n\tIgnoreProviders []string `json:\",omitempty\"`\n\n\t// Simplified configuration used by default when Routing.Type=auto|autoclient\n\tDelegatedRouters []string\n\n\t// Advanced configuration used when Routing.Type=custom\n\tRouters Routers `json:\",omitempty\"`\n\tMethods Methods `json:\",omitempty\"`\n}\n\ntype Router struct {\n\t// Router type ID. See RouterType for more info.\n\tType RouterType\n\n\t// Parameters are extra configuration that this router might need.\n\t// A common one for HTTP router is \"Endpoint\".\n\tParameters any\n}\n\ntype (\n\tRouters map[string]RouterParser\n\tMethods map[MethodName]Method\n)\n\nfunc (m Methods) Check() error {\n\t// Check supported methods\n\tfor _, mn := range MethodNameList {\n\t\t_, ok := m[mn]\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"method name %q is missing from Routing.Methods config param\", mn)\n\t\t}\n\t}\n\n\t// Check unsupported methods\n\tfor k := range m {\n\t\tseen := slices.Contains(MethodNameList, k)\n\n\t\tif seen {\n\t\t\tcontinue\n\t\t}\n\n\t\treturn fmt.Errorf(\"method name %q is not a supported method on Routing.Methods config param\", k)\n\t}\n\n\treturn nil\n}\n\ntype RouterParser struct {\n\tRouter\n}\n\nfunc (r *RouterParser) UnmarshalJSON(b []byte) error {\n\tout := Router{}\n\tout.Parameters = &json.RawMessage{}\n\tif err := json.Unmarshal(b, &out); err != nil {\n\t\treturn err\n\t}\n\traw := out.Parameters.(*json.RawMessage)\n\n\tvar p any\n\tswitch out.Type {\n\tcase RouterTypeHTTP:\n\t\tp = &HTTPRouterParams{}\n\tcase RouterTypeDHT:\n\t\tp = &DHTRouterParams{}\n\tcase RouterTypeSequential:\n\t\tp = &ComposableRouterParams{}\n\tcase RouterTypeParallel:\n\t\tp = &ComposableRouterParams{}\n\t}\n\n\tif err := json.Unmarshal(*raw, &p); err != nil {\n\t\treturn err\n\t}\n\n\tr.Router.Type = out.Type\n\tr.Router.Parameters = p\n\n\treturn nil\n}\n\n// Type is the routing type.\n// Depending of the type we need to instantiate different Routing implementations.\ntype RouterType string\n\nconst (\n\tRouterTypeHTTP RouterType = \"http\" // HTTP JSON API for delegated routing systems (IPIP-337).\n\tRouterTypeDHT RouterType = \"dht\" // DHT router.\n\tRouterTypeSequential RouterType = \"sequential\" // Router helper to execute several routers sequentially.\n\tRouterTypeParallel RouterType = \"parallel\" // Router helper to execute several routers in parallel.\n)\n\ntype DHTMode string\n\nconst (\n\tDHTModeServer DHTMode = \"server\"\n\tDHTModeClient DHTMode = \"client\"\n\tDHTModeAuto DHTMode = \"auto\"\n)\n\ntype MethodName string\n\nconst (\n\tMethodNameProvide MethodName = \"provide\"\n\tMethodNameFindProviders MethodName = \"find-providers\"\n\tMethodNameFindPeers MethodName = \"find-peers\"\n\tMethodNameGetIPNS MethodName = \"get-ipns\"\n\tMethodNamePutIPNS MethodName = \"put-ipns\"\n)\n\nvar MethodNameList = []MethodName{MethodNameProvide, MethodNameFindPeers, MethodNameFindProviders, MethodNameGetIPNS, MethodNamePutIPNS}\n\ntype HTTPRouterParams struct {\n\t// Endpoint is the URL where the routing implementation will point to get the information.\n\tEndpoint string\n\n\t// MaxProvideBatchSize determines the maximum amount of CIDs sent per batch.\n\t// Servers might not accept more than 100 elements per batch. 100 elements by default.\n\tMaxProvideBatchSize int\n\n\t// MaxProvideConcurrency determines the number of threads used when providing content. GOMAXPROCS by default.\n\tMaxProvideConcurrency int\n}\n\nfunc (hrp *HTTPRouterParams) FillDefaults() {\n\tif hrp.MaxProvideBatchSize == 0 {\n\t\thrp.MaxProvideBatchSize = 100\n\t}\n\n\tif hrp.MaxProvideConcurrency == 0 {\n\t\thrp.MaxProvideConcurrency = runtime.GOMAXPROCS(0)\n\t}\n}\n\ntype DHTRouterParams struct {\n\tMode DHTMode\n\tAcceleratedDHTClient bool `json:\",omitempty\"`\n\tPublicIPNetwork bool\n}\n\ntype ComposableRouterParams struct {\n\tRouters []ConfigRouter\n\tTimeout *OptionalDuration `json:\",omitempty\"`\n}\n\ntype ConfigRouter struct {\n\tRouterName string\n\tTimeout Duration\n\tIgnoreErrors bool\n\tExecuteAfter *OptionalDuration `json:\",omitempty\"`\n}\n\ntype Method struct {\n\tRouterName string\n}\n\n// getEnvOrDefault reads space or comma separated strings from env if present,\n// and uses provided defaultValue as a fallback\nfunc getEnvOrDefault(key string, defaultValue []string) []string {\n\tif value, exists := os.LookupEnv(key); exists {\n\t\tsplitFunc := func(r rune) bool { return r == ',' || r == ' ' }\n\t\treturn strings.FieldsFunc(value, splitFunc)\n\t}\n\treturn defaultValue\n}\n\n// HasHTTPProviderConfigured checks if the node is configured to use HTTP routers\n// for providing content announcements. This is used when determining if the node\n// can provide content even when not connected to libp2p peers.\n//\n// Note: Right now we only support delegated HTTP content providing if Routing.Type=custom\n// and Routing.Routers are configured according to:\n// https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md#configuration-file-example\n//\n// This uses the `ProvideBitswap` request type that is not documented anywhere,\n// because we hoped something like IPIP-378 (https://github.com/ipfs/specs/pull/378)\n// would get finalized and we'd switch to that. It never happened due to politics,\n// and now we are stuck with ProvideBitswap being the only API that works.\n// Some people have reverse engineered it (example:\n// https://discuss.ipfs.tech/t/only-peers-found-from-dht-seem-to-be-getting-used-as-relays-so-cant-use-http-routers/19545/9)\n// and use it, so what we do here is the bare minimum to ensure their use case works\n// using this old API until something better is available.\nfunc (c *Config) HasHTTPProviderConfigured() bool {\n\tif len(c.Routing.Routers) == 0 {\n\t\t// No \"custom\" routers\n\t\treturn false\n\t}\n\tmethod, ok := c.Routing.Methods[MethodNameProvide]\n\tif !ok {\n\t\t// No provide method configured\n\t\treturn false\n\t}\n\treturn c.routerSupportsHTTPProviding(method.RouterName)\n}\n\n// routerSupportsHTTPProviding checks if the supplied custom router is or\n// includes an HTTP-based router.\nfunc (c *Config) routerSupportsHTTPProviding(routerName string) bool {\n\trp, ok := c.Routing.Routers[routerName]\n\tif !ok {\n\t\t// Router configured for providing doesn't exist\n\t\treturn false\n\t}\n\n\tswitch rp.Type {\n\tcase RouterTypeHTTP:\n\t\treturn true\n\tcase RouterTypeParallel, RouterTypeSequential:\n\t\t// Check if any child router supports HTTP\n\t\tif children, ok := rp.Parameters.(*ComposableRouterParams); ok {\n\t\t\tfor _, childRouter := range children.Routers {\n\t\t\t\tif c.routerSupportsHTTPProviding(childRouter.RouterName) {\n\t\t\t\t\treturn true\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn false\n}\n" }, { "path": "config/routing_test.go", "content": "package config\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRouterParameters(t *testing.T) {\n\trequire := require.New(t)\n\tsec := time.Second\n\tmin := time.Minute\n\tr := Routing{\n\t\tType: NewOptionalString(\"custom\"),\n\t\tRouters: map[string]RouterParser{\n\t\t\t\"router-dht\": {Router{\n\t\t\t\tType: RouterTypeDHT,\n\t\t\t\tParameters: DHTRouterParams{\n\t\t\t\t\tMode: \"auto\",\n\t\t\t\t\tAcceleratedDHTClient: true,\n\t\t\t\t\tPublicIPNetwork: false,\n\t\t\t\t},\n\t\t\t}},\n\t\t\t\"router-parallel\": {\n\t\t\t\tRouter{\n\t\t\t\t\tType: RouterTypeParallel,\n\t\t\t\t\tParameters: ComposableRouterParams{\n\t\t\t\t\t\tRouters: []ConfigRouter{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tRouterName: \"router-dht\",\n\t\t\t\t\t\t\t\tTimeout: Duration{10 * time.Second},\n\t\t\t\t\t\t\t\tIgnoreErrors: true,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tRouterName: \"router-dht\",\n\t\t\t\t\t\t\t\tTimeout: Duration{10 * time.Second},\n\t\t\t\t\t\t\t\tIgnoreErrors: false,\n\t\t\t\t\t\t\t\tExecuteAfter: &OptionalDuration{&sec},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tTimeout: &OptionalDuration{&min},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"router-sequential\": {\n\t\t\t\tRouter{\n\t\t\t\t\tType: RouterTypeSequential,\n\t\t\t\t\tParameters: ComposableRouterParams{\n\t\t\t\t\t\tRouters: []ConfigRouter{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tRouterName: \"router-dht\",\n\t\t\t\t\t\t\t\tTimeout: Duration{10 * time.Second},\n\t\t\t\t\t\t\t\tIgnoreErrors: true,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tRouterName: \"router-dht\",\n\t\t\t\t\t\t\t\tTimeout: Duration{10 * time.Second},\n\t\t\t\t\t\t\t\tIgnoreErrors: false,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tTimeout: &OptionalDuration{&min},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tMethods: Methods{\n\t\t\tMethodNameFindPeers: {\n\t\t\t\tRouterName: \"router-dht\",\n\t\t\t},\n\t\t\tMethodNameFindProviders: {\n\t\t\t\tRouterName: \"router-dht\",\n\t\t\t},\n\t\t\tMethodNameGetIPNS: {\n\t\t\t\tRouterName: \"router-sequential\",\n\t\t\t},\n\t\t\tMethodNameProvide: {\n\t\t\t\tRouterName: \"router-parallel\",\n\t\t\t},\n\t\t\tMethodNamePutIPNS: {\n\t\t\t\tRouterName: \"router-parallel\",\n\t\t\t},\n\t\t},\n\t}\n\n\tout, err := json.Marshal(r)\n\trequire.NoError(err)\n\n\tr2 := &Routing{}\n\n\terr = json.Unmarshal(out, r2)\n\trequire.NoError(err)\n\n\trequire.Equal(5, len(r2.Methods))\n\n\tdhtp := r2.Routers[\"router-dht\"].Parameters\n\trequire.IsType(&DHTRouterParams{}, dhtp)\n\n\tsp := r2.Routers[\"router-sequential\"].Parameters\n\trequire.IsType(&ComposableRouterParams{}, sp)\n\n\tpp := r2.Routers[\"router-parallel\"].Parameters\n\trequire.IsType(&ComposableRouterParams{}, pp)\n}\n\nfunc TestMethods(t *testing.T) {\n\trequire := require.New(t)\n\n\tmethodsOK := Methods{\n\t\tMethodNameFindPeers: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNameFindProviders: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNameGetIPNS: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNameProvide: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNamePutIPNS: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t}\n\n\trequire.NoError(methodsOK.Check())\n\n\tmethodsMissing := Methods{\n\t\tMethodNameFindPeers: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNameGetIPNS: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNameProvide: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t\tMethodNamePutIPNS: {\n\t\t\tRouterName: \"router-wrong\",\n\t\t},\n\t}\n\n\trequire.Error(methodsMissing.Check())\n}\n" }, { "path": "config/serialize/serialize.go", "content": "package fsrepo\n\nimport (\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/ipfs/kubo/config\"\n\n\t\"github.com/facebookgo/atomicfile\"\n)\n\n// ErrNotInitialized is returned when we fail to read the config because the\n// repo doesn't exist.\nvar ErrNotInitialized = errors.New(\"ipfs not initialized, please run 'ipfs init'\")\n\n// ReadConfigFile reads the config from `filename` into `cfg`.\nfunc ReadConfigFile(filename string, cfg any) error {\n\tf, err := os.Open(filename)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\terr = ErrNotInitialized\n\t\t}\n\t\treturn err\n\t}\n\tdefer f.Close()\n\tif err := json.NewDecoder(f).Decode(cfg); err != nil {\n\t\treturn fmt.Errorf(\"failure to decode config: %w\", err)\n\t}\n\treturn nil\n}\n\n// WriteConfigFile writes the config from `cfg` into `filename`.\nfunc WriteConfigFile(filename string, cfg any) error {\n\terr := os.MkdirAll(filepath.Dir(filename), 0o755)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tf, err := atomicfile.New(filename, 0o600)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer f.Close()\n\n\treturn encode(f, cfg)\n}\n\n// encode configuration with JSON.\nfunc encode(w io.Writer, value any) error {\n\t// need to prettyprint, hence MarshalIndent, instead of Encoder\n\tbuf, err := config.Marshal(value)\n\tif err != nil {\n\t\treturn err\n\t}\n\t_, err = w.Write(buf)\n\treturn err\n}\n\n// Load reads given file and returns the read config, or error.\nfunc Load(filename string) (*config.Config, error) {\n\tvar cfg config.Config\n\terr := ReadConfigFile(filename, &cfg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &cfg, err\n}\n" }, { "path": "config/serialize/serialize_test.go", "content": "package fsrepo\n\nimport (\n\t\"os\"\n\t\"runtime\"\n\t\"testing\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n)\n\nfunc TestConfig(t *testing.T) {\n\tconst filename = \".ipfsconfig\"\n\tcfgWritten := new(config.Config)\n\tcfgWritten.Identity.PeerID = \"faketest\"\n\n\terr := WriteConfigFile(filename, cfgWritten)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tcfgRead, err := Load(filename)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif cfgWritten.Identity.PeerID != cfgRead.Identity.PeerID {\n\t\tt.Fatal()\n\t}\n\tst, err := os.Stat(filename)\n\tif err != nil {\n\t\tt.Fatalf(\"cannot stat config file: %v\", err)\n\t}\n\n\tif runtime.GOOS != \"windows\" { // see https://golang.org/src/os/types_windows.go\n\t\tif g := st.Mode().Perm(); g&0o117 != 0 {\n\t\t\tt.Fatalf(\"config file should not be executable or accessible to world: %v\", g)\n\t\t}\n\t}\n}\n" }, { "path": "config/swarm.go", "content": "package config\n\ntype SwarmConfig struct {\n\t// AddrFilters specifies a set libp2p addresses that we should never\n\t// dial or receive connections from.\n\tAddrFilters []string\n\n\t// DisableBandwidthMetrics disables recording of bandwidth metrics for a\n\t// slight reduction in memory usage. You probably don't need to set this\n\t// flag.\n\tDisableBandwidthMetrics bool\n\n\t// DisableNatPortMap turns off NAT port mapping (UPnP, etc.).\n\tDisableNatPortMap bool\n\n\t// RelayClient controls the client side of \"auto relay\" feature.\n\t// When enabled, the node will use relays if it is not publicly reachable.\n\tRelayClient RelayClient\n\n\t// RelayService.* controls the \"relay service\".\n\t// When enabled, node will provide a limited relay service to other peers.\n\tRelayService RelayService\n\n\t// EnableHolePunching enables the hole punching service.\n\tEnableHolePunching Flag `json:\",omitempty\"`\n\n\t// Transports contains flags to enable/disable libp2p transports.\n\tTransports Transports\n\n\t// ConnMgr configures the connection manager.\n\tConnMgr ConnMgr\n\n\t// ResourceMgr configures the libp2p Network Resource Manager\n\tResourceMgr ResourceMgr\n}\n\ntype RelayClient struct {\n\t// Enables the auto relay feature: will use relays if it is not publicly reachable.\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// StaticRelays configures static relays to use when this node is not\n\t// publicly reachable. If set, auto relay will not try to find any\n\t// other relay servers.\n\tStaticRelays []string `json:\",omitempty\"`\n}\n\n// RelayService configures the resources of the circuit v2 relay.\n// For every field a reasonable default will be defined in go-ipfs.\ntype RelayService struct {\n\t// Enables the limited relay service for other peers (circuit v2 relay).\n\tEnabled Flag `json:\",omitempty\"`\n\n\t// ConnectionDurationLimit is the time limit before resetting a relayed connection.\n\tConnectionDurationLimit *OptionalDuration `json:\",omitempty\"`\n\t// ConnectionDataLimit is the limit of data relayed (on each direction) before resetting the connection.\n\tConnectionDataLimit *OptionalInteger `json:\",omitempty\"`\n\n\t// ReservationTTL is the duration of a new (or refreshed reservation).\n\tReservationTTL *OptionalDuration `json:\",omitempty\"`\n\n\t// MaxReservations is the maximum number of active relay slots.\n\tMaxReservations *OptionalInteger `json:\",omitempty\"`\n\t// MaxCircuits is the maximum number of open relay connections for each peer; defaults to 16.\n\tMaxCircuits *OptionalInteger `json:\",omitempty\"`\n\t// BufferSize is the size of the relayed connection buffers.\n\tBufferSize *OptionalInteger `json:\",omitempty\"`\n\n\t// MaxReservationsPerIP is the maximum number of reservations originating from the same IP address.\n\tMaxReservationsPerIP *OptionalInteger `json:\",omitempty\"`\n\t// MaxReservationsPerASN is the maximum number of reservations origination from the same ASN.\n\tMaxReservationsPerASN *OptionalInteger `json:\",omitempty\"`\n}\n\ntype Transports struct {\n\t// Network specifies the base transports we'll use for dialing. To\n\t// listen on a transport, add the transport to your Addresses.Swarm.\n\tNetwork struct {\n\t\t// All default to on.\n\t\tQUIC Flag `json:\",omitempty\"`\n\t\tTCP Flag `json:\",omitempty\"`\n\t\tWebsocket Flag `json:\",omitempty\"`\n\t\tRelay Flag `json:\",omitempty\"`\n\t\tWebTransport Flag `json:\",omitempty\"`\n\t\t// except WebRTCDirect which is experimental and opt-in.\n\t\tWebRTCDirect Flag `json:\",omitempty\"`\n\t}\n\n\t// Security specifies the transports used to encrypt insecure network\n\t// transports.\n\tSecurity struct {\n\t\t// Defaults to 100.\n\t\tTLS Priority `json:\",omitempty\"`\n\t\t// Defaults to 300.\n\t\tNoise Priority `json:\",omitempty\"`\n\t}\n\n\t// Multiplexers specifies the transports used to multiplex multiple\n\t// connections over a single duplex connection.\n\tMultiplexers struct {\n\t\t// Defaults to 100.\n\t\tYamux Priority `json:\",omitempty\"`\n\t}\n}\n\n// ConnMgr defines configuration options for the libp2p connection manager.\ntype ConnMgr struct {\n\tType *OptionalString `json:\",omitempty\"`\n\tLowWater *OptionalInteger `json:\",omitempty\"`\n\tHighWater *OptionalInteger `json:\",omitempty\"`\n\tGracePeriod *OptionalDuration `json:\",omitempty\"`\n\tSilencePeriod *OptionalDuration `json:\",omitempty\"`\n}\n\n// ResourceMgr defines configuration options for the libp2p Network Resource Manager\n// \ntype ResourceMgr struct {\n\t// Enables the Network Resource Manager feature, default to on.\n\tEnabled Flag `json:\",omitempty\"`\n\tLimits swarmLimits `json:\",omitempty\"`\n\n\tMaxMemory *OptionalBytes `json:\",omitempty\"`\n\tMaxFileDescriptors *OptionalInteger `json:\",omitempty\"`\n\n\t// A list of multiaddrs that can bypass normal system limits (but are still\n\t// limited by the allowlist scope). Convenience config around\n\t// https://pkg.go.dev/github.com/libp2p/go-libp2p/p2p/host/resource-manager#Allowlist.Add\n\tAllowlist []string `json:\",omitempty\"`\n}\n\nconst (\n\tResourceMgrSystemScope = \"system\"\n\tResourceMgrTransientScope = \"transient\"\n\tResourceMgrServiceScopePrefix = \"svc:\"\n\tResourceMgrProtocolScopePrefix = \"proto:\"\n\tResourceMgrPeerScopePrefix = \"peer:\"\n)\n" }, { "path": "config/types.go", "content": "package config\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n)\n\n// Strings is a helper type that (un)marshals a single string to/from a single\n// JSON string and a slice of strings to/from a JSON array of strings.\ntype Strings []string\n\n// UnmarshalJSON conforms to the json.Unmarshaler interface.\nfunc (o *Strings) UnmarshalJSON(data []byte) error {\n\tif data[0] == '[' {\n\t\treturn json.Unmarshal(data, (*[]string)(o))\n\t}\n\tvar value string\n\tif err := json.Unmarshal(data, &value); err != nil {\n\t\treturn err\n\t}\n\tif len(value) == 0 {\n\t\t*o = []string{}\n\t} else {\n\t\t*o = []string{value}\n\t}\n\treturn nil\n}\n\n// MarshalJSON conforms to the json.Marshaler interface.\nfunc (o Strings) MarshalJSON() ([]byte, error) {\n\tswitch len(o) {\n\tcase 0:\n\t\treturn json.Marshal(nil)\n\tcase 1:\n\t\treturn json.Marshal(o[0])\n\tdefault:\n\t\treturn json.Marshal([]string(o))\n\t}\n}\n\nvar (\n\t_ json.Unmarshaler = (*Strings)(nil)\n\t_ json.Marshaler = (*Strings)(nil)\n)\n\n// Flag represents a ternary value: false (-1), default (0), or true (+1).\n//\n// When encoded in json, False is \"false\", Default is \"null\" (or empty), and True\n// is \"true\".\ntype Flag int8\n\nconst (\n\tFalse Flag = -1\n\tDefault Flag = 0\n\tTrue Flag = 1\n)\n\n// WithDefault resolves the value of the flag given the provided default value.\n//\n// Panics if Flag is an invalid value.\nfunc (f Flag) WithDefault(defaultValue bool) bool {\n\tswitch f {\n\tcase False:\n\t\treturn false\n\tcase Default:\n\t\treturn defaultValue\n\tcase True:\n\t\treturn true\n\tdefault:\n\t\tpanic(fmt.Sprintf(\"invalid flag value %d\", f))\n\t}\n}\n\nfunc (f Flag) MarshalJSON() ([]byte, error) {\n\tswitch f {\n\tcase Default:\n\t\treturn json.Marshal(nil)\n\tcase True:\n\t\treturn json.Marshal(true)\n\tcase False:\n\t\treturn json.Marshal(false)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"invalid flag value: %d\", f)\n\t}\n}\n\nfunc (f *Flag) UnmarshalJSON(input []byte) error {\n\tswitch string(input) {\n\tcase \"null\":\n\t\t*f = Default\n\tcase \"false\":\n\t\t*f = False\n\tcase \"true\":\n\t\t*f = True\n\tdefault:\n\t\treturn fmt.Errorf(\"failed to unmarshal %q into a flag: must be null/undefined, true, or false\", string(input))\n\t}\n\treturn nil\n}\n\nfunc (f Flag) String() string {\n\tswitch f {\n\tcase Default:\n\t\treturn \"default\"\n\tcase True:\n\t\treturn \"true\"\n\tcase False:\n\t\treturn \"false\"\n\tdefault:\n\t\treturn fmt.Sprintf(\"\", f)\n\t}\n}\n\n// ResolveBoolFromConfig returns the resolved boolean value based on:\n// - If userSet is true, returns userValue (user explicitly set the flag)\n// - Otherwise, uses configFlag.WithDefault(defaultValue) (respects config or falls back to default)\nfunc ResolveBoolFromConfig(userValue bool, userSet bool, configFlag Flag, defaultValue bool) bool {\n\tif userSet {\n\t\treturn userValue\n\t}\n\treturn configFlag.WithDefault(defaultValue)\n}\n\nvar (\n\t_ json.Unmarshaler = (*Flag)(nil)\n\t_ json.Marshaler = (*Flag)(nil)\n)\n\n// Priority represents a value with a priority where 0 means \"default\" and -1\n// means \"disabled\".\n//\n// When encoded in json, Default is encoded as \"null\" and Disabled is encoded as\n// \"false\".\ntype Priority int64\n\nconst (\n\tDefaultPriority Priority = 0\n\tDisabled Priority = -1\n)\n\n// WithDefault resolves the priority with the given default.\n//\n// If defaultPriority is Default/0, this function will return 0.\n//\n// Panics if the priority has an invalid value (e.g., not DefaultPriority,\n// Disabled, or > 0).\nfunc (p Priority) WithDefault(defaultPriority Priority) (priority int64, enabled bool) {\n\tswitch p {\n\tcase Disabled:\n\t\treturn 0, false\n\tcase DefaultPriority:\n\t\tswitch defaultPriority {\n\t\tcase Disabled:\n\t\t\treturn 0, false\n\t\tcase DefaultPriority:\n\t\t\treturn 0, true\n\t\tdefault:\n\t\t\tif defaultPriority <= 0 {\n\t\t\t\tpanic(fmt.Sprintf(\"invalid priority %d < 0\", int64(defaultPriority)))\n\t\t\t}\n\t\t\treturn int64(defaultPriority), true\n\t\t}\n\tdefault:\n\t\tif p <= 0 {\n\t\t\tpanic(fmt.Sprintf(\"invalid priority %d < 0\", int64(p)))\n\t\t}\n\t\treturn int64(p), true\n\t}\n}\n\nfunc (p Priority) MarshalJSON() ([]byte, error) {\n\t// > 0 == Priority\n\tif p > 0 {\n\t\treturn json.Marshal(int64(p))\n\t}\n\t// <= 0 == special\n\tswitch p {\n\tcase DefaultPriority:\n\t\treturn json.Marshal(nil)\n\tcase Disabled:\n\t\treturn json.Marshal(false)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"invalid priority value: %d\", p)\n\t}\n}\n\nfunc (p *Priority) UnmarshalJSON(input []byte) error {\n\tswitch string(input) {\n\tcase \"null\", \"undefined\":\n\t\t*p = DefaultPriority\n\tcase \"false\":\n\t\t*p = Disabled\n\tcase \"true\":\n\t\treturn fmt.Errorf(\"'true' is not a valid priority\")\n\tdefault:\n\t\tvar priority int64\n\t\terr := json.Unmarshal(input, &priority)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif priority <= 0 {\n\t\t\treturn fmt.Errorf(\"priority must be positive: %d <= 0\", priority)\n\t\t}\n\t\t*p = Priority(priority)\n\t}\n\treturn nil\n}\n\nfunc (p Priority) String() string {\n\tif p > 0 {\n\t\treturn fmt.Sprintf(\"%d\", p)\n\t}\n\tswitch p {\n\tcase DefaultPriority:\n\t\treturn \"default\"\n\tcase Disabled:\n\t\treturn \"false\"\n\tdefault:\n\t\treturn fmt.Sprintf(\"\", p)\n\t}\n}\n\nvar (\n\t_ json.Unmarshaler = (*Priority)(nil)\n\t_ json.Marshaler = (*Priority)(nil)\n)\n\n// OptionalDuration wraps time.Duration to provide json serialization and deserialization.\n//\n// NOTE: the zero value encodes to JSON nill.\ntype OptionalDuration struct {\n\tvalue *time.Duration\n}\n\n// NewOptionalDuration returns an OptionalDuration from a string.\nfunc NewOptionalDuration(d time.Duration) *OptionalDuration {\n\treturn &OptionalDuration{value: &d}\n}\n\nfunc (d *OptionalDuration) UnmarshalJSON(input []byte) error {\n\tswitch string(input) {\n\tcase \"null\", \"undefined\", \"\\\"null\\\"\", \"\", \"default\", \"\\\"\\\"\", \"\\\"default\\\"\":\n\t\t*d = OptionalDuration{}\n\t\treturn nil\n\tdefault:\n\t\ttext := strings.Trim(string(input), \"\\\"\")\n\t\tvalue, err := time.ParseDuration(text)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\t*d = OptionalDuration{value: &value}\n\t\treturn nil\n\t}\n}\n\nfunc (d *OptionalDuration) IsDefault() bool {\n\treturn d == nil || d.value == nil\n}\n\nfunc (d *OptionalDuration) WithDefault(defaultValue time.Duration) time.Duration {\n\tif d == nil || d.value == nil {\n\t\treturn defaultValue\n\t}\n\treturn *d.value\n}\n\nfunc (d OptionalDuration) MarshalJSON() ([]byte, error) {\n\tif d.value == nil {\n\t\treturn json.Marshal(nil)\n\t}\n\treturn json.Marshal(d.value.String())\n}\n\nfunc (d OptionalDuration) String() string {\n\tif d.value == nil {\n\t\treturn \"default\"\n\t}\n\treturn d.value.String()\n}\n\nvar (\n\t_ json.Unmarshaler = (*OptionalDuration)(nil)\n\t_ json.Marshaler = (*OptionalDuration)(nil)\n)\n\ntype Duration struct {\n\ttime.Duration\n}\n\nfunc (d Duration) MarshalJSON() ([]byte, error) {\n\treturn json.Marshal(d.String())\n}\n\nfunc (d *Duration) UnmarshalJSON(b []byte) error {\n\tvar v any\n\tif err := json.Unmarshal(b, &v); err != nil {\n\t\treturn err\n\t}\n\tswitch value := v.(type) {\n\tcase float64:\n\t\td.Duration = time.Duration(value)\n\t\treturn nil\n\tcase string:\n\t\tvar err error\n\t\td.Duration, err = time.ParseDuration(value)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\tdefault:\n\t\treturn fmt.Errorf(\"unable to parse duration, expected a duration string or a float, but got %T\", v)\n\t}\n}\n\nvar (\n\t_ json.Unmarshaler = (*Duration)(nil)\n\t_ json.Marshaler = (*Duration)(nil)\n)\n\n// OptionalInteger represents an integer that has a default value\n//\n// When encoded in json, Default is encoded as \"null\".\ntype OptionalInteger struct {\n\tvalue *int64\n}\n\n// NewOptionalInteger returns an OptionalInteger from a int64.\nfunc NewOptionalInteger(v int64) *OptionalInteger {\n\treturn &OptionalInteger{value: &v}\n}\n\n// WithDefault resolves the integer with the given default.\nfunc (p *OptionalInteger) WithDefault(defaultValue int64) (value int64) {\n\tif p == nil || p.value == nil {\n\t\treturn defaultValue\n\t}\n\treturn *p.value\n}\n\n// IsDefault returns if this is a default optional integer.\nfunc (p *OptionalInteger) IsDefault() bool {\n\treturn p == nil || p.value == nil\n}\n\nfunc (p OptionalInteger) MarshalJSON() ([]byte, error) {\n\tif p.value != nil {\n\t\treturn json.Marshal(p.value)\n\t}\n\treturn json.Marshal(nil)\n}\n\nfunc (p *OptionalInteger) UnmarshalJSON(input []byte) error {\n\tswitch string(input) {\n\tcase \"null\", \"undefined\":\n\t\t*p = OptionalInteger{}\n\tdefault:\n\t\tvar value int64\n\t\terr := json.Unmarshal(input, &value)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\t*p = OptionalInteger{value: &value}\n\t}\n\treturn nil\n}\n\nfunc (p OptionalInteger) String() string {\n\tif p.value == nil {\n\t\treturn \"default\"\n\t}\n\treturn fmt.Sprintf(\"%d\", *p.value)\n}\n\nvar (\n\t_ json.Unmarshaler = (*OptionalInteger)(nil)\n\t_ json.Marshaler = (*OptionalInteger)(nil)\n)\n\n// OptionalString represents a string that has a default value\n//\n// When encoded in json, Default is encoded as \"null\".\ntype OptionalString struct {\n\tvalue *string\n}\n\n// NewOptionalString returns an OptionalString from a string.\nfunc NewOptionalString(s string) *OptionalString {\n\treturn &OptionalString{value: &s}\n}\n\n// WithDefault resolves the integer with the given default.\nfunc (p *OptionalString) WithDefault(defaultValue string) (value string) {\n\tif p == nil || p.value == nil {\n\t\treturn defaultValue\n\t}\n\treturn *p.value\n}\n\n// IsDefault returns if this is a default optional integer.\nfunc (p *OptionalString) IsDefault() bool {\n\treturn p == nil || p.value == nil\n}\n\nfunc (p OptionalString) MarshalJSON() ([]byte, error) {\n\tif p.value != nil {\n\t\treturn json.Marshal(p.value)\n\t}\n\treturn json.Marshal(nil)\n}\n\nfunc (p *OptionalString) UnmarshalJSON(input []byte) error {\n\tswitch string(input) {\n\tcase \"null\", \"undefined\":\n\t\t*p = OptionalString{}\n\tdefault:\n\t\tvar value string\n\t\terr := json.Unmarshal(input, &value)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\t*p = OptionalString{value: &value}\n\t}\n\treturn nil\n}\n\nfunc (p OptionalString) String() string {\n\tif p.value == nil {\n\t\treturn \"default\"\n\t}\n\treturn *p.value\n}\n\nvar (\n\t_ json.Unmarshaler = (*OptionalString)(nil)\n\t_ json.Marshaler = (*OptionalString)(nil)\n)\n\n// OptionalBytes represents a byte size that has a default value\n//\n// When encoded in json, Default is encoded as \"null\".\n// Stores the original string representation and parses on access.\n// Embeds OptionalString to share common functionality.\ntype OptionalBytes struct {\n\tOptionalString\n}\n\n// NewOptionalBytes returns an OptionalBytes from a string.\nfunc NewOptionalBytes(s string) *OptionalBytes {\n\treturn &OptionalBytes{OptionalString{value: &s}}\n}\n\n// IsDefault returns if this is a default optional byte value.\nfunc (p *OptionalBytes) IsDefault() bool {\n\tif p == nil {\n\t\treturn true\n\t}\n\treturn p.OptionalString.IsDefault()\n}\n\n// WithDefault resolves the byte size with the given default.\n// Parses the stored string value using humanize.ParseBytes.\nfunc (p *OptionalBytes) WithDefault(defaultValue uint64) (value uint64) {\n\tif p.IsDefault() {\n\t\treturn defaultValue\n\t}\n\tstrValue := p.OptionalString.WithDefault(\"\")\n\tbytes, err := humanize.ParseBytes(strValue)\n\tif err != nil {\n\t\t// This should never happen as values are validated during UnmarshalJSON.\n\t\t// If it does, it indicates either config corruption or a programming error.\n\t\tpanic(fmt.Sprintf(\"invalid byte size in OptionalBytes: %q - %v\", strValue, err))\n\t}\n\treturn bytes\n}\n\n// UnmarshalJSON validates the input is a parseable byte size.\nfunc (p *OptionalBytes) UnmarshalJSON(input []byte) error {\n\tswitch string(input) {\n\tcase \"null\", \"undefined\":\n\t\t*p = OptionalBytes{}\n\tdefault:\n\t\tvar value any\n\t\terr := json.Unmarshal(input, &value)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tswitch v := value.(type) {\n\t\tcase float64:\n\t\t\tstr := fmt.Sprintf(\"%.0f\", v)\n\t\t\tp.value = &str\n\t\tcase string:\n\t\t\t_, err := humanize.ParseBytes(v)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tp.value = &v\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"unable to parse byte size, expected a size string (e.g., \\\"5GiB\\\") or a number, but got %T\", v)\n\t\t}\n\t}\n\treturn nil\n}\n\nvar (\n\t_ json.Unmarshaler = (*OptionalBytes)(nil)\n\t_ json.Marshaler = (*OptionalBytes)(nil)\n)\n\ntype swarmLimits doNotUse\n\nvar _ json.Unmarshaler = swarmLimits(false)\n\nfunc (swarmLimits) UnmarshalJSON(b []byte) error {\n\td := json.NewDecoder(bytes.NewReader(b))\n\tfor {\n\t\tswitch tok, err := d.Token(); err {\n\t\tcase io.EOF:\n\t\t\treturn nil\n\t\tcase nil:\n\t\t\tswitch tok {\n\t\t\tcase json.Delim('{'), json.Delim('}'):\n\t\t\t\t// accept empty objects\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t//nolint\n\t\t\treturn fmt.Errorf(\"The Swarm.ResourceMgr.Limits configuration has been removed in Kubo 0.19 and should be empty or not present. To set custom libp2p limits, read https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#user-supplied-override-limits\")\n\t\tdefault:\n\t\t\treturn err\n\t\t}\n\t}\n}\n\ntype experimentalAcceleratedDHTClient doNotUse\n\nvar _ json.Unmarshaler = experimentalAcceleratedDHTClient(false)\n\nfunc (experimentalAcceleratedDHTClient) UnmarshalJSON(b []byte) error {\n\td := json.NewDecoder(bytes.NewReader(b))\n\tfor {\n\t\tswitch tok, err := d.Token(); err {\n\t\tcase io.EOF:\n\t\t\treturn nil\n\t\tcase nil:\n\t\t\tswitch tok {\n\t\t\tcase json.Delim('{'), json.Delim('}'):\n\t\t\t\t// accept empty objects\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t//nolint\n\t\t\treturn fmt.Errorf(\"The Experimental.AcceleratedDHTClient key has been moved to Routing.AcceleratedDHTClient in Kubo 0.21, please use this new key and remove the old one.\")\n\t\tdefault:\n\t\t\treturn err\n\t\t}\n\t}\n}\n\n// doNotUse is a type you must not use, it should be struct{} but encoding/json\n// does not support omitempty on structs and I can't be bothered to write custom\n// marshalers on all structs that have a doNotUse field.\ntype doNotUse bool\n\ntype graphsyncEnabled doNotUse\n\nvar _ json.Unmarshaler = graphsyncEnabled(false)\n\nfunc (graphsyncEnabled) UnmarshalJSON(b []byte) error {\n\td := json.NewDecoder(bytes.NewReader(b))\n\tfor {\n\t\tswitch tok, err := d.Token(); err {\n\t\tcase io.EOF:\n\t\t\treturn nil\n\t\tcase nil:\n\t\t\tswitch tok {\n\t\t\tcase json.Delim('{'), json.Delim('}'), false:\n\t\t\t\t// accept empty objects and false\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t//nolint\n\t\t\treturn fmt.Errorf(\"Support for Experimental.GraphsyncEnabled has been removed in Kubo 0.25.0, please remove this key. For more details see https://github.com/ipfs/kubo/pull/9747.\")\n\t\tdefault:\n\t\t\treturn err\n\t\t}\n\t}\n}\n" }, { "path": "config/types_test.go", "content": "package config\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestOptionalDuration(t *testing.T) {\n\tmakeDurationPointer := func(d time.Duration) *time.Duration { return &d }\n\n\tt.Run(\"marshalling and unmarshalling\", func(t *testing.T) {\n\t\tout, err := json.Marshal(OptionalDuration{value: makeDurationPointer(time.Second)})\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\texpected := \"\\\"1s\\\"\"\n\t\tif string(out) != expected {\n\t\t\tt.Fatalf(\"expected %s, got %s\", expected, string(out))\n\t\t}\n\t\tvar d OptionalDuration\n\n\t\tif err := json.Unmarshal(out, &d); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif *d.value != time.Second {\n\t\t\tt.Fatal(\"expected a second\")\n\t\t}\n\t})\n\n\tt.Run(\"default value\", func(t *testing.T) {\n\t\tfor _, jsonStr := range []string{\"null\", \"\\\"null\\\"\", \"\\\"\\\"\", \"\\\"default\\\"\"} {\n\t\t\tvar d OptionalDuration\n\t\t\tif !d.IsDefault() {\n\t\t\t\tt.Fatal(\"expected value to be the default initially\")\n\t\t\t}\n\t\t\tif err := json.Unmarshal([]byte(jsonStr), &d); err != nil {\n\t\t\t\tt.Fatalf(\"%s failed to unmarshall with %s\", jsonStr, err)\n\t\t\t}\n\t\t\tif dur := d.WithDefault(time.Hour); dur != time.Hour {\n\t\t\t\tt.Fatalf(\"expected default value to be used, got %s\", dur)\n\t\t\t}\n\t\t\tif !d.IsDefault() {\n\t\t\t\tt.Fatal(\"expected value to be the default\")\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"omitempty with default value\", func(t *testing.T) {\n\t\ttype Foo struct {\n\t\t\tD *OptionalDuration `json:\",omitempty\"`\n\t\t}\n\t\t// marshall to JSON without empty field\n\t\tout, err := json.Marshal(new(Foo))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif string(out) != \"{}\" {\n\t\t\tt.Fatalf(\"expected omitempty to omit the duration, got %s\", out)\n\t\t}\n\t\t// unmarshall missing value and get the default\n\t\tvar foo2 Foo\n\t\tif err := json.Unmarshal(out, &foo2); err != nil {\n\t\t\tt.Fatalf(\"%s failed to unmarshall with %s\", string(out), err)\n\t\t}\n\t\tif dur := foo2.D.WithDefault(time.Hour); dur != time.Hour {\n\t\t\tt.Fatalf(\"expected default value to be used, got %s\", dur)\n\t\t}\n\t\tif !foo2.D.IsDefault() {\n\t\t\tt.Fatal(\"expected value to be the default\")\n\t\t}\n\t})\n\n\tt.Run(\"roundtrip including the default values\", func(t *testing.T) {\n\t\tfor jsonStr, goValue := range map[string]OptionalDuration{\n\t\t\t// there are various footguns user can hit, normalize them to the canonical default\n\t\t\t\"null\": {}, // JSON null → default value\n\t\t\t\"\\\"null\\\"\": {}, // JSON string \"null\" sent/set by \"ipfs config\" cli → default value\n\t\t\t\"\\\"default\\\"\": {}, // explicit \"default\" as string\n\t\t\t\"\\\"\\\"\": {}, // user removed custom value, empty string should also parse as default\n\t\t\t\"\\\"1s\\\"\": {value: makeDurationPointer(time.Second)},\n\t\t\t\"\\\"42h1m3s\\\"\": {value: makeDurationPointer(42*time.Hour + 1*time.Minute + 3*time.Second)},\n\t\t} {\n\t\t\tvar d OptionalDuration\n\t\t\terr := json.Unmarshal([]byte(jsonStr), &d)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\tif goValue.value == nil && d.value == nil {\n\t\t\t} else if goValue.value == nil && d.value != nil {\n\t\t\t\tt.Errorf(\"expected nil for %s, got %s\", jsonStr, d)\n\t\t\t} else if *d.value != *goValue.value {\n\t\t\t\tt.Fatalf(\"expected %s for %s, got %s\", goValue, jsonStr, d)\n\t\t\t}\n\n\t\t\t// Test Reverse\n\t\t\tout, err := json.Marshal(goValue)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\tif goValue.value == nil {\n\t\t\t\tif !bytes.Equal(out, []byte(\"null\")) {\n\t\t\t\t\tt.Fatalf(\"expected JSON null for %s, got %s\", jsonStr, string(out))\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif string(out) != jsonStr {\n\t\t\t\tt.Fatalf(\"expected %s, got %s\", jsonStr, string(out))\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"invalid duration values\", func(t *testing.T) {\n\t\tfor _, invalid := range []string{\n\t\t\t\"\\\"s\\\"\", \"\\\"1ę\\\"\", \"\\\"-1\\\"\", \"\\\"1H\\\"\", \"\\\"day\\\"\",\n\t\t} {\n\t\t\tvar d OptionalDuration\n\t\t\terr := json.Unmarshal([]byte(invalid), &d)\n\t\t\tif err == nil {\n\t\t\t\tt.Errorf(\"expected to fail to decode %s as an OptionalDuration, got %s instead\", invalid, d)\n\t\t\t}\n\t\t}\n\t})\n}\n\nfunc TestOneStrings(t *testing.T) {\n\tout, err := json.Marshal(Strings{\"one\"})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"\\\"one\\\"\"\n\tif string(out) != expected {\n\t\tt.Fatalf(\"expected %s, got %s\", expected, string(out))\n\t}\n}\n\nfunc TestNoStrings(t *testing.T) {\n\tout, err := json.Marshal(Strings{})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"null\"\n\tif string(out) != expected {\n\t\tt.Fatalf(\"expected %s, got %s\", expected, string(out))\n\t}\n}\n\nfunc TestManyStrings(t *testing.T) {\n\tout, err := json.Marshal(Strings{\"one\", \"two\"})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"[\\\"one\\\",\\\"two\\\"]\"\n\tif string(out) != expected {\n\t\tt.Fatalf(\"expected %s, got %s\", expected, string(out))\n\t}\n}\n\nfunc TestFunkyStrings(t *testing.T) {\n\ttoParse := \" [ \\\"one\\\", \\\"two\\\" ] \"\n\tvar s Strings\n\tif err := json.Unmarshal([]byte(toParse), &s); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(s) != 2 || s[0] != \"one\" && s[1] != \"two\" {\n\t\tt.Fatalf(\"unexpected result: %v\", s)\n\t}\n}\n\nfunc TestFlag(t *testing.T) {\n\t// make sure we have the right zero value.\n\tvar defaultFlag Flag\n\tif defaultFlag != Default {\n\t\tt.Errorf(\"expected default flag to be %q, got %q\", Default, defaultFlag)\n\t}\n\n\tif defaultFlag.WithDefault(true) != true {\n\t\tt.Error(\"expected default & true to be true\")\n\t}\n\n\tif defaultFlag.WithDefault(false) != false {\n\t\tt.Error(\"expected default & false to be false\")\n\t}\n\n\tif True.WithDefault(false) != true {\n\t\tt.Error(\"default should only apply to default\")\n\t}\n\n\tif False.WithDefault(true) != false {\n\t\tt.Error(\"default should only apply to default\")\n\t}\n\n\tif True.WithDefault(true) != true {\n\t\tt.Error(\"true & true is true\")\n\t}\n\n\tif False.WithDefault(true) != false {\n\t\tt.Error(\"false & false is false\")\n\t}\n\n\tfor jsonStr, goValue := range map[string]Flag{\n\t\t\"null\": Default,\n\t\t\"true\": True,\n\t\t\"false\": False,\n\t} {\n\t\tvar d Flag\n\t\terr := json.Unmarshal([]byte(jsonStr), &d)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif d != goValue {\n\t\t\tt.Fatalf(\"expected %s, got %s\", goValue, d)\n\t\t}\n\n\t\t// Reverse\n\t\tout, err := json.Marshal(goValue)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif string(out) != jsonStr {\n\t\t\tt.Fatalf(\"expected %s, got %s\", jsonStr, string(out))\n\t\t}\n\t}\n\n\ttype Foo struct {\n\t\tF Flag `json:\",omitempty\"`\n\t}\n\tout, err := json.Marshal(new(Foo))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"{}\"\n\tif string(out) != expected {\n\t\tt.Fatal(\"expected omitempty to omit the flag\")\n\t}\n}\n\nfunc TestPriority(t *testing.T) {\n\t// make sure we have the right zero value.\n\tvar defaultPriority Priority\n\tif defaultPriority != DefaultPriority {\n\t\tt.Errorf(\"expected default priority to be %q, got %q\", DefaultPriority, defaultPriority)\n\t}\n\n\tif _, ok := defaultPriority.WithDefault(Disabled); ok {\n\t\tt.Error(\"should have been disabled\")\n\t}\n\n\tif p, ok := defaultPriority.WithDefault(1); !ok || p != 1 {\n\t\tt.Errorf(\"priority should have been 1, got %d\", p)\n\t}\n\n\tif p, ok := defaultPriority.WithDefault(DefaultPriority); !ok || p != 0 {\n\t\tt.Errorf(\"priority should have been 0, got %d\", p)\n\t}\n\n\tfor jsonStr, goValue := range map[string]Priority{\n\t\t\"null\": DefaultPriority,\n\t\t\"false\": Disabled,\n\t\t\"1\": 1,\n\t\t\"2\": 2,\n\t\t\"100\": 100,\n\t} {\n\t\tvar d Priority\n\t\terr := json.Unmarshal([]byte(jsonStr), &d)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif d != goValue {\n\t\t\tt.Fatalf(\"expected %s, got %s\", goValue, d)\n\t\t}\n\n\t\t// Reverse\n\t\tout, err := json.Marshal(goValue)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif string(out) != jsonStr {\n\t\t\tt.Fatalf(\"expected %s, got %s\", jsonStr, string(out))\n\t\t}\n\t}\n\n\ttype Foo struct {\n\t\tP Priority `json:\",omitempty\"`\n\t}\n\tout, err := json.Marshal(new(Foo))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"{}\"\n\tif string(out) != expected {\n\t\tt.Fatal(\"expected omitempty to omit the flag\")\n\t}\n\tfor _, invalid := range []string{\n\t\t\"0\", \"-1\", \"-2\", \"1.1\", \"0.0\",\n\t} {\n\t\tvar p Priority\n\t\terr := json.Unmarshal([]byte(invalid), &p)\n\t\tif err == nil {\n\t\t\tt.Errorf(\"expected to fail to decode %s as a priority\", invalid)\n\t\t}\n\t}\n}\n\nfunc TestOptionalInteger(t *testing.T) {\n\tmakeInt64Pointer := func(v int64) *int64 {\n\t\treturn &v\n\t}\n\n\tvar defaultOptionalInt OptionalInteger\n\tif !defaultOptionalInt.IsDefault() {\n\t\tt.Fatal(\"should be the default\")\n\t}\n\tif val := defaultOptionalInt.WithDefault(0); val != 0 {\n\t\tt.Errorf(\"optional integer should have been 0, got %d\", val)\n\t}\n\n\tif val := defaultOptionalInt.WithDefault(1); val != 1 {\n\t\tt.Errorf(\"optional integer should have been 1, got %d\", val)\n\t}\n\n\tif val := defaultOptionalInt.WithDefault(-1); val != -1 {\n\t\tt.Errorf(\"optional integer should have been -1, got %d\", val)\n\t}\n\n\tvar filledInt OptionalInteger\n\tfilledInt = OptionalInteger{value: makeInt64Pointer(1)}\n\tif filledInt.IsDefault() {\n\t\tt.Fatal(\"should not be the default\")\n\t}\n\tif val := filledInt.WithDefault(0); val != 1 {\n\t\tt.Errorf(\"optional integer should have been 1, got %d\", val)\n\t}\n\n\tif val := filledInt.WithDefault(-1); val != 1 {\n\t\tt.Errorf(\"optional integer should have been 1, got %d\", val)\n\t}\n\n\tfilledInt = OptionalInteger{value: makeInt64Pointer(0)}\n\tif val := filledInt.WithDefault(1); val != 0 {\n\t\tt.Errorf(\"optional integer should have been 0, got %d\", val)\n\t}\n\n\tfor jsonStr, goValue := range map[string]OptionalInteger{\n\t\t\"null\": {},\n\t\t\"0\": {value: makeInt64Pointer(0)},\n\t\t\"1\": {value: makeInt64Pointer(1)},\n\t\t\"-1\": {value: makeInt64Pointer(-1)},\n\t} {\n\t\tvar d OptionalInteger\n\t\terr := json.Unmarshal([]byte(jsonStr), &d)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif goValue.value == nil && d.value == nil {\n\t\t} else if goValue.value == nil && d.value != nil {\n\t\t\tt.Errorf(\"expected default, got %s\", d)\n\t\t} else if *d.value != *goValue.value {\n\t\t\tt.Fatalf(\"expected %s, got %s\", goValue, d)\n\t\t}\n\n\t\t// Reverse\n\t\tout, err := json.Marshal(goValue)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif string(out) != jsonStr {\n\t\t\tt.Fatalf(\"expected %s, got %s\", jsonStr, string(out))\n\t\t}\n\t}\n\n\t// marshal with omitempty\n\ttype Foo struct {\n\t\tI *OptionalInteger `json:\",omitempty\"`\n\t}\n\tout, err := json.Marshal(new(Foo))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"{}\"\n\tif string(out) != expected {\n\t\tt.Fatal(\"expected omitempty to omit the optional integer\")\n\t}\n\n\t// unmarshal from omitempty output and get default value\n\tvar foo2 Foo\n\tif err := json.Unmarshal(out, &foo2); err != nil {\n\t\tt.Fatalf(\"%s failed to unmarshall with %s\", string(out), err)\n\t}\n\tif i := foo2.I.WithDefault(42); i != 42 {\n\t\tt.Fatalf(\"expected default value to be used, got %d\", i)\n\t}\n\tif !foo2.I.IsDefault() {\n\t\tt.Fatal(\"expected value to be the default\")\n\t}\n\n\t// test invalid values\n\tfor _, invalid := range []string{\n\t\t\"foo\", \"-1.1\", \"1.1\", \"0.0\", \"[]\",\n\t} {\n\t\tvar p OptionalInteger\n\t\terr := json.Unmarshal([]byte(invalid), &p)\n\t\tif err == nil {\n\t\t\tt.Errorf(\"expected to fail to decode %s as a priority\", invalid)\n\t\t}\n\t}\n}\n\nfunc TestOptionalString(t *testing.T) {\n\tmakeStringPointer := func(v string) *string {\n\t\treturn &v\n\t}\n\n\tvar defaultOptionalString OptionalString\n\tif !defaultOptionalString.IsDefault() {\n\t\tt.Fatal(\"should be the default\")\n\t}\n\tif val := defaultOptionalString.WithDefault(\"\"); val != \"\" {\n\t\tt.Errorf(\"optional string should have been empty, got %s\", val)\n\t}\n\tif val := defaultOptionalString.String(); val != \"default\" {\n\t\tt.Fatalf(\"default optional string should be the 'default' string, got %s\", val)\n\t}\n\tif val := defaultOptionalString.WithDefault(\"foo\"); val != \"foo\" {\n\t\tt.Errorf(\"optional string should have been foo, got %s\", val)\n\t}\n\n\tvar filledStr OptionalString\n\tfilledStr = OptionalString{value: makeStringPointer(\"foo\")}\n\tif filledStr.IsDefault() {\n\t\tt.Fatal(\"should not be the default\")\n\t}\n\tif val := filledStr.WithDefault(\"bar\"); val != \"foo\" {\n\t\tt.Errorf(\"optional string should have been foo, got %s\", val)\n\t}\n\tif val := filledStr.String(); val != \"foo\" {\n\t\tt.Fatalf(\"optional string should have been foo, got %s\", val)\n\t}\n\tfilledStr = OptionalString{value: makeStringPointer(\"\")}\n\tif val := filledStr.WithDefault(\"foo\"); val != \"\" {\n\t\tt.Errorf(\"optional string should have been 0, got %s\", val)\n\t}\n\n\tfor jsonStr, goValue := range map[string]OptionalString{\n\t\t\"null\": {},\n\t\t\"\\\"0\\\"\": {value: makeStringPointer(\"0\")},\n\t\t\"\\\"\\\"\": {value: makeStringPointer(\"\")},\n\t\t`\"1\"`: {value: makeStringPointer(\"1\")},\n\t\t`\"-1\"`: {value: makeStringPointer(\"-1\")},\n\t\t`\"qwerty\"`: {value: makeStringPointer(\"qwerty\")},\n\t} {\n\t\tvar d OptionalString\n\t\terr := json.Unmarshal([]byte(jsonStr), &d)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif goValue.value == nil && d.value == nil {\n\t\t} else if goValue.value == nil && d.value != nil {\n\t\t\tt.Errorf(\"expected default, got %s\", d)\n\t\t} else if *d.value != *goValue.value {\n\t\t\tt.Fatalf(\"expected %s, got %s\", goValue, d)\n\t\t}\n\n\t\t// Reverse\n\t\tout, err := json.Marshal(goValue)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif string(out) != jsonStr {\n\t\t\tt.Fatalf(\"expected %s, got %s\", jsonStr, string(out))\n\t\t}\n\t}\n\n\t// marshal with omitempty\n\ttype Foo struct {\n\t\tS *OptionalString `json:\",omitempty\"`\n\t}\n\tout, err := json.Marshal(new(Foo))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\texpected := \"{}\"\n\tif string(out) != expected {\n\t\tt.Fatal(\"expected omitempty to omit the optional integer\")\n\t}\n\t// unmarshal from omitempty output and get default value\n\tvar foo2 Foo\n\tif err := json.Unmarshal(out, &foo2); err != nil {\n\t\tt.Fatalf(\"%s failed to unmarshall with %s\", string(out), err)\n\t}\n\tif s := foo2.S.WithDefault(\"foo\"); s != \"foo\" {\n\t\tt.Fatalf(\"expected default value to be used, got %s\", s)\n\t}\n\tif !foo2.S.IsDefault() {\n\t\tt.Fatal(\"expected value to be the default\")\n\t}\n\n\tfor _, invalid := range []string{\n\t\t\"[]\", \"{}\", \"0\", \"a\", \"'b'\",\n\t} {\n\t\tvar p OptionalString\n\t\terr := json.Unmarshal([]byte(invalid), &p)\n\t\tif err == nil {\n\t\t\tt.Errorf(\"expected to fail to decode %s as an optional string\", invalid)\n\t\t}\n\t}\n}\n\nfunc TestOptionalBytes(t *testing.T) {\n\tmakeStringPointer := func(v string) *string { return &v }\n\n\tt.Run(\"default value\", func(t *testing.T) {\n\t\tvar b OptionalBytes\n\t\tassert.True(t, b.IsDefault())\n\t\tassert.Equal(t, uint64(0), b.WithDefault(0))\n\t\tassert.Equal(t, uint64(1024), b.WithDefault(1024))\n\t\tassert.Equal(t, \"default\", b.String())\n\t})\n\n\tt.Run(\"non-default value\", func(t *testing.T) {\n\t\tb := OptionalBytes{OptionalString{value: makeStringPointer(\"1MiB\")}}\n\t\tassert.False(t, b.IsDefault())\n\t\tassert.Equal(t, uint64(1048576), b.WithDefault(512))\n\t\tassert.Equal(t, \"1MiB\", b.String())\n\t})\n\n\tt.Run(\"JSON roundtrip\", func(t *testing.T) {\n\t\ttestCases := []struct {\n\t\t\tjsonInput string\n\t\t\tjsonOutput string\n\t\t\texpectedValue string\n\t\t}{\n\t\t\t{\"null\", \"null\", \"\"},\n\t\t\t{\"\\\"256KiB\\\"\", \"\\\"256KiB\\\"\", \"256KiB\"},\n\t\t\t{\"\\\"1MiB\\\"\", \"\\\"1MiB\\\"\", \"1MiB\"},\n\t\t\t{\"\\\"5GiB\\\"\", \"\\\"5GiB\\\"\", \"5GiB\"},\n\t\t\t{\"\\\"256KB\\\"\", \"\\\"256KB\\\"\", \"256KB\"},\n\t\t\t{\"1048576\", \"\\\"1048576\\\"\", \"1048576\"},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.jsonInput, func(t *testing.T) {\n\t\t\t\tvar b OptionalBytes\n\t\t\t\terr := json.Unmarshal([]byte(tc.jsonInput), &b)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tif tc.expectedValue == \"\" {\n\t\t\t\t\tassert.Nil(t, b.value)\n\t\t\t\t} else {\n\t\t\t\t\trequire.NotNil(t, b.value)\n\t\t\t\t\tassert.Equal(t, tc.expectedValue, *b.value)\n\t\t\t\t}\n\n\t\t\t\tout, err := json.Marshal(b)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.jsonOutput, string(out))\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"parsing byte sizes\", func(t *testing.T) {\n\t\ttestCases := []struct {\n\t\t\tinput string\n\t\t\texpected uint64\n\t\t}{\n\t\t\t{\"256KiB\", 262144},\n\t\t\t{\"1MiB\", 1048576},\n\t\t\t{\"5GiB\", 5368709120},\n\t\t\t{\"256KB\", 256000},\n\t\t\t{\"1048576\", 1048576},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.input, func(t *testing.T) {\n\t\t\t\tvar b OptionalBytes\n\t\t\t\terr := json.Unmarshal([]byte(\"\\\"\"+tc.input+\"\\\"\"), &b)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expected, b.WithDefault(0))\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"omitempty\", func(t *testing.T) {\n\t\ttype Foo struct {\n\t\t\tB *OptionalBytes `json:\",omitempty\"`\n\t\t}\n\n\t\tout, err := json.Marshal(new(Foo))\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"{}\", string(out))\n\n\t\tvar foo2 Foo\n\t\terr = json.Unmarshal(out, &foo2)\n\t\trequire.NoError(t, err)\n\n\t\tif foo2.B != nil {\n\t\t\tassert.Equal(t, uint64(1024), foo2.B.WithDefault(1024))\n\t\t\tassert.True(t, foo2.B.IsDefault())\n\t\t} else {\n\t\t\t// When field is omitted, pointer is nil which is also considered default\n\t\t\tt.Log(\"B is nil, which is acceptable for omitempty\")\n\t\t}\n\t})\n\n\tt.Run(\"invalid values\", func(t *testing.T) {\n\t\tinvalidInputs := []string{\n\t\t\t\"\\\"5XiB\\\"\", \"\\\"invalid\\\"\", \"\\\"\\\"\", \"[]\", \"{}\",\n\t\t}\n\n\t\tfor _, invalid := range invalidInputs {\n\t\t\tt.Run(invalid, func(t *testing.T) {\n\t\t\t\tvar b OptionalBytes\n\t\t\t\terr := json.Unmarshal([]byte(invalid), &b)\n\t\t\t\tassert.Error(t, err)\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"panic on invalid stored value\", func(t *testing.T) {\n\t\t// This tests that if somehow an invalid value gets stored\n\t\t// (bypassing UnmarshalJSON validation), WithDefault will panic\n\t\tinvalidValue := \"invalid-size\"\n\t\tb := OptionalBytes{OptionalString{value: &invalidValue}}\n\n\t\tassert.Panics(t, func() {\n\t\t\tb.WithDefault(1024)\n\t\t}, \"should panic on invalid stored value\")\n\t})\n}\n" }, { "path": "config/version.go", "content": "package config\n\nconst DefaultSwarmCheckPercentThreshold = 5\n\n// Version allows controlling things like custom user agent and update checks.\ntype Version struct {\n\t// Optional suffix to the AgentVersion presented by `ipfs id` and exposed\n\t// via libp2p identify protocol.\n\tAgentSuffix *OptionalString `json:\",omitempty\"`\n\n\t// Detect when to warn about new version when observed via libp2p identify\n\tSwarmCheckEnabled Flag `json:\",omitempty\"`\n\tSwarmCheckPercentThreshold *OptionalInteger `json:\",omitempty\"`\n}\n" }, { "path": "core/.gitignore", "content": ".testdb\n" }, { "path": "core/builder.go", "content": "package core\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/bootstrap\"\n\t\"github.com/ipfs/kubo/core/node\"\n\n\t\"github.com/ipfs/go-metrics-interface\"\n\t\"go.uber.org/dig\"\n\t\"go.uber.org/fx\"\n)\n\n// FXNodeInfo contains information useful for adding fx options.\n// This is the extension point for providing more info/context to fx plugins\n// to make decisions about what options to include.\ntype FXNodeInfo struct {\n\tFXOptions []fx.Option\n}\n\n// fxOptFunc takes in some info about the IPFS node and returns the full set of fx opts to use.\ntype fxOptFunc func(FXNodeInfo) ([]fx.Option, error)\n\nvar fxOptionFuncs []fxOptFunc\n\n// RegisterFXOptionFunc registers a function that is run before the fx app is initialized.\n// Functions are invoked in the order they are registered,\n// and the resulting options are passed into the next function's FXNodeInfo.\n//\n// Note that these are applied globally, by all invocations of NewNode.\n// There are multiple places in Kubo that construct nodes, such as:\n// - Repo initialization\n// - Daemon initialization\n// - When running migrations\n// - etc.\n//\n// If your fx options are doing anything sophisticated, you should keep this in mind.\n//\n// For example, if you plug in a blockservice that disallows non-allowlisted CIDs,\n// this may break migrations that fetch migration code over IPFS.\nfunc RegisterFXOptionFunc(optFunc fxOptFunc) {\n\tfxOptionFuncs = append(fxOptionFuncs, optFunc)\n}\n\n// from https://stackoverflow.com/a/59348871\ntype valueContext struct {\n\tcontext.Context\n}\n\nfunc (valueContext) Deadline() (deadline time.Time, ok bool) { return }\nfunc (valueContext) Done() <-chan struct{} { return nil }\nfunc (valueContext) Err() error { return nil }\n\ntype BuildCfg = node.BuildCfg // Alias for compatibility until we properly refactor the constructor interface\n\n// NewNode constructs and returns an IpfsNode using the given cfg.\nfunc NewNode(ctx context.Context, cfg *BuildCfg) (*IpfsNode, error) {\n\t// save this context as the \"lifetime\" ctx.\n\tlctx := ctx\n\n\t// derive a new context that ignores cancellations from the lifetime ctx.\n\tctx, cancel := context.WithCancel(valueContext{ctx})\n\n\t// add a metrics scope.\n\tctx = metrics.CtxScope(ctx, \"ipfs\")\n\n\tn := &IpfsNode{\n\t\tctx: ctx,\n\t}\n\n\topts := []fx.Option{\n\t\tnode.IPFS(ctx, cfg),\n\t\tfx.NopLogger,\n\t}\n\tfor _, optFunc := range fxOptionFuncs {\n\t\tvar err error\n\t\topts, err = optFunc(FXNodeInfo{FXOptions: opts})\n\t\tif err != nil {\n\t\t\tcancel()\n\t\t\treturn nil, fmt.Errorf(\"building fx opts: %w\", err)\n\t\t}\n\t}\n\t//nolint:staticcheck // https://github.com/ipfs/kubo/pull/9423#issuecomment-1341038770\n\topts = append(opts, fx.Extract(n))\n\n\tapp := fx.New(opts...)\n\n\tvar once sync.Once\n\tvar stopErr error\n\tn.stop = func() error {\n\t\tonce.Do(func() {\n\t\t\tstopErr = app.Stop(context.Background())\n\t\t\tif stopErr != nil {\n\t\t\t\tlog.Error(\"failure on stop: \", stopErr)\n\t\t\t}\n\t\t\t// Cancel the context _after_ the app has stopped.\n\t\t\tcancel()\n\t\t})\n\t\treturn stopErr\n\t}\n\tn.IsOnline = cfg.Online\n\n\tgo func() {\n\t\t// Shut down the application if the lifetime context is canceled.\n\t\t// NOTE: we _should_ stop the application by calling `Close()`\n\t\t// on the process. But we currently manage everything with contexts.\n\t\tselect {\n\t\tcase <-lctx.Done():\n\t\t\terr := n.stop()\n\t\t\tif err != nil {\n\t\t\t\tlog.Error(\"failure on stop: \", err)\n\t\t\t}\n\t\tcase <-ctx.Done():\n\t\t}\n\t}()\n\n\tif app.Err() != nil {\n\t\treturn nil, logAndUnwrapFxError(app.Err())\n\t}\n\n\tif err := app.Start(ctx); err != nil {\n\t\treturn nil, logAndUnwrapFxError(err)\n\t}\n\n\t// TODO: How soon will bootstrap move to libp2p?\n\tif !cfg.Online {\n\t\treturn n, nil\n\t}\n\n\treturn n, n.Bootstrap(bootstrap.DefaultBootstrapConfig)\n}\n\n// Log the entire `app.Err()` but return only the innermost one to the user\n// given the full error can be very long (as it can expose the entire build\n// graph in a single string).\n//\n// The fx.App error exposed through `app.Err()` normally contains un-exported\n// errors from its low-level `dig` package:\n// * https://github.com/uber-go/dig/blob/5e5a20d/error.go#L82\n// These usually wrap themselves in many layers to expose where in the build\n// chain did the error happen. Although useful for a developer that needs to\n// debug it, it can be very confusing for a user that just wants the IPFS error\n// that he can probably fix without being aware of the entire chain.\n// Unwrapping everything is not the best solution as there can be useful\n// information in the intermediate errors, mainly in the next to last error\n// that locates which component is the build error coming from, but it's the\n// best we can do at the moment given all errors in dig are private and we\n// just have the generic `RootCause` API.\nfunc logAndUnwrapFxError(fxAppErr error) error {\n\tif fxAppErr == nil {\n\t\treturn nil\n\t}\n\n\tlog.Error(\"constructing the node: \", fxAppErr)\n\n\terr := fxAppErr\n\tfor {\n\t\textractedErr := dig.RootCause(err)\n\t\t// Note that the `RootCause` name is misleading as it just unwraps only\n\t\t// *one* error layer at a time, so we need to continuously call it.\n\t\tif !reflect.TypeOf(extractedErr).Comparable() {\n\t\t\t// Some internal errors are not comparable (e.g., `dig.errMissingTypes`\n\t\t\t// which is a slice) and we can't go further.\n\t\t\tbreak\n\t\t}\n\t\tif extractedErr == err {\n\t\t\t// We didn't unwrap any new error in the last call, reached the innermost one.\n\t\t\tbreak\n\t\t}\n\t\terr = extractedErr\n\t}\n\n\treturn fmt.Errorf(\"constructing the node (see log for full detail): %w\", err)\n}\n" }, { "path": "core/commands/active.go", "content": "package commands\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"slices\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nconst (\n\tverboseOptionName = \"verbose\"\n)\n\nvar ActiveReqsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List commands run on this IPFS node.\",\n\t\tShortDescription: `\nLists running and recently run commands.\n`,\n\t},\n\tNoLocal: true,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tctx := env.(*oldcmds.Context)\n\t\treturn cmds.EmitOnce(res, ctx.ReqLog.Report())\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(verboseOptionName, \"v\", \"Print extra information.\"),\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"clear\": clearInactiveCmd,\n\t\t\"set-time\": setRequestClearCmd,\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *[]*cmds.ReqLogEntry) error {\n\t\t\tverbose, _ := req.Options[verboseOptionName].(bool)\n\n\t\t\ttw := tabwriter.NewWriter(w, 4, 4, 2, ' ', 0)\n\t\t\tif verbose {\n\t\t\t\tfmt.Fprint(tw, \"ID\\t\")\n\t\t\t}\n\t\t\tfmt.Fprint(tw, \"Command\\t\")\n\t\t\tif verbose {\n\t\t\t\tfmt.Fprint(tw, \"Arguments\\tOptions\\t\")\n\t\t\t}\n\t\t\tfmt.Fprintln(tw, \"Active\\tStartTime\\tRunTime\")\n\n\t\t\tfor _, req := range *out {\n\t\t\t\tif verbose {\n\t\t\t\t\tfmt.Fprintf(tw, \"%d\\t\", req.ID)\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(tw, \"%s\\t\", req.Command)\n\t\t\t\tif verbose {\n\t\t\t\t\tfmt.Fprintf(tw, \"%v\\t[\", req.Args)\n\t\t\t\t\tvar keys []string\n\t\t\t\t\tfor k := range req.Options {\n\t\t\t\t\t\tkeys = append(keys, k)\n\t\t\t\t\t}\n\t\t\t\t\tslices.Sort(keys)\n\n\t\t\t\t\tfor _, k := range keys {\n\t\t\t\t\t\tfmt.Fprintf(tw, \"%s=%v,\", k, req.Options[k])\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(tw, \"]\\t\")\n\t\t\t\t}\n\n\t\t\t\tvar live time.Duration\n\t\t\t\tif req.Active {\n\t\t\t\t\tlive = time.Since(req.StartTime)\n\t\t\t\t} else {\n\t\t\t\t\tlive = req.EndTime.Sub(req.StartTime)\n\t\t\t\t}\n\t\t\t\tt := req.StartTime.Format(time.Stamp)\n\t\t\t\tfmt.Fprintf(tw, \"%t\\t%s\\t%s\\n\", req.Active, t, live)\n\t\t\t}\n\t\t\ttw.Flush()\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: []*cmds.ReqLogEntry{},\n}\n\nvar clearInactiveCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Clear inactive requests from the log.\",\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tctx := env.(*oldcmds.Context)\n\t\tctx.ReqLog.ClearInactive()\n\t\treturn nil\n\t},\n}\n\nvar setRequestClearCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Set how long to keep inactive requests in the log.\",\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"time\", true, false, \"Time to keep inactive requests in log.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\ttval, err := time.ParseDuration(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tctx := env.(*oldcmds.Context)\n\t\tctx.ReqLog.SetKeepTime(tval)\n\n\t\treturn nil\n\t},\n}\n" }, { "path": "core/commands/add.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\tgopath \"path\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\t\"github.com/cheggaaa/pb\"\n\t\"github.com/ipfs/boxo/files\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\tmfs \"github.com/ipfs/boxo/mfs\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/boxo/verifcid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\n// ErrDepthLimitExceeded indicates that the max depth has been exceeded.\nvar ErrDepthLimitExceeded = errors.New(\"depth limit exceeded\")\n\ntype AddEvent struct {\n\tName string\n\tHash string `json:\",omitempty\"`\n\tBytes int64 `json:\",omitempty\"`\n\tSize string `json:\",omitempty\"`\n\tMode string `json:\",omitempty\"`\n\tMtime int64 `json:\",omitempty\"`\n\tMtimeNsecs int `json:\",omitempty\"`\n}\n\nconst (\n\tpinNameOptionName = \"pin-name\"\n\tquietOptionName = \"quiet\"\n\tquieterOptionName = \"quieter\"\n\tsilentOptionName = \"silent\"\n\tprogressOptionName = \"progress\"\n\ttrickleOptionName = \"trickle\"\n\twrapOptionName = \"wrap-with-directory\"\n\tonlyHashOptionName = \"only-hash\"\n\tchunkerOptionName = \"chunker\"\n\tpinOptionName = \"pin\"\n\trawLeavesOptionName = \"raw-leaves\"\n\tmaxFileLinksOptionName = \"max-file-links\"\n\tmaxDirectoryLinksOptionName = \"max-directory-links\"\n\tmaxHAMTFanoutOptionName = \"max-hamt-fanout\"\n\tnoCopyOptionName = \"nocopy\"\n\tfstoreCacheOptionName = \"fscache\"\n\tcidVersionOptionName = \"cid-version\"\n\thashOptionName = \"hash\"\n\tinlineOptionName = \"inline\"\n\tinlineLimitOptionName = \"inline-limit\"\n\ttoFilesOptionName = \"to-files\"\n\n\tpreserveModeOptionName = \"preserve-mode\"\n\tpreserveMtimeOptionName = \"preserve-mtime\"\n\tmodeOptionName = \"mode\"\n\tmtimeOptionName = \"mtime\"\n\tmtimeNsecsOptionName = \"mtime-nsecs\"\n\tfastProvideRootOptionName = \"fast-provide-root\"\n\tfastProvideWaitOptionName = \"fast-provide-wait\"\n\temptyDirsOptionName = \"empty-dirs\"\n)\n\nconst (\n\tadderOutChanSize = 8\n)\n\nvar AddCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add a file or directory to IPFS.\",\n\t\tShortDescription: `\nAdds the content of to IPFS. Use -r to add directories (recursively).\n\nFAST PROVIDE OPTIMIZATION:\n\nWhen you add content to IPFS, the sweep provider queues it for efficient\nDHT provides over time. While this is resource-efficient, other peers won't\nfind your content immediately after 'ipfs add' completes.\n\nTo make sharing faster, 'ipfs add' does an immediate provide of the root CID\nto the DHT in addition to the regular queue. This complements the sweep provider:\nfast-provide handles the urgent case (root CIDs that users share and reference),\nwhile the sweep provider efficiently provides all blocks according to\nProvide.Strategy over time.\n\nBy default, this immediate provide runs in the background without blocking\nthe command. If you need certainty that the root CID is discoverable before\nthe command returns (e.g., sharing a link immediately), use --fast-provide-wait\nto wait for the provide to complete. Use --fast-provide-root=false to skip\nthis optimization.\n\nThis works best with the sweep provider and accelerated DHT client.\nAutomatically skipped when DHT is not available.\n`,\n\t\tLongDescription: `\nAdds the content of to IPFS. Use -r to add directories.\nNote that directories are added recursively, and big files are chunked,\nto form the IPFS MerkleDAG. Learn more: https://docs.ipfs.tech/concepts/merkle-dag/\n\nIf the daemon is not running, it will just add locally to the repo at $IPFS_PATH.\nIf the daemon is started later, it will be advertised after a few\nseconds when the provide system runs.\n\nBASIC EXAMPLES:\n\nThe wrap option, '-w', wraps the file (or files, if using the\nrecursive option) in a directory. This directory contains only\nthe files which have been added, and means that the file retains\nits filename. For example:\n\n > ipfs add example.jpg\n added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH example.jpg\n > ipfs add example.jpg -w\n added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH example.jpg\n added QmaG4FuMqEBnQNn3C8XJ5bpW8kLs7zq2ZXgHptJHbKDDVx\n\nYou can now refer to the added file in a gateway, like so:\n\n /ipfs/QmaG4FuMqEBnQNn3C8XJ5bpW8kLs7zq2ZXgHptJHbKDDVx/example.jpg\n\nFiles imported with 'ipfs add' are protected from GC (implicit '--pin=true'),\nbut it is up to you to remember the returned CID to get the data back later.\n\nIf you need to back up or transport content-addressed data using a non-IPFS\nmedium, CID can be preserved with CAR files.\nSee 'dag export' and 'dag import' for more information.\n\nMFS INTEGRATION:\n\nPassing '--to-files' creates a reference in Files API (MFS), making it easier\nto find it in the future:\n\n > ipfs files mkdir -p /myfs/dir\n > ipfs add example.jpg --to-files /myfs/dir/\n > ipfs files ls /myfs/dir/\n example.jpg\n\nSee 'ipfs files --help' to learn more about using MFS\nfor keeping track of added files and directories.\n\nSYMLINK HANDLING:\n\nBy default, symbolic links are preserved as UnixFS symlink nodes that store\nthe target path. Use --dereference-symlinks to resolve symlinks to their\ntarget content instead:\n\n > ipfs add -r --dereference-symlinks ./mydir\n\nThis resolves all symlinks, including CLI arguments and those found inside\ndirectories. Symlinks to files become regular file content, symlinks to\ndirectories are traversed and their contents are added.\n\nCHUNKING EXAMPLES:\n\nThe chunker option, '-s', specifies the chunking strategy that dictates\nhow to break files into blocks. Blocks with same content can\nbe deduplicated. Different chunking strategies will produce different\nhashes for the same file. The default is a fixed block size of\n256 * 1024 bytes, 'size-262144'. Alternatively, you can use the\nBuzhash or Rabin fingerprint chunker for content defined chunking by\nspecifying buzhash or rabin-[min]-[avg]-[max] (where min/avg/max refer\nto the desired chunk sizes in bytes), e.g. 'rabin-262144-524288-1048576'.\n\nThe maximum accepted value for 'size-N' and rabin 'max' parameter is\n2MiB minus 256 bytes (2096896 bytes). The 256-byte overhead budget is\nreserved for protobuf/UnixFS framing so that serialized blocks stay\nwithin the 2MiB block size limit from the bitswap spec. The buzhash\nchunker uses a fixed internal maximum of 512KiB and is not affected.\n\nOnly the fixed-size chunker ('size-N') guarantees that the same data\nwill always produce the same CID. The rabin and buzhash chunkers may\nchange their internal parameters in a future release.\n\nThe following examples use very small byte sizes to demonstrate the\nproperties of the different chunkers on a small file. You'll likely\nwant to use a 1024 times larger chunk sizes for most files.\n\n > ipfs add --chunker=size-2048 ipfs-logo.svg\n added QmafrLBfzRLV4XSH1XcaMMeaXEUhDJjmtDfsYU95TrWG87 ipfs-logo.svg\n > ipfs add --chunker=rabin-512-1024-2048 ipfs-logo.svg\n added Qmf1hDN65tR55Ubh2RN1FPxr69xq3giVBz1KApsresY8Gn ipfs-logo.svg\n\nYou can now check what blocks have been created by:\n\n > ipfs ls QmafrLBfzRLV4XSH1XcaMMeaXEUhDJjmtDfsYU95TrWG87\n QmY6yj1GsermExDXoosVE3aSPxdMNYr6aKuw3nA8LoWPRS 2059\n Qmf7ZQeSxq2fJVJbCmgTrLLVN9tDR9Wy5k75DxQKuz5Gyt 1195\n > ipfs ls Qmf1hDN65tR55Ubh2RN1FPxr69xq3giVBz1KApsresY8Gn\n QmY6yj1GsermExDXoosVE3aSPxdMNYr6aKuw3nA8LoWPRS 2059\n QmerURi9k4XzKCaaPbsK6BL5pMEjF7PGphjDvkkjDtsVf3 868\n QmQB28iwSriSUSMqG2nXDTLtdPHgWb4rebBrU7Q1j4vxPv 338\n\nADVANCED CONFIGURATION:\n\nFinally, a note on hash (CID) determinism and 'ipfs add' command.\n\nAlmost all the flags provided by this command will change the final CID, and\nnew flags may be added in the future. It is not guaranteed for the implicit\ndefaults of 'ipfs add' to remain the same in future Kubo releases, or for other\nIPFS software to use the same import parameters as Kubo.\n\nNote: CIDv1 is automatically used when using non-default options like custom\nhash functions or when raw-leaves is explicitly enabled.\n\nUse Import.* configuration options to override global implicit defaults:\nhttps://github.com/ipfs/kubo/blob/master/docs/config.md#import\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"path\", true, true, \"The path to a file to be added to IPFS.\").EnableRecursive().EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\t// Input Processing\n\t\tcmds.OptionRecursivePath, // a builtin option that allows recursive paths (-r, --recursive)\n\t\tcmds.OptionDerefArgs, // DEPRECATED: use --dereference-symlinks instead\n\t\tcmds.OptionStdinName, // a builtin option that optionally allows wrapping stdin into a named file\n\t\tcmds.OptionHidden,\n\t\tcmds.OptionIgnore,\n\t\tcmds.OptionIgnoreRules,\n\t\tcmds.BoolOption(emptyDirsOptionName, \"E\", \"Include empty directories in the import.\").WithDefault(config.DefaultUnixFSIncludeEmptyDirs),\n\t\tcmds.OptionDerefSymlinks, // resolve symlinks to their target content\n\t\t// Output Control\n\t\tcmds.BoolOption(quietOptionName, \"q\", \"Write minimal output.\"),\n\t\tcmds.BoolOption(quieterOptionName, \"Q\", \"Write only final hash.\"),\n\t\tcmds.BoolOption(silentOptionName, \"Write no output.\"),\n\t\tcmds.BoolOption(progressOptionName, \"p\", \"Stream progress data.\"),\n\t\t// Basic Add Behavior\n\t\tcmds.BoolOption(onlyHashOptionName, \"n\", \"Only chunk and hash - do not write to disk.\"),\n\t\tcmds.BoolOption(wrapOptionName, \"w\", \"Wrap files with a directory object.\"),\n\t\tcmds.BoolOption(pinOptionName, \"Pin locally to protect added files from garbage collection.\").WithDefault(true),\n\t\tcmds.StringOption(pinNameOptionName, \"Name to use for the pin. Requires explicit value (e.g., --pin-name=myname).\"),\n\t\t// MFS Integration\n\t\tcmds.StringOption(toFilesOptionName, \"Add reference to Files API (MFS) at the provided path.\"),\n\t\t// CID & Hashing\n\t\tcmds.IntOption(cidVersionOptionName, \"CID version (0 or 1). CIDv1 automatically enables raw-leaves and is required for non-sha2-256 hashes. Default: Import.CidVersion\"),\n\t\tcmds.StringOption(hashOptionName, \"Hash function to use. Implies CIDv1 if not sha2-256. Default: Import.HashFunction\"),\n\t\tcmds.BoolOption(rawLeavesOptionName, \"Use raw blocks for leaf nodes. Note: CIDv1 automatically enables raw-leaves. Default: false for CIDv0, true for CIDv1 (Import.UnixFSRawLeaves)\"),\n\t\t// Chunking & DAG Structure\n\t\tcmds.StringOption(chunkerOptionName, \"s\", \"Chunking algorithm, size-[bytes], rabin-[min]-[avg]-[max] or buzhash. Files larger than chunk size are split into multiple blocks. Default: Import.UnixFSChunker\"),\n\t\tcmds.BoolOption(trickleOptionName, \"t\", \"Use trickle-dag format for dag generation.\"),\n\t\t// Advanced UnixFS Limits\n\t\tcmds.IntOption(maxFileLinksOptionName, \"Limit the maximum number of links in UnixFS file nodes to this value. WARNING: experimental. Default: Import.UnixFSFileMaxLinks\"),\n\t\tcmds.IntOption(maxDirectoryLinksOptionName, \"Limit the maximum number of links in UnixFS basic directory nodes to this value. WARNING: experimental, Import.UnixFSHAMTDirectorySizeThreshold is safer. Default: Import.UnixFSDirectoryMaxLinks\"),\n\t\tcmds.IntOption(maxHAMTFanoutOptionName, \"Limit the maximum number of links of a UnixFS HAMT directory node to this (power of 2, between 8 and 1024). WARNING: experimental, Import.UnixFSHAMTDirectorySizeThreshold is safer. Default: Import.UnixFSHAMTDirectoryMaxFanout\"),\n\t\t// Experimental Features\n\t\tcmds.BoolOption(inlineOptionName, \"Inline small blocks into CIDs. WARNING: experimental\"),\n\t\tcmds.IntOption(inlineLimitOptionName, fmt.Sprintf(\"Maximum block size to inline. Maximum: %d bytes. WARNING: experimental\", verifcid.DefaultMaxIdentityDigestSize)).WithDefault(32),\n\t\tcmds.BoolOption(noCopyOptionName, \"Add the file using filestore. Implies raw-leaves. WARNING: experimental\"),\n\t\tcmds.BoolOption(fstoreCacheOptionName, \"Check the filestore for pre-existing blocks. WARNING: experimental\"),\n\t\tcmds.BoolOption(preserveModeOptionName, \"Apply existing POSIX permissions to created UnixFS entries. WARNING: experimental, forces dag-pb for root block, disables raw-leaves\"),\n\t\tcmds.BoolOption(preserveMtimeOptionName, \"Apply existing POSIX modification time to created UnixFS entries. WARNING: experimental, forces dag-pb for root block, disables raw-leaves\"),\n\t\tcmds.UintOption(modeOptionName, \"Custom POSIX file mode to store in created UnixFS entries. WARNING: experimental, forces dag-pb for root block, disables raw-leaves\"),\n\t\tcmds.Int64Option(mtimeOptionName, \"Custom POSIX modification time to store in created UnixFS entries (seconds before or after the Unix Epoch). WARNING: experimental, forces dag-pb for root block, disables raw-leaves\"),\n\t\tcmds.UintOption(mtimeNsecsOptionName, \"Custom POSIX modification time (optional time fraction in nanoseconds)\"),\n\t\tcmds.BoolOption(fastProvideRootOptionName, \"Immediately provide root CID to DHT in addition to regular queue, for faster discovery. Default: Import.FastProvideRoot\"),\n\t\tcmds.BoolOption(fastProvideWaitOptionName, \"Block until the immediate provide completes before returning. Default: Import.FastProvideWait\"),\n\t},\n\tPreRun: func(req *cmds.Request, env cmds.Environment) error {\n\t\tquiet, _ := req.Options[quietOptionName].(bool)\n\t\tquieter, _ := req.Options[quieterOptionName].(bool)\n\t\tquiet = quiet || quieter\n\t\tsilent, _ := req.Options[silentOptionName].(bool)\n\n\t\tif !quiet && !silent {\n\t\t\t// ipfs cli progress bar defaults to true unless quiet or silent is used\n\t\t\t_, found := req.Options[progressOptionName].(bool)\n\t\t\tif !found {\n\t\t\t\treq.Options[progressOptionName] = true\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err := nd.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tprogress, _ := req.Options[progressOptionName].(bool)\n\t\ttrickle, trickleSet := req.Options[trickleOptionName].(bool)\n\t\twrap, _ := req.Options[wrapOptionName].(bool)\n\t\tonlyHash, _ := req.Options[onlyHashOptionName].(bool)\n\t\tsilent, _ := req.Options[silentOptionName].(bool)\n\t\tchunker, _ := req.Options[chunkerOptionName].(string)\n\t\tdopin, _ := req.Options[pinOptionName].(bool)\n\t\tpinName, pinNameSet := req.Options[pinNameOptionName].(string)\n\t\trawblks, rbset := req.Options[rawLeavesOptionName].(bool)\n\t\tmaxFileLinks, maxFileLinksSet := req.Options[maxFileLinksOptionName].(int)\n\t\tmaxDirectoryLinks, maxDirectoryLinksSet := req.Options[maxDirectoryLinksOptionName].(int)\n\t\tmaxHAMTFanout, maxHAMTFanoutSet := req.Options[maxHAMTFanoutOptionName].(int)\n\t\tvar sizeEstimationMode uio.SizeEstimationMode\n\t\tnocopy, _ := req.Options[noCopyOptionName].(bool)\n\t\tfscache, _ := req.Options[fstoreCacheOptionName].(bool)\n\t\tcidVer, cidVerSet := req.Options[cidVersionOptionName].(int)\n\t\thashFunStr, _ := req.Options[hashOptionName].(string)\n\t\tinline, _ := req.Options[inlineOptionName].(bool)\n\t\tinlineLimit, _ := req.Options[inlineLimitOptionName].(int)\n\n\t\t// Validate inline-limit doesn't exceed the maximum identity digest size\n\t\tif inline && inlineLimit > verifcid.DefaultMaxIdentityDigestSize {\n\t\t\treturn fmt.Errorf(\"inline-limit %d exceeds maximum allowed size of %d bytes\", inlineLimit, verifcid.DefaultMaxIdentityDigestSize)\n\t\t}\n\n\t\t// Validate pin name\n\t\tif pinNameSet {\n\t\t\tif err := cmdutils.ValidatePinName(pinName); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\ttoFilesStr, toFilesSet := req.Options[toFilesOptionName].(string)\n\t\tpreserveMode, _ := req.Options[preserveModeOptionName].(bool)\n\t\tpreserveMtime, _ := req.Options[preserveMtimeOptionName].(bool)\n\t\tmode, _ := req.Options[modeOptionName].(uint)\n\t\tmtime, _ := req.Options[mtimeOptionName].(int64)\n\t\tmtimeNsecs, _ := req.Options[mtimeNsecsOptionName].(uint)\n\t\tfastProvideRoot, fastProvideRootSet := req.Options[fastProvideRootOptionName].(bool)\n\t\tfastProvideWait, fastProvideWaitSet := req.Options[fastProvideWaitOptionName].(bool)\n\t\temptyDirs, _ := req.Options[emptyDirsOptionName].(bool)\n\n\t\t// Note: --dereference-args is deprecated but still works for backwards compatibility.\n\t\t// The help text marks it as DEPRECATED. Users should use --dereference-symlinks instead,\n\t\t// which is a superset (resolves both CLI arg symlinks AND nested symlinks in directories).\n\n\t\t// Wire --trickle from config\n\t\tif !trickleSet && !cfg.Import.UnixFSDAGLayout.IsDefault() {\n\t\t\tlayout := cfg.Import.UnixFSDAGLayout.WithDefault(config.DefaultUnixFSDAGLayout)\n\t\t\ttrickle = layout == config.DAGLayoutTrickle\n\t\t}\n\n\t\tif chunker == \"\" {\n\t\t\tchunker = cfg.Import.UnixFSChunker.WithDefault(config.DefaultUnixFSChunker)\n\t\t}\n\n\t\tif hashFunStr == \"\" {\n\t\t\thashFunStr = cfg.Import.HashFunction.WithDefault(config.DefaultHashFunction)\n\t\t}\n\n\t\tif !cidVerSet && !cfg.Import.CidVersion.IsDefault() {\n\t\t\tcidVerSet = true\n\t\t\tcidVer = int(cfg.Import.CidVersion.WithDefault(config.DefaultCidVersion))\n\t\t}\n\n\t\t// Pin names are only used when explicitly provided via --pin-name=value\n\n\t\tif !rbset && cfg.Import.UnixFSRawLeaves != config.Default {\n\t\t\trbset = true\n\t\t\trawblks = cfg.Import.UnixFSRawLeaves.WithDefault(config.DefaultUnixFSRawLeaves)\n\t\t}\n\n\t\tif !maxFileLinksSet && !cfg.Import.UnixFSFileMaxLinks.IsDefault() {\n\t\t\tmaxFileLinksSet = true\n\t\t\tmaxFileLinks = int(cfg.Import.UnixFSFileMaxLinks.WithDefault(config.DefaultUnixFSFileMaxLinks))\n\t\t}\n\n\t\tif !maxDirectoryLinksSet && !cfg.Import.UnixFSDirectoryMaxLinks.IsDefault() {\n\t\t\tmaxDirectoryLinksSet = true\n\t\t\tmaxDirectoryLinks = int(cfg.Import.UnixFSDirectoryMaxLinks.WithDefault(config.DefaultUnixFSDirectoryMaxLinks))\n\t\t}\n\n\t\tif !maxHAMTFanoutSet && !cfg.Import.UnixFSHAMTDirectoryMaxFanout.IsDefault() {\n\t\t\tmaxHAMTFanoutSet = true\n\t\t\tmaxHAMTFanout = int(cfg.Import.UnixFSHAMTDirectoryMaxFanout.WithDefault(config.DefaultUnixFSHAMTDirectoryMaxFanout))\n\t\t}\n\n\t\t// SizeEstimationMode is always set from config (no CLI flag)\n\t\tsizeEstimationMode = cfg.Import.HAMTSizeEstimationMode()\n\n\t\tfastProvideRoot = config.ResolveBoolFromConfig(fastProvideRoot, fastProvideRootSet, cfg.Import.FastProvideRoot, config.DefaultFastProvideRoot)\n\t\tfastProvideWait = config.ResolveBoolFromConfig(fastProvideWait, fastProvideWaitSet, cfg.Import.FastProvideWait, config.DefaultFastProvideWait)\n\n\t\t// Storing optional mode or mtime (UnixFS 1.5) requires root block\n\t\t// to always be 'dag-pb' and not 'raw'. Below adjusts raw-leaves setting, if possible.\n\t\tif preserveMode || preserveMtime || mode != 0 || mtime != 0 {\n\t\t\t// Error if --raw-leaves flag was explicitly passed by the user.\n\t\t\t// (let user make a decision to manually disable it and retry)\n\t\t\tif rbset && rawblks {\n\t\t\t\treturn fmt.Errorf(\"%s can't be used with UnixFS metadata like mode or modification time\", rawLeavesOptionName)\n\t\t\t}\n\t\t\t// No explicit preference from user, disable raw-leaves and continue\n\t\t\trbset = true\n\t\t\trawblks = false\n\t\t}\n\n\t\tif onlyHash && toFilesSet {\n\t\t\treturn fmt.Errorf(\"%s and %s options are not compatible\", onlyHashOptionName, toFilesOptionName)\n\t\t}\n\t\tif !dopin && pinNameSet {\n\t\t\treturn fmt.Errorf(\"%s option requires %s to be set\", pinNameOptionName, pinOptionName)\n\t\t}\n\t\tif wrap && toFilesSet {\n\t\t\treturn fmt.Errorf(\"%s and %s options are not compatible\", wrapOptionName, toFilesOptionName)\n\t\t}\n\n\t\thashFunCode, ok := mh.Names[strings.ToLower(hashFunStr)]\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"unrecognized hash function: %q\", strings.ToLower(hashFunStr))\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttoadd := req.Files\n\t\tif wrap {\n\t\t\ttoadd = files.NewSliceDirectory([]files.DirEntry{\n\t\t\t\tfiles.FileEntry(\"\", req.Files),\n\t\t\t})\n\t\t}\n\n\t\topts := []options.UnixfsAddOption{\n\t\t\toptions.Unixfs.Hash(hashFunCode),\n\n\t\t\toptions.Unixfs.Inline(inline),\n\t\t\toptions.Unixfs.InlineLimit(inlineLimit),\n\n\t\t\toptions.Unixfs.Chunker(chunker),\n\n\t\t\toptions.Unixfs.Pin(dopin, pinName),\n\t\t\toptions.Unixfs.HashOnly(onlyHash),\n\t\t\toptions.Unixfs.FsCache(fscache),\n\t\t\toptions.Unixfs.Nocopy(nocopy),\n\n\t\t\toptions.Unixfs.Progress(progress),\n\t\t\toptions.Unixfs.Silent(silent),\n\n\t\t\toptions.Unixfs.PreserveMode(preserveMode),\n\t\t\toptions.Unixfs.PreserveMtime(preserveMtime),\n\n\t\t\toptions.Unixfs.IncludeEmptyDirs(emptyDirs),\n\t\t}\n\n\t\tif mode != 0 {\n\t\t\topts = append(opts, options.Unixfs.Mode(os.FileMode(mode)))\n\t\t}\n\n\t\tif mtime != 0 {\n\t\t\topts = append(opts, options.Unixfs.Mtime(mtime, uint32(mtimeNsecs)))\n\t\t} else if mtimeNsecs != 0 {\n\t\t\treturn fmt.Errorf(\"option %q requires %q to be provided as well\", mtimeNsecsOptionName, mtimeOptionName)\n\t\t}\n\n\t\tif cidVerSet {\n\t\t\topts = append(opts, options.Unixfs.CidVersion(cidVer))\n\t\t}\n\n\t\tif rbset {\n\t\t\topts = append(opts, options.Unixfs.RawLeaves(rawblks))\n\t\t}\n\n\t\tif maxFileLinksSet {\n\t\t\topts = append(opts, options.Unixfs.MaxFileLinks(maxFileLinks))\n\t\t}\n\n\t\tif maxDirectoryLinksSet {\n\t\t\topts = append(opts, options.Unixfs.MaxDirectoryLinks(maxDirectoryLinks))\n\t\t}\n\n\t\tif maxHAMTFanoutSet {\n\t\t\topts = append(opts, options.Unixfs.MaxHAMTFanout(maxHAMTFanout))\n\t\t}\n\n\t\t// SizeEstimationMode is always set from config\n\t\topts = append(opts, options.Unixfs.SizeEstimationMode(sizeEstimationMode))\n\n\t\tif trickle {\n\t\t\topts = append(opts, options.Unixfs.Layout(options.TrickleLayout))\n\t\t}\n\n\t\topts = append(opts, nil) // events option placeholder\n\n\t\tipfsNode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tvar added int\n\t\tvar fileAddedToMFS bool\n\t\tvar lastRootCid path.ImmutablePath // Track the root CID for fast-provide\n\t\taddit := toadd.Entries()\n\t\tfor addit.Next() {\n\t\t\t_, dir := addit.Node().(files.Directory)\n\t\t\terrCh := make(chan error, 1)\n\t\t\tevents := make(chan any, adderOutChanSize)\n\t\t\topts[len(opts)-1] = options.Unixfs.Events(events)\n\n\t\t\tgo func() {\n\t\t\t\tvar err error\n\t\t\t\tdefer close(events)\n\t\t\t\tpathAdded, err := api.Unixfs().Add(req.Context, addit.Node(), opts...)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrCh <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\t// Store the root CID for potential fast-provide operation\n\t\t\t\tlastRootCid = pathAdded\n\n\t\t\t\t// creating MFS pointers when optional --to-files is set\n\t\t\t\tif toFilesSet {\n\t\t\t\t\tif addit.Name() == \"\" {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: cannot add unnamed files to MFS\", toFilesOptionName)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\n\t\t\t\t\tif toFilesStr == \"\" {\n\t\t\t\t\t\ttoFilesStr = \"/\"\n\t\t\t\t\t}\n\t\t\t\t\ttoFilesDst, err := checkPath(toFilesStr)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: %w\", toFilesOptionName, err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tdstAsDir := toFilesDst[len(toFilesDst)-1] == '/'\n\n\t\t\t\t\tif dstAsDir {\n\t\t\t\t\t\tmfsNode, err := mfs.Lookup(ipfsNode.FilesRoot, toFilesDst)\n\t\t\t\t\t\t// confirm dst exists\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: MFS destination directory %q does not exist: %w\", toFilesOptionName, toFilesDst, err)\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\t// confirm dst is a dir\n\t\t\t\t\t\tif mfsNode.Type() != mfs.TDir {\n\t\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: MFS destination %q is not a directory\", toFilesOptionName, toFilesDst)\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\t// if MFS destination is a dir, append filename to the dir path\n\t\t\t\t\t\ttoFilesDst += gopath.Base(addit.Name())\n\t\t\t\t\t}\n\n\t\t\t\t\t// error if we try to overwrite a preexisting file destination\n\t\t\t\t\tif fileAddedToMFS && !dstAsDir {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: MFS destination is a file: only one entry can be copied to %q\", toFilesOptionName, toFilesDst)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\n\t\t\t\t\t_, err = mfs.Lookup(ipfsNode.FilesRoot, gopath.Dir(toFilesDst))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: MFS destination parent %q %q does not exist: %w\", toFilesOptionName, toFilesDst, gopath.Dir(toFilesDst), err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\n\t\t\t\t\tvar nodeAdded ipld.Node\n\t\t\t\t\tnodeAdded, err = api.Dag().Get(req.Context, pathAdded.RootCid())\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrCh <- err\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\terr = mfs.PutNode(ipfsNode.FilesRoot, toFilesDst, nodeAdded)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"%s: cannot put node in path %q: %w\", toFilesOptionName, toFilesDst, err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tfileAddedToMFS = true\n\t\t\t\t}\n\t\t\t\terrCh <- err\n\t\t\t}()\n\n\t\t\tfor event := range events {\n\t\t\t\toutput, ok := event.(*coreiface.AddEvent)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn errors.New(\"unknown event type\")\n\t\t\t\t}\n\n\t\t\t\th := \"\"\n\t\t\t\tif (output.Path != path.ImmutablePath{}) {\n\t\t\t\t\th = enc.Encode(output.Path.RootCid())\n\t\t\t\t}\n\n\t\t\t\tif !dir && addit.Name() != \"\" {\n\t\t\t\t\toutput.Name = addit.Name()\n\t\t\t\t} else {\n\t\t\t\t\toutput.Name = gopath.Join(addit.Name(), output.Name)\n\t\t\t\t}\n\n\t\t\t\toutput.Mode = addit.Node().Mode()\n\t\t\t\tif ts := addit.Node().ModTime(); !ts.IsZero() {\n\t\t\t\t\toutput.Mtime = addit.Node().ModTime().Unix()\n\t\t\t\t\toutput.MtimeNsecs = addit.Node().ModTime().Nanosecond()\n\t\t\t\t}\n\n\t\t\t\taddEvent := AddEvent{\n\t\t\t\t\tName: output.Name,\n\t\t\t\t\tHash: h,\n\t\t\t\t\tBytes: output.Bytes,\n\t\t\t\t\tSize: output.Size,\n\t\t\t\t\tMtime: output.Mtime,\n\t\t\t\t\tMtimeNsecs: output.MtimeNsecs,\n\t\t\t\t}\n\n\t\t\t\tif output.Mode != 0 {\n\t\t\t\t\taddEvent.Mode = \"0\" + strconv.FormatUint(uint64(output.Mode), 8)\n\t\t\t\t}\n\n\t\t\t\tif output.Mtime > 0 {\n\t\t\t\t\taddEvent.Mtime = output.Mtime\n\t\t\t\t\tif output.MtimeNsecs > 0 {\n\t\t\t\t\t\taddEvent.MtimeNsecs = output.MtimeNsecs\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tif err := res.Emit(&addEvent); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif err := <-errCh; err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tadded++\n\t\t}\n\n\t\tif addit.Err() != nil {\n\t\t\treturn addit.Err()\n\t\t}\n\n\t\tif added == 0 {\n\t\t\treturn fmt.Errorf(\"expected a file argument\")\n\t\t}\n\n\t\t// Apply fast-provide-root if the flag is enabled\n\t\tif fastProvideRoot && (lastRootCid != path.ImmutablePath{}) {\n\t\t\tcfg, err := ipfsNode.Repo.Config()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif err := cmdenv.ExecuteFastProvide(req.Context, ipfsNode, cfg, lastRootCid.RootCid(), fastProvideWait, dopin, dopin, toFilesSet); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t} else if !fastProvideRoot {\n\t\t\tif fastProvideWait {\n\t\t\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"disabled by flag or config\", \"wait-flag-ignored\", true)\n\t\t\t} else {\n\t\t\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"disabled by flag or config\")\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tsizeChan := make(chan int64, 1)\n\t\t\toutChan := make(chan any)\n\t\t\treq := res.Request()\n\n\t\t\t// Could be slow.\n\t\t\tgo func() {\n\t\t\t\tsize, err := req.Files.Size()\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Warnf(\"error getting files size: %s\", err)\n\t\t\t\t\t// see comment above\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tsizeChan <- size\n\t\t\t}()\n\n\t\t\tprogressBar := func(wait chan struct{}) {\n\t\t\t\tdefer close(wait)\n\n\t\t\t\tquiet, _ := req.Options[quietOptionName].(bool)\n\t\t\t\tquieter, _ := req.Options[quieterOptionName].(bool)\n\t\t\t\tquiet = quiet || quieter\n\n\t\t\t\tprogress, _ := req.Options[progressOptionName].(bool)\n\n\t\t\t\tvar bar *pb.ProgressBar\n\t\t\t\tif progress {\n\t\t\t\t\tbar = pb.New64(0).SetUnits(pb.U_BYTES)\n\t\t\t\t\tbar.ManualUpdate = true\n\t\t\t\t\tbar.ShowTimeLeft = false\n\t\t\t\t\tbar.ShowPercent = false\n\t\t\t\t\tbar.Output = os.Stderr\n\t\t\t\t\tbar.Start()\n\t\t\t\t}\n\n\t\t\t\tlastFile := \"\"\n\t\t\t\tlastHash := \"\"\n\t\t\t\tvar totalProgress, prevFiles, lastBytes int64\n\n\t\t\tLOOP:\n\t\t\t\tfor {\n\t\t\t\t\tselect {\n\t\t\t\t\tcase out, ok := <-outChan:\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\tif quieter {\n\t\t\t\t\t\t\t\tfmt.Fprintln(os.Stdout, lastHash)\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t\tbreak LOOP\n\t\t\t\t\t\t}\n\t\t\t\t\t\toutput := out.(*AddEvent)\n\t\t\t\t\t\tif len(output.Hash) > 0 {\n\t\t\t\t\t\t\tlastHash = output.Hash\n\t\t\t\t\t\t\tif quieter {\n\t\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t\tif progress {\n\t\t\t\t\t\t\t\t// clear progress bar line before we print \"added x\" output\n\t\t\t\t\t\t\t\tfmt.Fprintf(os.Stderr, \"\\033[2K\\r\")\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tif quiet {\n\t\t\t\t\t\t\t\tfmt.Fprintf(os.Stdout, \"%s\\n\", output.Hash)\n\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\tfmt.Fprintf(os.Stdout, \"added %s %s\\n\", output.Hash, cmdenv.EscNonPrint(output.Name))\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tif !progress {\n\t\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t\tif len(lastFile) == 0 {\n\t\t\t\t\t\t\t\tlastFile = output.Name\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tif output.Name != lastFile || output.Bytes < lastBytes {\n\t\t\t\t\t\t\t\tprevFiles += lastBytes\n\t\t\t\t\t\t\t\tlastFile = output.Name\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tlastBytes = output.Bytes\n\t\t\t\t\t\t\tdelta := prevFiles + lastBytes - totalProgress\n\t\t\t\t\t\t\ttotalProgress = bar.Add64(delta)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif progress {\n\t\t\t\t\t\t\tbar.Update()\n\t\t\t\t\t\t}\n\t\t\t\t\tcase size := <-sizeChan:\n\t\t\t\t\t\tif progress {\n\t\t\t\t\t\t\tbar.Total = size\n\t\t\t\t\t\t\tbar.ShowPercent = true\n\t\t\t\t\t\t\tbar.ShowBar = true\n\t\t\t\t\t\t\tbar.ShowTimeLeft = true\n\t\t\t\t\t\t}\n\t\t\t\t\tcase <-req.Context.Done():\n\t\t\t\t\t\t// don't set or print error here, that happens in the goroutine below\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tif progress && bar.Total == 0 && bar.Get() != 0 {\n\t\t\t\t\tbar.Total = bar.Get()\n\t\t\t\t\tbar.ShowPercent = true\n\t\t\t\t\tbar.ShowBar = true\n\t\t\t\t\tbar.ShowTimeLeft = true\n\t\t\t\t\tbar.Update()\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif e := res.Error(); e != nil {\n\t\t\t\tclose(outChan)\n\t\t\t\treturn e\n\t\t\t}\n\n\t\t\twait := make(chan struct{})\n\t\t\tgo progressBar(wait)\n\n\t\t\tdefer func() { <-wait }()\n\t\t\tdefer close(outChan)\n\n\t\t\tfor {\n\t\t\t\tv, err := res.Next()\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err == io.EOF {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tselect {\n\t\t\t\tcase outChan <- v:\n\t\t\t\tcase <-req.Context.Done():\n\t\t\t\t\treturn req.Context.Err()\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t},\n\tType: AddEvent{},\n}\n" }, { "path": "core/commands/bitswap.go", "content": "package commands\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n\tbitswap \"github.com/ipfs/boxo/bitswap\"\n\t\"github.com/ipfs/boxo/bitswap/server\"\n\tcidutil \"github.com/ipfs/go-cidutil\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nvar BitswapCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with the bitswap agent.\",\n\t\tShortDescription: ``,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"stat\": bitswapStatCmd,\n\t\t\"wantlist\": showWantlistCmd,\n\t\t\"ledger\": ledgerCmd,\n\t\t\"reprovide\": deprecatedBitswapReprovideCmd,\n\t},\n}\n\nconst (\n\tpeerOptionName = \"peer\"\n)\n\nvar deprecatedBitswapReprovideCmd = &cmds.Command{\n\tStatus: cmds.Deprecated,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated command to announce to bitswap. Use 'ipfs routing reprovide' instead.\",\n\t\tShortDescription: `\n'ipfs bitswap reprovide' is a legacy plumbing command used to announce to DHT.\nDeprecated, use modern 'ipfs routing reprovide' instead.`,\n\t},\n\tRun: reprovideRoutingCmd.Run, // alias to routing reprovide to not break existing users\n}\n\nvar showWantlistCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show blocks currently on the wantlist.\",\n\t\tShortDescription: `\nPrint out all blocks currently on the bitswap wantlist for the local peer.`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(peerOptionName, \"p\", \"Specify which peer to show wantlist for. Default: self.\"),\n\t},\n\tType: KeyList{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tbs := nd.Bitswap\n\n\t\tpstr, found := req.Options[peerOptionName].(string)\n\t\tif found {\n\t\t\tpid, err := peer.Decode(pstr)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif pid != nd.Identity {\n\t\t\t\treturn cmds.EmitOnce(res, &KeyList{bs.WantlistForPeer(pid)})\n\t\t\t}\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &KeyList{bs.GetWantlist()})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *KeyList) error {\n\t\t\tenc, err := cmdenv.GetLowLevelCidEncoder(req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t// sort the keys first\n\t\t\tcidutil.Sort(out.Keys)\n\t\t\tfor _, key := range out.Keys {\n\t\t\t\tfmt.Fprintln(w, enc.Encode(key))\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nconst (\n\tbitswapVerboseOptionName = \"verbose\"\n\tbitswapHumanOptionName = \"human\"\n)\n\nvar bitswapStatCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show some diagnostic information on the bitswap agent.\",\n\t\tShortDescription: ``,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(bitswapVerboseOptionName, \"v\", \"Print extra information\"),\n\t\tcmds.BoolOption(bitswapHumanOptionName, \"Print sizes in human readable format (e.g., 1K 234M 2G)\"),\n\t},\n\tType: bitswap.Stat{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"unable to run offline: %s\", ErrNotOnline)\n\t\t}\n\n\t\tst, err := nd.Bitswap.Stat()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, st)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, s *bitswap.Stat) error {\n\t\t\tenc, err := cmdenv.GetLowLevelCidEncoder(req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tverbose, _ := req.Options[bitswapVerboseOptionName].(bool)\n\t\t\thuman, _ := req.Options[bitswapHumanOptionName].(bool)\n\n\t\t\tfmt.Fprintln(w, \"bitswap status\")\n\t\t\tfmt.Fprintf(w, \"\\tblocks received: %d\\n\", s.BlocksReceived)\n\t\t\tfmt.Fprintf(w, \"\\tblocks sent: %d\\n\", s.BlocksSent)\n\t\t\tif human {\n\t\t\t\tfmt.Fprintf(w, \"\\tdata received: %s\\n\", humanize.Bytes(s.DataReceived))\n\t\t\t\tfmt.Fprintf(w, \"\\tdata sent: %s\\n\", humanize.Bytes(s.DataSent))\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(w, \"\\tdata received: %d\\n\", s.DataReceived)\n\t\t\t\tfmt.Fprintf(w, \"\\tdata sent: %d\\n\", s.DataSent)\n\t\t\t}\n\t\t\tfmt.Fprintf(w, \"\\tdup blocks received: %d\\n\", s.DupBlksReceived)\n\t\t\tif human {\n\t\t\t\tfmt.Fprintf(w, \"\\tdup data received: %s\\n\", humanize.Bytes(s.DupDataReceived))\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(w, \"\\tdup data received: %d\\n\", s.DupDataReceived)\n\t\t\t}\n\t\t\tfmt.Fprintf(w, \"\\twantlist [%d keys]\\n\", len(s.Wantlist))\n\t\t\tfor _, k := range s.Wantlist {\n\t\t\t\tfmt.Fprintf(w, \"\\t\\t%s\\n\", enc.Encode(k))\n\t\t\t}\n\n\t\t\tfmt.Fprintf(w, \"\\tpartners [%d]\\n\", len(s.Peers))\n\t\t\tif verbose {\n\t\t\t\tfor _, p := range s.Peers {\n\t\t\t\t\tfmt.Fprintf(w, \"\\t\\t%s\\n\", p)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar ledgerCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show the current ledger for a peer.\",\n\t\tShortDescription: `\nThe Bitswap decision engine tracks the number of bytes exchanged between IPFS\nnodes, and stores this information as a collection of ledgers. This command\nprints the ledger associated with a given peer.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peer\", true, false, \"The PeerID (B58) of the ledger to inspect.\"),\n\t},\n\tType: server.Receipt{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tpartner, err := peer.Decode(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, nd.Bitswap.LedgerForPeer(partner))\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *server.Receipt) error {\n\t\t\tfmt.Fprintf(w, \"Ledger for %s\\n\"+\n\t\t\t\t\"Debt ratio:\\t%f\\n\"+\n\t\t\t\t\"Exchanges:\\t%d\\n\"+\n\t\t\t\t\"Bytes sent:\\t%d\\n\"+\n\t\t\t\t\"Bytes received:\\t%d\\n\\n\",\n\t\t\t\tout.Peer, out.Value, out.Exchanged,\n\t\t\t\tout.Sent, out.Recv)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/block.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/ipfs/boxo/files\"\n\n\t\"github.com/ipfs/kubo/config\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\ntype BlockStat struct {\n\tKey string\n\tSize int\n}\n\nfunc (bs BlockStat) String() string {\n\treturn fmt.Sprintf(\"Key: %s\\nSize: %d\\n\", bs.Key, bs.Size)\n}\n\nvar BlockCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with raw IPFS blocks.\",\n\t\tShortDescription: `\n'ipfs block' is a plumbing command used to manipulate raw IPFS blocks.\nReads from stdin or writes to stdout. A block is identified by a Multihash\npassed with a valid CID.\n`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"stat\": blockStatCmd,\n\t\t\"get\": blockGetCmd,\n\t\t\"put\": blockPutCmd,\n\t\t\"rm\": blockRmCmd,\n\t},\n}\n\nvar blockStatCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Print information of a raw IPFS block.\",\n\t\tShortDescription: `\n'ipfs block stat' is a plumbing command for retrieving information\non raw IPFS blocks. It outputs the following to stdout:\n\n\tKey - the CID of the block\n\tSize - the size of the block in bytes\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", true, false, \"The CID of an existing block to stat.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tb, err := api.Block().Stat(req.Context, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &BlockStat{\n\t\t\tKey: b.Path().RootCid().String(),\n\t\t\tSize: b.Size(),\n\t\t})\n\t},\n\tType: BlockStat{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, bs *BlockStat) error {\n\t\t\t_, err := fmt.Fprintf(w, \"%s\", bs)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\nvar blockGetCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Get a raw IPFS block.\",\n\t\tShortDescription: `\n'ipfs block get' is a plumbing command for retrieving raw IPFS blocks.\nIt takes a , and outputs the block to stdout.\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tResponseContentType: \"application/vnd.ipld.raw\",\n\t\t},\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", true, false, \"The CID of an existing block to get.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := api.Block().Get(req.Context, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tres.SetEncodingType(cmds.OctetStream)\n\t\tres.SetContentType(\"application/vnd.ipld.raw\")\n\t\treturn res.Emit(r)\n\t},\n}\n\nconst (\n\tblockFormatOptionName = \"format\"\n\tblockCidCodecOptionName = \"cid-codec\"\n\tmhtypeOptionName = \"mhtype\"\n\tmhlenOptionName = \"mhlen\"\n)\n\nvar blockPutCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Store input as an IPFS block.\",\n\t\tShortDescription: `\n'ipfs block put' is a plumbing command for storing raw IPFS blocks.\nIt reads data from stdin, and outputs the block's CID to stdout.\n\nUnless cid-codec is specified, this command returns raw (0x55) CIDv1 CIDs.\n\nPassing alternative --cid-codec does not modify imported data, nor run any\nvalidation. It is provided solely for convenience for users who create blocks\nin userland.\n\nNOTE:\nDo not use --format for any new code. It got superseded by --cid-codec and left\nonly for backward compatibility when a legacy CIDv0 is required (--format=v0).\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"data\", true, true, \"The data to be stored as an IPFS block.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(blockCidCodecOptionName, \"Multicodec to use in returned CID\").WithDefault(\"raw\"),\n\t\tcmds.StringOption(mhtypeOptionName, \"Multihash hash function\"),\n\t\tcmds.IntOption(mhlenOptionName, \"Multihash hash length\").WithDefault(-1),\n\t\tcmds.BoolOption(pinOptionName, \"Pin added blocks recursively\").WithDefault(false),\n\t\tcmdutils.AllowBigBlockOption,\n\t\tcmds.StringOption(blockFormatOptionName, \"f\", \"Use legacy format for returned CID (DEPRECATED)\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err := nd.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tmhtype, _ := req.Options[mhtypeOptionName].(string)\n\t\tif mhtype == \"\" {\n\t\t\tmhtype = cfg.Import.HashFunction.WithDefault(config.DefaultHashFunction)\n\t\t}\n\n\t\tmhtval, ok := mh.Names[mhtype]\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"unrecognized multihash function: %s\", mhtype)\n\t\t}\n\n\t\tmhlen, ok := req.Options[mhlenOptionName].(int)\n\t\tif !ok {\n\t\t\treturn errors.New(\"missing option \\\"mhlen\\\"\")\n\t\t}\n\n\t\tcidCodec, _ := req.Options[blockCidCodecOptionName].(string)\n\t\tformat, _ := req.Options[blockFormatOptionName].(string) // deprecated\n\n\t\t// use of legacy 'format' needs to suppress 'cid-codec'\n\t\tif format != \"\" {\n\t\t\tif cidCodec != \"\" && cidCodec != \"raw\" {\n\t\t\t\treturn fmt.Errorf(\"unable to use %q (deprecated) and a custom %q at the same time\", blockFormatOptionName, blockCidCodecOptionName)\n\t\t\t}\n\t\t\tcidCodec = \"\" // makes it no-op\n\t\t}\n\n\t\tpin, _ := req.Options[pinOptionName].(bool)\n\n\t\tit := req.Files.Entries()\n\t\tfor it.Next() {\n\t\t\tfile := files.FileFromEntry(it)\n\t\t\tif file == nil {\n\t\t\t\treturn errors.New(\"expected a file\")\n\t\t\t}\n\n\t\t\tp, err := api.Block().Put(req.Context, file,\n\t\t\t\toptions.Block.Hash(mhtval, mhlen),\n\t\t\t\toptions.Block.CidCodec(cidCodec),\n\t\t\t\toptions.Block.Format(format),\n\t\t\t\toptions.Block.Pin(pin))\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif err := cmdutils.CheckBlockSize(req, uint64(p.Size())); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\terr = res.Emit(&BlockStat{\n\t\t\t\tKey: p.Path().RootCid().String(),\n\t\t\t\tSize: p.Size(),\n\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn it.Err()\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, bs *BlockStat) error {\n\t\t\t_, err := fmt.Fprintf(w, \"%s\\n\", bs.Key)\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: BlockStat{},\n}\n\nconst (\n\tforceOptionName = \"force\"\n\tblockQuietOptionName = \"quiet\"\n)\n\ntype removedBlock struct {\n\tHash string `json:\",omitempty\"`\n\tError string `json:\",omitempty\"`\n}\n\nvar blockRmCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove IPFS block(s) from the local datastore.\",\n\t\tShortDescription: `\n'ipfs block rm' is a plumbing command for removing raw ipfs blocks.\nIt takes a list of CIDs to remove from the local datastore..\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", true, true, \"CIDs of block(s) to remove.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(forceOptionName, \"f\", \"Ignore nonexistent blocks.\"),\n\t\tcmds.BoolOption(blockQuietOptionName, \"q\", \"Write minimal output.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tforce, _ := req.Options[forceOptionName].(bool)\n\t\tquiet, _ := req.Options[blockQuietOptionName].(bool)\n\n\t\t// TODO: use batching coreapi when done\n\t\tfor _, b := range req.Arguments {\n\t\t\tp, err := cmdutils.PathOrCidPath(b)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\trp, _, err := api.ResolvePath(req.Context, p)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\terr = api.Block().Rm(req.Context, rp, options.Block.Force(force))\n\t\t\tif err != nil {\n\t\t\t\tif err := res.Emit(&removedBlock{\n\t\t\t\t\tHash: rp.RootCid().String(),\n\t\t\t\t\tError: err.Error(),\n\t\t\t\t}); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif !quiet {\n\t\t\t\terr := res.Emit(&removedBlock{\n\t\t\t\t\tHash: rp.RootCid().String(),\n\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tsomeFailed := false\n\t\t\tfor {\n\t\t\t\tres, err := res.Next()\n\t\t\t\tif err == io.EOF {\n\t\t\t\t\tbreak\n\t\t\t\t} else if err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tr := res.(*removedBlock)\n\t\t\t\tif r.Hash == \"\" && r.Error != \"\" {\n\t\t\t\t\treturn fmt.Errorf(\"aborted: %s\", r.Error)\n\t\t\t\t} else if r.Error != \"\" {\n\t\t\t\t\tsomeFailed = true\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"cannot remove %s: %s\\n\", r.Hash, r.Error)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(os.Stdout, \"removed %s\\n\", r.Hash)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif someFailed {\n\t\t\t\treturn fmt.Errorf(\"some blocks not removed\")\n\t\t\t}\n\t\t\treturn nil\n\t\t},\n\t},\n\tType: removedBlock{},\n}\n" }, { "path": "core/commands/bootstrap.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"slices\"\n\t\"strings\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\trepo \"github.com/ipfs/kubo/repo\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\ntype BootstrapOutput struct {\n\tPeers []string\n}\n\nvar peerOptionDesc = \"A peer to add to the bootstrap list (in the format '/')\"\n\nvar BootstrapCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show or edit the list of bootstrap peers.\",\n\t\tShortDescription: `\nRunning 'ipfs bootstrap' with no arguments will run 'ipfs bootstrap list'.\n` + bootstrapSecurityWarning,\n\t},\n\n\tRun: bootstrapListCmd.Run,\n\tEncoders: bootstrapListCmd.Encoders,\n\tType: bootstrapListCmd.Type,\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"list\": bootstrapListCmd,\n\t\t\"add\": bootstrapAddCmd,\n\t\t\"rm\": bootstrapRemoveCmd,\n\t},\n}\n\nvar bootstrapAddCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add peers to the bootstrap list.\",\n\t\tShortDescription: `Outputs a list of peers that were added (that weren't already\nin the bootstrap list).\n\nThe special values 'default' and 'auto' can be used to add the default\nbootstrap peers. Both are equivalent and will add the 'auto' placeholder to\nthe bootstrap list, which gets resolved using the AutoConf system.\n` + bootstrapSecurityWarning,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peer\", false, true, peerOptionDesc).EnableStdin(),\n\t},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tinputPeers := req.Arguments\n\n\t\tif len(inputPeers) == 0 {\n\t\t\treturn errors.New(\"no bootstrap peers to add\")\n\t\t}\n\n\t\t// Convert \"default\" to \"auto\" for backward compatibility\n\t\tfor i, peer := range inputPeers {\n\t\t\tif peer == \"default\" {\n\t\t\t\tinputPeers[i] = \"auto\"\n\t\t\t}\n\t\t}\n\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tcfg, err := r.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Check if trying to add \"auto\" when AutoConf is disabled\n\t\tfor _, peer := range inputPeers {\n\t\t\tif peer == config.AutoPlaceholder && !cfg.AutoConf.Enabled.WithDefault(config.DefaultAutoConfEnabled) {\n\t\t\t\treturn errors.New(\"cannot add default bootstrap peers: AutoConf is disabled (AutoConf.Enabled=false). Enable AutoConf by setting AutoConf.Enabled=true in your config, or add specific peer addresses instead\")\n\t\t\t}\n\t\t}\n\n\t\tadded, err := bootstrapAdd(r, cfg, inputPeers)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &BootstrapOutput{added})\n\t},\n\tType: BootstrapOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *BootstrapOutput) error {\n\t\t\treturn bootstrapWritePeers(w, \"added \", out.Peers)\n\t\t}),\n\t},\n}\n\nconst (\n\tbootstrapAllOptionName = \"all\"\n)\n\nvar bootstrapRemoveCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove peers from the bootstrap list.\",\n\t\tShortDescription: `Outputs the list of peers that were removed.\n` + bootstrapSecurityWarning,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peer\", false, true, peerOptionDesc).EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(bootstrapAllOptionName, \"Remove all bootstrap peers. (Deprecated, use 'all' subcommand)\"),\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"all\": bootstrapRemoveAllCmd,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tall, _ := req.Options[bootstrapAllOptionName].(bool)\n\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tcfg, err := r.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar removed []string\n\t\tif all {\n\t\t\tremoved, err = bootstrapRemoveAll(r, cfg)\n\t\t} else {\n\t\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tremoved, err = bootstrapRemove(r, cfg, req.Arguments)\n\t\t}\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &BootstrapOutput{removed})\n\t},\n\tType: BootstrapOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *BootstrapOutput) error {\n\t\t\treturn bootstrapWritePeers(w, \"removed \", out.Peers)\n\t\t}),\n\t},\n}\n\nvar bootstrapRemoveAllCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove all peers from the bootstrap list.\",\n\t\tShortDescription: `Outputs the list of peers that were removed.`,\n\t},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tcfg, err := r.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tremoved, err := bootstrapRemoveAll(r, cfg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &BootstrapOutput{removed})\n\t},\n\tType: BootstrapOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *BootstrapOutput) error {\n\t\t\treturn bootstrapWritePeers(w, \"removed \", out.Peers)\n\t\t}),\n\t},\n}\n\nvar bootstrapListCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show peers in the bootstrap list.\",\n\t\tShortDescription: \"Peers are output in the format '/'.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(configExpandAutoName, \"Expand 'auto' placeholders from AutoConf service.\"),\n\t},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tcfg, err := r.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Check if user wants to expand auto values\n\t\texpandAuto, _ := req.Options[configExpandAutoName].(bool)\n\t\tif expandAuto {\n\t\t\t// Use the same expansion method as the daemon\n\t\t\texpandedBootstrap := cfg.BootstrapWithAutoConf()\n\t\t\treturn cmds.EmitOnce(res, &BootstrapOutput{expandedBootstrap})\n\t\t}\n\n\t\t// Simply return the bootstrap config as-is, including any \"auto\" values\n\t\treturn cmds.EmitOnce(res, &BootstrapOutput{cfg.Bootstrap})\n\t},\n\tType: BootstrapOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *BootstrapOutput) error {\n\t\t\treturn bootstrapWritePeers(w, \"\", out.Peers)\n\t\t}),\n\t},\n}\n\nfunc bootstrapWritePeers(w io.Writer, prefix string, peers []string) error {\n\tslices.SortStableFunc(peers, func(a, b string) int {\n\t\treturn strings.Compare(a, b)\n\t})\n\tfor _, peer := range peers {\n\t\t_, err := w.Write([]byte(prefix + peer + \"\\n\"))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc bootstrapAdd(r repo.Repo, cfg *config.Config, peers []string) ([]string, error) {\n\t// Validate peers - skip validation for \"auto\" placeholder\n\tfor _, p := range peers {\n\t\tif p == config.AutoPlaceholder {\n\t\t\tcontinue // Skip validation for \"auto\" placeholder\n\t\t}\n\t\tm, err := ma.NewMultiaddr(p)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\ttpt, p2ppart := ma.SplitLast(m)\n\t\tif p2ppart == nil || p2ppart.Protocol().Code != ma.P_P2P {\n\t\t\treturn nil, fmt.Errorf(\"invalid bootstrap address: %s\", p)\n\t\t}\n\t\tif tpt == nil {\n\t\t\treturn nil, fmt.Errorf(\"bootstrap address without a transport: %s\", p)\n\t\t}\n\t}\n\n\taddedMap := map[string]struct{}{}\n\taddedList := make([]string, 0, len(peers))\n\n\t// re-add cfg bootstrap peers to rm dupes\n\tbpeers := cfg.Bootstrap\n\tcfg.Bootstrap = nil\n\n\t// add new peers\n\tfor _, s := range peers {\n\t\tif _, found := addedMap[s]; found {\n\t\t\tcontinue\n\t\t}\n\n\t\tcfg.Bootstrap = append(cfg.Bootstrap, s)\n\t\taddedList = append(addedList, s)\n\t\taddedMap[s] = struct{}{}\n\t}\n\n\t// add back original peers. in this order so that we output them.\n\tfor _, s := range bpeers {\n\t\tif _, found := addedMap[s]; found {\n\t\t\tcontinue\n\t\t}\n\n\t\tcfg.Bootstrap = append(cfg.Bootstrap, s)\n\t\taddedMap[s] = struct{}{}\n\t}\n\n\tif err := r.SetConfig(cfg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn addedList, nil\n}\n\nfunc bootstrapRemove(r repo.Repo, cfg *config.Config, toRemove []string) ([]string, error) {\n\t// Check if bootstrap contains \"auto\"\n\thasAuto := slices.Contains(cfg.Bootstrap, config.AutoPlaceholder)\n\n\tif hasAuto && cfg.AutoConf.Enabled.WithDefault(config.DefaultAutoConfEnabled) {\n\t\t// Cannot selectively remove peers when using \"auto\" bootstrap\n\t\t// Users should either disable AutoConf or replace \"auto\" with specific peers\n\t\treturn nil, fmt.Errorf(\"cannot remove individual bootstrap peers when using 'auto' placeholder: the 'auto' value is managed by AutoConf. Either disable AutoConf by setting AutoConf.Enabled=false and replace 'auto' with specific peer addresses, or use 'ipfs bootstrap rm --all' to remove all peers\")\n\t}\n\n\t// Original logic for non-auto bootstrap\n\tremoved := make([]peer.AddrInfo, 0, len(toRemove))\n\tkeep := make([]peer.AddrInfo, 0, len(cfg.Bootstrap))\n\n\ttoRemoveAddr, err := config.ParseBootstrapPeers(toRemove)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\ttoRemoveMap := make(map[peer.ID][]ma.Multiaddr, len(toRemoveAddr))\n\tfor _, addr := range toRemoveAddr {\n\t\ttoRemoveMap[addr.ID] = addr.Addrs\n\t}\n\n\tpeers, err := cfg.BootstrapPeers()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, p := range peers {\n\t\taddrs, ok := toRemoveMap[p.ID]\n\t\t// not in the remove set?\n\t\tif !ok {\n\t\t\tkeep = append(keep, p)\n\t\t\tcontinue\n\t\t}\n\t\t// remove the entire peer?\n\t\tif len(addrs) == 0 {\n\t\t\tremoved = append(removed, p)\n\t\t\tcontinue\n\t\t}\n\t\tvar keptAddrs, removedAddrs []ma.Multiaddr\n\t\t// remove specific addresses\n\tfilter:\n\t\tfor _, addr := range p.Addrs {\n\t\t\tfor _, addr2 := range addrs {\n\t\t\t\tif addr.Equal(addr2) {\n\t\t\t\t\tremovedAddrs = append(removedAddrs, addr)\n\t\t\t\t\tcontinue filter\n\t\t\t\t}\n\t\t\t}\n\t\t\tkeptAddrs = append(keptAddrs, addr)\n\t\t}\n\t\tif len(removedAddrs) > 0 {\n\t\t\tremoved = append(removed, peer.AddrInfo{ID: p.ID, Addrs: removedAddrs})\n\t\t}\n\n\t\tif len(keptAddrs) > 0 {\n\t\t\tkeep = append(keep, peer.AddrInfo{ID: p.ID, Addrs: keptAddrs})\n\t\t}\n\t}\n\tcfg.SetBootstrapPeers(keep)\n\n\tif err := r.SetConfig(cfg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn config.BootstrapPeerStrings(removed), nil\n}\n\nfunc bootstrapRemoveAll(r repo.Repo, cfg *config.Config) ([]string, error) {\n\t// Check if bootstrap contains \"auto\" - if so, we need special handling\n\thasAuto := slices.Contains(cfg.Bootstrap, config.AutoPlaceholder)\n\n\tvar removed []string\n\tif hasAuto {\n\t\t// When \"auto\" is present, we can't parse it as peer.AddrInfo\n\t\t// Just return the raw bootstrap list as strings for display\n\t\tremoved = slices.Clone(cfg.Bootstrap)\n\t} else {\n\t\t// Original logic for configs without \"auto\"\n\t\tremovedPeers, err := cfg.BootstrapPeers()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tremoved = config.BootstrapPeerStrings(removedPeers)\n\t}\n\n\tcfg.Bootstrap = nil\n\tif err := r.SetConfig(cfg); err != nil {\n\t\treturn nil, err\n\t}\n\treturn removed, nil\n}\n\nconst bootstrapSecurityWarning = `\nSECURITY WARNING:\n\nThe bootstrap command manipulates the \"bootstrap list\", which contains\nthe addresses of bootstrap nodes. These are the *trusted peers* from\nwhich to learn about other peers in the network. Only edit this list\nif you understand the risks of adding or removing nodes from this list.\n\n`\n" }, { "path": "core/commands/cat.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\t\"github.com/cheggaaa/pb\"\n\t\"github.com/ipfs/boxo/files\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n)\n\nconst (\n\tprogressBarMinSize = 1024 * 1024 * 8 // show progress bar for outputs > 8MiB\n\toffsetOptionName = \"offset\"\n\tlengthOptionName = \"length\"\n)\n\nvar CatCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show IPFS object data.\",\n\t\tShortDescription: \"Displays the data contained by an IPFS or IPNS object(s) at the given path.\",\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, true, \"The path to the IPFS object(s) to be outputted.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.Int64Option(offsetOptionName, \"o\", \"Byte offset to begin reading from.\"),\n\t\tcmds.Int64Option(lengthOptionName, \"l\", \"Maximum number of bytes to read.\"),\n\t\tcmds.BoolOption(progressOptionName, \"p\", \"Stream progress data.\").WithDefault(true),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toffset, _ := req.Options[offsetOptionName].(int64)\n\t\tif offset < 0 {\n\t\t\treturn errors.New(\"cannot specify negative offset\")\n\t\t}\n\n\t\tmax, found := req.Options[lengthOptionName].(int64)\n\n\t\tif max < 0 {\n\t\t\treturn errors.New(\"cannot specify negative length\")\n\t\t}\n\t\tif !found {\n\t\t\tmax = -1\n\t\t}\n\n\t\terr = req.ParseBodyArgs()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treaders, length, err := cat(req.Context, api, req.Arguments, int64(offset), int64(max))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t/*\n\t\t\tif err := corerepo.ConditionalGC(req.Context, node, length); err != nil {\n\t\t\t\tre.SetError(err, cmds.ErrNormal)\n\t\t\t\treturn\n\t\t\t}\n\t\t*/\n\n\t\tres.SetLength(length)\n\t\treader := io.MultiReader(readers...)\n\n\t\t// Since the reader returns the error that a block is missing, and that error is\n\t\t// returned from io.Copy inside Emit, we need to take Emit errors and send\n\t\t// them to the client. Usually we don't do that because it means the connection\n\t\t// is broken or we supplied an illegal argument etc.\n\t\treturn res.Emit(reader)\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tif res.Length() > 0 && res.Length() < progressBarMinSize {\n\t\t\t\treturn cmds.Copy(re, res)\n\t\t\t}\n\n\t\t\tfor {\n\t\t\t\tv, err := res.Next()\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err == io.EOF {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tswitch val := v.(type) {\n\t\t\t\tcase io.Reader:\n\t\t\t\t\treader := val\n\n\t\t\t\t\treq := res.Request()\n\t\t\t\t\tprogress, _ := req.Options[progressOptionName].(bool)\n\t\t\t\t\tif progress {\n\t\t\t\t\t\tvar bar *pb.ProgressBar\n\t\t\t\t\t\tbar, reader = progressBarForReader(os.Stderr, val, int64(res.Length()))\n\t\t\t\t\t\tbar.Start()\n\t\t\t\t\t\tdefer bar.Finish()\n\t\t\t\t\t}\n\n\t\t\t\t\terr = re.Emit(reader)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\tdefault:\n\t\t\t\t\tlog.Warnf(\"cat postrun: received unexpected type %T\", val)\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t},\n}\n\nfunc cat(ctx context.Context, api iface.CoreAPI, paths []string, offset int64, max int64) ([]io.Reader, uint64, error) {\n\treaders := make([]io.Reader, 0, len(paths))\n\tlength := uint64(0)\n\tif max == 0 {\n\t\treturn nil, 0, nil\n\t}\n\tfor _, pString := range paths {\n\t\tp, err := cmdutils.PathOrCidPath(pString)\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\tf, err := api.Unixfs().Get(ctx, p)\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\tvar file files.File\n\t\tswitch f := f.(type) {\n\t\tcase files.File:\n\t\t\tfile = f\n\t\tcase files.Directory:\n\t\t\treturn nil, 0, iface.ErrIsDir\n\t\tdefault:\n\t\t\treturn nil, 0, iface.ErrNotSupported\n\t\t}\n\n\t\tfsize, err := file.Size()\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\tif offset > fsize {\n\t\t\toffset = offset - fsize\n\t\t\tcontinue\n\t\t}\n\n\t\tcount, err := file.Seek(offset, io.SeekStart)\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\t\toffset = 0\n\n\t\tfsize, err = file.Size()\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\tsize := uint64(fsize - count)\n\t\tlength += size\n\t\tif max > 0 && length >= uint64(max) {\n\t\t\tvar r io.Reader = file\n\t\t\tif overshoot := int64(length - uint64(max)); overshoot != 0 {\n\t\t\t\tr = io.LimitReader(file, int64(size)-overshoot)\n\t\t\t\tlength = uint64(max)\n\t\t\t}\n\t\t\treaders = append(readers, r)\n\t\t\tbreak\n\t\t}\n\t\treaders = append(readers, file)\n\t}\n\treturn readers, length, nil\n}\n" }, { "path": "core/commands/cid.go", "content": "package commands\n\nimport (\n\t\"cmp\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"slices\"\n\t\"strings\"\n\t\"unicode\"\n\n\tverifcid \"github.com/ipfs/boxo/verifcid\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcidutil \"github.com/ipfs/go-cidutil\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipldmulticodec \"github.com/ipld/go-ipld-prime/multicodec\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n\tmc \"github.com/multiformats/go-multicodec\"\n\tmhash \"github.com/multiformats/go-multihash\"\n)\n\nvar CidCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Convert and discover properties of CIDs\",\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"inspect\": inspectCmd,\n\t\t\"format\": cidFmtCmd,\n\t\t\"base32\": base32Cmd,\n\t\t\"bases\": basesCmd,\n\t\t\"codecs\": codecsCmd,\n\t\t\"hashes\": hashesCmd,\n\t},\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\nconst (\n\tcidFormatOptionName = \"f\"\n\tcidToVersionOptionName = \"v\"\n\tcidCodecOptionName = \"mc\"\n\tcidMultibaseOptionName = \"b\"\n)\n\nvar cidFmtCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Format and convert a CID in various useful ways.\",\n\t\tLongDescription: `\nFormat and converts 's in various useful ways.\n\nFor a human-readable breakdown of a CID, see 'ipfs cid inspect'.\n\nThe optional format string is a printf style format string:\n` + cidutil.FormatRef,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", true, true, \"CIDs to format.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(cidFormatOptionName, \"Printf style format string.\").WithDefault(\"%s\"),\n\t\tcmds.StringOption(cidToVersionOptionName, \"CID version to convert to.\"),\n\t\tcmds.StringOption(cidCodecOptionName, \"CID multicodec to convert to.\"),\n\t\tcmds.StringOption(cidMultibaseOptionName, \"Multibase to display CID in.\"),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tfmtStr, _ := req.Options[cidFormatOptionName].(string)\n\t\tverStr, _ := req.Options[cidToVersionOptionName].(string)\n\t\tcodecStr, _ := req.Options[cidCodecOptionName].(string)\n\t\tbaseStr, _ := req.Options[cidMultibaseOptionName].(string)\n\n\t\topts := cidFormatOpts{}\n\n\t\tif strings.IndexByte(fmtStr, '%') == -1 {\n\t\t\treturn fmt.Errorf(\"invalid format string: %q\", fmtStr)\n\t\t}\n\t\topts.fmtStr = fmtStr\n\n\t\tif codecStr != \"\" {\n\t\t\tvar codec mc.Code\n\t\t\terr := codec.Set(codecStr)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\topts.newCodec = uint64(codec)\n\t\t} // otherwise, leave it as 0 (not a valid IPLD codec)\n\n\t\tswitch verStr {\n\t\tcase \"\":\n\t\t\tif baseStr != \"\" {\n\t\t\t\topts.verConv = toCidV1\n\t\t\t}\n\t\tcase \"0\":\n\t\t\tif opts.newCodec != 0 && opts.newCodec != cid.DagProtobuf {\n\t\t\t\treturn errors.New(\"cannot convert to CIDv0 with any codec other than dag-pb\")\n\t\t\t}\n\t\t\tif baseStr != \"\" && baseStr != \"base58btc\" {\n\t\t\t\treturn errors.New(\"cannot convert to CIDv0 with any multibase other than the implicit base58btc\")\n\t\t\t}\n\t\t\topts.verConv = toCidV0\n\t\tcase \"1\":\n\t\t\topts.verConv = toCidV1\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"invalid cid version: %q\", verStr)\n\t\t}\n\n\t\tif baseStr != \"\" {\n\t\t\tencoder, err := mbase.EncoderByName(baseStr)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\topts.newBase = encoder.Encoding()\n\t\t} else {\n\t\t\topts.newBase = mbase.Encoding(-1)\n\t\t}\n\n\t\treturn emitCids(req, resp, opts)\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: streamResult(func(v any, out io.Writer) nonFatalError {\n\t\t\tr := v.(*CidFormatRes)\n\t\t\tif r.ErrorMsg != \"\" {\n\t\t\t\treturn nonFatalError(fmt.Sprintf(\"%s: %s\", r.CidStr, r.ErrorMsg))\n\t\t\t}\n\t\t\tfmt.Fprintf(out, \"%s\\n\", r.Formatted)\n\t\t\treturn \"\"\n\t\t}),\n\t},\n\tType: CidFormatRes{},\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\ntype CidFormatRes struct {\n\tCidStr string // Original Cid String passed in\n\tFormatted string // Formatted Result\n\tErrorMsg string // Error\n}\n\nvar base32Cmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Convert CIDs to Base32 CID version 1.\",\n\t\tShortDescription: `\n'ipfs cid base32' normalizes passed CIDs to their canonical case-insensitive encoding.\nUseful when processing third-party CIDs which could come with arbitrary formats.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", true, true, \"CIDs to convert.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\topts := cidFormatOpts{\n\t\t\tfmtStr: \"%s\",\n\t\t\tnewBase: mbase.Encoding(mbase.Base32),\n\t\t\tverConv: toCidV1,\n\t\t}\n\t\treturn emitCids(req, resp, opts)\n\t},\n\tPostRun: cidFmtCmd.PostRun,\n\tType: cidFmtCmd.Type,\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\ntype cidFormatOpts struct {\n\tfmtStr string\n\tnewBase mbase.Encoding\n\tverConv func(cid cid.Cid) (cid.Cid, error)\n\tnewCodec uint64\n}\n\ntype argumentIterator struct {\n\targs []string\n\tbody cmds.StdinArguments\n}\n\nfunc (i *argumentIterator) next() (string, bool) {\n\tif len(i.args) > 0 {\n\t\targ := i.args[0]\n\t\ti.args = i.args[1:]\n\t\treturn arg, true\n\t}\n\tif i.body == nil || !i.body.Scan() {\n\t\treturn \"\", false\n\t}\n\treturn strings.TrimSpace(i.body.Argument()), true\n}\n\nfunc (i *argumentIterator) err() error {\n\tif i.body == nil {\n\t\treturn nil\n\t}\n\treturn i.body.Err()\n}\n\nfunc emitCids(req *cmds.Request, resp cmds.ResponseEmitter, opts cidFormatOpts) error {\n\titr := argumentIterator{req.Arguments, req.BodyArgs()}\n\tvar emitErr error\n\tfor emitErr == nil {\n\t\tcidStr, ok := itr.next()\n\t\tif !ok {\n\t\t\tbreak\n\t\t}\n\t\tres := &CidFormatRes{CidStr: cidStr}\n\t\tc, err := cid.Decode(cidStr)\n\t\tif err != nil {\n\t\t\tres.ErrorMsg = err.Error()\n\t\t\temitErr = resp.Emit(res)\n\t\t\tcontinue\n\t\t}\n\n\t\tif opts.newCodec != 0 && opts.newCodec != c.Type() {\n\t\t\tc = cid.NewCidV1(opts.newCodec, c.Hash())\n\t\t}\n\n\t\tif opts.verConv != nil {\n\t\t\tc, err = opts.verConv(c)\n\t\t\tif err != nil {\n\t\t\t\tres.ErrorMsg = err.Error()\n\t\t\t\temitErr = resp.Emit(res)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\tbase := opts.newBase\n\t\tif base == -1 {\n\t\t\tif c.Version() == 0 {\n\t\t\t\tbase = mbase.Base58BTC\n\t\t\t} else {\n\t\t\t\tbase, _ = cid.ExtractEncoding(cidStr)\n\t\t\t}\n\t\t}\n\n\t\tstr, err := cidutil.Format(opts.fmtStr, base, c)\n\t\tif _, ok := err.(cidutil.FormatStringError); ok {\n\t\t\t// no point in continuing if there is a problem with the format string\n\t\t\treturn err\n\t\t}\n\t\tif err != nil {\n\t\t\tres.ErrorMsg = err.Error()\n\t\t} else {\n\t\t\tres.Formatted = str\n\t\t}\n\t\temitErr = resp.Emit(res)\n\t}\n\tif emitErr != nil {\n\t\treturn emitErr\n\t}\n\terr := itr.err()\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc toCidV0(c cid.Cid) (cid.Cid, error) {\n\tif c.Type() != cid.DagProtobuf {\n\t\treturn cid.Cid{}, fmt.Errorf(\"can't convert non-dag-pb nodes to cidv0\")\n\t}\n\treturn cid.NewCidV0(c.Hash()), nil\n}\n\nfunc toCidV1(c cid.Cid) (cid.Cid, error) {\n\treturn cid.NewCidV1(c.Type(), c.Hash()), nil\n}\n\ntype CodeAndName struct {\n\tCode int\n\tName string\n}\n\nconst (\n\tprefixOptionName = \"prefix\"\n\tnumericOptionName = \"numeric\"\n)\n\nvar basesCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List available multibase encodings.\",\n\t\tShortDescription: `\n'ipfs cid bases' relies on https://github.com/multiformats/go-multibase\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(prefixOptionName, \"also include the single letter prefixes in addition to the code\"),\n\t\tcmds.BoolOption(numericOptionName, \"also include numeric codes\"),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tvar res []CodeAndName\n\t\t// use EncodingToStr in case at some point there are multiple names for a given code\n\t\tfor code, name := range mbase.EncodingToStr {\n\t\t\tres = append(res, CodeAndName{int(code), name})\n\t\t}\n\t\treturn cmds.EmitOnce(resp, res)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, val []CodeAndName) error {\n\t\t\tprefixes, _ := req.Options[prefixOptionName].(bool)\n\t\t\tnumeric, _ := req.Options[numericOptionName].(bool)\n\t\t\tmultibaseSorter{val}.Sort()\n\t\t\tfor _, v := range val {\n\t\t\t\tcode := v.Code\n\t\t\t\tif !unicode.IsPrint(rune(code)) {\n\t\t\t\t\t// don't display non-printable prefixes\n\t\t\t\t\tcode = ' '\n\t\t\t\t}\n\t\t\t\tswitch {\n\t\t\t\tcase prefixes && numeric:\n\t\t\t\t\tfmt.Fprintf(w, \"%c %7d %s\\n\", code, v.Code, v.Name)\n\t\t\t\tcase prefixes:\n\t\t\t\t\tfmt.Fprintf(w, \"%c %s\\n\", code, v.Name)\n\t\t\t\tcase numeric:\n\t\t\t\t\tfmt.Fprintf(w, \"%7d %s\\n\", v.Code, v.Name)\n\t\t\t\tdefault:\n\t\t\t\t\tfmt.Fprintf(w, \"%s\\n\", v.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: []CodeAndName{},\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\nconst (\n\tcodecsNumericOptionName = \"numeric\"\n\tcodecsSupportedOptionName = \"supported\"\n)\n\nvar codecsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List available CID multicodecs.\",\n\t\tShortDescription: `\n'ipfs cid codecs' relies on https://github.com/multiformats/go-multicodec\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(codecsNumericOptionName, \"n\", \"also include numeric codes\"),\n\t\tcmds.BoolOption(codecsSupportedOptionName, \"s\", \"list only codecs supported by go-ipfs commands\"),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tlistSupported, _ := req.Options[codecsSupportedOptionName].(bool)\n\t\tsupportedCodecs := make(map[uint64]struct{})\n\t\tif listSupported {\n\t\t\tfor _, code := range ipldmulticodec.ListEncoders() {\n\t\t\t\tsupportedCodecs[code] = struct{}{}\n\t\t\t}\n\t\t\tfor _, code := range ipldmulticodec.ListDecoders() {\n\t\t\t\tsupportedCodecs[code] = struct{}{}\n\t\t\t}\n\t\t\t// add libp2p-key\n\t\t\tsupportedCodecs[uint64(mc.Libp2pKey)] = struct{}{}\n\t\t}\n\n\t\tvar res []CodeAndName\n\t\tfor _, code := range mc.KnownCodes() {\n\t\t\tif code.Tag() == \"ipld\" {\n\t\t\t\tif listSupported {\n\t\t\t\t\tif _, ok := supportedCodecs[uint64(code)]; !ok {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tres = append(res, CodeAndName{int(code), mc.Code(code).String()})\n\t\t\t}\n\t\t}\n\t\treturn cmds.EmitOnce(resp, res)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, val []CodeAndName) error {\n\t\t\tnumeric, _ := req.Options[codecsNumericOptionName].(bool)\n\t\t\tcodeAndNameSorter{val}.Sort()\n\t\t\tfor _, v := range val {\n\t\t\t\tif numeric {\n\t\t\t\t\tfmt.Fprintf(w, \"%5d %s\\n\", v.Code, v.Name)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(w, \"%s\\n\", v.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: []CodeAndName{},\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\nvar hashesCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List available multihashes.\",\n\t\tShortDescription: `\n'ipfs cid hashes' relies on https://github.com/multiformats/go-multihash\n`,\n\t},\n\tOptions: codecsCmd.Options,\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tvar res []CodeAndName\n\t\t// use mhash.Codes in case at some point there are multiple names for a given code\n\t\tfor code, name := range mhash.Codes {\n\t\t\tif !verifcid.DefaultAllowlist.IsAllowed(code) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tres = append(res, CodeAndName{int(code), name})\n\t\t}\n\t\treturn cmds.EmitOnce(resp, res)\n\t},\n\tEncoders: codecsCmd.Encoders,\n\tType: codecsCmd.Type,\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\n// CidInspectRes represents the response from the inspect command.\ntype CidInspectRes struct {\n\tCid string `json:\"cid\"`\n\tVersion int `json:\"version\"`\n\tMultibase CidInspectBase `json:\"multibase\"`\n\tMulticodec CidInspectCodec `json:\"multicodec\"`\n\tMultihash CidInspectHash `json:\"multihash\"`\n\tCidV0 string `json:\"cidV0,omitempty\"`\n\tCidV1 string `json:\"cidV1\"`\n\tErrorMsg string `json:\"errorMsg,omitempty\"`\n}\n\ntype CidInspectBase struct {\n\tPrefix string `json:\"prefix\"`\n\tName string `json:\"name\"`\n}\n\ntype CidInspectCodec struct {\n\tCode uint64 `json:\"code\"`\n\tName string `json:\"name\"`\n}\n\ntype CidInspectHash struct {\n\tCode uint64 `json:\"code\"`\n\tName string `json:\"name\"`\n\tLength int `json:\"length\"`\n\tDigest string `json:\"digest\"`\n}\n\nvar inspectCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Inspect and display detailed information about a CID.\",\n\t\tShortDescription: `\n'ipfs cid inspect' breaks down a CID and displays its components:\n- CID version (0 or 1)\n- Multibase encoding (explicit for CIDv1, implicit for CIDv0)\n- Multicodec (DAG type)\n- Multihash (hash algorithm, length, and digest)\n- Equivalent CIDv0 and CIDv1 representations\n\nFor CIDv0, multibase, multicodec, and multihash are marked as\nimplicit because they are not explicitly encoded in the binary.\n\nIf a PeerID string is provided instead of a CID, a helpful error\nwith the equivalent CID representation is returned.\n\nUse --enc=json for machine-readable output same as the HTTP RPC API.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", true, false, \"CID to inspect.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcidStr := req.Arguments[0]\n\n\t\tc, err := cid.Decode(cidStr)\n\t\tif err != nil {\n\t\t\terrMsg := fmt.Sprintf(\"invalid CID: %s\", err)\n\t\t\t// PeerID fallback: try peer.Decode for legacy PeerIDs (12D3KooW..., Qm...)\n\t\t\tif pid, pidErr := peer.Decode(cidStr); pidErr == nil {\n\t\t\t\tpidCid := peer.ToCid(pid)\n\t\t\t\tcidV1, _ := pidCid.StringOfBase(mbase.Base36)\n\t\t\t\terrMsg += fmt.Sprintf(\"\\nNote: the value is a PeerID; inspect its CID representation instead:\\n %s\", cidV1)\n\t\t\t}\n\t\t\treturn cmds.EmitOnce(resp, &CidInspectRes{Cid: cidStr, ErrorMsg: errMsg})\n\t\t}\n\n\t\tres := &CidInspectRes{\n\t\t\tCid: cidStr,\n\t\t\tVersion: int(c.Version()),\n\t\t}\n\n\t\t// Multibase: always populated; CIDv0 uses implicit base58btc\n\t\tif c.Version() == 0 {\n\t\t\tres.Multibase = CidInspectBase{Prefix: \"z\", Name: \"base58btc\"}\n\t\t} else {\n\t\t\tbaseCode, _ := cid.ExtractEncoding(cidStr)\n\t\t\tres.Multibase = CidInspectBase{\n\t\t\t\tPrefix: string(rune(baseCode)),\n\t\t\t\tName: mbase.EncodingToStr[baseCode],\n\t\t\t}\n\t\t}\n\n\t\t// Multicodec\n\t\tcodecName := mc.Code(c.Type()).String()\n\t\tif codecName == \"\" || strings.HasPrefix(codecName, \"Code(\") {\n\t\t\tcodecName = \"unknown\"\n\t\t}\n\t\tres.Multicodec = CidInspectCodec{Code: c.Type(), Name: codecName}\n\n\t\t// Multihash\n\t\tdmh, err := mhash.Decode(c.Hash())\n\t\tif err != nil {\n\t\t\treturn cmds.EmitOnce(resp, &CidInspectRes{\n\t\t\t\tCid: cidStr,\n\t\t\t\tErrorMsg: fmt.Sprintf(\"failed to decode multihash: %s\", err),\n\t\t\t})\n\t\t}\n\t\thashName := mhash.Codes[dmh.Code]\n\t\tif hashName == \"\" {\n\t\t\thashName = \"unknown\"\n\t\t}\n\t\tres.Multihash = CidInspectHash{\n\t\t\tCode: dmh.Code,\n\t\t\tName: hashName,\n\t\t\tLength: dmh.Length,\n\t\t\tDigest: hex.EncodeToString(dmh.Digest),\n\t\t}\n\n\t\t// CIDv0: only possible with dag-pb + sha2-256-256\n\t\tif c.Type() == cid.DagProtobuf && dmh.Code == mhash.SHA2_256 && dmh.Length == 32 {\n\t\t\tres.CidV0 = cid.NewCidV0(c.Hash()).String()\n\t\t}\n\n\t\t// CIDv1: use base36 for libp2p-key, base32 for everything else\n\t\tv1 := cid.NewCidV1(c.Type(), c.Hash())\n\t\tv1Base := mbase.Encoding(mbase.Base32)\n\t\tif c.Type() == uint64(mc.Libp2pKey) {\n\t\t\tv1Base = mbase.Base36\n\t\t}\n\t\tv1Str, err := v1.StringOfBase(v1Base)\n\t\tif err != nil {\n\t\t\tv1Str = v1.String()\n\t\t}\n\t\tres.CidV1 = v1Str\n\n\t\treturn cmds.EmitOnce(resp, res)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, res *CidInspectRes) error {\n\t\t\tif res.ErrorMsg != \"\" {\n\t\t\t\treturn fmt.Errorf(\"%s\", res.ErrorMsg)\n\t\t\t}\n\n\t\t\timplicit := \"\"\n\t\t\tif res.Version == 0 {\n\t\t\t\timplicit = \", implicit\"\n\t\t\t}\n\n\t\t\tfmt.Fprintf(w, \"CID: %s\\n\", res.Cid)\n\t\t\tfmt.Fprintf(w, \"Version: %d\\n\", res.Version)\n\t\t\tif res.Version == 0 {\n\t\t\t\tfmt.Fprintf(w, \"Multibase: %s (implicit)\\n\", res.Multibase.Name)\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(w, \"Multibase: %s (%s)\\n\", res.Multibase.Name, res.Multibase.Prefix)\n\t\t\t}\n\t\t\tfmt.Fprintf(w, \"Multicodec: %s (0x%x%s)\\n\", res.Multicodec.Name, res.Multicodec.Code, implicit)\n\t\t\tfmt.Fprintf(w, \"Multihash: %s (0x%x%s)\\n\", res.Multihash.Name, res.Multihash.Code, implicit)\n\t\t\tfmt.Fprintf(w, \" Length: %d bytes\\n\", res.Multihash.Length)\n\t\t\tfmt.Fprintf(w, \" Digest: %s\\n\", res.Multihash.Digest)\n\n\t\t\tif res.CidV0 != \"\" {\n\t\t\t\tfmt.Fprintf(w, \"CIDv0: %s\\n\", res.CidV0)\n\t\t\t} else if res.Multicodec.Code != cid.DagProtobuf {\n\t\t\t\tfmt.Fprintf(w, \"CIDv0: not possible, requires dag-pb (0x70), got %s (0x%x)\\n\",\n\t\t\t\t\tres.Multicodec.Name, res.Multicodec.Code)\n\t\t\t} else if res.Multihash.Code != mhash.SHA2_256 {\n\t\t\t\tfmt.Fprintf(w, \"CIDv0: not possible, requires sha2-256 (0x12), got %s (0x%x)\\n\",\n\t\t\t\t\tres.Multihash.Name, res.Multihash.Code)\n\t\t\t} else if res.Multihash.Length != 32 {\n\t\t\t\tfmt.Fprintf(w, \"CIDv0: not possible, requires 32-byte digest, got %d\\n\",\n\t\t\t\t\tres.Multihash.Length)\n\t\t\t}\n\n\t\t\tfmt.Fprintf(w, \"CIDv1: %s\\n\", res.CidV1)\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: CidInspectRes{},\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\ntype multibaseSorter struct {\n\tdata []CodeAndName\n}\n\nfunc (s multibaseSorter) Sort() {\n\tslices.SortFunc(s.data, func(a, b CodeAndName) int {\n\t\tif n := cmp.Compare(unicode.ToLower(rune(a.Code)), unicode.ToLower(rune(b.Code))); n != 0 {\n\t\t\treturn n\n\t\t}\n\t\t// lowercase letters should come before uppercase\n\t\treturn cmp.Compare(b.Code, a.Code)\n\t})\n}\n\ntype codeAndNameSorter struct {\n\tdata []CodeAndName\n}\n\nfunc (s codeAndNameSorter) Sort() {\n\tslices.SortFunc(s.data, func(a, b CodeAndName) int {\n\t\treturn cmp.Compare(a.Code, b.Code)\n\t})\n}\n" }, { "path": "core/commands/cid_test.go", "content": "package commands\n\nimport (\n\t\"testing\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/multiformats/go-multibase\"\n)\n\nfunc TestCidFmtCmd(t *testing.T) {\n\tt.Parallel()\n\n\t// Test 'error when -v 0 is present and a custom -b is passed'\n\tt.Run(\"ipfs cid format -b z -v 0\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\ttype testV0PresentAndCustomBaseCase struct {\n\t\t\tMultibaseName string\n\t\t\tExpectedErrMsg string\n\t\t}\n\n\t\tvar testV0PresentAndCustomBaseCases []testV0PresentAndCustomBaseCase\n\n\t\tfor _, e := range multibase.EncodingToStr {\n\t\t\tvar testCase testV0PresentAndCustomBaseCase\n\n\t\t\tif e == \"base58btc\" {\n\t\t\t\ttestCase.MultibaseName = e\n\t\t\t\ttestCase.ExpectedErrMsg = \"\"\n\t\t\t\ttestV0PresentAndCustomBaseCases = append(testV0PresentAndCustomBaseCases, testCase)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\ttestCase.MultibaseName = e\n\t\t\ttestCase.ExpectedErrMsg = \"cannot convert to CIDv0 with any multibase other than the implicit base58btc\"\n\t\t\ttestV0PresentAndCustomBaseCases = append(testV0PresentAndCustomBaseCases, testCase)\n\t\t}\n\n\t\tfor _, e := range testV0PresentAndCustomBaseCases {\n\n\t\t\t// Mock request\n\t\t\treq := &cmds.Request{\n\t\t\t\tOptions: map[string]any{\n\t\t\t\t\tcidToVersionOptionName: \"0\",\n\t\t\t\t\tcidMultibaseOptionName: e.MultibaseName,\n\t\t\t\t\tcidFormatOptionName: \"%s\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Response emitter\n\t\t\tresp := cmds.ResponseEmitter(nil)\n\n\t\t\t// Call the CidFmtCmd function with the mock request and response\n\t\t\terr := cidFmtCmd.Run(req, resp, nil)\n\t\t\tif err == nil && e.MultibaseName == \"base58btc\" {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\terrMsg := err.Error()\n\t\t\tif errMsg != e.ExpectedErrMsg {\n\t\t\t\tt.Errorf(\"Expected %s, got %s instead\", e.ExpectedErrMsg, errMsg)\n\t\t\t}\n\t\t}\n\t})\n\n\t// Test 'upgrade CID to v1 when passing a custom -b and no -v is specified'\n\tt.Run(\"ipfs cid format -b z\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\ttype testImplicitVersionAndCustomMultibaseCase struct {\n\t\t\tVer string\n\t\t\tCidV1 string\n\t\t\tCidV0 string\n\t\t\tMultibaseName string\n\t\t}\n\n\t\tvar testCases = []testImplicitVersionAndCustomMultibaseCase{\n\t\t\t{\n\t\t\t\tVer: \"\",\n\t\t\t\tCidV1: \"zdj7WWwMSWGoyxYkkT7mHgYvr6tV8CYd77aYxxqSbg9HsiMcE\",\n\t\t\t\tCidV0: \"QmPr755CxWUwt39C2Yiw4UGKrv16uZhSgeZJmoHUUS9TSJ\",\n\t\t\t\tMultibaseName: \"z\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tVer: \"\",\n\t\t\t\tCidV1: \"CAFYBEIDI7ZABPGG3S63QW3AJG2XAZNE4NJQPN777WLWYRAIDG3TE5QFN3A======\",\n\t\t\t\tCidV0: \"QmVQVyEijmLb2cBQrowNQsaPbnUnJhfDK1sYe3wepm6ySf\",\n\t\t\t\tMultibaseName: \"base32padupper\",\n\t\t\t},\n\t\t}\n\t\tfor _, e := range testCases {\n\t\t\t// Mock request\n\t\t\treq := &cmds.Request{\n\t\t\t\tOptions: map[string]any{\n\t\t\t\t\tcidToVersionOptionName: e.Ver,\n\t\t\t\t\tcidMultibaseOptionName: e.MultibaseName,\n\t\t\t\t\tcidFormatOptionName: \"%s\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Response emitter\n\t\t\tresp := cmds.ResponseEmitter(nil)\n\n\t\t\t// Call the CidFmtCmd function with the mock request and response\n\t\t\terr := cidFmtCmd.Run(req, resp, nil)\n\n\t\t\tif err != nil {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t})\n}\n" }, { "path": "core/commands/cmdenv/cidbase.go", "content": "package cmdenv\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\tcid \"github.com/ipfs/go-cid\"\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nvar (\n\tOptionCidBase = cmds.StringOption(\"cid-base\", \"Multibase encoding used for version 1 CIDs in output.\")\n\tOptionUpgradeCidV0InOutput = cmds.BoolOption(\"upgrade-cidv0-in-output\", \"Upgrade version 0 to version 1 CIDs in output.\")\n)\n\n// GetCidEncoder processes the `cid-base` and `output-cidv1` options and\n// returns an encoder to use based on those parameters.\nfunc GetCidEncoder(req *cmds.Request) (cidenc.Encoder, error) {\n\treturn getCidBase(req, true)\n}\n\n// GetLowLevelCidEncoder is like GetCidEncoder but meant to be used by lower\n// level commands. It differs from GetCidEncoder in that CIDv0 are not, by\n// default, auto-upgraded to CIDv1.\nfunc GetLowLevelCidEncoder(req *cmds.Request) (cidenc.Encoder, error) {\n\treturn getCidBase(req, false)\n}\n\nfunc getCidBase(req *cmds.Request, autoUpgrade bool) (cidenc.Encoder, error) {\n\tbase, _ := req.Options[OptionCidBase.Name()].(string)\n\tupgrade, upgradeDefined := req.Options[OptionUpgradeCidV0InOutput.Name()].(bool)\n\n\te := cidenc.Default()\n\n\tif base != \"\" {\n\t\tvar err error\n\t\te.Base, err = mbase.EncoderByName(base)\n\t\tif err != nil {\n\t\t\treturn e, err\n\t\t}\n\t\tif autoUpgrade {\n\t\t\te.Upgrade = true\n\t\t}\n\t}\n\n\tif upgradeDefined {\n\t\te.Upgrade = upgrade\n\t}\n\n\treturn e, nil\n}\n\n// CidBaseDefined returns true if the `cid-base` option is specified on the\n// command line\nfunc CidBaseDefined(req *cmds.Request) bool {\n\tbase, _ := req.Options[\"cid-base\"].(string)\n\treturn base != \"\"\n}\n\n// CidEncoderFromPath creates a new encoder that is influenced from the encoded\n// Cid in a Path. For CIDv0 the multibase from the base encoder is used and\n// automatic upgrades are disabled. For CIDv1 the multibase from the CID is\n// used and upgrades are enabled.\n//\n// This logic is intentionally fuzzy and matches anything of the form\n// `CidLike`, `CidLike/...`, or `/namespace/CidLike/...`.\n//\n// For example:\n//\n// * Qm...\n// * Qm.../...\n// * /ipfs/Qm...\n// * /ipns/bafybeiahnxfi7fpmr5wtxs2imx4abnyn7fdxeiox7xxjem6zuiioqkh6zi/...\n// * /bzz/bafybeiahnxfi7fpmr5wtxs2imx4abnyn7fdxeiox7xxjem6zuiioqkh6zi/...\nfunc CidEncoderFromPath(p string) (cidenc.Encoder, error) {\n\tcomponents := strings.SplitN(p, \"/\", 4)\n\n\tvar maybeCid string\n\tif components[0] != \"\" {\n\t\t// No leading slash, first component is likely CID-like.\n\t\tmaybeCid = components[0]\n\t} else if len(components) < 3 {\n\t\t// Not enough components to include a CID.\n\t\treturn cidenc.Encoder{}, fmt.Errorf(\"no cid in path: %s\", p)\n\t} else {\n\t\tmaybeCid = components[2]\n\t}\n\tc, err := cid.Decode(maybeCid)\n\tif err != nil {\n\t\t// Ok, not a CID-like thing. Keep the current encoder.\n\t\treturn cidenc.Encoder{}, fmt.Errorf(\"no cid in path: %s\", p)\n\t}\n\tif c.Version() == 0 {\n\t\t// Version 0, use the base58 non-upgrading encoder.\n\t\treturn cidenc.Default(), nil\n\t}\n\n\t// Version 1+, extract multibase encoding.\n\tencoding, _, err := mbase.Decode(maybeCid)\n\tif err != nil {\n\t\t// This should be impossible, we've already decoded the cid.\n\t\tpanic(fmt.Sprintf(\"BUG: failed to get multibase decoder for CID %s\", maybeCid))\n\t}\n\n\treturn cidenc.Encoder{Base: mbase.MustNewEncoder(encoding), Upgrade: true}, nil\n}\n" }, { "path": "core/commands/cmdenv/cidbase_test.go", "content": "package cmdenv\n\nimport (\n\t\"testing\"\n\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nfunc TestEncoderFromPath(t *testing.T) {\n\ttest := func(path string, expected cidenc.Encoder) {\n\t\tactual, err := CidEncoderFromPath(path)\n\t\tif err != nil {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif actual != expected {\n\t\t\tt.Errorf(\"CidEncoderFromPath(%s) failed: expected %#v but got %#v\", path, expected, actual)\n\t\t}\n\t}\n\tp := \"QmRqVG8VGdKZ7KARqR96MV7VNHgWvEQifk94br5HpURpfu\"\n\tenc := cidenc.Default()\n\ttest(p, enc)\n\ttest(p+\"/a\", enc)\n\ttest(p+\"/a/b\", enc)\n\ttest(p+\"/a/b/\", enc)\n\ttest(p+\"/a/b/c\", enc)\n\ttest(\"/ipfs/\"+p, enc)\n\ttest(\"/ipfs/\"+p+\"/b\", enc)\n\n\tp = \"zb2rhfkM4FjkMLaUnygwhuqkETzbYXnUDf1P9MSmdNjW1w1Lk\"\n\tenc = cidenc.Encoder{\n\t\tBase: mbase.MustNewEncoder(mbase.Base58BTC),\n\t\tUpgrade: true,\n\t}\n\ttest(p, enc)\n\ttest(p+\"/a\", enc)\n\ttest(p+\"/a/b\", enc)\n\ttest(p+\"/a/b/\", enc)\n\ttest(p+\"/a/b/c\", enc)\n\ttest(\"/ipfs/\"+p, enc)\n\ttest(\"/ipfs/\"+p+\"/b\", enc)\n\ttest(\"/ipld/\"+p, enc)\n\ttest(\"/ipns/\"+p, enc) // even IPNS should work.\n\n\tp = \"bafyreifrcnyjokuw4i4ggkzg534tjlc25lqgt3ttznflmyv5fftdgu52hm\"\n\tenc = cidenc.Encoder{\n\t\tBase: mbase.MustNewEncoder(mbase.Base32),\n\t\tUpgrade: true,\n\t}\n\ttest(p, enc)\n\ttest(\"/ipfs/\"+p, enc)\n\ttest(\"/ipld/\"+p, enc)\n\n\tfor _, badPath := range []string{\n\t\t\"/ipld/\",\n\t\t\"/ipld\",\n\t\t\"/ipld//\",\n\t\t\"ipld//\",\n\t\t\"ipld\",\n\t\t\"\",\n\t\t\"ipns\",\n\t\t\"/ipfs/asdf\",\n\t\t\"/ipfs/...\",\n\t\t\"...\",\n\t\t\"abcdefg\",\n\t\t\"boo\",\n\t} {\n\t\t_, err := CidEncoderFromPath(badPath)\n\t\tif err == nil {\n\t\t\tt.Errorf(\"expected error extracting encoder from bad path: %s\", badPath)\n\t\t}\n\t}\n}\n" }, { "path": "core/commands/cmdenv/env.go", "content": "package cmdenv\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n\n\t\"github.com/ipfs/kubo/commands\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nvar log = logging.Logger(\"core/commands/cmdenv\")\n\n// GetNode extracts the node from the environment.\nfunc GetNode(env any) (*core.IpfsNode, error) {\n\tctx, ok := env.(*commands.Context)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"expected env to be of type %T, got %T\", ctx, env)\n\t}\n\n\treturn ctx.GetNode()\n}\n\n// GetApi extracts CoreAPI instance from the environment.\nfunc GetApi(env cmds.Environment, req *cmds.Request) (coreiface.CoreAPI, error) { //nolint\n\tctx, ok := env.(*commands.Context)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"expected env to be of type %T, got %T\", ctx, env)\n\t}\n\n\toffline, _ := req.Options[\"offline\"].(bool)\n\tif !offline {\n\t\toffline, _ = req.Options[\"local\"].(bool)\n\t\tif offline {\n\t\t\tlog.Errorf(\"Command '%s', --local is deprecated, use --offline instead\", strings.Join(req.Path, \" \"))\n\t\t}\n\t}\n\tapi, err := ctx.GetAPI()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif offline {\n\t\treturn api.WithOptions(options.Api.Offline(offline))\n\t}\n\n\treturn api, nil\n}\n\n// GetConfigRoot extracts the config root from the environment\nfunc GetConfigRoot(env cmds.Environment) (string, error) {\n\tctx, ok := env.(*commands.Context)\n\tif !ok {\n\t\treturn \"\", fmt.Errorf(\"expected env to be of type %T, got %T\", ctx, env)\n\t}\n\n\treturn ctx.ConfigRoot, nil\n}\n\n// EscNonPrint converts non-printable characters and backslash into Go escape\n// sequences. This is done to display all characters in a string, including\n// those that would otherwise not be displayed or have an undesirable effect on\n// the display.\nfunc EscNonPrint(s string) string {\n\tif !needEscape(s) {\n\t\treturn s\n\t}\n\n\tesc := strconv.Quote(s)\n\t// Remove first and last quote, and unescape quotes.\n\treturn strings.ReplaceAll(esc[1:len(esc)-1], `\\\"`, `\"`)\n}\n\nfunc needEscape(s string) bool {\n\tif strings.ContainsRune(s, '\\\\') {\n\t\treturn true\n\t}\n\tfor _, r := range s {\n\t\tif !strconv.IsPrint(r) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// provideCIDSync performs a synchronous/blocking provide operation to announce\n// the given CID to the DHT.\n//\n// - If the accelerated DHT client is used, a DHT lookup isn't needed, we\n// directly allocate provider records to closest peers.\n// - If Provide.DHT.SweepEnabled=true or OptimisticProvide=true, we make an\n// optimistic provide call.\n// - Else we make a standard provide call (much slower).\n//\n// IMPORTANT: The caller MUST verify DHT availability using HasActiveDHTClient()\n// before calling this function. Calling with a nil or invalid router will cause\n// a panic - this is the caller's responsibility to prevent.\nfunc provideCIDSync(ctx context.Context, router routing.Routing, c cid.Cid) error {\n\treturn router.Provide(ctx, c, true)\n}\n\n// ExecuteFastProvide immediately provides a root CID to the DHT, bypassing the regular\n// provide queue for faster content discovery. This function is reusable across commands\n// that add or import content, such as ipfs add and ipfs dag import.\n//\n// Parameters:\n// - ctx: context for synchronous provides\n// - ipfsNode: the IPFS node instance\n// - cfg: node configuration\n// - rootCid: the CID to provide\n// - wait: whether to block until provide completes (sync mode)\n// - isPinned: whether content is pinned\n// - isPinnedRoot: whether this is a pinned root CID\n// - isMFS: whether content is in MFS\n//\n// Return value:\n// - Returns nil if operation succeeded or was skipped (preconditions not met)\n// - Returns error only in sync mode (wait=true) when provide operation fails\n// - In async mode (wait=false), always returns nil (errors logged in goroutine)\n//\n// The function handles all precondition checks (Provide.Enabled, DHT availability,\n// strategy matching) and logs appropriately. In async mode, it launches a goroutine\n// with a detached context and timeout.\nfunc ExecuteFastProvide(\n\tctx context.Context,\n\tipfsNode *core.IpfsNode,\n\tcfg *config.Config,\n\trootCid cid.Cid,\n\twait bool,\n\tisPinned bool,\n\tisPinnedRoot bool,\n\tisMFS bool,\n) error {\n\tlog.Debugw(\"fast-provide-root: enabled\", \"wait\", wait)\n\n\t// Check preconditions for providing\n\tswitch {\n\tcase !cfg.Provide.Enabled.WithDefault(config.DefaultProvideEnabled):\n\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"Provide.Enabled is false\")\n\t\treturn nil\n\tcase cfg.Provide.DHT.Interval.WithDefault(config.DefaultProvideDHTInterval) == 0:\n\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"Provide.DHT.Interval is 0\")\n\t\treturn nil\n\tcase !ipfsNode.HasActiveDHTClient():\n\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"DHT not available\")\n\t\treturn nil\n\t}\n\n\t// Check if strategy allows providing this content\n\tstrategyStr := cfg.Provide.Strategy.WithDefault(config.DefaultProvideStrategy)\n\tstrategy := config.ParseProvideStrategy(strategyStr)\n\tshouldProvide := config.ShouldProvideForStrategy(strategy, isPinned, isPinnedRoot, isMFS)\n\n\tif !shouldProvide {\n\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"strategy does not match content\", \"strategy\", strategyStr, \"pinned\", isPinned, \"pinnedRoot\", isPinnedRoot, \"mfs\", isMFS)\n\t\treturn nil\n\t}\n\n\t// Execute provide operation\n\tif wait {\n\t\t// Synchronous mode: block until provide completes, return error on failure\n\t\tlog.Debugw(\"fast-provide-root: providing synchronously\", \"cid\", rootCid)\n\t\tif err := provideCIDSync(ctx, ipfsNode.DHTClient, rootCid); err != nil {\n\t\t\tlog.Warnw(\"fast-provide-root: sync provide failed\", \"cid\", rootCid, \"error\", err)\n\t\t\treturn fmt.Errorf(\"fast-provide: %w\", err)\n\t\t}\n\t\tlog.Debugw(\"fast-provide-root: sync provide completed\", \"cid\", rootCid)\n\t\treturn nil\n\t}\n\n\t// Asynchronous mode (default): fire-and-forget, don't block, always return nil\n\tlog.Debugw(\"fast-provide-root: providing asynchronously\", \"cid\", rootCid)\n\tgo func() {\n\t\t// Use detached context with timeout to prevent hanging on network issues\n\t\tctx, cancel := context.WithTimeout(context.Background(), config.DefaultFastProvideTimeout)\n\t\tdefer cancel()\n\t\tif err := provideCIDSync(ctx, ipfsNode.DHTClient, rootCid); err != nil {\n\t\t\tlog.Warnw(\"fast-provide-root: async provide failed\", \"cid\", rootCid, \"error\", err)\n\t\t} else {\n\t\t\tlog.Debugw(\"fast-provide-root: async provide completed\", \"cid\", rootCid)\n\t\t}\n\t}()\n\treturn nil\n}\n" }, { "path": "core/commands/cmdenv/env_test.go", "content": "package cmdenv\n\nimport (\n\t\"strconv\"\n\t\"testing\"\n)\n\nfunc TestEscNonPrint(t *testing.T) {\n\tb := []byte(\"hello\")\n\tb[2] = 0x7f\n\ts := string(b)\n\tif !needEscape(s) {\n\t\tt.Fatal(\"string needs escaping\")\n\t}\n\tif !hasNonPrintable(s) {\n\t\tt.Fatal(\"expected non-printable\")\n\t}\n\tif hasNonPrintable(EscNonPrint(s)) {\n\t\tt.Fatal(\"escaped string has non-printable\")\n\t}\n\tif EscNonPrint(`hel\\lo`) != `hel\\\\lo` {\n\t\tt.Fatal(\"backslash not escaped\")\n\t}\n\n\ts = `hello`\n\tif needEscape(s) {\n\t\tt.Fatal(\"string does not need escaping\")\n\t}\n\tif EscNonPrint(s) != s {\n\t\tt.Fatal(\"string should not have changed\")\n\t}\n\ts = `\"hello\"`\n\tif EscNonPrint(s) != s {\n\t\tt.Fatal(\"string should not have changed\")\n\t}\n\tif EscNonPrint(`\"hel\\\"lo\"`) != `\"hel\\\\\"lo\"` {\n\t\tt.Fatal(\"did not get expected escaped string\")\n\t}\n}\n\nfunc hasNonPrintable(s string) bool {\n\tfor _, r := range s {\n\t\tif !strconv.IsPrint(r) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n" }, { "path": "core/commands/cmdenv/file.go", "content": "package cmdenv\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/ipfs/boxo/files\"\n)\n\n// GetFileArg returns the next file from the directory or an error\nfunc GetFileArg(it files.DirIterator) (files.File, error) {\n\tif !it.Next() {\n\t\terr := it.Err()\n\t\tif err == nil {\n\t\t\terr = fmt.Errorf(\"expected a file argument\")\n\t\t}\n\t\treturn nil, err\n\t}\n\tfile := files.FileFromEntry(it)\n\tif file == nil {\n\t\treturn nil, fmt.Errorf(\"file argument was nil\")\n\t}\n\treturn file, nil\n}\n" }, { "path": "core/commands/cmdutils/sanitize.go", "content": "package cmdutils\n\nimport (\n\t\"strings\"\n\t\"unicode\"\n)\n\nconst maxRunes = 128\n\n// CleanAndTrim sanitizes untrusted strings from remote peers to prevent display issues\n// across web UIs, terminals, and logs. It replaces control characters, format characters,\n// and surrogates with U+FFFD (�), then enforces a maximum length of 128 runes.\n//\n// This follows the libp2p identify specification and RFC 9839 guidance:\n// replacing problematic code points is preferred over deletion as deletion\n// is a known security risk.\nfunc CleanAndTrim(str string) string {\n\t// Build sanitized result\n\tvar result []rune\n\tfor _, r := range str {\n\t\t// Replace control characters (Cc) with U+FFFD - prevents terminal escapes, CR, LF, etc.\n\t\tif unicode.Is(unicode.Cc, r) {\n\t\t\tresult = append(result, '\\uFFFD')\n\t\t\tcontinue\n\t\t}\n\t\t// Replace format characters (Cf) with U+FFFD - prevents RTL/LTR overrides, zero-width chars\n\t\tif unicode.Is(unicode.Cf, r) {\n\t\t\tresult = append(result, '\\uFFFD')\n\t\t\tcontinue\n\t\t}\n\t\t// Replace surrogate characters (Cs) with U+FFFD - invalid in UTF-8\n\t\tif unicode.Is(unicode.Cs, r) {\n\t\t\tresult = append(result, '\\uFFFD')\n\t\t\tcontinue\n\t\t}\n\t\t// Private use characters (Co) are preserved per spec\n\t\tresult = append(result, r)\n\t}\n\n\t// Convert to string and trim whitespace\n\tsanitized := strings.TrimSpace(string(result))\n\n\t// Enforce maximum length (128 runes, not bytes)\n\trunes := []rune(sanitized)\n\tif len(runes) > maxRunes {\n\t\treturn string(runes[:maxRunes])\n\t}\n\n\treturn sanitized\n}\n" }, { "path": "core/commands/cmdutils/utils.go", "content": "package cmdutils\n\nimport (\n\t\"fmt\"\n\t\"slices\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nconst (\n\tAllowBigBlockOptionName = \"allow-big-block\"\n\t// SoftBlockLimit is the maximum block size for bitswap transfer.\n\t// If this value changes, update the \"2MiB\" strings in error messages below.\n\tSoftBlockLimit = 2 * 1024 * 1024 // https://specs.ipfs.tech/bitswap-protocol/#block-sizes\n\tMaxPinNameBytes = 255 // Maximum number of bytes allowed for a pin name\n)\n\nvar AllowBigBlockOption cmds.Option\n\nfunc init() {\n\tAllowBigBlockOption = cmds.BoolOption(AllowBigBlockOptionName, \"Disable block size check and allow creation of blocks bigger than 2MiB. WARNING: such blocks won't be transferable over the standard bitswap.\").WithDefault(false)\n}\n\nfunc CheckCIDSize(req *cmds.Request, c cid.Cid, dagAPI coreiface.APIDagService) error {\n\tn, err := dagAPI.Get(req.Context, c)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"CheckCIDSize: getting dag: %w\", err)\n\t}\n\n\tnodeSize, err := n.Size()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"CheckCIDSize: getting node size: %w\", err)\n\t}\n\n\treturn CheckBlockSize(req, nodeSize)\n}\n\nfunc CheckBlockSize(req *cmds.Request, size uint64) error {\n\tallowAnyBlockSize, _ := req.Options[AllowBigBlockOptionName].(bool)\n\tif allowAnyBlockSize {\n\t\treturn nil\n\t}\n\n\t// Block size is limited to SoftBlockLimit (2MiB) as defined in the bitswap spec.\n\t// https://specs.ipfs.tech/bitswap-protocol/#block-sizes\n\tif size > SoftBlockLimit {\n\t\treturn fmt.Errorf(\"produced block is over 2MiB: big blocks can't be exchanged with other peers. consider using UnixFS for automatic chunking of bigger files, or pass --allow-big-block to override\")\n\t}\n\treturn nil\n}\n\n// ValidatePinName validates that a pin name does not exceed the maximum allowed byte length.\n// Returns an error if the name exceeds MaxPinNameBytes (255 bytes).\nfunc ValidatePinName(name string) error {\n\tif name == \"\" {\n\t\t// Empty names are allowed\n\t\treturn nil\n\t}\n\n\tnameBytes := len([]byte(name))\n\tif nameBytes > MaxPinNameBytes {\n\t\treturn fmt.Errorf(\"pin name is %d bytes (max %d bytes)\", nameBytes, MaxPinNameBytes)\n\t}\n\treturn nil\n}\n\n// PathOrCidPath returns a path.Path built from the argument. It keeps the old\n// behaviour by building a path from a CID string.\nfunc PathOrCidPath(str string) (path.Path, error) {\n\tp, err := path.NewPath(str)\n\tif err == nil {\n\t\treturn p, nil\n\t}\n\n\t// Save the original error before attempting fallback\n\toriginalErr := err\n\n\tif p, err := path.NewPath(\"/ipfs/\" + str); err == nil {\n\t\treturn p, nil\n\t}\n\n\t// Send back original err.\n\treturn nil, originalErr\n}\n\n// CloneAddrInfo returns a copy of the AddrInfo with a cloned Addrs slice.\n// This prevents data races if the sender reuses the backing array.\n// See: https://github.com/ipfs/kubo/issues/11116\nfunc CloneAddrInfo(ai peer.AddrInfo) peer.AddrInfo {\n\treturn peer.AddrInfo{\n\t\tID: ai.ID,\n\t\tAddrs: slices.Clone(ai.Addrs),\n\t}\n}\n" }, { "path": "core/commands/cmdutils/utils_test.go", "content": "package cmdutils\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPathOrCidPath(t *testing.T) {\n\tt.Run(\"valid path is returned as-is\", func(t *testing.T) {\n\t\tvalidPath := \"/ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG\"\n\t\tp, err := PathOrCidPath(validPath)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, validPath, p.String())\n\t})\n\n\tt.Run(\"valid CID is converted to /ipfs/ path\", func(t *testing.T) {\n\t\tcid := \"QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG\"\n\t\tp, err := PathOrCidPath(cid)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"/ipfs/\"+cid, p.String())\n\t})\n\n\tt.Run(\"valid ipns path is returned as-is\", func(t *testing.T) {\n\t\tvalidPath := \"/ipns/example.com\"\n\t\tp, err := PathOrCidPath(validPath)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, validPath, p.String())\n\t})\n\n\tt.Run(\"returns original error when both attempts fail\", func(t *testing.T) {\n\t\tinvalidInput := \"invalid!@#path\"\n\t\t_, err := PathOrCidPath(invalidInput)\n\t\trequire.Error(t, err)\n\n\t\t// The error should reference the original input attempt.\n\t\t// This ensures users get meaningful error messages about their actual input.\n\t\tassert.Contains(t, err.Error(), invalidInput,\n\t\t\t\"error should mention the original input\")\n\t\tassert.Contains(t, err.Error(), \"path does not have enough components\",\n\t\t\t\"error should describe the problem with the original input\")\n\t})\n\n\tt.Run(\"empty string returns error about original input\", func(t *testing.T) {\n\t\t_, err := PathOrCidPath(\"\")\n\t\trequire.Error(t, err)\n\n\t\t// Verify we're not getting an error about \"/ipfs/\" (the fallback)\n\t\terrMsg := err.Error()\n\t\tassert.NotContains(t, errMsg, \"/ipfs/\",\n\t\t\t\"error should be about empty input, not the fallback path\")\n\t})\n\n\tt.Run(\"invalid characters return error about original input\", func(t *testing.T) {\n\t\tinvalidInput := \"not a valid path or CID with spaces and /@#$%\"\n\t\t_, err := PathOrCidPath(invalidInput)\n\t\trequire.Error(t, err)\n\n\t\t// The error message should help debug the original input\n\t\tassert.True(t, strings.Contains(err.Error(), invalidInput) ||\n\t\t\tstrings.Contains(err.Error(), \"invalid\"),\n\t\t\t\"error should reference original problematic input\")\n\t})\n\n\tt.Run(\"CID with path is converted correctly\", func(t *testing.T) {\n\t\tcidWithPath := \"QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/file.txt\"\n\t\tp, err := PathOrCidPath(cidWithPath)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"/ipfs/\"+cidWithPath, p.String())\n\t})\n}\n\nfunc TestValidatePinName(t *testing.T) {\n\tt.Run(\"valid pin name is accepted\", func(t *testing.T) {\n\t\terr := ValidatePinName(\"my-pin-name\")\n\t\tassert.NoError(t, err)\n\t})\n\n\tt.Run(\"empty pin name is accepted\", func(t *testing.T) {\n\t\terr := ValidatePinName(\"\")\n\t\tassert.NoError(t, err)\n\t})\n\n\tt.Run(\"pin name at max length is accepted\", func(t *testing.T) {\n\t\tmaxName := strings.Repeat(\"a\", MaxPinNameBytes)\n\t\terr := ValidatePinName(maxName)\n\t\tassert.NoError(t, err)\n\t})\n\n\tt.Run(\"pin name exceeding max length is rejected\", func(t *testing.T) {\n\t\ttooLong := strings.Repeat(\"a\", MaxPinNameBytes+1)\n\t\terr := ValidatePinName(tooLong)\n\t\trequire.Error(t, err)\n\t\tassert.Contains(t, err.Error(), \"max\")\n\t})\n\n\tt.Run(\"pin name with unicode is counted by bytes\", func(t *testing.T) {\n\t\t// Unicode character can be multiple bytes\n\t\tunicodeName := strings.Repeat(\"🔒\", MaxPinNameBytes/4+1) // emoji is 4 bytes\n\t\terr := ValidatePinName(unicodeName)\n\t\trequire.Error(t, err)\n\t\tassert.Contains(t, err.Error(), \"bytes\")\n\t})\n}\n" }, { "path": "core/commands/commands.go", "content": "// Package commands implements the ipfs command interface\n//\n// Using github.com/ipfs/kubo/commands to define the command line and HTTP\n// APIs. This is the interface available to folks using IPFS from outside of\n// the Go language.\npackage commands\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"slices\"\n\t\"strings\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\ntype commandEncoder struct {\n\tw io.Writer\n}\n\nfunc (e *commandEncoder) Encode(v any) error {\n\tvar (\n\t\tcmd *Command\n\t\tok bool\n\t)\n\n\tif cmd, ok = v.(*Command); !ok {\n\t\treturn fmt.Errorf(`core/commands: unexpected type %T, expected *\"core/commands\".Command`, v)\n\t}\n\n\tfor _, s := range cmdPathStrings(cmd, cmd.showOpts) {\n\t\t_, err := e.w.Write([]byte(s + \"\\n\"))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\ntype Command struct {\n\tName string\n\tSubcommands []Command\n\tOptions []Option\n\n\tshowOpts bool\n}\n\ntype Option struct {\n\tNames []string\n}\n\nconst (\n\tflagsOptionName = \"flags\"\n)\n\n// CommandsCmd takes in a root command,\n// and returns a command that lists the subcommands in that root\nfunc CommandsCmd(root *cmds.Command) *cmds.Command {\n\treturn &cmds.Command{\n\t\tHelptext: cmds.HelpText{\n\t\t\tTagline: \"List all available commands.\",\n\t\t\tShortDescription: `Lists all available commands (and subcommands) and exits.`,\n\t\t},\n\t\tSubcommands: map[string]*cmds.Command{\n\t\t\t\"completion\": CompletionCmd(root),\n\t\t},\n\t\tOptions: []cmds.Option{\n\t\t\tcmds.BoolOption(flagsOptionName, \"f\", \"Show command flags\"),\n\t\t},\n\t\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n\t\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t\trootCmd := cmd2outputCmd(\"ipfs\", root)\n\t\t\trootCmd.showOpts, _ = req.Options[flagsOptionName].(bool)\n\t\t\treturn cmds.EmitOnce(res, &rootCmd)\n\t\t},\n\t\tEncoders: cmds.EncoderMap{\n\t\t\tcmds.Text: func(req *cmds.Request) func(io.Writer) cmds.Encoder {\n\t\t\t\treturn func(w io.Writer) cmds.Encoder { return &commandEncoder{w} }\n\t\t\t},\n\t\t},\n\t\tType: Command{},\n\t}\n}\n\nfunc cmd2outputCmd(name string, cmd *cmds.Command) Command {\n\topts := make([]Option, len(cmd.Options))\n\tfor i, opt := range cmd.Options {\n\t\topts[i] = Option{opt.Names()}\n\t}\n\n\toutput := Command{\n\t\tName: name,\n\t\tSubcommands: make([]Command, 0, len(cmd.Subcommands)),\n\t\tOptions: opts,\n\t}\n\n\tfor name, sub := range cmd.Subcommands {\n\t\toutput.Subcommands = append(output.Subcommands, cmd2outputCmd(name, sub))\n\t}\n\n\treturn output\n}\n\nfunc cmdPathStrings(cmd *Command, showOptions bool) []string {\n\tvar cmds []string\n\n\tvar recurse func(prefix string, cmd *Command)\n\trecurse = func(prefix string, cmd *Command) {\n\t\tnewPrefix := prefix + cmd.Name\n\t\tcmds = append(cmds, newPrefix)\n\t\tif prefix != \"\" && showOptions {\n\t\t\tfor _, options := range cmd.Options {\n\t\t\t\tvar cmdOpts []string\n\t\t\t\tfor _, flag := range options.Names {\n\t\t\t\t\tif len(flag) == 1 {\n\t\t\t\t\t\tflag = \"-\" + flag\n\t\t\t\t\t} else {\n\t\t\t\t\t\tflag = \"--\" + flag\n\t\t\t\t\t}\n\t\t\t\t\tcmdOpts = append(cmdOpts, newPrefix+\" \"+flag)\n\t\t\t\t}\n\t\t\t\tcmds = append(cmds, strings.Join(cmdOpts, \" / \"))\n\t\t\t}\n\t\t}\n\t\tfor _, sub := range cmd.Subcommands {\n\t\t\trecurse(newPrefix+\" \", &sub)\n\t\t}\n\t}\n\n\trecurse(\"\", cmd)\n\tslices.Sort(cmds)\n\treturn cmds\n}\n\nfunc CompletionCmd(root *cmds.Command) *cmds.Command {\n\treturn &cmds.Command{\n\t\tHelptext: cmds.HelpText{\n\t\t\tTagline: \"Generate shell completions.\",\n\t\t},\n\t\tNoRemote: true,\n\t\tSubcommands: map[string]*cmds.Command{\n\t\t\t\"bash\": {\n\t\t\t\tHelptext: cmds.HelpText{\n\t\t\t\t\tTagline: \"Generate bash shell completions.\",\n\t\t\t\t\tShortDescription: \"Generates command completions for the bash shell.\",\n\t\t\t\t\tLongDescription: `\nGenerates command completions for the bash shell.\n\nThe simplest way to see it working is write the completions\nto a file and then source it:\n\n > ipfs commands completion bash > ipfs-completion.bash\n > source ./ipfs-completion.bash\n\nTo install the completions permanently, they can be moved to\n/etc/bash_completion.d or sourced from your ~/.bashrc file.\n`,\n\t\t\t\t},\n\t\t\t\tNoRemote: true,\n\t\t\t\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t\t\t\tvar buf bytes.Buffer\n\t\t\t\t\tif err := writeBashCompletions(root, &buf); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\tres.SetLength(uint64(buf.Len()))\n\t\t\t\t\treturn res.Emit(&buf)\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"zsh\": {\n\t\t\t\tHelptext: cmds.HelpText{\n\t\t\t\t\tTagline: \"Generate zsh shell completions.\",\n\t\t\t\t\tShortDescription: \"Generates command completions for the zsh shell.\",\n\t\t\t\t\tLongDescription: `\nGenerates command completions for the zsh shell.\n\nThe simplest way to see it working is write the completions\nto a file and then source it:\n\n > ipfs commands completion zsh > ipfs-completion.zsh\n > source ./ipfs-completion.zsh\n\nTo install the completions permanently, they can be moved to\n/etc/zsh/completions or sourced from your ~/.zshrc file.\n`,\n\t\t\t\t},\n\t\t\t\tNoRemote: true,\n\t\t\t\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t\t\t\tvar buf bytes.Buffer\n\t\t\t\t\tif err := writeZshCompletions(root, &buf); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\tres.SetLength(uint64(buf.Len()))\n\t\t\t\t\treturn res.Emit(&buf)\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"fish\": {\n\t\t\t\tHelptext: cmds.HelpText{\n\t\t\t\t\tTagline: \"Generate fish shell completions.\",\n\t\t\t\t\tShortDescription: \"Generates command completions for the fish shell.\",\n\t\t\t\t\tLongDescription: `\nGenerates command completions for the fish shell.\n\nThe simplest way to see it working is write the completions\nto a file and then source it:\n\n > ipfs commands completion fish > ipfs-completion.fish\n > source ./ipfs-completion.fish\n\nTo install the completions permanently, they can be moved to\n/etc/fish/completions or ~/.config/fish/completions or sourced from your ~/.config/fish/config.fish file.\n`,\n\t\t\t\t},\n\t\t\t\tNoRemote: true,\n\t\t\t\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t\t\t\tvar buf bytes.Buffer\n\t\t\t\t\tif err := writeFishCompletions(root, &buf); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\tres.SetLength(uint64(buf.Len()))\n\t\t\t\t\treturn res.Emit(&buf)\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n}\n\ntype nonFatalError string\n\n// streamResult is a helper function to stream results that possibly\n// contain non-fatal errors. The helper function is allowed to panic\n// on internal errors.\nfunc streamResult(procVal func(any, io.Writer) nonFatalError) func(cmds.Response, cmds.ResponseEmitter) error {\n\treturn func(res cmds.Response, re cmds.ResponseEmitter) (rerr error) {\n\t\tdefer func() {\n\t\t\tif r := recover(); r != nil {\n\t\t\t\trerr = fmt.Errorf(\"internal error: %v\", r)\n\t\t\t}\n\t\t}()\n\n\t\tvar errors bool\n\t\tfor {\n\t\t\tv, err := res.Next()\n\t\t\tif err != nil {\n\t\t\t\tif err == io.EOF {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\trerr = err\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\terrorMsg := procVal(v, os.Stdout)\n\n\t\t\tif errorMsg != \"\" {\n\t\t\t\terrors = true\n\t\t\t\tfmt.Fprintf(os.Stderr, \"%s\\n\", errorMsg)\n\t\t\t}\n\t\t}\n\n\t\tif errors {\n\t\t\trerr = fmt.Errorf(\"errors while displaying some entries\")\n\t\t}\n\t\treturn\n\t}\n}\n" }, { "path": "core/commands/commands_test.go", "content": "package commands\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc collectPaths(prefix string, cmd *cmds.Command, out map[string]struct{}) {\n\tfor name, sub := range cmd.Subcommands {\n\t\tpath := prefix + \"/\" + name\n\t\tout[path] = struct{}{}\n\t\tcollectPaths(path, sub, out)\n\t}\n}\n\nfunc TestCommands(t *testing.T) {\n\tlist := []string{\n\t\t\"/add\",\n\t\t\"/bitswap\",\n\t\t\"/bitswap/ledger\",\n\t\t\"/bitswap/reprovide\",\n\t\t\"/bitswap/stat\",\n\t\t\"/bitswap/wantlist\",\n\t\t\"/block\",\n\t\t\"/block/get\",\n\t\t\"/block/put\",\n\t\t\"/block/rm\",\n\t\t\"/block/stat\",\n\t\t\"/bootstrap\",\n\t\t\"/bootstrap/add\",\n\t\t\"/bootstrap/list\",\n\t\t\"/bootstrap/rm\",\n\t\t\"/bootstrap/rm/all\",\n\t\t\"/cat\",\n\t\t\"/cid\",\n\t\t\"/cid/base32\",\n\t\t\"/cid/bases\",\n\t\t\"/cid/codecs\",\n\t\t\"/cid/format\",\n\t\t\"/cid/hashes\",\n\t\t\"/cid/inspect\",\n\t\t\"/commands\",\n\t\t\"/commands/completion\",\n\t\t\"/commands/completion/bash\",\n\t\t\"/commands/completion/fish\",\n\t\t\"/commands/completion/zsh\",\n\t\t\"/config\",\n\t\t\"/config/edit\",\n\t\t\"/config/profile\",\n\t\t\"/config/profile/apply\",\n\t\t\"/config/replace\",\n\t\t\"/config/show\",\n\t\t\"/dag\",\n\t\t\"/dag/export\",\n\t\t\"/dag/get\",\n\t\t\"/dag/import\",\n\t\t\"/dag/put\",\n\t\t\"/dag/resolve\",\n\t\t\"/dag/stat\",\n\t\t\"/dht\",\n\t\t\"/dht/query\",\n\t\t\"/dht/findprovs\",\n\t\t\"/dht/findpeer\",\n\t\t\"/dht/get\",\n\t\t\"/dht/provide\",\n\t\t\"/dht/put\",\n\t\t\"/routing\",\n\t\t\"/routing/put\",\n\t\t\"/routing/get\",\n\t\t\"/routing/findpeer\",\n\t\t\"/routing/findprovs\",\n\t\t\"/routing/provide\",\n\t\t\"/routing/reprovide\",\n\t\t\"/diag\",\n\t\t\"/diag/cmds\",\n\t\t\"/diag/cmds/clear\",\n\t\t\"/diag/cmds/set-time\",\n\t\t\"/diag/datastore\",\n\t\t\"/diag/datastore/count\",\n\t\t\"/diag/datastore/get\",\n\t\t\"/diag/datastore/put\",\n\t\t\"/diag/profile\",\n\t\t\"/diag/sys\",\n\t\t\"/files\",\n\t\t\"/files/chcid\",\n\t\t\"/files/cp\",\n\t\t\"/files/flush\",\n\t\t\"/files/ls\",\n\t\t\"/files/mkdir\",\n\t\t\"/files/mv\",\n\t\t\"/files/read\",\n\t\t\"/files/rm\",\n\t\t\"/files/stat\",\n\t\t\"/files/write\",\n\t\t\"/files/chmod\",\n\t\t\"/files/chroot\",\n\t\t\"/files/touch\",\n\t\t\"/filestore\",\n\t\t\"/filestore/dups\",\n\t\t\"/filestore/ls\",\n\t\t\"/filestore/verify\",\n\t\t\"/get\",\n\t\t\"/id\",\n\t\t\"/key\",\n\t\t\"/key/export\",\n\t\t\"/key/gen\",\n\t\t\"/key/import\",\n\t\t\"/key/list\",\n\t\t\"/key/ls\",\n\t\t\"/key/rename\",\n\t\t\"/key/rm\",\n\t\t\"/key/rotate\",\n\t\t\"/key/sign\",\n\t\t\"/key/verify\",\n\t\t\"/log\",\n\t\t\"/log/level\",\n\t\t\"/log/ls\",\n\t\t\"/log/tail\",\n\t\t\"/ls\",\n\t\t\"/mount\",\n\t\t\"/multibase\",\n\t\t\"/multibase/decode\",\n\t\t\"/multibase/encode\",\n\t\t\"/multibase/transcode\",\n\t\t\"/multibase/list\",\n\t\t\"/name\",\n\t\t\"/name/get\",\n\t\t\"/name/inspect\",\n\t\t\"/name/publish\",\n\t\t\"/name/pubsub\",\n\t\t\"/name/pubsub/cancel\",\n\t\t\"/name/pubsub/state\",\n\t\t\"/name/pubsub/subs\",\n\t\t\"/name/put\",\n\t\t\"/name/resolve\",\n\t\t\"/object\",\n\t\t\"/object/data\",\n\t\t\"/object/diff\",\n\t\t\"/object/get\",\n\t\t\"/object/links\",\n\t\t\"/object/new\",\n\t\t\"/object/patch\",\n\t\t\"/object/patch/add-link\",\n\t\t\"/object/patch/append-data\",\n\t\t\"/object/patch/rm-link\",\n\t\t\"/object/patch/set-data\",\n\t\t\"/object/put\",\n\t\t\"/object/stat\",\n\t\t\"/p2p\",\n\t\t\"/p2p/close\",\n\t\t\"/p2p/forward\",\n\t\t\"/p2p/listen\",\n\t\t\"/p2p/ls\",\n\t\t\"/p2p/stream\",\n\t\t\"/p2p/stream/close\",\n\t\t\"/p2p/stream/ls\",\n\t\t\"/pin\",\n\t\t\"/pin/add\",\n\t\t\"/pin/ls\",\n\t\t\"/pin/remote\",\n\t\t\"/pin/remote/add\",\n\t\t\"/pin/remote/ls\",\n\t\t\"/pin/remote/rm\",\n\t\t\"/pin/remote/service\",\n\t\t\"/pin/remote/service/add\",\n\t\t\"/pin/remote/service/ls\",\n\t\t\"/pin/remote/service/rm\",\n\t\t\"/pin/rm\",\n\t\t\"/pin/update\",\n\t\t\"/pin/verify\",\n\t\t\"/ping\",\n\t\t\"/provide\",\n\t\t\"/provide/clear\",\n\t\t\"/provide/stat\",\n\t\t\"/pubsub\",\n\t\t\"/pubsub/ls\",\n\t\t\"/pubsub/peers\",\n\t\t\"/pubsub/pub\",\n\t\t\"/pubsub/reset\",\n\t\t\"/pubsub/sub\",\n\t\t\"/refs\",\n\t\t\"/refs/local\",\n\t\t\"/repo\",\n\t\t\"/repo/gc\",\n\t\t\"/repo/migrate\",\n\t\t\"/repo/stat\",\n\t\t\"/repo/verify\",\n\t\t\"/repo/version\",\n\t\t\"/repo/ls\",\n\t\t\"/resolve\",\n\t\t\"/shutdown\",\n\t\t\"/stats\",\n\t\t\"/stats/bitswap\",\n\t\t\"/stats/bw\",\n\t\t\"/stats/dht\",\n\t\t\"/stats/provide\",\n\t\t\"/stats/reprovide\",\n\t\t\"/stats/repo\",\n\t\t\"/swarm\",\n\t\t\"/swarm/addrs\",\n\t\t\"/swarm/addrs/autonat\",\n\t\t\"/swarm/addrs/listen\",\n\t\t\"/swarm/addrs/local\",\n\t\t\"/swarm/connect\",\n\t\t\"/swarm/disconnect\",\n\t\t\"/swarm/filters\",\n\t\t\"/swarm/filters/add\",\n\t\t\"/swarm/filters/rm\",\n\t\t\"/swarm/peers\",\n\t\t\"/swarm/peering\",\n\t\t\"/swarm/peering/add\",\n\t\t\"/swarm/peering/ls\",\n\t\t\"/swarm/peering/rm\",\n\t\t\"/swarm/resources\",\n\t\t\"/update\",\n\t\t\"/version\",\n\t\t\"/version/check\",\n\t\t\"/version/deps\",\n\t}\n\n\tcmdSet := make(map[string]struct{})\n\tcollectPaths(\"\", Root, cmdSet)\n\n\tfor _, path := range list {\n\t\tif _, ok := cmdSet[path]; !ok {\n\t\t\tt.Errorf(\"%q not in result\", path)\n\t\t} else {\n\t\t\tdelete(cmdSet, path)\n\t\t}\n\t}\n\n\tfor path := range cmdSet {\n\t\tt.Errorf(\"%q in result but shouldn't be\", path)\n\t}\n\n\tfor _, path := range list {\n\t\tpath = path[1:] // remove leading slash\n\t\tsplit := strings.Split(path, \"/\")\n\t\tsub, err := Root.Get(split)\n\t\tif err != nil {\n\t\t\tt.Errorf(\"error getting subcommand %q: %v\", path, err)\n\t\t} else if sub == nil {\n\t\t\tt.Errorf(\"subcommand %q is nil even though there was no error\", path)\n\t\t}\n\t}\n}\n" }, { "path": "core/commands/completion.go", "content": "package commands\n\nimport (\n\t\"io\"\n\t\"slices\"\n\t\"strings\"\n\t\"text/template\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\ntype completionCommand struct {\n\tName string\n\tFullName string\n\tDescription string\n\tSubcommands []*completionCommand\n\tFlags []*singleOption\n\tOptions []*singleOption\n\tShortFlags []string\n\tShortOptions []string\n\tLongFlags []string\n\tLongOptions []string\n\tIsFinal bool\n}\n\ntype singleOption struct {\n\tLongNames []string\n\tShortNames []string\n\tDescription string\n}\n\nfunc commandToCompletions(name string, fullName string, cmd *cmds.Command) *completionCommand {\n\tparsed := &completionCommand{\n\t\tName: name,\n\t\tFullName: fullName,\n\t\tDescription: cmd.Helptext.Tagline,\n\t\tIsFinal: len(cmd.Subcommands) == 0,\n\t}\n\tfor name, subCmd := range cmd.Subcommands {\n\t\tparsed.Subcommands = append(parsed.Subcommands,\n\t\t\tcommandToCompletions(name, fullName+\" \"+name, subCmd))\n\t}\n\tslices.SortFunc(parsed.Subcommands, func(a, b *completionCommand) int {\n\t\treturn strings.Compare(a.Name, b.Name)\n\t})\n\n\tfor _, opt := range cmd.Options {\n\t\tflag := &singleOption{Description: opt.Description()}\n\t\tflag.LongNames = append(flag.LongNames, opt.Name())\n\t\tif opt.Type() == cmds.Bool {\n\t\t\tparsed.LongFlags = append(parsed.LongFlags, opt.Name())\n\t\t\tfor _, name := range opt.Names() {\n\t\t\t\tif len(name) == 1 {\n\t\t\t\t\tparsed.ShortFlags = append(parsed.ShortFlags, name)\n\t\t\t\t\tflag.ShortNames = append(flag.ShortNames, name)\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tparsed.Flags = append(parsed.Flags, flag)\n\t\t} else {\n\t\t\tparsed.LongOptions = append(parsed.LongOptions, opt.Name())\n\t\t\tfor _, name := range opt.Names() {\n\t\t\t\tif len(name) == 1 {\n\t\t\t\t\tparsed.ShortOptions = append(parsed.ShortOptions, name)\n\t\t\t\t\tflag.ShortNames = append(flag.ShortNames, name)\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tparsed.Options = append(parsed.Options, flag)\n\t\t}\n\t}\n\tslices.Sort(parsed.LongFlags)\n\tslices.Sort(parsed.ShortFlags)\n\tslices.Sort(parsed.LongOptions)\n\tslices.Sort(parsed.ShortOptions)\n\treturn parsed\n}\n\nvar bashCompletionTemplate, fishCompletionTemplate, zshCompletionTemplate *template.Template\n\nfunc init() {\n\tcommandTemplate := template.Must(template.New(\"command\").Parse(`\nwhile [[ ${index} -lt ${COMP_CWORD} ]]; do\n case \"${COMP_WORDS[index]}\" in\n -*)\n\t let index++\n continue\n\t ;;\n {{ range .Subcommands }}\n\t\"{{ .Name }}\")\n\t let index++\n\t {{ template \"command\" . }}\n\t return 0\n ;;\n {{ end }}\n esac\n break\ndone\n\nif [[ \"${word}\" == -* ]]; then\n{{ if .ShortFlags -}}\n _ipfs_compgen -W $'{{ range .ShortFlags }}-{{.}} \\n{{end}}' -- \"${word}\"\n{{ end -}}\n{{- if .ShortOptions -}}\n _ipfs_compgen -S = -W $'{{ range .ShortOptions }}-{{.}}\\n{{end}}' -- \"${word}\"\n{{ end -}}\n{{- if .LongFlags -}}\n _ipfs_compgen -W $'{{ range .LongFlags }}--{{.}} \\n{{end}}' -- \"${word}\"\n{{ end -}}\n{{- if .LongOptions -}}\n _ipfs_compgen -S = -W $'{{ range .LongOptions }}--{{.}}\\n{{end}}' -- \"${word}\"\n{{ end -}}\n return 0\nfi\n\nwhile [[ ${index} -lt ${COMP_CWORD} ]]; do\n if [[ \"${COMP_WORDS[index]}\" != -* ]]; then\n let argidx++\n fi\n let index++\ndone\n\n{{- if .Subcommands }}\nif [[ \"${argidx}\" -eq 0 ]]; then\n _ipfs_compgen -W $'{{ range .Subcommands }}{{.Name}} \\n{{end}}' -- \"${word}\"\nfi\n{{ end -}}\n`))\n\n\tbashCompletionTemplate = template.Must(commandTemplate.New(\"root\").Parse(`#!/bin/bash\n\n_ipfs_compgen() {\n local oldifs=\"$IFS\"\n IFS=$'\\n'\n while read -r line; do\n COMPREPLY+=(\"$line\")\n done < <(compgen \"$@\")\n IFS=\"$oldifs\"\n}\n\n_ipfs() {\n COMPREPLY=()\n local index=1\n local argidx=0\n local word=\"${COMP_WORDS[COMP_CWORD]}\"\n {{ template \"command\" . }}\n}\ncomplete -o nosort -o nospace -o default -F _ipfs ipfs\n`))\n\n\tzshCompletionTemplate = template.Must(commandTemplate.New(\"root\").Parse(`#!bin/zsh\nautoload bashcompinit\nbashcompinit\n_ipfs_compgen() {\nlocal oldifs=\"$IFS\"\nIFS=$'\\n'\nwhile read -r line; do\n\tCOMPREPLY+=(\"$line\")\ndone < <(compgen \"$@\")\nIFS=\"$oldifs\"\n}\n\n_ipfs() {\nCOMPREPLY=()\nlocal index=1\nlocal argidx=0\nlocal word=\"${COMP_WORDS[COMP_CWORD]}\"\n{{ template \"command\" . }}\n}\ncomplete -o nosort -o nospace -o default -F _ipfs ipfs\n`))\n\n\tfishCommandTemplate := template.Must(template.New(\"command\").Parse(`\n{{- if .IsFinal -}}\ncomplete -c ipfs -n '__fish_ipfs_seen_all_subcommands_from{{ .FullName }}' -F\n{{ end -}}\n{{- range .Flags -}}\n complete -c ipfs -n '__fish_ipfs_seen_all_subcommands_from{{ $.FullName }}' {{ range .ShortNames }}-s {{.}} {{end}}{{ range .LongNames }}-l {{.}} {{end}}-d \"{{ .Description }}\"\n{{ end -}}\n{{- range .Options -}}\n complete -c ipfs -n '__fish_ipfs_seen_all_subcommands_from{{ $.FullName }}' -r {{ range .ShortNames }}-s {{.}} {{end}}{{ range .LongNames }}-l {{.}} {{end}}-d \"{{ .Description }}\"\n{{ end -}}\n\n{{- range .Subcommands }}\n#{{ .FullName }}\ncomplete -c ipfs -n '__fish_ipfs_use_subcommand{{ .FullName }}' -a {{ .Name }} -d \"{{ .Description }}\"\n{{ template \"command\" . }}\n{{ end -}}\n\t`))\n\tfishCompletionTemplate = template.Must(fishCommandTemplate.New(\"root\").Parse(`#!/usr/bin/env fish\nfunction __fish_ipfs_seen_all_subcommands_from\n set -l cmd (commandline -poc)\n set -e cmd[1]\n for c in $argv\n if not contains -- $c $cmd\n return 1\n end\n end\n return 0\nend\n\nfunction __fish_ipfs_use_subcommand\n\tset -e argv[-1]\n\tset -l cmd (commandline -poc)\n\tset -e cmd[1]\n\tfor i in $cmd\n\t switch $i\n\t\t case '-*'\n\t\t\t continue\n case $argv[1]\n set argv $argv[2..]\n continue\n case '*'\n return 1\n end\n\tend\n\ttest -z \"$argv\"\nend\n\ncomplete -c ipfs -l help -d \"Show the full command help text.\"\n\ncomplete -c ipfs --keep-order --no-files\n\n{{ template \"command\" . }}\n`))\n}\n\n// writeBashCompletions generates a bash completion script for the given command tree.\nfunc writeBashCompletions(cmd *cmds.Command, out io.Writer) error {\n\tcmds := commandToCompletions(\"ipfs\", \"\", cmd)\n\treturn bashCompletionTemplate.Execute(out, cmds)\n}\n\n// writeFishCompletions generates a fish completion script for the given command tree.\nfunc writeFishCompletions(cmd *cmds.Command, out io.Writer) error {\n\tcmds := commandToCompletions(\"ipfs\", \"\", cmd)\n\treturn fishCompletionTemplate.Execute(out, cmds)\n}\n\nfunc writeZshCompletions(cmd *cmds.Command, out io.Writer) error {\n\tcmds := commandToCompletions(\"ipfs\", \"\", cmd)\n\treturn zshCompletionTemplate.Execute(out, cmds)\n}\n" }, { "path": "core/commands/config.go", "content": "package commands\n\nimport (\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"maps\"\n\t\"os\"\n\t\"os/exec\"\n\t\"slices\"\n\t\"strings\"\n\n\t\"github.com/anmitsu/go-shlex\"\n\t\"github.com/elgris/jsondiff\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n)\n\n// ConfigUpdateOutput is config profile apply command's output\ntype ConfigUpdateOutput struct {\n\tOldCfg map[string]any\n\tNewCfg map[string]any\n}\n\ntype ConfigField struct {\n\tKey string\n\tValue any\n}\n\nconst (\n\tconfigBoolOptionName = \"bool\"\n\tconfigJSONOptionName = \"json\"\n\tconfigDryRunOptionName = \"dry-run\"\n\tconfigExpandAutoName = \"expand-auto\"\n)\n\nvar ConfigCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Get and set IPFS config values.\",\n\t\tShortDescription: `\n'ipfs config' controls configuration variables. It works like 'git config'.\nThe configuration values are stored in a config file inside your IPFS_PATH.`,\n\t\tLongDescription: `\n'ipfs config' controls configuration variables. It works\nmuch like 'git config'. The configuration values are stored in a config\nfile inside your IPFS repository (IPFS_PATH).\n\nExamples:\n\nGet the value of the 'Routing.Type' key:\n\n $ ipfs config Routing.Type\n\nSet the value of the 'Routing.Type' key:\n\n $ ipfs config Routing.Type auto\n\nSet multiple values in the 'Addresses.AppendAnnounce' array:\n\n $ ipfs config Addresses.AppendAnnounce --json \\\n '[\"/dns4/a.example.com/tcp/4001\", \"/dns4/b.example.com/tcp/4002\"]'\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"show\": configShowCmd,\n\t\t\"edit\": configEditCmd,\n\t\t\"replace\": configReplaceCmd,\n\t\t\"profile\": configProfileCmd,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, false, \"The key of the config entry (e.g. \\\"Addresses.API\\\").\"),\n\t\tcmds.StringArg(\"value\", false, false, \"The value to set the config entry to.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(configBoolOptionName, \"Set a boolean value.\"),\n\t\tcmds.BoolOption(configJSONOptionName, \"Parse stringified JSON.\"),\n\t\tcmds.BoolOption(configExpandAutoName, \"Expand 'auto' placeholders to their expanded values from AutoConf service.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\targs := req.Arguments\n\t\tkey := args[0]\n\n\t\tvar output *ConfigField\n\n\t\t// This is a temporary fix until we move the private key out of the config file\n\t\tswitch strings.ToLower(key) {\n\t\tcase \"identity\", \"identity.privkey\":\n\t\t\treturn errors.New(\"cannot show or change private key through API\")\n\t\tdefault:\n\t\t}\n\n\t\t// Temporary fix until we move ApiKey secrets out of the config file\n\t\t// (remote services are a map, so more advanced blocking is required)\n\t\tif blocked := matchesGlobPrefix(key, config.PinningConcealSelector); blocked {\n\t\t\treturn errors.New(\"cannot show or change pinning services credentials\")\n\t\t}\n\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tif len(args) == 2 {\n\t\t\t// Check if user is trying to write config with expand flag\n\t\t\tif expandAuto, _ := req.Options[configExpandAutoName].(bool); expandAuto {\n\t\t\t\treturn fmt.Errorf(\"--expand-auto can only be used for reading config values, not for setting them\")\n\t\t\t}\n\n\t\t\tvalue := args[1]\n\n\t\t\tif parseJSON, _ := req.Options[configJSONOptionName].(bool); parseJSON {\n\t\t\t\tvar jsonVal any\n\t\t\t\tif err := json.Unmarshal([]byte(value), &jsonVal); err != nil {\n\t\t\t\t\terr = fmt.Errorf(\"failed to unmarshal json. %s\", err)\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\toutput, err = setConfig(r, key, jsonVal)\n\t\t\t} else if isbool, _ := req.Options[configBoolOptionName].(bool); isbool {\n\t\t\t\toutput, err = setConfig(r, key, value == \"true\")\n\t\t\t} else {\n\t\t\t\toutput, err = setConfig(r, key, value)\n\t\t\t}\n\t\t} else {\n\t\t\t// Check if user wants to expand auto values for getter\n\t\t\texpandAuto, _ := req.Options[configExpandAutoName].(bool)\n\t\t\tif expandAuto {\n\t\t\t\toutput, err = getConfigWithAutoExpand(r, key)\n\t\t\t} else {\n\t\t\t\toutput, err = getConfig(r, key)\n\t\t\t}\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, output)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *ConfigField) error {\n\t\t\tif len(req.Arguments) == 2 {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tbuf, err := config.HumanOutput(out.Value)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tbuf = append(buf, byte('\\n'))\n\n\t\t\t_, err = w.Write(buf)\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: ConfigField{},\n}\n\n// matchesGlobPrefix returns true if and only if the key matches the glob.\n// The key is a sequence of string \"parts\", separated by commas.\n// The glob is a sequence of string \"patterns\".\n// matchesGlobPrefix tries to match all of the first K parts to the first K patterns, respectively,\n// where K is the length of the shorter of key or glob.\n// A pattern matches a part if and only if the pattern is \"*\" or the lowercase pattern equals the lowercase part.\n//\n// For example:\n//\n//\tmatchesGlobPrefix(\"foo.bar\", []string{\"*\", \"bar\", \"baz\"}) returns true\n//\tmatchesGlobPrefix(\"foo.bar.baz\", []string{\"*\", \"bar\"}) returns true\n//\tmatchesGlobPrefix(\"foo.bar\", []string{\"baz\", \"*\"}) returns false\nfunc matchesGlobPrefix(key string, glob []string) bool {\n\tk := strings.Split(key, \".\")\n\tfor i, g := range glob {\n\t\tif i >= len(k) {\n\t\t\tbreak\n\t\t}\n\t\tif g == \"*\" {\n\t\t\tcontinue\n\t\t}\n\t\tif !strings.EqualFold(k[i], g) {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\nvar configShowCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Output config file contents.\",\n\t\tShortDescription: `\nNOTE: For security reasons, this command will omit your private key and remote services. If you would like to make a full backup of your config (private key included), you must copy the config file from your repo.\n`,\n\t},\n\tType: make(map[string]any),\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tconfigFileOpt, _ := req.Options[ConfigFileOption].(string)\n\t\tfname, err := config.Filename(cfgRoot, configFileOpt)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdata, err := os.ReadFile(fname)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar cfg map[string]any\n\t\terr = json.Unmarshal(data, &cfg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Check if user wants to expand auto values\n\t\texpandAuto, _ := req.Options[configExpandAutoName].(bool)\n\t\tif expandAuto {\n\t\t\t// Load full config to use resolution methods\n\t\t\tvar fullCfg config.Config\n\t\t\terr = json.Unmarshal(data, &fullCfg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// Expand auto values and update the map\n\t\t\tcfg, err = fullCfg.ExpandAutoConfValues(cfg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tcfg, err = scrubValue(cfg, []string{config.IdentityTag, config.PrivKeyTag})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err = scrubValue(cfg, []string{config.APITag, config.AuthorizationTag})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err = scrubOptionalValue(cfg, config.PinningConcealSelector)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &cfg)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: HumanJSONEncoder,\n\t},\n}\n\nvar HumanJSONEncoder = cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *map[string]any) error {\n\tbuf, err := config.HumanOutput(out)\n\tif err != nil {\n\t\treturn err\n\t}\n\tbuf = append(buf, byte('\\n'))\n\t_, err = w.Write(buf)\n\treturn err\n})\n\n// Scrubs value and returns error if missing\nfunc scrubValue(m map[string]any, key []string) (map[string]any, error) {\n\treturn scrubMapInternal(m, key, false)\n}\n\n// Scrubs value and returns no error if missing\nfunc scrubOptionalValue(m map[string]any, key []string) (map[string]any, error) {\n\treturn scrubMapInternal(m, key, true)\n}\n\nfunc scrubEither(u any, key []string, okIfMissing bool) (any, error) {\n\tm, ok := u.(map[string]any)\n\tif ok {\n\t\treturn scrubMapInternal(m, key, okIfMissing)\n\t}\n\treturn scrubValueInternal(m, key, okIfMissing)\n}\n\nfunc scrubValueInternal(v any, key []string, okIfMissing bool) (any, error) {\n\tif v == nil && !okIfMissing {\n\t\treturn nil, errors.New(\"failed to find specified key\")\n\t}\n\treturn nil, nil\n}\n\nfunc scrubMapInternal(m map[string]any, key []string, okIfMissing bool) (map[string]any, error) {\n\tif len(key) == 0 {\n\t\treturn make(map[string]any), nil // delete value\n\t}\n\tn := map[string]any{}\n\tfor k, v := range m {\n\t\tif key[0] == \"*\" || strings.EqualFold(key[0], k) {\n\t\t\tu, err := scrubEither(v, key[1:], okIfMissing)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tif u != nil {\n\t\t\t\tn[k] = u\n\t\t\t}\n\t\t} else {\n\t\t\tn[k] = v\n\t\t}\n\t}\n\treturn n, nil\n}\n\nvar configEditCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Open the config file for editing in $EDITOR.\",\n\t\tShortDescription: `\nTo use 'ipfs config edit', you must have the $EDITOR environment\nvariable set to your preferred text editor.\n`,\n\t},\n\tNoRemote: true,\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tconfigFileOpt, _ := req.Options[ConfigFileOption].(string)\n\t\tfilename, err := config.Filename(cfgRoot, configFileOpt)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn editConfig(filename)\n\t},\n}\n\nvar configReplaceCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Replace the config with .\",\n\t\tShortDescription: `\nMake sure to back up the config file first if necessary, as this operation\ncan't be undone.\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"file\", true, false, \"The file to use as the new config.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\treturn replaceConfig(r, file)\n\t},\n}\n\nvar configProfileCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Apply profiles to config.\",\n\t\tShortDescription: fmt.Sprintf(`\nAvailable profiles:\n%s\n`, buildProfileHelp()),\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"apply\": configProfileApplyCmd,\n\t},\n}\n\nvar configProfileApplyCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Apply profile to config.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(configDryRunOptionName, \"print difference between the current config and the config that would be generated\"),\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"profile\", true, false, \"The profile to apply to the config.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tprofile, ok := config.Profiles[req.Arguments[0]]\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"%s is not a profile\", req.Arguments[0])\n\t\t}\n\n\t\tdryRun, _ := req.Options[configDryRunOptionName].(bool)\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toldCfg, newCfg, err := transformConfig(cfgRoot, req.Arguments[0], profile.Transform, dryRun)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toldCfgMap, err := scrubPrivKey(oldCfg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnewCfgMap, err := scrubPrivKey(newCfg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &ConfigUpdateOutput{\n\t\t\tOldCfg: oldCfgMap,\n\t\t\tNewCfg: newCfgMap,\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *ConfigUpdateOutput) error {\n\t\t\tdiff := jsondiff.Compare(out.OldCfg, out.NewCfg)\n\t\t\tbuf := jsondiff.Format(diff)\n\n\t\t\t_, err := w.Write(buf)\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: ConfigUpdateOutput{},\n}\n\nfunc buildProfileHelp() string {\n\tvar out string\n\n\tfor _, name := range slices.Sorted(maps.Keys(config.Profiles)) {\n\t\tprofile := config.Profiles[name]\n\t\tdlines := strings.Split(profile.Description, \"\\n\")\n\t\tfor i := range dlines {\n\t\t\tdlines[i] = \" \" + dlines[i]\n\t\t}\n\n\t\tout = out + fmt.Sprintf(\" '%s':\\n%s\\n\", name, strings.Join(dlines, \"\\n\"))\n\t}\n\n\treturn out\n}\n\n// scrubPrivKey scrubs private key for security reasons.\nfunc scrubPrivKey(cfg *config.Config) (map[string]any, error) {\n\tcfgMap, err := config.ToMap(cfg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcfgMap, err = scrubValue(cfgMap, []string{config.IdentityTag, config.PrivKeyTag})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn cfgMap, nil\n}\n\n// transformConfig returns old config and new config instead of difference between them,\n// because apply command can provide stable API through this way.\n// If dryRun is true, repo's config should not be updated and persisted\n// to storage. Otherwise, repo's config should be updated and persisted\n// to storage.\nfunc transformConfig(configRoot string, configName string, transformer config.Transformer, dryRun bool) (*config.Config, *config.Config, error) {\n\tr, err := fsrepo.Open(configRoot)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\tdefer r.Close()\n\n\toldCfg, err := r.Config()\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\t// make a copy to avoid updating repo's config unintentionally\n\tnewCfg, err := oldCfg.Clone()\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\terr = transformer(newCfg)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tif !dryRun {\n\t\t_, err = r.BackupConfig(\"pre-\" + configName + \"-\")\n\t\tif err != nil {\n\t\t\treturn nil, nil, err\n\t\t}\n\n\t\terr = r.SetConfig(newCfg)\n\t\tif err != nil {\n\t\t\treturn nil, nil, err\n\t\t}\n\t}\n\n\treturn oldCfg, newCfg, nil\n}\n\nfunc getConfig(r repo.Repo, key string) (*ConfigField, error) {\n\tvalue, err := r.GetConfigKey(key)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to get config value: %q\", err)\n\t}\n\treturn &ConfigField{\n\t\tKey: key,\n\t\tValue: value,\n\t}, nil\n}\n\nfunc getConfigWithAutoExpand(r repo.Repo, key string) (*ConfigField, error) {\n\t// First get the current value\n\tvalue, err := r.GetConfigKey(key)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to get config value: %q\", err)\n\t}\n\n\t// Load full config for resolution\n\tfullCfg, err := r.Config()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to load config: %q\", err)\n\t}\n\n\t// Expand auto values based on the key\n\texpandedValue := fullCfg.ExpandConfigField(key, value)\n\n\treturn &ConfigField{\n\t\tKey: key,\n\t\tValue: expandedValue,\n\t}, nil\n}\n\nfunc setConfig(r repo.Repo, key string, value any) (*ConfigField, error) {\n\terr := r.SetConfigKey(key, value)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to set config value: %s (maybe use --json?)\", err)\n\t}\n\treturn getConfig(r, key)\n}\n\n// parseEditorCommand parses the EDITOR environment variable into command and arguments\nfunc parseEditorCommand(editor string) ([]string, error) {\n\treturn shlex.Split(editor, true)\n}\n\nfunc editConfig(filename string) error {\n\teditor := os.Getenv(\"EDITOR\")\n\tif editor == \"\" {\n\t\treturn errors.New(\"ENV variable $EDITOR not set\")\n\t}\n\n\teditorAndArgs, err := parseEditorCommand(editor)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot parse $EDITOR value: %s\", err)\n\t}\n\teditor = editorAndArgs[0]\n\targs := append(editorAndArgs[1:], filename)\n\n\tcmd := exec.Command(editor, args...)\n\tcmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr\n\treturn cmd.Run()\n}\n\nfunc replaceConfig(r repo.Repo, file io.Reader) error {\n\tvar newCfg config.Config\n\tif err := json.NewDecoder(file).Decode(&newCfg); err != nil {\n\t\treturn errors.New(\"failed to decode file as config\")\n\t}\n\n\t// Handle Identity.PrivKey (secret)\n\n\tif len(newCfg.Identity.PrivKey) != 0 {\n\t\treturn errors.New(\"setting private key with API is not supported\")\n\t}\n\n\tkeyF, err := getConfig(r, config.PrivKeySelector)\n\tif err != nil {\n\t\treturn errors.New(\"failed to get PrivKey\")\n\t}\n\n\tpkstr, ok := keyF.Value.(string)\n\tif !ok {\n\t\treturn errors.New(\"private key in config was not a string\")\n\t}\n\n\tnewCfg.Identity.PrivKey = pkstr\n\n\t// Handle Pinning.RemoteServices (API.Key of each service is a secret)\n\n\tnewServices := newCfg.Pinning.RemoteServices\n\toldServices, err := getRemotePinningServices(r)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to load remote pinning services info (%v)\", err)\n\t}\n\n\t// fail fast if service lists are obviously different\n\tif len(newServices) != len(oldServices) {\n\t\treturn errors.New(\"cannot add or remove remote pinning services with 'config replace'\")\n\t}\n\n\t// re-apply API details and confirm every modified service already existed\n\tfor name, oldSvc := range oldServices {\n\t\tif newSvc, hadSvc := newServices[name]; hadSvc {\n\t\t\t// fail if input changes any of API details\n\t\t\t// (interop with config show: allow Endpoint as long it did not change)\n\t\t\tif len(newSvc.API.Key) != 0 || (len(newSvc.API.Endpoint) != 0 && newSvc.API.Endpoint != oldSvc.API.Endpoint) {\n\t\t\t\treturn errors.New(\"cannot change remote pinning services api info with `config replace`\")\n\t\t\t}\n\t\t\t// re-apply API details and store service in updated config\n\t\t\tnewSvc.API = oldSvc.API\n\t\t\tnewCfg.Pinning.RemoteServices[name] = newSvc\n\t\t} else {\n\t\t\t// error on service rm attempt\n\t\t\treturn errors.New(\"cannot add or remove remote pinning services with 'config replace'\")\n\t\t}\n\t}\n\n\treturn r.SetConfig(&newCfg)\n}\n\nfunc getRemotePinningServices(r repo.Repo) (map[string]config.RemotePinningService, error) {\n\tvar oldServices map[string]config.RemotePinningService\n\tif remoteServicesTag, err := getConfig(r, config.RemoteServicesPath); err == nil {\n\t\t// seems that golang cannot type assert map[string]interface{} to map[string]config.RemotePinningService\n\t\t// so we have to manually copy the data :-|\n\t\tif val, ok := remoteServicesTag.Value.(map[string]any); ok {\n\t\t\tjsonString, err := json.Marshal(val)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\terr = json.Unmarshal(jsonString, &oldServices)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\t}\n\treturn oldServices, nil\n}\n" }, { "path": "core/commands/config_test.go", "content": "package commands\n\nimport \"testing\"\n\nfunc TestScrubMapInternalDelete(t *testing.T) {\n\tm, err := scrubMapInternal(nil, nil, true)\n\tif err != nil {\n\t\tt.Error(err)\n\t}\n\tif m == nil {\n\t\tt.Errorf(\"expecting an empty map, got nil\")\n\t}\n\tif len(m) != 0 {\n\t\tt.Errorf(\"expecting an empty map, got a non-empty map\")\n\t}\n}\n\nfunc TestEditorParsing(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinput string\n\t\texpected []string\n\t\thasError bool\n\t}{\n\t\t{\n\t\t\tname: \"simple editor\",\n\t\t\tinput: \"vim\",\n\t\t\texpected: []string{\"vim\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"editor with single flag\",\n\t\t\tinput: \"emacs -nw\",\n\t\t\texpected: []string{\"emacs\", \"-nw\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VS Code with wait flag (issue #9375)\",\n\t\t\tinput: \"code --wait\",\n\t\t\texpected: []string{\"code\", \"--wait\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VS Code with full path and wait flag (issue #9375)\",\n\t\t\tinput: \"/opt/homebrew/bin/code --wait\",\n\t\t\texpected: []string{\"/opt/homebrew/bin/code\", \"--wait\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"editor with quoted path containing spaces\",\n\t\t\tinput: \"\\\"/Applications/Visual Studio Code.app/Contents/Resources/app/bin/code\\\" --wait\",\n\t\t\texpected: []string{\"/Applications/Visual Studio Code.app/Contents/Resources/app/bin/code\", \"--wait\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"sublime text with wait flag\",\n\t\t\tinput: \"subl -w\",\n\t\t\texpected: []string{\"subl\", \"-w\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"nano editor\",\n\t\t\tinput: \"nano\",\n\t\t\texpected: []string{\"nano\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"gedit editor\",\n\t\t\tinput: \"gedit\",\n\t\t\texpected: []string{\"gedit\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"editor with multiple flags\",\n\t\t\tinput: \"vim -c 'set number' -c 'set hlsearch'\",\n\t\t\texpected: []string{\"vim\", \"-c\", \"set number\", \"-c\", \"set hlsearch\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"trailing backslash (POSIX edge case)\",\n\t\t\tinput: \"editor\\\\\",\n\t\t\texpected: nil,\n\t\t\thasError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"double quoted editor name with spaces\",\n\t\t\tinput: \"\\\"code with spaces\\\" --wait\",\n\t\t\texpected: []string{\"code with spaces\", \"--wait\"},\n\t\t\thasError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"single quoted editor with flags\",\n\t\t\tinput: \"'my editor' -flag\",\n\t\t\texpected: []string{\"my editor\", \"-flag\"},\n\t\t\thasError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult, err := parseEditorCommand(tc.input)\n\n\t\t\tif tc.hasError {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"Expected error for input '%s', but got none\", tc.input)\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"Unexpected error for input '%s': %v\", tc.input, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif len(result) != len(tc.expected) {\n\t\t\t\tt.Errorf(\"Expected %d args, got %d for input '%s'\", len(tc.expected), len(result), tc.input)\n\t\t\t\tt.Errorf(\"Expected: %v\", tc.expected)\n\t\t\t\tt.Errorf(\"Got: %v\", result)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tfor i, expected := range tc.expected {\n\t\t\t\tif result[i] != expected {\n\t\t\t\t\tt.Errorf(\"Expected arg %d to be '%s', got '%s' for input '%s'\", i, expected, result[i], tc.input)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "core/commands/dag/dag.go", "content": "package dagcmd\n\nimport (\n\t\"encoding/csv\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"path\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tcid \"github.com/ipfs/go-cid\"\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nconst (\n\tpinRootsOptionName = \"pin-roots\"\n\tprogressOptionName = \"progress\"\n\tsilentOptionName = \"silent\"\n\tstatsOptionName = \"stats\"\n\tfastProvideRootOptionName = \"fast-provide-root\"\n\tfastProvideWaitOptionName = \"fast-provide-wait\"\n)\n\n// DagCmd provides a subset of commands for interacting with ipld dag objects\nvar DagCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with IPLD DAG objects.\",\n\t\tShortDescription: `\n'ipfs dag' is used for creating and manipulating DAG objects/hierarchies.\n\nThis subcommand is intended to deprecate and replace\nthe existing 'ipfs object' command moving forward.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"put\": DagPutCmd,\n\t\t\"get\": DagGetCmd,\n\t\t\"resolve\": DagResolveCmd,\n\t\t\"import\": DagImportCmd,\n\t\t\"export\": DagExportCmd,\n\t\t\"stat\": DagStatCmd,\n\t},\n}\n\n// OutputObject is the output type of 'dag put' command\ntype OutputObject struct {\n\tCid cid.Cid\n}\n\n// ResolveOutput is the output type of 'dag resolve' command\ntype ResolveOutput struct {\n\tCid cid.Cid\n\tRemPath string\n}\n\ntype CarImportStats struct {\n\tBlockCount uint64\n\tBlockBytesCount uint64\n}\n\n// CarImportOutput is the output type of the 'dag import' commands\ntype CarImportOutput struct {\n\tRoot *RootMeta `json:\",omitempty\"`\n\tStats *CarImportStats `json:\",omitempty\"`\n}\n\n// RootMeta is the metadata for a root pinning response\ntype RootMeta struct {\n\tCid cid.Cid\n\tPinErrorMsg string\n}\n\n// DagPutCmd is a command for adding a dag node\nvar DagPutCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add a DAG node to IPFS.\",\n\t\tShortDescription: `\n'ipfs dag put' accepts input from a file or stdin and parses it\ninto an object of the specified format.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"object data\", true, true, \"The object to put\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(\"store-codec\", \"Codec that the stored object will be encoded with\").WithDefault(\"dag-cbor\"),\n\t\tcmds.StringOption(\"input-codec\", \"Codec that the input object is encoded in\").WithDefault(\"dag-json\"),\n\t\tcmds.BoolOption(\"pin\", \"Pin this object when adding.\"),\n\t\tcmds.StringOption(\"hash\", \"Hash function to use\"),\n\t\tcmdutils.AllowBigBlockOption,\n\t},\n\tRun: dagPut,\n\tType: OutputObject{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *OutputObject) error {\n\t\t\tenc, err := cmdenv.GetLowLevelCidEncoder(req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tfmt.Fprintln(w, enc.Encode(out.Cid))\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// DagGetCmd is a command for getting a dag node from IPFS\nvar DagGetCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Get a DAG node from IPFS.\",\n\t\tShortDescription: `\n'ipfs dag get' fetches a DAG node from IPFS and prints it out in the specified\nformat.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ref\", true, false, \"The object to get\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(\"output-codec\", \"Format that the object will be encoded as.\").WithDefault(\"dag-json\"),\n\t},\n\tRun: dagGet,\n}\n\n// DagResolveCmd returns address of highest block within a path and a path remainder\nvar DagResolveCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Resolve IPLD block.\",\n\t\tShortDescription: `\n'ipfs dag resolve' fetches a DAG node from IPFS, prints its address and remaining path.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ref\", true, false, \"The path to resolve\").EnableStdin(),\n\t},\n\tRun: dagResolve,\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *ResolveOutput) error {\n\t\t\tvar (\n\t\t\t\tenc cidenc.Encoder\n\t\t\t\terr error\n\t\t\t)\n\t\t\tswitch {\n\t\t\tcase !cmdenv.CidBaseDefined(req):\n\t\t\t\t// Not specified, check the path.\n\t\t\t\tenc, err = cmdenv.CidEncoderFromPath(req.Arguments[0])\n\t\t\t\tif err == nil {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\t// Nope, fallback on the default.\n\t\t\t\tfallthrough\n\t\t\tdefault:\n\t\t\t\tenc, err = cmdenv.GetLowLevelCidEncoder(req)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\tp := enc.Encode(out.Cid)\n\t\t\tif out.RemPath != \"\" {\n\t\t\t\tp = path.Join(p, out.RemPath)\n\t\t\t}\n\n\t\t\tfmt.Fprint(w, p)\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: ResolveOutput{},\n}\n\n// DagImportCmd is a command for importing a car to ipfs\nvar DagImportCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Import the contents of .car files\",\n\t\tShortDescription: `\n'ipfs dag import' imports all blocks present in supplied .car\n( Content Address aRchive ) files, recursively pinning any roots\nspecified in the CAR file headers, unless --pin-roots is set to false.\n\nNote:\n This command will import all blocks in the CAR file, not just those\n reachable from the specified roots. However, these other blocks will\n not be pinned and may be garbage collected later.\n\n The pinning of the roots happens after all car files are processed,\n permitting import of DAGs spanning multiple files.\n\n Pinning takes place in offline-mode exclusively, one root at a time.\n If the combination of blocks from the imported CAR files and what is\n currently present in the blockstore does not represent a complete DAG,\n pinning of that individual root will fail.\n\nFAST PROVIDE OPTIMIZATION:\n\nRoot CIDs from CAR headers are immediately provided to the DHT in addition\nto the regular provide queue, allowing other peers to discover your content\nright away. This complements the sweep provider, which efficiently provides\nall blocks according to Provide.Strategy over time.\n\nBy default, the provide happens in the background without blocking the\ncommand. Use --fast-provide-wait to wait for the provide to complete, or\n--fast-provide-root=false to skip it. Works even with --pin-roots=false.\nAutomatically skipped when DHT is not available.\n\nMaximum supported CAR version: 2\nSpecification of CAR formats: https://ipld.io/specs/transport/car/\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"path\", true, true, \"The path of a .car file.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(pinRootsOptionName, \"Pin optional roots listed in the .car headers after importing.\").WithDefault(true),\n\t\tcmds.BoolOption(silentOptionName, \"No output.\"),\n\t\tcmds.BoolOption(statsOptionName, \"Output stats.\"),\n\t\tcmds.BoolOption(fastProvideRootOptionName, \"Immediately provide root CIDs to DHT in addition to regular queue, for faster discovery. Default: Import.FastProvideRoot\"),\n\t\tcmds.BoolOption(fastProvideWaitOptionName, \"Block until the immediate provide completes before returning. Default: Import.FastProvideWait\"),\n\t\tcmdutils.AllowBigBlockOption,\n\t},\n\tType: CarImportOutput{},\n\tRun: dagImport,\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, event *CarImportOutput) error {\n\t\t\tsilent, _ := req.Options[silentOptionName].(bool)\n\t\t\tif silent {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\t// event should have only one of `Root` or `Stats` set, not both\n\t\t\tif event.Root == nil {\n\t\t\t\tif event.Stats == nil {\n\t\t\t\t\treturn fmt.Errorf(\"unexpected message from DAG import\")\n\t\t\t\t}\n\t\t\t\tstats, _ := req.Options[statsOptionName].(bool)\n\t\t\t\tif stats {\n\t\t\t\t\tfmt.Fprintf(w, \"Imported %d blocks (%d bytes)\\n\", event.Stats.BlockCount, event.Stats.BlockBytesCount)\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif event.Stats != nil {\n\t\t\t\treturn fmt.Errorf(\"unexpected message from DAG import\")\n\t\t\t}\n\n\t\t\tenc, err := cmdenv.GetLowLevelCidEncoder(req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif event.Root.PinErrorMsg != \"\" {\n\t\t\t\treturn fmt.Errorf(\"pinning root %q FAILED: %s\", enc.Encode(event.Root.Cid), event.Root.PinErrorMsg)\n\t\t\t}\n\n\t\t\tevent.Root.PinErrorMsg = \"success\"\n\n\t\t\t_, err = fmt.Fprintf(\n\t\t\t\tw,\n\t\t\t\t\"Pinned root\\t%s\\t%s\\n\",\n\t\t\t\tenc.Encode(event.Root.Cid),\n\t\t\t\tevent.Root.PinErrorMsg,\n\t\t\t)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\n// DagExportCmd is a command for exporting an ipfs dag to a car\nvar DagExportCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Streams the selected DAG as a .car stream on stdout.\",\n\t\tShortDescription: `\n'ipfs dag export' fetches a DAG and streams it out as a well-formed .car file.\nNote that at present only single root selections / .car files are supported.\nThe output of blocks happens in strict DAG-traversal, first-seen, order.\nCAR file follows the CARv1 format: https://ipld.io/specs/transport/car/carv1/\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tResponseContentType: \"application/vnd.ipld.car\",\n\t\t},\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"root\", true, false, \"CID of a root to recursively export\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(progressOptionName, \"p\", \"Display progress on CLI. Defaults to true when STDERR is a TTY.\"),\n\t},\n\tRun: dagExport,\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: finishCLIExport,\n\t},\n}\n\n// DagStat is a dag stat command response\ntype DagStat struct {\n\tCid cid.Cid\n\tSize uint64 `json:\",omitempty\"`\n\tNumBlocks int64 `json:\",omitempty\"`\n}\n\nfunc (s *DagStat) String() string {\n\treturn fmt.Sprintf(\"%s %d %d\", s.Cid.String()[:20], s.Size, s.NumBlocks)\n}\n\nfunc (s *DagStat) MarshalJSON() ([]byte, error) {\n\ttype Alias DagStat\n\t/*\n\t\tWe can't rely on cid.Cid.MarshalJSON since it uses the {\"/\": \"...\"}\n\t\tformat. To make the output consistent and follow the Kubo API patterns\n\t\twe use the Cid.String method\n\t*/\n\treturn json.Marshal(struct {\n\t\tCid string `json:\"Cid\"`\n\t\t*Alias\n\t}{\n\t\tCid: s.Cid.String(),\n\t\tAlias: (*Alias)(s),\n\t})\n}\n\nfunc (s *DagStat) UnmarshalJSON(data []byte) error {\n\t/*\n\t\tWe can't rely on cid.Cid.UnmarshalJSON since it uses the {\"/\": \"...\"}\n\t\tformat. To make the output consistent and follow the Kubo API patterns\n\t\twe use the Cid.Parse method\n\t*/\n\ttype Alias DagStat\n\taux := struct {\n\t\tCid string `json:\"Cid\"`\n\t\t*Alias\n\t}{\n\t\tAlias: (*Alias)(s),\n\t}\n\tif err := json.Unmarshal(data, &aux); err != nil {\n\t\treturn err\n\t}\n\tCid, err := cid.Parse(aux.Cid)\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.Cid = Cid\n\treturn nil\n}\n\ntype DagStatSummary struct {\n\tredundantSize uint64 `json:\"-\"`\n\tUniqueBlocks int `json:\",omitempty\"`\n\tTotalSize uint64 `json:\",omitempty\"`\n\tSharedSize uint64 `json:\",omitempty\"`\n\tRatio float32 `json:\",omitempty\"`\n\tDagStatsArray []*DagStat `json:\"DagStats,omitempty\"`\n}\n\nfunc (s *DagStatSummary) String() string {\n\treturn fmt.Sprintf(\"Total Size: %d (%s)\\nUnique Blocks: %d\\nShared Size: %d (%s)\\nRatio: %f\",\n\t\ts.TotalSize, humanize.Bytes(s.TotalSize),\n\t\ts.UniqueBlocks,\n\t\ts.SharedSize, humanize.Bytes(s.SharedSize),\n\t\ts.Ratio)\n}\n\nfunc (s *DagStatSummary) incrementTotalSize(size uint64) {\n\ts.TotalSize += size\n}\n\nfunc (s *DagStatSummary) incrementRedundantSize(size uint64) {\n\ts.redundantSize += size\n}\n\nfunc (s *DagStatSummary) appendStats(stats *DagStat) {\n\ts.DagStatsArray = append(s.DagStatsArray, stats)\n}\n\nfunc (s *DagStatSummary) calculateSummary() {\n\ts.Ratio = float32(s.redundantSize) / float32(s.TotalSize)\n\ts.SharedSize = s.redundantSize - s.TotalSize\n}\n\n// DagStatCmd is a command for getting size information about an ipfs-stored dag\nvar DagStatCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Gets stats for a DAG.\",\n\t\tShortDescription: `\n'ipfs dag stat' fetches a DAG and returns various statistics about it.\nStatistics include size and number of blocks.\n\nNote: This command skips duplicate blocks in reporting both size and the number of blocks\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"root\", true, true, \"CID of a DAG root to get statistics for\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(progressOptionName, \"p\", \"Show progress on stderr. Auto-detected if stderr is a terminal.\"),\n\t},\n\tRun: dagStat,\n\tType: DagStatSummary{},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: finishCLIStat,\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, event *DagStatSummary) error {\n\t\t\tfmt.Fprintln(w)\n\t\t\tcsvWriter := csv.NewWriter(w)\n\t\t\tcsvWriter.Comma = '\\t'\n\t\t\tcidSpacing := len(event.DagStatsArray[0].Cid.String())\n\t\t\theader := []string{fmt.Sprintf(\"%-*s\", cidSpacing, \"CID\"), fmt.Sprintf(\"%-15s\", \"Blocks\"), \"Size\"}\n\t\t\tif err := csvWriter.Write(header); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tfor _, dagStat := range event.DagStatsArray {\n\t\t\t\tnumBlocksStr := fmt.Sprint(dagStat.NumBlocks)\n\t\t\t\terr := csvWriter.Write([]string{\n\t\t\t\t\tdagStat.Cid.String(),\n\t\t\t\t\tfmt.Sprintf(\"%-15s\", numBlocksStr),\n\t\t\t\t\tfmt.Sprint(dagStat.Size),\n\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\tcsvWriter.Flush()\n\t\t\tfmt.Fprint(w, \"\\nSummary\\n\")\n\t\t\t_, err := fmt.Fprintf(\n\t\t\t\tw,\n\t\t\t\t\"%v\\n\",\n\t\t\t\tevent,\n\t\t\t)\n\t\t\tfmt.Fprint(w, \"\\n\\n\")\n\t\t\treturn err\n\t\t}),\n\t\tcmds.JSON: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, event *DagStatSummary) error {\n\t\t\treturn json.NewEncoder(w).Encode(event)\n\t\t},\n\t\t),\n\t},\n}\n" }, { "path": "core/commands/dag/export.go", "content": "package dagcmd\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/cheggaaa/pb\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tgocar \"github.com/ipld/go-car/v2\"\n\tcidlink \"github.com/ipld/go-ipld-prime/linking/cid\"\n\tselectorparse \"github.com/ipld/go-ipld-prime/traversal/selector/parse\"\n)\n\nfunc dagExport(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t// Accept CID or a content path\n\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tapi, err := cmdenv.GetApi(env, req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Resolve path and confirm the root block is available, fail fast if not\n\tb, err := api.Block().Stat(req.Context, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\tc := b.Path().RootCid()\n\n\tpipeR, pipeW := io.Pipe()\n\n\terrCh := make(chan error, 2) // we only report the 1st error\n\tgo func() {\n\t\tdefer func() {\n\t\t\tif err := pipeW.Close(); err != nil {\n\t\t\t\terrCh <- fmt.Errorf(\"stream flush failed: %s\", err)\n\t\t\t}\n\t\t\tclose(errCh)\n\t\t}()\n\n\t\tlsys := cidlink.DefaultLinkSystem()\n\t\tlsys.SetReadStorage(&dagStore{dag: api.Dag(), ctx: req.Context})\n\n\t\t// Uncomment the following to support CARv2 output.\n\t\t/*\n\t\t\tcar, err := gocar.NewSelectiveWriter(req.Context, &lsys, c, selectorparse.CommonSelector_ExploreAllRecursively, gocar.AllowDuplicatePuts(false))\n\t\t\tif err != nil {\n\t\t\t\terrCh <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif _, err = car.WriteTo(pipeW); err != nil {\n\t\t\t\terrCh <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t*/\n\t\t_, err := gocar.TraverseV1(req.Context, &lsys, c, selectorparse.CommonSelector_ExploreAllRecursively, pipeW, gocar.AllowDuplicatePuts(false))\n\t\tif err != nil {\n\t\t\terrCh <- err\n\t\t\treturn\n\t\t}\n\n\t}()\n\n\tres.SetEncodingType(cmds.OctetStream)\n\tres.SetContentType(\"application/vnd.ipld.car\")\n\tif err := res.Emit(pipeR); err != nil {\n\t\tpipeR.Close() // ignore the error if any\n\t\treturn err\n\t}\n\n\terr = <-errCh\n\n\t// minimal user friendliness\n\tif errors.Is(err, ipld.ErrNotFound{}) {\n\t\texplicitOffline, _ := req.Options[\"offline\"].(bool)\n\t\tif explicitOffline {\n\t\t\terr = fmt.Errorf(\"%s (currently offline, perhaps retry without the offline flag)\", err)\n\t\t} else {\n\t\t\tnode, envErr := cmdenv.GetNode(env)\n\t\t\tif envErr == nil && !node.IsOnline {\n\t\t\t\terr = fmt.Errorf(\"%s (currently offline, perhaps retry after attaching to the network)\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn err\n}\n\nfunc finishCLIExport(res cmds.Response, re cmds.ResponseEmitter) error {\n\tvar showProgress bool\n\tval, specified := res.Request().Options[progressOptionName]\n\tif !specified {\n\t\t// default based on TTY availability\n\t\terrStat, _ := os.Stderr.Stat()\n\t\tif (errStat.Mode() & os.ModeCharDevice) != 0 {\n\t\t\tshowProgress = true\n\t\t}\n\t} else if val.(bool) {\n\t\tshowProgress = true\n\t}\n\n\t// simple passthrough, no progress\n\tif !showProgress {\n\t\treturn cmds.Copy(re, res)\n\t}\n\n\tbar := pb.New64(0).SetUnits(pb.U_BYTES)\n\tbar.Output = os.Stderr\n\tbar.ShowSpeed = true\n\tbar.ShowElapsedTime = true\n\tbar.RefreshRate = 500 * time.Millisecond\n\tbar.Start()\n\n\tvar processedOneResponse bool\n\tfor {\n\t\tv, err := res.Next()\n\t\tif err != nil {\n\t\t\tif errors.Is(err, io.EOF) {\n\t\t\t\t// We only write the final bar update on success\n\t\t\t\t// On error it looks too weird\n\t\t\t\tbar.Finish()\n\t\t\t\treturn re.Close()\n\t\t\t}\n\t\t\treturn re.CloseWithError(err)\n\t\t}\n\n\t\tif processedOneResponse {\n\t\t\treturn re.CloseWithError(errors.New(\"unexpected multipart response during emit, please file a bugreport\"))\n\t\t}\n\n\t\tr, ok := v.(io.Reader)\n\t\tif !ok {\n\t\t\t// some sort of encoded response, this should not be happening\n\t\t\treturn errors.New(\"unexpected non-stream passed to PostRun: please file a bugreport\")\n\t\t}\n\n\t\tprocessedOneResponse = true\n\n\t\tif err = re.Emit(bar.NewProxyReader(r)); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n}\n\ntype dagStore struct {\n\tdag iface.APIDagService\n\tctx context.Context\n}\n\nfunc (ds *dagStore) Get(ctx context.Context, key string) ([]byte, error) {\n\tif ctx.Err() != nil {\n\t\treturn nil, ctx.Err()\n\t}\n\n\tc, err := cidFromBinString(key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tblock, err := ds.dag.Get(ds.ctx, c)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn block.RawData(), nil\n}\n\nfunc (ds *dagStore) Has(ctx context.Context, key string) (bool, error) {\n\t_, err := ds.Get(ctx, key)\n\tif err != nil {\n\t\tif errors.Is(err, ipld.ErrNotFound{}) {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, err\n\t}\n\treturn true, nil\n}\n\nfunc cidFromBinString(key string) (cid.Cid, error) {\n\tl, k, err := cid.CidFromBytes([]byte(key))\n\tif err != nil {\n\t\treturn cid.Undef, fmt.Errorf(\"dagStore: key was not a cid: %w\", err)\n\t}\n\tif l != len(key) {\n\t\treturn cid.Undef, fmt.Errorf(\"dagSore: key was not a cid: had %d bytes leftover\", len(key)-l)\n\t}\n\treturn k, nil\n}\n" }, { "path": "core/commands/dag/get.go", "content": "package dagcmd\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tipldlegacy \"github.com/ipfs/go-ipld-legacy\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\t\"github.com/ipld/go-ipld-prime\"\n\t\"github.com/ipld/go-ipld-prime/multicodec\"\n\t\"github.com/ipld/go-ipld-prime/traversal\"\n\tmc \"github.com/multiformats/go-multicodec\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc dagGet(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\tapi, err := cmdenv.GetApi(env, req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcodecStr, _ := req.Options[\"output-codec\"].(string)\n\tvar codec mc.Code\n\tif err := codec.Set(codecStr); err != nil {\n\t\treturn err\n\t}\n\n\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\tif err != nil {\n\t\treturn err\n\t}\n\n\trp, remainder, err := api.ResolvePath(req.Context, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tobj, err := api.Dag().Get(req.Context, rp.RootCid())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tuniversal, ok := obj.(ipldlegacy.UniversalNode)\n\tif !ok {\n\t\treturn fmt.Errorf(\"%T is not a valid IPLD node\", obj)\n\t}\n\n\tfinalNode := universal.(ipld.Node)\n\n\tif len(remainder) > 0 {\n\t\tremainderPath := ipld.ParsePath(path.SegmentsToString(remainder...))\n\n\t\tfinalNode, err = traversal.Get(finalNode, remainderPath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tencoder, err := multicodec.LookupEncoder(uint64(codec))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid encoding: %s - %s\", codec, err)\n\t}\n\n\tr, w := io.Pipe()\n\tgo func() {\n\t\tdefer w.Close()\n\t\tif err := encoder(finalNode, w); err != nil {\n\t\t\t_ = w.CloseWithError(err)\n\t\t}\n\t}()\n\n\treturn res.Emit(r)\n}\n" }, { "path": "core/commands/dag/import.go", "content": "package dagcmd\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tblocks \"github.com/ipfs/go-block-format\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tipldlegacy \"github.com/ipfs/go-ipld-legacy\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\tgocarv2 \"github.com/ipld/go-car/v2\"\n\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n)\n\nvar log = logging.Logger(\"core/commands\")\n\nfunc dagImport(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\tnode, err := cmdenv.GetNode(env)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcfg, err := node.Repo.Config()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tapi, err := cmdenv.GetApi(env, req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tblockDecoder := ipldlegacy.NewDecoder()\n\n\t// on import ensure we do not reach out to the network for any reason\n\t// if a pin based on what is imported + what is in the blockstore\n\t// isn't possible: tough luck\n\tapi, err = api.WithOptions(options.Api.Offline(true))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdoPinRoots, _ := req.Options[pinRootsOptionName].(bool)\n\n\tfastProvideRoot, fastProvideRootSet := req.Options[fastProvideRootOptionName].(bool)\n\tfastProvideWait, fastProvideWaitSet := req.Options[fastProvideWaitOptionName].(bool)\n\n\tfastProvideRoot = config.ResolveBoolFromConfig(fastProvideRoot, fastProvideRootSet, cfg.Import.FastProvideRoot, config.DefaultFastProvideRoot)\n\tfastProvideWait = config.ResolveBoolFromConfig(fastProvideWait, fastProvideWaitSet, cfg.Import.FastProvideWait, config.DefaultFastProvideWait)\n\n\t// grab a pinlock ( which doubles as a GC lock ) so that regardless of the\n\t// size of the streamed-in cars nothing will disappear on us before we had\n\t// a chance to roots that may show up at the very end\n\t// This is especially important for use cases like dagger:\n\t// ipfs dag import $( ... | ipfs-dagger --stdout=carfifos )\n\t//\n\tif doPinRoots {\n\t\tunlocker := node.Blockstore.PinLock(req.Context)\n\t\tdefer unlocker.Unlock(req.Context)\n\t}\n\n\t// this is *not* a transaction\n\t// it is simply a way to relieve pressure on the blockstore\n\t// similar to pinner.Pin/pinner.Flush\n\tbatch := ipld.NewBatch(req.Context, api.Dag(),\n\t\t// Default: 128. Means 128 file descriptors needed in flatfs\n\t\tipld.MaxNodesBatchOption(int(cfg.Import.BatchMaxNodes.WithDefault(config.DefaultBatchMaxNodes))),\n\t\t// Default 100MiB. When setting block size to 1MiB, we can add\n\t\t// ~100 nodes maximum. With default 256KiB block-size, we will\n\t\t// hit the max nodes limit at 32MiB.p\n\t\tipld.MaxSizeBatchOption(int(cfg.Import.BatchMaxSize.WithDefault(config.DefaultBatchMaxSize))),\n\t)\n\n\troots := cid.NewSet()\n\tvar blockCount, blockBytesCount uint64\n\n\t// remember last valid block and provide a meaningful error message\n\t// when a truncated/mangled CAR is being imported\n\timportError := func(previous blocks.Block, current blocks.Block, err error) error {\n\t\tif current != nil {\n\t\t\treturn fmt.Errorf(\"import failed at block %q: %w\", current.Cid(), err)\n\t\t}\n\t\tif previous != nil {\n\t\t\treturn fmt.Errorf(\"import failed after block %q: %w\", previous.Cid(), err)\n\t\t}\n\t\treturn fmt.Errorf(\"import failed: %w\", err)\n\t}\n\n\tit := req.Files.Entries()\n\tfor it.Next() {\n\t\tfile := files.FileFromEntry(it)\n\t\tif file == nil {\n\t\t\treturn errors.New(\"expected a file handle\")\n\t\t}\n\n\t\t// import blocks\n\t\terr = func() error {\n\t\t\t// wrap a defer-closer-scope\n\t\t\t//\n\t\t\t// every single file in it() is already open before we start\n\t\t\t// just close here sooner rather than later for neatness\n\t\t\t// and to surface potential errors writing on closed fifos\n\t\t\t// this won't/can't help with not running out of handles\n\t\t\tdefer file.Close()\n\n\t\t\tvar previous blocks.Block\n\n\t\t\tcar, err := gocarv2.NewBlockReader(file)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tfor _, c := range car.Roots {\n\t\t\t\troots.Add(c)\n\t\t\t}\n\n\t\t\tfor {\n\t\t\t\tblock, err := car.Next()\n\t\t\t\tif err != nil && err != io.EOF {\n\t\t\t\t\treturn importError(previous, block, err)\n\t\t\t\t} else if block == nil {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tif err := cmdutils.CheckBlockSize(req, uint64(len(block.RawData()))); err != nil {\n\t\t\t\t\treturn importError(previous, block, err)\n\t\t\t\t}\n\n\t\t\t\t// the double-decode is suboptimal, but we need it for batching\n\t\t\t\tnd, err := blockDecoder.DecodeNode(req.Context, block)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn importError(previous, block, err)\n\t\t\t\t}\n\n\t\t\t\tif err := batch.Add(req.Context, nd); err != nil {\n\t\t\t\t\treturn importError(previous, block, err)\n\t\t\t\t}\n\t\t\t\tblockCount++\n\t\t\t\tblockBytesCount += uint64(len(block.RawData()))\n\t\t\t\tprevious = block\n\t\t\t}\n\t\t\treturn nil\n\t\t}()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif err := batch.Commit(); err != nil {\n\t\treturn err\n\t}\n\n\t// It is not guaranteed that a root in a header is actually present in the same ( or any )\n\t// .car file. This is the case in version 1, and ideally in further versions too.\n\t// Accumulate any root CID seen in a header, and supplement its actual node if/when encountered\n\t// We will attempt a pin *only* at the end in case all car files were well-formed.\n\n\t// opportunistic pinning: try whatever sticks\n\tif doPinRoots {\n\t\terr = roots.ForEach(func(c cid.Cid) error {\n\t\t\tret := RootMeta{Cid: c}\n\n\t\t\t// This will trigger a full read of the DAG in the pinner, to make sure we have all blocks.\n\t\t\t// Ideally we would do colloring of the pinning state while importing the blocks\n\t\t\t// and ensure the gray bucket is empty at the end (or use the network to download missing blocks).\n\t\t\tif block, err := node.Blockstore.Get(req.Context, c); err != nil {\n\t\t\t\tret.PinErrorMsg = err.Error()\n\t\t\t} else if nd, err := blockDecoder.DecodeNode(req.Context, block); err != nil {\n\t\t\t\tret.PinErrorMsg = err.Error()\n\t\t\t} else if err := node.Pinning.Pin(req.Context, nd, true, \"\"); err != nil {\n\t\t\t\tret.PinErrorMsg = err.Error()\n\t\t\t} else if err := node.Pinning.Flush(req.Context); err != nil {\n\t\t\t\tret.PinErrorMsg = err.Error()\n\t\t\t}\n\n\t\t\treturn res.Emit(&CarImportOutput{Root: &ret})\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tstats, _ := req.Options[statsOptionName].(bool)\n\tif stats {\n\t\terr = res.Emit(&CarImportOutput{\n\t\t\tStats: &CarImportStats{\n\t\t\t\tBlockCount: blockCount,\n\t\t\t\tBlockBytesCount: blockBytesCount,\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Fast-provide roots for faster discovery\n\tif fastProvideRoot {\n\t\terr = roots.ForEach(func(c cid.Cid) error {\n\t\t\treturn cmdenv.ExecuteFastProvide(req.Context, node, cfg, c, fastProvideWait, doPinRoots, doPinRoots, false)\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t} else {\n\t\tif fastProvideWait {\n\t\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"disabled by flag or config\", \"wait-flag-ignored\", true)\n\t\t} else {\n\t\t\tlog.Debugw(\"fast-provide-root: skipped\", \"reason\", \"disabled by flag or config\")\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "core/commands/dag/put.go", "content": "package dagcmd\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/go-cid\"\n\tipldlegacy \"github.com/ipfs/go-ipld-legacy\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\t\"github.com/ipld/go-ipld-prime/multicodec\"\n\tbasicnode \"github.com/ipld/go-ipld-prime/node/basic\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tmc \"github.com/multiformats/go-multicodec\"\n\n\t// Expected minimal set of available format/ienc codecs.\n\t_ \"github.com/ipld/go-codec-dagpb\"\n\t_ \"github.com/ipld/go-ipld-prime/codec/cbor\"\n\t_ \"github.com/ipld/go-ipld-prime/codec/dagcbor\"\n\t_ \"github.com/ipld/go-ipld-prime/codec/dagjson\"\n\t_ \"github.com/ipld/go-ipld-prime/codec/json\"\n\t_ \"github.com/ipld/go-ipld-prime/codec/raw\"\n)\n\nfunc dagPut(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\tapi, err := cmdenv.GetApi(env, req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tnd, err := cmdenv.GetNode(env)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcfg, err := nd.Repo.Config()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tinputCodec, _ := req.Options[\"input-codec\"].(string)\n\tstoreCodec, _ := req.Options[\"store-codec\"].(string)\n\thash, _ := req.Options[\"hash\"].(string)\n\tdopin, _ := req.Options[\"pin\"].(bool)\n\n\tif hash == \"\" {\n\t\thash = cfg.Import.HashFunction.WithDefault(config.DefaultHashFunction)\n\t}\n\n\tvar icodec mc.Code\n\tif err := icodec.Set(inputCodec); err != nil {\n\t\treturn err\n\t}\n\tvar scodec mc.Code\n\tif err := scodec.Set(storeCodec); err != nil {\n\t\treturn err\n\t}\n\tvar mhType mc.Code\n\tif err := mhType.Set(hash); err != nil {\n\t\treturn err\n\t}\n\n\tcidPrefix := cid.Prefix{\n\t\tVersion: 1,\n\t\tCodec: uint64(scodec),\n\t\tMhType: uint64(mhType),\n\t\tMhLength: -1,\n\t}\n\n\tdecoder, err := multicodec.LookupDecoder(uint64(icodec))\n\tif err != nil {\n\t\treturn err\n\t}\n\tencoder, err := multicodec.LookupEncoder(uint64(scodec))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar adder ipld.NodeAdder = api.Dag()\n\tif dopin {\n\t\tadder = api.Dag().Pinning()\n\t}\n\tb := ipld.NewBatch(req.Context, adder)\n\n\tit := req.Files.Entries()\n\tfor it.Next() {\n\t\tfile := files.FileFromEntry(it)\n\t\tif file == nil {\n\t\t\treturn fmt.Errorf(\"expected a regular file\")\n\t\t}\n\n\t\tnode := basicnode.Prototype.Any.NewBuilder()\n\t\tif err := decoder(node, file); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tn := node.Build()\n\n\t\tbd := bytes.NewBuffer([]byte{})\n\t\tif err := encoder(n, bd); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tblockCid, err := cidPrefix.Sum(bd.Bytes())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tblk, err := blocks.NewBlockWithCid(bd.Bytes(), blockCid)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tln := ipldlegacy.LegacyNode{\n\t\t\tBlock: blk,\n\t\t\tNode: n,\n\t\t}\n\n\t\tif err := cmdutils.CheckBlockSize(req, uint64(bd.Len())); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := b.Add(req.Context, &ln); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcid := ln.Cid()\n\t\tif err := res.Emit(&OutputObject{Cid: cid}); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\tif it.Err() != nil {\n\t\treturn it.Err()\n\t}\n\n\tif err := b.Commit(); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n" }, { "path": "core/commands/dag/resolve.go", "content": "package dagcmd\n\nimport (\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc dagResolve(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\tapi, err := cmdenv.GetApi(env, req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\tif err != nil {\n\t\treturn err\n\t}\n\n\trp, remainder, err := api.ResolvePath(req.Context, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn cmds.EmitOnce(res, &ResolveOutput{\n\t\tCid: rp.RootCid(),\n\t\tRemPath: path.SegmentsToString(remainder...),\n\t})\n}\n" }, { "path": "core/commands/dag/stat.go", "content": "package dagcmd\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/dustin/go-humanize\"\n\tmdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipld/merkledag/traverse\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\t\"github.com/ipfs/kubo/core/commands/e\"\n)\n\n// TODO cache every cid traversal in a dp cache\n// if the cid exists in the cache, don't traverse it, and use the cached result\n// to compute the new state\n\nfunc dagStat(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t// Default to true (emit intermediate states) for HTTP/RPC clients that want progress\n\tprogressive := true\n\tif val, specified := req.Options[progressOptionName].(bool); specified {\n\t\tprogressive = val\n\t}\n\tapi, err := cmdenv.GetApi(env, req)\n\tif err != nil {\n\t\treturn err\n\t}\n\tnodeGetter := mdag.NewSession(req.Context, api.Dag())\n\n\tcidSet := cid.NewSet()\n\tdagStatSummary := &DagStatSummary{DagStatsArray: []*DagStat{}}\n\tfor _, a := range req.Arguments {\n\t\tp, err := cmdutils.PathOrCidPath(a)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\trp, remainder, err := api.ResolvePath(req.Context, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif len(remainder) > 0 {\n\t\t\treturn fmt.Errorf(\"cannot return size for anything other than a DAG with a root CID\")\n\t\t}\n\n\t\tobj, err := nodeGetter.Get(req.Context, rp.RootCid())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdagstats := &DagStat{Cid: rp.RootCid()}\n\t\tdagStatSummary.appendStats(dagstats)\n\t\terr = traverse.Traverse(obj, traverse.Options{\n\t\t\tDAG: nodeGetter,\n\t\t\tOrder: traverse.DFSPre,\n\t\t\tFunc: func(current traverse.State) error {\n\t\t\t\tcurrentNodeSize := uint64(len(current.Node.RawData()))\n\t\t\t\tdagstats.Size += currentNodeSize\n\t\t\t\tdagstats.NumBlocks++\n\t\t\t\tif !cidSet.Has(current.Node.Cid()) {\n\t\t\t\t\tdagStatSummary.incrementTotalSize(currentNodeSize)\n\t\t\t\t}\n\t\t\t\tdagStatSummary.incrementRedundantSize(currentNodeSize)\n\t\t\t\tcidSet.Add(current.Node.Cid())\n\t\t\t\tif progressive {\n\t\t\t\t\tif err := res.Emit(dagStatSummary); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tErrFunc: nil,\n\t\t\tSkipDuplicates: true,\n\t\t})\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error traversing DAG: %w\", err)\n\t\t}\n\t}\n\n\tdagStatSummary.UniqueBlocks = cidSet.Len()\n\tdagStatSummary.calculateSummary()\n\n\tif err := res.Emit(dagStatSummary); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc finishCLIStat(res cmds.Response, re cmds.ResponseEmitter) error {\n\t// Determine whether to show progress based on TTY detection or explicit flag\n\tvar showProgress bool\n\tval, specified := res.Request().Options[progressOptionName]\n\tif !specified {\n\t\t// Auto-detect: show progress only if stderr is a TTY\n\t\tif errStat, err := os.Stderr.Stat(); err == nil {\n\t\t\tshowProgress = (errStat.Mode() & os.ModeCharDevice) != 0\n\t\t}\n\t} else {\n\t\tshowProgress = val.(bool)\n\t}\n\n\tvar dagStats *DagStatSummary\n\tfor {\n\t\tv, err := res.Next()\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t\tswitch out := v.(type) {\n\t\tcase *DagStatSummary:\n\t\t\tdagStats = out\n\t\t\t// Ratio == 0 means this is a progress update (not final result)\n\t\t\tif showProgress && dagStats.Ratio == 0 {\n\t\t\t\t// Sum up total progress across all DAGs being scanned\n\t\t\t\tvar totalBlocks int64\n\t\t\t\tvar totalSize uint64\n\t\t\t\tfor _, stat := range dagStats.DagStatsArray {\n\t\t\t\t\ttotalBlocks += stat.NumBlocks\n\t\t\t\t\ttotalSize += stat.Size\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(os.Stderr, \"Fetched/Processed %d blocks, %d bytes (%s)\\r\", totalBlocks, totalSize, humanize.Bytes(totalSize))\n\t\t\t}\n\t\tdefault:\n\t\t\treturn e.TypeErr(out, v)\n\t\t}\n\t}\n\n\t// Clear the progress line before final output\n\tif showProgress {\n\t\tfmt.Fprint(os.Stderr, \"\\033[2K\\r\")\n\t}\n\n\treturn re.Emit(dagStats)\n}\n" }, { "path": "core/commands/dht.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n)\n\nvar ErrNotDHT = errors.New(\"routing service is not a DHT\")\n\nvar DhtCmd = &cmds.Command{\n\tStatus: cmds.Deprecated,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Issue commands directly through the DHT.\",\n\t\tShortDescription: ``,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"query\": queryDhtCmd,\n\t\t\"findprovs\": RemovedDHTCmd,\n\t\t\"findpeer\": RemovedDHTCmd,\n\t\t\"get\": RemovedDHTCmd,\n\t\t\"put\": RemovedDHTCmd,\n\t\t\"provide\": RemovedDHTCmd,\n\t},\n}\n\n// kademlia extends the routing interface with a command to get the peers closest to the target\ntype kademlia interface {\n\trouting.Routing\n\tGetClosestPeers(ctx context.Context, key string) ([]peer.ID, error)\n}\n\nvar queryDhtCmd = &cmds.Command{\n\tStatus: cmds.Deprecated,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Find the closest Peer IDs to a given Peer ID by querying the DHT.\",\n\t\tShortDescription: \"Outputs a list of newline-delimited Peer IDs.\",\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peerID\", true, true, \"The peerID to run the query against.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(dhtVerboseOptionName, \"v\", \"Print extra information.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.HasActiveDHTClient() {\n\t\t\treturn ErrNotDHT\n\t\t}\n\n\t\tid, err := peer.Decode(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn cmds.ClientError(\"invalid peer ID\")\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tdefer cancel()\n\t\tctx, events := routing.RegisterForQueryEvents(ctx)\n\n\t\tclient := nd.DHTClient\n\t\tif nd.DHT != nil && client == nd.DHT {\n\t\t\tclient = nd.DHT.WAN\n\t\t\tif !nd.DHT.WANActive() {\n\t\t\t\tclient = nd.DHT.LAN\n\t\t\t}\n\t\t}\n\n\t\tif d, ok := client.(kademlia); !ok {\n\t\t\treturn errors.New(\"dht client does not support GetClosestPeers\")\n\t\t} else {\n\t\t\terrCh := make(chan error, 1)\n\t\t\tgo func() {\n\t\t\t\tdefer close(errCh)\n\t\t\t\tdefer cancel()\n\t\t\t\tclosestPeers, err := d.GetClosestPeers(ctx, string(id))\n\t\t\t\tfor _, p := range closestPeers {\n\t\t\t\t\trouting.PublishQueryEvent(ctx, &routing.QueryEvent{\n\t\t\t\t\t\tID: p,\n\t\t\t\t\t\tType: routing.FinalPeer,\n\t\t\t\t\t})\n\t\t\t\t}\n\n\t\t\t\tif err != nil {\n\t\t\t\t\terrCh <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}()\n\n\t\t\tfor e := range events {\n\t\t\t\tif err := res.Emit(e); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn <-errCh\n\t\t}\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *routing.QueryEvent) error {\n\t\t\tpfm := pfuncMap{\n\t\t\t\trouting.FinalPeer: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tfmt.Fprintf(out, \"%s\\n\", obj.ID)\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t}\n\t\t\tverbose, _ := req.Options[dhtVerboseOptionName].(bool)\n\t\t\treturn printEvent(out, w, verbose, pfm)\n\t\t}),\n\t},\n\tType: routing.QueryEvent{},\n}\nvar RemovedDHTCmd = &cmds.Command{\n\tStatus: cmds.Removed,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Removed, use 'ipfs routing' instead.\",\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\treturn errors.New(\"removed, use 'ipfs routing' instead\")\n\t},\n}\n" }, { "path": "core/commands/dht_test.go", "content": "package commands\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n\n\tipns \"github.com/ipfs/boxo/ipns\"\n\t\"github.com/libp2p/go-libp2p/core/test\"\n)\n\nfunc TestKeyTranslation(t *testing.T) {\n\tpid := test.RandPeerIDFatal(t)\n\tpkname := namesys.PkRoutingKey(pid)\n\tipnsname := ipns.NameFromPeer(pid).RoutingKey()\n\n\tpkk, err := escapeDhtKey(\"/pk/\" + pid.String())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tipnsk, err := escapeDhtKey(\"/ipns/\" + pid.String())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif pkk != pkname {\n\t\tt.Fatal(\"keys didn't match!\")\n\t}\n\n\tif ipnsk != string(ipnsname) {\n\t\tt.Fatal(\"keys didn't match!\")\n\t}\n}\n" }, { "path": "core/commands/diag.go", "content": "package commands\n\nimport (\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/go-datastore\"\n\t\"github.com/ipfs/go-datastore/mount\"\n\t\"github.com/ipfs/go-datastore/query\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\tnode \"github.com/ipfs/kubo/core/node\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n)\n\nvar DiagCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Generate diagnostic reports.\",\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"sys\": sysDiagCmd,\n\t\t\"cmds\": ActiveReqsCmd,\n\t\t\"profile\": sysProfileCmd,\n\t\t\"datastore\": diagDatastoreCmd,\n\t},\n}\n\nvar diagDatastoreCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Low-level datastore inspection for debugging and testing.\",\n\t\tShortDescription: `\n'ipfs diag datastore' provides low-level access to the datastore for debugging\nand testing purposes.\n\nWARNING: FOR DEBUGGING/TESTING ONLY\n\nThese commands expose internal datastore details and should not be used\nin production workflows. The datastore format may change between versions.\n\nThe daemon must not be running when calling these commands.\n\nWhen the provider keystore datastores exist on disk (nodes with\nProvide.DHT.SweepEnabled=true), they are automatically mounted into the\ndatastore view under /provider/keystore/0/ and /provider/keystore/1/.\n\nEXAMPLES\n\nInspecting pubsub seqno validator state:\n\n $ ipfs diag datastore count /pubsub/seqno/\n 2\n $ ipfs diag datastore get --hex /pubsub/seqno/12D3KooW...\n Key: /pubsub/seqno/12D3KooW...\n Hex Dump:\n 00000000 18 81 81 c8 91 c0 ea f6 |........|\n\nWriting a test key (debugging only):\n\n $ ipfs diag datastore put /test/mykey \"hello\"\n\nInspecting provider keystore (requires SweepEnabled):\n\n $ ipfs diag datastore count /provider/keystore/0/\n $ ipfs diag datastore count /provider/keystore/1/\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"get\": diagDatastoreGetCmd,\n\t\t\"put\": diagDatastorePutCmd,\n\t\t\"count\": diagDatastoreCountCmd,\n\t},\n}\n\nconst diagDatastoreHexOptionName = \"hex\"\n\ntype diagDatastoreGetResult struct {\n\tKey string `json:\"key\"`\n\tValue []byte `json:\"value\"`\n\tHexDump string `json:\"hex_dump,omitempty\"`\n}\n\n// openDiagDatastore opens the repo datastore and conditionally mounts any\n// provider keystore datastores that exist on disk. It returns the composite\n// datastore and a cleanup function that must be called when done.\nfunc openDiagDatastore(env cmds.Environment) (datastore.Datastore, func(), error) {\n\tcctx := env.(*oldcmds.Context)\n\trepo, err := fsrepo.Open(cctx.ConfigRoot)\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"failed to open repo: %w\", err)\n\t}\n\n\textraMounts, extraCloser, err := node.MountKeystoreDatastores(repo)\n\tif err != nil {\n\t\trepo.Close()\n\t\treturn nil, nil, err\n\t}\n\n\tcloser := func() {\n\t\textraCloser()\n\t\trepo.Close()\n\t}\n\n\tif len(extraMounts) == 0 {\n\t\treturn repo.Datastore(), closer, nil\n\t}\n\n\tmounts := []mount.Mount{{Prefix: datastore.NewKey(\"/\"), Datastore: repo.Datastore()}}\n\tmounts = append(mounts, extraMounts...)\n\treturn mount.New(mounts), closer, nil\n}\n\nvar diagDatastoreGetCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Read a raw key from the datastore.\",\n\t\tShortDescription: `\nReturns the value stored at the given datastore key.\nDefault output is raw bytes. Use --hex for human-readable hex dump.\n\nThe daemon must not be running when using this command.\n\nWARNING: FOR DEBUGGING/TESTING ONLY\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, false, \"Datastore key to read (e.g., /pubsub/seqno/)\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(diagDatastoreHexOptionName, \"Output hex dump instead of raw bytes\"),\n\t},\n\tNoRemote: true,\n\tPreRun: DaemonNotRunning,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tds, closer, err := openDiagDatastore(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer closer()\n\n\t\tkeyStr := req.Arguments[0]\n\t\tkey := datastore.NewKey(keyStr)\n\n\t\tval, err := ds.Get(req.Context, key)\n\t\tif err != nil {\n\t\t\tif errors.Is(err, datastore.ErrNotFound) {\n\t\t\t\treturn fmt.Errorf(\"key not found: %s\", keyStr)\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"failed to read key: %w\", err)\n\t\t}\n\n\t\tresult := &diagDatastoreGetResult{\n\t\t\tKey: keyStr,\n\t\t\tValue: val,\n\t\t}\n\n\t\tif hexDump, _ := req.Options[diagDatastoreHexOptionName].(bool); hexDump {\n\t\t\tresult.HexDump = hex.Dump(val)\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, result)\n\t},\n\tType: diagDatastoreGetResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, result *diagDatastoreGetResult) error {\n\t\t\tif result.HexDump != \"\" {\n\t\t\t\tfmt.Fprintf(w, \"Key: %s\\nHex Dump:\\n%s\", result.Key, result.HexDump)\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\t// Raw bytes output\n\t\t\t_, err := w.Write(result.Value)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\nvar diagDatastorePutCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Write a raw key-value pair to the datastore.\",\n\t\tShortDescription: `\nStores the given value at the specified datastore key.\n\nThe daemon must not be running when using this command.\n\nWARNING: FOR DEBUGGING/TESTING ONLY\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, false, \"Datastore key (e.g., /test/mykey)\"),\n\t\tcmds.StringArg(\"value\", true, false, \"Value to store (as a string)\"),\n\t},\n\tNoRemote: true,\n\tPreRun: DaemonNotRunning,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tds, closer, err := openDiagDatastore(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer closer()\n\n\t\tkey := datastore.NewKey(req.Arguments[0])\n\t\tif err := ds.Put(req.Context, key, []byte(req.Arguments[1])); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to put key: %w\", err)\n\t\t}\n\t\tif err := ds.Sync(req.Context, key); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to sync: %w\", err)\n\t\t}\n\t\treturn nil\n\t},\n}\n\ntype diagDatastoreCountResult struct {\n\tPrefix string `json:\"prefix\"`\n\tCount int64 `json:\"count\"`\n}\n\nvar diagDatastoreCountCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Count entries matching a datastore prefix.\",\n\t\tShortDescription: `\nCounts the number of datastore entries whose keys start with the given prefix.\n\nThe daemon must not be running when using this command.\n\nWARNING: FOR DEBUGGING/TESTING ONLY\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"prefix\", true, false, \"Datastore key prefix (e.g., /pubsub/seqno/)\"),\n\t},\n\tNoRemote: true,\n\tPreRun: DaemonNotRunning,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tds, closer, err := openDiagDatastore(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer closer()\n\n\t\tprefix := req.Arguments[0]\n\n\t\tq := query.Query{\n\t\t\tPrefix: prefix,\n\t\t\tKeysOnly: true,\n\t\t}\n\n\t\tresults, err := ds.Query(req.Context, q)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to query datastore: %w\", err)\n\t\t}\n\t\tdefer results.Close()\n\n\t\tvar count int64\n\t\tfor result := range results.Next() {\n\t\t\tif result.Error != nil {\n\t\t\t\treturn fmt.Errorf(\"query error: %w\", result.Error)\n\t\t\t}\n\t\t\tcount++\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &diagDatastoreCountResult{\n\t\t\tPrefix: prefix,\n\t\t\tCount: count,\n\t\t})\n\t},\n\tType: diagDatastoreCountResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, result *diagDatastoreCountResult) error {\n\t\t\t_, err := fmt.Fprintf(w, \"%d\\n\", result.Count)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/e/error.go", "content": "package e\n\nimport (\n\t\"fmt\"\n\t\"runtime/debug\"\n)\n\n// TypeErr returns an error with a string that explains what error was expected and what was received.\nfunc TypeErr(expected, actual any) error {\n\treturn fmt.Errorf(\"expected type %T, got %T\", expected, actual)\n}\n\n// compile time type check that HandlerError is an error\nvar _ error = New(nil)\n\n// HandlerError adds a stack trace to an error\ntype HandlerError struct {\n\tErr error\n\tStack []byte\n}\n\n// Error makes HandlerError implement error\nfunc (err HandlerError) Error() string {\n\treturn fmt.Sprintf(\"%s in:\\n%s\", err.Err.Error(), err.Stack)\n}\n\n// New returns a new HandlerError\nfunc New(err error) HandlerError {\n\treturn HandlerError{Err: err, Stack: debug.Stack()}\n}\n" }, { "path": "core/commands/external.go", "content": "package commands\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc ExternalBinary(instructions string) *cmds.Command {\n\treturn &cmds.Command{\n\t\tArguments: []cmds.Argument{\n\t\t\tcmds.StringArg(\"args\", false, true, \"Arguments for subcommand.\"),\n\t\t},\n\t\tExternal: true,\n\t\tNoRemote: true,\n\t\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t\tbinname := strings.Join(append([]string{\"ipfs\"}, req.Path...), \"-\")\n\t\t\t_, err := exec.LookPath(binname)\n\t\t\tif err != nil {\n\t\t\t\t// special case for '--help' on uninstalled binaries.\n\t\t\t\tfor _, arg := range req.Arguments {\n\t\t\t\t\tif arg == \"--help\" || arg == \"-h\" {\n\t\t\t\t\t\tbuf := new(bytes.Buffer)\n\t\t\t\t\t\tfmt.Fprintf(buf, \"%s is an 'external' command.\\n\", binname)\n\t\t\t\t\t\tfmt.Fprintf(buf, \"It does not currently appear to be installed.\\n\")\n\t\t\t\t\t\tfmt.Fprintf(buf, \"%s\\n\", instructions)\n\t\t\t\t\t\treturn res.Emit(buf)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\treturn fmt.Errorf(\"%s not installed\", binname)\n\t\t\t}\n\n\t\t\tr, w := io.Pipe()\n\n\t\t\tcmd := exec.Command(binname, req.Arguments...)\n\n\t\t\t// TODO: make commands lib be able to pass stdin through daemon\n\t\t\t// cmd.Stdin = req.Stdin()\n\t\t\tcmd.Stdin = io.LimitReader(nil, 0)\n\t\t\tcmd.Stdout = w\n\t\t\tcmd.Stderr = w\n\n\t\t\t// setup env of child program\n\t\t\tosenv := os.Environ()\n\n\t\t\tcmd.Env = osenv\n\n\t\t\terr = cmd.Start()\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to start subcommand: %s\", err)\n\t\t\t}\n\n\t\t\terrC := make(chan error)\n\n\t\t\tgo func() {\n\t\t\t\tvar err error\n\t\t\t\tdefer func() { errC <- err }()\n\t\t\t\terr = cmd.Wait()\n\t\t\t\tw.Close()\n\t\t\t}()\n\n\t\t\terr = res.Emit(r)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\treturn <-errC\n\t\t},\n\t}\n}\n" }, { "path": "core/commands/extra.go", "content": "package commands\n\nimport (\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc CreateCmdExtras(opts ...func(e *cmds.Extra)) *cmds.Extra {\n\te := new(cmds.Extra)\n\tfor _, o := range opts {\n\t\to(e)\n\t}\n\treturn e\n}\n\ntype doesNotUseRepo struct{}\n\nfunc SetDoesNotUseRepo(val bool) func(e *cmds.Extra) {\n\treturn func(e *cmds.Extra) {\n\t\te.SetValue(doesNotUseRepo{}, val)\n\t}\n}\n\nfunc GetDoesNotUseRepo(e *cmds.Extra) (val bool, found bool) {\n\treturn getBoolFlag(e, doesNotUseRepo{})\n}\n\n// doesNotUseConfigAsInput describes commands that do not use the config as\n// input. These commands either initialize the config or perform operations\n// that don't require access to the config.\n//\n// pre-command hooks that require configs must not be run before these\n// commands.\ntype doesNotUseConfigAsInput struct{}\n\nfunc SetDoesNotUseConfigAsInput(val bool) func(e *cmds.Extra) {\n\treturn func(e *cmds.Extra) {\n\t\te.SetValue(doesNotUseConfigAsInput{}, val)\n\t}\n}\n\nfunc GetDoesNotUseConfigAsInput(e *cmds.Extra) (val bool, found bool) {\n\treturn getBoolFlag(e, doesNotUseConfigAsInput{})\n}\n\n// preemptsAutoUpdate describes commands that must be executed without the\n// auto-update pre-command hook\ntype preemptsAutoUpdate struct{}\n\nfunc SetPreemptsAutoUpdate(val bool) func(e *cmds.Extra) {\n\treturn func(e *cmds.Extra) {\n\t\te.SetValue(preemptsAutoUpdate{}, val)\n\t}\n}\n\nfunc GetPreemptsAutoUpdate(e *cmds.Extra) (val bool, found bool) {\n\treturn getBoolFlag(e, preemptsAutoUpdate{})\n}\n\nfunc getBoolFlag(e *cmds.Extra, key any) (val bool, found bool) {\n\tvar ival any\n\tival, found = e.GetValue(key)\n\tif !found {\n\t\treturn false, false\n\t}\n\tval = ival.(bool)\n\treturn val, found\n}\n" }, { "path": "core/commands/files.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\tgopath \"path\"\n\t\"slices\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/node\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\n\tbservice \"github.com/ipfs/boxo/blockservice\"\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\tmfs \"github.com/ipfs/boxo/mfs\"\n\t\"github.com/ipfs/boxo/path\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\t\"github.com/ipfs/go-datastore\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nvar flog = logging.Logger(\"cmds/files\")\n\n// Global counter for unflushed MFS operations\nvar noFlushOperationCounter atomic.Int64\n\n// Cached limit value (read once on first use)\nvar (\n\tnoFlushLimit int64\n\tnoFlushLimitInit sync.Once\n)\n\n// updateNoFlushCounter manages the counter for unflushed operations\nfunc updateNoFlushCounter(nd *core.IpfsNode, flush bool) error {\n\tif flush {\n\t\t// Reset counter when flushing\n\t\tnoFlushOperationCounter.Store(0)\n\t\treturn nil\n\t}\n\n\t// Cache the limit on first use (config doesn't change at runtime)\n\tnoFlushLimitInit.Do(func() {\n\t\tnoFlushLimit = int64(config.DefaultMFSNoFlushLimit)\n\t\tif cfg, err := nd.Repo.Config(); err == nil && cfg.Internal.MFSNoFlushLimit != nil {\n\t\t\tnoFlushLimit = cfg.Internal.MFSNoFlushLimit.WithDefault(int64(config.DefaultMFSNoFlushLimit))\n\t\t}\n\t})\n\n\t// Check if limit reached\n\tif noFlushLimit > 0 && noFlushOperationCounter.Load() >= noFlushLimit {\n\t\treturn fmt.Errorf(\"reached limit of %d unflushed MFS operations. \"+\n\t\t\t\"To resolve: 1) run 'ipfs files flush' to persist changes, \"+\n\t\t\t\"2) use --flush=true (default), or \"+\n\t\t\t\"3) increase Internal.MFSNoFlushLimit in config\", noFlushLimit)\n\t}\n\n\tnoFlushOperationCounter.Add(1)\n\treturn nil\n}\n\n// FilesCmd is the 'ipfs files' command\nvar FilesCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with unixfs files.\",\n\t\tShortDescription: `\nFiles is an API for manipulating IPFS objects as if they were a Unix\nfilesystem.\n\nThe files facility interacts with MFS (Mutable File System). MFS acts as a\nsingle, dynamic filesystem mount. MFS has a root CID that is transparently\nupdated when a change happens (and can be checked with \"ipfs files stat /\").\n\nAll files and folders within MFS are respected and will not be deleted\nduring garbage collections. However, a DAG may be referenced in MFS without\nbeing fully available locally (MFS content is lazy loaded when accessed).\nMFS is independent from the list of pinned items (\"ipfs pin ls\"). Calls to\n\"ipfs pin add\" and \"ipfs pin rm\" will add and remove pins independently of\nMFS. If MFS content that was additionally pinned is removed by calling\n\"ipfs files rm\", it will still remain pinned.\n\nContent added with \"ipfs add\" (which by default also becomes pinned), is not\nadded to MFS. Any content can be lazily referenced from MFS with the command\n\"ipfs files cp /ipfs/ /some/path/\" (see ipfs files cp --help).\n\nNOTE: Most of the subcommands of 'ipfs files' accept the '--flush' flag. It\ndefaults to true and ensures two things: 1) that the changes are reflected in\nthe full MFS structure (updated CIDs) 2) that the parent-folder's cache is\ncleared. Use caution when setting this flag to false. It will improve\nperformance for large numbers of file operations, but it does so at the cost\nof consistency guarantees. If the daemon is unexpectedly killed before running\n'ipfs files flush' on the files in question, then data may be lost. This also\napplies to run 'ipfs repo gc' concurrently with '--flush=false' operations.\n\nWhen using '--flush=false', operations are limited to prevent unbounded\nmemory growth. After reaching Internal.MFSNoFlushLimit operations, further\noperations will fail until you run 'ipfs files flush'. This explicit failure\n(instead of auto-flushing) ensures you maintain control over when data is\npersisted, preventing unexpected partial states and making batch operations\npredictable. We recommend flushing paths regularly, especially folders with\nmany write operations, to clear caches, free memory, and maintain good\nperformance.`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(filesFlushOptionName, \"f\", \"Flush target and ancestors after write.\").WithDefault(true),\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"read\": filesReadCmd,\n\t\t\"write\": filesWriteCmd,\n\t\t\"mv\": filesMvCmd,\n\t\t\"cp\": filesCpCmd,\n\t\t\"ls\": filesLsCmd,\n\t\t\"mkdir\": filesMkdirCmd,\n\t\t\"stat\": filesStatCmd,\n\t\t\"rm\": filesRmCmd,\n\t\t\"flush\": filesFlushCmd,\n\t\t\"chcid\": filesChcidCmd,\n\t\t\"chmod\": filesChmodCmd,\n\t\t\"chroot\": filesChrootCmd,\n\t\t\"touch\": filesTouchCmd,\n\t},\n}\n\nconst (\n\tfilesCidVersionOptionName = \"cid-version\"\n\tfilesHashOptionName = \"hash\"\n)\n\nvar (\n\tcidVersionOption = cmds.IntOption(filesCidVersionOptionName, \"cid-ver\", \"Cid version to use. (experimental)\")\n\thashOption = cmds.StringOption(filesHashOptionName, \"Hash function to use. Will set Cid version to 1 if used. (experimental)\")\n)\n\nvar errFormat = errors.New(\"format was set by multiple options. Only one format option is allowed\")\n\ntype statOutput struct {\n\tHash string\n\tSize uint64\n\tCumulativeSize uint64\n\tBlocks int\n\tType string\n\tWithLocality bool `json:\",omitempty\"`\n\tLocal bool `json:\",omitempty\"`\n\tSizeLocal uint64 `json:\",omitempty\"`\n\tMode uint32 `json:\",omitempty\"`\n\tMtime int64 `json:\",omitempty\"`\n\tMtimeNsecs int `json:\",omitempty\"`\n}\n\nfunc (s *statOutput) MarshalJSON() ([]byte, error) {\n\ttype so statOutput\n\tout := &struct {\n\t\t*so\n\t\tMode string `json:\",omitempty\"`\n\t}{so: (*so)(s)}\n\n\tif s.Mode != 0 {\n\t\tout.Mode = fmt.Sprintf(\"%04o\", s.Mode)\n\t}\n\treturn json.Marshal(out)\n}\n\nfunc (s *statOutput) UnmarshalJSON(data []byte) error {\n\tvar err error\n\ttype so statOutput\n\ttmp := &struct {\n\t\t*so\n\t\tMode string `json:\",omitempty\"`\n\t}{so: (*so)(s)}\n\n\tif err := json.Unmarshal(data, &tmp); err != nil {\n\t\treturn err\n\t}\n\n\tif tmp.Mode != \"\" {\n\t\tmode, err := strconv.ParseUint(tmp.Mode, 8, 32)\n\t\tif err == nil {\n\t\t\ts.Mode = uint32(mode)\n\t\t}\n\t}\n\treturn err\n}\n\nconst (\n\tdefaultStatFormat = `\nSize: \nCumulativeSize: \nChildBlocks: \nType: \nMode: ()\nMtime: `\n\tfilesFormatOptionName = \"format\"\n\tfilesSizeOptionName = \"size\"\n\tfilesWithLocalOptionName = \"with-local\"\n\tfilesStatUnspecified = \"not set\"\n)\n\nvar filesStatCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Display file status.\",\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", true, false, \"Path to node to stat.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(filesFormatOptionName, \"Print statistics in given format. Allowed tokens: \"+\n\t\t\t\" and optional .\"+\n\t\t\t\"Conflicts with other format options.\").WithDefault(defaultStatFormat),\n\t\tcmds.BoolOption(filesHashOptionName, \"Print only hash. Implies '--format='. Conflicts with other format options.\"),\n\t\tcmds.BoolOption(filesSizeOptionName, \"Print only size. Implies '--format='. Conflicts with other format options.\"),\n\t\tcmds.BoolOption(filesWithLocalOptionName, \"Compute the amount of the dag that is local, and if possible the total size\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t_, err := statGetFormatOptions(req)\n\t\tif err != nil {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"invalid parameters: %s\", err)\n\t\t}\n\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpath, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\twithLocal, _ := req.Options[filesWithLocalOptionName].(bool)\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar dagserv ipld.DAGService\n\t\tif withLocal {\n\t\t\t// an offline DAGService will not fetch from the network\n\t\t\tdagserv = dag.NewDAGService(bservice.New(\n\t\t\t\tnode.Blockstore,\n\t\t\t\toffline.Exchange(node.Blockstore),\n\t\t\t))\n\t\t} else {\n\t\t\tdagserv = node.DAG\n\t\t}\n\n\t\tnd, err := getNodeFromPath(req.Context, node, api, path)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\to, err := statNode(nd, enc)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !withLocal {\n\t\t\treturn cmds.EmitOnce(res, o)\n\t\t}\n\n\t\tlocal, sizeLocal, err := walkBlock(req.Context, dagserv, nd)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\to.WithLocality = true\n\t\to.Local = local\n\t\to.SizeLocal = sizeLocal\n\n\t\treturn cmds.EmitOnce(res, o)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *statOutput) error {\n\t\t\tmode, modeo := filesStatUnspecified, filesStatUnspecified\n\t\t\tif out.Mode != 0 {\n\t\t\t\tmode = strings.ToLower(os.FileMode(out.Mode).String())\n\t\t\t\tmodeo = \"0\" + strconv.FormatInt(int64(out.Mode&0x1FF), 8)\n\t\t\t}\n\t\t\tmtime, mtimes, mtimens := filesStatUnspecified, filesStatUnspecified, filesStatUnspecified\n\t\t\tif out.Mtime > 0 {\n\t\t\t\tmtime = time.Unix(out.Mtime, int64(out.MtimeNsecs)).UTC().Format(\"2 Jan 2006, 15:04:05 MST\")\n\t\t\t\tmtimes = strconv.FormatInt(out.Mtime, 10)\n\t\t\t\tmtimens = strconv.Itoa(out.MtimeNsecs)\n\t\t\t}\n\n\t\t\ts, _ := statGetFormatOptions(req)\n\t\t\ts = strings.Replace(s, \"\", out.Hash, -1)\n\t\t\ts = strings.Replace(s, \"\", fmt.Sprintf(\"%d\", out.Size), -1)\n\t\t\ts = strings.Replace(s, \"\", fmt.Sprintf(\"%d\", out.CumulativeSize), -1)\n\t\t\ts = strings.Replace(s, \"\", fmt.Sprintf(\"%d\", out.Blocks), -1)\n\t\t\ts = strings.Replace(s, \"\", out.Type, -1)\n\t\t\ts = strings.Replace(s, \"\", mode, -1)\n\t\t\ts = strings.Replace(s, \"\", modeo, -1)\n\t\t\ts = strings.Replace(s, \"\", mtime, -1)\n\t\t\ts = strings.Replace(s, \"\", mtimes, -1)\n\t\t\ts = strings.Replace(s, \"\", mtimens, -1)\n\n\t\t\tfmt.Fprintln(w, s)\n\n\t\t\tif out.WithLocality {\n\t\t\t\tfmt.Fprintf(w, \"Local: %s of %s (%.2f%%)\\n\",\n\t\t\t\t\thumanize.Bytes(out.SizeLocal),\n\t\t\t\t\thumanize.Bytes(out.CumulativeSize),\n\t\t\t\t\t100.0*float64(out.SizeLocal)/float64(out.CumulativeSize),\n\t\t\t\t)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: statOutput{},\n}\n\nfunc moreThanOne(a, b, c bool) bool {\n\treturn a && b || b && c || a && c\n}\n\nfunc statGetFormatOptions(req *cmds.Request) (string, error) {\n\thash, _ := req.Options[filesHashOptionName].(bool)\n\tsize, _ := req.Options[filesSizeOptionName].(bool)\n\tformat, _ := req.Options[filesFormatOptionName].(string)\n\n\tif moreThanOne(hash, size, format != defaultStatFormat) {\n\t\treturn \"\", errFormat\n\t}\n\n\tif hash {\n\t\treturn \"\", nil\n\t} else if size {\n\t\treturn \"\", nil\n\t} else {\n\t\treturn format, nil\n\t}\n}\n\nfunc statNode(nd ipld.Node, enc cidenc.Encoder) (*statOutput, error) {\n\tc := nd.Cid()\n\n\tcumulsize, err := nd.Size()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tswitch n := nd.(type) {\n\tcase *dag.ProtoNode:\n\t\treturn statProtoNode(n, enc, c, cumulsize)\n\tcase *dag.RawNode:\n\t\treturn &statOutput{\n\t\t\tHash: enc.Encode(c),\n\t\t\tBlocks: 0,\n\t\t\tSize: cumulsize,\n\t\t\tCumulativeSize: cumulsize,\n\t\t\tType: \"file\",\n\t\t}, nil\n\tdefault:\n\t\treturn nil, errors.New(\"not unixfs node (proto or raw)\")\n\t}\n}\n\nfunc statProtoNode(n *dag.ProtoNode, enc cidenc.Encoder, cid cid.Cid, cumulsize uint64) (*statOutput, error) {\n\td, err := ft.FSNodeFromBytes(n.Data())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tstat := statOutput{\n\t\tHash: enc.Encode(cid),\n\t\tBlocks: len(n.Links()),\n\t\tSize: d.FileSize(),\n\t\tCumulativeSize: cumulsize,\n\t}\n\n\tswitch d.Type() {\n\tcase ft.TDirectory, ft.THAMTShard:\n\t\tstat.Type = \"directory\"\n\tcase ft.TFile, ft.TSymlink, ft.TMetadata, ft.TRaw:\n\t\tstat.Type = \"file\"\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unrecognized node type: %s\", d.Type())\n\t}\n\n\tif mode := d.Mode(); mode != 0 {\n\t\tstat.Mode = uint32(mode)\n\t} else if d.Type() == ft.TSymlink {\n\t\tstat.Mode = uint32(os.ModeSymlink | 0x1FF)\n\t}\n\n\tif mt := d.ModTime(); !mt.IsZero() {\n\t\tstat.Mtime = mt.Unix()\n\t\tif ns := mt.Nanosecond(); ns > 0 {\n\t\t\tstat.MtimeNsecs = ns\n\t\t}\n\t}\n\n\treturn &stat, nil\n}\n\nfunc walkBlock(ctx context.Context, dagserv ipld.DAGService, nd ipld.Node) (bool, uint64, error) {\n\t// Start with the block data size\n\tsizeLocal := uint64(len(nd.RawData()))\n\n\tlocal := true\n\n\tfor _, link := range nd.Links() {\n\t\tchild, err := dagserv.Get(ctx, link.Cid)\n\n\t\tif ipld.IsNotFound(err) {\n\t\t\tlocal = false\n\t\t\tcontinue\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn local, sizeLocal, err\n\t\t}\n\n\t\tchildLocal, childLocalSize, err := walkBlock(ctx, dagserv, child)\n\t\tif err != nil {\n\t\t\treturn local, sizeLocal, err\n\t\t}\n\n\t\t// Recursively add the child size\n\t\tlocal = local && childLocal\n\t\tsizeLocal += childLocalSize\n\t}\n\n\treturn local, sizeLocal, nil\n}\n\nvar errFilesCpInvalidUnixFS = errors.New(\"cp: source must be a valid UnixFS (dag-pb or raw codec)\")\nvar filesCpCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add references to IPFS files and directories in MFS (or copy within MFS).\",\n\t\tShortDescription: `\n\"ipfs files cp\" can be used to add references to any IPFS file or directory\n(usually in the form /ipfs/, but also any resolvable path) into MFS.\nThis performs a lazy copy: the full DAG will not be fetched, only the root\nnode being copied.\n\nIt can also be used to copy files within MFS, but in the case when an\nIPFS-path matches an existing MFS path, the IPFS path wins.\n\nIn order to add content to MFS from disk, you can use \"ipfs add\" to obtain the\nIPFS Content Identifier and then \"ipfs files cp\" to copy it into MFS:\n\n$ ipfs add --quieter --pin=false \n# ...\n# ... outputs the root CID at the end\n$ ipfs files cp /ipfs/ /your/desired/mfs/path\n\nIf you wish to fully copy content from a different IPFS peer into MFS, do not\nforget to force IPFS to fetch the full DAG after doing a \"cp\" operation. i.e:\n\n$ ipfs files cp /ipfs/ /your/desired/mfs/path\n$ ipfs pin add \n\nThe lazy-copy feature can also be used to protect partial DAG contents from\ngarbage collection. i.e. adding the Wikipedia root to MFS would not download\nall the Wikipedia, but will prevent any downloaded Wikipedia-DAG content from\nbeing GC'ed.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"source\", true, false, \"Source IPFS or MFS path to copy.\"),\n\t\tcmds.StringArg(\"dest\", true, false, \"Destination within MFS.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(forceOptionName, \"Force overwrite of existing files.\"),\n\t\tcmds.BoolOption(filesParentsOptionName, \"p\", \"Make parent directories as needed.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err := nd.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tprefix, err := getPrefixNew(req, &cfg.Import)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsrc, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tsrc = strings.TrimRight(src, \"/\")\n\n\t\tdst, err := checkPath(req.Arguments[1])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif dst[len(dst)-1] == '/' {\n\t\t\tdst += gopath.Base(src)\n\t\t}\n\n\t\tnode, err := getNodeFromPath(req.Context, nd, api, src)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"cp: cannot get node from path %s: %s\", src, err)\n\t\t}\n\n\t\t// Sanity-check: ensure root CID is a valid UnixFS (dag-pb or raw block)\n\t\t// Context: https://github.com/ipfs/kubo/issues/10331\n\t\tsrcCidType := node.Cid().Type()\n\t\tswitch srcCidType {\n\t\tcase cid.Raw:\n\t\t\tif _, ok := node.(*dag.RawNode); !ok {\n\t\t\t\treturn errFilesCpInvalidUnixFS\n\t\t\t}\n\t\tcase cid.DagProtobuf:\n\t\t\tif _, ok := node.(*dag.ProtoNode); !ok {\n\t\t\t\treturn errFilesCpInvalidUnixFS\n\t\t\t}\n\t\t\tif _, err = ft.FSNodeFromBytes(node.(*dag.ProtoNode).Data()); err != nil {\n\t\t\t\treturn fmt.Errorf(\"%w: %v\", errFilesCpInvalidUnixFS, err)\n\t\t\t}\n\t\tdefault:\n\t\t\treturn errFilesCpInvalidUnixFS\n\t\t}\n\n\t\tmkParents, _ := req.Options[filesParentsOptionName].(bool)\n\t\tif mkParents {\n\t\t\tmaxDirLinks := int(cfg.Import.UnixFSDirectoryMaxLinks.WithDefault(config.DefaultUnixFSDirectoryMaxLinks))\n\t\t\tsizeEstimationMode := cfg.Import.HAMTSizeEstimationMode()\n\t\t\terr := ensureContainingDirectoryExists(nd.FilesRoot, dst, prefix, maxDirLinks, &sizeEstimationMode)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tforce, _ := req.Options[forceOptionName].(bool)\n\t\tif force {\n\t\t\tif err = unlinkNodeIfExists(nd, dst); err != nil {\n\t\t\t\treturn fmt.Errorf(\"cp: cannot unlink existing file: %s\", err)\n\t\t\t}\n\t\t}\n\n\t\tflush, _ := req.Options[filesFlushOptionName].(bool)\n\n\t\tif err := updateNoFlushCounter(nd, flush); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\terr = mfs.PutNode(nd.FilesRoot, dst, node)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"cp: cannot put node in path %s: %s\", dst, err)\n\t\t}\n\t\tif flush {\n\t\t\tif _, err := mfs.FlushPath(req.Context, nd.FilesRoot, dst); err != nil {\n\t\t\t\treturn fmt.Errorf(\"cp: cannot flush the created file %s: %s\", dst, err)\n\t\t\t}\n\t\t\t// Flush parent to clear directory cache and free memory.\n\t\t\tparent := gopath.Dir(dst)\n\t\t\tif _, err = mfs.FlushPath(req.Context, nd.FilesRoot, parent); err != nil {\n\t\t\t\treturn fmt.Errorf(\"cp: cannot flush the created file's parent folder %s: %s\", dst, err)\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n}\n\nfunc getNodeFromPath(ctx context.Context, node *core.IpfsNode, api iface.CoreAPI, p string) (ipld.Node, error) {\n\tswitch {\n\tcase strings.HasPrefix(p, \"/ipfs/\"):\n\t\tpth, err := path.NewPath(p)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn api.ResolveNode(ctx, pth)\n\tdefault:\n\t\tfsn, err := mfs.Lookup(node.FilesRoot, p)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn fsn.GetNode()\n\t}\n}\n\nfunc unlinkNodeIfExists(node *core.IpfsNode, path string) error {\n\tdir, name := gopath.Split(path)\n\tparent, err := mfs.Lookup(node.FilesRoot, dir)\n\tif err != nil {\n\t\tif errors.Is(err, os.ErrNotExist) {\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\n\tpdir, ok := parent.(*mfs.Directory)\n\tif !ok {\n\t\treturn fmt.Errorf(\"not a directory: %s\", dir)\n\t}\n\n\t// Attempt to unlink if child is a file, ignore error since\n\t// we are only concerned with unlinking an existing file.\n\tchild, err := pdir.Child(name)\n\tif err != nil {\n\t\treturn nil // no child file, nothing to unlink\n\t}\n\n\tif child.Type() != mfs.TFile {\n\t\treturn fmt.Errorf(\"not a file: %s\", path)\n\t}\n\n\treturn pdir.Unlink(name)\n}\n\ntype filesLsOutput struct {\n\tEntries []mfs.NodeListing\n}\n\nconst (\n\tlongOptionName = \"long\"\n\tdontSortOptionName = \"U\"\n)\n\nvar filesLsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List directories in the local mutable namespace.\",\n\t\tShortDescription: `\nList directories in the local mutable namespace (works on both IPFS and MFS paths).\n\nExamples:\n\n $ ipfs files ls /welcome/docs/\n about\n contact\n help\n quick-start\n readme\n security-notes\n\n $ ipfs files ls /myfiles/a/b/c/d\n foo\n bar\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", false, false, \"Path to show listing for. Defaults to '/'.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(longOptionName, \"l\", \"Use long listing format.\"),\n\t\tcmds.BoolOption(dontSortOptionName, \"Do not sort; list entries in directory order.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tvar arg string\n\n\t\tif len(req.Arguments) == 0 {\n\t\t\targ = \"/\"\n\t\t} else {\n\t\t\targ = req.Arguments[0]\n\t\t}\n\n\t\tpath, err := checkPath(arg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfsn, err := mfs.Lookup(nd.FilesRoot, path)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tlong, _ := req.Options[longOptionName].(bool)\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tswitch fsn := fsn.(type) {\n\t\tcase *mfs.Directory:\n\t\t\tif !long {\n\t\t\t\tvar output []mfs.NodeListing\n\t\t\t\tnames, err := fsn.ListNames(req.Context)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tfor _, name := range names {\n\t\t\t\t\toutput = append(output, mfs.NodeListing{\n\t\t\t\t\t\tName: name,\n\t\t\t\t\t})\n\t\t\t\t}\n\t\t\t\treturn cmds.EmitOnce(res, &filesLsOutput{output})\n\t\t\t}\n\t\t\tlisting, err := fsn.List(req.Context)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn cmds.EmitOnce(res, &filesLsOutput{listing})\n\t\tcase *mfs.File:\n\t\t\t_, name := gopath.Split(path)\n\t\t\tout := &filesLsOutput{[]mfs.NodeListing{{Name: name}}}\n\t\t\tif long {\n\t\t\t\tout.Entries[0].Type = int(fsn.Type())\n\n\t\t\t\tsize, err := fsn.Size()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tout.Entries[0].Size = size\n\n\t\t\t\tnd, err := fsn.GetNode()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tout.Entries[0].Hash = enc.Encode(nd.Cid())\n\t\t\t}\n\t\t\treturn cmds.EmitOnce(res, out)\n\t\tdefault:\n\t\t\treturn errors.New(\"unrecognized type\")\n\t\t}\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *filesLsOutput) error {\n\t\t\tnoSort, _ := req.Options[dontSortOptionName].(bool)\n\t\t\tif !noSort {\n\t\t\t\tslices.SortFunc(out.Entries, func(a, b mfs.NodeListing) int {\n\t\t\t\t\treturn strings.Compare(a.Name, b.Name)\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tlong, _ := req.Options[longOptionName].(bool)\n\t\t\tfor _, o := range out.Entries {\n\t\t\t\tif long {\n\t\t\t\t\tif o.Type == int(mfs.TDir) {\n\t\t\t\t\t\to.Name += \"/\"\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(w, \"%s\\t%s\\t%d\\n\", o.Name, o.Hash, o.Size)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(w, \"%s\\n\", o.Name)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: filesLsOutput{},\n}\n\nconst (\n\tfilesOffsetOptionName = \"offset\"\n\tfilesCountOptionName = \"count\"\n)\n\nvar filesReadCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Read a file from MFS.\",\n\t\tShortDescription: `\nRead a specified number of bytes from a file at a given offset. By default,\nit will read the entire file similar to the Unix cat.\n\nExamples:\n\n $ ipfs files read /test/hello\n hello\n\t`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", true, false, \"Path to file to be read.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.Int64Option(filesOffsetOptionName, \"o\", \"Byte offset to begin reading from.\"),\n\t\tcmds.Int64Option(filesCountOptionName, \"n\", \"Maximum number of bytes to read.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpath, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfsn, err := mfs.Lookup(nd.FilesRoot, path)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"%s: %w\", path, err)\n\t\t}\n\n\t\tfi, ok := fsn.(*mfs.File)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"%s was not a file\", path)\n\t\t}\n\n\t\trfd, err := fi.Open(mfs.Flags{Read: true})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdefer rfd.Close()\n\n\t\toffset, _ := req.Options[offsetOptionName].(int64)\n\t\tif offset < 0 {\n\t\t\treturn fmt.Errorf(\"cannot specify negative offset\")\n\t\t}\n\n\t\tfilen, err := rfd.Size()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif int64(offset) > filen {\n\t\t\treturn fmt.Errorf(\"offset was past end of file (%d > %d)\", offset, filen)\n\t\t}\n\n\t\t_, err = rfd.Seek(int64(offset), io.SeekStart)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar r io.Reader = &contextReaderWrapper{R: rfd, ctx: req.Context}\n\t\tcount, found := req.Options[filesCountOptionName].(int64)\n\t\tif found {\n\t\t\tif count < 0 {\n\t\t\t\treturn fmt.Errorf(\"cannot specify negative 'count'\")\n\t\t\t}\n\t\t\tr = io.LimitReader(r, int64(count))\n\t\t}\n\t\treturn res.Emit(r)\n\t},\n}\n\ntype contextReader interface {\n\tCtxReadFull(context.Context, []byte) (int, error)\n}\n\ntype contextReaderWrapper struct {\n\tR contextReader\n\tctx context.Context\n}\n\nfunc (crw *contextReaderWrapper) Read(b []byte) (int, error) {\n\treturn crw.R.CtxReadFull(crw.ctx, b)\n}\n\nvar filesMvCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Move files.\",\n\t\tShortDescription: `\nMove files around. Just like the traditional Unix mv.\n\nExample:\n\n $ ipfs files mv /myfs/a/b/c /myfs/foo/newc\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"source\", true, false, \"Source file to move.\"),\n\t\tcmds.StringArg(\"dest\", true, false, \"Destination path for file to be moved to.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tflush, _ := req.Options[filesFlushOptionName].(bool)\n\n\t\tif err := updateNoFlushCounter(nd, flush); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsrc, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdst, err := checkPath(req.Arguments[1])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\terr = mfs.Mv(nd.FilesRoot, src, dst)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif flush {\n\t\t\tparentSrc := gopath.Dir(src)\n\t\t\tparentDst := gopath.Dir(dst)\n\t\t\t// Flush parent to clear directory cache and free memory.\n\t\t\tif _, err = mfs.FlushPath(req.Context, nd.FilesRoot, parentDst); err != nil {\n\t\t\t\treturn fmt.Errorf(\"cp: cannot flush the destination file's parent folder %s: %s\", dst, err)\n\t\t\t}\n\n\t\t\t// Avoid re-flushing when moving within the same folder.\n\t\t\tif parentSrc != parentDst {\n\t\t\t\tif _, err = mfs.FlushPath(req.Context, nd.FilesRoot, parentSrc); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"cp: cannot flush the source's file's parent folder %s: %s\", dst, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif _, err = mfs.FlushPath(req.Context, nd.FilesRoot, \"/\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n}\n\nconst (\n\tfilesCreateOptionName = \"create\"\n\tfilesParentsOptionName = \"parents\"\n\tfilesTruncateOptionName = \"truncate\"\n\tfilesRawLeavesOptionName = \"raw-leaves\"\n\tfilesFlushOptionName = \"flush\"\n)\n\nvar filesWriteCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Append to (modify) a file in MFS.\",\n\t\tShortDescription: `\nA low-level MFS command that allows you to append data to a file. If you want\nto add a file without modifying an existing one, use 'ipfs add --to-files'\ninstead.\n`,\n\t\tLongDescription: `\nA low-level MFS command that allows you to append data at the end of a file, or\nspecify a beginning offset within a file to write to. The entire length of the\ninput will be written.\n\nIf the '--create' option is specified, the file will be created if it does not\nexist. Nonexistent intermediate directories will not be created unless the\n'--parents' option is specified.\n\nNewly created files will have the same CID version and hash function of the\nparent directory unless the '--cid-version' and '--hash' options are used.\n\nNewly created leaves will be in the legacy format (Protobuf) if the\nCID version is 0, or raw if the CID version is non-zero. Use of the\n'--raw-leaves' option will override this behavior.\n\nIf the '--flush' option is set to false, changes will not be propagated to the\nmerkledag root. This can make operations much faster when doing a large number\nof writes to a deeper directory structure.\n\nEXAMPLE:\n\n echo \"hello world\" | ipfs files write --create --parents /myfs/a/b/file\n echo \"hello world\" | ipfs files write --truncate /myfs/a/b/file\n\nWARNING:\n\nUsage of the '--flush=false' option does not guarantee data durability until\nthe tree has been flushed. This can be accomplished by running 'ipfs files\nstat' on the file or any of its ancestors.\n\nWARNING:\n\nThe CID produced by 'files write' will be different from 'ipfs add' because\n'ipfs files write' creates a trickle-dag optimized for append-only operations.\nSee '--trickle' in 'ipfs add --help' for more information.\n\nNOTE: The 'Import.UnixFSFileMaxLinks' config option does not apply to this command.\nTrickle DAG has a fixed internal structure optimized for append operations.\nTo use configurable max-links, use 'ipfs add' with balanced DAG layout.\n\nIf you want to add a file without modifying an existing one,\nuse 'ipfs add' with '--to-files':\n\n > ipfs files mkdir -p /myfs/dir\n > ipfs add example.jpg --to-files /myfs/dir/\n > ipfs files ls /myfs/dir/\n example.jpg\n\nSee '--to-files' in 'ipfs add --help' for more information.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", true, false, \"Path to write to.\"),\n\t\tcmds.FileArg(\"data\", true, false, \"Data to write.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.Int64Option(filesOffsetOptionName, \"o\", \"Byte offset to begin writing at.\"),\n\t\tcmds.BoolOption(filesCreateOptionName, \"e\", \"Create the file if it does not exist.\"),\n\t\tcmds.BoolOption(filesParentsOptionName, \"p\", \"Make parent directories as needed.\"),\n\t\tcmds.BoolOption(filesTruncateOptionName, \"t\", \"Truncate the file to size zero before writing.\"),\n\t\tcmds.Int64Option(filesCountOptionName, \"n\", \"Maximum number of bytes to read.\"),\n\t\tcmds.BoolOption(filesRawLeavesOptionName, \"Use raw blocks for newly created leaf nodes. (experimental)\"),\n\t\tcidVersionOption,\n\t\thashOption,\n\t},\n\tRun: func(req *cmds.Request, re cmds.ResponseEmitter, env cmds.Environment) (retErr error) {\n\t\tpath, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err := nd.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcreate, _ := req.Options[filesCreateOptionName].(bool)\n\t\tmkParents, _ := req.Options[filesParentsOptionName].(bool)\n\t\ttrunc, _ := req.Options[filesTruncateOptionName].(bool)\n\t\tflush, _ := req.Options[filesFlushOptionName].(bool)\n\t\trawLeaves, rawLeavesDef := req.Options[filesRawLeavesOptionName].(bool)\n\n\t\tif err := updateNoFlushCounter(nd, flush); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !rawLeavesDef && cfg.Import.UnixFSRawLeaves != config.Default {\n\t\t\trawLeavesDef = true\n\t\t\trawLeaves = cfg.Import.UnixFSRawLeaves.WithDefault(config.DefaultUnixFSRawLeaves)\n\t\t}\n\n\t\tprefix, err := getPrefixNew(req, &cfg.Import)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toffset, _ := req.Options[filesOffsetOptionName].(int64)\n\t\tif offset < 0 {\n\t\t\treturn fmt.Errorf(\"cannot have negative write offset\")\n\t\t}\n\n\t\tif mkParents {\n\t\t\tmaxDirLinks := int(cfg.Import.UnixFSDirectoryMaxLinks.WithDefault(config.DefaultUnixFSDirectoryMaxLinks))\n\t\t\tsizeEstimationMode := cfg.Import.HAMTSizeEstimationMode()\n\t\t\terr := ensureContainingDirectoryExists(nd.FilesRoot, path, prefix, maxDirLinks, &sizeEstimationMode)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tfi, err := getFileHandle(nd.FilesRoot, path, create, prefix)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif rawLeavesDef {\n\t\t\tfi.RawLeaves = rawLeaves\n\t\t}\n\n\t\twfd, err := fi.Open(mfs.Flags{Write: true, Sync: flush})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdefer func() {\n\t\t\terr := wfd.Close()\n\t\t\tif err != nil {\n\t\t\t\tif retErr == nil {\n\t\t\t\t\tretErr = err\n\t\t\t\t} else {\n\t\t\t\t\tflog.Error(\"files: error closing file mfs file descriptor\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif flush {\n\t\t\t\t// Flush parent to clear directory cache and free memory.\n\t\t\t\tparent := gopath.Dir(path)\n\t\t\t\tif _, err := mfs.FlushPath(req.Context, nd.FilesRoot, parent); err != nil {\n\t\t\t\t\tif retErr == nil {\n\t\t\t\t\t\tretErr = err\n\t\t\t\t\t} else {\n\t\t\t\t\t\tflog.Error(\"files: flushing the parent folder\", err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\tif trunc {\n\t\t\tif err := wfd.Truncate(0); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tcount, countfound := req.Options[filesCountOptionName].(int64)\n\t\tif countfound && count < 0 {\n\t\t\treturn fmt.Errorf(\"cannot have negative byte count\")\n\t\t}\n\n\t\t_, err = wfd.Seek(int64(offset), io.SeekStart)\n\t\tif err != nil {\n\t\t\tflog.Error(\"seekfail: \", err)\n\t\t\treturn err\n\t\t}\n\n\t\tvar r io.Reader\n\t\tr, err = cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif countfound {\n\t\t\tr = io.LimitReader(r, int64(count))\n\t\t}\n\n\t\t_, err = io.Copy(wfd, r)\n\t\treturn err\n\t},\n}\n\nvar filesMkdirCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Make directories.\",\n\t\tShortDescription: `\nCreate the directory if it does not already exist.\n\nThe directory will have the same CID version and hash function of the\nparent directory unless the --cid-version and --hash options are used.\n\nNOTE: All paths must be absolute.\n\nExamples:\n\n $ ipfs files mkdir /test/newdir\n $ ipfs files mkdir -p /test/does/not/exist/yet\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", true, false, \"Path to dir to make.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(filesParentsOptionName, \"p\", \"No error if existing, make parent directories as needed.\"),\n\t\tcidVersionOption,\n\t\thashOption,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcfg, err := n.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdashp, _ := req.Options[filesParentsOptionName].(bool)\n\t\tdirtomake, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tflush, _ := req.Options[filesFlushOptionName].(bool)\n\n\t\tif err := updateNoFlushCounter(n, flush); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tprefix, err := getPrefix(req, &cfg.Import)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\troot := n.FilesRoot\n\n\t\tmaxDirLinks := int(cfg.Import.UnixFSDirectoryMaxLinks.WithDefault(config.DefaultUnixFSDirectoryMaxLinks))\n\t\tsizeEstimationMode := cfg.Import.HAMTSizeEstimationMode()\n\n\t\terr = mfs.Mkdir(root, dirtomake, mfs.MkdirOpts{\n\t\t\tMkparents: dashp,\n\t\t\tFlush: flush,\n\t\t\tCidBuilder: prefix,\n\t\t\tMaxLinks: maxDirLinks,\n\t\t\tSizeEstimationMode: &sizeEstimationMode,\n\t\t})\n\n\t\treturn err\n\t},\n}\n\ntype flushRes struct {\n\tCid string\n}\n\nvar filesFlushCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Flush a given path's data to disk.\",\n\t\tShortDescription: `\nFlush a given path to the disk. This is only useful when other commands\nare run with the '--flush=false'.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", false, false, \"Path to flush. Default: '/'.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpath := \"/\"\n\t\tif len(req.Arguments) > 0 {\n\t\t\tpath = req.Arguments[0]\n\t\t}\n\n\t\tn, err := mfs.FlushPath(req.Context, nd.FilesRoot, path)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Reset the counter (flush always resets)\n\t\tnoFlushOperationCounter.Store(0)\n\n\t\treturn cmds.EmitOnce(res, &flushRes{enc.Encode(n.Cid())})\n\t},\n\tType: flushRes{},\n}\n\nvar filesChcidCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Change the CID version or hash function of the root node of a given path.\",\n\t\tShortDescription: `\nChange the CID version or hash function of the root node of a given path.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", false, false, \"Path to change. Default: '/'.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcidVersionOption,\n\t\thashOption,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpath := \"/\"\n\t\tif len(req.Arguments) > 0 {\n\t\t\tpath = req.Arguments[0]\n\t\t}\n\n\t\tflush, _ := req.Options[filesFlushOptionName].(bool)\n\n\t\t// Note: files chcid is for explicitly changing CID format, so we don't\n\t\t// fall back to Import config here. If no options are provided, it does nothing.\n\t\tprefix, err := getPrefix(req, nil)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := updatePath(nd.FilesRoot, path, prefix); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif flush {\n\t\t\tif _, err = mfs.FlushPath(req.Context, nd.FilesRoot, path); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t// Flush parent to clear directory cache and free memory.\n\t\t\tparent := gopath.Dir(path)\n\t\t\tif _, err = mfs.FlushPath(req.Context, nd.FilesRoot, parent); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t},\n}\n\nfunc updatePath(rt *mfs.Root, pth string, builder cid.Builder) error {\n\tif builder == nil {\n\t\treturn nil\n\t}\n\n\tnd, err := mfs.Lookup(rt, pth)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tswitch n := nd.(type) {\n\tcase *mfs.Directory:\n\t\tn.SetCidBuilder(builder)\n\tdefault:\n\t\treturn fmt.Errorf(\"can only update directories\")\n\t}\n\n\treturn nil\n}\n\nvar filesRmCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove a file from MFS.\",\n\t\tShortDescription: `\nRemove files or directories.\n\n $ ipfs files rm /foo\n $ ipfs files ls /bar\n cat\n dog\n fish\n $ ipfs files rm -r /bar\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", true, true, \"File to remove.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(recursiveOptionName, \"r\", \"Recursively remove directories.\"),\n\t\tcmds.BoolOption(forceOptionName, \"Forcibly remove target at path; implies -r for directories\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t// Check if user explicitly set --flush=false\n\t\tif flushOpt, ok := req.Options[filesFlushOptionName]; ok {\n\t\t\tif flush, ok := flushOpt.(bool); ok && !flush {\n\t\t\t\treturn fmt.Errorf(\"files rm always flushes for safety. The --flush flag cannot be set to false for this command\")\n\t\t\t}\n\t\t}\n\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\t// if '--force' specified, it will remove anything else,\n\t\t// including file, directory, corrupted node, etc\n\t\tforce, _ := req.Options[forceOptionName].(bool)\n\t\tdashr, _ := req.Options[recursiveOptionName].(bool)\n\t\tvar errs []error\n\t\tfor _, arg := range req.Arguments {\n\t\t\tpath, err := checkPath(arg)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"%s is not a valid path: %w\", arg, err))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif err := removePath(nd.FilesRoot, path, force, dashr); err != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"%s: %w\", path, err))\n\t\t\t}\n\t\t}\n\t\tif len(errs) > 0 {\n\t\t\tfor _, err = range errs {\n\t\t\t\te := res.Emit(err.Error())\n\t\t\t\tif e != nil {\n\t\t\t\t\treturn e\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"can't remove some files\")\n\t\t}\n\t\treturn nil\n\t},\n}\n\nfunc removePath(filesRoot *mfs.Root, path string, force bool, dashr bool) error {\n\tif path == \"/\" {\n\t\treturn fmt.Errorf(\"cannot delete root\")\n\t}\n\n\t// 'rm a/b/c/' will fail unless we trim the slash at the end\n\tif path[len(path)-1] == '/' {\n\t\tpath = path[:len(path)-1]\n\t}\n\n\tdir, name := gopath.Split(path)\n\n\tpdir, err := getParentDir(filesRoot, dir)\n\tif err != nil {\n\t\tif force && err == os.ErrNotExist {\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\n\tif force {\n\t\terr := pdir.Unlink(name)\n\t\tif err != nil {\n\t\t\tif err == os.ErrNotExist {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t\treturn pdir.Flush()\n\t}\n\n\t// get child node by name, when the node is corrupted and nonexistent,\n\t// it will return specific error.\n\tchild, err := pdir.Child(name)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tswitch child.(type) {\n\tcase *mfs.Directory:\n\t\tif !dashr {\n\t\t\treturn fmt.Errorf(\"path is a directory, use -r to remove directories\")\n\t\t}\n\t}\n\n\terr = pdir.Unlink(name)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn pdir.Flush()\n}\n\nfunc getPrefixNew(req *cmds.Request, importCfg *config.Import) (cid.Builder, error) {\n\tcidVer, cidVerSet := req.Options[filesCidVersionOptionName].(int)\n\thashFunStr, hashFunSet := req.Options[filesHashOptionName].(string)\n\n\t// Fall back to Import config if CLI options not set\n\tif !cidVerSet && importCfg != nil && !importCfg.CidVersion.IsDefault() {\n\t\tcidVer = int(importCfg.CidVersion.WithDefault(config.DefaultCidVersion))\n\t\tcidVerSet = true\n\t}\n\tif !hashFunSet && importCfg != nil && !importCfg.HashFunction.IsDefault() {\n\t\thashFunStr = importCfg.HashFunction.WithDefault(config.DefaultHashFunction)\n\t\thashFunSet = true\n\t}\n\n\tif !cidVerSet && !hashFunSet {\n\t\treturn nil, nil\n\t}\n\n\tif hashFunSet && cidVer == 0 {\n\t\tcidVer = 1\n\t}\n\n\tprefix, err := dag.PrefixForCidVersion(cidVer)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif hashFunSet {\n\t\thashFunCode, ok := mh.Names[strings.ToLower(hashFunStr)]\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"unrecognized hash function: %s\", strings.ToLower(hashFunStr))\n\t\t}\n\t\tprefix.MhType = hashFunCode\n\t\tprefix.MhLength = -1\n\t}\n\n\treturn &prefix, nil\n}\n\nfunc getPrefix(req *cmds.Request, importCfg *config.Import) (cid.Builder, error) {\n\tcidVer, cidVerSet := req.Options[filesCidVersionOptionName].(int)\n\thashFunStr, hashFunSet := req.Options[filesHashOptionName].(string)\n\n\t// Fall back to Import config if CLI options not set\n\tif !cidVerSet && importCfg != nil && !importCfg.CidVersion.IsDefault() {\n\t\tcidVer = int(importCfg.CidVersion.WithDefault(config.DefaultCidVersion))\n\t\tcidVerSet = true\n\t}\n\tif !hashFunSet && importCfg != nil && !importCfg.HashFunction.IsDefault() {\n\t\thashFunStr = importCfg.HashFunction.WithDefault(config.DefaultHashFunction)\n\t\thashFunSet = true\n\t}\n\n\tif !cidVerSet && !hashFunSet {\n\t\treturn nil, nil\n\t}\n\n\tif hashFunSet && cidVer == 0 {\n\t\tcidVer = 1\n\t}\n\n\tprefix, err := dag.PrefixForCidVersion(cidVer)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif hashFunSet {\n\t\thashFunCode, ok := mh.Names[strings.ToLower(hashFunStr)]\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"unrecognized hash function: %s\", strings.ToLower(hashFunStr))\n\t\t}\n\t\tprefix.MhType = hashFunCode\n\t\tprefix.MhLength = -1\n\t}\n\n\treturn &prefix, nil\n}\n\nfunc ensureContainingDirectoryExists(r *mfs.Root, path string, builder cid.Builder, maxLinks int, sizeEstimationMode *uio.SizeEstimationMode) error {\n\tdirtomake := gopath.Dir(path)\n\n\tif dirtomake == \"/\" {\n\t\treturn nil\n\t}\n\n\treturn mfs.Mkdir(r, dirtomake, mfs.MkdirOpts{\n\t\tMkparents: true,\n\t\tCidBuilder: builder,\n\t\tMaxLinks: maxLinks,\n\t\tSizeEstimationMode: sizeEstimationMode,\n\t})\n}\n\nfunc getFileHandle(r *mfs.Root, path string, create bool, builder cid.Builder) (*mfs.File, error) {\n\ttarget, err := mfs.Lookup(r, path)\n\tswitch err {\n\tcase nil:\n\t\tfi, ok := target.(*mfs.File)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"%s was not a file\", path)\n\t\t}\n\t\treturn fi, nil\n\n\tcase os.ErrNotExist:\n\t\tif !create {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// if create is specified and the file doesn't exist, we create the file\n\t\tdirname, fname := gopath.Split(path)\n\t\tpdir, err := getParentDir(r, dirname)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif builder == nil {\n\t\t\tbuilder = pdir.GetCidBuilder()\n\t\t}\n\n\t\tnd := dag.NodeWithData(ft.FilePBData(nil, 0))\n\t\terr = nd.SetCidBuilder(builder)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\terr = pdir.AddChild(fname, nd)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tfsn, err := pdir.Child(fname)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tfi, ok := fsn.(*mfs.File)\n\t\tif !ok {\n\t\t\treturn nil, errors.New(\"expected *mfs.File, didn't get it. This is likely a race condition\")\n\t\t}\n\t\treturn fi, nil\n\n\tdefault:\n\t\treturn nil, err\n\t}\n}\n\nfunc checkPath(p string) (string, error) {\n\tif len(p) == 0 {\n\t\treturn \"\", fmt.Errorf(\"paths must not be empty\")\n\t}\n\n\tif p[0] != '/' {\n\t\treturn \"\", fmt.Errorf(\"paths must start with a leading slash\")\n\t}\n\n\tcleaned := gopath.Clean(p)\n\tif p[len(p)-1] == '/' && p != \"/\" {\n\t\tcleaned += \"/\"\n\t}\n\treturn cleaned, nil\n}\n\nfunc getParentDir(root *mfs.Root, dir string) (*mfs.Directory, error) {\n\tparent, err := mfs.Lookup(root, dir)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpdir, ok := parent.(*mfs.Directory)\n\tif !ok {\n\t\treturn nil, errors.New(\"expected *mfs.Directory, didn't get it. This is likely a race condition\")\n\t}\n\treturn pdir, nil\n}\n\nvar filesChmodCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Change optional POSIX mode permissions\",\n\t\tShortDescription: `\nThe mode argument must be specified in Unix numeric notation.\n\n $ ipfs files chmod 0644 /foo\n $ ipfs files stat /foo\n ...\n Type: file\n Mode: -rw-r--r-- (0644)\n ...\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"mode\", true, false, \"Mode to apply to node (numeric notation)\"),\n\t\tcmds.StringArg(\"path\", true, false, \"Path to apply mode\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpath, err := checkPath(req.Arguments[1])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tmode, err := strconv.ParseInt(req.Arguments[0], 8, 32)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn mfs.Chmod(nd.FilesRoot, path, os.FileMode(mode))\n\t},\n}\n\nvar filesTouchCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Set or change optional POSIX modification times.\",\n\t\tShortDescription: `\nExamples:\n # set modification time to now.\n $ ipfs files touch /foo\n # set a custom modification time.\n $ ipfs files touch --mtime=1630937926 /foo\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"path\", true, false, \"Path of target to update.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.Int64Option(mtimeOptionName, \"Modification time in seconds before or since the Unix Epoch to apply to created UnixFS entries.\"),\n\t\tcmds.UintOption(mtimeNsecsOptionName, \"Modification time fraction in nanoseconds\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpath, err := checkPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tmtime, _ := req.Options[mtimeOptionName].(int64)\n\t\tnsecs, _ := req.Options[mtimeNsecsOptionName].(uint)\n\n\t\tvar ts time.Time\n\t\tif mtime != 0 {\n\t\t\tts = time.Unix(mtime, int64(nsecs)).UTC()\n\t\t} else {\n\t\t\tts = time.Now().UTC()\n\t\t}\n\n\t\treturn mfs.Touch(nd.FilesRoot, path, ts)\n\t},\n}\n\nconst chrootConfirmOptionName = \"confirm\"\n\nvar filesChrootCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Change the MFS root CID.\",\n\t\tShortDescription: `\n'ipfs files chroot' changes the root CID used by MFS (Mutable File System).\nThis is a recovery command for when MFS becomes corrupted and prevents the\ndaemon from starting.\n\nWhen run without a CID argument, resets MFS to an empty directory.\n\nWARNING: The old MFS root and its unpinned children will be removed during\nthe next garbage collection. Pin the old root first if you want to preserve.\n\nThis command can only run when the daemon is not running.\n\nExamples:\n\n # Reset MFS to empty directory (recovery from corruption)\n $ ipfs files chroot --confirm\n\n # Restore MFS to a known good directory CID\n $ ipfs files chroot --confirm QmYourBackupCID\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"cid\", false, false, \"New root CID (defaults to empty directory if not specified).\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(chrootConfirmOptionName, \"Confirm this potentially destructive operation.\"),\n\t},\n\tNoRemote: true,\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tconfirm, _ := req.Options[chrootConfirmOptionName].(bool)\n\t\tif !confirm {\n\t\t\treturn errors.New(\"this is a potentially destructive operation; pass --confirm to proceed\")\n\t\t}\n\n\t\t// Determine new root CID\n\t\tvar newRootCid cid.Cid\n\t\tif len(req.Arguments) > 0 {\n\t\t\tvar err error\n\t\t\tnewRootCid, err = cid.Decode(req.Arguments[0])\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"invalid CID %q: %w\", req.Arguments[0], err)\n\t\t\t}\n\t\t} else {\n\t\t\t// Default to empty directory\n\t\t\tnewRootCid = ft.EmptyDirNode().Cid()\n\t\t}\n\n\t\t// Get config root to open repo directly\n\t\tcctx := env.(*oldcmds.Context)\n\t\tcfgRoot := cctx.ConfigRoot\n\n\t\t// Open repo directly (daemon must not be running)\n\t\trepo, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"opening repo (is the daemon running?): %w\", err)\n\t\t}\n\t\tdefer repo.Close()\n\n\t\tlocalDS := repo.Datastore()\n\t\tbs := bstore.NewBlockstore(localDS)\n\n\t\t// Check new root exists locally and is a directory\n\t\thasBlock, err := bs.Has(req.Context, newRootCid)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"checking if new root exists: %w\", err)\n\t\t}\n\t\tif !hasBlock {\n\t\t\t// Special case: empty dir is always available (hardcoded in boxo)\n\t\t\temptyDirCid := ft.EmptyDirNode().Cid()\n\t\t\tif !newRootCid.Equals(emptyDirCid) {\n\t\t\t\treturn fmt.Errorf(\"new root %s does not exist locally; fetch it first with 'ipfs block get'\", newRootCid)\n\t\t\t}\n\t\t}\n\n\t\t// Validate it's a directory (not a file)\n\t\tif hasBlock {\n\t\t\tblk, err := bs.Get(req.Context, newRootCid)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"reading new root block: %w\", err)\n\t\t\t}\n\t\t\tpbNode, err := dag.DecodeProtobuf(blk.RawData())\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"new root is not a valid dag-pb node: %w\", err)\n\t\t\t}\n\t\t\tfsNode, err := ft.FSNodeFromBytes(pbNode.Data())\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"new root is not a valid UnixFS node: %w\", err)\n\t\t\t}\n\t\t\tif fsNode.Type() != ft.TDirectory && fsNode.Type() != ft.THAMTShard {\n\t\t\t\treturn fmt.Errorf(\"new root must be a directory, got %s\", fsNode.Type())\n\t\t\t}\n\t\t}\n\n\t\t// Get old root for display (if exists)\n\t\tvar oldRootStr string\n\t\toldRootBytes, err := localDS.Get(req.Context, node.FilesRootDatastoreKey)\n\t\tif err == nil {\n\t\t\toldRootCid, err := cid.Cast(oldRootBytes)\n\t\t\tif err == nil {\n\t\t\t\toldRootStr = oldRootCid.String()\n\t\t\t}\n\t\t} else if !errors.Is(err, datastore.ErrNotFound) {\n\t\t\treturn fmt.Errorf(\"reading current MFS root: %w\", err)\n\t\t}\n\n\t\t// Write new root\n\t\terr = localDS.Put(req.Context, node.FilesRootDatastoreKey, newRootCid.Bytes())\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"writing new MFS root: %w\", err)\n\t\t}\n\n\t\t// Build output message\n\t\tvar msg string\n\t\tif oldRootStr != \"\" {\n\t\t\tmsg = fmt.Sprintf(\"MFS root changed from %s to %s\\n\", oldRootStr, newRootCid)\n\t\t\tmsg += fmt.Sprintf(\"The old root %s will be garbage collected unless pinned.\\n\", oldRootStr)\n\t\t} else {\n\t\t\tmsg = fmt.Sprintf(\"MFS root set to %s\\n\", newRootCid)\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &MessageOutput{Message: msg})\n\t},\n\tType: MessageOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *MessageOutput) error {\n\t\t\t_, err := fmt.Fprint(w, out.Message)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/files_test.go", "content": "package commands\n\nimport (\n\t\"io\"\n\t\"testing\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcoremock \"github.com/ipfs/kubo/core/mock\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestFilesCp_DagCborNodeFails(t *testing.T) {\n\tctx := t.Context()\n\n\tcmdCtx, err := coremock.MockCmdsCtx()\n\trequire.NoError(t, err)\n\n\tnode, err := cmdCtx.ConstructNode()\n\trequire.NoError(t, err)\n\n\tinvalidData := []byte{0x00}\n\tprotoNode := dag.NodeWithData(invalidData)\n\terr = node.DAG.Add(ctx, protoNode)\n\trequire.NoError(t, err)\n\n\treq := &cmds.Request{\n\t\tContext: ctx,\n\t\tArguments: []string{\n\t\t\t\"/ipfs/\" + protoNode.Cid().String(),\n\t\t\t\"/test-destination\",\n\t\t},\n\t\tOptions: map[string]any{\n\t\t\t\"force\": false,\n\t\t},\n\t}\n\n\t_, pw := io.Pipe()\n\tres, err := cmds.NewWriterResponseEmitter(pw, req)\n\trequire.NoError(t, err)\n\n\terr = filesCpCmd.Run(req, res, &cmdCtx)\n\trequire.Error(t, err)\n\trequire.ErrorContains(t, err, \"cp: source must be a valid UnixFS (dag-pb or raw codec)\")\n}\n" }, { "path": "core/commands/filestore.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\tfilestore \"github.com/ipfs/boxo/filestore\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcore \"github.com/ipfs/kubo/core\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\te \"github.com/ipfs/kubo/core/commands/e\"\n\n\t\"github.com/ipfs/go-cid\"\n)\n\nvar FileStoreCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with filestore objects.\",\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"ls\": lsFileStore,\n\t\t\"verify\": verifyFileStore,\n\t\t\"dups\": dupsFileStore,\n\t},\n}\n\nconst (\n\tfileOrderOptionName = \"file-order\"\n\tremoveBadBlocksOptionName = \"remove-bad-blocks\"\n)\n\nvar lsFileStore = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List objects in filestore.\",\n\t\tLongDescription: `\nList objects in the filestore.\n\nIf one or more is specified only list those specific objects,\notherwise list all objects.\n\nThe output is:\n\n \n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"obj\", false, true, \"Cid of objects to list.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(fileOrderOptionName, \"sort the results based on the path of the backing file\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t_, fs, err := getFilestore(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\targs := req.Arguments\n\t\tif len(args) > 0 {\n\t\t\treturn listByArgs(req.Context, res, fs, args, false)\n\t\t}\n\n\t\tfileOrder, _ := req.Options[fileOrderOptionName].(bool)\n\t\tnext, err := filestore.ListAll(req.Context, fs, fileOrder)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor {\n\t\t\tr := next(req.Context)\n\t\t\tif r == nil {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif err := res.Emit(r); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tenc, err := cmdenv.GetCidEncoder(res.Request())\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn streamResult(func(v any, out io.Writer) nonFatalError {\n\t\t\t\tr := v.(*filestore.ListRes)\n\t\t\t\tif r.ErrorMsg != \"\" {\n\t\t\t\t\treturn nonFatalError(r.ErrorMsg)\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(out, \"%s\\n\", r.FormatLong(enc.Encode))\n\t\t\t\treturn \"\"\n\t\t\t})(res, re)\n\t\t},\n\t},\n\tType: filestore.ListRes{},\n}\n\nvar verifyFileStore = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Verify objects in filestore.\",\n\t\tLongDescription: `\nVerify objects in the filestore.\n\nIf one or more is specified only verify those specific objects,\notherwise verify all objects.\n\nThe output is:\n\n []\n\nWhere is one of:\nok: the block can be reconstructed\nchanged: the contents of the backing file have changed\nno-file: the backing file could not be found\nerror: there was some other problem reading the file\nmissing: could not be found in the filestore\nERROR: internal error, most likely due to a corrupt database\n\nWhere is present only when removing bad blocks and is one of:\nremove: link to the block will be removed from datastore\nkeep: keep link, nothing to do\n\nFor ERROR entries the error will also be printed to stderr.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"obj\", false, true, \"Cid of objects to verify.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(fileOrderOptionName, \"verify the objects based on the order of the backing file\"),\n\t\tcmds.BoolOption(removeBadBlocksOptionName, \"remove bad blocks. WARNING: This may remove pinned data. You should run 'ipfs pin verify' after running this command and correct any issues.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t_, fs, err := getFilestore(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tremoveBadBlocks, _ := req.Options[removeBadBlocksOptionName].(bool)\n\t\targs := req.Arguments\n\t\tif len(args) > 0 {\n\t\t\treturn listByArgs(req.Context, res, fs, args, removeBadBlocks)\n\t\t}\n\n\t\tfileOrder, _ := req.Options[fileOrderOptionName].(bool)\n\t\tnext, err := filestore.VerifyAll(req.Context, fs, fileOrder)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor {\n\t\t\tr := next(req.Context)\n\t\t\tif r == nil {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\tif removeBadBlocks && (r.Status != filestore.StatusOk) && (r.Status != filestore.StatusOtherError) {\n\t\t\t\tif err = fs.FileManager().DeleteBlock(req.Context, r.Key); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif err = res.Emit(r); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tenc, err := cmdenv.GetCidEncoder(res.Request())\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\treq := res.Request()\n\t\t\tremoveBadBlocks, _ := req.Options[removeBadBlocksOptionName].(bool)\n\t\t\tfor {\n\t\t\t\tv, err := res.Next()\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err == io.EOF {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tlist, ok := v.(*filestore.ListRes)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn e.TypeErr(list, v)\n\t\t\t\t}\n\n\t\t\t\tif list.Status == filestore.StatusOtherError {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"%s\\n\", list.ErrorMsg)\n\t\t\t\t}\n\n\t\t\t\tif removeBadBlocks {\n\t\t\t\t\taction := \"keep\"\n\t\t\t\t\tif removeBadBlocks && (list.Status != filestore.StatusOk) && (list.Status != filestore.StatusOtherError) {\n\t\t\t\t\t\taction = \"remove\"\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(os.Stdout, \"%s %s %s\\n\", list.Status.Format(), list.FormatLong(enc.Encode), action)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(os.Stdout, \"%s %s\\n\", list.Status.Format(), list.FormatLong(enc.Encode))\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t},\n\tType: filestore.ListRes{},\n}\n\nvar dupsFileStore = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List blocks that are both in the filestore and standard block storage.\",\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\t_, fs, err := getFilestore(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tch, err := fs.FileManager().AllKeysChan(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor cid := range ch {\n\t\t\thave, err := fs.MainBlockstore().Has(req.Context, cid)\n\t\t\tif err != nil {\n\t\t\t\treturn res.Emit(&RefWrapper{Err: err.Error()})\n\t\t\t}\n\t\t\tif have {\n\t\t\t\tif err := res.Emit(&RefWrapper{Ref: enc.Encode(cid)}); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tEncoders: refsEncoderMap,\n\tType: RefWrapper{},\n}\n\nfunc getFilestore(env cmds.Environment) (*core.IpfsNode, *filestore.Filestore, error) {\n\tn, err := cmdenv.GetNode(env)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\tfs := n.Filestore\n\tif fs == nil {\n\t\treturn n, nil, filestore.ErrFilestoreNotEnabled\n\t}\n\treturn n, fs, err\n}\n\nfunc listByArgs(ctx context.Context, res cmds.ResponseEmitter, fs *filestore.Filestore, args []string, removeBadBlocks bool) error {\n\tfor _, arg := range args {\n\t\tc, err := cid.Decode(arg)\n\t\tif err != nil {\n\t\t\tret := &filestore.ListRes{\n\t\t\t\tStatus: filestore.StatusOtherError,\n\t\t\t\tErrorMsg: fmt.Sprintf(\"%s: %v\", arg, err),\n\t\t\t}\n\t\t\tif err := res.Emit(ret); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\t\tr := filestore.Verify(ctx, fs, c)\n\n\t\tif removeBadBlocks && (r.Status != filestore.StatusOk) && (r.Status != filestore.StatusOtherError) {\n\t\t\tif err = fs.FileManager().DeleteBlock(ctx, r.Key); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif err = res.Emit(r); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "core/commands/get.go", "content": "package commands\n\nimport (\n\tgotar \"archive/tar\"\n\t\"bufio\"\n\t\"compress/gzip\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\tgopath \"path\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\t\"github.com/ipfs/kubo/core/commands/e\"\n\n\t\"github.com/cheggaaa/pb\"\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/tar\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nvar ErrInvalidCompressionLevel = errors.New(\"compression level must be between 1 and 9\")\n\nconst (\n\toutputOptionName = \"output\"\n\tarchiveOptionName = \"archive\"\n\tcompressOptionName = \"compress\"\n\tcompressionLevelOptionName = \"compression-level\"\n)\n\nvar GetCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Download IPFS objects.\",\n\t\tShortDescription: `\nStores to disk the data contained an IPFS or IPNS object(s) at the given path.\n\nBy default, the output will be stored at './', but an alternate\npath can be specified with '--output=' or '-o='.\n\nTo output a TAR archive instead of unpacked files, use '--archive' or '-a'.\n\nTo compress the output with GZIP compression, use '--compress' or '-C'. You\nmay also specify the level of compression by specifying '-l=<1-9>'.\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tResponseContentType: \"application/x-tar, or application/gzip when compress=true\",\n\t\t},\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, false, \"The path to the IPFS object(s) to be outputted.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(outputOptionName, \"o\", \"The path where the output should be stored.\"),\n\t\tcmds.BoolOption(archiveOptionName, \"a\", \"Output a TAR archive.\"),\n\t\tcmds.BoolOption(compressOptionName, \"C\", \"Compress the output with GZIP compression.\"),\n\t\tcmds.IntOption(compressionLevelOptionName, \"l\", \"The level of compression (1-9).\"),\n\t\tcmds.BoolOption(progressOptionName, \"p\", \"Stream progress data.\").WithDefault(true),\n\t},\n\tPreRun: func(req *cmds.Request, env cmds.Environment) error {\n\t\t_, err := getCompressOptions(req)\n\t\treturn err\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tctx := req.Context\n\t\tcmplvl, err := getCompressOptions(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfile, err := api.Unixfs().Get(ctx, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsize, err := file.Size()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tres.SetLength(uint64(size))\n\n\t\tarchive, _ := req.Options[archiveOptionName].(bool)\n\t\treader, err := fileArchive(file, p.String(), archive, cmplvl)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tgo func() {\n\t\t\t// We cannot defer a close in the response writer (like we should)\n\t\t\t// Because the cmd framework outsmart us and doesn't call response\n\t\t\t// if the context is over.\n\t\t\t<-ctx.Done()\n\t\t\treader.Close()\n\t\t}()\n\n\t\t// Set Content-Type based on output format.\n\t\t// When compression is enabled, output is gzip (or tar.gz for directories).\n\t\t// Otherwise, tar is used as the transport format.\n\t\tres.SetEncodingType(cmds.OctetStream)\n\t\tif cmplvl != gzip.NoCompression {\n\t\t\tres.SetContentType(\"application/gzip\")\n\t\t} else {\n\t\t\tres.SetContentType(\"application/x-tar\")\n\t\t}\n\n\t\treturn res.Emit(reader)\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\treq := res.Request()\n\n\t\t\tv, err := res.Next()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\toutReader, ok := v.(io.Reader)\n\t\t\tif !ok {\n\t\t\t\treturn e.New(e.TypeErr(outReader, v))\n\t\t\t}\n\n\t\t\toutPath := getOutPath(req)\n\n\t\t\tcmplvl, err := getCompressOptions(req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tarchive, _ := req.Options[archiveOptionName].(bool)\n\t\t\tprogress, _ := req.Options[progressOptionName].(bool)\n\n\t\t\tgw := getWriter{\n\t\t\t\tOut: os.Stdout,\n\t\t\t\tErr: os.Stderr,\n\t\t\t\tArchive: archive,\n\t\t\t\tCompression: cmplvl,\n\t\t\t\tSize: int64(res.Length()),\n\t\t\t\tProgress: progress,\n\t\t\t}\n\n\t\t\treturn gw.Write(outReader, outPath)\n\t\t},\n\t},\n}\n\ntype clearlineReader struct {\n\tio.Reader\n\tout io.Writer\n}\n\nfunc (r *clearlineReader) Read(p []byte) (n int, err error) {\n\tn, err = r.Reader.Read(p)\n\tif err == io.EOF {\n\t\t// callback\n\t\tfmt.Fprintf(r.out, \"\\033[2K\\r\") // clear progress bar line on EOF\n\t}\n\treturn\n}\n\nfunc progressBarForReader(out io.Writer, r io.Reader, l int64) (*pb.ProgressBar, io.Reader) {\n\tbar := makeProgressBar(out, l)\n\tbarR := bar.NewProxyReader(r)\n\treturn bar, &clearlineReader{barR, out}\n}\n\nfunc makeProgressBar(out io.Writer, l int64) *pb.ProgressBar {\n\t// setup bar reader\n\t// TODO: get total length of files\n\tbar := pb.New64(l).SetUnits(pb.U_BYTES)\n\tbar.Output = out\n\n\t// the progress bar lib doesn't give us a way to get the width of the output,\n\t// so as a hack we just use a callback to measure the output, then get rid of it\n\tbar.Callback = func(line string) {\n\t\tterminalWidth := len(line)\n\t\tbar.Callback = nil\n\t\tlog.Infof(\"terminal width: %v\\n\", terminalWidth)\n\t}\n\treturn bar\n}\n\nfunc getOutPath(req *cmds.Request) string {\n\toutPath, _ := req.Options[outputOptionName].(string)\n\tif outPath == \"\" {\n\t\ttrimmed := strings.TrimRight(req.Arguments[0], \"/\")\n\t\t_, outPath = filepath.Split(trimmed)\n\t\toutPath = filepath.Clean(outPath)\n\t}\n\treturn outPath\n}\n\ntype getWriter struct {\n\tOut io.Writer // for output to user\n\tErr io.Writer // for progress bar output\n\n\tArchive bool\n\tCompression int\n\tSize int64\n\tProgress bool\n}\n\nfunc (gw *getWriter) Write(r io.Reader, fpath string) error {\n\tif gw.Archive || gw.Compression != gzip.NoCompression {\n\t\treturn gw.writeArchive(r, fpath)\n\t}\n\treturn gw.writeExtracted(r, fpath)\n}\n\nfunc (gw *getWriter) writeArchive(r io.Reader, fpath string) error {\n\t// adjust file name if tar\n\tif gw.Archive {\n\t\tif !strings.HasSuffix(fpath, \".tar\") && !strings.HasSuffix(fpath, \".tar.gz\") {\n\t\t\tfpath += \".tar\"\n\t\t}\n\t}\n\n\t// adjust file name if gz\n\tif gw.Compression != gzip.NoCompression {\n\t\tif !strings.HasSuffix(fpath, \".gz\") {\n\t\t\tfpath += \".gz\"\n\t\t}\n\t}\n\n\t// create file\n\tfile, err := os.Create(fpath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer file.Close()\n\n\tfmt.Fprintf(gw.Out, \"Saving archive to %s\\n\", fpath)\n\tif gw.Progress {\n\t\tvar bar *pb.ProgressBar\n\t\tbar, r = progressBarForReader(gw.Err, r, gw.Size)\n\t\tbar.Start()\n\t\tdefer bar.Finish()\n\t}\n\n\t_, err = io.Copy(file, r)\n\treturn err\n}\n\nfunc (gw *getWriter) writeExtracted(r io.Reader, fpath string) error {\n\tfmt.Fprintf(gw.Out, \"Saving file(s) to %s\\n\", fpath)\n\tvar progressCb func(int64) int64\n\tif gw.Progress {\n\t\tbar := makeProgressBar(gw.Err, gw.Size)\n\t\tbar.Start()\n\t\tdefer bar.Finish()\n\t\tdefer bar.Set64(gw.Size)\n\t\tprogressCb = bar.Add64\n\t}\n\n\textractor := &tar.Extractor{Path: fpath, Progress: progressCb}\n\treturn extractor.Extract(r)\n}\n\nfunc getCompressOptions(req *cmds.Request) (int, error) {\n\tcmprs, _ := req.Options[compressOptionName].(bool)\n\tcmplvl, cmplvlFound := req.Options[compressionLevelOptionName].(int)\n\tswitch {\n\tcase !cmprs:\n\t\treturn gzip.NoCompression, nil\n\tcase cmprs && !cmplvlFound:\n\t\treturn gzip.DefaultCompression, nil\n\tcase cmprs && (cmplvl < 1 || cmplvl > 9):\n\t\treturn gzip.NoCompression, ErrInvalidCompressionLevel\n\t}\n\treturn cmplvl, nil\n}\n\n// DefaultBufSize is the buffer size for gets. for now, 1MiB, which is ~4 blocks.\n// TODO: does this need to be configurable?\nvar DefaultBufSize = 1048576\n\ntype identityWriteCloser struct {\n\tw io.Writer\n}\n\nfunc (i *identityWriteCloser) Write(p []byte) (int, error) {\n\treturn i.w.Write(p)\n}\n\nfunc (i *identityWriteCloser) Close() error {\n\treturn nil\n}\n\nfunc fileArchive(f files.Node, name string, archive bool, compression int) (io.ReadCloser, error) {\n\tcleaned := gopath.Clean(name)\n\t_, filename := gopath.Split(cleaned)\n\n\t// need to connect a writer to a reader\n\tpiper, pipew := io.Pipe()\n\tcheckErrAndClosePipe := func(err error) bool {\n\t\tif err != nil {\n\t\t\t_ = pipew.CloseWithError(err)\n\t\t\treturn true\n\t\t}\n\t\treturn false\n\t}\n\n\t// use a buffered writer to parallelize task\n\tbufw := bufio.NewWriterSize(pipew, DefaultBufSize)\n\n\t// compression determines whether to use gzip compression.\n\tmaybeGzw, err := newMaybeGzWriter(bufw, compression)\n\tif checkErrAndClosePipe(err) {\n\t\treturn nil, err\n\t}\n\n\tcloseGzwAndPipe := func() {\n\t\tif err := maybeGzw.Close(); checkErrAndClosePipe(err) {\n\t\t\treturn\n\t\t}\n\t\tif err := bufw.Flush(); checkErrAndClosePipe(err) {\n\t\t\treturn\n\t\t}\n\t\tpipew.Close() // everything seems to be ok.\n\t}\n\n\tif !archive && compression != gzip.NoCompression {\n\t\t// the case when the node is a file\n\t\tr := files.ToFile(f)\n\t\tif r == nil {\n\t\t\treturn nil, errors.New(\"file is not regular\")\n\t\t}\n\n\t\tgo func() {\n\t\t\tif _, err := io.Copy(maybeGzw, r); checkErrAndClosePipe(err) {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tcloseGzwAndPipe() // everything seems to be ok\n\t\t}()\n\t} else {\n\t\t// the case for 1. archive, and 2. not archived and not compressed, in\n\t\t// which tar is used anyway as a transport format\n\n\t\t// construct the tar writer\n\t\tw, err := files.NewTarWriter(maybeGzw)\n\t\tif checkErrAndClosePipe(err) {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// if not creating an archive set the format to PAX in order to preserve nanoseconds\n\t\tif !archive {\n\t\t\tw.SetFormat(gotar.FormatPAX)\n\t\t}\n\n\t\tgo func() {\n\t\t\t// write all the nodes recursively\n\t\t\tif err := w.WriteFile(f, filename); checkErrAndClosePipe(err) {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tw.Close() // close tar writer\n\t\t\tcloseGzwAndPipe() // everything seems to be ok\n\t\t}()\n\t}\n\n\treturn piper, nil\n}\n\nfunc newMaybeGzWriter(w io.Writer, compression int) (io.WriteCloser, error) {\n\tif compression != gzip.NoCompression {\n\t\treturn gzip.NewWriterLevel(w, compression)\n\t}\n\treturn &identityWriteCloser{w}, nil\n}\n" }, { "path": "core/commands/get_test.go", "content": "package commands\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc TestGetOutputPath(t *testing.T) {\n\tcases := []struct {\n\t\targs []string\n\t\topts cmds.OptMap\n\t\toutPath string\n\t}{\n\t\t{\n\t\t\targs: []string{\"/ipns/multiformats.io/\"},\n\t\t\topts: map[string]any{\n\t\t\t\t\"output\": \"takes-precedence\",\n\t\t\t},\n\t\t\toutPath: \"takes-precedence\",\n\t\t},\n\t\t{\n\t\t\targs: []string{\"/ipns/multiformats.io/\", \"some-other-arg-to-be-ignored\"},\n\t\t\topts: cmds.OptMap{\n\t\t\t\t\"output\": \"takes-precedence\",\n\t\t\t},\n\t\t\toutPath: \"takes-precedence\",\n\t\t},\n\t\t{\n\t\t\targs: []string{\"/ipns/multiformats.io/\"},\n\t\t\toutPath: \"multiformats.io\",\n\t\t\topts: cmds.OptMap{},\n\t\t},\n\t\t{\n\t\t\targs: []string{\"/ipns/multiformats.io/logo.svg/\"},\n\t\t\toutPath: \"logo.svg\",\n\t\t\topts: cmds.OptMap{},\n\t\t},\n\t\t{\n\t\t\targs: []string{\"/ipns/multiformats.io\", \"some-other-arg-to-be-ignored\"},\n\t\t\toutPath: \"multiformats.io\",\n\t\t\topts: cmds.OptMap{},\n\t\t},\n\t}\n\n\t_, err := GetCmd.GetOptions([]string{})\n\tif err != nil {\n\t\tt.Fatalf(\"error getting default command options: %v\", err)\n\t}\n\n\tfor i, tc := range cases {\n\t\tt.Run(fmt.Sprintf(\"%s-%d\", t.Name(), i), func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\treq, err := cmds.NewRequest(ctx, []string{}, tc.opts, tc.args, nil, GetCmd)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"error creating a command request: %v\", err)\n\t\t\t}\n\n\t\t\tif outPath := getOutPath(req); outPath != tc.outPath {\n\t\t\t\tt.Errorf(\"expected outPath %s to be %s\", outPath, tc.outPath)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "core/commands/helptext_test.go", "content": "package commands\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nfunc checkHelptextRecursive(t *testing.T, name []string, c *cmds.Command) {\n\tc.ProcessHelp()\n\n\tt.Run(strings.Join(name, \"_\"), func(t *testing.T) {\n\t\tif c.External {\n\t\t\tt.Skip(\"external\")\n\t\t}\n\n\t\tt.Run(\"tagline\", func(t *testing.T) {\n\t\t\tif c.Helptext.Tagline == \"\" {\n\t\t\t\tt.Error(\"no Tagline!\")\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"longDescription\", func(t *testing.T) {\n\t\t\tt.Skip(\"not everywhere yet\")\n\t\t\tif c.Helptext.LongDescription == \"\" {\n\t\t\t\tt.Error(\"no LongDescription!\")\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"shortDescription\", func(t *testing.T) {\n\t\t\tt.Skip(\"not everywhere yet\")\n\t\t\tif c.Helptext.ShortDescription == \"\" {\n\t\t\t\tt.Error(\"no ShortDescription!\")\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"synopsis\", func(t *testing.T) {\n\t\t\tt.Skip(\"autogenerated in go-ipfs-cmds\")\n\t\t\tif c.Helptext.Synopsis == \"\" {\n\t\t\t\tt.Error(\"no Synopsis!\")\n\t\t\t}\n\t\t})\n\t})\n\n\tfor subname, sub := range c.Subcommands {\n\t\tcheckHelptextRecursive(t, append(name, subname), sub)\n\t}\n}\n\nfunc TestHelptexts(t *testing.T) {\n\tRoot.ProcessHelp()\n\tcheckHelptextRecursive(t, []string{\"ipfs\"}, Root)\n}\n" }, { "path": "core/commands/id.go", "content": "package commands\n\nimport (\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"slices\"\n\t\"strings\"\n\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tke \"github.com/ipfs/kubo/core/commands/keyencode\"\n\tkb \"github.com/libp2p/go-libp2p-kbucket\"\n\tic \"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n)\n\nconst offlineIDErrorMessage = \"'ipfs id' cannot query information on remote peers without a running daemon; if you only want to convert --peerid-base, pass --offline option\"\n\ntype IdOutput struct { // nolint\n\tID string\n\tPublicKey string\n\tAddresses []string\n\tAgentVersion string\n\tProtocols []protocol.ID\n}\n\nconst (\n\tformatOptionName = \"format\"\n\tidFormatOptionName = \"peerid-base\"\n)\n\nvar IDCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show IPFS node id info.\",\n\t\tShortDescription: `\nPrints out information about the specified peer.\nIf no peer is specified, prints out information for local peers.\n\n'ipfs id' supports the format option for output with the following keys:\n : The peers id.\n: Agent version.\n: Protocol version.\n: Public key.\n: Addresses (newline delimited).\n: Libp2p Protocol registrations (newline delimited).\n\nEXAMPLE:\n\n ipfs id Qmece2RkXhsKe5CRooNisBTh4SK119KrXXGmoK6V3kb8aH -f=\"\\n\"\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peerid\", false, false, \"Peer.ID of node to look up.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(formatOptionName, \"f\", \"Optional output format.\"),\n\t\tcmds.StringOption(idFormatOptionName, \"Encoding used for peer IDs: Can either be a multibase encoded CID or a base58btc encoded multihash. Takes {b58mh|base36|k|base32|b...}.\").WithDefault(\"b58mh\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[idFormatOptionName].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar id peer.ID\n\t\tif len(req.Arguments) > 0 {\n\t\t\tvar err error\n\t\t\tid, err = peer.Decode(req.Arguments[0])\n\t\t\tif err != nil {\n\t\t\t\treturn errors.New(\"invalid peer id\")\n\t\t\t}\n\t\t} else {\n\t\t\tid = n.Identity\n\t\t}\n\n\t\tif id == n.Identity {\n\t\t\toutput, err := printSelf(keyEnc, n)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn cmds.EmitOnce(res, output)\n\t\t}\n\n\t\toffline, _ := req.Options[OfflineOption].(bool)\n\t\tif !offline && !n.IsOnline {\n\t\t\treturn errors.New(offlineIDErrorMessage)\n\t\t}\n\n\t\tif !offline {\n\t\t\t// We need to actually connect to run identify.\n\t\t\terr = n.PeerHost.Connect(req.Context, peer.AddrInfo{ID: id})\n\t\t\tswitch err {\n\t\t\tcase nil:\n\t\t\tcase kb.ErrLookupFailure:\n\t\t\t\treturn errors.New(offlineIDErrorMessage)\n\t\t\tdefault:\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\toutput, err := printPeer(keyEnc, n.Peerstore, id)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn cmds.EmitOnce(res, output)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *IdOutput) error {\n\t\t\tformat, found := req.Options[formatOptionName].(string)\n\t\t\tif found {\n\t\t\t\toutput := format\n\t\t\t\toutput = strings.Replace(output, \"\", out.ID, -1)\n\t\t\t\toutput = strings.Replace(output, \"\", out.AgentVersion, -1)\n\t\t\t\toutput = strings.Replace(output, \"\", out.PublicKey, -1)\n\t\t\t\toutput = strings.Replace(output, \"\", strings.Join(out.Addresses, \"\\n\"), -1)\n\t\t\t\toutput = strings.Replace(output, \"\", strings.Join(protocol.ConvertToStrings(out.Protocols), \"\\n\"), -1)\n\t\t\t\toutput = strings.Replace(output, \"\\\\n\", \"\\n\", -1)\n\t\t\t\toutput = strings.Replace(output, \"\\\\t\", \"\\t\", -1)\n\t\t\t\tfmt.Fprint(w, output)\n\t\t\t} else {\n\t\t\t\tmarshaled, err := json.MarshalIndent(out, \"\", \"\\t\")\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tmarshaled = append(marshaled, byte('\\n'))\n\t\t\t\tfmt.Fprintln(w, string(marshaled))\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: IdOutput{},\n}\n\nfunc printPeer(keyEnc ke.KeyEncoder, ps pstore.Peerstore, p peer.ID) (any, error) {\n\tif p == \"\" {\n\t\treturn nil, errors.New(\"attempted to print nil peer\")\n\t}\n\n\tinfo := new(IdOutput)\n\tinfo.ID = keyEnc.FormatID(p)\n\n\tif pk := ps.PubKey(p); pk != nil {\n\t\tpkb, err := ic.MarshalPublicKey(pk)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tinfo.PublicKey = base64.StdEncoding.EncodeToString(pkb)\n\t}\n\n\taddrInfo := ps.PeerInfo(p)\n\taddrs, err := peer.AddrInfoToP2pAddrs(&addrInfo)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, a := range addrs {\n\t\tinfo.Addresses = append(info.Addresses, a.String())\n\t}\n\tslices.Sort(info.Addresses)\n\n\tprotocols, _ := ps.GetProtocols(p) // don't care about errors here.\n\tfor _, proto := range protocols {\n\t\tinfo.Protocols = append(info.Protocols, protocol.ID(cmdutils.CleanAndTrim(string(proto))))\n\t}\n\tslices.Sort(info.Protocols)\n\n\tif v, err := ps.Get(p, \"AgentVersion\"); err == nil {\n\t\tif vs, ok := v.(string); ok {\n\t\t\tinfo.AgentVersion = cmdutils.CleanAndTrim(vs)\n\t\t}\n\t}\n\n\treturn info, nil\n}\n\n// printing self is special cased as we get values differently.\nfunc printSelf(keyEnc ke.KeyEncoder, node *core.IpfsNode) (any, error) {\n\tinfo := new(IdOutput)\n\tinfo.ID = keyEnc.FormatID(node.Identity)\n\n\tpk := node.PrivateKey.GetPublic()\n\tpkb, err := ic.MarshalPublicKey(pk)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tinfo.PublicKey = base64.StdEncoding.EncodeToString(pkb)\n\n\tif node.PeerHost != nil {\n\t\taddrs, err := peer.AddrInfoToP2pAddrs(host.InfoFromHost(node.PeerHost))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tfor _, a := range addrs {\n\t\t\tinfo.Addresses = append(info.Addresses, a.String())\n\t\t}\n\t\tslices.Sort(info.Addresses)\n\t\tinfo.Protocols = node.PeerHost.Mux().Protocols()\n\t\tslices.Sort(info.Protocols)\n\t}\n\tinfo.AgentVersion = version.GetUserAgentVersion()\n\treturn info, nil\n}\n" }, { "path": "core/commands/keyencode/keyencode.go", "content": "package keyencode\n\nimport (\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nconst ipnsKeyFormatOptionName = \"ipns-base\"\n\nvar OptionIPNSBase = cmds.StringOption(ipnsKeyFormatOptionName, \"Encoding used for keys: Can either be a multibase encoded CID or a base58btc encoded multihash. Takes {b58mh|base36|k|base32|b...}.\").WithDefault(\"base36\")\n\ntype KeyEncoder struct {\n\tbaseEnc *mbase.Encoder\n}\n\nfunc KeyEncoderFromString(formatLabel string) (KeyEncoder, error) {\n\tswitch formatLabel {\n\tcase \"b58mh\", \"v0\":\n\t\treturn KeyEncoder{}, nil\n\tdefault:\n\t\tif enc, err := mbase.EncoderByName(formatLabel); err != nil {\n\t\t\treturn KeyEncoder{}, err\n\t\t} else {\n\t\t\treturn KeyEncoder{&enc}, nil\n\t\t}\n\t}\n}\n\nfunc (enc KeyEncoder) FormatID(id peer.ID) string {\n\tif enc.baseEnc == nil {\n\t\treturn id.String()\n\t}\n\tif s, err := peer.ToCid(id).StringOfBase(enc.baseEnc.Encoding()); err != nil {\n\t\tpanic(err)\n\t} else {\n\t\treturn s\n\t}\n}\n" }, { "path": "core/commands/keystore.go", "content": "package commands\n\nimport (\n\t\"bytes\"\n\t\"crypto/ed25519\"\n\t\"crypto/x509\"\n\t\"encoding/pem\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\n\tkeystore \"github.com/ipfs/boxo/keystore\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/e\"\n\tke \"github.com/ipfs/kubo/core/commands/keyencode\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\tmigrations \"github.com/ipfs/kubo/repo/fsrepo/migrations\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nvar KeyCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Create and list IPNS name keypairs\",\n\t\tShortDescription: `\n'ipfs key gen' generates a new keypair for usage with IPNS and 'ipfs name\npublish'.\n\n > ipfs key gen --type=rsa --size=2048 mykey\n > ipfs name publish --key=mykey QmSomeHash\n\n'ipfs key ls' lists the available keys.\n\n > ipfs key ls\n self\n mykey\n\t\t`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"gen\": keyGenCmd,\n\t\t\"export\": keyExportCmd,\n\t\t\"import\": keyImportCmd,\n\t\t\"list\": keyListDeprecatedCmd,\n\t\t\"ls\": keyListCmd,\n\t\t\"rename\": keyRenameCmd,\n\t\t\"rm\": keyRmCmd,\n\t\t\"rotate\": keyRotateCmd,\n\t\t\"sign\": keySignCmd,\n\t\t\"verify\": keyVerifyCmd,\n\t},\n}\n\ntype KeyOutput struct {\n\tName string\n\tId string //nolint\n}\n\ntype KeyOutputList struct {\n\tKeys []KeyOutput\n}\n\n// KeyRenameOutput define the output type of keyRenameCmd\ntype KeyRenameOutput struct {\n\tWas string\n\tNow string\n\tId string //nolint\n\tOverwrite bool\n}\n\nconst (\n\tkeyStoreAlgorithmDefault = options.Ed25519Key\n\tkeyStoreTypeOptionName = \"type\"\n\tkeyStoreSizeOptionName = \"size\"\n\toldKeyOptionName = \"oldkey\"\n)\n\nvar keyGenCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Create a new keypair\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(keyStoreTypeOptionName, \"t\", \"type of the key to create: rsa, ed25519\").WithDefault(keyStoreAlgorithmDefault),\n\t\tcmds.IntOption(keyStoreSizeOptionName, \"s\", \"size of the key to generate\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"name of key to create\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttyp, f := req.Options[keyStoreTypeOptionName].(string)\n\t\tif !f {\n\t\t\treturn errors.New(\"please specify a key type with --type\")\n\t\t}\n\n\t\tname := req.Arguments[0]\n\t\tif name == \"self\" {\n\t\t\treturn errors.New(\"cannot create key with name 'self'\")\n\t\t}\n\n\t\topts := []options.KeyGenerateOption{options.Key.Type(typ)}\n\n\t\tsize, sizefound := req.Options[keyStoreSizeOptionName].(int)\n\t\tif sizefound {\n\t\t\topts = append(opts, options.Key.Size(size))\n\t\t}\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tkey, err := api.Key().Generate(req.Context, name, opts...)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &KeyOutput{\n\t\t\tName: name,\n\t\t\tId: keyEnc.FormatID(key.ID()),\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ko *KeyOutput) error {\n\t\t\t_, err := w.Write([]byte(ko.Id + \"\\n\"))\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: KeyOutput{},\n}\n\nconst (\n\t// Key format options used both for importing and exporting.\n\tkeyFormatOptionName = \"format\"\n\tkeyFormatPemCleartextOption = \"pem-pkcs8-cleartext\"\n\tkeyFormatLibp2pCleartextOption = \"libp2p-protobuf-cleartext\"\n\tkeyAllowAnyTypeOptionName = \"allow-any-key-type\"\n)\n\nvar keyExportCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Export a keypair\",\n\t\tShortDescription: `\nExports a named libp2p key to disk.\n\nBy default, the output will be stored at './.key', but an alternate\npath can be specified with '--output=' or '-o='.\n\nIt is possible to export a private key to interoperable PEM PKCS8 format by explicitly\npassing '--format=pem-pkcs8-cleartext'. The resulting PEM file can then be consumed\nelsewhere. For example, using openssl to get a PEM with public key:\n\n $ ipfs key export testkey --format=pem-pkcs8-cleartext -o privkey.pem\n $ openssl pkey -in privkey.pem -pubout > pubkey.pem\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"name of key to export\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(outputOptionName, \"o\", \"The path where the output should be stored.\"),\n\t\tcmds.StringOption(keyFormatOptionName, \"f\", \"The format of the exported private key, libp2p-protobuf-cleartext or pem-pkcs8-cleartext.\").WithDefault(keyFormatLibp2pCleartextOption),\n\t},\n\tNoRemote: true,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tname := req.Arguments[0]\n\n\t\tif name == \"self\" {\n\t\t\treturn fmt.Errorf(\"cannot export key with name 'self'\")\n\t\t}\n\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Check repo version, and error out if not matching\n\t\tver, err := migrations.RepoVersion(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif ver != fsrepo.RepoVersion {\n\t\t\treturn fmt.Errorf(\"key export expects repo version (%d) but found (%d)\", fsrepo.RepoVersion, ver)\n\t\t}\n\n\t\t// Export is read-only: safe to read it without acquiring repo lock\n\t\t// (this makes export work when ipfs daemon is already running)\n\t\tksp := filepath.Join(cfgRoot, \"keystore\")\n\t\tks, err := keystore.NewFSKeystore(ksp)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsk, err := ks.Get(name)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"key with name '%s' doesn't exist\", name)\n\t\t}\n\n\t\texportFormat, _ := req.Options[keyFormatOptionName].(string)\n\t\tvar formattedKey []byte\n\t\tswitch exportFormat {\n\t\tcase keyFormatPemCleartextOption:\n\t\t\tstdKey, err := crypto.PrivKeyToStdKey(sk)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"converting libp2p private key to std Go key: %w\", err)\n\t\t\t}\n\t\t\t// For some reason the ed25519.PrivateKey does not use pointer\n\t\t\t// receivers, so we need to convert it for MarshalPKCS8PrivateKey.\n\t\t\t// (We should probably change this upstream in PrivKeyToStdKey).\n\t\t\tif ed25519KeyPointer, ok := stdKey.(*ed25519.PrivateKey); ok {\n\t\t\t\tstdKey = *ed25519KeyPointer\n\t\t\t}\n\t\t\t// This function supports a restricted list of public key algorithms,\n\t\t\t// but we generate and use only the RSA and ed25519 types that are on that list.\n\t\t\tformattedKey, err = x509.MarshalPKCS8PrivateKey(stdKey)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"marshalling key to PKCS8 format: %w\", err)\n\t\t\t}\n\n\t\tcase keyFormatLibp2pCleartextOption:\n\t\t\tformattedKey, err = crypto.MarshalPrivateKey(sk)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"unrecognized export format: %s\", exportFormat)\n\t\t}\n\n\t\treturn res.Emit(bytes.NewReader(formattedKey))\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\treq := res.Request()\n\n\t\t\tv, err := res.Next()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\toutReader, ok := v.(io.Reader)\n\t\t\tif !ok {\n\t\t\t\treturn e.New(e.TypeErr(outReader, v))\n\t\t\t}\n\n\t\t\toutPath, _ := req.Options[outputOptionName].(string)\n\t\t\texportFormat, _ := req.Options[keyFormatOptionName].(string)\n\t\t\tif outPath == \"\" {\n\t\t\t\tvar fileExtension string\n\t\t\t\tswitch exportFormat {\n\t\t\t\tcase keyFormatPemCleartextOption:\n\t\t\t\t\tfileExtension = \"pem\"\n\t\t\t\tcase keyFormatLibp2pCleartextOption:\n\t\t\t\t\tfileExtension = \"key\"\n\t\t\t\t}\n\t\t\t\ttrimmed := strings.TrimRight(fmt.Sprintf(\"%s.%s\", req.Arguments[0], fileExtension), \"/\")\n\t\t\t\t_, outPath = filepath.Split(trimmed)\n\t\t\t\toutPath = filepath.Clean(outPath)\n\t\t\t}\n\n\t\t\t// create file\n\t\t\tfile, err := os.Create(outPath)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer file.Close()\n\n\t\t\tswitch exportFormat {\n\t\t\tcase keyFormatPemCleartextOption:\n\t\t\t\tprivKeyBytes, err := io.ReadAll(outReader)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\terr = pem.Encode(file, &pem.Block{\n\t\t\t\t\tType: \"PRIVATE KEY\",\n\t\t\t\t\tBytes: privKeyBytes,\n\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"encoding PEM block: %w\", err)\n\t\t\t\t}\n\n\t\t\tcase keyFormatLibp2pCleartextOption:\n\t\t\t\t_, err = io.Copy(file, outReader)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t},\n\t},\n}\n\nvar keyImportCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Import a key and prints imported key id\",\n\t\tShortDescription: `\nImports a key and stores it under the provided name.\n\nBy default, the key is assumed to be in 'libp2p-protobuf-cleartext' format,\nhowever it is possible to import private keys wrapped in interoperable PEM PKCS8\nby passing '--format=pem-pkcs8-cleartext'.\n\nThe PEM format allows for key generation outside of the IPFS node:\n\n $ openssl genpkey -algorithm ED25519 > ed25519.pem\n $ ipfs key import test-openssl -f pem-pkcs8-cleartext ed25519.pem\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tke.OptionIPNSBase,\n\t\tcmds.StringOption(keyFormatOptionName, \"f\", \"The format of the private key to import, libp2p-protobuf-cleartext or pem-pkcs8-cleartext.\").WithDefault(keyFormatLibp2pCleartextOption),\n\t\tcmds.BoolOption(keyAllowAnyTypeOptionName, \"Allow importing any key type.\").WithDefault(false),\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"name to associate with key in keychain\"),\n\t\tcmds.FileArg(\"key\", true, false, \"key provided by generate or export\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tname := req.Arguments[0]\n\n\t\tif name == \"self\" {\n\t\t\treturn fmt.Errorf(\"cannot import key with name 'self'\")\n\t\t}\n\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\tdata, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\timportFormat, _ := req.Options[keyFormatOptionName].(string)\n\t\tvar sk crypto.PrivKey\n\t\tswitch importFormat {\n\t\tcase keyFormatPemCleartextOption:\n\t\t\tpemBlock, rest := pem.Decode(data)\n\t\t\tif pemBlock == nil {\n\t\t\t\treturn fmt.Errorf(\"PEM block not found in input data:\\n%s\", rest)\n\t\t\t}\n\n\t\t\tif pemBlock.Type != \"PRIVATE KEY\" {\n\t\t\t\treturn fmt.Errorf(\"expected PRIVATE KEY type in PEM block but got: %s\", pemBlock.Type)\n\t\t\t}\n\n\t\t\tstdKey, err := x509.ParsePKCS8PrivateKey(pemBlock.Bytes)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"parsing PKCS8 format: %w\", err)\n\t\t\t}\n\n\t\t\t// In case ed25519.PrivateKey is returned we need the pointer for\n\t\t\t// conversion to libp2p (see export command for more details).\n\t\t\tif ed25519KeyPointer, ok := stdKey.(ed25519.PrivateKey); ok {\n\t\t\t\tstdKey = &ed25519KeyPointer\n\t\t\t}\n\n\t\t\tsk, _, err = crypto.KeyPairFromStdKey(stdKey)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"converting std Go key to libp2p key: %w\", err)\n\t\t\t}\n\t\tcase keyFormatLibp2pCleartextOption:\n\t\t\tsk, err = crypto.UnmarshalPrivateKey(data)\n\t\t\tif err != nil {\n\t\t\t\t// check if data is PEM, if so, provide user with hint\n\t\t\t\tpemBlock, _ := pem.Decode(data)\n\t\t\t\tif pemBlock != nil {\n\t\t\t\t\treturn fmt.Errorf(\"unexpected PEM block for format=%s: try again with format=%s\", keyFormatLibp2pCleartextOption, keyFormatPemCleartextOption)\n\t\t\t\t}\n\t\t\t\treturn fmt.Errorf(\"unable to unmarshall format=%s: %w\", keyFormatLibp2pCleartextOption, err)\n\t\t\t}\n\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"unrecognized import format: %s\", importFormat)\n\t\t}\n\n\t\t// We only allow importing keys of the same type we generate (see list in\n\t\t// https://github.com/ipfs/interface-go-ipfs-core/blob/1c3d8fc/options/key.go#L58-L60),\n\t\t// unless explicitly stated by the user.\n\t\tallowAnyKeyType, _ := req.Options[keyAllowAnyTypeOptionName].(bool)\n\t\tif !allowAnyKeyType {\n\t\t\tswitch t := sk.(type) {\n\t\t\tcase *crypto.RsaPrivateKey, *crypto.Ed25519PrivateKey:\n\t\t\tdefault:\n\t\t\t\treturn fmt.Errorf(\"key type %T is not allowed to be imported, only RSA or Ed25519;\"+\n\t\t\t\t\t\" use flag --%s if you are sure of what you're doing\",\n\t\t\t\t\tt, keyAllowAnyTypeOptionName)\n\t\t\t}\n\t\t}\n\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\n\t\t_, err = r.Keystore().Get(name)\n\t\tif err == nil {\n\t\t\treturn fmt.Errorf(\"key with name '%s' already exists\", name)\n\t\t}\n\n\t\terr = r.Keystore().Put(name, sk)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpid, err := peer.IDFromPrivateKey(sk)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &KeyOutput{\n\t\t\tName: name,\n\t\t\tId: keyEnc.FormatID(pid),\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ko *KeyOutput) error {\n\t\t\t_, err := w.Write([]byte(ko.Id + \"\\n\"))\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: KeyOutput{},\n}\n\nvar keyListCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List all local keypairs.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(\"l\", \"Show extra information about keys.\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"cannot get key encoder: %w\", err)\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tkeys, err := api.Key().List(req.Context)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"listing keys failed: %w\", err)\n\t\t}\n\n\t\tlist := make([]KeyOutput, 0, len(keys))\n\n\t\tfor _, key := range keys {\n\t\t\tlist = append(list, KeyOutput{\n\t\t\t\tName: key.Name(),\n\t\t\t\tId: keyEnc.FormatID(key.ID()),\n\t\t\t})\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &KeyOutputList{list})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: keyOutputListEncoders(),\n\t},\n\tType: KeyOutputList{},\n}\n\nvar keyListDeprecatedCmd = &cmds.Command{\n\tStatus: cmds.Deprecated,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated: use 'ipfs key ls' instead.\",\n\t},\n\tOptions: keyListCmd.Options,\n\tRun: keyListCmd.Run,\n\tEncoders: keyListCmd.Encoders,\n\tType: keyListCmd.Type,\n}\n\nconst (\n\tkeyStoreForceOptionName = \"force\"\n)\n\nvar keyRenameCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Rename a keypair.\",\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"name of key to rename\"),\n\t\tcmds.StringArg(\"newName\", true, false, \"new name of the key\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(keyStoreForceOptionName, \"f\", \"Allow to overwrite an existing key.\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tname := req.Arguments[0]\n\t\tnewName := req.Arguments[1]\n\t\tforce, _ := req.Options[keyStoreForceOptionName].(bool)\n\n\t\tkey, overwritten, err := api.Key().Rename(req.Context, name, newName, options.Key.Force(force))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &KeyRenameOutput{\n\t\t\tWas: name,\n\t\t\tNow: newName,\n\t\t\tId: keyEnc.FormatID(key.ID()),\n\t\t\tOverwrite: overwritten,\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, kro *KeyRenameOutput) error {\n\t\t\tif kro.Overwrite {\n\t\t\t\tfmt.Fprintf(w, \"Key %s renamed to %s with overwriting\\n\", kro.Id, cmdenv.EscNonPrint(kro.Now))\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(w, \"Key %s renamed to %s\\n\", kro.Id, cmdenv.EscNonPrint(kro.Now))\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: KeyRenameOutput{},\n}\n\nvar keyRmCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove a keypair.\",\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, true, \"names of keys to remove\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(\"l\", \"Show extra information about keys.\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnames := req.Arguments\n\n\t\tlist := make([]KeyOutput, 0, len(names))\n\t\tfor _, name := range names {\n\t\t\tkey, err := api.Key().Remove(req.Context, name)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tlist = append(list, KeyOutput{\n\t\t\t\tName: name,\n\t\t\t\tId: keyEnc.FormatID(key.ID()),\n\t\t\t})\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &KeyOutputList{list})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: keyOutputListEncoders(),\n\t},\n\tType: KeyOutputList{},\n}\n\nvar keyRotateCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Rotates the IPFS identity.\",\n\t\tShortDescription: `\nGenerates a new ipfs identity and saves it to the ipfs config file.\nYour existing identity key will be backed up in the Keystore.\nThe daemon must not be running when calling this command.\n\nipfs uses a repository in the local file system. By default, the repo is\nlocated at ~/.ipfs. To change the repo location, set the $IPFS_PATH\nenvironment variable:\n\n export IPFS_PATH=/path/to/ipfsrepo\n`,\n\t},\n\tArguments: []cmds.Argument{},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(oldKeyOptionName, \"o\", \"Keystore name to use for backing up your existing identity\"),\n\t\tcmds.StringOption(keyStoreTypeOptionName, \"t\", \"type of the key to create: rsa, ed25519\").WithDefault(keyStoreAlgorithmDefault),\n\t\tcmds.IntOption(keyStoreSizeOptionName, \"s\", \"size of the key to generate\"),\n\t},\n\tNoRemote: true,\n\tPreRun: DaemonNotRunning,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcctx := env.(*oldcmds.Context)\n\t\tnBitsForKeypair, nBitsGiven := req.Options[keyStoreSizeOptionName].(int)\n\t\talgorithm, _ := req.Options[keyStoreTypeOptionName].(string)\n\t\toldKey, ok := req.Options[oldKeyOptionName].(string)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"keystore name for backing up old key must be provided\")\n\t\t}\n\t\tif oldKey == \"self\" {\n\t\t\treturn fmt.Errorf(\"keystore name for back up cannot be named 'self'\")\n\t\t}\n\t\treturn doRotate(os.Stdout, cctx.ConfigRoot, oldKey, algorithm, nBitsForKeypair, nBitsGiven)\n\t},\n}\n\nfunc doRotate(out io.Writer, repoRoot string, oldKey string, algorithm string, nBitsForKeypair int, nBitsGiven bool) error {\n\t// Open repo\n\trepo, err := fsrepo.Open(repoRoot)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"opening repo (%v)\", err)\n\t}\n\tdefer repo.Close()\n\n\t// Read config file from repo\n\tcfg, err := repo.Config()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"reading config from repo (%v)\", err)\n\t}\n\n\t// Generate new identity\n\tvar identity config.Identity\n\tif nBitsGiven {\n\t\tidentity, err = config.CreateIdentity(out, []options.KeyGenerateOption{\n\t\t\toptions.Key.Size(nBitsForKeypair),\n\t\t\toptions.Key.Type(algorithm),\n\t\t})\n\t} else {\n\t\tidentity, err = config.CreateIdentity(out, []options.KeyGenerateOption{\n\t\t\toptions.Key.Type(algorithm),\n\t\t})\n\t}\n\tif err != nil {\n\t\treturn fmt.Errorf(\"creating identity (%v)\", err)\n\t}\n\n\t// Save old identity to keystore\n\toldPrivKey, err := cfg.Identity.DecodePrivateKey(\"\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"decoding old private key (%v)\", err)\n\t}\n\tkeystore := repo.Keystore()\n\tif err := keystore.Put(oldKey, oldPrivKey); err != nil {\n\t\treturn fmt.Errorf(\"saving old key in keystore (%v)\", err)\n\t}\n\n\t// Update identity\n\tcfg.Identity = identity\n\n\t// Write config file to repo\n\tif err = repo.SetConfig(cfg); err != nil {\n\t\treturn fmt.Errorf(\"saving new key to config (%v)\", err)\n\t}\n\treturn nil\n}\n\nfunc keyOutputListEncoders() cmds.EncoderFunc {\n\treturn cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, list *KeyOutputList) error {\n\t\twithID, _ := req.Options[\"l\"].(bool)\n\n\t\ttw := tabwriter.NewWriter(w, 1, 2, 1, ' ', 0)\n\t\tfor _, s := range list.Keys {\n\t\t\tif withID {\n\t\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\t\\n\", s.Id, cmdenv.EscNonPrint(s.Name))\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(tw, \"%s\\n\", cmdenv.EscNonPrint(s.Name))\n\t\t\t}\n\t\t}\n\t\ttw.Flush()\n\t\treturn nil\n\t})\n}\n\ntype KeySignOutput struct {\n\tKey KeyOutput\n\tSignature string\n}\n\nvar keySignCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Generates a signature for the given data with a specified key. Useful for proving the key ownership.\",\n\t\tLongDescription: `\nSign arbitrary bytes, such as to prove ownership of a Peer ID or an IPNS Name.\nTo avoid signature reuse, the signed payload is always prefixed with\n\"libp2p-key signed message:\".\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(\"key\", \"k\", \"The name of the key to use for signing.\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"data\", true, false, \"The data to sign.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tname, _ := req.Options[\"key\"].(string)\n\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\tdata, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tkey, signature, err := api.Key().Sign(req.Context, name, data)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tencodedSignature, err := mbase.Encode(mbase.Base64url, signature)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn res.Emit(&KeySignOutput{\n\t\t\tKey: KeyOutput{\n\t\t\t\tName: key.Name(),\n\t\t\t\tId: keyEnc.FormatID(key.ID()),\n\t\t\t},\n\t\t\tSignature: encodedSignature,\n\t\t})\n\t},\n\tType: KeySignOutput{},\n}\n\ntype KeyVerifyOutput struct {\n\tKey KeyOutput\n\tSignatureValid bool\n}\n\nvar keyVerifyCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Verify that the given data and signature match.\",\n\t\tLongDescription: `\nVerify if the given data and signatures match. To avoid the signature reuse,\nthe signed payload is always prefixed with \"libp2p-key signed message:\".\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(\"key\", \"k\", \"The name of the key to use for verifying.\"),\n\t\tcmds.StringOption(\"signature\", \"s\", \"Multibase-encoded signature to verify.\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"data\", true, false, \"The data to verify against the given signature.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tname, _ := req.Options[\"key\"].(string)\n\t\tencodedSignature, _ := req.Options[\"signature\"].(string)\n\n\t\t_, signature, err := mbase.Decode(encodedSignature)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\tdata, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tkey, valid, err := api.Key().Verify(req.Context, name, signature, data)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn res.Emit(&KeyVerifyOutput{\n\t\t\tKey: KeyOutput{\n\t\t\t\tName: key.Name(),\n\t\t\t\tId: keyEnc.FormatID(key.ID()),\n\t\t\t},\n\t\t\tSignatureValid: valid,\n\t\t})\n\t},\n\tType: KeyVerifyOutput{},\n}\n\n// DaemonNotRunning checks to see if the ipfs repo is locked, indicating that\n// the daemon is running, and returns and error if the daemon is running.\nfunc DaemonNotRunning(req *cmds.Request, env cmds.Environment) error {\n\tcctx := env.(*oldcmds.Context)\n\tdaemonLocked, err := fsrepo.LockedByOtherProcess(cctx.ConfigRoot)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlog.Info(\"checking if daemon is running...\")\n\tif daemonLocked {\n\t\tlog.Debug(\"ipfs daemon is running\")\n\t\te := \"ipfs daemon is running. please stop it to run this command\"\n\t\treturn cmds.ClientError(e)\n\t}\n\n\treturn nil\n}\n" }, { "path": "core/commands/log.go", "content": "package commands\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"slices\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n)\n\nconst (\n\t// allLogSubsystems is used to specify all log subsystems when setting the\n\t// log level.\n\tallLogSubsystems = \"*\"\n\t// allLogSubsystemsAlias is a convenience alias for allLogSubsystems that\n\t// doesn't require shell escaping.\n\tallLogSubsystemsAlias = \"all\"\n\t// defaultLogLevel is used to request and to identify the default log\n\t// level.\n\tdefaultLogLevel = \"default\"\n\t// defaultSubsystemKey is the subsystem name that is used to denote the\n\t// default log level. We use parentheses for UI clarity to distinguish it\n\t// from regular subsystem names.\n\tdefaultSubsystemKey = \"(default)\"\n\t// logLevelOption is an option for the tail subcommand to select the log\n\t// level to output.\n\tlogLevelOption = \"log-level\"\n\t// noSubsystemSpecified is used when no subsystem argument is provided\n\tnoSubsystemSpecified = \"\"\n)\n\ntype logLevelOutput struct {\n\tLevels map[string]string `json:\",omitempty\"`\n\tMessage string `json:\",omitempty\"`\n}\n\nvar LogCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with the daemon log output.\",\n\t\tShortDescription: `\n'ipfs log' contains utility commands to affect or read the logging\noutput of a running daemon.\n\nThere are also two environmental variables that direct the logging\nsystem (not just for the daemon logs, but all commands):\n GOLOG_LOG_LEVEL - sets the level of verbosity of the logging.\n One of: debug, info, warn, error, dpanic, panic, fatal\n GOLOG_LOG_FMT - sets formatting of the log output.\n One of: color, nocolor, json\n`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"level\": logLevelCmd,\n\t\t\"ls\": logLsCmd,\n\t\t\"tail\": logTailCmd,\n\t},\n}\n\nvar logLevelCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Change or get the logging level.\",\n\t\tShortDescription: `\nGet or change the logging level of one or all logging subsystems.\n\nThis command provides a runtime alternative to the GOLOG_LOG_LEVEL\nenvironment variable for debugging and troubleshooting.\n\nUNDERSTANDING DEFAULT vs '*':\n\nThe \"default\" level is the fallback used by unconfigured subsystems.\nYou cannot set the default level directly - it only changes when you use '*'.\n\nThe '*' wildcard represents ALL subsystems including the default level.\nSetting '*' changes everything at once, including the default.\n\nEXAMPLES - Getting levels:\n\n ipfs log level # Show only the default fallback level\n ipfs log level all # Show all subsystem levels (100+ lines)\n ipfs log level core # Show level for 'core' subsystem only\n\nEXAMPLES - Setting levels:\n\n ipfs log level core debug # Set 'core' to 'debug' (default unchanged)\n ipfs log level all info # Set ALL to 'info' (including default)\n ipfs log level core default # Reset 'core' to use current default level\n\nWILDCARD OPTIONS:\n\nUse 'all' (convenient) or '*' (requires escaping) to affect all subsystems:\n ipfs log level all debug # Convenient - no shell escaping needed\n ipfs log level '*' debug # Equivalent but needs quotes: '*' or \"*\" or \\*\n\nBEHAVIOR EXAMPLES:\n\nInitial state (all using default 'error'):\n $ ipfs log level => error\n $ ipfs log level core => error\n\nAfter setting one subsystem:\n $ ipfs log level core debug\n $ ipfs log level => error (default unchanged!)\n $ ipfs log level core => debug (explicitly set)\n $ ipfs log level dht => error (still uses default)\n\nAfter setting everything with 'all':\n $ ipfs log level all info\n $ ipfs log level => info (default changed!)\n $ ipfs log level core => info (all changed)\n $ ipfs log level dht => info (all changed)\n\nThe 'default' keyword always refers to the current default level:\n $ ipfs log level => error\n $ ipfs log level core default # Sets core to 'error'\n $ ipfs log level all info # Changes default to 'info'\n $ ipfs log level core default # Now sets core to 'info'\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"subsystem\", false, false, fmt.Sprintf(\"The subsystem logging identifier. Use '%s' or '%s' to get or set the log level of all subsystems including the default. If not specified, only show the default log level.\", allLogSubsystemsAlias, allLogSubsystems)),\n\t\tcmds.StringArg(\"level\", false, false, fmt.Sprintf(\"The log level, with 'debug' as the most verbose and 'fatal' the least verbose. Use '%s' to set to the current default level. One of: debug, info, warn, error, dpanic, panic, fatal, %s\", defaultLogLevel, defaultLogLevel)),\n\t},\n\tNoLocal: true,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tvar level, subsystem string\n\n\t\tif len(req.Arguments) > 0 {\n\t\t\tsubsystem = req.Arguments[0]\n\t\t\tif len(req.Arguments) > 1 {\n\t\t\t\tlevel = req.Arguments[1]\n\t\t\t}\n\n\t\t\t// Normalize aliases to the canonical \"*\" form\n\t\t\tif subsystem == allLogSubsystems || subsystem == allLogSubsystemsAlias {\n\t\t\t\tsubsystem = \"*\"\n\t\t\t}\n\t\t}\n\n\t\t// If a level is specified, then set the log level.\n\t\tif level != \"\" {\n\t\t\tif level == defaultLogLevel {\n\t\t\t\tlevel = logging.DefaultLevel().String()\n\t\t\t}\n\n\t\t\tif err := logging.SetLogLevel(subsystem, level); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\ts := fmt.Sprintf(\"Changed log level of '%s' to '%s'\\n\", subsystem, level)\n\t\t\tlog.Info(s)\n\n\t\t\treturn cmds.EmitOnce(res, &logLevelOutput{Message: s})\n\t\t}\n\n\t\t// Get the level for the requested subsystem.\n\t\tswitch subsystem {\n\t\tcase noSubsystemSpecified:\n\t\t\t// Return the default log level\n\t\t\tlevelMap := map[string]string{logging.DefaultName: logging.DefaultLevel().String()}\n\t\t\treturn cmds.EmitOnce(res, &logLevelOutput{Levels: levelMap})\n\t\tcase allLogSubsystems, allLogSubsystemsAlias:\n\t\t\t// Return levels for all subsystems (default behavior)\n\t\t\tlevels := logging.SubsystemLevelNames()\n\n\t\t\t// Replace default subsystem key with defaultSubsystemKey.\n\t\t\tlevels[defaultSubsystemKey] = levels[logging.DefaultName]\n\t\t\tdelete(levels, logging.DefaultName)\n\t\t\treturn cmds.EmitOnce(res, &logLevelOutput{Levels: levels})\n\t\tdefault:\n\t\t\t// Return level for a specific subsystem.\n\t\t\tlevel, err := logging.SubsystemLevelName(subsystem)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tlevelMap := map[string]string{subsystem: level}\n\t\t\treturn cmds.EmitOnce(res, &logLevelOutput{Levels: levelMap})\n\t\t}\n\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *logLevelOutput) error {\n\t\t\tif out.Message != \"\" {\n\t\t\t\tfmt.Fprint(w, out.Message)\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\t// Check if this is an RPC call by looking for the encoding option\n\t\t\tencoding, _ := req.Options[\"encoding\"].(string)\n\t\t\tisRPC := encoding == \"json\"\n\n\t\t\t// Determine whether to show subsystem names in output.\n\t\t\t// Show subsystem names when:\n\t\t\t// 1. It's an RPC call (needs JSON structure with named fields)\n\t\t\t// 2. Multiple subsystems are displayed (for clarity when showing many levels)\n\t\t\tshowNames := isRPC || len(out.Levels) > 1\n\n\t\t\tlevelNames := make([]string, 0, len(out.Levels))\n\t\t\tfor subsystem, level := range out.Levels {\n\t\t\t\tif showNames {\n\t\t\t\t\t// Show subsystem name when it's RPC or when showing multiple subsystems\n\t\t\t\t\tlevelNames = append(levelNames, fmt.Sprintf(\"%s: %s\", subsystem, level))\n\t\t\t\t} else {\n\t\t\t\t\t// For CLI calls with single subsystem, only show the level\n\t\t\t\t\tlevelNames = append(levelNames, level)\n\t\t\t\t}\n\t\t\t}\n\t\t\tslices.Sort(levelNames)\n\t\t\tfor _, ln := range levelNames {\n\t\t\t\tfmt.Fprintln(w, ln)\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: logLevelOutput{},\n}\n\nvar logLsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List the logging subsystems.\",\n\t\tShortDescription: `\n'ipfs log ls' is a utility command used to list the logging\nsubsystems of a running daemon.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\treturn cmds.EmitOnce(res, &stringList{logging.GetSubsystems()})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, list *stringList) error {\n\t\t\tfor _, s := range list.Strings {\n\t\t\t\tfmt.Fprintln(w, s)\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: stringList{},\n}\n\nvar logTailCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Read and output log messages.\",\n\t\tShortDescription: `\nOutputs log messages as they are generated.\n\nNOTE: --log-level requires the server to be logging at least at this level\n\nExample:\n\n GOLOG_LOG_LEVEL=\"error,bitswap=debug\" ipfs daemon\n ipfs log tail --log-level info\n\nThis will only return 'info' logs from bitswap and skip 'debug'.\n`,\n\t},\n\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(logLevelOption, \"Log level to listen to.\").WithDefault(\"\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tvar pipeReader *logging.PipeReader\n\t\tlogLevelString, _ := req.Options[logLevelOption].(string)\n\t\tif logLevelString != \"\" {\n\t\t\tlogLevel, err := logging.Parse(logLevelString)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"setting log level %s: %w\", logLevelString, err)\n\t\t\t}\n\t\t\tpipeReader = logging.NewPipeReader(logging.PipeLevel(logLevel))\n\t\t} else {\n\t\t\tpipeReader = logging.NewPipeReader()\n\t\t}\n\n\t\tgo func() {\n\t\t\t<-req.Context.Done()\n\t\t\tpipeReader.Close()\n\t\t}()\n\t\treturn res.Emit(pipeReader)\n\t},\n}\n" }, { "path": "core/commands/ls.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"slices\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tunixfs \"github.com/ipfs/boxo/ipld/unixfs\"\n\tunixfs_pb \"github.com/ipfs/boxo/ipld/unixfs/pb\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\n// LsLink contains printable data for a single ipld link in ls output\ntype LsLink struct {\n\tName, Hash string\n\tSize uint64\n\tType unixfs_pb.Data_DataType\n\tTarget string\n\tMode os.FileMode\n\tModTime time.Time\n}\n\n// LsObject is an element of LsOutput\n// It can represent all or part of a directory\ntype LsObject struct {\n\tHash string\n\tLinks []LsLink\n}\n\n// LsOutput is a set of printable data for directories,\n// it can be complete or partial\ntype LsOutput struct {\n\tObjects []LsObject\n}\n\nconst (\n\tlsHeadersOptionNameTime = \"headers\"\n\tlsResolveTypeOptionName = \"resolve-type\"\n\tlsSizeOptionName = \"size\"\n\tlsStreamOptionName = \"stream\"\n\tlsLongOptionName = \"long\"\n)\n\nvar LsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List directory contents for Unix filesystem objects.\",\n\t\tShortDescription: `\nDisplays the contents of an IPFS or IPNS object(s) at the given path, with\nthe following format:\n\n \n\nWith the --long (-l) option, display optional file mode (permissions) and\nmodification time in a format similar to Unix 'ls -l':\n\n \n\nMode and mtime are optional UnixFS metadata. They are only present if the\ncontent was imported with 'ipfs add --preserve-mode' and '--preserve-mtime'.\nWithout preserved metadata, both mode and mtime display '-'. Times are in UTC.\n\nExample with --long and preserved metadata:\n\n -rw-r--r-- QmZULkCELmmk5XNf... 1234 Jan 15 10:30 document.txt\n -rwxr-xr-x QmaRGe7bVmVaLmxb... 5678 Dec 01 2023 script.sh\n drwxr-xr-x QmWWEQhcLufF3qPm... - Nov 20 2023 subdir/\n\nExample with --long without preserved metadata:\n\n - QmZULkCELmmk5XNf... 1234 - document.txt\n\nThe JSON output contains type information.\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, true, \"The path to the IPFS object(s) to list links from.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(lsHeadersOptionNameTime, \"v\", \"Print table headers (Hash, Size, Name).\"),\n\t\tcmds.BoolOption(lsResolveTypeOptionName, \"Resolve linked objects to find out their types.\").WithDefault(true),\n\t\tcmds.BoolOption(lsSizeOptionName, \"Resolve linked objects to find out their file size.\").WithDefault(true),\n\t\tcmds.BoolOption(lsStreamOptionName, \"s\", \"Enable experimental streaming of directory entries as they are traversed.\"),\n\t\tcmds.BoolOption(lsLongOptionName, \"l\", \"Use a long listing format, showing file mode and modification time.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tresolveType, _ := req.Options[lsResolveTypeOptionName].(bool)\n\t\tresolveSize, _ := req.Options[lsSizeOptionName].(bool)\n\t\tstream, _ := req.Options[lsStreamOptionName].(bool)\n\n\t\terr = req.ParseBodyArgs()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tpaths := req.Arguments\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar processLink func(path string, link LsLink) error\n\t\tvar dirDone func(i int)\n\n\t\tprocessDir := func() (func(path string, link LsLink) error, func(i int)) {\n\t\t\treturn func(path string, link LsLink) error {\n\t\t\t\toutput := []LsObject{{\n\t\t\t\t\tHash: path,\n\t\t\t\t\tLinks: []LsLink{link},\n\t\t\t\t}}\n\t\t\t\treturn res.Emit(&LsOutput{output})\n\t\t\t}, func(i int) {}\n\t\t}\n\t\tdone := func() error { return nil }\n\n\t\tif !stream {\n\t\t\toutput := make([]LsObject, len(req.Arguments))\n\n\t\t\tprocessDir = func() (func(path string, link LsLink) error, func(i int)) {\n\t\t\t\t// for each dir\n\t\t\t\toutputLinks := make([]LsLink, 0)\n\t\t\t\treturn func(path string, link LsLink) error {\n\t\t\t\t\t\t// for each link\n\t\t\t\t\t\toutputLinks = append(outputLinks, link)\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}, func(i int) {\n\t\t\t\t\t\t// after each dir\n\t\t\t\t\t\tslices.SortFunc(outputLinks, func(a, b LsLink) int {\n\t\t\t\t\t\t\treturn strings.Compare(a.Name, b.Name)\n\t\t\t\t\t\t})\n\n\t\t\t\t\t\toutput[i] = LsObject{\n\t\t\t\t\t\t\tHash: paths[i],\n\t\t\t\t\t\t\tLinks: outputLinks,\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t}\n\n\t\t\tdone = func() error {\n\t\t\t\treturn cmds.EmitOnce(res, &LsOutput{output})\n\t\t\t}\n\t\t}\n\n\t\tlsCtx, cancel := context.WithCancel(req.Context)\n\t\tdefer cancel()\n\n\t\tfor i, fpath := range paths {\n\t\t\tpth, err := cmdutils.PathOrCidPath(fpath)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tresults := make(chan iface.DirEntry)\n\t\t\tlsErr := make(chan error, 1)\n\t\t\tgo func() {\n\t\t\t\tlsErr <- api.Unixfs().Ls(lsCtx, pth, results,\n\t\t\t\t\toptions.Unixfs.ResolveChildren(resolveSize || resolveType))\n\t\t\t}()\n\n\t\t\tprocessLink, dirDone = processDir()\n\t\t\tfor link := range results {\n\t\t\t\tvar ftype unixfs_pb.Data_DataType\n\t\t\t\tswitch link.Type {\n\t\t\t\tcase iface.TFile:\n\t\t\t\t\tftype = unixfs.TFile\n\t\t\t\tcase iface.TDirectory:\n\t\t\t\t\tftype = unixfs.TDirectory\n\t\t\t\tcase iface.TSymlink:\n\t\t\t\t\tftype = unixfs.TSymlink\n\t\t\t\t}\n\t\t\t\tlsLink := LsLink{\n\t\t\t\t\tName: link.Name,\n\t\t\t\t\tHash: enc.Encode(link.Cid),\n\n\t\t\t\t\tSize: link.Size,\n\t\t\t\t\tType: ftype,\n\t\t\t\t\tTarget: link.Target,\n\n\t\t\t\t\tMode: link.Mode,\n\t\t\t\t\tModTime: link.ModTime,\n\t\t\t\t}\n\t\t\t\tif err = processLink(paths[i], lsLink); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\tif err = <-lsErr; err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdirDone(i)\n\t\t}\n\t\treturn done()\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\treq := res.Request()\n\t\t\tlastObjectHash := \"\"\n\n\t\t\tfor {\n\t\t\t\tv, err := res.Next()\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err == io.EOF {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tout := v.(*LsOutput)\n\t\t\t\tlastObjectHash = tabularOutput(req, os.Stdout, out, lastObjectHash, false)\n\t\t\t}\n\t\t},\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *LsOutput) error {\n\t\t\t// when streaming over HTTP using a text encoder, we cannot render breaks\n\t\t\t// between directories because we don't know the hash of the last\n\t\t\t// directory encoder\n\t\t\tignoreBreaks, _ := req.Options[lsStreamOptionName].(bool)\n\t\t\ttabularOutput(req, w, out, \"\", ignoreBreaks)\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: LsOutput{},\n}\n\n// formatMode converts os.FileMode to a 10-character Unix ls-style string.\n//\n// Format: [type][owner rwx][group rwx][other rwx]\n//\n// Type indicators: - (regular), d (directory), l (symlink), p (named pipe),\n// s (socket), c (char device), b (block device).\n//\n// Special bits replace the execute position: setuid on owner (s/S),\n// setgid on group (s/S), sticky on other (t/T). Lowercase when the\n// underlying execute bit is also set, uppercase when not.\nfunc formatMode(mode os.FileMode) string {\n\tvar buf [10]byte\n\n\t// File type - handle all special file types like ls does\n\tswitch {\n\tcase mode&os.ModeDir != 0:\n\t\tbuf[0] = 'd'\n\tcase mode&os.ModeSymlink != 0:\n\t\tbuf[0] = 'l'\n\tcase mode&os.ModeNamedPipe != 0:\n\t\tbuf[0] = 'p'\n\tcase mode&os.ModeSocket != 0:\n\t\tbuf[0] = 's'\n\tcase mode&os.ModeDevice != 0:\n\t\tif mode&os.ModeCharDevice != 0 {\n\t\t\tbuf[0] = 'c'\n\t\t} else {\n\t\t\tbuf[0] = 'b'\n\t\t}\n\tdefault:\n\t\tbuf[0] = '-'\n\t}\n\n\t// Owner permissions (bits 8,7,6)\n\tbuf[1] = permBit(mode, 0400, 'r') // read\n\tbuf[2] = permBit(mode, 0200, 'w') // write\n\t// Handle setuid bit for owner execute\n\tif mode&os.ModeSetuid != 0 {\n\t\tif mode&0100 != 0 {\n\t\t\tbuf[3] = 's'\n\t\t} else {\n\t\t\tbuf[3] = 'S'\n\t\t}\n\t} else {\n\t\tbuf[3] = permBit(mode, 0100, 'x') // execute\n\t}\n\n\t// Group permissions (bits 5,4,3)\n\tbuf[4] = permBit(mode, 0040, 'r') // read\n\tbuf[5] = permBit(mode, 0020, 'w') // write\n\t// Handle setgid bit for group execute\n\tif mode&os.ModeSetgid != 0 {\n\t\tif mode&0010 != 0 {\n\t\t\tbuf[6] = 's'\n\t\t} else {\n\t\t\tbuf[6] = 'S'\n\t\t}\n\t} else {\n\t\tbuf[6] = permBit(mode, 0010, 'x') // execute\n\t}\n\n\t// Other permissions (bits 2,1,0)\n\tbuf[7] = permBit(mode, 0004, 'r') // read\n\tbuf[8] = permBit(mode, 0002, 'w') // write\n\t// Handle sticky bit for other execute\n\tif mode&os.ModeSticky != 0 {\n\t\tif mode&0001 != 0 {\n\t\t\tbuf[9] = 't'\n\t\t} else {\n\t\t\tbuf[9] = 'T'\n\t\t}\n\t} else {\n\t\tbuf[9] = permBit(mode, 0001, 'x') // execute\n\t}\n\n\treturn string(buf[:])\n}\n\n// permBit returns the permission character if the bit is set.\nfunc permBit(mode os.FileMode, bit os.FileMode, char byte) byte {\n\tif mode&bit != 0 {\n\t\treturn char\n\t}\n\treturn '-'\n}\n\n// formatModTime formats time.Time for display, following Unix ls conventions.\n//\n// Returns \"-\" for zero time. Otherwise returns a 12-character string:\n// recent files (within 6 months) show \"Jan 02 15:04\",\n// older or future files show \"Jan 02 2006\".\n//\n// The output uses the timezone embedded in t (UTC for IPFS metadata).\nfunc formatModTime(t time.Time) string {\n\tif t.IsZero() {\n\t\treturn \"-\"\n\t}\n\n\t// Format: \"Jan 02 15:04\" for times within the last 6 months\n\t// Format: \"Jan 02 2006\" for older times (similar to ls)\n\tnow := time.Now()\n\tsixMonthsAgo := now.AddDate(0, -6, 0)\n\n\tif t.After(sixMonthsAgo) && t.Before(now.Add(24*time.Hour)) {\n\t\treturn t.Format(\"Jan 02 15:04\")\n\t}\n\treturn t.Format(\"Jan 02 2006\")\n}\n\nfunc tabularOutput(req *cmds.Request, w io.Writer, out *LsOutput, lastObjectHash string, ignoreBreaks bool) string {\n\theaders, _ := req.Options[lsHeadersOptionNameTime].(bool)\n\tstream, _ := req.Options[lsStreamOptionName].(bool)\n\tsize, _ := req.Options[lsSizeOptionName].(bool)\n\tlong, _ := req.Options[lsLongOptionName].(bool)\n\n\t// in streaming mode we can't automatically align the tabs\n\t// so we take a best guess\n\tvar minTabWidth int\n\tif stream {\n\t\tminTabWidth = 10\n\t} else {\n\t\tminTabWidth = 1\n\t}\n\n\tmultipleFolders := len(req.Arguments) > 1\n\n\ttw := tabwriter.NewWriter(w, minTabWidth, 2, 1, ' ', 0)\n\n\tfor _, object := range out.Objects {\n\n\t\tif !ignoreBreaks && object.Hash != lastObjectHash {\n\t\t\tif multipleFolders {\n\t\t\t\tif lastObjectHash != \"\" {\n\t\t\t\t\tfmt.Fprintln(tw)\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(tw, \"%s:\\n\", object.Hash)\n\t\t\t}\n\t\t\tif headers {\n\t\t\t\tvar s string\n\t\t\t\tif long {\n\t\t\t\t\t// Long format: Mode Hash [Size] ModTime Name\n\t\t\t\t\tif size {\n\t\t\t\t\t\ts = \"Mode\\tHash\\tSize\\tModTime\\tName\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\ts = \"Mode\\tHash\\tModTime\\tName\"\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\t// Standard format: Hash [Size] Name\n\t\t\t\t\tif size {\n\t\t\t\t\t\ts = \"Hash\\tSize\\tName\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\ts = \"Hash\\tName\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tfmt.Fprintln(tw, s)\n\t\t\t}\n\t\t\tlastObjectHash = object.Hash\n\t\t}\n\n\t\tfor _, link := range object.Links {\n\t\t\tvar s string\n\t\t\tisDir := link.Type == unixfs.TDirectory || link.Type == unixfs.THAMTShard || link.Type == unixfs.TMetadata\n\n\t\t\tif long {\n\t\t\t\t// Long format: Mode Hash Size ModTime Name\n\t\t\t\tvar mode string\n\t\t\t\tif link.Mode == 0 {\n\t\t\t\t\t// No mode metadata preserved. Show \"-\" to indicate\n\t\t\t\t\t// \"not available\" rather than \"----------\" (mode 0000).\n\t\t\t\t\tmode = \"-\"\n\t\t\t\t} else {\n\t\t\t\t\tmode = formatMode(link.Mode)\n\t\t\t\t}\n\t\t\t\tmodTime := formatModTime(link.ModTime)\n\n\t\t\t\tif isDir {\n\t\t\t\t\tif size {\n\t\t\t\t\t\ts = \"%s\\t%s\\t-\\t%s\\t%s/\\n\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\ts = \"%s\\t%s\\t%s\\t%s/\\n\"\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(tw, s, mode, link.Hash, modTime, cmdenv.EscNonPrint(link.Name))\n\t\t\t\t} else {\n\t\t\t\t\tif size {\n\t\t\t\t\t\ts = \"%s\\t%s\\t%v\\t%s\\t%s\\n\"\n\t\t\t\t\t\tfmt.Fprintf(tw, s, mode, link.Hash, link.Size, modTime, cmdenv.EscNonPrint(link.Name))\n\t\t\t\t\t} else {\n\t\t\t\t\t\ts = \"%s\\t%s\\t%s\\t%s\\n\"\n\t\t\t\t\t\tfmt.Fprintf(tw, s, mode, link.Hash, modTime, cmdenv.EscNonPrint(link.Name))\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// Standard format: Hash [Size] Name\n\t\t\t\tswitch {\n\t\t\t\tcase isDir:\n\t\t\t\t\tif size {\n\t\t\t\t\t\ts = \"%[1]s\\t-\\t%[3]s/\\n\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\ts = \"%[1]s\\t%[3]s/\\n\"\n\t\t\t\t\t}\n\t\t\t\tdefault:\n\t\t\t\t\tif size {\n\t\t\t\t\t\ts = \"%s\\t%v\\t%s\\n\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\ts = \"%[1]s\\t%[3]s\\n\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(tw, s, link.Hash, link.Size, cmdenv.EscNonPrint(link.Name))\n\t\t\t}\n\t\t}\n\t}\n\ttw.Flush()\n\treturn lastObjectHash\n}\n" }, { "path": "core/commands/ls_test.go", "content": "package commands\n\nimport (\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestFormatMode(t *testing.T) {\n\tt.Parallel()\n\n\ttests := []struct {\n\t\tname string\n\t\tmode os.FileMode\n\t\texpected string\n\t}{\n\t\t// File types\n\t\t{\n\t\t\tname: \"regular file with rw-r--r--\",\n\t\t\tmode: 0644,\n\t\t\texpected: \"-rw-r--r--\",\n\t\t},\n\t\t{\n\t\t\tname: \"regular file with rwxr-xr-x\",\n\t\t\tmode: 0755,\n\t\t\texpected: \"-rwxr-xr-x\",\n\t\t},\n\t\t{\n\t\t\tname: \"regular file with no permissions\",\n\t\t\tmode: 0,\n\t\t\texpected: \"----------\",\n\t\t},\n\t\t{\n\t\t\tname: \"regular file with full permissions\",\n\t\t\tmode: 0777,\n\t\t\texpected: \"-rwxrwxrwx\",\n\t\t},\n\t\t{\n\t\t\tname: \"directory with rwxr-xr-x\",\n\t\t\tmode: os.ModeDir | 0755,\n\t\t\texpected: \"drwxr-xr-x\",\n\t\t},\n\t\t{\n\t\t\tname: \"directory with rwx------\",\n\t\t\tmode: os.ModeDir | 0700,\n\t\t\texpected: \"drwx------\",\n\t\t},\n\t\t{\n\t\t\tname: \"symlink with rwxrwxrwx\",\n\t\t\tmode: os.ModeSymlink | 0777,\n\t\t\texpected: \"lrwxrwxrwx\",\n\t\t},\n\t\t{\n\t\t\tname: \"named pipe with rw-r--r--\",\n\t\t\tmode: os.ModeNamedPipe | 0644,\n\t\t\texpected: \"prw-r--r--\",\n\t\t},\n\t\t{\n\t\t\tname: \"socket with rw-rw-rw-\",\n\t\t\tmode: os.ModeSocket | 0666,\n\t\t\texpected: \"srw-rw-rw-\",\n\t\t},\n\t\t{\n\t\t\tname: \"block device with rw-rw----\",\n\t\t\tmode: os.ModeDevice | 0660,\n\t\t\texpected: \"brw-rw----\",\n\t\t},\n\t\t{\n\t\t\tname: \"character device with rw-rw-rw-\",\n\t\t\tmode: os.ModeDevice | os.ModeCharDevice | 0666,\n\t\t\texpected: \"crw-rw-rw-\",\n\t\t},\n\n\t\t// Special permission bits - setuid\n\t\t{\n\t\t\tname: \"setuid with execute\",\n\t\t\tmode: os.ModeSetuid | 0755,\n\t\t\texpected: \"-rwsr-xr-x\",\n\t\t},\n\t\t{\n\t\t\tname: \"setuid without execute\",\n\t\t\tmode: os.ModeSetuid | 0644,\n\t\t\texpected: \"-rwSr--r--\",\n\t\t},\n\n\t\t// Special permission bits - setgid\n\t\t{\n\t\t\tname: \"setgid with execute\",\n\t\t\tmode: os.ModeSetgid | 0755,\n\t\t\texpected: \"-rwxr-sr-x\",\n\t\t},\n\t\t{\n\t\t\tname: \"setgid without execute\",\n\t\t\tmode: os.ModeSetgid | 0745,\n\t\t\texpected: \"-rwxr-Sr-x\",\n\t\t},\n\n\t\t// Special permission bits - sticky\n\t\t{\n\t\t\tname: \"sticky with execute\",\n\t\t\tmode: os.ModeSticky | 0755,\n\t\t\texpected: \"-rwxr-xr-t\",\n\t\t},\n\t\t{\n\t\t\tname: \"sticky without execute\",\n\t\t\tmode: os.ModeSticky | 0754,\n\t\t\texpected: \"-rwxr-xr-T\",\n\t\t},\n\n\t\t// Combined special bits\n\t\t{\n\t\t\tname: \"setuid + setgid + sticky all with execute\",\n\t\t\tmode: os.ModeSetuid | os.ModeSetgid | os.ModeSticky | 0777,\n\t\t\texpected: \"-rwsrwsrwt\",\n\t\t},\n\t\t{\n\t\t\tname: \"setuid + setgid + sticky none with execute\",\n\t\t\tmode: os.ModeSetuid | os.ModeSetgid | os.ModeSticky | 0666,\n\t\t\texpected: \"-rwSrwSrwT\",\n\t\t},\n\n\t\t// Directory with special bits\n\t\t{\n\t\t\tname: \"directory with sticky bit\",\n\t\t\tmode: os.ModeDir | os.ModeSticky | 0755,\n\t\t\texpected: \"drwxr-xr-t\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresult := formatMode(tc.mode)\n\t\t\tassert.Equal(t, tc.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestFormatModTime(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"zero time returns dash\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresult := formatModTime(time.Time{})\n\t\tassert.Equal(t, \"-\", result)\n\t})\n\n\tt.Run(\"old time shows year format\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Use a time clearly in the past (more than 6 months ago)\n\t\toldTime := time.Date(2020, time.March, 15, 10, 30, 0, 0, time.UTC)\n\t\tresult := formatModTime(oldTime)\n\t\t// Format: \"Jan 02 2006\" (note: two spaces before year)\n\t\tassert.Equal(t, \"Mar 15 2020\", result)\n\t})\n\n\tt.Run(\"very old time shows year format\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tveryOldTime := time.Date(2000, time.January, 1, 0, 0, 0, 0, time.UTC)\n\t\tresult := formatModTime(veryOldTime)\n\t\tassert.Equal(t, \"Jan 01 2000\", result)\n\t})\n\n\tt.Run(\"future time shows year format\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Times more than 24h in the future should show year format\n\t\tfutureTime := time.Now().AddDate(1, 0, 0)\n\t\tresult := formatModTime(futureTime)\n\t\t// Should contain the future year\n\t\tassert.Contains(t, result, \" \") // two spaces before year\n\t\tassert.Regexp(t, `^[A-Z][a-z]{2} \\d{2} \\d{4}$`, result) // matches \"Mon DD YYYY\"\n\t\tassert.Contains(t, result, futureTime.Format(\"2006\")) // contains the year\n\t})\n\n\tt.Run(\"format lengths are consistent\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Both formats should produce 12-character strings for alignment\n\t\toldTime := time.Date(2020, time.March, 15, 10, 30, 0, 0, time.UTC)\n\t\toldResult := formatModTime(oldTime)\n\t\tassert.Len(t, oldResult, 12, \"old time format should be 12 chars\")\n\n\t\t// Recent time: use 1 month ago to ensure it's always within the 6-month window\n\t\trecentTime := time.Now().AddDate(0, -1, 0)\n\t\trecentResult := formatModTime(recentTime)\n\t\tassert.Len(t, recentResult, 12, \"recent time format should be 12 chars\")\n\t})\n}\n" }, { "path": "core/commands/mount_nofuse.go", "content": "//go:build !windows && nofuse\n\npackage commands\n\nimport (\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nvar MountCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Mounts ipfs to the filesystem (disabled).\",\n\t\tShortDescription: `\nThis version of ipfs is compiled without fuse support, which is required\nfor mounting. If you'd like to be able to mount, please use a version of\nKubo compiled with fuse.\n\nFor the latest instructions, please check the project's repository:\n http://github.com/ipfs/kubo\n https://github.com/ipfs/kubo/blob/master/docs/fuse.md\n`,\n\t},\n}\n" }, { "path": "core/commands/mount_unix.go", "content": "//go:build !windows && !nofuse\n\npackage commands\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tnodeMount \"github.com/ipfs/kubo/fuse/node\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tconfig \"github.com/ipfs/kubo/config\"\n)\n\nconst (\n\tmountIPFSPathOptionName = \"ipfs-path\"\n\tmountIPNSPathOptionName = \"ipns-path\"\n\tmountMFSPathOptionName = \"mfs-path\"\n)\n\nvar MountCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Mounts IPFS to the filesystem (read-only).\",\n\t\tShortDescription: `\nMount IPFS at a read-only mountpoint on the OS (default: /ipfs, /ipns, /mfs).\nAll IPFS objects will be accessible under that directory. Note that the\nroot will not be listable, as it is virtual. Access known paths directly.\n\nYou may have to create /ipfs and /ipns before using 'ipfs mount':\n\n> sudo mkdir /ipfs /ipns /mfs\n> sudo chown $(whoami) /ipfs /ipns /mfs\n> ipfs daemon &\n> ipfs mount\n`,\n\t\tLongDescription: `\nMount IPFS at a read-only mountpoint on the OS. The default, /ipfs and /ipns,\nare set in the configuration file, but can be overridden by the options.\nAll IPFS objects will be accessible under this directory. Note that the\nroot will not be listable, as it is virtual. Access known paths directly.\n\nYou may have to create /ipfs and /ipns before using 'ipfs mount':\n\n> sudo mkdir /ipfs /ipns /mfs\n> sudo chown $(whoami) /ipfs /ipns /mfs\n> ipfs daemon &\n> ipfs mount\n\nExample:\n\n# setup\n> mkdir foo\n> echo \"baz\" > foo/bar\n> ipfs add -r foo\nadded QmWLdkp93sNxGRjnFHPaYg8tCQ35NBY3XPn6KiETd3Z4WR foo/bar\nadded QmSh5e7S6fdcu75LAbXNZAFY2nGyZUJXyLCJDvn2zRkWyC foo\n> ipfs ls QmSh5e7S6fdcu75LAbXNZAFY2nGyZUJXyLCJDvn2zRkWyC\nQmWLdkp93sNxGRjnFHPaYg8tCQ35NBY3XPn6KiETd3Z4WR 12 bar\n> ipfs cat QmWLdkp93sNxGRjnFHPaYg8tCQ35NBY3XPn6KiETd3Z4WR\nbaz\n\n# mount\n> ipfs daemon &\n> ipfs mount\nIPFS mounted at: /ipfs\nIPNS mounted at: /ipns\nMFS mounted at: /mfs\n> cd /ipfs/QmSh5e7S6fdcu75LAbXNZAFY2nGyZUJXyLCJDvn2zRkWyC\n> ls\nbar\n> cat bar\nbaz\n> cat /ipfs/QmSh5e7S6fdcu75LAbXNZAFY2nGyZUJXyLCJDvn2zRkWyC/bar\nbaz\n> cat /ipfs/QmWLdkp93sNxGRjnFHPaYg8tCQ35NBY3XPn6KiETd3Z4WR\nbaz\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(mountIPFSPathOptionName, \"f\", \"The path where IPFS should be mounted.\"),\n\t\tcmds.StringOption(mountIPNSPathOptionName, \"n\", \"The path where IPNS should be mounted.\"),\n\t\tcmds.StringOption(mountMFSPathOptionName, \"m\", \"The path where MFS should be mounted.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfg, err := env.(*oldcmds.Context).GetConfig()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// error if we aren't running node in online mode\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tfsdir, found := req.Options[mountIPFSPathOptionName].(string)\n\t\tif !found {\n\t\t\tfsdir = cfg.Mounts.IPFS // use default value\n\t\t}\n\n\t\t// get default mount points\n\t\tnsdir, found := req.Options[mountIPNSPathOptionName].(string)\n\t\tif !found {\n\t\t\tnsdir = cfg.Mounts.IPNS // NB: be sure to not redeclare!\n\t\t}\n\n\t\tmfsdir, found := req.Options[mountMFSPathOptionName].(string)\n\t\tif !found {\n\t\t\tmfsdir = cfg.Mounts.MFS\n\t\t}\n\n\t\terr = nodeMount.Mount(nd, fsdir, nsdir, mfsdir)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar output config.Mounts\n\t\toutput.IPFS = fsdir\n\t\toutput.IPNS = nsdir\n\t\toutput.MFS = mfsdir\n\t\treturn cmds.EmitOnce(res, &output)\n\t},\n\tType: config.Mounts{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, mounts *config.Mounts) error {\n\t\t\tfmt.Fprintf(w, \"IPFS mounted at: %s\\n\", cmdenv.EscNonPrint(mounts.IPFS))\n\t\t\tfmt.Fprintf(w, \"IPNS mounted at: %s\\n\", cmdenv.EscNonPrint(mounts.IPNS))\n\t\t\tfmt.Fprintf(w, \"MFS mounted at: %s\\n\", cmdenv.EscNonPrint(mounts.MFS))\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/mount_windows.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nvar MountCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Not yet implemented on Windows.\",\n\t\tShortDescription: \"Not yet implemented on Windows. :(\",\n\t},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\treturn errors.New(\"Mount isn't compatible with Windows yet\")\n\t},\n}\n" }, { "path": "core/commands/multibase.go", "content": "package commands\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nvar MbaseCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Encode and decode files or stdin with multibase format\",\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"encode\": mbaseEncodeCmd,\n\t\t\"decode\": mbaseDecodeCmd,\n\t\t\"transcode\": mbaseTranscodeCmd,\n\t\t\"list\": basesCmd,\n\t},\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n}\n\nconst (\n\tmbaseOptionName = \"b\"\n)\n\nvar mbaseEncodeCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Encode data into multibase string\",\n\t\tLongDescription: `\nThis command expects a file name or data provided via stdin.\n\nBy default it will use URL-safe base64url encoding,\nbut one can customize used base with -b:\n\n > echo hello | ipfs multibase encode -b base16 > output_file\n > cat output_file\n f68656c6c6f0a\n\n > echo hello > input_file\n > ipfs multibase encode -b base16 input_file\n f68656c6c6f0a\n `,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"file\", true, false, \"data to encode\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(mbaseOptionName, \"multibase encoding\").WithDefault(\"base64url\"),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tencoderName, _ := req.Options[mbaseOptionName].(string)\n\t\tencoder, err := mbase.EncoderByName(encoderName)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tfiles := req.Files.Entries()\n\t\tfile, err := cmdenv.GetFileArg(files)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to access file: %w\", err)\n\t\t}\n\t\tbuf, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to read file contents: %w\", err)\n\t\t}\n\t\tencoded := encoder.Encode(buf)\n\t\treader := strings.NewReader(encoded)\n\t\treturn resp.Emit(reader)\n\t},\n}\n\nvar mbaseDecodeCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Decode multibase string\",\n\t\tLongDescription: `\nThis command expects multibase inside of a file or via stdin:\n\n > echo -n hello | ipfs multibase encode -b base16 > file\n > cat file\n f68656c6c6f\n\n > ipfs multibase decode file\n hello\n\n > cat file | ipfs multibase decode\n hello\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"encoded_file\", true, false, \"encoded data to decode\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tfiles := req.Files.Entries()\n\t\tfile, err := cmdenv.GetFileArg(files)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to access file: %w\", err)\n\t\t}\n\t\tencodedData, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to read file contents: %w\", err)\n\t\t}\n\t\t_, data, err := mbase.Decode(string(encodedData))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to decode multibase: %w\", err)\n\t\t}\n\t\treader := bytes.NewReader(data)\n\t\treturn resp.Emit(reader)\n\t},\n}\n\nvar mbaseTranscodeCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Transcode multibase string between bases\",\n\t\tLongDescription: `\nThis command expects multibase inside of a file or via stdin.\n\nBy default it will use URL-safe base64url encoding,\nbut one can customize used base with -b:\n\n > echo -n hello | ipfs multibase encode > file\n > cat file\n uaGVsbG8\n\n > ipfs multibase transcode file -b base16 > transcoded_file\n > cat transcoded_file\n f68656c6c6f\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"encoded_file\", true, false, \"encoded data to decode\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(mbaseOptionName, \"multibase encoding\").WithDefault(\"base64url\"),\n\t},\n\tRun: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tencoderName, _ := req.Options[mbaseOptionName].(string)\n\t\tencoder, err := mbase.EncoderByName(encoderName)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tfiles := req.Files.Entries()\n\t\tfile, err := cmdenv.GetFileArg(files)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to access file: %w\", err)\n\t\t}\n\t\tencodedData, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to read file contents: %w\", err)\n\t\t}\n\t\t_, data, err := mbase.Decode(string(encodedData))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to decode multibase: %w\", err)\n\t\t}\n\t\tencoded := encoder.Encode(data)\n\t\treader := strings.NewReader(encoded)\n\t\treturn resp.Emit(reader)\n\t},\n}\n" }, { "path": "core/commands/name/ipns.go", "content": "package name\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/path\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nvar log = logging.Logger(\"core/commands/ipns\")\n\ntype ResolvedPath struct {\n\tPath string\n}\n\nconst (\n\trecursiveOptionName = \"recursive\"\n\tnocacheOptionName = \"nocache\"\n\tdhtRecordCountOptionName = \"dht-record-count\"\n\tdhtTimeoutOptionName = \"dht-timeout\"\n\tstreamOptionName = \"stream\"\n)\n\nvar IpnsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Resolve IPNS names.\",\n\t\tShortDescription: `\nIPNS is a PKI namespace, where names are the hashes of public keys, and\nthe private key enables publishing new (signed) values. In both publish\nand resolve, the default name used is the node's own PeerID,\nwhich is the hash of its public key.\n`,\n\t\tLongDescription: `\nIPNS is a PKI namespace, where names are the hashes of public keys, and\nthe private key enables publishing new (signed) values. In both publish\nand resolve, the default name used is the node's own PeerID,\nwhich is the hash of its public key.\n\nYou can use the 'ipfs key' commands to list and generate more names and their\nrespective keys.\n\nExamples:\n\nResolve the value of your name:\n\n > ipfs name resolve\n /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n\nResolve the value of another name:\n\n > ipfs name resolve QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\n /ipfs/QmSiTko9JZyabH56y2fussEt1A5oDqsFXB3CkvAqraFryz\n\nResolve the value of a dnslink:\n\n > ipfs name resolve ipfs.io\n /ipfs/QmaBvfZooxWkrv7D3r8LS9moNjzD2o525XMZze69hhoxf5\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", false, false, \"The IPNS name to resolve. Defaults to your node's peerID.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(recursiveOptionName, \"r\", \"Resolve until the result is not an IPNS name.\").WithDefault(true),\n\t\tcmds.BoolOption(nocacheOptionName, \"n\", \"Do not use cached entries.\"),\n\t\tcmds.UintOption(dhtRecordCountOptionName, \"dhtrc\", \"Number of records to request for DHT resolution.\").WithDefault(uint(namesys.DefaultResolverDhtRecordCount)),\n\t\tcmds.StringOption(dhtTimeoutOptionName, \"dhtt\", \"Max time to collect values during DHT resolution e.g. \\\"30s\\\". Pass 0 for no timeout.\").WithDefault(namesys.DefaultResolverDhtTimeout.String()),\n\t\tcmds.BoolOption(streamOptionName, \"s\", \"Stream entries as they are found.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnocache, _ := req.Options[\"nocache\"].(bool)\n\n\t\tvar name string\n\t\tif len(req.Arguments) == 0 {\n\t\t\tself, err := api.Key().Self(req.Context)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tname = self.ID().String()\n\t\t} else {\n\t\t\tname = req.Arguments[0]\n\t\t}\n\n\t\trecursive, _ := req.Options[recursiveOptionName].(bool)\n\t\trc, rcok := req.Options[dhtRecordCountOptionName].(uint)\n\t\tdhtt, dhttok := req.Options[dhtTimeoutOptionName].(string)\n\t\tstream, _ := req.Options[streamOptionName].(bool)\n\n\t\topts := []options.NameResolveOption{\n\t\t\toptions.Name.Cache(!nocache),\n\t\t}\n\n\t\tif !recursive {\n\t\t\topts = append(opts, options.Name.ResolveOption(namesys.ResolveWithDepth(1)))\n\t\t}\n\t\tif rcok {\n\t\t\topts = append(opts, options.Name.ResolveOption(namesys.ResolveWithDhtRecordCount(rc)))\n\t\t}\n\t\tif dhttok {\n\t\t\td, err := time.ParseDuration(dhtt)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif d < 0 {\n\t\t\t\treturn errors.New(\"DHT timeout value must be >= 0\")\n\t\t\t}\n\t\t\topts = append(opts, options.Name.ResolveOption(namesys.ResolveWithDhtTimeout(d)))\n\t\t}\n\n\t\tif !strings.HasPrefix(name, \"/ipns/\") {\n\t\t\tname = \"/ipns/\" + name\n\t\t}\n\n\t\tif !stream {\n\t\t\toutput, err := api.Name().Resolve(req.Context, name, opts...)\n\t\t\tif err != nil && (recursive || err != namesys.ErrResolveRecursion) {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tpth, err := path.NewPath(output.String())\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\treturn cmds.EmitOnce(res, &ResolvedPath{pth.String()})\n\t\t}\n\n\t\toutput, err := api.Name().Search(req.Context, name, opts...)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor v := range output {\n\t\t\tif v.Err != nil && (recursive || v.Err != namesys.ErrResolveRecursion) {\n\t\t\t\treturn v.Err\n\t\t\t}\n\t\t\tif err := res.Emit(&ResolvedPath{v.Path.String()}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t}\n\n\t\treturn nil\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, rp *ResolvedPath) error {\n\t\t\t_, err := fmt.Fprintln(w, rp.Path)\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: ResolvedPath{},\n}\n" }, { "path": "core/commands/name/ipnsps.go", "content": "package name\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tke \"github.com/ipfs/kubo/core/commands/keyencode\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\ntype ipnsPubsubState struct {\n\tEnabled bool\n}\n\ntype ipnsPubsubCancel struct {\n\tCanceled bool\n}\n\ntype stringList struct {\n\tStrings []string\n}\n\n// IpnsPubsubCmd is the subcommand that allows us to manage the IPNS pubsub system\nvar IpnsPubsubCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"IPNS pubsub management\",\n\t\tShortDescription: `\nManage and inspect the state of the IPNS pubsub resolver.\n\nNote: this command is experimental and subject to change as the system is refined\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"state\": ipnspsStateCmd,\n\t\t\"subs\": ipnspsSubsCmd,\n\t\t\"cancel\": ipnspsCancelCmd,\n\t},\n}\n\nvar ipnspsStateCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Query the state of IPNS pubsub.\",\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &ipnsPubsubState{n.PSRouter != nil})\n\t},\n\tType: ipnsPubsubState{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ips *ipnsPubsubState) error {\n\t\t\tvar state string\n\t\t\tif ips.Enabled {\n\t\t\t\tstate = \"enabled\"\n\t\t\t} else {\n\t\t\t\tstate = \"disabled\"\n\t\t\t}\n\n\t\t\t_, err := fmt.Fprintln(w, state)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\nvar ipnspsSubsCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show current name subscriptions.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tke.OptionIPNSBase,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tkeyEnc, err := ke.KeyEncoderFromString(req.Options[ke.OptionIPNSBase.Name()].(string))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif n.PSRouter == nil {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"IPNS pubsub subsystem is not enabled\")\n\t\t}\n\t\tvar paths []string\n\t\tfor _, key := range n.PSRouter.GetSubscriptions() {\n\t\t\tns, k, err := record.SplitKey(key)\n\t\t\tif err != nil || ns != \"ipns\" {\n\t\t\t\t// Not necessarily an error.\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpid, err := peer.IDFromBytes([]byte(k))\n\t\t\tif err != nil {\n\t\t\t\tlog.Errorf(\"ipns key not a valid peer ID: %s\", err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpaths = append(paths, \"/ipns/\"+keyEnc.FormatID(pid))\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &stringList{paths})\n\t},\n\tType: stringList{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: stringListEncoder(),\n\t},\n}\n\nvar ipnspsCancelCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Cancel a name subscription.\",\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif n.PSRouter == nil {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"IPNS pubsub subsystem is not enabled\")\n\t\t}\n\n\t\tname := req.Arguments[0]\n\t\tname = strings.TrimPrefix(name, \"/ipns/\")\n\t\tpid, err := peer.Decode(name)\n\t\tif err != nil {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"not a valid IPNS name: %s\", err)\n\t\t}\n\n\t\tok, err := n.PSRouter.Cancel(\"/ipns/\" + string(pid))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn cmds.EmitOnce(res, &ipnsPubsubCancel{ok})\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"Name to cancel the subscription for.\"),\n\t},\n\tType: ipnsPubsubCancel{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ipc *ipnsPubsubCancel) error {\n\t\t\tvar state string\n\t\t\tif ipc.Canceled {\n\t\t\t\tstate = \"canceled\"\n\t\t\t} else {\n\t\t\t\tstate = \"no subscription\"\n\t\t\t}\n\n\t\t\t_, err := fmt.Fprintln(w, state)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\nfunc stringListEncoder() cmds.EncoderFunc {\n\treturn cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, list *stringList) error {\n\t\tfor _, s := range list.Strings {\n\t\t\t_, err := fmt.Fprintln(w, s)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t})\n}\n" }, { "path": "core/commands/name/name.go", "content": "package name\n\nimport (\n\t\"bytes\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\tipns_pb \"github.com/ipfs/boxo/ipns/pb\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"google.golang.org/protobuf/proto\"\n)\n\ntype IpnsEntry struct {\n\tName string\n\tValue string\n}\n\nvar NameCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Publish and resolve IPNS names.\",\n\t\tShortDescription: `\nIPNS is a PKI namespace, where names are the hashes of public keys, and\nthe private key enables publishing new (signed) values. In both publish\nand resolve, the default name used is the node's own PeerID,\nwhich is the hash of its public key.\n`,\n\t\tLongDescription: `\nIPNS is a PKI namespace, where names are the hashes of public keys, and\nthe private key enables publishing new (signed) values. In both publish\nand resolve, the default name used is the node's own PeerID,\nwhich is the hash of its public key.\n\nYou can use the 'ipfs key' commands to list and generate more names and their\nrespective keys.\n\nExamples:\n\nPublish an with your default name:\n\n > ipfs name publish /ipfs/bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4\n Published to k51qzi5uqu5dgklc20hksmmzhoy5lfrn5xcnryq6xp4r50b5yc0vnivpywfu9p: /ipfs/bafk...\n\nPublish an with another name, added by an 'ipfs key' command:\n\n > ipfs key gen --type=ed25519 mykey\n k51qzi5uqu5dlz49qkb657myg6f1buu6rauv8c6b489a9i1e4dkt7a3yo9j2wr\n > ipfs name publish --key=mykey /ipfs/bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4\n Published to k51qzi5uqu5dlz49qkb657myg6f1buu6rauv8c6b489a9i1e4dkt7a3yo9j2wr: /ipfs/bafk...\n\nResolve the value of your name:\n\n > ipfs name resolve\n /ipfs/bafk...\n\nResolve the value of another name:\n\n > ipfs name resolve k51qzi5uqu5dlz49qkb657myg6f1buu6rauv8c6b489a9i1e4dkt7a3yo9j2wr\n /ipfs/bafk...\n\nResolve the value of a dnslink:\n\n > ipfs name resolve specs.ipfs.tech\n /ipfs/bafy...\n\n`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"publish\": PublishCmd,\n\t\t\"resolve\": IpnsCmd,\n\t\t\"pubsub\": IpnsPubsubCmd,\n\t\t\"inspect\": IpnsInspectCmd,\n\t\t\"get\": IpnsGetCmd,\n\t\t\"put\": IpnsPutCmd,\n\t},\n}\n\ntype IpnsInspectValidation struct {\n\tValid bool\n\tReason string\n\tName string\n}\n\n// IpnsInspectEntry contains the deserialized values from an IPNS Entry:\n// https://github.com/ipfs/specs/blob/main/ipns/IPNS.md#record-serialization-format\ntype IpnsInspectEntry struct {\n\tValue string\n\tValidityType *ipns.ValidityType\n\tValidity *time.Time\n\tSequence *uint64\n\tTTL *time.Duration\n}\n\ntype IpnsInspectResult struct {\n\tEntry IpnsInspectEntry\n\tPbSize int\n\tSignatureType string\n\tHexDump string\n\tValidation *IpnsInspectValidation\n}\n\nvar IpnsInspectCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Inspects an IPNS Record\",\n\t\tShortDescription: `\nPrints values inside of IPNS Record protobuf and its DAG-CBOR Data field.\nPassing --verify will verify signature against provided public key.\n`,\n\t\tLongDescription: `\nPrints values inside of IPNS Record protobuf and its DAG-CBOR Data field.\n\nThe input can be a file or STDIN, the output can be JSON:\n\n $ ipfs routing get \"/ipns/$PEERID\" > ipns_record\n $ ipfs name inspect --enc=json < ipns_record\n\nValues in PublicKey, SignatureV1 and SignatureV2 fields are raw bytes encoded\nin Multibase. The Data field is DAG-CBOR represented as DAG-JSON.\n\nPassing --verify will verify signature against provided public key.\n\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tDescription: \"Request body should be `multipart/form-data` with the IPNS record bytes.\",\n\t\t},\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.FileArg(\"record\", true, false, \"The IPNS record payload to be verified.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(\"verify\", \"CID of the public IPNS key to validate against.\"),\n\t\tcmds.BoolOption(\"dump\", \"Include a full hex dump of the raw Protobuf record.\").WithDefault(true),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\tvar b bytes.Buffer\n\n\t\t_, err = io.Copy(&b, file)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trec, err := ipns.UnmarshalRecord(b.Bytes())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tresult := &IpnsInspectResult{\n\t\t\tEntry: IpnsInspectEntry{},\n\t\t}\n\n\t\t// Best effort to get the fields. Show everything we can.\n\t\tif v, err := rec.Value(); err == nil {\n\t\t\tresult.Entry.Value = v.String()\n\t\t}\n\n\t\tif v, err := rec.ValidityType(); err == nil {\n\t\t\tresult.Entry.ValidityType = &v\n\t\t}\n\n\t\tif v, err := rec.Validity(); err == nil {\n\t\t\tresult.Entry.Validity = &v\n\t\t}\n\n\t\tif v, err := rec.Sequence(); err == nil {\n\t\t\tresult.Entry.Sequence = &v\n\t\t}\n\n\t\tif v, err := rec.TTL(); err == nil {\n\t\t\tresult.Entry.TTL = &v\n\t\t}\n\n\t\t// Here we need the raw protobuf just to decide the version.\n\t\tvar pbRecord ipns_pb.IpnsRecord\n\t\terr = proto.Unmarshal(b.Bytes(), &pbRecord)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif len(pbRecord.SignatureV1) != 0 || len(pbRecord.Value) != 0 {\n\t\t\tresult.SignatureType = \"V1+V2\"\n\t\t} else if pbRecord.Data != nil {\n\t\t\tresult.SignatureType = \"V2\"\n\t\t} else {\n\t\t\tresult.SignatureType = \"Unknown\"\n\t\t}\n\t\tresult.PbSize = proto.Size(&pbRecord)\n\n\t\tif verify, ok := req.Options[\"verify\"].(string); ok {\n\t\t\tname, err := ipns.NameFromString(verify)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tresult.Validation = &IpnsInspectValidation{\n\t\t\t\tName: name.String(),\n\t\t\t}\n\n\t\t\terr = ipns.ValidateWithName(rec, name)\n\t\t\tif err == nil {\n\t\t\t\tresult.Validation.Valid = true\n\t\t\t} else {\n\t\t\t\tresult.Validation.Reason = err.Error()\n\t\t\t}\n\t\t}\n\n\t\tif dump, ok := req.Options[\"dump\"].(bool); ok && dump {\n\t\t\tresult.HexDump = hex.Dump(b.Bytes())\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, result)\n\t},\n\tType: IpnsInspectResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *IpnsInspectResult) error {\n\t\t\ttw := tabwriter.NewWriter(w, 0, 0, 1, ' ', 0)\n\t\t\tdefer tw.Flush()\n\n\t\t\tif out.Entry.Value != \"\" {\n\t\t\t\tfmt.Fprintf(tw, \"Value:\\t%q\\n\", out.Entry.Value)\n\t\t\t}\n\n\t\t\tif out.Entry.ValidityType != nil {\n\t\t\t\tfmt.Fprintf(tw, \"Validity Type:\\t%d\\n\", *out.Entry.ValidityType)\n\t\t\t}\n\n\t\t\tif out.Entry.Validity != nil {\n\t\t\t\tfmt.Fprintf(tw, \"Validity:\\t%q\\n\", out.Entry.Validity.Format(time.RFC3339Nano))\n\t\t\t}\n\n\t\t\tif out.Entry.Sequence != nil {\n\t\t\t\tfmt.Fprintf(tw, \"Sequence:\\t%d\\n\", *out.Entry.Sequence)\n\t\t\t}\n\n\t\t\tif out.Entry.TTL != nil {\n\t\t\t\tfmt.Fprintf(tw, \"TTL:\\t%s\\n\", out.Entry.TTL.String())\n\t\t\t}\n\n\t\t\tfmt.Fprintf(tw, \"Protobuf Size:\\t%d\\n\", out.PbSize)\n\t\t\tfmt.Fprintf(tw, \"Signature Type:\\t%s\\n\", out.SignatureType)\n\n\t\t\tif out.Validation == nil {\n\t\t\t\ttw.Flush()\n\t\t\t\tfmt.Fprintf(w, \"\\nThis record was not verified. Pass '--verify k51...' to verify.\\n\")\n\t\t\t} else {\n\t\t\t\ttw.Flush()\n\t\t\t\tfmt.Fprintf(w, \"\\nValidation results:\\n\")\n\n\t\t\t\tfmt.Fprintf(tw, \"\\tValid:\\t%v\\n\", out.Validation.Valid)\n\t\t\t\tif out.Validation.Reason != \"\" {\n\t\t\t\t\tfmt.Fprintf(tw, \"\\tReason:\\t%s\\n\", out.Validation.Reason)\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(tw, \"\\tName:\\t%s\\n\", out.Validation.Name)\n\t\t\t}\n\n\t\t\tif out.HexDump != \"\" {\n\t\t\t\ttw.Flush()\n\n\t\t\t\tfmt.Fprintf(w, \"\\nHex Dump:\\n%s\", out.HexDump)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar IpnsGetCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Retrieve a signed IPNS record.\",\n\t\tShortDescription: `\nRetrieves the signed IPNS record for a given name from the routing system.\n\nThe output is the raw IPNS record (protobuf) as defined in the IPNS spec:\nhttps://specs.ipfs.tech/ipns/ipns-record/\n\nThe record can be inspected with 'ipfs name inspect':\n\n ipfs name get | ipfs name inspect\n\nThis is equivalent to 'ipfs routing get /ipns/' but only accepts\nIPNS names (not arbitrary routing keys).\n\nNote: The routing system returns the \"best\" IPNS record it knows about.\nFor IPNS, \"best\" means the record with the highest sequence number.\nIf multiple records exist (e.g., after using 'ipfs name put'), this command\nreturns the one the routing system considers most current.\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tResponseContentType: \"application/vnd.ipfs.ipns-record\",\n\t\t},\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"The IPNS name to look up.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Normalize the argument: accept both \"k51...\" and \"/ipns/k51...\"\n\t\tname := req.Arguments[0]\n\t\tif !strings.HasPrefix(name, \"/ipns/\") {\n\t\t\tname = \"/ipns/\" + name\n\t\t}\n\n\t\tdata, err := api.Routing().Get(req.Context, name)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tres.SetEncodingType(cmds.OctetStream)\n\t\tres.SetContentType(\"application/vnd.ipfs.ipns-record\")\n\t\treturn res.Emit(bytes.NewReader(data))\n\t},\n}\n\nconst (\n\tforceOptionName = \"force\"\n\tputAllowOfflineOption = \"allow-offline\"\n\tallowDelegatedOption = \"allow-delegated\"\n\tputQuietOptionName = \"quiet\"\n\tmaxIPNSRecordSize = 10 << 10 // 10 KiB per IPNS spec\n)\n\nvar errPutAllowOffline = errors.New(\"can't put while offline: pass `--allow-offline` to store locally or `--allow-delegated` if Ipns.DelegatedPublishers are set up\")\n\nvar IpnsPutCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Store a pre-signed IPNS record in the routing system.\",\n\t\tShortDescription: `\nStores a pre-signed IPNS record in the routing system.\n\nThis command accepts a raw IPNS record (protobuf) as defined in the IPNS spec:\nhttps://specs.ipfs.tech/ipns/ipns-record/\n\nThe record must be signed by the private key corresponding to the IPNS name.\nUse 'ipfs name get' to retrieve records and 'ipfs name inspect' to examine.\n`,\n\t\tLongDescription: `\nStores a pre-signed IPNS record in the routing system.\n\nThis command accepts a raw IPNS record (protobuf) as defined in the IPNS spec:\nhttps://specs.ipfs.tech/ipns/ipns-record/\n\nThe record must be signed by the private key corresponding to the IPNS name.\nUse 'ipfs name get' to retrieve records and 'ipfs name inspect' to examine.\n\nUse Cases:\n\n - Re-publishing third-party records: store someone else's signed record\n - Cross-node sync: import records exported from another node\n - Backup/restore: export with 'name get', restore with 'name put'\n\nValidation:\n\nBy default, the command validates that:\n\n - The record is a valid IPNS record (protobuf)\n - The record size is within 10 KiB limit\n - The signature matches the provided IPNS name\n - The record's sequence number is higher than any existing record\n (identical records are allowed for republishing)\n\nThe --force flag skips this command's validation and passes the record\ndirectly to the routing system. Note that --force only affects this command;\nit does not control how the routing system handles the record. The routing\nsystem may still reject invalid records or prefer records with higher sequence\nnumbers. Use --force primarily for testing (e.g., to observe how the routing\nsystem reacts to incorrectly signed or malformed records).\n\nImportant: Even after a successful 'name put', a subsequent 'name get' may\nreturn a different record if one with a higher sequence number exists.\nThis is expected IPNS behavior, not a bug.\n\nPublishing Modes:\n\nBy default, IPNS records are published to both the DHT and any configured\nHTTP delegated publishers. You can control this behavior with:\n\n --allow-offline Store locally without requiring network connectivity\n --allow-delegated Publish via HTTP delegated publishers only (no DHT)\n\nExamples:\n\nExport and re-import a record:\n\n > ipfs name get k51... > record.bin\n > ipfs name put k51... record.bin\n\nStore a record received from someone else:\n\n > ipfs name put k51... third-party-record.bin\n\nForce store a record to test routing validation:\n\n > ipfs name put --force k51... possibly-invalid-record.bin\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tDescription: \"Request body should be `multipart/form-data` with the IPNS record bytes.\",\n\t\t},\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"The IPNS name to store the record for (e.g., k51... or /ipns/k51...).\"),\n\t\tcmds.FileArg(\"record\", true, false, \"Path to file containing the signed IPNS record.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(forceOptionName, \"f\", \"Skip validation (signature, sequence, size).\"),\n\t\tcmds.BoolOption(putAllowOfflineOption, \"Store locally without broadcasting to the network.\"),\n\t\tcmds.BoolOption(allowDelegatedOption, \"Publish via HTTP delegated publishers only (no DHT).\"),\n\t\tcmds.BoolOption(putQuietOptionName, \"q\", \"Write no output.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Parse options\n\t\tforce, _ := req.Options[forceOptionName].(bool)\n\t\tallowOffline, _ := req.Options[putAllowOfflineOption].(bool)\n\t\tallowDelegated, _ := req.Options[allowDelegatedOption].(bool)\n\n\t\t// Validate flag combinations\n\t\tif allowOffline && allowDelegated {\n\t\t\treturn errors.New(\"cannot use both --allow-offline and --allow-delegated flags\")\n\t\t}\n\n\t\t// Handle different publishing modes\n\t\tif allowDelegated {\n\t\t\t// AllowDelegated mode: check if delegated publishers are configured\n\t\t\tcfg, err := nd.Repo.Config()\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to read config: %w\", err)\n\t\t\t}\n\t\t\tdelegatedPublishers := cfg.DelegatedPublishersWithAutoConf()\n\t\t\tif len(delegatedPublishers) == 0 {\n\t\t\t\treturn errors.New(\"no delegated publishers configured: add Ipns.DelegatedPublishers or use --allow-offline for local-only publishing\")\n\t\t\t}\n\t\t\t// For allow-delegated mode, we proceed even if offline\n\t\t\t// since we're using HTTP publishing via delegated publishers\n\t\t}\n\n\t\t// Parse the IPNS name argument\n\t\tnameArg := req.Arguments[0]\n\t\tif !strings.HasPrefix(nameArg, \"/ipns/\") {\n\t\t\tnameArg = \"/ipns/\" + nameArg\n\t\t}\n\t\t// Extract the name part after /ipns/\n\t\tnamePart := strings.TrimPrefix(nameArg, \"/ipns/\")\n\t\tname, err := ipns.NameFromString(namePart)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"invalid IPNS name: %w\", err)\n\t\t}\n\n\t\t// Read raw record bytes from file/stdin\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\t// Read record data (limit to 1 MiB for memory safety)\n\t\tdata, err := io.ReadAll(io.LimitReader(file, 1<<20))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to read record: %w\", err)\n\t\t}\n\t\tif len(data) == 0 {\n\t\t\treturn errors.New(\"record is empty\")\n\t\t}\n\n\t\t// Validate unless --force\n\t\tif !force {\n\t\t\t// Check size limit per IPNS spec\n\t\t\tif len(data) > maxIPNSRecordSize {\n\t\t\t\treturn fmt.Errorf(\"record exceeds maximum size of %d bytes, use --force to skip size check\", maxIPNSRecordSize)\n\t\t\t}\n\t\t\trec, err := ipns.UnmarshalRecord(data)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"invalid IPNS record: %w\", err)\n\t\t\t}\n\n\t\t\t// Validate signature against provided name\n\t\t\terr = ipns.ValidateWithName(rec, name)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"record validation failed: %w\", err)\n\t\t\t}\n\n\t\t\t// Check for sequence conflicts with existing record\n\t\t\texistingData, err := api.Routing().Get(req.Context, nameArg)\n\t\t\tif err == nil {\n\t\t\t\t// Allow republishing the exact same record (common use case:\n\t\t\t\t// get a third-party record and put it back to refresh DHT)\n\t\t\t\tif !bytes.Equal(existingData, data) {\n\t\t\t\t\texistingRec, parseErr := ipns.UnmarshalRecord(existingData)\n\t\t\t\t\tif parseErr == nil {\n\t\t\t\t\t\texistingSeq, seqErr := existingRec.Sequence()\n\t\t\t\t\t\tnewSeq, newSeqErr := rec.Sequence()\n\t\t\t\t\t\tif seqErr == nil && newSeqErr == nil && existingSeq >= newSeq {\n\t\t\t\t\t\t\treturn fmt.Errorf(\"existing IPNS record has sequence %d >= new record sequence %d, use 'ipfs name put --force' to skip this check\", existingSeq, newSeq)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\t// If Get fails (no existing record), that's fine - proceed with put\n\t\t}\n\n\t\t// Publish the original bytes as-is\n\t\t// When allowDelegated is true, we set allowOffline to allow the operation\n\t\t// even without DHT connectivity (delegated publishers use HTTP)\n\t\topts := []options.RoutingPutOption{\n\t\t\toptions.Routing.AllowOffline(allowOffline || allowDelegated),\n\t\t}\n\n\t\terr = api.Routing().Put(req.Context, nameArg, data, opts...)\n\t\tif err != nil {\n\t\t\tif err.Error() == \"can't put while offline\" {\n\t\t\t\treturn errPutAllowOffline\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\n\t\t// Extract value from the record for the response\n\t\tvalue := \"\"\n\t\tif rec, err := ipns.UnmarshalRecord(data); err == nil {\n\t\t\tif v, err := rec.Value(); err == nil {\n\t\t\t\tvalue = v.String()\n\t\t\t}\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &IpnsEntry{\n\t\t\tName: name.String(),\n\t\t\tValue: value,\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ie *IpnsEntry) error {\n\t\t\tquiet, _ := req.Options[putQuietOptionName].(bool)\n\t\t\tif quiet {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\t_, err := fmt.Fprintln(w, cmdenv.EscNonPrint(ie.Name))\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: IpnsEntry{},\n}\n" }, { "path": "core/commands/name/publish.go", "content": "package name\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"time\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tipns \"github.com/ipfs/boxo/ipns\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tke \"github.com/ipfs/kubo/core/commands/keyencode\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nvar errAllowOffline = errors.New(\"can't publish while offline: pass `--allow-offline` to override or `--allow-delegated` if Ipns.DelegatedPublishers are set up\")\n\nconst (\n\tipfsPathOptionName = \"ipfs-path\"\n\tresolveOptionName = \"resolve\"\n\tallowOfflineOptionName = \"allow-offline\"\n\tallowDelegatedOptionName = \"allow-delegated\"\n\tlifeTimeOptionName = \"lifetime\"\n\tttlOptionName = \"ttl\"\n\tkeyOptionName = \"key\"\n\tquieterOptionName = \"quieter\"\n\tv1compatOptionName = \"v1compat\"\n\tsequenceOptionName = \"sequence\"\n)\n\nvar PublishCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Publish IPNS names.\",\n\t\tShortDescription: `\nIPNS is a PKI namespace, where names are the hashes of public keys, and\nthe private key enables publishing new (signed) values. In both publish\nand resolve, the default name used is the node's own PeerID,\nwhich is the hash of its public key.\n`,\n\t\tLongDescription: `\nIPNS is a PKI namespace, where names are the hashes of public keys, and\nthe private key enables publishing new (signed) values. In both publish\nand resolve, the default name used is the node's own PeerID,\nwhich is the hash of its public key.\n\nYou can use the 'ipfs key' commands to list and generate more names and their\nrespective keys.\n\nPublishing Modes:\n\nBy default, IPNS records are published to both the DHT and any configured\nHTTP delegated publishers. You can control this behavior with the following flags:\n\n --allow-offline Allow publishing when offline (publishes to local datastore, network operations are optional)\n --allow-delegated Allow publishing without DHT connectivity (local + HTTP delegated publishers only)\n\nExamples:\n\nPublish an with your default name:\n\n > ipfs name publish /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n Published to QmbCMUZw6JFeZ7Wp9jkzbye3Fzp2GGcPgC3nmeUjfVF87n: /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n\nPublish without DHT (HTTP delegated publishers only):\n\n > ipfs name publish --allow-delegated /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n Published to QmbCMUZw6JFeZ7Wp9jkzbye3Fzp2GGcPgC3nmeUjfVF87n: /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n\nPublish when offline (local publish, network optional):\n\n > ipfs name publish --allow-offline /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n Published to QmbCMUZw6JFeZ7Wp9jkzbye3Fzp2GGcPgC3nmeUjfVF87n: /ipfs/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n\nNotes:\n\nThe --ttl option specifies the time duration for caching IPNS records.\nLower values like '1m' enable faster updates but increase network load,\nwhile the default of 1 hour reduces traffic but may delay propagation.\nGateway operators may override this with Ipns.MaxCacheTTL configuration.\n\nThe --sequence option sets a custom sequence number for the IPNS record.\nThe sequence number must be monotonically increasing (greater than the\ncurrent record's sequence). This is useful for manually coordinating\nupdates across multiple writers. If not specified, the sequence number\nincrements automatically.\n\nFor faster IPNS updates, consider:\n- Using a lower --ttl value (e.g., '1m' for quick updates)\n- Enabling PubSub via Ipns.UsePubsub in the config\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(ipfsPathOptionName, true, false, \"ipfs path of the object to be published.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(keyOptionName, \"k\", \"Name of the key to be used or a valid PeerID, as listed by 'ipfs key list -l'.\").WithDefault(\"self\"),\n\t\tcmds.BoolOption(resolveOptionName, \"Check if the given path can be resolved before publishing.\").WithDefault(true),\n\t\tcmds.StringOption(lifeTimeOptionName, \"t\", `Time duration the signed record will be valid for. Accepts durations such as \"300s\", \"1.5h\" or \"7d2h45m\"`).WithDefault(ipns.DefaultRecordLifetime.String()),\n\t\tcmds.StringOption(ttlOptionName, \"Time duration hint, akin to --lifetime, indicating how long to cache this record before checking for updates.\").WithDefault(ipns.DefaultRecordTTL.String()),\n\t\tcmds.BoolOption(quieterOptionName, \"Q\", \"Write only final IPNS Name encoded as CIDv1 (for use in /ipns content paths).\"),\n\t\tcmds.BoolOption(v1compatOptionName, \"Produce a backward-compatible IPNS Record by including fields for both V1 and V2 signatures.\").WithDefault(true),\n\t\tcmds.BoolOption(allowOfflineOptionName, \"Allow publishing when offline - publishes to local datastore without requiring network connectivity.\"),\n\t\tcmds.BoolOption(allowDelegatedOptionName, \"Allow publishing without DHT connectivity - uses local datastore and HTTP delegated publishers only.\"),\n\t\tcmds.Uint64Option(sequenceOptionName, \"Set a custom sequence number for the IPNS record (must be higher than current).\"),\n\t\tke.OptionIPNSBase,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tallowOffline, _ := req.Options[allowOfflineOptionName].(bool)\n\t\tallowDelegated, _ := req.Options[allowDelegatedOptionName].(bool)\n\t\tcompatibleWithV1, _ := req.Options[v1compatOptionName].(bool)\n\t\tkname, _ := req.Options[keyOptionName].(string)\n\n\t\t// Validate flag combinations\n\t\tif allowOffline && allowDelegated {\n\t\t\treturn errors.New(\"cannot use both --allow-offline and --allow-delegated flags\")\n\t\t}\n\n\t\tvalidTimeOpt, _ := req.Options[lifeTimeOptionName].(string)\n\t\tvalidTime, err := time.ParseDuration(validTimeOpt)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error parsing lifetime option: %s\", err)\n\t\t}\n\n\t\topts := []options.NamePublishOption{\n\t\t\toptions.Name.AllowOffline(allowOffline),\n\t\t\toptions.Name.AllowDelegated(allowDelegated),\n\t\t\toptions.Name.Key(kname),\n\t\t\toptions.Name.ValidTime(validTime),\n\t\t\toptions.Name.CompatibleWithV1(compatibleWithV1),\n\t\t}\n\n\t\tif ttl, found := req.Options[ttlOptionName].(string); found {\n\t\t\td, err := time.ParseDuration(ttl)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\topts = append(opts, options.Name.TTL(d))\n\t\t}\n\n\t\tif sequence, found := req.Options[sequenceOptionName].(uint64); found {\n\t\t\topts = append(opts, options.Name.Sequence(sequence))\n\t\t}\n\n\t\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif verifyExists, _ := req.Options[resolveOptionName].(bool); verifyExists {\n\t\t\t_, err := api.ResolveNode(req.Context, p)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tname, err := api.Name().Publish(req.Context, p, opts...)\n\t\tif err != nil {\n\t\t\tif err == iface.ErrOffline {\n\t\t\t\terr = errAllowOffline\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &IpnsEntry{\n\t\t\tName: name.String(),\n\t\t\tValue: p.String(),\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ie *IpnsEntry) error {\n\t\t\tvar err error\n\t\t\tquieter, _ := req.Options[quieterOptionName].(bool)\n\t\t\tif quieter {\n\t\t\t\t_, err = fmt.Fprintln(w, cmdenv.EscNonPrint(ie.Name))\n\t\t\t} else {\n\t\t\t\t_, err = fmt.Fprintf(w, \"Published to %s: %s\\n\", cmdenv.EscNonPrint(ie.Name), cmdenv.EscNonPrint(ie.Value))\n\t\t\t}\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: IpnsEntry{},\n}\n" }, { "path": "core/commands/object/diff.go", "content": "package objectcmd\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/ipld/merkledag/dagutils\"\n\t\"github.com/ipfs/boxo/path\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n)\n\nconst (\n\tverboseOptionName = \"verbose\"\n)\n\ntype Changes struct {\n\tChanges []*dagutils.Change\n}\n\nvar ObjectDiffCmd = &cmds.Command{\n\tStatus: cmds.Deprecated, // https://github.com/ipfs/kubo/issues/7936\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Display the diff between two IPFS objects.\",\n\t\tShortDescription: `\n'ipfs object diff' is a command used to show the differences between\ntwo IPFS objects.`,\n\t\tLongDescription: `\n'ipfs object diff' is a command used to show the differences between\ntwo IPFS objects.\n\nExample:\n\n > ls foo\n bar baz/ giraffe\n > ipfs add -r foo\n ...\n Added QmegHcnrPgMwC7tBiMxChD54fgQMBUecNw9nE9UUU4x1bz foo\n > OBJ_A=QmegHcnrPgMwC7tBiMxChD54fgQMBUecNw9nE9UUU4x1bz\n > echo \"different content\" > foo/bar\n > ipfs add -r foo\n ...\n Added QmcmRptkSPWhptCttgHg27QNDmnV33wAJyUkCnAvqD3eCD foo\n > OBJ_B=QmcmRptkSPWhptCttgHg27QNDmnV33wAJyUkCnAvqD3eCD\n > ipfs object diff -v $OBJ_A $OBJ_B\n Changed \"bar\" from QmNgd5cz2jNftnAHBhcRUGdtiaMzb5Rhjqd4etondHHST8 to QmRfFVsjSXkhFxrfWnLpMae2M4GBVsry6VAuYYcji5MiZb.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"obj_a\", true, false, \"Object to diff against.\"),\n\t\tcmds.StringArg(\"obj_b\", true, false, \"Object to diff.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(verboseOptionName, \"v\", \"Print extra information.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpa, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpb, err := cmdutils.PathOrCidPath(req.Arguments[1])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tchanges, err := api.Object().Diff(req.Context, pa, pb)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tout := make([]*dagutils.Change, len(changes))\n\t\tfor i, change := range changes {\n\t\t\tout[i] = &dagutils.Change{\n\t\t\t\tType: dagutils.ChangeType(change.Type),\n\t\t\t\tPath: change.Path,\n\t\t\t}\n\n\t\t\tif (change.Before != path.ImmutablePath{}) {\n\t\t\t\tout[i].Before = change.Before.RootCid()\n\t\t\t}\n\n\t\t\tif (change.After != path.ImmutablePath{}) {\n\t\t\t\tout[i].After = change.After.RootCid()\n\t\t\t}\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &Changes{out})\n\t},\n\tType: Changes{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *Changes) error {\n\t\t\tverbose, _ := req.Options[verboseOptionName].(bool)\n\n\t\t\tfor _, change := range out.Changes {\n\t\t\t\tif verbose {\n\t\t\t\t\tswitch change.Type {\n\t\t\t\t\tcase dagutils.Add:\n\t\t\t\t\t\tfmt.Fprintf(w, \"Added new link %q pointing to %s.\\n\", change.Path, change.After)\n\t\t\t\t\tcase dagutils.Mod:\n\t\t\t\t\t\tfmt.Fprintf(w, \"Changed %q from %s to %s.\\n\", change.Path, change.Before, change.After)\n\t\t\t\t\tcase dagutils.Remove:\n\t\t\t\t\t\tfmt.Fprintf(w, \"Removed link %q (was %s).\\n\", change.Path, change.Before)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tswitch change.Type {\n\t\t\t\t\tcase dagutils.Add:\n\t\t\t\t\t\tfmt.Fprintf(w, \"+ %s %q\\n\", change.After, change.Path)\n\t\t\t\t\tcase dagutils.Mod:\n\t\t\t\t\t\tfmt.Fprintf(w, \"~ %s %s %q\\n\", change.Before, change.After, change.Path)\n\t\t\t\t\tcase dagutils.Remove:\n\t\t\t\t\t\tfmt.Fprintf(w, \"- %s %q\\n\", change.Before, change.Path)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/object/object.go", "content": "package objectcmd\n\nimport (\n\t\"errors\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\ntype Link struct {\n\tName, Hash string\n\tSize uint64\n}\n\ntype Object struct {\n\tHash string `json:\"Hash,omitempty\"`\n\tLinks []Link `json:\"Links,omitempty\"`\n}\n\nvar ErrDataEncoding = errors.New(\"unknown data field encoding\")\n\nvar ObjectCmd = &cmds.Command{\n\tStatus: cmds.Deprecated, // https://github.com/ipfs/kubo/issues/7936\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated commands to interact with dag-pb objects. Use 'dag' or 'files' instead.\",\n\t\tShortDescription: `\n'ipfs object' is a legacy plumbing command used to manipulate dag-pb objects\ndirectly. Deprecated, use more modern 'ipfs dag' and 'ipfs files' instead.`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"data\": RemovedObjectCmd,\n\t\t\"diff\": ObjectDiffCmd,\n\t\t\"get\": RemovedObjectCmd,\n\t\t\"links\": RemovedObjectCmd,\n\t\t\"new\": RemovedObjectCmd,\n\t\t\"patch\": ObjectPatchCmd,\n\t\t\"put\": RemovedObjectCmd,\n\t\t\"stat\": RemovedObjectCmd,\n\t},\n}\n\nvar RemovedObjectCmd = &cmds.Command{\n\tStatus: cmds.Removed,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Removed, use 'ipfs dag' or 'ipfs files' instead.\",\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\treturn errors.New(\"removed, use 'ipfs dag' or 'ipfs files' instead\")\n\t},\n}\n" }, { "path": "core/commands/object/patch.go", "content": "package objectcmd\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nvar ObjectPatchCmd = &cmds.Command{\n\tStatus: cmds.Deprecated, // https://github.com/ipfs/kubo/issues/7936\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated way to create a new merkledag object based on an existing one. Use MFS with 'files cp|rm' instead.\",\n\t\tShortDescription: `\n'ipfs object patch ' is a plumbing command used to\nbuild custom dag-pb objects. It mutates objects, creating new objects as a\nresult. This is the Merkle-DAG version of modifying an object.\n\nDEPRECATED and provided for legacy reasons.\nFor modern use cases, use MFS with 'files' commands: 'ipfs files --help'.\n\n $ ipfs files cp /ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn /some-dir\n $ ipfs files cp /ipfs/Qmayz4F4UzqcAMitTzU4zCSckDofvxstDuj3y7ajsLLEVs /some-dir/added-file.jpg\n $ ipfs files stat --hash /some-dir\n\n The above will add 'added-file.jpg' to the directory placed under /some-dir\n and the CID of updated directory is returned by 'files stat'\n\n 'files cp' does not download the data, only the root block, which makes it\n possible to build arbitrary directory trees without fetching them in full to\n the local node.\n`,\n\t},\n\tArguments: []cmds.Argument{},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"append-data\": RemovedObjectCmd,\n\t\t\"add-link\": patchAddLinkCmd,\n\t\t\"rm-link\": patchRmLinkCmd,\n\t\t\"set-data\": RemovedObjectCmd,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmdutils.AllowBigBlockOption,\n\t},\n}\n\nvar patchRmLinkCmd = &cmds.Command{\n\tStatus: cmds.Deprecated, // https://github.com/ipfs/kubo/issues/7936\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated way to remove a link from dag-pb object.\",\n\t\tShortDescription: `\nRemove a Merkle-link from the given object and return the hash of the result.\n\nDEPRECATED and provided for legacy reasons. Use 'files rm' instead.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"root\", true, false, \"The hash of the node to modify.\"),\n\t\tcmds.StringArg(\"name\", true, false, \"Name of the link to remove.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\troot, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tname := req.Arguments[1]\n\t\tp, err := api.Object().RmLink(req.Context, root, name)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := cmdutils.CheckCIDSize(req, p.RootCid(), api.Dag()); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &Object{Hash: p.RootCid().String()})\n\t},\n\tType: Object{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *Object) error {\n\t\t\tfmt.Fprintln(w, out.Hash)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nconst (\n\tcreateOptionName = \"create\"\n)\n\nvar patchAddLinkCmd = &cmds.Command{\n\tStatus: cmds.Deprecated, // https://github.com/ipfs/kubo/issues/7936\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated way to add a link to a given dag-pb.\",\n\t\tShortDescription: `\nAdd a Merkle-link to the given object and return the hash of the result.\n\nDEPRECATED and provided for legacy reasons.\n\nUse MFS and 'files' commands instead:\n\n $ ipfs files cp /ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn /some-dir\n $ ipfs files cp /ipfs/Qmayz4F4UzqcAMitTzU4zCSckDofvxstDuj3y7ajsLLEVs /some-dir/added-file.jpg\n $ ipfs files stat --hash /some-dir\n\n The above will add 'added-file.jpg' to the directory placed under /some-dir\n and the CID of updated directory is returned by 'files stat'\n\n 'files cp' does not download the data, only the root block, which makes it\n possible to build arbitrary directory trees without fetching them in full to\n the local node.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"root\", true, false, \"The hash of the node to modify.\"),\n\t\tcmds.StringArg(\"name\", true, false, \"Name of link to create.\"),\n\t\tcmds.StringArg(\"ref\", true, false, \"IPFS object to add link to.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(createOptionName, \"p\", \"Create intermediary nodes.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\troot, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tname := req.Arguments[1]\n\n\t\tchild, err := cmdutils.PathOrCidPath(req.Arguments[2])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcreate, _ := req.Options[createOptionName].(bool)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tp, err := api.Object().AddLink(req.Context, root, name, child,\n\t\t\toptions.Object.Create(create))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := cmdutils.CheckCIDSize(req, p.RootCid(), api.Dag()); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &Object{Hash: p.RootCid().String()})\n\t},\n\tType: Object{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *Object) error {\n\t\t\tfmt.Fprintln(w, out.Hash)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/p2p.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strconv\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tp2p \"github.com/ipfs/kubo/p2p\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\tprotocol \"github.com/libp2p/go-libp2p/core/protocol\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n)\n\n// P2PProtoPrefix is the default required prefix for protocol names\nconst P2PProtoPrefix = \"/x/\"\n\n// P2PListenerInfoOutput is output type of ls command\ntype P2PListenerInfoOutput struct {\n\tProtocol string\n\tListenAddress string\n\tTargetAddress string\n}\n\n// P2PStreamInfoOutput is output type of streams command\ntype P2PStreamInfoOutput struct {\n\tHandlerID string\n\tProtocol string\n\tOriginAddress string\n\tTargetAddress string\n}\n\n// P2PLsOutput is output type of ls command\ntype P2PLsOutput struct {\n\tListeners []P2PListenerInfoOutput\n}\n\n// P2PStreamsOutput is output type of streams command\ntype P2PStreamsOutput struct {\n\tStreams []P2PStreamInfoOutput\n}\n\n// P2PForegroundOutput is output type for foreground mode status messages\ntype P2PForegroundOutput struct {\n\tStatus string // \"active\" or \"closing\"\n\tProtocol string\n\tAddress string\n}\n\nconst (\n\tallowCustomProtocolOptionName = \"allow-custom-protocol\"\n\treportPeerIDOptionName = \"report-peer-id\"\n\tforegroundOptionName = \"foreground\"\n)\n\nvar resolveTimeout = 10 * time.Second\n\n// P2PCmd is the 'ipfs p2p' command\nvar P2PCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Libp2p stream mounting.\",\n\t\tShortDescription: `\nCreate and use tunnels to remote peers over libp2p\n\nNote: this command is experimental and subject to change as usecases and APIs\nare refined`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"stream\": p2pStreamCmd,\n\t\t\"forward\": p2pForwardCmd,\n\t\t\"listen\": p2pListenCmd,\n\t\t\"close\": p2pCloseCmd,\n\t\t\"ls\": p2pLsCmd,\n\t},\n}\n\nvar p2pForwardCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Forward connections to libp2p service.\",\n\t\tShortDescription: `\nForward connections made to to via libp2p.\n\nCreates a local TCP listener that tunnels connections through libp2p to a\nremote peer's p2p listener. Similar to SSH port forwarding (-L flag).\n\nARGUMENTS:\n\n Protocol name (must start with '` + P2PProtoPrefix + `')\n Local multiaddr (e.g., /ip4/127.0.0.1/tcp/3000)\n Remote peer multiaddr (e.g., /p2p/PeerID)\n\nFOREGROUND MODE (--foreground, -f):\n\n By default, the forwarder runs in the daemon and the command returns\n immediately. Use --foreground to block until interrupted:\n\n - Ctrl+C or SIGTERM: Removes the forwarder and exits\n - 'ipfs p2p close': Removes the forwarder and exits\n - Daemon shutdown: Forwarder is automatically removed\n\n Useful for systemd services or scripts that need cleanup on exit.\n\nEXAMPLES:\n\n # Persistent forwarder (command returns immediately)\n ipfs p2p forward /x/myapp /ip4/127.0.0.1/tcp/3000 /p2p/PeerID\n\n # Temporary forwarder (removed when command exits)\n ipfs p2p forward -f /x/myapp /ip4/127.0.0.1/tcp/3000 /p2p/PeerID\n\nLearn more: https://github.com/ipfs/kubo/blob/master/docs/p2p-tunnels.md\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"protocol\", true, false, \"Protocol name.\"),\n\t\tcmds.StringArg(\"listen-address\", true, false, \"Listening endpoint.\"),\n\t\tcmds.StringArg(\"target-address\", true, false, \"Target endpoint.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(allowCustomProtocolOptionName, \"Don't require /x/ prefix\"),\n\t\tcmds.BoolOption(foregroundOptionName, \"f\", \"Run in foreground; forwarder is removed when command exits\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := p2pGetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tprotoOpt := req.Arguments[0]\n\t\tlistenOpt := req.Arguments[1]\n\t\ttargetOpt := req.Arguments[2]\n\n\t\tproto := protocol.ID(protoOpt)\n\n\t\tlisten, err := ma.NewMultiaddr(listenOpt)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttargets, err := parseIpfsAddr(targetOpt)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tallowCustom, _ := req.Options[allowCustomProtocolOptionName].(bool)\n\n\t\tif !allowCustom && !strings.HasPrefix(string(proto), P2PProtoPrefix) {\n\t\t\treturn errors.New(\"protocol name must be within '\" + P2PProtoPrefix + \"' namespace\")\n\t\t}\n\n\t\tlistener, err := forwardLocal(n.Context(), n.P2P, n.Peerstore, proto, listen, targets)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tforeground, _ := req.Options[foregroundOptionName].(bool)\n\t\tif foreground {\n\t\t\tif err := res.Emit(&P2PForegroundOutput{\n\t\t\t\tStatus: \"active\",\n\t\t\t\tProtocol: protoOpt,\n\t\t\t\tAddress: listenOpt,\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t// Wait for either context cancellation (Ctrl+C/daemon shutdown)\n\t\t\t// or listener removal (ipfs p2p close)\n\t\t\tselect {\n\t\t\tcase <-req.Context.Done():\n\t\t\t\t// SIGTERM/Ctrl+C - cleanup silently (CLI stream already closing)\n\t\t\t\tn.P2P.ListenersLocal.Close(func(l p2p.Listener) bool {\n\t\t\t\t\treturn l == listener\n\t\t\t\t})\n\t\t\t\treturn nil\n\t\t\tcase <-listener.Done():\n\t\t\t\t// Closed via \"ipfs p2p close\" - emit closing message\n\t\t\t\treturn res.Emit(&P2PForegroundOutput{\n\t\t\t\t\tStatus: \"closing\",\n\t\t\t\t\tProtocol: protoOpt,\n\t\t\t\t\tAddress: listenOpt,\n\t\t\t\t})\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tType: P2PForegroundOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *P2PForegroundOutput) error {\n\t\t\tif out.Status == \"active\" {\n\t\t\t\tfmt.Fprintf(w, \"Forwarding %s to %s, waiting for interrupt...\\n\", out.Protocol, out.Address)\n\t\t\t} else if out.Status == \"closing\" {\n\t\t\t\tfmt.Fprintf(w, \"Received interrupt, removing forwarder for %s\\n\", out.Protocol)\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// parseIpfsAddr is a function that takes in addr string and return ipfsAddrs\nfunc parseIpfsAddr(addr string) (*peer.AddrInfo, error) {\n\tmultiaddr, err := ma.NewMultiaddr(addr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpi, err := peer.AddrInfoFromP2pAddr(multiaddr)\n\tif err == nil {\n\t\treturn pi, nil\n\t}\n\n\t// resolve multiaddr whose protocol is not ma.P_IPFS\n\tctx, cancel := context.WithTimeout(context.Background(), resolveTimeout)\n\tdefer cancel()\n\taddrs, err := madns.Resolve(ctx, multiaddr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif len(addrs) == 0 {\n\t\treturn nil, errors.New(\"fail to resolve the multiaddr:\" + multiaddr.String())\n\t}\n\tvar info peer.AddrInfo\n\tfor _, addr := range addrs {\n\t\ttaddr, id := peer.SplitAddr(addr)\n\t\tif id == \"\" {\n\t\t\t// not an ipfs addr, skipping.\n\t\t\tcontinue\n\t\t}\n\t\tswitch info.ID {\n\t\tcase \"\":\n\t\t\tinfo.ID = id\n\t\tcase id:\n\t\tdefault:\n\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\"ambiguous multiaddr %s could refer to %s or %s\",\n\t\t\t\tmultiaddr,\n\t\t\t\tinfo.ID,\n\t\t\t\tid,\n\t\t\t)\n\t\t}\n\t\tinfo.Addrs = append(info.Addrs, taddr)\n\t}\n\treturn &info, nil\n}\n\nvar p2pListenCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Create libp2p service.\",\n\t\tShortDescription: `\nCreate a libp2p protocol handler that forwards incoming connections to\n.\n\nWhen a remote peer connects using 'ipfs p2p forward', the connection is\nforwarded to your local service. Similar to SSH port forwarding (server side).\n\nARGUMENTS:\n\n Protocol name (must start with '` + P2PProtoPrefix + `')\n Local multiaddr (e.g., /ip4/127.0.0.1/tcp/3000)\n\nFOREGROUND MODE (--foreground, -f):\n\n By default, the listener runs in the daemon and the command returns\n immediately. Use --foreground to block until interrupted:\n\n - Ctrl+C or SIGTERM: Removes the listener and exits\n - 'ipfs p2p close': Removes the listener and exits\n - Daemon shutdown: Listener is automatically removed\n\n Useful for systemd services or scripts that need cleanup on exit.\n\nEXAMPLES:\n\n # Persistent listener (command returns immediately)\n ipfs p2p listen /x/myapp /ip4/127.0.0.1/tcp/3000\n\n # Temporary listener (removed when command exits)\n ipfs p2p listen -f /x/myapp /ip4/127.0.0.1/tcp/3000\n\n # Report connecting peer ID to the target application\n ipfs p2p listen -r /x/myapp /ip4/127.0.0.1/tcp/3000\n\nLearn more: https://github.com/ipfs/kubo/blob/master/docs/p2p-tunnels.md\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"protocol\", true, false, \"Protocol name.\"),\n\t\tcmds.StringArg(\"target-address\", true, false, \"Target endpoint.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(allowCustomProtocolOptionName, \"Don't require /x/ prefix\"),\n\t\tcmds.BoolOption(reportPeerIDOptionName, \"r\", \"Send remote base58 peerid to target when a new connection is established\"),\n\t\tcmds.BoolOption(foregroundOptionName, \"f\", \"Run in foreground; listener is removed when command exits\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := p2pGetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tprotoOpt := req.Arguments[0]\n\t\ttargetOpt := req.Arguments[1]\n\n\t\tproto := protocol.ID(protoOpt)\n\n\t\ttarget, err := ma.NewMultiaddr(targetOpt)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// port can't be 0\n\t\tif err := checkPort(target); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tallowCustom, _ := req.Options[allowCustomProtocolOptionName].(bool)\n\t\treportPeerID, _ := req.Options[reportPeerIDOptionName].(bool)\n\n\t\tif !allowCustom && !strings.HasPrefix(string(proto), P2PProtoPrefix) {\n\t\t\treturn errors.New(\"protocol name must be within '\" + P2PProtoPrefix + \"' namespace\")\n\t\t}\n\n\t\tlistener, err := n.P2P.ForwardRemote(n.Context(), proto, target, reportPeerID)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tforeground, _ := req.Options[foregroundOptionName].(bool)\n\t\tif foreground {\n\t\t\tif err := res.Emit(&P2PForegroundOutput{\n\t\t\t\tStatus: \"active\",\n\t\t\t\tProtocol: protoOpt,\n\t\t\t\tAddress: targetOpt,\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t// Wait for either context cancellation (Ctrl+C/daemon shutdown)\n\t\t\t// or listener removal (ipfs p2p close)\n\t\t\tselect {\n\t\t\tcase <-req.Context.Done():\n\t\t\t\t// SIGTERM/Ctrl+C - cleanup silently (CLI stream already closing)\n\t\t\t\tn.P2P.ListenersP2P.Close(func(l p2p.Listener) bool {\n\t\t\t\t\treturn l == listener\n\t\t\t\t})\n\t\t\t\treturn nil\n\t\t\tcase <-listener.Done():\n\t\t\t\t// Closed via \"ipfs p2p close\" - emit closing message\n\t\t\t\treturn res.Emit(&P2PForegroundOutput{\n\t\t\t\t\tStatus: \"closing\",\n\t\t\t\t\tProtocol: protoOpt,\n\t\t\t\t\tAddress: targetOpt,\n\t\t\t\t})\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tType: P2PForegroundOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *P2PForegroundOutput) error {\n\t\t\tif out.Status == \"active\" {\n\t\t\t\tfmt.Fprintf(w, \"Listening on %s, forwarding to %s, waiting for interrupt...\\n\", out.Protocol, out.Address)\n\t\t\t} else if out.Status == \"closing\" {\n\t\t\t\tfmt.Fprintf(w, \"Received interrupt, removing listener for %s\\n\", out.Protocol)\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// checkPort checks whether target multiaddr contains tcp or udp protocol\n// and whether the port is equal to 0\nfunc checkPort(target ma.Multiaddr) error {\n\t// get tcp or udp port from multiaddr\n\tgetPort := func() (string, error) {\n\t\tsport, _ := target.ValueForProtocol(ma.P_TCP)\n\t\tif sport != \"\" {\n\t\t\treturn sport, nil\n\t\t}\n\n\t\tsport, _ = target.ValueForProtocol(ma.P_UDP)\n\t\tif sport != \"\" {\n\t\t\treturn sport, nil\n\t\t}\n\t\treturn \"\", errors.New(\"address does not contain tcp or udp protocol\")\n\t}\n\n\tsport, err := getPort()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tport, err := strconv.Atoi(sport)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif port == 0 {\n\t\treturn errors.New(\"port can not be 0\")\n\t}\n\n\treturn nil\n}\n\n// forwardLocal forwards local connections to a libp2p service\nfunc forwardLocal(ctx context.Context, p *p2p.P2P, ps pstore.Peerstore, proto protocol.ID, bindAddr ma.Multiaddr, addr *peer.AddrInfo) (p2p.Listener, error) {\n\tps.AddAddrs(addr.ID, addr.Addrs, pstore.TempAddrTTL)\n\treturn p.ForwardLocal(ctx, addr.ID, proto, bindAddr)\n}\n\nconst (\n\tp2pHeadersOptionName = \"headers\"\n)\n\nvar p2pLsCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List active p2p listeners.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(p2pHeadersOptionName, \"v\", \"Print table headers (Protocol, Listen, Target).\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := p2pGetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toutput := &P2PLsOutput{}\n\n\t\tn.P2P.ListenersLocal.Lock()\n\t\tfor _, listener := range n.P2P.ListenersLocal.Listeners {\n\t\t\toutput.Listeners = append(output.Listeners, P2PListenerInfoOutput{\n\t\t\t\tProtocol: string(listener.Protocol()),\n\t\t\t\tListenAddress: listener.ListenAddress().String(),\n\t\t\t\tTargetAddress: listener.TargetAddress().String(),\n\t\t\t})\n\t\t}\n\t\tn.P2P.ListenersLocal.Unlock()\n\n\t\tn.P2P.ListenersP2P.Lock()\n\t\tfor _, listener := range n.P2P.ListenersP2P.Listeners {\n\t\t\toutput.Listeners = append(output.Listeners, P2PListenerInfoOutput{\n\t\t\t\tProtocol: string(listener.Protocol()),\n\t\t\t\tListenAddress: listener.ListenAddress().String(),\n\t\t\t\tTargetAddress: listener.TargetAddress().String(),\n\t\t\t})\n\t\t}\n\t\tn.P2P.ListenersP2P.Unlock()\n\n\t\treturn cmds.EmitOnce(res, output)\n\t},\n\tType: P2PLsOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *P2PLsOutput) error {\n\t\t\theaders, _ := req.Options[p2pHeadersOptionName].(bool)\n\t\t\ttw := tabwriter.NewWriter(w, 1, 2, 1, ' ', 0)\n\t\t\tfor _, listener := range out.Listeners {\n\t\t\t\tif headers {\n\t\t\t\t\tfmt.Fprintln(tw, \"Protocol\\tListen Address\\tTarget Address\")\n\t\t\t\t}\n\n\t\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\t%s\\n\", listener.Protocol, listener.ListenAddress, listener.TargetAddress)\n\t\t\t}\n\t\t\ttw.Flush()\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nconst (\n\tp2pAllOptionName = \"all\"\n\tp2pProtocolOptionName = \"protocol\"\n\tp2pListenAddressOptionName = \"listen-address\"\n\tp2pTargetAddressOptionName = \"target-address\"\n)\n\nvar p2pCloseCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Stop listening for new connections to forward.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(p2pAllOptionName, \"a\", \"Close all listeners.\"),\n\t\tcmds.StringOption(p2pProtocolOptionName, \"p\", \"Match protocol name\"),\n\t\tcmds.StringOption(p2pListenAddressOptionName, \"l\", \"Match listen address\"),\n\t\tcmds.StringOption(p2pTargetAddressOptionName, \"t\", \"Match target address\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := p2pGetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcloseAll, _ := req.Options[p2pAllOptionName].(bool)\n\t\tprotoOpt, p := req.Options[p2pProtocolOptionName].(string)\n\t\tlistenOpt, l := req.Options[p2pListenAddressOptionName].(string)\n\t\ttargetOpt, t := req.Options[p2pTargetAddressOptionName].(string)\n\n\t\tproto := protocol.ID(protoOpt)\n\n\t\tvar target, listen ma.Multiaddr\n\n\t\tif l {\n\t\t\tlisten, err = ma.NewMultiaddr(listenOpt)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif t {\n\t\t\ttarget, err = ma.NewMultiaddr(targetOpt)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif !(closeAll || p || l || t) {\n\t\t\treturn errors.New(\"no matching options given\")\n\t\t}\n\n\t\tif closeAll && (p || l || t) {\n\t\t\treturn errors.New(\"can't combine --all with other matching options\")\n\t\t}\n\n\t\tmatch := func(listener p2p.Listener) bool {\n\t\t\tif closeAll {\n\t\t\t\treturn true\n\t\t\t}\n\t\t\tif p && proto != listener.Protocol() {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tif l && !listen.Equal(listener.ListenAddress()) {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tif t && !target.Equal(listener.TargetAddress()) {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\treturn true\n\t\t}\n\n\t\tdone := n.P2P.ListenersLocal.Close(match)\n\t\tdone += n.P2P.ListenersP2P.Close(match)\n\n\t\treturn cmds.EmitOnce(res, done)\n\t},\n\tType: int(0),\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out int) error {\n\t\t\tfmt.Fprintf(w, \"Closed %d stream(s)\\n\", out)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n///////\n// Stream\n//\n\n// p2pStreamCmd is the 'ipfs p2p stream' command\nvar p2pStreamCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"P2P stream management.\",\n\t\tShortDescription: \"Create and manage p2p streams\",\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"ls\": p2pStreamLsCmd,\n\t\t\"close\": p2pStreamCloseCmd,\n\t},\n}\n\nvar p2pStreamLsCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List active p2p streams.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(p2pHeadersOptionName, \"v\", \"Print table headers (ID, Protocol, Local, Remote).\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := p2pGetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toutput := &P2PStreamsOutput{}\n\n\t\tn.P2P.Streams.Lock()\n\t\tfor id, s := range n.P2P.Streams.Streams {\n\t\t\toutput.Streams = append(output.Streams, P2PStreamInfoOutput{\n\t\t\t\tHandlerID: strconv.FormatUint(id, 10),\n\n\t\t\t\tProtocol: string(s.Protocol),\n\n\t\t\t\tOriginAddress: s.OriginAddr.String(),\n\t\t\t\tTargetAddress: s.TargetAddr.String(),\n\t\t\t})\n\t\t}\n\t\tn.P2P.Streams.Unlock()\n\n\t\treturn cmds.EmitOnce(res, output)\n\t},\n\tType: P2PStreamsOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *P2PStreamsOutput) error {\n\t\t\theaders, _ := req.Options[p2pHeadersOptionName].(bool)\n\t\t\ttw := tabwriter.NewWriter(w, 1, 2, 1, ' ', 0)\n\t\t\tfor _, stream := range out.Streams {\n\t\t\t\tif headers {\n\t\t\t\t\tfmt.Fprintln(tw, \"ID\\tProtocol\\tOrigin\\tTarget\")\n\t\t\t\t}\n\n\t\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\t%s\\t%s\\n\", stream.HandlerID, stream.Protocol, stream.OriginAddress, stream.TargetAddress)\n\t\t\t}\n\t\t\ttw.Flush()\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar p2pStreamCloseCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Close active p2p stream.\",\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"id\", false, false, \"Stream identifier\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(p2pAllOptionName, \"a\", \"Close all streams.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := p2pGetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcloseAll, _ := req.Options[p2pAllOptionName].(bool)\n\t\tvar handlerID uint64\n\n\t\tif !closeAll {\n\t\t\tif len(req.Arguments) == 0 {\n\t\t\t\treturn errors.New(\"no id specified\")\n\t\t\t}\n\n\t\t\thandlerID, err = strconv.ParseUint(req.Arguments[0], 10, 64)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\ttoClose := make([]*p2p.Stream, 0, 1)\n\t\tn.P2P.Streams.Lock()\n\t\tfor id, stream := range n.P2P.Streams.Streams {\n\t\t\tif !closeAll && handlerID != id {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\ttoClose = append(toClose, stream)\n\t\t\tif !closeAll {\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tn.P2P.Streams.Unlock()\n\n\t\tfor _, s := range toClose {\n\t\t\tn.P2P.Streams.Reset(s)\n\t\t}\n\n\t\treturn nil\n\t},\n}\n\nfunc p2pGetNode(env cmds.Environment) (*core.IpfsNode, error) {\n\tnd, err := cmdenv.GetNode(env)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tconfig, err := nd.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif !config.Experimental.Libp2pStreamMounting {\n\t\treturn nil, errors.New(\"libp2p stream mounting not enabled\")\n\t}\n\n\tif !nd.IsOnline {\n\t\treturn nil, ErrNotOnline\n\t}\n\n\treturn nd, nil\n}\n" }, { "path": "core/commands/pin/pin.go", "content": "package pin\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/dustin/go-humanize\"\n\tbserv \"github.com/ipfs/boxo/blockservice\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\tverifcid \"github.com/ipfs/boxo/verifcid\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\te \"github.com/ipfs/kubo/core/commands/e\"\n)\n\nvar PinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Pin (and unpin) objects to local storage.\",\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"add\": addPinCmd,\n\t\t\"rm\": rmPinCmd,\n\t\t\"ls\": listPinCmd,\n\t\t\"verify\": verifyPinCmd,\n\t\t\"update\": updatePinCmd,\n\t\t\"remote\": remotePinCmd,\n\t},\n}\n\ntype PinOutput struct {\n\tPins []string\n}\n\ntype AddPinOutput struct {\n\tPins []string `json:\",omitempty\"`\n\tProgress int `json:\",omitempty\"`\n\tBytes uint64 `json:\",omitempty\"`\n}\n\nconst (\n\tpinRecursiveOptionName = \"recursive\"\n\tpinProgressOptionName = \"progress\"\n)\n\nvar addPinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Pin objects to local storage.\",\n\t\tShortDescription: \"Stores an IPFS object(s) from a given path locally to disk.\",\n\t\tLongDescription: `\nCreate a pin for the given object, protecting resolved CID from being garbage\ncollected.\n\nAn optional name can be provided, and read back via 'ipfs pin ls --names'.\n\nBe mindful of defaults:\n\nDefault pin type is 'recursive' (entire DAG).\nPass -r=false to create a direct pin for a single block.\nUse 'pin ls -t recursive' to only list roots of recursively pinned DAGs\n(significantly faster when many big DAGs are pinned recursively)\n\nDefault pin name is empty. Pass '--name' to 'pin add' to set one\nand use 'pin ls --names' to see it. Pinning a second time with a different\nname will update the name of the pin.\n\nIf daemon is running, any missing blocks will be retrieved from the network.\nIt may take some time. Pass '--progress' to track the progress.\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, true, \"Path to object(s) to be pinned.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(pinRecursiveOptionName, \"r\", \"Recursively pin the object linked to by the specified object(s).\").WithDefault(true),\n\t\tcmds.StringOption(pinNameOptionName, \"n\", \"An optional name for created pin(s).\"),\n\t\tcmds.BoolOption(pinProgressOptionName, \"Show progress\"),\n\t},\n\tType: AddPinOutput{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// set recursive flag\n\t\trecursive, _ := req.Options[pinRecursiveOptionName].(bool)\n\t\tname, _ := req.Options[pinNameOptionName].(string)\n\t\tshowProgress, _ := req.Options[pinProgressOptionName].(bool)\n\n\t\t// Validate pin name\n\t\tif err := cmdutils.ValidatePinName(name); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !showProgress {\n\t\t\tadded, err := pinAddMany(req.Context, api, enc, req.Arguments, recursive, name)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\treturn cmds.EmitOnce(res, &AddPinOutput{Pins: added})\n\t\t}\n\n\t\tv := new(dag.ProgressTracker)\n\t\tctx := v.DeriveContext(req.Context)\n\n\t\ttype pinResult struct {\n\t\t\tpins []string\n\t\t\terr error\n\t\t}\n\n\t\tch := make(chan pinResult, 1)\n\t\tgo func() {\n\t\t\tadded, err := pinAddMany(ctx, api, enc, req.Arguments, recursive, name)\n\t\t\tch <- pinResult{pins: added, err: err}\n\t\t}()\n\n\t\tticker := time.NewTicker(500 * time.Millisecond)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase val := <-ch:\n\t\t\t\tif val.err != nil {\n\t\t\t\t\treturn val.err\n\t\t\t\t}\n\n\t\t\t\tif ps := v.ProgressStat(); ps.Nodes != 0 {\n\t\t\t\t\tif err := res.Emit(&AddPinOutput{Progress: ps.Nodes, Bytes: ps.Bytes}); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn res.Emit(&AddPinOutput{Pins: val.pins})\n\t\t\tcase <-ticker.C:\n\t\t\t\tps := v.ProgressStat()\n\t\t\t\tif err := res.Emit(&AddPinOutput{Progress: ps.Nodes, Bytes: ps.Bytes}); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\tcase <-ctx.Done():\n\t\t\t\tlog.Error(ctx.Err())\n\t\t\t\treturn ctx.Err()\n\t\t\t}\n\t\t}\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *AddPinOutput) error {\n\t\t\trec, found := req.Options[\"recursive\"].(bool)\n\t\t\tvar pintype string\n\t\t\tif rec || !found {\n\t\t\t\tpintype = \"recursively\"\n\t\t\t} else {\n\t\t\t\tpintype = \"directly\"\n\t\t\t}\n\n\t\t\tfor _, k := range out.Pins {\n\t\t\t\tfmt.Fprintf(w, \"pinned %s %s\\n\", k, pintype)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tfor {\n\t\t\t\tv, err := res.Next()\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err == io.EOF {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tout, ok := v.(*AddPinOutput)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn e.TypeErr(out, v)\n\t\t\t\t}\n\t\t\t\tif out.Pins == nil {\n\t\t\t\t\t// this can only happen if the progress option is set\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"Fetched/Processed %d nodes (%s)\\r\", out.Progress, humanize.Bytes(out.Bytes))\n\t\t\t\t} else {\n\t\t\t\t\terr = re.Emit(out)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t},\n}\n\nfunc pinAddMany(ctx context.Context, api coreiface.CoreAPI, enc cidenc.Encoder, paths []string, recursive bool, name string) ([]string, error) {\n\tadded := make([]string, len(paths))\n\tfor i, b := range paths {\n\t\tp, err := cmdutils.PathOrCidPath(b)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\trp, _, err := api.ResolvePath(ctx, p)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := api.Pin().Add(ctx, rp, options.Pin.Recursive(recursive), options.Pin.Name(name)); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tadded[i] = enc.Encode(rp.RootCid())\n\t}\n\n\treturn added, nil\n}\n\nvar rmPinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove object from pin-list.\",\n\t\tShortDescription: `\nRemoves the pin from the given object allowing it to be garbage\ncollected if needed. (By default, recursively. Use -r=false for direct pins.)\n`,\n\t\tLongDescription: `\nRemoves the pin from the given object allowing it to be garbage\ncollected if needed. (By default, recursively. Use -r=false for direct pins.)\n\nA pin may not be removed because the specified object is not pinned or pinned\nindirectly. To determine if the object is pinned indirectly, use the command:\nipfs pin ls -t indirect \n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, true, \"Path to object(s) to be unpinned.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(pinRecursiveOptionName, \"r\", \"Recursively unpin the object linked to by the specified object(s).\").WithDefault(true),\n\t},\n\tType: PinOutput{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// set recursive flag\n\t\trecursive, _ := req.Options[pinRecursiveOptionName].(bool)\n\n\t\tif err := req.ParseBodyArgs(); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpins := make([]string, 0, len(req.Arguments))\n\t\tfor _, b := range req.Arguments {\n\t\t\tp, err := cmdutils.PathOrCidPath(b)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\trp, _, err := api.ResolvePath(req.Context, p)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tid := enc.Encode(rp.RootCid())\n\t\t\tpins = append(pins, id)\n\t\t\tif err := api.Pin().Rm(req.Context, rp, options.Pin.RmRecursive(recursive)); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &PinOutput{pins})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *PinOutput) error {\n\t\t\tfor _, k := range out.Pins {\n\t\t\t\tfmt.Fprintf(w, \"unpinned %s\\n\", k)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nconst (\n\tpinTypeOptionName = \"type\"\n\tpinQuietOptionName = \"quiet\"\n\tpinStreamOptionName = \"stream\"\n\tpinNamesOptionName = \"names\"\n)\n\nvar listPinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List objects pinned to local storage.\",\n\t\tShortDescription: `\nReturns a list of objects that are pinned locally.\nBy default, all pinned objects are returned, but the '--type' flag or\narguments can restrict that to a specific pin type or to some specific objects\nrespectively.\n`,\n\t\tLongDescription: `\nReturns a list of objects that are pinned locally.\n\nBy default, all pinned objects are returned, but the '--type' flag or\narguments can restrict that to a specific pin type or to some specific objects\nrespectively.\n\nUse --type= to specify the type of pinned keys to list.\nValid values are:\n * \"direct\": pin that specific object.\n * \"recursive\": pin that specific object, and indirectly pin all its\n descendants\n * \"indirect\": pinned indirectly by an ancestor (like a refcount)\n * \"all\"\n\nBy default, pin names are not included (returned as empty).\nPass '--names' flag to return pin names (set with '--name' from 'pin add').\n\nWith arguments, the command fails if any of the arguments is not a pinned\nobject. And if --type= is additionally used, the command will also fail\nif any of the arguments is not of the specified type.\n\nExample:\n\t$ echo \"hello\" | ipfs add -q\n\tQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\n\t$ ipfs pin ls\n\tQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN recursive\n\t# now remove the pin, and repin it directly\n\t$ ipfs pin rm QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\n\tunpinned QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\n\t$ ipfs pin add -r=false QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\n\tpinned QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN directly\n\t$ ipfs pin ls --type=direct\n\tQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN direct\n\t$ ipfs pin ls QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\n\tQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN direct\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", false, true, \"Path to object(s) to be listed.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(pinTypeOptionName, \"t\", \"The type of pinned keys to list. Can be \\\"direct\\\", \\\"indirect\\\", \\\"recursive\\\", or \\\"all\\\".\").WithDefault(\"all\"),\n\t\tcmds.BoolOption(pinQuietOptionName, \"q\", \"Output only the CIDs of pins.\"),\n\t\tcmds.StringOption(pinNameOptionName, \"n\", \"Limit returned pins to ones with names that contain the value provided (case-sensitive, partial match). Implies --names=true.\"),\n\t\tcmds.BoolOption(pinStreamOptionName, \"s\", \"Enable streaming of pins as they are discovered.\"),\n\t\tcmds.BoolOption(pinNamesOptionName, \"Include pin names in the output (slower, disabled by default).\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif n.Pinning == nil {\n\t\t\treturn fmt.Errorf(\"pinning service not available\")\n\t\t}\n\n\t\ttypeStr, _ := req.Options[pinTypeOptionName].(string)\n\t\tstream, _ := req.Options[pinStreamOptionName].(bool)\n\t\tdisplayNames, _ := req.Options[pinNamesOptionName].(bool)\n\t\tname, _ := req.Options[pinNameOptionName].(string)\n\n\t\t// Validate name filter\n\t\tif err := cmdutils.ValidatePinName(name); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tmode, ok := pin.StringToMode(typeStr)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"invalid type '%s', must be one of {direct, indirect, recursive, all}\", typeStr)\n\t\t}\n\n\t\t// For backward compatibility, we accumulate the pins in the same output type as before.\n\t\tvar emit func(PinLsOutputWrapper) error\n\t\tlgcList := map[string]PinLsType{}\n\t\tif !stream {\n\t\t\temit = func(v PinLsOutputWrapper) error {\n\t\t\t\tlgcList[v.PinLsObject.Cid] = PinLsType{Type: v.PinLsObject.Type, Name: v.PinLsObject.Name}\n\t\t\t\treturn nil\n\t\t\t}\n\t\t} else {\n\t\t\temit = func(v PinLsOutputWrapper) error {\n\t\t\t\treturn res.Emit(v)\n\t\t\t}\n\t\t}\n\n\t\tif len(req.Arguments) > 0 {\n\t\t\terr = pinLsKeys(req, mode, displayNames || name != \"\", n.Pinning, api, emit)\n\t\t} else {\n\t\t\terr = pinLsAll(req, typeStr, displayNames || name != \"\", name, api, emit)\n\t\t}\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !stream {\n\t\t\treturn cmds.EmitOnce(res, PinLsOutputWrapper{\n\t\t\t\tPinLsList: PinLsList{Keys: lgcList},\n\t\t\t})\n\t\t}\n\n\t\treturn nil\n\t},\n\tType: PinLsOutputWrapper{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.JSON: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out PinLsOutputWrapper) error {\n\t\t\tstream, _ := req.Options[pinStreamOptionName].(bool)\n\n\t\t\tenc := json.NewEncoder(w)\n\n\t\t\tif stream {\n\t\t\t\treturn enc.Encode(out.PinLsObject)\n\t\t\t}\n\n\t\t\treturn enc.Encode(out.PinLsList)\n\t\t}),\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out PinLsOutputWrapper) error {\n\t\t\tquiet, _ := req.Options[pinQuietOptionName].(bool)\n\t\t\tstream, _ := req.Options[pinStreamOptionName].(bool)\n\n\t\t\tif stream {\n\t\t\t\tif quiet {\n\t\t\t\t\tfmt.Fprintf(w, \"%s\\n\", out.PinLsObject.Cid)\n\t\t\t\t} else if out.PinLsObject.Name == \"\" {\n\t\t\t\t\tfmt.Fprintf(w, \"%s %s\\n\", out.PinLsObject.Cid, out.PinLsObject.Type)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(w, \"%s %s %s\\n\", out.PinLsObject.Cid, out.PinLsObject.Type, out.PinLsObject.Name)\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tfor k, v := range out.PinLsList.Keys {\n\t\t\t\tif quiet {\n\t\t\t\t\tfmt.Fprintf(w, \"%s\\n\", k)\n\t\t\t\t} else if v.Name == \"\" {\n\t\t\t\t\tfmt.Fprintf(w, \"%s %s\\n\", k, v.Type)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(w, \"%s %s %s\\n\", k, v.Type, v.Name)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// PinLsOutputWrapper is the output type of the pin ls command.\n// Pin ls needs to output two different type depending on if it's streamed or not.\n// We use this to bypass the cmds lib refusing to have interface{}\ntype PinLsOutputWrapper struct {\n\tPinLsList\n\tPinLsObject\n}\n\n// PinLsList is a set of pins with their type\ntype PinLsList struct {\n\tKeys map[string]PinLsType `json:\",omitempty\"`\n}\n\n// PinLsType contains the type of a pin\ntype PinLsType struct {\n\tType string\n\tName string\n}\n\n// PinLsObject contains the description of a pin\ntype PinLsObject struct {\n\tCid string `json:\",omitempty\"`\n\tName string `json:\",omitempty\"`\n\tType string `json:\",omitempty\"`\n}\n\nfunc pinLsKeys(req *cmds.Request, mode pin.Mode, displayNames bool, pinner pin.Pinner, api coreiface.CoreAPI, emit func(value PinLsOutputWrapper) error) error {\n\tenc, err := cmdenv.GetCidEncoder(req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Collect CIDs to check\n\tcids := make([]cid.Cid, 0, len(req.Arguments))\n\tfor _, p := range req.Arguments {\n\t\tp, err := cmdutils.PathOrCidPath(p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trp, _, err := api.ResolvePath(req.Context, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcids = append(cids, rp.RootCid())\n\t}\n\n\t// Check pins using the new type-specific method\n\tpinned, err := pinner.CheckIfPinnedWithType(req.Context, mode, displayNames, cids...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Process results\n\tfor i, p := range pinned {\n\t\tif !p.Pinned() {\n\t\t\treturn fmt.Errorf(\"path '%s' is not pinned\", req.Arguments[i])\n\t\t}\n\n\t\tpinType, _ := pin.ModeToString(p.Mode)\n\t\tif p.Mode == pin.Indirect && p.Via.Defined() {\n\t\t\tpinType = \"indirect through \" + enc.Encode(p.Via)\n\t\t}\n\n\t\terr = emit(PinLsOutputWrapper{\n\t\t\tPinLsObject: PinLsObject{\n\t\t\t\tType: pinType,\n\t\t\t\tCid: enc.Encode(cids[i]),\n\t\t\t\tName: p.Name,\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc pinLsAll(req *cmds.Request, typeStr string, detailed bool, name string, api coreiface.CoreAPI, emit func(value PinLsOutputWrapper) error) error {\n\tenc, err := cmdenv.GetCidEncoder(req)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, ok := pin.StringToMode(typeStr)\n\tif !ok {\n\t\treturn fmt.Errorf(\"invalid type '%s', must be one of {direct, indirect, recursive, all}\", typeStr)\n\t}\n\n\topt, err := options.Pin.Ls.Type(typeStr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tpins := make(chan coreiface.Pin)\n\tlsErr := make(chan error, 1)\n\tlsCtx, cancel := context.WithCancel(req.Context)\n\tdefer cancel()\n\n\tgo func() {\n\t\tlsErr <- api.Pin().Ls(lsCtx, pins, opt, options.Pin.Ls.Detailed(detailed), options.Pin.Ls.Name(name))\n\t}()\n\n\tfor p := range pins {\n\t\terr = emit(PinLsOutputWrapper{\n\t\t\tPinLsObject: PinLsObject{\n\t\t\t\tType: p.Type(),\n\t\t\t\tName: p.Name(),\n\t\t\t\tCid: enc.Encode(p.Path().RootCid()),\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn <-lsErr\n}\n\nconst (\n\tpinUnpinOptionName = \"unpin\"\n)\n\nvar updatePinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Update a recursive pin.\",\n\t\tShortDescription: `\nEfficiently pins a new object based on differences from an existing one and,\nby default, removes the old pin.\n\nThis command is useful when the new pin contains many similarities or is a\nderivative of an existing one, particularly for large objects. This allows a more\nefficient DAG-traversal which fully skips already-pinned branches from the old\nobject. As a requirement, the old object needs to be an existing recursive\npin.\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"from-path\", true, false, \"Path to old object.\"),\n\t\tcmds.StringArg(\"to-path\", true, false, \"Path to a new object to be pinned.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(pinUnpinOptionName, \"Remove the old pin.\").WithDefault(true),\n\t},\n\tType: PinOutput{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tunpin, _ := req.Options[pinUnpinOptionName].(bool)\n\n\t\tfromPath, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttoPath, err := cmdutils.PathOrCidPath(req.Arguments[1])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Resolve the paths ahead of time so we can return the actual CIDs\n\t\tfrom, _, err := api.ResolvePath(req.Context, fromPath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tto, _, err := api.ResolvePath(req.Context, toPath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\terr = api.Pin().Update(req.Context, from, to, options.Pin.Unpin(unpin))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &PinOutput{Pins: []string{enc.Encode(from.RootCid()), enc.Encode(to.RootCid())}})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *PinOutput) error {\n\t\t\tfmt.Fprintf(w, \"updated %s to %s\\n\", out.Pins[0], out.Pins[1])\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nconst (\n\tpinVerboseOptionName = \"verbose\"\n)\n\nvar verifyPinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Verify that recursive pins are complete.\",\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(pinVerboseOptionName, \"Also write the hashes of non-broken pins.\"),\n\t\tcmds.BoolOption(pinQuietOptionName, \"q\", \"Write just hashes of broken pins.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tverbose, _ := req.Options[pinVerboseOptionName].(bool)\n\t\tquiet, _ := req.Options[pinQuietOptionName].(bool)\n\n\t\tif verbose && quiet {\n\t\t\treturn fmt.Errorf(\"the --verbose and --quiet options can not be used at the same time\")\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\topts := pinVerifyOpts{\n\t\t\texplain: !quiet,\n\t\t\tincludeOk: verbose,\n\t\t}\n\t\tout, err := pinVerify(req.Context, n, opts, enc)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn res.Emit(out)\n\t},\n\tType: PinVerifyRes{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *PinVerifyRes) error {\n\t\t\tquiet, _ := req.Options[pinQuietOptionName].(bool)\n\n\t\t\tif quiet && !out.Ok {\n\t\t\t\tfmt.Fprintf(w, \"%s\\n\", out.Cid)\n\t\t\t} else if !quiet {\n\t\t\t\tout.Format(w)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// PinVerifyRes is the result returned for each pin checked in \"pin verify\"\ntype PinVerifyRes struct {\n\tCid string `json:\",omitempty\"`\n\tErr string `json:\",omitempty\"`\n\tPinStatus\n}\n\n// PinStatus is part of PinVerifyRes, do not use directly\ntype PinStatus struct {\n\tOk bool `json:\",omitempty\"`\n\tBadNodes []BadNode `json:\",omitempty\"`\n}\n\n// BadNode is used in PinVerifyRes\ntype BadNode struct {\n\tCid string\n\tErr string\n}\n\ntype pinVerifyOpts struct {\n\texplain bool\n\tincludeOk bool\n}\n\n// FIXME: this implementation is duplicated sith core/coreapi.PinAPI.Verify, remove this one and exclusively rely on CoreAPI.\nfunc pinVerify(ctx context.Context, n *core.IpfsNode, opts pinVerifyOpts, enc cidenc.Encoder) (<-chan any, error) {\n\tvisited := make(map[cid.Cid]PinStatus)\n\n\tbs := n.Blocks.Blockstore()\n\tDAG := dag.NewDAGService(bserv.New(bs, offline.Exchange(bs)))\n\tgetLinks := dag.GetLinksWithDAG(DAG)\n\n\tvar checkPin func(root cid.Cid) PinStatus\n\tcheckPin = func(root cid.Cid) PinStatus {\n\t\tkey := root\n\t\tif status, ok := visited[key]; ok {\n\t\t\treturn status\n\t\t}\n\n\t\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, root); err != nil {\n\t\t\tstatus := PinStatus{Ok: false}\n\t\t\tif opts.explain {\n\t\t\t\tstatus.BadNodes = []BadNode{{Cid: enc.Encode(key), Err: err.Error()}}\n\t\t\t}\n\t\t\tvisited[key] = status\n\t\t\treturn status\n\t\t}\n\n\t\tlinks, err := getLinks(ctx, root)\n\t\tif err != nil {\n\t\t\tstatus := PinStatus{Ok: false}\n\t\t\tif opts.explain {\n\t\t\t\tstatus.BadNodes = []BadNode{{Cid: enc.Encode(key), Err: err.Error()}}\n\t\t\t}\n\t\t\tvisited[key] = status\n\t\t\treturn status\n\t\t}\n\n\t\tstatus := PinStatus{Ok: true}\n\t\tfor _, lnk := range links {\n\t\t\tres := checkPin(lnk.Cid)\n\t\t\tif !res.Ok {\n\t\t\t\tstatus.Ok = false\n\t\t\t\tstatus.BadNodes = append(status.BadNodes, res.BadNodes...)\n\t\t\t}\n\t\t}\n\n\t\tvisited[key] = status\n\t\treturn status\n\t}\n\n\tout := make(chan any)\n\tgo func() {\n\t\tdefer close(out)\n\t\tfor p := range n.Pinning.RecursiveKeys(ctx, false) {\n\t\t\tif p.Err != nil {\n\t\t\t\tout <- PinVerifyRes{Err: p.Err.Error()}\n\t\t\t\treturn\n\t\t\t}\n\t\t\tpinStatus := checkPin(p.Pin.Key)\n\t\t\tif !pinStatus.Ok || opts.includeOk {\n\t\t\t\tselect {\n\t\t\t\tcase out <- PinVerifyRes{Cid: enc.Encode(p.Pin.Key), PinStatus: pinStatus}:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn out, nil\n}\n\n// Format formats PinVerifyRes\nfunc (r PinVerifyRes) Format(out io.Writer) {\n\tif r.Err != \"\" {\n\t\tfmt.Fprintf(out, \"error: %s\\n\", r.Err)\n\t\treturn\n\t}\n\n\tif r.Ok {\n\t\tfmt.Fprintf(out, \"%s ok\\n\", r.Cid)\n\t\treturn\n\t}\n\n\tfmt.Fprintf(out, \"%s broken\\n\", r.Cid)\n\tfor _, e := range r.BadNodes {\n\t\tfmt.Fprintf(out, \" %s: %s\\n\", e.Cid, e.Err)\n\t}\n}\n" }, { "path": "core/commands/pin/remotepin.go", "content": "package pin\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\tneturl \"net/url\"\n\tgopath \"path\"\n\n\t\"golang.org/x/sync/errgroup\"\n\n\tpinclient \"github.com/ipfs/boxo/pinning/remote/client\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nvar log = logging.Logger(\"core/commands/cmdenv\")\n\nvar remotePinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Pin (and unpin) objects to remote pinning service.\",\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"add\": addRemotePinCmd,\n\t\t\"ls\": listRemotePinCmd,\n\t\t\"rm\": rmRemotePinCmd,\n\t\t\"service\": remotePinServiceCmd,\n\t},\n}\n\nvar remotePinServiceCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Configure remote pinning services.\",\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"add\": addRemotePinServiceCmd,\n\t\t\"ls\": lsRemotePinServiceCmd,\n\t\t\"rm\": rmRemotePinServiceCmd,\n\t},\n}\n\nconst (\n\tpinNameOptionName = \"name\"\n\tpinCIDsOptionName = \"cid\"\n\tpinStatusOptionName = \"status\"\n\tpinServiceNameOptionName = \"service\"\n\tpinServiceNameArgName = pinServiceNameOptionName\n\tpinServiceEndpointArgName = \"endpoint\"\n\tpinServiceKeyArgName = \"key\"\n\tpinServiceStatOptionName = \"stat\"\n\tpinBackgroundOptionName = \"background\"\n\tpinForceOptionName = \"force\"\n)\n\ntype RemotePinOutput struct {\n\tStatus string\n\tCid string\n\tName string\n}\n\nfunc toRemotePinOutput(ps pinclient.PinStatusGetter) RemotePinOutput {\n\treturn RemotePinOutput{\n\t\tName: ps.GetPin().GetName(),\n\t\tStatus: ps.GetStatus().String(),\n\t\tCid: ps.GetPin().GetCid().String(),\n\t}\n}\n\nfunc printRemotePinDetails(w io.Writer, out *RemotePinOutput) {\n\ttw := tabwriter.NewWriter(w, 0, 0, 1, ' ', 0)\n\tdefer tw.Flush()\n\tfw := func(k string, v string) {\n\t\tfmt.Fprintf(tw, \"%s:\\t%s\\n\", k, v)\n\t}\n\tfw(\"CID\", out.Cid)\n\tfw(\"Name\", out.Name)\n\tfw(\"Status\", out.Status)\n}\n\n// remote pin commands\n\nvar pinServiceNameOption = cmds.StringOption(pinServiceNameOptionName, \"Name of the remote pinning service to use (mandatory).\")\n\nvar addRemotePinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Pin object to remote pinning service.\",\n\t\tShortDescription: \"Asks remote pinning service to pin an IPFS object from a given path.\",\n\t\tLongDescription: `\nAsks remote pinning service to pin an IPFS object from a given path or a CID.\n\nTo pin CID 'bafkqaaa' to service named 'mysrv' under a pin named 'mypin':\n\n $ ipfs pin remote add --service=mysrv --name=mypin bafkqaaa\n\nThe above command will block until remote service returns 'pinned' status,\nwhich may take time depending on the size and available providers of the pinned\ndata.\n\nIf you prefer to not wait for pinning confirmation and return immediately\nafter remote service confirms 'queued' status, add the '--background' flag:\n\n $ ipfs pin remote add --service=mysrv --name=mypin --background bafkqaaa\n\nStatus of background pin requests can be inspected with the 'ls' command.\n\nTo list all pins for the CID across all statuses:\n\n $ ipfs pin remote ls --service=mysrv --cid=bafkqaaa --status=queued \\\n --status=pinning --status=pinned --status=failed\n\nNOTE: a comma-separated notation is supported in CLI for convenience:\n\n $ ipfs pin remote ls --service=mysrv --cid=bafkqaaa --status=queued,pinning,pinned,failed\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, false, \"CID or Path to be pinned.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tpinServiceNameOption,\n\t\tcmds.StringOption(pinNameOptionName, \"An optional name for the pin.\"),\n\t\tcmds.BoolOption(pinBackgroundOptionName, \"Add to the queue on the remote service and return immediately (does not wait for pinned status).\").WithDefault(false),\n\t},\n\tType: RemotePinOutput{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tdefer cancel()\n\n\t\t// Get remote service\n\t\tc, err := getRemotePinServiceFromRequest(req, env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Prepare value for Pin.cid\n\t\tif len(req.Arguments) != 1 {\n\t\t\treturn fmt.Errorf(\"expecting one CID argument\")\n\t\t}\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tp, err := cmdutils.PathOrCidPath(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trp, _, err := api.ResolvePath(ctx, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Prepare Pin.name\n\t\topts := []pinclient.AddOption{}\n\t\tif name, nameFound := req.Options[pinNameOptionName]; nameFound {\n\t\t\tnameStr := name.(string)\n\t\t\t// Validate pin name\n\t\t\tif err := cmdutils.ValidatePinName(nameStr); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\topts = append(opts, pinclient.PinOpts.WithName(nameStr))\n\t\t}\n\n\t\t// Prepare Pin.origins\n\t\t// If CID in blockstore, add own multiaddrs to the 'origins' array\n\t\t// so pinning service can use that as a hint and connect back to us.\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tisInBlockstore, err := node.Blockstore.Has(req.Context, rp.RootCid())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif isInBlockstore && node.PeerHost != nil {\n\t\t\taddrs, err := peer.AddrInfoToP2pAddrs(host.InfoFromHost(node.PeerHost))\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\topts = append(opts, pinclient.PinOpts.WithOrigins(addrs...))\n\t\t} else if isInBlockstore && !node.IsOnline && cmds.GetEncoding(req, cmds.Text) == cmds.Text {\n\t\t\tfmt.Fprintf(os.Stdout, \"WARNING: the local node is offline and remote pinning may fail if there is no other provider for this CID\\n\")\n\t\t}\n\n\t\t// Execute remote pin request\n\t\t// TODO: fix panic when pinning service is down\n\t\tps, err := c.Add(ctx, rp.RootCid(), opts...)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Act on PinStatus.delegates\n\t\t// If Pinning Service returned any delegates, proactively try to\n\t\t// connect to them to facilitate data exchange without waiting for DHT\n\t\t// lookup\n\t\tfor _, d := range ps.GetDelegates() {\n\t\t\t// TODO: confirm this works as expected\n\t\t\tp, err := peer.AddrInfoFromP2pAddr(d)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif err := api.Swarm().Connect(ctx, *p); err != nil {\n\t\t\t\tlog.Infof(\"error connecting to remote pin delegate %v : %w\", d, err)\n\t\t\t}\n\t\t}\n\n\t\t// Block unless --background=true is passed\n\t\tif !req.Options[pinBackgroundOptionName].(bool) {\n\t\t\tconst pinWaitTime = 500 * time.Millisecond\n\t\t\tvar timer *time.Timer\n\t\t\trequestID := ps.GetRequestId()\n\t\t\tfor {\n\t\t\t\tps, err = c.GetStatusByID(ctx, requestID)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"failed to check pin status for requestid=%q due to error: %v\", requestID, err)\n\t\t\t\t}\n\t\t\t\tif ps.GetRequestId() != requestID {\n\t\t\t\t\treturn fmt.Errorf(\"failed to check pin status for requestid=%q, remote service sent unexpected requestid=%q\", requestID, ps.GetRequestId())\n\t\t\t\t}\n\t\t\t\ts := ps.GetStatus()\n\t\t\t\tif s == pinclient.StatusPinned {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tif s == pinclient.StatusFailed {\n\t\t\t\t\treturn fmt.Errorf(\"remote service failed to pin requestid=%q\", requestID)\n\t\t\t\t}\n\t\t\t\tif timer == nil {\n\t\t\t\t\ttimer = time.NewTimer(pinWaitTime)\n\t\t\t\t} else {\n\t\t\t\t\ttimer.Reset(pinWaitTime)\n\t\t\t\t}\n\t\t\t\tselect {\n\t\t\t\tcase <-timer.C:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\ttimer.Stop()\n\t\t\t\t\treturn fmt.Errorf(\"waiting for pin interrupted, requestid=%q remains on remote service\", requestID)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn res.Emit(toRemotePinOutput(ps))\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *RemotePinOutput) error {\n\t\t\tprintRemotePinDetails(w, out)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar listRemotePinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List objects pinned to remote pinning service.\",\n\t\tShortDescription: `\nReturns a list of objects that are pinned to a remote pinning service.\n`,\n\t\tLongDescription: `\nReturns a list of objects that are pinned to a remote pinning service.\n\nNOTE: By default, it will only show matching objects in 'pinned' state.\nPass '--status=queued,pinning,pinned,failed' to list pins in all states.\n`,\n\t},\n\n\tArguments: []cmds.Argument{},\n\tOptions: []cmds.Option{\n\t\tpinServiceNameOption,\n\t\tcmds.StringOption(pinNameOptionName, \"Return pins with names that contain the value provided (case-sensitive, exact match).\"),\n\t\tcmds.DelimitedStringsOption(\",\", pinCIDsOptionName, \"Return pins for the specified CIDs (comma-separated).\"),\n\t\tcmds.DelimitedStringsOption(\",\", pinStatusOptionName, \"Return pins with the specified statuses (queued,pinning,pinned,failed).\").WithDefault([]string{\"pinned\"}),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tc, err := getRemotePinServiceFromRequest(req, env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tdefer cancel()\n\n\t\tpsCh := make(chan pinclient.PinStatusGetter)\n\t\tlsErr := make(chan error, 1)\n\t\tgo func() {\n\t\t\tlsErr <- lsRemote(ctx, req, c, psCh)\n\t\t}()\n\t\tfor ps := range psCh {\n\t\t\tif err := res.Emit(toRemotePinOutput(ps)); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn <-lsErr\n\t},\n\tType: RemotePinOutput{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *RemotePinOutput) error {\n\t\t\t// pin remote ls produces a flat output similar to legacy pin ls\n\t\t\tfmt.Fprintf(w, \"%s\\t%s\\t%s\\n\", out.Cid, out.Status, cmdenv.EscNonPrint(out.Name))\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// Executes GET /pins/?query-with-filters\nfunc lsRemote(ctx context.Context, req *cmds.Request, c *pinclient.Client, out chan<- pinclient.PinStatusGetter) error {\n\topts := []pinclient.LsOption{}\n\tif name, nameFound := req.Options[pinNameOptionName]; nameFound {\n\t\tnameStr := name.(string)\n\t\t// Validate name filter\n\t\tif err := cmdutils.ValidatePinName(nameStr); err != nil {\n\t\t\tclose(out)\n\t\t\treturn err\n\t\t}\n\t\topts = append(opts, pinclient.PinOpts.FilterName(nameStr))\n\t}\n\n\tif cidsRaw, cidsFound := req.Options[pinCIDsOptionName]; cidsFound {\n\t\tcidsRawArr := cidsRaw.([]string)\n\t\tvar parsedCIDs []cid.Cid\n\t\tfor _, rawCID := range cidsRawArr {\n\t\t\tparsedCID, err := cid.Decode(rawCID)\n\t\t\tif err != nil {\n\t\t\t\tclose(out)\n\t\t\t\treturn fmt.Errorf(\"CID %q cannot be parsed: %v\", rawCID, err)\n\t\t\t}\n\t\t\tparsedCIDs = append(parsedCIDs, parsedCID)\n\t\t}\n\t\topts = append(opts, pinclient.PinOpts.FilterCIDs(parsedCIDs...))\n\t}\n\tif statusRaw, statusFound := req.Options[pinStatusOptionName]; statusFound {\n\t\tstatusRawArr := statusRaw.([]string)\n\t\tvar parsedStatuses []pinclient.Status\n\t\tfor _, rawStatus := range statusRawArr {\n\t\t\ts := pinclient.Status(rawStatus)\n\t\t\tif s.String() == string(pinclient.StatusUnknown) {\n\t\t\t\tclose(out)\n\t\t\t\treturn fmt.Errorf(\"status %q is not valid\", rawStatus)\n\t\t\t}\n\t\t\tparsedStatuses = append(parsedStatuses, s)\n\t\t}\n\t\topts = append(opts, pinclient.PinOpts.FilterStatus(parsedStatuses...))\n\t}\n\n\treturn c.Ls(ctx, out, opts...)\n}\n\nvar rmRemotePinCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove pins from remote pinning service.\",\n\t\tShortDescription: \"Removes the remote pin, allowing it to be garbage-collected if needed.\",\n\t\tLongDescription: `\nRemoves remote pins, allowing them to be garbage-collected if needed.\n\nThis command accepts the same search query parameters as 'ls', and it is good\npractice to execute 'ls' before 'rm' to confirm the list of pins to be removed.\n\nTo remove a single pin for a specific CID:\n\n $ ipfs pin remote ls --service=mysrv --cid=bafkqaaa\n $ ipfs pin remote rm --service=mysrv --cid=bafkqaaa\n\nWhen more than one pin matches the query on the remote service, an error is\nreturned. To confirm the removal of multiple pins, pass '--force':\n\n $ ipfs pin remote ls --service=mysrv --name=popular-name\n $ ipfs pin remote rm --service=mysrv --name=popular-name --force\n\nNOTE: When no '--status' is passed, implicit '--status=pinned' is used.\nTo list and then remove all pending pin requests, pass an explicit status list:\n\n $ ipfs pin remote ls --service=mysrv --status=queued,pinning,failed\n $ ipfs pin remote rm --service=mysrv --status=queued,pinning,failed --force\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{},\n\tOptions: []cmds.Option{\n\t\tpinServiceNameOption,\n\t\tcmds.StringOption(pinNameOptionName, \"Remove pins with names that contain provided value (case-sensitive, exact match).\"),\n\t\tcmds.DelimitedStringsOption(\",\", pinCIDsOptionName, \"Remove pins for the specified CIDs.\"),\n\t\tcmds.DelimitedStringsOption(\",\", pinStatusOptionName, \"Remove pins with the specified statuses (queued,pinning,pinned,failed).\").WithDefault([]string{\"pinned\"}),\n\t\tcmds.BoolOption(pinForceOptionName, \"Allow removal of multiple pins matching the query without additional confirmation.\").WithDefault(false),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tc, err := getRemotePinServiceFromRequest(req, env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trmIDs := []string{}\n\t\tif len(req.Arguments) != 0 {\n\t\t\treturn fmt.Errorf(\"unexpected argument %q\", req.Arguments[0])\n\t\t}\n\n\t\tpsCh := make(chan pinclient.PinStatusGetter)\n\t\terrCh := make(chan error, 1)\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tdefer cancel()\n\n\t\tgo func() {\n\t\t\terrCh <- lsRemote(ctx, req, c, psCh)\n\t\t}()\n\t\tfor ps := range psCh {\n\t\t\trmIDs = append(rmIDs, ps.GetRequestId())\n\t\t}\n\t\tif err = <-errCh; err != nil {\n\t\t\treturn fmt.Errorf(\"error while listing remote pins: %v\", err)\n\t\t}\n\n\t\tif len(rmIDs) > 1 && !req.Options[pinForceOptionName].(bool) {\n\t\t\treturn fmt.Errorf(\"multiple remote pins are matching this query, add --force to confirm the bulk removal\")\n\t\t}\n\n\t\tfor _, rmID := range rmIDs {\n\t\t\tif err = c.DeleteByID(ctx, rmID); err != nil {\n\t\t\t\treturn fmt.Errorf(\"removing pin identified by requestid=%q failed: %v\", rmID, err)\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t},\n}\n\n// remote service commands\n\nvar addRemotePinServiceCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add remote pinning service.\",\n\t\tShortDescription: \"Add credentials for access to a remote pinning service.\",\n\t\tLongDescription: `\nAdd credentials for access to a remote pinning service and store them in the\nconfig under Pinning.RemoteServices map.\n\nTIP:\n\n To add services and test them by fetching pin count stats:\n\n $ ipfs pin remote service add goodsrv https://pin-api.example.com secret-key\n $ ipfs pin remote service add badsrv https://bad-api.example.com invalid-key\n $ ipfs pin remote service ls --stat\n goodsrv https://pin-api.example.com 0/0/0/0\n badsrv https://bad-api.example.com invalid\n\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(pinServiceNameArgName, true, false, \"Service name.\"),\n\t\tcmds.StringArg(pinServiceEndpointArgName, true, false, \"Service endpoint.\"),\n\t\tcmds.StringArg(pinServiceKeyArgName, true, false, \"Service key.\"),\n\t},\n\tType: nil,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\trepo, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer repo.Close()\n\n\t\tif len(req.Arguments) < 3 {\n\t\t\treturn fmt.Errorf(\"expecting three arguments: service name, endpoint and key\")\n\t\t}\n\n\t\tname := req.Arguments[0]\n\t\tendpoint, err := normalizeEndpoint(req.Arguments[1])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tkey := req.Arguments[2]\n\n\t\tcfg, err := repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif cfg.Pinning.RemoteServices != nil {\n\t\t\tif _, present := cfg.Pinning.RemoteServices[name]; present {\n\t\t\t\treturn fmt.Errorf(\"service already present\")\n\t\t\t}\n\t\t} else {\n\t\t\tcfg.Pinning.RemoteServices = map[string]config.RemotePinningService{}\n\t\t}\n\n\t\tcfg.Pinning.RemoteServices[name] = config.RemotePinningService{\n\t\t\tAPI: config.RemotePinningServiceAPI{\n\t\t\t\tEndpoint: endpoint,\n\t\t\t\tKey: key,\n\t\t\t},\n\t\t\tPolicies: config.RemotePinningServicePolicies{},\n\t\t}\n\n\t\treturn repo.SetConfig(cfg)\n\t},\n}\n\nvar rmRemotePinServiceCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove remote pinning service.\",\n\t\tShortDescription: \"Remove credentials for access to a remote pinning service.\",\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(pinServiceNameOptionName, true, false, \"Name of remote pinning service to remove.\"),\n\t},\n\tOptions: []cmds.Option{},\n\tType: nil,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\trepo, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer repo.Close()\n\n\t\tif len(req.Arguments) != 1 {\n\t\t\treturn fmt.Errorf(\"expecting one argument: name\")\n\t\t}\n\t\tname := req.Arguments[0]\n\n\t\tcfg, err := repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif cfg.Pinning.RemoteServices != nil {\n\t\t\tdelete(cfg.Pinning.RemoteServices, name)\n\t\t}\n\t\treturn repo.SetConfig(cfg)\n\t},\n}\n\nvar lsRemotePinServiceCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List remote pinning services.\",\n\t\tShortDescription: \"List remote pinning services.\",\n\t\tLongDescription: `\nList remote pinning services.\n\nBy default, only a name and an endpoint are listed; however, one can pass\n'--stat' to test each endpoint by fetching pin counts for each state:\n\n $ ipfs pin remote service ls --stat\n goodsrv https://pin-api.example.com 0/0/0/0\n badsrv https://bad-api.example.com invalid\n\nTIP: pass '--enc=json' for more useful JSON output.\n`,\n\t},\n\tArguments: []cmds.Argument{},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(pinServiceStatOptionName, \"Try to fetch and display current pin count on remote service (queued/pinning/pinned/failed).\").WithDefault(false),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tdefer cancel()\n\n\t\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\trepo, err := fsrepo.Open(cfgRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer repo.Close()\n\n\t\tcfg, err := repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif cfg.Pinning.RemoteServices == nil {\n\t\t\treturn cmds.EmitOnce(res, &PinServicesList{make([]ServiceDetails, 0)})\n\t\t}\n\t\tservices := cfg.Pinning.RemoteServices\n\t\tresult := PinServicesList{make([]ServiceDetails, 0, len(services))}\n\t\tfor svcName, svcConfig := range services {\n\t\t\tsvcDetails := ServiceDetails{svcName, svcConfig.API.Endpoint, nil}\n\n\t\t\t// if --pin-count is passed, we try to fetch pin numbers from remote service\n\t\t\tif req.Options[pinServiceStatOptionName].(bool) {\n\t\t\t\tlsRemotePinCount := func(ctx context.Context, env cmds.Environment, svcName string) (*PinCount, error) {\n\t\t\t\t\tc, err := getRemotePinService(env, svcName)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn nil, err\n\t\t\t\t\t}\n\t\t\t\t\t// we only care about total count, so requesting smallest batch\n\t\t\t\t\tbatch := pinclient.PinOpts.Limit(1)\n\t\t\t\t\tfs := pinclient.PinOpts.FilterStatus\n\n\t\t\t\t\tstatuses := []pinclient.Status{\n\t\t\t\t\t\tpinclient.StatusQueued,\n\t\t\t\t\t\tpinclient.StatusPinning,\n\t\t\t\t\t\tpinclient.StatusPinned,\n\t\t\t\t\t\tpinclient.StatusFailed,\n\t\t\t\t\t}\n\n\t\t\t\t\tg, ctx := errgroup.WithContext(ctx)\n\t\t\t\t\tpc := &PinCount{}\n\n\t\t\t\t\tfor _, s := range statuses {\n\t\t\t\t\t\tstatus := s // lol https://golang.org/doc/faq#closures_and_goroutines\n\t\t\t\t\t\tg.Go(func() error {\n\t\t\t\t\t\t\t_, n, err := c.LsBatchSync(ctx, batch, fs(status))\n\t\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\t\treturn err\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tswitch status {\n\t\t\t\t\t\t\tcase pinclient.StatusQueued:\n\t\t\t\t\t\t\t\tpc.Queued = n\n\t\t\t\t\t\t\tcase pinclient.StatusPinning:\n\t\t\t\t\t\t\t\tpc.Pinning = n\n\t\t\t\t\t\t\tcase pinclient.StatusPinned:\n\t\t\t\t\t\t\t\tpc.Pinned = n\n\t\t\t\t\t\t\tcase pinclient.StatusFailed:\n\t\t\t\t\t\t\t\tpc.Failed = n\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\treturn nil\n\t\t\t\t\t\t})\n\t\t\t\t\t}\n\t\t\t\t\tif err := g.Wait(); err != nil {\n\t\t\t\t\t\treturn nil, err\n\t\t\t\t\t}\n\n\t\t\t\t\treturn pc, nil\n\t\t\t\t}\n\n\t\t\t\tpinCount, err := lsRemotePinCount(ctx, env, svcName)\n\n\t\t\t\t// PinCount is present only if we were able to fetch counts.\n\t\t\t\t// We don't want to break listing of services so this is best-effort.\n\t\t\t\t// (verbose err is returned by 'pin remote ls', if needed)\n\t\t\t\tsvcDetails.Stat = &Stat{}\n\t\t\t\tif err == nil {\n\t\t\t\t\tsvcDetails.Stat.Status = \"valid\"\n\t\t\t\t\tsvcDetails.Stat.PinCount = pinCount\n\t\t\t\t} else {\n\t\t\t\t\tsvcDetails.Stat.Status = \"invalid\"\n\t\t\t\t}\n\t\t\t}\n\t\t\tresult.RemoteServices = append(result.RemoteServices, svcDetails)\n\t\t}\n\t\tsort.Sort(result)\n\t\treturn cmds.EmitOnce(res, &result)\n\t},\n\tType: PinServicesList{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, list *PinServicesList) error {\n\t\t\ttw := tabwriter.NewWriter(w, 1, 2, 1, ' ', 0)\n\t\t\twithStat := req.Options[pinServiceStatOptionName].(bool)\n\t\t\tfor _, s := range list.RemoteServices {\n\t\t\t\tif withStat {\n\t\t\t\t\tstat := s.Stat.Status\n\t\t\t\t\tpc := s.Stat.PinCount\n\t\t\t\t\tif s.Stat.PinCount != nil {\n\t\t\t\t\t\tstat = fmt.Sprintf(\"%d/%d/%d/%d\", pc.Queued, pc.Pinning, pc.Pinned, pc.Failed)\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\t%s\\n\", s.Service, s.ApiEndpoint, stat)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\n\", s.Service, s.ApiEndpoint)\n\t\t\t\t}\n\t\t\t}\n\t\t\ttw.Flush()\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\ntype ServiceDetails struct {\n\tService string\n\tApiEndpoint string //nolint\n\tStat *Stat `json:\",omitempty\"` // present only when --stat not passed\n}\n\ntype Stat struct {\n\tStatus string\n\tPinCount *PinCount `json:\",omitempty\"` // missing when --stat is passed but the service is offline\n}\n\ntype PinCount struct {\n\tQueued int\n\tPinning int\n\tPinned int\n\tFailed int\n}\n\n// Struct returned by ipfs pin remote service ls --enc=json | jq\ntype PinServicesList struct {\n\tRemoteServices []ServiceDetails\n}\n\nfunc (l PinServicesList) Len() int {\n\treturn len(l.RemoteServices)\n}\n\nfunc (l PinServicesList) Swap(i, j int) {\n\ts := l.RemoteServices\n\ts[i], s[j] = s[j], s[i]\n}\n\nfunc (l PinServicesList) Less(i, j int) bool {\n\ts := l.RemoteServices\n\treturn s[i].Service < s[j].Service\n}\n\nfunc getRemotePinServiceFromRequest(req *cmds.Request, env cmds.Environment) (*pinclient.Client, error) {\n\tservice, serviceFound := req.Options[pinServiceNameOptionName]\n\tif !serviceFound {\n\t\treturn nil, fmt.Errorf(\"a service name must be passed\")\n\t}\n\n\tserviceStr := service.(string)\n\tvar err error\n\tc, err := getRemotePinService(env, serviceStr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn c, nil\n}\n\nfunc getRemotePinService(env cmds.Environment, name string) (*pinclient.Client, error) {\n\tif name == \"\" {\n\t\treturn nil, fmt.Errorf(\"remote pinning service name not specified\")\n\t}\n\tendpoint, key, err := getRemotePinServiceInfo(env, name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn pinclient.NewClient(endpoint, key), nil\n}\n\nfunc getRemotePinServiceInfo(env cmds.Environment, name string) (endpoint, key string, err error) {\n\tcfgRoot, err := cmdenv.GetConfigRoot(env)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\trepo, err := fsrepo.Open(cfgRoot)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\tdefer repo.Close()\n\tcfg, err := repo.Config()\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\tif cfg.Pinning.RemoteServices == nil {\n\t\treturn \"\", \"\", fmt.Errorf(\"service not known\")\n\t}\n\tservice, present := cfg.Pinning.RemoteServices[name]\n\tif !present {\n\t\treturn \"\", \"\", fmt.Errorf(\"service not known\")\n\t}\n\tendpoint, err = normalizeEndpoint(service.API.Endpoint)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\treturn endpoint, service.API.Key, nil\n}\n\nfunc normalizeEndpoint(endpoint string) (string, error) {\n\turi, err := neturl.ParseRequestURI(endpoint)\n\tif err != nil || !(uri.Scheme == \"http\" || uri.Scheme == \"https\") {\n\t\treturn \"\", fmt.Errorf(\"service endpoint must be a valid HTTP URL\")\n\t}\n\n\t// cleanup trailing and duplicate slashes (https://github.com/ipfs/kubo/issues/7826)\n\turi.Path = gopath.Clean(uri.Path)\n\turi.Path = strings.TrimSuffix(uri.Path, \".\")\n\turi.Path = strings.TrimSuffix(uri.Path, \"/\")\n\n\t// remove any query params\n\tif uri.RawQuery != \"\" {\n\t\treturn \"\", fmt.Errorf(\"service endpoint should be provided without any query parameters\")\n\t}\n\n\tif strings.HasSuffix(uri.Path, \"/pins\") {\n\t\treturn \"\", fmt.Errorf(\"service endpoint should be provided without the /pins suffix\")\n\t}\n\n\treturn uri.String(), nil\n}\n" }, { "path": "core/commands/pin/remotepin_test.go", "content": "package pin\n\nimport (\n\t\"testing\"\n)\n\nfunc TestNormalizeEndpoint(t *testing.T) {\n\tcases := []struct {\n\t\tin string\n\t\terr string\n\t\tout string\n\t}{\n\t\t{\n\t\t\tin: \"https://1.example.com\",\n\t\t\terr: \"\",\n\t\t\tout: \"https://1.example.com\",\n\t\t},\n\t\t{\n\t\t\tin: \"https://2.example.com/\",\n\t\t\terr: \"\",\n\t\t\tout: \"https://2.example.com\",\n\t\t},\n\t\t{\n\t\t\tin: \"https://3.example.com/pins/\",\n\t\t\terr: \"service endpoint should be provided without the /pins suffix\",\n\t\t\tout: \"\",\n\t\t},\n\t\t{\n\t\t\tin: \"https://4.example.com/pins\",\n\t\t\terr: \"service endpoint should be provided without the /pins suffix\",\n\t\t\tout: \"\",\n\t\t},\n\t\t{\n\t\t\tin: \"https://5.example.com/./some//nonsense/../path/../path/\",\n\t\t\terr: \"\",\n\t\t\tout: \"https://5.example.com/some/path\",\n\t\t},\n\t\t{\n\t\t\tin: \"https://6.example.com/endpoint/?query=val\",\n\t\t\terr: \"service endpoint should be provided without any query parameters\",\n\t\t\tout: \"\",\n\t\t},\n\t\t{\n\t\t\tin: \"http://192.168.0.5:45000/\",\n\t\t\terr: \"\",\n\t\t\tout: \"http://192.168.0.5:45000\",\n\t\t},\n\t\t{\n\t\t\tin: \"foo://4.example.com/pins\",\n\t\t\terr: \"service endpoint must be a valid HTTP URL\",\n\t\t\tout: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range cases {\n\t\tout, err := normalizeEndpoint(tc.in)\n\t\tif err != nil && tc.err != err.Error() {\n\t\t\tt.Errorf(\"unexpected error for %q: expected %q; got %q\", tc.in, tc.err, err)\n\t\t\tcontinue\n\t\t}\n\t\tif out != tc.out {\n\t\t\tt.Errorf(\"unexpected endpoint for %q: expected %q; got %q\", tc.in, tc.out, out)\n\t\t\tcontinue\n\t\t}\n\t}\n}\n" }, { "path": "core/commands/ping.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\tping \"github.com/libp2p/go-libp2p/p2p/protocol/ping\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\nconst kPingTimeout = 10 * time.Second\n\ntype PingResult struct {\n\tSuccess bool\n\tTime time.Duration\n\tText string\n}\n\nconst (\n\tpingCountOptionName = \"count\"\n)\n\n// ErrPingSelf is returned when the user attempts to ping themself.\nvar ErrPingSelf = errors.New(\"error: can't ping self\")\n\nvar PingCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Send echo request packets to IPFS hosts.\",\n\t\tShortDescription: `\n'ipfs ping' is a tool to test sending data to other nodes. It finds nodes\nvia the routing system, sends pings, waits for pongs, and prints out round-\ntrip latency information.\n\t\t`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peer ID\", true, true, \"ID of peer to be pinged.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.IntOption(pingCountOptionName, \"n\", \"Number of ping messages to send.\").WithDefault(10),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Must be online!\n\t\tif !n.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\taddr, pid, err := ParsePeerParam(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to parse peer address '%s': %s\", req.Arguments[0], err)\n\t\t}\n\n\t\tif pid == n.Identity {\n\t\t\treturn ErrPingSelf\n\t\t}\n\n\t\tif addr != nil {\n\t\t\tn.Peerstore.AddAddr(pid, addr, pstore.TempAddrTTL) // temporary\n\t\t}\n\n\t\tnumPings, _ := req.Options[pingCountOptionName].(int)\n\t\tif numPings <= 0 {\n\t\t\treturn fmt.Errorf(\"ping count must be greater than 0, was %d\", numPings)\n\t\t}\n\n\t\tif len(n.Peerstore.Addrs(pid)) == 0 {\n\t\t\t// Make sure we can find the node in question\n\t\t\tif err := res.Emit(&PingResult{\n\t\t\t\tText: fmt.Sprintf(\"Looking up peer %s\", pid),\n\t\t\t\tSuccess: true,\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tctx, cancel := context.WithTimeout(req.Context, kPingTimeout)\n\t\t\tp, err := n.Routing.FindPeer(ctx, pid)\n\t\t\tcancel()\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"peer lookup failed: %s\", err)\n\t\t\t}\n\t\t\tn.Peerstore.AddAddrs(p.ID, p.Addrs, pstore.TempAddrTTL)\n\t\t}\n\n\t\tif err := res.Emit(&PingResult{\n\t\t\tText: fmt.Sprintf(\"PING %s.\", pid),\n\t\t\tSuccess: true,\n\t\t}); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tctx, cancel := context.WithTimeout(req.Context, kPingTimeout*time.Duration(numPings))\n\t\tdefer cancel()\n\t\tpings := ping.Ping(ctx, n.PeerHost, pid)\n\n\t\tvar (\n\t\t\tcount int\n\t\t\ttotal time.Duration\n\t\t)\n\t\tticker := time.NewTicker(time.Second)\n\t\tdefer ticker.Stop()\n\n\t\tfor range numPings {\n\t\t\tr, ok := <-pings\n\t\t\tif !ok {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\tif r.Error != nil {\n\t\t\t\terr = res.Emit(&PingResult{\n\t\t\t\t\tSuccess: false,\n\t\t\t\t\tText: fmt.Sprintf(\"Ping error: %s\", r.Error),\n\t\t\t\t})\n\t\t\t} else {\n\t\t\t\tcount++\n\t\t\t\ttotal += r.RTT\n\t\t\t\terr = res.Emit(&PingResult{\n\t\t\t\t\tSuccess: true,\n\t\t\t\t\tTime: r.RTT,\n\t\t\t\t})\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn ctx.Err()\n\t\t\t}\n\t\t}\n\t\tif count == 0 {\n\t\t\treturn fmt.Errorf(\"ping failed\")\n\t\t}\n\t\taveragems := total.Seconds() * 1000 / float64(count)\n\t\treturn res.Emit(&PingResult{\n\t\t\tSuccess: true,\n\t\t\tText: fmt.Sprintf(\"Average latency: %.2fms\", averagems),\n\t\t})\n\t},\n\tType: PingResult{},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tvar (\n\t\t\t\ttotal time.Duration\n\t\t\t\tcount int\n\t\t\t)\n\n\t\t\tfor {\n\t\t\t\tevent, err := res.Next()\n\t\t\t\tswitch err {\n\t\t\t\tcase nil:\n\t\t\t\tcase io.EOF:\n\t\t\t\t\treturn nil\n\t\t\t\tcase context.Canceled, context.DeadlineExceeded:\n\t\t\t\t\tif count == 0 {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\taveragems := total.Seconds() * 1000 / float64(count)\n\t\t\t\t\treturn re.Emit(&PingResult{\n\t\t\t\t\t\tSuccess: true,\n\t\t\t\t\t\tText: fmt.Sprintf(\"Average latency: %.2fms\", averagems),\n\t\t\t\t\t})\n\t\t\t\tdefault:\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tpr := event.(*PingResult)\n\t\t\t\tif pr.Success && pr.Text == \"\" {\n\t\t\t\t\ttotal += pr.Time\n\t\t\t\t\tcount++\n\t\t\t\t}\n\t\t\t\terr = re.Emit(event)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *PingResult) error {\n\t\t\tif len(out.Text) > 0 {\n\t\t\t\tfmt.Fprintln(w, out.Text)\n\t\t\t} else if out.Success {\n\t\t\t\tfmt.Fprintf(w, \"Pong received: time=%.2f ms\\n\", out.Time.Seconds()*1000)\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(w, \"Pong failed\\n\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nfunc ParsePeerParam(text string) (ma.Multiaddr, peer.ID, error) {\n\t// Multiaddr\n\tif strings.HasPrefix(text, \"/\") {\n\t\tmaddr, err := ma.NewMultiaddr(text)\n\t\tif err != nil {\n\t\t\treturn nil, \"\", err\n\t\t}\n\t\ttransport, id := peer.SplitAddr(maddr)\n\t\tif id == \"\" {\n\t\t\treturn nil, \"\", peer.ErrInvalidAddr\n\t\t}\n\t\treturn transport, id, nil\n\t}\n\t// Raw peer ID\n\tp, err := peer.Decode(text)\n\treturn nil, p, err\n}\n" }, { "path": "core/commands/profile.go", "content": "package commands\n\nimport (\n\t\"archive/zip\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/e\"\n\t\"github.com/ipfs/kubo/profile\"\n)\n\n// time format that works in filenames on windows.\nvar timeFormat = strings.ReplaceAll(time.RFC3339, \":\", \"_\")\n\ntype profileResult struct {\n\tFile string\n}\n\nconst (\n\tcollectorsOptionName = \"collectors\"\n\tprofileTimeOption = \"profile-time\"\n\tmutexProfileFractionOption = \"mutex-profile-fraction\"\n\tblockProfileRateOption = \"block-profile-rate\"\n)\n\nvar sysProfileCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Collect a performance profile for debugging.\",\n\t\tShortDescription: `\nCollects profiles from a running go-ipfs daemon into a single zip file.\nTo aid in debugging, this command also attempts to include a copy of\nthe running go-ipfs binary.\n`,\n\t\tLongDescription: `\nCollects profiles from a running go-ipfs daemon into a single zipfile.\nTo aid in debugging, this command also attempts to include a copy of\nthe running go-ipfs binary.\n\nProfiles can be examined using 'go tool pprof', some tips can be found at\nhttps://github.com/ipfs/kubo/blob/master/docs/debug-guide.md.\n\nPrivacy Notice:\n\nThe output file includes:\n\n- A list of running goroutines.\n- A CPU profile.\n- A heap inuse profile.\n- A heap allocation profile.\n- A mutex profile.\n- A block profile.\n- Your copy of go-ipfs.\n- The output of 'ipfs version --all'.\n\nIt does not include:\n\n- Any of your IPFS data or metadata.\n- Your config or private key.\n- Your IP address.\n- The contents of your computer's memory, filesystem, etc.\n\nHowever, it could reveal:\n\n- Your build path, if you built go-ipfs yourself.\n- If and how a command/feature is being used (inferred from running functions).\n- Memory offsets of various data structures.\n- Any modifications you've made to go-ipfs.\n`,\n\t\tHTTP: &cmds.HTTPHelpText{\n\t\t\tResponseContentType: \"application/zip\",\n\t\t},\n\t},\n\tNoLocal: true,\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(outputOptionName, \"o\", \"The path where the output .zip should be stored. Default: ./ipfs-profile-[timestamp].zip\"),\n\t\tcmds.DelimitedStringsOption(\",\", collectorsOptionName, \"The list of collectors to use for collecting diagnostic data.\").\n\t\t\tWithDefault([]string{\n\t\t\t\tprofile.CollectorGoroutinesStack,\n\t\t\t\tprofile.CollectorGoroutinesPprof,\n\t\t\t\tprofile.CollectorVersion,\n\t\t\t\tprofile.CollectorHeap,\n\t\t\t\tprofile.CollectorAllocs,\n\t\t\t\tprofile.CollectorBin,\n\t\t\t\tprofile.CollectorCPU,\n\t\t\t\tprofile.CollectorMutex,\n\t\t\t\tprofile.CollectorBlock,\n\t\t\t\tprofile.CollectorTrace,\n\t\t\t}),\n\t\tcmds.StringOption(profileTimeOption, \"The amount of time spent profiling. If this is set to 0, then sampling profiles are skipped.\").WithDefault(\"30s\"),\n\t\tcmds.IntOption(mutexProfileFractionOption, \"The fraction 1/n of mutex contention events that are reported in the mutex profile.\").WithDefault(4),\n\t\tcmds.StringOption(blockProfileRateOption, \"The duration to wait between sampling goroutine-blocking events for the blocking profile.\").WithDefault(\"1ms\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcollectors := req.Options[collectorsOptionName].([]string)\n\n\t\tprofileTimeStr, _ := req.Options[profileTimeOption].(string)\n\t\tprofileTime, err := time.ParseDuration(profileTimeStr)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to parse profile duration %q: %w\", profileTimeStr, err)\n\t\t}\n\n\t\tblockProfileRateStr, _ := req.Options[blockProfileRateOption].(string)\n\t\tblockProfileRate, err := time.ParseDuration(blockProfileRateStr)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to parse block profile rate %q: %w\", blockProfileRateStr, err)\n\t\t}\n\n\t\tmutexProfileFraction, _ := req.Options[mutexProfileFractionOption].(int)\n\n\t\tr, w := io.Pipe()\n\n\t\tgo func() {\n\t\t\tarchive := zip.NewWriter(w)\n\t\t\terr = profile.WriteProfiles(req.Context, archive, profile.Options{\n\t\t\t\tCollectors: collectors,\n\t\t\t\tProfileDuration: profileTime,\n\t\t\t\tMutexProfileFraction: mutexProfileFraction,\n\t\t\t\tBlockProfileRate: blockProfileRate,\n\t\t\t})\n\t\t\tarchive.Close()\n\t\t\t_ = w.CloseWithError(err)\n\t\t}()\n\t\tres.SetEncodingType(cmds.OctetStream)\n\t\tres.SetContentType(\"application/zip\")\n\t\treturn res.Emit(r)\n\t},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tv, err := res.Next()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\toutReader, ok := v.(io.Reader)\n\t\t\tif !ok {\n\t\t\t\treturn e.New(e.TypeErr(outReader, v))\n\t\t\t}\n\n\t\t\toutPath, _ := res.Request().Options[outputOptionName].(string)\n\t\t\tif outPath == \"\" {\n\t\t\t\toutPath = \"ipfs-profile-\" + time.Now().Format(timeFormat) + \".zip\"\n\t\t\t}\n\t\t\tfi, err := os.Create(outPath)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer fi.Close()\n\n\t\t\t_, err = io.Copy(fi, outReader)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn re.Emit(&profileResult{File: outPath})\n\t\t},\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *profileResult) error {\n\t\t\tfmt.Fprintf(w, \"Wrote profiles to: %s\\n\", out.File)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/provide.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\t\"time\"\n\t\"unicode/utf8\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n\tboxoprovider \"github.com/ipfs/boxo/provider\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/provider\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/provider/buffered\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/provider/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/provider/stats\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n\t\"github.com/probe-lab/go-libdht/kad/key\"\n\t\"golang.org/x/exp/constraints\"\n)\n\nconst (\n\tprovideQuietOptionName = \"quiet\"\n\tprovideLanOptionName = \"lan\"\n\n\tprovideStatAllOptionName = \"all\"\n\tprovideStatCompactOptionName = \"compact\"\n\tprovideStatNetworkOptionName = \"network\"\n\tprovideStatConnectivityOptionName = \"connectivity\"\n\tprovideStatOperationsOptionName = \"operations\"\n\tprovideStatTimingsOptionName = \"timings\"\n\tprovideStatScheduleOptionName = \"schedule\"\n\tprovideStatQueuesOptionName = \"queues\"\n\tprovideStatWorkersOptionName = \"workers\"\n\n\t// lowWorkerThreshold is the threshold below which worker availability warnings are shown\n\tlowWorkerThreshold = 2\n)\n\nvar ProvideCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Control and monitor content providing\",\n\t\tShortDescription: `\nControl providing operations.\n\nOVERVIEW:\n\nThe provider system advertises content by publishing provider records,\nallowing other nodes to discover which peers have specific content.\nContent is reprovided periodically (every Provide.DHT.Interval)\naccording to Provide.Strategy.\n\nCONFIGURATION:\n\nLearn more: https://github.com/ipfs/kubo/blob/master/docs/config.md#provide\n\nSEE ALSO:\n\nFor ad-hoc one-time provide, see 'ipfs routing provide'\n`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"clear\": provideClearCmd,\n\t\t\"stat\": provideStatCmd,\n\t},\n}\n\nvar provideClearCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Clear all CIDs from the provide queue.\",\n\t\tShortDescription: `\nClear all CIDs pending to be provided for the first time.\n\nBEHAVIOR:\n\nThis command removes CIDs from the provide queue that are waiting to be\nadvertised to the DHT for the first time. It does not affect content that\nis already being reprovided on schedule.\n\nAUTOMATIC CLEARING:\n\nKubo will automatically clear the queue when it detects a change of\nProvide.Strategy upon a restart.\n\nLearn: https://github.com/ipfs/kubo/blob/master/docs/config.md#providestrategy\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(provideQuietOptionName, \"q\", \"Do not write output.\"),\n\t},\n\tRun: func(req *cmds.Request, re cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tquiet, _ := req.Options[provideQuietOptionName].(bool)\n\t\tif n.Provider == nil {\n\t\t\treturn nil\n\t\t}\n\n\t\tcleared := n.Provider.Clear()\n\t\tif quiet {\n\t\t\treturn nil\n\t\t}\n\t\t_ = re.Emit(cleared)\n\n\t\treturn nil\n\t},\n\tType: int(0),\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, cleared int) error {\n\t\t\tquiet, _ := req.Options[provideQuietOptionName].(bool)\n\t\t\tif quiet {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\t_, err := fmt.Fprintf(w, \"removed %d items from provide queue\\n\", cleared)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\ntype provideStats struct {\n\tSweep *stats.Stats\n\tLegacy *boxoprovider.ReproviderStats\n\tFullRT bool // only used for legacy stats\n}\n\n// extractSweepingProvider extracts a SweepingProvider from the given provider interface.\n// It handles unwrapping buffered and dual providers, selecting LAN or WAN as specified.\n// Returns nil if the provider is not a sweeping provider type.\nfunc extractSweepingProvider(prov any, useLAN bool) *provider.SweepingProvider {\n\tswitch p := prov.(type) {\n\tcase *provider.SweepingProvider:\n\t\treturn p\n\tcase *dual.SweepingProvider:\n\t\tif useLAN {\n\t\t\treturn p.LAN\n\t\t}\n\t\treturn p.WAN\n\tcase *buffered.SweepingProvider:\n\t\t// Recursively extract from the inner provider\n\t\treturn extractSweepingProvider(p.Provider, useLAN)\n\tdefault:\n\t\treturn nil\n\t}\n}\n\nvar provideStatCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show statistics about the provider system\",\n\t\tShortDescription: `\nReturns statistics about the node's provider system.\n\nOVERVIEW:\n\nThe provide system advertises content to the DHT by publishing provider\nrecords that map CIDs to your peer ID. These records expire after a fixed\nTTL to account for node churn, so content must be reprovided periodically\nto stay discoverable.\n\nTwo provider types exist:\n\n- Sweep provider: Divides the DHT keyspace into regions and systematically\n sweeps through them over the reprovide interval. Batches CIDs allocated\n to the same DHT servers, reducing lookups from N (one per CID) to a\n small static number based on DHT size (~3k for 10k DHT servers). Spreads\n work evenly over time to prevent resource spikes and ensure announcements\n happen just before records expire.\n\n- Legacy provider: Processes each CID individually with separate DHT\n lookups. Attempts to reprovide all content as quickly as possible at the\n start of each cycle. Works well for small datasets but struggles with\n large collections.\n\nLearn more:\n- Config: https://github.com/ipfs/kubo/blob/master/docs/config.md#provide\n- Metrics: https://github.com/ipfs/kubo/blob/master/docs/provide-stats.md\n\nDEFAULT OUTPUT:\n\nShows a brief summary including queue sizes, scheduled items, average record\nholders, ongoing/total provides, and worker warnings.\n\nDETAILED OUTPUT:\n\nUse --all for detailed statistics with these sections: connectivity, queues,\nschedule, timings, network, operations, and workers. Individual sections can\nbe displayed with their flags (e.g., --network, --operations). Multiple flags\ncan be combined.\n\nUse --compact for monitoring-friendly 2-column output (requires --all).\n\nEXAMPLES:\n\nMonitor provider statistics in real-time with 2-column layout:\n\n watch ipfs provide stat --all --compact\n\nGet statistics in JSON format for programmatic processing:\n\n ipfs provide stat --enc=json | jq\n\nNOTES:\n\n- This interface is experimental and may change between releases\n- Legacy provider shows basic stats only (no flags supported)\n- \"Regions\" are keyspace divisions for spreading reprovide work\n- For Dual DHT: use --lan for LAN provider stats (default is WAN)\n`,\n\t},\n\tArguments: []cmds.Argument{},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(provideLanOptionName, \"Show stats for LAN DHT only (for Sweep+Dual DHT only)\"),\n\t\tcmds.BoolOption(provideStatAllOptionName, \"a\", \"Display all provide sweep stats\"),\n\t\tcmds.BoolOption(provideStatCompactOptionName, \"Display stats in 2-column layout (requires --all)\"),\n\t\tcmds.BoolOption(provideStatConnectivityOptionName, \"Display DHT connectivity status\"),\n\t\tcmds.BoolOption(provideStatNetworkOptionName, \"Display network stats (peers, reachability, region size)\"),\n\t\tcmds.BoolOption(provideStatScheduleOptionName, \"Display reprovide schedule (CIDs/regions scheduled, next reprovide time)\"),\n\t\tcmds.BoolOption(provideStatTimingsOptionName, \"Display timing information (uptime, cycle start, reprovide interval)\"),\n\t\tcmds.BoolOption(provideStatWorkersOptionName, \"Display worker pool stats (active/available/queued workers)\"),\n\t\tcmds.BoolOption(provideStatOperationsOptionName, \"Display operation stats (ongoing/past provides, rates, errors)\"),\n\t\tcmds.BoolOption(provideStatQueuesOptionName, \"Display provide and reprovide queue sizes\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tlanStats, _ := req.Options[provideLanOptionName].(bool)\n\n\t\t// Handle legacy provider\n\t\tif legacySys, ok := nd.Provider.(boxoprovider.System); ok {\n\t\t\tif lanStats {\n\t\t\t\treturn errors.New(\"LAN stats only available for Sweep provider with Dual DHT\")\n\t\t\t}\n\t\t\tstats, err := legacySys.Stat()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, fullRT := nd.DHTClient.(*fullrt.FullRT)\n\t\t\treturn res.Emit(provideStats{Legacy: &stats, FullRT: fullRT})\n\t\t}\n\n\t\t// Extract sweeping provider (handles buffered and dual unwrapping)\n\t\tsweepingProvider := extractSweepingProvider(nd.Provider, lanStats)\n\t\tif sweepingProvider == nil {\n\t\t\tif lanStats {\n\t\t\t\treturn errors.New(\"LAN stats only available for Sweep provider with Dual DHT\")\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"stats not available with current routing system %T\", nd.Provider)\n\t\t}\n\n\t\ts := sweepingProvider.Stats()\n\t\treturn res.Emit(provideStats{Sweep: &s})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, s provideStats) error {\n\t\t\twtr := tabwriter.NewWriter(w, 1, 2, 1, ' ', 0)\n\t\t\tdefer wtr.Flush()\n\n\t\t\tall, _ := req.Options[provideStatAllOptionName].(bool)\n\t\t\tcompact, _ := req.Options[provideStatCompactOptionName].(bool)\n\t\t\tconnectivity, _ := req.Options[provideStatConnectivityOptionName].(bool)\n\t\t\tqueues, _ := req.Options[provideStatQueuesOptionName].(bool)\n\t\t\tschedule, _ := req.Options[provideStatScheduleOptionName].(bool)\n\t\t\tnetwork, _ := req.Options[provideStatNetworkOptionName].(bool)\n\t\t\ttimings, _ := req.Options[provideStatTimingsOptionName].(bool)\n\t\t\toperations, _ := req.Options[provideStatOperationsOptionName].(bool)\n\t\t\tworkers, _ := req.Options[provideStatWorkersOptionName].(bool)\n\n\t\t\tflagCount := 0\n\t\t\tfor _, enabled := range []bool{all, connectivity, queues, schedule, network, timings, operations, workers} {\n\t\t\t\tif enabled {\n\t\t\t\t\tflagCount++\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif s.Legacy != nil {\n\t\t\t\tif flagCount > 0 {\n\t\t\t\t\treturn errors.New(\"cannot use flags with legacy provide stats\")\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(wtr, \"TotalReprovides:\\t%s\\n\", humanNumber(s.Legacy.TotalReprovides))\n\t\t\t\tfmt.Fprintf(wtr, \"AvgReprovideDuration:\\t%s\\n\", humanDuration(s.Legacy.AvgReprovideDuration))\n\t\t\t\tfmt.Fprintf(wtr, \"LastReprovideDuration:\\t%s\\n\", humanDuration(s.Legacy.LastReprovideDuration))\n\t\t\t\tif !s.Legacy.LastRun.IsZero() {\n\t\t\t\t\tfmt.Fprintf(wtr, \"LastReprovide:\\t%s\\n\", humanTime(s.Legacy.LastRun))\n\t\t\t\t\tif s.FullRT {\n\t\t\t\t\t\tfmt.Fprintf(wtr, \"NextReprovide:\\t%s\\n\", humanTime(s.Legacy.LastRun.Add(s.Legacy.ReprovideInterval)))\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif s.Sweep == nil {\n\t\t\t\treturn errors.New(\"no provide stats available\")\n\t\t\t}\n\n\t\t\t// Sweep provider stats\n\t\t\tif s.Sweep.Closed {\n\t\t\t\tfmt.Fprintf(wtr, \"Provider is closed\\n\")\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif compact && !all {\n\t\t\t\treturn errors.New(\"--compact requires --all flag\")\n\t\t\t}\n\n\t\t\tbrief := flagCount == 0\n\t\t\tshowHeadings := flagCount > 1 || all\n\n\t\t\tcompactMode := all && compact\n\t\t\tvar cols [2][]string\n\t\t\tcol0MaxWidth := 0\n\t\t\t// formatLine handles both normal and compact output modes:\n\t\t\t// - Normal mode: all lines go to cols[0], col parameter is ignored\n\t\t\t// - Compact mode: col 0 for left column, col 1 for right column\n\t\t\tformatLine := func(col int, format string, a ...any) {\n\t\t\t\tif compactMode {\n\t\t\t\t\ts := fmt.Sprintf(format, a...)\n\t\t\t\t\tcols[col] = append(cols[col], s)\n\t\t\t\t\tif col == 0 {\n\t\t\t\t\t\tcol0MaxWidth = max(col0MaxWidth, utf8.RuneCountInString(s))\n\t\t\t\t\t}\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tformat = strings.Replace(format, \": \", \":\\t\", 1)\n\t\t\t\tformat = strings.Replace(format, \", \", \",\\t\", 1)\n\t\t\t\tcols[0] = append(cols[0], fmt.Sprintf(format, a...))\n\t\t\t}\n\t\t\taddBlankLine := func(col int) {\n\t\t\t\tif !brief {\n\t\t\t\t\tformatLine(col, \"\")\n\t\t\t\t}\n\t\t\t}\n\t\t\tsectionTitle := func(col int, title string) {\n\t\t\t\tif !brief && showHeadings {\n\t\t\t\t\tformatLine(col, \"%s:\", title)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tindent := \" \"\n\t\t\tif brief || !showHeadings {\n\t\t\t\tindent = \"\"\n\t\t\t}\n\n\t\t\t// Connectivity\n\t\t\tif all || connectivity || brief && s.Sweep.Connectivity.Status != \"online\" {\n\t\t\t\tsectionTitle(1, \"Connectivity\")\n\t\t\t\tsince := s.Sweep.Connectivity.Since\n\t\t\t\tif since.IsZero() {\n\t\t\t\t\tformatLine(1, \"%sStatus: %s\", indent, s.Sweep.Connectivity.Status)\n\t\t\t\t} else {\n\t\t\t\t\tformatLine(1, \"%sStatus: %s (%s)\", indent, s.Sweep.Connectivity.Status, humanTime(since))\n\t\t\t\t}\n\t\t\t\taddBlankLine(1)\n\t\t\t}\n\n\t\t\t// Queues\n\t\t\tif all || queues || brief {\n\t\t\t\tsectionTitle(1, \"Queues\")\n\t\t\t\tformatLine(1, \"%sProvide queue: %s CIDs, %s regions\", indent, humanSI(s.Sweep.Queues.PendingKeyProvides, 1), humanSI(s.Sweep.Queues.PendingRegionProvides, 1))\n\t\t\t\tformatLine(1, \"%sReprovide queue: %s regions\", indent, humanSI(s.Sweep.Queues.PendingRegionReprovides, 1))\n\t\t\t\taddBlankLine(1)\n\t\t\t}\n\n\t\t\t// Schedule\n\t\t\tif all || schedule || brief {\n\t\t\t\tsectionTitle(0, \"Schedule\")\n\t\t\t\tformatLine(0, \"%sCIDs scheduled: %s\", indent, humanNumber(s.Sweep.Schedule.Keys))\n\t\t\t\tformatLine(0, \"%sRegions scheduled: %s\", indent, humanNumberOrNA(s.Sweep.Schedule.Regions))\n\t\t\t\tif !brief {\n\t\t\t\t\tformatLine(0, \"%sAvg prefix length: %s\", indent, humanFloatOrNA(s.Sweep.Schedule.AvgPrefixLength))\n\t\t\t\t\tnextPrefix := key.BitString(s.Sweep.Schedule.NextReprovidePrefix)\n\t\t\t\t\tif nextPrefix == \"\" {\n\t\t\t\t\t\tnextPrefix = \"N/A\"\n\t\t\t\t\t}\n\t\t\t\t\tformatLine(0, \"%sNext region prefix: %s\", indent, nextPrefix)\n\t\t\t\t\tnextReprovideAt := s.Sweep.Schedule.NextReprovideAt.Format(\"15:04:05\")\n\t\t\t\t\tif s.Sweep.Schedule.NextReprovideAt.IsZero() {\n\t\t\t\t\t\tnextReprovideAt = \"N/A\"\n\t\t\t\t\t}\n\t\t\t\t\tformatLine(0, \"%sNext region reprovide: %s\", indent, nextReprovideAt)\n\t\t\t\t}\n\t\t\t\taddBlankLine(0)\n\t\t\t}\n\n\t\t\t// Timings\n\t\t\tif all || timings {\n\t\t\t\tsectionTitle(1, \"Timings\")\n\t\t\t\tformatLine(1, \"%sUptime: %s (%s)\", indent, humanDuration(s.Sweep.Timing.Uptime), humanTime(time.Now().Add(-s.Sweep.Timing.Uptime)))\n\t\t\t\tformatLine(1, \"%sCurrent time offset: %s\", indent, humanDuration(s.Sweep.Timing.CurrentTimeOffset))\n\t\t\t\tformatLine(1, \"%sCycle started: %s\", indent, humanTime(s.Sweep.Timing.CycleStart))\n\t\t\t\tformatLine(1, \"%sReprovide interval: %s\", indent, humanDuration(s.Sweep.Timing.ReprovidesInterval))\n\t\t\t\taddBlankLine(1)\n\t\t\t}\n\n\t\t\t// Network\n\t\t\tif all || network || brief {\n\t\t\t\tsectionTitle(0, \"Network\")\n\t\t\t\tformatLine(0, \"%sAvg record holders: %s\", indent, humanFloatOrNA(s.Sweep.Network.AvgHolders))\n\t\t\t\tif !brief {\n\t\t\t\t\tformatLine(0, \"%sPeers swept: %s\", indent, humanInt(s.Sweep.Network.Peers))\n\t\t\t\t\tformatLine(0, \"%sFull keyspace coverage: %t\", indent, s.Sweep.Network.CompleteKeyspaceCoverage)\n\t\t\t\t\tif s.Sweep.Network.Peers > 0 {\n\t\t\t\t\t\tformatLine(0, \"%sReachable peers: %s (%s%%)\", indent, humanInt(s.Sweep.Network.Reachable), humanNumber(100*s.Sweep.Network.Reachable/s.Sweep.Network.Peers))\n\t\t\t\t\t} else {\n\t\t\t\t\t\tformatLine(0, \"%sReachable peers: %s\", indent, humanInt(s.Sweep.Network.Reachable))\n\t\t\t\t\t}\n\t\t\t\t\tformatLine(0, \"%sAvg region size: %s\", indent, humanFloatOrNA(s.Sweep.Network.AvgRegionSize))\n\t\t\t\t\tformatLine(0, \"%sReplication factor: %s\", indent, humanNumber(s.Sweep.Network.ReplicationFactor))\n\t\t\t\t\taddBlankLine(0)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Operations\n\t\t\tif all || operations || brief {\n\t\t\t\tsectionTitle(1, \"Operations\")\n\t\t\t\t// Ongoing operations\n\t\t\t\tformatLine(1, \"%sOngoing provides: %s CIDs, %s regions\", indent, humanSI(s.Sweep.Operations.Ongoing.KeyProvides, 1), humanSI(s.Sweep.Operations.Ongoing.RegionProvides, 1))\n\t\t\t\tformatLine(1, \"%sOngoing reprovides: %s CIDs, %s regions\", indent, humanSI(s.Sweep.Operations.Ongoing.KeyReprovides, 1), humanSI(s.Sweep.Operations.Ongoing.RegionReprovides, 1))\n\t\t\t\t// Past operations summary\n\t\t\t\tformatLine(1, \"%sTotal CIDs provided: %s\", indent, humanNumber(s.Sweep.Operations.Past.KeysProvided))\n\t\t\t\tif !brief {\n\t\t\t\t\tformatLine(1, \"%sTotal records provided: %s\", indent, humanNumber(s.Sweep.Operations.Past.RecordsProvided))\n\t\t\t\t\tformatLine(1, \"%sTotal provide errors: %s\", indent, humanNumber(s.Sweep.Operations.Past.KeysFailed))\n\t\t\t\t\tformatLine(1, \"%sCIDs provided/min/worker: %s\", indent, humanFloatOrNA(s.Sweep.Operations.Past.KeysProvidedPerMinute))\n\t\t\t\t\tformatLine(1, \"%sCIDs reprovided/min/worker: %s\", indent, humanFloatOrNA(s.Sweep.Operations.Past.KeysReprovidedPerMinute))\n\t\t\t\t\tformatLine(1, \"%sRegion reprovide duration: %s\", indent, humanDurationOrNA(s.Sweep.Operations.Past.RegionReprovideDuration))\n\t\t\t\t\tformatLine(1, \"%sAvg CIDs/reprovide: %s\", indent, humanFloatOrNA(s.Sweep.Operations.Past.AvgKeysPerReprovide))\n\t\t\t\t\tformatLine(1, \"%sRegions reprovided (last cycle): %s\", indent, humanNumber(s.Sweep.Operations.Past.RegionReprovidedLastCycle))\n\t\t\t\t\taddBlankLine(1)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Workers\n\t\t\tdisplayWorkers := all || workers\n\t\t\tif displayWorkers || brief {\n\t\t\t\tavailableReservedBurst := max(0, s.Sweep.Workers.DedicatedBurst-s.Sweep.Workers.ActiveBurst)\n\t\t\t\tavailableReservedPeriodic := max(0, s.Sweep.Workers.DedicatedPeriodic-s.Sweep.Workers.ActivePeriodic)\n\t\t\t\tavailableFreeWorkers := max(0, s.Sweep.Workers.Max-max(s.Sweep.Workers.DedicatedBurst, s.Sweep.Workers.ActiveBurst)-max(s.Sweep.Workers.DedicatedPeriodic, s.Sweep.Workers.ActivePeriodic))\n\t\t\t\tavailableBurst := availableFreeWorkers + availableReservedBurst\n\t\t\t\tavailablePeriodic := availableFreeWorkers + availableReservedPeriodic\n\n\t\t\t\tif displayWorkers || availableBurst <= lowWorkerThreshold || availablePeriodic <= lowWorkerThreshold {\n\t\t\t\t\t// Either we want to display workers information, or we are low on\n\t\t\t\t\t// available workers and want to warn the user.\n\t\t\t\t\tsectionTitle(0, \"Workers\")\n\t\t\t\t\tspecifyWorkers := \" workers\"\n\t\t\t\t\tif compactMode {\n\t\t\t\t\t\tspecifyWorkers = \"\"\n\t\t\t\t\t}\n\t\t\t\t\tformatLine(0, \"%sActive%s: %s / %s (max)\", indent, specifyWorkers, humanInt(s.Sweep.Workers.Active), humanInt(s.Sweep.Workers.Max))\n\t\t\t\t\tif brief {\n\t\t\t\t\t\t// Brief mode - show condensed worker info\n\t\t\t\t\t\tformatLine(0, \"%sPeriodic%s: %s active, %s available, %s queued\", indent, specifyWorkers,\n\t\t\t\t\t\t\thumanInt(s.Sweep.Workers.ActivePeriodic), humanInt(availablePeriodic), humanInt(s.Sweep.Workers.QueuedPeriodic))\n\t\t\t\t\t\tformatLine(0, \"%sBurst%s: %s active, %s available, %s queued\\n\", indent, specifyWorkers,\n\t\t\t\t\t\t\thumanInt(s.Sweep.Workers.ActiveBurst), humanInt(availableBurst), humanInt(s.Sweep.Workers.QueuedBurst))\n\t\t\t\t\t} else {\n\t\t\t\t\t\tformatLine(0, \"%sFree%s: %s\", indent, specifyWorkers, humanInt(availableFreeWorkers))\n\t\t\t\t\t\tformatLine(0, \"%s %-14s %-9s %s\", indent, \"Workers stats:\", \"Periodic\", \"Burst\")\n\t\t\t\t\t\tformatLine(0, \"%s %-14s %-9s %s\", indent, \"Active:\", humanInt(s.Sweep.Workers.ActivePeriodic), humanInt(s.Sweep.Workers.ActiveBurst))\n\t\t\t\t\t\tformatLine(0, \"%s %-14s %-9s %s\", indent, \"Dedicated:\", humanInt(s.Sweep.Workers.DedicatedPeriodic), humanInt(s.Sweep.Workers.DedicatedBurst))\n\t\t\t\t\t\tformatLine(0, \"%s %-14s %-9s %s\", indent, \"Available:\", humanInt(availablePeriodic), humanInt(availableBurst))\n\t\t\t\t\t\tformatLine(0, \"%s %-14s %-9s %s\", indent, \"Queued:\", humanInt(s.Sweep.Workers.QueuedPeriodic), humanInt(s.Sweep.Workers.QueuedBurst))\n\t\t\t\t\t\tformatLine(0, \"%sMax connections/worker: %s\", indent, humanInt(s.Sweep.Workers.MaxProvideConnsPerWorker))\n\t\t\t\t\t\taddBlankLine(0)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tif compactMode {\n\t\t\t\tcol0Width := col0MaxWidth + 2\n\t\t\t\t// Print both columns side by side\n\t\t\t\tmaxRows := max(len(cols[0]), len(cols[1]))\n\t\t\t\tif maxRows == 0 {\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t\tfor i := range maxRows - 1 { // last line is empty\n\t\t\t\t\tvar left, right string\n\t\t\t\t\tif i < len(cols[0]) {\n\t\t\t\t\t\tleft = cols[0][i]\n\t\t\t\t\t}\n\t\t\t\t\tif i < len(cols[1]) {\n\t\t\t\t\t\tright = cols[1][i]\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(wtr, \"%-*s %s\\n\", col0Width, left, right)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif !brief {\n\t\t\t\t\tcols[0] = cols[0][:len(cols[0])-1] // remove last blank line\n\t\t\t\t}\n\t\t\t\tfor _, line := range cols[0] {\n\t\t\t\t\tfmt.Fprintln(wtr, line)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: provideStats{},\n}\n\nfunc humanDuration(val time.Duration) string {\n\tif val > time.Second {\n\t\treturn val.Truncate(100 * time.Millisecond).String()\n\t}\n\treturn val.Truncate(time.Microsecond).String()\n}\n\nfunc humanDurationOrNA(val time.Duration) string {\n\tif val <= 0 {\n\t\treturn \"N/A\"\n\t}\n\treturn humanDuration(val)\n}\n\nfunc humanTime(val time.Time) string {\n\tif val.IsZero() {\n\t\treturn \"N/A\"\n\t}\n\treturn val.Format(\"2006-01-02 15:04:05\")\n}\n\nfunc humanNumber[T constraints.Float | constraints.Integer](n T) string {\n\tnf := float64(n)\n\tstr := humanSI(nf, 0)\n\tfullStr := humanFull(nf, 0)\n\tif str != fullStr {\n\t\treturn fmt.Sprintf(\"%s\\t(%s)\", str, fullStr)\n\t}\n\treturn str\n}\n\n// humanNumberOrNA is like humanNumber but returns \"N/A\" for non-positive values.\nfunc humanNumberOrNA[T constraints.Float | constraints.Integer](n T) string {\n\tif n <= 0 {\n\t\treturn \"N/A\"\n\t}\n\treturn humanNumber(n)\n}\n\n// humanFloatOrNA formats a float with 1 decimal place, returning \"N/A\" for non-positive values.\n// This is separate from humanNumberOrNA because it provides simple decimal formatting for\n// continuous metrics (averages, rates) rather than SI unit formatting used for discrete counts.\nfunc humanFloatOrNA(val float64) string {\n\tif val <= 0 {\n\t\treturn \"N/A\"\n\t}\n\treturn humanFull(val, 1)\n}\n\nfunc humanSI[T constraints.Float | constraints.Integer](val T, decimals int) string {\n\tv, unit := humanize.ComputeSI(float64(val))\n\treturn fmt.Sprintf(\"%s%s\", humanFull(v, decimals), unit)\n}\n\nfunc humanInt[T constraints.Integer](val T) string {\n\treturn humanFull(float64(val), 0)\n}\n\nfunc humanFull(val float64, decimals int) string {\n\treturn humanize.CommafWithDigits(val, decimals)\n}\n\n// provideCIDSync performs a synchronous/blocking provide operation to announce\n// the given CID to the DHT.\n//\n// - If the accelerated DHT client is used, a DHT lookup isn't needed, we\n// directly allocate provider records to closest peers.\n// - If Provide.DHT.SweepEnabled=true or OptimisticProvide=true, we make an\n// optimistic provide call.\n// - Else we make a standard provide call (much slower).\n//\n// IMPORTANT: The caller MUST verify DHT availability using HasActiveDHTClient()\n// before calling this function. Calling with a nil or invalid router will cause\n// a panic - this is the caller's responsibility to prevent.\nfunc provideCIDSync(ctx context.Context, router routing.Routing, c cid.Cid) error {\n\treturn router.Provide(ctx, c, true)\n}\n" }, { "path": "core/commands/pubsub.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"slices\"\n\n\t\"github.com/ipfs/go-datastore\"\n\t\"github.com/ipfs/go-datastore/query\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nvar PubsubCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"An experimental publish-subscribe system on ipfs.\",\n\t\tShortDescription: `\nipfs pubsub allows you to publish messages to a given topic, and also to\nsubscribe to new messages on a given topic.\n\nEXPERIMENTAL FEATURE\n\n This is an opt-in feature optimized for IPNS over PubSub\n (https://specs.ipfs.tech/ipns/ipns-pubsub-router/).\n\n The default message validator is designed for IPNS record protocol.\n For custom pubsub applications requiring different validation logic,\n use go-libp2p-pubsub (https://github.com/libp2p/go-libp2p-pubsub)\n directly in a dedicated binary.\n\n To enable, set 'Pubsub.Enabled' config to true.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"pub\": PubsubPubCmd,\n\t\t\"sub\": PubsubSubCmd,\n\t\t\"ls\": PubsubLsCmd,\n\t\t\"peers\": PubsubPeersCmd,\n\t\t\"reset\": PubsubResetCmd,\n\t},\n}\n\ntype pubsubMessage struct {\n\tFrom string `json:\"from,omitempty\"`\n\tData string `json:\"data,omitempty\"`\n\tSeqno string `json:\"seqno,omitempty\"`\n\tTopicIDs []string `json:\"topicIDs,omitempty\"`\n}\n\nvar PubsubSubCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Subscribe to messages on a given topic.\",\n\t\tShortDescription: `\nipfs pubsub sub subscribes to messages on a given topic.\n\nEXPERIMENTAL FEATURE\n\n This is an opt-in feature optimized for IPNS over PubSub\n (https://specs.ipfs.tech/ipns/ipns-pubsub-router/).\n\n To enable, set 'Pubsub.Enabled' config to true.\n\nPEER ENCODING\n\n Peer IDs in From fields are encoded using the default text representation\n from go-libp2p. This ensures the same string values as in 'ipfs pubsub peers'.\n\nTOPIC AND DATA ENCODING\n\n Topics, Data and Seqno are binary data. To ensure all bytes are transferred\n correctly the RPC client and server will use multibase encoding behind\n the scenes.\n\n You can inspect the format by passing --enc=json. The ipfs multibase commands\n can be used for encoding/decoding multibase strings in the userland.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"topic\", true, false, \"Name of topic to subscribe to (multibase encoded when sent over HTTP RPC).\"),\n\t},\n\tPreRun: urlArgsEncoder,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif err := urlArgsDecoder(req, env); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttopic := req.Arguments[0]\n\n\t\tsub, err := api.PubSub().Subscribe(req.Context, topic)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer sub.Close()\n\n\t\tif f, ok := res.(http.Flusher); ok {\n\t\t\tf.Flush()\n\t\t}\n\n\t\tfor {\n\t\t\tmsg, err := sub.Next(req.Context)\n\t\t\tif err == io.EOF || err == context.Canceled {\n\t\t\t\treturn nil\n\t\t\t} else if err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// turn bytes into strings\n\t\t\tencoder, _ := mbase.EncoderByName(\"base64url\")\n\t\t\tpsm := pubsubMessage{\n\t\t\t\tData: encoder.Encode(msg.Data()),\n\t\t\t\tFrom: msg.From().String(),\n\t\t\t\tSeqno: encoder.Encode(msg.Seq()),\n\t\t\t}\n\t\t\tfor _, topic := range msg.Topics() {\n\t\t\t\tpsm.TopicIDs = append(psm.TopicIDs, encoder.Encode([]byte(topic)))\n\t\t\t}\n\t\t\tif err := res.Emit(&psm); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, psm *pubsubMessage) error {\n\t\t\t_, dec, err := mbase.Decode(psm.Data)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, err = w.Write(dec)\n\t\t\treturn err\n\t\t}),\n\t\t// DEPRECATED, undocumented format we used in tests, but not anymore\n\t\t// \\n\\n\n\t\t\"ndpayload\": cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, psm *pubsubMessage) error {\n\t\t\treturn errors.New(\"--enc=ndpayload was removed, use --enc=json instead\")\n\t\t}),\n\t\t// DEPRECATED, uncodumented format we used in tests, but not anymore\n\t\t// \n\t\t\"lenpayload\": cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, psm *pubsubMessage) error {\n\t\t\treturn errors.New(\"--enc=lenpayload was removed, use --enc=json instead\")\n\t\t}),\n\t},\n\tType: pubsubMessage{},\n}\n\nvar PubsubPubCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Publish data to a given pubsub topic.\",\n\t\tShortDescription: `\nipfs pubsub pub publishes a message to a specified topic.\nIt reads binary data from stdin or a file.\n\nEXPERIMENTAL FEATURE\n\n This is an opt-in feature optimized for IPNS over PubSub\n (https://specs.ipfs.tech/ipns/ipns-pubsub-router/).\n\n To enable, set 'Pubsub.Enabled' config to true.\n\nHTTP RPC ENCODING\n\n The data to be published is sent in HTTP request body as multipart/form-data.\n\n Topic names are binary data too. To ensure all bytes are transferred\n correctly via URL params, the RPC client and server will use multibase\n encoding behind the scenes.\n\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"topic\", true, false, \"Topic to publish to (multibase encoded when sent over HTTP RPC).\"),\n\t\tcmds.FileArg(\"data\", true, false, \"The data to be published.\").EnableStdin(),\n\t},\n\tPreRun: urlArgsEncoder,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif err := urlArgsDecoder(req, env); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttopic := req.Arguments[0]\n\n\t\t// read data passed as a file\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\t\tdata, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// publish\n\t\treturn api.PubSub().Publish(req.Context, topic, data)\n\t},\n}\n\nvar PubsubLsCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List subscribed topics by name.\",\n\t\tShortDescription: `\nipfs pubsub ls lists out the names of topics you are currently subscribed to.\n\nEXPERIMENTAL FEATURE\n\n This is an opt-in feature optimized for IPNS over PubSub\n (https://specs.ipfs.tech/ipns/ipns-pubsub-router/).\n\n To enable, set 'Pubsub.Enabled' config to true.\n\nTOPIC ENCODING\n\n Topic names are a binary data. To ensure all bytes are transferred\n correctly RPC client and server will use multibase encoding behind\n the scenes.\n\n You can inspect the format by passing --enc=json. ipfs multibase commands\n can be used for encoding/decoding multibase strings in the userland.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tl, err := api.PubSub().Ls(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// emit topics encoded in multibase\n\t\tencoder, _ := mbase.EncoderByName(\"base64url\")\n\t\tfor n, topic := range l {\n\t\t\tl[n] = encoder.Encode([]byte(topic))\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, stringList{l})\n\t},\n\tType: stringList{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(multibaseDecodedStringListEncoder),\n\t},\n}\n\nfunc multibaseDecodedStringListEncoder(req *cmds.Request, w io.Writer, list *stringList) error {\n\tfor n, mb := range list.Strings {\n\t\t_, data, err := mbase.Decode(mb)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tlist.Strings[n] = string(data)\n\t}\n\treturn safeTextListEncoder(req, w, list)\n}\n\n// converts list of strings to text representation where each string is placed\n// in separate line with non-printable/unsafe characters escaped\n// (this protects terminal output from being mangled by non-ascii topic names)\nfunc safeTextListEncoder(req *cmds.Request, w io.Writer, list *stringList) error {\n\tfor _, str := range list.Strings {\n\t\t_, err := fmt.Fprintf(w, \"%s\\n\", cmdenv.EscNonPrint(str))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nvar PubsubPeersCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List peers we are currently pubsubbing with.\",\n\t\tShortDescription: `\nipfs pubsub peers with no arguments lists out the pubsub peers you are\ncurrently connected to. If given a topic, it will list connected peers who are\nsubscribed to the named topic.\n\nEXPERIMENTAL FEATURE\n\n This is an opt-in feature optimized for IPNS over PubSub\n (https://specs.ipfs.tech/ipns/ipns-pubsub-router/).\n\n To enable, set 'Pubsub.Enabled' config to true.\n\nTOPIC AND DATA ENCODING\n\n Topic names are a binary data. To ensure all bytes are transferred\n correctly RPC client and server will use multibase encoding behind\n the scenes.\n\n You can inspect the format by passing --enc=json. ipfs multibase commands\n can be used for encoding/decoding multibase strings in the userland.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"topic\", false, false, \"Topic to list connected peers of.\"),\n\t},\n\tPreRun: urlArgsEncoder,\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif err := urlArgsDecoder(req, env); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar topic string\n\t\tif len(req.Arguments) == 1 {\n\t\t\ttopic = req.Arguments[0]\n\t\t}\n\n\t\tpeers, err := api.PubSub().Peers(req.Context, options.PubSub.Topic(topic))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tlist := &stringList{make([]string, 0, len(peers))}\n\n\t\tfor _, peer := range peers {\n\t\t\tlist.Strings = append(list.Strings, peer.String())\n\t\t}\n\t\tslices.Sort(list.Strings)\n\t\treturn cmds.EmitOnce(res, list)\n\t},\n\tType: stringList{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n}\n\n// TODO: move to cmdenv?\n// Encode binary data to be passed as multibase string in URL arguments.\n// (avoiding issues described in https://github.com/ipfs/kubo/issues/7939)\nfunc urlArgsEncoder(req *cmds.Request, env cmds.Environment) error {\n\tencoder, _ := mbase.EncoderByName(\"base64url\")\n\tfor n, arg := range req.Arguments {\n\t\treq.Arguments[n] = encoder.Encode([]byte(arg))\n\t}\n\treturn nil\n}\n\n// Decode binary data passed as multibase string in URL arguments.\n// (avoiding issues described in https://github.com/ipfs/kubo/issues/7939)\nfunc urlArgsDecoder(req *cmds.Request, env cmds.Environment) error {\n\tfor n, arg := range req.Arguments {\n\t\tencoding, data, err := mbase.Decode(arg)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"URL arg must be multibase encoded: %w\", err)\n\t\t}\n\n\t\t// Enforce URL-safe encoding is used for data passed via URL arguments\n\t\t// - without this we get data corruption similar to https://github.com/ipfs/kubo/issues/7939\n\t\t// - we can't just deny base64, because there may be other bases that\n\t\t// are not URL-safe – better to force base64url which is known to be\n\t\t// safe in URL context\n\t\tif encoding != mbase.Base64url {\n\t\t\treturn errors.New(\"URL arg must be base64url encoded\")\n\t\t}\n\n\t\treq.Arguments[n] = string(data)\n\t}\n\treturn nil\n}\n\ntype pubsubResetResult struct {\n\tDeleted int64 `json:\"deleted\"`\n}\n\nvar PubsubResetCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Reset pubsub validator state.\",\n\t\tShortDescription: `\nClears persistent sequence number state used by the pubsub validator.\n\nWARNING: FOR TESTING ONLY - DO NOT USE IN PRODUCTION\n\nResets validator state that protects against replay attacks. After reset,\npreviously seen messages may be accepted again until their sequence numbers\nare re-learned.\n\nUse cases:\n- Testing pubsub functionality\n- Recovery from a peer sending artificially high sequence numbers\n (which would cause subsequent messages from that peer to be rejected)\n\nThe --peer flag limits the reset to a specific peer's state.\nWithout --peer, all validator state is cleared.\n\nNOTE: This only resets the persistent seqno validator state. The in-memory\nseen messages cache (Pubsub.SeenMessagesTTL) auto-expires and can only be\nfully cleared by restarting the daemon.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(peerOptionName, \"p\", \"Only reset state for this peer ID\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tds := n.Repo.Datastore()\n\t\tctx := req.Context\n\n\t\tpeerOpt, _ := req.Options[peerOptionName].(string)\n\n\t\tvar deleted int64\n\t\tif peerOpt != \"\" {\n\t\t\t// Reset specific peer\n\t\t\tpid, err := peer.Decode(peerOpt)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"invalid peer ID: %w\", err)\n\t\t\t}\n\t\t\tkey := datastore.NewKey(libp2p.SeqnoStorePrefix + pid.String())\n\t\t\texists, err := ds.Has(ctx, key)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to check seqno state: %w\", err)\n\t\t\t}\n\t\t\tif exists {\n\t\t\t\tif err := ds.Delete(ctx, key); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"failed to delete seqno state: %w\", err)\n\t\t\t\t}\n\t\t\t\tdeleted = 1\n\t\t\t}\n\t\t} else {\n\t\t\t// Reset all peers using batched delete for efficiency\n\t\t\tq := query.Query{\n\t\t\t\tPrefix: libp2p.SeqnoStorePrefix,\n\t\t\t\tKeysOnly: true,\n\t\t\t}\n\t\t\tresults, err := ds.Query(ctx, q)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to query seqno state: %w\", err)\n\t\t\t}\n\t\t\tdefer results.Close()\n\n\t\t\tbatch, err := ds.Batch(ctx)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create batch: %w\", err)\n\t\t\t}\n\n\t\t\tfor result := range results.Next() {\n\t\t\t\tif result.Error != nil {\n\t\t\t\t\treturn fmt.Errorf(\"query error: %w\", result.Error)\n\t\t\t\t}\n\t\t\t\tif err := batch.Delete(ctx, datastore.NewKey(result.Key)); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"failed to batch delete key %s: %w\", result.Key, err)\n\t\t\t\t}\n\t\t\t\tdeleted++\n\t\t\t}\n\n\t\t\tif err := batch.Commit(ctx); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to commit batch delete: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\t// Sync to ensure deletions are persisted\n\t\tif err := ds.Sync(ctx, datastore.NewKey(libp2p.SeqnoStorePrefix)); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to sync datastore: %w\", err)\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &pubsubResetResult{Deleted: deleted})\n\t},\n\tType: pubsubResetResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, result *pubsubResetResult) error {\n\t\t\tpeerOpt, _ := req.Options[peerOptionName].(string)\n\t\t\tif peerOpt != \"\" {\n\t\t\t\tif result.Deleted == 0 {\n\t\t\t\t\t_, err := fmt.Fprintf(w, \"No validator state found for peer %s\\n\", peerOpt)\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\t_, err := fmt.Fprintf(w, \"Reset validator state for peer %s\\n\", peerOpt)\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, err := fmt.Fprintf(w, \"Reset validator state for %d peer(s)\\n\", result.Deleted)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/refs.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tmerkledag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n)\n\nvar refsEncoderMap = cmds.EncoderMap{\n\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *RefWrapper) error {\n\t\tif out.Err != \"\" {\n\t\t\treturn errors.New(out.Err)\n\t\t}\n\t\tfmt.Fprintln(w, out.Ref)\n\n\t\treturn nil\n\t}),\n}\n\n// KeyList is a general type for outputting lists of keys\ntype KeyList struct {\n\tKeys []cid.Cid\n}\n\nconst (\n\trefsFormatOptionName = \"format\"\n\trefsEdgesOptionName = \"edges\"\n\trefsUniqueOptionName = \"unique\"\n\trefsRecursiveOptionName = \"recursive\"\n\trefsMaxDepthOptionName = \"max-depth\"\n)\n\n// RefsCmd is the `ipfs refs` command\nvar RefsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List links (references) from an object.\",\n\t\tShortDescription: `\nLists the hashes of all the links an IPFS or IPNS object(s) contains,\nwith the following format:\n\n \n\nList all references recursively by using the flag '-r'.\n\nNOTE: Like most other commands, Kubo will try to fetch the blocks of the passed path if they can't be found in the local store if it is running in online mode.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"local\": RefsLocalCmd,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ipfs-path\", true, true, \"Path to the object(s) to list refs from.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(refsFormatOptionName, \"Emit edges with given format. Available tokens: .\").WithDefault(\"\"),\n\t\tcmds.BoolOption(refsEdgesOptionName, \"e\", \"Emit edge format: ` -> `.\"),\n\t\tcmds.BoolOption(refsUniqueOptionName, \"u\", \"Omit duplicate refs from output.\"),\n\t\tcmds.BoolOption(refsRecursiveOptionName, \"r\", \"Recursively list links of child nodes.\"),\n\t\tcmds.IntOption(refsMaxDepthOptionName, \"Only for recursive refs, limits fetch and listing to the given depth\").WithDefault(-1),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\terr := req.ParseBodyArgs()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tctx := req.Context\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tenc, err := cmdenv.GetCidEncoder(req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tunique, _ := req.Options[refsUniqueOptionName].(bool)\n\t\trecursive, _ := req.Options[refsRecursiveOptionName].(bool)\n\t\tmaxDepth, _ := req.Options[refsMaxDepthOptionName].(int)\n\t\tedges, _ := req.Options[refsEdgesOptionName].(bool)\n\t\tformat, _ := req.Options[refsFormatOptionName].(string)\n\n\t\tif !recursive {\n\t\t\tmaxDepth = 1 // write only direct refs\n\t\t}\n\n\t\tif edges {\n\t\t\tif format != \"\" {\n\t\t\t\treturn errors.New(\"using format argument with edges is not allowed\")\n\t\t\t}\n\n\t\t\tformat = \" -> \"\n\t\t}\n\n\t\t// TODO: use session for resolving as well.\n\t\tobjs, err := objectsForPaths(ctx, api, req.Arguments)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trw := RefWriter{\n\t\t\tres: res,\n\t\t\tDAG: merkledag.NewSession(ctx, api.Dag()),\n\t\t\tCtx: ctx,\n\t\t\tUnique: unique,\n\t\t\tPrintFmt: format,\n\t\t\tMaxDepth: maxDepth,\n\t\t}\n\n\t\tfor _, o := range objs {\n\t\t\tif _, err := rw.WriteRefs(o, enc); err != nil {\n\t\t\t\tif err := res.Emit(&RefWrapper{Err: err.Error()}); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tEncoders: refsEncoderMap,\n\tType: RefWrapper{},\n}\n\nvar RefsLocalCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List all local references.\",\n\t\tShortDescription: `\nDisplays the hashes of all local objects. NOTE: This treats all local objects as \"raw blocks\" and returns CIDv1-Raw CIDs.\n`,\n\t},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tctx := req.Context\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// todo: make async\n\t\tallKeys, err := n.Blockstore.AllKeysChan(ctx)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor k := range allKeys {\n\t\t\terr := res.Emit(&RefWrapper{Ref: k.String()})\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tEncoders: refsEncoderMap,\n\tType: RefWrapper{},\n}\n\nfunc objectsForPaths(ctx context.Context, n iface.CoreAPI, paths []string) ([]cid.Cid, error) {\n\troots := make([]cid.Cid, len(paths))\n\tfor i, sp := range paths {\n\t\tp, err := cmdutils.PathOrCidPath(sp)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\to, _, err := n.ResolvePath(ctx, p)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\troots[i] = o.RootCid()\n\t}\n\treturn roots, nil\n}\n\ntype RefWrapper struct {\n\tRef string\n\tErr string\n}\n\ntype RefWriter struct {\n\tres cmds.ResponseEmitter\n\tDAG ipld.NodeGetter\n\tCtx context.Context\n\n\tUnique bool\n\tMaxDepth int\n\tPrintFmt string\n\n\tseen map[string]int\n}\n\n// WriteRefs writes refs of the given object to the underlying writer.\nfunc (rw *RefWriter) WriteRefs(c cid.Cid, enc cidenc.Encoder) (int, error) {\n\tn, err := rw.DAG.Get(rw.Ctx, c)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\treturn rw.writeRefsRecursive(n, 0, enc)\n}\n\nfunc (rw *RefWriter) writeRefsRecursive(n ipld.Node, depth int, enc cidenc.Encoder) (int, error) {\n\tnc := n.Cid()\n\n\tvar count int\n\tfor i, ng := range ipld.GetDAG(rw.Ctx, rw.DAG, n) {\n\t\tlc := n.Links()[i].Cid\n\t\tgoDeeper, shouldWrite := rw.visit(lc, depth+1) // The children are at depth+1\n\n\t\t// Avoid \"Get()\" on the node and continue with next Link.\n\t\t// We can do this if:\n\t\t// - We printed it before (thus it was already seen and\n\t\t// fetched with Get()\n\t\t// - AND we must not go deeper.\n\t\t// This is an optimization for pruned branches which have been\n\t\t// visited before.\n\t\tif !shouldWrite && !goDeeper {\n\t\t\tcontinue\n\t\t}\n\n\t\t// We must Get() the node because:\n\t\t// - it is new (never written)\n\t\t// - OR we need to go deeper.\n\t\t// This ensures printed refs are always fetched.\n\t\tnd, err := ng.Get(rw.Ctx)\n\t\tif err != nil {\n\t\t\treturn count, err\n\t\t}\n\n\t\t// Write this node if not done before (or !Unique)\n\t\tif shouldWrite {\n\t\t\tif err := rw.WriteEdge(nc, lc, n.Links()[i].Name, enc); err != nil {\n\t\t\t\treturn count, err\n\t\t\t}\n\t\t\tcount++\n\t\t}\n\n\t\t// Keep going deeper. This happens:\n\t\t// - On unexplored branches\n\t\t// - On branches not explored deep enough\n\t\t// Note when !Unique, branches are always considered\n\t\t// unexplored and only depth limits apply.\n\t\tif goDeeper {\n\t\t\tc, err := rw.writeRefsRecursive(nd, depth+1, enc)\n\t\t\tcount += c\n\t\t\tif err != nil {\n\t\t\t\treturn count, err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn count, nil\n}\n\n// visit returns two values:\n// - the first boolean is true if we should keep traversing the DAG\n// - the second boolean is true if we should print the CID\n//\n// visit will do branch pruning depending on rw.MaxDepth, previously visited\n// cids and whether rw.Unique is set. i.e. rw.Unique = false and\n// rw.MaxDepth = -1 disables any pruning. But setting rw.Unique to true will\n// prune already visited branches at the cost of keeping as set of visited\n// CIDs in memory.\nfunc (rw *RefWriter) visit(c cid.Cid, depth int) (bool, bool) {\n\tatMaxDepth := rw.MaxDepth >= 0 && depth == rw.MaxDepth\n\toverMaxDepth := rw.MaxDepth >= 0 && depth > rw.MaxDepth\n\n\t// Shortcut when we are over max depth. In practice, this\n\t// only applies when calling refs with --maxDepth=0, as root's\n\t// children are already over max depth. Otherwise nothing should\n\t// hit this.\n\tif overMaxDepth {\n\t\treturn false, false\n\t}\n\n\t// We can shortcut right away if we don't need unique output:\n\t// - we keep traversing when not atMaxDepth\n\t// - always print\n\tif !rw.Unique {\n\t\treturn !atMaxDepth, true\n\t}\n\n\t// Unique == true from this point.\n\t// Thus, we keep track of seen Cids, and their depth.\n\tif rw.seen == nil {\n\t\trw.seen = make(map[string]int)\n\t}\n\tkey := string(c.Bytes())\n\toldDepth, ok := rw.seen[key]\n\n\t// Unique == true && depth < MaxDepth (or unlimited) from this point\n\n\t// Branch pruning cases:\n\t// - We saw the Cid before and either:\n\t// - Depth is unlimited (MaxDepth = -1)\n\t// - We saw it higher (smaller depth) in the DAG (means we must have\n\t// explored deep enough before)\n\t// Because we saw the CID, we don't print it again.\n\tif ok && (rw.MaxDepth < 0 || oldDepth <= depth) {\n\t\treturn false, false\n\t}\n\n\t// Final case, we must keep exploring the DAG from this CID\n\t// (unless we hit the depth limit).\n\t// We note down its depth because it was either not seen\n\t// or is lower than last time.\n\t// We print if it was not seen.\n\trw.seen[key] = depth\n\treturn !atMaxDepth, !ok\n}\n\n// Write one edge\nfunc (rw *RefWriter) WriteEdge(from, to cid.Cid, linkname string, enc cidenc.Encoder) error {\n\tif rw.Ctx != nil {\n\t\tselect {\n\t\tcase <-rw.Ctx.Done(): // just in case.\n\t\t\treturn rw.Ctx.Err()\n\t\tdefault:\n\t\t}\n\t}\n\n\tvar s string\n\tswitch {\n\tcase rw.PrintFmt != \"\":\n\t\ts = rw.PrintFmt\n\t\ts = strings.Replace(s, \"\", enc.Encode(from), -1)\n\t\ts = strings.Replace(s, \"\", enc.Encode(to), -1)\n\t\ts = strings.Replace(s, \"\", linkname, -1)\n\tdefault:\n\t\ts += enc.Encode(to)\n\t}\n\n\treturn rw.res.Emit(&RefWrapper{Ref: s})\n}\n" }, { "path": "core/commands/repo.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"runtime\"\n\t\"strings\"\n\t\"sync\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcorerepo \"github.com/ipfs/kubo/core/corerepo\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/path\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\ntype RepoVersion struct {\n\tVersion string\n}\n\nvar RepoCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Manipulate the IPFS repo.\",\n\t\tShortDescription: `\n'ipfs repo' is a plumbing command used to manipulate the repo.\n`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"stat\": repoStatCmd,\n\t\t\"gc\": repoGcCmd,\n\t\t\"version\": repoVersionCmd,\n\t\t\"verify\": repoVerifyCmd,\n\t\t\"migrate\": repoMigrateCmd,\n\t\t\"ls\": RefsLocalCmd,\n\t},\n}\n\n// GcResult is the result returned by \"repo gc\" command.\ntype GcResult struct {\n\tKey cid.Cid\n\tError string `json:\",omitempty\"`\n}\n\nconst (\n\trepoStreamErrorsOptionName = \"stream-errors\"\n\trepoQuietOptionName = \"quiet\"\n\trepoSilentOptionName = \"silent\"\n\trepoAllowDowngradeOptionName = \"allow-downgrade\"\n\trepoToVersionOptionName = \"to\"\n)\n\nvar repoGcCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Perform a garbage collection sweep on the repo.\",\n\t\tShortDescription: `\n'ipfs repo gc' is a plumbing command that will sweep the local\nset of stored objects and remove ones that are not pinned in\norder to reclaim hard disk space.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(repoStreamErrorsOptionName, \"Stream errors.\"),\n\t\tcmds.BoolOption(repoQuietOptionName, \"q\", \"Write minimal output.\"),\n\t\tcmds.BoolOption(repoSilentOptionName, \"Write no output.\"),\n\t},\n\tRun: func(req *cmds.Request, re cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsilent, _ := req.Options[repoSilentOptionName].(bool)\n\t\tstreamErrors, _ := req.Options[repoStreamErrorsOptionName].(bool)\n\n\t\tgcOutChan := corerepo.GarbageCollectAsync(n, req.Context)\n\n\t\tif streamErrors {\n\t\t\terrs := false\n\t\t\tfor res := range gcOutChan {\n\t\t\t\tif res.Error != nil {\n\t\t\t\t\tif err := re.Emit(&GcResult{Error: res.Error.Error()}); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\terrs = true\n\t\t\t\t} else {\n\t\t\t\t\tif err := re.Emit(&GcResult{Key: res.KeyRemoved}); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tif errs {\n\t\t\t\treturn errors.New(\"encountered errors during gc run\")\n\t\t\t}\n\t\t} else {\n\t\t\terr := corerepo.CollectResult(req.Context, gcOutChan, func(k cid.Cid) {\n\t\t\t\tif silent {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\t// Nothing to do with this error, really. This\n\t\t\t\t// most likely means that the client is gone but\n\t\t\t\t// we still need to let the GC finish.\n\t\t\t\t_ = re.Emit(&GcResult{Key: k})\n\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tType: GcResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, gcr *GcResult) error {\n\t\t\tquiet, _ := req.Options[repoQuietOptionName].(bool)\n\t\t\tsilent, _ := req.Options[repoSilentOptionName].(bool)\n\n\t\t\tif silent {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif gcr.Error != \"\" {\n\t\t\t\t_, err := fmt.Fprintf(w, \"Error: %s\\n\", gcr.Error)\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tprefix := \"removed \"\n\t\t\tif quiet {\n\t\t\t\tprefix = \"\"\n\t\t\t}\n\n\t\t\t_, err := fmt.Fprintf(w, \"%s%s\\n\", prefix, gcr.Key)\n\t\t\treturn err\n\t\t}),\n\t},\n}\n\nconst (\n\trepoSizeOnlyOptionName = \"size-only\"\n\trepoHumanOptionName = \"human\"\n)\n\nvar repoStatCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Get stats for the currently used repo.\",\n\t\tShortDescription: `\n'ipfs repo stat' provides information about the local set of\nstored objects. It outputs:\n\nRepoSize int Size in bytes that the repo is currently taking.\nStorageMax string Maximum datastore size (from configuration)\nNumObjects int Number of objects in the local repo.\nRepoPath string The path to the repo being currently used.\nVersion string The repo version.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(repoSizeOnlyOptionName, \"s\", \"Only report RepoSize and StorageMax.\"),\n\t\tcmds.BoolOption(repoHumanOptionName, \"H\", \"Print sizes in human readable format (e.g., 1K 234M 2G)\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsizeOnly, _ := req.Options[repoSizeOnlyOptionName].(bool)\n\t\tif sizeOnly {\n\t\t\tsizeStat, err := corerepo.RepoSize(req.Context, n)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn cmds.EmitOnce(res, &corerepo.Stat{\n\t\t\t\tSizeStat: sizeStat,\n\t\t\t})\n\t\t}\n\n\t\tstat, err := corerepo.RepoStat(req.Context, n)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &stat)\n\t},\n\tType: &corerepo.Stat{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, stat *corerepo.Stat) error {\n\t\t\twtr := tabwriter.NewWriter(w, 0, 0, 1, ' ', 0)\n\t\t\tdefer wtr.Flush()\n\n\t\t\thuman, _ := req.Options[repoHumanOptionName].(bool)\n\t\t\tsizeOnly, _ := req.Options[repoSizeOnlyOptionName].(bool)\n\n\t\t\tprintSize := func(name string, size uint64) {\n\t\t\t\tsizeStr := fmt.Sprintf(\"%d\", size)\n\t\t\t\tif human {\n\t\t\t\t\tsizeStr = humanize.Bytes(size)\n\t\t\t\t}\n\n\t\t\t\tfmt.Fprintf(wtr, \"%s:\\t%s\\n\", name, sizeStr)\n\t\t\t}\n\n\t\t\tif !sizeOnly {\n\t\t\t\tfmt.Fprintf(wtr, \"NumObjects:\\t%d\\n\", stat.NumObjects)\n\t\t\t}\n\n\t\t\tprintSize(\"RepoSize\", stat.RepoSize)\n\t\t\tprintSize(\"StorageMax\", stat.StorageMax)\n\n\t\t\tif !sizeOnly {\n\t\t\t\tfmt.Fprintf(wtr, \"RepoPath:\\t%s\\n\", stat.RepoPath)\n\t\t\t\tfmt.Fprintf(wtr, \"Version:\\t%s\\n\", stat.Version)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\n// VerifyProgress reports verification progress to the user.\n// It contains either a message about a corrupt block or a progress counter.\ntype VerifyProgress struct {\n\tMsg string // Message about a corrupt/healed block (empty for valid blocks)\n\tProgress int // Number of blocks processed so far\n}\n\n// verifyState represents the state of a block after verification.\n// States track both the verification result and any remediation actions taken.\ntype verifyState int\n\nconst (\n\tverifyStateValid verifyState = iota // Block is valid and uncorrupted\n\tverifyStateCorrupt // Block is corrupt, no action taken\n\tverifyStateCorruptRemoved // Block was corrupt and successfully removed\n\tverifyStateCorruptRemoveFailed // Block was corrupt but removal failed\n\tverifyStateCorruptHealed // Block was corrupt, removed, and successfully re-fetched\n\tverifyStateCorruptHealFailed // Block was corrupt and removed, but re-fetching failed\n)\n\nconst (\n\t// verifyWorkerMultiplier determines worker pool size relative to CPU count.\n\t// Since block verification is I/O-bound (disk reads + potential network fetches),\n\t// we use more workers than CPU cores to maximize throughput.\n\tverifyWorkerMultiplier = 2\n)\n\n// verifyResult contains the outcome of verifying a single block.\n// It includes the block's CID, its verification state, and an optional\n// human-readable message describing what happened.\ntype verifyResult struct {\n\tcid cid.Cid // CID of the block that was verified\n\tstate verifyState // Final state after verification and any remediation\n\tmsg string // Human-readable message (empty for valid blocks)\n}\n\n// verifyWorkerRun processes CIDs from the keys channel, verifying their integrity.\n// If shouldDrop is true, corrupt blocks are removed from the blockstore.\n// If shouldHeal is true (implies shouldDrop), removed blocks are re-fetched from the network.\n// The api parameter must be non-nil when shouldHeal is true.\n// healTimeout specifies the maximum time to wait for each block heal (0 = no timeout).\nfunc verifyWorkerRun(ctx context.Context, wg *sync.WaitGroup, keys <-chan cid.Cid, results chan<- *verifyResult, bs bstore.Blockstore, api coreiface.CoreAPI, shouldDrop, shouldHeal bool, healTimeout time.Duration) {\n\tdefer wg.Done()\n\n\tsendResult := func(r *verifyResult) bool {\n\t\tselect {\n\t\tcase results <- r:\n\t\t\treturn true\n\t\tcase <-ctx.Done():\n\t\t\treturn false\n\t\t}\n\t}\n\n\tfor k := range keys {\n\t\t_, err := bs.Get(ctx, k)\n\t\tif err != nil {\n\t\t\t// Block is corrupt\n\t\t\tresult := &verifyResult{cid: k, state: verifyStateCorrupt}\n\n\t\t\tif !shouldDrop {\n\t\t\t\tresult.msg = fmt.Sprintf(\"block %s was corrupt (%s)\", k, err)\n\t\t\t\tif !sendResult(result) {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Try to delete\n\t\t\tif delErr := bs.DeleteBlock(ctx, k); delErr != nil {\n\t\t\t\tresult.state = verifyStateCorruptRemoveFailed\n\t\t\t\tresult.msg = fmt.Sprintf(\"block %s was corrupt (%s), failed to remove (%s)\", k, err, delErr)\n\t\t\t\tif !sendResult(result) {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif !shouldHeal {\n\t\t\t\tresult.state = verifyStateCorruptRemoved\n\t\t\t\tresult.msg = fmt.Sprintf(\"block %s was corrupt (%s), removed\", k, err)\n\t\t\t\tif !sendResult(result) {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Try to heal by re-fetching from network (api is guaranteed non-nil here)\n\t\t\thealCtx := ctx\n\t\t\tvar healCancel context.CancelFunc\n\t\t\tif healTimeout > 0 {\n\t\t\t\thealCtx, healCancel = context.WithTimeout(ctx, healTimeout)\n\t\t\t}\n\n\t\t\tif _, healErr := api.Block().Get(healCtx, path.FromCid(k)); healErr != nil {\n\t\t\t\tresult.state = verifyStateCorruptHealFailed\n\t\t\t\tresult.msg = fmt.Sprintf(\"block %s was corrupt (%s), removed, failed to heal (%s)\", k, err, healErr)\n\t\t\t} else {\n\t\t\t\tresult.state = verifyStateCorruptHealed\n\t\t\t\tresult.msg = fmt.Sprintf(\"block %s was corrupt (%s), removed, healed\", k, err)\n\t\t\t}\n\n\t\t\tif healCancel != nil {\n\t\t\t\thealCancel()\n\t\t\t}\n\n\t\t\tif !sendResult(result) {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\t// Block is valid\n\t\tif !sendResult(&verifyResult{cid: k, state: verifyStateValid}) {\n\t\t\treturn\n\t\t}\n\t}\n}\n\n// verifyResultChan creates a channel of verification results by spawning multiple worker goroutines\n// to process blocks in parallel. It returns immediately with a channel that will receive results.\nfunc verifyResultChan(ctx context.Context, keys <-chan cid.Cid, bs bstore.Blockstore, api coreiface.CoreAPI, shouldDrop, shouldHeal bool, healTimeout time.Duration) <-chan *verifyResult {\n\tresults := make(chan *verifyResult)\n\n\tgo func() {\n\t\tdefer close(results)\n\n\t\tvar wg sync.WaitGroup\n\n\t\tfor i := 0; i < runtime.NumCPU()*verifyWorkerMultiplier; i++ {\n\t\t\twg.Add(1)\n\t\t\tgo verifyWorkerRun(ctx, &wg, keys, results, bs, api, shouldDrop, shouldHeal, healTimeout)\n\t\t}\n\n\t\twg.Wait()\n\t}()\n\n\treturn results\n}\n\nvar repoVerifyCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Verify all blocks in repo are not corrupted.\",\n\t\tShortDescription: `\n'ipfs repo verify' checks integrity of all blocks in the local datastore.\nEach block is read and validated against its CID to ensure data integrity.\n\nWithout any flags, this is a SAFE, read-only check that only reports corrupt\nblocks without modifying the repository. This can be used as a \"dry run\" to\npreview what --drop or --heal would do.\n\nUse --drop to remove corrupt blocks, or --heal to remove and re-fetch from\nthe network.\n\nExamples:\n ipfs repo verify # safe read-only check, reports corrupt blocks\n ipfs repo verify --drop # remove corrupt blocks\n ipfs repo verify --heal # remove and re-fetch corrupt blocks\n\nExit Codes:\n 0: All blocks are valid, OR all corrupt blocks were successfully remediated\n (with --drop or --heal)\n 1: Corrupt blocks detected (without flags), OR remediation failed (block\n removal or healing failed with --drop or --heal)\n\nNote: --heal requires the daemon to be running in online mode with network\nconnectivity to nodes that have the missing blocks. Make sure the daemon is\nonline and connected to other peers. Healing will attempt to re-fetch each\ncorrupt block from the network after removing it. If a block cannot be found\non the network, it will remain deleted.\n\nWARNING: Both --drop and --heal are DESTRUCTIVE operations that permanently\ndelete corrupt blocks from your repository. Once deleted, blocks cannot be\nrecovered unless --heal successfully fetches them from the network. Blocks\nthat cannot be healed will remain permanently deleted. Always backup your\nrepository before using these options.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(\"drop\", \"Remove corrupt blocks from datastore (destructive operation).\"),\n\t\tcmds.BoolOption(\"heal\", \"Remove corrupt blocks and re-fetch from network (destructive operation, implies --drop).\"),\n\t\tcmds.StringOption(\"heal-timeout\", \"Maximum time to wait for each block heal (e.g., \\\"30s\\\"). Only applies with --heal.\").WithDefault(\"30s\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdrop, _ := req.Options[\"drop\"].(bool)\n\t\theal, _ := req.Options[\"heal\"].(bool)\n\n\t\tif heal {\n\t\t\tdrop = true // heal implies drop\n\t\t}\n\n\t\t// Parse and validate heal-timeout\n\t\ttimeoutStr, _ := req.Options[\"heal-timeout\"].(string)\n\t\thealTimeout, err := time.ParseDuration(timeoutStr)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"invalid heal-timeout: %w\", err)\n\t\t}\n\t\tif healTimeout < 0 {\n\t\t\treturn errors.New(\"heal-timeout must be >= 0\")\n\t\t}\n\n\t\t// Check online mode and API availability for healing operation\n\t\tvar api coreiface.CoreAPI\n\t\tif heal {\n\t\t\tif !nd.IsOnline {\n\t\t\t\treturn ErrNotOnline\n\t\t\t}\n\t\t\tapi, err = cmdenv.GetApi(env, req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif api == nil {\n\t\t\t\treturn fmt.Errorf(\"healing requested but API is not available - make sure daemon is online and connected to other peers\")\n\t\t\t}\n\t\t}\n\n\t\tbs := &bstore.ValidatingBlockstore{Blockstore: bstore.NewBlockstore(nd.Repo.Datastore())}\n\n\t\tkeys, err := bs.AllKeysChan(req.Context)\n\t\tif err != nil {\n\t\t\tlog.Error(err)\n\t\t\treturn err\n\t\t}\n\n\t\tresults := verifyResultChan(req.Context, keys, bs, api, drop, heal, healTimeout)\n\n\t\t// Track statistics for each type of outcome\n\t\tvar corrupted, removed, removeFailed, healed, healFailed int\n\t\tvar i int\n\n\t\tfor result := range results {\n\t\t\t// Update counters based on the block's final state\n\t\t\tswitch result.state {\n\t\t\tcase verifyStateCorrupt:\n\t\t\t\t// Block is corrupt but no action was taken (--drop not specified)\n\t\t\t\tcorrupted++\n\t\t\tcase verifyStateCorruptRemoved:\n\t\t\t\t// Block was corrupt and successfully removed (--drop specified)\n\t\t\t\tcorrupted++\n\t\t\t\tremoved++\n\t\t\tcase verifyStateCorruptRemoveFailed:\n\t\t\t\t// Block was corrupt but couldn't be removed\n\t\t\t\tcorrupted++\n\t\t\t\tremoveFailed++\n\t\t\tcase verifyStateCorruptHealed:\n\t\t\t\t// Block was corrupt, removed, and successfully re-fetched (--heal specified)\n\t\t\t\tcorrupted++\n\t\t\t\tremoved++\n\t\t\t\thealed++\n\t\t\tcase verifyStateCorruptHealFailed:\n\t\t\t\t// Block was corrupt and removed, but re-fetching failed\n\t\t\t\tcorrupted++\n\t\t\t\tremoved++\n\t\t\t\thealFailed++\n\t\t\tdefault:\n\t\t\t\t// verifyStateValid blocks are not counted (they're the expected case)\n\t\t\t}\n\n\t\t\t// Emit progress message for corrupt blocks\n\t\t\tif result.state != verifyStateValid && result.msg != \"\" {\n\t\t\t\tif err := res.Emit(&VerifyProgress{Msg: result.msg}); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\ti++\n\t\t\tif err := res.Emit(&VerifyProgress{Progress: i}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif err := req.Context.Err(); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif corrupted > 0 {\n\t\t\t// Build a summary of what happened with corrupt blocks\n\t\t\tsummary := fmt.Sprintf(\"verify complete, %d blocks corrupt\", corrupted)\n\t\t\tif removed > 0 {\n\t\t\t\tsummary += fmt.Sprintf(\", %d removed\", removed)\n\t\t\t}\n\t\t\tif removeFailed > 0 {\n\t\t\t\tsummary += fmt.Sprintf(\", %d failed to remove\", removeFailed)\n\t\t\t}\n\t\t\tif healed > 0 {\n\t\t\t\tsummary += fmt.Sprintf(\", %d healed\", healed)\n\t\t\t}\n\t\t\tif healFailed > 0 {\n\t\t\t\tsummary += fmt.Sprintf(\", %d failed to heal\", healFailed)\n\t\t\t}\n\n\t\t\t// Determine success/failure based on operation mode\n\t\t\tshouldFail := false\n\n\t\t\tif !drop {\n\t\t\t\t// Detection-only mode: always fail if corruption found\n\t\t\t\tshouldFail = true\n\t\t\t} else if heal {\n\t\t\t\t// Heal mode: fail if any removal or heal failed\n\t\t\t\tshouldFail = (removeFailed > 0 || healFailed > 0)\n\t\t\t} else {\n\t\t\t\t// Drop mode: fail if any removal failed\n\t\t\t\tshouldFail = (removeFailed > 0)\n\t\t\t}\n\n\t\t\tif shouldFail {\n\t\t\t\treturn errors.New(summary)\n\t\t\t}\n\n\t\t\t// Success: emit summary as a message instead of error\n\t\t\treturn res.Emit(&VerifyProgress{Msg: summary})\n\t\t}\n\n\t\treturn res.Emit(&VerifyProgress{Msg: \"verify complete, all blocks validated.\"})\n\t},\n\tType: &VerifyProgress{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, obj *VerifyProgress) error {\n\t\t\tif strings.Contains(obj.Msg, \"was corrupt\") {\n\t\t\t\tfmt.Fprintln(w, obj.Msg)\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif obj.Msg != \"\" {\n\t\t\t\tif len(obj.Msg) < 20 {\n\t\t\t\t\tobj.Msg += \" \"\n\t\t\t\t}\n\t\t\t\tfmt.Fprintln(w, obj.Msg)\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tfmt.Fprintf(w, \"%d blocks processed.\\r\", obj.Progress)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar repoVersionCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show the repo version.\",\n\t\tShortDescription: `\n'ipfs repo version' returns the current repo version.\n`,\n\t},\n\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(repoQuietOptionName, \"q\", \"Write minimal output.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\treturn cmds.EmitOnce(res, &RepoVersion{\n\t\t\tVersion: fmt.Sprint(fsrepo.RepoVersion),\n\t\t})\n\t},\n\tType: RepoVersion{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *RepoVersion) error {\n\t\t\tquiet, _ := req.Options[repoQuietOptionName].(bool)\n\n\t\t\tif quiet {\n\t\t\t\tfmt.Fprintf(w, \"fs-repo@%s\\n\", out.Version)\n\t\t\t} else {\n\t\t\t\tfmt.Fprintf(w, \"ipfs repo version fs-repo@%s\\n\", out.Version)\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar repoMigrateCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Apply repository migrations to a specific version.\",\n\t\tShortDescription: `\n'ipfs repo migrate' applies repository migrations to bring the repository\nto a specific version. By default, migrates to the latest version supported\nby this IPFS binary.\n\nExamples:\n ipfs repo migrate # Migrate to latest version\n ipfs repo migrate --to=17 # Migrate to version 17\n ipfs repo migrate --to=16 --allow-downgrade # Downgrade to version 16\n\nWARNING: Downgrading a repository may cause data loss and requires using\nan older IPFS binary that supports the target version. After downgrading,\nyou must use an IPFS implementation compatible with that repository version.\n\nRepository versions 16+ use embedded migrations for faster, more reliable\nmigration. Versions below 16 require external migration tools.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.IntOption(repoToVersionOptionName, \"Target repository version\").WithDefault(fsrepo.RepoVersion),\n\t\tcmds.BoolOption(repoAllowDowngradeOptionName, \"Allow downgrading to a lower repo version\"),\n\t},\n\tNoRemote: true,\n\t// SetDoesNotUseRepo(true) might seem counter-intuitive since migrations\n\t// do access the repo, but it's correct - we need direct filesystem access\n\t// without going through the daemon. Migrations handle their own locking.\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true)),\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tcctx := env.(*oldcmds.Context)\n\t\tallowDowngrade, _ := req.Options[repoAllowDowngradeOptionName].(bool)\n\t\ttargetVersion, _ := req.Options[repoToVersionOptionName].(int)\n\n\t\t// Get current repo version\n\t\tcurrentVersion, err := migrations.RepoVersion(cctx.ConfigRoot)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not get current repo version: %w\", err)\n\t\t}\n\n\t\t// Check if migration is needed\n\t\tif currentVersion == targetVersion {\n\t\t\tfmt.Printf(\"Repository is already at version %d.\\n\", targetVersion)\n\t\t\treturn nil\n\t\t}\n\n\t\t// Validate downgrade request\n\t\tif targetVersion < currentVersion && !allowDowngrade {\n\t\t\treturn fmt.Errorf(\"downgrade from version %d to %d requires --allow-downgrade flag\", currentVersion, targetVersion)\n\t\t}\n\n\t\tfmt.Printf(\"Migrating repository from version %d to %d...\\n\", currentVersion, targetVersion)\n\n\t\t// Use hybrid migration strategy that intelligently combines external and embedded migrations\n\t\t// Use req.Context instead of cctx.Context() to avoid opening the repo before migrations run,\n\t\t// which would acquire the lock that migrations need\n\t\terr = migrations.RunHybridMigrations(req.Context, targetVersion, cctx.ConfigRoot, allowDowngrade)\n\t\tif err != nil {\n\t\t\tfmt.Println(\"Repository migration failed:\")\n\t\t\tfmt.Printf(\" %s\\n\", err)\n\t\t\tfmt.Println(\"If you think this is a bug, please file an issue and include this whole log output.\")\n\t\t\tfmt.Println(\" https://github.com/ipfs/kubo\")\n\t\t\treturn err\n\t\t}\n\n\t\tfmt.Printf(\"Repository successfully migrated to version %d.\\n\", targetVersion)\n\t\tif targetVersion < fsrepo.RepoVersion {\n\t\t\tfmt.Println(\"WARNING: After downgrading, you must use an IPFS binary compatible with this repository version.\")\n\t\t}\n\t\treturn nil\n\t},\n}\n" }, { "path": "core/commands/repo_verify_test.go", "content": "//go:build go1.25\n\npackage commands\n\n// This file contains unit tests for the --heal-timeout flag functionality\n// using testing/synctest to avoid waiting for real timeouts.\n//\n// End-to-end tests for the full 'ipfs repo verify' command (including --drop\n// and --heal flags) are located in test/cli/repo_verify_test.go.\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"sync\"\n\t\"testing\"\n\t\"testing/synctest\"\n\t\"time\"\n\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/ipfs/boxo/path\"\n)\n\nfunc TestVerifyWorkerHealTimeout(t *testing.T) {\n\tt.Run(\"heal succeeds before timeout\", func(t *testing.T) {\n\t\tsynctest.Test(t, func(t *testing.T) {\n\t\t\tconst healTimeout = 5 * time.Second\n\t\t\ttestCID := cid.MustParse(\"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\n\t\t\t// Setup channels\n\t\t\tkeys := make(chan cid.Cid, 1)\n\t\t\tkeys <- testCID\n\t\t\tclose(keys)\n\t\t\tresults := make(chan *verifyResult, 1)\n\n\t\t\t// Mock blockstore that returns error (simulating corruption)\n\t\t\tmockBS := &mockBlockstore{\n\t\t\t\tgetError: errors.New(\"corrupt block\"),\n\t\t\t}\n\n\t\t\t// Mock API where Block().Get() completes before timeout\n\t\t\tmockAPI := &mockCoreAPI{\n\t\t\t\tblockAPI: &mockBlockAPI{\n\t\t\t\t\tgetDelay: 2 * time.Second, // Less than healTimeout\n\t\t\t\t\tdata: []byte(\"healed data\"),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tvar wg sync.WaitGroup\n\t\t\twg.Add(1)\n\n\t\t\t// Run worker\n\t\t\tgo verifyWorkerRun(t.Context(), &wg, keys, results, mockBS, mockAPI, true, true, healTimeout)\n\n\t\t\t// Advance time past the mock delay but before timeout\n\t\t\ttime.Sleep(3 * time.Second)\n\t\t\tsynctest.Wait()\n\n\t\t\twg.Wait()\n\t\t\tclose(results)\n\n\t\t\t// Verify heal succeeded\n\t\t\tresult := <-results\n\t\t\trequire.NotNil(t, result)\n\t\t\tassert.Equal(t, verifyStateCorruptHealed, result.state)\n\t\t\tassert.Contains(t, result.msg, \"healed\")\n\t\t})\n\t})\n\n\tt.Run(\"heal fails due to timeout\", func(t *testing.T) {\n\t\tsynctest.Test(t, func(t *testing.T) {\n\t\t\tconst healTimeout = 2 * time.Second\n\t\t\ttestCID := cid.MustParse(\"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\n\t\t\t// Setup channels\n\t\t\tkeys := make(chan cid.Cid, 1)\n\t\t\tkeys <- testCID\n\t\t\tclose(keys)\n\t\t\tresults := make(chan *verifyResult, 1)\n\n\t\t\t// Mock blockstore that returns error (simulating corruption)\n\t\t\tmockBS := &mockBlockstore{\n\t\t\t\tgetError: errors.New(\"corrupt block\"),\n\t\t\t}\n\n\t\t\t// Mock API where Block().Get() takes longer than healTimeout\n\t\t\tmockAPI := &mockCoreAPI{\n\t\t\t\tblockAPI: &mockBlockAPI{\n\t\t\t\t\tgetDelay: 5 * time.Second, // More than healTimeout\n\t\t\t\t\tdata: []byte(\"healed data\"),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tvar wg sync.WaitGroup\n\t\t\twg.Add(1)\n\n\t\t\t// Run worker\n\t\t\tgo verifyWorkerRun(t.Context(), &wg, keys, results, mockBS, mockAPI, true, true, healTimeout)\n\n\t\t\t// Advance time past timeout\n\t\t\ttime.Sleep(3 * time.Second)\n\t\t\tsynctest.Wait()\n\n\t\t\twg.Wait()\n\t\t\tclose(results)\n\n\t\t\t// Verify heal failed due to timeout\n\t\t\tresult := <-results\n\t\t\trequire.NotNil(t, result)\n\t\t\tassert.Equal(t, verifyStateCorruptHealFailed, result.state)\n\t\t\tassert.Contains(t, result.msg, \"failed to heal\")\n\t\t\tassert.Contains(t, result.msg, \"context deadline exceeded\")\n\t\t})\n\t})\n\n\tt.Run(\"heal with zero timeout still attempts heal\", func(t *testing.T) {\n\t\tsynctest.Test(t, func(t *testing.T) {\n\t\t\tconst healTimeout = 0 // Zero timeout means no timeout\n\t\t\ttestCID := cid.MustParse(\"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\n\t\t\t// Setup channels\n\t\t\tkeys := make(chan cid.Cid, 1)\n\t\t\tkeys <- testCID\n\t\t\tclose(keys)\n\t\t\tresults := make(chan *verifyResult, 1)\n\n\t\t\t// Mock blockstore that returns error (simulating corruption)\n\t\t\tmockBS := &mockBlockstore{\n\t\t\t\tgetError: errors.New(\"corrupt block\"),\n\t\t\t}\n\n\t\t\t// Mock API that succeeds quickly\n\t\t\tmockAPI := &mockCoreAPI{\n\t\t\t\tblockAPI: &mockBlockAPI{\n\t\t\t\t\tgetDelay: 100 * time.Millisecond,\n\t\t\t\t\tdata: []byte(\"healed data\"),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tvar wg sync.WaitGroup\n\t\t\twg.Add(1)\n\n\t\t\t// Run worker\n\t\t\tgo verifyWorkerRun(t.Context(), &wg, keys, results, mockBS, mockAPI, true, true, healTimeout)\n\n\t\t\t// Advance time to let heal complete\n\t\t\ttime.Sleep(200 * time.Millisecond)\n\t\t\tsynctest.Wait()\n\n\t\t\twg.Wait()\n\t\t\tclose(results)\n\n\t\t\t// Verify heal succeeded even with zero timeout\n\t\t\tresult := <-results\n\t\t\trequire.NotNil(t, result)\n\t\t\tassert.Equal(t, verifyStateCorruptHealed, result.state)\n\t\t\tassert.Contains(t, result.msg, \"healed\")\n\t\t})\n\t})\n\n\tt.Run(\"multiple blocks with different timeout outcomes\", func(t *testing.T) {\n\t\tsynctest.Test(t, func(t *testing.T) {\n\t\t\tconst healTimeout = 3 * time.Second\n\t\t\ttestCID1 := cid.MustParse(\"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\t\ttestCID2 := cid.MustParse(\"bafybeihvvulpp4evxj7x7armbqcyg6uezzuig6jp3lktpbovlqfkjtgyby\")\n\n\t\t\t// Setup channels\n\t\t\tkeys := make(chan cid.Cid, 2)\n\t\t\tkeys <- testCID1\n\t\t\tkeys <- testCID2\n\t\t\tclose(keys)\n\t\t\tresults := make(chan *verifyResult, 2)\n\n\t\t\t// Mock blockstore that always returns error (all blocks corrupt)\n\t\t\tmockBS := &mockBlockstore{\n\t\t\t\tgetError: errors.New(\"corrupt block\"),\n\t\t\t}\n\n\t\t\t// Create two mock block APIs with different delays\n\t\t\t// We'll need to alternate which one gets used\n\t\t\t// For simplicity, use one that succeeds fast\n\t\t\tmockAPI := &mockCoreAPI{\n\t\t\t\tblockAPI: &mockBlockAPI{\n\t\t\t\t\tgetDelay: 1 * time.Second, // Less than healTimeout - will succeed\n\t\t\t\t\tdata: []byte(\"healed data\"),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tvar wg sync.WaitGroup\n\t\t\twg.Add(2) // Two workers\n\n\t\t\t// Run two workers\n\t\t\tgo verifyWorkerRun(t.Context(), &wg, keys, results, mockBS, mockAPI, true, true, healTimeout)\n\t\t\tgo verifyWorkerRun(t.Context(), &wg, keys, results, mockBS, mockAPI, true, true, healTimeout)\n\n\t\t\t// Advance time to let both complete\n\t\t\ttime.Sleep(2 * time.Second)\n\t\t\tsynctest.Wait()\n\n\t\t\twg.Wait()\n\t\t\tclose(results)\n\n\t\t\t// Collect results\n\t\t\tvar healedCount int\n\t\t\tfor result := range results {\n\t\t\t\tif result.state == verifyStateCorruptHealed {\n\t\t\t\t\thealedCount++\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Both should heal successfully (both under timeout)\n\t\t\tassert.Equal(t, 2, healedCount)\n\t\t})\n\t})\n\n\tt.Run(\"valid block is not healed\", func(t *testing.T) {\n\t\tsynctest.Test(t, func(t *testing.T) {\n\t\t\tconst healTimeout = 5 * time.Second\n\t\t\ttestCID := cid.MustParse(\"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\n\t\t\t// Setup channels\n\t\t\tkeys := make(chan cid.Cid, 1)\n\t\t\tkeys <- testCID\n\t\t\tclose(keys)\n\t\t\tresults := make(chan *verifyResult, 1)\n\n\t\t\t// Mock blockstore that returns valid block (no error)\n\t\t\tmockBS := &mockBlockstore{\n\t\t\t\tblock: blocks.NewBlock([]byte(\"valid data\")),\n\t\t\t}\n\n\t\t\t// Mock API (won't be called since block is valid)\n\t\t\tmockAPI := &mockCoreAPI{\n\t\t\t\tblockAPI: &mockBlockAPI{},\n\t\t\t}\n\n\t\t\tvar wg sync.WaitGroup\n\t\t\twg.Add(1)\n\n\t\t\t// Run worker with heal enabled\n\t\t\tgo verifyWorkerRun(t.Context(), &wg, keys, results, mockBS, mockAPI, false, true, healTimeout)\n\n\t\t\tsynctest.Wait()\n\n\t\t\twg.Wait()\n\t\t\tclose(results)\n\n\t\t\t// Verify block is marked valid, not healed\n\t\t\tresult := <-results\n\t\t\trequire.NotNil(t, result)\n\t\t\tassert.Equal(t, verifyStateValid, result.state)\n\t\t\tassert.Empty(t, result.msg)\n\t\t})\n\t})\n}\n\n// mockBlockstore implements a minimal blockstore for testing\ntype mockBlockstore struct {\n\tgetError error\n\tblock blocks.Block\n}\n\nfunc (m *mockBlockstore) Get(ctx context.Context, c cid.Cid) (blocks.Block, error) {\n\tif m.getError != nil {\n\t\treturn nil, m.getError\n\t}\n\treturn m.block, nil\n}\n\nfunc (m *mockBlockstore) DeleteBlock(ctx context.Context, c cid.Cid) error {\n\treturn nil\n}\n\nfunc (m *mockBlockstore) Has(ctx context.Context, c cid.Cid) (bool, error) {\n\treturn m.block != nil, nil\n}\n\nfunc (m *mockBlockstore) GetSize(ctx context.Context, c cid.Cid) (int, error) {\n\tif m.block != nil {\n\t\treturn len(m.block.RawData()), nil\n\t}\n\treturn 0, errors.New(\"block not found\")\n}\n\nfunc (m *mockBlockstore) Put(ctx context.Context, b blocks.Block) error {\n\treturn nil\n}\n\nfunc (m *mockBlockstore) PutMany(ctx context.Context, bs []blocks.Block) error {\n\treturn nil\n}\n\nfunc (m *mockBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error) {\n\treturn nil, errors.New(\"not implemented\")\n}\n\nfunc (m *mockBlockstore) HashOnRead(enabled bool) {\n}\n\n// mockBlockAPI implements BlockAPI for testing\ntype mockBlockAPI struct {\n\tgetDelay time.Duration\n\tgetError error\n\tdata []byte\n}\n\nfunc (m *mockBlockAPI) Get(ctx context.Context, p path.Path) (io.Reader, error) {\n\tif m.getDelay > 0 {\n\t\tselect {\n\t\tcase <-time.After(m.getDelay):\n\t\t\t// Delay completed\n\t\tcase <-ctx.Done():\n\t\t\treturn nil, ctx.Err()\n\t\t}\n\t}\n\tif m.getError != nil {\n\t\treturn nil, m.getError\n\t}\n\treturn bytes.NewReader(m.data), nil\n}\n\nfunc (m *mockBlockAPI) Put(ctx context.Context, r io.Reader, opts ...options.BlockPutOption) (coreiface.BlockStat, error) {\n\treturn nil, errors.New(\"not implemented\")\n}\n\nfunc (m *mockBlockAPI) Rm(ctx context.Context, p path.Path, opts ...options.BlockRmOption) error {\n\treturn errors.New(\"not implemented\")\n}\n\nfunc (m *mockBlockAPI) Stat(ctx context.Context, p path.Path) (coreiface.BlockStat, error) {\n\treturn nil, errors.New(\"not implemented\")\n}\n\n// mockCoreAPI implements minimal CoreAPI for testing\ntype mockCoreAPI struct {\n\tblockAPI *mockBlockAPI\n}\n\nfunc (m *mockCoreAPI) Block() coreiface.BlockAPI {\n\treturn m.blockAPI\n}\n\nfunc (m *mockCoreAPI) Unixfs() coreiface.UnixfsAPI { return nil }\nfunc (m *mockCoreAPI) Dag() coreiface.APIDagService { return nil }\nfunc (m *mockCoreAPI) Name() coreiface.NameAPI { return nil }\nfunc (m *mockCoreAPI) Key() coreiface.KeyAPI { return nil }\nfunc (m *mockCoreAPI) Pin() coreiface.PinAPI { return nil }\nfunc (m *mockCoreAPI) Object() coreiface.ObjectAPI { return nil }\nfunc (m *mockCoreAPI) Swarm() coreiface.SwarmAPI { return nil }\nfunc (m *mockCoreAPI) PubSub() coreiface.PubSubAPI { return nil }\nfunc (m *mockCoreAPI) Routing() coreiface.RoutingAPI { return nil }\n\nfunc (m *mockCoreAPI) ResolvePath(ctx context.Context, p path.Path) (path.ImmutablePath, []string, error) {\n\treturn path.ImmutablePath{}, nil, errors.New(\"not implemented\")\n}\n\nfunc (m *mockCoreAPI) ResolveNode(ctx context.Context, p path.Path) (ipld.Node, error) {\n\treturn nil, errors.New(\"not implemented\")\n}\n\nfunc (m *mockCoreAPI) WithOptions(...options.ApiOption) (coreiface.CoreAPI, error) {\n\treturn nil, errors.New(\"not implemented\")\n}\n" }, { "path": "core/commands/resolve.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\tns \"github.com/ipfs/boxo/namesys\"\n\tcidenc \"github.com/ipfs/go-cidutil/cidenc\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\tncmd \"github.com/ipfs/kubo/core/commands/name\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nconst (\n\tresolveRecursiveOptionName = \"recursive\"\n\tresolveDhtRecordCountOptionName = \"dht-record-count\"\n\tresolveDhtTimeoutOptionName = \"dht-timeout\"\n)\n\nvar ResolveCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Resolve the value of names to IPFS.\",\n\t\tShortDescription: `\nThere are a number of mutable name protocols that can link among\nthemselves and into IPNS. This command accepts any of these\nidentifiers and resolves them to the referenced item.\n`,\n\t\tLongDescription: `\nThere are a number of mutable name protocols that can link among\nthemselves and into IPNS. For example IPNS references can (currently)\npoint at an IPFS object, and DNS links can point at other DNS links, IPNS\nentries, or IPFS objects. This command accepts any of these\nidentifiers and resolves them to the referenced item.\n\nEXAMPLES\n\nResolve the value of your identity:\n\n $ ipfs resolve /ipns/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n /ipfs/Qmcqtw8FfrVSBaRmbWwHxt3AuySBhJLcvmFYi3Lbc4xnwj\n\nResolve the value of another name:\n\n $ ipfs resolve /ipns/QmbCMUZw6JFeZ7Wp9jkzbye3Fzp2GGcPgC3nmeUjfVF87n\n /ipns/QmatmE9msSfkKxoffpHwNLNKgwZG8eT9Bud6YoPab52vpy\n\nResolve the value of another name recursively:\n\n $ ipfs resolve -r /ipns/QmbCMUZw6JFeZ7Wp9jkzbye3Fzp2GGcPgC3nmeUjfVF87n\n /ipfs/Qmcqtw8FfrVSBaRmbWwHxt3AuySBhJLcvmFYi3Lbc4xnwj\n\nResolve the value of an IPFS DAG path:\n\n $ ipfs resolve /ipfs/QmeZy1fGbwgVSrqbfh9fKQrAWgeyRnj7h8fsHS1oy3k99x/beep/boop\n /ipfs/QmYRMjyvAiHKN9UTi8Bzt1HUspmSRD8T8DwxfSMzLgBon1\n\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"name\", true, false, \"The name to resolve.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(resolveRecursiveOptionName, \"r\", \"Resolve until the result is an IPFS name.\").WithDefault(true),\n\t\tcmds.IntOption(resolveDhtRecordCountOptionName, \"dhtrc\", \"Number of records to request for DHT resolution.\"),\n\t\tcmds.StringOption(resolveDhtTimeoutOptionName, \"dhtt\", \"Max time to collect values during DHT resolution e.g. \\\"30s\\\". Pass 0 for no timeout.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tname := req.Arguments[0]\n\t\trecursive, _ := req.Options[resolveRecursiveOptionName].(bool)\n\n\t\t// the case when ipns is resolved step by step\n\t\tif strings.HasPrefix(name, \"/ipns/\") && !recursive {\n\t\t\trc, rcok := req.Options[resolveDhtRecordCountOptionName].(uint)\n\t\t\tdhtt, dhttok := req.Options[resolveDhtTimeoutOptionName].(string)\n\t\t\tropts := []options.NameResolveOption{\n\t\t\t\toptions.Name.ResolveOption(ns.ResolveWithDepth(1)),\n\t\t\t}\n\n\t\t\tif rcok {\n\t\t\t\tropts = append(ropts, options.Name.ResolveOption(ns.ResolveWithDhtRecordCount(rc)))\n\t\t\t}\n\t\t\tif dhttok {\n\t\t\t\td, err := time.ParseDuration(dhtt)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tif d < 0 {\n\t\t\t\t\treturn errors.New(\"DHT timeout value must be >= 0\")\n\t\t\t\t}\n\t\t\t\tropts = append(ropts, options.Name.ResolveOption(ns.ResolveWithDhtTimeout(d)))\n\t\t\t}\n\t\t\tp, err := api.Name().Resolve(req.Context, name, ropts...)\n\t\t\t// ErrResolveRecursion is fine\n\t\t\tif err != nil && err != ns.ErrResolveRecursion {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn cmds.EmitOnce(res, &ncmd.ResolvedPath{Path: p.String()})\n\t\t}\n\n\t\tvar enc cidenc.Encoder\n\t\tswitch {\n\t\tcase !cmdenv.CidBaseDefined(req) && !strings.HasPrefix(name, \"/ipns/\"):\n\t\t\t// Not specified, check the path.\n\t\t\tenc, err = cmdenv.CidEncoderFromPath(name)\n\t\t\tif err == nil {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\t// Nope, fallback on the default.\n\t\t\tfallthrough\n\t\tdefault:\n\t\t\tenc, err = cmdenv.GetCidEncoder(req)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tp, err := cmdutils.PathOrCidPath(name)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// else, ipfs path or ipns with recursive flag\n\t\trp, remainder, err := api.ResolvePath(req.Context, p)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Trick to encode path with correct encoding.\n\t\tencodedPath := \"/\" + rp.Namespace() + \"/\" + enc.Encode(rp.RootCid())\n\t\tif len(remainder) != 0 {\n\t\t\tencodedPath += path.SegmentsToString(remainder...)\n\t\t}\n\n\t\t// Ensure valid and sanitized.\n\t\tep, err := path.NewPath(encodedPath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &ncmd.ResolvedPath{Path: ep.String()})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, rp *ncmd.ResolvedPath) error {\n\t\t\tfmt.Fprintln(w, rp.Path)\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: ncmd.ResolvedPath{},\n}\n" }, { "path": "core/commands/root.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tdag \"github.com/ipfs/kubo/core/commands/dag\"\n\tname \"github.com/ipfs/kubo/core/commands/name\"\n\tocmd \"github.com/ipfs/kubo/core/commands/object\"\n\t\"github.com/ipfs/kubo/core/commands/pin\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n)\n\nvar log = logging.Logger(\"core/commands\")\n\nvar (\n\tErrNotOnline = errors.New(\"this command must be run in online mode. Try running 'ipfs daemon' first\")\n\tErrSelfUnsupported = errors.New(\"finding your own node in the DHT is currently not supported\")\n)\n\nconst (\n\tRepoDirOption = \"repo-dir\"\n\tConfigFileOption = \"config-file\"\n\tConfigOption = \"config\"\n\tDebugOption = \"debug\"\n\tLocalOption = \"local\" // DEPRECATED: use OfflineOption\n\tOfflineOption = \"offline\"\n\tApiOption = \"api\" //nolint\n\tApiAuthOption = \"api-auth\" //nolint\n)\n\nvar Root = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Global p2p merkle-dag filesystem.\",\n\t\tSynopsis: \"ipfs [--config= | -c] [--debug | -D] [--help] [-h] [--api=] [--offline] [--cid-base=] [--upgrade-cidv0-in-output] [--encoding= | --enc] [--timeout=] ...\",\n\t\tSubcommands: `\nBASIC COMMANDS\n init Initialize local IPFS configuration\n add Add a file to IPFS\n cat Show IPFS object data\n get Download IPFS objects\n ls List links from an object\n refs List hashes of links from an object\n\nDATA STRUCTURE COMMANDS\n dag Interact with IPLD DAG nodes\n files Interact with files as if they were a unix filesystem\n block Interact with raw blocks in the datastore\n\nTEXT ENCODING COMMANDS\n cid Convert and discover properties of CIDs\n multibase Encode and decode data with Multibase format\n\nADVANCED COMMANDS\n daemon Start a long-running daemon process\n shutdown Shut down the daemon process\n resolve Resolve any type of content path\n name Publish and resolve IPNS names\n key Create and list IPNS name keypairs\n pin Pin objects to local storage\n repo Manipulate the IPFS repository\n stats Various operational stats\n p2p Libp2p stream mounting (experimental)\n filestore Manage the filestore (experimental)\n mount Mount an IPFS read-only mount point (experimental)\n provide Control providing operations\n\nNETWORK COMMANDS\n id Show info about IPFS peers\n bootstrap Add or remove bootstrap peers\n swarm Manage connections to the p2p network\n dht Query the DHT for values or peers\n routing Issue routing commands\n ping Measure the latency of a connection\n bitswap Inspect bitswap state\n pubsub Send and receive messages via pubsub\n\nTOOL COMMANDS\n config Manage configuration\n version Show IPFS version information\n diag Generate diagnostic reports\n update Download and apply go-ipfs updates\n commands List all available commands\n log Manage and show logs of running daemon\n\nUse 'ipfs --help' to learn more about each command.\n\nipfs uses a repository in the local file system. By default, the repo is\nlocated at ~/.ipfs. To change the repo location, set the $IPFS_PATH\nenvironment variable:\n\n export IPFS_PATH=/path/to/ipfsrepo\n\nEXIT STATUS\n\nThe CLI will exit with one of the following values:\n\n0 Successful execution.\n1 Failed executions.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(RepoDirOption, \"Path to the repository directory to use.\"),\n\t\tcmds.StringOption(ConfigFileOption, \"Path to the configuration file to use.\"),\n\t\tcmds.StringOption(ConfigOption, \"c\", \"[DEPRECATED] Path to the configuration file to use.\"),\n\t\tcmds.BoolOption(DebugOption, \"D\", \"Operate in debug mode.\"),\n\t\tcmds.BoolOption(cmds.OptLongHelp, \"Show the full command help text.\"),\n\t\tcmds.BoolOption(cmds.OptShortHelp, \"Show a short version of the command help text.\"),\n\t\tcmds.BoolOption(LocalOption, \"L\", \"Run the command locally, instead of using the daemon. DEPRECATED: use --offline.\"),\n\t\tcmds.BoolOption(OfflineOption, \"Run the command offline.\"),\n\t\tcmds.StringOption(ApiOption, \"Use a specific API instance (defaults to /ip4/127.0.0.1/tcp/5001)\"),\n\t\tcmds.StringOption(ApiAuthOption, \"Optional RPC API authorization secret (defined as AuthSecret in API.Authorizations config)\"),\n\n\t\t// global options, added to every command\n\t\tcmdenv.OptionCidBase,\n\t\tcmdenv.OptionUpgradeCidV0InOutput,\n\n\t\tcmds.OptionEncodingType,\n\t\tcmds.OptionStreamChannels,\n\t\tcmds.OptionTimeout,\n\t},\n}\n\nvar CommandsDaemonCmd = CommandsCmd(Root)\n\nvar rootSubcommands = map[string]*cmds.Command{\n\t\"add\": AddCmd,\n\t\"bitswap\": BitswapCmd,\n\t\"block\": BlockCmd,\n\t\"cat\": CatCmd,\n\t\"commands\": CommandsDaemonCmd,\n\t\"files\": FilesCmd,\n\t\"filestore\": FileStoreCmd,\n\t\"get\": GetCmd,\n\t\"provide\": ProvideCmd,\n\t\"pubsub\": PubsubCmd,\n\t\"repo\": RepoCmd,\n\t\"stats\": StatsCmd,\n\t\"bootstrap\": BootstrapCmd,\n\t\"config\": ConfigCmd,\n\t\"dag\": dag.DagCmd,\n\t\"dht\": DhtCmd,\n\t\"routing\": RoutingCmd,\n\t\"diag\": DiagCmd,\n\t\"id\": IDCmd,\n\t\"key\": KeyCmd,\n\t\"log\": LogCmd,\n\t\"ls\": LsCmd,\n\t\"mount\": MountCmd,\n\t\"name\": name.NameCmd,\n\t\"object\": ocmd.ObjectCmd,\n\t\"pin\": pin.PinCmd,\n\t\"ping\": PingCmd,\n\t\"p2p\": P2PCmd,\n\t\"refs\": RefsCmd,\n\t\"resolve\": ResolveCmd,\n\t\"swarm\": SwarmCmd,\n\t\"update\": ExternalBinary(\"Please see https://github.com/ipfs/ipfs-update/blob/master/README.md#install for installation instructions.\"),\n\t\"version\": VersionCmd,\n\t\"shutdown\": daemonShutdownCmd,\n\t\"cid\": CidCmd,\n\t\"multibase\": MbaseCmd,\n}\n\nfunc init() {\n\tRoot.ProcessHelp()\n\tRoot.Subcommands = rootSubcommands\n}\n\ntype MessageOutput struct {\n\tMessage string\n}\n" }, { "path": "core/commands/root_test.go", "content": "package commands\n\nimport (\n\t\"testing\"\n)\n\nfunc TestCommandTree(t *testing.T) {\n\tprintErrors := func(errs map[string][]error) {\n\t\tif errs == nil {\n\t\t\treturn\n\t\t}\n\t\tt.Error(\"In Root command tree:\")\n\t\tfor cmd, err := range errs {\n\t\t\tt.Errorf(\" In X command %s:\", cmd)\n\t\t\tfor _, e := range err {\n\t\t\t\tt.Errorf(\" %s\", e)\n\t\t\t}\n\t\t}\n\t}\n\tprintErrors(Root.DebugValidate())\n}\n" }, { "path": "core/commands/routing.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\t\"github.com/ipfs/kubo/core/node\"\n\tmh \"github.com/multiformats/go-multihash\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/provider\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n)\n\nvar errAllowOffline = errors.New(\"can't put while offline: pass `--allow-offline` to override\")\n\nconst (\n\tdhtVerboseOptionName = \"verbose\"\n\tnumProvidersOptionName = \"num-providers\"\n\tallowOfflineOptionName = \"allow-offline\"\n)\n\nvar RoutingCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Issue routing commands.\",\n\t\tShortDescription: ``,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"findprovs\": findProvidersRoutingCmd,\n\t\t\"findpeer\": findPeerRoutingCmd,\n\t\t\"get\": getValueRoutingCmd,\n\t\t\"put\": putValueRoutingCmd,\n\t\t\"provide\": provideRefRoutingCmd,\n\t\t\"reprovide\": reprovideRoutingCmd,\n\t},\n}\n\nvar findProvidersRoutingCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Find peers that can provide a specific value, given a key.\",\n\t\tShortDescription: \"Outputs a list of newline-delimited provider Peer IDs.\",\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, true, \"The key to find providers for.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(dhtVerboseOptionName, \"v\", \"Print extra information.\"),\n\t\tcmds.IntOption(numProvidersOptionName, \"n\", \"The number of providers to find.\").WithDefault(20),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !n.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tnumProviders, _ := req.Options[numProvidersOptionName].(int)\n\t\tif numProviders < 1 {\n\t\t\treturn errors.New(\"number of providers must be greater than 0\")\n\t\t}\n\n\t\tc, err := cid.Parse(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tctx, events := routing.RegisterForQueryEvents(ctx)\n\n\t\tgo func() {\n\t\t\tdefer cancel()\n\t\t\tpchan := n.Routing.FindProvidersAsync(ctx, c, numProviders)\n\t\t\tfor p := range pchan {\n\t\t\t\tnp := cmdutils.CloneAddrInfo(p)\n\t\t\t\trouting.PublishQueryEvent(ctx, &routing.QueryEvent{\n\t\t\t\t\tType: routing.Provider,\n\t\t\t\t\tResponses: []*peer.AddrInfo{&np},\n\t\t\t\t})\n\t\t\t}\n\t\t}()\n\t\tfor e := range events {\n\t\t\tif err := res.Emit(e); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *routing.QueryEvent) error {\n\t\t\tpfm := pfuncMap{\n\t\t\t\trouting.FinalPeer: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tif verbose {\n\t\t\t\t\t\tfmt.Fprintf(out, \"* closest peer %s\\n\", obj.ID)\n\t\t\t\t\t}\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t\trouting.Provider: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tprov := obj.Responses[0]\n\t\t\t\t\tif verbose {\n\t\t\t\t\t\tfmt.Fprintf(out, \"provider: \")\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(out, \"%s\\n\", prov.ID)\n\t\t\t\t\tif verbose {\n\t\t\t\t\t\tfor _, a := range prov.Addrs {\n\t\t\t\t\t\t\tfmt.Fprintf(out, \"\\t%s\\n\", a)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tverbose, _ := req.Options[dhtVerboseOptionName].(bool)\n\t\t\treturn printEvent(out, w, verbose, pfm)\n\t\t}),\n\t},\n\tType: routing.QueryEvent{},\n}\n\nconst (\n\trecursiveOptionName = \"recursive\"\n)\n\nvar provideRefRoutingCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Announce to the network that you are providing given values.\",\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, true, \"The key[s] to send provide records for.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(dhtVerboseOptionName, \"v\", \"Print extra information.\"),\n\t\tcmds.BoolOption(recursiveOptionName, \"r\", \"Recursively provide entire graph.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\t\t// respect global config\n\t\tcfg, err := nd.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !cfg.Provide.Enabled.WithDefault(config.DefaultProvideEnabled) {\n\t\t\treturn errors.New(\"invalid configuration: Provide.Enabled is set to 'false'\")\n\t\t}\n\n\t\tif len(nd.PeerHost.Network().Conns()) == 0 && !cfg.HasHTTPProviderConfigured() {\n\t\t\t// Node is depending on DHT for providing (no custom HTTP provider\n\t\t\t// configured) and currently has no connected peers.\n\t\t\treturn errors.New(\"cannot provide, no connected peers\")\n\t\t}\n\n\t\t// If we reach here with no connections but HTTP provider configured,\n\t\t// we proceed with the provide operation via HTTP\n\n\t\t// Needed to parse stdin args.\n\t\t// TODO: Lazy Load\n\t\terr = req.ParseBodyArgs()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trec, _ := req.Options[recursiveOptionName].(bool)\n\n\t\tvar cids []cid.Cid\n\t\tfor _, arg := range req.Arguments {\n\t\t\tc, err := cid.Decode(arg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\thas, err := nd.Blockstore.Has(req.Context, c)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif !has {\n\t\t\t\treturn fmt.Errorf(\"block %s not found locally, cannot provide\", c)\n\t\t\t}\n\n\t\t\tcids = append(cids, c)\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tctx, events := routing.RegisterForQueryEvents(ctx)\n\n\t\tvar provideErr error\n\t\t// TODO: not sure if necessary to call StartProviding for `ipfs routing\n\t\t// provide `, since either cid is already being provided, or it will\n\t\t// be garbage collected and not reprovided anyway. So we may simply stick\n\t\t// with a single (optimistic) provide, and skip StartProviding call.\n\t\tgo func() {\n\t\t\tdefer cancel()\n\t\t\tif rec {\n\t\t\t\tprovideErr = provideCidsRec(ctx, nd.Provider, nd.DAG, cids)\n\t\t\t} else {\n\t\t\t\tprovideErr = provideCids(nd.Provider, cids)\n\t\t\t}\n\t\t\tif provideErr != nil {\n\t\t\t\trouting.PublishQueryEvent(ctx, &routing.QueryEvent{\n\t\t\t\t\tType: routing.QueryError,\n\t\t\t\t\tExtra: provideErr.Error(),\n\t\t\t\t})\n\t\t\t}\n\t\t}()\n\n\t\tif nd.HasActiveDHTClient() {\n\t\t\t// If node has a DHT client, provide immediately the supplied cids before\n\t\t\t// returning.\n\t\t\tfor _, c := range cids {\n\t\t\t\tif err = provideCIDSync(req.Context, nd.DHTClient, c); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"error providing cid: %w\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tfor e := range events {\n\t\t\tif err := res.Emit(e); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn provideErr\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *routing.QueryEvent) error {\n\t\t\tpfm := pfuncMap{\n\t\t\t\trouting.FinalPeer: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tif verbose {\n\t\t\t\t\t\tfmt.Fprintf(out, \"sending provider record to peer %s\\n\", obj.ID)\n\t\t\t\t\t}\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tverbose, _ := req.Options[dhtVerboseOptionName].(bool)\n\t\t\treturn printEvent(out, w, verbose, pfm)\n\t\t}),\n\t},\n\tType: routing.QueryEvent{},\n}\n\nvar reprovideRoutingCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Trigger reprovider.\",\n\t\tShortDescription: `\nTrigger reprovider to announce our data to network.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\t// respect global config\n\t\tcfg, err := nd.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !cfg.Provide.Enabled.WithDefault(config.DefaultProvideEnabled) {\n\t\t\treturn errors.New(\"invalid configuration: Provide.Enabled is set to 'false'\")\n\t\t}\n\t\tif cfg.Provide.DHT.Interval.WithDefault(config.DefaultProvideDHTInterval) == 0 {\n\t\t\treturn errors.New(\"invalid configuration: Provide.DHT.Interval is set to '0'\")\n\t\t}\n\t\tprovideSys, ok := nd.Provider.(provider.Reprovider)\n\t\tif !ok {\n\t\t\treturn errors.New(\"manual reprovide only available with legacy provider (Provide.DHT.SweepEnabled=false)\")\n\t\t}\n\n\t\terr = provideSys.Reprovide(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn nil\n\t},\n}\n\nfunc provideCids(prov node.DHTProvider, cids []cid.Cid) error {\n\tmhs := make([]mh.Multihash, len(cids))\n\tfor i, c := range cids {\n\t\tmhs[i] = c.Hash()\n\t}\n\t// providing happens asynchronously\n\treturn prov.StartProviding(true, mhs...)\n}\n\nfunc provideCidsRec(ctx context.Context, prov node.DHTProvider, dserv ipld.DAGService, cids []cid.Cid) error {\n\tfor _, c := range cids {\n\t\tkset := cid.NewSet()\n\t\terr := dag.Walk(ctx, dag.GetLinksDirect(dserv), c, kset.Visit)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif err = provideCids(prov, kset.Keys()); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nvar findPeerRoutingCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Find the multiaddresses associated with a Peer ID.\",\n\t\tShortDescription: \"Outputs a list of newline-delimited multiaddresses.\",\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"peerID\", true, true, \"The ID of the peer to search for.\"),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(dhtVerboseOptionName, \"v\", \"Print extra information.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tpid, err := peer.Decode(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif pid == nd.Identity {\n\t\t\treturn ErrSelfUnsupported\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(req.Context)\n\t\tctx, events := routing.RegisterForQueryEvents(ctx)\n\n\t\tvar findPeerErr error\n\t\tgo func() {\n\t\t\tdefer cancel()\n\t\t\tvar pi peer.AddrInfo\n\t\t\tpi, findPeerErr = nd.Routing.FindPeer(ctx, pid)\n\t\t\tif findPeerErr != nil {\n\t\t\t\trouting.PublishQueryEvent(ctx, &routing.QueryEvent{\n\t\t\t\t\tType: routing.QueryError,\n\t\t\t\t\tExtra: findPeerErr.Error(),\n\t\t\t\t})\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trouting.PublishQueryEvent(ctx, &routing.QueryEvent{\n\t\t\t\tType: routing.FinalPeer,\n\t\t\t\tResponses: []*peer.AddrInfo{&pi},\n\t\t\t})\n\t\t}()\n\n\t\tfor e := range events {\n\t\t\tif err := res.Emit(e); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn findPeerErr\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *routing.QueryEvent) error {\n\t\t\tpfm := pfuncMap{\n\t\t\t\trouting.FinalPeer: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tpi := obj.Responses[0]\n\t\t\t\t\tfor _, a := range pi.Addrs {\n\t\t\t\t\t\tfmt.Fprintf(out, \"%s\\n\", a)\n\t\t\t\t\t}\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tverbose, _ := req.Options[dhtVerboseOptionName].(bool)\n\t\t\treturn printEvent(out, w, verbose, pfm)\n\t\t}),\n\t},\n\tType: routing.QueryEvent{},\n}\n\nvar getValueRoutingCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Given a key, query the routing system for its best value.\",\n\t\tShortDescription: `\nOutputs the best value for the given key.\n\nThere may be several different values for a given key stored in the routing\nsystem; in this context 'best' means the record that is most desirable. There is\nno one metric for 'best': it depends entirely on the key type. For IPNS, 'best'\nis the record that is both valid and has the highest sequence number (freshest).\nDifferent key types can specify other 'best' rules.\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, true, \"The key to find a value for.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tr, err := api.Routing().Get(req.Context, req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn res.Emit(routing.QueryEvent{\n\t\t\tExtra: base64.StdEncoding.EncodeToString(r),\n\t\t\tType: routing.Value,\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, obj *routing.QueryEvent) error {\n\t\t\tres, err := base64.StdEncoding.DecodeString(obj.Extra)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, err = w.Write(res)\n\t\t\treturn err\n\t\t}),\n\t},\n\tType: routing.QueryEvent{},\n}\n\nvar putValueRoutingCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Write a key/value pair to the routing system.\",\n\t\tShortDescription: `\nGiven a key of the form /foo/bar and a valid value for that key, this will write\nthat value to the routing system with that key.\n\nKeys have two parts: a keytype (foo) and the key name (bar). IPNS uses the\n/ipns keytype, and expects the key name to be a Peer ID. IPNS entries are\nspecifically formatted (protocol buffer).\n\nYou may only use keytypes that are supported in your ipfs binary: currently\nthis is only /ipns. Unless you have a relatively deep understanding of the\ngo-ipfs routing internals, you likely want to be using 'ipfs name publish' instead\nof this.\n\nThe value must be a valid value for the given key type. For example, if the key\nis /ipns/QmFoo, the value must be IPNS record (protobuf) signed with the key\nidentified by QmFoo.\n`,\n\t},\n\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"key\", true, false, \"The key to store the value at.\"),\n\t\tcmds.FileArg(\"value-file\", true, false, \"A path to a file containing the value to store.\").EnableStdin(),\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(allowOfflineOptionName, \"When offline, save the IPNS record to the local datastore without broadcasting to the network instead of simply failing.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfile, err := cmdenv.GetFileArg(req.Files.Entries())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer file.Close()\n\n\t\tdata, err := io.ReadAll(file)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tallowOffline, _ := req.Options[allowOfflineOptionName].(bool)\n\n\t\topts := []options.RoutingPutOption{\n\t\t\toptions.Put.AllowOffline(allowOffline),\n\t\t}\n\n\t\tipnsName, err := ipns.NameFromString(req.Arguments[0])\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\terr = api.Routing().Put(req.Context, req.Arguments[0], data, opts...)\n\t\tif err != nil {\n\t\t\tif err == iface.ErrOffline {\n\t\t\t\terr = errAllowOffline\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\n\t\treturn res.Emit(routing.QueryEvent{\n\t\t\tType: routing.Value,\n\t\t\tID: ipnsName.Peer(),\n\t\t})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *routing.QueryEvent) error {\n\t\t\tpfm := pfuncMap{\n\t\t\t\trouting.FinalPeer: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tif verbose {\n\t\t\t\t\t\tfmt.Fprintf(out, \"* closest peer %s\\n\", obj.ID)\n\t\t\t\t\t}\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t\trouting.Value: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {\n\t\t\t\t\tfmt.Fprintf(out, \"%s\\n\", obj.ID)\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tverbose, _ := req.Options[dhtVerboseOptionName].(bool)\n\n\t\t\treturn printEvent(out, w, verbose, pfm)\n\t\t}),\n\t},\n\tType: routing.QueryEvent{},\n}\n\ntype (\n\tprintFunc func(obj *routing.QueryEvent, out io.Writer, verbose bool) error\n\tpfuncMap map[routing.QueryEventType]printFunc\n)\n\nfunc printEvent(obj *routing.QueryEvent, out io.Writer, verbose bool, override pfuncMap) error {\n\tif verbose {\n\t\tfmt.Fprintf(out, \"%s: \", time.Now().Format(\"15:04:05.000\"))\n\t}\n\n\tif override != nil {\n\t\tif pf, ok := override[obj.Type]; ok {\n\t\t\treturn pf(obj, out, verbose)\n\t\t}\n\t}\n\n\tswitch obj.Type {\n\tcase routing.SendingQuery:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"* querying %s\\n\", obj.ID)\n\t\t}\n\tcase routing.Value:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"got value: '%s'\\n\", obj.Extra)\n\t\t} else {\n\t\t\tfmt.Fprint(out, obj.Extra)\n\t\t}\n\tcase routing.PeerResponse:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"* %s says use \", obj.ID)\n\t\t\tfor _, p := range obj.Responses {\n\t\t\t\tfmt.Fprintf(out, \"%s \", p.ID)\n\t\t\t}\n\t\t\tfmt.Fprintln(out)\n\t\t}\n\tcase routing.QueryError:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"error: %s\\n\", obj.Extra)\n\t\t}\n\tcase routing.DialingPeer:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"dialing peer: %s\\n\", obj.ID)\n\t\t}\n\tcase routing.AddingPeer:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"adding peer to query: %s\\n\", obj.ID)\n\t\t}\n\tcase routing.FinalPeer:\n\tdefault:\n\t\tif verbose {\n\t\t\tfmt.Fprintf(out, \"unrecognized event type: %d\\n\", obj.Type)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc escapeDhtKey(s string) (string, error) {\n\tparts := strings.Split(s, \"/\")\n\tif len(parts) != 3 ||\n\t\tparts[0] != \"\" ||\n\t\t!(parts[1] == \"ipns\" || parts[1] == \"pk\") {\n\t\treturn \"\", errors.New(\"invalid key\")\n\t}\n\n\tk, err := peer.Decode(parts[2])\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn strings.Join(append(parts[:2], string(k)), \"/\"), nil\n}\n" }, { "path": "core/commands/shutdown.go", "content": "package commands\n\nimport (\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n)\n\nvar daemonShutdownCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Shut down the IPFS daemon.\",\n\t},\n\tRun: func(req *cmds.Request, re cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsDaemon {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"daemon not running\")\n\t\t}\n\n\t\tif err := nd.Close(); err != nil {\n\t\t\tlog.Error(\"error while shutting down ipfs daemon:\", err)\n\t\t}\n\n\t\treturn nil\n\t},\n}\n" }, { "path": "core/commands/stat.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tmetrics \"github.com/libp2p/go-libp2p/core/metrics\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tprotocol \"github.com/libp2p/go-libp2p/core/protocol\"\n)\n\nvar StatsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Query IPFS statistics.\",\n\t\tShortDescription: `'ipfs stats' is a set of commands to help look at statistics\nfor your IPFS node.\n`,\n\t\tLongDescription: `'ipfs stats' is a set of commands to help look at statistics\nfor your IPFS node.`,\n\t},\n\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"bw\": statBwCmd,\n\t\t\"repo\": repoStatCmd,\n\t\t\"bitswap\": bitswapStatCmd,\n\t\t\"dht\": statDhtCmd,\n\t\t\"provide\": statProvideCmd,\n\t\t\"reprovide\": statReprovideCmd,\n\t},\n}\n\nconst (\n\tstatPeerOptionName = \"peer\"\n\tstatProtoOptionName = \"proto\"\n\tstatPollOptionName = \"poll\"\n\tstatIntervalOptionName = \"interval\"\n)\n\nvar statBwCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Print IPFS bandwidth information.\",\n\t\tShortDescription: `'ipfs stats bw' prints bandwidth information for the ipfs daemon.\nIt displays: TotalIn, TotalOut, RateIn, RateOut.\n\t\t`,\n\t\tLongDescription: `'ipfs stats bw' prints bandwidth information for the ipfs daemon.\nIt displays: TotalIn, TotalOut, RateIn, RateOut.\n\nBy default, overall bandwidth and all protocols are shown. To limit bandwidth\nto a particular peer, use the 'peer' option along with that peer's multihash\nid. To specify a specific protocol, use the 'proto' option. The 'peer' and\n'proto' options cannot be specified simultaneously. The protocols that are\nqueried using this method are outlined in the specification:\nhttps://github.com/libp2p/specs/blob/master/_archive/7-properties.md#757-protocol-multicodecs\n\nExample protocol options:\n - /ipfs/id/1.0.0\n - /ipfs/bitswap\n - /ipfs/dht\n\nExample:\n\n > ipfs stats bw -t /ipfs/bitswap\n Bandwidth\n TotalIn: 5.0MB\n TotalOut: 0B\n RateIn: 343B/s\n RateOut: 0B/s\n > ipfs stats bw -p QmepgFW7BHEtU4pZJdxaNiv75mKLLRQnPi1KaaXmQN4V1a\n Bandwidth\n TotalIn: 4.9MB\n TotalOut: 12MB\n RateIn: 0B/s\n RateOut: 0B/s\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.StringOption(statPeerOptionName, \"p\", \"Specify a peer to print bandwidth for.\"),\n\t\tcmds.StringOption(statProtoOptionName, \"t\", \"Specify a protocol to print bandwidth for.\"),\n\t\tcmds.BoolOption(statPollOptionName, \"Print bandwidth at an interval.\"),\n\t\tcmds.StringOption(statIntervalOptionName, \"i\", `Time interval to wait between updating output, if 'poll' is true.\n\n This accepts durations such as \"300s\", \"1.5h\" or \"2h45m\". Valid time units are:\n \"ns\", \"us\" (or \"µs\"), \"ms\", \"s\", \"m\", \"h\".`).WithDefault(\"1s\"),\n\t},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Must be online!\n\t\tif !nd.IsOnline {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"unable to run offline: %s\", ErrNotOnline)\n\t\t}\n\n\t\tif nd.Reporter == nil {\n\t\t\treturn errors.New(\"bandwidth reporter disabled in config\")\n\t\t}\n\n\t\tpstr, pfound := req.Options[statPeerOptionName].(string)\n\t\ttstr, tfound := req.Options[\"proto\"].(string)\n\t\tif pfound && tfound {\n\t\t\treturn cmds.Errorf(cmds.ErrClient, \"please only specify peer OR protocol\")\n\t\t}\n\n\t\tvar pid peer.ID\n\t\tif pfound {\n\t\t\tcheckpid, err := peer.Decode(pstr)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tpid = checkpid\n\t\t}\n\n\t\ttimeS, _ := req.Options[statIntervalOptionName].(string)\n\t\tinterval, err := time.ParseDuration(timeS)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdoPoll, _ := req.Options[statPollOptionName].(bool)\n\t\tfor {\n\t\t\tif pfound {\n\t\t\t\tstats := nd.Reporter.GetBandwidthForPeer(pid)\n\t\t\t\tif err := res.Emit(&stats); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t} else if tfound {\n\t\t\t\tprotoID := protocol.ID(tstr)\n\t\t\t\tstats := nd.Reporter.GetBandwidthForProtocol(protoID)\n\t\t\t\tif err := res.Emit(&stats); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\ttotals := nd.Reporter.GetBandwidthTotals()\n\t\t\t\tif err := res.Emit(&totals); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\tif !doPoll {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase <-time.After(interval):\n\t\t\tcase <-req.Context.Done():\n\t\t\t\treturn req.Context.Err()\n\t\t\t}\n\t\t}\n\t},\n\tType: metrics.Stats{},\n\tPostRun: cmds.PostRunMap{\n\t\tcmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {\n\t\t\tpolling, _ := res.Request().Options[statPollOptionName].(bool)\n\n\t\t\tif polling {\n\t\t\t\tfmt.Fprintln(os.Stdout, \"Total Up Total Down Rate Up Rate Down\")\n\t\t\t}\n\t\t\tfor {\n\t\t\t\tv, err := res.Next()\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err == io.EOF {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tbs := v.(*metrics.Stats)\n\n\t\t\t\tif !polling {\n\t\t\t\t\tprintStats(os.Stdout, bs)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\tfmt.Fprintf(os.Stdout, \"%8s \", humanize.Bytes(uint64(bs.TotalOut)))\n\t\t\t\tfmt.Fprintf(os.Stdout, \"%8s \", humanize.Bytes(uint64(bs.TotalIn)))\n\t\t\t\tfmt.Fprintf(os.Stdout, \"%8s/s \", humanize.Bytes(uint64(bs.RateOut)))\n\t\t\t\tfmt.Fprintf(os.Stdout, \"%8s/s \\r\", humanize.Bytes(uint64(bs.RateIn)))\n\t\t\t}\n\t\t},\n\t},\n}\n\nfunc printStats(out io.Writer, bs *metrics.Stats) {\n\tfmt.Fprintln(out, \"Bandwidth\")\n\tfmt.Fprintf(out, \"TotalIn: %s\\n\", humanize.Bytes(uint64(bs.TotalIn)))\n\tfmt.Fprintf(out, \"TotalOut: %s\\n\", humanize.Bytes(uint64(bs.TotalOut)))\n\tfmt.Fprintf(out, \"RateIn: %s/s\\n\", humanize.Bytes(uint64(bs.RateIn)))\n\tfmt.Fprintf(out, \"RateOut: %s/s\\n\", humanize.Bytes(uint64(bs.RateOut)))\n}\n" }, { "path": "core/commands/stat_dht.go", "content": "package commands\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tdht \"github.com/libp2p/go-libp2p-kad-dht\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\tkbucket \"github.com/libp2p/go-libp2p-kbucket\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n)\n\ntype dhtPeerInfo struct {\n\tID string\n\tConnected bool\n\tAgentVersion string\n\tLastUsefulAt string\n\tLastQueriedAt string\n}\n\ntype dhtStat struct {\n\tName string\n\tBuckets []dhtBucket\n}\n\ntype dhtBucket struct {\n\tLastRefresh string\n\tPeers []dhtPeerInfo\n}\n\nvar statDhtCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Returns statistics about the node's DHT(s).\",\n\t\tShortDescription: `\nReturns statistics about the DHT(s) the node is participating in.\n\nThis interface is not stable and may change from release to release.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"dht\", false, true, \"The DHT whose table should be listed (wanserver, lanserver, wan, lan). \"+\n\t\t\t\"wan and lan refer to client routing tables. When using the experimental DHT client only WAN is supported. Defaults to wan and lan.\"),\n\t},\n\tOptions: []cmds.Option{},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tif nd.DHT == nil {\n\t\t\treturn ErrNotDHT\n\t\t}\n\n\t\tid := kbucket.ConvertPeerID(nd.Identity)\n\n\t\tdhts := req.Arguments\n\t\tif len(dhts) == 0 {\n\t\t\tdhts = []string{\"wan\", \"lan\"}\n\t\t}\n\n\tdhttypeloop:\n\t\tfor _, name := range dhts {\n\t\t\tvar dht *dht.IpfsDHT\n\n\t\t\tvar separateClient bool\n\t\t\t// Check if using separate DHT client (e.g., accelerated DHT)\n\t\t\tif nd.HasActiveDHTClient() && nd.DHTClient != nd.DHT {\n\t\t\t\tseparateClient = true\n\t\t\t}\n\n\t\t\tswitch name {\n\t\t\tcase \"wan\":\n\t\t\t\tif separateClient {\n\t\t\t\t\tclient, ok := nd.DHTClient.(*fullrt.FullRT)\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\treturn cmds.Errorf(cmds.ErrClient, \"could not generate stats for the WAN DHT client type\")\n\t\t\t\t\t}\n\t\t\t\t\tpeerMap := client.Stat()\n\t\t\t\t\tbuckets := make([]dhtBucket, 1)\n\t\t\t\t\tb := &dhtBucket{}\n\t\t\t\t\tfor _, p := range peerMap {\n\t\t\t\t\t\tinfo := dhtPeerInfo{ID: p.String()}\n\n\t\t\t\t\t\tif ver, err := nd.Peerstore.Get(p, \"AgentVersion\"); err == nil {\n\t\t\t\t\t\t\tif vs, ok := ver.(string); ok {\n\t\t\t\t\t\t\t\tinfo.AgentVersion = cmdutils.CleanAndTrim(vs)\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t} else if err == pstore.ErrNotFound {\n\t\t\t\t\t\t\t// ignore\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t// this is a bug, usually.\n\t\t\t\t\t\t\tlog.Errorw(\n\t\t\t\t\t\t\t\t\"failed to get agent version from peerstore\",\n\t\t\t\t\t\t\t\t\"error\", err,\n\t\t\t\t\t\t\t)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tinfo.Connected = nd.PeerHost.Network().Connectedness(p) == network.Connected\n\t\t\t\t\t\tb.Peers = append(b.Peers, info)\n\t\t\t\t\t}\n\t\t\t\t\tbuckets[0] = *b\n\n\t\t\t\t\tif err := res.Emit(dhtStat{\n\t\t\t\t\t\tName: name,\n\t\t\t\t\t\tBuckets: buckets,\n\t\t\t\t\t}); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\tcontinue dhttypeloop\n\t\t\t\t}\n\t\t\t\tfallthrough\n\t\t\tcase \"wanserver\":\n\t\t\t\tdht = nd.DHT.WAN\n\t\t\tcase \"lan\":\n\t\t\t\tif separateClient {\n\t\t\t\t\treturn cmds.Errorf(cmds.ErrClient, \"no LAN client found\")\n\t\t\t\t}\n\t\t\t\tfallthrough\n\t\t\tcase \"lanserver\":\n\t\t\t\tdht = nd.DHT.LAN\n\t\t\tdefault:\n\t\t\t\treturn cmds.Errorf(cmds.ErrClient, \"unknown dht type: %s\", name)\n\t\t\t}\n\n\t\t\trt := dht.RoutingTable()\n\t\t\tlastRefresh := rt.GetTrackedCplsForRefresh()\n\t\t\tinfos := rt.GetPeerInfos()\n\t\t\tbuckets := make([]dhtBucket, 0, len(lastRefresh))\n\t\t\tfor _, pi := range infos {\n\t\t\t\tcpl := kbucket.CommonPrefixLen(id, kbucket.ConvertPeerID(pi.Id))\n\t\t\t\tif len(buckets) <= cpl {\n\t\t\t\t\tbuckets = append(buckets, make([]dhtBucket, 1+cpl-len(buckets))...)\n\t\t\t\t}\n\n\t\t\t\tinfo := dhtPeerInfo{ID: pi.Id.String()}\n\n\t\t\t\tif ver, err := nd.Peerstore.Get(pi.Id, \"AgentVersion\"); err == nil {\n\t\t\t\t\tif vs, ok := ver.(string); ok {\n\t\t\t\t\t\tinfo.AgentVersion = cmdutils.CleanAndTrim(vs)\n\t\t\t\t\t}\n\t\t\t\t} else if err == pstore.ErrNotFound {\n\t\t\t\t\t// ignore\n\t\t\t\t} else {\n\t\t\t\t\t// this is a bug, usually.\n\t\t\t\t\tlog.Errorw(\n\t\t\t\t\t\t\"failed to get agent version from peerstore\",\n\t\t\t\t\t\t\"error\", err,\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t\tif !pi.LastUsefulAt.IsZero() {\n\t\t\t\t\tinfo.LastUsefulAt = pi.LastUsefulAt.Format(time.RFC3339)\n\t\t\t\t}\n\n\t\t\t\tif !pi.LastSuccessfulOutboundQueryAt.IsZero() {\n\t\t\t\t\tinfo.LastQueriedAt = pi.LastSuccessfulOutboundQueryAt.Format(time.RFC3339)\n\t\t\t\t}\n\n\t\t\t\tinfo.Connected = nd.PeerHost.Network().Connectedness(pi.Id) == network.Connected\n\n\t\t\t\tbuckets[cpl].Peers = append(buckets[cpl].Peers, info)\n\t\t\t}\n\t\t\tfor i := 0; i < len(buckets) && i < len(lastRefresh); i++ {\n\t\t\t\trefreshTime := lastRefresh[i]\n\t\t\t\tif !refreshTime.IsZero() {\n\t\t\t\t\tbuckets[i].LastRefresh = refreshTime.Format(time.RFC3339)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif err := res.Emit(dhtStat{\n\t\t\t\tName: name,\n\t\t\t\tBuckets: buckets,\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out dhtStat) error {\n\t\t\ttw := tabwriter.NewWriter(w, 4, 4, 2, ' ', 0)\n\t\t\tdefer tw.Flush()\n\n\t\t\t// Formats a time into XX ago and remove any decimal\n\t\t\t// parts. That is, change \"2m3.00010101s\" to \"2m3s ago\".\n\t\t\tnow := time.Now()\n\t\t\tsince := func(t time.Time) string {\n\t\t\t\treturn now.Sub(t).Round(time.Second).String() + \" ago\"\n\t\t\t}\n\n\t\t\tcount := 0\n\t\t\tfor _, bucket := range out.Buckets {\n\t\t\t\tcount += len(bucket.Peers)\n\t\t\t}\n\n\t\t\tfmt.Fprintf(tw, \"DHT %s (%d peers):\\t\\t\\t\\n\", out.Name, count)\n\n\t\t\tfor i, bucket := range out.Buckets {\n\t\t\t\tlastRefresh := \"never\"\n\t\t\t\tif bucket.LastRefresh != \"\" {\n\t\t\t\t\tt, err := time.Parse(time.RFC3339, bucket.LastRefresh)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\tlastRefresh = since(t)\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(tw, \" Bucket %2d (%d peers) - refreshed %s:\\t\\t\\t\\n\", i, len(bucket.Peers), lastRefresh)\n\t\t\t\tfmt.Fprintln(tw, \" Peer\\tlast useful\\tlast queried\\tAgent Version\")\n\n\t\t\t\tfor _, p := range bucket.Peers {\n\t\t\t\t\tlastUseful := \"never\"\n\t\t\t\t\tif p.LastUsefulAt != \"\" {\n\t\t\t\t\t\tt, err := time.Parse(time.RFC3339, p.LastUsefulAt)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn err\n\t\t\t\t\t\t}\n\t\t\t\t\t\tlastUseful = since(t)\n\t\t\t\t\t}\n\n\t\t\t\t\tlastQueried := \"never\"\n\t\t\t\t\tif p.LastUsefulAt != \"\" {\n\t\t\t\t\t\tt, err := time.Parse(time.RFC3339, p.LastQueriedAt)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn err\n\t\t\t\t\t\t}\n\t\t\t\t\t\tlastQueried = since(t)\n\t\t\t\t\t}\n\n\t\t\t\t\tstate := \" \"\n\t\t\t\t\tif p.Connected {\n\t\t\t\t\t\tstate = \"@\"\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintf(tw, \" %s %s\\t%s\\t%s\\t%s\\n\", state, p.ID, lastUseful, lastQueried, p.AgentVersion)\n\t\t\t\t}\n\t\t\t\tfmt.Fprintln(tw, \"\\t\\t\\t\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: dhtStat{},\n}\n" }, { "path": "core/commands/stat_provide.go", "content": "package commands\n\nimport (\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nvar statProvideCmd = &cmds.Command{\n\tStatus: cmds.Deprecated,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated command, use 'ipfs provide stat' instead.\",\n\t\tShortDescription: `\n'ipfs stats provide' is deprecated because provide and reprovide operations\nare now distinct. This command may be replaced by provide only stats in the\nfuture.\n`,\n\t},\n\tArguments: provideStatCmd.Arguments,\n\tOptions: provideStatCmd.Options,\n\tRun: provideStatCmd.Run,\n\tEncoders: provideStatCmd.Encoders,\n\tType: provideStatCmd.Type,\n}\n" }, { "path": "core/commands/stat_reprovide.go", "content": "package commands\n\nimport (\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n)\n\nvar statReprovideCmd = &cmds.Command{\n\tStatus: cmds.Deprecated,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Deprecated command, use 'ipfs provide stat' instead.\",\n\t\tShortDescription: `\n'ipfs stats reprovide' is deprecated because provider stats are now\navailable from 'ipfs provide stat'.\n`,\n\t},\n\tArguments: provideStatCmd.Arguments,\n\tOptions: provideStatCmd.Options,\n\tRun: provideStatCmd.Run,\n\tEncoders: provideStatCmd.Encoders,\n\tType: provideStatCmd.Type,\n}\n" }, { "path": "core/commands/swarm.go", "content": "package commands\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"path\"\n\t\"slices\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/commands\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tic \"github.com/libp2p/go-libp2p/core/crypto\"\n\tinet \"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n\tmamask \"github.com/whyrusleeping/multiaddr-filter\"\n)\n\nconst (\n\tdnsResolveTimeout = 10 * time.Second\n)\n\ntype stringList struct {\n\tStrings []string\n}\n\ntype addrMap struct {\n\tAddrs map[string][]string\n}\n\nvar SwarmCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Interact with the swarm.\",\n\t\tShortDescription: `\n'ipfs swarm' is a tool to manipulate the network swarm. The swarm is the\ncomponent that opens, listens for, and maintains connections to other\nipfs peers in the internet.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"addrs\": swarmAddrsCmd,\n\t\t\"connect\": swarmConnectCmd,\n\t\t\"disconnect\": swarmDisconnectCmd,\n\t\t\"filters\": swarmFiltersCmd,\n\t\t\"peers\": swarmPeersCmd,\n\t\t\"peering\": swarmPeeringCmd,\n\t\t\"resources\": swarmResourcesCmd, // libp2p Network Resource Manager\n\n\t},\n}\n\nconst (\n\tswarmVerboseOptionName = \"verbose\"\n\tswarmStreamsOptionName = \"streams\"\n\tswarmLatencyOptionName = \"latency\"\n\tswarmDirectionOptionName = \"direction\"\n\tswarmResetLimitsOptionName = \"reset\"\n\tswarmUsedResourcesPercentageName = \"min-used-limit-perc\"\n\tswarmIdentifyOptionName = \"identify\"\n)\n\ntype peeringResult struct {\n\tID peer.ID\n\tStatus string\n}\n\nvar swarmPeeringCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Modify the peering subsystem.\",\n\t\tShortDescription: `\n'ipfs swarm peering' manages the peering subsystem.\nPeers in the peering subsystem are maintained to be connected, reconnected\non disconnect with a back-off.\nThe changes are not saved to the config.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"add\": swarmPeeringAddCmd,\n\t\t\"ls\": swarmPeeringLsCmd,\n\t\t\"rm\": swarmPeeringRmCmd,\n\t},\n}\n\nvar swarmPeeringAddCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add peers into the peering subsystem.\",\n\t\tShortDescription: `\n'ipfs swarm peering add' will add the new address to the peering subsystem as one that should always be connected to.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"address\", true, true, \"address of peer to add into the peering subsystem\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\taddrs := make([]ma.Multiaddr, len(req.Arguments))\n\n\t\tfor i, arg := range req.Arguments {\n\t\t\taddr, err := ma.NewMultiaddr(arg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\taddrs[i] = addr\n\t\t}\n\n\t\taddInfos, err := peer.AddrInfosFromP2pAddrs(addrs...)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !node.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tfor _, addrinfo := range addInfos {\n\t\t\tnode.Peering.AddPeer(addrinfo)\n\t\t\terr = res.Emit(peeringResult{addrinfo.ID, \"success\"})\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, pr *peeringResult) error {\n\t\t\tfmt.Fprintf(w, \"add %s %s\\n\", pr.ID.String(), pr.Status)\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: peeringResult{},\n}\n\nvar swarmPeeringLsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List peers registered in the peering subsystem.\",\n\t\tShortDescription: `\n'ipfs swarm peering ls' lists the peers that are registered in the peering subsystem and to which the daemon is always connected.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !node.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tpeers := node.Peering.ListPeers()\n\t\treturn cmds.EmitOnce(res, addrInfos{Peers: peers})\n\t},\n\tType: addrInfos{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ai *addrInfos) error {\n\t\t\tfor _, info := range ai.Peers {\n\t\t\t\tfmt.Fprintf(w, \"%s\\n\", info.ID)\n\t\t\t\tfor _, addr := range info.Addrs {\n\t\t\t\t\tfmt.Fprintf(w, \"\\t%s\\n\", addr)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\ntype addrInfos struct {\n\tPeers []peer.AddrInfo\n}\n\nvar swarmPeeringRmCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove a peer from the peering subsystem.\",\n\t\tShortDescription: `\n'ipfs swarm peering rm' will remove the given ID from the peering subsystem and remove it from the always-on connection.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"ID\", true, true, \"ID of peer to remove from the peering subsystem\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !node.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tfor _, arg := range req.Arguments {\n\t\t\tid, err := peer.Decode(arg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tnode.Peering.RemovePeer(id)\n\t\t\tif err = res.Emit(peeringResult{id, \"success\"}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t},\n\tType: peeringResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, pr *peeringResult) error {\n\t\t\tfmt.Fprintf(w, \"remove %s %s\\n\", pr.ID.String(), pr.Status)\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nvar swarmPeersCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List peers with open connections.\",\n\t\tShortDescription: `\n'ipfs swarm peers' lists the set of peers this node is connected to.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(swarmVerboseOptionName, \"v\", \"display all extra information\"),\n\t\tcmds.BoolOption(swarmStreamsOptionName, \"Also list information about open streams for each peer\"),\n\t\tcmds.BoolOption(swarmLatencyOptionName, \"Also list information about latency to each peer\"),\n\t\tcmds.BoolOption(swarmDirectionOptionName, \"Also list information about the direction of connection\"),\n\t\tcmds.BoolOption(swarmIdentifyOptionName, \"Also list information about peers identify\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tverbose, _ := req.Options[swarmVerboseOptionName].(bool)\n\t\tlatency, _ := req.Options[swarmLatencyOptionName].(bool)\n\t\tstreams, _ := req.Options[swarmStreamsOptionName].(bool)\n\t\tdirection, _ := req.Options[swarmDirectionOptionName].(bool)\n\t\tidentify, _ := req.Options[swarmIdentifyOptionName].(bool)\n\n\t\tconns, err := api.Swarm().Peers(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar out connInfos\n\t\tfor _, c := range conns {\n\t\t\tci := connInfo{\n\t\t\t\tAddr: c.Address().String(),\n\t\t\t\tPeer: c.ID().String(),\n\t\t\t}\n\n\t\t\tif verbose || direction {\n\t\t\t\t// set direction\n\t\t\t\tci.Direction = c.Direction()\n\t\t\t}\n\n\t\t\tif verbose || latency {\n\t\t\t\tlat, err := c.Latency()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tif lat == 0 {\n\t\t\t\t\tci.Latency = \"n/a\"\n\t\t\t\t} else {\n\t\t\t\t\tci.Latency = lat.String()\n\t\t\t\t}\n\t\t\t}\n\t\t\tif verbose || streams {\n\t\t\t\tstrs, err := c.Streams()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tfor _, s := range strs {\n\t\t\t\t\tci.Streams = append(ci.Streams, streamInfo{Protocol: cmdutils.CleanAndTrim(string(s))})\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif verbose || identify {\n\t\t\t\tn, err := cmdenv.GetNode(env)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tidentifyResult, _ := ci.identifyPeer(n.Peerstore, c.ID())\n\t\t\t\tci.Identify = identifyResult\n\t\t\t}\n\t\t\tci.Sort()\n\t\t\tout.Peers = append(out.Peers, ci)\n\t\t}\n\n\t\tout.Sort()\n\t\treturn cmds.EmitOnce(res, &out)\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, ci *connInfos) error {\n\t\t\tpipfs := ma.ProtocolWithCode(ma.P_IPFS).Name\n\t\t\tfor _, info := range ci.Peers {\n\t\t\t\tfmt.Fprintf(w, \"%s/%s/%s\", info.Addr, pipfs, info.Peer)\n\t\t\t\tif info.Latency != \"\" {\n\t\t\t\t\tfmt.Fprintf(w, \" %s\", info.Latency)\n\t\t\t\t}\n\n\t\t\t\tif info.Direction != inet.DirUnknown {\n\t\t\t\t\tfmt.Fprintf(w, \" %s\", directionString(info.Direction))\n\t\t\t\t}\n\t\t\t\tfmt.Fprintln(w)\n\n\t\t\t\tfor _, s := range info.Streams {\n\t\t\t\t\tif s.Protocol == \"\" {\n\t\t\t\t\t\ts.Protocol = \"\"\n\t\t\t\t\t}\n\n\t\t\t\t\tfmt.Fprintf(w, \" %s\\n\", s.Protocol)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: connInfos{},\n}\n\nvar swarmResourcesCmd = &cmds.Command{\n\tStatus: cmds.Experimental,\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Get a summary of all resources accounted for by the libp2p Resource Manager.\",\n\t\tLongDescription: `\nGet a summary of all resources accounted for by the libp2p Resource Manager.\nThis includes the limits and the usage against those limits.\nThis can output a human readable table and JSON encoding.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif node.ResourceManager == nil {\n\t\t\treturn libp2p.ErrNoResourceMgr\n\t\t}\n\n\t\tcfg, err := node.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tuserResourceOverrides, err := node.Repo.UserResourceOverrides()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// FIXME: we shouldn't recompute limits, either save them or load them from libp2p (https://github.com/libp2p/go-libp2p/issues/2166)\n\t\tlimitConfig, _, err := libp2p.LimitConfig(cfg.Swarm, userResourceOverrides)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trapi, ok := node.ResourceManager.(rcmgr.ResourceManagerState)\n\t\tif !ok { // NullResourceManager\n\t\t\treturn libp2p.ErrNoResourceMgr\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, libp2p.MergeLimitsAndStatsIntoLimitsConfigAndUsage(limitConfig, rapi.Stat()))\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.JSON: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, limitsAndUsage libp2p.LimitsConfigAndUsage) error {\n\t\t\treturn json.NewEncoder(w).Encode(limitsAndUsage)\n\t\t}),\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, limitsAndUsage libp2p.LimitsConfigAndUsage) error {\n\t\t\ttw := tabwriter.NewWriter(w, 20, 8, 0, '\\t', 0)\n\t\t\tdefer tw.Flush()\n\n\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\t%s\\t%s\\t%s\\t\\n\", \"Scope\", \"Limit Name\", \"Limit Value\", \"Limit Usage Amount\", \"Limit Usage Percent\")\n\t\t\tfor _, ri := range libp2p.LimitConfigsToInfo(limitsAndUsage) {\n\t\t\t\tvar limit, percentage string\n\t\t\t\tswitch ri.LimitValue {\n\t\t\t\tcase rcmgr.Unlimited64:\n\t\t\t\t\tlimit = \"unlimited\"\n\t\t\t\t\tpercentage = \"n/a\"\n\t\t\t\tcase rcmgr.BlockAllLimit64:\n\t\t\t\t\tlimit = \"blockAll\"\n\t\t\t\t\tpercentage = \"n/a\"\n\t\t\t\tdefault:\n\t\t\t\t\tlimit = strconv.FormatInt(int64(ri.LimitValue), 10)\n\t\t\t\t\tif ri.CurrentUsage == 0 {\n\t\t\t\t\t\tpercentage = \"0%\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\tpercentage = strconv.FormatFloat(float64(ri.CurrentUsage)/float64(ri.LimitValue)*100, 'f', 1, 64) + \"%\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tfmt.Fprintf(tw, \"%s\\t%s\\t%s\\t%d\\t%s\\t\\n\",\n\t\t\t\t\tri.ScopeName,\n\t\t\t\t\tri.LimitName,\n\t\t\t\t\tlimit,\n\t\t\t\t\tri.CurrentUsage,\n\t\t\t\t\tpercentage,\n\t\t\t\t)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: libp2p.LimitsConfigAndUsage{},\n}\n\ntype streamInfo struct {\n\tProtocol string\n}\n\ntype connInfo struct {\n\tAddr string `json:\",omitempty\"`\n\tPeer string `json:\",omitempty\"`\n\tLatency string `json:\",omitempty\"`\n\tMuxer string `json:\",omitempty\"`\n\tDirection inet.Direction `json:\",omitempty\"`\n\tStreams []streamInfo `json:\",omitempty\"`\n\tIdentify IdOutput\n}\n\nfunc (ci *connInfo) Sort() {\n\tslices.SortFunc(ci.Streams, func(a, b streamInfo) int {\n\t\treturn strings.Compare(a.Protocol, b.Protocol)\n\t})\n}\n\ntype connInfos struct {\n\tPeers []connInfo\n}\n\nfunc (ci *connInfos) Sort() {\n\tslices.SortFunc(ci.Peers, func(a, b connInfo) int {\n\t\treturn strings.Compare(a.Addr, b.Addr)\n\t})\n}\n\nfunc (ci *connInfo) identifyPeer(ps pstore.Peerstore, p peer.ID) (IdOutput, error) {\n\tvar info IdOutput\n\tinfo.ID = p.String()\n\n\tif pk := ps.PubKey(p); pk != nil {\n\t\tpkb, err := ic.MarshalPublicKey(pk)\n\t\tif err != nil {\n\t\t\treturn IdOutput{}, err\n\t\t}\n\t\tinfo.PublicKey = base64.StdEncoding.EncodeToString(pkb)\n\t}\n\n\taddrInfo := ps.PeerInfo(p)\n\taddrs, err := peer.AddrInfoToP2pAddrs(&addrInfo)\n\tif err != nil {\n\t\treturn IdOutput{}, err\n\t}\n\n\tfor _, a := range addrs {\n\t\tinfo.Addresses = append(info.Addresses, a.String())\n\t}\n\tslices.Sort(info.Addresses)\n\n\tif protocols, err := ps.GetProtocols(p); err == nil {\n\t\tfor _, proto := range protocols {\n\t\t\tinfo.Protocols = append(info.Protocols, protocol.ID(cmdutils.CleanAndTrim(string(proto))))\n\t\t}\n\t\tslices.Sort(info.Protocols)\n\t}\n\n\tif v, err := ps.Get(p, \"AgentVersion\"); err == nil {\n\t\tif vs, ok := v.(string); ok {\n\t\t\tinfo.AgentVersion = cmdutils.CleanAndTrim(vs)\n\t\t}\n\t}\n\n\treturn info, nil\n}\n\n// directionString transfers to string\nfunc directionString(d inet.Direction) string {\n\tswitch d {\n\tcase inet.DirInbound:\n\t\treturn \"inbound\"\n\tcase inet.DirOutbound:\n\t\treturn \"outbound\"\n\tdefault:\n\t\treturn \"\"\n\t}\n}\n\nvar swarmAddrsCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List known addresses. Useful for debugging.\",\n\t\tShortDescription: `\n'ipfs swarm addrs' lists all addresses this node is aware of.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"autonat\": swarmAddrsAutoNATCmd,\n\t\t\"local\": swarmAddrsLocalCmd,\n\t\t\"listen\": swarmAddrsListenCmd,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\taddrs, err := api.Swarm().KnownAddrs(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tout := make(map[string][]string)\n\t\tfor p, paddrs := range addrs {\n\t\t\ts := p.String()\n\t\t\tfor _, a := range paddrs {\n\t\t\t\tout[s] = append(out[s], a.String())\n\t\t\t}\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &addrMap{Addrs: out})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, am *addrMap) error {\n\t\t\t// sort the ids first\n\t\t\tids := make([]string, 0, len(am.Addrs))\n\t\t\tfor p := range am.Addrs {\n\t\t\t\tids = append(ids, p)\n\t\t\t}\n\t\t\tslices.Sort(ids)\n\n\t\t\tfor _, p := range ids {\n\t\t\t\tpaddrs := am.Addrs[p]\n\t\t\t\tfmt.Fprintf(w, \"%s (%d)\\n\", p, len(paddrs))\n\t\t\t\tfor _, addr := range paddrs {\n\t\t\t\t\tfmt.Fprintf(w, \"\\t%s\\n\", addr)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: addrMap{},\n}\n\nvar swarmAddrsLocalCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List local addresses.\",\n\t\tShortDescription: `\n'ipfs swarm addrs local' lists all local listening addresses announced to the network.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(\"id\", \"Show peer ID in addresses.\"),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tshowid, _ := req.Options[\"id\"].(bool)\n\t\tself, err := api.Key().Self(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tmaddrs, err := api.Swarm().LocalAddrs(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar addrs []string\n\t\tp2pProtocolName := ma.ProtocolWithCode(ma.P_P2P).Name\n\t\tfor _, addr := range maddrs {\n\t\t\tsaddr := addr.String()\n\t\t\tif showid {\n\t\t\t\tsaddr = path.Join(saddr, p2pProtocolName, self.ID().String())\n\t\t\t}\n\t\t\taddrs = append(addrs, saddr)\n\t\t}\n\t\tslices.Sort(addrs)\n\t\treturn cmds.EmitOnce(res, &stringList{addrs})\n\t},\n\tType: stringList{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n}\n\nvar swarmAddrsListenCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"List interface listening addresses.\",\n\t\tShortDescription: `\n'ipfs swarm addrs listen' lists all interface addresses the node is listening on.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tvar addrs []string\n\t\tmaddrs, err := api.Swarm().ListenAddrs(req.Context)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor _, addr := range maddrs {\n\t\t\taddrs = append(addrs, addr.String())\n\t\t}\n\t\tslices.Sort(addrs)\n\n\t\treturn cmds.EmitOnce(res, &stringList{addrs})\n\t},\n\tType: stringList{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n}\n\nvar swarmConnectCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Open connection to a given peer.\",\n\t\tShortDescription: `\n'ipfs swarm connect' attempts to ensure a connection to a given peer.\n\nMultiaddresses given are advisory, for example the node may already be aware of other addresses for a given peer or may already have an established connection to the peer.\n\nThe address format is a libp2p multiaddr:\n\nipfs swarm connect /ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"address\", true, true, \"Address of peer to connect to.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\taddrs := req.Arguments\n\n\t\tpis, err := parseAddresses(req.Context, addrs, node.DNSResolver)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toutput := make([]string, len(pis))\n\t\tfor i, pi := range pis {\n\t\t\toutput[i] = \"connect \" + pi.ID.String()\n\n\t\t\terr := api.Swarm().Connect(req.Context, pi)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"%s failure: %s\", output[i], err)\n\t\t\t}\n\t\t\toutput[i] += \" success\"\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &stringList{output})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n\tType: stringList{},\n}\n\nvar swarmDisconnectCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Close connection to a given address.\",\n\t\tShortDescription: `\n'ipfs swarm disconnect' closes a connection to a peer address. The address\nformat is an IPFS multiaddr:\n\nipfs swarm disconnect /ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\n\nThe disconnect is not permanent; if ipfs needs to talk to that address later,\nit will reconnect.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"address\", true, true, \"Address of peer to disconnect from.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnode, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tapi, err := cmdenv.GetApi(env, req)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\taddrs, err := parseAddresses(req.Context, req.Arguments, node.DNSResolver)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toutput := make([]string, 0, len(addrs))\n\t\tfor _, ainfo := range addrs {\n\t\t\tmaddrs, err := peer.AddrInfoToP2pAddrs(&ainfo)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t// FIXME: This will print:\n\t\t\t//\n\t\t\t// disconnect QmFoo success\n\t\t\t// disconnect QmFoo success\n\t\t\t// ...\n\t\t\t//\n\t\t\t// Once per address specified. However, I'm not sure of\n\t\t\t// a good backwards compat solution. Right now, I'm just\n\t\t\t// preserving the current behavior.\n\t\t\tfor _, addr := range maddrs {\n\t\t\t\tmsg := \"disconnect \" + ainfo.ID.String()\n\t\t\t\tif err := api.Swarm().Disconnect(req.Context, addr); err != nil {\n\t\t\t\t\tmsg += \" failure: \" + err.Error()\n\t\t\t\t} else {\n\t\t\t\t\tmsg += \" success\"\n\t\t\t\t}\n\t\t\t\toutput = append(output, msg)\n\t\t\t}\n\t\t}\n\t\treturn cmds.EmitOnce(res, &stringList{output})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n\tType: stringList{},\n}\n\n// parseAddresses is a function that takes in a slice of string peer addresses\n// (multiaddr + peerid) and returns a slice of properly constructed peers\nfunc parseAddresses(ctx context.Context, addrs []string, rslv *madns.Resolver) ([]peer.AddrInfo, error) {\n\t// resolve addresses\n\tmaddrs, err := resolveAddresses(ctx, addrs, rslv)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn peer.AddrInfosFromP2pAddrs(maddrs...)\n}\n\n// resolveAddresses resolves addresses parallelly\nfunc resolveAddresses(ctx context.Context, addrs []string, rslv *madns.Resolver) ([]ma.Multiaddr, error) {\n\tctx, cancel := context.WithTimeout(ctx, dnsResolveTimeout)\n\tdefer cancel()\n\n\tvar maddrs []ma.Multiaddr\n\tvar wg sync.WaitGroup\n\tresolveErrC := make(chan error, len(addrs))\n\n\tmaddrC := make(chan ma.Multiaddr)\n\n\tfor _, addr := range addrs {\n\t\tmaddr, err := ma.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// check whether address ends in `ipfs/Qm...`\n\t\tif _, last := ma.SplitLast(maddr); last.Protocol().Code == ma.P_IPFS {\n\t\t\tmaddrs = append(maddrs, maddr)\n\t\t\tcontinue\n\t\t}\n\t\twg.Add(1)\n\t\tgo func(maddr ma.Multiaddr) {\n\t\t\tdefer wg.Done()\n\t\t\traddrs, err := rslv.Resolve(ctx, maddr)\n\t\t\tif err != nil {\n\t\t\t\tresolveErrC <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\t// filter out addresses that still doesn't end in `ipfs/Qm...`\n\t\t\tfound := 0\n\t\t\tfor _, raddr := range raddrs {\n\t\t\t\tif _, last := ma.SplitLast(raddr); last != nil && last.Protocol().Code == ma.P_IPFS {\n\t\t\t\t\tmaddrC <- raddr\n\t\t\t\t\tfound++\n\t\t\t\t}\n\t\t\t}\n\t\t\tif found == 0 {\n\t\t\t\tresolveErrC <- fmt.Errorf(\"found no ipfs peers at %s\", maddr)\n\t\t\t}\n\t\t}(maddr)\n\t}\n\tgo func() {\n\t\twg.Wait()\n\t\tclose(maddrC)\n\t}()\n\n\tfor maddr := range maddrC {\n\t\tmaddrs = append(maddrs, maddr)\n\t}\n\n\tselect {\n\tcase err := <-resolveErrC:\n\t\treturn nil, err\n\tdefault:\n\t}\n\n\treturn maddrs, nil\n}\n\nvar swarmFiltersCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Manipulate address filters.\",\n\t\tShortDescription: `\n'ipfs swarm filters' will list out currently applied filters. Its subcommands\ncan be used to add or remove said filters. Filters are specified using the\nmultiaddr-filter format:\n\nExample:\n\n /ip4/192.168.0.0/ipcidr/16\n\nWhere the above is equivalent to the standard CIDR:\n\n 192.168.0.0/16\n\nFilters default to those specified under the \"Swarm.AddrFilters\" config key.\n`,\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"add\": swarmFiltersAddCmd,\n\t\t\"rm\": swarmFiltersRmCmd,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !n.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tvar output []string\n\t\tfor _, f := range n.Filters.FiltersForAction(ma.ActionDeny) {\n\t\t\ts, err := mamask.ConvertIPNet(&f)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\toutput = append(output, s)\n\t\t}\n\t\treturn cmds.EmitOnce(res, &stringList{output})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n\tType: stringList{},\n}\n\nvar swarmFiltersAddCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Add an address filter.\",\n\t\tShortDescription: `\n'ipfs swarm filters add' will add an address filter to the daemons swarm.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"address\", true, true, \"Multiaddr to filter.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !n.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tif len(req.Arguments) == 0 {\n\t\t\treturn errors.New(\"no filters to add\")\n\t\t}\n\n\t\tr, err := fsrepo.Open(env.(*commands.Context).ConfigRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tcfg, err := r.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor _, arg := range req.Arguments {\n\t\t\tmask, err := mamask.NewMask(arg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tn.Filters.AddFilter(*mask, ma.ActionDeny)\n\t\t}\n\n\t\tadded, err := filtersAdd(r, cfg, req.Arguments)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &stringList{added})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n\tType: stringList{},\n}\n\nvar swarmFiltersRmCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Remove an address filter.\",\n\t\tShortDescription: `\n'ipfs swarm filters rm' will remove an address filter from the daemons swarm.\n`,\n\t},\n\tArguments: []cmds.Argument{\n\t\tcmds.StringArg(\"address\", true, true, \"Multiaddr filter to remove.\").EnableStdin(),\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tn, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !n.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tr, err := fsrepo.Open(env.(*commands.Context).ConfigRoot)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer r.Close()\n\t\tcfg, err := r.Config()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif req.Arguments[0] == \"all\" || req.Arguments[0] == \"*\" {\n\t\t\tfs := n.Filters.FiltersForAction(ma.ActionDeny)\n\t\t\tfor _, f := range fs {\n\t\t\t\tn.Filters.RemoveLiteral(f)\n\t\t\t}\n\n\t\t\tremoved, err := filtersRemoveAll(r, cfg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\treturn cmds.EmitOnce(res, &stringList{removed})\n\t\t}\n\n\t\tfor _, arg := range req.Arguments {\n\t\t\tmask, err := mamask.NewMask(arg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tn.Filters.RemoveLiteral(*mask)\n\t\t}\n\n\t\tremoved, err := filtersRemove(r, cfg, req.Arguments)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, &stringList{removed})\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(safeTextListEncoder),\n\t},\n\tType: stringList{},\n}\n\nfunc filtersAdd(r repo.Repo, cfg *config.Config, filters []string) ([]string, error) {\n\taddedMap := map[string]struct{}{}\n\taddedList := make([]string, 0, len(filters))\n\n\t// re-add cfg swarm filters to rm dupes\n\toldFilters := cfg.Swarm.AddrFilters\n\tcfg.Swarm.AddrFilters = nil\n\n\t// add new filters\n\tfor _, filter := range filters {\n\t\tif _, found := addedMap[filter]; found {\n\t\t\tcontinue\n\t\t}\n\n\t\tcfg.Swarm.AddrFilters = append(cfg.Swarm.AddrFilters, filter)\n\t\taddedList = append(addedList, filter)\n\t\taddedMap[filter] = struct{}{}\n\t}\n\n\t// add back original filters. in this order so that we output them.\n\tfor _, filter := range oldFilters {\n\t\tif _, found := addedMap[filter]; found {\n\t\t\tcontinue\n\t\t}\n\n\t\tcfg.Swarm.AddrFilters = append(cfg.Swarm.AddrFilters, filter)\n\t\taddedMap[filter] = struct{}{}\n\t}\n\n\tif err := r.SetConfig(cfg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn addedList, nil\n}\n\nfunc filtersRemoveAll(r repo.Repo, cfg *config.Config) ([]string, error) {\n\tremoved := cfg.Swarm.AddrFilters\n\tcfg.Swarm.AddrFilters = nil\n\n\tif err := r.SetConfig(cfg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn removed, nil\n}\n\nfunc filtersRemove(r repo.Repo, cfg *config.Config, toRemoveFilters []string) ([]string, error) {\n\tremoved := make([]string, 0, len(toRemoveFilters))\n\tkeep := make([]string, 0, len(cfg.Swarm.AddrFilters))\n\n\toldFilters := cfg.Swarm.AddrFilters\n\n\tfor _, oldFilter := range oldFilters {\n\t\tfound := false\n\t\tfor _, toRemoveFilter := range toRemoveFilters {\n\t\t\tif oldFilter == toRemoveFilter {\n\t\t\t\tfound = true\n\t\t\t\tremoved = append(removed, toRemoveFilter)\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif !found {\n\t\t\tkeep = append(keep, oldFilter)\n\t\t}\n\t}\n\tcfg.Swarm.AddrFilters = keep\n\n\tif err := r.SetConfig(cfg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn removed, nil\n}\n" }, { "path": "core/commands/swarm_addrs_autonat.go", "content": "package commands\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\n// reachabilityHost provides access to the AutoNAT reachability status.\ntype reachabilityHost interface {\n\tReachability() network.Reachability\n}\n\n// confirmedAddrsHost provides access to per-address reachability from AutoNAT V2.\ntype confirmedAddrsHost interface {\n\tConfirmedAddrs() (reachable, unreachable, unknown []ma.Multiaddr)\n}\n\n// autoNATResult represents the AutoNAT reachability information.\ntype autoNATResult struct {\n\tReachability string `json:\"reachability\"`\n\tReachable []string `json:\"reachable,omitempty\"`\n\tUnreachable []string `json:\"unreachable,omitempty\"`\n\tUnknown []string `json:\"unknown,omitempty\"`\n}\n\nfunc multiaddrsToStrings(addrs []ma.Multiaddr) []string {\n\tout := make([]string, len(addrs))\n\tfor i, a := range addrs {\n\t\tout[i] = a.String()\n\t}\n\treturn out\n}\n\nfunc writeAddrSection(w io.Writer, label string, addrs []string) {\n\tif len(addrs) > 0 {\n\t\tfmt.Fprintf(w, \" %s:\\n\", label)\n\t\tfor _, addr := range addrs {\n\t\t\tfmt.Fprintf(w, \" %s\\n\", addr)\n\t\t}\n\t}\n}\n\nvar swarmAddrsAutoNATCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show address reachability as determined by AutoNAT V2.\",\n\t\tShortDescription: `\n'ipfs swarm addrs autonat' shows the reachability status of your node's\naddresses as determined by AutoNAT V2.\n`,\n\t\tLongDescription: `\n'ipfs swarm addrs autonat' shows the reachability status of your node's\naddresses as verified by AutoNAT V2.\n\nAutoNAT V2 probes your node's addresses to determine if they are reachable\nfrom the public internet. This helps understand whether other peers can\ndial your node directly.\n\nThe output shows:\n- Reachability: Overall status (Public, Private, or Unknown)\n- Reachable: Addresses confirmed to be publicly reachable\n- Unreachable: Addresses that failed reachability checks\n- Unknown: Addresses that haven't been tested yet\n\nFor more information on AutoNAT V2, see:\nhttps://github.com/libp2p/specs/blob/master/autonat/autonat-v2.md\n\nExample:\n\n > ipfs swarm addrs autonat\n AutoNAT V2 Status:\n Reachability: Public\n\n Per-Address Reachability:\n Reachable:\n /ip4/203.0.113.42/tcp/4001\n /ip4/203.0.113.42/udp/4001/quic-v1\n Unreachable:\n /ip6/2001:db8::1/tcp/4001\n Unknown:\n /ip4/203.0.113.42/udp/4001/webrtc-direct\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tresult := autoNATResult{\n\t\t\tReachability: network.ReachabilityUnknown.String(),\n\t\t}\n\n\t\t// Get per-address reachability from AutoNAT V2.\n\t\t// The host embeds *BasicHost (closableBasicHost, closableRoutedHost)\n\t\t// which implements ConfirmedAddrs.\n\t\tif h, ok := nd.PeerHost.(confirmedAddrsHost); ok {\n\t\t\treachable, unreachable, unknown := h.ConfirmedAddrs()\n\t\t\tresult.Reachable = multiaddrsToStrings(reachable)\n\t\t\tresult.Unreachable = multiaddrsToStrings(unreachable)\n\t\t\tresult.Unknown = multiaddrsToStrings(unknown)\n\t\t}\n\n\t\t// Get overall reachability status.\n\t\tif h, ok := nd.PeerHost.(reachabilityHost); ok {\n\t\t\tresult.Reachability = h.Reachability().String()\n\t\t}\n\n\t\treturn cmds.EmitOnce(res, result)\n\t},\n\tType: autoNATResult{},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, result autoNATResult) error {\n\t\t\tfmt.Fprintln(w, \"AutoNAT V2 Status:\")\n\t\t\tfmt.Fprintf(w, \" Reachability: %s\\n\", result.Reachability)\n\n\t\t\tfmt.Fprintln(w)\n\t\t\tfmt.Fprintln(w, \"Per-Address Reachability:\")\n\n\t\t\twriteAddrSection(w, \"Reachable\", result.Reachable)\n\t\t\twriteAddrSection(w, \"Unreachable\", result.Unreachable)\n\t\t\twriteAddrSection(w, \"Unknown\", result.Unknown)\n\n\t\t\tif len(result.Reachable) == 0 && len(result.Unreachable) == 0 && len(result.Unknown) == 0 {\n\t\t\t\tfmt.Fprintln(w, \" (no address reachability data available)\")\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n" }, { "path": "core/commands/sysdiag.go", "content": "package commands\n\nimport (\n\t\"os\"\n\t\"runtime\"\n\n\t\"github.com/ipfs/go-ipfs-cmds\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\tcmdenv \"github.com/ipfs/kubo/core/commands/cmdenv\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n\tsysi \"github.com/whyrusleeping/go-sysinfo\"\n)\n\nvar sysDiagCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Print system diagnostic information.\",\n\t\tShortDescription: `\nPrints out information about your computer to aid in easier debugging.\n`,\n\t},\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tinfo, err := getInfo(nd)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn cmds.EmitOnce(res, info)\n\t},\n}\n\nfunc getInfo(nd *core.IpfsNode) (map[string]any, error) {\n\tinfo := make(map[string]any)\n\terr := runtimeInfo(info)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = envVarInfo(info)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = diskSpaceInfo(info)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = memInfo(info)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = netInfo(nd.IsOnline, info)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tinfo[\"ipfs_version\"] = version.CurrentVersionNumber\n\tinfo[\"ipfs_commit\"] = version.CurrentCommit\n\treturn info, nil\n}\n\nfunc runtimeInfo(out map[string]any) error {\n\trt := make(map[string]any)\n\trt[\"os\"] = runtime.GOOS\n\trt[\"arch\"] = runtime.GOARCH\n\trt[\"compiler\"] = runtime.Compiler\n\trt[\"version\"] = runtime.Version()\n\trt[\"numcpu\"] = runtime.NumCPU()\n\trt[\"gomaxprocs\"] = runtime.GOMAXPROCS(0)\n\trt[\"numgoroutines\"] = runtime.NumGoroutine()\n\n\tout[\"runtime\"] = rt\n\treturn nil\n}\n\nfunc envVarInfo(out map[string]any) error {\n\tev := make(map[string]any)\n\tev[\"GOPATH\"] = os.Getenv(\"GOPATH\")\n\tev[config.EnvDir] = os.Getenv(config.EnvDir)\n\n\tout[\"environment\"] = ev\n\treturn nil\n}\n\nfunc diskSpaceInfo(out map[string]any) error {\n\tpathRoot, err := config.PathRoot()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdinfo, err := sysi.DiskUsage(pathRoot)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tout[\"diskinfo\"] = map[string]any{\n\t\t\"fstype\": dinfo.FsType,\n\t\t\"total_space\": dinfo.Total,\n\t\t\"free_space\": dinfo.Free,\n\t}\n\n\treturn nil\n}\n\nfunc memInfo(out map[string]any) error {\n\tm := make(map[string]any)\n\n\tmeminf, err := sysi.MemoryInfo()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tm[\"swap\"] = meminf.Swap\n\tm[\"virt\"] = meminf.Used\n\tout[\"memory\"] = m\n\treturn nil\n}\n\nfunc netInfo(online bool, out map[string]any) error {\n\tn := make(map[string]any)\n\taddrs, err := manet.InterfaceMultiaddrs()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tstraddrs := make([]string, len(addrs))\n\tfor i, a := range addrs {\n\t\tstraddrs[i] = a.String()\n\t}\n\n\tn[\"interface_addresses\"] = straddrs\n\tn[\"online\"] = online\n\tout[\"net\"] = n\n\treturn nil\n}\n" }, { "path": "core/commands/version.go", "content": "package commands\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"runtime/debug\"\n\t\"strings\"\n\n\tversioncmp \"github.com/hashicorp/go-version\"\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/commands/cmdenv\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n)\n\nconst (\n\tversionNumberOptionName = \"number\"\n\tversionCommitOptionName = \"commit\"\n\tversionRepoOptionName = \"repo\"\n\tversionAllOptionName = \"all\"\n\tversionCheckThresholdOptionName = \"min-percent\"\n)\n\nvar VersionCmd = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Show IPFS version information.\",\n\t\tShortDescription: \"Returns the current version of IPFS and exits.\",\n\t},\n\tSubcommands: map[string]*cmds.Command{\n\t\t\"deps\": depsVersionCommand,\n\t\t\"check\": checkVersionCommand,\n\t},\n\n\tOptions: []cmds.Option{\n\t\tcmds.BoolOption(versionNumberOptionName, \"n\", \"Only show the version number.\"),\n\t\tcmds.BoolOption(versionCommitOptionName, \"Show the commit hash.\"),\n\t\tcmds.BoolOption(versionRepoOptionName, \"Show repo version.\"),\n\t\tcmds.BoolOption(versionAllOptionName, \"Show all version information\"),\n\t},\n\t// must be permitted to run before init\n\tExtra: CreateCmdExtras(SetDoesNotUseRepo(true), SetDoesNotUseConfigAsInput(true)),\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\treturn cmds.EmitOnce(res, version.GetVersionInfo())\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, version *version.VersionInfo) error {\n\t\t\tall, _ := req.Options[versionAllOptionName].(bool)\n\t\t\tif all {\n\t\t\t\tver := version.Version\n\t\t\t\tif version.Commit != \"\" {\n\t\t\t\t\tver += \"-\" + version.Commit\n\t\t\t\t}\n\t\t\t\tout := fmt.Sprintf(\"Kubo version: %s\\n\"+\n\t\t\t\t\t\"Repo version: %s\\nSystem version: %s\\nGolang version: %s\\n\",\n\t\t\t\t\tver, version.Repo, version.System, version.Golang)\n\t\t\t\tfmt.Fprint(w, out)\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tcommit, _ := req.Options[versionCommitOptionName].(bool)\n\t\t\tcommitTxt := \"\"\n\t\t\tif commit && version.Commit != \"\" {\n\t\t\t\tcommitTxt = \"-\" + version.Commit\n\t\t\t}\n\n\t\t\trepo, _ := req.Options[versionRepoOptionName].(bool)\n\t\t\tif repo {\n\t\t\t\tfmt.Fprintln(w, version.Repo)\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tnumber, _ := req.Options[versionNumberOptionName].(bool)\n\t\t\tif number {\n\t\t\t\tfmt.Fprintln(w, version.Version+commitTxt)\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tfmt.Fprintf(w, \"ipfs version %s%s\\n\", version.Version, commitTxt)\n\t\t\treturn nil\n\t\t}),\n\t},\n\tType: version.VersionInfo{},\n}\n\ntype Dependency struct {\n\tPath string\n\tVersion string\n\tReplacedBy string\n\tSum string\n}\n\nconst pkgVersionFmt = \"%s@%s\"\n\nvar depsVersionCommand = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Shows information about dependencies used for build.\",\n\t\tShortDescription: `\nPrint out all dependencies and their versions.`,\n\t},\n\tType: Dependency{},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tinfo, ok := debug.ReadBuildInfo()\n\t\tif !ok {\n\t\t\treturn errors.New(\"no embedded dependency information\")\n\t\t}\n\t\ttoDependency := func(mod *debug.Module) (dep Dependency) {\n\t\t\tdep.Path = mod.Path\n\t\t\tdep.Version = mod.Version\n\t\t\tdep.Sum = mod.Sum\n\t\t\tif repl := mod.Replace; repl != nil {\n\t\t\t\tdep.ReplacedBy = fmt.Sprintf(pkgVersionFmt, repl.Path, repl.Version)\n\t\t\t}\n\t\t\treturn\n\t\t}\n\t\tif err := res.Emit(toDependency(&info.Main)); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tfor _, dep := range info.Deps {\n\t\t\tif err := res.Emit(toDependency(dep)); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t},\n\tEncoders: cmds.EncoderMap{\n\t\tcmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, dep Dependency) error {\n\t\t\tfmt.Fprintf(w, pkgVersionFmt, dep.Path, dep.Version)\n\t\t\tif dep.ReplacedBy != \"\" {\n\t\t\t\tfmt.Fprintf(w, \" => %s\", dep.ReplacedBy)\n\t\t\t}\n\t\t\tfmt.Fprintf(w, \"\\n\")\n\t\t\treturn nil\n\t\t}),\n\t},\n}\n\nconst DefaultMinimalVersionFraction = 0.05 // 5%\n\ntype VersionCheckOutput struct {\n\tUpdateAvailable bool\n\tRunningVersion string\n\tGreatestVersion string\n\tPeersSampled int\n\tWithGreaterVersion int\n}\n\nvar checkVersionCommand = &cmds.Command{\n\tHelptext: cmds.HelpText{\n\t\tTagline: \"Checks Kubo version against connected peers.\",\n\t\tShortDescription: `\nThis command uses the libp2p identify protocol to check the 'AgentVersion'\nof connected peers and see if the Kubo version we're running is outdated.\n\nPeers with an AgentVersion that doesn't start with 'kubo/' are ignored.\n'UpdateAvailable' is set to true only if the 'min-fraction' criteria are met.\n\nThe 'ipfs daemon' does the same check regularly and logs when a new version\nis available. You can stop these regular checks by setting\nVersion.SwarmCheckEnabled:false in the config.\n`,\n\t},\n\tOptions: []cmds.Option{\n\t\tcmds.IntOption(versionCheckThresholdOptionName, \"t\", \"Percentage (1-100) of sampled peers with the new Kubo version needed to trigger an update warning.\").WithDefault(config.DefaultSwarmCheckPercentThreshold),\n\t},\n\tType: VersionCheckOutput{},\n\n\tRun: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {\n\t\tnd, err := cmdenv.GetNode(env)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !nd.IsOnline {\n\t\t\treturn ErrNotOnline\n\t\t}\n\n\t\tminPercent, _ := req.Options[versionCheckThresholdOptionName].(int64)\n\t\toutput, err := DetectNewKuboVersion(nd, minPercent)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := cmds.EmitOnce(res, output); err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t},\n}\n\n// DetectNewKuboVersion observers kubo version reported by other peers via\n// libp2p identify protocol and notifies when threshold fraction of seen swarm\n// is running updated Kubo. It is used by RPC and CLI at 'ipfs version check'\n// and also periodically when 'ipfs daemon' is running.\nfunc DetectNewKuboVersion(nd *core.IpfsNode, minPercent int64) (VersionCheckOutput, error) {\n\tourVersion, err := versioncmp.NewVersion(version.CurrentVersionNumber)\n\tif err != nil {\n\t\treturn VersionCheckOutput{}, fmt.Errorf(\"could not parse our own version %q: %w\",\n\t\t\tversion.CurrentVersionNumber, err)\n\t}\n\t// MAJOR.MINOR.PATCH without any suffix\n\tourVersion = ourVersion.Core()\n\n\tgreatestVersionSeen := ourVersion\n\ttotalPeersSampled := 1 // Us (and to avoid division-by-zero edge case)\n\twithGreaterVersion := 0\n\n\trecordPeerVersion := func(agentVersion string) {\n\t\t// We process the version as is it assembled in GetUserAgentVersion\n\t\tsegments := strings.Split(agentVersion, \"/\")\n\t\tif len(segments) < 2 {\n\t\t\treturn\n\t\t}\n\t\tif segments[0] != \"kubo\" {\n\t\t\treturn\n\t\t}\n\t\tversionNumber := segments[1] // As in our CurrentVersionNumber\n\n\t\tpeerVersion, err := versioncmp.NewVersion(versionNumber)\n\t\tif err != nil {\n\t\t\t// Do not error on invalid remote versions, just ignore\n\t\t\treturn\n\t\t}\n\n\t\t// Ignore prereleases and development releases (-dev, -rcX)\n\t\tif peerVersion.Metadata() != \"\" || peerVersion.Prerelease() != \"\" {\n\t\t\treturn\n\t\t}\n\n\t\t// MAJOR.MINOR.PATCH without any suffix\n\t\tpeerVersion = peerVersion.Core()\n\n\t\t// Valid peer version number\n\t\ttotalPeersSampled += 1\n\t\tif ourVersion.LessThan(peerVersion) {\n\t\t\twithGreaterVersion += 1\n\t\t}\n\t\tif peerVersion.GreaterThan(greatestVersionSeen) {\n\t\t\tgreatestVersionSeen = peerVersion\n\t\t}\n\t}\n\n\tprocessPeerstoreEntry := func(id peer.ID) {\n\t\tif v, err := nd.Peerstore.Get(id, \"AgentVersion\"); err == nil {\n\t\t\trecordPeerVersion(v.(string))\n\t\t} else if errors.Is(err, pstore.ErrNotFound) { // ignore noop\n\t\t} else { // a bug, usually.\n\t\t\tlog.Errorw(\"failed to get agent version from peerstore\", \"error\", err)\n\t\t}\n\t}\n\n\t// Amino DHT client keeps information about previously seen peers\n\tif nd.HasActiveDHTClient() && nd.DHTClient != nd.DHT {\n\t\tclient, ok := nd.DHTClient.(*fullrt.FullRT)\n\t\tif !ok {\n\t\t\treturn VersionCheckOutput{}, errors.New(\"could not perform version check due to missing or incompatible DHT configuration\")\n\t\t}\n\t\tfor _, p := range client.Stat() {\n\t\t\tprocessPeerstoreEntry(p)\n\t\t}\n\t} else if nd.DHT != nil && nd.DHT.WAN != nil {\n\t\tfor _, pi := range nd.DHT.WAN.RoutingTable().GetPeerInfos() {\n\t\t\tprocessPeerstoreEntry(pi.Id)\n\t\t}\n\t} else if nd.DHT != nil && nd.DHT.LAN != nil {\n\t\tfor _, pi := range nd.DHT.LAN.RoutingTable().GetPeerInfos() {\n\t\t\tprocessPeerstoreEntry(pi.Id)\n\t\t}\n\t} else {\n\t\treturn VersionCheckOutput{}, errors.New(\"could not perform version check due to missing or incompatible DHT configuration\")\n\t}\n\n\tif minPercent < 1 || minPercent > 100 {\n\t\tif minPercent == 0 {\n\t\t\tminPercent = config.DefaultSwarmCheckPercentThreshold\n\t\t} else {\n\t\t\treturn VersionCheckOutput{}, errors.New(\"Version.SwarmCheckPercentThreshold must be between 1 and 100\")\n\t\t}\n\t}\n\n\tminFraction := float64(minPercent) / 100.0\n\n\t// UpdateAvailable flag is set only if minFraction was reached\n\tgreaterFraction := float64(withGreaterVersion) / float64(totalPeersSampled)\n\n\t// Gathered metric are returned every time\n\treturn VersionCheckOutput{\n\t\tUpdateAvailable: (greaterFraction >= minFraction),\n\t\tRunningVersion: ourVersion.String(),\n\t\tGreatestVersion: greatestVersionSeen.String(),\n\t\tPeersSampled: totalPeersSampled,\n\t\tWithGreaterVersion: withGreaterVersion,\n\t}, nil\n}\n" }, { "path": "core/core.go", "content": "/*\nPackage core implements the IpfsNode object and related methods.\n\nPackages underneath core/ provide a (relatively) stable, low-level API\nto carry out most IPFS-related tasks. For more details on the other\ninterfaces and how core/... fits into the bigger IPFS picture, see:\n\n\t$ godoc github.com/ipfs/go-ipfs\n*/\npackage core\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/filestore\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/go-datastore\"\n\n\tbitswap \"github.com/ipfs/boxo/bitswap\"\n\tbserv \"github.com/ipfs/boxo/blockservice\"\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\texchange \"github.com/ipfs/boxo/exchange\"\n\t\"github.com/ipfs/boxo/fetcher\"\n\tmfs \"github.com/ipfs/boxo/mfs\"\n\tpathresolver \"github.com/ipfs/boxo/path/resolver\"\n\tprovider \"github.com/ipfs/boxo/provider\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tddht \"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\tpubsub \"github.com/libp2p/go-libp2p-pubsub\"\n\tpsrouter \"github.com/libp2p/go-libp2p-pubsub-router\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\tconnmgr \"github.com/libp2p/go-libp2p/core/connmgr\"\n\tic \"github.com/libp2p/go-libp2p/core/crypto\"\n\tp2phost \"github.com/libp2p/go-libp2p/core/host\"\n\tmetrics \"github.com/libp2p/go-libp2p/core/metrics\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n\t\"github.com/libp2p/go-libp2p/p2p/discovery/mdns\"\n\tp2pbhost \"github.com/libp2p/go-libp2p/p2p/host/basic\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n\n\t\"github.com/ipfs/boxo/bootstrap\"\n\t\"github.com/ipfs/boxo/namesys\"\n\tipnsrp \"github.com/ipfs/boxo/namesys/republisher\"\n\t\"github.com/ipfs/boxo/peering\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/fuse/mount\"\n\t\"github.com/ipfs/kubo/p2p\"\n\t\"github.com/ipfs/kubo/repo\"\n\tirouting \"github.com/ipfs/kubo/routing\"\n)\n\nvar log = logging.Logger(\"core\")\n\n// IpfsNode is IPFS Core module. It represents an IPFS instance.\ntype IpfsNode struct {\n\t// Self\n\tIdentity peer.ID // the local node's identity\n\n\tRepo repo.Repo\n\n\t// Local node\n\tPinning pin.Pinner // the pinning manager\n\tMounts Mounts `optional:\"true\"` // current mount state, if any.\n\tPrivateKey ic.PrivKey `optional:\"true\"` // the local node's private Key\n\tPNetFingerprint libp2p.PNetFingerprint `optional:\"true\"` // fingerprint of private network\n\n\t// Services\n\tPeerstore pstore.Peerstore `optional:\"true\"` // storage for other Peer instances\n\tBlockstore bstore.GCBlockstore // the block store (lower level)\n\tFilestore *filestore.Filestore `optional:\"true\"` // the filestore blockstore\n\tBaseBlocks node.BaseBlocks // the raw blockstore, no filestore wrapping\n\tGCLocker bstore.GCLocker // the locker used to protect the blockstore during gc\n\tBlocks bserv.BlockService // the block service, get/add blocks.\n\tDAG ipld.DAGService // the merkle dag service, get/add objects.\n\tIPLDFetcherFactory fetcher.Factory `name:\"ipldFetcher\"` // fetcher that paths over the IPLD data model\n\tUnixFSFetcherFactory fetcher.Factory `name:\"unixfsFetcher\"` // fetcher that interprets UnixFS data\n\tOfflineIPLDFetcherFactory fetcher.Factory `name:\"offlineIpldFetcher\"` // fetcher that paths over the IPLD data model without fetching new blocks\n\tOfflineUnixFSFetcherFactory fetcher.Factory `name:\"offlineUnixfsFetcher\"` // fetcher that interprets UnixFS data without fetching new blocks\n\tReporter *metrics.BandwidthCounter `optional:\"true\"`\n\tDiscovery mdns.Service `optional:\"true\"`\n\tFilesRoot *mfs.Root\n\tRecordValidator record.Validator\n\n\t// Online\n\tPeerHost p2phost.Host `optional:\"true\"` // the network host (server+client)\n\tPeering *peering.PeeringService `optional:\"true\"`\n\tFilters *ma.Filters `optional:\"true\"`\n\tBootstrapper io.Closer `optional:\"true\"` // the periodic bootstrapper\n\tContentDiscovery routing.ContentDiscovery `optional:\"true\"` // the discovery part of the routing system\n\tDNSResolver *madns.Resolver // the DNS resolver\n\tIPLDPathResolver pathresolver.Resolver `name:\"ipldPathResolver\"` // The IPLD path resolver\n\tUnixFSPathResolver pathresolver.Resolver `name:\"unixFSPathResolver\"` // The UnixFS path resolver\n\tOfflineIPLDPathResolver pathresolver.Resolver `name:\"offlineIpldPathResolver\"` // The IPLD path resolver that uses only locally available blocks\n\tOfflineUnixFSPathResolver pathresolver.Resolver `name:\"offlineUnixFSPathResolver\"` // The UnixFS path resolver that uses only locally available blocks\n\tExchange exchange.Interface // the block exchange + strategy\n\tBitswap *bitswap.Bitswap `optional:\"true\"` // The Bitswap instance\n\tNamesys namesys.NameSystem // the name system, resolves paths to hashes\n\tProvidingStrategy config.ProvideStrategy `optional:\"true\"`\n\tProvidingKeyChanFunc provider.KeyChanFunc `optional:\"true\"`\n\tIpnsRepub *ipnsrp.Republisher `optional:\"true\"`\n\tResourceManager network.ResourceManager `optional:\"true\"`\n\n\tPubSub *pubsub.PubSub `optional:\"true\"`\n\tPSRouter *psrouter.PubsubValueStore `optional:\"true\"`\n\n\tRouting irouting.ProvideManyRouter `optional:\"true\"` // the routing system. recommend ipfs-dht\n\tProvider node.DHTProvider // the value provider system\n\tDHT *ddht.DHT `optional:\"true\"`\n\tDHTClient routing.Routing `name:\"dhtc\" optional:\"true\"`\n\n\tP2P *p2p.P2P `optional:\"true\"`\n\n\tctx context.Context\n\n\tstop func() error\n\n\t// Flags\n\tIsOnline bool `optional:\"true\"` // Online is set when networking is enabled.\n\tIsDaemon bool `optional:\"true\"` // Daemon is set when running on a long-running daemon.\n}\n\n// Mounts defines what the node's mount state is. This should\n// perhaps be moved to the daemon or mount. It's here because\n// it needs to be accessible across daemon requests.\ntype Mounts struct {\n\tIpfs mount.Mount\n\tIpns mount.Mount\n\tMfs mount.Mount\n}\n\n// Close calls Close() on the App object\nfunc (n *IpfsNode) Close() error {\n\treturn n.stop()\n}\n\n// HasActiveDHTClient checks if the node's DHT client is active and usable for DHT operations.\n//\n// Returns false for:\n// - nil DHTClient\n// - typed nil pointers (e.g., (*ddht.DHT)(nil))\n// - no-op routers (routinghelpers.Null)\n//\n// Note: This method only checks for known DHT client types (ddht.DHT, fullrt.FullRT).\n// Custom routing.Routing implementations are not explicitly validated.\n//\n// This method prevents the \"typed nil interface\" bug where an interface contains\n// a nil pointer of a concrete type, which passes nil checks but panics when methods\n// are called.\nfunc (n *IpfsNode) HasActiveDHTClient() bool {\n\tif n.DHTClient == nil {\n\t\treturn false\n\t}\n\n\t// Check for no-op router (Routing.Type=none)\n\tif _, ok := n.DHTClient.(routinghelpers.Null); ok {\n\t\treturn false\n\t}\n\n\t// Check for typed nil *ddht.DHT (common when Routing.Type=delegated or HTTP-only)\n\tif d, ok := n.DHTClient.(*ddht.DHT); ok && d == nil {\n\t\treturn false\n\t}\n\n\t// Check for typed nil *fullrt.FullRT (accelerated DHT client)\n\tif f, ok := n.DHTClient.(*fullrt.FullRT); ok && f == nil {\n\t\treturn false\n\t}\n\n\treturn true\n}\n\n// Context returns the IpfsNode context\nfunc (n *IpfsNode) Context() context.Context {\n\tif n.ctx == nil {\n\t\tn.ctx = context.TODO()\n\t}\n\treturn n.ctx\n}\n\n// Bootstrap will set and call the IpfsNodes bootstrap function.\nfunc (n *IpfsNode) Bootstrap(cfg bootstrap.BootstrapConfig) error {\n\t// TODO what should return value be when in offlineMode?\n\tif n.Routing == nil {\n\t\treturn nil\n\t}\n\n\tif n.Bootstrapper != nil {\n\t\tn.Bootstrapper.Close() // stop previous bootstrap process.\n\t}\n\n\t// if the caller did not specify a bootstrap peer function, get the\n\t// freshest bootstrap peers from config. this responds to live changes.\n\tif cfg.BootstrapPeers == nil {\n\t\tcfg.BootstrapPeers = func() []peer.AddrInfo {\n\t\t\tps, err := n.loadBootstrapPeers()\n\t\t\tif err != nil {\n\t\t\t\tlog.Warn(\"failed to parse bootstrap peers from config\")\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn ps\n\t\t}\n\t}\n\tif load, _ := cfg.BackupPeers(); load == nil {\n\t\tsave := func(ctx context.Context, peerList []peer.AddrInfo) {\n\t\t\terr := n.saveTempBootstrapPeers(ctx, peerList)\n\t\t\tif err != nil {\n\t\t\t\tlog.Warnf(\"saveTempBootstrapPeers failed: %s\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t\tload = func(ctx context.Context) []peer.AddrInfo {\n\t\t\tpeerList, err := n.loadTempBootstrapPeers(ctx)\n\t\t\tif err != nil {\n\t\t\t\tlog.Warnf(\"loadTempBootstrapPeers failed: %s\", err)\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn peerList\n\t\t}\n\t\tcfg.SetBackupPeers(load, save)\n\t}\n\n\trepoConf, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn err\n\t}\n\tif repoConf.Internal.BackupBootstrapInterval != nil {\n\t\tcfg.BackupBootstrapInterval = repoConf.Internal.BackupBootstrapInterval.WithDefault(time.Hour)\n\t}\n\n\tn.Bootstrapper, err = bootstrap.Bootstrap(n.Identity, n.PeerHost, n.Routing, cfg)\n\treturn err\n}\n\nvar TempBootstrapPeersKey = datastore.NewKey(\"/local/temp_bootstrap_peers\")\n\nfunc (n *IpfsNode) loadBootstrapPeers() ([]peer.AddrInfo, error) {\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Use auto-config resolution for actual bootstrap connectivity\n\treturn cfg.BootstrapPeersWithAutoConf()\n}\n\nfunc (n *IpfsNode) saveTempBootstrapPeers(ctx context.Context, peerList []peer.AddrInfo) error {\n\tds := n.Repo.Datastore()\n\tbytes, err := json.Marshal(config.BootstrapPeerStrings(peerList))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := ds.Put(ctx, TempBootstrapPeersKey, bytes); err != nil {\n\t\treturn err\n\t}\n\treturn ds.Sync(ctx, TempBootstrapPeersKey)\n}\n\nfunc (n *IpfsNode) loadTempBootstrapPeers(ctx context.Context) ([]peer.AddrInfo, error) {\n\tds := n.Repo.Datastore()\n\tbytes, err := ds.Get(ctx, TempBootstrapPeersKey)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar addrs []string\n\tif err := json.Unmarshal(bytes, &addrs); err != nil {\n\t\treturn nil, err\n\t}\n\treturn config.ParseBootstrapPeers(addrs)\n}\n\ntype ConstructPeerHostOpts struct {\n\tAddrsFactory p2pbhost.AddrsFactory\n\tDisableNatPortMap bool\n\tDisableRelay bool\n\tEnableRelayHop bool\n\tConnectionManager connmgr.ConnManager\n}\n" }, { "path": "core/core_test.go", "content": "package core\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\tcontext \"context\"\n\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"github.com/ipfs/boxo/filestore\"\n\t\"github.com/ipfs/boxo/keystore\"\n\tdatastore \"github.com/ipfs/go-datastore\"\n\tsyncds \"github.com/ipfs/go-datastore/sync\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\tgolib \"github.com/libp2p/go-libp2p\"\n\tddht \"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\nfunc TestInitialization(t *testing.T) {\n\tctx := context.Background()\n\tid := testIdentity\n\n\tgood := []*config.Config{\n\t\t{\n\t\t\tIdentity: id,\n\t\t\tAddresses: config.Addresses{\n\t\t\t\tSwarm: []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/udp/4001/quic-v1\"},\n\t\t\t\tAPI: []string{\"/ip4/127.0.0.1/tcp/8000\"},\n\t\t\t},\n\t\t},\n\n\t\t{\n\t\t\tIdentity: id,\n\t\t\tAddresses: config.Addresses{\n\t\t\t\tSwarm: []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/udp/4001/quic-v1\"},\n\t\t\t\tAPI: []string{\"/ip4/127.0.0.1/tcp/8000\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tbad := []*config.Config{\n\t\t{},\n\t}\n\n\tfor i, c := range good {\n\t\tr := &repo.Mock{\n\t\t\tC: *c,\n\t\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t\t}\n\t\tn, err := NewNode(ctx, &BuildCfg{Repo: r})\n\t\tif n == nil || err != nil {\n\t\t\tt.Error(\"Should have constructed.\", i, err)\n\t\t}\n\t}\n\n\tfor i, c := range bad {\n\t\tr := &repo.Mock{\n\t\t\tC: *c,\n\t\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t\t}\n\t\tn, err := NewNode(ctx, &BuildCfg{Repo: r})\n\t\tif n != nil || err == nil {\n\t\t\tt.Error(\"Should have failed to construct.\", i)\n\t\t}\n\t}\n}\n\nvar testIdentity = config.Identity{\n\tPeerID: \"QmNgdzLieYi8tgfo2WfTUzNVH5hQK9oAYGVf6dxN12NrHt\",\n\tPrivKey: \"CAASrRIwggkpAgEAAoICAQCwt67GTUQ8nlJhks6CgbLKOx7F5tl1r9zF4m3TUrG3Pe8h64vi+ILDRFd7QJxaJ/n8ux9RUDoxLjzftL4uTdtv5UXl2vaufCc/C0bhCRvDhuWPhVsD75/DZPbwLsepxocwVWTyq7/ZHsCfuWdoh/KNczfy+Gn33gVQbHCnip/uhTVxT7ARTiv8Qa3d7qmmxsR+1zdL/IRO0mic/iojcb3Oc/PRnYBTiAZFbZdUEit/99tnfSjMDg02wRayZaT5ikxa6gBTMZ16Yvienq7RwSELzMQq2jFA4i/TdiGhS9uKywltiN2LrNDBcQJSN02pK12DKoiIy+wuOCRgs2NTQEhU2sXCk091v7giTTOpFX2ij9ghmiRfoSiBFPJA5RGwiH6ansCHtWKY1K8BS5UORM0o3dYk87mTnKbCsdz4bYnGtOWafujYwzueGx8r+IWiys80IPQKDeehnLW6RgoyjszKgL/2XTyP54xMLSW+Qb3BPgDcPaPO0hmop1hW9upStxKsefW2A2d46Ds4HEpJEry7PkS5M4gKL/zCKHuxuXVk14+fZQ1rstMuvKjrekpAC2aVIKMI9VRA3awtnje8HImQMdj+r+bPmv0N8rTTr3eS4J8Yl7k12i95LLfK+fWnmUh22oTNzkRlaiERQrUDyE4XNCtJc0xs1oe1yXGqazCIAQIDAQABAoICAQCk1N/ftahlRmOfAXk//8wNl7FvdJD3le6+YSKBj0uWmN1ZbUSQk64chr12iGCOM2WY180xYjy1LOS44PTXaeW5bEiTSnb3b3SH+HPHaWCNM2EiSogHltYVQjKW+3tfH39vlOdQ9uQ+l9Gh6iTLOqsCRyszpYPqIBwi1NMLY2Ej8PpVU7ftnFWouHZ9YKS7nAEiMoowhTu/7cCIVwZlAy3AySTuKxPMVj9LORqC32PVvBHZaMPJ+X1Xyijqg6aq39WyoztkXg3+Xxx5j5eOrK6vO/Lp6ZUxaQilHDXoJkKEJjgIBDZpluss08UPfOgiWAGkW+L4fgUxY0qDLDAEMhyEBAn6KOKVL1JhGTX6GjhWziI94bddSpHKYOEIDzUy4H8BXnKhtnyQV6ELS65C2hj9D0IMBTj7edCF1poJy0QfdK0cuXgMvxHLeUO5uc2YWfbNosvKxqygB9rToy4b22YvNwsZUXsTY6Jt+p9V2OgXSKfB5VPeRbjTJL6xqvvUJpQytmII/C9JmSDUtCbYceHj6X9jgigLk20VV6nWHqCTj3utXD6NPAjoycVpLKDlnWEgfVELDIk0gobxUqqSm3jTPEKRPJgxkgPxbwxYumtw++1UY2y35w3WRDc2xYPaWKBCQeZy+mL6ByXp9bWlNvxS3Knb6oZp36/ovGnf2pGvdQKCAQEAyKpipz2lIUySDyE0avVWAmQb2tWGKXALPohzj7AwkcfEg2GuwoC6GyVE2sTJD1HRazIjOKn3yQORg2uOPeG7sx7EKHxSxCKDrbPawkvLCq8JYSy9TLvhqKUVVGYPqMBzu2POSLEA81QXas+aYjKOFWA2Zrjq26zV9ey3+6Lc6WULePgRQybU8+RHJc6fdjUCCfUxgOrUO2IQOuTJ+FsDpVnrMUGlokmWn23OjL4qTL9wGDnWGUs2pjSzNbj3qA0d8iqaiMUyHX/D/VS0wpeT1osNBSm8suvSibYBn+7wbIApbwXUxZaxMv2OHGz3empae4ckvNZs7r8wsI9UwFt8mwKCAQEA4XK6gZkv9t+3YCcSPw2ensLvL/xU7i2bkC9tfTGdjnQfzZXIf5KNdVuj/SerOl2S1s45NMs3ysJbADwRb4ahElD/V71nGzV8fpFTitC20ro9fuX4J0+twmBolHqeH9pmeGTjAeL1rvt6vxs4FkeG/yNft7GdXpXTtEGaObn8Mt0tPY+aB3UnKrnCQoQAlPyGHFrVRX0UEcp6wyyNGhJCNKeNOvqCHTFObhbhO+KWpWSN0MkVHnqaIBnIn1Te8FtvP/iTwXGnKc0YXJUG6+LM6LmOguW6tg8ZqiQeYyyR+e9eCFH4csLzkrTl1GxCxwEsoSLIMm7UDcjttW6tYEghkwKCAQEAmeCO5lCPYImnN5Lu71ZTLmI2OgmjaANTnBBnDbi+hgv61gUCToUIMejSdDCTPfwv61P3TmyIZs0luPGxkiKYHTNqmOE9Vspgz8Mr7fLRMNApESuNvloVIY32XVImj/GEzh4rAfM6F15U1sN8T/EUo6+0B/Glp+9R49QzAfRSE2g48/rGwgf1JVHYfVWFUtAzUA+GdqWdOixo5cCsYJbqpNHfWVZN/bUQnBFIYwUwysnC29D+LUdQEQQ4qOm+gFAOtrWU62zMkXJ4iLt8Ify6kbrvsRXgbhQIzzGS7WH9XDarj0eZciuslr15TLMC1Azadf+cXHLR9gMHA13mT9vYIQKCAQA/DjGv8cKCkAvf7s2hqROGYAs6Jp8yhrsN1tYOwAPLRhtnCs+rLrg17M2vDptLlcRuI/vIElamdTmylRpjUQpX7yObzLO73nfVhpwRJVMdGU394iBIDncQ+JoHfUwgqJskbUM40dvZdyjbrqc/Q/4z+hbZb+oN/GXb8sVKBATPzSDMKQ/xqgisYIw+wmDPStnPsHAaIWOtni47zIgilJzD0WEk78/YjmPbUrboYvWziK5JiRRJFA1rkQqV1c0M+OXixIm+/yS8AksgCeaHr0WUieGcJtjT9uE8vyFop5ykhRiNxy9wGaq6i7IEecsrkd6DqxDHWkwhFuO1bSE83q/VAoIBAEA+RX1i/SUi08p71ggUi9WFMqXmzELp1L3hiEjOc2AklHk2rPxsaTh9+G95BvjhP7fRa/Yga+yDtYuyjO99nedStdNNSg03aPXILl9gs3r2dPiQKUEXZJ3FrH6tkils/8BlpOIRfbkszrdZIKTO9GCdLWQ30dQITDACs8zV/1GFGrHFrqnnMe/NpIFHWNZJ0/WZMi8wgWO6Ik8jHEpQtVXRiXLqy7U6hk170pa4GHOzvftfPElOZZjy9qn7KjdAQqy6spIrAE94OEL+fBgbHQZGLpuTlj6w6YGbMtPU8uo7sXKoc6WOCb68JWft3tejGLDa1946HAWqVM9B/UcneNc=\",\n}\n\n// mockHostOption creates a HostOption that uses the provided mocknet.\n// Inlined to avoid import cycle with core/mock package.\nfunc mockHostOption(mn mocknet.Mocknet) libp2p.HostOption {\n\treturn func(id peer.ID, ps pstore.Peerstore, opts ...golib.Option) (host.Host, error) {\n\t\tvar cfg golib.Config\n\t\tif err := cfg.Apply(opts...); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// The mocknet does not use the provided libp2p.Option. This options include\n\t\t// the listening addresses we want our peer listening on. Therefore, we have\n\t\t// to manually parse the configuration and add them here.\n\t\tps.AddAddrs(id, cfg.ListenAddrs, pstore.PermanentAddrTTL)\n\t\treturn mn.AddPeerWithPeerstore(id, ps)\n\t}\n}\n\nfunc TestHasActiveDHTClient(t *testing.T) {\n\t// Test 1: nil DHTClient\n\tt.Run(\"nil DHTClient\", func(t *testing.T) {\n\t\tnode := &IpfsNode{\n\t\t\tDHTClient: nil,\n\t\t}\n\t\tif node.HasActiveDHTClient() {\n\t\t\tt.Error(\"Expected false for nil DHTClient\")\n\t\t}\n\t})\n\n\t// Test 2: Typed nil *ddht.DHT (common case when Routing.Type=delegated)\n\tt.Run(\"typed nil ddht.DHT\", func(t *testing.T) {\n\t\tnode := &IpfsNode{\n\t\t\tDHTClient: (*ddht.DHT)(nil),\n\t\t}\n\t\tif node.HasActiveDHTClient() {\n\t\t\tt.Error(\"Expected false for typed nil *ddht.DHT\")\n\t\t}\n\t})\n\n\t// Test 3: Typed nil *fullrt.FullRT (accelerated DHT client)\n\tt.Run(\"typed nil fullrt.FullRT\", func(t *testing.T) {\n\t\tnode := &IpfsNode{\n\t\t\tDHTClient: (*fullrt.FullRT)(nil),\n\t\t}\n\t\tif node.HasActiveDHTClient() {\n\t\t\tt.Error(\"Expected false for typed nil *fullrt.FullRT\")\n\t\t}\n\t})\n\n\t// Test 4: routinghelpers.Null no-op router (Routing.Type=none)\n\tt.Run(\"routinghelpers.Null\", func(t *testing.T) {\n\t\tnode := &IpfsNode{\n\t\t\tDHTClient: routinghelpers.Null{},\n\t\t}\n\t\tif node.HasActiveDHTClient() {\n\t\t\tt.Error(\"Expected false for routinghelpers.Null\")\n\t\t}\n\t})\n\n\t// Test 5: Valid standard dual DHT (Routing.Type=auto/dht/dhtclient)\n\tt.Run(\"valid standard dual DHT\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\tmn := mocknet.New()\n\t\tdefer mn.Close()\n\n\t\tds := syncds.MutexWrap(datastore.NewMapDatastore())\n\t\tc := config.Config{}\n\t\tc.Identity = testIdentity\n\t\tc.Addresses.Swarm = []string{\"/ip4/0.0.0.0/tcp/4001\"}\n\n\t\tr := &repo.Mock{\n\t\t\tC: c,\n\t\t\tD: ds,\n\t\t\tK: keystore.NewMemKeystore(),\n\t\t\tF: filestore.NewFileManager(ds, filepath.Dir(os.TempDir())),\n\t\t}\n\n\t\tnode, err := NewNode(ctx, &BuildCfg{\n\t\t\tRouting: libp2p.DHTServerOption,\n\t\t\tRepo: r,\n\t\t\tHost: mockHostOption(mn),\n\t\t\tOnline: true,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create node with DHT: %v\", err)\n\t\t}\n\t\tdefer node.Close()\n\n\t\t// First verify test setup created the expected DHT type\n\t\tif node.DHTClient == nil {\n\t\t\tt.Fatalf(\"Test setup failed: DHTClient is nil\")\n\t\t}\n\n\t\tif _, ok := node.DHTClient.(*ddht.DHT); !ok {\n\t\t\tt.Fatalf(\"Test setup failed: expected DHTClient to be *ddht.DHT, got %T\", node.DHTClient)\n\t\t}\n\n\t\t// Now verify HasActiveDHTClient() correctly identifies it as active\n\t\tif !node.HasActiveDHTClient() {\n\t\t\tt.Error(\"Expected true for valid dual DHT client\")\n\t\t}\n\t})\n\n\t// Test 6: Valid accelerated DHT client (Routing.Type=autoclient)\n\tt.Run(\"valid accelerated DHT client\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\tmn := mocknet.New()\n\t\tdefer mn.Close()\n\n\t\tds := syncds.MutexWrap(datastore.NewMapDatastore())\n\t\tc := config.Config{}\n\t\tc.Identity = testIdentity\n\t\tc.Addresses.Swarm = []string{\"/ip4/0.0.0.0/tcp/4001\"}\n\t\tc.Routing.AcceleratedDHTClient = config.True\n\n\t\tr := &repo.Mock{\n\t\t\tC: c,\n\t\t\tD: ds,\n\t\t\tK: keystore.NewMemKeystore(),\n\t\t\tF: filestore.NewFileManager(ds, filepath.Dir(os.TempDir())),\n\t\t}\n\n\t\tnode, err := NewNode(ctx, &BuildCfg{\n\t\t\tRouting: libp2p.DHTOption,\n\t\t\tRepo: r,\n\t\t\tHost: mockHostOption(mn),\n\t\t\tOnline: true,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create node with accelerated DHT: %v\", err)\n\t\t}\n\t\tdefer node.Close()\n\n\t\t// First verify test setup created the expected accelerated DHT type\n\t\tif node.DHTClient == nil {\n\t\t\tt.Fatalf(\"Test setup failed: DHTClient is nil\")\n\t\t}\n\n\t\tif _, ok := node.DHTClient.(*fullrt.FullRT); !ok {\n\t\t\tt.Fatalf(\"Test setup failed: expected DHTClient to be *fullrt.FullRT, got %T\", node.DHTClient)\n\t\t}\n\n\t\t// Now verify HasActiveDHTClient() correctly identifies it as active\n\t\tif !node.HasActiveDHTClient() {\n\t\t\tt.Error(\"Expected true for valid accelerated DHT client\")\n\t\t}\n\t})\n}\n" }, { "path": "core/coreapi/block.go", "content": "package coreapi\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\tblocks \"github.com/ipfs/go-block-format\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n\n\tutil \"github.com/ipfs/kubo/blocks/blockstoreutil\"\n\t\"github.com/ipfs/kubo/tracing\"\n)\n\ntype BlockAPI CoreAPI\n\ntype BlockStat struct {\n\tpath path.ImmutablePath\n\tsize int\n}\n\nfunc (api *BlockAPI) Put(ctx context.Context, src io.Reader, opts ...caopts.BlockPutOption) (coreiface.BlockStat, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.BlockAPI\", \"Put\")\n\tdefer span.End()\n\n\tsettings, err := caopts.BlockPutOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdata, err := io.ReadAll(src)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tbcid, err := settings.CidPrefix.Sum(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tb, err := blocks.NewBlockWithCid(data, bcid)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif settings.Pin {\n\t\tdefer api.blockstore.PinLock(ctx).Unlock(ctx)\n\t}\n\n\terr = api.blocks.AddBlock(ctx, b)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif settings.Pin {\n\t\tif err = api.pinning.PinWithMode(ctx, b.Cid(), pin.Recursive, \"\"); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := api.pinning.Flush(ctx); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn &BlockStat{path: path.FromCid(b.Cid()), size: len(data)}, nil\n}\n\nfunc (api *BlockAPI) Get(ctx context.Context, p path.Path) (io.Reader, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.BlockAPI\", \"Get\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tb, err := api.blocks.GetBlock(ctx, rp.RootCid())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn bytes.NewReader(b.RawData()), nil\n}\n\nfunc (api *BlockAPI) Rm(ctx context.Context, p path.Path, opts ...caopts.BlockRmOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.BlockAPI\", \"Rm\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tsettings, err := caopts.BlockRmOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\tcids := []cid.Cid{rp.RootCid()}\n\to := util.RmBlocksOpts{Force: settings.Force}\n\n\tout, err := util.RmBlocks(ctx, api.blockstore, api.pinning, cids, o)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tselect {\n\tcase res, ok := <-out:\n\t\tif !ok {\n\t\t\treturn nil\n\t\t}\n\n\t\tremBlock, ok := res.(*util.RemovedBlock)\n\t\tif !ok {\n\t\t\treturn errors.New(\"got unexpected output from util.RmBlocks\")\n\t\t}\n\n\t\tif remBlock.Error != nil {\n\t\t\treturn remBlock.Error\n\t\t}\n\t\treturn nil\n\tcase <-ctx.Done():\n\t\treturn ctx.Err()\n\t}\n}\n\nfunc (api *BlockAPI) Stat(ctx context.Context, p path.Path) (coreiface.BlockStat, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.BlockAPI\", \"Stat\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tb, err := api.blocks.GetBlock(ctx, rp.RootCid())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &BlockStat{\n\t\tpath: path.FromCid(b.Cid()),\n\t\tsize: len(b.RawData()),\n\t}, nil\n}\n\nfunc (bs *BlockStat) Size() int {\n\treturn bs.size\n}\n\nfunc (bs *BlockStat) Path() path.ImmutablePath {\n\treturn bs.path\n}\n\nfunc (api *BlockAPI) core() coreiface.CoreAPI {\n\treturn (*CoreAPI)(api)\n}\n" }, { "path": "core/coreapi/coreapi.go", "content": "/*\n**NOTE: this package is experimental.**\n\nPackage coreapi provides direct access to the core commands in IPFS. If you are\nembedding IPFS directly in your Go program, this package is the public\ninterface you should use to read and write files or otherwise control IPFS.\n\nIf you are running IPFS as a separate process, you should use `client/rpc` to\nwork with it via HTTP.\n*/\npackage coreapi\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\n\tbserv \"github.com/ipfs/boxo/blockservice\"\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\texchange \"github.com/ipfs/boxo/exchange\"\n\tofflinexch \"github.com/ipfs/boxo/exchange/offline\"\n\t\"github.com/ipfs/boxo/fetcher\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tpathresolver \"github.com/ipfs/boxo/path/resolver\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\tofflineroute \"github.com/ipfs/boxo/routing/offline\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\t\"github.com/ipfs/kubo/config\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\tpubsub \"github.com/libp2p/go-libp2p-pubsub\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\tci \"github.com/libp2p/go-libp2p/core/crypto\"\n\tp2phost \"github.com/libp2p/go-libp2p/core/host\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/node\"\n\t\"github.com/ipfs/kubo/repo\"\n)\n\ntype CoreAPI struct {\n\tnctx context.Context\n\n\tidentity peer.ID\n\tprivateKey ci.PrivKey\n\n\trepo repo.Repo\n\tblockstore blockstore.GCBlockstore\n\tbaseBlocks blockstore.Blockstore\n\tpinning pin.Pinner\n\n\tblocks bserv.BlockService\n\tdag ipld.DAGService\n\tipldFetcherFactory fetcher.Factory\n\tunixFSFetcherFactory fetcher.Factory\n\tpeerstore pstore.Peerstore\n\tpeerHost p2phost.Host\n\trecordValidator record.Validator\n\texchange exchange.Interface\n\n\tnamesys namesys.NameSystem\n\trouting routing.Routing\n\tdnsResolver *madns.Resolver\n\tipldPathResolver pathresolver.Resolver\n\tunixFSPathResolver pathresolver.Resolver\n\n\tprovider node.DHTProvider\n\tprovidingStrategy config.ProvideStrategy\n\n\tpubSub *pubsub.PubSub\n\n\tcheckPublishAllowed func() error\n\tcheckOnline func(allowOffline bool) error\n\n\t// ONLY for re-applying options in WithOptions, DO NOT USE ANYWHERE ELSE\n\tnd *core.IpfsNode\n\tparentOpts options.ApiSettings\n}\n\n// NewCoreAPI creates new instance of IPFS CoreAPI backed by go-ipfs Node.\nfunc NewCoreAPI(n *core.IpfsNode, opts ...options.ApiOption) (coreiface.CoreAPI, error) {\n\tparentOpts, err := options.ApiOptions()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn (&CoreAPI{nd: n, parentOpts: *parentOpts}).WithOptions(opts...)\n}\n\n// Unixfs returns the UnixfsAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Unixfs() coreiface.UnixfsAPI {\n\treturn (*UnixfsAPI)(api)\n}\n\n// Block returns the BlockAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Block() coreiface.BlockAPI {\n\treturn (*BlockAPI)(api)\n}\n\n// Dag returns the DagAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Dag() coreiface.APIDagService {\n\treturn &dagAPI{\n\t\tapi.dag,\n\t\tapi,\n\t}\n}\n\n// Name returns the NameAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Name() coreiface.NameAPI {\n\treturn (*NameAPI)(api)\n}\n\n// Key returns the KeyAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Key() coreiface.KeyAPI {\n\treturn (*KeyAPI)(api)\n}\n\n// Object returns the ObjectAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Object() coreiface.ObjectAPI {\n\treturn (*ObjectAPI)(api)\n}\n\n// Pin returns the PinAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Pin() coreiface.PinAPI {\n\treturn (*PinAPI)(api)\n}\n\n// Swarm returns the SwarmAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) Swarm() coreiface.SwarmAPI {\n\treturn (*SwarmAPI)(api)\n}\n\n// PubSub returns the PubSubAPI interface implementation backed by the go-ipfs node\nfunc (api *CoreAPI) PubSub() coreiface.PubSubAPI {\n\treturn (*PubSubAPI)(api)\n}\n\n// Routing returns the RoutingAPI interface implementation backed by the kubo node\nfunc (api *CoreAPI) Routing() coreiface.RoutingAPI {\n\treturn (*RoutingAPI)(api)\n}\n\n// WithOptions returns api with global options applied\nfunc (api *CoreAPI) WithOptions(opts ...options.ApiOption) (coreiface.CoreAPI, error) {\n\tsettings := api.parentOpts // make sure to copy\n\t_, err := options.ApiOptionsTo(&settings, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif api.nd == nil {\n\t\treturn nil, errors.New(\"cannot apply options to api without node\")\n\t}\n\n\tn := api.nd\n\n\tsubAPI := &CoreAPI{\n\t\tnctx: n.Context(),\n\n\t\tidentity: n.Identity,\n\t\tprivateKey: n.PrivateKey,\n\n\t\trepo: n.Repo,\n\t\tblockstore: n.Blockstore,\n\t\tbaseBlocks: n.BaseBlocks,\n\t\tpinning: n.Pinning,\n\n\t\tblocks: n.Blocks,\n\t\tdag: n.DAG,\n\t\tipldFetcherFactory: n.IPLDFetcherFactory,\n\t\tunixFSFetcherFactory: n.UnixFSFetcherFactory,\n\n\t\tpeerstore: n.Peerstore,\n\t\tpeerHost: n.PeerHost,\n\t\tnamesys: n.Namesys,\n\t\trecordValidator: n.RecordValidator,\n\t\texchange: n.Exchange,\n\t\trouting: n.Routing,\n\t\tdnsResolver: n.DNSResolver,\n\t\tipldPathResolver: n.IPLDPathResolver,\n\t\tunixFSPathResolver: n.UnixFSPathResolver,\n\n\t\tprovider: n.Provider,\n\t\tprovidingStrategy: n.ProvidingStrategy,\n\n\t\tpubSub: n.PubSub,\n\n\t\tnd: n,\n\t\tparentOpts: settings,\n\t}\n\n\tsubAPI.checkOnline = func(allowOffline bool) error {\n\t\tif !n.IsOnline && !allowOffline {\n\t\t\treturn coreiface.ErrOffline\n\t\t}\n\t\treturn nil\n\t}\n\n\tsubAPI.checkPublishAllowed = func() error {\n\t\tif n.Mounts.Ipns != nil && n.Mounts.Ipns.IsActive() {\n\t\t\treturn errors.New(\"cannot manually publish while IPNS is mounted\")\n\t\t}\n\t\treturn nil\n\t}\n\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif settings.Offline {\n\t\tcs := cfg.Ipns.ResolveCacheSize\n\t\tif cs == 0 {\n\t\t\tcs = node.DefaultIpnsCacheSize\n\t\t}\n\t\tif cs < 0 {\n\t\t\treturn nil, errors.New(\"cannot specify negative resolve cache size\")\n\t\t}\n\n\t\tnsOptions := []namesys.Option{\n\t\t\tnamesys.WithDatastore(subAPI.repo.Datastore()),\n\t\t\tnamesys.WithDNSResolver(subAPI.dnsResolver),\n\t\t\tnamesys.WithCache(cs),\n\t\t\tnamesys.WithMaxCacheTTL(cfg.Ipns.MaxCacheTTL.WithDefault(config.DefaultIpnsMaxCacheTTL)),\n\t\t}\n\n\t\tsubAPI.routing = offlineroute.NewOfflineRouter(subAPI.repo.Datastore(), subAPI.recordValidator)\n\n\t\tsubAPI.namesys, err = namesys.NewNameSystem(subAPI.routing, nsOptions...)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error constructing namesys: %w\", err)\n\t\t}\n\n\t\tsubAPI.peerstore = nil\n\t\tsubAPI.peerHost = nil\n\t\tsubAPI.recordValidator = nil\n\t}\n\n\tif settings.Offline || !settings.FetchBlocks {\n\t\tsubAPI.exchange = offlinexch.Exchange(subAPI.blockstore)\n\t\tsubAPI.blocks = bserv.New(subAPI.blockstore, subAPI.exchange,\n\t\t\tbserv.WriteThrough(cfg.Datastore.WriteThrough.WithDefault(config.DefaultWriteThrough)),\n\t\t)\n\t\tsubAPI.dag = dag.NewDAGService(subAPI.blocks)\n\t}\n\n\treturn subAPI, nil\n}\n\n// getSession returns new api backed by the same node with a read-only session DAG\nfunc (api *CoreAPI) getSession(ctx context.Context) *CoreAPI {\n\tsesAPI := *api\n\n\t// TODO: We could also apply this to api.blocks, and compose into writable api,\n\t// but this requires some changes in blockservice/merkledag\n\tsesAPI.dag = dag.NewReadOnlyDagService(dag.NewSession(ctx, api.dag))\n\n\treturn &sesAPI\n}\n" }, { "path": "core/coreapi/dag.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\tcid \"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n\n\t\"github.com/ipfs/kubo/tracing\"\n)\n\ntype dagAPI struct {\n\tipld.DAGService\n\n\tcore *CoreAPI\n}\n\ntype pinningAdder CoreAPI\n\nfunc (adder *pinningAdder) Add(ctx context.Context, nd ipld.Node) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinningAdder\", \"Add\", trace.WithAttributes(attribute.String(\"node\", nd.String())))\n\tdefer span.End()\n\tdefer adder.blockstore.PinLock(ctx).Unlock(ctx)\n\n\tif err := adder.dag.Add(ctx, nd); err != nil {\n\t\treturn err\n\t}\n\n\tif err := adder.pinning.PinWithMode(ctx, nd.Cid(), pin.Recursive, \"\"); err != nil {\n\t\treturn err\n\t}\n\n\treturn adder.pinning.Flush(ctx)\n}\n\nfunc (adder *pinningAdder) AddMany(ctx context.Context, nds []ipld.Node) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinningAdder\", \"AddMany\", trace.WithAttributes(attribute.Int(\"nodes.count\", len(nds))))\n\tdefer span.End()\n\tdefer adder.blockstore.PinLock(ctx).Unlock(ctx)\n\n\tif err := adder.dag.AddMany(ctx, nds); err != nil {\n\t\treturn err\n\t}\n\n\tcids := cid.NewSet()\n\n\tfor _, nd := range nds {\n\t\tc := nd.Cid()\n\t\tif cids.Visit(c) {\n\t\t\tif err := adder.pinning.PinWithMode(ctx, c, pin.Recursive, \"\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn adder.pinning.Flush(ctx)\n}\n\nfunc (api *dagAPI) Pinning() ipld.NodeAdder {\n\treturn (*pinningAdder)(api.core)\n}\n\nfunc (api *dagAPI) Session(ctx context.Context) ipld.NodeGetter {\n\treturn dag.NewSession(ctx, api.DAGService)\n}\n\nvar (\n\t_ ipld.DAGService = (*dagAPI)(nil)\n\t_ dag.SessionMaker = (*dagAPI)(nil)\n)\n" }, { "path": "core/coreapi/key.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"crypto/rand\"\n\t\"errors\"\n\t\"fmt\"\n\t\"sort\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/path\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/tracing\"\n\tcrypto \"github.com/libp2p/go-libp2p/core/crypto\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n)\n\ntype KeyAPI CoreAPI\n\ntype key struct {\n\tname string\n\tpeerID peer.ID\n\tpath path.Path\n}\n\nfunc newKey(name string, pid peer.ID) (*key, error) {\n\tp, err := path.NewPath(\"/ipns/\" + ipns.NameFromPeer(pid).String())\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"cannot create new key: %w\", err)\n\t}\n\treturn &key{\n\t\tname: name,\n\t\tpeerID: pid,\n\t\tpath: p,\n\t}, nil\n}\n\n// Name returns the key name\nfunc (k *key) Name() string {\n\treturn k.name\n}\n\n// Path returns the path of the key.\nfunc (k *key) Path() path.Path {\n\treturn k.path\n}\n\n// ID returns key PeerID\nfunc (k *key) ID() peer.ID {\n\treturn k.peerID\n}\n\n// Generate generates new key, stores it in the keystore under the specified\n// name and returns a base58 encoded multihash of its public key.\nfunc (api *KeyAPI) Generate(ctx context.Context, name string, opts ...caopts.KeyGenerateOption) (coreiface.Key, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.KeyAPI\", \"Generate\", trace.WithAttributes(attribute.String(\"name\", name)))\n\tdefer span.End()\n\n\toptions, err := caopts.KeyGenerateOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif name == \"self\" {\n\t\treturn nil, errors.New(\"cannot create key with name 'self'\")\n\t}\n\n\t_, err = api.repo.Keystore().Get(name)\n\tif err == nil {\n\t\treturn nil, fmt.Errorf(\"key with name '%s' already exists\", name)\n\t}\n\n\tvar sk crypto.PrivKey\n\tvar pk crypto.PubKey\n\n\tswitch options.Algorithm {\n\tcase \"rsa\":\n\t\tif options.Size == -1 {\n\t\t\toptions.Size = caopts.DefaultRSALen\n\t\t}\n\n\t\tpriv, pub, err := crypto.GenerateKeyPairWithReader(crypto.RSA, options.Size, rand.Reader)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tsk = priv\n\t\tpk = pub\n\tcase \"ed25519\":\n\t\tpriv, pub, err := crypto.GenerateEd25519Key(rand.Reader)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tsk = priv\n\t\tpk = pub\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unrecognized key type: %s\", options.Algorithm)\n\t}\n\n\terr = api.repo.Keystore().Put(name, sk)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpid, err := peer.IDFromPublicKey(pk)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn newKey(name, pid)\n}\n\n// List returns a list keys stored in keystore.\nfunc (api *KeyAPI) List(ctx context.Context) ([]coreiface.Key, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.KeyAPI\", \"List\")\n\tdefer span.End()\n\n\tkeys, err := api.repo.Keystore().List()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"cannot list keys in keystore: %w\", err)\n\t}\n\n\tsort.Strings(keys)\n\n\tout := make([]coreiface.Key, 1, len(keys)+1)\n\tout[0], err = newKey(\"self\", api.identity)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, k := range keys {\n\t\tprivKey, err := api.repo.Keystore().Get(k)\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"cannot get key from keystore: %s\", err)\n\t\t\tcontinue\n\t\t}\n\n\t\tpubKey := privKey.GetPublic()\n\n\t\tpid, err := peer.IDFromPublicKey(pubKey)\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"cannot decode public key: %s\", err)\n\t\t\tcontinue\n\t\t}\n\n\t\tk, err := newKey(k, pid)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tout = append(out, k)\n\t}\n\treturn out, nil\n}\n\n// Rename renames `oldName` to `newName`. Returns the key and whether another\n// key was overwritten, or an error.\nfunc (api *KeyAPI) Rename(ctx context.Context, oldName string, newName string, opts ...caopts.KeyRenameOption) (coreiface.Key, bool, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.KeyAPI\", \"Rename\", trace.WithAttributes(attribute.String(\"oldname\", oldName), attribute.String(\"newname\", newName)))\n\tdefer span.End()\n\n\toptions, err := caopts.KeyRenameOptions(opts...)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\tspan.SetAttributes(attribute.Bool(\"force\", options.Force))\n\n\tks := api.repo.Keystore()\n\n\tif oldName == \"self\" {\n\t\treturn nil, false, errors.New(\"cannot rename key with name 'self'\")\n\t}\n\n\tif newName == \"self\" {\n\t\treturn nil, false, errors.New(\"cannot overwrite key with name 'self'\")\n\t}\n\n\toldKey, err := ks.Get(oldName)\n\tif err != nil {\n\t\treturn nil, false, fmt.Errorf(\"no key named %s was found\", oldName)\n\t}\n\n\tpubKey := oldKey.GetPublic()\n\n\tpid, err := peer.IDFromPublicKey(pubKey)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\t// This is important, because future code will delete key `oldName`\n\t// even if it is the same as newName.\n\tif newName == oldName {\n\t\tk, err := newKey(oldName, pid)\n\t\treturn k, false, err\n\t}\n\n\toverwrite := false\n\tif options.Force {\n\t\texist, err := ks.Has(newName)\n\t\tif err != nil {\n\t\t\treturn nil, false, err\n\t\t}\n\n\t\tif exist {\n\t\t\toverwrite = true\n\t\t\terr := ks.Delete(newName)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, false, err\n\t\t\t}\n\t\t}\n\t}\n\n\terr = ks.Put(newName, oldKey)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\terr = ks.Delete(oldName)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\tk, err := newKey(newName, pid)\n\treturn k, overwrite, err\n}\n\n// Remove removes keys from keystore. Returns ipns path of the removed key.\nfunc (api *KeyAPI) Remove(ctx context.Context, name string) (coreiface.Key, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.KeyAPI\", \"Remove\", trace.WithAttributes(attribute.String(\"name\", name)))\n\tdefer span.End()\n\n\tks := api.repo.Keystore()\n\n\tif name == \"self\" {\n\t\treturn nil, errors.New(\"cannot remove key with name 'self'\")\n\t}\n\n\tremoved, err := ks.Get(name)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"no key named %s was found\", name)\n\t}\n\n\tpubKey := removed.GetPublic()\n\n\tpid, err := peer.IDFromPublicKey(pubKey)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = ks.Delete(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn newKey(\"\", pid)\n}\n\nfunc (api *KeyAPI) Self(ctx context.Context) (coreiface.Key, error) {\n\tif api.identity == \"\" {\n\t\treturn nil, errors.New(\"identity not loaded\")\n\t}\n\n\treturn newKey(\"self\", api.identity)\n}\n\nconst signedMessagePrefix = \"libp2p-key signed message:\"\n\nfunc (api *KeyAPI) Sign(ctx context.Context, name string, data []byte) (coreiface.Key, []byte, error) {\n\tvar (\n\t\tsk crypto.PrivKey\n\t\terr error\n\t)\n\tif name == \"\" || name == \"self\" {\n\t\tname = \"self\"\n\t\tsk = api.privateKey\n\t} else {\n\t\tsk, err = api.repo.Keystore().Get(name)\n\t}\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tpid, err := peer.IDFromPrivateKey(sk)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tkey, err := newKey(name, pid)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tdata = append([]byte(signedMessagePrefix), data...)\n\n\tsig, err := sk.Sign(data)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\treturn key, sig, nil\n}\n\nfunc (api *KeyAPI) Verify(ctx context.Context, keyOrName string, signature, data []byte) (coreiface.Key, bool, error) {\n\tvar (\n\t\tname string\n\t\tpk crypto.PubKey\n\t\terr error\n\t)\n\tif keyOrName == \"\" || keyOrName == \"self\" {\n\t\tname = \"self\"\n\t\tpk = api.privateKey.GetPublic()\n\t} else if sk, err := api.repo.Keystore().Get(keyOrName); err == nil {\n\t\tname = keyOrName\n\t\tpk = sk.GetPublic()\n\t} else if ipnsName, err := ipns.NameFromString(keyOrName); err == nil {\n\t\t// This works for both IPNS names and Peer IDs.\n\t\tname = \"\"\n\t\tpk, err = ipnsName.Peer().ExtractPublicKey()\n\t\tif err != nil {\n\t\t\treturn nil, false, err\n\t\t}\n\t} else {\n\t\treturn nil, false, fmt.Errorf(\"'%q' is not a known key, an IPNS Name, or a valid PeerID\", keyOrName)\n\t}\n\n\tpid, err := peer.IDFromPublicKey(pk)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\tkey, err := newKey(name, pid)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\tdata = append([]byte(signedMessagePrefix), data...)\n\n\tvalid, err := pk.Verify(data, signature)\n\tif err != nil {\n\t\treturn nil, false, err\n\t}\n\n\treturn key, valid, nil\n}\n" }, { "path": "core/coreapi/name.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\tkeystore \"github.com/ipfs/boxo/keystore\"\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/kubo/tracing\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\tci \"github.com/libp2p/go-libp2p/core/crypto\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n)\n\ntype NameAPI CoreAPI\n\n// Publish announces new IPNS name and returns the new IPNS entry.\nfunc (api *NameAPI) Publish(ctx context.Context, p path.Path, opts ...caopts.NamePublishOption) (ipns.Name, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.NameAPI\", \"Publish\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tif err := api.checkPublishAllowed(); err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\n\toptions, err := caopts.NamePublishOptions(opts...)\n\tif err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\tspan.SetAttributes(\n\t\tattribute.Bool(\"allowoffline\", options.AllowOffline),\n\t\tattribute.String(\"key\", options.Key),\n\t\tattribute.Float64(\"validtime\", options.ValidTime.Seconds()),\n\t)\n\tif options.TTL != nil {\n\t\tspan.SetAttributes(attribute.Float64(\"ttl\", options.TTL.Seconds()))\n\t}\n\n\t// Handle different publishing modes\n\tif options.AllowDelegated {\n\t\t// AllowDelegated mode: check if delegated publishers are configured\n\t\tcfg, err := api.repo.Config()\n\t\tif err != nil {\n\t\t\treturn ipns.Name{}, fmt.Errorf(\"failed to read config: %w\", err)\n\t\t}\n\t\tdelegatedPublishers := cfg.DelegatedPublishersWithAutoConf()\n\t\tif len(delegatedPublishers) == 0 {\n\t\t\treturn ipns.Name{}, errors.New(\"no delegated publishers configured: add Ipns.DelegatedPublishers or use --allow-offline for local-only publishing\")\n\t\t}\n\t\t// For allow-delegated mode, we only require that we have delegated publishers configured\n\t\t// The node doesn't need P2P connectivity since we're using HTTP publishing\n\t} else {\n\t\t// Normal mode: check online status with allow-offline flag\n\t\terr = api.checkOnline(options.AllowOffline)\n\t\tif err != nil {\n\t\t\treturn ipns.Name{}, err\n\t\t}\n\t}\n\n\tk, err := keylookup(api.privateKey, api.repo.Keystore(), options.Key)\n\tif err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\n\teol := time.Now().Add(options.ValidTime)\n\n\tpublishOptions := []namesys.PublishOption{\n\t\tnamesys.PublishWithEOL(eol),\n\t\tnamesys.PublishWithIPNSOption(ipns.WithV1Compatibility(options.CompatibleWithV1)),\n\t}\n\n\tif options.TTL != nil {\n\t\tpublishOptions = append(publishOptions, namesys.PublishWithTTL(*options.TTL))\n\t}\n\n\tif options.Sequence != nil {\n\t\tpublishOptions = append(publishOptions, namesys.PublishWithSequence(*options.Sequence))\n\t}\n\n\terr = api.namesys.Publish(ctx, k, p, publishOptions...)\n\tif err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\n\tpid, err := peer.IDFromPrivateKey(k)\n\tif err != nil {\n\t\treturn ipns.Name{}, err\n\t}\n\n\treturn ipns.NameFromPeer(pid), nil\n}\n\nfunc (api *NameAPI) Search(ctx context.Context, name string, opts ...caopts.NameResolveOption) (<-chan coreiface.IpnsResult, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.NameAPI\", \"Search\", trace.WithAttributes(attribute.String(\"name\", name)))\n\tdefer span.End()\n\n\toptions, err := caopts.NameResolveOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tspan.SetAttributes(attribute.Bool(\"cache\", options.Cache))\n\n\terr = api.checkOnline(true)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar resolver namesys.Resolver = api.namesys\n\tif !options.Cache {\n\t\tresolver, err = namesys.NewNameSystem(api.routing,\n\t\t\tnamesys.WithDatastore(api.repo.Datastore()),\n\t\t\tnamesys.WithDNSResolver(api.dnsResolver))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif !strings.HasPrefix(name, \"/ipns/\") {\n\t\tname = \"/ipns/\" + name\n\t}\n\n\tp, err := path.NewPath(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tout := make(chan coreiface.IpnsResult)\n\tgo func() {\n\t\tdefer close(out)\n\t\tfor res := range resolver.ResolveAsync(ctx, p, options.ResolveOpts...) {\n\t\t\tselect {\n\t\t\tcase out <- coreiface.IpnsResult{Path: res.Path, Err: res.Err}:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn out, nil\n}\n\n// Resolve attempts to resolve the newest version of the specified name and\n// returns its path.\nfunc (api *NameAPI) Resolve(ctx context.Context, name string, opts ...caopts.NameResolveOption) (path.Path, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.NameAPI\", \"Resolve\", trace.WithAttributes(attribute.String(\"name\", name)))\n\tdefer span.End()\n\n\tctx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\tresults, err := api.Search(ctx, name, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = coreiface.ErrResolveFailed\n\tvar p path.Path\n\n\tfor res := range results {\n\t\tp, err = res.Path, res.Err\n\t\tif err != nil {\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn p, err\n}\n\nfunc keylookup(self ci.PrivKey, kstore keystore.Keystore, k string) (ci.PrivKey, error) {\n\t////////////////////\n\t// Lookup by name //\n\t////////////////////\n\n\t// First, lookup self.\n\tif k == \"self\" {\n\t\treturn self, nil\n\t}\n\n\t// Then, look in the keystore.\n\tres, err := kstore.Get(k)\n\tif res != nil {\n\t\treturn res, nil\n\t}\n\n\tif err != nil && err != keystore.ErrNoSuchKey {\n\t\treturn nil, err\n\t}\n\n\tkeys, err := kstore.List()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t//////////////////\n\t// Lookup by ID //\n\t//////////////////\n\ttargetPid, err := peer.Decode(k)\n\tif err != nil {\n\t\treturn nil, keystore.ErrNoSuchKey\n\t}\n\n\t// First, check self.\n\tpid, err := peer.IDFromPrivateKey(self)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to determine peer ID for private key: %w\", err)\n\t}\n\tif pid == targetPid {\n\t\treturn self, nil\n\t}\n\n\t// Then, look in the keystore.\n\tfor _, key := range keys {\n\t\tprivKey, err := kstore.Get(key)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tpid, err := peer.IDFromPrivateKey(privKey)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif targetPid == pid {\n\t\t\treturn privKey, nil\n\t\t}\n\t}\n\n\treturn nil, errors.New(\"no key by the given name or PeerID was found\")\n}\n" }, { "path": "core/coreapi/object.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipld/merkledag/dagutils\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/path\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n\n\t\"github.com/ipfs/kubo/tracing\"\n)\n\ntype ObjectAPI CoreAPI\n\ntype Link struct {\n\tName, Hash string\n\tSize uint64\n}\n\ntype Node struct {\n\tLinks []Link\n\tData string\n}\n\nfunc (api *ObjectAPI) AddLink(ctx context.Context, base path.Path, name string, child path.Path, opts ...caopts.ObjectAddLinkOption) (path.ImmutablePath, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.ObjectAPI\", \"AddLink\", trace.WithAttributes(\n\t\tattribute.String(\"base\", base.String()),\n\t\tattribute.String(\"name\", name),\n\t\tattribute.String(\"child\", child.String()),\n\t))\n\tdefer span.End()\n\n\toptions, err := caopts.ObjectAddLinkOptions(opts...)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\tspan.SetAttributes(attribute.Bool(\"create\", options.Create))\n\n\tbaseNd, err := api.core().ResolveNode(ctx, base)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tchildNd, err := api.core().ResolveNode(ctx, child)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tbasePb, ok := baseNd.(*dag.ProtoNode)\n\tif !ok {\n\t\treturn path.ImmutablePath{}, dag.ErrNotProtobuf\n\t}\n\n\tvar createfunc func() *dag.ProtoNode\n\tif options.Create {\n\t\tcreatefunc = ft.EmptyDirNode\n\t}\n\n\te := dagutils.NewDagEditor(basePb, api.dag)\n\n\terr = e.InsertNodeAtPath(ctx, name, childNd, createfunc)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tnnode, err := e.Finalize(ctx, api.dag)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn path.FromCid(nnode.Cid()), nil\n}\n\nfunc (api *ObjectAPI) RmLink(ctx context.Context, base path.Path, link string) (path.ImmutablePath, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.ObjectAPI\", \"RmLink\", trace.WithAttributes(\n\t\tattribute.String(\"base\", base.String()),\n\t\tattribute.String(\"link\", link)),\n\t)\n\tdefer span.End()\n\n\tbaseNd, err := api.core().ResolveNode(ctx, base)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tbasePb, ok := baseNd.(*dag.ProtoNode)\n\tif !ok {\n\t\treturn path.ImmutablePath{}, dag.ErrNotProtobuf\n\t}\n\n\te := dagutils.NewDagEditor(basePb, api.dag)\n\n\terr = e.RmLink(ctx, link)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tnnode, err := e.Finalize(ctx, api.dag)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn path.FromCid(nnode.Cid()), nil\n}\n\nfunc (api *ObjectAPI) Diff(ctx context.Context, before path.Path, after path.Path) ([]coreiface.ObjectChange, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.ObjectAPI\", \"Diff\", trace.WithAttributes(\n\t\tattribute.String(\"before\", before.String()),\n\t\tattribute.String(\"after\", after.String()),\n\t))\n\tdefer span.End()\n\n\tbeforeNd, err := api.core().ResolveNode(ctx, before)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tafterNd, err := api.core().ResolveNode(ctx, after)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tchanges, err := dagutils.Diff(ctx, api.dag, beforeNd, afterNd)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tout := make([]coreiface.ObjectChange, len(changes))\n\tfor i, change := range changes {\n\t\tout[i] = coreiface.ObjectChange{\n\t\t\tType: coreiface.ChangeType(change.Type),\n\t\t\tPath: change.Path,\n\t\t}\n\n\t\tif change.Before.Defined() {\n\t\t\tout[i].Before = path.FromCid(change.Before)\n\t\t}\n\n\t\tif change.After.Defined() {\n\t\t\tout[i].After = path.FromCid(change.After)\n\t\t}\n\t}\n\n\treturn out, nil\n}\n\nfunc (api *ObjectAPI) core() coreiface.CoreAPI {\n\treturn (*CoreAPI)(api)\n}\n" }, { "path": "core/coreapi/path.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/kubo/tracing\"\n\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tipfspathresolver \"github.com/ipfs/boxo/path/resolver\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n)\n\n// ResolveNode resolves the path `p` using Unixfs resolver, gets and returns the\n// resolved Node.\nfunc (api *CoreAPI) ResolveNode(ctx context.Context, p path.Path) (ipld.Node, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI\", \"ResolveNode\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\trp, _, err := api.ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnode, err := api.dag.Get(ctx, rp.RootCid())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn node, nil\n}\n\n// ResolvePath resolves the path `p` using Unixfs resolver, returns the\n// resolved path.\nfunc (api *CoreAPI) ResolvePath(ctx context.Context, p path.Path) (path.ImmutablePath, []string, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI\", \"ResolvePath\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tres, err := namesys.Resolve(ctx, api.namesys, p)\n\tif errors.Is(err, namesys.ErrNoNamesys) {\n\t\treturn path.ImmutablePath{}, nil, coreiface.ErrOffline\n\t} else if err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\tp = res.Path\n\n\tvar resolver ipfspathresolver.Resolver\n\tswitch p.Namespace() {\n\tcase path.IPLDNamespace:\n\t\tresolver = api.ipldPathResolver\n\tcase path.IPFSNamespace:\n\t\tresolver = api.unixFSPathResolver\n\tdefault:\n\t\treturn path.ImmutablePath{}, nil, fmt.Errorf(\"unsupported path namespace: %s\", p.Namespace())\n\t}\n\n\timPath, err := path.NewImmutablePath(p)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\tnode, remainder, err := resolver.ResolveToLastNode(ctx, imPath)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\tsegments := []string{p.Namespace(), node.String()}\n\tsegments = append(segments, remainder...)\n\n\tp, err = path.NewPathFromSegments(segments...)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\timPath, err = path.NewImmutablePath(p)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, nil, err\n\t}\n\n\treturn imPath, remainder, nil\n}\n" }, { "path": "core/coreapi/pin.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\tbserv \"github.com/ipfs/boxo/blockservice\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\t\"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/path\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/go-cid\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n\n\t\"github.com/ipfs/kubo/tracing\"\n)\n\ntype PinAPI CoreAPI\n\nfunc (api *PinAPI) Add(ctx context.Context, p path.Path, opts ...caopts.PinAddOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"Add\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tdagNode, err := api.core().ResolveNode(ctx, p)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"pin: %s\", err)\n\t}\n\n\tsettings, err := caopts.PinAddOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tspan.SetAttributes(attribute.Bool(\"recursive\", settings.Recursive))\n\n\tdefer api.blockstore.PinLock(ctx).Unlock(ctx)\n\n\terr = api.pinning.Pin(ctx, dagNode, settings.Recursive, settings.Name)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"pin: %s\", err)\n\t}\n\n\treturn api.pinning.Flush(ctx)\n}\n\nfunc (api *PinAPI) Ls(ctx context.Context, pins chan<- coreiface.Pin, opts ...caopts.PinLsOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"Ls\")\n\tdefer span.End()\n\n\tsettings, err := caopts.PinLsOptions(opts...)\n\tif err != nil {\n\t\tclose(pins)\n\t\treturn err\n\t}\n\n\tspan.SetAttributes(attribute.String(\"type\", settings.Type))\n\n\tswitch settings.Type {\n\tcase \"all\", \"direct\", \"indirect\", \"recursive\":\n\tdefault:\n\t\tclose(pins)\n\t\treturn fmt.Errorf(\"invalid type '%s', must be one of {direct, indirect, recursive, all}\", settings.Type)\n\t}\n\n\treturn api.pinLsAll(ctx, settings.Type, settings.Detailed, settings.Name, pins)\n}\n\nfunc (api *PinAPI) IsPinned(ctx context.Context, p path.Path, opts ...caopts.PinIsPinnedOption) (string, bool, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"IsPinned\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tresolved, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn \"\", false, fmt.Errorf(\"error resolving path: %s\", err)\n\t}\n\n\tsettings, err := caopts.PinIsPinnedOptions(opts...)\n\tif err != nil {\n\t\treturn \"\", false, err\n\t}\n\n\tspan.SetAttributes(attribute.String(\"withtype\", settings.WithType))\n\n\tmode, ok := pin.StringToMode(settings.WithType)\n\tif !ok {\n\t\treturn \"\", false, fmt.Errorf(\"invalid type '%s', must be one of {direct, indirect, recursive, all}\", settings.WithType)\n\t}\n\n\treturn api.pinning.IsPinnedWithType(ctx, resolved.RootCid(), mode)\n}\n\n// Rm pin rm api\nfunc (api *PinAPI) Rm(ctx context.Context, p path.Path, opts ...caopts.PinRmOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"Rm\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tsettings, err := caopts.PinRmOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tspan.SetAttributes(attribute.Bool(\"recursive\", settings.Recursive))\n\n\t// Note: after unpin the pin sets are flushed to the blockstore, so we need\n\t// to take a lock to prevent a concurrent garbage collection\n\tdefer api.blockstore.PinLock(ctx).Unlock(ctx)\n\n\tif err = api.pinning.Unpin(ctx, rp.RootCid(), settings.Recursive); err != nil {\n\t\treturn err\n\t}\n\n\treturn api.pinning.Flush(ctx)\n}\n\nfunc (api *PinAPI) Update(ctx context.Context, from path.Path, to path.Path, opts ...caopts.PinUpdateOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"Update\", trace.WithAttributes(\n\t\tattribute.String(\"from\", from.String()),\n\t\tattribute.String(\"to\", to.String()),\n\t))\n\tdefer span.End()\n\n\tsettings, err := caopts.PinUpdateOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tspan.SetAttributes(attribute.Bool(\"unpin\", settings.Unpin))\n\n\tfp, _, err := api.core().ResolvePath(ctx, from)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttp, _, err := api.core().ResolvePath(ctx, to)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer api.blockstore.PinLock(ctx).Unlock(ctx)\n\n\terr = api.pinning.Update(ctx, fp.RootCid(), tp.RootCid(), settings.Unpin)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn api.pinning.Flush(ctx)\n}\n\ntype pinStatus struct {\n\terr error\n\tcid cid.Cid\n\tok bool\n\tbadNodes []coreiface.BadPinNode\n}\n\n// BadNode is used in PinVerifyRes\ntype badNode struct {\n\tpath path.ImmutablePath\n\terr error\n}\n\nfunc (s *pinStatus) Ok() bool {\n\treturn s.ok\n}\n\nfunc (s *pinStatus) BadNodes() []coreiface.BadPinNode {\n\treturn s.badNodes\n}\n\nfunc (s *pinStatus) Err() error {\n\treturn s.err\n}\n\nfunc (n *badNode) Path() path.ImmutablePath {\n\treturn n.path\n}\n\nfunc (n *badNode) Err() error {\n\treturn n.err\n}\n\nfunc (api *PinAPI) Verify(ctx context.Context) (<-chan coreiface.PinStatus, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"Verify\")\n\tdefer span.End()\n\n\tvisited := make(map[cid.Cid]*pinStatus)\n\tbs := api.blockstore\n\tDAG := merkledag.NewDAGService(bserv.New(bs, offline.Exchange(bs)))\n\tgetLinks := merkledag.GetLinksWithDAG(DAG)\n\n\tvar checkPin func(root cid.Cid) *pinStatus\n\tcheckPin = func(root cid.Cid) *pinStatus {\n\t\tctx, span := tracing.Span(ctx, \"CoreAPI.PinAPI\", \"Verify.CheckPin\", trace.WithAttributes(attribute.String(\"cid\", root.String())))\n\t\tdefer span.End()\n\n\t\tif status, ok := visited[root]; ok {\n\t\t\treturn status\n\t\t}\n\n\t\tlinks, err := getLinks(ctx, root)\n\t\tif err != nil {\n\t\t\tstatus := &pinStatus{ok: false, cid: root}\n\t\t\tstatus.badNodes = []coreiface.BadPinNode{&badNode{path: path.FromCid(root), err: err}}\n\t\t\tvisited[root] = status\n\t\t\treturn status\n\t\t}\n\n\t\tstatus := &pinStatus{ok: true, cid: root}\n\t\tfor _, lnk := range links {\n\t\t\tres := checkPin(lnk.Cid)\n\t\t\tif !res.ok {\n\t\t\t\tstatus.ok = false\n\t\t\t\tstatus.badNodes = append(status.badNodes, res.badNodes...)\n\t\t\t}\n\t\t}\n\n\t\tvisited[root] = status\n\t\treturn status\n\t}\n\n\tout := make(chan coreiface.PinStatus)\n\n\tgo func() {\n\t\tdefer close(out)\n\t\tfor p := range api.pinning.RecursiveKeys(ctx, false) {\n\t\t\tvar res *pinStatus\n\t\t\tif p.Err != nil {\n\t\t\t\tres = &pinStatus{err: p.Err}\n\t\t\t} else {\n\t\t\t\tres = checkPin(p.Pin.Key)\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase out <- res:\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn out, nil\n}\n\ntype pinInfo struct {\n\tpinType string\n\tpath path.ImmutablePath\n\tname string\n}\n\nfunc (p *pinInfo) Path() path.ImmutablePath {\n\treturn p.path\n}\n\nfunc (p *pinInfo) Type() string {\n\treturn p.pinType\n}\n\nfunc (p *pinInfo) Name() string {\n\treturn p.name\n}\n\n// pinLsAll is an internal function for returning a list of pins\n//\n// The caller must keep reading results until the channel is closed to prevent\n// leaking the goroutine that is fetching pins.\nfunc (api *PinAPI) pinLsAll(ctx context.Context, typeStr string, detailed bool, name string, out chan<- coreiface.Pin) error {\n\tdefer close(out)\n\temittedSet := cid.NewSet()\n\n\tAddToResultKeys := func(c cid.Cid, pinName, typeStr string) error {\n\t\tif emittedSet.Visit(c) && (name == \"\" || strings.Contains(pinName, name)) {\n\t\t\tselect {\n\t\t\tcase out <- &pinInfo{\n\t\t\t\tpinType: typeStr,\n\t\t\t\tname: pinName,\n\t\t\t\tpath: path.FromCid(c),\n\t\t\t}:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn ctx.Err()\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t}\n\n\tvar rkeys []cid.Cid\n\tvar err error\n\tif typeStr == \"recursive\" || typeStr == \"all\" {\n\t\tfor streamedCid := range api.pinning.RecursiveKeys(ctx, detailed) {\n\t\t\tif streamedCid.Err != nil {\n\t\t\t\treturn streamedCid.Err\n\t\t\t}\n\t\t\tif err = AddToResultKeys(streamedCid.Pin.Key, streamedCid.Pin.Name, \"recursive\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\trkeys = append(rkeys, streamedCid.Pin.Key)\n\t\t}\n\t}\n\tif typeStr == \"direct\" || typeStr == \"all\" {\n\t\tfor streamedCid := range api.pinning.DirectKeys(ctx, detailed) {\n\t\t\tif streamedCid.Err != nil {\n\t\t\t\treturn streamedCid.Err\n\t\t\t}\n\t\t\tif err = AddToResultKeys(streamedCid.Pin.Key, streamedCid.Pin.Name, \"direct\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\tif typeStr == \"indirect\" {\n\t\t// We need to first visit the direct pins that have priority\n\t\t// without emitting them\n\n\t\tfor streamedCid := range api.pinning.DirectKeys(ctx, detailed) {\n\t\t\tif streamedCid.Err != nil {\n\t\t\t\treturn streamedCid.Err\n\t\t\t}\n\t\t\temittedSet.Add(streamedCid.Pin.Key)\n\t\t}\n\n\t\tfor streamedCid := range api.pinning.RecursiveKeys(ctx, detailed) {\n\t\t\tif streamedCid.Err != nil {\n\t\t\t\treturn streamedCid.Err\n\t\t\t}\n\t\t\temittedSet.Add(streamedCid.Pin.Key)\n\t\t\trkeys = append(rkeys, streamedCid.Pin.Key)\n\t\t}\n\t}\n\tif typeStr == \"indirect\" || typeStr == \"all\" {\n\t\tif len(rkeys) == 0 {\n\t\t\treturn nil\n\t\t}\n\t\tvar addErr error\n\t\twalkingSet := cid.NewSet()\n\t\tfor _, k := range rkeys {\n\t\t\terr = merkledag.Walk(\n\t\t\t\tctx, merkledag.GetLinksWithDAG(api.dag), k,\n\t\t\t\tfunc(c cid.Cid) bool {\n\t\t\t\t\tif !walkingSet.Visit(c) {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t\tif emittedSet.Has(c) {\n\t\t\t\t\t\treturn true // skipped\n\t\t\t\t\t}\n\t\t\t\t\taddErr = AddToResultKeys(c, \"\", \"indirect\")\n\t\t\t\t\treturn addErr == nil\n\t\t\t\t},\n\t\t\t\tmerkledag.SkipRoot(), merkledag.Concurrent(),\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif addErr != nil {\n\t\t\t\treturn addErr\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (api *PinAPI) core() coreiface.CoreAPI {\n\treturn (*CoreAPI)(api)\n}\n" }, { "path": "core/coreapi/pubsub.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"errors\"\n\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/tracing\"\n\tpubsub \"github.com/libp2p/go-libp2p-pubsub\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n)\n\ntype PubSubAPI CoreAPI\n\ntype pubSubSubscription struct {\n\tsubscription *pubsub.Subscription\n}\n\ntype pubSubMessage struct {\n\tmsg *pubsub.Message\n}\n\nfunc (api *PubSubAPI) Ls(ctx context.Context) ([]string, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.PubSubAPI\", \"Ls\")\n\tdefer span.End()\n\n\t_, err := api.checkNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api.pubSub.GetTopics(), nil\n}\n\nfunc (api *PubSubAPI) Peers(ctx context.Context, opts ...caopts.PubSubPeersOption) ([]peer.ID, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.PubSubAPI\", \"Peers\")\n\tdefer span.End()\n\n\t_, err := api.checkNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tsettings, err := caopts.PubSubPeersOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tspan.SetAttributes(attribute.String(\"topic\", settings.Topic))\n\n\treturn api.pubSub.ListPeers(settings.Topic), nil\n}\n\nfunc (api *PubSubAPI) Publish(ctx context.Context, topic string, data []byte) error {\n\t_, span := tracing.Span(ctx, \"CoreAPI.PubSubAPI\", \"Publish\", trace.WithAttributes(attribute.String(\"topic\", topic)))\n\tdefer span.End()\n\n\t_, err := api.checkNode()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t//nolint deprecated\n\treturn api.pubSub.Publish(topic, data)\n}\n\nfunc (api *PubSubAPI) Subscribe(ctx context.Context, topic string, opts ...caopts.PubSubSubscribeOption) (coreiface.PubSubSubscription, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.PubSubAPI\", \"Subscribe\", trace.WithAttributes(attribute.String(\"topic\", topic)))\n\tdefer span.End()\n\n\t// Parse the options to avoid introducing silent failures for invalid\n\t// options. However, we don't currently have any use for them. The only\n\t// subscription option, discovery, is now a no-op as it's handled by\n\t// pubsub itself.\n\t_, err := caopts.PubSubSubscribeOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t_, err = api.checkNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t//nolint deprecated\n\tsub, err := api.pubSub.Subscribe(topic)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &pubSubSubscription{sub}, nil\n}\n\nfunc (api *PubSubAPI) checkNode() (routing.Routing, error) {\n\tif api.pubSub == nil {\n\t\treturn nil, errors.New(\"experimental pubsub feature not enabled, run daemon with --enable-pubsub-experiment to use\")\n\t}\n\n\terr := api.checkOnline(false)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api.routing, nil\n}\n\nfunc (sub *pubSubSubscription) Close() error {\n\tsub.subscription.Cancel()\n\treturn nil\n}\n\nfunc (sub *pubSubSubscription) Next(ctx context.Context) (coreiface.PubSubMessage, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.PubSubSubscription\", \"Next\")\n\tdefer span.End()\n\n\tmsg, err := sub.subscription.Next(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &pubSubMessage{msg}, nil\n}\n\nfunc (msg *pubSubMessage) From() peer.ID {\n\treturn peer.ID(msg.msg.From)\n}\n\nfunc (msg *pubSubMessage) Data() []byte {\n\treturn msg.msg.Data\n}\n\nfunc (msg *pubSubMessage) Seq() []byte {\n\treturn msg.msg.Seqno\n}\n\nfunc (msg *pubSubMessage) Topics() []string {\n\t// TODO: handle breaking downstream changes by returning a single string.\n\tif msg.msg.Topic == nil {\n\t\treturn nil\n\t}\n\treturn []string{*msg.msg.Topic}\n}\n" }, { "path": "core/coreapi/routing.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\n\tblockservice \"github.com/ipfs/boxo/blockservice\"\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/path\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcidutil \"github.com/ipfs/go-cidutil\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\tcaopts \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/core/node\"\n\t\"github.com/ipfs/kubo/tracing\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tmh \"github.com/multiformats/go-multihash\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n)\n\ntype RoutingAPI CoreAPI\n\nfunc (api *RoutingAPI) Get(ctx context.Context, key string) ([]byte, error) {\n\tif !api.nd.IsOnline {\n\t\treturn nil, coreiface.ErrOffline\n\t}\n\n\tdhtKey, err := normalizeKey(key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api.routing.GetValue(ctx, dhtKey)\n}\n\nfunc (api *RoutingAPI) Put(ctx context.Context, key string, value []byte, opts ...caopts.RoutingPutOption) error {\n\toptions, err := caopts.RoutingPutOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = api.checkOnline(options.AllowOffline)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdhtKey, err := normalizeKey(key)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn api.routing.PutValue(ctx, dhtKey, value)\n}\n\nfunc normalizeKey(s string) (string, error) {\n\tparts := strings.Split(s, \"/\")\n\tif len(parts) != 3 ||\n\t\tparts[0] != \"\" ||\n\t\t!(parts[1] == \"ipns\" || parts[1] == \"pk\") {\n\t\treturn \"\", errors.New(\"invalid key\")\n\t}\n\n\tk, err := peer.Decode(parts[2])\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn strings.Join(append(parts[:2], string(k)), \"/\"), nil\n}\n\nfunc (api *RoutingAPI) FindPeer(ctx context.Context, p peer.ID) (peer.AddrInfo, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.DhtAPI\", \"FindPeer\", trace.WithAttributes(attribute.String(\"peer\", p.String())))\n\tdefer span.End()\n\terr := api.checkOnline(false)\n\tif err != nil {\n\t\treturn peer.AddrInfo{}, err\n\t}\n\n\tpi, err := api.routing.FindPeer(ctx, peer.ID(p))\n\tif err != nil {\n\t\treturn peer.AddrInfo{}, err\n\t}\n\n\treturn pi, nil\n}\n\nfunc (api *RoutingAPI) FindProviders(ctx context.Context, p path.Path, opts ...caopts.RoutingFindProvidersOption) (<-chan peer.AddrInfo, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.DhtAPI\", \"FindProviders\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tsettings, err := caopts.RoutingFindProvidersOptions(opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tspan.SetAttributes(attribute.Int(\"numproviders\", settings.NumProviders))\n\n\terr = api.checkOnline(false)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trp, _, err := api.core().ResolvePath(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnumProviders := settings.NumProviders\n\tif numProviders < 1 {\n\t\treturn nil, errors.New(\"number of providers must be greater than 0\")\n\t}\n\n\tpchan := api.routing.FindProvidersAsync(ctx, rp.RootCid(), numProviders)\n\treturn pchan, nil\n}\n\nfunc (api *RoutingAPI) Provide(ctx context.Context, path path.Path, opts ...caopts.RoutingProvideOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.DhtAPI\", \"Provide\", trace.WithAttributes(attribute.String(\"path\", path.String())))\n\tdefer span.End()\n\n\tsettings, err := caopts.RoutingProvideOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\tspan.SetAttributes(attribute.Bool(\"recursive\", settings.Recursive))\n\n\terr = api.checkOnline(false)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\trp, _, err := api.core().ResolvePath(ctx, path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc := rp.RootCid()\n\n\thas, err := api.blockstore.Has(ctx, c)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif !has {\n\t\treturn fmt.Errorf(\"block %s not found locally, cannot provide\", c)\n\t}\n\n\tif settings.Recursive {\n\t\terr = provideKeysRec(ctx, api.provider, api.blockstore, []cid.Cid{c})\n\t} else {\n\t\terr = api.provider.StartProviding(false, c.Hash())\n\t}\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc provideKeysRec(ctx context.Context, prov node.DHTProvider, bs blockstore.Blockstore, cids []cid.Cid) error {\n\tprovided := cidutil.NewStreamingSet()\n\n\t// Error channel with buffer size 1 to avoid blocking the goroutine\n\terrCh := make(chan error, 1)\n\tgo func() {\n\t\t// Always close provided.New to signal completion\n\t\tdefer close(provided.New)\n\t\t// Also close error channel to distinguish between \"no error\" and \"pending error\"\n\t\tdefer close(errCh)\n\n\t\tdserv := dag.NewDAGService(blockservice.New(bs, offline.Exchange(bs)))\n\t\tfor _, c := range cids {\n\t\t\tif err := dag.Walk(ctx, dag.GetLinksDirect(dserv), c, provided.Visitor(ctx)); err != nil {\n\t\t\t\t// Send error to channel. If context is cancelled while trying to send,\n\t\t\t\t// exit immediately as the main loop will return ctx.Err()\n\t\t\t\tselect {\n\t\t\t\tcase errCh <- err:\n\t\t\t\t\t// Error sent successfully, exit goroutine\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t// Context cancelled, exit without sending error\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t\t// All CIDs walked successfully, goroutine will exit and channels will close\n\t}()\n\n\tkeys := make([]mh.Multihash, 0)\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\t// Context cancelled, return immediately\n\t\t\treturn ctx.Err()\n\t\tcase err := <-errCh:\n\t\t\t// Received error from DAG walk, return it\n\t\t\treturn err\n\t\tcase c, ok := <-provided.New:\n\t\t\tif !ok {\n\t\t\t\t// Channel closed means goroutine finished.\n\t\t\t\t// CRITICAL: Check for any error that was sent just before channel closure.\n\t\t\t\t// This handles the race where error is sent to errCh but main loop\n\t\t\t\t// sees provided.New close first.\n\t\t\t\tselect {\n\t\t\t\tcase err := <-errCh:\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\t// errCh closed with nil, meaning success\n\t\t\t\tdefault:\n\t\t\t\t\t// No pending error in errCh\n\t\t\t\t}\n\t\t\t\t// All CIDs successfully processed, start providing\n\t\t\t\treturn prov.StartProviding(true, keys...)\n\t\t\t}\n\t\t\t// Accumulate the CID for providing\n\t\t\tkeys = append(keys, c.Hash())\n\t\t}\n\t}\n}\n\nfunc (api *RoutingAPI) core() coreiface.CoreAPI {\n\treturn (*CoreAPI)(api)\n}\n" }, { "path": "core/coreapi/swarm.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"sort\"\n\t\"time\"\n\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/tracing\"\n\tinet \"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\t\"github.com/libp2p/go-libp2p/p2p/net/swarm\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n)\n\ntype SwarmAPI CoreAPI\n\ntype connInfo struct {\n\tpeerstore pstore.Peerstore\n\tconn inet.Conn\n\tdir inet.Direction\n\n\taddr ma.Multiaddr\n\tpeer peer.ID\n}\n\n// tag used in the connection manager when explicitly connecting to a peer.\nconst (\n\tconnectionManagerTag = \"user-connect\"\n\tconnectionManagerWeight = 100\n)\n\nfunc (api *SwarmAPI) Connect(ctx context.Context, pi peer.AddrInfo) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.SwarmAPI\", \"Connect\", trace.WithAttributes(attribute.String(\"peerid\", pi.ID.String())))\n\tdefer span.End()\n\n\tif api.peerHost == nil {\n\t\treturn coreiface.ErrOffline\n\t}\n\n\tif swrm, ok := api.peerHost.Network().(*swarm.Swarm); ok {\n\t\tswrm.Backoff().Clear(pi.ID)\n\t}\n\n\tif err := api.peerHost.Connect(ctx, pi); err != nil {\n\t\treturn err\n\t}\n\n\tapi.peerHost.ConnManager().TagPeer(pi.ID, connectionManagerTag, connectionManagerWeight)\n\treturn nil\n}\n\nfunc (api *SwarmAPI) Disconnect(ctx context.Context, addr ma.Multiaddr) error {\n\t_, span := tracing.Span(ctx, \"CoreAPI.SwarmAPI\", \"Disconnect\", trace.WithAttributes(attribute.String(\"addr\", addr.String())))\n\tdefer span.End()\n\n\tif api.peerHost == nil {\n\t\treturn coreiface.ErrOffline\n\t}\n\n\ttaddr, id := peer.SplitAddr(addr)\n\tif id == \"\" {\n\t\treturn peer.ErrInvalidAddr\n\t}\n\n\tspan.SetAttributes(attribute.String(\"peerid\", id.String()))\n\n\tnet := api.peerHost.Network()\n\tif taddr == nil {\n\t\tif net.Connectedness(id) != inet.Connected {\n\t\t\treturn coreiface.ErrNotConnected\n\t\t}\n\t\tif err := net.ClosePeer(id); err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t}\n\tfor _, conn := range net.ConnsToPeer(id) {\n\t\tif !conn.RemoteMultiaddr().Equal(taddr) {\n\t\t\tcontinue\n\t\t}\n\n\t\treturn conn.Close()\n\t}\n\treturn coreiface.ErrConnNotFound\n}\n\nfunc (api *SwarmAPI) KnownAddrs(ctx context.Context) (map[peer.ID][]ma.Multiaddr, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.SwarmAPI\", \"KnownAddrs\")\n\tdefer span.End()\n\n\tif api.peerHost == nil {\n\t\treturn nil, coreiface.ErrOffline\n\t}\n\n\taddrs := make(map[peer.ID][]ma.Multiaddr)\n\tps := api.peerHost.Network().Peerstore()\n\tfor _, p := range ps.Peers() {\n\t\taddrs[p] = append(addrs[p], ps.Addrs(p)...)\n\t\tsort.Slice(addrs[p], func(i, j int) bool {\n\t\t\treturn addrs[p][i].String() < addrs[p][j].String()\n\t\t})\n\t}\n\n\treturn addrs, nil\n}\n\nfunc (api *SwarmAPI) LocalAddrs(ctx context.Context) ([]ma.Multiaddr, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.SwarmAPI\", \"LocalAddrs\")\n\tdefer span.End()\n\n\tif api.peerHost == nil {\n\t\treturn nil, coreiface.ErrOffline\n\t}\n\n\treturn api.peerHost.Addrs(), nil\n}\n\nfunc (api *SwarmAPI) ListenAddrs(ctx context.Context) ([]ma.Multiaddr, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.SwarmAPI\", \"ListenAddrs\")\n\tdefer span.End()\n\n\tif api.peerHost == nil {\n\t\treturn nil, coreiface.ErrOffline\n\t}\n\n\treturn api.peerHost.Network().InterfaceListenAddresses()\n}\n\nfunc (api *SwarmAPI) Peers(ctx context.Context) ([]coreiface.ConnectionInfo, error) {\n\t_, span := tracing.Span(ctx, \"CoreAPI.SwarmAPI\", \"Peers\")\n\tdefer span.End()\n\n\tif api.peerHost == nil {\n\t\treturn nil, coreiface.ErrOffline\n\t}\n\n\tconns := api.peerHost.Network().Conns()\n\n\tout := make([]coreiface.ConnectionInfo, 0, len(conns))\n\tfor _, c := range conns {\n\n\t\tci := &connInfo{\n\t\t\tpeerstore: api.peerstore,\n\t\t\tconn: c,\n\t\t\tdir: c.Stat().Direction,\n\n\t\t\taddr: c.RemoteMultiaddr(),\n\t\t\tpeer: c.RemotePeer(),\n\t\t}\n\n\t\t/*\n\t\t\t// FIXME(steb):\n\t\t\tswcon, ok := c.(*swarm.Conn)\n\t\t\tif ok {\n\t\t\t\tci.muxer = fmt.Sprintf(\"%T\", swcon.StreamConn().Conn())\n\t\t\t}\n\t\t*/\n\n\t\tout = append(out, ci)\n\t}\n\n\treturn out, nil\n}\n\nfunc (ci *connInfo) ID() peer.ID {\n\treturn ci.peer\n}\n\nfunc (ci *connInfo) Address() ma.Multiaddr {\n\treturn ci.addr\n}\n\nfunc (ci *connInfo) Direction() inet.Direction {\n\treturn ci.dir\n}\n\nfunc (ci *connInfo) Latency() (time.Duration, error) {\n\treturn ci.peerstore.LatencyEWMA(peer.ID(ci.ID())), nil\n}\n\nfunc (ci *connInfo) Streams() ([]protocol.ID, error) {\n\tstreams := ci.conn.GetStreams()\n\n\tout := make([]protocol.ID, len(streams))\n\tfor i, s := range streams {\n\t\tout[i] = s.Protocol()\n\t}\n\n\treturn out, nil\n}\n" }, { "path": "core/coreapi/test/api_test.go", "content": "package test\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/bootstrap\"\n\t\"github.com/ipfs/boxo/filestore\"\n\tkeystore \"github.com/ipfs/boxo/keystore\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\tmock \"github.com/ipfs/kubo/core/mock\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"github.com/ipfs/go-datastore\"\n\tsyncds \"github.com/ipfs/go-datastore/sync\"\n\t\"github.com/ipfs/kubo/config\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/tests\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\nconst testPeerID = \"QmTFauExutTsy4XP6JbMFcw2Wa9645HJt2bTqL6qYDCKfe\"\n\ntype NodeProvider struct{}\n\nfunc (NodeProvider) MakeAPISwarm(t *testing.T, ctx context.Context, fullIdentity bool, online bool, n int) ([]coreiface.CoreAPI, error) {\n\tmn := mocknet.New()\n\n\tnodes := make([]*core.IpfsNode, n)\n\tapis := make([]coreiface.CoreAPI, n)\n\n\tfor i := range n {\n\t\tvar ident config.Identity\n\t\tif fullIdentity {\n\t\t\tsk, pk, err := crypto.GenerateKeyPair(crypto.RSA, 2048)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tid, err := peer.IDFromPublicKey(pk)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tkbytes, err := crypto.MarshalPrivateKey(sk)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tident = config.Identity{\n\t\t\t\tPeerID: id.String(),\n\t\t\t\tPrivKey: base64.StdEncoding.EncodeToString(kbytes),\n\t\t\t}\n\t\t} else {\n\t\t\tident = config.Identity{\n\t\t\t\tPeerID: testPeerID,\n\t\t\t}\n\t\t}\n\n\t\tc := config.Config{}\n\t\tc.Addresses.Swarm = []string{fmt.Sprintf(\"/ip4/18.0.%d.1/tcp/4001\", i)}\n\t\tc.Identity = ident\n\t\tc.Experimental.FilestoreEnabled = true\n\t\tc.AutoTLS.Enabled = config.False // disable so no /ws listener is added\n\t\t// For provider tests, avoid that content gets\n\t\t// auto-provided without calling \"provide\" (unless pinned).\n\t\tc.Provide.Strategy = config.NewOptionalString(\"roots\")\n\n\t\tds := syncds.MutexWrap(datastore.NewMapDatastore())\n\t\tr := &repo.Mock{\n\t\t\tC: c,\n\t\t\tD: ds,\n\t\t\tK: keystore.NewMemKeystore(),\n\t\t\tF: filestore.NewFileManager(ds, filepath.Dir(os.TempDir())),\n\t\t}\n\n\t\tnode, err := core.NewNode(ctx, &core.BuildCfg{\n\t\t\tRouting: libp2p.DHTServerOption,\n\t\t\tRepo: r,\n\t\t\tHost: mock.MockHostOption(mn),\n\t\t\tOnline: online,\n\t\t\tExtraOpts: map[string]bool{\n\t\t\t\t\"pubsub\": true,\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tnodes[i] = node\n\t\tapis[i], err = coreapi.NewCoreAPI(node)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\terr := mn.LinkAll()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif online {\n\t\tbsinf := bootstrap.BootstrapConfigWithPeers(\n\t\t\t[]peer.AddrInfo{\n\t\t\t\tnodes[0].Peerstore.PeerInfo(nodes[0].Identity),\n\t\t\t},\n\t\t)\n\n\t\tfor _, n := range nodes[1:] {\n\t\t\tif err := n.Bootstrap(bsinf); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn apis, nil\n}\n\nfunc TestIface(t *testing.T) {\n\ttests.TestApi(NodeProvider{})(t)\n}\n" }, { "path": "core/coreapi/test/path_test.go", "content": "package test\n\nimport (\n\t\"context\"\n\t\"strconv\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/ipld/merkledag\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipld/go-ipld-prime\"\n)\n\nfunc TestPathUnixFSHAMTPartial(t *testing.T) {\n\tctx := t.Context()\n\n\t// Create a node\n\tapis, err := NodeProvider{}.MakeAPISwarm(t, ctx, true, true, 1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\ta := apis[0]\n\n\t// Setting this after instantiating the swarm so that it's not clobbered by loading the go-ipfs config\n\tprevVal := uio.HAMTShardingSize\n\tuio.HAMTShardingSize = 1\n\tdefer func() {\n\t\tuio.HAMTShardingSize = prevVal\n\t}()\n\n\t// Create and add a sharded directory\n\tdir := make(map[string]files.Node)\n\t// Make sure we have at least two levels of sharding\n\tfor i := 0; i < uio.DefaultShardWidth+1; i++ {\n\t\tdir[strconv.Itoa(i)] = files.NewBytesFile([]byte(strconv.Itoa(i)))\n\t}\n\n\tr, err := a.Unixfs().Add(ctx, files.NewMapDirectory(dir), options.Unixfs.Pin(false, \"\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Get the root of the directory\n\tnd, err := a.Dag().Get(ctx, r.RootCid())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Make sure the root is a DagPB node (this API might change in the future to account for ADLs)\n\t_ = nd.(ipld.Node)\n\tpbNode := nd.(*merkledag.ProtoNode)\n\n\t// Remove one of the sharded directory blocks\n\tif err := a.Block().Rm(ctx, path.FromCid(pbNode.Links()[0].Cid)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Try and resolve each of the entries in the sharded directory which will result in pathing over the missing block\n\t//\n\t// Note: we could just check a particular path here, but it would require either greater use of the HAMT internals\n\t// or some hard coded values in the test both of which would be a pain to follow.\n\tfor k := range dir {\n\t\t// The node will go out to the (non-existent) network looking for the missing block. Make sure we're erroring\n\t\t// because we exceeded the timeout on our query\n\t\ttimeoutCtx, timeoutCancel := context.WithTimeout(ctx, time.Second*1)\n\t\tnewPath, err := path.Join(r, k)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t_, err = a.ResolveNode(timeoutCtx, newPath)\n\t\tif err != nil {\n\t\t\tif timeoutCtx.Err() == nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\ttimeoutCancel()\n\t}\n}\n" }, { "path": "core/coreapi/unixfs.go", "content": "package coreapi\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\n\tblockservice \"github.com/ipfs/boxo/blockservice\"\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/files\"\n\tfilestore \"github.com/ipfs/boxo/filestore\"\n\tmerkledag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tdagtest \"github.com/ipfs/boxo/ipld/merkledag/test\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\tunixfile \"github.com/ipfs/boxo/ipld/unixfs/file\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\t\"github.com/ipfs/boxo/mfs\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/boxo/provider\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcidutil \"github.com/ipfs/go-cidutil\"\n\tds \"github.com/ipfs/go-datastore\"\n\tdssync \"github.com/ipfs/go-datastore/sync\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/core/coreunix\"\n\t\"github.com/ipfs/kubo/tracing\"\n\tmh \"github.com/multiformats/go-multihash\"\n\t\"go.opentelemetry.io/otel/attribute\"\n\t\"go.opentelemetry.io/otel/trace\"\n)\n\nvar log = logging.Logger(\"coreapi\")\n\ntype UnixfsAPI CoreAPI\n\n// Add builds a merkledag node from a reader, adds it to the blockstore,\n// and returns the key representing that node.\nfunc (api *UnixfsAPI) Add(ctx context.Context, files files.Node, opts ...options.UnixfsAddOption) (path.ImmutablePath, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.UnixfsAPI\", \"Add\")\n\tdefer span.End()\n\n\tsettings, prefix, err := options.UnixfsAddOptions(opts...)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tspan.SetAttributes(\n\t\tattribute.String(\"chunker\", settings.Chunker),\n\t\tattribute.Int(\"cidversion\", settings.CidVersion),\n\t\tattribute.Bool(\"inline\", settings.Inline),\n\t\tattribute.Int(\"inlinelimit\", settings.InlineLimit),\n\t\tattribute.Bool(\"rawleaves\", settings.RawLeaves),\n\t\tattribute.Bool(\"rawleavesset\", settings.RawLeavesSet),\n\t\tattribute.Int(\"maxfilelinks\", settings.MaxFileLinks),\n\t\tattribute.Bool(\"maxfilelinksset\", settings.MaxFileLinksSet),\n\t\tattribute.Int(\"maxdirectorylinks\", settings.MaxDirectoryLinks),\n\t\tattribute.Bool(\"maxdirectorylinksset\", settings.MaxDirectoryLinksSet),\n\t\tattribute.Int(\"maxhamtfanout\", settings.MaxHAMTFanout),\n\t\tattribute.Bool(\"maxhamtfanoutset\", settings.MaxHAMTFanoutSet),\n\t\tattribute.Int(\"layout\", int(settings.Layout)),\n\t\tattribute.Bool(\"pin\", settings.Pin),\n\t\tattribute.String(\"pin-name\", settings.PinName),\n\t\tattribute.Bool(\"onlyhash\", settings.OnlyHash),\n\t\tattribute.Bool(\"fscache\", settings.FsCache),\n\t\tattribute.Bool(\"nocopy\", settings.NoCopy),\n\t\tattribute.Bool(\"silent\", settings.Silent),\n\t\tattribute.Bool(\"progress\", settings.Progress),\n\t)\n\n\tcfg, err := api.repo.Config()\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\t// check if repo will exceed storage limit if added\n\t// TODO: this doesn't handle the case if the hashed file is already in blocks (deduplicated)\n\t// TODO: conditional GC is disabled due to it is somehow not possible to pass the size to the daemon\n\t//if err := corerepo.ConditionalGC(req.Context(), n, uint64(size)); err != nil {\n\t//\tres.SetError(err, cmds.ErrNormal)\n\t//\treturn\n\t//}\n\n\tif settings.NoCopy && !(cfg.Experimental.FilestoreEnabled || cfg.Experimental.UrlstoreEnabled) {\n\t\treturn path.ImmutablePath{}, errors.New(\"either the filestore or the urlstore must be enabled to use nocopy, see: https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-filestore\")\n\t}\n\n\taddblockstore := api.blockstore\n\tif !(settings.FsCache || settings.NoCopy) {\n\t\taddblockstore = bstore.NewGCBlockstore(api.baseBlocks, api.blockstore)\n\t}\n\texch := api.exchange\n\tpinning := api.pinning\n\n\tif settings.OnlyHash {\n\t\t// setup a /dev/null pipeline to simulate adding the data\n\t\tdstore := dssync.MutexWrap(ds.NewNullDatastore())\n\t\tbs := bstore.NewBlockstore(dstore, bstore.WriteThrough(true)) // we use NewNullDatastore, so ok to always WriteThrough when OnlyHash\n\t\taddblockstore = bstore.NewGCBlockstore(bs, nil) // gclocker will never be used\n\t\texch = nil // exchange will never be used\n\t\tpinning = nil // pinner will never be used\n\t}\n\n\tbserv := blockservice.New(addblockstore, exch,\n\t\tblockservice.WriteThrough(cfg.Datastore.WriteThrough.WithDefault(config.DefaultWriteThrough)),\n\t) // hash security 001\n\n\tvar dserv ipld.DAGService = merkledag.NewDAGService(bserv)\n\n\t// wrap the DAGService in a providingDAG service which provides every block written.\n\t// note about strategies:\n\t// - \"all\" gets handled directly at the blockstore so no need to provide\n\t// - \"roots\" gets handled in the pinner\n\t// - \"mfs\" gets handled in mfs\n\t// We need to provide the \"pinned\" cases only. Added blocks are not\n\t// going to be provided by the blockstore (wrong strategy for that),\n\t// nor by the pinner (the pinner doesn't traverse the pinned DAG itself, it only\n\t// handles roots). This wrapping ensures all blocks of pinned content get provided.\n\tif settings.Pin && !settings.OnlyHash &&\n\t\t(api.providingStrategy&config.ProvideStrategyPinned) != 0 {\n\t\tdserv = &providingDagService{dserv, api.provider}\n\t}\n\n\t// add a sync call to the DagService\n\t// this ensures that data written to the DagService is persisted to the underlying datastore\n\t// TODO: propagate the Sync function from the datastore through the blockstore, blockservice and dagservice\n\tvar syncDserv *syncDagService\n\tif settings.OnlyHash {\n\t\tsyncDserv = &syncDagService{\n\t\t\tDAGService: dserv,\n\t\t\tsyncFn: func() error { return nil },\n\t\t}\n\t} else {\n\t\tsyncDserv = &syncDagService{\n\t\t\tDAGService: dserv,\n\t\t\tsyncFn: func() error {\n\t\t\t\trds := api.repo.Datastore()\n\t\t\t\tif err := rds.Sync(ctx, bstore.BlockPrefix); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\treturn rds.Sync(ctx, filestore.FilestorePrefix)\n\t\t\t},\n\t\t}\n\t}\n\n\t// Note: the dag service gets wrapped multiple times:\n\t// 1. providingDagService (if pinned strategy) - provides blocks as they're added\n\t// 2. syncDagService - ensures data persistence\n\t// 3. batchingDagService (in coreunix.Adder) - batches operations for efficiency\n\n\tfileAdder, err := coreunix.NewAdder(ctx, pinning, addblockstore, syncDserv)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\tfileAdder.Chunker = settings.Chunker\n\tif settings.Events != nil {\n\t\tfileAdder.Out = settings.Events\n\t\tfileAdder.Progress = settings.Progress\n\t}\n\tfileAdder.Pin = settings.Pin && !settings.OnlyHash\n\tif settings.Pin {\n\t\tfileAdder.PinName = settings.PinName\n\t}\n\tfileAdder.Silent = settings.Silent\n\tfileAdder.RawLeaves = settings.RawLeaves\n\tif settings.MaxFileLinksSet {\n\t\tfileAdder.MaxLinks = settings.MaxFileLinks\n\t}\n\tif settings.MaxDirectoryLinksSet {\n\t\tfileAdder.MaxDirectoryLinks = settings.MaxDirectoryLinks\n\t}\n\n\tif settings.MaxHAMTFanoutSet {\n\t\tfileAdder.MaxHAMTFanout = settings.MaxHAMTFanout\n\t}\n\tif settings.SizeEstimationModeSet {\n\t\tfileAdder.SizeEstimationMode = settings.SizeEstimationMode\n\t}\n\tfileAdder.NoCopy = settings.NoCopy\n\tfileAdder.CidBuilder = prefix\n\tfileAdder.PreserveMode = settings.PreserveMode\n\tfileAdder.PreserveMtime = settings.PreserveMtime\n\tfileAdder.FileMode = settings.Mode\n\tfileAdder.FileMtime = settings.Mtime\n\tif settings.IncludeEmptyDirsSet {\n\t\tfileAdder.IncludeEmptyDirs = settings.IncludeEmptyDirs\n\t}\n\n\tswitch settings.Layout {\n\tcase options.BalancedLayout:\n\t\t// Default\n\tcase options.TrickleLayout:\n\t\tfileAdder.Trickle = true\n\tdefault:\n\t\treturn path.ImmutablePath{}, fmt.Errorf(\"unknown layout: %d\", settings.Layout)\n\t}\n\n\tif settings.Inline {\n\t\tfileAdder.CidBuilder = cidutil.InlineBuilder{\n\t\t\tBuilder: fileAdder.CidBuilder,\n\t\t\tLimit: settings.InlineLimit,\n\t\t}\n\t}\n\n\tif settings.OnlyHash {\n\t\tmd := dagtest.Mock()\n\t\temptyDirNode := ft.EmptyDirNode()\n\t\t// Use the same prefix for the \"empty\" MFS root as for the file adder.\n\t\terr := emptyDirNode.SetCidBuilder(fileAdder.CidBuilder)\n\t\tif err != nil {\n\t\t\treturn path.ImmutablePath{}, err\n\t\t}\n\t\t// MFS root for OnlyHash mode: provider is nil since we're not storing/providing anything\n\t\tmr, err := mfs.NewRoot(ctx, md, emptyDirNode, nil, nil)\n\t\tif err != nil {\n\t\t\treturn path.ImmutablePath{}, err\n\t\t}\n\n\t\tfileAdder.SetMfsRoot(mr)\n\t}\n\n\tnd, err := fileAdder.AddAllAndPin(ctx, files)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn path.FromCid(nd.Cid()), nil\n}\n\nfunc (api *UnixfsAPI) Get(ctx context.Context, p path.Path) (files.Node, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.UnixfsAPI\", \"Get\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tses := api.core().getSession(ctx)\n\n\tnd, err := ses.ResolveNode(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn unixfile.NewUnixfsFile(ctx, ses.dag, nd)\n}\n\n// Ls returns the contents of an IPFS or IPNS object(s) at path p, with the format:\n// ` `\nfunc (api *UnixfsAPI) Ls(ctx context.Context, p path.Path, out chan<- coreiface.DirEntry, opts ...options.UnixfsLsOption) error {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.UnixfsAPI\", \"Ls\", trace.WithAttributes(attribute.String(\"path\", p.String())))\n\tdefer span.End()\n\n\tdefer close(out)\n\n\tsettings, err := options.UnixfsLsOptions(opts...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tspan.SetAttributes(attribute.Bool(\"resolvechildren\", settings.ResolveChildren))\n\n\tses := api.core().getSession(ctx)\n\tuses := (*UnixfsAPI)(ses)\n\n\tdagnode, err := ses.ResolveNode(ctx, p)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdir, err := uio.NewDirectoryFromNode(ses.dag, dagnode)\n\tif err != nil {\n\t\tif errors.Is(err, uio.ErrNotADir) {\n\t\t\treturn uses.lsFromLinks(ctx, dagnode.Links(), settings, out)\n\t\t}\n\t\treturn err\n\t}\n\n\treturn uses.lsFromDirLinks(ctx, dir, settings, out)\n}\n\nfunc (api *UnixfsAPI) processLink(ctx context.Context, linkres ft.LinkResult, settings *options.UnixfsLsSettings) (coreiface.DirEntry, error) {\n\tctx, span := tracing.Span(ctx, \"CoreAPI.UnixfsAPI\", \"ProcessLink\")\n\tdefer span.End()\n\tif linkres.Link != nil {\n\t\tspan.SetAttributes(attribute.String(\"linkname\", linkres.Link.Name), attribute.String(\"cid\", linkres.Link.Cid.String()))\n\t}\n\n\tif linkres.Err != nil {\n\t\treturn coreiface.DirEntry{}, linkres.Err\n\t}\n\n\tlnk := coreiface.DirEntry{\n\t\tName: linkres.Link.Name,\n\t\tCid: linkres.Link.Cid,\n\t}\n\n\tswitch lnk.Cid.Type() {\n\tcase cid.Raw:\n\t\t// No need to check with raw leaves\n\t\tlnk.Type = coreiface.TFile\n\t\tlnk.Size = linkres.Link.Size\n\tcase cid.DagProtobuf:\n\t\tif settings.ResolveChildren {\n\t\t\tlinkNode, err := linkres.Link.GetNode(ctx, api.dag)\n\t\t\tif err != nil {\n\t\t\t\treturn coreiface.DirEntry{}, err\n\t\t\t}\n\n\t\t\tif pn, ok := linkNode.(*merkledag.ProtoNode); ok {\n\t\t\t\td, err := ft.FSNodeFromBytes(pn.Data())\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn coreiface.DirEntry{}, err\n\t\t\t\t}\n\t\t\t\tswitch d.Type() {\n\t\t\t\tcase ft.TFile, ft.TRaw:\n\t\t\t\t\tlnk.Type = coreiface.TFile\n\t\t\t\tcase ft.THAMTShard, ft.TDirectory, ft.TMetadata:\n\t\t\t\t\tlnk.Type = coreiface.TDirectory\n\t\t\t\tcase ft.TSymlink:\n\t\t\t\t\tlnk.Type = coreiface.TSymlink\n\t\t\t\t\tlnk.Target = string(d.Data())\n\t\t\t\t}\n\t\t\t\tif !settings.UseCumulativeSize {\n\t\t\t\t\tlnk.Size = d.FileSize()\n\t\t\t\t}\n\t\t\t\tlnk.Mode = d.Mode()\n\t\t\t\tlnk.ModTime = d.ModTime()\n\t\t\t}\n\t\t}\n\n\t\tif settings.UseCumulativeSize {\n\t\t\tlnk.Size = linkres.Link.Size\n\t\t}\n\t}\n\n\treturn lnk, nil\n}\n\nfunc (api *UnixfsAPI) lsFromDirLinks(ctx context.Context, dir uio.Directory, settings *options.UnixfsLsSettings, out chan<- coreiface.DirEntry) error {\n\tfor l := range dir.EnumLinksAsync(ctx) {\n\t\tdirEnt, err := api.processLink(ctx, l, settings) // TODO: perf: processing can be done in background and in parallel\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tselect {\n\t\tcase out <- dirEnt:\n\t\tcase <-ctx.Done():\n\t\t\treturn nil\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (api *UnixfsAPI) lsFromLinks(ctx context.Context, ndlinks []*ipld.Link, settings *options.UnixfsLsSettings, out chan<- coreiface.DirEntry) error {\n\t// Create links channel large enough to not block when writing to out is slower.\n\tlinks := make(chan coreiface.DirEntry, len(ndlinks))\n\terrs := make(chan error, 1)\n\tgo func() {\n\t\tdefer close(links)\n\t\tdefer close(errs)\n\t\tfor _, l := range ndlinks {\n\t\t\tlr := ft.LinkResult{Link: &ipld.Link{Name: l.Name, Size: l.Size, Cid: l.Cid}}\n\t\t\tlnk, err := api.processLink(ctx, lr, settings) // TODO: can be parallel if settings.Async\n\t\t\tif err != nil {\n\t\t\t\terrs <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase links <- lnk:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\tfor lnk := range links {\n\t\tout <- lnk\n\t}\n\treturn <-errs\n}\n\nfunc (api *UnixfsAPI) core() *CoreAPI {\n\treturn (*CoreAPI)(api)\n}\n\n// syncDagService is used by the Adder to ensure blocks get persisted to the underlying datastore\ntype syncDagService struct {\n\tipld.DAGService\n\tsyncFn func() error\n}\n\nfunc (s *syncDagService) Sync() error {\n\treturn s.syncFn()\n}\n\ntype providingDagService struct {\n\tipld.DAGService\n\tprovider.MultihashProvider\n}\n\nfunc (pds *providingDagService) Add(ctx context.Context, n ipld.Node) error {\n\tif err := pds.DAGService.Add(ctx, n); err != nil {\n\t\treturn err\n\t}\n\t// Provider errors are logged but not propagated.\n\t// We don't want DAG operations to fail due to providing issues.\n\t// The user's data is still stored successfully even if the\n\t// announcement to the routing system fails temporarily.\n\tif err := pds.StartProviding(false, n.Cid().Hash()); err != nil {\n\t\tlog.Errorf(\"failed to provide new block: %s\", err)\n\t}\n\treturn nil\n}\n\nfunc (pds *providingDagService) AddMany(ctx context.Context, nds []ipld.Node) error {\n\tif err := pds.DAGService.AddMany(ctx, nds); err != nil {\n\t\treturn err\n\t}\n\tkeys := make([]mh.Multihash, len(nds))\n\tfor i, n := range nds {\n\t\tkeys[i] = n.Cid().Hash()\n\t}\n\t// Same error handling philosophy as Add(): log but don't fail.\n\tif err := pds.StartProviding(false, keys...); err != nil {\n\t\tlog.Errorf(\"failed to provide new blocks: %s\", err)\n\t}\n\treturn nil\n}\n\nvar _ ipld.DAGService = (*providingDagService)(nil)\n" }, { "path": "core/corehttp/commands.go", "content": "package corehttp\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\n\tcmds \"github.com/ipfs/go-ipfs-cmds\"\n\tcmdsHttp \"github.com/ipfs/go-ipfs-cmds/http\"\n\tversion \"github.com/ipfs/kubo\"\n\toldcmds \"github.com/ipfs/kubo/commands\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\tcorecommands \"github.com/ipfs/kubo/core/commands\"\n\t\"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp\"\n)\n\nvar errAPIVersionMismatch = errors.New(\"api version mismatch\")\n\nconst (\n\toriginEnvKey = \"API_ORIGIN\"\n\toriginEnvKeyDeprecate = `You are using the ` + originEnvKey + `ENV Variable.\nThis functionality is deprecated, and will be removed in future versions.\nInstead, try either adding headers to the config, or passing them via\ncli arguments:\n\n\tipfs config API.HTTPHeaders --json '{\"Access-Control-Allow-Origin\": [\"*\"]}'\n\tipfs daemon\n`\n)\n\n// APIPath is the path at which the API is mounted.\nconst APIPath = \"/api/v0\"\n\nvar defaultLocalhostOrigins = []string{\n\t\"http://127.0.0.1:\",\n\t\"https://127.0.0.1:\",\n\t\"http://[::1]:\",\n\t\"https://[::1]:\",\n\t\"http://localhost:\",\n\t\"https://localhost:\",\n}\n\nvar companionBrowserExtensionOrigins = []string{\n\t\"chrome-extension://nibjojkomfdiaoajekhjakgkdhaomnch\", // ipfs-companion\n\t\"chrome-extension://hjoieblefckbooibpepigmacodalfndh\", // ipfs-companion-beta\n}\n\nfunc addCORSFromEnv(c *cmdsHttp.ServerConfig) {\n\torigin := os.Getenv(originEnvKey)\n\tif origin != \"\" {\n\t\tlog.Warn(originEnvKeyDeprecate)\n\t\tc.AppendAllowedOrigins(origin)\n\t}\n}\n\nfunc addHeadersFromConfig(c *cmdsHttp.ServerConfig, nc *config.Config) {\n\tlog.Info(\"Using API.HTTPHeaders:\", nc.API.HTTPHeaders)\n\n\tif acao := nc.API.HTTPHeaders[cmdsHttp.ACAOrigin]; acao != nil {\n\t\tc.SetAllowedOrigins(acao...)\n\t}\n\tif acam := nc.API.HTTPHeaders[cmdsHttp.ACAMethods]; acam != nil {\n\t\tc.SetAllowedMethods(acam...)\n\t}\n\tfor _, v := range nc.API.HTTPHeaders[cmdsHttp.ACACredentials] {\n\t\tc.SetAllowCredentials(strings.ToLower(v) == \"true\")\n\t}\n\n\tc.Headers = make(map[string][]string, len(nc.API.HTTPHeaders)+1)\n\n\t// Copy these because the config is shared and this function is called\n\t// in multiple places concurrently. Updating these in-place *is* racy.\n\tfor h, v := range nc.API.HTTPHeaders {\n\t\th = http.CanonicalHeaderKey(h)\n\t\tswitch h {\n\t\tcase cmdsHttp.ACAOrigin, cmdsHttp.ACAMethods, cmdsHttp.ACACredentials:\n\t\t\t// these are handled by the CORs library.\n\t\tdefault:\n\t\t\tc.Headers[h] = v\n\t\t}\n\t}\n\tc.Headers[\"Server\"] = []string{\"kubo/\" + version.CurrentVersionNumber}\n}\n\nfunc addCORSDefaults(c *cmdsHttp.ServerConfig) {\n\t// always safelist certain origins\n\tc.AppendAllowedOrigins(defaultLocalhostOrigins...)\n\tc.AppendAllowedOrigins(companionBrowserExtensionOrigins...)\n\n\t// by default, use GET, PUT, POST\n\tif len(c.AllowedMethods()) == 0 {\n\t\tc.SetAllowedMethods(http.MethodGet, http.MethodPost, http.MethodPut)\n\t}\n}\n\nfunc patchCORSVars(c *cmdsHttp.ServerConfig, addr net.Addr) {\n\t// we have to grab the port from an addr, which may be an ip6 addr.\n\t// TODO: this should take multiaddrs and derive port from there.\n\tport := \"\"\n\tif tcpaddr, ok := addr.(*net.TCPAddr); ok {\n\t\tport = strconv.Itoa(tcpaddr.Port)\n\t} else if udpaddr, ok := addr.(*net.UDPAddr); ok {\n\t\tport = strconv.Itoa(udpaddr.Port)\n\t}\n\n\t// we're listening on tcp/udp with ports. (\"udp!?\" you say? yeah... it happens...)\n\toldOrigins := c.AllowedOrigins()\n\tnewOrigins := make([]string, len(oldOrigins))\n\tfor i, o := range oldOrigins {\n\t\t// TODO: allow replacing . tricky, ip4 and ip6 and hostnames...\n\t\tif port != \"\" {\n\t\t\to = strings.Replace(o, \"\", port, -1)\n\t\t}\n\t\tnewOrigins[i] = o\n\t}\n\tc.SetAllowedOrigins(newOrigins...)\n}\n\nfunc commandsOption(cctx oldcmds.Context, command *cmds.Command) ServeOption {\n\treturn func(n *core.IpfsNode, l net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tcfg := cmdsHttp.NewServerConfig()\n\n\t\tcfg.AddAllowedHeaders(\"Origin\", \"Accept\", \"Content-Type\", \"X-Requested-With\")\n\t\tcfg.SetAllowedMethods(http.MethodPost)\n\n\t\tcfg.APIPath = APIPath\n\t\trcfg, err := n.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\taddHeadersFromConfig(cfg, rcfg)\n\t\taddCORSFromEnv(cfg)\n\t\taddCORSDefaults(cfg)\n\t\tpatchCORSVars(cfg, l.Addr())\n\n\t\tcmdHandler := cmdsHttp.NewHandler(&cctx, command, cfg)\n\n\t\tif len(rcfg.API.Authorizations) > 0 {\n\t\t\tauthorizations := convertAuthorizationsMap(rcfg.API.Authorizations)\n\t\t\tcmdHandler = withAuthSecrets(authorizations, cmdHandler)\n\t\t}\n\n\t\tcmdHandler = otelhttp.NewHandler(cmdHandler, \"corehttp.cmdsHandler\",\n\t\t\totelhttp.WithMetricAttributesFn(staticServerDomainAttrFn(\"api\")),\n\t\t)\n\t\tmux.Handle(APIPath+\"/\", cmdHandler)\n\t\treturn mux, nil\n\t}\n}\n\ntype rpcAuthScopeWithUser struct {\n\tconfig.RPCAuthScope\n\tUser string\n}\n\nfunc convertAuthorizationsMap(authScopes map[string]*config.RPCAuthScope) map[string]rpcAuthScopeWithUser {\n\t// authorizations is a map where we can just check for the header value to match.\n\tauthorizations := map[string]rpcAuthScopeWithUser{}\n\tfor user, authScope := range authScopes {\n\t\texpectedHeader := config.ConvertAuthSecret(authScope.AuthSecret)\n\t\tif expectedHeader != \"\" {\n\t\t\tauthorizations[expectedHeader] = rpcAuthScopeWithUser{\n\t\t\t\tRPCAuthScope: *authScopes[user],\n\t\t\t\tUser: user,\n\t\t\t}\n\t\t}\n\t}\n\n\treturn authorizations\n}\n\nfunc withAuthSecrets(authorizations map[string]rpcAuthScopeWithUser, next http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tauthorizationHeader := r.Header.Get(\"Authorization\")\n\t\tauth, ok := authorizations[authorizationHeader]\n\n\t\tif ok {\n\t\t\t// version check is implicitly allowed\n\t\t\tif r.URL.Path == \"/api/v0/version\" {\n\t\t\t\tnext.ServeHTTP(w, r)\n\t\t\t\treturn\n\t\t\t}\n\t\t\t// everything else has to be safelisted via AllowedPaths\n\t\t\tfor _, prefix := range auth.AllowedPaths {\n\t\t\t\tif strings.HasPrefix(r.URL.Path, prefix) {\n\t\t\t\t\tnext.ServeHTTP(w, r)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\thttp.Error(w, \"Kubo RPC Access Denied: Please provide a valid authorization token as defined in the API.Authorizations configuration.\", http.StatusForbidden)\n\t})\n}\n\n// CommandsOption constructs a ServerOption for hooking the commands into the\n// HTTP server. It will NOT allow GET requests.\nfunc CommandsOption(cctx oldcmds.Context) ServeOption {\n\treturn commandsOption(cctx, corecommands.Root)\n}\n\n// CheckVersionOption returns a ServeOption that checks whether the client ipfs version matches. Does nothing when the user agent string does not contain `/kubo/` or `/go-ipfs/`\nfunc CheckVersionOption() ServeOption {\n\tdaemonVersion := version.ApiVersion\n\n\treturn func(n *core.IpfsNode, l net.Listener, parent *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux := http.NewServeMux()\n\t\tparent.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif strings.HasPrefix(r.URL.Path, APIPath) {\n\t\t\t\tcmdqry := r.URL.Path[len(APIPath):]\n\t\t\t\tpth := strings.Split(cmdqry, \"/\")\n\n\t\t\t\t// backwards compatibility to previous version check\n\t\t\t\tif len(pth) >= 2 && pth[1] != \"version\" {\n\t\t\t\t\tclientVersion := r.UserAgent()\n\t\t\t\t\t// skips check if client is not kubo (go-ipfs)\n\t\t\t\t\tif (strings.Contains(clientVersion, \"/go-ipfs/\") || strings.Contains(clientVersion, \"/kubo/\")) && daemonVersion != clientVersion {\n\t\t\t\t\t\thttp.Error(w, fmt.Sprintf(\"%s (%s != %s)\", errAPIVersionMismatch, daemonVersion, clientVersion), http.StatusBadRequest)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tmux.ServeHTTP(w, r)\n\t\t})\n\n\t\treturn mux, nil\n\t}\n}\n" }, { "path": "core/corehttp/corehttp.go", "content": "/*\nPackage corehttp provides utilities for the webui, gateways, and other\nhigh-level HTTP interfaces to IPFS.\n*/\npackage corehttp\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tcore \"github.com/ipfs/kubo/core\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nvar log = logging.Logger(\"core/server\")\n\n// shutdownTimeout is the timeout after which we'll stop waiting for hung\n// commands to return on shutdown.\nconst shutdownTimeout = 30 * time.Second\n\n// ServeOption registers any HTTP handlers it provides on the given mux.\n// It returns the mux to expose to future options, which may be a new mux if it\n// is interested in mediating requests to future options, or the same mux\n// initially passed in if not.\ntype ServeOption func(*core.IpfsNode, net.Listener, *http.ServeMux) (*http.ServeMux, error)\n\n// MakeHandler turns a list of ServeOptions into a http.Handler that implements\n// all of the given options, in order.\nfunc MakeHandler(n *core.IpfsNode, l net.Listener, options ...ServeOption) (http.Handler, error) {\n\ttopMux := http.NewServeMux()\n\tmux := topMux\n\tfor _, option := range options {\n\t\tvar err error\n\t\tmux, err = option(n, l, mux)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\thandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t// ServeMux does not support requests with CONNECT method,\n\t\t// so we need to handle them separately\n\t\t// https://golang.org/src/net/http/request.go#L111\n\t\tif r.Method == http.MethodConnect {\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\treturn\n\t\t}\n\t\ttopMux.ServeHTTP(w, r)\n\t})\n\treturn handler, nil\n}\n\n// ListenAndServe runs an HTTP server listening at |listeningMultiAddr| with\n// the given serve options. The address must be provided in multiaddr format.\n//\n// TODO intelligently parse address strings in other formats so long as they\n// unambiguously map to a valid multiaddr. e.g. for convenience, \":8080\" should\n// map to \"/ip4/0.0.0.0/tcp/8080\".\nfunc ListenAndServe(n *core.IpfsNode, listeningMultiAddr string, options ...ServeOption) error {\n\taddr, err := ma.NewMultiaddr(listeningMultiAddr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlist, err := manet.Listen(addr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// we might have listened to /tcp/0 - let's see what we are listing on\n\taddr = list.Multiaddr()\n\tfmt.Printf(\"RPC API server listening on %s\\n\", addr)\n\n\treturn Serve(n, manet.NetListener(list), options...)\n}\n\n// Serve accepts incoming HTTP connections on the listener and passes them\n// to ServeOption handlers.\nfunc Serve(node *core.IpfsNode, lis net.Listener, options ...ServeOption) error {\n\treturn ServeWithReady(node, lis, nil, options...)\n}\n\n// ServeWithReady is like Serve but signals on the ready channel when the\n// server is about to accept connections. The channel is closed right before\n// server.Serve() is called.\n//\n// This is useful for callers that need to perform actions (like writing\n// address files) only after the server is guaranteed to be accepting\n// connections, avoiding race conditions where clients see the file before\n// the server is ready.\n//\n// Passing nil for ready is equivalent to calling Serve().\nfunc ServeWithReady(node *core.IpfsNode, lis net.Listener, ready chan<- struct{}, options ...ServeOption) error {\n\t// make sure we close this no matter what.\n\tdefer lis.Close()\n\n\thandler, err := MakeHandler(node, lis, options...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\taddr, err := manet.FromNetAddr(lis.Addr())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tselect {\n\tcase <-node.Context().Done():\n\t\treturn fmt.Errorf(\"failed to start server, process closing\")\n\tdefault:\n\t}\n\n\tserver := &http.Server{\n\t\tHandler: handler,\n\t}\n\n\tvar serverError error\n\tserverClosed := make(chan struct{})\n\tgo func() {\n\t\tif ready != nil {\n\t\t\tclose(ready)\n\t\t}\n\t\tserverError = server.Serve(lis)\n\t\tclose(serverClosed)\n\t}()\n\n\t// wait for server to exit.\n\tselect {\n\tcase <-serverClosed:\n\t// if node being closed before server exits, close server\n\tcase <-node.Context().Done():\n\t\tlog.Infof(\"server at %s terminating...\", addr)\n\n\t\tgo func() {\n\t\t\tticker := time.NewTicker(5 * time.Second)\n\t\t\tdefer ticker.Stop()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ticker.C:\n\t\t\t\t\tlog.Infof(\"waiting for server at %s to terminate...\", addr)\n\t\t\t\tcase <-serverClosed:\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\t// This timeout shouldn't be necessary if all of our commands\n\t\t// are obeying their contexts but we should have *some* timeout.\n\t\tctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)\n\t\tdefer cancel()\n\t\terr := server.Shutdown(ctx)\n\n\t\t// Should have already closed but we still need to wait for it\n\t\t// to set the error.\n\t\t<-serverClosed\n\t\tserverError = err\n\t}\n\n\tlog.Infof(\"server at %s terminated\", addr)\n\treturn serverError\n}\n" }, { "path": "core/corehttp/gateway.go", "content": "package corehttp\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"maps\"\n\t\"net\"\n\t\"net/http\"\n\t\"slices\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/blockservice\"\n\t\"github.com/ipfs/boxo/exchange/offline\"\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/gateway\"\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/path\"\n\tofflineroute \"github.com/ipfs/boxo/routing/offline\"\n\t\"github.com/ipfs/go-cid\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/node\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\t\"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp\"\n\t\"go.opentelemetry.io/otel/attribute\"\n)\n\nfunc GatewayOption(paths ...string) ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tconfig, headers, err := getGatewayConfig(n)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tbackend, err := newGatewayBackend(n)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\thandler := gateway.NewHandler(config, backend)\n\t\thandler = gateway.NewHeaders(headers).ApplyCors().Wrap(handler)\n\t\tvar otelOpts []otelhttp.Option\n\t\tif fn := newServerDomainAttrFn(n); fn != nil {\n\t\t\totelOpts = append(otelOpts, otelhttp.WithMetricAttributesFn(fn))\n\t\t}\n\t\thandler = otelhttp.NewHandler(handler, \"Gateway\", otelOpts...)\n\n\t\tfor _, p := range paths {\n\t\t\tmux.Handle(p+\"/\", handler)\n\t\t}\n\n\t\treturn mux, nil\n\t}\n}\n\nfunc HostnameOption() ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tconfig, headers, err := getGatewayConfig(n)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tbackend, err := newGatewayBackend(n)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tchildMux := http.NewServeMux()\n\n\t\tvar handler http.Handler\n\t\thandler = gateway.NewHostnameHandler(config, backend, childMux)\n\t\thandler = gateway.NewHeaders(headers).ApplyCors().Wrap(handler)\n\t\tvar otelOpts []otelhttp.Option\n\t\tif fn := newServerDomainAttrFn(n); fn != nil {\n\t\t\totelOpts = append(otelOpts, otelhttp.WithMetricAttributesFn(fn))\n\t\t}\n\t\thandler = otelhttp.NewHandler(handler, \"HostnameGateway\", otelOpts...)\n\n\t\tmux.Handle(\"/\", handler)\n\t\treturn childMux, nil\n\t}\n}\n\nfunc VersionOption() ServeOption {\n\treturn func(_ *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux.HandleFunc(\"/version\", func(w http.ResponseWriter, r *http.Request) {\n\t\t\tfmt.Fprintf(w, \"Commit: %s\\n\", version.CurrentCommit)\n\t\t\tfmt.Fprintf(w, \"Client Version: %s\\n\", version.GetUserAgentVersion())\n\t\t})\n\t\treturn mux, nil\n\t}\n}\n\nfunc Libp2pGatewayOption() ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tbserv := blockservice.New(n.Blocks.Blockstore(), offline.Exchange(n.Blocks.Blockstore()))\n\n\t\tbackend, err := gateway.NewBlocksBackend(bserv,\n\t\t\t// GatewayOverLibp2p only returns things that are in local blockstore\n\t\t\t// (same as Gateway.NoFetch=true), we have to pass offline path resolver\n\t\t\tgateway.WithResolver(n.OfflineUnixFSPathResolver),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Get gateway configuration from the node's config\n\t\tcfg, err := n.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tgwConfig := gateway.Config{\n\t\t\t// Keep these constraints for security\n\t\t\tDeserializedResponses: false, // Trustless-only\n\t\t\tNoDNSLink: true, // No DNS resolution\n\t\t\tDisableHTMLErrors: true, // Plain text errors only\n\t\t\tPublicGateways: nil,\n\t\t\tMenu: nil,\n\t\t\t// Apply timeout and concurrency limits from user config\n\t\t\tRetrievalTimeout: cfg.Gateway.RetrievalTimeout.WithDefault(config.DefaultRetrievalTimeout),\n\t\t\tMaxRequestDuration: cfg.Gateway.MaxRequestDuration.WithDefault(config.DefaultMaxRequestDuration),\n\t\t\tMaxConcurrentRequests: int(cfg.Gateway.MaxConcurrentRequests.WithDefault(int64(config.DefaultMaxConcurrentRequests))),\n\t\t\tMaxRangeRequestFileSize: int64(cfg.Gateway.MaxRangeRequestFileSize.WithDefault(uint64(config.DefaultMaxRangeRequestFileSize))),\n\t\t\tDiagnosticServiceURL: \"\", // Not used since DisableHTMLErrors=true\n\t\t}\n\n\t\thandler := gateway.NewHandler(gwConfig, &offlineGatewayErrWrapper{gwimpl: backend})\n\t\thandler = otelhttp.NewHandler(handler, \"Libp2p-Gateway\",\n\t\t\totelhttp.WithMetricAttributesFn(staticServerDomainAttrFn(\"libp2p\")),\n\t\t)\n\n\t\tmux.Handle(\"/ipfs/\", handler)\n\n\t\treturn mux, nil\n\t}\n}\n\nfunc newGatewayBackend(n *core.IpfsNode) (gateway.IPFSBackend, error) {\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tbserv := n.Blocks\n\tvar vsRouting routing.ValueStore = n.Routing\n\tnsys := n.Namesys\n\tpathResolver := n.UnixFSPathResolver\n\n\tif cfg.Gateway.NoFetch {\n\t\tbserv = blockservice.New(bserv.Blockstore(), offline.Exchange(bserv.Blockstore()))\n\n\t\tcs := cfg.Ipns.ResolveCacheSize\n\t\tif cs == 0 {\n\t\t\tcs = node.DefaultIpnsCacheSize\n\t\t}\n\t\tif cs < 0 {\n\t\t\treturn nil, fmt.Errorf(\"cannot specify negative resolve cache size\")\n\t\t}\n\n\t\tnsOptions := []namesys.Option{\n\t\t\tnamesys.WithDatastore(n.Repo.Datastore()),\n\t\t\tnamesys.WithDNSResolver(n.DNSResolver),\n\t\t\tnamesys.WithCache(cs),\n\t\t\tnamesys.WithMaxCacheTTL(cfg.Ipns.MaxCacheTTL.WithDefault(config.DefaultIpnsMaxCacheTTL)),\n\t\t}\n\n\t\tvsRouting = offlineroute.NewOfflineRouter(n.Repo.Datastore(), n.RecordValidator)\n\t\tnsys, err = namesys.NewNameSystem(vsRouting, nsOptions...)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error constructing namesys: %w\", err)\n\t\t}\n\n\t\t// Gateway.NoFetch=true requires offline path resolver\n\t\t// to avoid fetching missing blocks during path traversal\n\t\tpathResolver = n.OfflineUnixFSPathResolver\n\t}\n\n\tbackend, err := gateway.NewBlocksBackend(bserv,\n\t\tgateway.WithValueStore(vsRouting),\n\t\tgateway.WithNameSystem(nsys),\n\t\tgateway.WithResolver(pathResolver),\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn &offlineGatewayErrWrapper{gwimpl: backend}, nil\n}\n\ntype offlineGatewayErrWrapper struct {\n\tgwimpl gateway.IPFSBackend\n}\n\nfunc offlineErrWrap(err error) error {\n\tif errors.Is(err, iface.ErrOffline) {\n\t\treturn fmt.Errorf(\"%s : %w\", err.Error(), gateway.ErrServiceUnavailable)\n\t}\n\treturn err\n}\n\nfunc (o *offlineGatewayErrWrapper) Get(ctx context.Context, path path.ImmutablePath, ranges ...gateway.ByteRange) (gateway.ContentPathMetadata, *gateway.GetResponse, error) {\n\tmd, n, err := o.gwimpl.Get(ctx, path, ranges...)\n\terr = offlineErrWrap(err)\n\treturn md, n, err\n}\n\nfunc (o *offlineGatewayErrWrapper) GetAll(ctx context.Context, path path.ImmutablePath) (gateway.ContentPathMetadata, files.Node, error) {\n\tmd, n, err := o.gwimpl.GetAll(ctx, path)\n\terr = offlineErrWrap(err)\n\treturn md, n, err\n}\n\nfunc (o *offlineGatewayErrWrapper) GetBlock(ctx context.Context, path path.ImmutablePath) (gateway.ContentPathMetadata, files.File, error) {\n\tmd, n, err := o.gwimpl.GetBlock(ctx, path)\n\terr = offlineErrWrap(err)\n\treturn md, n, err\n}\n\nfunc (o *offlineGatewayErrWrapper) Head(ctx context.Context, path path.ImmutablePath) (gateway.ContentPathMetadata, *gateway.HeadResponse, error) {\n\tmd, n, err := o.gwimpl.Head(ctx, path)\n\terr = offlineErrWrap(err)\n\treturn md, n, err\n}\n\nfunc (o *offlineGatewayErrWrapper) ResolvePath(ctx context.Context, path path.ImmutablePath) (gateway.ContentPathMetadata, error) {\n\tmd, err := o.gwimpl.ResolvePath(ctx, path)\n\terr = offlineErrWrap(err)\n\treturn md, err\n}\n\nfunc (o *offlineGatewayErrWrapper) GetCAR(ctx context.Context, path path.ImmutablePath, params gateway.CarParams) (gateway.ContentPathMetadata, io.ReadCloser, error) {\n\tmd, data, err := o.gwimpl.GetCAR(ctx, path, params)\n\terr = offlineErrWrap(err)\n\treturn md, data, err\n}\n\nfunc (o *offlineGatewayErrWrapper) IsCached(ctx context.Context, path path.Path) bool {\n\treturn o.gwimpl.IsCached(ctx, path)\n}\n\nfunc (o *offlineGatewayErrWrapper) GetIPNSRecord(ctx context.Context, c cid.Cid) ([]byte, error) {\n\trec, err := o.gwimpl.GetIPNSRecord(ctx, c)\n\terr = offlineErrWrap(err)\n\treturn rec, err\n}\n\nfunc (o *offlineGatewayErrWrapper) ResolveMutable(ctx context.Context, path path.Path) (path.ImmutablePath, time.Duration, time.Time, error) {\n\timPath, ttl, lastMod, err := o.gwimpl.ResolveMutable(ctx, path)\n\terr = offlineErrWrap(err)\n\treturn imPath, ttl, lastMod, err\n}\n\nfunc (o *offlineGatewayErrWrapper) GetDNSLinkRecord(ctx context.Context, s string) (path.Path, error) {\n\tp, err := o.gwimpl.GetDNSLinkRecord(ctx, s)\n\terr = offlineErrWrap(err)\n\treturn p, err\n}\n\nvar _ gateway.IPFSBackend = (*offlineGatewayErrWrapper)(nil)\n\nvar defaultPaths = []string{\"/ipfs/\", \"/ipns/\", \"/p2p/\"}\n\n// serverDomainAttrKey is the OTel attribute key for the logical server domain.\n// It replaces the high-cardinality server.address attribute (dropped by the\n// View in cmd/ipfs/kubo/daemon.go) with a bounded set of values: configured\n// Gateway.PublicGateways suffixes, \"localhost\", \"loopback\", \"api\", \"libp2p\",\n// or \"other\".\nvar serverDomainAttrKey = attribute.Key(\"server.domain\")\n\n// staticServerDomainAttrFn returns a MetricAttributesFn that always returns\n// a fixed server.domain value. Use for handlers where the domain is known\n// statically (e.g. \"api\", \"libp2p\") to keep the label set consistent across\n// all http_server_* metrics.\nfunc staticServerDomainAttrFn(domain string) func(*http.Request) []attribute.KeyValue {\n\tattrs := []attribute.KeyValue{serverDomainAttrKey.String(domain)}\n\treturn func(*http.Request) []attribute.KeyValue { return attrs }\n}\n\n// newServerDomainAttrFn returns an otelhttp.WithMetricAttributesFn callback\n// that adds a server.domain attribute grouping requests by their matching\n// Gateway.PublicGateways hostname suffix (e.g. \"dweb.link\", \"ipfs.io\").\n// Requests that don't match any configured gateway get \"other\".\n//\n// All return values are pre-allocated at setup time so the per-request\n// closure is zero-allocation.\nfunc newServerDomainAttrFn(n *core.IpfsNode) func(*http.Request) []attribute.KeyValue {\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn nil\n\t}\n\n\t// Collect non-nil gateway domain suffixes, sorted longest-first\n\t// so more-specific suffixes match before shorter ones.\n\t// Strip ports from keys to match boxo's fallback behavior\n\t// (boxo tries exact match with port, then strips port and retries).\n\tseen := make(map[string]struct{}, len(cfg.Gateway.PublicGateways))\n\tsuffixes := make([]string, 0, len(cfg.Gateway.PublicGateways))\n\tfor hostname, gw := range cfg.Gateway.PublicGateways {\n\t\tif gw == nil {\n\t\t\tcontinue\n\t\t}\n\t\tif h, _, err := net.SplitHostPort(hostname); err == nil {\n\t\t\thostname = h\n\t\t}\n\t\tif _, ok := seen[hostname]; ok {\n\t\t\tcontinue\n\t\t}\n\t\tseen[hostname] = struct{}{}\n\t\tsuffixes = append(suffixes, hostname)\n\t}\n\tslices.SortFunc(suffixes, func(a, b string) int {\n\t\treturn len(b) - len(a)\n\t})\n\n\t// Pre-allocate attribute slices so the per-request closure only returns\n\t// existing slices and does not allocate.\n\tsuffixAttrs := make([][]attribute.KeyValue, len(suffixes))\n\tfor i, s := range suffixes {\n\t\tsuffixAttrs[i] = []attribute.KeyValue{serverDomainAttrKey.String(s)}\n\t}\n\tlocalhostAttr := []attribute.KeyValue{serverDomainAttrKey.String(\"localhost\")}\n\tloopbackAttr := []attribute.KeyValue{serverDomainAttrKey.String(\"loopback\")}\n\totherAttr := []attribute.KeyValue{serverDomainAttrKey.String(\"other\")}\n\n\treturn func(r *http.Request) []attribute.KeyValue {\n\t\thost := r.Host\n\t\tif h, _, err := net.SplitHostPort(host); err == nil {\n\t\t\thost = h\n\t\t}\n\n\t\t// Check localhost/loopback before iterating suffixes.\n\t\t// \"localhost\" is an implicit default gateway (defaultKnownGateways)\n\t\t// not present in cfg.Gateway.PublicGateways, so it won't appear\n\t\t// in suffixes.\n\t\tif host == \"localhost\" || strings.HasSuffix(host, \".localhost\") {\n\t\t\treturn localhostAttr\n\t\t}\n\t\tif strings.HasPrefix(host, \"127.\") || host == \"::1\" {\n\t\t\treturn loopbackAttr\n\t\t}\n\n\t\tfor i, suffix := range suffixes {\n\t\t\tif strings.HasSuffix(host, suffix) {\n\t\t\t\treturn suffixAttrs[i]\n\t\t\t}\n\t\t}\n\n\t\treturn otherAttr\n\t}\n}\n\nvar subdomainGatewaySpec = &gateway.PublicGateway{\n\tPaths: defaultPaths,\n\tUseSubdomains: true,\n}\n\nvar defaultKnownGateways = map[string]*gateway.PublicGateway{\n\t\"localhost\": subdomainGatewaySpec,\n}\n\nfunc getGatewayConfig(n *core.IpfsNode) (gateway.Config, map[string][]string, error) {\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn gateway.Config{}, nil, err\n\t}\n\n\t// Initialize gateway configuration, with empty PublicGateways, handled after.\n\tgwCfg := gateway.Config{\n\t\tDeserializedResponses: cfg.Gateway.DeserializedResponses.WithDefault(config.DefaultDeserializedResponses),\n\t\tAllowCodecConversion: cfg.Gateway.AllowCodecConversion.WithDefault(config.DefaultAllowCodecConversion),\n\t\tDisableHTMLErrors: cfg.Gateway.DisableHTMLErrors.WithDefault(config.DefaultDisableHTMLErrors),\n\t\tNoDNSLink: cfg.Gateway.NoDNSLink,\n\t\tPublicGateways: map[string]*gateway.PublicGateway{},\n\t\tRetrievalTimeout: cfg.Gateway.RetrievalTimeout.WithDefault(config.DefaultRetrievalTimeout),\n\t\tMaxRequestDuration: cfg.Gateway.MaxRequestDuration.WithDefault(config.DefaultMaxRequestDuration),\n\t\tMaxConcurrentRequests: int(cfg.Gateway.MaxConcurrentRequests.WithDefault(int64(config.DefaultMaxConcurrentRequests))),\n\t\tMaxRangeRequestFileSize: int64(cfg.Gateway.MaxRangeRequestFileSize.WithDefault(uint64(config.DefaultMaxRangeRequestFileSize))),\n\t\tDiagnosticServiceURL: cfg.Gateway.DiagnosticServiceURL.WithDefault(config.DefaultDiagnosticServiceURL),\n\t}\n\n\t// Add default implicit known gateways, such as subdomain gateway on localhost.\n\tmaps.Copy(gwCfg.PublicGateways, defaultKnownGateways)\n\n\t// Apply values from cfg.Gateway.PublicGateways if they exist.\n\tfor hostname, gw := range cfg.Gateway.PublicGateways {\n\t\tif gw == nil {\n\t\t\t// Remove any implicit defaults, if present. This is useful when one\n\t\t\t// wants to disable subdomain gateway on localhost, etc.\n\t\t\tdelete(gwCfg.PublicGateways, hostname)\n\t\t\tcontinue\n\t\t}\n\n\t\tgwCfg.PublicGateways[hostname] = &gateway.PublicGateway{\n\t\t\tPaths: gw.Paths,\n\t\t\tNoDNSLink: gw.NoDNSLink,\n\t\t\tUseSubdomains: gw.UseSubdomains,\n\t\t\tInlineDNSLink: gw.InlineDNSLink.WithDefault(config.DefaultInlineDNSLink),\n\t\t\tDeserializedResponses: gw.DeserializedResponses.WithDefault(gwCfg.DeserializedResponses),\n\t\t}\n\t}\n\n\treturn gwCfg, cfg.Gateway.HTTPHeaders, nil\n}\n" }, { "path": "core/corehttp/gateway_test.go", "content": "package corehttp\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-datastore\"\n\tsyncds \"github.com/ipfs/go-datastore/sync\"\n\t\"github.com/ipfs/kubo/config\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\tci \"github.com/libp2p/go-libp2p/core/crypto\"\n)\n\ntype mockNamesys map[string]path.Path\n\nfunc (m mockNamesys) Resolve(ctx context.Context, p path.Path, opts ...namesys.ResolveOption) (namesys.Result, error) {\n\tcfg := namesys.DefaultResolveOptions()\n\tfor _, o := range opts {\n\t\to(&cfg)\n\t}\n\tdepth := cfg.Depth\n\tif depth == namesys.UnlimitedDepth {\n\t\t// max uint\n\t\tdepth = ^uint(0)\n\t}\n\tvar (\n\t\tvalue path.Path\n\t)\n\tname := path.SegmentsToString(p.Segments()[:2]...)\n\tfor strings.HasPrefix(name, \"/ipns/\") {\n\t\tif depth == 0 {\n\t\t\treturn namesys.Result{Path: value}, namesys.ErrResolveRecursion\n\t\t}\n\t\tdepth--\n\n\t\tv, ok := m[name]\n\t\tif !ok {\n\t\t\treturn namesys.Result{}, namesys.ErrResolveFailed\n\t\t}\n\t\tvalue = v\n\t\tname = value.String()\n\t}\n\n\tvalue, err := path.Join(value, p.Segments()[2:]...)\n\treturn namesys.Result{Path: value}, err\n}\n\nfunc (m mockNamesys) ResolveAsync(ctx context.Context, p path.Path, opts ...namesys.ResolveOption) <-chan namesys.AsyncResult {\n\tout := make(chan namesys.AsyncResult, 1)\n\tres, err := m.Resolve(ctx, p, opts...)\n\tout <- namesys.AsyncResult{Path: res.Path, TTL: res.TTL, LastMod: res.LastMod, Err: err}\n\tclose(out)\n\treturn out\n}\n\nfunc (m mockNamesys) Publish(ctx context.Context, name ci.PrivKey, value path.Path, opts ...namesys.PublishOption) error {\n\treturn errors.New(\"not implemented for mockNamesys\")\n}\n\nfunc (m mockNamesys) GetResolver(subs string) (namesys.Resolver, bool) {\n\treturn nil, false\n}\n\nfunc newNodeWithMockNamesys(ns mockNamesys) (*core.IpfsNode, error) {\n\tc := config.Config{\n\t\tIdentity: config.Identity{\n\t\t\tPeerID: \"QmTFauExutTsy4XP6JbMFcw2Wa9645HJt2bTqL6qYDCKfe\", // required by offline node\n\t\t},\n\t}\n\tr := &repo.Mock{\n\t\tC: c,\n\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t}\n\tn, err := core.NewNode(context.Background(), &core.BuildCfg{Repo: r})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tn.Namesys = ns\n\treturn n, nil\n}\n\ntype delegatedHandler struct {\n\thttp.Handler\n}\n\nfunc (dh *delegatedHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tdh.Handler.ServeHTTP(w, r)\n}\n\nfunc doWithoutRedirect(req *http.Request) (*http.Response, error) {\n\ttag := \"without-redirect\"\n\tc := &http.Client{\n\t\tCheckRedirect: func(req *http.Request, via []*http.Request) error {\n\t\t\treturn errors.New(tag)\n\t\t},\n\t}\n\tres, err := c.Do(req)\n\tif err != nil && !strings.Contains(err.Error(), tag) {\n\t\treturn nil, err\n\t}\n\treturn res, nil\n}\n\nfunc newTestServerAndNode(t *testing.T, ns mockNamesys) (*httptest.Server, iface.CoreAPI, context.Context) {\n\tn, err := newNodeWithMockNamesys(ns)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// need this variable here since we need to construct handler with\n\t// listener, and server with handler. yay cycles.\n\tdh := &delegatedHandler{}\n\tts := httptest.NewServer(dh)\n\tt.Cleanup(func() { ts.Close() })\n\n\tdh.Handler, err = MakeHandler(n,\n\t\tts.Listener,\n\t\tHostnameOption(),\n\t\tGatewayOption(\"/ipfs\", \"/ipns\"),\n\t\tVersionOption(),\n\t)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tapi, err := coreapi.NewCoreAPI(n)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treturn ts, api, n.Context()\n}\n\nfunc TestVersion(t *testing.T) {\n\tversion.CurrentCommit = \"theshortcommithash\"\n\n\tns := mockNamesys{}\n\tts, _, _ := newTestServerAndNode(t, ns)\n\tt.Logf(\"test server url: %s\", ts.URL)\n\n\treq, err := http.NewRequest(http.MethodGet, ts.URL+\"/version\", nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := doWithoutRedirect(req)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tbody, err := io.ReadAll(res.Body)\n\tif err != nil {\n\t\tt.Fatalf(\"error reading response: %s\", err)\n\t}\n\ts := string(body)\n\n\tif !strings.Contains(s, \"Commit: theshortcommithash\") {\n\t\tt.Fatalf(\"response doesn't contain commit:\\n%s\", s)\n\t}\n\n\tif !strings.Contains(s, \"Client Version: \"+version.GetUserAgentVersion()) {\n\t\tt.Fatalf(\"response doesn't contain client version:\\n%s\", s)\n\t}\n}\n\nfunc TestDeserializedResponsesInheritance(t *testing.T) {\n\tfor _, testCase := range []struct {\n\t\tglobalSetting config.Flag\n\t\tgatewaySetting config.Flag\n\t\texpectedGatewaySetting bool\n\t}{\n\t\t{config.True, config.Default, true},\n\t\t{config.False, config.Default, false},\n\t\t{config.False, config.True, true},\n\t\t{config.True, config.False, false},\n\t} {\n\t\tc := config.Config{\n\t\t\tIdentity: config.Identity{\n\t\t\t\tPeerID: \"QmTFauExutTsy4XP6JbMFcw2Wa9645HJt2bTqL6qYDCKfe\", // required by offline node\n\t\t\t},\n\t\t\tGateway: config.Gateway{\n\t\t\t\tDeserializedResponses: testCase.globalSetting,\n\t\t\t\tPublicGateways: map[string]*config.GatewaySpec{\n\t\t\t\t\t\"example.com\": {\n\t\t\t\t\t\tDeserializedResponses: testCase.gatewaySetting,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\tr := &repo.Mock{\n\t\t\tC: c,\n\t\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t\t}\n\t\tn, err := core.NewNode(context.Background(), &core.BuildCfg{Repo: r})\n\t\tassert.NoError(t, err)\n\n\t\tgwCfg, _, err := getGatewayConfig(n)\n\t\tassert.NoError(t, err)\n\n\t\tassert.Contains(t, gwCfg.PublicGateways, \"example.com\")\n\t\tassert.Equal(t, testCase.expectedGatewaySetting, gwCfg.PublicGateways[\"example.com\"].DeserializedResponses)\n\t}\n}\n" }, { "path": "core/corehttp/logs.go", "content": "package corehttp\n\nimport (\n\t\"bufio\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\nfunc LogOption() ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux.HandleFunc(\"/logs\", func(w http.ResponseWriter, r *http.Request) {\n\t\t\t// The log data comes from an io.Reader, and we need to constantly\n\t\t\t// read from it and then write to the HTTP response.\n\t\t\tpipeReader := logging.NewPipeReader()\n\t\t\tdone := make(chan struct{})\n\n\t\t\t// Close the pipe reader if the request context is canceled. This\n\t\t\t// is necessary to avoiding blocking on reading from the pipe\n\t\t\t// reader when the client terminates the request.\n\t\t\tgo func() {\n\t\t\t\tselect {\n\t\t\t\tcase <-r.Context().Done(): // Client canceled request\n\t\t\t\tcase <-n.Context().Done(): // Node shutdown\n\t\t\t\tcase <-done: // log reader goroutine exitex\n\t\t\t\t}\n\t\t\t\tpipeReader.Close()\n\t\t\t}()\n\n\t\t\terrs := make(chan error, 1)\n\n\t\t\tgo func() {\n\t\t\t\tdefer close(errs)\n\t\t\t\tdefer close(done)\n\n\t\t\t\trdr := bufio.NewReader(pipeReader)\n\t\t\t\tfor {\n\t\t\t\t\t// Read a line of log data and send it to the client.\n\t\t\t\t\tline, err := rdr.ReadString('\\n')\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrs <- fmt.Errorf(\"error reading log message: %s\", err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\t_, err = w.Write([]byte(line))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t// Failed to write to client, probably disconnected.\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tif f, ok := w.(http.Flusher); ok {\n\t\t\t\t\t\tf.Flush()\n\t\t\t\t\t}\n\t\t\t\t\tif r.Context().Err() != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}()\n\t\t\tlog.Info(\"log API client connected\")\n\t\t\terr := <-errs\n\t\t\tif err != nil {\n\t\t\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\t\t\treturn\n\t\t\t}\n\t\t})\n\t\treturn mux, nil\n\t}\n}\n" }, { "path": "core/corehttp/metrics.go", "content": "package corehttp\n\nimport (\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\t\"go.opencensus.io/stats/view\"\n\t\"go.opencensus.io/zpages\"\n\n\tocprom \"contrib.go.opencensus.io/exporter/prometheus\"\n\tprometheus \"github.com/prometheus/client_golang/prometheus\"\n\tpromhttp \"github.com/prometheus/client_golang/prometheus/promhttp\"\n)\n\n// MetricsScrapingOption adds the scraping endpoint which Prometheus uses to fetch metrics.\nfunc MetricsScrapingOption(path string) ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux.Handle(path, promhttp.HandlerFor(prometheus.DefaultGatherer, promhttp.HandlerOpts{}))\n\t\treturn mux, nil\n\t}\n}\n\n// This adds collection of OpenCensus metrics\nfunc MetricsOpenCensusCollectionOption() ServeOption {\n\treturn func(_ *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tlog.Info(\"Init OpenCensus\")\n\n\t\tpromRegistry := prometheus.NewRegistry()\n\t\tpe, err := ocprom.NewExporter(ocprom.Options{\n\t\t\tNamespace: \"ipfs_oc\",\n\t\t\tRegistry: promRegistry,\n\t\t\tOnError: func(err error) {\n\t\t\t\tlog.Errorw(\"OC ERROR\", \"error\", err)\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// register prometheus with opencensus\n\t\tview.RegisterExporter(pe)\n\t\tview.SetReportingPeriod(2 * time.Second)\n\n\t\t// Construct the mux\n\t\tzpages.Handle(mux, \"/debug/metrics/oc/debugz\")\n\t\tmux.Handle(\"/debug/metrics/oc\", pe)\n\n\t\treturn mux, nil\n\t}\n}\n\n// MetricsOpenCensusDefaultPrometheusRegistry registers the default prometheus\n// registry as an exporter to OpenCensus metrics. This means that OpenCensus\n// metrics will show up in the prometheus metrics endpoint\nfunc MetricsOpenCensusDefaultPrometheusRegistry() ServeOption {\n\treturn func(_ *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tlog.Info(\"Init OpenCensus with default prometheus registry\")\n\n\t\tpe, err := ocprom.NewExporter(ocprom.Options{\n\t\t\tRegistry: prometheus.DefaultRegisterer.(*prometheus.Registry),\n\t\t\tOnError: func(err error) {\n\t\t\t\tlog.Errorw(\"OC default registry ERROR\", \"error\", err)\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// register prometheus with opencensus\n\t\tview.RegisterExporter(pe)\n\n\t\treturn mux, nil\n\t}\n}\n\n// MetricsCollectionOption adds collection of net/http-related metrics.\nfunc MetricsCollectionOption(handlerName string) ServeOption {\n\treturn func(_ *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\t// Adapted from github.com/prometheus/client_golang/prometheus/http.go\n\t\t// Work around https://github.com/prometheus/client_golang/pull/311\n\t\topts := prometheus.SummaryOpts{\n\t\t\tNamespace: \"ipfs\",\n\t\t\tSubsystem: \"http\",\n\t\t\tConstLabels: prometheus.Labels{\"handler\": handlerName},\n\t\t\tObjectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},\n\t\t}\n\n\t\t// Legacy metric - new metrics are provided by boxo/gateway as gw_http_responses_total\n\t\treqCnt := prometheus.NewCounterVec(\n\t\t\tprometheus.CounterOpts{\n\t\t\t\tNamespace: opts.Namespace,\n\t\t\t\tSubsystem: opts.Subsystem,\n\t\t\t\tName: \"requests_total\",\n\t\t\t\tHelp: \"Total number of HTTP requests made.\",\n\t\t\t\tConstLabels: opts.ConstLabels,\n\t\t\t},\n\t\t\t[]string{\"method\", \"code\"},\n\t\t)\n\t\tif err := prometheus.Register(reqCnt); err != nil {\n\t\t\tif are, ok := err.(prometheus.AlreadyRegisteredError); ok {\n\t\t\t\treqCnt = are.ExistingCollector.(*prometheus.CounterVec)\n\t\t\t} else {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\topts.Name = \"request_duration_seconds\"\n\t\topts.Help = \"The HTTP request latencies in seconds.\"\n\t\treqDur := prometheus.NewSummaryVec(opts, nil)\n\t\tif err := prometheus.Register(reqDur); err != nil {\n\t\t\tif are, ok := err.(prometheus.AlreadyRegisteredError); ok {\n\t\t\t\treqDur = are.ExistingCollector.(*prometheus.SummaryVec)\n\t\t\t} else {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\topts.Name = \"request_size_bytes\"\n\t\topts.Help = \"The HTTP request sizes in bytes.\"\n\t\treqSz := prometheus.NewSummaryVec(opts, nil)\n\t\tif err := prometheus.Register(reqSz); err != nil {\n\t\t\tif are, ok := err.(prometheus.AlreadyRegisteredError); ok {\n\t\t\t\treqSz = are.ExistingCollector.(*prometheus.SummaryVec)\n\t\t\t} else {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\topts.Name = \"response_size_bytes\"\n\t\topts.Help = \"The HTTP response sizes in bytes.\"\n\t\tresSz := prometheus.NewSummaryVec(opts, nil)\n\t\tif err := prometheus.Register(resSz); err != nil {\n\t\t\tif are, ok := err.(prometheus.AlreadyRegisteredError); ok {\n\t\t\t\tresSz = are.ExistingCollector.(*prometheus.SummaryVec)\n\t\t\t} else {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\t// Construct the mux\n\t\tchildMux := http.NewServeMux()\n\t\tvar promMux http.Handler = childMux\n\t\tpromMux = promhttp.InstrumentHandlerResponseSize(resSz, promMux)\n\t\tpromMux = promhttp.InstrumentHandlerRequestSize(reqSz, promMux)\n\t\tpromMux = promhttp.InstrumentHandlerDuration(reqDur, promMux)\n\t\tpromMux = promhttp.InstrumentHandlerCounter(reqCnt, promMux)\n\t\tmux.Handle(\"/\", promMux)\n\n\t\treturn childMux, nil\n\t}\n}\n\nvar peersTotalMetric = prometheus.NewDesc(\n\tprometheus.BuildFQName(\"ipfs\", \"p2p\", \"peers_total\"),\n\t\"Number of connected peers\",\n\t[]string{\"transport\"},\n\tnil,\n)\n\ntype IpfsNodeCollector struct {\n\tNode *core.IpfsNode\n}\n\nfunc (IpfsNodeCollector) Describe(ch chan<- *prometheus.Desc) {\n\tch <- peersTotalMetric\n}\n\nfunc (c IpfsNodeCollector) Collect(ch chan<- prometheus.Metric) {\n\tfor tr, val := range c.PeersTotalValues() {\n\t\tch <- prometheus.MustNewConstMetric(\n\t\t\tpeersTotalMetric,\n\t\t\tprometheus.GaugeValue,\n\t\t\tval,\n\t\t\ttr,\n\t\t)\n\t}\n}\n\nfunc (c IpfsNodeCollector) PeersTotalValues() map[string]float64 {\n\tvals := make(map[string]float64)\n\tif c.Node.PeerHost == nil {\n\t\treturn vals\n\t}\n\tfor _, peerID := range c.Node.PeerHost.Network().Peers() {\n\t\t// Each peer may have more than one connection (see for an explanation\n\t\t// https://github.com/libp2p/go-libp2p-swarm/commit/0538806), so we grab\n\t\t// only one, the first (an arbitrary and non-deterministic choice), which\n\t\t// according to ConnsToPeer is the oldest connection in the list\n\t\t// (https://github.com/libp2p/go-libp2p-swarm/blob/v0.2.6/swarm.go#L362-L364).\n\t\tconns := c.Node.PeerHost.Network().ConnsToPeer(peerID)\n\t\tif len(conns) == 0 {\n\t\t\tcontinue\n\t\t}\n\t\ttr := \"\"\n\t\tfor _, proto := range conns[0].RemoteMultiaddr().Protocols() {\n\t\t\ttr = tr + \"/\" + proto.Name\n\t\t}\n\t\tvals[tr] = vals[tr] + 1\n\t}\n\treturn vals\n}\n" }, { "path": "core/corehttp/metrics_test.go", "content": "package corehttp\n\nimport (\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/core\"\n\n\tinet \"github.com/libp2p/go-libp2p/core/network\"\n\tbhost \"github.com/libp2p/go-libp2p/p2p/host/basic\"\n\tswarmt \"github.com/libp2p/go-libp2p/p2p/net/swarm/testing\"\n)\n\n// This test is based on go-libp2p/p2p/net/swarm.TestConnectednessCorrect\n// It builds 4 nodes and connects them, one being the sole center.\n// Then it checks that the center reports the correct number of peers.\nfunc TestPeersTotal(t *testing.T) {\n\tctx := context.Background()\n\n\thosts := make([]*bhost.BasicHost, 4)\n\tfor i := range 4 {\n\t\tvar err error\n\t\thosts[i], err = bhost.NewHost(swarmt.GenSwarm(t), nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tdial := func(a, b inet.Network) {\n\t\tswarmt.DivulgeAddresses(b, a)\n\t\tif _, err := a.DialPeer(ctx, b.LocalPeer()); err != nil {\n\t\t\tt.Fatalf(\"Failed to dial: %s\", err)\n\t\t}\n\t}\n\n\tdial(hosts[0].Network(), hosts[1].Network())\n\tdial(hosts[0].Network(), hosts[2].Network())\n\tdial(hosts[0].Network(), hosts[3].Network())\n\n\t// there's something wrong with dial, i think. it's not finishing\n\t// completely. there must be some async stuff.\n\t<-time.After(100 * time.Millisecond)\n\n\tnode := &core.IpfsNode{PeerHost: hosts[0]}\n\tcollector := IpfsNodeCollector{Node: node}\n\tpeersTransport := collector.PeersTotalValues()\n\tif len(peersTransport) > 2 {\n\t\tt.Fatalf(\"expected at most 2 peers transport (tcp and upd/quic), got %d, transport map %v\",\n\t\t\tlen(peersTransport), peersTransport)\n\t}\n\ttotalPeers := peersTransport[\"/ip4/tcp\"] + peersTransport[\"/ip4/udp/quic-v1\"]\n\tif totalPeers != 3 {\n\t\tt.Fatalf(\"expected 3 peers in either tcp or upd/quic transport, got %f\", totalPeers)\n\t}\n}\n" }, { "path": "core/corehttp/mutex_profile.go", "content": "package corehttp\n\nimport (\n\t\"net\"\n\t\"net/http\"\n\t\"runtime\"\n\t\"strconv\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\n// MutexFractionOption allows to set runtime.SetMutexProfileFraction via HTTP\n// using POST request with parameter 'fraction'.\nfunc MutexFractionOption(path string) ServeOption {\n\treturn func(_ *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux.HandleFunc(path, func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif r.Method != http.MethodPost {\n\t\t\t\thttp.Error(w, \"only POST allowed\", http.StatusMethodNotAllowed)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err := r.ParseForm(); err != nil {\n\t\t\t\thttp.Error(w, err.Error(), http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tasfr := r.Form.Get(\"fraction\")\n\t\t\tif len(asfr) == 0 {\n\t\t\t\thttp.Error(w, \"parameter 'fraction' must be set\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tfr, err := strconv.Atoi(asfr)\n\t\t\tif err != nil {\n\t\t\t\thttp.Error(w, err.Error(), http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tlog.Infof(\"Setting MutexProfileFraction to %d\", fr)\n\t\t\truntime.SetMutexProfileFraction(fr)\n\t\t})\n\n\t\treturn mux, nil\n\t}\n}\n\n// BlockProfileRateOption allows to set runtime.SetBlockProfileRate via HTTP\n// using POST request with parameter 'rate'.\n// The profiler tries to sample 1 event every nanoseconds.\n// If rate == 1, then the profiler samples every blocking event.\n// To disable, set rate = 0.\nfunc BlockProfileRateOption(path string) ServeOption {\n\treturn func(_ *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux.HandleFunc(path, func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif r.Method != http.MethodPost {\n\t\t\t\thttp.Error(w, \"only POST allowed\", http.StatusMethodNotAllowed)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err := r.ParseForm(); err != nil {\n\t\t\t\thttp.Error(w, err.Error(), http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trateStr := r.Form.Get(\"rate\")\n\t\t\tif len(rateStr) == 0 {\n\t\t\t\thttp.Error(w, \"parameter 'rate' must be set\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trate, err := strconv.Atoi(rateStr)\n\t\t\tif err != nil {\n\t\t\t\thttp.Error(w, err.Error(), http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tlog.Infof(\"Setting BlockProfileRate to %d\", rate)\n\t\t\truntime.SetBlockProfileRate(rate)\n\t\t})\n\t\treturn mux, nil\n\t}\n}\n" }, { "path": "core/corehttp/option_test.go", "content": "package corehttp\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"testing\"\n\n\tversion \"github.com/ipfs/kubo\"\n)\n\ntype testcasecheckversion struct {\n\tuserAgent string\n\turi string\n\tshouldHandle bool\n\tresponseBody string\n\tresponseCode int\n}\n\nfunc (tc testcasecheckversion) body() string {\n\tif !tc.shouldHandle && tc.responseBody == \"\" {\n\t\treturn fmt.Sprintf(\"%s (%s != %s)\\n\", errAPIVersionMismatch, version.ApiVersion, tc.userAgent)\n\t}\n\n\treturn tc.responseBody\n}\n\nfunc TestCheckVersionOption(t *testing.T) {\n\ttcs := []testcasecheckversion{\n\t\t{\"/go-ipfs/0.1/\", APIPath + \"/test/\", false, \"\", http.StatusBadRequest},\n\t\t{\"/go-ipfs/0.1/\", APIPath + \"/version\", true, \"check!\", http.StatusOK},\n\t\t{version.ApiVersion, APIPath + \"/test\", true, \"check!\", http.StatusOK},\n\t\t{\"Mozilla Firefox/no go-ipfs node\", APIPath + \"/test\", true, \"check!\", http.StatusOK},\n\t\t{\"/go-ipfs/0.1/\", \"/webui\", true, \"check!\", http.StatusOK},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Logf(\"%#v\", tc)\n\t\tr := httptest.NewRequest(http.MethodPost, tc.uri, nil)\n\t\tr.Header.Add(\"User-Agent\", tc.userAgent) // old version, should fail\n\n\t\tcalled := false\n\t\troot := http.NewServeMux()\n\t\tmux, err := CheckVersionOption()(nil, nil, root)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tmux.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {\n\t\t\tcalled = true\n\t\t\tif !tc.shouldHandle {\n\t\t\t\tt.Error(\"handler was called even though version didn't match\")\n\t\t\t}\n\t\t\tif _, err := io.WriteString(w, \"check!\"); err != nil {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t})\n\n\t\tw := httptest.NewRecorder()\n\n\t\troot.ServeHTTP(w, r)\n\n\t\tif tc.shouldHandle && !called {\n\t\t\tt.Error(\"handler wasn't called even though it should have\")\n\t\t}\n\n\t\tif w.Code != tc.responseCode {\n\t\t\tt.Errorf(\"expected code %d but got %d\", tc.responseCode, w.Code)\n\t\t}\n\n\t\tif w.Body.String() != tc.body() {\n\t\t\tt.Errorf(\"expected error message %q, got %q\", tc.body(), w.Body.String())\n\t\t}\n\t}\n}\n" }, { "path": "core/corehttp/p2p_proxy.go", "content": "package corehttp\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/httputil\"\n\t\"net/url\"\n\t\"strings\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\n\tp2phttp \"github.com/libp2p/go-libp2p-http\"\n\tprotocol \"github.com/libp2p/go-libp2p/core/protocol\"\n)\n\n// P2PProxyOption is an endpoint for proxying a HTTP request to another ipfs peer\nfunc P2PProxyOption() ServeOption {\n\treturn func(ipfsNode *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tmux.HandleFunc(\"/p2p/\", func(w http.ResponseWriter, request *http.Request) {\n\t\t\t// parse request\n\t\t\tparsedRequest, err := parseRequest(request)\n\t\t\tif err != nil {\n\t\t\t\thandleError(w, \"failed to parse request\", err, 400)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequest.Host = \"\" // Let URL's Host take precedence.\n\t\t\trequest.URL.Path = parsedRequest.httpPath\n\t\t\ttarget, err := url.Parse(fmt.Sprintf(\"libp2p://%s\", parsedRequest.target))\n\t\t\tif err != nil {\n\t\t\t\thandleError(w, \"failed to parse url\", err, 400)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trt := p2phttp.NewTransport(ipfsNode.PeerHost, p2phttp.ProtocolOption(parsedRequest.name))\n\t\t\tproxy := &httputil.ReverseProxy{\n\t\t\t\tTransport: rt,\n\t\t\t\tRewrite: func(r *httputil.ProxyRequest) {\n\t\t\t\t\tr.SetURL(target)\n\t\t\t\t\tr.SetXForwarded()\n\t\t\t\t},\n\t\t\t}\n\t\t\tproxy.ServeHTTP(w, request)\n\t\t})\n\t\treturn mux, nil\n\t}\n}\n\ntype proxyRequest struct {\n\ttarget string\n\tname protocol.ID\n\thttpPath string // path to send to the proxy-host\n}\n\n// from the url path parse the peer-ID, name and http path\n// /p2p/$peer_id/http/$http_path\n// or\n// /p2p/$peer_id/x/$protocol/http/$http_path\nfunc parseRequest(request *http.Request) (*proxyRequest, error) {\n\tpath := request.URL.Path\n\n\tsplit := strings.SplitN(path, \"/\", 5)\n\tif len(split) < 5 {\n\t\treturn nil, fmt.Errorf(\"invalid request path '%s'\", path)\n\t}\n\n\tif _, err := peer.Decode(split[2]); err != nil {\n\t\treturn nil, fmt.Errorf(\"invalid request path '%s'\", path)\n\t}\n\n\tif split[3] == \"http\" {\n\t\treturn &proxyRequest{split[2], protocol.ID(\"/http\"), split[4]}, nil\n\t}\n\n\tsplit = strings.SplitN(path, \"/\", 7)\n\tif len(split) < 7 || split[3] != \"x\" || split[5] != \"http\" {\n\t\treturn nil, fmt.Errorf(\"invalid request path '%s'\", path)\n\t}\n\n\treturn &proxyRequest{split[2], protocol.ID(\"/x/\" + split[4] + \"/http\"), split[6]}, nil\n}\n\nfunc handleError(w http.ResponseWriter, msg string, err error, code int) {\n\thttp.Error(w, fmt.Sprintf(\"%s: %s\", msg, err), code)\n}\n" }, { "path": "core/corehttp/p2p_proxy_test.go", "content": "package corehttp\n\nimport (\n\t\"net/http\"\n\t\"strings\"\n\t\"testing\"\n\n\tprotocol \"github.com/libp2p/go-libp2p/core/protocol\"\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype TestCase struct {\n\turlprefix string\n\ttarget string\n\tname string\n\tpath string\n}\n\nvar validtestCases = []TestCase{\n\t{\"http://localhost:5001\", \"QmT8JtU54XSmC38xSb1XHFSMm775VuTeajg7LWWWTAwzxT\", \"/http\", \"path/to/index.txt\"},\n\t{\"http://localhost:5001\", \"QmT8JtU54XSmC38xSb1XHFSMm775VuTeajg7LWWWTAwzxT\", \"/x/custom/http\", \"path/to/index.txt\"},\n\t{\"http://localhost:5001\", \"QmT8JtU54XSmC38xSb1XHFSMm775VuTeajg7LWWWTAwzxT\", \"/x/custom/http\", \"http/path/to/index.txt\"},\n}\n\nfunc TestParseRequest(t *testing.T) {\n\tfor _, tc := range validtestCases {\n\t\turl := tc.urlprefix + \"/p2p/\" + tc.target + tc.name + \"/\" + tc.path\n\t\treq, _ := http.NewRequest(http.MethodGet, url, strings.NewReader(\"\"))\n\n\t\tparsed, err := parseRequest(req)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, tc.path, parsed.httpPath, \"proxy request path\")\n\t\trequire.Equal(t, protocol.ID(tc.name), parsed.name, \"proxy request name\")\n\t\trequire.Equal(t, tc.target, parsed.target, \"proxy request peer-id\")\n\t}\n}\n\nvar invalidtestCases = []string{\n\t\"http://localhost:5001/p2p/http/foobar\",\n\t\"http://localhost:5001/p2p/QmT8JtU54XSmC38xSb1XHFSMm775VuTeajg7LWWWTAwzxT/x/custom/foobar\",\n}\n\nfunc TestParseRequestInvalidPath(t *testing.T) {\n\tfor _, tc := range invalidtestCases {\n\t\turl := tc\n\t\treq, _ := http.NewRequest(http.MethodGet, url, strings.NewReader(\"\"))\n\n\t\t_, err := parseRequest(req)\n\t\trequire.Error(t, err)\n\t}\n}\n" }, { "path": "core/corehttp/redirect.go", "content": "package corehttp\n\nimport (\n\t\"net\"\n\t\"net/http\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\nfunc RedirectOption(path string, redirect string) ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\tcfg, err := n.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\thandler := &redirectHandler{redirect, cfg.API.HTTPHeaders}\n\n\t\tif len(path) > 0 {\n\t\t\tmux.Handle(\"/\"+path+\"/\", handler)\n\t\t} else {\n\t\t\tmux.Handle(\"/\", handler)\n\t\t}\n\t\treturn mux, nil\n\t}\n}\n\ntype redirectHandler struct {\n\tpath string\n\theaders map[string][]string\n}\n\nfunc (i *redirectHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tfor k, v := range i.headers {\n\t\tw.Header()[http.CanonicalHeaderKey(k)] = v\n\t}\n\n\thttp.Redirect(w, r, i.path, http.StatusFound)\n}\n" }, { "path": "core/corehttp/routing.go", "content": "package corehttp\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/gateway\"\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/routing/http/server\"\n\t\"github.com/ipfs/boxo/routing/http/types\"\n\t\"github.com/ipfs/boxo/routing/http/types/iter\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcore \"github.com/ipfs/kubo/core\"\n\tdht \"github.com/libp2p/go-libp2p-kad-dht\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n)\n\nfunc RoutingOption() ServeOption {\n\treturn func(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\t\t_, headers, err := getGatewayConfig(n)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\thandler := server.Handler(&contentRouter{n})\n\t\thandler = gateway.NewHeaders(headers).ApplyCors().Wrap(handler)\n\t\tmux.Handle(\"/routing/v1/\", handler)\n\t\treturn mux, nil\n\t}\n}\n\ntype contentRouter struct {\n\tn *core.IpfsNode\n}\n\nfunc (r *contentRouter) FindProviders(ctx context.Context, key cid.Cid, limit int) (iter.ResultIter[types.Record], error) {\n\tctx, cancel := context.WithCancel(ctx)\n\tch := r.n.Routing.FindProvidersAsync(ctx, key, limit)\n\treturn iter.ToResultIter[types.Record](&peerChanIter{\n\t\tch: ch,\n\t\tcancel: cancel,\n\t}), nil\n}\n\n// nolint deprecated\nfunc (r *contentRouter) ProvideBitswap(ctx context.Context, req *server.BitswapWriteProvideRequest) (time.Duration, error) {\n\treturn 0, routing.ErrNotSupported\n}\n\nfunc (r *contentRouter) FindPeers(ctx context.Context, pid peer.ID, limit int) (iter.ResultIter[*types.PeerRecord], error) {\n\tctx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\taddr, err := r.n.Routing.FindPeer(ctx, pid)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trec := &types.PeerRecord{\n\t\tSchema: types.SchemaPeer,\n\t\tID: &addr.ID,\n\t}\n\n\tfor _, addr := range addr.Addrs {\n\t\trec.Addrs = append(rec.Addrs, types.Multiaddr{Multiaddr: addr})\n\t}\n\n\treturn iter.ToResultIter[*types.PeerRecord](iter.FromSlice[*types.PeerRecord]([]*types.PeerRecord{rec})), nil\n}\n\nfunc (r *contentRouter) GetIPNS(ctx context.Context, name ipns.Name) (*ipns.Record, error) {\n\tctx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\traw, err := r.n.Routing.GetValue(ctx, string(name.RoutingKey()))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ipns.UnmarshalRecord(raw)\n}\n\nfunc (r *contentRouter) PutIPNS(ctx context.Context, name ipns.Name, record *ipns.Record) error {\n\tctx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\traw, err := ipns.MarshalRecord(record)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// The caller guarantees that name matches the record. This is double checked\n\t// by the internals of PutValue.\n\treturn r.n.Routing.PutValue(ctx, string(name.RoutingKey()), raw)\n}\n\nfunc (r *contentRouter) GetClosestPeers(ctx context.Context, key cid.Cid) (iter.ResultIter[*types.PeerRecord], error) {\n\t// Per the spec, if the peer ID is empty, we should use self.\n\tif key == cid.Undef {\n\t\treturn nil, errors.New(\"GetClosestPeers key is undefined\")\n\t}\n\n\tkeyStr := string(key.Hash())\n\tvar peers []peer.ID\n\tvar err error\n\n\tif r.n.DHTClient == nil {\n\t\treturn nil, fmt.Errorf(\"GetClosestPeers not supported: DHT is not available\")\n\t}\n\n\tswitch dhtClient := r.n.DHTClient.(type) {\n\tcase *dual.DHT:\n\t\t// Only use WAN DHT for public HTTP Routing API.\n\t\t// LAN DHT contains private network peers that should not be exposed publicly.\n\t\tif dhtClient.WAN == nil {\n\t\t\treturn nil, fmt.Errorf(\"GetClosestPeers not supported: WAN DHT is not available\")\n\t\t}\n\t\tpeers, err = dhtClient.WAN.GetClosestPeers(ctx, keyStr)\n\tcase *fullrt.FullRT:\n\t\tpeers, err = dhtClient.GetClosestPeers(ctx, keyStr)\n\tcase *dht.IpfsDHT:\n\t\tpeers, err = dhtClient.GetClosestPeers(ctx, keyStr)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"GetClosestPeers not supported for DHT type %T\", r.n.DHTClient)\n\t}\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// We have some DHT-closest peers. Find addresses for them.\n\t// The addresses should be in the peerstore.\n\trecords := make([]*types.PeerRecord, 0, len(peers))\n\tfor _, p := range peers {\n\t\taddrs := r.n.Peerstore.Addrs(p)\n\t\trAddrs := make([]types.Multiaddr, len(addrs))\n\t\tfor i, addr := range addrs {\n\t\t\trAddrs[i] = types.Multiaddr{Multiaddr: addr}\n\t\t}\n\t\trecord := types.PeerRecord{\n\t\t\tID: &p,\n\t\t\tSchema: types.SchemaPeer,\n\t\t\tAddrs: rAddrs,\n\t\t}\n\t\trecords = append(records, &record)\n\t}\n\n\treturn iter.ToResultIter(iter.FromSlice(records)), nil\n}\n\ntype peerChanIter struct {\n\tch <-chan peer.AddrInfo\n\tcancel context.CancelFunc\n\tnext *peer.AddrInfo\n}\n\nfunc (it *peerChanIter) Next() bool {\n\taddr, ok := <-it.ch\n\tif ok {\n\t\tit.next = &addr\n\t\treturn true\n\t}\n\tit.next = nil\n\treturn false\n}\n\nfunc (it *peerChanIter) Val() types.Record {\n\tif it.next == nil {\n\t\treturn nil\n\t}\n\n\trec := &types.PeerRecord{\n\t\tSchema: types.SchemaPeer,\n\t\tID: &it.next.ID,\n\t}\n\n\tfor _, addr := range it.next.Addrs {\n\t\trec.Addrs = append(rec.Addrs, types.Multiaddr{Multiaddr: addr})\n\t}\n\n\treturn rec\n}\n\nfunc (it *peerChanIter) Close() error {\n\tit.cancel()\n\treturn nil\n}\n" }, { "path": "core/corehttp/webui.go", "content": "package corehttp\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"strings\"\n\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/config\"\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\n// WebUI version confirmed to work with this Kubo version\nconst WebUIPath = \"/ipfs/bafybeihxglpcfyarpm7apn7xpezbuoqgk3l5chyk7w4gvrjwk45rqohlmm\" // v4.12.0\n\n// WebUIPaths is a list of all past webUI paths.\nvar WebUIPaths = []string{\n\tWebUIPath,\n\t\"/ipfs/bafybeiddnr2jz65byk67sjt6jsu6g7tueddr7odhzzpzli3rgudlbnc6iq\", // v4.11.1\n\t\"/ipfs/bafybeidfgbcqy435sdbhhejifdxq4o64tlsezajc272zpyxcsmz47uyc64\", // v4.11.0\n\t\"/ipfs/bafybeidsjptidvb6wf6benznq2pxgnt5iyksgtecpmjoimlmswhtx2u5ua\", // v4.10.0\n\t\"/ipfs/bafybeicg7e6o2eszkfdzxg5233gmuip2a7kfzoloh7voyvt2r6ivdet54u\", // v4.9.1\n\t\"/ipfs/bafybeifplj2s3yegn7ko7tdnwpoxa4c5uaqnk2ajnw5geqm34slcj6b6mu\", // v4.8.0\n\t\"/ipfs/bafybeibfd5kbebqqruouji6ct5qku3tay273g7mt24mmrfzrsfeewaal5y\", // v4.7.0\n\t\"/ipfs/bafybeibpaa5kqrj4gkemiswbwndjqiryl65cks64ypwtyerxixu56gnvvm\", // v4.6.0\n\t\"/ipfs/bafybeiata4qg7xjtwgor6r5dw63jjxyouenyromrrb4lrewxrlvav7gzgi\", // v4.5.0\n\t\"/ipfs/bafybeigp3zm7cqoiciqk5anlheenqjsgovp7j7zq6hah4nu6iugdgb4nby\", // v4.4.2\n\t\"/ipfs/bafybeiatztgdllxnp5p6zu7bdwhjmozsmd7jprff4bdjqjljxtylitvss4\", // v4.4.1\n\t\"/ipfs/bafybeibgic2ex3fvzkinhy6k6aqyv3zy2o7bkbsmrzvzka24xetv7eeadm\", // v4.4.0\n\t\"/ipfs/bafybeid4uxz7klxcu3ffsnmn64r7ihvysamlj4ohl5h2orjsffuegcpaeq\", // v4.3.3\n\t\"/ipfs/bafybeif6abowqcavbkz243biyh7pde7ick5kkwwytrh7pd2hkbtuqysjxy\", // v4.3.2\n\t\"/ipfs/bafybeihatzsgposbr3hrngo42yckdyqcc56yean2rynnwpzxstvdlphxf4\",\n\t\"/ipfs/bafybeigggyffcf6yfhx5irtwzx3cgnk6n3dwylkvcpckzhqqrigsxowjwe\",\n\t\"/ipfs/bafybeidf7cpkwsjkq6xs3r6fbbxghbugilx3jtezbza7gua3k5wjixpmba\",\n\t\"/ipfs/bafybeiamycmd52xvg6k3nzr6z3n33de6a2teyhquhj4kspdtnvetnkrfim\",\n\t\"/ipfs/bafybeieqdeoqkf7xf4aozd524qncgiloh33qgr25lyzrkusbcre4c3fxay\",\n\t\"/ipfs/bafybeicyp7ssbnj3hdzehcibmapmpuc3atrsc4ch3q6acldfh4ojjdbcxe\",\n\t\"/ipfs/bafybeigs6d53gpgu34553mbi5bbkb26e4ikruoaaar75jpfdywpup2r3my\",\n\t\"/ipfs/bafybeic4gops3d3lyrisqku37uio33nvt6fqxvkxihrwlqsuvf76yln4fm\",\n\t\"/ipfs/bafybeifeqt7mvxaniphyu2i3qhovjaf3sayooxbh5enfdqtiehxjv2ldte\", // v2.22.0\n\t\"/ipfs/bafybeiequgo72mrvuml56j4gk7crewig5bavumrrzhkqbim6b3s2yqi7ty\",\n\t\"/ipfs/bafybeibjbq3tmmy7wuihhhwvbladjsd3gx3kfjepxzkq6wylik6wc3whzy\", // v2.20.0\n\t\"/ipfs/bafybeiavrvt53fks6u32n5p2morgblcmck4bh4ymf4rrwu7ah5zsykmqqa\", // v2.19.0\n\t\"/ipfs/bafybeiageaoxg6d7npaof6eyzqbwvbubyler7bq44hayik2hvqcggg7d2y\", // v2.18.1\n\t\"/ipfs/bafybeidb5eryh72zajiokdggzo7yct2d6hhcflncji5im2y5w26uuygdsm\", // v2.18.0\n\t\"/ipfs/bafybeibozpulxtpv5nhfa2ue3dcjx23ndh3gwr5vwllk7ptoyfwnfjjr4q\", // v2.15.1\n\t\"/ipfs/bafybeiednzu62vskme5wpoj4bjjikeg3xovfpp4t7vxk5ty2jxdi4mv4bu\", // v2.15.0\n\t\"/ipfs/bafybeihcyruaeza7uyjd6ugicbcrqumejf6uf353e5etdkhotqffwtguva\", // v2.13.0\n\t\"/ipfs/bafybeiflkjt66aetfgcrgvv75izymd5kc47g6luepqmfq6zsf5w6ueth6y\",\n\t\"/ipfs/bafybeid26vjplsejg7t3nrh7mxmiaaxriebbm4xxrxxdunlk7o337m5sqq\",\n\t\"/ipfs/bafybeif4zkmu7qdhkpf3pnhwxipylqleof7rl6ojbe7mq3fzogz6m4xk3i\", // v2.11.4\n\t\"/ipfs/bafybeianwe4vy7sprht5sm3hshvxjeqhwcmvbzq73u55sdhqngmohkjgs4\",\n\t\"/ipfs/bafybeicitin4p7ggmyjaubqpi3xwnagrwarsy6hiihraafk5rcrxqxju6m\",\n\t\"/ipfs/bafybeihpetclqvwb4qnmumvcn7nh4pxrtugrlpw4jgjpqicdxsv7opdm6e\",\n\t\"/ipfs/bafybeibnnxd4etu4tq5fuhu3z5p4rfu3buabfkeyr3o3s4h6wtesvvw6mu\",\n\t\"/ipfs/bafybeid6luolenf4fcsuaw5rgdwpqbyerce4x3mi3hxfdtp5pwco7h7qyq\",\n\t\"/ipfs/bafybeigkbbjnltbd4ewfj7elajsbnjwinyk6tiilczkqsibf3o7dcr6nn4\",\n\t\"/ipfs/bafybeicp23nbcxtt2k2twyfivcbrc6kr3l5lnaiv3ozvwbemtrb7v52r6i\",\n\t\"/ipfs/bafybeidatpz2hli6fgu3zul5woi27ujesdf5o5a7bu622qj6ugharciwjq\",\n\t\"/ipfs/QmfQkD8pBSBCBxWEwFSu4XaDVSWK6bjnNuaWZjMyQbyDub\",\n\t\"/ipfs/QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ\",\n\t\"/ipfs/QmenEBWcAk3tN94fSKpKFtUMwty1qNwSYw3DMDFV6cPBXA\",\n\t\"/ipfs/QmUnXcWZC5Ve21gUseouJsH5mLAyz5JPp8aHsg8qVUUK8e\",\n\t\"/ipfs/QmSDgpiHco5yXdyVTfhKxr3aiJ82ynz8V14QcGKicM3rVh\",\n\t\"/ipfs/QmRuvWJz1Fc8B9cTsAYANHTXqGmKR9DVfY5nvMD1uA2WQ8\",\n\t\"/ipfs/QmQLXHs7K98JNQdWrBB2cQLJahPhmupbDjRuH1b9ibmwVa\",\n\t\"/ipfs/QmXX7YRpU7nNBKfw75VG7Y1c3GwpSAGHRev67XVPgZFv9R\",\n\t\"/ipfs/QmXdu7HWdV6CUaUabd9q2ZeA4iHZLVyDRj3Gi4dsJsWjbr\",\n\t\"/ipfs/QmaaqrHyAQm7gALkRW8DcfGX3u8q9rWKnxEMmf7m9z515w\",\n\t\"/ipfs/QmSHDxWsMPuJQKWmVA1rB5a3NX2Eme5fPqNb63qwaqiqSp\",\n\t\"/ipfs/QmctngrQAt9fjpQUZr7Bx3BsXUcif52eZGTizWhvcShsjz\",\n\t\"/ipfs/QmS2HL9v5YeKgQkkWMvs1EMnFtUowTEdFfSSeMT4pos1e6\",\n\t\"/ipfs/QmR9MzChjp1MdFWik7NjEjqKQMzVmBkdK3dz14A6B5Cupm\",\n\t\"/ipfs/QmRyWyKWmphamkMRnJVjUTzSFSAAZowYP4rnbgnfMXC9Mr\",\n\t\"/ipfs/QmU3o9bvfenhTKhxUakbYrLDnZU7HezAVxPM6Ehjw9Xjqy\",\n\t\"/ipfs/QmPhnvn747LqwPYMJmQVorMaGbMSgA7mRRoyyZYz3DoZRQ\",\n\t\"/ipfs/QmQNHd1suZTktPRhP7DD4nKWG46ZRSxkwHocycHVrK3dYW\",\n\t\"/ipfs/QmNyMYhwJUS1cVvaWoVBhrW8KPj1qmie7rZcWo8f1Bvkhz\",\n\t\"/ipfs/QmVTiRTQ72qiH4usAGT4c6qVxCMv4hFMUH9fvU6mktaXdP\",\n\t\"/ipfs/QmYcP4sp1nraBiCYi6i9kqdaKobrK32yyMpTrM5JDA8a2C\",\n\t\"/ipfs/QmUtMmxgHnDvQq4bpH6Y9MaLN1hpfjJz5LZcq941BEqEXs\",\n\t\"/ipfs/QmPURAjo3oneGH53ovt68UZEBvsc8nNmEhQZEpsVEQUMZE\",\n\t\"/ipfs/QmeSXt32frzhvewLKwA1dePTSjkTfGVwTh55ZcsJxrCSnk\",\n\t\"/ipfs/QmcjeTciMNgEBe4xXvEaA4TQtwTRkXucx7DmKWViXSmX7m\",\n\t\"/ipfs/QmfNbSskgvTXYhuqP8tb9AKbCkyRcCy3WeiXwD9y5LeoqK\",\n\t\"/ipfs/QmPkojhjJkJ5LEGBDrAvdftrjAYmi9GU5Cq27mWvZTDieW\",\n\t\"/ipfs/Qmexhq2sBHnXQbvyP2GfUdbnY7HCagH2Mw5vUNSBn2nxip\",\n}\n\n// WebUIOption provides the WebUI handler for the RPC API.\nfunc WebUIOption(n *core.IpfsNode, _ net.Listener, mux *http.ServeMux) (*http.ServeMux, error) {\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thandler := &webUIHandler{\n\t\theaders: cfg.API.HTTPHeaders,\n\t\tnode: n,\n\t\tnoFetch: cfg.Gateway.NoFetch,\n\t\tdeserializedResponses: cfg.Gateway.DeserializedResponses.WithDefault(config.DefaultDeserializedResponses),\n\t}\n\n\tmux.Handle(\"/webui/\", handler)\n\treturn mux, nil\n}\n\ntype webUIHandler struct {\n\theaders map[string][]string\n\tnode *core.IpfsNode\n\tnoFetch bool\n\tdeserializedResponses bool\n}\n\nfunc (h *webUIHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tfor k, v := range h.headers {\n\t\tw.Header()[http.CanonicalHeaderKey(k)] = v\n\t}\n\n\t// Check if WebUI is incompatible with current configuration\n\tif !h.deserializedResponses {\n\t\th.writeIncompatibleError(w)\n\t\treturn\n\t}\n\n\t// Check if WebUI is available locally when Gateway.NoFetch is true\n\tif h.noFetch {\n\t\tcidStr := strings.TrimPrefix(WebUIPath, \"/ipfs/\")\n\t\twebUICID, err := cid.Parse(cidStr)\n\t\tif err != nil {\n\t\t\t// This should never happen with hardcoded constant\n\t\t\tlog.Errorf(\"failed to parse WebUI CID: %v\", err)\n\t\t} else {\n\t\t\thas, err := h.node.Blockstore.Has(r.Context(), webUICID)\n\t\t\tif err != nil {\n\t\t\t\tlog.Debugf(\"error checking WebUI availability: %v\", err)\n\t\t\t} else if !has {\n\t\t\t\th.writeNotAvailableError(w)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}\n\n\t// Default behavior: redirect to the WebUI path\n\thttp.Redirect(w, r, WebUIPath, http.StatusFound)\n}\n\nfunc (h *webUIHandler) writeIncompatibleError(w http.ResponseWriter) {\n\tw.Header().Set(\"Content-Type\", \"text/plain; charset=utf-8\")\n\tw.WriteHeader(http.StatusServiceUnavailable)\n\tfmt.Fprintf(w, `IPFS WebUI Incompatible\n\nWebUI is not compatible with Gateway.DeserializedResponses=false.\n\nThe WebUI requires deserializing IPFS responses to render the interface.\nTo use the WebUI, set Gateway.DeserializedResponses=true in your config.\n`)\n}\n\nfunc (h *webUIHandler) writeNotAvailableError(w http.ResponseWriter) {\n\tw.Header().Set(\"Content-Type\", \"text/plain; charset=utf-8\")\n\tw.WriteHeader(http.StatusServiceUnavailable)\n\tfmt.Fprintf(w, `IPFS WebUI Not Available\n\nWebUI at %s is not in your local node due to Gateway.NoFetch=true.\n\nTo use the WebUI, either:\n1. Run: ipfs pin add --progress --name ipfs-webui %s\n2. Download from https://github.com/ipfs/ipfs-webui/releases and import with: ipfs dag import ipfs-webui.car\n`, WebUIPath, WebUIPath)\n}\n" }, { "path": "core/coreiface/block.go", "content": "package iface\n\nimport (\n\t\"context\"\n\t\"io\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\n// BlockStat contains information about a block\ntype BlockStat interface {\n\t// Size is the size of a block\n\tSize() int\n\n\t// Path returns path to the block\n\tPath() path.ImmutablePath\n}\n\n// BlockAPI specifies the interface to the block layer\ntype BlockAPI interface {\n\t// Put imports raw block data, hashing it using specified settings.\n\tPut(context.Context, io.Reader, ...options.BlockPutOption) (BlockStat, error)\n\n\t// Get attempts to resolve the path and return a reader for data in the block\n\tGet(context.Context, path.Path) (io.Reader, error)\n\n\t// Rm removes the block specified by the path from local blockstore.\n\t// By default an error will be returned if the block can't be found locally.\n\t//\n\t// NOTE: If the specified block is pinned it won't be removed and no error\n\t// will be returned\n\tRm(context.Context, path.Path, ...options.BlockRmOption) error\n\n\t// Stat returns information on\n\tStat(context.Context, path.Path) (BlockStat, error)\n}\n" }, { "path": "core/coreiface/coreapi.go", "content": "// Package iface defines IPFS Core API which is a set of interfaces used to\n// interact with IPFS nodes.\npackage iface\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\n\tipld \"github.com/ipfs/go-ipld-format\"\n)\n\n// CoreAPI defines an unified interface to IPFS for Go programs\ntype CoreAPI interface {\n\t// Unixfs returns an implementation of Unixfs API\n\tUnixfs() UnixfsAPI\n\n\t// Block returns an implementation of Block API\n\tBlock() BlockAPI\n\n\t// Dag returns an implementation of Dag API\n\tDag() APIDagService\n\n\t// Name returns an implementation of Name API\n\tName() NameAPI\n\n\t// Key returns an implementation of Key API\n\tKey() KeyAPI\n\n\t// Pin returns an implementation of Pin API\n\tPin() PinAPI\n\n\t// Object returns an implementation of Object API\n\tObject() ObjectAPI\n\n\t// Swarm returns an implementation of Swarm API\n\tSwarm() SwarmAPI\n\n\t// PubSub returns an implementation of PubSub API\n\tPubSub() PubSubAPI\n\n\t// Routing returns an implementation of Routing API\n\tRouting() RoutingAPI\n\n\t// ResolvePath resolves the path using UnixFS resolver, and returns the resolved\n\t// immutable path, and the remainder of the path segments that cannot be resolved\n\t// within UnixFS.\n\tResolvePath(context.Context, path.Path) (path.ImmutablePath, []string, error)\n\n\t// ResolveNode resolves the path (if not resolved already) using Unixfs\n\t// resolver, gets and returns the resolved Node\n\tResolveNode(context.Context, path.Path) (ipld.Node, error)\n\n\t// WithOptions creates new instance of CoreAPI based on this instance with\n\t// a set of options applied\n\tWithOptions(...options.ApiOption) (CoreAPI, error)\n}\n" }, { "path": "core/coreiface/dag.go", "content": "package iface\n\nimport (\n\tipld \"github.com/ipfs/go-ipld-format\"\n)\n\n// APIDagService extends ipld.DAGService\ntype APIDagService interface {\n\tipld.DAGService\n\n\t// Pinning returns special NodeAdder which recursively pins added nodes\n\tPinning() ipld.NodeAdder\n}\n" }, { "path": "core/coreiface/errors.go", "content": "package iface\n\nimport \"errors\"\n\nvar (\n\tErrIsDir = errors.New(\"this dag node is a directory\")\n\tErrNotFile = errors.New(\"this dag node is not a regular file\")\n\tErrOffline = errors.New(\"this action must be run in online mode, try running 'ipfs daemon' first\")\n\tErrNotSupported = errors.New(\"operation not supported\")\n)\n" }, { "path": "core/coreiface/idfmt.go", "content": "package iface\n\nimport (\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n)\n\nfunc FormatKeyID(id peer.ID) string {\n\tif s, err := peer.ToCid(id).StringOfBase(mbase.Base36); err != nil {\n\t\tpanic(err)\n\t} else {\n\t\treturn s\n\t}\n}\n\n// FormatKey formats the given IPNS key in a canonical way.\nfunc FormatKey(key Key) string {\n\treturn FormatKeyID(key.ID())\n}\n" }, { "path": "core/coreiface/key.go", "content": "package iface\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\n// Key specifies the interface to Keys in KeyAPI Keystore\ntype Key interface {\n\t// Key returns key name\n\tName() string\n\n\t// Path returns key path\n\tPath() path.Path\n\n\t// ID returns key PeerID\n\tID() peer.ID\n}\n\n// KeyAPI specifies the interface to Keystore\ntype KeyAPI interface {\n\t// Generate generates new key, stores it in the keystore under the specified\n\t// name and returns a base58 encoded multihash of it's public key\n\tGenerate(ctx context.Context, name string, opts ...options.KeyGenerateOption) (Key, error)\n\n\t// Rename renames oldName key to newName. Returns the key and whether another\n\t// key was overwritten, or an error\n\tRename(ctx context.Context, oldName string, newName string, opts ...options.KeyRenameOption) (Key, bool, error)\n\n\t// List lists keys stored in keystore\n\tList(ctx context.Context) ([]Key, error)\n\n\t// Self returns the 'main' node key\n\tSelf(ctx context.Context) (Key, error)\n\n\t// Remove removes keys from keystore. Returns ipns path of the removed key\n\tRemove(ctx context.Context, name string) (Key, error)\n\n\t// Sign signs the given data with the key named name. Returns the key used\n\t// for signing, the signature, and an error.\n\tSign(ctx context.Context, name string, data []byte) (Key, []byte, error)\n\n\t// Verify verifies if the given data and signatures match. Returns the key used\n\t// for verification, whether signature and data match, and an error.\n\tVerify(ctx context.Context, keyOrName string, signature, data []byte) (Key, bool, error)\n}\n" }, { "path": "core/coreiface/name.go", "content": "package iface\n\nimport (\n\t\"context\"\n\t\"errors\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nvar ErrResolveFailed = errors.New(\"could not resolve name\")\n\ntype IpnsResult struct {\n\tpath.Path\n\tErr error\n}\n\n// NameAPI specifies the interface to IPNS.\n//\n// IPNS is a PKI namespace, where names are the hashes of public keys, and the\n// private key enables publishing new (signed) values. In both publish and\n// resolve, the default name used is the node's own PeerID, which is the hash of\n// its public key.\n//\n// You can use .Key API to list and generate more names and their respective keys.\ntype NameAPI interface {\n\t// Publish announces new IPNS name\n\tPublish(ctx context.Context, path path.Path, opts ...options.NamePublishOption) (ipns.Name, error)\n\n\t// Resolve attempts to resolve the newest version of the specified name\n\tResolve(ctx context.Context, name string, opts ...options.NameResolveOption) (path.Path, error)\n\n\t// Search is a version of Resolve which outputs paths as they are discovered,\n\t// reducing the time to first entry\n\t//\n\t// Note: by default, all paths read from the channel are considered unsafe,\n\t// except the latest (last path in channel read buffer).\n\tSearch(ctx context.Context, name string, opts ...options.NameResolveOption) (<-chan IpnsResult, error)\n}\n" }, { "path": "core/coreiface/object.go", "content": "package iface\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\n// ChangeType denotes type of change in ObjectChange\ntype ChangeType int\n\nconst (\n\t// DiffAdd is set when a link was added to the graph\n\tDiffAdd ChangeType = iota\n\n\t// DiffRemove is set when a link was removed from the graph\n\tDiffRemove\n\n\t// DiffMod is set when a link was changed in the graph\n\tDiffMod\n)\n\n// ObjectChange represents a change ia a graph\ntype ObjectChange struct {\n\t// Type of the change, either:\n\t// * DiffAdd - Added a link\n\t// * DiffRemove - Removed a link\n\t// * DiffMod - Modified a link\n\tType ChangeType\n\n\t// Path to the changed link\n\tPath string\n\n\t// Before holds the link path before the change. Note that when a link is\n\t// added, this will be nil.\n\tBefore path.ImmutablePath\n\n\t// After holds the link path after the change. Note that when a link is\n\t// removed, this will be nil.\n\tAfter path.ImmutablePath\n}\n\n// ObjectAPI specifies the interface to MerkleDAG and contains useful utilities\n// for manipulating MerkleDAG data structures.\ntype ObjectAPI interface {\n\t// AddLink adds a link under the specified path. child path can point to a\n\t// subdirectory within the patent which must be present (can be overridden\n\t// with WithCreate option).\n\tAddLink(ctx context.Context, base path.Path, name string, child path.Path, opts ...options.ObjectAddLinkOption) (path.ImmutablePath, error)\n\n\t// RmLink removes a link from the node\n\tRmLink(ctx context.Context, base path.Path, link string) (path.ImmutablePath, error)\n\n\t// Diff returns a set of changes needed to transform the first object into the\n\t// second.\n\tDiff(context.Context, path.Path, path.Path) ([]ObjectChange, error)\n}\n" }, { "path": "core/coreiface/options/block.go", "content": "package options\n\nimport (\n\t\"fmt\"\n\n\tcid \"github.com/ipfs/go-cid\"\n\tmc \"github.com/multiformats/go-multicodec\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\ntype BlockPutSettings struct {\n\tCidPrefix cid.Prefix\n\tPin bool\n}\n\ntype BlockRmSettings struct {\n\tForce bool\n}\n\ntype (\n\tBlockPutOption func(*BlockPutSettings) error\n\tBlockRmOption func(*BlockRmSettings) error\n)\n\nfunc BlockPutOptions(opts ...BlockPutOption) (*BlockPutSettings, error) {\n\tvar cidPrefix cid.Prefix\n\n\t// Baseline is CIDv1 raw sha2-255-32 (can be tweaked later via opts)\n\tcidPrefix.Version = 1\n\tcidPrefix.Codec = uint64(mc.Raw)\n\tcidPrefix.MhType = mh.SHA2_256\n\tcidPrefix.MhLength = -1 // -1 means len is to be calculated during mh.Sum()\n\n\toptions := &BlockPutSettings{\n\t\tCidPrefix: cidPrefix,\n\t\tPin: false,\n\t}\n\n\t// Apply any overrides\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\nfunc BlockRmOptions(opts ...BlockRmOption) (*BlockRmSettings, error) {\n\toptions := &BlockRmSettings{\n\t\tForce: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\ntype blockOpts struct{}\n\nvar Block blockOpts\n\n// CidCodec is the modern option for Block.Put which specifies the multicodec to use\n// in the CID returned by the Block.Put operation.\n// It uses correct codes from go-multicodec and replaces the old Format now with CIDv1 as the default.\nfunc (blockOpts) CidCodec(codecName string) BlockPutOption {\n\treturn func(settings *BlockPutSettings) error {\n\t\tif codecName == \"\" {\n\t\t\treturn nil\n\t\t}\n\t\tcode, err := codeFromName(codecName)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tsettings.CidPrefix.Codec = uint64(code)\n\t\treturn nil\n\t}\n}\n\n// Map string to code from go-multicodec\nfunc codeFromName(codecName string) (mc.Code, error) {\n\tvar cidCodec mc.Code\n\terr := cidCodec.Set(codecName)\n\treturn cidCodec, err\n}\n\n// Format is a legacy option for Block.Put which specifies the multicodec to\n// use to serialize the object.\n// Provided for backward-compatibility only. Use CidCodec instead.\nfunc (blockOpts) Format(format string) BlockPutOption {\n\treturn func(settings *BlockPutSettings) error {\n\t\tif format == \"\" {\n\t\t\treturn nil\n\t\t}\n\t\t// Opt-in CIDv0 support for backward-compatibility\n\t\tif format == \"v0\" {\n\t\t\tsettings.CidPrefix.Version = 0\n\t\t}\n\n\t\t// Fixup a legacy (invalid) names for dag-pb (0x70)\n\t\tif format == \"v0\" || format == \"protobuf\" {\n\t\t\tformat = \"dag-pb\"\n\t\t}\n\n\t\t// Fixup invalid name for dag-cbor (0x71)\n\t\tif format == \"cbor\" {\n\t\t\tformat = \"dag-cbor\"\n\t\t}\n\n\t\t// Set code based on name passed as \"format\"\n\t\tcode, err := codeFromName(format)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tsettings.CidPrefix.Codec = uint64(code)\n\n\t\t// If CIDv0, ensure all parameters are compatible\n\t\t// (in theory go-cid would validate this anyway, but we want to provide better errors)\n\t\tpref := settings.CidPrefix\n\t\tif pref.Version == 0 {\n\t\t\tif pref.Codec != uint64(mc.DagPb) {\n\t\t\t\treturn fmt.Errorf(\"only dag-pb is allowed with CIDv0\")\n\t\t\t}\n\t\t\tif pref.MhType != mh.SHA2_256 || (pref.MhLength != -1 && pref.MhLength != 32) {\n\t\t\t\treturn fmt.Errorf(\"only sha2-255-32 is allowed with CIDv0\")\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t}\n}\n\n// Hash is an option for Block.Put which specifies the multihash settings to use\n// when hashing the object. Default is mh.SHA2_256 (0x12).\n// If mhLen is set to -1, default length for the hash will be used\nfunc (blockOpts) Hash(mhType uint64, mhLen int) BlockPutOption {\n\treturn func(settings *BlockPutSettings) error {\n\t\tsettings.CidPrefix.MhType = mhType\n\t\tsettings.CidPrefix.MhLength = mhLen\n\t\treturn nil\n\t}\n}\n\n// Pin is an option for Block.Put which specifies whether to (recursively) pin\n// added blocks\nfunc (blockOpts) Pin(pin bool) BlockPutOption {\n\treturn func(settings *BlockPutSettings) error {\n\t\tsettings.Pin = pin\n\t\treturn nil\n\t}\n}\n\n// Force is an option for Block.Rm which, when set to true, will ignore\n// non-existing blocks\nfunc (blockOpts) Force(force bool) BlockRmOption {\n\treturn func(settings *BlockRmSettings) error {\n\t\tsettings.Force = force\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/dht.go", "content": "package options\n\n// nolint deprecated\n// Deprecated: use [RoutingProvideSettings] instead.\ntype DhtProvideSettings = RoutingProvideSettings\n\n// nolint deprecated\n// Deprecated: use [RoutingFindProvidersSettings] instead.\ntype DhtFindProvidersSettings = RoutingFindProvidersSettings\n\n// nolint deprecated\n// Deprecated: use [RoutingProvideOption] instead.\ntype DhtProvideOption = RoutingProvideOption\n\n// nolint deprecated\n// Deprecated: use [RoutingFindProvidersOption] instead.\ntype DhtFindProvidersOption = RoutingFindProvidersOption\n\n// nolint deprecated\n// Deprecated: use [RoutingProvideOptions] instead.\nvar DhtProvideOptions = RoutingProvideOptions\n\n// nolint deprecated\n// Deprecated: use [RoutingFindProvidersOptions] instead.\nvar DhtFindProvidersOptions = RoutingFindProvidersOptions\n\n// nolint deprecated\n// Deprecated: use [Routing] instead.\nvar Dht = Routing\n" }, { "path": "core/coreiface/options/global.go", "content": "package options\n\ntype ApiSettings struct {\n\tOffline bool\n\tFetchBlocks bool\n}\n\ntype ApiOption func(*ApiSettings) error\n\nfunc ApiOptions(opts ...ApiOption) (*ApiSettings, error) {\n\toptions := &ApiSettings{\n\t\tOffline: false,\n\t\tFetchBlocks: true,\n\t}\n\n\treturn ApiOptionsTo(options, opts...)\n}\n\nfunc ApiOptionsTo(options *ApiSettings, opts ...ApiOption) (*ApiSettings, error) {\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\ntype apiOpts struct{}\n\nvar Api apiOpts\n\nfunc (apiOpts) Offline(offline bool) ApiOption {\n\treturn func(settings *ApiSettings) error {\n\t\tsettings.Offline = offline\n\t\treturn nil\n\t}\n}\n\n// FetchBlocks when set to false prevents api from fetching blocks from the\n// network while allowing other services such as IPNS to still be online\nfunc (apiOpts) FetchBlocks(fetch bool) ApiOption {\n\treturn func(settings *ApiSettings) error {\n\t\tsettings.FetchBlocks = fetch\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/key.go", "content": "package options\n\nconst (\n\tRSAKey = \"rsa\"\n\tEd25519Key = \"ed25519\"\n\n\tDefaultRSALen = 2048\n)\n\ntype KeyGenerateSettings struct {\n\tAlgorithm string\n\tSize int\n}\n\ntype KeyRenameSettings struct {\n\tForce bool\n}\n\ntype (\n\tKeyGenerateOption func(*KeyGenerateSettings) error\n\tKeyRenameOption func(*KeyRenameSettings) error\n)\n\nfunc KeyGenerateOptions(opts ...KeyGenerateOption) (*KeyGenerateSettings, error) {\n\toptions := &KeyGenerateSettings{\n\t\tAlgorithm: RSAKey,\n\t\tSize: -1,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\nfunc KeyRenameOptions(opts ...KeyRenameOption) (*KeyRenameSettings, error) {\n\toptions := &KeyRenameSettings{\n\t\tForce: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\ntype keyOpts struct{}\n\nvar Key keyOpts\n\n// Type is an option for Key.Generate which specifies which algorithm\n// should be used for the key. Default is options.RSAKey\n//\n// Supported key types:\n// * options.RSAKey\n// * options.Ed25519Key\nfunc (keyOpts) Type(algorithm string) KeyGenerateOption {\n\treturn func(settings *KeyGenerateSettings) error {\n\t\tsettings.Algorithm = algorithm\n\t\treturn nil\n\t}\n}\n\n// Size is an option for Key.Generate which specifies the size of the key to\n// generated. Default is -1\n//\n// value of -1 means 'use default size for key type':\n// - 2048 for RSA\nfunc (keyOpts) Size(size int) KeyGenerateOption {\n\treturn func(settings *KeyGenerateSettings) error {\n\t\tsettings.Size = size\n\t\treturn nil\n\t}\n}\n\n// Force is an option for Key.Rename which specifies whether to allow to\n// replace existing keys.\nfunc (keyOpts) Force(force bool) KeyRenameOption {\n\treturn func(settings *KeyRenameSettings) error {\n\t\tsettings.Force = force\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/name.go", "content": "package options\n\nimport (\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n)\n\nconst (\n\tDefaultNameValidTime = 24 * time.Hour\n)\n\ntype NamePublishSettings struct {\n\tValidTime time.Duration\n\tKey string\n\tTTL *time.Duration\n\tCompatibleWithV1 bool\n\tAllowOffline bool\n\tAllowDelegated bool\n\tSequence *uint64\n}\n\ntype NameResolveSettings struct {\n\tCache bool\n\n\tResolveOpts []namesys.ResolveOption\n}\n\ntype (\n\tNamePublishOption func(*NamePublishSettings) error\n\tNameResolveOption func(*NameResolveSettings) error\n)\n\nfunc NamePublishOptions(opts ...NamePublishOption) (*NamePublishSettings, error) {\n\toptions := &NamePublishSettings{\n\t\tValidTime: DefaultNameValidTime,\n\t\tKey: \"self\",\n\n\t\tAllowOffline: false,\n\t\tAllowDelegated: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\nfunc NameResolveOptions(opts ...NameResolveOption) (*NameResolveSettings, error) {\n\toptions := &NameResolveSettings{\n\t\tCache: true,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\ntype nameOpts struct{}\n\nvar Name nameOpts\n\n// ValidTime is an option for Name.Publish which specifies for how long the\n// entry will remain valid. Default value is 24h\nfunc (nameOpts) ValidTime(validTime time.Duration) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.ValidTime = validTime\n\t\treturn nil\n\t}\n}\n\n// Key is an option for Name.Publish which specifies the key to use for\n// publishing. Default value is \"self\" which is the node's own PeerID.\n// The key parameter must be either PeerID or keystore key alias.\n//\n// You can use KeyAPI to list and generate more names and their respective keys.\nfunc (nameOpts) Key(key string) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.Key = key\n\t\treturn nil\n\t}\n}\n\n// AllowOffline is an option for Name.Publish which specifies whether to allow\n// publishing when the node is offline. Default value is false\nfunc (nameOpts) AllowOffline(allow bool) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.AllowOffline = allow\n\t\treturn nil\n\t}\n}\n\n// AllowDelegated is an option for Name.Publish which allows publishing without\n// DHT connectivity, using local datastore and HTTP delegated publishers only.\n// Default value is false\nfunc (nameOpts) AllowDelegated(allowDelegated bool) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.AllowDelegated = allowDelegated\n\t\treturn nil\n\t}\n}\n\n// TTL is an option for Name.Publish which specifies the time duration the\n// published record should be cached for (caution: experimental).\nfunc (nameOpts) TTL(ttl time.Duration) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.TTL = &ttl\n\t\treturn nil\n\t}\n}\n\n// Sequence is an option for Name.Publish which specifies the sequence number of\n// a namesys record.\nfunc (nameOpts) Sequence(seq uint64) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.Sequence = &seq\n\t\treturn nil\n\t}\n}\n\n// CompatibleWithV1 is an option for [Name.Publish] which specifies if the\n// created record should be backwards compatible with V1 IPNS Records.\nfunc (nameOpts) CompatibleWithV1(compatible bool) NamePublishOption {\n\treturn func(settings *NamePublishSettings) error {\n\t\tsettings.CompatibleWithV1 = compatible\n\t\treturn nil\n\t}\n}\n\n// Cache is an option for Name.Resolve which specifies if cache should be used.\n// Default value is true\nfunc (nameOpts) Cache(cache bool) NameResolveOption {\n\treturn func(settings *NameResolveSettings) error {\n\t\tsettings.Cache = cache\n\t\treturn nil\n\t}\n}\n\nfunc (nameOpts) ResolveOption(opt namesys.ResolveOption) NameResolveOption {\n\treturn func(settings *NameResolveSettings) error {\n\t\tsettings.ResolveOpts = append(settings.ResolveOpts, opt)\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/object.go", "content": "package options\n\ntype ObjectAddLinkSettings struct {\n\tCreate bool\n}\n\ntype (\n\tObjectAddLinkOption func(*ObjectAddLinkSettings) error\n)\n\nfunc ObjectAddLinkOptions(opts ...ObjectAddLinkOption) (*ObjectAddLinkSettings, error) {\n\toptions := &ObjectAddLinkSettings{\n\t\tCreate: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\ntype objectOpts struct{}\n\nvar Object objectOpts\n\n// Create is an option for Object.AddLink which specifies whether create required\n// directories for the child\nfunc (objectOpts) Create(create bool) ObjectAddLinkOption {\n\treturn func(settings *ObjectAddLinkSettings) error {\n\t\tsettings.Create = create\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/pin.go", "content": "package options\n\nimport \"fmt\"\n\n// PinAddSettings represent the settings for PinAPI.Add\ntype PinAddSettings struct {\n\tRecursive bool\n\tName string\n}\n\n// PinLsSettings represent the settings for PinAPI.Ls\ntype PinLsSettings struct {\n\tType string\n\tDetailed bool\n\tName string\n}\n\n// PinIsPinnedSettings represent the settings for PinAPI.IsPinned\ntype PinIsPinnedSettings struct {\n\tWithType string\n}\n\n// PinRmSettings represents the settings for PinAPI.Rm\ntype PinRmSettings struct {\n\tRecursive bool\n}\n\n// PinUpdateSettings represent the settings for PinAPI.Update\ntype PinUpdateSettings struct {\n\tUnpin bool\n}\n\n// PinAddOption is the signature of an option for PinAPI.Add\ntype PinAddOption func(*PinAddSettings) error\n\n// PinLsOption is the signature of an option for PinAPI.Ls\ntype PinLsOption func(*PinLsSettings) error\n\n// PinIsPinnedOption is the signature of an option for PinAPI.IsPinned\ntype PinIsPinnedOption func(*PinIsPinnedSettings) error\n\n// PinRmOption is the signature of an option for PinAPI.Rm\ntype PinRmOption func(*PinRmSettings) error\n\n// PinUpdateOption is the signature of an option for PinAPI.Update\ntype PinUpdateOption func(*PinUpdateSettings) error\n\n// PinAddOptions compile a series of PinAddOption into a ready to use\n// PinAddSettings and set the default values.\nfunc PinAddOptions(opts ...PinAddOption) (*PinAddSettings, error) {\n\toptions := &PinAddSettings{\n\t\tRecursive: true,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// PinLsOptions compile a series of PinLsOption into a ready to use\n// PinLsSettings and set the default values.\nfunc PinLsOptions(opts ...PinLsOption) (*PinLsSettings, error) {\n\toptions := &PinLsSettings{\n\t\tType: \"all\",\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// PinIsPinnedOptions compile a series of PinIsPinnedOption into a ready to use\n// PinIsPinnedSettings and set the default values.\nfunc PinIsPinnedOptions(opts ...PinIsPinnedOption) (*PinIsPinnedSettings, error) {\n\toptions := &PinIsPinnedSettings{\n\t\tWithType: \"all\",\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// PinRmOptions compile a series of PinRmOption into a ready to use\n// PinRmSettings and set the default values.\nfunc PinRmOptions(opts ...PinRmOption) (*PinRmSettings, error) {\n\toptions := &PinRmSettings{\n\t\tRecursive: true,\n\t}\n\n\tfor _, opt := range opts {\n\t\tif err := opt(options); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// PinUpdateOptions compile a series of PinUpdateOption into a ready to use\n// PinUpdateSettings and set the default values.\nfunc PinUpdateOptions(opts ...PinUpdateOption) (*PinUpdateSettings, error) {\n\toptions := &PinUpdateSettings{\n\t\tUnpin: true,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\ntype pinOpts struct {\n\tLs pinLsOpts\n\tIsPinned pinIsPinnedOpts\n}\n\n// Pin provide an access to all the options for the Pin API.\nvar Pin pinOpts\n\ntype pinLsOpts struct{}\n\n// All is an option for Pin.Ls which will make it return all pins. It is\n// the default\nfunc (pinLsOpts) All() PinLsOption {\n\treturn Pin.Ls.pinType(\"all\")\n}\n\n// Recursive is an option for Pin.Ls which will make it only return recursive\n// pins\nfunc (pinLsOpts) Recursive() PinLsOption {\n\treturn Pin.Ls.pinType(\"recursive\")\n}\n\n// Direct is an option for Pin.Ls which will make it only return direct (non\n// recursive) pins\nfunc (pinLsOpts) Direct() PinLsOption {\n\treturn Pin.Ls.pinType(\"direct\")\n}\n\n// Indirect is an option for Pin.Ls which will make it only return indirect pins\n// (objects referenced by other recursively pinned objects)\nfunc (pinLsOpts) Indirect() PinLsOption {\n\treturn Pin.Ls.pinType(\"indirect\")\n}\n\n// Type is an option for Pin.Ls which will make it only return pins of the given\n// type.\n//\n// Supported values:\n// - \"direct\" - directly pinned objects\n// - \"recursive\" - roots of recursive pins\n// - \"indirect\" - indirectly pinned objects (referenced by recursively pinned\n// objects)\n// - \"all\" - all pinned objects (default)\nfunc (pinLsOpts) Type(typeStr string) (PinLsOption, error) {\n\tswitch typeStr {\n\tcase \"all\", \"direct\", \"indirect\", \"recursive\":\n\t\treturn Pin.Ls.pinType(typeStr), nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"invalid type '%s', must be one of {direct, indirect, recursive, all}\", typeStr)\n\t}\n}\n\n// pinType is an option for Pin.Ls which allows to specify which pin types should\n// be returned\n//\n// Supported values:\n// - \"direct\" - directly pinned objects\n// - \"recursive\" - roots of recursive pins\n// - \"indirect\" - indirectly pinned objects (referenced by recursively pinned\n// objects)\n// - \"all\" - all pinned objects (default)\nfunc (pinLsOpts) pinType(t string) PinLsOption {\n\treturn func(settings *PinLsSettings) error {\n\t\tsettings.Type = t\n\t\treturn nil\n\t}\n}\n\n// Detailed is an option for [Pin.Ls] which sets whether or not to return\n// detailed information, such as pin names and modes.\nfunc (pinLsOpts) Detailed(detailed bool) PinLsOption {\n\treturn func(settings *PinLsSettings) error {\n\t\tsettings.Detailed = detailed\n\t\treturn nil\n\t}\n}\n\nfunc (pinLsOpts) Name(name string) PinLsOption {\n\treturn func(settings *PinLsSettings) error {\n\t\tsettings.Name = name\n\t\treturn nil\n\t}\n}\n\ntype pinIsPinnedOpts struct{}\n\n// All is an option for Pin.IsPinned which will make it search in all type of pins.\n// It is the default\nfunc (pinIsPinnedOpts) All() PinIsPinnedOption {\n\treturn Pin.IsPinned.pinType(\"all\")\n}\n\n// Recursive is an option for Pin.IsPinned which will make it only search in\n// recursive pins\nfunc (pinIsPinnedOpts) Recursive() PinIsPinnedOption {\n\treturn Pin.IsPinned.pinType(\"recursive\")\n}\n\n// Direct is an option for Pin.IsPinned which will make it only search in direct\n// (non recursive) pins\nfunc (pinIsPinnedOpts) Direct() PinIsPinnedOption {\n\treturn Pin.IsPinned.pinType(\"direct\")\n}\n\n// Indirect is an option for Pin.IsPinned which will make it only search indirect\n// pins (objects referenced by other recursively pinned objects)\nfunc (pinIsPinnedOpts) Indirect() PinIsPinnedOption {\n\treturn Pin.IsPinned.pinType(\"indirect\")\n}\n\n// Type is an option for Pin.IsPinned which will make it only search pins of the given\n// type.\n//\n// Supported values:\n// - \"direct\" - directly pinned objects\n// - \"recursive\" - roots of recursive pins\n// - \"indirect\" - indirectly pinned objects (referenced by recursively pinned\n// objects)\n// - \"all\" - all pinned objects (default)\nfunc (pinIsPinnedOpts) Type(typeStr string) (PinIsPinnedOption, error) {\n\tswitch typeStr {\n\tcase \"all\", \"direct\", \"indirect\", \"recursive\":\n\t\treturn Pin.IsPinned.pinType(typeStr), nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"invalid type '%s', must be one of {direct, indirect, recursive, all}\", typeStr)\n\t}\n}\n\n// pinType is an option for Pin.IsPinned which allows to specify which pin type the given\n// pin is expected to be, speeding up the research.\n//\n// Supported values:\n// - \"direct\" - directly pinned objects\n// - \"recursive\" - roots of recursive pins\n// - \"indirect\" - indirectly pinned objects (referenced by recursively pinned\n// objects)\n// - \"all\" - all pinned objects (default)\nfunc (pinIsPinnedOpts) pinType(t string) PinIsPinnedOption {\n\treturn func(settings *PinIsPinnedSettings) error {\n\t\tsettings.WithType = t\n\t\treturn nil\n\t}\n}\n\n// Recursive is an option for Pin.Add which specifies whether to pin an entire\n// object tree or just one object. Default: true\nfunc (pinOpts) Recursive(recursive bool) PinAddOption {\n\treturn func(settings *PinAddSettings) error {\n\t\tsettings.Recursive = recursive\n\t\treturn nil\n\t}\n}\n\n// Name is an option for Pin.Add which specifies an optional name to add to the pin.\nfunc (pinOpts) Name(name string) PinAddOption {\n\treturn func(settings *PinAddSettings) error {\n\t\tsettings.Name = name\n\t\treturn nil\n\t}\n}\n\n// RmRecursive is an option for Pin.Rm which specifies whether to recursively\n// unpin the object linked to by the specified object(s). This does not remove\n// indirect pins referenced by other recursive pins.\nfunc (pinOpts) RmRecursive(recursive bool) PinRmOption {\n\treturn func(settings *PinRmSettings) error {\n\t\tsettings.Recursive = recursive\n\t\treturn nil\n\t}\n}\n\n// Unpin is an option for Pin.Update which specifies whether to remove the old pin.\n// Default is true.\nfunc (pinOpts) Unpin(unpin bool) PinUpdateOption {\n\treturn func(settings *PinUpdateSettings) error {\n\t\tsettings.Unpin = unpin\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/pubsub.go", "content": "package options\n\ntype PubSubPeersSettings struct {\n\tTopic string\n}\n\ntype PubSubSubscribeSettings struct {\n\tDiscover bool\n}\n\ntype (\n\tPubSubPeersOption func(*PubSubPeersSettings) error\n\tPubSubSubscribeOption func(*PubSubSubscribeSettings) error\n)\n\nfunc PubSubPeersOptions(opts ...PubSubPeersOption) (*PubSubPeersSettings, error) {\n\toptions := &PubSubPeersSettings{\n\t\tTopic: \"\",\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\nfunc PubSubSubscribeOptions(opts ...PubSubSubscribeOption) (*PubSubSubscribeSettings, error) {\n\toptions := &PubSubSubscribeSettings{\n\t\tDiscover: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\ntype pubsubOpts struct{}\n\nvar PubSub pubsubOpts\n\nfunc (pubsubOpts) Topic(topic string) PubSubPeersOption {\n\treturn func(settings *PubSubPeersSettings) error {\n\t\tsettings.Topic = topic\n\t\treturn nil\n\t}\n}\n\nfunc (pubsubOpts) Discover(discover bool) PubSubSubscribeOption {\n\treturn func(settings *PubSubSubscribeSettings) error {\n\t\tsettings.Discover = discover\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/routing.go", "content": "package options\n\ntype RoutingPutSettings struct {\n\tAllowOffline bool\n}\n\ntype RoutingPutOption func(*RoutingPutSettings) error\n\nfunc RoutingPutOptions(opts ...RoutingPutOption) (*RoutingPutSettings, error) {\n\toptions := &RoutingPutSettings{\n\t\tAllowOffline: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// nolint deprecated\n// Deprecated: use [Routing] instead.\nvar Put = Routing\n\ntype RoutingProvideSettings struct {\n\tRecursive bool\n}\n\ntype RoutingFindProvidersSettings struct {\n\tNumProviders int\n}\n\ntype (\n\tRoutingProvideOption func(*DhtProvideSettings) error\n\tRoutingFindProvidersOption func(*DhtFindProvidersSettings) error\n)\n\nfunc RoutingProvideOptions(opts ...RoutingProvideOption) (*RoutingProvideSettings, error) {\n\toptions := &RoutingProvideSettings{\n\t\tRecursive: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\nfunc RoutingFindProvidersOptions(opts ...RoutingFindProvidersOption) (*RoutingFindProvidersSettings, error) {\n\toptions := &RoutingFindProvidersSettings{\n\t\tNumProviders: 20,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn options, nil\n}\n\ntype routingOpts struct{}\n\nvar Routing routingOpts\n\n// Recursive is an option for [Routing.Provide] which specifies whether to provide\n// the given path recursively.\nfunc (routingOpts) Recursive(recursive bool) RoutingProvideOption {\n\treturn func(settings *DhtProvideSettings) error {\n\t\tsettings.Recursive = recursive\n\t\treturn nil\n\t}\n}\n\n// NumProviders is an option for [Routing.FindProviders] which specifies the\n// number of peers to look for. Default is 20.\nfunc (routingOpts) NumProviders(numProviders int) RoutingFindProvidersOption {\n\treturn func(settings *DhtFindProvidersSettings) error {\n\t\tsettings.NumProviders = numProviders\n\t\treturn nil\n\t}\n}\n\n// AllowOffline is an option for [Routing.Put] which specifies whether to allow\n// publishing when the node is offline. Default value is false\nfunc (routingOpts) AllowOffline(allow bool) RoutingPutOption {\n\treturn func(settings *RoutingPutSettings) error {\n\t\tsettings.AllowOffline = allow\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/unixfs.go", "content": "package options\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipld/unixfs/importer/helpers\"\n\t\"github.com/ipfs/boxo/ipld/unixfs/io\"\n\tcid \"github.com/ipfs/go-cid\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\ntype Layout int\n\nconst (\n\tBalancedLayout Layout = iota\n\tTrickleLayout\n)\n\ntype UnixfsAddSettings struct {\n\tCidVersion int\n\tMhType uint64\n\n\tInline bool\n\tInlineLimit int\n\tRawLeaves bool\n\tRawLeavesSet bool\n\tMaxFileLinks int\n\tMaxFileLinksSet bool\n\tMaxDirectoryLinks int\n\tMaxDirectoryLinksSet bool\n\tMaxHAMTFanout int\n\tMaxHAMTFanoutSet bool\n\tSizeEstimationMode *io.SizeEstimationMode\n\tSizeEstimationModeSet bool\n\n\tChunker string\n\tLayout Layout\n\n\tPin bool\n\tPinName string\n\tOnlyHash bool\n\tFsCache bool\n\tNoCopy bool\n\n\tEvents chan<- any\n\tSilent bool\n\tProgress bool\n\n\tPreserveMode bool\n\tPreserveMtime bool\n\tMode os.FileMode\n\tMtime time.Time\n\tIncludeEmptyDirs bool\n\tIncludeEmptyDirsSet bool\n}\n\ntype UnixfsLsSettings struct {\n\tResolveChildren bool\n\tUseCumulativeSize bool\n}\n\ntype (\n\tUnixfsAddOption func(*UnixfsAddSettings) error\n\tUnixfsLsOption func(*UnixfsLsSettings) error\n)\n\nfunc UnixfsAddOptions(opts ...UnixfsAddOption) (*UnixfsAddSettings, cid.Prefix, error) {\n\toptions := &UnixfsAddSettings{\n\t\tCidVersion: -1,\n\t\tMhType: mh.SHA2_256,\n\n\t\tInline: false,\n\t\tInlineLimit: 32,\n\t\tRawLeaves: false,\n\t\tRawLeavesSet: false,\n\t\tMaxFileLinks: helpers.DefaultLinksPerBlock,\n\t\tMaxFileLinksSet: false,\n\t\tMaxDirectoryLinks: 0,\n\t\tMaxDirectoryLinksSet: false,\n\t\tMaxHAMTFanout: io.DefaultShardWidth,\n\t\tMaxHAMTFanoutSet: false,\n\n\t\tChunker: \"size-262144\",\n\t\tLayout: BalancedLayout,\n\n\t\tPin: false,\n\t\tPinName: \"\",\n\t\tOnlyHash: false,\n\t\tFsCache: false,\n\t\tNoCopy: false,\n\n\t\tEvents: nil,\n\t\tSilent: false,\n\t\tProgress: false,\n\n\t\tPreserveMode: false,\n\t\tPreserveMtime: false,\n\t\tMode: 0,\n\t\tMtime: time.Time{},\n\t\tIncludeEmptyDirs: true, // default: include empty directories\n\t\tIncludeEmptyDirsSet: false,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, cid.Prefix{}, err\n\t\t}\n\t}\n\n\t// nocopy -> rawblocks\n\tif options.NoCopy && !options.RawLeaves {\n\t\t// fixed?\n\t\tif options.RawLeavesSet {\n\t\t\treturn nil, cid.Prefix{}, fmt.Errorf(\"nocopy option requires '--raw-leaves' to be enabled as well\")\n\t\t}\n\n\t\t// No, satisfy mandatory constraint.\n\t\toptions.RawLeaves = true\n\t}\n\n\t// (hash != \"sha2-256\") -> CIDv1\n\tif options.MhType != mh.SHA2_256 {\n\t\tswitch options.CidVersion {\n\t\tcase 0:\n\t\t\treturn nil, cid.Prefix{}, errors.New(\"CIDv0 only supports sha2-256\")\n\t\tcase 1, -1:\n\t\t\toptions.CidVersion = 1\n\t\tdefault:\n\t\t\treturn nil, cid.Prefix{}, fmt.Errorf(\"unknown CID version: %d\", options.CidVersion)\n\t\t}\n\t} else {\n\t\tif options.CidVersion < 0 {\n\t\t\t// Default to CIDv0\n\t\t\toptions.CidVersion = 0\n\t\t}\n\t}\n\n\tif !options.Mtime.IsZero() && options.PreserveMtime {\n\t\toptions.PreserveMtime = false\n\t}\n\n\tif options.Mode != 0 && options.PreserveMode {\n\t\toptions.PreserveMode = false\n\t}\n\n\t// cidV1 -> raw blocks (by default)\n\tif options.CidVersion > 0 && !options.RawLeavesSet {\n\t\toptions.RawLeaves = true\n\t}\n\n\tprefix, err := dag.PrefixForCidVersion(options.CidVersion)\n\tif err != nil {\n\t\treturn nil, cid.Prefix{}, err\n\t}\n\n\tprefix.MhType = options.MhType\n\tprefix.MhLength = -1\n\n\treturn options, prefix, nil\n}\n\nfunc UnixfsLsOptions(opts ...UnixfsLsOption) (*UnixfsLsSettings, error) {\n\toptions := &UnixfsLsSettings{\n\t\tResolveChildren: true,\n\t}\n\n\tfor _, opt := range opts {\n\t\terr := opt(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\ntype unixfsOpts struct{}\n\nvar Unixfs unixfsOpts\n\n// CidVersion specifies which CID version to use. Defaults to 0 unless an option\n// that depends on CIDv1 is passed.\nfunc (unixfsOpts) CidVersion(version int) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.CidVersion = version\n\t\treturn nil\n\t}\n}\n\n// Hash function to use. Implies CIDv1 if not set to sha2-256 (default).\n//\n// Table of functions is declared in https://github.com/multiformats/go-multihash/blob/master/multihash.go\nfunc (unixfsOpts) Hash(mhtype uint64) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.MhType = mhtype\n\t\treturn nil\n\t}\n}\n\n// RawLeaves specifies whether to use raw blocks for leaves (data nodes with no\n// links) instead of wrapping them with unixfs structures.\nfunc (unixfsOpts) RawLeaves(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.RawLeaves = enable\n\t\tsettings.RawLeavesSet = true\n\t\treturn nil\n\t}\n}\n\n// MaxFileLinks specifies the maximum number of children for UnixFS file\n// nodes.\nfunc (unixfsOpts) MaxFileLinks(n int) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.MaxFileLinks = n\n\t\tsettings.MaxFileLinksSet = true\n\t\treturn nil\n\t}\n}\n\n// MaxDirectoryLinks specifies the maximum number of children for UnixFS basic\n// directory nodes.\nfunc (unixfsOpts) MaxDirectoryLinks(n int) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.MaxDirectoryLinks = n\n\t\tsettings.MaxDirectoryLinksSet = true\n\t\treturn nil\n\t}\n}\n\n// MaxHAMTFanout specifies the maximum width of the HAMT directory shards.\n// Per the UnixFS spec, the value must be a power of 2, minimum 8\n// (for byte-aligned bitfields), and maximum 1024.\nfunc (unixfsOpts) MaxHAMTFanout(n int) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tif n < 8 || n&(n-1) != 0 || n > 1024 {\n\t\t\treturn fmt.Errorf(\"HAMT fanout must be a power of 2, between 8 and 1024 (got %d)\", n)\n\t\t}\n\t\tsettings.MaxHAMTFanout = n\n\t\tsettings.MaxHAMTFanoutSet = true\n\t\treturn nil\n\t}\n}\n\n// SizeEstimationMode specifies how directory size is estimated for HAMT sharding decisions.\nfunc (unixfsOpts) SizeEstimationMode(mode io.SizeEstimationMode) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.SizeEstimationMode = &mode\n\t\tsettings.SizeEstimationModeSet = true\n\t\treturn nil\n\t}\n}\n\n// Inline tells the adder to inline small blocks into CIDs\nfunc (unixfsOpts) Inline(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Inline = enable\n\t\treturn nil\n\t}\n}\n\n// InlineLimit sets the amount of bytes below which blocks will be encoded\n// directly into CID instead of being stored and addressed by it's hash.\n// Specifying this option won't enable block inlining. For that use `Inline`\n// option. Default: 32 bytes\n//\n// Note that while there is no hard limit on the number of bytes, it should be\n// kept at a reasonably low value, such as 64; implementations may choose to\n// reject anything larger.\nfunc (unixfsOpts) InlineLimit(limit int) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.InlineLimit = limit\n\t\treturn nil\n\t}\n}\n\n// Chunker specifies settings for the chunking algorithm to use.\n//\n// Default: size-262144, formats:\n// size-[bytes] - Simple chunker splitting data into blocks of n bytes\n// rabin-[min]-[avg]-[max] - Rabin chunker\nfunc (unixfsOpts) Chunker(chunker string) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Chunker = chunker\n\t\treturn nil\n\t}\n}\n\n// Layout tells the adder how to balance data between leaves.\n// options.BalancedLayout is the default, it's optimized for static seekable\n// files.\n// options.TrickleLayout is optimized for streaming data,\nfunc (unixfsOpts) Layout(layout Layout) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Layout = layout\n\t\treturn nil\n\t}\n}\n\n// Pin tells the adder to pin the file root recursively after adding\nfunc (unixfsOpts) Pin(pin bool, pinName string) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Pin = pin\n\t\tif pin {\n\t\t\tsettings.PinName = pinName\n\t\t}\n\t\treturn nil\n\t}\n}\n\n// HashOnly will make the adder calculate data hash without storing it in the\n// blockstore or announcing it to the network\nfunc (unixfsOpts) HashOnly(hashOnly bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.OnlyHash = hashOnly\n\t\treturn nil\n\t}\n}\n\n// Events specifies channel which will be used to report events about ongoing\n// Add operation.\n//\n// Note that if this channel blocks it may slowdown the adder\nfunc (unixfsOpts) Events(sink chan<- any) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Events = sink\n\t\treturn nil\n\t}\n}\n\n// Silent reduces event output\nfunc (unixfsOpts) Silent(silent bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Silent = silent\n\t\treturn nil\n\t}\n}\n\n// Progress tells the adder whether to enable progress events\nfunc (unixfsOpts) Progress(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Progress = enable\n\t\treturn nil\n\t}\n}\n\n// FsCache tells the adder to check the filestore for pre-existing blocks\n//\n// Experimental\nfunc (unixfsOpts) FsCache(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.FsCache = enable\n\t\treturn nil\n\t}\n}\n\n// NoCopy tells the adder to add the files using filestore. Implies RawLeaves.\n//\n// Experimental\nfunc (unixfsOpts) Nocopy(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.NoCopy = enable\n\t\treturn nil\n\t}\n}\n\nfunc (unixfsOpts) ResolveChildren(resolve bool) UnixfsLsOption {\n\treturn func(settings *UnixfsLsSettings) error {\n\t\tsettings.ResolveChildren = resolve\n\t\treturn nil\n\t}\n}\n\nfunc (unixfsOpts) UseCumulativeSize(use bool) UnixfsLsOption {\n\treturn func(settings *UnixfsLsSettings) error {\n\t\tsettings.UseCumulativeSize = use\n\t\treturn nil\n\t}\n}\n\n// PreserveMode tells the adder to store the file permissions\nfunc (unixfsOpts) PreserveMode(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.PreserveMode = enable\n\t\treturn nil\n\t}\n}\n\n// PreserveMtime tells the adder to store the file modification time\nfunc (unixfsOpts) PreserveMtime(enable bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.PreserveMtime = enable\n\t\treturn nil\n\t}\n}\n\n// Mode represents a unix file mode\nfunc (unixfsOpts) Mode(mode os.FileMode) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.Mode = mode\n\t\treturn nil\n\t}\n}\n\n// Mtime represents a unix file mtime\nfunc (unixfsOpts) Mtime(seconds int64, nsecs uint32) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tif nsecs > 999999999 {\n\t\t\treturn errors.New(\"mtime nanoseconds must be in range [1, 999999999]\")\n\t\t}\n\t\tsettings.Mtime = time.Unix(seconds, int64(nsecs))\n\t\treturn nil\n\t}\n}\n\n// IncludeEmptyDirs tells the adder to include empty directories in the DAG\nfunc (unixfsOpts) IncludeEmptyDirs(include bool) UnixfsAddOption {\n\treturn func(settings *UnixfsAddSettings) error {\n\t\tsettings.IncludeEmptyDirs = include\n\t\tsettings.IncludeEmptyDirsSet = true\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/coreiface/options/unixfs_test.go", "content": "package options\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestMaxHAMTFanoutValidation(t *testing.T) {\n\tvalid := []int{8, 16, 32, 64, 128, 256, 512, 1024}\n\tfor _, v := range valid {\n\t\t_, _, err := UnixfsAddOptions(Unixfs.MaxHAMTFanout(v))\n\t\trequire.NoError(t, err, \"fanout %d should be valid\", v)\n\t}\n\n\tinvalid := []int{-1, 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 12, 24, 48, 100, 2048, 4096, 999999}\n\tfor _, v := range invalid {\n\t\t_, _, err := UnixfsAddOptions(Unixfs.MaxHAMTFanout(v))\n\t\trequire.Error(t, err, \"fanout %d should be invalid\", v)\n\t\trequire.Contains(t, err.Error(), \"HAMT fanout must be\")\n\t}\n}\n" }, { "path": "core/coreiface/pin.go", "content": "package iface\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\n// Pin holds information about pinned resource\ntype Pin interface {\n\t// Path to the pinned object\n\tPath() path.ImmutablePath\n\n\t// Name is the name of the pin.\n\tName() string\n\n\t// Type of the pin\n\tType() string\n}\n\n// PinStatus holds information about pin health\ntype PinStatus interface {\n\t// Ok indicates whether the pin has been verified to be correct\n\tOk() bool\n\n\t// BadNodes returns any bad (usually missing) nodes from the pin\n\tBadNodes() []BadPinNode\n\n\t// if not nil, an error happened. Everything else should be ignored.\n\tErr() error\n}\n\n// BadPinNode is a node that has been marked as bad by Pin.Verify\ntype BadPinNode interface {\n\t// Path is the path of the node\n\tPath() path.ImmutablePath\n\n\t// Err is the reason why the node has been marked as bad\n\tErr() error\n}\n\n// PinAPI specifies the interface to pining\ntype PinAPI interface {\n\t// Add creates new pin, be default recursive - pinning the whole referenced\n\t// tree\n\tAdd(context.Context, path.Path, ...options.PinAddOption) error\n\n\t// Ls returns this node's pinned objects on the provided channel. The\n\t// channel is closed when there are no more pins and an error is returned.\n\tLs(context.Context, chan<- Pin, ...options.PinLsOption) error\n\n\t// IsPinned returns whether or not the given cid is pinned\n\t// and an explanation of why its pinned\n\tIsPinned(context.Context, path.Path, ...options.PinIsPinnedOption) (string, bool, error)\n\n\t// Rm removes pin for object specified by the path\n\tRm(context.Context, path.Path, ...options.PinRmOption) error\n\n\t// Update changes one pin to another, skipping checks for matching paths in\n\t// the old tree\n\tUpdate(ctx context.Context, from path.Path, to path.Path, opts ...options.PinUpdateOption) error\n\n\t// Verify verifies the integrity of pinned objects\n\tVerify(context.Context) (<-chan PinStatus, error)\n}\n" }, { "path": "core/coreiface/pubsub.go", "content": "package iface\n\nimport (\n\t\"context\"\n\t\"io\"\n\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\n// PubSubSubscription is an active PubSub subscription\ntype PubSubSubscription interface {\n\tio.Closer\n\n\t// Next return the next incoming message\n\tNext(context.Context) (PubSubMessage, error)\n}\n\n// PubSubMessage is a single PubSub message\ntype PubSubMessage interface {\n\t// From returns id of a peer from which the message has arrived\n\tFrom() peer.ID\n\n\t// Data returns the message body\n\tData() []byte\n\n\t// Seq returns message identifier\n\tSeq() []byte\n\n\t// Topics returns list of topics this message was set to\n\tTopics() []string\n}\n\n// PubSubAPI specifies the interface to PubSub\ntype PubSubAPI interface {\n\t// Ls lists subscribed topics by name\n\tLs(context.Context) ([]string, error)\n\n\t// Peers list peers we are currently pubsubbing with\n\tPeers(context.Context, ...options.PubSubPeersOption) ([]peer.ID, error)\n\n\t// Publish a message to a given pubsub topic\n\tPublish(context.Context, string, []byte) error\n\n\t// Subscribe to messages on a given topic\n\tSubscribe(context.Context, string, ...options.PubSubSubscribeOption) (PubSubSubscription, error)\n}\n" }, { "path": "core/coreiface/routing.go", "content": "package iface\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\n// RoutingAPI specifies the interface to the routing layer.\ntype RoutingAPI interface {\n\t// Get retrieves the best value for a given key\n\tGet(context.Context, string) ([]byte, error)\n\n\t// Put sets a value for a given key\n\tPut(ctx context.Context, key string, value []byte, opts ...options.RoutingPutOption) error\n\n\t// FindPeer queries the routing system for all the multiaddresses associated\n\t// with the given [peer.ID].\n\tFindPeer(context.Context, peer.ID) (peer.AddrInfo, error)\n\n\t// FindProviders finds the peers in the routing system who can provide a specific\n\t// value given a key.\n\tFindProviders(context.Context, path.Path, ...options.RoutingFindProvidersOption) (<-chan peer.AddrInfo, error)\n\n\t// Provide announces to the network that you are providing given values\n\tProvide(context.Context, path.Path, ...options.RoutingProvideOption) error\n}\n" }, { "path": "core/coreiface/swarm.go", "content": "package iface\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"time\"\n\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\nvar (\n\tErrNotConnected = errors.New(\"not connected\")\n\tErrConnNotFound = errors.New(\"conn not found\")\n)\n\n// ConnectionInfo contains information about a peer\ntype ConnectionInfo interface {\n\t// ID returns PeerID\n\tID() peer.ID\n\n\t// Address returns the multiaddress via which we are connected with the peer\n\tAddress() ma.Multiaddr\n\n\t// Direction returns which way the connection was established\n\tDirection() network.Direction\n\n\t// Latency returns last known round trip time to the peer\n\tLatency() (time.Duration, error)\n\n\t// Streams returns list of streams established with the peer\n\tStreams() ([]protocol.ID, error)\n}\n\n// SwarmAPI specifies the interface to libp2p swarm\ntype SwarmAPI interface {\n\t// Connect to a given peer\n\tConnect(context.Context, peer.AddrInfo) error\n\n\t// Disconnect from a given address\n\tDisconnect(context.Context, ma.Multiaddr) error\n\n\t// Peers returns the list of peers we are connected to\n\tPeers(context.Context) ([]ConnectionInfo, error)\n\n\t// KnownAddrs returns the list of all addresses this node is aware of\n\tKnownAddrs(context.Context) (map[peer.ID][]ma.Multiaddr, error)\n\n\t// LocalAddrs returns the list of announced listening addresses\n\tLocalAddrs(context.Context) ([]ma.Multiaddr, error)\n\n\t// ListenAddrs returns the list of all listening addresses\n\tListenAddrs(context.Context) ([]ma.Multiaddr, error)\n}\n" }, { "path": "core/coreiface/tests/api.go", "content": "package tests\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"testing\"\n\t\"time\"\n\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n)\n\nvar errAPINotImplemented = errors.New(\"api not implemented\")\n\ntype Provider interface {\n\t// Make creates n nodes. fullIdentity set to false can be ignored\n\tMakeAPISwarm(t *testing.T, ctx context.Context, fullIdentity bool, online bool, n int) ([]coreiface.CoreAPI, error)\n}\n\nfunc (tp *TestSuite) makeAPISwarm(t *testing.T, ctx context.Context, fullIdentity bool, online bool, n int) ([]coreiface.CoreAPI, error) {\n\tif tp.apis != nil {\n\t\ttp.apis <- 1\n\t\tgo func() {\n\t\t\t<-ctx.Done()\n\t\t\ttp.apis <- -1\n\t\t}()\n\t}\n\n\treturn tp.Provider.MakeAPISwarm(t, ctx, fullIdentity, online, n)\n}\n\nfunc (tp *TestSuite) makeAPI(t *testing.T, ctx context.Context) (coreiface.CoreAPI, error) {\n\tapi, err := tp.makeAPISwarm(t, ctx, false, false, 1)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api[0], nil\n}\n\nfunc (tp *TestSuite) makeAPIWithIdentityAndOffline(t *testing.T, ctx context.Context) (coreiface.CoreAPI, error) {\n\tapi, err := tp.makeAPISwarm(t, ctx, true, false, 1)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn api[0], nil\n}\n\nfunc (tp *TestSuite) MakeAPISwarm(t *testing.T, ctx context.Context, n int) ([]coreiface.CoreAPI, error) {\n\treturn tp.makeAPISwarm(t, ctx, true, true, n)\n}\n\ntype TestSuite struct {\n\tProvider\n\n\tapis chan int\n}\n\nfunc TestApi(p Provider) func(t *testing.T) {\n\trunning := 1\n\tapis := make(chan int)\n\tzeroRunning := make(chan struct{})\n\tgo func() {\n\t\tfor i := range apis {\n\t\t\trunning += i\n\t\t\tif running < 1 {\n\t\t\t\tclose(zeroRunning)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\ttp := &TestSuite{Provider: p, apis: apis}\n\n\treturn func(t *testing.T) {\n\t\tt.Run(\"Block\", tp.TestBlock)\n\t\tt.Run(\"Dag\", tp.TestDag)\n\t\tt.Run(\"Key\", tp.TestKey)\n\t\tt.Run(\"Name\", tp.TestName)\n\t\tt.Run(\"Object\", tp.TestObject)\n\t\tt.Run(\"Path\", tp.TestPath)\n\t\tt.Run(\"Pin\", tp.TestPin)\n\t\tt.Run(\"PubSub\", tp.TestPubSub)\n\t\tt.Run(\"Routing\", tp.TestRouting)\n\t\tt.Run(\"Unixfs\", tp.TestUnixfs)\n\n\t\tapis <- -1\n\t\tt.Run(\"TestsCancelCtx\", func(t *testing.T) {\n\t\t\tselect {\n\t\t\tcase <-zeroRunning:\n\t\t\tcase <-time.After(time.Second):\n\t\t\t\tt.Errorf(\"%d test swarms(s) not closed\", running)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc (tp *TestSuite) hasApi(t *testing.T, tf func(coreiface.CoreAPI) error) {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := tf(api); err != nil {\n\t\tt.Fatal(api)\n\t}\n}\n" }, { "path": "core/coreiface/tests/block.go", "content": "package tests\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\topt \"github.com/ipfs/kubo/core/coreiface/options\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nvar (\n\tpbCidV0 = \"QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\" // dag-pb\n\tpbCid = \"bafybeiffndsajwhk3lwjewwdxqntmjm4b5wxaaanokonsggenkbw6slwk4\" // dag-pb\n\trawCid = \"bafkreiffndsajwhk3lwjewwdxqntmjm4b5wxaaanokonsggenkbw6slwk4\" // raw bytes\n\tcborCid = \"bafyreicnga62zhxnmnlt6ymq5hcbsg7gdhqdu6z4ehu3wpjhvqnflfy6nm\" // dag-cbor\n\tcborKCid = \"bafyr2qgsohbwdlk7ajmmbb4lhoytmest4wdbe5xnexfvtxeatuyqqmwv3fgxp3pmhpc27gwey2cct56gloqefoqwcf3yqiqzsaqb7p4jefhcw\" // dag-cbor keccak-512\n)\n\n// dag-pb\nfunc pbBlock() io.Reader {\n\treturn bytes.NewReader([]byte{10, 12, 8, 2, 18, 6, 104, 101, 108, 108, 111, 10, 24, 6})\n}\n\n// dag-cbor\nfunc cborBlock() io.Reader {\n\treturn bytes.NewReader([]byte{101, 72, 101, 108, 108, 111})\n}\n\nfunc (tp *TestSuite) TestBlock(t *testing.T) {\n\ttp.hasApi(t, func(api coreiface.CoreAPI) error {\n\t\tif api.Block() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestBlockPut (get raw CIDv1)\", tp.TestBlockPut)\n\tt.Run(\"TestBlockPutCidCodec: dag-pb\", tp.TestBlockPutCidCodecDagPb)\n\tt.Run(\"TestBlockPutCidCodec: dag-cbor\", tp.TestBlockPutCidCodecDagCbor)\n\tt.Run(\"TestBlockPutFormat (legacy): cbor → dag-cbor\", tp.TestBlockPutFormatDagCbor)\n\tt.Run(\"TestBlockPutFormat (legacy): protobuf → dag-pb\", tp.TestBlockPutFormatDagPb)\n\tt.Run(\"TestBlockPutFormat (legacy): v0 → CIDv0\", tp.TestBlockPutFormatV0)\n\tt.Run(\"TestBlockPutHash\", tp.TestBlockPutHash)\n\tt.Run(\"TestBlockGet\", tp.TestBlockGet)\n\tt.Run(\"TestBlockRm\", tp.TestBlockRm)\n\tt.Run(\"TestBlockStat\", tp.TestBlockStat)\n\tt.Run(\"TestBlockPin\", tp.TestBlockPin)\n}\n\n// when no opts are passed, produced CID has 'raw' codec\nfunc (tp *TestSuite) TestBlockPut(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, pbBlock())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != rawCid {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\n// Format is deprecated, it used invalid codec names.\n// Confirm 'cbor' gets fixed to 'dag-cbor'\nfunc (tp *TestSuite) TestBlockPutFormatDagCbor(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, cborBlock(), opt.Block.Format(\"cbor\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != cborCid {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\n// Format is deprecated, it used invalid codec names.\n// Confirm 'protobuf' got fixed to 'dag-pb'\nfunc (tp *TestSuite) TestBlockPutFormatDagPb(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, pbBlock(), opt.Block.Format(\"protobuf\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != pbCid {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\n// Format is deprecated, it used invalid codec names.\n// Confirm fake codec 'v0' got fixed to CIDv0 (with implicit dag-pb codec)\nfunc (tp *TestSuite) TestBlockPutFormatV0(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, pbBlock(), opt.Block.Format(\"v0\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != pbCidV0 {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockPutCidCodecDagCbor(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, cborBlock(), opt.Block.CidCodec(\"dag-cbor\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != cborCid {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockPutCidCodecDagPb(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, pbBlock(), opt.Block.CidCodec(\"dag-pb\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != pbCid {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockPutHash(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(\n\t\tctx,\n\t\tcborBlock(),\n\t\topt.Block.Hash(mh.KECCAK_512, -1),\n\t\topt.Block.CidCodec(\"dag-cbor\"),\n\t)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif res.Path().RootCid().String() != cborKCid {\n\t\tt.Errorf(\"got wrong cid: %s\", res.Path().RootCid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockGet(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, strings.NewReader(`Hello`), opt.Block.Format(\"raw\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr, err := api.Block().Get(ctx, res.Path())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\td, err := io.ReadAll(r)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif string(d) != \"Hello\" {\n\t\tt.Error(\"didn't get correct data back\")\n\t}\n\n\tp := path.FromCid(res.Path().RootCid())\n\n\trp, _, err := api.ResolvePath(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif rp.RootCid().String() != res.Path().RootCid().String() {\n\t\tt.Error(\"paths didn't match\")\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockRm(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, strings.NewReader(`Hello`), opt.Block.Format(\"raw\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr, err := api.Block().Get(ctx, res.Path())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\td, err := io.ReadAll(r)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif string(d) != \"Hello\" {\n\t\tt.Error(\"didn't get correct data back\")\n\t}\n\n\terr = api.Block().Rm(ctx, res.Path())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, err = api.Block().Get(ctx, res.Path())\n\tif err == nil {\n\t\tt.Fatal(\"expected err to exist\")\n\t}\n\tif !ipld.IsNotFound(err) {\n\t\tt.Errorf(\"unexpected error; %s\", err.Error())\n\t}\n\n\terr = api.Block().Rm(ctx, res.Path())\n\tif err == nil {\n\t\tt.Fatal(\"expected err to exist\")\n\t}\n\tif !ipld.IsNotFound(err) {\n\t\tt.Errorf(\"unexpected error; %s\", err.Error())\n\t}\n\n\terr = api.Block().Rm(ctx, res.Path(), opt.Block.Force(true))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockStat(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(ctx, strings.NewReader(`Hello`), opt.Block.Format(\"raw\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tstat, err := api.Block().Stat(ctx, res.Path())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif stat.Path().String() != res.Path().String() {\n\t\tt.Error(\"paths don't match\")\n\t}\n\n\tif stat.Size() != len(\"Hello\") {\n\t\tt.Error(\"length doesn't match\")\n\t}\n}\n\nfunc (tp *TestSuite) TestBlockPin(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, err = api.Block().Put(ctx, strings.NewReader(`Hello`), opt.Block.Format(\"raw\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpinCh := make(chan coreiface.Pin)\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, pinCh)\n\t}()\n\n\tfor range pinCh {\n\t\tt.Fatal(\"expected 0 pins\")\n\t}\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Block().Put(\n\t\tctx,\n\t\tstrings.NewReader(`Hello`),\n\t\topt.Block.Pin(true),\n\t\topt.Block.Format(\"raw\"),\n\t)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpins, err := accPins(ctx, api)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(pins) != 1 {\n\t\tt.Fatal(\"expected 1 pin\")\n\t}\n\tif pins[0].Type() != \"recursive\" {\n\t\tt.Error(\"expected a recursive pin\")\n\t}\n\tif pins[0].Path().String() != res.Path().String() {\n\t\tt.Error(\"pin path didn't match\")\n\t}\n}\n" }, { "path": "core/coreiface/tests/dag.go", "content": "package tests\n\nimport (\n\t\"math\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\n\tipldcbor \"github.com/ipfs/go-ipld-cbor\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nfunc (tp *TestSuite) TestDag(t *testing.T) {\n\ttp.hasApi(t, func(api coreiface.CoreAPI) error {\n\t\tif api.Dag() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestPut\", tp.TestPut)\n\tt.Run(\"TestPutWithHash\", tp.TestPutWithHash)\n\tt.Run(\"TestPath\", tp.TestDagPath)\n\tt.Run(\"TestTree\", tp.TestTree)\n\tt.Run(\"TestBatch\", tp.TestBatch)\n}\n\nvar treeExpected = map[string]struct{}{\n\t\"a\": {},\n\t\"b\": {},\n\t\"c\": {},\n\t\"c/d\": {},\n\t\"c/e\": {},\n}\n\nfunc (tp *TestSuite) TestPut(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`\"Hello\"`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Dag().Add(ctx, nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif nd.Cid().String() != \"bafyreicnga62zhxnmnlt6ymq5hcbsg7gdhqdu6z4ehu3wpjhvqnflfy6nm\" {\n\t\tt.Errorf(\"got wrong cid: %s\", nd.Cid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestPutWithHash(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`\"Hello\"`), mh.SHA3_256, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Dag().Add(ctx, nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif nd.Cid().String() != \"bafyrmifu7haikttpqqgc5ewvmp76z3z4ebp7h2ph4memw7dq4nt6btmxny\" {\n\t\tt.Errorf(\"got wrong cid: %s\", nd.Cid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestDagPath(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsnd, err := ipldcbor.FromJSON(strings.NewReader(`\"foo\"`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Dag().Add(ctx, snd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`{\"lnk\": {\"/\": \"`+snd.Cid().String()+`\"}}`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Dag().Add(ctx, nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp, err := path.Join(path.FromCid(nd.Cid()), \"lnk\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\trp, _, err := api.ResolvePath(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tndd, err := api.Dag().Get(ctx, rp.RootCid())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif ndd.Cid().String() != snd.Cid().String() {\n\t\tt.Errorf(\"got unexpected cid %s, expected %s\", ndd.Cid().String(), snd.Cid().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestTree(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`{\"a\": 123, \"b\": \"foo\", \"c\": {\"d\": 321, \"e\": 111}}`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Dag().Add(ctx, nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tres, err := api.Dag().Get(ctx, nd.Cid())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tlst := res.Tree(\"\", -1)\n\tif len(lst) != len(treeExpected) {\n\t\tt.Errorf(\"tree length of %d doesn't match expected %d\", len(lst), len(treeExpected))\n\t}\n\n\tfor _, ent := range lst {\n\t\tif _, ok := treeExpected[ent]; !ok {\n\t\t\tt.Errorf(\"unexpected tree entry %s\", ent)\n\t\t}\n\t}\n}\n\nfunc (tp *TestSuite) TestBatch(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`\"Hello\"`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif nd.Cid().String() != \"bafyreicnga62zhxnmnlt6ymq5hcbsg7gdhqdu6z4ehu3wpjhvqnflfy6nm\" {\n\t\tt.Errorf(\"got wrong cid: %s\", nd.Cid().String())\n\t}\n\n\t_, err = api.Dag().Get(ctx, nd.Cid())\n\tif err == nil || !strings.Contains(err.Error(), \"not found\") {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := api.Dag().AddMany(ctx, []ipld.Node{nd}); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, err = api.Dag().Get(ctx, nd.Cid())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n" }, { "path": "core/coreiface/tests/key.go", "content": "package tests\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/go-cid\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\topt \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmbase \"github.com/multiformats/go-multibase\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc (tp *TestSuite) TestKey(t *testing.T) {\n\ttp.hasApi(t, func(api iface.CoreAPI) error {\n\t\tif api.Key() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestListSelf\", tp.TestListSelf)\n\tt.Run(\"TestRenameSelf\", tp.TestRenameSelf)\n\tt.Run(\"TestRemoveSelf\", tp.TestRemoveSelf)\n\tt.Run(\"TestGenerate\", tp.TestGenerate)\n\tt.Run(\"TestGenerateSize\", tp.TestGenerateSize)\n\tt.Run(\"TestGenerateType\", tp.TestGenerateType)\n\tt.Run(\"TestGenerateExisting\", tp.TestGenerateExisting)\n\tt.Run(\"TestList\", tp.TestList)\n\tt.Run(\"TestRename\", tp.TestRename)\n\tt.Run(\"TestRenameToSelf\", tp.TestRenameToSelf)\n\tt.Run(\"TestRenameToSelfForce\", tp.TestRenameToSelfForce)\n\tt.Run(\"TestRenameOverwriteNoForce\", tp.TestRenameOverwriteNoForce)\n\tt.Run(\"TestRenameOverwrite\", tp.TestRenameOverwrite)\n\tt.Run(\"TestRenameSameNameNoForce\", tp.TestRenameSameNameNoForce)\n\tt.Run(\"TestRenameSameName\", tp.TestRenameSameName)\n\tt.Run(\"TestSign\", tp.TestSign)\n\tt.Run(\"TestVerify\", tp.TestVerify)\n}\n\nfunc (tp *TestSuite) TestListSelf(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tself, err := api.Key().Self(ctx)\n\trequire.NoError(t, err)\n\n\tkeys, err := api.Key().List(ctx)\n\trequire.NoError(t, err)\n\trequire.Len(t, keys, 1)\n\tassert.Equal(t, \"self\", keys[0].Name())\n\tassert.Equal(t, \"/ipns/\"+iface.FormatKeyID(self.ID()), keys[0].Path().String())\n}\n\nfunc (tp *TestSuite) TestRenameSelf(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, _, err = api.Key().Rename(ctx, \"self\", \"foo\")\n\trequire.ErrorContains(t, err, \"cannot rename key with name 'self'\")\n\n\t_, _, err = api.Key().Rename(ctx, \"self\", \"foo\", opt.Key.Force(true))\n\trequire.ErrorContains(t, err, \"cannot rename key with name 'self'\")\n}\n\nfunc (tp *TestSuite) TestRemoveSelf(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Remove(ctx, \"self\")\n\trequire.ErrorContains(t, err, \"cannot remove key with name 'self'\")\n}\n\nfunc (tp *TestSuite) TestGenerate(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tk, err := api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"foo\", k.Name())\n\n\tverifyIPNSPath(t, k.Path().String())\n}\n\nfunc verifyIPNSPath(t *testing.T, p string) {\n\tt.Helper()\n\n\trequire.True(t, strings.HasPrefix(p, \"/ipns/\"))\n\n\tk := p[len(\"/ipns/\"):]\n\tc, err := cid.Decode(k)\n\trequire.NoError(t, err)\n\n\tb36, err := c.StringOfBase(mbase.Base36)\n\trequire.NoError(t, err)\n\trequire.Equal(t, k, b36)\n}\n\nfunc (tp *TestSuite) TestGenerateSize(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tk, err := api.Key().Generate(ctx, \"foo\", opt.Key.Size(2048))\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"foo\", k.Name())\n\n\tverifyIPNSPath(t, k.Path().String())\n}\n\nfunc (tp *TestSuite) TestGenerateType(t *testing.T) {\n\tt.Skip(\"disabled until libp2p/specs#111 is fixed\")\n\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tk, err := api.Key().Generate(ctx, \"bar\", opt.Key.Type(opt.Ed25519Key))\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"bar\", k.Name())\n\t// Expected to be an inlined identity hash.\n\trequire.True(t, strings.HasPrefix(k.Path().String(), \"/ipns/12\"))\n}\n\nfunc (tp *TestSuite) TestGenerateExisting(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.ErrorContains(t, err, \"key with name 'foo' already exists\")\n\n\t_, err = api.Key().Generate(ctx, \"self\")\n\trequire.ErrorContains(t, err, \"cannot create key with name 'self'\")\n}\n\nfunc (tp *TestSuite) TestList(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\tl, err := api.Key().List(ctx)\n\trequire.NoError(t, err)\n\trequire.Len(t, l, 2)\n\trequire.Equal(t, \"self\", l[0].Name())\n\trequire.Equal(t, \"foo\", l[1].Name())\n\n\tverifyIPNSPath(t, l[0].Path().String())\n\tverifyIPNSPath(t, l[1].Path().String())\n}\n\nfunc (tp *TestSuite) TestRename(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\tk, overwrote, err := api.Key().Rename(ctx, \"foo\", \"bar\")\n\trequire.NoError(t, err)\n\tassert.False(t, overwrote)\n\tassert.Equal(t, \"bar\", k.Name())\n}\n\nfunc (tp *TestSuite) TestRenameToSelf(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\t_, _, err = api.Key().Rename(ctx, \"foo\", \"self\")\n\trequire.ErrorContains(t, err, \"cannot overwrite key with name 'self'\")\n}\n\nfunc (tp *TestSuite) TestRenameToSelfForce(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\t_, _, err = api.Key().Rename(ctx, \"foo\", \"self\", opt.Key.Force(true))\n\trequire.ErrorContains(t, err, \"cannot overwrite key with name 'self'\")\n}\n\nfunc (tp *TestSuite) TestRenameOverwriteNoForce(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"bar\")\n\trequire.NoError(t, err)\n\n\t_, _, err = api.Key().Rename(ctx, \"foo\", \"bar\")\n\trequire.ErrorContains(t, err, \"key by that name already exists, refusing to overwrite\")\n}\n\nfunc (tp *TestSuite) TestRenameOverwrite(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tkfoo, err := api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"bar\")\n\trequire.NoError(t, err)\n\n\tk, overwrote, err := api.Key().Rename(ctx, \"foo\", \"bar\", opt.Key.Force(true))\n\trequire.NoError(t, err)\n\trequire.True(t, overwrote)\n\tassert.Equal(t, \"bar\", k.Name())\n\tassert.Equal(t, kfoo.Path().String(), k.Path().String())\n}\n\nfunc (tp *TestSuite) TestRenameSameNameNoForce(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\tk, overwrote, err := api.Key().Rename(ctx, \"foo\", \"foo\")\n\trequire.NoError(t, err)\n\tassert.False(t, overwrote)\n\tassert.Equal(t, \"foo\", k.Name())\n}\n\nfunc (tp *TestSuite) TestRenameSameName(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t_, err = api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\tk, overwrote, err := api.Key().Rename(ctx, \"foo\", \"foo\", opt.Key.Force(true))\n\trequire.NoError(t, err)\n\tassert.False(t, overwrote)\n\tassert.Equal(t, \"foo\", k.Name())\n}\n\nfunc (tp *TestSuite) TestRemove(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tk, err := api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\tl, err := api.Key().List(ctx)\n\trequire.NoError(t, err)\n\trequire.Len(t, l, 2)\n\n\tp, err := api.Key().Remove(ctx, \"foo\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, p.Path().String(), k.Path().String())\n\n\tl, err = api.Key().List(ctx)\n\trequire.NoError(t, err)\n\trequire.Len(t, l, 1)\n\tassert.Equal(t, \"self\", l[0].Name())\n}\n\nfunc (tp *TestSuite) TestSign(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tkey1, err := api.Key().Generate(ctx, \"foo\", opt.Key.Type(opt.Ed25519Key))\n\trequire.NoError(t, err)\n\n\tdata := []byte(\"hello world\")\n\n\tkey2, signature, err := api.Key().Sign(ctx, \"foo\", data)\n\trequire.NoError(t, err)\n\n\trequire.Equal(t, key1.Name(), key2.Name())\n\trequire.Equal(t, key1.ID(), key2.ID())\n\n\tpk, err := key1.ID().ExtractPublicKey()\n\trequire.NoError(t, err)\n\n\tvalid, err := pk.Verify(append([]byte(\"libp2p-key signed message:\"), data...), signature)\n\trequire.NoError(t, err)\n\trequire.True(t, valid)\n}\n\nfunc (tp *TestSuite) TestVerify(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"Verify Own Key\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tctx := t.Context()\n\n\t\tapi, err := tp.makeAPI(t, ctx)\n\t\trequire.NoError(t, err)\n\n\t\t_, err = api.Key().Generate(ctx, \"foo\", opt.Key.Type(opt.Ed25519Key))\n\t\trequire.NoError(t, err)\n\n\t\tdata := []byte(\"hello world\")\n\n\t\t_, signature, err := api.Key().Sign(ctx, \"foo\", data)\n\t\trequire.NoError(t, err)\n\n\t\t_, valid, err := api.Key().Verify(ctx, \"foo\", signature, data)\n\t\trequire.NoError(t, err)\n\t\trequire.True(t, valid)\n\t})\n\n\tt.Run(\"Verify Self\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tctx := t.Context()\n\n\t\tapi, err := tp.makeAPIWithIdentityAndOffline(t, ctx)\n\t\trequire.NoError(t, err)\n\n\t\tdata := []byte(\"hello world\")\n\n\t\t_, signature, err := api.Key().Sign(ctx, \"\", data)\n\t\trequire.NoError(t, err)\n\n\t\t_, valid, err := api.Key().Verify(ctx, \"\", signature, data)\n\t\trequire.NoError(t, err)\n\t\trequire.True(t, valid)\n\t})\n\n\tt.Run(\"Verify With Key In Different Formats\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Spin some node and get signature out.\n\t\tctx := t.Context()\n\n\t\tapi, err := tp.makeAPI(t, ctx)\n\t\trequire.NoError(t, err)\n\n\t\tkey, err := api.Key().Generate(ctx, \"foo\", opt.Key.Type(opt.Ed25519Key))\n\t\trequire.NoError(t, err)\n\n\t\tdata := []byte(\"hello world\")\n\n\t\t_, signature, err := api.Key().Sign(ctx, \"foo\", data)\n\t\trequire.NoError(t, err)\n\n\t\tfor _, testCase := range [][]string{\n\t\t\t{\"Base58 Encoded Peer ID\", key.ID().String()},\n\t\t\t{\"CIDv1 Encoded Peer ID\", peer.ToCid(key.ID()).String()},\n\t\t\t{\"CIDv1 Encoded IPNS Name\", ipns.NameFromPeer(key.ID()).String()},\n\t\t\t{\"Prefixed IPNS Path\", ipns.NameFromPeer(key.ID()).AsPath().String()},\n\t\t} {\n\t\t\tt.Run(testCase[0], func(t *testing.T) {\n\t\t\t\tctx := t.Context()\n\n\t\t\t\t// Spin new node.\n\t\t\t\tapi, err := tp.makeAPI(t, ctx)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t_, valid, err := api.Key().Verify(ctx, testCase[1], signature, data)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.True(t, valid)\n\t\t\t})\n\t\t}\n\t})\n}\n" }, { "path": "core/coreiface/tests/name.go", "content": "package tests\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/path\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\topt \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc (tp *TestSuite) TestName(t *testing.T) {\n\ttp.hasApi(t, func(api coreiface.CoreAPI) error {\n\t\tif api.Name() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestPublishResolve\", tp.TestPublishResolve)\n\tt.Run(\"TestBasicPublishResolveKey\", tp.TestBasicPublishResolveKey)\n\tt.Run(\"TestBasicPublishResolveTimeout\", tp.TestBasicPublishResolveTimeout)\n}\n\nvar rnd = rand.New(rand.NewSource(0x62796532303137))\n\nfunc addTestObject(ctx context.Context, api coreiface.CoreAPI) (path.Path, error) {\n\treturn api.Unixfs().Add(ctx, files.NewReaderFile(&io.LimitedReader{R: rnd, N: 4092}))\n}\n\nfunc (tp *TestSuite) TestPublishResolve(t *testing.T) {\n\tctx := t.Context()\n\tinit := func() (coreiface.CoreAPI, path.Path) {\n\t\tapis, err := tp.MakeAPISwarm(t, ctx, 5)\n\t\trequire.NoError(t, err)\n\t\tapi := apis[0]\n\n\t\tp, err := addTestObject(ctx, api)\n\t\trequire.NoError(t, err)\n\t\treturn api, p\n\t}\n\trun := func(t *testing.T, ropts []opt.NameResolveOption) {\n\t\tt.Run(\"basic\", func(t *testing.T) {\n\t\t\tapi, p := init()\n\t\t\tname, err := api.Name().Publish(ctx, p)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tself, err := api.Key().Self(ctx)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, name.String(), ipns.NameFromPeer(self.ID()).String())\n\n\t\t\tresPath, err := api.Name().Resolve(ctx, name.String(), ropts...)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, p.String(), resPath.String())\n\t\t})\n\n\t\tt.Run(\"publishPath\", func(t *testing.T) {\n\t\t\tapi, p := init()\n\t\t\tp, err := path.Join(p, \"/test\")\n\t\t\trequire.NoError(t, err)\n\n\t\t\tname, err := api.Name().Publish(ctx, p)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tself, err := api.Key().Self(ctx)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, name.String(), ipns.NameFromPeer(self.ID()).String())\n\n\t\t\tresPath, err := api.Name().Resolve(ctx, name.String(), ropts...)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, p.String(), resPath.String())\n\t\t})\n\n\t\tt.Run(\"revolvePath\", func(t *testing.T) {\n\t\t\tapi, p := init()\n\t\t\tname, err := api.Name().Publish(ctx, p)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tself, err := api.Key().Self(ctx)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, name.String(), ipns.NameFromPeer(self.ID()).String())\n\n\t\t\tresPath, err := api.Name().Resolve(ctx, name.String()+\"/test\", ropts...)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, p.String()+\"/test\", resPath.String())\n\t\t})\n\n\t\tt.Run(\"publishRevolvePath\", func(t *testing.T) {\n\t\t\tapi, p := init()\n\t\t\tp, err := path.Join(p, \"/a\")\n\t\t\trequire.NoError(t, err)\n\n\t\t\tname, err := api.Name().Publish(ctx, p)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tself, err := api.Key().Self(ctx)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, name.String(), ipns.NameFromPeer(self.ID()).String())\n\n\t\t\tresPath, err := api.Name().Resolve(ctx, name.String()+\"/b\", ropts...)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, p.String()+\"/b\", resPath.String())\n\t\t})\n\t}\n\n\tt.Run(\"default\", func(t *testing.T) {\n\t\trun(t, []opt.NameResolveOption{})\n\t})\n\n\tt.Run(\"nocache\", func(t *testing.T) {\n\t\trun(t, []opt.NameResolveOption{opt.Name.Cache(false)})\n\t})\n}\n\nfunc (tp *TestSuite) TestBasicPublishResolveKey(t *testing.T) {\n\tctx := t.Context()\n\tapis, err := tp.MakeAPISwarm(t, ctx, 5)\n\trequire.NoError(t, err)\n\tapi := apis[0]\n\n\tk, err := api.Key().Generate(ctx, \"foo\")\n\trequire.NoError(t, err)\n\n\tp, err := addTestObject(ctx, api)\n\trequire.NoError(t, err)\n\n\tname, err := api.Name().Publish(ctx, p, opt.Name.Key(k.Name()))\n\trequire.NoError(t, err)\n\trequire.Equal(t, name.String(), ipns.NameFromPeer(k.ID()).String())\n\n\tresPath, err := api.Name().Resolve(ctx, name.String())\n\trequire.NoError(t, err)\n\trequire.Equal(t, p.String(), resPath.String())\n}\n\nfunc (tp *TestSuite) TestBasicPublishResolveTimeout(t *testing.T) {\n\tctx := t.Context()\n\tapis, err := tp.MakeAPISwarm(t, ctx, 5)\n\trequire.NoError(t, err)\n\tapi := apis[0]\n\tp, err := addTestObject(ctx, api)\n\trequire.NoError(t, err)\n\n\tself, err := api.Key().Self(ctx)\n\trequire.NoError(t, err)\n\n\tname, err := api.Name().Publish(ctx, p, opt.Name.ValidTime(time.Second*1))\n\trequire.NoError(t, err)\n\trequire.Equal(t, name.String(), ipns.NameFromPeer(self.ID()).String())\n\n\t// First resolve should succeed (before expiration)\n\tresPath, err := api.Name().Resolve(ctx, name.String())\n\trequire.NoError(t, err)\n\trequire.Equal(t, p.String(), resPath.String())\n\n\t// Wait for record to expire (1 second ValidTime + buffer)\n\ttime.Sleep(time.Second * 2)\n\n\t// Second resolve should now fail after ValidTime expiration (cached)\n\t_, err = api.Name().Resolve(ctx, name.String())\n\trequire.Error(t, err, \"IPNS resolution should fail after ValidTime expires (cached)\")\n\n\t// Third resolve should also fail after ValidTime expiration (non-cached)\n\t_, err = api.Name().Resolve(ctx, name.String(), opt.Name.Cache(false))\n\trequire.Error(t, err, \"IPNS resolution should fail after ValidTime expires (non-cached)\")\n}\n\n// TODO: When swarm api is created, add multinode tests\n" }, { "path": "core/coreiface/tests/object.go", "content": "package tests\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/path\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\topt \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc (tp *TestSuite) TestObject(t *testing.T) {\n\ttp.hasApi(t, func(api iface.CoreAPI) error {\n\t\tif api.Object() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestObjectAddLink\", tp.TestObjectAddLink)\n\tt.Run(\"TestObjectAddLinkCreate\", tp.TestObjectAddLinkCreate)\n\tt.Run(\"TestObjectRmLink\", tp.TestObjectRmLink)\n\tt.Run(\"TestDiffTest\", tp.TestDiffTest)\n}\n\nfunc putDagPbNode(t *testing.T, ctx context.Context, api iface.CoreAPI, data string, links []*ipld.Link) path.ImmutablePath {\n\tdagnode := new(dag.ProtoNode)\n\n\tif data != \"\" {\n\t\tdagnode.SetData([]byte(data))\n\t}\n\n\tif links != nil {\n\t\terr := dagnode.SetLinks(links)\n\t\trequire.NoError(t, err)\n\t}\n\n\terr := api.Dag().Add(ctx, dagnode)\n\trequire.NoError(t, err)\n\n\treturn path.FromCid(dagnode.Cid())\n}\n\nfunc (tp *TestSuite) TestObjectAddLink(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tp1 := putDagPbNode(t, ctx, api, \"foo\", nil)\n\tp2 := putDagPbNode(t, ctx, api, \"bazz\", []*ipld.Link{\n\t\t{\n\t\t\tName: \"bar\",\n\t\t\tCid: p1.RootCid(),\n\t\t\tSize: 3,\n\t\t},\n\t})\n\n\tp3, err := api.Object().AddLink(ctx, p2, \"abc\", p2)\n\trequire.NoError(t, err)\n\n\tnd, err := api.Dag().Get(ctx, p3.RootCid())\n\trequire.NoError(t, err)\n\n\tlinks := nd.Links()\n\trequire.Len(t, links, 2)\n\trequire.Equal(t, \"abc\", links[0].Name)\n\trequire.Equal(t, \"bar\", links[1].Name)\n}\n\nfunc (tp *TestSuite) TestObjectAddLinkCreate(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tp1 := putDagPbNode(t, ctx, api, \"foo\", nil)\n\tp2 := putDagPbNode(t, ctx, api, \"bazz\", []*ipld.Link{\n\t\t{\n\t\t\tName: \"bar\",\n\t\t\tCid: p1.RootCid(),\n\t\t\tSize: 3,\n\t\t},\n\t})\n\n\t_, err = api.Object().AddLink(ctx, p2, \"abc/d\", p2)\n\trequire.ErrorContains(t, err, \"no link by that name\")\n\n\tp3, err := api.Object().AddLink(ctx, p2, \"abc/d\", p2, opt.Object.Create(true))\n\trequire.NoError(t, err)\n\n\tnd, err := api.Dag().Get(ctx, p3.RootCid())\n\trequire.NoError(t, err)\n\n\tlinks := nd.Links()\n\trequire.Len(t, links, 2)\n\trequire.Equal(t, \"abc\", links[0].Name)\n\trequire.Equal(t, \"bar\", links[1].Name)\n}\n\nfunc (tp *TestSuite) TestObjectRmLink(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tp1 := putDagPbNode(t, ctx, api, \"foo\", nil)\n\tp2 := putDagPbNode(t, ctx, api, \"bazz\", []*ipld.Link{\n\t\t{\n\t\t\tName: \"bar\",\n\t\t\tCid: p1.RootCid(),\n\t\t\tSize: 3,\n\t\t},\n\t})\n\n\tp3, err := api.Object().RmLink(ctx, p2, \"bar\")\n\trequire.NoError(t, err)\n\n\tnd, err := api.Dag().Get(ctx, p3.RootCid())\n\trequire.NoError(t, err)\n\n\tlinks := nd.Links()\n\trequire.Len(t, links, 0)\n}\n\nfunc (tp *TestSuite) TestDiffTest(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tp1 := putDagPbNode(t, ctx, api, \"foo\", nil)\n\tp2 := putDagPbNode(t, ctx, api, \"bar\", nil)\n\n\tchanges, err := api.Object().Diff(ctx, p1, p2)\n\trequire.NoError(t, err)\n\trequire.Len(t, changes, 1)\n\trequire.Equal(t, iface.DiffMod, changes[0].Type)\n\trequire.Equal(t, p1.String(), changes[0].Before.String())\n\trequire.Equal(t, p2.String(), changes[0].After.String())\n}\n" }, { "path": "core/coreiface/tests/path.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tipldcbor \"github.com/ipfs/go-ipld-cbor\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc newIPLDPath(t *testing.T, cid cid.Cid) path.ImmutablePath {\n\tp, err := path.NewPath(fmt.Sprintf(\"/%s/%s\", path.IPLDNamespace, cid.String()))\n\trequire.NoError(t, err)\n\tim, err := path.NewImmutablePath(p)\n\trequire.NoError(t, err)\n\treturn im\n}\n\nfunc (tp *TestSuite) TestPath(t *testing.T) {\n\tt.Run(\"TestMutablePath\", tp.TestMutablePath)\n\tt.Run(\"TestPathRemainder\", tp.TestPathRemainder)\n\tt.Run(\"TestEmptyPathRemainder\", tp.TestEmptyPathRemainder)\n\tt.Run(\"TestInvalidPathRemainder\", tp.TestInvalidPathRemainder)\n\tt.Run(\"TestPathRoot\", tp.TestPathRoot)\n\tt.Run(\"TestPathJoin\", tp.TestPathJoin)\n}\n\nfunc (tp *TestSuite) TestMutablePath(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\tblk, err := api.Block().Put(ctx, strings.NewReader(`foo`))\n\trequire.NoError(t, err)\n\trequire.False(t, blk.Path().Mutable())\n\trequire.NotNil(t, api.Key())\n\n\tkeys, err := api.Key().List(ctx)\n\trequire.NoError(t, err)\n\trequire.True(t, keys[0].Path().Mutable())\n}\n\nfunc (tp *TestSuite) TestPathRemainder(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, api.Dag())\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`{\"foo\": {\"bar\": \"baz\"}}`), math.MaxUint64, -1)\n\trequire.NoError(t, err)\n\n\terr = api.Dag().Add(ctx, nd)\n\trequire.NoError(t, err)\n\n\tp, err := path.Join(path.FromCid(nd.Cid()), \"foo\", \"bar\")\n\trequire.NoError(t, err)\n\n\t_, remainder, err := api.ResolvePath(ctx, p)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"/foo/bar\", path.SegmentsToString(remainder...))\n}\n\nfunc (tp *TestSuite) TestEmptyPathRemainder(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, api.Dag())\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`{\"foo\": {\"bar\": \"baz\"}}`), math.MaxUint64, -1)\n\trequire.NoError(t, err)\n\n\terr = api.Dag().Add(ctx, nd)\n\trequire.NoError(t, err)\n\n\t_, remainder, err := api.ResolvePath(ctx, path.FromCid(nd.Cid()))\n\trequire.NoError(t, err)\n\trequire.Empty(t, remainder)\n}\n\nfunc (tp *TestSuite) TestInvalidPathRemainder(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, api.Dag())\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`{\"foo\": {\"bar\": \"baz\"}}`), math.MaxUint64, -1)\n\trequire.NoError(t, err)\n\n\terr = api.Dag().Add(ctx, nd)\n\trequire.NoError(t, err)\n\n\tp, err := path.Join(newIPLDPath(t, nd.Cid()), \"/bar/baz\")\n\trequire.NoError(t, err)\n\n\t_, _, err = api.ResolvePath(ctx, p)\n\trequire.NotNil(t, err)\n\trequire.ErrorContains(t, err, `no link named \"bar\"`)\n}\n\nfunc (tp *TestSuite) TestPathRoot(t *testing.T) {\n\tctx := t.Context()\n\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, api.Block())\n\n\tblk, err := api.Block().Put(ctx, strings.NewReader(`foo`), options.Block.Format(\"raw\"))\n\trequire.NoError(t, err)\n\trequire.NotNil(t, api.Dag())\n\n\tnd, err := ipldcbor.FromJSON(strings.NewReader(`{\"foo\": {\"/\": \"`+blk.Path().RootCid().String()+`\"}}`), math.MaxUint64, -1)\n\trequire.NoError(t, err)\n\n\terr = api.Dag().Add(ctx, nd)\n\trequire.NoError(t, err)\n\n\tp, err := path.Join(newIPLDPath(t, nd.Cid()), \"/foo\")\n\trequire.NoError(t, err)\n\n\trp, _, err := api.ResolvePath(ctx, p)\n\trequire.NoError(t, err)\n\trequire.Equal(t, rp.RootCid().String(), blk.Path().RootCid().String())\n}\n\nfunc (tp *TestSuite) TestPathJoin(t *testing.T) {\n\tp1, err := path.NewPath(\"/ipfs/QmYNmQKp6SuaVrpgWRsPTgCQCnpxUYGq76YEKBXuj2N4H6/bar/baz\")\n\trequire.NoError(t, err)\n\n\tp2, err := path.Join(p1, \"foo\")\n\trequire.NoError(t, err)\n\n\trequire.Equal(t, \"/ipfs/QmYNmQKp6SuaVrpgWRsPTgCQCnpxUYGq76YEKBXuj2N4H6/bar/baz/foo\", p2.String())\n}\n" }, { "path": "core/coreiface/tests/pin.go", "content": "package tests\n\nimport (\n\t\"context\"\n\t\"math\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tipldcbor \"github.com/ipfs/go-ipld-cbor\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\topt \"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc (tp *TestSuite) TestPin(t *testing.T) {\n\ttp.hasApi(t, func(api iface.CoreAPI) error {\n\t\tif api.Pin() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestPinAdd\", tp.TestPinAdd)\n\tt.Run(\"TestPinSimple\", tp.TestPinSimple)\n\tt.Run(\"TestPinRecursive\", tp.TestPinRecursive)\n\tt.Run(\"TestPinLsIndirect\", tp.TestPinLsIndirect)\n\tt.Run(\"TestPinLsPrecedence\", tp.TestPinLsPrecedence)\n\tt.Run(\"TestPinIsPinned\", tp.TestPinIsPinned)\n\tt.Run(\"TestPinNames\", tp.TestPinNames)\n}\n\nfunc (tp *TestSuite) TestPinAdd(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp, err := api.Unixfs().Add(ctx, strFile(\"foo\")())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc (tp *TestSuite) TestPinSimple(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp, err := api.Unixfs().Add(ctx, strFile(\"foo\")())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tlist, err := accPins(ctx, api)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(list) != 1 {\n\t\tt.Errorf(\"unexpected pin list len: %d\", len(list))\n\t}\n\n\tif list[0].Path().RootCid().String() != p.RootCid().String() {\n\t\tt.Error(\"paths don't match\")\n\t}\n\n\tif list[0].Type() != \"recursive\" {\n\t\tt.Error(\"unexpected pin type\")\n\t}\n\n\tassertIsPinned(t, ctx, api, p, \"recursive\")\n\n\terr = api.Pin().Rm(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tlist, err = accPins(ctx, api)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(list) != 0 {\n\t\tt.Errorf(\"unexpected pin list len: %d\", len(list))\n\t}\n}\n\nfunc (tp *TestSuite) TestPinRecursive(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp0, err := api.Unixfs().Add(ctx, strFile(\"foo\")())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp1, err := api.Unixfs().Add(ctx, strFile(\"bar\")())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd2, err := ipldcbor.FromJSON(strings.NewReader(`{\"lnk\": {\"/\": \"`+p0.RootCid().String()+`\"}}`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd3, err := ipldcbor.FromJSON(strings.NewReader(`{\"lnk\": {\"/\": \"`+p1.RootCid().String()+`\"}}`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := api.Dag().AddMany(ctx, []ipld.Node{nd2, nd3}); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(nd2.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(nd3.Cid()), opt.Pin.Recursive(false))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tlist, err := accPins(ctx, api)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(list) != 3 {\n\t\tt.Errorf(\"unexpected pin list len: %d\", len(list))\n\t}\n\n\tlist, err = accPins(ctx, api, opt.Pin.Ls.Direct())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(list) != 1 {\n\t\tt.Errorf(\"unexpected pin list len: %d\", len(list))\n\t}\n\n\tif list[0].Path().String() != path.FromCid(nd3.Cid()).String() {\n\t\tt.Errorf(\"unexpected path, %s != %s\", list[0].Path().String(), path.FromCid(nd3.Cid()).String())\n\t}\n\n\tlist, err = accPins(ctx, api, opt.Pin.Ls.Recursive())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(list) != 1 {\n\t\tt.Errorf(\"unexpected pin list len: %d\", len(list))\n\t}\n\n\tif list[0].Path().String() != path.FromCid(nd2.Cid()).String() {\n\t\tt.Errorf(\"unexpected path, %s != %s\", list[0].Path().String(), path.FromCid(nd2.Cid()).String())\n\t}\n\n\tlist, err = accPins(ctx, api, opt.Pin.Ls.Indirect())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(list) != 1 {\n\t\tt.Errorf(\"unexpected pin list len: %d\", len(list))\n\t}\n\n\tif list[0].Path().RootCid().String() != p0.RootCid().String() {\n\t\tt.Errorf(\"unexpected path, %s != %s\", list[0].Path().RootCid().String(), p0.RootCid().String())\n\t}\n\n\tres, err := api.Pin().Verify(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tn := 0\n\tfor r := range res {\n\t\tif err := r.Err(); err != nil {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif !r.Ok() {\n\t\t\tt.Error(\"expected pin to be ok\")\n\t\t}\n\t\tn++\n\t}\n\n\tif n != 1 {\n\t\tt.Errorf(\"unexpected verify result count: %d\", n)\n\t}\n\n\t// TODO: figure out a way to test verify without touching IpfsNode\n\t/*\n\t\terr = api.Block().Rm(ctx, p0, opt.Block.Force(true))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tres, err = api.Pin().Verify(ctx)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tn = 0\n\t\tfor r := range res {\n\t\t\tif r.Ok() {\n\t\t\t\tt.Error(\"expected pin to not be ok\")\n\t\t\t}\n\n\t\t\tif len(r.BadNodes()) != 1 {\n\t\t\t\tt.Fatalf(\"unexpected badNodes len\")\n\t\t\t}\n\n\t\t\tif r.BadNodes()[0].Path().Cid().String() != p0.Cid().String() {\n\t\t\t\tt.Error(\"unexpected badNode path\")\n\t\t\t}\n\n\t\t\tif r.BadNodes()[0].Err().Error() != \"merkledag: not found\" {\n\t\t\t\tt.Errorf(\"unexpected badNode error: %s\", r.BadNodes()[0].Err().Error())\n\t\t\t}\n\t\t\tn++\n\t\t}\n\n\t\tif n != 1 {\n\t\t\tt.Errorf(\"unexpected verify result count: %d\", n)\n\t\t}\n\t*/\n}\n\n// TestPinLsIndirect verifies that indirect nodes are listed by pin ls even if a parent node is directly pinned\nfunc (tp *TestSuite) TestPinLsIndirect(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tleaf, parent, grandparent := getThreeChainedNodes(t, ctx, api, \"foo\")\n\n\terr = api.Pin().Add(ctx, path.FromCid(grandparent.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(parent.Cid()), opt.Pin.Recursive(false))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinTypes(t, ctx, api, []cidContainer{grandparent}, []cidContainer{parent}, []cidContainer{leaf})\n}\n\n// TestPinLsPrecedence verifies the precedence of pins (recursive > direct > indirect)\nfunc (tp *TestSuite) TestPinLsPrecedence(t *testing.T) {\n\t// Testing precedence of recursive, direct and indirect pins\n\t// Results should be recursive > indirect, direct > indirect, and recursive > direct\n\n\tt.Run(\"TestPinLsPredenceRecursiveIndirect\", tp.TestPinLsPredenceRecursiveIndirect)\n\tt.Run(\"TestPinLsPrecedenceDirectIndirect\", tp.TestPinLsPrecedenceDirectIndirect)\n\tt.Run(\"TestPinLsPrecedenceRecursiveDirect\", tp.TestPinLsPrecedenceRecursiveDirect)\n}\n\nfunc (tp *TestSuite) TestPinLsPredenceRecursiveIndirect(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Test recursive > indirect\n\tleaf, parent, grandparent := getThreeChainedNodes(t, ctx, api, \"recursive > indirect\")\n\n\terr = api.Pin().Add(ctx, path.FromCid(grandparent.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(parent.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinTypes(t, ctx, api, []cidContainer{grandparent, parent}, []cidContainer{}, []cidContainer{leaf})\n}\n\nfunc (tp *TestSuite) TestPinLsPrecedenceDirectIndirect(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Test direct > indirect\n\tleaf, parent, grandparent := getThreeChainedNodes(t, ctx, api, \"direct > indirect\")\n\n\terr = api.Pin().Add(ctx, path.FromCid(grandparent.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(parent.Cid()), opt.Pin.Recursive(false))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinTypes(t, ctx, api, []cidContainer{grandparent}, []cidContainer{parent}, []cidContainer{leaf})\n}\n\nfunc (tp *TestSuite) TestPinLsPrecedenceRecursiveDirect(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Test recursive > direct\n\tleaf, parent, grandparent := getThreeChainedNodes(t, ctx, api, \"recursive + direct = error\")\n\n\terr = api.Pin().Add(ctx, path.FromCid(parent.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(parent.Cid()), opt.Pin.Recursive(false))\n\tif err == nil {\n\t\tt.Fatal(\"expected error directly pinning a recursively pinned node\")\n\t}\n\n\tassertPinTypes(t, ctx, api, []cidContainer{parent}, []cidContainer{}, []cidContainer{leaf})\n\n\terr = api.Pin().Add(ctx, path.FromCid(grandparent.Cid()), opt.Pin.Recursive(false))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Pin().Add(ctx, path.FromCid(grandparent.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinTypes(t, ctx, api, []cidContainer{grandparent, parent}, []cidContainer{}, []cidContainer{leaf})\n}\n\nfunc (tp *TestSuite) TestPinIsPinned(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tleaf, parent, grandparent := getThreeChainedNodes(t, ctx, api, \"foofoo\")\n\n\tassertNotPinned(t, ctx, api, newIPLDPath(t, grandparent.Cid()))\n\tassertNotPinned(t, ctx, api, newIPLDPath(t, parent.Cid()))\n\tassertNotPinned(t, ctx, api, newIPLDPath(t, leaf.Cid()))\n\n\terr = api.Pin().Add(ctx, newIPLDPath(t, parent.Cid()), opt.Pin.Recursive(true))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertNotPinned(t, ctx, api, newIPLDPath(t, grandparent.Cid()))\n\tassertIsPinned(t, ctx, api, newIPLDPath(t, parent.Cid()), \"recursive\")\n\tassertIsPinned(t, ctx, api, newIPLDPath(t, leaf.Cid()), \"indirect\")\n\n\terr = api.Pin().Add(ctx, newIPLDPath(t, grandparent.Cid()), opt.Pin.Recursive(false))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertIsPinned(t, ctx, api, newIPLDPath(t, grandparent.Cid()), \"direct\")\n\tassertIsPinned(t, ctx, api, newIPLDPath(t, parent.Cid()), \"recursive\")\n\tassertIsPinned(t, ctx, api, newIPLDPath(t, leaf.Cid()), \"indirect\")\n}\n\ntype cidContainer interface {\n\tCid() cid.Cid\n}\n\ntype immutablePathCidContainer struct {\n\tpath.ImmutablePath\n}\n\nfunc (i immutablePathCidContainer) Cid() cid.Cid {\n\treturn i.RootCid()\n}\n\nfunc getThreeChainedNodes(t *testing.T, ctx context.Context, api iface.CoreAPI, leafData string) (cidContainer, cidContainer, cidContainer) {\n\tleaf, err := api.Unixfs().Add(ctx, strFile(leafData)())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tparent, err := ipldcbor.FromJSON(strings.NewReader(`{\"lnk\": {\"/\": \"`+leaf.RootCid().String()+`\"}}`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tgrandparent, err := ipldcbor.FromJSON(strings.NewReader(`{\"lnk\": {\"/\": \"`+parent.Cid().String()+`\"}}`), math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := api.Dag().AddMany(ctx, []ipld.Node{parent, grandparent}); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treturn immutablePathCidContainer{leaf}, parent, grandparent\n}\n\nfunc assertPinTypes(t *testing.T, ctx context.Context, api iface.CoreAPI, recursive, direct, indirect []cidContainer) {\n\tassertPinLsAllConsistency(t, ctx, api)\n\n\tlist, err := accPins(ctx, api, opt.Pin.Ls.Recursive())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinCids(t, list, recursive...)\n\n\tlist, err = accPins(ctx, api, opt.Pin.Ls.Direct())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinCids(t, list, direct...)\n\n\tlist, err = accPins(ctx, api, opt.Pin.Ls.Indirect())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tassertPinCids(t, list, indirect...)\n}\n\n// assertPinCids verifies that the pins match the expected cids\nfunc assertPinCids(t *testing.T, pins []iface.Pin, cids ...cidContainer) {\n\tt.Helper()\n\n\tif expected, actual := len(cids), len(pins); expected != actual {\n\t\tt.Fatalf(\"expected pin list to have len %d, was %d\", expected, actual)\n\t}\n\n\tcSet := cid.NewSet()\n\tfor _, c := range cids {\n\t\tcSet.Add(c.Cid())\n\t}\n\n\tvalid := true\n\tfor _, p := range pins {\n\t\tc := p.Path().RootCid()\n\t\tif cSet.Has(c) {\n\t\t\tcSet.Remove(c)\n\t\t} else {\n\t\t\tvalid = false\n\t\t\tbreak\n\t\t}\n\t}\n\n\tvalid = valid && cSet.Len() == 0\n\n\tif !valid {\n\t\tpinStrs := make([]string, len(pins))\n\t\tfor i, p := range pins {\n\t\t\tpinStrs[i] = p.Path().RootCid().String()\n\t\t}\n\t\tpathStrs := make([]string, len(cids))\n\t\tfor i, c := range cids {\n\t\t\tpathStrs[i] = c.Cid().String()\n\t\t}\n\t\tt.Fatalf(\"expected: %s \\nactual: %s\", strings.Join(pathStrs, \", \"), strings.Join(pinStrs, \", \"))\n\t}\n}\n\n// assertPinLsAllConsistency verifies that listing all pins gives the same result as listing the pin types individually\nfunc assertPinLsAllConsistency(t *testing.T, ctx context.Context, api iface.CoreAPI) {\n\tt.Helper()\n\tallPins, err := accPins(ctx, api)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\ttype pinTypeProps struct {\n\t\t*cid.Set\n\t\topt.PinLsOption\n\t}\n\n\tall, recursive, direct, indirect := cid.NewSet(), cid.NewSet(), cid.NewSet(), cid.NewSet()\n\ttypeMap := map[string]*pinTypeProps{\n\t\t\"recursive\": {recursive, opt.Pin.Ls.Recursive()},\n\t\t\"direct\": {direct, opt.Pin.Ls.Direct()},\n\t\t\"indirect\": {indirect, opt.Pin.Ls.Indirect()},\n\t}\n\n\tfor _, p := range allPins {\n\t\tif !all.Visit(p.Path().RootCid()) {\n\t\t\tt.Fatalf(\"pin ls returned the same cid multiple times\")\n\t\t}\n\n\t\ttypeStr := p.Type()\n\t\tif typeSet, ok := typeMap[p.Type()]; ok {\n\t\t\ttypeSet.Add(p.Path().RootCid())\n\t\t} else {\n\t\t\tt.Fatalf(\"unknown pin type: %s\", typeStr)\n\t\t}\n\t}\n\n\tfor typeStr, pinProps := range typeMap {\n\t\tpins, err := accPins(ctx, api, pinProps.PinLsOption)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif expected, actual := len(pins), pinProps.Set.Len(); expected != actual {\n\t\t\tt.Fatalf(\"pin ls all has %d pins of type %s, but pin ls for the type has %d\", expected, typeStr, actual)\n\t\t}\n\n\t\tfor _, p := range pins {\n\t\t\tif pinType := p.Type(); pinType != typeStr {\n\t\t\t\tt.Fatalf(\"returned wrong pin type: expected %s, got %s\", typeStr, pinType)\n\t\t\t}\n\n\t\t\tif c := p.Path().RootCid(); !pinProps.Has(c) {\n\t\t\t\tt.Fatalf(\"%s expected to be in pin ls all as type %s\", c.String(), typeStr)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc assertIsPinned(t *testing.T, ctx context.Context, api iface.CoreAPI, p path.Path, typeStr string) {\n\tt.Helper()\n\twithType, err := opt.Pin.IsPinned.Type(typeStr)\n\tif err != nil {\n\t\tt.Fatal(\"unhandled pin type\")\n\t}\n\n\twhyPinned, pinned, err := api.Pin().IsPinned(ctx, p, withType)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !pinned {\n\t\tt.Fatalf(\"%s expected to be pinned with type %s\", p, typeStr)\n\t}\n\n\tswitch typeStr {\n\tcase \"recursive\", \"direct\":\n\t\tif typeStr != whyPinned {\n\t\t\tt.Fatalf(\"reason for pinning expected to be %s for %s, got %s\", typeStr, p, whyPinned)\n\t\t}\n\tcase \"indirect\":\n\t\tif whyPinned == \"\" {\n\t\t\tt.Fatalf(\"expected to have a pin reason for %s\", p)\n\t\t}\n\t}\n}\n\nfunc (tp *TestSuite) TestPinNames(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\trequire.NoError(t, err)\n\n\t// Create test content\n\tp1, err := api.Unixfs().Add(ctx, strFile(\"content1\")())\n\trequire.NoError(t, err)\n\n\tp2, err := api.Unixfs().Add(ctx, strFile(\"content2\")())\n\trequire.NoError(t, err)\n\n\tp3, err := api.Unixfs().Add(ctx, strFile(\"content3\")())\n\trequire.NoError(t, err)\n\n\tp4, err := api.Unixfs().Add(ctx, strFile(\"content4\")())\n\trequire.NoError(t, err)\n\n\t// Test 1: Pin with name\n\terr = api.Pin().Add(ctx, p1, opt.Pin.Name(\"test-pin-1\"))\n\trequire.NoError(t, err, \"failed to add pin with name\")\n\n\t// Test 2: Pin without name\n\terr = api.Pin().Add(ctx, p2)\n\trequire.NoError(t, err, \"failed to add pin without name\")\n\n\t// Test 3: List pins with detailed option to get names\n\tpins := make(chan iface.Pin)\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, pins, opt.Pin.Ls.Detailed(true))\n\t}()\n\n\tpinMap := make(map[string]string)\n\tfor pin := range pins {\n\t\tpinMap[pin.Path().String()] = pin.Name()\n\t}\n\trequire.NoError(t, err, \"failed to list pins with names\")\n\n\t// Verify pin names\n\tname1, ok := pinMap[p1.String()]\n\trequire.True(t, ok, \"pin for %s not found\", p1)\n\trequire.Equal(t, \"test-pin-1\", name1, \"unexpected pin name for %s\", p1)\n\n\tname2, ok := pinMap[p2.String()]\n\trequire.True(t, ok, \"pin for %s not found\", p2)\n\trequire.Empty(t, name2, \"expected empty pin name for %s, got '%s'\", p2, name2)\n\n\t// Test 4: Pin update preserves name\n\terr = api.Pin().Add(ctx, p3, opt.Pin.Name(\"updatable-pin\"))\n\trequire.NoError(t, err, \"failed to add pin with name for update test\")\n\n\terr = api.Pin().Update(ctx, p3, p4)\n\trequire.NoError(t, err, \"failed to update pin\")\n\n\t// Verify name was preserved after update\n\tpins2 := make(chan iface.Pin)\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, pins2, opt.Pin.Ls.Detailed(true))\n\t}()\n\n\tupdatedPinMap := make(map[string]string)\n\tfor pin := range pins2 {\n\t\tupdatedPinMap[pin.Path().String()] = pin.Name()\n\t}\n\trequire.NoError(t, err, \"failed to list pins after update\")\n\n\t// Old pin should not exist\n\t_, oldExists := updatedPinMap[p3.String()]\n\trequire.False(t, oldExists, \"old pin %s should not exist after update\", p3)\n\n\t// New pin should have the preserved name\n\tname4, ok := updatedPinMap[p4.String()]\n\trequire.True(t, ok, \"updated pin for %s not found\", p4)\n\trequire.Equal(t, \"updatable-pin\", name4, \"pin name not preserved after update from %s to %s\", p3, p4)\n\n\t// Test 5: Re-pinning with different name updates the name\n\terr = api.Pin().Add(ctx, p1, opt.Pin.Name(\"new-name-for-p1\"))\n\trequire.NoError(t, err, \"failed to re-pin with new name\")\n\n\t// Verify name was updated\n\tpins3 := make(chan iface.Pin)\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, pins3, opt.Pin.Ls.Detailed(true))\n\t}()\n\n\trepinMap := make(map[string]string)\n\tfor pin := range pins3 {\n\t\trepinMap[pin.Path().String()] = pin.Name()\n\t}\n\trequire.NoError(t, err, \"failed to list pins after re-pin\")\n\n\trePinnedName, ok := repinMap[p1.String()]\n\trequire.True(t, ok, \"re-pinned content %s not found\", p1)\n\trequire.Equal(t, \"new-name-for-p1\", rePinnedName, \"pin name not updated after re-pinning %s\", p1)\n\n\t// Test 6: Direct pin with name\n\tp5, err := api.Unixfs().Add(ctx, strFile(\"direct-content\")())\n\trequire.NoError(t, err)\n\n\terr = api.Pin().Add(ctx, p5, opt.Pin.Recursive(false), opt.Pin.Name(\"direct-pin-name\"))\n\trequire.NoError(t, err, \"failed to add direct pin with name\")\n\n\t// Verify direct pin has name\n\tdirectPins := make(chan iface.Pin)\n\ttypeOpt, err := opt.Pin.Ls.Type(\"direct\")\n\trequire.NoError(t, err, \"failed to create type option\")\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, directPins, typeOpt, opt.Pin.Ls.Detailed(true))\n\t}()\n\n\tdirectPinMap := make(map[string]string)\n\tfor pin := range directPins {\n\t\tdirectPinMap[pin.Path().String()] = pin.Name()\n\t}\n\trequire.NoError(t, err, \"failed to list direct pins\")\n\n\tdirectName, ok := directPinMap[p5.String()]\n\trequire.True(t, ok, \"direct pin %s not found\", p5)\n\trequire.Equal(t, \"direct-pin-name\", directName, \"unexpected name for direct pin %s\", p5)\n\n\t// Test 7: List without detailed option doesn't return names\n\tpinsNoDetails := make(chan iface.Pin)\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, pinsNoDetails)\n\t}()\n\n\tnoDetailsMap := make(map[string]string)\n\tfor pin := range pinsNoDetails {\n\t\tnoDetailsMap[pin.Path().String()] = pin.Name()\n\t}\n\trequire.NoError(t, err, \"failed to list pins without detailed option\")\n\n\t// All names should be empty without detailed option\n\tfor path, name := range noDetailsMap {\n\t\trequire.Empty(t, name, \"expected empty name for %s without detailed option, got '%s'\", path, name)\n\t}\n}\n\nfunc assertNotPinned(t *testing.T, ctx context.Context, api iface.CoreAPI, p path.Path) {\n\tt.Helper()\n\n\t_, pinned, err := api.Pin().IsPinned(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif pinned {\n\t\tt.Fatalf(\"%s expected to not be pinned\", p)\n\t}\n}\n\nfunc accPins(ctx context.Context, api iface.CoreAPI, opts ...opt.PinLsOption) ([]iface.Pin, error) {\n\tvar err error\n\tpins := make(chan iface.Pin)\n\tgo func() {\n\t\terr = api.Pin().Ls(ctx, pins, opts...)\n\t}()\n\n\tvar results []iface.Pin\n\tfor pin := range pins {\n\t\tresults = append(results, pin)\n\t}\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn results, nil\n}\n" }, { "path": "core/coreiface/tests/pubsub.go", "content": "package tests\n\nimport (\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nfunc (tp *TestSuite) TestPubSub(t *testing.T) {\n\ttp.hasApi(t, func(api iface.CoreAPI) error {\n\t\tif api.PubSub() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestBasicPubSub\", tp.TestBasicPubSub)\n}\n\nfunc (tp *TestSuite) TestBasicPubSub(t *testing.T) {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\tapis, err := tp.MakeAPISwarm(t, ctx, 2)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsub, err := apis[0].PubSub().Subscribe(ctx, \"testch\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdone := make(chan struct{})\n\tgo func() {\n\t\tdefer close(done)\n\n\t\tticker := time.NewTicker(100 * time.Millisecond)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\terr := apis[1].PubSub().Publish(ctx, \"testch\", []byte(\"hello world\"))\n\t\t\tswitch err {\n\t\t\tcase nil:\n\t\t\tcase context.Canceled:\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t\tt.Error(err)\n\t\t\t\tcancel()\n\t\t\t\treturn\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\t// Wait for the sender to finish before we return.\n\t// Otherwise, we can get random errors as publish fails.\n\tdefer func() {\n\t\tcancel()\n\t\t<-done\n\t}()\n\n\tm, err := sub.Next(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif string(m.Data()) != \"hello world\" {\n\t\tt.Errorf(\"got invalid data: %s\", string(m.Data()))\n\t}\n\n\tself1, err := apis[1].Key().Self(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif m.From() != self1.ID() {\n\t\tt.Errorf(\"m.From didn't match\")\n\t}\n\n\tpeers, err := apis[1].PubSub().Peers(ctx, options.PubSub.Topic(\"testch\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(peers) != 1 {\n\t\tt.Fatalf(\"got incorrect number of peers: %d\", len(peers))\n\t}\n\n\tself0, err := apis[0].Key().Self(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif peers[0] != self0.ID() {\n\t\tt.Errorf(\"peer didn't match\")\n\t}\n\n\tpeers, err = apis[1].PubSub().Peers(ctx, options.PubSub.Topic(\"nottestch\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(peers) != 0 {\n\t\tt.Fatalf(\"got incorrect number of peers: %d\", len(peers))\n\t}\n\n\ttopics, err := apis[0].PubSub().Ls(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(topics) != 1 {\n\t\tt.Fatalf(\"got incorrect number of topics: %d\", len(peers))\n\t}\n\n\tif topics[0] != \"testch\" {\n\t\tt.Errorf(\"topic didn't match\")\n\t}\n\n\ttopics, err = apis[1].PubSub().Ls(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(topics) != 0 {\n\t\tt.Fatalf(\"got incorrect number of topics: %d\", len(peers))\n\t}\n}\n" }, { "path": "core/coreiface/tests/routing.go", "content": "package tests\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/path\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc (tp *TestSuite) TestRouting(t *testing.T) {\n\ttp.hasApi(t, func(api iface.CoreAPI) error {\n\t\tif api.Routing() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestRoutingGet\", tp.TestRoutingGet)\n\tt.Run(\"TestRoutingPut\", tp.TestRoutingPut)\n\tt.Run(\"TestRoutingPutOffline\", tp.TestRoutingPutOffline)\n\tt.Run(\"TestRoutingFindPeer\", tp.TestRoutingFindPeer)\n\tt.Run(\"TestRoutingFindProviders\", tp.TestRoutingFindProviders)\n\tt.Run(\"TestRoutingProvide\", tp.TestRoutingProvide)\n}\n\nfunc (tp *TestSuite) testRoutingPublishKey(t *testing.T, ctx context.Context, api iface.CoreAPI, opts ...options.NamePublishOption) (path.Path, ipns.Name) {\n\tp, err := addTestObject(ctx, api)\n\trequire.NoError(t, err)\n\n\tname, err := api.Name().Publish(ctx, p, opts...)\n\trequire.NoError(t, err)\n\n\ttime.Sleep(3 * time.Second)\n\treturn p, name\n}\n\nfunc (tp *TestSuite) TestRoutingGet(t *testing.T) {\n\tctx := t.Context()\n\n\tapis, err := tp.MakeAPISwarm(t, ctx, 2)\n\trequire.NoError(t, err)\n\n\t// Node 1: publishes an IPNS name\n\tp, name := tp.testRoutingPublishKey(t, ctx, apis[0])\n\n\t// Node 2: retrieves the best value for the IPNS name.\n\tdata, err := apis[1].Routing().Get(ctx, ipns.NamespacePrefix+name.String())\n\trequire.NoError(t, err)\n\n\trec, err := ipns.UnmarshalRecord(data)\n\trequire.NoError(t, err)\n\n\tval, err := rec.Value()\n\trequire.NoError(t, err)\n\trequire.Equal(t, p.String(), val.String())\n}\n\nfunc (tp *TestSuite) TestRoutingPut(t *testing.T) {\n\tctx := t.Context()\n\tapis, err := tp.MakeAPISwarm(t, ctx, 2)\n\trequire.NoError(t, err)\n\n\t// Create and publish IPNS entry.\n\t_, name := tp.testRoutingPublishKey(t, ctx, apis[0])\n\n\t// Get valid routing value.\n\tdata, err := apis[0].Routing().Get(ctx, ipns.NamespacePrefix+name.String())\n\trequire.NoError(t, err)\n\n\t// Put routing value.\n\terr = apis[1].Routing().Put(ctx, ipns.NamespacePrefix+name.String(), data)\n\trequire.NoError(t, err)\n}\n\nfunc (tp *TestSuite) TestRoutingPutOffline(t *testing.T) {\n\tctx := t.Context()\n\n\t// init a swarm & publish an IPNS entry to get a valid payload\n\tapis, err := tp.MakeAPISwarm(t, ctx, 2)\n\trequire.NoError(t, err)\n\n\t_, name := tp.testRoutingPublishKey(t, ctx, apis[0], options.Name.AllowOffline(true))\n\tdata, err := apis[0].Routing().Get(ctx, ipns.NamespacePrefix+name.String())\n\trequire.NoError(t, err)\n\n\t// init our offline node and try to put the payload\n\tapi, err := tp.makeAPIWithIdentityAndOffline(t, ctx)\n\trequire.NoError(t, err)\n\n\terr = api.Routing().Put(ctx, ipns.NamespacePrefix+name.String(), data)\n\trequire.Error(t, err, \"this operation should fail because we are offline\")\n\n\terr = api.Routing().Put(ctx, ipns.NamespacePrefix+name.String(), data, options.Routing.AllowOffline(true))\n\trequire.NoError(t, err)\n}\n\nfunc (tp *TestSuite) TestRoutingFindPeer(t *testing.T) {\n\tctx := t.Context()\n\tapis, err := tp.MakeAPISwarm(t, ctx, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tself0, err := apis[0].Key().Self(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tladdrs0, err := apis[0].Swarm().LocalAddrs(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(laddrs0) != 1 {\n\t\tt.Fatal(\"unexpected number of local addrs\")\n\t}\n\n\ttime.Sleep(3 * time.Second)\n\n\tpi, err := apis[2].Routing().FindPeer(ctx, self0.ID())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif pi.Addrs[0].String() != laddrs0[0].String() {\n\t\tt.Errorf(\"got unexpected address from FindPeer: %s\", pi.Addrs[0].String())\n\t}\n\n\tself2, err := apis[2].Key().Self(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpi, err = apis[1].Routing().FindPeer(ctx, self2.ID())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tladdrs2, err := apis[2].Swarm().LocalAddrs(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(laddrs2) != 1 {\n\t\tt.Fatal(\"unexpected number of local addrs\")\n\t}\n\n\tif pi.Addrs[0].String() != laddrs2[0].String() {\n\t\tt.Errorf(\"got unexpected address from FindPeer: %s\", pi.Addrs[0].String())\n\t}\n}\n\nfunc (tp *TestSuite) TestRoutingFindProviders(t *testing.T) {\n\tctx := t.Context()\n\tapis, err := tp.MakeAPISwarm(t, ctx, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp, err := addTestObject(ctx, apis[0])\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Pin so that it is provided, given that providing strategy is\n\t// \"roots\" and addTestObject does not pin.\n\terr = apis[0].Pin().Add(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\ttime.Sleep(3 * time.Second)\n\n\tout, err := apis[2].Routing().FindProviders(ctx, p, options.Routing.NumProviders(1))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tprovider := <-out\n\n\tself0, err := apis[0].Key().Self(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif provider.ID.String() != self0.ID().String() {\n\t\tt.Errorf(\"got wrong provider: %s != %s\", provider.ID.String(), self0.ID().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestRoutingProvide(t *testing.T) {\n\tctx := t.Context()\n\tapis, err := tp.MakeAPISwarm(t, ctx, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\toff0, err := apis[0].WithOptions(options.Api.Offline(true))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\ts, err := off0.Block().Put(ctx, &io.LimitedReader{R: rnd, N: 4092})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp := s.Path()\n\n\ttime.Sleep(3 * time.Second)\n\n\tout, err := apis[2].Routing().FindProviders(ctx, p, options.Routing.NumProviders(1))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, ok := <-out\n\n\tif ok {\n\t\tt.Fatal(\"did not expect to find any providers\")\n\t}\n\n\tself0, err := apis[0].Key().Self(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = apis[0].Routing().Provide(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tmaxAttempts := 5\n\tsuccess := false\n\tfor range maxAttempts {\n\t\t// We may need to try again as Provide() doesn't block until the CID is\n\t\t// actually provided.\n\t\tout, err = apis[2].Routing().FindProviders(ctx, p, options.Routing.NumProviders(1))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tprovider := <-out\n\n\t\tif provider.ID.String() == self0.ID().String() {\n\t\t\tsuccess = true\n\t\t\tbreak\n\t\t}\n\t\tif len(provider.ID.String()) > 0 {\n\t\t\tt.Errorf(\"got wrong provider: %s != %s\", provider.ID.String(), self0.ID().String())\n\t\t}\n\t\ttime.Sleep(time.Second)\n\t}\n\tif !success {\n\t\tt.Errorf(\"missing provider after %d attempts\", maxAttempts)\n\t}\n}\n" }, { "path": "core/coreiface/tests/unixfs.go", "content": "package tests\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/hex\"\n\t\"fmt\"\n\t\"io\"\n\t\"math\"\n\t\"math/rand\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/path\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\n\t\"github.com/ipfs/boxo/files\"\n\tmdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/ipld/unixfs/importer/helpers\"\n\t\"github.com/ipfs/go-cid\"\n\tcbor \"github.com/ipfs/go-ipld-cbor\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tmh \"github.com/multiformats/go-multihash\"\n)\n\nfunc (tp *TestSuite) TestUnixfs(t *testing.T) {\n\ttp.hasApi(t, func(api coreiface.CoreAPI) error {\n\t\tif api.Unixfs() == nil {\n\t\t\treturn errAPINotImplemented\n\t\t}\n\t\treturn nil\n\t})\n\n\tt.Run(\"TestAdd\", tp.TestAdd)\n\tt.Run(\"TestAddPinned\", tp.TestAddPinned)\n\tt.Run(\"TestAddHashOnly\", tp.TestAddHashOnly)\n\tt.Run(\"TestGetEmptyFile\", tp.TestGetEmptyFile)\n\tt.Run(\"TestGetDir\", tp.TestGetDir)\n\tt.Run(\"TestGetNonUnixfs\", tp.TestGetNonUnixfs)\n\tt.Run(\"TestLs\", tp.TestLs)\n\tt.Run(\"TestEntriesExpired\", tp.TestEntriesExpired)\n\tt.Run(\"TestLsEmptyDir\", tp.TestLsEmptyDir)\n\tt.Run(\"TestLsNonUnixfs\", tp.TestLsNonUnixfs)\n\tt.Run(\"TestAddCloses\", tp.TestAddCloses)\n\tt.Run(\"TestGetSeek\", tp.TestGetSeek)\n\tt.Run(\"TestGetReadAt\", tp.TestGetReadAt)\n}\n\n// `echo -n 'hello, world!' | ipfs add`\nvar (\n\thello = \"/ipfs/QmQy2Dw4Wk7rdJKjThjYXzfFJNaRKRHhHP5gHHXroJMYxk\"\n\thelloStr = \"hello, world!\"\n)\n\n// `echo -n | ipfs add`\nvar emptyFile = \"/ipfs/QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH\"\n\nfunc strFile(data string) func() files.Node {\n\treturn func() files.Node {\n\t\treturn files.NewBytesFile([]byte(data))\n\t}\n}\n\nfunc twoLevelDir() func() files.Node {\n\treturn func() files.Node {\n\t\treturn files.NewMapDirectory(map[string]files.Node{\n\t\t\t\"abc\": files.NewMapDirectory(map[string]files.Node{\n\t\t\t\t\"def\": files.NewBytesFile([]byte(\"world\")),\n\t\t\t}),\n\n\t\t\t\"bar\": files.NewBytesFile([]byte(\"hello2\")),\n\t\t\t\"foo\": files.NewBytesFile([]byte(\"hello1\")),\n\t\t})\n\t}\n}\n\nfunc flatDir() files.Node {\n\treturn files.NewMapDirectory(map[string]files.Node{\n\t\t\"bar\": files.NewBytesFile([]byte(\"hello2\")),\n\t\t\"foo\": files.NewBytesFile([]byte(\"hello1\")),\n\t})\n}\n\nfunc wrapped(names ...string) func(f files.Node) files.Node {\n\treturn func(f files.Node) files.Node {\n\t\tfor i := range names {\n\t\t\tf = files.NewMapDirectory(map[string]files.Node{\n\t\t\t\tnames[len(names)-i-1]: f,\n\t\t\t})\n\t\t}\n\t\treturn f\n\t}\n}\n\nfunc (tp *TestSuite) TestAdd(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp := func(h string) path.ImmutablePath {\n\t\tc, err := cid.Parse(h)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\treturn path.FromCid(c)\n\t}\n\n\trf, err := os.CreateTemp(os.TempDir(), \"unixfs-add-real\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\trfp := rf.Name()\n\n\tif _, err := rf.Write([]byte(helloStr)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tstat, err := rf.Stat()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := rf.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer os.Remove(rfp)\n\n\trealFile := func() files.Node {\n\t\tn, err := files.NewReaderPathFile(rfp, io.NopCloser(strings.NewReader(helloStr)), stat)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\treturn n\n\t}\n\n\tcases := []struct {\n\t\tname string\n\t\tdata func() files.Node\n\t\texpect func(files.Node) files.Node\n\n\t\tapiOpts []options.ApiOption\n\n\t\tpath string\n\t\terr string\n\n\t\twrap string\n\n\t\tevents []coreiface.AddEvent\n\n\t\topts []options.UnixfsAddOption\n\t}{\n\t\t// Simple cases\n\t\t{\n\t\t\tname: \"simpleAdd\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: hello,\n\t\t\topts: []options.UnixfsAddOption{},\n\t\t},\n\t\t{\n\t\t\tname: \"addEmpty\",\n\t\t\tdata: strFile(\"\"),\n\t\t\tpath: emptyFile,\n\t\t},\n\t\t// CIDv1 version / rawLeaves\n\t\t{\n\t\t\tname: \"addCidV1\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.CidVersion(1)},\n\t\t},\n\t\t{\n\t\t\tname: \"addCidV1NoLeaves\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafybeibhbcn7k7o2m6xsqkrlfiokod3nxwe47viteynhruh6uqx7hvkjfu\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.CidVersion(1), options.Unixfs.RawLeaves(false)},\n\t\t},\n\t\t// Non sha256 hash vs CID\n\t\t{\n\t\t\tname: \"addCidSha3\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafkrmichjflejeh6aren53o7pig7zk3m3vxqcoc2i5dv326k3x6obh7jry\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Hash(mh.SHA3_256)},\n\t\t},\n\t\t{\n\t\t\tname: \"addCidSha3Cid0\",\n\t\t\tdata: strFile(helloStr),\n\t\t\terr: \"CIDv0 only supports sha2-256\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.CidVersion(0), options.Unixfs.Hash(mh.SHA3_256)},\n\t\t},\n\t\t// Inline\n\t\t{\n\t\t\tname: \"addInline\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafyaafikcmeaeeqnnbswy3dpfqqho33snrsccgan\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Inline(true)},\n\t\t},\n\t\t{\n\t\t\tname: \"addInlineLimit\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafyaafikcmeaeeqnnbswy3dpfqqho33snrsccgan\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.InlineLimit(32), options.Unixfs.Inline(true)},\n\t\t},\n\t\t{\n\t\t\tname: \"addInlineZero\",\n\t\t\tdata: strFile(\"\"),\n\t\t\tpath: \"/ipfs/bafkqaaa\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.InlineLimit(0), options.Unixfs.Inline(true), options.Unixfs.RawLeaves(true)},\n\t\t},\n\t\t{ // TODO: after coreapi add is used in `ipfs add`, consider making this default for inline\n\t\t\tname: \"addInlineRaw\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafkqadlimvwgy3zmeb3w64tmmqqq\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.InlineLimit(32), options.Unixfs.Inline(true), options.Unixfs.RawLeaves(true)},\n\t\t},\n\t\t// Chunker / Layout\n\t\t{\n\t\t\tname: \"addChunks\",\n\t\t\tdata: strFile(strings.Repeat(\"aoeuidhtns\", 200)),\n\t\t\tpath: \"/ipfs/QmRo11d4QJrST47aaiGVJYwPhoNA4ihRpJ5WaxBWjWDwbX\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Chunker(\"size-4\")},\n\t\t},\n\t\t{\n\t\t\tname: \"addChunksTrickle\",\n\t\t\tdata: strFile(strings.Repeat(\"aoeuidhtns\", 200)),\n\t\t\tpath: \"/ipfs/QmNNhDGttafX3M1wKWixGre6PrLFGjnoPEDXjBYpTv93HP\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Chunker(\"size-4\"), options.Unixfs.Layout(options.TrickleLayout)},\n\t\t},\n\t\t// Local\n\t\t{\n\t\t\tname: \"addLocal\", // better cases in sharness\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: hello,\n\t\t\tapiOpts: []options.ApiOption{options.Api.Offline(true)},\n\t\t},\n\t\t{\n\t\t\tname: \"hashOnly\", // test (non)fetchability\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: hello,\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.HashOnly(true)},\n\t\t},\n\t\t// multi file\n\t\t{\n\t\t\tname: \"simpleDirNoWrap\",\n\t\t\tdata: flatDir,\n\t\t\tpath: \"/ipfs/QmRKGpFfR32FVXdvJiHfo4WJ5TDYBsM1P9raAp1p6APWSp\",\n\t\t},\n\t\t{\n\t\t\tname: \"simpleDir\",\n\t\t\tdata: flatDir,\n\t\t\twrap: \"t\",\n\t\t\texpect: wrapped(\"t\"),\n\t\t\tpath: \"/ipfs/Qmc3nGXm1HtUVCmnXLQHvWcNwfdZGpfg2SRm1CxLf7Q2Rm\",\n\t\t},\n\t\t{\n\t\t\tname: \"twoLevelDir\",\n\t\t\tdata: twoLevelDir(),\n\t\t\twrap: \"t\",\n\t\t\texpect: wrapped(\"t\"),\n\t\t\tpath: \"/ipfs/QmPwsL3T5sWhDmmAWZHAzyjKtMVDS9a11aHNRqb3xoVnmg\",\n\t\t},\n\t\t// wrapped\n\t\t{\n\t\t\tname: \"addWrapped\",\n\t\t\tpath: \"/ipfs/QmVE9rNpj5doj7XHzp5zMUxD7BJgXEqx4pe3xZ3JBReWHE\",\n\t\t\tdata: func() files.Node {\n\t\t\t\treturn files.NewBytesFile([]byte(helloStr))\n\t\t\t},\n\t\t\twrap: \"foo\",\n\t\t\texpect: wrapped(\"foo\"),\n\t\t},\n\t\t// hidden\n\t\t{\n\t\t\tname: \"hiddenFilesAdded\",\n\t\t\tdata: func() files.Node {\n\t\t\t\treturn files.NewMapDirectory(map[string]files.Node{\n\t\t\t\t\t\".bar\": files.NewBytesFile([]byte(\"hello2\")),\n\t\t\t\t\t\"bar\": files.NewBytesFile([]byte(\"hello2\")),\n\t\t\t\t\t\"foo\": files.NewBytesFile([]byte(\"hello1\")),\n\t\t\t\t})\n\t\t\t},\n\t\t\twrap: \"t\",\n\t\t\texpect: wrapped(\"t\"),\n\t\t\tpath: \"/ipfs/QmPXLSBX382vJDLrGakcbrZDkU3grfkjMox7EgSC9KFbtQ\",\n\t\t},\n\t\t// NoCopy\n\t\t{\n\t\t\tname: \"simpleNoCopy\",\n\t\t\tdata: realFile,\n\t\t\tpath: \"/ipfs/bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Nocopy(true)},\n\t\t},\n\t\t{\n\t\t\tname: \"noCopyNoRaw\",\n\t\t\tdata: realFile,\n\t\t\tpath: \"/ipfs/bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Nocopy(true), options.Unixfs.RawLeaves(false)},\n\t\t\terr: \"nocopy option requires '--raw-leaves' to be enabled as well\",\n\t\t},\n\t\t{\n\t\t\tname: \"noCopyNoPath\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Nocopy(true)},\n\t\t\terr: helpers.ErrMissingFsRef.Error(),\n\t\t},\n\t\t// Events / Progress\n\t\t{\n\t\t\tname: \"simpleAddEvent\",\n\t\t\tdata: strFile(helloStr),\n\t\t\tpath: \"/ipfs/bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\",\n\t\t\tevents: []coreiface.AddEvent{\n\t\t\t\t{Name: \"bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\", Path: p(\"bafkreidi4zlleupgp2bvrpxyja5lbvi4mym7hz5bvhyoowby2qp7g2hxfa\"), Size: strconv.Itoa(len(helloStr))},\n\t\t\t},\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.RawLeaves(true)},\n\t\t},\n\t\t{\n\t\t\tname: \"silentAddEvent\",\n\t\t\tdata: twoLevelDir(),\n\t\t\tpath: \"/ipfs/QmVG2ZYCkV1S4TK8URA3a4RupBF17A8yAr4FqsRDXVJASr\",\n\t\t\tevents: []coreiface.AddEvent{\n\t\t\t\t{Name: \"abc\", Path: p(\"QmU7nuGs2djqK99UNsNgEPGh6GV4662p6WtsgccBNGTDxt\"), Size: \"62\"},\n\t\t\t\t{Name: \"\", Path: p(\"QmVG2ZYCkV1S4TK8URA3a4RupBF17A8yAr4FqsRDXVJASr\"), Size: \"229\"},\n\t\t\t},\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Silent(true)},\n\t\t},\n\t\t{\n\t\t\tname: \"dirAddEvents\",\n\t\t\tdata: twoLevelDir(),\n\t\t\tpath: \"/ipfs/QmVG2ZYCkV1S4TK8URA3a4RupBF17A8yAr4FqsRDXVJASr\",\n\t\t\tevents: []coreiface.AddEvent{\n\t\t\t\t{Name: \"abc/def\", Path: p(\"QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk\"), Size: \"13\"},\n\t\t\t\t{Name: \"bar\", Path: p(\"QmS21GuXiRMvJKHos4ZkEmQDmRBqRaF5tQS2CQCu2ne9sY\"), Size: \"14\"},\n\t\t\t\t{Name: \"foo\", Path: p(\"QmfAjGiVpTN56TXi6SBQtstit5BEw3sijKj1Qkxn6EXKzJ\"), Size: \"14\"},\n\t\t\t\t{Name: \"abc\", Path: p(\"QmU7nuGs2djqK99UNsNgEPGh6GV4662p6WtsgccBNGTDxt\"), Size: \"62\"},\n\t\t\t\t{Name: \"\", Path: p(\"QmVG2ZYCkV1S4TK8URA3a4RupBF17A8yAr4FqsRDXVJASr\"), Size: \"229\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"progress1M\",\n\t\t\tdata: func() files.Node {\n\t\t\t\treturn files.NewReaderFile(bytes.NewReader(bytes.Repeat([]byte{0}, 1000000)))\n\t\t\t},\n\t\t\tpath: \"/ipfs/QmXXNNbwe4zzpdMg62ZXvnX1oU7MwSrQ3vAEtuwFKCm1oD\",\n\t\t\tevents: []coreiface.AddEvent{\n\t\t\t\t{Name: \"\", Bytes: 262144},\n\t\t\t\t{Name: \"\", Bytes: 524288},\n\t\t\t\t{Name: \"\", Bytes: 786432},\n\t\t\t\t{Name: \"\", Bytes: 1000000},\n\t\t\t\t{Name: \"QmXXNNbwe4zzpdMg62ZXvnX1oU7MwSrQ3vAEtuwFKCm1oD\", Path: p(\"QmXXNNbwe4zzpdMg62ZXvnX1oU7MwSrQ3vAEtuwFKCm1oD\"), Size: \"1000256\"},\n\t\t\t},\n\t\t\twrap: \"\",\n\t\t\topts: []options.UnixfsAddOption{options.Unixfs.Progress(true)},\n\t\t},\n\t}\n\n\tfor _, testCase := range cases {\n\t\tt.Run(testCase.name, func(t *testing.T) {\n\t\t\tctx, cancel := context.WithCancel(ctx)\n\t\t\tdefer cancel()\n\n\t\t\t// recursive logic\n\n\t\t\tdata := testCase.data()\n\t\t\tif testCase.wrap != \"\" {\n\t\t\t\tdata = files.NewMapDirectory(map[string]files.Node{\n\t\t\t\t\ttestCase.wrap: data,\n\t\t\t\t})\n\t\t\t}\n\n\t\t\t// handle events if relevant to test case\n\n\t\t\topts := testCase.opts\n\t\t\teventOut := make(chan any)\n\t\t\tvar evtWg sync.WaitGroup\n\t\t\tif len(testCase.events) > 0 {\n\t\t\t\topts = append(opts, options.Unixfs.Events(eventOut))\n\n\t\t\t\tevtWg.Go(func() {\n\t\t\t\t\texpected := testCase.events\n\n\t\t\t\t\tfor evt := range eventOut {\n\t\t\t\t\t\tevent, ok := evt.(*coreiface.AddEvent)\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\tt.Error(\"unexpected event type\")\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif len(expected) < 1 {\n\t\t\t\t\t\t\tt.Error(\"got more events than expected\")\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif expected[0].Size != event.Size {\n\t\t\t\t\t\t\tt.Errorf(\"Event.Size didn't match, %s != %s\", expected[0].Size, event.Size)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif expected[0].Name != event.Name {\n\t\t\t\t\t\t\tt.Errorf(\"Event.Name didn't match, %s != %s\", expected[0].Name, event.Name)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif (expected[0].Path != path.ImmutablePath{} && event.Path != path.ImmutablePath{}) {\n\t\t\t\t\t\t\tif expected[0].Path.RootCid().String() != event.Path.RootCid().String() {\n\t\t\t\t\t\t\t\tt.Errorf(\"Event.Hash didn't match, %s != %s\", expected[0].Path, event.Path)\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t} else if event.Path != expected[0].Path {\n\t\t\t\t\t\t\tt.Errorf(\"Event.Hash didn't match, %s != %s\", expected[0].Path, event.Path)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif expected[0].Bytes != event.Bytes {\n\t\t\t\t\t\t\tt.Errorf(\"Event.Bytes didn't match, %d != %d\", expected[0].Bytes, event.Bytes)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\texpected = expected[1:]\n\t\t\t\t\t}\n\n\t\t\t\t\tif len(expected) > 0 {\n\t\t\t\t\t\tt.Errorf(\"%d event(s) didn't arrive\", len(expected))\n\t\t\t\t\t}\n\t\t\t\t})\n\t\t\t}\n\n\t\t\ttapi, err := api.WithOptions(testCase.apiOpts...)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\t// Add!\n\n\t\t\tp, err := tapi.Unixfs().Add(ctx, data, opts...)\n\t\t\tclose(eventOut)\n\t\t\tevtWg.Wait()\n\t\t\tif testCase.err != \"\" {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Fatalf(\"expected an error: %s\", testCase.err)\n\t\t\t\t}\n\t\t\t\tif err.Error() != testCase.err {\n\t\t\t\t\tt.Fatalf(\"expected an error: '%s' != '%s'\", err.Error(), testCase.err)\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\tif p.String() != testCase.path {\n\t\t\t\tt.Errorf(\"expected path %s, got: %s\", testCase.path, p)\n\t\t\t}\n\n\t\t\t// compare file structure with Unixfs().Get\n\n\t\t\tvar cmpFile func(origName string, orig files.Node, gotName string, got files.Node)\n\t\t\tcmpFile = func(origName string, orig files.Node, gotName string, got files.Node) {\n\t\t\t\t_, origDir := orig.(files.Directory)\n\t\t\t\t_, gotDir := got.(files.Directory)\n\n\t\t\t\tif origName != gotName {\n\t\t\t\t\tt.Errorf(\"file name mismatch, orig='%s', got='%s'\", origName, gotName)\n\t\t\t\t}\n\n\t\t\t\tif origDir != gotDir {\n\t\t\t\t\tt.Fatalf(\"file type mismatch on %s\", origName)\n\t\t\t\t}\n\n\t\t\t\tif !gotDir {\n\t\t\t\t\tdefer orig.Close()\n\t\t\t\t\tdefer got.Close()\n\n\t\t\t\t\tdo, err := io.ReadAll(orig.(files.File))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tt.Fatal(err)\n\t\t\t\t\t}\n\n\t\t\t\t\tdg, err := io.ReadAll(got.(files.File))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tt.Fatal(err)\n\t\t\t\t\t}\n\n\t\t\t\t\tif !bytes.Equal(do, dg) {\n\t\t\t\t\t\tt.Fatal(\"data not equal\")\n\t\t\t\t\t}\n\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\torigIt := orig.(files.Directory).Entries()\n\t\t\t\tgotIt := got.(files.Directory).Entries()\n\n\t\t\t\tfor {\n\t\t\t\t\tif origIt.Next() {\n\t\t\t\t\t\tif !gotIt.Next() {\n\t\t\t\t\t\t\tt.Fatal(\"gotIt out of entries before origIt\")\n\t\t\t\t\t\t}\n\t\t\t\t\t} else {\n\t\t\t\t\t\tif gotIt.Next() {\n\t\t\t\t\t\t\tt.Fatal(\"origIt out of entries before gotIt\")\n\t\t\t\t\t\t}\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\n\t\t\t\t\tcmpFile(origIt.Name(), origIt.Node(), gotIt.Name(), gotIt.Node())\n\t\t\t\t}\n\t\t\t\tif origIt.Err() != nil {\n\t\t\t\t\tt.Fatal(origIt.Err())\n\t\t\t\t}\n\t\t\t\tif gotIt.Err() != nil {\n\t\t\t\t\tt.Fatal(gotIt.Err())\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tf, err := tapi.Unixfs().Get(ctx, p)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\torig := testCase.data()\n\t\t\tif testCase.expect != nil {\n\t\t\t\torig = testCase.expect(orig)\n\t\t\t}\n\n\t\t\tcmpFile(\"\", orig, \"\", f)\n\t\t})\n\t}\n}\n\nfunc (tp *TestSuite) TestAddPinned(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, err = api.Unixfs().Add(ctx, strFile(helloStr)(), options.Unixfs.Pin(true, \"\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpins, err := accPins(ctx, api)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(pins) != 1 {\n\t\tt.Fatalf(\"expected 1 pin, got %d\", len(pins))\n\t}\n\n\tif pins[0].Path().String() != \"/ipfs/QmQy2Dw4Wk7rdJKjThjYXzfFJNaRKRHhHP5gHHXroJMYxk\" {\n\t\tt.Fatalf(\"got unexpected pin: %s\", pins[0].Path().String())\n\t}\n}\n\nfunc (tp *TestSuite) TestAddHashOnly(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp, err := api.Unixfs().Add(ctx, strFile(helloStr)(), options.Unixfs.HashOnly(true))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif p.String() != hello {\n\t\tt.Errorf(\"unexpected path: %s\", p.String())\n\t}\n\n\t_, err = api.Block().Get(ctx, p)\n\tif err == nil {\n\t\tt.Fatal(\"expected an error\")\n\t}\n\tif !ipld.IsNotFound(err) {\n\t\tt.Errorf(\"unexpected error: %s\", err.Error())\n\t}\n}\n\nfunc (tp *TestSuite) TestGetEmptyFile(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, err = api.Unixfs().Add(ctx, files.NewBytesFile([]byte{}))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\temptyFilePath, err := path.NewPath(emptyFile)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr, err := api.Unixfs().Get(ctx, emptyFilePath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tbuf := make([]byte, 1) // non-zero so that Read() actually tries to read\n\tn, err := io.ReadFull(r.(files.File), buf)\n\tif err != nil && err != io.EOF {\n\t\tt.Error(err)\n\t}\n\tif !bytes.HasPrefix(buf, []byte{0x00}) {\n\t\tt.Fatalf(\"expected empty data, got [%s] [read=%d]\", buf, n)\n\t}\n}\n\nfunc (tp *TestSuite) TestGetDir(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tedir := unixfs.EmptyDirNode()\n\terr = api.Dag().Add(ctx, edir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tp := path.FromCid(edir.Cid())\n\n\tif p.String() != path.FromCid(edir.Cid()).String() {\n\t\tt.Fatalf(\"expected path %s, got: %s\", edir.Cid(), p.String())\n\t}\n\n\tr, err := api.Unixfs().Get(ctx, path.FromCid(edir.Cid()))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, ok := r.(files.Directory); !ok {\n\t\tt.Fatalf(\"expected a directory\")\n\t}\n}\n\nfunc (tp *TestSuite) TestGetNonUnixfs(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd := new(mdag.ProtoNode)\n\terr = api.Dag().Add(ctx, nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t_, err = api.Unixfs().Get(ctx, path.FromCid(nd.Cid()))\n\tif !strings.Contains(err.Error(), \"proto:\") || !strings.Contains(err.Error(), \"required field\") {\n\t\tt.Fatalf(\"expected \\\"proto: required field\\\", got: %q\", err)\n\t}\n}\n\nfunc (tp *TestSuite) TestLs(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr := strings.NewReader(\"content-of-file\")\n\tp, err := api.Unixfs().Add(ctx, files.NewMapDirectory(map[string]files.Node{\n\t\t\"name-of-file\": files.NewReaderFile(r),\n\t\t\"name-of-symlink\": files.NewLinkFile(\"/foo/bar\", nil),\n\t}))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terrCh := make(chan error, 1)\n\tentries := make(chan coreiface.DirEntry)\n\tgo func() {\n\t\terrCh <- api.Unixfs().Ls(ctx, p, entries)\n\t}()\n\n\tentry, ok := <-entries\n\tif !ok {\n\t\tt.Fatal(\"expected another entry\")\n\t}\n\tif entry.Size != 15 {\n\t\tt.Errorf(\"expected size = 15, got %d\", entry.Size)\n\t}\n\tif entry.Name != \"name-of-file\" {\n\t\tt.Errorf(\"expected name = name-of-file, got %s\", entry.Name)\n\t}\n\tif entry.Type != coreiface.TFile {\n\t\tt.Errorf(\"wrong type %s\", entry.Type)\n\t}\n\tif entry.Cid.String() != \"QmX3qQVKxDGz3URVC3861Z3CKtQKGBn6ffXRBBWGMFz9Lr\" {\n\t\tt.Errorf(\"expected cid = QmX3qQVKxDGz3URVC3861Z3CKtQKGBn6ffXRBBWGMFz9Lr, got %s\", entry.Cid)\n\t}\n\tentry, ok = <-entries\n\tif !ok {\n\t\tt.Fatal(\"expected another entry\")\n\t}\n\tif entry.Type != coreiface.TSymlink {\n\t\tt.Errorf(\"wrong type %s\", entry.Type)\n\t}\n\tif entry.Name != \"name-of-symlink\" {\n\t\tt.Errorf(\"expected name = name-of-symlink, got %s\", entry.Name)\n\t}\n\tif entry.Target != \"/foo/bar\" {\n\t\tt.Errorf(\"expected symlink target to be /foo/bar, got %s\", entry.Target)\n\t}\n\n\t_, ok = <-entries\n\tif ok {\n\t\tt.Errorf(\"didn't expect a another link\")\n\t}\n\tif err = <-errCh; err != nil {\n\t\tt.Error(err)\n\t}\n}\n\nfunc (tp *TestSuite) TestEntriesExpired(t *testing.T) {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr := strings.NewReader(\"content-of-file\")\n\tp, err := api.Unixfs().Add(ctx, files.NewMapDirectory(map[string]files.Node{\n\t\t\"name-of-file\": files.NewReaderFile(r),\n\t}))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tctx, cancel = context.WithCancel(ctx)\n\n\tnd, err := api.Unixfs().Get(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tcancel()\n\n\tit := files.ToDir(nd).Entries()\n\tif it == nil {\n\t\tt.Fatal(\"it was nil\")\n\t}\n\n\tif it.Next() {\n\t\tt.Fatal(\"Next succeeded\")\n\t}\n\n\tif it.Err() != context.Canceled {\n\t\tt.Fatalf(\"unexpected error %s\", it.Err())\n\t}\n\n\tif it.Next() {\n\t\tt.Fatal(\"Next succeeded\")\n\t}\n}\n\nfunc (tp *TestSuite) TestLsEmptyDir(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tp, err := api.Unixfs().Add(ctx, files.NewSliceDirectory([]files.DirEntry{}))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terrCh := make(chan error, 1)\n\tlinks := make(chan coreiface.DirEntry)\n\tgo func() {\n\t\terrCh <- api.Unixfs().Ls(ctx, p, links)\n\t}()\n\n\tvar count int\n\tfor range links {\n\t\tcount++\n\t}\n\tif err = <-errCh; err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif count != 0 {\n\t\tt.Fatalf(\"expected 0 links, got %d\", count)\n\t}\n}\n\n// TODO(lgierth) this should test properly, with len(links) > 0\nfunc (tp *TestSuite) TestLsNonUnixfs(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnd, err := cbor.WrapObject(map[string]any{\"foo\": \"bar\"}, math.MaxUint64, -1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = api.Dag().Add(ctx, nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terrCh := make(chan error, 1)\n\tlinks := make(chan coreiface.DirEntry)\n\tgo func() {\n\t\terrCh <- api.Unixfs().Ls(ctx, path.FromCid(nd.Cid()), links)\n\t}()\n\n\tvar count int\n\tfor range links {\n\t\tcount++\n\t}\n\tif err = <-errCh; err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif count != 0 {\n\t\tt.Fatalf(\"expected 0 links, got %d\", count)\n\t}\n}\n\ntype closeTestF struct {\n\tfiles.File\n\tclosed bool\n\n\tt *testing.T\n}\n\ntype closeTestD struct {\n\tfiles.Directory\n\tclosed bool\n\n\tt *testing.T\n}\n\nfunc (f *closeTestD) Close() error {\n\tf.t.Helper()\n\tif f.closed {\n\t\tf.t.Fatal(\"already closed\")\n\t}\n\tf.closed = true\n\treturn nil\n}\n\nfunc (f *closeTestF) Close() error {\n\tif f.closed {\n\t\tf.t.Fatal(\"already closed\")\n\t}\n\tf.closed = true\n\treturn nil\n}\n\nfunc (tp *TestSuite) TestAddCloses(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tn4 := &closeTestF{files.NewBytesFile([]byte(\"foo\")), false, t}\n\td3 := &closeTestD{files.NewMapDirectory(map[string]files.Node{\n\t\t\"sub\": n4,\n\t}), false, t}\n\tn2 := &closeTestF{files.NewBytesFile([]byte(\"bar\")), false, t}\n\tn1 := &closeTestF{files.NewBytesFile([]byte(\"baz\")), false, t}\n\td0 := &closeTestD{files.NewMapDirectory(map[string]files.Node{\n\t\t\"a\": d3,\n\t\t\"b\": n1,\n\t\t\"c\": n2,\n\t}), false, t}\n\n\t_, err = api.Unixfs().Add(ctx, d0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfor i, n := range []*closeTestF{n1, n2, n4} {\n\t\tif !n.closed {\n\t\t\tt.Errorf(\"file %d not closed!\", i)\n\t\t}\n\t}\n\n\tfor i, n := range []*closeTestD{d0, d3} {\n\t\tif !n.closed {\n\t\t\tt.Errorf(\"dir %d not closed!\", i)\n\t\t}\n\t}\n}\n\nfunc (tp *TestSuite) TestGetSeek(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdataSize := int64(100000)\n\ttf := files.NewReaderFile(io.LimitReader(rand.New(rand.NewSource(1403768328)), dataSize))\n\n\tp, err := api.Unixfs().Add(ctx, tf, options.Unixfs.Chunker(\"size-100\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr, err := api.Unixfs().Get(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tf := files.ToFile(r)\n\tif f == nil {\n\t\tt.Fatal(\"not a file\")\n\t}\n\n\torig := make([]byte, dataSize)\n\tif _, err := io.ReadFull(f, orig); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tf.Close()\n\n\torigR := bytes.NewReader(orig)\n\n\tr, err = api.Unixfs().Get(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tf = files.ToFile(r)\n\tif f == nil {\n\t\tt.Fatal(\"not a file\")\n\t}\n\n\ttest := func(offset int64, whence int, read int, expect int64, shouldEof bool) {\n\t\tt.Run(fmt.Sprintf(\"seek%d+%d-r%d-%d\", whence, offset, read, expect), func(t *testing.T) {\n\t\t\tn, err := f.Seek(offset, whence)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\torigN, err := origR.Seek(offset, whence)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\tif n != origN {\n\t\t\t\tt.Fatalf(\"offsets didn't match, expected %d, got %d\", origN, n)\n\t\t\t}\n\n\t\t\tbuf := make([]byte, read)\n\t\t\torigBuf := make([]byte, read)\n\t\t\torigRead, err := origR.Read(origBuf)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"orig: %s\", err)\n\t\t\t}\n\t\t\tr, err := io.ReadFull(f, buf)\n\t\t\tswitch {\n\t\t\tcase shouldEof && err != nil && err != io.ErrUnexpectedEOF:\n\t\t\t\tfallthrough\n\t\t\tcase !shouldEof && err != nil:\n\t\t\t\tt.Fatalf(\"f: %s\", err)\n\t\t\tcase shouldEof:\n\t\t\t\t_, err := f.Read([]byte{0})\n\t\t\t\tif err != io.EOF {\n\t\t\t\t\tt.Fatal(\"expected EOF\")\n\t\t\t\t}\n\t\t\t\t_, err = origR.Read([]byte{0})\n\t\t\t\tif err != io.EOF {\n\t\t\t\t\tt.Fatal(\"expected EOF (orig)\")\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif int64(r) != expect {\n\t\t\t\tt.Fatal(\"read wrong amount of data\")\n\t\t\t}\n\t\t\tif r != origRead {\n\t\t\t\tt.Fatal(\"read different amount of data than bytes.Reader\")\n\t\t\t}\n\t\t\tif !bytes.Equal(buf, origBuf) {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"original:\\n%s\\n\", hex.Dump(origBuf))\n\t\t\t\tfmt.Fprintf(os.Stderr, \"got:\\n%s\\n\", hex.Dump(buf))\n\t\t\t\tt.Fatal(\"data didn't match\")\n\t\t\t}\n\t\t})\n\t}\n\n\ttest(3, io.SeekCurrent, 10, 10, false)\n\ttest(3, io.SeekCurrent, 10, 10, false)\n\ttest(500, io.SeekCurrent, 10, 10, false)\n\ttest(350, io.SeekStart, 100, 100, false)\n\ttest(-123, io.SeekCurrent, 100, 100, false)\n\ttest(0, io.SeekStart, int(dataSize), dataSize, false)\n\ttest(dataSize-50, io.SeekStart, 100, 50, true)\n\ttest(-5, io.SeekEnd, 100, 5, true)\n}\n\nfunc (tp *TestSuite) TestGetReadAt(t *testing.T) {\n\tctx := t.Context()\n\tapi, err := tp.makeAPI(t, ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdataSize := int64(100000)\n\ttf := files.NewReaderFile(io.LimitReader(rand.New(rand.NewSource(1403768328)), dataSize))\n\n\tp, err := api.Unixfs().Add(ctx, tf, options.Unixfs.Chunker(\"size-100\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tr, err := api.Unixfs().Get(ctx, p)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tf, ok := r.(interface {\n\t\tfiles.File\n\t\tio.ReaderAt\n\t})\n\tif !ok {\n\t\tt.Skip(\"ReaderAt not implemented\")\n\t}\n\n\torig := make([]byte, dataSize)\n\tif _, err := io.ReadFull(f, orig); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tf.Close()\n\n\torigR := bytes.NewReader(orig)\n\n\tif _, err := api.Unixfs().Get(ctx, p); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\ttest := func(offset int64, read int, expect int64, shouldEof bool) {\n\t\tt.Run(fmt.Sprintf(\"readat%d-r%d-%d\", offset, read, expect), func(t *testing.T) {\n\t\t\torigBuf := make([]byte, read)\n\t\t\torigRead, err := origR.ReadAt(origBuf, offset)\n\t\t\tif err != nil && err != io.EOF {\n\t\t\t\tt.Fatalf(\"orig: %s\", err)\n\t\t\t}\n\t\t\tbuf := make([]byte, read)\n\t\t\tr, err := f.ReadAt(buf, offset)\n\t\t\tif shouldEof {\n\t\t\t\tif err != io.EOF {\n\t\t\t\t\tt.Fatal(\"expected EOF, got: \", err)\n\t\t\t\t}\n\t\t\t} else if err != nil {\n\t\t\t\tt.Fatal(\"got: \", err)\n\t\t\t}\n\n\t\t\tif int64(r) != expect {\n\t\t\t\tt.Fatal(\"read wrong amount of data\")\n\t\t\t}\n\t\t\tif r != origRead {\n\t\t\t\tt.Fatal(\"read different amount of data than bytes.Reader\")\n\t\t\t}\n\t\t\tif !bytes.Equal(buf, origBuf) {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"original:\\n%s\\n\", hex.Dump(origBuf))\n\t\t\t\tfmt.Fprintf(os.Stderr, \"got:\\n%s\\n\", hex.Dump(buf))\n\t\t\t\tt.Fatal(\"data didn't match\")\n\t\t\t}\n\t\t})\n\t}\n\n\ttest(3, 10, 10, false)\n\ttest(13, 10, 10, false)\n\ttest(513, 10, 10, false)\n\ttest(350, 100, 100, false)\n\ttest(0, int(dataSize), dataSize, false)\n\ttest(dataSize-50, 100, 50, true)\n}\n" }, { "path": "core/coreiface/unixfs.go", "content": "package iface\n\nimport (\n\t\"context\"\n\t\"iter\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\ntype AddEvent struct {\n\tName string\n\tPath path.ImmutablePath\n\tBytes int64 `json:\",omitempty\"`\n\tSize string `json:\",omitempty\"`\n\tMode os.FileMode `json:\",omitempty\"`\n\tMtime int64 `json:\",omitempty\"`\n\tMtimeNsecs int `json:\",omitempty\"`\n}\n\n// FileType is an enum of possible UnixFS file types.\ntype FileType int32\n\nconst (\n\t// TUnknown means the file type isn't known (e.g., it hasn't been\n\t// resolved).\n\tTUnknown FileType = iota\n\t// TFile is a regular file.\n\tTFile\n\t// TDirectory is a directory.\n\tTDirectory\n\t// TSymlink is a symlink.\n\tTSymlink\n)\n\nfunc (t FileType) String() string {\n\tswitch t {\n\tcase TUnknown:\n\t\treturn \"unknown\"\n\tcase TFile:\n\t\treturn \"file\"\n\tcase TDirectory:\n\t\treturn \"directory\"\n\tcase TSymlink:\n\t\treturn \"symlink\"\n\tdefault:\n\t\treturn \"\"\n\t}\n}\n\n// DirEntry is a directory entry returned by `Ls`.\ntype DirEntry struct {\n\tName string\n\tCid cid.Cid\n\n\t// Only filled when asked to resolve the directory entry.\n\tSize uint64 // The size of the file in bytes (or the size of the symlink).\n\tType FileType // The type of the file.\n\tTarget string // The symlink target (if a symlink).\n\n\tMode os.FileMode\n\tModTime time.Time\n}\n\n// UnixfsAPI is the basic interface to immutable files in IPFS\n// NOTE: This API is heavily WIP, things are guaranteed to break frequently\ntype UnixfsAPI interface {\n\t// Add imports the data from the reader into merkledag file\n\t//\n\t// TODO: a long useful comment on how to use this for many different scenarios\n\tAdd(context.Context, files.Node, ...options.UnixfsAddOption) (path.ImmutablePath, error)\n\n\t// Get returns a read-only handle to a file tree referenced by a path\n\t//\n\t// Note that some implementations of this API may apply the specified context\n\t// to operations performed on the returned file\n\tGet(context.Context, path.Path) (files.Node, error)\n\n\t// Ls writes the links in a directory to the DirEntry channel. Links aren't\n\t// guaranteed to be returned in order. If an error occurs or the context is\n\t// canceled, the DirEntry channel is closed and an error is returned.\n\t//\n\t// Example:\n\t//\n\t// dirs := make(chan DirEntry)\n\t// lsErr := make(chan error, 1)\n\t// go func() {\n\t//\t\tlsErr <- Ls(ctx, p, dirs)\n\t// }()\n\t//\tfor dirEnt := range dirs {\n\t//\t\tfmt.Println(\"Dir name:\", dirEnt.Name)\n\t//\t}\n\t//\terr := <-lsErr\n\t//\tif err != nil {\n\t//\t\treturn fmt.Errorf(\"error listing directory: %w\", err)\n\t//\t}\n\tLs(context.Context, path.Path, chan<- DirEntry, ...options.UnixfsLsOption) error\n}\n\n// LsIter returns a go iterator that allows ranging over DirEntry results.\n// Iteration stops if the context is canceled or if the iterator yields an\n// error.\n//\n// Example:\n//\n//\tfor dirEnt, err := LsIter(ctx, ufsAPI, p) {\n//\t\tif err != nil {\n//\t\t\treturn fmt.Errorf(\"error listing directory: %w\", err)\n//\t\t}\n//\t\tfmt.Println(\"Dir name:\", dirEnt.Name)\n//\t}\nfunc LsIter(ctx context.Context, api UnixfsAPI, p path.Path, opts ...options.UnixfsLsOption) iter.Seq2[DirEntry, error] {\n\treturn func(yield func(DirEntry, error) bool) {\n\t\tctx, cancel := context.WithCancel(ctx)\n\t\tdefer cancel() // cancel Ls if done iterating early\n\n\t\tdirs := make(chan DirEntry)\n\t\tlsErr := make(chan error, 1)\n\t\tgo func() {\n\t\t\tlsErr <- api.Ls(ctx, p, dirs, opts...)\n\t\t}()\n\t\tfor dirEnt := range dirs {\n\t\t\tif !yield(dirEnt, nil) {\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t\tif err := <-lsErr; err != nil {\n\t\t\tyield(DirEntry{}, err)\n\t\t}\n\t}\n}\n" }, { "path": "core/coreiface/util.go", "content": "package iface\n\nimport (\n\t\"context\"\n\t\"io\"\n)\n\ntype Reader interface {\n\tReadSeekCloser\n\tSize() uint64\n\tCtxReadFull(context.Context, []byte) (int, error)\n}\n\n// A ReadSeekCloser implements interfaces to read, copy, seek and close.\ntype ReadSeekCloser interface {\n\tio.Reader\n\tio.Seeker\n\tio.Closer\n\tio.WriterTo\n}\n" }, { "path": "core/corerepo/gc.go", "content": "package corerepo\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/gc\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/ipfs/boxo/mfs\"\n\t\"github.com/ipfs/go-cid\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n)\n\nvar log = logging.Logger(\"corerepo\")\n\nvar ErrMaxStorageExceeded = errors.New(\"maximum storage limit exceeded. Try to unpin some files\")\n\ntype GC struct {\n\tNode *core.IpfsNode\n\tRepo repo.Repo\n\tStorageMax uint64\n\tStorageGC uint64\n\tSlackGB uint64\n\tStorage uint64\n}\n\nfunc NewGC(n *core.IpfsNode) (*GC, error) {\n\tr := n.Repo\n\tcfg, err := r.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// check if cfg has these fields initialized\n\t// TODO: there should be a general check for all of the cfg fields\n\t// maybe distinguish between user config file and default struct?\n\tif cfg.Datastore.StorageMax == \"\" {\n\t\tif err := r.SetConfigKey(\"Datastore.StorageMax\", \"10GB\"); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tcfg.Datastore.StorageMax = \"10GB\"\n\t}\n\tif cfg.Datastore.StorageGCWatermark == 0 {\n\t\tif err := r.SetConfigKey(\"Datastore.StorageGCWatermark\", 90); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tcfg.Datastore.StorageGCWatermark = 90\n\t}\n\n\tstorageMax, err := humanize.ParseBytes(cfg.Datastore.StorageMax)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tstorageGC := storageMax * uint64(cfg.Datastore.StorageGCWatermark) / 100\n\n\t// calculate the slack space between StorageMax and StorageGCWatermark\n\t// used to limit GC duration\n\tslackGB := max((storageMax-storageGC)/10e9, 1)\n\n\treturn &GC{\n\t\tNode: n,\n\t\tRepo: r,\n\t\tStorageMax: storageMax,\n\t\tStorageGC: storageGC,\n\t\tSlackGB: slackGB,\n\t}, nil\n}\n\nfunc BestEffortRoots(filesRoot *mfs.Root) ([]cid.Cid, error) {\n\trootDag, err := filesRoot.GetDirectory().GetNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn []cid.Cid{rootDag.Cid()}, nil\n}\n\nfunc GarbageCollect(n *core.IpfsNode, ctx context.Context) error {\n\troots, err := BestEffortRoots(n.FilesRoot)\n\tif err != nil {\n\t\treturn err\n\t}\n\trmed := gc.GC(ctx, n.Blockstore, n.Repo.Datastore(), n.Pinning, roots)\n\n\treturn CollectResult(ctx, rmed, nil)\n}\n\n// CollectResult collects the output of a garbage collection run and calls the\n// given callback for each object removed. It also collects all errors into a\n// MultiError which is returned after the gc is completed.\nfunc CollectResult(ctx context.Context, gcOut <-chan gc.Result, cb func(cid.Cid)) error {\n\tvar errors []error\nloop:\n\tfor {\n\t\tselect {\n\t\tcase res, ok := <-gcOut:\n\t\t\tif !ok {\n\t\t\t\tbreak loop\n\t\t\t}\n\t\t\tif res.Error != nil {\n\t\t\t\terrors = append(errors, res.Error)\n\t\t\t} else if res.KeyRemoved.Defined() && cb != nil {\n\t\t\t\tcb(res.KeyRemoved)\n\t\t\t}\n\t\tcase <-ctx.Done():\n\t\t\terrors = append(errors, ctx.Err())\n\t\t\tbreak loop\n\t\t}\n\t}\n\n\tswitch len(errors) {\n\tcase 0:\n\t\treturn nil\n\tcase 1:\n\t\treturn errors[0]\n\tdefault:\n\t\treturn NewMultiError(errors...)\n\t}\n}\n\n// NewMultiError creates a new MultiError object from a given slice of errors.\nfunc NewMultiError(errs ...error) *MultiError {\n\treturn &MultiError{errs[:len(errs)-1], errs[len(errs)-1]}\n}\n\n// MultiError contains the results of multiple errors.\ntype MultiError struct {\n\tErrors []error\n\tSummary error\n}\n\nfunc (e *MultiError) Error() string {\n\tvar buf bytes.Buffer\n\tfor _, err := range e.Errors {\n\t\tbuf.WriteString(err.Error())\n\t\tbuf.WriteString(\"; \")\n\t}\n\tbuf.WriteString(e.Summary.Error())\n\treturn buf.String()\n}\n\nfunc GarbageCollectAsync(n *core.IpfsNode, ctx context.Context) <-chan gc.Result {\n\troots, err := BestEffortRoots(n.FilesRoot)\n\tif err != nil {\n\t\tout := make(chan gc.Result)\n\t\tout <- gc.Result{Error: err}\n\t\tclose(out)\n\t\treturn out\n\t}\n\n\treturn gc.GC(ctx, n.Blockstore, n.Repo.Datastore(), n.Pinning, roots)\n}\n\nfunc PeriodicGC(ctx context.Context, node *core.IpfsNode) error {\n\tcfg, err := node.Repo.Config()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif cfg.Datastore.GCPeriod == \"\" {\n\t\tcfg.Datastore.GCPeriod = \"1h\"\n\t}\n\n\tperiod, err := time.ParseDuration(cfg.Datastore.GCPeriod)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif int64(period) == 0 {\n\t\t// if duration is 0, it means GC is disabled.\n\t\treturn nil\n\t}\n\n\tgc, err := NewGC(node)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn nil\n\t\tcase <-time.After(period):\n\t\t\t// the private func maybeGC doesn't compute storageMax, storageGC, slackGC so that they are not re-computed for every cycle\n\t\t\tif err := gc.maybeGC(ctx, 0); err != nil {\n\t\t\t\tlog.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc ConditionalGC(ctx context.Context, node *core.IpfsNode, offset uint64) error {\n\tgc, err := NewGC(node)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn gc.maybeGC(ctx, offset)\n}\n\nfunc (gc *GC) maybeGC(ctx context.Context, offset uint64) error {\n\tstorage, err := gc.Repo.GetStorageUsage(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif storage+offset > gc.StorageGC {\n\t\tif storage+offset > gc.StorageMax {\n\t\t\tlog.Warnf(\"pre-GC: %s\", ErrMaxStorageExceeded)\n\t\t}\n\n\t\t// Do GC here\n\t\tlog.Info(\"Watermark exceeded. Starting repo GC...\")\n\n\t\tif err := GarbageCollect(gc.Node, ctx); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tlog.Infof(\"Repo GC done. See `ipfs repo stat` to see how much space got freed.\\n\")\n\t}\n\treturn nil\n}\n" }, { "path": "core/corerepo/stat.go", "content": "package corerepo\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\n\tcontext \"context\"\n\n\t\"github.com/ipfs/kubo/core\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n)\n\n// SizeStat wraps information about the repository size and its limit.\ntype SizeStat struct {\n\tRepoSize uint64 // size in bytes\n\tStorageMax uint64 // size in bytes\n}\n\n// Stat wraps information about the objects stored on disk.\ntype Stat struct {\n\tSizeStat\n\tNumObjects uint64\n\tRepoPath string\n\tVersion string\n}\n\n// NoLimit represents the value for unlimited storage\nconst NoLimit uint64 = math.MaxUint64\n\n// RepoStat returns a *Stat object with all the fields set.\nfunc RepoStat(ctx context.Context, n *core.IpfsNode) (Stat, error) {\n\tsizeStat, err := RepoSize(ctx, n)\n\tif err != nil {\n\t\treturn Stat{}, err\n\t}\n\n\tallKeys, err := n.Blockstore.AllKeysChan(ctx)\n\tif err != nil {\n\t\treturn Stat{}, err\n\t}\n\n\tcount := uint64(0)\n\tfor range allKeys {\n\t\tcount++\n\t}\n\n\tpath, err := fsrepo.BestKnownPath()\n\tif err != nil {\n\t\treturn Stat{}, err\n\t}\n\n\treturn Stat{\n\t\tSizeStat: SizeStat{\n\t\t\tRepoSize: sizeStat.RepoSize,\n\t\t\tStorageMax: sizeStat.StorageMax,\n\t\t},\n\t\tNumObjects: count,\n\t\tRepoPath: path,\n\t\tVersion: fmt.Sprintf(\"fs-repo@%d\", fsrepo.RepoVersion),\n\t}, nil\n}\n\n// RepoSize returns a *Stat object with the RepoSize and StorageMax fields set.\nfunc RepoSize(ctx context.Context, n *core.IpfsNode) (SizeStat, error) {\n\tr := n.Repo\n\n\tcfg, err := r.Config()\n\tif err != nil {\n\t\treturn SizeStat{}, err\n\t}\n\n\tusage, err := r.GetStorageUsage(ctx)\n\tif err != nil {\n\t\treturn SizeStat{}, err\n\t}\n\n\tstorageMax := NoLimit\n\tif cfg.Datastore.StorageMax != \"\" {\n\t\tstorageMax, err = humanize.ParseBytes(cfg.Datastore.StorageMax)\n\t\tif err != nil {\n\t\t\treturn SizeStat{}, err\n\t\t}\n\t}\n\n\treturn SizeStat{\n\t\tRepoSize: usage,\n\t\tStorageMax: storageMax,\n\t}, nil\n}\n" }, { "path": "core/coreunix/add.go", "content": "package coreunix\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\tgopath \"path\"\n\t\"strconv\"\n\t\"time\"\n\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\tchunker \"github.com/ipfs/boxo/chunker\"\n\t\"github.com/ipfs/boxo/files\"\n\tposinfo \"github.com/ipfs/boxo/filestore/posinfo\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/ipld/unixfs/importer/balanced\"\n\tihelper \"github.com/ipfs/boxo/ipld/unixfs/importer/helpers\"\n\t\"github.com/ipfs/boxo/ipld/unixfs/importer/trickle\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\t\"github.com/ipfs/boxo/mfs\"\n\t\"github.com/ipfs/boxo/path\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n\n\t\"github.com/ipfs/kubo/tracing\"\n)\n\nvar log = logging.Logger(\"coreunix\")\n\n// how many bytes of progress to wait before sending a progress update message\nconst progressReaderIncrement = 1024 * 256\n\nvar liveCacheSize = uint64(256 << 10)\n\ntype Link struct {\n\tName, Hash string\n\tSize uint64\n}\n\ntype syncer interface {\n\tSync() error\n}\n\n// NewAdder Returns a new Adder used for a file add operation.\nfunc NewAdder(ctx context.Context, p pin.Pinner, bs bstore.GCLocker, ds ipld.DAGService) (*Adder, error) {\n\tbufferedDS := ipld.NewBufferedDAG(ctx, ds)\n\n\treturn &Adder{\n\t\tctx: ctx,\n\t\tpinning: p,\n\t\tgcLocker: bs,\n\t\tdagService: ds,\n\t\tbufferedDS: bufferedDS,\n\t\tProgress: false,\n\t\tPin: true,\n\t\tTrickle: false,\n\t\tMaxLinks: ihelper.DefaultLinksPerBlock,\n\t\tMaxHAMTFanout: uio.DefaultShardWidth,\n\t\tChunker: \"\",\n\t\tIncludeEmptyDirs: config.DefaultUnixFSIncludeEmptyDirs,\n\t}, nil\n}\n\n// Adder holds the switches passed to the `add` command.\ntype Adder struct {\n\tctx context.Context\n\tpinning pin.Pinner\n\tgcLocker bstore.GCLocker\n\tdagService ipld.DAGService\n\tbufferedDS *ipld.BufferedDAG\n\tOut chan<- any\n\tProgress bool\n\tPin bool\n\tPinName string\n\tTrickle bool\n\tRawLeaves bool\n\tMaxLinks int\n\tMaxDirectoryLinks int\n\tMaxHAMTFanout int\n\tSizeEstimationMode *uio.SizeEstimationMode\n\tSilent bool\n\tNoCopy bool\n\tChunker string\n\tmroot *mfs.Root\n\tunlocker bstore.Unlocker\n\ttempRoot cid.Cid\n\tCidBuilder cid.Builder\n\tliveNodes uint64\n\n\tPreserveMode bool\n\tPreserveMtime bool\n\tFileMode os.FileMode\n\tFileMtime time.Time\n\tIncludeEmptyDirs bool\n}\n\nfunc (adder *Adder) mfsRoot() (*mfs.Root, error) {\n\tif adder.mroot != nil {\n\t\treturn adder.mroot, nil\n\t}\n\n\t// Note, this adds it to DAGService already.\n\tmr, err := mfs.NewEmptyRoot(adder.ctx, adder.dagService, nil, nil, mfs.MkdirOpts{\n\t\tCidBuilder: adder.CidBuilder,\n\t\tMaxLinks: adder.MaxDirectoryLinks,\n\t\tMaxHAMTFanout: adder.MaxHAMTFanout,\n\t\tSizeEstimationMode: adder.SizeEstimationMode,\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tadder.mroot = mr\n\treturn adder.mroot, nil\n}\n\n// SetMfsRoot sets `r` as the root for Adder.\nfunc (adder *Adder) SetMfsRoot(r *mfs.Root) {\n\tadder.mroot = r\n}\n\n// Constructs a node from reader's data, and adds it. Doesn't pin.\nfunc (adder *Adder) add(reader io.Reader) (ipld.Node, error) {\n\tchnk, err := chunker.FromString(reader, adder.Chunker)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmaxLinks := ihelper.DefaultLinksPerBlock\n\tif adder.MaxLinks > 0 {\n\t\tmaxLinks = adder.MaxLinks\n\t}\n\n\tparams := ihelper.DagBuilderParams{\n\t\tDagserv: adder.bufferedDS,\n\t\tRawLeaves: adder.RawLeaves,\n\t\tMaxlinks: maxLinks,\n\t\tNoCopy: adder.NoCopy,\n\t\tCidBuilder: adder.CidBuilder,\n\t\tFileMode: adder.FileMode,\n\t\tFileModTime: adder.FileMtime,\n\t}\n\n\tdb, err := params.New(chnk)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tvar nd ipld.Node\n\tif adder.Trickle {\n\t\tnd, err = trickle.Layout(db)\n\t} else {\n\t\tnd, err = balanced.Layout(db)\n\t}\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn nd, adder.bufferedDS.Commit()\n}\n\n// RootNode returns the mfs root node\nfunc (adder *Adder) curRootNode() (ipld.Node, error) {\n\tmr, err := adder.mfsRoot()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\troot, err := mr.GetDirectory().GetNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// if one root file, use that hash as root.\n\tif len(root.Links()) == 1 {\n\t\tnd, err := root.Links()[0].GetNode(adder.ctx, adder.dagService)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\troot = nd\n\t}\n\n\treturn root, err\n}\n\n// PinRoot recursively pins the root node of Adder with an optional name and\n// writes the pin state to the backing datastore. If name is empty, the pin\n// will be created without a name.\nfunc (adder *Adder) PinRoot(ctx context.Context, root ipld.Node, name string) error {\n\tctx, span := tracing.Span(ctx, \"CoreUnix.Adder\", \"PinRoot\")\n\tdefer span.End()\n\n\tif !adder.Pin {\n\t\treturn nil\n\t}\n\n\trnk := root.Cid()\n\n\terr := adder.dagService.Add(ctx, root)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif adder.tempRoot.Defined() {\n\t\terr := adder.pinning.Unpin(ctx, adder.tempRoot, true)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tadder.tempRoot = rnk\n\t}\n\n\terr = adder.pinning.PinWithMode(ctx, rnk, pin.Recursive, name)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn adder.pinning.Flush(ctx)\n}\n\nfunc (adder *Adder) outputDirs(path string, fsn mfs.FSNode) error {\n\tswitch fsn := fsn.(type) {\n\tcase *mfs.File:\n\t\treturn nil\n\tcase *mfs.Directory:\n\t\tnames, err := fsn.ListNames(adder.ctx)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor _, name := range names {\n\t\t\tchild, err := fsn.Child(name)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tchildpath := gopath.Join(path, name)\n\t\t\terr = adder.outputDirs(childpath, child)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tfsn.Uncache(name)\n\t\t}\n\t\tnd, err := fsn.GetNode()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn outputDagnode(adder.Out, path, nd)\n\tdefault:\n\t\treturn fmt.Errorf(\"unrecognized fsn type: %#v\", fsn)\n\t}\n}\n\nfunc (adder *Adder) addNode(node ipld.Node, path string) error {\n\t// patch it into the root\n\tif path == \"\" {\n\t\tpath = node.Cid().String()\n\t}\n\n\tif pi, ok := node.(*posinfo.FilestoreNode); ok {\n\t\tnode = pi.Node\n\t}\n\n\tmr, err := adder.mfsRoot()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdir := gopath.Dir(path)\n\tif dir != \".\" {\n\t\topts := mfs.MkdirOpts{\n\t\t\tMkparents: true,\n\t\t\tFlush: false,\n\t\t\tCidBuilder: adder.CidBuilder,\n\t\t\tMaxLinks: adder.MaxDirectoryLinks,\n\t\t\tMaxHAMTFanout: adder.MaxHAMTFanout,\n\t\t\tSizeEstimationMode: adder.SizeEstimationMode,\n\t\t}\n\t\tif err := mfs.Mkdir(mr, dir, opts); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif err := mfs.PutNode(mr, path, node); err != nil {\n\t\treturn err\n\t}\n\n\tif !adder.Silent {\n\t\treturn outputDagnode(adder.Out, path, node)\n\t}\n\treturn nil\n}\n\n// AddAllAndPin adds the given request's files and pin them.\nfunc (adder *Adder) AddAllAndPin(ctx context.Context, file files.Node) (ipld.Node, error) {\n\tctx, span := tracing.Span(ctx, \"CoreUnix.Adder\", \"AddAllAndPin\")\n\tdefer span.End()\n\n\tif adder.Pin {\n\t\tadder.unlocker = adder.gcLocker.PinLock(ctx)\n\t}\n\tdefer func() {\n\t\tif adder.unlocker != nil {\n\t\t\tadder.unlocker.Unlock(ctx)\n\t\t}\n\t}()\n\n\tif err := adder.addFileNode(ctx, \"\", file, true); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// get root\n\tmr, err := adder.mfsRoot()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tvar root mfs.FSNode\n\trootdir := mr.GetDirectory()\n\troot = rootdir\n\n\terr = root.Flush()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// if adding a file without wrapping, swap the root to it (when adding a\n\t// directory, mfs root is the directory)\n\t_, dir := file.(files.Directory)\n\tvar name string\n\tif !dir {\n\t\tchildren, err := rootdir.ListNames(adder.ctx)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif len(children) == 0 {\n\t\t\treturn nil, fmt.Errorf(\"expected at least one child dir, got none\")\n\t\t}\n\n\t\t// Replace root with the first child\n\t\tname = children[0]\n\t\troot, err = rootdir.Child(name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\terr = mr.Close()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnd, err := root.GetNode()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// output directory events\n\terr = adder.outputDirs(name, root)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif asyncDagService, ok := adder.dagService.(syncer); ok {\n\t\terr = asyncDagService.Sync()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif !adder.Pin {\n\t\treturn nd, nil\n\t}\n\n\tif err := adder.PinRoot(ctx, nd, adder.PinName); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn nd, nil\n}\n\nfunc (adder *Adder) addFileNode(ctx context.Context, path string, file files.Node, toplevel bool) error {\n\tctx, span := tracing.Span(ctx, \"CoreUnix.Adder\", \"AddFileNode\")\n\tdefer span.End()\n\n\tdefer file.Close()\n\n\terr := adder.maybePauseForGC(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif adder.PreserveMtime {\n\t\tadder.FileMtime = file.ModTime()\n\t}\n\n\tif adder.PreserveMode {\n\t\tadder.FileMode = file.Mode()\n\t}\n\n\tif adder.liveNodes >= liveCacheSize {\n\t\t// TODO: A smarter cache that uses some sort of lru cache with an eviction handler\n\t\tmr, err := adder.mfsRoot()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif err := mr.FlushMemFree(adder.ctx); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tadder.liveNodes = 0\n\t}\n\tadder.liveNodes++\n\n\tswitch f := file.(type) {\n\tcase files.Directory:\n\t\treturn adder.addDir(ctx, path, f, toplevel)\n\tcase *files.Symlink:\n\t\treturn adder.addSymlink(ctx, path, f)\n\tcase files.File:\n\t\treturn adder.addFile(path, f)\n\tdefault:\n\t\treturn errors.New(\"unknown file type\")\n\t}\n}\n\nfunc (adder *Adder) addSymlink(ctx context.Context, path string, l *files.Symlink) error {\n\tsdata, err := unixfs.SymlinkData(l.Target)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif !adder.FileMtime.IsZero() {\n\t\tfsn, err := unixfs.FSNodeFromBytes(sdata)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfsn.SetModTime(adder.FileMtime)\n\t\tif sdata, err = fsn.GetBytes(); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tdagnode := dag.NodeWithData(sdata)\n\terr = dagnode.SetCidBuilder(adder.CidBuilder)\n\tif err != nil {\n\t\treturn err\n\t}\n\terr = adder.dagService.Add(adder.ctx, dagnode)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn adder.addNode(dagnode, path)\n}\n\nfunc (adder *Adder) addFile(path string, file files.File) error {\n\t// if the progress flag was specified, wrap the file so that we can send\n\t// progress updates to the client (over the output channel)\n\tvar reader io.Reader = file\n\tif adder.Progress {\n\t\trdr := &progressReader{file: reader, path: path, out: adder.Out}\n\t\tif fi, ok := file.(files.FileInfo); ok {\n\t\t\treader = &progressReader2{rdr, fi}\n\t\t} else {\n\t\t\treader = rdr\n\t\t}\n\t}\n\n\tdagnode, err := adder.add(reader)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// patch it into the root\n\treturn adder.addNode(dagnode, path)\n}\n\nfunc (adder *Adder) addDir(ctx context.Context, path string, dir files.Directory, toplevel bool) error {\n\tlog.Infof(\"adding directory: %s\", path)\n\n\t// Peek at first entry to check if directory is empty.\n\t// We advance the iterator once here and continue from this position\n\t// in the processing loop below. This avoids allocating a slice to\n\t// collect all entries just to check for emptiness.\n\tit := dir.Entries()\n\thasEntry := it.Next()\n\tif !hasEntry {\n\t\tif err := it.Err(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\t// Directory is empty. Skip it unless IncludeEmptyDirs is set or\n\t\t// this is the toplevel directory (we always include the root).\n\t\tif !adder.IncludeEmptyDirs && !toplevel {\n\t\t\tlog.Debugf(\"skipping empty directory: %s\", path)\n\t\t\treturn nil\n\t\t}\n\t}\n\n\t// if we need to store mode or modification time then create a new root which includes that data\n\tif toplevel && (adder.FileMode != 0 || !adder.FileMtime.IsZero()) {\n\t\tmr, err := mfs.NewEmptyRoot(ctx, adder.dagService, nil, nil,\n\t\t\tmfs.MkdirOpts{\n\t\t\t\tCidBuilder: adder.CidBuilder,\n\t\t\t\tMaxLinks: adder.MaxDirectoryLinks,\n\t\t\t\tMaxHAMTFanout: adder.MaxHAMTFanout,\n\t\t\t\tModTime: adder.FileMtime,\n\t\t\t\tMode: adder.FileMode,\n\t\t\t\tSizeEstimationMode: adder.SizeEstimationMode,\n\t\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tadder.SetMfsRoot(mr)\n\t}\n\n\tif !(toplevel && path == \"\") {\n\t\tmr, err := adder.mfsRoot()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\terr = mfs.Mkdir(mr, path, mfs.MkdirOpts{\n\t\t\tMkparents: true,\n\t\t\tFlush: false,\n\t\t\tCidBuilder: adder.CidBuilder,\n\t\t\tMode: adder.FileMode,\n\t\t\tModTime: adder.FileMtime,\n\t\t\tMaxLinks: adder.MaxDirectoryLinks,\n\t\t\tMaxHAMTFanout: adder.MaxHAMTFanout,\n\t\t\tSizeEstimationMode: adder.SizeEstimationMode,\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Process directory entries. The iterator was already advanced once above\n\t// to peek for emptiness, so we start from that position.\n\tfor hasEntry {\n\t\tfpath := gopath.Join(path, it.Name())\n\t\tif err := adder.addFileNode(ctx, fpath, it.Node(), false); err != nil {\n\t\t\treturn err\n\t\t}\n\t\thasEntry = it.Next()\n\t}\n\n\treturn it.Err()\n}\n\nfunc (adder *Adder) maybePauseForGC(ctx context.Context) error {\n\tctx, span := tracing.Span(ctx, \"CoreUnix.Adder\", \"MaybePauseForGC\")\n\tdefer span.End()\n\n\tif adder.unlocker != nil && adder.gcLocker.GCRequested(ctx) {\n\t\trn, err := adder.curRootNode()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\terr = adder.PinRoot(ctx, rn, \"\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tadder.unlocker.Unlock(ctx)\n\t\tadder.unlocker = adder.gcLocker.PinLock(ctx)\n\t}\n\treturn nil\n}\n\n// outputDagnode sends dagnode info over the output channel\nfunc outputDagnode(out chan<- any, name string, dn ipld.Node) error {\n\tif out == nil {\n\t\treturn nil\n\t}\n\n\to, err := getOutput(dn)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tout <- &coreiface.AddEvent{\n\t\tPath: o.Path,\n\t\tName: name,\n\t\tSize: o.Size,\n\t}\n\n\treturn nil\n}\n\n// from core/commands/object.go\nfunc getOutput(dagnode ipld.Node) (*coreiface.AddEvent, error) {\n\tc := dagnode.Cid()\n\ts, err := dagnode.Size()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\toutput := &coreiface.AddEvent{\n\t\tPath: path.FromCid(c),\n\t\tSize: strconv.FormatUint(s, 10),\n\t}\n\n\treturn output, nil\n}\n\ntype progressReader struct {\n\tfile io.Reader\n\tpath string\n\tout chan<- any\n\tbytes int64\n\tlastProgress int64\n}\n\nfunc (i *progressReader) Read(p []byte) (int, error) {\n\tn, err := i.file.Read(p)\n\n\ti.bytes += int64(n)\n\tif i.bytes-i.lastProgress >= progressReaderIncrement || err == io.EOF {\n\t\ti.lastProgress = i.bytes\n\t\ti.out <- &coreiface.AddEvent{\n\t\t\tName: i.path,\n\t\t\tBytes: i.bytes,\n\t\t}\n\t}\n\n\treturn n, err\n}\n\ntype progressReader2 struct {\n\t*progressReader\n\tfiles.FileInfo\n}\n\nfunc (i *progressReader2) Read(p []byte) (int, error) {\n\treturn i.progressReader.Read(p)\n}\n" }, { "path": "core/coreunix/add_test.go", "content": "package coreunix\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"math/rand\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/gc\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"github.com/ipfs/boxo/blockservice\"\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/files\"\n\tpi \"github.com/ipfs/boxo/filestore/posinfo\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-datastore\"\n\tsyncds \"github.com/ipfs/go-datastore/sync\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n)\n\nconst testPeerID = \"QmTFauExutTsy4XP6JbMFcw2Wa9645HJt2bTqL6qYDCKfe\"\n\nfunc TestAddMultipleGCLive(t *testing.T) {\n\tctx := t.Context()\n\tr := &repo.Mock{\n\t\tC: config.Config{\n\t\t\tIdentity: config.Identity{\n\t\t\t\tPeerID: testPeerID, // required by offline node\n\t\t\t},\n\t\t},\n\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t}\n\tnode, err := core.NewNode(ctx, &core.BuildCfg{Repo: r})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tout := make(chan any, 10)\n\tadder, err := NewAdder(ctx, node.Pinning, node.Blockstore, node.DAG)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tadder.Out = out\n\n\t// make two files with pipes so we can 'pause' the add for timing of the test\n\tpiper1, pipew1 := io.Pipe()\n\thangfile1 := files.NewReaderFile(piper1)\n\n\tpiper2, pipew2 := io.Pipe()\n\thangfile2 := files.NewReaderFile(piper2)\n\n\trfc := files.NewBytesFile([]byte(\"testfileA\"))\n\n\tslf := files.NewMapDirectory(map[string]files.Node{\n\t\t\"a\": hangfile1,\n\t\t\"b\": hangfile2,\n\t\t\"c\": rfc,\n\t})\n\n\tgo func() {\n\t\tdefer close(out)\n\t\t_, _ = adder.AddAllAndPin(ctx, slf)\n\t\t// Ignore errors for clarity - the real bug would be gc'ing files while adding them, not this resultant error\n\t}()\n\n\t// Start writing the first file but don't close the stream\n\tif _, err := pipew1.Write([]byte(\"some data for file a\")); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar gc1out <-chan gc.Result\n\tgc1started := make(chan struct{})\n\tgo func() {\n\t\tdefer close(gc1started)\n\t\tgc1out = gc.GC(ctx, node.Blockstore, node.Repo.Datastore(), node.Pinning, nil)\n\t}()\n\n\t// Give GC goroutine time to reach GCLock (will block there waiting for adder)\n\ttime.Sleep(time.Millisecond * 100)\n\n\t// GC shouldn't get the lock until after the file is completely added\n\tselect {\n\tcase <-gc1started:\n\t\tt.Fatal(\"gc shouldn't have started yet\")\n\tdefault:\n\t}\n\n\t// finish write and unblock gc\n\tpipew1.Close()\n\n\t// Wait for GC to acquire the lock\n\t// The adder needs to finish processing file 'a' and call maybePauseForGC\n\t// when starting file 'b' before GC can proceed\n\tselect {\n\tcase <-gc1started:\n\t\t// GC got the lock as expected\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"timeout waiting for GC to start - possible deadlock\")\n\t}\n\n\tremovedHashes := make(map[string]struct{})\n\tfor r := range gc1out {\n\t\tif r.Error != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tremovedHashes[r.KeyRemoved.String()] = struct{}{}\n\t}\n\n\tif _, err := pipew2.Write([]byte(\"some data for file b\")); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar gc2out <-chan gc.Result\n\tgc2started := make(chan struct{})\n\tgo func() {\n\t\tdefer close(gc2started)\n\t\tgc2out = gc.GC(ctx, node.Blockstore, node.Repo.Datastore(), node.Pinning, nil)\n\t}()\n\n\t// Give GC goroutine time to reach GCLock\n\ttime.Sleep(time.Millisecond * 100)\n\n\tselect {\n\tcase <-gc2started:\n\t\tt.Fatal(\"gc shouldn't have started yet\")\n\tdefault:\n\t}\n\n\tpipew2.Close()\n\n\t// Wait for second GC to acquire the lock\n\t// The adder needs to finish processing file 'b' and call maybePauseForGC\n\t// when starting file 'c' before GC can proceed\n\tselect {\n\tcase <-gc2started:\n\t\t// GC got the lock as expected\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"timeout waiting for second GC to start - possible deadlock\")\n\t}\n\n\tfor r := range gc2out {\n\t\tif r.Error != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tremovedHashes[r.KeyRemoved.String()] = struct{}{}\n\t}\n\n\tfor o := range out {\n\t\tif _, ok := removedHashes[o.(*coreiface.AddEvent).Path.RootCid().String()]; ok {\n\t\t\tt.Fatal(\"gc'ed a hash we just added\")\n\t\t}\n\t}\n}\n\nfunc TestAddGCLive(t *testing.T) {\n\tctx := t.Context()\n\tr := &repo.Mock{\n\t\tC: config.Config{\n\t\t\tIdentity: config.Identity{\n\t\t\t\tPeerID: testPeerID, // required by offline node\n\t\t\t},\n\t\t},\n\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t}\n\tnode, err := core.NewNode(ctx, &core.BuildCfg{Repo: r})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tout := make(chan any)\n\tadder, err := NewAdder(ctx, node.Pinning, node.Blockstore, node.DAG)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tadder.Out = out\n\n\trfa := files.NewBytesFile([]byte(\"testfileA\"))\n\n\t// make two files with pipes so we can 'pause' the add for timing of the test\n\tpiper, pipew := io.Pipe()\n\thangfile := files.NewReaderFile(piper)\n\n\trfd := files.NewBytesFile([]byte(\"testfileD\"))\n\n\tslf := files.NewMapDirectory(map[string]files.Node{\n\t\t\"a\": rfa,\n\t\t\"b\": hangfile,\n\t\t\"d\": rfd,\n\t})\n\n\taddDone := make(chan struct{})\n\tgo func() {\n\t\tdefer close(addDone)\n\t\tdefer close(out)\n\t\t_, err := adder.AddAllAndPin(ctx, slf)\n\t\tif err != nil {\n\t\t\tt.Error(err)\n\t\t}\n\t}()\n\n\taddedHashes := make(map[string]struct{})\n\tselect {\n\tcase o := <-out:\n\t\taddedHashes[o.(*coreiface.AddEvent).Path.RootCid().String()] = struct{}{}\n\tcase <-addDone:\n\t\tt.Fatal(\"add shouldn't complete yet\")\n\t}\n\n\tvar gcout <-chan gc.Result\n\tgcstarted := make(chan struct{})\n\tgo func() {\n\t\tdefer close(gcstarted)\n\t\tgcout = gc.GC(ctx, node.Blockstore, node.Repo.Datastore(), node.Pinning, nil)\n\t}()\n\n\t// gc shouldn't start until we let the add finish its current file.\n\tif _, err := pipew.Write([]byte(\"some data for file b\")); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tselect {\n\tcase <-gcstarted:\n\t\tt.Fatal(\"gc shouldn't have started yet\")\n\tdefault:\n\t}\n\n\ttime.Sleep(time.Millisecond * 100) // make sure gc gets to requesting lock\n\n\t// finish write and unblock gc\n\tpipew.Close()\n\n\t// receive next object from adder\n\to := <-out\n\taddedHashes[o.(*coreiface.AddEvent).Path.RootCid().String()] = struct{}{}\n\n\t<-gcstarted\n\n\tfor r := range gcout {\n\t\tif r.Error != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, ok := addedHashes[r.KeyRemoved.String()]; ok {\n\t\t\tt.Fatal(\"gc'ed a hash we just added\")\n\t\t}\n\t}\n\n\tvar last cid.Cid\n\tfor a := range out {\n\t\t// wait for it to finish\n\t\tc, err := cid.Decode(a.(*coreiface.AddEvent).Path.RootCid().String())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tlast = c\n\t}\n\n\tset := cid.NewSet()\n\terr = dag.Walk(ctx, dag.GetLinksWithDAG(node.DAG), last, set.Visit)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc testAddWPosInfo(t *testing.T, rawLeaves bool) {\n\tr := &repo.Mock{\n\t\tC: config.Config{\n\t\t\tIdentity: config.Identity{\n\t\t\t\tPeerID: testPeerID, // required by offline node\n\t\t\t},\n\t\t},\n\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t}\n\tnode, err := core.NewNode(context.Background(), &core.BuildCfg{Repo: r})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tbs := &testBlockstore{GCBlockstore: node.Blockstore, expectedPath: filepath.Join(os.TempDir(), \"foo.txt\"), t: t}\n\tbserv := blockservice.New(bs, node.Exchange)\n\tdserv := dag.NewDAGService(bserv)\n\tadder, err := NewAdder(context.Background(), node.Pinning, bs, dserv)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tout := make(chan any)\n\tadder.Out = out\n\tadder.Progress = true\n\tadder.RawLeaves = rawLeaves\n\tadder.NoCopy = true\n\n\tdata := make([]byte, 5*1024*1024)\n\trand.New(rand.NewSource(2)).Read(data) // Rand.Read never returns an error\n\tfileData := io.NopCloser(bytes.NewBuffer(data))\n\tfileInfo := dummyFileInfo{\"foo.txt\", int64(len(data)), time.Now()}\n\tfile, _ := files.NewReaderPathFile(filepath.Join(os.TempDir(), \"foo.txt\"), fileData, &fileInfo)\n\n\tgo func() {\n\t\tdefer close(adder.Out)\n\t\t_, err = adder.AddAllAndPin(context.Background(), file)\n\t\tif err != nil {\n\t\t\tt.Error(err)\n\t\t}\n\t}()\n\tfor range out {\n\t}\n\n\texp := 0\n\tnonOffZero := 0\n\tif rawLeaves {\n\t\texp = 1\n\t\tnonOffZero = 19\n\t}\n\tif bs.countAtOffsetZero != exp {\n\t\tt.Fatalf(\"expected %d blocks with an offset at zero (one root and one leaf), got %d\", exp, bs.countAtOffsetZero)\n\t}\n\tif bs.countAtOffsetNonZero != nonOffZero {\n\t\t// note: the exact number will depend on the size and the sharding algo. used\n\t\tt.Fatalf(\"expected %d blocks with an offset > 0, got %d\", nonOffZero, bs.countAtOffsetNonZero)\n\t}\n}\n\nfunc TestAddWPosInfo(t *testing.T) {\n\ttestAddWPosInfo(t, false)\n}\n\nfunc TestAddWPosInfoAndRawLeafs(t *testing.T) {\n\ttestAddWPosInfo(t, true)\n}\n\ntype testBlockstore struct {\n\tblockstore.GCBlockstore\n\texpectedPath string\n\tt *testing.T\n\tcountAtOffsetZero int\n\tcountAtOffsetNonZero int\n}\n\nfunc (bs *testBlockstore) Put(ctx context.Context, block blocks.Block) error {\n\tbs.CheckForPosInfo(block)\n\treturn bs.GCBlockstore.Put(ctx, block)\n}\n\nfunc (bs *testBlockstore) PutMany(ctx context.Context, blocks []blocks.Block) error {\n\tfor _, blk := range blocks {\n\t\tbs.CheckForPosInfo(blk)\n\t}\n\treturn bs.GCBlockstore.PutMany(ctx, blocks)\n}\n\nfunc (bs *testBlockstore) CheckForPosInfo(block blocks.Block) {\n\tfsn, ok := block.(*pi.FilestoreNode)\n\tif ok {\n\t\tposInfo := fsn.PosInfo\n\t\tif posInfo.FullPath != bs.expectedPath {\n\t\t\tbs.t.Fatal(\"PosInfo does not have the expected path\")\n\t\t}\n\t\tif posInfo.Offset == 0 {\n\t\t\tbs.countAtOffsetZero += 1\n\t\t} else {\n\t\t\tbs.countAtOffsetNonZero += 1\n\t\t}\n\t}\n}\n\ntype dummyFileInfo struct {\n\tname string\n\tsize int64\n\tmodTime time.Time\n}\n\nfunc (fi *dummyFileInfo) Name() string { return fi.name }\nfunc (fi *dummyFileInfo) Size() int64 { return fi.size }\nfunc (fi *dummyFileInfo) Mode() os.FileMode { return 0 }\nfunc (fi *dummyFileInfo) ModTime() time.Time { return fi.modTime }\nfunc (fi *dummyFileInfo) IsDir() bool { return false }\nfunc (fi *dummyFileInfo) Sys() any { return nil }\n" }, { "path": "core/coreunix/metadata.go", "content": "package coreunix\n\nimport (\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\tcid \"github.com/ipfs/go-cid\"\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\nfunc AddMetadataTo(n *core.IpfsNode, skey string, m *ft.Metadata) (string, error) {\n\tc, err := cid.Decode(skey)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tnd, err := n.DAG.Get(n.Context(), c)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tmdnode := new(dag.ProtoNode)\n\tmdata, err := ft.BytesForMetadata(m)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tmdnode.SetData(mdata)\n\tif err := mdnode.AddNodeLink(\"file\", nd); err != nil {\n\t\treturn \"\", err\n\t}\n\n\terr = n.DAG.Add(n.Context(), mdnode)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn mdnode.Cid().String(), nil\n}\n\nfunc Metadata(n *core.IpfsNode, skey string) (*ft.Metadata, error) {\n\tc, err := cid.Decode(skey)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnd, err := n.DAG.Get(n.Context(), c)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpbnd, ok := nd.(*dag.ProtoNode)\n\tif !ok {\n\t\treturn nil, dag.ErrNotProtobuf\n\t}\n\n\treturn ft.MetadataFromBytes(pbnd.Data())\n}\n" }, { "path": "core/coreunix/metadata_test.go", "content": "package coreunix\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"testing\"\n\n\tbserv \"github.com/ipfs/boxo/blockservice\"\n\tmerkledag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\timporter \"github.com/ipfs/boxo/ipld/unixfs/importer\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\tcore \"github.com/ipfs/kubo/core\"\n\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\tchunker \"github.com/ipfs/boxo/chunker\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\tcid \"github.com/ipfs/go-cid\"\n\tds \"github.com/ipfs/go-datastore\"\n\tdssync \"github.com/ipfs/go-datastore/sync\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\t\"github.com/ipfs/go-test/random\"\n)\n\nfunc getDagserv(t *testing.T) ipld.DAGService {\n\tdb := dssync.MutexWrap(ds.NewMapDatastore())\n\tbs := bstore.NewBlockstore(db)\n\tblockserv := bserv.New(bs, offline.Exchange(bs))\n\treturn merkledag.NewDAGService(blockserv)\n}\n\nfunc TestMetadata(t *testing.T) {\n\tctx := context.Background()\n\t// Make some random node\n\tds := getDagserv(t)\n\tdata := make([]byte, 1000)\n\t_, err := io.ReadFull(random.NewRand(), data)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tr := bytes.NewReader(data)\n\tnd, err := importer.BuildDagFromReader(ds, chunker.DefaultSplitter(r))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tc := nd.Cid()\n\n\tm := new(ft.Metadata)\n\tm.MimeType = \"THIS IS A TEST\"\n\n\t// Such effort, many compromise\n\tipfsnode := &core.IpfsNode{DAG: ds}\n\n\tmdk, err := AddMetadataTo(ipfsnode, c.String(), m)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\trec, err := Metadata(ipfsnode, mdk)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif rec.MimeType != m.MimeType {\n\t\tt.Fatalf(\"something went wrong in conversion: '%s' != '%s'\", rec.MimeType, m.MimeType)\n\t}\n\n\tcdk, err := cid.Decode(mdk)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tretnode, err := ds.Get(ctx, cdk)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\trtnpb, ok := retnode.(*merkledag.ProtoNode)\n\tif !ok {\n\t\tt.Fatal(\"expected protobuf node\")\n\t}\n\n\tndr, err := uio.NewDagReader(ctx, rtnpb, ds)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tout, err := io.ReadAll(ndr)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !bytes.Equal(out, data) {\n\t\tt.Fatal(\"read incorrect data\")\n\t}\n}\n" }, { "path": "core/coreunix/test/data/colors/orange", "content": "orange\n" }, { "path": "core/coreunix/test/data/corps/apple", "content": "apple\n" }, { "path": "core/coreunix/test/data/fruits/apple", "content": "apple\n" }, { "path": "core/coreunix/test/data/fruits/orange", "content": "orange\n" }, { "path": "core/mock/mock.go", "content": "package coremock\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\n\tlibp2p2 \"github.com/ipfs/kubo/core/node/libp2p\"\n\n\t\"github.com/ipfs/kubo/commands\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"github.com/ipfs/go-datastore\"\n\tsyncds \"github.com/ipfs/go-datastore/sync\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\ttestutil \"github.com/libp2p/go-libp2p-testing/net\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\n// NewMockNode constructs an IpfsNode for use in tests.\nfunc NewMockNode() (*core.IpfsNode, error) {\n\t// effectively offline, only peer in its network\n\treturn core.NewNode(context.Background(), &core.BuildCfg{\n\t\tOnline: true,\n\t\tHost: MockHostOption(mocknet.New()),\n\t})\n}\n\nfunc MockHostOption(mn mocknet.Mocknet) libp2p2.HostOption {\n\treturn func(id peer.ID, ps pstore.Peerstore, opts ...libp2p.Option) (host.Host, error) {\n\t\tvar cfg libp2p.Config\n\t\tif err := cfg.Apply(opts...); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// The mocknet does not use the provided libp2p.Option. This options include\n\t\t// the listening addresses we want our peer listening on. Therefore, we have\n\t\t// to manually parse the configuration and add them here.\n\t\tps.AddAddrs(id, cfg.ListenAddrs, pstore.PermanentAddrTTL)\n\t\treturn mn.AddPeerWithPeerstore(id, ps)\n\t}\n}\n\nfunc MockCmdsCtx() (commands.Context, error) {\n\t// Generate Identity\n\tident, err := testutil.RandIdentity()\n\tif err != nil {\n\t\treturn commands.Context{}, err\n\t}\n\tp := ident.ID()\n\n\tconf := config.Config{\n\t\tIdentity: config.Identity{\n\t\t\tPeerID: p.String(),\n\t\t},\n\t}\n\n\tr := &repo.Mock{\n\t\tD: syncds.MutexWrap(datastore.NewMapDatastore()),\n\t\tC: conf,\n\t}\n\n\tnode, err := core.NewNode(context.Background(), &core.BuildCfg{\n\t\tRepo: r,\n\t})\n\tif err != nil {\n\t\treturn commands.Context{}, err\n\t}\n\n\treturn commands.Context{\n\t\tConfigRoot: \"/tmp/.mockipfsconfig\",\n\t\tConstructNode: func() (*core.IpfsNode, error) {\n\t\t\treturn node, nil\n\t\t},\n\t}, nil\n}\n\nfunc MockPublicNode(ctx context.Context, mn mocknet.Mocknet) (*core.IpfsNode, error) {\n\tds := syncds.MutexWrap(datastore.NewMapDatastore())\n\tcfg, err := config.Init(io.Discard, 2048)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tcount := len(mn.Peers())\n\tcfg.Addresses.Swarm = []string{\n\t\tfmt.Sprintf(\"/ip4/18.0.%d.%d/tcp/4001\", count>>16, count&0xFF),\n\t}\n\tcfg.Datastore = config.Datastore{}\n\treturn core.NewNode(ctx, &core.BuildCfg{\n\t\tOnline: true,\n\t\tRouting: libp2p2.DHTServerOption,\n\t\tRepo: &repo.Mock{\n\t\t\tC: *cfg,\n\t\t\tD: ds,\n\t\t},\n\t\tHost: MockHostOption(mn),\n\t})\n}\n" }, { "path": "core/node/bitswap.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"time\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/ipfs/boxo/bitswap\"\n\t\"github.com/ipfs/boxo/bitswap/client\"\n\t\"github.com/ipfs/boxo/bitswap/network\"\n\tbsnet \"github.com/ipfs/boxo/bitswap/network/bsnet\"\n\t\"github.com/ipfs/boxo/bitswap/network/httpnet\"\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\texchange \"github.com/ipfs/boxo/exchange\"\n\trpqm \"github.com/ipfs/boxo/routing/providerquerymanager\"\n\t\"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\t\"go.uber.org/fx\"\n\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n)\n\n// Docs: https://github.com/ipfs/kubo/blob/master/docs/config.md#internalbitswap\nconst (\n\tDefaultEngineBlockstoreWorkerCount = 128\n\tDefaultTaskWorkerCount = 8\n\tDefaultEngineTaskWorkerCount = 8\n\tDefaultMaxOutstandingBytesPerPeer = 1 << 20\n\tDefaultProviderSearchDelay = 1000 * time.Millisecond\n\tDefaultMaxProviders = 10 // matching BitswapClientDefaultMaxProviders from https://github.com/ipfs/boxo/blob/v0.29.1/bitswap/internal/defaults/defaults.go#L15\n\tDefaultWantHaveReplaceSize = 1024\n)\n\ntype bitswapOptionsOut struct {\n\tfx.Out\n\n\tBitswapOpts []bitswap.Option `group:\"bitswap-options,flatten\"`\n}\n\n// BitswapOptions creates configuration options for Bitswap from the config file\n// and whether to provide data.\nfunc BitswapOptions(cfg *config.Config) any {\n\treturn func() bitswapOptionsOut {\n\t\tvar internalBsCfg config.InternalBitswap\n\t\tif cfg.Internal.Bitswap != nil {\n\t\t\tinternalBsCfg = *cfg.Internal.Bitswap\n\t\t}\n\n\t\topts := []bitswap.Option{\n\t\t\tbitswap.ProviderSearchDelay(internalBsCfg.ProviderSearchDelay.WithDefault(DefaultProviderSearchDelay)), // See https://github.com/ipfs/go-ipfs/issues/8807 for rationale\n\t\t\tbitswap.EngineBlockstoreWorkerCount(int(internalBsCfg.EngineBlockstoreWorkerCount.WithDefault(DefaultEngineBlockstoreWorkerCount))),\n\t\t\tbitswap.TaskWorkerCount(int(internalBsCfg.TaskWorkerCount.WithDefault(DefaultTaskWorkerCount))),\n\t\t\tbitswap.EngineTaskWorkerCount(int(internalBsCfg.EngineTaskWorkerCount.WithDefault(DefaultEngineTaskWorkerCount))),\n\t\t\tbitswap.MaxOutstandingBytesPerPeer(int(internalBsCfg.MaxOutstandingBytesPerPeer.WithDefault(DefaultMaxOutstandingBytesPerPeer))),\n\t\t\tbitswap.WithWantHaveReplaceSize(int(internalBsCfg.WantHaveReplaceSize.WithDefault(DefaultWantHaveReplaceSize))),\n\t\t}\n\n\t\treturn bitswapOptionsOut{BitswapOpts: opts}\n\t}\n}\n\ntype bitswapIn struct {\n\tfx.In\n\n\tMctx helpers.MetricsCtx\n\tCfg *config.Config\n\tHost host.Host\n\tDiscovery routing.ContentDiscovery\n\tBs blockstore.GCBlockstore\n\tBitswapOpts []bitswap.Option `group:\"bitswap-options\"`\n}\n\n// Bitswap creates the BitSwap server/client instance.\n// If Bitswap.ServerEnabled is false, the node will act only as a client\n// using an empty blockstore to prevent serving blocks to other peers.\nfunc Bitswap(serverEnabled, libp2pEnabled, httpEnabled bool) any {\n\treturn func(in bitswapIn, lc fx.Lifecycle) (*bitswap.Bitswap, error) {\n\t\tvar bitswapNetworks, bitswapLibp2p network.BitSwapNetwork\n\t\tvar bitswapBlockstore blockstore.Blockstore = in.Bs\n\n\t\tconnEvtMgr := network.NewConnectEventManager()\n\n\t\tlibp2pEnabled := in.Cfg.Bitswap.Libp2pEnabled.WithDefault(config.DefaultBitswapLibp2pEnabled)\n\t\tif libp2pEnabled {\n\t\t\tbitswapLibp2p = bsnet.NewFromIpfsHost(\n\t\t\t\tin.Host,\n\t\t\t\tbsnet.WithConnectEventManager(connEvtMgr),\n\t\t\t)\n\t\t}\n\n\t\tif httpEnabled {\n\t\t\thttpCfg := in.Cfg.HTTPRetrieval\n\t\t\tmaxBlockSize, err := humanize.ParseBytes(httpCfg.MaxBlockSize.WithDefault(config.DefaultHTTPRetrievalMaxBlockSize))\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tlogger.Infof(\"HTTP Retrieval enabled: Allowlist: %t. Denylist: %t\",\n\t\t\t\thttpCfg.Allowlist != nil,\n\t\t\t\thttpCfg.Denylist != nil,\n\t\t\t)\n\n\t\t\tbitswapHTTP := httpnet.New(in.Host,\n\t\t\t\thttpnet.WithHTTPWorkers(int(httpCfg.NumWorkers.WithDefault(config.DefaultHTTPRetrievalNumWorkers))),\n\t\t\t\thttpnet.WithAllowlist(httpCfg.Allowlist),\n\t\t\t\thttpnet.WithDenylist(httpCfg.Denylist),\n\t\t\t\thttpnet.WithInsecureSkipVerify(httpCfg.TLSInsecureSkipVerify.WithDefault(config.DefaultHTTPRetrievalTLSInsecureSkipVerify)),\n\t\t\t\thttpnet.WithMaxBlockSize(int64(maxBlockSize)),\n\t\t\t\thttpnet.WithUserAgent(version.GetUserAgentVersion()),\n\t\t\t\thttpnet.WithMetricsLabelsForEndpoints(httpCfg.Allowlist),\n\t\t\t\thttpnet.WithConnectEventManager(connEvtMgr),\n\t\t\t)\n\t\t\tbitswapNetworks = network.New(in.Host.Peerstore(), bitswapLibp2p, bitswapHTTP)\n\t\t} else if libp2pEnabled {\n\t\t\tbitswapNetworks = bitswapLibp2p\n\t\t} else {\n\t\t\treturn nil, errors.New(\"invalid configuration: Bitswap.Libp2pEnabled and HTTPRetrieval.Enabled are both disabled, unable to initialize Bitswap\")\n\t\t}\n\n\t\t// Kubo uses own, customized ProviderQueryManager\n\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.WithDefaultProviderQueryManager(false)))\n\t\tvar maxProviders int = DefaultMaxProviders\n\n\t\tvar bcDisposition string\n\t\tif in.Cfg.Internal.Bitswap != nil {\n\t\t\tmaxProviders = int(in.Cfg.Internal.Bitswap.ProviderSearchMaxResults.WithDefault(DefaultMaxProviders))\n\t\t\tif in.Cfg.Internal.Bitswap.BroadcastControl != nil {\n\t\t\t\tbcCfg := in.Cfg.Internal.Bitswap.BroadcastControl\n\t\t\t\tbcEnable := bcCfg.Enable.WithDefault(config.DefaultBroadcastControlEnable)\n\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlEnable(bcEnable)))\n\t\t\t\tif bcEnable {\n\t\t\t\t\tbcDisposition = \"enabled\"\n\t\t\t\t\tbcMaxPeers := int(bcCfg.MaxPeers.WithDefault(config.DefaultBroadcastControlMaxPeers))\n\t\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlMaxPeers(bcMaxPeers)))\n\n\t\t\t\t\tbcLocalPeers := bcCfg.LocalPeers.WithDefault(config.DefaultBroadcastControlLocalPeers)\n\t\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlLocalPeers(bcLocalPeers)))\n\n\t\t\t\t\tbcPeeredPeers := bcCfg.PeeredPeers.WithDefault(config.DefaultBroadcastControlPeeredPeers)\n\t\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlPeeredPeers(bcPeeredPeers)))\n\n\t\t\t\t\tbcMaxRandomPeers := int(bcCfg.MaxRandomPeers.WithDefault(config.DefaultBroadcastControlMaxRandomPeers))\n\t\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlMaxRandomPeers(bcMaxRandomPeers)))\n\n\t\t\t\t\tbcSendToPendingPeers := bcCfg.SendToPendingPeers.WithDefault(config.DefaultBroadcastControlSendToPendingPeers)\n\t\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlSendToPendingPeers(bcSendToPendingPeers)))\n\t\t\t\t} else {\n\t\t\t\t\tbcDisposition = \"disabled\"\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// If broadcast control is not configured, then configure with defaults.\n\t\tif bcDisposition == \"\" {\n\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlEnable(config.DefaultBroadcastControlEnable)))\n\t\t\tif config.DefaultBroadcastControlEnable {\n\t\t\t\tbcDisposition = \"enabled\"\n\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlMaxPeers(config.DefaultBroadcastControlMaxPeers)))\n\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlLocalPeers(config.DefaultBroadcastControlLocalPeers)))\n\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlPeeredPeers(config.DefaultBroadcastControlPeeredPeers)))\n\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlMaxRandomPeers(config.DefaultBroadcastControlMaxRandomPeers)))\n\t\t\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithClientOption(client.BroadcastControlSendToPendingPeers(config.DefaultBroadcastControlSendToPendingPeers)))\n\t\t\t} else {\n\t\t\t\tbcDisposition = \"enabled\"\n\t\t\t}\n\t\t}\n\t\tlogger.Infof(\"bitswap client broadcast control %s\", bcDisposition)\n\n\t\tignoredPeerIDs := make([]peer.ID, 0, len(in.Cfg.Routing.IgnoreProviders))\n\t\tfor _, str := range in.Cfg.Routing.IgnoreProviders {\n\t\t\tpid, err := peer.Decode(str)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tignoredPeerIDs = append(ignoredPeerIDs, pid)\n\t\t}\n\t\tproviderQueryMgr, err := rpqm.New(bitswapNetworks,\n\t\t\tin.Discovery,\n\t\t\trpqm.WithMaxProviders(maxProviders),\n\t\t\trpqm.WithIgnoreProviders(ignoredPeerIDs...),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Explicitly enable/disable server\n\t\tin.BitswapOpts = append(in.BitswapOpts, bitswap.WithServerEnabled(serverEnabled))\n\n\t\tbs := bitswap.New(helpers.LifecycleCtx(in.Mctx, lc), bitswapNetworks, providerQueryMgr, bitswapBlockstore, in.BitswapOpts...)\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\treturn bs.Close()\n\t\t\t},\n\t\t})\n\t\treturn bs, nil\n\t}\n}\n\n// OnlineExchange creates new LibP2P backed block exchange.\n// Returns a no-op exchange if Bitswap is disabled.\nfunc OnlineExchange(isBitswapActive bool) any {\n\treturn func(in *bitswap.Bitswap, lc fx.Lifecycle) exchange.Interface {\n\t\tif !isBitswapActive {\n\t\t\treturn &noopExchange{closer: in}\n\t\t}\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\treturn in.Close()\n\t\t\t},\n\t\t})\n\t\treturn in\n\t}\n}\n\ntype noopExchange struct {\n\tcloser io.Closer\n}\n\nfunc (e *noopExchange) GetBlock(ctx context.Context, c cid.Cid) (blocks.Block, error) {\n\treturn nil, ipld.ErrNotFound{Cid: c}\n}\n\nfunc (e *noopExchange) GetBlocks(ctx context.Context, cids []cid.Cid) (<-chan blocks.Block, error) {\n\tch := make(chan blocks.Block)\n\tclose(ch)\n\treturn ch, nil\n}\n\nfunc (e *noopExchange) NotifyNewBlocks(ctx context.Context, blocks ...blocks.Block) error {\n\treturn nil\n}\n\nfunc (e *noopExchange) Close() error {\n\treturn e.closer.Close()\n}\n" }, { "path": "core/node/builder.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"crypto/rand\"\n\t\"encoding/base64\"\n\n\t\"go.uber.org/fx\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\tds \"github.com/ipfs/go-datastore\"\n\tdsync \"github.com/ipfs/go-datastore/sync\"\n\tcfg \"github.com/ipfs/kubo/config\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n)\n\ntype BuildCfg struct {\n\t// If online is set, the node will have networking enabled\n\tOnline bool\n\n\t// ExtraOpts is a map of extra options used to configure the ipfs nodes creation\n\tExtraOpts map[string]bool\n\n\t// If permanent then node should run more expensive processes\n\t// that will improve performance in long run\n\tPermanent bool\n\n\t// DisableEncryptedConnections disables connection encryption *entirely*.\n\t// DO NOT SET THIS UNLESS YOU'RE TESTING.\n\tDisableEncryptedConnections bool\n\n\tRouting libp2p.RoutingOption\n\tHost libp2p.HostOption\n\tRepo repo.Repo\n}\n\nfunc (cfg *BuildCfg) getOpt(key string) bool {\n\tif cfg.ExtraOpts == nil {\n\t\treturn false\n\t}\n\n\treturn cfg.ExtraOpts[key]\n}\n\nfunc (cfg *BuildCfg) fillDefaults() error {\n\tif cfg.Repo == nil {\n\t\tr, err := defaultRepo(dsync.MutexWrap(ds.NewMapDatastore()))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tcfg.Repo = r\n\t}\n\n\tif cfg.Routing == nil {\n\t\tcfg.Routing = libp2p.DHTOption\n\t}\n\n\tif cfg.Host == nil {\n\t\tcfg.Host = libp2p.DefaultHostOption\n\t}\n\n\treturn nil\n}\n\n// options creates fx option group from this build config\nfunc (cfg *BuildCfg) options(ctx context.Context) (fx.Option, *cfg.Config) {\n\terr := cfg.fillDefaults()\n\tif err != nil {\n\t\treturn fx.Error(err), nil\n\t}\n\n\trepoOption := fx.Provide(func(lc fx.Lifecycle) repo.Repo {\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\treturn cfg.Repo.Close()\n\t\t\t},\n\t\t})\n\n\t\treturn cfg.Repo\n\t})\n\n\tmetricsCtx := fx.Provide(func() helpers.MetricsCtx {\n\t\treturn helpers.MetricsCtx(ctx)\n\t})\n\n\thostOption := fx.Provide(func() libp2p.HostOption {\n\t\treturn cfg.Host\n\t})\n\n\troutingOption := fx.Provide(func() libp2p.RoutingOption {\n\t\treturn cfg.Routing\n\t})\n\n\tconf, err := cfg.Repo.Config()\n\tif err != nil {\n\t\treturn fx.Error(err), nil\n\t}\n\n\treturn fx.Options(\n\t\trepoOption,\n\t\thostOption,\n\t\troutingOption,\n\t\tmetricsCtx,\n\t), conf\n}\n\nfunc defaultRepo(dstore repo.Datastore) (repo.Repo, error) {\n\tc := cfg.Config{}\n\tpriv, pub, err := crypto.GenerateKeyPairWithReader(crypto.RSA, 2048, rand.Reader)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpid, err := peer.IDFromPublicKey(pub)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tprivkeyb, err := crypto.MarshalPrivateKey(priv)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tc.Bootstrap = autoconf.FallbackBootstrapPeers\n\tc.Addresses.Swarm = []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/udp/4001/quic-v1\"}\n\tc.Identity.PeerID = pid.String()\n\tc.Identity.PrivKey = base64.StdEncoding.EncodeToString(privkeyb)\n\n\treturn &repo.Mock{\n\t\tD: dstore,\n\t\tC: c,\n\t}, nil\n}\n" }, { "path": "core/node/core.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\n\t\"github.com/ipfs/boxo/blockservice\"\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\texchange \"github.com/ipfs/boxo/exchange\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\t\"github.com/ipfs/boxo/fetcher\"\n\tbsfetcher \"github.com/ipfs/boxo/fetcher/impl/blockservice\"\n\t\"github.com/ipfs/boxo/filestore\"\n\t\"github.com/ipfs/boxo/ipld/merkledag\"\n\t\"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/mfs\"\n\tpathresolver \"github.com/ipfs/boxo/path/resolver\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/boxo/pinning/pinner/dspinner\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-datastore\"\n\tformat \"github.com/ipfs/go-ipld-format\"\n\t\"github.com/ipfs/go-unixfsnode\"\n\tdagpb \"github.com/ipld/go-codec-dagpb\"\n\t\"go.uber.org/fx\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/repo\"\n)\n\n// FilesRootDatastoreKey is the datastore key for the MFS files root CID.\nvar FilesRootDatastoreKey = datastore.NewKey(\"/local/filesroot\")\n\n// BlockService creates new blockservice which provides an interface to fetch content-addressable blocks\nfunc BlockService(cfg *config.Config) func(lc fx.Lifecycle, bs blockstore.Blockstore, rem exchange.Interface) blockservice.BlockService {\n\treturn func(lc fx.Lifecycle, bs blockstore.Blockstore, rem exchange.Interface) blockservice.BlockService {\n\t\tbsvc := blockservice.New(bs, rem,\n\t\t\tblockservice.WriteThrough(cfg.Datastore.WriteThrough.WithDefault(config.DefaultWriteThrough)),\n\t\t)\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\treturn bsvc.Close()\n\t\t\t},\n\t\t})\n\n\t\treturn bsvc\n\t}\n}\n\n// Pinning creates new pinner which tells GC which blocks should be kept\nfunc Pinning(strategy string) func(bstore blockstore.Blockstore, ds format.DAGService, repo repo.Repo, prov DHTProvider) (pin.Pinner, error) {\n\t// Parse strategy at function creation time (not inside the returned function)\n\t// This happens before the provider is created, which is why we pass the strategy\n\t// string and parse it here, rather than using fx-provided ProvidingStrategy.\n\tstrategyFlag := config.ParseProvideStrategy(strategy)\n\n\treturn func(bstore blockstore.Blockstore,\n\t\tds format.DAGService,\n\t\trepo repo.Repo,\n\t\tprov DHTProvider,\n\t) (pin.Pinner, error) {\n\t\trootDS := repo.Datastore()\n\n\t\tsyncFn := func(ctx context.Context) error {\n\t\t\tif err := rootDS.Sync(ctx, blockstore.BlockPrefix); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn rootDS.Sync(ctx, filestore.FilestorePrefix)\n\t\t}\n\t\tsyncDs := &syncDagService{ds, syncFn}\n\n\t\tctx := context.TODO()\n\n\t\tvar opts []dspinner.Option\n\t\troots := (strategyFlag & config.ProvideStrategyRoots) != 0\n\t\tpinned := (strategyFlag & config.ProvideStrategyPinned) != 0\n\n\t\t// Important: Only one of WithPinnedProvider or WithRootsProvider should be active.\n\t\t// Having both would cause duplicate root advertisements since \"pinned\" includes all\n\t\t// pinned content (roots + children), while \"roots\" is just the root CIDs.\n\t\t// We prioritize \"pinned\" if both are somehow set (though this shouldn't happen\n\t\t// with proper strategy parsing).\n\t\tif pinned {\n\t\t\topts = append(opts, dspinner.WithPinnedProvider(prov))\n\t\t} else if roots {\n\t\t\topts = append(opts, dspinner.WithRootsProvider(prov))\n\t\t}\n\n\t\tpinning, err := dspinner.New(ctx, rootDS, syncDs, opts...)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn pinning, nil\n\t}\n}\n\nvar (\n\t_ merkledag.SessionMaker = new(syncDagService)\n\t_ format.DAGService = new(syncDagService)\n)\n\n// syncDagService is used by the Pinner to ensure data gets persisted to the underlying datastore\ntype syncDagService struct {\n\tformat.DAGService\n\tsyncFn func(context.Context) error\n}\n\nfunc (s *syncDagService) Sync(ctx context.Context) error {\n\treturn s.syncFn(ctx)\n}\n\nfunc (s *syncDagService) Session(ctx context.Context) format.NodeGetter {\n\treturn merkledag.NewSession(ctx, s.DAGService)\n}\n\n// FetchersOut allows injection of fetchers.\ntype FetchersOut struct {\n\tfx.Out\n\tIPLDFetcher fetcher.Factory `name:\"ipldFetcher\"`\n\tUnixfsFetcher fetcher.Factory `name:\"unixfsFetcher\"`\n\tOfflineIPLDFetcher fetcher.Factory `name:\"offlineIpldFetcher\"`\n\tOfflineUnixfsFetcher fetcher.Factory `name:\"offlineUnixfsFetcher\"`\n}\n\n// FetchersIn allows using fetchers for other dependencies.\ntype FetchersIn struct {\n\tfx.In\n\tIPLDFetcher fetcher.Factory `name:\"ipldFetcher\"`\n\tUnixfsFetcher fetcher.Factory `name:\"unixfsFetcher\"`\n\tOfflineIPLDFetcher fetcher.Factory `name:\"offlineIpldFetcher\"`\n\tOfflineUnixfsFetcher fetcher.Factory `name:\"offlineUnixfsFetcher\"`\n}\n\n// FetcherConfig returns a fetcher config that can build new fetcher instances\nfunc FetcherConfig(bs blockservice.BlockService) FetchersOut {\n\tipldFetcher := bsfetcher.NewFetcherConfig(bs)\n\tipldFetcher.PrototypeChooser = dagpb.AddSupportToChooser(bsfetcher.DefaultPrototypeChooser)\n\tunixFSFetcher := ipldFetcher.WithReifier(unixfsnode.Reify)\n\n\t// Construct offline versions which we can safely use in contexts where\n\t// path resolution should not fetch new blocks via exchange.\n\tofflineBs := blockservice.New(bs.Blockstore(), offline.Exchange(bs.Blockstore()))\n\tofflineIpldFetcher := bsfetcher.NewFetcherConfig(offlineBs)\n\tofflineIpldFetcher.SkipNotFound = true // carries onto the UnixFSFetcher below\n\tofflineIpldFetcher.PrototypeChooser = dagpb.AddSupportToChooser(bsfetcher.DefaultPrototypeChooser)\n\tofflineUnixFSFetcher := offlineIpldFetcher.WithReifier(unixfsnode.Reify)\n\n\treturn FetchersOut{\n\t\tIPLDFetcher: ipldFetcher,\n\t\tUnixfsFetcher: unixFSFetcher,\n\t\tOfflineIPLDFetcher: offlineIpldFetcher,\n\t\tOfflineUnixfsFetcher: offlineUnixFSFetcher,\n\t}\n}\n\n// PathResolversOut allows injection of path resolvers\ntype PathResolversOut struct {\n\tfx.Out\n\tIPLDPathResolver pathresolver.Resolver `name:\"ipldPathResolver\"`\n\tUnixFSPathResolver pathresolver.Resolver `name:\"unixFSPathResolver\"`\n\tOfflineIPLDPathResolver pathresolver.Resolver `name:\"offlineIpldPathResolver\"`\n\tOfflineUnixFSPathResolver pathresolver.Resolver `name:\"offlineUnixFSPathResolver\"`\n}\n\n// PathResolverConfig creates path resolvers with the given fetchers.\nfunc PathResolverConfig(fetchers FetchersIn) PathResolversOut {\n\treturn PathResolversOut{\n\t\tIPLDPathResolver: pathresolver.NewBasicResolver(fetchers.IPLDFetcher),\n\t\tUnixFSPathResolver: pathresolver.NewBasicResolver(fetchers.UnixfsFetcher),\n\t\tOfflineIPLDPathResolver: pathresolver.NewBasicResolver(fetchers.OfflineIPLDFetcher),\n\t\tOfflineUnixFSPathResolver: pathresolver.NewBasicResolver(fetchers.OfflineUnixfsFetcher),\n\t}\n}\n\n// Dag creates new DAGService\nfunc Dag(bs blockservice.BlockService) format.DAGService {\n\treturn merkledag.NewDAGService(bs)\n}\n\n// Files loads persisted MFS root\nfunc Files(strategy string) func(mctx helpers.MetricsCtx, lc fx.Lifecycle, repo repo.Repo, dag format.DAGService, bs blockstore.Blockstore, prov DHTProvider) (*mfs.Root, error) {\n\treturn func(mctx helpers.MetricsCtx, lc fx.Lifecycle, repo repo.Repo, dag format.DAGService, bs blockstore.Blockstore, prov DHTProvider) (*mfs.Root, error) {\n\t\tpf := func(ctx context.Context, c cid.Cid) error {\n\t\t\trootDS := repo.Datastore()\n\t\t\tif err := rootDS.Sync(ctx, blockstore.BlockPrefix); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif err := rootDS.Sync(ctx, filestore.FilestorePrefix); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif err := rootDS.Put(ctx, FilesRootDatastoreKey, c.Bytes()); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn rootDS.Sync(ctx, FilesRootDatastoreKey)\n\t\t}\n\n\t\tvar nd *merkledag.ProtoNode\n\t\tctx := helpers.LifecycleCtx(mctx, lc)\n\t\tval, err := repo.Datastore().Get(ctx, FilesRootDatastoreKey)\n\n\t\tswitch {\n\t\tcase errors.Is(err, datastore.ErrNotFound):\n\t\t\tnd = unixfs.EmptyDirNode()\n\t\t\terr := dag.Add(ctx, nd)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"failure writing filesroot to dagstore: %s\", err)\n\t\t\t}\n\t\tcase err == nil:\n\t\t\tc, err := cid.Cast(val)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tofflineDag := merkledag.NewDAGService(blockservice.New(bs, offline.Exchange(bs)))\n\t\t\trnd, err := offlineDag.Get(ctx, c)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"error loading filesroot from dagservice: %s\", err)\n\t\t\t}\n\n\t\t\tpbnd, ok := rnd.(*merkledag.ProtoNode)\n\t\t\tif !ok {\n\t\t\t\treturn nil, merkledag.ErrNotProtobuf\n\t\t\t}\n\n\t\t\tnd = pbnd\n\t\tdefault:\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// MFS (Mutable File System) provider integration: Only pass the provider\n\t\t// to MFS when the strategy includes \"mfs\". MFS will call StartProviding()\n\t\t// on every DAGService.Add() operation, which is sufficient for the \"mfs\"\n\t\t// strategy - it ensures all MFS content gets announced as it's added or\n\t\t// modified. For non-mfs strategies, we set provider to nil to avoid\n\t\t// unnecessary providing.\n\t\tstrategyFlag := config.ParseProvideStrategy(strategy)\n\t\tif strategyFlag&config.ProvideStrategyMFS == 0 {\n\t\t\tprov = nil\n\t\t}\n\n\t\t// Get configured settings from Import config\n\t\tcfg, err := repo.Config()\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to get config: %w\", err)\n\t\t}\n\t\tchunkerGen := cfg.Import.UnixFSSplitterFunc()\n\t\tmaxDirLinks := int(cfg.Import.UnixFSDirectoryMaxLinks.WithDefault(config.DefaultUnixFSDirectoryMaxLinks))\n\t\tmaxHAMTFanout := int(cfg.Import.UnixFSHAMTDirectoryMaxFanout.WithDefault(config.DefaultUnixFSHAMTDirectoryMaxFanout))\n\t\thamtShardingSize := int(cfg.Import.UnixFSHAMTDirectorySizeThreshold.WithDefault(config.DefaultUnixFSHAMTDirectorySizeThreshold))\n\t\tsizeEstimationMode := cfg.Import.HAMTSizeEstimationMode()\n\n\t\troot, err := mfs.NewRoot(ctx, dag, nd, pf, prov,\n\t\t\tmfs.WithChunker(chunkerGen),\n\t\t\tmfs.WithMaxLinks(maxDirLinks),\n\t\t\tmfs.WithMaxHAMTFanout(maxHAMTFanout),\n\t\t\tmfs.WithHAMTShardingSize(hamtShardingSize),\n\t\t\tmfs.WithSizeEstimationMode(sizeEstimationMode),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to initialize MFS root from %s stored at %s: %w. \"+\n\t\t\t\t\"If corrupted, use 'ipfs files chroot' to reset (see --help)\", nd.Cid(), FilesRootDatastoreKey, err)\n\t\t}\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\treturn root.Close()\n\t\t\t},\n\t\t})\n\n\t\treturn root, err\n\t}\n}\n" }, { "path": "core/node/dns.go", "content": "package node\n\nimport (\n\t\"math\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/gateway\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tdoh \"github.com/libp2p/go-doh-resolver\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n)\n\n// Compile-time interface check: *madns.Resolver (returned by gateway.NewDNSResolver\n// and madns.NewResolver) must implement madns.BasicResolver for p2pForgeResolver fallback.\nvar _ madns.BasicResolver = (*madns.Resolver)(nil)\n\nfunc DNSResolver(cfg *config.Config) (*madns.Resolver, error) {\n\tvar dohOpts []doh.Option\n\tif !cfg.DNS.MaxCacheTTL.IsDefault() {\n\t\tdohOpts = append(dohOpts, doh.WithMaxCacheTTL(cfg.DNS.MaxCacheTTL.WithDefault(time.Duration(math.MaxUint32)*time.Second)))\n\t}\n\n\t// Replace \"auto\" DNS resolver placeholders with autoconf values\n\tresolvers := cfg.DNSResolversWithAutoConf()\n\n\t// Get base resolver from boxo (handles custom DoH resolvers per eTLD)\n\tbaseResolver, err := gateway.NewDNSResolver(resolvers, dohOpts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Check if we should skip network DNS lookups for p2p-forge domains\n\tskipAutoTLSDNS := cfg.AutoTLS.SkipDNSLookup.WithDefault(config.DefaultAutoTLSSkipDNSLookup)\n\tif !skipAutoTLSDNS {\n\t\t// Local resolution disabled, use network DNS for everything\n\t\treturn baseResolver, nil\n\t}\n\n\t// Build list of p2p-forge domains to resolve locally without network I/O.\n\t// AutoTLS hostnames encode IP addresses directly (e.g., 1-2-3-4.peerID.libp2p.direct),\n\t// so DNS lookups are wasteful. We resolve these in-memory when possible.\n\tforgeDomains := []string{config.DefaultDomainSuffix}\n\tcustomDomain := cfg.AutoTLS.DomainSuffix.WithDefault(config.DefaultDomainSuffix)\n\tif customDomain != config.DefaultDomainSuffix {\n\t\tforgeDomains = append(forgeDomains, customDomain)\n\t}\n\tforgeResolver := NewP2PForgeResolver(forgeDomains, baseResolver)\n\n\t// Register p2p-forge resolver for each domain, fallback to baseResolver for others\n\topts := []madns.Option{madns.WithDefaultResolver(baseResolver)}\n\tfor _, domain := range forgeDomains {\n\t\topts = append(opts, madns.WithDomainResolver(domain+\".\", forgeResolver))\n\t}\n\n\treturn madns.NewResolver(opts...)\n}\n" }, { "path": "core/node/groups.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"regexp\"\n\t\"strings\"\n\t\"time\"\n\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\tutil \"github.com/ipfs/boxo/util\"\n\t\"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/p2p\"\n\tpubsub \"github.com/libp2p/go-libp2p-pubsub\"\n\t\"github.com/libp2p/go-libp2p-pubsub/timecache\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\t\"go.uber.org/fx\"\n)\n\nvar logger = log.Logger(\"core:constructor\")\n\nvar BaseLibP2P = fx.Options(\n\tfx.Provide(libp2p.PNet),\n\tfx.Provide(libp2p.ConnectionManager),\n\tfx.Provide(libp2p.Host),\n\tfx.Provide(libp2p.MultiaddrResolver),\n\n\tfx.Provide(libp2p.DiscoveryHandler),\n\n\tfx.Invoke(libp2p.PNetChecker),\n)\n\nfunc LibP2P(bcfg *BuildCfg, cfg *config.Config, userResourceOverrides rcmgr.PartialLimitConfig) fx.Option {\n\tvar connmgr fx.Option\n\n\t// set connmgr based on Swarm.ConnMgr.Type\n\tconnMgrType := cfg.Swarm.ConnMgr.Type.WithDefault(config.DefaultConnMgrType)\n\tswitch connMgrType {\n\tcase \"none\":\n\t\tconnmgr = fx.Options() // noop\n\tcase \"\", \"basic\":\n\t\tgrace := cfg.Swarm.ConnMgr.GracePeriod.WithDefault(config.DefaultConnMgrGracePeriod)\n\t\tlow := int(cfg.Swarm.ConnMgr.LowWater.WithDefault(config.DefaultConnMgrLowWater))\n\t\thigh := int(cfg.Swarm.ConnMgr.HighWater.WithDefault(config.DefaultConnMgrHighWater))\n\t\tsilence := cfg.Swarm.ConnMgr.SilencePeriod.WithDefault(config.DefaultConnMgrSilencePeriod)\n\t\tconnmgr = fx.Provide(libp2p.ConnectionManager(low, high, grace, silence))\n\n\tdefault:\n\t\treturn fx.Error(fmt.Errorf(\"unrecognized Swarm.ConnMgr.Type: %q\", connMgrType))\n\t}\n\n\t// parse PubSub config\n\n\tps, disc := fx.Options(), fx.Options()\n\tif bcfg.getOpt(\"pubsub\") || bcfg.getOpt(\"ipnsps\") {\n\t\tdisc = fx.Provide(libp2p.TopicDiscovery())\n\n\t\tvar pubsubOptions []pubsub.Option\n\t\tpubsubOptions = append(\n\t\t\tpubsubOptions,\n\t\t\tpubsub.WithMessageSigning(!cfg.Pubsub.DisableSigning),\n\t\t\tpubsub.WithSeenMessagesTTL(cfg.Pubsub.SeenMessagesTTL.WithDefault(pubsub.TimeCacheDuration)),\n\t\t)\n\n\t\tvar seenMessagesStrategy timecache.Strategy\n\t\tconfigSeenMessagesStrategy := cfg.Pubsub.SeenMessagesStrategy.WithDefault(config.DefaultSeenMessagesStrategy)\n\t\tswitch configSeenMessagesStrategy {\n\t\tcase config.LastSeenMessagesStrategy:\n\t\t\tseenMessagesStrategy = timecache.Strategy_LastSeen\n\t\tcase config.FirstSeenMessagesStrategy:\n\t\t\tseenMessagesStrategy = timecache.Strategy_FirstSeen\n\t\tdefault:\n\t\t\treturn fx.Error(fmt.Errorf(\"unsupported Pubsub.SeenMessagesStrategy %q\", configSeenMessagesStrategy))\n\t\t}\n\t\tpubsubOptions = append(pubsubOptions, pubsub.WithSeenMessagesStrategy(seenMessagesStrategy))\n\n\t\tswitch cfg.Pubsub.Router {\n\t\tcase \"\":\n\t\t\tfallthrough\n\t\tcase \"gossipsub\":\n\t\t\tps = fx.Provide(libp2p.GossipSub(pubsubOptions...))\n\t\tcase \"floodsub\":\n\t\t\tps = fx.Provide(libp2p.FloodSub(pubsubOptions...))\n\t\tdefault:\n\t\t\treturn fx.Error(fmt.Errorf(\"unknown pubsub router %s\", cfg.Pubsub.Router))\n\t\t}\n\t}\n\n\tautonat := fx.Options()\n\n\tswitch cfg.AutoNAT.ServiceMode {\n\tdefault:\n\t\tpanic(\"BUG: unhandled autonat service mode\")\n\tcase config.AutoNATServiceDisabled:\n\tcase config.AutoNATServiceUnset:\n\t\t// TODO\n\t\t//\n\t\t// We're enabling the AutoNAT service by default on _all_ nodes\n\t\t// for the moment.\n\t\t//\n\t\t// We should consider disabling it by default if the dht is set\n\t\t// to dhtclient.\n\t\tfallthrough\n\tcase config.AutoNATServiceEnabled:\n\t\tautonat = fx.Provide(libp2p.AutoNATService(cfg.AutoNAT.Throttle, false))\n\tcase config.AutoNATServiceEnabledV1Only:\n\t\tautonat = fx.Provide(libp2p.AutoNATService(cfg.AutoNAT.Throttle, true))\n\t}\n\n\tenableTCPTransport := cfg.Swarm.Transports.Network.TCP.WithDefault(true)\n\tenableWebsocketTransport := cfg.Swarm.Transports.Network.Websocket.WithDefault(true)\n\tenableRelayTransport := cfg.Swarm.Transports.Network.Relay.WithDefault(true) // nolint\n\tenableRelayService := cfg.Swarm.RelayService.Enabled.WithDefault(enableRelayTransport)\n\tenableRelayClient := cfg.Swarm.RelayClient.Enabled.WithDefault(enableRelayTransport)\n\tenableAutoTLS := cfg.AutoTLS.Enabled.WithDefault(config.DefaultAutoTLSEnabled)\n\tenableAutoWSS := cfg.AutoTLS.AutoWSS.WithDefault(config.DefaultAutoWSS)\n\tatlsLog := log.Logger(\"autotls\")\n\n\t// Log error when relay subsystem could not be initialized due to missing dependency\n\tif !enableRelayTransport {\n\t\tif enableRelayService {\n\t\t\tlogger.Fatal(\"Failed to enable `Swarm.RelayService`, it requires `Swarm.Transports.Network.Relay` to be true.\")\n\t\t}\n\t\tif enableRelayClient {\n\t\t\tlogger.Fatal(\"Failed to enable `Swarm.RelayClient`, it requires `Swarm.Transports.Network.Relay` to be true.\")\n\t\t}\n\t}\n\n\tswitch {\n\tcase enableAutoTLS && enableTCPTransport && enableWebsocketTransport:\n\t\t// AutoTLS for Secure WebSockets: ensure WSS listeners are in place (manual or automatic)\n\t\twssWildcard := fmt.Sprintf(\"/tls/sni/*.%s/ws\", cfg.AutoTLS.DomainSuffix.WithDefault(config.DefaultDomainSuffix))\n\t\twssWildcardPresent := false\n\t\tcustomWsPresent := false\n\t\tcustomWsRegex := regexp.MustCompile(`/wss?$`)\n\t\ttcpRegex := regexp.MustCompile(`/tcp/\\d+$`)\n\n\t\t// inspect listeners defined in config at Addresses.Swarm\n\t\tvar tcpListeners []string\n\t\tfor _, listener := range cfg.Addresses.Swarm {\n\t\t\t// detect if user manually added /tls/sni/.../ws listener matching AutoTLS.DomainSuffix\n\t\t\tif strings.Contains(listener, wssWildcard) {\n\t\t\t\tatlsLog.Infof(\"found compatible wildcard listener in Addresses.Swarm. AutoTLS will be used on %s\", listener)\n\t\t\t\twssWildcardPresent = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t\t// detect if user manually added own /ws or /wss listener that is\n\t\t\t// not related to AutoTLS feature\n\t\t\tif customWsRegex.MatchString(listener) {\n\t\t\t\tatlsLog.Infof(\"found custom /ws listener set by user in Addresses.Swarm. AutoTLS will not be used on %s.\", listener)\n\t\t\t\tcustomWsPresent = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t\t// else, remember /tcp listeners that can be reused for /tls/sni/../ws\n\t\t\tif tcpRegex.MatchString(listener) {\n\t\t\t\ttcpListeners = append(tcpListeners, listener)\n\t\t\t}\n\t\t}\n\n\t\t// Append AutoTLS's wildcard listener\n\t\t// if no manual /ws listener was set by the user\n\t\tif enableAutoWSS && !wssWildcardPresent && !customWsPresent {\n\t\t\tif len(tcpListeners) == 0 {\n\t\t\t\tlogger.Error(\"Invalid configuration, AutoTLS will be disabled: AutoTLS.AutoWSS=true requires at least one /tcp listener present in Addresses.Swarm, see https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls\")\n\t\t\t\tenableAutoTLS = false\n\t\t\t}\n\t\t\tfor _, tcpListener := range tcpListeners {\n\t\t\t\twssListener := tcpListener + wssWildcard\n\t\t\t\tcfg.Addresses.Swarm = append(cfg.Addresses.Swarm, wssListener)\n\t\t\t\tatlsLog.Infof(\"appended AutoWSS listener: %s\", wssListener)\n\t\t\t}\n\t\t}\n\n\t\tif !wssWildcardPresent && !enableAutoWSS {\n\t\t\tlogger.Error(fmt.Sprintf(\"Invalid configuration, AutoTLS will be disabled: AutoTLS.Enabled=true requires a /tcp listener ending with %q to be present in Addresses.Swarm or AutoTLS.AutoWSS=true, see https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls\", wssWildcard))\n\t\t\tenableAutoTLS = false\n\t\t}\n\tcase enableAutoTLS && !enableTCPTransport:\n\t\tlogger.Error(\"Invalid configuration: AutoTLS.Enabled=true requires Swarm.Transports.Network.TCP to be true as well. AutoTLS will be disabled.\")\n\t\tenableAutoTLS = false\n\tcase enableAutoTLS && !enableWebsocketTransport:\n\t\tlogger.Error(\"Invalid configuration: AutoTLS.Enabled=true requires Swarm.Transports.Network.Websocket to be true as well. AutoTLS will be disabled.\")\n\t\tenableAutoTLS = false\n\t}\n\n\t// Gather all the options\n\topts := fx.Options(\n\t\tBaseLibP2P,\n\n\t\t// identify's AgentVersion (incl. optional agent-version-suffix)\n\t\tfx.Provide(libp2p.UserAgent()),\n\n\t\t// Services (resource management)\n\t\tfx.Provide(libp2p.ResourceManager(bcfg.Repo.Path(), cfg.Swarm, userResourceOverrides)),\n\t\tmaybeProvide(libp2p.P2PForgeCertMgr(bcfg.Repo.Path(), cfg.AutoTLS, atlsLog), enableAutoTLS),\n\t\tmaybeInvoke(libp2p.StartP2PAutoTLS, enableAutoTLS),\n\t\tfx.Provide(libp2p.AddrFilters(cfg.Swarm.AddrFilters)),\n\t\tfx.Provide(libp2p.AddrsFactory(cfg.Addresses.Announce, cfg.Addresses.AppendAnnounce, cfg.Addresses.NoAnnounce)),\n\t\tfx.Provide(libp2p.SmuxTransport(cfg.Swarm.Transports)),\n\t\tfx.Provide(libp2p.RelayTransport(enableRelayTransport)),\n\t\tfx.Provide(libp2p.RelayService(enableRelayService, cfg.Swarm.RelayService)),\n\t\tfx.Provide(libp2p.Transports(cfg.Swarm.Transports)),\n\t\tfx.Provide(libp2p.ListenOn(cfg.Addresses.Swarm)),\n\t\tfx.Invoke(libp2p.SetupDiscovery(cfg.Discovery.MDNS.Enabled)),\n\t\tfx.Provide(libp2p.ForceReachability(cfg.Internal.Libp2pForceReachability)),\n\t\tfx.Provide(libp2p.HolePunching(cfg.Swarm.EnableHolePunching, enableRelayClient)),\n\n\t\tfx.Provide(libp2p.Security(!bcfg.DisableEncryptedConnections, cfg.Swarm.Transports)),\n\n\t\tfx.Provide(libp2p.Routing),\n\t\tfx.Provide(libp2p.ContentRouting),\n\t\tfx.Provide(libp2p.ContentDiscovery),\n\n\t\tfx.Provide(libp2p.BaseRouting(cfg)),\n\t\tmaybeProvide(libp2p.PubsubRouter, bcfg.getOpt(\"ipnsps\")),\n\n\t\tmaybeProvide(libp2p.BandwidthCounter, !cfg.Swarm.DisableBandwidthMetrics),\n\t\tmaybeProvide(libp2p.NatPortMap, !cfg.Swarm.DisableNatPortMap),\n\t\tlibp2p.MaybeAutoRelay(cfg.Swarm.RelayClient.StaticRelays, cfg.Peering, enableRelayClient),\n\t\tautonat,\n\t\tconnmgr,\n\t\tps,\n\t\tdisc,\n\t)\n\n\treturn opts\n}\n\n// Storage groups units which setup datastore based persistence and blockstore layers\nfunc Storage(bcfg *BuildCfg, cfg *config.Config) fx.Option {\n\tcacheOpts := blockstore.DefaultCacheOpts()\n\tcacheOpts.HasBloomFilterSize = cfg.Datastore.BloomFilterSize\n\tcacheOpts.HasTwoQueueCacheSize = int(cfg.Datastore.BlockKeyCacheSize.WithDefault(config.DefaultBlockKeyCacheSize))\n\tif !bcfg.Permanent {\n\t\tcacheOpts.HasBloomFilterSize = 0\n\t}\n\n\tfinalBstore := fx.Provide(GcBlockstoreCtor)\n\tif cfg.Experimental.FilestoreEnabled || cfg.Experimental.UrlstoreEnabled {\n\t\tfinalBstore = fx.Provide(FilestoreBlockstoreCtor)\n\t}\n\n\treturn fx.Options(\n\t\tfx.Provide(RepoConfig),\n\t\tfx.Provide(Datastore),\n\t\tfx.Provide(BaseBlockstoreCtor(\n\t\t\tcacheOpts,\n\t\t\tcfg.Datastore.HashOnRead,\n\t\t\tcfg.Datastore.WriteThrough.WithDefault(config.DefaultWriteThrough),\n\t\t\tcfg.Provide.Strategy.WithDefault(config.DefaultProvideStrategy),\n\t\t)),\n\t\tfinalBstore,\n\t)\n}\n\n// Identity groups units providing cryptographic identity\nfunc Identity(cfg *config.Config) fx.Option {\n\t// PeerID\n\n\tcid := cfg.Identity.PeerID\n\tif cid == \"\" {\n\t\treturn fx.Error(errors.New(\"identity was not set in config (was 'ipfs init' run?)\"))\n\t}\n\tif len(cid) == 0 {\n\t\treturn fx.Error(errors.New(\"no peer ID in config! (was 'ipfs init' run?)\"))\n\t}\n\n\tid, err := peer.Decode(cid)\n\tif err != nil {\n\t\treturn fx.Error(fmt.Errorf(\"peer ID invalid: %s\", err))\n\t}\n\n\t// Private Key\n\n\tif cfg.Identity.PrivKey == \"\" {\n\t\treturn fx.Options( // No PK (usually in tests)\n\t\t\tfx.Provide(PeerID(id)),\n\t\t\tfx.Provide(libp2p.Peerstore),\n\t\t)\n\t}\n\n\tsk, err := cfg.Identity.DecodePrivateKey(\"passphrase todo!\")\n\tif err != nil {\n\t\treturn fx.Error(err)\n\t}\n\n\treturn fx.Options( // Full identity\n\t\tfx.Provide(PeerID(id)),\n\t\tfx.Provide(PrivateKey(sk)),\n\t\tfx.Provide(libp2p.Peerstore),\n\n\t\tfx.Invoke(libp2p.PstoreAddSelfKeys),\n\t)\n}\n\n// IPNS groups namesys related units\nvar IPNS = fx.Options(\n\tfx.Provide(RecordValidator),\n)\n\n// Online groups online-only units\nfunc Online(bcfg *BuildCfg, cfg *config.Config, userResourceOverrides rcmgr.PartialLimitConfig) fx.Option {\n\t// Namesys params\n\n\tipnsCacheSize := cfg.Ipns.ResolveCacheSize\n\tif ipnsCacheSize == 0 {\n\t\tipnsCacheSize = DefaultIpnsCacheSize\n\t}\n\tif ipnsCacheSize < 0 {\n\t\treturn fx.Error(errors.New(\"cannot specify negative resolve cache size\"))\n\t}\n\n\t// Republisher params\n\n\tvar repubPeriod, recordLifetime time.Duration\n\n\tif cfg.Ipns.RepublishPeriod != \"\" {\n\t\td, err := time.ParseDuration(cfg.Ipns.RepublishPeriod)\n\t\tif err != nil {\n\t\t\treturn fx.Error(fmt.Errorf(\"failure to parse config setting IPNS.RepublishPeriod: %s\", err))\n\t\t}\n\n\t\tif !util.Debug && (d < time.Minute || d > (time.Hour*24)) {\n\t\t\treturn fx.Error(fmt.Errorf(\"config setting IPNS.RepublishPeriod is not between 1min and 1day: %s\", d))\n\t\t}\n\n\t\trepubPeriod = d\n\t}\n\n\tif cfg.Ipns.RecordLifetime != \"\" {\n\t\td, err := time.ParseDuration(cfg.Ipns.RecordLifetime)\n\t\tif err != nil {\n\t\t\treturn fx.Error(fmt.Errorf(\"failure to parse config setting IPNS.RecordLifetime: %s\", err))\n\t\t}\n\n\t\trecordLifetime = d\n\t}\n\n\tisBitswapLibp2pEnabled := cfg.Bitswap.Libp2pEnabled.WithDefault(config.DefaultBitswapLibp2pEnabled)\n\tisBitswapServerEnabled := cfg.Bitswap.ServerEnabled.WithDefault(config.DefaultBitswapServerEnabled)\n\tisHTTPRetrievalEnabled := cfg.HTTPRetrieval.Enabled.WithDefault(config.DefaultHTTPRetrievalEnabled)\n\n\t// The Provide system handles both new CID announcements and periodic re-announcements.\n\t// Disabling is controlled by Provide.Enabled=false or setting Interval to 0.\n\tisProviderEnabled := cfg.Provide.Enabled.WithDefault(config.DefaultProvideEnabled) && cfg.Provide.DHT.Interval.WithDefault(config.DefaultProvideDHTInterval) != 0\n\n\treturn fx.Options(\n\t\tfx.Provide(BitswapOptions(cfg)),\n\t\tfx.Provide(Bitswap(isBitswapServerEnabled, isBitswapLibp2pEnabled, isHTTPRetrievalEnabled)),\n\t\tfx.Provide(OnlineExchange(isBitswapLibp2pEnabled)),\n\t\tfx.Provide(DNSResolver),\n\t\tfx.Provide(Namesys(ipnsCacheSize, cfg.Ipns.MaxCacheTTL.WithDefault(config.DefaultIpnsMaxCacheTTL))),\n\t\tfx.Provide(Peering),\n\t\tPeerWith(cfg.Peering.Peers...),\n\n\t\tfx.Invoke(IpnsRepublisher(repubPeriod, recordLifetime)),\n\n\t\tfx.Provide(p2p.New),\n\n\t\tLibP2P(bcfg, cfg, userResourceOverrides),\n\t\tOnlineProviders(isProviderEnabled, cfg),\n\t)\n}\n\n// Offline groups offline alternatives to Online units\nfunc Offline(cfg *config.Config) fx.Option {\n\treturn fx.Options(\n\t\tfx.Provide(offline.Exchange),\n\t\tfx.Provide(DNSResolver),\n\t\tfx.Provide(Namesys(0, 0)),\n\t\tfx.Provide(libp2p.Routing),\n\t\tfx.Provide(libp2p.ContentRouting),\n\t\tfx.Provide(libp2p.OfflineRouting),\n\t\tfx.Provide(libp2p.ContentDiscovery),\n\t\tOfflineProviders(),\n\t)\n}\n\n// Core groups basic IPFS services\nvar Core = fx.Options(\n\tfx.Provide(Dag),\n\tfx.Provide(FetcherConfig),\n\tfx.Provide(PathResolverConfig),\n)\n\nfunc Networked(bcfg *BuildCfg, cfg *config.Config, userResourceOverrides rcmgr.PartialLimitConfig) fx.Option {\n\tif bcfg.Online {\n\t\treturn Online(bcfg, cfg, userResourceOverrides)\n\t}\n\treturn Offline(cfg)\n}\n\n// IPFS builds a group of fx Options based on the passed BuildCfg\nfunc IPFS(ctx context.Context, bcfg *BuildCfg) fx.Option {\n\tif bcfg == nil {\n\t\tbcfg = new(BuildCfg)\n\t}\n\n\tbcfgOpts, cfg := bcfg.options(ctx)\n\tif cfg == nil {\n\t\treturn bcfgOpts // error\n\t}\n\n\tuserResourceOverrides, err := bcfg.Repo.UserResourceOverrides()\n\tif err != nil {\n\t\treturn fx.Error(err)\n\t}\n\n\t// Migrate users of deprecated Experimental.ShardingEnabled flag\n\tif cfg.Experimental.ShardingEnabled {\n\t\tlogger.Fatal(\"The `Experimental.ShardingEnabled` field is no longer used, please remove it from the config. Use Import.UnixFSHAMTDirectorySizeThreshold instead.\")\n\t}\n\tif !cfg.Internal.UnixFSShardingSizeThreshold.IsDefault() {\n\t\tmsg := \"The `Internal.UnixFSShardingSizeThreshold` field was renamed to `Import.UnixFSHAMTDirectorySizeThreshold`. Please update your config.\\n\"\n\t\tif !cfg.Import.UnixFSHAMTDirectorySizeThreshold.IsDefault() {\n\t\t\tlogger.Fatal(msg) // conflicting values, hard fail\n\t\t}\n\t\tlogger.Error(msg)\n\t\t// Migrate the old OptionalString value to the new OptionalBytes field.\n\t\t// Since OptionalBytes embeds OptionalString, we can construct it directly\n\t\t// with the old value, preserving the user's original string (e.g., \"256KiB\").\n\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = config.OptionalBytes{OptionalString: *cfg.Internal.UnixFSShardingSizeThreshold}\n\t}\n\n\t// Validate Import configuration\n\tif err := config.ValidateImportConfig(&cfg.Import); err != nil {\n\t\treturn fx.Error(err)\n\t}\n\n\t// Validate Provide configuration\n\tif err := config.ValidateProvideConfig(&cfg.Provide); err != nil {\n\t\treturn fx.Error(err)\n\t}\n\n\t// Directory sharding settings from Import config.\n\t// These globals affect both `ipfs add` and MFS (`ipfs files` API).\n\tshardSizeThreshold := cfg.Import.UnixFSHAMTDirectorySizeThreshold.WithDefault(config.DefaultUnixFSHAMTDirectorySizeThreshold)\n\tshardMaxFanout := cfg.Import.UnixFSHAMTDirectoryMaxFanout.WithDefault(config.DefaultUnixFSHAMTDirectoryMaxFanout)\n\tuio.HAMTShardingSize = int(shardSizeThreshold)\n\tuio.DefaultShardWidth = int(shardMaxFanout)\n\tuio.HAMTSizeEstimation = cfg.Import.HAMTSizeEstimationMode()\n\n\tproviderStrategy := cfg.Provide.Strategy.WithDefault(config.DefaultProvideStrategy)\n\n\treturn fx.Options(\n\t\tbcfgOpts,\n\n\t\tStorage(bcfg, cfg),\n\t\tIdentity(cfg),\n\t\tIPNS,\n\t\tNetworked(bcfg, cfg, userResourceOverrides),\n\t\tfx.Provide(BlockService(cfg)),\n\t\tfx.Provide(Pinning(providerStrategy)),\n\t\tfx.Provide(Files(providerStrategy)),\n\t\tCore,\n\t)\n}\n" }, { "path": "core/node/helpers/helpers.go", "content": "package helpers\n\nimport (\n\t\"context\"\n\n\t\"go.uber.org/fx\"\n)\n\ntype MetricsCtx context.Context\n\n// LifecycleCtx creates a context which will be canceled when lifecycle stops\n//\n// This is a hack which we need because most of our services use contexts in a\n// wrong way\nfunc LifecycleCtx(mctx MetricsCtx, lc fx.Lifecycle) context.Context {\n\tctx, cancel := context.WithCancel(mctx)\n\tlc.Append(fx.Hook{\n\t\tOnStop: func(_ context.Context) error {\n\t\t\tcancel()\n\t\t\treturn nil\n\t\t},\n\t})\n\treturn ctx\n}\n" }, { "path": "core/node/helpers.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"errors\"\n\n\t\"go.uber.org/fx\"\n)\n\ntype lcStartStop struct {\n\tfx.In\n\n\tLC fx.Lifecycle\n}\n\n// Append wraps a function into a fx.Hook and appends it to the fx.Lifecycle.\nfunc (lcss *lcStartStop) Append(f func() func()) {\n\t// Hooks are guaranteed to run in sequence. If a hook fails to start, its\n\t// OnStop won't be executed.\n\tvar stopFunc func()\n\n\tlcss.LC.Append(fx.Hook{\n\t\tOnStart: func(ctx context.Context) error {\n\t\t\tif ctx.Err() != nil {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tstopFunc = f()\n\t\t\treturn nil\n\t\t},\n\t\tOnStop: func(ctx context.Context) error {\n\t\t\tif ctx.Err() != nil {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tif stopFunc == nil { // Theoretically this shouldn't ever happen\n\t\t\t\treturn errors.New(\"lcStatStop: stopFunc was nil\")\n\t\t\t}\n\t\t\tstopFunc()\n\t\t\treturn nil\n\t\t},\n\t})\n}\n\nfunc maybeProvide(opt any, enable bool) fx.Option {\n\tif enable {\n\t\treturn fx.Provide(opt)\n\t}\n\treturn fx.Options()\n}\n\n// nolint unused\nfunc maybeInvoke(opt any, enable bool) fx.Option {\n\tif enable {\n\t\treturn fx.Invoke(opt)\n\t}\n\treturn fx.Options()\n}\n" }, { "path": "core/node/identity.go", "content": "package node\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nfunc PeerID(id peer.ID) func() peer.ID {\n\treturn func() peer.ID {\n\t\treturn id\n\t}\n}\n\n// PrivateKey loads the private key from config\nfunc PrivateKey(sk crypto.PrivKey) func(id peer.ID) (crypto.PrivKey, error) {\n\treturn func(id peer.ID) (crypto.PrivKey, error) {\n\t\tid2, err := peer.IDFromPrivateKey(sk)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif id2 != id {\n\t\t\treturn nil, fmt.Errorf(\"private key in config does not match id: %s != %s\", id, id2)\n\t\t}\n\t\treturn sk, nil\n\t}\n}\n" }, { "path": "core/node/ipns.go", "content": "package node\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\tutil \"github.com/ipfs/boxo/util\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/peerstore\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/namesys/republisher\"\n\t\"github.com/ipfs/kubo/repo\"\n\tirouting \"github.com/ipfs/kubo/routing\"\n)\n\nconst DefaultIpnsCacheSize = 128\n\n// RecordValidator provides namesys compatible routing record validator\nfunc RecordValidator(ps peerstore.Peerstore) record.Validator {\n\treturn record.NamespacedValidator{\n\t\t\"pk\": record.PublicKeyValidator{},\n\t\t\"ipns\": ipns.Validator{KeyBook: ps},\n\t}\n}\n\n// Namesys creates new name system\nfunc Namesys(cacheSize int, cacheMaxTTL time.Duration) func(rt irouting.ProvideManyRouter, rslv *madns.Resolver, repo repo.Repo) (namesys.NameSystem, error) {\n\treturn func(rt irouting.ProvideManyRouter, rslv *madns.Resolver, repo repo.Repo) (namesys.NameSystem, error) {\n\t\topts := []namesys.Option{\n\t\t\tnamesys.WithDatastore(repo.Datastore()),\n\t\t\tnamesys.WithDNSResolver(rslv),\n\t\t\tnamesys.WithMaxCacheTTL(cacheMaxTTL),\n\t\t}\n\n\t\tif cacheSize > 0 {\n\t\t\topts = append(opts, namesys.WithCache(cacheSize))\n\t\t}\n\n\t\treturn namesys.NewNameSystem(rt, opts...)\n\t}\n}\n\n// IpnsRepublisher runs new IPNS republisher service\nfunc IpnsRepublisher(repubPeriod time.Duration, recordLifetime time.Duration) func(lcStartStop, namesys.NameSystem, repo.Repo, crypto.PrivKey) error {\n\treturn func(lc lcStartStop, namesys namesys.NameSystem, repo repo.Repo, privKey crypto.PrivKey) error {\n\t\trepub := republisher.NewRepublisher(namesys, repo.Datastore(), privKey, repo.Keystore())\n\n\t\tif repubPeriod != 0 {\n\t\t\tif !util.Debug && (repubPeriod < time.Minute || repubPeriod > (time.Hour*24)) {\n\t\t\t\treturn fmt.Errorf(\"config setting IPNS.RepublishPeriod is not between 1min and 1day: %s\", repubPeriod)\n\t\t\t}\n\n\t\t\trepub.Interval = repubPeriod\n\t\t}\n\n\t\tif recordLifetime != 0 {\n\t\t\trepub.RecordLifetime = recordLifetime\n\t\t}\n\n\t\tlc.Append(repub.Run)\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/node/libp2p/addrs.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\tp2pforge \"github.com/ipshipyard/p2p-forge/client\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\tp2pbhost \"github.com/libp2p/go-libp2p/p2p/host/basic\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmamask \"github.com/whyrusleeping/multiaddr-filter\"\n\n\t\"github.com/caddyserver/certmagic\"\n\t\"go.uber.org/fx\"\n)\n\nfunc AddrFilters(filters []string) func() (*ma.Filters, Libp2pOpts, error) {\n\treturn func() (filter *ma.Filters, opts Libp2pOpts, err error) {\n\t\tfilter = ma.NewFilters()\n\t\topts.Opts = append(opts.Opts, libp2p.ConnectionGater((*filtersConnectionGater)(filter)))\n\t\tfor _, s := range filters {\n\t\t\tf, err := mamask.NewMask(s)\n\t\t\tif err != nil {\n\t\t\t\treturn filter, opts, fmt.Errorf(\"incorrectly formatted address filter in config: %s\", s)\n\t\t\t}\n\t\t\tfilter.AddFilter(*f, ma.ActionDeny)\n\t\t}\n\t\treturn filter, opts, nil\n\t}\n}\n\nfunc makeAddrsFactory(announce []string, appendAnnounce []string, noAnnounce []string) (p2pbhost.AddrsFactory, error) {\n\tvar err error // To assign to the slice in the for loop\n\texisting := make(map[string]bool) // To avoid duplicates\n\n\tannAddrs := make([]ma.Multiaddr, len(announce))\n\tfor i, addr := range announce {\n\t\tannAddrs[i], err = ma.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\texisting[addr] = true\n\t}\n\n\tvar appendAnnAddrs []ma.Multiaddr\n\tfor _, addr := range appendAnnounce {\n\t\tif existing[addr] {\n\t\t\t// skip AppendAnnounce that is on the Announce list already\n\t\t\tcontinue\n\t\t}\n\t\tappendAddr, err := ma.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tappendAnnAddrs = append(appendAnnAddrs, appendAddr)\n\t}\n\n\tfilters := ma.NewFilters()\n\tnoAnnAddrs := map[string]bool{}\n\tfor _, addr := range noAnnounce {\n\t\tf, err := mamask.NewMask(addr)\n\t\tif err == nil {\n\t\t\tfilters.AddFilter(*f, ma.ActionDeny)\n\t\t\tcontinue\n\t\t}\n\t\tmaddr, err := ma.NewMultiaddr(addr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tnoAnnAddrs[string(maddr.Bytes())] = true\n\t}\n\n\treturn func(allAddrs []ma.Multiaddr) []ma.Multiaddr {\n\t\tvar addrs []ma.Multiaddr\n\t\tif len(annAddrs) > 0 {\n\t\t\taddrs = annAddrs\n\t\t} else {\n\t\t\taddrs = allAddrs\n\t\t}\n\t\taddrs = append(addrs, appendAnnAddrs...)\n\n\t\tvar out []ma.Multiaddr\n\t\tfor _, maddr := range addrs {\n\t\t\t// check for exact matches\n\t\t\tok := noAnnAddrs[string(maddr.Bytes())]\n\t\t\t// check for /ipcidr matches\n\t\t\tif !ok && !filters.AddrBlocked(maddr) {\n\t\t\t\tout = append(out, maddr)\n\t\t\t}\n\t\t}\n\t\treturn out\n\t}, nil\n}\n\nfunc AddrsFactory(announce []string, appendAnnounce []string, noAnnounce []string) any {\n\treturn func(params struct {\n\t\tfx.In\n\t\tForgeMgr *p2pforge.P2PForgeCertMgr `optional:\"true\"`\n\t},\n\t) (opts Libp2pOpts, err error) {\n\t\tvar addrsFactory p2pbhost.AddrsFactory\n\t\tannounceAddrsFactory, err := makeAddrsFactory(announce, appendAnnounce, noAnnounce)\n\t\tif err != nil {\n\t\t\treturn opts, err\n\t\t}\n\t\tif params.ForgeMgr == nil {\n\t\t\taddrsFactory = announceAddrsFactory\n\t\t} else {\n\t\t\taddrsFactory = func(multiaddrs []ma.Multiaddr) []ma.Multiaddr {\n\t\t\t\tforgeProcessing := params.ForgeMgr.AddressFactory()(multiaddrs)\n\t\t\t\tannounceProcessing := announceAddrsFactory(forgeProcessing)\n\t\t\t\treturn announceProcessing\n\t\t\t}\n\t\t}\n\t\topts.Opts = append(opts.Opts, libp2p.AddrsFactory(addrsFactory))\n\t\treturn\n\t}\n}\n\nfunc ListenOn(addresses []string) any {\n\treturn func() (opts Libp2pOpts) {\n\t\treturn Libp2pOpts{\n\t\t\tOpts: []libp2p.Option{\n\t\t\t\tlibp2p.ListenAddrStrings(addresses...),\n\t\t\t},\n\t\t}\n\t}\n}\n\nfunc P2PForgeCertMgr(repoPath string, cfg config.AutoTLS, atlsLog *logging.ZapEventLogger) any {\n\treturn func() (*p2pforge.P2PForgeCertMgr, error) {\n\t\tstoragePath := filepath.Join(repoPath, \"p2p-forge-certs\")\n\t\trawLogger := atlsLog.Desugar()\n\n\t\t// TODO: this should not be necessary after\n\t\t// https://github.com/ipshipyard/p2p-forge/pull/42 but keep it here for\n\t\t// now to help tracking down any remaining conditions causing\n\t\t// https://github.com/ipshipyard/p2p-forge/issues/8\n\t\tcertmagic.Default.Logger = rawLogger.Named(\"default_fixme\")\n\t\tcertmagic.DefaultACME.Logger = rawLogger.Named(\"default_acme_client_fixme\")\n\n\t\tregistrationDelay := cfg.RegistrationDelay.WithDefault(config.DefaultAutoTLSRegistrationDelay)\n\t\tif cfg.Enabled == config.True && cfg.RegistrationDelay.IsDefault() {\n\t\t\t// Skip delay if user explicitly enabled AutoTLS.Enabled in config\n\t\t\t// and did not set custom AutoTLS.RegistrationDelay\n\t\t\tregistrationDelay = 0 * time.Second\n\t\t}\n\n\t\tcertStorage := &certmagic.FileStorage{Path: storagePath}\n\t\tcertMgr, err := p2pforge.NewP2PForgeCertMgr(\n\t\t\tp2pforge.WithLogger(rawLogger.Sugar()),\n\t\t\tp2pforge.WithForgeDomain(cfg.DomainSuffix.WithDefault(config.DefaultDomainSuffix)),\n\t\t\tp2pforge.WithForgeRegistrationEndpoint(cfg.RegistrationEndpoint.WithDefault(config.DefaultRegistrationEndpoint)),\n\t\t\tp2pforge.WithRegistrationDelay(registrationDelay),\n\t\t\tp2pforge.WithCAEndpoint(cfg.CAEndpoint.WithDefault(config.DefaultCAEndpoint)),\n\t\t\tp2pforge.WithForgeAuth(cfg.RegistrationToken.WithDefault(os.Getenv(p2pforge.ForgeAuthEnv))),\n\t\t\tp2pforge.WithUserAgent(version.GetUserAgentVersion()),\n\t\t\tp2pforge.WithCertificateStorage(certStorage),\n\t\t\tp2pforge.WithShortForgeAddrs(cfg.ShortAddrs.WithDefault(config.DefaultAutoTLSShortAddrs)),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn certMgr, nil\n\t}\n}\n\nfunc StartP2PAutoTLS(lc fx.Lifecycle, certMgr *p2pforge.P2PForgeCertMgr, h host.Host) {\n\tlc.Append(fx.Hook{\n\t\tOnStart: func(ctx context.Context) error {\n\t\t\tcertMgr.ProvideHost(h)\n\t\t\treturn certMgr.Start()\n\t\t},\n\t\tOnStop: func(ctx context.Context) error {\n\t\t\tcertMgr.Stop()\n\t\t\treturn nil\n\t\t},\n\t})\n}\n" }, { "path": "core/node/libp2p/discovery.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/p2p/discovery/mdns\"\n\n\t\"go.uber.org/fx\"\n\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n)\n\nconst discoveryConnTimeout = time.Second * 30\n\ntype discoveryHandler struct {\n\tctx context.Context\n\thost host.Host\n}\n\nfunc (dh *discoveryHandler) HandlePeerFound(p peer.AddrInfo) {\n\tlog.Info(\"connecting to discovered peer: \", p)\n\tctx, cancel := context.WithTimeout(dh.ctx, discoveryConnTimeout)\n\tdefer cancel()\n\tif err := dh.host.Connect(ctx, p); err != nil {\n\t\tlog.Warnf(\"failed to connect to peer %s found by discovery: %s\", p.ID, err)\n\t}\n}\n\nfunc DiscoveryHandler(mctx helpers.MetricsCtx, lc fx.Lifecycle, host host.Host) *discoveryHandler {\n\treturn &discoveryHandler{\n\t\tctx: helpers.LifecycleCtx(mctx, lc),\n\t\thost: host,\n\t}\n}\n\nfunc SetupDiscovery(useMdns bool) func(helpers.MetricsCtx, fx.Lifecycle, host.Host, *discoveryHandler) error {\n\treturn func(mctx helpers.MetricsCtx, lc fx.Lifecycle, host host.Host, handler *discoveryHandler) error {\n\t\tif useMdns {\n\t\t\tservice := mdns.NewMdnsService(host, mdns.ServiceName, handler)\n\t\t\tif err := service.Start(); err != nil {\n\t\t\t\tlog.Error(\"error starting mdns service: \", err)\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t}\n}\n" }, { "path": "core/node/libp2p/dns.go", "content": "package libp2p\n\nimport (\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/p2p/net/swarm\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n)\n\nfunc MultiaddrResolver(rslv *madns.Resolver) (opts Libp2pOpts, err error) {\n\topts.Opts = append(opts.Opts, libp2p.MultiaddrResolver(swarm.ResolverFromMaDNS{Resolver: rslv}))\n\treturn opts, nil\n}\n" }, { "path": "core/node/libp2p/fd/sys_not_unix.go", "content": "//go:build !linux && !darwin && !windows\n\npackage fd\n\nfunc GetNumFDs() int {\n\treturn 0\n}\n" }, { "path": "core/node/libp2p/fd/sys_unix.go", "content": "//go:build linux || darwin\n\npackage fd\n\nimport (\n\t\"golang.org/x/sys/unix\"\n)\n\nfunc GetNumFDs() int {\n\tvar l unix.Rlimit\n\tif err := unix.Getrlimit(unix.RLIMIT_NOFILE, &l); err != nil {\n\t\treturn 0\n\t}\n\treturn int(l.Cur)\n}\n" }, { "path": "core/node/libp2p/fd/sys_windows.go", "content": "//go:build windows\n\npackage fd\n\nimport (\n\t\"math\"\n)\n\nfunc GetNumFDs() int {\n\treturn math.MaxInt\n}\n" }, { "path": "core/node/libp2p/filters.go", "content": "package libp2p\n\nimport (\n\t\"github.com/libp2p/go-libp2p/core/connmgr\"\n\t\"github.com/libp2p/go-libp2p/core/control\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\n// filtersConnectionGater is an adapter that turns multiaddr.Filter into a\n// connmgr.ConnectionGater.\ntype filtersConnectionGater ma.Filters\n\nvar _ connmgr.ConnectionGater = (*filtersConnectionGater)(nil)\n\nfunc (f *filtersConnectionGater) InterceptAddrDial(_ peer.ID, addr ma.Multiaddr) (allow bool) {\n\treturn !(*ma.Filters)(f).AddrBlocked(addr)\n}\n\nfunc (f *filtersConnectionGater) InterceptPeerDial(p peer.ID) (allow bool) {\n\treturn true\n}\n\nfunc (f *filtersConnectionGater) InterceptAccept(connAddr network.ConnMultiaddrs) (allow bool) {\n\treturn !(*ma.Filters)(f).AddrBlocked(connAddr.RemoteMultiaddr())\n}\n\nfunc (f *filtersConnectionGater) InterceptSecured(_ network.Direction, _ peer.ID, connAddr network.ConnMultiaddrs) (allow bool) {\n\treturn !(*ma.Filters)(f).AddrBlocked(connAddr.RemoteMultiaddr())\n}\n\nfunc (f *filtersConnectionGater) InterceptUpgraded(_ network.Conn) (allow bool, reason control.DisconnectReason) {\n\treturn true, 0\n}\n" }, { "path": "core/node/libp2p/host.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\troutedhost \"github.com/libp2p/go-libp2p/p2p/host/routed\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"go.uber.org/fx\"\n)\n\ntype P2PHostIn struct {\n\tfx.In\n\n\tRepo repo.Repo\n\tValidator record.Validator\n\tHostOption HostOption\n\tRoutingOption RoutingOption\n\tID peer.ID\n\tPeerstore peerstore.Peerstore\n\n\tOpts [][]libp2p.Option `group:\"libp2p\"`\n}\n\ntype P2PHostOut struct {\n\tfx.Out\n\n\tHost host.Host\n\tRouting routing.Routing `name:\"initialrouting\"`\n}\n\nfunc Host(mctx helpers.MetricsCtx, lc fx.Lifecycle, params P2PHostIn) (out P2PHostOut, err error) {\n\topts := []libp2p.Option{libp2p.NoListenAddrs}\n\tfor _, o := range params.Opts {\n\t\topts = append(opts, o...)\n\t}\n\n\tctx := helpers.LifecycleCtx(mctx, lc)\n\tcfg, err := params.Repo.Config()\n\tif err != nil {\n\t\treturn out, err\n\t}\n\t// Use auto-config resolution for actual connectivity\n\tbootstrappers, err := cfg.BootstrapPeersWithAutoConf()\n\tif err != nil {\n\t\treturn out, err\n\t}\n\n\t// Optimistic provide is enabled either via dedicated expierimental flag, or when DHT Provide Sweep is enabled.\n\t// When DHT Provide Sweep is enabled, all provide operations go through the\n\t// `SweepingProvider`, hence the provides don't use the optimistic provide\n\t// logic. Provides use `SweepingProvider.StartProviding()` and not\n\t// `IpfsDHT.Provide()`, which is where the optimistic provide logic is\n\t// implemented. However, `IpfsDHT.Provide()` is used to quickly provide roots\n\t// when user manually adds content with the `--fast-provide` flag enabled. In\n\t// this case we want to use optimistic provide logic to quickly announce the\n\t// content to the network. This should be the only use case of\n\t// `IpfsDHT.Provide()` when DHT Provide Sweep is enabled.\n\toptimisticProvide := cfg.Experimental.OptimisticProvide || cfg.Provide.DHT.SweepEnabled.WithDefault(config.DefaultProvideDHTSweepEnabled)\n\n\troutingOptArgs := RoutingOptionArgs{\n\t\tCtx: ctx,\n\t\tDatastore: params.Repo.Datastore(),\n\t\tValidator: params.Validator,\n\t\tBootstrapPeers: bootstrappers,\n\t\tOptimisticProvide: optimisticProvide,\n\t\tOptimisticProvideJobsPoolSize: cfg.Experimental.OptimisticProvideJobsPoolSize,\n\t\tLoopbackAddressesOnLanDHT: cfg.Routing.LoopbackAddressesOnLanDHT.WithDefault(config.DefaultLoopbackAddressesOnLanDHT),\n\t}\n\topts = append(opts, libp2p.Routing(func(h host.Host) (routing.PeerRouting, error) {\n\t\targs := routingOptArgs\n\t\targs.Host = h\n\t\tr, err := params.RoutingOption(args)\n\t\tout.Routing = r\n\t\treturn r, err\n\t}))\n\n\tout.Host, err = params.HostOption(params.ID, params.Peerstore, opts...)\n\tif err != nil {\n\t\treturn P2PHostOut{}, err\n\t}\n\n\troutingOptArgs.Host = out.Host\n\n\t// this code is necessary just for tests: mock network constructions\n\t// ignore the libp2p constructor options that actually construct the routing!\n\tif out.Routing == nil {\n\t\tr, err := params.RoutingOption(routingOptArgs)\n\t\tif err != nil {\n\t\t\treturn P2PHostOut{}, err\n\t\t}\n\t\tout.Routing = r\n\t\tout.Host = routedhost.Wrap(out.Host, out.Routing)\n\t}\n\n\tlc.Append(fx.Hook{\n\t\tOnStop: func(ctx context.Context) error {\n\t\t\treturn out.Host.Close()\n\t\t},\n\t})\n\n\treturn out, err\n}\n" }, { "path": "core/node/libp2p/hostopt.go", "content": "package libp2p\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/peerstore\"\n)\n\ntype HostOption func(id peer.ID, ps peerstore.Peerstore, options ...libp2p.Option) (host.Host, error)\n\nvar DefaultHostOption HostOption = constructPeerHost\n\n// isolates the complex initialization steps\nfunc constructPeerHost(id peer.ID, ps peerstore.Peerstore, options ...libp2p.Option) (host.Host, error) {\n\tpkey := ps.PrivKey(id)\n\tif pkey == nil {\n\t\treturn nil, fmt.Errorf(\"missing private key for node ID: %s\", id)\n\t}\n\toptions = append([]libp2p.Option{libp2p.Identity(pkey), libp2p.Peerstore(ps)}, options...)\n\treturn libp2p.New(options...)\n}\n" }, { "path": "core/node/libp2p/libp2p.go", "content": "package libp2p\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"time\"\n\n\tversion \"github.com/ipfs/kubo\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/p2p/net/connmgr\"\n\t\"go.uber.org/fx\"\n)\n\nvar log = logging.Logger(\"p2pnode\")\n\ntype Libp2pOpts struct {\n\tfx.Out\n\n\tOpts []libp2p.Option `group:\"libp2p\"`\n}\n\nfunc ConnectionManager(low, high int, grace, silence time.Duration) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\tcm, err := connmgr.NewConnManager(low, high,\n\t\t\tconnmgr.WithGracePeriod(grace),\n\t\t\tconnmgr.WithSilencePeriod(silence),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn opts, err\n\t\t}\n\t\topts.Opts = append(opts.Opts, libp2p.ConnectionManager(cm))\n\t\treturn\n\t}\n}\n\nfunc PstoreAddSelfKeys(id peer.ID, sk crypto.PrivKey, ps peerstore.Peerstore) error {\n\tif err := ps.AddPubKey(id, sk.GetPublic()); err != nil {\n\t\treturn err\n\t}\n\n\treturn ps.AddPrivKey(id, sk)\n}\n\nfunc UserAgent() func() (opts Libp2pOpts, err error) {\n\treturn simpleOpt(libp2p.UserAgent(version.GetUserAgentVersion()))\n}\n\nfunc simpleOpt(opt libp2p.Option) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\topts.Opts = append(opts.Opts, opt)\n\t\treturn\n\t}\n}\n\ntype priorityOption struct {\n\tpriority, defaultPriority config.Priority\n\topt libp2p.Option\n}\n\nfunc prioritizeOptions(opts []priorityOption) libp2p.Option {\n\ttype popt struct {\n\t\tpriority int64 // lower priority values mean higher priority\n\t\topt libp2p.Option\n\t}\n\tenabledOptions := make([]popt, 0, len(opts))\n\tfor _, o := range opts {\n\t\tif prio, ok := o.priority.WithDefault(o.defaultPriority); ok {\n\t\t\tenabledOptions = append(enabledOptions, popt{\n\t\t\t\tpriority: prio,\n\t\t\t\topt: o.opt,\n\t\t\t})\n\t\t}\n\t}\n\tsort.Slice(enabledOptions, func(i, j int) bool {\n\t\treturn enabledOptions[i].priority < enabledOptions[j].priority\n\t})\n\tp2pOpts := make([]libp2p.Option, len(enabledOptions))\n\tfor i, opt := range enabledOptions {\n\t\tp2pOpts[i] = opt.opt\n\t}\n\treturn libp2p.ChainOptions(p2pOpts...)\n}\n\nfunc ForceReachability(val *config.OptionalString) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\tif val.IsDefault() {\n\t\t\treturn\n\t\t}\n\t\tv := val.WithDefault(\"unrecognized\")\n\t\tswitch v {\n\t\tcase \"public\":\n\t\t\topts.Opts = append(opts.Opts, libp2p.ForceReachabilityPublic())\n\t\tcase \"private\":\n\t\t\topts.Opts = append(opts.Opts, libp2p.ForceReachabilityPrivate())\n\t\tdefault:\n\t\t\treturn opts, fmt.Errorf(\"unrecognized reachability option: %s\", v)\n\t\t}\n\t\treturn\n\t}\n}\n" }, { "path": "core/node/libp2p/libp2p_test.go", "content": "package libp2p\n\nimport (\n\t\"fmt\"\n\t\"strconv\"\n\t\"testing\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPrioritize(t *testing.T) {\n\t// The option is encoded into the port number of a TCP multiaddr.\n\t// By extracting the port numbers obtained from the applied option, we can make sure that\n\t// prioritization sorted the options correctly.\n\tnewOption := func(num int) libp2p.Option {\n\t\treturn func(cfg *libp2p.Config) error {\n\t\t\tcfg.ListenAddrs = append(cfg.ListenAddrs, ma.StringCast(fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", num)))\n\t\t\treturn nil\n\t\t}\n\t}\n\n\textractNums := func(cfg *libp2p.Config) []int {\n\t\taddrs := cfg.ListenAddrs\n\t\tnums := make([]int, 0, len(addrs))\n\t\tfor _, addr := range addrs {\n\t\t\t_, comp := ma.SplitLast(addr)\n\t\t\tnum, err := strconv.Atoi(comp.Value())\n\t\t\trequire.NoError(t, err)\n\t\t\tnums = append(nums, num)\n\t\t}\n\t\treturn nums\n\t}\n\n\tt.Run(\"using default priorities\", func(t *testing.T) {\n\t\topts := []priorityOption{\n\t\t\t{defaultPriority: 200, opt: newOption(200)},\n\t\t\t{defaultPriority: 1, opt: newOption(1)},\n\t\t\t{defaultPriority: 300, opt: newOption(300)},\n\t\t}\n\t\tvar cfg libp2p.Config\n\t\trequire.NoError(t, prioritizeOptions(opts)(&cfg))\n\t\trequire.Equal(t, extractNums(&cfg), []int{1, 200, 300})\n\t})\n\n\tt.Run(\"using custom priorities\", func(t *testing.T) {\n\t\topts := []priorityOption{\n\t\t\t{defaultPriority: 200, priority: 1, opt: newOption(1)},\n\t\t\t{defaultPriority: 1, priority: 300, opt: newOption(300)},\n\t\t\t{defaultPriority: 300, priority: 20, opt: newOption(20)},\n\t\t}\n\t\tvar cfg libp2p.Config\n\t\trequire.NoError(t, prioritizeOptions(opts)(&cfg))\n\t\trequire.Equal(t, extractNums(&cfg), []int{1, 20, 300})\n\t})\n}\n" }, { "path": "core/node/libp2p/nat.go", "content": "package libp2p\n\nimport (\n\t\"time\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/libp2p/go-libp2p\"\n)\n\nvar NatPortMap = simpleOpt(libp2p.NATPortMap())\n\nfunc AutoNATService(throttle *config.AutoNATThrottleConfig, v1only bool) func() Libp2pOpts {\n\treturn func() (opts Libp2pOpts) {\n\t\topts.Opts = append(opts.Opts, libp2p.EnableNATService())\n\t\tif throttle != nil {\n\t\t\topts.Opts = append(opts.Opts,\n\t\t\t\tlibp2p.AutoNATServiceRateLimit(\n\t\t\t\t\tthrottle.GlobalLimit,\n\t\t\t\t\tthrottle.PeerLimit,\n\t\t\t\t\tthrottle.Interval.WithDefault(time.Minute),\n\t\t\t\t),\n\t\t\t)\n\t\t}\n\n\t\t// While V1 still exists and V2 rollout is in progress\n\t\t// (https://github.com/ipfs/kubo/issues/10091) we check a flag that\n\t\t// allows users to disable V2 and run V1-only mode\n\t\tif !v1only {\n\t\t\topts.Opts = append(opts.Opts, libp2p.EnableAutoNATv2())\n\t\t}\n\t\treturn opts\n\t}\n}\n" }, { "path": "core/node/libp2p/peerstore.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\n\t\"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem\"\n\t\"go.uber.org/fx\"\n)\n\nfunc Peerstore(lc fx.Lifecycle) (peerstore.Peerstore, error) {\n\tpstore, err := pstoremem.NewPeerstore()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tlc.Append(fx.Hook{\n\t\tOnStop: func(ctx context.Context) error {\n\t\t\treturn pstore.Close()\n\t\t},\n\t})\n\n\treturn pstore, nil\n}\n" }, { "path": "core/node/libp2p/pnet.go", "content": "package libp2p\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/repo\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/pnet\"\n\t\"go.uber.org/fx\"\n\t\"golang.org/x/crypto/salsa20\"\n\t\"golang.org/x/crypto/sha3\"\n)\n\ntype PNetFingerprint []byte\n\nfunc PNet(repo repo.Repo) (opts Libp2pOpts, fp PNetFingerprint, err error) {\n\tswarmkey, err := repo.SwarmKey()\n\tif err != nil || swarmkey == nil {\n\t\treturn opts, nil, err\n\t}\n\n\tpsk, err := pnet.DecodeV1PSK(bytes.NewReader(swarmkey))\n\tif err != nil {\n\t\treturn opts, nil, fmt.Errorf(\"failed to configure private network: %s\", err)\n\t}\n\n\topts.Opts = append(opts.Opts, libp2p.PrivateNetwork(psk))\n\n\treturn opts, pnetFingerprint(psk), nil\n}\n\nfunc PNetChecker(repo repo.Repo, ph host.Host, lc fx.Lifecycle) error {\n\t// TODO: better check?\n\tswarmkey, err := repo.SwarmKey()\n\tif err != nil || swarmkey == nil {\n\t\treturn err\n\t}\n\n\tdone := make(chan struct{})\n\tlc.Append(fx.Hook{\n\t\tOnStart: func(_ context.Context) error {\n\t\t\tgo func() {\n\t\t\t\tt := time.NewTicker(30 * time.Second)\n\t\t\t\tdefer t.Stop()\n\n\t\t\t\t<-t.C // swallow one tick\n\t\t\t\tfor {\n\t\t\t\t\tselect {\n\t\t\t\t\tcase <-t.C:\n\t\t\t\t\t\tif len(ph.Network().Peers()) == 0 {\n\t\t\t\t\t\t\tlog.Warn(\"We are in private network and have no peers.\")\n\t\t\t\t\t\t\tlog.Warn(\"This might be configuration mistake.\")\n\t\t\t\t\t\t}\n\t\t\t\t\tcase <-done:\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}()\n\t\t\treturn nil\n\t\t},\n\t\tOnStop: func(_ context.Context) error {\n\t\t\tclose(done)\n\t\t\treturn nil\n\t\t},\n\t})\n\treturn nil\n}\n\nfunc pnetFingerprint(psk pnet.PSK) []byte {\n\tvar pskArr [32]byte\n\tcopy(pskArr[:], psk)\n\n\tenc := make([]byte, 64)\n\tzeros := make([]byte, 64)\n\tout := make([]byte, 16)\n\n\t// We encrypt data first so we don't feed PSK to hash function.\n\t// Salsa20 function is not reversible thus increasing our security margin.\n\tsalsa20.XORKeyStream(enc, zeros, []byte(\"finprint\"), &pskArr)\n\n\t// Then do Shake-128 hash to reduce its length.\n\t// This way if for some reason Shake is broken and Salsa20 preimage is possible,\n\t// attacker has only half of the bytes necessary to recreate psk.\n\tsha3.ShakeSum128(out, enc)\n\n\treturn out\n}\n" }, { "path": "core/node/libp2p/pubsub.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"log/slog\"\n\n\t\"github.com/ipfs/go-datastore\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tpubsub \"github.com/libp2p/go-libp2p-pubsub\"\n\t\"github.com/libp2p/go-libp2p/core/discovery\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"go.uber.org/fx\"\n\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/repo\"\n)\n\ntype pubsubParams struct {\n\tfx.In\n\n\tRepo repo.Repo\n\tHost host.Host\n\tDiscovery discovery.Discovery\n}\n\nfunc FloodSub(pubsubOptions ...pubsub.Option) any {\n\treturn func(mctx helpers.MetricsCtx, lc fx.Lifecycle, params pubsubParams) (service *pubsub.PubSub, err error) {\n\t\treturn pubsub.NewFloodSub(\n\t\t\thelpers.LifecycleCtx(mctx, lc),\n\t\t\tparams.Host,\n\t\t\tappend(pubsubOptions,\n\t\t\t\tpubsub.WithDiscovery(params.Discovery),\n\t\t\t\tpubsub.WithDefaultValidator(newSeqnoValidator(params.Repo.Datastore())))...,\n\t\t)\n\t}\n}\n\nfunc GossipSub(pubsubOptions ...pubsub.Option) any {\n\treturn func(mctx helpers.MetricsCtx, lc fx.Lifecycle, params pubsubParams) (service *pubsub.PubSub, err error) {\n\t\treturn pubsub.NewGossipSub(\n\t\t\thelpers.LifecycleCtx(mctx, lc),\n\t\t\tparams.Host,\n\t\t\tappend(pubsubOptions,\n\t\t\t\tpubsub.WithDiscovery(params.Discovery),\n\t\t\t\tpubsub.WithFloodPublish(true), // flood own publications to all peers for reliable IPNS delivery\n\t\t\t\tpubsub.WithDefaultValidator(newSeqnoValidator(params.Repo.Datastore())))...,\n\t\t)\n\t}\n}\n\nfunc newSeqnoValidator(ds datastore.Datastore) pubsub.ValidatorEx {\n\treturn pubsub.NewBasicSeqnoValidator(&seqnoStore{ds: ds}, slog.New(logging.SlogHandler()).With(\"logger\", \"pubsub\"))\n}\n\n// SeqnoStorePrefix is the datastore prefix for pubsub seqno validator state.\nconst SeqnoStorePrefix = \"/pubsub/seqno/\"\n\n// seqnoStore implements pubsub.PeerMetadataStore using the repo datastore.\n// It stores the maximum seen sequence number per peer to prevent message\n// cycles when network diameter exceeds the timecache span.\ntype seqnoStore struct {\n\tds datastore.Datastore\n}\n\nvar _ pubsub.PeerMetadataStore = (*seqnoStore)(nil)\n\n// Get returns the stored seqno for a peer, or (nil, nil) if the peer is unknown.\n// Returning (nil, nil) for unknown peers allows BasicSeqnoValidator to accept\n// the first message from any peer.\nfunc (s *seqnoStore) Get(ctx context.Context, p peer.ID) ([]byte, error) {\n\tkey := datastore.NewKey(SeqnoStorePrefix + p.String())\n\tval, err := s.ds.Get(ctx, key)\n\tif errors.Is(err, datastore.ErrNotFound) {\n\t\treturn nil, nil\n\t}\n\treturn val, err\n}\n\n// Put stores the seqno for a peer.\nfunc (s *seqnoStore) Put(ctx context.Context, p peer.ID, val []byte) error {\n\tkey := datastore.NewKey(SeqnoStorePrefix + p.String())\n\treturn s.ds.Put(ctx, key, val)\n}\n" }, { "path": "core/node/libp2p/pubsub_test.go", "content": "package libp2p\n\nimport (\n\t\"encoding/binary\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-datastore\"\n\tsyncds \"github.com/ipfs/go-datastore/sync\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestSeqnoStore tests the seqnoStore implementation which backs the\n// BasicSeqnoValidator. The validator prevents message cycles when network\n// diameter exceeds the timecache span by tracking the maximum sequence number\n// seen from each peer.\nfunc TestSeqnoStore(t *testing.T) {\n\tctx := t.Context()\n\tds := syncds.MutexWrap(datastore.NewMapDatastore())\n\tstore := &seqnoStore{ds: ds}\n\n\tpeerA, err := peer.Decode(\"12D3KooWGC6TvWhfapngX6wvJHMYvKpDMXPb3ZnCZ6dMoaMtimQ5\")\n\trequire.NoError(t, err)\n\tpeerB, err := peer.Decode(\"12D3KooWJRqDKTRjvXeGdUEgwkHNsoghYMBUagNYgLPdA4mqdTeo\")\n\trequire.NoError(t, err)\n\n\t// BasicSeqnoValidator expects Get to return (nil, nil) for unknown peers,\n\t// not an error. This allows the validator to accept the first message from\n\t// any peer without special-casing.\n\tt.Run(\"unknown peer returns nil without error\", func(t *testing.T) {\n\t\tval, err := store.Get(ctx, peerA)\n\t\trequire.NoError(t, err)\n\t\trequire.Nil(t, val, \"unknown peer should return nil, not empty slice\")\n\t})\n\n\t// Verify basic store/retrieve functionality with a sequence number encoded\n\t// as big-endian uint64, matching the format used by BasicSeqnoValidator.\n\tt.Run(\"stores and retrieves seqno\", func(t *testing.T) {\n\t\tseqno := uint64(12345)\n\t\tdata := make([]byte, 8)\n\t\tbinary.BigEndian.PutUint64(data, seqno)\n\n\t\terr := store.Put(ctx, peerA, data)\n\t\trequire.NoError(t, err)\n\n\t\tval, err := store.Get(ctx, peerA)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, seqno, binary.BigEndian.Uint64(val))\n\t})\n\n\t// Each peer must have isolated storage. If peer data leaked between peers,\n\t// the validator would incorrectly reject valid messages or accept replays.\n\tt.Run(\"isolates seqno per peer\", func(t *testing.T) {\n\t\tseqnoA := uint64(100)\n\t\tseqnoB := uint64(200)\n\t\tdataA := make([]byte, 8)\n\t\tdataB := make([]byte, 8)\n\t\tbinary.BigEndian.PutUint64(dataA, seqnoA)\n\t\tbinary.BigEndian.PutUint64(dataB, seqnoB)\n\n\t\terr := store.Put(ctx, peerA, dataA)\n\t\trequire.NoError(t, err)\n\t\terr = store.Put(ctx, peerB, dataB)\n\t\trequire.NoError(t, err)\n\n\t\tvalA, err := store.Get(ctx, peerA)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, seqnoA, binary.BigEndian.Uint64(valA))\n\n\t\tvalB, err := store.Get(ctx, peerB)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, seqnoB, binary.BigEndian.Uint64(valB))\n\t})\n\n\t// The validator updates the stored seqno when accepting messages with\n\t// higher seqnos. This test verifies that updates work correctly.\n\tt.Run(\"updates seqno to higher value\", func(t *testing.T) {\n\t\tseqno1 := uint64(1000)\n\t\tseqno2 := uint64(2000)\n\t\tdata1 := make([]byte, 8)\n\t\tdata2 := make([]byte, 8)\n\t\tbinary.BigEndian.PutUint64(data1, seqno1)\n\t\tbinary.BigEndian.PutUint64(data2, seqno2)\n\n\t\terr := store.Put(ctx, peerA, data1)\n\t\trequire.NoError(t, err)\n\n\t\terr = store.Put(ctx, peerA, data2)\n\t\trequire.NoError(t, err)\n\n\t\tval, err := store.Get(ctx, peerA)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, seqno2, binary.BigEndian.Uint64(val))\n\t})\n\n\t// Verify the datastore key format. This is important for:\n\t// 1. Debugging: operators can inspect/clear pubsub state\n\t// 2. Migrations: future changes need to know the key format\n\tt.Run(\"uses expected datastore key format\", func(t *testing.T) {\n\t\tseqno := uint64(42)\n\t\tdata := make([]byte, 8)\n\t\tbinary.BigEndian.PutUint64(data, seqno)\n\n\t\terr := store.Put(ctx, peerA, data)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify we can read directly from datastore with expected key\n\t\texpectedKey := datastore.NewKey(\"/pubsub/seqno/\" + peerA.String())\n\t\tval, err := ds.Get(ctx, expectedKey)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, seqno, binary.BigEndian.Uint64(val))\n\t})\n\n\t// Verify data persists when creating a new store instance with the same\n\t// underlying datastore. This simulates node restart.\n\tt.Run(\"persists across store instances\", func(t *testing.T) {\n\t\tseqno := uint64(99999)\n\t\tdata := make([]byte, 8)\n\t\tbinary.BigEndian.PutUint64(data, seqno)\n\n\t\terr := store.Put(ctx, peerB, data)\n\t\trequire.NoError(t, err)\n\n\t\t// Create new store instance with same datastore\n\t\tstore2 := &seqnoStore{ds: ds}\n\t\tval, err := store2.Get(ctx, peerB)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, seqno, binary.BigEndian.Uint64(val))\n\t})\n}\n" }, { "path": "core/node/libp2p/rcmgr.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/repo\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\t\"github.com/multiformats/go-multiaddr\"\n\t\"go.uber.org/fx\"\n)\n\nvar rcmgrLogger = logging.Logger(\"rcmgr\")\n\nconst NetLimitTraceFilename = \"rcmgr.json.gz\"\n\nvar ErrNoResourceMgr = errors.New(\"missing ResourceMgr: make sure the daemon is running with Swarm.ResourceMgr.Enabled\")\n\nfunc ResourceManager(repoPath string, cfg config.SwarmConfig, userResourceOverrides rcmgr.PartialLimitConfig) any {\n\treturn func(mctx helpers.MetricsCtx, lc fx.Lifecycle, repo repo.Repo) (network.ResourceManager, Libp2pOpts, error) {\n\t\tvar manager network.ResourceManager\n\t\tvar opts Libp2pOpts\n\n\t\tenabled := cfg.ResourceMgr.Enabled.WithDefault(true)\n\n\t\t// ENV overrides Config (if present)\n\t\tswitch os.Getenv(\"LIBP2P_RCMGR\") {\n\t\tcase \"0\", \"false\":\n\t\t\tenabled = false\n\t\tcase \"1\", \"true\":\n\t\t\tenabled = true\n\t\t}\n\n\t\tif enabled {\n\t\t\tlog.Debug(\"libp2p resource manager is enabled\")\n\n\t\t\tlimitConfig, msg, err := LimitConfig(cfg, userResourceOverrides)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, opts, fmt.Errorf(\"creating final Resource Manager config: %w\", err)\n\t\t\t}\n\n\t\t\tif !isPartialConfigEmpty(userResourceOverrides) {\n\t\t\t\trcmgrLogger.Info(`\nlibp2p-resource-limit-overrides.json has been loaded, \"default\" fields will be\nfilled in with autocomputed defaults.`)\n\t\t\t}\n\n\t\t\t// We want to see this message on startup, that's why we are using fmt instead of log.\n\t\t\trcmgrLogger.Info(msg)\n\n\t\t\tif err := ensureConnMgrMakeSenseVsResourceMgr(limitConfig, cfg); err != nil {\n\t\t\t\treturn nil, opts, err\n\t\t\t}\n\n\t\t\tstr, err := rcmgr.NewStatsTraceReporter()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, opts, err\n\t\t\t}\n\n\t\t\tropts := []rcmgr.Option{\n\t\t\t\trcmgr.WithTraceReporter(str),\n\t\t\t\trcmgr.WithLimitPerSubnet(\n\t\t\t\t\tnil,\n\t\t\t\t\t[]rcmgr.ConnLimitPerSubnet{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tConnCount: 16,\n\t\t\t\t\t\t\tPrefixLength: 56,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tConnCount: 8 * 16,\n\t\t\t\t\t\t\tPrefixLength: 48,\n\t\t\t\t\t\t},\n\t\t\t\t\t}),\n\t\t\t}\n\n\t\t\tif len(cfg.ResourceMgr.Allowlist) > 0 {\n\t\t\t\tvar mas []multiaddr.Multiaddr\n\t\t\t\tfor _, maStr := range cfg.ResourceMgr.Allowlist {\n\t\t\t\t\tma, err := multiaddr.NewMultiaddr(maStr)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tlog.Errorf(\"failed to parse multiaddr=%v for allowlist, skipping. err=%v\", maStr, err)\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tmas = append(mas, ma)\n\t\t\t\t}\n\t\t\t\tropts = append(ropts, rcmgr.WithAllowlistedMultiaddrs(mas))\n\t\t\t\tlog.Infof(\"Setting allowlist to: %v\", mas)\n\t\t\t}\n\n\t\t\tif os.Getenv(\"LIBP2P_DEBUG_RCMGR\") != \"\" {\n\t\t\t\ttraceFilePath := filepath.Join(repoPath, NetLimitTraceFilename)\n\t\t\t\tropts = append(ropts, rcmgr.WithTrace(traceFilePath))\n\t\t\t}\n\n\t\t\tlimiter := rcmgr.NewFixedLimiter(limitConfig)\n\n\t\t\tmanager, err = rcmgr.NewResourceManager(limiter, ropts...)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, opts, fmt.Errorf(\"creating libp2p resource manager: %w\", err)\n\t\t\t}\n\t\t\tlrm := &loggingResourceManager{\n\t\t\t\tlogger: &logging.Logger(\"resourcemanager\").SugaredLogger,\n\t\t\t\tdelegate: manager,\n\t\t\t}\n\t\t\tlrm.start(helpers.LifecycleCtx(mctx, lc))\n\t\t\tmanager = lrm\n\t\t} else {\n\t\t\trcmgrLogger.Info(\"go-libp2p resource manager protection disabled\")\n\t\t\tmanager = &network.NullResourceManager{}\n\t\t}\n\n\t\topts.Opts = append(opts.Opts, libp2p.ResourceManager(manager))\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(_ context.Context) error {\n\t\t\t\treturn manager.Close()\n\t\t\t},\n\t\t})\n\n\t\treturn manager, opts, nil\n\t}\n}\n\nfunc isPartialConfigEmpty(cfg rcmgr.PartialLimitConfig) bool {\n\tvar emptyResourceConfig rcmgr.ResourceLimits\n\tif cfg.System != emptyResourceConfig ||\n\t\tcfg.Transient != emptyResourceConfig ||\n\t\tcfg.AllowlistedSystem != emptyResourceConfig ||\n\t\tcfg.AllowlistedTransient != emptyResourceConfig ||\n\t\tcfg.ServiceDefault != emptyResourceConfig ||\n\t\tcfg.ServicePeerDefault != emptyResourceConfig ||\n\t\tcfg.ProtocolDefault != emptyResourceConfig ||\n\t\tcfg.ProtocolPeerDefault != emptyResourceConfig ||\n\t\tcfg.PeerDefault != emptyResourceConfig ||\n\t\tcfg.Conn != emptyResourceConfig ||\n\t\tcfg.Stream != emptyResourceConfig {\n\t\treturn false\n\t}\n\tfor _, v := range cfg.Service {\n\t\tif v != emptyResourceConfig {\n\t\t\treturn false\n\t\t}\n\t}\n\tfor _, v := range cfg.ServicePeer {\n\t\tif v != emptyResourceConfig {\n\t\t\treturn false\n\t\t}\n\t}\n\tfor _, v := range cfg.Protocol {\n\t\tif v != emptyResourceConfig {\n\t\t\treturn false\n\t\t}\n\t}\n\tfor _, v := range cfg.ProtocolPeer {\n\t\tif v != emptyResourceConfig {\n\t\t\treturn false\n\t\t}\n\t}\n\tfor _, v := range cfg.Peer {\n\t\tif v != emptyResourceConfig {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\n// LimitConfig returns the union of the Computed Default Limits and the User Supplied Override Limits.\nfunc LimitConfig(cfg config.SwarmConfig, userResourceOverrides rcmgr.PartialLimitConfig) (limitConfig rcmgr.ConcreteLimitConfig, logMessageForStartup string, err error) {\n\tlimitConfig, msg, err := createDefaultLimitConfig(cfg)\n\tif err != nil {\n\t\treturn rcmgr.ConcreteLimitConfig{}, msg, err\n\t}\n\n\t// The logic for defaults and overriding with specified userResourceOverrides\n\t// is documented in docs/libp2p-resource-management.md.\n\t// Any changes here should be reflected there.\n\n\t// This effectively overrides the computed default LimitConfig with any non-\"useDefault\" values from the userResourceOverrides file.\n\t// Because of how how Build works, any rcmgr.Default value in userResourceOverrides\n\t// will be overridden with a computed default value.\n\tlimitConfig = userResourceOverrides.Build(limitConfig)\n\n\treturn limitConfig, msg, nil\n}\n\ntype ResourceLimitsAndUsage struct {\n\t// This is duplicated from rcmgr.ResourceResourceLimits but adding *Usage fields.\n\tMemory rcmgr.LimitVal64\n\tMemoryUsage int64\n\tFD rcmgr.LimitVal\n\tFDUsage int\n\tConns rcmgr.LimitVal\n\tConnsUsage int\n\tConnsInbound rcmgr.LimitVal\n\tConnsInboundUsage int\n\tConnsOutbound rcmgr.LimitVal\n\tConnsOutboundUsage int\n\tStreams rcmgr.LimitVal\n\tStreamsUsage int\n\tStreamsInbound rcmgr.LimitVal\n\tStreamsInboundUsage int\n\tStreamsOutbound rcmgr.LimitVal\n\tStreamsOutboundUsage int\n}\n\nfunc (u ResourceLimitsAndUsage) ToResourceLimits() rcmgr.ResourceLimits {\n\treturn rcmgr.ResourceLimits{\n\t\tMemory: u.Memory,\n\t\tFD: u.FD,\n\t\tConns: u.Conns,\n\t\tConnsInbound: u.ConnsInbound,\n\t\tConnsOutbound: u.ConnsOutbound,\n\t\tStreams: u.Streams,\n\t\tStreamsInbound: u.StreamsInbound,\n\t\tStreamsOutbound: u.StreamsOutbound,\n\t}\n}\n\ntype LimitsConfigAndUsage struct {\n\t// This is duplicated from rcmgr.ResourceManagerStat but using ResourceLimitsAndUsage\n\t// instead of network.ScopeStat.\n\tSystem ResourceLimitsAndUsage\n\tTransient ResourceLimitsAndUsage\n\tServices map[string]ResourceLimitsAndUsage `json:\",omitempty\"`\n\tProtocols map[protocol.ID]ResourceLimitsAndUsage `json:\",omitempty\"`\n\tPeers map[peer.ID]ResourceLimitsAndUsage `json:\",omitempty\"`\n}\n\nfunc (u LimitsConfigAndUsage) MarshalJSON() ([]byte, error) {\n\t// we want to marshal the encoded peer id\n\tencodedPeerMap := make(map[string]ResourceLimitsAndUsage, len(u.Peers))\n\tfor p, v := range u.Peers {\n\t\tencodedPeerMap[p.String()] = v\n\t}\n\n\ttype Alias LimitsConfigAndUsage\n\treturn json.Marshal(&struct {\n\t\t*Alias\n\t\tPeers map[string]ResourceLimitsAndUsage `json:\",omitempty\"`\n\t}{\n\t\tAlias: (*Alias)(&u),\n\t\tPeers: encodedPeerMap,\n\t})\n}\n\nfunc (u LimitsConfigAndUsage) ToPartialLimitConfig() (result rcmgr.PartialLimitConfig) {\n\tresult.System = u.System.ToResourceLimits()\n\tresult.Transient = u.Transient.ToResourceLimits()\n\n\tresult.Service = make(map[string]rcmgr.ResourceLimits, len(u.Services))\n\tfor s, l := range u.Services {\n\t\tresult.Service[s] = l.ToResourceLimits()\n\t}\n\tresult.Protocol = make(map[protocol.ID]rcmgr.ResourceLimits, len(u.Protocols))\n\tfor p, l := range u.Protocols {\n\t\tresult.Protocol[p] = l.ToResourceLimits()\n\t}\n\tresult.Peer = make(map[peer.ID]rcmgr.ResourceLimits, len(u.Peers))\n\tfor p, l := range u.Peers {\n\t\tresult.Peer[p] = l.ToResourceLimits()\n\t}\n\n\treturn\n}\n\nfunc MergeLimitsAndStatsIntoLimitsConfigAndUsage(l rcmgr.ConcreteLimitConfig, stats rcmgr.ResourceManagerStat) LimitsConfigAndUsage {\n\tlimits := l.ToPartialLimitConfig()\n\n\treturn LimitsConfigAndUsage{\n\t\tSystem: mergeResourceLimitsAndScopeStatToResourceLimitsAndUsage(limits.System, stats.System),\n\t\tTransient: mergeResourceLimitsAndScopeStatToResourceLimitsAndUsage(limits.Transient, stats.Transient),\n\t\tServices: mergeLimitsAndStatsMapIntoLimitsConfigAndUsageMap(limits.Service, stats.Services),\n\t\tProtocols: mergeLimitsAndStatsMapIntoLimitsConfigAndUsageMap(limits.Protocol, stats.Protocols),\n\t\tPeers: mergeLimitsAndStatsMapIntoLimitsConfigAndUsageMap(limits.Peer, stats.Peers),\n\t}\n}\n\nfunc mergeLimitsAndStatsMapIntoLimitsConfigAndUsageMap[K comparable](limits map[K]rcmgr.ResourceLimits, stats map[K]network.ScopeStat) map[K]ResourceLimitsAndUsage {\n\tr := make(map[K]ResourceLimitsAndUsage, maxInt(len(limits), len(stats)))\n\tfor p, s := range stats {\n\t\tvar l rcmgr.ResourceLimits\n\t\tif limits != nil {\n\t\t\tif rl, ok := limits[p]; ok {\n\t\t\t\tl = rl\n\t\t\t}\n\t\t}\n\t\tr[p] = mergeResourceLimitsAndScopeStatToResourceLimitsAndUsage(l, s)\n\t}\n\tfor p, s := range limits {\n\t\tif _, ok := stats[p]; ok {\n\t\t\tcontinue // we already processed this element in the loop above\n\t\t}\n\n\t\tr[p] = mergeResourceLimitsAndScopeStatToResourceLimitsAndUsage(s, network.ScopeStat{})\n\t}\n\treturn r\n}\n\nfunc maxInt(x, y int) int {\n\tif x > y {\n\t\treturn x\n\t}\n\treturn y\n}\n\nfunc mergeResourceLimitsAndScopeStatToResourceLimitsAndUsage(rl rcmgr.ResourceLimits, ss network.ScopeStat) ResourceLimitsAndUsage {\n\treturn ResourceLimitsAndUsage{\n\t\tMemory: rl.Memory,\n\t\tMemoryUsage: ss.Memory,\n\t\tFD: rl.FD,\n\t\tFDUsage: ss.NumFD,\n\t\tConns: rl.Conns,\n\t\tConnsUsage: ss.NumConnsOutbound + ss.NumConnsInbound,\n\t\tConnsOutbound: rl.ConnsOutbound,\n\t\tConnsOutboundUsage: ss.NumConnsOutbound,\n\t\tConnsInbound: rl.ConnsInbound,\n\t\tConnsInboundUsage: ss.NumConnsInbound,\n\t\tStreams: rl.Streams,\n\t\tStreamsUsage: ss.NumStreamsOutbound + ss.NumStreamsInbound,\n\t\tStreamsOutbound: rl.StreamsOutbound,\n\t\tStreamsOutboundUsage: ss.NumStreamsOutbound,\n\t\tStreamsInbound: rl.StreamsInbound,\n\t\tStreamsInboundUsage: ss.NumStreamsInbound,\n\t}\n}\n\ntype ResourceInfos []ResourceInfo\n\ntype ResourceInfo struct {\n\tScopeName string\n\tLimitName string\n\tLimitValue rcmgr.LimitVal64\n\tCurrentUsage int64\n}\n\n// LimitConfigsToInfo gets limits and stats and generates a list of scopes and limits to be printed.\nfunc LimitConfigsToInfo(stats LimitsConfigAndUsage) ResourceInfos {\n\tresult := ResourceInfos{}\n\n\tresult = append(result, resourceLimitsAndUsageToResourceInfo(config.ResourceMgrSystemScope, stats.System)...)\n\tresult = append(result, resourceLimitsAndUsageToResourceInfo(config.ResourceMgrTransientScope, stats.Transient)...)\n\n\tfor i, s := range stats.Services {\n\t\tresult = append(result, resourceLimitsAndUsageToResourceInfo(\n\t\t\tconfig.ResourceMgrServiceScopePrefix+i,\n\t\t\ts,\n\t\t)...)\n\t}\n\n\tfor i, p := range stats.Protocols {\n\t\tresult = append(result, resourceLimitsAndUsageToResourceInfo(\n\t\t\tconfig.ResourceMgrProtocolScopePrefix+string(i),\n\t\t\tp,\n\t\t)...)\n\t}\n\n\tfor i, p := range stats.Peers {\n\t\tresult = append(result, resourceLimitsAndUsageToResourceInfo(\n\t\t\tconfig.ResourceMgrPeerScopePrefix+i.String(),\n\t\t\tp,\n\t\t)...)\n\t}\n\n\treturn result\n}\n\nconst (\n\tlimitNameMemory = \"Memory\"\n\tlimitNameFD = \"FD\"\n\tlimitNameConns = \"Conns\"\n\tlimitNameConnsInbound = \"ConnsInbound\"\n\tlimitNameConnsOutbound = \"ConnsOutbound\"\n\tlimitNameStreams = \"Streams\"\n\tlimitNameStreamsInbound = \"StreamsInbound\"\n\tlimitNameStreamsOutbound = \"StreamsOutbound\"\n)\n\nvar limits = []string{\n\tlimitNameMemory,\n\tlimitNameFD,\n\tlimitNameConns,\n\tlimitNameConnsInbound,\n\tlimitNameConnsOutbound,\n\tlimitNameStreams,\n\tlimitNameStreamsInbound,\n\tlimitNameStreamsOutbound,\n}\n\nfunc resourceLimitsAndUsageToResourceInfo(scopeName string, stats ResourceLimitsAndUsage) ResourceInfos {\n\tresult := ResourceInfos{}\n\tfor _, l := range limits {\n\t\tri := ResourceInfo{\n\t\t\tScopeName: scopeName,\n\t\t}\n\t\tswitch l {\n\t\tcase limitNameMemory:\n\t\t\tri.LimitName = limitNameMemory\n\t\t\tri.LimitValue = stats.Memory\n\t\t\tri.CurrentUsage = stats.MemoryUsage\n\t\tcase limitNameFD:\n\t\t\tri.LimitName = limitNameFD\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.FD)\n\t\t\tri.CurrentUsage = int64(stats.FDUsage)\n\t\tcase limitNameConns:\n\t\t\tri.LimitName = limitNameConns\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.Conns)\n\t\t\tri.CurrentUsage = int64(stats.ConnsUsage)\n\t\tcase limitNameConnsInbound:\n\t\t\tri.LimitName = limitNameConnsInbound\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.ConnsInbound)\n\t\t\tri.CurrentUsage = int64(stats.ConnsInboundUsage)\n\t\tcase limitNameConnsOutbound:\n\t\t\tri.LimitName = limitNameConnsOutbound\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.ConnsOutbound)\n\t\t\tri.CurrentUsage = int64(stats.ConnsOutboundUsage)\n\t\tcase limitNameStreams:\n\t\t\tri.LimitName = limitNameStreams\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.Streams)\n\t\t\tri.CurrentUsage = int64(stats.StreamsUsage)\n\t\tcase limitNameStreamsInbound:\n\t\t\tri.LimitName = limitNameStreamsInbound\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.StreamsInbound)\n\t\t\tri.CurrentUsage = int64(stats.StreamsInboundUsage)\n\t\tcase limitNameStreamsOutbound:\n\t\t\tri.LimitName = limitNameStreamsOutbound\n\t\t\tri.LimitValue = rcmgr.LimitVal64(stats.StreamsOutbound)\n\t\t\tri.CurrentUsage = int64(stats.StreamsOutboundUsage)\n\t\t}\n\n\t\tif ri.LimitValue == rcmgr.Unlimited64 || ri.LimitValue == rcmgr.DefaultLimit64 {\n\t\t\t// ignore unlimited and unset limits to remove noise from output.\n\t\t\tcontinue\n\t\t}\n\n\t\tresult = append(result, ri)\n\t}\n\n\treturn result\n}\n\nfunc ensureConnMgrMakeSenseVsResourceMgr(concreteLimits rcmgr.ConcreteLimitConfig, cfg config.SwarmConfig) error {\n\tif cfg.ConnMgr.Type.WithDefault(config.DefaultConnMgrType) == \"none\" || len(cfg.ResourceMgr.Allowlist) != 0 {\n\t\t// no connmgr OR\n\t\t// If an allowlist is set, a user may be enacting some form of DoS defense.\n\t\t// We don't want want to modify the System.ConnsInbound in that case for example\n\t\t// as it may make sense for it to be (and stay) as \"blockAll\"\n\t\t// so that only connections within the allowlist of multiaddrs get established.\n\t\treturn nil\n\t}\n\n\trcm := concreteLimits.ToPartialLimitConfig()\n\n\thighWater := cfg.ConnMgr.HighWater.WithDefault(config.DefaultConnMgrHighWater)\n\tif (rcm.System.Conns > rcmgr.DefaultLimit || rcm.System.Conns == rcmgr.BlockAllLimit) && int64(rcm.System.Conns) <= highWater {\n\t\t// nolint\n\t\treturn fmt.Errorf(`\nUnable to initialize libp2p due to conflicting resource manager limit configuration.\nresource manager System.Conns (%d) must be bigger than ConnMgr.HighWater (%d)\nSee: https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#how-does-the-resource-manager-resourcemgr-relate-to-the-connection-manager-connmgr\n`, rcm.System.Conns, highWater)\n\t}\n\tif (rcm.System.ConnsInbound > rcmgr.DefaultLimit || rcm.System.ConnsInbound == rcmgr.BlockAllLimit) && int64(rcm.System.ConnsInbound) <= highWater {\n\t\t// nolint\n\t\treturn fmt.Errorf(`\nUnable to initialize libp2p due to conflicting resource manager limit configuration.\nresource manager System.ConnsInbound (%d) must be bigger than ConnMgr.HighWater (%d)\nSee: https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#how-does-the-resource-manager-resourcemgr-relate-to-the-connection-manager-connmgr\n`, rcm.System.ConnsInbound, highWater)\n\t}\n\tif (rcm.System.Streams > rcmgr.DefaultLimit || rcm.System.Streams == rcmgr.BlockAllLimit) && int64(rcm.System.Streams) <= highWater {\n\t\t// nolint\n\t\treturn fmt.Errorf(`\nUnable to initialize libp2p due to conflicting resource manager limit configuration.\nresource manager System.Streams (%d) must be bigger than ConnMgr.HighWater (%d)\nSee: https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#how-does-the-resource-manager-resourcemgr-relate-to-the-connection-manager-connmgr\n`, rcm.System.Streams, highWater)\n\t}\n\tif (rcm.System.StreamsInbound > rcmgr.DefaultLimit || rcm.System.StreamsInbound == rcmgr.BlockAllLimit) && int64(rcm.System.StreamsInbound) <= highWater {\n\t\t// nolint\n\t\treturn fmt.Errorf(`\nUnable to initialize libp2p due to conflicting resource manager limit configuration.\nresource manager System.StreamsInbound (%d) must be bigger than ConnMgr.HighWater (%d)\nSee: https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#how-does-the-resource-manager-resourcemgr-relate-to-the-connection-manager-connmgr\n`, rcm.System.StreamsInbound, highWater)\n\t}\n\treturn nil\n}\n" }, { "path": "core/node/libp2p/rcmgr_defaults.go", "content": "package libp2p\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/libp2p/fd\"\n\t\"github.com/libp2p/go-libp2p\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\t\"github.com/pbnjay/memory\"\n)\n\nvar infiniteResourceLimits = rcmgr.InfiniteLimits.ToPartialLimitConfig().System\n\n// This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled\n\n// createDefaultLimitConfig creates LimitConfig to pass to libp2p's resource manager.\n// The defaults follow the documentation in docs/libp2p-resource-management.md.\n// Any changes in the logic here should be reflected there.\nfunc createDefaultLimitConfig(cfg config.SwarmConfig) (limitConfig rcmgr.ConcreteLimitConfig, logMessageForStartup string, err error) {\n\tmaxMemoryDefault := uint64(memory.TotalMemory()) / 2\n\tmaxMemory := cfg.ResourceMgr.MaxMemory.WithDefault(maxMemoryDefault)\n\n\tmaxMemoryMB := maxMemory / (1024 * 1024)\n\tmaxFD := int(cfg.ResourceMgr.MaxFileDescriptors.WithDefault(int64(fd.GetNumFDs()) / 2))\n\n\t// At least as of 2023-01-25, it's possible to open a connection that\n\t// doesn't ask for any memory usage with the libp2p Resource Manager/Accountant\n\t// (see https://github.com/libp2p/go-libp2p/issues/2010#issuecomment-1404280736).\n\t// As a result, we can't currently rely on Memory limits to full protect us.\n\t// Until https://github.com/libp2p/go-libp2p/issues/2010 is addressed,\n\t// we take a proxy now of restricting to 1 inbound connection per MB.\n\t// Note: this is more generous than go-libp2p's default autoscaled limits which do\n\t// 64 connections per 1GB\n\t// (see https://github.com/libp2p/go-libp2p/blob/master/p2p/host/resource-manager/limit_defaults.go#L357 ).\n\tsystemConnsInbound := int(1 * maxMemoryMB)\n\n\tpartialLimits := rcmgr.PartialLimitConfig{\n\t\tSystem: rcmgr.ResourceLimits{\n\t\t\tMemory: rcmgr.LimitVal64(maxMemory),\n\t\t\tFD: rcmgr.LimitVal(maxFD),\n\n\t\t\tConns: rcmgr.Unlimited,\n\t\t\tConnsInbound: rcmgr.LimitVal(systemConnsInbound),\n\t\t\tConnsOutbound: rcmgr.Unlimited,\n\n\t\t\tStreams: rcmgr.Unlimited,\n\t\t\tStreamsOutbound: rcmgr.Unlimited,\n\t\t\tStreamsInbound: rcmgr.Unlimited,\n\t\t},\n\n\t\t// Transient connections won't cause any memory to be accounted for by the resource manager/accountant.\n\t\t// Only established connections do.\n\t\t// As a result, we can't rely on System.Memory to protect us from a bunch of transient connection being opened.\n\t\t// We limit the same values as the System scope, but only allow the Transient scope to take 25% of what is allowed for the System scope.\n\t\tTransient: rcmgr.ResourceLimits{\n\t\t\tMemory: rcmgr.LimitVal64(maxMemory / 4),\n\t\t\tFD: rcmgr.LimitVal(maxFD / 4),\n\n\t\t\tConns: rcmgr.Unlimited,\n\t\t\tConnsInbound: rcmgr.LimitVal(systemConnsInbound / 4),\n\t\t\tConnsOutbound: rcmgr.Unlimited,\n\n\t\t\tStreams: rcmgr.Unlimited,\n\t\t\tStreamsInbound: rcmgr.Unlimited,\n\t\t\tStreamsOutbound: rcmgr.Unlimited,\n\t\t},\n\n\t\t// Lets get out of the way of the allow list functionality.\n\t\t// If someone specified \"Swarm.ResourceMgr.Allowlist\" we should let it go through.\n\t\tAllowlistedSystem: infiniteResourceLimits,\n\n\t\tAllowlistedTransient: infiniteResourceLimits,\n\n\t\t// Keep it simple by not having Service, ServicePeer, Protocol, ProtocolPeer, Conn, or Stream limits.\n\t\tServiceDefault: infiniteResourceLimits,\n\n\t\tServicePeerDefault: infiniteResourceLimits,\n\n\t\tProtocolDefault: infiniteResourceLimits,\n\n\t\tProtocolPeerDefault: infiniteResourceLimits,\n\n\t\tConn: infiniteResourceLimits,\n\n\t\tStream: infiniteResourceLimits,\n\n\t\t// Limit the resources consumed by a peer.\n\t\t// This doesn't protect us against intentional DoS attacks since an attacker can easily spin up multiple peers.\n\t\t// We specify this limit against unintentional DoS attacks (e.g., a peer has a bug and is sending too much traffic intentionally).\n\t\t// In that case we want to keep that peer's resource consumption contained.\n\t\t// To keep this simple, we only constrain inbound connections and streams.\n\t\tPeerDefault: rcmgr.ResourceLimits{\n\t\t\tMemory: rcmgr.Unlimited64,\n\t\t\tFD: rcmgr.Unlimited,\n\t\t\tConns: rcmgr.Unlimited,\n\t\t\tConnsInbound: rcmgr.DefaultLimit,\n\t\t\tConnsOutbound: rcmgr.Unlimited,\n\t\t\tStreams: rcmgr.Unlimited,\n\t\t\tStreamsInbound: rcmgr.DefaultLimit,\n\t\t\tStreamsOutbound: rcmgr.Unlimited,\n\t\t},\n\t}\n\n\tscalingLimitConfig := rcmgr.DefaultLimits\n\tlibp2p.SetDefaultServiceLimits(&scalingLimitConfig)\n\n\t// Anything set above in partialLimits that had a value of rcmgr.DefaultLimit will be overridden.\n\t// Anything in scalingLimitConfig that wasn't defined in partialLimits above will be added (e.g., libp2p's default service limits).\n\tpartialLimits = partialLimits.Build(scalingLimitConfig.Scale(int64(maxMemory), maxFD)).ToPartialLimitConfig()\n\n\t// Simple checks to override autoscaling ensuring limits make sense versus the connmgr values.\n\t// There are ways to break this, but this should catch most problems already.\n\t// We might improve this in the future.\n\t// See: https://github.com/ipfs/kubo/issues/9545\n\tif partialLimits.System.ConnsInbound > rcmgr.DefaultLimit && cfg.ConnMgr.Type.WithDefault(config.DefaultConnMgrType) != \"none\" {\n\t\tmaxInboundConns := int64(partialLimits.System.ConnsInbound)\n\t\tif connmgrHighWaterTimesTwo := cfg.ConnMgr.HighWater.WithDefault(config.DefaultConnMgrHighWater) * 2; maxInboundConns < connmgrHighWaterTimesTwo {\n\t\t\tmaxInboundConns = connmgrHighWaterTimesTwo\n\t\t}\n\n\t\tif maxInboundConns < config.DefaultResourceMgrMinInboundConns {\n\t\t\tmaxInboundConns = config.DefaultResourceMgrMinInboundConns\n\t\t}\n\n\t\t// Scale System.StreamsInbound as well, but use the existing ratio of StreamsInbound to ConnsInbound\n\t\tif partialLimits.System.StreamsInbound > rcmgr.DefaultLimit {\n\t\t\tpartialLimits.System.StreamsInbound = rcmgr.LimitVal(maxInboundConns * int64(partialLimits.System.StreamsInbound) / int64(partialLimits.System.ConnsInbound))\n\t\t}\n\t\tpartialLimits.System.ConnsInbound = rcmgr.LimitVal(maxInboundConns)\n\t}\n\n\tmsg := fmt.Sprintf(`\nComputed default go-libp2p Resource Manager limits based on:\n - 'Swarm.ResourceMgr.MaxMemory': %q\n - 'Swarm.ResourceMgr.MaxFileDescriptors': %d\n\nThese can be inspected with 'ipfs swarm resources'.\n\n`, humanize.Bytes(maxMemory), maxFD)\n\n\t// We already have a complete value thus pass in an empty ConcreteLimitConfig.\n\treturn partialLimits.Build(rcmgr.ConcreteLimitConfig{}), msg, nil\n}\n" }, { "path": "core/node/libp2p/rcmgr_logging.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"net\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\t\"go.uber.org/zap\"\n)\n\ntype loggingResourceManager struct {\n\tlogger *zap.SugaredLogger\n\tdelegate network.ResourceManager\n\tlogInterval time.Duration\n\n\tmut sync.Mutex\n\tlimitExceededErrs map[string]int\n}\n\ntype loggingScope struct {\n\tlogger *zap.SugaredLogger\n\tdelegate network.ResourceScope\n\tcountErrs func(error)\n}\n\nvar (\n\t_ network.ResourceManager = (*loggingResourceManager)(nil)\n\t_ rcmgr.ResourceManagerState = (*loggingResourceManager)(nil)\n)\n\nfunc (n *loggingResourceManager) start(ctx context.Context) {\n\tlogInterval := n.logInterval\n\tif logInterval == 0 {\n\t\tlogInterval = 10 * time.Second\n\t}\n\tticker := time.NewTicker(logInterval)\n\tgo func() {\n\t\tdefer ticker.Stop()\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\t\tn.mut.Lock()\n\t\t\t\terrs := n.limitExceededErrs\n\t\t\t\tn.limitExceededErrs = make(map[string]int)\n\n\t\t\t\tfor e, count := range errs {\n\t\t\t\t\tn.logger.Warnf(\"Protected from exceeding resource limits %d times. libp2p message: %q.\", count, e)\n\t\t\t\t}\n\n\t\t\t\tif len(errs) != 0 {\n\t\t\t\t\tn.logger.Warnf(\"Learn more about potential actions to take at: https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md\")\n\t\t\t\t}\n\n\t\t\t\tn.mut.Unlock()\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n}\n\nfunc (n *loggingResourceManager) countErrs(err error) {\n\tif errors.Is(err, network.ErrResourceLimitExceeded) {\n\t\tn.mut.Lock()\n\t\tif n.limitExceededErrs == nil {\n\t\t\tn.limitExceededErrs = make(map[string]int)\n\t\t}\n\n\t\t// we need to unwrap the error to get the limit scope and the kind of reached limit\n\t\teout := errors.Unwrap(err)\n\t\tif eout != nil {\n\t\t\tn.limitExceededErrs[eout.Error()]++\n\t\t}\n\n\t\tn.mut.Unlock()\n\t}\n}\n\nfunc (n *loggingResourceManager) ViewSystem(f func(network.ResourceScope) error) error {\n\treturn n.delegate.ViewSystem(f)\n}\n\nfunc (n *loggingResourceManager) ViewTransient(f func(network.ResourceScope) error) error {\n\treturn n.delegate.ViewTransient(func(s network.ResourceScope) error {\n\t\treturn f(&loggingScope{logger: n.logger, delegate: s, countErrs: n.countErrs})\n\t})\n}\n\nfunc (n *loggingResourceManager) ViewService(svc string, f func(network.ServiceScope) error) error {\n\treturn n.delegate.ViewService(svc, func(s network.ServiceScope) error {\n\t\treturn f(&loggingScope{logger: n.logger, delegate: s, countErrs: n.countErrs})\n\t})\n}\n\nfunc (n *loggingResourceManager) ViewProtocol(p protocol.ID, f func(network.ProtocolScope) error) error {\n\treturn n.delegate.ViewProtocol(p, func(s network.ProtocolScope) error {\n\t\treturn f(&loggingScope{logger: n.logger, delegate: s, countErrs: n.countErrs})\n\t})\n}\n\nfunc (n *loggingResourceManager) ViewPeer(p peer.ID, f func(network.PeerScope) error) error {\n\treturn n.delegate.ViewPeer(p, func(s network.PeerScope) error {\n\t\treturn f(&loggingScope{logger: n.logger, delegate: s, countErrs: n.countErrs})\n\t})\n}\n\nfunc (n *loggingResourceManager) OpenConnection(dir network.Direction, usefd bool, remote ma.Multiaddr) (network.ConnManagementScope, error) {\n\tconnMgmtScope, err := n.delegate.OpenConnection(dir, usefd, remote)\n\tn.countErrs(err)\n\treturn connMgmtScope, err\n}\n\nfunc (n *loggingResourceManager) OpenStream(p peer.ID, dir network.Direction) (network.StreamManagementScope, error) {\n\tconnMgmtScope, err := n.delegate.OpenStream(p, dir)\n\tn.countErrs(err)\n\treturn connMgmtScope, err\n}\n\nfunc (n *loggingResourceManager) Close() error {\n\treturn n.delegate.Close()\n}\n\nfunc (n *loggingResourceManager) ListServices() []string {\n\trapi, ok := n.delegate.(rcmgr.ResourceManagerState)\n\tif !ok {\n\t\treturn nil\n\t}\n\n\treturn rapi.ListServices()\n}\n\nfunc (n *loggingResourceManager) ListProtocols() []protocol.ID {\n\trapi, ok := n.delegate.(rcmgr.ResourceManagerState)\n\tif !ok {\n\t\treturn nil\n\t}\n\n\treturn rapi.ListProtocols()\n}\n\nfunc (n *loggingResourceManager) ListPeers() []peer.ID {\n\trapi, ok := n.delegate.(rcmgr.ResourceManagerState)\n\tif !ok {\n\t\treturn nil\n\t}\n\n\treturn rapi.ListPeers()\n}\n\nfunc (n *loggingResourceManager) Stat() rcmgr.ResourceManagerStat {\n\trapi, ok := n.delegate.(rcmgr.ResourceManagerState)\n\tif !ok {\n\t\treturn rcmgr.ResourceManagerStat{}\n\t}\n\n\treturn rapi.Stat()\n}\n\nfunc (n *loggingResourceManager) VerifySourceAddress(addr net.Addr) bool {\n\treturn n.delegate.VerifySourceAddress(addr)\n}\n\nfunc (s *loggingScope) ReserveMemory(size int, prio uint8) error {\n\terr := s.delegate.ReserveMemory(size, prio)\n\ts.countErrs(err)\n\treturn err\n}\n\nfunc (s *loggingScope) ReleaseMemory(size int) {\n\ts.delegate.ReleaseMemory(size)\n}\n\nfunc (s *loggingScope) Stat() network.ScopeStat {\n\treturn s.delegate.Stat()\n}\n\nfunc (s *loggingScope) BeginSpan() (network.ResourceScopeSpan, error) {\n\treturn s.delegate.BeginSpan()\n}\n\nfunc (s *loggingScope) Done() {\n\ts.delegate.(network.ResourceScopeSpan).Done()\n}\n\nfunc (s *loggingScope) Name() string {\n\treturn s.delegate.(network.ServiceScope).Name()\n}\n\nfunc (s *loggingScope) Protocol() protocol.ID {\n\treturn s.delegate.(network.ProtocolScope).Protocol()\n}\n\nfunc (s *loggingScope) Peer() peer.ID {\n\treturn s.delegate.(network.PeerScope).Peer()\n}\n\nfunc (s *loggingScope) PeerScope() network.PeerScope {\n\treturn s.delegate.(network.PeerScope)\n}\n\nfunc (s *loggingScope) SetPeer(p peer.ID) error {\n\terr := s.delegate.(network.ConnManagementScope).SetPeer(p)\n\ts.countErrs(err)\n\treturn err\n}\n\nfunc (s *loggingScope) ProtocolScope() network.ProtocolScope {\n\treturn s.delegate.(network.ProtocolScope)\n}\n\nfunc (s *loggingScope) SetProtocol(proto protocol.ID) error {\n\terr := s.delegate.(network.StreamManagementScope).SetProtocol(proto)\n\ts.countErrs(err)\n\treturn err\n}\n\nfunc (s *loggingScope) ServiceScope() network.ServiceScope {\n\treturn s.delegate.(network.ServiceScope)\n}\n\nfunc (s *loggingScope) SetService(srv string) error {\n\terr := s.delegate.(network.StreamManagementScope).SetService(srv)\n\ts.countErrs(err)\n\treturn err\n}\n\nfunc (s *loggingScope) Limit() rcmgr.Limit {\n\treturn s.delegate.(rcmgr.ResourceScopeLimiter).Limit()\n}\n\nfunc (s *loggingScope) SetLimit(limit rcmgr.Limit) {\n\ts.delegate.(rcmgr.ResourceScopeLimiter).SetLimit(limit)\n}\n" }, { "path": "core/node/libp2p/rcmgr_logging_test.go", "content": "package libp2p\n\nimport (\n\t\"testing\"\n\t\"testing/synctest\"\n\t\"time\"\n\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\t\"github.com/stretchr/testify/require\"\n\t\"go.uber.org/zap\"\n\t\"go.uber.org/zap/zaptest/observer\"\n)\n\nfunc TestLoggingResourceManager(t *testing.T) {\n\tsynctest.Test(t, func(t *testing.T) {\n\t\torig := rcmgr.DefaultLimits.AutoScale()\n\t\tlimits := orig.ToPartialLimitConfig()\n\t\tlimits.System.Conns = 1\n\t\tlimits.System.ConnsInbound = 1\n\t\tlimits.System.ConnsOutbound = 1\n\t\tlimiter := rcmgr.NewFixedLimiter(limits.Build(orig))\n\t\trm, err := rcmgr.NewResourceManager(limiter)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer rm.Close()\n\n\t\toCore, oLogs := observer.New(zap.WarnLevel)\n\t\toLogger := zap.New(oCore)\n\t\tlrm := &loggingResourceManager{\n\t\t\tlogger: oLogger.Sugar(),\n\t\t\tdelegate: rm,\n\t\t\tlogInterval: 1 * time.Second,\n\t\t}\n\n\t\t// 2 of these should result in resource limit exceeded errors and subsequent log messages\n\t\tfor range 3 {\n\t\t\t_, _ = lrm.OpenConnection(network.DirInbound, false, ma.StringCast(\"/ip4/127.0.0.1/tcp/1234\"))\n\t\t}\n\n\t\t// run the logger which will write an entry for those errors\n\t\tctx := t.Context()\n\t\tlrm.start(ctx)\n\t\ttime.Sleep(3 * time.Second)\n\n\t\ttimer := time.NewTimer(1 * time.Second)\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-timer.C:\n\t\t\t\tt.Fatalf(\"expected logs never arrived\")\n\t\t\tdefault:\n\t\t\t\tif oLogs.Len() == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\trequire.Equal(t, \"Protected from exceeding resource limits 2 times. libp2p message: \\\"system: cannot reserve inbound connection: resource limit exceeded\\\".\", oLogs.All()[0].Message)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t})\n}\n" }, { "path": "core/node/libp2p/relay.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/p2p/host/autorelay\"\n\t\"github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/relay\"\n\t\"go.uber.org/fx\"\n)\n\nfunc RelayTransport(enableRelay bool) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\tif enableRelay {\n\t\t\topts.Opts = append(opts.Opts, libp2p.EnableRelay())\n\t\t} else {\n\t\t\topts.Opts = append(opts.Opts, libp2p.DisableRelay())\n\t\t}\n\t\treturn\n\t}\n}\n\nfunc RelayService(enable bool, relayOpts config.RelayService) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\tif enable {\n\t\t\tdef := relay.DefaultResources()\n\t\t\t// Real defaults live in go-libp2p.\n\t\t\t// Here we apply any overrides from user config.\n\t\t\topts.Opts = append(opts.Opts, libp2p.EnableRelayService(relay.WithResources(relay.Resources{\n\t\t\t\tLimit: &relay.RelayLimit{\n\t\t\t\t\tData: relayOpts.ConnectionDataLimit.WithDefault(def.Limit.Data),\n\t\t\t\t\tDuration: relayOpts.ConnectionDurationLimit.WithDefault(def.Limit.Duration),\n\t\t\t\t},\n\t\t\t\tMaxCircuits: int(relayOpts.MaxCircuits.WithDefault(int64(def.MaxCircuits))),\n\t\t\t\tBufferSize: int(relayOpts.BufferSize.WithDefault(int64(def.BufferSize))),\n\t\t\t\tReservationTTL: relayOpts.ReservationTTL.WithDefault(def.ReservationTTL),\n\t\t\t\tMaxReservations: int(relayOpts.MaxReservations.WithDefault(int64(def.MaxReservations))),\n\t\t\t\tMaxReservationsPerIP: int(relayOpts.MaxReservationsPerIP.WithDefault(int64(def.MaxReservationsPerIP))),\n\t\t\t\tMaxReservationsPerASN: int(relayOpts.MaxReservationsPerASN.WithDefault(int64(def.MaxReservationsPerASN))),\n\t\t\t})))\n\t\t}\n\t\treturn\n\t}\n}\n\nfunc MaybeAutoRelay(staticRelays []string, cfgPeering config.Peering, enabled bool) fx.Option {\n\tif !enabled {\n\t\treturn fx.Options()\n\t}\n\n\tif len(staticRelays) > 0 {\n\t\treturn fx.Provide(func() (opts Libp2pOpts, err error) {\n\t\t\tif len(staticRelays) > 0 {\n\t\t\t\tstatic := make([]peer.AddrInfo, 0, len(staticRelays))\n\t\t\t\tfor _, s := range staticRelays {\n\t\t\t\t\tvar addr *peer.AddrInfo\n\t\t\t\t\taddr, err = peer.AddrInfoFromString(s)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tstatic = append(static, *addr)\n\t\t\t\t}\n\t\t\t\topts.Opts = append(opts.Opts, libp2p.EnableAutoRelayWithStaticRelays(static))\n\t\t\t}\n\t\t\treturn\n\t\t})\n\t}\n\n\tpeerChan := make(chan peer.AddrInfo)\n\treturn fx.Options(\n\t\t// Provide AutoRelay option\n\t\tfx.Provide(func() (opts Libp2pOpts, err error) {\n\t\t\topts.Opts = append(opts.Opts,\n\t\t\t\tlibp2p.EnableAutoRelayWithPeerSource(\n\t\t\t\t\tfunc(ctx context.Context, numPeers int) <-chan peer.AddrInfo {\n\t\t\t\t\t\t// TODO(9257): make this code smarter (have a state and actually try to grow the search outward) instead of a long running task just polling our K cluster.\n\t\t\t\t\t\tr := make(chan peer.AddrInfo)\n\t\t\t\t\t\tgo func() {\n\t\t\t\t\t\t\tdefer close(r)\n\t\t\t\t\t\t\tfor ; numPeers != 0; numPeers-- {\n\t\t\t\t\t\t\t\tselect {\n\t\t\t\t\t\t\t\tcase v, ok := <-peerChan:\n\t\t\t\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t\tselect {\n\t\t\t\t\t\t\t\t\tcase r <- v:\n\t\t\t\t\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}()\n\t\t\t\t\t\treturn r\n\t\t\t\t\t},\n\t\t\t\t\tautorelay.WithMinInterval(0),\n\t\t\t\t))\n\t\t\treturn\n\t\t}),\n\t\tautoRelayFeeder(cfgPeering, peerChan),\n\t)\n}\n\nfunc HolePunching(flag config.Flag, hasRelayClient bool) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\tif flag.WithDefault(true) {\n\t\t\tif !hasRelayClient {\n\t\t\t\t// If hole punching is explicitly enabled but the relay client is disabled then panic,\n\t\t\t\t// otherwise just silently disable hole punching\n\t\t\t\tif flag != config.Default {\n\t\t\t\t\tlog.Fatal(\"Failed to enable `Swarm.EnableHolePunching`, it requires `Swarm.RelayClient.Enabled` to be true.\")\n\t\t\t\t} else {\n\t\t\t\t\tlog.Info(\"HolePunching has been disabled due to the RelayClient being disabled.\")\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\t\t\topts.Opts = append(opts.Opts, libp2p.EnableHolePunching())\n\t\t}\n\t\treturn\n\t}\n}\n" }, { "path": "core/node/libp2p/routing.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"runtime/debug\"\n\t\"sort\"\n\t\"time\"\n\n\t\"github.com/cenkalti/backoff/v4\"\n\toffroute \"github.com/ipfs/boxo/routing/offline\"\n\tds \"github.com/ipfs/go-datastore\"\n\tdht \"github.com/libp2p/go-libp2p-kad-dht\"\n\tddht \"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\tpubsub \"github.com/libp2p/go-libp2p-pubsub\"\n\tnamesys \"github.com/libp2p/go-libp2p-pubsub-router\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\t\"go.uber.org/fx\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/repo\"\n\tirouting \"github.com/ipfs/kubo/routing\"\n)\n\ntype Router struct {\n\trouting.Routing\n\n\tPriority int // less = more important\n}\n\ntype p2pRouterOut struct {\n\tfx.Out\n\n\tRouter Router `group:\"routers\"`\n}\n\ntype processInitialRoutingIn struct {\n\tfx.In\n\n\tRouter routing.Routing `name:\"initialrouting\"`\n\n\t// For setting up experimental DHT client\n\tHost host.Host\n\tRepo repo.Repo\n\tValidator record.Validator\n}\n\ntype processInitialRoutingOut struct {\n\tfx.Out\n\n\tRouter Router `group:\"routers\"`\n\tContentRouter routing.ContentRouting `group:\"content-routers\"`\n\n\tDHT *ddht.DHT\n\tDHTClient routing.Routing `name:\"dhtc\"`\n}\n\ntype AddrInfoChan chan peer.AddrInfo\n\nfunc BaseRouting(cfg *config.Config) any {\n\treturn func(lc fx.Lifecycle, in processInitialRoutingIn) (out processInitialRoutingOut, err error) {\n\t\tvar dualDHT *ddht.DHT\n\t\tif dht, ok := in.Router.(*ddht.DHT); ok {\n\t\t\tdualDHT = dht\n\n\t\t\tlc.Append(fx.Hook{\n\t\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\t\treturn dualDHT.Close()\n\t\t\t\t},\n\t\t\t})\n\t\t}\n\n\t\tif cr, ok := in.Router.(routinghelpers.ComposableRouter); ok {\n\t\t\tfor _, r := range cr.Routers() {\n\t\t\t\tif dht, ok := r.(*ddht.DHT); ok {\n\t\t\t\t\tdualDHT = dht\n\t\t\t\t\tlc.Append(fx.Hook{\n\t\t\t\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\t\t\t\treturn dualDHT.Close()\n\t\t\t\t\t\t},\n\t\t\t\t\t})\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif dualDHT != nil && cfg.Routing.AcceleratedDHTClient.WithDefault(config.DefaultAcceleratedDHTClient) {\n\t\t\tcfg, err := in.Repo.Config()\n\t\t\tif err != nil {\n\t\t\t\treturn out, err\n\t\t\t}\n\t\t\t// Use auto-config resolution for actual connectivity\n\t\t\tbspeers, err := cfg.BootstrapPeersWithAutoConf()\n\t\t\tif err != nil {\n\t\t\t\treturn out, err\n\t\t\t}\n\n\t\t\tfullRTClient, err := fullrt.NewFullRT(in.Host,\n\t\t\t\tdht.DefaultPrefix,\n\t\t\t\tfullrt.DHTOption(\n\t\t\t\t\tdht.Validator(in.Validator),\n\t\t\t\t\tdht.Datastore(in.Repo.Datastore()),\n\t\t\t\t\tdht.BootstrapPeers(bspeers...),\n\t\t\t\t\tdht.BucketSize(20),\n\t\t\t\t),\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\treturn out, err\n\t\t\t}\n\n\t\t\tlc.Append(fx.Hook{\n\t\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\t\treturn fullRTClient.Close()\n\t\t\t\t},\n\t\t\t})\n\n\t\t\t// we want to also use the default HTTP routers, so wrap the FullRT client\n\t\t\t// in a parallel router that calls them in parallel\n\t\t\taddrFunc := httpRouterAddrFunc(in.Host, cfg.Addresses)\n\t\t\thttpRouters, err := constructDefaultHTTPRouters(cfg, addrFunc)\n\t\t\tif err != nil {\n\t\t\t\treturn out, err\n\t\t\t}\n\t\t\trouters := []*routinghelpers.ParallelRouter{\n\t\t\t\t{Router: fullRTClient, DoNotWaitForSearchValue: true},\n\t\t\t}\n\t\t\trouters = append(routers, httpRouters...)\n\t\t\trouter := routinghelpers.NewComposableParallel(routers)\n\n\t\t\treturn processInitialRoutingOut{\n\t\t\t\tRouter: Router{\n\t\t\t\t\tPriority: 1000,\n\t\t\t\t\tRouting: router,\n\t\t\t\t},\n\t\t\t\tDHT: dualDHT,\n\t\t\t\tDHTClient: fullRTClient,\n\t\t\t\tContentRouter: fullRTClient,\n\t\t\t}, nil\n\t\t}\n\n\t\treturn processInitialRoutingOut{\n\t\t\tRouter: Router{\n\t\t\t\tPriority: 1000,\n\t\t\t\tRouting: in.Router,\n\t\t\t},\n\t\t\tDHT: dualDHT,\n\t\t\tDHTClient: dualDHT,\n\t\t\tContentRouter: in.Router,\n\t\t}, nil\n\t}\n}\n\ntype p2pOnlineContentRoutingIn struct {\n\tfx.In\n\n\tContentRouter []routing.ContentRouting `group:\"content-routers\"`\n}\n\n// ContentRouting will get all routers that can do contentRouting and add them\n// all together using a TieredRouter. It will be used for topic discovery.\nfunc ContentRouting(in p2pOnlineContentRoutingIn) routing.ContentRouting {\n\tvar routers []routing.Routing\n\tfor _, cr := range in.ContentRouter {\n\t\trouters = append(routers,\n\t\t\t&routinghelpers.Compose{\n\t\t\t\tContentRouting: cr,\n\t\t\t},\n\t\t)\n\t}\n\n\treturn routinghelpers.Tiered{\n\t\tRouters: routers,\n\t}\n}\n\n// ContentDiscovery narrows down the given content routing facility so that it\n// only does discovery.\nfunc ContentDiscovery(in irouting.ProvideManyRouter) routing.ContentDiscovery {\n\treturn in\n}\n\ntype p2pOnlineRoutingIn struct {\n\tfx.In\n\n\tRouters []Router `group:\"routers\"`\n\tValidator record.Validator\n}\n\n// Routing will get all routers obtained from different methods (delegated\n// routers, pub-sub, and so on) and add them all together using a ParallelRouter.\nfunc Routing(in p2pOnlineRoutingIn) irouting.ProvideManyRouter {\n\trouters := in.Routers\n\n\tsort.SliceStable(routers, func(i, j int) bool {\n\t\treturn routers[i].Priority < routers[j].Priority\n\t})\n\n\tvar cRouters []*routinghelpers.ParallelRouter\n\tfor _, v := range routers {\n\t\tcRouters = append(cRouters, &routinghelpers.ParallelRouter{\n\t\t\tIgnoreError: true,\n\t\t\tDoNotWaitForSearchValue: true,\n\t\t\tRouter: v.Routing,\n\t\t})\n\t}\n\n\treturn routinghelpers.NewComposableParallel(cRouters)\n}\n\n// OfflineRouting provides a special Router to the routers list when we are\n// creating an offline node.\nfunc OfflineRouting(dstore ds.Datastore, validator record.Validator) p2pRouterOut {\n\treturn p2pRouterOut{\n\t\tRouter: Router{\n\t\t\tRouting: offroute.NewOfflineRouter(dstore, validator),\n\t\t\tPriority: 10000,\n\t\t},\n\t}\n}\n\ntype p2pPSRoutingIn struct {\n\tfx.In\n\n\tValidator record.Validator\n\tHost host.Host\n\tPubSub *pubsub.PubSub `optional:\"true\"`\n}\n\nfunc PubsubRouter(mctx helpers.MetricsCtx, lc fx.Lifecycle, in p2pPSRoutingIn) (p2pRouterOut, *namesys.PubsubValueStore, error) {\n\tpsRouter, err := namesys.NewPubsubValueStore(\n\t\thelpers.LifecycleCtx(mctx, lc),\n\t\tin.Host,\n\t\tin.PubSub,\n\t\tin.Validator,\n\t\tnamesys.WithRebroadcastInterval(time.Minute),\n\t)\n\tif err != nil {\n\t\treturn p2pRouterOut{}, nil, err\n\t}\n\n\treturn p2pRouterOut{\n\t\tRouter: Router{\n\t\t\tRouting: &routinghelpers.Compose{\n\t\t\t\tValueStore: &routinghelpers.LimitedValueStore{\n\t\t\t\t\tValueStore: psRouter,\n\t\t\t\t\tNamespaces: []string{\"ipns\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tPriority: 100,\n\t\t},\n\t}, psRouter, nil\n}\n\nfunc autoRelayFeeder(cfgPeering config.Peering, peerChan chan<- peer.AddrInfo) fx.Option {\n\treturn fx.Invoke(func(lc fx.Lifecycle, h host.Host, dht *ddht.DHT) {\n\t\tctx, cancel := context.WithCancel(context.Background())\n\t\tdone := make(chan struct{})\n\n\t\tdefer func() {\n\t\t\tif r := recover(); r != nil {\n\t\t\t\tfmt.Println(\"Recovering from unexpected error in AutoRelayFeeder:\", r)\n\t\t\t\tdebug.PrintStack()\n\t\t\t}\n\t\t}()\n\t\tgo func() {\n\t\t\tdefer close(done)\n\n\t\t\t// Feed peers more often right after the bootstrap, then backoff\n\t\t\tbo := backoff.NewExponentialBackOff()\n\t\t\tbo.InitialInterval = 15 * time.Second\n\t\t\tbo.Multiplier = 3\n\t\t\tbo.MaxInterval = 1 * time.Hour\n\t\t\tbo.MaxElapsedTime = 0 // never stop\n\t\t\tt := backoff.NewTicker(bo)\n\t\t\tdefer t.Stop()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-t.C:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\t// Always feed trusted IDs (Peering.Peers in the config)\n\t\t\t\tfor _, trustedPeer := range cfgPeering.Peers {\n\t\t\t\t\tif len(trustedPeer.Addrs) == 0 {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tselect {\n\t\t\t\t\tcase peerChan <- trustedPeer:\n\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Additionally, feed closest peers discovered via DHT\n\t\t\t\tif dht != nil {\n\t\t\t\t\tclosestPeers, err := dht.WAN.GetClosestPeers(ctx, h.ID().String())\n\t\t\t\t\tif err == nil {\n\t\t\t\t\t\tfor _, p := range closestPeers {\n\t\t\t\t\t\t\taddrs := h.Peerstore().Addrs(p)\n\t\t\t\t\t\t\tif len(addrs) == 0 {\n\t\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tdhtPeer := peer.AddrInfo{ID: p, Addrs: addrs}\n\t\t\t\t\t\t\tselect {\n\t\t\t\t\t\t\tcase peerChan <- dhtPeer:\n\t\t\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Additionally, feed all connected swarm peers as potential relay candidates.\n\t\t\t\t// This includes peers from HTTP routing, manual swarm connect, mDNS discovery, etc.\n\t\t\t\t// (fixes https://github.com/ipfs/kubo/issues/10899)\n\t\t\t\tconnectedPeers := h.Network().Peers()\n\t\t\t\tfor _, p := range connectedPeers {\n\t\t\t\t\taddrs := h.Peerstore().Addrs(p)\n\t\t\t\t\tif len(addrs) == 0 {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tswarmPeer := peer.AddrInfo{ID: p, Addrs: addrs}\n\t\t\t\t\tselect {\n\t\t\t\t\tcase peerChan <- swarmPeer:\n\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(_ context.Context) error {\n\t\t\t\tcancel()\n\t\t\t\t<-done\n\t\t\t\treturn nil\n\t\t\t},\n\t\t})\n\t})\n}\n" }, { "path": "core/node/libp2p/routingopt.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"slices\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/ipfs/go-datastore\"\n\t\"github.com/ipfs/kubo/config\"\n\tirouting \"github.com/ipfs/kubo/routing\"\n\tdht \"github.com/libp2p/go-libp2p-kad-dht\"\n\tdual \"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\thost \"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\trouting \"github.com/libp2p/go-libp2p/core/routing\"\n\tbasichost \"github.com/libp2p/go-libp2p/p2p/host/basic\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\ntype RoutingOptionArgs struct {\n\tCtx context.Context\n\tHost host.Host\n\tDatastore datastore.Batching\n\tValidator record.Validator\n\tBootstrapPeers []peer.AddrInfo\n\tOptimisticProvide bool\n\tOptimisticProvideJobsPoolSize int\n\tLoopbackAddressesOnLanDHT bool\n}\n\ntype RoutingOption func(args RoutingOptionArgs) (routing.Routing, error)\n\nvar noopRouter = routinghelpers.Null{}\n\n// EndpointSource tracks where a URL came from to determine appropriate capabilities\ntype EndpointSource struct {\n\tURL string\n\tSupportsRead bool // came from DelegatedRoutersWithAutoConf (Read operations)\n\tSupportsWrite bool // came from DelegatedPublishersWithAutoConf (Write operations)\n}\n\n// determineCapabilities determines endpoint capabilities based on URL path and source\nfunc determineCapabilities(endpoint EndpointSource) (string, autoconf.EndpointCapabilities, error) {\n\tparsed, err := autoconf.DetermineKnownCapabilities(endpoint.URL, endpoint.SupportsRead, endpoint.SupportsWrite)\n\tif err != nil {\n\t\tlog.Debugf(\"Skipping endpoint %q: %v\", endpoint.URL, err)\n\t\treturn \"\", autoconf.EndpointCapabilities{}, nil // Return empty caps, not error\n\t}\n\n\treturn parsed.BaseURL, parsed.Capabilities, nil\n}\n\n// collectAllEndpoints gathers URLs from both router and publisher sources\nfunc collectAllEndpoints(cfg *config.Config) []EndpointSource {\n\tvar endpoints []EndpointSource\n\n\t// Get router URLs (Read operations)\n\tvar routerURLs []string\n\tif envRouters := os.Getenv(config.EnvHTTPRouters); envRouters != \"\" {\n\t\t// Use environment variable override if set (space or comma separated)\n\t\tsplitFunc := func(r rune) bool { return r == ',' || r == ' ' }\n\t\trouterURLs = strings.FieldsFunc(envRouters, splitFunc)\n\t\tlog.Warnf(\"Using HTTP routers from %s environment variable instead of config/autoconf: %v\", config.EnvHTTPRouters, routerURLs)\n\t} else {\n\t\t// Use delegated routers from autoconf\n\t\trouterURLs = cfg.DelegatedRoutersWithAutoConf()\n\t\t// No fallback - if autoconf doesn't provide endpoints, use empty list\n\t\t// This exposes any autoconf issues rather than masking them with hardcoded defaults\n\t}\n\n\t// Add router URLs to collection\n\tfor _, url := range routerURLs {\n\t\tendpoints = append(endpoints, EndpointSource{\n\t\t\tURL: url,\n\t\t\tSupportsRead: true,\n\t\t\tSupportsWrite: false,\n\t\t})\n\t}\n\n\t// Get publisher URLs (Write operations)\n\tpublisherURLs := cfg.DelegatedPublishersWithAutoConf()\n\n\t// Add publisher URLs, merging with existing router URLs if they match\n\tfor _, url := range publisherURLs {\n\t\tfound := false\n\t\tfor i, existing := range endpoints {\n\t\t\tif existing.URL == url {\n\t\t\t\tendpoints[i].SupportsWrite = true\n\t\t\t\tfound = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif !found {\n\t\t\tendpoints = append(endpoints, EndpointSource{\n\t\t\t\tURL: url,\n\t\t\t\tSupportsRead: false,\n\t\t\t\tSupportsWrite: true,\n\t\t\t})\n\t\t}\n\t}\n\n\treturn endpoints\n}\n\nfunc constructDefaultHTTPRouters(cfg *config.Config, addrFunc func() []ma.Multiaddr) ([]*routinghelpers.ParallelRouter, error) {\n\tvar routers []*routinghelpers.ParallelRouter\n\thttpRetrievalEnabled := cfg.HTTPRetrieval.Enabled.WithDefault(config.DefaultHTTPRetrievalEnabled)\n\n\t// Collect URLs from both router and publisher sources\n\tendpoints := collectAllEndpoints(cfg)\n\n\t// Group endpoints by origin (base URL) and aggregate capabilities\n\toriginCapabilities := make(map[string]autoconf.EndpointCapabilities)\n\tfor _, endpoint := range endpoints {\n\t\t// Parse endpoint and determine capabilities based on source\n\t\tbaseURL, capabilities, err := determineCapabilities(endpoint)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to parse endpoint %q: %w\", endpoint.URL, err)\n\t\t}\n\n\t\t// Aggregate capabilities for this origin\n\t\texisting := originCapabilities[baseURL]\n\t\texisting.Merge(capabilities)\n\t\toriginCapabilities[baseURL] = existing\n\t}\n\n\t// Create single HTTP router and composer per origin\n\tfor baseURL, capabilities := range originCapabilities {\n\t\t// Construct HTTP router using base URL (without path)\n\t\thttpRouter, err := irouting.ConstructHTTPRouter(baseURL, cfg.Identity.PeerID, addrFunc, cfg.Identity.PrivKey, httpRetrievalEnabled)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Configure router operations based on aggregated capabilities\n\t\t// https://specs.ipfs.tech/routing/http-routing-v1/\n\t\tcomposer := &irouting.Composer{\n\t\t\tGetValueRouter: noopRouter, // Default disabled, enabled below based on capabilities\n\t\t\tPutValueRouter: noopRouter, // Default disabled, enabled below based on capabilities\n\t\t\tProvideRouter: noopRouter, // we don't have spec for sending provides to /routing/v1 (revisit once https://github.com/ipfs/specs/pull/378 or similar is ratified)\n\t\t\tFindPeersRouter: noopRouter, // Default disabled, enabled below based on capabilities\n\t\t\tFindProvidersRouter: noopRouter, // Default disabled, enabled below based on capabilities\n\t\t}\n\n\t\t// Enable specific capabilities\n\t\tif capabilities.IPNSGet {\n\t\t\tcomposer.GetValueRouter = httpRouter // GET /routing/v1/ipns for IPNS resolution\n\t\t}\n\t\tif capabilities.IPNSPut {\n\t\t\tcomposer.PutValueRouter = httpRouter // PUT /routing/v1/ipns for IPNS publishing\n\t\t}\n\t\tif capabilities.Peers {\n\t\t\tcomposer.FindPeersRouter = httpRouter // GET /routing/v1/peers\n\t\t}\n\t\tif capabilities.Providers {\n\t\t\tcomposer.FindProvidersRouter = httpRouter // GET /routing/v1/providers\n\t\t}\n\n\t\t// Handle special cases and backward compatibility\n\t\tif baseURL == config.CidContactRoutingURL {\n\t\t\t// Special-case: cid.contact only supports /routing/v1/providers/cid endpoint\n\t\t\t// Override any capabilities detected from URL path to ensure only providers is enabled\n\t\t\t// TODO: Consider moving this to configuration or removing once cid.contact adds more capabilities\n\t\t\tcomposer.GetValueRouter = noopRouter\n\t\t\tcomposer.PutValueRouter = noopRouter\n\t\t\tcomposer.ProvideRouter = noopRouter\n\t\t\tcomposer.FindPeersRouter = noopRouter\n\t\t\tcomposer.FindProvidersRouter = httpRouter // Only providers supported\n\t\t}\n\n\t\trouters = append(routers, &routinghelpers.ParallelRouter{\n\t\t\tRouter: composer,\n\t\t\tIgnoreError: true, // https://github.com/ipfs/kubo/pull/9475#discussion_r1042507387\n\t\t\tTimeout: 15 * time.Second, // 5x server value from https://github.com/ipfs/kubo/pull/9475#discussion_r1042428529\n\t\t\tDoNotWaitForSearchValue: true,\n\t\t\tExecuteAfter: 0,\n\t\t})\n\t}\n\treturn routers, nil\n}\n\n// ConstructDelegatedOnlyRouting returns routers used when Routing.Type is set to \"delegated\"\n// This provides HTTP-only routing without DHT, using only delegated routers and IPNS publishers.\n// Useful for environments where DHT connectivity is not available or desired\nfunc ConstructDelegatedOnlyRouting(cfg *config.Config) RoutingOption {\n\treturn func(args RoutingOptionArgs) (routing.Routing, error) {\n\t\t// Use only HTTP routers (includes both read and write capabilities) - no DHT\n\t\tvar routers []*routinghelpers.ParallelRouter\n\n\t\t// Add HTTP delegated routers (includes both router and publisher capabilities)\n\t\taddrFunc := httpRouterAddrFunc(args.Host, cfg.Addresses)\n\t\thttpRouters, err := constructDefaultHTTPRouters(cfg, addrFunc)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\trouters = append(routers, httpRouters...)\n\n\t\t// Validate that we have at least one router configured\n\t\tif len(routers) == 0 {\n\t\t\treturn nil, fmt.Errorf(\"no delegated routers or publishers configured for 'delegated' routing mode\")\n\t\t}\n\n\t\trouting := routinghelpers.NewComposableParallel(routers)\n\t\treturn routing, nil\n\t}\n}\n\n// ConstructDefaultRouting returns routers used when Routing.Type is unset or set to \"auto\"\nfunc ConstructDefaultRouting(cfg *config.Config, routingOpt RoutingOption) RoutingOption {\n\treturn func(args RoutingOptionArgs) (routing.Routing, error) {\n\t\t// Defined routers will be queried in parallel (optimizing for response speed)\n\t\t// Different trade-offs can be made by setting Routing.Type = \"custom\" with own Routing.Routers\n\t\tvar routers []*routinghelpers.ParallelRouter\n\n\t\tdhtRouting, err := routingOpt(args)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\trouters = append(routers, &routinghelpers.ParallelRouter{\n\t\t\tRouter: dhtRouting,\n\t\t\tIgnoreError: false,\n\t\t\tDoNotWaitForSearchValue: true,\n\t\t\tExecuteAfter: 0,\n\t\t})\n\n\t\taddrFunc := httpRouterAddrFunc(args.Host, cfg.Addresses)\n\t\thttpRouters, err := constructDefaultHTTPRouters(cfg, addrFunc)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\trouters = append(routers, httpRouters...)\n\n\t\trouting := routinghelpers.NewComposableParallel(routers)\n\t\treturn routing, nil\n\t}\n}\n\n// constructDHTRouting is used when Routing.Type = \"dht\"\nfunc constructDHTRouting(mode dht.ModeOpt) RoutingOption {\n\treturn func(args RoutingOptionArgs) (routing.Routing, error) {\n\t\tdhtOpts := []dht.Option{\n\t\t\tdht.Concurrency(10),\n\t\t\tdht.Mode(mode),\n\t\t\tdht.Datastore(args.Datastore),\n\t\t\tdht.Validator(args.Validator),\n\t\t}\n\t\tif args.OptimisticProvide {\n\t\t\tdhtOpts = append(dhtOpts, dht.EnableOptimisticProvide())\n\t\t}\n\t\tif args.OptimisticProvideJobsPoolSize != 0 {\n\t\t\tdhtOpts = append(dhtOpts, dht.OptimisticProvideJobsPoolSize(args.OptimisticProvideJobsPoolSize))\n\t\t}\n\t\twanOptions := []dht.Option{\n\t\t\tdht.BootstrapPeers(args.BootstrapPeers...),\n\t\t}\n\t\tlanOptions := []dht.Option{}\n\t\tif args.LoopbackAddressesOnLanDHT {\n\t\t\tlanOptions = append(lanOptions, dht.AddressFilter(nil))\n\t\t}\n\t\treturn dual.New(\n\t\t\targs.Ctx, args.Host,\n\t\t\tdual.DHTOption(dhtOpts...),\n\t\t\tdual.WanDHTOption(wanOptions...),\n\t\t\tdual.LanDHTOption(lanOptions...),\n\t\t)\n\t}\n}\n\n// ConstructDelegatedRouting is used when Routing.Type = \"custom\"\nfunc ConstructDelegatedRouting(routers config.Routers, methods config.Methods, peerID string, addrs config.Addresses, privKey string, httpRetrieval bool) RoutingOption {\n\treturn func(args RoutingOptionArgs) (routing.Routing, error) {\n\t\taddrFunc := httpRouterAddrFunc(args.Host, addrs)\n\t\treturn irouting.Parse(routers, methods,\n\t\t\t&irouting.ExtraDHTParams{\n\t\t\t\tBootstrapPeers: args.BootstrapPeers,\n\t\t\t\tHost: args.Host,\n\t\t\t\tValidator: args.Validator,\n\t\t\t\tDatastore: args.Datastore,\n\t\t\t\tContext: args.Ctx,\n\t\t\t},\n\t\t\t&irouting.ExtraHTTPParams{\n\t\t\t\tPeerID: peerID,\n\t\t\t\tAddrFunc: addrFunc,\n\t\t\t\tPrivKeyB64: privKey,\n\t\t\t\tHTTPRetrieval: httpRetrieval,\n\t\t\t},\n\t\t)\n\t}\n}\n\nfunc constructNilRouting(_ RoutingOptionArgs) (routing.Routing, error) {\n\treturn routinghelpers.Null{}, nil\n}\n\nvar (\n\tDHTOption RoutingOption = constructDHTRouting(dht.ModeAuto)\n\tDHTClientOption = constructDHTRouting(dht.ModeClient)\n\tDHTServerOption = constructDHTRouting(dht.ModeServer)\n\tNilRouterOption = constructNilRouting\n)\n\n// confirmedAddrsHost matches libp2p hosts that support AutoNAT V2 address confirmation.\ntype confirmedAddrsHost interface {\n\tConfirmedAddrs() (reachable, unreachable, unknown []ma.Multiaddr)\n}\n\n// Compile-time check: BasicHost must satisfy confirmedAddrsHost.\n// ConfirmedAddrs is not part of the core host.Host interface and is marked\n// experimental in go-libp2p. If BasicHost ever drops or changes this method,\n// this assertion will fail at build time. In that case, update\n// httpRouterAddrFunc (this file) and the swarm autonat command\n// (core/commands/swarm_addrs_autonat.go) which both type-assert to this\n// interface.\nvar _ confirmedAddrsHost = (*basichost.BasicHost)(nil)\n\n// httpRouterAddrFunc returns a function that resolves provider addresses for\n// HTTP routers at provide-time.\n//\n// Resolution logic:\n// - If Announce is set, use it as a static override (no dynamic resolution).\n// - Otherwise, prefer AutoNAT V2 confirmed reachable addresses when available,\n// falling back to static Swarm addresses (filtered by NoAnnounce).\n// - AppendAnnounce addresses are always appended.\nfunc httpRouterAddrFunc(h host.Host, cfgAddrs config.Addresses) func() []ma.Multiaddr {\n\tappendAddrs := parseMultiaddrs(cfgAddrs.AppendAnnounce)\n\n\t// If Announce is explicitly set, use it as a static override.\n\tif len(cfgAddrs.Announce) > 0 {\n\t\tstaticAddrs := slices.Concat(parseMultiaddrs(cfgAddrs.Announce), appendAddrs)\n\t\treturn func() []ma.Multiaddr { return staticAddrs }\n\t}\n\n\t// Precompute fallback: Swarm minus NoAnnounce plus AppendAnnounce.\n\tfallbackStrs := cfgAddrs.Swarm\n\tif len(cfgAddrs.NoAnnounce) > 0 {\n\t\tnoAnnounce := map[string]struct{}{}\n\t\tfor _, a := range cfgAddrs.NoAnnounce {\n\t\t\tnoAnnounce[a] = struct{}{}\n\t\t}\n\t\tfiltered := make([]string, 0, len(fallbackStrs))\n\t\tfor _, a := range fallbackStrs {\n\t\t\tif _, skip := noAnnounce[a]; !skip {\n\t\t\t\tfiltered = append(filtered, a)\n\t\t\t}\n\t\t}\n\t\tfallbackStrs = filtered\n\t}\n\tfallbackResult := slices.Concat(parseMultiaddrs(fallbackStrs), appendAddrs)\n\n\tch, hasConfirmed := h.(confirmedAddrsHost)\n\treturn func() []ma.Multiaddr {\n\t\tif hasConfirmed {\n\t\t\treachable, _, _ := ch.ConfirmedAddrs()\n\t\t\tif len(reachable) > 0 {\n\t\t\t\tif len(appendAddrs) == 0 {\n\t\t\t\t\treturn reachable\n\t\t\t\t}\n\t\t\t\treturn slices.Concat(reachable, appendAddrs)\n\t\t\t}\n\t\t}\n\t\treturn fallbackResult\n\t}\n}\n\nfunc parseMultiaddrs(strs []string) []ma.Multiaddr {\n\taddrs := make([]ma.Multiaddr, 0, len(strs))\n\tfor _, s := range strs {\n\t\ta, err := ma.NewMultiaddr(s)\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"ignoring invalid multiaddr %q: %s\", s, err)\n\t\t\tcontinue\n\t\t}\n\t\taddrs = append(addrs, a)\n\t}\n\treturn addrs\n}\n" }, { "path": "core/node/libp2p/routingopt_test.go", "content": "package libp2p\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/libp2p/go-libp2p/core/connmgr\"\n\t\"github.com/libp2p/go-libp2p/core/event\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestDetermineCapabilities(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tendpoint EndpointSource\n\t\texpectedBaseURL string\n\t\texpectedCapabilities autoconf.EndpointCapabilities\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"URL with no path should have all Read capabilities\",\n\t\t\tendpoint: EndpointSource{\n\t\t\t\tURL: \"https://example.com\",\n\t\t\t\tSupportsRead: true,\n\t\t\t\tSupportsWrite: false,\n\t\t\t},\n\t\t\texpectedBaseURL: \"https://example.com\",\n\t\t\texpectedCapabilities: autoconf.EndpointCapabilities{\n\t\t\t\tProviders: true,\n\t\t\t\tPeers: true,\n\t\t\t\tIPNSGet: true,\n\t\t\t\tIPNSPut: false,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"URL with trailing slash should have all Read capabilities\",\n\t\t\tendpoint: EndpointSource{\n\t\t\t\tURL: \"https://example.com/\",\n\t\t\t\tSupportsRead: true,\n\t\t\t\tSupportsWrite: false,\n\t\t\t},\n\t\t\texpectedBaseURL: \"https://example.com\",\n\t\t\texpectedCapabilities: autoconf.EndpointCapabilities{\n\t\t\t\tProviders: true,\n\t\t\t\tPeers: true,\n\t\t\t\tIPNSGet: true,\n\t\t\t\tIPNSPut: false,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"URL with IPNS path should have only IPNS capabilities\",\n\t\t\tendpoint: EndpointSource{\n\t\t\t\tURL: \"https://example.com/routing/v1/ipns\",\n\t\t\t\tSupportsRead: true,\n\t\t\t\tSupportsWrite: true,\n\t\t\t},\n\t\t\texpectedBaseURL: \"https://example.com\",\n\t\t\texpectedCapabilities: autoconf.EndpointCapabilities{\n\t\t\t\tProviders: false,\n\t\t\t\tPeers: false,\n\t\t\t\tIPNSGet: true,\n\t\t\t\tIPNSPut: true,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"URL with providers path should have only Providers capability\",\n\t\t\tendpoint: EndpointSource{\n\t\t\t\tURL: \"https://example.com/routing/v1/providers\",\n\t\t\t\tSupportsRead: true,\n\t\t\t\tSupportsWrite: false,\n\t\t\t},\n\t\t\texpectedBaseURL: \"https://example.com\",\n\t\t\texpectedCapabilities: autoconf.EndpointCapabilities{\n\t\t\t\tProviders: true,\n\t\t\t\tPeers: false,\n\t\t\t\tIPNSGet: false,\n\t\t\t\tIPNSPut: false,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"URL with peers path should have only Peers capability\",\n\t\t\tendpoint: EndpointSource{\n\t\t\t\tURL: \"https://example.com/routing/v1/peers\",\n\t\t\t\tSupportsRead: true,\n\t\t\t\tSupportsWrite: false,\n\t\t\t},\n\t\t\texpectedBaseURL: \"https://example.com\",\n\t\t\texpectedCapabilities: autoconf.EndpointCapabilities{\n\t\t\t\tProviders: false,\n\t\t\t\tPeers: true,\n\t\t\t\tIPNSGet: false,\n\t\t\t\tIPNSPut: false,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"URL with Write support only should enable IPNSPut for no-path endpoint\",\n\t\t\tendpoint: EndpointSource{\n\t\t\t\tURL: \"https://example.com\",\n\t\t\t\tSupportsRead: false,\n\t\t\t\tSupportsWrite: true,\n\t\t\t},\n\t\t\texpectedBaseURL: \"https://example.com\",\n\t\t\texpectedCapabilities: autoconf.EndpointCapabilities{\n\t\t\t\tProviders: false,\n\t\t\t\tPeers: false,\n\t\t\t\tIPNSGet: false,\n\t\t\t\tIPNSPut: true,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tbaseURL, capabilities, err := determineCapabilities(tt.endpoint)\n\n\t\t\tif tt.expectError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, tt.expectedBaseURL, baseURL)\n\t\t\tassert.Equal(t, tt.expectedCapabilities, capabilities)\n\t\t})\n\t}\n}\n\nfunc TestEndpointCapabilitiesReadWriteLogic(t *testing.T) {\n\tt.Run(\"Read endpoint with no path should enable read capabilities\", func(t *testing.T) {\n\t\tendpoint := EndpointSource{\n\t\t\tURL: \"https://example.com\",\n\t\t\tSupportsRead: true,\n\t\t\tSupportsWrite: false,\n\t\t}\n\t\t_, capabilities, err := determineCapabilities(endpoint)\n\t\trequire.NoError(t, err)\n\n\t\t// Read endpoint with no path should enable all read capabilities\n\t\tassert.True(t, capabilities.Providers)\n\t\tassert.True(t, capabilities.Peers)\n\t\tassert.True(t, capabilities.IPNSGet)\n\t\tassert.False(t, capabilities.IPNSPut) // Write capability should be false\n\t})\n\n\tt.Run(\"Write endpoint with no path should enable write capabilities\", func(t *testing.T) {\n\t\tendpoint := EndpointSource{\n\t\t\tURL: \"https://example.com\",\n\t\t\tSupportsRead: false,\n\t\t\tSupportsWrite: true,\n\t\t}\n\t\t_, capabilities, err := determineCapabilities(endpoint)\n\t\trequire.NoError(t, err)\n\n\t\t// Write endpoint with no path should only enable IPNS write capability\n\t\tassert.False(t, capabilities.Providers)\n\t\tassert.False(t, capabilities.Peers)\n\t\tassert.False(t, capabilities.IPNSGet)\n\t\tassert.True(t, capabilities.IPNSPut) // Only write capability should be true\n\t})\n\n\tt.Run(\"Specific path should only enable matching capabilities\", func(t *testing.T) {\n\t\tendpoint := EndpointSource{\n\t\t\tURL: \"https://example.com/routing/v1/ipns\",\n\t\t\tSupportsRead: true,\n\t\t\tSupportsWrite: true,\n\t\t}\n\t\t_, capabilities, err := determineCapabilities(endpoint)\n\t\trequire.NoError(t, err)\n\n\t\t// Specific IPNS path should only enable IPNS capabilities based on source\n\t\tassert.False(t, capabilities.Providers)\n\t\tassert.False(t, capabilities.Peers)\n\t\tassert.True(t, capabilities.IPNSGet) // Read capability enabled\n\t\tassert.True(t, capabilities.IPNSPut) // Write capability enabled\n\t})\n\n\tt.Run(\"Unsupported paths should result in empty capabilities\", func(t *testing.T) {\n\t\tendpoint := EndpointSource{\n\t\t\tURL: \"https://example.com/routing/v1/unsupported\",\n\t\t\tSupportsRead: true,\n\t\t\tSupportsWrite: false,\n\t\t}\n\t\t_, capabilities, err := determineCapabilities(endpoint)\n\t\trequire.NoError(t, err)\n\n\t\t// Unsupported paths should result in no capabilities\n\t\tassert.False(t, capabilities.Providers)\n\t\tassert.False(t, capabilities.Peers)\n\t\tassert.False(t, capabilities.IPNSGet)\n\t\tassert.False(t, capabilities.IPNSPut)\n\t})\n}\n\n// stubHost is a minimal host.Host stub for testing httpRouterAddrFunc.\n// Only the methods checked via type assertion (confirmedAddrsHost) matter;\n// all other methods panic if called.\ntype stubHost struct {\n\treachable []ma.Multiaddr\n}\n\nfunc (h *stubHost) ConfirmedAddrs() (reachable, unreachable, unknown []ma.Multiaddr) {\n\treturn h.reachable, nil, nil\n}\n\nfunc (h *stubHost) ID() peer.ID { panic(\"unused\") }\nfunc (h *stubHost) Addrs() []ma.Multiaddr { panic(\"unused\") }\nfunc (h *stubHost) Peerstore() peerstore.Peerstore { panic(\"unused\") }\nfunc (h *stubHost) Network() network.Network { panic(\"unused\") }\nfunc (h *stubHost) Mux() protocol.Switch { panic(\"unused\") }\nfunc (h *stubHost) Connect(context.Context, peer.AddrInfo) error { panic(\"unused\") }\nfunc (h *stubHost) SetStreamHandler(protocol.ID, network.StreamHandler) { panic(\"unused\") }\nfunc (h *stubHost) SetStreamHandlerMatch(protocol.ID, func(protocol.ID) bool, network.StreamHandler) {\n\tpanic(\"unused\")\n}\nfunc (h *stubHost) RemoveStreamHandler(protocol.ID) { panic(\"unused\") }\nfunc (h *stubHost) NewStream(context.Context, peer.ID, ...protocol.ID) (network.Stream, error) {\n\tpanic(\"unused\")\n}\nfunc (h *stubHost) Close() error { panic(\"unused\") }\nfunc (h *stubHost) ConnManager() connmgr.ConnManager { panic(\"unused\") }\nfunc (h *stubHost) EventBus() event.Bus { panic(\"unused\") }\n\nfunc TestHttpRouterAddrFunc(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\treachable []string // autonat confirmed addrs (nil = none)\n\t\tcfg config.Addresses\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"prefers autonat confirmed reachable addrs over swarm fallback\",\n\t\t\treachable: []string{\"/ip4/1.2.3.4/tcp/4001\", \"/ip4/1.2.3.4/udp/4001/quic-v1\"},\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/udp/4001/quic-v1\"}},\n\t\t\twant: []string{\"/ip4/1.2.3.4/tcp/4001\", \"/ip4/1.2.3.4/udp/4001/quic-v1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"falls back to swarm when autonat has no confirmed addrs\",\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\"}},\n\t\t\twant: []string{\"/ip4/0.0.0.0/tcp/4001\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Announce overrides autonat and swarm\",\n\t\t\treachable: []string{\"/ip4/1.2.3.4/tcp/4001\"},\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\"}, Announce: []string{\"/ip4/5.6.7.8/tcp/4001\"}},\n\t\t\twant: []string{\"/ip4/5.6.7.8/tcp/4001\"},\n\t\t},\n\t\t{\n\t\t\tname: \"AppendAnnounce added to autonat addrs\",\n\t\t\treachable: []string{\"/ip4/1.2.3.4/tcp/4001\"},\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\"}, AppendAnnounce: []string{\"/ip4/10.0.0.1/tcp/4001\"}},\n\t\t\twant: []string{\"/ip4/1.2.3.4/tcp/4001\", \"/ip4/10.0.0.1/tcp/4001\"},\n\t\t},\n\t\t{\n\t\t\tname: \"AppendAnnounce added to swarm fallback\",\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\"}, AppendAnnounce: []string{\"/ip4/10.0.0.1/tcp/4001\"}},\n\t\t\twant: []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/10.0.0.1/tcp/4001\"},\n\t\t},\n\t\t{\n\t\t\tname: \"NoAnnounce filters swarm fallback\",\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/udp/4001/quic-v1\"}, NoAnnounce: []string{\"/ip4/0.0.0.0/tcp/4001\"}},\n\t\t\twant: []string{\"/ip4/0.0.0.0/udp/4001/quic-v1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"AppendAnnounce added to Announce\",\n\t\t\tcfg: config.Addresses{Swarm: []string{\"/ip4/0.0.0.0/tcp/4001\"}, Announce: []string{\"/ip4/5.6.7.8/tcp/4001\"}, AppendAnnounce: []string{\"/ip4/10.0.0.1/tcp/4001\"}},\n\t\t\twant: []string{\"/ip4/5.6.7.8/tcp/4001\", \"/ip4/10.0.0.1/tcp/4001\"},\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\th := &stubHost{reachable: parseMultiaddrs(tt.reachable)}\n\t\t\tfn := httpRouterAddrFunc(h, tt.cfg)\n\t\t\tassert.Equal(t, parseMultiaddrs(tt.want), fn())\n\t\t})\n\t}\n}\n" }, { "path": "core/node/libp2p/sec.go", "content": "package libp2p\n\nimport (\n\t\"github.com/ipfs/kubo/config\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/p2p/security/noise\"\n\ttls \"github.com/libp2p/go-libp2p/p2p/security/tls\"\n)\n\nfunc Security(enabled bool, tptConfig config.Transports) any {\n\tif !enabled {\n\t\treturn func() (opts Libp2pOpts) {\n\t\t\tlog.Errorf(`Your IPFS node has been configured to run WITHOUT ENCRYPTED CONNECTIONS.\n\t\tYou will not be able to connect to any nodes configured to use encrypted connections`)\n\t\t\topts.Opts = append(opts.Opts, libp2p.NoSecurity)\n\t\t\treturn opts\n\t\t}\n\t}\n\n\t// Using the new config options.\n\treturn func() (opts Libp2pOpts) {\n\t\topts.Opts = append(opts.Opts, prioritizeOptions([]priorityOption{{\n\t\t\tpriority: tptConfig.Security.TLS,\n\t\t\tdefaultPriority: 100,\n\t\t\topt: libp2p.Security(tls.ID, tls.New),\n\t\t}, {\n\t\t\tpriority: tptConfig.Security.Noise,\n\t\t\tdefaultPriority: 200,\n\t\t\topt: libp2p.Security(noise.ID, noise.New),\n\t\t}}))\n\t\treturn opts\n\t}\n}\n" }, { "path": "core/node/libp2p/smux.go", "content": "package libp2p\n\nimport (\n\t\"errors\"\n\t\"os\"\n\n\t\"github.com/ipfs/kubo/config\"\n\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/p2p/muxer/yamux\"\n)\n\nfunc makeSmuxTransportOption(tptConfig config.Transports) (libp2p.Option, error) {\n\tif prefs := os.Getenv(\"LIBP2P_MUX_PREFS\"); prefs != \"\" {\n\t\treturn nil, errors.New(\"configuring muxers with LIBP2P_MUX_PREFS is no longer supported, use Swarm.Transports.Multiplexers\")\n\t}\n\tif tptConfig.Multiplexers.Yamux < 0 {\n\t\treturn nil, errors.New(\"running libp2p with Swarm.Transports.Multiplexers.Yamux disabled is not supported\")\n\t}\n\n\treturn libp2p.Muxer(yamux.ID, yamux.DefaultTransport), nil\n}\n\nfunc SmuxTransport(tptConfig config.Transports) func() (opts Libp2pOpts, err error) {\n\treturn func() (opts Libp2pOpts, err error) {\n\t\tres, err := makeSmuxTransportOption(tptConfig)\n\t\tif err != nil {\n\t\t\treturn opts, err\n\t\t}\n\t\topts.Opts = append(opts.Opts, res)\n\t\treturn opts, nil\n\t}\n}\n" }, { "path": "core/node/libp2p/topicdiscovery.go", "content": "package libp2p\n\nimport (\n\t\"math/rand\"\n\t\"time\"\n\n\t\"github.com/libp2p/go-libp2p/core/discovery\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/p2p/discovery/backoff\"\n\tdisc \"github.com/libp2p/go-libp2p/p2p/discovery/routing\"\n\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n)\n\nfunc TopicDiscovery() any {\n\treturn func(host host.Host, cr routing.ContentRouting) (service discovery.Discovery, err error) {\n\t\tbaseDisc := disc.NewRoutingDiscovery(cr)\n\t\tminBackoff, maxBackoff := time.Second*60, time.Hour\n\t\trng := rand.New(rand.NewSource(rand.Int63()))\n\t\td, err := backoff.NewBackoffDiscovery(\n\t\t\tbaseDisc,\n\t\t\tbackoff.NewExponentialBackoff(minBackoff, maxBackoff, backoff.FullJitter, time.Second, 5.0, 0, rng),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn d, nil\n\t}\n}\n" }, { "path": "core/node/libp2p/transport.go", "content": "package libp2p\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipshipyard/p2p-forge/client\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/metrics\"\n\tquic \"github.com/libp2p/go-libp2p/p2p/transport/quic\"\n\t\"github.com/libp2p/go-libp2p/p2p/transport/tcp\"\n\twebrtc \"github.com/libp2p/go-libp2p/p2p/transport/webrtc\"\n\t\"github.com/libp2p/go-libp2p/p2p/transport/websocket\"\n\twebtransport \"github.com/libp2p/go-libp2p/p2p/transport/webtransport\"\n\n\t\"go.uber.org/fx\"\n)\n\nfunc Transports(tptConfig config.Transports) any {\n\treturn func(params struct {\n\t\tfx.In\n\t\tFprint PNetFingerprint `optional:\"true\"`\n\t\tForgeMgr *client.P2PForgeCertMgr `optional:\"true\"`\n\t},\n\t) (opts Libp2pOpts, err error) {\n\t\tprivateNetworkEnabled := params.Fprint != nil\n\n\t\ttcpEnabled := tptConfig.Network.TCP.WithDefault(true)\n\t\twsEnabled := tptConfig.Network.Websocket.WithDefault(true)\n\t\tif tcpEnabled {\n\t\t\t// TODO(9290): Make WithMetrics configurable\n\t\t\topts.Opts = append(opts.Opts, libp2p.Transport(tcp.NewTCPTransport, tcp.WithMetrics()))\n\t\t}\n\n\t\tif wsEnabled {\n\t\t\tif params.ForgeMgr == nil {\n\t\t\t\topts.Opts = append(opts.Opts, libp2p.Transport(websocket.New))\n\t\t\t} else {\n\t\t\t\topts.Opts = append(opts.Opts, libp2p.Transport(websocket.New, websocket.WithTLSConfig(params.ForgeMgr.TLSConfig())))\n\t\t\t}\n\t\t}\n\n\t\tif tcpEnabled && wsEnabled && os.Getenv(\"LIBP2P_TCP_MUX\") != \"false\" {\n\t\t\tif privateNetworkEnabled {\n\t\t\t\tlog.Error(\"libp2p.ShareTCPListener() is not supported in private networks, please disable Swarm.Transports.Network.Websocket or run with LIBP2P_TCP_MUX=false to make this message go away\")\n\t\t\t} else {\n\t\t\t\topts.Opts = append(opts.Opts, libp2p.ShareTCPListener())\n\t\t\t}\n\t\t}\n\n\t\tif tptConfig.Network.QUIC.WithDefault(!privateNetworkEnabled) {\n\t\t\tif privateNetworkEnabled {\n\t\t\t\treturn opts, fmt.Errorf(\n\t\t\t\t\t\"QUIC transport does not support private networks, please disable Swarm.Transports.Network.QUIC\",\n\t\t\t\t)\n\t\t\t}\n\t\t\topts.Opts = append(opts.Opts, libp2p.Transport(quic.NewTransport))\n\t\t}\n\n\t\tif tptConfig.Network.WebTransport.WithDefault(!privateNetworkEnabled) {\n\t\t\tif privateNetworkEnabled {\n\t\t\t\treturn opts, fmt.Errorf(\n\t\t\t\t\t\"WebTransport transport does not support private networks, please disable Swarm.Transports.Network.WebTransport\",\n\t\t\t\t)\n\t\t\t}\n\t\t\topts.Opts = append(opts.Opts, libp2p.Transport(webtransport.New))\n\t\t}\n\n\t\tif tptConfig.Network.WebRTCDirect.WithDefault(!privateNetworkEnabled) {\n\t\t\tif privateNetworkEnabled {\n\t\t\t\treturn opts, fmt.Errorf(\n\t\t\t\t\t\"WebRTC Direct transport does not support private networks, please disable Swarm.Transports.Network.WebRTCDirect\",\n\t\t\t\t)\n\t\t\t}\n\t\t\topts.Opts = append(opts.Opts, libp2p.Transport(webrtc.New))\n\t\t}\n\n\t\treturn opts, nil\n\t}\n}\n\nfunc BandwidthCounter() (opts Libp2pOpts, reporter *metrics.BandwidthCounter) {\n\treporter = metrics.NewBandwidthCounter()\n\topts.Opts = append(opts.Opts, libp2p.BandwidthReporter(reporter))\n\treturn opts, reporter\n}\n" }, { "path": "core/node/p2pforge_resolver.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"net\"\n\t\"net/netip\"\n\t\"strings\"\n\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n)\n\n// p2pForgeResolver implements madns.BasicResolver for deterministic resolution\n// of p2p-forge domains (e.g., *.libp2p.direct) without network I/O for A/AAAA queries.\n//\n// p2p-forge encodes IP addresses in DNS hostnames:\n// - IPv4: 1-2-3-4.peerID.libp2p.direct -> 1.2.3.4\n// - IPv6: 2001-db8--1.peerID.libp2p.direct -> 2001:db8::1\n//\n// When local parsing fails (invalid format, invalid peerID, etc.), the resolver\n// falls back to network DNS. This ensures future .libp2p.direct records\n// can still resolve if the authoritative DNS adds support for them.\n//\n// TXT queries always delegate to the fallback resolver. This is important for\n// p2p-forge/client ACME DNS-01 challenges to work correctly, as Let's Encrypt\n// needs to verify TXT records at _acme-challenge.peerID.libp2p.direct.\n//\n// See: https://github.com/ipshipyard/p2p-forge\ntype p2pForgeResolver struct {\n\tsuffixes []string\n\tfallback madns.BasicResolver\n}\n\n// Compile-time check that p2pForgeResolver implements madns.BasicResolver.\nvar _ madns.BasicResolver = (*p2pForgeResolver)(nil)\n\n// NewP2PForgeResolver creates a resolver for the given p2p-forge domain suffixes.\n// Each suffix should be a bare domain like \"libp2p.direct\" (without leading dot).\n// When local IP parsing fails, queries fall back to the provided resolver.\n// TXT queries always delegate to the fallback resolver for ACME compatibility.\nfunc NewP2PForgeResolver(suffixes []string, fallback madns.BasicResolver) *p2pForgeResolver {\n\tnormalized := make([]string, len(suffixes))\n\tfor i, s := range suffixes {\n\t\tnormalized[i] = strings.ToLower(strings.TrimSuffix(s, \".\"))\n\t}\n\treturn &p2pForgeResolver{suffixes: normalized, fallback: fallback}\n}\n\n// LookupIPAddr parses IP addresses encoded in the hostname.\n//\n// Format: ..\n// - IPv4: 192-168-1-1.peerID.libp2p.direct -> [192.168.1.1]\n// - IPv6: 2001-db8--1.peerID.libp2p.direct -> [2001:db8::1]\n//\n// If the hostname doesn't match the expected format (wrong suffix, invalid peerID,\n// invalid IP encoding, or peerID-only), the lookup falls back to network DNS.\n// This allows future DNS records like .libp2p.direct to resolve normally.\nfunc (r *p2pForgeResolver) LookupIPAddr(ctx context.Context, hostname string) ([]net.IPAddr, error) {\n\t// DNS is case-insensitive, normalize to lowercase\n\thostname = strings.ToLower(strings.TrimSuffix(hostname, \".\"))\n\n\t// find matching suffix and extract subdomain\n\tvar subdomain string\n\tfor _, suffix := range r.suffixes {\n\t\tif sub, found := strings.CutSuffix(hostname, \".\"+suffix); found {\n\t\t\tsubdomain = sub\n\t\t\tbreak\n\t\t}\n\t}\n\tif subdomain == \"\" {\n\t\t// not a p2p-forge domain, fallback to network\n\t\treturn r.fallback.LookupIPAddr(ctx, hostname)\n\t}\n\n\t// split subdomain into parts: should be [ip-prefix, peerID]\n\tparts := strings.Split(subdomain, \".\")\n\tif len(parts) != 2 {\n\t\t// not the expected . format, fallback to network\n\t\treturn r.fallback.LookupIPAddr(ctx, hostname)\n\t}\n\n\tencodedIP := parts[0]\n\tpeerIDStr := parts[1]\n\n\t// validate peerID (same validation as libp2p.direct DNS server)\n\tif _, err := peer.Decode(peerIDStr); err != nil {\n\t\t// invalid peerID, fallback to network\n\t\treturn r.fallback.LookupIPAddr(ctx, hostname)\n\t}\n\n\t// RFC 1123: hostname labels cannot start or end with hyphen\n\tif len(encodedIP) == 0 || encodedIP[0] == '-' || encodedIP[len(encodedIP)-1] == '-' {\n\t\t// invalid hostname label, fallback to network\n\t\treturn r.fallback.LookupIPAddr(ctx, hostname)\n\t}\n\n\t// try parsing as IPv4 first: segments joined by \"-\" become \".\"\n\tsegments := strings.Split(encodedIP, \"-\")\n\tif len(segments) == 4 {\n\t\tipv4Str := strings.Join(segments, \".\")\n\t\tif ip, err := netip.ParseAddr(ipv4Str); err == nil && ip.Is4() {\n\t\t\treturn []net.IPAddr{{IP: ip.AsSlice()}}, nil\n\t\t}\n\t}\n\n\t// try parsing as IPv6: segments joined by \"-\" become \":\"\n\tipv6Str := strings.Join(segments, \":\")\n\tif ip, err := netip.ParseAddr(ipv6Str); err == nil && ip.Is6() {\n\t\treturn []net.IPAddr{{IP: ip.AsSlice()}}, nil\n\t}\n\n\t// IP parsing failed, fallback to network\n\treturn r.fallback.LookupIPAddr(ctx, hostname)\n}\n\n// LookupTXT delegates to the fallback resolver to support ACME DNS-01 challenges\n// and any other TXT record lookups on p2p-forge domains.\nfunc (r *p2pForgeResolver) LookupTXT(ctx context.Context, hostname string) ([]string, error) {\n\treturn r.fallback.LookupTXT(ctx, hostname)\n}\n" }, { "path": "core/node/p2pforge_resolver_test.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"net\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// Test constants matching p2p-forge production format\nconst (\n\t// testPeerID is a valid peerID in CIDv1 base36 format as used by p2p-forge.\n\t// Base36 is lowercase-only, making it safe for case-insensitive DNS.\n\t// Corresponds to 12D3KooWDpJ7As7BWAwRMfu1VU2WCqNjvq387JEYKDBj4kx6nXTN in base58btc.\n\ttestPeerID = \"k51qzi5uqu5dhnwe629wdlncpql6frppdpwnz4wtlcw816aysd5wwlk63g4wmh\"\n\n\t// domainSuffix is the default p2p-forge domain used in tests.\n\tdomainSuffix = config.DefaultDomainSuffix\n)\n\n// mockResolver implements madns.BasicResolver for testing\ntype mockResolver struct {\n\ttxtRecords map[string][]string\n\tipRecords map[string][]net.IPAddr\n\tipErr error\n}\n\nfunc (m *mockResolver) LookupIPAddr(_ context.Context, hostname string) ([]net.IPAddr, error) {\n\tif m.ipErr != nil {\n\t\treturn nil, m.ipErr\n\t}\n\tif m.ipRecords != nil {\n\t\treturn m.ipRecords[hostname], nil\n\t}\n\treturn nil, nil\n}\n\nfunc (m *mockResolver) LookupTXT(_ context.Context, name string) ([]string, error) {\n\tif m.txtRecords != nil {\n\t\treturn m.txtRecords[name], nil\n\t}\n\treturn nil, nil\n}\n\n// newTestResolver creates a p2pForgeResolver with default suffix.\nfunc newTestResolver(t *testing.T) *p2pForgeResolver {\n\tt.Helper()\n\treturn NewP2PForgeResolver([]string{domainSuffix}, &mockResolver{})\n}\n\n// assertLookupIP verifies that hostname resolves to wantIP.\nfunc assertLookupIP(t *testing.T, r *p2pForgeResolver, hostname, wantIP string) {\n\tt.Helper()\n\taddrs, err := r.LookupIPAddr(t.Context(), hostname)\n\trequire.NoError(t, err)\n\trequire.Len(t, addrs, 1)\n\tassert.Equal(t, wantIP, addrs[0].IP.String())\n}\n\nfunc TestP2PForgeResolver_LookupIPAddr(t *testing.T) {\n\tr := newTestResolver(t)\n\n\ttests := []struct {\n\t\tname string\n\t\thostname string\n\t\twantIP string\n\t}{\n\t\t// IPv4\n\t\t{\"ipv4/basic\", \"192-168-1-1.\" + testPeerID + \".\" + domainSuffix, \"192.168.1.1\"},\n\t\t{\"ipv4/zeros\", \"0-0-0-0.\" + testPeerID + \".\" + domainSuffix, \"0.0.0.0\"},\n\t\t{\"ipv4/max\", \"255-255-255-255.\" + testPeerID + \".\" + domainSuffix, \"255.255.255.255\"},\n\t\t{\"ipv4/trailing dot\", \"10-0-0-1.\" + testPeerID + \".\" + domainSuffix + \".\", \"10.0.0.1\"},\n\t\t{\"ipv4/uppercase suffix\", \"192-168-1-1.\" + testPeerID + \".LIBP2P.DIRECT\", \"192.168.1.1\"},\n\t\t// IPv6\n\t\t{\"ipv6/full\", \"2001-db8-0-0-0-0-0-1.\" + testPeerID + \".\" + domainSuffix, \"2001:db8::1\"},\n\t\t{\"ipv6/compressed\", \"2001-db8--1.\" + testPeerID + \".\" + domainSuffix, \"2001:db8::1\"},\n\t\t{\"ipv6/loopback\", \"0--1.\" + testPeerID + \".\" + domainSuffix, \"::1\"},\n\t\t{\"ipv6/all zeros\", \"0--0.\" + testPeerID + \".\" + domainSuffix, \"::\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tassertLookupIP(t, r, tt.hostname, tt.wantIP)\n\t\t})\n\t}\n}\n\nfunc TestP2PForgeResolver_LookupIPAddr_MultipleSuffixes(t *testing.T) {\n\tr := NewP2PForgeResolver([]string{domainSuffix, \"custom.example.com\"}, &mockResolver{})\n\n\ttests := []struct {\n\t\thostname string\n\t\twantIP string\n\t}{\n\t\t{\"192-168-1-1.\" + testPeerID + \".\" + domainSuffix, \"192.168.1.1\"},\n\t\t{\"10-0-0-1.\" + testPeerID + \".custom.example.com\", \"10.0.0.1\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.hostname, func(t *testing.T) {\n\t\t\tassertLookupIP(t, r, tt.hostname, tt.wantIP)\n\t\t})\n\t}\n}\n\nfunc TestP2PForgeResolver_LookupIPAddr_FallbackToNetwork(t *testing.T) {\n\tfallbackIP := []net.IPAddr{{IP: net.ParseIP(\"93.184.216.34\")}}\n\n\ttests := []struct {\n\t\tname string\n\t\thostname string\n\t}{\n\t\t{\"peerID only\", testPeerID + \".\" + domainSuffix},\n\t\t{\"invalid peerID\", \"192-168-1-1.invalid-peer-id.\" + domainSuffix},\n\t\t{\"invalid IP encoding\", \"not-an-ip.\" + testPeerID + \".\" + domainSuffix},\n\t\t{\"leading hyphen\", \"-192-168-1-1.\" + testPeerID + \".\" + domainSuffix},\n\t\t{\"too many parts\", \"extra.192-168-1-1.\" + testPeerID + \".\" + domainSuffix},\n\t\t{\"wrong suffix\", \"192-168-1-1.\" + testPeerID + \".example.com\"},\n\t}\n\n\t// Build fallback records from test cases\n\tipRecords := make(map[string][]net.IPAddr, len(tests))\n\tfor _, tt := range tests {\n\t\tipRecords[tt.hostname] = fallbackIP\n\t}\n\tfallback := &mockResolver{ipRecords: ipRecords}\n\tr := NewP2PForgeResolver([]string{domainSuffix}, fallback)\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\taddrs, err := r.LookupIPAddr(t.Context(), tt.hostname)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Len(t, addrs, 1, \"should fallback to network\")\n\t\t\tassert.Equal(t, \"93.184.216.34\", addrs[0].IP.String())\n\t\t})\n\t}\n}\n\nfunc TestP2PForgeResolver_LookupIPAddr_FallbackError(t *testing.T) {\n\texpectedErr := errors.New(\"network error\")\n\tr := NewP2PForgeResolver([]string{domainSuffix}, &mockResolver{ipErr: expectedErr})\n\n\t// peerID-only triggers fallback, which returns error\n\t_, err := r.LookupIPAddr(t.Context(), testPeerID+\".\"+domainSuffix)\n\trequire.ErrorIs(t, err, expectedErr)\n}\n\nfunc TestP2PForgeResolver_LookupTXT(t *testing.T) {\n\tt.Run(\"delegates to fallback for ACME DNS-01\", func(t *testing.T) {\n\t\tacmeHost := \"_acme-challenge.\" + testPeerID + \".\" + domainSuffix\n\t\tfallback := &mockResolver{\n\t\t\ttxtRecords: map[string][]string{acmeHost: {\"acme-token-value\"}},\n\t\t}\n\t\tr := NewP2PForgeResolver([]string{domainSuffix}, fallback)\n\n\t\trecords, err := r.LookupTXT(t.Context(), acmeHost)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, []string{\"acme-token-value\"}, records)\n\t})\n\n\tt.Run(\"returns empty when fallback has no records\", func(t *testing.T) {\n\t\tr := NewP2PForgeResolver([]string{domainSuffix}, &mockResolver{})\n\n\t\trecords, err := r.LookupTXT(t.Context(), \"anything.\"+domainSuffix)\n\t\trequire.NoError(t, err)\n\t\tassert.Empty(t, records)\n\t})\n}\n" }, { "path": "core/node/peering.go", "content": "package node\n\nimport (\n\t\"context\"\n\n\t\"github.com/ipfs/boxo/peering\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"go.uber.org/fx\"\n)\n\n// Peering constructs the peering service and hooks it into fx's lifetime\n// management system.\nfunc Peering(lc fx.Lifecycle, host host.Host) *peering.PeeringService {\n\tps := peering.NewPeeringService(host)\n\tlc.Append(fx.Hook{\n\t\tOnStart: func(context.Context) error {\n\t\t\treturn ps.Start()\n\t\t},\n\t\tOnStop: func(context.Context) error {\n\t\t\tps.Stop()\n\t\t\treturn nil\n\t\t},\n\t})\n\treturn ps\n}\n\n// PeerWith configures the peering service to peer with the specified peers.\nfunc PeerWith(peers ...peer.AddrInfo) fx.Option {\n\treturn fx.Invoke(func(ps *peering.PeeringService) {\n\t\tfor _, ai := range peers {\n\t\t\tps.AddPeer(ai)\n\t\t}\n\t})\n}\n" }, { "path": "core/node/provider.go", "content": "package node\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/fetcher\"\n\t\"github.com/ipfs/boxo/mfs\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/boxo/pinning/pinner/dspinner\"\n\t\"github.com/ipfs/boxo/provider\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-datastore\"\n\t\"github.com/ipfs/go-datastore/mount\"\n\t\"github.com/ipfs/go-datastore/namespace\"\n\t\"github.com/ipfs/go-datastore/query\"\n\tlog \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\tirouting \"github.com/ipfs/kubo/routing\"\n\tdht \"github.com/libp2p/go-libp2p-kad-dht\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/amino\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\tdht_pb \"github.com/libp2p/go-libp2p-kad-dht/pb\"\n\tdhtprovider \"github.com/libp2p/go-libp2p-kad-dht/provider\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/provider/buffered\"\n\tddhtprovider \"github.com/libp2p/go-libp2p-kad-dht/provider/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/provider/keystore\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\t\"github.com/libp2p/go-libp2p/core/host\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmh \"github.com/multiformats/go-multihash\"\n\t\"go.uber.org/fx\"\n)\n\nconst (\n\t// The size of a batch that will be used for calculating average announcement\n\t// time per CID, inside of boxo/provider.ThroughputReport\n\t// and in 'ipfs stats provide' report.\n\t// Used when Provide.DHT.SweepEnabled=false\n\tsampledBatchSize = 1000\n\n\t// Datastore key used to store previous reprovide strategy.\n\treprovideStrategyKey = \"/reprovideStrategy\"\n\n\t// KeystoreDatastorePath is the base directory for the provider keystore datastores.\n\tKeystoreDatastorePath = \"provider-keystore\"\n)\n\nvar (\n\t// Datastore namespace key for provider data.\n\tproviderDatastoreKey = datastore.NewKey(\"provider\")\n\t// Datastore namespace key for provider keystore data.\n\tkeystoreDatastoreKey = datastore.NewKey(\"keystore\")\n)\n\nvar errAcceleratedDHTNotReady = errors.New(\"AcceleratedDHTClient: routing table not ready\")\n\n// validateKeystoreSuffix rejects any suffix other than \"0\" or \"1\".\n// The upstream library uses these two values as alternating namespace\n// identifiers. Validating here prevents accidental deletion of unrelated\n// directories via os.RemoveAll if the upstream ever changes its scheme.\nfunc validateKeystoreSuffix(suffix string) error {\n\tif suffix != \"0\" && suffix != \"1\" {\n\t\treturn fmt.Errorf(\"unexpected keystore suffix %q, expected \\\"0\\\" or \\\"1\\\"\", suffix)\n\t}\n\treturn nil\n}\n\n// Interval between reprovide queue monitoring checks for slow reprovide alerts.\n// Used when Provide.DHT.SweepEnabled=true\nconst reprovideAlertPollInterval = 15 * time.Minute\n\n// Number of consecutive polling intervals with sustained queue growth before\n// triggering a slow reprovide alert (3 intervals = 45 minutes).\n// Used when Provide.DHT.SweepEnabled=true\nconst consecutiveAlertsThreshold = 3\n\n// DHTProvider is an interface for providing keys to a DHT swarm. It holds a\n// state of keys to be advertised, and is responsible for periodically\n// publishing provider records for these keys to the DHT swarm before the\n// records expire.\ntype DHTProvider interface {\n\t// StartProviding ensures keys are periodically advertised to the DHT swarm.\n\t//\n\t// If the `keys` aren't currently being reprovided, they are added to the\n\t// queue to be provided to the DHT swarm as soon as possible, and scheduled\n\t// to be reprovided periodically. If `force` is set to true, all keys are\n\t// provided to the DHT swarm, regardless of whether they were already being\n\t// reprovided in the past. `keys` keep being reprovided until `StopProviding`\n\t// is called.\n\t//\n\t// This operation is asynchronous, it returns as soon as the `keys` are added\n\t// to the provide queue, and provides happens asynchronously.\n\t//\n\t// Returns an error if the keys couldn't be added to the provide queue. This\n\t// can happen if the provider is closed or if the node is currently Offline\n\t// (either never bootstrapped, or disconnected since more than `OfflineDelay`).\n\t// The schedule and provide queue depend on the network size, hence recent\n\t// network connectivity is essential.\n\tStartProviding(force bool, keys ...mh.Multihash) error\n\t// ProvideOnce sends provider records for the specified keys to the DHT swarm\n\t// only once. It does not automatically reprovide those keys afterward.\n\t//\n\t// Add the supplied multihashes to the provide queue, and return immediately.\n\t// The provide operation happens asynchronously.\n\t//\n\t// Returns an error if the keys couldn't be added to the provide queue. This\n\t// can happen if the provider is closed or if the node is currently Offline\n\t// (either never bootstrapped, or disconnected since more than `OfflineDelay`).\n\t// The schedule and provide queue depend on the network size, hence recent\n\t// network connectivity is essential.\n\tProvideOnce(keys ...mh.Multihash) error\n\t// Clear clears the all the keys from the provide queue and returns the number\n\t// of keys that were cleared.\n\t//\n\t// The keys are not deleted from the keystore, so they will continue to be\n\t// reprovided as scheduled.\n\tClear() int\n\t// RefreshSchedule scans the Keystore for any keys that are not currently\n\t// scheduled for reproviding. If such keys are found, it schedules their\n\t// associated keyspace region to be reprovided.\n\t//\n\t// This function doesn't remove prefixes that have no keys from the schedule.\n\t// This is done automatically during the reprovide operation if a region has no\n\t// keys.\n\t//\n\t// Returns an error if the provider is closed or if the node is currently\n\t// Offline (either never bootstrapped, or disconnected since more than\n\t// `OfflineDelay`). The schedule depends on the network size, hence recent\n\t// network connectivity is essential.\n\tRefreshSchedule() error\n\tClose() error\n}\n\nvar (\n\t_ DHTProvider = &ddhtprovider.SweepingProvider{}\n\t_ DHTProvider = &dhtprovider.SweepingProvider{}\n\t_ DHTProvider = &NoopProvider{}\n\t_ DHTProvider = &LegacyProvider{}\n)\n\n// NoopProvider is a no-operation provider implementation that does nothing.\n// It is used when providing is disabled or when no DHT is available.\n// All methods return successfully without performing any actual operations.\ntype NoopProvider struct{}\n\nfunc (r *NoopProvider) StartProviding(bool, ...mh.Multihash) error { return nil }\nfunc (r *NoopProvider) ProvideOnce(...mh.Multihash) error { return nil }\nfunc (r *NoopProvider) Clear() int { return 0 }\nfunc (r *NoopProvider) RefreshSchedule() error { return nil }\nfunc (r *NoopProvider) Close() error { return nil }\n\n// LegacyProvider is a wrapper around the boxo/provider.System that implements\n// the DHTProvider interface. This provider manages reprovides using a burst\n// strategy where it sequentially reprovides all keys at once during each\n// reprovide interval, rather than spreading the load over time.\n//\n// This is the legacy provider implementation that can cause resource spikes\n// during reprovide operations. For more efficient providing, consider using\n// the SweepingProvider which spreads the load over the reprovide interval.\ntype LegacyProvider struct {\n\tprovider.System\n}\n\nfunc (r *LegacyProvider) StartProviding(force bool, keys ...mh.Multihash) error {\n\treturn r.ProvideOnce(keys...)\n}\n\nfunc (r *LegacyProvider) ProvideOnce(keys ...mh.Multihash) error {\n\tif many, ok := r.System.(routinghelpers.ProvideManyRouter); ok {\n\t\treturn many.ProvideMany(context.Background(), keys)\n\t}\n\n\tfor _, k := range keys {\n\t\tif err := r.Provide(context.Background(), cid.NewCidV1(cid.Raw, k), true); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (r *LegacyProvider) Clear() int {\n\treturn r.System.Clear()\n}\n\nfunc (r *LegacyProvider) RefreshSchedule() error { return nil }\n\n// LegacyProviderOpt creates a LegacyProvider to be used as provider in the\n// IpfsNode\nfunc LegacyProviderOpt(reprovideInterval time.Duration, strategy string, acceleratedDHTClient bool, provideWorkerCount int) fx.Option {\n\tsystem := fx.Provide(\n\t\tfx.Annotate(func(lc fx.Lifecycle, cr irouting.ProvideManyRouter, repo repo.Repo) (*LegacyProvider, error) {\n\t\t\t// Initialize provider.System first, before pinner/blockstore/etc.\n\t\t\t// The KeyChanFunc will be set later via SetKeyProvider() once we have\n\t\t\t// created the pinner, blockstore and other dependencies.\n\t\t\topts := []provider.Option{\n\t\t\t\tprovider.Online(cr),\n\t\t\t\tprovider.ReproviderInterval(reprovideInterval),\n\t\t\t\tprovider.ProvideWorkerCount(provideWorkerCount),\n\t\t\t}\n\t\t\tif !acceleratedDHTClient && reprovideInterval > 0 {\n\t\t\t\t// The estimation kinda suck if you are running with accelerated DHT client,\n\t\t\t\t// given this message is just trying to push people to use the acceleratedDHTClient\n\t\t\t\t// let's not report on through if it's in use\n\t\t\t\topts = append(opts,\n\t\t\t\t\tprovider.ThroughputReport(func(reprovide bool, complete bool, keysProvided uint, duration time.Duration) bool {\n\t\t\t\t\t\tavgProvideSpeed := duration / time.Duration(keysProvided)\n\t\t\t\t\t\tcount := uint64(keysProvided)\n\n\t\t\t\t\t\tif !reprovide || !complete {\n\t\t\t\t\t\t\t// We don't know how many CIDs we have to provide, try to fetch it from the blockstore.\n\t\t\t\t\t\t\t// But don't try for too long as this might be very expensive if you have a huge datastore.\n\t\t\t\t\t\t\tctx, cancel := context.WithTimeout(context.Background(), time.Minute*5)\n\t\t\t\t\t\t\tdefer cancel()\n\n\t\t\t\t\t\t\t// FIXME: I want a running counter of blocks so size of blockstore can be an O(1) lookup.\n\t\t\t\t\t\t\t// Note: talk to datastore directly, as to not depend on Blockstore here.\n\t\t\t\t\t\t\tqr, err := repo.Datastore().Query(ctx, query.Query{\n\t\t\t\t\t\t\t\tPrefix: blockstore.BlockPrefix.String(),\n\t\t\t\t\t\t\t\tKeysOnly: true,\n\t\t\t\t\t\t\t})\n\t\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\t\tlogger.Errorf(\"fetching AllKeysChain in provider ThroughputReport: %v\", err)\n\t\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tdefer qr.Close()\n\t\t\t\t\t\t\tcount = 0\n\t\t\t\t\t\tcountLoop:\n\t\t\t\t\t\t\tfor {\n\t\t\t\t\t\t\t\tselect {\n\t\t\t\t\t\t\t\tcase _, ok := <-qr.Next():\n\t\t\t\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\t\t\t\tbreak countLoop\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t\tcount++\n\t\t\t\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\t\t\t\t// really big blockstore mode\n\n\t\t\t\t\t\t\t\t\t// how many blocks would be in a 10TiB blockstore with 128KiB blocks.\n\t\t\t\t\t\t\t\t\tconst probableBigBlockstore = (10 * 1024 * 1024 * 1024 * 1024) / (128 * 1024)\n\t\t\t\t\t\t\t\t\t// How long per block that lasts us.\n\t\t\t\t\t\t\t\t\texpectedProvideSpeed := reprovideInterval / probableBigBlockstore\n\t\t\t\t\t\t\t\t\tif avgProvideSpeed > expectedProvideSpeed {\n\t\t\t\t\t\t\t\t\t\tlogger.Errorf(`\n🔔🔔🔔 Reprovide Operations Too Slow 🔔🔔🔔\n\nYour node may be falling behind on DHT reprovides, which could affect content availability.\n\nObserved: %d keys at %v per key\nEstimated: Assuming 10TiB blockstore, would take %v to complete\n⏰ Must finish within %v (Provide.DHT.Interval)\n\nSolutions (try in order):\n1. Enable Provide.DHT.SweepEnabled=true (recommended)\n2. Increase Provide.DHT.MaxWorkers if needed\n3. Enable Routing.AcceleratedDHTClient=true (last resort, resource intensive)\n\nLearn more: https://github.com/ipfs/kubo/blob/master/docs/config.md#provide`,\n\t\t\t\t\t\t\t\t\t\t\tkeysProvided, avgProvideSpeed, avgProvideSpeed*probableBigBlockstore, reprovideInterval)\n\t\t\t\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\t// How long per block that lasts us.\n\t\t\t\t\t\texpectedProvideSpeed := reprovideInterval\n\t\t\t\t\t\tif count > 0 {\n\t\t\t\t\t\t\texpectedProvideSpeed = reprovideInterval / time.Duration(count)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif avgProvideSpeed > expectedProvideSpeed {\n\t\t\t\t\t\t\tlogger.Errorf(`\n🔔🔔🔔 Reprovide Operations Too Slow 🔔🔔🔔\n\nYour node is falling behind on DHT reprovides, which will affect content availability.\n\nObserved: %d keys at %v per key\nConfirmed: ~%d total CIDs requiring %v to complete\n⏰ Must finish within %v (Provide.DHT.Interval)\n\nSolutions (try in order):\n1. Enable Provide.DHT.SweepEnabled=true (recommended)\n2. Increase Provide.DHT.MaxWorkers if needed\n3. Enable Routing.AcceleratedDHTClient=true (last resort, resource intensive)\n\nLearn more: https://github.com/ipfs/kubo/blob/master/docs/config.md#provide`,\n\t\t\t\t\t\t\t\tkeysProvided, avgProvideSpeed, count, avgProvideSpeed*time.Duration(count), reprovideInterval)\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}, sampledBatchSize))\n\t\t\t}\n\n\t\t\tsys, err := provider.New(repo.Datastore(), opts...)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tlc.Append(fx.Hook{\n\t\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\t\treturn sys.Close()\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tprov := &LegacyProvider{sys}\n\t\t\thandleStrategyChange(strategy, prov, repo.Datastore())\n\n\t\t\treturn prov, nil\n\t\t},\n\t\t\tfx.As(new(provider.System)),\n\t\t\tfx.As(new(DHTProvider)),\n\t\t),\n\t)\n\tsetKeyProvider := fx.Invoke(func(lc fx.Lifecycle, system provider.System, keyProvider provider.KeyChanFunc) {\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStart: func(ctx context.Context) error {\n\t\t\t\t// SetKeyProvider breaks the circular dependency between provider, blockstore, and pinner.\n\t\t\t\t// We cannot create the blockstore without the provider (it needs to provide blocks),\n\t\t\t\t// and we cannot determine the reproviding strategy without the pinner/blockstore.\n\t\t\t\t// This deferred initialization allows us to create provider.System first,\n\t\t\t\t// then set the actual key provider function after all dependencies are ready.\n\t\t\t\tsystem.SetKeyProvider(keyProvider)\n\t\t\t\treturn nil\n\t\t\t},\n\t\t})\n\t})\n\treturn fx.Options(\n\t\tsystem,\n\t\tsetKeyProvider,\n\t)\n}\n\ntype dhtImpl interface {\n\trouting.Routing\n\tGetClosestPeers(context.Context, string) ([]peer.ID, error)\n\tHost() host.Host\n\tMessageSender() dht_pb.MessageSender\n}\n\ntype fullrtRouter struct {\n\t*fullrt.FullRT\n\tready bool\n\tlogger *log.ZapEventLogger\n}\n\nfunc newFullRTRouter(fr *fullrt.FullRT, loggerName string) *fullrtRouter {\n\treturn &fullrtRouter{\n\t\tFullRT: fr,\n\t\tready: true,\n\t\tlogger: log.Logger(loggerName),\n\t}\n}\n\n// GetClosestPeers overrides fullrt.FullRT's GetClosestPeers and returns an\n// error if the fullrt's initial network crawl isn't complete yet.\nfunc (fr *fullrtRouter) GetClosestPeers(ctx context.Context, key string) ([]peer.ID, error) {\n\tif fr.ready {\n\t\tif !fr.Ready() {\n\t\t\tfr.ready = false\n\t\t\tfr.logger.Info(\"AcceleratedDHTClient: waiting for routing table initialization (5-10 min, depends on DHT size and network) to complete before providing\")\n\t\t\treturn nil, errAcceleratedDHTNotReady\n\t\t}\n\t} else {\n\t\tif fr.Ready() {\n\t\t\tfr.ready = true\n\t\t\tfr.logger.Info(\"AcceleratedDHTClient: routing table ready, providing can begin\")\n\t\t} else {\n\t\t\treturn nil, errAcceleratedDHTNotReady\n\t\t}\n\t}\n\treturn fr.FullRT.GetClosestPeers(ctx, key)\n}\n\nvar (\n\t_ dhtImpl = &dht.IpfsDHT{}\n\t_ dhtImpl = &fullrtRouter{}\n)\n\ntype addrsFilter interface {\n\tFilteredAddrs() []ma.Multiaddr\n}\n\n// findRootDatastoreSpec extracts the leaf datastore spec for the root (\"/\")\n// mount from the repo's Datastore.Spec config. It unwraps mount (picks the \"/\"\n// mountpoint), measure, and log wrappers to find the actual backend spec\n// (e.g., levelds, pebbleds).\nfunc findRootDatastoreSpec(spec map[string]any) map[string]any {\n\tif spec == nil {\n\t\treturn nil\n\t}\n\tswitch spec[\"type\"] {\n\tcase \"mount\":\n\t\tmounts, ok := spec[\"mounts\"].([]any)\n\t\tif !ok {\n\t\t\treturn spec\n\t\t}\n\t\tfor _, m := range mounts {\n\t\t\tmnt, ok := m.(map[string]any)\n\t\t\tif !ok {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif mnt[\"mountpoint\"] == \"/\" {\n\t\t\t\treturn findRootDatastoreSpec(mnt)\n\t\t\t}\n\t\t}\n\t\t// No root mount found; return nil so callers fall back gracefully\n\t\t// (in-memory datastore or skip mounting) rather than passing a\n\t\t// mount-type spec to openDatastoreAt which expects a leaf backend.\n\t\treturn nil\n\tcase \"measure\", \"log\":\n\t\tif child, ok := spec[\"child\"].(map[string]any); ok {\n\t\t\treturn findRootDatastoreSpec(child)\n\t\t}\n\t\treturn spec\n\tdefault:\n\t\tif _, hasChild := spec[\"child\"]; hasChild {\n\t\t\tlogger.Warnw(\"unrecognized datastore wrapper type, using as-is\",\n\t\t\t\t\"type\", spec[\"type\"])\n\t\t}\n\t\treturn spec\n\t}\n}\n\n// MountKeystoreDatastores opens any provider keystore datastores that exist on\n// disk and returns them as mount.Mount entries ready to be combined with the\n// main repo datastore. The caller must call the returned cleanup function when\n// done. Returns nil mounts and a no-op closer if no keystores exist.\nfunc MountKeystoreDatastores(repo repo.Repo) ([]mount.Mount, func(), error) {\n\tcfg, err := repo.Config()\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"reading repo config: %w\", err)\n\t}\n\n\trootSpec := findRootDatastoreSpec(cfg.Datastore.Spec)\n\tif rootSpec == nil {\n\t\treturn nil, func() {}, nil\n\t}\n\n\tkeystoreBasePath := filepath.Join(repo.Path(), KeystoreDatastorePath)\n\tvar mounts []mount.Mount\n\tvar closers []func()\n\n\tfor _, suffix := range []string{\"0\", \"1\"} {\n\t\tdir := filepath.Join(keystoreBasePath, suffix)\n\t\tif _, err := os.Stat(dir); err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tds, err := openDatastoreAt(rootSpec, dir)\n\t\tif err != nil {\n\t\t\tfor _, c := range closers {\n\t\t\t\tc()\n\t\t\t}\n\t\t\treturn nil, nil, err\n\t\t}\n\t\tprefix := providerDatastoreKey.Child(keystoreDatastoreKey).ChildString(suffix)\n\t\tmounts = append(mounts, mount.Mount{Prefix: prefix, Datastore: ds})\n\t\tclosers = append(closers, func() { ds.Close() })\n\t}\n\n\tcloser := func() {\n\t\tfor _, c := range closers {\n\t\t\tc()\n\t\t}\n\t}\n\treturn mounts, closer, nil\n}\n\n// openDatastoreAt opens a datastore using the given spec at the specified path.\n// It deep-copies the spec to avoid mutating the original.\nfunc openDatastoreAt(rootSpec map[string]any, path string) (datastore.Batching, error) {\n\tspec := copySpec(rootSpec)\n\tspec[\"path\"] = path\n\tdsc, err := fsrepo.AnyDatastoreConfig(spec)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"creating datastore config for %s: %w\", path, err)\n\t}\n\treturn dsc.Create(\"\")\n}\n\n// copySpec deep-copies a datastore spec map so modifications (e.g., changing\n// the path) don't affect the original.\nfunc copySpec(spec map[string]any) map[string]any {\n\tif spec == nil {\n\t\treturn nil\n\t}\n\tcp := make(map[string]any, len(spec))\n\tfor k, v := range spec {\n\t\tswitch val := v.(type) {\n\t\tcase map[string]any:\n\t\t\tcp[k] = copySpec(val)\n\t\tcase []any:\n\t\t\ts := make([]any, len(val))\n\t\t\tfor i, elem := range val {\n\t\t\t\tif m, ok := elem.(map[string]any); ok {\n\t\t\t\t\ts[i] = copySpec(m)\n\t\t\t\t} else {\n\t\t\t\t\ts[i] = elem\n\t\t\t\t}\n\t\t\t}\n\t\t\tcp[k] = s\n\t\tdefault:\n\t\t\tcp[k] = v\n\t\t}\n\t}\n\treturn cp\n}\n\n// purgeBatchSize is the number of keys deleted per batch commit during\n// orphaned keystore cleanup. Each commit is a cancellation checkpoint.\nconst purgeBatchSize = 1 << 12 // 4096\n\n// purgeOrphanedKeystoreData deletes all keys under /provider/keystore/ from the\n// shared repo datastore. These were written by older Kubo versions that stored\n// provider keystore data inline in the shared datastore. The new code uses\n// separate filesystem datastores under /{KeystoreDatastorePath}/ instead.\n//\n// The operation is idempotent and safe to interrupt: partial completion is\n// fine because already-deleted keys are no-ops on re-run.\nfunc purgeOrphanedKeystoreData(ctx context.Context, ds datastore.Batching) error {\n\torphanedPrefix := providerDatastoreKey.Child(keystoreDatastoreKey).String()\n\tsyncKey := datastore.NewKey(orphanedPrefix)\n\n\tresults, err := ds.Query(ctx, query.Query{\n\t\tPrefix: orphanedPrefix,\n\t\tKeysOnly: true,\n\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"querying orphaned keystore data: %w\", err)\n\t}\n\tdefer results.Close()\n\n\tvar batch datastore.Batch\n\tvar count, pending int\n\tfor result := range results.Next() {\n\t\tif ctx.Err() != nil {\n\t\t\treturn ctx.Err()\n\t\t}\n\t\tif result.Error != nil {\n\t\t\treturn fmt.Errorf(\"iterating orphaned keystore data: %w\", result.Error)\n\t\t}\n\t\tif batch == nil {\n\t\t\tbatch, err = ds.Batch(ctx)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"creating batch for orphaned keystore cleanup: %w\", err)\n\t\t\t}\n\t\t}\n\t\tif err := batch.Delete(ctx, datastore.NewKey(result.Key)); err != nil {\n\t\t\treturn fmt.Errorf(\"batch deleting orphaned key %s: %w\", result.Key, err)\n\t\t}\n\t\tcount++\n\t\tpending++\n\t\tif pending >= purgeBatchSize {\n\t\t\tif err := batch.Commit(ctx); err != nil {\n\t\t\t\treturn fmt.Errorf(\"committing orphaned keystore cleanup batch: %w\", err)\n\t\t\t}\n\t\t\tif err := ds.Sync(ctx, syncKey); err != nil {\n\t\t\t\treturn fmt.Errorf(\"syncing orphaned keystore cleanup: %w\", err)\n\t\t\t}\n\t\t\tbatch = nil\n\t\t\tpending = 0\n\t\t}\n\t}\n\tif pending > 0 {\n\t\tif err := batch.Commit(ctx); err != nil {\n\t\t\treturn fmt.Errorf(\"committing orphaned keystore cleanup batch: %w\", err)\n\t\t}\n\t\tif err := ds.Sync(ctx, syncKey); err != nil {\n\t\t\treturn fmt.Errorf(\"syncing orphaned keystore cleanup: %w\", err)\n\t\t}\n\t}\n\tif count > 0 {\n\t\tlogger.Infow(\"purged orphaned provider keystore data from shared datastore\", \"keys\", count)\n\t}\n\treturn nil\n}\n\nfunc SweepingProviderOpt(cfg *config.Config) fx.Option {\n\treprovideInterval := cfg.Provide.DHT.Interval.WithDefault(config.DefaultProvideDHTInterval)\n\ttype providerInput struct {\n\t\tfx.In\n\t\tDHT routing.Routing `name:\"dhtc\"`\n\t\tRepo repo.Repo\n\t\tLc fx.Lifecycle\n\t}\n\tsweepingReprovider := fx.Provide(func(in providerInput) (DHTProvider, *keystore.ResettableKeystore, error) {\n\t\tds := namespace.Wrap(in.Repo.Datastore(), providerDatastoreKey)\n\n\t\t// Get repo path and config to determine datastore type\n\t\trepoPath := in.Repo.Path()\n\t\trepoCfg, err := in.Repo.Config()\n\t\tif err != nil {\n\t\t\treturn nil, nil, fmt.Errorf(\"getting repo config: %w\", err)\n\t\t}\n\n\t\t// Find the root datastore type (levelds, pebbleds, etc.)\n\t\trootSpec := findRootDatastoreSpec(repoCfg.Datastore.Spec)\n\n\t\t// Keystore datastores live at /{KeystoreDatastorePath}/\n\t\tkeystoreBasePath := filepath.Join(repoPath, KeystoreDatastorePath)\n\n\t\tcreateDs := func(suffix string) (datastore.Batching, error) {\n\t\t\tif err := validateKeystoreSuffix(suffix); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\t// When no datastore spec is configured (e.g., test/mock repos),\n\t\t\t// fall back to an in-memory datastore.\n\t\t\tif rootSpec == nil {\n\t\t\t\treturn datastore.NewMapDatastore(), nil\n\t\t\t}\n\t\t\tif err := os.MkdirAll(keystoreBasePath, 0o755); err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"creating keystore base directory: %w\", err)\n\t\t\t}\n\t\t\tds, err := openDatastoreAt(rootSpec, filepath.Join(keystoreBasePath, suffix))\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tlogger.Infow(\"provider keystore: opened datastore\", \"suffix\", suffix, \"path\", filepath.Join(keystoreBasePath, suffix))\n\t\t\treturn ds, nil\n\t\t}\n\n\t\tdestroyDs := func(suffix string) error {\n\t\t\tif err := validateKeystoreSuffix(suffix); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tlogger.Infow(\"provider keystore: removing datastore from disk\", \"suffix\", suffix, \"path\", filepath.Join(keystoreBasePath, suffix))\n\t\t\treturn os.RemoveAll(filepath.Join(keystoreBasePath, suffix))\n\t\t}\n\n\t\t// One-time cleanup of stale keystore data left by older Kubo in the\n\t\t// shared repo datastore under /provider/keystore/. New code stores\n\t\t// bulk key data in separate filesystem datastores under\n\t\t// /{KeystoreDatastorePath}/ while still using the same\n\t\t// /provider/keystore/ namespace in the shared datastore for metadata.\n\t\t//\n\t\t// The absence of the keystoreBasePath directory signals a first run\n\t\t// after upgrade: the directory is created later by createDs on first\n\t\t// use, so it doubles as a \"cleanup done\" flag. If the process dies\n\t\t// mid-purge the directory still won't exist and the cleanup re-runs\n\t\t// on next start (it is idempotent). Must run synchronously before\n\t\t// NewResettableKeystore to avoid racing with reads on the same\n\t\t// namespace.\n\t\tif _, statErr := os.Stat(keystoreBasePath); os.IsNotExist(statErr) {\n\t\t\tlogger.Infow(\"migrating provider keystore data from shared datastore to separate filesystem datastores\", \"path\", keystoreBasePath)\n\t\t\t// Create a cancellable context for the purge. The OnStop hook\n\t\t\t// below calls purgeCancel when the node receives a shutdown\n\t\t\t// signal (e.g., SIGINT), which interrupts the purge loop\n\t\t\t// instead of blocking indefinitely.\n\t\t\tpurgeCtx, purgeCancel := context.WithCancel(context.Background())\n\t\t\tin.Lc.Append(fx.Hook{\n\t\t\t\tOnStop: func(_ context.Context) error {\n\t\t\t\t\tpurgeCancel()\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t})\n\t\t\tif purgeErr := purgeOrphanedKeystoreData(purgeCtx, in.Repo.Datastore()); purgeErr != nil {\n\t\t\t\tif purgeCtx.Err() != nil {\n\t\t\t\t\tlogger.Infow(\"provider keystore migration interrupted by shutdown, will resume on next start\")\n\t\t\t\t} else {\n\t\t\t\t\tlogger.Warnw(\"provider keystore migration failed, will retry on next start\", \"error\", purgeErr)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tlogger.Infow(\"provider keystore migration completed\")\n\t\t\t}\n\t\t\tpurgeCancel()\n\t\t}\n\n\t\tkeystoreDs := namespace.Wrap(ds, keystoreDatastoreKey)\n\t\tks, err := keystore.NewResettableKeystore(keystoreDs,\n\t\t\tkeystore.WithDatastoreFactory(createDs, destroyDs),\n\t\t\tkeystore.KeystoreOption(\n\t\t\t\tkeystore.WithPrefixBits(16),\n\t\t\t\tkeystore.WithBatchSize(int(cfg.Provide.DHT.KeystoreBatchSize.WithDefault(config.DefaultProvideDHTKeystoreBatchSize))),\n\t\t\t),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, nil, err\n\t\t}\n\t\t// Constants for buffered provider configuration\n\t\t// These values match the upstream defaults from go-libp2p-kad-dht and have been battle-tested\n\t\tconst (\n\t\t\t// bufferedDsName is the datastore namespace used by the buffered provider.\n\t\t\t// The dsqueue persists operations here to handle large data additions without\n\t\t\t// being memory-bound, allowing operations on hardware with limited RAM and\n\t\t\t// enabling core operations to return instantly while processing happens async.\n\t\t\tbufferedDsName = \"bprov\"\n\n\t\t\t// bufferedBatchSize controls how many operations are dequeued and processed\n\t\t\t// together from the datastore queue. The worker processes up to this many\n\t\t\t// operations at once, grouping them by type for efficiency.\n\t\t\tbufferedBatchSize = 1 << 10 // 1024 items\n\n\t\t\t// bufferedIdleWriteTime is an implementation detail of go-dsqueue that controls\n\t\t\t// how long the datastore buffer waits for new multihashes to arrive before\n\t\t\t// flushing in-memory items to the datastore. This does NOT affect providing speed -\n\t\t\t// provides happen as fast as possible via a dedicated worker that continuously\n\t\t\t// processes the queue regardless of this timing.\n\t\t\tbufferedIdleWriteTime = time.Minute\n\n\t\t\t// loggerName is the name of the go-log logger used by the provider.\n\t\t\tloggerName = dhtprovider.DefaultLoggerName\n\t\t)\n\n\t\tbufferedProviderOpts := []buffered.Option{\n\t\t\tbuffered.WithBatchSize(bufferedBatchSize),\n\t\t\tbuffered.WithDsName(bufferedDsName),\n\t\t\tbuffered.WithIdleWriteTime(bufferedIdleWriteTime),\n\t\t}\n\t\tvar impl dhtImpl\n\t\tswitch inDht := in.DHT.(type) {\n\t\tcase *dht.IpfsDHT:\n\t\t\tif inDht != nil {\n\t\t\t\timpl = inDht\n\t\t\t}\n\t\tcase *dual.DHT:\n\t\t\tif inDht != nil {\n\t\t\t\tprov, err := ddhtprovider.New(inDht,\n\t\t\t\t\tddhtprovider.WithKeystore(ks),\n\t\t\t\t\tddhtprovider.WithDatastore(ds),\n\t\t\t\t\tddhtprovider.WithResumeCycle(cfg.Provide.DHT.ResumeEnabled.WithDefault(config.DefaultProvideDHTResumeEnabled)),\n\n\t\t\t\t\tddhtprovider.WithReprovideInterval(reprovideInterval),\n\t\t\t\t\tddhtprovider.WithMaxReprovideDelay(time.Hour),\n\t\t\t\t\tddhtprovider.WithOfflineDelay(cfg.Provide.DHT.OfflineDelay.WithDefault(config.DefaultProvideDHTOfflineDelay)),\n\t\t\t\t\tddhtprovider.WithConnectivityCheckOnlineInterval(1*time.Minute),\n\n\t\t\t\t\tddhtprovider.WithMaxWorkers(int(cfg.Provide.DHT.MaxWorkers.WithDefault(config.DefaultProvideDHTMaxWorkers))),\n\t\t\t\t\tddhtprovider.WithDedicatedPeriodicWorkers(int(cfg.Provide.DHT.DedicatedPeriodicWorkers.WithDefault(config.DefaultProvideDHTDedicatedPeriodicWorkers))),\n\t\t\t\t\tddhtprovider.WithDedicatedBurstWorkers(int(cfg.Provide.DHT.DedicatedBurstWorkers.WithDefault(config.DefaultProvideDHTDedicatedBurstWorkers))),\n\t\t\t\t\tddhtprovider.WithMaxProvideConnsPerWorker(int(cfg.Provide.DHT.MaxProvideConnsPerWorker.WithDefault(config.DefaultProvideDHTMaxProvideConnsPerWorker))),\n\n\t\t\t\t\tddhtprovider.WithLoggerName(loggerName),\n\t\t\t\t)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, nil, err\n\t\t\t\t}\n\t\t\t\treturn buffered.New(prov, ds, bufferedProviderOpts...), ks, nil\n\t\t\t}\n\t\tcase *fullrt.FullRT:\n\t\t\tif inDht != nil {\n\t\t\t\timpl = newFullRTRouter(inDht, loggerName)\n\t\t\t}\n\t\t}\n\t\tif impl == nil {\n\t\t\treturn &NoopProvider{}, nil, nil\n\t\t}\n\n\t\tvar selfAddrsFunc func() []ma.Multiaddr\n\t\tif imlpFilter, ok := impl.(addrsFilter); ok {\n\t\t\tselfAddrsFunc = imlpFilter.FilteredAddrs\n\t\t} else {\n\t\t\tselfAddrsFunc = func() []ma.Multiaddr { return impl.Host().Addrs() }\n\t\t}\n\t\topts := []dhtprovider.Option{\n\t\t\tdhtprovider.WithKeystore(ks),\n\t\t\tdhtprovider.WithDatastore(ds),\n\t\t\tdhtprovider.WithResumeCycle(cfg.Provide.DHT.ResumeEnabled.WithDefault(config.DefaultProvideDHTResumeEnabled)),\n\t\t\tdhtprovider.WithHost(impl.Host()),\n\t\t\tdhtprovider.WithRouter(impl),\n\t\t\tdhtprovider.WithMessageSender(impl.MessageSender()),\n\t\t\tdhtprovider.WithSelfAddrs(selfAddrsFunc),\n\t\t\tdhtprovider.WithAddLocalRecord(func(h mh.Multihash) error {\n\t\t\t\treturn impl.Provide(context.Background(), cid.NewCidV1(cid.Raw, h), false)\n\t\t\t}),\n\n\t\t\tdhtprovider.WithReplicationFactor(amino.DefaultBucketSize),\n\t\t\tdhtprovider.WithReprovideInterval(reprovideInterval),\n\t\t\tdhtprovider.WithMaxReprovideDelay(time.Hour),\n\t\t\tdhtprovider.WithOfflineDelay(cfg.Provide.DHT.OfflineDelay.WithDefault(config.DefaultProvideDHTOfflineDelay)),\n\t\t\tdhtprovider.WithConnectivityCheckOnlineInterval(1 * time.Minute),\n\n\t\t\tdhtprovider.WithMaxWorkers(int(cfg.Provide.DHT.MaxWorkers.WithDefault(config.DefaultProvideDHTMaxWorkers))),\n\t\t\tdhtprovider.WithDedicatedPeriodicWorkers(int(cfg.Provide.DHT.DedicatedPeriodicWorkers.WithDefault(config.DefaultProvideDHTDedicatedPeriodicWorkers))),\n\t\t\tdhtprovider.WithDedicatedBurstWorkers(int(cfg.Provide.DHT.DedicatedBurstWorkers.WithDefault(config.DefaultProvideDHTDedicatedBurstWorkers))),\n\t\t\tdhtprovider.WithMaxProvideConnsPerWorker(int(cfg.Provide.DHT.MaxProvideConnsPerWorker.WithDefault(config.DefaultProvideDHTMaxProvideConnsPerWorker))),\n\n\t\t\tdhtprovider.WithLoggerName(loggerName),\n\t\t}\n\n\t\tprov, err := dhtprovider.New(opts...)\n\t\tif err != nil {\n\t\t\treturn nil, nil, err\n\t\t}\n\t\treturn buffered.New(prov, ds, bufferedProviderOpts...), ks, nil\n\t})\n\n\ttype keystoreInput struct {\n\t\tfx.In\n\t\tProvider DHTProvider\n\t\tKeystore *keystore.ResettableKeystore\n\t\tKeyProvider provider.KeyChanFunc\n\t}\n\tinitKeystore := fx.Invoke(func(lc fx.Lifecycle, in keystoreInput) {\n\t\t// Skip keystore initialization for NoopProvider\n\t\tif _, ok := in.Provider.(*NoopProvider); ok {\n\t\t\treturn\n\t\t}\n\n\t\tvar (\n\t\t\tcancel context.CancelFunc\n\t\t\tdone = make(chan struct{})\n\t\t)\n\n\t\tsyncKeystore := func(ctx context.Context) error {\n\t\t\tkcf, err := in.KeyProvider(ctx)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif err := in.Keystore.ResetCids(ctx, kcf); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif err := in.Provider.RefreshSchedule(); err != nil {\n\t\t\t\tlogger.Infow(\"refreshing provider schedule\", \"err\", err)\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStart: func(ctx context.Context) error {\n\t\t\t\t// Set the KeyProvider as a garbage collection function for the\n\t\t\t\t// keystore. Periodically purge the Keystore from all its keys and\n\t\t\t\t// replace them with the keys that needs to be reprovided, coming from\n\t\t\t\t// the KeyChanFunc. So far, this is the less worse way to remove CIDs\n\t\t\t\t// that shouldn't be reprovided from the provider's state.\n\t\t\t\tgo func() {\n\t\t\t\t\t// Sync the keystore once at startup. This operation is async since\n\t\t\t\t\t// we need to walk the DAG of objects matching the provide strategy,\n\t\t\t\t\t// which can take a while.\n\t\t\t\t\tstrategy := cfg.Provide.Strategy.WithDefault(config.DefaultProvideStrategy)\n\t\t\t\t\tlogger.Infow(\"provider keystore sync started\", \"strategy\", strategy)\n\t\t\t\t\tif err := syncKeystore(ctx); err != nil {\n\t\t\t\t\t\tif ctx.Err() == nil {\n\t\t\t\t\t\t\tlogger.Errorw(\"provider keystore sync failed\", \"err\", err, \"strategy\", strategy)\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tlogger.Debugw(\"provider keystore sync interrupted by shutdown\", \"err\", err, \"strategy\", strategy)\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tlogger.Infow(\"provider keystore sync completed\", \"strategy\", strategy)\n\t\t\t\t}()\n\n\t\t\t\tgcCtx, c := context.WithCancel(context.Background())\n\t\t\t\tcancel = c\n\n\t\t\t\tgo func() { // garbage collection loop for cids to reprovide\n\t\t\t\t\tdefer close(done)\n\t\t\t\t\tticker := time.NewTicker(reprovideInterval)\n\t\t\t\t\tdefer ticker.Stop()\n\n\t\t\t\t\tfor {\n\t\t\t\t\t\tselect {\n\t\t\t\t\t\tcase <-gcCtx.Done():\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\tcase <-ticker.C:\n\t\t\t\t\t\t\tif err := syncKeystore(gcCtx); err != nil {\n\t\t\t\t\t\t\t\tlogger.Errorw(\"provider keystore sync\", \"err\", err)\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\tif cancel != nil {\n\t\t\t\t\tcancel()\n\t\t\t\t}\n\t\t\t\tselect {\n\t\t\t\tcase <-done:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn ctx.Err()\n\t\t\t\t}\n\t\t\t\t// Keystore will be closed by ensureProviderClosesBeforeKeystore hook\n\t\t\t\t// to guarantee provider closes before keystore.\n\t\t\t\treturn nil\n\t\t\t},\n\t\t})\n\t})\n\n\t// ensureProviderClosesBeforeKeystore manages the shutdown order between\n\t// provider and keystore to prevent race conditions.\n\t//\n\t// The provider's worker goroutines may call keystore methods during their\n\t// operation. If keystore closes while these operations are in-flight, we get\n\t// \"keystore is closed\" errors. By closing the provider first, we ensure all\n\t// worker goroutines exit and complete any pending keystore operations before\n\t// the keystore itself closes.\n\ttype providerKeystoreShutdownInput struct {\n\t\tfx.In\n\t\tProvider DHTProvider\n\t\tKeystore *keystore.ResettableKeystore\n\t}\n\tensureProviderClosesBeforeKeystore := fx.Invoke(func(lc fx.Lifecycle, in providerKeystoreShutdownInput) {\n\t\t// Skip for NoopProvider\n\t\tif _, ok := in.Provider.(*NoopProvider); ok {\n\t\t\treturn\n\t\t}\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\t// Close provider first - waits for all worker goroutines to exit.\n\t\t\t\t// This ensures no code can access keystore after this returns.\n\t\t\t\tif err := in.Provider.Close(); err != nil {\n\t\t\t\t\tlogger.Errorw(\"error closing provider during shutdown\", \"error\", err)\n\t\t\t\t}\n\n\t\t\t\t// Close keystore - safe now, provider is fully shut down\n\t\t\t\treturn in.Keystore.Close()\n\t\t\t},\n\t\t})\n\t})\n\n\t// extractSweepingProvider extracts a SweepingProvider from the given provider interface.\n\t// It handles unwrapping buffered and dual providers, always selecting WAN for dual DHT.\n\t// Returns nil if the provider is not a sweeping provider type.\n\tvar extractSweepingProvider func(prov any) *dhtprovider.SweepingProvider\n\textractSweepingProvider = func(prov any) *dhtprovider.SweepingProvider {\n\t\tswitch p := prov.(type) {\n\t\tcase *dhtprovider.SweepingProvider:\n\t\t\treturn p\n\t\tcase *ddhtprovider.SweepingProvider:\n\t\t\treturn p.WAN\n\t\tcase *buffered.SweepingProvider:\n\t\t\t// Recursively extract from the inner provider\n\t\t\treturn extractSweepingProvider(p.Provider)\n\t\tdefault:\n\t\t\treturn nil\n\t\t}\n\t}\n\n\ttype alertInput struct {\n\t\tfx.In\n\t\tProvider DHTProvider\n\t}\n\treprovideAlert := fx.Invoke(func(lc fx.Lifecycle, in alertInput) {\n\t\tprov := extractSweepingProvider(in.Provider)\n\t\tif prov == nil {\n\t\t\treturn\n\t\t}\n\n\t\tvar (\n\t\t\tcancel context.CancelFunc\n\t\t\tdone = make(chan struct{})\n\t\t)\n\n\t\tlc.Append(fx.Hook{\n\t\t\tOnStart: func(ctx context.Context) error {\n\t\t\t\tgcCtx, c := context.WithCancel(context.Background())\n\t\t\t\tcancel = c\n\t\t\t\tgo func() {\n\t\t\t\t\tdefer close(done)\n\n\t\t\t\t\tticker := time.NewTicker(reprovideAlertPollInterval)\n\t\t\t\t\tdefer ticker.Stop()\n\n\t\t\t\t\tvar (\n\t\t\t\t\t\tqueueSize, prevQueueSize int64\n\t\t\t\t\t\tqueuedWorkers, prevQueuedWorkers bool\n\t\t\t\t\t\tcount int\n\t\t\t\t\t)\n\n\t\t\t\t\tfor {\n\t\t\t\t\t\tselect {\n\t\t\t\t\t\tcase <-gcCtx.Done():\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\tcase <-ticker.C:\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tstats := prov.Stats()\n\t\t\t\t\t\tqueuedWorkers = stats.Workers.QueuedPeriodic > 0\n\t\t\t\t\t\tqueueSize = int64(stats.Queues.PendingRegionReprovides)\n\n\t\t\t\t\t\t// Alert if reprovide queue keeps growing and all periodic workers are busy.\n\t\t\t\t\t\t// Requires consecutiveAlertsThreshold intervals of sustained growth.\n\t\t\t\t\t\tif prevQueuedWorkers && queuedWorkers && queueSize > prevQueueSize {\n\t\t\t\t\t\t\tcount++\n\t\t\t\t\t\t\tif count >= consecutiveAlertsThreshold {\n\t\t\t\t\t\t\t\tlogger.Errorf(`\n🔔🔔🔔 Reprovide Operations Too Slow 🔔🔔🔔\n\nYour node is falling behind on DHT reprovides, which will affect content availability.\n\nKeyspace regions enqueued for reprovide:\n %s ago:\\t%d\n Now:\\t%d\n\nAll periodic workers are busy!\n Active workers:\\t%d / %d (max)\n Active workers types:\\t%d periodic, %d burst\n Dedicated workers:\\t%d periodic, %d burst\n\nSolutions (try in order):\n1. Increase Provide.DHT.MaxWorkers (current %d)\n2. Increase Provide.DHT.DedicatedPeriodicWorkers (current %d)\n3. Set Provide.DHT.SweepEnabled=false and Routing.AcceleratedDHTClient=true (last resort, not recommended)\n\nSee how the reprovide queue is processed in real-time with 'watch ipfs provide stat --all --compact'\n\nSee docs: https://github.com/ipfs/kubo/blob/master/docs/config.md#providedhtmaxworkers`,\n\t\t\t\t\t\t\t\t\treprovideAlertPollInterval.Truncate(time.Minute).String(), prevQueueSize, queueSize,\n\t\t\t\t\t\t\t\t\tstats.Workers.Active, stats.Workers.Max,\n\t\t\t\t\t\t\t\t\tstats.Workers.ActivePeriodic, stats.Workers.ActiveBurst,\n\t\t\t\t\t\t\t\t\tstats.Workers.DedicatedPeriodic, stats.Workers.DedicatedBurst,\n\t\t\t\t\t\t\t\t\tstats.Workers.Max, stats.Workers.DedicatedPeriodic)\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t} else if !queuedWorkers {\n\t\t\t\t\t\t\tcount = 0\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tprevQueueSize, prevQueuedWorkers = queueSize, queuedWorkers\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tOnStop: func(ctx context.Context) error {\n\t\t\t\t// Cancel the alert loop\n\t\t\t\tif cancel != nil {\n\t\t\t\t\tcancel()\n\t\t\t\t}\n\t\t\t\tselect {\n\t\t\t\tcase <-done:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn ctx.Err()\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t},\n\t\t})\n\t})\n\n\treturn fx.Options(\n\t\tsweepingReprovider,\n\t\tinitKeystore,\n\t\tensureProviderClosesBeforeKeystore,\n\t\treprovideAlert,\n\t)\n}\n\n// ONLINE/OFFLINE\n\n// hasDHTRouting checks if the routing configuration includes a DHT component.\n// Returns false for HTTP-only custom routing configurations (e.g., Routing.Type=\"custom\"\n// with only HTTP routers). This is used to determine whether SweepingProviderOpt\n// can be used, since it requires a DHT client.\nfunc hasDHTRouting(cfg *config.Config) bool {\n\troutingType := cfg.Routing.Type.WithDefault(config.DefaultRoutingType)\n\tswitch routingType {\n\tcase \"auto\", \"autoclient\", \"dht\", \"dhtclient\", \"dhtserver\":\n\t\treturn true\n\tcase \"custom\":\n\t\t// Check if any router in custom config is DHT-based\n\t\tfor _, router := range cfg.Routing.Routers {\n\t\t\tif routerIncludesDHT(router, cfg) {\n\t\t\t\treturn true\n\t\t\t}\n\t\t}\n\t\treturn false\n\tdefault: // \"none\", \"delegated\"\n\t\treturn false\n\t}\n}\n\n// routerIncludesDHT recursively checks if a router configuration includes DHT.\n// Handles parallel and sequential composite routers by checking their children.\nfunc routerIncludesDHT(rp config.RouterParser, cfg *config.Config) bool {\n\tswitch rp.Type {\n\tcase config.RouterTypeDHT:\n\t\treturn true\n\tcase config.RouterTypeParallel, config.RouterTypeSequential:\n\t\tif children, ok := rp.Parameters.(*config.ComposableRouterParams); ok {\n\t\t\tfor _, child := range children.Routers {\n\t\t\t\tif childRouter, exists := cfg.Routing.Routers[child.RouterName]; exists {\n\t\t\t\t\tif routerIncludesDHT(childRouter, cfg) {\n\t\t\t\t\t\treturn true\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn false\n}\n\n// OnlineProviders groups units managing provide routing records online\nfunc OnlineProviders(provide bool, cfg *config.Config) fx.Option {\n\tif !provide {\n\t\treturn OfflineProviders()\n\t}\n\n\tproviderStrategy := cfg.Provide.Strategy.WithDefault(config.DefaultProvideStrategy)\n\n\tstrategyFlag := config.ParseProvideStrategy(providerStrategy)\n\tif strategyFlag == 0 {\n\t\treturn fx.Error(fmt.Errorf(\"provider: unknown strategy %q\", providerStrategy))\n\t}\n\n\topts := []fx.Option{\n\t\tfx.Provide(setReproviderKeyProvider(providerStrategy)),\n\t}\n\n\tsweepEnabled := cfg.Provide.DHT.SweepEnabled.WithDefault(config.DefaultProvideDHTSweepEnabled)\n\tdhtAvailable := hasDHTRouting(cfg)\n\n\t// Use SweepingProvider only when both sweep is enabled AND DHT is available.\n\t// For HTTP-only routing (e.g., Routing.Type=\"custom\" with only HTTP routers),\n\t// fall back to LegacyProvider which works with ProvideManyRouter.\n\t// See https://github.com/ipfs/kubo/issues/11089\n\tif sweepEnabled && dhtAvailable {\n\t\topts = append(opts, SweepingProviderOpt(cfg))\n\t} else {\n\t\treprovideInterval := cfg.Provide.DHT.Interval.WithDefault(config.DefaultProvideDHTInterval)\n\t\tacceleratedDHTClient := cfg.Routing.AcceleratedDHTClient.WithDefault(config.DefaultAcceleratedDHTClient)\n\t\tprovideWorkerCount := int(cfg.Provide.DHT.MaxWorkers.WithDefault(config.DefaultProvideDHTMaxWorkers))\n\n\t\topts = append(opts, LegacyProviderOpt(reprovideInterval, providerStrategy, acceleratedDHTClient, provideWorkerCount))\n\t}\n\n\treturn fx.Options(opts...)\n}\n\n// OfflineProviders groups units managing provide routing records offline\nfunc OfflineProviders() fx.Option {\n\treturn fx.Provide(func() DHTProvider {\n\t\treturn &NoopProvider{}\n\t})\n}\n\nfunc mfsProvider(mfsRoot *mfs.Root, fetcher fetcher.Factory) provider.KeyChanFunc {\n\treturn func(ctx context.Context) (<-chan cid.Cid, error) {\n\t\terr := mfsRoot.FlushMemFree(ctx)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"provider: error flushing MFS, cannot provide MFS: %w\", err)\n\t\t}\n\t\trootNode, err := mfsRoot.GetDirectory().GetNode()\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"provider: error loading MFS root, cannot provide MFS: %w\", err)\n\t\t}\n\n\t\tkcf := provider.NewDAGProvider(rootNode.Cid(), fetcher)\n\t\treturn kcf(ctx)\n\t}\n}\n\ntype provStrategyIn struct {\n\tfx.In\n\tPinner pin.Pinner\n\tBlockstore blockstore.Blockstore\n\tOfflineIPLDFetcher fetcher.Factory `name:\"offlineIpldFetcher\"`\n\tOfflineUnixFSFetcher fetcher.Factory `name:\"offlineUnixfsFetcher\"`\n\tMFSRoot *mfs.Root\n\tRepo repo.Repo\n}\n\ntype provStrategyOut struct {\n\tfx.Out\n\tProvidingStrategy config.ProvideStrategy\n\tProvidingKeyChanFunc provider.KeyChanFunc\n}\n\n// createKeyProvider creates the appropriate KeyChanFunc based on strategy.\n// Each strategy has different behavior:\n// - \"roots\": Only root CIDs of pinned content\n// - \"pinned\": All pinned content (roots + children)\n// - \"mfs\": Only MFS content\n// - \"all\": all blocks\nfunc createKeyProvider(strategyFlag config.ProvideStrategy, in provStrategyIn) provider.KeyChanFunc {\n\tswitch strategyFlag {\n\tcase config.ProvideStrategyRoots:\n\t\treturn provider.NewBufferedProvider(dspinner.NewPinnedProvider(true, in.Pinner, in.OfflineIPLDFetcher))\n\tcase config.ProvideStrategyPinned:\n\t\treturn provider.NewBufferedProvider(dspinner.NewPinnedProvider(false, in.Pinner, in.OfflineIPLDFetcher))\n\tcase config.ProvideStrategyPinned | config.ProvideStrategyMFS:\n\t\treturn provider.NewPrioritizedProvider(\n\t\t\tprovider.NewBufferedProvider(dspinner.NewPinnedProvider(false, in.Pinner, in.OfflineIPLDFetcher)),\n\t\t\tmfsProvider(in.MFSRoot, in.OfflineUnixFSFetcher),\n\t\t)\n\tcase config.ProvideStrategyMFS:\n\t\treturn mfsProvider(in.MFSRoot, in.OfflineUnixFSFetcher)\n\tdefault: // \"all\", \"\", \"flat\" (compat)\n\t\treturn in.Blockstore.AllKeysChan\n\t}\n}\n\n// detectStrategyChange checks if the reproviding strategy has changed from what's persisted.\n// Returns: (previousStrategy, hasChanged, error)\nfunc detectStrategyChange(ctx context.Context, strategy string, ds datastore.Datastore) (string, bool, error) {\n\tstrategyKey := datastore.NewKey(reprovideStrategyKey)\n\n\tprev, err := ds.Get(ctx, strategyKey)\n\tif err != nil {\n\t\tif errors.Is(err, datastore.ErrNotFound) {\n\t\t\treturn \"\", strategy != \"\", nil\n\t\t}\n\t\treturn \"\", false, err\n\t}\n\n\tpreviousStrategy := string(prev)\n\treturn previousStrategy, previousStrategy != strategy, nil\n}\n\n// persistStrategy saves the current reproviding strategy to the datastore.\n// Empty string strategies are deleted rather than stored.\nfunc persistStrategy(ctx context.Context, strategy string, ds datastore.Datastore) error {\n\tstrategyKey := datastore.NewKey(reprovideStrategyKey)\n\n\tif strategy == \"\" {\n\t\treturn ds.Delete(ctx, strategyKey)\n\t}\n\treturn ds.Put(ctx, strategyKey, []byte(strategy))\n}\n\n// handleStrategyChange manages strategy change detection and queue clearing.\n// Strategy change detection: when the reproviding strategy changes,\n// we clear the provide queue to avoid unexpected behavior from mixing\n// strategies. This ensures a clean transition between different providing modes.\nfunc handleStrategyChange(strategy string, provider DHTProvider, ds datastore.Datastore) {\n\tctx := context.Background()\n\n\tprevious, changed, err := detectStrategyChange(ctx, strategy, ds)\n\tif err != nil {\n\t\tlogger.Error(\"cannot read previous reprovide strategy\", \"err\", err)\n\t\treturn\n\t}\n\n\tif !changed {\n\t\treturn\n\t}\n\n\tlogger.Infow(\"Provide.Strategy changed, clearing provide queue\", \"previous\", previous, \"current\", strategy)\n\tprovider.Clear()\n\n\tif err := persistStrategy(ctx, strategy, ds); err != nil {\n\t\tlogger.Error(\"cannot update reprovide strategy\", \"err\", err)\n\t}\n}\n\nfunc setReproviderKeyProvider(strategy string) func(in provStrategyIn) provStrategyOut {\n\tstrategyFlag := config.ParseProvideStrategy(strategy)\n\n\treturn func(in provStrategyIn) provStrategyOut {\n\t\t// Create the appropriate key provider based on strategy\n\t\tkcf := createKeyProvider(strategyFlag, in)\n\t\treturn provStrategyOut{\n\t\t\tProvidingStrategy: strategyFlag,\n\t\t\tProvidingKeyChanFunc: kcf,\n\t\t}\n\t}\n}\n" }, { "path": "core/node/storage.go", "content": "package node\n\nimport (\n\tblockstore \"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/go-datastore\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"go.uber.org/fx\"\n\n\t\"github.com/ipfs/boxo/filestore\"\n\t\"github.com/ipfs/kubo/core/node/helpers\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/thirdparty/verifbs\"\n)\n\n// RepoConfig loads configuration from the repo\nfunc RepoConfig(repo repo.Repo) (*config.Config, error) {\n\tcfg, err := repo.Config()\n\treturn cfg, err\n}\n\n// Datastore provides the datastore\nfunc Datastore(repo repo.Repo) datastore.Datastore {\n\treturn repo.Datastore()\n}\n\n// BaseBlocks is the lower level blockstore without GC or Filestore layers\ntype BaseBlocks blockstore.Blockstore\n\n// BaseBlockstoreCtor creates cached blockstore backed by the provided datastore\nfunc BaseBlockstoreCtor(\n\tcacheOpts blockstore.CacheOpts,\n\thashOnRead bool,\n\twriteThrough bool,\n\tprovidingStrategy string,\n) func(mctx helpers.MetricsCtx, repo repo.Repo, prov DHTProvider, lc fx.Lifecycle) (bs BaseBlocks, err error) {\n\treturn func(mctx helpers.MetricsCtx, repo repo.Repo, prov DHTProvider, lc fx.Lifecycle) (bs BaseBlocks, err error) {\n\t\topts := []blockstore.Option{blockstore.WriteThrough(writeThrough)}\n\n\t\t// Blockstore providing integration:\n\t\t// When strategy includes \"all\" the blockstore directly provides blocks as they're Put.\n\t\t// Important: Provide calls from blockstore are intentionally BLOCKING.\n\t\t// The Provider implementation (not the blockstore) should handle concurrency/queuing.\n\t\t// This avoids spawning unbounded goroutines for concurrent block additions.\n\t\tstrategyFlag := config.ParseProvideStrategy(providingStrategy)\n\t\tif strategyFlag&config.ProvideStrategyAll != 0 {\n\t\t\topts = append(opts, blockstore.Provider(prov))\n\t\t}\n\n\t\t// hash security\n\t\tbs = blockstore.NewBlockstore(\n\t\t\trepo.Datastore(),\n\t\t\topts...,\n\t\t)\n\t\tbs = &verifbs.VerifBS{Blockstore: bs}\n\t\tbs, err = blockstore.CachedBlockstore(helpers.LifecycleCtx(mctx, lc), bs, cacheOpts)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tbs = blockstore.NewIdStore(bs)\n\n\t\tif hashOnRead {\n\t\t\tbs = &blockstore.ValidatingBlockstore{Blockstore: bs}\n\t\t}\n\n\t\treturn\n\t}\n}\n\n// GcBlockstoreCtor wraps the base blockstore with GC and Filestore layers\nfunc GcBlockstoreCtor(bb BaseBlocks) (gclocker blockstore.GCLocker, gcbs blockstore.GCBlockstore, bs blockstore.Blockstore) {\n\tgclocker = blockstore.NewGCLocker()\n\tgcbs = blockstore.NewGCBlockstore(bb, gclocker)\n\n\tbs = gcbs\n\treturn\n}\n\n// FilestoreBlockstoreCtor wraps GcBlockstore and adds Filestore support\nfunc FilestoreBlockstoreCtor(repo repo.Repo, bb BaseBlocks, prov DHTProvider) (gclocker blockstore.GCLocker, gcbs blockstore.GCBlockstore, bs blockstore.Blockstore, fstore *filestore.Filestore) {\n\tgclocker = blockstore.NewGCLocker()\n\n\t// hash security\n\tfstore = filestore.NewFilestore(bb, repo.FileManager(), prov)\n\tgcbs = blockstore.NewGCBlockstore(fstore, gclocker)\n\tgcbs = &verifbs.VerifBSGC{GCBlockstore: gcbs}\n\n\tbs = gcbs\n\treturn\n}\n" }, { "path": "coverage/.gitignore", "content": "unitcover\nsharnesscover\nipfs\n\n" }, { "path": "coverage/Rules.mk", "content": "include mk/header.mk\n\nGOCC ?= go\n\n$(d)/coverage_deps: $$(DEPS_GO) cmd/ipfs/ipfs\n\trm -rf $(@D)/sharnesscover && mkdir $(@D)/sharnesscover\n\n.PHONY: $(d)/coverage_deps\n\n# unit tests coverage is now produced by test_unit target in mk/golang.mk\n# (outputs coverage/unit_tests.coverprofile and test/unit/gotest.json)\n\nTGTS_$(d) :=\n\n# sharness tests coverage\n$(d)/ipfs: GOTAGS += testrunmain\n$(d)/ipfs: $(d)/main\n\t$(go-build-relative)\n\nCLEAN += $(d)/ipfs\n\nifneq ($(filter coverage%,$(MAKECMDGOALS)),)\n\t# this is quite hacky but it is best way I could figure out\n\tDEPS_test/sharness += cmd/ipfs/ipfs-test-cover $(d)/coverage_deps $(d)/ipfs\nendif\n\nexport IPFS_COVER_DIR:= $(realpath $(d))/sharnesscover/\n\n$(d)/sharness_tests.coverprofile: export TEST_PLUGIN=0\n$(d)/sharness_tests.coverprofile: $(d)/ipfs cmd/ipfs/ipfs-test-cover $(d)/coverage_deps test/bin/gocovmerge test_sharness\n\t(cd $(@D)/sharnesscover && find . -type f | gocovmerge -list -) > $@\n\n\nPATH := $(realpath $(d)):$(PATH)\n\nTGTS_$(d) += $(d)/sharness_tests.coverprofile\n\nCLEAN += $(TGTS_$(d))\nCOVERAGE += $(TGTS_$(d))\n\ninclude mk/footer.mk\n" }, { "path": "coverage/main/main.go", "content": "//go:build testrunmain\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"os/signal\"\n\t\"strconv\"\n\t\"syscall\"\n)\n\nfunc main() {\n\tcoverDir := os.Getenv(\"IPFS_COVER_DIR\")\n\tif len(coverDir) == 0 {\n\t\tfmt.Println(\"IPFS_COVER_DIR not defined\")\n\t\tos.Exit(1)\n\t}\n\tcoverFile, err := os.CreateTemp(coverDir, \"coverage-\")\n\tif err != nil {\n\t\tfmt.Println(err.Error())\n\t\tos.Exit(1)\n\t}\n\n\tretFile, err := os.CreateTemp(\"\", \"cover-ret-file\")\n\tif err != nil {\n\t\tfmt.Println(err.Error())\n\t\tos.Exit(1)\n\t}\n\n\targs := []string{\"-test.run\", \"^TestRunMain$\", \"-test.coverprofile=\" + coverFile.Name(), \"--\"}\n\targs = append(args, os.Args[1:]...)\n\n\tp := exec.Command(\"ipfs-test-cover\", args...)\n\tp.Stdin = os.Stdin\n\tp.Stdout = os.Stdout\n\tp.Stderr = os.Stderr\n\tp.Env = append(os.Environ(), \"IPFS_COVER_RET_FILE=\"+retFile.Name())\n\n\tp.SysProcAttr = &syscall.SysProcAttr{\n\t\tPdeathsig: syscall.SIGTERM,\n\t}\n\n\tsig := make(chan os.Signal, 10)\n\tstart := make(chan struct{})\n\tgo func() {\n\t\t<-start\n\t\tfor {\n\t\t\tp.Process.Signal(<-sig)\n\t\t}\n\t}()\n\n\tsignal.Notify(sig, syscall.SIGHUP, syscall.SIGINT, syscall.SIGTERM)\n\n\terr = p.Start()\n\tif err != nil {\n\t\tfmt.Println(err.Error())\n\t\tos.Exit(1)\n\t}\n\n\tclose(start)\n\n\terr = p.Wait()\n\tif err != nil {\n\t\tfmt.Println(err.Error())\n\t\tos.Exit(1)\n\t}\n\n\tb, err := io.ReadAll(retFile)\n\tif err != nil {\n\t\tfmt.Println(err.Error())\n\t\tos.Exit(1)\n\t}\n\tb = b[:len(b)-1]\n\td, err := strconv.Atoi(string(b))\n\tif err != nil {\n\t\tfmt.Println(err.Error())\n\t\tos.Exit(1)\n\t}\n\tos.Exit(d)\n}\n" }, { "path": "doc.go", "content": "/*\nIPFS is a global, versioned, peer-to-peer filesystem\n*/\npackage ipfs\n" }, { "path": "docker-compose.yaml", "content": "version: '3.8'\nservices:\n ipfs:\n build: .\n restart: unless-stopped\n volumes:\n - ipfs_path:/data/ipfs\n - ipfs_fuse:/ipfs\n - ipns_fuse:/ipns\n environment:\n - IPFS_PATH=/data/ipfs\n ports:\n # Swarm listens on all interfaces, so is remotely reachable.\n - 4001:4001/tcp\n - 4001:4001/udp\n \n # The following ports only listen on the loopback interface, so are not remotely reachable by default.\n # If you want to override these or add more ports, see https://docs.docker.com/compose/extends/ .\n \n # API port, which includes admin operations, so you probably don't want this remotely accessible.\n - 127.0.0.1:5001:5001\n \n # HTTP Gateway\n - 127.0.0.1:8080:8080\nvolumes:\n ipfs_path:\n ipfs_fuse:\n ipns_fuse:\n" }, { "path": "docs/EARLY_TESTERS.md", "content": "# EARLY TESTERS PROGRAMME\n\n## What is it?\n\nThe early testers programme allows groups using Kubo in production to self-volunteer to help test `kubo` release candidates to ensure that no regressions that might affect production systems make it into the final release. While we invite the _entire_ community to help test releases, members of the early testers program are expected to participate directly and actively in every release.\n\n## What are the expectations?\n\nMembers of the early tester program are expected to work closely with us to:\n\n* Provide high quality, actionable feedback.\n* Work directly with us to debug regressions in the release.\n* Help ensure a rock-solid, timely release.\n\nWe will ask early testers to participate at two points in the process:\n\n* When Kubo enters the second release stage (public beta), early testers will be asked to test Kubo on non-production infrastructure. This may involve things like:\n - Running integration tests against the release candidate.\n - Running simulations/benchmarks on the release candidate.\n - Manually testing the release candidate to check for regressions.\n* When Kubo enters the third release stage (soft release), early testers will be asked to partially deploy the release candidate to production infrastructure. Release candidates at this stage are expected to be identical to the final release. However, this stage allows the Kubo team to fix any last-minute regressions without cutting an entirely new release.\n\n## Who has signed up?\n\n- [ ] Charity Engine (@rytiss, @tristanolive)\n- [ ] Fission (@bmann)\n- [ ] Infura (@MichaelMure)\n- [ ] OrbitDB (@haydenyoung)\n- [ ] Pinata (@obo20)\n- [ ] Shipyard (@cewood, @ns4plabs)\n- [ ] Siderus (@koalalorenzo)\n- [ ] Textile (@sanderpick)\n- [ ] @RubenKelevra\n\n## How to sign up?\n\nSimply submit a PR to this document by adding your project name and contact.\n" }, { "path": "docs/README.md", "content": "# Developer Documentation and Guides\n\nIf you're looking for User Documentation & Guides, visit [docs.ipfs.tech](https://docs.ipfs.tech/).\n\nIf you're experiencing an issue with IPFS, please [file an issue](https://github.com/ipfs/kubo/issues/new/choose) in this repository.\n\n## Configuration\n\n- [Configuration reference](config.md)\n - [Datastore configuration](datastores.md)\n - [Experimental features](experimental-features.md)\n- [Environment variables](environment-variables.md)\n\n## Running Kubo\n\n- [Gateway configuration](gateway.md)\n- [Delegated routing](delegated-routing.md)\n- [Content blocking](content-blocking.md) (for public node operators)\n- [libp2p resource management](libp2p-resource-management.md)\n- [Mounting IPFS with FUSE](fuse.md)\n\n## Metrics & Monitoring\n\n- [Prometheus metrics](metrics.md)\n- [Telemetry plugin](telemetry.md)\n- [Provider statistics](provide-stats.md)\n- [Performance debugging](debug-guide.md)\n\n## Development\n\n- **[Developer Guide](developer-guide.md)** - prerequisites, build, test, and contribute\n- **[AGENTS.md](../AGENTS.md)** - instructions for AI coding agents\n- Contributing Guidelines [for IPFS projects](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md) and for [Go code specifically](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md)\n- [Building on Windows](windows.md)\n- [Customizing Kubo](customizing.md)\n- [Installing plugins](plugins.md)\n- [Release checklist](releases.md)\n\n## Guides\n\n- [Transferring files over IPFS](file-transfer.md)\n- [How to implement an API client](implement-api-bindings.md)\n- [HTTP/RPC clients](http-rpc-clients.md)\n- [Websocket transports](transports.md)\n- [Command completion](command-completion.md)\n\n## Production\n\n- [Reverse proxy setup](production/reverse-proxy.md)\n\n## Specifications\n\n- [Repository structure](specifications/repository.md)\n- [Filesystem datastore](specifications/repository_fs.md)\n- [Keystore](specifications/keystore.md)\n\n## Examples\n\n- [Kubo as a library](examples/kubo-as-a-library/README.md)\n" }, { "path": "docs/RELEASE_CHECKLIST.md", "content": "\n\n# ✅ Release Checklist (vX.Y.Z[-rcN])\n\n**Release types:** RC (Release Candidate) | FINAL | PATCH\n\n## Prerequisites\n\n- [ ] [GPG signature](https://docs.github.com/en/authentication/managing-commit-signature-verification) configured in local git and GitHub\n- [ ] [Docker](https://docs.docker.com/get-docker/) installed on your system\n- [ ] [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed on your system\n- [ ] kubo repository cloned locally\n- [ ] **non-PATCH:** Upgrade Go in CI to latest patch from \n\n## 1. Prepare Release Branch\n\n- [ ] Fetch latest changes: `git fetch origin master release`\n- [ ] Create branch `release-vX.Y.Z` (base from: `master` if Z=0 for new minor/major, `release` if Z>0 for patch)\n- [ ] **RC1 only:** Switch to `master` branch and prepare for next release cycle:\n - [ ] Update [version.go](https://github.com/ipfs/kubo/blob/master/version.go) to `vX.Y+1.0-dev` (⚠️ double-check Y+1 is correct) ([example PR](https://github.com/ipfs/kubo/pull/9305))\n - [ ] Create `./docs/changelogs/vX.Y+1.md` and add link in [CHANGELOG.md](https://github.com/ipfs/kubo/blob/master/CHANGELOG.md)\n- [ ] Switch to `release-vX.Y.Z` branch and update [version.go](https://github.com/ipfs/kubo/blob/master/version.go) to `vX.Y.Z(-rcN)` (⚠️ double-check Y matches release) ([example](https://github.com/ipfs/kubo/pull/9394))\n- [ ] Create draft PR: `release-vX.Y.Z` → `release` ([example](https://github.com/ipfs/kubo/pull/9306))\n- [ ] Cherry-pick commits from `master` into `release-vX.Y.Z`: `git cherry-pick -x ` ([example](https://github.com/ipfs/kubo/pull/10636/commits/033de22e3bc6191dbb024ad6472f5b96b34e3ccf))\n - ⚠️ **NOTE:** `-x` flag records original commit SHA for traceability and cleaner merge history\n- [ ] Verify all CI checks on the PR are passing\n- [ ] **FINAL only:** Replace `Changelog` and `Contributors` sections in `release-vX.Y.Z` with `./bin/mkreleaselog` stdout (do **NOT** copy stderr)\n- [ ] **FINAL only:** Merge PR (`release-vX.Y.Z` → `release`) using `Create a merge commit`\n - ⚠️ do **NOT** use `Squash and merge` nor `Rebase and merge` -- we want the releaser's GPG signature on the merge commit\n - ⚠️ do **NOT** delete the `release-vX.Y.Z` branch (needed for future patch releases and git history)\n\n## 2. Tag & Publish\n\n### Create Tag\n⚠️ **POINT OF NO RETURN:** Once pushed, tags trigger automatic Docker/NPM publishing and are irreversible!\nIf you're making a release for the first time, do pair programming and have the release reviewer verify all commands.\n\n- [ ] **RC:** From `release-vX.Y.Z` branch: `git tag -s vX.Y.Z-rcN -m 'Prerelease X.Y.Z-rcN'`\n- [ ] **FINAL:** After PR merge, from `release` branch: `git tag -s vX.Y.Z -m 'Release X.Y.Z'`\n- [ ] ⚠️ Verify tag is signed and correct: `git show vX.Y.Z(-rcN)`\n- [ ] Push tag: `git push origin vX.Y.Z(-rcN)`\n - ⚠️ do **NOT** use `git push --tags` (pushes all local tags, polluting the repo with noise)\n- [ ] **STOP:** Wait for [Docker build](https://github.com/ipfs/kubo/actions/workflows/docker-image.yml) to complete before proceeding\n\n### Publish Artifacts\n\n> **Parallelism:** Docker and dist.ipfs.tech only depend on the pushed tag and can be started in parallel.\n> NPM and GitHub Release both depend on dist.ipfs.tech completing first.\n\n- [ ] **Docker:** Verify [docker-image CI](https://github.com/ipfs/kubo/actions/workflows/docker-image.yml) passed and image is available on [Docker Hub → tags](https://hub.docker.com/r/ipfs/kubo/tags)\n- [ ] **dist.ipfs.tech:** Publish to [dist.ipfs.tech](https://dist.ipfs.tech)\n - [ ] Check out [ipfs/distributions](https://github.com/ipfs/distributions)\n - [ ] Create branch: `git checkout -b release-kubo-X.Y.Z(-rcN)`\n - [ ] Verify `.tool-versions` golang matches [Kubo's CI](https://github.com/ipfs/kubo/blob/master/.github/workflows/gotest.yml) `go-version:` (update if needed)\n - [ ] Run: `./dist.sh add-version kubo vX.Y.Z(-rcN)` ([usage](https://github.com/ipfs/distributions#usage))\n - [ ] Create and merge PR (updates `dists/kubo/versions`, **FINAL** also updates `dists/kubo/current` - [example](https://github.com/ipfs/distributions/pull/1125))\n - [ ] Wait for [CI workflow](https://github.com/ipfs/distributions/actions/workflows/main.yml) triggered by merge\n - [ ] Verify release on [dist.ipfs.tech](https://dist.ipfs.tech/#kubo)\n- [ ] **NPM:** Publish to [NPM](https://www.npmjs.com/package/kubo?activeTab=versions)\n - [ ] Manually dispatch [Release to npm](https://github.com/ipfs/npm-kubo/actions/workflows/main.yml) workflow if not auto-triggered\n - [ ] Verify release on [NPM](https://www.npmjs.com/package/kubo?activeTab=versions)\n- [ ] **GitHub Release:** Publish to [GitHub](https://github.com/ipfs/kubo/releases)\n - [ ] [Create release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release) ([RC example](https://github.com/ipfs/kubo/releases/tag/v0.36.0-rc1), [FINAL example](https://github.com/ipfs/kubo/releases/tag/v0.35.0))\n - [ ] Use tag `vX.Y.Z(-rcN)`\n - [ ] Link to release issue\n - [ ] **RC:** Link to changelog, check `This is a pre-release`\n - [ ] **FINAL:** Copy changelog content (without header), do **NOT** check pre-release\n - [ ] Run [sync-release-assets](https://github.com/ipfs/kubo/actions/workflows/sync-release-assets.yml) workflow (requires dist.ipfs.tech)\n - [ ] Verify assets are attached to the GitHub release\n\n## 3. Post-Release\n\n### Technical Tasks\n\n- [ ] **FINAL only:** Merge `release` → `master`\n - [ ] Create branch `merge-release-vX.Y.Z` from `release`\n - [ ] Merge `master` to `merge-release-vX.Y.Z` first, and resolve conflict in `version.go`\n - ⚠️ **NOTE:** keep the `-dev` version from `master` in [version.go](https://github.com/ipfs/kubo/blob/master/version.go), discard version from `release`\n - [ ] Create and merge PR from `merge-release-vX.Y.Z` to `master` using `Create a merge commit`\n - ⚠️ do **NOT** use `Squash and merge` nor `Rebase and merge` -- only `Create a merge commit` preserves commit history and audit trail of what was merged where\n- [ ] Update [ipshipyard/waterworks-infra](https://github.com/ipshipyard/waterworks-infra)\n - [ ] Update Kubo staging environment ([Running Kubo tests on staging](https://www.notion.so/Running-Kubo-tests-on-staging-488578bb46154f9bad982e4205621af8))\n - [ ] **RC:** Test last release against current RC\n - [ ] **FINAL:** Latest release on both boxes\n - [ ] **FINAL:** Update collab cluster boxes to the tagged release\n - [ ] **FINAL:** Update libp2p bootstrappers to the tagged release\n- [ ] Update [ipfs-desktop](https://github.com/ipfs/ipfs-desktop)\n - [ ] Create PR updating kubo version in `package.json` and `package-lock.json`\n - [ ] Smoke test with [IPFS Companion Browser Extension](https://docs.ipfs.tech/install/ipfs-companion/) against the PR build\n - [ ] **FINAL:** Merge PR and ship new ipfs-desktop release\n- [ ] **FINAL only:** Update [docs.ipfs.tech](https://docs.ipfs.tech/): run [update-on-new-ipfs-tag.yml](https://github.com/ipfs/ipfs-docs/actions/workflows/update-on-new-ipfs-tag.yml) workflow and merge the PR\n\n### Promotion\n\n- [ ] Create [IPFS Discourse](https://discuss.ipfs.tech) topic ([RC example](https://discuss.ipfs.tech/t/kubo-v0-38-0-rc2-is-out/19772), [FINAL example](https://discuss.ipfs.tech/t/kubo-v0-38-0-is-out/19795))\n - [ ] Title: `Kubo vX.Y.Z(-rcN) is out!`, tag: `kubo`\n - [ ] Use title as heading (`##`) in description\n - [ ] Include: GitHub release link, IPNS binaries, docker pull command, release notes\n - [ ] Pin topic globally (make banner if no existing banner)\n- [ ] Verify bot posted to [#ipfs-chatter](https://discord.com/channels/669268347736686612/669268347736686615) (Discord) or [#ipfs-chatter:ipfs.io](https://matrix.to/#/#ipfs-chatter:ipfs.io) (Matrix)\n- [ ] **RC only:** Comment on release issue mentioning early testers ([example](https://github.com/ipfs/kubo/issues/9319#issuecomment-1311002478))\n- [ ] **FINAL only:** Comment on release issue with link ([example](https://github.com/ipfs/kubo/issues/9417#issuecomment-1400740975))\n- [ ] **FINAL only:** Create [blog.ipfs.tech](https://blog.ipfs.tech) entry ([example](https://github.com/ipfs/ipfs-blog/commit/32040d1e90279f21bad56b924fe4710bba5ba043))\n- [ ] **FINAL non-PATCH:** (optional) Post on social media ([bsky](https://bsky.app/profile/ipshipyard.com/post/3ltxcsrbn5s2k), [x.com](https://x.com/ipshipyard/status/1944867893226635603), [Reddit](https://www.reddit.com/r/ipfs/comments/1lzy6ze/release_v0360_ipfskubo/))\n\n### Final Steps\n\n- [ ] **FINAL non-PATCH:** Create dependency update PR\n - [ ] Review direct dependencies from root `go.mod` (⚠️ do **NOT** run `go get -u` as it will upgrade indirect dependencies which may cause problems)\n - [ ] Run `make mod_tidy`\n - [ ] Create PR with `go.mod` and `go.sum` updates\n - [ ] Add PR to next release milestone\n- [ ] **FINAL non-PATCH:** Create next release issue ([example](https://github.com/ipfs/kubo/issues/10816))\n- [ ] **FINAL only:** Close release issue" }, { "path": "docs/RELEASE_ISSUE_TEMPLATE.md", "content": "\n\n# Items to do upon creating the release issue\n\n- [ ] Fill in the Meta section\n- [ ] Assign the issue to the release owner and reviewer.\n- [ ] Name the issue \"Release vX.Y.Z\"\n- [ ] Set the proper values for X.Y.Z\n- [ ] Pin the issue\n\n\n\n# Meta\n\n* Release owner: @who\n* Release reviewer: @who\n* Expected RC date: week of YYYY-MM-DD\n* 🚢 Expected final release date: YYYY-MM-DD\n* Release PR: \n* Accompanying PR for improving the release process: ([example](https://github.com/ipfs/kubo/pull/9391))\n* Changelog: https://github.com/ipfs/kubo/blob/master/docs/changelogs/vX.Y.md\n\n# Items In Scope\n\n## Required\n\n\n\n## Nice To Have (Optional)\n\n\n" }, { "path": "docs/add-code-flow.md", "content": "# How `ipfs add` Works\n\nThis document explains what happens when you run `ipfs add` to import files into IPFS. Understanding this flow helps when debugging, optimizing imports, or building applications on top of IPFS.\n\n- [The Big Picture](#the-big-picture)\n- [Try It Yourself](#try-it-yourself)\n- [Step by Step](#step-by-step)\n - [Step 1: Chunking](#step-1-chunking)\n - [Step 2: Building the DAG](#step-2-building-the-dag)\n - [Step 3: Storing Blocks](#step-3-storing-blocks)\n - [Step 4: Pinning](#step-4-pinning)\n - [Alternative: Organizing with MFS](#alternative-organizing-with-mfs)\n- [Options](#options)\n- [UnixFS Format](#unixfs-format)\n- [Code Architecture](#code-architecture)\n - [Key Files](#key-files)\n - [The Adder](#the-adder)\n- [Further Reading](#further-reading)\n\n## The Big Picture\n\nWhen you add a file to IPFS, three main things happen:\n\n1. **Chunking** - The file is split into smaller pieces\n2. **DAG Building** - Those pieces are organized into a tree structure (a [Merkle DAG](https://docs.ipfs.tech/concepts/merkle-dag/))\n3. **Pinning** - The root of the tree is pinned so it persists in your local node\n\nThe result is a Content Identifier (CID) - a hash that uniquely identifies your content and can be used to retrieve it from anywhere in the IPFS network.\n\n```mermaid\nflowchart LR\n A[\"Your File
(bytes)\"] --> B[\"Chunker
(split data)\"]\n B --> C[\"DAG Builder
(tree)\"]\n C --> D[\"CID
(hash)\"]\n```\n\n## Try It Yourself\n\n```bash\n# Add a simple file\necho \"Hello World\" > hello.txt\nipfs add hello.txt\n# added QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u hello.txt\n\n# See what's inside\nipfs cat QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u\n# Hello World\n\n# View the DAG structure\nipfs dag get QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u\n```\n\n## Step by Step\n\n### Step 1: Chunking\n\nBig files are split into chunks because:\n\n- Large files need to be broken down for efficient transfer\n- Identical chunks across files are stored only once (deduplication)\n- You can fetch parts of a file without downloading the whole thing\n\n**Chunking strategies** (set with `--chunker`):\n\n| Strategy | Description | Best For |\n|----------|-------------|----------|\n| `size-N` | Fixed size chunks | General use |\n| `rabin` | Content-defined chunks using rolling hash | Deduplication across similar files |\n| `buzhash` | Alternative content-defined chunking | Similar to rabin |\n\nSee `ipfs add --help` for current defaults, or [Import](config.md#import) for making them permanent.\n\nContent-defined chunking (rabin/buzhash) finds natural boundaries in the data. This means if you edit the middle of a file, only the changed chunks need to be re-stored - the rest can be deduplicated.\n\n### Step 2: Building the DAG\n\nEach chunk becomes a leaf node in a tree. If a file has many chunks, intermediate nodes group them together. This creates a Merkle DAG (Directed Acyclic Graph) where:\n\n- Each node is identified by a hash of its contents\n- Parent nodes contain links (hashes) to their children\n- The root node's hash becomes the file's CID\n\n**Layout strategies**:\n\n**Balanced layout** (default):\n\n```mermaid\ngraph TD\n Root --> Node1[Node]\n Root --> Node2[Node]\n Node1 --> Leaf1[Leaf]\n Node1 --> Leaf2[Leaf]\n Node2 --> Leaf3[Leaf]\n```\n\nAll leaves at similar depth. Good for random access - you can jump to any part of the file efficiently.\n\n**Trickle layout** (`--trickle`):\n\n```mermaid\ngraph TD\n Root --> Leaf1[Leaf]\n Root --> Node1[Node]\n Root --> Node2[Node]\n Node1 --> Leaf2[Leaf]\n Node2 --> Leaf3[Leaf]\n```\n\nLeaves added progressively. Good for streaming - you can start reading before the whole file is added.\n\n### Step 3: Storing Blocks\n\nAs the DAG is built, each node is stored in the blockstore:\n\n- **Normal mode**: Data is copied into IPFS's internal storage (`~/.ipfs/blocks/`)\n- **Filestore mode** (`--nocopy`): Only references to the original file are stored (saves disk space but the original file must remain in place)\n\n### Step 4: Pinning\n\nBy default, added content is pinned (`ipfs add --pin=true`). This tells your IPFS node to keep this data - without pinning, content may eventually be removed to free up space.\n\n### Alternative: Organizing with MFS\n\nInstead of pinning, you can use the [Mutable File System (MFS)](https://docs.ipfs.tech/concepts/file-systems/#mutable-file-system-mfs) to organize content using familiar paths like `/photos/vacation.jpg` instead of raw CIDs:\n\n```bash\n# Add directly to MFS path\nipfs add --to-files=/backups/ myfile.txt\n\n# Or copy an existing CID into MFS\nipfs files cp /ipfs/QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u /docs/hello.txt\n```\n\nContent in MFS is implicitly pinned and stays organized across node restarts.\n\n## Options\n\nRun `ipfs add --help` to see all available options for controlling chunking, DAG layout, CID format, pinning behavior, and more.\n\n## UnixFS Format\n\nIPFS uses [UnixFS](https://specs.ipfs.tech/unixfs/) to represent files and directories. UnixFS is an abstraction layer that:\n\n- Gives names to raw data blobs (so you can have `/foo/bar.txt` instead of just hashes)\n- Represents directories as lists of named links to other nodes\n- Organizes large files as trees of smaller chunks\n- Makes these structures cryptographically verifiable - any tampering is detectable because it would change the hashes\n\nWith `--raw-leaves`, leaf nodes store raw data without the UnixFS wrapper. This is more efficient and is the default when using CIDv1.\n\n## Code Architecture\n\nThe add flow spans several layers:\n\n```mermaid\nflowchart TD\n subgraph CLI [\"CLI Layer (kubo)\"]\n A[\"core/commands/add.go
parses flags, shows progress\"]\n end\n subgraph API [\"CoreAPI Layer (kubo)\"]\n B[\"core/coreapi/unixfs.go
UnixfsAPI.Add() entry point\"]\n end\n subgraph Adder [\"Adder (kubo)\"]\n C[\"core/coreunix/add.go
orchestrates chunking, DAG building, MFS, pinning\"]\n end\n subgraph Boxo [\"boxo libraries\"]\n D[\"chunker/ - splits data into chunks\"]\n E[\"ipld/unixfs/ - DAG layout and UnixFS format\"]\n F[\"mfs/ - mutable filesystem abstraction\"]\n G[\"pinning/ - pin management\"]\n H[\"blockstore/ - block storage\"]\n end\n A --> B --> C --> Boxo\n```\n\n### Key Files\n\n| Component | Location |\n|-----------|----------|\n| CLI command | `core/commands/add.go` |\n| API implementation | `core/coreapi/unixfs.go` |\n| Adder logic | `core/coreunix/add.go` |\n| Chunking | [boxo/chunker](https://github.com/ipfs/boxo/tree/main/chunker) |\n| DAG layouts | [boxo/ipld/unixfs/importer](https://github.com/ipfs/boxo/tree/main/ipld/unixfs/importer) |\n| MFS | [boxo/mfs](https://github.com/ipfs/boxo/tree/main/mfs) |\n| Pinning | [boxo/pinning/pinner](https://github.com/ipfs/boxo/tree/main/pinning/pinner) |\n\n### The Adder\n\nThe `Adder` type in `core/coreunix/add.go` is the workhorse. It:\n\n1. **Creates an MFS root** - temporary in-memory filesystem for building the DAG\n2. **Processes files recursively** - chunks each file and builds DAG nodes\n3. **Commits to blockstore** - persists all blocks\n4. **Pins the result** - keeps content from being removed\n5. **Returns the root CID**\n\nKey methods:\n\n- `AddAllAndPin()` - main entry point\n- `addFileNode()` - handles a single file or directory\n- `add()` - chunks data and builds the DAG using boxo's layout builders\n\n## Further Reading\n\n- [UnixFS specification](https://specs.ipfs.tech/unixfs/)\n- [IPLD and Merkle DAGs](https://docs.ipfs.tech/concepts/merkle-dag/)\n- [Pinning](https://docs.ipfs.tech/concepts/persistence/)\n- [MFS (Mutable File System)](https://docs.ipfs.tech/concepts/file-systems/#mutable-file-system-mfs)\n" }, { "path": "docs/changelogs/v0.10.md", "content": "# go-ipfs changelog v0.10\n\n## v0.10.0 2021-09-30\n\nWe're happy to announce go-ipfs 0.10.0. This release brings some big changes to the IPLD internals of go-ipfs that make working with non-UnixFS DAGs easier than ever. There are also a variety of new commands and configuration options available.\n\nAs usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will _not_ be called out in the release notes. Please make sure to update ASAP. See our [release process](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md#security-fix-policy) for details.\n\n### 🛠 TLDR: BREAKING CHANGES\n\n- `ipfs dag get`\n - default output changed to [`dag-json`](https://ipld.io/specs/codecs/dag-json/spec/)\n - dag-pb (e.g. unixfs) field names changed - impacts userland code that works with `dag-pb` objects returned by `dag get`\n - no longer emits an additional new-line character at the end of the data output\n- `ipfs dag put`\n - defaults changed to reduce ambiguity and surprises: input is now assumed to be [`dag-json`](https://ipld.io/specs/codecs/dag-json/spec/), and data is serialized to [`dag-cbor`](https://ipld.io/specs/codecs/dag-cbor/spec/) at rest.\n - `--format` and `--input-enc` were removed and replaced with `--store-codec` and `--input-codec`\n - codec names now match the ones defined in the [multicodec table](https://github.com/multiformats/multicodec/blob/master/table.csv)\n - dag-pb (e.g. unixfs) field names changed - impacts userland code that works with `dag-pb` objects stored via `dag put`\n\nKeep reading to learn more details.\n\n### 🔦 Highlights\n\n#### 🌲 IPLD Levels Up\n\nThe handling of data serialization as well as many aspects of DAG traversal and pathing have been migrated from older libraries, including [go-merkledag](https://github.com/ipfs/go-merkledag) and [go-ipld-format](https://github.com/ipfs/go-ipld-format) to the new **[go-ipld-prime](https://github.com/ipld/go-ipld-prime)** library and its components. This allows us to use many of the newer tools afforded by go-ipld-prime, stricter and more uniform codec implementations, support for additional (pluggable) codecs, and some minor performance improvements.\n\nThis is significant refactor of a core component that touches many parts of IPFS, and does come with some **breaking changes**:\n\n* **IPLD plugins**:\n * The `PluginIPLD` interface has been changed to utilize go-ipld-prime. There is a demonstration of the change in the [bundled git plugin](./plugin/plugins/git/).\n* **The semantics of `dag put` and `dag get` change**:\n * `dag get` now takes the `output-codec` option which accepts a [multicodec](https://docs.ipfs.tech/concepts/glossary/#multicodec) name used to encode the output. By default this is `dag-json`, which is a strict and deterministic subset of JSON created by the IPLD team. Users may notice differences from the previously plain Go JSON output, particularly where bytes are concerned which are now encoded using a form similar to CIDs: `{\"/\":{\"bytes\":\"unpadded-base64-bytes\"}}` rather than the previously Go-specific plain padded base64 string. See the [dag-json specification](https://ipld.io/specs/codecs/dag-json/spec/) for an explanation of these forms.\n * `dag get` no longer prints an additional new-line character at the end of the encoded block output. This means that the output as presented by `dag get` are the exact bytes of the requested node. A round-trip of such bytes back in through `dag put` using the same codec should result in the same CID.\n * `dag put` uses the `input-codec` option to specify the multicodec name of the format data is being provided in, and the `store-codec` option to specify the multicodec name of the format the data should be stored in at rest. These formerly defaulted to `json` and `cbor` respectively. They now default to `dag-json` and `dag-cbor` respectively but may be changed to any supported codec (bundled or loaded via plugin) by its [multicodec name](https://github.com/multiformats/multicodec/blob/master/table.csv).\n * The `json` and `cbor` multicodec names (as used by `input-enc` and `format` options) are now no longer aliases for `dag-json` and `dag-cbor` respectively. Instead, they now refer to their proper [multicodec](https://github.com/multiformats/multicodec/blob/master/table.csv) types. `cbor` refers to a plain CBOR format, which will not encode CIDs and does not have strict deterministic encoding rules. `json` is a plain JSON format, which also won't encode CIDs and will encode bytes in the Go-specific padded base64 string format rather than the dag-json method of byte encoding. See https://ipld.io/specs/codecs/ for more information on IPLD codecs.\n * `protobuf` is no longer used as the codec name for `dag-pb`\n * The codec name `raw` is used to mean Bytes in the [IPLD Data Model](https://github.com/ipld/specs/blob/master/data-model-layer/data-model.md#bytes-kind)\n* **UnixFS refactor**. The **dag-pb codec**, which is used to encode UnixFS data for IPFS, is now represented through the `dag` API in a form that mirrors the protobuf schema used to define the binary format. This unifies the implementations and specification of dag-pb across the IPLD and IPFS stacks. Previously, additional layers of code for file and directory handling within IPFS between protobuf serialization and UnixFS obscured the protobuf representation. Much of this code has now been replaced and there are fewer layers of transformation. This means that interacting with dag-pb data via the `dag` API will use different forms:\n * Previously, using `dag get` on a dag-pb block would present the block serialized as JSON as `{\"data\":\"padded-base64-bytes\",\"links\":[{\"Name\":\"foo\",\"Size\":100,\"Cid\":{\"/\":\"Qm...\"}},...]}`.\n * Now, the dag-pb data with dag-json codec for output will be serialized using the data model from the [dag-pb specification](https://ipld.io/specs/codecs/dag-pb/spec/): `{\"Data\":{\"/\":{\"bytes\":\"unpadded-base64-bytes\"}},\"Links\":[{\"Name\":\"foo\",\"Tsize\":100,\"Hash\":{\"/\":\"Qm...\"}},...]}`. Aside from the change in byte formatting, most field names have changed: `data` → `Data`, `links` → `Links`, `Size` → `Tsize`, `Cid` → `Hash`. Note that this output can be changed now using the `output-codec` option to specify an alternative codec.\n * Similarly, using `dag put` and a `store-codec` option of `dag-pb` now requires that the input conform to this dag-pb specified form. Previously, input using `{\"data\":\"...\",\"links\":[...]}` was accepted, now it must be `{\"Data\":\"...\",\"Links\":[...]}`.\n * Previously it was not possible to use paths to navigate to any of these properties of a dag-pb node, the only possible paths were named links, e.g. `dag get QmFoo/NamedLink` where `NamedLink` was one of the links whose name was `NamedLink`. This functionality remains the same, but by prefixing the path with `/ipld/` we enter data model pathing semantics and can `dag get /ipld/QmFoo/Links/0/Hash` to navigate to links or `/ipld/QmFoo/Data` to simply retrieve the data section of the node, for example.\n * ℹ See the [dag-pb specification](https://ipld.io/specs/codecs/dag-pb/) for details on the codec and its data model representation.\n * ℹ See this [detailed write-up](https://github.com/ipld/ipld/blob/master/design/tricky-choices/dag-pb-forms-impl-and-use.md) for further background on these changes.\n\n#### Ⓜ Multibase Command\n\ngo-ipfs now provides utility commands for working with [multibase](https://docs.ipfs.tech/concepts/glossary/#multibase):\n\n```console\n$ echo -n hello | ipfs multibase encode -b base16 > file-mbase16\n$ cat file-mbase16\nf68656c6c6f\n\n$ ipfs multibase decode file-mbase16\nhello\n\n$ cat file-mbase16 | ipfs multibase decode\nhello\n\n$ ipfs multibase transcode -b base2 file-mbase16\n00110100001100101011011000110110001101111\n```\n\nSee `ipfs multibase --help` for more examples.\n\n#### 🔨 Bitswap now supports greater configurability\n\nThis release adds an [`Internal` section](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#internal) to the configuration file that is designed to help advanced users optimize their setups without needing a custom binary. The `Internal` section is not guaranteed to be the same from release to release and may not be covered by migrations. If you use the `Internal` section you should be making sure to check the config documentation between releases for any changes.\n\n#### 🐚 Programmatic shell completions command\n\n`ipfs commands completion bash` will generate a bash completion script for go-ipfs commands\n\n#### 📜 Profile collection command\n\nPerformance profiles can now be collected using `ipfs diag profile`. If you need to do some debugging or have an issue to submit the collected profiles are very useful to have around.\n\n#### 🍎 Mac OS notarized binaries\n\nThe go-ipfs and related migration binaries (for both Intel and Apple Silicon) are now signed and notarized to make Mac OS installation easier.\n\n#### 👨‍👩‍👦 Improved MDNS\n\nThere is a completed implementation of the revised libp2p MDNS spec. This should result in better MDNS discovery and better local/offline operation as a result.\n\n#### 🚗 CAR import statistics\n\n`dag import` command now supports `--stats` option which will include the number of imported blocks and their total size in the output.\n\n#### 🕸 Peering command\n\nThis release adds `swarm peering` command for easy management of the peering subsystem. Peer in the peering subsystem is maintained to be connected at all times, and gets reconnected on disconnect with a back-off.\n\nSee `ipfs swarm peering --help` for more details.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - fuse: load unixfs adls as their dagpb substrates\n - enable the legacy mDNS implementation\n - test: add dag get --output-codec test\n - change ipfs dag get flag name from format to output-codec\n - test: check behavior of loading UnixFS sharded directories with missing shards\n - remove dag put option shortcuts\n - change names of ipfs dag put flags to make changes clearer\n - feat: dag import --stats (#8237) ([ipfs/go-ipfs#8237](https://github.com/ipfs/go-ipfs/pull/8237))\n - feat: ipfs-webui v2.13.0 (#8430) ([ipfs/go-ipfs#8430](https://github.com/ipfs/go-ipfs/pull/8430))\n - feat(cli): add daemon option --agent-version-suffix (#8419) ([ipfs/go-ipfs#8419](https://github.com/ipfs/go-ipfs/pull/8419))\n - feat: multibase transcode command (#8403) ([ipfs/go-ipfs#8403](https://github.com/ipfs/go-ipfs/pull/8403))\n - fix: take the lock while listing peers\n - feature: 'ipfs swarm peering' command (#8147) ([ipfs/go-ipfs#8147](https://github.com/ipfs/go-ipfs/pull/8147))\n - fix(sharness): add extra check in flush=false in files write\n - chore: update IPFS Desktop testing steps (#8393) ([ipfs/go-ipfs#8393](https://github.com/ipfs/go-ipfs/pull/8393))\n - add more buttons; remove some sections covered in the docs; general cleanup\n - Cosmetic fixups in examples (#8325) ([ipfs/go-ipfs#8325](https://github.com/ipfs/go-ipfs/pull/8325))\n - perf: use performance-enhancing FUSE mount options\n - ci: publish Docker images for bifrost-* branches\n - chore: add comments to peerlog plugin about being unsupported\n - test: add unit tests for peerlog config parsing\n - ci: preload peerlog plugin, disable by default\n - fix(mkreleaselog): specify the parent commit when diffing\n - update go-libp2p to v0.15.0-rc.1 ([ipfs/go-ipfs#8354](https://github.com/ipfs/go-ipfs/pull/8354))\n - feat: add 'ipfs multibase' commands (#8180) ([ipfs/go-ipfs#8180](https://github.com/ipfs/go-ipfs/pull/8180))\n - support bitswap configurability (#8268) ([ipfs/go-ipfs#8268](https://github.com/ipfs/go-ipfs/pull/8268))\n - IPLD Prime In IPFS: Target Merge Branch (#7976) ([ipfs/go-ipfs#7976](https://github.com/ipfs/go-ipfs/pull/7976))\n - ci: upgrade to Go 1.16.7 on CI ([ipfs/go-ipfs#8324](https://github.com/ipfs/go-ipfs/pull/8324))\n - Add flag to create parent directories in files cp command ([ipfs/go-ipfs#8340](https://github.com/ipfs/go-ipfs/pull/8340))\n - fix: avoid out of bounds error when rendering short hashes ([ipfs/go-ipfs#8318](https://github.com/ipfs/go-ipfs/pull/8318))\n - fix: remove some deprecated calls ([ipfs/go-ipfs#8296](https://github.com/ipfs/go-ipfs/pull/8296))\n - perf: set an appropriate capacity ([ipfs/go-ipfs#8244](https://github.com/ipfs/go-ipfs/pull/8244))\n - Fix: Use a pointer type on IpfsNode.Peering ([ipfs/go-ipfs#8331](https://github.com/ipfs/go-ipfs/pull/8331))\n - fix: macos notarized fs-repo-migrations (#8333) ([ipfs/go-ipfs#8333](https://github.com/ipfs/go-ipfs/pull/8333))\n - README.md: Add MacPorts to install section ([ipfs/go-ipfs#8220](https://github.com/ipfs/go-ipfs/pull/8220))\n - feat: register first block metric by default ([ipfs/go-ipfs#8332](https://github.com/ipfs/go-ipfs/pull/8332))\n - Build a go-ipfs:extras docker image ([ipfs/go-ipfs#8142](https://github.com/ipfs/go-ipfs/pull/8142))\n - fix/go-ipfs-as-a-library ([ipfs/go-ipfs#8266](https://github.com/ipfs/go-ipfs/pull/8266))\n - Expose additional migration APIs (#8153) ([ipfs/go-ipfs#8153](https://github.com/ipfs/go-ipfs/pull/8153))\n - point ipfs to pinner that syncs on every pin (#8231) ([ipfs/go-ipfs#8231](https://github.com/ipfs/go-ipfs/pull/8231))\n - docs: chocolatey package name\n - Disambiguate online/offline naming in sharness tests ([ipfs/go-ipfs#8254](https://github.com/ipfs/go-ipfs/pull/8254))\n - Rename DOCKER_HOST to TEST_DOCKER_HOST to avoid conflicts ([ipfs/go-ipfs#8283](https://github.com/ipfs/go-ipfs/pull/8283))\n - feat: add an \"ipfs diag profile\" command ([ipfs/go-ipfs#8291](https://github.com/ipfs/go-ipfs/pull/8291))\n - Merge branch 'release'\n - feat: improve mkreleaslog ([ipfs/go-ipfs#8290](https://github.com/ipfs/go-ipfs/pull/8290))\n - Add test with expected failure for #3503 ([ipfs/go-ipfs#8280](https://github.com/ipfs/go-ipfs/pull/8280))\n - Create PATCH_RELEASE_TEMPLATE.md\n - fix document error ([ipfs/go-ipfs#8271](https://github.com/ipfs/go-ipfs/pull/8271))\n - feat: webui v2.12.4\n - programmatic shell completions ([ipfs/go-ipfs#8043](https://github.com/ipfs/go-ipfs/pull/8043))\n - test: gateway response for bafkqaaa\n - doc(README): update chat links (and misc fixes) ([ipfs/go-ipfs#8222](https://github.com/ipfs/go-ipfs/pull/8222))\n - link to the actual doc (#8126) ([ipfs/go-ipfs#8126](https://github.com/ipfs/go-ipfs/pull/8126))\n - Improve peer hints for pin remote add (#8143) ([ipfs/go-ipfs#8143](https://github.com/ipfs/go-ipfs/pull/8143))\n - fix(mkreleaselog): support multiple commit authors ([ipfs/go-ipfs#8214](https://github.com/ipfs/go-ipfs/pull/8214))\n - fix(mkreleaselog): handle commit 0 ([ipfs/go-ipfs#8121](https://github.com/ipfs/go-ipfs/pull/8121))\n - bump snap to build with Go 1.16\n - chore: update CHANGELOG\n - chore: switch tar-utils dep to ipfs org\n - feat: print error on bootstrap failure ([ipfs/go-ipfs#8166](https://github.com/ipfs/go-ipfs/pull/8166))\n - fix: typo in migration error\n - refactor: improved humanNumber and humanSI\n - feat: humanized durations in stat provide\n - feat: humanized numbers in stat provide\n - feat: add a text output encoding for the stats provide command\n - fix: webui-2.12.3\n - refactor(pinmfs): log error if pre-existing pin failed (#8056) ([ipfs/go-ipfs#8056](https://github.com/ipfs/go-ipfs/pull/8056))\n - config.md: fix typos/improve wording ([ipfs/go-ipfs#8031](https://github.com/ipfs/go-ipfs/pull/8031))\n - fix(peering_test) : Fix the peering_test to check the connection explicitly added ([ipfs/go-ipfs#8140](https://github.com/ipfs/go-ipfs/pull/8140))\n - build: ignore generated files in changelog ([ipfs/go-ipfs#7712](https://github.com/ipfs/go-ipfs/pull/7712))\n - update version to 0.10.0-dev ([ipfs/go-ipfs#8136](https://github.com/ipfs/go-ipfs/pull/8136))\n- github.com/ipfs/go-bitswap (v0.3.4 -> v0.4.0):\n - More stats, knobs and tunings (#514) ([ipfs/go-bitswap#514](https://github.com/ipfs/go-bitswap/pull/514))\n - fix: fix a map access race condition in the want index ([ipfs/go-bitswap#523](https://github.com/ipfs/go-bitswap/pull/523))\n - fix: make blockstore cancel test less timing dependent ([ipfs/go-bitswap#507](https://github.com/ipfs/go-bitswap/pull/507))\n - fix(decision): fix a datarace on disconnect ([ipfs/go-bitswap#508](https://github.com/ipfs/go-bitswap/pull/508))\n - optimize the lookup which peers are waiting for a given block ([ipfs/go-bitswap#486](https://github.com/ipfs/go-bitswap/pull/486))\n - fix: hold the task worker lock when starting task workers ([ipfs/go-bitswap#504](https://github.com/ipfs/go-bitswap/pull/504))\n - fix: Nil dereference while using SetSendDontHaves ([ipfs/go-bitswap#488](https://github.com/ipfs/go-bitswap/pull/488))\n - Fix flaky tests in message queue ([ipfs/go-bitswap#497](https://github.com/ipfs/go-bitswap/pull/497))\n - Fix flaky DontHaveTimeoutManger tests ([ipfs/go-bitswap#495](https://github.com/ipfs/go-bitswap/pull/495))\n - sync: update CI config files ([ipfs/go-bitswap#485](https://github.com/ipfs/go-bitswap/pull/485))\n- github.com/ipfs/go-blockservice (v0.1.4 -> v0.1.7):\n - update go-bitswap to v0.3.4 ([ipfs/go-blockservice#78](https://github.com/ipfs/go-blockservice/pull/78))\n - fix staticcheck ([ipfs/go-blockservice#75](https://github.com/ipfs/go-blockservice/pull/75))\n - fix: handle missing session exchange in Session ([ipfs/go-blockservice#73](https://github.com/ipfs/go-blockservice/pull/73))\n- github.com/ipfs/go-datastore (v0.4.5 -> v0.4.6):\n - sync: update CI config files ([ipfs/go-datastore#175](https://github.com/ipfs/go-datastore/pull/175))\n - speedup tests ([ipfs/go-datastore#177](https://github.com/ipfs/go-datastore/pull/177))\n - test: reduce element count when the race detector is enabled ([ipfs/go-datastore#176](https://github.com/ipfs/go-datastore/pull/176))\n - fix staticcheck ([ipfs/go-datastore#173](https://github.com/ipfs/go-datastore/pull/173))\n - remove Makefile ([ipfs/go-datastore#172](https://github.com/ipfs/go-datastore/pull/172))\n- github.com/ipfs/go-ds-badger (v0.2.6 -> v0.2.7):\n - Log start and end of GC rounds ([ipfs/go-ds-badger#115](https://github.com/ipfs/go-ds-badger/pull/115))\n- github.com/ipfs/go-fs-lock (v0.0.6 -> v0.0.7):\n - chore: update log ([ipfs/go-fs-lock#24](https://github.com/ipfs/go-fs-lock/pull/24))\n - sync: update CI config files ([ipfs/go-fs-lock#21](https://github.com/ipfs/go-fs-lock/pull/21))\n - fix TestLockedByOthers on Windows ([ipfs/go-fs-lock#19](https://github.com/ipfs/go-fs-lock/pull/19))\n- github.com/ipfs/go-ipfs-config (v0.14.0 -> v0.16.0):\n - feat: add Internal and Internal.Bitswap config options\n - feat: add an OptionalInteger type\n - fix: make sure the Priority type properly implements the JSON marshal/unmarshal interfaces\n - fix: remove deprecated calls ([ipfs/go-ipfs-config#138](https://github.com/ipfs/go-ipfs-config/pull/138))\n - sync: update CI config files ([ipfs/go-ipfs-config#132](https://github.com/ipfs/go-ipfs-config/pull/132))\n - remove period, fix staticcheck ([ipfs/go-ipfs-config#131](https://github.com/ipfs/go-ipfs-config/pull/131))\n- github.com/ipfs/go-ipfs-pinner (v0.1.1 -> v0.1.2):\n - Fix/minimize rebuild (#15) ([ipfs/go-ipfs-pinner#15](https://github.com/ipfs/go-ipfs-pinner/pull/15))\n - Define ErrNotPinned alongside the Pinner interface\n - fix staticcheck ([ipfs/go-ipfs-pinner#11](https://github.com/ipfs/go-ipfs-pinner/pull/11))\n - fix: remove the rest of the pb backed pinner ([ipfs/go-ipfs-pinner#9](https://github.com/ipfs/go-ipfs-pinner/pull/9))\n - Remove old ipldpinner that has been replaced by dspinner ([ipfs/go-ipfs-pinner#7](https://github.com/ipfs/go-ipfs-pinner/pull/7))\n - optimize CheckIfPinned ([ipfs/go-ipfs-pinner#6](https://github.com/ipfs/go-ipfs-pinner/pull/6))\n- github.com/ipfs/go-ipfs-provider (v0.5.1 -> v0.6.1):\n - Update to IPLD Prime (#32) ([ipfs/go-ipfs-provider#32](https://github.com/ipfs/go-ipfs-provider/pull/32))\n- github.com/ipfs/go-ipld-git (v0.0.4 -> v0.1.1):\n - return ErrUnexpectedEOF when Decode input is too short\n - Update go-ipld-git to a go-ipld-prime codec (#46) ([ipfs/go-ipld-git#46](https://github.com/ipfs/go-ipld-git/pull/46))\n - fix staticcheck ([ipfs/go-ipld-git#49](https://github.com/ipfs/go-ipld-git/pull/49))\n - change WriteTo to the standard signature ([ipfs/go-ipld-git#47](https://github.com/ipfs/go-ipld-git/pull/47))\n - don't copy mutexes ([ipfs/go-ipld-git#48](https://github.com/ipfs/go-ipld-git/pull/48))\n- github.com/ipfs/go-ipns (v0.1.0 -> v0.1.2):\n - fix: remove deprecated calls ([ipfs/go-ipns#30](https://github.com/ipfs/go-ipns/pull/30))\n - remove Makefile ([ipfs/go-ipns#27](https://github.com/ipfs/go-ipns/pull/27))\n- github.com/ipfs/go-log/v2 (v2.1.3 -> v2.3.0):\n - Stop defaulting to color output on non-TTY ([ipfs/go-log#116](https://github.com/ipfs/go-log/pull/116))\n - feat: add ability to use custom zap core ([ipfs/go-log#114](https://github.com/ipfs/go-log/pull/114))\n - fix staticcheck ([ipfs/go-log#112](https://github.com/ipfs/go-log/pull/112))\n - test: fix flaky label test ([ipfs/go-log#111](https://github.com/ipfs/go-log/pull/111))\n - per-subsystem log-levels ([ipfs/go-log#109](https://github.com/ipfs/go-log/pull/109))\n - fix: don't panic on invalid log labels ([ipfs/go-log#110](https://github.com/ipfs/go-log/pull/110))\n- github.com/ipfs/go-merkledag (v0.3.2 -> v0.4.0):\n - Use IPLD-prime: target merge branch ([ipfs/go-merkledag#67](https://github.com/ipfs/go-merkledag/pull/67))\n - sync: update CI config files ([ipfs/go-merkledag#70](https://github.com/ipfs/go-merkledag/pull/70))\n - staticcheck ([ipfs/go-merkledag#69](https://github.com/ipfs/go-merkledag/pull/69))\n - Fix bug in dagutils MergeDiffs. (#59) ([ipfs/go-merkledag#59](https://github.com/ipfs/go-merkledag/pull/59))\n - chore: add tests to verify allowable data layouts ([ipfs/go-merkledag#58](https://github.com/ipfs/go-merkledag/pull/58))\n- github.com/ipfs/go-namesys (v0.3.0 -> v0.3.1):\n - fix: remove deprecated call to pk.Bytes ([ipfs/go-namesys#19](https://github.com/ipfs/go-namesys/pull/19))\n- github.com/ipfs/go-path (v0.0.9 -> v0.1.2):\n - fix: give one minute timeouts to function calls instead of block retrievals ([ipfs/go-path#44](https://github.com/ipfs/go-path/pull/44))\n - IPLD Prime In IPFS: Target Merge Branch (#36) ([ipfs/go-path#36](https://github.com/ipfs/go-path/pull/36))\n - remove Makefile ([ipfs/go-path#40](https://github.com/ipfs/go-path/pull/40))\n - sync: update CI config files ([ipfs/go-path#39](https://github.com/ipfs/go-path/pull/39))\n- github.com/ipfs/go-peertaskqueue (v0.2.0 -> v0.4.0):\n - add stats\n - Have a configurable maximum active work per peer ([ipfs/go-peertaskqueue#10](https://github.com/ipfs/go-peertaskqueue/pull/10))\n - sync: update CI config files ([ipfs/go-peertaskqueue#13](https://github.com/ipfs/go-peertaskqueue/pull/13))\n - fix staticcheck ([ipfs/go-peertaskqueue#12](https://github.com/ipfs/go-peertaskqueue/pull/12))\n - fix go vet ([ipfs/go-peertaskqueue#11](https://github.com/ipfs/go-peertaskqueue/pull/11))\n- github.com/ipfs/go-unixfsnode (null -> v1.1.3):\n - make UnixFSHAMTShard implement the ADL interface (#11) ([ipfs/go-unixfsnode#11](https://github.com/ipfs/go-unixfsnode/pull/11))\n- github.com/ipfs/interface-go-ipfs-core (v0.4.0 -> v0.5.1):\n - IPLD In IPFS: Target Merge Branch (#67) ([ipfs/interface-go-ipfs-core#67](https://github.com/ipfs/interface-go-ipfs-core/pull/67))\n - fix staticcheck ([ipfs/interface-go-ipfs-core#72](https://github.com/ipfs/interface-go-ipfs-core/pull/72))\n - remove Makefile ([ipfs/interface-go-ipfs-core#70](https://github.com/ipfs/interface-go-ipfs-core/pull/70))\n- github.com/ipld/go-codec-dagpb (v1.2.0 -> v1.3.0):\n - fix staticcheck warnings ([ipld/go-codec-dagpb#29](https://github.com/ipld/go-codec-dagpb/pull/29))\n - update go-ipld-prime, use go:generate\n - allow decoding PBNode fields in any order\n - expose APIs without Reader/Writer overhead\n - preallocate 1KiB on the stack for marshals\n - encode directly with a []byte\n - decode directly with a []byte\n - remove unnecessary xerrors dep\n- github.com/ipld/go-ipld-prime (v0.9.1-0.20210324083106-dc342a9917db -> v0.12.2):\n - Printer feature ([ipld/go-ipld-prime#238](https://github.com/ipld/go-ipld-prime/pull/238))\n - schema: keep TypeSystem names ordered\n - schema/dmt: redesign with bindnode and add Compile\n - codec: make cbor and json codecs use ErrUnexpectedEOF\n - bindnode: fix for stringjoin struct emission when first field is the empty string ([ipld/go-ipld-prime#239](https://github.com/ipld/go-ipld-prime/pull/239))\n - schema: typekind names are not capitalized.\n - Bindnode fixes continued ([ipld/go-ipld-prime#233](https://github.com/ipld/go-ipld-prime/pull/233))\n - helper methods for encoding and decoding ([ipld/go-ipld-prime#232](https://github.com/ipld/go-ipld-prime/pull/232))\n - mark v0.12.0\n - Major refactor: extract datamodel package.\n ([ipld/go-ipld-prime#228](https://github.com/ipld/go-ipld-prime/pull/228))\n - Fix ExploreRecursive stopAt condition, add tests, add error return to Explore (#229) ([ipld/go-ipld-prime#229](https://github.com/ipld/go-ipld-prime/pull/229))\n - selector: add tests which are driven by language-agnostic spec fixtures. ([ipld/go-ipld-prime#231](https://github.com/ipld/go-ipld-prime/pull/231))\n - selector: Improve docs for implementers. (#227) ([ipld/go-ipld-prime#227](https://github.com/ipld/go-ipld-prime/pull/227))\n - Bindnode fixes of opportunity ([ipld/go-ipld-prime#226](https://github.com/ipld/go-ipld-prime/pull/226))\n - node/bindnode: redesign the shape of unions in Go ([ipld/go-ipld-prime#223](https://github.com/ipld/go-ipld-prime/pull/223))\n - summary of the v0.11.0 changelog should holler even more about how cool bindnode is.\n - mark v0.11.0\n - node/bindnode: mark as experimental in its godoc.\n - codecs: more docs, a terminology guide, consistency in options. ([ipld/go-ipld-prime#221](https://github.com/ipld/go-ipld-prime/pull/221))\n - Changelog backfill.\n - selectors: docs enhancements, new construction helpers. ([ipld/go-ipld-prime#199](https://github.com/ipld/go-ipld-prime/pull/199))\n - Changelog backfill.\n - Allow parsing of single Null tokens from refmt\n - Add link conditions for 'stop-at' expression in ExploreRecursive selector ([ipld/go-ipld-prime#214](https://github.com/ipld/go-ipld-prime/pull/214))\n - Remove base64 padding for dag-json bytes as per spec\n - node/bindnode: temporarily skip Links schema test\n - test: add test for traversal of typed node links\n - fix: typed links LinkTargetNodePrototype should return ReferencedType\n - Make `go vet` happy\n - Add MapSortMode to MarshalOptions\n - Add {Unm,M}arshalOptions for explicit mode switching for cbor vs dagcbor\n - Sort map entries marshalling dag-cbor\n - node/bindnode: first pass at inferring IPLD schemas\n - Add {Unm,M}arshalOptions for explicit mode switching for json vs dagjson\n - Make tests pass with sorted dag-json output\n - Sort map entries marshalling dag-json\n - Simplify refmt usage\n - Fix failing test using dagjson encoding\n - Fix some failing tests using dagjson\n - Remove pretty-printing\n - Update readme linking to specs and meta repo.\n - Fix example names so they render on go.pkg.dev.\n - fluent/quip: remove in favor of qp\n - node/basic: add Chooser\n - schema: add TypedPrototype\n - node/bindnode: rethink and better document APIs\n - node/tests: cover yet more interface methods\n - node/tests: cover more error cases for scalar kinds\n - node/tests: add more extensive scalar kind tests\n - node/bindnode: start running all schema tests\n - mark v0.10.0\n - More changelog grooming.\n - Changelog grooming.\n - node/tests: put most of the schema test cases here\n - Add more explicit discussion of indices to ListIterator.\n - node/bindnode: start of a reflect-based Node implementation\n - add DeepEqual and start using it in tests\n - Add enumerate methods to the multicodec registries. ([ipld/go-ipld-prime#176](https://github.com/ipld/go-ipld-prime/pull/176))\n - Make a multicodec.Registry type available. ([ipld/go-ipld-prime#172](https://github.com/ipld/go-ipld-prime/pull/172))\n - fluent/qp: don't panic on string panics\n - Allow emitting & parsing of bytes per dagjson codec spec ([ipld/go-ipld-prime#166](https://github.com/ipld/go-ipld-prime/pull/166))\n - Package docs for dag-cbor.\n - Update package docs.\n - schema/gen/go: apply gofmt automatically ([ipld/go-ipld-prime#163](https://github.com/ipld/go-ipld-prime/pull/163))\n - schema/gen/go: fix remaining vet warnings on generated code\n - schema/gen/go: batch file writes via a bytes.Buffer ([ipld/go-ipld-prime#161](https://github.com/ipld/go-ipld-prime/pull/161))\n - schema/gen/go: avoid Maybe pointers for small types\n - fix readme formatting typo\n - feat(linksystem): add reification to LinkSystem ([ipld/go-ipld-prime#158](https://github.com/ipld/go-ipld-prime/pull/158))\n- github.com/libp2p/go-addr-util (v0.0.2 -> v0.1.0):\n - stop using the deprecated go-multiaddr-net package ([libp2p/go-addr-util#34](https://github.com/libp2p/go-addr-util/pull/34))\n - Remove `IsFDCostlyTransport` ([libp2p/go-addr-util#31](https://github.com/libp2p/go-addr-util/pull/31))\n- github.com/libp2p/go-libp2p (v0.14.3 -> v0.15.0):\n - chore: update go-tcp-transport to v0.2.8\n - implement the new mDNS spec, move the old mDNS implementation (#1161) ([libp2p/go-libp2p#1161](https://github.com/libp2p/go-libp2p/pull/1161))\n - remove deprecated basichost.New constructor ([libp2p/go-libp2p#1156](https://github.com/libp2p/go-libp2p/pull/1156))\n - Make BasicHost.evtLocalAddrsUpdated event emitter stateful. ([libp2p/go-libp2p#1147](https://github.com/libp2p/go-libp2p/pull/1147))\n - fix: deflake multipro echo test ([libp2p/go-libp2p#1149](https://github.com/libp2p/go-libp2p/pull/1149))\n - fix(basic_host): stream not closed when context done ([libp2p/go-libp2p#1148](https://github.com/libp2p/go-libp2p/pull/1148))\n - chore: update deps ([libp2p/go-libp2p#1141](https://github.com/libp2p/go-libp2p/pull/1141))\n - remove secio from examples ([libp2p/go-libp2p#1143](https://github.com/libp2p/go-libp2p/pull/1143))\n - remove deprecated Filter option ([libp2p/go-libp2p#1132](https://github.com/libp2p/go-libp2p/pull/1132))\n - fix: remove deprecated call ([libp2p/go-libp2p#1136](https://github.com/libp2p/go-libp2p/pull/1136))\n - test: fix flaky example test ([libp2p/go-libp2p#1135](https://github.com/libp2p/go-libp2p/pull/1135))\n - remove deprecated identify.ClientVersion ([libp2p/go-libp2p#1133](https://github.com/libp2p/go-libp2p/pull/1133))\n - remove Go version requirement and note about Go modules from README ([libp2p/go-libp2p#1126](https://github.com/libp2p/go-libp2p/pull/1126))\n - Error assignment fix ([libp2p/go-libp2p#1124](https://github.com/libp2p/go-libp2p/pull/1124))\n - perf/basic_host: Don't handle address change if we hasn't anyone ([libp2p/go-libp2p#1115](https://github.com/libp2p/go-libp2p/pull/1115))\n- github.com/libp2p/go-libp2p-core (v0.8.5 -> v0.9.0):\n - feat: remove unused metrics (#208) ([libp2p/go-libp2p-core#208](https://github.com/libp2p/go-libp2p-core/pull/208))\n - feat: keep addresses for longer (#207) ([libp2p/go-libp2p-core#207](https://github.com/libp2p/go-libp2p-core/pull/207))\n - remove deprecated key stretching struct / function (#203) ([libp2p/go-libp2p-core#203](https://github.com/libp2p/go-libp2p-core/pull/203))\n - remove deprecated Bytes method from the Key interface (#204) ([libp2p/go-libp2p-core#204](https://github.com/libp2p/go-libp2p-core/pull/204))\n - remove deprecated functions in the peer package (#205) ([libp2p/go-libp2p-core#205](https://github.com/libp2p/go-libp2p-core/pull/205))\n - remove deprecated constructor for the insecure transport (#206) ([libp2p/go-libp2p-core#206](https://github.com/libp2p/go-libp2p-core/pull/206))\n - feat: add helper functions for working with addr infos (#202) ([libp2p/go-libp2p-core#202](https://github.com/libp2p/go-libp2p-core/pull/202))\n - fix: make timestamps strictly increasing (#201) ([libp2p/go-libp2p-core#201](https://github.com/libp2p/go-libp2p-core/pull/201))\n - ci: use github-actions for compatibility testing (#200) ([libp2p/go-libp2p-core#200](https://github.com/libp2p/go-libp2p-core/pull/200))\n - sync: update CI config files (#189) ([libp2p/go-libp2p-core#189](https://github.com/libp2p/go-libp2p-core/pull/189))\n - remove minimum Go version from README (#199) ([libp2p/go-libp2p-core#199](https://github.com/libp2p/go-libp2p-core/pull/199))\n - remove flaky tests (#194) ([libp2p/go-libp2p-core#194](https://github.com/libp2p/go-libp2p-core/pull/194))\n - reduce default timeouts to 15s (#192) ([libp2p/go-libp2p-core#192](https://github.com/libp2p/go-libp2p-core/pull/192))\n - fix benchmark of key verifications (#190) ([libp2p/go-libp2p-core#190](https://github.com/libp2p/go-libp2p-core/pull/190))\n - fix staticcheck errors (#191) ([libp2p/go-libp2p-core#191](https://github.com/libp2p/go-libp2p-core/pull/191))\n - doc: document Close on Transport (#188) ([libp2p/go-libp2p-core#188](https://github.com/libp2p/go-libp2p-core/pull/188))\n - add a helper function to go directly from a string to an AddrInfo (#184) ([libp2p/go-libp2p-core#184](https://github.com/libp2p/go-libp2p-core/pull/184))\n- github.com/libp2p/go-libp2p-http (v0.2.0 -> v0.2.1):\n - remove Makefile ([libp2p/go-libp2p-http#70](https://github.com/libp2p/go-libp2p-http/pull/70))\n - fix staticcheck ([libp2p/go-libp2p-http#67](https://github.com/libp2p/go-libp2p-http/pull/67))\n - Revert \"increase buffer size\"\n - Increase read buffer size to reduce poll system calls ([libp2p/go-libp2p-http#66](https://github.com/libp2p/go-libp2p-http/pull/66))\n- github.com/libp2p/go-libp2p-kad-dht (v0.12.2 -> v0.13.1):\n - Extract validation from ProtocolMessenger ([libp2p/go-libp2p-kad-dht#741](https://github.com/libp2p/go-libp2p-kad-dht/pull/741))\n - remove codecov.yml ([libp2p/go-libp2p-kad-dht#742](https://github.com/libp2p/go-libp2p-kad-dht/pull/742))\n - integrate some basic opentelemetry tracing ([libp2p/go-libp2p-kad-dht#734](https://github.com/libp2p/go-libp2p-kad-dht/pull/734))\n - feat: delete GetValues ([libp2p/go-libp2p-kad-dht#728](https://github.com/libp2p/go-libp2p-kad-dht/pull/728))\n - chore: skip flaky test when race detector is enabled ([libp2p/go-libp2p-kad-dht#731](https://github.com/libp2p/go-libp2p-kad-dht/pull/731))\n - Dont count connection times in usefulness ([libp2p/go-libp2p-kad-dht#660](https://github.com/libp2p/go-libp2p-kad-dht/pull/660))\n - Routing table refresh should NOT block ([libp2p/go-libp2p-kad-dht#705](https://github.com/libp2p/go-libp2p-kad-dht/pull/705))\n - update bootstrapPeers to be func() []peer.AddrInfo (#716) ([libp2p/go-libp2p-kad-dht#716](https://github.com/libp2p/go-libp2p-kad-dht/pull/716))\n- github.com/libp2p/go-libp2p-noise (v0.2.0 -> v0.2.2):\n - remove note about go modules in README ([libp2p/go-libp2p-noise#100](https://github.com/libp2p/go-libp2p-noise/pull/100))\n - fix: remove deprecated call to pk.Bytes ([libp2p/go-libp2p-noise#99](https://github.com/libp2p/go-libp2p-noise/pull/99))\n- github.com/libp2p/go-libp2p-peerstore (v0.2.7 -> v0.2.8):\n - Fix performance issue in updating addr book ([libp2p/go-libp2p-peerstore#141](https://github.com/libp2p/go-libp2p-peerstore/pull/141))\n - Fix test flakes ([libp2p/go-libp2p-peerstore#164](https://github.com/libp2p/go-libp2p-peerstore/pull/164))\n - Only remove records during GC ([libp2p/go-libp2p-peerstore#135](https://github.com/libp2p/go-libp2p-peerstore/pull/135))\n - sync: update CI config files ([libp2p/go-libp2p-peerstore#160](https://github.com/libp2p/go-libp2p-peerstore/pull/160))\n - fix: fix some race conditions in the ds address book ([libp2p/go-libp2p-peerstore#161](https://github.com/libp2p/go-libp2p-peerstore/pull/161))\n - address lints and test failures ([libp2p/go-libp2p-peerstore#159](https://github.com/libp2p/go-libp2p-peerstore/pull/159))\n - stop using the deprecated go-multiaddr-net package ([libp2p/go-libp2p-peerstore#158](https://github.com/libp2p/go-libp2p-peerstore/pull/158))\n- github.com/libp2p/go-libp2p-pubsub (v0.4.2 -> v0.5.4):\n - make slowness a warning, with a user configurable threshold\n - reduce log spam from empty heartbeat messages\n - fix: code review\n - add support for custom protocol matching function\n - fix: remove deprecated Bytes call (#436) ([libp2p/go-libp2p-pubsub#436](https://github.com/libp2p/go-libp2p-pubsub/pull/436))\n - cleanup: fix vet and staticcheck failures (#435) ([libp2p/go-libp2p-pubsub#435](https://github.com/libp2p/go-libp2p-pubsub/pull/435))\n - Revert noisy newline changes\n - fix: avoid panic when peer is blacklisted after connection\n - release priority locks early when handling batches\n - don't respawn writer if we fail to open a stream; declare it a peer error\n - batch process dead peer notifications\n - use a priority lock instead of a semaphore\n - do the notification in a goroutine\n - emit new peer notification without holding the semaphore\n - use a semaphore for new peer notifications so that we don't block the event loop\n - don't accumulate pending goroutines from new connections\n - rename RawTracer's DroppedInSubscribe into UndeliverableMessage\n - add a new RawTracer event to track messages dropped in Subscribe\n - add an option to configure the Subscription output queue length\n - fix some comments\n - expose more events for RawTracer\n - Make close concurrent safe\n - Fix close of closed channel\n - Update README to point to correct example directory (#424) ([libp2p/go-libp2p-pubsub#424](https://github.com/libp2p/go-libp2p-pubsub/pull/424))\n - fix: remove deprecated and never used topic descriptors (#423) ([libp2p/go-libp2p-pubsub#423](https://github.com/libp2p/go-libp2p-pubsub/pull/423))\n - Refactor Gossipsub Parameters To Make Them More Configurable (#421) ([libp2p/go-libp2p-pubsub#421](https://github.com/libp2p/go-libp2p-pubsub/pull/421))\n - add tests for gs features and custom protocols\n - add support for custom gossipsub protocols and feature tests\n - RIP travis, Long Live CircleCI (#414) ([libp2p/go-libp2p-pubsub#414](https://github.com/libp2p/go-libp2p-pubsub/pull/414))\n - Ignore transient connections (#412) ([libp2p/go-libp2p-pubsub#412](https://github.com/libp2p/go-libp2p-pubsub/pull/412))\n - demote log spam to debug\n - fix bug\n - add last amount of validation\n - add threshold validation\n - strengthen validation\n - rename checkSignature to checkSigningPolicy\n - rename validation.Publish to PushLocal\n - fix TestValidate, add TestValidate2\n - skip flaky test until we can fix it\n - implement synchronous validation for locally published messages\n - expose internalTracer as RawTracer\n - export rejection named string constants\n - more intelligent handling of ip whitelist check\n - remove obsolete explicit IP whitelisting in favor of subnets\n - add subnet whitelisting for IPColocation\n- github.com/libp2p/go-libp2p-quic-transport (v0.11.2 -> v0.12.0):\n - sync: update CI config files (#228) ([libp2p/go-libp2p-quic-transport#228](https://github.com/libp2p/go-libp2p-quic-transport/pull/228))\n - fix closing of streams in example ([libp2p/go-libp2p-quic-transport#221](https://github.com/libp2p/go-libp2p-quic-transport/pull/221))\n - close all UDP connections when the reuse is closed ([libp2p/go-libp2p-quic-transport#216](https://github.com/libp2p/go-libp2p-quic-transport/pull/216))\n - fix staticcheck ([libp2p/go-libp2p-quic-transport#217](https://github.com/libp2p/go-libp2p-quic-transport/pull/217))\n - sync: update CI config files (#214) ([libp2p/go-libp2p-quic-transport#214](https://github.com/libp2p/go-libp2p-quic-transport/pull/214))\n - implement a Transport.Close that waits for the reuse's GC to finish ([libp2p/go-libp2p-quic-transport#211](https://github.com/libp2p/go-libp2p-quic-transport/pull/211))\n - don't compare peer IDs when hole punching ([libp2p/go-libp2p-quic-transport#210](https://github.com/libp2p/go-libp2p-quic-transport/pull/210))\n - add hole punching support (#194) ([libp2p/go-libp2p-quic-transport#194](https://github.com/libp2p/go-libp2p-quic-transport/pull/194))\n- github.com/libp2p/go-libp2p-swarm (v0.5.0 -> v0.5.3):\n - sync: update CI config files ([libp2p/go-libp2p-swarm#263](https://github.com/libp2p/go-libp2p-swarm/pull/263))\n - remove incorrect call to InterceptAddrDial ([libp2p/go-libp2p-swarm#260](https://github.com/libp2p/go-libp2p-swarm/pull/260))\n - speed up the TestFDLimitUnderflow test ([libp2p/go-libp2p-swarm#262](https://github.com/libp2p/go-libp2p-swarm/pull/262))\n - sync: update CI config files (#248) ([libp2p/go-libp2p-swarm#248](https://github.com/libp2p/go-libp2p-swarm/pull/248))\n- github.com/libp2p/go-libp2p-testing (v0.4.0 -> v0.4.2):\n - fix deadlock in the transport's serve function ([libp2p/go-libp2p-testing#35](https://github.com/libp2p/go-libp2p-testing/pull/35))\n - fix: cleanup transport suite ([libp2p/go-libp2p-testing#34](https://github.com/libp2p/go-libp2p-testing/pull/34))\n - Address `go vet` and `saticcheck` issues ([libp2p/go-libp2p-testing#33](https://github.com/libp2p/go-libp2p-testing/pull/33))\n - Defer closing stream for reading ([libp2p/go-libp2p-testing#32](https://github.com/libp2p/go-libp2p-testing/pull/32))\n- github.com/libp2p/go-libp2p-tls (v0.1.3 -> v0.2.0):\n - fix: don't fail the handshake when the libp2p extension is critical ([libp2p/go-libp2p-tls#88](https://github.com/libp2p/go-libp2p-tls/pull/88))\n - fix deprecated call to key.Bytes ([libp2p/go-libp2p-tls#86](https://github.com/libp2p/go-libp2p-tls/pull/86))\n - fix usage of deprecated peer.IDB58Decode ([libp2p/go-libp2p-tls#77](https://github.com/libp2p/go-libp2p-tls/pull/77))\n - remove setting of the TLS 1.3 GODEBUG flag ([libp2p/go-libp2p-tls#68](https://github.com/libp2p/go-libp2p-tls/pull/68))\n - improve the error message returned when peer verification fails ([libp2p/go-libp2p-tls#57](https://github.com/libp2p/go-libp2p-tls/pull/57))\n - update to Go 1.14 ([libp2p/go-libp2p-tls#54](https://github.com/libp2p/go-libp2p-tls/pull/54))\n - Update deps and fix tests ([libp2p/go-libp2p-tls#43](https://github.com/libp2p/go-libp2p-tls/pull/43))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.4.2 -> v0.4.6):\n - chore: update deps ([libp2p/go-libp2p-transport-upgrader#78](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/78))\n - fix typo in error message ([libp2p/go-libp2p-transport-upgrader#77](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/77))\n - fix staticcheck ([libp2p/go-libp2p-transport-upgrader#74](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/74))\n - don't listen on all interfaces in tests ([libp2p/go-libp2p-transport-upgrader#73](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/73))\n - stop using the deprecated go-multiaddr-net ([libp2p/go-libp2p-transport-upgrader#72](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/72))\n- github.com/libp2p/go-libp2p-xor (v0.0.0-20200501025846-71e284145d58 -> v0.0.0-20210714161855-5c005aca55db):\n - Add immutable remove operation ([libp2p/go-libp2p-xor#14](https://github.com/libp2p/go-libp2p-xor/pull/14))\n - fix go vet and staticcheck ([libp2p/go-libp2p-xor#11](https://github.com/libp2p/go-libp2p-xor/pull/11))\n- github.com/libp2p/go-reuseport-transport (v0.0.4 -> v0.0.5):\n - remove note about Go modules in README ([libp2p/go-reuseport-transport#32](https://github.com/libp2p/go-reuseport-transport/pull/32))\n - stop using the deprecated go-multiaddr-net package ([libp2p/go-reuseport-transport#30](https://github.com/libp2p/go-reuseport-transport/pull/30))\n- github.com/libp2p/go-socket-activation (v0.0.2 -> v0.1.0):\n - chore: stop using the deprecated go-multiaddr-net package ([libp2p/go-socket-activation#16](https://github.com/libp2p/go-socket-activation/pull/16))\n - fix staticcheck ([libp2p/go-socket-activation#13](https://github.com/libp2p/go-socket-activation/pull/13))\n- github.com/libp2p/go-tcp-transport (v0.2.4 -> v0.2.8):\n - disable metrics collection on Windows ([libp2p/go-tcp-transport#93](https://github.com/libp2p/go-tcp-transport/pull/93))\n - sync: update CI config files (#90) ([libp2p/go-tcp-transport#90](https://github.com/libp2p/go-tcp-transport/pull/90))\n - chore: update go-libp2p-transport-upgrader and go-reuseport-transport ([libp2p/go-tcp-transport#84](https://github.com/libp2p/go-tcp-transport/pull/84))\n- github.com/libp2p/go-ws-transport (v0.4.0 -> v0.5.0):\n - chore: update go-libp2p-transport-upgrader and go-libp2p-core ([libp2p/go-ws-transport#103](https://github.com/libp2p/go-ws-transport/pull/103))\n - remove deprecated type ([libp2p/go-ws-transport#102](https://github.com/libp2p/go-ws-transport/pull/102))\n - sync: update CI config files ([libp2p/go-ws-transport#101](https://github.com/libp2p/go-ws-transport/pull/101))\n - chore: various cleanups required to get vet/staticcheck/test to pass ([libp2p/go-ws-transport#100](https://github.com/libp2p/go-ws-transport/pull/100))\n- github.com/lucas-clemente/quic-go (v0.21.2 -> v0.23.0):\n - update to Go 1.17.x ([lucas-clemente/quic-go#3258](https://github.com/lucas-clemente/quic-go/pull/3258))\n - quicvarint: export Min and Max (#3253) ([lucas-clemente/quic-go#3253](https://github.com/lucas-clemente/quic-go/pull/3253))\n - drop support for Go 1.15 ([lucas-clemente/quic-go#3247](https://github.com/lucas-clemente/quic-go/pull/3247))\n - quicvarint: add Reader and Writer interfaces (#3233) ([lucas-clemente/quic-go#3233](https://github.com/lucas-clemente/quic-go/pull/3233))\n - fix race when stream.Read and CancelRead are called concurrently ([lucas-clemente/quic-go#3241](https://github.com/lucas-clemente/quic-go/pull/3241))\n - also count coalesced 0-RTT packets in the integration tests ([lucas-clemente/quic-go#3251](https://github.com/lucas-clemente/quic-go/pull/3251))\n - remove draft versions 32 and 34 from README (#3244) ([lucas-clemente/quic-go#3244](https://github.com/lucas-clemente/quic-go/pull/3244))\n - update Changelog ([lucas-clemente/quic-go#3245](https://github.com/lucas-clemente/quic-go/pull/3245))\n - optimize hasOutstandingCryptoPackets in sentPacketHandler ([lucas-clemente/quic-go#3230](https://github.com/lucas-clemente/quic-go/pull/3230))\n - permit underlying conn to implement batch interface directly ([lucas-clemente/quic-go#3237](https://github.com/lucas-clemente/quic-go/pull/3237))\n - cancel the PTO timer when all Handshake packets are acknowledged ([lucas-clemente/quic-go#3231](https://github.com/lucas-clemente/quic-go/pull/3231))\n - fix flaky INVALID_TOKEN server test ([lucas-clemente/quic-go#3223](https://github.com/lucas-clemente/quic-go/pull/3223))\n - drop support for QUIC draft version 32 and 34 ([lucas-clemente/quic-go#3217](https://github.com/lucas-clemente/quic-go/pull/3217))\n - fix flaky 0-RTT integration test ([lucas-clemente/quic-go#3224](https://github.com/lucas-clemente/quic-go/pull/3224))\n - use batched reads ([lucas-clemente/quic-go#3142](https://github.com/lucas-clemente/quic-go/pull/3142))\n - add a config option to disable sending of Version Negotiation packets ([lucas-clemente/quic-go#3216](https://github.com/lucas-clemente/quic-go/pull/3216))\n - remove the RetireBugBackwardsCompatibilityMode ([lucas-clemente/quic-go#3213](https://github.com/lucas-clemente/quic-go/pull/3213))\n - remove outdated ackhandler test case ([lucas-clemente/quic-go#3212](https://github.com/lucas-clemente/quic-go/pull/3212))\n - remove unused StripGreasedVersions function ([lucas-clemente/quic-go#3214](https://github.com/lucas-clemente/quic-go/pull/3214))\n - fix incorrect usage of errors.Is ([lucas-clemente/quic-go#3215](https://github.com/lucas-clemente/quic-go/pull/3215))\n - return error on SendMessage when session is closed ([lucas-clemente/quic-go#3218](https://github.com/lucas-clemente/quic-go/pull/3218))\n - remove a redundant error check ([lucas-clemente/quic-go#3210](https://github.com/lucas-clemente/quic-go/pull/3210))\n - update golangci-lint to v1.41.1 ([lucas-clemente/quic-go#3205](https://github.com/lucas-clemente/quic-go/pull/3205))\n - Update doc for dialer in http3.RoundTripper ([lucas-clemente/quic-go#3208](https://github.com/lucas-clemente/quic-go/pull/3208))\n- github.com/multiformats/go-multiaddr (v0.3.3 -> v0.4.0):\n - remove forced dependency on deprecated go-maddr-filter ([multiformats/go-multiaddr#162](https://github.com/multiformats/go-multiaddr/pull/162))\n - remove deprecated SwapToP2pMultiaddrs ([multiformats/go-multiaddr#161](https://github.com/multiformats/go-multiaddr/pull/161))\n - remove Makefile ([multiformats/go-multiaddr#163](https://github.com/multiformats/go-multiaddr/pull/163))\n - remove deprecated filter functions ([multiformats/go-multiaddr#157](https://github.com/multiformats/go-multiaddr/pull/157))\n - remove deprecated NetCodec ([multiformats/go-multiaddr#159](https://github.com/multiformats/go-multiaddr/pull/159))\n - add Noise ([multiformats/go-multiaddr#156](https://github.com/multiformats/go-multiaddr/pull/156))\n - Add TLS protocol ([multiformats/go-multiaddr#153](https://github.com/multiformats/go-multiaddr/pull/153))\n- github.com/multiformats/go-multicodec (v0.2.0 -> v0.3.0):\n - Export reserved range constants (#53) ([multiformats/go-multicodec#53](https://github.com/multiformats/go-multicodec/pull/53))\n - make Code.Set accept valid code numbers\n - replace Of with Code.Set, implementing flag.Value\n - add multiformats/multicodec as a git submodule\n - update the generator with the \"status\" CSV column\n - Run `go generate` to generate the latest codecs\n - Add lookup for multicodec code by string name ([multiformats/go-multicodec#40](https://github.com/multiformats/go-multicodec/pull/40))\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Daniel Martí | 42 | +8549/-6587 | 170 |\n| Eric Myhre | 55 | +5883/-6715 | 395 |\n| Marten Seemann | 100 | +1814/-2028 | 275 |\n| Steven Allen | 80 | +1573/-1998 | 127 |\n| hannahhoward | 18 | +1721/-671 | 53 |\n| Will | 2 | +1114/-1217 | 18 |\n| Andrew Gillis | 2 | +1220/-720 | 14 |\n| gammazero | 3 | +43/-1856 | 10 |\n| Masih H. Derkani | 3 | +960/-896 | 8 |\n| Adin Schmahmann | 25 | +1458/-313 | 44 |\n| vyzo | 27 | +986/-353 | 60 |\n| Will Scott | 6 | +852/-424 | 16 |\n| Rod Vagg | 19 | +983/-255 | 66 |\n| Petar Maymounkov | 6 | +463/-179 | 22 |\n| web3-bot | 10 | +211/-195 | 24 |\n| adlrocha | 1 | +330/-75 | 15 |\n| RubenKelevra | 2 | +128/-210 | 2 |\n| Ian Davis | 3 | +200/-109 | 17 |\n| Cory Schwartz | 3 | +231/-33 | 7 |\n| Keenan Nemetz | 1 | +184/-71 | 2 |\n| Randy Reddig | 2 | +187/-53 | 8 |\n| Takashi Matsuda | 3 | +201/-2 | 7 |\n| guseggert | 4 | +161/-20 | 9 |\n| Lucas Molas | 5 | +114/-47 | 27 |\n| nisdas | 4 | +115/-45 | 7 |\n| Michael Muré | 6 | +107/-33 | 24 |\n| Richard Ramos | 2 | +113/-9 | 3 |\n| Marcin Rataj | 12 | +88/-24 | 13 |\n| Ondrej Prazak | 2 | +104/-6 | 4 |\n| Michal Dobaczewski | 2 | +77/-28 | 3 |\n| Jorropo | 3 | +9/-75 | 4 |\n| Andey Robins | 1 | +70/-3 | 3 |\n| Gus Eggert | 10 | +34/-31 | 12 |\n| noot | 1 | +54/-9 | 5 |\n| Maxim Merzhanov | 1 | +29/-24 | 1 |\n| Adrian Lanzafame | 1 | +30/-13 | 2 |\n| Bogdan Stirbat | 1 | +22/-16 | 2 |\n| Shad Sterling | 1 | +28/-3 | 1 |\n| Jesse Bouwman | 5 | +30/-0 | 5 |\n| Pavel Karpy | 1 | +19/-7 | 2 |\n| lasiar | 5 | +14/-10 | 5 |\n| Dennis Trautwein | 1 | +20/-4 | 2 |\n| Louis Thibault | 1 | +22/-1 | 2 |\n| whyrusleeping | 2 | +21/-1 | 2 |\n| aarshkshah1992 | 3 | +12/-8 | 3 |\n| Peter Rabbitson | 2 | +20/-0 | 2 |\n| bt90 | 2 | +17/-2 | 2 |\n| Dominic Della Valle | 1 | +13/-1 | 2 |\n| Audrius Butkevicius | 1 | +12/-1 | 1 |\n| Brian Strauch | 1 | +9/-3 | 1 |\n| Aarsh Shah | 2 | +1/-11 | 2 |\n| Whyrusleeping | 1 | +11/-0 | 1 |\n| Max | 1 | +7/-3 | 1 |\n| vallder | 1 | +3/-5 | 1 |\n| Michael Burns | 3 | +2/-6 | 3 |\n| Lasse Johnsen | 1 | +4/-4 | 2 |\n| snyh | 1 | +5/-2 | 1 |\n| Hector Sanjuan | 2 | +3/-2 | 2 |\n| 市川恭佑 (ebi) | 1 | +1/-3 | 1 |\n| godcong | 2 | +2/-1 | 2 |\n| Mathis Engelbart | 1 | +1/-2 | 1 |\n| folbrich | 1 | +1/-1 | 1 |\n| Med Mouine | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.11.md", "content": "# go-ipfs changelog v0.11\n\n## v0.11.1 2022-04-08\n\nThis patch release fixes a security issue wherein traversing some malformed DAGs can cause the node to panic.\n\nSee also the security advisory: https://github.com/ipfs/go-ipfs/security/advisories/GHSA-mcq2-w56r-5w2w\n\nNote: the v0.11.1 patch release contains the Docker compose fix from v0.12.1 as well\n\n### Changelog\n\n
\nFull Changelog\n- github.com/ipld/go-codec-dagpb (v1.3.0 -> v1.3.2):\n - fix: use protowire for Links bytes decoding\n
\n\n### ❤ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Rod Vagg | 1 | +34/-19 | 2 |\n\n## v0.11.0 2021-12-08\n\nWe're happy to announce go-ipfs 0.11.0. This release comes with improvements to the UnixFS Sharding and PubSub experiments as well as support for Circuit-Relay v2 which sets the network up for decentralized hole punching support.\n\nAs usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will _not_ be called out in the release notes. Please make sure to update ASAP. See our [release process](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md#security-fix-policy) for details.\n\n### 🛠 BREAKING CHANGES\n\n- UnixFS sharding is now automatic and enabled by default\n - HAMT-based sharding is applied to large directories (i.e. those that would serialize into [block](https://docs.ipfs.tech/concepts/glossary/#block) larger than ~256KiB)s. This means importing data via commands like `ipfs add -r ` may result in different [CID](https://docs.ipfs.tech/concepts/glossary/#cid)s due to the different [DAG](https://docs.ipfs.tech/concepts/glossary/#dag) representations.\n - Support for `Experimental.ShardingEnabled` is removed.\n- go-ipfs can no longer act as a [Circuit Relay](https://docs.ipfs.tech/concepts/glossary/#circuit-relay) v1\n - Node will refuse to start if `Swarm.EnableRelayHop` is set to `true`\n - If you depend on v1 relay service provider, see \"Removal of v1 relay service\" section for available migration options.\n- HTTP RPC wire format for experimental commands at `/api/v0/pubsub` changed.\n - If you use [js-ipfs-http-client](https://www.npmjs.com/package/ipfs-http-client) or [go-ipfs-http-client](https://github.com/ipfs/go-ipfs-http-client), just update to their latest version.\n - If you use something else, see \"Multibase in PubSub\" section below for migration details.\n\nKeep reading to learn more details.\n\n### 🔦 Highlights\n\n#### 🗃 Automatic UnixFS sharding\n\nTruly big directories can have so many items, that the root block with all of their names is too big to be exchanged with other peers. This was partially solved by [HAMT-sharding](https://docs.ipfs.tech/concepts/glossary/#hamt-sharding), which was introduced a while ago as opt-in. The main downside of the implementation was that it was a global flag that sharded all imported directories (big and small).\n\nThis release solves that inconvenience by making UnixFS sharding smarter and applies it only to larger directories (i.e. directories that would be at least ~256KiB). This is now the default behavior in `ipfs add` and `ipfs files` commands, where UnixFS sharding works out-of-the-box.\n\n#### 🔁 Circuit Relay v2\n\nThis release adds support for the [circuit relay v2](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md) protocol based on the reference implementation from [go-libp2p 0.16](https://github.com/libp2p/go-libp2p/releases/tag/v0.16.0).\n\nThis is the cornerstone for maximizing p2p connections between IPFS peers. Every publicly dialable peer can now act as a limited relay v2, which can be used for [hole punching](https://docs.ipfs.tech/concepts/glossary/#hole-punching) and other decentralized signaling protocols.\n\n##### Limited relay v2 configuration options\n\ngo-ipfs can now be configured to act as a [`RelayClient`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmrelayclient) that uses other peers for autorelay functionality when behind a NAT, or provide a limited [`RelayService`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmrelayservice) to other peers on the network.\n\nStarting with go-ipfs v0.11 every publicly dialable go-ipfs (based on AutoNAT determination) will start a limited `RelayService`. `RelayClient` remains disabled by default for now, as we want the network to update and get enough v2 service providers first.\n\nNote: the limited Circuit Relay v2 provided with this release only allows low-bandwidth protocols (identify, ping, holepunch) over transient connections. If you want to relay things like bitswap sessions, you need to set up a v1 relay by some other means. See details below.\n\n##### Removal of unlimited v1 relay service provider\n\nSwitching to v2 of the relay spec means removal or deprecation of configuration keys that were specific to v1.\n\n- Relay transport and client support circuit-relay v2:\n - `Swarm.EnableAutoRelay` was replaced by `Swarm.RelayClient.Enabled`.\n - `Swarm.DisableRelay` is deprecated, relay transport can be now disabled globally (both client and service) by setting `Swarm.Transports.Network.Relay` to `false`\n- Relay v1 service provider was replaced by v2:\n - `Swarm.EnableRelayHop` no longer starts an unlimited v1 relay. If you have it set to `true` the node will refuse to start and display an error message.\n - Existing users who choose to continue running a v1 relay should migrate their setups to relay v1 based on js-ipfs running in node, or the standalone [libp2p-relay-daemon](https://dist.ipfs.tech/#libp2p-relay-daemon) [configured](https://github.com/libp2p/go-libp2p-relay-daemon/#configuration) with `RelayV1.Enabled` set to `true`. Be mindful that v1 relays are unlimited, and one may want to set up some ACL based either on PeerIDs or Subnets.\n\n#### 🕳 Decentralized Hole Punching (DCUtR protocol client)\n\nWe are working towards enabling hole punching for NAT traversal when port forwarding is not possible.\n\n[go-libp2p 0.16](https://github.com/libp2p/go-libp2p/releases/tag/v0.16.0) provides an implementation of the [DCUtR (decentralized hole punching)](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md) protocol. It is hidden behind the `Swarm.EnableHolePunching` configuration flag.\n\nWhen enabled, go-ipfs will coordinate with the counterparty using a [relayed v2 connection](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md), to [upgrade to a direct connection](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md) through a NAT/firewall whenever possible.\n\nThis feature is disabled by default in this release, but we hope to enable it by default as soon the network updates to go-ipfs v0.11 and gains a healthy set of limited v2 relays.\n\n#### 💬 Multibase in PubSub HTTP RPC API\n\nThis release fixed some edge cases that were reported by users of the PubSub experiment, getting it closer to becoming a stable feature of go-ipfs. Some PubSub users will notice that the plaintext limitation is lifted: one can now use line breaks in messages published to non-ascii topic names, or even publish arbitrary bytes to arbitrary topics. It required a change to the wire format used when pubsub commands are executed over the HTTP RPC API at `/api/v0/pubsub/*`, and also modified the behavior of the `ipfs pubsub pub` command, which now is publishing only a single pubsub message with data read from a file or stdin.\n\n##### PubSub client migration tips\n\nIf you use the HTTP RPC API with the [go-ipfs-http-client](https://github.com/ipfs/go-ipfs-http-client) library, make sure to update to the latest version. The next version of [js-ipfs-http-client](https://www.npmjs.com/package/ipfs-http-client) will use the new wire format as well, so you don't need to do anything.\n\nIf you use `/api/v0/pubsub/*` directly or maintain your own client library, you must adjust your HTTP client code. Byte fields and URL args are now encoded in `base64url` [Multibase](https://docs.ipfs.tech/concepts/glossary/#multibase). Encode/decode bytes using the `ipfs multibase --help` commands, or use the multiformats libraries ([js-multiformats](https://github.com/multiformats/js-multiformats#readme), [go-multibase](https://github.com/multiformats/go-multibase)).\n\nLow level changes:\n- `topic` passed as URL `arg` in requests to `/api/v0/pubsub/*` must be encoded in URL-safe multibase (`base64url`)\n- `data`, `from`, `seqno` and `topicIDs` returned in JSON responses are now encoded in multibase\n- Peer IDs returned in `from` now use the same default text representation from go-libp2p and peerid encoder/decoder from libp2p. This means the same text representation as in as in `swarm peers`, which makes it possible to compare them without decoding multibase.\n- `/api/v0/pubsub/pub` no longer accepts `data` to be passed as URL, it has to be sent as `multipart/form-data`. This removes size limitations based on URL length, and enables regular HTTP clients to publish data to PubSub topics. For example, to publish `some-file` to topic named `test-topic` using vanilla `curl`, one would execute: `curl -X POST -v -F \"stdin=@some-file\" 'http://127.0.0.1:5001/api/v0/pubsub/pub?arg=$(echo -n test-topic | ipfs multibase encode -b base64url)'`\n- `ipfs pubsub pub` on the command line no longer accepts variadic `data` arguments. Instead, it expects a single file input or stream of bytes from stdin. This ensures arbitrary stream of bytes can be published, removing limitation around messages that include `\\n` or `\\r\\n`.\n\n#### ⚙ New configuration flags\n\n- [`Addresses.AppendAnnounce`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#addressesappendannounce) is an array of multiaddrs, similar to `Addresses.Announce`, except it does not override inferred swarm addresses, but appends custom ones to the list.\n- Pubsub experiments can now be enabled via config, removing the need for CLI flag to be passed every time daemon starts:\n - [`Pubsub.Enabled`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#pubsubenabled) enables the pubsub system.\n - [`Ipns.UsePubsub`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#ipnsusepubsub) enables IPFS over pubsub experiment for publishing IPNS records in real time.\n\n#### 🔐 Support for DAG-JOSE IPLD codec\n\nJOSE is a [standard](https://datatracker.ietf.org/wg/jose/documents/) for signing and encrypting JSON objects. [DAG-JOSE](https://ipld.io/specs/codecs/dag-jose/spec/) is an IPLD codec based on JOSE and represented in CBOR. Upon encountering the `dag-jose` multicodec indicator, implementations can expect that the block contains dag-cbor encoded data which matches the IPLD schema from the [DAG-JOSE spec](https://ipld.io/specs/codecs/dag-jose/spec/).\n\nThis work was [contributed](https://github.com/ipfs/go-ipfs/pull/8569) by [Ceramic](https://ceramic.network/) and acts as a template for future IPFS improvements driven by the real world needs of the IPFS community.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - docs: update changelog for v0.11.0\n - Release v0.11.0-rc2\n - fix(corehttp): adjust peer counting metrics (#8577) ([ipfs/go-ipfs#8577](https://github.com/ipfs/go-ipfs/pull/8577))\n - Release v0.11.0-rc1\n - feat: Swarm.EnableHolePunching flag (#8562) ([ipfs/go-ipfs#8562](https://github.com/ipfs/go-ipfs/pull/8562))\n - feat: enabling pubsub and ipns-pubsub via config flags (#8510) ([ipfs/go-ipfs#8510](https://github.com/ipfs/go-ipfs/pull/8510))\n - Integrate go-dag-jose plugin (#8569) ([ipfs/go-ipfs#8569](https://github.com/ipfs/go-ipfs/pull/8569))\n - feat: Addresses.AppendAnnounce (#8177) ([ipfs/go-ipfs#8177](https://github.com/ipfs/go-ipfs/pull/8177))\n - fix: multibase in pubsub http rpc (#8183) ([ipfs/go-ipfs#8183](https://github.com/ipfs/go-ipfs/pull/8183))\n - refactor: remove dir-index-html submodule ([ipfs/go-ipfs#8555](https://github.com/ipfs/go-ipfs/pull/8555))\n - feat: hard deprecation of IPFS_REUSEPORT\n - feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) ([ipfs/go-ipfs#8563](https://github.com/ipfs/go-ipfs/pull/8563))\n - chore: fix link in README.md (#8551) ([ipfs/go-ipfs#8551](https://github.com/ipfs/go-ipfs/pull/8551))\n - Updating release template based off some 0.10 learnings (#8491) ([ipfs/go-ipfs#8491](https://github.com/ipfs/go-ipfs/pull/8491))\n - fix: multiple subdomain gateways on same domain (#8556) ([ipfs/go-ipfs#8556](https://github.com/ipfs/go-ipfs/pull/8556))\n - Fix typos (#8548) ([ipfs/go-ipfs#8548](https://github.com/ipfs/go-ipfs/pull/8548))\n - Add support for multiple files to `ipfs files rm`.\n - add a docker-compose file (#8387) ([ipfs/go-ipfs#8387](https://github.com/ipfs/go-ipfs/pull/8387))\n - fix(sharness): use -Q option instead of pipe to tail cmd\n - Add Homebrew installation method. ([ipfs/go-ipfs#8545](https://github.com/ipfs/go-ipfs/pull/8545))\n - docs: fix ipfs files cp examples (#8533) ([ipfs/go-ipfs#8533](https://github.com/ipfs/go-ipfs/pull/8533))\n - fix(unixfs): check for errors before dereferencing the link ([ipfs/go-ipfs#8508](https://github.com/ipfs/go-ipfs/pull/8508))\n - chore: replace go-merkledag walk with go-ipld-prime traversal for dag export (#8506) ([ipfs/go-ipfs#8506](https://github.com/ipfs/go-ipfs/pull/8506))\n - test: add sharness test for reading ADLs with FUSE\n - fix: allow the levelds compression level to be unspecified\n - ([ipfs/go-ipfs#8457](https://github.com/ipfs/go-ipfs/pull/8457))\n - ([ipfs/go-ipfs#8482](https://github.com/ipfs/go-ipfs/pull/8482))\n - Added the missing heathcheck for the container (#8429) ([ipfs/go-ipfs#8429](https://github.com/ipfs/go-ipfs/pull/8429))\n - chore: update dir-index-html to v1.2.2\n - Update RELEASE_ISSUE_TEMPLATE.md\n - Update RELEASE_ISSUE_TEMPLATE.md\n - add more logging to flaky TestPeersTotal\n - Update RELEASE_ISSUE_TEMPLATE.md\n - Update RELEASE_ISSUE_TEMPLATE.md\n - Updating chocolatey to reference go-ipfs\n - chore: update changelog for v0.10.0\n - add testground plans to bitswap on CI\n - ci: move Docker image build to Actions (#8467) ([ipfs/go-ipfs#8467](https://github.com/ipfs/go-ipfs/pull/8467))\n - fix(cli): object add-link: do not allow blocks over BS limit (#8414) ([ipfs/go-ipfs#8414](https://github.com/ipfs/go-ipfs/pull/8414))\n - fuse: load unixfs adls as their dagpb substrates\n - enable the legacy mDNS implementation\n - change ipfs dag get flag name from format to output-codec ([ipfs/go-ipfs#8440](https://github.com/ipfs/go-ipfs/pull/8440))\n - change names of ipfs dag put flags to make changes clearer ([ipfs/go-ipfs#8439](https://github.com/ipfs/go-ipfs/pull/8439))\n - test: check behavior of loading UnixFS sharded directories with missing shards\n - ([ipfs/go-ipfs#8432](https://github.com/ipfs/go-ipfs/pull/8432))\n - feat: dag import --stats (#8237) ([ipfs/go-ipfs#8237](https://github.com/ipfs/go-ipfs/pull/8237))\n - feat: ipfs-webui v2.13.0 (#8430) ([ipfs/go-ipfs#8430](https://github.com/ipfs/go-ipfs/pull/8430))\n - feat(cli): add daemon option --agent-version-suffix (#8419) ([ipfs/go-ipfs#8419](https://github.com/ipfs/go-ipfs/pull/8419))\n - feat: multibase transcode command (#8403) ([ipfs/go-ipfs#8403](https://github.com/ipfs/go-ipfs/pull/8403))\n - fix: take the lock while listing peers\n - feature: 'ipfs swarm peering' command (#8147) ([ipfs/go-ipfs#8147](https://github.com/ipfs/go-ipfs/pull/8147))\n - chore: update IPFS Desktop testing steps (#8393) ([ipfs/go-ipfs#8393](https://github.com/ipfs/go-ipfs/pull/8393))\n - add more buttons; remove some sections covered in the docs; general cleanup ([ipfs/go-ipfs#8274](https://github.com/ipfs/go-ipfs/pull/8274))\n - Cosmetic fixups in examples (#8325) ([ipfs/go-ipfs#8325](https://github.com/ipfs/go-ipfs/pull/8325))\n - perf: use performance-enhancing FUSE mount options\n - ci: publish Docker images for bifrost-* branches\n - chore: add comments to peerlog plugin about being unsupported\n - test: add unit tests for peerlog config parsing\n - ci: preload peerlog plugin, disable by default\n - fix(mkreleaselog): specify the parent commit when diffing\n - update version to v0.11.0-dev\n- github.com/ipfs/go-bitswap (v0.4.0 -> v0.5.1):\n - Version 0.5.1\n - Change incorrect function name in README (#541) ([ipfs/go-bitswap#541](https://github.com/ipfs/go-bitswap/pull/541))\n - Version 0.5.0 (#540) ([ipfs/go-bitswap#540](https://github.com/ipfs/go-bitswap/pull/540))\n - feat: plumb through contexts (#539) ([ipfs/go-bitswap#539](https://github.com/ipfs/go-bitswap/pull/539))\n - sync: update CI config files (#538) ([ipfs/go-bitswap#538](https://github.com/ipfs/go-bitswap/pull/538))\n - fix: optimize handling for peers with lots of tasks ([ipfs/go-bitswap#537](https://github.com/ipfs/go-bitswap/pull/537))\n - Enable custom task prioritization logic ([ipfs/go-bitswap#535](https://github.com/ipfs/go-bitswap/pull/535))\n - feat: cache the materialized wantlist ([ipfs/go-bitswap#530](https://github.com/ipfs/go-bitswap/pull/530))\n - fix: reduce receive contention ([ipfs/go-bitswap#536](https://github.com/ipfs/go-bitswap/pull/536))\n - Fix ProviderQueryManager test timings ([ipfs/go-bitswap#534](https://github.com/ipfs/go-bitswap/pull/534))\n - fix: rename wiretap to tracer ([ipfs/go-bitswap#531](https://github.com/ipfs/go-bitswap/pull/531))\n - fix: fix race on \"responsive\" check ([ipfs/go-bitswap#528](https://github.com/ipfs/go-bitswap/pull/528))\n - fix: reduce log verbosity\n- github.com/ipfs/go-blockservice (v0.1.7 -> v0.2.1):\n - Version 0.2.1\n - Version 0.2.0 (#87) ([ipfs/go-blockservice#87](https://github.com/ipfs/go-blockservice/pull/87))\n - feat: add context to interfaces (#86) ([ipfs/go-blockservice#86](https://github.com/ipfs/go-blockservice/pull/86))\n - sync: update CI config files (#85) ([ipfs/go-blockservice#85](https://github.com/ipfs/go-blockservice/pull/85))\n - chore: update log ([ipfs/go-blockservice#84](https://github.com/ipfs/go-blockservice/pull/84))\n- github.com/ipfs/go-cid (v0.0.7 -> v0.1.0):\n - amend the CidFromReader slice extension math\n - implement CidFromReader\n - chore: fixups from running go vet, go fmt and staticcheck ([ipfs/go-cid#122](https://github.com/ipfs/go-cid/pull/122))\n - s/characters/bytes\n - Fix inaccurate comment for uvarint\n - coverage: more tests for cid\n - coverage: more tests for varint\n - coverage: more tests for builder\n - fix: make tests run with Go 1.15\n - Add the dagjose multiformat\n- github.com/ipfs/go-datastore (v0.4.6 -> v0.5.1):\n - Release v0.5.1\n - chore: add lots of interface assertions\n - fix: make NullDatastore satisfy the Batching interface again\n - Update version.json (#183) ([ipfs/go-datastore#183](https://github.com/ipfs/go-datastore/pull/183))\n - feat: add context to interfaces (#181) ([ipfs/go-datastore#181](https://github.com/ipfs/go-datastore/pull/181))\n - sync: update CI config files ([ipfs/go-datastore#182](https://github.com/ipfs/go-datastore/pull/182))\n- github.com/ipfs/go-ds-badger (v0.2.7 -> v0.3.0):\n - feat: plumb through contexts (#119) ([ipfs/go-ds-badger#119](https://github.com/ipfs/go-ds-badger/pull/119))\n- github.com/ipfs/go-ds-flatfs (v0.4.5 -> v0.5.1):\n - Update version.json\n - fix: add context to DiskUsage()\n - Version 0.5.0 (#99) ([ipfs/go-ds-flatfs#99](https://github.com/ipfs/go-ds-flatfs/pull/99))\n - feat: add contexts on datastore methods (#98) ([ipfs/go-ds-flatfs#98](https://github.com/ipfs/go-ds-flatfs/pull/98))\n - sync: update CI config files (#97) ([ipfs/go-ds-flatfs#97](https://github.com/ipfs/go-ds-flatfs/pull/97))\n - sync: update CI config files ([ipfs/go-ds-flatfs#96](https://github.com/ipfs/go-ds-flatfs/pull/96))\n - fix staticcheck ([ipfs/go-ds-flatfs#92](https://github.com/ipfs/go-ds-flatfs/pull/92))\n - fix typo in readme.go ([ipfs/go-ds-flatfs#89](https://github.com/ipfs/go-ds-flatfs/pull/89))\n- github.com/ipfs/go-ds-leveldb (v0.4.2 -> v0.5.0):\n - Version 0.5.0 (#58) ([ipfs/go-ds-leveldb#58](https://github.com/ipfs/go-ds-leveldb/pull/58))\n - feat: plumb through contexts (#57) ([ipfs/go-ds-leveldb#57](https://github.com/ipfs/go-ds-leveldb/pull/57))\n - sync: update CI config files (#56) ([ipfs/go-ds-leveldb#56](https://github.com/ipfs/go-ds-leveldb/pull/56))\n - fix closing of datastore in tests ([ipfs/go-ds-leveldb#52](https://github.com/ipfs/go-ds-leveldb/pull/52))\n - fix staticcheck ([ipfs/go-ds-leveldb#49](https://github.com/ipfs/go-ds-leveldb/pull/49))\n - fix typo in function documentation ([ipfs/go-ds-leveldb#46](https://github.com/ipfs/go-ds-leveldb/pull/46))\n- github.com/ipfs/go-ds-measure (v0.1.0 -> v0.2.0):\n - Version 0.2.0 (#39) ([ipfs/go-ds-measure#39](https://github.com/ipfs/go-ds-measure/pull/39))\n - feat: add contexts on datastore methods (#38) ([ipfs/go-ds-measure#38](https://github.com/ipfs/go-ds-measure/pull/38))\n - sync: update CI config files (#37) ([ipfs/go-ds-measure#37](https://github.com/ipfs/go-ds-measure/pull/37))\n- github.com/ipfs/go-fetcher (v1.5.0 -> v1.6.1):\n - Version 1.6.1\n - Version 1.6.0 (#29) ([ipfs/go-fetcher#29](https://github.com/ipfs/go-fetcher/pull/29))\n - feat: plumb through context changes (#28) ([ipfs/go-fetcher#28](https://github.com/ipfs/go-fetcher/pull/28))\n - sync: update CI config files (#27) ([ipfs/go-fetcher#27](https://github.com/ipfs/go-fetcher/pull/27))\n - add a fetcher constructor for the case where we already have a session ([ipfs/go-fetcher#26](https://github.com/ipfs/go-fetcher/pull/26))\n- github.com/ipfs/go-filestore (v0.0.3 -> v0.1.0):\n - feat: plumb through context changes (#56) ([ipfs/go-filestore#56](https://github.com/ipfs/go-filestore/pull/56))\n- github.com/ipfs/go-graphsync (v0.8.0 -> v0.11.0):\n - docs(CHANGELOG): update for v0.11.0 release\n - Merge branch 'release/v0.10.6'\n - update to context datastores (#275) ([ipfs/go-graphsync#275](https://github.com/ipfs/go-graphsync/pull/275))\n - feat!(requestmanager): remove request allocation backpressure (#272) ([ipfs/go-graphsync#272](https://github.com/ipfs/go-graphsync/pull/272))\n - message/pb: stop using gogo/protobuf (#277) ([ipfs/go-graphsync#277](https://github.com/ipfs/go-graphsync/pull/277))\n - mark all test helper funcs via t.Helper (#276) ([ipfs/go-graphsync#276](https://github.com/ipfs/go-graphsync/pull/276))\n - chore(queryexecutor): remove unused RunTraversal\n - chore(responsemanager): remove unused workSignal\n - chore(queryexecutor): fix tests for runtraversal refactor + clean up\n - feat(queryexecutor): merge RunTraversal into QueryExecutor\n - feat(responsemanager): QueryExecutor to separate module - use TaskQueue, add tests\n - Merge branch 'release/v0.10.5'\n - fix(responseassembler): don't hold block data reference in passed on subscribed block link (#268) ([ipfs/go-graphsync#268](https://github.com/ipfs/go-graphsync/pull/268))\n - sync: update CI config files (#266) ([ipfs/go-graphsync#266](https://github.com/ipfs/go-graphsync/pull/266))\n - Check IPLD context cancellation error type instead of string comparison\n - Use `context.CancelFunc` instead of `func()` (#257) ([ipfs/go-graphsync#257](https://github.com/ipfs/go-graphsync/pull/257))\n - fix: bail properly when budget exceeded\n - feat(requestmanager): report inProgressRequestCount on OutgoingRequests event\n - fix(requestmanager): remove failing racy test select block\n - feat(requestmanager): add OutgoingRequeustProcessingListener\n - Merge branch 'release/v0.10.4'\n - fix(allocator): prevent buffer overflow (#248) ([ipfs/go-graphsync#248](https://github.com/ipfs/go-graphsync/pull/248))\n - Merge branch 'release/v0.10.3'\n - Configure message parameters (#247) ([ipfs/go-graphsync#247](https://github.com/ipfs/go-graphsync/pull/247))\n - Stats! (#246) ([ipfs/go-graphsync#246](https://github.com/ipfs/go-graphsync/pull/246))\n - Limit simultaneous incoming requests on a per peer basis (#245) ([ipfs/go-graphsync#245](https://github.com/ipfs/go-graphsync/pull/245))\n - sync: update CI config files (#191) ([ipfs/go-graphsync#191](https://github.com/ipfs/go-graphsync/pull/191))\n - Merge branch 'release/v0.10.2'\n - test(responsemanager): fix flakiness TestCancellationViaCommand (#243) ([ipfs/go-graphsync#243](https://github.com/ipfs/go-graphsync/pull/243))\n - Fix deadlock on notifications (#242) ([ipfs/go-graphsync#242](https://github.com/ipfs/go-graphsync/pull/242))\n - Merge branch 'release/v0.10.1'\n - Free memory on request finish (#240) ([ipfs/go-graphsync#240](https://github.com/ipfs/go-graphsync/pull/240))\n - release: v1.10.0 ([ipfs/go-graphsync#238](https://github.com/ipfs/go-graphsync/pull/238))\n - Add support for IPLD prime's budgets feature in selectors (#235) ([ipfs/go-graphsync#235](https://github.com/ipfs/go-graphsync/pull/235))\n - feat(graphsync): add an index for blocks in the on new block hook (#234) ([ipfs/go-graphsync#234](https://github.com/ipfs/go-graphsync/pull/234))\n - Do not send first blocks extension (#230) ([ipfs/go-graphsync#230](https://github.com/ipfs/go-graphsync/pull/230))\n - Protect Libp2p Connections (#229) ([ipfs/go-graphsync#229](https://github.com/ipfs/go-graphsync/pull/229))\n - test(responsemanager): remove check (#228) ([ipfs/go-graphsync#228](https://github.com/ipfs/go-graphsync/pull/228))\n - feat(graphsync): give missing blocks a named error (#227) ([ipfs/go-graphsync#227](https://github.com/ipfs/go-graphsync/pull/227))\n - Add request limits (#224) ([ipfs/go-graphsync#224](https://github.com/ipfs/go-graphsync/pull/224))\n - Tech Debt Cleanup and Docs Update (#219) ([ipfs/go-graphsync#219](https://github.com/ipfs/go-graphsync/pull/219))\n - Release/v0.9.3 ([ipfs/go-graphsync#218](https://github.com/ipfs/go-graphsync/pull/218))\n - 0.9.2 release ([ipfs/go-graphsync#217](https://github.com/ipfs/go-graphsync/pull/217))\n - fix(requestmanager): remove main thread block on allocation (#216) ([ipfs/go-graphsync#216](https://github.com/ipfs/go-graphsync/pull/216))\n - feat(allocator): add debug logging (#213) ([ipfs/go-graphsync#213](https://github.com/ipfs/go-graphsync/pull/213))\n - fix: spurious warn log (#210) ([ipfs/go-graphsync#210](https://github.com/ipfs/go-graphsync/pull/210))\n - docs(CHANGELOG): update for v0.9.1 release (#212) ([ipfs/go-graphsync#212](https://github.com/ipfs/go-graphsync/pull/212))\n - fix(message): fix dropping of response extensions (#211) ([ipfs/go-graphsync#211](https://github.com/ipfs/go-graphsync/pull/211))\n - docs(CHANGELOG): update change log ([ipfs/go-graphsync#208](https://github.com/ipfs/go-graphsync/pull/208))\n - docs(README): add notice about branch rename\n - fix(graphsync): make sure linkcontext is passed (#207) ([ipfs/go-graphsync#207](https://github.com/ipfs/go-graphsync/pull/207))\n - Merge final v0.6.x commit history, and 0.8.0 changelog (#205) ([ipfs/go-graphsync#205](https://github.com/ipfs/go-graphsync/pull/205))\n - Fix broken link to IPLD selector documentation (#189) ([ipfs/go-graphsync#189](https://github.com/ipfs/go-graphsync/pull/189))\n - fix: check errors before deferring a close (#200) ([ipfs/go-graphsync#200](https://github.com/ipfs/go-graphsync/pull/200))\n - chore: fix checks (#197) ([ipfs/go-graphsync#197](https://github.com/ipfs/go-graphsync/pull/197))\n - Merge the v0.6.x commit history (#190) ([ipfs/go-graphsync#190](https://github.com/ipfs/go-graphsync/pull/190))\n - Ready for universal CI (#187) ([ipfs/go-graphsync#187](https://github.com/ipfs/go-graphsync/pull/187))\n - fix(requestmanager): pass through linksystem (#166) ([ipfs/go-graphsync#166](https://github.com/ipfs/go-graphsync/pull/166))\n - fix missing word in section title (#179) ([ipfs/go-graphsync#179](https://github.com/ipfs/go-graphsync/pull/179))\n- github.com/ipfs/go-ipfs-blockstore (v0.1.6 -> v0.2.1):\n - fix: revert back to go-ipfs-ds-help@v0.1.1 (#92) ([ipfs/go-ipfs-blockstore#92](https://github.com/ipfs/go-ipfs-blockstore/pull/92))\n - feat: add context to interfaces & plumb through datastore contexts (#89) ([ipfs/go-ipfs-blockstore#89](https://github.com/ipfs/go-ipfs-blockstore/pull/89))\n- github.com/ipfs/go-ipfs-config (v0.16.0 -> v0.18.0):\n - Release v0.18.0 (#159) ([ipfs/go-ipfs-config#159](https://github.com/ipfs/go-ipfs-config/pull/159))\n - feat: add Addresses.AppendAnnounce (#135) ([ipfs/go-ipfs-config#135](https://github.com/ipfs/go-ipfs-config/pull/135))\n - feat: omitempty Swarm.EnableRelayHop for circuit v1 migration (#157) ([ipfs/go-ipfs-config#157](https://github.com/ipfs/go-ipfs-config/pull/157))\n - chore: omitempty Experimental.ShardingEnabled (#158) ([ipfs/go-ipfs-config#158](https://github.com/ipfs/go-ipfs-config/pull/158))\n - chore: update comment to match struct\n - Release v0.17.0 (#156) ([ipfs/go-ipfs-config#156](https://github.com/ipfs/go-ipfs-config/pull/156))\n - feat: add a flag to enable the hole punching service (#155) ([ipfs/go-ipfs-config#155](https://github.com/ipfs/go-ipfs-config/pull/155))\n - improve AutoRelay configuration, add config option for static relays ([ipfs/go-ipfs-config#154](https://github.com/ipfs/go-ipfs-config/pull/154))\n - feat: Swarm.RelayService (circuit v2) (#146) ([ipfs/go-ipfs-config#146](https://github.com/ipfs/go-ipfs-config/pull/146))\n - fix: String method on the OptionalString (#153) ([ipfs/go-ipfs-config#153](https://github.com/ipfs/go-ipfs-config/pull/153))\n - sync: update CI config files (#152) ([ipfs/go-ipfs-config#152](https://github.com/ipfs/go-ipfs-config/pull/152))\n - feat: OptionalString type and UnixFSShardingSizeThreshold (#149) ([ipfs/go-ipfs-config#149](https://github.com/ipfs/go-ipfs-config/pull/149))\n - feat: pubsub and ipns pubsub flags (#145) ([ipfs/go-ipfs-config#145](https://github.com/ipfs/go-ipfs-config/pull/145))\n - feat: add an OptionalDuration type (#148) ([ipfs/go-ipfs-config#148](https://github.com/ipfs/go-ipfs-config/pull/148))\n- github.com/ipfs/go-ipfs-exchange-interface (v0.0.1 -> v0.1.0):\n - Update version.json (#20) ([ipfs/go-ipfs-exchange-interface#20](https://github.com/ipfs/go-ipfs-exchange-interface/pull/20))\n - sync: update CI config files (#19) ([ipfs/go-ipfs-exchange-interface#19](https://github.com/ipfs/go-ipfs-exchange-interface/pull/19))\n - feat: add context to interface (#18) ([ipfs/go-ipfs-exchange-interface#18](https://github.com/ipfs/go-ipfs-exchange-interface/pull/18))\n - doc: add a lead maintainer\n- github.com/ipfs/go-ipfs-exchange-offline (v0.0.1 -> v0.1.1):\n - Version 0.1.1\n - Version 0.1.0 (#43) ([ipfs/go-ipfs-exchange-offline#43](https://github.com/ipfs/go-ipfs-exchange-offline/pull/43))\n - feat: plumb through contexts (#42) ([ipfs/go-ipfs-exchange-offline#42](https://github.com/ipfs/go-ipfs-exchange-offline/pull/42))\n - sync: update CI config files (#41) ([ipfs/go-ipfs-exchange-offline#41](https://github.com/ipfs/go-ipfs-exchange-offline/pull/41))\n - fix staticcheck ([ipfs/go-ipfs-exchange-offline#35](https://github.com/ipfs/go-ipfs-exchange-offline/pull/35))\n - chore(gx): remove gx\n- github.com/ipfs/go-ipfs-files (v0.0.8 -> v0.0.9):\n - sync: update CI config files ([ipfs/go-ipfs-files#40](https://github.com/ipfs/go-ipfs-files/pull/40))\n - fix: manually parse the content disposition to preserve directories ([ipfs/go-ipfs-files#42](https://github.com/ipfs/go-ipfs-files/pull/42))\n - fix: round timestamps down by truncating them to seconds ([ipfs/go-ipfs-files#41](https://github.com/ipfs/go-ipfs-files/pull/41))\n - sync: update CI config files ([ipfs/go-ipfs-files#34](https://github.com/ipfs/go-ipfs-files/pull/34))\n - Fix test failure on Windows caused by nil `sys` in mock `FileInfo` ([ipfs/go-ipfs-files#39](https://github.com/ipfs/go-ipfs-files/pull/39))\n - fix staticcheck ([ipfs/go-ipfs-files#35](https://github.com/ipfs/go-ipfs-files/pull/35))\n - fix linters ([ipfs/go-ipfs-files#33](https://github.com/ipfs/go-ipfs-files/pull/33))\n- github.com/ipfs/go-ipfs-pinner (v0.1.2 -> v0.2.1):\n - feat: plumb through context changes (#18) ([ipfs/go-ipfs-pinner#18](https://github.com/ipfs/go-ipfs-pinner/pull/18))\n- github.com/ipfs/go-ipfs-provider (v0.6.1 -> v0.7.1):\n - Fix go vet and staticcheck ([ipfs/go-ipfs-provider#40](https://github.com/ipfs/go-ipfs-provider/pull/40))\n - feat: plumb through datastore contexts (#39) ([ipfs/go-ipfs-provider#39](https://github.com/ipfs/go-ipfs-provider/pull/39))\n- github.com/ipfs/go-ipfs-routing (v0.1.0 -> v0.2.1):\n - Version 0.2.1\n - Bump version to 0.2.0 (#29) ([ipfs/go-ipfs-routing#29](https://github.com/ipfs/go-ipfs-routing/pull/29))\n - feat: plumb through context changes (#28) ([ipfs/go-ipfs-routing#28](https://github.com/ipfs/go-ipfs-routing/pull/28))\n - sync: update CI config files (#27) ([ipfs/go-ipfs-routing#27](https://github.com/ipfs/go-ipfs-routing/pull/27))\n - fix staticcheck ([ipfs/go-ipfs-routing#24](https://github.com/ipfs/go-ipfs-routing/pull/24))\n- github.com/ipfs/go-merkledag (v0.4.0 -> v0.5.1):\n - Version 0.5.1\n - Version 0.5.0 (#79) ([ipfs/go-merkledag#79](https://github.com/ipfs/go-merkledag/pull/79))\n - feat: plumb through contexts (#78) ([ipfs/go-merkledag#78](https://github.com/ipfs/go-merkledag/pull/78))\n - sync: update CI config files (#77) ([ipfs/go-merkledag#77](https://github.com/ipfs/go-merkledag/pull/77))\n - expose session construction to other callers\n - fix RawNode incomplete stats\n- github.com/ipfs/go-mfs (v0.1.2 -> v0.2.1):\n - Version 0.2.1\n - Version 0.2.0 (#96) ([ipfs/go-mfs#96](https://github.com/ipfs/go-mfs/pull/96))\n - support threshold based automatic sharding and unsharding of directories (#88) ([ipfs/go-mfs#88](https://github.com/ipfs/go-mfs/pull/88))\n - sync: update CI config files (#94) ([ipfs/go-mfs#94](https://github.com/ipfs/go-mfs/pull/94))\n - Fix lint errors ([ipfs/go-mfs#90](https://github.com/ipfs/go-mfs/pull/90))\n - remove Makefile ([ipfs/go-mfs#89](https://github.com/ipfs/go-mfs/pull/89))\n- github.com/ipfs/go-namesys (v0.3.1 -> v0.4.0):\n - Release v0.4.0\n - feat: plumb through datastore contexts\n - sync: update CI config files (#23) ([ipfs/go-namesys#23](https://github.com/ipfs/go-namesys/pull/23))\n- github.com/ipfs/go-path (v0.1.2 -> v0.2.1):\n - Version 0.2.1\n - Version 0.2.0 (#48) ([ipfs/go-path#48](https://github.com/ipfs/go-path/pull/48))\n - feat: plumb through context changes (#47) ([ipfs/go-path#47](https://github.com/ipfs/go-path/pull/47))\n - sync: update CI config files (#46) ([ipfs/go-path#46](https://github.com/ipfs/go-path/pull/46))\n - Revert \"feat: plumb through context changes\"\n - feat: plumb through context changes\n- github.com/ipfs/go-peertaskqueue (v0.4.0 -> v0.7.0):\n - feat: optimize checking if a new task is \"better\" ([ipfs/go-peertaskqueue#19](https://github.com/ipfs/go-peertaskqueue/pull/19))\n - Adds customizable prioritization logic for peertracker and peertaskqueue ([ipfs/go-peertaskqueue#17](https://github.com/ipfs/go-peertaskqueue/pull/17))\n - When priority is equal, use FIFO ([ipfs/go-peertaskqueue#16](https://github.com/ipfs/go-peertaskqueue/pull/16))\n- github.com/ipfs/go-unixfs (v0.2.5 -> v0.3.1):\n - Version 0.3.1\n - Version 0.3.0 (#114) ([ipfs/go-unixfs#114](https://github.com/ipfs/go-unixfs/pull/114))\n - feat: plumb through datastore context changes\n - Size-based unsharding (#94) ([ipfs/go-unixfs#94](https://github.com/ipfs/go-unixfs/pull/94))\n - sync: update CI config files (#112) ([ipfs/go-unixfs#112](https://github.com/ipfs/go-unixfs/pull/112))\n - chore(deps): move bitfield to ipfs org ([ipfs/go-unixfs#98](https://github.com/ipfs/go-unixfs/pull/98))\n - fix staticcheck ([ipfs/go-unixfs#95](https://github.com/ipfs/go-unixfs/pull/95))\n - fix(directory): initialize size when computing it ([ipfs/go-unixfs#93](https://github.com/ipfs/go-unixfs/pull/93))\n - fix: always return upgradeable instead of basic dir (#92) ([ipfs/go-unixfs#92](https://github.com/ipfs/go-unixfs/pull/92))\n - feat: switch to HAMT based on size (#91) ([ipfs/go-unixfs#91](https://github.com/ipfs/go-unixfs/pull/91))\n - go fmt\n - fix: add pointer receiver\n - add test\n - feat: add UpgradeableDirectory\n- github.com/ipfs/interface-go-ipfs-core (v0.5.1 -> v0.5.2):\n - fix: check errors by string ([ipfs/interface-go-ipfs-core#76](https://github.com/ipfs/interface-go-ipfs-core/pull/76))\n- github.com/ipfs/tar-utils (v0.0.1 -> v0.0.2):\n - Release v0.0.2 (#8) ([ipfs/tar-utils#8](https://github.com/ipfs/tar-utils/pull/8))\n - sync: update CI config files ([ipfs/tar-utils#7](https://github.com/ipfs/tar-utils/pull/7))\n - sync: update CI config files (#6) ([ipfs/tar-utils#6](https://github.com/ipfs/tar-utils/pull/6))\n - allow .. in file and directory names ([ipfs/tar-utils#5](https://github.com/ipfs/tar-utils/pull/5))\n- github.com/ipld/go-car (v0.3.1 -> v0.3.2):\n - Expose selector traversal options for SelectiveCar ([ipld/go-car#251](https://github.com/ipld/go-car/pull/251))\n - Implement API to allow replacing root CIDs in a CARv1 or CARv2\n - blockstore: OpenReadWrite should not modify if it refuses to resume\n - clarify the relation between StoreIdentityCIDs and SetFullyIndexed\n - Implement options to handle `IDENTITY` CIDs gracefully\n - Combine API options for simplicity and logical coherence\n - Add test script for car verify (#236) ([ipld/go-car#236](https://github.com/ipld/go-car/pull/236))\n - cmd/car: add first testscript tests\n - integrate `car/` cli into `cmd/car` (#233) ([ipld/go-car#233](https://github.com/ipld/go-car/pull/233))\n - Add `car get-dag` command (#232) ([ipld/go-car#232](https://github.com/ipld/go-car/pull/232))\n - Separate CLI to separate module (#231) ([ipld/go-car#231](https://github.com/ipld/go-car/pull/231))\n - add `get block` to car cli (#230) ([ipld/go-car#230](https://github.com/ipld/go-car/pull/230))\n - use file size when loading from v1 car (#229) ([ipld/go-car#229](https://github.com/ipld/go-car/pull/229))\n - add interface describing iteration (#228) ([ipld/go-car#228](https://github.com/ipld/go-car/pull/228))\n - Add `list` and `filter` commands (#227) ([ipld/go-car#227](https://github.com/ipld/go-car/pull/227))\n - Add `car split` command (#226) ([ipld/go-car#226](https://github.com/ipld/go-car/pull/226))\n - Make `MultihashIndexSorted` the default index codec for CARv2\n - Add carve utility for updating the index of a car{v1,v2} file (#219) ([ipld/go-car#219](https://github.com/ipld/go-car/pull/219))\n - Ignore records with `IDENTITY` CID in `IndexSorted`\n - Fix index GetAll infinite loop if function always returns `true`\n - Expose the ability to iterate over records in `MultihasIndexSorted`\n - avoid another alloc per read byte\n - avoid allocating on every byte read\n - Implement new index type that also includes mutltihash code\n - Return `nil` as Index reader when reading indexless CARv2\n - Assert `OpenReader` from file does not panic after closure\n - Document performance caveats of `ExtractV1File` and address comments\n - Implement utility to extract CARv1 from a CARv2\n - v2/blockstore: add ReadWrite.Discard\n - update LICENSE files to point to the new gateway\n - re-add root LICENSE file\n - v2: stop using a symlink for LICENSE.md\n - Update the readme with link to examples\n - update package godocs and root level README for v2\n - blockstore: stop embedding ReadOnly in ReadWrite\n - Implement version agnostic streaming CAR block iterator\n - blockstore: use errors when API contracts are broken\n - add the first read-only benchmarks\n - Implement reader block iterator over CARv1 or CARv2\n - Propagate async `blockstore.AllKeysChan` errors via context\n - Add zero-length sections as EOF option to internal CARv1 reader\n - Improve error handing in tests\n - Allow `ReadOption`s to be set when getting or generating index\n - Use `ioutil.TempFile` to simplify file creation in index example\n - Avoid writing to files in testdata\n - blockstore: implement UseWholeCIDs\n - Merge wip-v2 into master (#178) ([ipld/go-car#178](https://github.com/ipld/go-car/pull/178))\n- github.com/ipld/go-ipld-prime (v0.12.2 -> v0.14.2):\n - dagcbor: coerce undef to null. ([ipld/go-ipld-prime#308](https://github.com/ipld/go-ipld-prime/pull/308))\n - fluent: add toInterface (#304) ([ipld/go-ipld-prime#304](https://github.com/ipld/go-ipld-prime/pull/304))\n - traversal: s/Walk/WalkLocal/\n - traversal: add a primitive walk function.\n - Remove dependency to `go-wish`\n - mark v0.14.0\n - ([ipld/go-ipld-prime#279](https://github.com/ipld/go-ipld-prime/pull/279))\n - Port `traversal` package tests to quicktest\n - Port `codec` package tests to quicktest\n - changelog: backfill.\n - Gracefully handle `TypedNode` with `nil` type of kind `Map`\n - Gracefully print typed nodes with `nil` type\n - Implement handling of `Link` and `[]byte` in `printer` (#294) ([ipld/go-ipld-prime#294](https://github.com/ipld/go-ipld-prime/pull/294))\n - changelog: backfill for the v0.12.x series.\n - readme: introduce a migration guide.\n - Port `fluent` package tests to quicktest\n - Port `datamodel` package tests to quicktest\n - Port `adl` package tests to quicktest\n - Port `node` package tests to quicktest\n - node/bindnode: support links in ProduceGoTypes\n - bump CI to Go 1.16 and 1.17\n - node/bindnode: support links in schema-type verification\n - node/bindnode: export ProduceGoTypes\n - all: fix \"an\" typos after the ipld->datamodel refactor\n - node/bindnode: fix test code after two PR merges\n - add LoadSchema APIs to the root package\n - storage: add 'Has' feature. ([ipld/go-ipld-prime#276](https://github.com/ipld/go-ipld-prime/pull/276))\n - node/bindnode: start verifying schema compatibility\n - linking: add LoadRaw and LoadPlusRaw functions to LinkSystem. ([ipld/go-ipld-prime#267](https://github.com/ipld/go-ipld-prime/pull/267))\n - node/bindnode: add support for lists behind kinded unions\n - node/bindnode: also run TestPrototype with just schemas\n - node/bindnode: polish a few TODO panics away\n - node/bindnode: add support for all scalars behind kinded unions\n - node/bindnode: get closer to passing the Links schema tests\n - start using Rod's schema tests from ipld/ipld\n - fully support parsing, encoding, and decoding the schema-schema\n - node/bindnode: add native support for cid.Cid\n - A more Featureful Approach to Storage APIs ([ipld/go-ipld-prime#265](https://github.com/ipld/go-ipld-prime/pull/265))\n - Add a cidlink.Memory storage option (#266) ([ipld/go-ipld-prime#266](https://github.com/ipld/go-ipld-prime/pull/266))\n - Improve docs for AssignNode; and datamodel.Copy function. ([ipld/go-ipld-prime#264](https://github.com/ipld/go-ipld-prime/pull/264))\n - schemadsl: assign the struct representation.\n - schema,tests,gen/go: more tests, gen union fixes. ([ipld/go-ipld-prime#257](https://github.com/ipld/go-ipld-prime/pull/257))\n - fix: deal with LinkRevisit->LinkVisitOnlyOnce change\n - traversal: the link-visit-only-once behavior should require opt-in, rather than defaulting to on.\n - chore: add LinkRevisit:false traversal test\n - traversal: track seen links, and revisit only if configured to do so.\n - fix: use datamodel.Node selectors\n - Revert encode round-trip to leave unencoded node test intact\n - Add more walk tests, including tests for use of SkipMe\n - Round-trip test nodes through custom codec to ensure stability\n - Don't abort block processing when encountering SkipMe\n - traversal: implement monotonically decrementing budgets. ([ipld/go-ipld-prime#260](https://github.com/ipld/go-ipld-prime/pull/260))\n - Use datamodel.Node for \"Common\" selector variants\n - schema/dmt: first pass at a parser ([ipld/go-ipld-prime#253](https://github.com/ipld/go-ipld-prime/pull/253))\n - drop codectools.\n - drop jst codec. It lives in https://github.com/warpfork/go-jst/ now.\n - drop dagjson2.\n - fix(traversal): properly wrap errors\n - printer: empty maps and lists and structs should stay on one line.\n - schema: turn TypeName into an alias\n - schema/dmt: sync with schema-schema changes, finish Compile\n - schema: add ways to set and access the ImplicitValue for a struct field.\n - schema: accessor for TypeEnum.Representation.\n - schema: finish minimum viable support for describing enum types.\n- github.com/libp2p/go-conn-security-multistream (v0.2.1 -> v0.3.0):\n - use the new SecureTransport and SecureMuxer interfaces (#36) ([libp2p/go-conn-security-multistream#36](https://github.com/libp2p/go-conn-security-multistream/pull/36))\n - fix go vet and staticcheck ([libp2p/go-conn-security-multistream#33](https://github.com/libp2p/go-conn-security-multistream/pull/33))\n- github.com/libp2p/go-libp2p (v0.15.0 -> v0.16.0):\n - release v0.16.0 ([libp2p/go-libp2p#1246](https://github.com/libp2p/go-libp2p/pull/1246))\n - allow the ping protocol on transient connections ([libp2p/go-libp2p#1244](https://github.com/libp2p/go-libp2p/pull/1244))\n - make the Type field required in the HolePunch protobuf ([libp2p/go-libp2p#1241](https://github.com/libp2p/go-libp2p/pull/1241))\n - reject hole punching attempts when we don't have any public addresses ([libp2p/go-libp2p#1214](https://github.com/libp2p/go-libp2p/pull/1214))\n - refactor the AutoRelay code ([libp2p/go-libp2p#1240](https://github.com/libp2p/go-libp2p/pull/1240))\n - remove dead API link in README ([libp2p/go-libp2p#1233](https://github.com/libp2p/go-libp2p/pull/1233))\n - pass static relays to EnableAutoRelay, deprecate libp2p.StaticRelays and libp2p.DefaultStaticRelays ([libp2p/go-libp2p#1239](https://github.com/libp2p/go-libp2p/pull/1239))\n - feat: plumb through peerstore context changes (#1237) ([libp2p/go-libp2p#1237](https://github.com/libp2p/go-libp2p/pull/1237))\n - emit the EvtPeerConnectednessChanged event ([libp2p/go-libp2p#1230](https://github.com/libp2p/go-libp2p/pull/1230))\n - update go-libp2p-swarm to v0.7.0 ([libp2p/go-libp2p#1226](https://github.com/libp2p/go-libp2p/pull/1226))\n - sync: update CI config files (#1225) ([libp2p/go-libp2p#1225](https://github.com/libp2p/go-libp2p/pull/1225))\n - simplify circuitv2 package structure ([libp2p/go-libp2p#1224](https://github.com/libp2p/go-libp2p/pull/1224))\n - use a random string for the mDNS peer-name ([libp2p/go-libp2p#1222](https://github.com/libp2p/go-libp2p/pull/1222))\n - remove {Un}RegisterNotifee functions from mDNS service ([libp2p/go-libp2p#1220](https://github.com/libp2p/go-libp2p/pull/1220))\n - fix structured logging in holepunch coordination ([libp2p/go-libp2p#1213](https://github.com/libp2p/go-libp2p/pull/1213))\n - fix flaky TestStBackpressureStreamWrite test ([libp2p/go-libp2p#1212](https://github.com/libp2p/go-libp2p/pull/1212))\n - properly close hosts in mDNS tests ([libp2p/go-libp2p#1216](https://github.com/libp2p/go-libp2p/pull/1216))\n - close the ObserverAddrManager when the ID service is closed ([libp2p/go-libp2p#1218](https://github.com/libp2p/go-libp2p/pull/1218))\n - make it possible to pass options to a transport constructor ([libp2p/go-libp2p#1205](https://github.com/libp2p/go-libp2p/pull/1205))\n - remove goprocess from the NATManager ([libp2p/go-libp2p#1193](https://github.com/libp2p/go-libp2p/pull/1193))\n - add an option to start the relay v2 ([libp2p/go-libp2p#1197](https://github.com/libp2p/go-libp2p/pull/1197))\n - fix flaky TestFastDisconnect identify test ([libp2p/go-libp2p#1200](https://github.com/libp2p/go-libp2p/pull/1200))\n - chore: update go-tcp-transport to v0.3.0 ([libp2p/go-libp2p#1203](https://github.com/libp2p/go-libp2p/pull/1203))\n - fix: skip variadic params in constructors ([libp2p/go-libp2p#1204](https://github.com/libp2p/go-libp2p/pull/1204))\n - fix flaky BasicHost tests ([libp2p/go-libp2p#1202](https://github.com/libp2p/go-libp2p/pull/1202))\n - remove dependency on github.com/ipfs/go-detect-race ([libp2p/go-libp2p#1201](https://github.com/libp2p/go-libp2p/pull/1201))\n - fix flaky TestEndToEndSimConnect holepunching test ([libp2p/go-libp2p#1191](https://github.com/libp2p/go-libp2p/pull/1191))\n - autorelay support for circuitv2 relays (#1198) ([libp2p/go-libp2p#1198](https://github.com/libp2p/go-libp2p/pull/1198))\n - reject circuitv2 reservations with nonsensical expiration times ([libp2p/go-libp2p#1199](https://github.com/libp2p/go-libp2p/pull/1199))\n - Tag relay hops in relay implementations ([libp2p/go-libp2p#1188](https://github.com/libp2p/go-libp2p/pull/1188))\n - Add standalone implementation of v1 Relay (#1186) ([libp2p/go-libp2p#1186](https://github.com/libp2p/go-libp2p/pull/1186))\n - remove the context from the libp2p and the Host constructor ([libp2p/go-libp2p#1190](https://github.com/libp2p/go-libp2p/pull/1190))\n - don't use a context to shut down the circuitv2 ([libp2p/go-libp2p#1185](https://github.com/libp2p/go-libp2p/pull/1185))\n - fix: remove v1 go-log dep ([libp2p/go-libp2p#1189](https://github.com/libp2p/go-libp2p/pull/1189))\n - don't use the context to shut down the relay ([libp2p/go-libp2p#1184](https://github.com/libp2p/go-libp2p/pull/1184))\n - Use circuitv2 code (#1183) ([libp2p/go-libp2p#1183](https://github.com/libp2p/go-libp2p/pull/1183))\n - clean up badges in README ([libp2p/go-libp2p#1179](https://github.com/libp2p/go-libp2p/pull/1179))\n - remove recommendation about Go module proxy from README ([libp2p/go-libp2p#1180](https://github.com/libp2p/go-libp2p/pull/1180))\n - merge branch 'hole-punching'\n - don't use a context for closing the ObservedAddrManager ([libp2p/go-libp2p#1175](https://github.com/libp2p/go-libp2p/pull/1175))\n - move the circuit v2 code here ([libp2p/go-libp2p#1174](https://github.com/libp2p/go-libp2p/pull/1174))\n - make QUIC a default transport ([libp2p/go-libp2p#1128](https://github.com/libp2p/go-libp2p/pull/1128))\n - stop using jbenet/go-cienv ([libp2p/go-libp2p#1176](https://github.com/libp2p/go-libp2p/pull/1176))\n - fix flaky TestObsAddrSet test ([libp2p/go-libp2p#1172](https://github.com/libp2p/go-libp2p/pull/1172))\n - clean up messy defer logic in IDService.sendIdentifyResp ([libp2p/go-libp2p#1169](https://github.com/libp2p/go-libp2p/pull/1169))\n - remove secio from README, add noise ([libp2p/go-libp2p#1165](https://github.com/libp2p/go-libp2p/pull/1165))\n- github.com/libp2p/go-libp2p-asn-util (v0.0.0-20200825225859-85005c6cf052 -> v0.1.0):\n - Update from upstream and make regeneration easier (#17) ([libp2p/go-libp2p-asn-util#17](https://github.com/libp2p/go-libp2p-asn-util/pull/17))\n - add license file so it can be found by go-licenses ([libp2p/go-libp2p-asn-util#10](https://github.com/libp2p/go-libp2p-asn-util/pull/10))\n - refactor: rename ASN table files ([libp2p/go-libp2p-asn-util#9](https://github.com/libp2p/go-libp2p-asn-util/pull/9))\n - Library for IP -> ASN mapping ([libp2p/go-libp2p-asn-util#1](https://github.com/libp2p/go-libp2p-asn-util/pull/1))\n- github.com/libp2p/go-libp2p-autonat (v0.4.2 -> v0.6.0):\n - Version 0.6.0 (#112) ([libp2p/go-libp2p-autonat#112](https://github.com/libp2p/go-libp2p-autonat/pull/112))\n - feat: plumb through contexts from peerstore (#111) ([libp2p/go-libp2p-autonat#111](https://github.com/libp2p/go-libp2p-autonat/pull/111))\n - sync: update CI config files (#110) ([libp2p/go-libp2p-autonat#110](https://github.com/libp2p/go-libp2p-autonat/pull/110))\n - remove context from constructor, implement a proper Close method ([libp2p/go-libp2p-autonat#109](https://github.com/libp2p/go-libp2p-autonat/pull/109))\n - fix stream deadlines ([libp2p/go-libp2p-autonat#107](https://github.com/libp2p/go-libp2p-autonat/pull/107))\n - disable failing integration test ([libp2p/go-libp2p-autonat#108](https://github.com/libp2p/go-libp2p-autonat/pull/108))\n - fix staticcheck ([libp2p/go-libp2p-autonat#103](https://github.com/libp2p/go-libp2p-autonat/pull/103))\n- github.com/libp2p/go-libp2p-core (v0.9.0 -> v0.11.0):\n - release v0.11.0 (#217) ([libp2p/go-libp2p-core#217](https://github.com/libp2p/go-libp2p-core/pull/217))\n - remove the ConnHandler (#214) ([libp2p/go-libp2p-core#214](https://github.com/libp2p/go-libp2p-core/pull/214))\n - sync: update CI config files (#216) ([libp2p/go-libp2p-core#216](https://github.com/libp2p/go-libp2p-core/pull/216))\n - remove the Process from the Network interface (#212) ([libp2p/go-libp2p-core#212](https://github.com/libp2p/go-libp2p-core/pull/212))\n - pass the peer ID to SecureInbound in the SecureTransport and SecureMuxer (#211) ([libp2p/go-libp2p-core#211](https://github.com/libp2p/go-libp2p-core/pull/211))\n - save the role (client, server) in the simultaneous connect context (#210) ([libp2p/go-libp2p-core#210](https://github.com/libp2p/go-libp2p-core/pull/210))\n - sync: update CI config files (#209) ([libp2p/go-libp2p-core#209](https://github.com/libp2p/go-libp2p-core/pull/209))\n- github.com/libp2p/go-libp2p-discovery (v0.5.1 -> v0.6.0):\n - feat: plumb peerstore contexts changes through (#75) ([libp2p/go-libp2p-discovery#75](https://github.com/libp2p/go-libp2p-discovery/pull/75))\n - remove deprecated types ([libp2p/go-libp2p-discovery#73](https://github.com/libp2p/go-libp2p-discovery/pull/73))\n- github.com/libp2p/go-libp2p-kad-dht (v0.13.1 -> v0.15.0):\n - Bump version to 0.15.0 (#755) ([libp2p/go-libp2p-kad-dht#755](https://github.com/libp2p/go-libp2p-kad-dht/pull/755))\n - sync: update CI config files (#754) ([libp2p/go-libp2p-kad-dht#754](https://github.com/libp2p/go-libp2p-kad-dht/pull/754))\n - feat: plumb through datastore contexts (#753) ([libp2p/go-libp2p-kad-dht#753](https://github.com/libp2p/go-libp2p-kad-dht/pull/753))\n - custom ProviderManager that brokers AddrInfos (#751) ([libp2p/go-libp2p-kad-dht#751](https://github.com/libp2p/go-libp2p-kad-dht/pull/751))\n - feat: make compatible with go-libp2p 0.15 ([libp2p/go-libp2p-kad-dht#747](https://github.com/libp2p/go-libp2p-kad-dht/pull/747))\n - sync: update CI config files ([libp2p/go-libp2p-kad-dht#743](https://github.com/libp2p/go-libp2p-kad-dht/pull/743))\n - Disallow GetPublicKey when DisableValues is passed ([libp2p/go-libp2p-kad-dht#604](https://github.com/libp2p/go-libp2p-kad-dht/pull/604))\n- github.com/libp2p/go-libp2p-nat (v0.0.6 -> v0.1.0):\n - remove Codecov config file ([libp2p/go-libp2p-nat#39](https://github.com/libp2p/go-libp2p-nat/pull/39))\n - stop using goprocess for shutdown ([libp2p/go-libp2p-nat#38](https://github.com/libp2p/go-libp2p-nat/pull/38))\n - chore: update go-log ([libp2p/go-libp2p-nat#37](https://github.com/libp2p/go-libp2p-nat/pull/37))\n - remove unused field permanent from mapping ([libp2p/go-libp2p-nat#33](https://github.com/libp2p/go-libp2p-nat/pull/33))\n- github.com/libp2p/go-libp2p-noise (v0.2.2 -> v0.3.0):\n - add the peer ID to SecureInbound ([libp2p/go-libp2p-noise#104](https://github.com/libp2p/go-libp2p-noise/pull/104))\n - update go-libp2p-core, remove integration test ([libp2p/go-libp2p-noise#102](https://github.com/libp2p/go-libp2p-noise/pull/102))\n- github.com/libp2p/go-libp2p-peerstore (v0.2.8 -> v0.4.0):\n - Update version.json (#178) ([libp2p/go-libp2p-peerstore#178](https://github.com/libp2p/go-libp2p-peerstore/pull/178))\n - limit the number of protocols we store per peer ([libp2p/go-libp2p-peerstore#172](https://github.com/libp2p/go-libp2p-peerstore/pull/172))\n - sync: update CI config files (#177) ([libp2p/go-libp2p-peerstore#177](https://github.com/libp2p/go-libp2p-peerstore/pull/177))\n - feat: plumb through datastore contexts (#176) ([libp2p/go-libp2p-peerstore#176](https://github.com/libp2p/go-libp2p-peerstore/pull/176))\n - remove leftover peerstore implementation in the root package ([libp2p/go-libp2p-peerstore#173](https://github.com/libp2p/go-libp2p-peerstore/pull/173))\n - fix: replace deprecated call ([libp2p/go-libp2p-peerstore#168](https://github.com/libp2p/go-libp2p-peerstore/pull/168))\n - feat: remove queue ([libp2p/go-libp2p-peerstore#166](https://github.com/libp2p/go-libp2p-peerstore/pull/166))\n - remove deprecated types ([libp2p/go-libp2p-peerstore#165](https://github.com/libp2p/go-libp2p-peerstore/pull/165))\n- github.com/libp2p/go-libp2p-pubsub (v0.5.4 -> v0.6.0):\n - feat: plumb through context changes (#459) ([libp2p/go-libp2p-pubsub#459](https://github.com/libp2p/go-libp2p-pubsub/pull/459))\n - support MinTopicSize without a discovery mechanism\n - clear peerPromises map when fulfilling a promise\n - README: remove obsolete notice, fix example code for tracing.\n - remove peer filter check from subscriptions (#453) ([libp2p/go-libp2p-pubsub#453](https://github.com/libp2p/go-libp2p-pubsub/pull/453))\n - Create peer filter option\n- github.com/libp2p/go-libp2p-pubsub-router (v0.4.0 -> v0.5.0):\n - Version 0.5.0\n - feat: plumb through datastore contexts\n - sync: update CI config files (#86) ([libp2p/go-libp2p-pubsub-router#86](https://github.com/libp2p/go-libp2p-pubsub-router/pull/86))\n - Remove arbitrary sleeps from tests ([libp2p/go-libp2p-pubsub-router#87](https://github.com/libp2p/go-libp2p-pubsub-router/pull/87))\n - cleanup: fix staticcheck failures ([libp2p/go-libp2p-pubsub-router#84](https://github.com/libp2p/go-libp2p-pubsub-router/pull/84))\n - Add WithDatastore option. ([libp2p/go-libp2p-pubsub-router#82](https://github.com/libp2p/go-libp2p-pubsub-router/pull/82))\n- github.com/libp2p/go-libp2p-quic-transport (v0.12.0 -> v0.15.0):\n - release v0.15.0 (#241) ([libp2p/go-libp2p-quic-transport#241](https://github.com/libp2p/go-libp2p-quic-transport/pull/241))\n - reuse the same router until we change listeners ([libp2p/go-libp2p-quic-transport#240](https://github.com/libp2p/go-libp2p-quic-transport/pull/240))\n - release v0.14.0 ([libp2p/go-libp2p-quic-transport#237](https://github.com/libp2p/go-libp2p-quic-transport/pull/237))\n - fix error assertions in the tracer ([libp2p/go-libp2p-quic-transport#234](https://github.com/libp2p/go-libp2p-quic-transport/pull/234))\n - sync: update CI config files (#235) ([libp2p/go-libp2p-quic-transport#235](https://github.com/libp2p/go-libp2p-quic-transport/pull/235))\n - read the client option from the simultaneous connect context ([libp2p/go-libp2p-quic-transport#230](https://github.com/libp2p/go-libp2p-quic-transport/pull/230))\n- github.com/libp2p/go-libp2p-swarm (v0.5.3 -> v0.8.0):\n - Version 0.8.0 (#292) ([libp2p/go-libp2p-swarm#292](https://github.com/libp2p/go-libp2p-swarm/pull/292))\n - feat: plumb contexts through from peerstore (#290) ([libp2p/go-libp2p-swarm#290](https://github.com/libp2p/go-libp2p-swarm/pull/290))\n - release v0.7.0 ([libp2p/go-libp2p-swarm#289](https://github.com/libp2p/go-libp2p-swarm/pull/289))\n - update go-tcp-transport to v0.4.0 ([libp2p/go-libp2p-swarm#287](https://github.com/libp2p/go-libp2p-swarm/pull/287))\n - remove the ConnHandler ([libp2p/go-libp2p-swarm#286](https://github.com/libp2p/go-libp2p-swarm/pull/286))\n - sync: update CI config files (#288) ([libp2p/go-libp2p-swarm#288](https://github.com/libp2p/go-libp2p-swarm/pull/288))\n - remove a lot of incorrect statements from the README ([libp2p/go-libp2p-swarm#284](https://github.com/libp2p/go-libp2p-swarm/pull/284))\n - unexport the DialSync ([libp2p/go-libp2p-swarm#281](https://github.com/libp2p/go-libp2p-swarm/pull/281))\n - add an error return value to the constructor ([libp2p/go-libp2p-swarm#280](https://github.com/libp2p/go-libp2p-swarm/pull/280))\n - use functional options to configure the swarm ([libp2p/go-libp2p-swarm#279](https://github.com/libp2p/go-libp2p-swarm/pull/279))\n - stop using goprocess to control teardown ([libp2p/go-libp2p-swarm#278](https://github.com/libp2p/go-libp2p-swarm/pull/278))\n - read and use the direction from the simultaneous connect context ([libp2p/go-libp2p-swarm#277](https://github.com/libp2p/go-libp2p-swarm/pull/277))\n - simplify the DialSync code ([libp2p/go-libp2p-swarm#272](https://github.com/libp2p/go-libp2p-swarm/pull/272))\n - remove redundant self-dialing check, simplify starting of dialWorkerLoop ([libp2p/go-libp2p-swarm#273](https://github.com/libp2p/go-libp2p-swarm/pull/273))\n - add a test case for the testing package ([libp2p/go-libp2p-swarm#276](https://github.com/libp2p/go-libp2p-swarm/pull/276))\n - simplify limiter by removing the injected isFdConsumingFnc ([libp2p/go-libp2p-swarm#274](https://github.com/libp2p/go-libp2p-swarm/pull/274))\n - update badges ([libp2p/go-libp2p-swarm#271](https://github.com/libp2p/go-libp2p-swarm/pull/271))\n - remove unused context in Swarm.dialWorkerLoop ([libp2p/go-libp2p-swarm#268](https://github.com/libp2p/go-libp2p-swarm/pull/268))\n - remove Codecov config ([libp2p/go-libp2p-swarm#270](https://github.com/libp2p/go-libp2p-swarm/pull/270))\n - fix race condition in TestFailFirst ([libp2p/go-libp2p-swarm#269](https://github.com/libp2p/go-libp2p-swarm/pull/269))\n- github.com/libp2p/go-libp2p-testing (v0.4.2 -> v0.5.0):\n - chore: update go-libp2p-core to v0.10.0 ([libp2p/go-libp2p-testing#38](https://github.com/libp2p/go-libp2p-testing/pull/38))\n - sync: update CI config files (#37) ([libp2p/go-libp2p-testing#37](https://github.com/libp2p/go-libp2p-testing/pull/37))\n- github.com/libp2p/go-libp2p-tls (v0.2.0 -> v0.3.1):\n - release v0.3.1 ([libp2p/go-libp2p-tls#101](https://github.com/libp2p/go-libp2p-tls/pull/101))\n - set a random certificate subject ([libp2p/go-libp2p-tls#100](https://github.com/libp2p/go-libp2p-tls/pull/100))\n - sync: update CI config files (#96) ([libp2p/go-libp2p-tls#96](https://github.com/libp2p/go-libp2p-tls/pull/96))\n - add the peer ID to SecureInbound ([libp2p/go-libp2p-tls#94](https://github.com/libp2p/go-libp2p-tls/pull/94))\n - sync: update CI config files ([libp2p/go-libp2p-tls#91](https://github.com/libp2p/go-libp2p-tls/pull/91))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.4.6 -> v0.5.0):\n - increase timeout in TestConnectionsClosedIfNotAccepted on CI ([libp2p/go-libp2p-transport-upgrader#85](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/85))\n - add the peer ID to SecureInbound ([libp2p/go-libp2p-transport-upgrader#83](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/83))\n- github.com/libp2p/go-msgio (v0.0.6 -> v0.1.0):\n - sync: update CI config files (#27) ([libp2p/go-msgio#27](https://github.com/libp2p/go-msgio/pull/27))\n - remove .gxignore file ([libp2p/go-msgio#24](https://github.com/libp2p/go-msgio/pull/24))\n - remove Codecov config ([libp2p/go-msgio#26](https://github.com/libp2p/go-msgio/pull/26))\n - remove \"Chan\" type ([libp2p/go-msgio#23](https://github.com/libp2p/go-msgio/pull/23))\n- github.com/libp2p/go-nat (v0.0.5 -> v0.1.0):\n - pass a context to DiscoverGateway ([libp2p/go-nat#23](https://github.com/libp2p/go-nat/pull/23))\n- github.com/libp2p/go-reuseport (v0.0.2 -> v0.1.0):\n - stop using github.com/pkg/errors ([libp2p/go-reuseport#85](https://github.com/libp2p/go-reuseport/pull/85))\n - sync: update CI config files (#84) ([libp2p/go-reuseport#84](https://github.com/libp2p/go-reuseport/pull/84))\n- github.com/libp2p/go-reuseport-transport (v0.0.5 -> v0.1.0):\n - remove Codecov config file ([libp2p/go-reuseport-transport#36](https://github.com/libp2p/go-reuseport-transport/pull/36))\n - chore: update go-log to v2 ([libp2p/go-reuseport-transport#35](https://github.com/libp2p/go-reuseport-transport/pull/35))\n - sync: update CI config files ([libp2p/go-reuseport-transport#31](https://github.com/libp2p/go-reuseport-transport/pull/31))\n- github.com/libp2p/go-tcp-transport (v0.2.8 -> v0.4.0):\n - release v0.4.0 ([libp2p/go-tcp-transport#108](https://github.com/libp2p/go-tcp-transport/pull/108))\n - sync: update CI config files (#107) ([libp2p/go-tcp-transport#107](https://github.com/libp2p/go-tcp-transport/pull/107))\n - remove the deprecated IPFS_REUSEPORT command line flag ([libp2p/go-tcp-transport#104](https://github.com/libp2p/go-tcp-transport/pull/104))\n - add options to the constructor ([libp2p/go-tcp-transport#99](https://github.com/libp2p/go-tcp-transport/pull/99))\n - remove the context from the libp2p constructor in README ([libp2p/go-tcp-transport#101](https://github.com/libp2p/go-tcp-transport/pull/101))\n - don't use libp2p.ChainOption in README ([libp2p/go-tcp-transport#102](https://github.com/libp2p/go-tcp-transport/pull/102))\n - remove incorrect statement about dns addresses in README ([libp2p/go-tcp-transport#100](https://github.com/libp2p/go-tcp-transport/pull/100))\n - use the assigned role when upgrading a sim open connection ([libp2p/go-tcp-transport#95](https://github.com/libp2p/go-tcp-transport/pull/95))\n - chore: update go-log to v2 ([libp2p/go-tcp-transport#97](https://github.com/libp2p/go-tcp-transport/pull/97))\n - simplify dial timeout context ([libp2p/go-tcp-transport#94](https://github.com/libp2p/go-tcp-transport/pull/94))\n- github.com/libp2p/go-yamux/v2 (v2.2.0 -> v2.3.0):\n - limit the number of concurrent incoming streams ([libp2p/go-yamux#66](https://github.com/libp2p/go-yamux/pull/66))\n - drastically reduce allocations in ring buffer implementation (#64) ([libp2p/go-yamux#64](https://github.com/libp2p/go-yamux/pull/64))\n - sync: update CI config files (#63) ([libp2p/go-yamux#63](https://github.com/libp2p/go-yamux/pull/63))\n - remove call to asyncNotify in Stream.Read\n- github.com/libp2p/zeroconf/v2 (v2.0.0 -> v2.1.1):\n - fix flaky TTL test ([libp2p/zeroconf#18](https://github.com/libp2p/zeroconf/pull/18))\n - implement a clean shutdown of the probe method ([libp2p/zeroconf#16](https://github.com/libp2p/zeroconf/pull/16))\n - remove dependency on the backoff library ([libp2p/zeroconf#17](https://github.com/libp2p/zeroconf/pull/17))\n - Don't stop browsing after ~15min ([libp2p/zeroconf#13](https://github.com/libp2p/zeroconf/pull/13))\n - fix delays when sending initial probe packets ([libp2p/zeroconf#14](https://github.com/libp2p/zeroconf/pull/14))\n - improve starting of mDNS service in tests, stop using pkg/errors ([libp2p/zeroconf#15](https://github.com/libp2p/zeroconf/pull/15))\n - update import path to include v2 in README ([libp2p/zeroconf#11](https://github.com/libp2p/zeroconf/pull/11))\n- github.com/lucas-clemente/quic-go (v0.23.0 -> v0.24.0):\n - don't unlock the receive stream mutex for copying from STREAM frames ([lucas-clemente/quic-go#3290](https://github.com/lucas-clemente/quic-go/pull/3290))\n - List projects using quic-go ([lucas-clemente/quic-go#3266](https://github.com/lucas-clemente/quic-go/pull/3266))\n - disable Path MTU Discovery on Windows ([lucas-clemente/quic-go#3276](https://github.com/lucas-clemente/quic-go/pull/3276))\n - enter the regular run loop if no undecryptable packet was processed ([lucas-clemente/quic-go#3268](https://github.com/lucas-clemente/quic-go/pull/3268))\n - Allow use of custom port value in Alt-Svc header. ([lucas-clemente/quic-go#3272](https://github.com/lucas-clemente/quic-go/pull/3272))\n - disable the goconst linter ([lucas-clemente/quic-go#3286](https://github.com/lucas-clemente/quic-go/pull/3286))\n - use x/net/ipv{4,6} to construct oob info when writing packets (#3278) ([lucas-clemente/quic-go#3278](https://github.com/lucas-clemente/quic-go/pull/3278))\n - run gofmt to add the new go:build tags ([lucas-clemente/quic-go#3277](https://github.com/lucas-clemente/quic-go/pull/3277))\n - fix log string in client example ([lucas-clemente/quic-go#3264](https://github.com/lucas-clemente/quic-go/pull/3264))\n- github.com/multiformats/go-multiaddr (v0.4.0 -> v0.4.1):\n - add the plaintextv2 protocol ([multiformats/go-multiaddr#165](https://github.com/multiformats/go-multiaddr/pull/165))\n- github.com/multiformats/go-multihash (v0.0.15 -> v0.1.0):\n - bump version to v0.1.0 ([multiformats/go-multihash#151](https://github.com/multiformats/go-multihash/pull/151))\n - add version.json per tooling convention.\n - murmur3 support (#150) ([multiformats/go-multihash#150](https://github.com/multiformats/go-multihash/pull/150))\n - Add variations of sha2 ([multiformats/go-multihash#149](https://github.com/multiformats/go-multihash/pull/149))\n - don't use pointers for Multihash.String\n - Add blake3 hash and sharness tests ([multiformats/go-multihash#147](https://github.com/multiformats/go-multihash/pull/147))\n - remove Makefile ([multiformats/go-multihash#142](https://github.com/multiformats/go-multihash/pull/142))\n - fix staticcheck ([multiformats/go-multihash#141](https://github.com/multiformats/go-multihash/pull/141))\n - New SumStream function reads from io.Reader ([multiformats/go-multihash#138](https://github.com/multiformats/go-multihash/pull/138))\n- github.com/warpfork/go-testmark (v0.3.0 -> v0.9.0):\n - testexec: will now always set up tmpdirs.\n - testexec: fix typo in error message.\n - testexec: subtest (\"then-*\") feature ([warpfork/go-testmark#7](https://github.com/warpfork/go-testmark/pull/7))\n - testexec: quote error from child; attribution better via more t.Helper.\n - Improve documentation of format.\n - Rename Hunk.BlockTag -> InfoString.\n - testexec: will now create tmpdirs and files for you if you have an 'fs' entry tree.\n - testexec: getting exit codes correctly. ([warpfork/go-testmark#6](https://github.com/warpfork/go-testmark/pull/6))\n - fix parsing CRLF files, part 3 ([warpfork/go-testmark#5](https://github.com/warpfork/go-testmark/pull/5))\n - fix parsing CRLF files, part 2 ([warpfork/go-testmark#4](https://github.com/warpfork/go-testmark/pull/4))\n - testexec: support both simple sequence and script mode.\n - Proper tests for read function.\n - avoid creeping extra linebreaks at the end of a patched document.\n - refrain from making double linebreaks when patching with content that ends in a linebreak.\n - Merge branch 'testexec'\n - add support for parsing CRLF line endings ([warpfork/go-testmark#3](https://github.com/warpfork/go-testmark/pull/3))\n - link to patch example code\n - More readme; and, parsing recommendations document.\n - Further improve readme.\n\n### ❤️ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Will | 13 | +73226/-130481 | 43 |\n| Masih H. Derkani | 99 | +10549/-5799 | 489 |\n| hannahhoward | 43 | +5515/-3293 | 233 |\n| Daniel Martí | 60 | +5312/-2883 | 208 |\n| Marten Seemann | 175 | +4839/-3254 | 396 |\n| Eric Myhre | 73 | +3924/-3328 | 175 |\n| Jessica Schilling | 52 | +2709/-2386 | 75 |\n| Rod Vagg | 30 | +2719/-1703 | 79 |\n| vyzo | 10 | +3516/-177 | 87 |\n| Gus Eggert | 64 | +1677/-1416 | 147 |\n| Adin Schmahmann | 23 | +1708/-381 | 95 |\n| Lucas Molas | 14 | +1557/-365 | 48 |\n| Will Scott | 7 | +1846/-15 | 34 |\n| Steven Allen | 32 | +537/-897 | 56 |\n| Cory Schwartz | 3 | +614/-109 | 12 |\n| rht | 3 | +576/-4 | 7 |\n| Simon Zhu | 9 | +352/-51 | 16 |\n| Petar Maymounkov | 7 | +173/-167 | 23 |\n| RubenKelevra | 1 | +107/-188 | 1 |\n| jwh | 2 | +212/-80 | 7 |\n| longfeiW | 1 | +4/-249 | 10 |\n| guseggert | 5 | +230/-21 | 11 |\n| Kevin Neaton | 8 | +137/-80 | 13 |\n| Takashi Matsuda | 1 | +199/-0 | 5 |\n| Andrey Kostakov | 1 | +107/-49 | 2 |\n| Jesse Bouwman | 1 | +151/-0 | 7 |\n| web3-bot | 39 | +136/-3 | 52 |\n| Marcin Rataj | 16 | +62/-57 | 25 |\n| Marco Munizaga | 1 | +118/-0 | 2 |\n| Aaron Riekenberg | 4 | +64/-52 | 6 |\n| Ian Davis | 4 | +81/-32 | 7 |\n| Jorropo | 2 | +79/-19 | 6 |\n| Mohsin Zaidi | 1 | +89/-1 | 20 |\n| Andey Robins | 1 | +70/-3 | 3 |\n| gammazero | 3 | +40/-25 | 4 |\n| Steve Loeppky | 2 | +26/-27 | 3 |\n| Dimitris Apostolou | 1 | +25/-25 | 15 |\n| Sudarshan Reddy | 1 | +9/-40 | 1 |\n| Richard Littauer | 2 | +42/-1 | 3 |\n| pymq | 1 | +32/-8 | 2 |\n| Dirk McCormick | 2 | +23/-1 | 2 |\n| Nicholas Bollweg | 1 | +21/-0 | 1 |\n| anorth | 1 | +14/-6 | 2 |\n| Jack Loughran | 1 | +16/-0 | 2 |\n| whyrusleeping | 2 | +11/-2 | 2 |\n| bt90 | 1 | +13/-0 | 1 |\n| Yi Cao | 1 | +10/-0 | 1 |\n| Max | 1 | +7/-3 | 1 |\n| Juan Batiz-Benet | 2 | +8/-2 | 2 |\n| Keenan Nemetz | 1 | +8/-0 | 1 |\n| muXxer | 1 | +3/-3 | 1 |\n| galargh | 2 | +3/-3 | 3 |\n| Didrik Nordström | 1 | +2/-4 | 1 |\n| Ben Lubar | 1 | +3/-3 | 1 |\n| arjunraghurama | 1 | +5/-0 | 1 |\n| Whyrusleeping | 1 | +3/-2 | 1 |\n| TUSF | 1 | +3/-2 | 3 |\n| mathew-cf | 1 | +3/-1 | 2 |\n| Stephen Whitmore | 1 | +2/-2 | 1 |\n| Song Zhu | 1 | +2/-2 | 1 |\n| Michael Muré | 1 | +4/-0 | 1 |\n| Alex Good | 1 | +4/-0 | 2 |\n| aarshkshah1992 | 1 | +2/-1 | 1 |\n| susarlanikhilesh | 1 | +1/-1 | 1 |\n| falstack | 1 | +1/-1 | 1 |\n| Michael Vorburger ⛑️ | 1 | +1/-1 | 1 |\n| Ismail Khoffi | 1 | +1/-1 | 1 |\n| George Xie | 1 | +1/-1 | 1 |\n| Bryan Stenson | 1 | +1/-1 | 1 |\n| Lars Gierth | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.12.md", "content": "# go-ipfs changelog v0.12\n\n## v0.12.2 2022-04-08\n\nThis patch release fixes a security issue wherein traversing some malformed DAGs can cause the node to panic.\n\nSee also the security advisory: https://github.com/ipfs/go-ipfs/security/advisories/GHSA-mcq2-w56r-5w2w\n\nNote: the v0.11.1 patch release contains the Docker compose fix from v0.12.1 as well\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipld/go-codec-dagpb (v1.3.0 -> v1.3.2):\n - fix: use protowire for Links bytes decoding\n\n
\n\n### ❤ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Rod Vagg | 1 | +34/-19 | 2 |\n\n## v0.12.1 2022-03-17\n\nThis patch release [fixes](https://github.com/ipfs/go-ipfs/commit/816a128aaf963d72c4930852ce32b9a4e31924a1) a security issue with the `docker-compose.yaml` file in which the IPFS daemon API listens on all interfaces instead of only the loopback interface, which could allow remote callers to control your IPFS daemon. If you use the included `docker-compose.yaml` file, it is recommended to upgrade.\n\nSee also the security advisory: https://github.com/ipfs/go-ipfs/security/advisories/GHSA-fx5p-f64h-93xc\n\nThanks to @LynHyper for finding and disclosing this.\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipfs/go-ipfs:\n - fix: listen on loopback for API and gateway ports in docker-compose.yaml\n\n
\n\n### ❤ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| guseggert | 1 | +10/-3 | 1 |\n\n## v0.12.0 2022-02-17\n\nWe're happy to announce go-ipfs 0.12.0. This release switches the storage of IPLD blocks to be keyed by multihash instead of CID.\n\nAs usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will _not_ be called out in the release notes. Please make sure to update ASAP. See our [release process](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md#security-fix-policy) for details.\n\n### 🛠 BREAKING CHANGES\n\n- `ipfs refs local` will now list all blocks as if they were [raw]() CIDv1 instead of with whatever CID version and IPLD codecs they were stored with. All other functionality should remain the same.\n\nNote: This change also effects [ipfs-update](https://github.com/ipfs/ipfs-update) so if you use that tool to manage your go-ipfs installation then grab ipfs-update v1.8.0 from [dist](https://dist.ipfs.tech/#ipfs-update).\n\nKeep reading to learn more details.\n\n#### 🔦 Highlights\n\nThere is only one change since 0.11:\n\n##### Blockstore migration from full CID to Multihash keys\n\nWe are switching the default low level [datastore](https://docs.ipfs.tech/concepts/glossary/#datastore) to be keyed only by the [Multihash](https://docs.ipfs.tech/concepts/glossary/#multihash) part of the [CID](https://docs.ipfs.tech/concepts/glossary/#cid), and deduplicate some [blocks](https://docs.ipfs.tech/concepts/glossary/#block) in the process. The blockstore will become [codec](https://docs.ipfs.tech/concepts/glossary/#codec)-agnostic.\n\n###### Rationale\n\nThe blockstore/datastore layers are not concerned with data interpretation, only with storage of binary blocks and verification that the Multihash they are addressed with (which comes from the CID), matches the block. In fact, different CIDs, with different codecs prefixes, may be carrying the same multihash, and referencing the same block. Carrying the CID abstraction so low on the stack means potentially fetching and storing the same blocks multiple times just because they are referenced by different CIDs. Prior to this change, a CIDv1 with a `dag-cbor` codec and a CIDv1 with a `raw` codec, both containing the same multihash, would result in two identical blocks stored. A CIDv0 and CIDv1 both being the same `dag-pb` block would also result in two copies.\n\n###### How migration works\n\nIn order to perform the switch, and start referencing all blocks by their multihash, a migration will occur on update. This migration will take the repository version from 11 (current) to 12.\n\nOne thing to note is that any content addressed CIDv0 (all the hashes that start with `Qm...`, the current default in go-ipfs), does not need any migration, as CIDv0 are raw multihashes already. This means the migration will be very lightweight for the majority of users.\n\nThe migration process will take care of re-keying any CIDv1 block so that it is only addressed by its multihash. Large nodes with lots of CIDv1-addressed content will need to go through a heavier process as the migration happens. This is how the migration works:\n\n1. Phase 1: The migration script will perform a pass for every block in the datastore and will add all CIDv1s found to a file named `11-to-12-cids.txt`, in the go-ipfs configuration folder. Nothing is written in this first phase and it only serves to identify keys that will be migrated in phase 2.\n2. Phase 2: The migration script will perform a second pass where every CIDv1 block will be read and re-written with its raw-multihash as key. There is 1 worker performing this task, although more can be configured. Every 100MiB-worth of blocks (this is configurable), each worker will trigger a datastore \"sync\" (to ensure all written data is flushed to disk) and delete the CIDv1-addressed blocks that were just renamed. This provides a good compromise between speed and resources needed to run the migration.\n\nAt every sync, the migration emits a log message showing how many blocks need to be rewritten and how far the process is.\n\n####### FlatFS specific migration\n\nFor those using a single FlatFS datastore as their backing blockstore (i.e. the default behavior), the migration (but not reversion) will take advantage of the ability to easily move/rename the blocks to improve migration performance.\n\nUnfortunately, other common datastores do not support renames which is what makes this FlatFS specific. If you are running a large custom datastore that supports renames you may want to consider running a fork of [fs-repo-11-to-12](https://github.com/ipfs/fs-repo-migrations/tree/master/fs-repo-11-to-12) specific to your datastore.\n\nIf you want to disable this behavior, set the environment variable `IPFS_FS_MIGRATION_11_TO_12_ENABLE_FLATFS_FASTPATH` to `false`.\n\n####### Migration configuration\n\nFor those who want to tune the migration more precisely for their setups, there are two environment variables to configure:\n\n- `IPFS_FS_MIGRATION_11_TO_12_NWORKERS` : an integer describing the number of migration workers - defaults to 1\n- `IPFS_FS_MIGRATION_11_TO_12_SYNC_SIZE_BYTES` : an integer describing the number of bytes after which migration workers will sync - defaults to 104857600 (i.e. 100MiB)\n\n###### Migration caveats\n\nLarge repositories with very large numbers of CIDv1s should be mindful of the migration process:\n\n* We recommend ensuring that IPFS runs with an appropriate (high) file-descriptor limit, particularly when Badger is use as datastore backend. Badger is known to open many tables when experiencing a high number of writes, which may trigger \"too many files open\" type of errors during the migrations. If this happens, the migration can be retried with a higher FD limit (see below).\n* Migrations using the Badger datastore may not immediately reclaim the space freed by the deletion of migrated blocks, thus space requirements may grow considerably. A periodic Badger-GC is run every 2 minutes, which will reclaim space used by deleted and de-duplicated blocks. The last portion of the space will only be reclaimed after go-ipfs starts (the Badger-GC cycle will trigger after 15 minutes).\n* While there is a revert process detailed below, we recommend keeping a backup of the repository, particularly for very large ones, in case an issue happens, so that the revert can happen immediately and cases of repository corruption due to crashes or unexpected circumstances are not catastrophic.\n\n###### Migration interruptions and retries\n\nIf a problem occurs during the migration, it is be possible to simply re-start and retry it:\n\n1. Phase 1 will never overwrite the `11-to-12-cids.txt` file, but only append to it (so that a list of things we were supposed to have migrated during our first attempt is not lost - this is important for reverts, see below).\n2. Phase 2 will proceed to continue re-keying blocks that were not re-keyed during previous attempts.\n\n###### Migration reverts\n\nIt is also possible to revert the migration after it has succeeded, for example to go to a previous go-ipfs version (<=[0.11](https://github.com/ipfs/go-ipfs/releases/tag/v0.11.0)), even after starting and using go-ipfs in the new version (>=0.12). The revert process works as follows:\n\n1. The `11-to-12-cids.txt` file is read, which has the list of all the CIDv1s that had to be rewritten for the migration.\n2. A CIDv1-addressed block is written for every item on the list. This work is performed by 1 worker (configurable), syncing every 100MiB (configurable).\n3. It is ensured that every CIDv1 pin, and every CIDv1 reference in MFS, are also written as CIDV1-addressed blocks, regardless of whether they were part of the original migration or were added later.\n\nThe revert process does not delete any blocks--it only makes sure that blocks that were accessible with CIDv1s before the migration are again keyed with CIDv1s. This may result in a datastore becoming twice as large (i.e. if all the blocks were CIDv1-addressed before the migration). This is however done this way to cover corner cases: user can add CIDv1s after migration, which may reference blocks that existed as CIDv0 before migration. The revert aims to ensure that no data becomes unavailable on downgrade.\n\nWhile go-ipfs will auto-run the migration for you, it will not run the reversion. To do so you can download the [latest migration binary](https://dist.ipfs.tech/fs-repo-11-to-12) or use [ipfs-update](https://dist.ipfs.tech/#ipfs-update).\n\n###### Custom datastores\n\nAs with previous migrations if you work with custom datastores and want to leverage the migration you can run a fork of [fs-repo-11-to-12](https://github.com/ipfs/fs-repo-migrations/tree/master/fs-repo-11-to-12) specific to your datastore. The repo includes instructions on building for different datastores.\n\nFor this migration, if your datastore has fast renames you may want to consider writing some code to leverage the particular efficiencies of your datastore similar to what was done for FlatFS.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - Release v0.12.0\n - docs: v0.12.0 release notes\n - chore: bump migrations dist.ipfs.tech CID to contain fs-repo-11-to-12 v1.0.2\n - feat: refactor Fetcher interface used for downloading migrations (#8728) ([ipfs/go-ipfs#8728](https://github.com/ipfs/go-ipfs/pull/8728))\n - feat: log multifetcher errors\n - Release v0.12.0-rc1\n - chore: bump Go version to 1.16.12\n - feat: switch to raw multihashes for blocks ([ipfs/go-ipfs#6816](https://github.com/ipfs/go-ipfs/pull/6816))\n - chore: add release template snippet for fetching artifact tarball\n - chore: bump Go version to 1.16.11\n - chore: add release steps for upgrading Go\n - Merge branch 'release'\n - fix(corehttp): adjust peer counting metrics (#8577) ([ipfs/go-ipfs#8577](https://github.com/ipfs/go-ipfs/pull/8577))\n - chore: update version to v0.12.0-dev\n- github.com/ipfs/go-filestore (v0.1.0 -> v1.1.0):\n - Version 1.1.0\n - feat: plumb through context changes\n - sync: update CI config files (#54) ([ipfs/go-filestore#54](https://github.com/ipfs/go-filestore/pull/54))\n - fix staticcheck ([ipfs/go-filestore#48](https://github.com/ipfs/go-filestore/pull/48))\n - Work with Multihashes directly (#21) ([ipfs/go-filestore#21](https://github.com/ipfs/go-filestore/pull/21))\n- github.com/ipfs/go-ipfs-blockstore (v0.2.1 -> v1.1.2):\n - Release v1.1.2\n - feat: per-cid locking\n - Version 1.1.1\n - fix: remove context from HashOnRead\n - Version 1.1.0 (#91) ([ipfs/go-ipfs-blockstore#91](https://github.com/ipfs/go-ipfs-blockstore/pull/91))\n - feat: add context to interfaces (#90) ([ipfs/go-ipfs-blockstore#90](https://github.com/ipfs/go-ipfs-blockstore/pull/90))\n - sync: update CI config files (#88) ([ipfs/go-ipfs-blockstore#88](https://github.com/ipfs/go-ipfs-blockstore/pull/88))\n - add constructor that doesn't mess with datastore keys ([ipfs/go-ipfs-blockstore#83](https://github.com/ipfs/go-ipfs-blockstore/pull/83))\n - Use bloom filter in GetSize\n - fix staticcheck ([ipfs/go-ipfs-blockstore#73](https://github.com/ipfs/go-ipfs-blockstore/pull/73))\n - add BenchmarkARCCacheConcurrentOps ([ipfs/go-ipfs-blockstore#70](https://github.com/ipfs/go-ipfs-blockstore/pull/70))\n - fix(arc): striped locking on last byte of CID ([ipfs/go-ipfs-blockstore#67](https://github.com/ipfs/go-ipfs-blockstore/pull/67))\n - make idstore implement io.Closer. (#60) ([ipfs/go-ipfs-blockstore#60](https://github.com/ipfs/go-ipfs-blockstore/pull/60))\n - add View() to all the various blockstores. (#59) ([ipfs/go-ipfs-blockstore#59](https://github.com/ipfs/go-ipfs-blockstore/pull/59))\n - Optimize id store ([ipfs/go-ipfs-blockstore#56](https://github.com/ipfs/go-ipfs-blockstore/pull/56))\n - add race fix for HashOnRead ([ipfs/go-ipfs-blockstore#50](https://github.com/ipfs/go-ipfs-blockstore/pull/50))\n - Add test to maintain Put contract of calling Has first ([ipfs/go-ipfs-blockstore#47](https://github.com/ipfs/go-ipfs-blockstore/pull/47))\n - Update readme and license ([ipfs/go-ipfs-blockstore#44](https://github.com/ipfs/go-ipfs-blockstore/pull/44))\n - feat: switch to raw multihashes for blocks ([ipfs/go-ipfs-blockstore#38](https://github.com/ipfs/go-ipfs-blockstore/pull/38))\n- github.com/ipfs/go-ipfs-ds-help (v0.1.1 -> v1.1.0):\n - Update version.json (#38) ([ipfs/go-ipfs-ds-help#38](https://github.com/ipfs/go-ipfs-ds-help/pull/38))\n - sync: update CI config files (#37) ([ipfs/go-ipfs-ds-help#37](https://github.com/ipfs/go-ipfs-ds-help/pull/37))\n - feat: switch to raw multihashes for blocks ([ipfs/go-ipfs-ds-help#18](https://github.com/ipfs/go-ipfs-ds-help/pull/18))\n\n### ❤️ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Gus Eggert | 10 | +333/-321 | 24 |\n| Steven Allen | 7 | +289/-190 | 13 |\n| Hector Sanjuan | 9 | +134/-109 | 18 |\n| Adin Schmahmann | 11 | +179/-55 | 21 |\n| Raúl Kripalani | 2 | +152/-42 | 5 |\n| Daniel Martí | 1 | +120/-1 | 1 |\n| frrist | 1 | +95/-13 | 2 |\n| Alex Trottier | 2 | +22/-11 | 4 |\n| Andrey Petrov | 1 | +32/-0 | 1 |\n| Lucas Molas | 1 | +18/-7 | 2 |\n| Marten Seemann | 2 | +11/-7 | 3 |\n| whyrusleeping | 1 | +10/-0 | 1 |\n| web3-bot | 3 | +9/-0 | 3 |\n| postables | 1 | +5/-3 | 1 |\n| Dr Ian Preston | 1 | +4/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.13.md", "content": "# go-ipfs changelog 2022\n\n## v0.13.1 2022-07-06\n\nThis release includes security fixes for various DOS vectors when importing untrusted user input with `ipfs dag import`\nand the [`v0/dag/import`](https://docs.ipfs.tech/reference/kubo/rpc/#api-v0-dag-import) endpoint.\n\nView the linked [security advisory](https://github.com/ipfs/go-ipfs/security/advisories/GHSA-f2gr-7299-487h) for more information.\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipfs/go-ipfs:\n - chore: update car\n- github.com/ipld/go-car (v0.3.2 -> v0.4.0) & (v2.1.1 -> v2.4.0):\n - Bump version in prep for releasing go-car `v0`\n - Revert changes to `insertionindex`\n - Revert changes to `index.Index` while keeping most of security fixes\n - Return error when section length is invalid `varint`\n - Drop repeated package name from `CarStats`\n - Benchmark `Reader.Inspect` with and without hash validation\n - Use consistent CID mismatch error in `Inspect` and `BlockReader.Next`\n - Use streaming APIs to verify the hash of blocks in CAR `Inspect`\n - test: add fuzzing for reader#Inspect\n - feat: add block hash validation to Inspect()\n - feat: add Reader#Inspect() function to check basic validity of a CAR and return stats\n - Remove support for `ForEach` enumeration from car-index-sorted\n - Use a fix code as the multihash code for `CarIndexSorted`\n - Fix testutil assertion logic and update index generation tests\n - fix: tighter constraint of singleWidthIndex width, add index recommendation docs\n - fix: explicitly disable serialization of insertionindex\n - feat: MaxAllowed{Header,Section}Size option\n - feat: MaxAllowedSectionSize default to 32M\n - fix: use CidFromReader() which has overread and OOM protection\n - fix: staticcheck catches\n - fix: revert to internalio.NewOffsetReadSeeker in Reader#IndexReader\n - fix index comparisons\n - feat: Refactor indexes to put storage considerations on consumers\n - test: v2 add fuzzing of the index\n - fix: v2 don't divide by zero in width indexes\n - fix: v2 don't allocate indexes too big\n - test: v2 add fuzzing to Reader\n - fix: v2 don't accept overflowing offsets while reading v2 headers\n - test: v2 add fuzzing to BlockReader\n - fix: v2 don't OOM if the header size is too big\n - test: add fuzzing of NewCarReader\n - fix: do bound check while checking for CIDv0\n - fix: don't OOM if the header size is too big\n - Add API to regenerate index from CARv1 or CARv2\n - PrototypeChooser support (#305) ([ipld/go-car#305](https://github.com/ipld/go-car/pull/305))\n - bump to newer blockstore err not found (#301) ([ipld/go-car#301](https://github.com/ipld/go-car/pull/301))\n - Car command supports for `largebytes` nodes (#296) ([ipld/go-car#296](https://github.com/ipld/go-car/pull/296))\n - fix(test): rootless fixture should have no roots, not null roots\n - Allow extraction of a raw unixfs file (#284) ([ipld/go-car#284](https://github.com/ipld/go-car/pull/284))\n - cmd/car: use a better install command in the README\n - feat: --version selector for `car create` & update deps\n - feat: add option to create blockstore that writes a plain CARv1 (#288) ([ipld/go-car#288](https://github.com/ipld/go-car/pull/288))\n - add `car detach-index list` to list detached index contents (#287) ([ipld/go-car#287](https://github.com/ipld/go-car/pull/287))\n - add `car root` command (#283) ([ipld/go-car#283](https://github.com/ipld/go-car/pull/283))\n - make specification of root cid in get-dag command optional (#281) ([ipld/go-car#281](https://github.com/ipld/go-car/pull/281))\n - Update `version.json` after manual tag push\n - Update v2 to context datastores (#275) ([ipld/go-car#275](https://github.com/ipld/go-car/pull/275))\n - update context datastore ([ipld/go-car#273](https://github.com/ipld/go-car/pull/273))\n - Traversal-based car creation (#269) ([ipld/go-car#269](https://github.com/ipld/go-car/pull/269))\n - Seek to start before index generation in `ReadOnly` blockstore\n - support extraction of unixfs content stored in car files (#263) ([ipld/go-car#263](https://github.com/ipld/go-car/pull/263))\n - Add a bare bones readme to the car CLI (#262) ([ipld/go-car#262](https://github.com/ipld/go-car/pull/262))\n - sync: update CI config files (#261) ([ipld/go-car#261](https://github.com/ipld/go-car/pull/261))\n - fix!: use -version=n instead of -v1 for index command\n - feat: fix get-dag and add version=1 option\n - creation of car from file / directory (#246) ([ipld/go-car#246](https://github.com/ipld/go-car/pull/246))\n - forEach iterates over index in stable order (#258) ([ipld/go-car#258](https://github.com/ipld/go-car/pull/258))\n- github.com/multiformats/go-multicodec (v0.4.1 -> v0.5.0):\n - Bump version to 0.5.0\n - Bump version to 0.4.2\n - deps: update stringer version in go generate command\n - docs(readme): improved usage examples (#66) ([multiformats/go-multicodec#66](https://github.com/multiformats/go-multicodec/pull/66))\n\n
\n\n### ❤ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Masih H. Derkani | 27 | +1494/-1446 | 100 |\n| Rod Vagg | 31 | +2021/-606 | 105 |\n| Will | 19 | +1898/-151 | 69 |\n| Jorropo | 27 | +1638/-248 | 76 |\n| Aayush Rajasekaran | 1 | +130/-100 | 10 |\n| whyrusleeping | 1 | +24/-22 | 4 |\n| Marcin Rataj | 1 | +27/-1 | 1 |\n\n## v0.13.0 2022-05-04\n\nWe're happy to announce go-ipfs 0.13.0, packed full of changes and improvements!\n\nAs usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will _not_ be called out in the release notes. Please make sure to update ASAP. See our [release process](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md#security-fix-policy) for details.\n\n### Overview\n\nBelow is an outline of all that is in this release, so you get a sense of all that's included.\n\n- [🛠 BREAKING CHANGES](#---breaking-changes)\n * [`ipfs block put` command](#-ipfs-block-put--command)\n * [`ipfs cid codecs` command](#-ipfs-cid-codecs--command)\n * [`Swarm` configuration](#-swarm--configuration)\n * [Circuit Relay V1 is deprecated](#circuit-relay-v1-is-deprecated)\n * [`ls` requests for `/multistream/1.0.0` are removed](#-ls--requests-for---multistream-100--are-removed)\n * [Gateway Items](#gateway-items)\n- [🔦 Highlights](#---highlights)\n * [🧑‍💼 libp2p Network Resource Manager (`Swarm.ResourceMgr`)](#------libp2p-network-resource-manager---swarmresourcemgr--)\n * [🔃 Relay V2 client with auto discovery (`Swarm.RelayClient`)](#---relay-v2-client-with-auto-discovery---swarmrelayclient--)\n * [🌉 HTTP Gateway improvements](#---http-gateway-improvements)\n + [🍱 Support for Block and CAR response formats](#---support-for-block-and-car-response-formats)\n + [🐎 Fast listing generation for huge directories](#---fast-listing-generation-for-huge--directories)\n + [🎫 Improved `Etag` and `If-None-Match` for bandwidth savings](#---improved--etag--and--if-none-match--for-bandwidth-savings)\n + [⛓️ Added X-Ipfs-Roots for smarter HTTP caches](#---added-x-ipfs-roots-for-smarter-http-caches)\n + [🌡️ Added metrics per response type](#----added-metrics-per-response-type)\n * [🕵️ OpenTelemetry tracing](#----opentelemetry-tracing)\n + [How to use Jaeger UI for visual tracing?](#how-to-use-jaeger-ui-for-visual-tracing-)\n * [🩺 Built-in `ipfs diag profile` to ease debugging](#---built-in--ipfs-diag-profile--to-ease-debugging)\n * [🔑 Support for PEM/PKCS8 for key import/export](#---support-for-pem-pkcs8-for-key-import-export)\n * [🧹 Using standard IPLD codec names across the CLI/HTTP API](#---using-standard-ipld-codec-names-across-the-cli-http-api)\n * [🐳 Custom initialization for Docker](#---custom-initialization-for-docker)\n * [RPC API docs for experimental and deprecated commands](#rpc-api-docs-for-experimental-and-deprecated-commands)\n * [Yamux over Mplex](#yamux-over-mplex)\n\n### 🛠 BREAKING CHANGES\n\n#### `ipfs block put` command\n\n`ipfs block put` command returns a CIDv1 with `raw` codec by default now.\n- `ipfs block put --cid-codec` makes `block put` return CID with alternative codec\n - This impacts only the returned CID; it does not trigger any validation or data transformation.\n - Retrieving a block with a different codec or CID version than it was put with is valid.\n - Codec names are validated against tables from [go-multicodec](https://github.com/multiformats/go-multicodec) library.\n- `ipfs block put --format` is deprecated. It used incorrect codec names and should be avoided for new deployments. Use it only if you need the old, invalid behavior, namely:\n - `ipfs block put --format=v0` will produce CIDv0 (implicit dag-pb)\n - `ipfs block put --format=cbor` will produce CIDv1 with dag-cbor (!)\n - `ipfs block put --format=protobuf` will produce CIDv1 with dag-pb (!)\n\n#### `ipfs cid codecs` command\n- Now lists codecs from [go-multicodec](https://github.com/multiformats/go-multicodec) library.\n- `ipfs cid codecs --supported` can be passed to only show codecs supported in various go-ipfs commands.\n\n#### `ipfs cid format` command\n- `--codec` was removed and replaced with `--mc` to ensure existing users are aware of the following changes:\n - `--mc protobuf` now correctly points to code `0x50` (was `0x70`, which is `dab-pg`)\n - `--mc cbor` now correctly points to code `0x51` (was `0x71`, which is `dag-cbor`)\n\n#### `Swarm` configuration\n- Daemon will refuse to start if long-deprecated RelayV1 config key `Swarm.EnableAutoRelay` or `Swarm.DisableRelay` is set to `true`.\n- If `Swarm.Transports.Network.Relay` is disabled, then `Swarm.RelayService` and `Swarm.RelayClient` are also disabled (unless they have been explicitly enabled).\n\n#### Circuit Relay V1 is deprecated\n- By default, `Swarm.RelayClient` does not use Circuit Relay V1. Circuit V1 support is only enabled when `Swarm.RelayClient.StaticRelays` are specified.\n\n#### `ls` requests for `/multistream/1.0.0` are removed\n- go-libp2p 0.19 removed support for undocumented `ls` command ([PR](https://github.com/multiformats/go-multistream/pull/76)). If you are still using it for internal testing, it is time to refactor ([example](https://github.com/ipfs/go-ipfs/commit/39047bcf61163096d1c965283d671c7c487c9173))\n\n#### Gateway Behavior\nDirectory listings returned by the HTTP Gateway won't have size column if the directory is bigger than [`Gateway.FastDirIndexThreshold`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#gatewayfastdirindexthreshold) config (default is 100).\n\nTo understand the wider context why we made these changes, read *Highlights* below.\n\n### 🔦 Highlights\n\n#### 🧑‍💼 libp2p Network Resource Manager (`Swarm.ResourceMgr`)\n\n*You can now easily bound how much resource usage libp2p consumes! This aids in protecting nodes from consuming more resources then are available to them.*\n\nThe [libp2p Network Resource Manager](https://github.com/libp2p/go-libp2p-resource-manager#readme) is disabled by default, but can be enabled via:\n\n`ipfs config --json Swarm.ResourceMgr.Enabled true`\n\nWhen enabled, it applies some safe defaults that can be inspected and adjusted with:\n\n- `ipfs swarm stats --help`\n- `ipfs swarm limit --help`\n\nUser changes persist to config at [`Swarm.ResourceMgr`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmresourcemgr).\n\nThe Resource Manager will be enabled by default in a future release.\n\n#### 🔃 Relay V2 client with auto discovery (`Swarm.RelayClient`)\n\n*All the pieces are enabled for [hole-punching](https://blog.ipfs.io/2022-01-20-libp2p-hole-punching/) by default, improving connecting with nodes behind NATs and Firewalls!*\n\nThis release enables [`Swarm.RelayClient`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmrelayclient) by default, along with circuit v2 relay discovery provided by go-libp2p [v0.19.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.19.0). This means:\n1. go-ipfs will coordinate with the counterparty using a [relayed connection](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md), to [upgrade to a direct connection](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md) through a NAT/firewall whenever possible.\n2. go-ipfs daemon will automatically use [public relays](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmrelayservice) if it detects that it cannot be reached from the public internet (e.g., it's behind a firewall). This results in a `/p2p-circuit` address from a public relay.\n\n**Notes:**\n- [`Swarm.RelayClient`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmrelayclient) does not use Circuit Relay V1 nodes any more. Circuit V1 support is only enabled when static relays are specified in [`Swarm.RelayClient.StaticRelays`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmrelayclientstaticrelays).\n- One can opt-out via [`Swarm.EnableHolePunching`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmenableholepunching).\n\n\n#### 🌉 HTTP Gateway improvements\n\nHTTP Gateway enables seamless interop with the existing Web, clients, user agents, tools, frameworks and libraries.\n\nThis release ships the first batch of improvements that enable creation of faster and smarter CDNs, and unblocks creation of light clients for Mobile and IoT.\n\nDetails below.\n\n##### 🍱 Support for Block and CAR response formats\n\n*Alternative response formats from Gateway can be requested to avoid needing to trust a gateway.*\n\nFor now, `{format}` is limited to two options:\n- `raw` – fetching single block\n- `car` – fetching entire DAG behind a CID as a [CARv1 stream](https://ipld.io/specs/transport/car/carv1/)\n\nWhen not set, the default UnixFS response is returned.\n\n*Why these two formats?* Requesting Block or CAR for `/ipfs/{cid}` allows a client to **use gateways in a trustless fashion**. These types of gateway responses can be verified locally and rejected if digest inside of requested CID does not match received bytes. This enables creation of \"light IPFS clients\" which use HTTP Gateways as inexpensive transport for [content-addressed](https://docs.ipfs.tech/concepts/content-addressing/) data, unlocking use in Mobile and IoT contexts.\n\n\nFuture releases will [add support for dag-json and dag-cbor responses](https://github.com/ipfs/go-ipfs/issues/8823).\n\nThere are two ways for requesting CID specific response format:\n1. HTTP header: `Accept: application/vnd.ipld.{format}`\n- Examples: [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car), [application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw)\n2. URL parameter: `?format=`\n- Useful for creating \"Download CAR\" links.\n\n*Usage examples:*\n\n1. Downloading a single raw Block and manually importing it to the local datastore:\n\n```console\n$ curl -H 'Accept: application/vnd.ipld.raw' \"http://127.0.0.1:8080/ipfs/QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\" --output block.bin\n$ cat block.bin | ipfs block put\n$ ipfs cat QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\nhello\n```\n\n2. Downloading entire DAG as a CAR file and importing it:\n\n```console\n$ ipfs resolve -r /ipns/webui.ipfs.io\n/ipfs/bafybeiednzu62vskme5wpoj4bjjikeg3xovfpp4t7vxk5ty2jxdi4mv4bu\n$ curl -H 'Accept: application/vnd.ipld.car' \"http://127.0.0.1:8080/ipfs/bafybeiednzu62vskme5wpoj4bjjikeg3xovfpp4t7vxk5ty2jxdi4mv4bu\" --output webui.car\n$ ipfs dag import webui.car\n$ ipfs dag stat bafybeiednzu62vskme5wpoj4bjjikeg3xovfpp4t7vxk5ty2jxdi4mv4bu --offline\nSize: 27684934, NumBlocks: 394\n```\n\nSee also:\n\n- [Content Addressable aRchives (CAR / .car) Specifications](https://ipld.io/specs/transport/car/)\n- [IANA media type](https://www.iana.org/assignments/media-types/media-types.xhtml) definitions: [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car), [application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw)\n- [ipfs-car](https://www.npmjs.com/package/ipfs-car) - CLI tool for verifying and unpacking CAR files\n- [go-car](https://github.com/ipld/go-car), [js-car](https://github.com/ipld/js-car/) – CAR libraries for GO and JS\n\n##### 🐎 Fast listing generation for huge directories\n\n*Added [`Gateway.FastDirIndexThreshold`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#gatewayfastdirindexthreshold) configuration, which allows for fast listings of big directories, without the linear slowdown caused by reading size metadata from child nodes.*\n\nAs an example, the CID `bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm` represents UnixFS directory with over 10k (10100) of files.\n\nOpening it with go-ipfs 0.12 would require fetching size information of each file, which would take a long long time, most likely causing timeout in the browser or CDN, and introducing unnecessary burden on the gateway node.\n\ngo-ipfs 0.13 opens it instantly, because the number of items is bigger than the default [`Gateway.FastDirIndexThreshold`](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#gatewayfastdirindexthreshold) and only the root UnixFS node needs to be resolved before the HTML Dir Index is returned to the user.\n\nNotes:\n- The default threshold is 100 items.\n- Setting to 0 will enable fast listings for all directories.\n- CLI users will note that this is equivalent to running `ipfs ls -s --size=false --resolve-type=false /ipfs/bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm`. Now the same speed is available on the gateways.\n\n##### 🎫 Improved `Etag` and `If-None-Match` for bandwidth savings\n\n*Every response type has an unique [`Etag`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag) which can be used by the client or CDN to save bandwidth, as a gateway does not need to resend a full response if the content was not changed.*\n\nGateway evaluates Etags sent by a client in [`If-None-Match`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match) and returns status code 304 (Not Modified) on strong or weak match ([RFC 7232, 2.3](https://datatracker.ietf.org/doc/html/rfc7232#section-2.3)).\n\n##### ⛓️ Added X-Ipfs-Roots for smarter HTTP caches\n\n`X-Ipfs-Roots` is now returned with every Gateway response. It is a way to indicate all CIDs required for resolving path segments from `X-Ipfs-Path`. Together, these two headers are meant to improve interop with existing HTTP software (load-balancers, caches, CDNs).\n\nThis additional information allows HTTP caches and CDNs to make better decisions around cache invalidation: not just invalidate everything under specific IPNS website when the root changes, but do more fine-grained cache invalidation by detecting when only a specific subdirectory (branch of a [DAG](https://docs.ipfs.tech/concepts/glossary/#dag)) changes.\n\n##### 🌡️ Added metrics per response type\n\nNew metrics can be found at `/debug/metrics/prometheus` on the RPC API port (`127.0.0.1:5001` is the default):\n\n- `gw_first_content_block_get_latency_seconds` – the time until the first content block is received on GET from the gateway (no matter the content or response types)\n- `gw_unixfs_file_get_duration_seconds` – the time to serve an entire UnixFS file from the gateway\n- `gw_unixfs_gen_dir_listing_get_duration_seconds` – the time to serve a generated UnixFS HTML directory listing from the gateway\n- `gw_car_stream_get_duration_seconds` – the time to GET an entire CAR stream from the gateway\n- `gw_raw_block_get_duration_seconds` – The time to GET an entire raw Block from the gateway\n\n\n#### 🕵️ OpenTelemetry tracing\n\n*Opt-in tracing support with many spans for tracing the duration of specific tasks performed by go-ipfs.*\n\nSee [Tracing](https://github.com/ipfs/go-ipfs/blob/master/docs/environment-variables.md#tracing) for details.\n\nWe will continue to add tracing instrumentation throughout IPFS subcomponents over time.\n\n##### How to use Jaeger UI for visual tracing?\n\nOne can use the `jaegertracing/all-in-one` Docker image to run a full Jaeger stack and configure go-ipfs to publish traces to it (here, in an ephemeral container):\n\n```console\n$ docker run --rm -it --name jaeger \\\n -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \\\n -p 5775:5775/udp \\\n -p 6831:6831/udp \\\n -p 6832:6832/udp \\\n -p 5778:5778 \\\n -p 16686:16686 \\\n -p 14268:14268 \\\n -p 14250:14250 \\\n -p 9411:9411 \\\n jaegertracing/all-in-one\n```\n\nThen, in other terminal, start go-ipfs with Jaeger tracing enabled:\n```\n$ OTEL_TRACES_EXPORTER=jaeger ipfs daemon\n```\n\nFinally, the [Jaeger UI](https://github.com/jaegertracing/jaeger-ui#readme) is available at http://localhost:16686\n\nBelow are examples of visual tracing for Gateway requests. (Note: this a preview how useful this insight is. Details may look different now, as we are constantly improving tracing annotations across the go-ipfs codebase.)\n\n| CAR | Block | File | Directory |\n| ---- | ---- | ---- | ---- |\n| ![2022-04-01_01-46](https://user-images.githubusercontent.com/157609/161167986-951d5c8c-9a5e-464d-bc20-81eb5ccbdc22.png) | ![block_2022-04-01_01-47](https://user-images.githubusercontent.com/157609/161167983-e8cac0ce-0575-4271-8cb8-4d44a0d5d786.png) | ![2022-04-01_01-49](https://user-images.githubusercontent.com/157609/161167978-e19aa44c-f5a4-45f4-b7c7-14c313ab1dee.png) | ![dir_2022-04-01_01-48](https://user-images.githubusercontent.com/157609/161167981-456ca52b-3e87-4042-916b-8db149071228.png) |\n\n#### 🩺 Built-in `ipfs diag profile` to ease debugging\nThe `diag profile` command has been expanded to include all information that was previously included in the `collect-profiles.sh` script, and the script has been removed. Profiles are now collected in parallel, so that profile collection is much faster. Specific profiles can also be selected for targeted debugging.\n\nSee `ipfs diag profile --help` for more details.\n\nFor general debugging information, see [the debug guide](https://github.com/ipfs/go-ipfs/blob/master/docs/debug-guide.md).\n\n#### 🔑 Support for PEM/PKCS8 for key import/export\n\nIt is now possible to import or export private keys wrapped in interoperable [PEM PKCS8](https://en.wikipedia.org/wiki/PKCS_8) by passing `--format=pem-pkcs8-cleartext` to `ipfs key import` and `export` commands.\n\nThis improved interop allows for key generation outside of the IPFS node:\n\n```console\n$ openssl genpkey -algorithm ED25519 > ed25519.pem\n$ ipfs key import test-openssl -f pem-pkcs8-cleartext ed25519.pem\n```\n\nOr using external tools like the standard `openssl` to get a PEM file with the public key:\n```console\n $ ipfs key export testkey --format=pem-pkcs8-cleartext -o privkey.pem\n $ openssl pkey -in privkey.pem -pubout > pubkey.pem\n```\n\n#### 🧹 Using standard IPLD codec names across the CLI/HTTP API\n\nThis release makes necessary (breaking) changes in effort to use canonical codec names from [multicodec/table.csv](https://github.com/multiformats/multicodec/blob/master/table.csv). We also switched to CIDv1 in `block put`. The breaking changes are discussed above.\n\n\n#### 🐳 Custom initialization for Docker\n\nDocker images published at https://hub.docker.com/r/ipfs/go-ipfs/ now support custom initialization by mounting scripts in the `/container-init.d` directory in the container. Scripts can set custom configuration using `ipfs config`, or otherwise customize the container before the daemon is started.\n\nScripts are executed sequentially and in lexicographic order, before the IPFS daemon is started and after `ipfs init` is run and the swarm keys are copied (if the IPFS repo needs initialization).\n\nFor more information, see:\n- Documentation: [ Run IPFS inside Docker](https://docs.ipfs.tech/how-to/run-ipfs-inside-docker/) \n- Examples in [ipfs-shipyard/go-ipfs-docker-examples](https://github.com/ipfs-shipyard/go-ipfs-docker-examples).\n\n#### RPC API docs for experimental and deprecated commands\n\nhttps://docs.ipfs.tech/reference/kubo/rpc/ now includes separate sections for _experimental_ and _deprecated_ commands.\n\nWe also display a warning in the command line:\n\n```console\n$ ipfs name pubsub state --help\nWARNING: EXPERIMENTAL, command may change in future releases\n```\n\n#### Yamux over Mplex\n\nThe more fully featured yamux stream multiplexer is now prioritized over mplex for outgoing connections.\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipfs/go-ipfs:\n - feat: upgrade to go-libp2p-kad-dht@v0.16.0 (#9005) ([ipfs/go-ipfs#9005](https://github.com/ipfs/go-ipfs/pull/9005))\n - docs: fix typo in the `swarm/peering` help text\n - feat: disable resource manager by default (#9003) ([ipfs/go-ipfs#9003](https://github.com/ipfs/go-ipfs/pull/9003))\n - fix: adjust rcmgr limits for accelerated DHT client rt refresh (#8982) ([ipfs/go-ipfs#8982](https://github.com/ipfs/go-ipfs/pull/8982))\n - fix(ci): make go-ipfs-as-a-library work without external peers (#8978) ([ipfs/go-ipfs#8978](https://github.com/ipfs/go-ipfs/pull/8978))\n - feat: log when resource manager limits are exceeded (#8980) ([ipfs/go-ipfs#8980](https://github.com/ipfs/go-ipfs/pull/8980))\n - fix: JS caching via Access-Control-Expose-Headers (#8984) ([ipfs/go-ipfs#8984](https://github.com/ipfs/go-ipfs/pull/8984))\n - docs: fix abstractions typo\n - fix: hanging goroutine in get fileArchive handler\n - fix(node/libp2p): disable rcmgr checkImplicitDefaults ([ipfs/go-ipfs#8965](https://github.com/ipfs/go-ipfs/pull/8965))\n - pubsub multibase encoding (#8933) ([ipfs/go-ipfs#8933](https://github.com/ipfs/go-ipfs/pull/8933))\n - 'pin rm' helptext: rewrite description as object is not removed from local storage (immediately) ([ipfs/go-ipfs#8947](https://github.com/ipfs/go-ipfs/pull/8947))\n - ([ipfs/go-ipfs#8934](https://github.com/ipfs/go-ipfs/pull/8934))\n - Add instructions to resolve repo migration error (#8946) ([ipfs/go-ipfs#8946](https://github.com/ipfs/go-ipfs/pull/8946))\n - fix: use path instead of filepath for asset embeds to support Windows\n - Release v0.13.0-rc1\n - docs: v0.13.0 changelog ([ipfs/go-ipfs#8941](https://github.com/ipfs/go-ipfs/pull/8941))\n - chore: build with go 1.18.1 ([ipfs/go-ipfs#8932](https://github.com/ipfs/go-ipfs/pull/8932))\n - docs(tracing): update env var docs for new tracing env vars\n - feat: enable Resource Manager by default\n - chore: Update test/dependencies to match go-ipfs dependencies. (#8928) ([ipfs/go-ipfs#8928](https://github.com/ipfs/go-ipfs/pull/8928))\n - chore: fix linting errors (#8930) ([ipfs/go-ipfs#8930](https://github.com/ipfs/go-ipfs/pull/8930))\n - docs: Swarm.ResourceMgr.Limits\n - feat: EnableHolePunching by default ([ipfs/go-ipfs#8748](https://github.com/ipfs/go-ipfs/pull/8748))\n - ci: add more golang strictness checks ([ipfs/go-ipfs#8931](https://github.com/ipfs/go-ipfs/pull/8931))\n - feat(gateway): Gateway.FastDirIndexThreshold (#8853) ([ipfs/go-ipfs#8853](https://github.com/ipfs/go-ipfs/pull/8853))\n - docs: replace all git.io links with their actual URLs\n - feat: relay v2 discovery (go-libp2p v0.19.0) (#8868) ([ipfs/go-ipfs#8868](https://github.com/ipfs/go-ipfs/pull/8868))\n - fix(cmds): add: reject files with different import dir\n - chore: mark 'log tail' experimental (#8912) ([ipfs/go-ipfs#8912](https://github.com/ipfs/go-ipfs/pull/8912))\n - feat: persist limits to Swarm.ResourceMgr.Limits (#8901) ([ipfs/go-ipfs#8901](https://github.com/ipfs/go-ipfs/pull/8901))\n - fix: build after Go 1.17 and Prometheus upgrades (#8916) ([ipfs/go-ipfs#8916](https://github.com/ipfs/go-ipfs/pull/8916))\n - feat(tracing): use OpenTelemetry env vars where possible (#8875) ([ipfs/go-ipfs#8875](https://github.com/ipfs/go-ipfs/pull/8875))\n - feat(cmds): allow to set the configuration file path ([ipfs/go-ipfs#8634](https://github.com/ipfs/go-ipfs/pull/8634))\n - chore: deprecate /api/v0/dns (#8893) ([ipfs/go-ipfs#8893](https://github.com/ipfs/go-ipfs/pull/8893))\n - fix(cmds): CIDv1 and correct multicodecs in 'block put' and 'cid codecs' (#8568) ([ipfs/go-ipfs#8568](https://github.com/ipfs/go-ipfs/pull/8568))\n - feat(gw): improved If-None-Match support (#8891) ([ipfs/go-ipfs#8891](https://github.com/ipfs/go-ipfs/pull/8891))\n - Update Go version to 1.17 (#8815) ([ipfs/go-ipfs#8815](https://github.com/ipfs/go-ipfs/pull/8815))\n - chore(gw): extract logical functions to improve readability (#8885) ([ipfs/go-ipfs#8885](https://github.com/ipfs/go-ipfs/pull/8885))\n - feat(docker): /container-init.d for advanced initialization (#6577) ([ipfs/go-ipfs#6577](https://github.com/ipfs/go-ipfs/pull/6577))\n - feat: port collect-profiles.sh to 'ipfs diag profile' (#8786) ([ipfs/go-ipfs#8786](https://github.com/ipfs/go-ipfs/pull/8786))\n - fix: assets: correctly use the argument err in the WalkDirFunc Hashing Files\n - Change `assets.Asset` from a `func` to the embed.FS\n - Remove gobindata\n - fix: fix context plumbing in gateway handlers (#8871) ([ipfs/go-ipfs#8871](https://github.com/ipfs/go-ipfs/pull/8871))\n - fix(gw): missing return if dir fails to finalize (#8806) ([ipfs/go-ipfs#8806](https://github.com/ipfs/go-ipfs/pull/8806))\n - fix(gw): update metrics only when payload data sent (#8827) ([ipfs/go-ipfs#8827](https://github.com/ipfs/go-ipfs/pull/8827))\n - Merge branch 'release'\n - feat: detect changes in go-libp2p-resource-manager (#8857) ([ipfs/go-ipfs#8857](https://github.com/ipfs/go-ipfs/pull/8857))\n - feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) ([ipfs/go-ipfs#8680](https://github.com/ipfs/go-ipfs/pull/8680))\n - feat(cmds): add support for CAR v2 imports (#8854) ([ipfs/go-ipfs#8854](https://github.com/ipfs/go-ipfs/pull/8854))\n - docs(logging): environment variables (#8833) ([ipfs/go-ipfs#8833](https://github.com/ipfs/go-ipfs/pull/8833))\n - fix: unknown fetcher type error (#8830) ([ipfs/go-ipfs#8830](https://github.com/ipfs/go-ipfs/pull/8830))\n - chore: deprecate tar commands (#8849) ([ipfs/go-ipfs#8849](https://github.com/ipfs/go-ipfs/pull/8849))\n - chore: bump go-ipld-format v0.4.0 and fix related sharness tests ([ipfs/go-ipfs#8838](https://github.com/ipfs/go-ipfs/pull/8838))\n - feat: add basic gateway tracing (#8595) ([ipfs/go-ipfs#8595](https://github.com/ipfs/go-ipfs/pull/8595))\n - fix(cli): ipfs add with multiple files of same name (#8493) ([ipfs/go-ipfs#8493](https://github.com/ipfs/go-ipfs/pull/8493))\n - fix(gw): validate requested CAR version (#8835) ([ipfs/go-ipfs#8835](https://github.com/ipfs/go-ipfs/pull/8835))\n - feat: re-enable docker sharness tests (#8808) ([ipfs/go-ipfs#8808](https://github.com/ipfs/go-ipfs/pull/8808))\n - docs: gateway.md (#8825) ([ipfs/go-ipfs#8825](https://github.com/ipfs/go-ipfs/pull/8825))\n - fix(core/commands): do not cache config (#8824) ([ipfs/go-ipfs#8824](https://github.com/ipfs/go-ipfs/pull/8824))\n - remove unused import (#8787) ([ipfs/go-ipfs#8787](https://github.com/ipfs/go-ipfs/pull/8787))\n - feat(cmds): document deprecated RPC API commands (#8802) ([ipfs/go-ipfs#8802](https://github.com/ipfs/go-ipfs/pull/8802))\n - fix(fsrepo): deep merge when setting config ([ipfs/go-ipfs#8793](https://github.com/ipfs/go-ipfs/pull/8793))\n - feat: add gateway histogram metrics (#8443) ([ipfs/go-ipfs#8443](https://github.com/ipfs/go-ipfs/pull/8443))\n - ErrNotFound changes: bubble tagged libraries. ([ipfs/go-ipfs#8803](https://github.com/ipfs/go-ipfs/pull/8803))\n - Fix typos\n ([ipfs/go-ipfs#8757](https://github.com/ipfs/go-ipfs/pull/8757))\n - feat(gateway): Block and CAR response formats (#8758) ([ipfs/go-ipfs#8758](https://github.com/ipfs/go-ipfs/pull/8758))\n - fix: allow ipfs-companion browser extension to access RPC API (#8690) ([ipfs/go-ipfs#8690](https://github.com/ipfs/go-ipfs/pull/8690))\n - fix(core/node): unwrap fx error in node construction ([ipfs/go-ipfs#8638](https://github.com/ipfs/go-ipfs/pull/8638))\n - Update PATCH_RELEASE_TEMPLATE.md\n - feat: add full goroutine stack dump (#8790) ([ipfs/go-ipfs#8790](https://github.com/ipfs/go-ipfs/pull/8790))\n - feat(cmds): extend block size check for dag|block put (#8751) ([ipfs/go-ipfs#8751](https://github.com/ipfs/go-ipfs/pull/8751))\n - feat: add endpoint for enabling block profiling (#8469) ([ipfs/go-ipfs#8469](https://github.com/ipfs/go-ipfs/pull/8469))\n - fix(cmds): option for progress bar in cat/get (#8686) ([ipfs/go-ipfs#8686](https://github.com/ipfs/go-ipfs/pull/8686))\n - docs: note the default reprovider strategy as all (#8603) ([ipfs/go-ipfs#8603](https://github.com/ipfs/go-ipfs/pull/8603))\n - fix: listen on loopback for API and gateway ports in docker-compose.yaml (#8773) ([ipfs/go-ipfs#8773](https://github.com/ipfs/go-ipfs/pull/8773))\n - fix(discovery): fix daemon not starting due to mdns startup failure (#8704) ([ipfs/go-ipfs#8704](https://github.com/ipfs/go-ipfs/pull/8704))\n ([ipfs/go-ipfs#8756](https://github.com/ipfs/go-ipfs/pull/8756))\n - feat: ipfs-webui v2.15 (#8712) ([ipfs/go-ipfs#8712](https://github.com/ipfs/go-ipfs/pull/8712))\n - feat: X-Ipfs-Roots for smarter HTTP caches (#8720) ([ipfs/go-ipfs#8720](https://github.com/ipfs/go-ipfs/pull/8720))\n - chore: add instructions for Chocolatey release\n - fix prioritization of stream muxers ([ipfs/go-ipfs#8750](https://github.com/ipfs/go-ipfs/pull/8750))\n - fix(cmds/keystore): do not allow to import keys we don't generate (#8733) ([ipfs/go-ipfs#8733](https://github.com/ipfs/go-ipfs/pull/8733))\n - docs: add Internal.UnixFSShardingSizeThreshold (#8723) ([ipfs/go-ipfs#8723](https://github.com/ipfs/go-ipfs/pull/8723))\n - feat(cmd): add silent option for repo gc (#7147) ([ipfs/go-ipfs#7147](https://github.com/ipfs/go-ipfs/pull/7147))\n - docs(changelog): update v0.12.0 release notes\n - Merge branch 'release'\n - fix: installation without sudo (#8715) ([ipfs/go-ipfs#8715](https://github.com/ipfs/go-ipfs/pull/8715))\n - Fix typos (#8726) ([ipfs/go-ipfs#8726](https://github.com/ipfs/go-ipfs/pull/8726))\n - fix(build): Recognize Go Beta versions in makefile (#8677) ([ipfs/go-ipfs#8677](https://github.com/ipfs/go-ipfs/pull/8677))\n - chore(cmds): encapsulate ipfs rm logic in another function (#8574) ([ipfs/go-ipfs#8574](https://github.com/ipfs/go-ipfs/pull/8574))\n - feat: warn user when 'pin remote add' while offline (#8621) ([ipfs/go-ipfs#8621](https://github.com/ipfs/go-ipfs/pull/8621))\n - chore(gateway): debug logging for the http requests (#8518) ([ipfs/go-ipfs#8518](https://github.com/ipfs/go-ipfs/pull/8518))\n - docker: build for arm cpu (#8633) ([ipfs/go-ipfs#8633](https://github.com/ipfs/go-ipfs/pull/8633))\n - feat: refactor Fetcher interface used for downloading migrations (#8728) ([ipfs/go-ipfs#8728](https://github.com/ipfs/go-ipfs/pull/8728))\n - feat: log multifetcher errors\n - docs: optionalInteger|String|Duration (#8729) ([ipfs/go-ipfs#8729](https://github.com/ipfs/go-ipfs/pull/8729))\n - feat: DNS.MaxCacheTTL for DNS-over-HTTPS resolvers (#8615) ([ipfs/go-ipfs#8615](https://github.com/ipfs/go-ipfs/pull/8615))\n - feat(cmds): ipfs id: support --offline option (#8626) ([ipfs/go-ipfs#8626](https://github.com/ipfs/go-ipfs/pull/8626))\n - feat(cmds): add cleartext PEM/PKCS8 for key import/export (#8616) ([ipfs/go-ipfs#8616](https://github.com/ipfs/go-ipfs/pull/8616))\n - docs: update badger section in config.md (#8662) ([ipfs/go-ipfs#8662](https://github.com/ipfs/go-ipfs/pull/8662))\n - docs: fix typo\n - Adding PowerShell to Minimal Go Installation\n - Fixed typos in docs/config.md\n - docs: add Snap note about customizing IPFS_PATH (#8584) ([ipfs/go-ipfs#8584](https://github.com/ipfs/go-ipfs/pull/8584))\n - Fix typo ([ipfs/go-ipfs#8625](https://github.com/ipfs/go-ipfs/pull/8625))\n - chore: update version to v0.13.0-dev\n- github.com/ipfs/go-bitswap (v0.5.1 -> v0.6.0):\n - v0.6.0\n - Use ipld.ErrNotFound\n - feat: add peer block filter option (#549) ([ipfs/go-bitswap#549](https://github.com/ipfs/go-bitswap/pull/549))\n - configurable target message size ([ipfs/go-bitswap#546](https://github.com/ipfs/go-bitswap/pull/546))\n- github.com/ipfs/go-blockservice (v0.2.1 -> v0.3.0):\n - v0.3.0\n - s/log/logger\n - Use ipld.ErrNotFound instead of ErrNotFound\n- github.com/ipfs/go-cid (v0.1.0 -> v0.2.0):\n - fix: remove invalid multicodec2string mappings (#137) ([ipfs/go-cid#137](https://github.com/ipfs/go-cid/pull/137))\n - sync: update CI config files (#136) ([ipfs/go-cid#136](https://github.com/ipfs/go-cid/pull/136))\n - Benchmark existing ways to check for `IDENTITY` CIDs\n - avoid double alloc in NewCidV1\n - sync: update CI config files ([ipfs/go-cid#131](https://github.com/ipfs/go-cid/pull/131))\n- github.com/ipfs/go-cidutil (v0.0.2 -> v0.1.0):\n ([ipfs/go-cidutil#36](https://github.com/ipfs/go-cidutil/pull/36))\n - sync: update CI config files ([ipfs/go-cidutil#35](https://github.com/ipfs/go-cidutil/pull/35))\n - sync: update CI config files (#34) ([ipfs/go-cidutil#34](https://github.com/ipfs/go-cidutil/pull/34))\n - fix staticcheck ([ipfs/go-cidutil#31](https://github.com/ipfs/go-cidutil/pull/31))\n - add license file so it can be found by go-licenses ([ipfs/go-cidutil#27](https://github.com/ipfs/go-cidutil/pull/27))\n - test: fix for base32 switch ([ipfs/go-cidutil#16](https://github.com/ipfs/go-cidutil/pull/16))\n - doc: add a lead maintainer\n- github.com/ipfs/go-filestore (v1.1.0 -> v1.2.0):\n - v1.2.0\n - refactor: follow the happy left practice in Filestore.DeleteBlock\n - Use ipld.ErrNotFound\n- github.com/ipfs/go-graphsync (v0.11.0 -> v0.13.1):\n - docs(CHANGELOG): update for v0.13.1\n - feat(ipld): wrap bindnode with panic protection (#368) ([ipfs/go-graphsync#368](https://github.com/ipfs/go-graphsync/pull/368))\n - docs(CHANGELOG): update for v0.13.0 (#366) ([ipfs/go-graphsync#366](https://github.com/ipfs/go-graphsync/pull/366))\n - fix(impl): delete file\n - Minimal alternate metadata type support (#365) ([ipfs/go-graphsync#365](https://github.com/ipfs/go-graphsync/pull/365))\n - Fix unixfs fetch (#364) ([ipfs/go-graphsync#364](https://github.com/ipfs/go-graphsync/pull/364))\n - [Feature] UUIDs, protocol versioning, v2 protocol w/ dag-cbor messaging (#332) ([ipfs/go-graphsync#332](https://github.com/ipfs/go-graphsync/pull/332))\n - feat(CHANGELOG): update for v0.12.0\n - Use do not send blocks for pause/resume & prevent processing of blocks on canceled requests (#333) ([ipfs/go-graphsync#333](https://github.com/ipfs/go-graphsync/pull/333))\n - Support unixfs reification in default linksystem (#329) ([ipfs/go-graphsync#329](https://github.com/ipfs/go-graphsync/pull/329))\n - Don't run hooks on blocks we didn't have (#331) ([ipfs/go-graphsync#331](https://github.com/ipfs/go-graphsync/pull/331))\n - feat(responsemanager): trace full messages via links to responses (#325) ([ipfs/go-graphsync#325](https://github.com/ipfs/go-graphsync/pull/325))\n - chore(requestmanager): rename processResponses internals for consistency (#328) ([ipfs/go-graphsync#328](https://github.com/ipfs/go-graphsync/pull/328))\n - Response message tracing (#327) ([ipfs/go-graphsync#327](https://github.com/ipfs/go-graphsync/pull/327))\n - fix(testutil): fix tracing span collection (#324) ([ipfs/go-graphsync#324](https://github.com/ipfs/go-graphsync/pull/324))\n - docs(CHANGELOG): update for v0.11.5 release\n - feat(requestmanager): add tracing for response messages & block processing (#322) ([ipfs/go-graphsync#322](https://github.com/ipfs/go-graphsync/pull/322))\n - ipldutil: simplify state synchronization, add docs (#300) ([ipfs/go-graphsync#300](https://github.com/ipfs/go-graphsync/pull/300))\n - docs(CHANGELOG): update for v0.11.4 release\n - Scrub response errors (#320) ([ipfs/go-graphsync#320](https://github.com/ipfs/go-graphsync/pull/320))\n - fix(responsemanager): remove unused maxInProcessRequests parameter (#319) ([ipfs/go-graphsync#319](https://github.com/ipfs/go-graphsync/pull/319))\n - feat(responsemanager): allow ctx augmentation via queued request hook\n - make go test with coverpkg=./...\n - docs(CHANGELOG): update for v0.11.3\n - Merge tag 'v0.10.9'\n - feat: add basic tracing for responses (#291) ([ipfs/go-graphsync#291](https://github.com/ipfs/go-graphsync/pull/291))\n - fix(impl): remove accidental legacy field (#310) ([ipfs/go-graphsync#310](https://github.com/ipfs/go-graphsync/pull/310))\n - docs(CHANGELOG): update for v0.11.2\n - Merge branch 'release/v0.10.8'\n - feat(taskqueue): fix race on peer state gather (#303) ([ipfs/go-graphsync#303](https://github.com/ipfs/go-graphsync/pull/303))\n - feat(responsemanager): clarify response completion (#304) ([ipfs/go-graphsync#304](https://github.com/ipfs/go-graphsync/pull/304))\n - docs(CHANGELOG): update for v0.11.1\n - Merge branch 'release/v0.10.7'\n - Expose task queue diagnostics (#302) ([ipfs/go-graphsync#302](https://github.com/ipfs/go-graphsync/pull/302))\n - chore: short-circuit unnecessary message processing\n - Add a bit of logging (#301) ([ipfs/go-graphsync#301](https://github.com/ipfs/go-graphsync/pull/301))\n - Peer Stats function (#298) ([ipfs/go-graphsync#298](https://github.com/ipfs/go-graphsync/pull/298))\n - fix: use sync.Cond to handle no-task blocking wait (#299) ([ipfs/go-graphsync#299](https://github.com/ipfs/go-graphsync/pull/299))\n - ipldutil: use chooser APIs from dagpb and basicnode (#292) ([ipfs/go-graphsync#292](https://github.com/ipfs/go-graphsync/pull/292))\n - testutil/chaintypes: simplify maintenance of codegen (#294) ([ipfs/go-graphsync#294](https://github.com/ipfs/go-graphsync/pull/294))\n - fix(test): increase 1s timeouts to 2s for slow CI (#289) ([ipfs/go-graphsync#289](https://github.com/ipfs/go-graphsync/pull/289))\n - docs(tests): document tracing test helper utilities\n - feat: add basic OT tracing for incoming requests\n - fix(responsemanager): make fix more global\n - fix(responsemanager): fix flaky tests\n - feat: add WorkerTaskQueue#WaitForNoActiveTasks() for tests (#284) ([ipfs/go-graphsync#284](https://github.com/ipfs/go-graphsync/pull/284))\n- github.com/ipfs/go-ipfs-blockstore (v1.1.2 -> v1.2.0):\n - v0.2.0 ([ipfs/go-ipfs-blockstore#98](https://github.com/ipfs/go-ipfs-blockstore/pull/98))\n - s/log/logger\n - Use ipld.ErrNotFound for NotFound errors\n- github.com/ipfs/go-ipfs-cmds (v0.6.0 -> v0.8.1):\n - fix(cli/parse): extract dir before name ([ipfs/go-ipfs-cmds#230](https://github.com/ipfs/go-ipfs-cmds/pull/230))\n - Version 0.8.0 ([ipfs/go-ipfs-cmds#228](https://github.com/ipfs/go-ipfs-cmds/pull/228))\n - fix(cli): use NewSliceDirectory for duplicate file paths ([ipfs/go-ipfs-cmds#220](https://github.com/ipfs/go-ipfs-cmds/pull/220))\n - chore: release v0.7.0 ([ipfs/go-ipfs-cmds#227](https://github.com/ipfs/go-ipfs-cmds/pull/227))\n - feat(Command): add status for the helptext ([ipfs/go-ipfs-cmds#225](https://github.com/ipfs/go-ipfs-cmds/pull/225))\n - allow header and set header in client ([ipfs/go-ipfs-cmds#226](https://github.com/ipfs/go-ipfs-cmds/pull/226))\n - sync: update CI config files (#221) ([ipfs/go-ipfs-cmds#221](https://github.com/ipfs/go-ipfs-cmds/pull/221))\n - fix: chanResponseEmitter cancel being ineffective\n - add: tests for postrun execution\n - fix: postrun's run condition in Execute\n - fix: exec deadlock when emitter is not Typer intf\n - sync: update CI config files ([ipfs/go-ipfs-cmds#207](https://github.com/ipfs/go-ipfs-cmds/pull/207))\n - fix: preserve windows file paths ([ipfs/go-ipfs-cmds#214](https://github.com/ipfs/go-ipfs-cmds/pull/214))\n - Resolve `staticcheck` issue in prep for unified CI ([ipfs/go-ipfs-cmds#212](https://github.com/ipfs/go-ipfs-cmds/pull/212))\n- github.com/ipfs/go-ipfs-exchange-offline (v0.1.1 -> v0.2.0):\n - v0.2.0\n - Improve NotFound error description\n- github.com/ipfs/go-ipfs-files (v0.0.9 -> v0.1.1):\n - Release v0.1.1\n - fix: add dragonfly build option for filewriter flags\n - fix: add freebsd build option for filewriter flags\n - Release v0.1.0\n - docs: fix community CONTRIBUTING.md link (#45) ([ipfs/go-ipfs-files#45](https://github.com/ipfs/go-ipfs-files/pull/45))\n - chore(filewriter): cleanup writes (#43) ([ipfs/go-ipfs-files#43](https://github.com/ipfs/go-ipfs-files/pull/43))\n - sync: update CI config files (#44) ([ipfs/go-ipfs-files#44](https://github.com/ipfs/go-ipfs-files/pull/44))\n- github.com/ipfs/go-ipld-format (v0.2.0 -> v0.4.0):\n - chore: release version v0.4.0\n - feat: use new more clearer format in ErrNotFound\n - chore: bump version to 0.3.1\n - fix: make Undef ErrNotFound string consistent with Def version\n - Version 0.3.0\n - ErrNotFound: change error string ([ipfs/go-ipld-format#69](https://github.com/ipfs/go-ipld-format/pull/69))\n - Revert \"Revert \"Add IsErrNotFound() method\"\" ([ipfs/go-ipld-format#68](https://github.com/ipfs/go-ipld-format/pull/68))\n - sync: update CI config files (#67) ([ipfs/go-ipld-format#67](https://github.com/ipfs/go-ipld-format/pull/67))\n - ignore statticheck error for EndOfDag ([ipfs/go-ipld-format#62](https://github.com/ipfs/go-ipld-format/pull/62))\n - remove Makefile ([ipfs/go-ipld-format#59](https://github.com/ipfs/go-ipld-format/pull/59))\n - fix staticcheck ([ipfs/go-ipld-format#60](https://github.com/ipfs/go-ipld-format/pull/60))\n - Allowing custom NavigableNode implementations ([ipfs/go-ipld-format#58](https://github.com/ipfs/go-ipld-format/pull/58))\n- github.com/ipfs/go-ipld-legacy (v0.1.0 -> v0.1.1):\n - feat(node): add json.Marshaller method ([ipfs/go-ipld-legacy#7](https://github.com/ipfs/go-ipld-legacy/pull/7))\n- github.com/ipfs/go-log/v2 (v2.3.0 -> v2.5.1):\n - feat: add logger option to skip a number of stack frames ([ipfs/go-log#132](https://github.com/ipfs/go-log/pull/132))\n - release v2.5.0 (#131) ([ipfs/go-log#131](https://github.com/ipfs/go-log/pull/131))\n - config inspection (#129) ([ipfs/go-log#129](https://github.com/ipfs/go-log/pull/129))\n - release v2.4.0 (#127) ([ipfs/go-log#127](https://github.com/ipfs/go-log/pull/127))\n - sync: update CI config files (#125) ([ipfs/go-log#125](https://github.com/ipfs/go-log/pull/125))\n - fix: cannot call SetPrimaryCore after using a Tee logger ([ipfs/go-log#121](https://github.com/ipfs/go-log/pull/121))\n - Document environment variables ([ipfs/go-log#120](https://github.com/ipfs/go-log/pull/120))\n - sync: update CI config files (#119) ([ipfs/go-log#119](https://github.com/ipfs/go-log/pull/119))\n - Add WithStacktrace utility ([ipfs/go-log#118](https://github.com/ipfs/go-log/pull/118))\n - In addition to StdOut/Err check the outfile for TTYness ([ipfs/go-log#117](https://github.com/ipfs/go-log/pull/117))\n- github.com/ipfs/go-merkledag (v0.5.1 -> v0.6.0):\n - v0.6.0\n - Improve ErrNotFound\n- github.com/ipfs/go-namesys (v0.4.0 -> v0.5.0):\n - Version 0.5.0\n - fix: CIDv1 error with go-libp2p 0.19 (#32) ([ipfs/go-namesys#32](https://github.com/ipfs/go-namesys/pull/32))\n - feat: add tracing (#30) ([ipfs/go-namesys#30](https://github.com/ipfs/go-namesys/pull/30))\n - fix(publisher): fix garbled code output (#28) ([ipfs/go-namesys#28](https://github.com/ipfs/go-namesys/pull/28))\n- github.com/ipfs/go-path (v0.2.1 -> v0.3.0):\n - Release v0.3.0 ([ipfs/go-path#55](https://github.com/ipfs/go-path/pull/55))\n - Resolver: convert to interface. ([ipfs/go-path#53](https://github.com/ipfs/go-path/pull/53))\n - Release v0.2.2 (#52) ([ipfs/go-path#52](https://github.com/ipfs/go-path/pull/52))\n - chore: improve error message for invalid ipfs paths ([ipfs/go-path#51](https://github.com/ipfs/go-path/pull/51))\n- github.com/ipfs/go-peertaskqueue (v0.7.0 -> v0.7.1):\n - Add topic inspector ([ipfs/go-peertaskqueue#20](https://github.com/ipfs/go-peertaskqueue/pull/20))\n- github.com/ipfs/go-pinning-service-http-client (v0.1.0 -> v0.1.1):\n - chore: release v0.1.1\n - fix: error handling while enumerating pins\n - sync: update CI config files (#15) ([ipfs/go-pinning-service-http-client#15](https://github.com/ipfs/go-pinning-service-http-client/pull/15))\n - Resolve lint issues prior to CI integration\n- github.com/ipfs/go-unixfsnode (v1.1.3 -> v1.4.0):\n - 1.4.0 release ([ipfs/go-unixfsnode#29](https://github.com/ipfs/go-unixfsnode/pull/29))\n - Partial file test ([ipfs/go-unixfsnode#26](https://github.com/ipfs/go-unixfsnode/pull/26))\n - Add unixfs to UnixFS path selector tail ([ipfs/go-unixfsnode#28](https://github.com/ipfs/go-unixfsnode/pull/28))\n - release v1.3.0 ([ipfs/go-unixfsnode#25](https://github.com/ipfs/go-unixfsnode/pull/25))\n - add AsLargeBytes support to unixfs files (#24) ([ipfs/go-unixfsnode#24](https://github.com/ipfs/go-unixfsnode/pull/24))\n - fix: add extra test to span the shard/no-shard boundary\n - fix: more Tsize fixes, fix HAMT and make it match go-unixfs output\n - fix: encode Tsize correctly everywhere (using wrapped LinkSystem)\n - docs(version): tag 1.2.0\n - Update deps for ADL selectors ([ipfs/go-unixfsnode#20](https://github.com/ipfs/go-unixfsnode/pull/20))\n - add license (#17) ([ipfs/go-unixfsnode#17](https://github.com/ipfs/go-unixfsnode/pull/17))\n - handle empty files (#15) ([ipfs/go-unixfsnode#15](https://github.com/ipfs/go-unixfsnode/pull/15))\n - Add ADL/single-node-view of a full unixFS file. (#14) ([ipfs/go-unixfsnode#14](https://github.com/ipfs/go-unixfsnode/pull/14))\n - sync: update CI config files (#13) ([ipfs/go-unixfsnode#13](https://github.com/ipfs/go-unixfsnode/pull/13))\n - Add builder for unixfs dags (#12) ([ipfs/go-unixfsnode#12](https://github.com/ipfs/go-unixfsnode/pull/12))\n- github.com/ipfs/interface-go-ipfs-core (v0.5.2 -> v0.7.0):\n - refactor(block): CIDv1 and BlockPutSettings CidPrefix (#80) ([ipfs/interface-go-ipfs-core#80](https://github.com/ipfs/interface-go-ipfs-core/pull/80))\n - chore: release v0.6.2\n - fix: use IPLD.ErrNotFound instead of string comparison in tests\n - fix: document error (#74) ([ipfs/interface-go-ipfs-core#74](https://github.com/ipfs/interface-go-ipfs-core/pull/74))\n - version: release 0.6.1\n - v0.6.0\n - Update tests to use ipld.IsNotFound to check for notfound errors\n - sync: update CI config files (#79) ([ipfs/interface-go-ipfs-core#79](https://github.com/ipfs/interface-go-ipfs-core/pull/79))\n- github.com/ipld/go-codec-dagpb (v1.3.2 -> v1.4.0):\n - bump to v1.4.0 given that we updated ipld-prime\n - add a decode-then-encode roundtrip fuzzer\n - 1.3.1\n - fix: use protowire for Links bytes decoding\n - delete useless code\n - sync: update CI config files (#33) ([ipld/go-codec-dagpb#33](https://github.com/ipld/go-codec-dagpb/pull/33))\n- github.com/ipld/go-ipld-prime (v0.14.2 -> v0.16.0):\n - mark v0.16.0\n - node/bindnode: enforce pointer requirement for nullable maps\n - Implement WalkTransforming traversal (#376) ([ipld/go-ipld-prime#376](https://github.com/ipld/go-ipld-prime/pull/376))\n - docs(datamodel): add comment to LargeBytesNode\n - Add partial-match traversal of large bytes (#375) ([ipld/go-ipld-prime#375](https://github.com/ipld/go-ipld-prime/pull/375))\n - Implement option to start traversals at a path ([ipld/go-ipld-prime#358](https://github.com/ipld/go-ipld-prime/pull/358))\n - add top-level \"go value with schema\" example\n - Support optional `LargeBytesNode` interface (#372) ([ipld/go-ipld-prime#372](https://github.com/ipld/go-ipld-prime/pull/372))\n - node/bindnode: support pointers to datamodel.Node to bind with Any\n - fix(bindnode): tuple struct iterator should handle absent fields properly\n - node/bindnode: make AssignNode work at the repr level\n - node/bindnode: add support for unsigned integers\n - node/bindnode: cover even more edge case panics\n - node/bindnode: polish some more AsT panics\n - schema/dmt: stop using a fake test to generate code ([ipld/go-ipld-prime#356](https://github.com/ipld/go-ipld-prime/pull/356))\n - schema: remove one review note; add another.\n - fix: minor EncodedLength fixes, add tests to fully exercise\n - feat: add dagcbor.EncodedLength(Node) to calculate length without encoding\n - chore: rename Garbage() to Generate()\n - fix: minor garbage nits\n - fix: Garbage() takes rand parameter, tweak algorithms, improve docs\n - feat: add Garbage() Node generator\n - node/bindnode: introduce an assembler that always errors\n - node/bindnode: polish panics on invalid AssignT calls\n - datamodel: don't panic when stringifying an empty KindSet\n - node/bindnode: start using ipld.LoadSchema APIs\n - selectors: fix for edge case around recursion clauses with an immediate edge. ([ipld/go-ipld-prime#334](https://github.com/ipld/go-ipld-prime/pull/334))\n - node/bindnode: improve support for pointer types\n - node/bindnode: subtract all absents in Length at the repr level\n - fix(codecs): error when encoding maps whose lengths don't match entry count\n - schema: avoid alloc and copy in Struct and Enum methods\n - node/bindnode: allow mapping int-repr enums with Go integers\n - schema,node/bindnode: add support for Any\n - signaling ADLs in selectors (#301) ([ipld/go-ipld-prime#301](https://github.com/ipld/go-ipld-prime/pull/301))\n - node/bindnode: add support for enums\n - schema/...: add support for enum int representations\n - node/bindnode: allow binding cidlink.Link to links\n - Update to context datastores (#312) ([ipld/go-ipld-prime#312](https://github.com/ipld/go-ipld-prime/pull/312))\n - schema: add support for struct tuple reprs\n - Allow parsing padding in dag-json bytes fields (#309) ([ipld/go-ipld-prime#309](https://github.com/ipld/go-ipld-prime/pull/309))\n- github.com/libp2p/go-doh-resolver (v0.3.1 -> v0.4.0):\n - Release v0.4.0 (#16) ([libp2p/go-doh-resolver#16](https://github.com/libp2p/go-doh-resolver/pull/16))\n - sync: update CI config files (#14) ([libp2p/go-doh-resolver#14](https://github.com/libp2p/go-doh-resolver/pull/14))\n - Add a max TTL for cached entries ([libp2p/go-doh-resolver#12](https://github.com/libp2p/go-doh-resolver/pull/12))\n - Perform test locally instead of using a live dns resolution ([libp2p/go-doh-resolver#13](https://github.com/libp2p/go-doh-resolver/pull/13))\n - sync: update CI config files (#7) ([libp2p/go-doh-resolver#7](https://github.com/libp2p/go-doh-resolver/pull/7))\n - fix staticcheck ([libp2p/go-doh-resolver#6](https://github.com/libp2p/go-doh-resolver/pull/6))\n- github.com/libp2p/go-libp2p (v0.16.0 -> v0.19.4):\n - update go-yamux to v3.1.2, release v0.19.4 (#1590) ([libp2p/go-libp2p#1590](https://github.com/libp2p/go-libp2p/pull/1590))\n - update quic-go to v0.27.1, release v0.19.3 (#1518) ([libp2p/go-libp2p#1518](https://github.com/libp2p/go-libp2p/pull/1518))\n - release v0.19.2\n - holepunch: fix incorrect message type for the SYNC message (#1478) ([libp2p/go-libp2p#1478](https://github.com/libp2p/go-libp2p/pull/1478))\n - fix race condition in holepunch service, release v0.19.1 ([libp2p/go-libp2p#1474](https://github.com/libp2p/go-libp2p/pull/1474))\n - release v0.19.0 (#1408) ([libp2p/go-libp2p#1408](https://github.com/libp2p/go-libp2p/pull/1408))\n - Close resource manager when host closes (#1343) ([libp2p/go-libp2p#1343](https://github.com/libp2p/go-libp2p/pull/1343))\n - fix flaky reconnect test (#1406) ([libp2p/go-libp2p#1406](https://github.com/libp2p/go-libp2p/pull/1406))\n - make sure to not oversubscribe to relays (#1404) ([libp2p/go-libp2p#1404](https://github.com/libp2p/go-libp2p/pull/1404))\n - rewrite the reconnect test (#1399) ([libp2p/go-libp2p#1399](https://github.com/libp2p/go-libp2p/pull/1399))\n - don't try to reconnect to already connected relays (#1401) ([libp2p/go-libp2p#1401](https://github.com/libp2p/go-libp2p/pull/1401))\n - reduce flakiness of AutoRelay TestBackoff test (#1400) ([libp2p/go-libp2p#1400](https://github.com/libp2p/go-libp2p/pull/1400))\n - improve AutoRelay v1 handling ([libp2p/go-libp2p#1396](https://github.com/libp2p/go-libp2p/pull/1396))\n - remove note about gx from README (#1385) ([libp2p/go-libp2p#1385](https://github.com/libp2p/go-libp2p/pull/1385))\n - use the vcs information from ReadBuildInfo in Go 1.18 ([libp2p/go-libp2p#1381](https://github.com/libp2p/go-libp2p/pull/1381))\n - fix race condition in AutoRelay candidate handling (#1383) ([libp2p/go-libp2p#1383](https://github.com/libp2p/go-libp2p/pull/1383))\n - implement relay v2 discovery ([libp2p/go-libp2p#1368](https://github.com/libp2p/go-libp2p/pull/1368))\n - fix go vet error in proxy example (#1377) ([libp2p/go-libp2p#1377](https://github.com/libp2p/go-libp2p/pull/1377))\n - Resolve addresses when creating a new stream (#1342) ([libp2p/go-libp2p#1342](https://github.com/libp2p/go-libp2p/pull/1342))\n - remove mplex from the list of default muxers (#1344) ([libp2p/go-libp2p#1344](https://github.com/libp2p/go-libp2p/pull/1344))\n - refactor the holepunching code ([libp2p/go-libp2p#1355](https://github.com/libp2p/go-libp2p/pull/1355))\n - speed up the connmgr tests (#1354) ([libp2p/go-libp2p#1354](https://github.com/libp2p/go-libp2p/pull/1354))\n - update go-libp2p-resource manager, release v0.18.0 (#1361) ([libp2p/go-libp2p#1361](https://github.com/libp2p/go-libp2p/pull/1361))\n - fix flaky BackoffConnector test (#1353) ([libp2p/go-libp2p#1353](https://github.com/libp2p/go-libp2p/pull/1353))\n - release v0.18.0-rc6 (#1350) ([libp2p/go-libp2p#1350](https://github.com/libp2p/go-libp2p/pull/1350))\n - release v0.18.0-rc5 ([libp2p/go-libp2p#1341](https://github.com/libp2p/go-libp2p/pull/1341))\n - update README (#1330) ([libp2p/go-libp2p#1330](https://github.com/libp2p/go-libp2p/pull/1330))\n - fix parsing of IP addresses for zeroconf initialization (#1338) ([libp2p/go-libp2p#1338](https://github.com/libp2p/go-libp2p/pull/1338))\n - fix flaky TestBackoffConnector test (#1328) ([libp2p/go-libp2p#1328](https://github.com/libp2p/go-libp2p/pull/1328))\n - release v0.18.0-rc4 ([libp2p/go-libp2p#1327](https://github.com/libp2p/go-libp2p/pull/1327))\n - fix (and speed up) flaky TestBackoffConnector test (#1316) ([libp2p/go-libp2p#1316](https://github.com/libp2p/go-libp2p/pull/1316))\n - fix flaky TestAutoRelay test (#1322) ([libp2p/go-libp2p#1322](https://github.com/libp2p/go-libp2p/pull/1322))\n - deflake resource manager tests, take 2 ([libp2p/go-libp2p#1318](https://github.com/libp2p/go-libp2p/pull/1318))\n - fix race condition causing TestAutoNATServiceDialError test failure (#1312) ([libp2p/go-libp2p#1312](https://github.com/libp2p/go-libp2p/pull/1312))\n - disable flaky relay example test on CI (#1219) ([libp2p/go-libp2p#1219](https://github.com/libp2p/go-libp2p/pull/1219))\n - fix flaky resource manager tests ([libp2p/go-libp2p#1315](https://github.com/libp2p/go-libp2p/pull/1315))\n - update deps, fixing nil peer scope pointer issues in connection upgrading ([libp2p/go-libp2p#1309](https://github.com/libp2p/go-libp2p/pull/1309))\n - release v0.18.0-rc2 ([libp2p/go-libp2p#1306](https://github.com/libp2p/go-libp2p/pull/1306))\n - add semaphore to control push/delta concurrency ([libp2p/go-libp2p#1305](https://github.com/libp2p/go-libp2p/pull/1305))\n - release v0.18.0-rc1 ([libp2p/go-libp2p#1300](https://github.com/libp2p/go-libp2p/pull/1300))\n - default connection manager ([libp2p/go-libp2p#1299](https://github.com/libp2p/go-libp2p/pull/1299))\n - Basic resource manager integration tests ([libp2p/go-libp2p#1296](https://github.com/libp2p/go-libp2p/pull/1296))\n - use the resource manager ([libp2p/go-libp2p#1275](https://github.com/libp2p/go-libp2p/pull/1275))\n - move the go-libp2p-connmgr here ([libp2p/go-libp2p#1297](https://github.com/libp2p/go-libp2p/pull/1297))\n - move go-libp2p-discovery here ([libp2p/go-libp2p#1291](https://github.com/libp2p/go-libp2p/pull/1291))\n - speed up identify tests ([libp2p/go-libp2p#1294](https://github.com/libp2p/go-libp2p/pull/1294))\n - don't close the connection when opening the identify stream fails ([libp2p/go-libp2p#1293](https://github.com/libp2p/go-libp2p/pull/1293))\n - use the netutil package that was moved to go-libp2p-testing (#1263) ([libp2p/go-libp2p#1263](https://github.com/libp2p/go-libp2p/pull/1263))\n - speed up the autorelay test, fix flaky TestAutoRelay test ([libp2p/go-libp2p#1272](https://github.com/libp2p/go-libp2p/pull/1272))\n - fix flaky TestStreamsStress test (#1288) ([libp2p/go-libp2p#1288](https://github.com/libp2p/go-libp2p/pull/1288))\n - add an option for the swarm dial timeout ([libp2p/go-libp2p#1271](https://github.com/libp2p/go-libp2p/pull/1271))\n - use the transport.Upgrader interface ([libp2p/go-libp2p#1277](https://github.com/libp2p/go-libp2p/pull/1277))\n - fix typo in options.go (#1274) ([libp2p/go-libp2p#1274](https://github.com/libp2p/go-libp2p/pull/1274))\n - remove direct dependency on libp2p/go-addr-util ([libp2p/go-libp2p#1279](https://github.com/libp2p/go-libp2p/pull/1279))\n - fix flaky TestNotifications test ([libp2p/go-libp2p#1278](https://github.com/libp2p/go-libp2p/pull/1278))\n - move go-libp2p-autonat to p2p/host/autonat ([libp2p/go-libp2p#1273](https://github.com/libp2p/go-libp2p/pull/1273))\n - require the expiration field of the circuit v2 Reservation protobuf ([libp2p/go-libp2p#1269](https://github.com/libp2p/go-libp2p/pull/1269))\n - run reconnect test using QUIC ([libp2p/go-libp2p#1268](https://github.com/libp2p/go-libp2p/pull/1268))\n - remove goprocess from the mock package ([libp2p/go-libp2p#1266](https://github.com/libp2p/go-libp2p/pull/1266))\n - release v0.17.0 (#1265) ([libp2p/go-libp2p#1265](https://github.com/libp2p/go-libp2p/pull/1265))\n - use the new network.ConnStats ([libp2p/go-libp2p#1262](https://github.com/libp2p/go-libp2p/pull/1262))\n - move the peerstoremanager to the host ([libp2p/go-libp2p#1258](https://github.com/libp2p/go-libp2p/pull/1258))\n - reduce the default stream protocol negotiation timeout (#1254) ([libp2p/go-libp2p#1254](https://github.com/libp2p/go-libp2p/pull/1254))\n - Doc: QUIC is default when no transports set (#1250) ([libp2p/go-libp2p#1250](https://github.com/libp2p/go-libp2p/pull/1250))\n - exclude web3-bot from mkreleaselog ([libp2p/go-libp2p#1248](https://github.com/libp2p/go-libp2p/pull/1248))\n - identify: also match observed against listening addresses ([libp2p/go-libp2p#1255](https://github.com/libp2p/go-libp2p/pull/1255))\n - make it possible to run the auto relays tests multiple times ([libp2p/go-libp2p#1253](https://github.com/libp2p/go-libp2p/pull/1253))\n- github.com/libp2p/go-libp2p-asn-util (v0.1.0 -> v0.2.0):\n - Release 0.2.0 (#21) ([libp2p/go-libp2p-asn-util#21](https://github.com/libp2p/go-libp2p-asn-util/pull/21))\n - perf: replace the ipv6 map by an array of struct (#20) ([libp2p/go-libp2p-asn-util#20](https://github.com/libp2p/go-libp2p-asn-util/pull/20))\n - sync: update CI config files (#18) ([libp2p/go-libp2p-asn-util#18](https://github.com/libp2p/go-libp2p-asn-util/pull/18))\n- github.com/libp2p/go-libp2p-blankhost (v0.2.0 -> v0.3.0):\n - add a WithEventBus constructor option ([libp2p/go-libp2p-blankhost#61](https://github.com/libp2p/go-libp2p-blankhost/pull/61))\n - emit the EvtPeerConnectednessChanged event ([libp2p/go-libp2p-blankhost#58](https://github.com/libp2p/go-libp2p-blankhost/pull/58))\n - chore: update go-log to v2 ([libp2p/go-libp2p-blankhost#59](https://github.com/libp2p/go-libp2p-blankhost/pull/59))\n - Remove invalid links ([libp2p/go-libp2p-blankhost#57](https://github.com/libp2p/go-libp2p-blankhost/pull/57))\n - fix go vet ([libp2p/go-libp2p-blankhost#53](https://github.com/libp2p/go-libp2p-blankhost/pull/53))\n- github.com/libp2p/go-libp2p-circuit (v0.4.0 -> v0.6.0):\n - release v0.6.0 (#151) ([libp2p/go-libp2p-circuit#151](https://github.com/libp2p/go-libp2p-circuit/pull/151))\n - chore: update go-log to v2 (#147) ([libp2p/go-libp2p-circuit#147](https://github.com/libp2p/go-libp2p-circuit/pull/147))\n - release v0.5.0 (#150) ([libp2p/go-libp2p-circuit#150](https://github.com/libp2p/go-libp2p-circuit/pull/150))\n - use the resource manager ([libp2p/go-libp2p-circuit#148](https://github.com/libp2p/go-libp2p-circuit/pull/148))\n - use the transport.Upgrader interface ([libp2p/go-libp2p-circuit#149](https://github.com/libp2p/go-libp2p-circuit/pull/149))\n - sync: update CI config files (#144) ([libp2p/go-libp2p-circuit#144](https://github.com/libp2p/go-libp2p-circuit/pull/144))\n - ([libp2p/go-libp2p-circuit#143](https://github.com/libp2p/go-libp2p-circuit/pull/143))\n - add a Close method, remove the context from the constructor ([libp2p/go-libp2p-circuit#141](https://github.com/libp2p/go-libp2p-circuit/pull/141))\n - chore: update go-libp2p-core, go-libp2p-swarm ([libp2p/go-libp2p-circuit#140](https://github.com/libp2p/go-libp2p-circuit/pull/140))\n - remove the circuit v2 code ([libp2p/go-libp2p-circuit#139](https://github.com/libp2p/go-libp2p-circuit/pull/139))\n - implement circuit v2 ([libp2p/go-libp2p-circuit#136](https://github.com/libp2p/go-libp2p-circuit/pull/136))\n - remove deprecated types ([libp2p/go-libp2p-circuit#135](https://github.com/libp2p/go-libp2p-circuit/pull/135))\n - fix race condition in TestActiveRelay ([libp2p/go-libp2p-circuit#133](https://github.com/libp2p/go-libp2p-circuit/pull/133))\n - minor staticcheck fixes ([libp2p/go-libp2p-circuit#126](https://github.com/libp2p/go-libp2p-circuit/pull/126))\n - Timeout Stream Read ([libp2p/go-libp2p-circuit#124](https://github.com/libp2p/go-libp2p-circuit/pull/124))\n- github.com/libp2p/go-libp2p-core (v0.11.0 -> v0.15.1):\n - release v0.15.1 (#246) ([libp2p/go-libp2p-core#246](https://github.com/libp2p/go-libp2p-core/pull/246))\n - feat: harden encoding/decoding functions against panics (#243) ([libp2p/go-libp2p-core#243](https://github.com/libp2p/go-libp2p-core/pull/243))\n - release v0.15.0 (#242) ([libp2p/go-libp2p-core#242](https://github.com/libp2p/go-libp2p-core/pull/242))\n - sync: update CI config files (#241) ([libp2p/go-libp2p-core#241](https://github.com/libp2p/go-libp2p-core/pull/241))\n - fix: switch to go-multicodec mappings (#240) ([libp2p/go-libp2p-core#240](https://github.com/libp2p/go-libp2p-core/pull/240))\n - chore: add `String()` method to `IDSlice` type (#238) ([libp2p/go-libp2p-core#238](https://github.com/libp2p/go-libp2p-core/pull/238))\n - release v0.14.0 (#235) ([libp2p/go-libp2p-core#235](https://github.com/libp2p/go-libp2p-core/pull/235))\n - Network Resource Manager interface (#229) ([libp2p/go-libp2p-core#229](https://github.com/libp2p/go-libp2p-core/pull/229))\n - introduce a transport.Upgrader interface (#232) ([libp2p/go-libp2p-core#232](https://github.com/libp2p/go-libp2p-core/pull/232))\n - remove the transport.AcceptTimeout (#231) ([libp2p/go-libp2p-core#231](https://github.com/libp2p/go-libp2p-core/pull/231))\n - remove the DialTimeout (#230) ([libp2p/go-libp2p-core#230](https://github.com/libp2p/go-libp2p-core/pull/230))\n - remove duplicate io.Closer on Network interface (#228) ([libp2p/go-libp2p-core#228](https://github.com/libp2p/go-libp2p-core/pull/228))\n - release v0.13.0 (#227) ([libp2p/go-libp2p-core#227](https://github.com/libp2p/go-libp2p-core/pull/227))\n - rename network.Stat to Stats, introduce ConnStats (#226) ([libp2p/go-libp2p-core#226](https://github.com/libp2p/go-libp2p-core/pull/226))\n - release v0.12.0 (#223) ([libp2p/go-libp2p-core#223](https://github.com/libp2p/go-libp2p-core/pull/223))\n - generate ecdsa public key from an input public key (#219) ([libp2p/go-libp2p-core#219](https://github.com/libp2p/go-libp2p-core/pull/219))\n - add RemovePeer method to PeerMetadata, Metrics, ProtoBook and Keybook (#218) ([libp2p/go-libp2p-core#218](https://github.com/libp2p/go-libp2p-core/pull/218))\n- github.com/libp2p/go-libp2p-kad-dht (v0.15.0 -> v0.16.0):\n - Version 0.16.0 (#774) ([libp2p/go-libp2p-kad-dht#774](https://github.com/libp2p/go-libp2p-kad-dht/pull/774))\n - feat: add error log when resource manager throttles crawler (#772) ([libp2p/go-libp2p-kad-dht#772](https://github.com/libp2p/go-libp2p-kad-dht/pull/772))\n - fix: incorrect format handling ([libp2p/go-libp2p-kad-dht#771](https://github.com/libp2p/go-libp2p-kad-dht/pull/771))\n - Upgrade to go-libp2p v0.16.0 (#756) ([libp2p/go-libp2p-kad-dht#756](https://github.com/libp2p/go-libp2p-kad-dht/pull/756))\n - sync: update CI config files ([libp2p/go-libp2p-kad-dht#758](https://github.com/libp2p/go-libp2p-kad-dht/pull/758))\n- github.com/libp2p/go-libp2p-mplex (v0.4.1 -> v0.7.0):\n - release v0.7.0 (#36) ([libp2p/go-libp2p-mplex#36](https://github.com/libp2p/go-libp2p-mplex/pull/36))\n - release v0.6.0 (#32) ([libp2p/go-libp2p-mplex#32](https://github.com/libp2p/go-libp2p-mplex/pull/32))\n - update mplex (#31) ([libp2p/go-libp2p-mplex#31](https://github.com/libp2p/go-libp2p-mplex/pull/31))\n - release v0.5.0 (#30) ([libp2p/go-libp2p-mplex#30](https://github.com/libp2p/go-libp2p-mplex/pull/30))\n - implement the new network.MuxedConn interface (#29) ([libp2p/go-libp2p-mplex#29](https://github.com/libp2p/go-libp2p-mplex/pull/29))\n - sync: update CI config files (#28) ([libp2p/go-libp2p-mplex#28](https://github.com/libp2p/go-libp2p-mplex/pull/28))\n - remove Makefile ([libp2p/go-libp2p-mplex#25](https://github.com/libp2p/go-libp2p-mplex/pull/25))\n- github.com/libp2p/go-libp2p-noise (v0.3.0 -> v0.4.0):\n - release v0.4.0 (#112) ([libp2p/go-libp2p-noise#112](https://github.com/libp2p/go-libp2p-noise/pull/112))\n - catch panics during the handshake (#111) ([libp2p/go-libp2p-noise#111](https://github.com/libp2p/go-libp2p-noise/pull/111))\n - sync: update CI config files (#106) ([libp2p/go-libp2p-noise#106](https://github.com/libp2p/go-libp2p-noise/pull/106))\n - update README to reflect that Noise is enabled by default ([libp2p/go-libp2p-noise#101](https://github.com/libp2p/go-libp2p-noise/pull/101))\n- github.com/libp2p/go-libp2p-peerstore (v0.4.0 -> v0.6.0):\n - release v0.6.0 ([libp2p/go-libp2p-peerstore#189](https://github.com/libp2p/go-libp2p-peerstore/pull/189))\n - remove the pstoremanager (will be moved to the Host) ([libp2p/go-libp2p-peerstore#188](https://github.com/libp2p/go-libp2p-peerstore/pull/188))\n - release v0.5.0 (#187) ([libp2p/go-libp2p-peerstore#187](https://github.com/libp2p/go-libp2p-peerstore/pull/187))\n - remove metadata interning ([libp2p/go-libp2p-peerstore#185](https://github.com/libp2p/go-libp2p-peerstore/pull/185))\n - when passed an event bus, automatically clean up disconnected peers ([libp2p/go-libp2p-peerstore#184](https://github.com/libp2p/go-libp2p-peerstore/pull/184))\n - implement the RemovePeer method ([libp2p/go-libp2p-peerstore#174](https://github.com/libp2p/go-libp2p-peerstore/pull/174))\n - chore: update go-log to v2 (#179) ([libp2p/go-libp2p-peerstore#179](https://github.com/libp2p/go-libp2p-peerstore/pull/179))\n- github.com/libp2p/go-libp2p-pubsub (v0.6.0 -> v0.6.1):\n - add tests for clearing the peerPromises map\n - properly clear the peerPromises map\n - more info\n - add to MinTopicSize godoc re topic size\n- github.com/libp2p/go-libp2p-quic-transport (v0.15.0 -> v0.17.0):\n - release v0.17.0 (#269) ([libp2p/go-libp2p-quic-transport#269](https://github.com/libp2p/go-libp2p-quic-transport/pull/269))\n - update quic-go to v0.27.0 (#264) ([libp2p/go-libp2p-quic-transport#264](https://github.com/libp2p/go-libp2p-quic-transport/pull/264))\n - release v0.16.1 (#261) ([libp2p/go-libp2p-quic-transport#261](https://github.com/libp2p/go-libp2p-quic-transport/pull/261))\n - Prevent data race in allowWindowIncrease (#259) ([libp2p/go-libp2p-quic-transport#259](https://github.com/libp2p/go-libp2p-quic-transport/pull/259))\n - release v0.16.0 (#258) ([libp2p/go-libp2p-quic-transport#258](https://github.com/libp2p/go-libp2p-quic-transport/pull/258))\n - use the Resource Manager ([libp2p/go-libp2p-quic-transport#249](https://github.com/libp2p/go-libp2p-quic-transport/pull/249))\n - don't start a Go routine for every connection dialed ([libp2p/go-libp2p-quic-transport#252](https://github.com/libp2p/go-libp2p-quic-transport/pull/252))\n - migrate to standard Go tests, stop using Ginkgo ([libp2p/go-libp2p-quic-transport#250](https://github.com/libp2p/go-libp2p-quic-transport/pull/250))\n - chore: remove Codecov config (#251) ([libp2p/go-libp2p-quic-transport#251](https://github.com/libp2p/go-libp2p-quic-transport/pull/251))\n - reduce the maximum number of incoming streams to 256 (#243) ([libp2p/go-libp2p-quic-transport#243](https://github.com/libp2p/go-libp2p-quic-transport/pull/243))\n - chore: update go-log to v2 (#242) ([libp2p/go-libp2p-quic-transport#242](https://github.com/libp2p/go-libp2p-quic-transport/pull/242))\n- github.com/libp2p/go-libp2p-swarm (v0.8.0 -> v0.10.2):\n - bump version to v0.10.2 ([libp2p/go-libp2p-swarm#316](https://github.com/libp2p/go-libp2p-swarm/pull/316))\n - Refactor dial worker loop into an object and fix bug ([libp2p/go-libp2p-swarm#315](https://github.com/libp2p/go-libp2p-swarm/pull/315))\n - release v0.10.1 ([libp2p/go-libp2p-swarm#313](https://github.com/libp2p/go-libp2p-swarm/pull/313))\n - release the stream scope if the conn fails to open a new stream ([libp2p/go-libp2p-swarm#312](https://github.com/libp2p/go-libp2p-swarm/pull/312))\n - release v0.10.0 (#311) ([libp2p/go-libp2p-swarm#311](https://github.com/libp2p/go-libp2p-swarm/pull/311))\n - add support for the resource manager ([libp2p/go-libp2p-swarm#308](https://github.com/libp2p/go-libp2p-swarm/pull/308))\n - use the transport.Upgrader interface ([libp2p/go-libp2p-swarm#309](https://github.com/libp2p/go-libp2p-swarm/pull/309))\n - remove dependency on go-addr-util ([libp2p/go-libp2p-swarm#300](https://github.com/libp2p/go-libp2p-swarm/pull/300))\n - stop using transport.DialTimeout in tests (#307) ([libp2p/go-libp2p-swarm#307](https://github.com/libp2p/go-libp2p-swarm/pull/307))\n - increment active dial counter in dial worker loop ([libp2p/go-libp2p-swarm#305](https://github.com/libp2p/go-libp2p-swarm/pull/305))\n - speed up the dial tests ([libp2p/go-libp2p-swarm#301](https://github.com/libp2p/go-libp2p-swarm/pull/301))\n - stop using the deprecated libp2p/go-maddr-filter ([libp2p/go-libp2p-swarm#303](https://github.com/libp2p/go-libp2p-swarm/pull/303))\n - add constructor options for timeout, stop using transport.DialTimeout ([libp2p/go-libp2p-swarm#302](https://github.com/libp2p/go-libp2p-swarm/pull/302))\n - release v0.9.0 (#299) ([libp2p/go-libp2p-swarm#299](https://github.com/libp2p/go-libp2p-swarm/pull/299))\n - count the number of streams on a connection for the stats ([libp2p/go-libp2p-swarm#298](https://github.com/libp2p/go-libp2p-swarm/pull/298))\n - chore: update go-log to v2 (#294) ([libp2p/go-libp2p-swarm#294](https://github.com/libp2p/go-libp2p-swarm/pull/294))\n- github.com/libp2p/go-libp2p-testing (v0.5.0 -> v0.9.2):\n - release v0.9.2 (#56) ([libp2p/go-libp2p-testing#56](https://github.com/libp2p/go-libp2p-testing/pull/56))\n - fix memory allocation check in SubtestStreamReset (#55) ([libp2p/go-libp2p-testing#55](https://github.com/libp2p/go-libp2p-testing/pull/55))\n - release v0.9.1 (#54) ([libp2p/go-libp2p-testing#54](https://github.com/libp2p/go-libp2p-testing/pull/54))\n - remove stray debug statements for memory allocations\n - release v0.9.0 (#53) ([libp2p/go-libp2p-testing#53](https://github.com/libp2p/go-libp2p-testing/pull/53))\n - add tests for memory management ([libp2p/go-libp2p-testing#52](https://github.com/libp2p/go-libp2p-testing/pull/52))\n - release v0.8.0 (#50) ([libp2p/go-libp2p-testing#50](https://github.com/libp2p/go-libp2p-testing/pull/50))\n - use io.ReadFull in muxer test, use require.Equal to compare buffers (#49) ([libp2p/go-libp2p-testing#49](https://github.com/libp2p/go-libp2p-testing/pull/49))\n - release v0.7.0 (#47) ([libp2p/go-libp2p-testing#47](https://github.com/libp2p/go-libp2p-testing/pull/47))\n - add mocks for the resource manager ([libp2p/go-libp2p-testing#46](https://github.com/libp2p/go-libp2p-testing/pull/46))\n - merge libp2p/go-libp2p-netutil into this repo ([libp2p/go-libp2p-testing#45](https://github.com/libp2p/go-libp2p-testing/pull/45))\n - reduce the number of connections in the stream muxer stress test (#44) ([libp2p/go-libp2p-testing#44](https://github.com/libp2p/go-libp2p-testing/pull/44))\n - release v0.6.0 (#42) ([libp2p/go-libp2p-testing#42](https://github.com/libp2p/go-libp2p-testing/pull/42))\n - expose a map, not a slice, of muxer tests (#41) ([libp2p/go-libp2p-testing#41](https://github.com/libp2p/go-libp2p-testing/pull/41))\n - sync: update CI config files (#40) ([libp2p/go-libp2p-testing#40](https://github.com/libp2p/go-libp2p-testing/pull/40))\n- github.com/libp2p/go-libp2p-tls (v0.3.1 -> v0.4.1):\n - release v0.4.1 (#112) ([libp2p/go-libp2p-tls#112](https://github.com/libp2p/go-libp2p-tls/pull/112))\n - feat: catch panics in TLS negotiation ([libp2p/go-libp2p-tls#111](https://github.com/libp2p/go-libp2p-tls/pull/111))\n - release v0.4.0 (#110) ([libp2p/go-libp2p-tls#110](https://github.com/libp2p/go-libp2p-tls/pull/110))\n - use tls.Conn.HandshakeContext instead of tls.Conn.Handshake (#106) ([libp2p/go-libp2p-tls#106](https://github.com/libp2p/go-libp2p-tls/pull/106))\n - remove paragraph about Go modules from README (#104) ([libp2p/go-libp2p-tls#104](https://github.com/libp2p/go-libp2p-tls/pull/104))\n - migrate to standard Go tests, stop using Ginkgo ([libp2p/go-libp2p-tls#105](https://github.com/libp2p/go-libp2p-tls/pull/105))\n - chore: remove Codecov config (#103) ([libp2p/go-libp2p-tls#103](https://github.com/libp2p/go-libp2p-tls/pull/103))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.5.0 -> v0.7.1):\n - release v0.7.1 ([libp2p/go-libp2p-transport-upgrader#105](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/105))\n - Fix nil peer scope issues ([libp2p/go-libp2p-transport-upgrader#104](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/104))\n - release v0.7.0 (#103) ([libp2p/go-libp2p-transport-upgrader#103](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/103))\n - use the Resource Manager ([libp2p/go-libp2p-transport-upgrader#99](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/99))\n - rename the package to upgrader ([libp2p/go-libp2p-transport-upgrader#101](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/101))\n - use the new transport.Upgrader interface ([libp2p/go-libp2p-transport-upgrader#100](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/100))\n - reset the temporary error catcher delay after successful accept ([libp2p/go-libp2p-transport-upgrader#97](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/97))\n - make the accept timeout configurable, stop using transport.AcceptTimeout ([libp2p/go-libp2p-transport-upgrader#98](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/98))\n - release v0.6.0 (#95) ([libp2p/go-libp2p-transport-upgrader#95](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/95))\n - remove note about go.mod and Go 1.11 in README (#94) ([libp2p/go-libp2p-transport-upgrader#94](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/94))\n - fix flaky TestAcceptQueueBacklogged test ([libp2p/go-libp2p-transport-upgrader#96](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/96))\n - chore: remove Codecov config (#93) ([libp2p/go-libp2p-transport-upgrader#93](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/93))\n - use the new network.ConnStats ([libp2p/go-libp2p-transport-upgrader#92](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/92))\n - sync: update CI config files (#89) ([libp2p/go-libp2p-transport-upgrader#89](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/89))\n - chore: update go-log ([libp2p/go-libp2p-transport-upgrader#88](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/88))\n- github.com/libp2p/go-libp2p-yamux (v0.6.0 -> v0.9.1):\n - release v0.9.1 (#55) ([libp2p/go-libp2p-yamux#55](https://github.com/libp2p/go-libp2p-yamux/pull/55))\n - release v0.9.0 (#53) ([libp2p/go-libp2p-yamux#53](https://github.com/libp2p/go-libp2p-yamux/pull/53))\n - release v0.8.2 (#50) ([libp2p/go-libp2p-yamux#50](https://github.com/libp2p/go-libp2p-yamux/pull/50))\n - disable the incoming streams limit (#49) ([libp2p/go-libp2p-yamux#49](https://github.com/libp2p/go-libp2p-yamux/pull/49))\n - Release v0.8.1 ([libp2p/go-libp2p-yamux#48](https://github.com/libp2p/go-libp2p-yamux/pull/48))\n - release v0.8.0 (#47) ([libp2p/go-libp2p-yamux#47](https://github.com/libp2p/go-libp2p-yamux/pull/47))\n - pass the PeerScope to yamux ( satisfying its MemoryManger interface) (#46) ([libp2p/go-libp2p-yamux#46](https://github.com/libp2p/go-libp2p-yamux/pull/46))\n - release v0.7.0 (#43) ([libp2p/go-libp2p-yamux#43](https://github.com/libp2p/go-libp2p-yamux/pull/43))\n - sync: update CI config files (#42) ([libp2p/go-libp2p-yamux#42](https://github.com/libp2p/go-libp2p-yamux/pull/42))\n - reduce the number of max incoming stream to 256 ([libp2p/go-libp2p-yamux#41](https://github.com/libp2p/go-libp2p-yamux/pull/41))\n- github.com/libp2p/go-mplex (v0.3.0 -> v0.7.0):\n - release v0.7.0 (#112) ([libp2p/go-mplex#112](https://github.com/libp2p/go-mplex/pull/112))\n - catch panics in handleIncoming and handleOutgoing (#109) ([libp2p/go-mplex#109](https://github.com/libp2p/go-mplex/pull/109))\n - remove benchmark tests (#111) ([libp2p/go-mplex#111](https://github.com/libp2p/go-mplex/pull/111))\n - release v0.6.0 (#105) ([libp2p/go-mplex#105](https://github.com/libp2p/go-mplex/pull/105))\n - fix incorrect reset of timer fired variable (#104) ([libp2p/go-mplex#104](https://github.com/libp2p/go-mplex/pull/104))\n - Mplex salvage operations, part II (#102) ([libp2p/go-mplex#102](https://github.com/libp2p/go-mplex/pull/102))\n - release v0.5.0 (#100) ([libp2p/go-mplex#100](https://github.com/libp2p/go-mplex/pull/100))\n - Salvage mplex in the age of resource management (#99) ([libp2p/go-mplex#99](https://github.com/libp2p/go-mplex/pull/99))\n - release v0.4.0 (#97) ([libp2p/go-mplex#97](https://github.com/libp2p/go-mplex/pull/97))\n - add a MemoryManager interface to control memory allocations ([libp2p/go-mplex#96](https://github.com/libp2p/go-mplex/pull/96))\n - sync: update CI config files (#93) ([libp2p/go-mplex#93](https://github.com/libp2p/go-mplex/pull/93))\n - chore: update go-log to v2 ([libp2p/go-mplex#92](https://github.com/libp2p/go-mplex/pull/92))\n - chore: remove Codecov config ([libp2p/go-mplex#91](https://github.com/libp2p/go-mplex/pull/91))\n - sync: update CI config files (#90) ([libp2p/go-mplex#90](https://github.com/libp2p/go-mplex/pull/90))\n - multiplex: add (*Multiplex).CloseChan ([libp2p/go-mplex#89](https://github.com/libp2p/go-mplex/pull/89))\n - add a Go Reference badge to the README ([libp2p/go-mplex#88](https://github.com/libp2p/go-mplex/pull/88))\n - sync: update CI config files ([libp2p/go-mplex#85](https://github.com/libp2p/go-mplex/pull/85))\n - Fix up tests & vet ([libp2p/go-mplex#84](https://github.com/libp2p/go-mplex/pull/84))\n - Bump lodash from 4.17.19 to 4.17.21 in /interop/js ([libp2p/go-mplex#83](https://github.com/libp2p/go-mplex/pull/83))\n- github.com/libp2p/go-msgio (v0.1.0 -> v0.2.0):\n - release v0.2.0 (#34) ([libp2p/go-msgio#34](https://github.com/libp2p/go-msgio/pull/34))\n - print recovered panics to stderr (#33) ([libp2p/go-msgio#33](https://github.com/libp2p/go-msgio/pull/33))\n - catch panics when reading / writing protobuf messages (#31) ([libp2p/go-msgio#31](https://github.com/libp2p/go-msgio/pull/31))\n - remove outdated section about channels from README (#32) ([libp2p/go-msgio#32](https://github.com/libp2p/go-msgio/pull/32))\n - sync: update CI config files (#28) ([libp2p/go-msgio#28](https://github.com/libp2p/go-msgio/pull/28))\n- github.com/libp2p/go-netroute (v0.1.6 -> v0.2.0):\n - release v0.2.0 (#21) ([libp2p/go-netroute#21](https://github.com/libp2p/go-netroute/pull/21))\n - move some functions from go-sockaddr here, remove go-sockaddr dependency ([libp2p/go-netroute#22](https://github.com/libp2p/go-netroute/pull/22))\n - ignore the error on the RouteMessage on Darwin ([libp2p/go-netroute#20](https://github.com/libp2p/go-netroute/pull/20))\n - sync: update CI config files (#19) ([libp2p/go-netroute#19](https://github.com/libp2p/go-netroute/pull/19))\n - sync: update CI config files (#18) ([libp2p/go-netroute#18](https://github.com/libp2p/go-netroute/pull/18))\n - skip loopback addr as indication of v6 routes ([libp2p/go-netroute#17](https://github.com/libp2p/go-netroute/pull/17))\n - fix staticcheck lint issues ([libp2p/go-netroute#15](https://github.com/libp2p/go-netroute/pull/15))\n- github.com/libp2p/go-stream-muxer-multistream (v0.3.0 -> v0.4.0):\n - release v0.4.0 (#23) ([libp2p/go-stream-muxer-multistream#23](https://github.com/libp2p/go-stream-muxer-multistream/pull/23))\n - implement the new Multiplexer.NewConn interface ([libp2p/go-stream-muxer-multistream#22](https://github.com/libp2p/go-stream-muxer-multistream/pull/22))\n - sync: update CI config files (#19) ([libp2p/go-stream-muxer-multistream#19](https://github.com/libp2p/go-stream-muxer-multistream/pull/19))\n- github.com/libp2p/go-tcp-transport (v0.4.0 -> v0.5.1):\n - release v0.5.1 (#116) ([libp2p/go-tcp-transport#116](https://github.com/libp2p/go-tcp-transport/pull/116))\n - fix: drop raw EINVAL (from keepalives) errors as well (#115) ([libp2p/go-tcp-transport#115](https://github.com/libp2p/go-tcp-transport/pull/115))\n - release v0.5.0 (#114) ([libp2p/go-tcp-transport#114](https://github.com/libp2p/go-tcp-transport/pull/114))\n - use the ResourceManager ([libp2p/go-tcp-transport#110](https://github.com/libp2p/go-tcp-transport/pull/110))\n - use the transport.Upgrader interface ([libp2p/go-tcp-transport#111](https://github.com/libp2p/go-tcp-transport/pull/111))\n - describe how to use options in README ([libp2p/go-tcp-transport#105](https://github.com/libp2p/go-tcp-transport/pull/105))\n- github.com/libp2p/go-ws-transport (v0.5.0 -> v0.6.0):\n - release v0.6.0 (#113) ([libp2p/go-ws-transport#113](https://github.com/libp2p/go-ws-transport/pull/113))\n - use the resource manager ([libp2p/go-ws-transport#109](https://github.com/libp2p/go-ws-transport/pull/109))\n - chore: remove Codecov config (#112) ([libp2p/go-ws-transport#112](https://github.com/libp2p/go-ws-transport/pull/112))\n - remove contexts from libp2p constructors in README (#111) ([libp2p/go-ws-transport#111](https://github.com/libp2p/go-ws-transport/pull/111))\n - use the transport.Upgrader interface ([libp2p/go-ws-transport#110](https://github.com/libp2p/go-ws-transport/pull/110))\n - sync: update CI config files (#108) ([libp2p/go-ws-transport#108](https://github.com/libp2p/go-ws-transport/pull/108))\n - sync: update CI config files (#106) ([libp2p/go-ws-transport#106](https://github.com/libp2p/go-ws-transport/pull/106))\n- github.com/lucas-clemente/quic-go (v0.24.0 -> v0.27.1):\n - don't send path MTU probe packets on a timer\n - stop using the deprecated net.Error.Temporary, update golangci-lint to v1.45.2 ([lucas-clemente/quic-go#3367](https://github.com/lucas-clemente/quic-go/pull/3367))\n - add support for serializing Extended CONNECT requests (#3360) ([lucas-clemente/quic-go#3360](https://github.com/lucas-clemente/quic-go/pull/3360))\n - improve the error thrown when building with an unsupported Go version ([lucas-clemente/quic-go#3364](https://github.com/lucas-clemente/quic-go/pull/3364))\n - remove nextdns from list of projects using quic-go (#3363) ([lucas-clemente/quic-go#3363](https://github.com/lucas-clemente/quic-go/pull/3363))\n - rename the Session to Connection ([lucas-clemente/quic-go#3361](https://github.com/lucas-clemente/quic-go/pull/3361))\n - respect the request context when dialing ([lucas-clemente/quic-go#3359](https://github.com/lucas-clemente/quic-go/pull/3359))\n - update HTTP/3 Datagram to draft-ietf-masque-h3-datagram-07 (#3355) ([lucas-clemente/quic-go#3355](https://github.com/lucas-clemente/quic-go/pull/3355))\n - add support for the Extended CONNECT method (#3357) ([lucas-clemente/quic-go#3357](https://github.com/lucas-clemente/quic-go/pull/3357))\n - remove the SkipSchemeCheck RoundTripOpt (#3353) ([lucas-clemente/quic-go#3353](https://github.com/lucas-clemente/quic-go/pull/3353))\n - remove parser logic for HTTP/3 DUPLICATE_PUSH frame (#3356) ([lucas-clemente/quic-go#3356](https://github.com/lucas-clemente/quic-go/pull/3356))\n - improve code coverage of random number generator test (#3358) ([lucas-clemente/quic-go#3358](https://github.com/lucas-clemente/quic-go/pull/3358))\n - advertise multiple listeners via Alt-Svc and improve perf of SetQuicHeaders (#3352) ([lucas-clemente/quic-go#3352](https://github.com/lucas-clemente/quic-go/pull/3352))\n - avoid recursion when skipping unknown HTTP/3 frames (#3354) ([lucas-clemente/quic-go#3354](https://github.com/lucas-clemente/quic-go/pull/3354))\n - Implement http3.Server.ServeListener (#3349) ([lucas-clemente/quic-go#3349](https://github.com/lucas-clemente/quic-go/pull/3349))\n - update for Go 1.18 ([lucas-clemente/quic-go#3345](https://github.com/lucas-clemente/quic-go/pull/3345))\n - don't print a receive buffer warning for closed connections (#3346) ([lucas-clemente/quic-go#3346](https://github.com/lucas-clemente/quic-go/pull/3346))\n - move set DF implementation to separate files & avoid the need for OOBCapablePacketConn (#3334) ([lucas-clemente/quic-go#3334](https://github.com/lucas-clemente/quic-go/pull/3334))\n - add env to disable the receive buffer warning (#3339) ([lucas-clemente/quic-go#3339](https://github.com/lucas-clemente/quic-go/pull/3339))\n - fix typo (#3333) ([lucas-clemente/quic-go#3333](https://github.com/lucas-clemente/quic-go/pull/3333))\n - sendQueue: ignore \"datagram too large\" error (#3328) ([lucas-clemente/quic-go#3328](https://github.com/lucas-clemente/quic-go/pull/3328))\n - add OONI Probe to list of projects in README (#3324) ([lucas-clemente/quic-go#3324](https://github.com/lucas-clemente/quic-go/pull/3324))\n - remove build status badges from README (#3325) ([lucas-clemente/quic-go#3325](https://github.com/lucas-clemente/quic-go/pull/3325))\n - add a AllowConnectionWindowIncrease config option ([lucas-clemente/quic-go#3317](https://github.com/lucas-clemente/quic-go/pull/3317))\n - Update README.md (#3315) ([lucas-clemente/quic-go#3315](https://github.com/lucas-clemente/quic-go/pull/3315))\n - fix some typos in documentation and tests\n - remove unneeded calls to goimports when generating mocks ([lucas-clemente/quic-go#3313](https://github.com/lucas-clemente/quic-go/pull/3313))\n - fix comment about congestionWindow value (#3310) ([lucas-clemente/quic-go#3310](https://github.com/lucas-clemente/quic-go/pull/3310))\n - fix typo *connections (#3309) ([lucas-clemente/quic-go#3309](https://github.com/lucas-clemente/quic-go/pull/3309))\n - add support for Go 1.18 ([lucas-clemente/quic-go#3298](https://github.com/lucas-clemente/quic-go/pull/3298))\n- github.com/multiformats/go-base32 (v0.0.3 -> v0.0.4):\n - optimize encode ([multiformats/go-base32#1](https://github.com/multiformats/go-base32/pull/1))\n - Fix `staticcheck` issue\n- github.com/multiformats/go-multiaddr (v0.4.1 -> v0.5.0):\n - release v0.5.0 (#171) ([multiformats/go-multiaddr#171](https://github.com/multiformats/go-multiaddr/pull/171))\n - remove wrong (and redundant) IsIpv6LinkLocal ([multiformats/go-multiaddr#170](https://github.com/multiformats/go-multiaddr/pull/170))\n - move ResolveUnspecifiedAddress(es) and FilterAddrs here from libp2p/go-addr-util ([multiformats/go-multiaddr#168](https://github.com/multiformats/go-multiaddr/pull/168))\n - sync: update CI config files (#167) ([multiformats/go-multiaddr#167](https://github.com/multiformats/go-multiaddr/pull/167))\n- github.com/multiformats/go-multicodec (v0.3.0 -> v0.4.1):\n - Version v0.4.1 ([multiformats/go-multicodec#64](https://github.com/multiformats/go-multicodec/pull/64))\n - update table with new codecs ([multiformats/go-multicodec#63](https://github.com/multiformats/go-multicodec/pull/63))\n - bump version to v0.4.0\n - sync: update CI config files (#60) ([multiformats/go-multicodec#60](https://github.com/multiformats/go-multicodec/pull/60))\n - add Code.Tag method\n - add the KnownCodes API\n - use \"go run pkg@version\" assuming Go 1.17 or later\n - update submodule and re-generate\n - update to newer multicodec table ([multiformats/go-multicodec#57](https://github.com/multiformats/go-multicodec/pull/57))\n - Update `multicodec` submodule to `1bcdc08` for CARv2 index codec\n- github.com/multiformats/go-multistream (v0.2.2 -> v0.3.0):\n - release v0.3.0 (#82) ([multiformats/go-multistream#82](https://github.com/multiformats/go-multistream/pull/82))\n - catch panics (#81) ([multiformats/go-multistream#81](https://github.com/multiformats/go-multistream/pull/81))\n - sync: update CI config files (#78) ([multiformats/go-multistream#78](https://github.com/multiformats/go-multistream/pull/78))\n - reduce the maximum read buffer size from 64 to 1 kB ([multiformats/go-multistream#77](https://github.com/multiformats/go-multistream/pull/77))\n - remove unused ls command ([multiformats/go-multistream#76](https://github.com/multiformats/go-multistream/pull/76))\n - chore: remove empty file cases.md ([multiformats/go-multistream#75](https://github.com/multiformats/go-multistream/pull/75))\n - chore: remove .gx ([multiformats/go-multistream#72](https://github.com/multiformats/go-multistream/pull/72))\n - don't commit the fuzzing binary ([multiformats/go-multistream#74](https://github.com/multiformats/go-multistream/pull/74))\n - sync: update CI config files (#71) ([multiformats/go-multistream#71](https://github.com/multiformats/go-multistream/pull/71))\n - remove Makefile ([multiformats/go-multistream#67](https://github.com/multiformats/go-multistream/pull/67))\n\n
\n\n### ❤ Contributors\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 347 | +14453/-12552 | 842 |\n| Rod Vagg | 28 | +8848/-4033 | 214 |\n| vyzo | 133 | +7959/-1783 | 231 |\n| hannahhoward | 40 | +3761/-1652 | 175 |\n| Will Scott | 39 | +2885/-1784 | 93 |\n| Daniel Martí | 32 | +3043/-969 | 103 |\n| Adin Schmahmann | 48 | +3439/-536 | 121 |\n| Gus Eggert | 29 | +2644/-788 | 123 |\n| Steven Allen | 87 | +2417/-840 | 135 |\n| Marcin Rataj | 29 | +2312/-942 | 75 |\n| Will | 11 | +2520/-62 | 56 |\n| Lucas Molas | 28 | +1602/-578 | 90 |\n| Raúl Kripalani | 18 | +1519/-271 | 38 |\n| Brian Tiger Chow | 20 | +833/-379 | 40 |\n| Masih H. Derkani | 5 | +514/-460 | 8 |\n| Jeromy Johnson | 53 | +646/-302 | 83 |\n| Łukasz Magiera | 26 | +592/-245 | 43 |\n| Artem Mikheev | 2 | +616/-120 | 5 |\n| Franky W | 2 | +49/-525 | 9 |\n| Laurent Senta | 3 | +468/-82 | 52 |\n| Hector Sanjuan | 32 | +253/-187 | 62 |\n| Juan Batiz-Benet | 8 | +285/-80 | 18 |\n| Justin Johnson | 2 | +181/-88 | 2 |\n| Thibault Meunier | 5 | +216/-28 | 8 |\n| James Wetter | 2 | +234/-1 | 2 |\n| web3-bot | 36 | +158/-66 | 62 |\n| gammazero | 7 | +140/-84 | 12 |\n| Rachel Chen | 2 | +165/-57 | 17 |\n| Jorropo | 18 | +108/-99 | 26 |\n| Toby | 2 | +107/-86 | 11 |\n| Antonio Navarro Perez | 4 | +82/-103 | 9 |\n| Dominic Della Valle | 4 | +148/-33 | 6 |\n| Ian Davis | 2 | +152/-28 | 6 |\n| Kyle Huntsman | 2 | +172/-6 | 5 |\n| huoju | 4 | +127/-41 | 6 |\n| Jeromy | 19 | +71/-58 | 31 |\n| Lars Gierth | 12 | +63/-54 | 20 |\n| Eric Myhre | 3 | +95/-15 | 8 |\n| Caian Benedicto | 1 | +69/-12 | 6 |\n| Raúl Kripalani | 2 | +63/-13 | 2 |\n| Anton Petrov | 1 | +73/-0 | 1 |\n| hunjixin | 2 | +67/-2 | 5 |\n| odanado | 1 | +61/-0 | 1 |\n| Andrew Gillis | 2 | +61/-0 | 3 |\n| Kevin Atkinson | 6 | +21/-34 | 7 |\n| Richard Ramos | 1 | +51/-0 | 2 |\n| Manuel Alonso | 1 | +42/-9 | 2 |\n| Jakub Sztandera | 10 | +37/-13 | 13 |\n| Aarsh Shah | 1 | +39/-5 | 2 |\n| Dave Justice | 1 | +32/-4 | 2 |\n| Tommi Virtanen | 3 | +23/-9 | 4 |\n| tarekbadr | 1 | +30/-1 | 1 |\n| whyrusleeping | 1 | +26/-4 | 3 |\n| Petar Maymounkov | 2 | +30/-0 | 4 |\n| rht | 3 | +17/-10 | 4 |\n| Miguel Mota | 1 | +23/-0 | 1 |\n| Manfred Touron | 1 | +21/-2 | 2 |\n| watjurk | 1 | +17/-5 | 1 |\n| SukkaW | 1 | +11/-11 | 5 |\n| Nicholas Bollweg | 1 | +21/-0 | 1 |\n| Ho-Sheng Hsiao | 2 | +11/-10 | 6 |\n| chblodg | 1 | +18/-2 | 1 |\n| Friedel Ziegelmayer | 2 | +18/-0 | 2 |\n| Shu Shen | 2 | +15/-2 | 3 |\n| Peter Rabbitson | 1 | +15/-2 | 1 |\n| galargh | 2 | +15/-0 | 2 |\n| ᴍᴀᴛᴛ ʙᴇʟʟ | 3 | +13/-1 | 4 |\n| aarshkshah1992 | 3 | +12/-2 | 3 |\n| RubenKelevra | 4 | +5/-8 | 5 |\n| Feiran Yang | 1 | +11/-0 | 2 |\n| zramsay | 2 | +0/-10 | 2 |\n| Teran McKinney | 1 | +8/-2 | 1 |\n| Richard Littauer | 2 | +5/-5 | 5 |\n| Elijah | 1 | +10/-0 | 1 |\n| Dimitris Apostolou | 2 | +5/-5 | 5 |\n| Michael Avila | 3 | +8/-1 | 4 |\n| siiky | 3 | +4/-4 | 3 |\n| Somajit | 1 | +4/-4 | 1 |\n| Sherod Taylor | 1 | +0/-8 | 2 |\n| Eclésio Junior | 1 | +8/-0 | 1 |\n| godcong | 3 | +4/-3 | 3 |\n| Piotr Galar | 3 | +3/-4 | 3 |\n| jwh | 1 | +6/-0 | 2 |\n| dependabot[bot] | 1 | +3/-3 | 1 |\n| Volker Mische | 1 | +4/-2 | 1 |\n| Aayush Rajasekaran | 1 | +3/-3 | 1 |\n| rene | 2 | +3/-2 | 2 |\n| keks | 1 | +5/-0 | 1 |\n| Hlib | 1 | +4/-1 | 2 |\n| Arash Payan | 1 | +5/-0 | 1 |\n| Wayback Archiver | 1 | +2/-2 | 1 |\n| T Mo | 1 | +2/-2 | 1 |\n| Ju Huo | 1 | +2/-2 | 1 |\n| Ivan | 2 | +2/-2 | 2 |\n| Ettore Di Giacinto | 2 | +3/-1 | 2 |\n| Christian Couder | 1 | +3/-1 | 1 |\n| ningmingxiao | 1 | +0/-3 | 1 |\n| 市川恭佑 (ebi) | 1 | +1/-1 | 1 |\n| star | 1 | +0/-2 | 1 |\n| alliswell | 1 | +0/-2 | 1 |\n| Preston Van Loon | 1 | +2/-0 | 1 |\n| Nguyễn Gia Phong | 1 | +1/-1 | 1 |\n| Nato Boram | 1 | +1/-1 | 1 |\n| Mildred Ki'Lya | 1 | +2/-0 | 2 |\n| Michael Burns | 1 | +1/-1 | 1 |\n| Glenn | 1 | +1/-1 | 1 |\n| George Antoniadis | 1 | +1/-1 | 1 |\n| David Florness | 1 | +1/-1 | 1 |\n| Daniel Norman | 1 | +1/-1 | 1 |\n| Coenie Beyers | 1 | +1/-1 | 1 |\n| Benedikt Spies | 1 | +1/-1 | 1 |\n| Abdul Rauf | 1 | +1/-1 | 1 |\n| makeworld | 1 | +1/-0 | 1 |\n| ignoramous | 1 | +0/-1 | 1 |\n| Omicron166 | 1 | +0/-1 | 1 |\n| Jan Winkelmann | 1 | +1/-0 | 1 |\n| Dr Ian Preston | 1 | +1/-0 | 1 |\n| Baptiste Jonglez | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.14.md", "content": "# Kubo changelog v0.14\n\n## v0.14.0\n\n### Overview\n\nBelow is an outline of all that is in this release, so you get a sense of all that's included.\n\n- [🛠 BREAKING CHANGES](#-breaking-changes)\n - [Removed `mdns_legacy` implementation](#removed-mdns_legacy-implementation)\n- [🔦 Highlights](#-highlights)\n - [🛣️ Delegated Routing](#-delegated-routing)\n - [👥 Rename to Kubo](#-rename-to-kubo)\n - [🎒 `ipfs repo migrate`](#-ipfs-repo-migrate)\n - [🚀 Emoji support in Multibase](#-emoji-support-in-multibase)\n\n### 🛠 BREAKING CHANGES\n\n#### Removed `mdns_legacy` implementation\n\nThe modern DNS-SD compatible [zeroconf implementation](https://github.com/libp2p/zeroconf#readme)\n(based on [this specification](https://github.com/libp2p/specs/blob/master/discovery/mdns.md))\nhas been running next to the `mdns_legacy` for a while (since v0.11). During\nthis transitional period Kubo nodes were sending twice as many LAN packets,\nwhich ends with this release: we've [removed](https://github.com/ipfs/kubo/pull/9048) the legacy implementation.\n\n### 🔦 Highlights\n\n#### 🛣️ Delegated Routing\n\nContent routing is the a term used to describe the problem of finding providers for a given piece of content.\nIf you have a hash, or CID of some data, how do you find who has it?\nIn IPFS, until now, only a DHT was used as a decentralized answer to content routing.\nNow, content routing can be handled by clients implementing the [Reframe protocol](https://github.com/ipfs/specs/tree/main/reframe#readme).\n\nExample configuration usage using the [Filecoin Network Indexer](https://docs.cid.contact/filecoin-network-indexer/overview):\n\n```\nipfs config Routing.Routers.CidContact --json '{\n \"Type\": \"reframe\",\n \"Parameters\": {\n \"Endpoint\": \"https://cid.contact/reframe\"\n }\n}'\n\n```\n\n#### 👥 Rename to Kubo\n\nWe've renamed Go-IPFS to Kubo ([details](https://github.com/ipfs/go-ipfs/issues/8959)).\n\nPublished artifacts use `kubo` now, and are available at:\n\n- https://dist.ipfs.tech/kubo/\n- https://hub.docker.com/r/ipfs/kubo/\n\nTo minimize the impact on infrastructure that autoupdates on a new release,\nthe same binaries are still published under the old name at:\n\n- https://dist.ipfs.tech/go-ipfs/\n- https://hub.docker.com/r/ipfs/go-ipfs/\n\nThe libp2p identify useragent of Kubo has also been changed from `go-ipfs` to `kubo`.\n\n#### 🎒 `ipfs repo migrate`\n\nThis new command allows the you to run the repo migration without starting the daemon.\n\nSee `ipfs repo migrate --help` for more info.\n\n#### 🚀 Emoji support in Multibase\n\nKubo now supports [`base256emoji`](https://github.com/multiformats/multibase/blob/master/rfcs/Base256Emoji.md) encoding in all [Multibase](https://docs.ipfs.tech/concepts/glossary/#multibase) contexts. Use it for testing Unicode support, as visual aid while explaining Multiformats, or just for fun:\n\n```console\n$ echo -n \"test\" | ipfs multibase encode -b base256emoji -\n🚀😈✋🌈😈\n\n$ echo -n \"🚀😈✋🌈😈\" | ipfs multibase decode -\ntest\n\n$ ipfs cid format -v 1 -b base256emoji bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\n🚀🪐⭐💻😅❓💎🌈🌸🌚💰💍🌒😵🐶💁🤐🌎👼🙃🙅☺🌚😞🤤⭐🚀😃✈🌕😚🍻💜🐷⚽✌😊\n```\n\n[`/ipfs/🚀🪐⭐💻😅❓💎🌈🌸🌚💰💍🌒😵🐶💁🤐🌎👼🙃🙅☺🌚😞🤤⭐🚀😃✈🌕😚🍻💜🐷⚽✌😊`](https://ipfs.io/ipfs/🚀🪐⭐💻😅❓💎🌈🌸🌚💰💍🌒😵🐶💁🤐🌎👼🙃🙅☺🌚😞🤤⭐🚀😃✈🌕😚🍻💜🐷⚽✌😊)\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipfs/kubo:\n - chore: bump to v0.14.0\n - docs(changelog): finish v0.14.0 changelog\n - fix(gw): cache-control of index.html websites\n - chore(license): fix broken link to apache-2.0\n - fix: kubo in daemon and cli stdout\n - backport: merge commit '839b0848a' into release-v0.14.0\n - chore: Release v0.14-rc1\n - docs: fix v0.14's changelog format\n - chore: update go-multibase 🚀\n - feat(routing): Delegated Routing (#8997) ([ipfs/kubo#8997](https://github.com/ipfs/kubo/pull/8997))\n - chore: changelogs split\n - feat(gw): Cache-Control: only-if-cached\n - chore(deps): webui v2.15.1\n - Follow-ups after repository rename\n ([ipfs/kubo#9098](https://github.com/ipfs/kubo/pull/9098))\n - docs: refine wording\n - docs: refine the wording of provider strategies\n - refactor: rename to kubo\n ([ipfs/kubo#8958](https://github.com/ipfs/kubo/pull/8958))\n - fix: correct cache-control in car responses\n - docs: v0.13.1 (#9093) ([ipfs/kubo#9093](https://github.com/ipfs/kubo/pull/9093))\n - chore: update go-car ([ipfs/kubo#9089](https://github.com/ipfs/kubo/pull/9089))\n - update go-libp2p to v0.20.3 ([ipfs/kubo#9038](https://github.com/ipfs/kubo/pull/9038))\n - docs: add SECURITY.md (#9062) ([ipfs/kubo#9062](https://github.com/ipfs/kubo/pull/9062))\n - fix: remove mdns_legacy & Discovery.MDNS.Interval\n - refactor: prealloc slices with known sizes (#8892) ([ipfs/kubo#8892](https://github.com/ipfs/kubo/pull/8892))\n - docs: fix typo in `cid/base32`\n - docs: mark Swarm.ResourceMgr as experimental\n - chore: replace ioutil with io and os (#8969) ([ipfs/kubo#8969](https://github.com/ipfs/kubo/pull/8969))\n - feat: add a public function on peering to get the state\n - fix: honor url filename when downloading as CAR/BLOCK\n - Merge branch 'release'\n - chore: GitHub format\n - fix(cmd/config): make config edit subcommand work on windows\n - chore: bump Go to 1.18.3 (#9021) ([ipfs/kubo#9021](https://github.com/ipfs/kubo/pull/9021))\n - feat: upgrade to go-libp2p-kad-dht@v0.16.0 (#9005) ([ipfs/kubo#9005](https://github.com/ipfs/kubo/pull/9005))\n - docs: fix typo in the `swarm/peering` help text\n - feat: disable resource manager by default (#9003) ([ipfs/kubo#9003](https://github.com/ipfs/kubo/pull/9003))\n - fix: adjust rcmgr limits for accelerated DHT client rt refresh (#8982) ([ipfs/kubo#8982](https://github.com/ipfs/kubo/pull/8982))\n - fix(ci): make go-ipfs-as-a-library work without external peers (#8978) ([ipfs/kubo#8978](https://github.com/ipfs/kubo/pull/8978))\n - feat: log when resource manager limits are exceeded (#8980) ([ipfs/kubo#8980](https://github.com/ipfs/kubo/pull/8980))\n - fix: JS caching via Access-Control-Expose-Headers (#8984) ([ipfs/kubo#8984](https://github.com/ipfs/kubo/pull/8984))\n - docs: fix abstractions typo\n - fix: hanging goroutine in get fileArchive handler\n - chore: mark fuse experimental (#8962) ([ipfs/kubo#8962](https://github.com/ipfs/kubo/pull/8962))\n - fix(node/libp2p): disable rcmgr checkImplicitDefaults ([ipfs/kubo#8965](https://github.com/ipfs/kubo/pull/8965))\n - Add 'ipfs repo migrate' command (#8428) ([ipfs/kubo#8428](https://github.com/ipfs/kubo/pull/8428))\n - pubsub multibase encoding (#8933) ([ipfs/kubo#8933](https://github.com/ipfs/kubo/pull/8933))\n - 'pin rm' helptext: rewrite description as object is not removed from local storage (immediately) ([ipfs/kubo#8947](https://github.com/ipfs/kubo/pull/8947))\n - ([ipfs/kubo#8934](https://github.com/ipfs/kubo/pull/8934))\n - Add instructions to resolve repo migration error (#8946) ([ipfs/kubo#8946](https://github.com/ipfs/kubo/pull/8946))\n - fix: use path instead of filepath for asset embeds to support Windows\n - chore: update version to v0.14.0-dev\n- github.com/ipfs/go-bitswap (v0.6.0 -> v0.7.0):\n - chore: release v0.7.0 (#566) ([ipfs/go-bitswap#566](https://github.com/ipfs/go-bitswap/pull/566))\n - feat: coalesce and queue connection event handling (#565) ([ipfs/go-bitswap#565](https://github.com/ipfs/go-bitswap/pull/565))\n - fix initialisation example in README (#552) ([ipfs/go-bitswap#552](https://github.com/ipfs/go-bitswap/pull/552))\n- github.com/ipfs/go-unixfs (v0.3.1 -> v0.4.0):\n - Set version to v0.3.2 ([ipfs/go-unixfs#122](https://github.com/ipfs/go-unixfs/pull/122))\n - Make switchToSharding more efficient\n- github.com/ipld/go-ipld-prime (v0.16.0 -> v0.17.0):\n failed to fetch repo\n- github.com/libp2p/go-libp2p (v0.19.4 -> v0.20.3):\n - Release 0.20.3 (#1623) ([libp2p/go-libp2p#1623](https://github.com/libp2p/go-libp2p/pull/1623))\n - release v0.20.2\n - feat: allow dialing wss peers using DNS multiaddrs\n - update go-yamux to v3.1.2, release v0.20.1 (#1591) ([libp2p/go-libp2p#1591](https://github.com/libp2p/go-libp2p/pull/1591))\n - release v0.20.0 (#1530) ([libp2p/go-libp2p#1530](https://github.com/libp2p/go-libp2p/pull/1530))\n - update go-libp2p-core, remove stream methods from network.Notifiee (#1521) ([libp2p/go-libp2p#1521](https://github.com/libp2p/go-libp2p/pull/1521))\n - autonat: return E_DIAL_REFUSED when skipping dial (#1527) ([libp2p/go-libp2p#1527](https://github.com/libp2p/go-libp2p/pull/1527))\n - move go-stream-muxer-multistream here ([libp2p/go-libp2p#1511](https://github.com/libp2p/go-libp2p/pull/1511))\n - remove dependency on go-libp2p-testing/suites/sec (#1510) ([libp2p/go-libp2p#1510](https://github.com/libp2p/go-libp2p/pull/1510))\n - backoff: fix flaky tests in backoff cache (#1516) ([libp2p/go-libp2p#1516](https://github.com/libp2p/go-libp2p/pull/1516))\n - identify: fix flaky tests (#1515) ([libp2p/go-libp2p#1515](https://github.com/libp2p/go-libp2p/pull/1515))\n - quic: increase timeout in hole punching test (#1495) ([libp2p/go-libp2p#1495](https://github.com/libp2p/go-libp2p/pull/1495))\n - Fix badge image in README (#1517) ([libp2p/go-libp2p#1517](https://github.com/libp2p/go-libp2p/pull/1517))\n - move go-libp2p-nat here ([libp2p/go-libp2p#1513](https://github.com/libp2p/go-libp2p/pull/1513))\n - move go-reuseport-transport here ([libp2p/go-libp2p#1459](https://github.com/libp2p/go-libp2p/pull/1459))\n - holepunch: fix flaky TestEndToEndSimConnect test (#1508) ([libp2p/go-libp2p#1508](https://github.com/libp2p/go-libp2p/pull/1508))\n - swarm: fix flaky TestDialExistingConnection test (#1509) ([libp2p/go-libp2p#1509](https://github.com/libp2p/go-libp2p/pull/1509))\n - tcp: limit the number of connections in tcp suite test on non-linux hosts (#1507) ([libp2p/go-libp2p#1507](https://github.com/libp2p/go-libp2p/pull/1507))\n - increase overly short require.Eventually intervals (#1501) ([libp2p/go-libp2p#1501](https://github.com/libp2p/go-libp2p/pull/1501))\n - tls: fix flaky handshake cancellation test (#1503) ([libp2p/go-libp2p#1503](https://github.com/libp2p/go-libp2p/pull/1503))\n - merge the transport test suite from go-libp2p-testing here ([libp2p/go-libp2p#1496](https://github.com/libp2p/go-libp2p/pull/1496))\n - fix racy connection comparison in TestDialWorkerLoopBasic (#1499) ([libp2p/go-libp2p#1499](https://github.com/libp2p/go-libp2p/pull/1499))\n - swarm: fix race condition in TestFailFirst (#1490) ([libp2p/go-libp2p#1490](https://github.com/libp2p/go-libp2p/pull/1490))\n - basichost: fix flaky TestSignedPeerRecordWithNoListenAddrs (#1488) ([libp2p/go-libp2p#1488](https://github.com/libp2p/go-libp2p/pull/1488))\n - swarm: fix flaky and racy TestDialExistingConnection (#1491) ([libp2p/go-libp2p#1491](https://github.com/libp2p/go-libp2p/pull/1491))\n - quic: adjust timeout for reuse garbage collector detection in tests (#1487) ([libp2p/go-libp2p#1487](https://github.com/libp2p/go-libp2p/pull/1487))\n - quic: fix flaky TestResourceManagerAcceptDenied (#1485) ([libp2p/go-libp2p#1485](https://github.com/libp2p/go-libp2p/pull/1485))\n - quic: deflake the holepunching test (#1484) ([libp2p/go-libp2p#1484](https://github.com/libp2p/go-libp2p/pull/1484))\n - holepunch: fix incorrect message type for the SYNC message (#1478) ([libp2p/go-libp2p#1478](https://github.com/libp2p/go-libp2p/pull/1478))\n - use real keys in tests instead of go-libp2p-testing/netutil fake keys (#1475) ([libp2p/go-libp2p#1475](https://github.com/libp2p/go-libp2p/pull/1475))\n - quic: fix flaky TestResourceManagerAcceptDenied ([libp2p/go-libp2p#1461](https://github.com/libp2p/go-libp2p/pull/1461))\n - move go-libp2p-pnet here ([libp2p/go-libp2p#1465](https://github.com/libp2p/go-libp2p/pull/1465))\n - move go-libp2p-tls here ([libp2p/go-libp2p#1466](https://github.com/libp2p/go-libp2p/pull/1466))\n - fix race condition in relayFinder ([libp2p/go-libp2p#1469](https://github.com/libp2p/go-libp2p/pull/1469))\n - fix race condition in holepunch service (#1473) ([libp2p/go-libp2p#1473](https://github.com/libp2p/go-libp2p/pull/1473))\n - Update README to include supported Go Versions (#1470) ([libp2p/go-libp2p#1470](https://github.com/libp2p/go-libp2p/pull/1470))\n - move go-libp2p-noise here ([libp2p/go-libp2p#1462](https://github.com/libp2p/go-libp2p/pull/1462))\n - move go-libp2p-transport-upgrader here ([libp2p/go-libp2p#1463](https://github.com/libp2p/go-libp2p/pull/1463))\n - move go-conn-security-multistream here ([libp2p/go-libp2p#1460](https://github.com/libp2p/go-libp2p/pull/1460))\n - move go-libp2p-mplex here ([libp2p/go-libp2p#1450](https://github.com/libp2p/go-libp2p/pull/1450))\n - use yamux instead of mplex in tests (#1456) ([libp2p/go-libp2p#1456](https://github.com/libp2p/go-libp2p/pull/1456))\n - rename the yamux package (#1452) ([libp2p/go-libp2p#1452](https://github.com/libp2p/go-libp2p/pull/1452))\n - swarm: don't check return value of str.Close in TestResourceManager (#1453) ([libp2p/go-libp2p#1453](https://github.com/libp2p/go-libp2p/pull/1453))\n - move go-libp2p-yamux here ([libp2p/go-libp2p#1439](https://github.com/libp2p/go-libp2p/pull/1439))\n - quic: fix flaky TestConnectionGating test (#1442) ([libp2p/go-libp2p#1442](https://github.com/libp2p/go-libp2p/pull/1442))\n - quic: fix flaky TestReuseGarbageCollect test (#1446) ([libp2p/go-libp2p#1446](https://github.com/libp2p/go-libp2p/pull/1446))\n - quic: fix flaky holepunching test (#1443) ([libp2p/go-libp2p#1443](https://github.com/libp2p/go-libp2p/pull/1443))\n - move go-libp2p-quic-transport here ([libp2p/go-libp2p#1424](https://github.com/libp2p/go-libp2p/pull/1424))\n - remove flaky TestTcpSimultaneousConnect (#1425) ([libp2p/go-libp2p#1425](https://github.com/libp2p/go-libp2p/pull/1425))\n - move go-ws-transport here ([libp2p/go-libp2p#1422](https://github.com/libp2p/go-libp2p/pull/1422))\n - update go-multistream, stop using deprecated NegotiateLazy (#1417) ([libp2p/go-libp2p#1417](https://github.com/libp2p/go-libp2p/pull/1417))\n - fix flaky TestResourceManagerAcceptStream test (#1420) ([libp2p/go-libp2p#1420](https://github.com/libp2p/go-libp2p/pull/1420))\n - move go-tcp-transport here ([libp2p/go-libp2p#1418](https://github.com/libp2p/go-libp2p/pull/1418))\n - move the go-libp2p-swarm here ([libp2p/go-libp2p#1414](https://github.com/libp2p/go-libp2p/pull/1414))\n - reduce flakiness of backoff cache tests (#1415) ([libp2p/go-libp2p#1415](https://github.com/libp2p/go-libp2p/pull/1415))\n - move the go-libp2p-blankhost here ([libp2p/go-libp2p#1411](https://github.com/libp2p/go-libp2p/pull/1411))\n- github.com/libp2p/go-libp2p-core (v0.15.1 -> v0.16.1):\n - release v0.16.1 (#255) ([libp2p/go-libp2p-core#255](https://github.com/libp2p/go-libp2p-core/pull/255))\n - force usage of github.com/btcsuite/btcd v0.22.1 or newer (#254) ([libp2p/go-libp2p-core#254](https://github.com/libp2p/go-libp2p-core/pull/254))\n - release v0.16.0 (#251) ([libp2p/go-libp2p-core#251](https://github.com/libp2p/go-libp2p-core/pull/251))\n - remove OpenedStream and ClosedStream from Notifiee interface (#250) ([libp2p/go-libp2p-core#250](https://github.com/libp2p/go-libp2p-core/pull/250))\n - deprecate Negotiator.NegotiateLazy (#249) ([libp2p/go-libp2p-core#249](https://github.com/libp2p/go-libp2p-core/pull/249))\n - update btcec dependency (#247) ([libp2p/go-libp2p-core#247](https://github.com/libp2p/go-libp2p-core/pull/247))\n- github.com/libp2p/go-libp2p-discovery (v0.6.0 -> v0.7.0):\n - deprecate this repo (#84) ([libp2p/go-libp2p-discovery#84](https://github.com/libp2p/go-libp2p-discovery/pull/84))\n - remove dependency on the go-libp2p-peerstore/addr package (#82) ([libp2p/go-libp2p-discovery#82](https://github.com/libp2p/go-libp2p-discovery/pull/82))\n - fix flaky TestBackoffDiscoveryMultipleBackoff test on CI (#80) ([libp2p/go-libp2p-discovery#80](https://github.com/libp2p/go-libp2p-discovery/pull/80))\n - chore: update go-log to v2 ([libp2p/go-libp2p-discovery#76](https://github.com/libp2p/go-libp2p-discovery/pull/76))\n - sync: update CI config files (#74) ([libp2p/go-libp2p-discovery#74](https://github.com/libp2p/go-libp2p-discovery/pull/74))\n- github.com/libp2p/go-libp2p-swarm (v0.10.2 -> v0.11.0):\n - deprecate this repo (#320) ([libp2p/go-libp2p-swarm#320](https://github.com/libp2p/go-libp2p-swarm/pull/320))\n - sync: update CI config files ([libp2p/go-libp2p-swarm#317](https://github.com/libp2p/go-libp2p-swarm/pull/317))\n- github.com/libp2p/go-reuseport (v0.1.0 -> v0.2.0):\n - release v0.2.0 (#90) ([libp2p/go-reuseport#90](https://github.com/libp2p/go-reuseport/pull/90))\n - sync: update CI config files (#86) ([libp2p/go-reuseport#86](https://github.com/libp2p/go-reuseport/pull/86))\n- github.com/multiformats/go-multibase (v0.0.3 -> v0.1.0):\n - chore: release v0.1.0\n - feat: add UTF-8 support and base256emoji\n - submodule: spec/\n - sync: update CI config files (#48) ([multiformats/go-multibase#48](https://github.com/multiformats/go-multibase/pull/48))\n - fix staticcheck ([multiformats/go-multibase#41](https://github.com/multiformats/go-multibase/pull/41))\n - Fix vet warnings about conversion of int to string ([multiformats/go-multibase#39](https://github.com/multiformats/go-multibase/pull/39))\n- github.com/multiformats/go-multihash (v0.1.0 -> v0.2.0):\n - chore: replace blake2b implementation by golang.org/x/crypto ([multiformats/go-multihash#157](https://github.com/multiformats/go-multihash/pull/157))\n - sync: update CI config files ([multiformats/go-multihash#156](https://github.com/multiformats/go-multihash/pull/156))\n- github.com/multiformats/go-multistream (v0.3.0 -> v0.3.3):\n - Release v0.3.3 ([multiformats/go-multistream#90](https://github.com/multiformats/go-multistream/pull/90))\n - Ignore error if can't write back multistream protocol id ([multiformats/go-multistream#89](https://github.com/multiformats/go-multistream/pull/89))\n - release v0.3.2 (#88) ([multiformats/go-multistream#88](https://github.com/multiformats/go-multistream/pull/88))\n - Ignore error if can't write back echoed protocol in negotiate (#87) ([multiformats/go-multistream#87](https://github.com/multiformats/go-multistream/pull/87))\n - release v0.3.1 (#86) ([multiformats/go-multistream#86](https://github.com/multiformats/go-multistream/pull/86))\n - deprecate NegotiateLazy (#85) ([multiformats/go-multistream#85](https://github.com/multiformats/go-multistream/pull/85))\n - return an ErrNotSupported when lazy negotiation fails (#84) ([multiformats/go-multistream#84](https://github.com/multiformats/go-multistream/pull/84))\n- github.com/warpfork/go-testmark (v0.9.0 -> v0.10.0):\n - testexec: support a hunk named 'input' for stdin.\n - readme: link to other implementations!\n - readme: discuss autopatching and fixture regeneration\n - readme: discuss extensions, and introduce testexec as an example.\n\n
\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 376 | +11584/-15055 | 894 |\n| Jorropo | 18 | +11649/-11249 | 81 |\n| noot | 43 | +5974/-3332 | 170 |\n| Steven Allen | 173 | +5206/-3124 | 282 |\n| Yusef Napora | 49 | +1911/-3606 | 124 |\n| Juan Batiz-Benet | 14 | +3933/-53 | 48 |\n| Jeromy | 84 | +2140/-1328 | 240 |\n| vyzo | 51 | +2057/-1126 | 79 |\n| Raúl Kripalani | 39 | +1993/-867 | 103 |\n| Jeromy Johnson | 52 | +1700/-1081 | 233 |\n| Antonio Navarro Perez | 4 | +1874/-729 | 34 |\n| Aarsh Shah | 24 | +1428/-504 | 54 |\n| Marcin Rataj | 19 | +1051/-855 | 251 |\n| Alex Browne | 25 | +1207/-582 | 49 |\n| Jakub Sztandera | 29 | +898/-335 | 63 |\n| Friedel Ziegelmayer | 11 | +491/-284 | 18 |\n| Will Scott | 6 | +240/-319 | 17 |\n| Marco Munizaga | 11 | +377/-141 | 17 |\n| Hlib | 8 | +269/-135 | 15 |\n| Gus Eggert | 5 | +325/-63 | 19 |\n| lnykww | 1 | +275/-50 | 4 |\n| Łukasz Magiera | 3 | +196/-58 | 7 |\n| Matt Joiner | 14 | +79/-55 | 17 |\n| Eric Myhre | 4 | +122/-6 | 5 |\n| Andrew Gillis | 1 | +111/-6 | 4 |\n| Fazlul Shahriar | 2 | +84/-31 | 5 |\n| tg | 1 | +70/-15 | 2 |\n| Cory Schwartz | 4 | +50/-28 | 11 |\n| Lars Gierth | 3 | +33/-26 | 3 |\n| Cole Brown | 2 | +37/-16 | 9 |\n| web3-bot | 7 | +38/-11 | 18 |\n| Alvin Reyes | 1 | +34/-14 | 1 |\n| Hector Sanjuan | 4 | +34/-8 | 5 |\n| Guilhem Fanton | 2 | +28/-10 | 6 |\n| Brian Meek | 1 | +14/-17 | 4 |\n| Hlib Kanunnikov | 1 | +25/-3 | 1 |\n| Adin Schmahmann | 5 | +15/-13 | 5 |\n| Henrique Dias | 1 | +24/-2 | 4 |\n| Dennis Trautwein | 1 | +20/-4 | 2 |\n| galargh | 2 | +18/-2 | 2 |\n| M. Hawn | 3 | +10/-10 | 7 |\n| Can ZHANG | 1 | +12/-3 | 1 |\n| Masih H. Derkani | 1 | +4/-10 | 2 |\n| gammazero | 1 | +6/-6 | 2 |\n| Ikko Ashimine | 1 | +6/-6 | 2 |\n| Daniel N | 2 | +6/-5 | 2 |\n| watjurk | 1 | +8/-2 | 1 |\n| John Steidley | 2 | +4/-4 | 3 |\n| Aaron Bieber | 1 | +6/-2 | 1 |\n| Kishan Mohanbhai Sagathiya | 1 | +6/-1 | 1 |\n| siiky | 3 | +3/-3 | 3 |\n| Lucas Molas | 1 | +5/-1 | 1 |\n| Kevin Atkinson | 1 | +3/-3 | 1 |\n| Aayush Rajasekaran | 1 | +5/-1 | 1 |\n| T Mo | 1 | +2/-2 | 1 |\n| Piotr Galar | 1 | +2/-2 | 1 |\n| Arber Avdullahu | 1 | +2/-2 | 1 |\n| Russell Dempsey | 1 | +2/-1 | 1 |\n| anders | 1 | +1/-1 | 1 |\n| RubenKelevra | 1 | +1/-1 | 1 |\n| Jonathan Rudenberg | 1 | +1/-1 | 1 |\n| Ettore Di Giacinto | 1 | +2/-0 | 1 |\n| Daniel Norman | 1 | +1/-1 | 1 |\n| Chawye Hsu | 1 | +1/-1 | 1 |\n| Aliabbas Merchant | 1 | +1/-1 | 1 |\n| can | 1 | +1/-0 | 1 |\n| Ed Mazurek | 1 | +0/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.15.md", "content": "# Kubo changelog v0.15\n\n## v0.15.0\n\n### Overview\n\nBelow is an outline of all that is in this release, so you get a sense of all that's included.\n\n- [Kubo changelog v0.15](#kubo-changelog-v015)\n - [v0.15.0](#v0150)\n - [Overview](#overview)\n - [🔦 Highlights](#-highlights)\n - [#️⃣ Blake 3 support](#️⃣-blake-3-support)\n - [💉 Fx Options plugin](#-fx-options-plugin)\n - [📁 `$IPFS_PATH/gateway` file](#-ipfs_pathgateway-file)\n - [Changelog](#changelog)\n - [Contributors](#contributors)\n\n\n### 🔦 Highlights\n\nThis is a release mainly with bugfixing and library updates.\nWe are improving release speed and cadence trying to have a new release every 5 weeks.\n\n#### #️⃣ Blake 3 support\n\nYou can now use `blake3` as a valid hash function:\n- `ipfs block put --mhtype=blake3`\n- `ipfs add --hash=blake3`\n\nIt uses a 32 bytes default size.\nAnd verify up to 128 bytes.\nBecause `blake3` is variable output hash function, you can use a different digest length, set `mhlen`: `ipfs block put --mhtype=blake3 --mhlen=64`, `ipfs add` doesn't have this option yet.\n\n#### 💉 Fx Options plugin\n\nThis adds a plugin interface that lets the plugin modify the fx options that are passed to fx when the app is initialized.\nThis means plugins can inject their own implementations of Kubo interfaces.\nThis enables granular customization of Kubo behavior by plugins, such as:\n\n- Bitswap with custom filters (e.g. for CID blocking)\n- Custom interface implementations such as Pinner or DAGService\n- Dynamic configuration of libp2p ...\n\nHere's an example plugin that overrides the default Pinner with a custom one:\n\n```go\nfunc (p *PinnerPlugin) Options(info core.FXNodeInfo) ([]fx.Option, error) {\n\tpinner := mypinner.New() \n\treturn append(info.FXOptions, fx.Replace(fx.Annotate(pinner, fx.As(new(pin.Pinner))))), nil\n}\n```\n\nExtra plugin info [here](https://github.com/ipfs/kubo/blob/master/docs/plugins.md#fx-experimental).\n\n#### 📁 `$IPFS_PATH/gateway` file\n\nThis adds a new file in the `IPFS_PATH` folder similar to `$IPFS_PATH/api` containing an address based on [`Addresses.Gateway`](https://github.com/ipfs/kubo/blob/master/docs/config.md#addressesgateway) configuration.\n\nThis file is in URL (RFC1738) format.\n\n```console\n$ cat ~/.ipfs/gateway\nhttp://127.0.0.1:8080\n```\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipfs/kubo:\n - chore: Release v0.15.0-rc1\n - Update RELEASE_ISSUE_TEMPLATE.md for 0.15\n - docs(add): skip binary name in helptext\n - docs(cli): clarify CID determinism in add command\n - docs(cli): clarify CAR format in dag export|import\n - test(gw): cors preflight with custom header\n - feat: make corehttp a reusable component ([ipfs/kubo#9070](https://github.com/ipfs/kubo/pull/9070))\n - feat: go-libp2p v0.21 (rcmgr auto scaling) ([ipfs/kubo#9074](https://github.com/ipfs/kubo/pull/9074))\n - ([ipfs/kubo#9024](https://github.com/ipfs/kubo/pull/9024))\n - ([ipfs/kubo#9100](https://github.com/ipfs/kubo/pull/9100))\n - ([ipfs/kubo#9095](https://github.com/ipfs/kubo/pull/9095))\n - chore(cmd): add shutdown to CLI help ([ipfs/kubo#9194](https://github.com/ipfs/kubo/pull/9194))\n - docs: add fx plugin documentation to plugins.md (#9191) ([ipfs/kubo#9191](https://github.com/ipfs/kubo/pull/9191))\n - chore: switch to dist.ipfs.tech\n - feat: add fx options plugin\n - feat: add blake3 support\n - Add reference to Experimental config doc (#9181) ([ipfs/kubo#9181](https://github.com/ipfs/kubo/pull/9181))\n - feat: add $IPFS_PATH/gateway file\n - docs: replace `docs.ipfs.io` with `docs.ipfs.tech` (#9158) ([ipfs/kubo#9158](https://github.com/ipfs/kubo/pull/9158))\n - chore: fix markdown link syntax typo for AutoNAT.ServiceMode\n - chore: bump go-blockservice to only do put once\n - docs: update Arch Linux installation instructions\n - chore: update kubo-as-a-library example\n - docs(readme): add maintainer info (#9141) ([ipfs/kubo#9141](https://github.com/ipfs/kubo/pull/9141))\n - fix(gw): 404 when a valid DAG is missing link\n - fix(gw): directory URL normalization ([ipfs/kubo#9123](https://github.com/ipfs/kubo/pull/9123))\n - docs(config): add link to someguy router\n - fix: typo in README\n - Reproducible Builds: Update GOFLAGS for -trimpath\n - Merge v0.14.0 back into master\n - fix(gw): cache-control of index.html websites\n - chore(license): fix broken link to apache-2.0\n - fix: kubo in daemon and cli stdout\n - docs(readme): move content to docs website (#9102) ([ipfs/kubo#9102](https://github.com/ipfs/kubo/pull/9102))\n - fix(gw): no backlink when listing root dir\n- github.com/ipfs/go-bitswap (v0.7.0 -> v0.9.0):\n - chore: release v0.9.0\n - feat: split client and server ([ipfs/go-bitswap#570](https://github.com/ipfs/go-bitswap/pull/570))\n - chore: remove goprocess from blockstoremanager\n - Don't add blocks to the datastore ([ipfs/go-bitswap#571](https://github.com/ipfs/go-bitswap/pull/571))\n - Remove dependency on travis package from go-libp2p-testing ([ipfs/go-bitswap#569](https://github.com/ipfs/go-bitswap/pull/569))\n - feat: add basic tracing (#562) ([ipfs/go-bitswap#562](https://github.com/ipfs/go-bitswap/pull/562))\n- github.com/ipfs/go-blockservice (v0.3.0 -> v0.4.0):\n - write blocks retrieved from the exchange to the blockstore ([ipfs/go-blockservice#92](https://github.com/ipfs/go-blockservice/pull/92))\n - feat: add basic tracing ([ipfs/go-blockservice#91](https://github.com/ipfs/go-blockservice/pull/91))\n- github.com/ipfs/go-ipfs-exchange-interface (v0.1.0 -> v0.2.0):\n - Rename HasBlock to NotifyNewBlocks, and make it accept multiple blocks ([ipfs/go-ipfs-exchange-interface#23](https://github.com/ipfs/go-ipfs-exchange-interface/pull/23))\n- github.com/ipfs/go-ipfs-exchange-offline (v0.2.0 -> v0.3.0):\n - Exchange don't add blocks on their own anymore ([ipfs/go-ipfs-exchange-offline#47](https://github.com/ipfs/go-ipfs-exchange-offline/pull/47))\n- github.com/ipfs/go-verifcid (v0.0.1 -> v0.0.2):\n - chore: release v0.0.2\n - feat: add blake3 as a good hash\n - sync: update CI config files (#12) ([ipfs/go-verifcid#12](https://github.com/ipfs/go-verifcid/pull/12))\n - Add license ([ipfs/go-verifcid#8](https://github.com/ipfs/go-verifcid/pull/8))\n- github.com/ipld/go-codec-dagpb (v1.4.0 -> v1.4.1):\n - v1.4.1 bump\n- github.com/libp2p/go-buffer-pool (v0.0.2 -> v0.1.0):\n - release v0.1.0 (#30) ([libp2p/go-buffer-pool#30](https://github.com/libp2p/go-buffer-pool/pull/30))\n - panic if a negative length is passed to BufferPool.Get (#28) ([libp2p/go-buffer-pool#28](https://github.com/libp2p/go-buffer-pool/pull/28))\n - sync: update CI config files (#22) ([libp2p/go-buffer-pool#22](https://github.com/libp2p/go-buffer-pool/pull/22))\n - sync: update CI config files (#20) ([libp2p/go-buffer-pool#20](https://github.com/libp2p/go-buffer-pool/pull/20))\n - test: fix gc test on go 1.16 ([libp2p/go-buffer-pool#18](https://github.com/libp2p/go-buffer-pool/pull/18))\n - fix staticcheck ([libp2p/go-buffer-pool#16](https://github.com/libp2p/go-buffer-pool/pull/16))\n - test: make sure we have the correct number of pools ([libp2p/go-buffer-pool#10](https://github.com/libp2p/go-buffer-pool/pull/10))\n- github.com/libp2p/go-libp2p (v0.20.3 -> v0.21.0):\n - Release v0.21.0 (#1648) ([libp2p/go-libp2p#1648](https://github.com/libp2p/go-libp2p/pull/1648))\n - ping: optimize random number generation (#1658) ([libp2p/go-libp2p#1658](https://github.com/libp2p/go-libp2p/pull/1658))\n - feat: switch noise to use minio's SHA256 implementation (#1657) ([libp2p/go-libp2p#1657](https://github.com/libp2p/go-libp2p/pull/1657))\n - swarm: mark dialing WebTransport addresses as expensive (#1650) ([libp2p/go-libp2p#1650](https://github.com/libp2p/go-libp2p/pull/1650))\n - routedhost: fix decoding of relay peer ID (#1644) ([libp2p/go-libp2p#1644](https://github.com/libp2p/go-libp2p/pull/1644))\n - Release v0.21.0 RC (#1638) ([libp2p/go-libp2p#1638](https://github.com/libp2p/go-libp2p/pull/1638))\n - fix: return the best _acceptable_ conn in NewStream (#1604) ([libp2p/go-libp2p#1604](https://github.com/libp2p/go-libp2p/pull/1604))\n - use autoscaling limits (#1637) ([libp2p/go-libp2p#1637](https://github.com/libp2p/go-libp2p/pull/1637))\n - docs: point to SetDefaultServiceLimits in ResourceManager option (#1636) ([libp2p/go-libp2p#1636](https://github.com/libp2p/go-libp2p/pull/1636))\n - chore: update deps (#1634) ([libp2p/go-libp2p#1634](https://github.com/libp2p/go-libp2p/pull/1634))\n - Pass endpoint information to resource manager's OpenConnection (#1633) ([libp2p/go-libp2p#1633](https://github.com/libp2p/go-libp2p/pull/1633))\n - Add canonical peer status logs (#1624) ([libp2p/go-libp2p#1624](https://github.com/libp2p/go-libp2p/pull/1624))\n - move go-libp2p-circuit here ([libp2p/go-libp2p#1626](https://github.com/libp2p/go-libp2p/pull/1626))\n - swarm: fix logging of accepted connections (#1629) ([libp2p/go-libp2p#1629](https://github.com/libp2p/go-libp2p/pull/1629))\n - fix: deny connections to peers in the right place (#1627) ([libp2p/go-libp2p#1627](https://github.com/libp2p/go-libp2p/pull/1627))\n - ping: fix flaky test (#1617) ([libp2p/go-libp2p#1617](https://github.com/libp2p/go-libp2p/pull/1617))\n - chore: use the new multiaddr.Contains function (#1618) ([libp2p/go-libp2p#1618](https://github.com/libp2p/go-libp2p/pull/1618))\n - chore: stop using the deprecated mux.MuxedConn (#1614) ([libp2p/go-libp2p#1614](https://github.com/libp2p/go-libp2p/pull/1614))\n - logging: Add canonical log for misbehaving peers (#1600) ([libp2p/go-libp2p#1600](https://github.com/libp2p/go-libp2p/pull/1600))\n - use multiaddr ipcidr to parse multiaddr filters (#1606) ([libp2p/go-libp2p#1606](https://github.com/libp2p/go-libp2p/pull/1606))\n - tcp: unexport TcpTransport.Upgrader (#1596) ([libp2p/go-libp2p#1596](https://github.com/libp2p/go-libp2p/pull/1596))\n - muxer: expose func to create MuxedConn from backing Conn (#1609) ([libp2p/go-libp2p#1609](https://github.com/libp2p/go-libp2p/pull/1609))\n - remove legacy mDNS implementation (#1192) ([libp2p/go-libp2p#1192](https://github.com/libp2p/go-libp2p/pull/1192))\n - feat: allow dialing wss peers using DNS multiaddrs\n - fix natManager to close natManager.nat (#1468) ([libp2p/go-libp2p#1468](https://github.com/libp2p/go-libp2p/pull/1468))\n - Expose DefaultPerPeerRateLimit as var (#1580) ([libp2p/go-libp2p#1580](https://github.com/libp2p/go-libp2p/pull/1580))\n - swarm: add ListenClose (#1586) ([libp2p/go-libp2p#1586](https://github.com/libp2p/go-libp2p/pull/1586))\n - identify: Fix flaky tests (#1555) ([libp2p/go-libp2p#1555](https://github.com/libp2p/go-libp2p/pull/1555))\n - autonat: fix flaky TestAutoNATPrivate (#1581) ([libp2p/go-libp2p#1581](https://github.com/libp2p/go-libp2p/pull/1581))\n - pstoremanager: fix test timeout (#1588) ([libp2p/go-libp2p#1588](https://github.com/libp2p/go-libp2p/pull/1588))\n - swarm: send notifications synchronously (#1562) ([libp2p/go-libp2p#1562](https://github.com/libp2p/go-libp2p/pull/1562))\n - basichost: fix flaky TestSignedPeerRecordWithNoListenAddrs (#1559) ([libp2p/go-libp2p#1559](https://github.com/libp2p/go-libp2p/pull/1559))\n - identify: fix flaky TestIdentifyDeltaOnProtocolChange (again) (#1582) ([libp2p/go-libp2p#1582](https://github.com/libp2p/go-libp2p/pull/1582))\n - tls: fix flaky TestInvalidCerts on Windows ([libp2p/go-libp2p#1560](https://github.com/libp2p/go-libp2p/pull/1560))\n - chore: log autorelay start failure error ([libp2p/go-libp2p#1583](https://github.com/libp2p/go-libp2p/pull/1583))\n - Add sanity check assertion (#1570) ([libp2p/go-libp2p#1570](https://github.com/libp2p/go-libp2p/pull/1570))\n - swarm: speed up the TestDialWorkerLoopConcurrentFailureStress test (#1573) ([libp2p/go-libp2p#1573](https://github.com/libp2p/go-libp2p/pull/1573))\n - chore: update examples to go-libp2p v0.20.0 (#1557) ([libp2p/go-libp2p#1557](https://github.com/libp2p/go-libp2p/pull/1557))\n - Wait a couple seconds for ID event (#1568) ([libp2p/go-libp2p#1568](https://github.com/libp2p/go-libp2p/pull/1568))\n - remove workspace and packages section from README (#1563) ([libp2p/go-libp2p#1563](https://github.com/libp2p/go-libp2p/pull/1563))\n - fix: mkreleaselog exclude autogenerated files (#1567) ([libp2p/go-libp2p#1567](https://github.com/libp2p/go-libp2p/pull/1567))\n - move resource manager integration tests to p2p/test/ (#1561) ([libp2p/go-libp2p#1561](https://github.com/libp2p/go-libp2p/pull/1561))\n - swarm: only dial a single transport in TestDialWorkerLoopBasic (#1526) ([libp2p/go-libp2p#1526](https://github.com/libp2p/go-libp2p/pull/1526))\n- github.com/libp2p/go-libp2p-core (v0.16.1 -> v0.19.1):\n - Update version.json\n - Remove btcsuite/btcd dep (#272) ([libp2p/go-libp2p-core#272](https://github.com/libp2p/go-libp2p-core/pull/272))\n - Release v0.19.0 (#271) ([libp2p/go-libp2p-core#271](https://github.com/libp2p/go-libp2p-core/pull/271))\n - Add endpoint parameter to the OpenConnection method for ResourceManager (#257) ([libp2p/go-libp2p-core#257](https://github.com/libp2p/go-libp2p-core/pull/257))\n - Release v0.18.0 (#270) ([libp2p/go-libp2p-core#270](https://github.com/libp2p/go-libp2p-core/pull/270))\n - Add canonical peer status logging with sampling (#269) ([libp2p/go-libp2p-core#269](https://github.com/libp2p/go-libp2p-core/pull/269))\n - canonicallog: reduce log level to warning (#268) ([libp2p/go-libp2p-core#268](https://github.com/libp2p/go-libp2p-core/pull/268))\n - Only log once if we failed to convert from netAddr (#264) ([libp2p/go-libp2p-core#264](https://github.com/libp2p/go-libp2p-core/pull/264))\n - remove deprecated mux package (#265) ([libp2p/go-libp2p-core#265](https://github.com/libp2p/go-libp2p-core/pull/265))\n - remove the peer.Set (#261) ([libp2p/go-libp2p-core#261](https://github.com/libp2p/go-libp2p-core/pull/261))\n - Bump version (#259) ([libp2p/go-libp2p-core#259](https://github.com/libp2p/go-libp2p-core/pull/259))\n - Add canonical log for misbehaving peers (#258) ([libp2p/go-libp2p-core#258](https://github.com/libp2p/go-libp2p-core/pull/258))\n- github.com/libp2p/go-libp2p-kad-dht (v0.16.0 -> v0.17.0):\n - Chore: bump version to v0.17.0\n - Update go-libp2p to v0.20.3 ([libp2p/go-libp2p-kad-dht#778](https://github.com/libp2p/go-libp2p-kad-dht/pull/778))\n- github.com/libp2p/go-libp2p-peerstore (v0.6.0 -> v0.7.1):\n - Release v0.7.1 ([libp2p/go-libp2p-peerstore#202](https://github.com/libp2p/go-libp2p-peerstore/pull/202))\n - stop using the peer.Set (#201) ([libp2p/go-libp2p-peerstore#201](https://github.com/libp2p/go-libp2p-peerstore/pull/201))\n - feat: Use a clock interface in pstoreds as well ([libp2p/go-libp2p-peerstore#200](https://github.com/libp2p/go-libp2p-peerstore/pull/200))\n - feat: use a clock interface to better support testing for pstoremem ([libp2p/go-libp2p-peerstore#199](https://github.com/libp2p/go-libp2p-peerstore/pull/199))\n - pstoremem: fix slice preallocation in GetProtocols (#198) ([libp2p/go-libp2p-peerstore#198](https://github.com/libp2p/go-libp2p-peerstore/pull/198))\n - remove all calls to peer.ID.Validate ([libp2p/go-libp2p-peerstore#194](https://github.com/libp2p/go-libp2p-peerstore/pull/194))\n - remove the addr package ([libp2p/go-libp2p-peerstore#195](https://github.com/libp2p/go-libp2p-peerstore/pull/195))\n - move AddrList to pstoremen, unexport it ([libp2p/go-libp2p-peerstore#193](https://github.com/libp2p/go-libp2p-peerstore/pull/193))\n - optimize allocations in the memory address book ([libp2p/go-libp2p-peerstore#191](https://github.com/libp2p/go-libp2p-peerstore/pull/191))\n - implement a clean shutdown for the memory address book ([libp2p/go-libp2p-peerstore#192](https://github.com/libp2p/go-libp2p-peerstore/pull/192))\n- github.com/libp2p/go-libp2p-resource-manager (v0.3.0 -> v0.5.3):\n - Chore: release patch v0.5.3 ([libp2p/go-libp2p-resource-manager#77](https://github.com/libp2p/go-libp2p-resource-manager/pull/77))\n - Add namespace to metrics ([libp2p/go-libp2p-resource-manager#79](https://github.com/libp2p/go-libp2p-resource-manager/pull/79))\n - Fix usage of make to reserve capacity, not values ([libp2p/go-libp2p-resource-manager#76](https://github.com/libp2p/go-libp2p-resource-manager/pull/76))\n - Add package docs ([libp2p/go-libp2p-resource-manager#75](https://github.com/libp2p/go-libp2p-resource-manager/pull/75))\n - chore: Release v0.5.2 ([libp2p/go-libp2p-resource-manager#74](https://github.com/libp2p/go-libp2p-resource-manager/pull/74))\n - Record which direction the resource was blocked ([libp2p/go-libp2p-resource-manager#72](https://github.com/libp2p/go-libp2p-resource-manager/pull/72))\n - Simplify mem graphs in stock Grafana dashboard ([libp2p/go-libp2p-resource-manager#73](https://github.com/libp2p/go-libp2p-resource-manager/pull/73))\n - feat: Handle multiple instances in stock Grafana dashboard ([libp2p/go-libp2p-resource-manager#70](https://github.com/libp2p/go-libp2p-resource-manager/pull/70))\n - Use templated version of Grafana dashboard json ([libp2p/go-libp2p-resource-manager#69](https://github.com/libp2p/go-libp2p-resource-manager/pull/69))\n - Release v0.5.1 ([libp2p/go-libp2p-resource-manager#66](https://github.com/libp2p/go-libp2p-resource-manager/pull/66))\n - Implement `json.Marshaler` interface for LimitConfig ([libp2p/go-libp2p-resource-manager#67](https://github.com/libp2p/go-libp2p-resource-manager/pull/67))\n - Don't wait for a chan that will never close ([libp2p/go-libp2p-resource-manager#65](https://github.com/libp2p/go-libp2p-resource-manager/pull/65))\n - release v0.5.0 ([libp2p/go-libp2p-resource-manager#60](https://github.com/libp2p/go-libp2p-resource-manager/pull/60))\n - Add docs around WithAllowlistedMultiaddrs. Expose allowlist ([libp2p/go-libp2p-resource-manager#63](https://github.com/libp2p/go-libp2p-resource-manager/pull/63))\n - fix marshalling of allowlisted scopes ([libp2p/go-libp2p-resource-manager#62](https://github.com/libp2p/go-libp2p-resource-manager/pull/62))\n - docs: describe how the limiter is configured, and how limits are scaled (#59) ([libp2p/go-libp2p-resource-manager#59](https://github.com/libp2p/go-libp2p-resource-manager/pull/59))\n - don't limit the number of FDs on Windows (#58) ([libp2p/go-libp2p-resource-manager#58](https://github.com/libp2p/go-libp2p-resource-manager/pull/58))\n - Add ability to configure allowlist limits ([libp2p/go-libp2p-resource-manager#57](https://github.com/libp2p/go-libp2p-resource-manager/pull/57))\n - rewrite limits to allow auto-scaling ([libp2p/go-libp2p-resource-manager#48](https://github.com/libp2p/go-libp2p-resource-manager/pull/48))\n - Release v0.4.0 ([libp2p/go-libp2p-resource-manager#56](https://github.com/libp2p/go-libp2p-resource-manager/pull/56))\n - feat: Out of the box metrics for resource manager ([libp2p/go-libp2p-resource-manager#54](https://github.com/libp2p/go-libp2p-resource-manager/pull/54))\n - feat: Allowlist ([libp2p/go-libp2p-resource-manager#47](https://github.com/libp2p/go-libp2p-resource-manager/pull/47))\n - trace the scope as a JSON object (#52) ([libp2p/go-libp2p-resource-manager#52](https://github.com/libp2p/go-libp2p-resource-manager/pull/52))\n - include current limits in debug messages ([libp2p/go-libp2p-resource-manager#42](https://github.com/libp2p/go-libp2p-resource-manager/pull/42))\n - add an ID to spans (#44) ([libp2p/go-libp2p-resource-manager#44](https://github.com/libp2p/go-libp2p-resource-manager/pull/44))\n - add a DefaultLimitConfig with infinite limits (#41) ([libp2p/go-libp2p-resource-manager#41](https://github.com/libp2p/go-libp2p-resource-manager/pull/41))\n - export the TraceEvt (#40) ([libp2p/go-libp2p-resource-manager#40](https://github.com/libp2p/go-libp2p-resource-manager/pull/40))\n - trace exact timestamps (#39) ([libp2p/go-libp2p-resource-manager#39](https://github.com/libp2p/go-libp2p-resource-manager/pull/39))\n - skip events that don't change anything in tracer (#38) ([libp2p/go-libp2p-resource-manager#38](https://github.com/libp2p/go-libp2p-resource-manager/pull/38))\n - fix typos in MetricsReporter docs\n - fix shadowing of service name (#37) ([libp2p/go-libp2p-resource-manager#37](https://github.com/libp2p/go-libp2p-resource-manager/pull/37))\n - add a timestamp to trace events (#34) ([libp2p/go-libp2p-resource-manager#34](https://github.com/libp2p/go-libp2p-resource-manager/pull/34))\n- github.com/libp2p/go-libp2p-testing (v0.9.2 -> v0.11.0):\n - Release v0.11.0 ([libp2p/go-libp2p-testing#64](https://github.com/libp2p/go-libp2p-testing/pull/64))\n - Remove unused bench file and dep ([libp2p/go-libp2p-testing#63](https://github.com/libp2p/go-libp2p-testing/pull/63))\n - Release v0.10.0 ([libp2p/go-libp2p-testing#62](https://github.com/libp2p/go-libp2p-testing/pull/62))\n - Update go-libp2p-core dep ([libp2p/go-libp2p-testing#61](https://github.com/libp2p/go-libp2p-testing/pull/61))\n - remove suites (#60) ([libp2p/go-libp2p-testing#60](https://github.com/libp2p/go-libp2p-testing/pull/60))\n - don't continue on read / write error in stream suite (#59) ([libp2p/go-libp2p-testing#59](https://github.com/libp2p/go-libp2p-testing/pull/59))\n - remove debug logging from stream and muxer suite ([libp2p/go-libp2p-testing#58](https://github.com/libp2p/go-libp2p-testing/pull/58))\n - remove Travis package (#57) ([libp2p/go-libp2p-testing#57](https://github.com/libp2p/go-libp2p-testing/pull/57))\n- github.com/lucas-clemente/quic-go (v0.27.1 -> v0.28.0):\n - update for Go 1.19beta1 (#3460) ([lucas-clemente/quic-go#3460](https://github.com/lucas-clemente/quic-go/pull/3460))\n - Deduplicate Alt-Svc header values (#3461) ([lucas-clemente/quic-go#3461](https://github.com/lucas-clemente/quic-go/pull/3461))\n - only set DF for sockets that can handle it (#3448) ([lucas-clemente/quic-go#3448](https://github.com/lucas-clemente/quic-go/pull/3448))\n - fix flaky HTTP/3 request body test (#3447) ([lucas-clemente/quic-go#3447](https://github.com/lucas-clemente/quic-go/pull/3447))\n - make the keep alive interval configurable (#3444) ([lucas-clemente/quic-go#3444](https://github.com/lucas-clemente/quic-go/pull/3444))\n - implement QUIC v2 ([lucas-clemente/quic-go#3432](https://github.com/lucas-clemente/quic-go/pull/3432))\n - allow HTTP clients and servers to take over the request stream ([lucas-clemente/quic-go#3437](https://github.com/lucas-clemente/quic-go/pull/3437))\n - remove the http3.DataStreamer (#3435) ([lucas-clemente/quic-go#3435](https://github.com/lucas-clemente/quic-go/pull/3435))\n - always reset header buffer, even when QPACK encoding fails (#3436) ([lucas-clemente/quic-go#3436](https://github.com/lucas-clemente/quic-go/pull/3436))\n - Change \"HTTP/3\" to \"HTTP/3.0\". (#3439) ([lucas-clemente/quic-go#3439](https://github.com/lucas-clemente/quic-go/pull/3439))\n - remove stray http3 connection file\n - pass frame / stream type parsing errors to the hijacker callbacks ([lucas-clemente/quic-go#3429](https://github.com/lucas-clemente/quic-go/pull/3429))\n - add test for bidirectional stream hijacker (#3434) ([lucas-clemente/quic-go#3434](https://github.com/lucas-clemente/quic-go/pull/3434))\n - make it possible to parse a varint at the end of a reader (#3428) ([lucas-clemente/quic-go#3428](https://github.com/lucas-clemente/quic-go/pull/3428))\n - don't ignore errors that occur when the TLS ClientHello is generated ([lucas-clemente/quic-go#3424](https://github.com/lucas-clemente/quic-go/pull/3424))\n - don't send path MTU probe packets on a timer (#3423) ([lucas-clemente/quic-go#3423](https://github.com/lucas-clemente/quic-go/pull/3423))\n - introduce a http3.RoundTripOpt to prevent closing of request stream (#3411) ([lucas-clemente/quic-go#3411](https://github.com/lucas-clemente/quic-go/pull/3411))\n - don't close the request stream when http3.DataStreamer was used (#3413) ([lucas-clemente/quic-go#3413](https://github.com/lucas-clemente/quic-go/pull/3413))\n - do not embed http.Server in http3.Server (#3397) ([lucas-clemente/quic-go#3397](https://github.com/lucas-clemente/quic-go/pull/3397))\n - remove error return value from ComposeVersionNegotiation (#3410) ([lucas-clemente/quic-go#3410](https://github.com/lucas-clemente/quic-go/pull/3410))\n - don't set receive buffer if it is already large enough (#3407) ([lucas-clemente/quic-go#3407](https://github.com/lucas-clemente/quic-go/pull/3407))\n - clone TLS conf in newClient (#3400) ([lucas-clemente/quic-go#3400](https://github.com/lucas-clemente/quic-go/pull/3400))\n - remove warning comments of stable implementation (#3399) ([lucas-clemente/quic-go#3399](https://github.com/lucas-clemente/quic-go/pull/3399))\n - fix parsing of request path for Extended CONNECT requests (#3388) ([lucas-clemente/quic-go#3388](https://github.com/lucas-clemente/quic-go/pull/3388))\n - update docs to reflect that we support RFC 9221 (Unreliable Datagrams) (#3382) ([lucas-clemente/quic-go#3382](https://github.com/lucas-clemente/quic-go/pull/3382))\n - fix deadlock on concurrent http3.Server.Serve and Close calls (#3387) ([lucas-clemente/quic-go#3387](https://github.com/lucas-clemente/quic-go/pull/3387))\n - reduce flakiness of deadline integration tests (#3383) ([lucas-clemente/quic-go#3383](https://github.com/lucas-clemente/quic-go/pull/3383))\n - protect against concurrent use of Stream.Write (#3381) ([lucas-clemente/quic-go#3381](https://github.com/lucas-clemente/quic-go/pull/3381))\n - protect against concurrent use of Stream.Read (#3380) ([lucas-clemente/quic-go#3380](https://github.com/lucas-clemente/quic-go/pull/3380))\n - Expose quic server closed err (#3395) ([lucas-clemente/quic-go#3395](https://github.com/lucas-clemente/quic-go/pull/3395))\n - implement HTTP/3 unidirectional stream hijacking (#3389) ([lucas-clemente/quic-go#3389](https://github.com/lucas-clemente/quic-go/pull/3389))\n - add LocalAddr and RemoteAddr functions to http3.StreamCreator (#3384) ([lucas-clemente/quic-go#3384](https://github.com/lucas-clemente/quic-go/pull/3384))\n - extend the HTTP/3 API for WebTransport support ([lucas-clemente/quic-go#3362](https://github.com/lucas-clemente/quic-go/pull/3362))\n - remove unneeded network from custom dial function used in HTTP/3 (#3368) ([lucas-clemente/quic-go#3368](https://github.com/lucas-clemente/quic-go/pull/3368))\n- github.com/multiformats/go-multiaddr (v0.5.0 -> v0.6.0):\n - release v0.6.0 ([multiformats/go-multiaddr#178](https://github.com/multiformats/go-multiaddr/pull/178))\n - add WebTransport multiaddr components ([multiformats/go-multiaddr#176](https://github.com/multiformats/go-multiaddr/pull/176))\n - add ipcidr support (#177) ([multiformats/go-multiaddr#177](https://github.com/multiformats/go-multiaddr/pull/177))\n - add a Contains function (#172) ([multiformats/go-multiaddr#172](https://github.com/multiformats/go-multiaddr/pull/172))\n- github.com/multiformats/go-multibase (v0.1.0 -> v0.1.1):\n - chore: release version 0.1.1\n - fix: add new emoji codepoint for Base256Emoji 🐉\n- github.com/multiformats/go-multihash (v0.2.0 -> v0.2.1):\n - chore: release v0.2.1\n - feat: adding tests and finish variable sized functions\n - feat: add support for variable length hash functions\n - adding blake3 tests and fixing an incorrect error message. (#158) ([multiformats/go-multihash#158](https://github.com/multiformats/go-multihash/pull/158))\n\n
\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 129 | +5612/-9895 | 345 |\n| Marco Munizaga | 109 | +7689/-3221 | 181 |\n| vyzo | 64 | +3972/-657 | 125 |\n| Jorropo | 19 | +1977/-1611 | 109 |\n| Steven Allen | 30 | +633/-593 | 54 |\n| Jeromy Johnson | 5 | +1032/-64 | 16 |\n| Marcin Rataj | 21 | +406/-200 | 59 |\n| Michael Muré | 6 | +335/-250 | 14 |\n| Gus Eggert | 8 | +336/-104 | 31 |\n| Claudia Richoux | 3 | +181/-63 | 9 |\n| Steve Loeppky | 11 | +95/-141 | 11 |\n| Ian Davis | 4 | +126/-58 | 6 |\n| hareku | 3 | +172/-6 | 7 |\n| Ivan Trubach | 1 | +98/-74 | 6 |\n| Raúl Kripalani | 2 | +69/-62 | 9 |\n| Seungbae Yu | 1 | +41/-41 | 13 |\n| Julien Muret | 1 | +60/-7 | 2 |\n| Mark Gaiser | 1 | +64/-0 | 5 |\n| Lars Gierth | 1 | +20/-29 | 4 |\n| Cole Brown | 4 | +27/-19 | 4 |\n| Chao Fei | 2 | +15/-30 | 9 |\n| Nuno Diegues | 2 | +25/-18 | 9 |\n| Jakub Sztandera | 1 | +37/-0 | 3 |\n| Wiktor Jurkiewicz | 1 | +13/-5 | 1 |\n| c r | 1 | +11/-6 | 3 |\n| Christian Stewart | 1 | +15/-2 | 4 |\n| Matt Robenolt | 1 | +15/-1 | 2 |\n| aarshkshah1992 | 2 | +8/-2 | 2 |\n| link2xt | 1 | +4/-4 | 1 |\n| Aaron Riekenberg | 1 | +4/-4 | 4 |\n| web3-bot | 3 | +7/-0 | 3 |\n| Adrian Lanzafame | 1 | +3/-3 | 1 |\n| Dmitriy Ryajov | 2 | +2/-3 | 2 |\n| Brendan O'Brien | 1 | +5/-0 | 1 |\n| millken | 1 | +1/-1 | 1 |\n| lostystyg | 1 | +1/-1 | 1 |\n| kpcyrd | 1 | +1/-1 | 1 |\n| anders | 1 | +1/-1 | 1 |\n| Rod Vagg | 1 | +1/-1 | 1 |\n| Matt Joiner | 1 | +1/-1 | 1 |\n| Leo Balduf | 1 | +1/-1 | 1 |\n| Didrik Nordström | 1 | +2/-0 | 1 |\n| Daniel Norman | 1 | +1/-1 | 1 |\n| Antonio Navarro Perez | 1 | +1/-1 | 1 |\n| Adin Schmahmann | 1 | +1/-1 | 1 |\n| Lucas Molas | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.16.md", "content": "# Kubo changelog v0.16\n\n## v0.16.0\n\n### Overview\n\nBelow is an outline of all that is in this release, so you get a sense of all that's included.\n\n- [Kubo changelog v0.16](#kubo-changelog-v016)\n - [v0.16.0](#v0160)\n - [Overview](#overview)\n - [🔦 Highlights](#-highlights)\n - [🛣️ More configurable delegated routing system](#️-more-configurable-delegated-routing-system)\n - [🌍 WebTransport new experimental Transport](#-webtransport-new-experimental-transport)\n - [🗃️ Hardened IPNS record verification](#-hardened-ipns-record-verification)\n - [🌉 Web Gateways now support _redirects files](#-web-gateways-now-support-_redirects-files)\n - [😻 Add files to MFS with ipfs add --to-files](#-add-files-to-mfs-with-ipfs-add---to-files)\n - [Changelog](#changelog)\n - [Contributors](#contributors)\n\n\n### 🔦 Highlights\n\n\n\n#### 🛣️ More configurable delegated routing system\n\nSince Kubo v0.14.0 [Reframe protocol](https://github.com/ipfs/specs/tree/main/reframe#readme) has been supported as a new routing system.\n\nNow, we allow to configure several routers working together, so you can have several `reframe` and `dht` routers making queries. You can use the special `parallel` and `sequential` routers to fill your needs.\n\nExample configuration usage using the [Filecoin Network Indexer](https://docs.cid.contact/filecoin-network-indexer/overview) and the DHT, making first a query to the indexer, and timing out after 3 seconds.\n\n```console\n$ ipfs config Routing.Type --json '\"custom\"'\n\n$ ipfs config Routing.Routers.CidContact --json '{\n \"Type\": \"reframe\",\n \"Parameters\": {\n \"Endpoint\": \"https://cid.contact/reframe\"\n }\n}'\n\n$ ipfs config Routing.Routers.WanDHT --json '{\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"auto\",\n \"PublicIPNetwork\": true,\n \"AcceleratedDHTClient\": false\n }\n}'\n\n$ ipfs config Routing.Routers.ParallelHelper --json '{\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\" : \"CidContact\",\n \"IgnoreErrors\" : true,\n \"Timeout\": \"3s\"\n },\n {\n \"RouterName\" : \"WanDHT\",\n \"IgnoreErrors\" : false,\n \"Timeout\": \"5m\",\n \"ExecuteAfter\": \"2s\"\n }\n ]\n }\n}'\n\n$ ipfs config Routing.Methods --json '{\n \"find-peers\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"find-providers\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"get-ipns\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"provide\": {\n \"RouterName\": \"WanDHT\"\n },\n \"put-ipns\": {\n \"RouterName\": \"ParallelHelper\"\n }\n }'\n\n```\n\n### 🌍 WebTransport new experimental Transport\n\nA new feature of [`go-libp2p`](https://github.com/libp2p/go-libp2p/releases/tag/v0.23.0) is [WebTransport](https://github.com/libp2p/go-libp2p/issues/1717).\n\nFor now it is **disabled by default** and considered **experimental**.\nIf you find issues running it please [report them to us](https://github.com/ipfs/kubo/issues/new).\n\nIn the future Kubo will listen on WebTransport by default for anyone already listening on QUIC addresses.\n\nWebTransport is a new transport protocol currently under development by the [IETF](https://datatracker.ietf.org/wg/webtrans/about/) and the [W3C](https://www.w3.org/TR/webtransport/), and [already implemented by Chrome](https://caniuse.com/webtransport).\nConceptually, it’s like WebSocket run over QUIC instead of TCP. Most importantly, it allows browsers to establish (secure!) connections to WebTransport servers without the need for CA-signed certificates,\nthereby enabling any js-libp2p node running in a browser to connect to any kubo node, with zero manual configuration involved.\n\nThe previous alternative is websocket secure, which require installing a reverse proxy and TLS certificates manually.\n\n#### How to enable WebTransport\n\nThose steps are temporary and won't be needed once we make it enabled by default.\n\n1. Enable the WebTransport transport:\n `ipfs config Swarm.Transports.Network.WebTransport --json true`\n1. Add a listener address for WebTransport to your `Addresses.Swarm` key, for example:\n ```json\n [\n \"/ip4/0.0.0.0/tcp/4001\",\n \"/ip4/0.0.0.0/udp/4001/quic\",\n \"/ip4/0.0.0.0/udp/4002/quic/webtransport\"\n ]\n ```\n1. Restart your daemon to apply the config changes.\n\n### 🗃️ Hardened IPNS record verification\n\nRecords that do not have a valid IPNS V2 signature, or exceed the max size\nlimit, will no longer pass verification, and will be ignored by Kubo when\nresolving `/ipns/{libp2p-key}` content paths.\n\nKubo continues publishing backward-compatible V1+V2 records that can be\nresolved by V1-only (go-ipfs <0.9.0) clients.\n\nMore details can be found in _Backward Compatibility_, _Record Creation_, and\n_Record Verification_ sections of the [updated IPNS\nspecification](https://github.com/ipfs/specs/pull/319/files).\n\n### 🌉 Web Gateways now support `_redirects` files\n\nThis feature enables support for redirects, single-page applications (SPA),\ncustom 404 pages, and moving to IPFS-backed website hosting\n[without breaking existing HTTP links](https://www.w3.org/Provider/Style/URI).\n\nIt is limited to websites hosted in web contexts with unique\n[Origins](https://en.wikipedia.org/wiki/Same-origin_policy), such as\n[subdomain](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway) and\n[DNSLink](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#dnslink-gateway) gateways.\nRedirect logic is evaluated only if the requested path is not in the DAG.\n\nSee more details and usage examples see\n[docs.ipfs.tech: _Redirects, custom 404s, and SPA support_](https://docs.ipfs.tech/how-to/websites-on-ipfs/redirects-and-custom-404s/).\n\n### 😻 Add files to MFS with `ipfs add --to-files`\n\nUsers no longer need to call `ipfs files cp` after `ipfs add` to create a\nreference in [MFS](https://docs.ipfs.tech/concepts/glossary/#mfs), or deal with\nlow level pins if they do not wish to do so. It is now possible to pass MFS\npath in an optional `--to-files` to add data directly to MFS, without creating\na low level pin.\n\nBefore (Kubo <0.16.0):\n\n\n```console\n$ ipfs add cat.jpg\nQmCID\n$ ipfs files cp /ipfs/QmCID /mfs-cats/cat.jpg\n$ ipfs pin rm QmCID # removing low level pin, since MFS is protecting from gc\n```\n\nKubo 0.16.0 collapses the above steps into one:\n\n```console\n$ ipfs add --pin=false cat.jpg --to-files /mfs-cats/\n```\n\nA recursive add to MFS works too (below line will create `/lots-of-cats/` directory in MFS):\n\n```console\n$ ipfs add -r ./lots-of-cats/ --to-files /\n```\n\nFor more information, see `ipfs add --help` and `ipfs files --help`.\n\n### Changelog\n\n
\nFull Changelog\n\n- github.com/ipfs/kubo:\n - fix: Set default Methods value to nil\n - docs: document remaining 0.16.0 features\n - docs: add WebTransport docs ([ipfs/kubo#9308](https://github.com/ipfs/kubo/pull/9308))\n - chore: bump version to 0.16.0-rc1\n - fix: ensure hasher is registered when using a hashing function\n - feat: add webtransport as an option transport ([ipfs/kubo#9293](https://github.com/ipfs/kubo/pull/9293))\n - feat(gateway): _redirects file support (#8890) ([ipfs/kubo#8890](https://github.com/ipfs/kubo/pull/8890))\n - docs: fix typo in changelog-v0.16.0.md\n - Readme: Rewrite introduction and featureset (#9211) ([ipfs/kubo#9211](https://github.com/ipfs/kubo/pull/9211))\n - feat: Delegated routing with custom configuration. (#9274) ([ipfs/kubo#9274](https://github.com/ipfs/kubo/pull/9274))\n - Add to `ipfs id -h` options (#9229) ([ipfs/kubo#9229](https://github.com/ipfs/kubo/pull/9229))\n - chore: bump go-libp2p v0.23.1 ([ipfs/kubo#9285](https://github.com/ipfs/kubo/pull/9285))\n - feat(cmds/add): --to-files option automates files cp (#8927) ([ipfs/kubo#8927](https://github.com/ipfs/kubo/pull/8927))\n - docs: fix broken ENS DoH example (#9281) ([ipfs/kubo#9281](https://github.com/ipfs/kubo/pull/9281))\n - ([ipfs/kubo#9258](https://github.com/ipfs/kubo/pull/9258))\n - ([ipfs/kubo#9213](https://github.com/ipfs/kubo/pull/9213))\n - docs: small typo in Dockerfile\n - feat: ipfs-webui v2.18.1\n - feat: ipfs-webui v2.18.0 (#9262) ([ipfs/kubo#9262](https://github.com/ipfs/kubo/pull/9262))\n - bump go-libp2p v0.22.0 & go1.18&go1.19 ([ipfs/kubo#9244](https://github.com/ipfs/kubo/pull/9244))\n - docs: change windows choco install command to point to go-ipfs\n - fix: pass the repo directory into the ignored_commit function\n - docs(cmds): daemon: update DHTClient description\n - fix(gw): send 200 for empty files\n - docs(readme): official vs unofficial packages\n - chore: remove Gateway.PathPrefixes\n - docs(readme): update Docker section\n - docs: fix markdown syntax typo in v0.15's changelog\n - chore: Release v0.15.0 ([ipfs/kubo#9236](https://github.com/ipfs/kubo/pull/9236))\n - chore: fix undiallable api and gateway files ([ipfs/kubo#9233](https://github.com/ipfs/kubo/pull/9233))\n - chore: Bump version to 0.16.0-dev\n- github.com/ipfs/go-bitswap (v0.9.0 -> v0.10.2):\n - chore: release v0.10.2\n - fix: create a copy of the protocol slice in network.processSettings\n - chore: release v0.10.1\n - fix: incorrect type in the WithTracer polyfill option\n - chore: fix incorrect log message when a bad option is passed\n - chore: release v0.10.0\n - chore: update go-libp2p v0.22.0\n- github.com/ipfs/go-cid (v0.2.0 -> v0.3.2):\n - chore: release v0.3.2\n - Revert \"fix: bring back, but deprecate CodecToStr and Codecs\"\n - chore: release v0.2.1\n - fix: bring back, but deprecate CodecToStr and Codecs\n - run gofmt -s\n - bump go.mod to Go 1.18 and run go fix\n - chore: release v0.3.0\n - fix: return nil Bytes() if the Cid in undef\n - Add MustParse ([ipfs/go-cid#139](https://github.com/ipfs/go-cid/pull/139))\n- github.com/ipfs/go-datastore (v0.5.1 -> v0.6.0):\n - Release v0.6.0 (#194) ([ipfs/go-datastore#194](https://github.com/ipfs/go-datastore/pull/194))\n - feat: add Features + datastore scoping\n - chore: Fix comment typo (#191) ([ipfs/go-datastore#191](https://github.com/ipfs/go-datastore/pull/191))\n- github.com/ipfs/go-delegated-routing (v0.3.0 -> v0.6.0):\n - ([ipfs/go-delegated-routing#53](https://github.com/ipfs/go-delegated-routing/pull/53))\n - ([ipfs/go-delegated-routing#52](https://github.com/ipfs/go-delegated-routing/pull/52))\n - Release v0.5.2 (#50) ([ipfs/go-delegated-routing#50](https://github.com/ipfs/go-delegated-routing/pull/50))\n - Fixed serialisation issue with multiadds (#49) ([ipfs/go-delegated-routing#49](https://github.com/ipfs/go-delegated-routing/pull/49))\n - Upgrade to IPLD `0.18.0`\n - Release v0.5.0 (#47) ([ipfs/go-delegated-routing#47](https://github.com/ipfs/go-delegated-routing/pull/47))\n - feat: use GET for FindProviders (#46) ([ipfs/go-delegated-routing#46](https://github.com/ipfs/go-delegated-routing/pull/46))\n - Update provide to take an array of keys, per spec (#45) ([ipfs/go-delegated-routing#45](https://github.com/ipfs/go-delegated-routing/pull/45))\n - ([ipfs/go-delegated-routing#44](https://github.com/ipfs/go-delegated-routing/pull/44))\n - fix: upgrade edelweiss and rerun 'go generate' (#42) ([ipfs/go-delegated-routing#42](https://github.com/ipfs/go-delegated-routing/pull/42))\n - ci: add check to ensure generated files are up-to-date (#41) ([ipfs/go-delegated-routing#41](https://github.com/ipfs/go-delegated-routing/pull/41))\n - Add Provide RPC (#37) ([ipfs/go-delegated-routing#37](https://github.com/ipfs/go-delegated-routing/pull/37))\n - upgrade to go-log/v2 (#34) ([ipfs/go-delegated-routing#34](https://github.com/ipfs/go-delegated-routing/pull/34))\n- github.com/ipfs/go-ipns (v0.1.2 -> v0.3.0):\n - fix: require V2 signatures ([ipfs/go-ipns#41](https://github.com/ipfs/go-ipns/pull/41))\n - update go-libp2p to v0.22.0, release v0.2.0 (#39) ([ipfs/go-ipns#39](https://github.com/ipfs/go-ipns/pull/39))\n - use peer.IDFromBytes instead of peer.IDFromString (#38) ([ipfs/go-ipns#38](https://github.com/ipfs/go-ipns/pull/38))\n - sync: update CI config files (#34) ([ipfs/go-ipns#34](https://github.com/ipfs/go-ipns/pull/34))\n- github.com/ipfs/go-pinning-service-http-client (v0.1.1 -> v0.1.2):\n - chore: release v0.1.2\n - fix: send up to nanosecond precision\n - refactor: cleanup Sprintf for Bearer token\n - sync: update CI config files ([ipfs/go-pinning-service-http-client#21](https://github.com/ipfs/go-pinning-service-http-client/pull/21))\n- github.com/ipld/edelweiss (v0.1.4 -> v0.2.0):\n - Release v0.2.0 (#60) ([ipld/edelweiss#60](https://github.com/ipld/edelweiss/pull/60))\n - feat: add cacheable modifier to methods (#48) ([ipld/edelweiss#48](https://github.com/ipld/edelweiss/pull/48))\n - adding licenses (#52) ([ipld/edelweiss#52](https://github.com/ipld/edelweiss/pull/52))\n - sync: update CI config files ([ipld/edelweiss#56](https://github.com/ipld/edelweiss/pull/56))\n - chore: replace deprecated ioutil with io/os ([ipld/edelweiss#59](https://github.com/ipld/edelweiss/pull/59))\n - Release v0.1.6\n - fix: iterate over BlueMap in deterministic order (#57) ([ipld/edelweiss#57](https://github.com/ipld/edelweiss/pull/57))\n - fix: wrap DAG-JSON serialization error (#55) ([ipld/edelweiss#55](https://github.com/ipld/edelweiss/pull/55))\n - update examples and harness\n - upgrade to go-log/v2 (#53) ([ipld/edelweiss#53](https://github.com/ipld/edelweiss/pull/53))\n- github.com/ipld/go-ipld-prime (v0.17.0 -> v0.18.0):\n - Prepare v0.18.0\n - feat(bindnode): add a BindnodeRegistry utility (#437) ([ipld/go-ipld-prime#437](https://github.com/ipld/go-ipld-prime/pull/437))\n - feat(bindnode): support full uint64 range\n - chore(bindnode): remove typed functions for options\n - chore(bindnode): docs and minor tweaks\n - feat(bindnode): make Any converters work for List and Map values\n - fix(bindnode): shorten converter option names, minor perf improvements\n - fix(bindnode): only custom convert AssignNull for Any converter\n - feat(bindnode): pass Null on to nullable custom converters\n - chore(bindnode): config helper refactor w/ short-circuit\n - feat(bindnode): add AddCustomTypeAnyConverter() to handle `Any` fields\n - feat(bindnode): add AddCustomTypeXConverter() options for most scalar kinds\n - chore(bindnode): back out of reflection for converters\n - feat(bindnode): switch to converter functions instead of type\n - feat(bindnode): allow custom type conversions with options\n - feat: add release checklist (#442) ([ipld/go-ipld-prime#442](https://github.com/ipld/go-ipld-prime/pull/442))\n- github.com/libp2p/go-flow-metrics (v0.0.3 -> v0.1.0):\n - introduce an API to set a mock clock (#20) ([libp2p/go-flow-metrics#20](https://github.com/libp2p/go-flow-metrics/pull/20))\n - chore: skip slow tests when short testing is specified ([libp2p/go-flow-metrics#16](https://github.com/libp2p/go-flow-metrics/pull/16))\n- github.com/libp2p/go-libp2p (v0.21.0 -> v0.23.2):\n - release v0.23.2 (#1781) ([libp2p/go-libp2p#1781](https://github.com/libp2p/go-libp2p/pull/1781))\n - webtransport: return error before wrapping opened / accepted streams (#1775) ([libp2p/go-libp2p#1775](https://github.com/libp2p/go-libp2p/pull/1775))\n - release v0.23.1 (#1773) ([libp2p/go-libp2p#1773](https://github.com/libp2p/go-libp2p/pull/1773))\n - websocket: fix nil pointer in tlsClientConf (#1770) ([libp2p/go-libp2p#1770](https://github.com/libp2p/go-libp2p/pull/1770))\n - release v0.23.0 (#1764) ([libp2p/go-libp2p#1764](https://github.com/libp2p/go-libp2p/pull/1764))\n - noise: switch to proto2, use the new NoiseExtensions protobuf ([libp2p/go-libp2p#1762](https://github.com/libp2p/go-libp2p/pull/1762))\n - webtransport: add custom resolver to add SNI (#1761) ([libp2p/go-libp2p#1761](https://github.com/libp2p/go-libp2p/pull/1761))\n - swarm: skip dialing WebTransport addresses when we have QUIC addresses (#1756) ([libp2p/go-libp2p#1756](https://github.com/libp2p/go-libp2p/pull/1756))\n - webtransport: have the server send the certificates (#1757) ([libp2p/go-libp2p#1757](https://github.com/libp2p/go-libp2p/pull/1757))\n - noise: make it possible for the server to send early data (#1750) ([libp2p/go-libp2p#1750](https://github.com/libp2p/go-libp2p/pull/1750))\n - swarm: fix selection of transport for dialing (#1653) ([libp2p/go-libp2p#1653](https://github.com/libp2p/go-libp2p/pull/1653))\n - autorelay: Add a context.Context to WithPeerSource callback (#1736) ([libp2p/go-libp2p#1736](https://github.com/libp2p/go-libp2p/pull/1736))\n - webtransport: add and check the ?type=noise URL parameter (#1749) ([libp2p/go-libp2p#1749](https://github.com/libp2p/go-libp2p/pull/1749))\n - webtransport: disable HTTP origin check (#1752) ([libp2p/go-libp2p#1752](https://github.com/libp2p/go-libp2p/pull/1752))\n - noise: don't fail handshake when early data is received without handler (#1746) ([libp2p/go-libp2p#1746](https://github.com/libp2p/go-libp2p/pull/1746))\n - Add Resolver interface to transport (#1719) ([libp2p/go-libp2p#1719](https://github.com/libp2p/go-libp2p/pull/1719))\n - use new /libp2p/go-libp2p/core pkg (#1745) ([libp2p/go-libp2p#1745](https://github.com/libp2p/go-libp2p/pull/1745))\n - yamux: pass constructors for peer resource scopes to session constructor (#1739) ([libp2p/go-libp2p#1739](https://github.com/libp2p/go-libp2p/pull/1739))\n - tcp: add an option to enable metrics (disabled by default) (#1734) ([libp2p/go-libp2p#1734](https://github.com/libp2p/go-libp2p/pull/1734))\n - move go-libp2p-webtransport to p2p/transport/webtransport ([libp2p/go-libp2p#1737](https://github.com/libp2p/go-libp2p/pull/1737))\n - autorelay: fix race condition in TestBackoff (#1731) ([libp2p/go-libp2p#1731](https://github.com/libp2p/go-libp2p/pull/1731))\n - rcmgr: increase default connection memory limit to 32 MB (#1740) ([libp2p/go-libp2p#1740](https://github.com/libp2p/go-libp2p/pull/1740))\n - quic: update quic-go to v0.29.0 (#1723) ([libp2p/go-libp2p#1723](https://github.com/libp2p/go-libp2p/pull/1723))\n - noise: implement an API to send and receive early data ([libp2p/go-libp2p#1728](https://github.com/libp2p/go-libp2p/pull/1728))\n - identify: make the protocol version configurable (#1724) ([libp2p/go-libp2p#1724](https://github.com/libp2p/go-libp2p/pull/1724))\n - Fix threshold calculation (#1722) ([libp2p/go-libp2p#1722](https://github.com/libp2p/go-libp2p/pull/1722))\n - connmgr: use clock interface (#1720) ([libp2p/go-libp2p#1720](https://github.com/libp2p/go-libp2p/pull/1720))\n - quic: increase the buffer size used for encoding qlogs (#1715) ([libp2p/go-libp2p#1715](https://github.com/libp2p/go-libp2p/pull/1715))\n - quic: add a WithMetrics option (#1716) ([libp2p/go-libp2p#1716](https://github.com/libp2p/go-libp2p/pull/1716))\n - add default listen addresses for QUIC (#1615) ([libp2p/go-libp2p#1615](https://github.com/libp2p/go-libp2p/pull/1615))\n - feat: inject DNS resolver (#1607) ([libp2p/go-libp2p#1607](https://github.com/libp2p/go-libp2p/pull/1607))\n - connmgr: prefer peers with no streams when closing connections (#1675) ([libp2p/go-libp2p#1675](https://github.com/libp2p/go-libp2p/pull/1675))\n - quic: add DisableReuseport option (#1476) ([libp2p/go-libp2p#1476](https://github.com/libp2p/go-libp2p/pull/1476))\n - release v0.22.0 ([libp2p/go-libp2p#1688](https://github.com/libp2p/go-libp2p/pull/1688))\n - fix: don't prefer local ports from other addresses when dialing (#1673) ([libp2p/go-libp2p#1673](https://github.com/libp2p/go-libp2p/pull/1673))\n - crypto: add better support for alternative backends (#1686) ([libp2p/go-libp2p#1686](https://github.com/libp2p/go-libp2p/pull/1686))\n - crypto/secp256k1: Remove btcsuite intermediary. (#1689) ([libp2p/go-libp2p#1689](https://github.com/libp2p/go-libp2p/pull/1689))\n - Update resource manager README (#1684) ([libp2p/go-libp2p#1684](https://github.com/libp2p/go-libp2p/pull/1684))\n - move go-libp2p-core here ([libp2p/go-libp2p#1683](https://github.com/libp2p/go-libp2p/pull/1683))\n - rcmgr: make scaling changes more intuitive (#1685) ([libp2p/go-libp2p#1685](https://github.com/libp2p/go-libp2p/pull/1685))\n - move go-eventbus here ([libp2p/go-libp2p#1681](https://github.com/libp2p/go-libp2p/pull/1681))\n - basichost: remove usage of MultistreamServerMatcher in test (#1680) ([libp2p/go-libp2p#1680](https://github.com/libp2p/go-libp2p/pull/1680))\n - sync: update CI config files (#1678) ([libp2p/go-libp2p#1678](https://github.com/libp2p/go-libp2p/pull/1678))\n - move go-libp2p-resource-manager to p2p/host/resource-manager ([libp2p/go-libp2p#1677](https://github.com/libp2p/go-libp2p/pull/1677))\n - chore: preallocate slices with known final size (#1679) ([libp2p/go-libp2p#1679](https://github.com/libp2p/go-libp2p/pull/1679))\n - autorelay: fix flaky TestMaxAge (#1676) ([libp2p/go-libp2p#1676](https://github.com/libp2p/go-libp2p/pull/1676))\n - move go-libp2p-peerstore to p2p/host/peerstore ([libp2p/go-libp2p#1667](https://github.com/libp2p/go-libp2p/pull/1667))\n - examples: remove ipfs components from echo (#1672) ([libp2p/go-libp2p#1672](https://github.com/libp2p/go-libp2p/pull/1672))\n - chore: update libp2p to v0.21 in examples (#1674) ([libp2p/go-libp2p#1674](https://github.com/libp2p/go-libp2p/pull/1674))\n - change the default key type to Ed25519 (#1576) ([libp2p/go-libp2p#1576](https://github.com/libp2p/go-libp2p/pull/1576))\n - autorelay: poll for new candidates when needed ([libp2p/go-libp2p#1587](https://github.com/libp2p/go-libp2p/pull/1587))\n - examples: fix unresponsive pubsub chat example (#1652) ([libp2p/go-libp2p#1652](https://github.com/libp2p/go-libp2p/pull/1652))\n - routed: respect force direct dial context (#1665) ([libp2p/go-libp2p#1665](https://github.com/libp2p/go-libp2p/pull/1665))\n - pstoremanager: fix flaky TestClose (#1649) ([libp2p/go-libp2p#1649](https://github.com/libp2p/go-libp2p/pull/1649))\n - Allow adding prologue to noise connections (#1663) ([libp2p/go-libp2p#1663](https://github.com/libp2p/go-libp2p/pull/1663))\n - connmgr: add nowatchdog go build tag (#1666) ([libp2p/go-libp2p#1666](https://github.com/libp2p/go-libp2p/pull/1666))\n - mdns: don't discover ourselves (#1661) ([libp2p/go-libp2p#1661](https://github.com/libp2p/go-libp2p/pull/1661))\n - Support generating custom x509 certificates (#1481) ([libp2p/go-libp2p#1481](https://github.com/libp2p/go-libp2p/pull/1481))\n- github.com/libp2p/go-libp2p-core (v0.19.1 -> v0.20.1):\n - chore: release v0.20.1\n - feat: forward crypto/pb\n - release v0.20.0\n - deprecate this repo\n - stop using the deprecated io/ioutil package (#279) ([libp2p/go-libp2p-core#279](https://github.com/libp2p/go-libp2p-core/pull/279))\n - use a mock clock in bandwidth tests (#276) ([libp2p/go-libp2p-core#276](https://github.com/libp2p/go-libp2p-core/pull/276))\n - remove unused MultistreamSemverMatcher (#277) ([libp2p/go-libp2p-core#277](https://github.com/libp2p/go-libp2p-core/pull/277))\n - remove peer.IDFromString (#274) ([libp2p/go-libp2p-core#274](https://github.com/libp2p/go-libp2p-core/pull/274))\n - deprecate peer.Encode in favor of peer.ID.String (#275) ([libp2p/go-libp2p-core#275](https://github.com/libp2p/go-libp2p-core/pull/275))\n - deprecate peer.ID.Pretty (#273) ([libp2p/go-libp2p-core#273](https://github.com/libp2p/go-libp2p-core/pull/273))\n- github.com/libp2p/go-libp2p-kad-dht (v0.17.0 -> v0.18.0):\n - update go-libp2p to v0.22.0, release v0.18.0 ([libp2p/go-libp2p-kad-dht#788](https://github.com/libp2p/go-libp2p-kad-dht/pull/788))\n - sync: update CI config files (#789) ([libp2p/go-libp2p-kad-dht#789](https://github.com/libp2p/go-libp2p-kad-dht/pull/789))\n- github.com/libp2p/go-libp2p-peerstore (v0.7.1 -> v0.8.0):\n - release v0.8.0\n - deprecate this repo\n - fix flaky TestGCDelay (#206) ([libp2p/go-libp2p-peerstore#206](https://github.com/libp2p/go-libp2p-peerstore/pull/206))\n - fix flaky EWMA test (#205) ([libp2p/go-libp2p-peerstore#205](https://github.com/libp2p/go-libp2p-peerstore/pull/205))\n- github.com/libp2p/go-libp2p-record (v0.1.3 -> v0.2.0):\n - update go-libp2p to v0.22.0, release v0.2.0 ([libp2p/go-libp2p-record#50](https://github.com/libp2p/go-libp2p-record/pull/50))\n - sync: update CI config files (#47) ([libp2p/go-libp2p-record#47](https://github.com/libp2p/go-libp2p-record/pull/47))\n - increase RSA key sizes in tests ([libp2p/go-libp2p-record#44](https://github.com/libp2p/go-libp2p-record/pull/44))\n - cleanup: fix staticcheck failures ([libp2p/go-libp2p-record#43](https://github.com/libp2p/go-libp2p-record/pull/43))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.2.3 -> v0.4.0):\n - ([libp2p/go-libp2p-routing-helpers#62](https://github.com/libp2p/go-libp2p-routing-helpers/pull/62))\n - ([libp2p/go-libp2p-routing-helpers#58](https://github.com/libp2p/go-libp2p-routing-helpers/pull/58))\n - Update version.json ([libp2p/go-libp2p-routing-helpers#60](https://github.com/libp2p/go-libp2p-routing-helpers/pull/60))\n - update go-libp2p to v0.22.0 ([libp2p/go-libp2p-routing-helpers#59](https://github.com/libp2p/go-libp2p-routing-helpers/pull/59))\n - sync: update CI config files (#53) ([libp2p/go-libp2p-routing-helpers#53](https://github.com/libp2p/go-libp2p-routing-helpers/pull/53))\n - fix staticcheck ([libp2p/go-libp2p-routing-helpers#49](https://github.com/libp2p/go-libp2p-routing-helpers/pull/49))\n - fix error handling in Parallel.search ([libp2p/go-libp2p-routing-helpers#48](https://github.com/libp2p/go-libp2p-routing-helpers/pull/48))\n- github.com/libp2p/go-libp2p-testing (v0.11.0 -> v0.12.0):\n - release v0.12.0 (#67) ([libp2p/go-libp2p-testing#67](https://github.com/libp2p/go-libp2p-testing/pull/67))\n - chore: update to go-libp2p v0.22.0 (#66) ([libp2p/go-libp2p-testing#66](https://github.com/libp2p/go-libp2p-testing/pull/66))\n - remove the resource manager mocks (#65) ([libp2p/go-libp2p-testing#65](https://github.com/libp2p/go-libp2p-testing/pull/65))\n- github.com/libp2p/go-openssl (v0.0.7 -> v0.1.0):\n - release v0.1.0 (#31) ([libp2p/go-openssl#31](https://github.com/libp2p/go-openssl/pull/31))\n - Fix build with OpenSSL 3.0 (#25) ([libp2p/go-openssl#25](https://github.com/libp2p/go-openssl/pull/25))\n - sync: update CI config files ([libp2p/go-openssl#24](https://github.com/libp2p/go-openssl/pull/24))\n - Add openssl.DialTimeout(network, addr, timeout, ctx, flags) call ([libp2p/go-openssl#26](https://github.com/libp2p/go-openssl/pull/26))\n - Add Ctx.SetMinProtoVersion and Ctx.SetMaxProtoVersion wrappers ([libp2p/go-openssl#27](https://github.com/libp2p/go-openssl/pull/27))\n - sync: update CI config files ([libp2p/go-openssl#17](https://github.com/libp2p/go-openssl/pull/17))\n - fix: unsafe pointer passing ([libp2p/go-openssl#18](https://github.com/libp2p/go-openssl/pull/18))\n - Update test RSA cert ([libp2p/go-openssl#15](https://github.com/libp2p/go-openssl/pull/15))\n - Fix tests ([libp2p/go-openssl#16](https://github.com/libp2p/go-openssl/pull/16))\n - Address `staticcheck` issues ([libp2p/go-openssl#14](https://github.com/libp2p/go-openssl/pull/14))\n - Enabled PEM files with CRLF line endings to be used (#10) ([libp2p/go-openssl#11](https://github.com/libp2p/go-openssl/pull/11))\n- github.com/libp2p/zeroconf/v2 (v2.1.1 -> v2.2.0):\n - Fix windows libp2p (#29) ([libp2p/zeroconf#29](https://github.com/libp2p/zeroconf/pull/29))\n - Fix compatibility with some IoT devices using avahi 0.8-rc1 (#27) ([libp2p/zeroconf#27](https://github.com/libp2p/zeroconf/pull/27))\n - Add TTL server option (#23) ([libp2p/zeroconf#23](https://github.com/libp2p/zeroconf/pull/23))\n- github.com/lucas-clemente/quic-go (v0.28.0 -> v0.29.1):\n - http3: fix double close of chan when using DontCloseRequestStream\n - add a logging.NullTracer and logging.NullConnectionTracer ([lucas-clemente/quic-go#3512](https://github.com/lucas-clemente/quic-go/pull/3512))\n - add support for providing a custom Connection ID generator via Config (#3452) ([lucas-clemente/quic-go#3452](https://github.com/lucas-clemente/quic-go/pull/3452))\n - fix typo in README\n - fix datagram support detection (#3511) ([lucas-clemente/quic-go#3511](https://github.com/lucas-clemente/quic-go/pull/3511))\n - use a single Go routine to send copies of CONNECTION_CLOSE packets ([lucas-clemente/quic-go#3514](https://github.com/lucas-clemente/quic-go/pull/3514))\n - add YoMo to list of projects in README (#3513) ([lucas-clemente/quic-go#3513](https://github.com/lucas-clemente/quic-go/pull/3513))\n - http3: fix listening on both QUIC and TCP (#3465) ([lucas-clemente/quic-go#3465](https://github.com/lucas-clemente/quic-go/pull/3465))\n - Disable anti-amplification limit by address validation token (#3326) ([lucas-clemente/quic-go#3326](https://github.com/lucas-clemente/quic-go/pull/3326))\n - fix typo in README\n - implement a new API to let servers control client address verification ([lucas-clemente/quic-go#3501](https://github.com/lucas-clemente/quic-go/pull/3501))\n - use a generic streams map for incoming streams ([lucas-clemente/quic-go#3489](https://github.com/lucas-clemente/quic-go/pull/3489))\n - fix unreachable code after log.Fatal in fuzzing corpus generator (#3496) ([lucas-clemente/quic-go#3496](https://github.com/lucas-clemente/quic-go/pull/3496))\n - use generic Min and Max functions ([lucas-clemente/quic-go#3483](https://github.com/lucas-clemente/quic-go/pull/3483))\n - add QPACK (RFC 9204) to the list of supported RFCs (#3485) ([lucas-clemente/quic-go#3485](https://github.com/lucas-clemente/quic-go/pull/3485))\n - add a function to distinguish between long and short header packets (#3498) ([lucas-clemente/quic-go#3498](https://github.com/lucas-clemente/quic-go/pull/3498))\n - use a generic streams map for outgoing streams (#3488) ([lucas-clemente/quic-go#3488](https://github.com/lucas-clemente/quic-go/pull/3488))\n - update golangci-lint action to v3, golangci-lint to v1.48.0 (#3499) ([lucas-clemente/quic-go#3499](https://github.com/lucas-clemente/quic-go/pull/3499))\n - use a generic linked list (#3487) ([lucas-clemente/quic-go#3487](https://github.com/lucas-clemente/quic-go/pull/3487))\n - drop support for Go 1.16 and 1.17 (#3482) ([lucas-clemente/quic-go#3482](https://github.com/lucas-clemente/quic-go/pull/3482))\n - optimize FirstOutstanding in the sent packet history (#3467) ([lucas-clemente/quic-go#3467](https://github.com/lucas-clemente/quic-go/pull/3467))\n - update supported RFCs in README (#3456) ([lucas-clemente/quic-go#3456](https://github.com/lucas-clemente/quic-go/pull/3456))\n - http3: ignore context after response when using DontCloseRequestStream (#3473) ([lucas-clemente/quic-go#3473](https://github.com/lucas-clemente/quic-go/pull/3473))\n- github.com/marten-seemann/webtransport-go (null -> v0.1.1):\n - release v0.1.1 (#31) ([marten-seemann/webtransport-go#31](https://github.com/marten-seemann/webtransport-go/pull/31))\n - fix double close of chan when using DontCloseRequestStream\n- github.com/multiformats/go-base32 (v0.0.4 -> v0.1.0):\n - chore: bump version to 0.1.0\n - fix: fix staticcheck complaints\n - run gofmt -s\n - sync: update CI config files (#5) ([multiformats/go-base32#5](https://github.com/multiformats/go-base32/pull/5))\n- github.com/multiformats/go-multiaddr (v0.6.0 -> v0.7.0):\n - Release v0.7.0 ([multiformats/go-multiaddr#183](https://github.com/multiformats/go-multiaddr/pull/183))\n - use decimal numbers for multicodecs ([multiformats/go-multiaddr#184](https://github.com/multiformats/go-multiaddr/pull/184))\n - Fix comment on Decapsulate ([multiformats/go-multiaddr#181](https://github.com/multiformats/go-multiaddr/pull/181))\n - ([multiformats/go-multiaddr#182](https://github.com/multiformats/go-multiaddr/pull/182))\n - sync: update CI config files (#180) ([multiformats/go-multiaddr#180](https://github.com/multiformats/go-multiaddr/pull/180))\n - Add webrtc (#179) ([multiformats/go-multiaddr#179](https://github.com/multiformats/go-multiaddr/pull/179))\n- github.com/multiformats/go-multicodec (v0.5.0 -> v0.6.0):\n - chore: version bump 0.6.0\n - fix: replace io/ioutil with io\n - bump go.mod to Go 1.18 and run go fix\n\n
\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 236 | +12637/-24326 | 1152 |\n| Raúl Kripalani | 118 | +11626/-4136 | 422 |\n| vyzo | 144 | +10129/-3665 | 230 |\n| galargh | 9 | +5293/-5298 | 26 |\n| Marco Munizaga | 83 | +7502/-3080 | 147 |\n| Antonio Navarro Perez | 33 | +4074/-1240 | 78 |\n| Steven Allen | 98 | +1974/-1693 | 202 |\n| Cole Brown | 57 | +2169/-1338 | 95 |\n| Rod Vagg | 21 | +2588/-768 | 56 |\n| Gus Eggert | 16 | +2011/-1226 | 36 |\n| Yusef Napora | 6 | +2738/-187 | 43 |\n| Raúl Kripalani | 2 | +1000/-889 | 18 |\n| Łukasz Magiera | 26 | +1312/-500 | 54 |\n| Will | 2 | +1593/-200 | 18 |\n| Jorropo | 31 | +924/-712 | 204 |\n| Juan Batiz-Benet | 2 | +1531/-9 | 21 |\n| Jeromy | 14 | +691/-468 | 51 |\n| Petar Maymounkov | 4 | +469/-285 | 25 |\n| Jeromy Johnson | 24 | +474/-204 | 116 |\n| Justin Johnson | 1 | +582/-93 | 7 |\n| Aarsh Shah | 24 | +377/-105 | 34 |\n| web3-bot | 18 | +246/-228 | 93 |\n| Masih H. Derkani | 2 | +197/-213 | 21 |\n| Marcin Rataj | 9 | +211/-176 | 16 |\n| adam | 4 | +235/-49 | 9 |\n| Jakub Sztandera | 9 | +203/-73 | 13 |\n| Guilhem Fanton | 1 | +216/-48 | 5 |\n| Lucas Molas | 1 | +219/-9 | 3 |\n| Peter Argue | 1 | +166/-36 | 3 |\n| Vibhav Pant | 4 | +186/-12 | 7 |\n| Adrian Lanzafame | 3 | +180/-16 | 5 |\n| Lars Gierth | 5 | +151/-41 | 25 |\n| João Oliveirinha | 1 | +124/-38 | 11 |\n| dignifiedquire | 3 | +122/-33 | 6 |\n| Chinmay Kousik | 2 | +128/-4 | 7 |\n| Toby | 1 | +89/-36 | 4 |\n| Oleg Jukovec | 3 | +111/-14 | 8 |\n| Whyrusleeping | 2 | +120/-0 | 6 |\n| KevinZønda | 1 | +81/-20 | 2 |\n| wzp | 2 | +86/-3 | 2 |\n| Benedikt Spies | 1 | +75/-12 | 8 |\n| nisainan | 1 | +33/-43 | 12 |\n| Tshaka Eric Lekholoane | 1 | +57/-19 | 6 |\n| cpuchip | 1 | +65/-6 | 2 |\n| Roman Proskuryakov | 2 | +69/-0 | 2 |\n| Arceliar | 2 | +36/-28 | 2 |\n| Maxim Merzhanov | 1 | +29/-24 | 1 |\n| Richard Ramos | 1 | +51/-0 | 2 |\n| Dave Collins | 1 | +25/-25 | 4 |\n| Leo Balduf | 2 | +37/-10 | 3 |\n| David Aronchick | 1 | +42/-0 | 3 |\n| Didrik Nordström | 1 | +35/-6 | 1 |\n| Vasco Santos | 1 | +20/-20 | 7 |\n| Jesse Bouwman | 1 | +19/-21 | 1 |\n| Ivan Schasny | 2 | +22/-14 | 4 |\n| MGMCN | 1 | +9/-24 | 2 |\n| Brian Meek | 1 | +14/-17 | 4 |\n| Ian Davis | 3 | +21/-9 | 5 |\n| Mars Zuo | 1 | +7/-18 | 1 |\n| RubenKelevra | 1 | +10/-10 | 1 |\n| mojatter | 1 | +9/-8 | 1 |\n| Cory Schwartz | 1 | +0/-17 | 1 |\n| Steve Loeppky | 6 | +7/-6 | 6 |\n| Matt Joiner | 2 | +10/-3 | 2 |\n| Winterhuman | 2 | +7/-5 | 2 |\n| Dmitry Yu Okunev | 1 | +5/-7 | 5 |\n| corverroos | 1 | +7/-4 | 2 |\n| Marcel Gregoriadis | 1 | +9/-0 | 1 |\n| Ignacio Hagopian | 2 | +7/-2 | 2 |\n| Julien Muret | 1 | +4/-4 | 2 |\n| Eclésio Junior | 1 | +8/-0 | 1 |\n| Stephan Eberle | 1 | +4/-3 | 1 |\n| muXxer | 1 | +3/-3 | 1 |\n| eth-limo | 1 | +3/-3 | 2 |\n| Russell Dempsey | 2 | +4/-2 | 2 |\n| Sergey | 1 | +1/-3 | 1 |\n| Jun10ng | 2 | +2/-2 | 2 |\n| Jorik Schellekens | 1 | +2/-2 | 1 |\n| Eli Wang | 1 | +2/-2 | 1 |\n| Andreas Linde | 1 | +4/-0 | 1 |\n| whyrusleeping | 1 | +2/-1 | 1 |\n| xiabin | 1 | +1/-1 | 1 |\n| star | 1 | +0/-2 | 1 |\n| fanweixiao | 1 | +1/-1 | 1 |\n| dbadoy4874 | 1 | +1/-1 | 1 |\n| bigs | 1 | +1/-1 | 1 |\n| Tarun Bansal | 1 | +1/-1 | 1 |\n| Mikerah | 1 | +1/-1 | 1 |\n| Mike Goelzer | 1 | +2/-0 | 1 |\n| Max Inden | 1 | +1/-1 | 1 |\n| Kevin Mai-Husan Chia | 1 | +1/-1 | 1 |\n| John B Nelson | 1 | +1/-1 | 1 |\n| Eli Bailey | 1 | +1/-1 | 1 |\n| Bryan Stenson | 1 | +1/-1 | 1 |\n| Alex Stokes | 1 | +1/-1 | 1 |\n| Abirdcfly | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.17.md", "content": "# Kubo changelog v0.17\n\n## v0.17.0\n\n### Overview\n\nBelow is an outline of all that is in this release, so you get a sense of all that's included.\n\n- [Kubo changelog v0.17](#kubo-changelog-v017)\n - [v0.17.0](#v0170)\n - [Overview](#overview)\n - [🔦 Highlights](#-highlights)\n - [libp2p resource management enabled by default](#libp2p-resource-management-enabled-by-default)\n - [Implicit connection manager limits](#implicit-connection-manager-limits)\n - [TAR Response Format on Gateways](#tar-response-format-on-gateways)\n - [Dialling `/wss` peer behind a reverse proxy](#dialling-wss-peer-behind-a-reverse-proxy)\n - [Changelog](#changelog)\n - [Contributors](#contributors)\n\n### 🔦 Highlights\n\n\n\n#### libp2p resource management enabled by default\n\nTo help protect nodes from DoS (resource exhaustion) and eclipse attacks, \ngo-libp2p released a [Network Resource Manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager) with a host of improvements throughout 2022. \n\nKubo first [exposed this functionality in Kubo 0.13](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.13.md#-libp2p-network-resource-manager-swarmresourcemgr), \nbut it was disabled by default.\n\nThe resource manager is now enabled by default to protect nodes. \nThe defaults balance providing protection from various attacks while still enabling normal usecases to work as expected.\n\nIf you want to adjust the defaults, then you can:\n1. bound the amount of memory and file descriptors that libp2p will use with [Swarm.ResourceMgr.MaxMemory](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmresourcemgrmaxmemory) \nand [Swarm.ResourceMgr.MaxFileDescriptors](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmresourcemgrmaxfiledescriptors) and/or\n2. override any specific resource scopes/limits with [Swarm.ResourceMgr.Limits](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmresourcemgrlimits)\n\nSee [Swarm.ResourceMgr](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#swarmresourcemgr) for\n1. what limits are set by default,\n2. example override configuration, \n3. how to access Prometheus metrics and view Grafana dashboards of resource usage, and \n4. how to set explicit \"allow lists\" to protect against eclipse attacks. \n\n#### Implicit connection manager limits\n\nStarting with this release, `ipfs init` will no longer store the default\n[Connection Manager](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmconnmgr)\nlimits in the user config under `Swarm.ConnMgr`.\n\nUsers are still free to use this setting to set custom values, but for most use\ncases, the defaults provided with the latest Kubo release should be sufficient.\n\nTo remove any custom limits and switch to the implicit defaults managed by Kubo:\n\n```console\n$ ipfs config --json Swarm.ConnMgr '{}'\n```\n\nWe will be adjusting defaults in the future releases.\n\n#### TAR Response Format on Gateways\n\nImplemented [IPIP-288](https://github.com/ipfs/specs/pull/288) which adds\nsupport for requesting deserialized UnixFS directory as a TAR stream.\n\nHTTP clients can request TAR response by passing the `?format=tar` URL\nparameter, or setting `Accept: application/x-tar` HTTP header:\n\n```console\n$ export DIR_CID=bafybeigccimv3zqm5g4jt363faybagywkvqbrismoquogimy7kvz2sj7sq\n$ curl -H \"Accept: application/x-tar\" \"http://127.0.0.1:8080/ipfs/$DIR_CID\" > dir.tar\n$ curl \"http://127.0.0.1:8080/ipfs/$DIR_CID?format=tar\" | tar xv\nbafybeigccimv3zqm5g4jt363faybagywkvqbrismoquogimy7kvz2sj7sq\nbafybeigccimv3zqm5g4jt363faybagywkvqbrismoquogimy7kvz2sj7sq/1 - Barrel - Part 1 - alt.txt\nbafybeigccimv3zqm5g4jt363faybagywkvqbrismoquogimy7kvz2sj7sq/1 - Barrel - Part 1 - transcript.txt\nbafybeigccimv3zqm5g4jt363faybagywkvqbrismoquogimy7kvz2sj7sq/1 - Barrel - Part 1.png\n```\n\n#### Dialling `/wss` peer behind a reverse proxy\n\nThis release resolves a regression introduced in Kubo 0.16, making it possible\nagain to connect to a peer over a WebSockets endpoint (`/wss`) that is\ndeployed behind a reverse proxy.\n\nMore details in [go-libp2p release notes](https://github.com/libp2p/go-libp2p/releases/tag/v0.23.3).\n\n### Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: bump version to v0.17.0 ([ipfs/kubo#9427](https://github.com/ipfs/kubo/pull/9427))\n - chore: bump version to v0.17.0-rc2 ([ipfs/kubo#9414](https://github.com/ipfs/kubo/pull/9414))\n - Doc improvements and changelog for resource manager (#9413) ([ipfs/kubo#9413](https://github.com/ipfs/kubo/pull/9413))\n - fix(docs): typo\n - docs: document /wss fixes in 0.17\n - refactor(config): remove Swarm.ConnMgr defaults\n - fix(config): skip nulls in ResourceMgr\n - Apply go fmt\n - Update core/node/libp2p/rcmgr_defaults.go\n - Remove limitation by HighWater param.\n - Fix RM errors when acceleratedDHT is active\n - docs: Deprecate Reframe on docs. (#9401) ([ipfs/kubo#9401](https://github.com/ipfs/kubo/pull/9401))\n - chore: bump version to v0.17.0-rc1 ([ipfs/kubo#9394](https://github.com/ipfs/kubo/pull/9394))\n - feat: Improve ResourceManager UX (#9338) ([ipfs/kubo#9338](https://github.com/ipfs/kubo/pull/9338))\n - feat: ipfs-webui 2.20.0\n - docs: note log tail is broken (#9383) ([ipfs/kubo#9383](https://github.com/ipfs/kubo/pull/9383))\n - feat(gateway): TAR response format (#9029) ([ipfs/kubo#9029](https://github.com/ipfs/kubo/pull/9029))\n - fix: error when using huge json limit file\n - chore: go-multicodec v0.7.0\n - fix: remove old unused buggy coredag code\n - feat: Add command line completion for fish\n - chore: delete snap configuration ([ipfs/kubo#9352](https://github.com/ipfs/kubo/pull/9352))\n - docs: update scoop package\n - docs: init release issue template improvement process v0.16.0 ([ipfs/kubo#9283](https://github.com/ipfs/kubo/pull/9283))\n - feat: add delegated routing metrics (#9354) ([ipfs/kubo#9354](https://github.com/ipfs/kubo/pull/9354))\n - chore: create v0.17.md changelog ([ipfs/kubo#9353](https://github.com/ipfs/kubo/pull/9353))\n - docs: pin remote arg\n - feat: webui@v2.19.0\n - test(car): export/import of (dag-)cbor/json codecs\n - add refs local alias repo ls (#9320) ([ipfs/kubo#9320](https://github.com/ipfs/kubo/pull/9320))\n - docs(cmds): Clarify block fetching of refs endpoint.\n - chore(cmds): dag import: use ipld legacy decode ([ipfs/kubo#9219](https://github.com/ipfs/kubo/pull/9219))\n - fix ipfs swarm peering crash in offline mode (#9261) ([ipfs/kubo#9261](https://github.com/ipfs/kubo/pull/9261))\n - feat: remove provider delay interval in bitswap (#9053) ([ipfs/kubo#9053](https://github.com/ipfs/kubo/pull/9053))\n - feat: --reset flag on swarm limit command (#9310) ([ipfs/kubo#9310](https://github.com/ipfs/kubo/pull/9310))\n - fix: add InlineDNSLink flag to PublicGateways config (#9328) ([ipfs/kubo#9328](https://github.com/ipfs/kubo/pull/9328))\n - docs: Fix typo and grammar in README\n - ci: add stylecheck to golangci-lint (#9334) ([ipfs/kubo#9334](https://github.com/ipfs/kubo/pull/9334))\n - Fix: `swarm stats all` command\n - Merge release v0.16.0 back into master ([ipfs/kubo#9324](https://github.com/ipfs/kubo/pull/9324))\n - fix: Set default Methods value to nil\n - docs: add WebTransport docs ([ipfs/kubo#9314](https://github.com/ipfs/kubo/pull/9314))\n - chore: bump version to 0.17.0-dev\n- github.com/ipfs/go-delegated-routing (v0.6.0 -> v0.7.0):\n - Release v0.7.0\n - feat: add latency & count metrics for content routing client (#59) ([ipfs/go-delegated-routing#59](https://github.com/ipfs/go-delegated-routing/pull/59))\n - docs: add basic readme ([ipfs/go-delegated-routing#57](https://github.com/ipfs/go-delegated-routing/pull/57))\n - sync: update CI config files ([ipfs/go-delegated-routing#40](https://github.com/ipfs/go-delegated-routing/pull/40))\n - added link to reframe blog post (#54) ([ipfs/go-delegated-routing#54](https://github.com/ipfs/go-delegated-routing/pull/54))\n- github.com/ipfs/go-ipfs-files (v0.1.1 -> v0.2.0):\n - Release v0.2.0\n - fix: error when TAR has files outside of root (#56) ([ipfs/go-ipfs-files#56](https://github.com/ipfs/go-ipfs-files/pull/56))\n - sync: update CI config files ([ipfs/go-ipfs-files#55](https://github.com/ipfs/go-ipfs-files/pull/55))\n - chore(Directory): add DirIterator API restriction: iterate only once\n- github.com/ipfs/go-unixfs (v0.4.0 -> v0.4.1):\n - Update version.json\n - Fix: panic when childer is nil (#127) ([ipfs/go-unixfs#127](https://github.com/ipfs/go-unixfs/pull/127))\n - sync: update CI config files (#125) ([ipfs/go-unixfs#125](https://github.com/ipfs/go-unixfs/pull/125))\n- github.com/ipld/go-ipld-prime (v0.18.0 -> v0.19.0):\n - Prepare v0.19.0\n - fix: correct json codec links & bytes handling\n - test(basicnode): increase test coverage for int and map types (#454) ([ipld/go-ipld-prime#454](https://github.com/ipld/go-ipld-prime/pull/454))\n - fix: remove reliance on ioutil\n - run gofmt -s\n - bump go.mod to Go 1.18 and run go fix\n - feat: add kinded union to gendemo\n- github.com/libp2p/go-libp2p (v0.23.2 -> v0.23.4):\n - Release v0.23.4 (#1864) ([libp2p/go-libp2p#1864](https://github.com/libp2p/go-libp2p/pull/1864))\n - release v0.23.3\n - websocket: set the HTTP host header in WSS\n- github.com/libp2p/go-netroute (v0.2.0 -> v0.2.1):\n - v0.2.1 ([libp2p/go-netroute#27](https://github.com/libp2p/go-netroute/pull/27))\n - fix(phys-addr-length): fix physical address length mismatch ([libp2p/go-netroute#29](https://github.com/libp2p/go-netroute/pull/29))\n - compare priority if route rule's dst mask is same size\n - compare priority if route rule's dst mask is same size\n - sync: update CI config files (#24) ([libp2p/go-netroute#24](https://github.com/libp2p/go-netroute/pull/24))\n- github.com/marten-seemann/qpack (v0.2.1 -> v0.3.0):\n - update to Ginkgo v2 (#30) ([marten-seemann/qpack#30](https://github.com/marten-seemann/qpack/pull/30))\n - return write error when encoding header fields (#28) ([marten-seemann/qpack#28](https://github.com/marten-seemann/qpack/pull/28))\n - update Go versions (#29) ([marten-seemann/qpack#29](https://github.com/marten-seemann/qpack/pull/29))\n - remove CircleCI build status from README\n - add link to QPACK RFC to README\n - remove build constraint from fuzzer ([marten-seemann/qpack#24](https://github.com/marten-seemann/qpack/pull/24))\n- github.com/multiformats/go-multicodec (v0.6.0 -> v0.7.0):\n - feat: update ./multicodec/table.csv ([multiformats/go-multicodec#71](https://github.com/multiformats/go-multicodec/pull/71))\n\n
\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Antonio Navarro Perez | 11 | +780/-987 | 31 |\n| Marcin Rataj | 14 | +791/-543 | 26 |\n| web3-bot | 7 | +393/-427 | 71 |\n| galargh | 20 | +309/-277 | 21 |\n| Gus Eggert | 5 | +358/-222 | 58 |\n| Henrique Dias | 3 | +409/-30 | 13 |\n| Dustin Long | 1 | +314/-0 | 2 |\n| Marco Munizaga | 2 | +211/-46 | 11 |\n| Rod Vagg | 4 | +188/-62 | 13 |\n| Jorropo | 2 | +4/-219 | 5 |\n| Steve Loeppky | 1 | +115/-72 | 4 |\n| Andreas Källberg | 1 | +145/-5 | 5 |\n| Lucas Molas | 3 | +76/-53 | 9 |\n| snyh | 2 | +36/-18 | 2 |\n| Piotr Galar | 2 | +31/-4 | 2 |\n| Ondrej Kokes | 1 | +25/-4 | 2 |\n| Marten Seemann | 6 | +14/-14 | 14 |\n| Yann Autissier | 1 | +14/-4 | 1 |\n| maxos | 1 | +8/-1 | 2 |\n| reidlw | 1 | +1/-4 | 1 |\n| Russell Dempsey | 2 | +4/-1 | 2 |\n| Ian Davis | 1 | +4/-0 | 2 |\n| Daniel Norman | 1 | +3/-1 | 1 |\n| Will Scott | 1 | +1/-1 | 1 |\n| Nikhilesh Susarla | 1 | +2/-0 | 2 |\n| Jamie Wilkinson | 1 | +1/-1 | 1 |\n| Will | 1 | +0/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.18.md", "content": "# Kubo changelog v0.18\n\n## v0.18.1\n\nThis release includes improvements around Pubsub message deduplication, libp2p resource management, and more.\n\n\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [New default Pubsub.SeenMessagesStrategy](#new-default-pubsubseenmessagesstrategy)\n - [Improving libp2p resource management integration](#improving-libp2p-resource-management-integration)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n\n\n### 🔦 Highlights\n\n#### New default `Pubsub.SeenMessagesStrategy`\n\nA new optional [`Pubsub.SeenMessagesStrategy`](../config.md#pubsubseenmessagesstrategy) configuration option has been added.\n\nThis option allows you to choose between two different strategies for\ndeduplicating messages: `first-seen` and `last-seen`.\n\nWhen unset, the default strategy is `last-seen`, which calculates the\ntime-to-live (TTL) countdown based on the last time a message is seen. This\nmeans that if a message is received and then seen again within the specified\nTTL window based on the last time it was seen, it won't be emitted.\n\nIf you prefer the old behavior, which calculates the TTL countdown based on the\nfirst time a message is seen, you can set `Pubsub.SeenMessagesStrategy` to\n`first-seen`.\n\n#### Improving libp2p resource management integration\n\nTL;DR: limit autoscaling improved, most users should start with default settings.\nIf you have old configuration, switch to implicit defaults:\n\n```\nipfs config --json -- Swarm.ResourceMgr '{}'\nipfs config --json -- Swarm.ConnMgr '{}'\n```\n\nIF you run a server and want to utilize more than half of memory and file descriptors to p2p work, adjust [`Swarm.ResourceMgr.MaxMemory`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgrmaxmemory) and [`Swarm.ResourceMgr.MaxFileDescriptors`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgrmaxfiledescriptors).\n\nThe 0.18.1 builds on the default protection nodes get against DoS (resource exhaustion) and eclipse attacks\nwith the [go-libp2p Network Resource Manager/Accountant](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md)\nthat was fine-tuned in [Kubo 0.18](https://github.com/ipfs/kubo/blob/biglep/resource-manager-example-of-what-want/docs/changelogs/v0.18.md#improving-libp2p-resource-management-integration).\n\nAdding default hard-limits from the Resource Manager/Accountant after the fact is tricky,\nand some additional improvements have been made to improve the [computed defaults](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#computed-default-limits).\n\nAs much as possible, the aim is for a user to only think about how much memory they want to bound libp2p to, \nand not need to think about translating that to hard numbers for connections, streams, etc.\nMore updates are likely in future Kubo releases, but with this release: \n1. ``System.StreamsInbound`` is no longer bounded directly\n2. ``System.ConnsInbound``, ``Transient.Memory``, ``Transient.ConnsInbound`` have higher default computed values.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - Add overview section\n - Adjust inbound connection limits depending on memory.\n - feat: Pubsub.SeenMessagesStrategy (#9543) ([ipfs/kubo#9543](https://github.com/ipfs/kubo/pull/9543))\n - chore: update version\n- github.com/libp2p/go-libp2p-pubsub (v0.8.2 -> v0.8.3):\n - feat: expire messages from the cache based on last seen time (#513) ([libp2p/go-libp2p-pubsub#513](https://github.com/libp2p/go-libp2p-pubsub/pull/513))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Mohsin Zaidi | 2 | +511/-55 | 12 |\n| Antonio Navarro Perez | 2 | +57/-57 | 5 |\n| galargh | 1 | +1/-1 | 1 |\n\n## v0.18.0\n\n### Overview\n\nBelow is an outline of all that is in this release, so you get a sense of all that's included.\n\n\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Content routing](#content-routing)\n - [Default InterPlanetary Network Indexer](#default-interplanetary-network-indexer)\n - [Increase provider record republish interval and expiration](#increase-provider-record-republish-interval-and-expiration)\n - [Gateways](#gateways)\n - [(DAG-)JSON and (DAG-)CBOR response formats](#dag-json-and-dag-cbor-response-formats)\n - [🐎 Fast directory listings with DAG sizes](#-fast-directory-listings-with-dag-sizes)\n - [QUIC and WebTransport](#quic-and-webtransport)\n - [WebTransport enabled by default](#webtransport-enabled-by-default)\n - [QUIC and WebTransport share a single port](#quic-and-webtransport-share-a-single-port)\n - [Differentiating QUIC versions](#differentiating-quic-versions)\n - [QUICv1 and WebTransport config migration](#quicv1-and-webtransport-config-migration)\n - [Improving libp2p resource management integration](#improving-libp2p-resource-management-integration)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n\n\n### 🔦 Highlights\n\n#### Content routing\n\n##### Default InterPlanetary Network Indexer\n\nContent routing is the process of discovering which peers provide a piece of content. Kubo has traditionally only supported [libp2p's implementation of Kademlia DHT](https://github.com/libp2p/specs/tree/master/kad-dht) for content routing.\n\nKubo can now bridge networks by including support for the [delegated routing HTTP API](https://github.com/ipfs/specs/pull/337). Users can compose content routers using the `Routing.Routers` config to pick content routers with different tradeoffs than a Kademlia DHT (e.g., high-performance and high-capacity centralized endpoints, dedicated Kademlia DHT nodes, routers with unique provider records, privacy-focused content routers).\n\nOne example is [InterPlanetary Network Indexers](https://github.com/ipni/specs/blob/main/IPNI.md#readme), which are HTTP endpoints that cache records from both the IPFS network and other sources such as web3.storage and Filecoin. This improves not only content availability by enabling Kubo to transparently fetch content directly from Filecoin storage providers, but also improves IPFS content routing latency by an order of magnitude and decreases resource consumption.\n\n> *Note:* it's possible to retrieve content stored by Filecoin Storage Providers (SPs) from Kubo if the SPs service Bitswap requests. As of this release, some SPs are advertising Bitswap. You can follow the roadmap progress for IPNIs and Bitswap in SPs [here](https://www.starmaps.app/roadmap/github.com/protocol/bedrock/issues/1).\n\nIn this release, the default content router is changed from `dht` to `auto`. The `auto` router includes the IPFS DHT in addition to the [cid.contact](https://cid.contact) IPNI instance. In future releases, we plan to expand the functionality of `auto` to encompass automatic discovery of content routers, which will improve performance and content availability (for example, see [IPIP-342](https://github.com/ipfs/specs/pull/342)).\n\nPrevious behavior can be restored by setting `Routing.Type` to `dht`.\n\nAlternative routing rules, including alternative IPNI endpoints, can be configured in `Routing.Routers` after setting `Routing.Type` to `custom`.\n\nLearn more in the [`Routing` docs](https://github.com/ipfs/kubo/blob/master/docs/config.md#routing).\n\n##### Increase provider record republish interval and expiration\n\nDefault `Reprovider.Interval` changed from 12h to 22h to match new defaults for the Provider Record Expiration (48h) in [go-libp2p-kad-dht v0.20.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.20.0).\n\nThe rationale for increasing this can be found in\n[RFM 17: Provider Record Liveness Report](https://github.com/protocol/network-measurements/blob/master/results/rfm17-provider-record-liveness.md),\n[kubo#9326](https://github.com/ipfs/kubo/pull/9326),\nand the upstream DHT specifications at [libp2p/specs#451](https://github.com/libp2p/specs/pull/451).\n\nLearn more in the [`Reprovider` config](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#reprovider).\n\n#### Gateways\n\n##### (DAG-)JSON and (DAG-)CBOR response formats\n\nThe IPFS project has reserved the corresponding media types at IANA:\n- [`application/vnd.ipld.dag-json`](https://www.iana.org/assignments/media-types/application/vnd.ipld.dag-json)\n- [`application/vnd.ipld.dag-cbor`](https://www.iana.org/assignments/media-types/application/vnd.ipld.dag-cbor)\n\nThis release implements them as part of [IPIP-328](https://github.com/ipfs/specs/pull/328)\nand adds Gateway support for CIDs with `json` (0x0200), `cbor` (0x51),\n[`dag-json`](https://ipld.io/specs/codecs/dag-json/) (0x0129)\nand [`dag-cbor`](https://ipld.io/specs/codecs/dag-cbor/spec/) (0x71) codecs.\n\nTo specify the response `Content-Type` explicitly, the HTTP client can override\nthe codec present in the CID by using the `format` parameter\nor setting the `Accept` HTTP header:\n\n- Plain JSON: `?format=json` or `Accept: application/json`\n- Plain CBOR: `?format=cbor` or `Accept: application/cbor`\n- DAG-JSON: `?format=dag-json` or `Accept: application/vnd.ipld.dag-json`\n- DAG-CBOR: `?format=dag-cbor` or `Accept: application/vnd.ipld.dag-cbor`\n\nIn addition, when DAG-JSON or DAG-CBOR is requested with the `Accept` header\nset to `text/html`, the Gateway will return a basic HTML page with download\noptions, improving the user experience in web browsers.\n\n###### Example 1: DAG-CBOR and DAG-JSON Conversion on Gateway\n\nThe Gateway supports conversion between DAG-CBOR and DAG-JSON for efficient\nend-to-end data structure management: author in CBOR or JSON, store as binary\nCBOR and retrieve as JSON via HTTP:\n\n```console\n$ echo '{\"test\": \"json\"}' | ipfs dag put # implicit --input-codec dag-json --store-codec dag-cbor\nbafyreico7mjtqtqhvawro3yud5uqn6sc33nzqb7b5j2d7pdmzer5nab4t4\n\n$ ipfs block get bafyreico7mjtqtqhvawro3yud5uqn6sc33nzqb7b5j2d7pdmzer5nab4t4 | xxd\n00000000: a164 7465 7374 646a 736f 6e .dtestdjson\n\n$ ipfs dag get bafyreico7mjtqtqhvawro3yud5uqn6sc33nzqb7b5j2d7pdmzer5nab4t4 # implicit --output-codec dag-json\n{\"test\":\"json\"}\n\n$ curl \"http://127.0.0.1:8080/ipfs/bafyreico7mjtqtqhvawro3yud5uqn6sc33nzqb7b5j2d7pdmzer5nab4t4?format=dag-json\"\n{\"test\":\"json\"}\n```\n\n###### Example 2: Traversing CBOR DAGs\n\nPlacing a CID in [CBOR Tag 42](https://github.com/ipld/cid-cbor/) enables the\ncreation of arbitrary DAGs. The equivalent DAG-JSON notation for linking\nto different blocks is represented by `{ \"/\": \"cid\" }`.\n\nThe Gateway supports traversing these links, enabling access to data\nreferenced by structures other than regular UnixFS directories:\n\n```console\n$ echo '{\"test.jpg\": {\"/\": \"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\"}}' | ipfs dag put\nbafyreihspwy3zlkzgphmec5d3xb5g5njrqwotd46lyubnelbzktnmsxkq4 # dag-cbor document linking to unixfs file\n\n$ ipfs resolve /ipfs/bafyreihspwy3zlkzgphmec5d3xb5g5njrqwotd46lyubnelbzktnmsxkq4/test.jpg\n/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\n\n$ ipfs dag stat bafyreihspwy3zlkzgphmec5d3xb5g5njrqwotd46lyubnelbzktnmsxkq4\nSize: 119827, NumBlocks: 2\n\n$ curl \"http://127.0.0.1:8080/ipfs/bafyreihspwy3zlkzgphmec5d3xb5g5njrqwotd46lyubnelbzktnmsxkq4/test.jpg\" > test.jpg\n```\n\n###### Example 3: UnixFS directory listing as JSON\n\nFinally, Gateway now supports the same [logical format projection](https://ipld.io/specs/codecs/dag-pb/spec/#logical-format) from\nDAG-PB to DAG-JSON as the `ipfs dag get` command, enabling the retrieval of directory listings as JSON instead of HTML:\n\n```console\n$ export DIR_CID=bafybeigccimv3zqm5g4jt363faybagywkvqbrismoquogimy7kvz2sj7sq\n$ curl -H \"Accept: application/vnd.ipld.dag-json\" \"http://127.0.0.1:8080/ipfs/$DIR_CID\" | jq\n$ curl \"http://127.0.0.1:8080/ipfs/$DIR_CID?format=dag-json\" | jq\n{\n \"Data\": {\n \"/\": {\n \"bytes\": \"CAE\"\n }\n },\n \"Links\": [\n {\n \"Hash\": {\n \"/\": \"Qmc3zqKcwzbbvw3MQm3hXdg8BQoFjGdZiGdAfXAyAGGdLi\"\n },\n \"Name\": \"1 - Barrel - Part 1 - alt.txt\",\n \"Tsize\": 21\n },\n {\n \"Hash\": {\n \"/\": \"QmdMxMx29KVYhHnaCc1icWYxQqXwUNCae6t1wS2NqruiHd\"\n },\n \"Name\": \"1 - Barrel - Part 1 - transcript.txt\",\n \"Tsize\": 195\n },\n {\n \"Hash\": {\n \"/\": \"QmawceGscqN4o8Y8Fv26UUmB454kn2bnkXV5tEQYc4jBd6\"\n },\n \"Name\": \"1 - Barrel - Part 1.png\",\n \"Tsize\": 24862\n }\n ]\n}\n$ ipfs dag get $DIR_CID\n{\"Data\":{\"/\":{\"bytes\":\"CAE\"}},\"Links\":[{\"Hash\":{\"/\":\"Qmc3zqKcwzbbvw3MQm3hXdg8BQoFjGdZiGdAfXAyAGGdLi\"},\"Name\":\"1 - Barrel - Part 1 - alt.txt\",\"Tsize\":21},{\"Hash\":{\"/\":\"QmdMxMx29KVYhHnaCc1icWYxQqXwUNCae6t1wS2NqruiHd\"},\"Name\":\"1 - Barrel - Part 1 - transcript.txt\",\"Tsize\":195},{\"Hash\":{\"/\":\"QmawceGscqN4o8Y8Fv26UUmB454kn2bnkXV5tEQYc4jBd6\"},\"Name\":\"1 - Barrel - Part 1.png\",\"Tsize\":24862}]}\n```\n\n##### 🐎 Fast directory listings with DAG sizes\n\nFast listings are now enabled for _all_ UnixFS directories: big and small.\nThere is no linear slowdown caused by reading size metadata from child nodes,\nand the size of DAG representing child items is always present.\n\nAs an example, the CID\n`bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm` represents a UnixFS\ndirectory with over 10k files. Listing big directories was fast\nsince Kubo 0.13, but in this release it will also include the size column.\n\n#### QUIC and WebTransport\n\n##### WebTransport enabled by default\n[WebTransport](https://web.archive.org/web/20260128152314/https://docs.libp2p.io/concepts/transports/webtransport/) is a new libp2p transport that [was introduced in v0.16](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.16.md#-webtransport-new-experimental-transport) that is based on top of QUIC and HTTP3.\n\nThis allows browser-based nodes to contact Kubo nodes, so now instead of just serving requests for other system-level application nodes, you can also serve requests directly to a node running inside a browser page.\n\nFor the full story see [connectivity.libp2p.io](https://web.archive.org/web/20251118040510/https://connectivity.libp2p.io/).\n\n##### QUIC and WebTransport share a single port\nWebTransport is enabled by default in part because [go-libp2p now supports running WebTransport and QUIC transports on the same QUIC listener](https://github.com/libp2p/go-libp2p/issues/1759). No additional port needs to be opened.\n\nTo use this feature, register two listen addresses on the same `/ipX/.../udp/XXX` prefix.\n\n##### Differentiating QUIC versions\ngo-libp2p now differentiates the first version of QUIC that was originally implemented, `Draft-29`, from the ratified protocol in [RFC9000](https://www.rfc-editor.org/rfc/rfc9000.html), `QUICv1`.\nThis was done for performance (time to first byte) reasons as [outlined here](https://github.com/multiformats/multiaddr/issues/145).\n\nThis manifests as two different multiaddr components `/quic` (old Draft-29) and `/quic-v1`.\ngo-libp2p do supports listening with both QUIC versions on one single listener.\nWebTransport has only supported QUICv1.\n`/webtransport` now needs to be prefixed by a `/quic-v1` component instead of a `/quic` component.\n\nSupport for QUIC Draft-29 will be removed at some point in 2023 ([tracking issue](https://github.com/ipfs/kubo/issues/9496)). As a result, new deployments should use `/quic-v1` instead of `/quic`.\n\n##### QUICv1 and WebTransport config migration\nTo support QUICv1 and WebTransport by default a new config migration (`v13`) is run which automatically adds entries in addresses-related fields:\n- Replace all `/quic/webtransport` to `/quic-v1/webtransport`.\n- For all `/quic` listeners, keep the Draft-29 listener, and on the same ip and port, add `/quic-v1` and `/quic-v1/webtransport` listeners.\n\n#### Improving libp2p resource management integration\nTo help protect nodes from DoS (resource exhaustion) and eclipse attacks,\nKubo enabled the [go-libp2p Network Resource Manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager)\nby default in [Kubo 0.17](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.17.md#libp2p-resource-management-enabled-by-default).\n\nIntroducing limits like this by default after the fact is tricky,\nand various improvements have been made to improve the UX including:\n1. [Dedicated docs concerning the resource manager integration](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md). This is a great place to go to learn more or get your FAQs answered.\n2. Increasing the default limits for the resource manager.\n3. Enabling the [`Swarm.ConnMgr`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmconnmgr) by default and reducing it thresholds so it can intelligently prune connections in many cases before the indiscriminate resource manager kicks in.\n4. Adjusted log messages and levels to make clear that the resource manager is likely doing your node a favor by bounding resources.\n5. [Other miscellaneous config and command bugs reported by users](https://github.com/ipfs/kubo/issues/9442).\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix: clarity: no user supplied rcmgr limits of 0 (#9563) ([ipfs/kubo#9563](https://github.com/ipfs/kubo/pull/9563))\n - fix(gateway): undesired conversions to dag-json and friends (#9566) ([ipfs/kubo#9566](https://github.com/ipfs/kubo/pull/9566))\n - fix: ensure connmgr is smaller then autoscalled resource limits\n - fix: typo in ensureConnMgrMakeSenseVsResourcesMgr\n - docs: clarify browser descriptions for webtransport\n - fix: update saxon download path\n - fix: refuse to start if connmgr is smaller than resource limits and not using none connmgr\n - fix: User-Agent sent to HTTP routers\n - test: port gateway sharness tests to Go tests\n - fix: do not download saxon in parallel\n - docs: improve docs/README (#9539) ([ipfs/kubo#9539](https://github.com/ipfs/kubo/pull/9539))\n - test: port CircleCI to GH Actions and improve sharness reporting (#9355) ([ipfs/kubo#9355](https://github.com/ipfs/kubo/pull/9355))\n - chore: migrate from go-ipfs-files to go-libipfs/files (#9535) ([ipfs/kubo#9535](https://github.com/ipfs/kubo/pull/9535))\n - fix: stats dht command when Routing.Type=auto (#9538) ([ipfs/kubo#9538](https://github.com/ipfs/kubo/pull/9538))\n - fix: hint people to changing from RSA peer ids\n - fix(gateway): JSON when Accept is a list\n - fix(test): retry flaky t0125-twonode.sh\n - docs: fix Router config Godoc (#9528) ([ipfs/kubo#9528](https://github.com/ipfs/kubo/pull/9528))\n - fix(ci): flaky sharness test\n - docs(config): ProviderSearchDelay (#9526) ([ipfs/kubo#9526](https://github.com/ipfs/kubo/pull/9526))\n - docs: clarify debug environment variables\n - chore: update version.go\n - fix(test): stabilize flaky provider tests\n - feat: port pins CLI test\n - Removing QRI from early tester ([ipfs/kubo#9503](https://github.com/ipfs/kubo/pull/9503))\n - fix: disable provide over HTTP with Routing.Type=auto (#9511) ([ipfs/kubo#9511](https://github.com/ipfs/kubo/pull/9511))\n - Update version.go\n - 'chore: update version.go'\n - Cleaned up 0.18 changelog for release ([ipfs/kubo#9497](https://github.com/ipfs/kubo/pull/9497))\n - feat: turn on WebTransport by default ([ipfs/kubo#9492](https://github.com/ipfs/kubo/pull/9492))\n - feat: fast directory listings with DAG Size column (#9481) ([ipfs/kubo#9481](https://github.com/ipfs/kubo/pull/9481))\n - feat: add basic CLI tests using Go Test\n - Changelog: v0.18.0 ([ipfs/kubo#9485](https://github.com/ipfs/kubo/pull/9485))\n - feat: go-libp2p-kad-dht with expiration 48h\n - chore: update go-libp2p to v0.24.1\n - fix: remove the imports work-around\n - fix: replace quic to quic-v1 for webtransport sharness\n - fix: silence staticcheck warning for fx.Extract usage\n - update go-libp2p to v0.24.0\n - stop using the deprecated go-libp2p-loggables package\n - docs(readme): update package managers section (#9488) ([ipfs/kubo#9488](https://github.com/ipfs/kubo/pull/9488))\n - fix: support /quic-v1 in webui v0.21\n - feat: Routing.Type=auto (DHT+IPNI) (#9475) ([ipfs/kubo#9475](https://github.com/ipfs/kubo/pull/9475))\n - feat: adjust ConnMgr target to 32-96 (#9483) ([ipfs/kubo#9483](https://github.com/ipfs/kubo/pull/9483))\n - feat: increase default Reprovider.Interval (#9326) ([ipfs/kubo#9326](https://github.com/ipfs/kubo/pull/9326))\n - feat: add response body limiter to routing HTTP client (#9478) ([ipfs/kubo#9478](https://github.com/ipfs/kubo/pull/9478))\n - docs: libp2p resource management (#9468) ([ipfs/kubo#9468](https://github.com/ipfs/kubo/pull/9468))\n - chore: upgrade libipfs for routing HTTP API schema changes (#9477) ([ipfs/kubo#9477](https://github.com/ipfs/kubo/pull/9477))\n - feat: lower connection pool\n - Add missing &&\n - Fix sharness test\n - Added a message when RM is disabled.\n - Requested changes.\n - Fix sharness checking daemon output\n - Update test/sharness/t0060-daemon.sh\n - Try to fix sharness test.\n - Fix: RM: Improve init RM message and fix final memory value.\n - Fix: Resource Manager: Filter stats correctly by %\n - Apply suggestions from code review\n - Increase MaxMemory param to use half of total memory.\n - Update libipfs dependency.\n - Add sharness tests and documentation\n - Fix variable name\n - feature: delegated-routing: Add HTTP delegated routing.\n - Fix: Change RM log output to WARN level\n - Fix: RM: Set no-limit value to 1e9 (1000000000).\n - Partial Revert \"Revert \"fix: ensure hasher is registered when using a hashing function\"\"\n - Add logs to the routing system\n - fix: apply agent-version-suffix to libp2p identify\n - chore: migrate ipfs/tar-utils to libipfs\n - feat(gateway): JSON and CBOR response formats (IPIP-328) (#9335) ([ipfs/kubo#9335](https://github.com/ipfs/kubo/pull/9335))\n - ([ipfs/kubo#9318](https://github.com/ipfs/kubo/pull/9318))\n - docs: release process updates from v0.17.0 ([ipfs/kubo#9391](https://github.com/ipfs/kubo/pull/9391))\n - fix(rcmgr): improve error phrasing\n - docs: Update CHANGELOG.md adding 0.17 link\n - feat(config): Pubsub.SeenMessagesTTL (#9372) ([ipfs/kubo#9372](https://github.com/ipfs/kubo/pull/9372))\n - docs: remove snap and chocolatey packages\n - Merge release v0.17.0 ([ipfs/kubo#9431](https://github.com/ipfs/kubo/pull/9431))\n - docs: ipfs-http-client -> kubo-rpc-client (#9331) ([ipfs/kubo#9331](https://github.com/ipfs/kubo/pull/9331))\n - docs(readme): improve tldr\n - Update config.md for resource management limits (#9421) ([ipfs/kubo#9421](https://github.com/ipfs/kubo/pull/9421))\n - Doc improvements and changelog for resource manager (#9413) ([ipfs/kubo#9413](https://github.com/ipfs/kubo/pull/9413))\n - Revert \"Doc improvements for rcmgr\"\n - Doc improvements for rcmgr\n - docs: document /wss fixes in 0.17\n - refactor(config): remove Swarm.ConnMgr defaults\n - fix(config): skip nulls in ResourceMgr\n - docs: replace tabcat with aphelionz in EARLY_TESTERS.md (#9404) ([ipfs/kubo#9404](https://github.com/ipfs/kubo/pull/9404))\n - docs: fix spoiler for 0.13.1 changelog\n - fix(docs): typo\n - chore: bump version to v0.18.0-dev ([ipfs/kubo#9393](https://github.com/ipfs/kubo/pull/9393))\n- github.com/ipfs/go-bitswap (v0.10.2 -> v0.11.0):\n - chore: release v0.11.0\n- github.com/ipfs/go-blockservice (v0.4.0 -> v0.5.0):\n - chore: release v0.5.0\n- github.com/ipfs/go-graphsync (v0.13.1 -> v0.14.1):\n - chore: version 0.14.1 (#400) ([ipfs/go-graphsync#400](https://github.com/ipfs/go-graphsync/pull/400))\n - chore: migrate files (#399) ([ipfs/go-graphsync#399](https://github.com/ipfs/go-graphsync/pull/399))\n - docs(CHANGELOG): update for v0.14.0 release\n - updates for libp2p v0.22 (#392) ([ipfs/go-graphsync#392](https://github.com/ipfs/go-graphsync/pull/392))\n - feat(ipld): use bindnode/registry (#386) ([ipfs/go-graphsync#386](https://github.com/ipfs/go-graphsync/pull/386))\n - Accept/Reject requests up front (#384) ([ipfs/go-graphsync#384](https://github.com/ipfs/go-graphsync/pull/384))\n - Remove protobuf protocol (#385) ([ipfs/go-graphsync#385](https://github.com/ipfs/go-graphsync/pull/385))\n - docs(CHANGELOG): update for v0.13.2\n - chore(deps): upgrade libp2p & ipld-prime (#389) ([ipfs/go-graphsync#389](https://github.com/ipfs/go-graphsync/pull/389))\n - chore(ipld): switch to using top-level ipld-prime codec helpers (#383) ([ipfs/go-graphsync#383](https://github.com/ipfs/go-graphsync/pull/383))\n - feat(requestmanager): read request from context (#381) ([ipfs/go-graphsync#381](https://github.com/ipfs/go-graphsync/pull/381))\n - fix: minor typo in error msg\n - fix(panics): lift panic recovery up to top of network handling\n - feat: expand use of panic handler to cover network and codec interaction\n - feat(panics): capture panics from selector execution\n- github.com/ipfs/go-ipfs-cmds (v0.8.1 -> v0.8.2):\n - chore: version v0.8.2 (#235) ([ipfs/go-ipfs-cmds#235](https://github.com/ipfs/go-ipfs-cmds/pull/235))\n - chore: migrate files (#233) ([ipfs/go-ipfs-cmds#233](https://github.com/ipfs/go-ipfs-cmds/pull/233))\n - sync: update CI config files (#229) ([ipfs/go-ipfs-cmds#229](https://github.com/ipfs/go-ipfs-cmds/pull/229))\n- github.com/ipfs/go-ipfs-keystore (v0.0.2 -> v0.1.0):\n - chore: release v0.1.0\n - chore: update go-libp2p\n - sync: update CI config files ([ipfs/go-ipfs-keystore#10](https://github.com/ipfs/go-ipfs-keystore/pull/10))\n - sync: update CI config files ([ipfs/go-ipfs-keystore#8](https://github.com/ipfs/go-ipfs-keystore/pull/8))\n - Add link to pkg.go.dev\n - README: this module does not use Gx\n- github.com/ipfs/go-ipfs-provider (v0.7.1 -> v0.8.1):\n - chore: release v0.8.1\n - fix: make queue 64bits on 32bits platforms too\n - sync: update CI config files ([ipfs/go-ipfs-provider#36](https://github.com/ipfs/go-ipfs-provider/pull/36))\n - chore: update go-lib2p, avoid depending on go-libp2p-core, bump go.mod version\n- github.com/ipfs/go-ipfs-routing (v0.2.1 -> v0.3.0):\n - release v0.3.0 ([ipfs/go-ipfs-routing#36](https://github.com/ipfs/go-ipfs-routing/pull/36))\n - chore: update go-libp2p to v0.22.0 ([ipfs/go-ipfs-routing#35](https://github.com/ipfs/go-ipfs-routing/pull/35))\n - sync: update CI config files (#34) ([ipfs/go-ipfs-routing#34](https://github.com/ipfs/go-ipfs-routing/pull/34))\n- github.com/ipfs/go-ipld-cbor (v0.0.5 -> v0.0.6):\n - Add contexts to IpldBlockstore ([ipfs/go-ipld-cbor#82](https://github.com/ipfs/go-ipld-cbor/pull/82))\n - sync: update CI config files (#81) ([ipfs/go-ipld-cbor#81](https://github.com/ipfs/go-ipld-cbor/pull/81))\n - sync: update CI config files ([ipfs/go-ipld-cbor#79](https://github.com/ipfs/go-ipld-cbor/pull/79))\n - Fix lint errors ([ipfs/go-ipld-cbor#78](https://github.com/ipfs/go-ipld-cbor/pull/78))\n - Add notice directing new projects to codec/dagcbor ([ipfs/go-ipld-cbor#77](https://github.com/ipfs/go-ipld-cbor/pull/77))\n- github.com/ipfs/go-merkledag (v0.6.0 -> v0.9.0):\n - chore: bump version to 0.9.0\n - chore: bump version to 0.8.1\n - feat: remove panic() from non-error methods\n - feat: improve broken cid.Builder testing for CidBuilder\n - chore: bump version to 0.8.0\n - doc: document potential panics and how to avoid them\n - fix: simplify Cid generation cache & usage\n - feat: check links on setting and sanitise on encoding\n - feat: check that the CidBuilder hasher is usable\n - chore: bump version to 0.7.0\n - fix: remove use of ioutil\n - run gofmt -s\n - fix!: keep deserialised state stable until explicit mutation\n - sync: update CI config files ([ipfs/go-merkledag#84](https://github.com/ipfs/go-merkledag/pull/84))\n- github.com/ipfs/go-namesys (v0.5.0 -> v0.6.0):\n - chore: release v0.6.0\n - chore: update go-libp2p to v0.23.4, update go.mod version to 1.18\n - stop using the deprecated io/ioutil package\n- github.com/ipfs/go-peertaskqueue (v0.7.1 -> v0.8.0):\n - Release v0.8.0 ([ipfs/go-peertaskqueue#25](https://github.com/ipfs/go-peertaskqueue/pull/25))\n - chore: update go-libp2p to v0.22.0 ([ipfs/go-peertaskqueue#24](https://github.com/ipfs/go-peertaskqueue/pull/24))\n - sync: update CI config files (#23) ([ipfs/go-peertaskqueue#23](https://github.com/ipfs/go-peertaskqueue/pull/23))\n - sync: update CI config files (#21) ([ipfs/go-peertaskqueue#21](https://github.com/ipfs/go-peertaskqueue/pull/21))\n- github.com/ipfs/go-unixfs (v0.4.1 -> v0.4.2):\n - chore: version 0.4.2 (#136) ([ipfs/go-unixfs#136](https://github.com/ipfs/go-unixfs/pull/136))\n - chore: migrate files (#134) ([ipfs/go-unixfs#134](https://github.com/ipfs/go-unixfs/pull/134))\n - ([ipfs/go-unixfs#128](https://github.com/ipfs/go-unixfs/pull/128))\n- github.com/ipfs/go-unixfsnode (v1.4.0 -> v1.5.1):\n - v1.5.1\n - fix a possible `index out of range` crash ([ipfs/go-unixfsnode#39](https://github.com/ipfs/go-unixfsnode/pull/39))\n - add an ADL to preload hamt loading ([ipfs/go-unixfsnode#38](https://github.com/ipfs/go-unixfsnode/pull/38))\n - chore: bump version to 1.5.0\n - fix: remove use of ioutil\n - run gofmt -s\n - bump go.mod to Go 1.18 and run go fix\n - test for reader / sizing behavior on large files ([ipfs/go-unixfsnode#34](https://github.com/ipfs/go-unixfsnode/pull/34))\n - add helper to approximate test creation pattern from ipfs-files ([ipfs/go-unixfsnode#32](https://github.com/ipfs/go-unixfsnode/pull/32))\n - chore: remove Stebalien/go-bitfield in favour of ipfs/go-bitfield\n- github.com/ipfs/interface-go-ipfs-core (v0.7.0 -> v0.8.2):\n - chore: version 0.8.2 (#100) ([ipfs/interface-go-ipfs-core#100](https://github.com/ipfs/interface-go-ipfs-core/pull/100))\n - chore: migrate files (#97) ([ipfs/interface-go-ipfs-core#97](https://github.com/ipfs/interface-go-ipfs-core/pull/97))\n - chore: release v0.8.1\n - feat: add UseCumulativeSize UnixfsLs option (#95) ([ipfs/interface-go-ipfs-core#95](https://github.com/ipfs/interface-go-ipfs-core/pull/95))\n - chore: release v0.8.0\n - chore: update go-libp2p to v0.23.4\n - sync: update CI config files (#87) ([ipfs/interface-go-ipfs-core#87](https://github.com/ipfs/interface-go-ipfs-core/pull/87))\n- github.com/ipld/go-car/v2 (v2.4.0 -> v2.5.1):\n - add a `SkipNext` method on block reader (#338) ([ipld/go-car#338](https://github.com/ipld/go-car/pull/338))\n - feat: Has() and Get() will respect StoreIdentityCIDs option\n - chore: bump version to 0.5.0\n - fix: remove use of ioutil\n - run gofmt -s\n - bump go.mod to Go 1.18 and run go fix\n - bump go.mod to Go 1.18 and run go fix\n - OpenReadWriteFile: add test\n - blockstore: allow to pass a file to write in (#323) ([ipld/go-car#323](https://github.com/ipld/go-car/pull/323))\n - feat: add `car inspect` command to cmd pkg (#320) ([ipld/go-car#320](https://github.com/ipld/go-car/pull/320))\n - Separate `index.ReadFrom` tests\n - Only read index codec during inspection\n - Upgrade to the latest `go-car/v2`\n - Empty identity CID should be indexed when options are set\n- github.com/ipld/go-codec-dagpb (v1.4.1 -> v1.5.0):\n - chore: version bump to 1.5.0\n - fix: replace io/ioutil with io\n - bump go.mod to Go 1.18 and run go fix\n - v1.4.2 bump\n- github.com/libp2p/go-libp2p (v0.23.4 -> v0.24.2):\n - release v0.24.2 (#1969) ([libp2p/go-libp2p#1969](https://github.com/libp2p/go-libp2p/pull/1969))\n - webtransport: initialize a NullResourceManager if none is provided (#1962) ([libp2p/go-libp2p#1962](https://github.com/libp2p/go-libp2p/pull/1962))\n - release v0.24.1 (#1945) ([libp2p/go-libp2p#1945](https://github.com/libp2p/go-libp2p/pull/1945))\n - routed host: return Connect error if FindPeer doesn't yield new addresses (#1946) ([libp2p/go-libp2p#1946](https://github.com/libp2p/go-libp2p/pull/1946))\n - chore: update examples to v0.24.0 (#1936) ([libp2p/go-libp2p#1936](https://github.com/libp2p/go-libp2p/pull/1936))\n - webtransport: fix flaky accept queue test (#1938) ([libp2p/go-libp2p#1938](https://github.com/libp2p/go-libp2p/pull/1938))\n - quic: fix race condition in TestClientCanDialDifferentQUICVersions (#1937) ([libp2p/go-libp2p#1937](https://github.com/libp2p/go-libp2p/pull/1937))\n - quic: update quic-go to v0.31.1 (#1942) ([libp2p/go-libp2p#1942](https://github.com/libp2p/go-libp2p/pull/1942))\n - release v0.24.0 (#1934) ([libp2p/go-libp2p#1934](https://github.com/libp2p/go-libp2p/pull/1934))\n - Disable support for signed/static TLS certificates in WebTransport (#1927) ([libp2p/go-libp2p#1927](https://github.com/libp2p/go-libp2p/pull/1927))\n - webtransport: add PSK to constructor, and fail if it is used (#1929) ([libp2p/go-libp2p#1929](https://github.com/libp2p/go-libp2p/pull/1929))\n - use a different set of default transports when PSK is enabled (#1921) ([libp2p/go-libp2p#1921](https://github.com/libp2p/go-libp2p/pull/1921))\n - transport.Listener,quic: Support multiple QUIC versions with the same Listener. Only return a single multiaddr per listener. (#1923) ([libp2p/go-libp2p#1923](https://github.com/libp2p/go-libp2p/pull/1923))\n - quic / webtransport: make it possible to listen on the same address / port (#1905) ([libp2p/go-libp2p#1905](https://github.com/libp2p/go-libp2p/pull/1905))\n - autorelay: fix flaky TestReconnectToStaticRelays (#1903) ([libp2p/go-libp2p#1903](https://github.com/libp2p/go-libp2p/pull/1903))\n - swarm / rcmgr: synchronize the concurrent outbound dials with limits (#1898) ([libp2p/go-libp2p#1898](https://github.com/libp2p/go-libp2p/pull/1898))\n - add QUIC v1 addresses to the default listen addresses (#1914) ([libp2p/go-libp2p#1914](https://github.com/libp2p/go-libp2p/pull/1914))\n - webtransport: update webtransport-go to v0.3.0 (#1895) ([libp2p/go-libp2p#1895](https://github.com/libp2p/go-libp2p/pull/1895))\n - tls: fix flaky TestHandshakeConnectionCancellations test (#1896) ([libp2p/go-libp2p#1896](https://github.com/libp2p/go-libp2p/pull/1896))\n - holepunch: disable the resource manager in tests (#1897) ([libp2p/go-libp2p#1897](https://github.com/libp2p/go-libp2p/pull/1897))\n - transports: expose the name of the transport in the ConnectionState (#1911) ([libp2p/go-libp2p#1911](https://github.com/libp2p/go-libp2p/pull/1911))\n - respect the user's security protocol preference order ([libp2p/go-libp2p#1912](https://github.com/libp2p/go-libp2p/pull/1912))\n - circuitv2: disable the resource manager in tests (#1899) ([libp2p/go-libp2p#1899](https://github.com/libp2p/go-libp2p/pull/1899))\n - expose the security protocol on the ConnectionState ([libp2p/go-libp2p#1907](https://github.com/libp2p/go-libp2p/pull/1907))\n - fix: autorelay: treat static relays as just another peer source (#1875) ([libp2p/go-libp2p#1875](https://github.com/libp2p/go-libp2p/pull/1875))\n - feat: quic,webtransport: enable both quic-draft29 and quic-v1 addrs on quic. only quic-v1 on webtransport (#1881) ([libp2p/go-libp2p#1881](https://github.com/libp2p/go-libp2p/pull/1881))\n - holepunch: add multiaddress filter (#1839) ([libp2p/go-libp2p#1839](https://github.com/libp2p/go-libp2p/pull/1839))\n - README: remove broken links from table of contents (#1893) ([libp2p/go-libp2p#1893](https://github.com/libp2p/go-libp2p/pull/1893))\n - quic: update quic-go to v0.31.0 (#1882) ([libp2p/go-libp2p#1882](https://github.com/libp2p/go-libp2p/pull/1882))\n - add an integration test for muxer selection ([libp2p/go-libp2p#1887](https://github.com/libp2p/go-libp2p/pull/1887))\n - core/network: fix typo\n - tls / noise: prefer the client's muxer preferences ([libp2p/go-libp2p#1888](https://github.com/libp2p/go-libp2p/pull/1888))\n - upgrader: absorb the muxer_multistream.Transport into the upgrader (#1885) ([libp2p/go-libp2p#1885](https://github.com/libp2p/go-libp2p/pull/1885))\n - Apply service peer default (#1878) ([libp2p/go-libp2p#1878](https://github.com/libp2p/go-libp2p/pull/1878))\n - webtransport: use deterministic TLS certificates (#1833) ([libp2p/go-libp2p#1833](https://github.com/libp2p/go-libp2p/pull/1833))\n - remove deprecated StaticRelays option (#1868) ([libp2p/go-libp2p#1868](https://github.com/libp2p/go-libp2p/pull/1868))\n - autorelay: remove the default static relay option (#1867) ([libp2p/go-libp2p#1867](https://github.com/libp2p/go-libp2p/pull/1867))\n - core/protocol: remove deprecated Negotiator.NegotiateLazy (#1869) ([libp2p/go-libp2p#1869](https://github.com/libp2p/go-libp2p/pull/1869))\n - config: use fx dependency injection to construct transports ([libp2p/go-libp2p#1858](https://github.com/libp2p/go-libp2p/pull/1858))\n - noise: add an option to allow unknown peer ID in SecureOutbound (#1823) ([libp2p/go-libp2p#1823](https://github.com/libp2p/go-libp2p/pull/1823))\n - Add some guard rails and docs (#1863) ([libp2p/go-libp2p#1863](https://github.com/libp2p/go-libp2p/pull/1863))\n - Fix concurrent map access in connmgr (#1860) ([libp2p/go-libp2p#1860](https://github.com/libp2p/go-libp2p/pull/1860))\n - fix: return filtered addrs (#1855) ([libp2p/go-libp2p#1855](https://github.com/libp2p/go-libp2p/pull/1855))\n - chore: preallocate slices (#1842) ([libp2p/go-libp2p#1842](https://github.com/libp2p/go-libp2p/pull/1842))\n - Close ping stream when we exit the loop (#1853) ([libp2p/go-libp2p#1853](https://github.com/libp2p/go-libp2p/pull/1853))\n - tls: don't set the deprecated tls.Config.PreferServerCipherSuites field (#1845) ([libp2p/go-libp2p#1845](https://github.com/libp2p/go-libp2p/pull/1845))\n - routed host: search for new multi addresses upon connect failure (#1835) ([libp2p/go-libp2p#1835](https://github.com/libp2p/go-libp2p/pull/1835))\n - core/peerstore: removed unused provider addr ttl constant (#1848) ([libp2p/go-libp2p#1848](https://github.com/libp2p/go-libp2p/pull/1848))\n - basichost: improve protocol negotiation debug message (#1846) ([libp2p/go-libp2p#1846](https://github.com/libp2p/go-libp2p/pull/1846))\n - noise: use Noise Extension to negotiate the muxer during the handshake (#1813) ([libp2p/go-libp2p#1813](https://github.com/libp2p/go-libp2p/pull/1813))\n - webtransport: use the rcmgr to control flow control window increases ([libp2p/go-libp2p#1832](https://github.com/libp2p/go-libp2p/pull/1832))\n - chore: update quic-go to v0.30.0 (#1838) ([libp2p/go-libp2p#1838](https://github.com/libp2p/go-libp2p/pull/1838))\n - roadmap: reorder priority, reorganize sections (#1831) ([libp2p/go-libp2p#1831](https://github.com/libp2p/go-libp2p/pull/1831))\n - websocket: set the HTTP host header in WSS(#1834) ([libp2p/go-libp2p#1834](https://github.com/libp2p/go-libp2p/pull/1834))\n - webtransport: make it possible to record qlogs (controlled by QLOGDIR env) ([libp2p/go-libp2p#1828](https://github.com/libp2p/go-libp2p/pull/1828))\n - ipfs /api/v0/id is post (#1819) ([libp2p/go-libp2p#1819](https://github.com/libp2p/go-libp2p/pull/1819))\n - examples: connect to all peers in example mdns chat app (#1798) ([libp2p/go-libp2p#1798](https://github.com/libp2p/go-libp2p/pull/1798))\n - roadmap: fix header level on \"Mid Q4\" (#1818) ([libp2p/go-libp2p#1818](https://github.com/libp2p/go-libp2p/pull/1818))\n - examples: use circuitv2 in relay example (#1795) ([libp2p/go-libp2p#1795](https://github.com/libp2p/go-libp2p/pull/1795))\n - add a roadmap for the next 6 months (#1784) ([libp2p/go-libp2p#1784](https://github.com/libp2p/go-libp2p/pull/1784))\n - tls: use ALPN to negotiate the stream multiplexer (#1772) ([libp2p/go-libp2p#1772](https://github.com/libp2p/go-libp2p/pull/1772))\n - tls: add tests for test vector from the spec (#1788) ([libp2p/go-libp2p#1788](https://github.com/libp2p/go-libp2p/pull/1788))\n - examples: update go-libp2p to v0.23.x (#1803) ([libp2p/go-libp2p#1803](https://github.com/libp2p/go-libp2p/pull/1803))\n - Try increasing timeouts if we're in CI for this test (#1796) ([libp2p/go-libp2p#1796](https://github.com/libp2p/go-libp2p/pull/1796))\n - Don't use rcmgr in this test (#1799) ([libp2p/go-libp2p#1799](https://github.com/libp2p/go-libp2p/pull/1799))\n - Bump timeout in CI for flaky test (#1800) ([libp2p/go-libp2p#1800](https://github.com/libp2p/go-libp2p/pull/1800))\n - Bump timeout in CI for flaky test (#1801) ([libp2p/go-libp2p#1801](https://github.com/libp2p/go-libp2p/pull/1801))\n - Fix comment in webtransport client auth handshake (#1793) ([libp2p/go-libp2p#1793](https://github.com/libp2p/go-libp2p/pull/1793))\n - examples: add basic pubsub-with-rendezvous example (#1738) ([libp2p/go-libp2p#1738](https://github.com/libp2p/go-libp2p/pull/1738))\n - quic: speed up the stateless reset test case (#1778) ([libp2p/go-libp2p#1778](https://github.com/libp2p/go-libp2p/pull/1778))\n - tls: fix flaky handshake cancellation test (#1779) ([libp2p/go-libp2p#1779](https://github.com/libp2p/go-libp2p/pull/1779))\n- github.com/libp2p/go-libp2p-gostream (v0.3.0 -> v0.5.0):\n - release v0.5.0 (#74) ([libp2p/go-libp2p-gostream#74](https://github.com/libp2p/go-libp2p-gostream/pull/74))\n - update go-libp2p to v0.22.0 (#73) ([libp2p/go-libp2p-gostream#73](https://github.com/libp2p/go-libp2p-gostream/pull/73))\n - Expose some read-only methods on the underlying Stream interface (#67) ([libp2p/go-libp2p-gostream#67](https://github.com/libp2p/go-libp2p-gostream/pull/67))\n - Update libp2p ([libp2p/go-libp2p-gostream#69](https://github.com/libp2p/go-libp2p-gostream/pull/69))\n - sync: update CI config files (#65) ([libp2p/go-libp2p-gostream#65](https://github.com/libp2p/go-libp2p-gostream/pull/65))\n - sync: update CI config files ([libp2p/go-libp2p-gostream#62](https://github.com/libp2p/go-libp2p-gostream/pull/62))\n - fix staticcheck ([libp2p/go-libp2p-gostream#61](https://github.com/libp2p/go-libp2p-gostream/pull/61))\n- github.com/libp2p/go-libp2p-http (v0.2.1 -> v0.4.0):\n - release v0.4.0 ([libp2p/go-libp2p-http#81](https://github.com/libp2p/go-libp2p-http/pull/81))\n - sync: update CI config files ([libp2p/go-libp2p-http#79](https://github.com/libp2p/go-libp2p-http/pull/79))\n - Update to latest go-libp2p ([libp2p/go-libp2p-http#80](https://github.com/libp2p/go-libp2p-http/pull/80))\n - Update to latest go-libp2p ([libp2p/go-libp2p-http#78](https://github.com/libp2p/go-libp2p-http/pull/78))\n - sync: update CI config files (#73) ([libp2p/go-libp2p-http#73](https://github.com/libp2p/go-libp2p-http/pull/73))\n- github.com/libp2p/go-libp2p-kad-dht (v0.18.0 -> v0.20.0):\n - release v0.20.0 (#803) ([libp2p/go-libp2p-kad-dht#803](https://github.com/libp2p/go-libp2p-kad-dht/pull/803))\n - feat: increase the max record age to 48h (PUT_VALUE, RFM17) (#794) ([libp2p/go-libp2p-kad-dht#794](https://github.com/libp2p/go-libp2p-kad-dht/pull/794))\n - feat: increase expiration time for Provider Records to 48h (RFM17)\n - release v0.19.0 (#801) ([libp2p/go-libp2p-kad-dht#801](https://github.com/libp2p/go-libp2p-kad-dht/pull/801))\n - define the ProviderAddrTTL in this repo (#797) ([libp2p/go-libp2p-kad-dht#797](https://github.com/libp2p/go-libp2p-kad-dht/pull/797))\n- github.com/libp2p/go-libp2p-kbucket (v0.4.7 -> v0.5.0):\n - chore: release 0.5.0 (#111) ([libp2p/go-libp2p-kbucket#111](https://github.com/libp2p/go-libp2p-kbucket/pull/111))\n - deprecate go-libp2p-core and use go-libp2p instead (#109) ([libp2p/go-libp2p-kbucket#109](https://github.com/libp2p/go-libp2p-kbucket/pull/109))\n - sync: update CI config files (#108) ([libp2p/go-libp2p-kbucket#108](https://github.com/libp2p/go-libp2p-kbucket/pull/108))\n - sync: update CI config files ([libp2p/go-libp2p-kbucket#107](https://github.com/libp2p/go-libp2p-kbucket/pull/107))\n - sync: update CI config files (#104) ([libp2p/go-libp2p-kbucket#104](https://github.com/libp2p/go-libp2p-kbucket/pull/104))\n - sync: update CI config files ([libp2p/go-libp2p-kbucket#101](https://github.com/libp2p/go-libp2p-kbucket/pull/101))\n - sync: update CI config files ([libp2p/go-libp2p-kbucket#99](https://github.com/libp2p/go-libp2p-kbucket/pull/99))\n - fix staticcheck ([libp2p/go-libp2p-kbucket#98](https://github.com/libp2p/go-libp2p-kbucket/pull/98))\n- github.com/libp2p/go-libp2p-pubsub (v0.6.1 -> v0.8.2):\n - Add docstring for WithAppSpecificRPCInspector (#510) ([libp2p/go-libp2p-pubsub#510](https://github.com/libp2p/go-libp2p-pubsub/pull/510))\n - Adds Application Specific RPC Inspector (#509) ([libp2p/go-libp2p-pubsub#509](https://github.com/libp2p/go-libp2p-pubsub/pull/509))\n - chore: ignore signing keys during WithLocalPublication publishing (#497) ([libp2p/go-libp2p-pubsub#497](https://github.com/libp2p/go-libp2p-pubsub/pull/497))\n - improve handling of dead peers (#508) ([libp2p/go-libp2p-pubsub#508](https://github.com/libp2p/go-libp2p-pubsub/pull/508))\n - perf: use pooled buffers for message writes (#507) ([libp2p/go-libp2p-pubsub#507](https://github.com/libp2p/go-libp2p-pubsub/pull/507))\n - perf: use msgio pooled buffers for received msgs (#500) ([libp2p/go-libp2p-pubsub#500](https://github.com/libp2p/go-libp2p-pubsub/pull/500))\n - Enables injectable GossipSub router (#503) ([libp2p/go-libp2p-pubsub#503](https://github.com/libp2p/go-libp2p-pubsub/pull/503))\n - Enables non-atomic validation for peer scoring parameters (#499) ([libp2p/go-libp2p-pubsub#499](https://github.com/libp2p/go-libp2p-pubsub/pull/499))\n - update go-libp2p to v0.22.0 (#498) ([libp2p/go-libp2p-pubsub#498](https://github.com/libp2p/go-libp2p-pubsub/pull/498))\n - fix handling of dead peers (#492) ([libp2p/go-libp2p-pubsub#492](https://github.com/libp2p/go-libp2p-pubsub/pull/492))\n - feat: WithLocalPublication option to enable local only publishing on a topic (#481) ([libp2p/go-libp2p-pubsub#481](https://github.com/libp2p/go-libp2p-pubsub/pull/481))\n - update pubsub deps (#491) ([libp2p/go-libp2p-pubsub#491](https://github.com/libp2p/go-libp2p-pubsub/pull/491))\n - Gossipsub: Unsubscribe backoff (#488) ([libp2p/go-libp2p-pubsub#488](https://github.com/libp2p/go-libp2p-pubsub/pull/488))\n - Adds exponential backoff to re-spawning new streams for supposedly dead peers (#483) ([libp2p/go-libp2p-pubsub#483](https://github.com/libp2p/go-libp2p-pubsub/pull/483))\n - Publishing option for signing a message with a custom private key (#486) ([libp2p/go-libp2p-pubsub#486](https://github.com/libp2p/go-libp2p-pubsub/pull/486))\n - fix unused GossipSubHistoryGossip, make seenMessages ttl configurable, make score params SeenMsgTTL configurable\n - Update README.md\n - Add in Backoff Check\n - Modify comment\n - Add Backoff For Pruned Peers\n - tests: new test for WithTopicMsgIdFunction\n - chore: better name\n - feat: detach WithMsgIdFunction\n - fix: use RawID in traceRPCMeta to avoid allocations\n - feat: extract RawID from ID\n - chore: hello mister mutex hat\n - chore: go fmt and return timecache named import\n - feat: new WithMsgIdFunction topic option to enable topics to have own msg id generation rules\n - feat: integrate msgIdGenerator\n - feat: introduce msgIdGenerator and add ID field to Message wrapper\n- github.com/libp2p/go-libp2p-pubsub-router (v0.5.0 -> v0.6.0):\n - release v0.6.0 (#99) ([libp2p/go-libp2p-pubsub-router#99](https://github.com/libp2p/go-libp2p-pubsub-router/pull/99))\n - sync: update CI config files (#93) ([libp2p/go-libp2p-pubsub-router#93](https://github.com/libp2p/go-libp2p-pubsub-router/pull/93))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.4.0 -> v0.6.0):\n - Update version.json\n - Change interface name\n - Add tests\n - Feat: retrieve routers from composable routers\n - Update version.json\n - Update version.json\n - chore: add regression test for compparallel deadlock\n - Add logs to composable parallel\n - Bump version to v0.4.1\n - ([libp2p/go-libp2p-routing-helpers#64](https://github.com/libp2p/go-libp2p-routing-helpers/pull/64))\n- github.com/lucas-clemente/quic-go (v0.29.1 -> v0.31.1):\n - qerr: include role (remote / local) in error string representations (#3629) ([lucas-clemente/quic-go#3629](https://github.com/lucas-clemente/quic-go/pull/3629))\n - introduce a type for the stateless reset key (#3621) ([lucas-clemente/quic-go#3621](https://github.com/lucas-clemente/quic-go/pull/3621))\n - limit the exponential PTO backoff to 60s (#3595) ([lucas-clemente/quic-go#3595](https://github.com/lucas-clemente/quic-go/pull/3595))\n - expose the QUIC version of a connection (#3620) ([lucas-clemente/quic-go#3620](https://github.com/lucas-clemente/quic-go/pull/3620))\n - expose function to convert byte slice to a connection ID (#3614) ([lucas-clemente/quic-go#3614](https://github.com/lucas-clemente/quic-go/pull/3614))\n - use `go run` for mockgen, goimports and ginkgo (#3616) ([lucas-clemente/quic-go#3616](https://github.com/lucas-clemente/quic-go/pull/3616))\n - fix client SNI handling (#3613) ([lucas-clemente/quic-go#3613](https://github.com/lucas-clemente/quic-go/pull/3613))\n - chore: fix multiple typos in comments (#3612) ([lucas-clemente/quic-go#3612](https://github.com/lucas-clemente/quic-go/pull/3612))\n - use the new zero-allocation control message parsing function from x/sys (#3609) ([lucas-clemente/quic-go#3609](https://github.com/lucas-clemente/quic-go/pull/3609))\n - http3: add support for parsing and writing HTTP/3 capsules (#3607) ([lucas-clemente/quic-go#3607](https://github.com/lucas-clemente/quic-go/pull/3607))\n - http3: add request to response (#3608) ([lucas-clemente/quic-go#3608](https://github.com/lucas-clemente/quic-go/pull/3608))\n - fix availability signaling of the send queue (#3597) ([lucas-clemente/quic-go#3597](https://github.com/lucas-clemente/quic-go/pull/3597))\n - http3: add a ConnectionState method to the StreamCreator interface (#3600) ([lucas-clemente/quic-go#3600](https://github.com/lucas-clemente/quic-go/pull/3600))\n - http3: add a Context method to the StreamCreator interface (#3601) ([lucas-clemente/quic-go#3601](https://github.com/lucas-clemente/quic-go/pull/3601))\n - rename the variable in quic.Config.AllowConnectionWindowIncrease (#3602) ([lucas-clemente/quic-go#3602](https://github.com/lucas-clemente/quic-go/pull/3602))\n - migrate to Ginkgo v2, remove benchmark test ([lucas-clemente/quic-go#3589](https://github.com/lucas-clemente/quic-go/pull/3589))\n - don't drop more than 10 consecutive packets in drop test (#3584) ([lucas-clemente/quic-go#3584](https://github.com/lucas-clemente/quic-go/pull/3584))\n - use a monotonous timer for the connection (#3570) ([lucas-clemente/quic-go#3570](https://github.com/lucas-clemente/quic-go/pull/3570))\n - http3: add http3.Server.ServeQUICConn to serve a single QUIC connection (#3587) ([lucas-clemente/quic-go#3587](https://github.com/lucas-clemente/quic-go/pull/3587))\n - http3: expose ALPN values (#3580) ([lucas-clemente/quic-go#3580](https://github.com/lucas-clemente/quic-go/pull/3580))\n - log the size of buffered packets (#3571) ([lucas-clemente/quic-go#3571](https://github.com/lucas-clemente/quic-go/pull/3571))\n - ackhandler: reject duplicate packets in ReceivedPacket (#3568) ([lucas-clemente/quic-go#3568](https://github.com/lucas-clemente/quic-go/pull/3568))\n - reduce max DATAGRAM frame size, so that DATAGRAMs fit in IPv6 packets (#3581) ([lucas-clemente/quic-go#3581](https://github.com/lucas-clemente/quic-go/pull/3581))\n - use a Peek / Pop API for the datagram queue (#3582) ([lucas-clemente/quic-go#3582](https://github.com/lucas-clemente/quic-go/pull/3582))\n - http3: handle ErrAbortHandler when the handler panics (#3575) ([lucas-clemente/quic-go#3575](https://github.com/lucas-clemente/quic-go/pull/3575))\n - http3: fix double close of chan when using DontCloseRequestStream (#3561) ([lucas-clemente/quic-go#3561](https://github.com/lucas-clemente/quic-go/pull/3561))\n - qlog: rename key_retired to key_discarded (#3463) ([lucas-clemente/quic-go#3463](https://github.com/lucas-clemente/quic-go/pull/3463))\n - simplify packing of ACK-only packets ([lucas-clemente/quic-go#3545](https://github.com/lucas-clemente/quic-go/pull/3545))\n - use a sync.Pool for ACK frames ([lucas-clemente/quic-go#3547](https://github.com/lucas-clemente/quic-go/pull/3547))\n - prioritize sending ACKs over sending new DATAGRAM frames (#3544) ([lucas-clemente/quic-go#3544](https://github.com/lucas-clemente/quic-go/pull/3544))\n - http3: reduce usage of bytes.Buffer (#3539) ([lucas-clemente/quic-go#3539](https://github.com/lucas-clemente/quic-go/pull/3539))\n - use a single bytes.Reader for frame parsing (#3536) ([lucas-clemente/quic-go#3536](https://github.com/lucas-clemente/quic-go/pull/3536))\n - split code paths for packing 0-RTT and 1-RTT packets in packet packer (#3540) ([lucas-clemente/quic-go#3540](https://github.com/lucas-clemente/quic-go/pull/3540))\n - remove the wire.ShortHeader in favor of more return values (#3535) ([lucas-clemente/quic-go#3535](https://github.com/lucas-clemente/quic-go/pull/3535))\n - preallocate the message buffers of the ipv4.Message passed to ReadBatch (#3541) ([lucas-clemente/quic-go#3541](https://github.com/lucas-clemente/quic-go/pull/3541))\n - introduce a separate code paths for Short Header packet handling ([lucas-clemente/quic-go#3534](https://github.com/lucas-clemente/quic-go/pull/3534))\n - fix usage of ackhandler.Packet pool for non-ack-eliciting packets (#3538) ([lucas-clemente/quic-go#3538](https://github.com/lucas-clemente/quic-go/pull/3538))\n - return an error when parsing a too long connection ID from a header (#3533) ([lucas-clemente/quic-go#3533](https://github.com/lucas-clemente/quic-go/pull/3533))\n - speed up marshaling of transport parameters (#3531) ([lucas-clemente/quic-go#3531](https://github.com/lucas-clemente/quic-go/pull/3531))\n - use a struct containing an array to represent Connection IDs ([lucas-clemente/quic-go#3529](https://github.com/lucas-clemente/quic-go/pull/3529))\n - reduce allocations of ackhandler.Packet ([lucas-clemente/quic-go#3525](https://github.com/lucas-clemente/quic-go/pull/3525))\n - serialize frames by appending to a byte slice, not to a bytes.Buffer ([lucas-clemente/quic-go#3530](https://github.com/lucas-clemente/quic-go/pull/3530))\n - fix datagram RFC number in documentation for quic.Config (#3523) ([lucas-clemente/quic-go#3523](https://github.com/lucas-clemente/quic-go/pull/3523))\n - add DPLPMTUD (RFC 8899) to list of supported RFCs in README (#3520) ([lucas-clemente/quic-go#3520](https://github.com/lucas-clemente/quic-go/pull/3520))\n - use the null tracers in the tracer integration tests (#3528) ([lucas-clemente/quic-go#3528](https://github.com/lucas-clemente/quic-go/pull/3528))\n- github.com/marten-seemann/webtransport-go (v0.1.1 -> v0.4.3):\n - release v0.4.3 (#57) ([marten-seemann/webtransport-go#57](https://github.com/marten-seemann/webtransport-go/pull/57))\n - return the correct error from OpenStreamSync when context is canceled (#55) ([marten-seemann/webtransport-go#55](https://github.com/marten-seemann/webtransport-go/pull/55))\n - release v0.4.2 (#54) ([marten-seemann/webtransport-go#54](https://github.com/marten-seemann/webtransport-go/pull/54))\n - use a buffered channel in the acceptQueue (#53) ([marten-seemann/webtransport-go#53](https://github.com/marten-seemann/webtransport-go/pull/53))\n - add a comment why using (a blocking) Read in the StreamHijacker is fine\n - release v0.4.1 (#52) ([marten-seemann/webtransport-go#52](https://github.com/marten-seemann/webtransport-go/pull/52))\n - release session mutex when an error occurs when closing (#51) ([marten-seemann/webtransport-go#51](https://github.com/marten-seemann/webtransport-go/pull/51))\n - release v0.4.0 (#48) ([marten-seemann/webtransport-go#48](https://github.com/marten-seemann/webtransport-go/pull/48))\n - add a Server.ServeQUICConn method (#47) ([marten-seemann/webtransport-go#47](https://github.com/marten-seemann/webtransport-go/pull/47))\n - release v0.3.0 (#46) ([marten-seemann/webtransport-go#46](https://github.com/marten-seemann/webtransport-go/pull/46))\n - read and write CLOSE_WEBTRANSPORT_SESSION capsules ([marten-seemann/webtransport-go#40](https://github.com/marten-seemann/webtransport-go/pull/40))\n - implement the SetDeadline method on the stream (#44) ([marten-seemann/webtransport-go#44](https://github.com/marten-seemann/webtransport-go/pull/44))\n - expose the QUIC stream ID on the stream interfaces (#43) ([marten-seemann/webtransport-go#43](https://github.com/marten-seemann/webtransport-go/pull/43))\n - release v0.2.0 (#38) ([marten-seemann/webtransport-go#38](https://github.com/marten-seemann/webtransport-go/pull/38))\n - expose quic-go's connection tracing ID on the Session.Context (#35) ([marten-seemann/webtransport-go#35](https://github.com/marten-seemann/webtransport-go/pull/35))\n - add a ConnectionState method to the Session (#33) ([marten-seemann/webtransport-go#33](https://github.com/marten-seemann/webtransport-go/pull/33))\n - chore: update quic-go to v0.30.0 (#36) ([marten-seemann/webtransport-go#36](https://github.com/marten-seemann/webtransport-go/pull/36))\n - fix interop build (#37) ([marten-seemann/webtransport-go#37](https://github.com/marten-seemann/webtransport-go/pull/37))\n - rename session receiver variable (#34) ([marten-seemann/webtransport-go#34](https://github.com/marten-seemann/webtransport-go/pull/34))\n - fix double close of chan when using DontCloseRequestStream (#30) ([marten-seemann/webtransport-go#30](https://github.com/marten-seemann/webtransport-go/pull/30))\n - add a simple integration test using Selenium and a headless Chrome (#28) ([marten-seemann/webtransport-go#28](https://github.com/marten-seemann/webtransport-go/pull/28))\n - use a generic accept queue for uni- and bidirectional streams (#26) ([marten-seemann/webtransport-go#26](https://github.com/marten-seemann/webtransport-go/pull/26))\n- github.com/multiformats/go-base36 (v0.1.0 -> v0.2.0):\n - v0.2.0\n - sync: update CI config files (#11) ([multiformats/go-base36#11](https://github.com/multiformats/go-base36/pull/11))\n - fix link to documentation (#9) ([multiformats/go-base36#9](https://github.com/multiformats/go-base36/pull/9))\n - sync: update CI config files (#8) ([multiformats/go-base36#8](https://github.com/multiformats/go-base36/pull/8))\n - Address `staticcheck` issue ([multiformats/go-base36#5](https://github.com/multiformats/go-base36/pull/5))\n - Feat/fasterer ([multiformats/go-base36#4](https://github.com/multiformats/go-base36/pull/4))\n- github.com/multiformats/go-multiaddr (v0.7.0 -> v0.8.0):\n - release v0.8.0 ([multiformats/go-multiaddr#187](https://github.com/multiformats/go-multiaddr/pull/187))\n - Add quic-v1 component ([multiformats/go-multiaddr#186](https://github.com/multiformats/go-multiaddr/pull/186))\n- github.com/multiformats/go-varint (v0.0.6 -> v0.0.7):\n - v0.0.7 ([multiformats/go-varint#18](https://github.com/multiformats/go-varint/pull/18))\n - feat: optimize decoding (#15) ([multiformats/go-varint#15](https://github.com/multiformats/go-varint/pull/15))\n - sync: update CI config files (#13) ([multiformats/go-varint#13](https://github.com/multiformats/go-varint/pull/13))\n - fix staticcheck ([multiformats/go-varint#9](https://github.com/multiformats/go-varint/pull/9))\n - tests for unbounded uvarint streams. ([multiformats/go-varint#7](https://github.com/multiformats/go-varint/pull/7))\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20210219115102-f37d292932f2 -> v0.0.0-20221220214510-0333c149dec0):\n - fix bug in consts code\n - Allow 'const' fields to be declared ([whyrusleeping/cbor-gen#78](https://github.com/whyrusleeping/cbor-gen/pull/78))\n - support omitting empty fields on map encoders ([whyrusleeping/cbor-gen#77](https://github.com/whyrusleeping/cbor-gen/pull/77))\n - support string pointers\n - feat: add the ability to encode a byte array (#76) ([whyrusleeping/cbor-gen#76](https://github.com/whyrusleeping/cbor-gen/pull/76))\n - support string slices (#73) ([whyrusleeping/cbor-gen#73](https://github.com/whyrusleeping/cbor-gen/pull/73))\n - Feat/size types ([whyrusleeping/cbor-gen#69](https://github.com/whyrusleeping/cbor-gen/pull/69))\n - Add CborReader and CborWriter (#67) ([whyrusleeping/cbor-gen#67](https://github.com/whyrusleeping/cbor-gen/pull/67))\n - Fix broken TestTimeIsh (#66) ([whyrusleeping/cbor-gen#66](https://github.com/whyrusleeping/cbor-gen/pull/66))\n - Return EOF and ErrUnexpectedEOF correctly (#64) ([whyrusleeping/cbor-gen#64](https://github.com/whyrusleeping/cbor-gen/pull/64))\n - fix: don't fail if we try to discard nothing at the end of an object ([whyrusleeping/cbor-gen#63](https://github.com/whyrusleeping/cbor-gen/pull/63))\n - Make peeker.ReadByte follow buffer.ReadByte semantics ([whyrusleeping/cbor-gen#61](https://github.com/whyrusleeping/cbor-gen/pull/61))\n - Fix read bug in readByteBuf ([whyrusleeping/cbor-gen#60](https://github.com/whyrusleeping/cbor-gen/pull/60))\n - support for cborgen struct field tags ([whyrusleeping/cbor-gen#58](https://github.com/whyrusleeping/cbor-gen/pull/58))\n - feat: take cbor adapters by-value when encoding ([whyrusleeping/cbor-gen#55](https://github.com/whyrusleeping/cbor-gen/pull/55))\n - fix: import \"math\" in generated code for uint8 unmarshalling ([whyrusleeping/cbor-gen#53](https://github.com/whyrusleeping/cbor-gen/pull/53))\n - doc: document Write*EncodersToFile functions ([whyrusleeping/cbor-gen#52](https://github.com/whyrusleeping/cbor-gen/pull/52))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 154 | +8826/-6369 | 911 |\n| Gus Eggert | 7 | +2792/-1444 | 40 |\n| Marco Munizaga | 26 | +2324/-752 | 101 |\n| hannahhoward | 7 | +695/-1587 | 50 |\n| Rod Vagg | 30 | +1508/-668 | 106 |\n| Henrique Dias | 13 | +1321/-431 | 85 |\n| Yahya Hassanzadeh | 4 | +984/-158 | 9 |\n| galargh | 17 | +519/-520 | 20 |\n| Steve Loeppky | 11 | +612/-418 | 25 |\n| Antonio Navarro Perez | 30 | +742/-88 | 47 |\n| Marcin Rataj | 19 | +377/-407 | 52 |\n| Ian Davis | 2 | +419/-307 | 7 |\n| whyrusleeping | 5 | +670/-28 | 17 |\n| Piotr Galar | 8 | +211/-417 | 25 |\n| web3-bot | 28 | +282/-264 | 75 |\n| Will Scott | 10 | +428/-103 | 19 |\n| julian88110 | 2 | +367/-55 | 27 |\n| Will | 5 | +282/-131 | 65 |\n| Jorropo | 25 | +263/-94 | 38 |\n| Wondertan | 10 | +203/-87 | 24 |\n| Mohsin Zaidi | 1 | +269/-0 | 4 |\n| Dennis Trautwein | 3 | +230/-21 | 7 |\n| Prithvi Shahi | 1 | +116/-77 | 1 |\n| Masih H. Derkani | 5 | +130/-37 | 11 |\n| Iulian Pascalau | 1 | +151/-16 | 2 |\n| Scott Martin | 1 | +166/-0 | 3 |\n| Daniel Vernall | 1 | +92/-45 | 2 |\n| Steven Allen | 7 | +114/-15 | 11 |\n| Hlib Kanunnikov | 4 | +100/-28 | 6 |\n| Peter Rabbitson | 4 | +59/-65 | 5 |\n| Lucas Molas | 1 | +60/-57 | 7 |\n| nisdas | 3 | +107/-6 | 5 |\n| why | 2 | +80/-20 | 5 |\n| ShengTao | 2 | +46/-45 | 16 |\n| nisainan | 2 | +40/-50 | 12 |\n| Mikel Cortes | 3 | +44/-36 | 10 |\n| Chinmay Kousik | 1 | +64/-14 | 6 |\n| ZenGround0 | 2 | +62/-15 | 6 |\n| Antonio Navarro | 3 | +58/-3 | 8 |\n| Michael Muré | 2 | +49/-2 | 2 |\n| Dirk McCormick | 1 | +3/-42 | 1 |\n| kixelated | 1 | +20/-20 | 4 |\n| Russell Dempsey | 1 | +19/-17 | 3 |\n| Karthik Nallabolu | 1 | +17/-17 | 1 |\n| protolambda | 1 | +26/-4 | 4 |\n| cliffc-spirent | 1 | +25/-5 | 2 |\n| Raúl Kripalani | 1 | +29/-0 | 1 |\n| Håvard Anda Estensen | 1 | +9/-19 | 6 |\n| vyzo | 1 | +11/-12 | 1 |\n| anorth | 1 | +15/-8 | 3 |\n| shade34321 | 1 | +21/-1 | 2 |\n| Toby | 2 | +9/-13 | 6 |\n| Nishant Das | 1 | +9/-9 | 5 |\n| Jeromy Johnson | 1 | +17/-0 | 3 |\n| Oleg | 1 | +14/-1 | 1 |\n| Hector Sanjuan | 6 | +4/-11 | 6 |\n| Łukasz Magiera | 2 | +10/-4 | 2 |\n| Aayush Rajasekaran | 1 | +7/-7 | 1 |\n| Adin Schmahmann | 1 | +4/-3 | 1 |\n| Stojan Dimitrovski | 1 | +6/-0 | 1 |\n| Mathew Jacob | 1 | +3/-3 | 1 |\n| Vladimir Ivanov | 1 | +2/-2 | 1 |\n| Nex Zhu | 1 | +4/-0 | 2 |\n| Michele Mastrogiovanni | 1 | +2/-2 | 1 |\n| Louis Thibault | 1 | +4/-0 | 1 |\n| Eric Myhre | 1 | +3/-1 | 1 |\n| Kubo Mage | 2 | +2/-1 | 2 |\n| tabcat | 1 | +1/-1 | 1 |\n| Viacheslav | 1 | +1/-1 | 1 |\n| Max Inden | 1 | +1/-1 | 1 |\n| Manic Security | 1 | +1/-1 | 1 |\n| Jc0803kevin | 1 | +1/-1 | 1 |\n| David Brouwer | 1 | +2/-0 | 2 |\n| Rong Zhou | 1 | +1/-0 | 1 |\n| Neel Virdy | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.19.md", "content": "\n# Kubo changelog v0.19\n\n## v0.19.2\n\n### Highlights\n\n#### FullRT DHT HTTP Routers\n\nThe default HTTP routers are now used when the FullRT DHT client is used. This fixes\nthe issue where cid.contact is not being queried by default when the accelerated\nDHT client was enabled. Read more in ([ipfs/kubo#9841](https://github.com/ipfs/kubo/pull/9841)).\n\n### Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix: use default HTTP routers when FullRT DHT client is used (#9841) ([ipfs/kubo#9841](https://github.com/ipfs/kubo/pull/9841))\n - chore: update version\n\n
\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Gus Eggert | 1 | +65/-53 | 4 |\n| Henrique Dias | 1 | +1/-1 | 1 |\n\n## v0.19.1\n\n### 🔦 Highlights\n\n#### DHT Timeouts\nIn v0.16.0, Kubo added the ability to configure custom content routers and DHTs with the `custom` router type, and as part of this added a default 5 minute timeout to all DHT operations. In some cases with large repos ([example](https://github.com/ipfs/kubo/issues/9722)), this can cause provide and reprovide operations to fail because the timeout is reached. This release removes these timeouts on DHT operations. If users desire these timeouts, they can be added back using [the `custom` router type](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingrouters-parameters).\n\n### Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - fix: remove timeout on default DHT operations (#9783) ([ipfs/kubo#9783](https://github.com/ipfs/kubo/pull/9783))\n - chore: update version\n- github.com/ipfs/go-blockservice (v0.5.0 -> v0.5.1):\n - chore: release v0.5.1\n - fix: remove busyloop in getBlocks by removing batching\n- github.com/libp2p/go-libp2p (v0.26.3 -> v0.26.4):\n - release v0.26.4\n - autorelay: fix busy loop bug and flaky tests in relay finder (#2208) ([libp2p/go-libp2p#2208](https://github.com/libp2p/go-libp2p/pull/2208))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.6.1 -> v0.6.2):\n - Release v0.6.2 (#73) ([libp2p/go-libp2p-routing-helpers#73](https://github.com/libp2p/go-libp2p-routing-helpers/pull/73))\n - feat: zero timeout on composed routers should disable timeout (#72) ([libp2p/go-libp2p-routing-helpers#72](https://github.com/libp2p/go-libp2p-routing-helpers/pull/72))\n\n
\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marco Munizaga | 1 | +347/-46 | 5 |\n| Gus Eggert | 3 | +119/-93 | 8 |\n| Jorropo | 2 | +20/-32 | 2 |\n| galargh | 2 | +2/-2 | 2 |\n| Marten Seemann | 1 | +2/-2 | 1 |\n\n\n## v0.19.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Improving the libp2p resource management integration](#improving-the-libp2p-resource-management-integration)\n - [Gateways](#gateways)\n - [Signed IPNS Record response format](#signed-ipns-record-response-format)\n - [Example fetch and inspect IPNS record](#example-fetch-and-inspect-ipns-record)\n - [Addition of \"autoclient\" router type](#addition-of-autoclient-router-type)\n - [Deprecation of the `ipfs pubsub` commands and matching HTTP endpoints](#deprecation-of-the-ipfs-pubsub-commands-and-matching-http-endpoints)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Improving the libp2p resource management integration\n\nThere are further followups up on libp2p resource manager improvements in Kubo [0.18.0](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.18.md#improving-libp2p-resource-management-integration-1)\nand [0.18.1](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.18.md#improving-libp2p-resource-management-integration):\n1. `ipfs swarm limits` and `ipfs swarm stats` have been replaced by `ipfs swarm resources` to provide a single/combined view for limits and their current usage in a more intuitive ordering.\n1. Removal of `Swarm.ResourceMgr.Limits` config. Instead [the power user can specify limits in a .json file that are fed directly to go-libp2p](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#user-supplied-override-limits). This allows the power user to take advantage of the [new resource manager types introduced in go-libp2p 0.25](https://github.com/libp2p/go-libp2p/blob/master/CHANGELOG.md#new-resource-manager-types-) including \"use default\", \"unlimited\", \"block all\".\n - Note: we don't expect most users to need these capabilities, but they are there if so.\n1. [Doc updates](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md).\n\n#### Gateways\n\n##### Signed IPNS Record response format\n\nThis release implements [IPIP-351](https://github.com/ipfs/specs/pull/351) and\nadds Gateway support for returning signed (verifiable) `ipns-record` (0x0300)\nwhen `/ipns/{libp2p-key}` is requested with either\n`Accept: application/vnd.ipfs.ipns-record` HTTP header\nor `?format=ipns-record` URL query parameter.\n\n\nThe Gateway in Kubo already supported [trustless, verifiable retrieval](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval) of immutable `/ipfs/` namespace.\nWith `?format=ipns-record`, light HTTP clients are now able to get the same level of verifiability for IPNS websites.\n\nTooling is limited at the moment, but we are working on [go-libipfs](https://github.com/ipfs/go-libipfs/) examples that illustrate the verifiable HTTP client pattern.\n\n##### Example: fetch IPNS record over HTTP and inspect it with `ipfs name inspect --verify`\n\n```console\n$ FILE_CID=$(echo \"Hello IPFS\" | ipfs add --cid-version 1 -q)\n$ IPNS_KEY=$(ipfs key gen test)\n$ ipfs name publish /ipfs/$FILE_CID --key=test --ttl=30m\nPublished to k51q..dvf1: /ipfs/bafk..z244\n$ curl \"http://127.0.0.1:8080/ipns/$IPNS_KEY?format=ipns-record\" > signed.ipns-record\n$ ipfs name inspect --verify $IPNS_KEY < signed.ipns-record\nValue: \"/ipfs/bafk...\"\nValidity Type: \"EOL\"\nValidity: 2023-03-09T23:13:34.032977468Z\nSequence: 0\nTTL: 1800000000000\nPublicKey: \"\"\nSignature V1: \"m...\"\nSignature V2: \"m...\"\nData: {...}\n\nValidation results:\n Valid: true\n PublicKey: 12D3...\n```\n\n#### Addition of \"autoclient\" router type\nA new routing type \"autoclient\" has been added. This mode is similar to \"auto\", in that it is a hybrid of content routers (including Kademlia and HTTP routers), but it does not run a DHT server. This is similar to the difference between \"dhtclient\" and \"dht\" router types.\n\nSee the [Routing.Type documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingtype) for more information.\n\n#### Deprecation of the `ipfs pubsub` commands and matching HTTP endpoints\n\nWe are deprecating `ipfs pubsub` and all `/api/v0/pubsub/` RPC endpoints and will remove them in the next release.\n\nFor more information and rational see [#9717](https://github.com/ipfs/kubo/issues/9717).\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - docs: 0.19 changelog ([ipfs/kubo#9707](https://github.com/ipfs/kubo/pull/9707))\n - fix: canonicalize user defined headers\n - fix: apply API.HTTPHeaders to /webui redirect\n - feat: add heap allocs to 'ipfs diag profile'\n - fix: future proof with > rcmgr.DefaultLimit for new enum rcmgr values\n - test: add test for presarvation of unlimited configs for inbound systems\n - fix: preserve Unlimited StreamsInbound in connmgr reconciliation\n - test: fix flaky rcmgr test\n - chore: deprecate the pubsub api\n - test: port peering test from sharness to Go\n - test: use `T.TempDir` to create temporary test directory\n - fix: --verify forgets the verified key\n - test: name --verify forgets the verified key\n - feat: add \"autoclient\" routing type\n - test: parallelize more of rcmgr Go tests\n - test: port legacy DHT tests to Go\n - fix: t0116-gateway-cache.sh ([ipfs/kubo#9696](https://github.com/ipfs/kubo/pull/9696))\n - docs: add bifrost to early testers ([ipfs/kubo#9699](https://github.com/ipfs/kubo/pull/9699))\n - fix: typo in documentation for install path\n - chore: update version\n - feat: Reduce RM code footprint\n - Doc updates/additions\n - ci: replace junit html generation with gh action\n - test: port rcmgr sharness tests to Go\n - test(gateway): use deterministic CAR fixtures ([ipfs/kubo#9657](https://github.com/ipfs/kubo/pull/9657))\n - feat(gateway): error handling improvements (500, 502, 504) (#9660) ([ipfs/kubo#9660](https://github.com/ipfs/kubo/pull/9660))\n - docs: be clear about swarm.addrfilters (#9661) ([ipfs/kubo#9661](https://github.com/ipfs/kubo/pull/9661))\n - chore: update go-libp2p to v0.26 (#9656) ([ipfs/kubo#9656](https://github.com/ipfs/kubo/pull/9656))\n - feat(pinning): connect some missing go context (#9557) ([ipfs/kubo#9557](https://github.com/ipfs/kubo/pull/9557))\n - fix(gateway): return HTTP 500 on ErrResolveFailed (#9589) ([ipfs/kubo#9589](https://github.com/ipfs/kubo/pull/9589))\n - docs: bulk spelling edits (#9544) ([ipfs/kubo#9544](https://github.com/ipfs/kubo/pull/9544))\n - docs: \"remote\" errors from resource manager (#9653) ([ipfs/kubo#9653](https://github.com/ipfs/kubo/pull/9653))\n - test: remove gateway tests migrated to go-libipfs\n - fix: update rcmgr for go-libp2p v0.25\n - chore: update go-libp2p to v0.25.1\n - docs(0.18.1): guide users to clean up limits (#9644) ([ipfs/kubo#9644](https://github.com/ipfs/kubo/pull/9644))\n - feat: add NewOptionalInteger function\n - fix: dereference int64 pointer in OptionalInteger.String() (#9640) ([ipfs/kubo#9640](https://github.com/ipfs/kubo/pull/9640))\n - fix: restore wire format for /api/v0/routing/get|put (#9639) ([ipfs/kubo#9639](https://github.com/ipfs/kubo/pull/9639))\n - refactor(gw): move Host (DNSLink and subdomain) handling to go-libipfs (#9624) ([ipfs/kubo#9624](https://github.com/ipfs/kubo/pull/9624))\n - refactor: new go-libipfs/gateway API, deprecate Gateway.Writable (#9616) ([ipfs/kubo#9616](https://github.com/ipfs/kubo/pull/9616))\n - Create Changelog: v0.19 ([ipfs/kubo#9617](https://github.com/ipfs/kubo/pull/9617))\n - refactor: use gateway from go-libipfs (#9588) ([ipfs/kubo#9588](https://github.com/ipfs/kubo/pull/9588))\n - Merge Release: v0.18.1 ([ipfs/kubo#9613](https://github.com/ipfs/kubo/pull/9613))\n - Add overview section\n - Adjust inbound connection limits depending on memory.\n - feat: ipfs-webui 2.22.0\n - chore: bump go-libipfs remove go-bitswap\n - docs: DefaultResourceMgrMinInboundConns\n - feat(gateway): IPNS record response format (IPIP-351) (#9399) ([ipfs/kubo#9399](https://github.com/ipfs/kubo/pull/9399))\n - fix(ipns): honour --ttl flag in 'ipfs name publish' (#9471) ([ipfs/kubo#9471](https://github.com/ipfs/kubo/pull/9471))\n - feat: Pubsub.SeenMessagesStrategy (#9543) ([ipfs/kubo#9543](https://github.com/ipfs/kubo/pull/9543))\n - chore: bump go-libipfs to replace go-block-format\n - Merge Kubo: v0.18 ([ipfs/kubo#9581](https://github.com/ipfs/kubo/pull/9581))\n - fix: clarity: no user supplied rcmgr limits of 0 (#9563) ([ipfs/kubo#9563](https://github.com/ipfs/kubo/pull/9563))\n - fix(gateway): undesired conversions to dag-json and friends (#9566) ([ipfs/kubo#9566](https://github.com/ipfs/kubo/pull/9566))\n - fix: ensure connmgr is smaller then autoscalled resource limits\n - fix: typo in ensureConnMgrMakeSenseVsResourcesMgr\n - docs: clarify browser descriptions for webtransport\n - fix: update saxon download path\n - fix: refuse to start if connmgr is smaller than resource limits and not using none connmgr\n - fix: User-Agent sent to HTTP routers\n - test: port gateway sharness tests to Go tests\n - fix: do not download saxon in parallel\n - docs: improve docs/README (#9539) ([ipfs/kubo#9539](https://github.com/ipfs/kubo/pull/9539))\n - test: port CircleCI to GH Actions and improve sharness reporting (#9355) ([ipfs/kubo#9355](https://github.com/ipfs/kubo/pull/9355))\n - chore: migrate from go-ipfs-files to go-libipfs/files (#9535) ([ipfs/kubo#9535](https://github.com/ipfs/kubo/pull/9535))\n - fix: stats dht command when Routing.Type=auto (#9538) ([ipfs/kubo#9538](https://github.com/ipfs/kubo/pull/9538))\n - fix: hint people to changing from RSA peer ids\n - fix(gateway): JSON when Accept is a list\n - fix(test): retry flaky t0125-twonode.sh\n - docs: fix Router config Godoc (#9528) ([ipfs/kubo#9528](https://github.com/ipfs/kubo/pull/9528))\n - fix(ci): flaky sharness test\n - docs(config): ProviderSearchDelay (#9526) ([ipfs/kubo#9526](https://github.com/ipfs/kubo/pull/9526))\n - docs: clarify debug environment variables\n - fix: disable provide over HTTP with Routing.Type=auto (#9511) ([ipfs/kubo#9511](https://github.com/ipfs/kubo/pull/9511))\n - fix(test): stabilize flaky provider tests\n - feat: port pins CLI test\n - Removing QRI from early tester ([ipfs/kubo#9503](https://github.com/ipfs/kubo/pull/9503))\n - Update Version (dev): v0.18 ([ipfs/kubo#9500](https://github.com/ipfs/kubo/pull/9500))\n- github.com/ipfs/go-bitfield (v1.0.0 -> v1.1.0):\n - Merge pull request from GHSA-2h6c-j3gf-xp9r\n - sync: update CI config files (#3) ([ipfs/go-bitfield#3](https://github.com/ipfs/go-bitfield/pull/3))\n- github.com/ipfs/go-block-format (v0.0.3 -> v0.1.1):\n - chore: release v0.1.1\n - docs: fix wrong copy paste in docs\n - chore: release v0.1.0\n - refactor: deprecate and add stub types to go-libipfs/blocks\n - sync: update CI config files (#34) ([ipfs/go-block-format#34](https://github.com/ipfs/go-block-format/pull/34))\n - remove Makefile ([ipfs/go-block-format#31](https://github.com/ipfs/go-block-format/pull/31))\n- github.com/ipfs/go-ipfs-files (v0.0.8 -> v0.3.0):\n - ([ipfs/go-ipfs-files#59](https://github.com/ipfs/go-ipfs-files/pull/59))\n - docs: add moved noticed [ci skip]\n - Release v0.2.0\n - fix: error when TAR has files outside of root (#56) ([ipfs/go-ipfs-files#56](https://github.com/ipfs/go-ipfs-files/pull/56))\n - sync: update CI config files ([ipfs/go-ipfs-files#55](https://github.com/ipfs/go-ipfs-files/pull/55))\n - chore(Directory): add DirIterator API restriction: iterate only once\n - Release v0.1.1\n - fix: add dragonfly build option for filewriter flags\n - fix: add freebsd build option for filewriter flags\n - Release v0.1.0\n - docs: fix community CONTRIBUTING.md link (#45) ([ipfs/go-ipfs-files#45](https://github.com/ipfs/go-ipfs-files/pull/45))\n - chore(filewriter): cleanup writes (#43) ([ipfs/go-ipfs-files#43](https://github.com/ipfs/go-ipfs-files/pull/43))\n - sync: update CI config files (#44) ([ipfs/go-ipfs-files#44](https://github.com/ipfs/go-ipfs-files/pull/44))\n - sync: update CI config files ([ipfs/go-ipfs-files#40](https://github.com/ipfs/go-ipfs-files/pull/40))\n - fix: manually parse the content disposition to preserve directories ([ipfs/go-ipfs-files#42](https://github.com/ipfs/go-ipfs-files/pull/42))\n - fix: round timestamps down by truncating them to seconds ([ipfs/go-ipfs-files#41](https://github.com/ipfs/go-ipfs-files/pull/41))\n - sync: update CI config files ([ipfs/go-ipfs-files#34](https://github.com/ipfs/go-ipfs-files/pull/34))\n - Fix test failure on Windows caused by nil `sys` in mock `FileInfo` ([ipfs/go-ipfs-files#39](https://github.com/ipfs/go-ipfs-files/pull/39))\n - fix staticcheck ([ipfs/go-ipfs-files#35](https://github.com/ipfs/go-ipfs-files/pull/35))\n - fix linters ([ipfs/go-ipfs-files#33](https://github.com/ipfs/go-ipfs-files/pull/33))\n- github.com/ipfs/go-ipfs-pinner (v0.2.1 -> v0.3.0):\n - chore: release v0.3.0 (#27) ([ipfs/go-ipfs-pinner#27](https://github.com/ipfs/go-ipfs-pinner/pull/27))\n - feat!: add and connect missing context, remove RemovePinWithMode (#23) ([ipfs/go-ipfs-pinner#23](https://github.com/ipfs/go-ipfs-pinner/pull/23))\n - sync: update CI config files ([ipfs/go-ipfs-pinner#16](https://github.com/ipfs/go-ipfs-pinner/pull/16))\n- github.com/ipfs/go-ipfs-pq (v0.0.2 -> v0.0.3):\n - chore: release v0.0.3\n - fix: enable early GC\n - sync: update CI config files (#10) ([ipfs/go-ipfs-pq#10](https://github.com/ipfs/go-ipfs-pq/pull/10))\n - sync: update CI config files ([ipfs/go-ipfs-pq#8](https://github.com/ipfs/go-ipfs-pq/pull/8))\n - remove Makefile ([ipfs/go-ipfs-pq#7](https://github.com/ipfs/go-ipfs-pq/pull/7))\n- github.com/ipfs/go-libipfs (v0.2.0 -> v0.6.2):\n - chore: release 0.6.2 (#211) ([ipfs/go-libipfs#211](https://github.com/ipfs/go-libipfs/pull/211))\n - fix(gateway): 500 on panic, recover on WithHostname\n - refactor: use assert in remaining gateway tests\n - chore: release 0.6.1\n - feat: support HTTP 429 with Retry-After (#194) ([ipfs/go-libipfs#194](https://github.com/ipfs/go-libipfs/pull/194))\n - docs: fix typo in README.md\n - fix(gateway): return 500 for all /ip[nf]s/id failures\n - chore: make gocritic happier\n - feat(gateway): improved error handling, support for 502 and 504 ([ipfs/go-libipfs#182](https://github.com/ipfs/go-libipfs/pull/182))\n - feat: add content path in request context (#184) ([ipfs/go-libipfs#184](https://github.com/ipfs/go-libipfs/pull/184))\n - sync: update CI config files ([ipfs/go-libipfs#159](https://github.com/ipfs/go-libipfs/pull/159))\n - fix(gateway): return HTTP 500 on namesys.ErrResolveFailed (#150) ([ipfs/go-libipfs#150](https://github.com/ipfs/go-libipfs/pull/150))\n - docs(examples): add UnixFS file download over Bitswap (#143) ([ipfs/go-libipfs#143](https://github.com/ipfs/go-libipfs/pull/143))\n - bitswap/server/internal/decision: fix: remove unused private type\n - chore: release v0.6.0\n - bitswap/server/internal/decision: add more non flaky tests\n - bitswap/server/internal/decision: add filtering on CIDs - Ignore cids that are too big. - Kill connection for peers that are using inline CIDs.\n - bitswap/server/internal/decision: rewrite ledger inversion\n - docs(readme): various updates for clarity (#171) ([ipfs/go-libipfs#171](https://github.com/ipfs/go-libipfs/pull/171))\n - feat: metric for implicit index.html in dirs\n - fix(gateway): ensure ipfs_http_gw_get_duration_seconds gets updated\n - test(gateway): migrate Go tests from Kubo ([ipfs/go-libipfs#156](https://github.com/ipfs/go-libipfs/pull/156))\n - docs: fix link (#165) ([ipfs/go-libipfs#165](https://github.com/ipfs/go-libipfs/pull/165))\n - fix: GetIPNSRecord example gateway implementation (#158) ([ipfs/go-libipfs#158](https://github.com/ipfs/go-libipfs/pull/158))\n - chore: release v0.5.0\n - chore: update go-libp2p to v0.25.1\n - fix(gateway): display correct error with 500 (#160) ([ipfs/go-libipfs#160](https://github.com/ipfs/go-libipfs/pull/160))\n - fix: gateway car example dnslink\n - feat(gateway): add TAR, IPNS Record, DAG-* histograms and spans (#155) ([ipfs/go-libipfs#155](https://github.com/ipfs/go-libipfs/pull/155))\n - feat(gateway): migrate subdomain and dnslink code (#153) ([ipfs/go-libipfs#153](https://github.com/ipfs/go-libipfs/pull/153))\n - docs: add example of gateway that proxies to ?format=raw (#151) ([ipfs/go-libipfs#151](https://github.com/ipfs/go-libipfs/pull/151))\n - docs: add example of gateway backed by CAR file (#147) ([ipfs/go-libipfs#147](https://github.com/ipfs/go-libipfs/pull/147))\n - undefined ([ipfs/go-libipfs#145](https://github.com/ipfs/go-libipfs/pull/145))\n - Extract Gateway Code From Kubo\n ([ipfs/go-libipfs#65](https://github.com/ipfs/go-libipfs/pull/65))\n - Migrate go-bitswap ([ipfs/go-libipfs#63](https://github.com/ipfs/go-libipfs/pull/63))\n - Use `PUT` as method to insert provider records\n - Migrate `go-block-format` ([ipfs/go-libipfs#58](https://github.com/ipfs/go-libipfs/pull/58))\n - chore: add codecov PR comment\n - chore: add a logo and some basics in the README (#37) ([ipfs/go-libipfs#37](https://github.com/ipfs/go-libipfs/pull/37))\n- github.com/ipfs/go-namesys (v0.6.0 -> v0.7.0):\n - chore: release 0.7.0 (#36) ([ipfs/go-namesys#36](https://github.com/ipfs/go-namesys/pull/36))\n - feat: use PublishOptions for publishing IPNS records (#35) ([ipfs/go-namesys#35](https://github.com/ipfs/go-namesys/pull/35))\n- github.com/ipfs/go-path (v0.3.0 -> v0.3.1):\n - chore: release v0.3.1 (#67) ([ipfs/go-path#67](https://github.com/ipfs/go-path/pull/67))\n - feat: expose ErrInvalidPath and implement .Is function (#66) ([ipfs/go-path#66](https://github.com/ipfs/go-path/pull/66))\n - sync: update CI config files (#60) ([ipfs/go-path#60](https://github.com/ipfs/go-path/pull/60))\n - feat: add basic tracing ([ipfs/go-path#59](https://github.com/ipfs/go-path/pull/59))\n- github.com/ipfs/go-peertaskqueue (v0.8.0 -> v0.8.1):\n - chore: release v0.8.1\n - feat: add PushTasksTruncated which only push a limited amount of tasks\n - feat: add (*PeerTaskQueue).Clear which fully removes a peer\n - sync: update CI config files (#26) ([ipfs/go-peertaskqueue#26](https://github.com/ipfs/go-peertaskqueue/pull/26))\n- github.com/ipfs/go-unixfs (v0.4.2 -> v0.4.4):\n - chore: release v0.4.4\n - fix: correctly handle return errors\n - fix: correctly handle errors in balancedbuilder's Layout\n - test: fix tests after hamt issues fixes\n - Merge pull request from GHSA-q264-w97q-q778\n- github.com/ipfs/go-unixfsnode (v1.5.1 -> v1.5.2):\n - Merge pull request from GHSA-4gj3-6r43-3wfc\n- github.com/ipfs/interface-go-ipfs-core (v0.8.2 -> v0.11.0):\n - chore: release v0.11.0\n - test: basic routing interface test\n - chore: release v0.10.0 (#102) ([ipfs/interface-go-ipfs-core#102](https://github.com/ipfs/interface-go-ipfs-core/pull/102))\n - feat: add RoutingAPI to CoreAPI\n - chore: release 0.9.0 (#101) ([ipfs/interface-go-ipfs-core#101](https://github.com/ipfs/interface-go-ipfs-core/pull/101))\n - feat: add namesys publish options (#94) ([ipfs/interface-go-ipfs-core#94](https://github.com/ipfs/interface-go-ipfs-core/pull/94))\n- github.com/ipld/go-car (v0.4.0 -> v0.5.0):\n - chore: bump version to 0.5.0\n - fix: remove use of ioutil\n - run gofmt -s\n - bump go.mod to Go 1.18 and run go fix\n - bump go.mod to Go 1.18 and run go fix\n - OpenReadWriteFile: add test\n - blockstore: allow to pass a file to write in (#323) ([ipld/go-car#323](https://github.com/ipld/go-car/pull/323))\n - feat: add `car inspect` command to cmd pkg (#320) ([ipld/go-car#320](https://github.com/ipld/go-car/pull/320))\n - Separate `index.ReadFrom` tests\n - Only read index codec during inspection\n - Upgrade to the latest `go-car/v2`\n - Empty identity CID should be indexed when options are set\n- github.com/libp2p/go-libp2p (v0.24.2 -> v0.26.3):\n - Release v0.26.3 (#2197) ([libp2p/go-libp2p#2197](https://github.com/libp2p/go-libp2p/pull/2197))\n - retract v0.26.1, release v0.26.2 (#2153) ([libp2p/go-libp2p#2153](https://github.com/libp2p/go-libp2p/pull/2153))\n - rcmgr: fix JSON marshalling of ResourceManagerStat peer map (#2156) ([libp2p/go-libp2p#2156](https://github.com/libp2p/go-libp2p/pull/2156))\n - release v0.26.1 ([libp2p/go-libp2p#2146](https://github.com/libp2p/go-libp2p/pull/2146))\n - release v0.26.0 (#2133) ([libp2p/go-libp2p#2133](https://github.com/libp2p/go-libp2p/pull/2133))\n - identify: add more detailed metrics (#2126) ([libp2p/go-libp2p#2126](https://github.com/libp2p/go-libp2p/pull/2126))\n - autorelay: refactor relay finder and start autorelay after identify (#2120) ([libp2p/go-libp2p#2120](https://github.com/libp2p/go-libp2p/pull/2120))\n - don't use the time value from the time.Ticker channel (#2127) ([libp2p/go-libp2p#2127](https://github.com/libp2p/go-libp2p/pull/2127))\n - Wrap conn with metrics (#2131) ([libp2p/go-libp2p#2131](https://github.com/libp2p/go-libp2p/pull/2131))\n - chore: update changelog for 0.26.0 (#2132) ([libp2p/go-libp2p#2132](https://github.com/libp2p/go-libp2p/pull/2132))\n - circuitv2: Update proto files to proto3 (#2121) ([libp2p/go-libp2p#2121](https://github.com/libp2p/go-libp2p/pull/2121))\n - swarm: remove parallel tests from swarm tests (#2130) ([libp2p/go-libp2p#2130](https://github.com/libp2p/go-libp2p/pull/2130))\n - circuitv2: add a relay option to disable limits (#2125) ([libp2p/go-libp2p#2125](https://github.com/libp2p/go-libp2p/pull/2125))\n - quic: fix stalled virtual listener (#2122) ([libp2p/go-libp2p#2122](https://github.com/libp2p/go-libp2p/pull/2122))\n - swarm: add early muxer selection to swarm metrics (#2119) ([libp2p/go-libp2p#2119](https://github.com/libp2p/go-libp2p/pull/2119))\n - metrics: add options to disable metrics and to set Prometheus registerer (#2116) ([libp2p/go-libp2p#2116](https://github.com/libp2p/go-libp2p/pull/2116))\n - swarm: add ip_version to metrics (#2114) ([libp2p/go-libp2p#2114](https://github.com/libp2p/go-libp2p/pull/2114))\n - Revert mistaken \"Bump timeout\"\n - Bump timeout\n - remove all circuit v1 related code (#2107) ([libp2p/go-libp2p#2107](https://github.com/libp2p/go-libp2p/pull/2107))\n - quic: don't send detailed error messages when closing connections (#2112) ([libp2p/go-libp2p#2112](https://github.com/libp2p/go-libp2p/pull/2112))\n - metrics: add no alloc metrics for eventbus, swarm, identify (#2108) ([libp2p/go-libp2p#2108](https://github.com/libp2p/go-libp2p/pull/2108))\n - chore: fix typo in Changelog (#2111) ([libp2p/go-libp2p#2111](https://github.com/libp2p/go-libp2p/pull/2111))\n - chore: update changelog (#2109) ([libp2p/go-libp2p#2109](https://github.com/libp2p/go-libp2p/pull/2109))\n - chore: unify dashboard location (#2110) ([libp2p/go-libp2p#2110](https://github.com/libp2p/go-libp2p/pull/2110))\n - autonat: add metrics (#2086) ([libp2p/go-libp2p#2086](https://github.com/libp2p/go-libp2p/pull/2086))\n - relaymanager: do not start new relay if one already exists (#2093) ([libp2p/go-libp2p#2093](https://github.com/libp2p/go-libp2p/pull/2093))\n - autonat: don't emit reachability changed events on address change (#2092) ([libp2p/go-libp2p#2092](https://github.com/libp2p/go-libp2p/pull/2092))\n - chore: modify changelog entries (#2101) ([libp2p/go-libp2p#2101](https://github.com/libp2p/go-libp2p/pull/2101))\n - Introduce a changelog (#2084) ([libp2p/go-libp2p#2084](https://github.com/libp2p/go-libp2p/pull/2084))\n - use atomic.Int32 and atomic.Int64 (#2096) ([libp2p/go-libp2p#2096](https://github.com/libp2p/go-libp2p/pull/2096))\n - change atomic.Value to atomic.Pointer (#2088) ([libp2p/go-libp2p#2088](https://github.com/libp2p/go-libp2p/pull/2088))\n - use atomic.Bool instead of int32 operations (#2089) ([libp2p/go-libp2p#2089](https://github.com/libp2p/go-libp2p/pull/2089))\n - sync: update CI config files (#2073) ([libp2p/go-libp2p#2073](https://github.com/libp2p/go-libp2p/pull/2073))\n - chore: update examples to v0.25.1 (#2080) ([libp2p/go-libp2p#2080](https://github.com/libp2p/go-libp2p/pull/2080))\n - v0.25.1 (#2082) ([libp2p/go-libp2p#2082](https://github.com/libp2p/go-libp2p/pull/2082))\n - Start host in mocknet (#2078) ([libp2p/go-libp2p#2078](https://github.com/libp2p/go-libp2p/pull/2078))\n - Release v0.25.0 (#2077) ([libp2p/go-libp2p#2077](https://github.com/libp2p/go-libp2p/pull/2077))\n - identify: add some basic metrics (#2069) ([libp2p/go-libp2p#2069](https://github.com/libp2p/go-libp2p/pull/2069))\n - p2p/test/quic: use contexts with a timeout for Connect calls (#2070) ([libp2p/go-libp2p#2070](https://github.com/libp2p/go-libp2p/pull/2070))\n - feat!: rcmgr: Change LimitConfig to use LimitVal type (#2000) ([libp2p/go-libp2p#2000](https://github.com/libp2p/go-libp2p/pull/2000))\n - identify: refactor sending of Identify pushes (#1984) ([libp2p/go-libp2p#1984](https://github.com/libp2p/go-libp2p/pull/1984))\n - Update interop to match spec (#2049) ([libp2p/go-libp2p#2049](https://github.com/libp2p/go-libp2p/pull/2049))\n - chore: git-ignore various flavors of qlog files (#2064) ([libp2p/go-libp2p#2064](https://github.com/libp2p/go-libp2p/pull/2064))\n - rcmgr: add libp2p prefix to all metrics (#2063) ([libp2p/go-libp2p#2063](https://github.com/libp2p/go-libp2p/pull/2063))\n - websocket: Replace gorilla websocket transport with nhooyr websocket transport (#1982) ([libp2p/go-libp2p#1982](https://github.com/libp2p/go-libp2p/pull/1982))\n - rcmgr: Use prometheus SDK for rcmgr metrics (#2044) ([libp2p/go-libp2p#2044](https://github.com/libp2p/go-libp2p/pull/2044))\n - autorelay: Split libp2p.EnableAutoRelay into 2 functions (#2022) ([libp2p/go-libp2p#2022](https://github.com/libp2p/go-libp2p/pull/2022))\n - set names for eventbus event subscriptions (#2057) ([libp2p/go-libp2p#2057](https://github.com/libp2p/go-libp2p/pull/2057))\n - Test cleanup (#2053) ([libp2p/go-libp2p#2053](https://github.com/libp2p/go-libp2p/pull/2053))\n - metrics: use a single slice pool for all metrics tracer (#2054) ([libp2p/go-libp2p#2054](https://github.com/libp2p/go-libp2p/pull/2054))\n - eventbus: add metrics (#2038) ([libp2p/go-libp2p#2038](https://github.com/libp2p/go-libp2p/pull/2038))\n - quic: disable sending of Version Negotiation packets (#2015) ([libp2p/go-libp2p#2015](https://github.com/libp2p/go-libp2p/pull/2015))\n - p2p/test: fix flaky notification test (#2051) ([libp2p/go-libp2p#2051](https://github.com/libp2p/go-libp2p/pull/2051))\n - quic, tcp: only register Prometheus counters when metrics are enabled ([libp2p/go-libp2p#1971](https://github.com/libp2p/go-libp2p/pull/1971))\n - p2p/test: add test for EvtLocalAddressesUpdated event (#2016) ([libp2p/go-libp2p#2016](https://github.com/libp2p/go-libp2p/pull/2016))\n - quic / webtransport: extend test to test dialing draft-29 and v1 (#1957) ([libp2p/go-libp2p#1957](https://github.com/libp2p/go-libp2p/pull/1957))\n - holepunch: fix flaky by not remove holepunch protocol handler (#1948) ([libp2p/go-libp2p#1948](https://github.com/libp2p/go-libp2p/pull/1948))\n - use quic-go and webtransport-go from quic-go organization (#2040) ([libp2p/go-libp2p#2040](https://github.com/libp2p/go-libp2p/pull/2040))\n - Migrate to test-plan composite action (#2039) ([libp2p/go-libp2p#2039](https://github.com/libp2p/go-libp2p/pull/2039))\n - chore: remove license files from the eventbus package (#2042) ([libp2p/go-libp2p#2042](https://github.com/libp2p/go-libp2p/pull/2042))\n - rcmgr: *: Always close connscope (#2037) ([libp2p/go-libp2p#2037](https://github.com/libp2p/go-libp2p/pull/2037))\n - chore: remove textual roadmap in favor for Starmap (#2036) ([libp2p/go-libp2p#2036](https://github.com/libp2p/go-libp2p/pull/2036))\n - swarm metrics: fix datasource for dashboard (#2024) ([libp2p/go-libp2p#2024](https://github.com/libp2p/go-libp2p/pull/2024))\n - consistently use protocol.ID instead of strings (#2004) ([libp2p/go-libp2p#2004](https://github.com/libp2p/go-libp2p/pull/2004))\n - swarm: add a basic metrics tracer (#1973) ([libp2p/go-libp2p#1973](https://github.com/libp2p/go-libp2p/pull/1973))\n - Expose muxer ids (#2012) ([libp2p/go-libp2p#2012](https://github.com/libp2p/go-libp2p/pull/2012))\n - Clean addresses with peer id before adding to addrbook (#2007) ([libp2p/go-libp2p#2007](https://github.com/libp2p/go-libp2p/pull/2007))\n - feat: ci test-plans: Parse test timeout parameter for interop test (#2014) ([libp2p/go-libp2p#2014](https://github.com/libp2p/go-libp2p/pull/2014))\n - Export resource manager errors (#2008) ([libp2p/go-libp2p#2008](https://github.com/libp2p/go-libp2p/pull/2008))\n - peerstore: make it possible to use an empty peer ID (#2006) ([libp2p/go-libp2p#2006](https://github.com/libp2p/go-libp2p/pull/2006))\n - Add ci flakiness score to readme (#2002) ([libp2p/go-libp2p#2002](https://github.com/libp2p/go-libp2p/pull/2002))\n - rcmgr: fix: Ignore zero values when marshalling Limits. (#1998) ([libp2p/go-libp2p#1998](https://github.com/libp2p/go-libp2p/pull/1998))\n - CI: Fast multidimensional Interop tests (#1991) ([libp2p/go-libp2p#1991](https://github.com/libp2p/go-libp2p/pull/1991))\n - feat: add some users to the readme (#1981) ([libp2p/go-libp2p#1981](https://github.com/libp2p/go-libp2p/pull/1981))\n - ci: run go generate as part of the go-check workflow (#1986) ([libp2p/go-libp2p#1986](https://github.com/libp2p/go-libp2p/pull/1986))\n - switch to Google's Protobuf library, make protobufs compile with go generate ([libp2p/go-libp2p#1979](https://github.com/libp2p/go-libp2p/pull/1979))\n - circuitv2: correctly set the transport in the ConnectionState (#1972) ([libp2p/go-libp2p#1972](https://github.com/libp2p/go-libp2p/pull/1972))\n - roadmap: remove optimizations of the TCP-based handshake (#1959) ([libp2p/go-libp2p#1959](https://github.com/libp2p/go-libp2p/pull/1959))\n - identify: remove support for Identify Delta ([libp2p/go-libp2p#1975](https://github.com/libp2p/go-libp2p/pull/1975))\n - core: remove introspection package (#1978) ([libp2p/go-libp2p#1978](https://github.com/libp2p/go-libp2p/pull/1978))\n - identify: remove old code targeting Go 1.17 (#1964) ([libp2p/go-libp2p#1964](https://github.com/libp2p/go-libp2p/pull/1964))\n - add WebTransport to the list of default transports (#1915) ([libp2p/go-libp2p#1915](https://github.com/libp2p/go-libp2p/pull/1915))\n - core/crypto: drop all OpenSSL code paths (#1953) ([libp2p/go-libp2p#1953](https://github.com/libp2p/go-libp2p/pull/1953))\n - chore: use generic LRU cache (#1980) ([libp2p/go-libp2p#1980](https://github.com/libp2p/go-libp2p/pull/1980))\n- github.com/libp2p/go-libp2p-kad-dht (v0.20.0 -> v0.21.1):\n - chore: bump to v0.21.1 (#821) ([libp2p/go-libp2p-kad-dht#821](https://github.com/libp2p/go-libp2p-kad-dht/pull/821))\n - feat: send FIND_NODE request to peers on routing table refresh (#810) ([libp2p/go-libp2p-kad-dht#810](https://github.com/libp2p/go-libp2p-kad-dht/pull/810))\n - chore: release v0.21.\n - chore: Update to go libp2p v0.25 ([libp2p/go-libp2p-kad-dht#815](https://github.com/libp2p/go-libp2p-kad-dht/pull/815))\n- github.com/libp2p/go-libp2p-pubsub (v0.8.3 -> v0.9.0):\n - chore: update to go-libp2p v0.25 (#517) ([libp2p/go-libp2p-pubsub#517](https://github.com/libp2p/go-libp2p-pubsub/pull/517))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.6.0 -> v0.6.1):\n - chore: release v0.6.1\n - fix: cancel parallel routers\n- github.com/libp2p/go-msgio (v0.2.0 -> v0.3.0):\n - release v0.3.0 (#39) ([libp2p/go-msgio#39](https://github.com/libp2p/go-msgio/pull/39))\n - switch from deprecated gogo to google.golang.org/protobuf ([libp2p/go-msgio#38](https://github.com/libp2p/go-msgio/pull/38))\n - sync: update CI config files (#36) ([libp2p/go-msgio#36](https://github.com/libp2p/go-msgio/pull/36))\n- github.com/lucas-clemente/quic-go (v0.31.1 -> v0.29.1):\n - http3: fix double close of chan when using DontCloseRequestStream\n- github.com/multiformats/go-multistream (v0.3.3 -> v0.4.1):\n - release v0.4.1 ([multiformats/go-multistream#101](https://github.com/multiformats/go-multistream/pull/101))\n - Fix errors Is checking ([multiformats/go-multistream#100](https://github.com/multiformats/go-multistream/pull/100))\n - release v0.4.0 (#93) ([multiformats/go-multistream#93](https://github.com/multiformats/go-multistream/pull/93))\n - switch to Go's native fuzzing (#96) ([multiformats/go-multistream#96](https://github.com/multiformats/go-multistream/pull/96))\n - Add not supported protocols to returned errors (#97) ([multiformats/go-multistream#97](https://github.com/multiformats/go-multistream/pull/97))\n - Make MultistreamMuxer and Client APIs generic (#95) ([multiformats/go-multistream#95](https://github.com/multiformats/go-multistream/pull/95))\n - remove MultistreamMuxer.NegotiateLazy (#92) ([multiformats/go-multistream#92](https://github.com/multiformats/go-multistream/pull/92))\n - sync: update CI config files (#91) ([multiformats/go-multistream#91](https://github.com/multiformats/go-multistream/pull/91))\n- github.com/warpfork/go-wish (v0.0.0-20200122115046-b9ea61034e4a -> v0.0.0-20220906213052-39a1cc7a02d0):\n - Update readme with deprecation info\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20221220214510-0333c149dec0 -> v0.0.0-20230126041949-52956bd4c9aa):\n - add setter to allow reuse of cborreader struct\n - fix typo\n - allow fields to be ignored ([whyrusleeping/cbor-gen#79](https://github.com/whyrusleeping/cbor-gen/pull/79))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Dirk McCormick | 128 | +16757/-7211 | 387 |\n| Henrique Dias | 69 | +7599/-10016 | 316 |\n| hannahhoward | 88 | +8503/-4397 | 271 |\n| Jeromy Johnson | 244 | +6544/-4034 | 774 |\n| Marten Seemann | 64 | +4870/-5628 | 266 |\n| Steven Allen | 296 | +4769/-3517 | 972 |\n| Brian Tiger Chow | 250 | +5520/-2579 | 435 |\n| Jorropo | 64 | +4237/-3548 | 302 |\n| Sukun | 18 | +4327/-1093 | 132 |\n| Marco Munizaga | 35 | +2809/-1294 | 94 |\n| Gus Eggert | 20 | +2523/-1476 | 99 |\n| Adin Schmahmann | 15 | +683/-2625 | 69 |\n| Marcin Rataj | 73 | +2348/-882 | 133 |\n| whyrusleeping | 12 | +1683/-1338 | 23 |\n| Jeromy | 99 | +1754/-1181 | 453 |\n| Juan Batiz-Benet | 69 | +1182/-678 | 149 |\n| Lars Gierth | 31 | +827/-358 | 92 |\n| Paul Wolneykien | 2 | +670/-338 | 9 |\n| Laurent Senta | 16 | +806/-134 | 53 |\n| Henry | 19 | +438/-372 | 36 |\n| Michael Muré | 8 | +400/-387 | 19 |\n| Łukasz Magiera | 56 | +413/-354 | 117 |\n| Jakub Sztandera | 40 | +413/-251 | 100 |\n| Justin Johnson | 2 | +479/-165 | 5 |\n| Piotr Galar | 7 | +227/-378 | 24 |\n| Kevin Atkinson | 11 | +252/-232 | 49 |\n| web3-bot | 17 | +236/-240 | 59 |\n| Petar Maymounkov | 2 | +348/-84 | 11 |\n| Hector Sanjuan | 38 | +206/-223 | 85 |\n| Antonio Navarro Perez | 9 | +259/-95 | 17 |\n| keks | 22 | +233/-118 | 24 |\n| Ho-Sheng Hsiao | 3 | +170/-170 | 30 |\n| Lucas Molas | 6 | +266/-54 | 16 |\n| Mildred Ki'Lya | 4 | +280/-35 | 7 |\n| Steve Loeppky | 5 | +147/-156 | 9 |\n| rht | 14 | +97/-188 | 20 |\n| Prithvi Shahi | 6 | +89/-193 | 11 |\n| Ian Davis | 6 | +198/-75 | 11 |\n| taylor | 1 | +180/-89 | 8 |\n| ᴍᴀᴛᴛ ʙᴇʟʟ | 14 | +158/-104 | 18 |\n| Chris Boddy | 6 | +190/-45 | 8 |\n| Rod Vagg | 3 | +203/-28 | 15 |\n| Masih H. Derkani | 8 | +165/-61 | 16 |\n| Kevin Wallace | 4 | +194/-27 | 7 |\n| Mohsin Zaidi | 1 | +179/-41 | 5 |\n| ElPaisano | 1 | +110/-110 | 22 |\n| Simon Zhu | 6 | +177/-32 | 8 |\n| galargh | 9 | +80/-120 | 14 |\n| Tomasz Zdybał | 1 | +180/-1 | 4 |\n| dgrisham | 3 | +176/-2 | 4 |\n| Michael Avila | 3 | +116/-59 | 8 |\n| Raúl Kripalani | 2 | +85/-77 | 34 |\n| Dr Ian Preston | 11 | +101/-48 | 11 |\n| JP Hastings-Spital | 1 | +145/-0 | 2 |\n| George Antoniadis | 6 | +59/-58 | 43 |\n| Kevin Neaton | 2 | +97/-16 | 4 |\n| Adrian Lanzafame | 6 | +81/-25 | 7 |\n| Dennis Trautwein | 3 | +89/-9 | 5 |\n| mathew-cf | 2 | +82/-9 | 5 |\n| tg | 1 | +41/-33 | 1 |\n| Eng Zer Jun | 1 | +15/-54 | 5 |\n| zramsay | 4 | +15/-53 | 12 |\n| muXxer | 1 | +28/-33 | 4 |\n| Thomas Eizinger | 1 | +24/-37 | 4 |\n| Remco Bloemen | 2 | +28/-18 | 3 |\n| Manuel Alonso | 1 | +36/-9 | 1 |\n| vyzo | 4 | +26/-12 | 13 |\n| Djalil Dreamski | 3 | +27/-9 | 3 |\n| Thomas Gardner | 2 | +32/-3 | 4 |\n| Jan Winkelmann | 2 | +23/-12 | 8 |\n| Artem Andreenko | 1 | +16/-19 | 1 |\n| James Stanley | 1 | +34/-0 | 1 |\n| Brendan McMillion | 1 | +10/-17 | 3 |\n| Jack Loughran | 1 | +22/-0 | 3 |\n| Peter Wu | 2 | +12/-9 | 2 |\n| Gowtham G | 4 | +14/-7 | 4 |\n| Tor Arne Vestbø | 3 | +19/-1 | 3 |\n| Cory Schwartz | 1 | +8/-12 | 5 |\n| Peter Rabbitson | 1 | +15/-4 | 1 |\n| David Dias | 1 | +9/-9 | 1 |\n| Will Scott | 1 | +13/-4 | 2 |\n| Eric Myhre | 1 | +15/-2 | 1 |\n| Stephen Whitmore | 1 | +8/-8 | 1 |\n| Rafael Ramalho | 5 | +11/-5 | 5 |\n| Christian Couder | 1 | +14/-2 | 1 |\n| W. Trevor King | 2 | +9/-6 | 3 |\n| Steven Vandevelde | 1 | +11/-3 | 1 |\n| Knut Ahlers | 3 | +9/-5 | 3 |\n| Bob Potter | 1 | +3/-10 | 1 |\n| Russell Dempsey | 4 | +8/-4 | 4 |\n| Diogo Silva | 4 | +8/-4 | 4 |\n| Dave Justice | 1 | +8/-4 | 1 |\n| Andy Leap | 2 | +2/-10 | 2 |\n| divingpetrel | 1 | +7/-4 | 2 |\n| Iaroslav Gridin | 1 | +9/-2 | 1 |\n| Dominic Della Valle | 3 | +5/-5 | 3 |\n| Vijayee Kulkaa | 1 | +3/-6 | 1 |\n| Friedel Ziegelmayer | 3 | +6/-3 | 3 |\n| Stephen Solka | 1 | +1/-7 | 1 |\n| Richard Littauer | 3 | +4/-4 | 3 |\n| Franky W | 2 | +4/-4 | 2 |\n| Dimitris Apostolou | 2 | +4/-4 | 3 |\n| Adrian Ulrich | 1 | +8/-0 | 1 |\n| Masashi Salvador Mitsuzawa | 1 | +5/-1 | 1 |\n| Gabe | 1 | +3/-3 | 1 |\n| zuuluuz | 1 | +4/-1 | 1 |\n| myml | 1 | +5/-0 | 1 |\n| swedneck | 1 | +3/-1 | 1 |\n| Wayback Archiver | 1 | +2/-2 | 1 |\n| Vladimir Ivanov | 1 | +2/-2 | 1 |\n| Péter Szilágyi | 1 | +2/-2 | 1 |\n| Karthik Bala | 1 | +2/-2 | 1 |\n| Etienne Laurin | 1 | +1/-3 | 1 |\n| Shotaro Yamada | 1 | +2/-1 | 1 |\n| Robert Carlsen | 1 | +2/-1 | 1 |\n| Oli Evans | 1 | +2/-1 | 1 |\n| Dan McQuillan | 1 | +2/-1 | 1 |\n| susarlanikhilesh | 1 | +1/-1 | 1 |\n| mateon1 | 1 | +1/-1 | 1 |\n| kpcyrd | 1 | +1/-1 | 1 |\n| bbenshoof | 1 | +1/-1 | 1 |\n| ZenGround0 | 1 | +1/-1 | 1 |\n| Will Hawkins | 1 | +1/-1 | 1 |\n| Tommi Virtanen | 1 | +1/-1 | 1 |\n| Seungbae Yu | 1 | +1/-1 | 1 |\n| Riishab Joshi | 1 | +1/-1 | 1 |\n| Kubo Mage | 1 | +1/-1 | 1 |\n| Ivan | 1 | +1/-1 | 1 |\n| Guillaume Renault | 1 | +1/-1 | 1 |\n| Anjor Kanekar | 1 | +1/-1 | 1 |\n| Andrew Chin | 1 | +1/-1 | 1 |\n| Abdul Rauf | 1 | +1/-1 | 1 |\n| makeworld | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.2.md", "content": "# go-ipfs changelog v0.2\n\n## 0.2.3 - 2015-03-01\n\n* Alpha Release\n\n## 2015-01-31:\n\n* bootstrap addresses now have .../ipfs/... in format\n config file Bootstrap field changed accordingly. users\n can upgrade cleanly with:\n\n ipfs bootstrap >bootstrap_peers\n ipfs bootstrap rm --all\n \n \n ipfs bootstrap add Full Changelog\n\n- github.com/ipfs/kubo:\n - fix: deadlock on retrieving WebTransport addresses (#9857) ([ipfs/kubo#9857](https://github.com/ipfs/kubo/pull/9857))\n - docs(config): remove mentions of relay v1 (#9860) ([ipfs/kubo#9860](https://github.com/ipfs/kubo/pull/9860))\n - Merge branch 'master' into merge-release-v0.19.2\n - docs: add changelog for v0.19.2\n - feat: webui@3.0.0 (#9835) ([ipfs/kubo#9835](https://github.com/ipfs/kubo/pull/9835))\n - fix: use default HTTP routers when FullRT DHT client is used (#9841) ([ipfs/kubo#9841](https://github.com/ipfs/kubo/pull/9841))\n - chore: update version\n - docs: add `ipfs pubsub` deprecation reminder to changelog (#9827) ([ipfs/kubo#9827](https://github.com/ipfs/kubo/pull/9827))\n - docs: preparing 0.20 changelog for release (#9799) ([ipfs/kubo#9799](https://github.com/ipfs/kubo/pull/9799))\n - feat: boxo tracing and traceparent support (#9811) ([ipfs/kubo#9811](https://github.com/ipfs/kubo/pull/9811))\n - chore: update version\n - chore: update version\n - update go-libp2p to v0.27.0\n - docs: add optimistic provide feature description\n - feat: add experimental optimistic provide\n - fix(ci): speed up docker build (#9800) ([ipfs/kubo#9800](https://github.com/ipfs/kubo/pull/9800))\n - feat(tracing): use OTEL_PROPAGATORS as per OTel spec (#9801) ([ipfs/kubo#9801](https://github.com/ipfs/kubo/pull/9801))\n - docs: fix jaeger command (#9797) ([ipfs/kubo#9797](https://github.com/ipfs/kubo/pull/9797))\n - Merge Release: v0.19.1 (#9794) ([ipfs/kubo#9794](https://github.com/ipfs/kubo/pull/9794))\n - chore: upgrade OpenTelemetry dependencies (#9736) ([ipfs/kubo#9736](https://github.com/ipfs/kubo/pull/9736))\n - test: fix flaky content routing over HTTP test (#9772) ([ipfs/kubo#9772](https://github.com/ipfs/kubo/pull/9772))\n - feat: allow injecting custom path resolvers (#9750) ([ipfs/kubo#9750](https://github.com/ipfs/kubo/pull/9750))\n - feat: add changelog entry for router timeouts for v0.19.1 (#9784) ([ipfs/kubo#9784](https://github.com/ipfs/kubo/pull/9784))\n - feat(gw): new metrics and HTTP range support (#9786) ([ipfs/kubo#9786](https://github.com/ipfs/kubo/pull/9786))\n - feat!: make --empty-repo default (#9758) ([ipfs/kubo#9758](https://github.com/ipfs/kubo/pull/9758))\n - fix: remove timeout on default DHT operations (#9783) ([ipfs/kubo#9783](https://github.com/ipfs/kubo/pull/9783))\n - refactor: switch gateway code to new API from go-libipfs (#9681) ([ipfs/kubo#9681](https://github.com/ipfs/kubo/pull/9681))\n - test: port remote pinning tests to Go (#9720) ([ipfs/kubo#9720](https://github.com/ipfs/kubo/pull/9720))\n - feat: add identify option to swarm peers command\n - test: port routing DHT tests to Go (#9709) ([ipfs/kubo#9709](https://github.com/ipfs/kubo/pull/9709))\n - test: fix autoclient flakiness (#9769) ([ipfs/kubo#9769](https://github.com/ipfs/kubo/pull/9769))\n - test: skip flaky pubsub test (#9770) ([ipfs/kubo#9770](https://github.com/ipfs/kubo/pull/9770))\n - chore: migrate go-libipfs to boxo\n - feat: add tracing to the commands client\n - feat: add client-side metrics for routing-v1 client\n - test: increase max wait time for peering assertion\n - feat: remove writable gateway (#9743) ([ipfs/kubo#9743](https://github.com/ipfs/kubo/pull/9743))\n - Process Improvement: v0.18.0 ([ipfs/kubo#9484](https://github.com/ipfs/kubo/pull/9484))\n - fix: deadlock while racing `ipfs dag import` and `ipfs repo gc`\n - feat: improve dag/import (#9721) ([ipfs/kubo#9721](https://github.com/ipfs/kubo/pull/9721))\n - ci: remove circleci config ([ipfs/kubo#9687](https://github.com/ipfs/kubo/pull/9687))\n - docs: use fx.Decorate instead of fx.Replace in examples (#9725) ([ipfs/kubo#9725](https://github.com/ipfs/kubo/pull/9725))\n - Create Changelog: v0.20 ([ipfs/kubo#9742](https://github.com/ipfs/kubo/pull/9742))\n - Merge Release: v0.19.0 ([ipfs/kubo#9741](https://github.com/ipfs/kubo/pull/9741))\n - feat(gateway): invalid CID returns 400 Bad Request (#9726) ([ipfs/kubo#9726](https://github.com/ipfs/kubo/pull/9726))\n - fix: remove outdated changelog part ([ipfs/kubo#9739](https://github.com/ipfs/kubo/pull/9739))\n - docs: 0.19 changelog ([ipfs/kubo#9707](https://github.com/ipfs/kubo/pull/9707))\n - fix: canonicalize user defined headers\n - fix: apply API.HTTPHeaders to /webui redirect\n - feat: add heap allocs to 'ipfs diag profile'\n - fix: future proof with > rcmgr.DefaultLimit for new enum rcmgr values\n - test: add test for presarvation of unlimited configs for inbound systems\n - fix: preserve Unlimited StreamsInbound in connmgr reconciliation\n - test: fix flaky rcmgr test\n - chore: deprecate the pubsub api\n - Revert \"chore: add hamt directory sharding test\"\n - chore: add hamt directory sharding test\n - test: port peering test from sharness to Go\n - test: use `T.TempDir` to create temporary test directory\n - fix: --verify forgets the verified key\n - test: name --verify forgets the verified key\n - chore: fix toc in changelog for 0.18\n - feat: add \"autoclient\" routing type\n - test: parallelize more of rcmgr Go tests\n - test: port legacy DHT tests to Go\n - fix: t0116-gateway-cache.sh ([ipfs/kubo#9696](https://github.com/ipfs/kubo/pull/9696))\n - docs: add bifrost to early testers ([ipfs/kubo#9699](https://github.com/ipfs/kubo/pull/9699))\n - fix: typo in documentation for install path\n - docs: fix typos\n - Update Version: v0.19 ([ipfs/kubo#9698](https://github.com/ipfs/kubo/pull/9698))\n- github.com/ipfs/go-block-format (v0.1.1 -> v0.1.2):\n - chore: release v0.1.2\n - Revert deprecation and go-libipfs/blocks stub types\n - docs: deprecation notice [ci skip]\n- github.com/ipfs/go-cid (v0.3.2 -> v0.4.1):\n - v0.4.1\n - Add unit test for unexpected eof\n - Update cid.go\n - CidFromReader should not wrap valid EOF return.\n - chore: version 0.4.0\n - feat: wrap parsing errors into ErrInvalidCid\n - fix: use crypto/rand.Read\n - Fix README.md example error (#146) ([ipfs/go-cid#146](https://github.com/ipfs/go-cid/pull/146))\n- github.com/ipfs/go-delegated-routing (v0.7.0 -> v0.8.0):\n - chore: release v0.8.0\n - chore: migrate from go-ipns to boxo\n - docs: add deprecation notice [ci skip]\n- github.com/ipfs/go-graphsync (v0.14.1 -> v0.14.4):\n - Update version to cover latest fixes (#419) ([ipfs/go-graphsync#419](https://github.com/ipfs/go-graphsync/pull/419))\n - Bring changes from #412\n - Bring changes from #391\n - fix: calling message queue Shutdown twice causes panic (because close is called twice on done channel) (#414) ([ipfs/go-graphsync#414](https://github.com/ipfs/go-graphsync/pull/414))\n - docs(CHANGELOG): update for v0.14.3\n - fix: wire up proper linksystem to traverser (#411) ([ipfs/go-graphsync#411](https://github.com/ipfs/go-graphsync/pull/411))\n - sync: update CI config files (#378) ([ipfs/go-graphsync#378](https://github.com/ipfs/go-graphsync/pull/378))\n - chore: remove social links (#398) ([ipfs/go-graphsync#398](https://github.com/ipfs/go-graphsync/pull/398))\n - Removes `main` branch callout.\n - release v0.14.2\n- github.com/ipfs/go-ipfs-blockstore (v1.2.0 -> v1.3.0):\n - chore: release v1.3.0\n - feat: stub and deprecate NewBlockstoreNoPrefix\n - Accept options for blockstore: start with WriteThrough and NoPrefix\n - Allow using a NewWriteThrough() blockstore.\n - sync: update CI config files (#105) ([ipfs/go-ipfs-blockstore#105](https://github.com/ipfs/go-ipfs-blockstore/pull/105))\n - feat: fast-path for PutMany, falling back to Put for single block call (#97) ([ipfs/go-ipfs-blockstore#97](https://github.com/ipfs/go-ipfs-blockstore/pull/97))\n- github.com/ipfs/go-ipfs-cmds (v0.8.2 -> v0.9.0):\n - chore: release v0.9.0\n - chore: change go-libipfs to boxo\n- github.com/ipfs/go-libipfs (v0.6.2 -> v0.7.0):\n - chore: bump to 0.7.0 (#213) ([ipfs/go-libipfs#213](https://github.com/ipfs/go-libipfs/pull/213))\n - feat: return 400 on /ipfs/invalid-cid (#205) ([ipfs/go-libipfs#205](https://github.com/ipfs/go-libipfs/pull/205))\n - docs: add note in README that go-libipfs is not comprehensive (#163) ([ipfs/go-libipfs#163](https://github.com/ipfs/go-libipfs/pull/163))\n- github.com/ipfs/go-merkledag (v0.9.0 -> v0.10.0):\n - chore: bump version to 0.10.0\n - fix: switch to crypto/rand.Read\n - stop using the deprecated io/ioutil package\n- github.com/ipfs/go-unixfs (v0.4.4 -> v0.4.5):\n - chore: release v0.4.5\n - chore: remove go-libipfs dependency\n- github.com/ipfs/go-unixfsnode (v1.5.2 -> v1.6.0):\n - chore: bump v1.6.0\n - feat: add UnixFSPathSelectorBuilder ([ipfs/go-unixfsnode#45](https://github.com/ipfs/go-unixfsnode/pull/45))\n - fix: update state to allow iter continuance on NotFound errors\n - chore!: make PBLinkItr private - not intended for public use\n - fix: propagate iteration errors\n- github.com/ipld/go-car/v2 (v2.5.1 -> v2.9.1-0.20230325062757-fff0e4397a3d):\n - chore: unmigrate from go-libipfs\n - Create CODEOWNERS\n - blockstore: give a direct access to the index for read operations\n - blockstore: only close the file on error in OpenReadWrite, not OpenReadWriteFile\n - fix: handle (and test) WholeCID vs not; fast Has() path for storage\n - ReadWrite: faster Has() by using the in-memory index instead of reading on disk\n - fix: let `extract` skip missing unixfs shard links\n - fix: error when no files extracted\n - fix: make -f optional, read from stdin if omitted\n - fix: update cmd/car/README with latest description\n - chore: add test cases for extract modes\n - feat: extract accepts '-' as an output path for stdout\n - feat: extract specific path, accept stdin as streaming input\n - fix: if we don't read the full block data, don't error on !EOF\n - blockstore: try to close during Finalize(), even in case of previous error\n - ReadWrite: add an alternative FinalizeReadOnly+Close flow\n - feat: add WithTrustedCar() reader option (#381) ([ipld/go-car#381](https://github.com/ipld/go-car/pull/381))\n - blockstore: fast path for AllKeysChan using the index\n - fix: switch to crypto/rand.Read\n - stop using the deprecated io/ioutil package\n - fix(doc): fix storage package doc formatting\n - fix: return errors for unsupported operations\n - chore: move insertionindex into store pkg\n - chore: add experimental note\n - fix: minor lint & windows fd test problems\n - feat: docs for StorageCar interfaces\n - feat: ReadableWritable; dedupe shared code\n - feat: add Writable functionality to StorageCar\n - feat: StorageCar as a Readable storage, separate from blockstore\n - feat(blockstore): implement a streaming read only storage\n - feat(cmd): add index create subcommand to create an external carv2 index ([ipld/go-car#350](https://github.com/ipld/go-car/pull/350))\n - chore: bump version to 0.6.0\n - fix: use goreleaser instead\n - Allow using WalkOption in WriteCar function ([ipld/go-car#357](https://github.com/ipld/go-car/pull/357))\n - fix: update go-block-format to the version that includes the stubs\n - feat: upgrade from go-block-format to go-libipfs/blocks\n - cleanup readme a bit to make the cli more discoverable (#353) ([ipld/go-car#353](https://github.com/ipld/go-car/pull/353))\n - Update install instructions in README.md\n - Add a debugging form for car files. (#341) ([ipld/go-car#341](https://github.com/ipld/go-car/pull/341))\n - ([ipld/go-car#340](https://github.com/ipld/go-car/pull/340))\n- github.com/ipld/go-codec-dagpb (v1.5.0 -> v1.6.0):\n - Update version.json\n- github.com/ipld/go-ipld-prime (v0.19.0 -> v0.20.0):\n - Prepare v0.20.0\n - fix(datamodel): add tests to Copy, make it complain on nil\n - feat(dagcbor): mode to allow parsing undelimited streamed objects\n - Fix mispatched package declaration.\n - Add several pieces of docs to schema/dmt.\n - Additional access to schema/dmt package; schema concatenation feature ([ipld/go-ipld-prime#483](https://github.com/ipld/go-ipld-prime/pull/483))\n - Fix hash mismatch error on matching link pointer\n - feat: support errors.Is for schema errors\n- github.com/ipld/go-ipld-prime/storage/bsadapter (v0.0.0-20211210234204-ce2a1c70cd73 -> v0.0.0-20230102063945-1a409dc236dd):\n - build(deps): bump github.com/ipfs/go-blockservice\n - Fix mispatched package declaration.\n - Add several pieces of docs to schema/dmt.\n - Additional access to schema/dmt package; schema concatenation feature ([ipld/go-ipld-prime/storage/bsadapter#483](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/483))\n - fix: go mod tidy\n - build(deps): bump github.com/frankban/quicktest from 1.14.3 to 1.14.4\n - Fix hash mismatch error on matching link pointer\n - build(deps): bump github.com/warpfork/go-testmark from 0.10.0 to 0.11.0\n - feat: support errors.Is for schema errors\n - build(deps): bump github.com/multiformats/go-multicodec\n - Prepare v0.19.0\n - fix: correct json codec links & bytes handling\n - build(deps): bump github.com/google/go-cmp from 0.5.8 to 0.5.9 (#468) ([ipld/go-ipld-prime/storage/bsadapter#468](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/468))\n - build(deps): bump github.com/ipfs/go-cid from 0.3.0 to 0.3.2 (#466) ([ipld/go-ipld-prime/storage/bsadapter#466](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/466))\n - build(deps): bump github.com/ipfs/go-cid in /storage/bsrvadapter (#464) ([ipld/go-ipld-prime/storage/bsadapter#464](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/464))\n - test(basicnode): increase test coverage for int and map types (#454) ([ipld/go-ipld-prime/storage/bsadapter#454](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/454))\n - build(deps): bump github.com/ipfs/go-cid in /storage/bsrvadapter\n - build(deps): bump github.com/ipfs/go-cid from 0.2.0 to 0.3.0\n - build(deps): bump github.com/multiformats/go-multicodec\n - fix: remove reliance on ioutil\n - fix: update sub-package modules\n - build(deps): bump github.com/multiformats/go-multihash\n - build(deps): bump github.com/ipfs/go-datastore in /storage/dsadapter\n - update .github/workflows/go-check.yml\n - update .github/workflows/go-test.yml\n - run gofmt -s\n - bump go.mod to Go 1.18 and run go fix\n - bump go.mod to Go 1.18 and run go fix\n - bump go.mod to Go 1.18 and run go fix\n - bump go.mod to Go 1.18 and run go fix\n - feat: add kinded union to gendemo\n - fix: go mod 1.17 compat problems\n - build(deps): bump github.com/ipfs/go-blockservice\n - Prepare v0.18.0\n - fix(deps): update benchmarks go.sum\n - build(deps): bump github.com/multiformats/go-multihash\n - feat(bindnode): add a BindnodeRegistry utility (#437) ([ipld/go-ipld-prime/storage/bsadapter#437](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/437))\n - feat(bindnode): support full uint64 range\n - chore(bindnode): remove typed functions for options\n - chore(bindnode): docs and minor tweaks\n - feat(bindnode): make Any converters work for List and Map values\n - fix(bindnode): shorten converter option names, minor perf improvements\n - fix(bindnode): only custom convert AssignNull for Any converter\n - feat(bindnode): pass Null on to nullable custom converters\n - chore(bindnode): config helper refactor w/ short-circuit\n - feat(bindnode): add AddCustomTypeAnyConverter() to handle `Any` fields\n - feat(bindnode): add AddCustomTypeXConverter() options for most scalar kinds\n - chore(bindnode): back out of reflection for converters\n - feat(bindnode): switch to converter functions instead of type\n - feat(bindnode): allow custom type conversions with options\n - feat: add release checklist (#442) ([ipld/go-ipld-prime/storage/bsadapter#442](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/442))\n - Prepare v0.17.0\n - feat: introduce UIntNode interface, used within DAG-CBOR codec\n - add option to not parse beyond end of structure (#435) ([ipld/go-ipld-prime/storage/bsadapter#435](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/435))\n - sync benchmarks go.sum\n - build(deps): bump github.com/multiformats/go-multicodec\n - patch: first draft. ([ipld/go-ipld-prime/storage/bsadapter#350](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/350))\n - feat(bindnode): infer links and Any from Go types (#432) ([ipld/go-ipld-prime/storage/bsadapter#432](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/432))\n - fix(codecs): error on cid.Undef links in dag{json,cbor} encoding (#433) ([ipld/go-ipld-prime/storage/bsadapter#433](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/433))\n - chore(bindnode): add test for sub-node unwrapping\n - fix(bindnode): more helpful error message for enum value footgun\n - fix(bindnode): panic early if API has been passed ptr-to-ptr\n - fix(deps): mod tidy for dependencies\n - build(deps): bump github.com/warpfork/go-testmark from 0.3.0 to 0.10.0\n - build(deps): bump github.com/multiformats/go-multicodec\n - build(deps): bump github.com/ipfs/go-cid from 0.0.4 to 0.2.0\n - build(deps): bump github.com/google/go-cmp from 0.5.7 to 0.5.8\n - build(deps): bump github.com/frankban/quicktest from 1.14.2 to 1.14.3\n - build(deps): bump github.com/ipfs/go-cid in /storage/bsrvadapter\n - chore(deps): expand dependabot to sub-modules\n - chore(deps): add dependabot config\n - printer: fix printing of floats\n - add version.json file (#411) ([ipld/go-ipld-prime/storage/bsadapter#411](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/411))\n - ci: use GOFLAGS to control test tags\n - ci: disable coverpkg using custom workflow insertion\n - ci: add initial web3 unified-ci files\n - fix: make 32-bit safe and stable & add to CI\n - ci: add go-check.yml workflow from unified-ci\n - ci: go mod tidy\n - fix: staticcheck and govet fixes\n - test: make tests work on Windows, add Windows to CI (#405) ([ipld/go-ipld-prime/storage/bsadapter#405](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/405))\n - schema: enable inline types through dsl parser & compiler (#404) ([ipld/go-ipld-prime/storage/bsadapter#404](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/404))\n - node/bindnode: allow nilable types for IPLD optional/nullable\n - test(ci): enable macos in GitHub Actions\n - test(gen-go): disable parallelism when testing on macos\n - storage: update deps\n - dsl support for stringjoin struct repr and stringprefix union repr ([ipld/go-ipld-prime/storage/bsadapter#397](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/397))\n - codec/dagcbor: add DecodeOptions.ExperimentalDeterminism\n - node/bindnode: add some more docs\n - start testing on Go 1.18.x, drop Go 1.16.x\n - readme: getting started pointers.\n - readme: bindnode definitely needs a mention!\n - Readme updates!\n - datamodel: document that repr prototypes produce type nodes\n - node/bindnode: minor fuzz improvements\n - gengo: update readme.\n - fix(dagcbor): don't accept trailing bytes\n - schema/dmt: reject duplicate or missing union repr members\n - node/bindnode: actually check schemadmt.Compile errors when fuzzing\n - node/bindnode: avoid OOM when inferring from cyclic IPLD schemas\n - schema/dmt: require enum reprs to refer valid members\n - skip NaN/Inf errors for dag-json\n - node/bindnode: refuse to decode empty union values\n - schema/dmt: error in Compile if union reprs refer to unknown members\n - node/bindnode: start fuzzing with schema/dmt and codec/dagcbor\n - mark v0.16.0\n - node/bindnode: enforce pointer requirement for nullable maps\n - Implement WalkTransforming traversal (#376) ([ipld/go-ipld-prime/storage/bsadapter#376](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/376))\n - docs(datamodel): add comment to LargeBytesNode\n - Add partial-match traversal of large bytes (#375) ([ipld/go-ipld-prime/storage/bsadapter#375](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/375))\n - Implement option to start traversals at a path ([ipld/go-ipld-prime/storage/bsadapter#358](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/358))\n - add top-level \"go value with schema\" example\n - Support optional `LargeBytesNode` interface (#372) ([ipld/go-ipld-prime/storage/bsadapter#372](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/372))\n - node/bindnode: support pointers to datamodel.Node to bind with Any\n - fix(bindnode): tuple struct iterator should handle absent fields properly\n - node/bindnode: make AssignNode work at the repr level\n - node/bindnode: add support for unsigned integers\n - node/bindnode: cover even more edge case panics\n - node/bindnode: polish some more AsT panics\n - schema/dmt: stop using a fake test to generate code ([ipld/go-ipld-prime/storage/bsadapter#356](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/356))\n - schema: remove one review note; add another.\n - fix: minor EncodedLength fixes, add tests to fully exercise\n - feat: add dagcbor.EncodedLength(Node) to calculate length without encoding\n - chore: rename Garbage() to Generate()\n - fix: minor garbage nits\n - fix: Garbage() takes rand parameter, tweak algorithms, improve docs\n - feat: add Garbage() Node generator\n - node/bindnode: introduce an assembler that always errors\n - node/bindnode: polish panics on invalid AssignT calls\n - datamodel: don't panic when stringifying an empty KindSet\n - node/bindnode: start using ipld.LoadSchema APIs\n - selectors: fix for edge case around recursion clauses with an immediate edge. ([ipld/go-ipld-prime/storage/bsadapter#334](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/334))\n - node/bindnode: improve support for pointer types\n - node/bindnode: subtract all absents in Length at the repr level\n - fix(codecs): error when encoding maps whose lengths don't match entry count\n - schema: avoid alloc and copy in Struct and Enum methods\n - node/bindnode: allow mapping int-repr enums with Go integers\n - schema,node/bindnode: add support for Any\n - signaling ADLs in selectors (#301) ([ipld/go-ipld-prime/storage/bsadapter#301](https://github.com/ipld/go-ipld-prime/storage/bsadapter/pull/301))\n - node/bindnode: add support for enums\n - schema/...: add support for enum int representations\n - node/bindnode: allow binding cidlink.Link to links\n- github.com/libp2p/go-libp2p (v0.26.4 -> v0.27.3):\n - release v0.27.3\n - quic virtual listener: don't panic when quic-go's accept call errors (#2276) ([libp2p/go-libp2p#2276](https://github.com/libp2p/go-libp2p/pull/2276))\n - Release v0.27.2 (#2270) ([libp2p/go-libp2p#2270](https://github.com/libp2p/go-libp2p/pull/2270))\n - release v0.27.1 (#2252) ([libp2p/go-libp2p#2252](https://github.com/libp2p/go-libp2p/pull/2252))\n - Infer public webtransport addrs from quic-v1 addrs. (#2251) ([libp2p/go-libp2p#2251](https://github.com/libp2p/go-libp2p/pull/2251))\n - basichost: don't allocate when deduplicating multiaddrs (#2206) ([libp2p/go-libp2p#2206](https://github.com/libp2p/go-libp2p/pull/2206))\n - identify: fix normalization of interface listen addresses (#2250) ([libp2p/go-libp2p#2250](https://github.com/libp2p/go-libp2p/pull/2250))\n - autonat: fix flaky TestAutoNATDialRefused (#2245) ([libp2p/go-libp2p#2245](https://github.com/libp2p/go-libp2p/pull/2245))\n - basichost: remove stray print statement in test (#2249) ([libp2p/go-libp2p#2249](https://github.com/libp2p/go-libp2p/pull/2249))\n - swarm: fix multiaddr comparison in ListenClose (#2247) ([libp2p/go-libp2p#2247](https://github.com/libp2p/go-libp2p/pull/2247))\n - release v0.27.0 (#2242) ([libp2p/go-libp2p#2242](https://github.com/libp2p/go-libp2p/pull/2242))\n - add a security policy (#2238) ([libp2p/go-libp2p#2238](https://github.com/libp2p/go-libp2p/pull/2238))\n - chore: 0.27.0 changelog entries (#2241) ([libp2p/go-libp2p#2241](https://github.com/libp2p/go-libp2p/pull/2241))\n - correctly handle WebTransport addresses without certhashes (#2239) ([libp2p/go-libp2p#2239](https://github.com/libp2p/go-libp2p/pull/2239))\n - autorelay: add metrics (#2185) ([libp2p/go-libp2p#2185](https://github.com/libp2p/go-libp2p/pull/2185))\n - autonat: don't change status on dial request refused (#2225) ([libp2p/go-libp2p#2225](https://github.com/libp2p/go-libp2p/pull/2225))\n - autonat: fix closing of listeners in dialPolicy tests (#2226) ([libp2p/go-libp2p#2226](https://github.com/libp2p/go-libp2p/pull/2226))\n - discovery (backoff): fix typo in comment (#2214) ([libp2p/go-libp2p#2214](https://github.com/libp2p/go-libp2p/pull/2214))\n - relaysvc: flaky TestReachabilityChangeEvent (#2215) ([libp2p/go-libp2p#2215](https://github.com/libp2p/go-libp2p/pull/2215))\n - Add wss transport to interop tester impl (#2178) ([libp2p/go-libp2p#2178](https://github.com/libp2p/go-libp2p/pull/2178))\n - tests: add a stream read deadline transport test (#2210) ([libp2p/go-libp2p#2210](https://github.com/libp2p/go-libp2p/pull/2210))\n - autorelay: fix busy loop bug and flaky tests in relay finder (#2208) ([libp2p/go-libp2p#2208](https://github.com/libp2p/go-libp2p/pull/2208))\n - tests: test mplex and Yamux, Noise and TLS in transport tests (#2209) ([libp2p/go-libp2p#2209](https://github.com/libp2p/go-libp2p/pull/2209))\n - tests: add some basic transport integration tests (#2207) ([libp2p/go-libp2p#2207](https://github.com/libp2p/go-libp2p/pull/2207))\n - autorelay: remove unused semaphore (#2184) ([libp2p/go-libp2p#2184](https://github.com/libp2p/go-libp2p/pull/2184))\n - basichost: prevent duplicate dials (#2196) ([libp2p/go-libp2p#2196](https://github.com/libp2p/go-libp2p/pull/2196))\n - websocket: don't set a WSS multiaddr for accepted unencrypted conns (#2199) ([libp2p/go-libp2p#2199](https://github.com/libp2p/go-libp2p/pull/2199))\n - websocket: Don't limit message sizes in the websocket reader (#2193) ([libp2p/go-libp2p#2193](https://github.com/libp2p/go-libp2p/pull/2193))\n - identify: fix stale comment (#2179) ([libp2p/go-libp2p#2179](https://github.com/libp2p/go-libp2p/pull/2179))\n - relay service: add metrics (#2154) ([libp2p/go-libp2p#2154](https://github.com/libp2p/go-libp2p/pull/2154))\n - identify: Fix IdentifyWait when Connected events happen out of order (#2173) ([libp2p/go-libp2p#2173](https://github.com/libp2p/go-libp2p/pull/2173))\n - chore: fix resource manager's README (#2168) ([libp2p/go-libp2p#2168](https://github.com/libp2p/go-libp2p/pull/2168))\n - relay: fix deadlock when closing (#2171) ([libp2p/go-libp2p#2171](https://github.com/libp2p/go-libp2p/pull/2171))\n - core: remove LocalPrivateKey method from network.Conn interface (#2144) ([libp2p/go-libp2p#2144](https://github.com/libp2p/go-libp2p/pull/2144))\n - routed host: return connection error instead of routing error (#2169) ([libp2p/go-libp2p#2169](https://github.com/libp2p/go-libp2p/pull/2169))\n - connmgr: reduce log level for closing connections (#2165) ([libp2p/go-libp2p#2165](https://github.com/libp2p/go-libp2p/pull/2165))\n - circuitv2: cleanup relay service properly (#2164) ([libp2p/go-libp2p#2164](https://github.com/libp2p/go-libp2p/pull/2164))\n - chore: add patch release to changelog (#2151) ([libp2p/go-libp2p#2151](https://github.com/libp2p/go-libp2p/pull/2151))\n - chore: remove superfluous testing section from README (#2150) ([libp2p/go-libp2p#2150](https://github.com/libp2p/go-libp2p/pull/2150))\n - autonat: don't use autonat for address discovery (#2148) ([libp2p/go-libp2p#2148](https://github.com/libp2p/go-libp2p/pull/2148))\n - swarm metrics: fix connection direction (#2147) ([libp2p/go-libp2p#2147](https://github.com/libp2p/go-libp2p/pull/2147))\n - connmgr: Use eventually equal helper in connmgr tests (#2128) ([libp2p/go-libp2p#2128](https://github.com/libp2p/go-libp2p/pull/2128))\n - swarm: emit PeerConnectedness event from swarm instead of from hosts (#1574) ([libp2p/go-libp2p#1574](https://github.com/libp2p/go-libp2p/pull/1574))\n - relay: initialize the ASN util when starting the service (#2143) ([libp2p/go-libp2p#2143](https://github.com/libp2p/go-libp2p/pull/2143))\n - Fix flaky TestMetricsNoAllocNoCover test (#2142) ([libp2p/go-libp2p#2142](https://github.com/libp2p/go-libp2p/pull/2142))\n - identify: Bump timeouts/sleep in tests (#2135) ([libp2p/go-libp2p#2135](https://github.com/libp2p/go-libp2p/pull/2135))\n - Add sleep to fix flaky test (#2129) ([libp2p/go-libp2p#2129](https://github.com/libp2p/go-libp2p/pull/2129))\n - basic_host: Fix flaky tests (#2136) ([libp2p/go-libp2p#2136](https://github.com/libp2p/go-libp2p/pull/2136))\n - swarm: Check context once more before dialing (#2139) ([libp2p/go-libp2p#2139](https://github.com/libp2p/go-libp2p/pull/2139))\n- github.com/libp2p/go-libp2p-asn-util (v0.2.0 -> v0.3.0):\n - release v0.3.0 (#26) ([libp2p/go-libp2p-asn-util#26](https://github.com/libp2p/go-libp2p-asn-util/pull/26))\n - initialize the store lazily (#25) ([libp2p/go-libp2p-asn-util#25](https://github.com/libp2p/go-libp2p-asn-util/pull/25))\n- github.com/libp2p/go-libp2p-gostream (v0.5.0 -> v0.6.0):\n - Update libp2p ([libp2p/go-libp2p-gostream#80](https://github.com/libp2p/go-libp2p-gostream/pull/80))\n - fix typo in README (#75) ([libp2p/go-libp2p-gostream#75](https://github.com/libp2p/go-libp2p-gostream/pull/75))\n- github.com/libp2p/go-libp2p-http (v0.4.0 -> v0.5.0):\n - sync: update CI config files ([libp2p/go-libp2p-http#82](https://github.com/libp2p/go-libp2p-http/pull/82))\n- github.com/libp2p/go-libp2p-kad-dht (v0.21.1 -> v0.23.0):\n - Release v0.23.0\n - Specified CODEOWNERS ([libp2p/go-libp2p-kad-dht#828](https://github.com/libp2p/go-libp2p-kad-dht/pull/828))\n - fix: optimistic provide ci checks in tests ([libp2p/go-libp2p-kad-dht#833](https://github.com/libp2p/go-libp2p-kad-dht/pull/833))\n - feat: add experimental optimistic provide (#783) ([libp2p/go-libp2p-kad-dht#783](https://github.com/libp2p/go-libp2p-kad-dht/pull/783))\n - feat: rework tracing a bit\n - feat: add basic tracing\n - chore: release v0.22.0\n - chore: migrate go-libipfs to boxo\n - Fix multiple ProviderAddrTTL definitions #795 ([libp2p/go-libp2p-kad-dht#831](https://github.com/libp2p/go-libp2p-kad-dht/pull/831))\n - Increase provider Multiaddress TTL ([libp2p/go-libp2p-kad-dht#795](https://github.com/libp2p/go-libp2p-kad-dht/pull/795))\n - Make provider manager options configurable in `fullrt` ([libp2p/go-libp2p-kad-dht#829](https://github.com/libp2p/go-libp2p-kad-dht/pull/829))\n - Adjust PeerSet logic in the DHT lookup process ([libp2p/go-libp2p-kad-dht#802](https://github.com/libp2p/go-libp2p-kad-dht/pull/802))\n - added maintainers in the README ([libp2p/go-libp2p-kad-dht#826](https://github.com/libp2p/go-libp2p-kad-dht/pull/826))\n - Allow DHT crawler to be swappable\n - Introduce options to parameterize config of the accelerated DHT client ([libp2p/go-libp2p-kad-dht#822](https://github.com/libp2p/go-libp2p-kad-dht/pull/822))\n- github.com/libp2p/go-libp2p-pubsub (v0.9.0 -> v0.9.3):\n - Fix Memory Leak In New Timecache Implementations (#528) ([libp2p/go-libp2p-pubsub#528](https://github.com/libp2p/go-libp2p-pubsub/pull/528))\n - Default validator support (#525) ([libp2p/go-libp2p-pubsub#525](https://github.com/libp2p/go-libp2p-pubsub/pull/525))\n - Refactor timecache implementations (#523) ([libp2p/go-libp2p-pubsub#523](https://github.com/libp2p/go-libp2p-pubsub/pull/523))\n - fix(timecache): remove panic in first seen cache on Add (#522) ([libp2p/go-libp2p-pubsub#522](https://github.com/libp2p/go-libp2p-pubsub/pull/522))\n - chore: update go version and dependencies (#516) ([libp2p/go-libp2p-pubsub#516](https://github.com/libp2p/go-libp2p-pubsub/pull/516))\n- github.com/multiformats/go-multiaddr (v0.8.0 -> v0.9.0):\n - Release v0.9.0 ([multiformats/go-multiaddr#196](https://github.com/multiformats/go-multiaddr/pull/196))\n - Update webrtc protocols after rename ([multiformats/go-multiaddr#195](https://github.com/multiformats/go-multiaddr/pull/195))\n- github.com/multiformats/go-multibase (v0.1.1 -> v0.2.0):\n - chore: bump v0.2.0\n - fix: math/rand -> crypto/rand\n - fuzz: add Decoder fuzzing\n- github.com/multiformats/go-multicodec (v0.7.0 -> v0.8.1):\n - Bump version to release `ipns-record` code\n - chore: update submodules and go generate\n - deps: upgrade stringer to compatible version\n - v0.8.0\n - chore: update submodules and go generate\n- github.com/warpfork/go-testmark (v0.10.0 -> v0.11.0):\n - Quick changelog to note we have an API update.\n - Index fix ([warpfork/go-testmark#13](https://github.com/warpfork/go-testmark/pull/13))\n - Link to python implementation in the readme!\n\n\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Rod Vagg | 40 | +4214/-1400 | 102 |\n| Sukun | 12 | +3541/-267 | 34 |\n| Gus Eggert | 22 | +2387/-1160 | 81 |\n| galargh | 23 | +1331/-1734 | 34 |\n| Henrique Dias | 23 | +681/-1167 | 79 |\n| Marco Munizaga | 19 | +1500/-187 | 55 |\n| Jorropo | 25 | +897/-597 | 180 |\n| Dennis Trautwein | 4 | +990/-60 | 14 |\n| Marten Seemann | 18 | +443/-450 | 53 |\n| vyzo | 2 | +595/-152 | 11 |\n| Michael Muré | 8 | +427/-182 | 18 |\n| Will | 2 | +536/-15 | 5 |\n| Adin Schmahmann | 3 | +327/-125 | 11 |\n| hannahhoward | 2 | +344/-1 | 4 |\n| Arthur Gavazza | 1 | +210/-50 | 4 |\n| Hector Sanjuan | 6 | +181/-77 | 13 |\n| Masih H. Derkani | 5 | +214/-42 | 12 |\n| Calvin Behling | 4 | +158/-58 | 11 |\n| Eric Myhre | 7 | +113/-27 | 15 |\n| Marcin Rataj | 5 | +72/-30 | 5 |\n| Steve Loeppky | 2 | +99/-0 | 2 |\n| Piotr Galar | 9 | +60/-18 | 9 |\n| gammazero | 4 | +69/-0 | 8 |\n| Prithvi Shahi | 2 | +55/-14 | 2 |\n| Eng Zer Jun | 1 | +15/-54 | 5 |\n| Laurent Senta | 3 | +44/-2 | 3 |\n| Ian Davis | 1 | +35/-0 | 1 |\n| web3-bot | 4 | +19/-13 | 7 |\n| guillaumemichel | 2 | +18/-14 | 3 |\n| Guillaume Michel - guissou | 4 | +24/-8 | 4 |\n| omahs | 1 | +9/-9 | 3 |\n| cortze | 3 | +9/-9 | 3 |\n| Nishant Das | 1 | +9/-5 | 3 |\n| Hlib Kanunnikov | 2 | +11/-3 | 3 |\n| Andrew Gillis | 3 | +6/-8 | 3 |\n| Johnny | 1 | +0/-10 | 1 |\n| Rafał Leszko | 1 | +4/-4 | 1 |\n| Dirk McCormick | 1 | +4/-1 | 1 |\n| Antonio Navarro Perez | 1 | +4/-1 | 1 |\n| RichΛrd | 2 | +2/-2 | 2 |\n| Russell Dempsey | 1 | +2/-1 | 1 |\n| Winterhuman | 1 | +1/-1 | 1 |\n| Will Hawkins | 1 | +1/-1 | 1 |\n| Nikhilesh Susarla | 1 | +1/-1 | 1 |\n| Kubo Mage | 1 | +1/-1 | 1 |\n| Bryan White | 1 | +1/-1 | 1 |\n\n\n" }, { "path": "docs/changelogs/v0.21.md", "content": "# Kubo changelog v0.21\n\n- [v0.21.1](#v0211)\n- [v0.21.0](#v0210)\n\n## v0.21.1\n\n- Update go-libp2p:\n - [v0.27.8](https://github.com/libp2p/go-libp2p/releases/tag/v0.27.8)\n - [v0.27.9](https://github.com/libp2p/go-libp2p/releases/tag/v0.27.9)\n- Update Boxo to v0.10.3 ([ipfs/boxo#412](https://github.com/ipfs/boxo/pull/412)).\n\n## v0.21.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Saving previously seen nodes for later bootstrapping](#saving-previously-seen-nodes-for-later-bootstrapping)\n - [Gateway: `DeserializedResponses` config flag](#gateway-deserializedresponses-config-flag)\n - [`client/rpc` migration of `go-ipfs-http-client`](#clientrpc-migration-of-go-ipfs-http-client)\n - [Gateway: DAG-CBOR/-JSON previews and improved error pages](#gateway-dag-cbor-json-previews-and-improved-error-pages)\n - [Gateway: subdomain redirects are now `text/html`](#gateway-subdomain-redirects-are-now-texthtml)\n - [Gateway: support for partial CAR export parameters (IPIP-402)](#gateway-support-for-partial-car-export-parameters-ipip-402)\n - [`ipfs dag stat` deduping statistics](#ipfs-dag-stat-deduping-statistics)\n - [Accelerated DHT Client is no longer experimental](#accelerated-dht-client-is-no-longer-experimental)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Saving previously seen nodes for later bootstrapping\n\nKubo now stores a subset of connected peers as backup bootstrap nodes ([kubo#8856](https://github.com/ipfs/kubo/pull/8856)).\nThese nodes are used in addition to the explicitly defined bootstrappers in the\n[`Bootstrap`](https://github.com/ipfs/kubo/blob/master/docs/config.md#bootstrap) configuration.\n\nThis enhancement improves the resiliency of the system, as it eliminates the\nnecessity of relying solely on the default bootstrappers operated by Protocol\nLabs for joining the public IPFS swarm. Previously, this level of robustness\nwas only available in LAN contexts with [mDNS peer discovery](https://github.com/ipfs/kubo/blob/master/docs/config.md#discoverymdns)\nenabled.\n\nWith this update, the same level of robustness is applied to peers that lack\nmDNS peers and solely rely on the public DHT.\n\n#### Gateway: `DeserializedResponses` config flag\n\nThis release introduces the\n[`Gateway.DeserializedResponses`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaydeserializedresponses)\nconfiguration flag.\n\nWith this flag, one can explicitly configure whether the gateway responds to\ndeserialized requests or not. By default, this flag is enabled.\n\nDisabling deserialized responses allows the\ngateway to operate\nas a [Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/)\nlimited to three [verifiable](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval)\nresponse types:\n[application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw),\n[application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car),\nand [application/vnd.ipfs.ipns-record](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record).\n\nWith deserialized responses disabled, the Kubo gateway can serve as a block\nbackend for other software (like\n[bifrost-gateway](https://github.com/ipfs/bifrost-gateway#readme),\n[IPFS in Chromium](https://github.com/little-bear-labs/ipfs-chromium/blob/main/README.md)\netc) without the usual risks associated with hosting deserialized data behind\nthird-party CIDs.\n\n#### `client/rpc` migration of `go-ipfs-http-client`\n\nThe [`go-ipfs-http-client`](https://github.com/ipfs/go-ipfs-http-client) RPC has\nbeen migrated into [`kubo/client/rpc`](../../client/rpc).\n\nWith this change the two will be kept in sync, in some previous releases we\nupdated the CoreAPI with new Kubo features but forgot to port those to the\nhttp-client, making it impossible to use them together with the same coreapi\nversion.\n\nFor smooth transition `v0.7.0` of `go-ipfs-http-client` provides updated stubs\nfor Kubo `v0.21`.\n\n#### Gateway: DAG-CBOR/-JSON previews and improved error pages\n\nIn this release, we improved the HTML templates of our HTTP gateway:\n\n1. You can now preview the contents of a DAG-CBOR and DAG-JSON document from your browser, as well as follow any IPLD Links ([CBOR Tag 42](https://github.com/ipld/cid-cbor/)) contained within them.\n2. The HTML directory listings now contain [updated, higher-definition icons](https://user-images.githubusercontent.com/5447088/241224419-5385793a-d3bb-40aa-8cb0-0382b5bc56a0.png).\n3. On gateway error, instead of a plain text error message, web browsers will now get a friendly HTML response with more details regarding the problem.\n\nHTML responses are returned when request's `Accept` header includes `text/html`.\n\n| DAG-CBOR Preview | Error Page |\n| ---- | ---- |\n| ![DAG-CBOR Preview](https://github.com/ipfs/boxo/assets/5447088/973f05d1-5731-4469-9da5-d1d776891899) | ![Error Page](https://github.com/ipfs/boxo/assets/5447088/14c453df-adbc-4634-b038-133121914550) |\n\n#### Gateway: subdomain redirects are now `text/html`\n\nHTTP 301 redirects [from path to subdomain](https://specs.ipfs.tech/http-gateways/subdomain-gateway/#migrating-from-path-to-subdomain-gateway)\nno longer include the target data in the body.\nThe data is returned only once, with the final HTTP 200 returned from the\ntarget subdomain.\n\nThe HTTP 301 body now includes human-readable `text/html` message\nfor clients that do not follow redirects by default:\n\n```console\n$ curl \"https://subdomain-gw.example.net/ipfs/${cid}/\"\nMoved Permanently.\n```\n\nRationale can be found in [kubo#9913](https://github.com/ipfs/kubo/pull/9913).\n\n#### Gateway: support for partial CAR export parameters (IPIP-402)\n\nThe gateway now supports optional CAR export parameters\n`dag-scope=block|entity|all` and `entity-bytes=from:to` as specified in\n[IPIP-402](https://specs.ipfs.tech/ipips/ipip-0402/).\n\nBatch block retrieval minimizes round trips, catering to the requirements of\nlight HTTP clients for directory enumeration, range requests, and content path\nresolution.\n\n#### `ipfs dag stat` deduping statistics\n\n`ipfs dat stat` now accept multiple CIDs and will dump advanced statistics\non the number of shared blocks and size of each CID.\n\n```console\n$ ipfs dag stat --progress=false QmfXuRxzyVy5H2LssLgtXrKCrNvDY8UBvMp2aoW8LS8AYA QmfZDyu2UFfUhL4VdHaw7Hofivmn5D4DdQj38Lwo86RsnB\n\nCID \tBlocks \tSize\nQmfXuRxzyVy5H2LssLgtXrKCrNvDY8UBvMp2aoW8LS8AYA\t3 \t2151\nQmfZDyu2UFfUhL4VdHaw7Hofivmn5D4DdQj38Lwo86RsnB\t4 \t3223\n\nSummary\nTotal Size: 3326\nUnique Blocks: 5\nShared Size: 2048\nRatio: 1.615755\n```\n\n`ipfs --enc=json dag stat`'s keys are a non breaking change, new keys have been added but old keys with previous semantics are still here.\n\n#### Accelerated DHT Client is no longer experimental\n\nThe [accelerated DHT client](docs/config.md#routingaccelerateddhtclient) is now\nthe main recommended solution for users who are hosting lots of data.\nBy trading some upfront DHT caching and increased memory usage,\none gets provider throughput improvements up to 6 millions times bigger dataset.\nSee [the docs](docs/config.md#routingaccelerateddhtclient) for more info.\n\nThe `Experimental.AcceleratedDHTClient` flag moved to [`Routing.AcceleratedDHTClient`](/docs/config.md#routingaccelerateddhtclient).\nA config migration has been added to handle this automatically.\n\nA new tracker estimates the providing speed and warns users if they\nshould be using AcceleratedDHTClient because they are falling behind.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix: correctly handle migration of configs\n - fix(gateway): include CORS on subdomain redirects (#9994) ([ipfs/kubo#9994](https://github.com/ipfs/kubo/pull/9994))\n - fix: docker repository initialization race condition\n - chore: update version\n - ([ipfs/kubo#9981](https://github.com/ipfs/kubo/pull/9981))\n - ([ipfs/kubo#9960](https://github.com/ipfs/kubo/pull/9960))\n - ([ipfs/kubo#9936](https://github.com/ipfs/kubo/pull/9936))\n- github.com/ipfs/boxo (v0.8.1 -> v0.10.2-0.20230629143123-2d3edc552442):\n - chore: version 0.10.2\n - fix(gateway): include CORS on subdomain redirects (#395) ([ipfs/boxo#395](https://github.com/ipfs/boxo/pull/395))\n - fix(gateway): ensure 'X-Ipfs-Root' header is valid (#337) ([ipfs/boxo#337](https://github.com/ipfs/boxo/pull/337))\n - docs: prepare changelog for next release [ci skip]\n - chore: version 0.10.1 (#359) ([ipfs/boxo#359](https://github.com/ipfs/boxo/pull/359))\n - fix(gateway): allow CAR trustless requests with path\n - blockstore: replace go.uber.org/atomic with sync/atomic\n - fix(gateway): remove handleUnsupportedHeaders after go-ipfs 0.13 (#350) ([ipfs/boxo#350](https://github.com/ipfs/boxo/pull/350))\n - docs: update RELEASE.md based on 0.9 release (#343) ([ipfs/boxo#343](https://github.com/ipfs/boxo/pull/343))\n - chore: v0.10.0 (#345) ([ipfs/boxo#345](https://github.com/ipfs/boxo/pull/345))\n - docs(changelog): car params from ipip-402\n - docs(changelog): add gateway deserialized responses (#341) ([ipfs/boxo#341](https://github.com/ipfs/boxo/pull/341))\n - feat(gateway): implement IPIP-402 extensions for gateway CAR requests (#303) ([ipfs/boxo#303](https://github.com/ipfs/boxo/pull/303))\n - chore: release v0.9.0\n - changelog: update for 0.8.1 and 0.9.0\n - provider: second round of reprovider refactor\n - feat(unixfs): change protobuf package name to unixfs.v1.pb to prevent collisions with go-unixfs. Also regenerate protobufs with latest gogo\n - feat(ipld/merkledag): remove use of go-ipld-format global registry\n - feat(ipld/merkledag): updated to use its own global go-ipld-legacy registry instead of a shared global registry\n - chore: do not rely on deprecated logger\n - changelog: add changelog for async pin listing (#336) ([ipfs/boxo#336](https://github.com/ipfs/boxo/pull/336))\n - pinner: change the interface to have async pin listing\n - provider: revert throughput callback and related refactor\n - fix(gateway): question marks in url.Path when redirecting (#313) ([ipfs/boxo#313](https://github.com/ipfs/boxo/pull/313))\n - fix(gateway)!: no duplicate payload during subdomain redirects (#326) ([ipfs/boxo#326](https://github.com/ipfs/boxo/pull/326))\n - provider: add breaking changes to the changelog (#330) ([ipfs/boxo#330](https://github.com/ipfs/boxo/pull/330))\n - relocated magic numbers, updated Reprovide Interval from 24h to 22h\n - provider: refactor to only maintain one batched implementation and add throughput callback\n - feat(gateway): HTML preview for dag-cbor and dag-json (#315) ([ipfs/boxo#315](https://github.com/ipfs/boxo/pull/315))\n - coreiface: add a testing.T argument to the provider\n - feat(gateway): improved templates, user friendly errors (#298) ([ipfs/boxo#298](https://github.com/ipfs/boxo/pull/298))\n - feat(gateway)!: deserialised responses turned off by default (#252) ([ipfs/boxo#252](https://github.com/ipfs/boxo/pull/252))\n - fix(gw): missing return in error case ([ipfs/boxo#319](https://github.com/ipfs/boxo/pull/319))\n - feat(routing/http): pass records limit on routing.FindProviders (#299) ([ipfs/boxo#299](https://github.com/ipfs/boxo/pull/299))\n - bitswap/client: fix PeerResponseTrackerProbabilityOneKnownOneUnknownPeer\n - feat(gw): add ipfs_http_gw_car_stream_fail_duration_seconds (#312) ([ipfs/boxo#312](https://github.com/ipfs/boxo/pull/312))\n - feat(gw): add ipfs_http_gw_request_types metric (#311) ([ipfs/boxo#311](https://github.com/ipfs/boxo/pull/311))\n - refactor: simplify ipns validation in example\n - feat: add deprecator\n - fix(routing/v1): add newline in NDJSON responses (#300) ([ipfs/boxo#300](https://github.com/ipfs/boxo/pull/300))\n - feat(gateway): redirect ipns b58mh to cid (#236) ([ipfs/boxo#236](https://github.com/ipfs/boxo/pull/236))\n - refactor: replace assert.Nil for assert.NoError\n - tar: add test cases for validatePlatformPath\n - feat(ipns): helper ValidateWithPeerID and UnmarshalIpnsEntry (#294) ([ipfs/boxo#294](https://github.com/ipfs/boxo/pull/294))\n - Revert \"feat: reusable ipns verify (#292)\"\n - feat: reusable ipns verify (#292) ([ipfs/boxo#292](https://github.com/ipfs/boxo/pull/292))\n - refactor: remove badger, leveldb dependencies (#286) ([ipfs/boxo#286](https://github.com/ipfs/boxo/pull/286))\n - feat(routing/http): add streaming support (#18) ([ipfs/boxo#18](https://github.com/ipfs/boxo/pull/18))\n - feat(routing): allow-offline with routing put (#278) ([ipfs/boxo#278](https://github.com/ipfs/boxo/pull/278))\n - refactor(gateway): switch to xxhash/v2 (#285) ([ipfs/boxo#285](https://github.com/ipfs/boxo/pull/285))\n- github.com/ipfs/go-ipfs-util (v0.0.2 -> v0.0.3):\n - docs: remove contribution section\n - chore: bump version\n - chore: deprecate types and readme\n - sync: update CI config files (#12) ([ipfs/go-ipfs-util#12](https://github.com/ipfs/go-ipfs-util/pull/12))\n - fix staticcheck ([ipfs/go-ipfs-util#9](https://github.com/ipfs/go-ipfs-util/pull/9))\n- github.com/ipfs/go-ipld-format (v0.4.0 -> v0.5.0):\n - chore: release version v0.5.0\n - feat: remove block decoding global registry\n - sync: update CI config files (#75) ([ipfs/go-ipld-format#75](https://github.com/ipfs/go-ipld-format/pull/75))\n - sync: update CI config files (#74) ([ipfs/go-ipld-format#74](https://github.com/ipfs/go-ipld-format/pull/74))\n- github.com/ipfs/go-ipld-legacy (v0.1.1 -> v0.2.1):\n - v0.2.1 ([ipfs/go-ipld-legacy#15](https://github.com/ipfs/go-ipld-legacy/pull/15))\n - Expose a constructor for making a decoder with an existing link system ([ipfs/go-ipld-legacy#14](https://github.com/ipfs/go-ipld-legacy/pull/14))\n - Update to v0.2.0 ([ipfs/go-ipld-legacy#13](https://github.com/ipfs/go-ipld-legacy/pull/13))\n - Remove global variable ([ipfs/go-ipld-legacy#12](https://github.com/ipfs/go-ipld-legacy/pull/12))\n - sync: update CI config files (#8) ([ipfs/go-ipld-legacy#8](https://github.com/ipfs/go-ipld-legacy/pull/8))\n- github.com/ipfs/go-unixfsnode (v1.6.0 -> v1.7.1):\n - chore: bump to v1.7.1\n - test: remove unnecessary t.Log\n - test: check if reader reads only necessary blocks\n - fix: do not read extra block if offset = at+childSize\n - doc: added simple doc for testutil package\n - bump v1.7.0\n - feat(testutil): add test data generation utils (extracted from Lassie)\n- github.com/libp2p/go-libp2p (v0.27.3 -> v0.27.7):\n - Release v0.27.7 (#2374) ([libp2p/go-libp2p#2374](https://github.com/libp2p/go-libp2p/pull/2374))\n - Release v0.27.6 (#2359) ([libp2p/go-libp2p#2359](https://github.com/libp2p/go-libp2p/pull/2359))\n - Release v0.27.5 (#2324) ([libp2p/go-libp2p#2324](https://github.com/libp2p/go-libp2p/pull/2324))\n - Bump version to v0.27.4\n - identify: reject signed peer records on peer ID mismatch\n - swarm: change maps with multiaddress keys to use strings (#2284) ([libp2p/go-libp2p#2284](https://github.com/libp2p/go-libp2p/pull/2284))\n - identify: avoid spuriously triggering pushes (#2299) ([libp2p/go-libp2p#2299](https://github.com/libp2p/go-libp2p/pull/2299))\n- github.com/libp2p/go-libp2p-kad-dht (v0.23.0 -> v0.24.2):\n - chore: release v0.24.2\n - chore: release v0.24.1\n - fix: decrease tests noise, update kbucket and fix fixRTIUfNeeded\n - refactor: remove goprocess\n - fix: leaking go routines\n - chore: release v0.24.0\n - fix: don't add unresponsive DHT servers to the Routing Table (#820) ([libp2p/go-libp2p-kad-dht#820](https://github.com/libp2p/go-libp2p-kad-dht/pull/820))\n- github.com/libp2p/go-libp2p-kbucket (v0.5.0 -> v0.6.3):\n - fix: fix abba bug in UsefulNewPeer ([libp2p/go-libp2p-kbucket#122](https://github.com/libp2p/go-libp2p-kbucket/pull/122))\n - chore: release v0.6.2 ([libp2p/go-libp2p-kbucket#121](https://github.com/libp2p/go-libp2p-kbucket/pull/121))\n - Replacing UsefulPeer() with UsefulNewPeer() ([libp2p/go-libp2p-kbucket#120](https://github.com/libp2p/go-libp2p-kbucket/pull/120))\n - chore: release 0.6.1 ([libp2p/go-libp2p-kbucket#119](https://github.com/libp2p/go-libp2p-kbucket/pull/119))\n - UsefulPeer function ([libp2p/go-libp2p-kbucket#113](https://github.com/libp2p/go-libp2p-kbucket/pull/113))\n - Fixed peer replacement with bucket size of 1. ([libp2p/go-libp2p-kbucket#117](https://github.com/libp2p/go-libp2p-kbucket/pull/117))\n - GenRandomKey function ([libp2p/go-libp2p-kbucket#116](https://github.com/libp2p/go-libp2p-kbucket/pull/116))\n - Removed maintainers from readme ([libp2p/go-libp2p-kbucket#115](https://github.com/libp2p/go-libp2p-kbucket/pull/115))\n - Add maintainers ([libp2p/go-libp2p-kbucket#114](https://github.com/libp2p/go-libp2p-kbucket/pull/114))\n - sync: update CI config files (#112) ([libp2p/go-libp2p-kbucket#112](https://github.com/libp2p/go-libp2p-kbucket/pull/112))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.6.2 -> v0.7.0):\n - chore: release v0.7.0\n - fix: iterate over keys manually in ProvideMany\n- github.com/libp2p/go-reuseport (v0.2.0 -> v0.3.0):\n - release v0.3.0 (#103) ([libp2p/go-reuseport#103](https://github.com/libp2p/go-reuseport/pull/103))\n - fix error handling when setting socket options (#102) ([libp2p/go-reuseport#102](https://github.com/libp2p/go-reuseport/pull/102))\n - minor README updates (#96) ([libp2p/go-reuseport#96](https://github.com/libp2p/go-reuseport/pull/96))\n - sync: update CI config files (#94) ([libp2p/go-reuseport#94](https://github.com/libp2p/go-reuseport/pull/94))\n - feat: add a DialTimeout function ([libp2p/go-reuseport#92](https://github.com/libp2p/go-reuseport/pull/92))\n- github.com/multiformats/go-multicodec (v0.8.1 -> v0.9.0):\n - Bump v0.9.0\n - Bump v0.8.2\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n- github.com/multiformats/go-multihash (v0.2.1 -> v0.2.3):\n - chore: release v0.2.3\n - perf: outline logic in Decode to allow for stack allocations\n - chore: release v0.2.2\n - sha256: drop minio in favor of crypto/sha256 for go1.21 and above\n - sync: update CI config files (#169) ([multiformats/go-multihash#169](https://github.com/multiformats/go-multihash/pull/169))\n - add handler for hasher.Write returned error ([multiformats/go-multihash#167](https://github.com/multiformats/go-multihash/pull/167))\n - sync: update CI config files (#165) ([multiformats/go-multihash#165](https://github.com/multiformats/go-multihash/pull/165))\n - test: add benchmark for all hash functions Sum\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Jorropo | 47 | +4394/-4458 | 202 |\n| Henrique Dias | 48 | +4344/-3962 | 205 |\n| Łukasz Magiera | 68 | +3604/-886 | 172 |\n| Adin Schmahmann | 8 | +1754/-1057 | 37 |\n| galargh | 7 | +1355/-1302 | 15 |\n| Gus Eggert | 7 | +1566/-655 | 33 |\n| rvagg | 1 | +396/-389 | 3 |\n| Michael Muré | 3 | +547/-202 | 14 |\n| Guillaume Michel - guissou | 5 | +153/-494 | 17 |\n| guillaumemichel | 15 | +446/-189 | 28 |\n| Laurent Senta | 4 | +472/-152 | 29 |\n| Rod Vagg | 6 | +554/-37 | 23 |\n| Marcin Rataj | 11 | +330/-82 | 21 |\n| Arthur Gavazza | 1 | +296/-87 | 7 |\n| Lucas Molas | 1 | +323/-56 | 6 |\n| Marco Munizaga | 5 | +227/-97 | 17 |\n| Alex | 8 | +163/-116 | 10 |\n| Steven Allen | 11 | +154/-114 | 14 |\n| Marten Seemann | 6 | +214/-41 | 12 |\n| web3-bot | 9 | +76/-75 | 28 |\n| Hector Sanjuan | 2 | +5/-96 | 4 |\n| Sukun | 1 | +83/-17 | 3 |\n| Steve Loeppky | 2 | +100/-0 | 2 |\n| Edgar Lee | 1 | +46/-46 | 12 |\n| Ivan Schasny | 1 | +67/-5 | 4 |\n| imthe1 | 1 | +65/-3 | 5 |\n| godcong | 2 | +30/-31 | 5 |\n| Will Scott | 4 | +36/-23 | 6 |\n| Petar Maymounkov | 1 | +45/-9 | 1 |\n| Ross Jones | 1 | +43/-1 | 2 |\n| William Entriken | 1 | +38/-0 | 1 |\n| João Pedro | 1 | +35/-0 | 1 |\n| jhertz | 1 | +21/-0 | 2 |\n| Nikhilesh Susarla | 1 | +21/-0 | 3 |\n| Matt Joiner | 1 | +11/-9 | 2 |\n| Vlad | 2 | +4/-2 | 2 |\n| Russell Dempsey | 2 | +4/-2 | 2 |\n| Will | 2 | +2/-2 | 2 |\n| Piotr Galar | 1 | +1/-1 | 1 |\n| Joel Gustafson | 1 | +1/-1 | 1 |\n| Dennis Trautwein | 1 | +1/-1 | 1 |\n| Bryan Stenson | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.22.md", "content": "# Kubo changelog v0.22\n\n- [v0.22.0](#v0220)\n\n## v0.22.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Gateway: support for `order=` and `dups=` parameters (IPIP-412)](#gateway-support-for-order-and-dups-parameters-ipip-412)\n - [`ipfs name publish` now supports V2 only IPNS records](#ipfs-name-publish-now-supports-v2-only-ipns-records)\n - [IPNS name resolution has been fixed](#ipns-name-resolution-has-been-fixed)\n - [go-libp2p v0.29.0 update with smart dialing](#go-libp2p-v0290-update-with-smart-dialing)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Gateway: support for `order=` and `dups=` parameters (IPIP-412)\n\nThe updated [`boxo/gateway` library](https://github.com/ipfs/boxo/tree/main/gateway)\nintroduces support for ordered CAR responses through the inclusion of optional\nCAR content type parameters: `order=dfs` and `dups=y|n` from\n[IPIP-412](https://github.com/ipfs/specs/pull/412).\n\nPreviously, Kubo already provided CARs in DFS order without duplicate blocks.\nWith the implementation of IPIP-412, this behavior is now explicitly defined\nrather than implied.\n\nIn the absence of `dups` or `order` in `Accept` request reader, the default CAR\nresponse will have the `Content-Type: application/vnd.ipld.car; version=1; order=dfs; dups=n`\nand the same blocks as Kubo 0.21.\n\nKubo 0.22 still only supports DFS block ordering (`order=dfs`). However, it is\nnow possible to request a DFS CAR stream with duplicate blocks by opting in via\n`Accept: application/vnd.ipld.car; order=dfs; dups=y`. This opt-in feature can be\nbeneficial for memory-constrained clients and IoT devices, as it allows for\nstreaming large DAGs without the need to store all previously encountered\nblocks in memory.\n\n#### `ipfs name publish` now supports V2 only IPNS records\n\nWhen publishing an IPNS record, you are now able to create v2 only records\nby passing `--v1compat=false`. By default, we still create V1+V2 records, such\nthat there is the highest chance of backwards compatibility. The goal is to move\nto V2 only in the future.\n\nFor more details, see [IPIP-428](https://specs.ipfs.tech/ipips/ipip-0428/)\nand the updated [IPNS Record Verification](https://specs.ipfs.tech/ipns/ipns-record/#record-verification) logic.\n\n#### IPNS name resolution has been fixed\n\nIPNS name resolution had a regression where if IPNS over PubSub was enabled, but the name was not also available via IPNS over PubSub it would take 1 minute to for the lookup to complete (if the record was not yet cached).\n\nThis has been fixed and as before will give the best record from either the DHT subsystem or IPNS over PubSub, whichever comes back first.\n\nFor details see [#9927](https://github.com/ipfs/kubo/issues/9927) and [#10020](https://github.com/ipfs/kubo/pull/10020).\n\n# go-libp2p v0.29.0 update with smart dialing\n\nWe updated from [go-libp2p](https://github.com/libp2p/go-libp2p) [v0.27.7](https://github.com/libp2p/go-libp2p/releases/tag/v0.27.7) to [v0.29.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.29.0). This release includes smart dialing, which is a prioritization algorithm that will try to rank addresses and protocols rather than attempting all options in parallel. Anecdotally, we have observed [Kubo nodes make 30% less dials](https://github.com/libp2p/go-libp2p/issues/2326#issuecomment-1644332863) with no to low latency impact.\n\nThis includes a breaking change to `ipfs id` and some of the `ipfs swarm` commands. We no longer report `ProtocolVersion`. This used to be hardcoded as `ipfs/0.1.0` and sent to other peers but was not providing any distinguishing value. See [libp2p/go-libp2p#2294](https://github.com/libp2p/go-libp2p/issues/2294) for more information.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: change version to v0.22.0\n - chore(misc/README.md): trim duplicated content\n - Merge branch 'release-v0.21' back into master\n - docs(readme): unofficial packages badge\n - chore: remove sharness tests ported to conformance testing (#9999) ([ipfs/kubo#9999](https://github.com/ipfs/kubo/pull/9999))\n - ci: switch from testing against js-ipfs to helia (#10042) ([ipfs/kubo#10042](https://github.com/ipfs/kubo/pull/10042))\n - chore: merge release back into master\n - chore: change orbitdb to haydenyoung EARLY_TESTERS\n - Fix usage numbers\n - chore: update early testers list (#9218) ([ipfs/kubo#9218](https://github.com/ipfs/kubo/pull/9218))\n - docs: changelog v0.21 fixes (#10037) ([ipfs/kubo#10037](https://github.com/ipfs/kubo/pull/10037))\n - refactor(ci): simplify Dockerfile and add docker image testing (#10021) ([ipfs/kubo#10021](https://github.com/ipfs/kubo/pull/10021))\n - chore: update version\n - fix(relay): apply user provider options\n - libp2p: stop reporting ProtocolVersion\n - chore: update go-libp2p to v0.29.0\n - chore: update go-libp2p to v0.28.1\n - fix: mark all routers DoNotWaitForSearchValue (#10020) ([ipfs/kubo#10020](https://github.com/ipfs/kubo/pull/10020))\n - feat(gateway): support for ipip-412 parameters\n - docs(commands): explain that swarm connect can reuse existing connections or known addresses (#10015) ([ipfs/kubo#10015](https://github.com/ipfs/kubo/pull/10015))\n - docs: add Brave to RELEASE_ISSUE_TEMPLATE.md (#10012) ([ipfs/kubo#10012](https://github.com/ipfs/kubo/pull/10012))\n - feat: webui@4.0.2\n - ([ipfs/kubo#10008](https://github.com/ipfs/kubo/pull/10008))\n - docs: skip check before prepare branch in RELEASE_ISSUE_TEMPLATE.md\n - docs: update RELEASE_ISSUE_TEMPLATE.md with a warning about npm publish\n - docs: update refs to kuboreleaser in RELEASE_ISSUE_TEMPLATE.md\n - docs: Gateway.HTTPHeaders\n - refactor: replace boxo/ipld/car by ipld/go-car\n - chore: bump to boxo master\n - fix: correctly handle migration of configs\n - fix(gateway): include CORS on subdomain redirects (#9994) ([ipfs/kubo#9994](https://github.com/ipfs/kubo/pull/9994))\n - fix: docker repository initialization race condition\n - feat(ipns): records with V2-only signatures (#9932) ([ipfs/kubo#9932](https://github.com/ipfs/kubo/pull/9932))\n - cmds/dag/import: pin roots by default (#9966) ([ipfs/kubo#9966](https://github.com/ipfs/kubo/pull/9966))\n - docs: fix 0.21 changelog\n - feat!: dag import - don't pin roots by default (#9926) ([ipfs/kubo#9926](https://github.com/ipfs/kubo/pull/9926))\n - fix(cmd): useful errors in dag import (#9945) ([ipfs/kubo#9945](https://github.com/ipfs/kubo/pull/9945))\n - feat: webui@4.0.1 (#9940) ([ipfs/kubo#9940](https://github.com/ipfs/kubo/pull/9940))\n - chore(docs): typo http→https\n - fix: more stable prometheus test (#9944) ([ipfs/kubo#9944](https://github.com/ipfs/kubo/pull/9944))\n - ([ipfs/kubo#9937](https://github.com/ipfs/kubo/pull/9937))\n- github.com/ipfs/boxo (v0.10.3 -> v0.11.0):\n - Release v0.11.0 ([ipfs/boxo#417](https://github.com/ipfs/boxo/pull/417))\n- github.com/ipfs/go-bitswap (null -> v0.11.0):\n - chore: release v0.11.0\n - chore: release v0.10.2\n - fix: create a copy of the protocol slice in network.processSettings\n - chore: release v0.10.1\n - fix: incorrect type in the WithTracer polyfill option\n - chore: fix incorrect log message when a bad option is passed\n - chore: release v0.10.0\n - chore: update go-libp2p v0.22.0\n - chore: release v0.9.0\n - feat: split client and server ([ipfs/go-bitswap#570](https://github.com/ipfs/go-bitswap/pull/570))\n - chore: remove goprocess from blockstoremanager\n - Don't add blocks to the datastore ([ipfs/go-bitswap#571](https://github.com/ipfs/go-bitswap/pull/571))\n - Remove dependency on travis package from go-libp2p-testing ([ipfs/go-bitswap#569](https://github.com/ipfs/go-bitswap/pull/569))\n - feat: add basic tracing (#562) ([ipfs/go-bitswap#562](https://github.com/ipfs/go-bitswap/pull/562))\n - chore: release v0.7.0 (#566) ([ipfs/go-bitswap#566](https://github.com/ipfs/go-bitswap/pull/566))\n - feat: coalesce and queue connection event handling (#565) ([ipfs/go-bitswap#565](https://github.com/ipfs/go-bitswap/pull/565))\n- github.com/ipfs/go-merkledag (v0.10.0 -> v0.11.0):\n - chore: update v0.11.0 (#106) ([ipfs/go-merkledag#106](https://github.com/ipfs/go-merkledag/pull/106))\n - update merkeldag to use the explicit decoder registry (#104) ([ipfs/go-merkledag#104](https://github.com/ipfs/go-merkledag/pull/104))\n - Update status in README.md and added CODEOWNERS (#101) ([ipfs/go-merkledag#101](https://github.com/ipfs/go-merkledag/pull/101))\n- github.com/ipld/go-car/v2 (v2.9.1-0.20230325062757-fff0e4397a3d -> v2.10.2-0.20230622090957-499d0c909d33):\n - feat: add inverse and version to filter cmd ([ipld/go-car#457](https://github.com/ipld/go-car/pull/457))\n - v0.6.1 bump\n - chore: update usage of merkledag by go-car (#437) ([ipld/go-car#437](https://github.com/ipld/go-car/pull/437))\n - feat(cmd/car): add '--no-wrap' option to 'create' command ([ipld/go-car#432](https://github.com/ipld/go-car/pull/432))\n - fix: remove github.com/ipfs/go-ipfs-blockstore dependency\n - feat: expose index for StorageCar\n - perf: reduce NewCarReader allocations\n - fix(deps): update deps for cmd (use master go-car and go-car/v2 for now)\n - fix: new error strings from go-cid\n - fix: tests should match stderr for verbose output\n - fix: reading from stdin should broadcast EOF to block loaders\n - refactor insertion index to be publicly accessible ([ipld/go-car#408](https://github.com/ipld/go-car/pull/408))\n- github.com/libp2p/go-libp2p (v0.27.9 -> v0.29.2):\n - release v0.29.2\n - release v0.29.1\n - swarm: don't open new streams over transient connections (#2450) ([libp2p/go-libp2p#2450](https://github.com/libp2p/go-libp2p/pull/2450))\n - core/crypto: restrict RSA keys to <= 8192 bits (#2454) ([libp2p/go-libp2p#2454](https://github.com/libp2p/go-libp2p/pull/2454))\n - Release version v0.29.0 (#2431) ([libp2p/go-libp2p#2431](https://github.com/libp2p/go-libp2p/pull/2431))\n - webtransport: reject listening on a multiaddr with a certhash (#2426) ([libp2p/go-libp2p#2426](https://github.com/libp2p/go-libp2p/pull/2426))\n - swarm: deprecate libp2p.DialRanker option (#2430) ([libp2p/go-libp2p#2430](https://github.com/libp2p/go-libp2p/pull/2430))\n - quic: Update to quic-go v0.36.2 (#2424) ([libp2p/go-libp2p#2424](https://github.com/libp2p/go-libp2p/pull/2424))\n - autonat: fix typo in WithSchedule option comment (#2425) ([libp2p/go-libp2p#2425](https://github.com/libp2p/go-libp2p/pull/2425))\n - identify: filter nat64 well-known prefix ipv6 addresses (#2392) ([libp2p/go-libp2p#2392](https://github.com/libp2p/go-libp2p/pull/2392))\n - update go-multiaddr to v0.10.1, use Unique function from there (#2407) ([libp2p/go-libp2p#2407](https://github.com/libp2p/go-libp2p/pull/2407))\n - swarm: enable smart dialing by default (#2420) ([libp2p/go-libp2p#2420](https://github.com/libp2p/go-libp2p/pull/2420))\n - transport integration tests: make TestMoreStreamsThanOurLimits less flaky (#2410) ([libp2p/go-libp2p#2410](https://github.com/libp2p/go-libp2p/pull/2410))\n - holepunch: skip racy TestDirectDialWorks (#2419) ([libp2p/go-libp2p#2419](https://github.com/libp2p/go-libp2p/pull/2419))\n - swarm: change relay dial delay to 500ms (#2421) ([libp2p/go-libp2p#2421](https://github.com/libp2p/go-libp2p/pull/2421))\n - identify: disable racy TestLargeIdentifyMessage with race detector (#2401) ([libp2p/go-libp2p#2401](https://github.com/libp2p/go-libp2p/pull/2401))\n - swarm: make black hole detection configurable (#2403) ([libp2p/go-libp2p#2403](https://github.com/libp2p/go-libp2p/pull/2403))\n - net/mock: support ConnectionGater in MockNet (#2297) ([libp2p/go-libp2p#2297](https://github.com/libp2p/go-libp2p/pull/2297))\n - docs: Add a Github workflow for checking dead links (#2406) ([libp2p/go-libp2p#2406](https://github.com/libp2p/go-libp2p/pull/2406))\n - rcmgr: enable metrics by default (#2389) (#2409) ([libp2p/go-libp2p#2409](https://github.com/libp2p/go-libp2p/pull/2409))\n - chore: remove outdated info in README and link to libp2p-implementers slack (#2405) ([libp2p/go-libp2p#2405](https://github.com/libp2p/go-libp2p/pull/2405))\n - metrics: deduplicate code in examples (#2404) ([libp2p/go-libp2p#2404](https://github.com/libp2p/go-libp2p/pull/2404))\n - transport tests: remove mplex tests (#2402) ([libp2p/go-libp2p#2402](https://github.com/libp2p/go-libp2p/pull/2402))\n - swarm: implement Happy Eyeballs ranking (#2365) ([libp2p/go-libp2p#2365](https://github.com/libp2p/go-libp2p/pull/2365))\n - docs: fix some comments (#2391) ([libp2p/go-libp2p#2391](https://github.com/libp2p/go-libp2p/pull/2391))\n - metrics: provide separate docker-compose files for OSX and Linux (#2397) ([libp2p/go-libp2p#2397](https://github.com/libp2p/go-libp2p/pull/2397))\n - identify: use zero-alloc slice sorting function (#2396) ([libp2p/go-libp2p#2396](https://github.com/libp2p/go-libp2p/pull/2396))\n - rcmgr: move StatsTraceReporter to rcmgr package (#2388) ([libp2p/go-libp2p#2388](https://github.com/libp2p/go-libp2p/pull/2388))\n - swarm: implement blackhole detection (#2320) ([libp2p/go-libp2p#2320](https://github.com/libp2p/go-libp2p/pull/2320))\n - basichost / blankhost: wrap errors (#2331) ([libp2p/go-libp2p#2331](https://github.com/libp2p/go-libp2p/pull/2331))\n - network: don't allocate in DedupAddrs (#2395) ([libp2p/go-libp2p#2395](https://github.com/libp2p/go-libp2p/pull/2395))\n - rcmgr: test snapshot defaults and that we keep consistent defaults (#2315) ([libp2p/go-libp2p#2315](https://github.com/libp2p/go-libp2p/pull/2315))\n - rcmgr: register prometheus metrics with the libp2p registerer (#2370) ([libp2p/go-libp2p#2370](https://github.com/libp2p/go-libp2p/pull/2370))\n - metrics: make it possible to spin up Grafana using docker-compose (#2383) ([libp2p/go-libp2p#2383](https://github.com/libp2p/go-libp2p/pull/2383))\n - identify: set stream deadlines for Identify and Identify Push streams (#2382) ([libp2p/go-libp2p#2382](https://github.com/libp2p/go-libp2p/pull/2382))\n - fix: in the swarm move Connectedness emit after releasing conns (#2373) ([libp2p/go-libp2p#2373](https://github.com/libp2p/go-libp2p/pull/2373))\n - metrics: add example for metrics and dashboard (#2232) ([libp2p/go-libp2p#2232](https://github.com/libp2p/go-libp2p/pull/2232))\n - dashboards: finish metrics effort (#2362) ([libp2p/go-libp2p#2362](https://github.com/libp2p/go-libp2p/pull/2362))\n - transport tests: many streams and lots of data (#2296) ([libp2p/go-libp2p#2296](https://github.com/libp2p/go-libp2p/pull/2296))\n - webtransport: close the challenge stream after the Noise handshake (#2305) ([libp2p/go-libp2p#2305](https://github.com/libp2p/go-libp2p/pull/2305))\n - test: document why InstantTimer is required (#2351) ([libp2p/go-libp2p#2351](https://github.com/libp2p/go-libp2p/pull/2351))\n - rcmgr: fix link to dashboards in README (#2363) ([libp2p/go-libp2p#2363](https://github.com/libp2p/go-libp2p/pull/2363))\n - docs: fix some comments errors (#2356) ([libp2p/go-libp2p#2356](https://github.com/libp2p/go-libp2p/pull/2356))\n - release v0.28.0 (#2344) ([libp2p/go-libp2p#2344](https://github.com/libp2p/go-libp2p/pull/2344))\n - nat: add HasDiscoveredNAT method for checking NAT environments (#2358) ([libp2p/go-libp2p#2358](https://github.com/libp2p/go-libp2p/pull/2358))\n - swarm: fix stale DialBackoff comment (#2353) ([libp2p/go-libp2p#2353](https://github.com/libp2p/go-libp2p/pull/2353))\n - swarm: use RLock for DialBackoff reads (#2354) ([libp2p/go-libp2p#2354](https://github.com/libp2p/go-libp2p/pull/2354))\n - Clear stream scope if we error (#2345) ([libp2p/go-libp2p#2345](https://github.com/libp2p/go-libp2p/pull/2345))\n - changelog: improve description of smart dialing (#2342) ([libp2p/go-libp2p#2342](https://github.com/libp2p/go-libp2p/pull/2342))\n - swarm: make smart-dialing opt in (#2340) ([libp2p/go-libp2p#2340](https://github.com/libp2p/go-libp2p/pull/2340))\n - swarm: cleanup address filtering logic (#2333) ([libp2p/go-libp2p#2333](https://github.com/libp2p/go-libp2p/pull/2333))\n - chore: add 0.28.0 changelog (#2335) ([libp2p/go-libp2p#2335](https://github.com/libp2p/go-libp2p/pull/2335))\n - swarm: improve documentation for the DefaultDialRanker (#2336) ([libp2p/go-libp2p#2336](https://github.com/libp2p/go-libp2p/pull/2336))\n - holepunch: add metrics (#2246) ([libp2p/go-libp2p#2246](https://github.com/libp2p/go-libp2p/pull/2246))\n - swarm: implement smart dialing logic (#2260) ([libp2p/go-libp2p#2260](https://github.com/libp2p/go-libp2p/pull/2260))\n - revert \"feat:add contexts to all peerstore methods (#2312)\" (#2328) ([libp2p/go-libp2p#2328](https://github.com/libp2p/go-libp2p/pull/2328))\n - identify: don't save signed peer records (#2325) ([libp2p/go-libp2p#2325](https://github.com/libp2p/go-libp2p/pull/2325))\n - feat:add contexts to all peerstore methods (#2312) ([libp2p/go-libp2p#2312](https://github.com/libp2p/go-libp2p/pull/2312))\n - swarm: Dedup addresses to dial (#2322) ([libp2p/go-libp2p#2322](https://github.com/libp2p/go-libp2p/pull/2322))\n - identify: filter received addresses based on the node's remote address (#2300) ([libp2p/go-libp2p#2300](https://github.com/libp2p/go-libp2p/pull/2300))\n - update go-nat to v0.2.0, use context on AddMapping and RemoveMapping (#2319) ([libp2p/go-libp2p#2319](https://github.com/libp2p/go-libp2p/pull/2319))\n - transport integration tests: add tests for resource manager (#2285) ([libp2p/go-libp2p#2285](https://github.com/libp2p/go-libp2p/pull/2285))\n - identify: reject signed peer records on peer ID mismatch\n - identify: don't send default protocol version (#2303) ([libp2p/go-libp2p#2303](https://github.com/libp2p/go-libp2p/pull/2303))\n - metrics: add instance filter to all dashboards (#2301) ([libp2p/go-libp2p#2301](https://github.com/libp2p/go-libp2p/pull/2301))\n - identify: avoid spuriously triggering pushes (#2299) ([libp2p/go-libp2p#2299](https://github.com/libp2p/go-libp2p/pull/2299))\n - net/mock: mimic Swarm's event and notification behavior in MockNet (#2287) ([libp2p/go-libp2p#2287](https://github.com/libp2p/go-libp2p/pull/2287))\n - examples: fix flaky multipro TestMain (#2289) ([libp2p/go-libp2p#2289](https://github.com/libp2p/go-libp2p/pull/2289))\n - swarm: change maps with multiaddress keys to use strings (#2284) ([libp2p/go-libp2p#2284](https://github.com/libp2p/go-libp2p/pull/2284))\n - tests: add comprehensive end-to-end tests for connection gating (#2200) ([libp2p/go-libp2p#2200](https://github.com/libp2p/go-libp2p/pull/2200))\n - swarm: log unexpected listener errors (#2277) ([libp2p/go-libp2p#2277](https://github.com/libp2p/go-libp2p/pull/2277))\n - websocket: switch back to the gorilla library (#2280) ([libp2p/go-libp2p#2280](https://github.com/libp2p/go-libp2p/pull/2280))\n - quic: prioritise listen connections for reuse (#2262) ([libp2p/go-libp2p#2262](https://github.com/libp2p/go-libp2p/pull/2262))\n - quic virtual listener: don't panic when quic-go's accept call errors (#2276) ([libp2p/go-libp2p#2276](https://github.com/libp2p/go-libp2p/pull/2276))\n - tests: add docks for debugging flaky tests (#2216) ([libp2p/go-libp2p#2216](https://github.com/libp2p/go-libp2p/pull/2216))\n - webtransport: only add cert hashes if we already started listening (#2271) ([libp2p/go-libp2p#2271](https://github.com/libp2p/go-libp2p/pull/2271))\n - Revert \"webtransport: initialize the certmanager when creating the transport (#2268)\" (#2273) ([libp2p/go-libp2p#2273](https://github.com/libp2p/go-libp2p/pull/2273))\n - webtransport: initialize the certmanager when creating the transport (#2268) ([libp2p/go-libp2p#2268](https://github.com/libp2p/go-libp2p/pull/2268))\n - move NAT mapping logic out of the host, add tests for NAT handling ([libp2p/go-libp2p#2248](https://github.com/libp2p/go-libp2p/pull/2248))\n - githooks: add a githook to check that the test-plans go.mod is tidied (#2256) ([libp2p/go-libp2p#2256](https://github.com/libp2p/go-libp2p/pull/2256))\n - quic: fix race condition when generating random holepunch packet (#2263) ([libp2p/go-libp2p#2263](https://github.com/libp2p/go-libp2p/pull/2263))\n - swarm: remove unused variable in addrDial (#2257) ([libp2p/go-libp2p#2257](https://github.com/libp2p/go-libp2p/pull/2257))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.7.0 -> v0.7.1):\n - chore: release v0.7.1\n - fix: for comparallel never return nil channel for FindProvidersAsync\n - chore: rename DoNotWaitForStreamingResponses to DoNotWaitForSearchValue\n - feat: add DoNotWaitForStreamingResponses to ParallelRouter\n - chore: cleanup error handling in compparallel\n - fix: correctly handle errors in compparallel\n - fix: make the ProvideMany docs clearer\n - perf: remove goroutine that just waits before closing with a synchronous waitgroup\n- github.com/libp2p/go-nat (v0.1.0 -> v0.2.0):\n - release v0.2.0 (#30) ([libp2p/go-nat#30](https://github.com/libp2p/go-nat/pull/30))\n - update deps, use contexts on UPnP functions (#29) ([libp2p/go-nat#29](https://github.com/libp2p/go-nat/pull/29))\n - sync: update CI config files (#28) ([libp2p/go-nat#28](https://github.com/libp2p/go-nat/pull/28))\n - sync: update CI config files (#24) ([libp2p/go-nat#24](https://github.com/libp2p/go-nat/pull/24))\n- github.com/libp2p/go-yamux/v4 (v4.0.0 -> v4.0.1):\n - Release v4.0.1 ([libp2p/go-yamux#106](https://github.com/libp2p/go-yamux/pull/106))\n - fix: sendWindowUpdate respects deadlines (#105) ([libp2p/go-yamux#105](https://github.com/libp2p/go-yamux/pull/105))\n- github.com/multiformats/go-multiaddr (v0.9.0 -> v0.10.1):\n - release v0.10.1 (#206) ([multiformats/go-multiaddr#206](https://github.com/multiformats/go-multiaddr/pull/206))\n - fix nat64 well-known prefix check (#205) ([multiformats/go-multiaddr#205](https://github.com/multiformats/go-multiaddr/pull/205))\n - release v0.10.0 (#204) ([multiformats/go-multiaddr#204](https://github.com/multiformats/go-multiaddr/pull/204))\n - add a Unique function (#203) ([multiformats/go-multiaddr#203](https://github.com/multiformats/go-multiaddr/pull/203))\n - manet: add function to test if address is NAT64 IPv4 converted IPv6 address (#202) ([multiformats/go-multiaddr#202](https://github.com/multiformats/go-multiaddr/pull/202))\n - sync: update CI config files (#190) ([multiformats/go-multiaddr#190](https://github.com/multiformats/go-multiaddr/pull/190))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Henrique Dias | 14 | +3735/-17889 | 185 |\n| Sukun | 28 | +5910/-957 | 100 |\n| Jorropo | 40 | +2913/-2112 | 205 |\n| Marten Seemann | 41 | +2926/-1833 | 163 |\n| Marco Munizaga | 20 | +1559/-586 | 81 |\n| Prem Chaitanya Prathi | 1 | +757/-740 | 61 |\n| Laurent Senta | 2 | +69/-1094 | 32 |\n| Marcin Rataj | 11 | +339/-198 | 22 |\n| Steven Allen | 2 | +313/-161 | 9 |\n| Will | 2 | +118/-211 | 9 |\n| Adin Schmahmann | 4 | +275/-41 | 8 |\n| Michael Muré | 1 | +113/-164 | 6 |\n| Rod Vagg | 8 | +228/-46 | 28 |\n| Gus Eggert | 5 | +156/-93 | 21 |\n| Adrian Sutton | 1 | +190/-17 | 4 |\n| Hlib Kanunnikov | 3 | +139/-40 | 9 |\n| VM | 2 | +80/-79 | 49 |\n| UnkwUsr | 1 | +0/-124 | 1 |\n| Piotr Galar | 4 | +51/-59 | 5 |\n| web3-bot | 3 | +22/-46 | 4 |\n| Will Scott | 2 | +29/-28 | 6 |\n| Prithvi Shahi | 2 | +40/-7 | 2 |\n| Brad Fitzpatrick | 1 | +42/-2 | 2 |\n| Steve Loeppky | 1 | +6/-23 | 2 |\n| Sahib Yar | 1 | +4/-4 | 3 |\n| Russell Dempsey | 2 | +4/-2 | 2 |\n| Mohamed MHAMDI | 1 | +3/-3 | 1 |\n| Bryan White | 1 | +2/-2 | 1 |\n| Dennis Trautwein | 1 | +1/-1 | 1 |\n| Antonio Navarro Perez | 1 | +0/-1 | 1 |\n\n" }, { "path": "docs/changelogs/v0.23.md", "content": "# Kubo changelog v0.23\n\n- [v0.23.0](#v0230)\n\n## v0.23.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Mplex deprecation](#mplex-deprecation)\n - [Gateway: meaningful CAR responses on Not Found errors](#gateway-meaningful-car-responses-on-not-found-errors)\n - [Gateway: added `Gateway.DisableHTMLErrors` configuration option](#gateway-added-gatewaydisablehtmlerrors-configuration-option)\n - [Binary characters in file names: no longer works with old clients and new Kubo servers](#binary-characters-in-file-names-no-longer-works-with-old-clients-and-new-kubo-servers)\n - [Self-hosting `/routing/v1` endpoint for delegated routing needs](#self-hosting-routingv1-endpoint-for-delegated-routing-needs)\n - [Trustless Gateway Over Libp2p Experiment](#trustless-gateway-over-libp2p-experiment)\n - [Removal of `/quic` (Draft 29) support](#removal-of-quic-draft-29-support)\n - [Better Caching of multiaddresses for providers in DHT servers](#better-caching-of-multiaddresses-for-providers-in-dht-servers)\n - [Fixed FUSE multiblock structures](#fixed-fuse-multiblock-structures)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Mplex deprecation\n\nMplex is being deprecated, this is because it is unreliable and\nrandomly drop streams when sending data *too fast*.\n\nNew pieces of code rely on backpressure, that means the stream will dynamically\nslow down the sending rate if data is getting backed up.\nBackpressure is provided by **Yamux** and **QUIC**.\n\nIn case you need compatibility with older implementations that do not ship with\nYamux (like default's JS-IPFS) you can turned it back ON in the config with:\n```console\n$ ipfs config --json Swarm.Transports.Multiplexers.Mplex 200\n```\n\nWe will completely remove Mplex in v0.24 as it makes protocols very bad to implement,\nif you are in this situation you need to add yamux support to your other implementation.\n\n#### Gateway: meaningful CAR responses on Not Found errors\n\nWhen requesting a CAR from the gateway, the root of the CAR might no longer be\nmeaningful. By default, the CAR root will be the last resolvable segment of the\npath. However, in situations where the path cannot be resolved, such as when\nthe path does not exist, a CAR will be sent with a root of `bafkqaaa` (empty CID).\n\nThis CAR will contain all blocks necessary to validate that the path does not\nexist without having to trust the gateway.\n\n#### Gateway: added `Gateway.DisableHTMLErrors` configuration option\n\nThe `Gateway.DisableHTMLErrors` configuration option forces errors to be\ndisplayed in browsers as plain text (`text/plain`) rather than HTML error\npages. It's especially beneficial for whitelabel or middleware deployments that\nwish to avoid IPFS branding and links on error pages in browsers.\n\n#### Binary characters in file names: no longer works with old clients and new Kubo servers\n\nIn this version, we updated Kubo to support Go 1.20+. In Go 1.20, a regression\nregarding multipart headers was [introduced](https://github.com/golang/go/issues/60674).\nThis only affects `ipfs add` when a file name has binary characters in its name.\nAs a consequence, we had to update the encoding of the file name headers. This is\nthe compatibility table:\n\n| | New Client | Old Client |\n|------------|------------|-------------|\n| New Server | ✅ | 🟡* |\n| Old Server | ✅ | ✅ |\n\n*Old clients can only send Unicode file paths to the server.\n\n#### Self-hosting `/routing/v1` endpoint for delegated routing needs\n\nThe `Routing` system configured in Kubo can be now exposed on the gateway port as a standard\nHTTP [Routing V1](https://specs.ipfs.tech/routing/http-routing-v1/) API endpoint. This allows \nself-hosting and experimentation with custom delegated routers. This is disabled by default,\nbut can be enabled by setting [`Gateway.ExposeRoutingAPI`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewayexposeroutingapi) to `true` .\n\n#### Trustless Gateway Over Libp2p Experiment\n\nIn this update, we've introduced an experimental opt-in feature allowing users to\nserve a subset of [Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/) responses,\nsuch as blocks and CARs, over libp2p. This enhancement leverages the ongoing\n[`/http/1.1` specification work in libp2p](https://github.com/libp2p/specs/pull/508)\nto make it easier to support HTTP semantics over libp2p streams.\n\nThis development means that if users wish to utilize the Trustless Gateway API\nfor data transport, they can now do so even in scenarios where standard HTTP\nmight be problematic, such as when the endpoint is behind a firewall or when\nattempting to serve data to a browser without a CA certificate.\n\nSee [HTTP Gateway over Libp2p](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#http-gateway-over-libp2p) for details about this experiment.\n\n#### Removal of `/quic` (Draft 29) support\n\nKubo no longer supports QUIC Draft 29. This means that older nodes aren't able to connect\nto newer nodes using QUIC Draft 29. However, they are still able to connect through any other\ntransport that both nodes talk (such as QUIC RFC 9000, or TCP). QUIC Draft 29 was a preliminary implementation of QUIC before\nthe official RFC 9000 was published, and it has now been dropped by [`go-libp2p`](https://github.com/libp2p/go-libp2p/releases/tag/v0.30.0)\nand therefore Kubo.\n\nIn [Kubo 0.18](https://github.com/ipfs/kubo/releases/tag/v0.18.0), we shipped a migration\nto have listeners for both `/quic` (Draft 29) and `/quic-v1` (RFC 9000). Similarly, in this\nversion we are shipping a migration to remove the current `/quic` addresses, maintaining\nthe `/quic-v1` addresses only. For more background information, check [issue #9496](https://github.com/ipfs/kubo/issues/9496).\n\n#### Better Caching of multiaddresses for providers in DHT servers\n\nThanks to [probelab.io's RFM17.1](https://github.com/plprobelab/network-measurements/blob/master/results/rfm17.1-sharing-prs-with-multiaddresses.md) DHT servers will [now cache the addresses of content hosts for the lifetime of the provider record](https://github.com/libp2p/go-libp2p-kad-dht/commit/777160f164b8c187c534debd293157031e9f3a02).\n\nThis means clients who resolve content from these servers get a responses which include both peer id and multiaddresses.\nIn most cases this enables skipping a second query which resolves the peer id to multiaddresses for stable enough peers.\n\nThis will improve content fetching lantency in the network overtime as servers updates.\n\n#### Fixed FUSE multiblock structures\n\n`ls`ing directories and reading dag-pb files on a fuse volume have been fixed. [#9044](https://github.com/ipfs/kubo/issues/9044)\nThx a lot @bmwiedemann for debugging this issue.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix: align systemd unit file with default IPFS installation path (#10163) ([ipfs/kubo#10163](https://github.com/ipfs/kubo/pull/10163))\n - docs: capitalize headers for consistency\n - Merge commit '695bf66674931a138862b6fa2cb0b16dc2f6ddd8' into release-v0.23.0\n - chore: update version\n - changelog: generalize fuse 9044's entry\n - changelog: update fuse 9044's entry\n - Update go-unixfsnode to 1.8.0 to fix FUSE\n - docs(readme): header improvements (#10144) ([ipfs/kubo#10144](https://github.com/ipfs/kubo/pull/10144))\n - fix(docker): allow nofuse builds for MacOS (#10135) ([ipfs/kubo#10135](https://github.com/ipfs/kubo/pull/10135))\n - docs: fix typos\n - docs: s/ipfs dht/amino dht/\n - changelog: mention probelab RFM17.1 dht improvement\n - tests: remove sharness ping tests\n - perf: make bootstrap saves O(N)\n - chore: update go-libp2p-kad-dht\n - chore: webui v4.1.1 (#10120) ([ipfs/kubo#10120](https://github.com/ipfs/kubo/pull/10120))\n - core/bootstrap: fix panic without backup bootstrap peer functions (#10029) ([ipfs/kubo#10029](https://github.com/ipfs/kubo/pull/10029))\n - feat: add Gateway.DisableHTMLErrors option (#10137) ([ipfs/kubo#10137](https://github.com/ipfs/kubo/pull/10137))\n - fix(migrations): use dweb.link (#10133) ([ipfs/kubo#10133](https://github.com/ipfs/kubo/pull/10133))\n - docs: add changelog info for QUIC Draft 29 (#10132) ([ipfs/kubo#10132](https://github.com/ipfs/kubo/pull/10132))\n - feat: add gateway to http over libp2p ([ipfs/kubo#10108](https://github.com/ipfs/kubo/pull/10108))\n - migration: update 14-to-15 to v1.0.1\n - chore: update to build with Go 1.21\n - refactor: stop using go-libp2p deprecated peer.ID.Pretty\n - docs(readonly): fix typo\n - docs(changelog): link to relevant IPIP\n - fix: hamt traversal in ipld-explorer (webui@4.1.0) (#10025) ([ipfs/kubo#10025](https://github.com/ipfs/kubo/pull/10025))\n - refactor: if statement (#10105) ([ipfs/kubo#10105](https://github.com/ipfs/kubo/pull/10105))\n - chore: bump repo version to 15\n - docs: remove link to deleted #accelerated-dht-client\n - feat(gateway): expose /routing/v1 server (opt-in) (#9877) ([ipfs/kubo#9877](https://github.com/ipfs/kubo/pull/9877))\n - improve error in fuse node failures\n - chore: update boxo, go-libp2p, and internalize mplex (#10095) ([ipfs/kubo#10095](https://github.com/ipfs/kubo/pull/10095))\n - dockerfile: reorder copy order for better layer caching\n - refactor: using error is instead of == (#10093) ([ipfs/kubo#10093](https://github.com/ipfs/kubo/pull/10093))\n - fix: use %-encoded headers in most compatible way\n - fix: open /dev/null with read write permissions\n - chore: bump to go 1.20\n - docs(readme): new logo and header\n - docker: change to releases that follow debian's updates\n - docker: bump debian version to bookworm\n - chore: restore exec perms for t0116-gateway-cache.sh and fixtures (#10085) ([ipfs/kubo#10085](https://github.com/ipfs/kubo/pull/10085))\n - fix(gw): useful IPIP-402 CARs on not found errors (#10084) ([ipfs/kubo#10084](https://github.com/ipfs/kubo/pull/10084))\n - feat: add zsh completions (#10040) ([ipfs/kubo#10040](https://github.com/ipfs/kubo/pull/10040))\n - style: remove commented imports [skip changelog]\n - style: gofumpt and godot [skip changelog] (#10081) ([ipfs/kubo#10081](https://github.com/ipfs/kubo/pull/10081))\n - chore: bump boxo for verifcid breaking changes\n - chore: remove outdated comment (#10077) ([ipfs/kubo#10077](https://github.com/ipfs/kubo/pull/10077))\n - chore: remove deprecated testground plans\n - feat: allow users to option again into mplex\n - feat: remove Mplex\n - docs(readme): minimal reqs (#10066) ([ipfs/kubo#10066](https://github.com/ipfs/kubo/pull/10066))\n - docs: add v0.23.md\n - docs: get ready for v0.23\n - chore: fix link in v0.22 changelog\n- github.com/ipfs/boxo (v0.11.0 -> v0.13.1):\n - Release v0.13.1 ([ipfs/boxo#469](https://github.com/ipfs/boxo/pull/469))\n - Release v0.13.0 ([ipfs/boxo#465](https://github.com/ipfs/boxo/pull/465))\n - Release v0.12 ([ipfs/boxo#446](https://github.com/ipfs/boxo/pull/446))\n- github.com/ipfs/go-graphsync (v0.14.4 -> v0.15.1):\n - v0.15.1 bump\n - fix: partial revert of 1be7c1a20; make traverser process identity CIDs\n - v0.15.0 bump\n - chore: add identity CID parse tests\n - fix: traverser should skip over identity CIDs\n - fix(ipld): update ipld deps, only slurp LargeBytesNode when matching\n - docs(version): update for v0.14.7\n - Handle context cancellation properly (#428) ([ipfs/go-graphsync#428](https://github.com/ipfs/go-graphsync/pull/428))\n - chore(version.json): update for v0.14.6\n - feat: MaxLinks for requests (#420) ([ipfs/go-graphsync#420](https://github.com/ipfs/go-graphsync/pull/420))\n - fix(responsemanager): network disconnect reliability (#425) ([ipfs/go-graphsync#425](https://github.com/ipfs/go-graphsync/pull/425))\n - Update version to reflect latest fixes (#424) ([ipfs/go-graphsync#424](https://github.com/ipfs/go-graphsync/pull/424))\n - Fix shutdown bug in #412 (#422) ([ipfs/go-graphsync#422](https://github.com/ipfs/go-graphsync/pull/422))\n- github.com/ipfs/go-ipfs-cmds (v0.9.0 -> v0.10.0):\n - chore: version 0.10.0\n - fix: panic when calling .SetLength for writerResponseEmitter\n - fix!: client with raw abs path option\n - doc: clarify flag inheritance explanation\n - ci: uci/copy-templates ([ipfs/go-ipfs-cmds#242](https://github.com/ipfs/go-ipfs-cmds/pull/242))\n - chore: remove dep on github.com/Kubuxu/go-os-helper\n- github.com/ipfs/go-unixfsnode (v1.7.1 -> v1.8.1):\n - v1.8.1 bump\n - testutil: relax DirEntry usage for non-dag-pb\n - v1.8.0 bump\n - fix: add cross-impl shard test\n - files returned from unixfsnode should be traversable back to their substrate\n - fix: better import name\n - chore: refactor and add tests with fixtures\n - fix: proper tsize encoding in sharded files\n - rel 1.7.4\n - Provide path for getting sizes on directory iteration ([ipfs/go-unixfsnode#60](https://github.com/ipfs/go-unixfsnode/pull/60))\n - tag 1.7.3 ([ipfs/go-unixfsnode#57](https://github.com/ipfs/go-unixfsnode/pull/57))\n - Fail to construct preload hamt shards when traversal fails ([ipfs/go-unixfsnode#55](https://github.com/ipfs/go-unixfsnode/pull/55))\n - fix: large files support io.SeekCurrent ([ipfs/go-unixfsnode#56](https://github.com/ipfs/go-unixfsnode/pull/56))\n - chore(version): update version number\n - feat: add entity matcher w/o preload, add matcher fn for consuming bytes ([ipfs/go-unixfsnode#52](https://github.com/ipfs/go-unixfsnode/pull/52))\n- github.com/ipld/go-ipld-prime (v0.20.0 -> v0.21.0):\n - v0.21.0 release\n - fix(selectors): document ranges in slice matcher\n - fix(selectors): update ipld/ipld submodule with latest fixtures\n - fix(selectors): more permissive with slice \"from\" underflow\n - chore: extract simpleBytes to testutil package\n - feat(selectors): negative values for slice matcher's From and To\n - chore: extract MultiByteNote to testutil package\n - feat(test): add matcher/slice selector test cases\n - feat: remove hard-error when slice matcher reaches non-string/bytes node\n - fix: cache offsets for sequential reads\n - feat: add inline union representation to schema parser\n - fix: basic.NewInt returns pointer (like others)\n - fix(bindnode): listpairs value assembly handles complex reprs\n - fix(bindnode): listpairs repr assembler handles AssignNode\n - fix(schema): handle parsing of \"listpairs\" in the DSL\n - fix: remove _skipAbsent labels\n - fix: make listpairs repr [[k1,v1],[k2,v2]...]\n - feat(bindnode): support listpairs struct representation\n - fix(windows,test): avoid \"already exists\" error on codegen tests for Windows\n - Make traversal.WalkTransforming() work\n - doc: clean up and expand on traversal pkg docs\n - doc: add lots of notes about using the preloader and the budget\n - doc: expand on preloader docs\n - fix: inline initialPhase() logic for clarity\n - feat: preload walk using phase state, call preloader once per link\n - fix: handle Budget & SeenLinks\n - chore: remove BufferedLoader\n - fix: recurse preloader at block level\n - fix: Context->PreloadContext for clarity and consistency with LinkContext\n - fix: replace ioutil.ReadAll\n - fix: fix tooling complaints\n - feat: add BufferedLoader\n - feat(traversal): allow preloading functionality\n - fix: address dodgy test case variable capture\n - stop using the deprecated io/ioutil package\n - stop using the deprecated io/ioutil package\n - stop using the deprecated io/ioutil package\n - fix: make StartAtPath work properly for matching walks\n- github.com/libp2p/go-libp2p (v0.29.2 -> v0.31.0):\n - release v0.31.0 (#2543) ([libp2p/go-libp2p#2543](https://github.com/libp2p/go-libp2p/pull/2543))\n - dashboards: improve naming for black hole panel (#2539) ([libp2p/go-libp2p#2539](https://github.com/libp2p/go-libp2p/pull/2539))\n - reuseport: use DialContext instead of Dial to fail quickly (#2541) ([libp2p/go-libp2p#2541](https://github.com/libp2p/go-libp2p/pull/2541))\n - swarm: track dial cancellation reason (#2532) ([libp2p/go-libp2p#2532](https://github.com/libp2p/go-libp2p/pull/2532))\n - p2p/http: cache json wellknown mappings in the .well-known handler (#2537) ([libp2p/go-libp2p#2537](https://github.com/libp2p/go-libp2p/pull/2537))\n - feat: Implement HTTP spec (#2438) ([libp2p/go-libp2p#2438](https://github.com/libp2p/go-libp2p/pull/2438))\n - move libp2p/go-libp2p-gostream to p2p/net/gostream ([libp2p/go-libp2p#2535](https://github.com/libp2p/go-libp2p/pull/2535))\n - host: disable black hole detection on autonat dialer (#2529) ([libp2p/go-libp2p#2529](https://github.com/libp2p/go-libp2p/pull/2529))\n - identify: disable racy test when running with race detector (#2526) ([libp2p/go-libp2p#2526](https://github.com/libp2p/go-libp2p/pull/2526))\n - swarm: return a more meaningful error when dialing QUIC draft-29 (#2524) ([libp2p/go-libp2p#2524](https://github.com/libp2p/go-libp2p/pull/2524))\n - swarm: fix Unwrap for DialError, implement Unwrap for TransportError (#2437) ([libp2p/go-libp2p#2437](https://github.com/libp2p/go-libp2p/pull/2437))\n - swarm: return errors on filtered addresses when dialing (#2461) ([libp2p/go-libp2p#2461](https://github.com/libp2p/go-libp2p/pull/2461))\n - core: add ErrPeerIDMismatch error type to replace ad-hoc errors (#2451) ([libp2p/go-libp2p#2451](https://github.com/libp2p/go-libp2p/pull/2451))\n - update quic-go to v0.38.1 (#2506) ([libp2p/go-libp2p#2506](https://github.com/libp2p/go-libp2p/pull/2506))\n - quic: don't claim to be able to dial draft-29 in CanDial (#2520) ([libp2p/go-libp2p#2520](https://github.com/libp2p/go-libp2p/pull/2520))\n - examples: update go-libp2p to v0.30.0 (#2507) ([libp2p/go-libp2p#2507](https://github.com/libp2p/go-libp2p/pull/2507))\n - metrics: update dashboard names from libp2p to go-libp2p (#2512) ([libp2p/go-libp2p#2512](https://github.com/libp2p/go-libp2p/pull/2512))\n - chore: be more descriptive about where public dashboards come from (#2508) ([libp2p/go-libp2p#2508](https://github.com/libp2p/go-libp2p/pull/2508))\n - release v0.30.0 (#2505) ([libp2p/go-libp2p#2505](https://github.com/libp2p/go-libp2p/pull/2505))\n - transport tests: add deadline tests (#2286) ([libp2p/go-libp2p#2286](https://github.com/libp2p/go-libp2p/pull/2286))\n - chore: remove unused and outdated package-list.json (#2499) ([libp2p/go-libp2p#2499](https://github.com/libp2p/go-libp2p/pull/2499))\n - muxer: remove support for mplex (#2498) ([libp2p/go-libp2p#2498](https://github.com/libp2p/go-libp2p/pull/2498))\n - transport tests: refactor workers in TestMoreStreamsThanOurLimits (#2472) ([libp2p/go-libp2p#2472](https://github.com/libp2p/go-libp2p/pull/2472))\n - use standard library sha256 implementation for Go 1.21 (#2309) ([libp2p/go-libp2p#2309](https://github.com/libp2p/go-libp2p/pull/2309))\n - quic: update quic-go to v0.37.5 (#2497) ([libp2p/go-libp2p#2497](https://github.com/libp2p/go-libp2p/pull/2497))\n - cleanup: add continue in case of failure in the (*BasicHost).Addrs certhash loop (#2492) ([libp2p/go-libp2p#2492](https://github.com/libp2p/go-libp2p/pull/2492))\n - tests: add a CertHashes testcase in TestInferWebtransportAddrsFromQuic (#2495) ([libp2p/go-libp2p#2495](https://github.com/libp2p/go-libp2p/pull/2495))\n - basichost: use byte representation of WebTransport multiaddr as map key (#2494) ([libp2p/go-libp2p#2494](https://github.com/libp2p/go-libp2p/pull/2494))\n - webtransport: check for UDP multiaddr component in address matcher (#2491) ([libp2p/go-libp2p#2491](https://github.com/libp2p/go-libp2p/pull/2491))\n - swarm: remove unnecessary reqno for pending request tracking (#2460) ([libp2p/go-libp2p#2460](https://github.com/libp2p/go-libp2p/pull/2460))\n - quic: drop support for QUIC draft-29 (#2487) ([libp2p/go-libp2p#2487](https://github.com/libp2p/go-libp2p/pull/2487))\n - metrics: add links to public dashboards (#2486) ([libp2p/go-libp2p#2486](https://github.com/libp2p/go-libp2p/pull/2486))\n - swarm: remove leftover TODO (#2474) ([libp2p/go-libp2p#2474](https://github.com/libp2p/go-libp2p/pull/2474))\n - peerstore: deprecate the database-backed peerstore (#2475) ([libp2p/go-libp2p#2475](https://github.com/libp2p/go-libp2p/pull/2475))\n - identify: fix sorting of observed addresses (#2476) ([libp2p/go-libp2p#2476](https://github.com/libp2p/go-libp2p/pull/2476))\n - update go-multiaddr to v0.11.0 (#2467) ([libp2p/go-libp2p#2467](https://github.com/libp2p/go-libp2p/pull/2467))\n - chore: update golang-lru to v2.0.4, fixing semver violation (#2448) ([libp2p/go-libp2p#2448](https://github.com/libp2p/go-libp2p/pull/2448))\n - swarm: don't open new streams over transient connections (#2450) ([libp2p/go-libp2p#2450](https://github.com/libp2p/go-libp2p/pull/2450))\n - core/crypto: restrict RSA keys to <= 8192 bits (#2454) ([libp2p/go-libp2p#2454](https://github.com/libp2p/go-libp2p/pull/2454))\n - chore: add notable project requirement (#2453) ([libp2p/go-libp2p#2453](https://github.com/libp2p/go-libp2p/pull/2453))\n - examples: update go-libp2p to v0.29.0 (#2432) ([libp2p/go-libp2p#2432](https://github.com/libp2p/go-libp2p/pull/2432))\n - examples: fix description of command line flags for pubsub (#2400) ([libp2p/go-libp2p#2400](https://github.com/libp2p/go-libp2p/pull/2400))\n - basichost: remove invalid comment (#2435) ([libp2p/go-libp2p#2435](https://github.com/libp2p/go-libp2p/pull/2435))\n- github.com/libp2p/go-libp2p-kad-dht (v0.24.2 -> v0.24.4):\n - Make v0.24.4 ([libp2p/go-libp2p-kad-dht#931](https://github.com/libp2p/go-libp2p-kad-dht/pull/931))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.7.1 -> v0.7.3):\n - chore: release v0.7.3\n - nit: invert if\n - fix: for getValueOrErrorParallel do not return values if they come with errors\n - test: add test to make sure we return not found when we get errors back with values\n - chore: release v0.7.2\n - tracing: do not leak goroutines when the context is canceled\n - tracing: allow for reuse of the tracing\n - tracing: add tracing to compose parallel's worker\n - tests: add more tests\n - tests: mark all tests Parallel\n - tracing: add highlevel APIs records on the composable routers\n- github.com/libp2p/go-reuseport (v0.3.0 -> v0.4.0):\n - release v0.4.0 (#111) ([libp2p/go-reuseport#111](https://github.com/libp2p/go-reuseport/pull/111))\n - use SO_REUSEPORT_LB on FreeBSD (#106) ([libp2p/go-reuseport#106](https://github.com/libp2p/go-reuseport/pull/106))\n- github.com/multiformats/go-multiaddr (v0.10.1 -> v0.11.0):\n - release v0.11.0 (#214) ([multiformats/go-multiaddr#214](https://github.com/multiformats/go-multiaddr/pull/214))\n - update golang.org/x/exp slice comparison to match standard library version (#210) ([multiformats/go-multiaddr#210](https://github.com/multiformats/go-multiaddr/pull/210))\n- github.com/warpfork/go-testmark (v0.11.0 -> v0.12.1):\n - suite: allow disabling file parallelism.\n - Suite feature ([warpfork/go-testmark#16](https://github.com/warpfork/go-testmark/pull/16))\n - fix unchecked error in a test\n - accept a simplification suggestion from linters\n - Trailing whitespace error ([warpfork/go-testmark#15](https://github.com/warpfork/go-testmark/pull/15))\n - FS implementation (#11) ([warpfork/go-testmark#11](https://github.com/warpfork/go-testmark/pull/11))\n - Add a readme for the testexec extension and its conventions. ([warpfork/go-testmark#14](https://github.com/warpfork/go-testmark/pull/14))\n - Strict mode for testexec structure ([warpfork/go-testmark#12](https://github.com/warpfork/go-testmark/pull/12))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Rod Vagg | 48 | +3578/-1789 | 110 |\n| Henrique Dias | 24 | +3173/-1128 | 104 |\n| Jorropo | 51 | +1721/-1297 | 252 |\n| Marco Munizaga | 6 | +1989/-505 | 39 |\n| Kay | 3 | +487/-474 | 163 |\n| hannahhoward | 8 | +626/-136 | 23 |\n| Calvin Behling | 6 | +496/-259 | 20 |\n| Eric Myhre | 9 | +610/-121 | 16 |\n| Adin Schmahmann | 17 | +659/-45 | 35 |\n| Marten Seemann | 17 | +218/-477 | 119 |\n| Sukun | 11 | +481/-174 | 29 |\n| CJB | 1 | +639/-2 | 5 |\n| Hector Sanjuan | 10 | +450/-127 | 21 |\n| Wondertan | 2 | +203/-127 | 8 |\n| Marcin Rataj | 11 | +148/-86 | 18 |\n| Andrew Gillis | 2 | +163/-14 | 5 |\n| P. Reis | 3 | +120/-4 | 4 |\n| Will Scott | 4 | +107/-12 | 6 |\n| Amir Mohammad Fakhimi | 1 | +97/-2 | 5 |\n| Ed Schouten | 1 | +55/-7 | 2 |\n| Icarus9913 | 1 | +30/-30 | 18 |\n| Dirk McCormick | 1 | +3/-42 | 1 |\n| Raúl Kripalani | 1 | +20/-18 | 4 |\n| Michael Muré | 1 | +26/-7 | 5 |\n| Prem Chaitanya Prathi | 1 | +28/-1 | 2 |\n| ShengTao | 1 | +13/-14 | 4 |\n| Prithvi Shahi | 3 | +14/-13 | 3 |\n| web3-bot | 5 | +12/-10 | 9 |\n| Alejandro Criado-Pérez | 1 | +11/-11 | 6 |\n| Steven Allen | 2 | +6/-10 | 2 |\n| Andrej Manduch | 1 | +5/-5 | 3 |\n| Russell Dempsey | 2 | +4/-2 | 2 |\n| Johannes Maria Frank | 1 | +4/-1 | 1 |\n| downIoads | 1 | +2/-2 | 1 |\n| Will | 2 | +2/-2 | 2 |\n| Marin Kirkov | 1 | +2/-2 | 2 |\n| Gus Eggert | 1 | +2/-2 | 1 |\n| Bernhard M. Wiedemann | 1 | +4/-0 | 1 |\n| Dennis Trautwein | 1 | +1/-2 | 1 |\n| “GheisMohammadi” | 1 | +1/-1 | 1 |\n| cce | 1 | +1/-1 | 1 |\n| Joao Andrade | 1 | +1/-1 | 1 |\n| guillaumemichel | 1 | +1/-0 | 1 |\n| Santiago Botto | 1 | +0/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.24.md", "content": "# Kubo changelog v0.24\n\n- [v0.24.0](#v0240)\n\n## v0.24.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Support for content blocking](#support-for-content-blocking)\n - [Gateway: the root of the CARs are no longer meaningful](#gateway-the-root-of-the-cars-are-no-longer-meaningful)\n - [IPNS: improved publishing defaults](#ipns-improved-publishing-defaults)\n - [IPNS: record TTL is used for caching](#ipns-record-ttl-is-used-for-caching)\n - [Experimental Transport: WebRTC Direct](#experimental-transport-webrtc-direct)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Support for content blocking\n\nThis Kubo release ships with built-in content-blocking subsystem [announced earlier this year](https://blog.ipfs.tech/2023-content-blocking-for-the-ipfs-stack/).\nContent blocking is an opt-in decision made by the operator of `ipfs daemon`.\nThe official build does not ship with any denylists.\n\nLearn more at [`/docs/content-blocking.md`](https://github.com/ipfs/kubo/blob/master/docs/content-blocking.md)\n\n#### Gateway: the root of the CARs are no longer meaningful\n\nWhen requesting a CAR from the gateway, the root of the CAR might no longer be\nmeaningful. By default, the CAR root will be the last resolvable segment of the\npath. However, in situations where the path cannot be resolved, such as when\nthe path does not exist, a CAR will be sent with a root of `bafkqaaa` (empty CID).\nThis CAR will contain all blocks necessary to validate that the path does not exist.\n\n#### IPNS: improved publishing defaults\n\nThis release changes the default values used when publishing IPNS record\nvia `ipfs name publish` command:\n\n- Default `--lifetime` increased from `24h` to `48h` to take full advantage of\n the increased expiration window of Amino DHT\n ([go-libp2p-kad-dht#793](https://github.com/libp2p/go-libp2p-kad-dht/pull/793))\n- Default `--ttl` increased from `1m` to `1h` to improve website caching and follow\n saner defaults present in similar systems like DNS\n ([specs#371](https://github.com/ipfs/specs/pull/371))\n\nThis change only impacts the implicit defaults, when mentioned parameters are omitted\nduring publishing. Users are free to override the default if different value\nmakes more sense for their use case.\n\n#### IPNS: record TTL is used for caching\n\nIn this release, we've made significant improvements to IPNS caching.\n\nPreviously, the TTL value in IPNS records was not utilized, and the\n`boxo/namesys` library maintained a static one-minute resolution cache.\n\nWith this update, IPNS publishers gain more control over how long a valid IPNS\nrecord remains cached before checking an upstream routing system, such as Amino\nDHT, for updates. The TTL value in the IPNS record now serves as a hint for:\n\n- `boxo/namesys`: the internal cache, determining how long the IPNS resolution\n result is cached before asking upstream routing systems for updates.\n- `boxo/gateway`: the `Cache-Control` HTTP header in responses to requests made\n for `/ipns/name` content paths.\n\nThese changes make it easier for rarely updated IPNS-hosted websites to be\ncached more efficiently and load faster in browser contexts.\n\n#### Experimental Transport: WebRTC Direct\n\nThis Kubo release includes the initial work towards WebRTC Direct\nintroduced in [`go-libp2p`](https://github.com/libp2p/go-libp2p/releases/tag/v0.32.0) v0.32:\n\n> [WebRTC Direct](https://github.com/libp2p/specs/blob/master/webrtc/webrtc-direct.md)\n> allows browser nodes to connect to go-libp2p nodes directly,\n> without any configuration (e.g. TLS certificates) needed on the go-libp2p\n> side. This is useful for browser nodes that aren’t able to use\n> [WebTransport](https://web.archive.org/web/20260107053250/https://blog.libp2p.io/2022-12-19-libp2p-webtransport/).\n\nThe `/webrtc-direct` transport is disabled by default in Kubo 0.24,\nand not ready for production use yet, but we plan to enable it in a future release.\n\nSee [`Swarm.Transports.Network.WebRTCDirect`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmtransportsnetworkwebrtcdirect)\nto learn how to enable it manually, and what current limitations are.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - fix: allow event emitting to happen in parallel with getting the query channel\n - fixes to routing put command (#10205) ([ipfs/kubo#10205](https://github.com/ipfs/kubo/pull/10205))\n - docs: fix accelerated-dht-client\n - docs/config: remove extra commas in PublicGateways example entries\n - chore: update version\n - docs: make it clear Web RTC Direct is experimental\n - feat: add WebRTC Direct support\n - docs: update EARLY_TESTERS.md (#10194) ([ipfs/kubo#10194](https://github.com/ipfs/kubo/pull/10194))\n - Release: v0.24.0-1 ([ipfs/kubo#10190](https://github.com/ipfs/kubo/pull/10190))\n- github.com/ipfs/boxo (v0.13.1 -> v0.15.0):\n - Release v0.15.0 ([ipfs/boxo#505](https://github.com/ipfs/boxo/pull/505))\n - Release v0.14.0 ([ipfs/boxo#500](https://github.com/ipfs/boxo/pull/500))\n- github.com/ipfs/go-block-format (v0.1.2 -> v0.2.0):\n - v0.2.0 bump\n- github.com/ipfs/go-graphsync (v0.15.1 -> v0.16.0):\n - chore: release 0.16.0\n - chore: bump go-libp2p to 0.32.0\n- github.com/ipfs/go-ipld-format (v0.5.0 -> v0.6.0):\n - v0.6.0 bump\n - chore: update deps\n - fix: stop using the deprecated io/ioutil package\n- github.com/libp2p/go-libp2p (v0.31.0 -> v0.32.1):\n - release v0.32.1 (#2637) ([libp2p/go-libp2p#2637](https://github.com/libp2p/go-libp2p/pull/2637))\n - swarm: fix timer Leak in the dial loop (#2636) ([libp2p/go-libp2p#2636](https://github.com/libp2p/go-libp2p/pull/2636))\n - release v0.32.0 (#2625) ([libp2p/go-libp2p#2625](https://github.com/libp2p/go-libp2p/pull/2625))\n - chore: update js-libp2p examples repo (#2624) ([libp2p/go-libp2p#2624](https://github.com/libp2p/go-libp2p/pull/2624))\n - identify: don't filter dns addresses based on remote addr type (#2553) ([libp2p/go-libp2p#2553](https://github.com/libp2p/go-libp2p/pull/2553))\n - webrtc: fix race in TestRemoveConnByUfrag (#2620) ([libp2p/go-libp2p#2620](https://github.com/libp2p/go-libp2p/pull/2620))\n - swarm: fix recursive resolving of DNS multiaddrs (#2564) ([libp2p/go-libp2p#2564](https://github.com/libp2p/go-libp2p/pull/2564))\n - ci: migrate to renamed interop test action (#2617) ([libp2p/go-libp2p#2617](https://github.com/libp2p/go-libp2p/pull/2617))\n - quic: update quic-go to v0.39.1, set a static resumption token generator key (#2572) ([libp2p/go-libp2p#2572](https://github.com/libp2p/go-libp2p/pull/2572))\n - test/basichost: fix flaky test due to rcmgr (#2613) ([libp2p/go-libp2p#2613](https://github.com/libp2p/go-libp2p/pull/2613))\n - swarm: use typed atomics (#2612) ([libp2p/go-libp2p#2612](https://github.com/libp2p/go-libp2p/pull/2612))\n - swarm: cleanup stream handler goroutine (#2610) ([libp2p/go-libp2p#2610](https://github.com/libp2p/go-libp2p/pull/2610))\n - circuitv2: don't check ASN for private addrs (#2611) ([libp2p/go-libp2p#2611](https://github.com/libp2p/go-libp2p/pull/2611))\n - swarm: use happy eyeballs ranking for TCP dials (#2573) ([libp2p/go-libp2p#2573](https://github.com/libp2p/go-libp2p/pull/2573))\n - webrtc: fix race in TestMuxedConnection (#2607) ([libp2p/go-libp2p#2607](https://github.com/libp2p/go-libp2p/pull/2607))\n - tcp: fix build on riscv64 (#2590) ([libp2p/go-libp2p#2590](https://github.com/libp2p/go-libp2p/pull/2590))\n - Fix missing deprecation tag (#2605) ([libp2p/go-libp2p#2605](https://github.com/libp2p/go-libp2p/pull/2605))\n - swarm: wait for transient connections to upgrade for NewStream (#2542) ([libp2p/go-libp2p#2542](https://github.com/libp2p/go-libp2p/pull/2542))\n - docs: fix typos (#2604) ([libp2p/go-libp2p#2604](https://github.com/libp2p/go-libp2p/pull/2604))\n - webrtc: correctly report incoming packet address on muxed connection (#2586) ([libp2p/go-libp2p#2586](https://github.com/libp2p/go-libp2p/pull/2586))\n - swarm: add loopback to low timeout filter (#2595) ([libp2p/go-libp2p#2595](https://github.com/libp2p/go-libp2p/pull/2595))\n - Fix typos in comments and a test failure message (#2600) ([libp2p/go-libp2p#2600](https://github.com/libp2p/go-libp2p/pull/2600))\n - libp2phttp: don't strip `/` suffix when mounting handler (#2552) ([libp2p/go-libp2p#2552](https://github.com/libp2p/go-libp2p/pull/2552))\n - interop: fix redis env var (#2585) ([libp2p/go-libp2p#2585](https://github.com/libp2p/go-libp2p/pull/2585))\n - quicreuse: remove QUIC metrics tracer (#2582) ([libp2p/go-libp2p#2582](https://github.com/libp2p/go-libp2p/pull/2582))\n - config: warn if connmgr limits conflict with rcmgr (#2527) ([libp2p/go-libp2p#2527](https://github.com/libp2p/go-libp2p/pull/2527))\n - update gomock to v0.3.0 (#2581) ([libp2p/go-libp2p#2581](https://github.com/libp2p/go-libp2p/pull/2581))\n - webrtc: fix deadlock on connection close (#2580) ([libp2p/go-libp2p#2580](https://github.com/libp2p/go-libp2p/pull/2580))\n - webrtc: put buffer back to pool (#2574) ([libp2p/go-libp2p#2574](https://github.com/libp2p/go-libp2p/pull/2574))\n - webrtc: fail Write early if deadline has exceeded before the call (#2578) ([libp2p/go-libp2p#2578](https://github.com/libp2p/go-libp2p/pull/2578))\n - swarm: fix DialPeer behaviour for transient connections (#2547) ([libp2p/go-libp2p#2547](https://github.com/libp2p/go-libp2p/pull/2547))\n - websocket: don't resolve /dnsaddr addresses (#2571) ([libp2p/go-libp2p#2571](https://github.com/libp2p/go-libp2p/pull/2571))\n - core/peer: remove deprecated ID.Pretty method (#2565) ([libp2p/go-libp2p#2565](https://github.com/libp2p/go-libp2p/pull/2565))\n - core/peer: remove deprecated Encode function (#2566) ([libp2p/go-libp2p#2566](https://github.com/libp2p/go-libp2p/pull/2566))\n - mock: use go.uber.org/mock (#2540) ([libp2p/go-libp2p#2540](https://github.com/libp2p/go-libp2p/pull/2540))\n - add WebRTC Direct transport implementation (#2337) ([libp2p/go-libp2p#2337](https://github.com/libp2p/go-libp2p/pull/2337))\n - upgrader: drop support for multistream simultaneous open (#2557) ([libp2p/go-libp2p#2557](https://github.com/libp2p/go-libp2p/pull/2557))\n - examples: stop using deprecated peer.ID.Pretty (#2563) ([libp2p/go-libp2p#2563](https://github.com/libp2p/go-libp2p/pull/2563))\n - swarm: don't dial unspecified addresses (#2560) ([libp2p/go-libp2p#2560](https://github.com/libp2p/go-libp2p/pull/2560))\n - basichost: handle the SetProtocol error in NewStream (#2555) ([libp2p/go-libp2p#2555](https://github.com/libp2p/go-libp2p/pull/2555))\n - libp2phttp: don't initialise ServeMux if not nil (#2548) ([libp2p/go-libp2p#2548](https://github.com/libp2p/go-libp2p/pull/2548))\n- github.com/libp2p/go-libp2p-pubsub (v0.9.3 -> v0.10.0):\n - chore: update go-libp2p to v0.32 (#548) ([libp2p/go-libp2p-pubsub#548](https://github.com/libp2p/go-libp2p-pubsub/pull/548))\n - remove usage of deprecated peerid.Pretty method (#542) ([libp2p/go-libp2p-pubsub#542](https://github.com/libp2p/go-libp2p-pubsub/pull/542))\n - Revert \"fix: topicscore params can't be set for dynamically subscribed topic (#540)\" (#541) ([libp2p/go-libp2p-pubsub#541](https://github.com/libp2p/go-libp2p-pubsub/pull/541))\n - fix: topicscore params can't be set for dynamically subscribed topic (#540) ([libp2p/go-libp2p-pubsub#540](https://github.com/libp2p/go-libp2p-pubsub/pull/540))\n- github.com/multiformats/go-multiaddr (v0.11.0 -> v0.12.0):\n - release v0.12.0 (#223) ([multiformats/go-multiaddr#223](https://github.com/multiformats/go-multiaddr/pull/223))\n - net: consider /dns/localhost as private address (#221) ([multiformats/go-multiaddr#221](https://github.com/multiformats/go-multiaddr/pull/221))\n - net: consider dns addresses as public (#220) ([multiformats/go-multiaddr#220](https://github.com/multiformats/go-multiaddr/pull/220))\n- github.com/multiformats/go-multistream (v0.4.1 -> v0.5.0):\n - remove support for the simultaneous open extension (#107) ([multiformats/go-multistream#107](https://github.com/multiformats/go-multistream/pull/107))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Henrique Dias | 27 | +4505/-3853 | 244 |\n| Marten Seemann | 18 | +4260/-1173 | 101 |\n| Sukun | 24 | +1499/-340 | 79 |\n| Andrew Gillis | 4 | +169/-1025 | 16 |\n| Adin Schmahmann | 4 | +788/-184 | 19 |\n| Hector Sanjuan | 6 | +619/-72 | 19 |\n| Steven Allen | 11 | +489/-101 | 14 |\n| Jorropo | 10 | +221/-192 | 28 |\n| Łukasz Magiera | 2 | +306/-9 | 3 |\n| Lucas Molas | 1 | +183/-52 | 2 |\n| Marcin Rataj | 5 | +160/-25 | 6 |\n| piersy | 1 | +57/-0 | 6 |\n| Raúl Kripalani | 1 | +25/-25 | 2 |\n| Alvin Reyes | 1 | +34/-14 | 1 |\n| Dennis Trautwein | 1 | +1/-40 | 2 |\n| Icarus9913 | 1 | +14/-14 | 10 |\n| Takashi Matsuda | 2 | +18/-1 | 3 |\n| gammazero | 4 | +8/-5 | 7 |\n| xiaolou86 | 1 | +6/-6 | 5 |\n| Daniel Martí | 1 | +9/-2 | 1 |\n| Rod Vagg | 3 | +5/-5 | 4 |\n| Andrej Manduch | 1 | +5/-5 | 3 |\n| vuittont60 | 1 | +4/-4 | 3 |\n| vyzo | 1 | +5/-1 | 1 |\n| tkzktk | 1 | +3/-3 | 3 |\n| tk | 1 | +3/-3 | 2 |\n| Prem Chaitanya Prathi | 1 | +1/-5 | 1 |\n| Kay | 2 | +2/-3 | 2 |\n| Thomas Eizinger | 1 | +2/-2 | 1 |\n| Steve Loeppky | 1 | +2/-2 | 1 |\n| Jonas Keunecke | 1 | +2/-2 | 1 |\n| Alejandro Criado-Pérez | 1 | +1/-1 | 1 |\n| web3-bot | 1 | +1/-0 | 1 |\n| Eric | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.25.md", "content": "# Kubo changelog v0.25\n\n- [v0.25.0](#v0250)\n\n## v0.25.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [WebUI: Updated Peers View](#webui-updated-peers-view)\n - [RPC `API.Authorizations`](#rpc-apiauthorizations)\n - [MPLEX Removal](#mplex-removal)\n - [Graphsync Experiment Removal](#graphsync-experiment-removal)\n - [Commands `ipfs key sign` and `ipfs key verify`](#commands-ipfs-key-sign-and-ipfs-key-verify)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### WebUI: Updated Peers View\n\nWebUI [v4.2.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.2.0) shipped\nwith updated [ipfs-geoip](https://www.npmjs.com/package/ipfs-geoip) dataset\nand [ability to filter the peers table](https://github.com/ipfs/ipfs-webui/pull/2181).\n\n#### RPC `API.Authorizations`\n\nKubo RPC API now supports optional HTTP Authorization.\n\nGranular control over user access to the RPC can be defined in the\n[`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations)\nmap in the configuration file, allowing different users or apps to have unique\naccess secrets and allowed paths.\n\nThis feature is opt-in. By default, no authorization is set up.\nFor configuration instructions,\nrefer to the [documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations).\n\n#### MPLEX Removal\n\nAfter deprecating and removing mplex support by default in [v0.23.0](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.23.md#mplex-deprecation).\n\nWe now fully removed it. If you still need mplex support to talk with other pieces of software,\nplease try updating them, and if they don't support yamux or QUIC [talk to us about it](https://github.com/ipfs/kubo/issues/new/choose).\n\nMplex is unreliable by design, it will drop data and generate errors when sending data *too fast*,\nyamux and QUIC support backpressure, that means if we send data faster than the remote machine can process it, we slows down to match the remote's speed.\n\n#### Graphsync Experiment Removal\n\nCurrently the Graphsync server is to our knowledge not used\ndue to lack of compatible software.\nAnd we are left to have to maintain the go-graphsync implementation when trying\nto update Kubo because some dependency changed and it fails to build anymore.\n\nFor more information see https://github.com/ipfs/kubo/pull/9747.\n\n##### Commands `ipfs key sign` and `ipfs key verify`\n\nThis allows the Kubo node to sign arbitrary bytes to prove ownership of a PeerID or an IPNS Name. To avoid signature reuse, the signed payload is always prefixed with `libp2p-key signed message:`.\n\nThese commands are also both available through the RPC client and implemented in `client/rpc`.\n\nFor more information see https://github.com/ipfs/kubo/issues/10230.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - fix: allow daemon to start correctly if the API is null (#10062) ([ipfs/kubo#10062](https://github.com/ipfs/kubo/pull/10062))\n - chore: update version\n - feat: ipfs key sign|verify (#10235) ([ipfs/kubo#10235](https://github.com/ipfs/kubo/pull/10235))\n - docs(cli): fix spelling\n - feat: webui v4.2.0 (#10241) ([ipfs/kubo#10241](https://github.com/ipfs/kubo/pull/10241))\n - Migrate coreiface ([ipfs/kubo#10237](https://github.com/ipfs/kubo/pull/10237))\n - docs: clarify WebRTCDirect cannot reuse the same port as QUIC\n - libp2p: remove mplex\n - graphsync: remove support for the server\n - docs: move kubo-specific docs (#10226) ([ipfs/kubo#10226](https://github.com/ipfs/kubo/pull/10226))\n - feat(rpc): Opt-in HTTP RPC API Authorization (#10218) ([ipfs/kubo#10218](https://github.com/ipfs/kubo/pull/10218))\n - docs: clarify ipfs id agent version\n - fix: regression in 'ipfs dns'\n - docs(changelog): clarify webrtc in v0.24\n - chore: create next changelog\n - Merge Release: v0.24.0 ([ipfs/kubo#10209](https://github.com/ipfs/kubo/pull/10209))\n - fix: allow event emitting to happen in parallel with getting the query channel\n - fixes to routing put command (#10205) ([ipfs/kubo#10205](https://github.com/ipfs/kubo/pull/10205))\n - docs: fix accelerated-dht-client\n - docs/config: remove extra commas in PublicGateways example entries\n - docs: make it clear Web RTC Direct is experimental\n - feat: add WebRTC Direct support\n - docs: update EARLY_TESTERS.md (#10194) ([ipfs/kubo#10194](https://github.com/ipfs/kubo/pull/10194))\n - Update Version: v0.24 ([ipfs/kubo#10191](https://github.com/ipfs/kubo/pull/10191))\n- github.com/ipfs/boxo (v0.15.0 -> v0.16.0):\n - Release 0.16.0 ([ipfs/boxo#518](https://github.com/ipfs/boxo/pull/518))\n- github.com/libp2p/go-libp2p (v0.32.1 -> v0.32.2):\n - release v0.32.2\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Łukasz Magiera | 149 | +7833/-2505 | 375 |\n| Henrique Dias | 26 | +2498/-7535 | 210 |\n| Steven Allen | 48 | +497/-373 | 129 |\n| Jorropo | 9 | +247/-604 | 49 |\n| Michael Muré | 6 | +306/-79 | 14 |\n| Adin Schmahmann | 3 | +275/-8 | 5 |\n| Lucas Molas | 1 | +181/-56 | 2 |\n| Laurent Senta | 1 | +109/-24 | 7 |\n| Lars Gierth | 6 | +82/-18 | 8 |\n| Petar Maymounkov | 1 | +66/-32 | 3 |\n| web3-bot | 1 | +47/-42 | 17 |\n| Marcin Rataj | 6 | +57/-23 | 8 |\n| Kevin Atkinson | 5 | +31/-31 | 17 |\n| Marten Seemann | 3 | +27/-28 | 16 |\n| Hector Sanjuan | 3 | +28/-14 | 10 |\n| Overbool | 2 | +36/-3 | 3 |\n| Raúl Kripalani | 1 | +11/-12 | 4 |\n| hannahhoward | 2 | +11/-7 | 6 |\n| Jeromy Johnson | 5 | +9/-9 | 5 |\n| ForrestWeston | 1 | +14/-1 | 1 |\n| Russell Dempsey | 1 | +10/-2 | 2 |\n| Will Scott | 1 | +8/-1 | 1 |\n| Jeromy | 2 | +4/-4 | 2 |\n| sukun | 1 | +2/-2 | 1 |\n| Steve Loeppky | 1 | +2/-2 | 1 |\n| Jonas Keunecke | 1 | +2/-2 | 1 |\n| Edgar Lee | 1 | +3/-1 | 1 |\n| Dreamacro | 1 | +2/-2 | 2 |\n| godcong | 1 | +1/-1 | 1 |\n| Cole Brown | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.26.md", "content": "# Kubo changelog v0.26\n\n- [v0.26.0](#v0260)\n\n## v0.26.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Several deprecated commands have been removed](#several-deprecated-commands-have-been-removed)\n - [Support optional pin names](#support-optional-pin-names)\n - [`jaeger` trace exporter has been removed](#jaeger-trace-exporter-has-been-removed)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Kubo binary imports\n\nFor users of [Kubo preloaded plugins](https://github.com/ipfs/kubo/blob/master/docs/plugins.md#preloaded-plugins) there is now a way to create a kubo instance with your plugins by depending on the `cmd/ipfs/kubo` package rather than rebuilding kubo with the included plugins.\n\nSee the [customization docs](https://github.com/ipfs/kubo/blob/master/docs/customizing.md) for more information.\n\n#### Several deprecated commands have been removed\n\nSeveral deprecated commands have been removed:\n\n- `ipfs urlstore` deprecated in [April 2019, Kubo 0.4.21](https://github.com/ipfs/kubo/commit/8beaee63b3fa634c59b85179286ad3873921a535), use `ipfs add -q --nocopy --cid-version=1 {url}` instead.\n- `ipfs repo fsck` deprecated in [July 2019, Kubo 0.5.0](https://github.com/ipfs/kubo/commit/288a83ce7dcbf4a2498e06e4a95245bbb5e30f45)\n- `ipfs file` (and `ipfs file ls`) deprecated in [November 2020, Kubo 0.8.0](https://github.com/ipfs/kubo/commit/ec64dc5c396e7114590e15909384fabce0035482), use `ipfs ls` and `ipfs files ls` instead.\n- `ipfs dns` deprecated in [April 2022, Kubo 0.13](https://github.com/ipfs/kubo/commit/76ae33a9f3f9abd166d1f6f23d6a8a0511510e3c), use `ipfs resolve /ipns/{name}` instead.\n- `ipfs tar` deprecated [April 2022, Kubo 0.13](https://github.com/ipfs/kubo/pull/8849)\n\n#### Support optional pin names\n\nYou can now add a name to a pin when pinning a CID. To do so, use `ipfs pin add --name \"Some Name\" bafy...`. You can list your pins, including their names, with `ipfs pin ls --names`.\n\n#### `jaeger` trace exporter has been removed\n\n`jaeger` exporter has been removed from upstream, you should use `otlp` exporter instead.\nSee the [boxo tracing docs](https://github.com/ipfs/boxo/blob/a391d02102875ee7075a692076154bec1fa871f3/docs/tracing.md) for an example.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - chore: update version\n - feat(pinning): allow for overwriting pin name\n - chore: update otlp\n - Revert \"build,docker: add support for riscv64\"\n - feat: support optional pin names (#10261) ([ipfs/kubo#10261](https://github.com/ipfs/kubo/pull/10261))\n - build,docker: add support for riscv64\n - feat(cmd/ipfs): Make it possible to depend on cmd/ipfs/kubo for easier preloaded plugin management ([ipfs/kubo#10219](https://github.com/ipfs/kubo/pull/10219))\n - docs: fix broken link in HTTP RPC client doc (#10267) ([ipfs/kubo#10267](https://github.com/ipfs/kubo/pull/10267))\n - Merge Release: v0.25.0 [skip changelog] ([ipfs/kubo#10260](https://github.com/ipfs/kubo/pull/10260))\n - docs: add detail to NOpfs instructions in content-blocking.md\n - commands: remove several deprecated commands\n - fix: allow daemon to start correctly if the API is null (#10062) ([ipfs/kubo#10062](https://github.com/ipfs/kubo/pull/10062))\n - chore: update version\n- github.com/ipfs/boxo (v0.16.0 -> v0.17.0):\n - Release v0.17.0 ([ipfs/boxo#542](https://github.com/ipfs/boxo/pull/542))\n- github.com/ipfs/go-ipld-cbor (v0.0.6 -> v0.1.0):\n - v0.1.0 bump\n - chore: add or force update version.json\n - allow configuration of ipldStores default hash function ([ipfs/go-ipld-cbor#86](https://github.com/ipfs/go-ipld-cbor/pull/86))\n - sync: update CI config files (#85) ([ipfs/go-ipld-cbor#85](https://github.com/ipfs/go-ipld-cbor/pull/85))\n- github.com/ipfs/go-unixfsnode (v1.8.1 -> v1.9.0):\n - v1.9.0 bump\n - feat: expose ToDirEntryFrom to allow sub-dag representation\n - feat: new UnixFS{File,Directory} with options pattern\n - feat: testutil generator enhancements\n- github.com/ipld/go-car/v2 (v2.10.2-0.20230622090957-499d0c909d33 -> v2.13.1):\n - fix: BlockMetadata#Offset should be for section, not block data\n - fix: add closed check, expose storage.ErrClosed\n - fix: switch constructor args to match storage.New*, make roots plural\n - feat: add DeferredCarWriter\n - feat: fix BlockReader#SkipNext & add SourceOffset property\n - v0.6.2 ([ipld/go-car#464](https://github.com/ipld/go-car/pull/464))\n - fix: opt-in way to allow empty list of roots in CAR headers ([ipld/go-car#461](https://github.com/ipld/go-car/pull/461))\n- github.com/libp2p/go-libp2p-asn-util (v0.3.0 -> v0.4.1):\n - chore: release v0.4.1\n - fix: add Init method on backward compat\n - chore: release v0.4.0\n - rewrite representation to a sorted binary list and embed it\n - docs: fix incorrect markdown === in README\n - ci: run go generate on CI (#27) ([libp2p/go-libp2p-asn-util#27](https://github.com/libp2p/go-libp2p-asn-util/pull/27))\n- github.com/multiformats/go-multiaddr (v0.12.0 -> v0.12.1):\n - v0.12.1 bump\n - manet: reduce allocations in resolve unspecified address\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20230126041949-52956bd4c9aa -> v0.0.0-20240109153615-66e95c3e8a87):\n - Add a feature to preserve nil slices (#88) ([whyrusleeping/cbor-gen#88](https://github.com/whyrusleeping/cbor-gen/pull/88))\n - some cleanup for easier reading ([whyrusleeping/cbor-gen#89](https://github.com/whyrusleeping/cbor-gen/pull/89))\n - Support gen for map with value type `string` (#83) ([whyrusleeping/cbor-gen#83](https://github.com/whyrusleeping/cbor-gen/pull/83))\n - feat: add support for pointers to CIDs in slices (#86) ([whyrusleeping/cbor-gen#86](https://github.com/whyrusleeping/cbor-gen/pull/86))\n - optimize anything using WriteString ([whyrusleeping/cbor-gen#85](https://github.com/whyrusleeping/cbor-gen/pull/85))\n - Implement *bool support and support omitempty for slices ([whyrusleeping/cbor-gen#81](https://github.com/whyrusleeping/cbor-gen/pull/81))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Henrique Dias | 11 | +493/-1184 | 48 |\n| Łukasz Magiera | 3 | +610/-582 | 16 |\n| Rod Vagg | 11 | +1030/-151 | 18 |\n| whyrusleeping | 6 | +553/-388 | 14 |\n| Jorropo | 13 | +561/-348 | 84 |\n| Jeromy Johnson | 1 | +771/-48 | 6 |\n| Steven Allen | 2 | +264/-135 | 4 |\n| Forrest | 1 | +214/-0 | 5 |\n| Marcin Rataj | 1 | +89/-24 | 2 |\n| sukun | 1 | +31/-11 | 5 |\n| Will Scott | 3 | +25/-10 | 3 |\n| Adin Schmahmann | 3 | +21/-5 | 3 |\n| web3-bot | 2 | +8/-8 | 3 |\n| Marten Seemann | 1 | +13/-1 | 1 |\n| Bumblefudge | 1 | +5/-2 | 1 |\n| Will | 1 | +1/-1 | 1 |\n| Nicholas Ericksen | 1 | +1/-1 | 1 |\n| 0xbasar | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.27.md", "content": "# Kubo changelog v0.27\n\n- [v0.27.0](#v0270)\n\n## v0.27.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Gateway: support for `/api/v0` is deprecated](#gateway-support-for-apiv0-is-deprecated)\n - [IPNS resolver cache's TTL can now be configured](#ipns-resolver-caches-ttl-can-now-be-configured)\n - [RPC client: deprecated DHT API, added Routing API](#rpc-client-deprecated-dht-api-added-routing-api)\n - [Deprecated DHT commands removed from `/api/v0/dht`](#deprecated-dht-commands-removed-from-apiv0dht)\n - [Repository migrations are now trustless](#repository-migrations-are-now-trustless)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Gateway: support for `/api/v0` is deprecated\n\nSupport for exposing the legacy subset of Kubo RPC via the Gateway port is deprecated and should not be used. It will be removed in the next version. You can read more in .\n\nIf you have a legacy software that relies on this behavior, and want to expose parts of `/api/v0` next to `/ipfs`, use reverse-proxy in front of Kubo to mount both Gateway and RPC on the same port. NOTE: exposing RPC to the internet comes with security risk: make sure to specify access control via [API.Authorizations](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations).\n\n#### IPNS resolver cache's TTL can now be configured\n\nYou can now configure the upper-bound of a cached IPNS entry's Time-To-Live via [`Ipns.MaxCacheTTL`](https://github.com/ipfs/kubo/blob/master/docs/config.md#ipnsmaxcachettl).\n\n#### RPC client: deprecated DHT API, added Routing API\n\nThe RPC client for GO (`kubo/client/rpc`) now includes a Routing API to match the available commands in `/api/v0/routing`. In addition, the DHT API has been marked as deprecated.\n\nIn the next version, all DHT deprecated methods will be removed from the Go RPC client.\n\n#### Deprecated DHT commands removed from `/api/v0/dht`\n\nAll the DHT commands that were deprecated for over a year were finally removed from `/api/v0/dht`. Users should switch to modern `/api/v0/routing` which works with [both Amino DHT and Delegated Routers](https://github.com/ipfs/kubo/blob/master/docs/config.md#routing).\n\n#### Repository migrations are now trustless\n\nKubo now only uses [trustless requests](https://specs.ipfs.tech/http-gateways/trustless-gateway/) (e.g., CAR files) when downloading repository migrations via HTTP. This further strengthens Kubo by not delegating trust to public gateways. The migration binaries are locally verified before being executed. \n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - chore: update version\n - test: cleanup content blocking tests (#10360) ([ipfs/kubo#10360](https://github.com/ipfs/kubo/pull/10360))\n - docs: improve release issue template\n - chore: update version\n - repo/fsrepo/migrations: verified HTTP migrations (#10324) ([ipfs/kubo#10324](https://github.com/ipfs/kubo/pull/10324))\n - chore: fix link\n - docs: clarify Gateway.ExposeRoutingAPI (#10337) ([ipfs/kubo#10337](https://github.com/ipfs/kubo/pull/10337))\n - commands/add: return an error when using --only-hash and --to-files\n - docs(config): mention routing v1 spec\n - core/commands: remove 'ipfs dht' commands, except 'query' (#10328) ([ipfs/kubo#10328](https://github.com/ipfs/kubo/pull/10328))\n - core: deprecate CoreAPI.Dht, introduce CoreAPI.Routing\n - refactor: superfluous namespace test redirects (#10322) ([ipfs/kubo#10322](https://github.com/ipfs/kubo/pull/10322))\n - feat: add Ipns.MaxCacheTTL\n - fix(gw): negative entity-bytes beyond file size (#10320) ([ipfs/kubo#10320](https://github.com/ipfs/kubo/pull/10320))\n - core/corehttp: wrap gateway with headers, deprecate gateway /api/v0\n - docs: add changelog link to release issue template\n - docs: remove whizzzkid\n - chore: create next changelog\n - Merge Release: v0.26.0 [skip changelog] ([ipfs/kubo#10313](https://github.com/ipfs/kubo/pull/10313))\n - config: remove all options that are marked as REMOVED\n - chore: remove Gateway.APICommands\n - docs(cli): name inspect --verify (#10308) ([ipfs/kubo#10308](https://github.com/ipfs/kubo/pull/10308))\n - docs: improve release issue template (#10305) ([ipfs/kubo#10305](https://github.com/ipfs/kubo/pull/10305))\n - core/corehttp: wrap hostname option with otelhttp\n - fix: profiling tests\n - profile: add trace\n - docs(config): clarify ReproviderStrategy roots\n - chore: update version\n - docs: in RELEASE_ISSUE_TEMPLATE ask releaser to ensure we are using the latest go release on the major branch\n- github.com/ipfs/boxo (v0.17.0 -> v0.18.0):\n - Release v0.18.0 ([ipfs/boxo#581](https://github.com/ipfs/boxo/pull/581))\n- github.com/libp2p/go-libp2p (v0.32.2 -> v0.33.0):\n - release v0.33.0 (#2715) ([libp2p/go-libp2p#2715](https://github.com/libp2p/go-libp2p/pull/2715))\n - chore: update deps for v0.33 (#2713) ([libp2p/go-libp2p#2713](https://github.com/libp2p/go-libp2p/pull/2713))\n - webrtc: wait for FIN_ACK before closing data channels (#2615) ([libp2p/go-libp2p#2615](https://github.com/libp2p/go-libp2p/pull/2615))\n - quic: upgrade quic-go to v0.41.0 (#2710) ([libp2p/go-libp2p#2710](https://github.com/libp2p/go-libp2p/pull/2710))\n - chore: remove unused GenerateEKeyPair function (#2711) ([libp2p/go-libp2p#2711](https://github.com/libp2p/go-libp2p/pull/2711))\n - chore: drop support for go1.20 (#2708) ([libp2p/go-libp2p#2708](https://github.com/libp2p/go-libp2p/pull/2708))\n - chore: testify fix got, expected transpositions (#2666) ([libp2p/go-libp2p#2666](https://github.com/libp2p/go-libp2p/pull/2666))\n - docs: fix broken link in README\n - chore: fix typos (#2694) ([libp2p/go-libp2p#2694](https://github.com/libp2p/go-libp2p/pull/2694))\n - libp2phttp: fix flaky ExampleHost_listenOnHTTPTransportAndStreams (#2697) ([libp2p/go-libp2p#2697](https://github.com/libp2p/go-libp2p/pull/2697))\n - chore(p2p/host): fix typos (#2683) ([libp2p/go-libp2p#2683](https://github.com/libp2p/go-libp2p/pull/2683))\n - chore: fix typos (#2689) ([libp2p/go-libp2p#2689](https://github.com/libp2p/go-libp2p/pull/2689))\n - defaults: do TLS by default for encryption (#2650) ([libp2p/go-libp2p#2650](https://github.com/libp2p/go-libp2p/pull/2650))\n - webrtc: fix flaky TestMaxInFlightRequests (#2682) ([libp2p/go-libp2p#2682](https://github.com/libp2p/go-libp2p/pull/2682))\n - chore: remove unnecessary conversions (#2680) ([libp2p/go-libp2p#2680](https://github.com/libp2p/go-libp2p/pull/2680))\n - chore: update chat-with-mdns example readme (#2678) ([libp2p/go-libp2p#2678](https://github.com/libp2p/go-libp2p/pull/2678))\n - examples: call NewStream from only one side (#2677) ([libp2p/go-libp2p#2677](https://github.com/libp2p/go-libp2p/pull/2677))\n - chore: fix typos in comment (#2674) ([libp2p/go-libp2p#2674](https://github.com/libp2p/go-libp2p/pull/2674))\n - chore: update go-libp2p-asn-util (#2673) ([libp2p/go-libp2p#2673](https://github.com/libp2p/go-libp2p/pull/2673))\n - chore: update go security policy url (#2665) ([libp2p/go-libp2p#2665](https://github.com/libp2p/go-libp2p/pull/2665))\n - security: remove separate licenses for Noise and TLS (#2663) ([libp2p/go-libp2p#2663](https://github.com/libp2p/go-libp2p/pull/2663))\n - webrtc: clarify that there is no reuseport functionality (#2652) ([libp2p/go-libp2p#2652](https://github.com/libp2p/go-libp2p/pull/2652))\n - rcmgr: fix connmgr connection limit conflict warning (#2648) ([libp2p/go-libp2p#2648](https://github.com/libp2p/go-libp2p/pull/2648))\n - tcp: fix build on loong64 (#2655) ([libp2p/go-libp2p#2655](https://github.com/libp2p/go-libp2p/pull/2655))\n - swarm: fix grafana dashboard templating (#2640) ([libp2p/go-libp2p#2640](https://github.com/libp2p/go-libp2p/pull/2640))\n - chore: fix typos (#2608) ([libp2p/go-libp2p#2608](https://github.com/libp2p/go-libp2p/pull/2608))\n - chore: add resource manager dashboard to docker-compose (#2641) ([libp2p/go-libp2p#2641](https://github.com/libp2p/go-libp2p/pull/2641))\n - pstoremanager: fix race condition when removing peers from peer store (#2644) ([libp2p/go-libp2p#2644](https://github.com/libp2p/go-libp2p/pull/2644))\n - examples: remove unused 'SetStreamHandler' (#2598) ([libp2p/go-libp2p#2598](https://github.com/libp2p/go-libp2p/pull/2598))\n - Update docs from RSA to Ed25519 (#2606) ([libp2p/go-libp2p#2606](https://github.com/libp2p/go-libp2p/pull/2606))\n- github.com/multiformats/go-multiaddr (v0.12.1 -> v0.12.2):\n - chore: release v0.12.2\n - tests: add round trip equality check to fuzz (#232) ([multiformats/go-multiaddr#232](https://github.com/multiformats/go-multiaddr/pull/232))\n - fix: correctly parse ports as uint16 and explicitly fail on overflows (#228) ([multiformats/go-multiaddr#228](https://github.com/multiformats/go-multiaddr/pull/228))\n - replace custom random tests with testing.F (#227) ([multiformats/go-multiaddr#227](https://github.com/multiformats/go-multiaddr/pull/227))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Henrique Dias | 26 | +1668/-1484 | 96 |\n| Sukun | 13 | +983/-618 | 68 |\n| Jorropo | 18 | +501/-222 | 32 |\n| Marten Seemann | 2 | +17/-244 | 5 |\n| dozyio | 1 | +117/-132 | 31 |\n| Marcin Rataj | 7 | +100/-20 | 8 |\n| Alexandr Burdiyan | 2 | +29/-54 | 2 |\n| Tyler | 1 | +17/-19 | 2 |\n| KeienWang | 2 | +14/-14 | 12 |\n| Håvard Anda Estensen | 1 | +14/-14 | 11 |\n| Halimao | 2 | +17/-4 | 2 |\n| hannahhoward | 1 | +14/-6 | 2 |\n| alex | 1 | +8/-8 | 4 |\n| shuoer86 | 1 | +7/-7 | 5 |\n| John Chase | 1 | +0/-12 | 1 |\n| GoodDaisy | 1 | +5/-5 | 4 |\n| Michael Muré | 1 | +6/-2 | 1 |\n| 吴小白 | 1 | +3/-3 | 3 |\n| Vehorny | 1 | +3/-3 | 2 |\n| Eric | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.28.md", "content": "# Kubo changelog v0.28\n\n- [v0.28.0](#v0280)\n\n## v0.28.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [RPC client: removed deprecated DHT API](#rpc-client-removed-deprecated-dht-api)\n - [Gateway: `/api/v0` is removed](#gateway-apiv0-is-removed)\n - [Removed deprecated Object API commands](#removed-deprecated-object-api-commands)\n - [No longer publishes loopback and private addresses on DHT](#no-longer-publishes-loopback-and-private-addresses-on-dht)\n - [Pin roots are now prioritized when announcing](#pin-roots-are-now-prioritized-when-announcing)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n#### RPC client: removed deprecated DHT API\n\nThe deprecated DHT API commands in the RPC client have been removed. Instead, use the Routing API.\n\n#### Gateway: `/api/v0` is removed\n\nThe legacy subset of the Kubo RPC that was available via the Gateway port and was deprecated is now completely removed. You can read more in .\n\nIf you have a legacy software that relies on this behavior, and want to expose parts of `/api/v0` next to `/ipfs`, use reverse-proxy in front of Kubo to mount both Gateway and RPC on the same port. NOTE: exposing RPC to the internet comes with security risk: make sure to specify access control via [API.Authorizations](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations).\n\n#### Removed deprecated Object API commands\n\nThe Object API commands deprecated back in [2021](https://github.com/ipfs/kubo/issues/7936) have been removed, except for `object diff`, `object patch add-link` and `object patch rm-link`, whose alternatives have not yet been built (see issues [4801](https://github.com/ipfs/kubo/issues/4801) and [4782](https://github.com/ipfs/kubo/issues/4782)).\n\n##### Kubo ignores loopback addresses on LAN DHT and private addresses on WAN DHT\n\nKubo no longer keeps track of loopback and private addresses on the LAN and WAN DHTs, respectively. This means that other nodes will not try to dial likely undialable addresses.\n\nTo support testing scenarios where multiple Kubo instances run on the same machine, [`Routing.LoopbackAddressesOnLanDHT`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingloopbackaddressesonlandht) is set to `true` when the `test` profile is applied.\n\n#### Pin roots are now prioritized when announcing\n\nThe root CIDs of pinned content are now prioritized when announcing to the Amino DHT with [`Reprovider.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy) set to `all` (default) or `pinned`, making the important CIDs accessible faster.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: update version\n - chore: update version\n - core/node: prioritize announcing pin roots, and flat strategy (#10376) ([ipfs/kubo#10376](https://github.com/ipfs/kubo/pull/10376))\n - chore: webui v4.2.1 (#10391) ([ipfs/kubo#10391](https://github.com/ipfs/kubo/pull/10391))\n - docs(config): clarify RPC vs Gateway\n - chore: upgrade go-libp2p-kad-dht (#10378) ([ipfs/kubo#10378](https://github.com/ipfs/kubo/pull/10378))\n - chore(config): make Routing.AcceleratedDHTClient a Flag (#10384) ([ipfs/kubo#10384](https://github.com/ipfs/kubo/pull/10384))\n - fix: switch lowpower profile to autoclient\n - core: fix some typos (#10382) ([ipfs/kubo#10382](https://github.com/ipfs/kubo/pull/10382))\n - docs: fix some typos (#10377) ([ipfs/kubo#10377](https://github.com/ipfs/kubo/pull/10377))\n - core/commands!: remove deprecated object APIs (#10375) ([ipfs/kubo#10375](https://github.com/ipfs/kubo/pull/10375))\n - docs: update default ipns lifetime\n - coreapi/unixfs: don't create an additional IpfsNode for --only-hash\n - chore: cleanup old workaround (#10369) ([ipfs/kubo#10369](https://github.com/ipfs/kubo/pull/10369))\n - chore: finish reframe removal\n - docs: remove repetitive words (#10370) ([ipfs/kubo#10370](https://github.com/ipfs/kubo/pull/10370))\n - docs: updated links and refs to external resources (#10368) ([ipfs/kubo#10368](https://github.com/ipfs/kubo/pull/10368))\n - core/corehttp!: remove /api/v0 from gateway port\n - client/rpc!: remove deprecated DHT commands\n - ci: upgrade to go 1.22 (#10355) ([ipfs/kubo#10355](https://github.com/ipfs/kubo/pull/10355))\n - chore: create next changelog\n - Merge Release: v0.27.0 [skip changelog] ([ipfs/kubo#10362](https://github.com/ipfs/kubo/pull/10362))\n - test: cleanup content blocking tests (#10360) ([ipfs/kubo#10360](https://github.com/ipfs/kubo/pull/10360))\n - docs: improve release issue template\n - chore: update version\n- github.com/ipfs/boxo (v0.18.0 -> v0.19.0):\n - Release v0.19.0 ([ipfs/boxo#598](https://github.com/ipfs/boxo/pull/598))\n- github.com/libp2p/go-libp2p (v0.33.0 -> v0.33.2):\n - chore: release v0.33.2 (#2755) ([libp2p/go-libp2p#2755](https://github.com/libp2p/go-libp2p/pull/2755))\n - Update quic-go to v0.42.0. Release v0.33.1 (#2741) ([libp2p/go-libp2p#2741](https://github.com/libp2p/go-libp2p/pull/2741))\n- github.com/libp2p/go-libp2p-kad-dht (v0.24.4 -> v0.25.2):\n - chore: release v0.25.2 ([libp2p/go-libp2p-kad-dht#961](https://github.com/libp2p/go-libp2p-kad-dht/pull/961))\n - add ctx canceled err check ([libp2p/go-libp2p-kad-dht#960](https://github.com/libp2p/go-libp2p-kad-dht/pull/960))\n - chore: release v0.25.1\n - perf: don't buffer the output of FindProvidersAsync\n - chore: use go-libp2p-routing-helpers for tracing needs\n - fix: properly iterate in tracing for protocol messenger\n - fix: apply addrFilters in the dht (#872) ([libp2p/go-libp2p-kad-dht#872](https://github.com/libp2p/go-libp2p-kad-dht/pull/872))\n - Add provider record addresses to peerstore ([libp2p/go-libp2p-kad-dht#870](https://github.com/libp2p/go-libp2p-kad-dht/pull/870))\n - chore: release v0.25.0\n - tracing: add protocol messages client tracing\n - Enhance handleNewMessage Server Mode Logging: Convert Error Logs to Debug Level ([libp2p/go-libp2p-kad-dht#860](https://github.com/libp2p/go-libp2p-kad-dht/pull/860))\n - tracing: fix DHT keys as string attribute not being valid utf-8 ([libp2p/go-libp2p-kad-dht#859](https://github.com/libp2p/go-libp2p-kad-dht/pull/859))\n - merge: fix: issues discovered in kubo v0.21.0-rc2 (#853) ([libp2p/go-libp2p-kad-dht#853](https://github.com/libp2p/go-libp2p-kad-dht/pull/853))\n - merge: fix: issues discovered in kubo v0.21.0-rc1 (#851) ([libp2p/go-libp2p-kad-dht#851](https://github.com/libp2p/go-libp2p-kad-dht/pull/851))\n - Release v0.24.0 ([libp2p/go-libp2p-kad-dht#844](https://github.com/libp2p/go-libp2p-kad-dht/pull/844))\n - fix: don't add unresponsive DHT servers to the Routing Table (#820) ([libp2p/go-libp2p-kad-dht#820](https://github.com/libp2p/go-libp2p-kad-dht/pull/820))\n - filter local addresses (for WAN) and localhost addresses (for LAN) ([libp2p/go-libp2p-kad-dht#839](https://github.com/libp2p/go-libp2p-kad-dht/pull/839))\n- github.com/multiformats/go-multiaddr (v0.12.2 -> v0.12.3):\n - chore: release v0.12.3 ([multiformats/go-multiaddr#240](https://github.com/multiformats/go-multiaddr/pull/240))\n - chore: Expand comment ForEach ([multiformats/go-multiaddr#238](https://github.com/multiformats/go-multiaddr/pull/238))\n - .Decapsulate by Components ([multiformats/go-multiaddr#239](https://github.com/multiformats/go-multiaddr/pull/239))\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20240109153615-66e95c3e8a87 -> v0.1.0):\n - Nullable ints (#93) ([whyrusleeping/cbor-gen#93](https://github.com/whyrusleeping/cbor-gen/pull/93))\n - Introduce Gen{} struct for configurability ([whyrusleeping/cbor-gen#94](https://github.com/whyrusleeping/cbor-gen/pull/94))\n - Transparent encoding ([whyrusleeping/cbor-gen#91](https://github.com/whyrusleeping/cbor-gen/pull/91))\n - turn max length consts into global vars ([whyrusleeping/cbor-gen#92](https://github.com/whyrusleeping/cbor-gen/pull/92))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Henrique Dias | 19 | +867/-2806 | 96 |\n| Rod Vagg | 7 | +921/-475 | 25 |\n| Marcin Rataj | 8 | +358/-344 | 18 |\n| Guillaume Michel - guissou | 1 | +145/-485 | 13 |\n| Jorropo | 8 | +429/-136 | 22 |\n| Łukasz Magiera | 4 | +284/-48 | 11 |\n| whyrusleeping | 1 | +90/-90 | 2 |\n| Michael Muré | 2 | +48/-73 | 9 |\n| Marco Munizaga | 6 | +86/-29 | 10 |\n| guillaumemichel | 3 | +93/-1 | 3 |\n| Marten Seemann | 1 | +31/-4 | 4 |\n| godeamon | 3 | +11/-8 | 3 |\n| shuangcui | 1 | +6/-6 | 5 |\n| occupyhabit | 1 | +3/-3 | 3 |\n| crazehang | 1 | +2/-2 | 1 |\n| Dennis Trautwein | 1 | +1/-2 | 1 |\n| “GheisMohammadi” | 1 | +1/-1 | 1 |\n| web3-bot | 1 | +2/-0 | 1 |\n| Daniel Norman | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.29.md", "content": "# Kubo changelog v0.29\n\n- [v0.29.0](#v0290)\n\n## v0.29.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Add search functionality for pin names](#add-search-functionality-for-pin-names)\n - [Customizing `ipfs add` defaults](#customizing-ipfs-add-defaults)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Add search functionality for pin names\n\nIt is now possible to search for pins by name via `ipfs pin ls --name \"SomeName\"`.\nThe search is case-sensitive and will return all pins that contain the specified substring in their name.\n\n> [!TIP]\n> The `ipfs pin ls -n` is now a shorthand for `ipfs pin ls --name`, mirroring the behavior of `ipfs pin add`.\n> See `ipfs pin ls --help` for more information.\n\n#### Customizing `ipfs add` defaults\n\nThis release supports overriding global data ingestion defaults used by commands like `ipfs add` via user-defined [`Import.*` configuration options](../config.md#import).\nThe hash function, CID version, or UnixFS raw leaves and chunker behaviors can be set once, and used as the new implicit default for `ipfs add`.\n\n> [!TIP]\n> As a convenience, two CID [profiles](../config.md#profile) are provided: `legacy-cid-v0` and `test-cid-v1`.\n> A test profile that defaults to modern CIDv1 can be applied via `ipfs config profile apply test-cid-v1`.\n> We encourage users to try it and report any issues in [kubo#4143](https://github.com/ipfs/kubo/issues/4143).\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix(cli): unify --name param in ls and add (#10439) ([ipfs/kubo#10439](https://github.com/ipfs/kubo/pull/10439))\n - chore: set version to 0.29.0-rc2\n - fix(libp2p): streams config validation in resource manager (#10435) ([ipfs/kubo#10435](https://github.com/ipfs/kubo/pull/10435))\n - chore: update version\n - chore: libp2p 0.34.1 (#10429) ([ipfs/kubo#10429](https://github.com/ipfs/kubo/pull/10429))\n - refactor: stop using github.com/pkg/errors (#10431) ([ipfs/kubo#10431](https://github.com/ipfs/kubo/pull/10431))\n - chore: fix --help text\n - config: introduce Import section (#10421) ([ipfs/kubo#10421](https://github.com/ipfs/kubo/pull/10421))\n - feat: enables searching pins by name (#10412) ([ipfs/kubo#10412](https://github.com/ipfs/kubo/pull/10412))\n - fix(fuse): ipfs path parsing (#10243) ([ipfs/kubo#10243](https://github.com/ipfs/kubo/pull/10243))\n - core/node: fix divide by zero fatal crash for reprovide rate check (#10411) ([ipfs/kubo#10411](https://github.com/ipfs/kubo/pull/10411))\n - chore: bump to go-ipfs-cmds @ v0.11\n - chore: create next changelog\n - Merge Release: v0.28.0 [skip changelog] ([ipfs/kubo#10402](https://github.com/ipfs/kubo/pull/10402))\n - docs: update release checklist (#10401) ([ipfs/kubo#10401](https://github.com/ipfs/kubo/pull/10401))\n - chore: update version\n- github.com/ipfs/boxo (v0.19.0 -> v0.20.0):\n - Release v0.20.0 ([ipfs/boxo#613](https://github.com/ipfs/boxo/pull/613))\n- github.com/ipfs/go-blockservice (v0.5.0 -> v0.5.2):\n - docs: remove contribution section\n - chore: bump version\n - chore: deprecate types and readme\n - chore: release v0.5.1\n - fix: remove busyloop in getBlocks by removing batching\n- github.com/ipfs/go-ipfs-blockstore (v1.3.0 -> v1.3.1):\n - docs: remove contribution section\n - chore: bump version\n - chore: deprecate types and readme\n- github.com/ipfs/go-ipfs-cmds (v0.10.0 -> v0.11.0):\n - chore: release v0.11.0 (#253) ([ipfs/go-ipfs-cmds#253](https://github.com/ipfs/go-ipfs-cmds/pull/253))\n - chore: update deps (#252) ([ipfs/go-ipfs-cmds#252](https://github.com/ipfs/go-ipfs-cmds/pull/252))\n - chore: release 0.10.2 (#251) ([ipfs/go-ipfs-cmds#251](https://github.com/ipfs/go-ipfs-cmds/pull/251))\n - fix(http): return error in case of panic (#250) ([ipfs/go-ipfs-cmds#250](https://github.com/ipfs/go-ipfs-cmds/pull/250))\n - chore: release v0.10.1\n- github.com/ipfs/go-ipfs-ds-help (v1.1.0 -> v1.1.1):\n - docs: remove contribution section\n - chore: bump version\n - chore: deprecate types and readme\n- github.com/ipfs/go-ipfs-exchange-interface (v0.2.0 -> v0.2.1):\n - chore: bump version\n - Deprecate types and readme (#29) ([ipfs/go-ipfs-exchange-interface#29](https://github.com/ipfs/go-ipfs-exchange-interface/pull/29))\n - docs: Add proper documentation to the interface.\n- github.com/ipfs/go-verifcid (v0.0.2 -> v0.0.3):\n - chore: bump version\n - chore: deprecate types and readme\n - Make poseidon hashes good hashes ([ipfs/go-verifcid#19](https://github.com/ipfs/go-verifcid/pull/19))\n - sync: update CI config files (#18) ([ipfs/go-verifcid#18](https://github.com/ipfs/go-verifcid/pull/18))\n- github.com/ipld/go-car (v0.5.0 -> v0.6.2):\n - v0.6.2 ([ipld/go-car#464](https://github.com/ipld/go-car/pull/464))\n - fix: opt-in way to allow empty list of roots in CAR headers ([ipld/go-car#461](https://github.com/ipld/go-car/pull/461))\n - feat: add inverse and version to filter cmd ([ipld/go-car#457](https://github.com/ipld/go-car/pull/457))\n - v0.6.1 bump\n - chore: update usage of merkledag by go-car (#437) ([ipld/go-car#437](https://github.com/ipld/go-car/pull/437))\n - feat(cmd/car): add '--no-wrap' option to 'create' command ([ipld/go-car#432](https://github.com/ipld/go-car/pull/432))\n - fix: remove github.com/ipfs/go-ipfs-blockstore dependency\n - feat: expose index for StorageCar\n - perf: reduce NewCarReader allocations\n - fix(deps): update deps for cmd (use master go-car and go-car/v2 for now)\n - fix: new error strings from go-cid\n - fix: tests should match stderr for verbose output\n - fix: reading from stdin should broadcast EOF to block loaders\n - refactor insertion index to be publicly accessible ([ipld/go-car#408](https://github.com/ipld/go-car/pull/408))\n - chore: unmigrate from go-libipfs\n - Create CODEOWNERS\n - blockstore: give a direct access to the index for read operations\n - blockstore: only close the file on error in OpenReadWrite, not OpenReadWriteFile\n - fix: handle (and test) WholeCID vs not; fast Has() path for storage\n - ReadWrite: faster Has() by using the in-memory index instead of reading on disk\n - fix: let `extract` skip missing unixfs shard links\n - fix: error when no files extracted\n - fix: make -f optional, read from stdin if omitted\n - fix: update cmd/car/README with latest description\n - chore: add test cases for extract modes\n - feat: extract accepts '-' as an output path for stdout\n - feat: extract specific path, accept stdin as streaming input\n - fix: if we don't read the full block data, don't error on !EOF\n - blockstore: try to close during Finalize(), even in case of previous error\n - ReadWrite: add an alternative FinalizeReadOnly+Close flow\n - feat: add WithTrustedCar() reader option (#381) ([ipld/go-car#381](https://github.com/ipld/go-car/pull/381))\n - blockstore: fast path for AllKeysChan using the index\n - fix: switch to crypto/rand.Read\n - stop using the deprecated io/ioutil package\n - fix(doc): fix storage package doc formatting\n - fix: return errors for unsupported operations\n - chore: move insertionindex into store pkg\n - chore: add experimental note\n - fix: minor lint & windows fd test problems\n - feat: docs for StorageCar interfaces\n - feat: ReadableWritable; dedupe shared code\n - feat: add Writable functionality to StorageCar\n - feat: StorageCar as a Readable storage, separate from blockstore\n - feat(blockstore): implement a streaming read only storage\n - feat(cmd): add index create subcommand to create an external carv2 index ([ipld/go-car#350](https://github.com/ipld/go-car/pull/350))\n - chore: bump version to 0.6.0\n - fix: use goreleaser instead\n - Allow using WalkOption in WriteCar function ([ipld/go-car#357](https://github.com/ipld/go-car/pull/357))\n - fix: update go-block-format to the version that includes the stubs\n - feat: upgrade from go-block-format to go-libipfs/blocks\n - cleanup readme a bit to make the cli more discoverable (#353) ([ipld/go-car#353](https://github.com/ipld/go-car/pull/353))\n - Update install instructions in README.md\n - Add a debugging form for car files. (#341) ([ipld/go-car#341](https://github.com/ipld/go-car/pull/341))\n - ([ipld/go-car#340](https://github.com/ipld/go-car/pull/340))\n - add a `SkipNext` method on block reader (#338) ([ipld/go-car#338](https://github.com/ipld/go-car/pull/338))\n - feat: Has() and Get() will respect StoreIdentityCIDs option\n- github.com/libp2p/go-libp2p (v0.33.2 -> v0.34.1):\n - release v0.34.1 (#2811) ([libp2p/go-libp2p#2811](https://github.com/libp2p/go-libp2p/pull/2811))\n - config: fix Insecure security constructor (#2810) ([libp2p/go-libp2p#2810](https://github.com/libp2p/go-libp2p/pull/2810))\n - rcmgr: Backwards compatibility if you wrap default impl (#2805) ([libp2p/go-libp2p#2805](https://github.com/libp2p/go-libp2p/pull/2805))\n - v0.34.0 (#2795) ([libp2p/go-libp2p#2795](https://github.com/libp2p/go-libp2p/pull/2795))\n - swarm: fix addr for TestBlackHoledAddrBlocked (#2803) ([libp2p/go-libp2p#2803](https://github.com/libp2p/go-libp2p/pull/2803))\n - Add backwards compatibility with old well-known resource (#2798) ([libp2p/go-libp2p#2798](https://github.com/libp2p/go-libp2p/pull/2798))\n - rcmgr: remove a connection only once from the limiter (#2800) ([libp2p/go-libp2p#2800](https://github.com/libp2p/go-libp2p/pull/2800))\n - Adhere to request.Context when roundtripping on a stream (#2796) ([libp2p/go-libp2p#2796](https://github.com/libp2p/go-libp2p/pull/2796))\n - fix: Set missing deadlines (#2794) ([libp2p/go-libp2p#2794](https://github.com/libp2p/go-libp2p/pull/2794))\n - rcmgr: Add conn_limiter to limit number of conns per ip cidr (#2788) ([libp2p/go-libp2p#2788](https://github.com/libp2p/go-libp2p/pull/2788))\n - identify: refactor observed address manager to do address mapping at thin waist(IP+TCP/UDP) layer (#2793) ([libp2p/go-libp2p#2793](https://github.com/libp2p/go-libp2p/pull/2793))\n - fix: DNS protocol address is not reserved (#2792) ([libp2p/go-libp2p#2792](https://github.com/libp2p/go-libp2p/pull/2792))\n - Update github.com/quic-go/quic-go dependency (#2780) ([libp2p/go-libp2p#2780](https://github.com/libp2p/go-libp2p/pull/2780))\n - webrtc: add webrtc addresses to host normalizer (#2784) ([libp2p/go-libp2p#2784](https://github.com/libp2p/go-libp2p/pull/2784))\n - Add a \"Limited\" network connectivity state (#2696) ([libp2p/go-libp2p#2696](https://github.com/libp2p/go-libp2p/pull/2696))\n - basichost: append certhash for webrtc addresses provided via address factory (#2774) ([libp2p/go-libp2p#2774](https://github.com/libp2p/go-libp2p/pull/2774))\n - Fix comment (#2775) ([libp2p/go-libp2p#2775](https://github.com/libp2p/go-libp2p/pull/2775))\n - Update: update incomplete readmes (#2767) ([libp2p/go-libp2p#2767](https://github.com/libp2p/go-libp2p/pull/2767))\n - libp2phttp: Return connection: close when doing http over streams (#2756) ([libp2p/go-libp2p#2756](https://github.com/libp2p/go-libp2p/pull/2756))\n - Identify: emit useful events after identification (#2759) ([libp2p/go-libp2p#2759](https://github.com/libp2p/go-libp2p/pull/2759))\n - Update chat with rendezvous example (#2769) ([libp2p/go-libp2p#2769](https://github.com/libp2p/go-libp2p/pull/2769))\n - Rename well-known resource (#2757) ([libp2p/go-libp2p#2757](https://github.com/libp2p/go-libp2p/pull/2757))\n - quic: make server cmd use RFC 9000 instead of draft-29 (#2753) ([libp2p/go-libp2p#2753](https://github.com/libp2p/go-libp2p/pull/2753))\n - autonat: Clean up after close (#2749) ([libp2p/go-libp2p#2749](https://github.com/libp2p/go-libp2p/pull/2749))\n - webrtc: run onDone callback immediately on close (#2729) ([libp2p/go-libp2p#2729](https://github.com/libp2p/go-libp2p/pull/2729))\n - fix: add NullResourceManager to webrtc, fixes panic (#2752) ([libp2p/go-libp2p#2752](https://github.com/libp2p/go-libp2p/pull/2752))\n - feat: add tls KeyLogWriter option (#2750) ([libp2p/go-libp2p#2750](https://github.com/libp2p/go-libp2p/pull/2750))\n - Use any port, not a specific one for examples (#2748) ([libp2p/go-libp2p#2748](https://github.com/libp2p/go-libp2p/pull/2748))\n - quicreuse: remove workaround for quic-go listener close deadlock (#2746) ([libp2p/go-libp2p#2746](https://github.com/libp2p/go-libp2p/pull/2746))\n - use Fx to start and stop the host, swarm, autorelay and quicreuse (#2118) ([libp2p/go-libp2p#2118](https://github.com/libp2p/go-libp2p/pull/2118))\n - webrtc: set sctp receive buffer size to 100kB (#2745) ([libp2p/go-libp2p#2745](https://github.com/libp2p/go-libp2p/pull/2745))\n - basichost: log more info when protocol selection fails (#2734) ([libp2p/go-libp2p#2734](https://github.com/libp2p/go-libp2p/pull/2734))\n - chore: bump quic-go (#2742) ([libp2p/go-libp2p#2742](https://github.com/libp2p/go-libp2p/pull/2742))\n - security: remove unnecessary noise code (#2738) ([libp2p/go-libp2p#2738](https://github.com/libp2p/go-libp2p/pull/2738))\n - webrtc: increase receive buffer size on listener (#2730) ([libp2p/go-libp2p#2730](https://github.com/libp2p/go-libp2p/pull/2730))\n - webrtc: fix bug with logger wrapper (#2727) ([libp2p/go-libp2p#2727](https://github.com/libp2p/go-libp2p/pull/2727))\n - dcutr: fix log format to actually print error (#2725) ([libp2p/go-libp2p#2725](https://github.com/libp2p/go-libp2p/pull/2725))\n - webrtc: use a common logger for all pion logging (#2718) ([libp2p/go-libp2p#2718](https://github.com/libp2p/go-libp2p/pull/2718))\n - chore: remove unreadable code, move a test function to test code, better locking in webrtc control reader\n - ping: use context.Afterfunc to avoid a lingering goroutine (#2723) ([libp2p/go-libp2p#2723](https://github.com/libp2p/go-libp2p/pull/2723))\n - webrtc: close mux when closing listener (#2717) ([libp2p/go-libp2p#2717](https://github.com/libp2p/go-libp2p/pull/2717))\n - webrtc: setup datachannel handlers before connecting to a peer (#2716) ([libp2p/go-libp2p#2716](https://github.com/libp2p/go-libp2p/pull/2716))\n- github.com/libp2p/go-libp2p-pubsub (v0.10.0 -> v0.11.0):\n - Fix: Own our CertifiedAddrBook (#555) ([libp2p/go-libp2p-pubsub#555](https://github.com/libp2p/go-libp2p-pubsub/pull/555))\n - chores: bump go-libp2p (#558) ([libp2p/go-libp2p-pubsub#558](https://github.com/libp2p/go-libp2p-pubsub/pull/558))\n - fix: Don't bother parsing an empty slice (#556) ([libp2p/go-libp2p-pubsub#556](https://github.com/libp2p/go-libp2p-pubsub/pull/556))\n - Replace fragmentRPC with appendOrMergeRPC (#557) ([libp2p/go-libp2p-pubsub#557](https://github.com/libp2p/go-libp2p-pubsub/pull/557))\n- github.com/multiformats/go-multiaddr (v0.12.3 -> v0.12.4):\n - Release v0.12.4 ([multiformats/go-multiaddr#245](https://github.com/multiformats/go-multiaddr/pull/245))\n - net: restrict unicast ip6 public address space (#235) ([multiformats/go-multiaddr#235](https://github.com/multiformats/go-multiaddr/pull/235))\n- github.com/whyrusleeping/cbor-gen (v0.1.0 -> v0.1.1):\n - fix: reduce memory held by deferred objects (#96) ([whyrusleeping/cbor-gen#96](https://github.com/whyrusleeping/cbor-gen/pull/96))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Henrique Dias | 33 | +4994/-579 | 115 |\n| Rod Vagg | 29 | +3781/-1367 | 90 |\n| sukun | 12 | +2026/-1215 | 39 |\n| Marco Munizaga | 18 | +1482/-382 | 47 |\n| Will | 5 | +769/-213 | 17 |\n| Steven Allen | 5 | +540/-115 | 24 |\n| Sukun | 4 | +274/-194 | 11 |\n| Michael Muré | 7 | +372/-55 | 16 |\n| Marten Seemann | 1 | +243/-141 | 10 |\n| Marcin Rataj | 7 | +244/-134 | 13 |\n| hannahhoward | 1 | +277/-0 | 2 |\n| Will Scott | 5 | +54/-38 | 9 |\n| Hector Sanjuan | 3 | +68/-20 | 5 |\n| Jorropo | 5 | +34/-47 | 15 |\n| Andrew Gillis | 2 | +67/-7 | 3 |\n| IGP | 1 | +59/-8 | 5 |\n| Adin Schmahmann | 2 | +50/-0 | 3 |\n| Laurent Senta | 1 | +40/-4 | 2 |\n| Brad Fitzpatrick | 1 | +42/-2 | 2 |\n| Fabio Bozzo | 1 | +36/-1 | 3 |\n| Yolan Romailler | 1 | +15/-19 | 4 |\n| Hlib Kanunnikov | 2 | +14/-14 | 6 |\n| Andreas Penzkofer | 1 | +22/-2 | 3 |\n| Matthias Fasching | 1 | +8/-10 | 1 |\n| gopherfarm | 2 | +16/-1 | 2 |\n| Dreamacro | 1 | +1/-10 | 1 |\n| web3-bot | 2 | +7/-3 | 4 |\n| Rafał Leszko | 1 | +4/-4 | 1 |\n| Oleg Kovalov | 1 | +4/-4 | 3 |\n| dbeal | 1 | +5/-1 | 1 |\n| Antonio Navarro Perez | 1 | +4/-1 | 1 |\n| dozyio | 1 | +3/-0 | 1 |\n| zhiqiangxu | 1 | +1/-1 | 1 |\n| the harder the luckier | 1 | +1/-1 | 1 |\n| Lukáš Lukáč | 1 | +1/-1 | 1 |\n| Steve Loeppky | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.3.md", "content": "# go-ipfs changelog 2015\n\n## v0.3.11 - 2016-01-12\n\nThis is the final ipfs version before the transition to v0.4.0.\nIt introduces a few stability improvements, bugfixes, and increased\ntest coverage.\n\n* Features\n * Add 'get' and 'patch' to the allowed gateway commands (@whyrusleeping)\n * Updated webui version (@dignifiedquire)\n\n* BugFixes\n * Fix path parsing for add command (@djdv)\n * namesys: Make paths with multiple segments work. Fixes #2059 (@Kubuxu)\n * Fix up panic catching in http handler funcs (@whyrusleeping)\n * Add correct access control headers to the default api config (@dignifiedquire)\n * Fix closenotify by not sending empty file set (@whyrusleeping)\n\n* Tool Changes\n * Have install.sh use the full path to ipfs binary if detected (@jedahan)\n * Install daemon system-wide if on El Capitan (@jedahan)\n * makefile: add -ldflags to install and nofuse tasks (@lgierth)\n\n* General Codebase\n * Clean up http client code (@whyrusleeping)\n * Move api version check to header (@rht)\n\n* Documentation\n * Improved release checklist (@jbenet)\n * Added quotes around command in long description (@RichardLitt)\n * Added a shutdown note to daemon description (@RichardLitt)\n\n* Testing\n * t0080: improve last tests (@chriscool)\n * t0080: improve 'ipfs refs --unique' test (@chriscool)\n * Fix t.Fatal usage in goroutines (@chriscool)\n * Add docker testing support to sharness (@chriscool)\n * sharness: add t0300-docker-image.sh (@chriscool)\n * Included more namesys tests. (@Kubuxu)\n * Add sharness test to verify requests look good (@whyrusleeping)\n * Re-enable ipns sharness test now that iptb is fixed (@whyrusleeping)\n * Force use of ipv4 in test (@whyrusleeping)\n * Travis-CI: use go 1.5.2 (@jbenet)\n\n## v0.3.10 - 2015-12-07\n\nThis patch update introduces the 'ipfs update' command which will be used for\nfuture ipfs updates along with a few other bug fixes and documentation\nimprovements.\n\n\n* Features\n * support for 'ipfs update' to call external binary (@whyrusleeping)\n * cache ipns entries to speed things up a little (@whyrusleeping)\n * add option to version command to print repo version (@whyrusleeping)\n * Add in some more notifications to help profile queries (@whyrusleeping)\n * gateway: add path prefix for directory listings (@lgierth)\n * gateway: add CurrentCommit to /version (@lgierth)\n\n* BugFixes\n * set data and links nil if not present (@whyrusleeping)\n * fix log hanging issue, and implement close-notify for commands (@whyrusleeping)\n * fix dial backoff (@whyrusleeping)\n * proper ndjson implementation (@whyrusleeping)\n * seccat: fix secio context (@lgierth)\n * Add newline to end of the output for a few commands. (@nham)\n * Add fixed period repo GC + test (@rht)\n\n* Tool Changes\n * Allow `ipfs cat` on ipns path (@rht)\n\n* General Codebase\n * rewrite of backoff mechanism (@whyrusleeping)\n * refactor net code to use transports, in rough accordance with libp2p (@whyrusleeping)\n * disable building fuse stuff on windows (@whyrusleeping)\n * repo: remove Log config (@lgierth)\n * commands: fix description of --api (@lgierth)\n\n* Documentation\n * --help: Add a note on using IPFS_PATH to the footer of the helptext. (@sahib)\n * Moved email juan to ipfs/contribute (@richardlitt)\n * Added commit sign off section (@richardlitt)\n * Added a security section (@richardlitt)\n * Moved TODO doc to issue #1929 (@richardlitt)\n\n* Testing\n * gateway: add tests for /version (@lgierth)\n * Add gc auto test (@rht)\n * t0020: cleanup dir with bad perms (@chriscool)\n\nNote: this commit introduces fixed-period repo gc, which will trigger gc\nafter a fixed period of time. This feature is introduced now, disabled by\ndefault, and can be enabled with `ipfs daemon --enable-gc`. If all goes well,\nin the future, it will be enabled by default.\n\n## v0.3.9 - 2015-10-30\n\nThis patch update includes a good number of bugfixes, notably, it fixes\nbuilds on windows, and puts newlines between streaming json objects for a\nproper ndjson format.\n\n* Features\n * Writable gateway enabled again (@cryptix)\n\n* Bugfixes\n * fix windows builds (@whyrusleeping)\n * content type on command responses default to text (@whyrusleeping)\n * add check to makefile to ensure windows builds don't fail silently (@whyrusleeping)\n * put newlines between streaming json output objects (@whyrusleeping)\n * fix streaming output to flush per write (@whyrusleeping)\n * purposely fail builds pre go1.5 (@whyrusleeping)\n * fix ipfs id (@whyrusleeping)\n * fix a few race conditions in mocknet (@whyrusleeping)\n * fix makefile failing when not in a git repo (@whyrusleeping)\n * fix cli flag orders (long, short) (@rht)\n * fix races in http cors (@miolini)\n * small webui update (some bugfixes) (@jbenet)\n\n* Tool Changes\n * make swarm connect return an error when it fails (@whyrusleeping)\n * Add short flag for `ipfs ls --headers` (v for verbose) (@rht)\n\n* General Codebase\n * bitswap: clean log printf and humanize dup data count (@cryptix)\n * config: update pluto's peerID (@lgierth)\n * config: update bootstrap list hostname (@lgierth)\n\n* Documentation\n * Pared down contribute to link to new go guidelines (@richardlitt)\n\n* Testing\n * t0010: add tests for 'ipfs commands --flags' (@chriscool)\n * ipns_test: fix namesys.NewNameSystem() call (@chriscool)\n * t0060: fail if no nc (@chriscool)\n\n## v0.3.8 - 2015-10-09\n\nThis patch update includes changes to make ipns more consistent and reliable,\nsymlink support in unixfs, mild performance improvements, new tooling features,\na plethora of bugfixes, and greatly improved tests.\n\nNOTICE: Version 0.3.8 also requires golang version 1.5.1 or higher.\n\n* Bugfixes\n * refactor ipns to be more consistent and reliable (@whyrusleeping)\n * fix 'ipfs refs' json output (@whyrusleeping)\n * fix setting null config maps (@rht)\n * fix output of dht commands (@whyrusleeping)\n * fix NAT spam dialing (@whyrusleeping)\n * fix random panics on 32 bit systems (@whyrusleeping)\n * limit total number of network fd's (@whyrusleeping)\n * fix http api content type (@WeMeetAgain)\n * fix writing of api file for port zero daemons (@whyrusleeping)\n * windows connection refused fixes (@mjanczyk)\n * use go1.5's built in trailers, no more failures (@whyrusleeping)\n * fix random bitswap hangs (@whyrusleeping)\n * rate limit fd usage (@whyrusleeping)\n * fix panic in bitswap rate limiting (@whyrusleeping)\n\n* Tool Changes\n * --empty-repo option for init (@prusnak)\n * implement symlinks (@whyrusleeping)\n * improve cmds lib files processing (@rht)\n * properly return errors through commands (@whyrusleeping)\n * bitswap unwant command (@whyrusleeping)\n * tar add/cat commands (@whyrusleeping)\n * fix gzip compression in get (@klauspost)\n * bitswap stat logs wasted bytes (@whyrusleeping)\n * resolve command now uses core.Resolve (@rht)\n * add `--local` flag to 'name resolve' (@whyrusleeping)\n * add `ipfs diag sys` command for debugging help (@whyrusleeping)\n\n* General Codebase\n * improvements to dag editor (@whyrusleeping)\n * swarm IPv6 in default config (Baptiste Jonglez)\n * improve dir listing css (@rht)\n * removed elliptic.P224 usage (@prusnak)\n * improve bitswap providing speed (@jbenet)\n * print panics that occur in cmds lib (@whyrusleeping)\n * ipfs api check test fixes (@rht)\n * update peerstream and datastore (@whyrusleeping)\n * cleaned up tar-reader code (@jbenet)\n * write context into coreunix.Cat (@rht)\n * move assets to separate repo (@rht)\n * fix proc/ctx wiring in bitswap (@jbenet)\n * rabin fingerprinting chunker (@whyrusleeping)\n * better notification on daemon ready (@rht)\n * coreunix cat cleanup (@rht)\n * extract logging into go-log (@whyrusleeping)\n * blockservice.New no longer errors (@whyrusleeping)\n * refactor ipfs get (@rht)\n * readonly api on gateway (@rht)\n * cleanup context usage all over (@rht)\n * add xml decoding to 'object put' (@ForrestWeston)\n * replace nodebuilder with NewNode method (@whyrusleeping)\n * add metrics to http handlers (@lgierth)\n * rm blockservice workers (@whyrusleeping)\n * decompose maybeGzWriter (@rht)\n * makefile sets git commit sha on build (@CaioAlonso)\n\n* Documentation\n * add contribute file (@RichardLitt)\n * add go devel guide to contribute.md (@whyrusleeping)\n\n* Testing\n * fix mock notifs test (@whyrusleeping)\n * test utf8 with object cmd (@chriscool)\n * make mocknet conn close idempotent (@jbenet)\n * fix fuse tests (@pnelson)\n * improve sharness test quoting (@chriscool)\n * sharness tests for chunker and add-cat (@rht)\n * generalize peerid check in sharness (@chriscool)\n * test_cmp argument cleanup (@chriscool)\n\n## v0.3.7 - 2015-08-02\n\nThis patch update fixes a problem we introduced in 0.3.6 and did not\ncatch: the webui failed to work with out-of-the-box CORS configs.\nThis has been fixed and now should work correctly. @jbenet\n\n## v0.3.6 - 2015-07-30\n\nThis patch improves the resource consumption of go-ipfs,\nintroduces a few new options on the CLI, and also\nfixes (yet again) windows builds.\n\n* Resource consumption:\n * fixed goprocess memory leak @rht\n * implement batching on datastore @whyrusleeping\n * Fix bitswap memory leak @whyrusleeping\n * let bitswap ignore temporary write errors @whyrusleeping\n * remove logging to disk in favor of api endpoint @whyrusleeping\n * --only-hash option for add to skip writing to disk @whyrusleeping\n\n* Tool changes\n * improved `ipfs daemon` output with all addresses @jbenet\n * improved `ipfs id -f` output, added `` and `\\n \\t` support @jbenet\n * `ipfs swarm addrs local` now shows the local node's addrs @jbenet\n * improved config json parsing @rht\n * improved Dockerfile to use alpine linux @Luzifer @lgierth\n * improved bash completion @MichaelMure\n * Improved 404 for gateway @cryptix\n * add unixfs ls to list correct filesizes @wking\n * ignore hidden files by default @gatesvp\n * global --timeout flag @whyrusleeping\n * fix random API failures by closing resp bodies @whyrusleeping\n * ipfs swarm filters @whyrusleeping\n * api returns errors in http trailers @whyrusleeping @jbenet\n * `ipfs patch` learned to create intermediate nodes @whyrusleeping\n * `ipfs object stat` now shows Hash @whyrusleeping\n * `ipfs cat` now clears progressbar on exit @rht\n * `ipfs add -w -r ` now wraps directories @jbenet\n * `ipfs add -w ` now wraps with one dir @jbenet\n * API + Gateway now support arbitrary HTTP Headers from config @jbenet\n * API now supports CORS properly from config @jbenet\n * **Deprecated:** `API_ORIGIN` env var (use config, see `ipfs daemon --help`) @jbenet\n\n* General Codebase\n * `nofuse` tag for windows @Luzifer\n * improved `ipfs add` code @gatesvp\n * started requiring license trailers @chriscool @jbenet\n * removed CtxCloser for goprocess @rht\n * remove deadcode @lgierth @whyrusleeping\n * reduced number of logging libs to 2 (soon to be 1) @rht\n * dial address filtering @whyrusleeping\n * Prometheus metrics @lgierth\n * new index page for gateway @krl @cryptix\n * move ping to separate protocol @whyrusleeping\n * add events to bitswap for a dashboard @whyrusleeping\n * add latency and bandwidth options to mocknet @heems\n * levenshtein distance cmd autosuggest @sbruce\n * refactor/cleanup of cmds http handler @whyrusleeping\n * cmds http stream reports errors in trailers @whyrusleeping\n\n* Bugfixes\n * fixed path resolution and validation @rht\n * fixed `ipfs get -C` output and progress bar @rht\n * Fixed install pkg dist bug @jbenet @Luzifer\n * Fix `ipfs get` silent failure @whyrusleeping\n * `ipfs get` tarx no longer times out @jbenet\n * `ipfs refs -r -u` is now correct @gatesvp\n * Fix `ipfs add -w -r ` wrapping bugs @jbenet\n * Fixed FUSE unmount failures @jbenet\n * Fixed `ipfs log tail` command (api + cli) @whyrusleeping\n\n* Testing\n * sharness updates @chriscool\n * ability to disable secio for testing @jbenet\n * fixed many random test failures, more reliable CI @whyrusleeping\n * Fixed racey notifier failures @whyrusleeping\n * `ipfs refs -r -u` test cases @jbenet\n * Fix failing pinning test @jbenet\n * Better CORS + Referer tests @jbenet\n * Added reversible gc test @rht\n * Fixed bugs in FUSE IPNS tests @whyrusleeping\n * Fixed bugs in FUSE IPFS tests @jbenet\n * Added `random-files` tool for easier sharness tests @jbenet\n\n* Documentation\n * Add link to init system examples @slang800\n * Add CORS documentation to daemon init @carver (Note: this will change soon)\n\n## v0.3.5 - 2015-06-11\n\nThis patch improves overall stability and performance\n\n* added 'object patch' and 'object new' commands @whyrusleeping\n* improved symmetric NAT avoidance @jbenet\n* move util.Key to blocks.Key @whyrusleeping\n* fix memory leak in provider store @whyrusleeping\n* updated webui to 0.2.0 @krl\n* improved bitswap performance @whyrusleeping\n* update fuse lib @cryptix\n* fix path resolution @wking\n* implement test_seq() in sharness @chriscool\n* improve parsing of stdin for commands @chriscool\n* fix 'ipfs refs' failing silently @whyrusleeping\n* fix serial dialing bug @jbenet\n* improved testing @chriscool @rht @jbenet\n* fixed domain resolving @luzifer\n* fix parsing of unwanted stdin @lgierth\n* added CORS handlers to gateway @NodeGuy\n* added `ipfs daemon --unrestricted-api` option @krl\n* general cleanup of dependencies\n\n## v0.3.4 - 2015-05-10\n\n* fix ipns append bug @whyrusleeping\n* fix out of memory panic @whyrusleeping\n* add in expvar metrics @tv42\n* bitswap improvements @whyrusleeping\n* fix write-cache in blockstore @tv42\n* vendoring cleanup @cryptix\n* added `launchctl` plist for OSX @grncdr\n* improved Dockerfile, changed root and mount paths @ehd\n* improved `pin ls` output to show types @vitorbaptista\n\n## v0.3.3 - 2015-04-28\n\nThis patch update fixes various issues, in particular:\n- windows support (0.3.0 had broken it)\n- command line parses spaces correctly.\n\n* much improved command line parsing by @AtnNn\n* improved dockerfile by @luzifer\n* add cmd cleanup by @wking\n* fix flatfs windows support by @tv42 and @gatesvp\n* test case improvements by @chriscool\n* ipns resolution timeout bug fix by @whyrusleeping\n* new cluster tests with iptb by @whyrusleeping\n* fix log callstack printing bug by @whyrusleeping\n* document bash completion by @dylanPowers\n\n## v0.3.2 - 2015-04-22\n\nThis patch update implements multicast dns as well as fixing a few test issues.\n\n* implement mdns peer discovery @whyrusleeping\n* fix mounting issues in sharness tests @chriscool\n\n## v0.3.1 - 2015-04-21\n\nThis patch update fixes a few bugs:\n\n* harden shutdown logic by @torarnv\n* daemon locking fixes by @travisperson\n* don't re-add entire dirs by @whyrusleeping\n* tests now wait for graceful shutdown by @jbenet\n* default key size is now 2048 by @jbenet\n\n## v0.3.0 - 2015-04-20\n\nWe've just released version 0.3.0, which contains many\nperformance improvements, bugfixes, and new features.\nPerhaps the most noticeable change is moving block storage\nfrom leveldb to flat files in the filesystem.\n\nWhat to expect:\n\n* _much faster_ performance\n\n* Repo format 2\n * moved default location from ~/.go-ipfs -> ~/.ipfs\n * renamed lock filename daemon.lock -> repo.lock\n * now using a flat-file datastore for local blocks\n\n* Fixed lots of bugs\n * proper ipfs-path in various commands\n * fixed two pinning bugs (recursive pins)\n * increased yamux streams window (for speed)\n * increased bitswap workers (+ env var)\n * fixed memory leaks\n * ipfs add error returns\n * daemon exit bugfix\n * set proper UID and GID on fuse mounts\n\n* Gateway\n * Added support for HEAD requests\n\n* configuration\n * env var to turn off SO_REUSEPORT: IPFS_REUSEPORT=false\n * env var to increase bitswap workers: IPFS_BITSWAP_TASK_WORKERS=n\n\n* other\n * bash completion is now available\n * ipfs stats bw -- bandwidth metering\n\nAnd many more things.\n" }, { "path": "docs/changelogs/v0.30.md", "content": "# Kubo changelog v0.30\n\n- [v0.30.0](#v0300)\n\n## v0.30.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Improved P2P connectivity](#improved-p2p-connectivity)\n - [Refactored Bitswap and dag-pb chunker](#refactored-bitswap-and-dag-pb-chunker)\n - [WebRTC-Direct Transport enabled by default](#webrtc-direct-transport-enabled-by-default)\n - [UnixFS 1.5: Mode and Modification Time Support](#unixfs-15-mode-and-modification-time-support)\n - [AutoNAT V2 Service Introduced Alongside V1](#autonat-v2-service-introduced-alongside-v1)\n - [Automated `ipfs version check`](#automated-ipfs-version-check)\n - [Version Suffix Configuration](#version-suffix-configuration)\n - [`/unix/` socket support in `Addresses.API`](#unix-socket-support-in-addressesapi)\n - [Cleaned Up `ipfs daemon` Startup Log](#cleaned-up-ipfs-daemon-startup-log)\n - [Commands Preserve Specified Hostname](#commands-preserve-specified-hostname)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\nThis release took longer and is more packed with fixes and features than usual.\n\n> [!IMPORTANT]\n> TLDR: update, it contains many, many fixes.\n\n#### Improved P2P connectivity\n\nThis release comes with significant go-libp2p update from v0.34.1 to v0.36.3 ([release notes](https://github.com/libp2p/go-libp2p/releases/)).\n\nIt includes multiple fixes to key protocols: [QUIC](https://github.com/libp2p/specs/tree/master/quic)/[Webtransport](https://github.com/libp2p/specs/tree/master/webtransport)/[WebRTC](https://github.com/libp2p/specs/tree/master/webrtc), Connection Upgrades through Relay ([DCUtR](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md)), and [Secure WebSockets](https://github.com/libp2p/specs/pull/624).\n\nAlso, peers that are behind certain types of NAT will now be more reachable. For this alone, Kubo users are highly encouraged to upgrade.\n\n#### Refactored Bitswap and dag-pb chunker\n\nSome workloads may experience improved memory profile thanks to optimizations from Boxo SDK [v0.23.0](https://github.com/ipfs/boxo/releases/tag/v0.23.0).\n\n> [!IMPORTANT]\n> Storage providers should upgrade to take advantage of the Bitswap server fix, which resolves the issue of greedy peers depleting available wantlist slots for their PeerID, resulting in stalled downloads.\n\n#### WebRTC-Direct Transport enabled by default\n\nKubo now ships with [WebRTC Direct](https://github.com/libp2p/specs/blob/master/webrtc/webrtc-direct.md) listener enabled by default: `/udp/4001/webrtc-direct`.\n\nWebRTC Direct complements existing `/wss` (Secure WebSockets) and `/webtransport` transports. Unlike `/wss`, which requires a domain name and a CA-issued TLS certificate, WebRTC Direct works with IPs and can be enabled by default on all Kubo nodes.\n\nLearn more: [`Swarm.Transports.Network.WebRTCDirect`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmtransportsnetworkwebrtcdirect)\n\n> [!NOTE]\n> Kubo 0.30 includes a migration for existing users that adds `/webrtc-direct` listener on the same UDP port as `/udp/{port}/quic-v1`. This supports the WebRTC-Direct rollout by reusing preexisting UDP firewall settings and port mappings created for QUIC.\n\n#### UnixFS 1.5: Mode and Modification Time Support\n\nKubo now allows users to opt-in to store mode and modification time for files, directories, and symbolic links.\nBy default, if you do not opt-in, the old behavior remains unchanged, and the same CIDs will be generated as before.\n\nThe `ipfs add` CLI options `--preserve-mode` and `--preserve-mtime` can be used to store the original mode and last modified time of the file being added, and `ipfs files stat /ipfs/CID` can be used for inspecting these optional attributes:\n\n```console\n$ touch ./file\n$ chmod 654 ./file\n$ ipfs add --preserve-mode --preserve-mtime -Q ./file\nQmczQr4XS1rRnWVopyg5Chr9EQ7JKpbhgnrjpb5kTQ1DKQ\n\n$ ipfs files stat /ipfs/QmczQr4XS1rRnWVopyg5Chr9EQ7JKpbhgnrjpb5kTQ1DKQ\nQmczQr4XS1rRnWVopyg5Chr9EQ7JKpbhgnrjpb5kTQ1DKQ\nSize: 0\nCumulativeSize: 22\nChildBlocks: 0\nType: file\nMode: -rw-r-xr-- (0654)\nMtime: 13 Aug 2024, 21:15:31 UTC\n```\n\nThe CLI and HTTP RPC options `--mode`, `--mtime` and `--mtime-nsecs` can be used to set them to arbitrary values.\n\nOpt-in support for `mode` and `mtime` was also added to MFS (`ipfs files --help`). For more information see `--help` text of `ipfs files touch|stat|chmod` commands.\n\nModification time support was also added to the Gateway. If present, value from file's dag-pb is returned in `Last-Modified` HTTP header and requests made with `If-Modified-Since` can produce HTTP 304 not modified response.\n\n> [!NOTE]\n> Storing `mode` and `mtime` requires root block to be `dag-pb` and disabled `raw-leaves` setting to create envelope for storing the metadata.\n\n#### AutoNAT V2 Service Introduced Alongside V1\n\nThe AutoNAT service enables nodes to determine their public reachability on the internet. [AutoNAT V2](https://github.com/libp2p/specs/pull/538) enhances this protocol with improved features. In this release, Kubo will offer both V1 and V2 services to other peers, although it will continue to use only V1 when acting as a client. Future releases will phase out V1, transitioning clients to utilize V2 exclusively.\n\nFor more details, see the [Deployment Plan for AutoNAT V2](https://github.com/ipfs/kubo/issues/10091) and [`AutoNAT`](https://github.com/ipfs/kubo/blob/master/docs/config.md#autonat) configuration options.\n\n#### Automated `ipfs version check`\n\nKubo now performs privacy-preserving version checks using the [libp2p identify protocol](https://github.com/libp2p/specs/blob/master/identify/README.md) on peers detected by the Amino DHT client.\nIf more than 5% of Kubo peers seen by your node are running a newer version, you will receive a log message notification.\n\n- For manual checks, refer to `ipfs version check --help` for details.\n- To disable automated checks, set [`Version.SwarmCheckEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#versionswarmcheckenabled) to `false`.\n\n#### Version Suffix Configuration\n\nDefining the optional agent version suffix is now simpler. The [`Version.AgentSuffix`](https://github.com/ipfs/kubo/blob/master/docs/config.md#agentsuffix) value from the Kubo config takes precedence over any value provided via `ipfs daemon --agent-version-suffix` (which is still supported).\n\n> [!NOTE]\n> Setting a custom version suffix helps with ecosystem analysis, such as Amino DHT reports published at https://stats.ipfs.network\n\n#### `/unix/` socket support in `Addresses.API`\n\nThis release fixes a bug which blocked users from using Unix domain sockets for [Kubo's RPC](https://docs.ipfs.tech/reference/kubo/rpc/) (instead of a local HTTP port).\n\n```console\n$ ipfs config Addresses.API \"/unix/tmp/kubo.socket\"\n$ ipfs daemon # start with rpc socket\n...\nRPC API server listening on /unix/tmp/kubo.socket\n\n$ # cli client, in different terminal can find socket via /api file\n$ cat $IPFS_PATH/api\n/unix/tmp/kubo.socket\n\n$ # or have it passed via --api\n$ ipfs --api=/unix/tmp/kubo.socket id\n```\n\n#### Cleaned Up `ipfs daemon` Startup Log\n\nThe `ipfs daemon` startup output has been streamlined to enhance clarity and usability:\n\n```console\n$ ipfs daemon\nInitializing daemon...\nKubo version: 0.30.0\nRepo version: 16\nSystem version: amd64/linux\nGolang version: go1.22.5\nPeerID: 12D3KooWQ73s1CQsm4jWwQvdCAtc5w8LatyQt7QLQARk5xdhK9CE\nSwarm listening on 127.0.0.1:4001 (TCP+UDP)\nSwarm listening on 192.0.2.10:4001 (TCP+UDP)\nSwarm listening on [::1]:4001 (TCP+UDP)\nSwarm listening on [2001:0db8::10]:4001 (TCP+UDP)\nRun 'ipfs id' to inspect announced and discovered multiaddrs of this node.\nRPC API server listening on /ip4/127.0.0.1/tcp/5001\nWebUI: http://127.0.0.1:5001/webui\nGateway server listening on /ip4/127.0.0.1/tcp/8080\nDaemon is ready\n```\n\nThe previous lengthy listing of all listener and announced multiaddrs has been removed due to its complexity, especially with modern libp2p nodes sharing multiple transports and long lists of `/webtransport` and `/webrtc-direct` certhashes.\nThe output now features a simplified list of swarm listeners, displayed in the format `host:port (TCP+UDP)`, which provides essential information for debugging connectivity issues, particularly related to port forwarding.\nAnnounced libp2p addresses are no longer printed on startup, because libp2p may change or augment them based on AutoNAT, relay, and UPnP state. Instead, users are prompted to run `ipfs id` to obtain up-to-date list of listeners and announced multiaddrs in libp2p format.\n\n#### Commands Preserve Specified Hostname\n\nWhen executing a [CLI command](https://docs.ipfs.tech/reference/kubo/cli/) over [Kubo RPC API](https://docs.ipfs.tech/reference/kubo/rpc/), if a hostname is specified by `--api=/dns4//` the resulting HTTP request now contains the hostname, instead of the the IP address that the hostname resolved to, as was the previous behavior. This makes it easier for those trying to run Kubo behind a reverse proxy using hostname-based rules.\n\n#### Commands Preserve Specified Hostname\n\nWhen executing a [CLI command](https://docs.ipfs.tech/reference/kubo/cli/) over [Kubo RPC API](https://docs.ipfs.tech/reference/kubo/rpc/), if a hostname is specified by `--api=/dns4//` the resulting HTTP request now contains the hostname, instead of the the IP address that the hostname resolved to, as was the previous behavior. This makes it easier for those trying to run Kubo behind a reverse proxy using hostname-based rules.\n \n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: set version to 0.30.0\n - chore: bump CurrentVersionNumber\n - chore: boxo v0.23.0 and go-libp2p v0.36.3 (#10507) ([ipfs/kubo#10507](https://github.com/ipfs/kubo/pull/10507))\n - fix: switch back to go 1.22 (#10502) ([ipfs/kubo#10502](https://github.com/ipfs/kubo/pull/10502))\n - chore: update go-unixfsnode, cmds, and boxo (#10494) ([ipfs/kubo#10494](https://github.com/ipfs/kubo/pull/10494))\n - fix(cli): preserve hostname specified with --api in http request headers (#10497) ([ipfs/kubo#10497](https://github.com/ipfs/kubo/pull/10497))\n - chore: upgrade to go 1.23 (#10486) ([ipfs/kubo#10486](https://github.com/ipfs/kubo/pull/10486))\n - fix: error during config when running benchmarks (#10495) ([ipfs/kubo#10495](https://github.com/ipfs/kubo/pull/10495))\n - chore: update version to rc-2\n - chore: update version\n - chore: fix function name (#10481) ([ipfs/kubo#10481](https://github.com/ipfs/kubo/pull/10481))\n - feat: Support storing UnixFS 1.5 Mode and ModTime (#10478) ([ipfs/kubo#10478](https://github.com/ipfs/kubo/pull/10478))\n - fix(rpc): cross-platform support for /unix/ socket maddrs in Addresses.API ([ipfs/kubo#10019](https://github.com/ipfs/kubo/pull/10019))\n - chore(daemon): sort listeners (#10480) ([ipfs/kubo#10480](https://github.com/ipfs/kubo/pull/10480))\n - feat(daemon): improve stdout on startup (#10472) ([ipfs/kubo#10472](https://github.com/ipfs/kubo/pull/10472))\n - fix(daemon): panic in kubo/daemon.go:595 (#10473) ([ipfs/kubo#10473](https://github.com/ipfs/kubo/pull/10473))\n - feat: webui v4.3.0 (#10477) ([ipfs/kubo#10477](https://github.com/ipfs/kubo/pull/10477))\n - docs(readme): add Gentoo Linux (#10474) ([ipfs/kubo#10474](https://github.com/ipfs/kubo/pull/10474))\n - libp2p: default to preferring TLS ([ipfs/kubo#10227](https://github.com/ipfs/kubo/pull/10227))\n - docs: document unofficial Ubuntu PPA ([ipfs/kubo#10467](https://github.com/ipfs/kubo/pull/10467))\n - feat: run AutoNAT V2 service in addition to V1 (#10468) ([ipfs/kubo#10468](https://github.com/ipfs/kubo/pull/10468))\n - feat: go-libp2p 0.36 and /webrtc-direct listener (#10463) ([ipfs/kubo#10463](https://github.com/ipfs/kubo/pull/10463))\n - chore: update dependencies (#10462)(#10466) ([ipfs/kubo#10466](https://github.com/ipfs/kubo/pull/10466))\n - feat: periodic version check and json config (#10438) ([ipfs/kubo#10438](https://github.com/ipfs/kubo/pull/10438))\n - docs: clarify pnet limitations\n - docs: \"error mounting: could not resolve name\" (#10449) ([ipfs/kubo#10449](https://github.com/ipfs/kubo/pull/10449))\n - docs: update ipfs-swarm-key-gen example (#10453) ([ipfs/kubo#10453](https://github.com/ipfs/kubo/pull/10453))\n - chore: update deps incl. boxo v0.21.0 (#10444) ([ipfs/kubo#10444](https://github.com/ipfs/kubo/pull/10444))\n - chore: go-libp2p 0.35.1 (#10430) ([ipfs/kubo#10430](https://github.com/ipfs/kubo/pull/10430))\n - docsa: update RELEASE_CHECKLIST.md\n - chore: create next changelog (#10443) ([ipfs/kubo#10443](https://github.com/ipfs/kubo/pull/10443))\n - Merge Release: v0.29.0 [skip changelog] ([ipfs/kubo#10442](https://github.com/ipfs/kubo/pull/10442))\n - fix(cli): unify --name param in ls and add (#10439) ([ipfs/kubo#10439](https://github.com/ipfs/kubo/pull/10439))\n - fix(libp2p): streams config validation in resource manager (#10435) ([ipfs/kubo#10435](https://github.com/ipfs/kubo/pull/10435))\n - chore: fix some typos (#10396) ([ipfs/kubo#10396](https://github.com/ipfs/kubo/pull/10396))\n - chore: update version\n- github.com/ipfs/boxo (v0.20.0 -> v0.23.0):\n - Release v0.23.0 ([ipfs/boxo#669](https://github.com/ipfs/boxo/pull/669))\n - docs(changelog): move entry to correct release\n - Release v0.22.0 ([ipfs/boxo#654](https://github.com/ipfs/boxo/pull/654))\n - Release v0.21.0 ([ipfs/boxo#622](https://github.com/ipfs/boxo/pull/622))\n- github.com/ipfs/go-ipfs-cmds (v0.11.0 -> v0.13.0):\n - chore: release v0.13.0 (#261) ([ipfs/go-ipfs-cmds#261](https://github.com/ipfs/go-ipfs-cmds/pull/261))\n - chore: release v0.12.0 (#259) ([ipfs/go-ipfs-cmds#259](https://github.com/ipfs/go-ipfs-cmds/pull/259))\n- github.com/ipfs/go-unixfsnode (v1.9.0 -> v1.9.1):\n - Update release version ([ipfs/go-unixfsnode#76](https://github.com/ipfs/go-unixfsnode/pull/76))\n - chore: update dependencies ([ipfs/go-unixfsnode#75](https://github.com/ipfs/go-unixfsnode/pull/75))\n- github.com/libp2p/go-libp2p (v0.34.1 -> v0.36.3):\n - Release v0.36.3\n - Fix: WebSocket: Clone TLS config before creating a new listener\n - fix: enable dctur when interface address is public (#2931) ([libp2p/go-libp2p#2931](https://github.com/libp2p/go-libp2p/pull/2931))\n - fix: QUIC/Webtransport Transports now will prefer their owned listeners for dialing out (#2936) ([libp2p/go-libp2p#2936](https://github.com/libp2p/go-libp2p/pull/2936))\n - ci: uci/update-go (#2937) ([libp2p/go-libp2p#2937](https://github.com/libp2p/go-libp2p/pull/2937))\n - fix: slice append value (#2938) ([libp2p/go-libp2p#2938](https://github.com/libp2p/go-libp2p/pull/2938))\n - webrtc: wait for listener context before dropping connection (#2932) ([libp2p/go-libp2p#2932](https://github.com/libp2p/go-libp2p/pull/2932))\n - ci: use go1.23, drop go1.21 (#2933) ([libp2p/go-libp2p#2933](https://github.com/libp2p/go-libp2p/pull/2933))\n - Fail on any test timeout (#2929) ([libp2p/go-libp2p#2929](https://github.com/libp2p/go-libp2p/pull/2929))\n - test: Try to fix test timeout (#2930) ([libp2p/go-libp2p#2930](https://github.com/libp2p/go-libp2p/pull/2930))\n - ci: Out of the tarpit (#2923) ([libp2p/go-libp2p#2923](https://github.com/libp2p/go-libp2p/pull/2923))\n - Fix proto import paths (#2920) ([libp2p/go-libp2p#2920](https://github.com/libp2p/go-libp2p/pull/2920))\n - Release v0.36.2\n - webrtc: reduce loglevel for pion logs (#2915) ([libp2p/go-libp2p#2915](https://github.com/libp2p/go-libp2p/pull/2915))\n - webrtc: close connection when remote closes (#2914) ([libp2p/go-libp2p#2914](https://github.com/libp2p/go-libp2p/pull/2914))\n - basic_host: close swarm on Close (#2916) ([libp2p/go-libp2p#2916](https://github.com/libp2p/go-libp2p/pull/2916))\n - Revert \"Create funding.json\" (#2919) ([libp2p/go-libp2p#2919](https://github.com/libp2p/go-libp2p/pull/2919))\n - Create funding.json\n - Release v0.36.1\n - Release v0.36.0 (#2905) ([libp2p/go-libp2p#2905](https://github.com/libp2p/go-libp2p/pull/2905))\n - swarm: add a default timeout to conn.NewStream (#2907) ([libp2p/go-libp2p#2907](https://github.com/libp2p/go-libp2p/pull/2907))\n - udpmux: Don't log an error if canceled because of shutdown (#2903) ([libp2p/go-libp2p#2903](https://github.com/libp2p/go-libp2p/pull/2903))\n - ObsAddrManager: Infer external addresses for transports that share the same listening address. (#2892) ([libp2p/go-libp2p#2892](https://github.com/libp2p/go-libp2p/pull/2892))\n - feat: WebRTC reuse QUIC conn (#2889) ([libp2p/go-libp2p#2889](https://github.com/libp2p/go-libp2p/pull/2889))\n - examples/chat-with-mdns: default to a random port (#2896) ([libp2p/go-libp2p#2896](https://github.com/libp2p/go-libp2p/pull/2896))\n - allow findpeers limit to be 0 (#2894) ([libp2p/go-libp2p#2894](https://github.com/libp2p/go-libp2p/pull/2894))\n - quic: add support for quic-go metrics (#2823) ([libp2p/go-libp2p#2823](https://github.com/libp2p/go-libp2p/pull/2823))\n - webrtc: remove experimental tag, enable by default (#2887) ([libp2p/go-libp2p#2887](https://github.com/libp2p/go-libp2p/pull/2887))\n - config: fix AddrFactory for AutoNAT (#2868) ([libp2p/go-libp2p#2868](https://github.com/libp2p/go-libp2p/pull/2868))\n - chore: /quic → /quic-v1 (#2888) ([libp2p/go-libp2p#2888](https://github.com/libp2p/go-libp2p/pull/2888))\n - basichost: reset stream if SetProtocol fails (#2875) ([libp2p/go-libp2p#2875](https://github.com/libp2p/go-libp2p/pull/2875))\n - feat: libp2phttp `/http-path` (#2850) ([libp2p/go-libp2p#2850](https://github.com/libp2p/go-libp2p/pull/2850))\n - readme: update per latest multiversx rename (#2874) ([libp2p/go-libp2p#2874](https://github.com/libp2p/go-libp2p/pull/2874))\n - websocket: don't return transport.ErrListenerClosed on closing listener (#2867) ([libp2p/go-libp2p#2867](https://github.com/libp2p/go-libp2p/pull/2867))\n - Added tau to README.md (#2870) ([libp2p/go-libp2p#2870](https://github.com/libp2p/go-libp2p/pull/2870))\n - basichost: reset new stream if rcmgr blocks (#2869) ([libp2p/go-libp2p#2869](https://github.com/libp2p/go-libp2p/pull/2869))\n - transport integration tests: test conn attempt is dropped when the rcmgr blocks for WebRTC (#2856) ([libp2p/go-libp2p#2856](https://github.com/libp2p/go-libp2p/pull/2856))\n - webtransport: close underlying h3 connection (#2862) ([libp2p/go-libp2p#2862](https://github.com/libp2p/go-libp2p/pull/2862))\n - peerstore: don't intern protocols (#2860) ([libp2p/go-libp2p#2860](https://github.com/libp2p/go-libp2p/pull/2860))\n - autonatv2: add server metrics for dial requests (#2848) ([libp2p/go-libp2p#2848](https://github.com/libp2p/go-libp2p/pull/2848))\n - PR Comments\n - Add a transport level test to ensure we close conns after rejecting them by the rcmgr\n - Close quic conns when wrapping conn fails\n - libp2p: use rcmgr for autonat dials (#2842) ([libp2p/go-libp2p#2842](https://github.com/libp2p/go-libp2p/pull/2842))\n - metricshelper: improve checks for ip and transport (#2849) ([libp2p/go-libp2p#2849](https://github.com/libp2p/go-libp2p/pull/2849))\n - Don't reuse the URL, make a new one\n - Use default transport to make using the Host cheaper\n - cleanup\n - Add future test\n - HTTP Host implements RoundTripper\n - swarm: improve dial worker performance for common case\n - pstoremanager: fix connectedness check\n - autonatv2: implement autonatv2 spec (#2469) ([libp2p/go-libp2p#2469](https://github.com/libp2p/go-libp2p/pull/2469))\n - webrtc: add a test for establishing many connections (#2801) ([libp2p/go-libp2p#2801](https://github.com/libp2p/go-libp2p/pull/2801))\n - webrtc: fix ufrag prefix for dialing (#2832) ([libp2p/go-libp2p#2832](https://github.com/libp2p/go-libp2p/pull/2832))\n - circuitv2: improve voucher validation (#2826) ([libp2p/go-libp2p#2826](https://github.com/libp2p/go-libp2p/pull/2826))\n - libp2phttp: workaround for ResponseWriter's CloseNotifier (#2821) ([libp2p/go-libp2p#2821](https://github.com/libp2p/go-libp2p/pull/2821))\n - Update README.md (#2830) ([libp2p/go-libp2p#2830](https://github.com/libp2p/go-libp2p/pull/2830))\n - identify: add test for observed address handling (#2828) ([libp2p/go-libp2p#2828](https://github.com/libp2p/go-libp2p/pull/2828))\n - identify: fix bug in observed address handling (#2825) ([libp2p/go-libp2p#2825](https://github.com/libp2p/go-libp2p/pull/2825))\n - identify: Don't filter addr if remote is neither public nor private (#2820) ([libp2p/go-libp2p#2820](https://github.com/libp2p/go-libp2p/pull/2820))\n - limit ping duration to 30s (#1358) ([libp2p/go-libp2p#1358](https://github.com/libp2p/go-libp2p/pull/1358))\n - Remove out-dated code in example readme (#2818) ([libp2p/go-libp2p#2818](https://github.com/libp2p/go-libp2p/pull/2818))\n - v0.35.0 (#2812) ([libp2p/go-libp2p#2812](https://github.com/libp2p/go-libp2p/pull/2812))\n - rcmgr: Support specific network prefix in conn limiter (#2807) ([libp2p/go-libp2p#2807](https://github.com/libp2p/go-libp2p/pull/2807))\n- github.com/libp2p/go-libp2p-kad-dht (v0.25.2 -> v0.26.1):\n - Release v0.26.1 ([libp2p/go-libp2p-kad-dht#983](https://github.com/libp2p/go-libp2p-kad-dht/pull/983))\n - fix: Unexport hasValidConnectedness to make a patch release ([libp2p/go-libp2p-kad-dht#982](https://github.com/libp2p/go-libp2p-kad-dht/pull/982))\n - correctly merging fix from https://github.com/libp2p/go-libp2p-kad-dht/pull/976 ([libp2p/go-libp2p-kad-dht#980](https://github.com/libp2p/go-libp2p-kad-dht/pull/980))\n - Release v0.26.0 ([libp2p/go-libp2p-kad-dht#979](https://github.com/libp2p/go-libp2p-kad-dht/pull/979))\n - chore: update deps ([libp2p/go-libp2p-kad-dht#974](https://github.com/libp2p/go-libp2p-kad-dht/pull/974))\n - Upgrade to go-log v2.5.1 ([libp2p/go-libp2p-kad-dht#971](https://github.com/libp2p/go-libp2p-kad-dht/pull/971))\n - Fix: don't perform lookupCheck if not enough peers in routing table ([libp2p/go-libp2p-kad-dht#970](https://github.com/libp2p/go-libp2p-kad-dht/pull/970))\n - findnode(self) should return multiple peers ([libp2p/go-libp2p-kad-dht#968](https://github.com/libp2p/go-libp2p-kad-dht/pull/968))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.7.3 -> v0.7.4):\n - chore: release v0.7.4 (#85) ([libp2p/go-libp2p-routing-helpers#85](https://github.com/libp2p/go-libp2p-routing-helpers/pull/85))\n - fix: composable parallel router tracing by index (#84) ([libp2p/go-libp2p-routing-helpers#84](https://github.com/libp2p/go-libp2p-routing-helpers/pull/84))\n- github.com/multiformats/go-multiaddr (v0.12.4 -> v0.13.0):\n - Release v0.13.0 ([multiformats/go-multiaddr#248](https://github.com/multiformats/go-multiaddr/pull/248))\n - Add support for http-path ([multiformats/go-multiaddr#246](https://github.com/multiformats/go-multiaddr/pull/246))\n- github.com/whyrusleeping/cbor-gen (v0.1.1 -> v0.1.2):\n - properly extend strings (#95) ([whyrusleeping/cbor-gen#95](https://github.com/whyrusleeping/cbor-gen/pull/95))\n - ioutil to io (#98) ([whyrusleeping/cbor-gen#98](https://github.com/whyrusleeping/cbor-gen/pull/98))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Andrew Gillis | 14 | +4920/-1714 | 145 |\n| sukun | 26 | +4402/-448 | 79 |\n| Marco Munizaga | 32 | +2287/-536 | 73 |\n| Marcin Rataj | 41 | +685/-193 | 86 |\n| Patryk | 1 | +312/-10 | 8 |\n| guillaumemichel | 7 | +134/-105 | 14 |\n| Adin Schmahmann | 5 | +145/-80 | 9 |\n| Henrique Dias | 2 | +190/-1 | 6 |\n| Josh Klopfenstein | 1 | +90/-35 | 27 |\n| gammazero | 5 | +90/-28 | 11 |\n| Jeromy Johnson | 1 | +116/-0 | 5 |\n| Daniel N | 3 | +27/-25 | 9 |\n| Daniel Norman | 2 | +28/-19 | 4 |\n| Ivan Shvedunov | 2 | +25/-10 | 2 |\n| Michael Muré | 2 | +22/-9 | 4 |\n| Dominic Della Valle | 1 | +23/-4 | 1 |\n| Andrei Vukolov | 1 | +27/-0 | 1 |\n| chris erway | 1 | +9/-9 | 9 |\n| Vitaly Zdanevich | 1 | +12/-0 | 1 |\n| Guillaume Michel | 1 | +4/-7 | 1 |\n| swedneck | 1 | +10/-0 | 1 |\n| Jorropo | 2 | +5/-5 | 3 |\n| omahs | 1 | +4/-4 | 4 |\n| THAT ONE GUY | 1 | +3/-5 | 2 |\n| vyzo | 1 | +5/-2 | 1 |\n| looklose | 1 | +3/-3 | 2 |\n| web3-bot | 2 | +2/-3 | 4 |\n| Dave Huseby | 1 | +5/-0 | 1 |\n| shenpengfeng | 1 | +1/-1 | 1 |\n| bytetigers | 1 | +1/-1 | 1 |\n| Sorin Stanculeanu | 1 | +1/-1 | 1 |\n| Lukáš Lukáč | 1 | +1/-1 | 1 |\n| Gabe | 1 | +1/-1 | 1 |\n| Bryan Stenson | 1 | +1/-1 | 1 |\n| Samy Fodil | 1 | +1/-0 | 1 |\n| Lane Rettig | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.31.md", "content": "# Kubo changelog v0.31\n\n- [v0.31.0](#v0310)\n\n## v0.31.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Experimental Pebble Datastore](#experimental-pebble-datastore)\n - [New metrics](#new-metrics)\n - [`lowpower` profile no longer breaks DHT announcements](#lowpower-profile-no-longer-breaks-dht-announcements)\n - [go 1.23, boxo 0.24 and go-libp2p 0.36.5](#go-123-boxo-024-and-go-libp2p-0365)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Experimental Pebble Datastore\n\n[Pebble](https://github.com/ipfs/kubo/blob/master/docs/config.md#pebbleds-profile) provides a high-performance alternative to leveldb as the datastore, and provides a modern replacement for [legacy badgerv1](https://github.com/ipfs/kubo/blob/master/docs/config.md#badgerds-profile).\n\nA fresh Kubo node can be initialized with [`pebbleds` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#pebbleds-profile) via `ipfs init --profile pebbleds`.\n\nThere are a number of parameters available for tuning pebble's performance to your specific needs. Default values are used for any parameters that are not configured or are set to their zero-value.\nFor a description of the available tuning parameters, see [kubo/docs/datastores.md#pebbleds](https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds).\n\n#### New metrics\n\n- Added 3 new go metrics: `go_gc_gogc_percent`, `go_gc_gomemlimit_bytes` and `go_sched_gomaxprocs_threads` as those are [recommended by the Go team](https://github.com/prometheus/client_golang/pull/1559)\n- Added [network usage metrics](https://github.com/prometheus/client_golang/pull/1555): `process_network_receive_bytes_total` and `process_network_transmit_bytes_total`\n- Removed `go_memstat_lookups_total` metric [which was always 0](https://github.com/prometheus/client_golang/pull/1577)\n\n#### `lowpower` profile no longer breaks DHT announcements\n\nWe've notices users were applying `lowpower` profile, and then reporting content routing issues. This was because `lowpower` disabled reprovider system and locally hosted data was no longer announced on Amino DHT.\n\nThis release changes [`lowpower` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#lowpower-profile) to not change reprovider settings, ensuring the new users are not sabotaging themselves. It also adds [`announce-on`](https://github.com/ipfs/kubo/blob/master/docs/config.md#announce-on-profile) and [`announce-off`](https://github.com/ipfs/kubo/blob/master/docs/config.md#announce-off-profile) profiles for controlling announcement settings separately.\n\n> [!IMPORTANT]\n> If you've ever applied the `lowpower` profile before, there is a high chance your node is not announcing to DHT anymore.\n> If you have `Reprovider.Interval` set to `0` you may want to set it to `22h` (or run `ipfs config profile apply announce-on`) to fix your system.\n>\n> As a convenience, `ipfs daemon` will warn if reprovide system is disabled, creating oportinity to fix configuration if it was not intentional.\n\n#### go 1.23, boxo 0.24 and go-libp2p 0.36.5\n\nVarious bugfixes. Please update.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix: go 1.23(.2) (#10540) ([ipfs/kubo#10540](https://github.com/ipfs/kubo/pull/10540))\n - chore: bump version to 0.32.0-dev\n - feat(routing/http): support IPIP-484 and streaming (#10534) ([ipfs/kubo#10534](https://github.com/ipfs/kubo/pull/10534))\n - fix(daemon): webui URL when rpc is catch-all (#10520) ([ipfs/kubo#10520](https://github.com/ipfs/kubo/pull/10520))\n - chore: update changelog and config doc with more info about pebble (#10533) ([ipfs/kubo#10533](https://github.com/ipfs/kubo/pull/10533))\n - feat: pebbleds profile and plugin (#10530) ([ipfs/kubo#10530](https://github.com/ipfs/kubo/pull/10530))\n - chore: dependency updates for 0.31 (#10511) ([ipfs/kubo#10511](https://github.com/ipfs/kubo/pull/10511))\n - feat: explicit announce-on/off profiles (#10524) ([ipfs/kubo#10524](https://github.com/ipfs/kubo/pull/10524))\n - fix(core): look for MFS root in local repo only (#8661) ([ipfs/kubo#8661](https://github.com/ipfs/kubo/pull/8661))\n - Fix issue in ResourceManager and nopfsPlugin about repo path (#10492) ([ipfs/kubo#10492](https://github.com/ipfs/kubo/pull/10492))\n - feat(bitswap): allow configuring WithWantHaveReplaceSize (#10512) ([ipfs/kubo#10512](https://github.com/ipfs/kubo/pull/10512))\n - refactor: simplify logic for MFS pinning (#10506) ([ipfs/kubo#10506](https://github.com/ipfs/kubo/pull/10506))\n - docs: clarify Gateway.PublicGateways (#10525) ([ipfs/kubo#10525](https://github.com/ipfs/kubo/pull/10525))\n - chore: clarify dep update in RELEASE_CHECKLIST.md (#10518) ([ipfs/kubo#10518](https://github.com/ipfs/kubo/pull/10518))\n - feat: ipfs-webui v4.3.2 (#10523) ([ipfs/kubo#10523](https://github.com/ipfs/kubo/pull/10523))\n - docs(config): add useful references\n - docs(config): improve profile descriptions (#10517) ([ipfs/kubo#10517](https://github.com/ipfs/kubo/pull/10517))\n - docs: update RELEASE_CHECKLIST.md (#10496) ([ipfs/kubo#10496](https://github.com/ipfs/kubo/pull/10496))\n - chore: create next changelog (#10510) ([ipfs/kubo#10510](https://github.com/ipfs/kubo/pull/10510))\n - Merge Release: v0.30.0 [skip changelog] ([ipfs/kubo#10508](https://github.com/ipfs/kubo/pull/10508))\n - chore: boxo v0.23.0 and go-libp2p v0.36.3 (#10507) ([ipfs/kubo#10507](https://github.com/ipfs/kubo/pull/10507))\n - docs: replace outdated package paths described in rpc README (#10505) ([ipfs/kubo#10505](https://github.com/ipfs/kubo/pull/10505))\n - fix: switch back to go 1.22 (#10502) ([ipfs/kubo#10502](https://github.com/ipfs/kubo/pull/10502))\n - fix(cli): preserve hostname specified with --api in http request headers (#10497) ([ipfs/kubo#10497](https://github.com/ipfs/kubo/pull/10497))\n - chore: upgrade to go 1.23 (#10486) ([ipfs/kubo#10486](https://github.com/ipfs/kubo/pull/10486))\n - fix: error during config when running benchmarks (#10495) ([ipfs/kubo#10495](https://github.com/ipfs/kubo/pull/10495))\n - chore: update go-unixfsnode, cmds, and boxo (#10494) ([ipfs/kubo#10494](https://github.com/ipfs/kubo/pull/10494))\n - Docs fix spelling issues (#10493) ([ipfs/kubo#10493](https://github.com/ipfs/kubo/pull/10493))\n - chore: update version (#10491) ([ipfs/kubo#10491](https://github.com/ipfs/kubo/pull/10491))\n- github.com/ipfs/boxo (v0.23.0 -> v0.24.0):\n - Release v0.24.0 ([ipfs/boxo#683](https://github.com/ipfs/boxo/pull/683))\n- github.com/ipfs/go-ipld-cbor (v0.1.0 -> v0.2.0):\n - v0.2.0\n - deprecate DumpObject() in favor of better named Encode()\n - add an EncodeWriter method, using the pooled marshallers\n - fix expCid vs actualCid guard\n- github.com/ipld/go-car/v2 (v2.13.1 -> v2.14.2):\n - v2.14.2 bump\n - fix: goreleaser v2 compat, trigger release-binaries with workflow_run\n - v2.14.1 bump\n - chore: update fuzz to Go 1.22\n - v2.14.0 bump\n - fix(cmd): properly pick up --inverse and --cid-file args ([ipld/go-car#531](https://github.com/ipld/go-car/pull/531))\n - Re-factor cmd functions to library ([ipld/go-car#524](https://github.com/ipld/go-car/pull/524))\n - ci: uci/copy-templates ([ipld/go-car#521](https://github.com/ipld/go-car/pull/521))\n - Add a `car ls --unixfs-blocks` to render two-column output ([ipld/go-car#514](https://github.com/ipld/go-car/pull/514))\n- github.com/libp2p/go-libp2p (v0.36.3 -> v0.36.5):\n - chore: remove Roadmap file (#2954) ([libp2p/go-libp2p#2954](https://github.com/libp2p/go-libp2p/pull/2954))\n - fix: Release v0.36.5\n - autonatv2: recover from panics (#2992) ([libp2p/go-libp2p#2992](https://github.com/libp2p/go-libp2p/pull/2992))\n - basichost: ensure no duplicates in Addrs output (#2980) ([libp2p/go-libp2p#2980](https://github.com/libp2p/go-libp2p/pull/2980))\n - Release v0.36.4\n - peerstore: better GC in membacked peerstore (#2960) ([libp2p/go-libp2p#2960](https://github.com/libp2p/go-libp2p/pull/2960))\n - fix: use quic.Version instead of the deprecated quic.VersionNumber (#2955) ([libp2p/go-libp2p#2955](https://github.com/libp2p/go-libp2p/pull/2955))\n - tcp: fix metrics for multiple calls to Close (#2953) ([libp2p/go-libp2p#2953](https://github.com/libp2p/go-libp2p/pull/2953))\n- github.com/libp2p/go-libp2p-kbucket (v0.6.3 -> v0.6.4):\n - release v0.6.4 ([libp2p/go-libp2p-kbucket#135](https://github.com/libp2p/go-libp2p-kbucket/pull/135))\n - feat: add log printing when peer added and removed table ([libp2p/go-libp2p-kbucket#134](https://github.com/libp2p/go-libp2p-kbucket/pull/134))\n - Upgrade to go-log v2.5.1 ([libp2p/go-libp2p-kbucket#132](https://github.com/libp2p/go-libp2p-kbucket/pull/132))\n - chore: update go-libp2p-asn-util\n- github.com/multiformats/go-multiaddr-dns (v0.3.1 -> v0.4.0):\n - Release v0.4.0 (#64) ([multiformats/go-multiaddr-dns#64](https://github.com/multiformats/go-multiaddr-dns/pull/64))\n - Limit total number of resolved addresses from DNS response (#63) ([multiformats/go-multiaddr-dns#63](https://github.com/multiformats/go-multiaddr-dns/pull/63))\n - fix!: Only resolve the first DNS-like component (#61) ([multiformats/go-multiaddr-dns#61](https://github.com/multiformats/go-multiaddr-dns/pull/61))\n - sync: update CI config files (#43) ([multiformats/go-multiaddr-dns#43](https://github.com/multiformats/go-multiaddr-dns/pull/43))\n - remove deprecated types ([multiformats/go-multiaddr-dns#37](https://github.com/multiformats/go-multiaddr-dns/pull/37))\n - remove Jenkinsfile ([multiformats/go-multiaddr-dns#40](https://github.com/multiformats/go-multiaddr-dns/pull/40))\n - sync: update CI config files (#29) ([multiformats/go-multiaddr-dns#29](https://github.com/multiformats/go-multiaddr-dns/pull/29))\n - use net.IP.Equal to compare IP addresses ([multiformats/go-multiaddr-dns#30](https://github.com/multiformats/go-multiaddr-dns/pull/30))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Will Scott | 3 | +731/-581 | 14 |\n| Daniel N | 17 | +1034/-191 | 33 |\n| Marco Munizaga | 5 | +721/-404 | 12 |\n| Andrew Gillis | 9 | +765/-266 | 35 |\n| Marcin Rataj | 17 | +568/-323 | 41 |\n| Daniel Norman | 3 | +232/-111 | 10 |\n| sukun | 4 | +93/-8 | 8 |\n| Jorropo | 2 | +48/-45 | 5 |\n| Marten Seemann | 3 | +19/-47 | 5 |\n| fengzie | 1 | +29/-26 | 5 |\n| Rod Vagg | 7 | +27/-11 | 9 |\n| gopherfarm | 1 | +14/-14 | 6 |\n| web3-bot | 3 | +13/-10 | 3 |\n| Michael Muré | 2 | +16/-5 | 4 |\n| i-norden | 1 | +9/-9 | 1 |\n| Elias Rad | 1 | +7/-7 | 4 |\n| Prithvi Shahi | 1 | +0/-11 | 2 |\n| Lucas Molas | 1 | +5/-4 | 1 |\n| elecbug | 1 | +6/-2 | 1 |\n| gammazero | 2 | +2/-2 | 2 |\n| chris erway | 1 | +2/-2 | 2 |\n| Russell Dempsey | 1 | +2/-1 | 1 |\n| guillaumemichel | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.32.md", "content": "# Kubo changelog v0.32\n\n- [v0.32.0](#v0320)\n\n## v0.32.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [🎯 AutoTLS: Automatic Certificates for libp2p WebSockets via `libp2p.direct`](#-autotls-automatic-certificates-for-libp2p-websockets-via-libp2pdirect)\n - [📦️ Dependency updates](#-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### 🎯 AutoTLS: Automatic Certificates for libp2p WebSockets via `libp2p.direct`\n\n\n\nThis release introduces an experimental feature that significantly improves how browsers ([Helia](https://helia.io/), [Service Worker](https://inbrowser.link)) can connect to Kubo node.\n\nOpt-in configuration allows a publicly dialable Kubo nodes (public IP, port forwarding, or NAT with uPnP) to obtain CA-signed TLS certificates for [libp2p Secure WebSocket (WSS)](https://github.com/libp2p/specs/blob/master/websockets/README.md) connections automatically.\n\n> [!TIP]\n> To enable this feature, set `AutoTLS.Enabled` to `true` and add a listener for `/tls/sni/*.libp2p.direct/ws` on a separate TCP port:\n> ```diff\n> {\n> + \"AutoTLS\": { \"Enabled\": true },\n> \"Addresses\": {\n> \"Swarm\": {\n> \"/ip4/0.0.0.0/tcp/4001\",\n> + \"/ip4/0.0.0.0/tcp/4002/tls/sni/*.libp2p.direct/ws\",\n> \"/ip6/::/tcp/4001\",\n> + \"/ip6/::/tcp/4002/tls/sni/*.libp2p.direct/ws\",\n> ```\n> After restarting your node for the first time you may need to wait 5-15 minutes to pass all checks and for the changes to take effect.\n> We are working on sharing the same TCP port with other transports ([go-libp2p#2984](https://github.com/libp2p/go-libp2p/pull/2984)).\n\nSee [`AutoTLS` configuration](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls) for more details how to enable it and what to expect.\n\nThis is an early preview, we appreciate you testing and filling bug reports or feedback in the tracking issue at [kubo#10560](https://github.com/ipfs/kubo/issues/10560).\n\n#### 📦️ Dependency updates\n\n- update `ipfs-webui` to [v4.4.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.0)\n- update `boxo` to [v0.24.1](https://github.com/ipfs/boxo/releases/tag/v0.24.1) + [v0.24.2](https://github.com/ipfs/boxo/releases/tag/v0.24.2) + [v0.24.3](https://github.com/ipfs/boxo/releases/tag/v0.24.3)\n - This includes a number of fixes and bitswap improvements, and support for filtering from [IPIP-484](https://specs.ipfs.tech/ipips/ipip-0484/) in delegated HTTP routing and IPNI queries.\n- update `go-libp2p` to [v0.37.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.37.0)\n - This update required removal of `Swarm.RelayService.MaxReservationsPerPeer` configuration option from Kubo. If you had it set, remove it from your configuration file.\n- update `go-libp2p-kad-dht` to [v0.27.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.27.0) + [v0.28.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.0) + [v0.28.1](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.1)\n- update `go-libp2p-pubsub` to [v0.12.0](https://github.com/libp2p/go-libp2p-pubsub/releases/tag/v0.12.0)\n- update `p2p-forge/client` to [v0.0.2](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.0.2)\n- removed `go-homedir`\n - The `github.com/mitchellh/go-homedir` repo is archived, no longer needed, and no longer maintained.\n - `homedir.Dir` is replaced by the stdlib `os.UserHomeDir`\n - `homedir.Expand` is replaced by `fsutil.ExpandHome` in the `github.com/ipfs/kubo/misc/fsutil` package.\n - The new `github.com/ipfs/kubo/misc/fsutil` package contains file utility code previously located elsewhere in kubo.\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: 0.32.0\n - fix: go-libp2p-kad-dht v0.28.0 (#10578) ([ipfs/kubo#10578](https://github.com/ipfs/kubo/pull/10578))\n - chore: 0.32.0-rc2\n - feat: ipfs-webui v4.4.0 (#10574) ([ipfs/kubo#10574](https://github.com/ipfs/kubo/pull/10574))\n - chore: label implicit loggers\n - chore: boxo v0.24.3 and p2p-forge v0.0.2 (#10572) ([ipfs/kubo#10572](https://github.com/ipfs/kubo/pull/10572))\n - chore: stop using go-homedir (#10568) ([ipfs/kubo#10568](https://github.com/ipfs/kubo/pull/10568))\n - fix(autotls): store certificates at the location from the repo path (#10566) ([ipfs/kubo#10566](https://github.com/ipfs/kubo/pull/10566))\n - chore: 0.32.0-rc1\n - docs(autotls): add note about separate port use (#10562) ([ipfs/kubo#10562](https://github.com/ipfs/kubo/pull/10562))\n - feat(AutoTLS): opt-in WSS certs from p2p-forge at libp2p.direct (#10521) ([ipfs/kubo#10521](https://github.com/ipfs/kubo/pull/10521))\n - chore: upgrade to boxo v0.24.2 (#10559) ([ipfs/kubo#10559](https://github.com/ipfs/kubo/pull/10559))\n - refactor: update to go-libp2p v0.37.0 (#10554) ([ipfs/kubo#10554](https://github.com/ipfs/kubo/pull/10554))\n - docs(config): explain what multiaddr is\n - chore: update dependencies (#10548) ([ipfs/kubo#10548](https://github.com/ipfs/kubo/pull/10548))\n - chore: update test dependencies (#10555) ([ipfs/kubo#10555](https://github.com/ipfs/kubo/pull/10555))\n - chore(ci): adjust verbosity\n - chore(ci): verbose build of test/bin deps\n - chore(ci): build docker images for staging branch\n - Create Changelog: v0.32 ([ipfs/kubo#10546](https://github.com/ipfs/kubo/pull/10546))\n - Merge release v0.31.0 ([ipfs/kubo#10545](https://github.com/ipfs/kubo/pull/10545))\n - chore: update RELEASE_CHECKLIST.md (#10544) ([ipfs/kubo#10544](https://github.com/ipfs/kubo/pull/10544))\n - feat: ipfs-webui v4.3.3 (#10543) ([ipfs/kubo#10543](https://github.com/ipfs/kubo/pull/10543))\n - chore: update RELEASE_CHECKLIST.md (#10542) ([ipfs/kubo#10542](https://github.com/ipfs/kubo/pull/10542))\n - Add full changelog to release changelog\n - fix: go 1.23(.2) (#10540) ([ipfs/kubo#10540](https://github.com/ipfs/kubo/pull/10540))\n - chore: bump version to 0.32.0-dev\n- github.com/ipfs/boxo (v0.24.0 -> v0.24.3):\n - Release v0.24.3 ([ipfs/boxo#713](https://github.com/ipfs/boxo/pull/713))\n - Merge branch 'main' into release\n - Release v0.24.2 ([ipfs/boxo#707](https://github.com/ipfs/boxo/pull/707))\n - Release v0.24.1 ([ipfs/boxo#706](https://github.com/ipfs/boxo/pull/706))\n- github.com/ipfs/go-ipfs-cmds (v0.13.0 -> v0.14.0):\n - chore: release v0.14.0 (#269) ([ipfs/go-ipfs-cmds#269](https://github.com/ipfs/go-ipfs-cmds/pull/269))\n- github.com/ipfs/go-ipfs-redirects-file (v0.1.1 -> v0.1.2):\n - chore: v0.1.2 (#29) ([ipfs/go-ipfs-redirects-file#29](https://github.com/ipfs/go-ipfs-redirects-file/pull/29))\n - docs(readme): refer specs and ipip\n - chore: update dependencies (#28) ([ipfs/go-ipfs-redirects-file#28](https://github.com/ipfs/go-ipfs-redirects-file/pull/28))\n- github.com/ipfs/go-metrics-prometheus (v0.0.2 -> v0.0.3):\n - chore: release v0.0.3 (#24) ([ipfs/go-metrics-prometheus#24](https://github.com/ipfs/go-metrics-prometheus/pull/24))\n - chore: update deps and update go-log to v2 (#23) ([ipfs/go-metrics-prometheus#23](https://github.com/ipfs/go-metrics-prometheus/pull/23))\n - sync: update CI config files (#9) ([ipfs/go-metrics-prometheus#9](https://github.com/ipfs/go-metrics-prometheus/pull/9))\n- github.com/ipfs/go-unixfsnode (v1.9.1 -> v1.9.2):\n - New release version ([ipfs/go-unixfsnode#78](https://github.com/ipfs/go-unixfsnode/pull/78))\n - chore: update dependencies\n- github.com/libp2p/go-flow-metrics (v0.1.0 -> v0.2.0):\n - chore: release v0.2.0 (#33) ([libp2p/go-flow-metrics#33](https://github.com/libp2p/go-flow-metrics/pull/33))\n - chore: cleanup readme (#31) ([libp2p/go-flow-metrics#31](https://github.com/libp2p/go-flow-metrics/pull/31))\n - ci: uci/update-go ([libp2p/go-flow-metrics#27](https://github.com/libp2p/go-flow-metrics/pull/27))\n - fix(ewma): reduce the chances of fake bandwidth spikes (#8) ([libp2p/go-flow-metrics#8](https://github.com/libp2p/go-flow-metrics/pull/8))\n - chore: switch to typed atomics (#24) ([libp2p/go-flow-metrics#24](https://github.com/libp2p/go-flow-metrics/pull/24))\n - test: use mock clocks for all tests (#25) ([libp2p/go-flow-metrics#25](https://github.com/libp2p/go-flow-metrics/pull/25))\n - ci: uci/copy-templates ([libp2p/go-flow-metrics#21](https://github.com/libp2p/go-flow-metrics/pull/21))\n- github.com/libp2p/go-libp2p (v0.36.5 -> v0.37.0):\n - Release v0.37.0 (#3013) ([libp2p/go-libp2p#3013](https://github.com/libp2p/go-libp2p/pull/3013))\n - feat: Add WithFxOption (#2956) ([libp2p/go-libp2p#2956](https://github.com/libp2p/go-libp2p/pull/2956))\n - chore: update imports to use slices package (#3007) ([libp2p/go-libp2p#3007](https://github.com/libp2p/go-libp2p/pull/3007))\n - Change latency metrics buckets (#3012) ([libp2p/go-libp2p#3012](https://github.com/libp2p/go-libp2p/pull/3012))\n - chore: bump deps in preparation for v0.37.0 (#3011) ([libp2p/go-libp2p#3011](https://github.com/libp2p/go-libp2p/pull/3011))\n - autonat: fix interaction with autorelay (#2967) ([libp2p/go-libp2p#2967](https://github.com/libp2p/go-libp2p/pull/2967))\n - swarm: add a peer dial latency metric (#2959) ([libp2p/go-libp2p#2959](https://github.com/libp2p/go-libp2p/pull/2959))\n - peerstore: limit number of non connected peers in addrbook (#2971) ([libp2p/go-libp2p#2971](https://github.com/libp2p/go-libp2p/pull/2971))\n - fix: swarm: refactor address resolution (#2990) ([libp2p/go-libp2p#2990](https://github.com/libp2p/go-libp2p/pull/2990))\n - Add backoff for updating local IP addresses on error (#2999) ([libp2p/go-libp2p#2999](https://github.com/libp2p/go-libp2p/pull/2999))\n - libp2phttp: HTTP Peer ID Authentication (#2854) ([libp2p/go-libp2p#2854](https://github.com/libp2p/go-libp2p/pull/2854))\n - relay: make only 1 reservation per peer (#2974) ([libp2p/go-libp2p#2974](https://github.com/libp2p/go-libp2p/pull/2974))\n - autonatv2: recover from panics (#2992) ([libp2p/go-libp2p#2992](https://github.com/libp2p/go-libp2p/pull/2992))\n - basichost: ensure no duplicates in Addrs output (#2980) ([libp2p/go-libp2p#2980](https://github.com/libp2p/go-libp2p/pull/2980))\n - fix(websocket): re-enable websocket transport test (#2987) ([libp2p/go-libp2p#2987](https://github.com/libp2p/go-libp2p/pull/2987))\n - feat(websocket): switch the underlying http server logger to use ipfs/go-log (#2985) ([libp2p/go-libp2p#2985](https://github.com/libp2p/go-libp2p/pull/2985))\n - peerstore: better GC in membacked peerstore (#2960) ([libp2p/go-libp2p#2960](https://github.com/libp2p/go-libp2p/pull/2960))\n - connmgr: reduce log level for untagging untracked peers ([libp2p/go-libp2p#2961](https://github.com/libp2p/go-libp2p/pull/2961))\n - fix: use quic.Version instead of the deprecated quic.VersionNumber (#2955) ([libp2p/go-libp2p#2955](https://github.com/libp2p/go-libp2p/pull/2955))\n - tcp: fix metrics for multiple calls to Close (#2953) ([libp2p/go-libp2p#2953](https://github.com/libp2p/go-libp2p/pull/2953))\n - chore: remove Roadmap file (#2954) ([libp2p/go-libp2p#2954](https://github.com/libp2p/go-libp2p/pull/2954))\n - chore: add a funding JSON file to apply for Optimism rPGF round 5 (#2940) ([libp2p/go-libp2p#2940](https://github.com/libp2p/go-libp2p/pull/2940))\n - Fix: WebSocket: Clone TLS config before creating a new listener\n - fix: enable dctur when interface address is public (#2931) ([libp2p/go-libp2p#2931](https://github.com/libp2p/go-libp2p/pull/2931))\n - fix: QUIC/Webtransport Transports now will prefer their owned listeners for dialing out (#2936) ([libp2p/go-libp2p#2936](https://github.com/libp2p/go-libp2p/pull/2936))\n - ci: uci/update-go (#2937) ([libp2p/go-libp2p#2937](https://github.com/libp2p/go-libp2p/pull/2937))\n - fix: slice append value (#2938) ([libp2p/go-libp2p#2938](https://github.com/libp2p/go-libp2p/pull/2938))\n - webrtc: wait for listener context before dropping connection (#2932) ([libp2p/go-libp2p#2932](https://github.com/libp2p/go-libp2p/pull/2932))\n - ci: use go1.23, drop go1.21 (#2933) ([libp2p/go-libp2p#2933](https://github.com/libp2p/go-libp2p/pull/2933))\n - Fail on any test timeout (#2929) ([libp2p/go-libp2p#2929](https://github.com/libp2p/go-libp2p/pull/2929))\n - test: Try to fix test timeout (#2930) ([libp2p/go-libp2p#2930](https://github.com/libp2p/go-libp2p/pull/2930))\n - ci: Out of the tarpit (#2923) ([libp2p/go-libp2p#2923](https://github.com/libp2p/go-libp2p/pull/2923))\n - Make BlackHoleState type public (#2917) ([libp2p/go-libp2p#2917](https://github.com/libp2p/go-libp2p/pull/2917))\n - Fix proto import paths (#2920) ([libp2p/go-libp2p#2920](https://github.com/libp2p/go-libp2p/pull/2920))\n- github.com/libp2p/go-libp2p-kad-dht (v0.26.1 -> v0.28.0):\n - chore: release v0.28.0 (#998) ([libp2p/go-libp2p-kad-dht#998](https://github.com/libp2p/go-libp2p-kad-dht/pull/998))\n - fix: set context timeout for `queryPeer` (#996) ([libp2p/go-libp2p-kad-dht#996](https://github.com/libp2p/go-libp2p-kad-dht/pull/996))\n - refactor: document and expose Amino DHT defaults (#990) ([libp2p/go-libp2p-kad-dht#990](https://github.com/libp2p/go-libp2p-kad-dht/pull/990))\n - Use timeout context for NewStream call ([libp2p/go-libp2p-kad-dht#994](https://github.com/libp2p/go-libp2p-kad-dht/pull/994))\n - release v0.27.0 ([libp2p/go-libp2p-kad-dht#992](https://github.com/libp2p/go-libp2p-kad-dht/pull/992))\n - Add new DHT option to provide custom pb.MessageSender ([libp2p/go-libp2p-kad-dht#991](https://github.com/libp2p/go-libp2p-kad-dht/pull/991))\n - fix: replace deprecated Boxo function ([libp2p/go-libp2p-kad-dht#987](https://github.com/libp2p/go-libp2p-kad-dht/pull/987))\n - fix(query): reverting changes on TestRTEvictionOnFailedQuery ([libp2p/go-libp2p-kad-dht#984](https://github.com/libp2p/go-libp2p-kad-dht/pull/984))\n- github.com/libp2p/go-libp2p-pubsub (v0.11.0 -> v0.12.0):\n - chore: upgrade go-libp2p (#575) ([libp2p/go-libp2p-pubsub#575](https://github.com/libp2p/go-libp2p-pubsub/pull/575))\n - GossipSub v1.2: IDONTWANT control message and priority queue. (#553) ([libp2p/go-libp2p-pubsub#553](https://github.com/libp2p/go-libp2p-pubsub/pull/553))\n - Re-enable disabled gossipsub test (#566) ([libp2p/go-libp2p-pubsub#566](https://github.com/libp2p/go-libp2p-pubsub/pull/566))\n - chore: staticcheck\n - chore: update rand usage\n - chore: go fmt\n - chore: add or force update version.json\n - added missing Close call on the AddrBook member of GossipSubRouter (#568) ([libp2p/go-libp2p-pubsub#568](https://github.com/libp2p/go-libp2p-pubsub/pull/568))\n - test: test notify protocols updated (#567) ([libp2p/go-libp2p-pubsub#567](https://github.com/libp2p/go-libp2p-pubsub/pull/567))\n - Switch to the new peer notify mechanism (#564) ([libp2p/go-libp2p-pubsub#564](https://github.com/libp2p/go-libp2p-pubsub/pull/564))\n - test: use the regular libp2p host (#565) ([libp2p/go-libp2p-pubsub#565](https://github.com/libp2p/go-libp2p-pubsub/pull/565))\n - Missing flood protection check for number of message IDs when handling `Ihave` messages (#560) ([libp2p/go-libp2p-pubsub#560](https://github.com/libp2p/go-libp2p-pubsub/pull/560))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marco Munizaga | 16 | +4253/-545 | 81 |\n| Pop Chunhapanya | 1 | +1423/-137 | 15 |\n| sukun | 10 | +752/-425 | 35 |\n| Steven Allen | 11 | +518/-541 | 35 |\n| Andrew Gillis | 19 | +348/-194 | 50 |\n| Marcin Rataj | 26 | +343/-132 | 47 |\n| Adin Schmahmann | 4 | +269/-29 | 12 |\n| gammazero | 12 | +154/-18 | 13 |\n| Josh Klopfenstein | 1 | +90/-35 | 27 |\n| galargh | 3 | +42/-44 | 13 |\n| Daniel Norman | 2 | +30/-16 | 4 |\n| Mikel Cortes | 3 | +25/-4 | 4 |\n| gopherfarm | 1 | +14/-14 | 6 |\n| Carlos Peliciari | 1 | +12/-12 | 4 |\n| Prithvi Shahi | 2 | +5/-11 | 3 |\n| web3-bot | 6 | +12/-3 | 6 |\n| guillaumemichel | 3 | +7/-6 | 3 |\n| Jorropo | 1 | +11/-0 | 1 |\n| Sorin Stanculeanu | 1 | +8/-0 | 1 |\n| Hlib Kanunnikov | 2 | +6/-2 | 4 |\n| André Bierlein | 1 | +4/-3 | 1 |\n| bytetigers | 1 | +1/-1 | 1 |\n| Wondertan | 2 | +2/-0 | 2 |\n| Alexandr Burdiyan | 1 | +1/-1 | 1 |\n| Guillaume Michel | 1 | +0/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.33.md", "content": "# Kubo changelog v0.33\n\n- [v0.33.0](#v0330)\n- [v0.33.1](#v0331)\n- [v0.33.2](#v0332)\n\n## v0.33.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Shared TCP listeners](#shared-tcp-listeners)\n - [AutoTLS takes care of Secure WebSockets setup](#autotls-takes-care-of-secure-websockets-setup)\n - [Bitswap improvements from Boxo](#bitswap-improvements-from-boxo)\n - [Using default `libp2p_rcmgr` metrics](#using-default-libp2p_rcmgr--metrics)\n - [Flatfs does not `sync` on each write](#flatfs-does-not-sync-on-each-write)\n - [`ipfs add --to-files` no longer works with `--wrap`](#ipfs-add---to-files-no-longer-works-with---wrap)\n - [`ipfs --api` supports HTTPS RPC endpoints](#ipfs---api-supports-https-rpc-endpoints)\n - [New options for faster writes: `WriteThrough`, `BlockKeyCacheSize`, `BatchMaxNodes`, `BatchMaxSize`](#new-options-for-faster-writes-writethrough-blockkeycachesize-batchmaxnodes-batchmaxsize)\n - [MFS stability with large number of writes](#mfs-stability-with-large-number-of-writes)\n - [New DoH resolvers for non-ICANN DNSLinks](#new-doh-resolvers-for-non-icann-dnslinks)\n - [Reliability improvements to the WebRTC Direct listener](#reliability-improvements-to-the-webrtc-direct-listener)\n - [Bitswap improvements from Boxo](#bitswap-improvements-from-boxo-1)\n - [📦️ Important dependency updates](#-important-dependency-updates)\n - [Escape Redirect URL for Directory](#escape-redirect-url-for-directory)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### Shared TCP listeners\n\nKubo now supports sharing the same TCP port (`4001` by default) by both [raw TCP](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmtransportsnetworktcp) and [WebSockets](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmtransportsnetworkwebsocket) libp2p transports.\n\nThis feature is not yet compatible with Private Networks and can be disabled by setting `LIBP2P_TCP_MUX=false` if causes any issues.\n\n#### AutoTLS takes care of Secure WebSockets setup\n\nIt is no longer necessary to manually add `/tcp/../ws` listeners to `Addresses.Swarm` when [`AutoTLS.Enabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotlsenabled) is set to `true`. Kubo will detect if `/ws` listener is missing and add one on the same port as pre-existing TCP (e.g. `/tcp/4001`), removing the need for any extra configuration.\n> [!TIP]\n> Give it a try:\n> ```console\n> $ ipfs config --json AutoTLS.Enabled true\n> ```\n> And restart the node. If you are behind NAT, make sure your node is publicly diallable (uPnP or port forwarding), and wait a few minutes to pass all checks and for the changes to take effect.\n\nSee [`AutoTLS`](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls) for more information.\n\n#### Bitswap improvements from Boxo\n\nThis release includes some refactorings and improvements affecting Bitswap which should improve reliability. One of the changes affects blocks providing. Previously, the bitswap layer took care itself of announcing new blocks -added or received- with the configured provider (i.e. DHT). This bypassed the \"Reprovider\", that is, the system that manages precisely \"providing\" the blocks stored by Kubo. The Reprovider knows how to take advantage of the [AcceleratedDHTClient](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingaccelerateddhtclient), is able to handle priorities, logs statistics and is able to resume on daemon reboot where it left off. From now on, Bitswap will not be doing any providing on-the-side and all announcements are managed by the reprovider. In some cases, when the reproviding queue is full with other elements, this may cause additional delays, but more likely this will result in improved block-providing behaviour overall.\n\n#### Using default `libp2p_rcmgr` metrics\n\nBespoke rcmgr metrics [were removed](https://github.com/ipfs/kubo/pull/9947), Kubo now exposes only the default `libp2p_rcmgr` metrics from go-libp2p.\nThis makes it easier to compare Kubo with custom implementations based on go-libp2p.\nIf you depended on removed ones, please fill an issue to add them to the upstream [go-libp2p](https://github.com/libp2p/go-libp2p).\n\n#### Flatfs does not `sync` on each write\n\nNew repositories initialized with `flatfs` in `Datastore.Spec` will have `sync` set to `false`.\n\nThe old default was overly conservative and caused performance issues in big repositories that did a lot of writes. There is usually no need to flush on every block write to disk before continuing. Setting this to false is safe as kubo will automatically flush writes to disk before and after performing critical operations like pinning. However, we still provide users with ability to set this to true to be extra-safe (at the cost of a slowdown when adding files in bulk).\n\n#### `ipfs add --to-files` no longer works with `--wrap`\n\nOnboarding files and directories with `ipfs add --to-files` now requires non-empty names. due to this, The `--to-files` and `--wrap` options are now mutually exclusive ([#10612](https://github.com/ipfs/kubo/issues/10612)).\n\n#### `ipfs --api` supports HTTPS RPC endpoints\n\nCLI and RPC client now supports accessing Kubo RPC over `https://` protocol when multiaddr ending with `/https` or `/tls/http` is passed to `ipfs --api`:\n\n```console\n$ ipfs id --api /dns/kubo-rpc.example.net/tcp/5001/tls/http\n# → https://kubo-rpc.example.net:5001\n```\n\n#### New options for faster writes: `WriteThrough`, `BlockKeyCacheSize`, `BatchMaxNodes`, `BatchMaxSize`\n\nNow that Kubo supports [`pebble`](https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds) as an _experimental_ datastore backend, it becomes very useful to expose some additional configuration options for how the blockservice/blockstore/datastore combo behaves.\n\nUsually, LSM-tree based datastore like Pebble or Badger have very fast write performance (blocks are streamed to disk) while incurring in read-amplification penalties (blocks need to be looked up in the index to know where they are on disk), specially noticeable on spinning disks.\n\nPrior to this version, `BlockService` and `Blockstore` implementations performed a `Has(cid)` for every block that was going to be written, skipping the writes altogether if the block was already present in the datastore. The performance impact of this `Has()` call can vary. The `Datastore` implementation itself might include block-caching and things like bloom-filters to speed up lookups and mitigate read-penalties. Our `Blockstore` implementation also supports a bloom-filter (controlled by `BloomFilterSize` and disabled by default), and a two-queue cache for keys and block sizes. If we assume that most of the blocks added to Kubo are new blocks, not already present in the datastore, or that the datastore itself includes mechanisms to optimize writes and avoid writing the same data twice, the calls to `Has()` at both BlockService and Blockstore layers seem superfluous to they point they even harm write performance.\n\nFor these reasons, from now on, the default is to use a \"write-through\" mode for the Blockservice and the Blockstore. We have added a new option `Datastore.WriteThrough`, which defaults to `true`. Previous behaviour can be obtained by manually setting it to `false`.\n\nWe have also made the size of the two-queue blockstore cache configurable with another option: `Datastore.BlockKeyCacheSize`, which defaults to `65536` (64KiB). Additionally, this caching layer can be disabled altogether by setting it to `0`. In particular, this option controls the size of a blockstore caching layer that records whether the blockstore has certain block and their sizes (but does not cache the contents, so it stays relativey small in general).\n\nFinally, we have added two new options to the `Import` section to control the maximum size of write-batches: `BatchMaxNodes` and `BatchMaxSize`. These are set by default to `128` nodes and `20MiB`. Increasing them will batch more items together when importing data with `ipfs dag import`, which can speed things up. It is importance to find a balance between available memory (used to hold the batch), disk latencies (when writing the batch) and processing power (when preparing the batch, as nodes are sorted and duplicates removed).\n\nAs a reminder, details from all the options are explained in the [configuration documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md).\n\nWe recommend users trying Pebble as a datastore backend to disable both blockstore bloom-filter and key caching layers and enable write through as a way to evaluate the raw performance of the underlying datastore, which includes its own bloom-filter and caching layers (default cache size is `8MiB` and can be configured in the [options](https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds).\n\n#### MFS stability with large number of writes\n\nWe have fixed a number of issues that were triggered by writing or copying many files onto an MFS folder: increased memory usage first, then CPU, disk usage, and eventually a deadlock on write operations. The details of the fixes can be read at [#10630](https://github.com/ipfs/kubo/pull/10630) and [#10623](https://github.com/ipfs/kubo/pull/10623). The result is that writing large amounts of files to an MFS folder should now be possible without major issues. It is possible, as before, to speed up the operations using the `ipfs files --flush=false ...` flag, but it is recommended to switch to `ipfs files --flush=true ...` regularly, or call `ipfs files flush` on the working directory regularly, as this will flush, clear the directory cache and speed up reads. \n\n#### New DoH resolvers for non-ICANN DNSLinks\n\n- `.eth` TLD DNSLinks are now resolved via [DNS-over-HTTPS](https://en.wikipedia.org/wiki/DNS_over_HTTPS) endpoint at `https://dns.eth.limo/dns-query`\n- `.crypto` TLD DNSLinks are now resolved via DoH endpoint at `https://resolver.unstoppable.io/dns-query`\n\n#### Reliability improvements to the WebRTC Direct listener\n\nTwo fixes in go-libp2p improve the reliability of the WebRTC Direct listener in Kubo, and by extension dialability from browsers.\n\nRelevant changes in go-libp2p:\n- [Deprioritising outgoing `/webrtc-direct`](https://github.com/libp2p/go-libp2p/pull/3078) dials.\n- [Allows more concurrent handshakes by default](https://github.com/libp2p/go-libp2p/pull/3040/).\n\n#### Bitswap improvements from Boxo\n\nThis release includes performance and reliability improvements and fixes for minor resource leaks.\n\n#### 📦️ Important dependency updates\n\n- update `boxo` to [v0.27.4](https://github.com/ipfs/boxo/releases/tag/v0.27.4) (incl. [v0.25.0](https://github.com/ipfs/boxo/releases/tag/v0.25.0) + [v0.26.0](https://github.com/ipfs/boxo/releases/tag/v0.26.0) + [v0.27.0](https://github.com/ipfs/boxo/releases/tag/v0.27.0) + [v0.27.1](https://github.com/ipfs/boxo/releases/tag/v0.27.1) + [v0.27.2](https://github.com/ipfs/boxo/releases/tag/v0.27.2) + [v0.27.3](https://github.com/ipfs/boxo/releases/tag/v0.27.3))\n- update `go-libp2p` to [v0.38.2](https://github.com/libp2p/go-libp2p/releases/tag/v0.38.2) (incl. [v0.37.1](https://github.com/libp2p/go-libp2p/releases/tag/v0.37.1) + [v0.37.2](https://github.com/libp2p/go-libp2p/releases/tag/v0.37.2) + [v0.38.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.38.0) + [v0.38.1](https://github.com/libp2p/go-libp2p/releases/tag/v0.38.1))\n- update `go-libp2p-kad-dht` to [v0.28.2](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.2)\n- update `quic-go` to [v0.49.0](https://github.com/quic-go/quic-go/releases/tag/v0.49.0)\n- update `p2p-forge/client` to [v0.3.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.3.0) (incl. [v0.1.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.1.0), [v0.2.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0), [v0.2.1](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.1), [v0.2.2](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.2))\n- update `ipfs-webui` to [v4.4.2](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.2) (incl. [v4.4.1](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.1))\n\n#### Escape Redirect URL for Directory\n\nWhen navigating to a subdirectory, served by the Kubo web server, a subdirectory without a trailing slash gets redirected to a URL with a trailing slash. If there are special characters such as \"%\" in the subdirectory name then these must be escaped in the redirect URL. Previously this was not being done and was preventing navigation to such subdirectories, requiring the user to manually add a trailing slash to the subdirectory URL. This is now fixed to handle the redirect to URLs with characters that must be escaped.\n\n### 📝 Changelog\n\n
Full Changelog v0.33.0\n\n- github.com/ipfs/kubo:\n - test: fix the socat tests after the ubuntu 24.04 upgrade (#10683) ([ipfs/kubo#10683](https://github.com/ipfs/kubo/pull/10683))\n - chore: 0.33.0-rc3\n - fix: quic-go v0.49.0 (#10673) ([ipfs/kubo#10673](https://github.com/ipfs/kubo/pull/10673))\n - Upgrade to Boxo v0.27.2 (#10672) ([ipfs/kubo#10672](https://github.com/ipfs/kubo/pull/10672))\n - chore: 0.33.0-rc2\n - Upgrade to Boxo v0.27.1 (#10671) ([ipfs/kubo#10671](https://github.com/ipfs/kubo/pull/10671))\n - fix(autotls): renewal and AutoTLS.ShortAddrs (#10669) ([ipfs/kubo#10669](https://github.com/ipfs/kubo/pull/10669))\n - update changelog for boxo and go-libp2p (#10668) ([ipfs/kubo#10668](https://github.com/ipfs/kubo/pull/10668))\n - Upgrade to Boxo v0.27.0 (#10665) ([ipfs/kubo#10665](https://github.com/ipfs/kubo/pull/10665))\n - update dependencies (#10664) ([ipfs/kubo#10664](https://github.com/ipfs/kubo/pull/10664))\n - fix(dns): update default DNSLink resolvers (#10655) ([ipfs/kubo#10655](https://github.com/ipfs/kubo/pull/10655))\n - chore: p2p-forge v0.2.2 + go-libp2p-kad-dht v0.28.2 (#10663) ([ipfs/kubo#10663](https://github.com/ipfs/kubo/pull/10663))\n - fix(cli): support HTTPS in ipfs --api (#10659) ([ipfs/kubo#10659](https://github.com/ipfs/kubo/pull/10659))\n - chore: fix typos and comment formatting (#10653) ([ipfs/kubo#10653](https://github.com/ipfs/kubo/pull/10653))\n - fix/gateway: escape directory redirect url (#10649) ([ipfs/kubo#10649](https://github.com/ipfs/kubo/pull/10649))\n - Add example of setting array to config command help\n - collection of typo fixes (#10647) ([ipfs/kubo#10647](https://github.com/ipfs/kubo/pull/10647))\n - chore: 0.33.0-rc1\n - fix: ipfs-webui v4.4.2 (#10635) ([ipfs/kubo#10635](https://github.com/ipfs/kubo/pull/10635))\n - feat(libp2p): shared TCP listeners and AutoTLS.AutoWSS (#10565) ([ipfs/kubo#10565](https://github.com/ipfs/kubo/pull/10565))\n - feat(flatfs): default to sync=false (#10632) ([ipfs/kubo#10632](https://github.com/ipfs/kubo/pull/10632))\n - Minor spelling and wording changes (#10634) ([ipfs/kubo#10634](https://github.com/ipfs/kubo/pull/10634))\n - docs: clarify Swarm.ResourceMgr.MaxMemory (#10622) ([ipfs/kubo#10622](https://github.com/ipfs/kubo/pull/10622))\n - feat: expose BlockKeyCacheSize and enable WriteThrough datastore options (#10614) ([ipfs/kubo#10614](https://github.com/ipfs/kubo/pull/10614))\n - cmd/files: flush parent folders (#10630) ([ipfs/kubo#10630](https://github.com/ipfs/kubo/pull/10630))\n - Upgrade to Boxo v0.26.0 (#10631) ([ipfs/kubo#10631](https://github.com/ipfs/kubo/pull/10631))\n - [skip changelog] pinmfs: mitigate slow mfs writes when it triggers (#10623) ([ipfs/kubo#10623](https://github.com/ipfs/kubo/pull/10623))\n - chore: use errors.New to replace fmt.Errorf with no parameters (#10617) ([ipfs/kubo#10617](https://github.com/ipfs/kubo/pull/10617))\n - chore: boxo v0.25.0 (#10619) ([ipfs/kubo#10619](https://github.com/ipfs/kubo/pull/10619))\n - fix(cmds/add): disallow --wrap with --to-files (#10612) ([ipfs/kubo#10612](https://github.com/ipfs/kubo/pull/10612))\n - refactor(cmds): do not return errors embedded in result type (#10527) ([ipfs/kubo#10527](https://github.com/ipfs/kubo/pull/10527))\n - fix: ipfs-webui v4.4.1 (#10608) ([ipfs/kubo#10608](https://github.com/ipfs/kubo/pull/10608))\n - chore: fix broken url in comment (#10606) ([ipfs/kubo#10606](https://github.com/ipfs/kubo/pull/10606))\n - refactor(rcmgr): use default libp2p rcmgr metrics (#9947) ([ipfs/kubo#9947](https://github.com/ipfs/kubo/pull/9947))\n - docs(changelog/v0.33): bitswap reprovide changes (#10604) ([ipfs/kubo#10604](https://github.com/ipfs/kubo/pull/10604))\n - tests(cli/harness): use unused Verbose flag to pipe daemon outputs (#10601) ([ipfs/kubo#10601](https://github.com/ipfs/kubo/pull/10601))\n - chore: p2p-forge/client v0.1.0 (#10605) ([ipfs/kubo#10605](https://github.com/ipfs/kubo/pull/10605))\n - fix: go-libp2p v0.37.2 (#10603) ([ipfs/kubo#10603](https://github.com/ipfs/kubo/pull/10603))\n - docs: typos (#10602) ([ipfs/kubo#10602](https://github.com/ipfs/kubo/pull/10602))\n - tests/cli: fix flapping tests (#10600) ([ipfs/kubo#10600](https://github.com/ipfs/kubo/pull/10600))\n - Update to boxo with refactored providerQueryManager. (#10595) ([ipfs/kubo#10595](https://github.com/ipfs/kubo/pull/10595))\n - fix some typos in docs (#10598) ([ipfs/kubo#10598](https://github.com/ipfs/kubo/pull/10598))\n - feat(bootstrap): add JS-based va1.bootstrap.libp2p.io (#10575) ([ipfs/kubo#10575](https://github.com/ipfs/kubo/pull/10575))\n - fix: increase provider sample size (#10589) ([ipfs/kubo#10589](https://github.com/ipfs/kubo/pull/10589))\n - Typos Update config.md (#10591) ([ipfs/kubo#10591](https://github.com/ipfs/kubo/pull/10591))\n - refactor: update to boxo without goprocess (#10567) ([ipfs/kubo#10567](https://github.com/ipfs/kubo/pull/10567))\n - fix: go-libp2p-kad-dht v0.28.1 (#10581) ([ipfs/kubo#10581](https://github.com/ipfs/kubo/pull/10581))\n - docs: update RELEASE_CHECKLIST.md (#10564) ([ipfs/kubo#10564](https://github.com/ipfs/kubo/pull/10564))\n - Merge release v0.32.0 ([ipfs/kubo#10579](https://github.com/ipfs/kubo/pull/10579))\n - fix: go-libp2p-kad-dht v0.28.0 (#10578) ([ipfs/kubo#10578](https://github.com/ipfs/kubo/pull/10578))\n - feat: ipfs-webui v4.4.0 (#10574) ([ipfs/kubo#10574](https://github.com/ipfs/kubo/pull/10574))\n - chore: boxo v0.24.3 and p2p-forge v0.0.2 (#10572) ([ipfs/kubo#10572](https://github.com/ipfs/kubo/pull/10572))\n - chore: stop using go-homedir (#10568) ([ipfs/kubo#10568](https://github.com/ipfs/kubo/pull/10568))\n - fix(autotls): store certificates at the location from the repo path (#10566) ([ipfs/kubo#10566](https://github.com/ipfs/kubo/pull/10566))\n - chore: bump master to 0.33.0-dev\n- github.com/ipfs-shipyard/nopfs (v0.0.12 -> v0.0.14):\n - Fix error when no doublehash db exists (#42) ([ipfs-shipyard/nopfs#42](https://github.com/ipfs-shipyard/nopfs/pull/42))\n - Improve support for IPNS double-hashed entries (#41) ([ipfs-shipyard/nopfs#41](https://github.com/ipfs-shipyard/nopfs/pull/41))\n- github.com/ipfs-shipyard/nopfs/ipfs (v0.13.2-0.20231027223058-cde3b5ba964c -> v0.25.0):\n failed to fetch repo\n- github.com/ipfs/boxo (v0.24.3 -> v0.27.2):\n - Release v0.27.2 ([ipfs/boxo#811](https://github.com/ipfs/boxo/pull/811))\n - Revert peer exclude cancel ([ipfs/boxo#809](https://github.com/ipfs/boxo/pull/809))\n - Release v0.27.1 ([ipfs/boxo#807](https://github.com/ipfs/boxo/pull/807))\n - fix sending cancels when excluding peer ([ipfs/boxo#805](https://github.com/ipfs/boxo/pull/805))\n - Release v0.27.0 ([ipfs/boxo#802](https://github.com/ipfs/boxo/pull/802))\n - Remove want-block sent tracking from sessionWantSender (#759) ([ipfs/boxo#759](https://github.com/ipfs/boxo/pull/759))\n - Upgrade to go-libp2p v0.38.2 (#804) ([ipfs/boxo#804](https://github.com/ipfs/boxo/pull/804))\n - [skip changelog] Use routing.ContentRouting interface (#803) ([ipfs/boxo#803](https://github.com/ipfs/boxo/pull/803))\n - fix potential crash in unixfs directory (#798) ([ipfs/boxo#798](https://github.com/ipfs/boxo/pull/798))\n - prefer slices.SortFunc to sort.Sort (#796) ([ipfs/boxo#796](https://github.com/ipfs/boxo/pull/796))\n - fix: ipns protobuf namespace conflict (#794) ([ipfs/boxo#794](https://github.com/ipfs/boxo/pull/794))\n - update release procedure (#773) ([ipfs/boxo#773](https://github.com/ipfs/boxo/pull/773))\n - reduce default number of routing in-process requests (#793) ([ipfs/boxo#793](https://github.com/ipfs/boxo/pull/793))\n - Do not return unused values from wantlists (#792) ([ipfs/boxo#792](https://github.com/ipfs/boxo/pull/792))\n - Create FUNDING.json [skip changelog] (#795) ([ipfs/boxo#795](https://github.com/ipfs/boxo/pull/795))\n - refactor: using slices.Contains to simplify the code (#791) ([ipfs/boxo#791](https://github.com/ipfs/boxo/pull/791))\n - do not send cancel message to peer that sent block (#784) ([ipfs/boxo#784](https://github.com/ipfs/boxo/pull/784))\n - Define a `go_package` for protobuf, rename to a more unique `ipns-record.proto` ([ipfs/boxo#789](https://github.com/ipfs/boxo/pull/789))\n - bitswap: messagequeue: lock only needed sections (#787) ([ipfs/boxo#787](https://github.com/ipfs/boxo/pull/787))\n - Update libp2p-kad-dht to v0.28.2 (#786) ([ipfs/boxo#786](https://github.com/ipfs/boxo/pull/786))\n - feat(gateway): allow localhost http:// DoH resolvers (#645) ([ipfs/boxo#645](https://github.com/ipfs/boxo/pull/645))\n - fix(gateway): update DoH resolver for .crypto DNSLink (#782) ([ipfs/boxo#782](https://github.com/ipfs/boxo/pull/782))\n - fix(gateway): update DoH resolver for .eth DNSLink (#781) ([ipfs/boxo#781](https://github.com/ipfs/boxo/pull/781))\n - chore: pass options to tracer start (#775) ([ipfs/boxo#775](https://github.com/ipfs/boxo/pull/775))\n - escape redirect urls (#783) ([ipfs/boxo#783](https://github.com/ipfs/boxo/pull/783))\n - fix/gateway: escape directory redirect url (#779) ([ipfs/boxo#779](https://github.com/ipfs/boxo/pull/779))\n - fix spelling in comments (#778) ([ipfs/boxo#778](https://github.com/ipfs/boxo/pull/778))\n - trivial spelling changes in comments (#777) ([ipfs/boxo#777](https://github.com/ipfs/boxo/pull/777))\n - Release v0.26.0 ([ipfs/boxo#770](https://github.com/ipfs/boxo/pull/770))\n - Minor spelling and wording changes (#768) ([ipfs/boxo#768](https://github.com/ipfs/boxo/pull/768))\n - update go-libp2p and go-libp2p-kad-dht ([ipfs/boxo#767](https://github.com/ipfs/boxo/pull/767))\n - [skip changelog] fix: Drop stream references on Close/Reset ([ipfs/boxo#760](https://github.com/ipfs/boxo/pull/760))\n - Update go-libp2p to v0.38.0 (#764) ([ipfs/boxo#764](https://github.com/ipfs/boxo/pull/764))\n - Fix leak due to cid queue never getting cleaned up (#756) ([ipfs/boxo#756](https://github.com/ipfs/boxo/pull/756))\n - Do not reset the broadcast timer if there are no wants (#758) ([ipfs/boxo#758](https://github.com/ipfs/boxo/pull/758))\n - Replace mock time implementation (#762) ([ipfs/boxo#762](https://github.com/ipfs/boxo/pull/762))\n - mfs: clean cache on sync ([ipfs/boxo#751](https://github.com/ipfs/boxo/pull/751))\n - Remove peer's count of first responses when peer becomes unavailable (#757) ([ipfs/boxo#757](https://github.com/ipfs/boxo/pull/757))\n - Remove unnecessary CID copying in SessionInterestManager (#761) ([ipfs/boxo#761](https://github.com/ipfs/boxo/pull/761))\n - [bitswap/peermanager] take read-lock for read-only operation (#755) ([ipfs/boxo#755](https://github.com/ipfs/boxo/pull/755))\n - bitswap/client/messagequeue: expose dontHaveTimeoutMgr configuration (#750) ([ipfs/boxo#750](https://github.com/ipfs/boxo/pull/750))\n - improve mfs republisher (#754) ([ipfs/boxo#754](https://github.com/ipfs/boxo/pull/754))\n - blockstore/blockservice: change option to `WriteThrough(enabled bool)` ([ipfs/boxo#749](https://github.com/ipfs/boxo/pull/749))\n - Merge release v0.25.0 ([ipfs/boxo#748](https://github.com/ipfs/boxo/pull/748))\n - Use deque instead of slice for queues (#742) ([ipfs/boxo#742](https://github.com/ipfs/boxo/pull/742))\n - chore: no lifecycle context to shutdown ProviderQueryManager (#734) ([ipfs/boxo#734](https://github.com/ipfs/boxo/pull/734))\n - removed Startup function from ProviderQueryManager (#741) ([ipfs/boxo#741](https://github.com/ipfs/boxo/pull/741))\n - Re-enable flaky bitswap tests (#740) ([ipfs/boxo#740](https://github.com/ipfs/boxo/pull/740))\n - feat(session): do not record erroneous session want sends (#452) ([ipfs/boxo#452](https://github.com/ipfs/boxo/pull/452))\n - feat(filestore): add mmap reader option (#665) ([ipfs/boxo#665](https://github.com/ipfs/boxo/pull/665))\n - chore: update to latest go-libp2p (#739) ([ipfs/boxo#739](https://github.com/ipfs/boxo/pull/739))\n - refactor(remote/pinning): `Ls` to take results channel instead of returning one (#738) ([ipfs/boxo#738](https://github.com/ipfs/boxo/pull/738))\n - Bitswap default ProviderQueryManager uses explicit options (#737) ([ipfs/boxo#737](https://github.com/ipfs/boxo/pull/737))\n - chore: minor examples cleanup (#736) ([ipfs/boxo#736](https://github.com/ipfs/boxo/pull/736))\n - misc comments and spelling (#735) ([ipfs/boxo#735](https://github.com/ipfs/boxo/pull/735))\n - chore: fix invalid url in docs (#733) ([ipfs/boxo#733](https://github.com/ipfs/boxo/pull/733))\n - [skip changelog] bitswap/client: fix wiring when passing custom providerFinder ([ipfs/boxo#732](https://github.com/ipfs/boxo/pull/732))\n - Add debug logging for deduplicated queries (#729) ([ipfs/boxo#729](https://github.com/ipfs/boxo/pull/729))\n - [skip changelog] staticcheck fixes / remove unused variables (#730) ([ipfs/boxo#730](https://github.com/ipfs/boxo/pull/730))\n - refactor: default to prometheus.DefaultRegisterer (#722) ([ipfs/boxo#722](https://github.com/ipfs/boxo/pull/722))\n - chore: minor Improvements to providerquerymanager (#728) ([ipfs/boxo#728](https://github.com/ipfs/boxo/pull/728))\n - dspinner: RecursiveKeys(): do not hang on cancellations (#727) ([ipfs/boxo#727](https://github.com/ipfs/boxo/pull/727))\n - Tests can signal immediate rebroadcast (#726) ([ipfs/boxo#726](https://github.com/ipfs/boxo/pull/726))\n - fix(bitswap/client/msgq): prevent duplicate requests (#691) ([ipfs/boxo#691](https://github.com/ipfs/boxo/pull/691))\n - Bitswap: move providing -> Exchange-layer, providerQueryManager -> routing (#641) ([ipfs/boxo#641](https://github.com/ipfs/boxo/pull/641))\n - fix(bitswap/client/providerquerymanager): don't end trace span until … (#725) ([ipfs/boxo#725](https://github.com/ipfs/boxo/pull/725))\n - fix(routing/http/server): adjust bucket sizes for http metrics ([ipfs/boxo#724](https://github.com/ipfs/boxo/pull/724))\n - fix(bitswap/client/providerquerymanager): use non-timed out context for tracing (#721) ([ipfs/boxo#721](https://github.com/ipfs/boxo/pull/721))\n - fix(bitswap/server): pass context to server engine to register metrics (#723) ([ipfs/boxo#723](https://github.com/ipfs/boxo/pull/723))\n - docs: fix url of tracing env vars (#719) ([ipfs/boxo#719](https://github.com/ipfs/boxo/pull/719))\n - feat(routing/http/server): add routing timeout (#720) ([ipfs/boxo#720](https://github.com/ipfs/boxo/pull/720))\n - feat(routing/http/server): expose prometheus metrics (#718) ([ipfs/boxo#718](https://github.com/ipfs/boxo/pull/718))\n - Remove dependency on goprocess ([ipfs/boxo#710](https://github.com/ipfs/boxo/pull/710))\n - Merge release v0.24.3 ([ipfs/boxo#714](https://github.com/ipfs/boxo/pull/714))\n - fix(bitswap): log unexpected blocks to debug level (#711) ([ipfs/boxo#711](https://github.com/ipfs/boxo/pull/711))\n - Release v0.24.2 ([ipfs/boxo#708](https://github.com/ipfs/boxo/pull/708))\n- github.com/ipfs/go-ds-pebble (v0.4.0 -> v0.4.2):\n - new version (#44) ([ipfs/go-ds-pebble#44](https://github.com/ipfs/go-ds-pebble/pull/44))\n - new version for pebble minor version update (#42) ([ipfs/go-ds-pebble#42](https://github.com/ipfs/go-ds-pebble/pull/42))\n- github.com/ipfs/go-ipfs-cmds (v0.14.0 -> v0.14.1):\n - fix(NewClient): support https:// URLs (#277) ([ipfs/go-ipfs-cmds#277](https://github.com/ipfs/go-ipfs-cmds/pull/277))\n- github.com/ipfs/go-peertaskqueue (v0.8.1 -> v0.8.2):\n - new version ([ipfs/go-peertaskqueue#39](https://github.com/ipfs/go-peertaskqueue/pull/39))\n - Replace mock time implementation ([ipfs/go-peertaskqueue#37](https://github.com/ipfs/go-peertaskqueue/pull/37))\n - fix: staticcheck feedback\n- github.com/libp2p/go-doh-resolver (v0.4.0 -> v0.5.0):\n - chore: release v0.5.0\n - fix: include url on HTTP error (#29) ([libp2p/go-doh-resolver#29](https://github.com/libp2p/go-doh-resolver/pull/29))\n - feat: allow localhost http endpoints (#28) ([libp2p/go-doh-resolver#28](https://github.com/libp2p/go-doh-resolver/pull/28))\n - sync: update CI config files (#20) ([libp2p/go-doh-resolver#20](https://github.com/libp2p/go-doh-resolver/pull/20))\n- github.com/libp2p/go-libp2p (v0.37.0 -> v0.38.2):\n - Release v0.38.2 (#3147) ([libp2p/go-libp2p#3147](https://github.com/libp2p/go-libp2p/pull/3147))\n - chore: release v0.38.1\n - fix(httpauth): Correctly handle concurrent requests on server (#3111) ([libp2p/go-libp2p#3111](https://github.com/libp2p/go-libp2p/pull/3111))\n - ci: Install specific protoc version when generating protobufs (#3112) ([libp2p/go-libp2p#3112](https://github.com/libp2p/go-libp2p/pull/3112))\n - fix(autorelay): Move relayFinder peer disconnect cleanup to separate goroutine (#3105) ([libp2p/go-libp2p#3105](https://github.com/libp2p/go-libp2p/pull/3105))\n - chore: Release v0.38.0 (#3106) ([libp2p/go-libp2p#3106](https://github.com/libp2p/go-libp2p/pull/3106))\n - peerstore: remove sync.Pool for expiringAddrs (#3093) ([libp2p/go-libp2p#3093](https://github.com/libp2p/go-libp2p/pull/3093))\n - webtransport: close quic conn on dial error (#3104) ([libp2p/go-libp2p#3104](https://github.com/libp2p/go-libp2p/pull/3104))\n - peerstore: fix addressbook benchmark timing (#3092) ([libp2p/go-libp2p#3092](https://github.com/libp2p/go-libp2p/pull/3092))\n - swarm: record conn metrics only once (#3091) ([libp2p/go-libp2p#3091](https://github.com/libp2p/go-libp2p/pull/3091))\n - fix(sampledconn): Correctly handle slow bytes and closed conns (#3080) ([libp2p/go-libp2p#3080](https://github.com/libp2p/go-libp2p/pull/3080))\n - peerstore: pass options to addrbook constructor (#3090) ([libp2p/go-libp2p#3090](https://github.com/libp2p/go-libp2p/pull/3090))\n - fix(swarm): remove stray print stmt (#3086) ([libp2p/go-libp2p#3086](https://github.com/libp2p/go-libp2p/pull/3086))\n - feat(swarm): delay /webrtc-direct dials by 1 second (#3078) ([libp2p/go-libp2p#3078](https://github.com/libp2p/go-libp2p/pull/3078))\n - chore: Update dependencies and fix deprecated function in relay example (#3023) ([libp2p/go-libp2p#3023](https://github.com/libp2p/go-libp2p/pull/3023))\n - chore: fix broken link to record envelope protobuf file (#3070) ([libp2p/go-libp2p#3070](https://github.com/libp2p/go-libp2p/pull/3070))\n - chore(core): fix function name in interface comment (#3056) ([libp2p/go-libp2p#3056](https://github.com/libp2p/go-libp2p/pull/3056))\n - basichost: avoid modifying slice returned by AddrsFactory (#3068) ([libp2p/go-libp2p#3068](https://github.com/libp2p/go-libp2p/pull/3068))\n - fix(swarm): check after we split for empty multiaddr (#3063) ([libp2p/go-libp2p#3063](https://github.com/libp2p/go-libp2p/pull/3063))\n - feat: allow passing options to memoryAddrBook (#3062) ([libp2p/go-libp2p#3062](https://github.com/libp2p/go-libp2p/pull/3062))\n - fix(libp2phttp): Return ErrServerClosed on Close (#3050) ([libp2p/go-libp2p#3050](https://github.com/libp2p/go-libp2p/pull/3050))\n - chore(dashboard/alertmanager): update api version from v1 to v2 (#3054) ([libp2p/go-libp2p#3054](https://github.com/libp2p/go-libp2p/pull/3054))\n - fix(tcpreuse): handle connection that failed to be sampled (#3036) ([libp2p/go-libp2p#3036](https://github.com/libp2p/go-libp2p/pull/3036))\n - fix(tcpreuse): remove windows specific code (#3039) ([libp2p/go-libp2p#3039](https://github.com/libp2p/go-libp2p/pull/3039))\n - refactor(libp2phttp): don't require specific port for the HTTP host example (#3047) ([libp2p/go-libp2p#3047](https://github.com/libp2p/go-libp2p/pull/3047))\n - refactor(core/routing): split ContentRouting interface (#3048) ([libp2p/go-libp2p#3048](https://github.com/libp2p/go-libp2p/pull/3048))\n - fix(holepunch/tracer): replace inline peer struct with peerInfo type (#3049) ([libp2p/go-libp2p#3049](https://github.com/libp2p/go-libp2p/pull/3049))\n - fix: Defer resource usage cleanup until the very end (#3042) ([libp2p/go-libp2p#3042](https://github.com/libp2p/go-libp2p/pull/3042))\n - fix(eventbus): Idempotent wildcardSub close (#3045) ([libp2p/go-libp2p#3045](https://github.com/libp2p/go-libp2p/pull/3045))\n - fix: obsaddr: do not record observations over relayed conn (#3043) ([libp2p/go-libp2p#3043](https://github.com/libp2p/go-libp2p/pull/3043))\n - fix(identify): push should not dial a new connection (#3035) ([libp2p/go-libp2p#3035](https://github.com/libp2p/go-libp2p/pull/3035))\n - webrtc: handshake more connections in parallel (#3040) ([libp2p/go-libp2p#3040](https://github.com/libp2p/go-libp2p/pull/3040))\n - eventbus: dont panic on closing Subscription twice (#3034) ([libp2p/go-libp2p#3034](https://github.com/libp2p/go-libp2p/pull/3034))\n - fix(swarm): incorrect error message format order (#3037) ([libp2p/go-libp2p#3037](https://github.com/libp2p/go-libp2p/pull/3037))\n - feat: eventbus: log error on slow consumers (#3031) ([libp2p/go-libp2p#3031](https://github.com/libp2p/go-libp2p/pull/3031))\n - chore: make funding.json uppercase to follow meta convention (#3028) ([libp2p/go-libp2p#3028](https://github.com/libp2p/go-libp2p/pull/3028))\n - chore: add drips entry to funding.json for Filecoin rPGF round 2\n - tcp: parameterize metrics collector (#3026) ([libp2p/go-libp2p#3026](https://github.com/libp2p/go-libp2p/pull/3026))\n - fix: basichost: Use NegotiationTimeout as fallback timeout for NewStream (#3020) ([libp2p/go-libp2p#3020](https://github.com/libp2p/go-libp2p/pull/3020))\n - feat(tcpreuse): add options for sharing TCP listeners amongst TCP, WS and WSS transports (#2984) ([libp2p/go-libp2p#2984](https://github.com/libp2p/go-libp2p/pull/2984))\n - pnet: wrap underlying error when reading nonce fails (#2975) ([libp2p/go-libp2p#2975](https://github.com/libp2p/go-libp2p/pull/2975))\n- github.com/libp2p/go-libp2p-kad-dht (v0.28.1 -> v0.28.2):\n - Release v0.28.2 (#1010) ([libp2p/go-libp2p-kad-dht#1010](https://github.com/libp2p/go-libp2p-kad-dht/pull/1010))\n - accelerated-dht: cleanup peer from message sender on disconnection (#1009) ([libp2p/go-libp2p-kad-dht#1009](https://github.com/libp2p/go-libp2p-kad-dht/pull/1009))\n - chore: fix some function names in comment ([libp2p/go-libp2p-kad-dht#1004](https://github.com/libp2p/go-libp2p-kad-dht/pull/1004))\n - feat: add more attributes to traces ([libp2p/go-libp2p-kad-dht#1002](https://github.com/libp2p/go-libp2p-kad-dht/pull/1002))\n- github.com/libp2p/go-netroute (v0.2.1 -> v0.2.2):\n - v0.2.2 Includes v4/v6 confusion fix for bsd route parsing\n - #50, Don't transform v4 routes to their v6 form on bsd ([libp2p/go-netroute#51](https://github.com/libp2p/go-netroute/pull/51))\n - Using syscall.RtMsg on Linux ([libp2p/go-netroute#43](https://github.com/libp2p/go-netroute/pull/43))\n - add wasi build constraint for netroute_stub ([libp2p/go-netroute#38](https://github.com/libp2p/go-netroute/pull/38))\n - Stricter filtering of degenerate routes ([libp2p/go-netroute#33](https://github.com/libp2p/go-netroute/pull/33))\n - sync: update CI config files (#30) ([libp2p/go-netroute#30](https://github.com/libp2p/go-netroute/pull/30))\n- github.com/multiformats/go-multiaddr (v0.13.0 -> v0.14.0):\n - Release v0.14.0 ([multiformats/go-multiaddr#258](https://github.com/multiformats/go-multiaddr/pull/258))\n - feat: memory multiaddrs ([multiformats/go-multiaddr#256](https://github.com/multiformats/go-multiaddr/pull/256))\n - nit: validate ipcidr ([multiformats/go-multiaddr#247](https://github.com/multiformats/go-multiaddr/pull/247))\n - check for nil interfaces (#251) ([multiformats/go-multiaddr#251](https://github.com/multiformats/go-multiaddr/pull/251))\n - Make it safe to roundtrip SplitXXX and Join (#250) ([multiformats/go-multiaddr#250](https://github.com/multiformats/go-multiaddr/pull/250))\n- github.com/multiformats/go-multiaddr-dns (v0.4.0 -> v0.4.1):\n - Release v0.4.1\n - fix: If decapsulating is empty, skip it. (#65) ([multiformats/go-multiaddr-dns#65](https://github.com/multiformats/go-multiaddr-dns/pull/65))\n- github.com/multiformats/go-multistream (v0.5.0 -> v0.6.0):\n - release v0.6.0 ([multiformats/go-multistream#116](https://github.com/multiformats/go-multistream/pull/116))\n - fix: finish reading handshake on lazyConn close\n - feat: New error to highlight unrecognized responses\n - release v0.5.0 (#108) ([multiformats/go-multistream#108](https://github.com/multiformats/go-multistream/pull/108))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Andrew Gillis | 57 | +1995/-1718 | 191 |\n| Adin Schmahmann | 7 | +2552/-719 | 84 |\n| Marco Munizaga | 27 | +1036/-261 | 51 |\n| Hector Sanjuan | 21 | +789/-362 | 65 |\n| gammazero | 20 | +407/-419 | 40 |\n| sukun | 13 | +519/-233 | 30 |\n| Marcin Rataj | 34 | +426/-142 | 59 |\n| Marten Seemann | 2 | +11/-261 | 5 |\n| Dreamacro | 2 | +161/-68 | 5 |\n| Hlib Kanunnikov | 1 | +34/-65 | 4 |\n| bashkarev | 1 | +78/-5 | 2 |\n| Daniel Norman | 4 | +68/-12 | 6 |\n| Andi | 1 | +37/-32 | 20 |\n| hannahhoward | 1 | +35/-17 | 7 |\n| Carlos Peliciari | 2 | +19/-26 | 2 |\n| Cole Brown | 1 | +32/-0 | 3 |\n| Will Scott | 2 | +19/-7 | 3 |\n| Guillaume Michel | 1 | +21/-2 | 4 |\n| 7sunarni | 1 | +3/-19 | 1 |\n| Srdjan S | 1 | +11/-2 | 2 |\n| web3-bot | 2 | +6/-6 | 3 |\n| dashangcun | 1 | +2/-10 | 1 |\n| John | 3 | +6/-6 | 5 |\n| Daniel N | 3 | +8/-3 | 3 |\n| Ivan Shvedunov | 1 | +4/-6 | 2 |\n| Piotr Galar | 1 | +4/-4 | 2 |\n| Derek Nola | 2 | +4/-4 | 4 |\n| Bryer | 1 | +4/-4 | 1 |\n| Prithvi Shahi | 2 | +6/-1 | 2 |\n| Cameron Wood | 1 | +7/-0 | 1 |\n| wangjingcun | 1 | +3/-3 | 2 |\n| cuibuwei | 1 | +2/-2 | 2 |\n| Jorropo | 1 | +1/-3 | 1 |\n| 未月 | 1 | +1/-1 | 1 |\n| Ubuntu | 1 | +1/-1 | 1 |\n| Ryan MacArthur | 1 | +1/-1 | 1 |\n| Reymon | 1 | +1/-1 | 1 |\n| guillaumemichel | 1 | +1/-0 | 1 |\n\n## v0.33.1\n\n### 🔦 Highlights\n\n#### Bitswap improvements from Boxo\n\nThis release includes performance and reliability improvements and fixes for minor resource leaks. One of the performance changes [greatly improves the bitswap clients ability to operate under high load](https://github.com/ipfs/boxo/pull/817#pullrequestreview-2587207745), that could previously result in an out of memory condition.\n\n#### Improved IPNS interop\n\nImproved compatibility with third-party IPNS publishers by restoring support for compact binary CIDs in the `Value` field of IPNS Records ([IPNS Specs](https://specs.ipfs.tech/ipns/ipns-record/)). As long the signature is valid, Kubo will now resolve such records (likely created by non-Kubo nodes) and convert raw CIDs into valid `/ipfs/cid` content paths.\n**Note:** This only adds support for resolving externally created records—Kubo’s IPNS record creation remains unchanged. IPNS records with empty `Value` fields default to zero-length `/ipfs/bafkqaaa` to maintain backward compatibility with code expecting a valid content path.\n\n#### 📦️ Important dependency updates\n\n- update `boxo` to [v0.27.4](https://github.com/ipfs/boxo/releases/tag/v0.27.4) (incl. [v0.27.3](https://github.com/ipfs/boxo/releases/tag/v0.27.3))\n\n### 📝 Changelog\n\n
Full Changelog v0.33.1\n\n- github.com/ipfs/kubo:\n - chore: v0.33.1\n - fix: boxo v0.27.4 (#10692) ([ipfs/kubo#10692](https://github.com/ipfs/kubo/pull/10692))\n - docs: add webrtc-direct fixes to 0.33 release changelog (#10688) ([ipfs/kubo#10688](https://github.com/ipfs/kubo/pull/10688))\n - fix: config help (#10686) ([ipfs/kubo#10686](https://github.com/ipfs/kubo/pull/10686))\n- github.com/ipfs/boxo (v0.27.2 -> v0.27.4):\n - Release v0.27.4 ([ipfs/boxo#832](https://github.com/ipfs/boxo/pull/832))\n - fix(ipns): reading records with raw []byte Value (#830) ([ipfs/boxo#830](https://github.com/ipfs/boxo/pull/830))\n - fix(bitswap): blockpresencemanager leak (#833) ([ipfs/boxo#833](https://github.com/ipfs/boxo/pull/833))\n - Always send cancels even if peer has no interest (#829) ([ipfs/boxo#829](https://github.com/ipfs/boxo/pull/829))\n - tidy changelog ([ipfs/boxo#828](https://github.com/ipfs/boxo/pull/828))\n - Update changelog (#827) ([ipfs/boxo#827](https://github.com/ipfs/boxo/pull/827))\n - fix(bitswap): filter interests from received messages (#822) ([ipfs/boxo#822](https://github.com/ipfs/boxo/pull/822))\n - Reduce unnecessary logging work (#826) ([ipfs/boxo#826](https://github.com/ipfs/boxo/pull/826))\n - fix: bitswap lock contention under high load (#817) ([ipfs/boxo#817](https://github.com/ipfs/boxo/pull/817))\n - fix: bitswap simplify cancel (#824) ([ipfs/boxo#824](https://github.com/ipfs/boxo/pull/824))\n - fix(bitswap): simplify SessionInterestManager (#821) ([ipfs/boxo#821](https://github.com/ipfs/boxo/pull/821))\n - feat: Better self-service commands for DHT providing (#815) ([ipfs/boxo#815](https://github.com/ipfs/boxo/pull/815))\n - bitswap/client: fewer wantlist iterations in sendCancels (#819) ([ipfs/boxo#819](https://github.com/ipfs/boxo/pull/819))\n - style: cleanup code by golangci-lint (#797) ([ipfs/boxo#797](https://github.com/ipfs/boxo/pull/797))\n - Move long messagequeue comment to doc.go (#814) ([ipfs/boxo#814](https://github.com/ipfs/boxo/pull/814))\n - Describe how bitswap message queue works ([ipfs/boxo#813](https://github.com/ipfs/boxo/pull/813))\n\n
\n\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Dreamacro | 1 | +304/-376 | 119 |\n| Andrew Gillis | 7 | +306/-200 | 20 |\n| Guillaume Michel | 5 | +122/-98 | 14 |\n| Marcin Rataj | 2 | +113/-7 | 4 |\n| gammazero | 6 | +41/-11 | 6 |\n| Sergey Gorbunov | 1 | +14/-2 | 2 |\n| Daniel Norman | 1 | +9/-0 | 1 |\n\n## v0.33.2\n\n### 🔦 Highlights\n\n#### 📦️ Important dependency updates\n\n- update `go-libp2p` to [v0.38.3](https://github.com/libp2p/go-libp2p/releases/tag/v0.38.3)\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: v0.33.2\n- github.com/libp2p/go-libp2p (v0.38.2 -> v0.38.3):\n - Release v0.38.3 (#3184) ([libp2p/go-libp2p#3184](https://github.com/libp2p/go-libp2p/pull/3184))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| sukun | 1 | +122/-23 | 7 |\n| Marcin Rataj | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.34.md", "content": "# Kubo changelog v0.34\n\n\n\nThis release was brought to you by the [Shipyard](http://ipshipyard.com/) team.\n\n- [v0.34.0](#v0340)\n- [v0.34.1](#v0341)\n\n## v0.34.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [AutoTLS now enabled by default for nodes with 1 hour uptime](#autotls-now-enabled-by-default-for-nodes-with-1-hour-uptime)\n - [New WebUI features](#new-webui-features)\n - [RPC and CLI command changes](#rpc-and-cli-command-changes)\n - [Bitswap improvements from Boxo](#bitswap-improvements-from-boxo)\n - [IPNS publishing TTL change](#ipns-publishing-ttl-change)\n - [`IPFS_LOG_LEVEL` deprecated](#ipfs_log_level-deprecated)\n - [Pebble datastore format update](#pebble-datastore-format-update)\n - [Badger datastore update](#badger-datastore-update)\n - [Datastore Implementation Updates](#datastore-implementation-updates)\n - [One Multi-error Package](#one-multi-error-package)\n - [Fix hanging pinset operations during reprovides](#fix-hanging-pinset-operations-during-reprovides)\n - [📦️ Important dependency updates](#-important-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### AutoTLS now enabled by default for nodes with 1 hour uptime\n\nStarting now, any publicly dialable Kubo node with a `/tcp` listener that remains online for at least one hour will receive a TLS certificate through the [`AutoTLS`](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls) feature.\nThis occurs automatically, with no need for manual setup.\n\nTo bypass the 1-hour delay and enable AutoTLS immediately, users can explicitly opt-in by running the following commands:\n\n```console\n$ ipfs config --json AutoTLS.Enabled true\n$ ipfs config --json AutoTLS.RegistrationDelay 0\n```\n\nAutoTLS will remain disabled under the following conditions:\n\n- The node already has a manually configured `/ws` (WebSocket) listener\n- A private network is in use with a `swarm.key`\n- TCP or WebSocket transports are disabled, or there is no `/tcp` listener\n\nTo troubleshoot, use `GOLOG_LOG_LEVEL=\"error,autotls=info`.\n\nFor more details, check out the [`AutoTLS` configuration documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls) or dive deeper with [AutoTLS libp2p blog post](https://web.archive.org/web/20260112031855/https://blog.libp2p.io/autotls/).\n\n#### New WebUI features\n\nThe WebUI, accessible at http://127.0.0.1:5001/webui/, now includes support for CAR file import and QR code sharing directly from the Files view. Additionally, the Peers screen has been updated with the latest [`ipfs-geoip`](https://www.npmjs.com/package/ipfs-geoip) dataset.\n\n#### RPC and CLI command changes\n\n- `ipfs config` is now validating json fields ([#10679](https://github.com/ipfs/kubo/pull/10679)).\n- Deprecated the `bitswap reprovide` command. Make sure to switch to modern `routing reprovide`. ([#10677](https://github.com/ipfs/kubo/pull/10677))\n- The `stats reprovide` command now shows additional stats for [`Routing.AcceleratedDHTClient`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingaccelerateddhtclient), indicating the last and next `reprovide` times. ([#10677](https://github.com/ipfs/kubo/pull/10677))\n- `ipfs files cp` now performs basic codec check and will error when source is not a valid UnixFS (only `dag-pb` and `raw` codecs are allowed in MFS)\n\n#### Bitswap improvements from Boxo\n\nThis release includes performance and reliability improvements and fixes for minor resource leaks. One of the performance changes [greatly improves the bitswap clients ability to operate under high load](https://github.com/ipfs/boxo/pull/817#pullrequestreview-2587207745), that could previously result in an out of memory condition.\n\n#### IPNS publishing TTL change\n\nMany complaints about IPNS being slow are tied to the default `--ttl` in `ipfs name publish`, which was set to 1 hour. To address this, we’ve lowered the default [IPNS Record TTL](https://specs.ipfs.tech/ipns/ipns-record/#ttl-uint64) during publishing to 5 minutes, matching similar TTL defaults in DNS. This update is now part of `boxo/ipfs` (GO, [boxo#859](https://github.com/ipfs/boxo/pull/859)) and `@helia/ipns` (JS, [helia#749](https://github.com/ipfs/helia/pull/749)).\n\n> [!TIP]\n> IPNS TTL recommendations when even faster update propagation is desired:\n> - **As a Publisher:** Lower the `--ttl` (e.g., `ipfs name publish --ttl=1m`) to further reduce caching delays. If using DNSLink, ensure the DNS TXT record TTL matches the IPNS record TTL.\n> - **As a Gateway Operator:** Override publisher TTLs for faster updates using configurations like [`Ipns.MaxCacheTTL`](https://github.com/ipfs/kubo/blob/master/docs/config.md#ipnsmaxcachettl) in Kubo or [`RAINBOW_IPNS_MAX_CACHE_TTL`](https://github.com/ipfs/rainbow/blob/main/docs/environment-variables.md#rainbow_ipns_max_cache_ttl) in [Rainbow](https://github.com/ipfs/rainbow/).\n\n#### `IPFS_LOG_LEVEL` deprecated\n\nThe variable has been deprecated. Please use [`GOLOG_LOG_LEVEL`](https://github.com/ipfs/kubo/blob/master/docs/environment-variables.md#golog_log_level) instead for configuring logging levels.\n\n#### Pebble datastore format update\n\nIf the pebble database format is not explicitly set in the config, then automatically upgrade it to the latest format version supported by the release ob pebble used by kubo. This will ensure that the database format is sufficiently up-to-date to be compatible with a major version upgrade of pebble. This is necessary before upgrading to use pebble v2.\n\n#### Badger datastore update\n\nAn update was made to the badger v1 datastore that avoids use of mmap in 32-bit environments, which has been seen to cause issues on some platforms. Please be aware that this could lead to a performance regression for users of badger in a 32-bit environment. Badger users are advised to move to the flatds or pebble datastore.\n\n#### Datastore Implementation Updates\n\nThe go-ds-xxx datastore implementations have been updated to support the updated `go-datastore` [v0.8.2](https://github.com/ipfs/go-datastore/releases/tag/v0.8.2) query API. This update removes the datastore implementations' dependency on `goprocess` and updates the query API.\n\n#### One Multi-error Package\n\nKubo previously depended on multiple multi-error packages, `github.com/hashicorp/go-multierror` and `go.uber.org/multierr`. These have nearly identical functionality so there was no need to use both. Therefore, `go.uber.org/multierr` was selected as the package to depend on. Any future code needing multi-error functionality should use `go.uber.org/multierr` to avoid introducing unneeded dependencies.\n\n#### Fix hanging pinset operations during reprovides\n\nThe reprovide process can be quite slow. In default settings, the reprovide process will start reading CIDs that belong to the pinset. During this operation, starvation can occur for other operations that need pinset access (see https://github.com/ipfs/kubo/issues/10596).\n\nWe have now switch to buffering pinset-related cids that are going to be reprovided in memory, so that we can free pinset mutexes as soon as possible so that pinset-writes and subsequent read operations can proceed. The downside is larger pinsets will need some extra memory, with an estimation of ~1GiB of RAM memory-use per 20 million items to be reprovided.\n\nUse [`Reprovider.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy) to balance announcement prioritization, speed, and memory utilization.\n\n#### 📦️ Important dependency updates\n\n- update `go-libp2p` to [v0.41.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.41.0) (incl. [v0.40.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.40.0))\n- update `go-libp2p-kad-dht` to [v0.30.2](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.30.2) (incl. [v0.29.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.29.0), [v0.29.1](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.29.1), [v0.29.2](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.29.2), [v0.30.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.30.0), [v0.30.1](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.30.1))\n- update `boxo` to [v0.29.1](https://github.com/ipfs/boxo/releases/tag/v0.29.1) (incl. [v0.28.0](https://github.com/ipfs/boxo/releases/tag/v0.28.0) [v0.29.0](https://github.com/ipfs/boxo/releases/tag/v0.29.0))\n- update `ipfs-webui` to [v4.6.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.6.0) (incl. [v4.5.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.5.0))\n- update `p2p-forge/client` to [v0.4.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.4.0)\n- update `go-datastore` to [v0.8.2](https://github.com/ipfs/go-datastore/releases/tag/v0.8.2) (incl. [v0.7.0](https://github.com/ipfs/go-datastore/releases/tag/v0.7.0), [v0.8.0](https://github.com/ipfs/go-datastore/releases/tag/v0.8.0))\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: v0.34.0\n - chore: v0.34.0-rc2\n - docs: mention Reprovider.Strategy config\n - docs: ipns ttl change\n - feat: ipfs-webui v4.6 (#10756) ([ipfs/kubo#10756](https://github.com/ipfs/kubo/pull/10756))\n - docs(readme): update min. requirements + cleanup (#10750) ([ipfs/kubo#10750](https://github.com/ipfs/kubo/pull/10750))\n - Upgrade to Boxo v0.29.1 (#10755) ([ipfs/kubo#10755](https://github.com/ipfs/kubo/pull/10755))\n - Nonfunctional (#10753) ([ipfs/kubo#10753](https://github.com/ipfs/kubo/pull/10753))\n - Update docs/changelogs/v0.34.md\n - provider: buffer pin providers.\n - chore: 0.34.0-rc1\n - fix(mfs): basic UnixFS sanity checks in `files cp` (#10701) ([ipfs/kubo#10701](https://github.com/ipfs/kubo/pull/10701))\n - Upgrade to Boxo v0.29.0 (#10742) ([ipfs/kubo#10742](https://github.com/ipfs/kubo/pull/10742))\n - use go-datastore without go-process (#10736) ([ipfs/kubo#10736](https://github.com/ipfs/kubo/pull/10736))\n - docs(config): add security considerations for rpc (#10739) ([ipfs/kubo#10739](https://github.com/ipfs/kubo/pull/10739))\n - chore: update go-libp2p to v0.41.0 (#10733) ([ipfs/kubo#10733](https://github.com/ipfs/kubo/pull/10733))\n - feat: ipfs-webui v4.5.0 (#10735) ([ipfs/kubo#10735](https://github.com/ipfs/kubo/pull/10735))\n - Create FUNDING.json (#10734) ([ipfs/kubo#10734](https://github.com/ipfs/kubo/pull/10734))\n - feat(AutoTLS): enabled by default with 1h RegistrationDelay (#10724) ([ipfs/kubo#10724](https://github.com/ipfs/kubo/pull/10724))\n - Upgrade to Boxo v0.28.0 (#10725) ([ipfs/kubo#10725](https://github.com/ipfs/kubo/pull/10725))\n - Upgrade to go1.24 (#10726) ([ipfs/kubo#10726](https://github.com/ipfs/kubo/pull/10726))\n - Replace go-random with random-data from go-test package (#10731) ([ipfs/kubo#10731](https://github.com/ipfs/kubo/pull/10731))\n - Update to new go-test (#10729) ([ipfs/kubo#10729](https://github.com/ipfs/kubo/pull/10729))\n - Update go-test and use new random-files generator (#10728) ([ipfs/kubo#10728](https://github.com/ipfs/kubo/pull/10728))\n - docs(readme): update docker section (#10716) ([ipfs/kubo#10716](https://github.com/ipfs/kubo/pull/10716))\n - Update go-ds-badger to v0.3.1 (#10722) ([ipfs/kubo#10722](https://github.com/ipfs/kubo/pull/10722))\n - Update pebble db to latest format by default (#10720) ([ipfs/kubo#10720](https://github.com/ipfs/kubo/pull/10720))\n - fix: switch away from IPFS_LOG_LEVEL (#10694) ([ipfs/kubo#10694](https://github.com/ipfs/kubo/pull/10694))\n - Merge release v0.33.2 ([ipfs/kubo#10713](https://github.com/ipfs/kubo/pull/10713))\n - Remove unused TimeParts struct (#10708) ([ipfs/kubo#10708](https://github.com/ipfs/kubo/pull/10708))\n - fix(rpc): restore and deprecate `bitswap reprovide` (#10699) ([ipfs/kubo#10699](https://github.com/ipfs/kubo/pull/10699))\n - docs(release): update RELEASE_CHECKLIST.md after v0.33.1 (#10697) ([ipfs/kubo#10697](https://github.com/ipfs/kubo/pull/10697))\n - docs: update min requirements (#10687) ([ipfs/kubo#10687](https://github.com/ipfs/kubo/pull/10687))\n - Merge release v0.33.1 ([ipfs/kubo#10698](https://github.com/ipfs/kubo/pull/10698))\n - fix: boxo v0.27.4 (#10692) ([ipfs/kubo#10692](https://github.com/ipfs/kubo/pull/10692))\n - fix: Issue #9364 JSON config validation (#10679) ([ipfs/kubo#10679](https://github.com/ipfs/kubo/pull/10679))\n - docs: RELEASE_CHECKLIST.md update for 0.33 (#10674) ([ipfs/kubo#10674](https://github.com/ipfs/kubo/pull/10674))\n - feat: Better self-service commands for DHT providing (#10677) ([ipfs/kubo#10677](https://github.com/ipfs/kubo/pull/10677))\n - docs: add webrtc-direct fixes to 0.33 release changelog (#10688) ([ipfs/kubo#10688](https://github.com/ipfs/kubo/pull/10688))\n - fix: config help (#10686) ([ipfs/kubo#10686](https://github.com/ipfs/kubo/pull/10686))\n - feat: Add CI for Spell Checking (#10637) ([ipfs/kubo#10637](https://github.com/ipfs/kubo/pull/10637))\n - Merge release v0.33.0 ([ipfs/kubo#10684](https://github.com/ipfs/kubo/pull/10684))\n - test: fix the socat tests after the ubuntu 24.04 upgrade (#10683) ([ipfs/kubo#10683](https://github.com/ipfs/kubo/pull/10683))\n - fix: quic-go v0.49.0 (#10673) ([ipfs/kubo#10673](https://github.com/ipfs/kubo/pull/10673))\n - Upgrade to Boxo v0.27.2 (#10672) ([ipfs/kubo#10672](https://github.com/ipfs/kubo/pull/10672))\n - Upgrade to Boxo v0.27.1 (#10671) ([ipfs/kubo#10671](https://github.com/ipfs/kubo/pull/10671))\n - fix(autotls): renewal and AutoTLS.ShortAddrs (#10669) ([ipfs/kubo#10669](https://github.com/ipfs/kubo/pull/10669))\n - update changelog for boxo and go-libp2p (#10668) ([ipfs/kubo#10668](https://github.com/ipfs/kubo/pull/10668))\n - Upgrade to Boxo v0.27.0 (#10665) ([ipfs/kubo#10665](https://github.com/ipfs/kubo/pull/10665))\n - update dependencies (#10664) ([ipfs/kubo#10664](https://github.com/ipfs/kubo/pull/10664))\n - docs(readme): add unofficial Fedora COPR (#10660) ([ipfs/kubo#10660](https://github.com/ipfs/kubo/pull/10660))\n - fix(dns): update default DNSLink resolvers (#10655) ([ipfs/kubo#10655](https://github.com/ipfs/kubo/pull/10655))\n - chore: p2p-forge v0.2.2 + go-libp2p-kad-dht v0.28.2 (#10663) ([ipfs/kubo#10663](https://github.com/ipfs/kubo/pull/10663))\n - fix(cli): support HTTPS in ipfs --api (#10659) ([ipfs/kubo#10659](https://github.com/ipfs/kubo/pull/10659))\n - chore: fix typos and comment formatting (#10653) ([ipfs/kubo#10653](https://github.com/ipfs/kubo/pull/10653))\n - fix/gateway: escape directory redirect url (#10649) ([ipfs/kubo#10649](https://github.com/ipfs/kubo/pull/10649))\n - Add example of setting array to config command help ([ipfs/kubo#10650](https://github.com/ipfs/kubo/pull/10650))\n - collection of typo fixes (#10647) ([ipfs/kubo#10647](https://github.com/ipfs/kubo/pull/10647))\n - chore: bump master to 0.34.0-dev\n- github.com/ipfs/boxo (v0.27.4 -> v0.29.1):\n - Release v0.29.1 ([ipfs/boxo#885](https://github.com/ipfs/boxo/pull/885))\n - fix(provider): call reprovider throughput callback only if reprovide is enabled (#871) ([ipfs/boxo#871](https://github.com/ipfs/boxo/pull/871))\n - bitswap/httpnet: do not follow redirects (#878) ([ipfs/boxo#878](https://github.com/ipfs/boxo/pull/878))\n - Refactor(hostname): Skip DNSLink for local IP addresses to avoid DNS queries (#880) ([ipfs/boxo#880](https://github.com/ipfs/boxo/pull/880))\n - Nonfunctional (#882) ([ipfs/boxo#882](https://github.com/ipfs/boxo/pull/882))\n - fix(bitswap/client): dont set nil for DontHaveTimeoutConfig (#872) ([ipfs/boxo#872](https://github.com/ipfs/boxo/pull/872))\n - provider: add a buffered KeyChanFunc. ([ipfs/boxo#870](https://github.com/ipfs/boxo/pull/870))\n - Release v0.29.0 (#869) ([ipfs/boxo#869](https://github.com/ipfs/boxo/pull/869))\n - Do not use multiple multi-error packages, pick one (#867) ([ipfs/boxo#867](https://github.com/ipfs/boxo/pull/867))\n - feat(bitswap/client): MinTimeout for DontHaveTimeoutConfig (#865) ([ipfs/boxo#865](https://github.com/ipfs/boxo/pull/865))\n - use go-datastore without go-process (#858) ([ipfs/boxo#858](https://github.com/ipfs/boxo/pull/858))\n - minimize peermanager lock scope (#860) ([ipfs/boxo#860](https://github.com/ipfs/boxo/pull/860))\n - chore(ipns): lower `DefaultRecordTTL` to 5m (#859) ([ipfs/boxo#859](https://github.com/ipfs/boxo/pull/859))\n - httpnet: bitswap network for HTTP block retrieval over trustless gateway endpoints. ([ipfs/boxo#747](https://github.com/ipfs/boxo/pull/747))\n - chore: Update FUNDING.json for Optimism RPF (#857) ([ipfs/boxo#857](https://github.com/ipfs/boxo/pull/857))\n - Release v0.28.0 (#854) ([ipfs/boxo#854](https://github.com/ipfs/boxo/pull/854))\n - Update deps (#852) ([ipfs/boxo#852](https://github.com/ipfs/boxo/pull/852))\n - fix: gateway/blocks-backend: GetBlock should not perform IPLD decoding (#845) ([ipfs/boxo#845](https://github.com/ipfs/boxo/pull/845))\n - Protobuf pkg name (#850) ([ipfs/boxo#850](https://github.com/ipfs/boxo/pull/850))\n - Fix intermittent test failure (#849) ([ipfs/boxo#849](https://github.com/ipfs/boxo/pull/849))\n - move `ipld/merkledag` from gogo protobuf (#841) ([ipfs/boxo#841](https://github.com/ipfs/boxo/pull/841))\n - move `ipld/unixfs` from gogo protobuf (#840) ([ipfs/boxo#840](https://github.com/ipfs/boxo/pull/840))\n - Start moving from gogo protobuf (#839) ([ipfs/boxo#839](https://github.com/ipfs/boxo/pull/839))\n - ci: uci/update-go (#848) ([ipfs/boxo#848](https://github.com/ipfs/boxo/pull/848))\n - expose DontHaveTimeoutConfig (#846) ([ipfs/boxo#846](https://github.com/ipfs/boxo/pull/846))\n - Upgrade go-libp2p to v0.39.1 (#843) ([ipfs/boxo#843](https://github.com/ipfs/boxo/pull/843))\n - feat: Prevent multiple instances of \"ipfs routing reprovide\" running together. (#834) ([ipfs/boxo#834](https://github.com/ipfs/boxo/pull/834))\n - Upgrade to go-libp2p v0.39.0 (#837) ([ipfs/boxo#837](https://github.com/ipfs/boxo/pull/837))\n - bitswap/client/internal/messagequeue: run tests in parallel (#835) ([ipfs/boxo#835](https://github.com/ipfs/boxo/pull/835))\n- github.com/ipfs/go-cid (v0.4.1 -> v0.5.0):\n - v0.5.0 bump (#172) ([ipfs/go-cid#172](https://github.com/ipfs/go-cid/pull/172))\n - move _rsrch/cidiface into an internal package\n- github.com/ipfs/go-datastore (v0.6.0 -> v0.8.2):\n - bump version (#231) ([ipfs/go-datastore#231](https://github.com/ipfs/go-datastore/pull/231))\n - Results.Close should return error (#230) ([ipfs/go-datastore#230](https://github.com/ipfs/go-datastore/pull/230))\n - new version (#229) ([ipfs/go-datastore#229](https://github.com/ipfs/go-datastore/pull/229))\n - Update fuzz module dependencies (#228) ([ipfs/go-datastore#228](https://github.com/ipfs/go-datastore/pull/228))\n - new version (#225) ([ipfs/go-datastore#225](https://github.com/ipfs/go-datastore/pull/225))\n - No goprocess (#223) ([ipfs/go-datastore#223](https://github.com/ipfs/go-datastore/pull/223))\n - Release version 0.7.0 (#213) ([ipfs/go-datastore#213](https://github.com/ipfs/go-datastore/pull/213))\n - query result ordering does not create additional goroutine (#221) ([ipfs/go-datastore#221](https://github.com/ipfs/go-datastore/pull/221))\n - Remove unneeded dependencies (#220) ([ipfs/go-datastore#220](https://github.com/ipfs/go-datastore/pull/220))\n - Add traced datastore (#209) ([ipfs/go-datastore#209](https://github.com/ipfs/go-datastore/pull/209))\n - Add root namespace method to Key (#208) ([ipfs/go-datastore#208](https://github.com/ipfs/go-datastore/pull/208))\n - ci: uci/copy-templates (#207) ([ipfs/go-datastore#207](https://github.com/ipfs/go-datastore/pull/207))\n - test: fix fuzz commands\n - fix fuzz tests by adding the missing context.Context argument (#198) ([ipfs/go-datastore#198](https://github.com/ipfs/go-datastore/pull/198))\n - sync: update CI config files (#195) ([ipfs/go-datastore#195](https://github.com/ipfs/go-datastore/pull/195))\n- github.com/ipfs/go-ds-badger (v0.3.0 -> v0.3.4):\n - new version (#137) ([ipfs/go-ds-badger#137](https://github.com/ipfs/go-ds-badger/pull/137))\n - new version (#135) ([ipfs/go-ds-badger#135](https://github.com/ipfs/go-ds-badger/pull/135))\n - new version (#132) ([ipfs/go-ds-badger#132](https://github.com/ipfs/go-ds-badger/pull/132))\n - Update to use go-datastore without go-process (#131) ([ipfs/go-ds-badger#131](https://github.com/ipfs/go-ds-badger/pull/131))\n - new version ([ipfs/go-ds-badger#128](https://github.com/ipfs/go-ds-badger/pull/128))\n - Update dependencies and minimum go version ([ipfs/go-ds-badger#127](https://github.com/ipfs/go-ds-badger/pull/127))\n - ci: uci/update-go ([ipfs/go-ds-badger#123](https://github.com/ipfs/go-ds-badger/pull/123))\n - ci: uci/copy-templates ([ipfs/go-ds-badger#122](https://github.com/ipfs/go-ds-badger/pull/122))\n - chore: check PersistentDatastore conformance at build time (#120) ([ipfs/go-ds-badger#120](https://github.com/ipfs/go-ds-badger/pull/120))\n- github.com/ipfs/go-ds-flatfs (v0.5.1 -> v0.5.5):\n - bump version (#130) ([ipfs/go-ds-flatfs#130](https://github.com/ipfs/go-ds-flatfs/pull/130))\n - new version (#128) ([ipfs/go-ds-flatfs#128](https://github.com/ipfs/go-ds-flatfs/pull/128))\n - new version (#126) ([ipfs/go-ds-flatfs#126](https://github.com/ipfs/go-ds-flatfs/pull/126))\n - Fix race condition due to concurrent use of rand source (#125) ([ipfs/go-ds-flatfs#125](https://github.com/ipfs/go-ds-flatfs/pull/125))\n - new version ([ipfs/go-ds-flatfs#124](https://github.com/ipfs/go-ds-flatfs/pull/124))\n - Use go-datastore without go-process ([ipfs/go-ds-flatfs#123](https://github.com/ipfs/go-ds-flatfs/pull/123))\n - ci: uci/update-go (#122) ([ipfs/go-ds-flatfs#122](https://github.com/ipfs/go-ds-flatfs/pull/122))\n - fix: actually use the size hint in util_windows.go\n - perf: do not use virtual call when passing os.Rename as rename\n - chore(logging): update go-log v2 (#117) ([ipfs/go-ds-flatfs#117](https://github.com/ipfs/go-ds-flatfs/pull/117))\n - ci: uci/copy-templates ([ipfs/go-ds-flatfs#116](https://github.com/ipfs/go-ds-flatfs/pull/116))\n - sync: update CI config files ([ipfs/go-ds-flatfs#111](https://github.com/ipfs/go-ds-flatfs/pull/111))\n - possibly fix a bug in renameAndUpdateDiskUsage\n - add documentation and comment\n - perf: avoid syncing directories when they already existed (#107) ([ipfs/go-ds-flatfs#107](https://github.com/ipfs/go-ds-flatfs/pull/107))\n - test: faster TestNoCluster by batching the 3200 Puts ([ipfs/go-ds-flatfs#108](https://github.com/ipfs/go-ds-flatfs/pull/108))\n - query: also teard down on ctx done (#106) ([ipfs/go-ds-flatfs#106](https://github.com/ipfs/go-ds-flatfs/pull/106))\n- github.com/ipfs/go-ds-leveldb (v0.5.0 -> v0.5.2):\n - new version (#75) ([ipfs/go-ds-leveldb#75](https://github.com/ipfs/go-ds-leveldb/pull/75))\n - Results close needs to return error (#74) ([ipfs/go-ds-leveldb#74](https://github.com/ipfs/go-ds-leveldb/pull/74))\n - new version ([ipfs/go-ds-leveldb#73](https://github.com/ipfs/go-ds-leveldb/pull/73))\n - use go-datastore without go-process ([ipfs/go-ds-leveldb#72](https://github.com/ipfs/go-ds-leveldb/pull/72))\n - sync: update CI config files (#62) ([ipfs/go-ds-leveldb#62](https://github.com/ipfs/go-ds-leveldb/pull/62))\n - chore: add PersistentDatastore and Batching interface checks\n- github.com/ipfs/go-ds-measure (v0.2.0 -> v0.2.2):\n - new version ([ipfs/go-ds-measure#54](https://github.com/ipfs/go-ds-measure/pull/54))\n - new version ([ipfs/go-ds-measure#52](https://github.com/ipfs/go-ds-measure/pull/52))\n- github.com/ipfs/go-ds-pebble (v0.4.2 -> v0.4.4):\n - new version (#51) ([ipfs/go-ds-pebble#51](https://github.com/ipfs/go-ds-pebble/pull/51))\n - new version (#48) ([ipfs/go-ds-pebble#48](https://github.com/ipfs/go-ds-pebble/pull/48))\n - Use go-datastore without go-process (#47) ([ipfs/go-ds-pebble#47](https://github.com/ipfs/go-ds-pebble/pull/47))\n- github.com/ipfs/go-metrics-interface (v0.0.1 -> v0.3.0):\n - CounterVec: even more ergonomic ([ipfs/go-metrics-interface#22](https://github.com/ipfs/go-metrics-interface/pull/22))\n - Improve CounterVec abstraction ([ipfs/go-metrics-interface#21](https://github.com/ipfs/go-metrics-interface/pull/21))\n - v0.1.0 ([ipfs/go-metrics-interface#20](https://github.com/ipfs/go-metrics-interface/pull/20))\n - Feat: Add CounterVec type. ([ipfs/go-metrics-interface#19](https://github.com/ipfs/go-metrics-interface/pull/19))\n - sync: update CI config files (#10) ([ipfs/go-metrics-interface#10](https://github.com/ipfs/go-metrics-interface/pull/10))\n - sync: update CI config files (#8) ([ipfs/go-metrics-interface#8](https://github.com/ipfs/go-metrics-interface/pull/8))\n - use a struct as a key for the context ([ipfs/go-metrics-interface#4](https://github.com/ipfs/go-metrics-interface/pull/4))\n- github.com/ipfs/go-metrics-prometheus (v0.0.3 -> v0.1.0):\n - Implement the CounterVec type. ([ipfs/go-metrics-prometheus#26](https://github.com/ipfs/go-metrics-prometheus/pull/26))\n- github.com/ipfs/go-test (v0.0.4 -> v0.2.1):\n - new version (#20) ([ipfs/go-test#20](https://github.com/ipfs/go-test/pull/20))\n - No newline at end of random raw data (#19) ([ipfs/go-test#19](https://github.com/ipfs/go-test/pull/19))\n - new-version (#18) ([ipfs/go-test#18](https://github.com/ipfs/go-test/pull/18))\n - new version (#15) ([ipfs/go-test#15](https://github.com/ipfs/go-test/pull/15))\n - refactor: Make go-multiaddr v0.15 forward compatible change (#16) ([ipfs/go-test#16](https://github.com/ipfs/go-test/pull/16))\n - Move cli apps (#17) ([ipfs/go-test#17](https://github.com/ipfs/go-test/pull/17))\n - Update help text (#14) ([ipfs/go-test#14](https://github.com/ipfs/go-test/pull/14))\n - Add package to generate random filesystem hierarchies for testing (#13) ([ipfs/go-test#13](https://github.com/ipfs/go-test/pull/13))\n- github.com/ipfs/go-unixfsnode (v1.9.2 -> v1.10.0):\n - new version ([ipfs/go-unixfsnode#81](https://github.com/ipfs/go-unixfsnode/pull/81))\n - upgrade to boxo v0.27.4 ([ipfs/go-unixfsnode#80](https://github.com/ipfs/go-unixfsnode/pull/80))\n- github.com/libp2p/go-libp2p (v0.38.3 -> v0.41.0):\n - Release v0.41.0 (#3210) ([libp2p/go-libp2p#3210](https://github.com/libp2p/go-libp2p/pull/3210))\n - fix(libp2phttp): Fix relative to absolute multiaddr URI logic (#3208) ([libp2p/go-libp2p#3208](https://github.com/libp2p/go-libp2p/pull/3208))\n - fix(dcutr): Fix end to end tests and add legacy behavior flag (default=true) (#3044) ([libp2p/go-libp2p#3044](https://github.com/libp2p/go-libp2p/pull/3044))\n - feat(libp2phttp): More ergonomic auth (#3188) ([libp2p/go-libp2p#3188](https://github.com/libp2p/go-libp2p/pull/3188))\n - chore(identify): move log to debug level (#3206) ([libp2p/go-libp2p#3206](https://github.com/libp2p/go-libp2p/pull/3206))\n - chore: Update go-multiaddr to v0.15 (#3145) ([libp2p/go-libp2p#3145](https://github.com/libp2p/go-libp2p/pull/3145))\n - chore: update quic-go to v0.50.0 (#3204) ([libp2p/go-libp2p#3204](https://github.com/libp2p/go-libp2p/pull/3204))\n - chore: move go-nat to internal package\n - basichost: add certhashes to addrs in place (#3200) ([libp2p/go-libp2p#3200](https://github.com/libp2p/go-libp2p/pull/3200))\n - autorelay: send addresses on eventbus; dont wrap address factory (#3071) ([libp2p/go-libp2p#3071](https://github.com/libp2p/go-libp2p/pull/3071))\n - chore: update ci for go1.24 (#3195) ([libp2p/go-libp2p#3195](https://github.com/libp2p/go-libp2p/pull/3195))\n - Release v0.40.0 (#3192) ([libp2p/go-libp2p#3192](https://github.com/libp2p/go-libp2p/pull/3192))\n - chore: bump deps for v0.40.0 (#3191) ([libp2p/go-libp2p#3191](https://github.com/libp2p/go-libp2p/pull/3191))\n - autonatv2: allow multiple concurrent requests per peer (#3187) ([libp2p/go-libp2p#3187](https://github.com/libp2p/go-libp2p/pull/3187))\n - feat: add AutoTLS example (#3103) ([libp2p/go-libp2p#3103](https://github.com/libp2p/go-libp2p/pull/3103))\n - feat(swarm): logging waitForDirectConn return error (#3183) ([libp2p/go-libp2p#3183](https://github.com/libp2p/go-libp2p/pull/3183))\n - tcpreuse: fix Scope() for *tls.Conn (#3181) ([libp2p/go-libp2p#3181](https://github.com/libp2p/go-libp2p/pull/3181))\n - test(p2p/protocol/identify): fix user agent assertion in Go 1.24 (#3177) ([libp2p/go-libp2p#3177](https://github.com/libp2p/go-libp2p/pull/3177))\n - swarm: remove unnecessary error log (#3128) ([libp2p/go-libp2p#3128](https://github.com/libp2p/go-libp2p/pull/3128))\n - Implement error codes spec (#2927) ([libp2p/go-libp2p#2927](https://github.com/libp2p/go-libp2p/pull/2927))\n - chore: update pion/ice to v4 (#3175) ([libp2p/go-libp2p#3175](https://github.com/libp2p/go-libp2p/pull/3175))\n - chore: release v0.39.0 (#3174) ([libp2p/go-libp2p#3174](https://github.com/libp2p/go-libp2p/pull/3174))\n - feat(holepunch): add logging when DirectConnect execution fails (#3146) ([libp2p/go-libp2p#3146](https://github.com/libp2p/go-libp2p/pull/3146))\n - feat: Implement Custom TCP Dialers (#3166) ([libp2p/go-libp2p#3166](https://github.com/libp2p/go-libp2p/pull/3166))\n - Update quic-go to v0.49.0 (#3153) ([libp2p/go-libp2p#3153](https://github.com/libp2p/go-libp2p/pull/3153))\n - feat(transport/websocket): support SOCKS proxy with ws(s) (#3137) ([libp2p/go-libp2p#3137](https://github.com/libp2p/go-libp2p/pull/3137))\n - tcpreuse: fix rcmgr accounting when tcp metrics are enabled (#3142) ([libp2p/go-libp2p#3142](https://github.com/libp2p/go-libp2p/pull/3142))\n - fix(net/nat): data race problem of `extAddr` (#3140) ([libp2p/go-libp2p#3140](https://github.com/libp2p/go-libp2p/pull/3140))\n - test: fix failing test (#3141) ([libp2p/go-libp2p#3141](https://github.com/libp2p/go-libp2p/pull/3141))\n - quicreuse: make it possible to use an application-constructed quic.Transport (#3122) ([libp2p/go-libp2p#3122](https://github.com/libp2p/go-libp2p/pull/3122))\n - nat: ignore mapping if external port is 0 (#3094) ([libp2p/go-libp2p#3094](https://github.com/libp2p/go-libp2p/pull/3094))\n - tcpreuse: error on using tcpreuse with pnet (#3129) ([libp2p/go-libp2p#3129](https://github.com/libp2p/go-libp2p/pull/3129))\n - chore: Update contribution guidelines (#3134) ([libp2p/go-libp2p#3134](https://github.com/libp2p/go-libp2p/pull/3134))\n - tcp: fix metrics test build directive (#3052) ([libp2p/go-libp2p#3052](https://github.com/libp2p/go-libp2p/pull/3052))\n - webrtc: upgrade pion/webrtc to v4 (#3098) ([libp2p/go-libp2p#3098](https://github.com/libp2p/go-libp2p/pull/3098))\n - webtransport: fix docstring comment for getCurrentBucketStartTime\n - chore: release v0.38.1 (#3114) ([libp2p/go-libp2p#3114](https://github.com/libp2p/go-libp2p/pull/3114))\n- github.com/libp2p/go-libp2p-kad-dht (v0.28.2 -> v0.30.2):\n - new version (#1059) ([libp2p/go-libp2p-kad-dht#1059](https://github.com/libp2p/go-libp2p-kad-dht/pull/1059))\n - do not use multiple multi-error packages, pick one (#1058) ([libp2p/go-libp2p-kad-dht#1058](https://github.com/libp2p/go-libp2p-kad-dht/pull/1058))\n - update version (#1057) ([libp2p/go-libp2p-kad-dht#1057](https://github.com/libp2p/go-libp2p-kad-dht/pull/1057))\n - chore: release v0.30.0 (#1054) ([libp2p/go-libp2p-kad-dht#1054](https://github.com/libp2p/go-libp2p-kad-dht/pull/1054))\n - fix: crawler polluting peerstore (#1053) ([libp2p/go-libp2p-kad-dht#1053](https://github.com/libp2p/go-libp2p-kad-dht/pull/1053))\n - new version (#1052) ([libp2p/go-libp2p-kad-dht#1052](https://github.com/libp2p/go-libp2p-kad-dht/pull/1052))\n - use go-datastore without go-process (#1051) ([libp2p/go-libp2p-kad-dht#1051](https://github.com/libp2p/go-libp2p-kad-dht/pull/1051))\n - feat: use OTEL for metrics (removes opencensus) (#1045) ([libp2p/go-libp2p-kad-dht#1045](https://github.com/libp2p/go-libp2p-kad-dht/pull/1045))\n - release v0.29.1 (#1042) ([libp2p/go-libp2p-kad-dht#1042](https://github.com/libp2p/go-libp2p-kad-dht/pull/1042))\n - fix: flaky TestInvalidServer (#1049) ([libp2p/go-libp2p-kad-dht#1049](https://github.com/libp2p/go-libp2p-kad-dht/pull/1049))\n - chore: update deps (#1048) ([libp2p/go-libp2p-kad-dht#1048](https://github.com/libp2p/go-libp2p-kad-dht/pull/1048))\n - fix addrsSoFar comparison (#1046) ([libp2p/go-libp2p-kad-dht#1046](https://github.com/libp2p/go-libp2p-kad-dht/pull/1046))\n - fix: flaky TestInvalidServer (#1043) ([libp2p/go-libp2p-kad-dht#1043](https://github.com/libp2p/go-libp2p-kad-dht/pull/1043))\n - add verbose to TestFindProviderAsync (dual) (#1040) ([libp2p/go-libp2p-kad-dht#1040](https://github.com/libp2p/go-libp2p-kad-dht/pull/1040))\n - test: cover dns addresses in TestAddrFilter (#1041) ([libp2p/go-libp2p-kad-dht#1041](https://github.com/libp2p/go-libp2p-kad-dht/pull/1041))\n - fix: flaky TestSearchValue (dual) (#1038) ([libp2p/go-libp2p-kad-dht#1038](https://github.com/libp2p/go-libp2p-kad-dht/pull/1038))\n - fix: flaky TestClientModeConnect (#1037) ([libp2p/go-libp2p-kad-dht#1037](https://github.com/libp2p/go-libp2p-kad-dht/pull/1037))\n - fix: flaky TestFindPeerQueryMinimal (#1036) ([libp2p/go-libp2p-kad-dht#1036](https://github.com/libp2p/go-libp2p-kad-dht/pull/1036))\n - fix: flaky TestInvalidServer (#1032) ([libp2p/go-libp2p-kad-dht#1032](https://github.com/libp2p/go-libp2p-kad-dht/pull/1032))\n - fix: flaky TestFindPeerWithQueryFilter (#1034) ([libp2p/go-libp2p-kad-dht#1034](https://github.com/libp2p/go-libp2p-kad-dht/pull/1034))\n - fix: Flaky TestInvalidServer (#1029) ([libp2p/go-libp2p-kad-dht#1029](https://github.com/libp2p/go-libp2p-kad-dht/pull/1029))\n - fix: flaky TestClientModeConnect (#1028) ([libp2p/go-libp2p-kad-dht#1028](https://github.com/libp2p/go-libp2p-kad-dht/pull/1028))\n - fix: increase timeout in TestProvidesMany (#1027) ([libp2p/go-libp2p-kad-dht#1027](https://github.com/libp2p/go-libp2p-kad-dht/pull/1027))\n - fix(tests): cleanup of skipped tests (#1025) ([libp2p/go-libp2p-kad-dht#1025](https://github.com/libp2p/go-libp2p-kad-dht/pull/1025))\n - fix: don't skip TestProvidesExpire (#1024) ([libp2p/go-libp2p-kad-dht#1024](https://github.com/libp2p/go-libp2p-kad-dht/pull/1024))\n - fixing flaky TestFindPeerQueryMinimal (#1020) ([libp2p/go-libp2p-kad-dht#1020](https://github.com/libp2p/go-libp2p-kad-dht/pull/1020))\n - fix flaky TestSkipRefreshOnGapCpls (#1021) ([libp2p/go-libp2p-kad-dht#1021](https://github.com/libp2p/go-libp2p-kad-dht/pull/1021))\n - fix: don't skip TestContextShutDown (#1022) ([libp2p/go-libp2p-kad-dht#1022](https://github.com/libp2p/go-libp2p-kad-dht/pull/1022))\n - comments formatting and typos (#1019) ([libp2p/go-libp2p-kad-dht#1019](https://github.com/libp2p/go-libp2p-kad-dht/pull/1019))\n - log peers rejected for diversity (#759) ([libp2p/go-libp2p-kad-dht#759](https://github.com/libp2p/go-libp2p-kad-dht/pull/759))\n - docs: update fullrt docs (#768) ([libp2p/go-libp2p-kad-dht#768](https://github.com/libp2p/go-libp2p-kad-dht/pull/768))\n - query cleanup (#1017) ([libp2p/go-libp2p-kad-dht#1017](https://github.com/libp2p/go-libp2p-kad-dht/pull/1017))\n - better variable names (#787) ([libp2p/go-libp2p-kad-dht#787](https://github.com/libp2p/go-libp2p-kad-dht/pull/787))\n - release v0.29.0 (#1014) ([libp2p/go-libp2p-kad-dht#1014](https://github.com/libp2p/go-libp2p-kad-dht/pull/1014))\n - Move from gogo protobuf (#975) ([libp2p/go-libp2p-kad-dht#975](https://github.com/libp2p/go-libp2p-kad-dht/pull/975))\n - fix: don't copy message to OnRequestHook ([libp2p/go-libp2p-kad-dht#1012](https://github.com/libp2p/go-libp2p-kad-dht/pull/1012))\n - chore: remove boxo/util deps ([libp2p/go-libp2p-kad-dht#1013](https://github.com/libp2p/go-libp2p-kad-dht/pull/1013))\n - feat: add request callback config option ([libp2p/go-libp2p-kad-dht#1011](https://github.com/libp2p/go-libp2p-kad-dht/pull/1011))\n- github.com/libp2p/go-libp2p-kbucket (v0.6.4 -> v0.6.5):\n - upgrading deps (#137) ([libp2p/go-libp2p-kbucket#137](https://github.com/libp2p/go-libp2p-kbucket/pull/137))\n- github.com/libp2p/go-libp2p-pubsub (v0.12.0 -> v0.13.0):\n - Release v0.13.0 (#593) ([libp2p/go-libp2p-pubsub#593](https://github.com/libp2p/go-libp2p-pubsub/pull/593))\n - Allow cancelling IWANT using IDONTWANT (#591) ([libp2p/go-libp2p-pubsub#591](https://github.com/libp2p/go-libp2p-pubsub/pull/591))\n - Improve IDONTWANT Flood Protection (#590) ([libp2p/go-libp2p-pubsub#590](https://github.com/libp2p/go-libp2p-pubsub/pull/590))\n - Fix the Router's Ability to Prune the Mesh Periodically (#589) ([libp2p/go-libp2p-pubsub#589](https://github.com/libp2p/go-libp2p-pubsub/pull/589))\n - Add Function to Enable Application Layer to Send Direct Control Messages (#562) ([libp2p/go-libp2p-pubsub#562](https://github.com/libp2p/go-libp2p-pubsub/pull/562))\n - Do not format expensive debug messages in non-debug levels in doDropRPC (#580) ([libp2p/go-libp2p-pubsub#580](https://github.com/libp2p/go-libp2p-pubsub/pull/580))\n- github.com/libp2p/go-libp2p-record (v0.2.0 -> v0.3.1):\n - fix: missing protobuf package (#64) ([libp2p/go-libp2p-record#64](https://github.com/libp2p/go-libp2p-record/pull/64))\n - release: v0.3.0 (#63) ([libp2p/go-libp2p-record#63](https://github.com/libp2p/go-libp2p-record/pull/63))\n - fix: protobuf namespace conflicts (#62) ([libp2p/go-libp2p-record#62](https://github.com/libp2p/go-libp2p-record/pull/62))\n - Remove gogo protobuf (#60) ([libp2p/go-libp2p-record#60](https://github.com/libp2p/go-libp2p-record/pull/60))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.7.4 -> v0.7.5):\n - new version ([libp2p/go-libp2p-routing-helpers#90](https://github.com/libp2p/go-libp2p-routing-helpers/pull/90))\n - Consolidate multi-error packages by choosing one ([libp2p/go-libp2p-routing-helpers#88](https://github.com/libp2p/go-libp2p-routing-helpers/pull/88))\n - update dependencies ([libp2p/go-libp2p-routing-helpers#89](https://github.com/libp2p/go-libp2p-routing-helpers/pull/89))\n- github.com/multiformats/go-multiaddr (v0.14.0 -> v0.15.0):\n - chore: release v0.15.0 (#266) ([multiformats/go-multiaddr#266](https://github.com/multiformats/go-multiaddr/pull/266))\n - refactor: Backwards compatible Encapsulate/Decapsulate/Join/NewComponent (#272) ([multiformats/go-multiaddr#272](https://github.com/multiformats/go-multiaddr/pull/272))\n - refactor: keep same api as v0.14.0 for SplitFirst/SplitLast (#271) ([multiformats/go-multiaddr#271](https://github.com/multiformats/go-multiaddr/pull/271))\n - refactor: Follows up on #261 (#264) ([multiformats/go-multiaddr#264](https://github.com/multiformats/go-multiaddr/pull/264))\n - refactor!: make the API harder to misuse (#261) ([multiformats/go-multiaddr#261](https://github.com/multiformats/go-multiaddr/pull/261))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Hector Sanjuan | 100 | +4777/-1495 | 200 |\n| Marco Munizaga | 22 | +3482/-1632 | 122 |\n| Andrew Gillis | 69 | +1628/-1509 | 191 |\n| sukun | 13 | +1240/-288 | 67 |\n| Simon Menke | 7 | +766/-97 | 16 |\n| Guillaume Michel | 33 | +438/-383 | 62 |\n| Marcin Rataj | 24 | +494/-266 | 47 |\n| Sergey Gorbunov | 4 | +384/-103 | 20 |\n| AvyChanna | 1 | +294/-193 | 9 |\n| gammazero | 22 | +208/-217 | 28 |\n| Dennis Trautwein | 3 | +425/-0 | 8 |\n| web3-bot | 18 | +193/-184 | 46 |\n| Steven Allen | 8 | +204/-82 | 13 |\n| Marten Seemann | 5 | +215/-63 | 11 |\n| Daniel Norman | 2 | +225/-0 | 6 |\n| Abhinav Prakash | 1 | +190/-2 | 4 |\n| guillaumemichel | 3 | +93/-56 | 15 |\n| youyyytrok | 1 | +84/-63 | 29 |\n| Nishant Das | 2 | +111/-1 | 4 |\n| Pop Chunhapanya | 1 | +109/-0 | 2 |\n| Michael Muré | 7 | +78/-29 | 15 |\n| Jorropo | 4 | +53/-20 | 7 |\n| Ryan Skidmore | 1 | +62/-0 | 2 |\n| GITSRC | 1 | +44/-0 | 3 |\n| Russell Dempsey | 1 | +22/-17 | 10 |\n| Adin Schmahmann | 2 | +29/-8 | 3 |\n| Gabriel Cruz | 1 | +13/-13 | 1 |\n| Wlynxg | 3 | +12/-9 | 3 |\n| Khaled Yakdan | 1 | +11/-10 | 1 |\n| Yahya Hassanzadeh, Ph.D. | 1 | +17/-0 | 1 |\n| Can ZHANG | 2 | +15/-2 | 3 |\n| Pavel Zbitskiy | 1 | +13/-1 | 2 |\n| Yuttakhan B. | 1 | +6/-6 | 6 |\n| Hlib Kanunnikov | 2 | +9/-2 | 4 |\n| Petar Maymounkov | 1 | +7/-2 | 1 |\n| Prithvi Shahi | 2 | +8/-0 | 2 |\n| Piotr Galar | 1 | +4/-4 | 2 |\n| Michael Vorburger | 1 | +6/-0 | 1 |\n| Gus Eggert | 2 | +6/-0 | 2 |\n| Raúl Kripalani | 1 | +4/-0 | 1 |\n| linchizhen | 1 | +1/-1 | 1 |\n| achingbrain | 1 | +1/-1 | 1 |\n| Rod Vagg | 1 | +1/-1 | 1 |\n| Ian Davis | 1 | +1/-1 | 1 |\n| Fabio Bozzo | 1 | +1/-1 | 1 |\n\n## v0.34.1\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [📦️ Important dependency updates](#-important-dependency-updates)\n\n### Overview\n\n### 🔦 Highlights\n\n#### 📦️ Important dependency updates\n\n- update `go-libp2p` to [v0.41.1](https://github.com/libp2p/go-libp2p/releases/tag/v0.41.1)\n - high impact fix from [go-libp2p#3221](https://github.com/libp2p/go-libp2p/pull/3221) improves [hole punching](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md) success rate\n- update `quic-go` to [v0.50.1](https://github.com/quic-go/quic-go/releases/tag/v0.50.1)\n" }, { "path": "docs/changelogs/v0.35.md", "content": "# Kubo changelog v0.35\n\n\n\nThis release was brought to you by the [Shipyard](http://ipshipyard.com/) team.\n\n- [v0.35.0](#v0340)\n\n## v0.35.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [Opt-in HTTP Retrieval client](#opt-in-http-retrieval-client)\n - [Dedicated `Reprovider.Strategy` for MFS](#dedicated-reproviderstrategy-for-mfs)\n - [Experimental support for MFS as a FUSE mount point](#experimental-support-for-mfs-as-a-fuse-mount-point)\n - [Grid view in WebUI](#grid-view-in-webui)\n - [Enhanced DAG-Shaping Controls](#enhanced-dag-shaping-controls)\n - [New DAG-Shaping `ipfs add` Options](#new-dag-shaping-ipfs-add-options)\n - [Persistent DAG-Shaping `Import.*` Configuration](#persistent-dag-shaping-import-configuration)\n - [Updated DAG-Shaping `Import` Profiles](#updated-dag-shaping-import-profiles)\n - [`Datastore` Metrics Now Opt-In](#datastore-metrics-now-opt-in)\n - [Improved performance of data onboarding](#improved-performance-of-data-onboarding)\n - [Fast `ipfs add` in online mode](#fast-ipfs-add-in-online-mode)\n - [Optimized, dedicated queue for providing fresh CIDs](#optimized-dedicated-queue-for-providing-fresh-cids)\n - [Deprecated `ipfs stats provider`](#deprecated-ipfs-stats-provider)\n - [New `Bitswap` configuration options](#new-bitswap-configuration-options)\n - [New `Routing` configuration options](#new-routing-configuration-options)\n - [New Pebble database format config](#new-pebble-database-format-config)\n - [New environment variables](#new-environment-variables)\n - [Improved Log Output Setting](#improved-log-output-setting)\n - [New Repo Lock Optional Wait](#new-repo-lock-optional-wait)\n - [📦️ Important dependency updates](#-important-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\nThis release brings significant UX and performance improvements to data onboarding, provisioning, and retrieval systems.\n\nNew configuration options let you customize the shape of UnixFS DAGs generated during the data import, control the scope of DAGs announced on the Amino DHT, select which delegated routing endpoints are queried, and choose whether to enable HTTP retrieval alongside Bitswap over Libp2p.\n\nContinue reading for more details.\n\n\n### 🔦 Highlights\n\n#### Opt-in HTTP Retrieval client\n\nThis release adds experimental support for retrieving blocks directly over HTTPS (HTTP/2), complementing the existing Bitswap over Libp2p.\n\nThe opt-in client enables Kubo to use [delegated routing](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters) results with `/tls/http` multiaddrs, connecting to HTTPS servers that support [Trustless HTTP Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway)'s Block Responses (`?format=raw`, `application/vnd.ipld.raw`). Fetching blocks via HTTPS (HTTP/2) simplifies infrastructure and reduces costs for storage providers by leveraging HTTP caching and CDNs.\n\nTo enable this feature for testing and feedback, set:\n\n```console\n$ ipfs config --json HTTPRetrieval.Enabled true\n```\n\nSee [`HTTPRetrieval`](https://github.com/ipfs/kubo/blob/master/docs/config.md#httpretrieval) for more details.\n\n#### Dedicated `Reprovider.Strategy` for MFS\n\nThe [Mutable File System (MFS)](https://docs.ipfs.tech/concepts/glossary/#mfs) in Kubo is a UnixFS filesystem managed with [`ipfs files`](https://docs.ipfs.tech/reference/kubo/cli/#ipfs-files) commands. It supports familiar file operations like cp and mv within a folder-tree structure, automatically updating a MerkleDAG and a \"root CID\" that reflects the current MFS state. Files in MFS are protected from garbage collection, offering a simpler alternative to `ipfs pin`. This makes it a popular choice for tools like [IPFS Desktop](https://docs.ipfs.tech/install/ipfs-desktop/) and the [WebUI](https://github.com/ipfs/ipfs-webui/#readme).\n\nPreviously, the `pinned` reprovider strategy required manual pin management: each dataset update meant pinning the new version and unpinning the old one. Now, new strategies—`mfs` and `pinned+mfs`—let users limit announcements to data explicitly placed in MFS. This simplifies updating datasets and announcing only the latest version to the Amino DHT.\n\nUsers relying on the `pinned` strategy can switch to `pinned+mfs` and use MFS alone to manage updates and announcements, eliminating the need for manual pinning and unpinning. We hope this makes it easier to publish just the data that matters to you.\n\nSee [`Reprovider.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy) for more details.\n\n#### Experimental support for MFS as a FUSE mount point\n\nThe MFS root (filesystem behind the `ipfs files` API) is now available as a read/write FUSE mount point at `Mounts.MFS`. This filesystem is mounted in the same way as `Mounts.IPFS` and `Mounts.IPNS` when running `ipfs mount` or `ipfs daemon --mount`.\n\nNote that the operations supported by the MFS FUSE mountpoint are limited, since MFS doesn't store file attributes.\n\nSee [`Mounts`](https://github.com/ipfs/kubo/blob/master/docs/config.md#mounts) and [`docs/fuse.md`](https://github.com/ipfs/kubo/blob/master/docs/fuse.md) for more details.\n\n#### Grid view in WebUI\n\nThe WebUI, accessible at http://127.0.0.1:5001/webui/, now includes support for the grid view on the _Files_ screen:\n\n> ![image](https://github.com/user-attachments/assets/80dcf0d0-8103-426f-ae91-416fb25d32b6)\n\n#### Enhanced DAG-Shaping Controls\n\nThis release advances CIDv1 support by introducing fine-grained control over UnixFS DAG shaping during data ingestion with the `ipfs add` command.\n\nWider DAG trees (more links per node, higher fanout, larger thresholds) are beneficial for large files and directories with many files, reducing tree depth and lookup latency in high-latency networks, but they increase node size, straining memory and CPU on resource-constrained devices. Narrower trees (lower link count, lower fanout, smaller thresholds) are preferable for smaller directories, frequent updates, or low-power clients, minimizing overhead and ensuring compatibility, though they may increase traversal steps for very large datasets.\n\nKubo now allows users to act on these tradeoffs and customize the width of the DAG created by `ipfs add` command.\n\n##### New DAG-Shaping `ipfs add` Options\n\nThree new options allow you to override default settings for specific import operations:\n\n- `--max-file-links`: Sets the maximum number of child links for a single file chunk.\n- `--max-directory-links`: Defines the maximum number of child entries in a \"basic\" (single-chunk) directory.\n - Note: Directories exceeding this limit or the `Import.UnixFSHAMTDirectorySizeThreshold` are converted to HAMT-based (sharded across multiple blocks) structures.\n- `--max-hamt-fanout`: Specifies the maximum number of child nodes for HAMT internal structures.\n\n##### Persistent DAG-Shaping `Import.*` Configuration\n\nYou can set default values for these options using the following configuration settings:\n- [`Import.UnixFSFileMaxLinks`](https://github.com/ipfs/kubo/blob/master/docs/config.md#importunixfsfilemaxlinks)\n- [`Import.UnixFSDirectoryMaxLinks`](https://github.com/ipfs/kubo/blob/master/docs/config.md#importunixfsdirectorymaxlinks)\n- [`Import.UnixFSHAMTDirectoryMaxFanout`](https://github.com/ipfs/kubo/blob/master/docs/config.md#importunixfshamtdirectorymaxfanout)\n- [`Import.UnixFSHAMTDirectorySizeThreshold`](https://github.com/ipfs/kubo/blob/master/docs/config.md#importunixfshamtdirectorysizethreshold)\n\n##### Updated DAG-Shaping `Import` Profiles\n\nThe release updated configuration [profiles](https://github.com/ipfs/kubo/blob/master/docs/config.md#profiles) to incorporate these new `Import.*` settings:\n- Updated Profile: `test-cid-v1` now includes current defaults as explicit `Import.UnixFSFileMaxLinks=174`, `Import.UnixFSDirectoryMaxLinks=0`, `Import.UnixFSHAMTDirectoryMaxFanout=256` and `Import.UnixFSHAMTDirectorySizeThreshold=256KiB`\n- New Profile: `test-cid-v1-wide` adopts experimental directory DAG-shaping defaults, increasing the maximum file DAG width from 174 to 1024, HAMT fanout from 256 to 1024, and raising the HAMT directory sharding threshold from 256KiB to 1MiB, aligning with 1MiB file chunks.\n - Feedback: Try it out and share your thoughts at [discuss.ipfs.tech/t/should-we-profile-cids](https://discuss.ipfs.tech/t/should-we-profile-cids/18507) or [ipfs/specs#499](https://github.com/ipfs/specs/pull/499).\n\n> [!TIP]\n> Apply one of CIDv1 test [profiles](https://github.com/ipfs/kubo/blob/master/docs/config.md#profiles) with `ipfs config profile apply test-cid-v1[-wide]`.\n\n#### `Datastore` Metrics Now Opt-In\n\nTo reduce overhead in the default configuration, datastore metrics are no longer enabled by default when initializing a Kubo repository with `ipfs init`.\nMetrics prefixed with `_datastore` (e.g., `flatfs_datastore_...`, `leveldb_datastore_...`) are not exposed unless explicitly enabled. For a complete list of affected default metrics, refer to [`prometheus_metrics_added_by_measure_profile`](https://github.com/ipfs/kubo/blob/master/test/sharness/t0119-prometheus-data/prometheus_metrics_added_by_measure_profile).\n\nConvenience opt-in [profiles](https://github.com/ipfs/kubo/blob/master/docs/config.md#profiles) can be enabled at initialization time with `ipfs init --profile`: `flatfs-measure`, `pebbleds-measure`, `badgerds-measure`\n\nIt is also possible to manually add the `measure` wrapper. See examples in [`Datastore.Spec`](https://github.com/ipfs/kubo/blob/master/docs/config.md#datastorespec) documentation.\n\n#### Improved performance of data onboarding\n\nThis Kubo release significantly improves both the speed of ingesting data via `ipfs add` and announcing newly produced CIDs to Amino DHT.\n\n##### Fast `ipfs add` in online mode\n\nAdding a large directory of data when `ipfs daemon` was running in online mode took a long time. A significant amount of this time was spent writing to and reading from the persisted provider queue. Due to this, many users had to shut down the daemon and perform data import in offline mode. This release fixes this known limitation, significantly improving the speed of `ipfs add`.\n\n> [!IMPORTANT]\n> Performing `ipfs add` of 10GiB file would take about 30 minutes.\n> Now it takes close to 30 seconds.\n\nKubo v0.34:\n\n```console\n$ time kubo/cmd/ipfs/ipfs add -r /tmp/testfiles-100M > /dev/null\n 100.00 MiB / 100.00 MiB [=====================================================================] 100.00%\nreal\t0m6.464s\n\n$ time kubo/cmd/ipfs/ipfs add -r /tmp/testfiles-1G > /dev/null\n 1000.00 MiB / 1000.00 MiB [===================================================================] 100.00%\nreal\t1m10.542s\n\n$ time kubo/cmd/ipfs/ipfs add -r /tmp/testfiles-10G > /dev/null\n 10.00 GiB / 10.00 GiB [=======================================================================] 100.00%\nreal\t24m5.744s\n```\n\nKubo v0.35:\n\n```console\n$ time kubo/cmd/ipfs/ipfs add -r /tmp/testfiles-100M > /dev/null\n 100.00 MiB / 100.00 MiB [=====================================================================] 100.00%\nreal\t0m0.326s\n\n$ time kubo/cmd/ipfs/ipfs add -r /tmp/testfiles-1G > /dev/null\n 1.00 GiB / 1.00 GiB [=========================================================================] 100.00%\nreal\t0m2.819s\n\n$ time kubo/cmd/ipfs/ipfs add -r /tmp/testfiles-10G > /dev/null\n 10.00 GiB / 10.00 GiB [=======================================================================] 100.00%\nreal\t0m28.405s\n```\n\n##### Optimized, dedicated queue for providing fresh CIDs\n\nFrom `kubo` [`v0.33.0`](https://github.com/ipfs/kubo/releases/tag/v0.33.0),\nBitswap stopped advertising newly added and received blocks to the DHT. Since\nthen `boxo/provider` is responsible for the first time provide and the recurring reprovide logic. Prior\nto `v0.35.0`, provides and reprovides were handled together in batches, leading\nto delays in initial advertisements (provides).\n\nProvides and Reprovides now have separate queues, allowing for immediate\nprovide of new CIDs and optimised batching of reprovides.\n\n###### New `Provider` configuration options\n\nThis change introduces a new configuration options:\n\n- [`Provider.Enabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providerenabled) is a global flag for disabling both [Provider](https://github.com/ipfs/kubo/blob/master/docs/config.md#provider) and [Reprovider](https://github.com/ipfs/kubo/blob/master/docs/config.md#reprovider) systems (announcing new/old CIDs to amino DHT).\n- [`Provider.WorkerCount`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providerworkercount) for limiting the number of concurrent provide operations, allows for fine-tuning the trade-off between announcement speed and system load when announcing new CIDs.\n- Removed `Experimental.StrategicProviding`. Superseded by `Provider.Enabled`, `Reprovider.Interval` and [`Reprovider.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy).\n\n> [!TIP]\n> Users who need to provide large volumes of content immediately should consider setting `Routing.AcceleratedDHTClient` to `true`. If that is not enough, consider adjusting `Provider.WorkerCount` to a higher value.\n\n###### Deprecated `ipfs stats provider`\n\nSince the `ipfs stats provider` command was displaying statistics for both\nprovides and reprovides, this command isn't relevant anymore after separating\nthe two queues.\n\nThe successor command is `ipfs stats reprovide`, showing the same statistics,\nbut for reprovides only.\n\n> [!NOTE]\n> `ipfs stats provider` still works, but is marked as deprecated and will be removed in a future release. Be mindful that the command provides only statistics about reprovides (similar to `ipfs stats reprovide`) and not the new provide queue (this will be fixed as a part of wider refactor planned for a future release).\n\n#### New `Bitswap` configuration options\n\n- [`Bitswap.Libp2pEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#bitswaplibp2penabled) determines whether Kubo will use Bitswap over libp2p (both client and server).\n- [`Bitswap.ServerEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#bitswapserverenabled) controls whether Kubo functions as a Bitswap server to host and respond to block requests.\n- [`Internal.Bitswap.ProviderSearchMaxResults`](https://github.com/ipfs/kubo/blob/master/docs/config.md#internalbitswapprovidersearchmaxresults) for adjusting the maximum number of providers bitswap client should aim at before it stops searching for new ones.\n\n#### New `Routing` configuration options\n\n- [`Routing.IgnoreProviders`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingignoreproviders) allows ignoring specific peer IDs when returned by the content routing system as providers of content.\n - Simplifies testing `HTTPRetrieval.Enabled` in setups where Bitswap over Libp2p and HTTP retrieval is served under different PeerIDs.\n- [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters) allows customizing HTTP routers used by Kubo when `Routing.Type` is set to `auto` or `autoclient`.\n - Users are now able to adjust the default routing system and directly query custom routers for increased resiliency or when dataset is too big and CIDs are not announced on Amino DHT.\n\n> [!TIP]\n>\n> For example, to use Pinata's routing endpoint in addition to IPNI at `cid.contact`:\n>\n> ```console\n> $ ipfs config --json Routing.DelegatedRouters '[\"https://cid.contact\",\"https://indexer.pinata.cloud\"]'\n> ```\n\n#### New Pebble database format config\n\nThis Kubo release provides node operators with more control over [Pebble's `FormatMajorVersion`](https://github.com/cockroachdb/pebble/tree/master?tab=readme-ov-file#format-major-versions). This allows testing a new Kubo release without automatically migrating Pebble datastores, keeping the ability to switch back to older Kubo.\n\nWhen IPFS is initialized to use the pebbleds datastore (opt-in via `ipfs init --profile=pebbleds`), the latest pebble database format is configured in the pebble datastore config as `\"formatMajorVersion\"`. Setting this in the datastore config prevents automatically upgrading to the latest available version when Kubo is upgraded. If a later version becomes available, the Kubo daemon prints a startup message to indicate this. The user can them update the config to use the latest format when they are certain a downgrade will not be necessary.\n\nWithout the `\"formatMajorVersion\"` in the pebble datastore config, the database format is automatically upgraded to the latest version. If this happens, then it is possible a downgrade back to the previous version of Kubo will not work if new format is not compatible with the pebble datastore in the previous version of Kubo.\n\nWhen installing a new version of Kubo when `\"formatMajorVersion\"` is configured, automatic repository migration (`ipfs daemon with --migrate=true`) does not upgrade this to the latest available version. This is done because a user may have reasons not to upgrade the pebble database format, and may want to be able to downgrade Kubo if something else is not working in the new version. If the configured pebble database format in the old Kubo is not supported in the new Kubo, then the configured version must be updated and the old Kubo run, before installing the new Kubo.\n\nSee other caveats and configuration options at [`kubo/docs/datastores.md#pebbleds`](https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds)\n\n#### New environment variables\n\nThe [`environment-variables.md`](https://github.com/ipfs/kubo/blob/master/docs/environment-variables.md) was extended with two new features:\n\n##### Improved Log Output Setting\n\nWhen stderr and/or stdout options are configured or specified by the `GOLOG_OUTPUT` environ variable, log only to the output(s) specified. For example:\n\n- `GOLOG_OUTPUT=\"stderr\"` logs only to stderr\n- `GOLOG_OUTPUT=\"stdout\"` logs only to stdout\n- `GOLOG_OUTPUT=\"stderr+stdout\"` logs to both stderr and stdout\n\n##### New Repo Lock Optional Wait\n\nThe environment variable `IPFS_WAIT_REPO_LOCK` specifies the amount of time to wait for the repo lock. Set the value of this variable to a string that can be [parsed](https://pkg.go.dev/time@go1.24.3#ParseDuration) as a golang `time.Duration`. For example:\n```\nIPFS_WAIT_REPO_LOCK=\"15s\"\n```\n\nIf the lock cannot be acquired because someone else has the lock, and `IPFS_WAIT_REPO_LOCK` is set to a valid value, then acquiring the lock is retried every second until the lock is acquired or the specified wait time has elapsed.\n\n#### 📦️ Important dependency updates\n\n- update `boxo` to [v0.30.0](https://github.com/ipfs/boxo/releases/tag/v0.30.0)\n- update `ipfs-webui` to [v4.7.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.7.0)\n- update `go-ds-pebble` to [v0.5.0](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.0)\n - update `pebble` to [v2.0.3](https://github.com/cockroachdb/pebble/releases/tag/v2.0.3)\n- update `go-libp2p-pubsub` to [v0.13.1](https://github.com:/libp2p/go-libp2p-pubsub/releases/tag/v0.13.1)\n- update `go-libp2p-kad-dht` to [v0.33.1](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.33.1) (incl. [v0.33.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.33.0), [v0.32.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.32.0), [v0.31.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.31.0))\n- update `go-log` to [v2.6.0](https://github.com/ipfs/go-log/releases/tag/v2.6.0)\n- update `p2p-forge/client` to [v0.5.1](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.5.1)\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore(version): 0.35.0\n - fix: go-libp2p-kad-dht v0.33.1 (#10814) ([ipfs/kubo#10814](https://github.com/ipfs/kubo/pull/10814))\n - fix: p2p-forge v0.5.1 ignoring /p2p-circuit (#10813) ([ipfs/kubo#10813](https://github.com/ipfs/kubo/pull/10813))\n - chore(version): 0.35.0-rc2\n - fix(fuse): ipns error handling and friendly errors (#10807) ([ipfs/kubo#10807](https://github.com/ipfs/kubo/pull/10807))\n - fix(config): wire up `Provider.Enabled` flag (#10804) ([ipfs/kubo#10804](https://github.com/ipfs/kubo/pull/10804))\n - docs(changelog): go-libp2p-kad-dht\n - chore(version): 0.35.0-rc1\n - feat: IPFS_WAIT_REPO_LOCK (#10797) ([ipfs/kubo#10797](https://github.com/ipfs/kubo/pull/10797))\n - logging: upgrade to go-log/v2 v2.6.0 (#10798) ([ipfs/kubo#10798](https://github.com/ipfs/kubo/pull/10798))\n - chore: ensure /mfs is present in docker\n - feat(fuse): Expose MFS as FUSE mount point (#10781) ([ipfs/kubo#10781](https://github.com/ipfs/kubo/pull/10781))\n - feat: opt-in http retrieval client (#10772) ([ipfs/kubo#10772](https://github.com/ipfs/kubo/pull/10772))\n - Update go-libp2p-pubsub to v0.13.1 (#10795) ([ipfs/kubo#10795](https://github.com/ipfs/kubo/pull/10795))\n - feat(config): ability to disable Bitswap fully or just server (#10782) ([ipfs/kubo#10782](https://github.com/ipfs/kubo/pull/10782))\n - refactor: make datastore metrics opt-in (#10788) ([ipfs/kubo#10788](https://github.com/ipfs/kubo/pull/10788))\n - feat(pebble): support pinning `FormatMajorVersion` (#10789) ([ipfs/kubo#10789](https://github.com/ipfs/kubo/pull/10789))\n - feat: `Provider.WorkerCount` and `stats reprovide` (#10779) ([ipfs/kubo#10779](https://github.com/ipfs/kubo/pull/10779))\n - Upgrade to Boxo v0.30.0 (#10794) ([ipfs/kubo#10794](https://github.com/ipfs/kubo/pull/10794))\n - docs: use latest fuse package (#10791) ([ipfs/kubo#10791](https://github.com/ipfs/kubo/pull/10791))\n - remove duplicate words (#10790) ([ipfs/kubo#10790](https://github.com/ipfs/kubo/pull/10790))\n - feat(config): `ipfs add` and `Import` options for controlling UnixFS DAG Width (#10774) ([ipfs/kubo#10774](https://github.com/ipfs/kubo/pull/10774))\n - feat(config): expose ProviderSearchMaxResults (#10773) ([ipfs/kubo#10773](https://github.com/ipfs/kubo/pull/10773))\n - feat: ipfs-webui v4.7.0 (#10780) ([ipfs/kubo#10780](https://github.com/ipfs/kubo/pull/10780))\n - feat: partial DAG provides with Reprovider.Strategy=mfs|pinned+mfs (#10754) ([ipfs/kubo#10754](https://github.com/ipfs/kubo/pull/10754))\n - chore: update url\n - docs: known issues with file/urlstores (#10768) ([ipfs/kubo#10768](https://github.com/ipfs/kubo/pull/10768))\n - fix: Add IPFS & IPNS path details to error (re. #10762) (#10770) ([ipfs/kubo#10770](https://github.com/ipfs/kubo/pull/10770))\n - docs: Fix typo in v0.34 changelog (#10771) ([ipfs/kubo#10771](https://github.com/ipfs/kubo/pull/10771))\n - Support WithIgnoreProviders() in provider query manager ([ipfs/kubo#10765](https://github.com/ipfs/kubo/pull/10765))\n - Merge release v0.34.1 ([ipfs/kubo#10766](https://github.com/ipfs/kubo/pull/10766))\n - fix: reprovides warning (#10761) ([ipfs/kubo#10761](https://github.com/ipfs/kubo/pull/10761))\n - Merge release v0.34.0 ([ipfs/kubo#10759](https://github.com/ipfs/kubo/pull/10759))\n - feat: ipfs-webui v4.6 (#10756) ([ipfs/kubo#10756](https://github.com/ipfs/kubo/pull/10756))\n - docs(readme): update min. requirements + cleanup (#10750) ([ipfs/kubo#10750](https://github.com/ipfs/kubo/pull/10750))\n - Upgrade to Boxo v0.29.1 (#10755) ([ipfs/kubo#10755](https://github.com/ipfs/kubo/pull/10755))\n - Nonfunctional (#10753) ([ipfs/kubo#10753](https://github.com/ipfs/kubo/pull/10753))\n - provider: buffer pin providers ([ipfs/kubo#10746](https://github.com/ipfs/kubo/pull/10746))\n - chore: 0.35.0-dev\n- github.com/ipfs/boxo (v0.29.1 -> v0.30.0):\n - Release v0.30.0 ([ipfs/boxo#915](https://github.com/ipfs/boxo/pull/915))\n - feat(bitswap): add option to disable Bitswap server (#911) ([ipfs/boxo#911](https://github.com/ipfs/boxo/pull/911))\n - provider: dedicated provide queue (#907) ([ipfs/boxo#907](https://github.com/ipfs/boxo/pull/907))\n - provider: deduplicate cids in queue (#910) ([ipfs/boxo#910](https://github.com/ipfs/boxo/pull/910))\n - feat(unixfs/mfs): support MaxLinks and MaxHAMTFanout (#906) ([ipfs/boxo#906](https://github.com/ipfs/boxo/pull/906))\n - feat(ipld/unixfs): DagModifier: allow specifying MaxLinks per file (#898) ([ipfs/boxo#898](https://github.com/ipfs/boxo/pull/898))\n - feat: NewDAGProvider to walk partial DAGs in offline mode (#905) ([ipfs/boxo#905](https://github.com/ipfs/boxo/pull/905))\n - gateway: check for UseSubdomains with IP addresses (#903) ([ipfs/boxo#903](https://github.com/ipfs/boxo/pull/903))\n - feat(gateway): add cid copy button to directory listings (#899) ([ipfs/boxo#899](https://github.com/ipfs/boxo/pull/899))\n - Improve performance of data onboarding (#888) ([ipfs/boxo#888](https://github.com/ipfs/boxo/pull/888))\n - bitswap: add requestsInFlight metric ([ipfs/boxo#904](https://github.com/ipfs/boxo/pull/904))\n - provider: simplify reprovide (#890) ([ipfs/boxo#890](https://github.com/ipfs/boxo/pull/890))\n - Upgrade to go-libp2p v0.41.1 ([ipfs/boxo#896](https://github.com/ipfs/boxo/pull/896))\n - Update RELEASE.md ([ipfs/boxo#892](https://github.com/ipfs/boxo/pull/892))\n - changelog: document bsnet import path change ([ipfs/boxo#891](https://github.com/ipfs/boxo/pull/891))\n - fix(gateway): preserve query parameters on _redirects ([ipfs/boxo#886](https://github.com/ipfs/boxo/pull/886))\n - bitswap/httpnet: Add WithDenylist option ([ipfs/boxo#877](https://github.com/ipfs/boxo/pull/877))\n- github.com/ipfs/go-block-format (v0.2.0 -> v0.2.1):\n - Update version (#60) ([ipfs/go-block-format#60](https://github.com/ipfs/go-block-format/pull/60))\n - Update go-ipfs-util to use boxo (#52) ([ipfs/go-block-format#52](https://github.com/ipfs/go-block-format/pull/52))\n- github.com/ipfs/go-ds-pebble (v0.4.4 -> v0.5.0):\n - new version (#53) ([ipfs/go-ds-pebble#53](https://github.com/ipfs/go-ds-pebble/pull/53))\n - Upgrade to pebble v2.0.3 (#45) ([ipfs/go-ds-pebble#45](https://github.com/ipfs/go-ds-pebble/pull/45))\n- github.com/ipfs/go-fs-lock (v0.0.7 -> v0.1.1):\n - new version (#48) ([ipfs/go-fs-lock#48](https://github.com/ipfs/go-fs-lock/pull/48))\n - Return original error when WaitLock times out (#47) ([ipfs/go-fs-lock#47](https://github.com/ipfs/go-fs-lock/pull/47))\n - new version (#45) ([ipfs/go-fs-lock#45](https://github.com/ipfs/go-fs-lock/pull/45))\n - Add WaitLock function (#44) ([ipfs/go-fs-lock#44](https://github.com/ipfs/go-fs-lock/pull/44))\n - sync: update CI config files ([ipfs/go-fs-lock#30](https://github.com/ipfs/go-fs-lock/pull/30))\n - sync: update CI config files (#27) ([ipfs/go-fs-lock#27](https://github.com/ipfs/go-fs-lock/pull/27))\n - sync: update CI config files ([ipfs/go-fs-lock#25](https://github.com/ipfs/go-fs-lock/pull/25))\n- github.com/ipfs/go-log/v2 (v2.5.1 -> v2.6.0):\n - new version (#155) ([ipfs/go-log#155](https://github.com/ipfs/go-log/pull/155))\n - feat: only log to stderr or to stdout or both if configured (#154) ([ipfs/go-log#154](https://github.com/ipfs/go-log/pull/154))\n - ci: uci/copy-templates ([ipfs/go-log#145](https://github.com/ipfs/go-log/pull/145))\n - sync: update CI config files (#137) ([ipfs/go-log#137](https://github.com/ipfs/go-log/pull/137))\n- github.com/libp2p/go-libp2p-kad-dht (v0.30.2 -> v0.33.1):\n - chore: release v0.33.1 (#1088) ([libp2p/go-libp2p-kad-dht#1088](https://github.com/libp2p/go-libp2p-kad-dht/pull/1088))\n - fix(fullrt): mutex cleanup (#1087) ([libp2p/go-libp2p-kad-dht#1087](https://github.com/libp2p/go-libp2p-kad-dht/pull/1087))\n - fix: use correct mutex for reading keyToPeerMap (#1086) ([libp2p/go-libp2p-kad-dht#1086](https://github.com/libp2p/go-libp2p-kad-dht/pull/1086))\n - fix: fullrt kMapLk unlock (#1085) ([libp2p/go-libp2p-kad-dht#1085](https://github.com/libp2p/go-libp2p-kad-dht/pull/1085))\n - chore: release v0.33.0 (#1083) ([libp2p/go-libp2p-kad-dht#1083](https://github.com/libp2p/go-libp2p-kad-dht/pull/1083))\n - fix/updates to use context passed in New function for context cancellation (#1081) ([libp2p/go-libp2p-kad-dht#1081](https://github.com/libp2p/go-libp2p-kad-dht/pull/1081))\n - chore: release v0.31.1 (#1079) ([libp2p/go-libp2p-kad-dht#1079](https://github.com/libp2p/go-libp2p-kad-dht/pull/1079))\n - fix: netsize warning (#1077) ([libp2p/go-libp2p-kad-dht#1077](https://github.com/libp2p/go-libp2p-kad-dht/pull/1077))\n - fix: use correct message type attribute in metrics (#1076) ([libp2p/go-libp2p-kad-dht#1076](https://github.com/libp2p/go-libp2p-kad-dht/pull/1076))\n - chore: bump go-log to v2 (#1074) ([libp2p/go-libp2p-kad-dht#1074](https://github.com/libp2p/go-libp2p-kad-dht/pull/1074))\n - release v0.31.0 (#1072) ([libp2p/go-libp2p-kad-dht#1072](https://github.com/libp2p/go-libp2p-kad-dht/pull/1072))\n - query: ip diversity filter (#1070) ([libp2p/go-libp2p-kad-dht#1070](https://github.com/libp2p/go-libp2p-kad-dht/pull/1070))\n - tests: fix flaky TestProvidesExpire (#1069) ([libp2p/go-libp2p-kad-dht#1069](https://github.com/libp2p/go-libp2p-kad-dht/pull/1069))\n - refactor: replace fmt.Errorf with errors.New when not formatting is required (#1067) ([libp2p/go-libp2p-kad-dht#1067](https://github.com/libp2p/go-libp2p-kad-dht/pull/1067))\n - fix: error on no valid provs (#1065) ([libp2p/go-libp2p-kad-dht#1065](https://github.com/libp2p/go-libp2p-kad-dht/pull/1065))\n - cleanup: remove deprecated opt package (#1064) ([libp2p/go-libp2p-kad-dht#1064](https://github.com/libp2p/go-libp2p-kad-dht/pull/1064))\n - cleanup: fullrt ([libp2p/go-libp2p-kad-dht#1062](https://github.com/libp2p/go-libp2p-kad-dht/pull/1062))\n - fix: remove peerstore no-op (#1063) ([libp2p/go-libp2p-kad-dht#1063](https://github.com/libp2p/go-libp2p-kad-dht/pull/1063))\n - tests: flaky TestSearchValue (dual) (#1060) ([libp2p/go-libp2p-kad-dht#1060](https://github.com/libp2p/go-libp2p-kad-dht/pull/1060))\n- github.com/libp2p/go-libp2p-kbucket (v0.6.5 -> v0.7.0):\n - chore: release v0.7.0 (#143) ([libp2p/go-libp2p-kbucket#143](https://github.com/libp2p/go-libp2p-kbucket/pull/143))\n - peerdiversity: export IPGroupKey (#141) ([libp2p/go-libp2p-kbucket#141](https://github.com/libp2p/go-libp2p-kbucket/pull/141))\n - fix: flaky TestUsefulNewPeer (#140) ([libp2p/go-libp2p-kbucket#140](https://github.com/libp2p/go-libp2p-kbucket/pull/140))\n - fix: flaky TestTableFindMultipleBuckets (#139) ([libp2p/go-libp2p-kbucket#139](https://github.com/libp2p/go-libp2p-kbucket/pull/139))\n- github.com/libp2p/go-libp2p-pubsub (v0.13.0 -> v0.13.1):\n - feat: WithValidatorData publishing option (#603) ([libp2p/go-libp2p-pubsub#603](https://github.com/libp2p/go-libp2p-pubsub/pull/603))\n - feat: avoid repeated checksum calculations (#599) ([libp2p/go-libp2p-pubsub#599](https://github.com/libp2p/go-libp2p-pubsub/pull/599))\n- github.com/libp2p/go-yamux/v4 (v4.0.1 -> v4.0.2):\n - Release v4.0.2 (#124) ([libp2p/go-yamux#124](https://github.com/libp2p/go-yamux/pull/124))\n - fix: remove noisy logs (#116) ([libp2p/go-yamux#116](https://github.com/libp2p/go-yamux/pull/116))\n - check deadline before sending a message (#114) ([libp2p/go-yamux#114](https://github.com/libp2p/go-yamux/pull/114))\n - only check KeepAliveInterval if keep-alive are enabled (#113) ([libp2p/go-yamux#113](https://github.com/libp2p/go-yamux/pull/113))\n - ci: uci/copy-templates ([libp2p/go-yamux#109](https://github.com/libp2p/go-yamux/pull/109))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Hector Sanjuan | 16 | +2662/-590 | 71 |\n| Guillaume Michel | 27 | +1339/-714 | 69 |\n| Andrew Gillis | 22 | +1056/-377 | 54 |\n| Sergey Gorbunov | 1 | +962/-42 | 26 |\n| Marcin Rataj | 19 | +714/-133 | 47 |\n| IGP | 2 | +419/-35 | 11 |\n| GITSRC | 1 | +90/-1 | 3 |\n| guillaumemichel | 1 | +21/-43 | 1 |\n| blockchainluffy | 1 | +27/-26 | 8 |\n| web3-bot | 9 | +21/-22 | 13 |\n| VersaliX | 1 | +31/-2 | 4 |\n| gammazero | 5 | +18/-5 | 5 |\n| Hlib Kanunnikov | 1 | +14/-4 | 1 |\n| diogo464 | 1 | +6/-7 | 1 |\n| Asutorufa | 2 | +7/-1 | 2 |\n| Russell Dempsey | 1 | +6/-1 | 1 |\n| Steven Allen | 1 | +1/-5 | 1 |\n| Michael Vorburger | 2 | +3/-3 | 2 |\n| Aayush Rajasekaran | 1 | +2/-2 | 1 |\n| sukun | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.36.md", "content": "# Kubo changelog v0.36\n\n\n\nThis release was brought to you by the [Interplanetary Shipyard](https://ipshipyard.com/) team.\n\n- [v0.36.0](#v0360)\n\n## v0.36.0\n\n[](https://github.com/user-attachments/assets/0d830631-7b92-48ca-8ce9-b537e1479dfb)\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [HTTP Retrieval Client Now Enabled by Default](#http-retrieval-client-now-enabled-by-default)\n - [Bitswap Broadcast Reduction](#bitswap-broadcast-reduction)\n - [Update go-log to v2](#update-go-log-to-v2)\n - [Kubo now uses AutoNATv2 as a client](#kubo-now-uses-autonatv2-as-a-client)\n - [Smarter AutoTLS registration](#smarter-autotls-registration)\n - [Overwrite option for files cp command](#overwrite-option-for-files-cp-command)\n - [Gateway now supports negative HTTP Range requests](#gateway-now-supports-negative-http-range-requests)\n - [Option for `filestore` command to remove bad blocks](#option-for-filestore-command-to-remove-bad-blocks)\n - [`ConnMgr.SilencePeriod` configuration setting exposed](#connmgrsilenceperiod-configuration-setting-exposed)\n - [Fix handling of EDITOR env var](#fix-handling-of-editor-env-var)\n - [📦️ Important dependency updates](#-important-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### HTTP Retrieval Client Now Enabled by Default\n\nThis release promotes the HTTP Retrieval client from an experimental feature to a standard feature that is enabled by default. When possible, Kubo will retrieve blocks over plain HTTPS (HTTP/2) without any extra user configuration.\n\nSee [`HTTPRetrieval`](https://github.com/ipfs/kubo/blob/master/docs/config.md#httpretrieval) for more details.\n\n#### Bitswap Broadcast Reduction\n\nThe Bitswap client now supports broadcast reduction logic, which is enabled by default. This feature significantly reduces the number of broadcast messages sent to peers, resulting in lower bandwidth usage during load spikes.\n\nThe overall logic works by sending to non-local peers only if those peers have previously replied that they have data blocks. To minimize impact on existing workloads, by default, broadcasts are still always sent to peers on the local network, or the ones defined in `Peering.Peers`.\n\nAt Shipyard, we conducted A/B testing on our internal Kubo staging gateway with organic CID requests to `ipfs.io`. While these results may not exactly match your specific workload, the benefits proved significant enough to make this feature default. Here are the key findings:\n\n- **Dramatic Resource Usage Reduction:** Internal testing demonstrated a reduction in Bitswap broadcast messages by 80-98% and network bandwidth savings of 50-95%, with the greatest improvements occurring during high traffic and peer spikes. These efficiency gains lower operational costs of running Kubo under high load and improve the IPFS Mainnet (which is >80% Kubo-based) by reducing ambient traffic for all connected peers.\n- **Improved Memory Stability:** Memory stays stable even during major CID request spikes that increase peer count, preventing the out-of-memory (OOM) issues found in earlier Kubo versions.\n- **Data Retrieval Performance Remains Strong:** Our tests suggest that Kubo gateway hosts with broadcast reduction enabled achieve similar or better HTTP 200 success rates compared to version 0.35, while maintaining equivalent or higher want-have responses and unique blocks received.\n\nFor more information about our A/B tests, see [kubo#10825](https://github.com/ipfs/kubo/pull/10825).\n\nTo revert to the previous behavior for your own A/B testing, set `Internal.Bitswap.BroadcastControl.Enable` to `false` and monitor relevant metrics (`ipfs_bitswap_bcast_skips_total`, `ipfs_bitswap_haves_received`, `ipfs_bitswap_unique_blocks_received`, `ipfs_bitswap_wanthaves_broadcast`, HTTP 200 success rate).\n\nFor a description of the configuration items, see the documentation of [`Internal.Bitswap.BroadcastControl`](https://github.com/ipfs/kubo/blob/master/docs/config.md#internalbitswapbroadcastcontrol).\n\n#### Update go-log to v2\n\ngo-log v2 has been out for quite a while now and it's time to deprecate v1.\n\n- Replace all use of `go-log` with `go-log/v2`\n- Makes `/api/v0/log/tail` useful over HTTP\n- Fixes `ipfs log tail`\n- Removes support for `ContextWithLoggable` as this is not needed for tracing-like functionality\n\n#### Kubo now uses AutoNATv2 as a client\n\nThis Kubo release starts utilizing [AutoNATv2](https://github.com/libp2p/specs/blob/master/autonat/autonat-v2.md) client functionality. go-libp2p v0.42 supports and depends on both AutoNATv1 and v2, and Autorelay feature continues to use v1. go-libp2p v0.43+ will discontinue internal use of AutoNATv1. We will maintain support for both v1 and v2 until then, though v1 will gradually be deprecated and ultimately removed.\n\n##### Smarter AutoTLS registration\n\nThis update to libp2p and [AutoTLS](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotls) incorporates AutoNATv2 changes. It aims to reduce false-positive scenarios where AutoTLS certificate registration occurred before a publicly dialable multiaddr was available. This should result in fewer error logs during node start, especially when IPv6 and/or IPv4 NATs with UPnP/PCP/NAT-PMP are at play.\n\n#### Overwrite option for files cp command\n\nThe `ipfs files cp` command has a `--force` option to allow it to overwrite existing files. Attempting to overwrite an existing directory results in an error.\n\n#### Gateway now supports negative HTTP Range requests\n\nThe latest update to `boxo/gateway` adds support for negative HTTP Range requests, achieving [gateway-conformance@v0.8](https://github.com/ipfs/gateway-conformance/releases/tag/v0.8.0) compatibility.\nThis provides greater interoperability with generic HTTP-based tools. For example, [WebRecorder](https://webrecorder.net/archivewebpage/)'s https://replayweb.page/ can now directly load website snapshots from Kubo-backed URLs.\n\n#### Option for `filestore` command to remove bad blocks\n\nThe [experimental `filestore`](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-filestore) command has a new option, `--remove-bad-blocks`, to verify objects in the filestore and remove those that fail verification.\n\n#### `ConnMgr.SilencePeriod` configuration setting exposed\n\nThis connection manager option controls how often connections are swept and potentially terminated. See the [ConnMgr documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmconnmgrsilenceperiod).\n\n#### Fix handling of EDITOR env var\n\nThe `ipfs config edit` command did not correctly handle the `EDITOR` environment variable when its value contains flags and arguments, i.e. `EDITOR=emacs -nw`. The command was treating the entire value of `$EDITOR` as the name of the editor command. This has been fixed to parse the value of `$EDITOR` into separate args, respecting shell quoting.\n\n#### 📦️ Important dependency updates\n\n- update `go-libp2p` to [v0.42.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.42.0)\n- update `go-libp2p-kad-dht` to [v0.33.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.33.0)\n- update `boxo` to [v0.33.0](https://github.com/ipfs/boxo/releases/tag/v0.33.0) (incl. [v0.32.0](https://github.com/ipfs/boxo/releases/tag/v0.32.0))\n- update `gateway-conformance` to [v0.8](https://github.com/ipfs/gateway-conformance/releases/tag/v0.8.0)\n- update `p2p-forge/client` to [v0.6.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.6.0)\n- update `github.com/cockroachdb/pebble/v2` to [v2.0.6](https://github.com/cockroachdb/pebble/releases/tag/v2.0.6) for Go 1.25 support\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: 0.36.0\n - chore: update links in markdown\n - chore: 0.36.0-rc2\n - feat(httpnet): gather metrics for allowlist\n - chore: changelog\n - test: TestEditorParsing\n - fix: handling of EDITOR env var (#10855) ([ipfs/kubo#10855](https://github.com/ipfs/kubo/pull/10855))\n - refactor: use slices.Sort where appropriate (#10858) ([ipfs/kubo#10858](https://github.com/ipfs/kubo/pull/10858))\n - Upgrade to Boxo v0.33.0 (#10857) ([ipfs/kubo#10857](https://github.com/ipfs/kubo/pull/10857))\n - chore: Upgrade github.com/cockroachdb/pebble/v2 to v2.0.6 for Go 1.25 support (#10850) ([ipfs/kubo#10850](https://github.com/ipfs/kubo/pull/10850))\n - core:constructor: add a log line about http retrieval\n - chore: p2p-forge v0.6.0 + go-libp2p 0.42.0 (#10840) ([ipfs/kubo#10840](https://github.com/ipfs/kubo/pull/10840))\n - docs: fix minor typos (#10849) ([ipfs/kubo#10849](https://github.com/ipfs/kubo/pull/10849))\n - Replace use of go-car v1 with go-car/v2 (#10845) ([ipfs/kubo#10845](https://github.com/ipfs/kubo/pull/10845))\n - chore: v0.36.0-rc1\n - chore: deduplicate 0.36 changelog\n - feat(config): connmgr: expose silence period (#10827) ([ipfs/kubo#10827](https://github.com/ipfs/kubo/pull/10827))\n - bitswap/client: configurable broadcast reduction (#10825) ([ipfs/kubo#10825](https://github.com/ipfs/kubo/pull/10825))\n - Upgrade to Boxo v0.32.0 (#10839) ([ipfs/kubo#10839](https://github.com/ipfs/kubo/pull/10839))\n - feat: HTTP retrieval enabled by default (#10836) ([ipfs/kubo#10836](https://github.com/ipfs/kubo/pull/10836))\n - feat: AutoTLS with AutoNATv2 client (#10835) ([ipfs/kubo#10835](https://github.com/ipfs/kubo/pull/10835))\n - commands: add `--force` option to `files cp` command (#10823) ([ipfs/kubo#10823](https://github.com/ipfs/kubo/pull/10823))\n - docs/env variables: Document LIBP2P_SWARM_FD_LIMIT ([ipfs/kubo#10828](https://github.com/ipfs/kubo/pull/10828))\n - test: fix \"invert\" commands in sharness tests (#9652) ([ipfs/kubo#9652](https://github.com/ipfs/kubo/pull/9652))\n - Ivan386/filestore fix (#7474) ([ipfs/kubo#7474](https://github.com/ipfs/kubo/pull/7474))\n - wrap user-facing mfs.Lookup error (#10821) ([ipfs/kubo#10821](https://github.com/ipfs/kubo/pull/10821))\n - Update fuse docs with FreeBSD specifics (#10820) ([ipfs/kubo#10820](https://github.com/ipfs/kubo/pull/10820))\n - Minor wording fixes in docs (#10822) ([ipfs/kubo#10822](https://github.com/ipfs/kubo/pull/10822))\n - fix(gateway): gateway-conformance v0.8 (#10818) ([ipfs/kubo#10818](https://github.com/ipfs/kubo/pull/10818))\n - Upgrade to Boxo v0.31.0 (#10819) ([ipfs/kubo#10819](https://github.com/ipfs/kubo/pull/10819))\n - Merge release v0.35.0 ([ipfs/kubo#10815](https://github.com/ipfs/kubo/pull/10815))\n - fix: go-libp2p-kad-dht v0.33.1 (#10814) ([ipfs/kubo#10814](https://github.com/ipfs/kubo/pull/10814))\n - fix: p2p-forge v0.5.1 ignoring /p2p-circuit (#10813) ([ipfs/kubo#10813](https://github.com/ipfs/kubo/pull/10813))\n - Upgrade go-libp2p-kad-dht to v0.33.0 (#10811) ([ipfs/kubo#10811](https://github.com/ipfs/kubo/pull/10811))\n - chore: use go-log/v2 (#10801) ([ipfs/kubo#10801](https://github.com/ipfs/kubo/pull/10801))\n - fix(fuse): ipns error handling and friendly errors (#10807) ([ipfs/kubo#10807](https://github.com/ipfs/kubo/pull/10807))\n - fix(config): wire up `Provider.Enabled` flag (#10804) ([ipfs/kubo#10804](https://github.com/ipfs/kubo/pull/10804))\n - chore: bump version to 0.36.0-dev\n- github.com/ipfs/boxo (v0.30.0 -> v0.33.0):\n - Release v0.33.0 ([ipfs/boxo#974](https://github.com/ipfs/boxo/pull/974))\n - [skip changelog] fix sending empty want from #968 (#975) ([ipfs/boxo#975](https://github.com/ipfs/boxo/pull/975))\n - minor typo fixes (#972) ([ipfs/boxo#972](https://github.com/ipfs/boxo/pull/972))\n - fix: normalize delegated /routing/v1 urls (#971) ([ipfs/boxo#971](https://github.com/ipfs/boxo/pull/971))\n - bitswap/client: Set DontHaveTimeout MinTimeout to 50ms (#965) ([ipfs/boxo#965](https://github.com/ipfs/boxo/pull/965))\n - remove unused code (#967) ([ipfs/boxo#967](https://github.com/ipfs/boxo/pull/967))\n - Fix sending extra wants (#968) ([ipfs/boxo#968](https://github.com/ipfs/boxo/pull/968))\n - Handle Bitswap messages without `Wantlist` (#961) ([ipfs/boxo#961](https://github.com/ipfs/boxo/pull/961))\n - bitswap/httpnet: limit metric cardinality ([ipfs/boxo#957](https://github.com/ipfs/boxo/pull/957))\n - bitswap/httpnet: Sanitize allow/denylist inputs ([ipfs/boxo#964](https://github.com/ipfs/boxo/pull/964))\n - Bitswap: Set DontHaveTimeout/MinTimeout to 200ms. ([ipfs/boxo#959](https://github.com/ipfs/boxo/pull/959))\n - upgrade go-libp2p to v0.42.0 (#960) ([ipfs/boxo#960](https://github.com/ipfs/boxo/pull/960))\n - refactor: use the built-in max/min to simplify the code [skip changelog] (#941) ([ipfs/boxo#941](https://github.com/ipfs/boxo/pull/941))\n - bitswap/httpnet: adjust error logging (#958) ([ipfs/boxo#958](https://github.com/ipfs/boxo/pull/958))\n - docs: reprovider metrics name in changelog (#953) ([ipfs/boxo#953](https://github.com/ipfs/boxo/pull/953))\n - Release v0.32.0 (#952) ([ipfs/boxo#952](https://github.com/ipfs/boxo/pull/952))\n - Remove redundant loop over published blocks (#950) ([ipfs/boxo#950](https://github.com/ipfs/boxo/pull/950))\n - Fix links in README.md (#948) ([ipfs/boxo#948](https://github.com/ipfs/boxo/pull/948))\n - chore(provider): meaningful info level log (#940) ([ipfs/boxo#940](https://github.com/ipfs/boxo/pull/940))\n - feat(provider): reprovide metrics (#944) ([ipfs/boxo#944](https://github.com/ipfs/boxo/pull/944))\n - ci: set up golangci lint in boxo (#943) ([ipfs/boxo#943](https://github.com/ipfs/boxo/pull/943))\n - Do not return error from notify blocks when bitswap shutdown (#947) ([ipfs/boxo#947](https://github.com/ipfs/boxo/pull/947))\n - bitswap/client: broadcast reduction and metrics (#937) ([ipfs/boxo#937](https://github.com/ipfs/boxo/pull/937))\n - fix: typo in HAMT error message ([ipfs/boxo#945](https://github.com/ipfs/boxo/pull/945))\n - bitswap/httpnet: expose the errors on connect when connection impossible ([ipfs/boxo#939](https://github.com/ipfs/boxo/pull/939))\n - fix(unixfs): int check (#936) ([ipfs/boxo#936](https://github.com/ipfs/boxo/pull/936))\n - Remove WithPeerLedger option and PeerLedger interface (#938) ([ipfs/boxo#938](https://github.com/ipfs/boxo/pull/938))\n - fix(gateway): support suffix range requests (#922) ([ipfs/boxo#922](https://github.com/ipfs/boxo/pull/922))\n - Release v0.31.0 ([ipfs/boxo#934](https://github.com/ipfs/boxo/pull/934))\n - Revert \"Remove an unused timestamp from traceability.Block\" (#931) ([ipfs/boxo#931](https://github.com/ipfs/boxo/pull/931))\n - update changelog (#930) ([ipfs/boxo#930](https://github.com/ipfs/boxo/pull/930))\n - Deprecate WithPeerLedger option for bitswap server (#929) ([ipfs/boxo#929](https://github.com/ipfs/boxo/pull/929))\n - refactor: use a more efficient querying method (#921) ([ipfs/boxo#921](https://github.com/ipfs/boxo/pull/921))\n - Use go-car/v2 for reading CAR files in gateway backend (#927) ([ipfs/boxo#927](https://github.com/ipfs/boxo/pull/927))\n - Upgrade go-libp2p-kad-dht v0.33.1 (#924) ([ipfs/boxo#924](https://github.com/ipfs/boxo/pull/924))\n - bitswap/httpnet: Disconnect peers after client errors ([ipfs/boxo#919](https://github.com/ipfs/boxo/pull/919))\n - Remove an unused timestamp from traceability.Block (#923) ([ipfs/boxo#923](https://github.com/ipfs/boxo/pull/923))\n - fix(bitswap/httpnet): idempotent Stop() (#920) ([ipfs/boxo#920](https://github.com/ipfs/boxo/pull/920))\n - Update dependencies (#916) ([ipfs/boxo#916](https://github.com/ipfs/boxo/pull/916))\n- github.com/ipfs/go-block-format (v0.2.1 -> v0.2.2):\n - new version (#62) ([ipfs/go-block-format#62](https://github.com/ipfs/go-block-format/pull/62))\n - Use value receivers for `BasicBlock` methods (#61) ([ipfs/go-block-format#61](https://github.com/ipfs/go-block-format/pull/61))\n- github.com/ipfs/go-ds-badger4 (v0.1.5 -> v0.1.8):\n - new version (#7) ([ipfs/go-ds-badger4#7](https://github.com/ipfs/go-ds-badger4/pull/7))\n - update version (#5) ([ipfs/go-ds-badger4#5](https://github.com/ipfs/go-ds-badger4/pull/5))\n - update dependencies (#4) ([ipfs/go-ds-badger4#4](https://github.com/ipfs/go-ds-badger4/pull/4))\n - new version ([ipfs/go-ds-badger4#3](https://github.com/ipfs/go-ds-badger4/pull/3))\n - use go-datastore without goprocess ([ipfs/go-ds-badger4#2](https://github.com/ipfs/go-ds-badger4/pull/2))\n- github.com/ipfs/go-ds-pebble (v0.5.0 -> v0.5.1):\n - new version (#55) ([ipfs/go-ds-pebble#55](https://github.com/ipfs/go-ds-pebble/pull/55))\n- github.com/ipfs/go-ipfs-cmds (v0.14.1 -> v0.15.0):\n - new version (#287) ([ipfs/go-ipfs-cmds#287](https://github.com/ipfs/go-ipfs-cmds/pull/287))\n - minor document updates (#286) ([ipfs/go-ipfs-cmds#286](https://github.com/ipfs/go-ipfs-cmds/pull/286))\n - Update go log v2 (#285) ([ipfs/go-ipfs-cmds#285](https://github.com/ipfs/go-ipfs-cmds/pull/285))\n - ci: uci/update-go (#281) ([ipfs/go-ipfs-cmds#281](https://github.com/ipfs/go-ipfs-cmds/pull/281))\n- github.com/ipfs/go-ipld-format (v0.6.0 -> v0.6.2):\n - new version (#96) ([ipfs/go-ipld-format#96](https://github.com/ipfs/go-ipld-format/pull/96))\n - bump version (#94) ([ipfs/go-ipld-format#94](https://github.com/ipfs/go-ipld-format/pull/94))\n- github.com/ipfs/go-ipld-legacy (v0.2.1 -> v0.2.2):\n - new version ([ipfs/go-ipld-legacy#25](https://github.com/ipfs/go-ipld-legacy/pull/25))\n- github.com/ipfs/go-test (v0.2.1 -> v0.2.2):\n - new version (#25) ([ipfs/go-test#25](https://github.com/ipfs/go-test/pull/25))\n - Update README.md (#24) ([ipfs/go-test#24](https://github.com/ipfs/go-test/pull/24))\n- github.com/ipfs/go-unixfsnode (v1.10.0 -> v1.10.1):\n - new version ([ipfs/go-unixfsnode#84](https://github.com/ipfs/go-unixfsnode/pull/84))\n- github.com/ipld/go-car/v2 (v2.14.2 -> v2.14.3):\n - bump version ([ipld/go-car#579](https://github.com/ipld/go-car/pull/579))\n - chore: update to boxo merkledag package\n - feat: car debug handles the zero length block ([ipld/go-car#569](https://github.com/ipld/go-car/pull/569))\n - chore(deps): bump github.com/rogpeppe/go-internal from 1.13.1 to 1.14.1 in /cmd ([ipld/go-car#566](https://github.com/ipld/go-car/pull/566))\n - Add a concatenation cli utility ([ipld/go-car#565](https://github.com/ipld/go-car/pull/565))\n- github.com/ipld/go-codec-dagpb (v1.6.0 -> v1.7.0):\n - chore: v1.7.0 bump\n- github.com/libp2p/go-flow-metrics (v0.2.0 -> v0.3.0):\n - chore: release v0.3.0 ([libp2p/go-flow-metrics#38](https://github.com/libp2p/go-flow-metrics/pull/38))\n - go-clock migration ([libp2p/go-flow-metrics#36](https://github.com/libp2p/go-flow-metrics/pull/36))\n- github.com/libp2p/go-libp2p (v0.41.1 -> v0.42.0):\n - Release v0.42.0 (#3318) ([libp2p/go-libp2p#3318](https://github.com/libp2p/go-libp2p/pull/3318))\n - mocknet: notify listeners on listen (#3310) ([libp2p/go-libp2p#3310](https://github.com/libp2p/go-libp2p/pull/3310))\n - autonatv2: add metrics (#3308) ([libp2p/go-libp2p#3308](https://github.com/libp2p/go-libp2p/pull/3308))\n - chore: fix errors reported by golangci-lint ([libp2p/go-libp2p#3295](https://github.com/libp2p/go-libp2p/pull/3295))\n - autonatv2: add Unknown addrs to event (#3305) ([libp2p/go-libp2p#3305](https://github.com/libp2p/go-libp2p/pull/3305))\n - transport: rate limit new connections (#3283) ([libp2p/go-libp2p#3283](https://github.com/libp2p/go-libp2p/pull/3283))\n - basichost: use autonatv2 to verify reachability (#3231) ([libp2p/go-libp2p#3231](https://github.com/libp2p/go-libp2p/pull/3231))\n - chore: Revert \"go-clock migration\" (#3303) ([libp2p/go-libp2p#3303](https://github.com/libp2p/go-libp2p/pull/3303))\n - tcp: ensure tcpGatedMaListener wrapping happens always (#3275) ([libp2p/go-libp2p#3275](https://github.com/libp2p/go-libp2p/pull/3275))\n - go-clock migration ([libp2p/go-libp2p#3293](https://github.com/libp2p/go-libp2p/pull/3293))\n - swarm_test: support more transports for GenSwarm (#3130) ([libp2p/go-libp2p#3130](https://github.com/libp2p/go-libp2p/pull/3130))\n - eventbus: change slow consumer event from error to warn (#3286) ([libp2p/go-libp2p#3286](https://github.com/libp2p/go-libp2p/pull/3286))\n - quicreuse: add some documentation for the package (#3279) ([libp2p/go-libp2p#3279](https://github.com/libp2p/go-libp2p/pull/3279))\n - identify: rate limit id push protocol (#3266) ([libp2p/go-libp2p#3266](https://github.com/libp2p/go-libp2p/pull/3266))\n - fix(pstoreds): add missing log for failed GC record unmarshalling in `purgeStore()` (#3273) ([libp2p/go-libp2p#3273](https://github.com/libp2p/go-libp2p/pull/3273))\n - nat: improve port mapping failure logging (#3261) ([libp2p/go-libp2p#3261](https://github.com/libp2p/go-libp2p/pull/3261))\n - ci: add golangci-lint for linting (#3269) ([libp2p/go-libp2p#3269](https://github.com/libp2p/go-libp2p/pull/3269))\n - build(test_analysis): use `modernc.org/sqlite` directly (#3227) ([libp2p/go-libp2p#3227](https://github.com/libp2p/go-libp2p/pull/3227))\n - chore(certificate): update test vectors (#3242) ([libp2p/go-libp2p#3242](https://github.com/libp2p/go-libp2p/pull/3242))\n - rcmgr: use netip.Prefix as map key instead of string (#3264) ([libp2p/go-libp2p#3264](https://github.com/libp2p/go-libp2p/pull/3264))\n - webrtc: support receiving 256kB messages (#3255) ([libp2p/go-libp2p#3255](https://github.com/libp2p/go-libp2p/pull/3255))\n - peerstore: remove leveldb tests (#3260) ([libp2p/go-libp2p#3260](https://github.com/libp2p/go-libp2p/pull/3260))\n - identify: reduce timeout to 5 seconds (#3259) ([libp2p/go-libp2p#3259](https://github.com/libp2p/go-libp2p/pull/3259))\n - fix(relay): fix data-race in relayFinder (#3258) ([libp2p/go-libp2p#3258](https://github.com/libp2p/go-libp2p/pull/3258))\n - chore: update p2p-forge to v0.5.0 for autotls example (#3257) ([libp2p/go-libp2p#3257](https://github.com/libp2p/go-libp2p/pull/3257))\n - peerstore: remove unused badger tests (#3252) ([libp2p/go-libp2p#3252](https://github.com/libp2p/go-libp2p/pull/3252))\n - chore: using t.TempDir() instead of os.MkdirTemp (#3222) ([libp2p/go-libp2p#3222](https://github.com/libp2p/go-libp2p/pull/3222))\n - chore(examples): p2p-forge/client v0.4.0 (#3211) ([libp2p/go-libp2p#3211](https://github.com/libp2p/go-libp2p/pull/3211))\n - transport: add GatedMaListener type (#3186) ([libp2p/go-libp2p#3186](https://github.com/libp2p/go-libp2p/pull/3186))\n - autonatv2: explicitly handle dns addrs (#3249) ([libp2p/go-libp2p#3249](https://github.com/libp2p/go-libp2p/pull/3249))\n - autonatv2: fix server dial data request policy (#3247) ([libp2p/go-libp2p#3247](https://github.com/libp2p/go-libp2p/pull/3247))\n - webtransport: wrap underlying transport error on stream resets (#3237) ([libp2p/go-libp2p#3237](https://github.com/libp2p/go-libp2p/pull/3237))\n - connmgr: remove WithEmergencyTrim (#3217) ([libp2p/go-libp2p#3217](https://github.com/libp2p/go-libp2p/pull/3217))\n - connmgr: fix transport association bug (#3221) ([libp2p/go-libp2p#3221](https://github.com/libp2p/go-libp2p/pull/3221))\n - webrtc: fix memory leak with udpmux.muxedConnection context (#3243) ([libp2p/go-libp2p#3243](https://github.com/libp2p/go-libp2p/pull/3243))\n - fix(libp2phttp): bound NewStream timeout (#3225) ([libp2p/go-libp2p#3225](https://github.com/libp2p/go-libp2p/pull/3225))\n - conngater: fix incorrect err return value (#3219) ([libp2p/go-libp2p#3219](https://github.com/libp2p/go-libp2p/pull/3219))\n - addrsmanager: extract out addressing logic from basichost (#3075) ([libp2p/go-libp2p#3075](https://github.com/libp2p/go-libp2p/pull/3075))\n- github.com/libp2p/go-socket-activation (v0.1.0 -> v0.1.1):\n - new version (#35) ([libp2p/go-socket-activation#35](https://github.com/libp2p/go-socket-activation/pull/35))\n - Upgrade to go-log/v2 v2.6.0 (#33) ([libp2p/go-socket-activation#33](https://github.com/libp2p/go-socket-activation/pull/33))\n - sync: update CI config files (#20) ([libp2p/go-socket-activation#20](https://github.com/libp2p/go-socket-activation/pull/20))\n - sync: update CI config files (#18) ([libp2p/go-socket-activation#18](https://github.com/libp2p/go-socket-activation/pull/18))\n - sync: update CI config files (#17) ([libp2p/go-socket-activation#17](https://github.com/libp2p/go-socket-activation/pull/17))\n- github.com/libp2p/go-yamux/v5 (v5.0.0 -> v5.0.1):\n - Release v5.0.1\n - fix: deadlock on close (#130) ([libp2p/go-yamux#130](https://github.com/libp2p/go-yamux/pull/130))\n- github.com/multiformats/go-multiaddr (v0.15.0 -> v0.16.0):\n - Release v0.16.0 (#279) ([multiformats/go-multiaddr#279](https://github.com/multiformats/go-multiaddr/pull/279))\n - Rename CaptureStringVal to CaptureString (#278) ([multiformats/go-multiaddr#278](https://github.com/multiformats/go-multiaddr/pull/278))\n - Megular Expressions (#263) ([multiformats/go-multiaddr#263](https://github.com/multiformats/go-multiaddr/pull/263))\n- github.com/multiformats/go-multicodec (v0.9.0 -> v0.9.2):\n - v0.9.2 bump\n - chore: update submodules and go generate\n - chore: v0.9.1 bump\n - chore: update submodules and go generate\n - ci: uci/update-go (#97) ([multiformats/go-multicodec#97](https://github.com/multiformats/go-multicodec/pull/97))\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n - chore: update submodules and go generate\n- github.com/multiformats/go-multistream (v0.6.0 -> v0.6.1):\n - Release v0.6.1 ([multiformats/go-multistream#121](https://github.com/multiformats/go-multistream/pull/121))\n - refactor(lazyClientConn): Use synctest friendly once func ([multiformats/go-multistream#120](https://github.com/multiformats/go-multistream/pull/120))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| sukun | 25 | +7274/-1586 | 140 |\n| galargh | 13 | +1714/-1680 | 115 |\n| rvagg | 2 | +1383/-960 | 6 |\n| Andrew Gillis | 46 | +1226/-564 | 140 |\n| Marco Munizaga | 6 | +1643/-36 | 24 |\n| Hector Sanjuan | 20 | +624/-202 | 40 |\n| Marcin Rataj | 24 | +583/-175 | 49 |\n| Dennis Trautwein | 1 | +134/-14 | 4 |\n| Piotr Galar | 1 | +73/-71 | 23 |\n| Guillaume Michel | 4 | +58/-44 | 23 |\n| Ivan | 1 | +90/-9 | 3 |\n| Will Scott | 1 | +97/-0 | 2 |\n| gammazero | 11 | +47/-30 | 13 |\n| guillaumemichel | 3 | +40/-35 | 21 |\n| Adin Schmahmann | 1 | +58/-17 | 8 |\n| Laurent Senta | 1 | +26/-24 | 4 |\n| pullmerge | 1 | +20/-16 | 5 |\n| vladopajic | 1 | +20/-14 | 1 |\n| Probot | 1 | +18/-4 | 1 |\n| Dmitry Markin | 1 | +13/-9 | 2 |\n| overallteach | 1 | +4/-12 | 3 |\n| web3-bot | 5 | +9/-6 | 7 |\n| Pavel Zbitskiy | 1 | +14/-1 | 1 |\n| Rod Vagg | 5 | +7/-7 | 5 |\n| argentpapa | 1 | +3/-10 | 1 |\n| GarmashAlex | 1 | +8/-3 | 1 |\n| huochexizhan | 1 | +3/-3 | 1 |\n| VolodymyrBg | 1 | +2/-3 | 1 |\n| levisyin | 1 | +2/-2 | 2 |\n| b00f | 1 | +3/-0 | 1 |\n| achingbrain | 1 | +1/-1 | 1 |\n| Ocenka | 1 | +1/-1 | 1 |\n| Dreamacro | 1 | +1/-1 | 1 |\n| Štefan Baebler | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.37.md", "content": "# Kubo changelog v0.37\n\n\n\nThis release was brought to you by the [Shipyard](https://ipshipyard.com/) team.\n\n- [v0.37.0](#v0370)\n\n## v0.37.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [🚀 Repository migration from v16 to v17 with embedded tooling](#-repository-migration-from-v16-to-v17-with-embedded-tooling)\n - [🚦 Gateway concurrent request limits and retrieval timeouts](#-gateway-concurrent-request-limits-and-retrieval-timeouts)\n - [🔧 AutoConf: Complete control over network defaults](#-autoconf-complete-control-over-network-defaults)\n - [🗑️ Clear provide queue when reprovide strategy changes](#-clear-provide-queue-when-reprovide-strategy-changes)\n - [🪵 Revamped `ipfs log level` command](#-revamped-ipfs-log-level-command)\n - [📌 Named pins in `ipfs add` command](#-named-pins-in-ipfs-add-command)\n - [📝 New IPNS publishing options](#-new-ipns-publishing-options)\n - [🔢 Custom sequence numbers in `ipfs name publish`](#-custom-sequence-numbers-in-ipfs-name-publish)\n - [⚙️ `Reprovider.Strategy` is now consistently respected](#-reprovider-strategy-is-now-consistently-respected)\n - [⚙️ `Reprovider.Strategy=all`: improved memory efficiency](#-reproviderstrategyall-improved-memory-efficiency)\n - [🧹 Removed unnecessary dependencies](#-removed-unnecessary-dependencies)\n - [🔍 Improved `ipfs cid`](#-improved-ipfs-cid)\n - [⚠️ Deprecated `ipfs stats reprovide`](#-deprecated-ipfs-stats-reprovide)\n - [🔄 AutoRelay now uses all connected peers for relay discovery](#-autorelay-now-uses-all-connected-peers-for-relay-discovery)\n - [📊 Anonymous telemetry for better feature prioritization](#-anonymous-telemetry-for-better-feature-prioritization)\n- [📦️ Important dependency updates](#-important-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\nKubo 0.37.0 introduces embedded repository migrations, gateway resource protection, complete AutoConf control, improved reprovider strategies, and anonymous telemetry for better feature prioritization. This release significantly improves memory efficiency, network configuration flexibility, and operational reliability while maintaining full backward compatibility.\n\n### 🔦 Highlights\n\n#### 🚀 Repository migration from v16 to v17 with embedded tooling\n\nThis release migrates the Kubo repository from version 16 to version 17. Migrations are now built directly into the binary - completing in milliseconds without internet access or external downloads.\n\n`ipfs daemon --migrate` performs migrations automatically. Manual migration: `ipfs repo migrate --to=17` (or `--to=16 --allow-downgrade` for compatibility). Embedded migrations apply to v17+; older versions still require external tools.\n\n**Legacy migration deprecation**: Support for legacy migrations that download binaries from the internet will be removed in a future version. Only embedded migrations for the last 3 releases will be supported. Users with very old repositories should update in stages rather than skipping multiple versions.\n\n#### 🚦 Gateway concurrent request limits and retrieval timeouts\n\nNew configurable limits protect gateway resources during high load:\n\n- **[`Gateway.RetrievalTimeout`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewayretrievaltimeout)** (default: 30s): Maximum duration for content retrieval. Returns 504 Gateway Timeout when exceeded - applies to both initial retrieval (time to first byte) and between subsequent writes.\n- **[`Gateway.MaxConcurrentRequests`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaymaxconcurrentrequests)** (default: 4096): Limits concurrent HTTP requests. Returns 429 Too Many Requests when exceeded. Protects nodes from traffic spikes and resource exhaustion, especially useful behind reverse proxies without rate-limiting.\n\nNew Prometheus metrics for monitoring:\n\n- `ipfs_http_gw_concurrent_requests`: Current requests being processed\n- `ipfs_http_gw_responses_total`: HTTP responses by status code\n- `ipfs_http_gw_retrieval_timeouts_total`: Timeouts by status code and truncation status\n\nTuning tips:\n\n- Monitor metrics to understand gateway behavior and adjust based on observations\n- Watch `ipfs_http_gw_concurrent_requests` for saturation\n- Track `ipfs_http_gw_retrieval_timeouts_total` vs success rates to identify timeout patterns indicating routing or storage provider issues\n\n#### 🔧 AutoConf: Complete control over network defaults\n\nConfiguration fields now support `[\"auto\"]` placeholders that resolve to network defaults from [`AutoConf.URL`](https://github.com/ipfs/kubo/blob/master/docs/config.md#autoconfurl). These defaults can be inspected, replaced with custom values, or disabled entirely. Previously, empty configuration fields like `Routing.DelegatedRouters: []` would use hardcoded defaults - this system makes those defaults explicit through `\"auto\"` values. When upgrading to Kubo 0.37, custom configurations remain unchanged.\n\nNew `--expand-auto` flag shows resolved values for any config field:\n\n```bash\nipfs config show --expand-auto # View all resolved endpoints\nipfs config Bootstrap --expand-auto # Check specific values\nipfs config Routing.DelegatedRouters --expand-auto\nipfs config DNS.Resolvers --expand-auto\n```\n\nConfiguration can be managed via:\n- Replace `\"auto\"` with custom endpoints or set `[]` to disable features\n- Switch modes with `--profile=autoconf-on|autoconf-off`\n- Configure via `AutoConf.Enabled` and custom manifests via `AutoConf.URL`\n\n```bash\n# Enable automatic configuration\nipfs config profiles apply autoconf-on\n\n# Or manually set specific fields\nipfs config Bootstrap '[\"auto\"]'\nipfs config --json DNS.Resolvers '{\".\": [\"https://dns.example.com/dns-query\"], \"eth.\": [\"auto\"]}'\n```\n\nOrganizations can host custom AutoConf manifests for private networks. See [AutoConf documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#autoconf) and format spec at https://conf.ipfs-mainnet.org/\n\n#### 🗑️ Clear provide queue when reprovide strategy changes\n\nChanging [`Reprovider.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy) and restarting Kubo now automatically clears the provide queue. Only content matching the new strategy will be announced.\n\nManual queue clearing is also available:\n\n- `ipfs provide clear` - clear all queued content announcements\n\n> [!NOTE]\n> Upgrading to Kubo 0.37 will automatically clear any preexisting provide queue. The next time `Reprovider.Interval` hits, `Reprovider.Strategy` will be executed on a clean slate, ensuring consistent behavior with your current configuration.\n\n#### 🪵 Revamped `ipfs log level` command\n\nThe `ipfs log level` command has been completely revamped to support both getting and setting log levels with a unified interface.\n\n**New: Getting log levels**\n\n- `ipfs log level` - Shows default level only\n- `ipfs log level all` - Shows log level for every subsystem, including default level\n- `ipfs log level foo` - Shows log level for a specific subsystem only\n- Kubo RPC API: `POST /api/v0/log/level?arg=`\n\n**Enhanced: Setting log levels**\n\n- `ipfs log level foo debug` - Sets \"foo\" subsystem to \"debug\" level\n- `ipfs log level all info` - Sets all subsystems to \"info\" level (convenient, no escaping)\n- `ipfs log level '*' info` - Equivalent to above but requires shell escaping\n- `ipfs log level foo default` - Sets \"foo\" subsystem to current default level\n\nThe command now provides full visibility into your current logging configuration while maintaining full backward compatibility. Both `all` and `*` work for specifying all subsystems, with `all` being more convenient since it doesn't require shell escaping.\n\n#### 🧷 Named pins in `ipfs add` command\n\nAdded `--pin-name` flag to `ipfs add` for assigning names to pins.\n\n```console\n$ ipfs add --pin-name=testname cat.jpg\nadded bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi cat.jpg\n\n$ ipfs pin ls --names\nbafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi recursive testname\n```\n\n#### 📝 New IPNS publishing options\n\nAdded support for controlling IPNS record publishing strategies with new command flags and configuration.\n\n**New command flags:**\n```bash\n# Publish without network connectivity (local datastore only)\nipfs name publish --allow-offline /ipfs/QmHash\n\n# Publish without DHT connectivity (uses local datastore and HTTP delegated publishers)\nipfs name publish --allow-delegated /ipfs/QmHash\n```\n\n**Delegated publishers configuration:**\n\n[`Ipns.DelegatedPublishers`](https://github.com/ipfs/kubo/blob/master/docs/config.md#ipnsdelegatedpublishers) configures HTTP endpoints for IPNS publishing. Supports `\"auto\"` for network defaults or custom HTTP endpoints. The `--allow-delegated` flag enables publishing through these endpoints without requiring DHT connectivity, useful for nodes behind restrictive networks or during testing.\n\n#### 🔢 Custom sequence numbers in `ipfs name publish`\n\nAdded `--sequence` flag to `ipfs name publish` for setting custom sequence numbers in IPNS records. This enables advanced use cases like manually coordinating updates across multiple nodes. See `ipfs name publish --help` for details.\n\n#### ⚙️ `Reprovider.Strategy` is now consistently respected\n\nPrior to this version, files added, blocks received etc. were \"provided\" to the network (announced on the DHT) regardless of the [\"reproviding strategy\" setting](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy). For example:\n\n- Strategy set to \"pinned\" + `ipfs add --pin=false` → file was provided regardless\n- Strategy set to \"roots\" + `ipfs pin add` → all blocks (not only the root) were provided\n\nOnly the periodic \"reproviding\" action (runs every 22h by default) respected the strategy.\n\nThis was inefficient as content that should not be provided was getting provided once. Now all operations respect `Reprovider.Strategy`. If set to \"roots\", no blocks other than pin roots will be provided regardless of what is fetched, added etc.\n\n> [!NOTE]\n> **Behavior change:** The `--offline` flag no longer affects providing behavior. Both `ipfs add` and `ipfs --offline add` now provide blocks according to the reproviding strategy when run against an online daemon (previously `--offline add` did not provide). Since `ipfs add` has been nearly as fast as offline mode [since v0.35](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.35.md#fast-ipfs-add-in-online-mode), `--offline` is rarely needed. To run truly offline operations, use `ipfs --offline daemon`.\n\n#### ⚙️ `Reprovider.Strategy=all`: improved memory efficiency\n\nThe memory cost of `Reprovider.Strategy=all` no longer grows with the number of pins. The strategy now processes blocks directly from the datastore in undefined order, eliminating the memory pressure tied to the number of pins.\n\nAs part of this improvement, the `flat` reprovider strategy has been renamed to `all` (the default). This cleanup removes the workaround introduced in v0.28 for pin root prioritization. With the introduction of more granular strategies like [`pinned+mfs`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy), we can now optimize the default `all` strategy for lower memory usage without compromising users who need pin root prioritization ([rationale](https://github.com/ipfs/kubo/pull/10928#issuecomment-3211040182)).\n\n> [!NOTE]\n> **Migration guidance:** If you experience undesired announcement delays of root CIDs with the new `all` strategy, switch to `pinned+mfs` for root prioritization.\n\n#### 🧹 Removed unnecessary dependencies\n\nKubo has been cleaned up by removing unnecessary dependencies and packages:\n\n- Removed `thirdparty/assert` (replaced by `github.com/stretchr/testify/require`)\n- Removed `thirdparty/dir` (replaced by `misc/fsutil`)\n- Removed `thirdparty/notifier` (unused)\n- Removed `goprocess` dependency (replaced with native Go `context` patterns)\n\nThese changes reduce the dependency footprint while improving code maintainability and following Go best practices.\n\n#### 🔍 Improved `ipfs cid`\n\nCertain `ipfs cid` commands can now be run without a daemon or repository, and return correct exit code 1 on error, making it easier to perform CID conversion in scripts and CI/CD pipelines.\n\nWhile at it, we also fixed unicode support in `ipfs cid bases --prefix` to correctly show `base256emoji` 🚀 :-)\n\n#### ⚠️ Deprecated `ipfs stats reprovide`\n\nThe `ipfs stats reprovide` command has moved to `ipfs provide stat`. This was done to organize provider commands in one location.\n\n> [!NOTE]\n> `ipfs stats reprovide` still works, but is marked as deprecated and will be removed in a future release.\n\n#### 🔄 AutoRelay now uses all connected peers for relay discovery\n\nAutoRelay's relay discovery now includes all connected peers as potential relay candidates, not just peers discovered through the DHT. This allows peers connected via HTTP routing and manual `ipfs swarm connect` commands to serve as relays, improving connectivity for nodes using non-DHT routing configurations.\n\n#### 📊 Anonymous telemetry for better feature prioritization\n\nPer a suggestion from the IPFS Foundation, Kubo now sends optional anonymized telemetry information to Shipyard [maintainers](https://github.com/ipshipyard/roadmaps/issues/20).\n\n**Privacy first**: The telemetry system collects only anonymous data - no personally identifiable information, file paths, or content data. A random UUID is generated on first run for anonymous identification. Users are notified before any data is sent and have time to opt-out.\n\n**Why**: We want to better understand Kubo usage across the ecosystem so we can better direct funding and work efforts. For example, we have little insights into how many nodes are NAT'ed and rely on AutoNAT for reachability. Some of the information can be inferred by crawling the network or logging `/identify` details in the bootstrappers, but users have no way of opting out from that, so we believe it is more transparent to concentrate this functionality in one place.\n\n**What**: Currently, we send the following anonymous metrics:\n\n
Click to see telemetry metrics example\n\n```\n \"uuid\": \"\",\n \"agent_version\": \"kubo/0.37.0-dev\",\n \"private_network\": false,\n \"bootstrappers_custom\": false,\n \"repo_size_bucket\": 1073741824,\n \"uptime_bucket\": 86400000000000,\n \"reprovider_strategy\": \"pinned\",\n \"routing_type\": \"auto\",\n \"routing_accelerated_dht_client\": false,\n \"routing_delegated_count\": 0,\n \"autonat_service_mode\": \"enabled\",\n \"autonat_reachability\": \"\",\n \"autoconf\": true,\n \"autoconf_custom\": false,\n \"swarm_enable_hole_punching\": true,\n \"swarm_circuit_addresses\": false,\n \"swarm_ipv4_public_addresses\": true,\n \"swarm_ipv6_public_addresses\": true,\n \"auto_tls_auto_wss\": true,\n \"auto_tls_domain_suffix_custom\": false,\n \"discovery_mdns_enabled\": true,\n \"platform_os\": \"linux\",\n \"platform_arch\": \"amd64\",\n \"platform_containerized\": false,\n \"platform_vm\": false\n```\n\n
\n\nThe exact data sent for your node can be inspected by setting `GOLOG_LOG_LEVEL=\"telemetry=debug\"`. Users will see an informative message the first time they launch a telemetry-enabled daemon, with time to opt-out before any data is collected. Telemetry data is sent every 24h, with the first collection starting 15 minutes after daemon launch.\n\n**User control**: You can opt-out at any time:\n\n- Set environment variable `IPFS_TELEMETRY=off` before starting the daemon\n- Or run `ipfs config Plugins.Plugins.telemetry.Config.Mode off` and restart the daemon\n\nThe telemetry plugin code lives in `plugin/plugins/telemetry`.\n\nLearn more: [`/kubo/docs/telemetry.md`](https://github.com/ipfs/kubo/blob/master/docs/telemetry.md)\n\n### 📦️ Important dependency updates\n\n- update `boxo` to [v0.34.0](https://github.com/ipfs/boxo/releases/tag/v0.34.0) (incl. [v0.33.1](https://github.com/ipfs/boxo/releases/tag/v0.33.1))\n- update `go-libp2p` to [v0.43.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.43.0)\n- update `go-libp2p-kad-dht` to [v0.34.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.34.0)\n- update `go-libp2p-pubsub` to [v0.14.2](https://github.com/libp2p/go-libp2p-pubsub/releases/tag/v0.14.2) (incl. [v0.14.1](https://github.com/libp2p/go-libp2p-pubsub/releases/tag/v0.14.1), [v0.14.0](https://github.com/libp2p/go-libp2p-pubsub/releases/tag/v0.14.0))\n- update `ipfs-webui` to [v4.8.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.8.0)\n- update to [Go 1.25](https://go.dev/doc/go1.25)\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: set version to v0.37.0\n - feat(ci): docker linting (#10927) ([ipfs/kubo#10927](https://github.com/ipfs/kubo/pull/10927))\n - fix: disable telemetry in test profile (#10931) ([ipfs/kubo#10931](https://github.com/ipfs/kubo/pull/10931))\n - fix: harness tests random panic (#10933) ([ipfs/kubo#10933](https://github.com/ipfs/kubo/pull/10933))\n - chore: v0.37.0-rc1\n - feat: Reprovider.Strategy: rename \"flat\" to \"all\" (#10928) ([ipfs/kubo#10928](https://github.com/ipfs/kubo/pull/10928))\n - docs: improve `ipfs add --help` (#10926) ([ipfs/kubo#10926](https://github.com/ipfs/kubo/pull/10926))\n - feat: optimize docker builds (#10925) ([ipfs/kubo#10925](https://github.com/ipfs/kubo/pull/10925))\n - feat(config): AutoConf with \"auto\" placeholders (#10883) ([ipfs/kubo#10883](https://github.com/ipfs/kubo/pull/10883))\n - fix(ci): make NewRandPort thread-safe (#10921) ([ipfs/kubo#10921](https://github.com/ipfs/kubo/pull/10921))\n - fix: resolve TestAddMultipleGCLive race condition (#10916) ([ipfs/kubo#10916](https://github.com/ipfs/kubo/pull/10916))\n - feat: telemetry plugin (#10866) ([ipfs/kubo#10866](https://github.com/ipfs/kubo/pull/10866))\n - fix typos in docs and comments (#10920) ([ipfs/kubo#10920](https://github.com/ipfs/kubo/pull/10920))\n - Upgrade to Boxo v0.34.0 (#10917) ([ipfs/kubo#10917](https://github.com/ipfs/kubo/pull/10917))\n - test: fix flaky repo verify (#10743) ([ipfs/kubo#10743](https://github.com/ipfs/kubo/pull/10743))\n - feat(config): `Gateway.RetrievalTimeout|MaxConcurrentRequests` (#10905) ([ipfs/kubo#10905](https://github.com/ipfs/kubo/pull/10905))\n - chore: replace random test utils with equivalents in go-test/random (#10915) ([ipfs/kubo#10915](https://github.com/ipfs/kubo/pull/10915))\n - feat: require go1.25 for building kubo (#10913) ([ipfs/kubo#10913](https://github.com/ipfs/kubo/pull/10913))\n - feat(ci): reusable spellcheck from unified CI (#10873) ([ipfs/kubo#10873](https://github.com/ipfs/kubo/pull/10873))\n - fix(ci): docker build (#10914) ([ipfs/kubo#10914](https://github.com/ipfs/kubo/pull/10914))\n - Replace `uber-go/multierr` with `errors.Join` (#10912) ([ipfs/kubo#10912](https://github.com/ipfs/kubo/pull/10912))\n - feat(ipns): support passing custom sequence number during publishing (#10851) ([ipfs/kubo#10851](https://github.com/ipfs/kubo/pull/10851))\n - fix(relay): feed connected peers to AutoRelay discovery (#10901) ([ipfs/kubo#10901](https://github.com/ipfs/kubo/pull/10901))\n - fix(sharness): no blocking on unclean FUSE unmount (#10906) ([ipfs/kubo#10906](https://github.com/ipfs/kubo/pull/10906))\n - feat: add query functionality to log level command (#10885) ([ipfs/kubo#10885](https://github.com/ipfs/kubo/pull/10885))\n - fix(ci): switch to debian:bookworm-slim\n - Fix failing FUSE test (#10904) ([ipfs/kubo#10904](https://github.com/ipfs/kubo/pull/10904))\n - fix(cmd): exit 1 on error (#10903) ([ipfs/kubo#10903](https://github.com/ipfs/kubo/pull/10903))\n - feat: go-libp2p v0.43.0 (#10892) ([ipfs/kubo#10892](https://github.com/ipfs/kubo/pull/10892))\n - fix: `ipfs cid` without repo (#10897) ([ipfs/kubo#10897](https://github.com/ipfs/kubo/pull/10897))\n - client/rpc: re-enable tests on windows. (#10895) ([ipfs/kubo#10895](https://github.com/ipfs/kubo/pull/10895))\n - fix: Provide according to Reprovider.Strategy (#10886) ([ipfs/kubo#10886](https://github.com/ipfs/kubo/pull/10886))\n - feat: ipfs-webui v4.8.0 (#10902) ([ipfs/kubo#10902](https://github.com/ipfs/kubo/pull/10902))\n - refactor: move `ipfs stat provide/reprovide` to `ipfs provide stat` (#10896) ([ipfs/kubo#10896](https://github.com/ipfs/kubo/pull/10896))\n - Bitswap: use a single ConnectEventManager. ([ipfs/kubo#10889](https://github.com/ipfs/kubo/pull/10889))\n - feat(add): add support for naming pinned CIDs (#10877) ([ipfs/kubo#10877](https://github.com/ipfs/kubo/pull/10877))\n - refactor: remove goprocess (#10872) ([ipfs/kubo#10872](https://github.com/ipfs/kubo/pull/10872))\n - feat(daemon): accelerated client startup note (#10859) ([ipfs/kubo#10859](https://github.com/ipfs/kubo/pull/10859))\n - docs:added GOLOG_LOG_LEVEL to debug-guide for logging more info (#10894) ([ipfs/kubo#10894](https://github.com/ipfs/kubo/pull/10894))\n - core: Add a ContentDiscovery field ([ipfs/kubo#10890](https://github.com/ipfs/kubo/pull/10890))\n - chore: update go-libp2p and p2p-forge (#10887) ([ipfs/kubo#10887](https://github.com/ipfs/kubo/pull/10887))\n - Upgrade to Boxo v0.33.1 (#10888) ([ipfs/kubo#10888](https://github.com/ipfs/kubo/pull/10888))\n - remove unneeded thirdparty packages (#10871) ([ipfs/kubo#10871](https://github.com/ipfs/kubo/pull/10871))\n - provider: clear provide queue when reprovide strategy changes (#10863) ([ipfs/kubo#10863](https://github.com/ipfs/kubo/pull/10863))\n - chore: merge release v0.36.0 ([ipfs/kubo#10868](https://github.com/ipfs/kubo/pull/10868))\n - docs: release checklist fixes from 0.36 (#10861) ([ipfs/kubo#10861](https://github.com/ipfs/kubo/pull/10861))\n - docs(config): add network exposure considerations (#10856) ([ipfs/kubo#10856](https://github.com/ipfs/kubo/pull/10856))\n - fix: handling of EDITOR env var (#10855) ([ipfs/kubo#10855](https://github.com/ipfs/kubo/pull/10855))\n - refactor: use slices.Sort where appropriate (#10858) ([ipfs/kubo#10858](https://github.com/ipfs/kubo/pull/10858))\n - Upgrade to Boxo v0.33.0 (#10857) ([ipfs/kubo#10857](https://github.com/ipfs/kubo/pull/10857))\n - chore: Upgrade github.com/cockroachdb/pebble/v2 to v2.0.6 for Go 1.25 support (#10850) ([ipfs/kubo#10850](https://github.com/ipfs/kubo/pull/10850))\n - core:constructor: add a log line about http retrieval ([ipfs/kubo#10852](https://github.com/ipfs/kubo/pull/10852))\n - chore: p2p-forge v0.6.0 + go-libp2p 0.42.0 (#10840) ([ipfs/kubo#10840](https://github.com/ipfs/kubo/pull/10840))\n - docs: fix minor typos (#10849) ([ipfs/kubo#10849](https://github.com/ipfs/kubo/pull/10849))\n - Replace use of go-car v1 with go-car/v2 (#10845) ([ipfs/kubo#10845](https://github.com/ipfs/kubo/pull/10845))\n - chore: 0.37.0-dev\n- github.com/ipfs/boxo (v0.33.0 -> v0.34.0):\n - Release v0.34.0 ([ipfs/boxo#1003](https://github.com/ipfs/boxo/pull/1003))\n - blockstore: remove HashOnRead ([ipfs/boxo#1001](https://github.com/ipfs/boxo/pull/1001))\n - Update go-log to v2.8.1 ([ipfs/boxo#998](https://github.com/ipfs/boxo/pull/998))\n - feat: autoconf client library (#997) ([ipfs/boxo#997](https://github.com/ipfs/boxo/pull/997))\n - feat(gateway): concurrency and retrieval timeout limits (#994) ([ipfs/boxo#994](https://github.com/ipfs/boxo/pull/994))\n - update dependencies ([ipfs/boxo#999](https://github.com/ipfs/boxo/pull/999))\n - fix: cidqueue gc must iterate all elements in queue ([ipfs/boxo#1000](https://github.com/ipfs/boxo/pull/1000))\n - Replace `uber-go/multierr` with `errors.Join` ([ipfs/boxo#996](https://github.com/ipfs/boxo/pull/996))\n - feat(namesys/IPNSPublisher): expose ability to set Sequence (#962) ([ipfs/boxo#962](https://github.com/ipfs/boxo/pull/962))\n - upgrade to go-libp2p v0.43.0 ([ipfs/boxo#993](https://github.com/ipfs/boxo/pull/993))\n - Remove providing Exchange. Call Provide() from relevant places. ([ipfs/boxo#976](https://github.com/ipfs/boxo/pull/976))\n - reprovider: s/initial/initial ([ipfs/boxo#992](https://github.com/ipfs/boxo/pull/992))\n - Release v0.33.1 ([ipfs/boxo#991](https://github.com/ipfs/boxo/pull/991))\n - fix(bootstrap): filter-out peers behind relays (#987) ([ipfs/boxo#987](https://github.com/ipfs/boxo/pull/987))\n - Bitswap: fix double-worker in connectEventManager. Logging improvements. ([ipfs/boxo#986](https://github.com/ipfs/boxo/pull/986))\n - upgrade to go-libp2p v0.42.1 (#988) ([ipfs/boxo#988](https://github.com/ipfs/boxo/pull/988))\n - bitswap/httpnet: fix sudden stop of http retrieval requests (#984) ([ipfs/boxo#984](https://github.com/ipfs/boxo/pull/984))\n - bitswap/client: disable use of traceability block by default (#956) ([ipfs/boxo#956](https://github.com/ipfs/boxo/pull/956))\n - test(gateway): fix race in TestCarBackendTar (#985) ([ipfs/boxo#985](https://github.com/ipfs/boxo/pull/985))\n - Shutdown the sessionWantSender changes queue when session is shutdown (#983) ([ipfs/boxo#983](https://github.com/ipfs/boxo/pull/983))\n - bitswap/httpnet: start pinging before signaling Connected ([ipfs/boxo#982](https://github.com/ipfs/boxo/pull/982))\n - Queue all changes in order using non-blocking async queue ([ipfs/boxo#981](https://github.com/ipfs/boxo/pull/981))\n - bitswap/httpnet: fix peers silently stopping from doing http requests ([ipfs/boxo#980](https://github.com/ipfs/boxo/pull/980))\n - provider: clear provide queue (#978) ([ipfs/boxo#978](https://github.com/ipfs/boxo/pull/978))\n - update dependencies ([ipfs/boxo#977](https://github.com/ipfs/boxo/pull/977))\n- github.com/ipfs/go-datastore (v0.8.2 -> v0.8.3):\n - new version (#245) ([ipfs/go-datastore#245](https://github.com/ipfs/go-datastore/pull/245))\n - sort using slices.Sort (#243) ([ipfs/go-datastore#243](https://github.com/ipfs/go-datastore/pull/243))\n - Replace `uber-go/multierr` with `errors.Join` (#242) ([ipfs/go-datastore#242](https://github.com/ipfs/go-datastore/pull/242))\n - replace gopkg.in/check.v1 with github.com/stretchr/testify (#241) ([ipfs/go-datastore#241](https://github.com/ipfs/go-datastore/pull/241))\n- github.com/ipfs/go-ipld-cbor (v0.2.0 -> v0.2.1):\n - new version ([ipfs/go-ipld-cbor#111](https://github.com/ipfs/go-ipld-cbor/pull/111))\n - update dependencies ([ipfs/go-ipld-cbor#110](https://github.com/ipfs/go-ipld-cbor/pull/110))\n- github.com/ipfs/go-log/v2 (v2.6.0 -> v2.8.1):\n - new version (#171) ([ipfs/go-log#171](https://github.com/ipfs/go-log/pull/171))\n - feat: add LevelEnabled function to check if log level enabled (#170) ([ipfs/go-log#170](https://github.com/ipfs/go-log/pull/170))\n - Replace `uber-go/multierr` with `errors.Join` (#168) ([ipfs/go-log#168](https://github.com/ipfs/go-log/pull/168))\n - new version (#167) ([ipfs/go-log#167](https://github.com/ipfs/go-log/pull/167))\n - Test using testify package (#166) ([ipfs/go-log#166](https://github.com/ipfs/go-log/pull/166))\n - Revise the loglevel API to be more golang idiomatic (#165) ([ipfs/go-log#165](https://github.com/ipfs/go-log/pull/165))\n - new version (#164) ([ipfs/go-log#164](https://github.com/ipfs/go-log/pull/164))\n - feat: add GetLogLevel and GetAllLogLevels (#160) ([ipfs/go-log#160](https://github.com/ipfs/go-log/pull/160))\n- github.com/ipfs/go-test (v0.2.2 -> v0.2.3):\n - new version (#30) ([ipfs/go-test#30](https://github.com/ipfs/go-test/pull/30))\n - fix: multihash random generation (#28) ([ipfs/go-test#28](https://github.com/ipfs/go-test/pull/28))\n - Add RandomName function to generate random filename (#26) ([ipfs/go-test#26](https://github.com/ipfs/go-test/pull/26))\n- github.com/libp2p/go-libp2p (v0.42.0 -> v0.43.0):\n - Release v0.43 (#3353) ([libp2p/go-libp2p#3353](https://github.com/libp2p/go-libp2p/pull/3353))\n - basichost: fix deadlock with addrs_manager (#3348) ([libp2p/go-libp2p#3348](https://github.com/libp2p/go-libp2p/pull/3348))\n - basichost: fix Addrs docstring (#3341) ([libp2p/go-libp2p#3341](https://github.com/libp2p/go-libp2p/pull/3341))\n - quic: upgrade quic-go to v0.53 (#3323) ([libp2p/go-libp2p#3323](https://github.com/libp2p/go-libp2p/pull/3323))\n- github.com/libp2p/go-libp2p-kad-dht (v0.33.1 -> v0.34.0):\n - chore: release v0.34.0 (#1130) ([libp2p/go-libp2p-kad-dht#1130](https://github.com/libp2p/go-libp2p-kad-dht/pull/1130))\n - make crawler protocol messenger configurable (#1128) ([libp2p/go-libp2p-kad-dht#1128](https://github.com/libp2p/go-libp2p-kad-dht/pull/1128))\n - fix: move non-error log to warning level (#1119) ([libp2p/go-libp2p-kad-dht#1119](https://github.com/libp2p/go-libp2p-kad-dht/pull/1119))\n - migrate providers package (#1094) ([libp2p/go-libp2p-kad-dht#1094](https://github.com/libp2p/go-libp2p-kad-dht/pull/1094))\n- github.com/libp2p/go-libp2p-pubsub (v0.13.1 -> v0.14.2):\n - Release v0.14.2 (#629) ([libp2p/go-libp2p-pubsub#629](https://github.com/libp2p/go-libp2p-pubsub/pull/629))\n - Fix test races and enable race tests in CI (#626) ([libp2p/go-libp2p-pubsub#626](https://github.com/libp2p/go-libp2p-pubsub/pull/626))\n - Fix race when calling Preprocess and msg ID generator(#627) ([libp2p/go-libp2p-pubsub#627](https://github.com/libp2p/go-libp2p-pubsub/pull/627))\n - Release v0.14.1 (#623) ([libp2p/go-libp2p-pubsub#623](https://github.com/libp2p/go-libp2p-pubsub/pull/623))\n - fix(BatchPublishing): Make topic.AddToBatch threadsafe (#622) ([libp2p/go-libp2p-pubsub#622](https://github.com/libp2p/go-libp2p-pubsub/pull/622))\n - Release v0.14.0 (#614) ([libp2p/go-libp2p-pubsub#614](https://github.com/libp2p/go-libp2p-pubsub/pull/614))\n - refactor: 10x faster RPC splitting (#615) ([libp2p/go-libp2p-pubsub#615](https://github.com/libp2p/go-libp2p-pubsub/pull/615))\n - test: Fix flaky TestMessageBatchPublish (#616) ([libp2p/go-libp2p-pubsub#616](https://github.com/libp2p/go-libp2p-pubsub/pull/616))\n - Send IDONTWANT before first publish (#612) ([libp2p/go-libp2p-pubsub#612](https://github.com/libp2p/go-libp2p-pubsub/pull/612))\n - feat(gossipsub): Add MessageBatch (#607) ([libp2p/go-libp2p-pubsub#607](https://github.com/libp2p/go-libp2p-pubsub/pull/607))\n - fix(IDONTWANT)!: Do not IDONTWANT your sender (#609) ([libp2p/go-libp2p-pubsub#609](https://github.com/libp2p/go-libp2p-pubsub/pull/609))\n- github.com/multiformats/go-multiaddr (v0.16.0 -> v0.16.1):\n - Release v0.16.1 (#281) ([multiformats/go-multiaddr#281](https://github.com/multiformats/go-multiaddr/pull/281))\n - reduce allocations in Bytes() and manet methods (#280) ([multiformats/go-multiaddr#280](https://github.com/multiformats/go-multiaddr/pull/280))\n- github.com/whyrusleeping/cbor-gen (v0.1.2 -> v0.3.1):\n - fix: capture field count early for \"optional\" length check (#112) ([whyrusleeping/cbor-gen#112](https://github.com/whyrusleeping/cbor-gen/pull/112))\n - doc: basic cbor-gen documentation (#110) ([whyrusleeping/cbor-gen#110](https://github.com/whyrusleeping/cbor-gen/pull/110))\n - feat: add support for optional fields at the end of tuple structs (#109) ([whyrusleeping/cbor-gen#109](https://github.com/whyrusleeping/cbor-gen/pull/109))\n - Regenerate test files ([whyrusleeping/cbor-gen#107](https://github.com/whyrusleeping/cbor-gen/pull/107))\n - improve allocations in map serialization ([whyrusleeping/cbor-gen#105](https://github.com/whyrusleeping/cbor-gen/pull/105))\n - fixed array in struct instead of heap slice ([whyrusleeping/cbor-gen#104](https://github.com/whyrusleeping/cbor-gen/pull/104))\n - optionally sort type names in generated code file ([whyrusleeping/cbor-gen#102](https://github.com/whyrusleeping/cbor-gen/pull/102))\n - fix handling of an []*string field ([whyrusleeping/cbor-gen#101](https://github.com/whyrusleeping/cbor-gen/pull/101))\n - fix: reject negative big integers ([whyrusleeping/cbor-gen#100](https://github.com/whyrusleeping/cbor-gen/pull/100))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marcin Rataj | 26 | +16033/-755 | 176 |\n| Andrew Gillis | 35 | +2656/-1911 | 142 |\n| Hector Sanjuan | 30 | +2638/-760 | 114 |\n| Marco Munizaga | 11 | +1244/-362 | 41 |\n| Russell Dempsey | 2 | +1031/-33 | 7 |\n| Guillaume Michel | 4 | +899/-65 | 15 |\n| whyrusleeping | 4 | +448/-177 | 15 |\n| sukun | 9 | +312/-191 | 31 |\n| gammazero | 23 | +239/-216 | 45 |\n| Brian Olson | 5 | +343/-16 | 11 |\n| Steven Allen | 3 | +294/-7 | 9 |\n| Sergey Gorbunov | 2 | +247/-11 | 9 |\n| Kapil Sareen | 1 | +86/-13 | 10 |\n| Masih H. Derkani | 1 | +72/-24 | 1 |\n| Piotr Galar | 1 | +40/-55 | 23 |\n| Rod Vagg | 1 | +13/-11 | 3 |\n| Ankita Sahu | 1 | +2/-0 | 1 |\n| Štefan Baebler | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.38.md", "content": "# Kubo changelog v0.38\n\n\n\nThis release was brought to you by the [Shipyard](https://ipshipyard.com/) team.\n\n- [v0.38.0](#v0380)\n- [v0.38.1](#v0381)\n- [v0.38.2](#v0382)\n\n## v0.38.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [🚀 Repository migration: simplified provide configuration](#-repository-migration-simplified-provide-configuration)\n - [🧹 Experimental Sweeping DHT Provider](#-experimental-sweeping-dht-provider)\n - [📊 Exposed DHT metrics](#-exposed-dht-metrics)\n - [🚨 Improved gateway error pages with diagnostic tools](#-improved-gateway-error-pages-with-diagnostic-tools)\n - [🎨 Updated WebUI](#-updated-webui)\n - [📌 Pin name improvements](#-pin-name-improvements)\n - [🛠️ Identity CID size enforcement and `ipfs files write` fixes](#️-identity-cid-size-enforcement-and-ipfs-files-write-fixes)\n - [📤 Provide Filestore and Urlstore blocks on write](#-provide-filestore-and-urlstore-blocks-on-write)\n - [🚦 MFS operation limit for --flush=false](#-mfs-operation-limit-for---flush=false)\n- [📦️ Important dependency updates](#-important-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\nKubo 0.38.0 simplifies content announcement configuration, introduces an experimental sweeping DHT provider for efficient large-scale operations, and includes various performance improvements.\n\n### 🔦 Highlights\n\n#### 🚀 Repository migration: simplified provide configuration\n\nThis release migrates the repository from version 17 to version 18, simplifying how you configure content announcements.\n\nThe old `Provider` and `Reprovider` sections are now combined into a single [`Provide`](https://github.com/ipfs/kubo/blob/master/docs/config.md#provide) section. Your existing settings are automatically migrated - no manual changes needed.\n\n**Migration happens automatically** when you run `ipfs daemon --migrate`. For manual migration: `ipfs repo migrate --to=18`.\n\nRead more about the new system below.\n\n#### 🧹 Experimental Sweeping DHT Provider\n\nA new experimental DHT provider is available as an alternative to both the default provider and the resource-intensive [accelerated DHT client](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingaccelerateddhtclient). Enable it via [`Provide.DHT.SweepEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedhtsweepenabled).\n\n**How it works:** Instead of providing keys one-by-one, the sweep provider systematically explores DHT keyspace regions in batches.\n\n> \n> \n> \n> \"Reprovide\n> \n>\n> The diagram shows how sweep mode avoids the hourly traffic spikes of Accelerated DHT while maintaining similar effectiveness. By grouping CIDs into keyspace regions and processing them in batches, sweep mode reduces memory overhead and creates predictable network patterns.\n\n**Benefits for large-scale operations:** Handles hundreds of thousands of CIDs with reduced memory and network connections, spreads operations evenly to eliminate resource spikes, maintains state across restarts through persistent keystore, and provides better metrics visibility.\n\n**Monitoring and debugging:** Legacy mode (`SweepEnabled=false`) tracks `provider_reprovider_provide_count` and `provider_reprovider_reprovide_count`, while sweep mode (`SweepEnabled=true`) tracks `total_provide_count_total`. Enable debug logging with `GOLOG_LOG_LEVEL=error,provider=debug,dht/provider=debug` to see detailed logs from either system.\n\n> [!IMPORTANT]\n> The metric `total_provide_count_total` was renamed to `provider_provides_total` in Kubo v0.39 to follow OpenTelemetry naming conventions. If you have dashboards or alerts monitoring this metric, update them accordingly.\n\n> [!NOTE]\n> This feature is experimental and opt-in. In the future, it will become the default and replace the legacy system. Some commands like `ipfs stats provide` and `ipfs routing provide` are not yet available with sweep mode. Run `ipfs provide --help` for alternatives.\n\nFor configuration details, see [`Provide.DHT`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedht). For metrics documentation, see [Provide metrics](https://github.com/ipfs/kubo/blob/master/docs/metrics.md#provide).\n\n#### 📊 Exposed DHT metrics\n\nKubo now exposes DHT metrics from [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht/), including `total_provide_count_total` for sweep provider operations and RPC metrics prefixed with `rpc_inbound_` and `rpc_outbound_` for DHT message traffic. See [Kubo metrics documentation](https://github.com/ipfs/kubo/blob/master/docs/metrics.md) for details.\n\n> [!IMPORTANT]\n> The metric `total_provide_count_total` was renamed to `provider_provides_total` in Kubo v0.39 to follow OpenTelemetry naming conventions. If you have dashboards or alerts monitoring this metric, update them accordingly.\n\n#### 🚨 Improved gateway error pages with diagnostic tools\n\nGateway error pages now provide more actionable information during content retrieval failures. When a 504 Gateway Timeout occurs, users see detailed retrieval state information including which phase failed and a sample of providers that were attempted:\n\n> ![Improved gateway error page showing retrieval diagnostics](https://github.com/user-attachments/assets/18432c74-a5e0-4bbf-9815-7c780779dc98)\n>\n> - **[`Gateway.DiagnosticServiceURL`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaydiagnosticserviceurl)** (default: `https://check.ipfs.network`): Configures the diagnostic service URL. When set, 504 errors show a \"Check CID retrievability\" button that links to this service with `?cid=` for external diagnostics. Set to empty string to disable.\n> - **Enhanced error details**: Timeout errors now display the retrieval phase where failure occurred (e.g., \"connecting to providers\", \"fetching data\") and up to 3 peer IDs that were attempted but couldn't deliver the content, making it easier to diagnose network or provider issues.\n> - **Retry button on all error pages**: Every gateway error page now includes a retry button for quick page refresh without manual URL re-entry.\n\n#### 🎨 Updated WebUI\n\nThe Web UI has been updated to [v4.9](https://github.com/ipfs/ipfs-webui/releases/tag/v4.9.0) with a new **Diagnostics** screen for troubleshooting and system monitoring. Access it at `http://127.0.0.1:5001/webui` when running your local IPFS node.\n\n| Diagnostics: Logs | Files: Check Retrieval | Diagnostics: Retrieval Results |\n|:---:|:---:|:---:|\n| ![Diagnostics logs](https://github.com/user-attachments/assets/a1560fd2-6f4e-4e4f-9506-85ecb10f96e5) | ![Retrieval check interface](https://github.com/user-attachments/assets/6efa8bf1-705e-4256-8c66-282455daf789) | ![Retrieval check results](https://github.com/user-attachments/assets/970f2de3-94a3-4d48-b0a4-46832f73c2e9) |\n| Debug issues in real-time by adjusting [log level](https://github.com/ipfs/kubo/blob/master/docs/environment-variables.md#golog_log_level) without restart (global or per-subsystem like bitswap) | Check if content is available to other peers directly from Files screen | Find out why content won't load or who is providing it to the network |\n\n| Peers: Agent Versions | Files: Custom Sorting |\n|:---:|:---:|\n| ![Peers with Agent Version](https://github.com/user-attachments/assets/4bf95e72-193a-415d-9428-dd222795107a) | ![File sorting options](https://github.com/user-attachments/assets/fd7a1807-c487-4393-ab60-a16ae087e6cd) |\n| Know what software peers run | Find files faster with new sorting |\n\nAdditional improvements include a close button in the file viewer, better error handling, and fixed navigation highlighting.\n\n#### 📌 Pin name improvements\n\n`ipfs pin ls --names` now correctly returns pin names for specific CIDs ([#10649](https://github.com/ipfs/kubo/issues/10649), [boxo#1035](https://github.com/ipfs/boxo/pull/1035)), RPC no longer incorrectly returns names from other pins ([#10966](https://github.com/ipfs/kubo/pull/10966)), and pin names are now limited to 255 bytes for better cross-platform compatibility ([#10981](https://github.com/ipfs/kubo/pull/10981)).\n\n#### 🛠️ Identity CID size enforcement and `ipfs files write` fixes\n\n**Identity CID size limits are now enforced**\n\nThis release enforces a maximum of 128 bytes for identity CIDs ([IPIP-512](https://github.com/ipfs/specs/pull/512)) - attempting to exceed this limit will return a clear error message.\n\nIdentity CIDs use [multihash `0x00`](https://github.com/multiformats/multicodec/blob/master/table.csv#L2) to embed data directly in the CID without hashing. This experimental optimization was designed for tiny data where a CID reference would be larger than the data itself, but without size limits it was easy to misuse and could turn into an anti-pattern that wastes resources and enables abuse.\n\n- `ipfs add --inline-limit` and `--hash=identity` now enforce the 128-byte maximum (error when exceeded)\n- `ipfs files write` prevents creation of oversized identity CIDs\n\n**Multiple `ipfs files write` bugs have been fixed**\n\nThis release resolves several long-standing MFS issues: raw nodes now preserve their codec instead of being forced to dag-pb, append operations on raw nodes work correctly by converting to UnixFS when needed, and identity CIDs properly inherit the full CID prefix from parent directories.\n\n#### 📤 Provide Filestore and Urlstore blocks on write\n\nImprovements to the providing system in the last release (provide blocks according to the configured [Strategy](https://github.com/ipfs/kubo/blob/master/docs/config.md#providestrategy)) left out [Filestore](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-filestore) and [Urlstore](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-urlstore) blocks when the \"all\" strategy was used. They would only be reprovided but not provided on write. This is now fixed, and both Filestore blocks (local file references) and Urlstore blocks (HTTP/HTTPS URL references) will be provided correctly shortly after initial add.\n\n#### 🚦 MFS operation limit for --flush=false\n\nThe new [`Internal.MFSNoFlushLimit`](https://github.com/ipfs/kubo/blob/master/docs/config.md#internalmfsnoflushlimit) configuration option prevents unbounded memory growth when using `--flush=false` with `ipfs files` commands. After performing the configured number of operations without flushing (default: 256), further operations will fail with a clear error message instructing users to flush manually.\n\n### 📦️ Important dependency updates\n\n- update `boxo` to [v0.35.0](https://github.com/ipfs/boxo/releases/tag/v0.35.0)\n- update `go-libp2p-kad-dht` to [v0.35.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.35.0)\n- update `ipfs-webui` to [v4.9.1](https://github.com/ipfs/ipfs-webui/releases/tag/v4.9.1) (incl. [v4.9.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.9.0))\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: v0.38.0\n - chore: bump go-libp2p-kad-dht to v0.35.0 (#11002) ([ipfs/kubo#11002](https://github.com/ipfs/kubo/pull/11002))\n - docs: add sweeping provide worker count recommendation (#11001) ([ipfs/kubo#11001](https://github.com/ipfs/kubo/pull/11001))\n - Upgrade to Boxo v0.35.0 (#10999) ([ipfs/kubo#10999](https://github.com/ipfs/kubo/pull/10999))\n - chore: 0.38.0-rc2\n - chore: update boxo and kad-dht dependencies (#10995) ([ipfs/kubo#10995](https://github.com/ipfs/kubo/pull/10995))\n - fix: update webui to v4.9.1 (#10994) ([ipfs/kubo#10994](https://github.com/ipfs/kubo/pull/10994))\n - fix: provider merge conflicts (#10989) ([ipfs/kubo#10989](https://github.com/ipfs/kubo/pull/10989))\n - fix(mfs): add soft limit for `--flush=false` (#10985) ([ipfs/kubo#10985](https://github.com/ipfs/kubo/pull/10985))\n - fix: provide Filestore nodes (#10990) ([ipfs/kubo#10990](https://github.com/ipfs/kubo/pull/10990))\n - feat: limit pin names to 255 bytes (#10981) ([ipfs/kubo#10981](https://github.com/ipfs/kubo/pull/10981))\n - fix: SweepingProvider slow start (#10980) ([ipfs/kubo#10980](https://github.com/ipfs/kubo/pull/10980))\n - chore: release v0.38.0-rc1\n - fix: SweepingProvider shouldn't error when missing DHT (#10975) ([ipfs/kubo#10975](https://github.com/ipfs/kubo/pull/10975))\n - fix: allow custom http provide when libp2p node is offline (#10974) ([ipfs/kubo#10974](https://github.com/ipfs/kubo/pull/10974))\n - docs(provide): validation and reprovide cycle visualization (#10977) ([ipfs/kubo#10977](https://github.com/ipfs/kubo/pull/10977))\n - refactor(ci): optimize build workflows (#10973) ([ipfs/kubo#10973](https://github.com/ipfs/kubo/pull/10973))\n - fix(cmds): cleanup unicode identify strings (#9465) ([ipfs/kubo#9465](https://github.com/ipfs/kubo/pull/9465))\n - feat: ipfs-webui v4.9.0 with retrieval diagnostics (#10969) ([ipfs/kubo#10969](https://github.com/ipfs/kubo/pull/10969))\n - fix(mfs): unbound cache growth with `flush=false` (#10971) ([ipfs/kubo#10971](https://github.com/ipfs/kubo/pull/10971))\n - fix: `ipfs pin ls --names` (#10970) ([ipfs/kubo#10970](https://github.com/ipfs/kubo/pull/10970))\n - refactor(config): migration 17-to-18 to unify Provider/Reprovider into Provide.DHT (#10951) ([ipfs/kubo#10951](https://github.com/ipfs/kubo/pull/10951))\n - feat: opt-in new Sweep provide system (#10834) ([ipfs/kubo#10834](https://github.com/ipfs/kubo/pull/10834))\n - rpc: retrieve pin names when Detailed option provided (#10966) ([ipfs/kubo#10966](https://github.com/ipfs/kubo/pull/10966))\n - fix: enforce identity CID size limits (#10949) ([ipfs/kubo#10949](https://github.com/ipfs/kubo/pull/10949))\n - docs: kubo logo sources (#10964) ([ipfs/kubo#10964](https://github.com/ipfs/kubo/pull/10964))\n - feat(config): validate Import config at daemon startup (#10957) ([ipfs/kubo#10957](https://github.com/ipfs/kubo/pull/10957))\n - fix(telemetry): improve vm/container detection (#10944) ([ipfs/kubo#10944](https://github.com/ipfs/kubo/pull/10944))\n - feat(gateway): improved error page with retrieval state details (#10950) ([ipfs/kubo#10950](https://github.com/ipfs/kubo/pull/10950))\n - close files opened during migration (#10956) ([ipfs/kubo#10956](https://github.com/ipfs/kubo/pull/10956))\n - fix ctrl-c prompt during run migrations prompt (#10947) ([ipfs/kubo#10947](https://github.com/ipfs/kubo/pull/10947))\n - repo: use config api to get node root path (#10934) ([ipfs/kubo#10934](https://github.com/ipfs/kubo/pull/10934))\n - docs: simplify release process (#10870) ([ipfs/kubo#10870](https://github.com/ipfs/kubo/pull/10870))\n - Merge release v0.37.0 ([ipfs/kubo#10943](https://github.com/ipfs/kubo/pull/10943))\n - feat(ci): docker linting (#10927) ([ipfs/kubo#10927](https://github.com/ipfs/kubo/pull/10927))\n - fix: disable telemetry in test profile (#10931) ([ipfs/kubo#10931](https://github.com/ipfs/kubo/pull/10931))\n - fix: harness tests random panic (#10933) ([ipfs/kubo#10933](https://github.com/ipfs/kubo/pull/10933))\n - chore: 0.38.0-dev\n- github.com/ipfs/boxo (v0.34.0 -> v0.35.0):\n - Release v0.35.0 ([ipfs/boxo#1046](https://github.com/ipfs/boxo/pull/1046))\n - feat(gateway): add `MaxRangeRequestFileSize` protection (#1043) ([ipfs/boxo#1043](https://github.com/ipfs/boxo/pull/1043))\n - revert: remove MFS auto-flush mechanism (#1041) ([ipfs/boxo#1041](https://github.com/ipfs/boxo/pull/1041))\n - Filestore: add Provider option to provide filestore blocks. (#1042) ([ipfs/boxo#1042](https://github.com/ipfs/boxo/pull/1042))\n - fix(pinner): restore indirect pin detection and add context cancellation (#1039) ([ipfs/boxo#1039](https://github.com/ipfs/boxo/pull/1039))\n - fix(mfs): limit cache growth by default (#1037) ([ipfs/boxo#1037](https://github.com/ipfs/boxo/pull/1037))\n - update dependencies (#1038) ([ipfs/boxo#1038](https://github.com/ipfs/boxo/pull/1038))\n - feat(pinner): add `CheckIfPinnedWithType` for efficient checks with names (#1035) ([ipfs/boxo#1035](https://github.com/ipfs/boxo/pull/1035))\n - fix(routing/http): don't cancel batch prematurely (#1036) ([ipfs/boxo#1036](https://github.com/ipfs/boxo/pull/1036))\n - refactor: use the new Reprovide Sweep interface (#995) ([ipfs/boxo#995](https://github.com/ipfs/boxo/pull/995))\n - Update go-dsqueue to latest (#1034) ([ipfs/boxo#1034](https://github.com/ipfs/boxo/pull/1034))\n - feat(routing/http): return 200 for empty results per IPIP-513 (#1032) ([ipfs/boxo#1032](https://github.com/ipfs/boxo/pull/1032))\n - replace provider queue with go-dsqueue (#1033) ([ipfs/boxo#1033](https://github.com/ipfs/boxo/pull/1033))\n - refactor: use slices package to simplify slice manipulation (#1031) ([ipfs/boxo#1031](https://github.com/ipfs/boxo/pull/1031))\n - bitswap/network: fix read/write data race in bitswap network test (#1030) ([ipfs/boxo#1030](https://github.com/ipfs/boxo/pull/1030))\n - fix(verifcid): enforce size limit for identity CIDs (#1018) ([ipfs/boxo#1018](https://github.com/ipfs/boxo/pull/1018))\n - docs: boxo logo source files (#1028) ([ipfs/boxo#1028](https://github.com/ipfs/boxo/pull/1028))\n - feat(gateway): enhance 504 timeout errors with diagnostic UX (#1023) ([ipfs/boxo#1023](https://github.com/ipfs/boxo/pull/1023))\n - Use `time.Duration` for rebroadcast delay (#1027) ([ipfs/boxo#1027](https://github.com/ipfs/boxo/pull/1027))\n - refactor(bitswap/client/internal): close session with Close method instead of context (#1011) ([ipfs/boxo#1011](https://github.com/ipfs/boxo/pull/1011))\n - fix: use %q for logging routing keys with binary data (#1025) ([ipfs/boxo#1025](https://github.com/ipfs/boxo/pull/1025))\n - rename `retrieval.RetrievalState` to `retrieval.State` (#1026) ([ipfs/boxo#1026](https://github.com/ipfs/boxo/pull/1026))\n - feat(gateway): add retrieval state tracking for timeout diagnostics (#1015) ([ipfs/boxo#1015](https://github.com/ipfs/boxo/pull/1015))\n - Nonfunctional changes (#1017) ([ipfs/boxo#1017](https://github.com/ipfs/boxo/pull/1017))\n - fix: flaky TestCancelOverridesPendingWants (#1016) ([ipfs/boxo#1016](https://github.com/ipfs/boxo/pull/1016))\n - bitswap/client: GetBlocks cancels session when finished (#1007) ([ipfs/boxo#1007](https://github.com/ipfs/boxo/pull/1007))\n - Remove unused context ([ipfs/boxo#1006](https://github.com/ipfs/boxo/pull/1006))\n- github.com/ipfs/go-block-format (v0.2.2 -> v0.2.3):\n - new version (#66) ([ipfs/go-block-format#66](https://github.com/ipfs/go-block-format/pull/66))\n - Replace CI badge and add GoDoc link in README (#65) ([ipfs/go-block-format#65](https://github.com/ipfs/go-block-format/pull/65))\n- github.com/ipfs/go-datastore (v0.8.3 -> v0.9.0):\n - new version (#255) ([ipfs/go-datastore#255](https://github.com/ipfs/go-datastore/pull/255))\n - feat(keytransform): support transaction feature (#239) ([ipfs/go-datastore#239](https://github.com/ipfs/go-datastore/pull/239))\n - feat: context datastore (#238) ([ipfs/go-datastore#238](https://github.com/ipfs/go-datastore/pull/238))\n - new version (#254) ([ipfs/go-datastore#254](https://github.com/ipfs/go-datastore/pull/254))\n - fix comment (#253) ([ipfs/go-datastore#253](https://github.com/ipfs/go-datastore/pull/253))\n - feat: query iterator (#244) ([ipfs/go-datastore#244](https://github.com/ipfs/go-datastore/pull/244))\n - Update readme links (#246) ([ipfs/go-datastore#246](https://github.com/ipfs/go-datastore/pull/246))\n- github.com/ipfs/go-ipld-format (v0.6.2 -> v0.6.3):\n - new version (#100) ([ipfs/go-ipld-format#100](https://github.com/ipfs/go-ipld-format/pull/100))\n - avoid unnecessary slice allocation (#99) ([ipfs/go-ipld-format#99](https://github.com/ipfs/go-ipld-format/pull/99))\n- github.com/ipfs/go-unixfsnode (v1.10.1 -> v1.10.2):\n - new version ([ipfs/go-unixfsnode#88](https://github.com/ipfs/go-unixfsnode/pull/88))\n- github.com/ipld/go-car/v2 (v2.14.3 -> v2.15.0):\n - v2.15.0 bump (#606) ([ipld/go-car#606](https://github.com/ipld/go-car/pull/606))\n - feat: add NextReader to BlockReader (#603) ([ipld/go-car#603](https://github.com/ipld/go-car/pull/603))\n - Remove `@masih` form CODEOWNERS ([ipld/go-car#605](https://github.com/ipld/go-car/pull/605))\n- github.com/libp2p/go-libp2p-kad-dht (v0.34.0 -> v0.35.0):\n - chore: release v0.35.0 (#1162) ([libp2p/go-libp2p-kad-dht#1162](https://github.com/libp2p/go-libp2p-kad-dht/pull/1162))\n - refactor: adjust FIND_NODE response exceptions (#1158) ([libp2p/go-libp2p-kad-dht#1158](https://github.com/libp2p/go-libp2p-kad-dht/pull/1158))\n - refactor: remove provider status command (#1157) ([libp2p/go-libp2p-kad-dht#1157](https://github.com/libp2p/go-libp2p-kad-dht/pull/1157))\n - refactor(provider): closestPeerToPrefix coverage trie (#1156) ([libp2p/go-libp2p-kad-dht#1156](https://github.com/libp2p/go-libp2p-kad-dht/pull/1156))\n - fix: don't empty mapdatastore keystore on close (#1155) ([libp2p/go-libp2p-kad-dht#1155](https://github.com/libp2p/go-libp2p-kad-dht/pull/1155))\n - provider: default options (#1153) ([libp2p/go-libp2p-kad-dht#1153](https://github.com/libp2p/go-libp2p-kad-dht/pull/1153))\n - fix(keystore): use new batch after commit (#1154) ([libp2p/go-libp2p-kad-dht#1154](https://github.com/libp2p/go-libp2p-kad-dht/pull/1154))\n - provider: more minor fixes (#1152) ([libp2p/go-libp2p-kad-dht#1152](https://github.com/libp2p/go-libp2p-kad-dht/pull/1152))\n - rename KeyStore -> Keystore (#1151) ([libp2p/go-libp2p-kad-dht#1151](https://github.com/libp2p/go-libp2p-kad-dht/pull/1151))\n - provider: minor fixes (#1150) ([libp2p/go-libp2p-kad-dht#1150](https://github.com/libp2p/go-libp2p-kad-dht/pull/1150))\n - buffered provider (#1149) ([libp2p/go-libp2p-kad-dht#1149](https://github.com/libp2p/go-libp2p-kad-dht/pull/1149))\n - keystore: remove mutex (#1147) ([libp2p/go-libp2p-kad-dht#1147](https://github.com/libp2p/go-libp2p-kad-dht/pull/1147))\n - provider: ResettableKeyStore (#1146) ([libp2p/go-libp2p-kad-dht#1146](https://github.com/libp2p/go-libp2p-kad-dht/pull/1146))\n - keystore: revamp (#1142) ([libp2p/go-libp2p-kad-dht#1142](https://github.com/libp2p/go-libp2p-kad-dht/pull/1142))\n - provider: use synctest for testing time (#1136) ([libp2p/go-libp2p-kad-dht#1136](https://github.com/libp2p/go-libp2p-kad-dht/pull/1136))\n - provider: connectivity state machine (#1135) ([libp2p/go-libp2p-kad-dht#1135](https://github.com/libp2p/go-libp2p-kad-dht/pull/1135))\n - provider: minor fixes (#1133) ([libp2p/go-libp2p-kad-dht#1133](https://github.com/libp2p/go-libp2p-kad-dht/pull/1133))\n - dual: provider (#1132) ([libp2p/go-libp2p-kad-dht#1132](https://github.com/libp2p/go-libp2p-kad-dht/pull/1132))\n - provider: refresh schedule (#1131) ([libp2p/go-libp2p-kad-dht#1131](https://github.com/libp2p/go-libp2p-kad-dht/pull/1131))\n - provider: integration tests (#1127) ([libp2p/go-libp2p-kad-dht#1127](https://github.com/libp2p/go-libp2p-kad-dht/pull/1127))\n - provider: daemon (#1126) ([libp2p/go-libp2p-kad-dht#1126](https://github.com/libp2p/go-libp2p-kad-dht/pull/1126))\n - provide: handle reprovide (#1125) ([libp2p/go-libp2p-kad-dht#1125](https://github.com/libp2p/go-libp2p-kad-dht/pull/1125))\n - provider: options (#1124) ([libp2p/go-libp2p-kad-dht#1124](https://github.com/libp2p/go-libp2p-kad-dht/pull/1124))\n - provider: catchup pending work (#1123) ([libp2p/go-libp2p-kad-dht#1123](https://github.com/libp2p/go-libp2p-kad-dht/pull/1123))\n - provider: batch reprovide (#1122) ([libp2p/go-libp2p-kad-dht#1122](https://github.com/libp2p/go-libp2p-kad-dht/pull/1122))\n - provider: batch provide (#1121) ([libp2p/go-libp2p-kad-dht#1121](https://github.com/libp2p/go-libp2p-kad-dht/pull/1121))\n - provider: swarm exploration (#1120) ([libp2p/go-libp2p-kad-dht#1120](https://github.com/libp2p/go-libp2p-kad-dht/pull/1120))\n - provider: handleProvide (#1118) ([libp2p/go-libp2p-kad-dht#1118](https://github.com/libp2p/go-libp2p-kad-dht/pull/1118))\n - provider: schedule (#1117) ([libp2p/go-libp2p-kad-dht#1117](https://github.com/libp2p/go-libp2p-kad-dht/pull/1117))\n - provider: schedule prefix length (#1116) ([libp2p/go-libp2p-kad-dht#1116](https://github.com/libp2p/go-libp2p-kad-dht/pull/1116))\n - provider: ProvideStatus interface (#1110) ([libp2p/go-libp2p-kad-dht#1110](https://github.com/libp2p/go-libp2p-kad-dht/pull/1110))\n - provider: network operations (#1115) ([libp2p/go-libp2p-kad-dht#1115](https://github.com/libp2p/go-libp2p-kad-dht/pull/1115))\n - provider: adding provide and reprovide queue (#1114) ([libp2p/go-libp2p-kad-dht#1114](https://github.com/libp2p/go-libp2p-kad-dht/pull/1114))\n - provider: trie allocation helper (#1108) ([libp2p/go-libp2p-kad-dht#1108](https://github.com/libp2p/go-libp2p-kad-dht/pull/1108))\n - add missing ShortestCoveredPrefix ([libp2p/go-libp2p-kad-dht@d0b110d](https://github.com/libp2p/go-libp2p-kad-dht/commit/d0b110d))\n - provider: keyspace helpers ([libp2p/go-libp2p-kad-dht@af3ce09](https://github.com/libp2p/go-libp2p-kad-dht/commit/af3ce09))\n - provider: helpers package rename (#1111) ([libp2p/go-libp2p-kad-dht#1111](https://github.com/libp2p/go-libp2p-kad-dht/pull/1111))\n - provider: trie region helpers (#1109) ([libp2p/go-libp2p-kad-dht#1109](https://github.com/libp2p/go-libp2p-kad-dht/pull/1109))\n - provider: PruneSubtrie helper (#1107) ([libp2p/go-libp2p-kad-dht#1107](https://github.com/libp2p/go-libp2p-kad-dht/pull/1107))\n - provider: NextNonEmptyLeaf trie helper (#1106) ([libp2p/go-libp2p-kad-dht#1106](https://github.com/libp2p/go-libp2p-kad-dht/pull/1106))\n - provider: find subtrie helper (#1105) ([libp2p/go-libp2p-kad-dht#1105](https://github.com/libp2p/go-libp2p-kad-dht/pull/1105))\n - provider: helpers trie find prefix (#1104) ([libp2p/go-libp2p-kad-dht#1104](https://github.com/libp2p/go-libp2p-kad-dht/pull/1104))\n - provider: trie items listing helpers (#1103) ([libp2p/go-libp2p-kad-dht#1103](https://github.com/libp2p/go-libp2p-kad-dht/pull/1103))\n - provider: add ShortestCoveredPrefix helper (#1102) ([libp2p/go-libp2p-kad-dht#1102](https://github.com/libp2p/go-libp2p-kad-dht/pull/1102))\n - provider: key helpers (#1101) ([libp2p/go-libp2p-kad-dht#1101](https://github.com/libp2p/go-libp2p-kad-dht/pull/1101))\n - provider: Connectivity Checker (#1099) ([libp2p/go-libp2p-kad-dht#1099](https://github.com/libp2p/go-libp2p-kad-dht/pull/1099))\n - provider: SweepingProvider interface (#1098) ([libp2p/go-libp2p-kad-dht#1098](https://github.com/libp2p/go-libp2p-kad-dht/pull/1098))\n - provider: keystore (#1096) ([libp2p/go-libp2p-kad-dht#1096](https://github.com/libp2p/go-libp2p-kad-dht/pull/1096))\n - provider initial commit ([libp2p/go-libp2p-kad-dht@70d21a8](https://github.com/libp2p/go-libp2p-kad-dht/commit/70d21a8))\n - test GCP result order (#1097) ([libp2p/go-libp2p-kad-dht#1097](https://github.com/libp2p/go-libp2p-kad-dht/pull/1097))\n - refactor: apply suggestions in records (#1113) ([libp2p/go-libp2p-kad-dht#1113](https://github.com/libp2p/go-libp2p-kad-dht/pull/1113))\n- github.com/libp2p/go-libp2p-kbucket (v0.7.0 -> v0.8.0):\n - chore: release v0.8.0 (#147) ([libp2p/go-libp2p-kbucket#147](https://github.com/libp2p/go-libp2p-kbucket/pull/147))\n - feat: generic find PeerID with CPL (#145) ([libp2p/go-libp2p-kbucket#145](https://github.com/libp2p/go-libp2p-kbucket/pull/145))\n- github.com/multiformats/go-varint (v0.0.7 -> v0.1.0):\n - v0.1.0 bump (#29) ([multiformats/go-varint#29](https://github.com/multiformats/go-varint/pull/29))\n - chore: optimise UvarintSize (#28) ([multiformats/go-varint#28](https://github.com/multiformats/go-varint/pull/28))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Guillaume Michel | 62 | +15401/-5657 | 209 |\n| Marcin Rataj | 33 | +9540/-1734 | 215 |\n| Andrew Gillis | 29 | +771/-1093 | 70 |\n| Hlib Kanunnikov | 2 | +350/-0 | 5 |\n| Rod Vagg | 3 | +260/-9 | 4 |\n| Hector Sanjuan | 4 | +188/-33 | 11 |\n| Jakub Sztandera | 1 | +67/-15 | 3 |\n| Masih H. Derkani | 1 | +1/-2 | 2 |\n| Dominic Della Valle | 1 | +2/-1 | 1 |\n\n## v0.38.1\n\nFixes migration panic on Windows when upgrading from v0.37 to v0.38 (\"panic: error can't be dealt with transactionally: Access is denied\").\n\nUpdates go-ds-pebble to v0.5.3 (pebble v2.1.0).\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: v0.38.1\n - fix: migrations for Windows (#11010) ([ipfs/kubo#11010](https://github.com/ipfs/kubo/pull/11010))\n - Upgrade go-ds-pebble to v0.5.3 (#11011) ([ipfs/kubo#11011](https://github.com/ipfs/kubo/pull/11011))\n - upgrade go-ds-pebble to v0.5.2 (#11000) ([ipfs/kubo#11000](https://github.com/ipfs/kubo/pull/11000))\n- github.com/ipfs/go-ds-pebble (v0.5.1 -> v0.5.3):\n - new version (#62) ([ipfs/go-ds-pebble#62](https://github.com/ipfs/go-ds-pebble/pull/62))\n - fix panic when batch is reused after commit (#61) ([ipfs/go-ds-pebble#61](https://github.com/ipfs/go-ds-pebble/pull/61))\n - new version (#60) ([ipfs/go-ds-pebble#60](https://github.com/ipfs/go-ds-pebble/pull/60))\n - Upgrade to pebble v2.1.0 (#59) ([ipfs/go-ds-pebble#59](https://github.com/ipfs/go-ds-pebble/pull/59))\n - update readme (#57) ([ipfs/go-ds-pebble#57](https://github.com/ipfs/go-ds-pebble/pull/57))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marcin Rataj | 2 | +613/-267 | 15 |\n| Andrew Gillis | 6 | +148/-22 | 8 |\n\n## v0.38.2\n\n- Updates [boxo v0.35.1](https://github.com/ipfs/boxo/releases/tag/v0.35.1) with bitswap and HTTP retrieval fixes:\n - Fixed bitswap trace context not being passed to sessions, restoring observability for monitoring tools\n - Kubo now fetches from HTTP gateways that return errors in legacy IPLD format, improving compatibility with older providers\n - Better handling of rate-limited HTTP endpoints and clearer timeout error messages\n- Updates [go-libp2p-kad-dht v0.35.1](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.35.1) with memory optimizations for nodes using `Provide.DHT.SweepEnabled=true`\n- Updates [quic-go v0.55.0](https://github.com/quic-go/quic-go/releases/tag/v0.55.0) to fix memory pooling where stream frames weren't returned to the pool on cancellation\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - chore: boxo and kad-dht updates\n - fix: update quic-go to v0.55.0\n- github.com/ipfs/boxo (v0.35.0 -> v0.35.1):\n - Release v0.35.1 ([ipfs/boxo#1063](https://github.com/ipfs/boxo/pull/1063))\n - bitswap/httpnet: improve \"Connect\"/testCid check (#1057) ([ipfs/boxo#1057](https://github.com/ipfs/boxo/pull/1057))\n - fix: revert go-libp2p to v0.43.0 (#1061) ([ipfs/boxo#1061](https://github.com/ipfs/boxo/pull/1061))\n - bitswap/client: propagate trace state when calling `GetBlocks` ([ipfs/boxo#1060](https://github.com/ipfs/boxo/pull/1060))\n - fix(tracing): use context to pass trace and retrieval state to session ([ipfs/boxo#1059](https://github.com/ipfs/boxo/pull/1059))\n - bitswap: link traces ([ipfs/boxo#1053](https://github.com/ipfs/boxo/pull/1053))\n - fix(gateway): deduplicate peer IDs in retrieval diagnostics (#1058) ([ipfs/boxo#1058](https://github.com/ipfs/boxo/pull/1058))\n - update go-dsqueue to v0.1.0 ([ipfs/boxo#1049](https://github.com/ipfs/boxo/pull/1049))\n - Update go-libp2p to v0.44 ([ipfs/boxo#1048](https://github.com/ipfs/boxo/pull/1048))\n- github.com/ipfs/go-dsqueue (v0.0.5 -> v0.1.0):\n - new version (#24) ([ipfs/go-dsqueue#24](https://github.com/ipfs/go-dsqueue/pull/24))\n - Do not reuse datastore Batch (#23) ([ipfs/go-dsqueue#23](https://github.com/ipfs/go-dsqueue/pull/23))\n- github.com/ipfs/go-log/v2 (v2.8.1 -> v2.8.2):\n - new version (#175) ([ipfs/go-log#175](https://github.com/ipfs/go-log/pull/175))\n - fix: revert removal of LevelFromString to avoid breaking change (#174) ([ipfs/go-log#174](https://github.com/ipfs/go-log/pull/174))\n- github.com/ipld/go-car/v2 (v2.15.0 -> v2.16.0):\n - v2.16.0 bump (#625) ([ipld/go-car#625](https://github.com/ipld/go-car/pull/625))\n- github.com/ipld/go-ipld-prime/storage/bsadapter (v0.0.0-20230102063945-1a409dc236dd -> v0.0.0-20250821084354-a425e60cd714):\n- github.com/libp2p/go-libp2p-kad-dht (v0.35.0 -> v0.35.1):\n - chore: release v0.35.1 (#1165) ([libp2p/go-libp2p-kad-dht#1165](https://github.com/libp2p/go-libp2p-kad-dht/pull/1165))\n - feat(provider): use Trie.AddMany (#1164) ([libp2p/go-libp2p-kad-dht#1164](https://github.com/libp2p/go-libp2p-kad-dht/pull/1164))\n - fix(provider): memory usage (#1163) ([libp2p/go-libp2p-kad-dht#1163](https://github.com/libp2p/go-libp2p-kad-dht/pull/1163))\n- github.com/libp2p/go-netroute (v0.2.2 -> v0.3.0):\n - release v0.3.0\n - remove google/gopacket dependency\n - Query routes via routesocket ([libp2p/go-netroute#57](https://github.com/libp2p/go-netroute/pull/57))\n - ci: uci/update-go (#52) ([libp2p/go-netroute#52](https://github.com/libp2p/go-netroute/pull/52))\n- github.com/multiformats/go-multicodec (v0.9.2 -> v0.10.0):\n - chore: v0.10.0 bump\n - chore: update submodules and go generate\n - chore(deps): update stringer to v0.38.0\n - ci: uci/update-go ([multiformats/go-multicodec#104](https://github.com/multiformats/go-multicodec/pull/104))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| rvagg | 1 | +537/-481 | 3 |\n| Carlos Hernandez | 9 | +556/-218 | 11 |\n| Guillaume Michel | 3 | +139/-105 | 6 |\n| gammazero | 8 | +101/-97 | 14 |\n| Hector Sanjuan | 1 | +87/-28 | 5 |\n| Marcin Rataj | 4 | +57/-9 | 7 |\n| Marco Munizaga | 2 | +42/-14 | 7 |\n| Dennis Trautwein | 2 | +19/-7 | 7 |\n| Andrew Gillis | 3 | +3/-19 | 3 |\n| Rod Vagg | 4 | +12/-3 | 4 |\n| web3-bot | 1 | +2/-1 | 1 |\n| galargh | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.39.md", "content": "# Kubo changelog v0.39\n\n\n\nThis release was brought to you by the [Shipyard](https://ipshipyard.com/) team.\n\n- [v0.39.0](#v0390)\n\n## v0.39.0\n\n[](https://github.com/user-attachments/assets/427702e8-b6b8-4ac2-8425-18069626c321)\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [🎯 DHT Sweep provider is now the default](#-dht-sweep-provider-is-now-the-default)\n - [⚡ Fast root CID providing for immediate content discovery](#-fast-root-cid-providing-for-immediate-content-discovery)\n - [⏯️ Provider state persists across restarts](#️-provider-state-persists-across-restarts)\n - [📊 Detailed statistics with `ipfs provide stat`](#-detailed-statistics-with-ipfs-provide-stat)\n - [🔔 Slow reprovide warnings](#-slow-reprovide-warnings)\n - [📊 Metric rename: `provider_provides_total`](#-metric-rename-provider_provides_total)\n - [🔧 Automatic UPnP recovery after router restarts](#-automatic-upnp-recovery-after-router-restarts)\n - [🪦 Deprecated `go-ipfs` name no longer published](#-deprecated-go-ipfs-name-no-longer-published)\n - [🚦 Gateway range request limits for CDN compatibility](#-gateway-range-request-limits-for-cdn-compatibility)\n - [🖥️ RISC-V support with prebuilt binaries](#️-risc-v-support-with-prebuilt-binaries)\n- [📦️ Important dependency updates](#-important-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\nKubo 0.39 makes self-hosting practical on consumer hardware and home networks. The DHT sweep provider (now default) announces your content to the network without traffic spikes that overwhelm residential connections. Automatic UPnP recovery means your node stays reachable after router restarts without manual intervention.\n\nNew content becomes findable immediately after `ipfs add`. The provider system persists state across restarts, alerts you when falling behind, and exposes detailed stats for monitoring. This release also finalizes the deprecation of the legacy `go-ipfs` name.\n\n### 🔦 Highlights\n\n#### 🎯 DHT Sweep provider is now the default\n\nThe Amino DHT Sweep provider system, introduced as experimental in v0.38, is now enabled by default (`Provide.DHT.SweepEnabled=true`).\n\n**What this means:** All nodes now benefit from efficient keyspace-sweeping content announcements that reduce memory overhead and create predictable network patterns, especially for nodes providing large content collections.\n\n**Migration:** The transition is automatic on upgrade. Your existing configuration is preserved:\n\n- If you explicitly set `Provide.DHT.SweepEnabled=false` in v0.38, you'll continue using the legacy provider\n- If you were using the default settings, you'll automatically get the sweep provider\n- To opt out and return to legacy behavior: `ipfs config --json Provide.DHT.SweepEnabled false`\n- Providers with medium to large datasets may need to adjust defaults; see [Capacity Planning](https://github.com/ipfs/kubo/blob/master/docs/provide-stats.md#capacity-planning)\n- When `Routing.AcceleratedDHTClient` is enabled, full sweep efficiency may not be available yet; consider disabling the accelerated client as sweep is sufficient for most workloads. See [caveat 4](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingaccelerateddhtclient).\n\n**New features available with sweep mode:**\n\n- Detailed statistics via `ipfs provide stat` ([see below](#-detailed-statistics-with-ipfs-provide-stat))\n- Automatic resume after restarts with persistent state ([see below](#️-provider-state-persists-across-restarts))\n- Proactive alerts when reproviding falls behind ([see below](#-slow-reprovide-warnings))\n- Better metrics for monitoring (`provider_provides_total`) ([see below](#-metric-rename-provider_provides_total))\n- Fast optimistic provide of new root CIDs ([see below](#-fast-root-cid-providing-for-immediate-content-discovery))\n\nFor background on the sweep provider design and motivations, see [`Provide.DHT.SweepEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedhtsweepenabled) and Shipyard's blogpost [Provide Sweep: Solving the DHT Provide Bottleneck](https://ipshipyard.com/blog/2025-dht-provide-sweep/).\n\n#### ⚡ Fast root CID providing for immediate content discovery\n\nWhen you add content to IPFS, the sweep provider queues it for efficient DHT provides over time. While this is resource-efficient, other peers won't find your content immediately after `ipfs add` or `ipfs dag import` completes.\n\nTo make sharing faster, `ipfs add` and `ipfs dag import` now do an immediate provide of root CIDs to the DHT in addition to the regular queue (controlled by the new `--fast-provide-root` flag, enabled by default). This complements the sweep provider system: fast-provide handles the urgent case (root CIDs that users share and reference), while the sweep provider efficiently provides all blocks according to `Provide.Strategy` over time.\n\nThis closes the gap between command completion and content shareability: root CIDs typically become discoverable on the network in under a second (compared to 30+ seconds previously). The feature uses optimistic DHT operations, which are significantly faster with the sweep provider (now enabled by default).\n\nBy default, this immediate provide runs in the background without blocking the command. For use cases requiring guaranteed discoverability before the command returns (e.g., sharing a link immediately), use `--fast-provide-wait` to block until the provide completes.\n\n**Simple examples:**\n\n```bash\nipfs add file.txt # Root provided immediately, blocks queued for sweep provider\nipfs add file.txt --fast-provide-wait # Wait for root provide to complete\nipfs dag import file.car # Same for CAR imports\n```\n\n**Configuration:** Set defaults via `Import.FastProvideRoot` (default: `true`) and `Import.FastProvideWait` (default: `false`). See `ipfs add --help` and `ipfs dag import --help` for more details and examples.\n\nFast root CID provide is automatically skipped when DHT routing is unavailable (e.g., `Routing.Type=none` or delegated-only configurations).\n\n#### ⏯️ Provider state persists across restarts\n\nThe Sweep provider now persists the reprovide cycle state and automatically resumes where it left off after a restart. This brings several improvements:\n\n- **Persistent progress**: The provider saves its position in the reprovide cycle to the datastore. On restart, it continues from where it stopped instead of starting from scratch.\n- **Catch-up reproviding**: If the node was offline for an extended period, all CIDs that haven't been reprovided within the configured reprovide interval are immediately queued for reproviding when the node starts up. This ensures content availability is maintained even after downtime.\n- **Persistent provide queue**: The provide queue is persisted to the datastore on shutdown. When the node restarts, queued CIDs are restored and provided as expected, preventing loss of pending provide operations.\n- **Resume control**: The resume behavior is controlled via [`Provide.DHT.ResumeEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedhtresumeenabled) (default: `true`). Set to `false` if you don't want to keep the persisted provider state from a previous run.\n\nThis feature improves reliability for nodes that experience intermittent connectivity or restarts.\n\n#### 📊 Detailed statistics with `ipfs provide stat`\n\nThe Sweep provider system now exposes detailed statistics through `ipfs provide stat`, helping you monitor provider health and troubleshoot issues.\n\nRun `ipfs provide stat` for a quick summary, or use `--all` to see complete metrics including connectivity status, queue sizes, reprovide schedules, network statistics, operation rates, and worker utilization. For real-time monitoring, use `watch ipfs provide stat --all --compact` to observe changes in a 2-column layout. Individual sections can be displayed with flags like `--network`, `--operations`, or `--workers`.\n\nFor Dual DHT configurations, use `--lan` to view LAN DHT statistics instead of the default WAN DHT stats.\n\nFor more information, run `ipfs provide stat --help` or see the [Provide Stats documentation](https://github.com/ipfs/kubo/blob/master/docs/provide-stats.md), including [Capacity Planning](https://github.com/ipfs/kubo/blob/master/docs/provide-stats.md#capacity-planning).\n\n> [!NOTE]\n> Legacy provider (when `Provide.DHT.SweepEnabled=false`) shows basic statistics without flag support.\n\n#### 🔔 Slow reprovide warnings\n\nKubo now monitors DHT reprovide operations when `Provide.DHT.SweepEnabled=true`\nand alerts you if your node is falling behind on reprovides.\n\nWhen the reprovide queue consistently grows and all periodic workers are busy,\na warning displays with:\n\n- Queue size and worker utilization details\n- Recommended solutions: increase `Provide.DHT.MaxWorkers` or `Provide.DHT.DedicatedPeriodicWorkers`\n- Command to monitor real-time progress: `watch ipfs provide stat --all --compact`\n\nThe alert polls every 15 minutes (to avoid alert fatigue while catching\npersistent issues) and only triggers after sustained growth across multiple\nintervals. The legacy provider is unaffected by this change.\n\n#### 📊 Metric rename: `provider_provides_total`\n\nThe Amino DHT Sweep provider metric has been renamed from `total_provide_count_total` to `provider_provides_total` to follow OpenTelemetry naming conventions and maintain consistency with other kad-dht metrics (which use dot notation like `rpc.inbound.messages`, `rpc.outbound.requests`, etc.).\n\n**Migration:** If you have Prometheus queries, dashboards, or alerts monitoring the old `total_provide_count_total` metric, update them to use `provider_provides_total` instead. This affects all nodes using sweep mode, which is now the default in v0.39 (previously opt-in experimental in v0.38).\n\n#### 🔧 Automatic UPnP recovery after router restarts\n\nKubo now automatically recovers UPnP port mappings when routers restart or\nbecome temporarily unavailable, fixing a critical connectivity issue that\naffected self-hosted nodes behind NAT.\n\n**Previous behavior:** When a UPnP-enabled router restarted, Kubo would lose\nits port mapping and fail to re-establish it automatically. Nodes would become\nunreachable to the network until the daemon was manually restarted, forcing\nreliance on relay connections which degraded performance.\n\n**New behavior:** The upgraded go-libp2p (v0.44.0) includes [Shipyard's fix](https://github.com/libp2p/go-libp2p/pull/3367)\nfor self-healing NAT mappings that automatically rediscover and re-establish\nport forwarding after router events. Nodes now maintain public connectivity\nwithout manual intervention.\n\n> [!NOTE]\n> If your node runs behind a router and you haven't manually configured port\n> forwarding, make sure [`Swarm.DisableNatPortMap=false`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmdisablenatportmap)\n> so UPnP can automatically handle port mapping (this is the default).\n\nThis significantly improves reliability for desktop and self-hosted IPFS nodes\nusing UPnP for NAT traversal.\n\n#### 🪦 Deprecated `go-ipfs` name no longer published\n\nThe `go-ipfs` name was deprecated in 2022 and renamed to `kubo`. Starting with this release, the legacy Docker image name has been replaced with a stub that displays an error message directing users to switch to `ipfs/kubo`.\n\n**Docker images:** The `ipfs/go-ipfs` image tags now contain only a stub script that exits with an error, instructing users to update their Docker configurations to use [`ipfs/kubo`](https://hub.docker.com/r/ipfs/kubo) instead. This ensures users are aware of the deprecation while allowing existing automation to fail explicitly rather than silently using outdated images.\n\n**Distribution binaries:** Download Kubo from or . The legacy `go-ipfs` distribution path should no longer be used.\n\nAll users should migrate to the `kubo` name in their scripts and configurations.\n\n#### 🚦 Gateway range request limits for CDN compatibility\n\nThe new [`Gateway.MaxRangeRequestFileSize`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaymaxrangerequestfilesize) configuration protects against CDN range request limitations that cause bandwidth overcharges on deserialized responses. Some CDNs convert range requests over large files into full file downloads, causing clients requesting small byte ranges to unknowingly download entire multi-gigabyte files.\n\nThis only impacts deserialized responses. Clients using verifiable block requests (`application/vnd.ipld.raw`) are not affected. See the [configuration documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaymaxrangerequestfilesize) for details.\n\n#### 🖥️ RISC-V support with prebuilt binaries\n\nKubo provides official `linux-riscv64` prebuilt binaries, bringing IPFS to [RISC-V](https://en.wikipedia.org/wiki/RISC-V) open hardware.\n\nAs RISC-V single-board computers and embedded systems become more accessible, the distributed web is now supported on open hardware architectures - a natural pairing of open technologies.\n\nDownload from or and look for the `linux-riscv64` archive.\n\n### 📦️ Important dependency updates\n\n- update `go-libp2p` to [v0.45.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.45.0) (incl. [v0.44.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.44.0)) with self-healing UPnP port mappings and go-log/slog interop fixes\n- update `quic-go` to [v0.55.0](https://github.com/quic-go/quic-go/releases/tag/v0.55.0)\n- update `go-log` to [v2.9.0](https://github.com/ipfs/go-log/releases/tag/v2.9.0) with slog integration for go-libp2p\n- update `go-ds-pebble` to [v0.5.7](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.7) (includes pebble [v2.1.2](https://github.com/cockroachdb/pebble/releases/tag/v2.1.2))\n- update `boxo` to [v0.35.2](https://github.com/ipfs/boxo/releases/tag/v0.35.2) (includes boxo [v0.35.1](https://github.com/ipfs/boxo/releases/tag/v0.35.1))\n- update `ipfs-webui` to [v4.10.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.10.0)\n- update `go-libp2p-kad-dht` to [v0.36.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.36.0)\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - docs: mkreleaselog for 0.39\n - chore: version 0.39.0\n - bin/mkreleaselog: add github handle resolution and deduplication\n - docs: restructure v0.39 changelog for clarity\n - upgrade go-libp2p-kad-dht to v0.36.0 (#11079) ([ipfs/kubo#11079](https://github.com/ipfs/kubo/pull/11079))\n - fix(docker): include symlinks in scanning for init scripts (#11077) ([ipfs/kubo#11077](https://github.com/ipfs/kubo/pull/11077))\n - Update deprecation message for Reprovider fields (#11072) ([ipfs/kubo#11072](https://github.com/ipfs/kubo/pull/11072))\n - chore: release v0.39.0-rc1\n - test: add regression tests for config secrets protection (#11061) ([ipfs/kubo#11061](https://github.com/ipfs/kubo/pull/11061))\n - test: add regression tests for API.Authorizations (#11060) ([ipfs/kubo#11060](https://github.com/ipfs/kubo/pull/11060))\n - test: verifyWorkerRun and helptext (#11063) ([ipfs/kubo#11063](https://github.com/ipfs/kubo/pull/11063))\n - test(cmdutils): add tests for PathOrCidPath and ValidatePinName (#11062) ([ipfs/kubo#11062](https://github.com/ipfs/kubo/pull/11062))\n - fix: return original error in PathOrCidPath fallback (#11059) ([ipfs/kubo#11059](https://github.com/ipfs/kubo/pull/11059))\n - feat: fast provide support in `dag import` (#11058) ([ipfs/kubo#11058](https://github.com/ipfs/kubo/pull/11058))\n - feat(cli/rpc/add): fast provide of root CID (#11046) ([ipfs/kubo#11046](https://github.com/ipfs/kubo/pull/11046))\n - feat(telemetry): collect high level provide DHT sweep settings (#11056) ([ipfs/kubo#11056](https://github.com/ipfs/kubo/pull/11056))\n - feat: enable DHT Provide Sweep by default (#10955) ([ipfs/kubo#10955](https://github.com/ipfs/kubo/pull/10955))\n - feat(config): optional Gateway.MaxRangeRequestFileSize (#10997) ([ipfs/kubo#10997](https://github.com/ipfs/kubo/pull/10997))\n - docs: clarify provide stats metric types and calculations (#11041) ([ipfs/kubo#11041](https://github.com/ipfs/kubo/pull/11041))\n - Upgrade to Boxo v0.35.2 (#11050) ([ipfs/kubo#11050](https://github.com/ipfs/kubo/pull/11050))\n - fix(go-log@2.9/go-libp2p@0.45): dynamic log level control and tail (#11039) ([ipfs/kubo#11039](https://github.com/ipfs/kubo/pull/11039))\n - chore: update webui to v4.10.0 (#11048) ([ipfs/kubo#11048](https://github.com/ipfs/kubo/pull/11048))\n - fix(provider/stats): number format (#11045) ([ipfs/kubo#11045](https://github.com/ipfs/kubo/pull/11045))\n - provider: protect libp2p connections (#11028) ([ipfs/kubo#11028](https://github.com/ipfs/kubo/pull/11028))\n - Merge release v0.38.2 ([ipfs/kubo#11044](https://github.com/ipfs/kubo/pull/11044))\n - Upgrade to Boxo v0.35.1 (#11043) ([ipfs/kubo#11043](https://github.com/ipfs/kubo/pull/11043))\n - feat(provider): resume cycle (#11031) ([ipfs/kubo#11031](https://github.com/ipfs/kubo/pull/11031))\n - chore: upgrade pebble to v2.1.1 (#11040) ([ipfs/kubo#11040](https://github.com/ipfs/kubo/pull/11040))\n - fix(cli): provide stat cosmetics (#11034) ([ipfs/kubo#11034](https://github.com/ipfs/kubo/pull/11034))\n - fix: go-libp2p v0.44 with self-healing UPnP port mappings (#11032) ([ipfs/kubo#11032](https://github.com/ipfs/kubo/pull/11032))\n - feat(provide): slow reprovide alerts when SweepEnabled (#11021) ([ipfs/kubo#11021](https://github.com/ipfs/kubo/pull/11021))\n - feat: trace delegated routing http client (#11017) ([ipfs/kubo#11017](https://github.com/ipfs/kubo/pull/11017))\n - feat(provide): detailed `ipfs provide stat` (#11019) ([ipfs/kubo#11019](https://github.com/ipfs/kubo/pull/11019))\n - config: increase default Provide.DHT.MaxProvideConnsPerWorker (#11016) ([ipfs/kubo#11016](https://github.com/ipfs/kubo/pull/11016))\n - docs: update release checklist based on v0.38.0 learnings (#11007) ([ipfs/kubo#11007](https://github.com/ipfs/kubo/pull/11007))\n - chore: merge release v0.38.1 ([ipfs/kubo#11020](https://github.com/ipfs/kubo/pull/11020))\n - fix: migrations for Windows (#11010) ([ipfs/kubo#11010](https://github.com/ipfs/kubo/pull/11010))\n - Upgrade go-ds-pebble to v0.5.3 (#11011) ([ipfs/kubo#11011](https://github.com/ipfs/kubo/pull/11011))\n - Merge release v0.38.0 ([ipfs/kubo#11006](https://github.com/ipfs/kubo/pull/11006))\n - feat: add docker stub for deprecated ipfs/go-ipfs name (#10998) ([ipfs/kubo#10998](https://github.com/ipfs/kubo/pull/10998))\n - docs: add sweeping provide worker count recommendation (#11001) ([ipfs/kubo#11001](https://github.com/ipfs/kubo/pull/11001))\n - chore: bump go-libp2p-kad-dht to v0.35.0 (#11002) ([ipfs/kubo#11002](https://github.com/ipfs/kubo/pull/11002))\n - upgrade go-ds-pebble to v0.5.2 (#11000) ([ipfs/kubo#11000](https://github.com/ipfs/kubo/pull/11000))\n - Upgrade to Boxo v0.35.0 (#10999) ([ipfs/kubo#10999](https://github.com/ipfs/kubo/pull/10999))\n - Non-functional changes (#10996) ([ipfs/kubo#10996](https://github.com/ipfs/kubo/pull/10996))\n - chore: update boxo and kad-dht dependencies (#10995) ([ipfs/kubo#10995](https://github.com/ipfs/kubo/pull/10995))\n - fix: update webui to v4.9.1 (#10994) ([ipfs/kubo#10994](https://github.com/ipfs/kubo/pull/10994))\n - fix: provider merge conflicts (#10989) ([ipfs/kubo#10989](https://github.com/ipfs/kubo/pull/10989))\n - fix(mfs): add soft limit for `--flush=false` (#10985) ([ipfs/kubo#10985](https://github.com/ipfs/kubo/pull/10985))\n - fix: provide Filestore nodes (#10990) ([ipfs/kubo#10990](https://github.com/ipfs/kubo/pull/10990))\n - feat: limit pin names to 255 bytes (#10981) ([ipfs/kubo#10981](https://github.com/ipfs/kubo/pull/10981))\n - fix: SweepingProvider slow start (#10980) ([ipfs/kubo#10980](https://github.com/ipfs/kubo/pull/10980))\n - chore: start v0.39.0 release cycle\n- github.com/gammazero/deque (v1.1.0 -> v1.2.0):\n - add slice operation functions (#40) ([gammazero/deque#40](https://github.com/gammazero/deque/pull/40))\n - maintain base capacity after IterPop iteration (#44) ([gammazero/deque#44](https://github.com/gammazero/deque/pull/44))\n- github.com/ipfs/boxo (v0.35.1 -> v0.35.2):\n - Release v0.35.2 ([ipfs/boxo#1068](https://github.com/ipfs/boxo/pull/1068))\n - fix(logs): upgrade go-libp2p to v0.45.0 and go-log to v2.9.0 ([ipfs/boxo#1066](https://github.com/ipfs/boxo/pull/1066))\n- github.com/ipfs/go-cid (v0.5.0 -> v0.6.0):\n - v0.6.0 bump (#178) ([ipfs/go-cid#178](https://github.com/ipfs/go-cid/pull/178))\n- github.com/ipfs/go-ds-pebble (v0.5.3 -> v0.5.7):\n - new version (#74) ([ipfs/go-ds-pebble#74](https://github.com/ipfs/go-ds-pebble/pull/74))\n - do not override logger if logger is provided (#72) ([ipfs/go-ds-pebble#72](https://github.com/ipfs/go-ds-pebble/pull/72))\n - new version (#70) ([ipfs/go-ds-pebble#70](https://github.com/ipfs/go-ds-pebble/pull/70))\n - new-version (#68) ([ipfs/go-ds-pebble#68](https://github.com/ipfs/go-ds-pebble/pull/68))\n - Do not allow batch to be reused after commit (#67) ([ipfs/go-ds-pebble#67](https://github.com/ipfs/go-ds-pebble/pull/67))\n - new version (#66) ([ipfs/go-ds-pebble#66](https://github.com/ipfs/go-ds-pebble/pull/66))\n - Make pebble write options configurable ([ipfs/go-ds-pebble#63](https://github.com/ipfs/go-ds-pebble/pull/63))\n- github.com/ipfs/go-dsqueue (v0.1.0 -> v0.1.1):\n - new version (#26) ([ipfs/go-dsqueue#26](https://github.com/ipfs/go-dsqueue/pull/26))\n - update deque package and add stress test (#25) ([ipfs/go-dsqueue#25](https://github.com/ipfs/go-dsqueue/pull/25))\n- github.com/ipfs/go-log/v2 (v2.8.2 -> v2.9.0):\n - chore: release v2.9.0 (#177) ([ipfs/go-log#177](https://github.com/ipfs/go-log/pull/177))\n - fix: go-libp2p and slog interop (#176) ([ipfs/go-log#176](https://github.com/ipfs/go-log/pull/176))\n- github.com/libp2p/go-libp2p (v0.43.0 -> v0.45.0):\n - Release v0.45.0 (#3424) ([libp2p/go-libp2p#3424](https://github.com/libp2p/go-libp2p/pull/3424))\n - feat(gologshim): Add SetDefaultHandler (#3418) ([libp2p/go-libp2p#3418](https://github.com/libp2p/go-libp2p/pull/3418))\n - Update Drips ownedBy address in FUNDING.json\n - fix(websocket): use debug level for http.Server errors\n - chore: release v0.44.0\n - autonatv2: fix normalization for websocket addrs\n - autonatv2: remove dependency on webrtc and webtransport\n - quicreuse: update libp2p/go-netroute (#3405) ([libp2p/go-libp2p#3405](https://github.com/libp2p/go-libp2p/pull/3405))\n - basichost: don't advertise unreachable addrs. (#3357) ([libp2p/go-libp2p#3357](https://github.com/libp2p/go-libp2p/pull/3357))\n - basichost: improve autonatv2 reachability logic (#3356) ([libp2p/go-libp2p#3356](https://github.com/libp2p/go-libp2p/pull/3356))\n - basichost: fix lint error\n - basichost: move EvtLocalAddrsChanged to addrs_manager (#3355) ([libp2p/go-libp2p#3355](https://github.com/libp2p/go-libp2p/pull/3355))\n - chore: gitignore go.work files\n - refactor!: move insecure transport outside of core\n - refactor: drop go-varint dependency\n - refactor!: move canonicallog package outside of core\n - fix: assignment to entry in nil map\n - docs: Update contribute section with mailing list and irc (#3387) ([libp2p/go-libp2p#3387](https://github.com/libp2p/go-libp2p/pull/3387))\n - README: remove Drand from notable users section\n - chore: add help comment\n - refactor: replace context.WithCancel with t.Context\n - feat(network): Add Conn.As\n - Skip mdns tests on macOS in CI\n - fix: deduplicate NAT port mapping requests\n - fix: heal NAT mappings after router restart\n - feat: relay: add option for custom filter function\n - docs: remove broken link (#3375) ([libp2p/go-libp2p#3375](https://github.com/libp2p/go-libp2p/pull/3375))\n - AI tooling must be disclosed for contributions (#3372) ([libp2p/go-libp2p#3372](https://github.com/libp2p/go-libp2p/pull/3372))\n - feat: Migrate to log/slog (#3364) ([libp2p/go-libp2p#3364](https://github.com/libp2p/go-libp2p/pull/3364))\n - basichost: move observed address manager to basichost (#3332) ([libp2p/go-libp2p#3332](https://github.com/libp2p/go-libp2p/pull/3332))\n - chore: support Go 1.24 & 1.25 (#3366) ([libp2p/go-libp2p#3366](https://github.com/libp2p/go-libp2p/pull/3366))\n - feat(simlibp2p): Simulated libp2p Networks (#3262) ([libp2p/go-libp2p#3262](https://github.com/libp2p/go-libp2p/pull/3262))\n - bandwidthcounter: add Reset and TrimIdle methods to reporter interface (#3343) ([libp2p/go-libp2p#3343](https://github.com/libp2p/go-libp2p/pull/3343))\n - network: rename NAT Types (#3331) ([libp2p/go-libp2p#3331](https://github.com/libp2p/go-libp2p/pull/3331))\n - refactor(quicreuse): use errors.Join in Close method (#3363) ([libp2p/go-libp2p#3363](https://github.com/libp2p/go-libp2p/pull/3363))\n - swarm: move AddCertHashes to swarm (#3330) ([libp2p/go-libp2p#3330](https://github.com/libp2p/go-libp2p/pull/3330))\n - quicreuse: clean up associations for closed listeners. (#3306) ([libp2p/go-libp2p#3306](https://github.com/libp2p/go-libp2p/pull/3306))\n- github.com/libp2p/go-libp2p-kad-dht (v0.35.1 -> v0.36.0):\n - new version (#1204) ([libp2p/go-libp2p-kad-dht#1204](https://github.com/libp2p/go-libp2p-kad-dht/pull/1204))\n - update dependencies (#1205) ([libp2p/go-libp2p-kad-dht#1205](https://github.com/libp2p/go-libp2p-kad-dht/pull/1205))\n - fix(provider): protect `SweepingProvider.wg` (#1200) ([libp2p/go-libp2p-kad-dht#1200](https://github.com/libp2p/go-libp2p-kad-dht/pull/1200))\n - fix(ResettableKeystore): race when closing during reset (#1201) ([libp2p/go-libp2p-kad-dht#1201](https://github.com/libp2p/go-libp2p-kad-dht/pull/1201))\n - fix(provider): conflict resolution (#1199) ([libp2p/go-libp2p-kad-dht#1199](https://github.com/libp2p/go-libp2p-kad-dht/pull/1199))\n - fix(provider): remove from trie by pruning prefix (#1198) ([libp2p/go-libp2p-kad-dht#1198](https://github.com/libp2p/go-libp2p-kad-dht/pull/1198))\n - fix(provider): rename metric to follow OpenTelemetry conventions (#1195) ([libp2p/go-libp2p-kad-dht#1195](https://github.com/libp2p/go-libp2p-kad-dht/pull/1195))\n - fix(provider): resume cycle from persisted keystore (#1193) ([libp2p/go-libp2p-kad-dht#1193](https://github.com/libp2p/go-libp2p-kad-dht/pull/1193))\n - feat(provider): connectivity callbacks (#1194) ([libp2p/go-libp2p-kad-dht#1194](https://github.com/libp2p/go-libp2p-kad-dht/pull/1194))\n - feat(provider): trie iterators (#1189) ([libp2p/go-libp2p-kad-dht#1189](https://github.com/libp2p/go-libp2p-kad-dht/pull/1189))\n - refactor(provider): optimize memory when allocating keys to peers (#1187) ([libp2p/go-libp2p-kad-dht#1187](https://github.com/libp2p/go-libp2p-kad-dht/pull/1187))\n - refactor(keystore): track size (#1181) ([libp2p/go-libp2p-kad-dht#1181](https://github.com/libp2p/go-libp2p-kad-dht/pull/1181))\n - Remove go-libp2p-maintainers from codeowners (#1192) ([libp2p/go-libp2p-kad-dht#1192](https://github.com/libp2p/go-libp2p-kad-dht/pull/1192))\n - switch to bit256.NewKeyFromArray (#1188) ([libp2p/go-libp2p-kad-dht#1188](https://github.com/libp2p/go-libp2p-kad-dht/pull/1188))\n - fix(provider): `RegionsFromPeers` may return multiple regions (#1185) ([libp2p/go-libp2p-kad-dht#1185](https://github.com/libp2p/go-libp2p-kad-dht/pull/1185))\n - feat(provider): skip bootstrap reprovide (#1186) ([libp2p/go-libp2p-kad-dht#1186](https://github.com/libp2p/go-libp2p-kad-dht/pull/1186))\n - refactor(provider): use adaptive deadline for CycleStats cleanup (#1183) ([libp2p/go-libp2p-kad-dht#1183](https://github.com/libp2p/go-libp2p-kad-dht/pull/1183))\n - refactor(provider/stats): use int64 to avoid overflows (#1182) ([libp2p/go-libp2p-kad-dht#1182](https://github.com/libp2p/go-libp2p-kad-dht/pull/1182))\n - provider: trigger connectivity check when missing libp2p addresses (#1180) ([libp2p/go-libp2p-kad-dht#1180](https://github.com/libp2p/go-libp2p-kad-dht/pull/1180))\n - fix(provider): resume cycle (#1176) ([libp2p/go-libp2p-kad-dht#1176](https://github.com/libp2p/go-libp2p-kad-dht/pull/1176))\n - tests: fix flaky TestProvidesExpire (#1179) ([libp2p/go-libp2p-kad-dht#1179](https://github.com/libp2p/go-libp2p-kad-dht/pull/1179))\n - tests: fix flaky TestFindPeerWithQueryFilter (#1178) ([libp2p/go-libp2p-kad-dht#1178](https://github.com/libp2p/go-libp2p-kad-dht/pull/1178))\n - tests: fix #1175 (#1177) ([libp2p/go-libp2p-kad-dht#1177](https://github.com/libp2p/go-libp2p-kad-dht/pull/1177))\n - feat(provider): exit early region exploration if no new peers discovered (#1174) ([libp2p/go-libp2p-kad-dht#1174](https://github.com/libp2p/go-libp2p-kad-dht/pull/1174))\n - provider: protect connections (#1172) ([libp2p/go-libp2p-kad-dht#1172](https://github.com/libp2p/go-libp2p-kad-dht/pull/1172))\n - feat(provider): resume reprovides (#1170) ([libp2p/go-libp2p-kad-dht#1170](https://github.com/libp2p/go-libp2p-kad-dht/pull/1170))\n - fix(provider): custom logger name (#1173) ([libp2p/go-libp2p-kad-dht#1173](https://github.com/libp2p/go-libp2p-kad-dht/pull/1173))\n - feat(provider): persist provide queue (#1167) ([libp2p/go-libp2p-kad-dht#1167](https://github.com/libp2p/go-libp2p-kad-dht/pull/1167))\n - provider: stats (#1144) ([libp2p/go-libp2p-kad-dht#1144](https://github.com/libp2p/go-libp2p-kad-dht/pull/1144))\n- github.com/probe-lab/go-libdht (v0.3.0 -> v0.4.0):\n - chore: release v0.4.0 (#26) ([probe-lab/go-libdht#26](https://github.com/probe-lab/go-libdht/pull/26))\n - feat(key/bit256): memory optimized constructor (#25) ([probe-lab/go-libdht#25](https://github.com/probe-lab/go-libdht/pull/25))\n - refactor(trie): AddMany memory optimization (#24) ([probe-lab/go-libdht#24](https://github.com/probe-lab/go-libdht/pull/24))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| [@guillaumemichel](https://github.com/guillaumemichel) | 41 | +9906/-1383 | 170 |\n| [@lidel](https://github.com/lidel) | 30 | +6652/-694 | 97 |\n| [@sukunrt](https://github.com/sukunrt) | 9 | +1618/-1524 | 39 |\n| [@MarcoPolo](https://github.com/MarcoPolo) | 17 | +1665/-1452 | 160 |\n| [@gammazero](https://github.com/gammazero) | 23 | +514/-53 | 29 |\n| [@Prabhat1308](https://github.com/Prabhat1308) | 1 | +197/-67 | 4 |\n| [@peterargue](https://github.com/peterargue) | 3 | +82/-25 | 5 |\n| [@cargoedit](https://github.com/cargoedit) | 1 | +35/-72 | 14 |\n| [@hsanjuan](https://github.com/hsanjuan) | 2 | +66/-29 | 5 |\n| [@shoriwe](https://github.com/shoriwe) | 1 | +68/-21 | 3 |\n| [@dennis-tra](https://github.com/dennis-tra) | 2 | +27/-2 | 2 |\n| [@Lil-Duckling-22](https://github.com/Lil-Duckling-22) | 1 | +4/-1 | 1 |\n| [@crStiv](https://github.com/crStiv) | 1 | +1/-3 | 1 |\n| [@cpeliciari](https://github.com/cpeliciari) | 1 | +3/-0 | 1 |\n| [@rvagg](https://github.com/rvagg) | 1 | +1/-1 | 1 |\n| [@p-shahi](https://github.com/p-shahi) | 1 | +1/-1 | 1 |\n| [@lbarrettanderson](https://github.com/lbarrettanderson) | 1 | +1/-1 | 1 |\n| [@filipremb](https://github.com/filipremb) | 1 | +1/-1 | 1 |\n| [@marten-seemann](https://github.com/marten-seemann) | 1 | +0/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.4.md", "content": "# go-ipfs changelog v0.4\n\n## v0.4.23 2020-01-29\n\nGiven the large number of fixes merged since 0.4.22, we've decided to cut another patch release.\n\nThis release contains critical fixes. Please upgrade ASAP. Importantly, we're strongly considering switching to TLS by default in go-ipfs 0.5.0 and dropping SECIO support. However, the current TLS transport in go-ipfs 0.4.22 has a bug that can cause connections to spontaneously disconnect during the handshake.\n\nThis release fixes that bug, among many other issues. Users that _don't_ upgrade may experience connectivity issues when the network upgrades to go-ipfs 0.5.0.\n\n### Highlights\n\n* Fixes build on go 1.13\n* Fixes an issue where we may not connect to providers in bitswap.\n* Fixes an issue on the TLS transport where we may abort a handshake unintentionally.\n* Fixes a common panic in the websocket transport.\n* Adds support for recursively resolving dnsaddrs (makes go-ipfs compatible with the new bootstrappers).\n* Fixes several potential panics/crashes.\n* Switches to using pre-defined autorelays instead of trying to find them in the DHT:\n * Avoids selecting random, potentially poor, relays.\n * Avoids spamming the DHT with requests trying to find relays.\n * Reduces the impact of accidentally enabling AutoRelay + RelayHop. I.e., the network won't try to DoS you.\n* Modifies the connection manager to not count connections in the grace period towards the connection limit.\n * Pro: New connections don't cause us to close useful, existing connections.\n * Con: Libp2p will keep more connections. Consider reducing your HighWater after applying this patch.\n* Improved peer usefulness tracking in bitswap. Frequently used peers will be marked as \"important\" and the connection manager will avoid closing connections to these peers.\n* Includes a new version of the WebUI to fix some issues with the peers map.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - feat: update the webui to fix some performance issues ([ipfs/go-ipfs#6844](https://github.com/ipfs/go-ipfs/pull/6844))\n - fix: limit SW registration to content root ([ipfs/go-ipfs#6801](https://github.com/ipfs/go-ipfs/pull/6801))\n - fix issue 6760, adding with hash-only, high CPU usage. ([ipfs/go-ipfs#6764](https://github.com/ipfs/go-ipfs/pull/6764))\n - fix(coreapi/add): close the fake repo used when adding with hash-only ([ipfs/go-ipfs#6747](https://github.com/ipfs/go-ipfs/pull/6747))\n - fix bug 6748 ([ipfs/go-ipfs#6754](https://github.com/ipfs/go-ipfs/pull/6754))\n - fix(pin): wait till after fetching to remove direct pin ([ipfs/go-ipfs#6708](https://github.com/ipfs/go-ipfs/pull/6708))\n - pin: fix pin update X Y where X==Y ([ipfs/go-ipfs#6669](https://github.com/ipfs/go-ipfs/pull/6669))\n - namesys: set the correct cache TTL on publish ([ipfs/go-ipfs#6667](https://github.com/ipfs/go-ipfs/pull/6667))\n - build: fix golangci again ([ipfs/go-ipfs#6641](https://github.com/ipfs/go-ipfs/pull/6641))\n - make: move all test deps to a separate module ([ipfs/go-ipfs#6637](https://github.com/ipfs/go-ipfs/pull/6637))\n - fix: close peerstore on stop ([ipfs/go-ipfs#6629](https://github.com/ipfs/go-ipfs/pull/6629))\n - build: fix build when we don't have a full git tree ([ipfs/go-ipfs#6626](https://github.com/ipfs/go-ipfs/pull/6626))\n- github.com/ipfs/go-bitswap (v0.0.8-cbb485998356 -> v0.0.8-e37498cf10d6):\n - fix: wait until we finish connecting before we cancel the context ([ipfs/go-bitswap#226](https://github.com/ipfs/go-bitswap/pull/226))\n - engine: tag peers based on usefulness ([ipfs/go-bitswap#191](https://github.com/ipfs/go-bitswap/pull/191))\n- github.com/ipfs/go-cid (v0.0.2 -> v0.0.4):\n - fix parsing issues and nits ([ipfs/go-cid#97](https://github.com/ipfs/go-cid/pull/97))\n - Verify that prefix is correct v0 prefix ([ipfs/go-cid#96](https://github.com/ipfs/go-cid/pull/96))\n- github.com/multiformats/go-multihash (v0.0.5 -> v0.0.10):\n - Ensure that length of multihash is properly handled ([multiformats/go-multihash#119](https://github.com/multiformats/go-multihash/pull/119))\n - fix murmur3 name ([multiformats/go-multihash#115](https://github.com/multiformats/go-multihash/pull/115))\n - rename ID to IDENTITY ([multiformats/go-multihash#113](https://github.com/multiformats/go-multihash/pull/113))\n ([multiformats/go-multihash#119](https://github.com/multiformats/go-multihash/pull/119))\n- github.com/libp2p/go-flow-metrics (v0.0.1 -> v0.0.3):\n - fix bug in meter traversal logic ([libp2p/go-flow-metrics#11](https://github.com/libp2p/go-flow-metrics/pull/11))\n- github.com/libp2p/go-libp2p (v0.0.28 -> v0.0.32):\n - options to configure known relays for autorelay ([libp2p/go-libp2p#705](https://github.com/libp2p/go-libp2p/pull/705))\n - feat(host): recursively resolve addresses ([libp2p/go-libp2p#764](https://github.com/libp2p/go-libp2p/pull/764))\n - mdns: always use interface addresses ([libp2p/go-libp2p#667](https://github.com/libp2p/go-libp2p/pull/667))\n- github.com/libp2p/go-libp2p-connmgr (v0.0.6 -> v0.2.1):\n - don't count connections in the grace period against the limit ([libp2p/go-libp2p-connmgr#50](https://github.com/libp2p/go-libp2p-connmgr/pull/50))\n- github.com/libp2p/go-libp2p-kad-dht (v0.0.13 -> v0.0.15):\n - metrics: fix memory leak ([libp2p/go-libp2p-kad-dht#390](https://github.com/libp2p/go-libp2p-kad-dht/pull/390))\n- github.com/libp2p/go-libp2p-tls (v0.0.1 -> v0.0.2):\n - close the underlying connection when the handshake fails ([libp2p/go-libp2p-tls#39](https://github.com/libp2p/go-libp2p-tls/pull/39))\n - make the error check for not receiving a public key more explicit ([libp2p/go-libp2p-tls#34](https://github.com/libp2p/go-libp2p-tls/pull/34))\n - Fix: Connection Closed after handshake ([libp2p/go-libp2p-tls#37](https://github.com/libp2p/go-libp2p-tls/pull/37))\n- github.com/libp2p/go-libp2p-swarm (v0.0.6 -> v0.0.7):\n - fix: don't assume that transports implement stringer ([libp2p/go-libp2p-swarm#134](https://github.com/libp2p/go-libp2p-swarm/pull/134))\n- github.com/libp2p/go-ws-transport (v0.0.4 -> v0.0.6):\n - Add mutex for write/close ([libp2p/go-ws-transport#65](https://github.com/libp2p/go-ws-transport/pull/65))\n\nOther:\n\nUpdate bloom filter libraries to remove unsound usage of the `unsafe` package.\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Steven Allen | 52 | +1866/-578 | 102 |\n| vyzo | 12 | +167/-90 | 22 |\n| whyrusleeping | 5 | +136/-52 | 7 |\n| Roman Proskuryakov | 7 | +94/-7 | 10 |\n| Jakub Sztandera | 3 | +58/-13 | 7 |\n| hcg1314 | 2 | +31/-11 | 2 |\n| Raúl Kripalani | 2 | +7/-33 | 6 |\n| Marten Seemann | 3 | +27/-10 | 5 |\n| Marcin Rataj | 2 | +26/-0 | 5 |\n| b5 | 1 | +2/-22 | 1 |\n| Hector Sanjuan | 1 | +11/-0 | 1 |\n| Yusef Napora | 1 | +4/-0 | 1 |\n\n## v0.4.22 2019-08-06\n\nWe're releasing a PATCH release of go-ipfs based on 0.4.21 containing some critical fixes.\n\nThe IPFS network has scaled to the point where small changes can have a\nwide-reaching impact on the entire network. To keep this situation from\nescalating, we've put a hold on releasing new features until we can improve our\n[release process](https://github.com/ipfs/go-ipfs/blob/master/docs/releases.md)\n(which we've trialed in this release) and [testing\nprocedures](https://github.com/ipfs/go-ipfs/issues/6483).\n\nThis release includes fixes for the following regressions:\n\n1. A major bitswap throughput regression introduced in 0.4.21\n ([ipfs/go-ipfs#6442](https://github.com/ipfs/go-ipfs/issues/6442)).\n2. High bitswap CPU usage when connected to many (e.g. 10,000) peers. See\n [ipfs/go-bitswap#154](https://github.com/ipfs/go-bitswap/issues/154).\n2. The local network discovery service sometimes initializes before the\n networking module, causing it to announce the wrong addresses and sometimes\n complain about not being able to determine the IP address\n ([ipfs/go-ipfs#6415](https://github.com/ipfs/go-ipfs/pull/6415)).\n\nIt also includes fixes for:\n\n1. Pins not being persisted after `ipfs block add --pin`\n ([ipfs/go-ipfs#6441](https://github.com/ipfs/go-ipfs/pull/6441)).\n2. Panic due to concurrent map access when adding and listing pins at the same\n time ([ipfs/go-ipfs#6419](https://github.com/ipfs/go-ipfs/pull/6419)).\n3. Potential pin-set corruption given a concurrent `ipfs repo gc` and `ipfs pin\n rm` ([ipfs/go-ipfs#6444](https://github.com/ipfs/go-ipfs/pull/6444)).\n4. Build failure due to a deleted git tag in one of our dependencies\n ([ipfs/go-ds-badger#64](https://github.com/ipfs/go-ds-badger/pull/65)).\n\nThanks to:\n\n* [@hannahhoward](https://github.com/hannahhoward) for fixing both bitswap issues.\n* [@sanderpick](https://github.com/sanderpick) for catching and fixing the local\n discovery bug.\n* [@campoy](https://github.com/campoy) for fixing the build issue.\n\n## v0.4.21 2019-05-30\n\nWe're happy to announce go-ipfs 0.4.21. This release has some critical bug fixes\nand a handful of new features so every user should upgrade.\n\nKey bug fixes:\n\n* Too many open file descriptors/too many peers\n ([#6237](https://github.com/ipfs/go-ipfs/issues/6237)).\n* Adding multiple files at the same time doesn't work\n ([#6254](https://github.com/ipfs/go-ipfs/pull/6255)).\n* CPU utilization spikes and then holds at 100%\n ([#5613](https://github.com/ipfs/go-ipfs/issues/5613)).\n\nKey features:\n\n* Experimental TLS1.3 support (to eventually replace secio).\n* OpenSSL support for SECIO handshakes (performance improvement).\n\n**IMPORTANT:** This release fixes a bug in our security transport that could\npotentially drop data from the channel. Note: This issue affects neither the\nprivacy nor the integrity of the data with respect to a third-party attacker.\nOnly the peer sending us data could trigger this bug.\n\n**ALL USERS MUST UPGRADE.** We intended to introduce a feature this release that,\nunfortunately, [reliably triggered this bug][secio-bug]. To avoid partitioning\nthe network, we've decided to postpone this feature for a release or two.\n\nSpecifically, we're going to provide a minimum _one month_ upgrade period. After\nthat, we'll start testing the impact of deploying the proposed changes.\n\nIf you're running the mainline go-ipfs, please upgrade ASAP. If you're building\na separate app or working on a forked go-ipfs, make sure to upgrade\ngithub.com/libp2p/go-libp2p-secio to _at least_ v0.0.3.\n\n[secio-bug]: https://github.com/libp2p/go-libp2p/issues/644\n\n### Contributors\n\nFirst off, we'd like to give a shout-out to all contributors that participated\nin this release (including contributions to ipld, libp2p, and multiformats):\n\n| Contributor | Commits | Lines ± | Files Changed |\n|----------------------------|---------|-------------|---------------|\n| Steven Allen | 220 | +6078/-4211 | 520 |\n| Łukasz Magiera | 53 | +5039/-4557 | 274 |\n| vyzo | 179 | +2929/-1704 | 238 |\n| Raúl Kripalani | 44 | +757/-1895 | 134 |\n| hannahhoward | 11 | +755/-1005 | 49 |\n| Marten Seemann | 16 | +862/-203 | 44 |\n| keks | 10 | +359/-110 | 12 |\n| Jan Winkelmann | 8 | +368/-26 | 16 |\n| Jakub Sztandera | 4 | +361/-8 | 7 |\n| Adrian Lanzafame | 1 | +287/-18 | 5 |\n| Erik Ingenito | 4 | +247/-28 | 8 |\n| Reid 'arrdem' McKenzie | 1 | +220/-20 | 3 |\n| Yusef Napora | 26 | +98/-130 | 26 |\n| Michael Avila | 3 | +116/-59 | 8 |\n| Raghav Gulati | 13 | +145/-26 | 13 |\n| tg | 1 | +41/-33 | 1 |\n| Matt Joiner | 6 | +41/-30 | 7 |\n| Cole Brown | 1 | +37/-25 | 1 |\n| Dominic Della Valle | 2 | +12/-40 | 4 |\n| Overbool | 1 | +50/-0 | 2 |\n| Christopher Buesser | 3 | +29/-16 | 10 |\n| myself659 | 1 | +38/-5 | 2 |\n| Alex Browne | 3 | +30/-8 | 3 |\n| jmank88 | 1 | +27/-4 | 2 |\n| Vikram | 1 | +25/-1 | 2 |\n| MollyM | 7 | +17/-9 | 7 |\n| Marcin Rataj | 1 | +17/-1 | 1 |\n| requilence | 1 | +11/-4 | 1 |\n| Teran McKinney | 1 | +8/-2 | 1 |\n| Oli Evans | 1 | +5/-5 | 1 |\n| Masashi Salvador Mitsuzawa | 1 | +5/-1 | 1 |\n| chenminjian | 1 | +4/-0 | 1 |\n| Edgar Lee | 1 | +3/-1 | 1 |\n| Dirk McCormick | 1 | +2/-2 | 2 |\n| ia | 1 | +1/-1 | 1 |\n| Alan Shaw | 1 | +1/-1 | 1 |\n\n### Bug Fixes And Enhancements\n\nThis release includes quite a number of critical bug fixes and\nperformance/reliability enhancements.\n\n#### Error when adding multiple files\n\nThe last release broke the simple command `ipfs add file1 file2`. It turns out\nwe simply lacked a test case for this. Both of these issues (the bug and the\nlack of a test case) have now been fixed.\n\n#### SECIO\n\nAs noted above, we've fixed a bug that could cause data to be dropped from a\nSECIO connection on read. Specifically, this happens when:\n\n1. The capacity of the read buffer is greater than the length.\n2. The remote peer sent more than the length but less than the capacity in a\n single secio \"frame\".\n\nIn this case, we'd fill the read buffer to it's capacity instead of its length.\n\n#### Too many open files, too many peers, etc.\n\nGo-ipfs automatically closes the least useful connections when it accumulates\ntoo many connections. Unfortunately, some relayed connections were blocking in\n`Close()`, halting the entire process.\n\n#### Out of control CPU usage\n\nMany users noted out of control CPU usage this release. This turned out to be a\nlong-standing issue with how the DHT handled provider records (records recording\nwhich peers have what content):\n\n1. It wasn't removing provider records for content until the set of providers\n completely emptied.\n2. It was loading every provider record into memory whenever we updated the set\n of providers.\n\nCombined, these two issues were trashing the provider record cache, forcing the\nDHT to repeatedly load and discard provider records.\n\n#### More Reliable Connection Management\n\nGo-ipfs has a subsystem called the \"connection manager\" to close the\nleast-useful connections when go-ipfs runs low on resources.\n\nUnfortunately, other IPFS subsystems may learn about connections _before_ the\nconnection manager. Previously, if some IPFS subsystem tried to mark a\nconnection as useful before the connection manager learned about it, the\nconnection manager would discard this information. We believe this was causing\n[#6271](https://github.com/ipfs/go-ipfs/issues/6271). [It no longer does\nthat](https://github.com/libp2p/go-libp2p-connmgr/pull/39).\n\n#### Improved Bitswap Connection Management\n\nBitswap now uses the connection manager to mark all peers downloading blocks as\nimportant (while downloading). Previously, it only marked peers from which _it_\nwas downloading blocks.\n\n#### Reduced Memory Usage\n\nThe most noticeable memory reduction in this release comes from fixing connection\nclosing. However, we've made a few additional improvements:\n\n* Bitswap's \"work queue\" no longer remembers every peer it has seen\n indefinitely.\n* The peerstore now interns protocol names.\n* The per-peer goroutine count has been reduced.\n* The DHT now wastes less memory on idle peers by pooling buffered writers and\n returning them to the pool when not actively using them.\n\n#### Increased File Descriptor Limit\n\nThe default file descriptor limit has been raised to 8192 (from 2048).\nUnfortunately, go-ipfs behaves poorly when it runs out of file descriptors and\nit uses a _lot_ of file descriptors.\n\nLuckily, most modern kernels can handle thousands of file descriptors without\nany difficulty.\n\n#### Decreased Connection Handshake Latency\n\nLibp2p now shaves off a couple of round trips when initiating connections by\nbeginning the protocol negotiation before the remote peer responds to the\ninitial handshake message.\n\nIn the optimal case (when the target peer speaks our preferred protocol), this\nreduces the number of handshake round-trips from 6 to 4 (including the TCP\nhandshake).\n\n### Commands\n\nThis release brings no new commands but does introduce a few changes, bug fixes,\nand enhancements. This section is hardly complete but it lists the most\nnoticeable changes.\n\nTake note: this release also introduces a few breaking changes.\n\n#### [DEPRECATION] The URLStore Command Deprecated\n\nThe experimental `ipfs urlstore` command is now deprecated. Please use `ipfs add\n--nocopy URL` instead.\n\n#### [BREAKING] The DHT Command Base64 Encodes Values\n\nWhen responding to an `ipfs dht get` command, the daemon now encodes the\nreturned value using base64. The `ipfs` command will automatically decode this\nvalue before returning it to the user so this change should only affect those\nusing the HTTP API directly.\n\nUnfortunately, this change was necessary as DHT records are arbitrary binary\nblobs which can't be directly stored in JSON strings.\n\n#### [BREAKING] Base32 Encoded v1 CIDs By Default\n\nBoth js-ipfs and go-ipfs now encode CIDv1 CIDs using base32 by default, instead\nof base58. Unfortunately, base58 is case-sensitive and doesn't play well with\nbrowsers (see [#4143](https://github.com/ipfs/go-ipfs/issues/4143).\n\n#### Human Readable Numbers\n\nThe `ipfs bitswap stat` and `ipfs object stat` commands now support a\n`--humanize` flag that formats numbers with human-readable units (GiB, MiB,\netc.).\n\n#### Improved Errors\n\nThis release improves two types of errors:\n\n1. Commands that take paths/multiaddrs now include the path/multiaddr in the\n error message when it fails to parse.\n2. `ipfs swarm connect` now returns a detailed error describing which addresses\n were tried and why the dial failed.\n\n#### Ping Improvements\n\nThe ping command has received some small improvements and fixes:\n\n1. It now exits with a non-zero exit status on failure.\n2. It no longer succeeds with zero successful pings if we have a zombie but\n non-functional connection to the peer being pinged\n ([#6298](https://github.com/ipfs/go-ipfs/issues/6298)).\n3. It now prints out the average latency when canceled with `^C` (like the unix\n `ping` command).\n\n#### Improved Help Text\n\nGo-ipfs now intelligently wraps help text for easier reading. On an 80 character\nwide terminal,\n\n**Before**\n\n```\nUSAGE\n ipfs add ... - Add a file or directory to ipfs.\n\nSYNOPSIS\n ipfs add [--recursive | -r] [--dereference-args] [--stdin-name=] [\n--hidden | -H] [--quiet | -q] [--quieter | -Q] [--silent] [--progress | -p] [--t\nrickle | -t] [--only-hash | -n] [--wrap-with-directory | -w] [--chunker= | -s] [--pin=false] [--raw-leaves] [--nocopy] [--fscache] [--cid-version=] [--hash=] [--inline] [--inline-limit=] [--] \n...\n\nARGUMENTS\n\n ... - The path to a file to be added to ipfs.\n\nOPTIONS\n\n -r, --recursive bool - Add directory paths recursive\nly.\n --dereference-args bool - Symlinks supplied in argument\ns are dereferenced.\n --stdin-name string - Assign a name if the file sou\nrce is stdin.\n -H, --hidden bool - Include files that are hidden\n. Only takes effect on recursive add.\n -q, --quiet bool - Write minimal output.\n -Q, --quieter bool - Write only final hash.\n --silent bool - Write no output.\n -p, --progress bool - Stream progress data.\n -t, --trickle bool - Use trickle-dag format for da\ng generation.\n -n, --only-hash bool - Only chunk and hash - do not\nwrite to disk.\n -w, --wrap-with-directory bool - Wrap files with a directory o\nobject.\n -s, --chunker string - Chunking algorithm, size-[byt\nes] or rabin-[min]-[avg]-[max]. Default: size-262144.\n --pin bool - Pin this object when adding.\nDefault: true.\n --raw-leaves bool - Use raw blocks for leaf nodes\n. (experimental).\n --nocopy bool - Add the file using filestore.\n Implies raw-leaves. (experimental).\n --fscache bool - Check the filestore for pre-e\nxisting blocks. (experimental).\n --cid-version int - CID version. Defaults to 0 un\nless an option that depends on CIDv1 is passed. (experimental).\n --hash string - Hash function to use. Implies\n CIDv1 if not sha2-256. (experimental). Default: sha2-256.\n --inline bool - Inline small blocks into CIDs\n. (experimental).\n --inline-limit int - Maximum block size to inline.\n (experimental). Default: 32.\n\n```\n\n\n**After**\n\n```\nUSAGE\n ipfs add ... - Add a file or directory to ipfs.\n\nSYNOPSIS\n ipfs add [--recursive | -r] [--dereference-args] [--stdin-name=]\n [--hidden | -H] [--quiet | -q] [--quieter | -Q] [--silent]\n [--progress | -p] [--trickle | -t] [--only-hash | -n]\n [--wrap-with-directory | -w] [--chunker= | -s] [--pin=false]\n [--raw-leaves] [--nocopy] [--fscache] [--cid-version=]\n [--hash=] [--inline] [--inline-limit=] [--]\n ...\n\nARGUMENTS\n\n ... - The path to a file to be added to ipfs.\n\nOPTIONS\n\n -r, --recursive bool - Add directory paths recursively.\n --dereference-args bool - Symlinks supplied in arguments are\n dereferenced.\n --stdin-name string - Assign a name if the file source is stdin.\n -H, --hidden bool - Include files that are hidden. Only takes\n effect on recursive add.\n -q, --quiet bool - Write minimal output.\n -Q, --quieter bool - Write only final hash.\n --silent bool - Write no output.\n -p, --progress bool - Stream progress data.\n -t, --trickle bool - Use trickle-dag format for dag generation.\n -n, --only-hash bool - Only chunk and hash - do not write to\n disk.\n -w, --wrap-with-directory bool - Wrap files with a directory object.\n -s, --chunker string - Chunking algorithm, size-[bytes] or\n rabin-[min]-[avg]-[max]. Default:\n size-262144.\n --pin bool - Pin this object when adding. Default:\n true.\n --raw-leaves bool - Use raw blocks for leaf nodes.\n (experimental).\n --nocopy bool - Add the file using filestore. Implies\n raw-leaves. (experimental).\n --fscache bool - Check the filestore for pre-existing\n blocks. (experimental).\n --cid-version int - CID version. Defaults to 0 unless an\n option that depends on CIDv1 is passed.\n (experimental).\n --hash string - Hash function to use. Implies CIDv1 if\n not sha2-256. (experimental). Default:\n sha2-256.\n --inline bool - Inline small blocks into CIDs.\n (experimental).\n --inline-limit int - Maximum block size to inline.\n (experimental). Default: 32.\n```\n\n### Features\n\nThis release is primarily a bug fix release but it still includes two nice\nfeatures from libp2p.\n\n#### Experimental TLS1.3 support\n\nGo-ipfs now has experimental TLS1.3 support. Currently, libp2p (IPFS's\nnetworking library) uses a custom TLS-like protocol we call SECIO. However, the\nconventional wisdom concerning custom security transports is \"just don't\" so we\nare working on replacing it with TLS1.3\n\nTo choose this protocol by default, set the `Experimental.PreferTLS` config\nvariable:\n\n```bash\n> ipfs config --bool Experimental.PreferTLS true\n```\n\nWhy TLS1.3 and not X (noise, etc.)?\n\n1. Libp2p allows negotiating transports so there's no reason not to add noise\n support to libp2p as well.\n2. TLS has wide language support which should make implementing libp2p for new\n languages significantly simpler.\n\n#### OpenSSL Support\n\nGo-ipfs can now (optionally) be built with OpenSSL support for improved\nperformance when establishing connections. This is primarily useful for nodes\nreceiving multiple inbound connections per second.\n\nTo enable openssl support, rebuild go-ipfs with:\n\n```bash\n> make build GOTAGS=openssl\n```\n\n### CoreAPI\n\nThe CoreAPI refactor is still underway and we've made significant progress\ntowards a usable ipfs-as-a-library constructor. Specifically, we've integrated\nthe [fx](https://go.uber.org/fx) dependency injection system and are\nnow working on cleaning up our initialization logic. This should make it easier\nto inject new services into a go-ipfs process without messing with the core\ninternals.\n\n### Build: `GOCC` Environment Variable\n\nBuild system now uses `GOCC` environment variable allowing for use of specific\ngo versions during builds.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - fix: use http.Error for sending errors ([ipfs/go-ipfs#6379](https://github.com/ipfs/go-ipfs/pull/6379))\n - core: call app.Stop once ([ipfs/go-ipfs#6380](https://github.com/ipfs/go-ipfs/pull/6380))\n - explain what dhtclient does ([ipfs/go-ipfs#6375](https://github.com/ipfs/go-ipfs/pull/6375))\n - ci: actually enable golangci-lint ([ipfs/go-ipfs#6362](https://github.com/ipfs/go-ipfs/pull/6362))\n - commands/swarm(fix): handle empty multiaddrs ([ipfs/go-ipfs#6355](https://github.com/ipfs/go-ipfs/pull/6355))\n - feat: improve errors when a path fails to parse ([ipfs/go-ipfs#6346](https://github.com/ipfs/go-ipfs/pull/6346))\n - fix vendoring dependencies when building the source tarball ([ipfs/go-ipfs#6349](https://github.com/ipfs/go-ipfs/pull/6349))\n - core: Use correct default for connmgr lowWater ([ipfs/go-ipfs#6352](https://github.com/ipfs/go-ipfs/pull/6352))\n - doc: remove out of date documentation ([ipfs/go-ipfs#6345](https://github.com/ipfs/go-ipfs/pull/6345))\n - Add generation of dependency changes to mkreleaselog ([ipfs/go-ipfs#6348](https://github.com/ipfs/go-ipfs/pull/6348))\n - readme: remove mention of DCO ([ipfs/go-ipfs#6344](https://github.com/ipfs/go-ipfs/pull/6344))\n - Add golangci-lint ([ipfs/go-ipfs#6321](https://github.com/ipfs/go-ipfs/pull/6321))\n - docs+mk: update guidance for unsupported platforms ([ipfs/go-ipfs#6338](https://github.com/ipfs/go-ipfs/pull/6338))\n - fix formatting in object get ([ipfs/go-ipfs#6340](https://github.com/ipfs/go-ipfs/pull/6340))\n - fail start when loading a plugin fails ([ipfs/go-ipfs#6339](https://github.com/ipfs/go-ipfs/pull/6339))\n - fix a typo in the issue template ([ipfs/go-ipfs#6335](https://github.com/ipfs/go-ipfs/pull/6335))\n - github: turn issue template into a multiple-choice question ([ipfs/go-ipfs#6333](https://github.com/ipfs/go-ipfs/pull/6333))\n - object put: Allow empty objects ([ipfs/go-ipfs#6330](https://github.com/ipfs/go-ipfs/pull/6330))\n - Update fuse.md ([ipfs/go-ipfs#6332](https://github.com/ipfs/go-ipfs/pull/6332))\n - work towards fixing dht commands ([ipfs/go-ipfs#6277](https://github.com/ipfs/go-ipfs/pull/6277))\n - fix setting ulimit ([ipfs/go-ipfs#6319](https://github.com/ipfs/go-ipfs/pull/6319))\n - switch to base32 by default for CIDv1 ([ipfs/go-ipfs#6300](https://github.com/ipfs/go-ipfs/pull/6300))\n - cmdkit -> cmds ([ipfs/go-ipfs#6318](https://github.com/ipfs/go-ipfs/pull/6318))\n - raise default fd limit to 8192 ([ipfs/go-ipfs#6266](https://github.com/ipfs/go-ipfs/pull/6266))\n - pin: don't walk all pinned blocks when removing a non-existent pin ([ipfs/go-ipfs#6311](https://github.com/ipfs/go-ipfs/pull/6311))\n - ping: fix a bunch of issues ([ipfs/go-ipfs#6312](https://github.com/ipfs/go-ipfs/pull/6312))\n - test(coreapi): use a thread-safe datastore everywhere ([ipfs/go-ipfs#6222](https://github.com/ipfs/go-ipfs/pull/6222))\n - fix(Dockerfile): Allow ipfs mount in Docker container ([ipfs/go-ipfs#5560](https://github.com/ipfs/go-ipfs/pull/5560))\n - docs: fix Routing section ([ipfs/go-ipfs#6309](https://github.com/ipfs/go-ipfs/pull/6309))\n - License update to dual MIT and Apache 2 ([ipfs/go-ipfs#6301](https://github.com/ipfs/go-ipfs/pull/6301))\n - Go test fix ([ipfs/go-ipfs#6293](https://github.com/ipfs/go-ipfs/pull/6293))\n - commands(pin update): return resolved CIDs instead of paths ([ipfs/go-ipfs#6275](https://github.com/ipfs/go-ipfs/pull/6275))\n - core: fix autonat construction ([ipfs/go-ipfs#6289](https://github.com/ipfs/go-ipfs/pull/6289))\n - Test and fix GC/pin bug ([ipfs/go-ipfs#6288](https://github.com/ipfs/go-ipfs/pull/6288))\n - GOCC implementation & fix in make & build scripts ([ipfs/go-ipfs#6282](https://github.com/ipfs/go-ipfs/pull/6282))\n - gc: cancel context ([ipfs/go-ipfs#6281](https://github.com/ipfs/go-ipfs/pull/6281))\n - fix: windows friendly daemon help ([ipfs/go-ipfs#6278](https://github.com/ipfs/go-ipfs/pull/6278))\n - Invert constructor config handling ([ipfs/go-ipfs#6276](https://github.com/ipfs/go-ipfs/pull/6276))\n - docs: document environment variables ([ipfs/go-ipfs#6268](https://github.com/ipfs/go-ipfs/pull/6268))\n - add: Return error from iterator ([ipfs/go-ipfs#6272](https://github.com/ipfs/go-ipfs/pull/6272))\n - commands(feat): use the coreapi in the urlstore command ([ipfs/go-ipfs#6259](https://github.com/ipfs/go-ipfs/pull/6259))\n - humanize for ipfs bitswap stat ([ipfs/go-ipfs#6258](https://github.com/ipfs/go-ipfs/pull/6258))\n - Revert \"raise default fd limit to 8192\" ([ipfs/go-ipfs#6265](https://github.com/ipfs/go-ipfs/pull/6265))\n - raise default fd limit to 8192 ([ipfs/go-ipfs#6261](https://github.com/ipfs/go-ipfs/pull/6261))\n - Fix AutoNAT service for private network ([ipfs/go-ipfs#6251](https://github.com/ipfs/go-ipfs/pull/6251))\n - add: Fix adding multiple files ([ipfs/go-ipfs#6255](https://github.com/ipfs/go-ipfs/pull/6255))\n - reprovider: Use goprocess ([ipfs/go-ipfs#6248](https://github.com/ipfs/go-ipfs/pull/6248))\n - core/corehttp/gateway_handler: pass a request ctx instead of the node ([ipfs/go-ipfs#6244](https://github.com/ipfs/go-ipfs/pull/6244))\n - constructor: cleanup some things ([ipfs/go-ipfs#6246](https://github.com/ipfs/go-ipfs/pull/6246))\n - Support --human flag in cmd/object-stat ([ipfs/go-ipfs#6241](https://github.com/ipfs/go-ipfs/pull/6241))\n - build: fix macos build with fuse ([ipfs/go-ipfs#6235](https://github.com/ipfs/go-ipfs/pull/6235))\n - add an experiment to prefer TLS 1.3 over secio ([ipfs/go-ipfs#6229](https://github.com/ipfs/go-ipfs/pull/6229))\n - fix two small nits in the go-ipfs constructor ([ipfs/go-ipfs#6234](https://github.com/ipfs/go-ipfs/pull/6234))\n - DI-based core.NewNode ([ipfs/go-ipfs#6162](https://github.com/ipfs/go-ipfs/pull/6162))\n - coreapi: Drop error from ParsePath ([ipfs/go-ipfs#6122](https://github.com/ipfs/go-ipfs/pull/6122))\n - fix the wrong path configuration in root redirection ([ipfs/go-ipfs#6215](https://github.com/ipfs/go-ipfs/pull/6215))\n- github.com/ipfs/go-bitswap (v0.0.4 -> v0.0.7):\n - feat(engine): tag peers with requests ([ipfs/go-bitswap#128](https://github.com/ipfs/go-bitswap/pull/128))\n - fix(network): add mutex to avoid data race ([ipfs/go-bitswap#127](https://github.com/ipfs/go-bitswap/pull/127))\n - Change bitswap provide toggle to not be static ([ipfs/go-bitswap#124](https://github.com/ipfs/go-bitswap/pull/124))\n - Use shared peer task queue with Graphsync ([ipfs/go-bitswap#119](https://github.com/ipfs/go-bitswap/pull/119))\n - Add missing godoc comments, refactor to avoid confusion ([ipfs/go-bitswap#117](https://github.com/ipfs/go-bitswap/pull/117))\n - fix(decision): cleanup request queues ([ipfs/go-bitswap#116](https://github.com/ipfs/go-bitswap/pull/116))\n - Control provider workers with experiment flag ([ipfs/go-bitswap#110](https://github.com/ipfs/go-bitswap/pull/110))\n - connmgr: give peers more weight when actively participating in a session ([ipfs/go-bitswap#111](https://github.com/ipfs/go-bitswap/pull/111))\n - make the WantlistManager own the PeerHandler ([ipfs/go-bitswap#78](https://github.com/ipfs/go-bitswap/pull/78))\n - remove IPFS_LOW_MEM flag support ([ipfs/go-bitswap#115](https://github.com/ipfs/go-bitswap/pull/115))\n- github.com/ipfs/go-cid (v0.0.1 -> v0.0.2):\n - default cidv1 to base32 ([ipfs/go-cid#85](https://github.com/ipfs/go-cid/pull/85))\n- github.com/ipfs/go-cidutil (v0.0.1 -> v0.0.2):\n - default cidv1 to base32 ([ipfs/go-cidutil#13](https://github.com/ipfs/go-cidutil/pull/13))\n- github.com/ipfs/go-datastore (v0.0.3 -> v0.0.5):\n - MapDatastore: obey KeysOnly ([ipfs/go-datastore#130](https://github.com/ipfs/go-datastore/pull/130))\n - fix the keytransform datastore's query implementation ([ipfs/go-datastore#127](https://github.com/ipfs/go-datastore/pull/127))\n - sync: apply entire query while locked ([ipfs/go-datastore#129](https://github.com/ipfs/go-datastore/pull/129))\n - filter: values are now always bytes ([ipfs/go-datastore#126](https://github.com/ipfs/go-datastore/pull/126))\n - autobatch: batch deletes ([ipfs/go-datastore#128](https://github.com/ipfs/go-datastore/pull/128))\n- github.com/ipfs/go-ipfs-cmds (v0.0.5 -> v0.0.8):\n - fix: use golang's http.Error to send errors ([ipfs/go-ipfs-cmds#167](https://github.com/ipfs/go-ipfs-cmds/pull/167))\n - improve help text on narrow terminals ([ipfs/go-ipfs-cmds#140](https://github.com/ipfs/go-ipfs-cmds/pull/140))\n - chore: remove an old hack ([ipfs/go-ipfs-cmds#165](https://github.com/ipfs/go-ipfs-cmds/pull/165))\n - http: use the request context ([ipfs/go-ipfs-cmds#163](https://github.com/ipfs/go-ipfs-cmds/pull/163))\n - merge in go-ipfs-cmdkit ([ipfs/go-ipfs-cmds#164](https://github.com/ipfs/go-ipfs-cmds/pull/164))\n - fix: return the correct error ([ipfs/go-ipfs-cmds#162](https://github.com/ipfs/go-ipfs-cmds/pull/162))\n- github.com/ipfs/go-ipfs-config (v0.0.1 -> v0.0.3):\n - Closes: #6284 Add appropriate IPv6 ranges to defaultServerFilters ([ipfs/go-ipfs-config#34](https://github.com/ipfs/go-ipfs-config/pull/34))\n - add an experiment to prefer TLS 1.3 over secio ([ipfs/go-ipfs-config#32](https://github.com/ipfs/go-ipfs-config/pull/32))\n- github.com/ipfs/go-ipfs-files (v0.0.2 -> v0.0.3):\n - webfile: make Size() work before Read ([ipfs/go-ipfs-files#18](https://github.com/ipfs/go-ipfs-files/pull/18))\n - check http status code during WebFile reads and return error for non-2XX ([ipfs/go-ipfs-files#17](https://github.com/ipfs/go-ipfs-files/pull/17))\n- github.com/ipfs/go-ipld-cbor (v0.0.1 -> v0.0.2):\n - switch to base32 by default ([ipfs/go-ipld-cbor#62](https://github.com/ipfs/go-ipld-cbor/pull/62))\n- github.com/ipfs/go-ipld-git (v0.0.1 -> v0.0.2):\n - switch to base32 by default ([ipfs/go-ipld-git#40](https://github.com/ipfs/go-ipld-git/pull/40))\n- github.com/ipfs/go-mfs (v0.0.4 -> v0.0.7):\n - Fix directory mv and add tests ([ipfs/go-mfs#76](https://github.com/ipfs/go-mfs/pull/76))\n - fix: not remove file by mistakes ([ipfs/go-mfs#73](https://github.com/ipfs/go-mfs/pull/73))\n- github.com/ipfs/go-path (v0.0.3 -> v0.0.4):\n - include the path in path errors ([ipfs/go-path#28](https://github.com/ipfs/go-path/pull/28))\n- github.com/ipfs/go-unixfs (v0.0.4 -> v0.0.6):\n - chore: remove URL field ([ipfs/go-unixfs#72](https://github.com/ipfs/go-unixfs/pull/72))\n- github.com/ipfs/interface-go-ipfs-core (v0.0.6 -> v0.0.8):\n - switch to base32 cidv1 by default ([ipfs/interface-go-ipfs-core#29](https://github.com/ipfs/interface-go-ipfs-core/pull/29))\n - path: drop error from ParsePath ([ipfs/interface-go-ipfs-core#22](https://github.com/ipfs/interface-go-ipfs-core/pull/22))\n - tests: fix a bunch of small test lints/issues ([ipfs/interface-go-ipfs-core#28](https://github.com/ipfs/interface-go-ipfs-core/pull/28))\n - Update Pin.RmRecursive docs to clarify shared indirect pins are not removed ([ipfs/interface-go-ipfs-core#26](https://github.com/ipfs/interface-go-ipfs-core/pull/26))\n- github.com/libp2p/go-buffer-pool (v0.0.1 -> v0.0.2):\n - feat: add buffered writer ([libp2p/go-buffer-pool#9](https://github.com/libp2p/go-buffer-pool/pull/9))\n- github.com/libp2p/go-conn-security-multistream (v0.0.1 -> v0.0.2):\n - block while writing ([libp2p/go-conn-security-multistream#10](https://github.com/libp2p/go-conn-security-multistream/pull/10))\n- github.com/libp2p/go-libp2p (v0.0.12 -> v0.0.28):\n - Close the connection manager ([libp2p/go-libp2p#639](https://github.com/libp2p/go-libp2p/pull/639))\n - Frequent Relay Advertisements ([libp2p/go-libp2p#637](https://github.com/libp2p/go-libp2p/pull/637))\n - ping: return a stream of results ([libp2p/go-libp2p#626](https://github.com/libp2p/go-libp2p/pull/626))\n - Use cancelable background context in identify ([libp2p/go-libp2p#624](https://github.com/libp2p/go-libp2p/pull/624))\n - avoid intermediate allocation in relayAddrs ([libp2p/go-libp2p#609](https://github.com/libp2p/go-libp2p/pull/609))\n - cache relayAddrs for a short period of time ([libp2p/go-libp2p#608](https://github.com/libp2p/go-libp2p/pull/608))\n - autorelay: break findRelays into multiple functions and avoid the goto ([libp2p/go-libp2p#606](https://github.com/libp2p/go-libp2p/pull/606))\n - autorelay: curtail addrsplosion ([libp2p/go-libp2p#598](https://github.com/libp2p/go-libp2p/pull/598))\n - Periodically schedule identify push if the address set has changed ([libp2p/go-libp2p#597](https://github.com/libp2p/go-libp2p/pull/597))\n - Replace peer addresses in identify ([libp2p/go-libp2p#599](https://github.com/libp2p/go-libp2p/pull/599))\n- github.com/libp2p/go-libp2p-circuit (v0.0.4 -> v0.0.8):\n - call Stream.Reset instead of Stream.Close ([libp2p/go-libp2p-circuit#76](https://github.com/libp2p/go-libp2p-circuit/pull/76))\n - Tag the hop relay when creating stop streams ([libp2p/go-libp2p-circuit#77](https://github.com/libp2p/go-libp2p-circuit/pull/77))\n - Tag peers with live hop streams ([libp2p/go-libp2p-circuit#75](https://github.com/libp2p/go-libp2p-circuit/pull/75))\n - Hard Limit the number of hop stream goroutines ([libp2p/go-libp2p-circuit#74](https://github.com/libp2p/go-libp2p-circuit/pull/74))\n - set deadline for stop handshake ([libp2p/go-libp2p-circuit#73](https://github.com/libp2p/go-libp2p-circuit/pull/73))\n- github.com/libp2p/go-libp2p-connmgr (v0.0.1 -> v0.0.6):\n - Background trimming ([libp2p/go-libp2p-connmgr#43](https://github.com/libp2p/go-libp2p-connmgr/pull/43))\n - Implement UpsertTag ([libp2p/go-libp2p-connmgr#38](https://github.com/libp2p/go-libp2p-connmgr/pull/38))\n - Add peer protection capability (implementation) ([libp2p/go-libp2p-connmgr#36](https://github.com/libp2p/go-libp2p-connmgr/pull/36))\n- github.com/libp2p/go-libp2p-crypto (v0.0.1 -> v0.0.2):\n - add openssl support ([libp2p/go-libp2p-crypto#61](https://github.com/libp2p/go-libp2p-crypto/pull/61))\n- github.com/libp2p/go-libp2p-discovery (v0.0.1 -> v0.0.4):\n - More consistent use of options ([libp2p/go-libp2p-discovery#25](https://github.com/libp2p/go-libp2p-discovery/pull/25))\n - Use 3hrs as routing advertisement ttl ([libp2p/go-libp2p-discovery#23](https://github.com/libp2p/go-libp2p-discovery/pull/23))\n- github.com/libp2p/go-libp2p-interface-connmgr (v0.0.1 -> v0.0.5):\n - Add Close method to the ConnManager interface ([libp2p/go-libp2p-interface-connmgr#18](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/18))\n - Add UpsertTag to the interface ([libp2p/go-libp2p-interface-connmgr#17](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/17))\n - Fix NullConnMgr to respect ConnManager interface ([libp2p/go-libp2p-interface-connmgr#15](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/15))\n - Add peer protection capability ([libp2p/go-libp2p-interface-connmgr#14](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/14))\n- github.com/libp2p/go-libp2p-kad-dht (v0.0.7 -> v0.0.13):\n - fix: reduce memory used by buffered writers ([libp2p/go-libp2p-kad-dht#332](https://github.com/libp2p/go-libp2p-kad-dht/pull/332))\n - query: fix a goroutine leak when the routing table is empty ([libp2p/go-libp2p-kad-dht#329](https://github.com/libp2p/go-libp2p-kad-dht/pull/329))\n - query: fix error \"leak\" ([libp2p/go-libp2p-kad-dht#328](https://github.com/libp2p/go-libp2p-kad-dht/pull/328))\n - providers: run datastore GC concurrently ([libp2p/go-libp2p-kad-dht#326](https://github.com/libp2p/go-libp2p-kad-dht/pull/326))\n - fix(providers): gc ([libp2p/go-libp2p-kad-dht#325](https://github.com/libp2p/go-libp2p-kad-dht/pull/325))\n - Remove the old protocol from the defaults ([libp2p/go-libp2p-kad-dht#320](https://github.com/libp2p/go-libp2p-kad-dht/pull/320))\n - Fix some provider subsystem performance issues ([libp2p/go-libp2p-kad-dht#319](https://github.com/libp2p/go-libp2p-kad-dht/pull/319))\n- github.com/libp2p/go-libp2p-peerstore (v0.0.2 -> v0.0.6):\n - segment the memory peerstore + granular locks ([libp2p/go-libp2p-peerstore#78](https://github.com/libp2p/go-libp2p-peerstore/pull/78))\n - don't delete under the read lock ([libp2p/go-libp2p-peerstore#76](https://github.com/libp2p/go-libp2p-peerstore/pull/76))\n - Read/Write locking ([libp2p/go-libp2p-peerstore#74](https://github.com/libp2p/go-libp2p-peerstore/pull/74))\n - optimize peerstore memory ([libp2p/go-libp2p-peerstore#71](https://github.com/libp2p/go-libp2p-peerstore/pull/71))\n - fix unmarshalling of peer IDs ([libp2p/go-libp2p-peerstore#72](https://github.com/libp2p/go-libp2p-peerstore/pull/72))\n - fix error handling in UpdateAddrs: return on error ([libp2p/go-libp2p-peerstore#70](https://github.com/libp2p/go-libp2p-peerstore/pull/70))\n- github.com/libp2p/go-libp2p-pubsub (v0.0.1 -> v0.0.3):\n - rework validator pipeline ([libp2p/go-libp2p-pubsub#176](https://github.com/libp2p/go-libp2p-pubsub/pull/176))\n - Test adversarial signing ([libp2p/go-libp2p-pubsub#181](https://github.com/libp2p/go-libp2p-pubsub/pull/181))\n - Strict message signing by default ([libp2p/go-libp2p-pubsub#180](https://github.com/libp2p/go-libp2p-pubsub/pull/180))\n- github.com/libp2p/go-libp2p-secio (v0.0.1 -> v0.0.3):\n - fix buffer size check ([libp2p/go-libp2p-secio#44](https://github.com/libp2p/go-libp2p-secio/pull/44))\n- github.com/libp2p/go-libp2p-swarm (v0.0.2 -> v0.0.6):\n - dial: return a nice custom dial error ([libp2p/go-libp2p-swarm#121](https://github.com/libp2p/go-libp2p-swarm/pull/121))\n- github.com/libp2p/go-libp2p-tls (null -> v0.0.1):\n - implement the new handshake ([libp2p/go-libp2p-tls#20](https://github.com/libp2p/go-libp2p-tls/pull/20))\n - use a prefix when signing the public key ([libp2p/go-libp2p-tls#26](https://github.com/libp2p/go-libp2p-tls/pull/26))\n - use ChaCha if one of the peers doesn't have AES hardware support ([libp2p/go-libp2p-tls#23](https://github.com/libp2p/go-libp2p-tls/pull/23))\n - improve peer verification ([libp2p/go-libp2p-tls#17](https://github.com/libp2p/go-libp2p-tls/pull/17))\n - add an example (mainly for development) ([libp2p/go-libp2p-tls#14](https://github.com/libp2p/go-libp2p-tls/pull/14))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.0.1 -> v0.0.4):\n - improve correctness of closing connections on failure ([libp2p/go-libp2p-transport-upgrader#19](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/19))\n- github.com/libp2p/go-maddr-filter (v0.0.1 -> v0.0.4):\n - fix filter listing ([libp2p/go-maddr-filter#13](https://github.com/libp2p/go-maddr-filter/pull/13))\n - Reinstate deprecated Remove() method to reverse breakage ([libp2p/go-maddr-filter#12](https://github.com/libp2p/go-maddr-filter/pull/12))\n - Implement support for whitelists, default-deny/allow ([libp2p/go-maddr-filter#8](https://github.com/libp2p/go-maddr-filter/pull/8))\n- github.com/libp2p/go-mplex (v0.0.1 -> v0.0.4):\n - disable write coalescing ([libp2p/go-mplex#61](https://github.com/libp2p/go-mplex/pull/61))\n - fix SetDeadline error conditions ([libp2p/go-mplex#59](https://github.com/libp2p/go-mplex/pull/59))\n - don't use contexts for deadlines ([libp2p/go-mplex#58](https://github.com/libp2p/go-mplex/pull/58))\n - don't reset on pathologies, just ignore the data ([libp2p/go-mplex#57](https://github.com/libp2p/go-mplex/pull/57))\n - coalesce writes ([libp2p/go-mplex#54](https://github.com/libp2p/go-mplex/pull/54))\n - read as much as we can in one go ([libp2p/go-mplex#53](https://github.com/libp2p/go-mplex/pull/53))\n - use timeouts when sending messages for stream open, close, and reset. ([libp2p/go-mplex#52](https://github.com/libp2p/go-mplex/pull/52))\n - fix: reset a stream even if closed remotely ([libp2p/go-mplex#50](https://github.com/libp2p/go-mplex/pull/50))\n - downgrade Error log to Warning ([libp2p/go-mplex#46](https://github.com/libp2p/go-mplex/pull/46))\n - Fix race condition by adding a mutex for deadline access ([libp2p/go-mplex#41](https://github.com/libp2p/go-mplex/pull/41))\n- github.com/libp2p/go-msgio (v0.0.1 -> v0.0.2):\n - fix: never claim to read more than read ([libp2p/go-msgio#12](https://github.com/libp2p/go-msgio/pull/12))\n- github.com/libp2p/go-ws-transport (v0.0.2 -> v0.0.4):\n - dep: import go-smux-* into the libp2p org ([libp2p/go-ws-transport#43](https://github.com/libp2p/go-ws-transport/pull/43))\n - replace gx instructions with note about gomod ([libp2p/go-ws-transport#42](https://github.com/libp2p/go-ws-transport/pull/42))\n\n\n## v0.4.20 2019-04-16\n\nWe're happy to release go-ipfs 0.4.20. This release includes some critical\nperformance and stability fixes so all users should upgrade ASAP.\n\nThis is also the first release to use go modules instead of GX. While GX has\nbeen a great way to dogfood an IPFS-based package manager, building and\nmaintaining a custom package manager is a _lot_ of work and we haven't been able\nto dedicate enough time to bring the user experience of gx to an acceptable\nlevel. You can read [#5850](https://github.com/ipfs/go-ipfs/issues/5850) for\nsome discussion on this matter.\n\n### Docker\n\nAs of this release, it's now much easier to run arbitrary IPFS commands within\nthe docker container:\n\n```bash\n> docker run --name my-ipfs ipfs/go-ipfs:v0.4.20 config profile apply server # apply the server profile\n> docker start my-ipfs # start the daemon\n```\n\nThis release also [reverts](https://github.com/ipfs/go-ipfs/pull/6040) a change that\ncaused some significant trouble in 0.4.19. If you've been running into Docker\npermission errors in 0.4.19, please upgrade.\n\n### WebUI\n\nThis release contains a major\n[WebUI](https://github.com/ipfs-shipyard/ipfs-webui) release with some\nsignificant improvements to the file browser and new opt-in, privately hosted,\nanonymous usage analytics.\n\n### Commands\n\nAs usual, we've made several changes and improvements to our commands. The most\nnotable changes are listed in this section.\n\n#### New: `ipfs version deps`\n\nThis release includes a new command, `ipfs version deps`, to list all\ndependencies (with versions) of the current go-ipfs build. This should make it\neasy to tell exactly how go-ipfs was built when tracking down issues.\n\n#### New: `ipfs add URL`\n\nThe `ipfs add` command has gained support for URLs. This means you can:\n\n1. Add files with `ipfs add URL` instead of downloading the file first.\n2. Replace all uses of the `ipfs urlstore` command with a call to `ipfs add\n --nocopy`. The `ipfs urlstore` command will be deprecated in a future\n release.\n\n\n#### Changed: `ipfs swarm connect`\n\nThe `ipfs swarm connect` command has a few new features:\n\nIt now marks the newly created connection as \"important\". This should ensure\nthat the connection manager won't come along later and close the connection if\nit doesn't think it's being used.\n\nIt can now resolve `/dnsaddr` addresses that _don't_ end in a peer ID. For\nexample, you can now run `ipfs swarm connect /dnsaddr/bootstrap.libp2p.io` to\nconnect to one of the bootstrap peers at random. NOTE: This could connect you to\nan _arbitrary_ peer as DNS is not secure (by default). Please do not rely on\nthis except for testing or unless you know what you're doing.\n\nFinally, `ipfs swarm connect` now returns _all_ errors on failure. This should\nmake it much easier to debug connectivity issues. For example, one might see an\nerror like:\n\n```\nError: connect QmYou failure: dial attempt failed: 6 errors occurred:\n\t* --> (/ip4/127.0.0.1/tcp/4001) dial attempt failed: dial tcp4 127.0.0.1:4001: connect: connection refused\n\t* --> (/ip6/::1/tcp/4001) dial attempt failed: dial tcp6 [::1]:4001: connect: connection refused\n\t* --> (/ip6/2604::1/tcp/4001) dial attempt failed: dial tcp6 [2604::1]:4001: connect: network is unreachable\n\t* --> (/ip6/2602::1/tcp/4001) dial attempt failed: dial tcp6 [2602::1]:4001: connect: network is unreachable\n\t* --> (/ip4/150.0.1.2/tcp/4001) dial attempt failed: dial tcp4 0.0.0.0:4001->150.0.1.2:4001: i/o timeout\n\t* --> (/ip4/200.0.1.2/tcp/4001) dial attempt failed: dial tcp4 0.0.0.0:4001->200.0.1.2:4001: i/o timeout\n```\n\n#### Changed: `ipfs bitswap stat`\n\n`ipfs bitswap stat` no longer lists bitswap partners unless the `-v` flag is\npassed. That is, it will now return:\n\n```\n> ipfs bitswap stat\nbitswap status\n\tprovides buffer: 0 / 256\n\tblocks received: 0\n\tblocks sent: 79\n\tdata received: 0\n\tdata sent: 672706\n\tdup blocks received: 0\n\tdup data received: 0 B\n\twantlist [0 keys]\n\tpartners [197]\n```\n\nInstead of:\n\n```\n> ipfs bitswap stat -v\nbitswap status\n\tprovides buffer: 0 / 256\n\tblocks received: 0\n\tblocks sent: 79\n\tdata received: 0\n\tdata sent: 672706\n\tdup blocks received: 0\n\tdup data received: 0 B\n\twantlist [0 keys]\n\tpartners [203]\n\t\tQmNQTTTRCDpCYCiiu6TYWCqEa7ShAUo9jrZJvWngfSu1mL\n\t\tQmNWaxbqERvdcgoWpqAhDMrbK2gKi3SMGk3LUEvfcqZcf4\n\t\tQmNgSVpgZVEd41pBX6DyCaHRof8UmUJLqQ3XH2qNL9xLvN\n ... omitting 200 lines ...\n```\n\n#### Changed: `ipfs repo stat --human`\n\nThe `--human` flag in the `ipfs repo stat` command now intelligently picks a\nsize unit instead of always using MiB.\n\n#### Changed: `ipfs resolve` (`ipfs dns`, `ipfs name resolve`)\n\nAll of the resolve commands now:\n\n1. Resolve _recursively_ (up to 32 steps) by default to better match user\n expectations (these commands used to be non-recursive by default). To turn\n recursion off, pass `-r false`.\n2. When resolving non-recursively, these commands no longer fail when partially\n resolving a name. Instead, they simply return the intermediate result.\n\n#### Changed: `ipfs files flush`\n\nThe `ipfs files flush` command now returns the CID of the flushed file.\n\n### Performance And Reliability\n\nThis release has the usual collection of performance and reliability\nimprovements.\n\n#### Badger Memory Usage\n\nThose of you using the badger datastore should notice reduced memory usage in\nthis release due to some upstream changes. Badger still uses significantly more\nmemory than the default datastore configuration but this will hopefully continue\nto improve.\n\n#### Bitswap\n\nWe fixed some critical CPU utilization regressions in bitswap for this release.\nIf you've been noticing CPU _regressions_ in go-ipfs 0.4.19, especially when\nrunning a public gateway, upgrading to 0.4.20 will likely fix them.\n\n#### Relays\n\nAfter AutoRelay was introduced in go-ipfs 0.4.19, the number of peers connecting\nthrough relays skyrocketed to over 120K concurrent peers. This highlighted some\nperformance issues that we've now fixed in this release. Specifically:\n\n* We've significantly reduced the amount of memory allocated per-peer.\n* We've fixed a bug where relays might, in rare cases, try to actively dial a\n peer to relay traffic. By default, relays only forward traffic between peers\n already connected to the relay.\n* We've fixed quite a number of performance issues that only show up when\n rapidly forming new connections. This will actually help _all_ nodes but will\n especially help relays.\n\nIf you've enabled relay _hop_ (`Swarm.EnableRelayHop`) in go-ipfs 0.4.19 and it\nhasn't burned down your machine yet, this release should improve things\nsignificantly. However, relays are still under heavy load so running an open\nrelay will continue to be resource intensive.\n\nWe're continuing to investigate this issue and have a few more patches on the\nway that, unfortunately, won't make it into this release.\n\n#### Panics\n\nWe've fixed two notable panics in this release:\n\n* We've fixed a frequent panic in the DHT.\n* We've fixed an occasional panic in the experimental QUIC transport.\n\n### Content Routing\n\nIPFS announces and finds content by sending and retrieving content routing\n(\"provider\") records to and from the DHT. Unfortunately, sending out these\nrecords can be quite resource intensive.\n\nThis release has two changes to alleviate this: a reduced number of initial\nprovide workers and a persistent provider queue.\n\nWe've reduced the number of parallel initial provide workers (workers that send\nout provider records when content is initially added to go-ipfs) from 512 to 6.\nEach provide request (currently, due to some issues in our DHT) tries to\nestablish hundreds of connections, significantly impacting the performance of\ngo-ipfs and [crashing some\nrouters](https://github.com/ipfs/go-ipfs/issues/3320).\n\nWe've introduced a new persistent provider queue for files added via `ipfs add`\nand `ipfs pin add`. When new directory trees are added to go-ipfs, go-ipfs will\nadd the root/final CID to this queue. Then, in the background, go-ipfs will walk\nthe queue, sequentially sending out provider records for each CID.\n\nThis ensures that root CIDs are sent out as soon as possible and are sent even\nwhen files are added when the go-ipfs daemon isn't running.\n\nBy example, let's add a directory tree to go-ipfs:\n\n```bash\n> # We're going to do this in \"online\" mode first so let's start the daemon.\n> ipfs daemon &\n...\nDaemon is ready\n> # Now, we're going to create a directory to add.\n> mkdir foo\n> for i in {0..1000}; do echo do echo $i > foo/$i; done\n> # finally, we're going to add it.\n> ipfs add -r foo\nadded QmUQcSjQx2bg4cSe2rUZyQi6F8QtJFJb74fWL7D784UWf9 foo/0\n...\nadded QmQac2chFyJ24yfG2Dfuqg1P5gipLcgUDuiuYkQ5ExwGap foo/990\nadded QmQWwz9haeQ5T2QmQeXzqspKdowzYELShBCLzLJjVa2DuV foo/991\nadded QmQ5D4MtHUN4LTS4n7mgyHyaUukieMMyCfvnzXQAAbgTJm foo/992\nadded QmZq4n4KRNq3k1ovzxJ4qdQXZSrarfJjnoLYPR3ztHd7EY foo/993\nadded QmdtrsuVf8Nf1s1MaSjLAd54iNqrn1KN9VoFNgKGnLgjbt foo/994\nadded QmbstvU9mnW2hsE94WFmw5WbrXdLTu2Sf9kWWSozrSDscL foo/995\nadded QmXFd7f35gAnmisjfFmfYKkjA3F3TSpvUYB9SXr6tLsdg8 foo/996\nadded QmV5BxS1YQ9V227Np2Cq124cRrFDAyBXNMqHHa6kpJ9cr6 foo/997\nadded QmcXsccUtwKeQ1SuYC3YgyFUeYmAR9CXwGGnT3LPeCg5Tx foo/998\nadded Qmc4mcQcpaNzyDQxQj5SyxwFg9ZYz5XBEeEZAuH4cQirj9 foo/999\nadded QmXpXzUhcS9edmFBuVafV5wFXKjfXkCQcjAUZsTs7qFf3G foo\n```\n\nIn 0.4.19, we would have sent out provider records for files `foo/{0..1000}`\n_before_ sending out a provider record for `foo`. If you were ask a friend to\ndownload /ipfs/QmUQcSjQx2bg4cSe2rUZyQi6F8QtJFJb74fWL7D784UWf9, they would\n(baring other issues) be able to find it pretty quickly as this is the first CID\nyou'll have announced to the network. However, if you ask your friend to\ndownload /ipfs/QmXpXzUhcS9edmFBuVafV5wFXKjfXkCQcjAUZsTs7qFf3G/0, they'll have to\nwait for you to finish telling the network about every file in `foo` first.\n\nIn 0.4.20, we _immediately_ tell the network about\n`QmXpXzUhcS9edmFBuVafV5wFXKjfXkCQcjAUZsTs7qFf3G` (the `foo` directory) as soon\nas we finish adding the directory to go-ipfs _without_ waiting to finish\nannouncing `foo/{0..1000}`. This is especially important in this release\nbecause we've drastically reduced the number of provide workers.\n\nThe second benefit is that this queue is persistent. That means go-ipfs won't\nforget to send out this record, even if it was offline when the content was\ninitially added. NOTE: go-ipfs _does_ continuously _re_-send provider records in\nthe background twice a day, it just might be a while before it gets around to\nsending one out any specific one.\n\n### Bitswap\n\nBitswap now periodically re-sends its wantlist to connected peers. This should\nhelp work around some race conditions we've seen in bitswap where one node wants\na block but the other doesn't know for some reason.\n\nYou can track this issue here: https://github.com/ipfs/go-ipfs/issues/5183.\n\n### Improved NAT Traversal\n\nWhile NATs are still p2p enemy #1, this release includes slightly improved\nsupport for traversing them.\n\nSpecifically, this release now:\n\n1. Better detects the \"gateway\" NAT, even when multiple devices on the network\n _claim_ to be NATs.\n2. Better guesses the external IP address when port mapping, even when the\n gateway lies.\n\n### Reduced AutoRelay Boot Time\n\nThe experimental AutoRelay feature can now detect NATs _much_ faster as we've\nreduced initial NAT detection delay to 15 seconds. There's still room for\nimprovement but this should make nodes that have enabled this feature dialable\nearlier on start.\n\n### Changelogs\n\n- github.com/ipfs/go-ipfs:\n - gitattributes: avoid normalizing known binary files ([ipfs/go-ipfs#6209](https://github.com/ipfs/go-ipfs/pull/6209))\n - gitattributes: default to LF ([ipfs/go-ipfs#6198](https://github.com/ipfs/go-ipfs/pull/6198))\n - Fix level db panic ([ipfs/go-ipfs#6186](https://github.com/ipfs/go-ipfs/pull/6186))\n - Dockerfile: Remove 2 year old deprecation warning ([ipfs/go-ipfs#6188](https://github.com/ipfs/go-ipfs/pull/6188))\n - align output for the command ipfs object stat ([ipfs/go-ipfs#6189](https://github.com/ipfs/go-ipfs/pull/6189))\n - provider queue: don't repeatedly retry the same item if we fail ([ipfs/go-ipfs#6187](https://github.com/ipfs/go-ipfs/pull/6187))\n - test: remove version/deps from ro commands test ([ipfs/go-ipfs#6185](https://github.com/ipfs/go-ipfs/pull/6185))\n - feat: add version deps command [modversion] ([ipfs/go-ipfs#6115](https://github.com/ipfs/go-ipfs/pull/6115))\n - readme: update for go modules ([ipfs/go-ipfs#6180](https://github.com/ipfs/go-ipfs/pull/6180))\n - Switch to Go 1.12 ([ipfs/go-ipfs#6144](https://github.com/ipfs/go-ipfs/pull/6144))\n - ci: avoid interleaving output from different sharness tests ([ipfs/go-ipfs#6175](https://github.com/ipfs/go-ipfs/pull/6175))\n - fix two bugs where the repo may not properly be closed ([ipfs/go-ipfs#6176](https://github.com/ipfs/go-ipfs/pull/6176))\n - fix error check in swarm connect ([ipfs/go-ipfs#6174](https://github.com/ipfs/go-ipfs/pull/6174))\n - feat(coreapi): tag all explicit connect requests in the connection manager ([ipfs/go-ipfs#6171](https://github.com/ipfs/go-ipfs/pull/6171))\n - chore: remove CODEOWNERS ([ipfs/go-ipfs#6172](https://github.com/ipfs/go-ipfs/pull/6172))\n - feat: update to IPFS Web UI 2.4.4 ([ipfs/go-ipfs#6169](https://github.com/ipfs/go-ipfs/pull/6169))\n - fix add error handling ([ipfs/go-ipfs#6156](https://github.com/ipfs/go-ipfs/pull/6156))\n - chore: remove waffle ([ipfs/go-ipfs#6157](https://github.com/ipfs/go-ipfs/pull/6157))\n - chore: fix a bunch of issues caught by golangci-lint ([ipfs/go-ipfs#6140](https://github.com/ipfs/go-ipfs/pull/6140))\n - docs/experimental-features.md: link to ipfs-ds-convert ([ipfs/go-ipfs#6154](https://github.com/ipfs/go-ipfs/pull/6154))\n - interrupt: fix send on closed ([ipfs/go-ipfs#6147](https://github.com/ipfs/go-ipfs/pull/6147))\n - docs: document Gateway.Writable not Gateway.Writeable ([ipfs/go-ipfs#6151](https://github.com/ipfs/go-ipfs/pull/6151))\n - Fuse fixes ([ipfs/go-ipfs#6135](https://github.com/ipfs/go-ipfs/pull/6135))\n - Remove duplicate blockstore from the package list ([ipfs/go-ipfs#6138](https://github.com/ipfs/go-ipfs/pull/6138))\n - Query for provider head/tail ([ipfs/go-ipfs#6125](https://github.com/ipfs/go-ipfs/pull/6125))\n - Remove dead link from ISSUE_TEMPLATE.md ([ipfs/go-ipfs#6128](https://github.com/ipfs/go-ipfs/pull/6128))\n - coreapi: remove Unixfs.Wrap ([ipfs/go-ipfs#6123](https://github.com/ipfs/go-ipfs/pull/6123))\n - coreapi unixfs: change Wrap logic to make more sense ([ipfs/go-ipfs#6019](https://github.com/ipfs/go-ipfs/pull/6019))\n - deps: switch back to jbenet go-is-domain ([ipfs/go-ipfs#6119](https://github.com/ipfs/go-ipfs/pull/6119))\n - command repo stat: add human flag tests to t0080-repo.sh ([ipfs/go-ipfs#6116](https://github.com/ipfs/go-ipfs/pull/6116))\n - gc: fix a potential deadlock ([ipfs/go-ipfs#6112](https://github.com/ipfs/go-ipfs/pull/6112))\n - fix config options in osxfuse error messages ([ipfs/go-ipfs#6105](https://github.com/ipfs/go-ipfs/pull/6105))\n - Command repo stat: improve human flag behavior ([ipfs/go-ipfs#6106](https://github.com/ipfs/go-ipfs/pull/6106))\n - Provide root node immediately on add and pin add ([ipfs/go-ipfs#6068](https://github.com/ipfs/go-ipfs/pull/6068))\n - gomod: Update Dockerfile, remove Dockerfile.fast ([ipfs/go-ipfs#6100](https://github.com/ipfs/go-ipfs/pull/6100))\n - Return CID from 'ipfs files flush' ([ipfs/go-ipfs#6102](https://github.com/ipfs/go-ipfs/pull/6102))\n - resolve: fix recursion ([ipfs/go-ipfs#6087](https://github.com/ipfs/go-ipfs/pull/6087))\n - fix(swarm): add dnsaddr support in swarm connect ([ipfs/go-ipfs#5535](https://github.com/ipfs/go-ipfs/pull/5535))\n - make in-memory datastore thread-safe ([ipfs/go-ipfs#6085](https://github.com/ipfs/go-ipfs/pull/6085))\n - Update package table to remove broken jenkins links ([ipfs/go-ipfs#6084](https://github.com/ipfs/go-ipfs/pull/6084))\n - mk: fix maketarball to work with gomod ([ipfs/go-ipfs#6078](https://github.com/ipfs/go-ipfs/pull/6078))\n - fix ls command to use the new coreinterface types ([ipfs/go-ipfs#6051](https://github.com/ipfs/go-ipfs/pull/6051))\n - mk: remove install_unsupported, leave a note ([ipfs/go-ipfs#6063](https://github.com/ipfs/go-ipfs/pull/6063))\n - mk: change git-hash command to include information about modifications ([ipfs/go-ipfs#6060](https://github.com/ipfs/go-ipfs/pull/6060))\n - mk: fix make install by not setting GOBIN ([ipfs/go-ipfs#6059](https://github.com/ipfs/go-ipfs/pull/6059))\n - go: require Golang 1.11.4 ([ipfs/go-ipfs#6057](https://github.com/ipfs/go-ipfs/pull/6057))\n - yamux: increase yamux window size to 8MiB. ([ipfs/go-ipfs#6049](https://github.com/ipfs/go-ipfs/pull/6049))\n - Introduce go modules [yey] ([ipfs/go-ipfs#6038](https://github.com/ipfs/go-ipfs/pull/6038))\n - cleanup daemon online logic ([ipfs/go-ipfs#6050](https://github.com/ipfs/go-ipfs/pull/6050))\n - ci: test on 32bit os ([ipfs/go-ipfs#5429](https://github.com/ipfs/go-ipfs/pull/5429))\n - feat/cmds: hide peers info default in bitswap stat ([ipfs/go-ipfs#5820](https://github.com/ipfs/go-ipfs/pull/5820))\n - Improve CLI help pages ([ipfs/go-ipfs#6013](https://github.com/ipfs/go-ipfs/pull/6013))\n - Close #6044 ([ipfs/go-ipfs#6045](https://github.com/ipfs/go-ipfs/pull/6045))\n - commands(dht): return final error ([ipfs/go-ipfs#6034](https://github.com/ipfs/go-ipfs/pull/6034))\n - Revert \"Really run as non-root user in docker container\" ([ipfs/go-ipfs#6040](https://github.com/ipfs/go-ipfs/pull/6040))\n- github.com/ipfs/go-bitswap:\n - feat(messagequeue): rebroadcast wantlist ([ipfs/go-bitswap#106](https://github.com/ipfs/go-bitswap/pull/106))\n - reduce provide workers to 6 ([ipfs/go-bitswap#93](https://github.com/ipfs/go-bitswap/pull/93))\n - Reduce memory allocation ([ipfs/go-bitswap#103](https://github.com/ipfs/go-bitswap/pull/103))\n - refactor(messagequeue): remove dead code ([ipfs/go-bitswap#98](https://github.com/ipfs/go-bitswap/pull/98))\n - fix: limit use of custom context type ([ipfs/go-bitswap#89](https://github.com/ipfs/go-bitswap/pull/89))\n - fix: remove non-error log message ([ipfs/go-bitswap#91](https://github.com/ipfs/go-bitswap/pull/91))\n - fix(messagequeue): Remove second run loop ([ipfs/go-bitswap#94](https://github.com/ipfs/go-bitswap/pull/94))\n- github.com/ipfs/go-blockservice:\n - Revert \"Remove verifcid as it is handled in go-cid\" ([ipfs/go-blockservice#25](https://github.com/ipfs/go-blockservice/pull/25))\n - Remove verifcid as it is handled in go-cid ([ipfs/go-blockservice#23](https://github.com/ipfs/go-blockservice/pull/23))\n- github.com/ipfs/go-datastore:\n - cleanup and optimize naive query filters ([ipfs/go-datastore#125](https://github.com/ipfs/go-datastore/pull/125))\n - Fix – sorted limited offset mount queries ([ipfs/go-datastore#124](https://github.com/ipfs/go-datastore/pull/124))\n - Fix function comments based on best practices from Effective Go ([ipfs/go-datastore#122](https://github.com/ipfs/go-datastore/pull/122))\n - remove ThreadSafeDatastore ([ipfs/go-datastore#120](https://github.com/ipfs/go-datastore/pull/120))\n - Splinter TTLDatastore interface into TTL + Datastore ([ipfs/go-datastore#118](https://github.com/ipfs/go-datastore/pull/118))\n- github.com/ipfs/go-ds-badger:\n - tweak the default options ([ipfs/go-ds-badger#52](https://github.com/ipfs/go-ds-badger/pull/52))\n - remove thread-safe assertion ([ipfs/go-ds-badger#55](https://github.com/ipfs/go-ds-badger/pull/55))\n - make memory-safe against concurrent closure/operations ([ipfs/go-ds-badger#53](https://github.com/ipfs/go-ds-badger/pull/53))\n - make badger use our logging framework ([ipfs/go-ds-badger#50](https://github.com/ipfs/go-ds-badger/pull/50))\n- github.com/ipfs/go-ds-flatfs:\n - remove thread-safe assertion ([ipfs/go-ds-flatfs#53](https://github.com/ipfs/go-ds-flatfs/pull/53))\n- github.com/ipfs/go-ds-leveldb:\n - Fast reverse query ([ipfs/go-ds-leveldb#28](https://github.com/ipfs/go-ds-leveldb/pull/28))\n - remove thread-safe assertion ([ipfs/go-ds-leveldb#27](https://github.com/ipfs/go-ds-leveldb/pull/27))\n- github.com/ipfs/go-ipfs-cmdkit:\n - Extract files package ([ipfs/go-ipfs-cmdkit#31](https://github.com/ipfs/go-ipfs-cmdkit/pull/31))\n- github.com/ipfs/go-ipfs-cmds:\n - sync: add yet another sync error ([ipfs/go-ipfs-cmds#161](https://github.com/ipfs/go-ipfs-cmds/pull/161))\n - Removed broken link from readme ([ipfs/go-ipfs-cmds#159](https://github.com/ipfs/go-ipfs-cmds/pull/159))\n - Fix broken link in readme ([ipfs/go-ipfs-cmds#160](https://github.com/ipfs/go-ipfs-cmds/pull/160))\n - set WebFile fpath to URL base ([ipfs/go-ipfs-cmds#158](https://github.com/ipfs/go-ipfs-cmds/pull/158))\n - Handle stdin name in cli/parse ([ipfs/go-ipfs-cmds#157](https://github.com/ipfs/go-ipfs-cmds/pull/157))\n - support url paths as files.WebFile ([ipfs/go-ipfs-cmds#154](https://github.com/ipfs/go-ipfs-cmds/pull/154))\n - typed encoder: improve pointer reflection ([ipfs/go-ipfs-cmds#155](https://github.com/ipfs/go-ipfs-cmds/pull/155))\n - cli: don't sync output to NUL on Windows ([ipfs/go-ipfs-cmds#153](https://github.com/ipfs/go-ipfs-cmds/pull/153))\n- github.com/ipfs/go-ipfs-files:\n - return url as AbsPath from WebFile to implement FileInfo ([ipfs/go-ipfs-files#13](https://github.com/ipfs/go-ipfs-files/pull/13))\n - fix the content disposition header ([ipfs/go-ipfs-files#14](https://github.com/ipfs/go-ipfs-files/pull/14))\n - go format ([ipfs/go-ipfs-files#15](https://github.com/ipfs/go-ipfs-files/pull/15))\n - simplify content type checking ([ipfs/go-ipfs-files#9](https://github.com/ipfs/go-ipfs-files/pull/9))\n - remove extra webfile test code ([ipfs/go-ipfs-files#12](https://github.com/ipfs/go-ipfs-files/pull/12))\n- github.com/ipfs/go-merkledag:\n - add function to marshal raw nodes to json ([ipfs/go-merkledag#36](https://github.com/ipfs/go-merkledag/pull/36))\n - fix some performance regressions when reading protobuf nodes ([ipfs/go-merkledag#34](https://github.com/ipfs/go-merkledag/pull/34))\n- github.com/ipfs/go-metrics-interface:\n - update the counter interface to match Prometheus ([ipfs/go-metrics-interface#2](https://github.com/ipfs/go-metrics-interface/pull/2))\n- github.com/ipfs/go-mfs:\n - Return node from FlushPath ([ipfs/go-mfs#72](https://github.com/ipfs/go-mfs/pull/72))\n - Wire up context to FlushPath ([ipfs/go-mfs#70](https://github.com/ipfs/go-mfs/pull/70))\n- github.com/ipfs/interface-go-ipfs-core:\n - don't close the top-level addr ([ipfs/interface-go-ipfs-core#25](https://github.com/ipfs/interface-go-ipfs-core/pull/25))\n - fix a bunch of small test \"bugs\" ([ipfs/interface-go-ipfs-core#24](https://github.com/ipfs/interface-go-ipfs-core/pull/24))\n - remove Wrap ([ipfs/interface-go-ipfs-core#21](https://github.com/ipfs/interface-go-ipfs-core/pull/21))\n - Unixfs.Wrap Fixes ([ipfs/interface-go-ipfs-core#10](https://github.com/ipfs/interface-go-ipfs-core/pull/10))\n - tweak the Ls interface ([ipfs/interface-go-ipfs-core#14](https://github.com/ipfs/interface-go-ipfs-core/pull/14))\n- github.com/libp2p/go-buffer-pool:\n - Enable tests ([libp2p/go-buffer-pool#6](https://github.com/libp2p/go-buffer-pool/pull/6))\n- github.com/libp2p/go-flow-metrics:\n - Just repair spelling mistake ([libp2p/go-flow-metrics#3](https://github.com/libp2p/go-flow-metrics/pull/3))\n- github.com/libp2p/go-libp2p:\n - Deprecate gx in readme & link to workspace repo ([libp2p/go-libp2p#591](https://github.com/libp2p/go-libp2p/pull/591))\n - Respect nodial option in routed host ([libp2p/go-libp2p#590](https://github.com/libp2p/go-libp2p/pull/590))\n - fix panic in observed address activation check ([libp2p/go-libp2p#586](https://github.com/libp2p/go-libp2p/pull/586))\n - Improve observed address handling ([libp2p/go-libp2p#585](https://github.com/libp2p/go-libp2p/pull/585))\n - identify: avoid parsing/printing multiaddrs ([libp2p/go-libp2p#583](https://github.com/libp2p/go-libp2p/pull/583))\n - move things outside of the lock in obsaddr ([libp2p/go-libp2p#582](https://github.com/libp2p/go-libp2p/pull/582))\n - identify: be more careful about the addresses we store ([libp2p/go-libp2p#577](https://github.com/libp2p/go-libp2p/pull/577))\n - relay: turn autorelay into a service and always filter out relay addresses ([libp2p/go-libp2p#578](https://github.com/libp2p/go-libp2p/pull/578))\n - chore: fail in the libp2p constructor if we fail to store the key ([libp2p/go-libp2p#576](https://github.com/libp2p/go-libp2p/pull/576))\n - Fix broken link in README.md ([libp2p/go-libp2p#580](https://github.com/libp2p/go-libp2p/pull/580))\n - Link to docs & discuss in readme ([libp2p/go-libp2p#571](https://github.com/libp2p/go-libp2p/pull/571))\n - Reduce autorelay boot delay and correctly handle private->public transition ([libp2p/go-libp2p#570](https://github.com/libp2p/go-libp2p/pull/570))\n - reduce nat error level ([libp2p/go-libp2p#568](https://github.com/libp2p/go-libp2p/pull/568))\n - relay: simplify declaration of multiaddr var ([libp2p/go-libp2p#563](https://github.com/libp2p/go-libp2p/pull/563))\n - Fix UDP listen on a Unspecified Address and Dial from the Unspecified Address ([libp2p/go-libp2p#561](https://github.com/libp2p/go-libp2p/pull/561))\n - Remove jenkins column from package table ([libp2p/go-libp2p#562](https://github.com/libp2p/go-libp2p/pull/562))\n - Fix typos in p2p/net/README.md ([libp2p/go-libp2p#555](https://github.com/libp2p/go-libp2p/pull/555))\n - better nat mapping ([libp2p/go-libp2p#549](https://github.com/libp2p/go-libp2p/pull/549))\n- github.com/libp2p/go-libp2p-autonat:\n - fully close the autonat client stream ([libp2p/go-libp2p-autonat#21](https://github.com/libp2p/go-libp2p-autonat/pull/21))\n - parallelize dialbacks ([libp2p/go-libp2p-autonat#20](https://github.com/libp2p/go-libp2p-autonat/pull/20))\n - Pacify the race detector ([libp2p/go-libp2p-autonat#17](https://github.com/libp2p/go-libp2p-autonat/pull/17))\n- github.com/libp2p/go-libp2p-autonat-svc:\n - full close the autonat stream ([libp2p/go-libp2p-autonat-svc#20](https://github.com/libp2p/go-libp2p-autonat-svc/pull/20))\n - reduce dialback timeout to 15s ([libp2p/go-libp2p-autonat-svc#17](https://github.com/libp2p/go-libp2p-autonat-svc/pull/17))\n- github.com/libp2p/go-libp2p-circuit:\n - use buffer pool in newDelimitedReader ([libp2p/go-libp2p-circuit#71](https://github.com/libp2p/go-libp2p-circuit/pull/71))\n - Use NoDial option when opening hop streams for non-active relays ([libp2p/go-libp2p-circuit#70](https://github.com/libp2p/go-libp2p-circuit/pull/70))\n - use io.CopyBuffer with explicitly allocated buffers ([libp2p/go-libp2p-circuit#69](https://github.com/libp2p/go-libp2p-circuit/pull/69))\n - docs and nits ([libp2p/go-libp2p-circuit#66](https://github.com/libp2p/go-libp2p-circuit/pull/66))\n- github.com/libp2p/go-libp2p-kad-dht:\n - dialQueue: start the control loop later ([libp2p/go-libp2p-kad-dht#312](https://github.com/libp2p/go-libp2p-kad-dht/pull/312))\n - make it work in wasm ([libp2p/go-libp2p-kad-dht#310](https://github.com/libp2p/go-libp2p-kad-dht/pull/310))\n - Revert \"GoModules: Checksum mismatch:\" ([libp2p/go-libp2p-kad-dht#309](https://github.com/libp2p/go-libp2p-kad-dht/pull/309))\n - defer dialqueue action until initial peers have been added ([libp2p/go-libp2p-kad-dht#301](https://github.com/libp2p/go-libp2p-kad-dht/pull/301))\n- github.com/libp2p/go-libp2p-nat:\n - switch to libp2p's go-nat fork ([libp2p/go-libp2p-nat#16](https://github.com/libp2p/go-libp2p-nat/pull/16))\n - remove all uses of multiaddrs ([libp2p/go-libp2p-nat#14](https://github.com/libp2p/go-libp2p-nat/pull/14))\n- github.com/libp2p/go-libp2p-net:\n - fix WithNoDial to return the context ([libp2p/go-libp2p-net#43](https://github.com/libp2p/go-libp2p-net/pull/43))\n - NoDial context option ([libp2p/go-libp2p-net#42](https://github.com/libp2p/go-libp2p-net/pull/42))\n- github.com/libp2p/go-libp2p-peer:\n - Let ID implement encoding.Binary[Un]Marshaler and encoding.Text[Un]Marshaler ([libp2p/go-libp2p-peer#44](https://github.com/libp2p/go-libp2p-peer/pull/44))\n- github.com/libp2p/go-libp2p-peerstore:\n - keep temp addresses for 2 minutes ([libp2p/go-libp2p-peerstore#67](https://github.com/libp2p/go-libp2p-peerstore/pull/67))\n - migrate to multiformats/go-base32 ([libp2p/go-libp2p-peerstore#61](https://github.com/libp2p/go-libp2p-peerstore/pull/61))\n- github.com/libp2p/go-libp2p-protocol:\n - update readme ([libp2p/go-libp2p-protocol#6](https://github.com/libp2p/go-libp2p-protocol/pull/6))\n - Enable standard Travis CI tests. ([libp2p/go-libp2p-protocol#5](https://github.com/libp2p/go-libp2p-protocol/pull/5))\n - Fix go get address. ([libp2p/go-libp2p-protocol#4](https://github.com/libp2p/go-libp2p-protocol/pull/4))\n - Add MIT license ([libp2p/go-libp2p-protocol#3](https://github.com/libp2p/go-libp2p-protocol/pull/3))\n - Standardized Readme ([libp2p/go-libp2p-protocol#2](https://github.com/libp2p/go-libp2p-protocol/pull/2))\n- github.com/libp2p/go-libp2p-pubsub-router:\n - gx publish 0.5.17 ([libp2p/go-libp2p-pubsub-router#26](https://github.com/libp2p/go-libp2p-pubsub-router/pull/26))\n- github.com/libp2p/go-libp2p-quic-transport:\n - update quic-go to v0.11.0 ([libp2p/go-libp2p-quic-transport#54](https://github.com/libp2p/go-libp2p-quic-transport/pull/54))\n- github.com/libp2p/go-libp2p-routing-helpers:\n - fix(put): fail if any router fails ([libp2p/go-libp2p-routing-helpers#19](https://github.com/libp2p/go-libp2p-routing-helpers/pull/19))\n- github.com/libp2p/go-libp2p-swarm:\n - Add context option to disable dialing when opening a new stream ([libp2p/go-libp2p-swarm#116](https://github.com/libp2p/go-libp2p-swarm/pull/116))\n - return all dial errors if dial has failed ([libp2p/go-libp2p-swarm#115](https://github.com/libp2p/go-libp2p-swarm/pull/115))\n - Differentiate no addresses error from no good addresses ([libp2p/go-libp2p-swarm#113](https://github.com/libp2p/go-libp2p-swarm/pull/113))\n- github.com/libp2p/go-libp2p-transport:\n - tests: constrain concurrency with race detector. ([libp2p/go-libp2p-transport#47](https://github.com/libp2p/go-libp2p-transport/pull/47))\n - pick test timeout from env var if available. ([libp2p/go-libp2p-transport#46](https://github.com/libp2p/go-libp2p-transport/pull/46))\n - increase test timeout. ([libp2p/go-libp2p-transport#45](https://github.com/libp2p/go-libp2p-transport/pull/45))\n- github.com/libp2p/go-msgio:\n - Improve test coverage ([libp2p/go-msgio#10](https://github.com/libp2p/go-msgio/pull/10))\n- github.com/libp2p/go-reuseport:\n - fix: add wasm build tag to wasm module ([libp2p/go-reuseport#70](https://github.com/libp2p/go-reuseport/pull/70))\n- github.com/libp2p/go-reuseport-transport:\n - don't set linger to 0 ([libp2p/go-reuseport-transport#14](https://github.com/libp2p/go-reuseport-transport/pull/14))\n- github.com/libp2p/go-tcp-transport:\n - set linger to 0 for both inbound and outbound connections ([libp2p/go-tcp-transport#36](https://github.com/libp2p/go-tcp-transport/pull/36))\n- github.com/libp2p/go-ws-transport:\n - modernize request handling ([libp2p/go-ws-transport#41](https://github.com/libp2p/go-ws-transport/pull/41))\n\n## v0.4.19 2019-03-01\n\nWe're happy to announce go 0.4.19. This release contains a bunch of important\nfixes and a slew of new and improved features. Get pumped and upgrade ASAP to benefit from all the new goodies! 🎁\n\n### Features\n\n#### 🔌 Initializing With Random Ports\n\nGo-ipfs can now be configured to listen on a random but _stable_ port (across\nrestarts) using the new `randomports` configuration profile. This should be\nhelpful when testing and/or running multiple go-ipfs instances on a single\nmachine.\n\nTo initialize a go-ipfs instance with a randomly chosen port, run:\n\n```bash\n> ipfs init --profile=randomports\n```\n\n#### 👂 Gateway Directory Listing\n\nIPNS (and/or DNSLink) directory listings on the gateway, e.g.\nhttps://ipfs.io/ipns/dist.ipfs.tech/go-ipfs/, will now display the _ipfs_ hash of\nthe current directory. This way users can more easily create permanent links to\notherwise mutable data.\n\n#### 📡 AutoRelay and AutoNAT\n\nThis release introduces two new experimental features (courtesy of libp2p):\nAutoRelay and AutoNAT.\n\nAutoRelay is a new service that automatically chooses a public relay when it\ndetects that the go-ipfs node is behind a NAT. While relaying connections\nthrough a third-party node isn't the most efficient way to route around NATs,\nit's a reliable fallback.\n\nTo enable AutoRelay, set the `Swarm.EnableAutoRelay` option in the config.\n\nAutoNAT is the service AutoRelay uses to detect if the node is behind a NAT. You\ndon't have to set any special config flags to enable it.\n\nIn this same config section, you may also notice options like `EnableRelayHop`,\n`EnableAutoNATService`, etc. You _do not_ need to enable these:\n\n* `EnableRelayHop` -- Allow _other_ nodes to use _your_ node as a relay\n (disabled by default).\n* `EnableAutoNATService` -- Help _other_ nodes detect if they're behind a NAT\n (disabled by default).\n\n#### 📵 Offline Operation\n\nThere are two new \"offline\" features in this release: a global `--offline` flag\nand an option to configure the gateway to not fetch files.\n\nMost go-ipfs commands now support the `--offline` flag. This causes IPFS to avoid\nnetwork operations when performing the requested operation. If you've ever used\nthe `--local` flag, the `--offline` flag is the (almost) universally supported\nreplacement.\n\nFor example:\n\n* If the daemon is started with `ipfs daemon --offline`, it won't even _connect_\n to the network. (note: this feature isn't new, just an example).\n* `ipfs add --offline some_file` won't send out provider records.\n* `ipfs cat --offline Qm...` won't fetch any blocks from the network.\n* `ipfs block stat --offline Qm...` is a great way to tell if a block is locally\n available.\n\nNote: It doesn't _yet_ work with the `refs`, `urlstore`, or `tar` commands\n([#6002](https://github.com/ipfs/go-ipfs/issues/6002)).\n\nOn to the gateway, there's a new `Gateway.NoFetch` option to configure the\ngateway to only serve locally present files. This makes it possible to run an\nIPFS node as a gateway to serve content of _your_ choosing without acting like a\npublic proxy. 🤫\n\n#### 📍 Adding And Pinning Content\n\nThere's a new `--pin` flag for both `ipfs block put` and `ipfs urlstore add` to\nmatch the `--pin` flag in `ipfs add`. This allows one to atomically add and pin\ncontent with these APIs.\n\n**NOTE 1:** For `ipfs urlstore add`, `--pin` has been enabled _by default_ to\nmatch the behavior in `ipfs add`. However, `ipfs block put` _does not_ pin by\ndefault to match the _current_ behavior.\n\n**NOTE 2:** If you had previously used the urlstore and _weren't_ explicitly\npinning content after adding it, it isn't pinned and running the garbage\ncollector will delete it. While technically documented in the `ipfs urlstore\nadd` helptext, this behavior was non-obvious and bears mentioning.\n\n#### 🗂 File Listing\n\nThe `ipfs ls` command has two significant changes this release: it reports\n_file_ sizes instead of _dag_ sizes and has gained a new `--stream` flag.\n\nFirst up, `ipfs ls` now reports _file_ sizes instead of _dag_ sizes. Previously,\nfor historical reasons, `ipfs ls` would report the size of a file/directory as\nseen by IPFS _including_ all the filesystem datastructures and metadata.\nHowever, this meant that `ls -l` and `ipfs ls` would print _different_ sizes:\n\n```bash\n> ipfs ls /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\n\nQmZTR5bcpQD7cFgTorqxZDYaew1Wqgfbd2ud9QqGPAkK2V 1688 about\nQmYCvbfNbCwFR45HiNP45rwJgvatpiW38D961L5qAhUM5Y 200 contact\nQmY5heUM5qgRubMDD1og9fhCPA6QdkMp3QCwd4s7gJsyE7 322 help\nQmejvEPop4D7YUadeGqYWmZxHhLc4JBUCzJJHWMzdcMe2y 12 ping\nQmXgqKTbzdh83pQtKFb19SpMCpDDcKR2ujqk3pKph9aCNF 1692 quick-start\nQmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB 1102 readme\nQmQ5vhrL7uv6tuoN9KeVBwd4PwfQkXdVVmDLUZuTNxqgvm 1173 security-notes\n\n> ipfs get /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\nSaving file(s) to QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\n 6.39 KiB / 6.39 KiB [================================] 100.00% 0s\n\n> ls -l QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\ntotal 28\n-rw------- 1 user group 1677 Feb 14 17:03 about\n-rw------- 1 user group 189 Feb 14 17:03 contact\n-rw------- 1 user group 311 Feb 14 17:03 help\n-rw------- 1 user group 4 Feb 14 17:03 ping\n-rw------- 1 user group 1681 Feb 14 17:03 quick-start\n-rw------- 1 user group 1091 Feb 14 17:03 readme\n-rw------- 1 user group 1162 Feb 14 17:03 security-notes\n```\n\nThis is now no longer the case. `ipfs ls` and `ls -l` now return the _same_\nsizes. 🙌\n\nSecond up, `ipfs ls` now has a new `--stream` flag. In IPFS, very large\ndirectories (e.g., Wikipedia) are split up into multiple chunks (shards) as\nthere are too many entries to fit in a single block. Unfortunately, `ipfs ls`\nbuffers the _entire_ file list in memory and then sorts it. This means that\n`ipfs ls /ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki` (Wikipedia)\nwill take a _very_ long time to return anything (it'll also use quite a bit of\nmemory).\n\nHowever, the new `--stream` flag makes it possible to stream a directory listing\nas new chunks are fetched from the network. To test this, you can run `ipfs ls\n--stream --size=false --resolve-type=false\n/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki`. You probably won't\nwant to wait for that command to finish, Wikipedia has a _lot_ of entries. 😉\n\n#### 🔁 HTTP Proxy\n\nThis release sees a new (experimental) feature contributed by our friends at\n[Peergos](https://peergos.org): HTTP proxy over libp2p. When enabled, the local\ngateway can act as an HTTP proxy and forward HTTP requests to libp2p peers. When\ncombined with the `ipfs p2p` command, users can use this to expose HTTP services\nto other go-ipfs nodes via their gateways. For details, check out the\n[documentation](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#p2p-http-proxy).\n\n### Performance And Reliability\n\nThis release introduces quite a few performance/reliability improvements and, as\nusual, fixes several memory leaks. Below is a non-exhaustive list of noticeable changes.\n\n#### 📞 DHT\n\nThis release includes an important DHT fix that should significantly:\n\n1. Reduce dialing.\n2. Speed up DHT queries.\n3. Improve performance of the gateways.\n\nBasically, in the worst case, a DHT query would turn into a random walk of the\nentire IPFS network. Yikes!\n\nRelevant PR: https://github.com/libp2p/go-libp2p-kad-dht/pull/237\n\n#### 🕸 Bitswap\n\nBitswap sessions have improved and are now used for _all_ requests. Sessions\nallow us to group related content and ask peers most likely to _have_ the\ncontent instead of broadcasting the request to all connected peers. This gives\nus two significant benefits:\n\n1. Less wasted upload bandwidth. Instead of broadcasting which blocks we want to\n everyone, we can ask fewer peers thus reducing the number of requests we send\n out.\n2. Less wasted download bandwidth. Because we _know_ which peers likely have\n content, we can ask an individual peer for a block and expect to get an\n answer. In the past, we'd ask every peer at the same time to optimize for\n latency at the expense of bandwidth (getting the same block from multiple\n peers). We had to do this because we had to assume that _most_ peers didn't\n have the requested block.\n\n#### ‼️ Pubsub\n\nThis release includes some significant reliability improvements in pubsub\nsubscription handling. If you've previously had issues with connected pubsub\npeers _not_ seeing each-other's messages, please upgrade ASAP.\n\n#### ♻️ Reuseport\n\nIn this release, we've rewritten our previously error-prone `go-reuseport`\nlibrary to _not_ duplicate a significant portion of Go's low-level networking\ncode. This was made possible by Go's new `Control`\n[`net.Dialer`](https://golang.org/pkg/net/#Dialer) option.\n\nIn the past, our first suggestion to anyone experiencing weird resource or\nconnectivity issues was to disable `REUSEPORT` (set `IPFS_REUSEPORT` to false).\nThis should no longer be necessary.\n\n#### 🐺 Badger Datastore\n\n[Badger has reached 1.0][badger-release]. This release brings an audit and\nnumerous reliability fixes. We are now reasonably confident that badger will\nbecome the default datastore in a future release. 👍\n\n[badger-release]: https://blog.dgraph.io/post/releasing-v1.0/\n\nThis release also adds a new `Truncate` configuration option for the badger\ndatastore (enabled by default for new IPFS nodes). When enabled, badger will\n_delete_ any un-synced data on start instead of simply refusing to start. This\nshould be safe on all filesystems where the `sync` operation is safe and removes\nthe need for manual intervention when restarting an IPFS node after a crash.\n\nAssuming you initialized your badger repo with `ipfs init --profile=badgerds`,\nyou can enable truncate on an existing repo by running: `ipfs config --json\n\"Datastore.Spec.child.truncate\" true`.\n\n### Refactors and Endeavors\n\n#### 🕹 Commands Library\n\nThe legacy commands library shim has now been completely removed. This won't\nmean much for many users but the go-ipfs team is happy to have this behind them.\n\n#### 🌐 Base32 CIDs\n\nThis release can now encode CIDs in responses in bases other than base58. This\nis primarily useful for web-browser integration as it allows us to (a) encode\nCIDs in a lower-case base (e.g., base32) and then use them in the _origin_ part\nof URLs. The take away is: this release brings us a step closer to better\nbrowser integration.\n\nSpecifically, this release adds two flags:\n\n1. `--cid-base`: When specified, the IPFS CLI will encode all CIDv1 CIDs using the\n requested base.\n2. `--upgrade-cidv0-in-output`: When specified, the IPFS CLI will _upgrade_ CIDv0\n CIDs to CIDv1 CIDs when returning them to the user. This upgrade is necessary\n because CIDv0 doesn't support multibase however, it's off by default as it\n changes the _binary_ representation of the CIDs (which could have unintended\n consequences).\n\n#### 🎛 CoreAPI\n\nThe work on the CoreAPI refactor ([ipfs/go-ipfs#4498][]) has progressed leaps and\nbounds this release. The CoreAPI is a comprehensive programmatic interface\ndesigned to allow go-ipfs be used as a daemon or a library interchangeably.\n\nAs of this release, go-ipfs now has:\n\n* External interface definitions in [ipfs/interface-go-ipfs-core][].\n* A work-in-progress implementation ([ipfs/go-ipfs-http-client][]) of these\n interfaces that uses the IPFS HTTP API. This will replace the\n ([ipfs/go-ipfs-api][]) library.\n* A new plugin type [\"Daemon\"][daemon-plugin]. Daemon plugins are started and\n stopped along with the go-ipfs daemon and are instantiated with a copy of the\n CoreAPI. This allows them to control and extend the go-ipfs daemon from within\n the daemon itself.\n\nThe next steps are:\n\n1. Finishing the remaining API surface area. At the moment, the two key missing\n parts are:\n 1. Config manipulation.\n 2. The `ipfs files` API.\n1. Finalizing the [ipfs/go-ipfs-http-client][] implementation.\n2. Creating a simple way to construct and initialize a go-ipfs node when using\n go-ipfs as a library.\n\n[ipfs/go-ipfs#4498]: https://github.com/ipfs/go-ipfs/issues/4498\n[ipfs/interface-go-ipfs-core]: https://github.com/ipfs/interface-go-ipfs-core\n[ipfs/go-ipfs-http-client]: https://github.com/ipfs/go-ipfs-http-client\n[ipfs/go-ipfs-api]: https://github.com/ipfs/go-ipfs-http-client\n[daemon-plugin]: https://github.com/ipfs/go-ipfs/blob/master/docs/plugins.md#daemon\n\n### Changelogs\n\n- github.com/ipfs/go-ipfs:\n - fix: show interactive output from install.sh ([ipfs/go-ipfs#6024](https://github.com/ipfs/go-ipfs/pull/6024))\n - fix: return the shortest, completely resolved path in the resolve command ([ipfs/go-ipfs#5704](https://github.com/ipfs/go-ipfs/pull/5704))\n - fix a few interop test issues ([ipfs/go-ipfs#6004](https://github.com/ipfs/go-ipfs/pull/6004))\n - fix HAMT bookmark ln ([ipfs/go-ipfs#6005](https://github.com/ipfs/go-ipfs/pull/6005))\n - docs: document Gateway.NoFetch ([ipfs/go-ipfs#5999](https://github.com/ipfs/go-ipfs/pull/5999))\n - Improve \"name publish\" ttl option documentation ([ipfs/go-ipfs#5979](https://github.com/ipfs/go-ipfs/pull/5979))\n - fix(cmd/mv): dst filename error ([ipfs/go-ipfs#5964](https://github.com/ipfs/go-ipfs/pull/5964))\n - coreapi: extract interface ([ipfs/go-ipfs#5978](https://github.com/ipfs/go-ipfs/pull/5978))\n - coreapi: cleanup non-gx references ([ipfs/go-ipfs#5976](https://github.com/ipfs/go-ipfs/pull/5976))\n - coreapi: fix seek test on http impl ([ipfs/go-ipfs#5971](https://github.com/ipfs/go-ipfs/pull/5971))\n - block put --pin ([ipfs/go-ipfs#5969](https://github.com/ipfs/go-ipfs/pull/5969))\n - Port `ipfs ls` to CoreAPI ([ipfs/go-ipfs#5962](https://github.com/ipfs/go-ipfs/pull/5962))\n - docs: duplicate default helptext in `name publish` ([ipfs/go-ipfs#5960](https://github.com/ipfs/go-ipfs/pull/5960))\n - plugin: add a daemon plugin with access to the CoreAPI ([ipfs/go-ipfs#5955](https://github.com/ipfs/go-ipfs/pull/5955))\n - coreapi: add some seeker tests ([ipfs/go-ipfs#5934](https://github.com/ipfs/go-ipfs/pull/5934))\n - Refactor ipfs get to use CoreAPI ([ipfs/go-ipfs#5943](https://github.com/ipfs/go-ipfs/pull/5943))\n - refact(cmd/init): change string option to const ([ipfs/go-ipfs#5949](https://github.com/ipfs/go-ipfs/pull/5949))\n - cmds/pin: use coreapi/pin ([ipfs/go-ipfs#5843](https://github.com/ipfs/go-ipfs/pull/5843))\n - Only perform DNSLink lookups on fully qualified domain names (FQDN) ([ipfs/go-ipfs#5950](https://github.com/ipfs/go-ipfs/pull/5950))\n - Fix DontCheckOSXFUSE config command example ([ipfs/go-ipfs#5951](https://github.com/ipfs/go-ipfs/pull/5951))\n - refact(cmd/config): change string option to const ([ipfs/go-ipfs#5948](https://github.com/ipfs/go-ipfs/pull/5948))\n - clarification the document of --resolve flag in name.publish ([ipfs/go-ipfs#5651](https://github.com/ipfs/go-ipfs/pull/5651))\n - Drop some coreunix code ([ipfs/go-ipfs#5938](https://github.com/ipfs/go-ipfs/pull/5938))\n - commands: fix verbose flag ([ipfs/go-ipfs#5940](https://github.com/ipfs/go-ipfs/pull/5940))\n - Fixes #4558 ([ipfs/go-ipfs#5937](https://github.com/ipfs/go-ipfs/pull/5937))\n - Port dag commansds to CoreAPI ([ipfs/go-ipfs#5939](https://github.com/ipfs/go-ipfs/pull/5939))\n - mfs: make sure to flush after mv and chcid ([ipfs/go-ipfs#5936](https://github.com/ipfs/go-ipfs/pull/5936))\n - docs/code-flow : Add code flow documentation for add cmd. ([ipfs/go-ipfs#5864](https://github.com/ipfs/go-ipfs/pull/5864))\n - coreapi: few more error check fixes ([ipfs/go-ipfs#5935](https://github.com/ipfs/go-ipfs/pull/5935))\n - Fixed and cleaned up TestIpfsStressRead ([ipfs/go-ipfs#5920](https://github.com/ipfs/go-ipfs/pull/5920))\n - Clarify that chunker sizes are in bytes ([ipfs/go-ipfs#5923](https://github.com/ipfs/go-ipfs/pull/5923))\n - refact(cmd/patch): change string to const ([ipfs/go-ipfs#5931](https://github.com/ipfs/go-ipfs/pull/5931))\n - refact(cmd/object): change option string to const ([ipfs/go-ipfs#5932](https://github.com/ipfs/go-ipfs/pull/5932))\n - coreapi: replace coreiface.DagAPI with ipld.DAGService ([ipfs/go-ipfs#5922](https://github.com/ipfs/go-ipfs/pull/5922))\n - Add global option to specify the multibase encoding (server side) ([ipfs/go-ipfs#5789](https://github.com/ipfs/go-ipfs/pull/5789))\n - coreapi: Adjust some tests for go-ipfs-http-api ([ipfs/go-ipfs#5926](https://github.com/ipfs/go-ipfs/pull/5926))\n - chore: update to Web UI v2.3.3 ([ipfs/go-ipfs#5928](https://github.com/ipfs/go-ipfs/pull/5928))\n - ls: Report real file size ([ipfs/go-ipfs#5906](https://github.com/ipfs/go-ipfs/pull/5906))\n - Improve the Filestore document ([ipfs/go-ipfs#5927](https://github.com/ipfs/go-ipfs/pull/5927))\n - [CORS] Bubble go-ipfs-cmds 2.0.10 - Updates CORS library ([ipfs/go-ipfs#5919](https://github.com/ipfs/go-ipfs/pull/5919))\n - reduce verbosity of daemon start ([ipfs/go-ipfs#5904](https://github.com/ipfs/go-ipfs/pull/5904))\n - feat: update to Web UI v2.3.2 ([ipfs/go-ipfs#5899](https://github.com/ipfs/go-ipfs/pull/5899))\n - CoreAPI: Don't panic when testing incomplete implementations ([ipfs/go-ipfs#5900](https://github.com/ipfs/go-ipfs/pull/5900))\n - gateway: fix CORs headers ([ipfs/go-ipfs#5893](https://github.com/ipfs/go-ipfs/pull/5893))\n - Local Gateway option ([ipfs/go-ipfs#5649](https://github.com/ipfs/go-ipfs/pull/5649))\n - Show hash on gateway ([ipfs/go-ipfs#5830](https://github.com/ipfs/go-ipfs/pull/5830))\n - fix: ulimit docs mistake ([ipfs/go-ipfs#5894](https://github.com/ipfs/go-ipfs/pull/5894))\n - Move coreapi tests to the interface ([ipfs/go-ipfs#5865](https://github.com/ipfs/go-ipfs/pull/5865))\n - Move checkHelptextRecursive forward a bit ([ipfs/go-ipfs#5889](https://github.com/ipfs/go-ipfs/pull/5889))\n - coreapi/unixfs: Use path instead of raw hash in AddEvent ([ipfs/go-ipfs#5854](https://github.com/ipfs/go-ipfs/pull/5854))\n - Fix name resolve --offline ([ipfs/go-ipfs#5885](https://github.com/ipfs/go-ipfs/pull/5885))\n - testing: slow down republisher sharness test ([ipfs/go-ipfs#5856](https://github.com/ipfs/go-ipfs/pull/5856))\n - docs: flesh out plugin documentation ([ipfs/go-ipfs#5876](https://github.com/ipfs/go-ipfs/pull/5876))\n - main: move InterruptHandler to util ([ipfs/go-ipfs#5872](https://github.com/ipfs/go-ipfs/pull/5872))\n - make: fix building source tarball on macos ([ipfs/go-ipfs#5860](https://github.com/ipfs/go-ipfs/pull/5860))\n - fix config data race ([ipfs/go-ipfs#5634](https://github.com/ipfs/go-ipfs/pull/5634))\n - CoreAPI: Global offline option ([ipfs/go-ipfs#5825](https://github.com/ipfs/go-ipfs/pull/5825))\n - Update for go-ipfs-files refactor ([ipfs/go-ipfs#5661](https://github.com/ipfs/go-ipfs/pull/5661))\n - feat: update Web UI to v2.3.0 ([ipfs/go-ipfs#5855](https://github.com/ipfs/go-ipfs/pull/5855))\n - Stateful plugin loading ([ipfs/go-ipfs#4806](https://github.com/ipfs/go-ipfs/pull/4806))\n - startup: always load the private key ([ipfs/go-ipfs#5844](https://github.com/ipfs/go-ipfs/pull/5844))\n - add --dereference-args parameter ([ipfs/go-ipfs#5801](https://github.com/ipfs/go-ipfs/pull/5801))\n - config: document the connection manager ([ipfs/go-ipfs#5839](https://github.com/ipfs/go-ipfs/pull/5839))\n - add pinning support to the urlstore ([ipfs/go-ipfs#5834](https://github.com/ipfs/go-ipfs/pull/5834))\n - refact(cmd/cat): remove useless code ([ipfs/go-ipfs#5836](https://github.com/ipfs/go-ipfs/pull/5836))\n - Really run as non-root user in docker container ([ipfs/go-ipfs#5048](https://github.com/ipfs/go-ipfs/pull/5048))\n - README: document guix package ([ipfs/go-ipfs#5832](https://github.com/ipfs/go-ipfs/pull/5832))\n - docs: Improve config documentation ([ipfs/go-ipfs#5829](https://github.com/ipfs/go-ipfs/pull/5829))\n - block: rm extra output ([ipfs/go-ipfs#5751](https://github.com/ipfs/go-ipfs/pull/5751))\n - merge github-issue-guide with the issue template ([ipfs/go-ipfs#4636](https://github.com/ipfs/go-ipfs/pull/4636))\n - docs: fix inconsistent capitalization of \"API\". ([ipfs/go-ipfs#5824](https://github.com/ipfs/go-ipfs/pull/5824))\n - Update README.md ([ipfs/go-ipfs#5818](https://github.com/ipfs/go-ipfs/pull/5818))\n - CONTRIBUTING.md link ([ipfs/go-ipfs#5811](https://github.com/ipfs/go-ipfs/pull/5811))\n - README: Update required Go version ([ipfs/go-ipfs#5813](https://github.com/ipfs/go-ipfs/pull/5813))\n - p2p: report-peer-id option for listen ([ipfs/go-ipfs#5771](https://github.com/ipfs/go-ipfs/pull/5771))\n - really fix netcat race ([ipfs/go-ipfs#5803](https://github.com/ipfs/go-ipfs/pull/5803))\n - [http_proxy_over_p2p] ([ipfs/go-ipfs#5526](https://github.com/ipfs/go-ipfs/pull/5526))\n - coreapi/pin: Use CID's directly in maps instead of converting to string ([ipfs/go-ipfs#5809](https://github.com/ipfs/go-ipfs/pull/5809))\n - Gx update go-merkledag and related deps. ([ipfs/go-ipfs#5802](https://github.com/ipfs/go-ipfs/pull/5802))\n - cmds: rm old lib ([ipfs/go-ipfs#5786](https://github.com/ipfs/go-ipfs/pull/5786))\n - badger: add truncate flag ([ipfs/go-ipfs#5625](https://github.com/ipfs/go-ipfs/pull/5625))\n - docker: allow IPFS_PROFILE to choose the profile for `ipfs init` ([ipfs/go-ipfs#5473](https://github.com/ipfs/go-ipfs/pull/5473))\n - Add --stream option to `ls` command ([ipfs/go-ipfs#5611](https://github.com/ipfs/go-ipfs/pull/5611))\n - Switch to using request.Context() ([ipfs/go-ipfs#5782](https://github.com/ipfs/go-ipfs/pull/5782))\n - Update go-ipfs-delay and assoc deps ([ipfs/go-ipfs#5762](https://github.com/ipfs/go-ipfs/pull/5762))\n - Suppress bootstrap error ([ipfs/go-ipfs#5769](https://github.com/ipfs/go-ipfs/pull/5769))\n - ISSUE_TEMPLATE: move the support question comment to the very top ([ipfs/go-ipfs#5770](https://github.com/ipfs/go-ipfs/pull/5770))\n - cmds: use MakeTypedEncoder ([ipfs/go-ipfs#5760](https://github.com/ipfs/go-ipfs/pull/5760))\n - cmds/bitswap: sort wantlist ([ipfs/go-ipfs#5759](https://github.com/ipfs/go-ipfs/pull/5759))\n - cmds/update: use new cmds lib ([ipfs/go-ipfs#5730](https://github.com/ipfs/go-ipfs/pull/5730))\n - cmds/file: use new cmds lib ([ipfs/go-ipfs#5756](https://github.com/ipfs/go-ipfs/pull/5756))\n - cmds: remove redundant func ([ipfs/go-ipfs#5750](https://github.com/ipfs/go-ipfs/pull/5750))\n - commands/refs: use new cmds ([ipfs/go-ipfs#5679](https://github.com/ipfs/go-ipfs/pull/5679))\n - commands/pin: use new cmds lib ([ipfs/go-ipfs#5674](https://github.com/ipfs/go-ipfs/pull/5674))\n - commands/bootstrap: use new cmds ([ipfs/go-ipfs#5678](https://github.com/ipfs/go-ipfs/pull/5678))\n - fix(cmd/add): progressbar output error when input is read from stdin ([ipfs/go-ipfs#5743](https://github.com/ipfs/go-ipfs/pull/5743))\n - unexport GOFLAGS ([ipfs/go-ipfs#5747](https://github.com/ipfs/go-ipfs/pull/5747))\n - refactor(cmds): use new cmds ([ipfs/go-ipfs#5659](https://github.com/ipfs/go-ipfs/pull/5659))\n - commands/filestore: use new cmds lib ([ipfs/go-ipfs#5673](https://github.com/ipfs/go-ipfs/pull/5673))\n - Fix broken links ([ipfs/go-ipfs#5721](https://github.com/ipfs/go-ipfs/pull/5721))\n - fix `ipfs help` bug #5557 ([ipfs/go-ipfs#5573](https://github.com/ipfs/go-ipfs/pull/5573))\n - commands/bitswap: use new cmds lib ([ipfs/go-ipfs#5676](https://github.com/ipfs/go-ipfs/pull/5676))\n - refact(cmd/repo): repo's sub cmds uses new cmd lib ([ipfs/go-ipfs#5677](https://github.com/ipfs/go-ipfs/pull/5677))\n - fix the maketarball script ([ipfs/go-ipfs#5718](https://github.com/ipfs/go-ipfs/pull/5718))\n - output link to WebUI on daemon startup ([ipfs/go-ipfs#5729](https://github.com/ipfs/go-ipfs/pull/5729))\n - Move persistent datastores to plugins ([ipfs/go-ipfs#5695](https://github.com/ipfs/go-ipfs/pull/5695))\n - Update IPTB test ([ipfs/go-ipfs#5636](https://github.com/ipfs/go-ipfs/pull/5636))\n - enhance(cmd/verify): add goroutine count to improve verify speed ([ipfs/go-ipfs#5710](https://github.com/ipfs/go-ipfs/pull/5710))\n - Update go-mfs and go-unixfs ([ipfs/go-ipfs#5714](https://github.com/ipfs/go-ipfs/pull/5714))\n - fix(flag/version): flag `all` should have a higher priority ([ipfs/go-ipfs#5719](https://github.com/ipfs/go-ipfs/pull/5719))\n - commands/p2p: use new cmds lib ([ipfs/go-ipfs#5672](https://github.com/ipfs/go-ipfs/pull/5672))\n - commands/dht: use new cmds lib ([ipfs/go-ipfs#5671](https://github.com/ipfs/go-ipfs/pull/5671))\n - commands/object: use new cmds ([ipfs/go-ipfs#5666](https://github.com/ipfs/go-ipfs/pull/5666))\n - commands/files: use new cmds ([ipfs/go-ipfs#5665](https://github.com/ipfs/go-ipfs/pull/5665))\n - cmds/env: add a config path helper ([ipfs/go-ipfs#5712](https://github.com/ipfs/go-ipfs/pull/5712))\n- github.com/ipfs/dir-index-html:\n - show hash if given ([ipfs/dir-index-html#21](https://github.com/ipfs/dir-index-html/pull/21))\n - Add \"jpeg\" as an alias to \"jpg\". ([ipfs/dir-index-html#16](https://github.com/ipfs/dir-index-html/pull/16))\n- github.com/libp2p/go-addr-util:\n - Improve test coverage ([libp2p/go-addr-util#14](https://github.com/libp2p/go-addr-util/pull/14))\n- github.com/ipfs/go-bitswap:\n - fix(prq): fix a bunch of goroutine leaks and deadlocks ([ipfs/go-bitswap#87](https://github.com/ipfs/go-bitswap/pull/87))\n - remove allocations round two ([ipfs/go-bitswap#84](https://github.com/ipfs/go-bitswap/pull/84))\n - fix(bitswap): remove CancelWants function ([ipfs/go-bitswap#80](https://github.com/ipfs/go-bitswap/pull/80))\n - Avoid allocating for wantlist entries ([ipfs/go-bitswap#79](https://github.com/ipfs/go-bitswap/pull/79))\n - ci(Jenkins): remove Jenkinsfile ([ipfs/go-bitswap#83](https://github.com/ipfs/go-bitswap/pull/83))\n - More specific wantlists ([ipfs/go-bitswap#74](https://github.com/ipfs/go-bitswap/pull/74))\n - fix(wantlist): remove races on setup ([ipfs/go-bitswap#72](https://github.com/ipfs/go-bitswap/pull/72))\n - fix multiple data races ([ipfs/go-bitswap#76](https://github.com/ipfs/go-bitswap/pull/76))\n - ci: add travis ([ipfs/go-bitswap#75](https://github.com/ipfs/go-bitswap/pull/75))\n - providers: don't add every connected node as a provider ([ipfs/go-bitswap#59](https://github.com/ipfs/go-bitswap/pull/59))\n - refactor(GetBlocks): Merge session/non-session ([ipfs/go-bitswap#64](https://github.com/ipfs/go-bitswap/pull/64))\n - Feat: A more robust provider finder for sessions (for now) and soon for all bitswap ([ipfs/go-bitswap#60](https://github.com/ipfs/go-bitswap/pull/60))\n - fix(tests): stabilize session tests ([ipfs/go-bitswap#63](https://github.com/ipfs/go-bitswap/pull/63))\n - contexts: make sure to abort when a context is canceled ([ipfs/go-bitswap#58](https://github.com/ipfs/go-bitswap/pull/58))\n - fix(sessions): explicitly connect found peers ([ipfs/go-bitswap#56](https://github.com/ipfs/go-bitswap/pull/56))\n - Speed up sessions Round #1 ([ipfs/go-bitswap#27](https://github.com/ipfs/go-bitswap/pull/27))\n - Fix debug log formatting issues ([ipfs/go-bitswap#37](https://github.com/ipfs/go-bitswap/pull/37))\n - Feat/bandwidth limited tests ([ipfs/go-bitswap#42](https://github.com/ipfs/go-bitswap/pull/42))\n - fix(tests): stabilize unreliable session tests ([ipfs/go-bitswap#44](https://github.com/ipfs/go-bitswap/pull/44))\n - Bitswap Refactor #4: Extract session peer manager from sessions ([ipfs/go-bitswap#26](https://github.com/ipfs/go-bitswap/pull/26))\n - Bitswap Refactor #3: Extract sessions to package ([ipfs/go-bitswap#30](https://github.com/ipfs/go-bitswap/pull/30))\n - docs(comments): end comment sentences to have full-stop ([ipfs/go-bitswap#33](https://github.com/ipfs/go-bitswap/pull/33))\n - Bitswap Refactor #2: Extract PeerManager From Want Manager + Unit Test ([ipfs/go-bitswap#29](https://github.com/ipfs/go-bitswap/pull/29))\n - Bitswap Refactor #1: Session Manager & Extract Want Manager ([ipfs/go-bitswap#28](https://github.com/ipfs/go-bitswap/pull/28))\n - fix(Receiver): Ignore unwanted blocks ([ipfs/go-bitswap#24](https://github.com/ipfs/go-bitswap/pull/24))\n - feat(Benchmarks): Add real world dup blocks test ([ipfs/go-bitswap#25](https://github.com/ipfs/go-bitswap/pull/25))\n - Feat/bitswap pr improvements ([ipfs/go-bitswap#19](https://github.com/ipfs/go-bitswap/pull/19))\n- github.com/ipfs/go-blockservice:\n - Don't return errors on closed exchange ([ipfs/go-blockservice#15](https://github.com/ipfs/go-blockservice/pull/15))\n- github.com/ipfs/go-cid:\n - fix inline CIDs generated by Prefix.Sum ([ipfs/go-cid#84](https://github.com/ipfs/go-cid/pull/84))\n - Let Cid implement Binary[Un]Marshaler and Text[Un]Marshaler interfaces. ([ipfs/go-cid#81](https://github.com/ipfs/go-cid/pull/81))\n - fix typo in comment ([ipfs/go-cid#80](https://github.com/ipfs/go-cid/pull/80))\n - add codecs for Dash blocks, tx ([ipfs/go-cid#78](https://github.com/ipfs/go-cid/pull/78))\n- github.com/ipfs/go-cidutil:\n - Fix Travis CI to run all tests. ([ipfs/go-cidutil#11](https://github.com/ipfs/go-cidutil/pull/11))\n - Changes needed for `--cid-base` option in go-ipfs (simplified version) ([ipfs/go-cidutil#10](https://github.com/ipfs/go-cidutil/pull/10))\n - add a utility method for sorting CID slices ([ipfs/go-cidutil#5](https://github.com/ipfs/go-cidutil/pull/5))\n- github.com/libp2p/go-conn-security:\n - fix link to usage example in README ([libp2p/go-conn-security#4](https://github.com/libp2p/go-conn-security/pull/4))\n- github.com/ipfs/go-datastore:\n - interfaces: make GetBacked* take a Read instead of a Datastore ([ipfs/go-datastore#115](https://github.com/ipfs/go-datastore/pull/115))\n - remove closer type assertions ([ipfs/go-datastore#112](https://github.com/ipfs/go-datastore/pull/112))\n - remove io.Closer from the transaction interface ([ipfs/go-datastore#113](https://github.com/ipfs/go-datastore/pull/113))\n - feat(datastore): expose datastore Close() ([ipfs/go-datastore#111](https://github.com/ipfs/go-datastore/pull/111))\n - query: make datastore ordering act like a user would expect ([ipfs/go-datastore#110](https://github.com/ipfs/go-datastore/pull/110))\n - delayed: implement io.Closer and export datastore type. ([ipfs/go-datastore#108](https://github.com/ipfs/go-datastore/pull/108))\n - split the datastore into a read and a write interface ([ipfs/go-datastore#107](https://github.com/ipfs/go-datastore/pull/107))\n - Describe behavior of Batching datastores ([ipfs/go-datastore#105](https://github.com/ipfs/go-datastore/pull/105))\n - handle concurrent puts/deletes in BasicBatch ([ipfs/go-datastore#103](https://github.com/ipfs/go-datastore/pull/103))\n - add a GetSize method ([ipfs/go-datastore#99](https://github.com/ipfs/go-datastore/pull/99))\n- github.com/ipfs/go-ds-badger:\n - removed additional/wasteful Prefix conversion ([ipfs/go-ds-badger#45](https://github.com/ipfs/go-ds-badger/pull/45))\n - Enable Jenkins ([ipfs/go-ds-badger#35](https://github.com/ipfs/go-ds-badger/pull/35))\n - fix application or ordering for interface change ([ipfs/go-ds-badger#44](https://github.com/ipfs/go-ds-badger/pull/44))\n - Update badger ([ipfs/go-ds-badger#40](https://github.com/ipfs/go-ds-badger/pull/40))\n- github.com/ipfs/go-ds-flatfs:\n - fix a goroutine leak killing the gateways ([ipfs/go-ds-flatfs#51](https://github.com/ipfs/go-ds-flatfs/pull/51))\n- github.com/ipfs/go-ds-leveldb:\n - Expose Datastore type ([ipfs/go-ds-leveldb#20](https://github.com/ipfs/go-ds-leveldb/pull/20))\n - fix application or ordering for interface change ([ipfs/go-ds-leveldb#23](https://github.com/ipfs/go-ds-leveldb/pull/23))\n- github.com/ipfs/go-ipfs-cmds:\n - fix sync error with go1.12 on darwin ([ipfs/go-ipfs-cmds#147](https://github.com/ipfs/go-ipfs-cmds/pull/147))\n - cli: fix ignoring std{out,err} sync errors on windows ([ipfs/go-ipfs-cmds#146](https://github.com/ipfs/go-ipfs-cmds/pull/146))\n - roundup of cleanup fixes ([ipfs/go-ipfs-cmds#144](https://github.com/ipfs/go-ipfs-cmds/pull/144))\n - Update cors library ([ipfs/go-ipfs-cmds#139](https://github.com/ipfs/go-ipfs-cmds/pull/139))\n - expand on the api error ([ipfs/go-ipfs-cmds#138](https://github.com/ipfs/go-ipfs-cmds/pull/138))\n - set the connection close header if we have a body to read ([ipfs/go-ipfs-cmds#116](https://github.com/ipfs/go-ipfs-cmds/pull/116))\n - print a nicer error on timeout/cancel ([ipfs/go-ipfs-cmds#137](https://github.com/ipfs/go-ipfs-cmds/pull/137))\n - Add link traversal option ([ipfs/go-ipfs-cmds#96](https://github.com/ipfs/go-ipfs-cmds/pull/96))\n - Don't skip stdin test on Windows ([ipfs/go-ipfs-cmds#136](https://github.com/ipfs/go-ipfs-cmds/pull/136))\n - MakeTypedEncoder: accept results by pointer or value ([ipfs/go-ipfs-cmds#134](https://github.com/ipfs/go-ipfs-cmds/pull/134))\n- github.com/ipfs/go-ipfs-config:\n - Gateway.NoFetch ([ipfs/go-ipfs-config#19](https://github.com/ipfs/go-ipfs-config/pull/19))\n - add a Clone function ([ipfs/go-ipfs-config#16](https://github.com/ipfs/go-ipfs-config/pull/16))\n - randomports: give user ability to init ipfs using random port for swarm. ([ipfs/go-ipfs-config#17](https://github.com/ipfs/go-ipfs-config/pull/17))\n - Allow the use of the User-Agent header ([ipfs/go-ipfs-config#15](https://github.com/ipfs/go-ipfs-config/pull/15))\n - autorelay options ([ipfs/go-ipfs-config#21](https://github.com/ipfs/go-ipfs-config/pull/21))\n - profile: add badger truncate option ([ipfs/go-ipfs-config#20](https://github.com/ipfs/go-ipfs-config/pull/20))\n- github.com/ipfs/go-ipfs-delay:\n - Feat/refactor wait time ([ipfs/go-ipfs-delay#1](https://github.com/ipfs/go-ipfs-delay/pull/1))\n- github.com/ipfs/go-ipfs-files:\n - multipart: fix handling of common prefixes ([ipfs/go-ipfs-files#7](https://github.com/ipfs/go-ipfs-files/pull/7))\n - create implicit directories from multipart requests ([ipfs/go-ipfs-files#6](https://github.com/ipfs/go-ipfs-files/pull/6))\n - TarWriter ([ipfs/go-ipfs-files#4](https://github.com/ipfs/go-ipfs-files/pull/4))\n - Refactor filename - file relation ([ipfs/go-ipfs-files#2](https://github.com/ipfs/go-ipfs-files/pull/2))\n- github.com/ipfs/go-ipld-cbor:\n - cbor: decode undefined as null ([ipfs/go-ipld-cbor#54](https://github.com/ipfs/go-ipld-cbor/pull/54))\n - error when trying to encode an empty link ([ipfs/go-ipld-cbor#52](https://github.com/ipfs/go-ipld-cbor/pull/52))\n - test for struct with both a cid and a bigint ([ipfs/go-ipld-cbor#51](https://github.com/ipfs/go-ipld-cbor/pull/51))\n- github.com/ipfs/go-ipld-format:\n - Add a DAG walker with support for IPLD `Node`s ([ipfs/go-ipld-format#39](https://github.com/ipfs/go-ipld-format/pull/39))\n - Add BufferedDAG wrapping Batch as a DAGService. ([ipfs/go-ipld-format#48](https://github.com/ipfs/go-ipld-format/pull/48))\n- github.com/ipfs/go-ipld-git:\n - Fix blob marshalling ([ipfs/go-ipld-git#37](https://github.com/ipfs/go-ipld-git/pull/37))\n - Re-enable assertion on commit size -- it is correct after #31 ([ipfs/go-ipld-git#33](https://github.com/ipfs/go-ipld-git/pull/33))\n - Use OS path separator in testing, fixes #30 ([ipfs/go-ipld-git#34](https://github.com/ipfs/go-ipld-git/pull/34))\n - Use rawdata length for size, fixes #7 ([ipfs/go-ipld-git#31](https://github.com/ipfs/go-ipld-git/pull/31))\n - Cache RawData for Commit, Tag, & Tree, fixes #6 ([ipfs/go-ipld-git#28](https://github.com/ipfs/go-ipld-git/pull/28))\n - Precompute Blob CID, fixes #21 ([ipfs/go-ipld-git#27](https://github.com/ipfs/go-ipld-git/pull/27))\n - Enable Jenkins ([ipfs/go-ipld-git#29](https://github.com/ipfs/go-ipld-git/pull/29))\n- github.com/ipfs/go-ipns:\n - fix community/CONTRIBUTING.md link in README.md ([ipfs/go-ipns#20](https://github.com/ipfs/go-ipns/pull/20))\n - fix typo in README.md ([ipfs/go-ipns#21](https://github.com/ipfs/go-ipns/pull/21))\n - testing: disable inline peer ID test ([ipfs/go-ipns#19](https://github.com/ipfs/go-ipns/pull/19))\n- github.com/libp2p/go-libp2p:\n - Fixed race conditions in mock package mock_stream and mock_conn ([libp2p/go-libp2p#535](https://github.com/libp2p/go-libp2p/pull/535))\n - increase initial relay advertisement delay to 30s ([libp2p/go-libp2p#534](https://github.com/libp2p/go-libp2p/pull/534))\n - Use PeerRouting in autorelay to find relay peer addresses ([libp2p/go-libp2p#531](https://github.com/libp2p/go-libp2p/pull/531))\n - docs: update broken links in NEWS.md ([libp2p/go-libp2p#517](https://github.com/libp2p/go-libp2p/pull/517))\n - don't advertise the raw public address in autorelay ([libp2p/go-libp2p#511](https://github.com/libp2p/go-libp2p/pull/511))\n - mock: export ratelimiter as RateLimiter ([libp2p/go-libp2p#507](https://github.com/libp2p/go-libp2p/pull/507))\n - readme: remove duplicate repo entries in README and package-list.json ([libp2p/go-libp2p#506](https://github.com/libp2p/go-libp2p/pull/506))\n - explicit option to enable autorelay ([libp2p/go-libp2p#500](https://github.com/libp2p/go-libp2p/pull/500))\n - Add delay in initial relay advertisement to allow the dht time to bootstrap ([libp2p/go-libp2p#495](https://github.com/libp2p/go-libp2p/pull/495))\n - suppressing error msg for NoSecurity option ([libp2p/go-libp2p#498](https://github.com/libp2p/go-libp2p/pull/498))\n - pulling updates ([libp2p/go-libp2p#4](https://github.com/libp2p/go-libp2p/pull/4))\n - fix contributing link in README ([libp2p/go-libp2p#494](https://github.com/libp2p/go-libp2p/pull/494))\n - Fix badges and links on README.md ([libp2p/go-libp2p#485](https://github.com/libp2p/go-libp2p/pull/485))\n - mocknet: fix NewStream and self dials ([libp2p/go-libp2p#480](https://github.com/libp2p/go-libp2p/pull/480))\n - deflake identify test ([libp2p/go-libp2p#479](https://github.com/libp2p/go-libp2p/pull/479))\n - mocknet: use peer ID in peer address ([libp2p/go-libp2p#476](https://github.com/libp2p/go-libp2p/pull/476))\n - autorelay ([libp2p/go-libp2p#454](https://github.com/libp2p/go-libp2p/pull/454))\n - Getting updates ([libp2p/go-libp2p#3](https://github.com/libp2p/go-libp2p/pull/3))\n- github.com/libp2p/go-libp2p-autonat:\n - track autonat peer addresses ([libp2p/go-libp2p-autonat#7](https://github.com/libp2p/go-libp2p-autonat/pull/7))\n- github.com/libp2p/go-libp2p-circuit:\n - Don't log raw binary ([libp2p/go-libp2p-circuit#53](https://github.com/libp2p/go-libp2p-circuit/pull/53))\n- github.com/libp2p/go-libp2p-connmgr:\n - Fix concurrency and silence period not being honoured ([libp2p/go-libp2p-connmgr#26](https://github.com/libp2p/go-libp2p-connmgr/pull/26))\n- github.com/libp2p/go-libp2p-crypto:\n - Fix: Remove redundant Ed25519 public key (#36). ([libp2p/go-libp2p-crypto#54](https://github.com/libp2p/go-libp2p-crypto/pull/54))\n - libp2p badges, remove IPFS ([libp2p/go-libp2p-crypto#52](https://github.com/libp2p/go-libp2p-crypto/pull/52))\n - Fix broken contribute link in README ([libp2p/go-libp2p-crypto#46](https://github.com/libp2p/go-libp2p-crypto/pull/46))\n - forbid RSA keys smaller than 512 bits ([libp2p/go-libp2p-crypto#43](https://github.com/libp2p/go-libp2p-crypto/pull/43))\n - Added ECDSA; Added RSA tests; Fixed linting errors; Handling all un-handled errors ([libp2p/go-libp2p-crypto#35](https://github.com/libp2p/go-libp2p-crypto/pull/35))\n - switch to the go-crypto ed25519 implementation ([libp2p/go-libp2p-crypto#38](https://github.com/libp2p/go-libp2p-crypto/pull/38))\n - update gogo protobuf ([libp2p/go-libp2p-crypto#37](https://github.com/libp2p/go-libp2p-crypto/pull/37))\n- github.com/libp2p/go-libp2p-discovery:\n - add a timeout to Provide in routing.Advertise ([libp2p/go-libp2p-discovery#12](https://github.com/libp2p/go-libp2p-discovery/pull/12))\n - correctly encode ns to CID ([libp2p/go-libp2p-discovery#11](https://github.com/libp2p/go-libp2p-discovery/pull/11))\n - use 6hrs as ttl for routing based advertisements ([libp2p/go-libp2p-discovery#8](https://github.com/libp2p/go-libp2p-discovery/pull/8))\n- github.com/libp2p/go-libp2p-host:\n - Helper to get PeerInfo from Host ([libp2p/go-libp2p-host#20](https://github.com/libp2p/go-libp2p-host/pull/20))\n- github.com/libp2p/go-libp2p-kad-dht:\n - fix(dialQueue): account for failed dials ([libp2p/go-libp2p-kad-dht#277](https://github.com/libp2p/go-libp2p-kad-dht/pull/277))\n - Fix Bootstrap sub-queries ([libp2p/go-libp2p-kad-dht#264](https://github.com/libp2p/go-libp2p-kad-dht/pull/264))\n - dial queue: fix possible goroutine leak ([libp2p/go-libp2p-kad-dht#262](https://github.com/libp2p/go-libp2p-kad-dht/pull/262))\n - Alter some logging ([libp2p/go-libp2p-kad-dht#269](https://github.com/libp2p/go-libp2p-kad-dht/pull/269))\n - Revert #236: Test go mod in travis and use major versioning in import paths ([libp2p/go-libp2p-kad-dht#259](https://github.com/libp2p/go-libp2p-kad-dht/pull/259))\n - fix tests on freebsd ([libp2p/go-libp2p-kad-dht#255](https://github.com/libp2p/go-libp2p-kad-dht/pull/255))\n - Fix \"no protocol with name dnsaddr\" error ([libp2p/go-libp2p-kad-dht#247](https://github.com/libp2p/go-libp2p-kad-dht/pull/247))\n - Fix a race in dial queue ([libp2p/go-libp2p-kad-dht#248](https://github.com/libp2p/go-libp2p-kad-dht/pull/248))\n - Fix races with DialQueue variables ([libp2p/go-libp2p-kad-dht#241](https://github.com/libp2p/go-libp2p-kad-dht/pull/241))\n - Fix CircleCI ([libp2p/go-libp2p-kad-dht#238](https://github.com/libp2p/go-libp2p-kad-dht/pull/238))\n - Adaptive queue for staging dials ([libp2p/go-libp2p-kad-dht#237](https://github.com/libp2p/go-libp2p-kad-dht/pull/237))\n - Add the full libp2p default bootstrap peer list ([libp2p/go-libp2p-kad-dht#226](https://github.com/libp2p/go-libp2p-kad-dht/pull/226))\n - Revert \"Tidy up bootstrapping\" ([libp2p/go-libp2p-kad-dht#232](https://github.com/libp2p/go-libp2p-kad-dht/pull/232))\n - Tidy up bootstrapping ([libp2p/go-libp2p-kad-dht#225](https://github.com/libp2p/go-libp2p-kad-dht/pull/225))\n - Revert \"Remove signal bootstrapping\" ([libp2p/go-libp2p-kad-dht#227](https://github.com/libp2p/go-libp2p-kad-dht/pull/227))\n - Remove signal bootstrapping ([libp2p/go-libp2p-kad-dht#224](https://github.com/libp2p/go-libp2p-kad-dht/pull/224))\n - fix a potential DHT query hang ([libp2p/go-libp2p-kad-dht#219](https://github.com/libp2p/go-libp2p-kad-dht/pull/219))\n - docs: duplicate pkg documentation ([libp2p/go-libp2p-kad-dht#218](https://github.com/libp2p/go-libp2p-kad-dht/pull/218))\n - tests: skip key inlining test ([libp2p/go-libp2p-kad-dht#212](https://github.com/libp2p/go-libp2p-kad-dht/pull/212))\n - Rephrase \"betterPeersToQuery\" method comment to be less cryptic ([libp2p/go-libp2p-kad-dht#206](https://github.com/libp2p/go-libp2p-kad-dht/pull/206))\n- github.com/libp2p/go-libp2p-loggables:\n - test: add unit tests ([libp2p/go-libp2p-loggables#21](https://github.com/libp2p/go-libp2p-loggables/pull/21))\n- github.com/libp2p/go-libp2p-netutil:\n - Add tests ([libp2p/go-libp2p-netutil#28](https://github.com/libp2p/go-libp2p-netutil/pull/28))\n- github.com/libp2p/go-libp2p-peer:\n - fix: re-enable peer ID inlining but make it configurable ([libp2p/go-libp2p-peer#42](https://github.com/libp2p/go-libp2p-peer/pull/42))\n - Protobuf and JSON (un-)marshalling methods for peer.ID ([libp2p/go-libp2p-peer#41](https://github.com/libp2p/go-libp2p-peer/pull/41))\n - disable key inlining ([libp2p/go-libp2p-peer#40](https://github.com/libp2p/go-libp2p-peer/pull/40))\n- github.com/libp2p/go-libp2p-peerstore:\n - Add unit test to verify AddAddr doesn't shorten TTL ([libp2p/go-libp2p-peerstore#52](https://github.com/libp2p/go-libp2p-peerstore/pull/52))\n - disable inline-peer id test ([libp2p/go-libp2p-peerstore#49](https://github.com/libp2p/go-libp2p-peerstore/pull/49))\n - README: Update contributing guideline linkrot. ([libp2p/go-libp2p-peerstore#48](https://github.com/libp2p/go-libp2p-peerstore/pull/48))\n - Deterministic benchmark order; Keybook interface benchmarks ([libp2p/go-libp2p-peerstore#43](https://github.com/libp2p/go-libp2p-peerstore/pull/43))\n - PeerInfo UnMarshal Error #393 ([libp2p/go-libp2p-peerstore#45](https://github.com/libp2p/go-libp2p-peerstore/pull/45))\n - fix the inline key test ([libp2p/go-libp2p-peerstore#44](https://github.com/libp2p/go-libp2p-peerstore/pull/44))\n- github.com/libp2p/go-libp2p-pubsub:\n - move timecache check/update after validation ([libp2p/go-libp2p-pubsub#156](https://github.com/libp2p/go-libp2p-pubsub/pull/156))\n - fix nonsensical check ([libp2p/go-libp2p-pubsub#154](https://github.com/libp2p/go-libp2p-pubsub/pull/154))\n - Extend validator interface to include message source ([libp2p/go-libp2p-pubsub#151](https://github.com/libp2p/go-libp2p-pubsub/pull/151))\n - Implement peer blacklist ([libp2p/go-libp2p-pubsub#149](https://github.com/libp2p/go-libp2p-pubsub/pull/149))\n - make timecache duration configurable ([libp2p/go-libp2p-pubsub#148](https://github.com/libp2p/go-libp2p-pubsub/pull/148))\n - godoc is not html either ([libp2p/go-libp2p-pubsub#147](https://github.com/libp2p/go-libp2p-pubsub/pull/147))\n - godoc documentation is not markdown ([libp2p/go-libp2p-pubsub#146](https://github.com/libp2p/go-libp2p-pubsub/pull/146))\n - Add documentation for subscribe's non-instantaneous semantics ([libp2p/go-libp2p-pubsub#145](https://github.com/libp2p/go-libp2p-pubsub/pull/145))\n - Some documentation ([libp2p/go-libp2p-pubsub#140](https://github.com/libp2p/go-libp2p-pubsub/pull/140))\n - rework peer tracking logic to handle multiple connections ([libp2p/go-libp2p-pubsub#132](https://github.com/libp2p/go-libp2p-pubsub/pull/132))\n- github.com/libp2p/go-libp2p-pubsub-router:\n - encode record-store keys in pubsub ([libp2p/go-libp2p-pubsub-router#17](https://github.com/libp2p/go-libp2p-pubsub-router/pull/17))\n- github.com/libp2p/go-libp2p-quic-transport:\n - fix badges in README ([libp2p/go-libp2p-quic-transport#39](https://github.com/libp2p/go-libp2p-quic-transport/pull/39))\n - Fix missing transport parameter in dialed connection ([libp2p/go-libp2p-quic-transport#38](https://github.com/libp2p/go-libp2p-quic-transport/pull/38))\n- github.com/libp2p/go-libp2p-routing:\n - Update the comment on IpfsRouting.Bootstrap ([libp2p/go-libp2p-routing#36](https://github.com/libp2p/go-libp2p-routing/pull/36))\n- github.com/libp2p/go-libp2p-swarm:\n - Make FD limits configurable by environment property ([libp2p/go-libp2p-swarm#102](https://github.com/libp2p/go-libp2p-swarm/pull/102))\n - Fix logging race ([libp2p/go-libp2p-swarm#100](https://github.com/libp2p/go-libp2p-swarm/pull/100))\n - Add CircleCI config ([libp2p/go-libp2p-swarm#99](https://github.com/libp2p/go-libp2p-swarm/pull/99))\n - Enhance debug logging in dial limiter ([libp2p/go-libp2p-swarm#98](https://github.com/libp2p/go-libp2p-swarm/pull/98))\n - dialer: handle dial cancel and/or completion before trying new addresses ([libp2p/go-libp2p-swarm#96](https://github.com/libp2p/go-libp2p-swarm/pull/96))\n - avoid spawning goroutines for canceled dials ([libp2p/go-libp2p-swarm#95](https://github.com/libp2p/go-libp2p-swarm/pull/95))\n - warn when we encounter a useless transport ([libp2p/go-libp2p-swarm#90](https://github.com/libp2p/go-libp2p-swarm/pull/90))\n- github.com/libp2p/go-libp2p-transport:\n - fix transport tests for quic ([libp2p/go-libp2p-transport#39](https://github.com/libp2p/go-libp2p-transport/pull/39))\n - fix: fully close streams before returning ([libp2p/go-libp2p-transport#37](https://github.com/libp2p/go-libp2p-transport/pull/37))\n - fix typo in README ([libp2p/go-libp2p-transport#36](https://github.com/libp2p/go-libp2p-transport/pull/36))\n- github.com/libp2p/go-libp2p-transport-upgrader:\n - annotate errors ([libp2p/go-libp2p-transport-upgrader#11](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/11))\n- github.com/ipfs/go-log:\n - uglify the (event) logs ([ipfs/go-log#53](https://github.com/ipfs/go-log/pull/53))\n - add environment variable for writing tracing information to a file ([ipfs/go-log#52](https://github.com/ipfs/go-log/pull/52))\n - correctly display the line number when FinishWithErr fails ([ipfs/go-log#51](https://github.com/ipfs/go-log/pull/51))\n- github.com/libp2p/go-maddr-filter:\n - test: extend test to improve coverage ([libp2p/go-maddr-filter#7](https://github.com/libp2p/go-maddr-filter/pull/7))\n- github.com/ipfs/go-merkledag:\n - Increase FetchGraphConcurrency to 32 ([ipfs/go-merkledag#29](https://github.com/ipfs/go-merkledag/pull/29))\n - Enable CI ([ipfs/go-merkledag#9](https://github.com/ipfs/go-merkledag/pull/9))\n - fix a fetch deadlock on error ([ipfs/go-merkledag#21](https://github.com/ipfs/go-merkledag/pull/21))\n - Wait for all go routines to finish before function returns ([ipfs/go-merkledag#19](https://github.com/ipfs/go-merkledag/pull/19))\n- github.com/ipfs/go-metrics-prometheus:\n - use Prometheus instead of gxed ([ipfs/go-metrics-prometheus#3](https://github.com/ipfs/go-metrics-prometheus/pull/3))\n- github.com/ipfs/go-mfs:\n - fix(mv): dst filename error ([ipfs/go-mfs#62](https://github.com/ipfs/go-mfs/pull/62))\n - fix over-wait in WaitPub ([ipfs/go-mfs#53](https://github.com/ipfs/go-mfs/pull/53))\n - Fix/32/pr ports from go-ipfs to go-mfs ([ipfs/go-mfs#49](https://github.com/ipfs/go-mfs/pull/49))\n - remove the `fullSync` option from `updateChildEntry` ([ipfs/go-mfs#45](https://github.com/ipfs/go-mfs/pull/45))\n - Various refactorings ([ipfs/go-mfs#36](https://github.com/ipfs/go-mfs/pull/36))\n - use RW lock for the `File`'s lock ([ipfs/go-mfs#43](https://github.com/ipfs/go-mfs/pull/43))\n - add documentation links in README ([ipfs/go-mfs#41](https://github.com/ipfs/go-mfs/pull/41))\n - [WIP] documentation notes ([ipfs/go-mfs#27](https://github.com/ipfs/go-mfs/pull/27))\n - feat(inode): add inode struct ([ipfs/go-mfs#12](https://github.com/ipfs/go-mfs/pull/12))\n- github.com/libp2p/go-mplex:\n - fix deadlock ([libp2p/go-mplex#39](https://github.com/libp2p/go-mplex/pull/39))\n - When a stream is closed, cancel pending writes ([libp2p/go-mplex#35](https://github.com/libp2p/go-mplex/pull/35))\n - make sure to but the buffer back in the pool ([libp2p/go-mplex#34](https://github.com/libp2p/go-mplex/pull/34))\n - reduce the packet count ([libp2p/go-mplex#29](https://github.com/libp2p/go-mplex/pull/29))\n- github.com/ipfs/go-path:\n - fix: no components error ([ipfs/go-path#18](https://github.com/ipfs/go-path/pull/18))\n - nit: validate CIDs in IPLD paths ([ipfs/go-path#16](https://github.com/ipfs/go-path/pull/16))\n- github.com/libp2p/go-reuseport:\n - Fix build on wasm ([libp2p/go-reuseport#59](https://github.com/libp2p/go-reuseport/pull/59))\n - Use Go Control API ([libp2p/go-reuseport#56](https://github.com/libp2p/go-reuseport/pull/56))\n - Support WASM ([libp2p/go-reuseport#54](https://github.com/libp2p/go-reuseport/pull/54))\n- github.com/libp2p/go-reuseport-transport:\n - Update to go-reuseport 0.2.0 ([libp2p/go-reuseport-transport#6](https://github.com/libp2p/go-reuseport-transport/pull/6))\n- github.com/libp2p/go-stream-muxer:\n - add standard reset error ([libp2p/go-stream-muxer#23](https://github.com/libp2p/go-stream-muxer/pull/23))\n - ci: fix ([libp2p/go-stream-muxer#24](https://github.com/libp2p/go-stream-muxer/pull/24))\n - Document Reset versus Close ([libp2p/go-stream-muxer#18](https://github.com/libp2p/go-stream-muxer/pull/18))\n - WIP document Conn.Close ([libp2p/go-stream-muxer#19](https://github.com/libp2p/go-stream-muxer/pull/19))\n- github.com/libp2p/go-tcp-transport:\n - Deprecate IPFS_REUSEPORT, use LIBP2P_TCP_REUSEPORT ([libp2p/go-tcp-transport#27](https://github.com/libp2p/go-tcp-transport/pull/27))\n- github.com/ipfs/go-unixfs:\n - unixfile: precalc dir size ([ipfs/go-unixfs#61](https://github.com/ipfs/go-unixfs/pull/61))\n - Archive refactor ([ipfs/go-unixfs#59](https://github.com/ipfs/go-unixfs/pull/59))\n - decouple the DAG traversal logic from the DAG reader (local branch) ([ipfs/go-unixfs#60](https://github.com/ipfs/go-unixfs/pull/60))\n - Unixfs: enforce refs on files when using nocopy ([ipfs/go-unixfs#56](https://github.com/ipfs/go-unixfs/pull/56))\n - Fix/handle overflow ([ipfs/go-unixfs#53](https://github.com/ipfs/go-unixfs/pull/53))\n - feat(Directory): Add EnumLinksAsync method ([ipfs/go-unixfs#39](https://github.com/ipfs/go-unixfs/pull/39))\n\n## v0.4.18 2018-10-26\n\nThis is probably one of the largest go-ipfs releases in recent history, 3 months\nin the making.\n\n### Features\n\nThe headline features this release are experimental QUIC support, the gossipsub\npubsub routing algorithm, pubsub message signing, and a refactored `ipfs p2p`\ncommand. However, that's just scratching the surface.\n\n#### QUIC\n\nFirst up, on the networking front, this release has also introduced experimental\nsupport for the QUIC protocol. QUIC is a new UDP-based network transport that\nsolves many of the long standing issues with TCP.\n\nFor us, this means (eventually):\n\n* **Fewer local resources.** TCP requires a file-descriptor per connection while\n QUIC (and most UDP based transports) can share a single file descriptor\n between all connections. This should allow us to dial faster and keep more\n connections open.\n* **Faster connection establishment.** When client authentication is included,\n QUIC has a three-way handshake like TCP. However, unlike TCP, this handshake\n brings us from all the way from 0 to a fully encrypted, authenticated, and\n multiplexed connection. In theory (not yet in practice), this should\n significantly reduce the latency of DHT queries.\n* **Behaves better on lossy networks.** When multiplexing multiple requests over\n a single TCP connection, a single dropped packet will bring the entire\n connection to a halt while the packet is re-transmitted. However, because QUIC\n handles multiplexing internally, dropping a single packets affects only the\n related stream.\n* **Better NAT traversal.** TL;DR: NAT hole-punching is significantly easier\n and, in many cases, more reliable with UDP than with TCP.\n\nHowever, we still have a long way to go. While we encourage users to test this,\nthe IETF QUIC protocol is still being actively developed and *will* change. You\ncan find instructions for enabling it\n[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#QUIC).\n\n#### Pubsub\n\nIn terms of pubsub, go-ipfs now supports the gossipsub routing algorithm and\nmessage signing.\n\nThe gossipsub routing algorithm is *significantly* more efficient than the\ncurrent floodsub routing algorithm. Even better, it's fully backwards compatible\nso you can enable it and still talk to nodes using the floodsub algorithm. You\ncan find instructions to enable gossipsub in go-ipfs\n[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#gossipsub).\n\nMessages are now signed by their authors. While signing has now been enabled by\ndefault, strict signature verification has not been and will not be for at least\none release (probably multiple) to avoid breaking existing applications. You can\nread about how to configure this feature\n[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#message-signing).\n\n#### Commands\n\nIn terms of new toys, this release introduces a new `ipfs cid` subcommand for\nworking with CIDs, a completely refactored `ipfs p2p` command, streaming name\nresolution, and complete inline block support.\n\nThe new `ipfs cid` command allows users to both inspect CIDs and convert them\nbetween various formats and versions. For example:\n\n```sh\n# Print out the CID metadata (prefix)\n> ipfs cid format -f %P QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o\ncidv0-protobuf-sha2-256-32\n\n# Get the hex sha256 hash from the CID.\n> ipfs cid format -b base16 -f '0x%D' QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o\n0x46d44814b9c5af141c3aaab7c05dc5e844ead5f91f12858b021eba45768b4c0e\n\n# Convert a base58 v0 CID to a base32 v1 CID.\n> ipfs cid base32 QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o\nbafybeicg2rebjoofv4kbyovkw7af3rpiitvnl6i7ckcywaq6xjcxnc2mby\n```\n\nThe refactored `ipfs p2p` command allows forwarding TCP streams through two IPFS\nnodes from one host to another. It's `ssh -L` but for IPFS. You can find\ndocumentation\n[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipfs-p2p).\nIt's still experimental but we don't expect too many breaking changes at this\npoint (it will very likely be stabilized in the next release). Quick summary of\nbreaking changes:\n\n* We don't stop listening for local (forwarded) connections after accepting a\n single connection.\n* `ipfs p2p stream ls` output now returns more useful output, first address is\n always the initiator address.\n* `ipfs p2p listener ls` is renamed to `ipfs p2p ls`\n* `ipfs p2p listener close` is renamed to `ipfs p2p close`\n* Protocol names have to be prefixed with `/x/` and are now just passed to\n libp2p as handler name. Previous version did this 'under the hood' and with\n `/p2p/` prefix. There is a `--allow-custom-protocol` flag which allows you\n to use any libp2p handler name.\n* `ipfs p2p listener open` and `ipfs p2p stream dial` got renamed:\n * `ipfs p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101`\n new becomes `ipfs p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101`\n * `ipfs p2p stream dial $NODE_A_PEERID p2p-test /ip4/127.0.0.1/tcp/10102`\n is now `ipfs p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /ipfs/$NODE_A_PEERID`\n\nThere is now a new flag for `ipfs name resolve` - `--stream`. When the command\nis invoked with the flag set, it will start returning results as soon as they\nare discovered in the DHT and other routing mechanisms. This enables certain\napplications to start prefetching/displaying data while the discovery is still\nrunning. Note that this command will likely return many outdated records\nbefore it finding and returning the latest. However, it will always return\n*valid* records (even if a bit stale).\n\nFinally, in the previous release, we added support for extracting blocks inlined\ninto CIDs. In this release, we've added support for creating these CIDs. You can\nnow run `ipfs add` with the `--inline` flag to inline blocks less than or equal\nto 32 bytes in length into a CID, instead of writing an actual block. This\nshould significantly reduce the size of filesystem trees with many empty\ndirectories and tiny files.\n\n#### IPNS\n\nYou can now publish and resolve paths with namespaces *other* than `/ipns` and\n`/ipfs` through IPNS. Critically, IPNS can now be used with IPLD paths (paths\nstarting with `/ipld`).\n\n#### WebUI\n\nFinally, this release includes the shiny [updated\nwebui](https://github.com/ipfs-shipyard/ipfs-webui). You can view it by\ninstalling go-ipfs and visiting http://localhost:5001/webui.\n\n### Performance\n\nThis release includes some significant performance improvements, both in terms\nof resource utilization and speed. This section will go into some technical\ndetails so feel free to skip it if you're just looking for shiny new features.\n\n#### Resource Utilization\n\nIn this release, we've (a) fixed a slow memory leak in libp2p and (b)\nsignificantly reduced the allocation load. Together, these should improve both\nmemory and CPU usage.\n\n##### Datastructures\n\nWe've changed two of our most frequently used datastructures, CIDs and\nMultiaddrs, to reduce allocation load.\n\nFirst, we now store CIDs *encode* as strings, instead of decoded in structs\n(behind pointers). In addition to being more compact, our `Cid` type is now a\nvalid `map` key so we no longer have to encode CIDs every time we want to use\nthem in a map/set. Allocations when inserting CIDs into maps/sets was showing up\nas a significant source of allocations under heavy load so this change should\nimprove memory usage.\n\nSecond, we've changed many of our multiaddr parsing/processing/formatting\nfunctions to allocate less. Much of our DHT related-work includes processing\nmultiaddrs so this should reduce CPU utilization when heavily using the DHT.\n\n##### Streams and Yamux\n\nStreams have always plagued us in terms of memory utilization. This was\npartially solved by introducing the connection manager, keeping our maximum\nconnection count to a reasonable number but they're still a major memory sink.\n\nThis release sees two improvements on this front:\n\n1. A memory [leak in identify](https://github.com/libp2p/go-libp2p/issues/419)\n has been fixed. This was slowly causing us to leak connections (locking up\n the memory used by the connections' streams).\n2. Yamux streams now use a buffer-pool backed, auto shrinking read buffer.\n Before, this read buffer would grow to its maximum size (a few megabytes) and\n never shrink but these buffers now shrink as they're emptied.\n\n#### Bitswap Performance\n\nBitswap will now pack *multiple* small blocks into a single message thanks\n[ipfs/go-bitswap#5](https://github.com/ipfs/go-bitswap/pull/5). While this won't\nhelp when transferring large files (with large blocks), this should help when\ntransferring many tiny files.\n\n### Refactors and Endeavors\n\nThis release saw yet another commands-library refactor, work towards the\nCoreAPI, and the first step towards reliable base32 CID support.\n\n#### Commands Lib\n\nWe've completely refactored our commands library (again). While it still needs\nquite a bit of work, it now requires significantly less boilerplate and should\nbe significantly more robust. The refactor immediately found two broken tests\nand probably fixed quite a few bugs around properly returning and handling\nerrors.\n\n#### CoreAPI\n\nCoreAPI is a new way to interact with IPFS from Go. While it's still not\nfinal, most things you can do via the CLI or HTTP interfaces, can now be done\nthrough the new API.\n\nCurrently there is only one implementation, backed by go-ipfs node, and there are\nplans to start http-api backed one soon. We are also looking into creating RPC\ninterface using this API, which could help performance in some use cases.\n\nYou can track progress in https://github.com/ipfs/go-ipfs/issues/4498\n\n#### IPLD paths\n\nWe introduced new path type which introduces distinction between IPLD and\nIPFS (unixfs) paths. From now on paths prefixed with `/ipld/` will always\nuse IPLD link traversal and `/ipfs/` will use unixfs path resolver, which\ntakes things like sharding into account.\n\nNote that this is only initial support and there likely are some bugs in\nhow the paths are handled internally, so consider this feature\nexperimental for now.\n\n#### CIDv1/Base32 Migration\n\nCurrently, IPFS is usually used in browsers by browsing to\n`https://SOME_GATEWAY/ipfs/CID/...`. There are two significant drawbacks to this\napproach:\n\n1. From a browser security standpoint, all IPFS \"sites\" will live under the same\n origin (SOME_GATEWAY).\n2. From a UX standpoint, this doesn't feel very \"native\" (even if the gateway is\n a local IPFS node).\n\nTo fix the security issue, we intend to switch IPFS gateway links\n`https://ipfs.io/ipfs/CID` to `https://CID.ipfs.dweb.link`. This way, the CID\nwill be a part of the\n[\"origin\"](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin) so\neach IPFS website will get a separate security origin.\n\nTo fix the UX issue, we've been working on adding support for `ipfs://CID/...`\nto web browsers through our\n[ipfs-companion](https://github.com/ipfs/ipfs-companion/) add-on and some new,\nexperimental extension APIs from Mozilla. This has the same effect of putting\nthe CID in the URL origin but has the added benefit of looking \"native\".\n\nUnfortunately, origins must be *case insensitive*. Currently, most CIDs users\nsee are *CIDv0* CIDs (those starting with `Qm`) which are *always* base58\nencoded and are therefore case-sensitive.\n\nFortunately, CIDv1 (the latest CID format) supports arbitrary bases using the\n[multibase](https://github.com/multiformats/multibase/) standard. Unfortunately,\nIPFS has always treated equivalent CIDv0 and CIDv1 CIDs as distinct. This means\nthat files added with CIDv0 CIDs (the default) can't be looked up using the\nequivalent CIDv1.\n\nThis release makes some significant progress towards solving this issue by\nintroducing two features:\n\n(1) The previous mentioned `ipfs cid base32` command for converting CID to a\ncase intensive encoding required by domain names. This command converts a CID to\nversion 1 and encodes it using base32.\n\n(2) A hack to allow locally looking up blocks associated with a CIDv0 CID using\nthe equivalent CIDv1 CID (or the reverse). This hack will eventually\nbe replaced with a multihash indexed blockstore, which is agnostic to both the\nCID version and multicodec content type.\n\n### go-ipfs changelog\n\nFeatures (i.e., users take heed):\n - gossipsub ([ipfs/go-ipfs#5373](https://github.com/ipfs/go-ipfs/pull/5373))\n - support /ipfs/CID in `ipfs dht findprovs` ([ipfs/go-ipfs#5329](https://github.com/ipfs/go-ipfs/pull/5329))\n - return a json object from config show ([ipfs/go-ipfs#5345](https://github.com/ipfs/go-ipfs/pull/5345))\n - Set filename in Content-Disposition if filename=x is passed in URI query ([ipfs/go-ipfs#4177](https://github.com/ipfs/go-ipfs/pull/4177))\n - Allow mfs files.write command to create parent directories ([ipfs/go-ipfs#5359](https://github.com/ipfs/go-ipfs/pull/5359))\n - Run DNS lookup for --api endpoint provided in CLI ([ipfs/go-ipfs#5372](https://github.com/ipfs/go-ipfs/pull/5372))\n - Add support for inlinling blocks into CIDs the id-hash ([ipfs/go-ipfs#5281](https://github.com/ipfs/go-ipfs/pull/5281))\n - depth limited refs -r ([ipfs/go-ipfs#5337](https://github.com/ipfs/go-ipfs/pull/5337))\n - remove bitswap unwant ([ipfs/go-ipfs#5308](https://github.com/ipfs/go-ipfs/pull/5308))\n - add experimental QUIC support ([ipfs/go-ipfs#5350](https://github.com/ipfs/go-ipfs/pull/5350))\n - add a --stdin-name flag for naming files from stdin ([ipfs/go-ipfs#5399](https://github.com/ipfs/go-ipfs/pull/5399))\n - Refactor `ipfs p2p` ([ipfs/go-ipfs#4929](https://github.com/ipfs/go-ipfs/pull/4929))\n - add dns support in`ipfs p2p forward` and refactor code ([ipfs/go-ipfs#5533](https://github.com/ipfs/go-ipfs/pull/5533))\n - feat(command): expose connection direction ([ipfs/go-ipfs#5457](https://github.com/ipfs/go-ipfs/pull/5457))\n - error when publishing ipns records without a running daemon ([ipfs/go-ipfs#5477](https://github.com/ipfs/go-ipfs/pull/5477))\n - feat(daemon): print version on start ([ipfs/go-ipfs#5503](https://github.com/ipfs/go-ipfs/pull/5503))\n - add quieter option to name publish ([ipfs/go-ipfs#5494](https://github.com/ipfs/go-ipfs/pull/5494))\n - Provide new \"cid\" sub-command. ([ipfs/go-ipfs#5385](https://github.com/ipfs/go-ipfs/pull/5385))\n - feat(command): add force flag for files rm ([ipfs/go-ipfs#5555](https://github.com/ipfs/go-ipfs/pull/5555))\n - Add support for datastore plugins ([ipfs/go-ipfs#5187](https://github.com/ipfs/go-ipfs/pull/5187))\n - files ls: append slash to directory names ([ipfs/go-ipfs#5605](https://github.com/ipfs/go-ipfs/pull/5605))\n - ipfs name resolve --stream ([ipfs/go-ipfs#5404](https://github.com/ipfs/go-ipfs/pull/5404))\n - update webui to 2.1.0 ([ipfs/go-ipfs#5627](https://github.com/ipfs/go-ipfs/pull/5627))\n - feat: add dry-run flag for config profile apply command ([ipfs/go-ipfs#5455](https://github.com/ipfs/go-ipfs/pull/5455))\n - configurable pubsub signing ([ipfs/go-ipfs#5647](https://github.com/ipfs/go-ipfs/pull/5647))\n\nFixes (i.e., users take note):\n - pin update fixes ([ipfs/go-ipfs#5265](https://github.com/ipfs/go-ipfs/pull/5265))\n - Fix inability to pin two things at once ([ipfs/go-ipfs#5512](https://github.com/ipfs/go-ipfs/pull/5512))\n - wait for all connections to close before exiting on shutdown. ([ipfs/go-ipfs#5322](https://github.com/ipfs/go-ipfs/pull/5322))\n - Fixed ipns address resolution in fuse unix mount ([ipfs/go-ipfs#5384](https://github.com/ipfs/go-ipfs/pull/5384))\n - core/commands/ls: wrap `NewDirectoryFromNode` error ([ipfs/go-ipfs#5166](https://github.com/ipfs/go-ipfs/pull/5166))\n - fix goroutine leaks in filestore.go ([ipfs/go-ipfs#5427](https://github.com/ipfs/go-ipfs/pull/5427))\n - move VersionOption after GatewayOption to fix #5422 ([ipfs/go-ipfs#5424](https://github.com/ipfs/go-ipfs/pull/5424))\n - fix(commands): fix filestore.go goroutine leak ([ipfs/go-ipfs#5439](https://github.com/ipfs/go-ipfs/pull/5439))\n - fix(commands): goroutine leaks in ping.go ([ipfs/go-ipfs#5444](https://github.com/ipfs/go-ipfs/pull/5444))\n - fix output of object command ([ipfs/go-ipfs#5459](https://github.com/ipfs/go-ipfs/pull/5459))\n - add warning when no bootstrap in config ([ipfs/go-ipfs#5445](https://github.com/ipfs/go-ipfs/pull/5445))\n - fix behaviour of key rename to same name ([ipfs/go-ipfs#5465](https://github.com/ipfs/go-ipfs/pull/5465))\n - fix(object): print object diff error ([ipfs/go-ipfs#5469](https://github.com/ipfs/go-ipfs/pull/5469))\n - fix(pin): goroutine leaks ([ipfs/go-ipfs#5453](https://github.com/ipfs/go-ipfs/pull/5453))\n - fix offline id bug ([ipfs/go-ipfs#5486](https://github.com/ipfs/go-ipfs/pull/5486))\n - files cp: improve flush error message ([ipfs/go-ipfs#5485](https://github.com/ipfs/go-ipfs/pull/5485))\n - resolve: fix unixfs resolution through sharded directories ([ipfs/go-ipfs#5484](https://github.com/ipfs/go-ipfs/pull/5484))\n - Switch name publish/resolve to coreapi ([ipfs/go-ipfs#5563](https://github.com/ipfs/go-ipfs/pull/5563))\n - use CoreAPI resolver everywhere (fixes sharded directory resolution) ([ipfs/go-ipfs#5492](https://github.com/ipfs/go-ipfs/pull/5492))\n - add pin lock in AddallPin function ([ipfs/go-ipfs#5506](https://github.com/ipfs/go-ipfs/pull/5506))\n - take the pinlock when updating pins ([ipfs/go-ipfs#5550](https://github.com/ipfs/go-ipfs/pull/5550))\n - fix(object): add support for raw leaves in object diff ([ipfs/go-ipfs#5472](https://github.com/ipfs/go-ipfs/pull/5472))\n - don't use the domain name as a filename in /ipns/a.com ([ipfs/go-ipfs#5564](https://github.com/ipfs/go-ipfs/pull/5564))\n - refactor(command): modify int to int64 ([ipfs/go-ipfs#5612](https://github.com/ipfs/go-ipfs/pull/5612))\n - fix(core): ipns config RecordLifetime panic ([ipfs/go-ipfs#5648](https://github.com/ipfs/go-ipfs/pull/5648))\n - simplify dag put and correctly take pin lock ([ipfs/go-ipfs#5667](https://github.com/ipfs/go-ipfs/pull/5667))\n - fix Prometheus concurrent map write bug ([ipfs/go-ipfs#5706](https://github.com/ipfs/go-ipfs/pull/5706))\n\nRegressions Fixes (fixes for bugs introduced since the last release):\n - namesys: properly attach path in name.Resolve ([ipfs/go-ipfs#5660](https://github.com/ipfs/go-ipfs/pull/5660))\n - fix(p2p): issue #5523 ([ipfs/go-ipfs#5529](https://github.com/ipfs/go-ipfs/pull/5529))\n - fix infinite loop in `stats bw` ([ipfs/go-ipfs#5598](https://github.com/ipfs/go-ipfs/pull/5598))\n - make warnings on no bootstrap peers less noisy ([ipfs/go-ipfs#5466](https://github.com/ipfs/go-ipfs/pull/5466))\n - fix two transport related bugs ([ipfs/go-ipfs#5417](https://github.com/ipfs/go-ipfs/pull/5417))\n - Fix pin ls output when hash is specified ([ipfs/go-ipfs#5699](https://github.com/ipfs/go-ipfs/pull/5699))\n - ping: switch to the ping service enabled in the libp2p constructor ([ipfs/go-ipfs#5698](https://github.com/ipfs/go-ipfs/pull/5698))\n - commands: fix a bunch of tiny commands-lib issues ([ipfs/go-ipfs#5697](https://github.com/ipfs/go-ipfs/pull/5697))\n - cleanup the ping command ([ipfs/go-ipfs#5680](https://github.com/ipfs/go-ipfs/pull/5680))\n - fix gossipsub goroutine explosion ([ipfs/go-ipfs#5688](https://github.com/ipfs/go-ipfs/pull/5688))\n - fix(cmd/gc): Run func does not return error when Emit func returns error ([ipfs/go-ipfs#5687](https://github.com/ipfs/go-ipfs/pull/5687))\n\nExtractions:\n - Extract bitswap to go-bitswap ([ipfs/go-ipfs#5294](https://github.com/ipfs/go-ipfs/pull/5294))\n - Extract blockservice and verifcid ([ipfs/go-ipfs#5296](https://github.com/ipfs/go-ipfs/pull/5296))\n - Extract merkledag package, move dagutils to top level ([ipfs/go-ipfs#5298](https://github.com/ipfs/go-ipfs/pull/5298))\n - Extract path and resolver ([ipfs/go-ipfs#5306](https://github.com/ipfs/go-ipfs/pull/5306))\n - Extract config package ([ipfs/go-ipfs#5277](https://github.com/ipfs/go-ipfs/pull/5277))\n - Extract unixfs and importers to go-unixfs ([ipfs/go-ipfs#5316](https://github.com/ipfs/go-ipfs/pull/5316))\n - delete unixfs code... ([ipfs/go-ipfs#5319](https://github.com/ipfs/go-ipfs/pull/5319))\n - Extract /mfs to github.com/ipfs/go-mfs ([ipfs/go-ipfs#5391](https://github.com/ipfs/go-ipfs/pull/5391))\n - re-format log output as ndjson ([ipfs/go-ipfs#5708](https://github.com/ipfs/go-ipfs/pull/5708))\n - error on resolving non-terminal paths ([ipfs/go-ipfs#5705](https://github.com/ipfs/go-ipfs/pull/5705))\n\nDocumentation:\n - document the fact that we now publish releases on GitHub ([ipfs/go-ipfs#5301](https://github.com/ipfs/go-ipfs/pull/5301))\n - docs: add url to dev weekly sync to the README ([ipfs/go-ipfs#5371](https://github.com/ipfs/go-ipfs/pull/5371))\n - docs: README refresh, add cli-http-api-core diagram ([ipfs/go-ipfs#5396](https://github.com/ipfs/go-ipfs/pull/5396))\n - add some basic gateway documentation ([ipfs/go-ipfs#5393](https://github.com/ipfs/go-ipfs/pull/5393))\n - fix the default gateway port ([ipfs/go-ipfs#5419](https://github.com/ipfs/go-ipfs/pull/5419))\n - fix order of events in the release process ([ipfs/go-ipfs#5434](https://github.com/ipfs/go-ipfs/pull/5434))\n - docs: add some minimal read-only API documentation ([ipfs/go-ipfs#5437](https://github.com/ipfs/go-ipfs/pull/5437))\n - feat: use package-table ([ipfs/go-ipfs#5395](https://github.com/ipfs/go-ipfs/pull/5395))\n - link to go-{libp2p,ipld} package tables ([ipfs/go-ipfs#5446](https://github.com/ipfs/go-ipfs/pull/5446))\n - api: fix outdated HTTPHeaders config documentation ([ipfs/go-ipfs#5451](https://github.com/ipfs/go-ipfs/pull/5451))\n - add version, usage, and planning info for urlstore ([ipfs/go-ipfs#5552](https://github.com/ipfs/go-ipfs/pull/5552))\n - debug-guide.md added memory statistics command ([ipfs/go-ipfs#5546](https://github.com/ipfs/go-ipfs/pull/5546))\n - Change to point to combined go contributing guidelines ([ipfs/go-ipfs#5607](https://github.com/ipfs/go-ipfs/pull/5607))\n - docs: Update link format ([ipfs/go-ipfs#5617](https://github.com/ipfs/go-ipfs/pull/5617))\n - Fix link in readme ([ipfs/go-ipfs#5632](https://github.com/ipfs/go-ipfs/pull/5632))\n - docs: add a note for dns command ([ipfs/go-ipfs#5629](https://github.com/ipfs/go-ipfs/pull/5629))\n - Dockerfile: Specifies comments on exposed ports ([ipfs/go-ipfs#5615](https://github.com/ipfs/go-ipfs/pull/5615))\n - document pubsub message signing ([ipfs/go-ipfs#5669](https://github.com/ipfs/go-ipfs/pull/5669))\n\nTesting:\n - Include cid-fmt binary in test/bin. ([ipfs/go-ipfs#5297](https://github.com/ipfs/go-ipfs/pull/5297))\n - wait for the nodes to fully stop ([ipfs/go-ipfs#5315](https://github.com/ipfs/go-ipfs/pull/5315))\n - apply timeout for build steps after getting node ([ipfs/go-ipfs#5313](https://github.com/ipfs/go-ipfs/pull/5313))\n - ci: check for gx deps dupes ([ipfs/go-ipfs#5338](https://github.com/ipfs/go-ipfs/pull/5338))\n - ci: call cleanWs after each step ([ipfs/go-ipfs#5374](https://github.com/ipfs/go-ipfs/pull/5374))\n - add correct test for GC completeness ([ipfs/go-ipfs#5364](https://github.com/ipfs/go-ipfs/pull/5364))\n - fix the urlstore tests ([ipfs/go-ipfs#5397](https://github.com/ipfs/go-ipfs/pull/5397))\n - improve gateway options test ([ipfs/go-ipfs#5433](https://github.com/ipfs/go-ipfs/pull/5433))\n - coreapi name: Increase test swarm size ([ipfs/go-ipfs#5481](https://github.com/ipfs/go-ipfs/pull/5481))\n - fix fuse unmount test ([ipfs/go-ipfs#5476](https://github.com/ipfs/go-ipfs/pull/5476))\n - test(add): add test for issue \\#5456 ([ipfs/go-ipfs#5493](https://github.com/ipfs/go-ipfs/pull/5493))\n - fixed tests of raised fd limits ([ipfs/go-ipfs#5496](https://github.com/ipfs/go-ipfs/pull/5496))\n - pprof: create HTTP endpoint for setting MutexProfileFraction ([ipfs/go-ipfs#5527](https://github.com/ipfs/go-ipfs/pull/5527))\n - fix(command):update `add --chunker` test ([ipfs/go-ipfs#5571](https://github.com/ipfs/go-ipfs/pull/5571))\n - switch to go 1.11 ([ipfs/go-ipfs#5483](https://github.com/ipfs/go-ipfs/pull/5483))\n - fix: sharness race in directory_size if file is removed ([ipfs/go-ipfs#5586](https://github.com/ipfs/go-ipfs/pull/5586))\n - Bump Go versions and use '.x' to always get latest minor versions ([ipfs/go-ipfs#5682](https://github.com/ipfs/go-ipfs/pull/5682))\n - add rabin min error test ([ipfs/go-ipfs#5449](https://github.com/ipfs/go-ipfs/pull/5449))\n - Use CircleCI 2.0 ([ipfs/go-ipfs#5691](https://github.com/ipfs/go-ipfs/pull/5691))\n\nInternal:\n - Add ability to retrieve blocks even if given using a different CID version ([ipfs/go-ipfs#5285](https://github.com/ipfs/go-ipfs/pull/5285))\n - update gogo-protobuf ([ipfs/go-ipfs#5355](https://github.com/ipfs/go-ipfs/pull/5355))\n - update protobuf files in go-ipfs ([ipfs/go-ipfs#5356](https://github.com/ipfs/go-ipfs/pull/5356))\n - string-backed CIDs ([ipfs/go-ipfs#5441](https://github.com/ipfs/go-ipfs/pull/5441))\n - commands: switch object to CoreAPI ([ipfs/go-ipfs#4643](https://github.com/ipfs/go-ipfs/pull/4643))\n - coreapi: dag: Batching interface ([ipfs/go-ipfs#5340](https://github.com/ipfs/go-ipfs/pull/5340))\n - key cmd: Refactor to use coreapi ([ipfs/go-ipfs#5339](https://github.com/ipfs/go-ipfs/pull/5339))\n - coreapi: DHT API ([ipfs/go-ipfs#4804](https://github.com/ipfs/go-ipfs/pull/4804))\n - block cmd: Use coreapi ([ipfs/go-ipfs#5331](https://github.com/ipfs/go-ipfs/pull/5331))\n - mk: embed CurrentCommit in the right place ([ipfs/go-ipfs#5507](https://github.com/ipfs/go-ipfs/pull/5507))\n - added binary executable files to .dockerignore ([ipfs/go-ipfs#5544](https://github.com/ipfs/go-ipfs/pull/5544))\n - Add sessions when fetching MerkleDAG in LS ([ipfs/go-ipfs#5509](https://github.com/ipfs/go-ipfs/pull/5509))\n - coreapi: Swarm API ([ipfs/go-ipfs#4803](https://github.com/ipfs/go-ipfs/pull/4803))\n - coreapi swarm: unify impl type with other apis ([ipfs/go-ipfs#5551](https://github.com/ipfs/go-ipfs/pull/5551))\n - Refactor UnixFS CoreAPI ([ipfs/go-ipfs#5501](https://github.com/ipfs/go-ipfs/pull/5501))\n - coreapi: PubSub API ([ipfs/go-ipfs#4805](https://github.com/ipfs/go-ipfs/pull/4805))\n - fix: maketarball.sh for OSX ([ipfs/go-ipfs#5575](https://github.com/ipfs/go-ipfs/pull/5575))\n - test the correct return value when checking directory size ([ipfs/go-ipfs#5580](https://github.com/ipfs/go-ipfs/pull/5580))\n - coreapi unixfs: remove Cat ([ipfs/go-ipfs#5574](https://github.com/ipfs/go-ipfs/pull/5574))\n - Explicitly use BufferedDAG after removing Batch from importers ([ipfs/go-ipfs#5626](https://github.com/ipfs/go-ipfs/pull/5626))\n\nCleanup:\n - Fix some weird code in core/coreunix/add.go ([ipfs/go-ipfs#5354](https://github.com/ipfs/go-ipfs/pull/5354))\n - name cmd: move subcommands to subdirectory ([ipfs/go-ipfs#5392](https://github.com/ipfs/go-ipfs/pull/5392))\n - directly parse peer IDs as peer IDs ([ipfs/go-ipfs#5409](https://github.com/ipfs/go-ipfs/pull/5409))\n - don't bother caching if we're using a nil repo ([ipfs/go-ipfs#5414](https://github.com/ipfs/go-ipfs/pull/5414))\n - object:refactor data encode error ([ipfs/go-ipfs#5426](https://github.com/ipfs/go-ipfs/pull/5426))\n - remove Godeps ([ipfs/go-ipfs#5440](https://github.com/ipfs/go-ipfs/pull/5440))\n - update for the go-ipfs-cmds refactor ([ipfs/go-ipfs#5035](https://github.com/ipfs/go-ipfs/pull/5035))\n - fix(unixfs): issue #5217 (Avoid use of `pb.Data`) ([ipfs/go-ipfs#5505](https://github.com/ipfs/go-ipfs/pull/5505))\n - fix(unixfs): issue #5055 ([ipfs/go-ipfs#5525](https://github.com/ipfs/go-ipfs/pull/5525))\n - add offline id test #4978 and refactor command code ([ipfs/go-ipfs#5562](https://github.com/ipfs/go-ipfs/pull/5562))\n - refact(command): replace option name with const string ([ipfs/go-ipfs#5642](https://github.com/ipfs/go-ipfs/pull/5642))\n - remove p2p-circuit addr hack in ipfs swarm peers ([ipfs/go-ipfs#5645](https://github.com/ipfs/go-ipfs/pull/5645))\n - refactor(commands/id): use new command ([ipfs/go-ipfs#5646](https://github.com/ipfs/go-ipfs/pull/5646))\n - object patch rm-link: change arg from 'link' to 'name' ([ipfs/go-ipfs#5638](https://github.com/ipfs/go-ipfs/pull/5638))\n - refactor(cmds): use new cmds lib in version, tar and dns ([ipfs/go-ipfs#5650](https://github.com/ipfs/go-ipfs/pull/5650))\n - cmds/dag: use new cmds lib ([ipfs/go-ipfs#5662](https://github.com/ipfs/go-ipfs/pull/5662))\n - commands/ping: use new cmds lib ([ipfs/go-ipfs#5675](https://github.com/ipfs/go-ipfs/pull/5675))\n\n### related changelogs\n\nChanges to sub-packages go-ipfs depends on. This *does not* include libp2p or multiformats.\n\ngithub.com/ipfs/go-log\n - update gogo protobuf ([ipfs/go-log#39](https://github.com/ipfs/go-log/pull/39))\n - rename the protobuf to loggabletracer ([ipfs/go-log#41](https://github.com/ipfs/go-log/pull/41))\n - protect loggers with rwmutex ([ipfs/go-log#44](https://github.com/ipfs/go-log/pull/44))\n - make logging prettier ([ipfs/go-log#45](https://github.com/ipfs/go-log/pull/45))\n - add env vars for logging to file and syslog ([ipfs/go-log#46](https://github.com/ipfs/go-log/pull/46))\n - remove syslogger ([ipfs/go-log#47](https://github.com/ipfs/go-log/pull/47))\n\ngithub.com/ipfs/go-datastore\n - implement DiskUsage for the rest of the datastores ([ipfs/go-datastore#86](https://github.com/ipfs/go-datastore/pull/86))\n - switch to google's uuid library ([ipfs/go-datastore#89](https://github.com/ipfs/go-datastore/pull/89))\n - return ErrNotFound from the NullDatastore instead of nil, nil ([ipfs/go-datastore#92](https://github.com/ipfs/go-datastore/pull/92))\n - Add TTL and Transactional interfaces ([ipfs/go-datastore#91](https://github.com/ipfs/go-datastore/pull/91))\n - improve testing ([ipfs/go-datastore#93](https://github.com/ipfs/go-datastore/pull/93))\n - Add support for querying entry expiration ([ipfs/go-datastore#96](https://github.com/ipfs/go-datastore/pull/96))\n - Allow ds.NewTransaction() to return an error ([ipfs/go-datastore#98](https://github.com/ipfs/go-datastore/pull/98))\n - add a GetSize method ([ipfs/go-datastore#99](https://github.com/ipfs/go-datastore/pull/99))\n\ngithub.com/ipfs/go-cid\n - Add tests for Set type ([ipfs/go-cid#63](https://github.com/ipfs/go-cid/pull/63))\n - Create new Builder interface for creating CIDs. ([ipfs/go-cid#53](https://github.com/ipfs/go-cid/pull/53))\n - cid-fmt enhancements ([ipfs/go-cid#61](https://github.com/ipfs/go-cid/pull/61))\n - add String benchmark ([ipfs/go-cid#44](https://github.com/ipfs/go-cid/pull/44))\n - add a streaming CID set ([ipfs/go-cid#67](https://github.com/ipfs/go-cid/pull/67))\n - Extract non-core functionality from go-cid into go-cidutil ([ipfs/go-cid#69](https://github.com/ipfs/go-cid/pull/69))\n - cid implementation research ([ipfs/go-cid#70](https://github.com/ipfs/go-cid/pull/70))\n - cid implementation variations++ ([ipfs/go-cid#72](https://github.com/ipfs/go-cid/pull/72))\n - Create a new Encode method that is like StringOfBase but never errors ([ipfs/go-cid#60](https://github.com/ipfs/go-cid/pull/60))\n - add codecs for Dash blocks, tx ([ipfs/go-cid#78](https://github.com/ipfs/go-cid/pull/78))\n\ngithub.com/ipfs/go-ds-flatfs\n - check error before defer-removing disk usage file ([ipfs/go-ds-flatfs#47](https://github.com/ipfs/go-ds-flatfs/pull/47))\n - add GetSize function ([ipfs/go-ds-flatfs#48](https://github.com/ipfs/go-ds-flatfs/pull/48))\n\ngithub.com/ipfs/go-ds-measure\n - ([ipfs/go-ds-measure#](https://github.com/ipfs/go-ds-measure/pull/))\n\ngithub.com/ipfs/go-ds-leveldb\n - recover datastore on corruption ([ipfs/go-ds-leveldb#15](https://github.com/ipfs/go-ds-leveldb/pull/15))\n - Add transactional support to leveldb datastore. ([ipfs/go-ds-leveldb#17](https://github.com/ipfs/go-ds-leveldb/pull/17))\n - implement GetSize ([ipfs/go-ds-leveldb#18](https://github.com/ipfs/go-ds-leveldb/pull/18))\n\ngithub.com/ipfs/go-metrics-prometheus\n - use an existing metric when it has already been registered ([ipfs/go-metrics-prometheus#1](https://github.com/ipfs/go-metrics-prometheus/pull/1))\n\ngithub.com/ipfs/go-metrics-interface\n - update the counter interface to match Prometheus ([ipfs/go-metrics-interface#2](https://github.com/ipfs/go-metrics-interface/pull/2))\n\ngithub.com/ipfs/go-ipld-format\n - add copy dagservice function ([ipfs/go-ipld-format#41](https://github.com/ipfs/go-ipld-format/pull/41))\n\ngithub.com/ipfs/go-ipld-cbor\n - Refactor to refmt ([ipfs/go-ipld-cbor#30](https://github.com/ipfs/go-ipld-cbor/pull/30))\n - import changes from the filecoin branch ([ipfs/go-ipld-cbor#41](https://github.com/ipfs/go-ipld-cbor/pull/41))\n - register the BitIntAtlasEntry for the tests ([ipfs/go-ipld-cbor#43](https://github.com/ipfs/go-ipld-cbor/pull/43))\n - attempt to allocate a bit less ([ipfs/go-ipld-cbor#45](https://github.com/ipfs/go-ipld-cbor/pull/45))\n\ngithub.com/ipfs/go-ipfs-cmds\n - check if we can decode an error before trying ([ipfs/go-ipfs-cmds#108](https://github.com/ipfs/go-ipfs-cmds/pull/108))\n - fix(option): print error message for error timeout option ([ipfs/go-ipfs-cmds#118](https://github.com/ipfs/go-ipfs-cmds/pull/118))\n - Create Jenkinsfile ([ipfs/go-ipfs-cmds#89](https://github.com/ipfs/go-ipfs-cmds/pull/89))\n - fix(add): refer to ipfs issue #5456 ([ipfs/go-ipfs-cmds#121](https://github.com/ipfs/go-ipfs-cmds/pull/121))\n - commands refactor 2.0 ([ipfs/go-ipfs-cmds#112](https://github.com/ipfs/go-ipfs-cmds/pull/112))\n - always assign keks to review new PRs ([ipfs/go-ipfs-cmds#123](https://github.com/ipfs/go-ipfs-cmds/pull/123))\n - extract go-ipfs-files ([ipfs/go-ipfs-cmds#125](https://github.com/ipfs/go-ipfs-cmds/pull/125))\n - split the value encoder and the error encoder ([ipfs/go-ipfs-cmds#128](https://github.com/ipfs/go-ipfs-cmds/pull/128))\n\ngithub.com/ipfs/go-ipfs-cmdkit\n - all: gofmt ([ipfs/go-ipfs-cmdkit#22](https://github.com/ipfs/go-ipfs-cmdkit/pull/22))\n - add standard ci scripts ([ipfs/go-ipfs-cmdkit#23](https://github.com/ipfs/go-ipfs-cmdkit/pull/23))\n - only count size for regular files ([ipfs/go-ipfs-cmdkit#25](https://github.com/ipfs/go-ipfs-cmdkit/pull/25))\n - Create Jenkinsfile ([ipfs/go-ipfs-cmdkit#16](https://github.com/ipfs/go-ipfs-cmdkit/pull/16))\n - Feat: add WebFile File implementation. ([ipfs/go-ipfs-cmdkit#26](https://github.com/ipfs/go-ipfs-cmdkit/pull/26))\n - feat(type): fix issue #28 ([ipfs/go-ipfs-cmdkit#29](https://github.com/ipfs/go-ipfs-cmdkit/pull/29))\n - Extract files package ([ipfs/go-ipfs-cmdkit#31](https://github.com/ipfs/go-ipfs-cmdkit/pull/31))\n\ngithub.com/ipfs/go-ds-badger\n - update protobuf ([ipfs/go-ds-badger#26](https://github.com/ipfs/go-ds-badger/pull/26))\n - exported type datastore => Datastore ([ipfs/go-ds-badger#1](https://github.com/ipfs/go-ds-badger/pull/1))\n - using exported Datastore type ([ipfs/go-ds-badger#2](https://github.com/ipfs/go-ds-badger/pull/2))\n - exported type datastore => Datastore ([ipfs/go-ds-badger#28](https://github.com/ipfs/go-ds-badger/pull/28))\n - Implement new TxDatastore and Txn interfaces ([ipfs/go-ds-badger#27](https://github.com/ipfs/go-ds-badger/pull/27))\n - Avoid discarding transaction too early in queries ([ipfs/go-ds-badger#31](https://github.com/ipfs/go-ds-badger/pull/31))\n - Ability to get entry expirations ([ipfs/go-ds-badger#32](https://github.com/ipfs/go-ds-badger/pull/32))\n - Update badger to 2.8.0 ([ipfs/go-ds-badger#33](https://github.com/ipfs/go-ds-badger/pull/33))\n - ds.NewTransaction() now returns an error parameter ([ipfs/go-ds-badger#36](https://github.com/ipfs/go-ds-badger/pull/36))\n - make has faster ([ipfs/go-ds-badger#37](https://github.com/ipfs/go-ds-badger/pull/37))\n - Implement GetSize and update badger ([ipfs/go-ds-badger#38](https://github.com/ipfs/go-ds-badger/pull/38))\n\ngithub.com/ipfs/go-ipfs-addr\n - Remove dependency on libp2p-circuit ([ipfs/go-ipfs-addr#7](https://github.com/ipfs/go-ipfs-addr/pull/7))\n\ngithub.com/ipfs/go-ipfs-chunker\n - return err when rabin min less than 16 ([ipfs/go-ipfs-chunker#3](https://github.com/ipfs/go-ipfs-chunker/pull/3))\n - switch to go-buffer-pool ([ipfs/go-ipfs-chunker#8](https://github.com/ipfs/go-ipfs-chunker/pull/8))\n - fix size-0 chunker bug ([ipfs/go-ipfs-chunker#9](https://github.com/ipfs/go-ipfs-chunker/pull/9))\n\ngithub.com/ipfs/go-ipfs-routing\n - update protobuf ([ipfs/go-ipfs-routing#8](https://github.com/ipfs/go-ipfs-routing/pull/8))\n - Implement SearchValue ([ipfs/go-ipfs-routing#12](https://github.com/ipfs/go-ipfs-routing/pull/12))\n\ngithub.com/ipfs/go-ipfs-blockstore\n - blockstore: Adding Stat method to map from Cid to BlockSize ([ipfs/go-ipfs-blockstore#5](https://github.com/ipfs/go-ipfs-blockstore/pull/5))\n - correctly convert the datastore not found errors ([ipfs/go-ipfs-blockstore#10](https://github.com/ipfs/go-ipfs-blockstore/pull/10))\n - Fix typo: Change 'should not' to 'should' ([ipfs/go-ipfs-blockstore#14](https://github.com/ipfs/go-ipfs-blockstore/pull/14))\n - fix test race condition ([ipfs/go-ipfs-blockstore#9](https://github.com/ipfs/go-ipfs-blockstore/pull/9))\n - make arccache.GetSize return ErrNotFound when not found ([ipfs/go-ipfs-blockstore#16](https://github.com/ipfs/go-ipfs-blockstore/pull/16))\n - use datastore.GetSize ([ipfs/go-ipfs-blockstore#17](https://github.com/ipfs/go-ipfs-blockstore/pull/17))\n\ngithub.com/ipfs/go-ipns\n - update gogo protobuf ([ipfs/go-ipns#16](https://github.com/ipfs/go-ipns/pull/16))\n - use new ExtractPublicKey signature ([ipfs/go-ipns#17](https://github.com/ipfs/go-ipns/pull/17))\n\ngithub.com/ipfs/go-bitswap\n - update gogo protobuf ([ipfs/go-bitswap#2](https://github.com/ipfs/go-bitswap/pull/2))\n - ci: add jenkins ([ipfs/go-bitswap#9](https://github.com/ipfs/go-bitswap/pull/9))\n - bitswap: Bitswap now sends multiple blocks per message ([ipfs/go-bitswap#5](https://github.com/ipfs/go-bitswap/pull/5))\n - reduce allocations ([ipfs/go-bitswap#12](https://github.com/ipfs/go-bitswap/pull/12))\n - buffer writes ([ipfs/go-bitswap#15](https://github.com/ipfs/go-bitswap/pull/15))\n - delay finding providers ([ipfs/go-bitswap#17](https://github.com/ipfs/go-bitswap/pull/17))\ngithub.com/ipfs/go-blockservice\n - Avoid allocating a session unless we need it ([ipfs/go-blockservice#6](https://github.com/ipfs/go-blockservice/pull/6))\n\ngithub.com/ipfs/go-cidutil\n - add a utility method for sorting CID slices ([ipfs/go-cidutil#5](https://github.com/ipfs/go-cidutil/pull/5))\n\ngithub.com/ipfs/go-ipfs-config\n - Add pubsub configuration options ([ipfs/go-ipfs-config#3](https://github.com/ipfs/go-ipfs-config/pull/3))\n - add QUIC experiment ([ipfs/go-ipfs-config#4](https://github.com/ipfs/go-ipfs-config/pull/4))\n - Add Gateway.APICommands for /api allowlists ([ipfs/go-ipfs-config#10](https://github.com/ipfs/go-ipfs-config/pull/10))\n - allow multiple API/Gateway addresses ([ipfs/go-ipfs-config#11](https://github.com/ipfs/go-ipfs-config/pull/11))\n - Fix handling of null strings ([ipfs/go-ipfs-config#12](https://github.com/ipfs/go-ipfs-config/pull/12))\n - add experiment for p2p http proxy ([ipfs/go-ipfs-config#13](https://github.com/ipfs/go-ipfs-config/pull/13))\n - add message signing config options ([ipfs/go-ipfs-config#18](https://github.com/ipfs/go-ipfs-config/pull/18))\n\ngithub.com/ipfs/go-merkledag\n - Add FetchGraphWithDepthLimit to specify depth-limited graph fetching. ([ipfs/go-merkledag#2](https://github.com/ipfs/go-merkledag/pull/2))\n - update gogo protobuf ([ipfs/go-merkledag#4](https://github.com/ipfs/go-merkledag/pull/4))\n - Update to use new Builder interface for creating CIDs. ([ipfs/go-merkledag#6](https://github.com/ipfs/go-merkledag/pull/6))\n - perf: avoid allocations when filtering nodes ([ipfs/go-merkledag#11](https://github.com/ipfs/go-merkledag/pull/11))\n\ngithub.com/ipfs/go-mfs\n - fix(unixfs): issue #6 ([ipfs/go-mfs#7](https://github.com/ipfs/go-mfs/pull/7))\n - fix(type): issue #13 ([ipfs/go-mfs#14](https://github.com/ipfs/go-mfs/pull/14))\n\ngithub.com/ipfs/go-path\n - fix: don't dag.Get in ResolveToLastNode when not needed ([ipfs/go-path#1](https://github.com/ipfs/go-path/pull/1))\n\ngithub.com/ipfs/go-unixfs\n - update gogo protobuf ([ipfs/go-unixfs#6](https://github.com/ipfs/go-unixfs/pull/6))\n - Update to use new Builder interface for creating CIDs. ([ipfs/go-unixfs#7](https://github.com/ipfs/go-unixfs/pull/7))\n - nit: make dagTruncate a method on DagModifier ([ipfs/go-unixfs#13](https://github.com/ipfs/go-unixfs/pull/13))\n - fix(fsnode): issue #17 ([ipfs/go-unixfs#18](https://github.com/ipfs/go-unixfs/pull/18))\n - Use EnumerateChildrenAsync in for enumerating HAMT links ([ipfs/go-unixfs#19](https://github.com/ipfs/go-unixfs/pull/19))\n\n## v0.4.17 2018-07-27\n\nIpfs 0.4.17 is a quick release to fix a major performance regression in bitswap\n(mostly affecting go-ipfs -> js-ipfs transfers). However, while motivated by\nthis fix, this release contains a few other goodies that will excite some users.\n\nThe headline feature in this release is [urlstore][] support. Urlstore is a\ngeneralization of the filestore backend that can fetch file blocks from remote\nURLs on-demand instead of storing them in the local datastore.\n\nAdditionally, we've added support for extracting inline blocks from CIDs (blocks\ninlined into CIDs using the identity hash function). However, go-ipfs won't yet\n*create* such CIDs so you're unlikely to see any in the wild.\n\n[urlstore]: https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipfs-urlstore\n\nFeatures:\n\n* URLStore ([ipfs/go-ipfs#4896](https://github.com/ipfs/go-ipfs/pull/4896))\n* Add trickle-dag support to the urlstore ([ipfs/go-ipfs#5245](https://github.com/ipfs/go-ipfs/pull/5245)).\n* Allow specifying how the data field in the `object get` is encoded ([ipfs/go-ipfs#5139](https://github.com/ipfs/go-ipfs/pull/5139))\n* Add a `-U` flag to `files ls` to disable sorting ([ipfs/go-ipfs#5219](https://github.com/ipfs/go-ipfs/pull/5219))\n* Add an efficient `--size-only` flag to the `repo stat` ([ipfs/go-ipfs#5010](https://github.com/ipfs/go-ipfs/pull/5010))\n* Inline blocks in CIDs ([ipfs/go-ipfs#5117](https://github.com/ipfs/go-ipfs/pull/5117))\n\nChanges/Fixes:\n\n* Make `ipfs files ls -l` correctly report the hash and size of files ([ipfs/go-ipfs#5045](https://github.com/ipfs/go-ipfs/pull/5045))\n* Fix sorting of `files ls` ([ipfs/go-ipfs#5219](https://github.com/ipfs/go-ipfs/pull/5219))\n* Improve prefetching in `ipfs cat` and related commands ([ipfs/go-ipfs#5162](https://github.com/ipfs/go-ipfs/pull/5162))\n* Better error message when `ipfs cp` fails ([ipfs/go-ipfs#5218](https://github.com/ipfs/go-ipfs/pull/5218))\n* Don't wait for the peer to close it's end of a bitswap stream before considering the block \"sent\" ([ipfs/go-ipfs#5258](https://github.com/ipfs/go-ipfs/pull/5258))\n* Fix resolving links in sharded directories via the gateway ([ipfs/go-ipfs#5271](https://github.com/ipfs/go-ipfs/pull/5271))\n* Fix building when there's a space in the current directory ([ipfs/go-ipfs#5261](https://github.com/ipfs/go-ipfs/pull/5261))\n\nDocumentation:\n\n* Improve documentation about the bloomfilter config options ([ipfs/go-ipfs#4924](https://github.com/ipfs/go-ipfs/pull/4924))\n\nGeneral refactorings and internal bug fixes:\n\n* Remove the `Offset()` method from the DAGReader ([ipfs/go-ipfs#5190](https://github.com/ipfs/go-ipfs/pull/5190))\n* Fix TestLargeWriteChunks seek behavior ([ipfs/go-ipfs#5276](https://github.com/ipfs/go-ipfs/pull/5276))\n* Add a build tag to disable dynamic plugins ([ipfs/go-ipfs#5274](https://github.com/ipfs/go-ipfs/pull/5274))\n* Use FSNode instead of the Protobuf structure in PBDagReader ([ipfs/go-ipfs#5189](https://github.com/ipfs/go-ipfs/pull/5189))\n* Remove support for non-directory MFS roots ([ipfs/go-ipfs#5170](https://github.com/ipfs/go-ipfs/pull/5170))\n* Remove `UnixfsNode` from the balanced builder ([ipfs/go-ipfs#5118](https://github.com/ipfs/go-ipfs/pull/5118))\n* Fix truncating files (internal) when already at the correct size ([ipfs/go-ipfs#5253](https://github.com/ipfs/go-ipfs/pull/5253))\n* Fix `dagTruncate` (internal) to preserve the node type ([ipfs/go-ipfs#5216](https://github.com/ipfs/go-ipfs/pull/5216))\n* Add an internal interface for unixfs directories ([ipfs/go-ipfs#5160](https://github.com/ipfs/go-ipfs/pull/5160))\n* Refactor the CoreAPI path types and interfaces ([ipfs/go-ipfs#4672](https://github.com/ipfs/go-ipfs/pull/4672))\n* Refactor `precalcNextBuf` in the dag reader ([ipfs/go-ipfs#5237](https://github.com/ipfs/go-ipfs/pull/5237))\n* Update a bunch of dependencies that haven't been updated for a while ([ipfs/go-ipfs#5268](https://github.com/ipfs/go-ipfs/pull/5268))\n\n## v0.4.16 2018-07-13\n\nIpfs 0.4.16 is a fairly small release in terms of changes to the ipfs codebase,\nbut it contains a huge amount of changes and improvements from the libraries we\ndepend on, notably libp2p.\n\nThis release includes small a repo migration to account for some changes to the\nDHT. It should only take a second to run but, depending on your configuration,\nyou may need to run it manually.\n\nYou can run a migration by either:\n\n1. Selecting \"Yes\" when the daemon prompts you to migrate.\n2. Running the daemon with the `--migrate=true` flag.\n3. Manually [running](https://github.com/ipfs/fs-repo-migrations/blob/master/run.md#running-repo-migrations) the migration.\n\n### Libp2p\n\nThis version of ipfs contains the changes made in libp2p from v5.0.14 through\nv6.0.5. In that time, we have made significant changes to the codebase to allow\nfor easier integration of future transports and modules along with the usual\nperformance and reliability improvements. You can find many of these\nimprovements in the libp2p 6.0 [release blog\npost](https://ipfs.io/blog/39-go-libp2p-6-0-0/).\n\nThe primary motivation for this refactor was adding support for network\ntransports like QUIC that have built-in support for encryption, authentication,\nand stream multiplexing. It will also allow us to plug-in new security\ntransports (like TLS) without hard-coding them.\n\nFor example, our [QUIC\ntransport](https://github.com/libp2p/go-libp2p-quic-transport) currently works,\nand can be plugged into libp2p manually (though note that it is still\nexperimental, as the upstream spec is still in flux). Further work is needed to\nmake enabling this inside ipfs easy and not require recompilation.\n\nOn the user-visible side of things, we've improved our dialing logic and\ntimeouts. We now abort dials to local subnets after 5 seconds and abort all\ndials if the TCP handshake takes longer than 5 seconds. This should\nsignificantly improve performance in some cases as we limit the number of\nconcurrent dials and slow dials to non-responsive peers have been known to clog\nthe dialer, blocking dials to reachable peers. Importantly, this should improve\nDHT performance as it tends to spend a disproportional amount of time connecting\nto peers.\n\nWe have also made a few noticeable changes to the DHT: we've significantly\nimproved the chances of finding a value on the DHT, tightened up some of our\nvalidation logic, and fixed some issues that should reduce traffic to nodes\nrunning in dhtclient mode over time.\n\nOf these, the first one will likely see the most impact. In the past, when\nputting a value (e.g., an IPNS entry) into the DHT, we'd try to put the value to\nK peers (where K for us is 20). However, we'd often fail to connect to many of\nthese peers so we'd end up putting the value to significantly fewer than K\npeers. We now try to put the value to the K peers we can actually connect to.\n\nFinally, we've fixed JavaScript interoperability in go-multiplex, the one stream\nmuxer that both go-libp2p and js-libp2p implement. This should significantly\nimprove go-libp2p and js-libp2p interoperability.\n\n### Multiformats\n\nWe are also changing the way that people write 'ipfs' multiaddrs. Currently,\nipfs multiaddrs look something like\n`/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ`.\nHowever, calling them 'ipfs' multiaddrs is a bit misleading, as this is actually\nthe multiaddr of a libp2p peer that happens to run ipfs. Other protocols built\non libp2p right now still have to use multiaddrs that say 'ipfs', even if they\nhave nothing to do with ipfs. Therefore, we are renaming them to 'p2p'\nmultiaddrs. Moving forward, these addresses will be written as:\n`/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ`.\n\nThis release adds support for *parsing* both types of addresses (`.../ipfs/...`\nand `.../p2p/...`) into the same network format, and the network format is\nremaining exactly the same. A future release will have the ipfs daemon switch to\n*printing* out addresses this way once a large enough portion of the network\nhas upgraded.\n\nN.B., these addresses are *not* related to IPFS *file* names (`/ipfs/Qm...`).\nDisambiguating the two was yet another motivation to switch the protocol name to\n`/p2p/`.\n\n### IPFS\n\nOn the ipfs side of things, we've started embedding public keys inside IPNS\nrecords and have enabled the Git plugin by default.\n\nEmbedding public keys inside IPNS records allows lookups to be faster as we only\nneed to fetch the record itself (and not the public key separately). It also\nfixes an issue where DHT peers wouldn't store a record for a peer if they didn't\nhave their public key already. Combined with some of the DHT and dialing fixes,\nthis should improve the performance of IPNS (once a majority of the network\nupdates).\n\nSecond, our public builds now include the Git plugin (in past builds, you could\nadd it yourself, but doing so was not easy). With this, ipfs can ingest and\noperate over Git repositories and commit graphs directly. For more information\non this, see [the go-ipld-git repo](https://github.com/ipfs/go-ipld-git).\n\nFinally, we've included many smaller bug fixes, refactorings, improved\ndocumentation, and a good bit more. For the full details, see the changelog\nbelow.\n\n## v0.4.16-rc3 2018-07-09\n- Bug fixes\n - Fix dht commands when ipns over pubsub is enabled ([ipfs/go-ipfs#5200](https://github.com/ipfs/go-ipfs/pull/5200))\n - Fix content routing when ipns over pubsub is enabled ([ipfs/go-ipfs#5200](https://github.com/ipfs/go-ipfs/pull/5200))\n - Correctly handle multi-hop dnslink resolution ([ipfs/go-ipfs#5202](https://github.com/ipfs/go-ipfs/pull/5202))\n\n## v0.4.16-rc2 2018-07-05\n- Bug fixes\n - Fix usage of file name vs path name in adder ([ipfs/go-ipfs#5167](https://github.com/ipfs/go-ipfs/pull/5167))\n - Fix `ipfs update` working with migrations ([ipfs/go-ipfs#5194](https://github.com/ipfs/go-ipfs/pull/5194))\n- Documentation\n - Grammar fix in fuse docs ([ipfs/go-ipfs#5164](https://github.com/ipfs/go-ipfs/pull/5164))\n\n## v0.4.16-rc1 2018-06-27\n- Features\n - Embed public keys inside ipns records, use for validation ([ipfs/go-ipfs#5079](https://github.com/ipfs/go-ipfs/pull/5079))\n - Preload git plugin by default ([ipfs/go-ipfs#4991](https://github.com/ipfs/go-ipfs/pull/4991))\n- Improvements\n - Only resolve dnslinks once in the gateway ([ipfs/go-ipfs#4977](https://github.com/ipfs/go-ipfs/pull/4977))\n - Libp2p transport refactor update ([ipfs/go-ipfs#4817](https://github.com/ipfs/go-ipfs/pull/4817))\n - Improve swarm connect/disconnect commands ([ipfs/go-ipfs#5107](https://github.com/ipfs/go-ipfs/pull/5107))\n- Documentation\n - Fix typo of sudo install command ([ipfs/go-ipfs#5001](https://github.com/ipfs/go-ipfs/pull/5001))\n - Fix experimental features Table of Contents ([ipfs/go-ipfs#4976](https://github.com/ipfs/go-ipfs/pull/4976))\n - Fix link to systemd init scripts in the README ([ipfs/go-ipfs#4968](https://github.com/ipfs/go-ipfs/pull/4968))\n - Add package overview comments to coreapi ([ipfs/go-ipfs#5108](https://github.com/ipfs/go-ipfs/pull/5108))\n - Add README to docs folder ([ipfs/go-ipfs#5095](https://github.com/ipfs/go-ipfs/pull/5095))\n - Add system requirements to README ([ipfs/go-ipfs#5137](https://github.com/ipfs/go-ipfs/pull/5137))\n- Bug fixes\n - Fix goroutine leak in pin verify ([ipfs/go-ipfs#5011](https://github.com/ipfs/go-ipfs/pull/5011))\n - Fix commit string in version ([ipfs/go-ipfs#4982](https://github.com/ipfs/go-ipfs/pull/4982))\n - Fix `key rename` command output error ([ipfs/go-ipfs#4962](https://github.com/ipfs/go-ipfs/pull/4962))\n - Report error source when failing to construct private network ([ipfs/go-ipfs#4952](https://github.com/ipfs/go-ipfs/pull/4952))\n - Fix build on DragonFlyBSD ([ipfs/go-ipfs#5031](https://github.com/ipfs/go-ipfs/pull/5031))\n - Fix goroutine leak in dag put ([ipfs/go-ipfs#5016](https://github.com/ipfs/go-ipfs/pull/5016))\n - Fix goroutine leaks in refs.go ([ipfs/go-ipfs#5018](https://github.com/ipfs/go-ipfs/pull/5018))\n - Fix panic, Don't handle errors with fallthrough ([ipfs/go-ipfs#5072](https://github.com/ipfs/go-ipfs/pull/5072))\n - Fix how filestore is hooked up with caching ([ipfs/go-ipfs#5122](https://github.com/ipfs/go-ipfs/pull/5122))\n - Add record validation to offline routing ([ipfs/go-ipfs#5116](https://github.com/ipfs/go-ipfs/pull/5116))\n - Fix `ipfs update` working with migrations ([ipfs/go-ipfs#5194](https://github.com/ipfs/go-ipfs/pull/5194))\n- General Changes and Refactorings\n - Remove leftover bits of dead code ([ipfs/go-ipfs#5022](https://github.com/ipfs/go-ipfs/pull/5022))\n - Remove fuse platform build constraints ([ipfs/go-ipfs#5033](https://github.com/ipfs/go-ipfs/pull/5033))\n - Warning when legacy NoSync setting is set ([ipfs/go-ipfs#5036](https://github.com/ipfs/go-ipfs/pull/5036))\n - Clean up and refactor namesys module ([ipfs/go-ipfs#5007](https://github.com/ipfs/go-ipfs/pull/5007))\n - When raw-leaves are used for empty files use 'Raw' nodes ([ipfs/go-ipfs#4693](https://github.com/ipfs/go-ipfs/pull/4693))\n - Update dist_root in build scripts ([ipfs/go-ipfs#5093](https://github.com/ipfs/go-ipfs/pull/5093))\n - Integrate `pb.Data` into `FSNode` to avoid duplicating fields ([ipfs/go-ipfs#5098](https://github.com/ipfs/go-ipfs/pull/5098))\n - Reduce log level when we can't republish ([ipfs/go-ipfs#5091](https://github.com/ipfs/go-ipfs/pull/5091))\n - Extract ipns record logic to go-ipns ([ipfs/go-ipfs#5124](https://github.com/ipfs/go-ipfs/pull/5124))\n- Testing\n - Collect test times for sharness ([ipfs/go-ipfs#4959](https://github.com/ipfs/go-ipfs/pull/4959))\n - Fix sharness iptb connect timeout ([ipfs/go-ipfs#4966](https://github.com/ipfs/go-ipfs/pull/4966))\n - Add more timeouts to the jenkins pipeline ([ipfs/go-ipfs#4958](https://github.com/ipfs/go-ipfs/pull/4958))\n - Use go 1.10 on jenkins ([ipfs/go-ipfs#5009](https://github.com/ipfs/go-ipfs/pull/5009))\n - Speed up multinode sharness test ([ipfs/go-ipfs#4967](https://github.com/ipfs/go-ipfs/pull/4967))\n - Print out iptb logs on iptb test failure (for debugging CI) ([ipfs/go-ipfs#5069](https://github.com/ipfs/go-ipfs/pull/5069))\n - Disable the MacOS tests in jenkins ([ipfs/go-ipfs#5119](https://github.com/ipfs/go-ipfs/pull/5119))\n - Make republisher test robust against timing issues ([ipfs/go-ipfs#5125](https://github.com/ipfs/go-ipfs/pull/5125))\n - Archive sharness trash dirs in jenkins ([ipfs/go-ipfs#5071](https://github.com/ipfs/go-ipfs/pull/5071))\n - Fix up DHT sharness tests ([ipfs/go-ipfs#5114](https://github.com/ipfs/go-ipfs/pull/5114))\n- Dependencies\n - Update go-ipld-git to fix mergetag resolving ([ipfs/go-ipfs#4988](https://github.com/ipfs/go-ipfs/pull/4988))\n - Fix duplicate /x/sys imports ([ipfs/go-ipfs#5068](https://github.com/ipfs/go-ipfs/pull/5068))\n - Update stream multiplexers ([ipfs/go-ipfs#5075](https://github.com/ipfs/go-ipfs/pull/5075))\n - Update dependencies: go-log, sys, go-crypto ([ipfs/go-ipfs#5100](https://github.com/ipfs/go-ipfs/pull/5100))\n - Explicitly import go-multiaddr-dns in config/bootstrap_peers ([ipfs/go-ipfs#5144](https://github.com/ipfs/go-ipfs/pull/5144))\n - Gx update with dht and dialing improvements ([ipfs/go-ipfs#5158](https://github.com/ipfs/go-ipfs/pull/5158))\n\n## v0.4.15 2018-05-09\n\nThis release is significantly smaller than the last as much of the work on\nimproving our datastores, and other libraries libp2p has yet to be merged.\nHowever, it still includes many welcome improvements.\n\nAs with 0.4.12 and 0.4.14 (0.4.13 was a patch), this release has a negative\ndiff-stat. Unfortunately, much of this code isn't actually going away but at\nleast it's being moved out into separate repositories.\n\nMuch of the work that made it into this release is under the hood. We've cleaned\nup some code, extracted several packages into their own repositories, and made\nsome long neglected optimizations (e.g., handling of sharded directories).\nAdditionally, this release includes a bunch of tests for our CLI commands that\nshould help us avoid some of the issues we've seen in the past few releases.\n\nMore visibly, thanks to @djdv's efforts, this release includes some significant\nWindows improvements (with more on the way). Specifically, this release includes\nbetter handling of repo lockfiles (no more `ipfs repo fsck`), stdin command-line\nsupport, and, last but not least, IPFS no longer writes random files with scary\ngarbage in the drive root. To read more about future windows improvements, take\na look at this [blog post](https://blog.ipfs.io/36-a-look-at-windows/).\n\nTo better support low-power devices, we've added a low-power config profile.\nThis can be enabled when initializing a repo by running `ipfs init` with the\n`--profile=lowpower` flag or later by running `ipfs config profile apply lowpower`.\n\nFinally, with this release we have begun distributing self-contained source\narchives of go-ipfs and its dependencies. This should be a welcome improvement\nfor both packagers and those living in countries with harmonized internet\naccess.\n\n- Features\n - Add options for record count and timeout for resolving DHT paths ([ipfs/go-ipfs#4733](https://github.com/ipfs/go-ipfs/pull/4733))\n - Add low power init profile ([ipfs/go-ipfs#4154](https://github.com/ipfs/go-ipfs/pull/4154))\n - Add Opentracing plugin support ([ipfs/go-ipfs#4506](https://github.com/ipfs/go-ipfs/pull/4506))\n - Add make target to build source tarballs ([ipfs/go-ipfs#4920](https://github.com/ipfs/go-ipfs/pull/4920))\n\n- Improvements\n - Add BlockedFetched/Added/Removed events to Blockservice ([ipfs/go-ipfs#4649](https://github.com/ipfs/go-ipfs/pull/4649))\n - Improve performance of HAMT code ([ipfs/go-ipfs#4889](https://github.com/ipfs/go-ipfs/pull/4889))\n - Avoid unnecessarily resolving child nodes when listing a sharded directory ([ipfs/go-ipfs#4884](https://github.com/ipfs/go-ipfs/pull/4884))\n - Tar writer now supports sharded ipfs directories ([ipfs/go-ipfs#4873](https://github.com/ipfs/go-ipfs/pull/4873))\n - Infer type from CID when possible in `ipfs ls` ([ipfs/go-ipfs#4890](https://github.com/ipfs/go-ipfs/pull/4890))\n - Deduplicate keys in GetMany ([ipfs/go-ipfs#4888](https://github.com/ipfs/go-ipfs/pull/4888))\n\n- Documentation\n - Fix spelling of retrieval ([ipfs/go-ipfs#4819](https://github.com/ipfs/go-ipfs/pull/4819))\n - Update broken links ([ipfs/go-ipfs#4798](https://github.com/ipfs/go-ipfs/pull/4798))\n - Remove roadmap.md ([ipfs/go-ipfs#4834](https://github.com/ipfs/go-ipfs/pull/4834))\n - Remove link to IPFS paper in contribute.md ([ipfs/go-ipfs#4812](https://github.com/ipfs/go-ipfs/pull/4812))\n - Fix broken todo link in readme.md ([ipfs/go-ipfs#4865](https://github.com/ipfs/go-ipfs/pull/4865))\n - Document ipns pubsub ([ipfs/go-ipfs#4903](https://github.com/ipfs/go-ipfs/pull/4903))\n - Fix missing profile docs ([ipfs/go-ipfs#4846](https://github.com/ipfs/go-ipfs/pull/4846))\n - Fix a few typos ([ipfs/go-ipfs#4835](https://github.com/ipfs/go-ipfs/pull/4835))\n - Fix typo in fsrepo error message ([ipfs/go-ipfs#4933](https://github.com/ipfs/go-ipfs/pull/4933))\n - Remove go-ipfs version from issue template ([ipfs/go-ipfs#4943](https://github.com/ipfs/go-ipfs/pull/4943))\n - Add docs for --profile=lowpower ([ipfs/go-ipfs#4970](https://github.com/ipfs/go-ipfs/pull/4970))\n - Improve Windows build documentation ([ipfs/go-ipfs#4691](https://github.com/ipfs/go-ipfs/pull/4691))\n\n- Bug fixes\n - Check CIDs in base case when diffing nodes ([ipfs/go-ipfs#4767](https://github.com/ipfs/go-ipfs/pull/4767))\n - Support for CIDv1 with custom mhtype in `ipfs block put` ([ipfs/go-ipfs#4563](https://github.com/ipfs/go-ipfs/pull/4563))\n - Clean path in DagArchive ([ipfs/go-ipfs#4743](https://github.com/ipfs/go-ipfs/pull/4743))\n - Set the prefix for MFS root in `ipfs add --hash-only` ([ipfs/go-ipfs#4755](https://github.com/ipfs/go-ipfs/pull/4755))\n - Fix get output path ([ipfs/go-ipfs#4809](https://github.com/ipfs/go-ipfs/pull/4809))\n - Fix incorrect Read calls ([ipfs/go-ipfs#4792](https://github.com/ipfs/go-ipfs/pull/4792))\n - Use prefix in bootstrapWritePeers ([ipfs/go-ipfs#4832](https://github.com/ipfs/go-ipfs/pull/4832))\n - Fix mfs Directory.Path not working ([ipfs/go-ipfs#4844](https://github.com/ipfs/go-ipfs/pull/4844))\n - Remove header in `ipfs stats bw` if not polling ([ipfs/go-ipfs#4856](https://github.com/ipfs/go-ipfs/pull/4856))\n - Match Go's GOPATH defaults behaviour in build scripts ([ipfs/go-ipfs#4678](https://github.com/ipfs/go-ipfs/pull/4678))\n - Fix default-net profile not reverting bootstrap config ([ipfs/go-ipfs#4845](https://github.com/ipfs/go-ipfs/pull/4845))\n - Fix excess goroutines in bitswap caused by insecure CIDs ([ipfs/go-ipfs#4946](https://github.com/ipfs/go-ipfs/pull/4946))\n\n- General Changes and Refactorings\n - Refactor trickle DAG builder ([ipfs/go-ipfs#4730](https://github.com/ipfs/go-ipfs/pull/4730))\n - Split the coreapi interface into multiple files ([ipfs/go-ipfs#4802](https://github.com/ipfs/go-ipfs/pull/4802))\n - Make `ipfs init` command use new cmds lib ([ipfs/go-ipfs#4732](https://github.com/ipfs/go-ipfs/pull/4732))\n - Extract thirdparty/tar package ([ipfs/go-ipfs#4857](https://github.com/ipfs/go-ipfs/pull/4857))\n - Reduce log level when for disconnected peers to info ([ipfs/go-ipfs#4811](https://github.com/ipfs/go-ipfs/pull/4811))\n - Only visit nodes in EnumerateChildrenAsync when asked ([ipfs/go-ipfs#4885](https://github.com/ipfs/go-ipfs/pull/4885))\n - Refactor coreapi options ([ipfs/go-ipfs#4807](https://github.com/ipfs/go-ipfs/pull/4807))\n - Fix error style for most errors ([ipfs/go-ipfs#4829](https://github.com/ipfs/go-ipfs/pull/4829))\n - Ensure `--help` always works, even with /dev/null stdin ([ipfs/go-ipfs#4849](https://github.com/ipfs/go-ipfs/pull/4849))\n - Deduplicate AddNodeLinkClean into AddNodeLink ([ipfs/go-ipfs#4940](https://github.com/ipfs/go-ipfs/pull/4940))\n - Remove some dead code ([ipfs/go-ipfs#4833](https://github.com/ipfs/go-ipfs/pull/4833))\n - Remove unused imports ([ipfs/go-ipfs#4955](https://github.com/ipfs/go-ipfs/pull/4955))\n - Fix go vet warnings ([ipfs/go-ipfs#4859](https://github.com/ipfs/go-ipfs/pull/4859))\n\n- Testing\n - Generate JUnit test reports for sharness tests ([ipfs/go-ipfs#4530](https://github.com/ipfs/go-ipfs/pull/4530))\n - Fix t0063-daemon-init.sh by adding test profile to daemon ([ipfs/go-ipfs#4816](https://github.com/ipfs/go-ipfs/pull/4816))\n - Remove circular dependencies in merkledag package tests ([ipfs/go-ipfs#4704](https://github.com/ipfs/go-ipfs/pull/4704))\n - Check that all the commands fail when passed a bad flag ([ipfs/go-ipfs#4848](https://github.com/ipfs/go-ipfs/pull/4848))\n - Allow for some small margin of code coverage dropping on commit ([ipfs/go-ipfs#4867](https://github.com/ipfs/go-ipfs/pull/4867))\n - Add confirmation to archive-branches script ([ipfs/go-ipfs#4797](https://github.com/ipfs/go-ipfs/pull/4797))\n\n- Dependencies\n - Update lock package ([ipfs/go-ipfs#4855](https://github.com/ipfs/go-ipfs/pull/4855))\n - Update to latest go-datastore. Remove thirdparty/datastore2 ([ipfs/go-ipfs#4742](https://github.com/ipfs/go-ipfs/pull/4742))\n - Extract fs lock into go-fs-lock ([ipfs/go-ipfs#4631](https://github.com/ipfs/go-ipfs/pull/4631))\n - Extract: exchange/interface.go, blocks/blocksutil, exchange/offline ([ipfs/go-ipfs#4912](https://github.com/ipfs/go-ipfs/pull/4912))\n - Remove unused lock dep ([ipfs/go-ipfs#4971](https://github.com/ipfs/go-ipfs/pull/4971))\n - Update iptb ([ipfs/go-ipfs#4965](https://github.com/ipfs/go-ipfs/pull/4965))\n - Update go-ipfs-cmds to fix stdin on windows ([ipfs/go-ipfs#4975](https://github.com/ipfs/go-ipfs/pull/4975))\n - Update go-ds-flatfs to fix windows corruption issue ([ipfs/go-ipfs#4872](https://github.com/ipfs/go-ipfs/pull/4872))\n\n## v0.4.14 2018-03-22\n\nIpfs 0.4.14 is a big release with a large number of improvements and bug fixes.\nIt is also the first release of 2018, and our first release in over three\nmonths. The release took longer than expected due to our refactoring and\nextracting of our commands library. This refactor had two stages. The first\nround of the refactor disentangled the commands code from core ipfs code,\nallowing us to move it out into a [separate\nrepository](https://github.com/ipfs/go-ipfs-cmds). The code was previously\nvery entangled with the go-ipfs codebase and not usable for other projects.\nThe second round of the refactor had the goal of fixing several major issues\naround streaming outputs, progress bars, and error handling. It also paved the\nway for us to more easily provide an API over other transports, such as\nwebsockets and unix domain sockets. It took a while to flush out all the kinks\non such a massive change. We're pretty sure we've got most of them, but if you\nnotice anything weird, please let us know.\n\nBeyond that, we've added a new experimental way to use IPNS. With the new\npubsub IPNS resolver and publisher, you can subscribe to updates of an IPNS\nentry, and the owner can publish out changes in real time. With this, IPNS can\nbecome nearly instantaneous. To make use of this, simply start your ipfs daemon\nwith the `--enable-namesys-pubsub` option, and all IPNS resolution and\npublishing will use pubsub. Note that resolving an IPNS name via pubsub without\nsomeone publishing it via pubsub will result in a fallback to using the DHT.\nPlease give this a try and let us know how it goes!\n\nMemory and CPU usage should see a noticeable improvement in this release. We\nhave spent considerable time fixing excess memory usage throughout the codebase\nand down into libp2p. Fixes in peer tracking, bitswap allocation, pinning, and\nmany other places have brought down both peak and average memory usage. An\nupgraded hashing library, base58 encoding library, and improved allocation\npatterns all contribute to overall lower CPU usage across the board. See the\nfull changelist below for more memory and CPU usage improvements.\n\nThis release also brings the beginning of the ipfs 'Core API'. Once finished,\nthe Core API will be the primary way to interact with go-ipfs using go. Both\nembedded nodes and nodes accessed over the http API will have the same\ninterface. Stay tuned for future updates and documentation.\n\nThese are only a sampling of the changes that made it into this release, the\nfull list (almost 100 PRs!) is below.\n\nFinally, I'd like to thank everyone who contributed to this release, whether\nyou're just contributing a typo fix or driving new features. We are really\ngrateful to everyone who has spent their their time pushing ipfs forward.\n\nSECURITY NOTE:\n\nThis release of ipfs disallows the usage of insecure hash functions and\nlengths. Ipfs does not create these insecure objects for any purpose, but it\ndid allow manually creating them and fetching them from other peers. If you\ncurrently have objects using insecure hashes in your local ipfs repo, please\nremove them before updating.\n\n#### Changes from rc2 to rc3\n- Fix bug in stdin argument parsing ([ipfs/go-ipfs#4827](https://github.com/ipfs/go-ipfs/pull/4827))\n- Revert commands back to sending a single response ([ipfs/go-ipfs#4822](https://github.com/ipfs/go-ipfs/pull/4822))\n\n#### Changes from rc1 to rc2\n- Fix issue in ipfs get caused by go1.10 changes ([ipfs/go-ipfs#4790](https://github.com/ipfs/go-ipfs/pull/4790))\n\n- Features\n - Pubsub IPNS Publisher and Resolver (experimental) ([ipfs/go-ipfs#4047](https://github.com/ipfs/go-ipfs/pull/4047))\n - Implement coreapi Dag interface ([ipfs/go-ipfs#4471](https://github.com/ipfs/go-ipfs/pull/4471))\n - Add --offset flag to ipfs cat ([ipfs/go-ipfs#4538](https://github.com/ipfs/go-ipfs/pull/4538))\n - Command to apply config profile after init ([ipfs/go-ipfs#4195](https://github.com/ipfs/go-ipfs/pull/4195))\n - Implement coreapi Name and Key interfaces ([ipfs/go-ipfs#4477](https://github.com/ipfs/go-ipfs/pull/4477))\n - Add --length flag to ipfs cat ([ipfs/go-ipfs#4553](https://github.com/ipfs/go-ipfs/pull/4553))\n - Implement coreapi Object interface ([ipfs/go-ipfs#4492](https://github.com/ipfs/go-ipfs/pull/4492))\n - Implement coreapi Block interface ([ipfs/go-ipfs#4548](https://github.com/ipfs/go-ipfs/pull/4548))\n - Implement coreapi Pin interface ([ipfs/go-ipfs#4575](https://github.com/ipfs/go-ipfs/pull/4575))\n - Add a --with-local flag to ipfs files stat ([ipfs/go-ipfs#4638](https://github.com/ipfs/go-ipfs/pull/4638))\n - Disallow usage of blocks with insecure hashes ([ipfs/go-ipfs#4751](https://github.com/ipfs/go-ipfs/pull/4751))\n- Improvements\n - Add uuid to event logs ([ipfs/go-ipfs#4392](https://github.com/ipfs/go-ipfs/pull/4392))\n - Add --quiet flag to object put ([ipfs/go-ipfs#4411](https://github.com/ipfs/go-ipfs/pull/4411))\n - Pinning memory improvements and fixes ([ipfs/go-ipfs#4451](https://github.com/ipfs/go-ipfs/pull/4451))\n - Update WebUI version ([ipfs/go-ipfs#4449](https://github.com/ipfs/go-ipfs/pull/4449))\n - Check strong and weak ETag validator ([ipfs/go-ipfs#3983](https://github.com/ipfs/go-ipfs/pull/3983))\n - Improve and refactor FD limit handling ([ipfs/go-ipfs#3801](https://github.com/ipfs/go-ipfs/pull/3801))\n - Support linking to non-dagpb objects in ipfs object patch ([ipfs/go-ipfs#4460](https://github.com/ipfs/go-ipfs/pull/4460))\n - Improve allocation patterns of slices in bitswap ([ipfs/go-ipfs#4458](https://github.com/ipfs/go-ipfs/pull/4458))\n - Secio handshake now happens synchronously ([libp2p/go-libp2p-secio#25](https://github.com/libp2p/go-libp2p-secio/pull/25))\n - Don't block closing connections on pending writes ([libp2p/go-msgio#7](https://github.com/libp2p/go-msgio/pull/7))\n - Improve memory usage of multiaddr parsing ([multiformats/go-multiaddr#56](https://github.com/multiformats/go-multiaddr/pull/56))\n - Don't lock up 256KiB buffers when adding small files ([ipfs/go-ipfs#4508](https://github.com/ipfs/go-ipfs/pull/4508))\n - Clear out memory after reads from the dagreader ([ipfs/go-ipfs#4525](https://github.com/ipfs/go-ipfs/pull/4525))\n - Improve error handling in ipfs ping ([ipfs/go-ipfs#4546](https://github.com/ipfs/go-ipfs/pull/4546))\n - Allow install.sh to be run without being the script dir ([ipfs/go-ipfs#4547](https://github.com/ipfs/go-ipfs/pull/4547))\n - Much faster base58 encoding ([libp2p/go-libp2p-peer#24](https://github.com/libp2p/go-libp2p-peer/pull/24))\n - Use faster sha256 and blake2b libs ([multiformats/go-multihash#63](https://github.com/multiformats/go-multihash/pull/63))\n - Greatly improve peerstore memory usage ([libp2p/go-libp2p-peerstore#22](https://github.com/libp2p/go-libp2p-peerstore/pull/22))\n - Improve dht memory usage and peer tracking ([libp2p/go-libp2p-kad-dht#111](https://github.com/libp2p/go-libp2p-kad-dht/pull/111))\n - New libp2p metrics lib with lower overhead ([libp2p/go-libp2p-metrics#8](https://github.com/libp2p/go-libp2p-metrics/pull/8))\n - Fix memory leak that occurred when dialing many peers ([libp2p/go-libp2p-swarm#51](https://github.com/libp2p/go-libp2p-swarm/pull/51))\n - Wire up new dag interfaces to make sessions easier ([ipfs/go-ipfs#4641](https://github.com/ipfs/go-ipfs/pull/4641))\n- Documentation\n - Correct StorageMax config description ([ipfs/go-ipfs#4388](https://github.com/ipfs/go-ipfs/pull/4388))\n - Add how to download IPFS with IPFS doc ([ipfs/go-ipfs#4390](https://github.com/ipfs/go-ipfs/pull/4390))\n - Document gx release checklist item ([ipfs/go-ipfs#4480](https://github.com/ipfs/go-ipfs/pull/4480))\n - Add some documentation to CoreAPI ([ipfs/go-ipfs#4493](https://github.com/ipfs/go-ipfs/pull/4493))\n - Add interop tests to the release checklist ([ipfs/go-ipfs#4501](https://github.com/ipfs/go-ipfs/pull/4501))\n - Add badgerds to experimental-features ToC ([ipfs/go-ipfs#4537](https://github.com/ipfs/go-ipfs/pull/4537))\n - Fix typos and inconsistencies in commands documentation ([ipfs/go-ipfs#4552](https://github.com/ipfs/go-ipfs/pull/4552))\n - Add a document to help troubleshoot data transfers ([ipfs/go-ipfs#4332](https://github.com/ipfs/go-ipfs/pull/4332))\n - Add a bunch of documentation on public interfaces ([ipfs/go-ipfs#4599](https://github.com/ipfs/go-ipfs/pull/4599))\n - Expand the issue template and remove the severity field ([ipfs/go-ipfs#4624](https://github.com/ipfs/go-ipfs/pull/4624))\n - Add godocs for importers module ([ipfs/go-ipfs#4640](https://github.com/ipfs/go-ipfs/pull/4640))\n - Document make targets ([ipfs/go-ipfs#4653](https://github.com/ipfs/go-ipfs/pull/4653))\n - Add godocs for merkledag module ([ipfs/go-ipfs#4665](https://github.com/ipfs/go-ipfs/pull/4665))\n - Add godocs for unixfs module ([ipfs/go-ipfs#4664](https://github.com/ipfs/go-ipfs/pull/4664))\n - Add sharding to experimental features list ([ipfs/go-ipfs#4569](https://github.com/ipfs/go-ipfs/pull/4569))\n - Add godocs for routing module ([ipfs/go-ipfs#4676](https://github.com/ipfs/go-ipfs/pull/4676))\n - Add godocs for path module ([ipfs/go-ipfs#4689](https://github.com/ipfs/go-ipfs/pull/4689))\n - Add godocs for pin module ([ipfs/go-ipfs#4696](https://github.com/ipfs/go-ipfs/pull/4696))\n - Update link to filestore experimental status ([ipfs/go-ipfs#4557](https://github.com/ipfs/go-ipfs/pull/4557))\n- Bug fixes\n - Remove trailing slash in ipfs get paths, fixes #3729 ([ipfs/go-ipfs#4365](https://github.com/ipfs/go-ipfs/pull/4365))\n - fix deadlock in bitswap sessions ([ipfs/go-ipfs#4407](https://github.com/ipfs/go-ipfs/pull/4407))\n - Fix two race conditions (and possibly go routine leaks) in commands ([ipfs/go-ipfs#4406](https://github.com/ipfs/go-ipfs/pull/4406))\n - Fix output delay in ipfs pubsub sub ([ipfs/go-ipfs#4402](https://github.com/ipfs/go-ipfs/pull/4402))\n - Use correct context in AddWithContext ([ipfs/go-ipfs#4433](https://github.com/ipfs/go-ipfs/pull/4433))\n - Fix various IPNS republisher issues ([ipfs/go-ipfs#4440](https://github.com/ipfs/go-ipfs/pull/4440))\n - Fix error handling in commands add and get ([ipfs/go-ipfs#4454](https://github.com/ipfs/go-ipfs/pull/4454))\n - Fix hamt (sharding) delete issue ([ipfs/go-ipfs#4398](https://github.com/ipfs/go-ipfs/pull/4398))\n - More correctly check for reuseport support ([libp2p/go-reuseport#40](https://github.com/libp2p/go-reuseport/pull/40))\n - Fix goroutine leak in websockets transport ([libp2p/go-ws-transport#21](https://github.com/libp2p/go-ws-transport/pull/21))\n - Update badgerds to fix i386 windows build ([ipfs/go-ipfs#4464](https://github.com/ipfs/go-ipfs/pull/4464))\n - Only construct bitswap event loggable if necessary ([ipfs/go-ipfs#4533](https://github.com/ipfs/go-ipfs/pull/4533))\n - Ensure that flush on the mfs root flushes its directory ([ipfs/go-ipfs#4509](https://github.com/ipfs/go-ipfs/pull/4509))\n - Fix deferred unlock of pin lock in AddR ([ipfs/go-ipfs#4562](https://github.com/ipfs/go-ipfs/pull/4562))\n - Fix iOS builds ([ipfs/go-ipfs#4610](https://github.com/ipfs/go-ipfs/pull/4610))\n - Calling repo gc now frees up space with badgerds ([ipfs/go-ipfs#4578](https://github.com/ipfs/go-ipfs/pull/4578))\n - Fix leak in bitswap sessions shutdown ([ipfs/go-ipfs#4658](https://github.com/ipfs/go-ipfs/pull/4658))\n - Fix make on windows ([ipfs/go-ipfs#4682](https://github.com/ipfs/go-ipfs/pull/4682))\n - Ignore invalid key files in keystore directory ([ipfs/go-ipfs#4700](https://github.com/ipfs/go-ipfs/pull/4700))\n- General Changes and Refactorings\n - Extract and refactor commands library ([ipfs/go-ipfs#3856](https://github.com/ipfs/go-ipfs/pull/3856))\n - Remove all instances of `Default(false)` ([ipfs/go-ipfs#4042](https://github.com/ipfs/go-ipfs/pull/4042))\n - Build for all supported platforms when testing ([ipfs/go-ipfs#4445](https://github.com/ipfs/go-ipfs/pull/4445))\n - Refine gateway and namesys logging ([ipfs/go-ipfs#4428](https://github.com/ipfs/go-ipfs/pull/4428))\n - Demote bitswap error to an info ([ipfs/go-ipfs#4472](https://github.com/ipfs/go-ipfs/pull/4472))\n - Extract posinfo package to github.com/ipfs/go-ipfs-posinfo ([ipfs/go-ipfs#4669](https://github.com/ipfs/go-ipfs/pull/4669))\n - Move signature verification to ipns validator ([ipfs/go-ipfs#4628](https://github.com/ipfs/go-ipfs/pull/4628))\n - Extract importers/chunk module as go-ipfs-chunker ([ipfs/go-ipfs#4661](https://github.com/ipfs/go-ipfs/pull/4661))\n - Extract go-detect-race from Godeps ([ipfs/go-ipfs#4686](https://github.com/ipfs/go-ipfs/pull/4686))\n - Extract flags, delay, ds-help ([ipfs/go-ipfs#4685](https://github.com/ipfs/go-ipfs/pull/4685))\n - Extract routing package to go-ipfs-routing ([ipfs/go-ipfs#4703](https://github.com/ipfs/go-ipfs/pull/4703))\n - Extract blocks/blockstore package to go-ipfs-blockstore ([ipfs/go-ipfs#4707](https://github.com/ipfs/go-ipfs/pull/4707))\n - Add exchange.SessionExchange interface for exchanges that support sessions ([ipfs/go-ipfs#4709](https://github.com/ipfs/go-ipfs/pull/4709))\n - Extract thirdparty/pq to go-ipfs-pq ([ipfs/go-ipfs#4711](https://github.com/ipfs/go-ipfs/pull/4711))\n - Separate \"path\" from \"path/resolver\" ([ipfs/go-ipfs#4713](https://github.com/ipfs/go-ipfs/pull/4713))\n- Testing\n - Increase verbosity of t0088-repo-stat-symlink.sh test ([ipfs/go-ipfs#4434](https://github.com/ipfs/go-ipfs/pull/4434))\n - Make repo size test pass deterministically ([ipfs/go-ipfs#4443](https://github.com/ipfs/go-ipfs/pull/4443))\n - Always set IPFS_PATH in test-lib.sh ([ipfs/go-ipfs#4469](https://github.com/ipfs/go-ipfs/pull/4469))\n - Fix sharness docker ([ipfs/go-ipfs#4489](https://github.com/ipfs/go-ipfs/pull/4489))\n - Fix loops in sharness tests to fail the test if the inner command fails ([ipfs/go-ipfs#4482](https://github.com/ipfs/go-ipfs/pull/4482))\n - Improve bitswap tests, fix race conditions ([ipfs/go-ipfs#4499](https://github.com/ipfs/go-ipfs/pull/4499))\n - Fix circleci cache directory list ([ipfs/go-ipfs#4564](https://github.com/ipfs/go-ipfs/pull/4564))\n - Only run the build test on test_go_expensive ([ipfs/go-ipfs#4645](https://github.com/ipfs/go-ipfs/pull/4645))\n - Fix go test on Windows ([ipfs/go-ipfs#4632](https://github.com/ipfs/go-ipfs/pull/4632))\n - Fix some tests on FreeBSD ([ipfs/go-ipfs#4662](https://github.com/ipfs/go-ipfs/pull/4662))\n\n## v0.4.13 2017-11-16\n\nIpfs 0.4.13 is a patch release that fixes two high priority issues that were\ndiscovered in the 0.4.12 release.\n\nBug fixes:\n - Fix periodic bitswap deadlock ([ipfs/go-ipfs#4386](https://github.com/ipfs/go-ipfs/pull/4386))\n - Fix badgerds crash on startup ([ipfs/go-ipfs#4384](https://github.com/ipfs/go-ipfs/pull/4384))\n\n\n## v0.4.12 2017-11-09\n\nIpfs 0.4.12 brings with it many important fixes for the huge spike in network\nsize we've seen this past month. These changes include the Connection Manager,\nfaster batching in `ipfs add`, libp2p fixes that reduce CPU usage, and a bunch\nof new documentation.\n\nThe most critical change is the 'Connection Manager': it allows an ipfs node to\nmaintain a limited set of connections to other peers in the network. By default\n(and with no config changes required by the user), ipfs nodes will now try to\nmaintain between 600 and 900 open connections. These limits are still likely\nhigher than needed, and future releases may lower the default recommendation,\nbut for now we want to make changes gradually. The rationale for this selection\nof numbers is as follows:\n\n- The DHT routing table for a large network may rise to around 400 peers\n- Bitswap connections tend to be separate from the DHT\n- PubSub connections also generally are another distinct set of peers\n (including js-ipfs nodes)\n\nBecause of this, we selected 600 as a 'LowWater' number, and 900 as a\n'HighWater' number to avoid having to clear out connections too frequently.\nYou can configure different numbers as you see fit via the `Swarm.ConnMgr`\nfield in your ipfs config file. See\n[here](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#connmgr) for\nmore details.\n\nDisk utilization during `ipfs add` has been optimized for large files by doing\nbatch writes in parallel. Previously, when adding a large file, users might have\nnoticed that the add progressed by about 8MB at a time, with brief pauses in between.\nThis was caused by quickly filling up the batch, then blocking while it was\nwriting to disk. We now write to disk in the background while continuing to add\nthe remainder of the file.\n\nOther changes in this release have noticeably reduced memory consumption and CPU\nusage. This was done by optimising some frequently called functions in libp2p\nthat were expensive in terms of both CPU usage and memory allocations. We also\nlowered the yamux accept buffer sizes which were raised over a year ago to\ncombat a separate bug that has since been fixed.\n\nAnd finally, thank you to everyone who filed bugs, tested out the release candidates,\nfiled pull requests, and contributed in any other way to this release!\n\n- Features\n - Implement Connection Manager ([ipfs/go-ipfs#4288](https://github.com/ipfs/go-ipfs/pull/4288))\n - Support multiple files in dag put ([ipfs/go-ipfs#4254](https://github.com/ipfs/go-ipfs/pull/4254))\n - Add 'raw' support to the dag put command ([ipfs/go-ipfs#4285](https://github.com/ipfs/go-ipfs/pull/4285))\n- Improvements\n - Parallelize dag batch flushing ([ipfs/go-ipfs#4296](https://github.com/ipfs/go-ipfs/pull/4296))\n - Update go-peerstream to improve CPU usage ([ipfs/go-ipfs#4323](https://github.com/ipfs/go-ipfs/pull/4323))\n - Add full support for CidV1 in Files API and Dag Modifier ([ipfs/go-ipfs#4026](https://github.com/ipfs/go-ipfs/pull/4026))\n - Lower yamux accept buffer size ([ipfs/go-ipfs#4326](https://github.com/ipfs/go-ipfs/pull/4326))\n - Optimise `ipfs pin update` command ([ipfs/go-ipfs#4348](https://github.com/ipfs/go-ipfs/pull/4348))\n- Documentation\n - Add some docs on plugins ([ipfs/go-ipfs#4255](https://github.com/ipfs/go-ipfs/pull/4255))\n - Add more info about private network bootstrap ([ipfs/go-ipfs#4270](https://github.com/ipfs/go-ipfs/pull/4270))\n - Add more info about `ipfs add` chunker option ([ipfs/go-ipfs#4306](https://github.com/ipfs/go-ipfs/pull/4306))\n - Remove cruft in readme and mention discourse forum ([ipfs/go-ipfs#4345](https://github.com/ipfs/go-ipfs/pull/4345))\n - Add note about updating before reporting issues ([ipfs/go-ipfs#4361](https://github.com/ipfs/go-ipfs/pull/4361))\n- Bug fixes\n - Fix FreeBSD build issues ([ipfs/go-ipfs#4275](https://github.com/ipfs/go-ipfs/pull/4275))\n - Don't crash when Datastore.StorageMax is not defined ([ipfs/go-ipfs#4246](https://github.com/ipfs/go-ipfs/pull/4246))\n - Do not call 'Connect' on NewStream in bitswap ([ipfs/go-ipfs#4317](https://github.com/ipfs/go-ipfs/pull/4317))\n - Filter out \"\" from active peers in bitswap sessions ([ipfs/go-ipfs#4316](https://github.com/ipfs/go-ipfs/pull/4316))\n - Fix \"seeker can't seek\" on specific files ([ipfs/go-ipfs#4320](https://github.com/ipfs/go-ipfs/pull/4320))\n - Do not set \"gecos\" field in Dockerfile ([ipfs/go-ipfs#4331](https://github.com/ipfs/go-ipfs/pull/4331))\n - Handle sym links in when calculating repo size ([ipfs/go-ipfs#4305](https://github.com/ipfs/go-ipfs/pull/4305))\n- General Changes and Refactorings\n - Fix indent in sharness tests ([ipfs/go-ipfs#4212](https://github.com/ipfs/go-ipfs/pull/4212))\n - Remove supernode routing ([ipfs/go-ipfs#4302](https://github.com/ipfs/go-ipfs/pull/4302))\n - Extract go-ipfs-addr ([ipfs/go-ipfs#4340](https://github.com/ipfs/go-ipfs/pull/4340))\n - Remove dead code and config files ([ipfs/go-ipfs#4357](https://github.com/ipfs/go-ipfs/pull/4357))\n - Update badgerds to 1.0 ([ipfs/go-ipfs#4327](https://github.com/ipfs/go-ipfs/pull/4327))\n - Wrap help descriptions under 80 chars ([ipfs/go-ipfs#4121](https://github.com/ipfs/go-ipfs/pull/4121))\n- Testing\n - Make sharness t0180-p2p less racy ([ipfs/go-ipfs#4310](https://github.com/ipfs/go-ipfs/pull/4310))\n\n\n### v0.4.11 2017-09-14\n\nIpfs 0.4.11 is a larger release that brings many long-awaited features and\nperformance improvements. These include new datastore options, more efficient\nbitswap transfers, greatly improved resource consumption, circuit relay\nsupport, ipld plugins, and more! Take a look at the full changelog below for a\ndetailed list of every change.\n\nThe ipfs datastore has, until now, been a combination of leveldb and a custom\ngit-like storage backend called 'flatfs'. This works well enough for the\naverage user, but different ipfs usecases demand different backend\nconfigurations. To address this, we have changed the configuration file format\nfor datastores to be a modular way of specifying exactly how you want the\ndatastore to be structured. You will now be able to configure ipfs to use\nflatfs, leveldb, badger, an in-memory datastore, and more to suit your needs.\nSee the new [datastore\ndocumentation](https://github.com/ipfs/go-ipfs/blob/master/docs/datastores.md)\nfor more information.\n\nBitswap received some much needed attention during this release cycle. The\nconcept of 'Bitswap Sessions' allows bitswap to associate requests for\ndifferent blocks to the same underlying session, and from that infer better\nways of requesting that data. In more concrete terms, parts of the ipfs\ncodebase that take advantage of sessions (currently, only `ipfs pin add`) will\ncause much less extra traffic than before. This is done by making optimistic\nguesses about which nodes might be providing given blocks and not sending\nwantlist updates to every connected bitswap partner, as well as searching the\nDHT for providers less frequently. In future releases we will migrate over more\nipfs commands to take advantage of bitswap sessions. As nodes update to this\nand future versions, expect to see idle bandwidth usage on the ipfs network\ngo down noticeably.\n\nThe never ending effort to reduce resource consumption had a few important\nupdates this release. First, the bitswap sessions changes discussed above will\nhelp with improving bandwidth usage. Aside from that there are two important\nlibp2p updates that improved things significantly. The first was a fix to a bug\nin the dial limiter code that was causing it to not limit outgoing dials\ncorrectly. This resulted in ipfs running out of file descriptors very\nfrequently (as well as incurring a decent amount of excess outgoing bandwidth),\nthis has now been fixed. Users who previously received \"too many open files\"\nerrors should see this much less often in 0.4.11. The second change was a\nmemory leak in the DHT that was identified and fixed. Streams being tracked in\na map in the DHT weren't being cleaned up after the peer disconnected leading\nto the multiplexer session not being cleaned up properly. This issue has been\nresolved, and now memory usage appears to be stable over time. There is still a\nlot of work to be done improving memory usage, but we feel this is a solid\nvictory.\n\nIt is often said that NAT traversal is the hardest problem in peer to peer\ntechnology, we tend to agree with this. In an effort to provide a more\nubiquitous p2p mesh, we have implemented a relay mechanism that allows willing\npeers to relay traffic for other peers who might not otherwise be able to\ncommunicate with each other. This feature is still pretty early, and currently\nusers have to manually connect through a relay. The next step in this endeavour\nis automatic relaying, and research for this is currently in progress. We\nexpect that when it lands, it will improve the perceived performance of ipfs by\nspending less time attempting connections to hard to reach nodes. A short guide\non using the circuit relay feature can be found\n[here](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#circuit-relay).\n\nThe last feature we want to highlight (but by no means the last feature in this\nrelease) is our new plugin system. There are many different workflows and\nusecases that ipfs should be able to support, but not everyone wants to be able\nto use every feature. We could simply merge in all these features, but that\ncauses problems for several reasons: first off, the size of the ipfs binary\nstarts to get very large very quickly. Second, each of these different pieces\nneeds to be maintained and updated independently, which would cause significant\nchurn in the codebase. To address this, we have come up with a system that\nallows users to install plugins to the vanilla ipfs daemon that augment its\ncapabilities. The first of these plugins are a [git\nplugin](https://github.com/ipfs/go-ipfs/blob/master/plugin/plugins/git/git.go)\nthat allows ipfs to natively address git objects and an [ethereum\nplugin](https://github.com/ipfs/go-ipld-eth) that lets ipfs ingest and operate\non all ethereum blockchain data. Soon to come are plugins for the bitcoin and\nzcash data formats. In the future, we will be adding plugins for other things\nlike datastore backends and specialized libp2p network transports.\nYou can read more on this topic in [Plugin docs](docs/plugins.md)\n\nIn order to simplify its integration with fs-repo-migrations, we've switched\nthe ipfs/go-ipfs docker image from a musl base to a glibc base. For most users\nthis will not be noticeable, but if you've been building your own images based\noff this image, you'll have to update your dockerfile. We recommend a\nmulti-stage dockerfile, where the build stage is based off of a regular Debian or\nother glibc-based image, and the assembly stage is based off of the ipfs/go-ipfs\nimage, and you copy build artifacts from the build stage to the assembly\nstage. Note, if you are using the docker image and see a deprecation message,\nplease update your usage. We will stop supporting the old method of starting\nthe dockerfile in the next release.\n\nFinally, I would like to thank all of our contributors, users, supporters, and\nfriends for helping us along the way. Ipfs would not be where it is without\nyou.\n\n\n- Features\n - Add `--pin` option to `ipfs dag put` ([ipfs/go-ipfs#4004](https://github.com/ipfs/go-ipfs/pull/4004))\n - Add `--pin` option to `ipfs object put` ([ipfs/go-ipfs#4095](https://github.com/ipfs/go-ipfs/pull/4095))\n - Implement `--profile` option on `ipfs init` ([ipfs/go-ipfs#4001](https://github.com/ipfs/go-ipfs/pull/4001))\n - Add CID Codecs to `ipfs block put` ([ipfs/go-ipfs#4022](https://github.com/ipfs/go-ipfs/pull/4022))\n - Bitswap sessions ([ipfs/go-ipfs#3867](https://github.com/ipfs/go-ipfs/pull/3867))\n - Create plugin API and loader, add ipld-git plugin ([ipfs/go-ipfs#4033](https://github.com/ipfs/go-ipfs/pull/4033))\n - Make announced swarm addresses configurable ([ipfs/go-ipfs#3948](https://github.com/ipfs/go-ipfs/pull/3948))\n - Reprovider strategies ([ipfs/go-ipfs#4113](https://github.com/ipfs/go-ipfs/pull/4113))\n - Circuit Relay integration ([ipfs/go-ipfs#4091](https://github.com/ipfs/go-ipfs/pull/4091))\n - More configurable datastore configs ([ipfs/go-ipfs#3575](https://github.com/ipfs/go-ipfs/pull/3575))\n - Add experimental support for badger datastore ([ipfs/go-ipfs#4007](https://github.com/ipfs/go-ipfs/pull/4007))\n- Improvements\n - Add better support for Raw Nodes in MFS and elsewhere ([ipfs/go-ipfs#3996](https://github.com/ipfs/go-ipfs/pull/3996))\n - Added file size to response of `ipfs add` command ([ipfs/go-ipfs#4082](https://github.com/ipfs/go-ipfs/pull/4082))\n - Add /dnsaddr bootstrap nodes ([ipfs/go-ipfs#4127](https://github.com/ipfs/go-ipfs/pull/4127))\n - Do not publish public keys extractable from ID ([ipfs/go-ipfs#4020](https://github.com/ipfs/go-ipfs/pull/4020))\n- Documentation\n - Adding documentation that PubSub Sub can be encoded. ([ipfs/go-ipfs#3909](https://github.com/ipfs/go-ipfs/pull/3909))\n - Add Comms items from js-ipfs, including blog ([ipfs/go-ipfs#3936](https://github.com/ipfs/go-ipfs/pull/3936))\n - Add Developer Certificate of Origin ([ipfs/go-ipfs#4006](https://github.com/ipfs/go-ipfs/pull/4006))\n - Add `transports.md` document ([ipfs/go-ipfs#4034](https://github.com/ipfs/go-ipfs/pull/4034))\n - Add `experimental-features.md` document ([ipfs/go-ipfs#4036](https://github.com/ipfs/go-ipfs/pull/4036))\n - Update release docs ([ipfs/go-ipfs#4165](https://github.com/ipfs/go-ipfs/pull/4165))\n - Add documentation for datastore configs ([ipfs/go-ipfs#4223](https://github.com/ipfs/go-ipfs/pull/4223))\n - General update and clean-up of docs ([ipfs/go-ipfs#4222](https://github.com/ipfs/go-ipfs/pull/4222))\n- Bug fixes\n - Fix shutdown check in t0023 ([ipfs/go-ipfs#3969](https://github.com/ipfs/go-ipfs/pull/3969))\n - Fix pinning of unixfs sharded directories ([ipfs/go-ipfs#3975](https://github.com/ipfs/go-ipfs/pull/3975))\n - Show escaped url in gateway 404 message ([ipfs/go-ipfs#4005](https://github.com/ipfs/go-ipfs/pull/4005))\n - Fix early opening of bitswap message sender ([ipfs/go-ipfs#4069](https://github.com/ipfs/go-ipfs/pull/4069))\n - Fix determination of 'root' node in dag put ([ipfs/go-ipfs#4072](https://github.com/ipfs/go-ipfs/pull/4072))\n - Fix bad multipart message panic in gateway ([ipfs/go-ipfs#4053](https://github.com/ipfs/go-ipfs/pull/4053))\n - Add blocks to the blockstore before returning them from blockservice sessions ([ipfs/go-ipfs#4169](https://github.com/ipfs/go-ipfs/pull/4169))\n - Various fixes for /ipfs fuse code ([ipfs/go-ipfs#4194](https://github.com/ipfs/go-ipfs/pull/4194))\n - Fix memory leak in dht stream tracking ([ipfs/go-ipfs#4251](https://github.com/ipfs/go-ipfs/pull/4251))\n- General Changes and Refactorings\n - Require go 1.8 ([ipfs/go-ipfs#4044](https://github.com/ipfs/go-ipfs/pull/4044))\n - Change IPFS to use the new pluggable Block to IPLD decoding framework. ([ipfs/go-ipfs#4060](https://github.com/ipfs/go-ipfs/pull/4060))\n - Remove tour command from ipfs ([ipfs/go-ipfs#4123](https://github.com/ipfs/go-ipfs/pull/4123))\n - Add support for Go 1.9 ([ipfs/go-ipfs#4156](https://github.com/ipfs/go-ipfs/pull/4156))\n - Remove some dead code ([ipfs/go-ipfs#4204](https://github.com/ipfs/go-ipfs/pull/4204))\n - Switch docker image from musl to glibc ([ipfs/go-ipfs#4219](https://github.com/ipfs/go-ipfs/pull/4219))\n\n### v0.4.10 - 2017-06-27\n\nIpfs 0.4.10 is a patch release that contains several exciting new features,\nbug fixes and general improvements. Including new commands, easier corruption\nrecovery, and a generally cleaner codebase.\n\nThe `ipfs pin` command has two new subcommands, `verify` and `update`. `ipfs\npin verify` is used to scan the repo for pinned object graphs and check their\nintegrity. Any issues are reported back with helpful error text to make error\nrecovery simpler. This subcommand was added to help recover from datastore\ncorruptions, particularly if using the experimental filestore and accidentally\ndeleting tracked files.\n`ipfs pin update` was added to make the task of keeping a large, frequently\nchanging object graph pinned. Previously users had to call `ipfs pin rm` on the\nold pin, and `ipfs pin add` on the new one. The 'new' `ipfs pin add` call would\nbe very expensive as it would need to verify the entirety of the graph again.\nThe `ipfs pin update` command takes shortcuts, portions of the graph that were\ncovered under the old pin are assumed to be fine, and the command skips\nchecking them.\n\nNext up, we have finally implemented an `ipfs shutdown` command so users can\nshut down their ipfs daemons via the API. This is especially useful on\nplatforms that make it difficult to control processes (Android, for example),\nand is also useful when needing to shut down a node remotely and you do not\nhave access to the machine itself.\n\n`ipfs add` has gained a new flag; the `--hash` flag allows you to select which\nhash function to use and we have given it the ability to select `blake2b-256`.\nThis pushes us one step closer to shifting over to using blake2b as the\ndefault. Blake2b is significantly faster than sha2-256, and also is conjectured\nto provide superior security.\n\nWe have also finally implemented a very early (and experimental) `ipfs p2p`.\nThis command and its subcommands will allow you to open up arbitrary streams to\nother ipfs peers through libp2p. The interfaces are a little bit clunky right\nnow, but shouldn't get in the way of anyone wanting to try building a fully\npeer to peer application on top of ipfs and libp2p. For more info on this\ncommand, to ask questions, or to provide feedback, head over to the [feedback\nissue](https://github.com/ipfs/go-ipfs/issues/3994) for the command.\n\nA few other subcommands and flags were added around the API, as well as many\nother requested improvements. See below for the full list of changes.\n\n\n- Features\n - Add support for specifying the hash function in `ipfs add` ([ipfs/go-ipfs#3919](https://github.com/ipfs/go-ipfs/pull/3919))\n - Implement `ipfs key {rm, rename}` ([ipfs/go-ipfs#3892](https://github.com/ipfs/go-ipfs/pull/3892))\n - Implement `ipfs shutdown` command ([ipfs/go-ipfs#3884](https://github.com/ipfs/go-ipfs/pull/3884))\n - Implement `ipfs pin update` ([ipfs/go-ipfs#3846](https://github.com/ipfs/go-ipfs/pull/3846))\n - Implement `ipfs pin verify` ([ipfs/go-ipfs#3843](https://github.com/ipfs/go-ipfs/pull/3843))\n - Implemented experimental p2p commands ([ipfs/go-ipfs#3943](https://github.com/ipfs/go-ipfs/pull/3943))\n- Improvements\n - Add MaxStorage field to output of \"repo stat\" ([ipfs/go-ipfs#3915](https://github.com/ipfs/go-ipfs/pull/3915))\n - Add Suborigin header to gateway responses ([ipfs/go-ipfs#3914](https://github.com/ipfs/go-ipfs/pull/3914))\n - Add \"--file-order\" option to \"filestore ls\" and \"verify\" ([ipfs/go-ipfs#3938](https://github.com/ipfs/go-ipfs/pull/3938))\n - Allow selecting ipns keys by Peer ID ([ipfs/go-ipfs#3882](https://github.com/ipfs/go-ipfs/pull/3882))\n - Don't redirect to trailing slash in gateway for `go get` ([ipfs/go-ipfs#3963](https://github.com/ipfs/go-ipfs/pull/3963))\n - Add 'ipfs dht findprovs --num-providers' to allow choosing number of providers to find ([ipfs/go-ipfs#3966](https://github.com/ipfs/go-ipfs/pull/3966))\n - Make sure all keystore keys get republished ([ipfs/go-ipfs#3951](https://github.com/ipfs/go-ipfs/pull/3951))\n- Documentation\n - Adding documentation on PubSub encodings ([ipfs/go-ipfs#3909](https://github.com/ipfs/go-ipfs/pull/3909))\n - Change 'necessary' to 'necessary' ([ipfs/go-ipfs#3941](https://github.com/ipfs/go-ipfs/pull/3941))\n - README.md: add Nix to the linux package managers ([ipfs/go-ipfs#3939](https://github.com/ipfs/go-ipfs/pull/3939))\n - More verbose errors in filestore ([ipfs/go-ipfs#3964](https://github.com/ipfs/go-ipfs/pull/3964))\n- Bug fixes\n - Fix typo in message when file size check fails ([ipfs/go-ipfs#3895](https://github.com/ipfs/go-ipfs/pull/3895))\n - Clean up bitswap ledgers when disconnecting ([ipfs/go-ipfs#3437](https://github.com/ipfs/go-ipfs/pull/3437))\n - Make odds of 'process added after close' panic less likely ([ipfs/go-ipfs#3940](https://github.com/ipfs/go-ipfs/pull/3940))\n- General Changes and Refactorings\n - Remove 'ipfs diag net' from codebase ([ipfs/go-ipfs#3916](https://github.com/ipfs/go-ipfs/pull/3916))\n - Update to dht code with provide announce option ([ipfs/go-ipfs#3928](https://github.com/ipfs/go-ipfs/pull/3928))\n - Apply the megacheck code vetting tool ([ipfs/go-ipfs#3949](https://github.com/ipfs/go-ipfs/pull/3949))\n - Expose port 8081 in docker container for /ws listener ([ipfs/go-ipfs#3954](https://github.com/ipfs/go-ipfs/pull/3954))\n\n### v0.4.9 - 2017-04-30\n\nIpfs 0.4.9 is a maintenance release that contains several useful bug fixes and\nimprovements. Notably, `ipfs add` has gained the ability to select which CID\nversion will be output. The common ipfs hash that looks like this:\n`QmRjNgF2mRLDT8AzCPsQbw1EYF2hDTFgfUmJokJPhCApYP` is a multihash. Multihashes\nallow us to specify the hashing algorithm that was used to verify the data, but\nit doesn't give us any indication of what format that data might be. To address\nthat issue, we are adding another couple of bytes to the prefix that will allow us\nto indicate the format of the data referenced by the hash. This new format is\ncalled a Content ID, or CID for short. The previous bare multihashes will still\nbe fully supported throughout the entire application as CID version 0. The new\nformat with the type information will be CID version 1. To give an example,\nthe content referenced by the hash above is \"Hello Ipfs!\". That same content,\nin the same format (dag-protobuf) using CIDv1 is\n`zb2rhkgXZVkT2xvDiuUsJENPSbWJy7fdYnsboLBzzEjjZMRoG`.\n\nCIDv1 hashes are supported in ipfs versions back to 0.4.5. Nodes running 0.4.4\nand older will not be able to load content via CIDv1 and we recommend that they\nupdate to a newer version.\n\nThere are many other use cases for CIDs. Plugins can be written to\nallow ipfs to natively address content from any other merkletree based system,\nsuch as git, bitcoin, zcash and ethereum -- a few systems we've already started work on.\n\nAside from the CID flag, there were many other changes as noted below:\n\n- Features\n - Add support for using CidV1 in 'ipfs add' ([ipfs/go-ipfs#3743](https://github.com/ipfs/go-ipfs/pull/3743))\n- Improvements\n - Use CID as an ETag strong validator ([ipfs/go-ipfs#3869](https://github.com/ipfs/go-ipfs/pull/3869))\n - Update go-multihash with keccak and bitcoin hashes ([ipfs/go-ipfs#3833](https://github.com/ipfs/go-ipfs/pull/3833))\n - Update go-is-domain to contain new gTLD ([ipfs/go-ipfs#3873](https://github.com/ipfs/go-ipfs/pull/3873))\n - Periodically flush cached directories during ipfs add ([ipfs/go-ipfs#3888](https://github.com/ipfs/go-ipfs/pull/3888))\n - improved gateway directory listing for sharded nodes ([ipfs/go-ipfs#3897](https://github.com/ipfs/go-ipfs/pull/3897))\n- Documentation\n - Change issue template to use Severity instead of Priority ([ipfs/go-ipfs#3834](https://github.com/ipfs/go-ipfs/pull/3834))\n - Fix link to commit hook script in contribute.md ([ipfs/go-ipfs#3863](https://github.com/ipfs/go-ipfs/pull/3863))\n - Fix install_unsupported for openbsd, add docs ([ipfs/go-ipfs#3880](https://github.com/ipfs/go-ipfs/pull/3880))\n- Bug fixes\n - Fix wantlist typo in Prometheus metric name ([ipfs/go-ipfs#3841](https://github.com/ipfs/go-ipfs/pull/3841))\n - Fix `make install` not using ldflags for git hash ([ipfs/go-ipfs#3838](https://github.com/ipfs/go-ipfs/pull/3838))\n - Fix `make install` not installing dependencies ([ipfs/go-ipfs#3848](https://github.com/ipfs/go-ipfs/pull/3848))\n - Fix erroneous Cache-Control: immutable on dir listings ([ipfs/go-ipfs#3870](https://github.com/ipfs/go-ipfs/pull/3870))\n - Fix bitswap accounting of 'BytesSent' in ledger ([ipfs/go-ipfs#3876](https://github.com/ipfs/go-ipfs/pull/3876))\n - Fix gateway handling of sharded directories ([ipfs/go-ipfs#3889](https://github.com/ipfs/go-ipfs/pull/3889))\n - Fix sharding memory growth, and fix resolver for unixfs paths ([ipfs/go-ipfs#3890](https://github.com/ipfs/go-ipfs/pull/3890))\n- General Changes and Refactorings\n - Use ctx var consistently in daemon.go ([ipfs/go-ipfs#3864](https://github.com/ipfs/go-ipfs/pull/3864))\n - Handle 404 correctly in dist_get tool ([ipfs/go-ipfs#3879](https://github.com/ipfs/go-ipfs/pull/3879))\n- Testing\n - Fix go fuse tests ([ipfs/go-ipfs#3840](https://github.com/ipfs/go-ipfs/pull/3840))\n\n### v0.4.8 - 2017-03-29\n\nIpfs 0.4.8 brings with it several improvements, bug fixes, documentation\nimprovements, and the long awaited directory sharding code.\n\nCurrently, when too many items are added into a unixfs directory, the object\ngets too large and you may experience issues. To prevent this problem, and\ngenerally make working really large directories more efficient, we have\nimplemented a HAMT structure for unixfs. To enable this feature, run:\n```\nipfs config --json Experimental.ShardingEnabled true\n```\n\nAnd restart your daemon if it was running.\n\nNote: With this setting enabled, the hashes of any newly added directories will\nbe different than they previously were, as the new code will use the sharded\nHAMT structure for all directories. Also, nodes running ipfs 0.4.7 and earlier\nwill not be able to access directories created with this option.\n\nThat said, please do give it a try, let us know how it goes, and then take a\nlook at all the other cool things added in 0.4.8 below.\n\n- Features\n\t- Implement unixfs directory sharding ([ipfs/go-ipfs#3042](https://github.com/ipfs/go-ipfs/pull/3042))\n\t- Add DisableNatPortMap option ([ipfs/go-ipfs#3798](https://github.com/ipfs/go-ipfs/pull/3798))\n\t- Basic Filestore utility commands ([ipfs/go-ipfs#3653](https://github.com/ipfs/go-ipfs/pull/3653))\n- Improvements\n\t- More Robust GC ([ipfs/go-ipfs#3712](https://github.com/ipfs/go-ipfs/pull/3712))\n\t- Automatically fix permissions for docker volumes ([ipfs/go-ipfs#3744](https://github.com/ipfs/go-ipfs/pull/3744))\n\t- Core API refinements and efficiency improvements ([ipfs/go-ipfs#3493](https://github.com/ipfs/go-ipfs/pull/3493))\n\t- Improve IsPinned() lookups for indirect pins ([ipfs/go-ipfs#3809](https://github.com/ipfs/go-ipfs/pull/3809))\n- Documentation\n\t- Improve 'name' and 'key' helptexts ([ipfs/go-ipfs#3806](https://github.com/ipfs/go-ipfs/pull/3806))\n\t- Update link to paper in dev.md ([ipfs/go-ipfs#3812](https://github.com/ipfs/go-ipfs/pull/3812))\n\t- Add test to enforce helptext on commands ([ipfs/go-ipfs#2648](https://github.com/ipfs/go-ipfs/pull/2648))\n- Bug fixes\n\t- Remove bloom filter check on Put call in blockstore ([ipfs/go-ipfs#3782](https://github.com/ipfs/go-ipfs/pull/3782))\n\t- Re-add the GOPATH checking functionality ([ipfs/go-ipfs#3787](https://github.com/ipfs/go-ipfs/pull/3787))\n\t- Use fsrepo.IsInitialized to test for initialization ([ipfs/go-ipfs#3805](https://github.com/ipfs/go-ipfs/pull/3805))\n\t- Return 404 Not Found for failed path resolutions ([ipfs/go-ipfs#3777](https://github.com/ipfs/go-ipfs/pull/3777))\n\t- Fix 'dist\\_get' failing without failing ([ipfs/go-ipfs#3818](https://github.com/ipfs/go-ipfs/pull/3818))\n\t- Update iptb with fix for t0130 hanging issue ([ipfs/go-ipfs#3823](https://github.com/ipfs/go-ipfs/pull/3823))\n\t- fix hidden file detection on windows ([ipfs/go-ipfs#3829](https://github.com/ipfs/go-ipfs/pull/3829))\n- General Changes and Refactorings\n\t- Fix multiple govet warnings ([ipfs/go-ipfs#3824](https://github.com/ipfs/go-ipfs/pull/3824))\n\t- Make Golint happy in the blocks submodule ([ipfs/go-ipfs#3827](https://github.com/ipfs/go-ipfs/pull/3827))\n- Testing\n\t- Enable codeclimate for automated linting and vetting ([ipfs/go-ipfs#3821](https://github.com/ipfs/go-ipfs/pull/3821))\n\t- Fix EOF test failure with Multipart.Read ([ipfs/go-ipfs#3804](https://github.com/ipfs/go-ipfs/pull/3804))\n\n### v0.4.7 - 2017-03-15\n\nIpfs 0.4.7 contains several exciting new features!\nFirst off, The long awaited filestore feature has been merged, allowing users\nthe option to not have ipfs store chunked copies of added files in the\nblockstore, pushing to burden of ensuring those files are not changed to the\nuser. The filestore feature is currently still experimental, and must be\nenabled in your config with:\n```\nipfs config --json Experimental.FilestoreEnabled true\n```\nbefore it can be used. Please see [this issue](https://github.com/ipfs/go-ipfs/issues/3397#issuecomment-284337564) for more details.\n\nNext up, We have merged initial support for ipfs 'Private Networks'. This\nfeature allows users to run ipfs in a mode that will only connect to other\npeers in the private network. This feature, like the filestore is being\nreleased experimentally, but if you're interested please try it out.\nInstructions for setting it up can be found\n[here](https://github.com/ipfs/go-ipfs/issues/3397#issuecomment-284341649).\n\nThis release also enables support for the 'mplex' stream muxer by default. This\nstream multiplexing protocol was available previously via the\n`--enable-mplex-experiment` daemon flag, but has now graduated to being 'less\nexperimental' and no longer requires the flag to use it.\n\nAside from those, we have a good number of bug fixes, perf improvements and new\ntests. Heres a list of highlights:\n\n- Features\n\t- Implement basic filestore 'no-copy' functionality ([ipfs/go-ipfs#3629](https://github.com/ipfs/go-ipfs/pull/3629))\n\t- Add support for private ipfs networks ([ipfs/go-ipfs#3697](https://github.com/ipfs/go-ipfs/pull/3697))\n\t- Enable 'mplex' stream muxer by default ([ipfs/go-ipfs#3725](https://github.com/ipfs/go-ipfs/pull/3725))\n\t- Add `--quieter` option to `ipfs add` ([ipfs/go-ipfs#3770](https://github.com/ipfs/go-ipfs/pull/3770))\n\t- Report progress during `pin add` via `--progress` ([ipfs/go-ipfs#3671](https://github.com/ipfs/go-ipfs/pull/3671))\n- Improvements\n\t- Allow `ipfs get` to handle content added with raw leaves option ([ipfs/go-ipfs#3757](https://github.com/ipfs/go-ipfs/pull/3757))\n\t- Fix accuracy of progress bar on `ipfs get` ([ipfs/go-ipfs#3758](https://github.com/ipfs/go-ipfs/pull/3758))\n\t- Limit number of objects in batches to prevent too many fds issue ([ipfs/go-ipfs#3756](https://github.com/ipfs/go-ipfs/pull/3756))\n\t- Add more info to bitswap stat ([ipfs/go-ipfs#3635](https://github.com/ipfs/go-ipfs/pull/3635))\n\t- Add multiple performance metrics ([ipfs/go-ipfs#3615](https://github.com/ipfs/go-ipfs/pull/3615))\n\t- Make `dist_get` fall back to other downloaders if one fails ([ipfs/go-ipfs#3692](https://github.com/ipfs/go-ipfs/pull/3692))\n- Documentation\n\t- Add Arch Linux install instructions to readme ([ipfs/go-ipfs#3742](https://github.com/ipfs/go-ipfs/pull/3742))\n\t- Improve release checklist document ([ipfs/go-ipfs#3717](https://github.com/ipfs/go-ipfs/pull/3717))\n- Bug fixes\n\t- Fix drive root parsing on windows ([ipfs/go-ipfs#3328](https://github.com/ipfs/go-ipfs/pull/3328))\n\t- Fix panic in ipfs get when passing no parameters to API ([ipfs/go-ipfs#3768](https://github.com/ipfs/go-ipfs/pull/3768))\n\t- Fix breakage of `ipfs pin add` api output ([ipfs/go-ipfs#3760](https://github.com/ipfs/go-ipfs/pull/3760))\n\t- Fix issue in DHT queries that was causing poor record replication ([ipfs/go-ipfs#3748](https://github.com/ipfs/go-ipfs/pull/3748))\n\t- Fix `ipfs mount` crashing if no name was published before ([ipfs/go-ipfs#3728](https://github.com/ipfs/go-ipfs/pull/3728))\n\t- Add `self` key to the `ipfs key list` listing ([ipfs/go-ipfs#3734](https://github.com/ipfs/go-ipfs/pull/3734))\n\t- Fix panic when shutting down `ipfs daemon` pre gateway setup ([ipfs/go-ipfs#3723](https://github.com/ipfs/go-ipfs/pull/3723))\n- General Changes and Refactorings\n\t- Refactor `EnumerateChildren` to avoid need for bestEffort parameter ([ipfs/go-ipfs#3700](https://github.com/ipfs/go-ipfs/pull/3700))\n\t- Update fuse dependency, fixing several issues ([ipfs/go-ipfs#3727](https://github.com/ipfs/go-ipfs/pull/3727))\n\t- Add `install_unsupported` makefile target for 'exotic' systems ([ipfs/go-ipfs#3719](https://github.com/ipfs/go-ipfs/pull/3719))\n\t- Deprecate implicit daemon argument in Dockerfile ([ipfs/go-ipfs#3685](https://github.com/ipfs/go-ipfs/pull/3685))\n- Testing\n\t- Add test to ensure helptext is under 80 columns wide ([ipfs/go-ipfs#3774](https://github.com/ipfs/go-ipfs/pull/3774))\n\t- Add unit tests for auto migration code ([ipfs/go-ipfs#3618](https://github.com/ipfs/go-ipfs/pull/3618))\n\t- Fix iptb stop issue in sharness tests ([ipfs/go-ipfs#3714](https://github.com/ipfs/go-ipfs/pull/3714))\n\n\n### v0.4.6 - 2017-02-21\n\nIpfs 0.4.6 contains several bug fixes related to migrations and also contains a\nfew other improvements to other parts of the codebase. Notably:\n\n- The default config will now contain some ipv6 addresses for bootstrap nodes.\n- `ipfs pin add` should be faster and consume less memory.\n- Pinning thousands of files no longer causes superlinear usage of storage space.\n\n- Improvements\n\t- Make pinset sharding deterministic ([ipfs/go-ipfs#3640](https://github.com/ipfs/go-ipfs/pull/3640))\n\t- Update to go-multihash with blake2 ([ipfs/go-ipfs#3649](https://github.com/ipfs/go-ipfs/pull/3649))\n\t- Pass cids instead of nodes around in EnumerateChildrenAsync ([ipfs/go-ipfs#3598](https://github.com/ipfs/go-ipfs/pull/3598))\n\t- Add /ip6 bootstrap nodes ([ipfs/go-ipfs#3523](https://github.com/ipfs/go-ipfs/pull/3523))\n\t- Add sub-object support to `dag get` command ([ipfs/go-ipfs#3687](https://github.com/ipfs/go-ipfs/pull/3687))\n\t- Add half-closed streams support to multiplex experiment ([ipfs/go-ipfs#3695](https://github.com/ipfs/go-ipfs/pull/3695))\n- Documentation\n\t- Add the snap installation instructions ([ipfs/go-ipfs#3663](https://github.com/ipfs/go-ipfs/pull/3663))\n\t- Add closed PRs, Issues throughput ([ipfs/go-ipfs#3602](https://github.com/ipfs/go-ipfs/pull/3602))\n- Bug fixes\n\t- Fix auto-migration on docker nodes ([ipfs/go-ipfs#3698](https://github.com/ipfs/go-ipfs/pull/3698))\n\t- Update flatfs to v1.1.2, fixing directory fd issue ([ipfs/go-ipfs#3711](https://github.com/ipfs/go-ipfs/pull/3711))\n- General Changes and Refactorings\n\t- Remove `FindProviders` from routing mocks ([ipfs/go-ipfs#3617](https://github.com/ipfs/go-ipfs/pull/3617))\n\t- Use Marshalers instead of PostRun to process `block rm` output ([ipfs/go-ipfs#3708](https://github.com/ipfs/go-ipfs/pull/3708))\n- Testing\n\t- Makefile rework and sharness test coverage ([ipfs/go-ipfs#3504](https://github.com/ipfs/go-ipfs/pull/3504))\n\t- Print out all daemon stderr files when iptb stop fails ([ipfs/go-ipfs#3701](https://github.com/ipfs/go-ipfs/pull/3701))\n\t- Add tests for recursively pinning a dag ([ipfs/go-ipfs#3691](https://github.com/ipfs/go-ipfs/pull/3691))\n\t- Fix lack of commit hash during build ([ipfs/go-ipfs#3705](https://github.com/ipfs/go-ipfs/pull/3705))\n\n### v0.4.5 - 2017-02-11\n\n#### Changes from rc3 to rc4\n- Update to fixed webui. ([ipfs/go-ipfs#3669](https://github.com/ipfs/go-ipfs/pull/3669))\n\n#### Changes from rc2 to rc3\n- Fix handling of null arrays in cbor ipld objects. ([ipfs/go-ipfs#3666](https://github.com/ipfs/go-ipfs/pull/3666))\n- Add env var to enable yamux debug logging. ([ipfs/go-ipfs#3668](https://github.com/ipfs/go-ipfs/pull/3668))\n- Fix libc check during auto-migrations. ([ipfs/go-ipfs#3665](https://github.com/ipfs/go-ipfs/pull/3665))\n\n#### Changes from rc1 to rc2\n- Fixed json output of ipld objects in `ipfs dag get` ([ipfs/go-ipfs#3655](https://github.com/ipfs/go-ipfs/pull/3655))\n\n#### Changes since 0.4.4\n\n- Notable changes\n\t- IPLD and CIDs\n\t - Rework go-ipfs to use Content IDs ([ipfs/go-ipfs#3187](https://github.com/ipfs/go-ipfs/pull/3187)) ([ipfs/go-ipfs#3290](https://github.com/ipfs/go-ipfs/pull/3290))\n\t - Turn merkledag.Node into an interface ([ipfs/go-ipfs#3301](https://github.com/ipfs/go-ipfs/pull/3301))\n\t - Implement cbor ipld nodes ([ipfs/go-ipfs#3325](https://github.com/ipfs/go-ipfs/pull/3325))\n\t - Allow cid format selection in block put command ([ipfs/go-ipfs#3324](https://github.com/ipfs/go-ipfs/pull/3324)) ([ipfs/go-ipfs#3483](https://github.com/ipfs/go-ipfs/pull/3483))\n\t - Bitswap protocol extension to handle cids ([ipfs/go-ipfs#3297](https://github.com/ipfs/go-ipfs/pull/3297))\n\t - Add dag get to read-only api ([ipfs/go-ipfs#3499](https://github.com/ipfs/go-ipfs/pull/3499))\n\t- Raw Nodes\n\t - Implement 'Raw Node' node type for addressing raw data ([ipfs/go-ipfs#3307](https://github.com/ipfs/go-ipfs/pull/3307))\n\t - Optimize DagService GetLinks for Raw Nodes. ([ipfs/go-ipfs#3351](https://github.com/ipfs/go-ipfs/pull/3351))\n\t- Experimental PubSub\n\t - Added a very basic pubsub implementation ([ipfs/go-ipfs#3202](https://github.com/ipfs/go-ipfs/pull/3202))\n\t- Core API\n\t - gateway: use core api for serving GET/HEAD/POST ([ipfs/go-ipfs#3244](https://github.com/ipfs/go-ipfs/pull/3244))\n\n- Improvements\n\t- Disable auto-gc check in 'ipfs cat' ([ipfs/go-ipfs#3100](https://github.com/ipfs/go-ipfs/pull/3100))\n\t- Add `bitswap ledger` command ([ipfs/go-ipfs#2852](https://github.com/ipfs/go-ipfs/pull/2852))\n\t- Add `ipfs block rm` command. ([ipfs/go-ipfs#2962](https://github.com/ipfs/go-ipfs/pull/2962))\n\t- Add config option to disable bandwidth metrics ([ipfs/go-ipfs#3381](https://github.com/ipfs/go-ipfs/pull/3381))\n\t- Add experimental dht 'client mode' flag ([ipfs/go-ipfs#3269](https://github.com/ipfs/go-ipfs/pull/3269))\n\t- Add config option to set reprovider interval ([ipfs/go-ipfs#3101](https://github.com/ipfs/go-ipfs/pull/3101))\n\t- Add `ipfs dht provide` command ([ipfs/go-ipfs#3106](https://github.com/ipfs/go-ipfs/pull/3106))\n\t- Add stream info to `ipfs swarm peers -v` ([ipfs/go-ipfs#3352](https://github.com/ipfs/go-ipfs/pull/3352))\n\t- Add option to enable go-multiplex experiment ([ipfs/go-ipfs#3447](https://github.com/ipfs/go-ipfs/pull/3447))\n\t- Basic Keystore implementation ([ipfs/go-ipfs#3472](https://github.com/ipfs/go-ipfs/pull/3472))\n\t- Make `ipfs add --local` not send providers messages ([ipfs/go-ipfs#3102](https://github.com/ipfs/go-ipfs/pull/3102))\n\t- Fix bug in `ipfs tar add` that buffered input in memory ([ipfs/go-ipfs#3334](https://github.com/ipfs/go-ipfs/pull/3334))\n\t- Make blockstore retry operations on temporary errors ([ipfs/go-ipfs#3091](https://github.com/ipfs/go-ipfs/pull/3091))\n\t- Don't hold the PinLock in adder when not pinning. ([ipfs/go-ipfs#3222](https://github.com/ipfs/go-ipfs/pull/3222))\n\t- Validate repo/api file and improve error message ([ipfs/go-ipfs#3219](https://github.com/ipfs/go-ipfs/pull/3219))\n\t- no longer hard code gomaxprocs ([ipfs/go-ipfs#3357](https://github.com/ipfs/go-ipfs/pull/3357))\n\t- Updated Bash complete script ([ipfs/go-ipfs#3377](https://github.com/ipfs/go-ipfs/pull/3377))\n\t- Remove expensive debug statement in blockstore AllKeysChan ([ipfs/go-ipfs#3384](https://github.com/ipfs/go-ipfs/pull/3384))\n\t- Remove GC timeout, fix GC tests ([ipfs/go-ipfs#3494](https://github.com/ipfs/go-ipfs/pull/3494))\n\t- Fix `ipfs pin add` resource consumption ([ipfs/go-ipfs#3495](https://github.com/ipfs/go-ipfs/pull/3495)) ([ipfs/go-ipfs#3571](https://github.com/ipfs/go-ipfs/pull/3571))\n\t- Add IPNS entry to DHT cache after publish ([ipfs/go-ipfs#3501](https://github.com/ipfs/go-ipfs/pull/3501))\n\t- Add in `--routing=none` daemon option ([ipfs/go-ipfs#3605](https://github.com/ipfs/go-ipfs/pull/3605))\n\n- Bitswap\n\t- Don't re-provide blocks we've provided very recently ([ipfs/go-ipfs#3105](https://github.com/ipfs/go-ipfs/pull/3105))\n\t- Add a deadline to sendmsg calls ([ipfs/go-ipfs#3445](https://github.com/ipfs/go-ipfs/pull/3445))\n\t- cleanup bitswap and handle message send failure slightly better ([ipfs/go-ipfs#3408](https://github.com/ipfs/go-ipfs/pull/3408))\n\t- Increase wantlist resend delay to one minute ([ipfs/go-ipfs#3448](https://github.com/ipfs/go-ipfs/pull/3448))\n\t- Fix issue where wantlist fullness wasn't included in messages ([ipfs/go-ipfs#3461](https://github.com/ipfs/go-ipfs/pull/3461))\n\t- Only pass keys down newBlocks chan in bitswap ([ipfs/go-ipfs#3271](https://github.com/ipfs/go-ipfs/pull/3271))\n\n- Bug fixes\n\t- gateway: fix --writable flag ([ipfs/go-ipfs#3206](https://github.com/ipfs/go-ipfs/pull/3206))\n\t- Fix relative seek in unixfs not expanding file properly ([ipfs/go-ipfs#3095](https://github.com/ipfs/go-ipfs/pull/3095))\n\t- Update multicodec service names for ipfs services ([ipfs/go-ipfs#3132](https://github.com/ipfs/go-ipfs/pull/3132))\n\t- dht: add missing protocol ID to newStream call ([ipfs/go-ipfs#3203](https://github.com/ipfs/go-ipfs/pull/3203))\n\t- Return immediately on namesys error ([ipfs/go-ipfs#3345](https://github.com/ipfs/go-ipfs/pull/3345))\n\t- Improve osxfuse handling ([ipfs/go-ipfs#3098](https://github.com/ipfs/go-ipfs/pull/3098)) ([ipfs/go-ipfs#3413](https://github.com/ipfs/go-ipfs/pull/3413))\n\t- commands: fix opt.Description panic when desc was empty ([ipfs/go-ipfs#3521](https://github.com/ipfs/go-ipfs/pull/3521))\n\t- Fixes #3133: Properly handle release candidates in version comparison ([ipfs/go-ipfs#3136](https://github.com/ipfs/go-ipfs/pull/3136))\n\t- Don't drop error in readStreamedJson. ([ipfs/go-ipfs#3276](https://github.com/ipfs/go-ipfs/pull/3276))\n\t- Error out on invalid `--routing` option ([ipfs/go-ipfs#3482](https://github.com/ipfs/go-ipfs/pull/3482))\n\t- Respect contexts when returning diagnostics responses ([ipfs/go-ipfs#3353](https://github.com/ipfs/go-ipfs/pull/3353))\n\t- Fix json marshalling of pbnode ([ipfs/go-ipfs#3507](https://github.com/ipfs/go-ipfs/pull/3507))\n\n- General changes and refactorings\n\t- Disable Suborigins the spec changed and our impl conflicts ([ipfs/go-ipfs#3519](https://github.com/ipfs/go-ipfs/pull/3519))\n\t- Avoid sending provide messages for pinsets ([ipfs/go-ipfs#3103](https://github.com/ipfs/go-ipfs/pull/3103))\n\t- Refactor cli handling to expose argument parsing functionality ([ipfs/go-ipfs#3308](https://github.com/ipfs/go-ipfs/pull/3308))\n\t- Create a FilestoreNode object to carry PosInfo ([ipfs/go-ipfs#3314](https://github.com/ipfs/go-ipfs/pull/3314))\n\t- Print 'n/a' instead of zero latency in `ipfs swarm peers` ([ipfs/go-ipfs#3491](https://github.com/ipfs/go-ipfs/pull/3491))\n\t- Add DAGService.GetLinks() method to optimize traversals. ([ipfs/go-ipfs#3255](https://github.com/ipfs/go-ipfs/pull/3255))\n\t- Make path resolver no longer require whole IpfsNode for construction ([ipfs/go-ipfs#3321](https://github.com/ipfs/go-ipfs/pull/3321))\n\t- Distinguish between Offline and Local Modes of daemon operation. ([ipfs/go-ipfs#3259](https://github.com/ipfs/go-ipfs/pull/3259))\n\t- Separate out the GC Locking from the Blockstore interface. ([ipfs/go-ipfs#3348](https://github.com/ipfs/go-ipfs/pull/3348))\n\t- Avoid unnecessary allocs in datastore key handling ([ipfs/go-ipfs#3407](https://github.com/ipfs/go-ipfs/pull/3407))\n\t- Use NextSync method for datastore queries ([ipfs/go-ipfs#3386](https://github.com/ipfs/go-ipfs/pull/3386))\n\t- Switch unixfs.Metadata.MimeType to optional ([ipfs/go-ipfs#3458](https://github.com/ipfs/go-ipfs/pull/3458))\n\t- Fix path parsing in `ipfs name publish` ([ipfs/go-ipfs#3592](https://github.com/ipfs/go-ipfs/pull/3592))\n\t- Fix inconsistent `ipfs stats bw` formatting ([ipfs/go-ipfs#3554](https://github.com/ipfs/go-ipfs/pull/3554))\n\t- Set the libp2p agent version based on version string ([ipfs/go-ipfs#3569](https://github.com/ipfs/go-ipfs/pull/3569))\n\n- Cross Platform Changes\n\t- Fix 'dist_get' script on BSDs. ([ipfs/go-ipfs#3264](https://github.com/ipfs/go-ipfs/pull/3264))\n\t- ulimit: Tune resource limits on BSDs ([ipfs/go-ipfs#3374](https://github.com/ipfs/go-ipfs/pull/3374))\n\n- Metrics\n\t- Introduce go-metrics-interface ([ipfs/go-ipfs#3189](https://github.com/ipfs/go-ipfs/pull/3189))\n\t- Fix metrics injection ([ipfs/go-ipfs#3315](https://github.com/ipfs/go-ipfs/pull/3315))\n\n- Misc\n\t- Bump Go requirement to 1.7 ([ipfs/go-ipfs#3111](https://github.com/ipfs/go-ipfs/pull/3111))\n\t- Merge 0.4.3 release candidate changes back into master ([ipfs/go-ipfs#3248](https://github.com/ipfs/go-ipfs/pull/3248))\n\t- Add security@ipfs.io GPG key to assets ([ipfs/go-ipfs#2997](https://github.com/ipfs/go-ipfs/pull/2997))\n\t- Improve makefiles ([ipfs/go-ipfs#2999](https://github.com/ipfs/go-ipfs/pull/2999)) ([ipfs/go-ipfs#3265](https://github.com/ipfs/go-ipfs/pull/3265))\n\t- Refactor install.sh script ([ipfs/go-ipfs#3194](https://github.com/ipfs/go-ipfs/pull/3194))\n\t- Add test check for go code formatting ([ipfs/go-ipfs#3421](https://github.com/ipfs/go-ipfs/pull/3421))\n\t- bin: dist_get script: prevents get_go_vars() returns same values twice ([ipfs/go-ipfs#3079](https://github.com/ipfs/go-ipfs/pull/3079))\n\n- Dependencies\n\t- Update libp2p to have fixed spdystream dep ([ipfs/go-ipfs#3210](https://github.com/ipfs/go-ipfs/pull/3210))\n\t- Update libp2p and dht packages ([ipfs/go-ipfs#3263](https://github.com/ipfs/go-ipfs/pull/3263))\n\t- Update to libp2p 4.0.1 and propagate other changes ([ipfs/go-ipfs#3284](https://github.com/ipfs/go-ipfs/pull/3284))\n\t- Update to libp2p 4.0.4 ([ipfs/go-ipfs#3361](https://github.com/ipfs/go-ipfs/pull/3361))\n\t- Update go-libp2p across codebase ([ipfs/go-ipfs#3406](https://github.com/ipfs/go-ipfs/pull/3406))\n\t- Update to go-libp2p 4.1.0 ([ipfs/go-ipfs#3373](https://github.com/ipfs/go-ipfs/pull/3373))\n\t- Update deps for libp2p 3.4.0 ([ipfs/go-ipfs#3110](https://github.com/ipfs/go-ipfs/pull/3110))\n\t- Update go-libp2p-swarm with deadlock fixes ([ipfs/go-ipfs#3339](https://github.com/ipfs/go-ipfs/pull/3339))\n\t- Update to new cid and ipld node packages ([ipfs/go-ipfs#3326](https://github.com/ipfs/go-ipfs/pull/3326))\n\t- Update to newer ipld node interface with Copy and better Tree ([ipfs/go-ipfs#3391](https://github.com/ipfs/go-ipfs/pull/3391))\n\t- Update experimental go-multiplex to 0.2.6 ([ipfs/go-ipfs#3475](https://github.com/ipfs/go-ipfs/pull/3475))\n\t- Rework routing interfaces to make separation easier ([ipfs/go-ipfs#3107](https://github.com/ipfs/go-ipfs/pull/3107))\n\t- Update to dht code with fixed GetClosestPeers ([ipfs/go-ipfs#3346](https://github.com/ipfs/go-ipfs/pull/3346))\n\t- Move go-is-domain to gx ([ipfs/go-ipfs#3077](https://github.com/ipfs/go-ipfs/pull/3077))\n\t- Extract thirdparty/loggables and thirdparty/peerset ([ipfs/go-ipfs#3204](https://github.com/ipfs/go-ipfs/pull/3204))\n\t- Completely remove go-key dep ([ipfs/go-ipfs#3439](https://github.com/ipfs/go-ipfs/pull/3439))\n\t- Remove randbo dep, its no longer needed ([ipfs/go-ipfs#3118](https://github.com/ipfs/go-ipfs/pull/3118))\n\t- Update libp2p for identify configuration updates ([ipfs/go-ipfs#3539](https://github.com/ipfs/go-ipfs/pull/3539))\n\t- Use newer flatfs sharding scheme ([ipfs/go-ipfs#3608](https://github.com/ipfs/go-ipfs/pull/3608))\n\n- Testing\n\t- fix test_fsh arg quoting in ipfs-test-lib ([ipfs/go-ipfs#3085](https://github.com/ipfs/go-ipfs/pull/3085))\n\t- 100% coverage for blocks/blocksutil ([ipfs/go-ipfs#3090](https://github.com/ipfs/go-ipfs/pull/3090))\n\t- 100% coverage on blocks/set ([ipfs/go-ipfs#3084](https://github.com/ipfs/go-ipfs/pull/3084))\n\t- 81% coverage on blockstore ([ipfs/go-ipfs#3074](https://github.com/ipfs/go-ipfs/pull/3074))\n\t- 80% coverage of unixfs/mod ([ipfs/go-ipfs#3096](https://github.com/ipfs/go-ipfs/pull/3096))\n\t- 82% coverage on blocks ([ipfs/go-ipfs#3086](https://github.com/ipfs/go-ipfs/pull/3086))\n\t- 87% coverage on unixfs ([ipfs/go-ipfs#3492](https://github.com/ipfs/go-ipfs/pull/3492))\n\t- Improve coverage on routing/offline ([ipfs/go-ipfs#3516](https://github.com/ipfs/go-ipfs/pull/3516))\n\t- Add test for flags package ([ipfs/go-ipfs#3449](https://github.com/ipfs/go-ipfs/pull/3449))\n\t- improve test coverage on merkledag package ([ipfs/go-ipfs#3113](https://github.com/ipfs/go-ipfs/pull/3113))\n\t- 80% coverage of unixfs/io ([ipfs/go-ipfs#3097](https://github.com/ipfs/go-ipfs/pull/3097))\n\t- Accept more than one digit in repo version tests ([ipfs/go-ipfs#3130](https://github.com/ipfs/go-ipfs/pull/3130))\n\t- Fix typo in hash in t0050 ([ipfs/go-ipfs#3170](https://github.com/ipfs/go-ipfs/pull/3170))\n\t- fix bug in pinsets and add a stress test for the scenario ([ipfs/go-ipfs#3273](https://github.com/ipfs/go-ipfs/pull/3273)) ([ipfs/go-ipfs#3302](https://github.com/ipfs/go-ipfs/pull/3302))\n\t- Report coverage to codecov ([ipfs/go-ipfs#3473](https://github.com/ipfs/go-ipfs/pull/3473))\n\t- Add test for 'ipfs config replace' ([ipfs/go-ipfs#3073](https://github.com/ipfs/go-ipfs/pull/3073))\n\t- Fix netcat on macOS not closing socket when the stdin sends EOF ([ipfs/go-ipfs#3515](https://github.com/ipfs/go-ipfs/pull/3515))\n\n- Documentation\n\t- Update dns help with a correct domain name ([ipfs/go-ipfs#3087](https://github.com/ipfs/go-ipfs/pull/3087))\n\t- Add period to `ipfs pin rm` ([ipfs/go-ipfs#3088](https://github.com/ipfs/go-ipfs/pull/3088))\n\t- Make all Taglines use imperative mood ([ipfs/go-ipfs#3041](https://github.com/ipfs/go-ipfs/pull/3041))\n\t- Document listing commands better ([ipfs/go-ipfs#3083](https://github.com/ipfs/go-ipfs/pull/3083))\n\t- Add notes to readme on building for uncommon systems ([ipfs/go-ipfs#3051](https://github.com/ipfs/go-ipfs/pull/3051))\n\t- Add branch naming conventions doc ([ipfs/go-ipfs#3035](https://github.com/ipfs/go-ipfs/pull/3035))\n\t- Replace keyword with <> ([ipfs/go-ipfs#3129](https://github.com/ipfs/go-ipfs/pull/3129))\n\t- Fix Add() docs regarding pinning ([ipfs/go-ipfs#3513](https://github.com/ipfs/go-ipfs/pull/3513))\n\t- Add sudo to install commands. ([ipfs/go-ipfs#3201](https://github.com/ipfs/go-ipfs/pull/3201))\n\t- Add docs for `\"commands\".Command.Run` ([ipfs/go-ipfs#3382](https://github.com/ipfs/go-ipfs/pull/3382))\n\t- Put config keys in proper case ([ipfs/go-ipfs#3365](https://github.com/ipfs/go-ipfs/pull/3365))\n\t- Fix link in `ipfs stats bw` help message ([ipfs/go-ipfs#3620](https://github.com/ipfs/go-ipfs/pull/3620))\n\n## v0.4.4 - 2016-10-11\n\nThis release contains an important hotfix for a bug we discovered in how pinning works.\nIf you had a large number of pins, new pins would overwrite existing pins.\nApart from the hotfix, this release is equal to the previous release 0.4.3.\n\n- Fix bug in pinsets fanout, and add stress test. (@whyrusleeping, [ipfs/go-ipfs#3273](https://github.com/ipfs/go-ipfs/pull/3273))\n\nWe published a [detailed account of the bug and fix in a blog post](https://ipfs.io/blog/21-go-ipfs-0-4-4-released/).\n\n## v0.4.3 - 2016-09-20\n\nThere have been no changes since the last release candidate 0.4.3-rc4. \\o/\n\n## v0.4.3-rc4 - 2016-09-09\n\nThis release candidate fixes issues in Bitswap and the `ipfs add` command, and improves testing.\nWe plan for this to be the last release candidate before the release of go-ipfs v0.4.3.\n\nWith this release candidate, we're also moving go-ipfs to Go 1.7, which we expect will yield improvements in runtime performance, memory usage, build time and size of the release binaries.\n\n- Require Go 1.7. (@whyrusleeping, @Kubuxu, @lgierth, [ipfs/go-ipfs#3163](https://github.com/ipfs/go-ipfs/pull/3163))\n - For this purpose, switch Docker image from Alpine 3.4 to Alpine Edge.\n- Fix cancellation of Bitswap `wantlist` entries. (@whyrusleeping, [ipfs/go-ipfs#3182](https://github.com/ipfs/go-ipfs/pull/3182))\n- Fix clearing of `active` state of Bitswap provider queries. (@whyrusleeping, [ipfs/go-ipfs#3169](https://github.com/ipfs/go-ipfs/pull/3169))\n- Fix a panic in the DHT code. (@Kubuxu, [ipfs/go-ipfs#3200](https://github.com/ipfs/go-ipfs/pull/3200))\n- Improve handling of `Identity` field in `ipfs config` command. (@Kubuxu, @whyrusleeping, [ipfs/go-ipfs#3141](https://github.com/ipfs/go-ipfs/pull/3141))\n- Fix explicit adding of symlinked files and directories. (@kevina, [ipfs/go-ipfs#3135](https://github.com/ipfs/go-ipfs/pull/3135))\n- Fix bash auto-completion of `ipfs daemon --unrestricted-api` option. (@lgierth, [ipfs/go-ipfs#3159](https://github.com/ipfs/go-ipfs/pull/3159))\n- Introduce a new timeout tool for tests to avoid licensing issues. (@Kubuxu, [ipfs/go-ipfs#3152](https://github.com/ipfs/go-ipfs/pull/3152))\n- Improve output for migrations of fs-repo. (@lgierth, [ipfs/go-ipfs#3158](https://github.com/ipfs/go-ipfs/pull/3158))\n- Fix info notice of commands taking input from stdin. (@Kubuxu, [ipfs/go-ipfs#3134](https://github.com/ipfs/go-ipfs/pull/3134))\n- Bring back a few tests for stdin handling of `ipfs cat` and `ipfs add`. (@Kubuxu, [ipfs/go-ipfs#3144](https://github.com/ipfs/go-ipfs/pull/3144))\n- Improve sharness tests for `ipfs repo verify` command. (@whyrusleeping, [ipfs/go-ipfs#3148](https://github.com/ipfs/go-ipfs/pull/3148))\n- Improve sharness tests for CORS headers on the gateway. (@Kubuxu, [ipfs/go-ipfs#3142](https://github.com/ipfs/go-ipfs/pull/3142))\n- Improve tests for pinning within `ipfs files`. (@kevina, [ipfs/go-ipfs#3151](https://github.com/ipfs/go-ipfs/pull/3151))\n- Improve tests for the automatic raising of file descriptor limits. (@whyrusleeping, [ipfs/go-ipfs#3149](https://github.com/ipfs/go-ipfs/pull/3149))\n\n## v0.4.3-rc3 - 2016-08-11\n\nThis release candidate fixes a panic that occurs when input from stdin was\nexpected, but none was given: [ipfs/go-ipfs#3050](https://github.com/ipfs/go-ipfs/pull/3050)\n\n## v0.4.3-rc2 - 2016-08-04\n\nThis release includes bug fixes and fixes for regressions that were introduced\nbetween 0.4.2 and 0.4.3-rc1.\n\n- Regressions\n - Fix daemon panic when there is no multipart input provided over the HTTP API.\n (@whyrusleeping, [ipfs/go-ipfs#2989](https://github.com/ipfs/go-ipfs/pull/2989))\n - Fix `ipfs refs --edges` not printing edges.\n (@Kubuxu, [ipfs/go-ipfs#3007](https://github.com/ipfs/go-ipfs/pull/3007))\n - Fix progress option for `ipfs add` defaulting to true on the HTTP API.\n (@whyrusleeping, [ipfs/go-ipfs#3025](https://github.com/ipfs/go-ipfs/pull/3025))\n - Fix erroneous printing of stdin reading message.\n (@whyrusleeping, [ipfs/go-ipfs#3033](https://github.com/ipfs/go-ipfs/pull/3033))\n - Fix panic caused by passing `--mount` and `--offline` flags to `ipfs daemon`.\n (@Kubuxu, [ipfs/go-ipfs#3022](https://github.com/ipfs/go-ipfs/pull/3022))\n - Fix symlink path resolution on windows.\n (@Kubuxu, [ipfs/go-ipfs#3023](https://github.com/ipfs/go-ipfs/pull/3023))\n - Add in code to prevent issue 3032 from crashing the daemon.\n (@whyrusleeping, [ipfs/go-ipfs#3037](https://github.com/ipfs/go-ipfs/pull/3037))\n\n\n## v0.4.3-rc1 - 2016-07-23\n\nThis is a maintenance release which comes with a couple of nice enhancements, and improves the performance of Storage, Bitswap, as well as Content and Peer Routing. It also introduces a handful of new commands and options, and fixes a good bunch of bugs.\n\nThis is the first Release Candidate. Unless there are vulnerabilities or regressions discovered, the final 0.4.3 release will happen about one week from now.\n\n- Security Vulnerability\n\n - The `master` branch if go-ipfs suffered from a vulnerability for about 3 weeks. It allowed an attacker to use an iframe to request malicious HTML and JS from the API of a local go-ipfs node. The attacker could then gain unrestricted access to the node's API, and e.g. extract the private key. We fixed this issue by reintroducing restrictions on which particular objects can be loaded through the API (@lgierth, [ipfs/go-ipfs#2949](https://github.com/ipfs/go-ipfs/pull/2949)), and by completely excluding the private key from the API (@Kubuxu, [ipfs/go-ipfs#2957](https://github.com/ipfs/go-ipfs/pull/2957)). We will also work on more hardening of the API in the next release.\n - **The previous release 0.4.2 is not vulnerable. That means if you're using official binaries from [dist.ipfs.tech](https://dist.ipfs.tech) you're not affected.** If you're running go-ipfs built from the `master` branch between June 17th ([ipfs/go-ipfs@1afebc21](https://github.com/ipfs/go-ipfs/commit/1afebc21f324982141ca8a29710da0d6f83ca804)) and July 7th ([ipfs/go-ipfs@39bef0d5](https://github.com/ipfs/go-ipfs/commit/39bef0d5b01f70abf679fca2c4d078a2d55620e2)), please update to v0.4.3-rc1 immediately.\n - We are grateful to the group of independent researchers who made us aware of this vulnerability. We wanna use this opportunity to reiterate that we're very happy about any additional review of pull requests and releases. You can contact us any time at security@ipfs.io (GPG [4B9665FB 92636D17 7C7A86D3 50AAE8A9 59B13AF3](https://pgp.mit.edu/pks/lookup?op=get&search=0x50AAE8A959B13AF3)).\n\n- Notable changes\n\n - Improve Bitswap performance. (@whyrusleeping, [ipfs/go-ipfs#2727](https://github.com/ipfs/go-ipfs/pull/2727), [ipfs/go-ipfs#2798](https://github.com/ipfs/go-ipfs/pull/2798))\n - Improve Content Routing and Peer Routing performance. (@whyrusleeping, [ipfs/go-ipfs#2817](https://github.com/ipfs/go-ipfs/pull/2817), [ipfs/go-ipfs#2841](https://github.com/ipfs/go-ipfs/pull/2841))\n - Improve datastore, blockstore, and dagstore performance. (@kevina, @Kubuxu, @whyrusleeping [ipfs/go-datastore#43](https://github.com/ipfs/go-datastore/pull/43), [ipfs/go-ipfs#2885](https://github.com/ipfs/go-ipfs/pull/2885), [ipfs/go-ipfs#2961](https://github.com/ipfs/go-ipfs/pull/2961), [ipfs/go-ipfs#2953](https://github.com/ipfs/go-ipfs/pull/2953), [ipfs/go-ipfs#2960](https://github.com/ipfs/go-ipfs/pull/2960))\n - Content Providers are now stored on disk to gain savings on process memory. (@whyrusleeping, [ipfs/go-ipfs#2804](https://github.com/ipfs/go-ipfs/pull/2804), [ipfs/go-ipfs#2860](https://github.com/ipfs/go-ipfs/pull/2860))\n - Migrations of the fs-repo (usually stored at `~/.ipfs`) now run automatically. If there's a TTY available, you'll get prompted when running `ipfs daemon`, and in addition you can use the `--migrate=true` or `--migrate=false` options to avoid the prompt. (@whyrusleeping, @lgierth, [ipfs/go-ipfs#2939](https://github.com/ipfs/go-ipfs/pull/2939))\n - The internal naming of blocks in the blockstore has changed, which requires a migration of the fs-repo, from version 3 to 4. (@whyrusleeping, [ipfs/go-ipfs#2903](https://github.com/ipfs/go-ipfs/pull/2903))\n - We now automatically raise the file descriptor limit to 1024 if necessary. (@whyrusleeping, [ipfs/go-ipfs#2884](https://github.com/ipfs/go-ipfs/pull/2884), [ipfs/go-ipfs#2891](https://github.com/ipfs/go-ipfs/pull/2891))\n - After a long struggle with deadlocks and hanging connections, we've decided to disable the uTP transport by default for now. (@whyrusleeping, [ipfs/go-ipfs#2840](https://github.com/ipfs/go-ipfs/pull/2840), [ipfs/go-libp2p-transport@88244000](https://github.com/ipfs/go-libp2p-transport/commit/88244000f0ce8851ffcfbac746ebc0794b71d2a4))\n - There is now documentation for the configuration options in `docs/config.md`. (@whyrusleeping, [ipfs/go-ipfs#2974](https://github.com/ipfs/go-ipfs/pull/2974))\n - All commands now sanely handle the combination of stdin and optional flags in certain edge cases. (@lgierth, [ipfs/go-ipfs#2952](https://github.com/ipfs/go-ipfs/pull/2952))\n\n- New Features\n\n - Add `--offline` option to `ipfs daemon` command, which disables all swarm networking. (@Kubuxu, [ipfs/go-ipfs#2696](https://github.com/ipfs/go-ipfs/pull/2696), [ipfs/go-ipfs#2867](https://github.com/ipfs/go-ipfs/pull/2867))\n - Add `Datastore.HashOnRead` option for verifying block hashes on read access. (@Kubuxu, [ipfs/go-ipfs#2904](https://github.com/ipfs/go-ipfs/pull/2904))\n - Add `Datastore.BloomFilterSize` option for tuning the blockstore's new lookup bloom filter. (@Kubuxu, [ipfs/go-ipfs#2973](https://github.com/ipfs/go-ipfs/pull/2973))\n\n- Bug fixes\n\n - Fix publishing of local IPNS entries, and more. (@whyrusleeping, [ipfs/go-ipfs#2943](https://github.com/ipfs/go-ipfs/pull/2943))\n - Fix progress bars in `ipfs add` and `ipfs get`. (@whyrusleeping, [ipfs/go-ipfs#2893](https://github.com/ipfs/go-ipfs/pull/2893), [ipfs/go-ipfs#2948](https://github.com/ipfs/go-ipfs/pull/2948))\n - Make sure files added through `ipfs files` are pinned and don't get GC'd. (@kevina, [ipfs/go-ipfs#2872](https://github.com/ipfs/go-ipfs/pull/2872))\n - Fix copying into directory using `ipfs files cp`. (@whyrusleeping, [ipfs/go-ipfs#2977](https://github.com/ipfs/go-ipfs/pull/2977))\n - Fix `ipfs version --commit` with Docker containers. (@lgierth, [ipfs/go-ipfs#2734](https://github.com/ipfs/go-ipfs/pull/2734))\n - Run `ipfs diag` commands in the daemon instead of the CLI. (@Kubuxu, [ipfs/go-ipfs#2761](https://github.com/ipfs/go-ipfs/pull/2761))\n - Fix protobuf encoding on the API and in commands. (@stebalien, [ipfs/go-ipfs#2516](https://github.com/ipfs/go-ipfs/pull/2516))\n - Fix goroutine leak in `/ipfs/ping` protocol handler. (@whyrusleeping, [ipfs/go-libp2p#58](https://github.com/ipfs/go-libp2p/pull/58))\n - Fix `--flags` option on `ipfs commands`. (@Kubuxu, [ipfs/go-ipfs#2773](https://github.com/ipfs/go-ipfs/pull/2773))\n - Fix the error channels in `namesys`. (@whyrusleeping, [ipfs/go-ipfs#2788](https://github.com/ipfs/go-ipfs/pull/2788))\n - Fix consumptions of observed swarm addresses. (@whyrusleeping, [ipfs/go-libp2p#63](https://github.com/ipfs/go-libp2p/pull/63), [ipfs/go-ipfs#2771](https://github.com/ipfs/go-ipfs/issues/2771))\n - Fix a rare DHT panic. (@whyrusleeping, [ipfs/go-ipfs#2856](https://github.com/ipfs/go-ipfs/pull/2856))\n - Fix go-ipfs/js-ipfs interoperability issues in SPDY. (@whyrusleeping, [whyrusleeping/go-smux-spdystream@fae17783](https://github.com/whyrusleeping/go-smux-spdystream/commit/fae1778302a9e029bb308cf71cf33f857f2d89e8))\n - Fix a logging race condition during shutdown. (@Kubuxu, [ipfs/go-log#3](https://github.com/ipfs/go-log/pull/3))\n - Prevent DHT connection hangs. (@whyrusleeping, [ipfs/go-ipfs#2826](https://github.com/ipfs/go-ipfs/pull/2826), [ipfs/go-ipfs#2863](https://github.com/ipfs/go-ipfs/pull/2863))\n - Fix NDJSON output of `ipfs refs local`. (@Kubuxu, [ipfs/go-ipfs#2812](https://github.com/ipfs/go-ipfs/pull/2812))\n - Fix race condition in NAT detection. (@whyrusleeping, [ipfs/go-libp2p#69](https://github.com/ipfs/go-libp2p/pull/69))\n - Fix error messages. (@whyrusleeping, @Kubuxu, [ipfs/go-ipfs#2905](https://github.com/ipfs/go-ipfs/pull/2905), [ipfs/go-ipfs#2928](https://github.com/ipfs/go-ipfs/pull/2928))\n\n- Enhancements\n\n - Increase maximum object size on `ipfs put` from 1 MiB to 2 MiB. The maximum object size on the wire including all framing is 4 MiB. (@kpcyrd, [ipfs/go-ipfs#2980](https://github.com/ipfs/go-ipfs/pull/2980))\n - Add CORS headers to the Gateway's default config. (@Kubuxu, [ipfs/go-ipfs#2778](https://github.com/ipfs/go-ipfs/pull/2778))\n - Clear the dial backoff for a peer when using `ipfs swarm connect`. (@whyrusleeping, [ipfs/go-ipfs#2941](https://github.com/ipfs/go-ipfs/pull/2941))\n - Allow passing options to daemon in Docker container. (@lgierth, [ipfs/go-ipfs#2955](https://github.com/ipfs/go-ipfs/pull/2955))\n - Add `-v/--verbose` to `ìpfs swarm peers` command. (@csasarak, [ipfs/go-ipfs#2713](https://github.com/ipfs/go-ipfs/pull/2713))\n - Add `--format`, `--hash`, and `--size` options to `ipfs files stat` command. (@Kubuxu, [ipfs/go-ipfs#2706](https://github.com/ipfs/go-ipfs/pull/2706))\n - Add `--all` option to `ipfs version` command. (@Kubuxu, [ipfs/go-ipfs#2790](https://github.com/ipfs/go-ipfs/pull/2790))\n - Add `ipfs repo version` command. (@pfista, [ipfs/go-ipfs#2598](https://github.com/ipfs/go-ipfs/pull/2598))\n - Add `ipfs repo verify` command. (@whyrusleeping, [ipfs/go-ipfs#2924](https://github.com/ipfs/go-ipfs/pull/2924), [ipfs/go-ipfs#2951](https://github.com/ipfs/go-ipfs/pull/2951))\n - Add `ipfs stats repo` and `ipfs stats bitswap` command aliases. (@pfista, [ipfs/go-ipfs#2810](https://github.com/ipfs/go-ipfs/pull/2810))\n - Add success indication to responses of `ipfs ping` command. (@Kubuxu, [ipfs/go-ipfs#2813](https://github.com/ipfs/go-ipfs/pull/2813))\n - Save changes made via `ipfs swarm filter` to the config file. (@yuvallanger, [ipfs/go-ipfs#2880](https://github.com/ipfs/go-ipfs/pull/2880))\n - Expand `ipfs_p2p_peers` metric to include libp2p transport. (@lgierth, [ipfs/go-ipfs#2728](https://github.com/ipfs/go-ipfs/pull/2728))\n - Rework `ipfs files add` internals to avoid caching and prevent memory leaks. (@whyrusleeping, [ipfs/go-ipfs#2795](https://github.com/ipfs/go-ipfs/pull/2795))\n - Support `GOPATH` with multiple path components. (@karalabe, @lgierth, @djdv, [ipfs/go-ipfs#2808](https://github.com/ipfs/go-ipfs/pull/2808), [ipfs/go-ipfs#2862](https://github.com/ipfs/go-ipfs/pull/2862), [ipfs/go-ipfs#2975](https://github.com/ipfs/go-ipfs/pull/2975))\n\n- General Codebase\n\n - Take steps towards the `filestore` datastore. (@kevina, [ipfs/go-ipfs#2792](https://github.com/ipfs/go-ipfs/pull/2792), [ipfs/go-ipfs#2634](https://github.com/ipfs/go-ipfs/pull/2634))\n - Update recommended Golang version to 1.6.2 (@Kubuxu, [ipfs/go-ipfs#2724](https://github.com/ipfs/go-ipfs/pull/2724))\n - Update to Gx 0.8.0 and Gx-Go 1.2.1, which is faster and less noisy. (@whyrusleeping, [ipfs/go-ipfs#2979](https://github.com/ipfs/go-ipfs/pull/2979))\n - Use `go4.org/lock` instead of `camlistore/lock` for locking. (@whyrusleeping, [ipfs/go-ipfs#2887](https://github.com/ipfs/go-ipfs/pull/2887))\n - Manage `go.uuid`, `hamming`, `backoff`, `proquint`, `pb`, `go-context`, `cors`, `go-datastore` packages with Gx. (@Kubuxu, [ipfs/go-ipfs#2733](https://github.com/ipfs/go-ipfs/pull/2733), [ipfs/go-ipfs#2736](https://github.com/ipfs/go-ipfs/pull/2736), [ipfs/go-ipfs#2757](https://github.com/ipfs/go-ipfs/pull/2757), [ipfs/go-ipfs#2825](https://github.com/ipfs/go-ipfs/pull/2825), [ipfs/go-ipfs#2838](https://github.com/ipfs/go-ipfs/pull/2838))\n - Clean up the gateway's surface. (@lgierth, [ipfs/go-ipfs#2874](https://github.com/ipfs/go-ipfs/pull/2874))\n - Simplify the API gateway's access restrictions. (@lgierth, [ipfs/go-ipfs#2949](https://github.com/ipfs/go-ipfs/pull/2949), [ipfs/go-ipfs#2956](https://github.com/ipfs/go-ipfs/pull/2956))\n - Update docker image to Alpine Linux 3.4 and remove Go version constraint. (@lgierth, [ipfs/go-ipfs#2901](https://github.com/ipfs/go-ipfs/pull/2901), [ipfs/go-ipfs#2929](https://github.com/ipfs/go-ipfs/pull/2929))\n - Clarify `Dockerfile` and `Dockerfile.fast`. (@lgierth, [ipfs/go-ipfs#2796](https://github.com/ipfs/go-ipfs/pull/2796))\n - Simplify resolution of Git commit refs in Dockerfiles. (@lgierth, [ipfs/go-ipfs#2754](https://github.com/ipfs/go-ipfs/pull/2754))\n - Consolidate `--verbose` description across commands. (@Kubuxu, [ipfs/go-ipfs#2746](https://github.com/ipfs/go-ipfs/pull/2746))\n - Allow setting position of default values in command option descriptions. (@Kubuxu, [ipfs/go-ipfs#2744](https://github.com/ipfs/go-ipfs/pull/2744))\n - Set explicit default values for boolean command options. (@RichardLitt, [ipfs/go-ipfs#2657](https://github.com/ipfs/go-ipfs/pull/2657))\n - Autogenerate command synopses. (@Kubuxu, [ipfs/go-ipfs#2785](https://github.com/ipfs/go-ipfs/pull/2785))\n - Fix and improve lots of documentation. (@RichardLitt, [ipfs/go-ipfs#2741](https://github.com/ipfs/go-ipfs/pull/2741), [ipfs/go-ipfs#2781](https://github.com/ipfs/go-ipfs/pull/2781))\n - Improve command descriptions to fit a width of 78 characters. (@RichardLitt, [ipfs/go-ipfs#2779](https://github.com/ipfs/go-ipfs/pull/2779), [ipfs/go-ipfs#2780](https://github.com/ipfs/go-ipfs/pull/2780), [ipfs/go-ipfs#2782](https://github.com/ipfs/go-ipfs/pull/2782))\n - Fix filename conflict in the debugging guide. (@Kubuxu, [ipfs/go-ipfs#2752](https://github.com/ipfs/go-ipfs/pull/2752))\n - Decapitalize log messages, according to Golang style guides. (@RichardLitt, [ipfs/go-ipfs#2853](https://github.com/ipfs/go-ipfs/pull/2853))\n - Add GitHub Issues HowTo guide. (@RichardLitt, @chriscool, [ipfs/go-ipfs#2889](https://github.com/ipfs/go-ipfs/pull/2889), [ipfs/go-ipfs#2895](https://github.com/ipfs/go-ipfs/pull/2895))\n - Add GitHub Issue template. (@chriscool, [ipfs/go-ipfs#2786](https://github.com/ipfs/go-ipfs/pull/2786))\n - Apply standard-readme to the README file. (@RichardLitt, [ipfs/go-ipfs#2883](https://github.com/ipfs/go-ipfs/pull/2883))\n - Fix issues pointed out by `govet`. (@Kubuxu, [ipfs/go-ipfs#2854](https://github.com/ipfs/go-ipfs/pull/2854))\n - Clarify `ipfs get` error message. (@whyrusleeping, [ipfs/go-ipfs#2886](https://github.com/ipfs/go-ipfs/pull/2886))\n - Remove dead code. (@whyrusleeping, [ipfs/go-ipfs#2819](https://github.com/ipfs/go-ipfs/pull/2819))\n - Add changelog for v0.4.3. (@lgierth, [ipfs/go-ipfs#2984](https://github.com/ipfs/go-ipfs/pull/2984))\n\n- Tests & CI\n\n - Fix flaky `ipfs mount` sharness test by using the `iptb` tool. (@noffle, [ipfs/go-ipfs#2707](https://github.com/ipfs/go-ipfs/pull/2707))\n - Fix flaky IP port selection in tests. (@Kubuxu, [ipfs/go-ipfs#2855](https://github.com/ipfs/go-ipfs/pull/2855))\n - Fix CLI tests on OSX by resolving /tmp symlink. (@Kubuxu, [ipfs/go-ipfs#2926](https://github.com/ipfs/go-ipfs/pull/2926))\n - Fix flaky GC test by running the daemon in offline mode. (@Kubuxu, [ipfs/go-ipfs#2908](https://github.com/ipfs/go-ipfs/pull/2908))\n - Add tests for `ipfs add` with hidden files. (@Kubuxu, [ipfs/go-ipfs#2756](https://github.com/ipfs/go-ipfs/pull/2756))\n - Add test to make sure the body of HEAD responses is empty. (@Kubuxu, [ipfs/go-ipfs#2775](https://github.com/ipfs/go-ipfs/pull/2775))\n - Add test to catch misdials. (@Kubuxu, [ipfs/go-ipfs#2831](https://github.com/ipfs/go-ipfs/pull/2831))\n - Mark flaky tests for `ipfs dht query` as known failure. (@noffle, [ipfs/go-ipfs#2720](https://github.com/ipfs/go-ipfs/pull/2720))\n - Remove failing blockstore-without-context test. (@Kubuxu, [ipfs/go-ipfs#2857](https://github.com/ipfs/go-ipfs/pull/2857))\n - Fix `--version` tests for versions with a suffix like `-dev` or `-rc1`. (@lgierth, [ipfs/go-ipfs#2937](https://github.com/ipfs/go-ipfs/pull/2937))\n - Make sharness tests work in cases where go-ipfs is symlinked into GOPATH. (@lgierth, [ipfs/go-ipfs#2937](https://github.com/ipfs/go-ipfs/pull/2937))\n - Add variable delays to blockstore mocks. (@rikonor, [ipfs/go-ipfs#2871](https://github.com/ipfs/go-ipfs/pull/2871))\n - Disable Travis CI email notifications. (@Kubuxu, [ipfs/go-ipfs#2896](https://github.com/ipfs/go-ipfs/pull/2896))\n\n\n## v0.4.2 - 2016-05-17\n\nThis is a patch release which fixes performance and networking bugs in go-libp2p,\nYou should see improvements in CPU and RAM usage, as well as speed of object lookups.\nThere are also a few other nice improvements.\n\n* Notable Fixes\n * Set a deadline for dialing attempts. This prevents a node from accumulating\n failed connections. (@whyrusleeping)\n * Avoid unnecessary string/byte conversions in go-multihash. (@whyrusleeping)\n * Fix a deadlock around the yamux stream muxer. (@whyrusleeping)\n * Fix a bug that left channels open, causing hangs. (@whyrusleeping)\n * Fix a bug around yamux which caused connection hangs. (@whyrusleeping)\n * Fix a crash caused by nil multiaddrs. (@whyrusleeping)\n\n* Enhancements\n * Add NetBSD support. (@erde74)\n * Set Cache-Control: immutable on /ipfs responses. (@kpcyrd)\n * Have `ipfs init` optionally accept a default configuration from stdin. (@sivachandran)\n * Add `ipfs log ls` command for listing logging subsystems. (@hsanjuan)\n * Allow bitswap to read multiple messages per stream. (@whyrusleeping)\n * Remove `make toolkit_upgrade` step. (@chriscool)\n\n* Documentation\n * Add a debug-guidelines document. (@richardlitt)\n * Update the contribute document. (@richardlitt)\n * Fix documentation of many `ipfs` commands. (@richardlitt)\n * Fall back to ShortDesc if LongDesc is missing. (@Kubuxu)\n\n* Removals\n * Remove -f option from `ipfs init` command. (@whyrusleeping)\n\n* Bug fixes\n * Fix `ipfs object patch` argument handling and validation. (@jbenet)\n * Fix `ipfs config edit` command by running it client-side. (@Kubuxu)\n * Set default value for `ipfs refs` arguments. (@richardlitt)\n * Fix parsing of incorrect command and argument permutations. (@thomas-gardner)\n * Update Dockerfile to latest go1.5.4-r0. (@chriscool)\n * Allow passing IPFS_LOGGING to Docker image. (@lgierth)\n * Fix dot path parsing on Windows. (@djdv)\n * Fix formatting of `ipfs log ls` output. (@richardlitt)\n\n* General Codebase\n * Refactor Makefile. (@kevina)\n * Wire context into bitswap requests more deeply. (@whyrusleeping)\n * Use gx for iptb. (@chriscool)\n * Update gx and gx-go. (@chriscool)\n * Make blocks.Block an interface. (@kevina)\n * Silence check for Docker existence. (@chriscool)\n * Add dist_get script for fetching tools from dist.ipfs.tech. (@whyrusleeping)\n * Add proper defaults to all `ipfs` commands. (@richardlitt)\n * Remove dead `count` option from `ipfs pin ls`. (@richardlitt)\n * Initialize pin mode strings only once. (@chriscool)\n * Add changelog for v0.4.2. (@lgierth)\n * Specify a dist.ipfs.tech hash for tool downloads instead of trusting DNS. (@lgierth)\n\n* CI\n * Fix t0170-dht sharness test. (@chriscool)\n * Increase timeout in t0060-daemon sharness test. (@Kubuxu)\n * Have CircleCI use `make deps` instead of `gx` directly. (@whyrusleeping)\n\n\n## v0.4.1 - 2016-04-25\n\nThis is a patch release that fixes a few bugs, and adds a few small (but not\ninsignificant) features. The primary reason for this release is the listener\nhang bugfix that was shipped in the 0.4.0 release.\n\n* Features\n * implemented ipfs object diff (@whyrusleeping)\n * allow promises (used in get, refs) to fail (@whyrusleeping)\n\n* Tool changes\n * Adds 'toolkit_upgrade' to the makefile help target (@achin)\n\n* General Codebase\n * Use extracted go-libp2p-crypto, -secio, -peer packages (@lgierth)\n * Update go-libp2p (@lgierth)\n * Fix package manifest fields (@lgierth)\n * remove incfusever dead-code (@whyrusleeping)\n * remove a ton of unused godeps (@whyrusleeping)\n * metrics: add Prometheus back (@lgierth)\n * clean up dead code and config fields (@whyrusleeping)\n * Add log events when blocks are added/removed from the blockstore (@michealmure)\n * repo: don't create logs directory, not used any longer (@lgierth)\n\n* Bug fixes\n * fixed ipfs name resolve --local multihash error (@pfista)\n * ipfs patch commands won't return null links field anymore (@whyrusleeping)\n * Make non recursive resolve print the result (@Kubuxu)\n * Output dirs on ipfs add -rn (@noffle)\n * update libp2p dep to fix hanging listeners problem (@whyrusleeping)\n * Fix Swarm.AddrFilters config setting with regard to `/ip6` addresses (@lgierth)\n * fix dht command key escaping (@whyrusleeping)\n\n* Testing\n * Adds tests to make sure 'object patch' writes. (@noffle)\n * small sharness test for promise failure checking (@whyrusleeping)\n * sharness/Makefile: clean all BINS when cleaning (@chriscool)\n\n* Documentation\n * Fix disconnect argument description (@richardlitt)\n * Added a note about swarm disconnect (@richardlitt)\n * Also fixed syntax for comment (@richardlitt)\n * Alphabetized swarm subcmds (@richardlitt)\n * Added note to ipfs stats bw interval option (@richardlitt)\n * Small syntax changes to repo stat man (@richardlitt)\n * update log command help text (@pfista)\n * Added a long description to add (@richardlitt)\n * Edited object patch set-data doc (@richardlitt)\n * add roadmap.md (@Jeromy)\n * Adds files api cmd to helptext (@noffle)\n\n\n## v0.4.0 - 2016-04-05\n\nThis is a major release with plenty of new features and bug fixes.\nIt also includes breaking changes which make it incompatible with v0.3.x\non the networking layer.\n\n* Major Changes\n * Multistream\n * The addition of multistream is a breaking change on the networking layer,\n but gives IPFS implementations the ability to mix and match different\n stream multiplexers, e.g. yamux, spdystream, or muxado.\n This adds a ton of flexibility on one of the lower layers of the protocol,\n and will help us avoid further breaking protocol changes in the future.\n * Files API\n * The new `files` command and API allow a program to interact with IPFS\n using familiar filesystem operations, namely: creating directories,\n reading, writing, and deleting files, listing out different directories,\n and so on. This feature enables any other application that uses a\n filesystem-like backend for storage, to use IPFS as its storage driver\n without having change the application logic at all.\n * Gx\n * go-ipfs now uses [gx](https://github.com/whyrusleeping/gx) to manage its\n dependencies. This means that under the hood, go-ipfs's dependencies are\n backed by IPFS itself! It also means that go-ipfs is no longer installed\n using `go get`. Use `make install` instead.\n* New Features\n * Web UI\n * Update to new version which is compatible with 0.4.0. (@dignifiedquire)\n * Networking\n * Implement uTP transport. (@whyrusleeping)\n * Allow multiple addresses per configured bootstrap node. (@whyrusleeping)\n * IPNS\n * Improve IPNS resolution performance. (@whyrusleeping)\n * Have dnslink prefer `TXT _dnslink.example.com`, allows usage of CNAME records. (@Kubuxu)\n * Prevent `ipfs name publish` when `/ipns` is mounted. (@noffle)\n * Repo\n * Improve performance of `ipfs add`. (@whyrusleeping)\n * Add `Datastore.NoSync` config option for flatfs. (@rht)\n * Implement mark-and-sweep GC. (@whyrusleeping)\n * Allow for GC during `ipfs add`. (@whyrusleeping)\n * Add `ipfs repo stat` command. (@tmg, @diasdavid)\n * General\n * Add support for HTTP OPTIONS requests. (@lidel)\n * Add `ipfs diag cmds` to view active API requests (@whyrusleeping)\n * Add an `IPFS_LOW_MEM` environment variable which relaxes Bitswap's memory usage. (@whyrusleeping)\n * The Docker image now lives at `ipfs/go-ipfs` and has been completely reworked. (@lgierth)\n* Security fixes\n * The gateway path prefix added in v0.3.10 was vulnerable to cross-site\n scripting attacks. This release introduces a configurable list of allowed\n path prefixes. It's called `Gateway.PathPrefixes` and takes a list of\n strings, e.g. `[\"/blog\", \"/foo/bar\"]`. The v0.3.x line will not receive any\n further updates, so please update to v0.4.0 as soon as possible. (@lgierth)\n* Incompatible Changes\n * Install using `make install` instead of `go get` (@whyrusleeping)\n * Rewrite pinning to store pins in IPFS objects. (@tv42)\n * Bump fs-repo version to 3. (@whyrusleeping)\n * Use multistream muxer (@whyrusleeping)\n * The default for `--type` in `ipfs pin ls` is now `all`. (@chriscool)\n* Bug Fixes\n * Remove msgio double wrap. (@jbenet)\n * Buffer msgio. (@whyrusleeping)\n * Perform various fixes to the FUSE code. (@tv42)\n * Compute `ipfs add` size in background to not stall add operation. (@whyrusleeping)\n * Add option to have `ipfs add` include top-level hidden files. (@noffle)\n * Fix CORS checks on the API. (@rht)\n * Fix `ipfs update` error message. (@tomgg)\n * Resolve paths in `ipfs pin rm` without network lookup. (@noffle)\n * Detect FUSE unmounts and track mount state. (@noffle)\n * Fix go1.6rc2 panic caused by CloseNotify being called from wrong goroutine. (@rwcarlsen)\n * Bump DHT kvalue from 10 to 20. (@whyrusleeping)\n * Put public key and IPNS entry to DHT in parallel. (@whyrusleeping)\n * Fix panic in CLI argument parsing. (@whyrusleeping)\n * Fix range error by using larger-than-zero-length buffer. (@noffle)\n * Fix yamux hanging issue by increasing AcceptBacklog. (@whyrusleeping)\n * Fix double Transport-Encoding header bug. (@whyrusleeping)\n * Fix uTP panic and file descriptor leak. (@whyrusleeping)\n* Tool Changes\n * Add `--pin` option to `ipfs add`, which defaults to `true` and allows `--pin=false`. (@eminence)\n * Add arguments to `ipfs pin ls`. (@chriscool)\n * Add `dns` and `resolve` commands to read-only API. (@Kubuxu)\n * Add option to display headers for `ipfs object links`. (@palkeo)\n* General Codebase Changes\n * Check Golang version in Makefile. (@chriscool)\n * Improve Makefile. (@tomgg)\n * Remove dead Jenkins CI code. (@lgierth)\n * Add locking interface to blockstore. (@whyrusleeping)\n * Add Merkledag FetchGraph and EnumerateChildren. (@whyrusleeping)\n * Rename Lock/RLock to GCLock/PinLock. (@jbenet)\n * Implement pluggable datastore types. (@tv42)\n * Record datastore metrics for non-default datastores. (@tv42)\n * Allow multistream to have zero-rtt stream opening. (@whyrusleeping)\n * Refactor `ipnsfs` into a more generic and well tested `mfs`. (@whyrusleeping)\n * Grab more peers if bucket doesn't contain enough. (@whyrusleeping)\n * Use CloseNotify in gateway. (@whyrusleeping)\n * Flatten multipart file transfers. (@whyrusleeping)\n * Send updated DHT record fixes to peers who sent outdated records. (@whyrusleeping)\n * Replace go-psutil with go-sysinfo. (@whyrusleeping)\n * Use ServeContent for index.html. (@AtnNn)\n * Refactor `object patch` API to not store data in URL. (@whyrusleeping)\n * Use mfs for `ipfs add`. (@whyrusleeping)\n * Add `Server` header to API responses. (@Kubuxu)\n * Wire context directly into HTTP requests. (@rht)\n * Wire context directly into GetDAG operations within GC. (@rht)\n * Vendor libp2p using gx. (@whyrusleeping)\n * Use gx vendored packages instead of Godeps. (@whyrusleeping)\n * Simplify merkledag package interface to ease IPLD inclusion. (@mildred)\n * Add default option value support to commands lib. (@whyrusleeping)\n * Refactor merkledag fetching methods. (@whyrusleeping)\n * Use net/url to escape paths within Web UI. (@noffle)\n * Deprecated key.Pretty(). (@MichealMure)\n* Documentation\n * Fix and update help text for **every** `ipfs` command. (@RichardLitt)\n * Change sample API origin settings from wildcard (`*`) to `example.com`. (@Kubuxu)\n * Improve documentation of installation process in README. (@whyrusleeping)\n * Improve windows.md. (@chriscool)\n * Clarify instructions for installing from source. (@noffle)\n * Make version checking more robust. (@jedahan)\n * Assert the source code is located within GOPATH. (@whyrusleeping)\n * Remove mentions of `/dns` from `ipfs dns` command docs. (@lgierth)\n* Testing\n * Refactor iptb tests. (@chriscool)\n * Improve t0240 sharness test. (@chriscool)\n * Make bitswap tests less flaky. (@whyrusleeping)\n * Use TCP port zero for ipfs daemon in sharness tests. (@whyrusleeping)\n * Improve sharness tests on AppVeyor. (@chriscool)\n * Add a pause to fix timing on t0065. (@whyrusleeping)\n * Add support for arbitrary TCP ports to t0060-daemon.sh. (@noffle)\n * Make t0060 sharness test use TCP port zero. (@whyrusleeping)\n * Randomized ipfs stress testing via randor (@dignifiedquire)\n * Stress test pinning and migrations (@whyrusleeping)\n" }, { "path": "docs/changelogs/v0.40.md", "content": "# Kubo changelog v0.40\n\n\n\nThis release was brought to you by the [Shipyard](https://ipshipyard.com/) team.\n\n- [v0.40.0](#v0400)\n- [v0.40.1](#v0401)\n\n## v0.40.0\n\n[](https://github.com/user-attachments/assets/c065d5e5-2a8a-4651-8142-d7baf3106623)\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [🔢 IPIP-499: UnixFS CID Profiles](#-ipip-499-unixfs-cid-profiles)\n - [🧹 Automatic cleanup of interrupted imports](#-automatic-cleanup-of-interrupted-imports)\n - [🌍 Light clients can now use your node for delegated routing](#-light-clients-can-now-use-your-node-for-delegated-routing)\n - [📊 See total size when pinning](#-see-total-size-when-pinning)\n - [🔀 IPIP-523: `?format=` takes precedence over `Accept` header](#-ipip-523-format-takes-precedence-over-accept-header)\n - [🚫 IPIP-524: Gateway codec conversion disabled by default](#-ipip-524-gateway-codec-conversion-disabled-by-default)\n - [✅ More reliable IPNS over PubSub](#-more-reliable-ipns-over-pubsub)\n - [🗄️ New `ipfs diag datastore` commands](#️-new-ipfs-diag-datastore-commands)\n - [🔍 New `ipfs swarm addrs autonat` command](#-new-ipfs-swarm-addrs-autonat-command)\n - [🚇 Improved `ipfs p2p` tunnels with foreground mode](#-improved-ipfs-p2p-tunnels-with-foreground-mode)\n - [📊 Friendlier `ipfs dag stat` output](#-friendlier-ipfs-dag-stat-output)\n - [🔑 `ipfs key` improvements](#-ipfs-key-improvements)\n - [🤝 More reliable content providing after startup](#-more-reliable-content-providing-after-startup)\n - [🌐 No unnecessary DNS lookups for AutoTLS addresses](#-no-unnecessary-dns-lookups-for-autotls-addresses)\n - [⏱️ Configurable gateway request duration limit](#️-configurable-gateway-request-duration-limit)\n - [🔧 Recovery from corrupted MFS root](#-recovery-from-corrupted-mfs-root)\n - [📡 RPC `Content-Type` headers for binary responses](#-rpc-content-type-headers-for-binary-responses)\n - [🔖 New `ipfs name get|put` commands](#-new-ipfs-name-getput-commands)\n - [📋 Long listing format for `ipfs ls`](#-long-listing-format-for-ipfs-ls)\n - [🖥️ WebUI Improvements](#-webui-improvements)\n - [📉 Fixed Prometheus metrics bloat on popular subdomain gateways](#-fixed-prometheus-metrics-bloat-on-popular-subdomain-gateways)\n - [📢 libp2p announces all interface addresses](#-libp2p-announces-all-interface-addresses)\n - [🗑️ Badger v1 datastore slated for removal this year](#-badger-v1-datastore-slated-for-removal-this-year)\n - [🐹 Go 1.26](#-go-126)\n - [📦️ Dependency updates](#-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\nThis release brings reproducible file imports (CID Profiles), automatic cleanup of interrupted operations, better connectivity diagnostics, and improved gateway behavior. It also ships with Go 1.26, lowering memory usage and GC overhead across the board.\n\n### 🔦 Highlights\n\n#### 🔢 IPIP-499: UnixFS CID Profiles\n\nCID Profiles are presets that pin down how files get split into blocks and organized into directories, so you get the same CID for the same data across different software or versions. Defined in [IPIP-499](https://specs.ipfs.tech/ipips/ipip-0499/).\n\n**New configuration [profiles](https://github.com/ipfs/kubo/blob/master/docs/config.md#profiles)**\n\n- `unixfs-v1-2025`: modern CIDv1 profile with improved defaults\n- `unixfs-v0-2015` (alias `legacy-cid-v0`): best-effort legacy CIDv0 behavior\n\nApply with: `ipfs config profile apply unixfs-v1-2025`\n\nThe `test-cid-v1` and `test-cid-v1-wide` profiles have been removed. Use `unixfs-v1-2025` or manually set specific `Import.*` settings instead.\n\n**New [`Import.*`](https://github.com/ipfs/kubo/blob/master/docs/config.md#import) options**\n\n- `Import.UnixFSHAMTDirectorySizeEstimation`: estimation mode (`links`, `block`, or `disabled`)\n- `Import.UnixFSDAGLayout`: DAG layout (`balanced` or `trickle`)\n\n**New [`ipfs add`](https://docs.ipfs.tech/reference/kubo/cli/#ipfs-add) CLI flags**\n\n- `--dereference-symlinks` resolves all symlinks to their target content, replacing the deprecated `--dereference-args` which only resolved CLI argument symlinks\n- `--empty-dirs` / `-E` controls inclusion of empty directories (default: true)\n- `--hidden` / `-H` includes hidden files (default: false)\n- `--trickle` implicit default can be adjusted via `Import.UnixFSDAGLayout`\n\n**`ipfs files write` fix for CIDv1 directories**\n\nWhen writing to MFS directories that use CIDv1 (via `--cid-version=1` or `ipfs files chcid`), single-block files now produce raw block CIDs (like `bafkrei...`), matching the behavior of `ipfs add --raw-leaves`. Previously, MFS would wrap single-block files in dag-pb even when raw leaves were enabled. CIDv0 directories continue to use dag-pb.\n\n**Block size limit raised to 2MiB**\n\n`ipfs block put`, `ipfs dag put`, and `ipfs dag import` now accept blocks up to 2MiB without `--allow-big-block`, matching the [bitswap spec](https://specs.ipfs.tech/bitswap-protocol/#block-sizes). The previous 1MiB limit was too restrictive and broke `ipfs dag import` of 1MiB-chunked non-raw-leaf data (protobuf wrapping pushes blocks slightly over 1MiB). The max `--chunker` value for `ipfs add` is `2MiB - 256 bytes` to leave room for protobuf framing. IPIP-499 profiles use lower chunk sizes (256KiB and 1MiB) and are not affected.\n\n**HAMT Threshold Fix**\n\nHAMT directory sharding threshold changed from `>=` to `>` to match the Go docs and JS implementation ([ipfs/boxo@6707376](https://github.com/ipfs/boxo/commit/6707376002a3d4ba64895749ce9be2e00d265ed5)). A directory exactly at 256 KiB now stays as a basic directory instead of converting to HAMT. This is a theoretical breaking change, but unlikely to impact real-world users as it requires a directory to be exactly at the threshold boundary. If you depend on the old behavior, adjust [`Import.UnixFSHAMTShardingSize`](https://github.com/ipfs/kubo/blob/master/docs/config.md#importunixfshamtshardingsize) to be 1 byte lower.\n\n#### 🧹 Automatic cleanup of interrupted imports\n\nIf you cancel `ipfs add` or `ipfs dag import` mid-operation, Kubo now automatically cleans up incomplete data on the next daemon start. Previously, interrupted imports would leave orphan blocks in your repository that were difficult to identify and remove without pins and running explicit garbage collection.\n\nBatch operations also use less memory now. Block data is written to disk immediately rather than held in RAM until the batch commits.\n\nUnder the hood, the block storage layer (flatfs) was rewritten to use atomic batch operations via a temporary staging directory. See [go-ds-flatfs#142](https://github.com/ipfs/go-ds-flatfs/pull/142) for details.\n\n#### 🌍 Light clients can now use your node for delegated routing\n\nThe [Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) is now exposed by default at `http://127.0.0.1:8080/routing/v1`. This allows light clients in browsers to use Kubo Gateway as a delegated routing backend instead of running a full DHT client. Support for [IPIP-476: Delegated Routing DHT Closest Peers API](https://specs.ipfs.tech/ipips/ipip-0476/) is included. Can be disabled via [`Gateway.ExposeRoutingAPI`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewayexposeroutingapi).\n\n#### 📊 See total size when pinning\n\n`ipfs pin add --progress` now shows the total size of the pinned DAG as it fetches blocks.\n\nExample output:\n\n```\nFetched/Processed 336 nodes (83 MB)\n```\n\n#### 🔀 IPIP-523: `?format=` takes precedence over `Accept` header\n\nThe `?format=` URL query parameter now always wins over the `Accept` header ([IPIP-523](https://specs.ipfs.tech/ipips/ipip-0523/)), giving you deterministic HTTP caching and protecting against CDN cache-key collisions. Browsers can also use `?format=` reliably even when they send `Accept` headers with specific content types.\n\nThe only breaking change is for edge cases where a client sends both a specific `Accept` header and a different `?format=` value for an explicitly supported format (`tar`, `raw`, `car`, `dag-json`, `dag-cbor`, etc.). Previously `Accept` would win. Now `?format=` always wins.\n\n#### 🚫 IPIP-524: Gateway codec conversion disabled by default\n\nGateways no longer convert between codecs by default ([IPIP-524](https://specs.ipfs.tech/ipips/ipip-0524/)). This removes gateways from a gatekeeping role: clients can adopt new codecs immediately without waiting for gateway operator updates. Requests for a format that differs from the block's codec now return `406 Not Acceptable`.\n\n**Migration**: Clients should fetch raw blocks (`?format=raw` or `Accept: application/vnd.ipld.raw`)\nand convert client-side using libraries like [@helia/verified-fetch](https://www.npmjs.com/package/@helia/verified-fetch).\n\nSet [`Gateway.AllowCodecConversion`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewayallowcodecconversion)\nto `true` to restore previous behavior.\n\n#### ✅ More reliable IPNS over PubSub\n\n[IPNS over PubSub](https://specs.ipfs.tech/ipns/ipns-pubsub-router/) implementation in Kubo is now more reliable. Duplicate messages are rejected even in large networks where messages may cycle back after the in-memory cache expires.\n\nKubo now persists the maximum seen sequence number per peer to the datastore ([go-libp2p-pubsub#BasicSeqnoValidator](https://pkg.go.dev/github.com/libp2p/go-libp2p-pubsub#BasicSeqnoValidator)), providing stronger duplicate detection that survives node restarts. This addresses message flooding issues reported in [#9665](https://github.com/ipfs/kubo/issues/9665).\n\nIPNS over PubSub is opt-in via [`Ipns.UsePubsub`](https://github.com/ipfs/kubo/blob/master/docs/config.md#ipnsusepubsub). Kubo's pubsub is optimized for IPNS use case. For custom pubsub applications requiring different validation logic, use [go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub) directly in a dedicated binary.\n\n#### 🗄️ New `ipfs diag datastore` commands\n\nNew experimental commands for low-level datastore inspection:\n\n- `ipfs diag datastore get ` - Read raw value at a datastore key (use `--hex` for hex dump)\n- `ipfs diag datastore count ` - Count entries matching a datastore prefix\n\nThe daemon must not be running when using these commands. Run `ipfs diag datastore --help` for usage examples.\n\n#### 🔍 New `ipfs swarm addrs autonat` command\n\nThe new `ipfs swarm addrs autonat` command shows the network reachability status of your node's addresses as verified by AutoNAT V2. AutoNAT V2 leverages other nodes in the IPFS network to test your node's external public reachability, providing a self-service way to debug connectivity.\n\nPublic reachability is important for:\n\n- **Direct data fetching**: Other nodes can fetch data directly from your node without NAT hole punching.\n- **Browser access**: Web browsers can connect to your node directly for content retrieval.\n- **DHT participation**: Your node can act as a DHT server, helping to maintain the distributed hash table and making content routing more robust.\n\nThe command displays:\n\n- Overall reachability status (public, private, or unknown)\n- Per-address reachability showing which specific addresses are reachable, unreachable, or unknown\n\nExample output:\n```\nAutoNAT V2 Status:\n Reachability: public\n\nPer-Address Reachability:\n Reachable:\n /ip4/203.0.113.42/tcp/4001\n /ip4/203.0.113.42/udp/4001/quic-v1\n Unreachable:\n /ip6/2001:db8::1/tcp/4001\n Unknown:\n /ip4/203.0.113.42/udp/4001/webrtc-direct\n```\n\nThis helps diagnose connectivity issues and understand if your node is publicly reachable. See the [AutoNAT V2 spec](https://github.com/libp2p/specs/blob/master/autonat/autonat-v2.md) for more details.\n\n#### 🚇 Improved `ipfs p2p` tunnels with foreground mode\n\nP2P tunnels can now run like SSH port forwarding: start a tunnel, use it, and it cleans up automatically when you're done.\n\nThe new `--foreground` (`-f`) flag for `ipfs p2p listen` and `ipfs p2p forward` keeps the command running until interrupted. When you Ctrl+C, send SIGTERM, or stop the service, the tunnel is removed automatically:\n\n```console\n$ ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 --foreground\nListening on /x/ssh, forwarding to /ip4/127.0.0.1/tcp/22, waiting for interrupt...\n^C\nReceived interrupt, removing listener for /x/ssh\n```\n\nWithout `--foreground`, commands return immediately and tunnels persist until explicitly closed (existing behavior).\n\nSee [docs/p2p-tunnels.md](https://github.com/ipfs/kubo/blob/master/docs/p2p-tunnels.md) for usage examples.\n\n#### 📊 Friendlier `ipfs dag stat` output\n\nThe `ipfs dag stat` command has been improved for better terminal UX:\n\n- Progress output now uses a single line with carriage return, avoiding terminal flooding\n- Progress is auto-detected: shown only in interactive terminals by default\n- Human-readable sizes are now displayed alongside raw byte counts\n\nExample progress (interactive terminal):\n```\nFetched/Processed 84 blocks, 2097152 bytes (2.1 MB)\n```\n\nExample summary output:\n```\nSummary\nTotal Size: 2097152 (2.1 MB)\nUnique Blocks: 42\nShared Size: 1048576 (1.0 MB)\nRatio: 1.500000\n```\n\nUse `--progress=true` to force progress even when piped, or `--progress=false` to disable it.\n\n#### 🔑 `ipfs key` improvements\n\n`ipfs key ls` is now the canonical command for listing keys, matching `ipfs pin ls` and `ipfs files ls`. The old `ipfs key list` still works but is deprecated.\n\nListing also became more resilient: bad keys are now skipped with an error log instead of failing the entire operation.\n\n#### 🤝 More reliable content providing after startup\n\nPreviously, provide operations could start before the Accelerated DHT Client discovered enough peers, causing sweep mode to lose its efficiency benefits. Now, providing waits for the initial network crawl (about 10 minutes). Your content will be properly distributed across DHT regions after initial DHT map is created. Check `ipfs provide stat` to see when providing begins.\n\n#### 🌐 No unnecessary DNS lookups for AutoTLS addresses\n\nKubo no longer makes DNS queries for [AutoTLS](https://web.archive.org/web/20260112031855/https://blog.libp2p.io/autotls/) addresses like `1-2-3-4.peerid.libp2p.direct`. Since the IP is encoded in the hostname (`1-2-3-4` means `1.2.3.4`), Kubo extracts it locally. This reduces load on the public good DNS servers at `libp2p.direct` run by [Shipyard](https://ipshipyard.com), reserving them for web browsers which lack direct DNS access and must rely on the browser's resolver.\n\nTo disable, set [`AutoTLS.SkipDNSLookup`](https://github.com/ipfs/kubo/blob/master/docs/config.md#autotlsskipdnslookup) to `false`.\n\n#### ⏱️ Configurable gateway request duration limit\n\n[`Gateway.MaxRequestDuration`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaymaxrequestduration) sets an absolute deadline for gateway requests. Unlike `RetrievalTimeout` (which resets on each data write and catches stalled transfers), this is a hard limit on the total time a request can take.\n\nThe default 1 hour limit (previously hardcoded) can now be adjusted to fit your deployment needs. This is a fallback that prevents requests from hanging indefinitely when subsystem timeouts are misconfigured or fail to trigger. Returns 504 Gateway Timeout when exceeded.\n\n#### 🔧 Recovery from corrupted MFS root\n\nIf your daemon fails to start because the MFS root is not a directory (due to misconfiguration, operational error, or disk corruption), you can now recover without deleting and recreating your repository in a new `IPFS_PATH`.\n\nThe new `ipfs files chroot` command lets you reset the MFS (Mutable File System) root or restore it to a known valid CID:\n\n```console\n# Reset MFS to an empty directory\n$ ipfs files chroot --confirm\n\n# Or restore from a previously saved directory CID\n$ ipfs files chroot --confirm QmYourBackupCID\n```\n\nSee `ipfs files chroot --help` for details.\n\n#### 📡 RPC `Content-Type` headers for binary responses\n\nHTTP RPC endpoints that return binary data now set appropriate `Content-Type` headers, making it easier to integrate with HTTP clients and tooling that rely on MIME types. On CLI these commands behave the same as before, but over HTTP RPC you now get proper headers:\n\n| Endpoint | Content-Type |\n|------------------------|-------------------------------------------|\n| `/api/v0/get` | `application/x-tar` or `application/gzip` |\n| `/api/v0/dag/export` | `application/vnd.ipld.car` |\n| `/api/v0/block/get` | `application/vnd.ipld.raw` |\n| `/api/v0/name/get` | `application/vnd.ipfs.ipns-record` |\n| `/api/v0/diag/profile` | `application/zip` |\n\n#### 🔖 New `ipfs name get|put` commands\n\nYou can now backup, restore, and share IPNS records without needing the private key.\n\n```console\n$ ipfs name get /ipns/k51... > record.bin\n$ ipfs name get /ipns/k51... | ipfs name inspect\n$ ipfs name put k51... record.bin\n```\n\nThese are low-level tools primarily for debugging and testing IPNS.\n\nThe `put` command validates records by default. Use `--force` to skip validation and test how routing systems handle malformed or outdated records. Note that `--force` only bypasses this command's checks; the routing system may still reject invalid records.\n\n#### 📋 Long listing format for `ipfs ls`\n\nThe `ipfs ls` command now supports `--long` (`-l`) flag for displaying Unix-style file permissions and modification times. This works with files added using `--preserve-mode` and `--preserve-mtime`. See `ipfs ls --help` for format details and examples.\n\n#### 🖥️ WebUI Improvements\n\nIPFS Web UI has been updated to [v4.11.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.11.0).\n\n##### Search and filter files\n\nYou can now search and filter files directly in the Files screen. Type a name, CID, or file extension and the list narrows down in real time. Works in both list and grid view.\n\n> ![Search and filter files](https://github.com/user-attachments/assets/cc266dbc-8424-4a8a-a6b7-a80a5a25683b)\n\n##### DHT Provide diagnostic screen\n\nNew screen under Diagnostics that shows the health of DHT Provide operations. You can see reprovide cycle progress, worker utilization, queue status, and network throughput at a glance, without having to use the [`ipfs provide stat`](https://docs.ipfs.tech/reference/kubo/cli/#ipfs-provide-stat) CLI.\n\n> ![DHT Provide diagnostic screen](https://github.com/user-attachments/assets/c577309a-2249-46f8-87d9-f0da42955f32)\n\n##### Better path handling in Files\n\nThe Inspect button now resolves `/ipfs/` and `/ipns/` paths to their final CID before opening the IPLD Explorer. The Explore form also accepts `ipfs://` and `ipns://` protocol URLs. Previously, these would show a blank screen or an infinite spinner. Path resolution errors now also show better error pages:\n\n> ![Better path handling in Files](https://github.com/user-attachments/assets/3494835b-0b93-4990-9971-078273671928)\n\n#### 📉 Fixed Prometheus metrics bloat on popular subdomain gateways\n\nMost Kubo users are unaffected by this change. It matters if you run Kubo as a public subdomain gateway (with [`Gateway.PublicGateways`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaypublicgateways) and `UseSubdomains: true`), where the `otelhttp` instrumentation was including the raw `Host` header as the `server_address` metric label. Every unique hostname (e.g., each `CID.ipfs.dweb.link`) created a separate time series, resulting in millions of metric lines, multi-gigabyte `/debug/metrics/prometheus` responses, and Prometheus scrape timeouts.\n\n**What changed:**\n\n- `http_server_*` metrics replace the unbounded `server_address` label with a new `server_domain` label that groups requests by gateway domain:\n - Gateway: matched [`Gateway.PublicGateways`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaypublicgateways) suffix (e.g., `dweb.link`, `ipfs.io`), or `localhost`, `loopback`, `other`\n - RPC API: `api` / Libp2p Gateway: `libp2p`\n- Prometheus exemplars are disabled to prevent log noise from long subdomain hostnames. Tracing spans are unaffected.\n\nIf you use [Rainbow](https://github.com/ipfs/rainbow) for your public gateway (recommended), this issue never applied to you -- Rainbow uses its own low-cardinality HTTP metrics.\n\n#### 📢 libp2p announces all interface addresses\n\ngo-libp2p [v0.47.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.47.0) includes a rewritten routing library ([`go-netroute`](https://github.com/libp2p/go-netroute/pull/64)) that fixes interop with VPN and WireGuard/Tailscale setups. A side effect: when listening on `0.0.0.0`, libp2p now returns addresses from all network interfaces instead of just the primary one ([go-libp2p#3460](https://github.com/libp2p/go-libp2p/issues/3460)).\n\nThis means easier connectivity and less manual configuration for most desktop, VPN, and self-hosted users. However, if you don't run with the [`server` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#server-profile) and have an empty [`Addresses.NoAnnounce`](https://github.com/ipfs/kubo/blob/master/docs/config.md#addressesnoannounce), your node may now announce internal addresses (e.g. Docker bridge `172.17.0.0/16` or Tailscale `100.64.0.0/10`) to the DHT. In the default setup, AutoNAT will probe and mark unreachable ones as offline and they won't be listed, but you can also filter them out explicitly.\n\nTo check what your node announces and filter out unwanted ranges:\n\n```console\n$ ipfs swarm addrs local\n$ ipfs config --json Addresses.NoAnnounce '[\"/ip4/172.17.0.0/ipcidr/16\"]'\n```\n\nThe [`server` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#server-profile) already [filters common private ranges](https://github.com/ipfs/kubo/blob/master/config/profile.go#L24-L43) via `Addresses.NoAnnounce`.\n\n#### 🗑️ Badger v1 datastore slated for removal this year\n\nThe `badgerds` datastore (based on badger 1.x) is slated for removal. Badger v1 has not been maintained by its upstream maintainers for years and has known bugs including startup timeouts, shutdown hangs, and file descriptor exhaustion. Starting with this release, every daemon start with a badger-based repository prints a loud deprecation error on stderr.\n\nSee the [`badgerds` profile documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#badgerds-profile) for migration guidance, and [#11186](https://github.com/ipfs/kubo/issues/11186) for background.\n\n#### 🐹 Go 1.26\n\nThis release is built with [Go 1.26](https://go.dev/doc/go1.26).\n\nYou should see lower memory usage and reduced GC pauses thanks to the new Green Tea garbage collector (10-40% less GC overhead). Reading block data and API responses is faster due to `io.ReadAll` improvements (~2x faster, ~50% less memory). On 64-bit platforms, heap base address randomization adds a layer of security hardening.\n\n> **Note:** [v0.40.1](#v0401) downgrades to Go 1.25 due to a Windows stability issue. If you run Kubo on Linux or macOS, staying on v0.40.0 is fine and you benefit from Go 1.26's GC improvements.\n\n#### 📦️ Dependency updates\n\n- update `go-libp2p` to [v0.47.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.47.0) (incl. [v0.46.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.46.0))\n - Reduced WebRTC log noise by using debug level for pion errors ([go-libp2p#3426](https://github.com/libp2p/go-libp2p/pull/3426)).\n - Fixed mDNS discovery on Windows and macOS by filtering addresses to reduce packet size ([go-libp2p#3434](https://github.com/libp2p/go-libp2p/pull/3434)).\n - AutoTLS addresses no longer get marked unreachable when peers lack WebSockets support. Swarm heals over time ([go-libp2p#3435](https://github.com/libp2p/go-libp2p/pull/3435)).\n - Fixed `stream.Close()` blocking indefinitely on unresponsive peers ([go-libp2p#3448](https://github.com/libp2p/go-libp2p/pull/3448)).\n- update `quic-go` to [v0.59.0](https://github.com/quic-go/quic-go/releases/tag/v0.59.0) (incl. [v0.58.0](https://github.com/quic-go/quic-go/releases/tag/v0.58.0) + [v0.57.0](https://github.com/quic-go/quic-go/releases/tag/v0.57.0))\n- update `p2p-forge` to [v0.7.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.7.0)\n- update `go-ds-pebble` to [v0.5.9](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.9)\n - updates `github.com/cockroachdb/pebble` to [v2.1.4](https://github.com/cockroachdb/pebble/releases/tag/v2.1.4) to enable Go 1.26 support\n- update `go-libp2p-pubsub` to [v0.15.0](https://github.com/libp2p/go-libp2p-pubsub/releases/tag/v0.15.0)\n- update `go-ipld-prime` to [v0.22.0](https://github.com/ipld/go-ipld-prime/releases/tag/v0.22.0)\n- update `boxo` to [v0.37.0](https://github.com/ipfs/boxo/releases/tag/v0.37.0) (incl. [v0.36.0](https://github.com/ipfs/boxo/releases/tag/v0.36.0))\n- update `go-libp2p-kad-dht` to [v0.38.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.38.0) (includes [v0.37.1](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.37.1), [v0.37.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.37.0))\n- update `ipfs-webui` to [v4.11.1](https://github.com/ipfs/ipfs-webui/releases/tag/v4.11.1) (incl. [v4.11.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.11.0))\n- update `gateway-conformance` tests to [v0.10](https://github.com/ipfs/gateway-conformance/releases/tag/v0.10.0) (incl. [v0.9](https://github.com/ipfs/gateway-conformance/releases/tag/v0.9.0))\n\n### 📝 Changelog\n\n
Full Changelog\n\n- github.com/ipfs/kubo:\n - fix(metrics): disable otel exemplars to prevent rune overflow (#11211) ([ipfs/kubo#11211](https://github.com/ipfs/kubo/pull/11211))\n - fix: drop high-cardinality server.address from http_server metrics (#11208) ([ipfs/kubo#11208](https://github.com/ipfs/kubo/pull/11208))\n - chore: set version to 0.40.0-rc2\n - fix(version): produce shorter user agent for tagged release builds\n - chore: update webui to v4.11.1 (#11204) ([ipfs/kubo#11204](https://github.com/ipfs/kubo/pull/11204))\n - fix: improve `ipfs name put` for IPNS record republishing (#11199) ([ipfs/kubo#11199](https://github.com/ipfs/kubo/pull/11199))\n - Upgrade to Boxo v0.37.0 (#11201) ([ipfs/kubo#11201](https://github.com/ipfs/kubo/pull/11201))\n - chore: set version to v0.40.0-rc1\n - refactor: apply go fix modernizers from Go 1.26 (#11190) ([ipfs/kubo#11190](https://github.com/ipfs/kubo/pull/11190))\n - feat: update to Go 1.26 (#11189) ([ipfs/kubo#11189](https://github.com/ipfs/kubo/pull/11189))\n - docs: clarify LevelDB compaction limitations and StorageMax scope (#11188) ([ipfs/kubo#11188](https://github.com/ipfs/kubo/pull/11188))\n - docs: loud deprecation of badger v1 datastore (#11187) ([ipfs/kubo#11187](https://github.com/ipfs/kubo/pull/11187))\n - docs(changelog): add highlight for libp2p AllAddrs behavior change (#11183) ([ipfs/kubo#11183](https://github.com/ipfs/kubo/pull/11183))\n - fix: allow dag import of 1MiB chunks wrapped in dag-pb (#11185) ([ipfs/kubo#11185](https://github.com/ipfs/kubo/pull/11185))\n - feat: `swarm addrs autonat` command (#11184) ([ipfs/kubo#11184](https://github.com/ipfs/kubo/pull/11184))\n - feat(gateway): IPIP-0524 Gateway.AllowCodecConversion config option (#11090) ([ipfs/kubo#11090](https://github.com/ipfs/kubo/pull/11090))\n - feat: update ipfs-webui to v4.11.0 (#11182) ([ipfs/kubo#11182](https://github.com/ipfs/kubo/pull/11182))\n - feat(config): add Import.* for CID Profiles from IPIP-499 (#11148) ([ipfs/kubo#11148](https://github.com/ipfs/kubo/pull/11148))\n - chore: replace libp2p.io URL with Internet Archive (#11181) ([ipfs/kubo#11181](https://github.com/ipfs/kubo/pull/11181))\n - test: IPIP-523 format query precedence over Accept header (#11086) ([ipfs/kubo#11086](https://github.com/ipfs/kubo/pull/11086))\n - docs: update go-libp2p changelog entry to v0.47.0\n - feat(rpc): Content-Type headers and IPNS record get/put (#11067) ([ipfs/kubo#11067](https://github.com/ipfs/kubo/pull/11067))\n - feat(key): add 'ipfs key ls' as alias for 'ipfs key list' (#11147) ([ipfs/kubo#11147](https://github.com/ipfs/kubo/pull/11147))\n - docs: cleanup broken links and outdated content (#11100) ([ipfs/kubo#11100](https://github.com/ipfs/kubo/pull/11100))\n - feat(dns): skip DNS lookups for AutoTLS hostnames (#11140) ([ipfs/kubo#11140](https://github.com/ipfs/kubo/pull/11140))\n - Upgrade to Boxo v0.36.0 (#11175) ([ipfs/kubo#11175](https://github.com/ipfs/kubo/pull/11175))\n - chore: upgrade go-ds-pebble to v0.5.9 (#11170) ([ipfs/kubo#11170](https://github.com/ipfs/kubo/pull/11170))\n - fix(commands/reprovide): update manual reprovide error message (#11151) ([ipfs/kubo#11151](https://github.com/ipfs/kubo/pull/11151))\n - feat(cli): ls --long (#11103) ([ipfs/kubo#11103](https://github.com/ipfs/kubo/pull/11103))\n - feat(pubsub): persistent validation and diagnostic commands (#11110) ([ipfs/kubo#11110](https://github.com/ipfs/kubo/pull/11110))\n - feat(config): add Gateway.MaxRequestDuration option (#11138) ([ipfs/kubo#11138](https://github.com/ipfs/kubo/pull/11138))\n - feat(provider): info log AcceleratedDHTClient crawl (#11143) ([ipfs/kubo#11143](https://github.com/ipfs/kubo/pull/11143))\n - ipfswatch: fix panic on broken link (#11145) ([ipfs/kubo#11145](https://github.com/ipfs/kubo/pull/11145))\n - upgrade go-libp2p-pubsub to v0.15.0 (#11144) ([ipfs/kubo#11144](https://github.com/ipfs/kubo/pull/11144))\n - feat(mfs): chroot command to change the root (#8648) ([ipfs/kubo#8648](https://github.com/ipfs/kubo/pull/8648))\n - feat: improved go-ds-flatfs (#11092) ([ipfs/kubo#11092](https://github.com/ipfs/kubo/pull/11092))\n - test: fix flaky ipfswatch test (#11142) ([ipfs/kubo#11142](https://github.com/ipfs/kubo/pull/11142))\n - docs: clarify Routing.Type=custom as experimental (#11111) ([ipfs/kubo#11111](https://github.com/ipfs/kubo/pull/11111))\n - fix(routing): defensive clone of AddrInfo from provider channel (#11120) ([ipfs/kubo#11120](https://github.com/ipfs/kubo/pull/11120))\n - fix(provider): wait for fullrt crawl completion before providing (#11137) ([ipfs/kubo#11137](https://github.com/ipfs/kubo/pull/11137))\n - fix(provide): do not output keystore error on shutdown (#11130) ([ipfs/kubo#11130](https://github.com/ipfs/kubo/pull/11130))\n - feat(p2p): add --foreground flag to listen and forward commands (#11099) ([ipfs/kubo#11099](https://github.com/ipfs/kubo/pull/11099))\n - feat(cli): improve ipfs dag stat output UX (#11097) ([ipfs/kubo#11097](https://github.com/ipfs/kubo/pull/11097))\n - docs: add production deployment guidance for gateway (#11117) ([ipfs/kubo#11117](https://github.com/ipfs/kubo/pull/11117))\n - fix(routing): use LegacyProvider for HTTP-only custom routing (#11112) ([ipfs/kubo#11112](https://github.com/ipfs/kubo/pull/11112))\n - shutdown daemon after test (#11135) ([ipfs/kubo#11135](https://github.com/ipfs/kubo/pull/11135))\n - fix(ci): parallelize gotest, cleanup output, flakiness (#11113) ([ipfs/kubo#11113](https://github.com/ipfs/kubo/pull/11113))\n - test: replace `go-clock` with `testing/synctest` (#11131) ([ipfs/kubo#11131](https://github.com/ipfs/kubo/pull/11131))\n - docs: improve README for first-time users (#11133) ([ipfs/kubo#11133](https://github.com/ipfs/kubo/pull/11133))\n - keys: skip bad keys when listing (#11115) ([ipfs/kubo#11115](https://github.com/ipfs/kubo/pull/11115))\n - docs: add developer guide for local development workflow (#11128) ([ipfs/kubo#11128](https://github.com/ipfs/kubo/pull/11128))\n - datastore: upgrade go-ds-pebble to v0.5.8 (#11129) ([ipfs/kubo#11129](https://github.com/ipfs/kubo/pull/11129))\n - output stdout and stderr on example test failure (#11119) ([ipfs/kubo#11119](https://github.com/ipfs/kubo/pull/11119))\n - chore: update go-libp2p 0.46 (#11105) ([ipfs/kubo#11105](https://github.com/ipfs/kubo/pull/11105))\n - fix(ipfswatch): loading datastore plugins (#11078) ([ipfs/kubo#11078](https://github.com/ipfs/kubo/pull/11078))\n - Add bytes progress tracker for ipfs pin add (#11074) ([ipfs/kubo#11074](https://github.com/ipfs/kubo/pull/11074))\n - docs: link sweep blogpost in Provide.DHT.SweepEnabled\n - docs: note sweep+accelerated DHT client limitation (#11084) ([ipfs/kubo#11084](https://github.com/ipfs/kubo/pull/11084))\n - refactor: replace context.WithCancel with t.Context (#11083) ([ipfs/kubo#11083](https://github.com/ipfs/kubo/pull/11083))\n - chore: remove deprecated go-ipfs Docker image publishing (#11081) ([ipfs/kubo#11081](https://github.com/ipfs/kubo/pull/11081))\n - Merge release v0.39.0 ([ipfs/kubo#11080](https://github.com/ipfs/kubo/pull/11080))\n - docs: move IPIP-476 feature to v0.40 changelog\n - upgrade go-libp2p-kad-dht to v0.36.0 (#11079) ([ipfs/kubo#11079](https://github.com/ipfs/kubo/pull/11079))\n - fix(docker): include symlinks in scanning for init scripts (#11077) ([ipfs/kubo#11077](https://github.com/ipfs/kubo/pull/11077))\n - Update deprecation message for Reprovider fields (#11072) ([ipfs/kubo#11072](https://github.com/ipfs/kubo/pull/11072))\n - fix doc string (#11068) ([ipfs/kubo#11068](https://github.com/ipfs/kubo/pull/11068))\n - feat: support GetClosesPeers (IPIP-476) and ExposeRoutingAPI by default (#10954) ([ipfs/kubo#10954](https://github.com/ipfs/kubo/pull/10954))\n - chore: start v0.40.0 release cycle\n- github.com/gammazero/chanqueue (v1.1.1 -> v1.1.2):\n - require go1.24 or later (#9) ([gammazero/chanqueue#9](https://github.com/gammazero/chanqueue/pull/9))\n - update workflow (#7) ([gammazero/chanqueue#7](https://github.com/gammazero/chanqueue/pull/7))\n - prefer range loops (#6) ([gammazero/chanqueue#6](https://github.com/gammazero/chanqueue/pull/6))\n- github.com/gammazero/deque (v1.2.0 -> v1.2.1):\n - fix panic after IterPopX leaves buffer exactly full (#51) ([gammazero/deque#51](https://github.com/gammazero/deque/pull/51))\n - fix panic if copying in exactly the buffer size (#50) ([gammazero/deque#50](https://github.com/gammazero/deque/pull/50))\n - refactor: prefer range loops (#49) ([gammazero/deque#49](https://github.com/gammazero/deque/pull/49))\n- github.com/ipfs/boxo (v0.35.2 -> v0.37.0):\n - Release v0.37.0 ([ipfs/boxo#1109](https://github.com/ipfs/boxo/pull/1109))\n - update dependencies (#1107) ([ipfs/boxo#1107](https://github.com/ipfs/boxo/pull/1107))\n - refactor: modernize code (#1105) ([ipfs/boxo#1105](https://github.com/ipfs/boxo/pull/1105))\n - ensure http response body is closed (#1103) ([ipfs/boxo#1103](https://github.com/ipfs/boxo/pull/1103))\n - update multiaddr dns and otel (#1102) ([ipfs/boxo#1102](https://github.com/ipfs/boxo/pull/1102))\n - fix: raise block size limits from 1MiB to 2MiB (#1101) ([ipfs/boxo#1101](https://github.com/ipfs/boxo/pull/1101))\n - test(gateway): add dag-pb to dag-json codec conversion tests\n - feat(gateway): IPIP-0524 + AllowCodecConversion config option (#1077) ([ipfs/boxo#1077](https://github.com/ipfs/boxo/pull/1077))\n - feat(unixfs): configurable CID Profiles from IPIP-499 (#1088) ([ipfs/boxo#1088](https://github.com/ipfs/boxo/pull/1088))\n - feat(gateway): IPIP-523 format query over Accept header (#1074) ([ipfs/boxo#1074](https://github.com/ipfs/boxo/pull/1074))\n - Release v0.36.0 ([ipfs/boxo#1099](https://github.com/ipfs/boxo/pull/1099))\n - upgrade go-libp2p-kad-dht to v0.37.1 (#1097) ([ipfs/boxo#1097](https://github.com/ipfs/boxo/pull/1097))\n - fix(routing): defensive nil checks for multiaddr handling (#1081) ([ipfs/boxo#1081](https://github.com/ipfs/boxo/pull/1081))\n - fix: flaky TestSessionBetweenPeers with shuffle enabled (#1022) ([ipfs/boxo#1022](https://github.com/ipfs/boxo/pull/1022))\n - upgrade to go-libp2p v0.47.0 (#1095) ([ipfs/boxo#1095](https://github.com/ipfs/boxo/pull/1095))\n - update dependencies (#1091) ([ipfs/boxo#1091](https://github.com/ipfs/boxo/pull/1091))\n - refactor: rewrite some flaky tests to testing/synctest (#1087) ([ipfs/boxo#1087](https://github.com/ipfs/boxo/pull/1087))\n - fix(routing): fix unknown record bytes unmarshalling (#1090) ([ipfs/boxo#1090](https://github.com/ipfs/boxo/pull/1090))\n - refactor: replace `go-clock` with `synctest` (#1082) ([ipfs/boxo#1082](https://github.com/ipfs/boxo/pull/1082))\n - chore: fix gofumpt formatting in dagreader.go\n - cosmetic fixes (#1086) ([ipfs/boxo#1086](https://github.com/ipfs/boxo/pull/1086))\n - feat(gateway): configurable fallback timeout for MaxRequestDuration (#1079) ([ipfs/boxo#1079](https://github.com/ipfs/boxo/pull/1079))\n - fix(bitswap/network): `stream.Close()` blocks indefinitely on unresponsive peers (#1083) ([ipfs/boxo#1083](https://github.com/ipfs/boxo/pull/1083))\n - test: cleanup goroutines at end of test (#1084) ([ipfs/boxo#1084](https://github.com/ipfs/boxo/pull/1084))\n - keystore: improve error messages and include key file name (#1080) ([ipfs/boxo#1080](https://github.com/ipfs/boxo/pull/1080))\n - docs: update deprecation comments to reference IPIP-526 (#1076) ([ipfs/boxo#1076](https://github.com/ipfs/boxo/pull/1076))\n - feat(ipld/merkledag): add total size of visited nodes in progress tracker ([ipfs/boxo#1071](https://github.com/ipfs/boxo/pull/1071))\n - upgrade to go-libp2p-kad-dht v0.36.0 ([ipfs/boxo#1072](https://github.com/ipfs/boxo/pull/1072))\n - routing/http: add support for GetClosestPeers (IPIP-476) (#1021) ([ipfs/boxo#1021](https://github.com/ipfs/boxo/pull/1021))\n - tar: fix name filter on windows ([ipfs/boxo#1047](https://github.com/ipfs/boxo/pull/1047))\n- github.com/ipfs/go-cidutil (v0.1.0 -> v0.1.1):\n - new version (#55) ([ipfs/go-cidutil#55](https://github.com/ipfs/go-cidutil/pull/55))\n - update dependencies (#53) ([ipfs/go-cidutil#53](https://github.com/ipfs/go-cidutil/pull/53))\n - ci: uci/copy-templates ([ipfs/go-cidutil#43](https://github.com/ipfs/go-cidutil/pull/43))\n - ci: uci/copy-templates ([ipfs/go-cidutil#42](https://github.com/ipfs/go-cidutil/pull/42))\n- github.com/ipfs/go-datastore (v0.9.0 -> v0.9.1):\n - new version (#266) ([ipfs/go-datastore#266](https://github.com/ipfs/go-datastore/pull/266))\n - update opentelemetry to v1.40.0 (#265) ([ipfs/go-datastore#265](https://github.com/ipfs/go-datastore/pull/265))\n - refactor: modernize code (#264) ([ipfs/go-datastore#264](https://github.com/ipfs/go-datastore/pull/264))\n - test suite: use a non-cancelled context to delete all keys (#259) ([ipfs/go-datastore#259](https://github.com/ipfs/go-datastore/pull/259))\n - Document not to reuse batch (#258) ([ipfs/go-datastore#258](https://github.com/ipfs/go-datastore/pull/258))\n - Revert \"Test that a second batch commit does not error (#256)\" (#257) ([ipfs/go-datastore#257](https://github.com/ipfs/go-datastore/pull/257))\n - Test that a second batch commit does not error (#256) ([ipfs/go-datastore#256](https://github.com/ipfs/go-datastore/pull/256))\n- github.com/ipfs/go-ds-flatfs (v0.5.5 -> v0.6.0):\n - new version (#144) ([ipfs/go-ds-flatfs#144](https://github.com/ipfs/go-ds-flatfs/pull/144))\n - refactor: rewrite batch mode to use temp directory (#142) ([ipfs/go-ds-flatfs#142](https://github.com/ipfs/go-ds-flatfs/pull/142))\n - Clarify the usage of RLock and why RUnlock is missing when applying ops (#141) ([ipfs/go-ds-flatfs#141](https://github.com/ipfs/go-ds-flatfs/pull/141))\n - update dependencies (#134) ([ipfs/go-ds-flatfs#134](https://github.com/ipfs/go-ds-flatfs/pull/134))\n- github.com/ipfs/go-ds-pebble (v0.5.7 -> v0.5.9):\n - new version (#79) ([ipfs/go-ds-pebble#79](https://github.com/ipfs/go-ds-pebble/pull/79))\n - update error checks to use errors.Is (#78) ([ipfs/go-ds-pebble#78](https://github.com/ipfs/go-ds-pebble/pull/78))\n - new version (#76) ([ipfs/go-ds-pebble#76](https://github.com/ipfs/go-ds-pebble/pull/76))\n- github.com/ipfs/go-dsqueue (v0.1.1 -> v0.2.0):\n - release v0.2.0 (#33) ([ipfs/go-dsqueue#33](https://github.com/ipfs/go-dsqueue/pull/33))\n - update dependencies and required go version (#32) ([ipfs/go-dsqueue#32](https://github.com/ipfs/go-dsqueue/pull/32))\n - new version (#31) ([ipfs/go-dsqueue#31](https://github.com/ipfs/go-dsqueue/pull/31))\n - refactor: put queued item decoding logic into function (#30) ([ipfs/go-dsqueue#30](https://github.com/ipfs/go-dsqueue/pull/30))\n - use testing/synctest for artificial clock (#29) ([ipfs/go-dsqueue#29](https://github.com/ipfs/go-dsqueue/pull/29))\n- github.com/ipfs/go-ipfs-cmds (v0.15.0 -> v0.16.0):\n - new version (#325) ([ipfs/go-ipfs-cmds#325](https://github.com/ipfs/go-ipfs-cmds/pull/325))\n - update to use WaitGroup.Go (#324) ([ipfs/go-ipfs-cmds#324](https://github.com/ipfs/go-ipfs-cmds/pull/324))\n - refactor: modernize code (#322) ([ipfs/go-ipfs-cmds#322](https://github.com/ipfs/go-ipfs-cmds/pull/322))\n - feat: add --dereference-symlinks flag for recursive symlink resolution (#315) ([ipfs/go-ipfs-cmds#315](https://github.com/ipfs/go-ipfs-cmds/pull/315))\n - fix: add remaining binary content types to MIMEEncodings\n - fix: recognize content-type application/x-tar (#320) ([ipfs/go-ipfs-cmds#320](https://github.com/ipfs/go-ipfs-cmds/pull/320))\n - feat(http): ability to control Content-Type of binary responses (#311) ([ipfs/go-ipfs-cmds#311](https://github.com/ipfs/go-ipfs-cmds/pull/311))\n - update dependencies and fix test (#318) ([ipfs/go-ipfs-cmds#318](https://github.com/ipfs/go-ipfs-cmds/pull/318))\n - update dependencies (#296) ([ipfs/go-ipfs-cmds#296](https://github.com/ipfs/go-ipfs-cmds/pull/296))\n- github.com/ipfs/go-ipfs-pq (v0.0.3 -> v0.0.4):\n - new version (#23) ([ipfs/go-ipfs-pq#23](https://github.com/ipfs/go-ipfs-pq/pull/23))\n - fix broken test (#22) ([ipfs/go-ipfs-pq#22](https://github.com/ipfs/go-ipfs-pq/pull/22))\n - Update readme links (#21) ([ipfs/go-ipfs-pq#21](https://github.com/ipfs/go-ipfs-pq/pull/21))\n- github.com/ipfs/go-log/v2 (v2.9.0 -> v2.9.1):\n - new version (#179) ([ipfs/go-log#179](https://github.com/ipfs/go-log/pull/179))\n- github.com/ipfs/go-peertaskqueue (v0.8.2 -> v0.8.3):\n - new version ([ipfs/go-peertaskqueue#51](https://github.com/ipfs/go-peertaskqueue/pull/51))\n - update dependencies ([ipfs/go-peertaskqueue#50](https://github.com/ipfs/go-peertaskqueue/pull/50))\n - Add README.md ([ipfs/go-peertaskqueue#49](https://github.com/ipfs/go-peertaskqueue/pull/49))\n - replace go-clock with synctest ([ipfs/go-peertaskqueue#47](https://github.com/ipfs/go-peertaskqueue/pull/47))\n- github.com/ipfs/go-unixfsnode (v1.10.2 -> v1.10.3):\n - new version ([ipfs/go-unixfsnode#92](https://github.com/ipfs/go-unixfsnode/pull/92))\n - refactor: modernize code ([ipfs/go-unixfsnode#91](https://github.com/ipfs/go-unixfsnode/pull/91))\n- github.com/ipld/go-ipld-prime (v0.21.0 -> v0.22.0):\n failed to fetch repo\n- github.com/ipshipyard/p2p-forge (v0.6.1 -> v0.7.0):\n - chore: release v0.7.0 (#81) ([ipshipyard/p2p-forge#81](https://github.com/ipshipyard/p2p-forge/pull/81))\n - feat: add IP denylist plugin for abuse prevention (#79) ([ipshipyard/p2p-forge#79](https://github.com/ipshipyard/p2p-forge/pull/79))\n - refactor/test/fix: ipparser hardening (#75) ([ipshipyard/p2p-forge#75](https://github.com/ipshipyard/p2p-forge/pull/75))\n- github.com/libp2p/go-libp2p (v0.45.0 -> v0.47.0):\n - Release v0.47.0 (#3454) ([libp2p/go-libp2p#3454](https://github.com/libp2p/go-libp2p/pull/3454))\n - rcmgr: expose resource limits to Prometheus (#3433) ([libp2p/go-libp2p#3433](https://github.com/libp2p/go-libp2p/pull/3433))\n - update webtransport-go to v0.10.0 and quic-go to v0.59.0 (#3452)f\n - fix: handle error from mh.Sum in IDFromPublicKey\n - fix(basic_host): set read deadline before multistream Close to prevent blocking\n - update simnet dependency\n - rename simconlibp2p package to simlibp2p\n - simlibp2p: add GetBasicHostPair helper\n - run synctest with Go 1.25\n - fix(autonatv2): secondary addrs inherit reachability from primary (#3435) ([libp2p/go-libp2p#3435](https://github.com/libp2p/go-libp2p/pull/3435))\n - Release v0.46.0\n - chore: update quic-go to v0.57.1 (#3439) ([libp2p/go-libp2p#3439](https://github.com/libp2p/go-libp2p/pull/3439))\n - fix(mdns): filter addresses to reduce packet size (#3434) ([libp2p/go-libp2p#3434](https://github.com/libp2p/go-libp2p/pull/3434))\n - chore: update quic-go to v0.56.0 (#3425) ([libp2p/go-libp2p#3425](https://github.com/libp2p/go-libp2p/pull/3425))\n - fix(webrtc): use debug level for pion errors (#3426) ([libp2p/go-libp2p#3426](https://github.com/libp2p/go-libp2p/pull/3426))\n- github.com/libp2p/go-libp2p-kad-dht (v0.36.0 -> v0.38.0):\n - Release v0.38.0 (#1236) ([libp2p/go-libp2p-kad-dht#1236](https://github.com/libp2p/go-libp2p-kad-dht/pull/1236))\n - fix(provider/keystore): protect keystore size during reset (#1227) ([libp2p/go-libp2p-kad-dht#1227](https://github.com/libp2p/go-libp2p-kad-dht/pull/1227))\n - update dependencies and minimum go version (#1230) ([libp2p/go-libp2p-kad-dht#1230](https://github.com/libp2p/go-libp2p-kad-dht/pull/1230))\n - refactor: apply go fix modernizers from Go 1.26 (#1231) ([libp2p/go-libp2p-kad-dht#1231](https://github.com/libp2p/go-libp2p-kad-dht/pull/1231))\n - chore: go-libdht org transfer (#1229) ([libp2p/go-libp2p-kad-dht#1229](https://github.com/libp2p/go-libp2p-kad-dht/pull/1229))\n - fix(provider): close datastore results (#1226) ([libp2p/go-libp2p-kad-dht#1226](https://github.com/libp2p/go-libp2p-kad-dht/pull/1226))\n - new version (#1225) ([libp2p/go-libp2p-kad-dht#1225](https://github.com/libp2p/go-libp2p-kad-dht/pull/1225))\n - replace multierr with errors.Join (#1224) ([libp2p/go-libp2p-kad-dht#1224](https://github.com/libp2p/go-libp2p-kad-dht/pull/1224))\n - fix(routing): add per-peer timeouts for PutValue and Provide (#1222) ([libp2p/go-libp2p-kad-dht#1222](https://github.com/libp2p/go-libp2p-kad-dht/pull/1222))\n - chore: release v0.37.0 (#1221) ([libp2p/go-libp2p-kad-dht#1221](https://github.com/libp2p/go-libp2p-kad-dht/pull/1221))\n - fix(provider): keyspace exploration should succeed with a single peer (#1220) ([libp2p/go-libp2p-kad-dht#1220](https://github.com/libp2p/go-libp2p-kad-dht/pull/1220))\n - fix(provider): hold scheduleLk when reading schedule.Size() in test (#1219) ([libp2p/go-libp2p-kad-dht#1219](https://github.com/libp2p/go-libp2p-kad-dht/pull/1219))\n - fix(provider): close worker pool before wg.Wait() (#1218) ([libp2p/go-libp2p-kad-dht#1218](https://github.com/libp2p/go-libp2p-kad-dht/pull/1218))\n - chore: remove deprecated providers pkg (#1211) ([libp2p/go-libp2p-kad-dht#1211](https://github.com/libp2p/go-libp2p-kad-dht/pull/1211))\n - fix(provider): don't discard peers if they all share CPL during exploration (#1216) ([libp2p/go-libp2p-kad-dht#1216](https://github.com/libp2p/go-libp2p-kad-dht/pull/1216))\n - fix(records): clone addresses received from peerstore (#1210) ([libp2p/go-libp2p-kad-dht#1210](https://github.com/libp2p/go-libp2p-kad-dht/pull/1210))\n - tests: fix flaky TestOptimisticProvide (#1213) ([libp2p/go-libp2p-kad-dht#1213](https://github.com/libp2p/go-libp2p-kad-dht/pull/1213))\n - tests: fix flaky TestHandleRemotePeerProtocolChanges (#1212) ([libp2p/go-libp2p-kad-dht#1212](https://github.com/libp2p/go-libp2p-kad-dht/pull/1212))\n - chore: bump go-libp2p to v0.46 (#1209) ([libp2p/go-libp2p-kad-dht#1209](https://github.com/libp2p/go-libp2p-kad-dht/pull/1209))\n- github.com/libp2p/go-libp2p-pubsub (v0.14.2 -> v0.15.0):\n - release v0.15.0\n - Fix data race in test\n - Skip flaky floodsub test\n - pubsub: remove redundant sends of hello packet\n - gossipsub: implement extensions\n - test: add skeleton gossipsub to drive a gossipsub peer\n - gossipsub: add plumbing for Gossipsub v1.3 support\n - pubsub: AddPeer now accepts reference to hello packet\n - pb: add extensions protobufs\n - feat: add px peer record reducer + pub addrs filter\n - Migrate to `log/slog`\n - chore: add params.Dscore validation\n - Release v0.14.3\n - Unexport Params.Validate to maintain patch release semantics\n - Merge pull request #642 for GossipSub Params validation\n - fix: Select ctx.Done() when preprocessing to avoid blocking on cancel (#635) ([libp2p/go-libp2p-pubsub#635](https://github.com/libp2p/go-libp2p-pubsub/pull/635))\n- github.com/libp2p/go-netroute (v0.3.0 -> v0.4.0):\n - v0.4.0\n - gofmt ([libp2p/go-netroute#65](https://github.com/libp2p/go-netroute/pull/65))\n - linux: use rtnetlink directly ([libp2p/go-netroute#64](https://github.com/libp2p/go-netroute/pull/64))\n- github.com/multiformats/go-multiaddr-dns (v0.4.1 -> v0.5.0):\n - Release v0.5.0\n - refactor: remove miekg/dns dep (#72) ([multiformats/go-multiaddr-dns#72](https://github.com/multiformats/go-multiaddr-dns/pull/72))\n\n
\n\n### 👨‍👩‍👧‍👦 Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| [@lidel](https://github.com/lidel) | 62 | +18446/-3513 | 406 |\n| [@gammazero](https://github.com/gammazero) | 84 | +5719/-2815 | 374 |\n| [@MarcoPolo](https://github.com/MarcoPolo) | 24 | +1275/-311 | 58 |\n| [@guillaumemichel](https://github.com/guillaumemichel) | 14 | +392/-967 | 41 |\n| [@hsanjuan](https://github.com/hsanjuan) | 4 | +1093/-43 | 15 |\n| [@sneaxhuh](https://github.com/sneaxhuh) | 2 | +840/-19 | 7 |\n| [@cortze](https://github.com/cortze) | 1 | +367/-35 | 2 |\n| [@schomatis](https://github.com/schomatis) | 1 | +288/-17 | 5 |\n| [@rifeplight](https://github.com/rifeplight) | 1 | +92/-195 | 18 |\n| [@sukunrt](https://github.com/sukunrt) | 3 | +22/-211 | 7 |\n| [@2color](https://github.com/2color) | 1 | +207/-2 | 5 |\n| [@djdv](https://github.com/djdv) | 8 | +96/-65 | 10 |\n| [@vlerdman](https://github.com/vlerdman) | 4 | +90/-38 | 8 |\n| [@v1rtl](https://github.com/v1rtl) | 1 | +71/-3 | 2 |\n| [@VedantMadane](https://github.com/VedantMadane) | 1 | +23/-8 | 4 |\n| [@dozyio](https://github.com/dozyio) | 1 | +26/-4 | 2 |\n| [@infrmtcs](https://github.com/infrmtcs) | 1 | +24/-1 | 2 |\n| [@marten-seemann](https://github.com/marten-seemann) | 1 | +5/-1 | 2 |\n| [@web3-bot](https://github.com/web3-bot) | 2 | +3/-2 | 2 |\n| [@MozirDmitriy](https://github.com/MozirDmitriy) | 1 | +4/-1 | 1 |\n| [@aschmahmann](https://github.com/aschmahmann) | 1 | +2/-1 | 1 |\n| [@willscott](https://github.com/willscott) | 1 | +1/-1 | 1 |\n| [@lbarrettanderson](https://github.com/lbarrettanderson) | 1 | +1/-1 | 1 |\n| [@filipremb](https://github.com/filipremb) | 1 | +1/-1 | 1 |\n\n## v0.40.1\n\n### 🔦 Highlights\n\n#### Windows stability fix\n\nIf you run Kubo on Windows, v0.40.0 can crash after running for a while. The daemon starts fine and works normally at first, but eventually hits a memory corruption in Go's network I/O layer and dies. This is likely caused by an upstream Go 1.26 regression in overlapped I/O handling that has known issues ([go#77142](https://github.com/golang/go/issues/77142), [#11214](https://github.com/ipfs/kubo/issues/11214)).\n\nThis patch release downgrades the Go toolchain from 1.26 to 1.25, which does not have this bug. If you are running Kubo on Windows, upgrade to v0.40.1. We will switch back to Go 1.26.x once the upstream fix lands.\n\n### 📝 Changelog\n\n
Full Changelog v0.40.1\n\n- github.com/ipfs/kubo:\n - chore: downgrade to Go 1.25 to fix Windows crash ([ipfs/kubo#11215](https://github.com/ipfs/kubo/pull/11215))\n\n
\n" }, { "path": "docs/changelogs/v0.41.md", "content": "# Kubo changelog v0.41\n\n\n\nThis release was brought to you by the [Shipyard](https://ipshipyard.com/) team.\n\n- [v0.41.0](#v0410)\n\n## v0.41.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n - [🗑️ Faster Provide Queue Disk Reclamation](#-faster-provide-queue-disk-reclamation)\n - [✨ New `ipfs cid inspect` command](#-new-ipfs-cid-inspect-command)\n - [🖥️ WebUI Improvements](#-webui-improvements)\n - [🔧 Correct provider addresses for custom HTTP routing](#-correct-provider-addresses-for-custom-http-routing)\n - [📦️ Dependency updates](#-dependency-updates)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n#### 🗑️ Faster Provide Queue Disk Reclamation\n\nNodes with significant amount of data and DHT provide sweep enabled\n(`Provide.DHT.SweepEnabled`, the default since Kubo 0.39) could see their\n`datastore/` directory grow continuously.\nEach reprovide cycle rewrote the provider keystore inside the shared repo\ndatastore, generating tombstones faster than the storage engine could compact\nthem, and in default configuration Kubo was slow to reclaim this space.\n\nThe provider keystore now lives in a dedicated datastore under\n`$IPFS_PATH/provider-keystore/`. After each reprovide cycle the old datastore\nis removed from disk entirely, so space is reclaimed immediately regardless\nof storage backend.\n\nOn first start after upgrading, stale keystore data is cleaned up from the\nshared datastore automatically.\n\nTo learn more, see [kubo#11096](https://github.com/ipfs/kubo/issues/11096),\n[kubo#11198](https://github.com/ipfs/kubo/pull/11198), and\n[go-libp2p-kad-dht#1233](https://github.com/libp2p/go-libp2p-kad-dht/pull/1233).\n\n#### ✨ New `ipfs cid inspect` command\n\nNew subcommand for breaking down a CID into its components. Works offline, supports `--enc=json`.\n\n```console\n$ ipfs cid inspect bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\nCID: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\nVersion: 1\nMultibase: base32 (b)\nMulticodec: dag-pb (0x70)\nMultihash: sha2-256 (0x12)\n Length: 32 bytes\n Digest: c3c4733ec8affd06cf9e9ff50ffc6bcd2ec85a6170004bb709669c31de94391a\nCIDv0: QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\nCIDv1: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\n```\n\nSee `ipfs cid --help` for all CID-related commands.\n\n#### 🖥️ WebUI Improvements\n\nIPFS Web UI has been updated to [v4.12.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.12.0).\n\n##### IPv6 peer geolocation and Peers screen optimizations\n\nThe Peers screen now resolves IPv6 addresses to geographic locations, and the geolocation database has been updated to `GeoLite2-City-CSV_20260220`. ([ipfs-geoip v9.3.0](https://github.com/ipfs-shipyard/ipfs-geoip/releases/tag/v9.3.0))\n\nPeer locations load faster thanks to UX optimizations in the underlying ipfs-geoip library.\n\n#### 🔧 Correct provider addresses for custom HTTP routing\n\nNodes using custom routing (`Routing.Type=custom`) with [IPIP-526](https://github.com/ipfs/specs/pull/526) could end up publishing unresolved `0.0.0.0` addresses in provider records. Addresses are now resolved at provide-time, and when AutoNAT V2 has confirmed publicly reachable addresses, those are preferred automatically. See [#11213](https://github.com/ipfs/kubo/issues/11213).\n\n#### 📦️ Dependency updates\n\n- update `ipfs-webui` to [v4.12.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.12.0)\n- update `gateway-conformance` tests to [v0.11](https://github.com/ipfs/gateway-conformance/releases/tag/v0.11.0)\n\n### 📝 Changelog\n\n### 👨‍👩‍👧‍👦 Contributors\n" }, { "path": "docs/changelogs/v0.42.md", "content": "# Kubo changelog v0.42\n\n\n\nThis release was brought to you by the [Shipyard](https://ipshipyard.com/) team.\n\n- [v0.42.0](#v0420)\n\n## v0.42.0\n\n- [Overview](#overview)\n- [🔦 Highlights](#-highlights)\n- [📝 Changelog](#-changelog)\n- [👨‍👩‍👧‍👦 Contributors](#-contributors)\n\n### Overview\n\n### 🔦 Highlights\n\n### 📝 Changelog\n\n### 👨‍👩‍👧‍👦 Contributors\n" }, { "path": "docs/changelogs/v0.5.md", "content": "# go-ipfs changelog v0.5\n\n## v0.5.1 2020-05-08\n\nHot on the heels of 0.5.0 is 0.5.1 with some important but small bug fixes. This release:\n\n1. Removes the 1 minute timeout for IPNS publishes (fixes #7244).\n2. Backport a DHT fix to reduce CPU usage for canceled requests.\n3. Fixes some timer leaks in the QUIC transport ([ipfs/go-ipfs#2515](https://github.com/lucas-clemente/quic-go/issues/2515)).\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - IPNS timeout patch from master ([ipfs/go-ipfs#7276](https://github.com/ipfs/go-ipfs/pull/7276))\n- github.com/libp2p/go-libp2p-core (v0.5.2 -> v0.5.3):\n - feat: add a function to tell if a context subscribes to query events ([libp2p/go-libp2p-core#147](https://github.com/libp2p/go-libp2p-core/pull/147))\n- github.com/libp2p/go-libp2p-kad-dht (v0.7.10 -> v0.7.11):\n - fix: optimize for the case where we're not subscribing to query events ([libp2p/go-libp2p-kad-dht#624](https://github.com/libp2p/go-libp2p-kad-dht/pull/624))\n - fix: don't spin when the event channel is closed ([libp2p/go-libp2p-kad-dht#622](https://github.com/libp2p/go-libp2p-kad-dht/pull/622))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.2.2 -> v0.2.3):\n - fix: avoid subscribing to query events unless necessary ([libp2p/go-libp2p-routing-helpers#43](https://github.com/libp2p/go-libp2p-routing-helpers/pull/43))\n- github.com/lucas-clemente/quic-go (v0.15.5 -> v0.15.7):\n - reset the PTO when dropping a packet number space\n - move deadlineTimer declaration out of the Read loop\n - stop the deadline timer in Stream.Read and Write\n - fix buffer use after it was released when sending an INVALID_TOKEN error\n - create the session timer at the beginning of the run loop\n - stop the timer when the session's run loop returns\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------------------|---------|---------|---------------|\n| Marten Seemann | 10 | +81/-62 | 19 |\n| Steven Allen | 5 | +42/-18 | 10 |\n| Adin Schmahmann | 1 | +2/-8 | 1 |\n| dependabot-preview[bot] | 2 | +6/-2 | 4 |\n\n## v0.5.0 2020-04-28\n\nWe're excited to announce go-ipfs 0.5.0! This is by far the largest go-ipfs release with ~2500 commits, 98 contributors, and over 650 PRs across ipfs, libp2p, and multiformats.\n\n### Highlights\n\n#### Content Routing\n\nThe primary focus of this release was on improving content routing. That is, advertising and finding content. To that end, this release heavily focuses on improving the DHT.\n\n##### Improved DHT\n\nThe distributed hash table (DHT) is how IPFS nodes keep track of who has what data. The DHT implementation has been almost completely rewritten in this release. Providing, finding content, and resolving IPNS records are now all much faster. However, there are risks involved with this update due to the significant amount of changes that have gone into this feature.\n\nThe current DHT suffers from three core issues addressed in this release:\n\n- Most peers in the DHT cannot be dialed (e.g., due to firewalls and NATs). Much of a DHT query time is wasted trying to connect to peers that cannot be reached.\n- The DHT query logic doesn't properly terminate when it hits the end of the query and, instead, aggressively keeps on searching.\n- The routing tables are poorly maintained. This can cause search performance to slow down linearly with network size, instead of logarithmically as expected.\n\n###### Reachability\n\nWe have addressed the problem of undialable nodes by having nodes wait to join the DHT as _server_ nodes until they've confirmed that they are reachable from the public internet.\n\nTo ensure that nodes which are not publicly reachable (ex behind VPNs, offline LANs, etc.) can still coordinate and share data, go-ipfs 0.5 will run two DHTs: one for private networks and one for the public internet. Every node will participate in a LAN DHT and a public WAN DHT. See [Dual DHT](#dual-dht) for more details.\n\n###### Dual DHT\n\nAll IPFS nodes will now run two DHTs: one for the public internet WAN, and one for their local network LAN.\n\n1. When connected to the public internet, IPFS will use both DHTs for finding peers, content, and IPNS records. Nodes only publish provider and IPNS records to the WAN DHT to avoid flooding the local network.\n2. When not connected to the public internet, nodes publish provider and IPNS records to the LAN DHT.\n\nThe WAN DHT includes all peers with at least one public IP address. This release will only consider an IPv6 address public if it is in the [public internet range `2000::/3`](https://www.iana.org/assignments/ipv6-address-space/ipv6-address-space.xhtml).\n\nThis feature should not have any noticeable impact on go-ipfs, performance, or otherwise. Everything should continue to work in all the currently supported network configurations: VPNs, disconnected LANs, public internet, etc.\n\n###### Query Logic\n\nWe've improved the DHT query logic to more closely follow Kademlia. This should significantly speed up:\n\n- Publishing IPNS & provider records.\n- Resolving IPNS addresses.\n\nPreviously, nodes would continue searching until they timed out or ran out of peers before stopping (putting or returning data found). Now, nodes will now stop as soon as they find the closest peers.\n\n###### Routing Tables\n\nFinally, we've addressed the poorly maintained routing tables by:\n\n- Reducing the likelihood that the connection manager will kill connections to peers in the routing table.\n- Keeping peers in the routing table, even if we get disconnected from them.\n- Actively and frequently querying the DHT to keep our routing table full.\n- Prioritizing useful peers that respond to queries quickly.\n\n##### Testing\n\nThe DHT rewrite was made possible by [Testground](https://github.com/ipfs/testground/), our new testing framework. Testground allows us to spin up multi-thousand node tests with simulated real-world network conditions. By combining Testground and some custom analysis tools, we were able to gain confidence that the new DHT implementation behaves correctly.\n\n##### Provider Record Changes\n\nWhen you add content to your IPFS node, you advertise this content to the network by announcing it in the DHT. We call this _providing_.\n\nHowever, go-ipfs has multiple ways to address the same underlying bytes. Specifically, we address content by content ID (CID) and the same underlying bytes can be addressed using (a) two different versions of CIDs (CIDv0 and CIDv1) and (b) with different _codecs_ depending on how we're interpreting the data.\n\nPrior to go-ipfs 0.5.0, we used the content id (CID) in the DHT when sending out provider records for content. Unfortunately, this meant that users trying to find data announced using one CID wouldn't find nodes providing the content under a different CID.\n\nIn go-ipfs 0.5.0, we're announcing data by _multihash_, not _CID_. This way, regardless of the CID version used by the peer adding the content, the peer trying to download the content should still be able to find it.\n\n**Warning:** as part of the network, this could impact finding content added with CIDv1. Because go-ipfs 0.5.0 will announce and search for content using the bare multihash (equivalent to the v0 CID), go-ipfs 0.5.0 will be unable to find CIDv1 content published by nodes prior to go-ipfs 0.5.0 and vice-versa. As CIDv1 is _not_ enabled by default so we believe this will have minimal impact. However, users are _strongly_ encouraged to upgrade as soon as possible.\n\n#### Content Transfer\n\nA secondary focus in this release was improving content _transfer_, our data exchange protocols.\n\n##### Refactored Bitswap\n\nThis release includes a major [Bitswap refactor](https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distribution/), running a new and backward compatible Bitswap protocol. We expect these changes to improve performance significantly.\n\nWith the refactored Bitswap, we expect:\n\n- Few to no duplicate blocks when fetching data from other nodes speaking the _new_ protocol.\n- Better parallelism when fetching from multiple peers.\n\nThe new Bitswap won't magically make downloading content any faster until both seeds and leaches have updated. If you're one of the first to upgrade to `0.5.0` and try downloading from peers that haven't upgraded, you're unlikely to see much of a performance improvement.\n\n[bitswap-refactor]: https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distribution/\n\n##### Server-Side Graphsync Support (Experimental)\n\nGraphsync is a new exchange protocol that operates at the IPLD Graph layer instead of the Block layer like bitswap.\n\nFor example, to download \"/ipfs/QmExample/index.html\":\n\n* Bitswap would download QmFoo, lookup \"index.html\" in the directory named by\nQmFoo, resolving it to a CID QmIndex. Finally, bitswap would download QmIndex.\n* Graphsync would ask peers for \"/ipfs/QmFoo/index.html\". Specifically, it would ask for the child named \"index.html\" of the object named by \"QmFoo\".\n\nThis saves us round-trips in exchange for some extra protocol complexity. Moreover, this protocol allows specifying more powerful queries like \"give me everything under QmFoo\". This can be used to quickly download a large amount of data with few round-trips.\n\nAt the moment, go-ipfs cannot use this protocol to download content from other peers. However, if enabled, go-ipfs can _serve_ content to other peers over this protocol. This may be useful for pinning services that wish to quickly replicate client data.\n\nTo enable, run:\n\n```bash\n> ipfs config --json Experimental.GraphsyncEnabled true\n```\n\n#### Datastores\n\nContinuing with the of improving our core data handling subsystems, both of the datastores used in go-ipfs, Badger and flatfs, have received important updates in this release:\n\n##### Badger\n\nBadger has been in go-ipfs for over a year as an experimental feature, and we're promoting it to stable (but not default). For this release, we've switched from writing to disk synchronously to explicitly syncing where appropriate, significantly increasing write throughput.\n\nThe current and default datastore used by go-ipfs is [FlatFS](https://github.com/ipfs/go-ds-flatfs). FlatFS essentially stores blocks of data as individual files on your file system. However, there are lots of optimizations a specialized database can do that a standard file system can not.\n\nThe benefit of Badger is that adding/fetching data to/from Badger is significantly faster than adding/fetching data to/from the default datastore, FlatFS. In some tests, adding data to Badger is 32x faster than FlatFS (in this release).\n\n###### Enable Badger\n\nIn this release, we're marking the badger datastore as stable. However, we're not yet enabling it by default. You can enable it at initialization by running: `ipfs init --profile=badgerds`\n\n###### Issues with Badger\n\nWhile Badger is a great solution, there are some issues you should consider before enabling it.\n\nBadger is complicated. FlatFS pushes all the complexity down into the filesystem itself. That means that FlatFS is only likely to lose your data if your underlying filesystem gets corrupted while there are more opportunities for Badger itself to get corrupted.\n\nBadger can use a lot of memory. In this release, we've tuned Badger to use `~20MB` of memory by default. However, it can still produce spikes as large as [`1GiB` of data](https://github.com/dgraph-io/badger/issues/1292) in memory usage when garbage collecting.\n\nFinally, Badger isn't very aggressive when it comes to garbage collection, and we're still investigating ways to get it to more aggressively clean up after itself.\n\nWe suggest you use Badger if:\n\n- Performance is your main requirement.\n- You rarely delete anything.\n- You have some memory to spare.\n\n##### Flatfs\n\nIn the flatfs datastore, we've fixed an issue where temporary files could be left behind in some cases. While this release will avoid leaving behind temporary files, you may want to remove any left behind by previous releases:\n\n```bash\n> rm ~/.ipfs/blocks/*/put-*\n> rm ~/.ipfs/blocks/du-*\n```\n\nWe've also hardened several edge-cases in flatfs to reduce the impact of file descriptor limits, spurious crashes, etc.\n\n#### Libp2p\n\nMany improvements and bug fixes were made to libp2p over the course of this release. These release notes only include the most important and those most relevant to the content routing improvements.\n\n##### Improved Backoff Logic\n\nWhen we fail to connect to a peer, we \"backoff\" and refuse to re-connect to that peer for a short period of time. This prevents us from wasting resources repeatedly failing to connect to the same unreachable peer.\n\nUnfortunately, the old backoff logic was flawed: if we failed to connect to a peer and entered the \"backoff\" state, we wouldn't try to re-connect to that peer even if we had learned new and potentially working addresses for the peer. We've fixed this by applying backoff to each _address_ instead of to the peer as a whole. This achieves the same result as we'll stop repeatedly trying to connect to the peer at known-bad addresses, but it allows us to reach the peer if we later learn about a good address.\n\n##### AutoNAT\n\nThis release uses Automatic NAT Detection (AutoNAT) - determining if the node is _reachable_ from the public internet - to make decisions about how to participate in IPFS. This subsystem is used to determine if the node should store some of the public DHT, and if it needs to use relays to be reached by others. In short:\n\n1. An AutoNAT client asks a node running an AutoNAT service if it can be reached at one of a set of guessed addresses.\n2. The AutoNAT service attempts to _dial back_ those addresses, with some restrictions. We won't dial back to a different IP address, for example.\n3. If the AutoNAT service succeeds, it reports back the address it successfully dialed, and the AutoNAT client knows that it is reachable from the public internet.\n\nAll nodes act as AutoNAT clients to determine if they should switch into DHT server mode. As of this release, nodes will by default run the service side of AutoNAT - verifying connectivity - for up to 30 peers every minute. This service should have minimal overhead and will be disabled for nodes in the `lowpower` configuration profile, and those which believe they are not publicly reachable.\n\nIn addition to enabling the AutoNAT service by default, this release changes the AutoNAT config options:\n\n1. The `Swarm.EnableAutoNATService` option has been removed.\n2. A new AutoNAT section has been added to the config. This section is empty by default.\n\n\n##### IPFS/Libp2p Address Format\n\nIf you've ever run a command like `ipfs swarm peers`, you've likely seen paths that look like `/ip4/193.45.1.24/tcp/4001/ipfs/QmSomePeerID`. These paths are _not_ file paths, they're multiaddrs; addresses of peers on the network.\n\nUnfortunately, `/ipfs/Qm...` is _also_ the same path format we use for files. This release, changes the multiaddr format from /ip4/193.45.1.24/tcp/4001/ipfs/QmSomePeerID to /ip4/193.45.1.24/tcp/4001/p2p/QmSomePeerID to make the distinction clear.\n\nWhat this means for users:\n\n* Old-style multiaddrs will still be accepted as inputs to IPFS.\n* If you were using a multiaddr library (go, js, etc.) to name _files_ because `/ipfs/QmSomePeerID` looks like `/ipfs/QmSomeFile`, your tool may break if you upgrade this library.\n* If you're manually parsing multiaddrs and are searching for the string `/ipfs/`..., you'll need to search for `/p2p/...`.\n\n##### Minimum RSA Key Size\n\nPreviously, IPFS did not enforce a minimum RSA key size. In this release, we've introduced a minimum 2048 bit RSA key size. IPFS generates 2048 bit RSA keys by default so this shouldn't be an issue for anyone in practice. However, users who explicitly chose a smaller key size will not be able to communicate with new nodes.\n\nUnfortunately, some of the bootstrap peers _did_ intentionally generate 1024 bit RSA keys so they'd have vanity peer addresses (starting with QmSoL for \"solar net\"). All IPFS nodes should _also_ have peers with >= 2048 bit RSA keys in their bootstrap list, but we've introduced a migration to ensure this.\n\nWe implemented this change to follow security best practices and to remove a potential foot-gun. However, in practice, the security impact of allowing insecure RSA keys should have been next to none because IPFS doesn't trust other peers on the network anyways.\n\n##### TLS By Default\n\nIn this release, we're switching TLS to be the _default_ transport. This means we'll try to encrypt the connection with TLS before re-trying with SECIO.\n\nContrary to the announcement in the go-ipfs 0.4.23 release notes, this release does not remove SECIO support to maintain compatibility with js-ipfs.\n\nNote: The `Experimental.PreferTLS` configuration option is now ignored.\n\n##### SECIO Deprecation Notice\n\nSECIO should be considered to be well on the way to deprecation and will be\ncompletely disabled in either the next release (0.6.0, ~mid May) or the one\nfollowing that (0.7.0, ~end of June). Before SECIO is disabled, support will be\nadded for the NOISE transport for compatibility with other IPFS implementations.\n\n##### QUIC Upgrade\n\nIf you've been using the experimental QUIC support, this release upgrades to a new and _incompatible_ version of the QUIC protocol (draft 27). Old and new go-ipfs nodes will still interoperate, but not over the QUIC transport.\n\nWe intend to standardize on this draft of the QUIC protocol and enable QUIC by default in the next release if all goes well.\n\nNOTE: QUIC does not yet support [private networks](./docs/experimental-features.md#private-networks).\n\n#### Gateway\n\nIn addition to a bunch of bug fixes, we've made two improvements to the gateway.\n\nYou can play with both of these features by visiting:\n\n> http://bafybeia6po64b6tfqq73lckadrhpihg2oubaxgqaoushquhcek46y3zumm.ipfs.localhost:8080\n\n##### Subdomain Gateway\n\nFirst up, we've changed how URLs in the IPFS gateway work for better browser\nsecurity. The gateway will now redirect from\n`http://localhost:8080/ipfs/CID/...` to `http://CID.ipfs.localhost:8080/...` by\ndefault. This:\n\n* Ensures that every dapp gets its own browser origin.\n* Makes it easier to write websites that \"just work\" with IPFS because absolute paths will now work (though you should still use relative links because they're better).\n\nPaths addressing the gateway by IP address (`http://127.0.0.1:5001/ipfs/CID`) will not be altered as IP addresses can't have subdomains.\n\nNote: cURL doesn't follow redirects by default. To avoid breaking cURL and other clients that don't support redirects, go-ipfs will return the requested file along with the redirect. Browsers will follow the redirect and abort the download while cURL will ignore the redirect and finish the download.\n\n##### Directory Listing\n\nThe second feature is a face-lift to the directory listing theme and color palette.\n\n> http://bafybeia6po64b6tfqq73lckadrhpihg2oubaxgqaoushquhcek46y3zumm.ipfs.localhost:8080\n\n#### IPNS\n\nThis release includes several new IPNS and IPNS-related features.\n\n##### ENS\n\nIPFS now resolves [ENS](https://ens.domains/) names (e.g., `/ipns/ipfs.eth`) via DNSLink provided by https://eth.link service.\n\n##### IPNS over PubSub\n\nIPFS has had experimental support for resolving IPNS over pubsub for a while. However, in the past, this feature was passive. When resolving an IPNS name, one would join a pubsub topic for the IPNS name and subscribe to _future_ updates. Unfortunately, this wouldn't speed-up initial IPNS lookups.\n\nIn this release, we've introduced a new \"record fetch\" protocol to speedup the initial lookup. Now, after subscribing to the pubsub topic for the IPNS key, nodes will use this new protocol to \"fetch\" the last-seen IPNS record from all peers subscribed to the topic.\n\nThis feature will be enabled by default in 0.6.0.\n\n##### IPNS with base32 PIDs\n\nIPNS names can now be expressed as special multibase CIDs. E.g.,\n\n> /ipns/bafzbeibxfjp4gaxc4cdn57257cyvc7jfa4rlp4e5min6geg44m57g6nx7e\n\nImportantly, this allows IPNS names to appear in subdomains in the new [subdomain gateway](#subdomain-gateway) feature.\n\n#### PubSub\n\nWe have made two major changes to the pubsub subsystem in this release:\n\n1. Pubsub now more aggressively finds and connects to other peers subscribing to the same topic.\n2. Go-ipfs has switched its default pubsub router from \"floodsub\", an inefficient but simple \"flooding\" pubsub implementation, to \"gossipsub\".\n\nPubSub will be stabilized in go-ipfs 0.6.0.\n\n#### CLI & API\n\nThe IPFS CLI and API have a couple of new features and changes.\n\n##### POST Only\n\nIPFS has two HTTP APIs:\n\n* Port 5001: http://localhost:5001/api/v0/... - the API\n* Port 8080: http://localhost:8080/api/v0/... - a read-only subset of the API, accessible via the gateway\n\nAs of this release, the main IPFS API (port 5001) will only accept POST requests. This change is necessary to tighten cross origin security in browsers.\n\nIf you're using the go-ipfs API in your application, you may need to change GET calls to POST calls or upgrade your libraries and tools.\n\n* go - go-ipfs-api - v0.0.3\n* js-ipfs-http-api - v0.41.1\n* orbit-db - v0.24.0 (unreleased)\n\n##### RIP \"Error: api not running\"\n\nIf you've ever seen [the error](https://github.com/ipfs/go-ipfs/issues/5784):\n\n> Error: api not running\n\nwhen trying to run a command without the daemon running, we have good news! You\nshould never see this error again. The `ipfs` command now correctly detects that the daemon is not, in fact, running, and directly opens the IPFS repo.\n\n##### RIP `ipfs repo fsck`\n\nThe `ipfs repo fsck` now does nothing but print an error message. Previously, it was used to cleanup some lock files: the \"api\" file that caused the aforementioned \"api not running\" error and the repo lock. However, this is no longer necessary.\n\n##### Init with config\n\nIt's now possible to initialize an IPFS node with an existing IPFS config by running:\n\n```bash\n> ipfs init /path/to/existing/config\n```\n\nThis will reuse the existing configuration in it's entirety (including the private key) and can be useful when:\n\n* Migrating a node's identity between machines without keeping the data.\n* Resetting the datastore.\n\n##### Ignoring Files\n\nFiles can now be ignored on add by passing the `--ignore` and/or\n`--ignore-rules-path` flags.\n\n* `--ignore=PATTERN` will ignore all files matching the gitignore rule PATTERN.\n* `--ignore-rules-path=FILENAME` will apply the gitignore rules from the specified file.\n\nFor example, to add a git repo while ignoring all files git would ignore, you could run:\n\n```bash\n> cd path/to/some/repo\n> ipfs add -r --hidden=false --ignore=.git --ignore-rules-path=.gitignore .\n```\n\n##### Named Pipes\n\nIt's now possible to add data directly from a named pipe:\n\n```bash\n> mkfifo foo\n> echo -n \"hello \" > foo &\n> echo -n \"world\" > bar &\n> ipfs add foo bar\n```\n\nThis can be useful when adding data from multiple streaming sources.\n\nNOTE: To avoid surprising users, IPFS will only add data from FIFOs _directly_ named on the command line, not FIFOs in a recursively added directory. Otherwise, `ipfs add` would halt whenever it encountered a FIFO with no data to be read leading to difficult to debug stalls.\n\n##### DAG import/export (.car)\n\nIPFS now allows rapid reading and writing of blocks in [`.car` format](https://github.com/ipld/specs/blob/master/block-layer/content-addressable-archives.md#readme). The functionality is accessible via the experimental `dag import` and `dag export` commands:\n\n```\n~$ ipfs dag export QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc \\\n| xz > welcome_to_ipfs.car.xz\n\n 0s 6.73 KiB / ? [-------=-------------------------------------] 5.16 MiB/s 0s\n\n```\nThen on another `ipfs` instance, not even connected to the network:\n```\n~$ xz -dc welcome_to_ipfs.car.xz | ipfs dag import\n\nPinned root\tQmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc\tsuccess\n\n```\n\n##### Pins\n\nWe've made two minor changes to the pinning subsystem:\n\n1. `ipfs pin ls --stream` allows streaming a pin listing.\n2. `ipfs pin update` no longer holds the global pin lock while fetching files from the network. This should hopefully make it significantly more useful.\n\n#### Daemon\n\n##### Zap Logging\n\nThe go-ipfs daemon has switched to using [Uber's Zap](https://go.uber.org/zap). Unlike our previous logging system, Zap supports _structured_ logging which can make parsing, filtering, and analyzing go-ipfs logs much simpler.\n\nTo enable structured logging, set the `IPFS_LOGGING_FMT` environment variable to \"json\".\n\nNote: while we've switched to using Zap as the logging backend, most of go-ipfs still logs strings.\n\n##### Systemd Support\n\nFor Linux users, this release includes support for two systemd features: socket activation and startup/shutdown notifications. This makes it possible to:\n\n* Start IPFS on demand on first use.\n* Wait for IPFS to finish starting before starting services that depend on it.\n\nYou can find the new systemd units in the go-ipfs repo under misc/systemd.\n\n##### IPFS API Over Unix Domain Sockets\n\nThis release supports exposing the IPFS API over a unix domain socket in the filesystem. You use this feature, run:\n\n```bash\n> ipfs config Addresses.API \"/unix/path/to/socket/location\"\n```\n\n##### Docker\n\nWe've made a few improvements to our docker image in this release:\n\n* It can now be cross-built for multiple architectures.\n* It now builds go-ipfs with OpenSSL support by default for faster libp2p handshakes.\n* A private-network \"swarm\" key can now be passed in to a docker image via either the `IPFS_SWARM_KEY=` or `IPFS_SWARM_KEY_FILE=` docker variables. Check out the Docker section of the README for more information.\n\n#### Plugins\n\ngo-ipfs plugins allow users to extend go-ipfs without modifying the original source-code. This release includes a few important changes.\n\nSee [docs/plugins.md](./docs/plugins.md) for details.\n\n##### MacOS Support\n\nPlugins are now supported on MacOS, in addition to Linux. Unfortunately, Go still doesn't [support plugins on Windows](https://github.com/golang/go/issues/19282).\n\n##### New Plugin Type: `InternalPlugin`\n\nThis release introduces a new `InternalPlugin` plugin type. When started, this plugin will be passed a raw `*IpfsNode` object, giving it access to all go-ipfs internals.\n\nThis plugin interface is permanently unstable as it has access to internals that can change frequently. However, it should allow power-users to develop deeply integrated extensions to go-ipfs, out-of-tree.\n\n##### Plugin Config\n\n**BREAKING**\n\nPlugins can now be configured and/or disabled via the [ipfs config file](./docs/plugins.md#configuration).\n\nTo make this possible, the plugin interface has changed. The `Init` function now takes an `*Environment` object. Specifically, the plugin signature has changed from:\n\n```go\ntype Plugin interface {\n\tName() string\n\tVersion() string\n\tInit() error\n}\n```\n\nto\n\n```go\ntype Environment struct {\n\t// Path to the IPFS repo.\n\tRepo string\n\n\t// The plugin's config, if specified.\n\tConfig interface{}\n}\n\ntype Plugin interface {\n\tName() string\n\tVersion() string\n\tInit(env *Environment) error\n}\n```\n\n#### Repo Migrations\n\nIPFS uses repo migrations to make structural changes to the \"repo\" (the config, data storage, etc.) on upgrade.\n\nThis release includes two very simple repo migrations: a config migration to ensure that the config contains working bootstrap nodes and a keystore migration to base32 encode all key filenames.\n\nIn general, migrations should not require significant manual intervention. However, you should be aware of migrations and plan for them.\n\n* If you update go-ipfs with `ipfs update`, `ipfs update` will run the migration for you. Note: `ipfs update` will refuse to run the migrations while ipfs itself is running.\n* If you start the ipfs daemon with `ipfs daemon --migrate`, ipfs will migrate your repo for you on start.\n\nOtherwise, if you want more control over the repo migration process, you can manually install and run the [repo migration tool](http://dist.ipfs.tech/#fs-repo-migrations).\n\n##### Bootstrap Peer Changes\n\n**AUTOMATIC MIGRATION REQUIRED**\n\nThe first migration will update the bootstrap peer list to:\n\n1. Replace the old bootstrap nodes (ones with peer IDs starting with QmSoL), with new bootstrap nodes (ones with addresses that start with `/dnsaddr/bootstrap.libp2p.io`).\n2. Rewrite the address format from `/ipfs/QmPeerID` to `/p2p/QmPeerID`.\n\nWe're migrating addresses for a few reasons:\n\n1. We're using DNS to address the new bootstrap nodes so we can change the underlying IP addresses as necessary.\n2. The new bootstrap nodes use 2048 bit keys while the old bootstrap nodes use 1024 bit keys.\n3. We're normalizing the address format to `/p2p/Qm...`.\n\nNote: This migration won't _add_ the new bootstrap peers to your config if you've explicitly removed the old bootstrap peers. It will also leave custom entries in the list alone. In other words, if you've customized your bootstrap list, this migration won't clobber your changes.\n\n##### Keystore Changes\n\n**AUTOMATIC MIGRATION REQUIRED**\n\ngo-ipfs stores additional keys (i.e., all keys other than the \"identity\" key) in the keystore. You can list these keys with `ipfs key`.\n\nCurrently, the keystore stores keys as regular files, named after the key itself. Unfortunately, filename restrictions and case-insensitivity are platform specific. To avoid platform specific issues, we're base32 encoding all key names and renaming all keys on-disk.\n\n#### Windows\n\nAs usual, this release contains several Windows specific fixes and improvements:\n\n* Double-clicking `ipfs.exe` will now start the daemon inside a console window.\n* `ipfs add -r` now correctly recognizes and ignores hidden files on Windows.\n* The default datastore, flatfs, now takes extra precautions to avoid \"file in use\" errors caused by both go-ipfs and external programs like anti-viruses. If you've ever seen go-ipfs print out an \"access denied\" or \"file in use\" error on Windows, this issue was likely the cause.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - fix: non-blocking peerlog logging ([ipfs/go-ipfs#7232](https://github.com/ipfs/go-ipfs/pull/7232))\n - doc: update go-ipfs docs for 0.5.0 release ([ipfs/go-ipfs#7229](https://github.com/ipfs/go-ipfs/pull/7229))\n - Add additional documentation links to the new issue screen ([ipfs/go-ipfs#7226](https://github.com/ipfs/go-ipfs/pull/7226))\n - docs: note that ShardingEnabled is a global flag ([ipfs/go-ipfs#7218](https://github.com/ipfs/go-ipfs/pull/7218))\n - update log helptext to match actual levels ([ipfs/go-ipfs#7199](https://github.com/ipfs/go-ipfs/pull/7199))\n - Chore/harden car test a bit harder ([ipfs/go-ipfs#7209](https://github.com/ipfs/go-ipfs/pull/7209))\n - fix: fix duplicate block issue in bitswap ([ipfs/go-ipfs#7202](https://github.com/ipfs/go-ipfs/pull/7202))\n - feat: update docker image ([ipfs/go-ipfs#7191](https://github.com/ipfs/go-ipfs/pull/7191))\n - feat: update dir index ([ipfs/go-ipfs#7192](https://github.com/ipfs/go-ipfs/pull/7192))\n - fix: update the dht to fix yggdrasil ([ipfs/go-ipfs#7186](https://github.com/ipfs/go-ipfs/pull/7186))\n - Choose architecture when download tini into docker container ([ipfs/go-ipfs#7187](https://github.com/ipfs/go-ipfs/pull/7187))\n - Fix typos and cleanup ([ipfs/go-ipfs#7181](https://github.com/ipfs/go-ipfs/pull/7181))\n - Fix typos ([ipfs/go-ipfs#7180](https://github.com/ipfs/go-ipfs/pull/7180))\n - feat: webui 2.7.5 ([ipfs/go-ipfs#7176](https://github.com/ipfs/go-ipfs/pull/7176))\n - integration test for the dual dht ([ipfs/go-ipfs#7151](https://github.com/ipfs/go-ipfs/pull/7151))\n - fix: subdomain redirect for dir CIDs ([ipfs/go-ipfs#7165](https://github.com/ipfs/go-ipfs/pull/7165))\n - add autonat config options ([ipfs/go-ipfs#7162](https://github.com/ipfs/go-ipfs/pull/7162))\n - docs: fix link to version.go ([ipfs/go-ipfs#7157](https://github.com/ipfs/go-ipfs/pull/7157))\n - feat: webui v2.7.4 ([ipfs/go-ipfs#7159](https://github.com/ipfs/go-ipfs/pull/7159))\n - fix the typo in the serveHTTPApi ([ipfs/go-ipfs#7156](https://github.com/ipfs/go-ipfs/pull/7156))\n - test(sharness): improve CAR tests to remove some potential races ([ipfs/go-ipfs#7154](https://github.com/ipfs/go-ipfs/pull/7154))\n - feat: introduce the dual WAN/LAN DHT ([ipfs/go-ipfs#7127](https://github.com/ipfs/go-ipfs/pull/7127))\n - fix: invalidate cache on failed publish ([ipfs/go-ipfs#7152](https://github.com/ipfs/go-ipfs/pull/7152))\n - Temporarily disable gc-race test ([ipfs/go-ipfs#7148](https://github.com/ipfs/go-ipfs/pull/7148))\n - Beef up and harden import/export tests ([ipfs/go-ipfs#7140](https://github.com/ipfs/go-ipfs/pull/7140))\n - Filter dials to blocked subnets, even when using DNS. ([ipfs/go-ipfs#6996](https://github.com/ipfs/go-ipfs/pull/6996))\n - Dag export command, complete ([ipfs/go-ipfs#7036](https://github.com/ipfs/go-ipfs/pull/7036))\n - Adding Fission to IPFS early testers page ([ipfs/go-ipfs#7119](https://github.com/ipfs/go-ipfs/pull/7119))\n - feat: bump version ([ipfs/go-ipfs#7110](https://github.com/ipfs/go-ipfs/pull/7110))\n - feat: initial update to the changelog for 0.5.0 ([ipfs/go-ipfs#6977](https://github.com/ipfs/go-ipfs/pull/6977))\n - feat(dht): update to cypress DHT in backwards compatibility mode ([ipfs/go-ipfs#7103](https://github.com/ipfs/go-ipfs/pull/7103))\n - update bash completion for `ipfs add` ([ipfs/go-ipfs#7102](https://github.com/ipfs/go-ipfs/pull/7102))\n - HTTP API: Only allow POST requests (plus OPTIONS) ([ipfs/go-ipfs#7097](https://github.com/ipfs/go-ipfs/pull/7097))\n - Revert last change (the default is now printed twice) ([ipfs/go-ipfs#7098](https://github.com/ipfs/go-ipfs/pull/7098))\n - Fix #4996: Improve help text for \"ipfs files cp\" ([ipfs/go-ipfs#7069](https://github.com/ipfs/go-ipfs/pull/7069))\n - changed brew to brew cask ([ipfs/go-ipfs#7072](https://github.com/ipfs/go-ipfs/pull/7072))\n - fix: remove internal relay discovery ([ipfs/go-ipfs#7064](https://github.com/ipfs/go-ipfs/pull/7064))\n - docs/experimental-features.md: typo ([ipfs/go-ipfs#7062](https://github.com/ipfs/go-ipfs/pull/7062))\n - fix: get rid of shutdown errors ([ipfs/go-ipfs#7058](https://github.com/ipfs/go-ipfs/pull/7058))\n - feat: tls by default ([ipfs/go-ipfs#7055](https://github.com/ipfs/go-ipfs/pull/7055))\n - fix: downgrade to go 1.13 ([ipfs/go-ipfs#7054](https://github.com/ipfs/go-ipfs/pull/7054))\n - Keystore: minor maintenance ([ipfs/go-ipfs#7043](https://github.com/ipfs/go-ipfs/pull/7043))\n - fix(keystore): avoid racy filesystem access ([ipfs/go-ipfs#6999](https://github.com/ipfs/go-ipfs/pull/6999))\n - Forgotten go-fmt ([ipfs/go-ipfs#7030](https://github.com/ipfs/go-ipfs/pull/7030))\n - feat: update go-libp2p & go-bitswap ([ipfs/go-ipfs#7028](https://github.com/ipfs/go-ipfs/pull/7028))\n - Introducing EncodedFSKeystore with base32 encoding (#5947) ([ipfs/go-ipfs#6955](https://github.com/ipfs/go-ipfs/pull/6955))\n - feat: improve key lookup ([ipfs/go-ipfs#7023](https://github.com/ipfs/go-ipfs/pull/7023))\n - feat(file-ignore): add ignore opts to add cmd ([ipfs/go-ipfs#7017](https://github.com/ipfs/go-ipfs/pull/7017))\n - feat: gateway subdomains + http proxy mode ([ipfs/go-ipfs#6096](https://github.com/ipfs/go-ipfs/pull/6096))\n - Chore/sharness fixes 2019 03 16 ([ipfs/go-ipfs#6997](https://github.com/ipfs/go-ipfs/pull/6997))\n - Support pipes when named on the cli explicitly ([ipfs/go-ipfs#6998](https://github.com/ipfs/go-ipfs/pull/6998))\n - Fix a typo ([ipfs/go-ipfs#7000](https://github.com/ipfs/go-ipfs/pull/7000))\n - fix: revert changes to the user agent ([ipfs/go-ipfs#6993](https://github.com/ipfs/go-ipfs/pull/6993))\n - feat(peerlog): log protocols/versions ([ipfs/go-ipfs#6972](https://github.com/ipfs/go-ipfs/pull/6972))\n - feat: docker build and tag from ci ([ipfs/go-ipfs#6949](https://github.com/ipfs/go-ipfs/pull/6949))\n - cmd: ipfs handle GUI environment on Windows ([ipfs/go-ipfs#6646](https://github.com/ipfs/go-ipfs/pull/6646))\n - Chore/macos sharness fixes ([ipfs/go-ipfs#6988](https://github.com/ipfs/go-ipfs/pull/6988))\n - Update to go-libp2p 0.6.0 ([ipfs/go-ipfs#6914](https://github.com/ipfs/go-ipfs/pull/6914))\n - mount: switch over to the CoreAPI ([ipfs/go-ipfs#6602](https://github.com/ipfs/go-ipfs/pull/6602))\n - doc(commands): document that `dht put` takes a file ([ipfs/go-ipfs#6960](https://github.com/ipfs/go-ipfs/pull/6960))\n - docs: update licence info in README ([ipfs/go-ipfs#6942](https://github.com/ipfs/go-ipfs/pull/6942))\n - docs: fix example for files.write ([ipfs/go-ipfs#6943](https://github.com/ipfs/go-ipfs/pull/6943))\n - feat(graphsync): mount the graphsync libp2p protocol ([ipfs/go-ipfs#6892](https://github.com/ipfs/go-ipfs/pull/6892))\n - feat: update go in docker container ([ipfs/go-ipfs#6933](https://github.com/ipfs/go-ipfs/pull/6933))\n - remove expired GPG key from README ([ipfs/go-ipfs#6931](https://github.com/ipfs/go-ipfs/pull/6931))\n - test(sharness): test our tests ([ipfs/go-ipfs#6908](https://github.com/ipfs/go-ipfs/pull/6908))\n - fix: broken interop tests ([ipfs/go-ipfs#6899](https://github.com/ipfs/go-ipfs/pull/6899))\n - feat: pass IPFS_PLUGINS to docker build ([ipfs/go-ipfs#6898](https://github.com/ipfs/go-ipfs/pull/6898))\n - doc(add): document hash stability ([ipfs/go-ipfs#6891](https://github.com/ipfs/go-ipfs/pull/6891))\n - feat: add peerlog plugin ([ipfs/go-ipfs#6887](https://github.com/ipfs/go-ipfs/pull/6887))\n - doc(plugin): document internal plugins ([ipfs/go-ipfs#6888](https://github.com/ipfs/go-ipfs/pull/6888))\n - Fix #6878: Improve MFS Cli documentation ([ipfs/go-ipfs#6882](https://github.com/ipfs/go-ipfs/pull/6882))\n - Update the license distributed with dist builds to the dual one ([ipfs/go-ipfs#6879](https://github.com/ipfs/go-ipfs/pull/6879))\n - doc: add license URLs so go's doc service can detect our license ([ipfs/go-ipfs#6874](https://github.com/ipfs/go-ipfs/pull/6874))\n - doc: rename COPYRIGHT to LICENSE ([ipfs/go-ipfs#6873](https://github.com/ipfs/go-ipfs/pull/6873))\n - fix: fix id addr format ([ipfs/go-ipfs#6872](https://github.com/ipfs/go-ipfs/pull/6872))\n - Help text update for 'ipfs key gen' ([ipfs/go-ipfs#6867](https://github.com/ipfs/go-ipfs/pull/6867))\n - fix: make rsa the default key type ([ipfs/go-ipfs#6864](https://github.com/ipfs/go-ipfs/pull/6864))\n - doc(config): cleanup ([ipfs/go-ipfs#6855](https://github.com/ipfs/go-ipfs/pull/6855))\n - Allow building non-amd64 Docker images ([ipfs/go-ipfs#6854](https://github.com/ipfs/go-ipfs/pull/6854))\n - doc(release): add Charity Engine to the early testers programme ([ipfs/go-ipfs#6850](https://github.com/ipfs/go-ipfs/pull/6850))\n - fix: fix a potential out of bounds issue in fuse ([ipfs/go-ipfs#6847](https://github.com/ipfs/go-ipfs/pull/6847))\n - fix(build): instruct users to use GOTAGS, not GOFLAGS ([ipfs/go-ipfs#6843](https://github.com/ipfs/go-ipfs/pull/6843))\n - doc(release): document how RCs should be communicated ([ipfs/go-ipfs#6845](https://github.com/ipfs/go-ipfs/pull/6845))\n - doc(release): move WebUI from manual tests to automated tests section ([ipfs/go-ipfs#6838](https://github.com/ipfs/go-ipfs/pull/6838))\n - test(sharness): fix typo ([ipfs/go-ipfs#6835](https://github.com/ipfs/go-ipfs/pull/6835))\n - test: E2E tests against ipfs-webui HEAD ([ipfs/go-ipfs#6825](https://github.com/ipfs/go-ipfs/pull/6825))\n - mkreleaslog: improve edge-cases ([ipfs/go-ipfs#6833](https://github.com/ipfs/go-ipfs/pull/6833))\n - fix: don't fail to collect profiles if no ipfs bin ([ipfs/go-ipfs#6829](https://github.com/ipfs/go-ipfs/pull/6829))\n - update dockerfile and use openssl ([ipfs/go-ipfs#6828](https://github.com/ipfs/go-ipfs/pull/6828))\n - docs: define Gateway.PathPrefixes ([ipfs/go-ipfs#6826](https://github.com/ipfs/go-ipfs/pull/6826))\n - fix(badgerds): turn off sync writes by default ([ipfs/go-ipfs#6819](https://github.com/ipfs/go-ipfs/pull/6819))\n - gateway cleanups ([ipfs/go-ipfs#6820](https://github.com/ipfs/go-ipfs/pull/6820))\n - make it possible to change the codec with the `ipfs cid` subcommand ([ipfs/go-ipfs#6817](https://github.com/ipfs/go-ipfs/pull/6817))\n - improve gateway symlink handling ([ipfs/go-ipfs#6680](https://github.com/ipfs/go-ipfs/pull/6680))\n - Inclusion of the presence of the go-ipfs package in Solus ([ipfs/go-ipfs#6809](https://github.com/ipfs/go-ipfs/pull/6809))\n - Fix Typos ([ipfs/go-ipfs#6807](https://github.com/ipfs/go-ipfs/pull/6807))\n - Sharness macos no brainer fixes ([ipfs/go-ipfs#6805](https://github.com/ipfs/go-ipfs/pull/6805))\n - Support Asynchronous Datastores ([ipfs/go-ipfs#6785](https://github.com/ipfs/go-ipfs/pull/6785))\n - update documentation for /ipfs -> /p2p multiaddr switch ([ipfs/go-ipfs#6538](https://github.com/ipfs/go-ipfs/pull/6538))\n - IPNS over PubSub as an Independent Transport ([ipfs/go-ipfs#6758](https://github.com/ipfs/go-ipfs/pull/6758))\n - docs: add information on how to enable experiments ([ipfs/go-ipfs#6792](https://github.com/ipfs/go-ipfs/pull/6792))\n - Change Reporter to BandwidthCounter in IpfsNode ([ipfs/go-ipfs#6793](https://github.com/ipfs/go-ipfs/pull/6793))\n - update go-datastore ([ipfs/go-ipfs#6791](https://github.com/ipfs/go-ipfs/pull/6791))\n - go fmt: go-ipfs-as-a-library ([ipfs/go-ipfs#6784](https://github.com/ipfs/go-ipfs/pull/6784))\n - feat: web ui 2.7.2 ([ipfs/go-ipfs#6778](https://github.com/ipfs/go-ipfs/pull/6778))\n - extract the pinner to go-ipfs-pinner and dagutils into go-merkledag ([ipfs/go-ipfs#6771](https://github.com/ipfs/go-ipfs/pull/6771))\n - fix #2203: omit the charset attribute when Content-Type is text/html ([ipfs/go-ipfs#6743](https://github.com/ipfs/go-ipfs/pull/6743))\n - Pin ls traverses all indirect pins ([ipfs/go-ipfs#6705](https://github.com/ipfs/go-ipfs/pull/6705))\n - fix: ignore nonexistent when force rm ([ipfs/go-ipfs#6773](https://github.com/ipfs/go-ipfs/pull/6773))\n - introduce IpfsNode Plugin ([ipfs/go-ipfs#6719](https://github.com/ipfs/go-ipfs/pull/6719))\n - improve documentation and fix dht put bug ([ipfs/go-ipfs#6750](https://github.com/ipfs/go-ipfs/pull/6750))\n - Adding alias for `ipfs repo stat`. ([ipfs/go-ipfs#6769](https://github.com/ipfs/go-ipfs/pull/6769))\n - doc(gateway): document dnslink ([ipfs/go-ipfs#6767](https://github.com/ipfs/go-ipfs/pull/6767))\n - pin: add context and error return to most of the Pinner functions ([ipfs/go-ipfs#6715](https://github.com/ipfs/go-ipfs/pull/6715))\n - feat: web ui 2.7.1 ([ipfs/go-ipfs#6762](https://github.com/ipfs/go-ipfs/pull/6762))\n - doc(README): document requirements for cross-compiling with OpenSSL support ([ipfs/go-ipfs#6738](https://github.com/ipfs/go-ipfs/pull/6738))\n - feat: web ui 2.6.0 ([ipfs/go-ipfs#6740](https://github.com/ipfs/go-ipfs/pull/6740))\n - Add high-level go-ipfs architecture diagram ([ipfs/go-ipfs#6727](https://github.com/ipfs/go-ipfs/pull/6727))\n - docs: remove extra ) on the example README ([ipfs/go-ipfs#6733](https://github.com/ipfs/go-ipfs/pull/6733))\n - update maintainer label ([ipfs/go-ipfs#6735](https://github.com/ipfs/go-ipfs/pull/6735))\n - ipfs namespace is now being provided to Prometheus ([ipfs/go-ipfs#6643](https://github.com/ipfs/go-ipfs/pull/6643))\n - feat: web ui 2.5.8 ([ipfs/go-ipfs#6718](https://github.com/ipfs/go-ipfs/pull/6718))\n - docs: add connmgr to config.md toc ([ipfs/go-ipfs#6712](https://github.com/ipfs/go-ipfs/pull/6712))\n - feat: web ui 2.5.7 ([ipfs/go-ipfs#6707](https://github.com/ipfs/go-ipfs/pull/6707))\n - README: improve build documentation ([ipfs/go-ipfs#6706](https://github.com/ipfs/go-ipfs/pull/6706))\n - Introduce buzhash chunker ([ipfs/go-ipfs#6701](https://github.com/ipfs/go-ipfs/pull/6701))\n - Pinning interop: Pin ls returns appropriate zero value ([ipfs/go-ipfs#6685](https://github.com/ipfs/go-ipfs/pull/6685))\n - fix(resolve): correctly handle .eth domains ([ipfs/go-ipfs#6700](https://github.com/ipfs/go-ipfs/pull/6700))\n - Update README.md ([ipfs/go-ipfs#6697](https://github.com/ipfs/go-ipfs/pull/6697))\n - daemon: support unix domain sockets for the API/gateway ([ipfs/go-ipfs#6678](https://github.com/ipfs/go-ipfs/pull/6678))\n - docs: guide users to the right locations for questions ([ipfs/go-ipfs#6691](https://github.com/ipfs/go-ipfs/pull/6691))\n - docs: readme improvements ([ipfs/go-ipfs#6693](https://github.com/ipfs/go-ipfs/pull/6693))\n - docs: link remaining docs available, guide people to the right locations ([ipfs/go-ipfs#6694](https://github.com/ipfs/go-ipfs/pull/6694))\n - docs: fix broken url ([ipfs/go-ipfs#6692](https://github.com/ipfs/go-ipfs/pull/6692))\n - add systemd support ([ipfs/go-ipfs#6675](https://github.com/ipfs/go-ipfs/pull/6675))\n - feat: add ipfs version info to prometheus metrics ([ipfs/go-ipfs#6688](https://github.com/ipfs/go-ipfs/pull/6688))\n - Fix typo ([ipfs/go-ipfs#6686](https://github.com/ipfs/go-ipfs/pull/6686))\n - github: migrate actions ([ipfs/go-ipfs#6681](https://github.com/ipfs/go-ipfs/pull/6681))\n - Add bridged chats ([ipfs/go-ipfs#6653](https://github.com/ipfs/go-ipfs/pull/6653))\n - doc(config): improve DisableNatPortMap documentation ([ipfs/go-ipfs#6655](https://github.com/ipfs/go-ipfs/pull/6655))\n - plugins: support Close() for Tracer plugins as well ([ipfs/go-ipfs#6672](https://github.com/ipfs/go-ipfs/pull/6672))\n - fix: make collect-profiles.sh work on mac ([ipfs/go-ipfs#6673](https://github.com/ipfs/go-ipfs/pull/6673))\n - namesys(test): test TTL on publish ([ipfs/go-ipfs#6671](https://github.com/ipfs/go-ipfs/pull/6671))\n - discovery: improve mdns warnings ([ipfs/go-ipfs#6665](https://github.com/ipfs/go-ipfs/pull/6665))\n - feat: web ui 2.5.4 ([ipfs/go-ipfs#6664](https://github.com/ipfs/go-ipfs/pull/6664))\n - cmds(help): fix swarm filter add/rm help text ([ipfs/go-ipfs#6654](https://github.com/ipfs/go-ipfs/pull/6654))\n - feat: webui 2.5.3 ([ipfs/go-ipfs#6638](https://github.com/ipfs/go-ipfs/pull/6638))\n - feat: web ui 2.5.1 ([ipfs/go-ipfs#6630](https://github.com/ipfs/go-ipfs/pull/6630))\n - docs: add multiple gateway and api addrs ([ipfs/go-ipfs#6631](https://github.com/ipfs/go-ipfs/pull/6631))\n - doc: add post-release checklist ([ipfs/go-ipfs#6625](https://github.com/ipfs/go-ipfs/pull/6625))\n - docs: add ship date and next release issue opening time ([ipfs/go-ipfs#6620](https://github.com/ipfs/go-ipfs/pull/6620))\n - docker: libdl dependency ([ipfs/go-ipfs#6624](https://github.com/ipfs/go-ipfs/pull/6624))\n - docs: improvements to the release doc ([ipfs/go-ipfs#6616](https://github.com/ipfs/go-ipfs/pull/6616))\n - plugins: add support for plugin configs ([ipfs/go-ipfs#6613](https://github.com/ipfs/go-ipfs/pull/6613))\n - Update README.md ([ipfs/go-ipfs#6615](https://github.com/ipfs/go-ipfs/pull/6615))\n - doc: remove gmake instructions ([ipfs/go-ipfs#6614](https://github.com/ipfs/go-ipfs/pull/6614))\n - feat: add ability to use existing config during init ([ipfs/go-ipfs#6489](https://github.com/ipfs/go-ipfs/pull/6489))\n - doc: expand and cleanup badger documentation ([ipfs/go-ipfs#6611](https://github.com/ipfs/go-ipfs/pull/6611))\n - feat: improve plugin preload logic ([ipfs/go-ipfs#6576](https://github.com/ipfs/go-ipfs/pull/6576))\n - version: don't print 'VERSION-' if no commit is specified ([ipfs/go-ipfs#6609](https://github.com/ipfs/go-ipfs/pull/6609))\n - Update go-libp2p, fix tests with weak RSA keys ([ipfs/go-ipfs#6555](https://github.com/ipfs/go-ipfs/pull/6555))\n - cmds/refs: fix ipfs refs for sharded directories ([ipfs/go-ipfs#6601](https://github.com/ipfs/go-ipfs/pull/6601))\n - fix: spammy mock when testing ([ipfs/go-ipfs#6583](https://github.com/ipfs/go-ipfs/pull/6583))\n - docker: update the docker image ([ipfs/go-ipfs#6582](https://github.com/ipfs/go-ipfs/pull/6582))\n - add release process graphic ([ipfs/go-ipfs#6568](https://github.com/ipfs/go-ipfs/pull/6568))\n - feat: web ui 2.5.0 ([ipfs/go-ipfs#6566](https://github.com/ipfs/go-ipfs/pull/6566))\n - Add swarm key variables to container daemon ([ipfs/go-ipfs#6554](https://github.com/ipfs/go-ipfs/pull/6554))\n - doc: update the release template ([ipfs/go-ipfs#6561](https://github.com/ipfs/go-ipfs/pull/6561))\n - merge changelog and bump version ([ipfs/go-ipfs#6559](https://github.com/ipfs/go-ipfs/pull/6559))\n - require GNU make ([ipfs/go-ipfs#6551](https://github.com/ipfs/go-ipfs/pull/6551))\n - tweak the release process ([ipfs/go-ipfs#6553](https://github.com/ipfs/go-ipfs/pull/6553))\n - Allow resolution of .eth names via .eth.link ([ipfs/go-ipfs#6448](https://github.com/ipfs/go-ipfs/pull/6448))\n - README: update minimum system requirements and recommend OpenSSL ([ipfs/go-ipfs#6543](https://github.com/ipfs/go-ipfs/pull/6543))\n - fix and improve the writable gateway ([ipfs/go-ipfs#6539](https://github.com/ipfs/go-ipfs/pull/6539))\n - feat: add install instructions for external commands ([ipfs/go-ipfs#6541](https://github.com/ipfs/go-ipfs/pull/6541))\n - fix: slightly faster gc ([ipfs/go-ipfs#6505](https://github.com/ipfs/go-ipfs/pull/6505))\n - fix {net,open}bsd build by disabling fuse on openbsd ([ipfs/go-ipfs#6535](https://github.com/ipfs/go-ipfs/pull/6535))\n - mk: handle stripping paths when GOPATH contains whitespace ([ipfs/go-ipfs#6536](https://github.com/ipfs/go-ipfs/pull/6536))\n - make gossipsub the default routing protocol for pubsub ([ipfs/go-ipfs#6512](https://github.com/ipfs/go-ipfs/pull/6512))\n - doc: align the early testers program description with its goal ([ipfs/go-ipfs#6529](https://github.com/ipfs/go-ipfs/pull/6529))\n - feat: add --long as alias for -l in files.ls ([ipfs/go-ipfs#6528](https://github.com/ipfs/go-ipfs/pull/6528))\n - switch to new merkledag walk functions ([ipfs/go-ipfs#6499](https://github.com/ipfs/go-ipfs/pull/6499))\n - readme: fix CI badge ([ipfs/go-ipfs#6521](https://github.com/ipfs/go-ipfs/pull/6521))\n - Adds Siderus in early testers ([ipfs/go-ipfs#6517](https://github.com/ipfs/go-ipfs/pull/6517))\n - Extract Filestore ([ipfs/go-ipfs#6511](https://github.com/ipfs/go-ipfs/pull/6511))\n - readme: fix scoop bucket command error ([ipfs/go-ipfs#6510](https://github.com/ipfs/go-ipfs/pull/6510))\n - sharness: test pin ls stream ([ipfs/go-ipfs#6504](https://github.com/ipfs/go-ipfs/pull/6504))\n - Improve pin/update description ([ipfs/go-ipfs#6501](https://github.com/ipfs/go-ipfs/pull/6501))\n - pin cmd: stream recursive pins ([ipfs/go-ipfs#6493](https://github.com/ipfs/go-ipfs/pull/6493))\n - Document the AddrFilters option ([ipfs/go-ipfs#6459](https://github.com/ipfs/go-ipfs/pull/6459))\n - feat: make it easier to load custom plugins ([ipfs/go-ipfs#6474](https://github.com/ipfs/go-ipfs/pull/6474))\n - document the debug script ([ipfs/go-ipfs#6486](https://github.com/ipfs/go-ipfs/pull/6486))\n - Extract provider module to `go-ipfs-provider` ([ipfs/go-ipfs#6421](https://github.com/ipfs/go-ipfs/pull/6421))\n - ignore stale API files and deprecate ipfs repo fsck ([ipfs/go-ipfs#6478](https://github.com/ipfs/go-ipfs/pull/6478))\n - Fix node construction queue error ([ipfs/go-ipfs#6480](https://github.com/ipfs/go-ipfs/pull/6480))\n - Update the required go version in the README ([ipfs/go-ipfs#6462](https://github.com/ipfs/go-ipfs/pull/6462))\n - gitmodules: use https so we don't need an ssh key ([ipfs/go-ipfs#6450](https://github.com/ipfs/go-ipfs/pull/6450))\n - doc: add another Windows package to README ([ipfs/go-ipfs#6440](https://github.com/ipfs/go-ipfs/pull/6440))\n - Close started plugins when one of them fails to start. ([ipfs/go-ipfs#6438](https://github.com/ipfs/go-ipfs/pull/6438))\n - Load plugins on darwin/macOS ([ipfs/go-ipfs#6439](https://github.com/ipfs/go-ipfs/pull/6439))\n - assets: move away from gx ([ipfs/go-ipfs#6414](https://github.com/ipfs/go-ipfs/pull/6414))\n - Fix a typo ([ipfs/go-ipfs#6432](https://github.com/ipfs/go-ipfs/pull/6432))\n - docs: fix install guide link ([ipfs/go-ipfs#6423](https://github.com/ipfs/go-ipfs/pull/6423))\n - Deps: update go-libp2p-http to its new libp2p location ([ipfs/go-ipfs#6422](https://github.com/ipfs/go-ipfs/pull/6422))\n - install.sh: Fix wrong destination path for ipfs binary ([ipfs/go-ipfs#6424](https://github.com/ipfs/go-ipfs/pull/6424))\n - build: strip GOPATH from build paths ([ipfs/go-ipfs#6412](https://github.com/ipfs/go-ipfs/pull/6412))\n - libp2p: moves discovery after host listen ([ipfs/go-ipfs#6415](https://github.com/ipfs/go-ipfs/pull/6415))\n - remove mentions of gx from windows build docs ([ipfs/go-ipfs#6413](https://github.com/ipfs/go-ipfs/pull/6413))\n - build: use protoc-gen-* from gomod ([ipfs/go-ipfs#6411](https://github.com/ipfs/go-ipfs/pull/6411))\n - add unixfs get metric ([ipfs/go-ipfs#6406](https://github.com/ipfs/go-ipfs/pull/6406))\n - Run JS interop in CircleCI ([ipfs/go-ipfs#6409](https://github.com/ipfs/go-ipfs/pull/6409))\n - Usage of context helper in Blockstore provider ([ipfs/go-ipfs#6399](https://github.com/ipfs/go-ipfs/pull/6399))\n - docs: default value for HashOnRead is false ([ipfs/go-ipfs#6401](https://github.com/ipfs/go-ipfs/pull/6401))\n - block cmd: allow adding multiple blocks at once ([ipfs/go-ipfs#6331](https://github.com/ipfs/go-ipfs/pull/6331))\n - Remove Repo from routing fx provider parameter ([ipfs/go-ipfs#6395](https://github.com/ipfs/go-ipfs/pull/6395))\n - migrate to go-libp2p-core. ([ipfs/go-ipfs#6384](https://github.com/ipfs/go-ipfs/pull/6384))\n - feat: update Web UI to v2.4.6 ([ipfs/go-ipfs#6392](https://github.com/ipfs/go-ipfs/pull/6392))\n - Introduce first strategic provider: do nothing ([ipfs/go-ipfs#6292](https://github.com/ipfs/go-ipfs/pull/6292))\n- github.com/ipfs/go-bitswap (v0.0.8-e37498cf10d6 -> v0.2.13):\n - refactor: remove WantManager ([ipfs/go-bitswap#374](https://github.com/ipfs/go-bitswap/pull/374))\n - Send CANCELs when session context is canceled ([ipfs/go-bitswap#375](https://github.com/ipfs/go-bitswap/pull/375))\n - refactor: remove unused code ([ipfs/go-bitswap#373](https://github.com/ipfs/go-bitswap/pull/373))\n - Change timing for DONT_HAVE timeouts to be more conservative ([ipfs/go-bitswap#371](https://github.com/ipfs/go-bitswap/pull/371))\n - fix: avoid calling ctx.SetDeadline() every time we send a message ([ipfs/go-bitswap#369](https://github.com/ipfs/go-bitswap/pull/369))\n - feat: optimize entry sorting in MessageQueue ([ipfs/go-bitswap#356](https://github.com/ipfs/go-bitswap/pull/356))\n - Move connection management into networking layer ([ipfs/go-bitswap#351](https://github.com/ipfs/go-bitswap/pull/351))\n - refactor: simplify messageQueue onSent ([ipfs/go-bitswap#349](https://github.com/ipfs/go-bitswap/pull/349))\n - feat: prioritize more important wants ([ipfs/go-bitswap#346](https://github.com/ipfs/go-bitswap/pull/346))\n - fix: in message queue only send cancel if want was sent ([ipfs/go-bitswap#345](https://github.com/ipfs/go-bitswap/pull/345))\n - fix: ensure wantlist gauge gets decremented on disconnect ([ipfs/go-bitswap#332](https://github.com/ipfs/go-bitswap/pull/332))\n - avoid copying messages and improve logging ([ipfs/go-bitswap#326](https://github.com/ipfs/go-bitswap/pull/326))\n - fix: log unexpected condition in peerWantManager.prepareSendWants() ([ipfs/go-bitswap#325](https://github.com/ipfs/go-bitswap/pull/325))\n - wait for sessionWantSender to shutdown before completing session shutdown ([ipfs/go-bitswap#317](https://github.com/ipfs/go-bitswap/pull/317))\n - Perf/message queue ([ipfs/go-bitswap#307](https://github.com/ipfs/go-bitswap/pull/307))\n - feat: add a custom CID type ([ipfs/go-bitswap#308](https://github.com/ipfs/go-bitswap/pull/308))\n - feat: expose the full wantlist through GetWantlist ([ipfs/go-bitswap#300](https://github.com/ipfs/go-bitswap/pull/300))\n - Clean up logs ([ipfs/go-bitswap#299](https://github.com/ipfs/go-bitswap/pull/299))\n - Fix order of session broadcast wants ([ipfs/go-bitswap#291](https://github.com/ipfs/go-bitswap/pull/291))\n - fix flaky TestRateLimitingRequests ([ipfs/go-bitswap#296](https://github.com/ipfs/go-bitswap/pull/296))\n - fix flaky TestDontHaveTimeoutMgrTimeout ([ipfs/go-bitswap#293](https://github.com/ipfs/go-bitswap/pull/293))\n - fix: re-export testinstance/testnet ([ipfs/go-bitswap#289](https://github.com/ipfs/go-bitswap/pull/289))\n - Simulate DONT_HAVE when peer doesn't respond to want-block (new peers) ([ipfs/go-bitswap#284](https://github.com/ipfs/go-bitswap/pull/284))\n - Be less aggressive when pruning peers from session ([ipfs/go-bitswap#276](https://github.com/ipfs/go-bitswap/pull/276))\n - fix: races in tests ([ipfs/go-bitswap#279](https://github.com/ipfs/go-bitswap/pull/279))\n - Refactor: simplify session peer management ([ipfs/go-bitswap#275](https://github.com/ipfs/go-bitswap/pull/275))\n - Prune peers that send too many consecutive DONT_HAVEs ([ipfs/go-bitswap#261](https://github.com/ipfs/go-bitswap/pull/261))\n - feat: debounce wants manually ([ipfs/go-bitswap#255](https://github.com/ipfs/go-bitswap/pull/255))\n - Fix bug with signaling peer availability to sessions ([ipfs/go-bitswap#247](https://github.com/ipfs/go-bitswap/pull/247))\n - feat: move internals to an internal package ([ipfs/go-bitswap#242](https://github.com/ipfs/go-bitswap/pull/242))\n - PoC of Bitswap protocol extensions implementation ([ipfs/go-bitswap#189](https://github.com/ipfs/go-bitswap/pull/189))\n - fix: abort when the context is canceled while getting blocks ([ipfs/go-bitswap#240](https://github.com/ipfs/go-bitswap/pull/240))\n - Add bridged chats ([ipfs/go-bitswap#198](https://github.com/ipfs/go-bitswap/pull/198))\n - reduce session contention ([ipfs/go-bitswap#188](https://github.com/ipfs/go-bitswap/pull/188))\n - Fix: don't ignore received blocks for pending wants ([ipfs/go-bitswap#174](https://github.com/ipfs/go-bitswap/pull/174))\n - Test: fix flakey session peer manager tests ([ipfs/go-bitswap#185](https://github.com/ipfs/go-bitswap/pull/185))\n - Refactor: use global pubsub notifier ([ipfs/go-bitswap#177](https://github.com/ipfs/go-bitswap/pull/177))\n - network: Allow specifying protocol prefix ([ipfs/go-bitswap#171](https://github.com/ipfs/go-bitswap/pull/171))\n - fix: memory leak in latency tracker on timeout after cancel ([ipfs/go-bitswap#164](https://github.com/ipfs/go-bitswap/pull/164))\n - Fix typo ([ipfs/go-bitswap#158](https://github.com/ipfs/go-bitswap/pull/158))\n - Feat: Track Session Peer Latency More Accurately ([ipfs/go-bitswap#149](https://github.com/ipfs/go-bitswap/pull/149))\n - ci(circleci): add benchmark comparisons ([ipfs/go-bitswap#147](https://github.com/ipfs/go-bitswap/pull/147))\n - aggressively free memory ([ipfs/go-bitswap#143](https://github.com/ipfs/go-bitswap/pull/143))\n - Enhanced logging for bitswap ([ipfs/go-bitswap#137](https://github.com/ipfs/go-bitswap/pull/137))\n - fix: rand.Intn(0) panics ([ipfs/go-bitswap#144](https://github.com/ipfs/go-bitswap/pull/144))\n - fix some naming nits and broadcast on search ([ipfs/go-bitswap#139](https://github.com/ipfs/go-bitswap/pull/139))\n - feat(sessions): add rebroadcasting, search backoff ([ipfs/go-bitswap#133](https://github.com/ipfs/go-bitswap/pull/133))\n - testutil: fix block generator ([ipfs/go-bitswap#135](https://github.com/ipfs/go-bitswap/pull/135))\n - migrate to go-libp2p-core. ([ipfs/go-bitswap#132](https://github.com/ipfs/go-bitswap/pull/132))\n- github.com/ipfs/go-blockservice (v0.0.3 -> v0.1.3):\n - fix ci badge and lints ([ipfs/go-blockservice#52](https://github.com/ipfs/go-blockservice/pull/52))\n - demote warning to debug log ([ipfs/go-blockservice#30](https://github.com/ipfs/go-blockservice/pull/30))\n - nil exchange is okay ([ipfs/go-blockservice#29](https://github.com/ipfs/go-blockservice/pull/29))\n - set the session context ([ipfs/go-blockservice#28](https://github.com/ipfs/go-blockservice/pull/28))\n - make blockservice AddBlocks return more quickly ([ipfs/go-blockservice#10](https://github.com/ipfs/go-blockservice/pull/10))\n - feat(session): instantiated sessions lazily ([ipfs/go-blockservice#27](https://github.com/ipfs/go-blockservice/pull/27))\n- github.com/ipfs/go-cid (v0.0.4 -> v0.0.5):\n - fix: enforce minimal encoding ([ipfs/go-cid#99](https://github.com/ipfs/go-cid/pull/99))\n- github.com/ipfs/go-datastore (v0.0.5 -> v0.4.4):\n - Fix test log message about number of values put ([ipfs/go-datastore#150](https://github.com/ipfs/go-datastore/pull/150))\n - test suite: Add ElemCount to control how many elements are added. ([ipfs/go-datastore#151](https://github.com/ipfs/go-datastore/pull/151))\n - fix: avoid filtering by prefix unless necessary ([ipfs/go-datastore#147](https://github.com/ipfs/go-datastore/pull/147))\n - feat: add upper-case keys at a known prefix ([ipfs/go-datastore#148](https://github.com/ipfs/go-datastore/pull/148))\n - test(suite): add a bunch of prefix tests for the new behavior ([ipfs/go-datastore#145](https://github.com/ipfs/go-datastore/pull/145))\n - Only count a key as an ancestor if there is a separator ([ipfs/go-datastore#141](https://github.com/ipfs/go-datastore/pull/141))\n - fix go-check path to use \"gopkg.in/check.v1\" ([ipfs/go-datastore#144](https://github.com/ipfs/go-datastore/pull/144))\n - LogDatastore fulfills the Datastore interface again ([ipfs/go-datastore#142](https://github.com/ipfs/go-datastore/pull/142))\n - Support Asynchronous Writing Datastores ([ipfs/go-datastore#140](https://github.com/ipfs/go-datastore/pull/140))\n - add a Size field to Query's Result ([ipfs/go-datastore#134](https://github.com/ipfs/go-datastore/pull/134))\n - Add clarifying comments on Query#String() ([ipfs/go-datastore#138](https://github.com/ipfs/go-datastore/pull/138))\n - Add a large test suite ([ipfs/go-datastore#136](https://github.com/ipfs/go-datastore/pull/136))\n - doc: add a lead maintainer ([ipfs/go-datastore#135](https://github.com/ipfs/go-datastore/pull/135))\n - feat: make not-found errors discoverable ([ipfs/go-datastore#133](https://github.com/ipfs/go-datastore/pull/133))\n - feat: make delete idempotent ([ipfs/go-datastore#132](https://github.com/ipfs/go-datastore/pull/132))\n - Misc Typo Fixes ([ipfs/go-datastore#131](https://github.com/ipfs/go-datastore/pull/131))\n- github.com/ipfs/go-ds-badger (v0.0.5 -> v0.2.4):\n - fix: verify that the datastore is still open when querying ([ipfs/go-ds-badger#87](https://github.com/ipfs/go-ds-badger/pull/87))\n - feat: switch to file io and shrink tables ([ipfs/go-ds-badger#83](https://github.com/ipfs/go-ds-badger/pull/83))\n - fix: update go-datastore ([ipfs/go-ds-badger#80](https://github.com/ipfs/go-ds-badger/pull/80))\n - update datastore Interface ([ipfs/go-ds-badger#77](https://github.com/ipfs/go-ds-badger/pull/77))\n - query: always return the size ([ipfs/go-ds-badger#78](https://github.com/ipfs/go-ds-badger/pull/78))\n - feat(gc): make it possible to disable GC ([ipfs/go-ds-badger#74](https://github.com/ipfs/go-ds-badger/pull/74))\n - feat(gc): improve periodic GC logic ([ipfs/go-ds-badger#73](https://github.com/ipfs/go-ds-badger/pull/73))\n - periodic GC for badger datastore ([ipfs/go-ds-badger#72](https://github.com/ipfs/go-ds-badger/pull/72))\n - Fix combining query filters, offsets, and limits ([ipfs/go-ds-badger#71](https://github.com/ipfs/go-ds-badger/pull/71))\n - doc: add lead maintainer ([ipfs/go-ds-badger#67](https://github.com/ipfs/go-ds-badger/pull/67))\n- github.com/ipfs/go-ds-flatfs (v0.0.2 -> v0.4.4):\n - move retries lower and retry rename ops ([ipfs/go-ds-flatfs#82](https://github.com/ipfs/go-ds-flatfs/pull/82))\n - cleanup putMany implementation ([ipfs/go-ds-flatfs#80](https://github.com/ipfs/go-ds-flatfs/pull/80))\n - feat: read harder ([ipfs/go-ds-flatfs#78](https://github.com/ipfs/go-ds-flatfs/pull/78))\n - fix: remove temporary files when multiple write operations conflict ([ipfs/go-ds-flatfs#76](https://github.com/ipfs/go-ds-flatfs/pull/76))\n - Windows CI + Fixes ([ipfs/go-ds-flatfs#73](https://github.com/ipfs/go-ds-flatfs/pull/73))\n - fix: close query when finished moving ([ipfs/go-ds-flatfs#74](https://github.com/ipfs/go-ds-flatfs/pull/74))\n - fix: ensure that we close the diskusage file, even if we fail to rename it ([ipfs/go-ds-flatfs#72](https://github.com/ipfs/go-ds-flatfs/pull/72))\n - feat: put all temporary files in the same directory and clean them up ([ipfs/go-ds-flatfs#69](https://github.com/ipfs/go-ds-flatfs/pull/69))\n - fix: only log when we find a file we don't expect ([ipfs/go-ds-flatfs#68](https://github.com/ipfs/go-ds-flatfs/pull/68))\n - Make flatfs robust ([ipfs/go-ds-flatfs#64](https://github.com/ipfs/go-ds-flatfs/pull/64))\n - Update Datastore Interface ([ipfs/go-ds-flatfs#60](https://github.com/ipfs/go-ds-flatfs/pull/60))\n - query: deny ReturnsSizes and ReturnExpirations instead of returning wrong result ([ipfs/go-ds-flatfs#59](https://github.com/ipfs/go-ds-flatfs/pull/59))\n - doc: add a lead maintainer ([ipfs/go-ds-flatfs#55](https://github.com/ipfs/go-ds-flatfs/pull/55))\n - make delete idempotent ([ipfs/go-ds-flatfs#54](https://github.com/ipfs/go-ds-flatfs/pull/54))\n- github.com/ipfs/go-ds-leveldb (v0.0.2 -> v0.4.2):\n - prevent closing concurrently with other operations. ([ipfs/go-ds-leveldb#42](https://github.com/ipfs/go-ds-leveldb/pull/42))\n - feat: update go-datastore ([ipfs/go-ds-leveldb#40](https://github.com/ipfs/go-ds-leveldb/pull/40))\n - update datastore Interface ([ipfs/go-ds-leveldb#36](https://github.com/ipfs/go-ds-leveldb/pull/36))\n - query: always return the size ([ipfs/go-ds-leveldb#35](https://github.com/ipfs/go-ds-leveldb/pull/35))\n - doc: add a lead maintainer ([ipfs/go-ds-leveldb#31](https://github.com/ipfs/go-ds-leveldb/pull/31))\n - make delete idempotent ([ipfs/go-ds-leveldb#30](https://github.com/ipfs/go-ds-leveldb/pull/30))\n- github.com/ipfs/go-ds-measure (v0.0.1 -> v0.1.0):\n - update datastore Interface ([ipfs/go-ds-measure#23](https://github.com/ipfs/go-ds-measure/pull/23))\n - Add Datastore Tests ([ipfs/go-ds-measure#24](https://github.com/ipfs/go-ds-measure/pull/24))\n - fix GetSize calls reported as Has ([ipfs/go-ds-measure#20](https://github.com/ipfs/go-ds-measure/pull/20))\n- github.com/ipfs/go-fs-lock (v0.0.1 -> v0.0.4):\n - fix: revert small breaking change ([ipfs/go-fs-lock#10](https://github.com/ipfs/go-fs-lock/pull/10))\n - Enh/improve error handling ([ipfs/go-fs-lock#9](https://github.com/ipfs/go-fs-lock/pull/9))\n - Use path/filepath instead of path ([ipfs/go-fs-lock#8](https://github.com/ipfs/go-fs-lock/pull/8))\n- github.com/ipfs/go-ipfs-blockstore (v0.0.1 -> v0.1.4):\n - return the correct size when only \"has\" is cached ([ipfs/go-ipfs-blockstore#36](https://github.com/ipfs/go-ipfs-blockstore/pull/36))\n - cache: switch to 2q ([ipfs/go-ipfs-blockstore#20](https://github.com/ipfs/go-ipfs-blockstore/pull/20))\n- github.com/ipfs/go-ipfs-chunker (v0.0.1 -> v0.0.5):\n - fix: don't return an empty block at the end ([ipfs/go-ipfs-chunker#22](https://github.com/ipfs/go-ipfs-chunker/pull/22))\n - Rigorous sizing checks ([ipfs/go-ipfs-chunker#21](https://github.com/ipfs/go-ipfs-chunker/pull/21))\n - Improve performance of buzhash ([ipfs/go-ipfs-chunker#17](https://github.com/ipfs/go-ipfs-chunker/pull/17))\n - Implement buzhash ([ipfs/go-ipfs-chunker#16](https://github.com/ipfs/go-ipfs-chunker/pull/16))\n - Add benchmarks ([ipfs/go-ipfs-chunker#15](https://github.com/ipfs/go-ipfs-chunker/pull/15))\n- github.com/ipfs/go-ipfs-cmds (v0.0.8 -> v0.2.2):\n - Fix: disallow POST without Origin nor Referer from specific user agents ([ipfs/go-ipfs-cmds#193](https://github.com/ipfs/go-ipfs-cmds/pull/193))\n - doc: document command fields ([ipfs/go-ipfs-cmds#192](https://github.com/ipfs/go-ipfs-cmds/pull/192))\n - change HandledMethods to AllowGet and cleanup method handling ([ipfs/go-ipfs-cmds#191](https://github.com/ipfs/go-ipfs-cmds/pull/191))\n - remove deprecated log.Warning(f) ([ipfs/go-ipfs-cmds#180](https://github.com/ipfs/go-ipfs-cmds/pull/180))\n - http: configurable allowed request methods for the API. ([ipfs/go-ipfs-cmds#190](https://github.com/ipfs/go-ipfs-cmds/pull/190))\n - #183 refactored the request options conversion code per the ticket requirements ([ipfs/go-ipfs-cmds#187](https://github.com/ipfs/go-ipfs-cmds/pull/187))\n - fix typo ([ipfs/go-ipfs-cmds#188](https://github.com/ipfs/go-ipfs-cmds/pull/188))\n - ([ipfs/go-ipfs-cmds#183](https://github.com/ipfs/go-ipfs-cmds/pull/183))\n - fix: normalize options when parsing them ([ipfs/go-ipfs-cmds#186](https://github.com/ipfs/go-ipfs-cmds/pull/186))\n - feat:add strings option; re-implement file ignore ([ipfs/go-ipfs-cmds#181](https://github.com/ipfs/go-ipfs-cmds/pull/181))\n - Special-case accepting explicitly supplied named pipes ([ipfs/go-ipfs-cmds#184](https://github.com/ipfs/go-ipfs-cmds/pull/184))\n - Chore/remove gx ([ipfs/go-ipfs-cmds#182](https://github.com/ipfs/go-ipfs-cmds/pull/182))\n - http: allow specifying a custom http client ([ipfs/go-ipfs-cmds#175](https://github.com/ipfs/go-ipfs-cmds/pull/175))\n - http: cleanup http related errors ([ipfs/go-ipfs-cmds#173](https://github.com/ipfs/go-ipfs-cmds/pull/173))\n - fix: too many arguments error text ([ipfs/go-ipfs-cmds#172](https://github.com/ipfs/go-ipfs-cmds/pull/172))\n - fallback executor support ([ipfs/go-ipfs-cmds#171](https://github.com/ipfs/go-ipfs-cmds/pull/171))\n - make ErrorType a valid error and implement Unwrap on Error ([ipfs/go-ipfs-cmds#170](https://github.com/ipfs/go-ipfs-cmds/pull/170))\n - feat: improve error codes ([ipfs/go-ipfs-cmds#168](https://github.com/ipfs/go-ipfs-cmds/pull/168))\n - Fix a typo ([ipfs/go-ipfs-cmds#169](https://github.com/ipfs/go-ipfs-cmds/pull/169))\n- github.com/ipfs/go-ipfs-config (v0.0.3 -> v0.5.3):\n - fix: correct the default-datastore config profile ([ipfs/go-ipfs-config#80](https://github.com/ipfs/go-ipfs-config/pull/80))\n - feat: disable autonat service when in lowpower mode ([ipfs/go-ipfs-config#77](https://github.com/ipfs/go-ipfs-config/pull/77))\n - feat: add and use a duration helper type ([ipfs/go-ipfs-config#76](https://github.com/ipfs/go-ipfs-config/pull/76))\n - feat: add an autonat config section ([ipfs/go-ipfs-config#75](https://github.com/ipfs/go-ipfs-config/pull/75))\n - feat: remove Routing.PrivateType ([ipfs/go-ipfs-config#74](https://github.com/ipfs/go-ipfs-config/pull/74))\n - feat: add private routing config field ([ipfs/go-ipfs-config#73](https://github.com/ipfs/go-ipfs-config/pull/73))\n - feat: mark badger as stable ([ipfs/go-ipfs-config#70](https://github.com/ipfs/go-ipfs-config/pull/70))\n - feat: remove PreferTLS experiment ([ipfs/go-ipfs-config#71](https://github.com/ipfs/go-ipfs-config/pull/71))\n - feat: remove old bootstrap peers ([ipfs/go-ipfs-config#67](https://github.com/ipfs/go-ipfs-config/pull/67))\n - add config options for proxy/subdomain ([ipfs/go-ipfs-config#30](https://github.com/ipfs/go-ipfs-config/pull/30))\n - feat: add graphsync option ([ipfs/go-ipfs-config#62](https://github.com/ipfs/go-ipfs-config/pull/62))\n - profile: badger profile now defaults to asynchronous writes ([ipfs/go-ipfs-config#60](https://github.com/ipfs/go-ipfs-config/pull/60))\n - migrate multiaddrs from /ipfs -> /p2p ([ipfs/go-ipfs-config#39](https://github.com/ipfs/go-ipfs-config/pull/39))\n - use key size constraints defined in libp2p ([ipfs/go-ipfs-config#57](https://github.com/ipfs/go-ipfs-config/pull/57))\n - plugins: don't omit empty config values ([ipfs/go-ipfs-config#46](https://github.com/ipfs/go-ipfs-config/pull/46))\n - make it easier to detect an uninitialized repo ([ipfs/go-ipfs-config#45](https://github.com/ipfs/go-ipfs-config/pull/45))\n - nit: omit empty plugin values ([ipfs/go-ipfs-config#44](https://github.com/ipfs/go-ipfs-config/pull/44))\n - add plugins config section ([ipfs/go-ipfs-config#43](https://github.com/ipfs/go-ipfs-config/pull/43))\n - Add very basic (possibly temporary) Provider configs ([ipfs/go-ipfs-config#38](https://github.com/ipfs/go-ipfs-config/pull/38))\n - fix string formatting of bootstrap peers ([ipfs/go-ipfs-config#37](https://github.com/ipfs/go-ipfs-config/pull/37))\n - migrate to the consolidated libp2p ([ipfs/go-ipfs-config#36](https://github.com/ipfs/go-ipfs-config/pull/36))\n - Add strategic provider system experiment flag ([ipfs/go-ipfs-config#33](https://github.com/ipfs/go-ipfs-config/pull/33))\n- github.com/ipfs/go-ipfs-files (v0.0.3 -> v0.0.8):\n - skip ignored files when calculating size ([ipfs/go-ipfs-files#30](https://github.com/ipfs/go-ipfs-files/pull/30))\n - Feat/add ignore rules ([ipfs/go-ipfs-files#26](https://github.com/ipfs/go-ipfs-files/pull/26))\n - revert(symlink): keep stat argument ([ipfs/go-ipfs-files#23](https://github.com/ipfs/go-ipfs-files/pull/23))\n - feat: correctly report the size of symlinks ([ipfs/go-ipfs-files#22](https://github.com/ipfs/go-ipfs-files/pull/22))\n - serialfile: fix handling of hidden paths on windows ([ipfs/go-ipfs-files#21](https://github.com/ipfs/go-ipfs-files/pull/21))\n - feat: add WriteTo function ([ipfs/go-ipfs-files#20](https://github.com/ipfs/go-ipfs-files/pull/20))\n - doc: fix formdata documentation ([ipfs/go-ipfs-files#19](https://github.com/ipfs/go-ipfs-files/pull/19))\n- github.com/ipfs/go-ipfs-pinner (v0.0.1 -> v0.0.4):\n - fix: don't hold the pin lock while updating pins ([ipfs/go-ipfs-pinner#2](https://github.com/ipfs/go-ipfs-pinner/pull/2))\n- github.com/ipfs/go-ipfs-pq (v0.0.1 -> v0.0.2):\n - Remove() ([ipfs/go-ipfs-pq#5](https://github.com/ipfs/go-ipfs-pq/pull/5))\n - Fix Peek() test ([ipfs/go-ipfs-pq#4](https://github.com/ipfs/go-ipfs-pq/pull/4))\n - add Peek() method ([ipfs/go-ipfs-pq#3](https://github.com/ipfs/go-ipfs-pq/pull/3))\n - add gomod support // tag v0.0.1. ([ipfs/go-ipfs-pq#1](https://github.com/ipfs/go-ipfs-pq/pull/1))\n- github.com/ipfs/go-ipfs-routing (v0.0.1 -> v0.1.0):\n - migrate to go-libp2p-core ([ipfs/go-ipfs-routing#22](https://github.com/ipfs/go-ipfs-routing/pull/22))\n- github.com/ipfs/go-ipld-cbor (v0.0.2 -> v0.0.4):\n - doc: add a lead maintainer ([ipfs/go-ipld-cbor#65](https://github.com/ipfs/go-ipld-cbor/pull/65))\n - fastpath CBOR ([ipfs/go-ipld-cbor#64](https://github.com/ipfs/go-ipld-cbor/pull/64))\n- github.com/ipfs/go-ipld-format (v0.0.2 -> v0.2.0):\n - fix: change the batch size to avoid buffering too much ([ipfs/go-ipld-format#56](https://github.com/ipfs/go-ipld-format/pull/56))\n - doc: add a lead maintainer ([ipfs/go-ipld-format#54](https://github.com/ipfs/go-ipld-format/pull/54))\n- github.com/ipfs/go-ipld-git (v0.0.2 -> v0.0.3):\n - Use RFC3339 to format dates, fixes #16 ([ipfs/go-ipld-git#32](https://github.com/ipfs/go-ipld-git/pull/32))\n - doc: add a lead maintainer ([ipfs/go-ipld-git#41](https://github.com/ipfs/go-ipld-git/pull/41))\n- github.com/ipfs/go-ipns (v0.0.1 -> v0.0.2):\n - readme: add a lead maintainer ([ipfs/go-ipns#25](https://github.com/ipfs/go-ipns/pull/25))\n- github.com/ipfs/go-log (v0.0.1 -> v1.0.4):\n - add IPFS_* env vars back for transitionary release of go-log ([ipfs/go-log#67](https://github.com/ipfs/go-log/pull/67))\n - Experimental: zap backend for go-log ([ipfs/go-log#61](https://github.com/ipfs/go-log/pull/61))\n - Spelling fix ([ipfs/go-log#63](https://github.com/ipfs/go-log/pull/63))\n - Deprecate EventLogging and Warning* functions ([ipfs/go-log#62](https://github.com/ipfs/go-log/pull/62))\n- github.com/ipfs/go-merkledag (v0.0.3 -> v0.3.2):\n - fix: correctly construct sessions ([ipfs/go-merkledag#56](https://github.com/ipfs/go-merkledag/pull/56))\n - Migrate dagutils from go-ipfs ([ipfs/go-merkledag#50](https://github.com/ipfs/go-merkledag/pull/50))\n - Make getPBNode Public ([ipfs/go-merkledag#49](https://github.com/ipfs/go-merkledag/pull/49))\n - Pull In Upstream Changes ([ipfs/go-merkledag#1](https://github.com/ipfs/go-merkledag/pull/1))\n - fix: slightly reduce memory usage when walking large directory trees ([ipfs/go-merkledag#45](https://github.com/ipfs/go-merkledag/pull/45))\n - fix: return ErrLinkNotFound when the _link_ isn't found ([ipfs/go-merkledag#44](https://github.com/ipfs/go-merkledag/pull/44))\n - fix: include root in searches by default ([ipfs/go-merkledag#43](https://github.com/ipfs/go-merkledag/pull/43))\n - rework the graph walking functions with functional options ([ipfs/go-merkledag#42](https://github.com/ipfs/go-merkledag/pull/42))\n - fix inconsistent EnumerateChildrenAsync behavior ([ipfs/go-merkledag#41](https://github.com/ipfs/go-merkledag/pull/41))\n- github.com/ipfs/go-mfs (v0.0.7 -> v0.1.1):\n - migrate to go-libp2p-core ([ipfs/go-mfs#77](https://github.com/ipfs/go-mfs/pull/77))\n- github.com/ipfs/go-peertaskqueue (v0.0.5-f09820a0a5b6 -> v0.2.0):\n - Extend peer task queue to work with want-have / want-block ([ipfs/go-peertaskqueue#8](https://github.com/ipfs/go-peertaskqueue/pull/8))\n - migrate to go-libp2p-core ([ipfs/go-peertaskqueue#4](https://github.com/ipfs/go-peertaskqueue/pull/4))\n- github.com/ipfs/go-unixfs (v0.0.6 -> v0.2.4):\n - fix: fix a panic when deleting ([ipfs/go-unixfs#81](https://github.com/ipfs/go-unixfs/pull/81))\n - fix(dagreader): remove a buggy workaround for a gateway issue ([ipfs/go-unixfs#80](https://github.com/ipfs/go-unixfs/pull/80))\n - fix: correctly handle symlink file sizes ([ipfs/go-unixfs#78](https://github.com/ipfs/go-unixfs/pull/78))\n - fix: return the correct error from RemoveChild ([ipfs/go-unixfs#76](https://github.com/ipfs/go-unixfs/pull/76))\n - update the last go-merkledag ([ipfs/go-unixfs#75](https://github.com/ipfs/go-unixfs/pull/75))\n - fix: enumerate children ([ipfs/go-unixfs#74](https://github.com/ipfs/go-unixfs/pull/74))\n- github.com/ipfs/interface-go-ipfs-core (v0.0.8 -> v0.2.7):\n - Add pin ls tests for indirect pin traversal and pin type precedence ([ipfs/interface-go-ipfs-core#47](https://github.com/ipfs/interface-go-ipfs-core/pull/47))\n - fix(test): fix a flaky pubsub test ([ipfs/interface-go-ipfs-core#45](https://github.com/ipfs/interface-go-ipfs-core/pull/45))\n - README: stub ([ipfs/interface-go-ipfs-core#44](https://github.com/ipfs/interface-go-ipfs-core/pull/44))\n - test: test ReadAt if implemented ([ipfs/interface-go-ipfs-core#43](https://github.com/ipfs/interface-go-ipfs-core/pull/43))\n - test: fix put with hash test ([ipfs/interface-go-ipfs-core#41](https://github.com/ipfs/interface-go-ipfs-core/pull/41))\n - Bump go-libp2p-core, up test key size to 2048 ([ipfs/interface-go-ipfs-core#39](https://github.com/ipfs/interface-go-ipfs-core/pull/39))\n - migrate to go-libp2p-core. ([ipfs/interface-go-ipfs-core#35](https://github.com/ipfs/interface-go-ipfs-core/pull/35))\n - tests: expose TestSuite ([ipfs/interface-go-ipfs-core#34](https://github.com/ipfs/interface-go-ipfs-core/pull/34))\n- github.com/libp2p/go-libp2p (v0.0.32 -> v0.8.2):\n - fix: keep observed addrs alive as long as their associated connections are alive ([libp2p/go-libp2p#899](https://github.com/libp2p/go-libp2p/pull/899))\n - fix: refactor logic for identifying connections ([libp2p/go-libp2p#898](https://github.com/libp2p/go-libp2p/pull/898))\n - fix: reduce log level of a noisy log line ([libp2p/go-libp2p#889](https://github.com/libp2p/go-libp2p/pull/889))\n - [discovery] missing defer .Stop on ticker ([libp2p/go-libp2p#888](https://github.com/libp2p/go-libp2p/pull/888))\n - deprioritize unspecified addresses in mock connections ([libp2p/go-libp2p#887](https://github.com/libp2p/go-libp2p/pull/887))\n - feat: support TLS by default ([libp2p/go-libp2p#884](https://github.com/libp2p/go-libp2p/pull/884))\n - Expose option for setting autonat throttling ([libp2p/go-libp2p#882](https://github.com/libp2p/go-libp2p/pull/882))\n - Clearer naming of nat override options ([libp2p/go-libp2p#878](https://github.com/libp2p/go-libp2p/pull/878))\n - fix: set the private key when constructing the autonat service ([libp2p/go-libp2p#853](https://github.com/libp2p/go-libp2p/pull/853))\n - Signal address change ([libp2p/go-libp2p#851](https://github.com/libp2p/go-libp2p/pull/851))\n - fix multiple issues in the mock tests ([libp2p/go-libp2p#850](https://github.com/libp2p/go-libp2p/pull/850))\n - fix: minimal autonat dialer ([libp2p/go-libp2p#849](https://github.com/libp2p/go-libp2p/pull/849))\n - Trigger Autorelay on NAT events ([libp2p/go-libp2p#807](https://github.com/libp2p/go-libp2p/pull/807))\n - Local addr updated event ([libp2p/go-libp2p#847](https://github.com/libp2p/go-libp2p/pull/847))\n - feat(mock): reliable notifications ([libp2p/go-libp2p#836](https://github.com/libp2p/go-libp2p/pull/836))\n - doc(options): fix autorelay documentation ([libp2p/go-libp2p#835](https://github.com/libp2p/go-libp2p/pull/835))\n - change PrivateNetwork to accept a PSK, update constructor magic ([libp2p/go-libp2p#796](https://github.com/libp2p/go-libp2p/pull/796))\n - docs: Update the README ([libp2p/go-libp2p#827](https://github.com/libp2p/go-libp2p/pull/827))\n - fix: remove an unnecessary goroutine ([libp2p/go-libp2p#820](https://github.com/libp2p/go-libp2p/pull/820))\n - EnableAutoRelay should work without ContentRouting if there are StaticRelays defined ([libp2p/go-libp2p#810](https://github.com/libp2p/go-libp2p/pull/810))\n - Use of mux.ErrReset in mocknet ([libp2p/go-libp2p#815](https://github.com/libp2p/go-libp2p/pull/815))\n - docs: uniform comment sentences ([libp2p/go-libp2p#826](https://github.com/libp2p/go-libp2p/pull/826))\n - enable non-public address port mapping announcement ([libp2p/go-libp2p#771](https://github.com/libp2p/go-libp2p/pull/771))\n - fix: demote stream deadline errors to debug logs ([libp2p/go-libp2p#768](https://github.com/libp2p/go-libp2p/pull/768))\n - small grammar fixes and updates to readme ([libp2p/go-libp2p#743](https://github.com/libp2p/go-libp2p/pull/743))\n - Identify: Make activation threshold configurable ([libp2p/go-libp2p#740](https://github.com/libp2p/go-libp2p/pull/740))\n - better user-agent handling ([libp2p/go-libp2p#702](https://github.com/libp2p/go-libp2p/pull/702))\n - Update deps, mocknet tests ([libp2p/go-libp2p#697](https://github.com/libp2p/go-libp2p/pull/697))\n - autorelay: ensure candidate relays can hop ([libp2p/go-libp2p#696](https://github.com/libp2p/go-libp2p/pull/696))\n - We don't use `cs` here, drop it. ([libp2p/go-libp2p#682](https://github.com/libp2p/go-libp2p/pull/682))\n - Fix racy and failing test cases. ([libp2p/go-libp2p#674](https://github.com/libp2p/go-libp2p/pull/674))\n - fix: use the goprocess for closing ([libp2p/go-libp2p#669](https://github.com/libp2p/go-libp2p/pull/669))\n - update package table after -core refactor ([libp2p/go-libp2p#661](https://github.com/libp2p/go-libp2p/pull/661))\n - basic_host: ensure we close correctly when the context is canceled ([libp2p/go-libp2p#656](https://github.com/libp2p/go-libp2p/pull/656))\n - Add go-libp2p-gostream and go-libp2p-http to readme ([libp2p/go-libp2p#655](https://github.com/libp2p/go-libp2p/pull/655))\n- github.com/libp2p/go-libp2p-autonat (v0.0.6 -> v0.2.2):\n - Run Autonat Service while in unknown connectivity mode ([libp2p/go-libp2p-autonat#75](https://github.com/libp2p/go-libp2p-autonat/pull/75))\n - Add option to force nat into a specified reachability state ([libp2p/go-libp2p-autonat#55](https://github.com/libp2p/go-libp2p-autonat/pull/55))\n - Merge Autonat-svc ([libp2p/go-libp2p-autonat#54](https://github.com/libp2p/go-libp2p-autonat/pull/54))\n - change autonat interface to use functional options ([libp2p/go-libp2p-autonat#53](https://github.com/libp2p/go-libp2p-autonat/pull/53))\n - Limiting autonat service responses/startup ([libp2p/go-libp2p-autonat#45](https://github.com/libp2p/go-libp2p-autonat/pull/45))\n - Emit events when NAT status changes ([libp2p/go-libp2p-autonat#37](https://github.com/libp2p/go-libp2p-autonat/pull/37))\n - Take eventbus events to completion ([libp2p/go-libp2p-autonat#38](https://github.com/libp2p/go-libp2p-autonat/pull/38))\n - Add missing syntax to autonat.proto ([libp2p/go-libp2p-autonat#26](https://github.com/libp2p/go-libp2p-autonat/pull/26))\n - full close the autonat stream ([libp2p/go-libp2p-autonat#20](https://github.com/libp2p/go-libp2p-autonat/pull/20))\n - reduce dialback timeout to 15s ([libp2p/go-libp2p-autonat#17](https://github.com/libp2p/go-libp2p-autonat/pull/17))\n - Extract service implementation from go-libp2p-autonat ([libp2p/go-libp2p-autonat#1](https://github.com/libp2p/go-libp2p-autonat/pull/1))\n- github.com/libp2p/go-libp2p-circuit (v0.0.9 -> v0.2.2):\n - fix: don't abort accept when accepting a single connection fails ([libp2p/go-libp2p-circuit#107](https://github.com/libp2p/go-libp2p-circuit/pull/107))\n - Revert \"feat: functional options\" ([libp2p/go-libp2p-circuit#103](https://github.com/libp2p/go-libp2p-circuit/pull/103))\n - feat: remove relay discovery and unspecified relay dialing ([libp2p/go-libp2p-circuit#101](https://github.com/libp2p/go-libp2p-circuit/pull/101))\n - move protocol definitions to go-multiaddr ([libp2p/go-libp2p-circuit#81](https://github.com/libp2p/go-libp2p-circuit/pull/81))\n - return the full address from conn.RemoteMultiaddr ([libp2p/go-libp2p-circuit#80](https://github.com/libp2p/go-libp2p-circuit/pull/80))\n - expose CanHop as a module function ([libp2p/go-libp2p-circuit#79](https://github.com/libp2p/go-libp2p-circuit/pull/79))\n- github.com/libp2p/go-libp2p-discovery (v0.0.5 -> v0.4.0):\n - Fix race with reuse of randomness ([libp2p/go-libp2p-discovery#54](https://github.com/libp2p/go-libp2p-discovery/pull/54))\n - Add Backoff Cache Discovery ([libp2p/go-libp2p-discovery#26](https://github.com/libp2p/go-libp2p-discovery/pull/26))\n - Discovery based Content Routing ([libp2p/go-libp2p-discovery#27](https://github.com/libp2p/go-libp2p-discovery/pull/27))\n- github.com/libp2p/go-libp2p-kad-dht (v0.0.15 -> v0.7.10):\n - fix: avoid blocking when bootstrapping ([libp2p/go-libp2p-kad-dht#610](https://github.com/libp2p/go-libp2p-kad-dht/pull/610))\n - fix: re-validate peers whenever their state changes ([libp2p/go-libp2p-kad-dht#607](https://github.com/libp2p/go-libp2p-kad-dht/pull/607))\n - intercept failing query events when finding providers ([libp2p/go-libp2p-kad-dht#603](https://github.com/libp2p/go-libp2p-kad-dht/pull/603))\n - feat: set provider manager options ([libp2p/go-libp2p-kad-dht#593](https://github.com/libp2p/go-libp2p-kad-dht/pull/593))\n - fix: optimize debug logging a bit ([libp2p/go-libp2p-kad-dht#598](https://github.com/libp2p/go-libp2p-kad-dht/pull/598))\n - stricter definition of public for DHT ([libp2p/go-libp2p-kad-dht#596](https://github.com/libp2p/go-libp2p-kad-dht/pull/596))\n - feat: reduce allocations ([libp2p/go-libp2p-kad-dht#588](https://github.com/libp2p/go-libp2p-kad-dht/pull/588))\n - query.go: Remove shuffle comment ([libp2p/go-libp2p-kad-dht#586](https://github.com/libp2p/go-libp2p-kad-dht/pull/586))\n - fix: optimize isRelay ([libp2p/go-libp2p-kad-dht#585](https://github.com/libp2p/go-libp2p-kad-dht/pull/585))\n - feat: expose WANActive ([libp2p/go-libp2p-kad-dht#580](https://github.com/libp2p/go-libp2p-kad-dht/pull/580))\n - fix: improve error handling in dual dht ([libp2p/go-libp2p-kad-dht#582](https://github.com/libp2p/go-libp2p-kad-dht/pull/582))\n - fix: deduplicate addresses ([libp2p/go-libp2p-kad-dht#581](https://github.com/libp2p/go-libp2p-kad-dht/pull/581))\n - Fix bug in periodic peer pinging ([libp2p/go-libp2p-kad-dht#579](https://github.com/libp2p/go-libp2p-kad-dht/pull/579))\n - Dual DHT scaffold ([libp2p/go-libp2p-kad-dht#570](https://github.com/libp2p/go-libp2p-kad-dht/pull/570))\n - fix: linting fixes ([libp2p/go-libp2p-kad-dht#578](https://github.com/libp2p/go-libp2p-kad-dht/pull/578))\n - fix: remove local provider check ([libp2p/go-libp2p-kad-dht#577](https://github.com/libp2p/go-libp2p-kad-dht/pull/577))\n - fix: use the routing table filter ([libp2p/go-libp2p-kad-dht#576](https://github.com/libp2p/go-libp2p-kad-dht/pull/576))\n - fix: handle empty keys ([libp2p/go-libp2p-kad-dht#562](https://github.com/libp2p/go-libp2p-kad-dht/pull/562))\n - Set record handlers for the default protocol prefix ([libp2p/go-libp2p-kad-dht#560](https://github.com/libp2p/go-libp2p-kad-dht/pull/560))\n - fix incorrect error handling during provider record lookups ([libp2p/go-libp2p-kad-dht#554](https://github.com/libp2p/go-libp2p-kad-dht/pull/554))\n - Proposed DHTv2 Changes ([libp2p/go-libp2p-kad-dht#473](https://github.com/libp2p/go-libp2p-kad-dht/pull/473))\n - fix: obey the context when sending messages to peers ([libp2p/go-libp2p-kad-dht#462](https://github.com/libp2p/go-libp2p-kad-dht/pull/462))\n - Close context correctly ([libp2p/go-libp2p-kad-dht#477](https://github.com/libp2p/go-libp2p-kad-dht/pull/477))\n - add benchmark for handleFindPeer ([libp2p/go-libp2p-kad-dht#475](https://github.com/libp2p/go-libp2p-kad-dht/pull/475))\n - give views names again ([libp2p/go-libp2p-kad-dht#474](https://github.com/libp2p/go-libp2p-kad-dht/pull/474))\n - metrics: record message/request event even in case of error ([libp2p/go-libp2p-kad-dht#464](https://github.com/libp2p/go-libp2p-kad-dht/pull/464))\n - fix(dialqueue): fix a timer leak ([libp2p/go-libp2p-kad-dht#466](https://github.com/libp2p/go-libp2p-kad-dht/pull/466))\n - fix(query): cancel the context when the query finishes ([libp2p/go-libp2p-kad-dht#467](https://github.com/libp2p/go-libp2p-kad-dht/pull/467))\n - fix(providers): upgrade warnings to errors ([libp2p/go-libp2p-kad-dht#455](https://github.com/libp2p/go-libp2p-kad-dht/pull/455))\n - Make the Routing Table's latency tolerance configurable. ([libp2p/go-libp2p-kad-dht#454](https://github.com/libp2p/go-libp2p-kad-dht/pull/454))\n - Adjust cluster level while encoding as well ([libp2p/go-libp2p-kad-dht#445](https://github.com/libp2p/go-libp2p-kad-dht/pull/445))\n - Remove incorrect doc ([libp2p/go-libp2p-kad-dht#443](https://github.com/libp2p/go-libp2p-kad-dht/pull/443))\n - feat: reduce stream idle timeout to 1m ([libp2p/go-libp2p-kad-dht#441](https://github.com/libp2p/go-libp2p-kad-dht/pull/441))\n - Provider records use multihashes instead of CIDs ([libp2p/go-libp2p-kad-dht#422](https://github.com/libp2p/go-libp2p-kad-dht/pull/422))\n - Fix flaky TestEmptyTableTest ([libp2p/go-libp2p-kad-dht#433](https://github.com/libp2p/go-libp2p-kad-dht/pull/433))\n - Refresh cpl's in dht ([libp2p/go-libp2p-kad-dht#428](https://github.com/libp2p/go-libp2p-kad-dht/pull/428))\n - fix: always send the result channel when triggering a refresh ([libp2p/go-libp2p-kad-dht#425](https://github.com/libp2p/go-libp2p-kad-dht/pull/425))\n - feat: allow disabling value and provider storage/messages ([libp2p/go-libp2p-kad-dht#400](https://github.com/libp2p/go-libp2p-kad-dht/pull/400))\n - fix: prioritize closer peers ([libp2p/go-libp2p-kad-dht#424](https://github.com/libp2p/go-libp2p-kad-dht/pull/424))\n - fix: try to re-add existing peers when the routing table is empty ([libp2p/go-libp2p-kad-dht#420](https://github.com/libp2p/go-libp2p-kad-dht/pull/420))\n - feat: refresh and wait ([libp2p/go-libp2p-kad-dht#418](https://github.com/libp2p/go-libp2p-kad-dht/pull/418))\n - Make max record age configurable ([libp2p/go-libp2p-kad-dht#410](https://github.com/libp2p/go-libp2p-kad-dht/pull/410))\n - fix and simplify some bootstrapping logic ([libp2p/go-libp2p-kad-dht#405](https://github.com/libp2p/go-libp2p-kad-dht/pull/405))\n - feat(bootstrap): take autobootstrap to completion ([libp2p/go-libp2p-kad-dht#403](https://github.com/libp2p/go-libp2p-kad-dht/pull/403))\n - Feature/correct bootstrapping ([libp2p/go-libp2p-kad-dht#384](https://github.com/libp2p/go-libp2p-kad-dht/pull/384))\n - Update tests to use Ed25519 when acceptable. ([libp2p/go-libp2p-kad-dht#380](https://github.com/libp2p/go-libp2p-kad-dht/pull/380))\n - Add timeout ([libp2p/go-libp2p-kad-dht#351](https://github.com/libp2p/go-libp2p-kad-dht/pull/351))\n - Feat/message size ([libp2p/go-libp2p-kad-dht#353](https://github.com/libp2p/go-libp2p-kad-dht/pull/353))\n - reduce background goroutines ([libp2p/go-libp2p-kad-dht#340](https://github.com/libp2p/go-libp2p-kad-dht/pull/340))\n- github.com/libp2p/go-libp2p-kbucket (v0.1.1 -> v0.4.1):\n - fix: use time.Duration for time, not floats ([libp2p/go-libp2p-kbucket#76](https://github.com/libp2p/go-libp2p-kbucket/pull/76))\n - Add LastUsefulAt and LastSuccessfulQueryAt for each peer ([libp2p/go-libp2p-kbucket#75](https://github.com/libp2p/go-libp2p-kbucket/pull/75))\n - fix: correctly track CPLs of never refreshed buckets ([libp2p/go-libp2p-kbucket#71](https://github.com/libp2p/go-libp2p-kbucket/pull/71))\n - Get Peer Infos ([libp2p/go-libp2p-kbucket#69](https://github.com/libp2p/go-libp2p-kbucket/pull/69))\n - fix: use accurate bucket logic ([libp2p/go-libp2p-kbucket#64](https://github.com/libp2p/go-libp2p-kbucket/pull/64))\n - Replace dead peers & increase replacement cache size ([libp2p/go-libp2p-kbucket#59](https://github.com/libp2p/go-libp2p-kbucket/pull/59))\n - Kbucket refactoring for Content Routing ([libp2p/go-libp2p-kbucket#54](https://github.com/libp2p/go-libp2p-kbucket/pull/54))\n - Disassociate RT membership from connectivity ([libp2p/go-libp2p-kbucket#50](https://github.com/libp2p/go-libp2p-kbucket/pull/50))\n - Unit Test for the util.Closer function ([libp2p/go-libp2p-kbucket#48](https://github.com/libp2p/go-libp2p-kbucket/pull/48))\n - Refresh Cpl's, not buckets ([libp2p/go-libp2p-kbucket#46](https://github.com/libp2p/go-libp2p-kbucket/pull/46))\n - Fix NearestPeers Doc ([libp2p/go-libp2p-kbucket#45](https://github.com/libp2p/go-libp2p-kbucket/pull/45))\n - fix: when the target bucket is empty or low, pull from all other buckets ([libp2p/go-libp2p-kbucket#43](https://github.com/libp2p/go-libp2p-kbucket/pull/43))\n - readme: replace IPFS contrib links with libp2p ([libp2p/go-libp2p-kbucket#34](https://github.com/libp2p/go-libp2p-kbucket/pull/34))\n - k-bucket support for peoper kad bootstrapping ([libp2p/go-libp2p-kbucket#38](https://github.com/libp2p/go-libp2p-kbucket/pull/38))\n - Fix bootstrapping id generation logic ([libp2p/go-libp2p-kbucket#1](https://github.com/libp2p/go-libp2p-kbucket/pull/1))\n - fix: avoid hashing under a lock ([libp2p/go-libp2p-kbucket#31](https://github.com/libp2p/go-libp2p-kbucket/pull/31))\n - dep: use a faster sha256 library ([libp2p/go-libp2p-kbucket#32](https://github.com/libp2p/go-libp2p-kbucket/pull/32))\n - Remove a lot of allocations, and fix some ambiguous naming ([libp2p/go-libp2p-kbucket#30](https://github.com/libp2p/go-libp2p-kbucket/pull/30))\n- github.com/libp2p/go-libp2p-mplex (v0.1.1 -> v0.2.3):\n - Respect mux.ErrReset ([libp2p/go-libp2p-mplex#9](https://github.com/libp2p/go-libp2p-mplex/pull/9))\n- github.com/libp2p/go-libp2p-nat (v0.0.4 -> v0.0.6):\n - typo and changed deprecated method ([libp2p/go-libp2p-nat#26](https://github.com/libp2p/go-libp2p-nat/pull/26))\n - nit: fix log format ([libp2p/go-libp2p-nat#19](https://github.com/libp2p/go-libp2p-nat/pull/19))\n - fix: remove notifier ([libp2p/go-libp2p-nat#18](https://github.com/libp2p/go-libp2p-nat/pull/18))\n- github.com/libp2p/go-libp2p-peerstore (v0.0.6 -> v0.2.3):\n - fix: handle nil peer IDs ([libp2p/go-libp2p-peerstore#88](https://github.com/libp2p/go-libp2p-peerstore/pull/88))\n - Fix memory store signed peer record bug ([libp2p/go-libp2p-peerstore#133](https://github.com/libp2p/go-libp2p-peerstore/pull/133))\n - fix: make closing the in-memory peerstore actually close it ([libp2p/go-libp2p-peerstore#131](https://github.com/libp2p/go-libp2p-peerstore/pull/131))\n - Correct path to peer.AddrInfo in deprecation ([libp2p/go-libp2p-peerstore#124](https://github.com/libp2p/go-libp2p-peerstore/pull/124))\n - fix multiple TTL bugs ([libp2p/go-libp2p-peerstore#92](https://github.com/libp2p/go-libp2p-peerstore/pull/92))\n - reduce allocations when adding addrs ([libp2p/go-libp2p-peerstore#86](https://github.com/libp2p/go-libp2p-peerstore/pull/86))\n - test: add metadata test ([libp2p/go-libp2p-peerstore#82](https://github.com/libp2p/go-libp2p-peerstore/pull/82))\n - set map in constructor ([libp2p/go-libp2p-peerstore#81](https://github.com/libp2p/go-libp2p-peerstore/pull/81))\n - improve interning ([libp2p/go-libp2p-peerstore#79](https://github.com/libp2p/go-libp2p-peerstore/pull/79))\n- github.com/libp2p/go-libp2p-pnet (v0.0.1 -> v0.2.0):\n - remove key serialization, construct conn from ipnet.PSK ([libp2p/go-libp2p-pnet#32](https://github.com/libp2p/go-libp2p-pnet/pull/32))\n - remove dependency on go-multicodec ([libp2p/go-libp2p-pnet#26](https://github.com/libp2p/go-libp2p-pnet/pull/26))\n- github.com/libp2p/go-libp2p-pubsub (v0.0.3 -> v0.2.7):\n - Replace LRU cache blacklist implementation with a time cache ([libp2p/go-libp2p-pubsub#258](https://github.com/libp2p/go-libp2p-pubsub/pull/258))\n - Configurable size of validate queue ([libp2p/go-libp2p-pubsub#255](https://github.com/libp2p/go-libp2p-pubsub/pull/255))\n - Rename VaidatorData to ValidatorData ([libp2p/go-libp2p-pubsub#251](https://github.com/libp2p/go-libp2p-pubsub/pull/251))\n - Configurable message id function ([libp2p/go-libp2p-pubsub#248](https://github.com/libp2p/go-libp2p-pubsub/pull/248))\n - tracing support ([libp2p/go-libp2p-pubsub#227](https://github.com/libp2p/go-libp2p-pubsub/pull/227))\n - add ValidatorData field to Message ([libp2p/go-libp2p-pubsub#231](https://github.com/libp2p/go-libp2p-pubsub/pull/231))\n - Configurable outbound peer queue sizes ([libp2p/go-libp2p-pubsub#230](https://github.com/libp2p/go-libp2p-pubsub/pull/230))\n - Topic handler bug fixes ([libp2p/go-libp2p-pubsub#225](https://github.com/libp2p/go-libp2p-pubsub/pull/225))\n - Add Discovery ([libp2p/go-libp2p-pubsub#184](https://github.com/libp2p/go-libp2p-pubsub/pull/184))\n - Expose the peer that propagates a message to the recipient ([libp2p/go-libp2p-pubsub#218](https://github.com/libp2p/go-libp2p-pubsub/pull/218))\n - gossip methods: renames and predicate adjustment ([libp2p/go-libp2p-pubsub#204](https://github.com/libp2p/go-libp2p-pubsub/pull/204))\n - godocs: clarify config params of MessageCache. ([libp2p/go-libp2p-pubsub#205](https://github.com/libp2p/go-libp2p-pubsub/pull/205))\n - minor bug fix: on join, source peers from gossip[topic] if insufficient peers in fanout[topic] ([libp2p/go-libp2p-pubsub#196](https://github.com/libp2p/go-libp2p-pubsub/pull/196))\n - add PubSub's context to Subscription ([libp2p/go-libp2p-pubsub#201](https://github.com/libp2p/go-libp2p-pubsub/pull/201))\n - Add the ability to handle newly subscribed peers ([libp2p/go-libp2p-pubsub#190](https://github.com/libp2p/go-libp2p-pubsub/pull/190))\n - Fix gossipsub race condition for heartbeat ([libp2p/go-libp2p-pubsub#188](https://github.com/libp2p/go-libp2p-pubsub/pull/188))\n- github.com/libp2p/go-libp2p-pubsub-router (v0.0.3 -> v0.2.1):\n - fix: ignore bad peers when fetching the latest value ([libp2p/go-libp2p-pubsub-router#54](https://github.com/libp2p/go-libp2p-pubsub-router/pull/54))\n - fix: rename MinimalPubsub -> Pubsub interface and improve docs ([libp2p/go-libp2p-pubsub-router#52](https://github.com/libp2p/go-libp2p-pubsub-router/pull/52))\n - Use Minimal PubSub Interface Instead Of Full PubSub Router ([libp2p/go-libp2p-pubsub-router#51](https://github.com/libp2p/go-libp2p-pubsub-router/pull/51))\n - Remove bootstrapping code ([libp2p/go-libp2p-pubsub-router#37](https://github.com/libp2p/go-libp2p-pubsub-router/pull/37))\n - readme: replace IPFS contrib links with libp2p ([libp2p/go-libp2p-pubsub-router#34](https://github.com/libp2p/go-libp2p-pubsub-router/pull/34))\n - Add Persistence Layer on top of PubSub ([libp2p/go-libp2p-pubsub-router#33](https://github.com/libp2p/go-libp2p-pubsub-router/pull/33))\n - Subscribe to PubSub topic before Publishing ([libp2p/go-libp2p-pubsub-router#30](https://github.com/libp2p/go-libp2p-pubsub-router/pull/30))\n - PutValue not blocked by Provide during bootstrapping ([libp2p/go-libp2p-pubsub-router#29](https://github.com/libp2p/go-libp2p-pubsub-router/pull/29))\n- github.com/libp2p/go-libp2p-quic-transport (v0.0.3 -> v0.3.5):\n - add command line client and server ([libp2p/go-libp2p-quic-transport#139](https://github.com/libp2p/go-libp2p-quic-transport/pull/139))\n - write qlogs to a temporary file first, then rename them when done ([libp2p/go-libp2p-quic-transport#136](https://github.com/libp2p/go-libp2p-quic-transport/pull/136))\n - export qlogs when the QLOGDIR env variable is set ([libp2p/go-libp2p-quic-transport#129](https://github.com/libp2p/go-libp2p-quic-transport/pull/129))\n - fix: avoid dialing/listening on dns addresses ([libp2p/go-libp2p-quic-transport#131](https://github.com/libp2p/go-libp2p-quic-transport/pull/131))\n - use a stateless reset key derived from the private key ([libp2p/go-libp2p-quic-transport#122](https://github.com/libp2p/go-libp2p-quic-transport/pull/122))\n - add support for multiaddr filtering ([libp2p/go-libp2p-quic-transport#125](https://github.com/libp2p/go-libp2p-quic-transport/pull/125))\n - use the resolved address for RemoteMultiaddr() ([libp2p/go-libp2p-quic-transport#127](https://github.com/libp2p/go-libp2p-quic-transport/pull/127))\n - accept a PSK in the transport constructor (and reject it) ([libp2p/go-libp2p-quic-transport#111](https://github.com/libp2p/go-libp2p-quic-transport/pull/111))\n - update quic-go to v0.15.0 ([libp2p/go-libp2p-quic-transport#114](https://github.com/libp2p/go-libp2p-quic-transport/pull/114))\n - increase the stream and connection receive windows ([libp2p/go-libp2p-quic-transport#108](https://github.com/libp2p/go-libp2p-quic-transport/pull/108))\n - fix key comparisons in tests ([libp2p/go-libp2p-quic-transport#110](https://github.com/libp2p/go-libp2p-quic-transport/pull/110))\n - make reuse work on Windows ([libp2p/go-libp2p-quic-transport#83](https://github.com/libp2p/go-libp2p-quic-transport/pull/83))\n - add a LICENSE ([libp2p/go-libp2p-quic-transport#78](https://github.com/libp2p/go-libp2p-quic-transport/pull/78))\n - Use specific netlink families for android ([libp2p/go-libp2p-quic-transport#75](https://github.com/libp2p/go-libp2p-quic-transport/pull/75))\n - implement a garbage-collector for unused reuse connections ([libp2p/go-libp2p-quic-transport#73](https://github.com/libp2p/go-libp2p-quic-transport/pull/73))\n - implement connection reuse ([libp2p/go-libp2p-quic-transport#63](https://github.com/libp2p/go-libp2p-quic-transport/pull/63))\n - update the README ([libp2p/go-libp2p-quic-transport#69](https://github.com/libp2p/go-libp2p-quic-transport/pull/69))\n - use the handshake logic from go-libp2p-tls ([libp2p/go-libp2p-quic-transport#67](https://github.com/libp2p/go-libp2p-quic-transport/pull/67))\n - update quic-go to v0.12.0 (supporting QUIC draft-22) ([libp2p/go-libp2p-quic-transport#68](https://github.com/libp2p/go-libp2p-quic-transport/pull/68))\n - when ListenUDP fails once, try again next time ([libp2p/go-libp2p-quic-transport#59](https://github.com/libp2p/go-libp2p-quic-transport/pull/59))\n- github.com/libp2p/go-libp2p-record (v0.0.1 -> v0.1.2):\n - readme: replace IPFS contrib links with libp2p ([libp2p/go-libp2p-record#25](https://github.com/libp2p/go-libp2p-record/pull/25))\n - Use peer ID utilities to go from pubkey to peer ID ([libp2p/go-libp2p-record#26](https://github.com/libp2p/go-libp2p-record/pull/26))\n- github.com/libp2p/go-libp2p-routing-helpers (v0.0.2 -> v0.2.2):\n - doc: document all types ([libp2p/go-libp2p-routing-helpers#40](https://github.com/libp2p/go-libp2p-routing-helpers/pull/40))\n - fix: fetch all providers when count is 0 ([libp2p/go-libp2p-routing-helpers#39](https://github.com/libp2p/go-libp2p-routing-helpers/pull/39))\n - feat: implement io.Closer ([libp2p/go-libp2p-routing-helpers#37](https://github.com/libp2p/go-libp2p-routing-helpers/pull/37))\n - readme: replace IPFS contrib links with libp2p ([libp2p/go-libp2p-routing-helpers#21](https://github.com/libp2p/go-libp2p-routing-helpers/pull/21))\n- github.com/libp2p/go-libp2p-secio (v0.0.3 -> v0.2.2):\n - feat: remove sha1 hmac ([libp2p/go-libp2p-secio#64](https://github.com/libp2p/go-libp2p-secio/pull/64))\n - readme: add context and links ([libp2p/go-libp2p-secio#55](https://github.com/libp2p/go-libp2p-secio/pull/55))\n - Update to latest go-libp2p-core, update tests ([libp2p/go-libp2p-secio#54](https://github.com/libp2p/go-libp2p-secio/pull/54))\n - Remove support for blowfish ([libp2p/go-libp2p-secio#52](https://github.com/libp2p/go-libp2p-secio/pull/52))\n - fix: wait for handshake to complete before returning ([libp2p/go-libp2p-secio#50](https://github.com/libp2p/go-libp2p-secio/pull/50))\n - avoid holding the message writer longer than necessary ([libp2p/go-libp2p-secio#49](https://github.com/libp2p/go-libp2p-secio/pull/49))\n- github.com/libp2p/go-libp2p-swarm (v0.0.7 -> v0.2.3):\n - don't expire backoffs until 2x backoff period ([libp2p/go-libp2p-swarm#193](https://github.com/libp2p/go-libp2p-swarm/pull/193))\n - fix: slightly simplify backoff logic ([libp2p/go-libp2p-swarm#192](https://github.com/libp2p/go-libp2p-swarm/pull/192))\n - change backoffs to per-address ([libp2p/go-libp2p-swarm#191](https://github.com/libp2p/go-libp2p-swarm/pull/191))\n - fix: set teardown after storing the context ([libp2p/go-libp2p-swarm#190](https://github.com/libp2p/go-libp2p-swarm/pull/190))\n - feat: handle no addresses ([libp2p/go-libp2p-swarm#185](https://github.com/libp2p/go-libp2p-swarm/pull/185))\n - fix: make sure to include peer in dial error ([libp2p/go-libp2p-swarm#180](https://github.com/libp2p/go-libp2p-swarm/pull/180))\n - Don't drop connections when simultaneous dialing occurs ([libp2p/go-libp2p-swarm#174](https://github.com/libp2p/go-libp2p-swarm/pull/174))\n - fix: fire a listen close event when closing the listener ([libp2p/go-libp2p-swarm#164](https://github.com/libp2p/go-libp2p-swarm/pull/164))\n - Link to godocs for Host instead of deprecated repo ([libp2p/go-libp2p-swarm#137](https://github.com/libp2p/go-libp2p-swarm/pull/137))\n - improve dial errors ([libp2p/go-libp2p-swarm#145](https://github.com/libp2p/go-libp2p-swarm/pull/145))\n - Minor Docstring correction ([libp2p/go-libp2p-swarm#143](https://github.com/libp2p/go-libp2p-swarm/pull/143))\n - test: close peerstore when closing the test swarm ([libp2p/go-libp2p-swarm#139](https://github.com/libp2p/go-libp2p-swarm/pull/139))\n - fix listen addrs race ([libp2p/go-libp2p-swarm#136](https://github.com/libp2p/go-libp2p-swarm/pull/136))\n - logging: make the swarm less noisy ([libp2p/go-libp2p-swarm#131](https://github.com/libp2p/go-libp2p-swarm/pull/131))\n - feat: cache interface addresses for 1 minute ([libp2p/go-libp2p-swarm#129](https://github.com/libp2p/go-libp2p-swarm/pull/129))\n- github.com/libp2p/go-libp2p-tls (v0.0.2 -> v0.1.3):\n - Readme: link to the libp2p-core docs ([libp2p/go-libp2p-tls#36](https://github.com/libp2p/go-libp2p-tls/pull/36))\n - expose the function to derive the peer's public key from the cert chain ([libp2p/go-libp2p-tls#33](https://github.com/libp2p/go-libp2p-tls/pull/33))\n - set an ALPN value in the tls.Config ([libp2p/go-libp2p-tls#32](https://github.com/libp2p/go-libp2p-tls/pull/32))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.0.4 -> v0.2.0):\n - use the ipnet.PSK instead of the ipnet.Protector for private networks ([libp2p/go-libp2p-transport-upgrader#45](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/45))\n - readme: add context & fix example code ([libp2p/go-libp2p-transport-upgrader#26](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/26))\n - fix an incorrect error message ([libp2p/go-libp2p-transport-upgrader#27](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/27))\n - Consolidate abstractions and core types into go-libp2p-core (#28) ([libp2p/go-libp2p-transport-upgrader#22](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/22))\n- github.com/libp2p/go-libp2p-yamux (v0.1.3 -> v0.2.7):\n - Respect mux.ErrReset ([libp2p/go-libp2p-yamux#10](https://github.com/libp2p/go-libp2p-yamux/pull/10))\n- github.com/libp2p/go-maddr-filter (v0.0.4 -> v0.0.5):\n - fix: check for blocked addrs without allocating ([libp2p/go-maddr-filter#14](https://github.com/libp2p/go-maddr-filter/pull/14))\n- github.com/libp2p/go-mplex (v0.0.4 -> v0.1.2):\n - remove deprecated log.Warning(f) ([libp2p/go-mplex#65](https://github.com/libp2p/go-mplex/pull/65))\n - Remove dependency on go-libp2p-core and introduce new errors. ([libp2p/go-mplex#72](https://github.com/libp2p/go-mplex/pull/72))\n - Bump lodash from 4.17.5 to 4.17.15 in /interop/js ([libp2p/go-mplex#66](https://github.com/libp2p/go-mplex/pull/66))\n - add test for deadlines ([libp2p/go-mplex#60](https://github.com/libp2p/go-mplex/pull/60))\n- github.com/libp2p/go-msgio (v0.0.2 -> v0.0.4):\n - make the maximum message size configurable ([libp2p/go-msgio#15](https://github.com/libp2p/go-msgio/pull/15))\n - combine writes and avoid a few more allocations ([libp2p/go-msgio#14](https://github.com/libp2p/go-msgio/pull/14))\n - avoid allocating unless we need to ([libp2p/go-msgio#13](https://github.com/libp2p/go-msgio/pull/13))\n- github.com/libp2p/go-nat (v0.0.3 -> v0.0.5):\n - feat: switch to go-netroute ([libp2p/go-nat#19](https://github.com/libp2p/go-nat/pull/19))\n - fix: really obey the context ([libp2p/go-nat#13](https://github.com/libp2p/go-nat/pull/13))\n - don't mask context ([libp2p/go-nat#10](https://github.com/libp2p/go-nat/pull/10))\n- github.com/libp2p/go-reuseport-transport (v0.0.2 -> v0.0.3):\n - fix: less confusing log message ([libp2p/go-reuseport-transport#22](https://github.com/libp2p/go-reuseport-transport/pull/22))\n - readme: replace IPFS contrib links with libp2p ([libp2p/go-reuseport-transport#16](https://github.com/libp2p/go-reuseport-transport/pull/16))\n - replace gx instructions with note about gomod ([libp2p/go-reuseport-transport#15](https://github.com/libp2p/go-reuseport-transport/pull/15))\n- github.com/libp2p/go-tcp-transport (v0.0.4 -> v0.2.0):\n - fix: don't allow dialing DNS addresses ([libp2p/go-tcp-transport#61](https://github.com/libp2p/go-tcp-transport/pull/61))\n - Use new constructor for insecure transport in tests ([libp2p/go-tcp-transport#42](https://github.com/libp2p/go-tcp-transport/pull/42))\n - readme: add install, usage & addressing info ([libp2p/go-tcp-transport#41](https://github.com/libp2p/go-tcp-transport/pull/41))\n- github.com/libp2p/go-ws-transport (v0.0.6 -> v0.3.1):\n - fix: add read/write locks ([libp2p/go-ws-transport#85](https://github.com/libp2p/go-ws-transport/pull/85))\n - fix: restrict dials to IP + TCP ([libp2p/go-ws-transport#84](https://github.com/libp2p/go-ws-transport/pull/84))\n - Revert \"add mutex for write/close\" ([libp2p/go-ws-transport#73](https://github.com/libp2p/go-ws-transport/pull/73))\n - feat: faster copy in wasm ([libp2p/go-ws-transport#68](https://github.com/libp2p/go-ws-transport/pull/68))\n - Add WebAssembly support and the ability to Dial from browsers ([libp2p/go-ws-transport#55](https://github.com/libp2p/go-ws-transport/pull/55))\n - fix: close gracefully ([libp2p/go-ws-transport#54](https://github.com/libp2p/go-ws-transport/pull/54))\n - move multiaddr protocol definitions to go-multiaddr ([libp2p/go-ws-transport#52](https://github.com/libp2p/go-ws-transport/pull/52))\n - Add install, usage & addressing info to README ([libp2p/go-ws-transport#49](https://github.com/libp2p/go-ws-transport/pull/49))\n- github.com/libp2p/go-yamux (v1.2.3 -> v1.3.5):\n - fix: synchronize when resetting the keepalive timer ([libp2p/go-yamux#21](https://github.com/libp2p/go-yamux/pull/21))\n - fix: don't keepalive when the connection is busy ([libp2p/go-yamux#16](https://github.com/libp2p/go-yamux/pull/16))\n - Rename errors ([libp2p/go-yamux#14](https://github.com/libp2p/go-yamux/pull/14))\n - fix(stream): set writeDeadline when cleanup and forceClose ([libp2p/go-yamux#12](https://github.com/libp2p/go-yamux/pull/12))\n - fixes a stream deadlock multiple ways ([libp2p/go-yamux#8](https://github.com/libp2p/go-yamux/pull/8))\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|----------------------------|---------|---------------|---------------|\n| Steven Allen | 858 | +27833/-15919 | 1906 |\n| Dirk McCormick | 134 | +18058/-8347 | 282 |\n| Aarsh Shah | 83 | +13458/-11883 | 241 |\n| Adin Schmahmann | 144 | +11878/-6236 | 397 |\n| Raúl Kripalani | 94 | +6894/-10214 | 598 |\n| vyzo | 60 | +8923/-1160 | 102 |\n| Will Scott | 79 | +3776/-1467 | 175 |\n| Michael Muré | 29 | +1734/-3290 | 104 |\n| dependabot[bot] | 365 | +3419/-361 | 728 |\n| Hector Sanjuan | 64 | +2053/-1321 | 132 |\n| Marten Seemann | 52 | +1922/-1268 | 147 |\n| Michael Avila | 29 | +828/-1733 | 70 |\n| Peter Rabbitson | 53 | +1073/-1197 | 100 |\n| Yusef Napora | 36 | +1610/-378 | 57 |\n| hannahhoward | 16 | +1342/-559 | 61 |\n| Łukasz Magiera | 9 | +277/-1623 | 41 |\n| Marcin Rataj | 9 | +1686/-99 | 32 |\n| Will | 7 | +936/-709 | 34 |\n| Alex Browne | 27 | +1019/-503 | 46 |\n| David Dias | 30 | +987/-431 | 43 |\n| Jakub Sztandera | 43 | +912/-436 | 77 |\n| Cole Brown | 21 | +646/-398 | 57 |\n| Oli Evans | 29 | +488/-466 | 43 |\n| Cornelius Toole | 3 | +827/-60 | 20 |\n| Hlib | 15 | +331/-185 | 28 |\n| Adrian Lanzafame | 9 | +123/-334 | 18 |\n| Petar Maymounkov | 1 | +385/-48 | 5 |\n| Alan Shaw | 18 | +262/-146 | 35 |\n| lnykww | 1 | +303/-52 | 6 |\n| Hannah Howard | 1 | +198/-27 | 3 |\n| Dominic Della Valle | 9 | +163/-52 | 14 |\n| Adam Uhlir | 1 | +211/-2 | 3 |\n| Dimitris Apostolou | 1 | +105/-105 | 64 |\n| Frrist | 1 | +186/-18 | 5 |\n| Henrique Dias | 22 | +119/-28 | 22 |\n| Gergely Tabiczky | 5 | +74/-60 | 7 |\n| Matt Joiner | 2 | +63/-62 | 4 |\n| @RubenKelevra | 12 | +46/-55 | 12 |\n| whyrusleeping | 6 | +87/-11 | 7 |\n| deepakgarg | 4 | +42/-43 | 4 |\n| protolambda | 2 | +49/-17 | 9 |\n| hucg | 2 | +47/-11 | 3 |\n| Arber Avdullahu | 3 | +31/-27 | 3 |\n| Sameer Puri | 1 | +46/-4 | 2 |\n| Hucg | 3 | +17/-33 | 3 |\n| Guilhem Fanton | 2 | +29/-10 | 7 |\n| Christian Muehlhaeuser | 6 | +20/-19 | 14 |\n| Djalil Dreamski | 3 | +27/-9 | 3 |\n| Caian | 2 | +36/-0 | 2 |\n| Topper Bowers | 2 | +31/-4 | 4 |\n| flowed | 1 | +16/-16 | 11 |\n| Vibhav Pant | 4 | +21/-10 | 5 |\n| frrist | 1 | +26/-4 | 1 |\n| Hlib Kanunnikov | 1 | +25/-3 | 1 |\n| george xie | 3 | +12/-15 | 11 |\n| optman | 1 | +13/-9 | 1 |\n| Roman Proskuryakov | 1 | +11/-11 | 2 |\n| Vasco Santos | 1 | +10/-10 | 5 |\n| Pretty Please Mark Darkly | 2 | +16/-2 | 2 |\n| Piotr Dyraga | 2 | +15/-2 | 2 |\n| Andrew Nesbitt | 1 | +5/-11 | 5 |\n| postables | 4 | +19/-8 | 4 |\n| Jim McDonald | 2 | +13/-1 | 2 |\n| PoorPockets McNewHold | 1 | +12/-0 | 1 |\n| Henri S | 1 | +6/-6 | 1 |\n| Igor Velkov | 1 | +8/-3 | 1 |\n| swedneck | 4 | +7/-3 | 4 |\n| Devin | 2 | +5/-5 | 4 |\n| iulianpascalau | 1 | +5/-3 | 2 |\n| MollyM | 3 | +7/-1 | 3 |\n| Jorropo | 2 | +5/-3 | 3 |\n| lukesolo | 1 | +6/-1 | 2 |\n| Wes Morgan | 1 | +3/-3 | 1 |\n| Kishan Mohanbhai Sagathiya | 1 | +3/-3 | 2 |\n| songjiayang | 1 | +4/-0 | 1 |\n| Terry Ding | 1 | +2/-2 | 1 |\n| Preston Van Loon | 2 | +3/-1 | 2 |\n| Jim Pick | 2 | +2/-2 | 2 |\n| Jakub Kaczmarzyk | 1 | +2/-2 | 1 |\n| Simon Menke | 2 | +2/-1 | 2 |\n| Jessica Schilling | 2 | +1/-2 | 2 |\n| Edgar Aroutiounian | 1 | +2/-1 | 1 |\n| hikerpig | 1 | +1/-1 | 1 |\n| ZenGround0 | 1 | +1/-1 | 1 |\n| Thomas Preindl | 1 | +1/-1 | 1 |\n| Sander Pick | 1 | +1/-1 | 1 |\n| Ronsor | 1 | +1/-1 | 1 |\n| Roman Khafizianov | 1 | +1/-1 | 1 |\n| Rod Vagg | 1 | +1/-1 | 1 |\n| Max Inden | 1 | +1/-1 | 1 |\n| Leo Arias | 1 | +1/-1 | 1 |\n| Kuro1 | 1 | +1/-1 | 1 |\n| Kirill Goncharov | 1 | +1/-1 | 1 |\n| John B Nelson | 1 | +1/-1 | 1 |\n| George Masgras | 1 | +1/-1 | 1 |\n| Aliabbas Merchant | 1 | +1/-1 | 1 |\n| Lorenzo Setale | 1 | +1/-0 | 1 |\n| Boris Mann | 1 | +1/-0 | 1 |\n" }, { "path": "docs/changelogs/v0.6.md", "content": "# go-ipfs changelog v0.6\n\n## v0.6.0 2020-06-19\n\nThis is a relatively small release in terms of code changes, but it contains some significant changes to the IPFS protocol.\n\n### Highlights\n\nThe highlights in this release include:\n\n* The QUIC transport is enabled by default. Furthermore, go-ipfs will automatically run a migration to listen on the QUIC transport (on the same address/port as the TCP transport) to make this upgrade process seamless.\n* The new NOISE security transport is now supported but won't be selected by default. This transport will replace SECIO as the default cross-language interoperability security transport. TLS 1.3 will still remain the default security transport between go-ipfs nodes for now.\n\n**MIGRATION:** This release contains a small config migration to enable listening on the QUIC transport in addition the TCP transport. This migration will:\n\n* Normalize multiaddrs in the bootstrap list to use the `/p2p/Qm...` syntax for multiaddrs instead of the `/ipfs/Qm...` syntax.\n* Add QUIC addresses for the default bootstrappers, as necessary. If you've removed the default bootstrappers from your bootstrap config, the migration won't add them back.\n* Add a QUIC listener address to mirror any TCP addresses present in your config. For example, if you're listening on `/ip4/0.0.0.0/tcp/1234`, this migration will add a listen address for `/ip4/0.0.0.0/udp/1234/quic`.\n\n#### QUIC by default\n\nThis release enables the QUIC transport (draft 28) by default for both inbound and outbound connections. When connecting to new peers, libp2p will continue to dial all advertised addresses (tcp + quic) in parallel so if the QUIC connection fails for some reason, the connection should still succeed.\n\nThe QUIC transport has several key benefits over the current TCP based transports:\n\n* It takes fewer round-trips to establish a connection. With the QUIC transport, the IPFS handshake takes two round trips (one to establish the QUIC connection, one for the libp2p handshake). In the future, we should be able to reduce this to one round trip for the initial connection, and zero round trips for subsequent connections to a previously seen peer. This is especially important for DHT requests that contact many new peers.\n* Because it's UDP based instead of TCP based, it uses fewer file descriptors. The QUIC transport will open one UDP socket per listen address instead of one socket per connection. This should, in the future, allow us to keep more connections open.\n* Because QUIC connections don't consume file descriptors, we're able to remove the rate limit on outbound QUIC connections, further speeding up DHT queries.\n\nUnfortunately, this change isn't without drawbacks: the QUIC transport may not be able to max out some links (usually due to [poorly tuned kernel parameters](https://github.com/lucas-clemente/quic-go/issues/2586#issuecomment-639247615)). On the other hand, it may also be _faster_ in some cases\n\nIf you hit this performance issue on Linux, you should tune the `net.core.rmem_default` and `net.core.rmem_max` sysctl parameters to increase your UDP receive buffer sizes.\n\nIf necessary, you can disable the QUIC transport by running:\n\n```bash\n> ipfs config --json Swarm.Transports.Network.QUIC false\n```\n\n**NOTE:** The QUIC transport included in this release is backwards incompatible with the experimental QUIC transport included in previous releases. Unfortunately, the QUIC protocol underwent some significant breaking changes and supporting multiple versions wasn't an option. In practice this degrades gracefully as go-ipfs will simply fall back on the TCP transport when dialing nodes with incompatible QUIC versions.\n\n#### Noise Transport\n\nThis go-ipfs release introduces a new security transport: [libp2p Noise](https://github.com/libp2p/specs/tree/master/noise) (built from the [Noise Protocol Framework](http://www.noiseprotocol.org/)). While TLS1.3 remains the default go-ipfs security transport, Noise is simpler to implement from scratch and will be the standard cross-platform libp2p security transport going forward.\n\nThis brings us one step closer to deprecating and removing support for SECIO.\n\nWhile enabled by default, Noise won't actually be _used_ by default it's negotiated. Given that TLS1.3 is still the default security transport for go-ipfs, this usually won't happen. If you'd like to prefer Noise over other security transports, you can change its priority in the [config](./docs/config.md) (`Swarm.Transports.Security.Noise`).\n\n#### Gateway\n\nThis release brings two gateway-relevant features: custom 404 pages and base36 support.\n\n##### Custom 404\n\nYou can now customize `404 Not Found` error pages by including an `ipfs-404.html` file somewhere in the request path. When a requested file isn't found, go-ipfs will look for an `ipfs-404.html` in the same directory as the requested file, and in each ancestor directory. If found, this file will be returned (with a 404 status code) instead of the usual error message.\n\n##### Support for Base36\n\nThis release adds support for a new multibase encoding: base36. Base36 is an optimally efficient case-insensitive alphanumeric encoding. Case-insensitive alphanumeric encodings are important for the subdomain gateway as domain names are case insensitive.\n\nWhile base32 (the current default encoding used in subdomains) is simpler than base36, it's not optimally efficient and base36 Ed25519 IPNS keys are 2 characters too big to fit into the 63 character subdomain length limit. The extra efficiency from base36 brings us under this limit and allows Ed25519 IPNS keys to work with the subdomain gateway.\n\nThis release adds support for base36 but won't use it by default. If you'd like to re-encode an Ed25519 IPNS key into base36, you can use the `ipfs cid format` command:\n\n```sh\n$ ipfs cid format -v 1 --codec libp2p-key -b base36 bafzaajaiaejca4syrpdu6gdx4wsdnokxkprgzxf4wrstuc34gxw5k5jrag2so5gk k51qzi5uqu5dj16qyiq0tajolkojyl9qdkr254920wxv7ghtuwcz593tp69z9m\n```\n\n#### Gossipsub Upgrade\n\nThis release brings a new gossipsub protocol version: 1.1. You can read about it in the [blog post](https://blog.ipfs.io/2020-05-20-gossipsub-v1.1/).\n\n#### Connectivity\n\nThis release introduces a new [\"peering\"](./docs/config.md#peering) feature. The peering subsystem configures go-ipfs to connect to, remain connected to, and reconnect to a set of nodes. Nodes should use this subsystem to create \"sticky\" links between frequently useful peers to improve reliability.\n\nUse-cases:\n\n* An IPFS gateway connected to an IPFS cluster should peer to ensure that the gateway can always fetch content from the cluster.\n* A dapp may peer embedded go-ipfs nodes with a set of pinning services or textile cafes/hubs.\n* A set of friends may peer to ensure that they can always fetch each other's content.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - fix 3 bugs responsible for a goroutine leak (plus one other bug) ([ipfs/go-ipfs#7491](https://github.com/ipfs/go-ipfs/pull/7491))\n - docs(config): update toc ([ipfs/go-ipfs#7483](https://github.com/ipfs/go-ipfs/pull/7483))\n - feat: transport config ([ipfs/go-ipfs#7479](https://github.com/ipfs/go-ipfs/pull/7479))\n - fix the minimal go version under 'Build from Source' ([ipfs/go-ipfs#7459](https://github.com/ipfs/go-ipfs/pull/7459))\n - fix(migration): migrate /ipfs/ bootstrappers to /p2p/\n - fix(migration): correctly migrate quic addresses\n - chore: add migration to listen on QUIC by default\n - backport fixes ([ipfs/go-ipfs#7405](https://github.com/ipfs/go-ipfs/pull/7405))\n - Use bitswap sessions for `ipfs refs`.\n - Update to webui 2.9.0\n - feat: add noise support ([ipfs/go-ipfs#7365](https://github.com/ipfs/go-ipfs/pull/7365))\n - feat: implement peering service ([ipfs/go-ipfs#7362](https://github.com/ipfs/go-ipfs/pull/7362))\n - Include the git blob id of the dir-index bundle in the ETag ([ipfs/go-ipfs#7360](https://github.com/ipfs/go-ipfs/pull/7360))\n - feat: bootstrap in dht when the routing table is empty ([ipfs/go-ipfs#7340](https://github.com/ipfs/go-ipfs/pull/7340))\n - quic: remove experimental status and add it to the default config ([ipfs/go-ipfs#7349](https://github.com/ipfs/go-ipfs/pull/7349))\n - fix: support directory listings even if a 404 page is present ([ipfs/go-ipfs#7339](https://github.com/ipfs/go-ipfs/pull/7339))\n - doc(plugin): document plugin config ([ipfs/go-ipfs#7309](https://github.com/ipfs/go-ipfs/pull/7309))\n - test(sharness): fix fuse tests ([ipfs/go-ipfs#7320](https://github.com/ipfs/go-ipfs/pull/7320))\n - docs: update experimental-features doc with IPNS over pubsub changes. ([ipfs/go-ipfs#7334](https://github.com/ipfs/go-ipfs/pull/7334))\n - docs: cleanup config formatting ([ipfs/go-ipfs#7336](https://github.com/ipfs/go-ipfs/pull/7336))\n - fix(gateway): ensure directory listings have Content-Type text/html ([ipfs/go-ipfs#7330](https://github.com/ipfs/go-ipfs/pull/7330))\n - test(sharness): test the local symlink ([ipfs/go-ipfs#7332](https://github.com/ipfs/go-ipfs/pull/7332))\n - misc config/experimental-features doc fixes ([ipfs/go-ipfs#7333](https://github.com/ipfs/go-ipfs/pull/7333))\n - fix: correctly trim resolved IPNS addresses ([ipfs/go-ipfs#7331](https://github.com/ipfs/go-ipfs/pull/7331))\n - Gateway renders pretty 404 pages if available ([ipfs/go-ipfs#4233](https://github.com/ipfs/go-ipfs/pull/4233))\n - feat: add a dht stat command ([ipfs/go-ipfs#7221](https://github.com/ipfs/go-ipfs/pull/7221))\n - fix: update dists url for OpenBSD support ([ipfs/go-ipfs#7311](https://github.com/ipfs/go-ipfs/pull/7311))\n - docs: X-Forwarded-Proto: https ([ipfs/go-ipfs#7306](https://github.com/ipfs/go-ipfs/pull/7306))\n - fix(mkreleaselog): make robust against running in different working directories ([ipfs/go-ipfs#7310](https://github.com/ipfs/go-ipfs/pull/7310))\n - fix(mkreleasenotes): include commits directly to master ([ipfs/go-ipfs#7296](https://github.com/ipfs/go-ipfs/pull/7296))\n - write api file automatically ([ipfs/go-ipfs#7282](https://github.com/ipfs/go-ipfs/pull/7282))\n - systemd: disable swap-usage for ipfs ([ipfs/go-ipfs#7299](https://github.com/ipfs/go-ipfs/pull/7299))\n - systemd: add helptext ([ipfs/go-ipfs#7265](https://github.com/ipfs/go-ipfs/pull/7265))\n - systemd: add the link to the docs ([ipfs/go-ipfs#7287](https://github.com/ipfs/go-ipfs/pull/7287))\n - systemd: add state directory setting ([ipfs/go-ipfs#7288](https://github.com/ipfs/go-ipfs/pull/7288))\n - Update go version required to build ([ipfs/go-ipfs#7289](https://github.com/ipfs/go-ipfs/pull/7289))\n - pin: implement pin/ls with only CoreApi ([ipfs/go-ipfs#6774](https://github.com/ipfs/go-ipfs/pull/6774))\n - update go-libp2p-quic-transport to v0.3.7 ([ipfs/go-ipfs#7278](https://github.com/ipfs/go-ipfs/pull/7278))\n - Docs: Delete section headers for removed features ([ipfs/go-ipfs#7277](https://github.com/ipfs/go-ipfs/pull/7277))\n - README.md: typo ([ipfs/go-ipfs#7061](https://github.com/ipfs/go-ipfs/pull/7061))\n - PR autocomment: Only comment for first-time contributors ([ipfs/go-ipfs#7270](https://github.com/ipfs/go-ipfs/pull/7270))\n - Fixed typo in config.md ([ipfs/go-ipfs#7267](https://github.com/ipfs/go-ipfs/pull/7267))\n - Fixes #7252 - Uses gabriel-vasile/mimetype to support additional content types ([ipfs/go-ipfs#7262](https://github.com/ipfs/go-ipfs/pull/7262))\n - update go-libp2p-quic-transport to v0.3.6 ([ipfs/go-ipfs#7266](https://github.com/ipfs/go-ipfs/pull/7266))\n - Updates bash completions to be compatible with zsh ([ipfs/go-ipfs#7261](https://github.com/ipfs/go-ipfs/pull/7261))\n - systemd service enhancements + run as system user ([ipfs/go-ipfs#7259](https://github.com/ipfs/go-ipfs/pull/7259))\n - upgrade to go 1.14.2 ([ipfs/go-ipfs#7130](https://github.com/ipfs/go-ipfs/pull/7130))\n - Add module files for go-ipfs-as-a-library example ([ipfs/go-ipfs#7146](https://github.com/ipfs/go-ipfs/pull/7146))\n - feat(gateway): show the absolute path and CID every time ([ipfs/go-ipfs#7219](https://github.com/ipfs/go-ipfs/pull/7219))\n - fix: do not use hard coded IPNS Publish maximum timeout duration ([ipfs/go-ipfs#7256](https://github.com/ipfs/go-ipfs/pull/7256))\n - Auto-comment on submitted PRs ([ipfs/go-ipfs#7248](https://github.com/ipfs/go-ipfs/pull/7248))\n - Fixes GitHub link. ([ipfs/go-ipfs#7239](https://github.com/ipfs/go-ipfs/pull/7239))\n - docs: fix subdomain examples in CHANGELOG ([ipfs/go-ipfs#7240](https://github.com/ipfs/go-ipfs/pull/7240))\n - doc: add snap to the release checklist ([ipfs/go-ipfs#7253](https://github.com/ipfs/go-ipfs/pull/7253))\n - Welcome message for users opening their first issue ([ipfs/go-ipfs#7247](https://github.com/ipfs/go-ipfs/pull/7247))\n - feat: bump to 0.6.0-dev ([ipfs/go-ipfs#7249](https://github.com/ipfs/go-ipfs/pull/7249))\n- github.com/ipfs/go-bitswap (v0.2.13 -> v0.2.19):\n - fix want gauge calculation ([ipfs/go-bitswap#416](https://github.com/ipfs/go-bitswap/pull/416))\n - Fix PeerManager signalAvailabiity() race ([ipfs/go-bitswap#417](https://github.com/ipfs/go-bitswap/pull/417))\n - fix: avoid taking accessing the peerQueues without taking the lock ([ipfs/go-bitswap#412](https://github.com/ipfs/go-bitswap/pull/412))\n - fix: update circleci ci-go ([ipfs/go-bitswap#396](https://github.com/ipfs/go-bitswap/pull/396))\n - fix: only track useful received data in the ledger (#411) ([ipfs/go-bitswap#411](https://github.com/ipfs/go-bitswap/pull/411))\n - If peer is first to send a block to session, protect connection ([ipfs/go-bitswap#406](https://github.com/ipfs/go-bitswap/pull/406))\n - Ensure sessions register with PeerManager ([ipfs/go-bitswap#405](https://github.com/ipfs/go-bitswap/pull/405))\n - Total wants gauge (#402) ([ipfs/go-bitswap#402](https://github.com/ipfs/go-bitswap/pull/402))\n - Improve peer manager performance ([ipfs/go-bitswap#395](https://github.com/ipfs/go-bitswap/pull/395))\n - fix: return wants from engine.WantlistForPeer() ([ipfs/go-bitswap#390](https://github.com/ipfs/go-bitswap/pull/390))\n - Add autocomment configuration\n - calculate message latency ([ipfs/go-bitswap#386](https://github.com/ipfs/go-bitswap/pull/386))\n - fix: use one less go-routine per session (#377) ([ipfs/go-bitswap#377](https://github.com/ipfs/go-bitswap/pull/377))\n - Add standard issue template\n- github.com/ipfs/go-cid (v0.0.5 -> v0.0.6):\n - feat: add Filecoin multicodecs ([ipfs/go-cid#104](https://github.com/ipfs/go-cid/pull/104))\n - Add autocomment configuration\n - avoid calling the method WriteTo if we don't satisfy its contract ([ipfs/go-cid#103](https://github.com/ipfs/go-cid/pull/103))\n - add a couple useful methods ([ipfs/go-cid#102](https://github.com/ipfs/go-cid/pull/102))\n - Add standard issue template\n- github.com/ipfs/go-fs-lock (v0.0.4 -> v0.0.5):\n - chore: remove xerrors ([ipfs/go-fs-lock#15](https://github.com/ipfs/go-fs-lock/pull/15))\n - Add autocomment configuration\n - Add standard issue template\n- github.com/ipfs/go-ipfs-cmds (v0.2.2 -> v0.2.9):\n - build(deps): bump github.com/ipfs/go-log from 1.0.3 to 1.0.4 ([ipfs/go-ipfs-cmds#194](https://github.com/ipfs/go-ipfs-cmds/pull/194))\n - Fix go-ipfs#7242: Remove \"HEAD\" from Allow methods ([ipfs/go-ipfs-cmds#195](https://github.com/ipfs/go-ipfs-cmds/pull/195))\n - Staticcheck fixes (#196) ([ipfs/go-ipfs-cmds#196](https://github.com/ipfs/go-ipfs-cmds/pull/196))\n - doc: update docs for interface changes ([ipfs/go-ipfs-cmds#197](https://github.com/ipfs/go-ipfs-cmds/pull/197))\n - Add standard issue template\n- github.com/ipfs/go-ipfs-config (v0.5.3 -> v0.8.0):\n - feat: add a transports section for enabling/disabling transports ([ipfs/go-ipfs-config#102](https://github.com/ipfs/go-ipfs-config/pull/102))\n - feat: add an option for security transport experiments ([ipfs/go-ipfs-config#97](https://github.com/ipfs/go-ipfs-config/pull/97))\n - feat: add peering service config section ([ipfs/go-ipfs-config#96](https://github.com/ipfs/go-ipfs-config/pull/96))\n - fix: include key size in key init method ([ipfs/go-ipfs-config#95](https://github.com/ipfs/go-ipfs-config/pull/95))\n - QUIC: remove experimental config option ([ipfs/go-ipfs-config#93](https://github.com/ipfs/go-ipfs-config/pull/93))\n - fix bootstrap peers ([ipfs/go-ipfs-config#94](https://github.com/ipfs/go-ipfs-config/pull/94))\n - default config: add QUIC listening ports + quic to mars.i.ipfs.io ([ipfs/go-ipfs-config#91](https://github.com/ipfs/go-ipfs-config/pull/91))\n - feat: remove strict signing pubsub option. ([ipfs/go-ipfs-config#90](https://github.com/ipfs/go-ipfs-config/pull/90))\n - Add autocomment configuration\n - Add Init Alternative allowing specification of ED25519 key ([ipfs/go-ipfs-config#78](https://github.com/ipfs/go-ipfs-config/pull/78))\n- github.com/ipfs/go-mfs (v0.1.1 -> v0.1.2):\n - Fix incorrect mutex unlock call in File.Open ([ipfs/go-mfs#82](https://github.com/ipfs/go-mfs/pull/82))\n - Add autocomment configuration\n - Add standard issue template\n - test: add Directory.ListNames test ([ipfs/go-mfs#81](https://github.com/ipfs/go-mfs/pull/81))\n - doc: add a lead maintainer\n - Update README.md with newer travis badge ([ipfs/go-mfs#78](https://github.com/ipfs/go-mfs/pull/78))\n- github.com/ipfs/interface-go-ipfs-core (v0.2.7 -> v0.3.0):\n - add Pin.IsPinned(..) ([ipfs/interface-go-ipfs-core#50](https://github.com/ipfs/interface-go-ipfs-core/pull/50))\n - Add autocomment configuration\n - Add standard issue template\n - extra time for dht spin-up ([ipfs/interface-go-ipfs-core#61](https://github.com/ipfs/interface-go-ipfs-core/pull/61))\n - feat: make the CoreAPI expose a streaming pin interface ([ipfs/interface-go-ipfs-core#49](https://github.com/ipfs/interface-go-ipfs-core/pull/49))\n - test: fail early on err to avoid an unrelated panic ([ipfs/interface-go-ipfs-core#57](https://github.com/ipfs/interface-go-ipfs-core/pull/57))\n- github.com/jbenet/go-is-domain (v1.0.3 -> v1.0.5):\n - Add OpenNIC domains to extended TLDs. ([jbenet/go-is-domain#15](https://github.com/jbenet/go-is-domain/pull/15))\n - feat: add .crypto and .zil from UnstoppableDomains ([jbenet/go-is-domain#17](https://github.com/jbenet/go-is-domain/pull/17))\n - chore: update IANA TLDs to version 2020051300 ([jbenet/go-is-domain#18](https://github.com/jbenet/go-is-domain/pull/18))\n- github.com/libp2p/go-addr-util (v0.0.1 -> v0.0.2):\n - fix discuss badge\n - add discuss link to readme\n - fix: fdcostly should take only the prefix into account ([libp2p/go-addr-util#5](https://github.com/libp2p/go-addr-util/pull/5))\n - add gomod support // tag v0.0.1 ([libp2p/go-addr-util#17](https://github.com/libp2p/go-addr-util/pull/17))\n- github.com/libp2p/go-libp2p (v0.8.3 -> v0.9.6):\n - fix(nat): use the right addresses when nat port mapping ([libp2p/go-libp2p#966](https://github.com/libp2p/go-libp2p/pull/966))\n - chore: update deps ([libp2p/go-libp2p#967](https://github.com/libp2p/go-libp2p/pull/967))\n - Fix peer handler race ([libp2p/go-libp2p#965](https://github.com/libp2p/go-libp2p/pull/965))\n - optimize numInbound count ([libp2p/go-libp2p#960](https://github.com/libp2p/go-libp2p/pull/960))\n - update go-libp2p-circuit ([libp2p/go-libp2p#962](https://github.com/libp2p/go-libp2p/pull/962))\n - Chunking large Identify responses with Signed Records ([libp2p/go-libp2p#958](https://github.com/libp2p/go-libp2p/pull/958))\n - gomod: update dependencies ([libp2p/go-libp2p#959](https://github.com/libp2p/go-libp2p/pull/959))\n - fixed compilation error (#956) ([libp2p/go-libp2p#956](https://github.com/libp2p/go-libp2p/pull/956))\n - Filter Interface Addresses (#936) ([libp2p/go-libp2p#936](https://github.com/libp2p/go-libp2p/pull/936))\n - fix: remove old addresses in identify immediately ([libp2p/go-libp2p#953](https://github.com/libp2p/go-libp2p/pull/953))\n - fix flaky test (#952) ([libp2p/go-libp2p#952](https://github.com/libp2p/go-libp2p/pull/952))\n - fix: group observations by zeroing port ([libp2p/go-libp2p#949](https://github.com/libp2p/go-libp2p/pull/949))\n - fix: fix connection gater in transport constructor ([libp2p/go-libp2p#948](https://github.com/libp2p/go-libp2p/pull/948))\n - Fix potential flakiness in TestIDService ([libp2p/go-libp2p#945](https://github.com/libp2p/go-libp2p/pull/945))\n - make the {F=>f}iltersConnectionGater private. (#946) ([libp2p/go-libp2p#946](https://github.com/libp2p/go-libp2p/pull/946))\n - Filter observed addresses (#917) ([libp2p/go-libp2p#917](https://github.com/libp2p/go-libp2p/pull/917))\n - fix: don't try to marshal a nil record ([libp2p/go-libp2p#943](https://github.com/libp2p/go-libp2p/pull/943))\n - add test to demo missing peer records after listen ([libp2p/go-libp2p#941](https://github.com/libp2p/go-libp2p/pull/941))\n - fix: don't leak a goroutine if a peer connects and immediately disconnects ([libp2p/go-libp2p#942](https://github.com/libp2p/go-libp2p/pull/942))\n - no signed peer records for mocknets (#934) ([libp2p/go-libp2p#934](https://github.com/libp2p/go-libp2p/pull/934))\n - implement connection gating at the top level (#881) ([libp2p/go-libp2p#881](https://github.com/libp2p/go-libp2p/pull/881))\n - various identify fixes and nits (#922) ([libp2p/go-libp2p#922](https://github.com/libp2p/go-libp2p/pull/922))\n - Remove race between ID, Push & Delta (#907) ([libp2p/go-libp2p#907](https://github.com/libp2p/go-libp2p/pull/907))\n - fix a compilation error introduced in 077a818. (#919) ([libp2p/go-libp2p#919](https://github.com/libp2p/go-libp2p/pull/919))\n - exchange signed routing records in identify (#747) ([libp2p/go-libp2p#747](https://github.com/libp2p/go-libp2p/pull/747))\n- github.com/libp2p/go-libp2p-autonat (v0.2.2 -> v0.2.3):\n - react to incoming events ([libp2p/go-libp2p-autonat#65](https://github.com/libp2p/go-libp2p-autonat/pull/65))\n- github.com/libp2p/go-libp2p-blankhost (v0.1.4 -> v0.1.6):\n - subscribe connmgr to net notifications ([libp2p/go-libp2p-blankhost#45](https://github.com/libp2p/go-libp2p-blankhost/pull/45))\n - add WithConnectionManager option to blankhost ([libp2p/go-libp2p-blankhost#44](https://github.com/libp2p/go-libp2p-blankhost/pull/44))\n - Blank host should support signed records ([libp2p/go-libp2p-blankhost#42](https://github.com/libp2p/go-libp2p-blankhost/pull/42))\n- github.com/libp2p/go-libp2p-circuit (v0.2.2 -> v0.2.3):\n - Use a fixed connection manager weight for peers with relay connections ([libp2p/go-libp2p-circuit#119](https://github.com/libp2p/go-libp2p-circuit/pull/119))\n- github.com/libp2p/go-libp2p-connmgr (v0.2.1 -> v0.2.4):\n - Implement IsProtected interface ([libp2p/go-libp2p-connmgr#76](https://github.com/libp2p/go-libp2p-connmgr/pull/76))\n - decaying tags: support removal and closure. (#72) ([libp2p/go-libp2p-connmgr#72](https://github.com/libp2p/go-libp2p-connmgr/pull/72))\n - implement decaying tags. (#61) ([libp2p/go-libp2p-connmgr#61](https://github.com/libp2p/go-libp2p-connmgr/pull/61))\n- github.com/libp2p/go-libp2p-core (v0.5.3 -> v0.5.7):\n - connmgr: add IsProtected interface (#158) ([libp2p/go-libp2p-core#158](https://github.com/libp2p/go-libp2p-core/pull/158))\n - eventbus: add wildcard subscription type; getter to enumerate known types (#153) ([libp2p/go-libp2p-core#153](https://github.com/libp2p/go-libp2p-core/pull/153))\n - events: add a generic DHT event. (#154) ([libp2p/go-libp2p-core#154](https://github.com/libp2p/go-libp2p-core/pull/154))\n - decaying tags: support removal and closure. (#151) ([libp2p/go-libp2p-core#151](https://github.com/libp2p/go-libp2p-core/pull/151))\n - implement Stringer for network.{Direction,Connectedness,Reachability}. (#150) ([libp2p/go-libp2p-core#150](https://github.com/libp2p/go-libp2p-core/pull/150))\n - connmgr: introduce abstractions and functions for decaying tags. (#104) ([libp2p/go-libp2p-core#104](https://github.com/libp2p/go-libp2p-core/pull/104))\n - Interface to verify if a peer supports a protocol without making allocations. ([libp2p/go-libp2p-core#148](https://github.com/libp2p/go-libp2p-core/pull/148))\n - add connection gating interfaces and types. (#139) ([libp2p/go-libp2p-core#139](https://github.com/libp2p/go-libp2p-core/pull/139))\n- github.com/libp2p/go-libp2p-kad-dht (v0.7.11 -> v0.8.2):\n - feat: protect all peers in low buckets, tag everyone else with 5\n - fix: lookup context cancellation race condition ([libp2p/go-libp2p-kad-dht#656](https://github.com/libp2p/go-libp2p-kad-dht/pull/656))\n - fix: protect useful peers in low buckets ([libp2p/go-libp2p-kad-dht#634](https://github.com/libp2p/go-libp2p-kad-dht/pull/634))\n - Double the usefulness interval for peers in the Routing Table (#651) ([libp2p/go-libp2p-kad-dht#651](https://github.com/libp2p/go-libp2p-kad-dht/pull/651))\n - enhancement/remove-unused-variable ([libp2p/go-libp2p-kad-dht#633](https://github.com/libp2p/go-libp2p-kad-dht/pull/633))\n - Put back TestSelfWalkOnAddressChange ([libp2p/go-libp2p-kad-dht#648](https://github.com/libp2p/go-libp2p-kad-dht/pull/648))\n - Routing Table Refresh manager (#601) ([libp2p/go-libp2p-kad-dht#601](https://github.com/libp2p/go-libp2p-kad-dht/pull/601))\n - bootstrap empty RT and Optimize allocs when we discover new peers (#631) ([libp2p/go-libp2p-kad-dht#631](https://github.com/libp2p/go-libp2p-kad-dht/pull/631))\n - fix all flaky tests ([libp2p/go-libp2p-kad-dht#628](https://github.com/libp2p/go-libp2p-kad-dht/pull/628))\n - Update default concurrency parameter ([libp2p/go-libp2p-kad-dht#605](https://github.com/libp2p/go-libp2p-kad-dht/pull/605))\n - clean up a channel that was dangling ([libp2p/go-libp2p-kad-dht#620](https://github.com/libp2p/go-libp2p-kad-dht/pull/620))\n- github.com/libp2p/go-libp2p-kbucket (v0.4.1 -> v0.4.2):\n - Reduce allocs in AddPeer (#81) ([libp2p/go-libp2p-kbucket#81](https://github.com/libp2p/go-libp2p-kbucket/pull/81))\n - NPeersForCpl and collapse empty buckets (#77) ([libp2p/go-libp2p-kbucket#77](https://github.com/libp2p/go-libp2p-kbucket/pull/77))\n- github.com/libp2p/go-libp2p-peerstore (v0.2.3 -> v0.2.6):\n - fix two bugs in signed address handling ([libp2p/go-libp2p-peerstore#155](https://github.com/libp2p/go-libp2p-peerstore/pull/155))\n - addrbook: fix races ([libp2p/go-libp2p-peerstore#154](https://github.com/libp2p/go-libp2p-peerstore/pull/154))\n - Implement the FirstSupportedProtocol API. ([libp2p/go-libp2p-peerstore#147](https://github.com/libp2p/go-libp2p-peerstore/pull/147))\n- github.com/libp2p/go-libp2p-pubsub (v0.2.7 -> v0.3.1):\n - fix outbound constraint satisfaction in oversubscription pruning\n - Gossipsub v0.3.0\n - set sendTo to remote peer id in trace events ([libp2p/go-libp2p-pubsub#268](https://github.com/libp2p/go-libp2p-pubsub/pull/268))\n - make wire protocol message size configurable. (#261) ([libp2p/go-libp2p-pubsub#261](https://github.com/libp2p/go-libp2p-pubsub/pull/261))\n- github.com/libp2p/go-libp2p-pubsub-router (v0.2.1 -> v0.3.0):\n - feat: update pubsub ([libp2p/go-libp2p-pubsub-router#76](https://github.com/libp2p/go-libp2p-pubsub-router/pull/76))\n- github.com/libp2p/go-libp2p-quic-transport (v0.3.7 -> v0.5.1):\n - close the connection when it is refused by InterceptSecured ([libp2p/go-libp2p-quic-transport#157](https://github.com/libp2p/go-libp2p-quic-transport/pull/157))\n - gate QUIC connections via new ConnectionGater (#152) ([libp2p/go-libp2p-quic-transport#152](https://github.com/libp2p/go-libp2p-quic-transport/pull/152))\n- github.com/libp2p/go-libp2p-record (v0.1.2 -> v0.1.3):\n - feat: add a better record error ([libp2p/go-libp2p-record#39](https://github.com/libp2p/go-libp2p-record/pull/39))\n- github.com/libp2p/go-libp2p-swarm (v0.2.3 -> v0.2.6):\n - Configure private key for test swarm ([libp2p/go-libp2p-swarm#223](https://github.com/libp2p/go-libp2p-swarm/pull/223))\n - Rank Dial addresses (#212) ([libp2p/go-libp2p-swarm#212](https://github.com/libp2p/go-libp2p-swarm/pull/212))\n - implement connection gating support: intercept peer, address dials, upgraded conns (#201) ([libp2p/go-libp2p-swarm#201](https://github.com/libp2p/go-libp2p-swarm/pull/201))\n - fix: avoid calling AddChild after the process may shutdown. ([libp2p/go-libp2p-swarm#207](https://github.com/libp2p/go-libp2p-swarm/pull/207))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.2.0 -> v0.3.0):\n - call the connection gater when accepting connections and after crypto handshake (#55) ([libp2p/go-libp2p-transport-upgrader#55](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/55))\n- github.com/libp2p/go-openssl (v0.0.4 -> v0.0.5):\n - add binding for OBJ_create ([libp2p/go-openssl#5](https://github.com/libp2p/go-openssl/pull/5))\n- github.com/libp2p/go-yamux (v1.3.5 -> v1.3.7):\n - tighten lock around appending new chunks of read data in stream ([libp2p/go-yamux#28](https://github.com/libp2p/go-yamux/pull/28))\n - fix: unlock recvLock in all cases. ([libp2p/go-yamux#25](https://github.com/libp2p/go-yamux/pull/25))\n- github.com/lucas-clemente/quic-go (v0.15.7 -> v0.16.2):\n - make it possible to use the transport with both draft-28 and draft-29\n - update the ALPN for draft-29 ([lucas-clemente/quic-go#2600](https://github.com/lucas-clemente/quic-go/pull/2600))\n - update initial salts and test vectors for draft-29 ([lucas-clemente/quic-go#2587](https://github.com/lucas-clemente/quic-go/pull/2587))\n - rename the SERVER_BUSY error to CONNECTION_REFUSED ([lucas-clemente/quic-go#2596](https://github.com/lucas-clemente/quic-go/pull/2596))\n - reduce calls to time.Now() from the flow controller ([lucas-clemente/quic-go#2591](https://github.com/lucas-clemente/quic-go/pull/2591))\n - remove redundant parenthesis and type conversion in flow controller ([lucas-clemente/quic-go#2592](https://github.com/lucas-clemente/quic-go/pull/2592))\n - use the receipt of a Retry packet to get a first RTT estimate ([lucas-clemente/quic-go#2588](https://github.com/lucas-clemente/quic-go/pull/2588))\n - fix debug message when returning an early session ([lucas-clemente/quic-go#2594](https://github.com/lucas-clemente/quic-go/pull/2594))\n - fix closing of the http.Request.Body ([lucas-clemente/quic-go#2584](https://github.com/lucas-clemente/quic-go/pull/2584))\n - split PTO calculation into a separate function ([lucas-clemente/quic-go#2576](https://github.com/lucas-clemente/quic-go/pull/2576))\n - add a unit test using the ChaCha20 test vector from the draft ([lucas-clemente/quic-go#2585](https://github.com/lucas-clemente/quic-go/pull/2585))\n - fix seed generation in frame sorter tests ([lucas-clemente/quic-go#2583](https://github.com/lucas-clemente/quic-go/pull/2583))\n - make sure that ACK frames are bundled with data ([lucas-clemente/quic-go#2543](https://github.com/lucas-clemente/quic-go/pull/2543))\n - add a Changelog for v0.16 ([lucas-clemente/quic-go#2582](https://github.com/lucas-clemente/quic-go/pull/2582))\n - authenticate connection IDs ([lucas-clemente/quic-go#2567](https://github.com/lucas-clemente/quic-go/pull/2567))\n - don't switch to PTO mode after using early loss detection ([lucas-clemente/quic-go#2581](https://github.com/lucas-clemente/quic-go/pull/2581))\n - only create a single session for duplicate Initials ([lucas-clemente/quic-go#2580](https://github.com/lucas-clemente/quic-go/pull/2580))\n - fix broken unit test in ackhandler\n - update the ALPN tokens to draft-28 ([lucas-clemente/quic-go#2570](https://github.com/lucas-clemente/quic-go/pull/2570))\n - drop duplicate packets ([lucas-clemente/quic-go#2569](https://github.com/lucas-clemente/quic-go/pull/2569))\n - remove noisy log statement in frame sorter test ([lucas-clemente/quic-go#2571](https://github.com/lucas-clemente/quic-go/pull/2571))\n - fix flaky qlog unit tests ([lucas-clemente/quic-go#2572](https://github.com/lucas-clemente/quic-go/pull/2572))\n - implement the 3x amplification limit ([lucas-clemente/quic-go#2536](https://github.com/lucas-clemente/quic-go/pull/2536))\n - rewrite the frame sorter ([lucas-clemente/quic-go#2561](https://github.com/lucas-clemente/quic-go/pull/2561))\n - retire conn IDs with sequence numbers smaller than the currently active ([lucas-clemente/quic-go#2563](https://github.com/lucas-clemente/quic-go/pull/2563))\n - remove unused readOffset member variable in receiveStream ([lucas-clemente/quic-go#2559](https://github.com/lucas-clemente/quic-go/pull/2559))\n - fix int overflow when parsing the transport parameters ([lucas-clemente/quic-go#2564](https://github.com/lucas-clemente/quic-go/pull/2564))\n - use struct{} instead of bool in window update queue ([lucas-clemente/quic-go#2555](https://github.com/lucas-clemente/quic-go/pull/2555))\n - update the protobuf library to google.golang.org/protobuf/proto ([lucas-clemente/quic-go#2554](https://github.com/lucas-clemente/quic-go/pull/2554))\n - use the correct error code for crypto stream errors ([lucas-clemente/quic-go#2546](https://github.com/lucas-clemente/quic-go/pull/2546))\n - bundle small writes on streams ([lucas-clemente/quic-go#2538](https://github.com/lucas-clemente/quic-go/pull/2538))\n - reduce the length of the unprocessed packet chan in the session ([lucas-clemente/quic-go#2534](https://github.com/lucas-clemente/quic-go/pull/2534))\n - fix flaky session unit test ([lucas-clemente/quic-go#2537](https://github.com/lucas-clemente/quic-go/pull/2537))\n - add a send stream test that randomly acknowledges and loses data ([lucas-clemente/quic-go#2535](https://github.com/lucas-clemente/quic-go/pull/2535))\n - fix size calculation for version negotiation packets ([lucas-clemente/quic-go#2542](https://github.com/lucas-clemente/quic-go/pull/2542))\n - run all unit tests with race detector ([lucas-clemente/quic-go#2528](https://github.com/lucas-clemente/quic-go/pull/2528))\n - add support for the ChaCha20 interop test case ([lucas-clemente/quic-go#2517](https://github.com/lucas-clemente/quic-go/pull/2517))\n - fix buffer use after it was released when sending an INVALID_TOKEN error ([lucas-clemente/quic-go#2524](https://github.com/lucas-clemente/quic-go/pull/2524))\n - run the internal and http3 tests with race detector on Travis ([lucas-clemente/quic-go#2385](https://github.com/lucas-clemente/quic-go/pull/2385))\n - reset the PTO when dropping a packet number space ([lucas-clemente/quic-go#2527](https://github.com/lucas-clemente/quic-go/pull/2527))\n - stop the deadline timer in Stream.Read and Write ([lucas-clemente/quic-go#2519](https://github.com/lucas-clemente/quic-go/pull/2519))\n - don't reset pto_count on Initial ACKs ([lucas-clemente/quic-go#2513](https://github.com/lucas-clemente/quic-go/pull/2513))\n - fix all race conditions in the session tests ([lucas-clemente/quic-go#2525](https://github.com/lucas-clemente/quic-go/pull/2525))\n - make sure that the server's run loop returned when closing ([lucas-clemente/quic-go#2526](https://github.com/lucas-clemente/quic-go/pull/2526))\n - fix flaky proxy test ([lucas-clemente/quic-go#2522](https://github.com/lucas-clemente/quic-go/pull/2522))\n - stop the timer when the session's run loop returns ([lucas-clemente/quic-go#2516](https://github.com/lucas-clemente/quic-go/pull/2516))\n - make it more likely that a STREAM frame is bundled with the FIN ([lucas-clemente/quic-go#2504](https://github.com/lucas-clemente/quic-go/pull/2504))\n- github.com/multiformats/go-multiaddr (v0.2.1 -> v0.2.2):\n - absorb go-maddr-filter; rm stale Makefile targets; upgrade deps (#124) ([multiformats/go-multiaddr#124](https://github.com/multiformats/go-multiaddr/pull/124))\n- github.com/multiformats/go-multibase (v0.0.2 -> v0.0.3):\n - Base36 implementation ([multiformats/go-multibase#36](https://github.com/multiformats/go-multibase/pull/36))\n - Even more tests/benchmarks, less repetition in-code ([multiformats/go-multibase#34](https://github.com/multiformats/go-multibase/pull/34))\n - Beef up tests before adding new codec ([multiformats/go-multibase#32](https://github.com/multiformats/go-multibase/pull/32))\n - Remove GX, bump spec submodule, fix tests ([multiformats/go-multibase#31](https://github.com/multiformats/go-multibase/pull/31))\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------------------|---------|-------------|---------------|\n| vyzo | 224 | +8016/-2810 | 304 |\n| Marten Seemann | 87 | +6081/-2607 | 215 |\n| Steven Allen | 157 | +4763/-1628 | 266 |\n| Aarsh Shah | 33 | +4619/-1634 | 128 |\n| Dirk McCormick | 26 | +3596/-1156 | 69 |\n| Yusef Napora | 66 | +2622/-785 | 98 |\n| Raúl Kripalani | 24 | +2424/-782 | 61 |\n| Hector Sanjuan | 30 | +999/-177 | 61 |\n| Louis Thibault | 2 | +1111/-4 | 4 |\n| Will Scott | 15 | +717/-219 | 31 |\n| dependabot-preview[bot] | 53 | +640/-64 | 106 |\n| Michael Muré | 7 | +456/-213 | 17 |\n| David Dias | 11 | +426/-88 | 15 |\n| Peter Rabbitson | 11 | +254/-189 | 31 |\n| Lukasz Zimnoch | 9 | +361/-49 | 13 |\n| Jakub Sztandera | 4 | +157/-104 | 9 |\n| Rod Vagg | 1 | +91/-83 | 2 |\n| RubenKelevra | 13 | +84/-84 | 30 |\n| JP Hastings-Spital | 1 | +145/-0 | 2 |\n| Adin Schmahmann | 11 | +67/-37 | 15 |\n| Marcin Rataj | 11 | +41/-43 | 11 |\n| Tiger | 5 | +53/-8 | 6 |\n| Akira | 2 | +35/-19 | 2 |\n| Casey Chance | 2 | +31/-22 | 2 |\n| Alan Shaw | 1 | +44/-0 | 2 |\n| Jessica Schilling | 4 | +20/-19 | 7 |\n| Gowtham G | 4 | +22/-14 | 6 |\n| Jeromy Johnson | 3 | +24/-6 | 3 |\n| Edgar Aroutiounian | 3 | +16/-8 | 3 |\n| Peter Wu | 2 | +12/-9 | 2 |\n| Sawood Alam | 2 | +7/-7 | 2 |\n| Command | 1 | +12/-0 | 1 |\n| Eric Myhre | 1 | +9/-2 | 1 |\n| mawei | 2 | +5/-5 | 2 |\n| decanus | 1 | +5/-5 | 1 |\n| Ignacio Hagopian | 2 | +7/-2 | 2 |\n| Alfonso Montero | 1 | +1/-5 | 1 |\n| Volker Mische | 1 | +2/-2 | 1 |\n| Shotaro Yamada | 1 | +2/-1 | 1 |\n| Mark Gaiser | 1 | +1/-1 | 1 |\n| Johnny | 1 | +1/-1 | 1 |\n| Ganesh Prasad Kumble | 1 | +1/-1 | 1 |\n| Dominic Della Valle | 1 | +1/-1 | 1 |\n| Corbin Page | 1 | +1/-1 | 1 |\n| Bryan Stenson | 1 | +1/-1 | 1 |\n| Bernhard M. Wiedemann | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.7.md", "content": "# go-ipfs changelog v0.7\n\n## v0.7.0 2020-09-22\n\n### Highlights\n\n#### Secio is now disabled by default\n\nAs part of deprecating and removing support for the Secio security transport, we have disabled it by default. TLS1.3 will remain the default security transport with fallback to Noise. You can read more about the deprecation in the blog post, https://blog.ipfs.io/2020-08-07-deprecating-secio/. If you're running IPFS older than 0.5, this may start to impact your performance on the public network.\n\n#### Ed25519 keys are now used by default\n\nPreviously go-ipfs generated 2048 bit RSA keys for new nodes, but it will now use ed25519 keys by default. This will not affect any existing keys, but newly created keys will be ed25519 by default. The main benefit of using ed25519 keys over RSA is that ed25519 keys have an inline public key. This means that someone only needs your PeerId to verify things you've signed, which means we don't have to worry about storing those bulky RSA public keys.\n\n##### Rotating keys\n\nAlong with switching the default, we've added support for rotating keys. If you would like to change the key type of your IPFS node, you can now do so with the rotate command. **NOTE: This will affect your Peer Id, so be sure you want to do this!** Your existing identity key will be backed up in the Keystore.\n\n```bash\nipfs key rotate -o my-old-key -t ed25519\n```\n\n#### Key export/import\n\nWe've added commands to allow you to export and import keys from the IPFS Keystore to a local .key file. This does not apply to the IPFS identity key, `self`.\n\n```bash\nipfs key gen mykey\nipfs key export -o mykey.key mykey # ./.key is the default path\nipfs key import mykey mykey.key # on another node\n```\n\n#### IPNS paths now encode the key name as a base36 CIDv1 by default\n\nPreviously go-ipfs encoded the key names for IPNS paths as base58btc multihashes (e.g. Qmabc...). We now encode them as base36 encoded CIDv1s as defined in the [peerID spec](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#string-representation) (e.g. k51xyz...) which also deals with encoding of public keys. This is nice because it means that IPNS keys will by default be case-insensitive and that they will fit into DNS labels (e.g. k51xyz...ipns.localhost) and therefore that subdomain gateway redirections (e.g. from localhost:8080/ipns/{key} to {key}.ipns.localhost) will look better to users in the default case.\n\nMany commands will accept a `--ipns-base` option that allows changing command outputs to use a particular encoding (i.e. base58btc multihash, or CIDv1 encoded in any supported base)\n\n#### Multiaddresses now accept PeerIDs encoded as CIDv1\n\nIn preparation for eventually changing the default PeerID representation multiaddresses can now contain strings like `/p2p/k51xyz...` in addition to the default `/p2p/Qmabc...`. There is a corresponding `--peerid-base` option to many functions that output peerIDs.\n\n#### `dag stat`\n\nInitial support has been added for the `ipfs dag stat` command. Running this command will traverse the DAG for the given root CID and report statistics. By default, progress will be shown as the DAG is traversed. Supported statistics currently include DAG size and number of blocks.\n\n```bash\nipfs dag stat bafybeihpetclqvwb4qnmumvcn7nh4pxrtugrlpw4jgjpqicdxsv7opdm6e # the IPFS webui\nSize: 30362191, NumBlocks: 346\n```\n\n#### Plugin build changes\n\nWe have changed the build flags used by the official binary distributions on dist.ipfs.tech (or `/ipns/dist.ipfs.tech`) to use the simpler and more reliable `-trimpath` flag instead of the more complicated and brittle `-asmflags=all=-trimpath=\"$(GOPATH)\" -gcflags=all=-trimpath=\"$(GOPATH)\"` flags, however the build flags used by default in go-ipfs remain the same.\n\nThe scripts in https://github.com/ipfs/go-ipfs-example-plugin have been updated to reflect this change. This is a breaking change to how people have been building plugins against the dist.ipfs.tech binary of go-ipfs and plugins should update their build processes accordingly see https://github.com/ipfs/go-ipfs-example-plugin/pull/9 for details.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - chore: bump webui version\n - fix: remove the (empty) alias for --peerid-base\n - Release v0.7.0-rc2\n - fix: use override GOFLAGS changes from 480defab689610550ee3d346e31441a2bb881fcb but keep trimpath usage as is\n - Revert \"fix: override GOFLAGS\"\n - fix: remove the (empty) alias for --ipns-base\n - refactor: put all --ipns-base options in one place\n - docs: update config to indicate SECIO deprecation\n - fix: ipfs dht put/get commands now work on keys encoded as peerIDs and fail early for namespaces other than /pk or /ipns\n - Release v0.7.0-rc1\n - chore: cleanup ([ipfs/go-ipfs#7628](https://github.com/ipfs/go-ipfs/pull/7628))\n - namesys: fixed IPNS republisher to not overwrite IPNS record lifetimes ([ipfs/go-ipfs#7627](https://github.com/ipfs/go-ipfs/pull/7627))\n - Fix #7624: Do not fetch dag nodes when checking if a pin exists ([ipfs/go-ipfs#7625](https://github.com/ipfs/go-ipfs/pull/7625))\n - chore: update dependencies ([ipfs/go-ipfs#7610](https://github.com/ipfs/go-ipfs/pull/7610))\n - use t.Cleanup() to reduce the need to clean up servers in tests ([ipfs/go-ipfs#7550](https://github.com/ipfs/go-ipfs/pull/7550))\n - fix: ipfs pin ls - ignore pins that have errors ([ipfs/go-ipfs#7612](https://github.com/ipfs/go-ipfs/pull/7612))\n - docs(config): fix Peering header ([ipfs/go-ipfs#7623](https://github.com/ipfs/go-ipfs/pull/7623))\n - sharness: use dnsaddr example in ipfs p2p command tests ([ipfs/go-ipfs#7620](https://github.com/ipfs/go-ipfs/pull/7620))\n - fix(key): don't allow backup key to be named 'self' ([ipfs/go-ipfs#7615](https://github.com/ipfs/go-ipfs/pull/7615))\n - [BOUNTY] Directory page UI improvements ([ipfs/go-ipfs#7536](https://github.com/ipfs/go-ipfs/pull/7536))\n - fix: make assets deterministic ([ipfs/go-ipfs#7609](https://github.com/ipfs/go-ipfs/pull/7609))\n - use ed25519 keys by default ([ipfs/go-ipfs#7579](https://github.com/ipfs/go-ipfs/pull/7579))\n - feat: wildcard support for public gateways ([ipfs/go-ipfs#7319](https://github.com/ipfs/go-ipfs/pull/7319))\n - fix: fix go-bindata import path ([ipfs/go-ipfs#7605](https://github.com/ipfs/go-ipfs/pull/7605))\n - Upgrade graphsync deps ([ipfs/go-ipfs#7598](https://github.com/ipfs/go-ipfs/pull/7598))\n - Add --peerid-base to ipfs id command ([ipfs/go-ipfs#7591](https://github.com/ipfs/go-ipfs/pull/7591))\n - use b36 keys by default for keys and IPNS ([ipfs/go-ipfs#7582](https://github.com/ipfs/go-ipfs/pull/7582))\n - add ipfs dag stat command (#7553) ([ipfs/go-ipfs#7553](https://github.com/ipfs/go-ipfs/pull/7553))\n - Move key rotation command to ipfs key rotate ([ipfs/go-ipfs#7599](https://github.com/ipfs/go-ipfs/pull/7599))\n - Disable secio by default ([ipfs/go-ipfs#7600](https://github.com/ipfs/go-ipfs/pull/7600))\n - Stop searching for public keys before doing an IPNS Get (#7549) ([ipfs/go-ipfs#7549](https://github.com/ipfs/go-ipfs/pull/7549))\n - feat: return supported protocols in id output ([ipfs/go-ipfs#7409](https://github.com/ipfs/go-ipfs/pull/7409))\n - docs: fix typo in default swarm addrs config docs ([ipfs/go-ipfs#7585](https://github.com/ipfs/go-ipfs/pull/7585))\n - feat: nice errors when failing to load plugins ([ipfs/go-ipfs#7429](https://github.com/ipfs/go-ipfs/pull/7429))\n - doc: document reverse proxy bug ([ipfs/go-ipfs#7478](https://github.com/ipfs/go-ipfs/pull/7478))\n - fix: ipfs name resolve --dht-record-count flag uses correct type and now works\n - refactor: get rid of cmdDetails awkwardness\n - IPNS format keys in b36cid ([ipfs/go-ipfs#7554](https://github.com/ipfs/go-ipfs/pull/7554))\n - Key import and export cli commands ([ipfs/go-ipfs#7546](https://github.com/ipfs/go-ipfs/pull/7546))\n - feat: add snap package configuration ([ipfs/go-ipfs#7529](https://github.com/ipfs/go-ipfs/pull/7529))\n - chore: bump webui version\n - repeat gateway subdomain test for all key types (#7542) ([ipfs/go-ipfs#7542](https://github.com/ipfs/go-ipfs/pull/7542))\n - fix: override GOFLAGS\n - update QUIC, enable the RetireBugBackwardsCompatibilityMode\n - Document add behavior when the daemon is not running ([ipfs/go-ipfs#7514](https://github.com/ipfs/go-ipfs/pull/7514))\n - ([ipfs/go-ipfs#7515](https://github.com/ipfs/go-ipfs/pull/7515))\n - Choose Key type at initialization ([ipfs/go-ipfs#7251](https://github.com/ipfs/go-ipfs/pull/7251))\n - feat: add flag to ipfs key and list to output keys in b36/CIDv1 (#7531) ([ipfs/go-ipfs#7531](https://github.com/ipfs/go-ipfs/pull/7531))\n - feat: support ED25519 libp2p-key in subdomains\n - chore: fix a typo\n - docs: document X-Forwarded-Host\n - feat: support X-Forwarded-Host when doing gateway redirect\n - chore: update test deps for graphsync\n - chore: bump test dependencies ([ipfs/go-ipfs#7524](https://github.com/ipfs/go-ipfs/pull/7524))\n - fix: use static binaries in docker container ([ipfs/go-ipfs#7505](https://github.com/ipfs/go-ipfs/pull/7505))\n - chore:bump webui version to 2.10.1 ([ipfs/go-ipfs#7504](https://github.com/ipfs/go-ipfs/pull/7504))\n - chore: bump webui version ([ipfs/go-ipfs#7501](https://github.com/ipfs/go-ipfs/pull/7501))\n - update version to 0.7.0-dev\n - Merge branch 'release' into master\n - systemd: specify repo path, to avoid unnecessary subdirectory ([ipfs/go-ipfs#7472](https://github.com/ipfs/go-ipfs/pull/7472))\n - doc(prod): start documenting production stuff ([ipfs/go-ipfs#7469](https://github.com/ipfs/go-ipfs/pull/7469))\n - Readme: Update link about init systems (and import old readme) ([ipfs/go-ipfs#7473](https://github.com/ipfs/go-ipfs/pull/7473))\n - doc(config): expand peering docs ([ipfs/go-ipfs#7466](https://github.com/ipfs/go-ipfs/pull/7466))\n - fix: Use the -p option in Dockerfile to make parents as needed ([ipfs/go-ipfs#7464](https://github.com/ipfs/go-ipfs/pull/7464))\n - systemd: enable systemd hardening features ([ipfs/go-ipfs#7286](https://github.com/ipfs/go-ipfs/pull/7286))\n - fix(migration): migrate /ipfs/ bootstrappers to /p2p/ ([ipfs/go-ipfs#7450](https://github.com/ipfs/go-ipfs/pull/7450))\n - readme: update go-version ([ipfs/go-ipfs#7447](https://github.com/ipfs/go-ipfs/pull/7447))\n - fix(migration): correctly migrate quic addresses ([ipfs/go-ipfs#7446](https://github.com/ipfs/go-ipfs/pull/7446))\n - chore: add migration to listen on QUIC by default ([ipfs/go-ipfs#7443](https://github.com/ipfs/go-ipfs/pull/7443))\n - go: bump minimal dependency to 1.14.4 ([ipfs/go-ipfs#7419](https://github.com/ipfs/go-ipfs/pull/7419))\n - fix: use bitswap sessions for ipfs refs ([ipfs/go-ipfs#7389](https://github.com/ipfs/go-ipfs/pull/7389))\n - fix(commands): print consistent addresses in ipfs id ([ipfs/go-ipfs#7397](https://github.com/ipfs/go-ipfs/pull/7397))\n - fix two pubsub issues. ([ipfs/go-ipfs#7394](https://github.com/ipfs/go-ipfs/pull/7394))\n - docs: add pacman.store (@RubenKelevra) to the early testers ([ipfs/go-ipfs#7368](https://github.com/ipfs/go-ipfs/pull/7368))\n - Update docs-beta links to final URLs ([ipfs/go-ipfs#7386](https://github.com/ipfs/go-ipfs/pull/7386))\n - feat: webui v2.9.0 ([ipfs/go-ipfs#7387](https://github.com/ipfs/go-ipfs/pull/7387))\n - chore: update WebUI to 2.8.0 ([ipfs/go-ipfs#7380](https://github.com/ipfs/go-ipfs/pull/7380))\n - mailmap support ([ipfs/go-ipfs#7375](https://github.com/ipfs/go-ipfs/pull/7375))\n - doc: update the release template for git flow changes ([ipfs/go-ipfs#7370](https://github.com/ipfs/go-ipfs/pull/7370))\n - chore: update deps ([ipfs/go-ipfs#7369](https://github.com/ipfs/go-ipfs/pull/7369))\n- github.com/ipfs/go-bitswap (v0.2.19 -> v0.2.20):\n - fix: don't say we're sending a full wantlist unless we are (#429) ([ipfs/go-bitswap#429](https://github.com/ipfs/go-bitswap/pull/429))\n- github.com/ipfs/go-cid (v0.0.6 -> v0.0.7):\n - feat: optimize cid.Prefix ([ipfs/go-cid#109](https://github.com/ipfs/go-cid/pull/109))\n- github.com/ipfs/go-datastore (v0.4.4 -> v0.4.5):\n - Add test to ensure that Delete returns no error for missing keys ([ipfs/go-datastore#162](https://github.com/ipfs/go-datastore/pull/162))\n - Fix typo in sync/sync.go ([ipfs/go-datastore#159](https://github.com/ipfs/go-datastore/pull/159))\n - Add the generated flatfs stub, since it cannot be auto-generated ([ipfs/go-datastore#158](https://github.com/ipfs/go-datastore/pull/158))\n - support flatfs fuzzing ([ipfs/go-datastore#157](https://github.com/ipfs/go-datastore/pull/157))\n - fuzzing harness (#153) ([ipfs/go-datastore#153](https://github.com/ipfs/go-datastore/pull/153))\n - feat(mount): don't give up on error ([ipfs/go-datastore#146](https://github.com/ipfs/go-datastore/pull/146))\n - /test: fix bad ElemCount/10 length (should not be divided) ([ipfs/go-datastore#152](https://github.com/ipfs/go-datastore/pull/152))\n- github.com/ipfs/go-ds-flatfs (v0.4.4 -> v0.4.5):\n - Add os.Rename wrapper for Plan 9 (#87) ([ipfs/go-ds-flatfs#87](https://github.com/ipfs/go-ds-flatfs/pull/87))\n- github.com/ipfs/go-fs-lock (v0.0.5 -> v0.0.6):\n - Fix build on Plan 9 ([ipfs/go-fs-lock#17](https://github.com/ipfs/go-fs-lock/pull/17))\n- github.com/ipfs/go-graphsync (v0.0.5 -> v0.1.1):\n - docs(CHANGELOG): update for v0.1.1\n - docs(CHANGELOG): update for v0.1.0 release ([ipfs/go-graphsync#84](https://github.com/ipfs/go-graphsync/pull/84))\n - Deduplicate by key extension (#83) ([ipfs/go-graphsync#83](https://github.com/ipfs/go-graphsync/pull/83))\n - Release infrastructure (#81) ([ipfs/go-graphsync#81](https://github.com/ipfs/go-graphsync/pull/81))\n - feat(persistenceoptions): add unregister ability (#80) ([ipfs/go-graphsync#80](https://github.com/ipfs/go-graphsync/pull/80))\n - fix(message): regen protobuf code (#79) ([ipfs/go-graphsync#79](https://github.com/ipfs/go-graphsync/pull/79))\n - feat(requestmanager): run response hooks on completed requests (#77) ([ipfs/go-graphsync#77](https://github.com/ipfs/go-graphsync/pull/77))\n - Revert \"add extensions on complete (#76)\"\n - add extensions on complete (#76) ([ipfs/go-graphsync#76](https://github.com/ipfs/go-graphsync/pull/76))\n - All changes to date including pause requests & start paused, along with new adds for cleanups and checking of execution (#75) ([ipfs/go-graphsync#75](https://github.com/ipfs/go-graphsync/pull/75))\n - More fine grained response controls (#71) ([ipfs/go-graphsync#71](https://github.com/ipfs/go-graphsync/pull/71))\n - Refactor request execution and use IPLD SkipMe functionality for proper partial results on a request (#70) ([ipfs/go-graphsync#70](https://github.com/ipfs/go-graphsync/pull/70))\n - feat(graphsync): implement do-no-send-cids extension (#69) ([ipfs/go-graphsync#69](https://github.com/ipfs/go-graphsync/pull/69))\n - Incoming Block Hooks (#68) ([ipfs/go-graphsync#68](https://github.com/ipfs/go-graphsync/pull/68))\n - fix(responsemanager): add nil check (#67) ([ipfs/go-graphsync#67](https://github.com/ipfs/go-graphsync/pull/67))\n - refactor(hooks): use external pubsub (#65) ([ipfs/go-graphsync#65](https://github.com/ipfs/go-graphsync/pull/65))\n - Update of IPLD Prime (#66) ([ipfs/go-graphsync#66](https://github.com/ipfs/go-graphsync/pull/66))\n - feat(responsemanager): add listener for completed responses (#64) ([ipfs/go-graphsync#64](https://github.com/ipfs/go-graphsync/pull/64))\n - Update Requests (#63) ([ipfs/go-graphsync#63](https://github.com/ipfs/go-graphsync/pull/63))\n - Add pausing and unpausing of requests (#62) ([ipfs/go-graphsync#62](https://github.com/ipfs/go-graphsync/pull/62))\n - Outgoing Request Hooks, swapping persistence layers (#61) ([ipfs/go-graphsync#61](https://github.com/ipfs/go-graphsync/pull/61))\n - Feat/request hook loader chooser (#60) ([ipfs/go-graphsync#60](https://github.com/ipfs/go-graphsync/pull/60))\n - Option to Reject requests by default (#58) ([ipfs/go-graphsync#58](https://github.com/ipfs/go-graphsync/pull/58))\n - Testify refactor (#56) ([ipfs/go-graphsync#56](https://github.com/ipfs/go-graphsync/pull/56))\n - Switch To Circle CI (#57) ([ipfs/go-graphsync#57](https://github.com/ipfs/go-graphsync/pull/57))\n - fix(deps): go mod tidy\n - docs(README): remove ipldbridge reference\n - Tech Debt: Remove IPLD Bridge ([ipfs/go-graphsync#55](https://github.com/ipfs/go-graphsync/pull/55))\n- github.com/ipfs/go-ipfs-cmds (v0.2.9 -> v0.4.0):\n - fix: allow requests from electron renderer (#201) ([ipfs/go-ipfs-cmds#201](https://github.com/ipfs/go-ipfs-cmds/pull/201))\n - refactor: move external command checks into commands lib (#198) ([ipfs/go-ipfs-cmds#198](https://github.com/ipfs/go-ipfs-cmds/pull/198))\n - Fix build on Plan 9 ([ipfs/go-ipfs-cmds#199](https://github.com/ipfs/go-ipfs-cmds/pull/199))\n- github.com/ipfs/go-ipfs-config (v0.8.0 -> v0.9.0):\n - error if bit size specified with ed25519 keys (#105) ([ipfs/go-ipfs-config#105](https://github.com/ipfs/go-ipfs-config/pull/105))\n- github.com/ipfs/go-log/v2 (v2.0.8 -> v2.1.1):\n failed to fetch repo\n- github.com/ipfs/go-path (v0.0.7 -> v0.0.8):\n - ResolveToLastNode no longer fetches nodes it does not need ([ipfs/go-path#30](https://github.com/ipfs/go-path/pull/30))\n - doc: add a lead maintainer\n- github.com/ipfs/interface-go-ipfs-core (v0.3.0 -> v0.4.0):\n - Add ID formatting functions, used by various IPFS cli commands ([ipfs/interface-go-ipfs-core#65](https://github.com/ipfs/interface-go-ipfs-core/pull/65))\n- github.com/ipld/go-car (v0.1.0 -> v0.1.1-0.20200429200904-c222d793c339):\n - Update go-ipld-prime to the era of NodeAssembler. ([ipld/go-car#31](https://github.com/ipld/go-car/pull/31))\n - fix: update the cli tool's car dep ([ipld/go-car#30](https://github.com/ipld/go-car/pull/30))\n- github.com/ipld/go-ipld-prime (v0.0.2-0.20191108012745-28a82f04c785 -> v0.0.2-0.20200428162820-8b59dc292b8e):\n - Add two basic examples of usage, as go tests.\n - Fix marshalling error ([ipld/go-ipld-prime#53](https://github.com/ipld/go-ipld-prime/pull/53))\n - Add more test specs for list and map nesting.\n - traversal.SkipMe feature ([ipld/go-ipld-prime#51](https://github.com/ipld/go-ipld-prime/pull/51))\n - Improvements to traversal docs.\n - Drop code coverage bot config. ([ipld/go-ipld-prime#50](https://github.com/ipld/go-ipld-prime/pull/50))\n - Promote NodeAssembler/NodeStyle interface rework to core, and use improved basicnode implementation. ([ipld/go-ipld-prime#49](https://github.com/ipld/go-ipld-prime/pull/49))\n - Merge branch 'traversal-benchmarks'\n - Merge branch 'cycle-breaking-and-traversal-benchmarks'\n - Merge branch 'assembler-upgrade-to-codecs'\n - Path clarifications ([ipld/go-ipld-prime#47](https://github.com/ipld/go-ipld-prime/pull/47))\n - Merge branch 'research-admissions'\n - Add a typed link node to allow traversal with code generated builders across links ([ipld/go-ipld-prime#41](https://github.com/ipld/go-ipld-prime/pull/41))\n - Merge branch 'research-admissions'\n - Library updates.\n - Feat/add code gen disclaimer ([ipld/go-ipld-prime#39](https://github.com/ipld/go-ipld-prime/pull/39))\n - Readme and key Node interface docs improvements.\n - fix(schema/gen): return value not reference ([ipld/go-ipld-prime#38](https://github.com/ipld/go-ipld-prime/pull/38))\n- github.com/ipld/go-ipld-prime-proto (v0.0.0-20191113031812-e32bd156a1e5 -> v0.0.0-20200428191222-c1ffdadc01e1):\n - feat(deps): upgrade to new IPLD prime ([ipld/go-ipld-prime-proto#1](https://github.com/ipld/go-ipld-prime-proto/pull/1))\n - Update to latest ipld before rework ([ipld/go-ipld-prime-proto#2](https://github.com/ipld/go-ipld-prime-proto/pull/2))\n- github.com/libp2p/go-libp2p (v0.9.6 -> v0.11.0):\n - Added parsing of IPv6 addresses for incoming mDNS requests ([libp2p/go-libp2p#990](https://github.com/libp2p/go-libp2p/pull/990))\n - Switch from SECIO to Noise ([libp2p/go-libp2p#972](https://github.com/libp2p/go-libp2p/pull/972))\n - fix tests ([libp2p/go-libp2p#995](https://github.com/libp2p/go-libp2p/pull/995))\n - Bump Autonat version & validate fixed call loop in `.Addrs` (#988) ([libp2p/go-libp2p#988](https://github.com/libp2p/go-libp2p/pull/988))\n - fix: use the correct external address when NAT port-mapping ([libp2p/go-libp2p#987](https://github.com/libp2p/go-libp2p/pull/987))\n - upgrade deps + interoperable uvarint delimited writer/reader. (#985) ([libp2p/go-libp2p#985](https://github.com/libp2p/go-libp2p/pull/985))\n - fix host can be dialed by autonat public addr, but lost the public addr to announce ([libp2p/go-libp2p#983](https://github.com/libp2p/go-libp2p/pull/983))\n - Fix address advertisement bugs (#974) ([libp2p/go-libp2p#974](https://github.com/libp2p/go-libp2p/pull/974))\n - fix: avoid a close deadlock in the natmanager ([libp2p/go-libp2p#971](https://github.com/libp2p/go-libp2p/pull/971))\n - upgrade swarm; add ID() on mock conns and streams. (#970) ([libp2p/go-libp2p#970](https://github.com/libp2p/go-libp2p/pull/970))\n- github.com/libp2p/go-libp2p-asn-util (null -> v0.0.0-20200825225859-85005c6cf052):\n - chore: go fmt\n - feat: use deferred initialization of the asnStore ([libp2p/go-libp2p-asn-util#3](https://github.com/libp2p/go-libp2p-asn-util/pull/3))\n - chore: switch to forked cidranger\n - fixed code\n - library for ASN mappings\n- github.com/libp2p/go-libp2p-autonat (v0.2.3 -> v0.3.2):\n - static nat shouldn't call host.Addrs()\n - upgrade deps + interoperable uvarint delimited writer/reader. (#95) ([libp2p/go-libp2p-autonat#95](https://github.com/libp2p/go-libp2p-autonat/pull/95))\n - fix: a type switch nit ([libp2p/go-libp2p-autonat#83](https://github.com/libp2p/go-libp2p-autonat/pull/83))\n- github.com/libp2p/go-libp2p-blankhost (v0.1.6 -> v0.2.0):\n - call reset where appropriate (and update deps) ([libp2p/go-libp2p-blankhost#52](https://github.com/libp2p/go-libp2p-blankhost/pull/52))\n- github.com/libp2p/go-libp2p-circuit (v0.2.3 -> v0.3.1):\n - upgrade deps + interoperable uvarints. (#122) ([libp2p/go-libp2p-circuit#122](https://github.com/libp2p/go-libp2p-circuit/pull/122))\n - Fix/remove deprecated logging ([libp2p/go-libp2p-circuit#85](https://github.com/libp2p/go-libp2p-circuit/pull/85))\n- github.com/libp2p/go-libp2p-core (v0.5.7 -> v0.6.1):\n - experimental introspection support (#159) ([libp2p/go-libp2p-core#159](https://github.com/libp2p/go-libp2p-core/pull/159))\n- github.com/libp2p/go-libp2p-discovery (v0.4.0 -> v0.5.0):\n - Put period at end of sentence ([libp2p/go-libp2p-discovery#65](https://github.com/libp2p/go-libp2p-discovery/pull/65))\n- github.com/libp2p/go-libp2p-kad-dht (v0.8.2 -> v0.9.0):\n - chore: update deps ([libp2p/go-libp2p-kad-dht#689](https://github.com/libp2p/go-libp2p-kad-dht/pull/689))\n - allow overwriting builtin dual DHT options ([libp2p/go-libp2p-kad-dht#688](https://github.com/libp2p/go-libp2p-kad-dht/pull/688))\n - Hardening Improvements: RT diversity and decreased RT churn ([libp2p/go-libp2p-kad-dht#687](https://github.com/libp2p/go-libp2p-kad-dht/pull/687))\n - Fix key log encoding ([libp2p/go-libp2p-kad-dht#682](https://github.com/libp2p/go-libp2p-kad-dht/pull/682))\n - upgrade deps + uvarint delimited writer/reader. (#684) ([libp2p/go-libp2p-kad-dht#684](https://github.com/libp2p/go-libp2p-kad-dht/pull/684))\n - periodicBootstrapInterval should be ticker? (#678) ([libp2p/go-libp2p-kad-dht#678](https://github.com/libp2p/go-libp2p-kad-dht/pull/678))\n - removes duplicate comment ([libp2p/go-libp2p-kad-dht#674](https://github.com/libp2p/go-libp2p-kad-dht/pull/674))\n - Revert \"Peer Diversity in the Routing Table (#658)\" ([libp2p/go-libp2p-kad-dht#670](https://github.com/libp2p/go-libp2p-kad-dht/pull/670))\n - Fixed problem with refresh logging ([libp2p/go-libp2p-kad-dht#667](https://github.com/libp2p/go-libp2p-kad-dht/pull/667))\n - feat: protect all peers in low buckets, tag everyone else with 5 ([libp2p/go-libp2p-kad-dht#666](https://github.com/libp2p/go-libp2p-kad-dht/pull/666))\n - Peer Diversity in the Routing Table (#658) ([libp2p/go-libp2p-kad-dht#658](https://github.com/libp2p/go-libp2p-kad-dht/pull/658))\n- github.com/libp2p/go-libp2p-kbucket (v0.4.2 -> v0.4.7):\n - chore: switch from go-multiaddr-net to go-multiaddr/net\n - Use crypto/rand for generating random prefixes\n - feat: when using the diversity filter for ipv6 addresses if the ASN cannot be found for a particular address then fallback on using the /32 mask of the address as the group name instead of simply rejecting the peer from routing table\n - simplify filter (#92) ([libp2p/go-libp2p-kbucket#92](https://github.com/libp2p/go-libp2p-kbucket/pull/92))\n - fix: switch to forked cid ranger dep ([libp2p/go-libp2p-kbucket#91](https://github.com/libp2p/go-libp2p-kbucket/pull/91))\n - Reduce Routing Table churn (#90) ([libp2p/go-libp2p-kbucket#90](https://github.com/libp2p/go-libp2p-kbucket/pull/90))\n - Peer Diversity for Routing Table and Querying (#88) ([libp2p/go-libp2p-kbucket#88](https://github.com/libp2p/go-libp2p-kbucket/pull/88))\n - fix bug in peer eviction (#87) ([libp2p/go-libp2p-kbucket#87](https://github.com/libp2p/go-libp2p-kbucket/pull/87))\n - feat: add an AddedAt timestamp (#84) ([libp2p/go-libp2p-kbucket#84](https://github.com/libp2p/go-libp2p-kbucket/pull/84))\n- github.com/libp2p/go-libp2p-pubsub (v0.3.1 -> v0.3.5):\n - regenerate protobufs (#381) ([libp2p/go-libp2p-pubsub#381](https://github.com/libp2p/go-libp2p-pubsub/pull/381))\n - track validation time\n - fulfill promise as soon as a message begins validation\n - don't apply penalty in self origin rejections\n - add behaviour penalty threshold\n - Add String() method to Topic.\n - add regression test for issue 371\n - don't add direct peers to fanout\n - reference spec change in comment.\n - fix backoff slack time\n - use the heartbeat interval for slack time\n - add slack time to prune backoff clearance\n - fix: call the correct tracer function in FloodSubRouter.Leave (#373) ([libp2p/go-libp2p-pubsub#373](https://github.com/libp2p/go-libp2p-pubsub/pull/373))\n - downgrade trace buffer overflow log to debug\n - track topics in Reject/Duplicate/Deliver events\n - add topics to Reject/Duplicate/Deliver events\n - fix flaky test\n - refactor ip colocation factor computation that is common for score and inspection\n - better handling of intermediate topic score snapshots\n - disallow duplicate score inspectors\n - make peer score inspect function types aliases\n - extended peer score inspection\n - upgrade deps + interoperable uvarint delimited writer/reader.\n - Add warning about messageIDs\n - Signing policy + optional Signature, From and Seqno ([libp2p/go-libp2p-pubsub#359](https://github.com/libp2p/go-libp2p-pubsub/pull/359))\n - Update pubsub.go\n - Define a public error ErrSubscriptionCancelled.\n - only do PX on leave if PX was enabled in the node\n - drop warning about failure to open stream to a debug log\n - reinstate tagging (now protection) tests\n - disable tests for direct/mesh tags, we don't have an interface to query the connman yet\n - protect direct and mesh peers in the connection manager\n - feat: add direct connect ticks option\n- github.com/libp2p/go-libp2p-pubsub-router (v0.3.0 -> v0.3.2):\n - upgrade deps + interoperable uvarint delimited writer/reader. (#79) ([libp2p/go-libp2p-pubsub-router#79](https://github.com/libp2p/go-libp2p-pubsub-router/pull/79))\n- github.com/libp2p/go-libp2p-quic-transport (v0.6.0 -> v0.8.0):\n - update quic-go to v0.18.0 (#171) ([libp2p/go-libp2p-quic-transport#171](https://github.com/libp2p/go-libp2p-quic-transport/pull/171))\n- github.com/libp2p/go-libp2p-swarm (v0.2.6 -> v0.2.8):\n - slim down dependencies ([libp2p/go-libp2p-swarm#225](https://github.com/libp2p/go-libp2p-swarm/pull/225))\n - `ID()` method on connections and streams + record opening time (#224) ([libp2p/go-libp2p-swarm#224](https://github.com/libp2p/go-libp2p-swarm/pull/224))\n- github.com/libp2p/go-libp2p-testing (v0.1.1 -> v0.2.0):\n - Add net benchmark harness ([libp2p/go-libp2p-testing#21](https://github.com/libp2p/go-libp2p-testing/pull/21))\n - Update suite to check that streams respect mux.ErrReset. ([libp2p/go-libp2p-testing#16](https://github.com/libp2p/go-libp2p-testing/pull/16))\n- github.com/libp2p/go-maddr-filter (v0.0.5 -> v0.1.0):\n - deprecate this package; moved to multiformats/go-multiaddr. (#23) ([libp2p/go-maddr-filter#23](https://github.com/libp2p/go-maddr-filter/pull/23))\n - chore(dep): update ([libp2p/go-maddr-filter#18](https://github.com/libp2p/go-maddr-filter/pull/18))\n- github.com/libp2p/go-msgio (v0.0.4 -> v0.0.6):\n - interoperable uvarints. (#21) ([libp2p/go-msgio#21](https://github.com/libp2p/go-msgio/pull/21))\n - upgrade deps + interoperable uvarint delimited writer/reader. (#20) ([libp2p/go-msgio#20](https://github.com/libp2p/go-msgio/pull/20))\n- github.com/libp2p/go-netroute (v0.1.2 -> v0.1.3):\n - add Plan 9 support\n- github.com/libp2p/go-openssl (v0.0.5 -> v0.0.7):\n - make ed25519 less special ([libp2p/go-openssl#7](https://github.com/libp2p/go-openssl/pull/7))\n - Add required bindings to support openssl in libp2p-tls ([libp2p/go-openssl#6](https://github.com/libp2p/go-openssl/pull/6))\n- github.com/libp2p/go-reuseport (v0.0.1 -> v0.0.2):\n - Fix build on Plan 9 ([libp2p/go-reuseport#79](https://github.com/libp2p/go-reuseport/pull/79))\n - farewell gx; thanks for serving us well.\n - update readme badges\n - remove Jenkinsfile.\n- github.com/libp2p/go-reuseport-transport (v0.0.3 -> v0.0.4):\n - Update go-netroute and go-reuseport for Plan 9 support\n - Fix build on Plan 9\n- github.com/lucas-clemente/quic-go (v0.16.2 -> v0.18.0):\n - create a milestone version for v0.18.x\n - add Changelog entries for v0.17 ([lucas-clemente/quic-go#2726](https://github.com/lucas-clemente/quic-go/pull/2726))\n - regenerate the testdata certificate with SAN instead of CommonName ([lucas-clemente/quic-go#2723](https://github.com/lucas-clemente/quic-go/pull/2723))\n - make it possible to use multiple qtls versions at the same time, add support for Go 1.15 ([lucas-clemente/quic-go#2720](https://github.com/lucas-clemente/quic-go/pull/2720))\n - add fuzzing for transport parameters ([lucas-clemente/quic-go#2713](https://github.com/lucas-clemente/quic-go/pull/2713))\n - run golangci-lint on GitHub Actions ([lucas-clemente/quic-go#2700](https://github.com/lucas-clemente/quic-go/pull/2700))\n - disallow values above 2^60 for Config.MaxIncoming{Uni}Streams ([lucas-clemente/quic-go#2711](https://github.com/lucas-clemente/quic-go/pull/2711))\n - never send a value larger than 2^60 in MAX_STREAMS frames ([lucas-clemente/quic-go#2710](https://github.com/lucas-clemente/quic-go/pull/2710))\n - run the check for go generated files on GitHub Actions instead of Travis ([lucas-clemente/quic-go#2703](https://github.com/lucas-clemente/quic-go/pull/2703))\n - update QUIC draft version information in README ([lucas-clemente/quic-go#2715](https://github.com/lucas-clemente/quic-go/pull/2715))\n - remove Fuzzit badge from README ([lucas-clemente/quic-go#2714](https://github.com/lucas-clemente/quic-go/pull/2714))\n - use the correct return values in Fuzz() functions ([lucas-clemente/quic-go#2705](https://github.com/lucas-clemente/quic-go/pull/2705))\n - simplify the connection, rename it to sendConn ([lucas-clemente/quic-go#2707](https://github.com/lucas-clemente/quic-go/pull/2707))\n - update qpack to v0.2.0 ([lucas-clemente/quic-go#2704](https://github.com/lucas-clemente/quic-go/pull/2704))\n - remove redundant error check in the stream ([lucas-clemente/quic-go#2718](https://github.com/lucas-clemente/quic-go/pull/2718))\n - put back the packet buffer when parsing the connection ID fails ([lucas-clemente/quic-go#2708](https://github.com/lucas-clemente/quic-go/pull/2708))\n - update fuzzing code for oss-fuzz ([lucas-clemente/quic-go#2702](https://github.com/lucas-clemente/quic-go/pull/2702))\n - fix travis script ([lucas-clemente/quic-go#2701](https://github.com/lucas-clemente/quic-go/pull/2701))\n - remove Fuzzit from Travis config ([lucas-clemente/quic-go#2699](https://github.com/lucas-clemente/quic-go/pull/2699))\n - add a script to check if go generated files are correct ([lucas-clemente/quic-go#2692](https://github.com/lucas-clemente/quic-go/pull/2692))\n - only arm the application data PTO timer after the handshake is confirmed ([lucas-clemente/quic-go#2689](https://github.com/lucas-clemente/quic-go/pull/2689))\n - fix tracing of congestion state updates ([lucas-clemente/quic-go#2691](https://github.com/lucas-clemente/quic-go/pull/2691))\n - fix reading of flag values in integration tests ([lucas-clemente/quic-go#2690](https://github.com/lucas-clemente/quic-go/pull/2690))\n - remove ACK decimation ([lucas-clemente/quic-go#2599](https://github.com/lucas-clemente/quic-go/pull/2599))\n - add a metric for PTOs ([lucas-clemente/quic-go#2686](https://github.com/lucas-clemente/quic-go/pull/2686))\n - remove the H3_EARLY_RESPONSE error ([lucas-clemente/quic-go#2687](https://github.com/lucas-clemente/quic-go/pull/2687))\n - implement tracing for congestion state changes ([lucas-clemente/quic-go#2684](https://github.com/lucas-clemente/quic-go/pull/2684))\n - remove the N connection simulation from the Reno code ([lucas-clemente/quic-go#2682](https://github.com/lucas-clemente/quic-go/pull/2682))\n - remove the SSLR (slow start large reduction) experiment ([lucas-clemente/quic-go#2680](https://github.com/lucas-clemente/quic-go/pull/2680))\n - remove unused connectionStats counters from the Reno implementation ([lucas-clemente/quic-go#2683](https://github.com/lucas-clemente/quic-go/pull/2683))\n - add an integration test that randomly sets tracers ([lucas-clemente/quic-go#2679](https://github.com/lucas-clemente/quic-go/pull/2679))\n - privatize some methods in the congestion controller package ([lucas-clemente/quic-go#2681](https://github.com/lucas-clemente/quic-go/pull/2681))\n - fix out-of-bounds read when creating a multiplexed tracer ([lucas-clemente/quic-go#2678](https://github.com/lucas-clemente/quic-go/pull/2678))\n - run integration tests with qlog and metrics on CircleCI ([lucas-clemente/quic-go#2677](https://github.com/lucas-clemente/quic-go/pull/2677))\n - add a metric for closed connections ([lucas-clemente/quic-go#2676](https://github.com/lucas-clemente/quic-go/pull/2676))\n - trace packets that are sent outside of a connection ([lucas-clemente/quic-go#2675](https://github.com/lucas-clemente/quic-go/pull/2675))\n - trace dropped packets that are dropped before they are passed to any session ([lucas-clemente/quic-go#2670](https://github.com/lucas-clemente/quic-go/pull/2670))\n - add a metric for sent packets ([lucas-clemente/quic-go#2673](https://github.com/lucas-clemente/quic-go/pull/2673))\n - add a metric for lost packets ([lucas-clemente/quic-go#2672](https://github.com/lucas-clemente/quic-go/pull/2672))\n - simplify the Tracer interface by combining the TracerFor... methods ([lucas-clemente/quic-go#2671](https://github.com/lucas-clemente/quic-go/pull/2671))\n - add a metrics package using OpenCensus, trace connections ([lucas-clemente/quic-go#2646](https://github.com/lucas-clemente/quic-go/pull/2646))\n - add a multiplexer for the tracer ([lucas-clemente/quic-go#2665](https://github.com/lucas-clemente/quic-go/pull/2665))\n - introduce a type for stateless reset tokens ([lucas-clemente/quic-go#2668](https://github.com/lucas-clemente/quic-go/pull/2668))\n - log all reasons why a connection is closed ([lucas-clemente/quic-go#2669](https://github.com/lucas-clemente/quic-go/pull/2669))\n - add integration tests using faulty packet conns ([lucas-clemente/quic-go#2663](https://github.com/lucas-clemente/quic-go/pull/2663))\n - don't block sendQueue.Send() if the runloop already exited. ([lucas-clemente/quic-go#2656](https://github.com/lucas-clemente/quic-go/pull/2656))\n - move the SupportedVersions slice out of the wire.Header ([lucas-clemente/quic-go#2664](https://github.com/lucas-clemente/quic-go/pull/2664))\n - add a flag to disable conn ID generation and the check for retired conn IDs ([lucas-clemente/quic-go#2660](https://github.com/lucas-clemente/quic-go/pull/2660))\n - put the session in the packet handler map directly (for client sessions) ([lucas-clemente/quic-go#2667](https://github.com/lucas-clemente/quic-go/pull/2667))\n - don't send write error in CONNECTION_CLOSE frames ([lucas-clemente/quic-go#2666](https://github.com/lucas-clemente/quic-go/pull/2666))\n - reset the PTO count before setting the timer when dropping a PN space ([lucas-clemente/quic-go#2657](https://github.com/lucas-clemente/quic-go/pull/2657))\n - enforce that a connection ID is not retired in a packet that uses that connection ID ([lucas-clemente/quic-go#2651](https://github.com/lucas-clemente/quic-go/pull/2651))\n - don't retire the conn ID that's in use when receiving a retransmission ([lucas-clemente/quic-go#2652](https://github.com/lucas-clemente/quic-go/pull/2652))\n - fix flaky cancellation integration test ([lucas-clemente/quic-go#2649](https://github.com/lucas-clemente/quic-go/pull/2649))\n - fix crash when the qlog callbacks returns a nil io.WriteCloser ([lucas-clemente/quic-go#2648](https://github.com/lucas-clemente/quic-go/pull/2648))\n - fix flaky server test on Travis ([lucas-clemente/quic-go#2645](https://github.com/lucas-clemente/quic-go/pull/2645))\n - fix a typo in the logging package test suite\n - introduce type aliases in the logging package ([lucas-clemente/quic-go#2643](https://github.com/lucas-clemente/quic-go/pull/2643))\n - rename frame fields to the names used in the draft ([lucas-clemente/quic-go#2644](https://github.com/lucas-clemente/quic-go/pull/2644))\n - split the qlog package into a logging and a qlog package, use a tracer interface in the quic.Config ([lucas-clemente/quic-go#2638](https://github.com/lucas-clemente/quic-go/pull/2638))\n - fix HTTP request writing if the Request.Body reads data and returns EOF ([lucas-clemente/quic-go#2642](https://github.com/lucas-clemente/quic-go/pull/2642))\n - handle Version Negotiation packets in the session ([lucas-clemente/quic-go#2640](https://github.com/lucas-clemente/quic-go/pull/2640))\n - increase the packet size of the client's Initial packet ([lucas-clemente/quic-go#2634](https://github.com/lucas-clemente/quic-go/pull/2634))\n - introduce an assertion in the server ([lucas-clemente/quic-go#2637](https://github.com/lucas-clemente/quic-go/pull/2637))\n - use the new qtls interface for (re)storing app data with a session state ([lucas-clemente/quic-go#2631](https://github.com/lucas-clemente/quic-go/pull/2631))\n - remove buffering of HTTP requests ([lucas-clemente/quic-go#2626](https://github.com/lucas-clemente/quic-go/pull/2626))\n - remove superfluous parameters logged when not doing 0-RTT ([lucas-clemente/quic-go#2632](https://github.com/lucas-clemente/quic-go/pull/2632))\n - return an infinite bandwidth if the RTT is zero ([lucas-clemente/quic-go#2636](https://github.com/lucas-clemente/quic-go/pull/2636))\n - drop support for Go 1.13 ([lucas-clemente/quic-go#2628](https://github.com/lucas-clemente/quic-go/pull/2628))\n - remove superfluous handleResetStreamFrame method on the stream ([lucas-clemente/quic-go#2623](https://github.com/lucas-clemente/quic-go/pull/2623))\n - implement a token-bucket pacing algorithm ([lucas-clemente/quic-go#2615](https://github.com/lucas-clemente/quic-go/pull/2615))\n - gracefully handle concurrent stream writes and cancellations ([lucas-clemente/quic-go#2624](https://github.com/lucas-clemente/quic-go/pull/2624))\n - log sent packets right before sending them out ([lucas-clemente/quic-go#2613](https://github.com/lucas-clemente/quic-go/pull/2613))\n - remove unused packet counter in the receivedPacketTracker ([lucas-clemente/quic-go#2611](https://github.com/lucas-clemente/quic-go/pull/2611))\n - rewrite the proxy to avoid packet reordering ([lucas-clemente/quic-go#2617](https://github.com/lucas-clemente/quic-go/pull/2617))\n - fix flaky INVALID_TOKEN integration test ([lucas-clemente/quic-go#2610](https://github.com/lucas-clemente/quic-go/pull/2610))\n - make DialEarly return EarlySession ([lucas-clemente/quic-go#2621](https://github.com/lucas-clemente/quic-go/pull/2621))\n - add debug logging to the packet handler map ([lucas-clemente/quic-go#2608](https://github.com/lucas-clemente/quic-go/pull/2608))\n - increase the minimum pacing delay to 1ms ([lucas-clemente/quic-go#2605](https://github.com/lucas-clemente/quic-go/pull/2605))\n- github.com/marten-seemann/qpack (v0.1.0 -> v0.2.0):\n - don't reuse the encoder in the integration tests ([marten-seemann/qpack#18](https://github.com/marten-seemann/qpack/pull/18))\n - use Huffman encoding for field names and values ([marten-seemann/qpack#16](https://github.com/marten-seemann/qpack/pull/16))\n - add more tests for encoding using the static table ([marten-seemann/qpack#15](https://github.com/marten-seemann/qpack/pull/15))\n - Encoder uses the static table. ([marten-seemann/qpack#10](https://github.com/marten-seemann/qpack/pull/10))\n - add gofmt to golangci-lint\n - update qifs to the current version ([marten-seemann/qpack#14](https://github.com/marten-seemann/qpack/pull/14))\n - use golangci-lint for linting ([marten-seemann/qpack#12](https://github.com/marten-seemann/qpack/pull/12))\n - add fuzzing ([marten-seemann/qpack#9](https://github.com/marten-seemann/qpack/pull/9))\n - update qifs\n - use https protocol for submodule clone ([marten-seemann/qpack#7](https://github.com/marten-seemann/qpack/pull/7))\n- github.com/marten-seemann/qtls (v0.9.1 -> v0.10.0):\n - add callbacks to store and restore app data along a session state\n - remove support for Go 1.13\n- github.com/marten-seemann/qtls-go1-15 (null -> v0.1.0):\n - use a prefix for client session cache keys\n - add callbacks to store and restore app data along a session state\n - don't use TLS 1.3 compatibility mode when using alternative record layer\n - delete the session ticket after attempting 0-RTT\n - reject 0-RTT when a different ALPN is chosen\n - encode the ALPN into the session ticket\n - add a field to the ConnectionState to tell if 0-RTT was used\n - add a callback to tell the client about rejection of 0-RTT\n - don't offer 0-RTT after a HelloRetryRequest\n - add Accept0RTT to Config callback to decide if 0-RTT should be accepted\n - add the option to encode application data into the session ticket\n - export the 0-RTT write key\n - abuse the nonce field of ClientSessionState to save max_early_data_size\n - export the 0-RTT read key\n - close connection if client attempts 0-RTT, but ticket didn't allow it\n - encode the max early data size into the session ticket\n - implement parsing of the early_data extension in the EncryptedExtensions\n - add a tls.Config.MaxEarlyData option to enable 0-RTT\n - accept TLS 1.3 cipher suites in Config.CipherSuites\n - introduce a function on the connection to generate a session ticket\n - add a config option to enforce selection of an application protocol\n - export Conn.HandlePostHandshakeMessage\n - export Alert\n - reject Configs that set MaxVersion < 1.3 when using a record layer\n - enforce TLS 1.3 when using an alternative record layer\n- github.com/multiformats/go-multiaddr (v0.2.2 -> v0.3.1):\n - dep: add \"codependencies\" for handling version conflicts ([multiformats/go-multiaddr#132](https://github.com/multiformats/go-multiaddr/pull/132))\n - Support /p2p addresses encoded as CIDs ([multiformats/go-multiaddr#130](https://github.com/multiformats/go-multiaddr/pull/130))\n - Merge go-multiaddr-net\n- github.com/multiformats/go-multiaddr-net (v0.1.5 -> v0.2.0):\n - Deprecate ([multiformats/go-multiaddr-net#72](https://github.com/multiformats/go-multiaddr-net/pull/72))\n- github.com/multiformats/go-multihash (v0.0.13 -> v0.0.14):\n - fix: only register one blake2s length ([multiformats/go-multihash#129](https://github.com/multiformats/go-multihash/pull/129))\n - feat: add two filecoin hashes, without Sum() implementations ([multiformats/go-multihash#128](https://github.com/multiformats/go-multihash/pull/128))\n - feat: reduce blake2b allocations by special-casing the 256/512 variants ([multiformats/go-multihash#126](https://github.com/multiformats/go-multihash/pull/126))\n- github.com/multiformats/go-multistream (v0.1.1 -> v0.1.2):\n - upgrade deps + interoperable varints. (#51) ([multiformats/go-multistream#51](https://github.com/multiformats/go-multistream/pull/51))\n- github.com/multiformats/go-varint (v0.0.5 -> v0.0.6):\n - fix minor interoperability issues. (#6) ([multiformats/go-varint#6](https://github.com/multiformats/go-varint/pull/6))\n- github.com/warpfork/go-wish (v0.0.0-20190328234359-8b3e70f8e830 -> v0.0.0-20200122115046-b9ea61034e4a):\n - Add ShouldBeSameTypeAs checker.\n - Integration test update for go versions.\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20200123233031-1cdf64d27158 -> v0.0.0-20200402171437-3d27c146c105):\n - Handle Nil values for cbg.Deferred ([whyrusleeping/cbor-gen#14](https://github.com/whyrusleeping/cbor-gen/pull/14))\n - add name of struct field to error messages\n - Support uint64 pointers ([whyrusleeping/cbor-gen#13](https://github.com/whyrusleeping/cbor-gen/pull/13))\n - int64 support in map encoders ([whyrusleeping/cbor-gen#12](https://github.com/whyrusleeping/cbor-gen/pull/12))\n - Fix uint64 typed array gen ([whyrusleeping/cbor-gen#10](https://github.com/whyrusleeping/cbor-gen/pull/10))\n - Fix cbg self referencing import path ([whyrusleeping/cbor-gen#8](https://github.com/whyrusleeping/cbor-gen/pull/8))\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 156 | +16428/-42621 | 979 |\n| hannahhoward | 42 | +15132/-9819 | 467 |\n| Eric Myhre | 114 | +13709/-6898 | 586 |\n| Steven Allen | 55 | +1211/-2714 | 95 |\n| Adin Schmahmann | 54 | +1660/-783 | 117 |\n| Petar Maymounkov | 23 | +1677/-671 | 75 |\n| Aarsh Shah | 10 | +1926/-341 | 39 |\n| Raúl Kripalani | 17 | +1134/-537 | 53 |\n| Will | 1 | +841/-0 | 9 |\n| rendaw | 3 | +425/-195 | 12 |\n| Will Scott | 8 | +302/-229 | 15 |\n| vyzo | 22 | +345/-166 | 23 |\n| Fazlul Shahriar | 7 | +452/-44 | 19 |\n| Peter Rabbitson | 1 | +353/-118 | 5 |\n| Hector Sanjuan | 10 | +451/-3 | 14 |\n| Marcin Rataj | 9 | +298/-106 | 16 |\n| Łukasz Magiera | 4 | +329/-51 | 12 |\n| RubenKelevra | 9 | +331/-7 | 12 |\n| Michael Muré | 2 | +259/-69 | 6 |\n| jstordeur | 1 | +252/-2 | 5 |\n| Diederik Loerakker | 1 | +168/-35 | 7 |\n| Tiger | 3 | +138/-52 | 8 |\n| Kevin Neaton | 3 | +103/-21 | 9 |\n| Rod Vagg | 1 | +50/-40 | 4 |\n| Oli Evans | 4 | +60/-9 | 6 |\n| achingbrain | 4 | +30/-30 | 5 |\n| Cyril Fougeray | 2 | +34/-24 | 2 |\n| Luke Tucker | 1 | +31/-1 | 2 |\n| sandman | 2 | +23/-7 | 3 |\n| Alan Shaw | 1 | +18/-9 | 2 |\n| Jacob Heun | 4 | +13/-3 | 4 |\n| Jessica Schilling | 3 | +7/-7 | 3 |\n| Rafael Ramalho | 4 | +9/-4 | 4 |\n| Jeromy Johnson | 2 | +6/-6 | 4 |\n| Nick Cabatoff | 1 | +7/-2 | 1 |\n| Stephen Solka | 1 | +1/-7 | 1 |\n| Preston Van Loon | 2 | +6/-2 | 2 |\n| Jakub Sztandera | 2 | +5/-2 | 2 |\n| llx | 1 | +3/-3 | 1 |\n| Adrian Lanzafame | 1 | +3/-3 | 1 |\n| Yusef Napora | 1 | +3/-2 | 1 |\n| Louis Thibault | 1 | +5/-0 | 1 |\n| Martín Triay | 1 | +4/-0 | 1 |\n| Hlib | 1 | +2/-2 | 1 |\n| Shotaro Yamada | 1 | +2/-1 | 1 |\n| phuslu | 1 | +1/-1 | 1 |\n| Zero King | 1 | +1/-1 | 1 |\n| Rüdiger Klaehn | 1 | +2/-0 | 1 |\n| Nex | 1 | +1/-1 | 1 |\n| Mark Gaiser | 1 | +1/-1 | 1 |\n| Luflosi | 1 | +1/-1 | 1 |\n| David Florness | 1 | +1/-1 | 1 |\n| Dean Eigenmann | 1 | +0/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.8.md", "content": " # go-ipfs changelog v0.8\n\n## v0.8.0 2021-02-18\n\nWe're happy to announce go-ipfs 0.8.0! This is planned to be a fairly small release focused on integrating in the new pinning service/remote pinning [API](https://github.com/ipfs/pinning-services-api-spec) that makes the experience of managing pins across pinning services easier and more uniform.\n\n### 🔦 Highlights\n\n#### 🧷 Remote pinning services\n\nThere is now support for asking remote services to pin data for you. This means anyone can implement the [spec](https://ipfs.github.io/pinning-services-api-spec/) (developed in this [repo](https://github.com/ipfs/pinning-services-api-spec)) and allow for pin management.\n\nAll of the CLI (and corresponding HTTP API) commands are available under `ipfs pin remote`.\n\nThis remote pinning service comes with a redesign of how we're thinking about pinning and includes some commonly requested features such as:\n- Pins can have names (and coming soon metadata)\n- The same content can be pinned multiple times, but of course stored only once\n - This allows applications using the same pinning service to manage their own pins without worrying about removing content important to another application\n- Data can be pinned in either the foreground or background\n\n\nExamples include:\n```\nipfs pin remote service add myservice https://myservice.tld:1234/api/path myaccess key\n\nipfs pin remote add /ipfs/bafymydata --service=myservice --name=myfile\nipfs pin remote ls --service=myservice --name=myfile\nipfs pin remote ls --service=myservice --cid=bafymydata\nipfs pin remote rm --service=myservice --name=myfile\n```\nA few notes:\n\nRemote pinning services work with recursive pins. This means commands like `ipfs pin remote ls` will not list indirectly pinned CIDs.\n\nWhile pinning service data is stored in the configuration file it cannot be edited directly via the `ipfs config` commands due to the sensitive nature of pinning service API keys. The `ipfs pin remote service` commands can be used for interacting with remote service settings.\n\n#### 📌 Faster local pinning and unpinning\n\nThe pinning subsystem has been redesigned to be much faster and more flexible in how it tracks pins. For users who are working with many pins this will lead to a big speed increase in listing and modifying the set of pinned items as well as decreased memory usage.\n\nPart of the redesign was setup to account for being able to interact with local pins the same way we can now interact with remote pins (e.g. names, being allowed to pin the same CID multiple times, etc.). Keep posted for more improvements to pinning.\n\n#### DNSLink names on https:// subdomains\n\nPreviously DNSLink names would have trouble loading over subdomain gateways with HTTPS support since there is no way to get multilevel wildcard certificates (e.g. `en.wikipedia-on-ipfs.org.ipns.dweb.link` cannot be covered by `*.ipns.dweb.link`). Therefore, when trying to load DNSLink names over https:// subdomains go-ipfs we now forward to an encoded DNS name. Since DNS names cannot contain `.` in them they are escaped using `-`.\n\n`/ipns/en.wikipedia-on-ipfs.org` →\n`ipns://en.wikipedia-on-ipfs.org` →\n`https://dweb.link/ipns/en.wikipedia-on-ipfs.org`\n`https://en-wikipedia--on--ipfs-org.ipns.dweb.link` :point_left: _a single DNS label, no TLS error_\n\n#### QUIC update\n\nQUIC support has received a number of upgrades, including the ability to take advantage of larger UDP receive buffers for increased performance.\n\nLinux users may notice a logged error on daemon startup if your system needs extra configuration to allow IPFS increase the buffer size. A helpful link for resolving this is in the log message as well as [here](https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size).\n\n#### 👋 No more Darwin 386 builds\n\nGo 1.15 (the latest version of Go) [no longer supports](https://github.com/golang/go/issues/34749) Darwin 386 and so we are dropping support as well.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - Release v0.8.0\n - docs: RepinInterval\n - style: docs/config.md\n - style: improved MFS PinName example\n - docs: Pinning.RemoteServices.Policies\n - fix: decrease log level of opencensus initialization ([ipfs/go-ipfs#7815](https://github.com/ipfs/go-ipfs/pull/7815))\n - Register oc metrics ([ipfs/go-ipfs#7593](https://github.com/ipfs/go-ipfs/pull/7593))\n - add remote pinning to ipfs command (#7661) ([ipfs/go-ipfs#7661](https://github.com/ipfs/go-ipfs/pull/7661))\n - More p2p proxy checks ([ipfs/go-ipfs#7797](https://github.com/ipfs/go-ipfs/pull/7797))\n - Use datastore based pinning ([ipfs/go-ipfs#7750](https://github.com/ipfs/go-ipfs/pull/7750))\n - fix: return an error when an unknown object type is passed ([ipfs/go-ipfs#7795](https://github.com/ipfs/go-ipfs/pull/7795))\n - clarify why ipfs file ls is being deprecated ([ipfs/go-ipfs#7755](https://github.com/ipfs/go-ipfs/pull/7755))\n - fix: ipfs dag export uses the CoreAPI and respects the offline flag ([ipfs/go-ipfs#7753](https://github.com/ipfs/go-ipfs/pull/7753))\n - return an error when trying to download fs-repo-migrations for linux + musl ([ipfs/go-ipfs#7735](https://github.com/ipfs/go-ipfs/pull/7735))\n - fix: do not create a new (unused) peerID when initializing from config ([ipfs/go-ipfs#7730](https://github.com/ipfs/go-ipfs/pull/7730))\n - docs: Add a link in config.md ([ipfs/go-ipfs#7780](https://github.com/ipfs/go-ipfs/pull/7780))\n - update libp2p for stream closure refactor ([ipfs/go-ipfs#7747](https://github.com/ipfs/go-ipfs/pull/7747))\n - Fix typo in ipfs dag stat command ([ipfs/go-ipfs#7761](https://github.com/ipfs/go-ipfs/pull/7761))\n - docs(readme): key rotation in docker (#7721) ([ipfs/go-ipfs#7721](https://github.com/ipfs/go-ipfs/pull/7721))\n - fix(dnslink-gw): breadcrumbs and CID column when dir listing ([ipfs/go-ipfs#7699](https://github.com/ipfs/go-ipfs/pull/7699))\n - fix(gw): preserve query on website redirect ([ipfs/go-ipfs#7727](https://github.com/ipfs/go-ipfs/pull/7727))\n - feat: ipfs-webui v2.11.4 ([ipfs/go-ipfs#7716](https://github.com/ipfs/go-ipfs/pull/7716))\n - docs: how the ipfs snap is built and published ([ipfs/go-ipfs#7725](https://github.com/ipfs/go-ipfs/pull/7725))\n - fix: webui on ipv6 localhost ([ipfs/go-ipfs#7731](https://github.com/ipfs/go-ipfs/pull/7731))\n - Add missing plugin support on FreeBSD ([ipfs/go-ipfs#7722](https://github.com/ipfs/go-ipfs/pull/7722))\n - fix error when computing coverage ([ipfs/go-ipfs#7726](https://github.com/ipfs/go-ipfs/pull/7726))\n - docs(config): X-Forwarded-Host ([ipfs/go-ipfs#7651](https://github.com/ipfs/go-ipfs/pull/7651))\n - chore: webui v2.11.2 ([ipfs/go-ipfs#7703](https://github.com/ipfs/go-ipfs/pull/7703))\n - Add task for updating CLI docs right after updating the HTTP-api docs ([ipfs/go-ipfs#7711](https://github.com/ipfs/go-ipfs/pull/7711))\n - feat(gateway): Content-Disposition improvements ([ipfs/go-ipfs#7677](https://github.com/ipfs/go-ipfs/pull/7677))\n - fix build on Plan 9 ([ipfs/go-ipfs#7690](https://github.com/ipfs/go-ipfs/pull/7690))\n - docs: update changelog for v0.7.0\n - chore: bump webui version\n - fix: remove the (empty) alias for --peerid-base\n - fix: use override GOFLAGS changes from 480defab689610550ee3d346e31441a2bb881fcb but keep trimpath usage as is\n - Revert \"fix: override GOFLAGS\"\n - Fix --ipns-base alias ([ipfs/go-ipfs#7659](https://github.com/ipfs/go-ipfs/pull/7659))\n - docs: update config to indicate SECIO deprecation ([ipfs/go-ipfs#7630](https://github.com/ipfs/go-ipfs/pull/7630))\n - fix: ipfs dht put/get commands with peerIDs encoded as CIDs ([ipfs/go-ipfs#7633](https://github.com/ipfs/go-ipfs/pull/7633))\n - update version to 0.8.0-dev ([ipfs/go-ipfs#7629](https://github.com/ipfs/go-ipfs/pull/7629))\n- github.com/ipfs/go-bitswap (v0.2.20 -> v0.3.3):\n - feat: configurable engine blockstore worker count (#449) ([ipfs/go-bitswap#449](https://github.com/ipfs/go-bitswap/pull/449))\n - fix: set the score ledger on start ([ipfs/go-bitswap#447](https://github.com/ipfs/go-bitswap/pull/447))\n - feat: update for go-libp2p-core 0.7.0 interface changes ([ipfs/go-bitswap#445](https://github.com/ipfs/go-bitswap/pull/445))\n - fix: guard access to the mock wiretap with a lock ([ipfs/go-bitswap#446](https://github.com/ipfs/go-bitswap/pull/446))\n - Add WireTap interface (#444) ([ipfs/go-bitswap#444](https://github.com/ipfs/go-bitswap/pull/444))\n - Fix: Increment stats.MessagesSent in msgToStream() function (#441) ([ipfs/go-bitswap#441](https://github.com/ipfs/go-bitswap/pull/441))\n - refactor: remove extraneous ledger field init (#437) ([ipfs/go-bitswap#437](https://github.com/ipfs/go-bitswap/pull/437))\n - Added `WithScoreLedger` Bitswap option (#430) ([ipfs/go-bitswap#430](https://github.com/ipfs/go-bitswap/pull/430))\n- github.com/ipfs/go-blockservice (v0.1.3 -> v0.1.4):\n - Avoid modifying passed in slice of cids ([ipfs/go-blockservice#65](https://github.com/ipfs/go-blockservice/pull/65))\n- github.com/ipfs/go-ds-badger (v0.2.4 -> v0.2.6):\n - Log error if batch not committed or canceled ([ipfs/go-ds-badger#108](https://github.com/ipfs/go-ds-badger/pull/108))\n - Add Cancel function; add finalizer to cleanup abandoned batch ([ipfs/go-ds-badger#105](https://github.com/ipfs/go-ds-badger/pull/105))\n - Do not implement batches using transactions ([ipfs/go-ds-badger#104](https://github.com/ipfs/go-ds-badger/pull/104))\n - readme: add information on Badger2 datastore ([ipfs/go-ds-badger#102](https://github.com/ipfs/go-ds-badger/pull/102))\n - update contributing link ([ipfs/go-ds-badger#91](https://github.com/ipfs/go-ds-badger/pull/91))\n - Use current go-log (#89) ([ipfs/go-ds-badger#89](https://github.com/ipfs/go-ds-badger/pull/89))\n- github.com/ipfs/go-graphsync (v0.1.1 -> v0.6.0):\n - docs(CHANGELOG): revise for 0.6.0\n - Merge branch 'master' into release/v0.6.0\n - docs(CHANGELOG): update for 0.6.0 release\n - move block allocation into message queue (#140) ([ipfs/go-graphsync#140](https://github.com/ipfs/go-graphsync/pull/140))\n - Response Assembler Refactor (#138) ([ipfs/go-graphsync#138](https://github.com/ipfs/go-graphsync/pull/138))\n - Add error listener on receiver (#136) ([ipfs/go-graphsync#136](https://github.com/ipfs/go-graphsync/pull/136))\n - Run testplan on in CI (#137) ([ipfs/go-graphsync#137](https://github.com/ipfs/go-graphsync/pull/137))\n - fix(responsemanager): fix network error propagation (#133) ([ipfs/go-graphsync#133](https://github.com/ipfs/go-graphsync/pull/133))\n - testground test for graphsync (#132) ([ipfs/go-graphsync#132](https://github.com/ipfs/go-graphsync/pull/132))\n - docs(CHANGELOG): update for v0.5.2 ([ipfs/go-graphsync#130](https://github.com/ipfs/go-graphsync/pull/130))\n - RegisterNetworkErrorListener should fire when there's an error connecting to the peer (#127) ([ipfs/go-graphsync#127](https://github.com/ipfs/go-graphsync/pull/127))\n - Permit multiple data subscriptions per original topic (#128) ([ipfs/go-graphsync#128](https://github.com/ipfs/go-graphsync/pull/128))\n - release: v0.5.1 (#123) ([ipfs/go-graphsync#123](https://github.com/ipfs/go-graphsync/pull/123))\n - feat(responsemanager): allow configuration of max requests (#122) ([ipfs/go-graphsync#122](https://github.com/ipfs/go-graphsync/pull/122))\n - docs(CHANGELOG): update for 0.5.0 ([ipfs/go-graphsync#120](https://github.com/ipfs/go-graphsync/pull/120))\n - feat: use go-libp2p-core 0.7.0 stream interfaces (#116) ([ipfs/go-graphsync#116](https://github.com/ipfs/go-graphsync/pull/116))\n - Merge branch 'release/v0.4.3'\n - chore(benchmarks): remove extra files\n - fix(peerresponsemanager): avoid race condition that could result in NPE in link tracker (#118) ([ipfs/go-graphsync#118](https://github.com/ipfs/go-graphsync/pull/118))\n - docs(CHANGELOG): update for 0.4.2 ([ipfs/go-graphsync#117](https://github.com/ipfs/go-graphsync/pull/117))\n - feat(memory): improve memory usage (#110) ([ipfs/go-graphsync#110](https://github.com/ipfs/go-graphsync/pull/110))\n - fix(notifications): fix lock in close (#115) ([ipfs/go-graphsync#115](https://github.com/ipfs/go-graphsync/pull/115))\n - docs(CHANGELOG): update for v0.4.1 ([ipfs/go-graphsync#114](https://github.com/ipfs/go-graphsync/pull/114))\n - fix(allocator): remove peer from peer status list\n - docs(CHANGELOG): update for v0.4.0\n - docs(CHANGELOG): update for 0.3.1 ([ipfs/go-graphsync#112](https://github.com/ipfs/go-graphsync/pull/112))\n - Add allocator for memory backpressure (#108) ([ipfs/go-graphsync#108](https://github.com/ipfs/go-graphsync/pull/108))\n - Shutdown notifications go routines (#109) ([ipfs/go-graphsync#109](https://github.com/ipfs/go-graphsync/pull/109))\n - Switch to google protobuf generator (#105) ([ipfs/go-graphsync#105](https://github.com/ipfs/go-graphsync/pull/105))\n - feat(CHANGELOG): update for 0.3.0 ([ipfs/go-graphsync#104](https://github.com/ipfs/go-graphsync/pull/104))\n - docs(CHANGELOG): update for 0.2.1 ([ipfs/go-graphsync#103](https://github.com/ipfs/go-graphsync/pull/103))\n - Track actual network operations in a response (#102) ([ipfs/go-graphsync#102](https://github.com/ipfs/go-graphsync/pull/102))\n - feat(responsecache): prune blocks more intelligently (#101) ([ipfs/go-graphsync#101](https://github.com/ipfs/go-graphsync/pull/101))\n - Release/0.2.0 ([ipfs/go-graphsync#99](https://github.com/ipfs/go-graphsync/pull/99))\n - fix(metadata): fix cbor-gen (#98) ([ipfs/go-graphsync#98](https://github.com/ipfs/go-graphsync/pull/98))\n - fix(selectorvalidator): memory optimization (#97) ([ipfs/go-graphsync#97](https://github.com/ipfs/go-graphsync/pull/97))\n - Update go-ipld-prime@v0.5.0 (#92) ([ipfs/go-graphsync#92](https://github.com/ipfs/go-graphsync/pull/92))\n - refactor(metadata): use cbor-gen encoding (#96) ([ipfs/go-graphsync#96](https://github.com/ipfs/go-graphsync/pull/96))\n - Release/v0.1.2 ([ipfs/go-graphsync#95](https://github.com/ipfs/go-graphsync/pull/95))\n - Return Request context canceled error (#93) ([ipfs/go-graphsync#93](https://github.com/ipfs/go-graphsync/pull/93))\n - feat(benchmarks): add p2p stress test (#91) ([ipfs/go-graphsync#91](https://github.com/ipfs/go-graphsync/pull/91))\n - Benchmark framework + First memory fixes (#89) ([ipfs/go-graphsync#89](https://github.com/ipfs/go-graphsync/pull/89))\n - docs(CHANGELOG): update for v0.1.1 ([ipfs/go-graphsync#85](https://github.com/ipfs/go-graphsync/pull/85))\n- github.com/ipfs/go-ipfs-cmds (v0.4.0 -> v0.6.0):\n - Added DelimitedStringsOption for enabling delimited strings on the CLI ([ipfs/go-ipfs-cmds#204](https://github.com/ipfs/go-ipfs-cmds/pull/204))\n - feat: support strings option over HTTP API ([ipfs/go-ipfs-cmds#203](https://github.com/ipfs/go-ipfs-cmds/pull/203))\n- github.com/ipfs/go-ipfs-config (v0.9.0 -> v0.12.0):\n - add support for pinning mfs (#116) ([ipfs/go-ipfs-config#116](https://github.com/ipfs/go-ipfs-config/pull/116))\n - add remote pinning services config ([ipfs/go-ipfs-config#113](https://github.com/ipfs/go-ipfs-config/pull/113))\n - Remove badger2 profile ([ipfs/go-ipfs-config#115](https://github.com/ipfs/go-ipfs-config/pull/115))\n - Add badger2 profile and config spec\n- github.com/ipfs/go-ipfs-pinner (v0.0.4 -> v0.1.1):\n - Avoid loading all pins into memory during migration (#5) ([ipfs/go-ipfs-pinner#5](https://github.com/ipfs/go-ipfs-pinner/pull/5))\n - Datastore based pinner (#4) ([ipfs/go-ipfs-pinner#4](https://github.com/ipfs/go-ipfs-pinner/pull/4))\n- github.com/ipfs/go-ipld-cbor (v0.0.4 -> v0.0.5):\n - add the ability to leverage zero-copy on blockstores. (#75) ([ipfs/go-ipld-cbor#75](https://github.com/ipfs/go-ipld-cbor/pull/75))\n - ipldstore: Also wrap Put serialization errors ([ipfs/go-ipld-cbor#74](https://github.com/ipfs/go-ipld-cbor/pull/74))\n - add helper constructor for inmem cbor store\n - docs: add comments describing methods & interfaces ([ipfs/go-ipld-cbor#71](https://github.com/ipfs/go-ipld-cbor/pull/71))\n- github.com/ipfs/go-path (v0.0.8 -> v0.0.9):\n - fix: improved error message on broken CIDv0 ([ipfs/go-path#33](https://github.com/ipfs/go-path/pull/33))\n- github.com/ipfs/go-pinning-service-http-client (null -> v0.1.0):\n - feat: LsBatchSync to fetch single batch of results ([ipfs/go-pinning-service-http-client#6](https://github.com/ipfs/go-pinning-service-http-client/pull/6))\n - Initial Implementation ([ipfs/go-pinning-service-http-client#1](https://github.com/ipfs/go-pinning-service-http-client/pull/1))\n- github.com/ipld/go-car (v0.1.1-0.20200429200904-c222d793c339 -> v0.1.1-0.20201015032735-ff6ccdc46acc):\n - Update ipld libs ([ipld/go-car#35](https://github.com/ipld/go-car/pull/35))\n- github.com/ipld/go-ipld-prime (v0.0.2-0.20200428162820-8b59dc292b8e -> v0.5.1-0.20201021195245-109253e8a018):\n - Merge branch 'codec-hardening'\n - Add fluent.MustReflect convenience method.\n - codegen: make error info available when tuples process data that is too long. ([ipld/go-ipld-prime#99](https://github.com/ipld/go-ipld-prime/pull/99))\n - Merge branch 'codegen-typofixes'\n - Implement resource budgets in dagcbor parsing. ([ipld/go-ipld-prime#85](https://github.com/ipld/go-ipld-prime/pull/85))\n - Codegen for links should emit the methods to conform to the schema.TypedLinkNode interface where applicable. ([ipld/go-ipld-prime#91](https://github.com/ipld/go-ipld-prime/pull/91))\n - Introduce fluent.Reflect convenience functions. ([ipld/go-ipld-prime#81](https://github.com/ipld/go-ipld-prime/pull/81))\n - schema/gen/go: make all top-level tests parallel\n - all: don't use buffers where readers suffice\n - fix typo in documentation\n - schema-schema codegen demo now includes unmarshal exercise ([ipld/go-ipld-prime#76](https://github.com/ipld/go-ipld-prime/pull/76))\n - Update tests for unions; several fixes ([ipld/go-ipld-prime#75](https://github.com/ipld/go-ipld-prime/pull/75))\n - New testcase system for exercising typed nodes; Revamp struct tests to use it. ([ipld/go-ipld-prime#66](https://github.com/ipld/go-ipld-prime/pull/66))\n - small docs fixes on an internal component.\n - Fix formatting in README.\n - fix(cidlink): check for byte buffer ([ipld/go-ipld-prime#70](https://github.com/ipld/go-ipld-prime/pull/70))\n - linking/cid: check a previously unused error ([ipld/go-ipld-prime#68](https://github.com/ipld/go-ipld-prime/pull/68))\n - all: make 'go test ./...' pass on Go 1.15 ([ipld/go-ipld-prime#67](https://github.com/ipld/go-ipld-prime/pull/67))\n - Merge branch 'kinded-union-gen'\n - Add traversal.Get function ([ipld/go-ipld-prime#65](https://github.com/ipld/go-ipld-prime/pull/65))\n - Kinded union gen ([ipld/go-ipld-prime#64](https://github.com/ipld/go-ipld-prime/pull/64))\n - Struct tuple representation codegen ([ipld/go-ipld-prime#63](https://github.com/ipld/go-ipld-prime/pull/63))\n - Merge branch 'moar-codegen'\n - Self-hosting gen of the schema-schema. ([ipld/go-ipld-prime#62](https://github.com/ipld/go-ipld-prime/pull/62))\n - Codegen: approaching self-host ([ipld/go-ipld-prime#61](https://github.com/ipld/go-ipld-prime/pull/61))\n - Codegen of unions, and their keyed representations ([ipld/go-ipld-prime#60](https://github.com/ipld/go-ipld-prime/pull/60))\n - mark v0.5\n - API updates for v0.5: the renamening ([ipld/go-ipld-prime#59](https://github.com/ipld/go-ipld-prime/pull/59))\n - mark v0.4\n - changelog: note the codegen work.\n - Codegen update -- Assemblers, and many new representations ([ipld/go-ipld-prime#52](https://github.com/ipld/go-ipld-prime/pull/52))\n - Merge branch 'json-tables-codec'\n - Merge branch 'docs-updates'\n - Introduce changelog!\n - Add examples of creating and loading links.\n- github.com/ipld/go-ipld-prime-proto (v0.0.0-20200428191222-c1ffdadc01e1 -> v0.1.0):\n - Update go-ipld-prime ([ipld/go-ipld-prime-proto#6](https://github.com/ipld/go-ipld-prime-proto/pull/6))\n - feat(coding use -1 instead of 0):\n - Update ipld prime, use proper code-gen ([ipld/go-ipld-prime-proto#5](https://github.com/ipld/go-ipld-prime-proto/pull/5))\n - Updates to dependencies ([ipld/go-ipld-prime-proto#4](https://github.com/ipld/go-ipld-prime-proto/pull/4))\n - Check for byte buffer on decode ([ipld/go-ipld-prime-proto#3](https://github.com/ipld/go-ipld-prime-proto/pull/3))\n- github.com/libp2p/go-libp2p (v0.11.0 -> v0.13.0):\n - use a context when opening streams ([libp2p/go-libp2p#1033](https://github.com/libp2p/go-libp2p/pull/1033))\n - fix: obey new stream timeout ([libp2p/go-libp2p#1029](https://github.com/libp2p/go-libp2p/pull/1029))\n - feat: update to go-libp2p-core 0.7.0 interface changes ([libp2p/go-libp2p#1001](https://github.com/libp2p/go-libp2p/pull/1001))\n - Basic Connection Gater Implementation ([libp2p/go-libp2p#1005](https://github.com/libp2p/go-libp2p/pull/1005))\n - Fixed bug for inbound connections gated by the deprecated filter option (#1004) ([libp2p/go-libp2p#1004](https://github.com/libp2p/go-libp2p/pull/1004))\n- github.com/libp2p/go-libp2p-autonat (v0.3.2 -> v0.4.0):\n - feat: update to go-libp2p-core 0.7.0 ([libp2p/go-libp2p-autonat#97](https://github.com/libp2p/go-libp2p-autonat/pull/97))\n- github.com/libp2p/go-libp2p-circuit (v0.3.1 -> v0.4.0):\n - feat: update to go-libp2p-core 0.7.0 ([libp2p/go-libp2p-circuit#123](https://github.com/libp2p/go-libp2p-circuit/pull/123))\n- github.com/libp2p/go-libp2p-core (v0.6.1 -> v0.8.0):\n - add a context to OpenStream and NewStream (#172) ([libp2p/go-libp2p-core#172](https://github.com/libp2p/go-libp2p-core/pull/172))\n - sec/insecure/insecure.go: Fix typo (#167) ([libp2p/go-libp2p-core#167](https://github.com/libp2p/go-libp2p-core/pull/167))\n - add CloseRead/CloseWrite on streams (#166) ([libp2p/go-libp2p-core#166](https://github.com/libp2p/go-libp2p-core/pull/166))\n - Fix typo in docs (#163) ([libp2p/go-libp2p-core#163](https://github.com/libp2p/go-libp2p-core/pull/163))\n- github.com/libp2p/go-libp2p-gostream (v0.2.1 -> v0.3.0):\n - feat: use go-libp2p-core 0.7.0 stream interfaces ([libp2p/go-libp2p-gostream#60](https://github.com/libp2p/go-libp2p-gostream/pull/60))\n- github.com/libp2p/go-libp2p-http (v0.1.5 -> v0.2.0):\n - Fix var name in README ([libp2p/go-libp2p-http#63](https://github.com/libp2p/go-libp2p-http/pull/63))\n - Fix var name in doc ([libp2p/go-libp2p-http#62](https://github.com/libp2p/go-libp2p-http/pull/62))\n- github.com/libp2p/go-libp2p-kad-dht (v0.9.0 -> v0.11.1):\n - Fix constructor ordering ([libp2p/go-libp2p-kad-dht#698](https://github.com/libp2p/go-libp2p-kad-dht/pull/698))\n - feat: update to go-libp2p-core 0.7.0 ([libp2p/go-libp2p-kad-dht#693](https://github.com/libp2p/go-libp2p-kad-dht/pull/693))\n - Run fixLowPeers on startup ([libp2p/go-libp2p-kad-dht#694](https://github.com/libp2p/go-libp2p-kad-dht/pull/694))\n - feat: add advanced V1ProtocolOverride option to be used by legacy networks\n - feat: remove dht v2 as it's not actually in use and could be confusing\n- github.com/libp2p/go-libp2p-mplex (v0.2.4 -> v0.4.1):\n - update go-mplex, use the context passed to OpenStream ([libp2p/go-libp2p-mplex#23](https://github.com/libp2p/go-libp2p-mplex/pull/23))\n - change OpenStream to accept a context ([libp2p/go-libp2p-mplex#21](https://github.com/libp2p/go-libp2p-mplex/pull/21))\n - feat: update stream interfaces ([libp2p/go-libp2p-mplex#20](https://github.com/libp2p/go-libp2p-mplex/pull/20))\n- github.com/libp2p/go-libp2p-noise (v0.1.1 -> v0.1.2):\n - optimize: reduce syscalls using a buffered reader.\n- github.com/libp2p/go-libp2p-pubsub (v0.3.5 -> v0.4.1):\n - defer stream removal instead of doing it inline.\n - add test for inbound stream deduplication\n - deduplicate inbound streams\n - populate receivedFrom field in delivery trace\n - add receivedFrom field in delivery trace\n - fix: reduce log spam (#394) ([libp2p/go-libp2p-pubsub#394](https://github.com/libp2p/go-libp2p-pubsub/pull/394))\n - fix: treat peers already connected to the host before pubsub is initialized as valid potential pubsub peers\n - test: add test for if nodes are connected before pubsub is started\n - feat: update to go-libp2p-core 0.7.0\n - Add go-libp2p example in README.md (#392) ([libp2p/go-libp2p-pubsub#392](https://github.com/libp2p/go-libp2p-pubsub/pull/392))\n - subscription filters\n - remove multi-topic message support\n - satisfy race detector\n - clean up\n - copy string topic\n - add test for score adjustment from topic params reset\n - prettify things\n - add test for topic score parameter reset method\n - add test for topic score parameter reset\n - add api for dynamically setting and resetting topic score parameters\n - add support for priority topic delivery weights\n - tweak duplicate/reject weights\n - decay global counters after 2 min\n - decouple global counter decay from source counter decay\n - add warning for failure to parse IP out of remote multiaddr\n - more docs\n - configure the peer gater using a parameter object, docs and stuff\n - disable codecov annotations, makes things unreadable\n - further tweak gate threshold weights\n - fix test races\n - use IPs for peer gater stat tracking\n - mix total accounting components with different weights\n - count all rejections by default\n - fix non-determinism in test\n - tweak probability threshold\n - also account for duplicates in gating decisions\n - test throttle code path in gossip tracer\n - add test for peer gater\n - more efficient promise processing on throttling\n - trace throttle peers to avoid breaking promises unfairly\n - better log messages around gating\n - implement peer gater\n - peer gater scaffolding\n - rich router acceptance semantics\n - reduce log verbosity; debug mostly\n- github.com/libp2p/go-libp2p-pubsub-router (v0.3.2 -> v0.4.0):\n - feat: use new stream interfaces from go-libp2p-core 0.7.0 ([libp2p/go-libp2p-pubsub-router#81](https://github.com/libp2p/go-libp2p-pubsub-router/pull/81))\n- github.com/libp2p/go-libp2p-quic-transport (v0.8.0 -> v0.10.0):\n - change OpenStream to accept a context ([libp2p/go-libp2p-quic-transport#189](https://github.com/libp2p/go-libp2p-quic-transport/pull/189))\n - update quic-go to v0.19.1 ([libp2p/go-libp2p-quic-transport#182](https://github.com/libp2p/go-libp2p-quic-transport/pull/182))\n - pass a conn that can be type asserted to a net.UDPConn to quic-go ([libp2p/go-libp2p-quic-transport#180](https://github.com/libp2p/go-libp2p-quic-transport/pull/180))\n - add more integration tests ([libp2p/go-libp2p-quic-transport#181](https://github.com/libp2p/go-libp2p-quic-transport/pull/181))\n - always close the connection in the cmd client ([libp2p/go-libp2p-quic-transport#175](https://github.com/libp2p/go-libp2p-quic-transport/pull/175))\n - use GitHub Actions to test interoperability of releases ([libp2p/go-libp2p-quic-transport#173](https://github.com/libp2p/go-libp2p-quic-transport/pull/173))\n - Implement CloseRead/CloseWrite ([libp2p/go-libp2p-quic-transport#174](https://github.com/libp2p/go-libp2p-quic-transport/pull/174))\n - enable quic-go metrics collection ([libp2p/go-libp2p-quic-transport#172](https://github.com/libp2p/go-libp2p-quic-transport/pull/172))\n- github.com/libp2p/go-libp2p-swarm (v0.2.8 -> v0.4.0):\n - use a context for OpenStream and NewStream ([libp2p/go-libp2p-swarm#232](https://github.com/libp2p/go-libp2p-swarm/pull/232))\n - fix: handle case where swarm closes before stream ([libp2p/go-libp2p-swarm#229](https://github.com/libp2p/go-libp2p-swarm/pull/229))\n - feat: update to latest go-libp2p-core interfaces ([libp2p/go-libp2p-swarm#228](https://github.com/libp2p/go-libp2p-swarm/pull/228))\n- github.com/libp2p/go-libp2p-testing (v0.2.0 -> v0.4.0):\n - pass contexts to OpenStream in tests ([libp2p/go-libp2p-testing#31](https://github.com/libp2p/go-libp2p-testing/pull/31))\n - chore: Adding LICENSE. ([libp2p/go-libp2p-testing#30](https://github.com/libp2p/go-libp2p-testing/pull/30))\n - feat: update to go-libp2p-core 0.7.0 ([libp2p/go-libp2p-testing#29](https://github.com/libp2p/go-libp2p-testing/pull/29))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.3.0 -> v0.4.0):\n - pass contexts to OpenStream in tests ([libp2p/go-libp2p-transport-upgrader#70](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/70))\n - fix int to string conversion in tests, update Go version on CI ([libp2p/go-libp2p-transport-upgrader#69](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/69))\n- github.com/libp2p/go-libp2p-yamux (v0.2.8 -> v0.5.1):\n - update go-yamux to v2.0.0, use context passed to OpenStream ([libp2p/go-libp2p-yamux#31](https://github.com/libp2p/go-libp2p-yamux/pull/31))\n - change OpenStream to accept a context ([libp2p/go-libp2p-yamux#29](https://github.com/libp2p/go-libp2p-yamux/pull/29))\n - feat: update to new stream interfaces ([libp2p/go-libp2p-yamux#27](https://github.com/libp2p/go-libp2p-yamux/pull/27))\n- github.com/libp2p/go-mplex (v0.1.2 -> v0.3.0):\n - add a context to NewStream, remove the NewStreamTimeout ([libp2p/go-mplex#82](https://github.com/libp2p/go-mplex/pull/82))\n - Implement new CloseWrite/CloseRead interface ([libp2p/go-mplex#81](https://github.com/libp2p/go-mplex/pull/81))\n - Bump lodash from 4.17.15 to 4.17.19 in /interop/js ([libp2p/go-mplex#79](https://github.com/libp2p/go-mplex/pull/79))\n - upgrade deps + interoperable varints. (#80) ([libp2p/go-mplex#80](https://github.com/libp2p/go-mplex/pull/80))\n - write benchmarks (#77) ([libp2p/go-mplex#77](https://github.com/libp2p/go-mplex/pull/77))\n- github.com/libp2p/go-ws-transport (v0.3.1 -> v0.4.0):\n - pass a context to OpenStream in tests ([libp2p/go-ws-transport#98](https://github.com/libp2p/go-ws-transport/pull/98))\n - Dependency: Remove deprecated multiaddr-net ([libp2p/go-ws-transport#97](https://github.com/libp2p/go-ws-transport/pull/97))\n - Update for go 1.14 Wasm changes ([libp2p/go-ws-transport#96](https://github.com/libp2p/go-ws-transport/pull/96))\n- github.com/libp2p/go-yamux (v1.3.7 -> v1.4.1):\n - feat: improve ping accuracy ([libp2p/go-yamux#35](https://github.com/libp2p/go-yamux/pull/35))\n - implement CloseRead/CloseWrite ([libp2p/go-yamux#5](https://github.com/libp2p/go-yamux/pull/5))\n - fix space accounting in the receive buffer ([libp2p/go-yamux#33](https://github.com/libp2p/go-yamux/pull/33))\n - Limit pings ([libp2p/go-yamux#32](https://github.com/libp2p/go-yamux/pull/32))\n - fix: simplify inflight fix ([libp2p/go-yamux#31](https://github.com/libp2p/go-yamux/pull/31))\n - Clearing inflight along with streams to avoid memory leak ([libp2p/go-yamux#30](https://github.com/libp2p/go-yamux/pull/30))\n- github.com/lucas-clemente/quic-go (v0.18.0 -> v0.19.3):\n - create a v0.19.x release\n - improve the warning about the UDP receive buffer size ([lucas-clemente/quic-go#2923](https://github.com/lucas-clemente/quic-go/pull/2923))\n - immediately remove reset tokens when retiring a connection ID ([lucas-clemente/quic-go#2897](https://github.com/lucas-clemente/quic-go/pull/2897))\n - add common temporary file patterns to .gitignore ([lucas-clemente/quic-go#2917](https://github.com/lucas-clemente/quic-go/pull/2917))\n - disable key updates when using HTTP/3 to avoid breaking Chrome 87 ([lucas-clemente/quic-go#2906](https://github.com/lucas-clemente/quic-go/pull/2906))\n - fix decoding of packet numbers in different packet number spaces ([lucas-clemente/quic-go#2903](https://github.com/lucas-clemente/quic-go/pull/2903))\n - log sent packet before logging its congestion / loss recovery effects ([lucas-clemente/quic-go#2912](https://github.com/lucas-clemente/quic-go/pull/2912))\n - fix a crash in the http3.Server when GetConfigForClient returns nil ([lucas-clemente/quic-go#2925](https://github.com/lucas-clemente/quic-go/pull/2925))\n - set the UDP receive buffer size on Windows ([lucas-clemente/quic-go#2896](https://github.com/lucas-clemente/quic-go/pull/2896))\n - remove superfluous sleep in packet handler map test ([lucas-clemente/quic-go#2894](https://github.com/lucas-clemente/quic-go/pull/2894))\n - fix setting of http.Handler in the example server ([lucas-clemente/quic-go#2900](https://github.com/lucas-clemente/quic-go/pull/2900))\n - remove stray print statement\n - remove unnecessary mutex locking in the stream flow controller ([lucas-clemente/quic-go#2869](https://github.com/lucas-clemente/quic-go/pull/2869))\n - only use syscalls on platforms that we're actually testing ([lucas-clemente/quic-go#2886](https://github.com/lucas-clemente/quic-go/pull/2886))\n - only write headers with a length that fits into 2 bytes in fuzz test ([lucas-clemente/quic-go#2884](https://github.com/lucas-clemente/quic-go/pull/2884))\n - fix packing of 1-RTT probe packets ([lucas-clemente/quic-go#2882](https://github.com/lucas-clemente/quic-go/pull/2882))\n - use PADDING frames to pad packets ([lucas-clemente/quic-go#2876](https://github.com/lucas-clemente/quic-go/pull/2876))\n - fix race condition when accepting streams ([lucas-clemente/quic-go#2874](https://github.com/lucas-clemente/quic-go/pull/2874))\n - only trace dropped 0-RTT packets when a tracer is set ([lucas-clemente/quic-go#2871](https://github.com/lucas-clemente/quic-go/pull/2871))\n - use consistent version numbers in client test ([lucas-clemente/quic-go#2870](https://github.com/lucas-clemente/quic-go/pull/2870))\n - replace the RWMutex with a Mutex in the flow controller ([lucas-clemente/quic-go#2865](https://github.com/lucas-clemente/quic-go/pull/2865))\n - replace the RWMutex with a Mutex in the packet handler map ([lucas-clemente/quic-go#2864](https://github.com/lucas-clemente/quic-go/pull/2864))\n - wait until the handshake is complete before updating the connection ID ([lucas-clemente/quic-go#2856](https://github.com/lucas-clemente/quic-go/pull/2856))\n - only check the SCID for Initial packets ([lucas-clemente/quic-go#2857](https://github.com/lucas-clemente/quic-go/pull/2857))\n - add the NO_VIABLE_PATH error ([lucas-clemente/quic-go#2861](https://github.com/lucas-clemente/quic-go/pull/2861))\n - implement qlogging of the preferred address in the transport parameters ([lucas-clemente/quic-go#2853](https://github.com/lucas-clemente/quic-go/pull/2853))\n - explicitly set the supported versions in the HTTP/3 server test ([lucas-clemente/quic-go#2854](https://github.com/lucas-clemente/quic-go/pull/2854))\n - allow an amplification factor of 3.x ([lucas-clemente/quic-go#2862](https://github.com/lucas-clemente/quic-go/pull/2862))\n - only allow the HTTP/3 client to dial with a single QUIC version ([lucas-clemente/quic-go#2848](https://github.com/lucas-clemente/quic-go/pull/2848))\n - send STREAMS_BLOCKED frame when MAX_STREAMS frame allows too few streams ([lucas-clemente/quic-go#2828](https://github.com/lucas-clemente/quic-go/pull/2828))\n - set the ALPN based on the QUIC version in the HTTP3 server ([lucas-clemente/quic-go#2847](https://github.com/lucas-clemente/quic-go/pull/2847))\n - pad datagrams containing ack-eliciting Initial packets sent by the server ([lucas-clemente/quic-go#2841](https://github.com/lucas-clemente/quic-go/pull/2841))\n - fix OpenStreamSync busy looping ([lucas-clemente/quic-go#2827](https://github.com/lucas-clemente/quic-go/pull/2827))\n - fix deadlock when closing the server and the connection at the same time ([lucas-clemente/quic-go#2849](https://github.com/lucas-clemente/quic-go/pull/2849))\n - run gofumpt, enable the gofumpt linter ([lucas-clemente/quic-go#2839](https://github.com/lucas-clemente/quic-go/pull/2839))\n - prepare for draft-32 ([lucas-clemente/quic-go#2831](https://github.com/lucas-clemente/quic-go/pull/2831))\n - update the invalid packet limit for AES ([lucas-clemente/quic-go#2825](https://github.com/lucas-clemente/quic-go/pull/2825))\n - increase UDP receive buffer size ([lucas-clemente/quic-go#2791](https://github.com/lucas-clemente/quic-go/pull/2791))\n - listen on both IPv4 and IPv6 in the interop runner server ([lucas-clemente/quic-go#2822](https://github.com/lucas-clemente/quic-go/pull/2822))\n - only send Version Negotiation packets for packets larger than 1200 bytes ([lucas-clemente/quic-go#2820](https://github.com/lucas-clemente/quic-go/pull/2820))\n - don't send a version negotiation packet in response to a version negotiation packet ([lucas-clemente/quic-go#2818](https://github.com/lucas-clemente/quic-go/pull/2818))\n - client: Add DialEarlyContext and DialAddrEarlyContext API ([lucas-clemente/quic-go#2814](https://github.com/lucas-clemente/quic-go/pull/2814))\n - qlog the key phase bit ([lucas-clemente/quic-go#2817](https://github.com/lucas-clemente/quic-go/pull/2817))\n - only include quic-trace when the quictrace build flag is set ([lucas-clemente/quic-go#2799](https://github.com/lucas-clemente/quic-go/pull/2799))\n - fix error handling when receiving post handshake messages ([lucas-clemente/quic-go#2807](https://github.com/lucas-clemente/quic-go/pull/2807))\n - add support for the ChaCha20 test on the server side ([lucas-clemente/quic-go#2816](https://github.com/lucas-clemente/quic-go/pull/2816))\n - allow the first key update immediately after handshake confirmation ([lucas-clemente/quic-go#2811](https://github.com/lucas-clemente/quic-go/pull/2811))\n - ignore temporary errors when reading from the packet conn ([lucas-clemente/quic-go#2806](https://github.com/lucas-clemente/quic-go/pull/2806))\n - fix linting error on OSX ([lucas-clemente/quic-go#2813](https://github.com/lucas-clemente/quic-go/pull/2813))\n - add the exhaustive linter, replace panics by return values in logging stringers ([lucas-clemente/quic-go#2729](https://github.com/lucas-clemente/quic-go/pull/2729))\n - include the error code in the string for CRYPTO_ERRORs ([lucas-clemente/quic-go#2805](https://github.com/lucas-clemente/quic-go/pull/2805))\n - fail the handshake if the quic_transport_parameter extension is missing ([lucas-clemente/quic-go#2804](https://github.com/lucas-clemente/quic-go/pull/2804))\n - fix logging of received Retry packets ([lucas-clemente/quic-go#2803](https://github.com/lucas-clemente/quic-go/pull/2803))\n - fix deadlock in crypto setup when it is closed while handling a message ([lucas-clemente/quic-go#2802](https://github.com/lucas-clemente/quic-go/pull/2802))\n - make the key update integration test more rigorous ([lucas-clemente/quic-go#2760](https://github.com/lucas-clemente/quic-go/pull/2760))\n - add support for the new keyupdate interop runner test case ([lucas-clemente/quic-go#2782](https://github.com/lucas-clemente/quic-go/pull/2782))\n - remove unneeded mutex in the client ([lucas-clemente/quic-go#2798](https://github.com/lucas-clemente/quic-go/pull/2798))\n - correctly handle key updates within the 3 PTO period ([lucas-clemente/quic-go#2787](https://github.com/lucas-clemente/quic-go/pull/2787))\n - introduce an ECNCapablePacketConn interface to determine ECN support ([lucas-clemente/quic-go#2788](https://github.com/lucas-clemente/quic-go/pull/2788))\n - use certificates from /certs directory for the server ([lucas-clemente/quic-go#2794](https://github.com/lucas-clemente/quic-go/pull/2794))\n - remove support for the ECN test case ([lucas-clemente/quic-go#2793](https://github.com/lucas-clemente/quic-go/pull/2793))\n - check that the peer updated its keys when acknowledging a key update ([lucas-clemente/quic-go#2781](https://github.com/lucas-clemente/quic-go/pull/2781))\n - fix flaky packet number skipping test ([lucas-clemente/quic-go#2786](https://github.com/lucas-clemente/quic-go/pull/2786))\n - read ECN bits and send ECN counters in ACK frames ([lucas-clemente/quic-go#2741](https://github.com/lucas-clemente/quic-go/pull/2741))\n - implement the limit of unsuccessful decryptions for the AEADs ([lucas-clemente/quic-go#2771](https://github.com/lucas-clemente/quic-go/pull/2771))\n - use the KEY_UPDATE_ERROR ([lucas-clemente/quic-go#2770](https://github.com/lucas-clemente/quic-go/pull/2770))\n - fix dropping of key phase 0 ([lucas-clemente/quic-go#2769](https://github.com/lucas-clemente/quic-go/pull/2769))\n - reduce the handshake timeout to two minutes in the handshake drop tests ([lucas-clemente/quic-go#2768](https://github.com/lucas-clemente/quic-go/pull/2768))\n - fix handling of multiple handshake messages in the case of errors ([lucas-clemente/quic-go#2777](https://github.com/lucas-clemente/quic-go/pull/2777))\n - enable more linters, update golangci-lint to v1.31 ([lucas-clemente/quic-go#2775](https://github.com/lucas-clemente/quic-go/pull/2775))\n - increase the threshold for the receive stream deadline test ([lucas-clemente/quic-go#2774](https://github.com/lucas-clemente/quic-go/pull/2774))\n - add an assertion that bytes_in_flight never becomes negative ([lucas-clemente/quic-go#2779](https://github.com/lucas-clemente/quic-go/pull/2779))\n - fix race condition in handshake fuzz code ([lucas-clemente/quic-go#2778](https://github.com/lucas-clemente/quic-go/pull/2778))\n - use more tls.Config options in the handshake fuzzer ([lucas-clemente/quic-go#2746](https://github.com/lucas-clemente/quic-go/pull/2746))\n - run two handshakes in the handshake fuzzer ([lucas-clemente/quic-go#2743](https://github.com/lucas-clemente/quic-go/pull/2743))\n - send post-handshake message in the handshake fuzzer ([lucas-clemente/quic-go#2742](https://github.com/lucas-clemente/quic-go/pull/2742))\n - skip a packet number when sending a 1-RTT PTO packet ([lucas-clemente/quic-go#2754](https://github.com/lucas-clemente/quic-go/pull/2754))\n - save dummy packets in the packet history when skipping packet numbers ([lucas-clemente/quic-go#2753](https://github.com/lucas-clemente/quic-go/pull/2753))\n - delete unacknowledged packets from the packet history after 3 PTOs ([lucas-clemente/quic-go#2750](https://github.com/lucas-clemente/quic-go/pull/2750))\n - add support for the HTTP CONNECT method (#2761) ([lucas-clemente/quic-go#2761](https://github.com/lucas-clemente/quic-go/pull/2761))\n - don't drop keys for key phase N before receiving a N+1-protected packet ([lucas-clemente/quic-go#2762](https://github.com/lucas-clemente/quic-go/pull/2762))\n - close session on errors unpacking errors other than decryption errors ([lucas-clemente/quic-go#2756](https://github.com/lucas-clemente/quic-go/pull/2756))\n - log when an old 1-RTT key is retired ([lucas-clemente/quic-go#2765](https://github.com/lucas-clemente/quic-go/pull/2765))\n - only return an invalid first key phase error for decryptable packets ([lucas-clemente/quic-go#2757](https://github.com/lucas-clemente/quic-go/pull/2757))\n - fix logging of locally initiated key updates ([lucas-clemente/quic-go#2764](https://github.com/lucas-clemente/quic-go/pull/2764))\n - test that both endpoints time out in the timeout integration test ([lucas-clemente/quic-go#2744](https://github.com/lucas-clemente/quic-go/pull/2744))\n - refactor RTT measurements to simplify the sentPacketHistory ([lucas-clemente/quic-go#2747](https://github.com/lucas-clemente/quic-go/pull/2747))\n - fix dropping of 0-RTT packets ([lucas-clemente/quic-go#2752](https://github.com/lucas-clemente/quic-go/pull/2752))\n - always qlog the generation of 1-RTT key updates ([lucas-clemente/quic-go#2763](https://github.com/lucas-clemente/quic-go/pull/2763))\n - move the PacketHeader struct from logging to qlog package ([lucas-clemente/quic-go#2766](https://github.com/lucas-clemente/quic-go/pull/2766))\n - use a uint8 for the EncryptionLevel ([lucas-clemente/quic-go#2751](https://github.com/lucas-clemente/quic-go/pull/2751))\n - make sure to only pass handshake messages that keys are available for ([lucas-clemente/quic-go#2739](https://github.com/lucas-clemente/quic-go/pull/2739))\n - only close the handshake fuzz runner once ([lucas-clemente/quic-go#2740](https://github.com/lucas-clemente/quic-go/pull/2740))\n - generate a self-signed certificate for the handshake fuzzer ([lucas-clemente/quic-go#2738](https://github.com/lucas-clemente/quic-go/pull/2738))\n - use the os.ErrDeadlineExceeded for stream deadline errors on Go 1.15 ([lucas-clemente/quic-go#2734](https://github.com/lucas-clemente/quic-go/pull/2734))\n - use GitHub Actions to run unit tests ([lucas-clemente/quic-go#2732](https://github.com/lucas-clemente/quic-go/pull/2732))\n - add a basic fuzzer for the handshake ([lucas-clemente/quic-go#2733](https://github.com/lucas-clemente/quic-go/pull/2733))\n - export seed corpus files using the SHA1 of the content as the filename ([lucas-clemente/quic-go#2731](https://github.com/lucas-clemente/quic-go/pull/2731))\n - add a fuzz target for the token generator ([lucas-clemente/quic-go#2730](https://github.com/lucas-clemente/quic-go/pull/2730))\n - fix typo in error message in sent packet handler\n - fix missing OnLost callback for frames sent in 0-RTT packets ([lucas-clemente/quic-go#2728](https://github.com/lucas-clemente/quic-go/pull/2728))\n - fix overflow of the max_ack_delay when parsing transport parameters ([lucas-clemente/quic-go#2725](https://github.com/lucas-clemente/quic-go/pull/2725))\n- github.com/marten-seemann/qpack (v0.2.0 -> v0.2.1):\n - run gofumpt, add a few more linters ([marten-seemann/qpack#21](https://github.com/marten-seemann/qpack/pull/21))\n - fix static table entry 80 ([marten-seemann/qpack#20](https://github.com/marten-seemann/qpack/pull/20))\n- github.com/marten-seemann/qtls-go1-15 (v0.1.0 -> v0.1.1):\n - use a prefix for client session cache keys\n - add callbacks to store and restore app data along a session state\n - don't use TLS 1.3 compatibility mode when using alternative record layer\n - delete the session ticket after attempting 0-RTT\n - reject 0-RTT when a different ALPN is chosen\n - encode the ALPN into the session ticket\n - add a field to the ConnectionState to tell if 0-RTT was used\n - add a callback to tell the client about rejection of 0-RTT\n - don't offer 0-RTT after a HelloRetryRequest\n - add Accept0RTT to Config callback to decide if 0-RTT should be accepted\n - add the option to encode application data into the session ticket\n - export the 0-RTT write key\n - abuse the nonce field of ClientSessionState to save max_early_data_size\n - export the 0-RTT read key\n - close connection if client attempts 0-RTT, but ticket didn't allow it\n - encode the max early data size into the session ticket\n - implement parsing of the early_data extension in the EncryptedExtensions\n - add a tls.Config.MaxEarlyData option to enable 0-RTT\n - accept TLS 1.3 cipher suites in Config.CipherSuites\n - introduce a function on the connection to generate a session ticket\n - add a config option to enforce selection of an application protocol\n - export Conn.HandlePostHandshakeMessage\n - export Alert\n - reject Configs that set MaxVersion < 1.3 when using a record layer\n - enforce TLS 1.3 when using an alternative record layer\n- github.com/multiformats/go-multistream (v0.1.2 -> v0.2.0):\n - improve negotiation flushing ([multiformats/go-multistream#52](https://github.com/multiformats/go-multistream/pull/52))\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20200402171437-3d27c146c105 -> v0.0.0-20200710004633-5379fc63235d):\n - correctly map typegen to cbg in all cases ([whyrusleeping/cbor-gen#26](https://github.com/whyrusleeping/cbor-gen/pull/26))\n - fix: clear struct state on unmarshal ([whyrusleeping/cbor-gen#22](https://github.com/whyrusleeping/cbor-gen/pull/22))\n - deferred: restrict max length ([whyrusleeping/cbor-gen#25](https://github.com/whyrusleeping/cbor-gen/pull/25))\n - reduce number of allocations in ScanForLinks ([whyrusleeping/cbor-gen#24](https://github.com/whyrusleeping/cbor-gen/pull/24))\n - attempt to allocate less by using shared buffers ([whyrusleeping/cbor-gen#18](https://github.com/whyrusleeping/cbor-gen/pull/18))\n - add benchmark\n - use new cid methods for less allocs ([whyrusleeping/cbor-gen#17](https://github.com/whyrusleeping/cbor-gen/pull/17))\n - properly handle roundtripping Deferred with 'null' value ([whyrusleeping/cbor-gen#16](https://github.com/whyrusleeping/cbor-gen/pull/16))\n - Support array types ([whyrusleeping/cbor-gen#15](https://github.com/whyrusleeping/cbor-gen/pull/15))\n- github.com/whyrusleeping/tar-utils (v0.0.0-20180509141711-8c6c8ba81d5c -> v0.0.0-20201201191210-20a61371de5b):\n - more closely match default tar errors (GNU + BSD binaries)\n\nContributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Eric Myhre | 180 | +26453/-11032 | 883 |\n| Marten Seemann | 212 | +14876/-9352 | 794 |\n| hannahhoward | 41 | +9195/-3113 | 186 |\n| Alex Cruikshank | 5 | +3323/-1895 | 58 |\n| Andrew Gillis | 3 | +3792/-581 | 21 |\n| vyzo | 49 | +2675/-949 | 95 |\n| Adin Schmahmann | 57 | +1473/-837 | 90 |\n| Steven Allen | 43 | +1252/-780 | 99 |\n| Petar Maymounkov | 3 | +1755/-113 | 17 |\n| Marcin Rataj | 35 | +979/-210 | 61 |\n| Paul Wolneykien | 2 | +670/-338 | 9 |\n| Jeromy Johnson | 9 | +525/-221 | 21 |\n| gammazero | 11 | +366/-101 | 26 |\n| Hector Sanjuan | 7 | +312/-0 | 11 |\n| Dirk McCormick | 4 | +190/-90 | 15 |\n| Will Scott | 1 | +252/-0 | 1 |\n| Oli Evans | 1 | +201/-0 | 1 |\n| Tomasz Zdybał | 2 | +182/-3 | 6 |\n| Daniel Martí | 6 | +104/-66 | 35 |\n| Sam | 3 | +76/-59 | 5 |\n| Łukasz Magiera | 2 | +92/-3 | 5 |\n| whyrusleeping | 3 | +77/-15 | 3 |\n| nisdas | 3 | +76/-15 | 4 |\n| Raúl Kripalani | 3 | +59/-31 | 5 |\n| Lucas Molas | 1 | +66/-3 | 2 |\n| Alex Towle | 1 | +52/-8 | 2 |\n| Dennis Trautwein | 1 | +58/-0 | 2 |\n| Adrian Lanzafame | 2 | +49/-7 | 4 |\n| klzgrad | 1 | +49/-5 | 2 |\n| Fazlul Shahriar | 1 | +35/-14 | 17 |\n| Yingrong Zhao | 1 | +45/-2 | 2 |\n| Jakub Sztandera | 2 | +22/-13 | 2 |\n| Chaitanya | 8 | +16/-16 | 8 |\n| Aarsh Shah | 1 | +27/-1 | 3 |\n| Rod Vagg | 1 | +23/-4 | 2 |\n| M. Hawn | 4 | +11/-11 | 8 |\n| Will | 1 | +12/-2 | 1 |\n| frrist | 1 | +7/-0 | 1 |\n| Rafael Ramalho | 2 | +5/-2 | 2 |\n| dependabot[bot] | 1 | +3/-3 | 1 |\n| Zaurbek Zhakupov | 1 | +3/-3 | 1 |\n| Tom Worrall | 1 | +4/-2 | 1 |\n| Jorropo | 2 | +5/-1 | 2 |\n| Chaitanya Raju | 1 | +3/-3 | 2 |\n| Egon Elbre | 1 | +0/-5 | 1 |\n| incognitomode | 1 | +2/-2 | 1 |\n| achingbrain | 1 | +2/-2 | 1 |\n| Michael Burns | 1 | +2/-2 | 1 |\n| David Florness | 2 | +2/-2 | 2 |\n| RubenKelevra | 1 | +2/-1 | 1 |\n| Andrew Nesbitt | 2 | +2/-1 | 2 |\n| Tarun Bansal | 1 | +1/-1 | 1 |\n| Max Inden | 1 | +1/-1 | 1 |\n| K | 1 | +2/-0 | 1 |\n| Jacob Heun | 1 | +1/-1 | 1 |\n| Henrique Dias | 1 | +1/-1 | 1 |\n| Bryan White | 1 | +1/-1 | 1 |\n| Bryan Stenson | 1 | +1/-1 | 1 |\n" }, { "path": "docs/changelogs/v0.9.md", "content": "# go-ipfs changelog v0.9\n\n## v0.9.1 2021-07-20\n\nThis is a small bug fix release resolving the following issues:\n1. A regression where the empty CID bafkqaaa could not resolve on gateways [#8230](https://github.com/ipfs/go-ipfs/issues/8230)\n2. A panic on OpenBSD [#8211](https://github.com/ipfs/go-ipfs/issues/8211)\n3. High CPU usage with QUIC [#8256](https://github.com/ipfs/go-ipfs/issues/8256)\n4. High memory usage with TCP [#8219](https://github.com/ipfs/go-ipfs/issues/8219)\n5. Some pubsub issues ([libp2p/go-libp2p-pubsub#427](https://github.com/libp2p/go-libp2p-pubsub/pull/427), [libp2p/go-libp2p-pubsub#430](https://github.com/libp2p/go-libp2p-pubsub/pull/430))\n6. Updated WebUI to [v2.12.4](https://github.com/ipfs/ipfs-webui/releases/tag/v2.12.4)\n7. Fixed the snap deployment [#8212](https://github.com/ipfs/go-ipfs/pull/8212)\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - chore: update deps\n - feat: webui v2.12.4\n - test: gateway response for bafkqaaa\n - fix: downgrade mimetype dependency\n - update go-libp2p to v0.14.3\n - bump snap to build with Go 1.16\n- github.com/libp2p/go-libp2p (v0.14.2 -> v0.14.3):\n - update go-tcp-transport to v0.2.3 and go-multiaddr to v0.3.3 ([libp2p/go-libp2p#1121](https://github.com/libp2p/go-libp2p/pull/1121))\n- github.com/libp2p/go-libp2p-pubsub (v0.4.1 -> v0.4.2):\n - release priority locks early when handling batches\n - don't respawn writer if we fail to open a stream; declare it a peer error\n - batch process dead peer notifications\n - use a priority lock instead of a semaphore\n - do the notification in a goroutine\n - emit new peer notification without holding the semaphore\n - use a semaphore for new peer notifications so that we don't block the event loop\n - don't accumulate pending goroutines from new connections\n - Make close concurrent safe\n - Fix close of closed channel\n- github.com/libp2p/go-libp2p-quic-transport (v0.11.1 -> v0.11.2):\n - update quic-go to v0.21.2\n- github.com/libp2p/go-tcp-transport (v0.2.2 -> v0.2.4):\n - collect metrics in a separate go routine ([libp2p/go-tcp-transport#82](https://github.com/libp2p/go-tcp-transport/pull/82))\n - fix: avoid logging \"invalid argument\" errors when setting keepalive ([libp2p/go-tcp-transport#83](https://github.com/libp2p/go-tcp-transport/pull/83))\n - Skip SetKeepAlivePeriod call on OpenBSD ([libp2p/go-tcp-transport#80](https://github.com/libp2p/go-tcp-transport/pull/80))\n - sync: update CI config files (#79) ([libp2p/go-tcp-transport#79](https://github.com/libp2p/go-tcp-transport/pull/79))\n- github.com/lucas-clemente/quic-go (v0.21.1 -> v0.21.2):\n - update qtls to include the crypto/tls fix of Go 1.16.6 / 1.15.14\n - cancel the PTO timer when all Handshake packets are acknowledged\n - update to Go 1.17rc1\n - update Ginkgo to v1.16.4 and Gomega to v1.13.0 ([lucas-clemente/quic-go#3139](https://github.com/lucas-clemente/quic-go/pull/3139))\n- github.com/multiformats/go-multiaddr (v0.3.2 -> v0.3.3):\n - guard against nil {Local,Remote}Addr() return values ([multiformats/go-multiaddr#155](https://github.com/multiformats/go-multiaddr/pull/155))\n - sync: update CI config files (#154) ([multiformats/go-multiaddr#154](https://github.com/multiformats/go-multiaddr/pull/154))\n\n### Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| vyzo | 8 | +205/-141 | 12 |\n| Marten Seemann | 7 | +127/-74 | 11 |\n| gammazero | 2 | +43/-5 | 3 |\n| Steven Allen | 1 | +13/-2 | 1 |\n| Adin Schmahmann | 3 | +13/-2 | 3 |\n| Marcin Rataj | 2 | +9/-1 | 2 |\n| Aaron Bieber | 1 | +6/-2 | 1 |\n\n## v0.9.0 2021-06-22\n\nWe're happy to announce go-ipfs 0.9.0. This release makes go-ipfs even more configurable with some fun experiments to boot. We're also deprecating or removing some uncommonly used features to make it easier for users to discover the easy ways to use go-ipfs safely and efficiently.\n\nAs usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will _not_ be called out in the release notes. Please make sure to update ASAP. See our [release process](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md#security-fix-policy) for details.\n\n### 🔦 Highlights\n\n#### 📦 Exporting of DAGs via Gateways\n\nGateways now support downloading arbitrary IPLD graphs via the `/api/v0/dag/export` endpoint. This endpoint works in the same way as the `ipfs dag export` command.\n\nOne major thing this enables is ability to verify data downloaded from public gateways. If you go to `https://somegateway.example.net/ipfs/bafyexample` you are using the old school HTTP transport, and trusting that the gateway is being well behaved. However, if you download the graph as a [DAG archive](https://github.com/ipld/specs/blob/master/block-layer/content-addressable-archives.md) then it is possible to verify that the data you downloaded does in fact match `bafyexample`.\n\nAdditionally, it was previously quite painful to download things other than UnixFS (files + directories) using gateways. It is now possible to download arbitrary IPLD graphs from gateways, making them useful as a general-purpose alternative to p2p transports.\n\nThis opens exciting opportunities in areas like thin clients, mobile browsers and IoT devices, which now can delegate IPFS resolution to any public gateway, and have ability to verify that the data received matches the requested hash.\n\n#### ☁ Custom DNS Resolvers\n\nResolution of DNS records for DNSLink and DNSAddrs means that names are sent in cleartext between the operating system and the DNS server provided by an ISP. In the past, the only way to customize DNS resolution in IPFS stack was to set up own DNS proxy server.\n\nThere is now the ability to [customize DNS resolution](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#dns) and override the default resolver from the OS with [DNS over HTTPS](https://en.wikipedia.org/wiki/DNS_over_HTTPS) (DoH) one. We made it really flexible: override can be applied globally, or per specific [TLD](https://en.wikipedia.org/wiki/Top-level_domain)/[FQDN](https://en.wikipedia.org/wiki/Fully_qualified_domain_name). Examples can be found in the [documentation](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#dns).\n\n#### 👪 Support for non-ICANN DNSLink names\n\nBuilding off of the support for custom DNS resolvers it is now possible to create DNSLink names not handled by ICANN and choose how that domain name will be resolved. An example of this is how ENS is supported, despite `.eth` not being an ICANN TLD you can point `.eth` to any ENS resolver you want (including a local one).\n\nWhile go-ipfs may have some DoH defaults for a few popular non-ICANN DNSLink names (e.g. ENS), you are free to use any protocol for a naming system and as long as it exposes a DNSLink record via a DNS endpoint you can make it work.\n\n#### 🖥️ Updated to the latest WebUI\n\nOur web interface now includes experimental support for pinning services, and various updates to _Files_ and _Peers_ screens.\n\nRemote pinning services added via the `ipfs pin remote service add` command are already detected, one can also add one from _Settings_ screen, and it will appear in _Set pinning_ interface on the _Files_ screen.\n\nData presented on the _Peers_ screen can now be copied by simply clicking on a specific cell, and a list of open streams gives better insight into how a local node interacts with a specific peer.\n\nSee release notes for [ipfs-webui v2.12](https://github.com/ipfs/ipfs-webui/releases/tag/v2.12.0) for screenshots and more details.\n\n#### 🔑 IPNS keys can now be exported via the CLI without stopping the daemon\n\n`ipfs key export` no longer requires interrupting `ipfs daemon` ✨\n\n#### 🕸 Experimental DHT Client and Provider System\n\nAn area of go-ipfs that has been historically tricky is how go-ipfs finds who has the data they are looking for. While the IPFS Public DHT is only one of the ways go-ipfs can find data it tends to be an important one. While since go-ipfs v0.5.0 the time to find content in the network has dropped significantly the time to put/get IPNS records or for a node to advertise the content it has still has much room for improvement.\n\nWe have been doing some experimenting and have an alternative DHT client that essentially trades off some resources and in return is much more performant. We have also included with the experimental DHT client a bulk provider system that takes advantage of the new client to more efficiently do many advertisements at a time\n\nThis work is quite new and still under development, however, the results so far have been promising especially for users with lots of data who have otherwise been having difficulty advertising their data into the IPFS Public DHT\n\nAs described in the experimental features [documentation](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#accelerated-dht-client) the experimental client can be enabled using the command below (or modifying the config file).\n\n```\nipfs config --json Experimental.AcceleratedDHTClient true\n```\n\nA few things to take note of when `AcceleratedDHTClient` is enabled:\n- go-ipfs will likely use more resources then previously\n- DHT queries will not be usable (i.e. finding which peers have some data, finding where a particular peer is, etc.) for the first 5-10 minutes of operation depending on your network conditions\n- There is an `ipfs stats provide` command that will help you track your provide/reprovide usage, if you are providing lots of data you may want to consider how to reduce the amount you are providing (e.g. [Reprovider Strategies](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#reproviderstrategy) and/or [Strategic Providing](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#strategic-providing))\n\nSee the [documentation](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#accelerated-dht-client) for more details.\n\n#### 🚶‍♀️ Migrations\n\n##### Migrations are now individually packaged\n\nWhile previously the go-ipfs [repo migration](https://github.com/ipfs/fs-repo-migrations) binary was monolithic and contained all migrations from previous go-ipfs versions the binaries are now packaged individually. However, the fs-repo-migrations binary is still there to help those who manually upgrade their repos to download all the individual migrations.\n\nThis means faster download times for upgrades, a much easier time building migrations for those who make use of custom plugins, and an easier time developing new migrations going forward.\n\n##### Configurable migration downloads enable downloading over IPFS\n\nPreviously the migration downloader built into go-ipfs downloaded the migrations from [dist.ipfs.tech](https://dist.ipfs.tech). While users could use tools like [ipfs-update](https://github.com/ipfs/ipfs-update) to download the migrations over IPFS or manually download the migrations (over IPFS or otherwise) themselves, this is now automated and configurable. Users can choose to download the migrations over IPFS or from any specified IPFS Gateway.\n\nThe configurable migration options are described in the config file [documentation](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#migration), although most users should not need to change the default settings.\n\nThe main benefit here is that users behind restrictive firewalls, or in offline/private deployments, won't have to run migrations manually, which is especially important for desktop use cases where go-ipfs is running inside of [IPFS Desktop](https://github.com/ipfs-shipyard/ipfs-desktop#readme) and [Brave](https://brave.com/ipfs-support/).\n\n#### 🍎 Published builds for Apple M1 hardware\n\nGo now supports building for Darwin ARM64, and we are now publishing those builds\n\n#### 👋 Deprecations and Feature Removals\n\n##### The `ipfs object` commands are now deprecated\n\nIn the last couple years most of the Object API's commands have become fulfillable using alternative APIs.\n\nThe utility of Object API's is limited to data in UnixFS-v1 (`dag-pb`) format. If you are still using it, it is highly recommended that you switch to the DAG `ipfs dag` (supports modern data types like `dag-cbor`) or Files `ipfs files` (more intuitive for working with `dag-pb`) APIs.\n\nWhile the Object API and commands are still usable they are now marked as deprecated and hidden from users on the command line to discourage further use. We also updated their `--help` text to point at the modern replacements.\n\n\n##### `X-Ipfs-Gateway-Prefix` is now deprecated\n\nIPFS community moved towards dedicated Origins (DNSLink and [subdomain gateways](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway)) which are much easier to isolate and reason about.\n\nSetting up `Gateway.PathPrefixes` and `X-Ipfs-Gateway-Prefix` is no longer necessary and support [will be removed in near future](https://github.com/ipfs/go-ipfs/issues/7702).\n\n##### Proquints support removed\n\nA little known feature that was not well used or documented and was more well known for the error message `Error: not a valid proquint string` users received when trying to download invalid IPNS or DNSLink names (e.g. `https://dweb.link/ipns/badname`). We have removed support for proquints as they were out of place and largely unused, however proquints are [valid multibases](https://github.com/multiformats/multibase/pull/78) so if there is renewed interest in them there is a way forward.\n\n##### SECIO support removed\n\nSECIO was deprecated and turned off by default given the prevalence of TLS and Noise support, SECIO support is now removed entirely.\n\n### Changelog\n\n- github.com/ipfs/go-ipfs:\n - chore: switch tar-utils dep to ipfs org\n - chore: update CHANGELOG\n - refactor: warning when bootstrap disabled by user\n - feat: print error on bootstrap failure\n - fix: typo in migration error\n - Release v0.9.0-rc2\n - refactor: improved humanNumber and humanSI\n - feat: humanized durations in stat provide\n - feat: humanized numbers in stat provide\n - feat: add a text output encoding for the stats provide command\n - fix: webui-2.12.3\n - refactor(pinmfs): log error if pre-existing pin failed (#8056) ([ipfs/go-ipfs#8056](https://github.com/ipfs/go-ipfs/pull/8056))\n - Release v0.9.0-rc1\n - Added support for an experimental DHT client and provider system via the Experiments.AcceleratedDHTClient config option ([ipfs/go-ipfs#8061](https://github.com/ipfs/go-ipfs/pull/8061))\n - feat: support DNSLink on non-ICANN DNS names ([ipfs/go-ipfs#8071](https://github.com/ipfs/go-ipfs/pull/8071))\n - update go-tcp-transport to v0.2.2 ([ipfs/go-ipfs#8129](https://github.com/ipfs/go-ipfs/pull/8129))\n - update quic-go to v0.21.0-rc.1 ([ipfs/go-ipfs#8125](https://github.com/ipfs/go-ipfs/pull/8125))\n - chore: bump minimum go version to 1.15 ([ipfs/go-ipfs#7944](https://github.com/ipfs/go-ipfs/pull/7944))\n - chore: update deps ([ipfs/go-ipfs#8128](https://github.com/ipfs/go-ipfs/pull/8128))\n - feat: allow key export in online mode ([ipfs/go-ipfs#8113](https://github.com/ipfs/go-ipfs/pull/8113))\n - Feat/migration ipfs download (#8064) ([ipfs/go-ipfs#8064](https://github.com/ipfs/go-ipfs/pull/8064))\n - feat: support custom DoH resolvers ([ipfs/go-ipfs#8068](https://github.com/ipfs/go-ipfs/pull/8068))\n - update go-libp2p to v0.14.0 ([ipfs/go-ipfs#8122](https://github.com/ipfs/go-ipfs/pull/8122))\n - feat(gw): expose /api/v0/dag/export on gateway port ([ipfs/go-ipfs#8111](https://github.com/ipfs/go-ipfs/pull/8111))\n - chore: update webui to 2.12.2 ([ipfs/go-ipfs#8097](https://github.com/ipfs/go-ipfs/pull/8097))\n - docs: deprecate object commands ([ipfs/go-ipfs#8098](https://github.com/ipfs/go-ipfs/pull/8098))\n - fix: omit empty pins slice when reporting pin progress ([ipfs/go-ipfs#8023](https://github.com/ipfs/go-ipfs/pull/8023))\n - Fix typo in comment ([ipfs/go-ipfs#8087](https://github.com/ipfs/go-ipfs/pull/8087))\n - fix: set systemd startup timeout to infinity ([ipfs/go-ipfs#8040](https://github.com/ipfs/go-ipfs/pull/8040))\n - fix(mkreleaselog): partially handle v2 modules ([ipfs/go-ipfs#8073](https://github.com/ipfs/go-ipfs/pull/8073))\n - Update migration sharness tests for new migrations (#8053) ([ipfs/go-ipfs#8053](https://github.com/ipfs/go-ipfs/pull/8053))\n - fix: make migrations log output to stdout ([ipfs/go-ipfs#8054](https://github.com/ipfs/go-ipfs/pull/8054))\n - fix(gw): remove hardcoded hostnames ([ipfs/go-ipfs#8069](https://github.com/ipfs/go-ipfs/pull/8069))\n - Fix transposed words in docs/config.md ([ipfs/go-ipfs#8051](https://github.com/ipfs/go-ipfs/pull/8051))\n - fix: update root help ([ipfs/go-ipfs#8052](https://github.com/ipfs/go-ipfs/pull/8052))\n - chore: don't docker tag rc as latest ([ipfs/go-ipfs#8055](https://github.com/ipfs/go-ipfs/pull/8055))\n - Add info to \"pin rm\" help about how to tell if pin is indirect ([ipfs/go-ipfs#8044](https://github.com/ipfs/go-ipfs/pull/8044))\n - build(deps): bump contrib.go.opencensus.io/exporter/prometheus from 0.2.0 to 0.3.0 ([ipfs/go-ipfs#8020](https://github.com/ipfs/go-ipfs/pull/8020))\n - fix(gw): remove use of Clear-Site-Data in subdomain router ([ipfs/go-ipfs#7890](https://github.com/ipfs/go-ipfs/pull/7890))\n - ([ipfs/go-ipfs#7857](https://github.com/ipfs/go-ipfs/pull/7857))\n - docs: clarification of the Strategic Providing functionality ([ipfs/go-ipfs#8035](https://github.com/ipfs/go-ipfs/pull/8035))\n - docs: cosmetic fixes of help text ([ipfs/go-ipfs#8010](https://github.com/ipfs/go-ipfs/pull/8010))\n - chore: deprecate Gateway.PathPrefixes ([ipfs/go-ipfs#7994](https://github.com/ipfs/go-ipfs/pull/7994))\n - Fix text contrast for dark mode ([ipfs/go-ipfs#8027](https://github.com/ipfs/go-ipfs/pull/8027))\n - Do not fetch recursive pins from pinner unnecessarily ([ipfs/go-ipfs#7883](https://github.com/ipfs/go-ipfs/pull/7883))\n - test(sharness): verify the list of exported metrics ([ipfs/go-ipfs#7987](https://github.com/ipfs/go-ipfs/pull/7987))\n - fix: return an error if repo verify is canceled ([ipfs/go-ipfs#7973](https://github.com/ipfs/go-ipfs/pull/7973))\n - doc: document security fix policy ([ipfs/go-ipfs#7991](https://github.com/ipfs/go-ipfs/pull/7991))\n - feat(gw): /ipfs/ipfs/{cid} → /ipfs/{cid} ([ipfs/go-ipfs#7930](https://github.com/ipfs/go-ipfs/pull/7930))\n - Fix: inaccuracies in MFS command documentation. ([ipfs/go-ipfs#8001](https://github.com/ipfs/go-ipfs/pull/8001))\n - Feat: Re-import InitializeKeyspace code from go-namesys ([ipfs/go-ipfs#7984](https://github.com/ipfs/go-ipfs/pull/7984))\n - revert registration of metrics against unexposed prom registry ([ipfs/go-ipfs#7986](https://github.com/ipfs/go-ipfs/pull/7986))\n - Extract the namesys and the keystore submodules ([ipfs/go-ipfs#7925](https://github.com/ipfs/go-ipfs/pull/7925))\n - split core/commands/dag into individual files for different subcommands ([ipfs/go-ipfs#7970](https://github.com/ipfs/go-ipfs/pull/7970))\n - test(sharness): pass correct timeout format to go-timeout ([ipfs/go-ipfs#7971](https://github.com/ipfs/go-ipfs/pull/7971))\n - fix race condition when logging requests ([ipfs/go-ipfs#7953](https://github.com/ipfs/go-ipfs/pull/7953))\n - fix some sharness-in-CI issues ([ipfs/go-ipfs#7946](https://github.com/ipfs/go-ipfs/pull/7946))\n - chore: update deps ([ipfs/go-ipfs#7941](https://github.com/ipfs/go-ipfs/pull/7941))\n - fix: correctly return pin ls errors ([ipfs/go-ipfs#7942](https://github.com/ipfs/go-ipfs/pull/7942))\n - feat: remove secio support ([ipfs/go-ipfs#7943](https://github.com/ipfs/go-ipfs/pull/7943))\n - Set supported platforms by go-version ([ipfs/go-ipfs#7927](https://github.com/ipfs/go-ipfs/pull/7927))\n - docs: tips on debugging Policies.MFS (#7929) ([ipfs/go-ipfs#7929](https://github.com/ipfs/go-ipfs/pull/7929))\n - docs: fix DNSLink gw recipe ([ipfs/go-ipfs#7932](https://github.com/ipfs/go-ipfs/pull/7932))\n - Merge branch 'release'\n - docs: RepinInterval\n - style: docs/config.md\n - style: improved MFS PinName example\n - docs: Pinning.RemoteServices.Policies\n - peering: add logs before many-second waits ([ipfs/go-ipfs#7904](https://github.com/ipfs/go-ipfs/pull/7904))\n - all: gofmt -s ([ipfs/go-ipfs#7900](https://github.com/ipfs/go-ipfs/pull/7900))\n- github.com/ipfs/go-bitswap (v0.3.3 -> v0.3.4):\n - remove Makefile ([ipfs/go-bitswap#483](https://github.com/ipfs/go-bitswap/pull/483))\n - test: deflake engine test ([ipfs/go-bitswap#480](https://github.com/ipfs/go-bitswap/pull/480))\n - test: deflake large-message test ([ipfs/go-bitswap#479](https://github.com/ipfs/go-bitswap/pull/479))\n - fix: fix alignment of stats struct in virtual network ([ipfs/go-bitswap#478](https://github.com/ipfs/go-bitswap/pull/478))\n - fix(network): impl: add timeout in newStreamToPeer call ([ipfs/go-bitswap#477](https://github.com/ipfs/go-bitswap/pull/477))\n - fix staticcheck ([ipfs/go-bitswap#474](https://github.com/ipfs/go-bitswap/pull/474))\n - ignore transient connections ([ipfs/go-bitswap#470](https://github.com/ipfs/go-bitswap/pull/470))\n - fix a startup race by creating the blockstoremanager process on init ([ipfs/go-bitswap#465](https://github.com/ipfs/go-bitswap/pull/465))\n- github.com/ipfs/go-block-format (v0.0.2 -> v0.0.3):\n - doc: add a lead maintainer ([ipfs/go-block-format#16](https://github.com/ipfs/go-block-format/pull/16))\n- github.com/ipfs/go-graphsync (v0.6.0 -> v0.8.0):\n - docs(CHANGELOG): update for v0.8.0\n - Update for LinkSystem (#161) ([ipfs/go-graphsync#161](https://github.com/ipfs/go-graphsync/pull/161))\n - Round out diagnostic parameters (#157) ([ipfs/go-graphsync#157](https://github.com/ipfs/go-graphsync/pull/157))\n - map response codes to names (#148) ([ipfs/go-graphsync#148](https://github.com/ipfs/go-graphsync/pull/148))\n - Discard http output (#156) ([ipfs/go-graphsync#156](https://github.com/ipfs/go-graphsync/pull/156))\n - Add debug logging (#121) ([ipfs/go-graphsync#121](https://github.com/ipfs/go-graphsync/pull/121))\n - Add optional HTTP comparison (#153) ([ipfs/go-graphsync#153](https://github.com/ipfs/go-graphsync/pull/153))\n - docs(architecture): update architecture docs (#154) ([ipfs/go-graphsync#154](https://github.com/ipfs/go-graphsync/pull/154))\n - release v0.7.0 ([ipfs/go-graphsync#152](https://github.com/ipfs/go-graphsync/pull/152))\n - chore: update deps (#151) ([ipfs/go-graphsync#151](https://github.com/ipfs/go-graphsync/pull/151))\n - Automatically record heap profiles in test plans (#147) ([ipfs/go-graphsync#147](https://github.com/ipfs/go-graphsync/pull/147))\n - feat(deps): update go-ipld-prime v0.7.0 (#145) ([ipfs/go-graphsync#145](https://github.com/ipfs/go-graphsync/pull/145))\n - Release/v0.6.0 ([ipfs/go-graphsync#144](https://github.com/ipfs/go-graphsync/pull/144))\n- github.com/ipfs/go-ipfs-blockstore (v0.1.4 -> v0.1.6):\n - use bloom filter in GetSize\n- github.com/ipfs/go-ipfs-config (v0.12.0 -> v0.14.0):\n - Added Experiments.AcceleratedDHTClient option ([ipfs/go-ipfs-config#125](https://github.com/ipfs/go-ipfs-config/pull/125))\n - Add config for downloading repo migrations ([ipfs/go-ipfs-config#128](https://github.com/ipfs/go-ipfs-config/pull/128))\n - remove duplicate entries in defaultServerFilters ([ipfs/go-ipfs-config#121](https://github.com/ipfs/go-ipfs-config/pull/121))\n - add custom DNS Resolver configuration ([ipfs/go-ipfs-config#126](https://github.com/ipfs/go-ipfs-config/pull/126))\n- github.com/ipfs/go-ipfs-provider (v0.4.3 -> v0.5.1):\n - Fix batched providing of empty keys ([ipfs/go-ipfs-provider#37](https://github.com/ipfs/go-ipfs-provider/pull/37))\n - Bulk Provide/Reproviding System (#34) ([ipfs/go-ipfs-provider#34](https://github.com/ipfs/go-ipfs-provider/pull/34))\n - chore: update the Usage part of readme ([ipfs/go-ipfs-provider#33](https://github.com/ipfs/go-ipfs-provider/pull/33))\n - Retract and revert 1.0.0 ([ipfs/go-ipfs-provider#31](https://github.com/ipfs/go-ipfs-provider/pull/31))\n - replace go-merkledag with go-fetcher ([ipfs/go-ipfs-provider#30](https://github.com/ipfs/go-ipfs-provider/pull/30))\n- github.com/ipfs/go-ipld-git (v0.0.3 -> v0.0.4):\n - add license file so it can be found by go-licenses ([ipfs/go-ipld-git#42](https://github.com/ipfs/go-ipld-git/pull/42))\n- github.com/ipfs/go-ipns (v0.0.2 -> v0.1.0):\n - Add support for extensible records (and v2 signature)\n- github.com/ipfs/go-log (v1.0.4 -> v1.0.5):\n - chore: update v1 deps ([ipfs/go-log#108](https://github.com/ipfs/go-log/pull/108))\n- github.com/ipfs/go-log/v2 (v2.1.1 -> v2.1.3):\n - doc(README): use circle-ci badge ([ipfs/go-log#106](https://github.com/ipfs/go-log/pull/106))\n - feat: add ability to specify labels for all loggers ([ipfs/go-log#105](https://github.com/ipfs/go-log/pull/105))\n - Add an option to pass URL to zap ([ipfs/go-log#101](https://github.com/ipfs/go-log/pull/101))\n - enable configuring several log outputs ([ipfs/go-log#98](https://github.com/ipfs/go-log/pull/98))\n - Fix caller not being added ([ipfs/go-log#96](https://github.com/ipfs/go-log/pull/96))\n- github.com/ipfs/go-unixfs (v0.2.4 -> v0.2.5):\n - correct file size for raw node ([ipfs/go-unixfs#88](https://github.com/ipfs/go-unixfs/pull/88))\n- github.com/ipld/go-car (v0.1.1-0.20201015032735-ff6ccdc46acc -> v0.3.1):\n - chore: make sure we get an error where we expect one\n - chore: refactor header tests to iterate over a struct\n - chore: add header error tests\n - fix: lint errors\n - fix: go mod tidy\n - chore: update go.mod to 1.15\n - fix: ReadHeader return value mismatch\n - Updates for ipld linksystem branch ([ipld/go-car#56](https://github.com/ipld/go-car/pull/56))\n - replace go-ipld-prime-proto with go-codec-dagpb\n - fix staticcheck errors ([ipld/go-car#67](https://github.com/ipld/go-car/pull/67))\n - chore: switch to a single license file ([ipld/go-car#59](https://github.com/ipld/go-car/pull/59))\n - chore: remove LICENSE ([ipld/go-car#58](https://github.com/ipld/go-car/pull/58))\n - chore: relicense ([ipld/go-car#57](https://github.com/ipld/go-car/pull/57))\n - ci: remove travis support ([ipld/go-car#55](https://github.com/ipld/go-car/pull/55))\n - run gofmt -s\n - Allow user defined block hooks when using two step write for selective cars ([ipld/go-car#37](https://github.com/ipld/go-car/pull/37))\n - feat: handle mid-varint EOF case as UnexpectedEOF\n - fix: main NewReader call\n- github.com/ipld/go-ipld-prime (v0.5.1-0.20201021195245-109253e8a018 -> v0.9.1-0.20210324083106-dc342a9917db):\n - Add option to tell link system storage is trusted and we can skip hash on read ([ipld/go-ipld-prime#149](https://github.com/ipld/go-ipld-prime/pull/149))\n - implement non-dag cbor codec ([ipld/go-ipld-prime#153](https://github.com/ipld/go-ipld-prime/pull/153))\n - add non-dag json codec ([ipld/go-ipld-prime#152](https://github.com/ipld/go-ipld-prime/pull/152))\n - typo fixes\n - mark v0.9.0\n - Changelog: more backfill :)\n - hackme: about merge strategies.\n - Dropping .gopath and other unmaintained scripts.\n - introduce LinkSystem ([ipld/go-ipld-prime#143](https://github.com/ipld/go-ipld-prime/pull/143))\n - Readme updates.\n - codec/raw: implement the raw codec\n - add an ADL interface type\n - schema/gen/go: cache genned code in os.TempDir\n - fluent/qp: finish writing all data model helpers\n - fluent: add qp, a different spin on quip\n - schema/gen/go: prevent some unkeyed literal vet errors\n - schema/gen/go: remove two common subtest levels\n - use %q in error strings\n - schema/gen/go: please vet a bit more\n - Introduce 'quip' data building helpers. ([ipld/go-ipld-prime#134](https://github.com/ipld/go-ipld-prime/pull/134))\n - gengo: support for unions with stringprefix representation. ([ipld/go-ipld-prime#133](https://github.com/ipld/go-ipld-prime/pull/133))\n - target of opportunity DRY improvement: use more shared templates for structs with stringjoin representations.\n - fix small consistency typo in gen function names.\n - drop old generation mechanisms that were already deprecated.\n - error type cleanup, and helpers.\n - v0.7.0 and changelog update\n - Revert \"rename AssignNode to ConvertFrom\"\n - Implement traversal.FocusedTransform. ([ipld/go-ipld-prime#130](https://github.com/ipld/go-ipld-prime/pull/130))\n - Update a few more lingering ReprKind references.\n - all: rename schema.Kind to TypeKind, ipld.ReprKind to Kind ([ipld/go-ipld-prime#127](https://github.com/ipld/go-ipld-prime/pull/127))\n - all: rename AssignNode to ConvertFrom\n - all: rewrite interfaces and APIs to support int64\n - mark v0.6.0\n - clean up node/gendemo regeneration ([ipld/go-ipld-prime#123](https://github.com/ipld/go-ipld-prime/pull/123))\n - cleanup: drop orphaned gitignore file.\n - Schema types rebased to use codegen types for the data ([ipld/go-ipld-prime#107](https://github.com/ipld/go-ipld-prime/pull/107))\n - codegen: assembler for struct with map representation validates all non-optional fields are present ([ipld/go-ipld-prime#121](https://github.com/ipld/go-ipld-prime/pull/121))\n - changelog: backfill.\n - fluent: finish out matrix of helper methods, and fix error handling of the non-Must methods.\n - all: fix a lot of \"unkeyed literal\" vet warnings\n - node/mixins: use simpler filenames\n - node/gendemo: use the new code generator\n - Merge pull request #96 , originally known as ipld/cidlink-only-usable-as-ptr\n - Codec revamp ([ipld/go-ipld-prime#112](https://github.com/ipld/go-ipld-prime/pull/112))\n - Allow overridden types (#116) ([ipld/go-ipld-prime#116](https://github.com/ipld/go-ipld-prime/pull/116))\n - add import to ipld in ipldsch_types.go ([ipld/go-ipld-prime#115](https://github.com/ipld/go-ipld-prime/pull/115))\n - Codegen output rearrange ([ipld/go-ipld-prime#105](https://github.com/ipld/go-ipld-prime/pull/105))\n - Validate struct builder sufficiency ([ipld/go-ipld-prime#111](https://github.com/ipld/go-ipld-prime/pull/111))\n - Fresh take on codec APIs, and some tokenization utilities. ([ipld/go-ipld-prime#101](https://github.com/ipld/go-ipld-prime/pull/101))\n - Add a demo ADL (rot13adl) ([ipld/go-ipld-prime#98](https://github.com/ipld/go-ipld-prime/pull/98))\n - Introduce traversal function that selects links out of a tree. ([ipld/go-ipld-prime#110](https://github.com/ipld/go-ipld-prime/pull/110))\n - Codegen various improvements ([ipld/go-ipld-prime#106](https://github.com/ipld/go-ipld-prime/pull/106))\n- github.com/libp2p/go-conn-security-multistream (v0.2.0 -> v0.2.1):\n - Implement support for simultaneous open (#14) ([libp2p/go-conn-security-multistream#14](https://github.com/libp2p/go-conn-security-multistream/pull/14))\n- github.com/libp2p/go-libp2p (v0.13.0 -> v0.14.2):\n - Fix race in adding connections to connsByPeer ([libp2p/go-libp2p#1116](https://github.com/libp2p/go-libp2p/pull/1116))\n - speed up the mock tests ([libp2p/go-libp2p#1103](https://github.com/libp2p/go-libp2p/pull/1103))\n - remove slow ObservedAddrManager test that doesn't test anything ([libp2p/go-libp2p#1104](https://github.com/libp2p/go-libp2p/pull/1104))\n - remove Codecov config ([libp2p/go-libp2p#1100](https://github.com/libp2p/go-libp2p/pull/1100))\n - doc: document standard connection manager ([libp2p/go-libp2p#1099](https://github.com/libp2p/go-libp2p/pull/1099))\n - run go mod tidy in the examples ([libp2p/go-libp2p#1098](https://github.com/libp2p/go-libp2p/pull/1098))\n - Cleanup some remaining examples nits ([libp2p/go-libp2p#1097](https://github.com/libp2p/go-libp2p/pull/1097))\n - chore: bring examples back into repository and add tests ([libp2p/go-libp2p#1092](https://github.com/libp2p/go-libp2p/pull/1092))\n - fix(mkreleasenotes): handle first commit ([libp2p/go-libp2p#1095](https://github.com/libp2p/go-libp2p/pull/1095))\n - doc: add a basic release process ([libp2p/go-libp2p#1080](https://github.com/libp2p/go-libp2p/pull/1080))\n - chore: update yamux ([libp2p/go-libp2p#1089](https://github.com/libp2p/go-libp2p/pull/1089))\n - fix: re-expose AutoNAT service on BasicHost ([libp2p/go-libp2p#1088](https://github.com/libp2p/go-libp2p/pull/1088))\n - remove NEWS.md ([libp2p/go-libp2p#1086](https://github.com/libp2p/go-libp2p/pull/1086))\n - test: deflake TestProtoDowngrade ([libp2p/go-libp2p#1084](https://github.com/libp2p/go-libp2p/pull/1084))\n - sync: update CI config files (and fix tests) ([libp2p/go-libp2p#1083](https://github.com/libp2p/go-libp2p/pull/1083))\n - static check fixes ([libp2p/go-libp2p#1076](https://github.com/libp2p/go-libp2p/pull/1076))\n - fix go vet ([libp2p/go-libp2p#1075](https://github.com/libp2p/go-libp2p/pull/1075))\n - option for custom dns resolver ([libp2p/go-libp2p#1073](https://github.com/libp2p/go-libp2p/pull/1073))\n - chore: update deps ([libp2p/go-libp2p#1066](https://github.com/libp2p/go-libp2p/pull/1066))\n - fix autonat race ([libp2p/go-libp2p#1062](https://github.com/libp2p/go-libp2p/pull/1062))\n - use transient connections in identify streams ([libp2p/go-libp2p#1061](https://github.com/libp2p/go-libp2p/pull/1061))\n - Emit event for User's NAT Type i.e. Hard NAT or Easy NAT (#1042) ([libp2p/go-libp2p#1042](https://github.com/libp2p/go-libp2p/pull/1042))\n - Finish and Test the simultaneous connect problem in libp2p peers (#1041) ([libp2p/go-libp2p#1041](https://github.com/libp2p/go-libp2p/pull/1041))\n - Close peerstore and document Host Close (#1037) ([libp2p/go-libp2p#1037](https://github.com/libp2p/go-libp2p/pull/1037))\n - Timeout all Identify stream reads (#1032) ([libp2p/go-libp2p#1032](https://github.com/libp2p/go-libp2p/pull/1032))\n- github.com/libp2p/go-libp2p-autonat (v0.4.0 -> v0.4.2):\n - Fix: Stream read timeout ([libp2p/go-libp2p-autonat#99](https://github.com/libp2p/go-libp2p-autonat/pull/99))\n - fix: simplify address replacement ([libp2p/go-libp2p-autonat#102](https://github.com/libp2p/go-libp2p-autonat/pull/102))\n - replace the port number for double NAT mapping ([libp2p/go-libp2p-autonat#101](https://github.com/libp2p/go-libp2p-autonat/pull/101))\n- github.com/libp2p/go-libp2p-core (v0.8.0 -> v0.8.5):\n - mind the dot.\n - context option for simultaneous connect\n - Event for user's NAT Device Type: Tell user if the node is behind an Easy or Hard NAT (#173) ([libp2p/go-libp2p-core#173](https://github.com/libp2p/go-libp2p-core/pull/173))\n - address aarshian nitpicks\n - make UseTransient context option take a reason argument, for consistency with other options\n - abstract Conn Stat interface for threading\n - Update network/context.go\n - add ErrTransientConn error\n - add support for transient connections\n - more docs for stream fncs (#183) ([libp2p/go-libp2p-core#183](https://github.com/libp2p/go-libp2p-core/pull/183))\n - refactor: use a helper type to decode AddrInfo from JSON (#178) ([libp2p/go-libp2p-core#178](https://github.com/libp2p/go-libp2p-core/pull/178))\n - fix stream docs (#182) ([libp2p/go-libp2p-core#182](https://github.com/libp2p/go-libp2p-core/pull/182))\n - context to force direct dial (#181) ([libp2p/go-libp2p-core#181](https://github.com/libp2p/go-libp2p-core/pull/181))\n - Secure Muxer Interface (#180) ([libp2p/go-libp2p-core#180](https://github.com/libp2p/go-libp2p-core/pull/180))\n- github.com/libp2p/go-libp2p-discovery (v0.5.0 -> v0.5.1):\n - Fix hang in BackoffDiscovery.FindPeers when requesting limit lower than number of peers available ([libp2p/go-libp2p-discovery#69](https://github.com/libp2p/go-libp2p-discovery/pull/69))\n - fix staticcheck ([libp2p/go-libp2p-discovery#70](https://github.com/libp2p/go-libp2p-discovery/pull/70))\n- github.com/libp2p/go-libp2p-kad-dht (v0.11.1 -> v0.12.2):\n - fullrt rework batching (#720) ([libp2p/go-libp2p-kad-dht#720](https://github.com/libp2p/go-libp2p-kad-dht/pull/720))\n - sync: update CI config files ([libp2p/go-libp2p-kad-dht#712](https://github.com/libp2p/go-libp2p-kad-dht/pull/712))\n - fix staticcheck ([libp2p/go-libp2p-kad-dht#721](https://github.com/libp2p/go-libp2p-kad-dht/pull/721))\n - fix: fullrt dht bug fixes ([libp2p/go-libp2p-kad-dht#719](https://github.com/libp2p/go-libp2p-kad-dht/pull/719))\n - Crawler based DHT client (#709) ([libp2p/go-libp2p-kad-dht#709](https://github.com/libp2p/go-libp2p-kad-dht/pull/709))\n - test: fix unique addr check ([libp2p/go-libp2p-kad-dht#714](https://github.com/libp2p/go-libp2p-kad-dht/pull/714))\n - chore: update deps ([libp2p/go-libp2p-kad-dht#713](https://github.com/libp2p/go-libp2p-kad-dht/pull/713))\n - Add basic crawler (#663) ([libp2p/go-libp2p-kad-dht#663](https://github.com/libp2p/go-libp2p-kad-dht/pull/663))\n - various staticcheck fixes ([libp2p/go-libp2p-kad-dht#710](https://github.com/libp2p/go-libp2p-kad-dht/pull/710))\n - findpeer should work even on peers that are not part of DHT queries ([libp2p/go-libp2p-kad-dht#711](https://github.com/libp2p/go-libp2p-kad-dht/pull/711))\n - Extract DHT message sender from the DHT ([libp2p/go-libp2p-kad-dht#659](https://github.com/libp2p/go-libp2p-kad-dht/pull/659))\n- github.com/libp2p/go-libp2p-noise (v0.1.2 -> v0.2.0):\n - Update github.com/flynn/noise to address nonce handling security issues ([libp2p/go-libp2p-noise#95](https://github.com/libp2p/go-libp2p-noise/pull/95))\n - fix staticcheck ([libp2p/go-libp2p-noise#96](https://github.com/libp2p/go-libp2p-noise/pull/96))\n - chore: update deps ([libp2p/go-libp2p-noise#94](https://github.com/libp2p/go-libp2p-noise/pull/94))\n - chore: relicense MIT/Apache-2.0 ([libp2p/go-libp2p-noise#93](https://github.com/libp2p/go-libp2p-noise/pull/93))\n- github.com/libp2p/go-libp2p-peerstore (v0.2.6 -> v0.2.7):\n - fix: delete addrs when \"updating\" them to zero ([libp2p/go-libp2p-peerstore#157](https://github.com/libp2p/go-libp2p-peerstore/pull/157))\n- github.com/libp2p/go-libp2p-quic-transport (v0.10.0 -> v0.11.1):\n - update quic-go, enable QUIC v1 (RFC 9000) ([libp2p/go-libp2p-quic-transport#207](https://github.com/libp2p/go-libp2p-quic-transport/pull/207))\n - update quic-go to v0.21.0-rc2 ([libp2p/go-libp2p-quic-transport#206](https://github.com/libp2p/go-libp2p-quic-transport/pull/206))\n - increase test timeout to reduce flakiness of test on Windows ([libp2p/go-libp2p-quic-transport#204](https://github.com/libp2p/go-libp2p-quic-transport/pull/204))\n - correctly export version negotiation failures to Prometheus ([libp2p/go-libp2p-quic-transport#205](https://github.com/libp2p/go-libp2p-quic-transport/pull/205))\n - update quic-go to v0.20.1 ([libp2p/go-libp2p-quic-transport#201](https://github.com/libp2p/go-libp2p-quic-transport/pull/201))\n - expose some Prometheus metrics ([libp2p/go-libp2p-quic-transport#200](https://github.com/libp2p/go-libp2p-quic-transport/pull/200))\n - update quic-go to v0.20.0 ([libp2p/go-libp2p-quic-transport#198](https://github.com/libp2p/go-libp2p-quic-transport/pull/198))\n - reduce the zstd window size from 8 MB to 32 KB ([libp2p/go-libp2p-quic-transport#195](https://github.com/libp2p/go-libp2p-quic-transport/pull/195))\n - compress qlogs when the QUIC connection is closed ([libp2p/go-libp2p-quic-transport#193](https://github.com/libp2p/go-libp2p-quic-transport/pull/193))\n - switch from gzip to zstd for qlog compression ([libp2p/go-libp2p-quic-transport#190](https://github.com/libp2p/go-libp2p-quic-transport/pull/190))\n- github.com/libp2p/go-libp2p-swarm (v0.4.0 -> v0.5.0):\n - run connection gating tests on both TCP and QUIC ([libp2p/go-libp2p-swarm#258](https://github.com/libp2p/go-libp2p-swarm/pull/258))\n - fix: avoid returning typed nils ([libp2p/go-libp2p-swarm#257](https://github.com/libp2p/go-libp2p-swarm/pull/257))\n - fix staticcheck ([libp2p/go-libp2p-swarm#255](https://github.com/libp2p/go-libp2p-swarm/pull/255))\n - fix go vet ([libp2p/go-libp2p-swarm#253](https://github.com/libp2p/go-libp2p-swarm/pull/253))\n - New Dialer ([libp2p/go-libp2p-swarm#243](https://github.com/libp2p/go-libp2p-swarm/pull/243))\n - fix: use 64bit stream/conn IDs ([libp2p/go-libp2p-swarm#247](https://github.com/libp2p/go-libp2p-swarm/pull/247))\n - feat: close transports that implement io.Closer ([libp2p/go-libp2p-swarm#227](https://github.com/libp2p/go-libp2p-swarm/pull/227))\n - fix swarm transient conn (#241) ([libp2p/go-libp2p-swarm#241](https://github.com/libp2p/go-libp2p-swarm/pull/241))\n - Support for Hole punching (#233) ([libp2p/go-libp2p-swarm#233](https://github.com/libp2p/go-libp2p-swarm/pull/233))\n - Treat transient connections as opt-in when opening new streams ([libp2p/go-libp2p-swarm#236](https://github.com/libp2p/go-libp2p-swarm/pull/236))\n - avoid assigning a function to a variable ([libp2p/go-libp2p-swarm#239](https://github.com/libp2p/go-libp2p-swarm/pull/239))\n - only listen on localhost in tests ([libp2p/go-libp2p-swarm#238](https://github.com/libp2p/go-libp2p-swarm/pull/238))\n - prevent dialing addresses that we're listening on ([libp2p/go-libp2p-swarm#237](https://github.com/libp2p/go-libp2p-swarm/pull/237))\n - Enable QUIC in Test Swarm (#235) ([libp2p/go-libp2p-swarm#235](https://github.com/libp2p/go-libp2p-swarm/pull/235))\n- github.com/libp2p/go-libp2p-transport-upgrader (v0.4.0 -> v0.4.2):\n - Expose underlying transport connection stat where available ([libp2p/go-libp2p-transport-upgrader#71](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/71))\n - Implement support for simultaneous open (#25) ([libp2p/go-libp2p-transport-upgrader#25](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/25))\n- github.com/libp2p/go-libp2p-yamux (v0.5.1 -> v0.5.4):\n - remove Makefile ([libp2p/go-libp2p-yamux#35](https://github.com/libp2p/go-libp2p-yamux/pull/35))\n- github.com/libp2p/go-netroute (v0.1.3 -> v0.1.6):\n - add js stub impl\n- github.com/libp2p/go-sockaddr (v0.0.2 -> v0.1.1):\n - fix: allocate \"any\" socket type then cast ([libp2p/go-sockaddr#20](https://github.com/libp2p/go-sockaddr/pull/20))\n - fix: remove CGO functions ([libp2p/go-sockaddr#18](https://github.com/libp2p/go-sockaddr/pull/18))\n- github.com/libp2p/go-tcp-transport (v0.2.1 -> v0.2.2):\n - use log.Warn instead of log.Warning ([libp2p/go-tcp-transport#77](https://github.com/libp2p/go-tcp-transport/pull/77))\n - add bandwidth-related metrics (for Linux and OSX) ([libp2p/go-tcp-transport#76](https://github.com/libp2p/go-tcp-transport/pull/76))\n - expose some Prometheus metrics ([libp2p/go-tcp-transport#75](https://github.com/libp2p/go-tcp-transport/pull/75))\n - enable TCP keepalives ([libp2p/go-tcp-transport#73](https://github.com/libp2p/go-tcp-transport/pull/73))\n - stop using the deprecated go-multiaddr-net package ([libp2p/go-tcp-transport#72](https://github.com/libp2p/go-tcp-transport/pull/72))\n- github.com/libp2p/go-yamux/v2 (v2.0.0 -> v2.2.0):\n - make the initial stream receive window configurable ([libp2p/go-yamux#59](https://github.com/libp2p/go-yamux/pull/59))\n - set initial window size to spec value (256 kB), remove config option ([libp2p/go-yamux#57](https://github.com/libp2p/go-yamux/pull/57))\n - fix: don't change the receive window if we're forcing an update ([libp2p/go-yamux#56](https://github.com/libp2p/go-yamux/pull/56))\n - sync: update CI config files ([libp2p/go-yamux#55](https://github.com/libp2p/go-yamux/pull/55))\n - increase the receive window size if we're sending updates to frequently ([libp2p/go-yamux#54](https://github.com/libp2p/go-yamux/pull/54))\n - remove unused Stream.Shrink() method ([libp2p/go-yamux#52](https://github.com/libp2p/go-yamux/pull/52))\n - remove misleading comment about the MaxMessageSize ([libp2p/go-yamux#50](https://github.com/libp2p/go-yamux/pull/50))\n - clean up the receive window check ([libp2p/go-yamux#49](https://github.com/libp2p/go-yamux/pull/49))\n - don't reslice byte slices taking from the buffer ([libp2p/go-yamux#48](https://github.com/libp2p/go-yamux/pull/48))\n - don't reimplement io.ReadFull ([libp2p/go-yamux#38](https://github.com/libp2p/go-yamux/pull/38))\n - remove the recvLock in the stream ([libp2p/go-yamux#42](https://github.com/libp2p/go-yamux/pull/42))\n - remove the sendLock in the stream ([libp2p/go-yamux#41](https://github.com/libp2p/go-yamux/pull/41))\n - remove misleading statement about NAT traversal ([libp2p/go-yamux#45](https://github.com/libp2p/go-yamux/pull/45))\n - remove .gx directory, add last gx version to README ([libp2p/go-yamux#43](https://github.com/libp2p/go-yamux/pull/43))\n - reduce usage of goto ([libp2p/go-yamux#40](https://github.com/libp2p/go-yamux/pull/40))\n - remove unused error return value in Stream.processFlags ([libp2p/go-yamux#39](https://github.com/libp2p/go-yamux/pull/39))\n- github.com/lucas-clemente/quic-go (v0.19.3 -> v0.21.1):\n - add support for Go 1.17 Beta 1 ([lucas-clemente/quic-go#3203](https://github.com/lucas-clemente/quic-go/pull/3203))\n - add a CI test that go mod vendor works ([lucas-clemente/quic-go#3202](https://github.com/lucas-clemente/quic-go/pull/3202))\n - prevent go mod vendor from stumbling over the Go 1.18 file ([lucas-clemente/quic-go#3195](https://github.com/lucas-clemente/quic-go/pull/3195))\n - remove CipherSuiteName and HkdfExtract for Go 1.17 ([lucas-clemente/quic-go#3192](https://github.com/lucas-clemente/quic-go/pull/3192))\n - fix relocation target for cipherSuiteTLS13ByID in Go 1.17\n - use HkdfExtract from x/crypto ([lucas-clemente/quic-go#3173](https://github.com/lucas-clemente/quic-go/pull/3173))\n - add support for QUIC v1, RFC 9000 ([lucas-clemente/quic-go#3190](https://github.com/lucas-clemente/quic-go/pull/3190))\n - use tls.CipherSuiteName, instead of wrapping it in the qtls package ([lucas-clemente/quic-go#3174](https://github.com/lucas-clemente/quic-go/pull/3174))\n - use a pre-generated test vectors to test hkdfExpandLabel ([lucas-clemente/quic-go#3175](https://github.com/lucas-clemente/quic-go/pull/3175))\n - reduce flakiness of packet number generation test ([lucas-clemente/quic-go#3181](https://github.com/lucas-clemente/quic-go/pull/3181))\n - simplify the qtls tests ([lucas-clemente/quic-go#3185](https://github.com/lucas-clemente/quic-go/pull/3185))\n - add support for Go 1.17 (tip) ([lucas-clemente/quic-go#3182](https://github.com/lucas-clemente/quic-go/pull/3182))\n - prevent quic-go from building on Go 1.17 ([lucas-clemente/quic-go#3180](https://github.com/lucas-clemente/quic-go/pull/3180))\n - fix DONT_FRAGMENT error when using a IPv6 connection on Windows ([lucas-clemente/quic-go#3178](https://github.com/lucas-clemente/quic-go/pull/3178))\n - use net.ErrClosed (for Go 1.16) ([lucas-clemente/quic-go#3163](https://github.com/lucas-clemente/quic-go/pull/3163))\n - use the new error types to log the reason why a connection is closed ([lucas-clemente/quic-go#3166](https://github.com/lucas-clemente/quic-go/pull/3166))\n - fix race condition in deadline integration test ([lucas-clemente/quic-go#3165](https://github.com/lucas-clemente/quic-go/pull/3165))\n - add support for QUIC v1 ([lucas-clemente/quic-go#3160](https://github.com/lucas-clemente/quic-go/pull/3160))\n - rework error return values ([lucas-clemente/quic-go#3159](https://github.com/lucas-clemente/quic-go/pull/3159))\n - declare Path MTU probe packets lost with the early retransmit timer ([lucas-clemente/quic-go#3152](https://github.com/lucas-clemente/quic-go/pull/3152))\n - declare the handshake confirmed when receiving an ACK for a 1-RTT packet ([lucas-clemente/quic-go#3148](https://github.com/lucas-clemente/quic-go/pull/3148))\n - trace and qlog version selection / negotiation ([lucas-clemente/quic-go#3153](https://github.com/lucas-clemente/quic-go/pull/3153))\n - set the don't fragment (DF) bit on Windows (#3155) ([lucas-clemente/quic-go#3155](https://github.com/lucas-clemente/quic-go/pull/3155))\n - fix doc comment for Tracer.TracerForConnection ([lucas-clemente/quic-go#3154](https://github.com/lucas-clemente/quic-go/pull/3154))\n - make it possible to associate a ConnectionTracer with a Session ([lucas-clemente/quic-go#3146](https://github.com/lucas-clemente/quic-go/pull/3146))\n - remove the .editorconfig ([lucas-clemente/quic-go#3147](https://github.com/lucas-clemente/quic-go/pull/3147))\n - don't use a lower RTT than 5ms after receiving a Retry packet ([lucas-clemente/quic-go#3129](https://github.com/lucas-clemente/quic-go/pull/3129))\n - don't pass the QUIC version to the StartedConnection event ([lucas-clemente/quic-go#3109](https://github.com/lucas-clemente/quic-go/pull/3109))\n - update the packet numbers in decoding test to the ones from the draft ([lucas-clemente/quic-go#3137](https://github.com/lucas-clemente/quic-go/pull/3137))\n - various amplification limit fixes ([lucas-clemente/quic-go#3132](https://github.com/lucas-clemente/quic-go/pull/3132))\n - fix calculation of the handshake idle timeout ([lucas-clemente/quic-go#3120](https://github.com/lucas-clemente/quic-go/pull/3120))\n - only start PMTUD after handshake confirmation ([lucas-clemente/quic-go#3138](https://github.com/lucas-clemente/quic-go/pull/3138))\n - don't regard PMTU probe packets as outstanding ([lucas-clemente/quic-go#3126](https://github.com/lucas-clemente/quic-go/pull/3126))\n - expose the draft-34 version ([lucas-clemente/quic-go#3100](https://github.com/lucas-clemente/quic-go/pull/3100))\n - clean up the testutils ([lucas-clemente/quic-go#3104](https://github.com/lucas-clemente/quic-go/pull/3104))\n - initialize the congestion controller with the actual max datagram size ([lucas-clemente/quic-go#3107](https://github.com/lucas-clemente/quic-go/pull/3107))\n - make it possible to trace acknowledged packets ([lucas-clemente/quic-go#3134](https://github.com/lucas-clemente/quic-go/pull/3134))\n - avoid type confusion between protocol.PacketType and logging.PacketType ([lucas-clemente/quic-go#3108](https://github.com/lucas-clemente/quic-go/pull/3108))\n - fix duplicate logging of errors when the first error was a timeout error ([lucas-clemente/quic-go#3112](https://github.com/lucas-clemente/quic-go/pull/3112))\n - use a tracer to make the packetization test more useful ([lucas-clemente/quic-go#3136](https://github.com/lucas-clemente/quic-go/pull/3136))\n - improve string representation of timeout errors ([lucas-clemente/quic-go#3118](https://github.com/lucas-clemente/quic-go/pull/3118))\n - fix flaky timeout test ([lucas-clemente/quic-go#3105](https://github.com/lucas-clemente/quic-go/pull/3105))\n - fix calculation of the time for the next keep alive\n - add a 0-RTT test with different connection ID lengths ([lucas-clemente/quic-go#3098](https://github.com/lucas-clemente/quic-go/pull/3098))\n - only run Ginkgo focus detection in staged files in pre-commit hook ([lucas-clemente/quic-go#3099](https://github.com/lucas-clemente/quic-go/pull/3099))\n - allow 0-RTT when flow control windows are increased ([lucas-clemente/quic-go#3096](https://github.com/lucas-clemente/quic-go/pull/3096))\n - improve the 0-RTT rejection integration test ([lucas-clemente/quic-go#3097](https://github.com/lucas-clemente/quic-go/pull/3097))\n - rename config values for flow control limits ([lucas-clemente/quic-go#3089](https://github.com/lucas-clemente/quic-go/pull/3089))\n - allow 0-RTT resumption if the server's stream limit was increased ([lucas-clemente/quic-go#3086](https://github.com/lucas-clemente/quic-go/pull/3086))\n - cache the serialized OOB in the conn, not in the packet info ([lucas-clemente/quic-go#3093](https://github.com/lucas-clemente/quic-go/pull/3093))\n - use code points from x/sys/unix for PKTINFO syscalls ([lucas-clemente/quic-go#3094](https://github.com/lucas-clemente/quic-go/pull/3094))\n - make it possible to detect version negotiation failures in logging, fix qlogging of those ([lucas-clemente/quic-go#3092](https://github.com/lucas-clemente/quic-go/pull/3092))\n - make the initial stream / connection flow control windows configurable ([lucas-clemente/quic-go#3083](https://github.com/lucas-clemente/quic-go/pull/3083))\n - only apply server's transport parameters after handshake completion ([lucas-clemente/quic-go#3085](https://github.com/lucas-clemente/quic-go/pull/3085))\n - fix documentation for baseFlowController.UpdateSendWindow ([lucas-clemente/quic-go#3087](https://github.com/lucas-clemente/quic-go/pull/3087))\n - set the Content-Length for HTTP/3 responses ([lucas-clemente/quic-go#3091](https://github.com/lucas-clemente/quic-go/pull/3091))\n - update the flow control windows of streams opened in 0-RTT ([lucas-clemente/quic-go#3088](https://github.com/lucas-clemente/quic-go/pull/3088))\n - Use the correct source IP when binding multiple IPs ([lucas-clemente/quic-go#3067](https://github.com/lucas-clemente/quic-go/pull/3067))\n - fix race condition when receiving 0-RTT packets ([lucas-clemente/quic-go#3074](https://github.com/lucas-clemente/quic-go/pull/3074))\n - require the application to handle 0-RTT rejection ([lucas-clemente/quic-go#3066](https://github.com/lucas-clemente/quic-go/pull/3066))\n - add an internal queue to signal that a datagram frame has been dequeued ([lucas-clemente/quic-go#3081](https://github.com/lucas-clemente/quic-go/pull/3081))\n - increase the maximum size of DATAGRAM frames ([lucas-clemente/quic-go#2966](https://github.com/lucas-clemente/quic-go/pull/2966))\n - remove non-functioning 0-RTT test with different conn ID lengths ([lucas-clemente/quic-go#3079](https://github.com/lucas-clemente/quic-go/pull/3079))\n - remove stray struct equality check ([lucas-clemente/quic-go#3078](https://github.com/lucas-clemente/quic-go/pull/3078))\n - fix issuing of connection IDs when dialing a 0-RTT connections ([lucas-clemente/quic-go#3058](https://github.com/lucas-clemente/quic-go/pull/3058))\n - only accept 0-RTT it the active_connection_id_limit didn't change ([lucas-clemente/quic-go#3060](https://github.com/lucas-clemente/quic-go/pull/3060))\n - remove unused error return value from HandleMaxStreamsFrame ([lucas-clemente/quic-go#3072](https://github.com/lucas-clemente/quic-go/pull/3072))\n - fix flaky accept queue integration test ([lucas-clemente/quic-go#3068](https://github.com/lucas-clemente/quic-go/pull/3068))\n - don't reset the QPACK encoder / decoder streams ([lucas-clemente/quic-go#3063](https://github.com/lucas-clemente/quic-go/pull/3063))\n - remove incorrect logging for client side retry packet ([lucas-clemente/quic-go#3071](https://github.com/lucas-clemente/quic-go/pull/3071))\n - allow sending 1xx responses (#3047) ([lucas-clemente/quic-go#3047](https://github.com/lucas-clemente/quic-go/pull/3047))\n - fix retry key and nonce for draft-34 ([lucas-clemente/quic-go#3062](https://github.com/lucas-clemente/quic-go/pull/3062))\n - implement DPLPMTUD ([lucas-clemente/quic-go#3028](https://github.com/lucas-clemente/quic-go/pull/3028))\n - only read multiple packets at a time after handshake completion ([lucas-clemente/quic-go#3041](https://github.com/lucas-clemente/quic-go/pull/3041))\n - make the certificate verification integration tests more explicit ([lucas-clemente/quic-go#3040](https://github.com/lucas-clemente/quic-go/pull/3040))\n - update gomock to v1.5.0, use mockgen source mode ([lucas-clemente/quic-go#3049](https://github.com/lucas-clemente/quic-go/pull/3049))\n - trace dropping of 0-RTT keys ([lucas-clemente/quic-go#3054](https://github.com/lucas-clemente/quic-go/pull/3054))\n - improve timeout measurement in the timeout test ([lucas-clemente/quic-go#3042](https://github.com/lucas-clemente/quic-go/pull/3042))\n - add a randomized test for the received_packet_history ([lucas-clemente/quic-go#3052](https://github.com/lucas-clemente/quic-go/pull/3052))\n - fix documentation of default values for MaxReceive{Stream, Connection}FlowControlWindow ([lucas-clemente/quic-go#3055](https://github.com/lucas-clemente/quic-go/pull/3055))\n - refactor merge packet number ranges ([lucas-clemente/quic-go#3051](https://github.com/lucas-clemente/quic-go/pull/3051))\n - add draft-34 to support versions in README\n - update README to reflect dropped Go 1.14 support\n - remove redundant nil-check in the packet packer ([lucas-clemente/quic-go#3048](https://github.com/lucas-clemente/quic-go/pull/3048))\n - avoid using rand.Source ([lucas-clemente/quic-go#3046](https://github.com/lucas-clemente/quic-go/pull/3046))\n - update Go to 1.16, drop support for 1.14 ([lucas-clemente/quic-go#3045](https://github.com/lucas-clemente/quic-go/pull/3045))\n - fix error message when the UDP receive buffer size can't be increased ([lucas-clemente/quic-go#3039](https://github.com/lucas-clemente/quic-go/pull/3039))\n - add the time_format field to qlog common_fields ([lucas-clemente/quic-go#3038](https://github.com/lucas-clemente/quic-go/pull/3038))\n - log connection IDs without the 0x prefix ([lucas-clemente/quic-go#3036](https://github.com/lucas-clemente/quic-go/pull/3036))\n - add support for QUIC draft-34 ([lucas-clemente/quic-go#3031](https://github.com/lucas-clemente/quic-go/pull/3031))\n - fix qtls imports in mockgen generated mocks ([lucas-clemente/quic-go#3037](https://github.com/lucas-clemente/quic-go/pull/3037))\n - improve error message when the read buffer size can't be set ([lucas-clemente/quic-go#3030](https://github.com/lucas-clemente/quic-go/pull/3030))\n - qlog the quic-go version ([lucas-clemente/quic-go#3033](https://github.com/lucas-clemente/quic-go/pull/3033))\n - remove the metrics package ([lucas-clemente/quic-go#3032](https://github.com/lucas-clemente/quic-go/pull/3032))\n - expose the constructor for the qlog connection tracer ([lucas-clemente/quic-go#3034](https://github.com/lucas-clemente/quic-go/pull/3034))\n - expose the constructor for the multipexed connection tracer ([lucas-clemente/quic-go#3035](https://github.com/lucas-clemente/quic-go/pull/3035))\n - make sure the server is stopped before closing all server sessions ([lucas-clemente/quic-go#3020](https://github.com/lucas-clemente/quic-go/pull/3020))\n - increase the size of the send queue ([lucas-clemente/quic-go#3016](https://github.com/lucas-clemente/quic-go/pull/3016))\n - prioritize receiving packets over sending out more packets ([lucas-clemente/quic-go#3015](https://github.com/lucas-clemente/quic-go/pull/3015))\n - re-enable key updates for HTTP/3 ([lucas-clemente/quic-go#3017](https://github.com/lucas-clemente/quic-go/pull/3017))\n - check for errors after handling each previously undecryptable packet ([lucas-clemente/quic-go#3011](https://github.com/lucas-clemente/quic-go/pull/3011))\n - fix flaky streams map test on Windows ([lucas-clemente/quic-go#3013](https://github.com/lucas-clemente/quic-go/pull/3013))\n - fix flaky stream cancellation integration test ([lucas-clemente/quic-go#3014](https://github.com/lucas-clemente/quic-go/pull/3014))\n - preallocate a slice of one frame when packing a packet ([lucas-clemente/quic-go#3018](https://github.com/lucas-clemente/quic-go/pull/3018))\n - allow sending of ACKs when pacing limited ([lucas-clemente/quic-go#3010](https://github.com/lucas-clemente/quic-go/pull/3010))\n - fix qlogging of the packet payload length ([lucas-clemente/quic-go#3004](https://github.com/lucas-clemente/quic-go/pull/3004))\n - corrupt more ACKs in the MITM test ([lucas-clemente/quic-go#3007](https://github.com/lucas-clemente/quic-go/pull/3007))\n - fix flaky key update integration test ([lucas-clemente/quic-go#3005](https://github.com/lucas-clemente/quic-go/pull/3005))\n - immediately complete streams that were canceled, drop retransmissions ([lucas-clemente/quic-go#3003](https://github.com/lucas-clemente/quic-go/pull/3003))\n - stop generating new packets when the send queue is full ([lucas-clemente/quic-go#2971](https://github.com/lucas-clemente/quic-go/pull/2971))\n - allow access to the underlying quic.Stream from a http.ResponseWriter ([lucas-clemente/quic-go#2993](https://github.com/lucas-clemente/quic-go/pull/2993))\n - remove stay print statement from session test\n - allow receiving of multiple packets before sending a packet ([lucas-clemente/quic-go#2984](https://github.com/lucas-clemente/quic-go/pull/2984))\n - use cryptographic random for determining skipped packet numbers ([lucas-clemente/quic-go#2940](https://github.com/lucas-clemente/quic-go/pull/2940))\n - fix interpretation of time.Time{} as a pacing deadline ([lucas-clemente/quic-go#2980](https://github.com/lucas-clemente/quic-go/pull/2980))\n - qlog restored transport parameters ([lucas-clemente/quic-go#2991](https://github.com/lucas-clemente/quic-go/pull/2991))\n - use a pkg.go.dev instead of a GoDoc badge ([lucas-clemente/quic-go#2982](https://github.com/lucas-clemente/quic-go/pull/2982))\n - introduce a separate queue for undecryptable packets ([lucas-clemente/quic-go#2988](https://github.com/lucas-clemente/quic-go/pull/2988))\n - improve 0-RTT queue ([lucas-clemente/quic-go#2990](https://github.com/lucas-clemente/quic-go/pull/2990))\n - simplify switch statement in the transport parameter parser ([lucas-clemente/quic-go#2995](https://github.com/lucas-clemente/quic-go/pull/2995))\n - remove unneeded overflow check when parsing the max_ack_delay ([lucas-clemente/quic-go#2996](https://github.com/lucas-clemente/quic-go/pull/2996))\n - remove unneeded check in receivedPacketHandler.IsPotentiallyDuplicate ([lucas-clemente/quic-go#2998](https://github.com/lucas-clemente/quic-go/pull/2998))\n - qlog the max_datagram_frame_size transport parameter ([lucas-clemente/quic-go#2997](https://github.com/lucas-clemente/quic-go/pull/2997))\n - qlog draft-02 fixes ([lucas-clemente/quic-go#2987](https://github.com/lucas-clemente/quic-go/pull/2987))\n - fix flaky qlog test ([lucas-clemente/quic-go#2981](https://github.com/lucas-clemente/quic-go/pull/2981))\n - only run gofumpt on .go files in pre-commit hook ([lucas-clemente/quic-go#2983](https://github.com/lucas-clemente/quic-go/pull/2983))\n - fix outdated comment for the http3.Server\n - make the OpenStreamSync cancellation test less flaky ([lucas-clemente/quic-go#2978](https://github.com/lucas-clemente/quic-go/pull/2978))\n - add some useful pre-commit hooks ([lucas-clemente/quic-go#2979](https://github.com/lucas-clemente/quic-go/pull/2979))\n - publicize QUIC varint reading and writing ([lucas-clemente/quic-go#2973](https://github.com/lucas-clemente/quic-go/pull/2973))\n - add a http3.RoundTripOpt to skip the request scheme check ([lucas-clemente/quic-go#2962](https://github.com/lucas-clemente/quic-go/pull/2962))\n - use the standard quic.Config in the deadline tests ([lucas-clemente/quic-go#2970](https://github.com/lucas-clemente/quic-go/pull/2970))\n - update golangci-lint to v1.34.1 ([lucas-clemente/quic-go#2964](https://github.com/lucas-clemente/quic-go/pull/2964))\n - update text about QUIC versions in the README ([lucas-clemente/quic-go#2975](https://github.com/lucas-clemente/quic-go/pull/2975))\n - remove stray TODO in the http3.Server\n - add support for Go 1.16 ([lucas-clemente/quic-go#2953](https://github.com/lucas-clemente/quic-go/pull/2953))\n - cancel reading on unidirectional streams when the stream type is unknown ([lucas-clemente/quic-go#2952](https://github.com/lucas-clemente/quic-go/pull/2952))\n - remove duplicate check of the URL scheme in the HTTP/3 client ([lucas-clemente/quic-go#2956](https://github.com/lucas-clemente/quic-go/pull/2956))\n - increase queueing duration in 0-RTT queue test to reduce flakiness ([lucas-clemente/quic-go#2954](https://github.com/lucas-clemente/quic-go/pull/2954))\n - implement the HTTP/3 Datagram negotiation ([lucas-clemente/quic-go#2951](https://github.com/lucas-clemente/quic-go/pull/2951))\n - implement HTTP/3 control stream handling ([lucas-clemente/quic-go#2949](https://github.com/lucas-clemente/quic-go/pull/2949))\n - fix flaky sentPacketHandler test ([lucas-clemente/quic-go#2950](https://github.com/lucas-clemente/quic-go/pull/2950))\n - don't retransmit PING frames added to ACK-only packets ([lucas-clemente/quic-go#2942](https://github.com/lucas-clemente/quic-go/pull/2942))\n - move the transport parameter stream limit check to the parser ([lucas-clemente/quic-go#2944](https://github.com/lucas-clemente/quic-go/pull/2944))\n - remove unused initialVersion variable in session ([lucas-clemente/quic-go#2946](https://github.com/lucas-clemente/quic-go/pull/2946))\n - remove unneeded check for the peer's transport parameters ([lucas-clemente/quic-go#2945](https://github.com/lucas-clemente/quic-go/pull/2945))\n - add the H3_MESSAGE_ERROR ([lucas-clemente/quic-go#2947](https://github.com/lucas-clemente/quic-go/pull/2947))\n - simplify Read and Write mock calls in http3 tests ([lucas-clemente/quic-go#2948](https://github.com/lucas-clemente/quic-go/pull/2948))\n - implement the datagram draft ([lucas-clemente/quic-go#2162](https://github.com/lucas-clemente/quic-go/pull/2162))\n - fix logging of bytes_in_flight when receiving an ACK ([lucas-clemente/quic-go#2937](https://github.com/lucas-clemente/quic-go/pull/2937))\n - trace when a packet is dropped because the receivedPackets chan is full ([lucas-clemente/quic-go#2939](https://github.com/lucas-clemente/quic-go/pull/2939))\n - various improvements to the packet number generator ([lucas-clemente/quic-go#2905](https://github.com/lucas-clemente/quic-go/pull/2905))\n - introduce a quic.Config.HandshakeIdleTimeout, remove HandshakeTimeout ([lucas-clemente/quic-go#2930](https://github.com/lucas-clemente/quic-go/pull/2930))\n - allow up to 20 byte for the initial connection IDs ([lucas-clemente/quic-go#2936](https://github.com/lucas-clemente/quic-go/pull/2936))\n - reduce memory footprint of undecryptable packet handling ([lucas-clemente/quic-go#2932](https://github.com/lucas-clemente/quic-go/pull/2932))\n - use a buffer from the pool for composing Retry packets ([lucas-clemente/quic-go#2934](https://github.com/lucas-clemente/quic-go/pull/2934))\n - release the packet buffer after sending a CONNECTION_CLOSE in the server ([lucas-clemente/quic-go#2935](https://github.com/lucas-clemente/quic-go/pull/2935))\n - move integration tests to GitHub Actions, disable Travis ([lucas-clemente/quic-go#2891](https://github.com/lucas-clemente/quic-go/pull/2891))\n - use golang.org/x/sys/unix instead of syscall ([lucas-clemente/quic-go#2927](https://github.com/lucas-clemente/quic-go/pull/2927))\n - add support for the connection_closed qlog event ([lucas-clemente/quic-go#2921](https://github.com/lucas-clemente/quic-go/pull/2921))\n - qlog tokens in NEW_TOKEN frames, Retry packets and Initial packets ([lucas-clemente/quic-go#2863](https://github.com/lucas-clemente/quic-go/pull/2863))\n - qlog the packet_type as part of the packet header, not the event itself ([lucas-clemente/quic-go#2758](https://github.com/lucas-clemente/quic-go/pull/2758))\n - use the new, streaming-friendly NDJSON-based qlog encoding ([lucas-clemente/quic-go#2736](https://github.com/lucas-clemente/quic-go/pull/2736))\n - add a generic Debug() function to the connection tracer ([lucas-clemente/quic-go#2909](https://github.com/lucas-clemente/quic-go/pull/2909))\n - remove unnecessary call to time.Now() when sending a packet ([lucas-clemente/quic-go#2911](https://github.com/lucas-clemente/quic-go/pull/2911))\n - remove support for quic-trace ([lucas-clemente/quic-go#2913](https://github.com/lucas-clemente/quic-go/pull/2913))\n - reduce the maximum number of ACK ranges ([lucas-clemente/quic-go#2887](https://github.com/lucas-clemente/quic-go/pull/2887))\n - don't allocate for acked packets ([lucas-clemente/quic-go#2899](https://github.com/lucas-clemente/quic-go/pull/2899))\n - avoid allocating when detecting lost packets ([lucas-clemente/quic-go#2898](https://github.com/lucas-clemente/quic-go/pull/2898))\n - use the string optimization for map keys in the packet handler map ([lucas-clemente/quic-go#2892](https://github.com/lucas-clemente/quic-go/pull/2892))\n - use a single map in the incoming streams map ([lucas-clemente/quic-go#2890](https://github.com/lucas-clemente/quic-go/pull/2890))\n- github.com/marten-seemann/qtls-go1-15 (v0.1.1 -> v0.1.4):\n - use a prefix for client session cache keys\n - add callbacks to store and restore app data along a session state\n - don't use TLS 1.3 compatibility mode when using alternative record layer\n - delete the session ticket after attempting 0-RTT\n - reject 0-RTT when a different ALPN is chosen\n - encode the ALPN into the session ticket\n - add a field to the ConnectionState to tell if 0-RTT was used\n - add a callback to tell the client about rejection of 0-RTT\n - don't offer 0-RTT after a HelloRetryRequest\n - add Accept0RTT to Config callback to decide if 0-RTT should be accepted\n- github.com/marten-seemann/qtls-go1-16 (null -> v0.1.3):\n - use a prefix for client session cache keys\n - add callbacks to store and restore app data along a session state\n - don't use TLS 1.3 compatibility mode when using alternative record layer\n - delete the session ticket after attempting 0-RTT\n - reject 0-RTT when a different ALPN is chosen\n- github.com/multiformats/go-multiaddr (v0.3.1 -> v0.3.2):\n - fix(net): export new net.Addr conversion registration functions ([multiformats/go-multiaddr#152](https://github.com/multiformats/go-multiaddr/pull/152))\n - sync: run go mod tidy (and set Go 1.15) and gofmt -s in copy workflow (#146) ([multiformats/go-multiaddr#146](https://github.com/multiformats/go-multiaddr/pull/146))\n - more linter fixes ([multiformats/go-multiaddr#145](https://github.com/multiformats/go-multiaddr/pull/145))\n - fix go vet and staticcheck failures ([multiformats/go-multiaddr#143](https://github.com/multiformats/go-multiaddr/pull/143))\n - don't listen on all interfaces in tests, unless on CI ([multiformats/go-multiaddr#136](https://github.com/multiformats/go-multiaddr/pull/136))\n - Fix Local Address on TCP connections ([multiformats/go-multiaddr#135](https://github.com/multiformats/go-multiaddr/pull/135))\n- github.com/multiformats/go-multiaddr-dns (v0.2.0 -> v0.3.1):\n - Normalize domains to fqdn for resolver selection ([multiformats/go-multiaddr-dns#27](https://github.com/multiformats/go-multiaddr-dns/pull/27))\n - refactor Resolver to support custom per-TLD resolvers ([multiformats/go-multiaddr-dns#26](https://github.com/multiformats/go-multiaddr-dns/pull/26))\n - feat: exposes backend ([multiformats/go-multiaddr-dns#25](https://github.com/multiformats/go-multiaddr-dns/pull/25))\n- github.com/multiformats/go-multihash (v0.0.14 -> v0.0.15):\n - Refactor registry system: no direct dependencies; expose standard hash.Hash; be a data carrier. ([multiformats/go-multihash#136](https://github.com/multiformats/go-multihash/pull/136))\n- github.com/multiformats/go-multistream (v0.2.0 -> v0.2.2):\n - change the simultaneous open protocol to /libp2p/simultaneous-connect ([multiformats/go-multistream#66](https://github.com/multiformats/go-multistream/pull/66))\n - fix the lazy stress read test on Windows ([multiformats/go-multistream#61](https://github.com/multiformats/go-multistream/pull/61))\n - fix go vet and staticcheck errors ([multiformats/go-multistream#60](https://github.com/multiformats/go-multistream/pull/60))\n - Implement simultaneous open extension ([multiformats/go-multistream#42](https://github.com/multiformats/go-multistream/pull/42))\n - reduce the number of streams in the stress tests, fix error handling ([multiformats/go-multistream#54](https://github.com/multiformats/go-multistream/pull/54))\n- github.com/whyrusleeping/cbor-gen (v0.0.0-20200710004633-5379fc63235d -> v0.0.0-20210219115102-f37d292932f2):\n - feat: allow unmarshalling of struct with more fields than marshaled struct ([whyrusleeping/cbor-gen#50](https://github.com/whyrusleeping/cbor-gen/pull/50))\n - chore: add a license file ([whyrusleeping/cbor-gen#49](https://github.com/whyrusleeping/cbor-gen/pull/49))\n - fix: enforce maxlen in ReadByteArray() ([whyrusleeping/cbor-gen#43](https://github.com/whyrusleeping/cbor-gen/pull/43))\n - use unix nanoseconds for encoding Cbortime ([whyrusleeping/cbor-gen#41](https://github.com/whyrusleeping/cbor-gen/pull/41))\n - add json marshalers to CborTime\n - add a helper for roundtripping time.time objects ([whyrusleeping/cbor-gen#40](https://github.com/whyrusleeping/cbor-gen/pull/40))\n - Add a validate function. ([whyrusleeping/cbor-gen#39](https://github.com/whyrusleeping/cbor-gen/pull/39))\n - Fix import handling ([whyrusleeping/cbor-gen#38](https://github.com/whyrusleeping/cbor-gen/pull/38))\n - Optimize discarding in ScanForLinks ([whyrusleeping/cbor-gen#36](https://github.com/whyrusleeping/cbor-gen/pull/36))\n - Always allocate scratch space when marshalling into a map. ([whyrusleeping/cbor-gen#37](https://github.com/whyrusleeping/cbor-gen/pull/37))\n - optimize byte reading ([whyrusleeping/cbor-gen#35](https://github.com/whyrusleeping/cbor-gen/pull/35))\n - Optimize decoding ([whyrusleeping/cbor-gen#34](https://github.com/whyrusleeping/cbor-gen/pull/34))\n - Fix named string issue ([whyrusleeping/cbor-gen#30](https://github.com/whyrusleeping/cbor-gen/pull/30))\n - Fix encoding/decoding fixed byte arrays ([whyrusleeping/cbor-gen#29](https://github.com/whyrusleeping/cbor-gen/pull/29))\n - fix overread on scanforlinks ([whyrusleeping/cbor-gen#28](https://github.com/whyrusleeping/cbor-gen/pull/28))\n\n### ❤️ Contributors\n\n| Contributor | Commits | Lines ± | Files Changed |\n|-------------|---------|---------|---------------|\n| Marten Seemann | 358 | +17444/-12000 | 1268 |\n| Eric Myhre | 82 | +9672/-2459 | 328 |\n| Ian Davis | 7 | +8421/-737 | 116 |\n| Daniel Martí | 18 | +2733/-4377 | 313 |\n| Adin Schmahmann | 46 | +5387/-1289 | 125 |\n| Steven Allen | 95 | +3278/-1861 | 200 |\n| hannahhoward | 14 | +1380/-3667 | 84 |\n| gammazero | 29 | +2520/-1161 | 88 |\n| Hector Sanjuan | 12 | +511/-3129 | 52 |\n| vyzo | 77 | +2198/-940 | 117 |\n| Will Scott | 12 | +912/-593 | 37 |\n| Dirk McCormick | 3 | +1384/-63 | 14 |\n| Andrew Gillis | 3 | +1231/-39 | 19 |\n| Marcin Rataj | 37 | +549/-308 | 72 |\n| Aarsh Shah | 13 | +668/-86 | 30 |\n| Olivier Poitrey | 1 | +469/-182 | 15 |\n| Rod Vagg | 9 | +364/-184 | 14 |\n| whyrusleeping | 5 | +253/-32 | 11 |\n| Cory Schwartz | 10 | +162/-115 | 37 |\n| Adrian Lanzafame | 8 | +212/-60 | 11 |\n| aarshkshah1992 | 7 | +102/-110 | 9 |\n| Jakub Sztandera | 7 | +126/-75 | 16 |\n| huoju | 4 | +127/-41 | 6 |\n| acruikshank | 6 | +32/-24 | 7 |\n| Toby | 1 | +41/-1 | 2 |\n| Naveen | 1 | +40/-0 | 1 |\n| Bogdan Stirbat | 1 | +22/-16 | 2 |\n| Kévin Dunglas | 1 | +32/-2 | 2 |\n| Nicholas Bollweg | 1 | +22/-0 | 1 |\n| q191201771 | 2 | +4/-11 | 2 |\n| Mathis Engelbart | 1 | +12/-2 | 1 |\n| requilence | 1 | +13/-0 | 1 |\n| divingpetrel | 1 | +7/-4 | 2 |\n| Oli Evans | 2 | +9/-2 | 3 |\n| Lucas Molas | 3 | +7/-3 | 3 |\n| RubenKelevra | 3 | +2/-6 | 3 |\n| Will | 1 | +1/-5 | 1 |\n| Jorropo | 1 | +4/-2 | 1 |\n| Ju Huo | 1 | +2/-2 | 1 |\n| zhoujiajie | 1 | +1/-1 | 1 |\n| Luflosi | 1 | +1/-1 | 1 |\n| Jonathan Rudenberg | 1 | +1/-1 | 1 |\n| David Pflug | 1 | +1/-1 | 1 |\n| Ari Mattila | 1 | +1/-1 | 1 |\n| Yingrong Zhao | 1 | +0/-1 | 1 |\n" }, { "path": "docs/command-completion.md", "content": "# Command Completion\n\nShell command completions can be generated by running one of the `ipfs commands completions`\nsub-commands.\n\nThe simplest way to \"eval\" the completions logic:\n\n```bash\n> eval \"$(ipfs commands completion bash)\"\n```\n\nTo install the completions permanently, they can be moved to\n`/etc/bash_completion.d` or sourced from your `~/.bashrc` file.\n\n## Fish\n\nThe fish shell is also supported:\n\nThe simplest way to use the completions logic:\n\n```bash\n> ipfs commands completion fish | source\n```\n\nTo install the completions permanently, they can be moved to\n`/etc/fish/completions` or `~/.config/fish/completions` or sourced from your `~/.config/fish/config.fish` file.\n\n## ZSH\n\nThe zsh shell is also supported:\n\nThe simplest way to \"eval\" the completions logic:\n\n```bash\n> eval \"$(ipfs commands completion zsh)\"\n```\n\nTo install the completions permanently, they can be moved to\n`/etc/bash_completion.d` or sourced from your `~/.zshrc` file.\n" }, { "path": "docs/config.md", "content": "# The Kubo config file\n\nThe Kubo config file is a JSON document located at `$IPFS_PATH/config`. It\nis read once at node instantiation, either for an offline command, or when\nstarting the daemon. Commands that execute on a running daemon do not read the\nconfig file at runtime.\n\n# Table of Contents\n\n- [The Kubo config file](#the-kubo-config-file)\n- [Table of Contents](#table-of-contents)\n - [`Addresses`](#addresses)\n - [`Addresses.API`](#addressesapi)\n - [`Addresses.Gateway`](#addressesgateway)\n - [`Addresses.Swarm`](#addressesswarm)\n - [`Addresses.Announce`](#addressesannounce)\n - [`Addresses.AppendAnnounce`](#addressesappendannounce)\n - [`Addresses.NoAnnounce`](#addressesnoannounce)\n - [`API`](#api)\n - [`API.HTTPHeaders`](#apihttpheaders)\n - [`API.Authorizations`](#apiauthorizations)\n - [`API.Authorizations: AuthSecret`](#apiauthorizations-authsecret)\n - [`API.Authorizations: AllowedPaths`](#apiauthorizations-allowedpaths)\n - [`AutoNAT`](#autonat)\n - [`AutoNAT.ServiceMode`](#autonatservicemode)\n - [`AutoNAT.Throttle`](#autonatthrottle)\n - [`AutoNAT.Throttle.GlobalLimit`](#autonatthrottlegloballimit)\n - [`AutoNAT.Throttle.PeerLimit`](#autonatthrottlepeerlimit)\n - [`AutoNAT.Throttle.Interval`](#autonatthrottleinterval)\n - [`AutoTLS`](#autotls)\n - [`AutoTLS.Enabled`](#autotlsenabled)\n - [`AutoTLS.AutoWSS`](#autotlsautowss)\n - [`AutoTLS.ShortAddrs`](#autotlsshortaddrs)\n - [`AutoTLS.DomainSuffix`](#autotlsdomainsuffix)\n - [`AutoTLS.RegistrationEndpoint`](#autotlsregistrationendpoint)\n - [`AutoTLS.RegistrationToken`](#autotlsregistrationtoken)\n - [`AutoTLS.RegistrationDelay`](#autotlsregistrationdelay)\n - [`AutoTLS.CAEndpoint`](#autotlscaendpoint)\n - [`AutoConf`](#autoconf)\n - [`AutoConf.URL`](#autoconfurl)\n - [`AutoConf.Enabled`](#autoconfenabled)\n - [`AutoConf.RefreshInterval`](#autoconfrefreshinterval)\n - [`AutoConf.TLSInsecureSkipVerify`](#autoconftlsinsecureskipverify)\n - [`Bitswap`](#bitswap)\n - [`Bitswap.Libp2pEnabled`](#bitswaplibp2penabled)\n - [`Bitswap.ServerEnabled`](#bitswapserverenabled)\n - [`Bootstrap`](#bootstrap)\n - [`Datastore`](#datastore)\n - [`Datastore.StorageMax`](#datastorestoragemax)\n - [`Datastore.StorageGCWatermark`](#datastorestoragegcwatermark)\n - [`Datastore.GCPeriod`](#datastoregcperiod)\n - [`Datastore.HashOnRead`](#datastorehashonread)\n - [`Datastore.BloomFilterSize`](#datastorebloomfiltersize)\n - [`Datastore.WriteThrough`](#datastorewritethrough)\n - [`Datastore.BlockKeyCacheSize`](#datastoreblockkeycachesize)\n - [`Datastore.Spec`](#datastorespec)\n - [`Discovery`](#discovery)\n - [`Discovery.MDNS`](#discoverymdns)\n - [`Discovery.MDNS.Enabled`](#discoverymdnsenabled)\n - [`Discovery.MDNS.Interval`](#discoverymdnsinterval)\n - [`Experimental`](#experimental)\n - [`Experimental.Libp2pStreamMounting`](#experimentallibp2pstreammounting)\n - [`Gateway`](#gateway)\n - [`Gateway.NoFetch`](#gatewaynofetch)\n - [`Gateway.NoDNSLink`](#gatewaynodnslink)\n - [`Gateway.DeserializedResponses`](#gatewaydeserializedresponses)\n - [`Gateway.AllowCodecConversion`](#gatewayallowcodecconversion)\n - [`Gateway.DisableHTMLErrors`](#gatewaydisablehtmlerrors)\n - [`Gateway.ExposeRoutingAPI`](#gatewayexposeroutingapi)\n - [`Gateway.RetrievalTimeout`](#gatewayretrievaltimeout)\n - [`Gateway.MaxRequestDuration`](#gatewaymaxrequestduration)\n - [`Gateway.MaxRangeRequestFileSize`](#gatewaymaxrangerequestfilesize)\n - [`Gateway.MaxConcurrentRequests`](#gatewaymaxconcurrentrequests)\n - [`Gateway.HTTPHeaders`](#gatewayhttpheaders)\n - [`Gateway.RootRedirect`](#gatewayrootredirect)\n - [`Gateway.DiagnosticServiceURL`](#gatewaydiagnosticserviceurl)\n - [`Gateway.FastDirIndexThreshold`](#gatewayfastdirindexthreshold)\n - [`Gateway.Writable`](#gatewaywritable)\n - [`Gateway.PathPrefixes`](#gatewaypathprefixes)\n - [`Gateway.PublicGateways`](#gatewaypublicgateways)\n - [`Gateway.PublicGateways: Paths`](#gatewaypublicgateways-paths)\n - [`Gateway.PublicGateways: UseSubdomains`](#gatewaypublicgateways-usesubdomains)\n - [`Gateway.PublicGateways: NoDNSLink`](#gatewaypublicgateways-nodnslink)\n - [`Gateway.PublicGateways: InlineDNSLink`](#gatewaypublicgateways-inlinednslink)\n - [`Gateway.PublicGateways: DeserializedResponses`](#gatewaypublicgateways-deserializedresponses)\n - [Implicit defaults of `Gateway.PublicGateways`](#implicit-defaults-of-gatewaypublicgateways)\n - [`Gateway` recipes](#gateway-recipes)\n - [`Identity`](#identity)\n - [`Identity.PeerID`](#identitypeerid)\n - [`Identity.PrivKey`](#identityprivkey)\n - [`Internal`](#internal)\n - [`Internal.Bitswap`](#internalbitswap)\n - [`Internal.Bitswap.TaskWorkerCount`](#internalbitswaptaskworkercount)\n - [`Internal.Bitswap.EngineBlockstoreWorkerCount`](#internalbitswapengineblockstoreworkercount)\n - [`Internal.Bitswap.EngineTaskWorkerCount`](#internalbitswapenginetaskworkercount)\n - [`Internal.Bitswap.MaxOutstandingBytesPerPeer`](#internalbitswapmaxoutstandingbytesperpeer)\n - [`Internal.Bitswap.ProviderSearchDelay`](#internalbitswapprovidersearchdelay)\n - [`Internal.Bitswap.ProviderSearchMaxResults`](#internalbitswapprovidersearchmaxresults)\n - [`Internal.Bitswap.BroadcastControl`](#internalbitswapbroadcastcontrol)\n - [`Internal.Bitswap.BroadcastControl.Enable`](#internalbitswapbroadcastcontrolenable)\n - [`Internal.Bitswap.BroadcastControl.MaxPeers`](#internalbitswapbroadcastcontrolmaxpeers)\n - [`Internal.Bitswap.BroadcastControl.LocalPeers`](#internalbitswapbroadcastcontrollocalpeers)\n - [`Internal.Bitswap.BroadcastControl.PeeredPeers`](#internalbitswapbroadcastcontrolpeeredpeers)\n - [`Internal.Bitswap.BroadcastControl.MaxRandomPeers`](#internalbitswapbroadcastcontrolmaxrandompeers)\n - [`Internal.Bitswap.BroadcastControl.SendToPendingPeers`](#internalbitswapbroadcastcontrolsendtopendingpeers)\n - [`Internal.UnixFSShardingSizeThreshold`](#internalunixfsshardingsizethreshold)\n - [`Ipns`](#ipns)\n - [`Ipns.RepublishPeriod`](#ipnsrepublishperiod)\n - [`Ipns.RecordLifetime`](#ipnsrecordlifetime)\n - [`Ipns.ResolveCacheSize`](#ipnsresolvecachesize)\n - [`Ipns.MaxCacheTTL`](#ipnsmaxcachettl)\n - [`Ipns.UsePubsub`](#ipnsusepubsub)\n - [`Ipns.DelegatedPublishers`](#ipnsdelegatedpublishers)\n - [`Migration`](#migration)\n - [`Migration.DownloadSources`](#migrationdownloadsources)\n - [`Migration.Keep`](#migrationkeep)\n - [`Mounts`](#mounts)\n - [`Mounts.IPFS`](#mountsipfs)\n - [`Mounts.IPNS`](#mountsipns)\n - [`Mounts.MFS`](#mountsmfs)\n - [`Mounts.FuseAllowOther`](#mountsfuseallowother)\n - [`Pinning`](#pinning)\n - [`Pinning.RemoteServices`](#pinningremoteservices)\n - [`Pinning.RemoteServices: API`](#pinningremoteservices-api)\n - [`Pinning.RemoteServices: API.Endpoint`](#pinningremoteservices-apiendpoint)\n - [`Pinning.RemoteServices: API.Key`](#pinningremoteservices-apikey)\n - [`Pinning.RemoteServices: Policies`](#pinningremoteservices-policies)\n - [`Pinning.RemoteServices: Policies.MFS`](#pinningremoteservices-policiesmfs)\n - [`Pinning.RemoteServices: Policies.MFS.Enabled`](#pinningremoteservices-policiesmfsenabled)\n - [`Pinning.RemoteServices: Policies.MFS.PinName`](#pinningremoteservices-policiesmfspinname)\n - [`Pinning.RemoteServices: Policies.MFS.RepinInterval`](#pinningremoteservices-policiesmfsrepininterval)\n - [`Provide`](#provide)\n - [`Provide.Enabled`](#provideenabled)\n - [`Provide.Strategy`](#providestrategy)\n - [`Provide.DHT`](#providedht)\n - [`Provide.DHT.MaxWorkers`](#providedhtmaxworkers)\n - [`Provide.DHT.Interval`](#providedhtinterval)\n - [`Provide.DHT.SweepEnabled`](#providedhtsweepenabled)\n - [`Provide.DHT.ResumeEnabled`](#providedhtresumeenabled)\n - [`Provide.DHT.DedicatedPeriodicWorkers`](#providedhtdedicatedperiodicworkers)\n - [`Provide.DHT.DedicatedBurstWorkers`](#providedhtdedicatedburstworkers)\n - [`Provide.DHT.MaxProvideConnsPerWorker`](#providedhtmaxprovideconnsperworker)\n - [`Provide.DHT.KeystoreBatchSize`](#providedhtkeystorebatchsize)\n - [`Provide.DHT.OfflineDelay`](#providedhtofflinedelay)\n - [`Provider`](#provider)\n - [`Provider.Enabled`](#providerenabled)\n - [`Provider.Strategy`](#providerstrategy)\n - [`Provider.WorkerCount`](#providerworkercount)\n - [`Pubsub`](#pubsub)\n - [When to use a dedicated pubsub node](#when-to-use-a-dedicated-pubsub-node)\n - [Message deduplication](#message-deduplication)\n - [`Pubsub.Enabled`](#pubsubenabled)\n - [`Pubsub.Router`](#pubsubrouter)\n - [`Pubsub.DisableSigning`](#pubsubdisablesigning)\n - [`Pubsub.SeenMessagesTTL`](#pubsubseenmessagesttl)\n - [`Pubsub.SeenMessagesStrategy`](#pubsubseenmessagesstrategy)\n - [`Peering`](#peering)\n - [`Peering.Peers`](#peeringpeers)\n - [`Reprovider`](#reprovider)\n - [`Reprovider.Interval`](#reproviderinterval)\n - [`Reprovider.Strategy`](#providestrategy)\n - [`Routing`](#routing)\n - [`Routing.Type`](#routingtype)\n - [`Routing.DelegatedRouters`](#routingdelegatedrouters)\n - [`Routing.AcceleratedDHTClient`](#routingaccelerateddhtclient)\n - [`Routing.LoopbackAddressesOnLanDHT`](#routingloopbackaddressesonlandht)\n - [`Routing.IgnoreProviders`](#routingignoreproviders)\n - [`Routing.Routers`](#routingrouters)\n - [`Routing.Routers.[name].Type`](#routingroutersnametype)\n - [`Routing.Routers.[name].Parameters`](#routingroutersnameparameters)\n - [`Routing.Methods`](#routingmethods)\n - [`Swarm`](#swarm)\n - [`Swarm.AddrFilters`](#swarmaddrfilters)\n - [`Swarm.DisableBandwidthMetrics`](#swarmdisablebandwidthmetrics)\n - [`Swarm.DisableNatPortMap`](#swarmdisablenatportmap)\n - [`Swarm.EnableHolePunching`](#swarmenableholepunching)\n - [`Swarm.EnableAutoRelay`](#swarmenableautorelay)\n - [`Swarm.RelayClient`](#swarmrelayclient)\n - [`Swarm.RelayClient.Enabled`](#swarmrelayclientenabled)\n - [`Swarm.RelayClient.StaticRelays`](#swarmrelayclientstaticrelays)\n - [`Swarm.RelayService`](#swarmrelayservice)\n - [`Swarm.RelayService.Enabled`](#swarmrelayserviceenabled)\n - [`Swarm.RelayService.Limit`](#swarmrelayservicelimit)\n - [`Swarm.RelayService.ConnectionDurationLimit`](#swarmrelayserviceconnectiondurationlimit)\n - [`Swarm.RelayService.ConnectionDataLimit`](#swarmrelayserviceconnectiondatalimit)\n - [`Swarm.RelayService.ReservationTTL`](#swarmrelayservicereservationttl)\n - [`Swarm.RelayService.MaxReservations`](#swarmrelayservicemaxreservations)\n - [`Swarm.RelayService.MaxCircuits`](#swarmrelayservicemaxcircuits)\n - [`Swarm.RelayService.BufferSize`](#swarmrelayservicebuffersize)\n - [`Swarm.RelayService.MaxReservationsPerPeer`](#swarmrelayservicemaxreservationsperpeer)\n - [`Swarm.RelayService.MaxReservationsPerIP`](#swarmrelayservicemaxreservationsperip)\n - [`Swarm.RelayService.MaxReservationsPerASN`](#swarmrelayservicemaxreservationsperasn)\n - [`Swarm.EnableRelayHop`](#swarmenablerelayhop)\n - [`Swarm.DisableRelay`](#swarmdisablerelay)\n - [`Swarm.EnableAutoNATService`](#swarmenableautonatservice)\n - [`Swarm.ConnMgr`](#swarmconnmgr)\n - [`Swarm.ConnMgr.Type`](#swarmconnmgrtype)\n - [Basic Connection Manager](#basic-connection-manager)\n - [`Swarm.ConnMgr.LowWater`](#swarmconnmgrlowwater)\n - [`Swarm.ConnMgr.HighWater`](#swarmconnmgrhighwater)\n - [`Swarm.ConnMgr.GracePeriod`](#swarmconnmgrgraceperiod)\n - [`Swarm.ConnMgr.SilencePeriod`](#swarmconnmgrsilenceperiod)\n - [`Swarm.ResourceMgr`](#swarmresourcemgr)\n - [`Swarm.ResourceMgr.Enabled`](#swarmresourcemgrenabled)\n - [`Swarm.ResourceMgr.MaxMemory`](#swarmresourcemgrmaxmemory)\n - [`Swarm.ResourceMgr.MaxFileDescriptors`](#swarmresourcemgrmaxfiledescriptors)\n - [`Swarm.ResourceMgr.Allowlist`](#swarmresourcemgrallowlist)\n - [`Swarm.Transports`](#swarmtransports)\n - [`Swarm.Transports.Network`](#swarmtransportsnetwork)\n - [`Swarm.Transports.Network.TCP`](#swarmtransportsnetworktcp)\n - [`Swarm.Transports.Network.Websocket`](#swarmtransportsnetworkwebsocket)\n - [`Swarm.Transports.Network.QUIC`](#swarmtransportsnetworkquic)\n - [`Swarm.Transports.Network.Relay`](#swarmtransportsnetworkrelay)\n - [`Swarm.Transports.Network.WebTransport`](#swarmtransportsnetworkwebtransport)\n - [`Swarm.Transports.Network.WebRTCDirect`](#swarmtransportsnetworkwebrtcdirect)\n - [`Swarm.Transports.Security`](#swarmtransportssecurity)\n - [`Swarm.Transports.Security.TLS`](#swarmtransportssecuritytls)\n - [`Swarm.Transports.Security.SECIO`](#swarmtransportssecuritysecio)\n - [`Swarm.Transports.Security.Noise`](#swarmtransportssecuritynoise)\n - [`Swarm.Transports.Multiplexers`](#swarmtransportsmultiplexers)\n - [`Swarm.Transports.Multiplexers.Yamux`](#swarmtransportsmultiplexersyamux)\n - [`Swarm.Transports.Multiplexers.Mplex`](#swarmtransportsmultiplexersmplex)\n - [`DNS`](#dns)\n - [`DNS.Resolvers`](#dnsresolvers)\n - [`DNS.MaxCacheTTL`](#dnsmaxcachettl)\n - [`HTTPRetrieval`](#httpretrieval)\n - [`HTTPRetrieval.Enabled`](#httpretrievalenabled)\n - [`HTTPRetrieval.Allowlist`](#httpretrievalallowlist)\n - [`HTTPRetrieval.Denylist`](#httpretrievaldenylist)\n - [`HTTPRetrieval.NumWorkers`](#httpretrievalnumworkers)\n - [`HTTPRetrieval.MaxBlockSize`](#httpretrievalmaxblocksize)\n - [`HTTPRetrieval.TLSInsecureSkipVerify`](#httpretrievaltlsinsecureskipverify)\n - [`Import`](#import)\n - [`Import.CidVersion`](#importcidversion)\n - [`Import.UnixFSRawLeaves`](#importunixfsrawleaves)\n - [`Import.UnixFSChunker`](#importunixfschunker)\n - [`Import.HashFunction`](#importhashfunction)\n - [`Import.FastProvideRoot`](#importfastprovideroot)\n - [`Import.FastProvideWait`](#importfastprovidewait)\n - [`Import.BatchMaxNodes`](#importbatchmaxnodes)\n - [`Import.BatchMaxSize`](#importbatchmaxsize)\n - [`Import.UnixFSFileMaxLinks`](#importunixfsfilemaxlinks)\n - [`Import.UnixFSDirectoryMaxLinks`](#importunixfsdirectorymaxlinks)\n - [`Import.UnixFSHAMTDirectoryMaxFanout`](#importunixfshamtdirectorymaxfanout)\n - [`Import.UnixFSHAMTDirectorySizeThreshold`](#importunixfshamtdirectorysizethreshold)\n - [`Import.UnixFSHAMTDirectorySizeEstimation`](#importunixfshamtdirectorysizeestimation)\n - [`Import.UnixFSDAGLayout`](#importunixfsdaglayout)\n - [`Version`](#version)\n - [`Version.AgentSuffix`](#versionagentsuffix)\n - [`Version.SwarmCheckEnabled`](#versionswarmcheckenabled)\n - [`Version.SwarmCheckPercentThreshold`](#versionswarmcheckpercentthreshold)\n - [Profiles](#profiles)\n - [`server` profile](#server-profile)\n - [`randomports` profile](#randomports-profile)\n - [`default-datastore` profile](#default-datastore-profile)\n - [`local-discovery` profile](#local-discovery-profile)\n - [`default-networking` profile](#default-networking-profile)\n - [`autoconf-on` profile](#autoconf-on-profile)\n - [`autoconf-off` profile](#autoconf-off-profile)\n - [`flatfs` profile](#flatfs-profile)\n - [`flatfs-measure` profile](#flatfs-measure-profile)\n - [`pebbleds` profile](#pebbleds-profile)\n - [`pebbleds-measure` profile](#pebbleds-measure-profile)\n - [`badgerds` profile](#badgerds-profile)\n - [`badgerds-measure` profile](#badgerds-measure-profile)\n - [`lowpower` profile](#lowpower-profile)\n - [`announce-off` profile](#announce-off-profile)\n - [`announce-on` profile](#announce-on-profile)\n - [`unixfs-v0-2015` profile](#unixfs-v0-2015-profile)\n - [`legacy-cid-v0` profile](#legacy-cid-v0-profile)\n - [`unixfs-v1-2025` profile](#unixfs-v1-2025-profile)\n - [Security](#security)\n - [Port and Network Exposure](#port-and-network-exposure)\n - [Security Best Practices](#security-best-practices)\n - [Types](#types)\n - [`flag`](#flag)\n - [`priority`](#priority)\n - [`strings`](#strings)\n - [`duration`](#duration)\n - [`optionalInteger`](#optionalinteger)\n - [`optionalBytes`](#optionalbytes)\n - [`optionalString`](#optionalstring)\n - [`optionalDuration`](#optionalduration)\n\n## `Addresses`\n\nContains information about various listener addresses to be used by this node.\n\n### `Addresses.API`\n\n[Multiaddr][multiaddr] or array of multiaddrs describing the addresses to serve\nthe local [Kubo RPC API](https://docs.ipfs.tech/reference/kubo/rpc/) (`/api/v0`).\n\nSupported Transports:\n\n- tcp/ip{4,6} - `/ipN/.../tcp/...`\n- unix - `/unix/path/to/socket`\n\n> [!CAUTION]\n> **NEVER EXPOSE UNPROTECTED ADMIN RPC TO LAN OR THE PUBLIC INTERNET**\n>\n> The RPC API grants admin-level access to your Kubo IPFS node, including\n> configuration and secret key management.\n>\n> By default, it is bound to localhost for security reasons. Exposing it to LAN\n> or the public internet is highly risky—similar to exposing a SQL database or\n> backend service without authentication middleware\n>\n> - If you need secure access to a subset of RPC, secure it with [`API.Authorizations`](#apiauthorizations) or custom auth middleware running in front of the localhost-only RPC port defined here.\n> - If you are looking for an interface designed for browsers and public internet, use [`Addresses.Gateway`](#addressesgateway) port instead.\n> - See [Security section](#security) for network exposure considerations.\n\nDefault: `/ip4/127.0.0.1/tcp/5001`\n\nType: `strings` ([multiaddrs][multiaddr])\n\n### `Addresses.Gateway`\n\n[Multiaddr][multiaddr] or array of multiaddrs describing the address to serve\nthe local [HTTP gateway](https://specs.ipfs.tech/http-gateways/) (`/ipfs`, `/ipns`) on.\n\nSupported Transports:\n\n- tcp/ip{4,6} - `/ipN/.../tcp/...`\n- unix - `/unix/path/to/socket`\n\n> [!CAUTION]\n> **SECURITY CONSIDERATIONS FOR GATEWAY EXPOSURE**\n>\n> By default, the gateway is bound to localhost for security. If you bind to `0.0.0.0`\n> or a public IP, anyone with access can trigger retrieval of arbitrary CIDs, causing\n> bandwidth usage and potential exposure to malicious content. Limit with\n> [`Gateway.NoFetch`](#gatewaynofetch). Consider firewall rules, authentication,\n> and [`Gateway.PublicGateways`](#gatewaypublicgateways) for public exposure.\n> See [Security section](#security) for network exposure considerations.\n\nDefault: `/ip4/127.0.0.1/tcp/8080`\n\nType: `strings` ([multiaddrs][multiaddr])\n\n### `Addresses.Swarm`\n\nAn array of [multiaddrs][multiaddr] describing which addresses to listen on for p2p swarm\nconnections.\n\nSupported Transports:\n\n- tcp/ip{4,6} - `/ipN/.../tcp/...`\n- websocket - `/ipN/.../tcp/.../ws`\n- quicv1 (RFC9000) - `/ipN/.../udp/.../quic-v1` - can share the same two tuple with `/quic-v1/webtransport`\n- webtransport `/ipN/.../udp/.../quic-v1/webtransport` - can share the same two tuple with `/quic-v1`\n\n> [!IMPORTANT]\n> Make sure your firewall rules allow incoming connections on both TCP and UDP ports defined here.\n> See [Security section](#security) for network exposure considerations.\n\nNote that quic (Draft-29) used to be supported with the format `/ipN/.../udp/.../quic`, but has since been [removed](https://github.com/libp2p/go-libp2p/releases/tag/v0.30.0).\n\nDefault:\n\n```json\n[\n \"/ip4/0.0.0.0/tcp/4001\",\n \"/ip6/::/tcp/4001\",\n \"/ip4/0.0.0.0/udp/4001/quic-v1\",\n \"/ip4/0.0.0.0/udp/4001/quic-v1/webtransport\",\n \"/ip6/::/udp/4001/quic-v1\",\n \"/ip6/::/udp/4001/quic-v1/webtransport\"\n]\n```\n\nType: `array[string]` ([multiaddrs][multiaddr])\n\n### `Addresses.Announce`\n\nIf non-empty, this array specifies the swarm addresses to announce to the\nnetwork. If empty, the daemon will announce inferred swarm addresses.\n\nDefault: `[]`\n\nType: `array[string]` ([multiaddrs][multiaddr])\n\n### `Addresses.AppendAnnounce`\n\nSimilar to [`Addresses.Announce`](#addressesannounce) except this doesn't\noverride inferred swarm addresses if non-empty.\n\nDefault: `[]`\n\nType: `array[string]` ([multiaddrs][multiaddr])\n\n### `Addresses.NoAnnounce`\n\nAn array of swarm addresses not to announce to the network.\nTakes precedence over `Addresses.Announce` and `Addresses.AppendAnnounce`.\n\n> [!TIP]\n> The [`server` configuration profile](#server-profile) fills up this list with sensible defaults,\n> preventing announcement of non-routable IP addresses (e.g., `/ip4/192.168.0.0/ipcidr/16`,\n> which is the [multiaddress][multiaddr] representation of `192.168.0.0/16`) but you should always\n> check settings against your own network and/or hosting provider.\n\nDefault: `[]`\n\nType: `array[string]` ([multiaddrs][multiaddr])\n\n## `API`\n\nContains information used by the [Kubo RPC API](https://docs.ipfs.tech/reference/kubo/rpc/).\n\n### `API.HTTPHeaders`\n\nMap of HTTP headers to set on responses from the RPC (`/api/v0`) HTTP server.\n\nExample:\n\n```json\n{\n \"Foo\": [\"bar\"]\n}\n```\n\nDefault: `null`\n\nType: `object[string -> array[string]]` (header names -> array of header values)\n\n### `API.Authorizations`\n\nThe `API.Authorizations` field defines user-based access restrictions for the\n[Kubo RPC API](https://docs.ipfs.tech/reference/kubo/rpc/), which is located at\n`Addresses.API` under `/api/v0` paths.\n\nBy default, the admin-level RPC API is accessible without restrictions as it is only\nexposed on `127.0.0.1` and safeguarded with Origin check and implicit\n[CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) headers that\nblock random websites from accessing the RPC.\n\nWhen entries are defined in `API.Authorizations`, RPC requests will be declined\nunless a corresponding secret is present in the HTTP [`Authorization` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization),\nand the requested path is included in the `AllowedPaths` list for that specific\nsecret.\n\n> [!CAUTION]\n> **NEVER EXPOSE UNPROTECTED ADMIN RPC TO LAN OR THE PUBLIC INTERNET**\n>\n> The RPC API is vast. It grants admin-level access to your Kubo IPFS node, including\n> configuration and secret key management.\n>\n> - If you need secure access to a subset of RPC, make sure you understand the risk, block everything by default and allow basic auth access with [`API.Authorizations`](#apiauthorizations) or custom auth middleware running in front of the localhost-only port defined in [`Addresses.API`](#addressesapi).\n> - If you are looking for an interface designed for browsers and public internet, use [`Addresses.Gateway`](#addressesgateway) port instead.\n\nDefault: `null`\n\nType: `object[string -> object]` (user name -> authorization object, see below)\n\nFor example, to limit RPC access to Alice (access `id` and MFS `files` commands with HTTP Basic Auth)\nand Bob (full access with Bearer token):\n\n```json\n{\n \"API\": {\n \"Authorizations\": {\n \"Alice\": {\n \"AuthSecret\": \"basic:alice:password123\",\n \"AllowedPaths\": [\"/api/v0/id\", \"/api/v0/files\"]\n },\n \"Bob\": {\n \"AuthSecret\": \"bearer:secret-token123\",\n \"AllowedPaths\": [\"/api/v0\"]\n }\n }\n }\n}\n\n```\n\n#### `API.Authorizations: AuthSecret`\n\nThe `AuthSecret` field denotes the secret used by a user to authenticate,\nusually via HTTP [`Authorization` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization).\n\nField format is `type:value`, and the following types are supported:\n\n- `bearer:` For secret Bearer tokens, set as `bearer:token`.\n - If no known `type:` prefix is present, `bearer:` is assumed.\n- `basic`: For HTTP Basic Auth introduced in [RFC7617](https://datatracker.ietf.org/doc/html/rfc7617). Value can be:\n - `basic:user:pass`\n - `basic:base64EncodedBasicAuth`\n\nOne can use the config value for authentication via the command line:\n\n```\nipfs id --api-auth basic:user:pass\n```\n\nType: `string`\n\n#### `API.Authorizations: AllowedPaths`\n\nThe `AllowedPaths` field is an array of strings containing allowed RPC path\nprefixes. Users authorized with the related `AuthSecret` will only be able to\naccess paths prefixed by the specified prefixes.\n\nFor instance:\n\n- If set to `[\"/api/v0\"]`, the user will have access to the complete RPC API.\n- If set to `[\"/api/v0/id\", \"/api/v0/files\"]`, the user will only have access\n to the `id` command and all MFS commands under `files`.\n\nNote that `/api/v0/version` is always permitted access to allow version check\nto ensure compatibility.\n\nDefault: `[]`\n\nType: `array[string]`\n\n## `AutoNAT`\n\nContains the configuration options for the libp2p's [AutoNAT](https://github.com/libp2p/specs/tree/master/autonat) service. The AutoNAT service\nhelps other nodes on the network determine if they're publicly reachable from\nthe rest of the internet.\n\n### `AutoNAT.ServiceMode`\n\nWhen unset (default), the AutoNAT service defaults to _enabled_. Otherwise, this\nfield can take one of two values:\n\n- `enabled` - Enable the V1+V2 service (unless the node determines that it,\n itself, isn't reachable by the public internet).\n- `legacy-v1` - **DEPRECATED** Same as `enabled` but only V1 service is enabled. Used for testing\n during as few releases as we [transition to V2](https://github.com/ipfs/kubo/issues/10091), will be removed in the future.\n- `disabled` - Disable the service.\n\nAdditional modes may be added in the future.\n\n> [!IMPORTANT]\n> We are in the progress of [rolling out AutoNAT V2](https://github.com/ipfs/kubo/issues/10091).\n> Right now, by default, a publicly dialable Kubo provides both V1 and V2 service to other peers,\n> and V1 is still used by Kubo for Autorelay feature. In a future release we will remove V1 and switch all features to use V2.\n\nDefault: `enabled`\n\nType: `optionalString`\n\n### `AutoNAT.Throttle`\n\nWhen set, this option configures the AutoNAT services throttling behavior. By\ndefault, Kubo will rate-limit the number of NAT checks performed for other\nnodes to 30 per minute, and 3 per peer.\n\n### `AutoNAT.Throttle.GlobalLimit`\n\nConfigures how many AutoNAT requests to service per `AutoNAT.Throttle.Interval`.\n\nDefault: 30\n\nType: `integer` (non-negative, `0` means unlimited)\n\n### `AutoNAT.Throttle.PeerLimit`\n\nConfigures how many AutoNAT requests per-peer to service per `AutoNAT.Throttle.Interval`.\n\nDefault: 3\n\nType: `integer` (non-negative, `0` means unlimited)\n\n### `AutoNAT.Throttle.Interval`\n\nConfigures the interval for the above limits.\n\nDefault: 1 Minute\n\nType: `duration` (when `0`/unset, the default value is used)\n\n## `AutoConf`\n\nThe AutoConf feature enables Kubo nodes to automatically fetch and apply network configuration from a remote JSON endpoint. This system allows dynamic configuration updates for bootstrap peers, DNS resolvers, delegated routing, and IPNS publishing endpoints without requiring manual updates to each node's local config.\n\nAutoConf works by using special `\"auto\"` placeholder values in configuration fields. When Kubo encounters these placeholders, it fetches the latest configuration from the specified URL and resolves the placeholders with the appropriate values at runtime. The original configuration file remains unchanged - `\"auto\"` values are preserved in the JSON and only resolved in memory during node operation.\n\n### Key Features\n\n- **Remote Configuration**: Fetch network defaults from a trusted URL\n- **Automatic Updates**: Periodic background checks for configuration updates\n- **Graceful Fallback**: Uses hardcoded IPFS Mainnet bootstrappers when remote config is unavailable\n- **Validation**: Ensures all fetched configuration values are valid multiaddrs and URLs\n- **Caching**: Stores multiple versions locally with ETags for efficient updates\n- **User Notification**: Logs ERROR when new configuration is available requiring node restart\n- **Debug Logging**: AutoConf operations can be inspected by setting `GOLOG_LOG_LEVEL=\"error,autoconf=debug\"`\n\n### Supported Fields\n\nAutoConf can resolve `\"auto\"` placeholders in the following configuration fields:\n\n- `Bootstrap` - Bootstrap peer addresses\n- `DNS.Resolvers` - DNS-over-HTTPS resolver endpoints\n- `Routing.DelegatedRouters` - Delegated routing HTTP API endpoints\n- `Ipns.DelegatedPublishers` - IPNS delegated publishing HTTP API endpoints\n\n### Usage Example\n\n```json\n{\n \"AutoConf\": {\n \"URL\": \"https://example.com/autoconf.json\",\n \"Enabled\": true,\n \"RefreshInterval\": \"24h\"\n },\n \"Bootstrap\": [\"auto\"],\n \"DNS\": {\n \"Resolvers\": {\n \".\": [\"auto\"],\n \"eth.\": [\"auto\"],\n \"custom.\": [\"https://dns.example.com/dns-query\"]\n }\n },\n \"Routing\": {\n \"DelegatedRouters\": [\"auto\", \"https://router.example.org/routing/v1\"]\n }\n}\n```\n\n**Notes:**\n\n- Configuration fetching happens at daemon startup and periodically in the background\n- When new configuration is detected, users must restart their node to apply changes\n- Mixed configurations are supported: you can use both `\"auto\"` and static values\n- If AutoConf is disabled but `\"auto\"` values exist, daemon startup will fail with validation errors\n- Cache is stored in `$IPFS_PATH/autoconf/` with up to 3 versions retained\n\n### Path-Based Routing Configuration\n\nAutoConf supports path-based routing URLs that automatically enable specific routing operations based on the URL path. This allows precise control over which HTTP Routing V1 endpoints are used for different operations:\n\n**Supported paths:**\n\n- `/routing/v1/providers` - Enables provider record lookups only\n- `/routing/v1/peers` - Enables peer routing lookups only \n- `/routing/v1/ipns` - Enables IPNS record operations only\n- No path - Enables all routing operations (backward compatibility)\n\n**AutoConf JSON structure with path-based routing:**\n\n```json\n{\n \"DelegatedRouters\": {\n \"mainnet-for-nodes-with-dht\": [\n \"https://cid.contact/routing/v1/providers\"\n ],\n \"mainnet-for-nodes-without-dht\": [\n \"https://delegated-ipfs.dev/routing/v1/providers\",\n \"https://delegated-ipfs.dev/routing/v1/peers\", \n \"https://delegated-ipfs.dev/routing/v1/ipns\"\n ]\n },\n \"DelegatedPublishers\": {\n \"mainnet-for-ipns-publishers-with-http\": [\n \"https://delegated-ipfs.dev/routing/v1/ipns\"\n ]\n }\n}\n```\n\n**Node type categories:**\n\n- `mainnet-for-nodes-with-dht`: Mainnet nodes with DHT enabled (typically only need additional provider lookups)\n- `mainnet-for-nodes-without-dht`: Mainnet nodes without DHT (need comprehensive routing services)\n- `mainnet-for-ipns-publishers-with-http`: Mainnet nodes that publish IPNS records via HTTP\n\nThis design enables efficient, selective routing where each endpoint URL automatically determines its capabilities based on the path, while maintaining semantic grouping by node configuration type.\n\nDefault: `{}`\n\nType: `object`\n\n### `AutoConf.Enabled`\n\nControls whether the AutoConf system is active. When enabled, Kubo will fetch configuration from the specified URL and resolve `\"auto\"` placeholders at runtime. When disabled, any `\"auto\"` values in the configuration will cause daemon startup to fail with validation errors.\n\nThis provides a safety mechanism to ensure nodes don't start with unresolved placeholders when AutoConf is intentionally disabled.\n\nDefault: `true`\n\nType: `flag`\n\n### `AutoConf.URL`\n\nSpecifies the HTTP(S) URL from which to fetch the autoconf JSON. The endpoint should return a JSON document containing Bootstrap peers, DNS resolvers, delegated routing endpoints, and IPNS publishing endpoints that will replace `\"auto\"` placeholders in the local configuration.\n\nThe URL must serve a JSON document matching the AutoConf schema. Kubo validates all multiaddr and URL values before caching to ensure they are properly formatted.\n\nWhen not specified in the configuration, the default mainnet URL is used automatically.\n\n\n\n> [!NOTE]\n> Public good autoconf manifest at `conf.ipfs-mainnet.org` is provided by the team at [Shipyard](https://ipshipyard.com).\n\nDefault: `\"https://conf.ipfs-mainnet.org/autoconf.json\"` (when not specified)\n\nType: `optionalString`\n\n### `AutoConf.RefreshInterval`\n\nSpecifies how frequently Kubo should refresh autoconf data. This controls both how often cached autoconf data is considered fresh and how frequently the background service checks for new configuration updates.\n\nWhen a new configuration version is detected during background updates, Kubo logs an ERROR message informing the user that a node restart is required to apply the changes to any `\"auto\"` entries in their configuration.\n\nDefault: `24h`\n\nType: `optionalDuration`\n\n### `AutoConf.TLSInsecureSkipVerify`\n\n**FOR TESTING ONLY** - Allows skipping TLS certificate verification when fetching autoconf from HTTPS URLs. This should never be enabled in production as it makes the configuration fetching vulnerable to man-in-the-middle attacks.\n\nDefault: `false`\n\nType: `flag`\n\n## `AutoTLS`\n\nThe [AutoTLS](https://web.archive.org/web/20260112031855/https://blog.libp2p.io/autotls/) feature enables publicly reachable Kubo nodes (those dialable from the public\ninternet) to automatically obtain a wildcard TLS certificate for a DNS name\nunique to their PeerID at `*.[PeerID].libp2p.direct`. This enables direct\nlibp2p connections and retrieval of IPFS content from browsers [Secure Context](https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts)\nusing transports such as [Secure WebSockets](https://github.com/libp2p/specs/blob/master/websockets/README.md),\nwithout requiring user to do any manual domain registration and certificate configuration.\n\nUnder the hood, [p2p-forge] client uses public utility service at `libp2p.direct` as an [ACME DNS-01 Challenge](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge)\nbroker enabling peer to obtain a wildcard TLS certificate tied to public key of their [PeerID](https://web.archive.org/web/20251112181025/https://docs.libp2p.io/concepts/fundamentals/peers/#peer-id).\n\nBy default, the certificates are requested from Let's Encrypt. Origin and rationale for this project can be found in [community.letsencrypt.org discussion](https://community.letsencrypt.org/t/feedback-on-raising-certificates-per-registered-domain-to-enable-peer-to-peer-networking/223003).\n\n\n\n> [!NOTE]\n> Public good DNS and [p2p-forge] infrastructure at `libp2p.direct` is run by the team at [Interplanetary Shipyard](https://ipshipyard.com).\n>\n[p2p-forge]: https://github.com/ipshipyard/p2p-forge\n\nDefault: `{}`\n\nType: `object`\n\n### `AutoTLS.Enabled`\n\nEnables the AutoTLS feature to provide DNS and TLS support for [libp2p Secure WebSocket](https://github.com/libp2p/specs/blob/master/websockets/README.md) over a `/tcp` port,\nto allow JS clients running in web browser [Secure Context](https://w3c.github.io/webappsec-secure-contexts/) to connect to Kubo directly.\n\nWhen activated, together with [`AutoTLS.AutoWSS`](#autotlsautowss) (default) or manually including a `/tcp/{port}/tls/sni/*.libp2p.direct/ws` multiaddr in [`Addresses.Swarm`](#addressesswarm)\n(with SNI suffix matching [`AutoTLS.DomainSuffix`](#autotlsdomainsuffix)), Kubo retrieves a trusted PKI TLS certificate for `*.{peerid}.libp2p.direct` and configures the `/ws` listener to use it.\n\n**Note:**\n\n- This feature requires a publicly reachable node. If behind NAT, manual port forwarding or UPnP (`Swarm.DisableNatPortMap=false`) is required.\n- The first time AutoTLS is used, it may take 5-15 minutes + [`AutoTLS.RegistrationDelay`](#autotlsregistrationdelay) before `/ws` listener is added. Be patient.\n- Avoid manual configuration. [`AutoTLS.AutoWSS=true`](#autotlsautowss) should automatically add `/ws` listener to existing, firewall-forwarded `/tcp` ports.\n- To troubleshoot, use `GOLOG_LOG_LEVEL=\"error,autotls=debug` for detailed logs, or `GOLOG_LOG_LEVEL=\"error,autotls=info` for quieter output.\n- Certificates are stored in `$IPFS_PATH/p2p-forge-certs`; deleting this directory and restarting the daemon forces a certificate rotation.\n- For now, the TLS cert applies solely to `/ws` libp2p WebSocket connections, not HTTP [`Gateway`](#gateway), which still need separate reverse proxy TLS setup with a custom domain.\n\nDefault: `true`\n\nType: `flag`\n\n### `AutoTLS.AutoWSS`\n\nOptional. Controls if Kubo should add `/tls/sni/*.libp2p.direct/ws` listener to every pre-existing `/tcp` port IFF no explicit `/ws` is defined in [`Addresses.Swarm`](#addressesswarm) already.\n\nDefault: `true` (if `AutoTLS.Enabled`)\n\nType: `flag`\n\n### `AutoTLS.ShortAddrs`\n\nOptional. Controls if final AutoTLS listeners are announced under shorter `/dnsX/A.B.C.D.peerid.libp2p.direct/tcp/4001/tls/ws` addresses instead of fully resolved `/ip4/A.B.C.D/tcp/4001/tls/sni/A-B-C-D.peerid.libp2p.direct/tls/ws`.\n\nThe main use for AutoTLS is allowing connectivity from Secure Context in a web browser, and DNS lookup needs to happen there anyway, making `/dnsX` a more compact, more interoperable option without obvious downside.\n\nDefault: `true`\n\nType: `flag`\n\n### `AutoTLS.SkipDNSLookup`\n\nOptional. Controls whether to skip network DNS lookups for [p2p-forge] domains like `*.libp2p.direct`.\n\nThis applies to DNS resolution performed via [`DNS.Resolvers`](#dnsresolvers), including `/dns*` multiaddrs resolved by go-libp2p (e.g., peer addresses from DHT or delegated routing).\n\nWhen enabled (default), A/AAAA queries for hostnames matching [`AutoTLS.DomainSuffix`](#autotlsdomainsuffix) are resolved locally by parsing the IP address directly from the hostname (e.g., `1-2-3-4.peerID.libp2p.direct` resolves to `1.2.3.4` without network I/O). This avoids unnecessary DNS queries since the IP is already encoded in the hostname.\n\nIf the hostname format is invalid (wrong peerID, malformed IP encoding), the resolver falls back to network DNS, ensuring forward compatibility with potential future DNS record types.\n\nSet to `false` to always use network DNS for these domains. This is primarily useful for debugging or if you need to override resolution behavior via [`DNS.Resolvers`](#dnsresolvers).\n\nDefault: `true`\n\nType: `flag`\n\n### `AutoTLS.DomainSuffix`\n\nOptional override of the parent domain suffix that will be used in DNS+TLS+WebSockets multiaddrs generated by [p2p-forge] client.\nDo not change this unless you self-host [p2p-forge].\n\nDefault: `libp2p.direct` (public good run by [Interplanetary Shipyard](https://ipshipyard.com))\n\nType: `optionalString`\n\n### `AutoTLS.RegistrationEndpoint`\n\nOptional override of [p2p-forge] HTTP registration API.\nDo not change this unless you self-host [p2p-forge] under own domain.\n\n> [!IMPORTANT]\n> The default endpoint performs [libp2p Peer ID Authentication over HTTP](https://github.com/libp2p/specs/blob/master/http/peer-id-auth.md)\n> (proving ownership of PeerID), probes if your Kubo node can correctly answer to a [libp2p Identify](https://github.com/libp2p/specs/tree/master/identify) query.\n> This ensures only a correctly configured, publicly dialable Kubo can initiate [ACME DNS-01 challenge](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) for `peerid.libp2p.direct`.\n\nDefault: `https://registration.libp2p.direct` (public good run by [Interplanetary Shipyard](https://ipshipyard.com))\n\nType: `optionalString`\n\n### `AutoTLS.RegistrationToken`\n\nOptional value for `Forge-Authorization` token sent with request to `RegistrationEndpoint`\n(useful for private/self-hosted/test instances of [p2p-forge], unset by default).\n\nDefault: `\"\"`\n\nType: `optionalString`\n\n### `AutoTLS.RegistrationDelay`\n\nAn additional delay applied before sending a request to the `RegistrationEndpoint`.\n\nThe default delay is bypassed if the user explicitly set `AutoTLS.Enabled=true` in the JSON configuration file.\nThis ensures that ephemeral nodes using the default configuration do not spam the`AutoTLS.CAEndpoint` with unnecessary ACME requests.\n\nDefault: `1h` (or `0` if explicit `AutoTLS.Enabled=true`)\n\nType: `optionalDuration`\n\n### `AutoTLS.CAEndpoint`\n\nOptional override of CA ACME API used by [p2p-forge] system.\nDo not change this unless you self-host [p2p-forge] under own domain.\n\n> [!IMPORTANT]\n> CAA DNS record at `libp2p.direct` limits CA choice to Let's Encrypt. If you want to use a different CA, use your own domain.\n\nDefault: [certmagic.LetsEncryptProductionCA](https://pkg.go.dev/github.com/caddyserver/certmagic#pkg-constants) (see [community.letsencrypt.org discussion](https://community.letsencrypt.org/t/feedback-on-raising-certificates-per-registered-domain-to-enable-peer-to-peer-networking/223003))\n\nType: `optionalString`\n\n## `Bitswap`\n\nHigh level client and server configuration of the [Bitswap Protocol](https://specs.ipfs.tech/bitswap-protocol/) over libp2p.\n\nFor internal configuration see [`Internal.Bitswap`](#internalbitswap).\n\nFor HTTP version see [`HTTPRetrieval`](#httpretrieval).\n\n### `Bitswap.Libp2pEnabled`\n\nDetermines whether Kubo will use Bitswap over libp2p.\n\nDisabling this, will remove `/ipfs/bitswap/*` protocol support from [libp2p identify](https://github.com/libp2p/specs/blob/master/identify/README.md) responses, effectively shutting down both Bitswap libp2p client and server.\n\n> [!WARNING]\n> Bitswap over libp2p is a core component of Kubo and the oldest way of exchanging blocks. Disabling it completely may cause unpredictable outcomes, such as retrieval failures, if the only providers were libp2p ones. Treat this as experimental and use it solely for testing purposes with `HTTPRetrieval.Enabled`.\n\nDefault: `true`\n\nType: `flag`\n\n### `Bitswap.ServerEnabled`\n\nDetermines whether Kubo functions as a Bitswap server to host and respond to block requests.\n\nDisabling the server retains client and protocol support in [libp2p identify](https://github.com/libp2p/specs/blob/master/identify/README.md) responses but causes Kubo to reply with \"don't have\" to all block requests.\n\nDefault: `true` (requires `Bitswap.Libp2pEnabled`)\n\nType: `flag`\n\n## `Bootstrap`\n\nBootstrap peers help your node discover and connect to the IPFS network when starting up. This array contains [multiaddrs][multiaddr] of trusted nodes that your node contacts first to find other peers and content.\n\nThe special value `\"auto\"` automatically uses curated, up-to-date bootstrap peers from [AutoConf](#autoconf), ensuring your node can always connect to the healthy network without manual maintenance.\n\n**What this gives you:**\n\n- **Reliable startup**: Your node can always find the network, even if some bootstrap peers go offline\n- **Automatic updates**: New bootstrap peers are added as the network evolves\n- **Custom control**: Add your own trusted peers alongside or instead of the defaults\n\nDefault: `[\"auto\"]`\n\nType: `array[string]` ([multiaddrs][multiaddr] or `\"auto\"`)\n\n## `Datastore`\n\nContains information related to the construction and operation of the on-disk\nstorage system.\n\n### `Datastore.StorageMax`\n\nA soft upper limit for the size of the ipfs repository's datastore. With `StorageGCWatermark`,\nis used to calculate whether to trigger a gc run (only if `--enable-gc` flag is set).\n\n> [!NOTE]\n> This only controls when automatic GC of raw blocks is triggered. It is not a\n> hard limit on total disk usage. The metadata stored alongside blocks (pins,\n> MFS, provider system state, pubsub message ID tracking, and other internal\n> data) is not counted against this limit. Always include extra headroom to\n> account for metadata overhead. See [datastores.md](datastores.md) for details\n> on how different datastore backends handle disk space reclamation.\n\nDefault: `\"10GB\"`\n\nType: `string` (size)\n\n### `Datastore.StorageGCWatermark`\n\nThe percentage of the `StorageMax` value at which a garbage collection will be\ntriggered automatically if the daemon was run with automatic gc enabled (that\noption defaults to false currently).\n\nDefault: `90`\n\nType: `integer` (0-100%)\n\n### `Datastore.GCPeriod`\n\nA time duration specifying how frequently to run a garbage collection. Only used\nif automatic gc is enabled.\n\nDefault: `1h`\n\nType: `duration` (an empty string means the default value)\n\n### `Datastore.HashOnRead`\n\nA boolean value. If set to true, all block reads from the disk will be hashed and\nverified. This will cause increased CPU utilization.\n\nDefault: `false`\n\nType: `bool`\n\n### `Datastore.BloomFilterSize`\n\nA number representing the size in bytes of the blockstore's [bloom\nfilter](https://en.wikipedia.org/wiki/Bloom_filter). A value of zero represents\nthe feature is disabled.\n\nThis site generates useful graphs for various bloom filter values:\n You may use it to find a\npreferred optimal value, where `m` is `BloomFilterSize` in bits. Remember to\nconvert the value `m` from bits, into bytes for use as `BloomFilterSize` in the\nconfig file. For example, for 1,000,000 blocks, expecting a 1% false-positive\nrate, you'd end up with a filter size of 9592955 bits, so for `BloomFilterSize`\nwe'd want to use 1199120 bytes. As of writing, [7 hash\nfunctions](https://github.com/ipfs/go-ipfs-blockstore/blob/547442836ade055cc114b562a3cc193d4e57c884/caching.go#L22)\nare used, so the constant `k` is 7 in the formula.\n\nEnabling the BloomFilter can provide performance improvements specially when\nresponding to many requests for inexistent blocks. It however requires a full\nsweep of all the datastore keys on daemon start. On very large datastores this\ncan be a very taxing operation, particularly if the datastore does not support\nquerying existing keys without reading their values at the same time (blocks).\n\nDefault: `0` (disabled)\n\nType: `integer` (non-negative, bytes)\n\n### `Datastore.WriteThrough`\n\nThis option controls whether a block that already exist in the datastore\nshould be written to it. When set to `false`, a `Has()` call is performed\nagainst the datastore prior to writing every block. If the block is already\nstored, the write is skipped. This check happens both on the Blockservice and\nthe Blockstore layers and this setting affects both.\n\nWhen set to `true`, no checks are performed and blocks are written to the\ndatastore, which depending on the implementation may perform its own checks.\n\nThis option can affect performance and the strategy should be taken in\nconjunction with [`BlockKeyCacheSize`](#datastoreblockkeycachesize) and\n[`BloomFilterSize`](#datastoreboomfiltersize`).\n\nDefault: `true`\n\nType: `bool`\n\n### `Datastore.BlockKeyCacheSize`\n\nA number representing the maximum size in bytes of the blockstore's Two-Queue\ncache, which caches block-cids and their block-sizes. Use `0` to disable.\n\nThis cache, once primed, can greatly speed up operations like `ipfs repo stat`\nas there is no need to read full blocks to know their sizes. Size should be\nadjusted depending on the number of CIDs on disk (`NumObjects in`ipfs repo stat`).\n\nDefault: `65536` (64KiB)\n\nType: `optionalInteger` (non-negative, bytes)\n\n### `Datastore.Spec`\n\nSpec defines the structure of the ipfs datastore. It is a composable structure,\nwhere each datastore is represented by a json object. Datastores can wrap other\ndatastores to provide extra functionality (eg metrics, logging, or caching).\n\n> [!NOTE]\n> For more information on possible values for this configuration option, see [`kubo/docs/datastores.md`](datastores.md)\n\nDefault:\n\n```\n{\n \"mounts\": [\n {\n \"mountpoint\": \"/blocks\",\n \"path\": \"blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": false,\n \"type\": \"flatfs\"\n },\n {\n \"compression\": \"none\",\n \"mountpoint\": \"/\",\n \"path\": \"datastore\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"levelds\"\n }\n ],\n \"type\": \"mount\"\n}\n```\n\nWith `flatfs-measure` profile:\n\n```\n{\n \"mounts\": [\n {\n \"child\": {\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": true,\n \"type\": \"flatfs\"\n },\n \"mountpoint\": \"/blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"type\": \"measure\"\n },\n {\n \"child\": {\n \"compression\": \"none\",\n \"path\": \"datastore\",\n \"type\": \"levelds\"\n },\n \"mountpoint\": \"/\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"measure\"\n }\n ],\n \"type\": \"mount\"\n}\n```\n\nType: `object`\n\n## `Discovery`\n\nContains options for configuring IPFS node discovery mechanisms.\n\n### `Discovery.MDNS`\n\nOptions for [ZeroConf](https://github.com/libp2p/zeroconf#readme) Multicast DNS-SD peer discovery.\n\n#### `Discovery.MDNS.Enabled`\n\nA boolean value to activate or deactivate Multicast DNS-SD.\n\nDefault: `true`\n\nType: `bool`\n\n#### `Discovery.MDNS.Interval`\n\n**REMOVED:** this is not configurable anymore\nin the [new mDNS implementation](https://github.com/libp2p/zeroconf#readme).\n\n## `Experimental`\n\nToggle and configure experimental features of Kubo. Experimental features are listed [here](./experimental-features.md).\n\n### `Experimental.Libp2pStreamMounting`\n\nEnables the `ipfs p2p` commands for tunneling TCP connections through libp2p\nstreams, similar to SSH port forwarding.\n\nSee [docs/p2p-tunnels.md](p2p-tunnels.md) for usage examples.\n\nDefault: `false`\n\nType: `bool`\n\n## `Gateway`\n\nOptions for the HTTP gateway.\n\n> [!IMPORTANT]\n> By default, Kubo's gateway is configured for local use at `127.0.0.1` and `localhost`.\n> To run a public gateway, configure your domain names in [`Gateway.PublicGateways`](#gatewaypublicgateways).\n> For production deployment considerations (reverse proxy, timeouts, rate limiting, CDN),\n> see [Running in Production](gateway.md#running-in-production).\n\n### `Gateway.NoFetch`\n\nWhen set to true, the gateway will only serve content already in the local repo\nand will not fetch files from the network.\n\nDefault: `false`\n\nType: `bool`\n\n### `Gateway.NoDNSLink`\n\nA boolean to configure whether DNSLink lookup for value in `Host` HTTP header\nshould be performed. If DNSLink is present, the content path stored in the DNS TXT\nrecord becomes the `/` and the respective payload is returned to the client.\n\nDefault: `false`\n\nType: `bool`\n\n### `Gateway.DeserializedResponses`\n\nAn optional flag to explicitly configure whether this gateway responds to deserialized\nrequests, or not. By default, it is enabled. When disabling this option, the gateway\noperates as a Trustless Gateway only: .\n\nDefault: `true`\n\nType: `flag`\n\n### `Gateway.AllowCodecConversion`\n\nAn optional flag to enable automatic conversion between codecs when the\nrequested format differs from the block's native codec (e.g., converting\ndag-pb or dag-cbor to dag-json).\n\nWhen disabled (the default), the gateway returns `406 Not Acceptable` for\ncodec mismatches, following behavior specified in\n[IPIP-524](https://specs.ipfs.tech/ipips/ipip-0524/).\n\nMost users should keep this disabled unless legacy\n[IPLD Logical Format](https://web.archive.org/web/20260204204727/https://ipld.io/specs/codecs/dag-pb/spec/#logical-format)\nsupport is needed as a stop-gap while switching clients to `?format=raw`\nand converting client-side.\n\nInstead of relying on gateway-side conversion, fetch the raw block using\n`?format=raw` (`application/vnd.ipld.raw`) and convert client-side. This:\n\n- Allows clients to use any codec without waiting for gateway support\n- Enables ecosystem innovation without gateway operator coordination\n- Works with libraries like [@helia/verified-fetch](https://www.npmjs.com/package/@helia/verified-fetch) in JavaScript\n\nDefault: `false`\n\nType: `flag`\n\n### `Gateway.DisableHTMLErrors`\n\nAn optional flag to disable the pretty HTML error pages of the gateway. Instead,\na `text/plain` page will be returned with the raw error message from Kubo.\n\nIt is useful for whitelabel or middleware deployments that wish to avoid\n`text/html` responses with IPFS branding and links on error pages in browsers.\n\nDefault: `false`\n\nType: `flag`\n\n### `Gateway.ExposeRoutingAPI`\n\nAn optional flag to expose Kubo `Routing` system on the gateway port\nas an [HTTP `/routing/v1`](https://specs.ipfs.tech/routing/http-routing-v1/) endpoint on `127.0.0.1`.\nUse reverse proxy to expose it on a different hostname.\n\nThis endpoint can be used by other Kubo instances, as illustrated in\n[`delegated_routing_v1_http_proxy_test.go`](https://github.com/ipfs/kubo/blob/master/test/cli/delegated_routing_v1_http_proxy_test.go).\nKubo will filter out routing results which are not actionable, for example, all\ngraphsync providers will be skipped. If you need a generic pass-through, see\nstandalone router implementation named [someguy](https://github.com/ipfs/someguy).\n\nDefault: `true`\n\nType: `flag`\n\n### `Gateway.RetrievalTimeout`\n\nMaximum duration Kubo will wait for content retrieval (new bytes to arrive).\n\n**Timeout behavior:**\n\n- **Time to first byte**: Returns 504 Gateway Timeout if the gateway cannot start writing within this duration (e.g., stuck searching for providers)\n- **Time between writes**: After first byte, timeout resets with each write. Response terminates if no new data can be written within this duration\n\n**Truncation handling:** When timeout occurs after HTTP 200 headers are sent (e.g., during CAR streams), the gateway:\n\n- Appends error message to indicate truncation\n- Forces TCP reset (RST) to prevent caching incomplete responses\n- Records in metrics with original status code and `truncated=true` flag\n\n**Monitoring:** Track `ipfs_http_gw_retrieval_timeouts_total` by status code and truncation status.\n\n**Tuning guidance:**\n\n- Compare timeout rates (`ipfs_http_gw_retrieval_timeouts_total`) with success rates (`ipfs_http_gw_responses_total{status=\"200\"}`)\n- High timeout rate: consider increasing timeout or scaling horizontally if hardware is constrained\n- Many 504s may indicate routing problems - check requested CIDs and provider availability using \n- `truncated=true` timeouts indicate retrieval stalled mid-file with no new bytes for the timeout duration\n\nA value of 0 disables this timeout.\n\nDefault: `30s`\n\nType: `optionalDuration`\n\n### `Gateway.MaxRequestDuration`\n\nAn absolute deadline for the entire gateway request. Unlike [`RetrievalTimeout`](#gatewayretrievaltimeout) (which resets on each data write and catches stalled transfers), this is a hard limit on the total time a request can take.\n\nReturns 504 Gateway Timeout when exceeded. This protects the gateway from edge cases and slow client attacks.\n\nDefault: `1h`\n\nType: `optionalDuration`\n\n### `Gateway.MaxRangeRequestFileSize`\n\nMaximum file size for HTTP range requests on deserialized responses. Range requests for files larger than this limit return 501 Not Implemented.\n\n**Why this exists:**\n\nSome CDNs like Cloudflare intercept HTTP range requests and convert them to full file downloads when files exceed their cache bucket limits. Cloudflare's default plan only caches range requests for files up to 5GiB. Files larger than this receive HTTP 200 with the entire file instead of HTTP 206 with the requested byte range. A client requesting 1MB from a 40GiB file would unknowingly download all 40GiB, causing bandwidth overcharges for the gateway operator, unexpected data costs for the client, and potential browser crashes.\n\nThis only affects deserialized responses. Clients fetching verifiable blocks as `application/vnd.ipld.raw` are not impacted because they work with small chunks that stay well below CDN cache limits.\n\n**How to use:**\n\nSet this to your CDN's range request cache limit (e.g., `\"5GiB\"` for Cloudflare's default plan). The gateway returns 501 Not Implemented for range requests over files larger than this limit, with an error message suggesting verifiable block requests as an alternative.\n\n> [!NOTE]\n> Cloudflare users running open gateway hosting deserialized responses should deploy additional protection via Cloudflare Snippets (requires Enterprise plan). The Kubo configuration alone is not sufficient because Cloudflare has already intercepted and cached the response by the time it reaches your origin. See [boxo#856](https://github.com/ipfs/boxo/issues/856#issuecomment-3523944976) for a snippet that aborts HTTP 200 responses when Content-Length exceeds the limit.\n\nDefault: `0` (no limit)\n\nType: [`optionalBytes`](#optionalbytes)\n\n### `Gateway.MaxConcurrentRequests`\n\nLimits concurrent HTTP requests. Requests beyond limit receive 429 Too Many Requests.\n\nProtects nodes from traffic spikes and resource exhaustion, especially behind reverse proxies without rate-limiting. Default (4096) aligns with common reverse proxy configurations (e.g., nginx: 8 workers × 1024 connections).\n\n**Monitoring:** `ipfs_http_gw_concurrent_requests` tracks current requests in flight.\n\n**Tuning guidance:**\n\n- Monitor `ipfs_http_gw_concurrent_requests` gauge for usage patterns\n- Track 429s (`ipfs_http_gw_responses_total{status=\"429\"}`) and success rate (`{status=\"200\"}`)\n- Near limit with low resource usage → increase value\n- Memory pressure or OOMs → decrease value and consider scaling\n- Set slightly below reverse proxy limit for graceful degradation\n- Start with default, adjust based on observed performance for your hardware\n\nA value of 0 disables the limit.\n\nDefault: `4096`\n\nType: `optionalInteger`\n\n### `Gateway.HTTPHeaders`\n\nHeaders to set on gateway responses.\n\nDefault: `{}` + implicit CORS headers from `boxo/gateway#AddAccessControlHeaders` and [ipfs/specs#423](https://github.com/ipfs/specs/issues/423)\n\nType: `object[string -> array[string]]`\n\n### `Gateway.RootRedirect`\n\nA URL to redirect requests for `/` to.\n\nDefault: `\"\"`\n\nType: `string` (url)\n\n### `Gateway.DiagnosticServiceURL`\n\nURL for a service to diagnose CID retrievability issues. When the gateway returns a 504 Gateway Timeout error, an \"Inspect retrievability of CID\" button will be shown that links to this service with the CID appended as `?cid=`.\n\nSet to empty string to disable the button.\n\nDefault: `\"https://check.ipfs.network\"`\n\nType: `optionalstring` (url)\n\n### `Gateway.FastDirIndexThreshold`\n\n**REMOVED**: this option is [no longer necessary](https://github.com/ipfs/kubo/pull/9481). Ignored since [Kubo 0.18](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.18.md).\n\n### `Gateway.Writable`\n\n**REMOVED**: this option no longer available as of [Kubo 0.20](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.20.md).\n\nWe are working on developing a modern replacement. To support our efforts, please leave a comment describing your use case in [ipfs/specs#375](https://github.com/ipfs/specs/issues/375).\n\n### `Gateway.PathPrefixes`\n\n**REMOVED:** see [go-ipfs#7702](https://github.com/ipfs/go-ipfs/issues/7702)\n\n### `Gateway.PublicGateways`\n\n> [!IMPORTANT]\n> This configuration is **NOT** for HTTP Client, it is for HTTP Server – use this ONLY if you want to run your own IPFS gateway.\n\n`PublicGateways` is a configuration map used for dictionary for customizing gateway behavior\non specified hostnames that point at your Kubo instance.\n\nIt is useful when you want to run [Path gateway](https://specs.ipfs.tech/http-gateways/path-gateway/) on `example.com/ipfs/cid`,\nand [Subdomain gateway](https://specs.ipfs.tech/http-gateways/subdomain-gateway/) on `cid.ipfs.example.org`,\nor limit `verifiable.example.net` to response types defined in [Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/) specification.\n\n> [!CAUTION]\n> Keys (Hostnames) MUST be unique. Do not use the same parent domain for multiple gateway types, it will break origin isolation.\n\nHostnames can optionally be defined with one or more wildcards.\n\nExamples:\n\n- `*.example.com` will match requests to `http://foo.example.com/ipfs/*` or `http://{cid}.ipfs.bar.example.com/*`.\n- `foo-*.example.com` will match requests to `http://foo-bar.example.com/ipfs/*` or `http://{cid}.ipfs.foo-xyz.example.com/*`.\n\n> [!IMPORTANT]\n> **Reverse Proxy:** If running behind nginx or another reverse proxy, ensure\n> `Host` and `X-Forwarded-*` headers are forwarded correctly.\n> See [Reverse Proxy Caveats](gateway.md#reverse-proxy) in gateway documentation.\n\n#### `Gateway.PublicGateways: Paths`\n\nAn array of paths that should be exposed on the hostname.\n\nExample:\n\n```json\n{\n \"Gateway\": {\n \"PublicGateways\": {\n \"example.com\": {\n \"Paths\": [\"/ipfs\"],\n }\n }\n }\n}\n```\n\nAbove enables `http://example.com/ipfs/*` but not `http://example.com/ipns/*`\n\nDefault: `[]`\n\nType: `array[string]`\n\n#### `Gateway.PublicGateways: UseSubdomains`\n\nA boolean to configure whether the gateway at the hostname should be\na [Subdomain Gateway](https://specs.ipfs.tech/http-gateways/subdomain-gateway/)\nand provide [Origin isolation](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy)\nbetween content roots.\n\n- `true` - enables [subdomain gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://*.{hostname}/`\n - **Requires whitelist:** make sure respective `Paths` are set.\n For example, `Paths: [\"/ipfs\", \"/ipns\"]` are required for `http://{cid}.ipfs.{hostname}` and `http://{foo}.ipns.{hostname}` to work:\n\n ```json\n \"Gateway\": {\n \"PublicGateways\": {\n \"dweb.link\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\"]\n }\n }\n }\n ```\n\n - **Backward-compatible:** requests for content paths such as `http://{hostname}/ipfs/{cid}` produce redirect to `http://{cid}.ipfs.{hostname}`\n\n- `false` - enables [path gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#path-gateway) at `http://{hostname}/*`\n - Example:\n\n ```json\n \"Gateway\": {\n \"PublicGateways\": {\n \"ipfs.io\": {\n \"UseSubdomains\": false,\n \"Paths\": [\"/ipfs\", \"/ipns\"]\n }\n }\n }\n ```\n\nDefault: `false`\n\nType: `bool`\n\n> [!IMPORTANT]\n> See [Reverse Proxy Caveats](gateway.md#reverse-proxy) if running behind nginx or another reverse proxy.\n\n#### `Gateway.PublicGateways: NoDNSLink`\n\nA boolean to configure whether DNSLink for hostname present in `Host`\nHTTP header should be resolved. Overrides global setting.\nIf `Paths` are defined, they take priority over DNSLink.\n\nDefault: `false` (DNSLink lookup enabled by default for every defined hostname)\n\nType: `bool`\n\n> [!IMPORTANT]\n> See [Reverse Proxy Caveats](gateway.md#reverse-proxy) if running behind nginx or another reverse proxy.\n\n#### `Gateway.PublicGateways: InlineDNSLink`\n\nAn optional flag to explicitly configure whether subdomain gateway's redirects\n(enabled by `UseSubdomains: true`) should always inline a DNSLink name (FQDN)\ninto a single DNS label ([specification](https://specs.ipfs.tech/http-gateways/subdomain-gateway/#host-request-header)):\n\n```\n//example.com/ipns/example.net → HTTP 301 → //example-net.ipns.example.com\n```\n\nDNSLink name inlining allows for HTTPS on public subdomain gateways with single\nlabel wildcard TLS certs (also enabled when passing `X-Forwarded-Proto: https`),\nand provides disjoint Origin per root CID when special rules like\n, or a custom localhost logic in browsers like Brave\nhas to be applied.\n\nDefault: `false`\n\nType: `flag`\n\n#### `Gateway.PublicGateways: DeserializedResponses`\n\nAn optional flag to explicitly configure whether this gateway responds to deserialized\nrequests, or not. By default, it is enabled.\n\nWhen disabled, the gateway operates strictly as a [Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/).\n\n> [!TIP]\n> Disabling deserialized responses will protect you from acting as a free web hosting,\n> while still allowing trustless clients like [@helia/verified-fetch](https://www.npmjs.com/package/@helia/verified-fetch)\n> to utilize it for [trustless, verifiable data retrieval](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval).\n\nDefault: same as global `Gateway.DeserializedResponses`\n\nType: `flag`\n\n#### Implicit defaults of `Gateway.PublicGateways`\n\nDefault entries for `localhost` hostname and loopback IPs are always present.\nIf additional config is provided for those hostnames, it will be merged on top of implicit values:\n\n```json\n{\n \"Gateway\": {\n \"PublicGateways\": {\n \"localhost\": {\n \"Paths\": [\"/ipfs\", \"/ipns\"],\n \"UseSubdomains\": true\n }\n }\n }\n}\n```\n\nIt is also possible to remove a default by setting it to `null`.\n\nFor example, to disable subdomain gateway on `localhost`\nand make that hostname act the same as `127.0.0.1`:\n\n```console\nipfs config --json Gateway.PublicGateways '{\"localhost\": null }'\n```\n\n### `Gateway` recipes\n\nBelow is a list of the most common gateway setups.\n\n> [!IMPORTANT]\n> See [Reverse Proxy Caveats](gateway.md#reverse-proxy) if running behind nginx or another reverse proxy.\n\n- Public [subdomain gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://{cid}.ipfs.dweb.link` (each content root gets its own Origin)\n\n ```console\n $ ipfs config --json Gateway.PublicGateways '{\n \"dweb.link\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\"]\n }\n }'\n ```\n\n - **Performance:** Consider enabling `Routing.AcceleratedDHTClient=true` to improve content routing lookups. Separately, gateway operators should decide if the gateway node should also co-host and provide (announce) fetched content to the DHT. If providing content, enable `Provide.DHT.SweepEnabled=true` for efficient announcements. If announcements are still not fast enough, adjust `Provide.DHT.MaxWorkers`. For a read-only gateway that doesn't announce content, use `Provide.Enabled=false`.\n - **Backward-compatible:** this feature enables automatic redirects from content paths to subdomains:\n\n `http://dweb.link/ipfs/{cid}` → `http://{cid}.ipfs.dweb.link`\n\n - **X-Forwarded-Proto:** if you run Kubo behind a reverse proxy that provides TLS, make it add a `X-Forwarded-Proto: https` HTTP header to ensure users are redirected to `https://`, not `http://`. It will also ensure DNSLink names are inlined to fit in a single DNS label, so they work fine with a wildcard TLS cert ([details](https://github.com/ipfs/in-web-browsers/issues/169)). The NGINX directive is `proxy_set_header X-Forwarded-Proto \"https\";`.:\n\n `http://dweb.link/ipfs/{cid}` → `https://{cid}.ipfs.dweb.link`\n\n `http://dweb.link/ipns/your-dnslink.site.example.com` → `https://your--dnslink-site-example-com.ipfs.dweb.link`\n\n - **X-Forwarded-Host:** we also support `X-Forwarded-Host: example.com` if you want to override subdomain gateway host from the original request:\n\n `http://dweb.link/ipfs/{cid}` → `http://{cid}.ipfs.example.com`\n\n- Public [path gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#path-gateway) at `http://ipfs.io/ipfs/{cid}` (no Origin separation)\n\n ```console\n $ ipfs config --json Gateway.PublicGateways '{\n \"ipfs.io\": {\n \"UseSubdomains\": false,\n \"Paths\": [\"/ipfs\", \"/ipns\"]\n }\n }'\n ```\n\n - **Performance:** Consider enabling `Routing.AcceleratedDHTClient=true` to improve content routing lookups. When running an open, recursive gateway, decide if the gateway should also co-host and provide (announce) fetched content to the DHT. If providing content, enable `Provide.DHT.SweepEnabled=true` for efficient announcements. If announcements are still not fast enough, adjust `Provide.DHT.MaxWorkers`. For a read-only gateway that doesn't announce content, use `Provide.Enabled=false`.\n\n- Public [DNSLink](https://dnslink.io/) gateway resolving every hostname passed in `Host` header.\n\n ```console\n ipfs config --json Gateway.NoDNSLink false\n ```\n\n - Note that `NoDNSLink: false` is the default (it works out of the box unless set to `true` manually)\n\n- Hardened, site-specific [DNSLink gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#dnslink-gateway).\n\n Disable fetching of remote data (`NoFetch: true`) and resolving DNSLink at unknown hostnames (`NoDNSLink: true`).\n Then, enable DNSLink gateway only for the specific hostname (for which data\n is already present on the node), without exposing any content-addressing `Paths`:\n\n ```console\n $ ipfs config --json Gateway.NoFetch true\n $ ipfs config --json Gateway.NoDNSLink true\n $ ipfs config --json Gateway.PublicGateways '{\n \"en.wikipedia-on-ipfs.org\": {\n \"NoDNSLink\": false,\n \"Paths\": []\n }\n }'\n ```\n\n## `Identity`\n\n### `Identity.PeerID`\n\nThe unique PKI identity label for this configs peer. Set on init and never read,\nit's merely here for convenience. Ipfs will always generate the peerID from its\nkeypair at runtime.\n\nType: `string` (peer ID)\n\n### `Identity.PrivKey`\n\nThe base64 encoded protobuf describing (and containing) the node's private key.\n\nType: `string` (base64 encoded)\n\n## `Internal`\n\nThis section includes internal knobs for various subsystems to allow advanced users with big or private infrastructures to fine-tune some behaviors without the need to recompile Kubo.\n\n**Be aware that making informed change here requires in-depth knowledge and most users should leave these untouched. All knobs listed here are subject to breaking changes between versions.**\n\n### `Internal.Bitswap`\n\n`Internal.Bitswap` contains knobs for tuning bitswap resource utilization.\n\n> [!TIP]\n> For high level configuration see [`Bitswap`](#bitswap).\n\nThe knobs (below) document how their value should related to each other.\nWhether their values should be raised or lowered should be determined\nbased on the metrics `ipfs_bitswap_active_tasks`, `ipfs_bitswap_pending_tasks`,\n`ipfs_bitswap_pending_block_tasks` and `ipfs_bitswap_active_block_tasks`\nreported by bitswap.\n\nThese metrics can be accessed as the Prometheus endpoint at `{Addresses.API}/debug/metrics/prometheus` (default: `http://127.0.0.1:5001/debug/metrics/prometheus`)\n\nThe value of `ipfs_bitswap_active_tasks` is capped by `EngineTaskWorkerCount`.\n\nThe value of `ipfs_bitswap_pending_tasks` is generally capped by the knobs below,\nhowever its exact maximum value is hard to predict as it depends on task sizes\nas well as number of requesting peers. However, as a rule of thumb,\nduring healthy operation this value should oscillate around a \"typical\" low value\n(without hitting a plateau continuously).\n\nIf `ipfs_bitswap_pending_tasks` is growing while `ipfs_bitswap_active_tasks` is at its maximum then\nthe node has reached its resource limits and new requests are unable to be processed as quickly as they are coming in.\nRaising resource limits (using the knobs below) could help, assuming the hardware can support the new limits.\n\nThe value of `ipfs_bitswap_active_block_tasks` is capped by `EngineBlockstoreWorkerCount`.\n\nThe value of `ipfs_bitswap_pending_block_tasks` is indirectly capped by `ipfs_bitswap_active_tasks`, but can be hard to\npredict as it depends on the number of blocks involved in a peer task which can vary.\n\nIf the value of `ipfs_bitswap_pending_block_tasks` is observed to grow,\nwhile `ipfs_bitswap_active_block_tasks` is at its maximum, there is indication that the number of\navailable block tasks is creating a bottleneck (either due to high-latency block operations,\nor due to high number of block operations per bitswap peer task).\nIn such cases, try increasing the `EngineBlockstoreWorkerCount`.\nIf this adjustment still does not increase the throughput of the node, there might\nbe hardware limitations like I/O or CPU.\n\n#### `Internal.Bitswap.TaskWorkerCount`\n\nNumber of threads (goroutines) sending outgoing messages.\nThrottles the number of concurrent send operations.\n\nType: `optionalInteger` (thread count, `null` means default which is 8)\n\n#### `Internal.Bitswap.EngineBlockstoreWorkerCount`\n\nNumber of threads for blockstore operations.\nUsed to throttle the number of concurrent requests to the block store.\nThe optimal value can be informed by the metrics `ipfs_bitswap_pending_block_tasks` and `ipfs_bitswap_active_block_tasks`.\nThis would be a number that depends on your hardware (I/O and CPU).\n\nType: `optionalInteger` (thread count, `null` means default which is 128)\n\n#### `Internal.Bitswap.EngineTaskWorkerCount`\n\nNumber of worker threads used for preparing and packaging responses before they are sent out.\nThis number should generally be equal to `TaskWorkerCount`.\n\nType: `optionalInteger` (thread count, `null` means default which is 8)\n\n#### `Internal.Bitswap.MaxOutstandingBytesPerPeer`\n\nMaximum number of bytes (across all tasks) pending to be processed and sent to any individual peer.\nThis number controls fairness and can vary from 250Kb (very fair) to 10Mb (less fair, with more work\ndedicated to peers who ask for more). Values below 250Kb could cause thrashing.\nValues above 10Mb open the potential for aggressively-wanting peers to consume all resources and\ndeteriorate the quality provided to less aggressively-wanting peers.\n\nType: `optionalInteger` (byte count, `null` means default which is 1MB)\n\n#### `Internal.Bitswap.ProviderSearchDelay`\n\nThis parameter determines how long to wait before looking for providers outside of bitswap.\nOther routing systems like the Amino DHT are able to provide results in less than a second, so lowering\nthis number will allow faster peers lookups in some cases.\n\nType: `optionalDuration` (`null` means default which is 1s)\n\n#### `Internal.Bitswap.ProviderSearchMaxResults`\n\nMaximum number of providers bitswap client should aim at before it stops searching for new ones.\nSetting to 0 means unlimited.\n\nType: `optionalInteger` (`null` means default which is 10)\n\n#### `Internal.Bitswap.BroadcastControl`\n\n`Internal.Bitswap.BroadcastControl` contains settings for the bitswap client's broadcast control functionality.\n\nBroadcast control tries to reduce the number of bitswap broadcast messages sent to peers by choosing a subset of of the peers to send to. Peers are chosen based on whether they have previously responded indicating they have wanted blocks, as well as other configurable criteria. The settings here change how peers are selected as broadcast targets. Broadcast control can also be completely disabled to return bitswap to its previous behavior before broadcast control was introduced.\n\nEnabling broadcast control should generally reduce the number of broadcasts significantly without significantly degrading the ability to discover which peers have wanted blocks. However, if block discovery on your network relies sufficiently on broadcasts to discover peers that have wanted blocks, then adjusting the broadcast control configuration or disabling it altogether, may be helpful.\n\n##### `Internal.Bitswap.BroadcastControl.Enable`\n\nEnables or disables broadcast control functionality. Setting this to `false` disables broadcast reduction logic and restores the previous (Kubo < 0.36) broadcast behavior of sending broadcasts to all peers. When disabled, all other `Bitswap.BroadcastControl` configuration items are ignored.\n\nDefault: `true` (Enabled)\n\nType: `flag`\n\n##### `Internal.Bitswap.BroadcastControl.MaxPeers`\n\nSets a hard limit on the number of peers to send broadcasts to. A value of `0` means no broadcasts are sent. A value of `-1` means there is no limit.\n\nDefault: `0` (no limit)\n\nType: `optionalInteger` (non-negative, 0 means no limit)\n\n##### `Internal.Bitswap.BroadcastControl.LocalPeers`\n\nEnables or disables broadcast control for peers on the local network. Peers that have private or loopback addresses are considered to be on the local network. If this setting is `false`, than always broadcast to peers on the local network. If `true`, apply broadcast control to local peers.\n\nDefault: `false` (Always broadcast to peers on local network)\n\nType: `flag`\n\n##### `Internal.Bitswap.BroadcastControl.PeeredPeers`\n\nEnables or disables broadcast reduction for peers configured for peering. If `false`, than always broadcast to peers configured for peering. If `true`, apply broadcast reduction to peered peers.\n\nDefault: `false` (Always broadcast to peers configured for peering)\n\nType: `flag`\n\n##### `Internal.Bitswap.BroadcastControl.MaxRandomPeers`\n\nSets the number of peers to broadcast to anyway, even though broadcast control logic has determined that they are not broadcast targets. Setting this to a non-zero value ensures at least this number of random peers receives a broadcast. This may be helpful in cases where peers that are not receiving broadcasts my have wanted blocks.\n\nDefault: `0` (do not send broadcasts to peers not already targeted broadcast control)\n\nType: `optionalInteger` (non-negative, 0 means do not broadcast to any random peers)\n\n##### `Internal.Bitswap.BroadcastControl.SendToPendingPeers`\n\nEnables or disables sending broadcasts to any peers to which there is a pending message to send. When enabled, this sends broadcasts to many more peers, but does so in a way that does not increase the number of separate broadcast messages. There is still the increased cost of the recipients having to process and respond to the broadcasts.\n\nDefault: `false` (Do not send broadcasts to all peers for which there are pending messages)\n\nType: `flag`\n\n### `Internal.UnixFSShardingSizeThreshold`\n\n**MOVED:** see [`Import.UnixFSHAMTDirectorySizeThreshold`](#importunixfshamtdirectorysizethreshold)\n\n### `Internal.MFSNoFlushLimit`\n\nControls the maximum number of consecutive MFS operations allowed with `--flush=false`\nbefore requiring a manual flush. This prevents unbounded memory growth and ensures\ndata consistency when using deferred flushing with `ipfs files` commands.\n\nWhen the limit is reached, further operations will fail with an error message\ninstructing the user to run `ipfs files flush`, use `--flush=true`, or increase\nthis limit in the configuration.\n\n**Why operations fail instead of auto-flushing:** Automatic flushing once the limit\nis reached was considered but rejected because it can lead to data corruption issues\nthat are difficult to debug. When the system decides to flush without user knowledge, it can:\n\n- Create partial states that violate user expectations about atomicity\n- Interfere with concurrent operations in unexpected ways\n- Make debugging and recovery much harder when issues occur\n\nBy failing explicitly, users maintain control over when their data is persisted,\nallowing them to:\n\n- Batch related operations together before flushing\n- Handle errors predictably at natural transaction boundaries\n- Understand exactly when and why their data is written to disk\n\nIf you expect automatic flushing behavior, simply use the default `--flush=true`\n(or omit the flag entirely) instead of `--flush=false`.\n\n**⚠️ WARNING:** Increasing this limit or disabling it (setting to 0) can lead to:\n\n- **Out-of-memory errors (OOM)** - Each unflushed operation consumes memory\n- **Data loss** - If the daemon crashes before flushing, all unflushed changes are lost\n- **Degraded performance** - Large unflushed caches slow down MFS operations\n\nDefault: `256`\n\nType: `optionalInteger` (0 disables the limit, strongly discouraged)\n\n**Note:** This is an EXPERIMENTAL feature and may change or be removed in future releases.\nSee [#10842](https://github.com/ipfs/kubo/issues/10842) for more information.\n\n## `Ipns`\n\n### `Ipns.RepublishPeriod`\n\nA time duration specifying how frequently to republish ipns records to ensure\nthey stay fresh on the network.\n\nDefault: 4 hours.\n\nType: `interval` or an empty string for the default.\n\n### `Ipns.RecordLifetime`\n\nA time duration specifying the value to set on ipns records for their validity\nlifetime.\n\nDefault: 48 hours.\n\nType: `interval` or an empty string for the default.\n\n### `Ipns.ResolveCacheSize`\n\nThe number of entries to store in an LRU cache of resolved ipns entries. Entries\nwill be kept cached until their lifetime is expired.\n\nDefault: `128`\n\nType: `integer` (non-negative, 0 means the default)\n\n### `Ipns.MaxCacheTTL`\n\nMaximum duration for which entries are valid in the name system cache. Applied\nto everything under `/ipns/` namespace, allows you to cap\nthe [Time-To-Live (TTL)](https://specs.ipfs.tech/ipns/ipns-record/#ttl-uint64) of\n[IPNS Records](https://specs.ipfs.tech/ipns/ipns-record/)\nAND also DNSLink TXT records (when DoH-specific [`DNS.MaxCacheTTL`](https://github.com/ipfs/kubo/blob/master/docs/config.md#dnsmaxcachettl)\nis not set to a lower value).\n\nWhen `Ipns.MaxCacheTTL` is set, it defines the upper bound limit of how long a\n[IPNS Name](https://specs.ipfs.tech/ipns/ipns-record/#ipns-name) lookup result\nwill be cached and read from cache before checking for updates.\n\n**Examples:**\n\n- `\"1m\"` IPNS results are cached 1m or less (good compromise for system where\n faster updates are desired).\n- `\"0s\"` IPNS caching is effectively turned off (useful for testing, bad for production use)\n - **Note:** setting this to `0` will turn off TTL-based caching entirely.\n This is discouraged in production environments. It will make IPNS websites\n artificially slow because IPNS resolution results will expire as soon as\n they are retrieved, forcing expensive IPNS lookup to happen on every\n request. If you want near-real-time IPNS, set it to a low, but still\n sensible value, such as `1m`.\n\nDefault: No upper bound, [TTL from IPNS Record](https://specs.ipfs.tech/ipns/ipns-record/#ttl-uint64) (see `ipns name publish --help`) is always respected.\n\nType: `optionalDuration`\n\n### `Ipns.UsePubsub`\n\nEnables [IPNS over PubSub](https://specs.ipfs.tech/ipns/ipns-pubsub-router/) for publishing and resolving IPNS records in real time.\n\n**EXPERIMENTAL:** read about current limitations at [experimental-features.md#ipns-pubsub](./experimental-features.md#ipns-pubsub).\n\nDefault: `disabled`\n\nType: `flag`\n\n### `Ipns.DelegatedPublishers`\n\nHTTP endpoints for delegated IPNS publishing operations. These endpoints must support the [IPNS API](https://specs.ipfs.tech/routing/http-routing-v1/#ipns-api) from the Delegated Routing V1 HTTP specification.\n\nThe special value `\"auto\"` loads delegated publishers from [AutoConf](#autoconf) when enabled.\n\n**Publishing behavior depends on routing configuration:**\n\n- `Routing.Type=auto` (default): Uses DHT for publishing, `\"auto\"` resolves to empty list\n- `Routing.Type=delegated`: Uses HTTP delegated publishers only, `\"auto\"` resolves to configured endpoints\n\nWhen using `\"auto\"`, inspect the effective publishers with: `ipfs config Ipns.DelegatedPublishers --expand-auto`\n\n**Command flags override publishing behavior:**\n\n- `--allow-offline` - Publishes to local datastore without requiring network connectivity\n- `--allow-delegated` - Uses local datastore and HTTP delegated publishers only (no DHT connectivity required)\n\nFor self-hosting, you can run your own `/routing/v1/ipns` endpoint using [someguy](https://github.com/ipfs/someguy/).\n\nDefault: `[\"auto\"]`\n\nType: `array[string]` (URLs or `\"auto\"`)\n\n## `Migration`\n\n> [!WARNING]\n> **DEPRECATED:** Only applies to legacy migrations (repo versions <16). Modern repos (v16+) use embedded migrations.\n> This section is optional and will not appear in new configurations.\n\n### `Migration.DownloadSources`\n\n**DEPRECATED:** Download sources for legacy migrations. Only `\"HTTPS\"` is supported.\n\nType: `array[string]` (optional)\n\nDefault: `[\"HTTPS\"]`\n\n### `Migration.Keep`\n\n**DEPRECATED:** Controls retention of legacy migration binaries. Options: `\"cache\"` (default), `\"discard\"`, `\"keep\"`.\n\nType: `string` (optional)\n\nDefault: `\"cache\"`\n\n## `Mounts`\n\n> [!CAUTION]\n> **EXPERIMENTAL:**\n> This feature is disabled by default, requires an explicit opt-in with `ipfs mount` or `ipfs daemon --mount`.\n>\n> Read about current limitations at [fuse.md](./fuse.md).\n\nFUSE mount point configuration options.\n\n### `Mounts.IPFS`\n\nMountpoint for `/ipfs/`.\n\nDefault: `/ipfs`\n\nType: `string` (filesystem path)\n\n### `Mounts.IPNS`\n\nMountpoint for `/ipns/`.\n\nDefault: `/ipns`\n\nType: `string` (filesystem path)\n\n### `Mounts.MFS`\n\nMountpoint for Mutable File System (MFS) behind the `ipfs files` API.\n\n> [!CAUTION]\n>\n> - Write support is highly experimental and not recommended for mission-critical deployments.\n> - Avoid storing lazy-loaded datasets in MFS. Exposing a partially local, lazy-loaded DAG risks operating system search indexers crawling it, which may trigger unintended network prefetching of non-local DAG components.\n\nDefault: `/mfs`\n\nType: `string` (filesystem path)\n\n### `Mounts.FuseAllowOther`\n\nSets the 'FUSE allow-other' option on the mount point.\n\n## `Pinning`\n\nPinning configures the options available for pinning content\n(i.e. keeping content longer-term instead of as temporarily cached storage).\n\n### `Pinning.RemoteServices`\n\n`RemoteServices` maps a name for a remote pinning service to its configuration.\n\nA remote pinning service is a remote service that exposes an API for managing\nthat service's interest in long-term data storage.\n\nThe exposed API conforms to the specification defined at\n\n\n#### `Pinning.RemoteServices: API`\n\nContains information relevant to utilizing the remote pinning service\n\nExample:\n\n```json\n{\n \"Pinning\": {\n \"RemoteServices\": {\n \"myPinningService\": {\n \"API\" : {\n \"Endpoint\" : \"https://pinningservice.tld:1234/my/api/path\",\n \"Key\" : \"someOpaqueKey\"\n }\n }\n }\n }\n}\n```\n\n##### `Pinning.RemoteServices: API.Endpoint`\n\nThe HTTP(S) endpoint through which to access the pinning service\n\nExample: \"\"\n\nType: `string`\n\n##### `Pinning.RemoteServices: API.Key`\n\nThe key through which access to the pinning service is granted\n\nType: `string`\n\n#### `Pinning.RemoteServices: Policies`\n\nContains additional opt-in policies for the remote pinning service.\n\n##### `Pinning.RemoteServices: Policies.MFS`\n\nWhen this policy is enabled, it follows changes to MFS\nand updates the pin for MFS root on the configured remote service.\n\nA pin request to the remote service is sent only when MFS root CID has changed\nand enough time has passed since the previous request (determined by `RepinInterval`).\n\nOne can observe MFS pinning details by enabling debug via `ipfs log level remotepinning/mfs debug` and switching back to `error` when done.\n\n###### `Pinning.RemoteServices: Policies.MFS.Enabled`\n\nControls if this policy is active.\n\nDefault: `false`\n\nType: `bool`\n\n###### `Pinning.RemoteServices: Policies.MFS.PinName`\n\nOptional name to use for a remote pin that represents the MFS root CID.\nWhen left empty, a default name will be generated.\n\nDefault: `\"policy/{PeerID}/mfs\"`, e.g. `\"policy/12.../mfs\"`\n\nType: `string`\n\n###### `Pinning.RemoteServices: Policies.MFS.RepinInterval`\n\nDefines how often (at most) the pin request should be sent to the remote service.\nIf left empty, the default interval will be used. Values lower than `1m` will be ignored.\n\nDefault: `\"5m\"`\n\nType: `duration`\n\n## `Provide`\n\nConfigures how your node advertises content to make it discoverable by other\npeers.\n\n**What is providing?** When your node stores content, it publishes provider\nrecords to the routing system announcing \"I have this content\". These records\nmap CIDs to your peer ID, enabling content discovery across the network.\n\nWhile designed to support multiple routing systems in the future, the current\ndefault configuration only supports [providing to the Amino DHT](#providedht).\n\n### `Provide.Enabled`\n\nControls whether Kubo provide and reprovide systems are enabled.\n\n> [!CAUTION]\n> Disabling this will prevent other nodes from discovering your content.\n> Your node will stop announcing data to the routing system, making it\n> inaccessible unless peers connect to you directly.\n\nDefault: `true`\n\nType: `flag`\n\n### `Provide.Strategy`\n\nTells the provide system what should be announced. Valid strategies are:\n\n- `\"all\"` - announce all CIDs of stored blocks\n- `\"pinned\"` - only announce recursively pinned CIDs (`ipfs pin add -r`, both roots and child blocks)\n - Order: root blocks of direct and recursive pins are announced first, then the child blocks of recursive pins\n- `\"roots\"` - only announce the root block of explicitly pinned CIDs (`ipfs pin add`)\n - **⚠️ BE CAREFUL:** node with `roots` strategy will not announce child blocks.\n It makes sense only for use cases where the entire DAG is fetched in full,\n and a graceful resume does not have to be guaranteed: the lack of child\n announcements means an interrupted retrieval won't be able to find\n providers for the missing block in the middle of a file, unless the peer\n happens to already be connected to a provider and asks for child CID over\n bitswap.\n- `\"mfs\"` - announce only the local CIDs that are part of the MFS (`ipfs files`)\n - Note: MFS is lazy-loaded. Only the MFS blocks present in local datastore are announced.\n- `\"pinned+mfs\"` - a combination of the `pinned` and `mfs` strategies.\n - **ℹ️ NOTE:** This is the suggested strategy for users who run without GC and don't want to provide everything in cache.\n - Order: first `pinned` and then the locally available part of `mfs`.\n\n**Strategy changes automatically clear the provide queue.** When you change `Provide.Strategy` and restart Kubo, the provide queue is automatically cleared to ensure only content matching your new strategy is announced. You can also manually clear the queue using `ipfs provide clear`.\n\n**Memory requirements:**\n\n- Reproviding larger pinsets using the `mfs`, `pinned`, `pinned+mfs` or `roots` strategies requires additional memory, with an estimated ~1 GiB of RAM per 20 million CIDs for reproviding to the Amino DHT.\n- This is due to the use of a buffered provider, which loads all CIDs into memory to avoid holding a lock on the entire pinset during the reprovide cycle.\n\nDefault: `\"all\"`\n\nType: `optionalString` (unset for the default)\n\n### `Provide.DHT`\n\nConfiguration for providing data to Amino DHT peers.\n\n**Provider record lifecycle:** On the Amino DHT, provider records expire after\n[`amino.DefaultProvideValidity`](https://github.com/libp2p/go-libp2p-kad-dht/blob/v0.34.0/amino/defaults.go#L40-L43).\nYour node must re-announce (reprovide) content periodically to keep it\ndiscoverable. The [`Provide.DHT.Interval`](#providedhtinterval) setting\ncontrols this timing, with the default ensuring records refresh well before\nexpiration or negative churn effects kick in.\n\n**Two provider systems:**\n\n- **Sweep provider**: Divides the DHT keyspace into regions and systematically\n sweeps through them over the reprovide interval. This batches CIDs allocated\n to the same DHT servers, dramatically reducing the number of DHT lookups and\n PUTs needed. Spreads work evenly over time with predictable resource usage.\n\n- **Legacy provider**: Processes each CID individually with separate DHT\n lookups. Works well for small content collections but struggles to complete\n reprovide cycles when managing thousands of CIDs.\n\n#### Monitoring Provide Operations\n\n**Quick command-line monitoring:** Use `ipfs provide stat` to view the current\nstate of the provider system. For real-time monitoring, run\n`watch ipfs provide stat --all --compact` to see detailed statistics refreshed\ncontinuously in a 2-column layout.\n\n**Long-term monitoring:** For in-depth or long-term monitoring, metrics are\nexposed at the Prometheus endpoint: `{Addresses.API}/debug/metrics/prometheus`\n(default: `http://127.0.0.1:5001/debug/metrics/prometheus`). Different metrics\nare available depending on whether you use legacy mode (`SweepEnabled=false`) or\nsweep mode (`SweepEnabled=true`). See [Provide metrics documentation](https://github.com/ipfs/kubo/blob/master/docs/metrics.md#provide)\nfor details.\n\n**Debug logging:** For troubleshooting, enable detailed logging by setting:\n\n```sh\nGOLOG_LOG_LEVEL=error,provider=debug,dht/provider=debug\n```\n\n- `provider=debug` enables generic logging (legacy provider and any non-dht operations)\n- `dht/provider=debug` enables logging for the sweep provider\n\n#### `Provide.DHT.Interval`\n\nSets how often to re-announce content to the DHT. Provider records on Amino DHT\nexpire after [`amino.DefaultProvideValidity`](https://github.com/libp2p/go-libp2p-kad-dht/blob/v0.34.0/amino/defaults.go#L40-L43).\n\n**Why this matters:** The interval must be shorter than the expiration window to\nensure provider records refresh before they expire. The default value is\napproximately half of [`amino.DefaultProvideValidity`](https://github.com/libp2p/go-libp2p-kad-dht/blob/v0.34.0/amino/defaults.go#L40-L43),\nwhich accounts for network churn and ensures records stay alive without\noverwhelming the network with unnecessary announcements.\n\n**With sweep mode enabled\n([`Provide.DHT.SweepEnabled`](#providedhtsweepenabled)):** The system spreads\nreprovide operations smoothly across this entire interval. Each keyspace region\nis reprovided at scheduled times throughout the period, ensuring each region's\nannouncements complete before records expire.\n\n**With legacy mode:** The system attempts to reprovide all CIDs as quickly as\npossible at the start of each interval. If reproviding takes longer than this\ninterval (common with large datasets), the next cycle is skipped and provider\nrecords may expire.\n\n- If unset, it uses the implicit safe default.\n- If set to the value `\"0\"` it will disable content reproviding to DHT.\n\n> [!CAUTION]\n> Disabling this will prevent other nodes from discovering your content via the DHT.\n> Your node will stop announcing data to the DHT, making it\n> inaccessible unless peers connect to you directly. Since provider\n> records expire after `amino.DefaultProvideValidity`, your content will become undiscoverable\n> after this period.\n\nDefault: `22h`\n\nType: `optionalDuration` (unset for the default)\n\n#### `Provide.DHT.MaxWorkers`\n\nSets the maximum number of _concurrent_ DHT provide operations.\n\n**When `Provide.DHT.SweepEnabled` is false (legacy mode):**\n\n- Controls NEW CID announcements only\n- Reprovide operations do **not** count against this limit\n- A value of `0` allows unlimited provide workers\n\n**When `Provide.DHT.SweepEnabled` is true:**\n\n- Controls the total worker pool for both provide and reprovide operations\n- Workers are split between periodic reprovides and burst provides\n- Use a positive value to control resource usage\n- See [`DedicatedPeriodicWorkers`](#providedhtdedicatedperiodicworkers) and [`DedicatedBurstWorkers`](#providedhtdedicatedburstworkers) for task allocation\n\nIf the [accelerated DHT client](#routingaccelerateddhtclient) is enabled, each\nprovide operation opens ~20 connections in parallel. With the standard DHT\nclient (accelerated disabled), each provide opens between 20 and 60\nconnections, with at most 10 active at once. Provides complete more quickly\nwhen using the accelerated client. Be mindful of how many simultaneous\nconnections this setting can generate.\n\n> [!CAUTION]\n> For nodes without strict connection limits that need to provide large volumes\n> of content, we recommend first trying `Provide.DHT.SweepEnabled=true` for efficient\n> announcements. If announcements are still not fast enough, adjust `Provide.DHT.MaxWorkers`.\n> As a last resort, consider enabling `Routing.AcceleratedDHTClient=true` but be aware that it is very resource hungry.\n>\n> At the same time, mind that raising this value too high may lead to increased load.\n> Proceed with caution, ensure proper hardware and networking are in place.\n\n> [!TIP]\n> **When `SweepEnabled` is true:** Users providing millions of CIDs or more\n> should increase the worker count accordingly. Underprovisioning can lead to\n> slow provides (burst workers) and inability to keep up with content\n> reproviding (periodic workers). For nodes with sufficient resources (CPU,\n> bandwidth, number of connections), dedicating `1024` for [periodic\n> workers](#providedhtdedicatedperiodicworkers) and `512` for [burst\n> workers](#providedhtdedicatedburstworkers), and `2048` [max\n> workers](#providedhtmaxworkers) should be adequate even for the largest\n> users. The system will only use workers as needed - unused resources won't be\n> consumed. Ensure you adjust the swarm [connection manager](#swarmconnmgr) and\n> [resource manager](#swarmresourcemgr) configuration accordingly.\n> See [Capacity Planning](https://github.com/ipfs/kubo/blob/master/docs/provide-stats.md#capacity-planning) for more details.\n\nDefault: `16`\n\nType: `optionalInteger` (non-negative; `0` means unlimited number of workers)\n\n#### `Provide.DHT.SweepEnabled`\n\nEnables the sweep provider for efficient content announcements. When disabled,\nthe legacy [`boxo/provider`](https://github.com/ipfs/boxo/tree/main/provider) is\nused instead.\n\n**The legacy provider problem:** The legacy system processes CIDs one at a\ntime, requiring a separate DHT lookup (10-20 seconds each) to find the 20\nclosest peers for each CID. This sequential approach typically handles less\nthan 10,000 CID over 22h ([`Provide.DHT.Interval`](#providedhtinterval)). If\nyour node has more CIDs than can be reprovided within\n[`Provide.DHT.Interval`](#providedhtinterval), provider records start expiring\nafter\n[`amino.DefaultProvideValidity`](https://github.com/libp2p/go-libp2p-kad-dht/blob/v0.34.0/amino/defaults.go#L40-L43),\nmaking content undiscoverable.\n\n**How sweep mode works:** The sweep provider divides the DHT keyspace into\nregions based on keyspace prefixes. It estimates the Amino DHT size, calculates\nhow many regions are needed (sized to contain at least 20 peers each), then\nschedules region processing evenly across\n[`Provide.DHT.Interval`](#providedhtinterval). When processing a region, it\ndiscovers the peers in that region once, then sends all provider records for\nCIDs allocated to those peers in a batch. This batching is the key efficiency:\ninstead of N lookups for N CIDs, the number of lookups is bounded by a constant\nfraction of the Amino DHT size (e.g., ~3,000 lookups when there are ~10,000 DHT\nservers), regardless of how many CIDs you're providing.\n\n**Efficiency gains:** For a node providing 100,000 CIDs, sweep mode reduces\nlookups by 97% compared to legacy. The work spreads smoothly over time rather\nthan completing in bursts, preventing resource spikes and duplicate\nannouncements. Long-running nodes reprovide systematically just before records\nwould expire, keeping content continuously discoverable without wasting\nbandwidth.\n\n**Implementation details:** The sweep provider tracks CIDs in a persistent\nkeystore. New content added via `StartProviding()` enters the provide queue and\ngets batched by keyspace region. The keystore is periodically refreshed at each\n[`Provide.DHT.Interval`](#providedhtinterval) with CIDs matching\n[`Provide.Strategy`](#providestrategy) to ensure only current content remains\nscheduled. This handles cases where content is unpinned or removed.\n\n**Persistent reprovide cycle state:** When Provide Sweep is enabled, the\nreprovide cycle state is persisted to the datastore by default. On restart, Kubo\nautomatically resumes from where it left off. If the node was offline for an\nextended period, all CIDs that haven't been reprovided within the configured\n[`Provide.DHT.Interval`](#providedhtinterval) are immediately queued for\nreproviding. Additionally, the provide queue is persisted on shutdown and\nrestored on startup, ensuring no pending provide operations are lost. If you\ndon't want to keep the persisted provider state from a previous run, you can\ndisable this behavior by setting [`Provide.DHT.ResumeEnabled`](#providedhtresumeenabled)\nto `false`.\n\n> \n> \n> \n> \"Reprovide\n> \n>\n> The diagram compares performance patterns:\n>\n> - **Legacy mode**: Sequential processing, one lookup per CID, struggles with large datasets\n> - **Sweep mode**: Smooth distribution over time, batched lookups by keyspace region, predictable resource usage\n> - **Accelerated DHT**: Hourly network crawls creating traffic spikes, high resource usage\n>\n> Sweep mode achieves similar effectiveness to the Accelerated DHT client but with steady resource consumption.\n\nFor background on the sweep provider design and motivations, see Shipyard's blogpost [Provide Sweep: Solving the DHT Provide Bottleneck](https://ipshipyard.com/blog/2025-dht-provide-sweep/).\n\nYou can compare the effectiveness of sweep mode vs legacy mode by monitoring the appropriate metrics (see [Monitoring Provide Operations](#monitoring-provide-operations) above).\n\n> [!NOTE]\n> This is the default provider system as of Kubo v0.39. To use the legacy provider instead, set `Provide.DHT.SweepEnabled=false`.\n\n> [!NOTE]\n> When DHT routing is unavailable (e.g., `Routing.Type=custom` with only HTTP routers), the provider automatically falls back to the legacy provider regardless of this setting.\n\nDefault: `true`\n\nType: `flag`\n\n#### `Provide.DHT.ResumeEnabled`\n\nControls whether the provider resumes from its previous state on restart. Only\napplies when `Provide.DHT.SweepEnabled` is true.\n\nWhen enabled (the default), the provider persists its reprovide cycle state and\nprovide queue to the datastore, and restores them on restart. This ensures:\n\n- The reprovide cycle continues from where it left off instead of starting over\n- Any CIDs in the provide queue during shutdown are restored and provided after\nrestart\n- CIDs that missed their reprovide window while the node was offline are queued\nfor immediate reproviding\n\nWhen disabled, the provider starts fresh on each restart, discarding any\nprevious reprovide cycle state and provide queue. On a fresh start, all CIDs\nmatching the [`Provide.Strategy`](#providestrategy) will be provided ASAP (as\nburst provides), and then keyspace regions are reprovided according to the\nregular schedule starting from the beginning of the reprovide cycle.\n\n> [!NOTE]\n> Disabling this option means the provider will provide all content matching\n> your strategy on every restart (which can be resource-intensive for large\n> datasets), then start from the beginning of the reprovide cycle. For nodes\n> with large datasets or frequent restarts, keeping this enabled (the default)\n> is recommended for better resource efficiency and more consistent reproviding\n> behavior.\n\nDefault: `true`\n\nType: `flag`\n\n#### `Provide.DHT.DedicatedPeriodicWorkers`\n\nNumber of workers dedicated to periodic keyspace region reprovides. Only\napplies when `Provide.DHT.SweepEnabled` is true.\n\nAmong the [`Provide.DHT.MaxWorkers`](#providedhtmaxworkers), this\nnumber of workers will be dedicated to the periodic region reprovide only. The sum of\n`DedicatedPeriodicWorkers` and `DedicatedBurstWorkers` should not exceed `MaxWorkers`.\nAny remaining workers (MaxWorkers - DedicatedPeriodicWorkers - DedicatedBurstWorkers)\nform a shared pool that can be used for either type of work as needed.\n\n> [!NOTE]\n> If the provider system isn't able to keep up with reproviding all your\n> content within the [Provide.DHT.Interval](#providedhtinterval), consider\n> increasing this value.\n\nDefault: `2`\n\nType: `optionalInteger` (`0` means there are no dedicated workers, but the\noperation can be performed by free non-dedicated workers)\n\n#### `Provide.DHT.DedicatedBurstWorkers`\n\nNumber of workers dedicated to burst provides. Only applies when `Provide.DHT.SweepEnabled` is true.\n\nBurst provides are triggered by:\n\n- Manual provide commands (`ipfs routing provide`)\n- New content matching your `Provide.Strategy` (blocks from `ipfs add`, bitswap, or trustless gateway requests)\n- Catch-up reprovides after being disconnected/offline for a while\n\nHaving dedicated burst workers ensures that bulk operations (like adding many CIDs\nor reconnecting to the network) don't delay regular periodic reprovides, and vice versa.\n\nAmong the [`Provide.DHT.MaxWorkers`](#providedhtmaxworkers), this\nnumber of workers will be dedicated to burst provides only. In addition to\nthese, if there are available workers in the pool, they can also be used for\nburst provides.\n\n> [!NOTE]\n> If CIDs aren't provided quickly enough to your taste, and you can afford more\n> CPU and bandwidth, consider increasing this value.\n\nDefault: `1`\n\nType: `optionalInteger` (`0` means there are no dedicated workers, but the\noperation can be performed by free non-dedicated workers)\n\n#### `Provide.DHT.MaxProvideConnsPerWorker`\n\nMaximum number of connections that a single worker can use to send provider\nrecords over the network.\n\nWhen reproviding CIDs corresponding to a keyspace region, the reprovider must\nsend a provider record to the 20 closest peers to the CID (in XOR distance) for\neach CID belonging to this keyspace region.\n\nThe reprovider opens a connection to a peer from that region, sends it all its\nallocated provider records. Once done, it opens a connection to the next peer\nfrom that keyspace region until all provider records are assigned.\n\nThis option defines how many such connections can be open concurrently by a\nsingle worker.\n\n> [!NOTE]\n> Increasing this value can speed up the provide operation, at the cost of\n> opening more simultaneous connections to DHT servers. A keyspace typically\n> has less than 60 peers, so you may hit a performance ceiling beyond which\n> increasing this value has no effect.\n\nDefault: `20`\n\nType: `optionalInteger` (non-negative)\n\n#### `Provide.DHT.KeystoreBatchSize`\n\nDuring the garbage collection, all keys stored in the Keystore are removed, and\nthe keys are streamed from a channel to fill the Keystore again with up-to-date\nkeys. Since a high number of CIDs to reprovide can easily fill up the memory,\nkeys are read and written in batches to optimize for memory usage.\n\nThis option defines how many multihashes should be contained within a batch. A\nmultihash is usually represented by 34 bytes.\n\nDefault: `16384` (~544 KiB per batch)\n\nType: `optionalInteger` (non-negative)\n\n#### `Provide.DHT.OfflineDelay`\n\nThe `SweepingProvider` has 3 states: `ONLINE`, `DISCONNECTED` and `OFFLINE`. It\nstarts `OFFLINE`, and as the node bootstraps, it changes its state to `ONLINE`.\n\nWhen the provider loses connection to all DHT peers, it switches to the\n`DISCONNECTED` state. In this state, new provides will be added to the provide\nqueue, and provided as soon as the node comes back online.\n\nAfter a node has been `DISCONNECTED` for `OfflineDelay`, it goes to `OFFLINE`\nstate. When `OFFLINE`, the provider drops the provide queue, and returns errors\nto new provide requests. However, when `OFFLINE` the provider still adds the\nkeys to its state, so keys will eventually be provided in the\n[`Provide.DHT.Interval`](#providedhtinterval) after the provider comes back\n`ONLINE`.\n\nDefault: `2h`\n\nType: `optionalDuration`\n\n## `Provider`\n\n### `Provider.Enabled`\n\n**REMOVED**\n\nReplaced with [`Provide.Enabled`](#provideenabled).\n\n### `Provider.Strategy`\n\n**REMOVED**\n\nThis field was unused. Use [`Provide.Strategy`](#providestrategy) instead.\n\n### `Provider.WorkerCount`\n\n**REMOVED**\n\nReplaced with [`Provide.DHT.MaxWorkers`](#providedhtmaxworkers).\n\n## `Pubsub`\n\nPubsub configures Kubo's opt-in, opinionated [libp2p pubsub](https://web.archive.org/web/20260116065034/https://docs.libp2p.io/concepts/pubsub/overview/) instance.\nTo enable, set `Pubsub.Enabled` to `true`.\n\n**EXPERIMENTAL:** This is an opt-in feature. Its primary use case is\n[IPNS over PubSub](https://specs.ipfs.tech/ipns/ipns-pubsub-router/), which\nenables real-time IPNS record propagation. See [`Ipns.UsePubsub`](#ipnsusepubsub)\nfor details.\n\nThe `ipfs pubsub` commands can also be used for basic publish/subscribe\noperations, but only if Kubo's built-in message validation (described below) is\nacceptable for your use case.\n\n### When to use a dedicated pubsub node\n\nKubo's pubsub is optimized for IPNS. It uses opinionated message validation\nthat may not fit all applications. If you need custom Message ID computation,\ndifferent deduplication logic, or validation rules beyond what Kubo provides,\nconsider building a dedicated pubsub node using\n[go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub) directly.\n\n### Message deduplication\n\nKubo uses two layers of message deduplication to handle duplicate messages that\nmay arrive via different network paths:\n\n**Layer 1: In-memory TimeCache (Message ID)**\n\nWhen a message arrives, Kubo computes its Message ID (hash of the message\ncontent) and checks an in-memory cache. If the ID was seen recently, the\nmessage is dropped. This cache is controlled by:\n\n- [`Pubsub.SeenMessagesTTL`](#pubsubseenmessagesttl) - how long Message IDs are remembered (default: 120s)\n- [`Pubsub.SeenMessagesStrategy`](#pubsubseenmessagesstrategy) - whether TTL resets on each sighting\n\nThis cache is fast but limited: it only works within the TTL window and is\ncleared on node restart.\n\n**Layer 2: Persistent Seqno Validator (per-peer)**\n\nFor stronger deduplication, Kubo tracks the maximum sequence number seen from\neach peer and persists it to the datastore. Messages with sequence numbers\nlower than the recorded maximum are rejected. This prevents replay attacks and\nhandles message cycles in large networks where messages may take longer than\nthe TimeCache TTL to propagate.\n\nThis layer survives node restarts. The state can be inspected or cleared using\n`ipfs pubsub reset` (for testing/recovery only).\n\n### `Pubsub.Enabled`\n\nEnables the pubsub system.\n\nDefault: `false`\n\nType: `flag`\n\n### `Pubsub.Router`\n\nSets the default router used by pubsub to route messages to peers. This can be one of:\n\n- `\"floodsub\"` - floodsub is a basic router that simply _floods_ messages to all\n connected peers. This router is extremely inefficient but _very_ reliable.\n- `\"gossipsub\"` - [gossipsub][] is a more advanced routing algorithm that will\n build an overlay mesh from a subset of the links in the network.\n\nDefault: `\"gossipsub\"`\n\nType: `string` (one of `\"floodsub\"`, `\"gossipsub\"`, or `\"\"` (apply default))\n\n[gossipsub]: https://github.com/libp2p/specs/tree/master/pubsub/gossipsub\n\n### `Pubsub.DisableSigning`\n\nDisables message signing and signature verification.\n\n**FOR TESTING ONLY - DO NOT USE IN PRODUCTION**\n\nIt is _not_ safe to disable signing even if you don't care _who_ sent the\nmessage because spoofed messages can be used to silence real messages by\nintentionally re-using the real message's message ID.\n\nDefault: `false`\n\nType: `bool`\n\n### `Pubsub.SeenMessagesTTL`\n\nControls the time window for the in-memory Message ID cache (Layer 1\ndeduplication). Messages with the same ID seen within this window are dropped.\n\nA smaller value reduces memory usage but may cause more duplicates in networks\nwith slow nodes. A larger value uses more memory but provides better duplicate\ndetection within the time window.\n\nDefault: see `TimeCacheDuration` from [go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub)\n\nType: `optionalDuration`\n\n### `Pubsub.SeenMessagesStrategy`\n\nDetermines how the TTL countdown for the Message ID cache works.\n\n- `last-seen` - Sliding window: TTL resets each time the message is seen again.\n Keeps frequently-seen messages in cache longer, preventing continued propagation.\n- `first-seen` - Fixed window: TTL counts from first sighting only. Messages are\n purged after the TTL regardless of how many times they're seen.\n\nDefault: `last-seen` (see [go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub))\n\nType: `optionalString`\n\n## `Peering`\n\nConfigures the peering subsystem. The peering subsystem configures Kubo to\nconnect to, remain connected to, and reconnect to a set of nodes. Nodes should\nuse this subsystem to create \"sticky\" links between frequently useful peers to\nimprove reliability.\n\nUse-cases:\n\n- An IPFS gateway connected to an IPFS cluster should peer to ensure that the\n gateway can always fetch content from the cluster.\n- A dapp may peer embedded Kubo nodes with a set of pinning services or\n textile cafes/hubs.\n- A set of friends may peer to ensure that they can always fetch each other's\n content.\n\nWhen a node is added to the set of peered nodes, Kubo will:\n\n1. Protect connections to this node from the connection manager. That is,\n Kubo will never automatically close the connection to this node and\n connections to this node will not count towards the connection limit.\n2. Connect to this node on startup.\n3. Repeatedly try to reconnect to this node if the last connection dies or the\n node goes offline. This repeated re-connect logic is governed by a randomized\n exponential backoff delay ranging from ~5 seconds to ~10 minutes to avoid\n repeatedly reconnect to a node that's offline.\n\nPeering can be asymmetric or symmetric:\n\n- When symmetric, the connection will be protected by both nodes and will likely\n be very stable.\n- When asymmetric, only one node (the node that configured peering) will protect\n the connection and attempt to re-connect to the peered node on disconnect. If\n the peered node is under heavy load and/or has a low connection limit, the\n connection may flap repeatedly. Be careful when asymmetrically peering to not\n overload peers.\n\n### `Peering.Peers`\n\nThe set of peers with which to peer.\n\n```json\n{\n \"Peering\": {\n \"Peers\": [\n {\n \"ID\": \"QmPeerID1\",\n \"Addrs\": [\"/ip4/18.1.1.1/tcp/4001\"]\n },\n {\n \"ID\": \"QmPeerID2\",\n \"Addrs\": [\"/ip4/18.1.1.2/tcp/4001\", \"/ip4/18.1.1.2/udp/4001/quic-v1\"]\n }\n ]\n }\n ...\n}\n```\n\nWhere `ID` is the peer ID and `Addrs` is a set of known addresses for the peer. If no addresses are specified, the Amino DHT will be queried.\n\nAdditional fields may be added in the future.\n\nDefault: empty.\n\nType: `array[peering]`\n\n## `Reprovider`\n\n### `Reprovider.Interval`\n\n**REMOVED**\n\nReplaced with [`Provide.DHT.Interval`](#providedhtinterval).\n\n### `Reprovider.Strategy`\n\n**REMOVED**\n\nReplaced with [`Provide.Strategy`](#providestrategy).\n\n## `Routing`\n\nContains options for content, peer, and IPNS routing mechanisms.\n\n### `Routing.Type`\n\nControls how your node discovers content and peers on the network.\n\n**Production options:**\n\n- **`auto`** (default): Uses both the public IPFS DHT (Amino) and HTTP routers\n from [`Routing.DelegatedRouters`](#routingdelegatedrouters) for faster lookups.\n Your node starts as a DHT client and automatically switches to server mode\n when reachable from the public internet.\n\n- **`autoclient`**: Same as `auto`, but never runs a DHT server.\n Use this if your node is behind a firewall or NAT.\n\n- **`dht`**: Uses only the Amino DHT (no HTTP routers). Automatically switches\n between client and server mode based on reachability.\n\n- **`dhtclient`**: DHT-only, always running as a client. Lower resource usage.\n\n- **`dhtserver`**: DHT-only, always running as a server.\n Only use this if your node is reachable from the public internet.\n\n- **`none`**: Disables all routing. You must manually connect to peers.\n\n**About DHT client vs server mode:**\nWhen the DHT is enabled, your node can operate as either a client or server.\nIn server mode, it queries other peers and responds to their queries - this helps\nthe network but uses more resources. In client mode, it only queries others without\nresponding, which is less resource-intensive. With `auto` or `dht`, your node starts\nas a client and switches to server when it detects public reachability.\n\n> [!CAUTION]\n> **`Routing.Type` Experimental options:**\n>\n> These modes are for research and testing only, not production use.\n> They may change without notice between releases.\n>\n> - **`delegated`**: Uses only HTTP routers from [`Routing.DelegatedRouters`](#routingdelegatedrouters)\n> and IPNS publishers from [`Ipns.DelegatedPublishers`](#ipnsdelegatedpublishers),\n> without initializing the DHT. Useful when peer-to-peer connectivity is unavailable.\n> Note: cannot provide content to the network (no DHT means no provider records).\n>\n> - **`custom`**: Disables all default routers. You define your own routing in\n> [`Routing.Routers`](#routingrouters). See [delegated-routing.md](delegated-routing.md).\n\nDefault: `auto`\n\nType: `optionalString` (`null`/missing means the default)\n\n### `Routing.DelegatedRouters`\n\nAn array of URL hostnames for delegated routers to be queried in addition to the Amino DHT when `Routing.Type` is set to `auto` (default) or `autoclient`.\nThese endpoints must support the [Delegated Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/).\n\nThe special value `\"auto\"` uses delegated routers from [AutoConf](#autoconf) when enabled.\nYou can combine `\"auto\"` with custom URLs (e.g., `[\"auto\", \"https://custom.example.com\"]`) to query both the default delegated routers and your own endpoints. The first `\"auto\"` entry gets substituted with autoconf values, and other URLs are preserved.\n\n> [!TIP]\n> Delegated routing allows IPFS implementations to offload tasks like content routing, peer routing, and naming to a separate process or server while also benefiting from HTTP caching.\n>\n> One can run their own delegated router either by implementing the [Delegated Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) themselves, or by using [Someguy](https://github.com/ipfs/someguy), a turn-key implementation that proxies requests to other routing systems. A public utility instance of Someguy is hosted at [`https://delegated-ipfs.dev`](https://docs.ipfs.tech/concepts/public-utilities/#delegated-routing).\n\nDefault: `[\"auto\"]`\n\nType: `array[string]` (URLs or `\"auto\"`)\n\n### `Routing.AcceleratedDHTClient`\n\nThis alternative Amino DHT client with a Full-Routing-Table strategy will\ndo a complete scan of the DHT every hour and record all nodes found.\nThen when a lookup is tried instead of having to go through multiple Kad hops it\nis able to find the 20 final nodes by looking up the in-memory recorded network table.\n\nThis means sustained higher memory to store the routing table\nand extra CPU and network bandwidth for each network scan.\nHowever the latency of individual read/write operations should be ~10x faster\nand provide throughput up to 6 million times faster on larger datasets!\n\nThis is not compatible with `Routing.Type` `custom`. If you are using composable routers\nyou can configure this individually on each router.\n\nWhen it is enabled:\n\n- Client DHT operations (reads and writes) should complete much faster\n- The provider will now use a keyspace sweeping mode allowing to keep alive\n CID sets that are multiple orders of magnitude larger.\n - **Note:** For improved provide/reprovide operations specifically, consider using\n [`Provide.DHT.SweepEnabled`](#providedhtsweepenabled) instead, which offers similar\n benefits without the hourly traffic spikes.\n - The standard Bucket-Routing-Table DHT will still run for the DHT server (if\n the DHT server is enabled). This means the classical routing table will\n still be used to answer other nodes.\n This is critical to maintain to not harm the network.\n- The operations `ipfs stats dht` will default to showing information about the accelerated DHT client\n\n> [!CAUTION]\n> **`Routing.AcceleratedDHTClient` Caveats:**\n>\n> 1. Running the accelerated client likely will result in more resource consumption (connections, RAM, CPU, bandwidth)\n> - Users that are limited in the number of parallel connections their machines/networks can perform will be most affected\n> - The resource usage is not smooth as the client crawls the network in rounds and reproviding is similarly done in rounds\n> - Users who previously had a lot of content but were unable to advertise it on the network will see an increase in\n> egress bandwidth as their nodes start to advertise all of their CIDs into the network. If you have lots of data\n> entering your node that you don't want to advertise, consider using [`Provide.*`](#provide) configuration\n> to control which CIDs are reprovided.\n> 2. Currently, the DHT is not usable for queries for the first 5-10 minutes of operation as the routing table is being\n> prepared. This means operations like searching the DHT for particular peers or content will not work initially.\n> - You can see if the DHT has been initially populated by running `ipfs stats dht`\n> 3. Currently, the accelerated DHT client is not compatible with LAN-based DHTs and will not perform operations against\n> them.\n\nDefault: `false`\n\nType: `flag`\n\n### `Routing.LoopbackAddressesOnLanDHT`\n\n**EXPERIMENTAL: `Routing.LoopbackAddressesOnLanDHT` configuration may change in future release**\n\nWhether loopback addresses (e.g. 127.0.0.1) should not be ignored on the local LAN DHT.\n\nMost users do not need this setting. It can be useful during testing, when multiple Kubo nodes run on the same machine but some of them do not have `Discovery.MDNS.Enabled`.\n\nDefault: `false`\n\nType: `bool` (missing means `false`)\n\n### `Routing.IgnoreProviders`\n\nAn array of [string-encoded PeerIDs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#string-representation). Any provider record associated to one of these peer IDs is ignored.\n\nApart from ignoring specific providers for reasons like misbehaviour etc. this\nsetting is useful to ignore providers as a way to indicate preference, when the same provider\nis found under different peerIDs (i.e. one for HTTP and one for Bitswap retrieval).\n\n> [!TIP]\n> This denylist operates on PeerIDs.\n> To deny specific HTTP Provider URL, use [`HTTPRetrieval.Denylist`](#httpretrievaldenylist) instead.\n\nDefault: `[]`\n\nType: `array[string]`\n\n### `Routing.Routers`\n\nAlternative configuration used when `Routing.Type=custom`.\n\n> [!CAUTION]\n> **EXPERIMENTAL: `Routing.Routers` is for research and testing only, not production use.**\n>\n> - The configuration format and behavior may change without notice between releases.\n> - Bugs and regressions may not be prioritized.\n> - HTTP-only configurations cannot reliably provide content. See [delegated-routing.md](delegated-routing.md#limitations).\n>\n> Most users should use `Routing.Type=auto` or `autoclient` with [`Routing.DelegatedRouters`](#routingdelegatedrouters).\n\nAllows for replacing the default routing (Amino DHT) with alternative Router\nimplementations.\n\nThe map key is a name of a Router, and the value is its configuration.\n\nDefault: `{}`\n\nType: `object[string->object]`\n\n#### `Routing.Routers.[name].Type`\n\n**⚠️ EXPERIMENTAL: For research and testing only. May change without notice.**\n\nIt specifies the routing type that will be created.\n\nCurrently supported types:\n\n- `http` simple delegated routing based on HTTP protocol from [IPIP-337](https://specs.ipfs.tech/ipips/ipip-0337/)\n- `dht` provides decentralized routing based on [libp2p's kad-dht](https://github.com/libp2p/specs/tree/master/kad-dht)\n- `parallel` and `sequential`: Helpers that can be used to run several routers sequentially or in parallel.\n\nType: `string`\n\n#### `Routing.Routers.[name].Parameters`\n\n**⚠️ EXPERIMENTAL: For research and testing only. May change without notice.**\n\nParameters needed to create the specified router. Supported params per router type:\n\nHTTP:\n\n- `Endpoint` (mandatory): URL that will be used to connect to a specified router.\n- `MaxProvideBatchSize`: This number determines the maximum amount of CIDs sent per batch. Servers might not accept more than 100 elements per batch. 100 elements by default.\n- `MaxProvideConcurrency`: It determines the number of threads used when providing content. GOMAXPROCS by default.\n\nDHT:\n\n- `\"Mode\"`: Mode used by the Amino DHT. Possible values: \"server\", \"client\", \"auto\"\n- `\"AcceleratedDHTClient\"`: Set to `true` if you want to use the acceleratedDHT.\n- `\"PublicIPNetwork\"`: Set to `true` to create a `WAN` DHT. Set to `false` to create a `LAN` DHT.\n\nParallel:\n\n- `Routers`: A list of routers that will be executed in parallel:\n - `Name:string`: Name of the router. It should be one of the previously added to `Routers` list.\n - `Timeout:duration`: Local timeout. It accepts strings compatible with Go `time.ParseDuration(string)` (`10s`, `1m`, `2h`). Time will start counting when this specific router is called, and it will stop when the router returns, or we reach the specified timeout.\n - `ExecuteAfter:duration`: Providing this param will delay the execution of that router at the specified time. It accepts strings compatible with Go `time.ParseDuration(string)` (`10s`, `1m`, `2h`).\n - `IgnoreErrors:bool`: It will specify if that router should be ignored if an error occurred.\n- `Timeout:duration`: Global timeout. It accepts strings compatible with Go `time.ParseDuration(string)` (`10s`, `1m`, `2h`).\n\nSequential:\n\n- `Routers`: A list of routers that will be executed in order:\n - `Name:string`: Name of the router. It should be one of the previously added to `Routers` list.\n - `Timeout:duration`: Local timeout. It accepts strings compatible with Go `time.ParseDuration(string)`. Time will start counting when this specific router is called, and it will stop when the router returns, or we reach the specified timeout.\n - `IgnoreErrors:bool`: It will specify if that router should be ignored if an error occurred.\n- `Timeout:duration`: Global timeout. It accepts strings compatible with Go `time.ParseDuration(string)`.\n\nDefault: `{}` (use the safe implicit defaults)\n\nType: `object[string->string]`\n\n### `Routing.Methods`\n\n`Methods:map` will define which routers will be executed per method used when `Routing.Type=custom`.\n\n> [!CAUTION]\n> **EXPERIMENTAL: `Routing.Methods` is for research and testing only, not production use.**\n>\n> - The configuration format and behavior may change without notice between releases.\n> - Bugs and regressions may not be prioritized.\n> - HTTP-only configurations cannot reliably provide content. See [delegated-routing.md](delegated-routing.md#limitations).\n>\n> Most users should use `Routing.Type=auto` or `autoclient` with [`Routing.DelegatedRouters`](#routingdelegatedrouters).\n\nThe key will be the name of the method: `\"provide\"`, `\"find-providers\"`, `\"find-peers\"`, `\"put-ipns\"`, `\"get-ipns\"`. All methods must be added to the list.\n\nThe value will contain:\n\n- `RouterName:string`: Name of the router. It should be one of the previously added to `Routing.Routers` list.\n\nType: `object[string->object]`\n\n**Examples:**\n\nComplete example using 2 Routers, Amino DHT (LAN/WAN) and parallel.\n\n```\n$ ipfs config Routing.Type --json '\"custom\"'\n\n$ ipfs config Routing.Routers.WanDHT --json '{\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"auto\",\n \"PublicIPNetwork\": true,\n \"AcceleratedDHTClient\": false\n }\n}'\n\n$ ipfs config Routing.Routers.LanDHT --json '{\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"auto\",\n \"PublicIPNetwork\": false,\n \"AcceleratedDHTClient\": false\n }\n}'\n\n$ ipfs config Routing.Routers.ParallelHelper --json '{\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\" : \"LanDHT\",\n \"IgnoreErrors\" : true,\n \"Timeout\": \"3s\"\n },\n {\n \"RouterName\" : \"WanDHT\",\n \"IgnoreErrors\" : false,\n \"Timeout\": \"5m\",\n \"ExecuteAfter\": \"2s\"\n }\n ]\n }\n}'\n\nipfs config Routing.Methods --json '{\n \"find-peers\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"find-providers\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"get-ipns\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"provide\": {\n \"RouterName\": \"ParallelHelper\"\n },\n \"put-ipns\": {\n \"RouterName\": \"ParallelHelper\"\n }\n }'\n\n```\n\n## `Swarm`\n\nOptions for configuring the swarm.\n\n### `Swarm.AddrFilters`\n\nAn array of addresses (multiaddr netmasks) to not dial. By default, IPFS nodes\nadvertise _all_ addresses, even internal ones. This makes it easier for nodes on\nthe same network to reach each other. Unfortunately, this means that an IPFS\nnode will try to connect to one or more private IP addresses whenever dialing\nanother node, even if this other node is on a different network. This may\ntrigger netscan alerts on some hosting providers or cause strain in some setups.\n\n> [!TIP]\n> The [`server` configuration profile](#server-profile) fills up this list with sensible defaults,\n> preventing dials to all non-routable IP addresses (e.g., `/ip4/192.168.0.0/ipcidr/16`,\n> which is the [multiaddress][multiaddr] representation of `192.168.0.0/16`) but you should always\n> check settings against your own network and/or hosting provider.\n\nDefault: `[]`\n\nType: `array[string]`\n\n### `Swarm.DisableBandwidthMetrics`\n\nA boolean value that when set to true, will cause ipfs to not keep track of\nbandwidth metrics. Disabling bandwidth metrics can lead to a slight performance\nimprovement, as well as a reduction in memory usage.\n\nDefault: `false`\n\nType: `bool`\n\n### `Swarm.DisableNatPortMap`\n\nDisable automatic NAT port forwarding (turn off [UPnP](https://en.wikipedia.org/wiki/Universal_Plug_and_Play)).\n\nWhen not disabled (default), Kubo asks NAT devices (e.g., routers), to open\nup an external port and forward it to the port Kubo is running on. When this\nworks (i.e., when your router supports NAT port forwarding), it makes the local\nKubo node accessible from the public internet.\n\nDefault: `false`\n\nType: `bool`\n\n### `Swarm.EnableHolePunching`\n\nEnable hole punching for NAT traversal\nwhen port forwarding is not possible.\n\nWhen enabled, Kubo will coordinate with the counterparty using\na [relayed connection](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md),\nto [upgrade to a direct connection](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md)\nthrough a NAT/firewall whenever possible.\nThis feature requires `Swarm.RelayClient.Enabled` to be set to `true`.\n\nDefault: `true`\n\nType: `flag`\n\n### `Swarm.EnableAutoRelay`\n\n**REMOVED**\n\nSee `Swarm.RelayClient` instead.\n\n### `Swarm.RelayClient`\n\nConfiguration options for the relay client to use relay services.\n\nDefault: `{}`\n\nType: `object`\n\n#### `Swarm.RelayClient.Enabled`\n\nEnables \"automatic relay user\" mode for this node.\n\nYour node will automatically _use_ public relays from the network if it detects\nthat it cannot be reached from the public internet (e.g., it's behind a\nfirewall) and get a `/p2p-circuit` address from a public relay.\n\nDefault: `true`\n\nType: `flag`\n\n#### `Swarm.RelayClient.StaticRelays`\n\nYour node will use these statically configured relay servers\ninstead of discovering public relays ([Circuit Relay v2](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md)) from the network.\n\nDefault: `[]`\n\nType: `array[string]`\n\n### `Swarm.RelayService`\n\nConfiguration options for the relay service that can be provided to _other_ peers\non the network ([Circuit Relay v2](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md)).\n\nDefault: `{}`\n\nType: `object`\n\n#### `Swarm.RelayService.Enabled`\n\nEnables providing `/p2p-circuit` v2 relay service to other peers on the network.\n\nNOTE: This is the service/server part of the relay system.\nDisabling this will prevent this node from running as a relay server.\nUse [`Swarm.RelayClient.Enabled`](#swarmrelayclientenabled) for turning your node into a relay user.\n\nDefault: `true`\n\nType: `flag`\n\n#### `Swarm.RelayService.Limit`\n\nLimits are applied to every relayed connection.\n\nDefault: `{}`\n\nType: `object[string -> string]`\n\n##### `Swarm.RelayService.ConnectionDurationLimit`\n\nTime limit before a relayed connection is reset.\n\nDefault: `\"2m\"`\n\nType: `duration`\n\n##### `Swarm.RelayService.ConnectionDataLimit`\n\nLimit of data relayed (in each direction) before a relayed connection is reset.\n\nDefault: `131072` (128 kb)\n\nType: `optionalInteger`\n\n#### `Swarm.RelayService.ReservationTTL`\n\nDuration of a new or refreshed reservation.\n\nDefault: `\"1h\"`\n\nType: `duration`\n\n#### `Swarm.RelayService.MaxReservations`\n\nMaximum number of active relay slots.\n\nDefault: `128`\n\nType: `optionalInteger`\n\n#### `Swarm.RelayService.MaxCircuits`\n\nMaximum number of open relay connections for each peer.\n\nDefault: `16`\n\nType: `optionalInteger`\n\n#### `Swarm.RelayService.BufferSize`\n\nSize of the relayed connection buffers.\n\nDefault: `2048`\n\nType: `optionalInteger`\n\n#### `Swarm.RelayService.MaxReservationsPerPeer`\n\n**REMOVED in kubo 0.32 due to [go-libp2p#2974](https://github.com/libp2p/go-libp2p/pull/2974)**\n\n#### `Swarm.RelayService.MaxReservationsPerIP`\n\nMaximum number of reservations originating from the same IP.\n\nDefault: `8`\n\nType: `optionalInteger`\n\n#### `Swarm.RelayService.MaxReservationsPerASN`\n\nMaximum number of reservations originating from the same ASN.\n\nDefault: `32`\n\nType: `optionalInteger`\n\n### `Swarm.EnableRelayHop`\n\n**REMOVED**\n\nReplaced with [`Swarm.RelayService.Enabled`](#swarmrelayserviceenabled).\n\n### `Swarm.DisableRelay`\n\n**REMOVED**\n\nSet `Swarm.Transports.Network.Relay` to `false` instead.\n\n### `Swarm.EnableAutoNATService`\n\n**REMOVED**\n\nPlease use [`AutoNAT.ServiceMode`](#autonatservicemode).\n\n### `Swarm.ConnMgr`\n\nThe connection manager determines which and how many connections to keep and can\nbe configured to keep. Kubo currently supports two connection managers:\n\n- none: never close idle connections.\n- basic: the default connection manager.\n\nBy default, this section is empty and the implicit defaults defined below\nare used.\n\n#### `Swarm.ConnMgr.Type`\n\nSets the type of connection manager to use, options are: `\"none\"` (no connection\nmanagement) and `\"basic\"`.\n\nDefault: \"basic\".\n\nType: `optionalString` (default when unset or empty)\n\n#### Basic Connection Manager\n\nThe basic connection manager uses a \"high water\", a \"low water\", and internal\nscoring to periodically close connections to free up resources. When a node\nusing the basic connection manager reaches `HighWater` idle connections, it\nwill close the least useful ones until it reaches `LowWater` idle\nconnections. The process of closing connections happens every `SilencePeriod`.\n\nThe connection manager considers a connection idle if:\n\n- It has not been explicitly _protected_ by some subsystem. For example, Bitswap\n will protect connections to peers from which it is actively downloading data,\n the DHT will protect some peers for routing, and the peering subsystem will\n protect all \"peered\" nodes.\n- It has existed for longer than the `GracePeriod`.\n\n**Example:**\n\n```json\n{\n \"Swarm\": {\n \"ConnMgr\": {\n \"Type\": \"basic\",\n \"LowWater\": 100,\n \"HighWater\": 200,\n \"GracePeriod\": \"30s\",\n \"SilencePeriod\": \"10s\"\n }\n }\n}\n```\n\n##### `Swarm.ConnMgr.LowWater`\n\nLowWater is the number of connections that the basic connection manager will\ntrim down to.\n\nDefault: `32`\n\nType: `optionalInteger`\n\n##### `Swarm.ConnMgr.HighWater`\n\nHighWater is the number of connections that, when exceeded, will trigger a\nconnection GC operation. Note: protected/recently formed connections don't count\ntowards this limit.\n\nDefault: `96`\n\nType: `optionalInteger`\n\n##### `Swarm.ConnMgr.GracePeriod`\n\nGracePeriod is a time duration that new connections are immune from being closed\nby the connection manager.\n\nDefault: `\"20s\"`\n\nType: `optionalDuration`\n\n##### `Swarm.ConnMgr.SilencePeriod`\n\nSilencePeriod is the time duration between connection manager runs, when connections that are idle are closed.\n\nDefault: `\"10s\"`\n\nType: `optionalDuration`\n\n### `Swarm.ResourceMgr`\n\nLearn more about Kubo's usage of libp2p Network Resource Manager\nin the [dedicated resource management docs](./libp2p-resource-management.md).\n\n#### `Swarm.ResourceMgr.Enabled`\n\nEnables the libp2p Resource Manager using limits based on the defaults and/or other configuration as discussed in [libp2p resource management](./libp2p-resource-management.md).\n\nDefault: `true`\nType: `flag`\n\n#### `Swarm.ResourceMgr.MaxMemory`\n\nThis is the max amount of memory to allow go-libp2p to use.\n\nlibp2p's resource manager will prevent additional resource creation while this limit is reached.\nThis value is also used to scale the limit on various resources at various scopes\nwhen the default limits (discussed in [libp2p resource management](./libp2p-resource-management.md)) are used.\nFor example, increasing this value will increase the default limit for incoming connections.\n\nIt is possible to inspect the runtime limits via `ipfs swarm resources --help`.\n\n> [!IMPORTANT]\n> `Swarm.ResourceMgr.MaxMemory` is the memory limit for go-libp2p networking stack alone, and not for entire Kubo or Bitswap.\n>\n> To set memory limit for the entire Kubo process, use [`GOMEMLIMIT` environment variable](http://web.archive.org/web/20240222201412/https://kupczynski.info/posts/go-container-aware/) which all Go programs recognize, and then set `Swarm.ResourceMgr.MaxMemory` to less than your custom `GOMEMLIMIT`.\n\nDefault: `[TOTAL_SYSTEM_MEMORY]/2`\nType: [`optionalBytes`](#optionalbytes)\n\n#### `Swarm.ResourceMgr.MaxFileDescriptors`\n\nThis is the maximum number of file descriptors to allow libp2p to use.\nlibp2p's resource manager will prevent additional file descriptor consumption while this limit is reached.\n\nThis param is ignored on Windows.\n\nDefault `[TOTAL_SYSTEM_FILE_DESCRIPTORS]/2`\nType: `optionalInteger`\n\n#### `Swarm.ResourceMgr.Allowlist`\n\nA list of [multiaddrs][libp2p-multiaddrs] that can bypass normal system limits (but are still limited by the allowlist scope).\nConvenience config around [go-libp2p-resource-manager#Allowlist.Add](https://pkg.go.dev/github.com/libp2p/go-libp2p/p2p/host/resource-manager#Allowlist.Add).\n\nDefault: `[]`\n\nType: `array[string]` ([multiaddrs][multiaddr])\n\n### `Swarm.Transports`\n\nConfiguration section for libp2p transports. An empty configuration will apply\nthe defaults.\n\n### `Swarm.Transports.Network`\n\nConfiguration section for libp2p _network_ transports. Transports enabled in\nthis section will be used for dialing. However, to receive connections on these\ntransports, multiaddrs for these transports must be added to `Addresses.Swarm`.\n\nSupported transports are: QUIC, TCP, WS, Relay, WebTransport and WebRTCDirect.\n\n> [!CAUTION]\n> **SECURITY CONSIDERATIONS FOR NETWORK TRANSPORTS**\n>\n> Enabling network transports allows your node to accept connections from the internet.\n> Ensure your firewall rules and [`Addresses.Swarm`](#addressesswarm) configuration\n> align with your security requirements.\n> See [Security section](#security) for network exposure considerations.\n\nEach field in this section is a `flag`.\n\n#### `Swarm.Transports.Network.TCP`\n\n[TCP](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) is a simple\nand widely deployed transport, it should be compatible with most implementations\nand network configurations. TCP doesn't directly support encryption and/or\nmultiplexing, so libp2p will layer a security & multiplexing transport over it.\n\nDefault: Enabled\n\nType: `flag`\n\nListen Addresses:\n\n- /ip4/0.0.0.0/tcp/4001 (default)\n- /ip6/::/tcp/4001 (default)\n\n#### `Swarm.Transports.Network.Websocket`\n\n[Websocket](https://en.wikipedia.org/wiki/WebSocket) is a transport usually used\nto connect to non-browser-based IPFS nodes from browser-based js-ipfs nodes.\n\nWhile it's enabled by default for dialing, Kubo doesn't listen on this\ntransport by default.\n\nDefault: Enabled\n\nType: `flag`\n\nListen Addresses:\n\n- /ip4/0.0.0.0/tcp/4001/ws\n- /ip6/::/tcp/4001/ws\n\n#### `Swarm.Transports.Network.QUIC`\n\n[QUIC](https://en.wikipedia.org/wiki/QUIC) is the most widely used transport by\nKubo nodes. It is a UDP-based transport with built-in encryption and\nmultiplexing. The primary benefits over TCP are:\n\n1. It takes 1 round trip to establish a connection (our TCP transport\n currently takes 4).\n2. No [Head-of-Line blocking](https://en.wikipedia.org/wiki/Head-of-line_blocking).\n3. It doesn't require a file descriptor per connection, easing the load on the OS.\n\nDefault: Enabled\n\nType: `flag`\n\nListen Addresses:\n\n- `/ip4/0.0.0.0/udp/4001/quic-v1` (default)\n- `/ip6/::/udp/4001/quic-v1` (default)\n\n#### `Swarm.Transports.Network.Relay`\n\n[Libp2p Relay](https://github.com/libp2p/specs/tree/master/relay) proxy\ntransport that forms connections by hopping between multiple libp2p nodes.\nAllows IPFS node to connect to other peers using their `/p2p-circuit`\n[multiaddrs][libp2p-multiaddrs]. This transport is primarily useful for bypassing firewalls and\nNATs.\n\nSee also:\n\n- Docs: [Libp2p Circuit Relay](https://web.archive.org/web/20260128152445/https://docs.libp2p.io/concepts/nat/circuit-relay/)\n- [`Swarm.RelayClient.Enabled`](#swarmrelayclientenabled) for getting a public\n- `/p2p-circuit` address when behind a firewall.\n- [`Swarm.EnableHolePunching`](#swarmenableholepunching) for direct connection upgrade through relay\n- [`Swarm.RelayService.Enabled`](#swarmrelayserviceenabled) for becoming a\n limited relay for other peers\n\nDefault: Enabled\n\nType: `flag`\n\nListen Addresses:\n\n- This transport is special. Any node that enables this transport can receive\n inbound connections on this transport, without specifying a listen address.\n\n#### `Swarm.Transports.Network.WebTransport`\n\nA new feature of [`go-libp2p`](https://github.com/libp2p/go-libp2p/releases/tag/v0.23.0)\nis the [WebTransport](https://github.com/libp2p/go-libp2p/issues/1717) transport.\n\nThis is a spiritual descendant of WebSocket but over `HTTP/3`.\nSince this runs on top of `HTTP/3` it uses `QUIC` under the hood.\nWe expect it to perform worst than `QUIC` because of the extra overhead,\nthis transport is really meant at agents that cannot do `TCP` or `QUIC` (like browsers).\n\nWebTransport is a new transport protocol currently under development by the IETF and the W3C, and already implemented by Chrome.\nConceptually, it’s like WebSocket run over QUIC instead of TCP. Most importantly, it allows browsers to establish (secure!) connections to WebTransport servers without the need for CA-signed certificates,\nthereby enabling any js-libp2p node running in a browser to connect to any kubo node, with zero manual configuration involved.\n\nThe previous alternative is websocket secure, which require installing a reverse proxy and TLS certificates manually.\n\nDefault: Enabled\n\nType: `flag`\n\nListen Addresses:\n\n- `/ip4/0.0.0.0/udp/4001/quic-v1/webtransport` (default)\n- `/ip6/::/udp/4001/quic-v1/webtransport` (default)\n\n#### `Swarm.Transports.Network.WebRTCDirect`\n\n[WebRTC Direct](https://github.com/libp2p/specs/blob/master/webrtc/webrtc-direct.md)\nis a transport protocol that provides another way for browsers to\nconnect to the rest of the libp2p network. WebRTC Direct allows for browser\nnodes to connect to other nodes without special configuration, such as TLS\ncertificates. This can be useful for browser nodes that do not yet support\n[WebTransport](https://web.archive.org/web/20260107053250/https://blog.libp2p.io/2022-12-19-libp2p-webtransport/),\nwhich is still relatively new and has [known issues](https://github.com/libp2p/js-libp2p/issues/2572).\n\nEnabling this transport allows Kubo node to act on `/udp/4001/webrtc-direct`\nlisteners defined in `Addresses.Swarm`, `Addresses.Announce` or\n`Addresses.AppendAnnounce`.\n\n> [!NOTE]\n> WebRTC Direct is browser-to-node. It cannot be used to connect a browser\n> node to a node that is behind a NAT or firewall (without UPnP port mapping).\n> The browser-to-private requires using normal\n> [WebRTC](https://github.com/libp2p/specs/blob/master/webrtc/webrtc.md),\n> which is currently being worked on in\n> [go-libp2p#2009](https://github.com/libp2p/go-libp2p/issues/2009).\n\nDefault: Enabled\n\nType: `flag`\n\nListen Addresses:\n\n- `/ip4/0.0.0.0/udp/4001/webrtc-direct` (default)\n- `/ip6/::/udp/4001/webrtc-direct` (default)\n\n### `Swarm.Transports.Security`\n\nConfiguration section for libp2p _security_ transports. Transports enabled in\nthis section will be used to secure unencrypted connections.\n\nThis does not concern all the QUIC transports which use QUIC's builtin encryption.\n\nSecurity transports are configured with the `priority` type.\n\nWhen establishing an _outbound_ connection, Kubo will try each security\ntransport in priority order (lower first), until it finds a protocol that the\nreceiver supports. When establishing an _inbound_ connection, Kubo will let\nthe initiator choose the protocol, but will refuse to use any of the disabled\ntransports.\n\nSupported transports are: TLS (priority 100) and Noise (priority 200).\n\nNo default priority will ever be less than 100. Lower values have precedence.\n\n#### `Swarm.Transports.Security.TLS`\n\n[TLS](https://github.com/libp2p/specs/tree/master/tls) (1.3) is the default\nsecurity transport as of Kubo 0.5.0. It's also the most scrutinized and\ntrusted security transport.\n\nDefault: `100`\n\nType: `priority`\n\n#### `Swarm.Transports.Security.SECIO`\n\n**REMOVED**: support for SECIO has been removed. Please remove this option from your config.\n\n#### `Swarm.Transports.Security.Noise`\n\n[Noise](https://github.com/libp2p/specs/tree/master/noise) is slated to replace\nTLS as the cross-platform, default libp2p protocol due to ease of\nimplementation. It is currently enabled by default but with low priority as it's\nnot yet widely supported.\n\nDefault: `200`\n\nType: `priority`\n\n### `Swarm.Transports.Multiplexers`\n\nConfiguration section for libp2p _multiplexer_ transports. Transports enabled in\nthis section will be used to multiplex duplex connections.\n\nThis does not concern all the QUIC transports which use QUIC's builtin muxing.\n\nMultiplexer transports are configured the same way security transports are, with\nthe `priority` type. Like with security transports, the initiator gets their\nfirst choice.\n\nSupported transport is only: Yamux (priority 100)\n\nNo default priority will ever be less than 100.\n\n### `Swarm.Transports.Multiplexers.Yamux`\n\nYamux is the default multiplexer used when communicating between Kubo nodes.\n\nDefault: `100`\n\nType: `priority`\n\n### `Swarm.Transports.Multiplexers.Mplex`\n\n**REMOVED**: See \n\nSupport for Mplex has been [removed from Kubo and go-libp2p](https://github.com/libp2p/specs/issues/553).\nPlease remove this option from your config.\n\n## `DNS`\n\nOptions for configuring DNS resolution for [DNSLink](https://docs.ipfs.tech/concepts/dnslink/) and `/dns*` [Multiaddrs][libp2p-multiaddrs] (including peer addresses discovered via DHT or delegated routing).\n\n### `DNS.Resolvers`\n\nMap of [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) to custom resolver URLs.\n\nThis allows for overriding the default DNS resolver provided by the operating system,\nand using different resolvers per domain or TLD (including ones from alternative, non-ICANN naming systems).\n\nExample:\n\n```json\n{\n \"DNS\": {\n \"Resolvers\": {\n \"eth.\": \"https://dns.eth.limo/dns-query\",\n \"crypto.\": \"https://resolver.unstoppable.io/dns-query\",\n \"libre.\": \"https://ns1.iriseden.fr/dns-query\",\n \".\": \"https://cloudflare-dns.com/dns-query\"\n }\n }\n}\n```\n\nBe mindful that:\n\n- Currently only `https://` URLs for [DNS over HTTPS (DoH)](https://en.wikipedia.org/wiki/DNS_over_HTTPS) endpoints are supported as values.\n- The default catch-all resolver is the cleartext one provided by your operating system. It can be overridden by adding a DoH entry for the DNS root indicated by `.` as illustrated above.\n- Out-of-the-box support for selected non-ICANN TLDs relies on third-party centralized services provided by respective communities on best-effort basis.\n- The special value `\"auto\"` uses DNS resolvers from [AutoConf](#autoconf) when enabled. For example: `{\".\": \"auto\"}` uses any custom DoH resolver (global or per TLD) provided by AutoConf system.\n- When [`AutoTLS.SkipDNSLookup`](#autotlsskipdnslookup) is enabled (default), domains matching [`AutoTLS.DomainSuffix`](#autotlsdomainsuffix) (default: `libp2p.direct`) are resolved locally by parsing the IP directly from the hostname. Set `AutoTLS.SkipDNSLookup=false` to force network DNS lookups for these domains.\n\nDefault: `{\".\": \"auto\"}`\n\nType: `object[string -> string]`\n\n### `DNS.MaxCacheTTL`\n\nMaximum duration for which entries are valid in the DoH cache.\n\nThis allows you to cap the Time-To-Live suggested by the DNS response ([RFC2181](https://datatracker.ietf.org/doc/html/rfc2181#section-8)).\nIf present, the upper bound is applied to DoH resolvers in [`DNS.Resolvers`](#dnsresolvers).\n\nNote: this does NOT work with Go's default DNS resolver. To make this a global setting, add a `.` entry to `DNS.Resolvers` first.\n\n**Examples:**\n\n- `\"1m\"` DNS entries are kept for 1 minute or less.\n- `\"0s\"` DNS entries expire as soon as they are retrieved.\n\nDefault: Respect DNS Response TTL\n\nType: `optionalDuration`\n\n## `HTTPRetrieval`\n\n`HTTPRetrieval` is configuration for pure HTTP retrieval based on Trustless HTTP Gateways'\n[Block Responses (`application/vnd.ipld.raw`)](https://specs.ipfs.tech/http-gateways/trustless-gateway/#block-responses-application-vnd-ipld-raw)\nwhich can be used in addition to or instead of retrieving blocks with [Bitswap over Libp2p](#bitswap).\n\nDefault: `{}`\n\nType: `object`\n\n### `HTTPRetrieval.Enabled`\n\nControls whether HTTP-based block retrieval is enabled.\n\nWhen enabled, Kubo will act on `/tls/http` (HTTP/2) providers ([Trustless HTTP Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) returned by the [`Routing.DelegatedRouters`](#routingdelegatedrouters)\nto perform pure HTTP [block retrievals](https://specs.ipfs.tech/http-gateways/trustless-gateway/#block-responses-application-vnd-ipld-raw)\n(`/ipfs/cid?format=raw`, `Accept: application/vnd.ipld.raw`)\nalongside [Bitswap over Libp2p](#bitswap).\n\nHTTP requests for `application/vnd.ipld.raw` will be made instead of Bitswap when a peer has a `/tls/http` multiaddr\nand the HTTPS server returns HTTP 200 for the [probe path](https://specs.ipfs.tech/http-gateways/trustless-gateway/#dedicated-probe-paths).\n\n> [!IMPORTANT]\n> This feature is relatively new. Please report any issues via [Github](https://github.com/ipfs/kubo/issues/new).\n>\n> Important notes:\n>\n> - TLS and HTTP/2 are required. For privacy reasons, and to maintain feature-parity with browsers, unencrypted `http://` providers are ignored and not used.\n> - This feature works in the same way as Bitswap: connected HTTP-peers receive optimistic block requests even for content that they are not announcing.\n> - For performance reasons, and to avoid loops, the HTTP client does not follow redirects. Providers should keep announcements up to date.\n> - IPFS ecosystem is working towards [supporting HTTP providers on Amino DHT](https://github.com/ipfs/specs/issues/496). Currently, HTTP providers are mostly limited to results from [`Routing.DelegatedRouters`](#routingdelegatedrouters) endpoints and requires `Routing.Type=auto|autoclient`.\n\nDefault: `true`\n\nType: `flag`\n\n### `HTTPRetrieval.Allowlist`\n\nOptional list of hostnames for which HTTP retrieval is allowed for.\nIf this list is not empty, only hosts matching these entries will be allowed for HTTP retrieval.\n\n> [!TIP]\n> To limit HTTP retrieval to a provider at `/dns4/example.com/tcp/443/tls/http` (which would serve `HEAD|GET https://example.com/ipfs/cid?format=raw`), set this to `[\"example.com\"]`\n\nDefault: `[]`\n\nType: `array[string]`\n\n### `HTTPRetrieval.Denylist`\n\nOptional list of hostnames for which HTTP retrieval is not allowed.\nDenylist entries take precedence over Allowlist entries.\n\n> [!TIP]\n> This denylist operates on HTTP endpoint hostnames.\n> To deny specific PeerID, use [`Routing.IgnoreProviders`](#routingignoreproviders) instead.\n\nDefault: `[]`\n\nType: `array[string]`\n\n### `HTTPRetrieval.NumWorkers`\n\nThe number of worker goroutines to use for concurrent HTTP retrieval operations.\nThis setting controls the level of parallelism for HTTP-based block retrieval operations.\nHigher values can improve performance when retrieving many blocks but may increase resource usage.\n\nDefault: `16`\n\nType: `optionalInteger`\n\n### `HTTPRetrieval.MaxBlockSize`\n\nSets the maximum size of a block that the HTTP retrieval client will accept.\n\n> [!NOTE]\n> This setting is a security feature designed to protect Kubo from malicious providers who might send excessively large or invalid data.\n> Increasing this value allows Kubo to retrieve larger blocks from compatible HTTP providers, but doing so reduces interoperability with Bitswap, and increases potential security risks.\n>\n> Learn more: [Supporting Large IPLD Blocks: Why block limits?](https://discuss.ipfs.tech/t/supporting-large-ipld-blocks/15093#why-block-limits-5)\n\nDefault: `2MiB` (matching [Bitswap size limit](https://specs.ipfs.tech/bitswap-protocol/#block-sizes))\n\nType: `optionalString`\n\n### `HTTPRetrieval.TLSInsecureSkipVerify`\n\nDisables TLS certificate validation.\nAllows making HTTPS connections to HTTP/2 test servers with self-signed TLS certificates.\nOnly for testing, do not use in production.\n\nDefault: `false`\n\nType: `flag`\n\n## `Import`\n\nOptions to configure the default parameters used for ingesting data, in commands such as `ipfs add` or `ipfs block put`. All affected commands are detailed per option.\n\nThese options implement [IPIP-499: UnixFS CID Profiles](https://specs.ipfs.tech/ipips/ipip-0499/) for reproducible CID generation across IPFS implementations. Instead of configuring individual options, you can apply a predefined profile with `ipfs config profile apply `. See [Profiles](#profiles) for available options like `unixfs-v1-2025`.\n\nNote that using CLI flags will override the options defined here.\n\n### `Import.CidVersion`\n\nThe default CID version. Commands affected: `ipfs add`.\n\nMust be either 0 or 1. CIDv0 uses SHA2-256 only, while CIDv1 supports multiple hash functions.\n\nDefault: `0`\n\nType: `optionalInteger`\n\n### `Import.UnixFSRawLeaves`\n\nThe default UnixFS raw leaves option. Commands affected: `ipfs add`, `ipfs files write`.\n\nDefault: `false` if `CidVersion=0`; `true` if `CidVersion=1`\n\nType: `flag`\n\n### `Import.UnixFSChunker`\n\nThe default UnixFS chunker. Commands affected: `ipfs add`.\n\nValid formats:\n\n- `size-` - fixed size chunker\n- `rabin---` - rabin fingerprint chunker\n- `buzhash` - buzhash chunker\n\nThe maximum accepted value for `size-` and rabin `max` parameter is\n`2MiB - 256 bytes` (2096896 bytes). The 256-byte overhead budget is reserved\nfor protobuf/UnixFS framing so that serialized blocks stay within the 2MiB\nblock size limit defined by the\n[bitswap spec](https://specs.ipfs.tech/bitswap-protocol/#block-sizes).\nThe `buzhash` chunker uses a fixed internal maximum of 512KiB and is not\naffected by this limit.\n\nOnly the fixed-size chunker (`size-`) guarantees that the same data\nwill always produce the same CID. The `rabin` and `buzhash` chunkers may\nchange their internal parameters in a future release.\n\nDefault: `size-262144`\n\nType: `optionalString`\n\n### `Import.HashFunction`\n\nThe default hash function. Commands affected: `ipfs add`, `ipfs block put`, `ipfs dag put`.\n\nMust be a valid multihash name (e.g., `sha2-256`, `blake3`) and must be allowed for use in IPFS according to security constraints.\n\nRun `ipfs cid hashes --supported` to see the full list of allowed hash functions.\n\nDefault: `sha2-256`\n\nType: `optionalString`\n\n### `Import.FastProvideRoot`\n\nImmediately provide root CIDs to the DHT in addition to the regular provide queue.\n\nThis complements the sweep provider system: fast-provide handles the urgent case (root CIDs that users share and reference), while the sweep provider efficiently provides all blocks according to the `Provide.Strategy` over time. Together, they optimize for both immediate discoverability of newly imported content and efficient resource usage for complete DAG provides.\n\nWhen disabled, only the sweep provider's queue is used.\n\nThis setting applies to both `ipfs add` and `ipfs dag import` commands and can be overridden per-command with the `--fast-provide-root` flag.\n\nIgnored when DHT is not available for routing (e.g., `Routing.Type=none` or delegated-only configurations).\n\nDefault: `true`\n\nType: `flag`\n\n### `Import.FastProvideWait`\n\nWait for the immediate root CID provide to complete before returning.\n\nWhen enabled, the command blocks until the provide completes, ensuring guaranteed discoverability before returning. When disabled (default), the provide happens asynchronously in the background without blocking the command.\n\nUse this when you need certainty that content is discoverable before the command returns (e.g., sharing a link immediately after adding).\n\nThis setting applies to both `ipfs add` and `ipfs dag import` commands and can be overridden per-command with the `--fast-provide-wait` flag.\n\nIgnored when DHT is not available for routing (e.g., `Routing.Type=none` or delegated-only configurations).\n\nDefault: `false`\n\nType: `flag`\n\n### `Import.BatchMaxNodes`\n\nThe maximum number of nodes in a write-batch. The total size of the batch is limited by `BatchMaxnodes` and `BatchMaxSize`.\n\nIncreasing this will batch more items together when importing data with `ipfs dag import`, which can speed things up.\n\nMust be positive (> 0). Setting to 0 would cause immediate batching after each node, which is inefficient.\n\nDefault: `128`\n\nType: `optionalInteger`\n\n### `Import.BatchMaxSize`\n\nThe maximum size of a single write-batch (computed as the sum of the sizes of the blocks). The total size of the batch is limited by `BatchMaxnodes` and `BatchMaxSize`.\n\nIncreasing this will batch more items together when importing data with `ipfs dag import`, which can speed things up.\n\nMust be positive (> 0). Setting to 0 would cause immediate batching after any data, which is inefficient.\n\nDefault: `20971520` (20MiB)\n\nType: `optionalInteger`\n\n### `Import.UnixFSFileMaxLinks`\n\nThe maximum number of links that a node part of a UnixFS File can have\nwhen building the DAG while importing.\n\nThis setting controls both the fanout in files that are chunked into several\nblocks and grouped as a Unixfs (dag-pb) DAG.\n\nMust be positive (> 0). Zero or negative values would break file DAG construction.\n\nDefault: `174`\n\nType: `optionalInteger`\n\n### `Import.UnixFSDirectoryMaxLinks`\n\nThe maximum number of links that a node part of a UnixFS basic directory can\nhave when building the DAG while importing.\n\nThis setting controls both the fanout for basic, non-HAMT folder nodes. It\nsets a limit after which directories are converted to a HAMT-based structure.\n\nWhen unset (0), no limit exists for children. Directories will be converted to\nHAMTs based on their estimated size only.\n\nThis setting will cause basic directories to be converted to HAMTs when they\nexceed the maximum number of children. This happens transparently during the\nadd process. The fanout of HAMT nodes is controlled by `MaxHAMTFanout`.\n\nMust be non-negative (>= 0). Zero means no limit, negative values are invalid.\n\nCommands affected: `ipfs add`\n\nDefault: `0` (no limit, because [`Import.UnixFSHAMTDirectorySizeThreshold`](#importunixfshamtdirectorysizethreshold) triggers controls when to switch to HAMT sharding when a directory grows too big)\n\nType: `optionalInteger`\n\n### `Import.UnixFSHAMTDirectoryMaxFanout`\n\nThe maximum number of children that a node part of a UnixFS HAMT directory\n(aka sharded directory) can have.\n\nHAMT directories have unlimited children and are used when basic directories\nbecome too big or reach `MaxLinks`. A HAMT is a structure made of UnixFS\nnodes that store the list of elements in the folder. This option controls the\nmaximum number of children that the HAMT nodes can have.\n\nAccording to the [UnixFS specification](https://specs.ipfs.tech/unixfs/#hamt-structure-and-parameters), this value must be a power of 2, between 8 (for byte-aligned bitfields) and 1024 (to prevent denial-of-service attacks).\n\nCommands affected: `ipfs add`, `ipfs daemon` (globally overrides [`boxo/ipld/unixfs/io.DefaultShardWidth`](https://github.com/ipfs/boxo/blob/6c5a07602aed248acc86598f30ab61923a54a83e/ipld/unixfs/io/directory.go#L30C5-L30C22))\n\nDefault: `256`\n\nType: `optionalInteger`\n\n### `Import.UnixFSHAMTDirectorySizeThreshold`\n\nThe sharding threshold to decide whether a basic UnixFS directory\nshould be sharded (converted into HAMT Directory) or not.\n\nThis value is not strictly related to the size of the UnixFS directory block\nand any increases in the threshold should come with being careful that block\nsizes stay under 2MiB in order for them to be reliably transferable through the\nnetworking stack. At the time of writing this, IPFS peers on the public swarm\ntend to ignore requests for blocks bigger than 2MiB.\n\nUses implementation from `boxo/ipld/unixfs/io/directory`, where the size is not\nthe _exact_ block size of the encoded directory but just the estimated size\nbased byte length of DAG-PB Links names and CIDs.\n\nSetting to `1B` is functionally equivalent to always using HAMT (useful in testing).\n\nCommands affected: `ipfs add`, `ipfs daemon` (globally overrides [`boxo/ipld/unixfs/io.HAMTShardingSize`](https://github.com/ipfs/boxo/blob/6c5a07602aed248acc86598f30ab61923a54a83e/ipld/unixfs/io/directory.go#L26))\n\nDefault: `256KiB` (may change, inspect `DefaultUnixFSHAMTDirectorySizeThreshold` to confirm)\n\nType: [`optionalBytes`](#optionalbytes)\n\n### `Import.UnixFSHAMTDirectorySizeEstimation`\n\nControls how directory size is estimated when deciding whether to switch\nfrom a basic UnixFS directory to HAMT sharding.\n\nAccepted values:\n\n- `links` (default): Legacy estimation using sum of link names and CID byte lengths.\n- `block`: Full serialized dag-pb block size for accurate threshold decisions.\n- `disabled`: Disable HAMT sharding entirely (directories always remain basic).\n\nThe `block` estimation is recommended for new profiles as it provides more\naccurate threshold decisions and better cross-implementation consistency.\nSee [IPIP-499](https://specs.ipfs.tech/ipips/ipip-0499/) for more details.\n\nCommands affected: `ipfs add`\n\nDefault: `links`\n\nType: `optionalString`\n\n### `Import.UnixFSDAGLayout`\n\nControls the DAG layout used when chunking files.\n\nAccepted values:\n\n- `balanced` (default): Balanced DAG layout with uniform leaf depth.\n- `trickle`: Trickle DAG layout optimized for streaming.\n\nCommands affected: `ipfs add`\n\nDefault: `balanced`\n\nType: `optionalString`\n\n## `Version`\n\nOptions to configure agent version announced to the swarm, and leveraging\nother peers version for detecting when there is time to update.\n\n### `Version.AgentSuffix`\n\nOptional suffix to the AgentVersion presented by `ipfs id` and exposed via [libp2p identify protocol](https://github.com/libp2p/specs/blob/master/identify/README.md#agentversion).\n\nThe value from config takes precedence over value passed via `ipfs daemon --agent-version-suffix`.\n\n> [!NOTE]\n> Setting a custom version suffix helps with ecosystem analysis, such as Amino DHT reports published at \n\nDefault: `\"\"` (no suffix, or value from `ipfs daemon --agent-version-suffix=`)\n\nType: `optionalString`\n\n### `Version.SwarmCheckEnabled`\n\nObserve the AgentVersion of swarm peers and log warning when\n`SwarmCheckPercentThreshold` of peers runs version higher than this node.\n\nDefault: `true`\n\nType: `flag`\n\n### `Version.SwarmCheckPercentThreshold`\n\nControl the percentage of `kubo/` peers running new version required to\ntrigger update warning.\n\nDefault: `5`\n\nType: `optionalInteger` (1-100)\n\n## Profiles\n\nConfiguration profiles allow to tweak configuration quickly. Profiles can be\napplied with the `--profile` flag to `ipfs init` or with the `ipfs config profile\napply` command. When a profile is applied a backup of the configuration file\nwill be created in `$IPFS_PATH`.\n\nConfiguration profiles can be applied additively. For example, both the `unixfs-v1-2025` and `lowpower` profiles can be applied one after the other.\nThe available configuration profiles are listed below. You can also find them\ndocumented in `ipfs config profile --help`.\n\n### `server` profile\n\nDisables local [`Discovery.MDNS`](#discoverymdns), [turns off uPnP NAT port mapping](#swarmdisablenatportmap), and blocks connections to\nIPv4 and IPv6 prefixes that are [private, local only, or unrouteable](https://github.com/ipfs/kubo/blob/b71cf0d15904bdef21fe2eee5f1118a274309a4d/config/profile.go#L24-L43).\n\nRecommended when running IPFS on machines with public IPv4 addresses (no NAT, no uPnP)\nat providers that interpret local IPFS discovery and traffic as netscan abuse ([example](https://github.com/ipfs/kubo/issues/10327)).\n\n### `randomports` profile\n\nUse a random port number for the incoming swarm connections.\nUsed for testing.\n\n### `default-datastore` profile\n\nConfigures the node to use the default datastore (flatfs).\n\nRead the \"flatfs\" profile description for more information on this datastore.\n\nThis profile may only be applied when first initializing the node.\n\n### `local-discovery` profile\n\nEnables local [`Discovery.MDNS`](#discoverymdns) (enabled by default).\n\nUseful to re-enable local discovery after it's disabled by another profile\n(e.g., the server profile).\n\n`test` profile\n\nReduces external interference of IPFS daemon, this\nis useful when using the daemon in test environments.\n\n### `default-networking` profile\n\nRestores default network settings.\nInverse profile of the test profile.\n\n### `autoconf-on` profile\n\nSafe default for joining the public IPFS Mainnet swarm with automatic configuration.\nCan also be used with custom AutoConf.URL for other networks.\n\n### `autoconf-off` profile\n\nDisables AutoConf and clears all networking fields for manual configuration.\nUse this for private networks or when you want explicit control over all endpoints.\n\n### `flatfs` profile\n\nConfigures the node to use the flatfs datastore.\nFlatfs is the default, most battle-tested and reliable datastore.\n\nYou should use this datastore if:\n\n- You need a very simple and very reliable datastore, and you trust your\n filesystem. This datastore stores each block as a separate file in the\n underlying filesystem so it's unlikely to lose data unless there's an issue\n with the underlying file system.\n- You need to run garbage collection in a way that reclaims free space as soon as possible.\n- You want to minimize memory usage.\n- You are ok with the default speed of data import, or prefer to use `--nocopy`.\n\n> [!WARNING]\n> This profile may only be applied when first initializing the node via `ipfs init --profile flatfs`\n\n> [!NOTE]\n> See caveats and configuration options at [`datastores.md#flatfs`](datastores.md#flatfs)\n\n### `flatfs-measure` profile\n\nConfigures the node to use the flatfs datastore with metrics. This is the same as [`flatfs` profile](#flatfs-profile) with the addition of the `measure` datastore wrapper.\n\n### `pebbleds` profile\n\nConfigures the node to use the pebble high-performance datastore.\n\nPebble is a LevelDB/RocksDB inspired key-value store focused on performance and internal usage by CockroachDB.\nYou should use this datastore if:\n\n- You need a datastore that is focused on performance.\n- You need a datastore that is good for multi-terabyte data sets.\n- You need reliability by default, but may choose to disable WAL for maximum performance when reliability is not critical.\n- You want a datastore that does not need GC cycles and does not use more space than necessary\n- You want a datastore that does not take several minutes to start with large repositories\n- You want a datastore that performs well even with default settings, but can optimized by setting configuration to tune it for your specific needs.\n\n> [!WARNING]\n> This profile may only be applied when first initializing the node via `ipfs init --profile pebbleds`\n\n> [!NOTE]\n> See other caveats and configuration options at [`datastores.md#pebbleds`](datastores.md#pebbleds)\n\n### `pebbleds-measure` profile\n\nConfigures the node to use the pebble datastore with metrics. This is the same as [`pebbleds` profile](#pebble-profile) with the addition of the `measure` datastore wrapper.\n\n### `badgerds` profile\n\nConfigures the node to use the **legacy** badgerv1 datastore.\n\n> [!CAUTION]\n> **Badger v1 datastore is deprecated and will be removed in a future Kubo release.**\n>\n> This is based on very old badger 1.x, which has not been maintained by its\n> upstream maintainers for years and has known bugs (startup timeouts, shutdown\n> hangs, file descriptor\n> exhaustion, and more). Do not use it for new deployments.\n>\n> **To migrate:** create a new `IPFS_PATH` with `flatfs`\n> (`ipfs init --profile=flatfs`), move pinned data via\n> `ipfs dag export/import` or `ipfs pin ls -t recursive|add`, and decommission the\n> old badger-based node. When it comes to block storage, use experimental\n> `pebbleds` only if you are sure modern `flatfs` does not serve your use case\n> (most users will be perfectly fine with `flatfs`, it is also possible to keep\n> `flatfs` for blocks and replace `leveldb` with `pebble` if preferred over\n> `leveldb`).\n\nAlso, be aware that:\n\n- This datastore will not properly reclaim space when your datastore is\n smaller than several gigabytes. If you run IPFS with `--enable-gc`, you plan on storing very little data in\n your IPFS node, and disk usage is more critical than performance, consider using\n `flatfs`.\n- This datastore uses up to several gigabytes of memory.\n- Good for medium-size datastores, but may run into performance issues if your dataset is bigger than a terabyte.\n\n> [!WARNING]\n> This profile may only be applied when first initializing the node via `ipfs init --profile badgerds`\n\n> [!NOTE]\n> See other caveats and configuration options at [`datastores.md#badgerds`](datastores.md#badgerds)\n\n### `badgerds-measure` profile\n\nConfigures the node to use the **legacy** badgerv1 datastore with metrics. This is the same as [`badgerds` profile](#badger-profile) with the addition of the `measure` datastore wrapper. This profile will be removed in a future Kubo release.\n\n### `lowpower` profile\n\nReduces daemon overhead on the system by disabling optional swarm services.\n\n- [`Routing.Type`](#routingtype) set to `autoclient` (no DHT server, only client).\n- `Swarm.ConnMgr` set to maintain minimum number of p2p connections at a time.\n- Disables [`AutoNAT`](#autonat).\n- Disables [`Swam.RelayService`](#swarmrelayservice).\n\n> [!NOTE]\n> This profile is provided for legacy reasons.\n> With modern Kubo setting the above should not be necessary.\n\n### `announce-off` profile\n\nDisables [Provide](#provide) system (and announcing to Amino DHT).\n\n> [!CAUTION]\n> The main use case for this is setups with manual Peering.Peers config.\n> Data from this node will not be announced on the DHT. This will make\n> DHT-based routing an data retrieval impossible if this node is the only\n> one hosting it, and other peers are not already connected to it.\n\n### `announce-on` profile\n\n(Re-)enables [Provide](#provide) system (reverts [`announce-off` profile](#announce-off-profile)).\n\n### `unixfs-v0-2015` profile\n\nLegacy UnixFS import profile for backward-compatible CID generation.\nProduces CIDv0 with no raw leaves, sha2-256, 256 KiB chunks, and\nlink-based HAMT size estimation.\n\nSee for exact [`Import.*`](#import) settings.\n\n> [!NOTE]\n> Use only when legacy CIDs are required. For new projects, use [`unixfs-v1-2025`](#unixfs-v1-2025-profile).\n>\n> See [IPIP-499](https://specs.ipfs.tech/ipips/ipip-0499/) for more details.\n\n### `legacy-cid-v0` profile\n\nAlias for [`unixfs-v0-2015`](#unixfs-v0-2015-profile) profile.\n\n### `unixfs-v1-2025` profile\n\nRecommended UnixFS import profile for cross-implementation CID determinism.\nUses CIDv1, raw leaves, sha2-256, 1 MiB chunks, 1024 links per file node,\n256 HAMT fanout, and block-based size estimation for HAMT threshold.\n\nSee for exact [`Import.*`](#import) settings.\n\n> [!NOTE]\n> This profile ensures CID consistency across different IPFS implementations.\n>\n> See [IPIP-499](https://specs.ipfs.tech/ipips/ipip-0499/) for more details.\n\n## Security\n\nThis section provides an overview of security considerations for configurations that expose network services.\n\n### Port and Network Exposure\n\nSeveral configuration options expose TCP or UDP ports that can make your Kubo node accessible from the network:\n\n- **[`Addresses.API`](#addressesapi)** - Exposes the admin RPC API (default: localhost:5001)\n- **[`Addresses.Gateway`](#addressesgateway)** - Exposes the HTTP gateway (default: localhost:8080)\n- **[`Addresses.Swarm`](#addressesswarm)** - Exposes P2P connectivity (default: 0.0.0.0:4001, both UDP and TCP)\n- **[`Swarm.Transports.Network`](#swarmtransportsnetwork)** - Controls which P2P transport protocols are enabled over TCP and UDP\n\n### Security Best Practices\n\n- Keep admin services ([`Addresses.API`](#addressesapi)) bound to localhost unless authentication ([`API.Authorizations`](#apiauthorizations)) is configured\n- Use [`Gateway.NoFetch`](#gatewaynofetch) to prevent arbitrary CID retrieval if Kubo is acting as a public gateway available to anyone\n- Configure firewall rules to restrict access to exposed ports. Note that [`Addresses.Swarm`](#addressesswarm) is special - all incoming traffic to swarm ports should be allowed to ensure proper P2P connectivity\n- Control which public-facing addresses are announced to other peers using [`Addresses.NoAnnounce`](#addressesnoannounce), [`Addresses.Announce`](#addressesannounce), and [`Addresses.AppendAnnounce`](#addressesappendannounce)\n- Consider using the [`server` profile](#server-profile) for production deployments\n\n## Types\n\nThis document refers to the standard JSON types (e.g., `null`, `string`,\n`number`, etc.), as well as a few custom types, described below.\n\n### `flag`\n\nFlags allow enabling and disabling features. However, unlike simple booleans,\nthey can also be `null` (or omitted) to indicate that the default value should\nbe chosen. This makes it easier for Kubo to change the defaults in the\nfuture unless the user _explicitly_ sets the flag to either `true` (enabled) or\n`false` (disabled). Flags have three possible states:\n\n- `null` or missing (apply the default value).\n- `true` (enabled)\n- `false` (disabled)\n\n### `priority`\n\nPriorities allow specifying the priority of a feature/protocol and disabling the\nfeature/protocol. Priorities can take one of the following values:\n\n- `null`/missing (apply the default priority, same as with flags)\n- `false` (disabled)\n- `1 - 2^63` (priority, lower is preferred)\n\n### `strings`\n\nStrings is a special type for conveniently specifying a single string, an array\nof strings, or null:\n\n- `null`\n- `\"a single string\"`\n- `[\"an\", \"array\", \"of\", \"strings\"]`\n\n### `duration`\n\nDuration is a type for describing lengths of time, using the same format go\ndoes (e.g, `\"1d2h4m40.01s\"`).\n\n### `optionalInteger`\n\nOptional integers allow specifying some numerical value which has\nan implicit default when missing from the config file:\n\n- `null`/missing will apply the default value defined in Kubo sources (`.WithDefault(value)`)\n- an integer between `-2^63` and `2^63-1` (i.e. `-9223372036854775808` to `9223372036854775807`)\n\n### `optionalBytes`\n\nOptional Bytes allow specifying some number of bytes which has\nan implicit default when missing from the config file:\n\n- `null`/missing (apply the default value defined in Kubo sources)\n- a string value indicating the number of bytes, including human readable representations:\n - [SI sizes](https://en.wikipedia.org/wiki/Metric_prefix#List_of_SI_prefixes) (metric units, powers of 1000), e.g. `1B`, `2kB`, `3MB`, `4GB`, `5TB`, …)\n - [IEC sizes](https://en.wikipedia.org/wiki/Binary_prefix#IEC_prefixes) (binary units, powers of 1024), e.g. `1B`, `2KiB`, `3MiB`, `4GiB`, `5TiB`, …)\n- a raw number (will be interpreted as bytes, e.g. `1048576` for 1MiB)\n\n### `optionalString`\n\nOptional strings allow specifying some string value which has\nan implicit default when missing from the config file:\n\n- `null`/missing will apply the default value defined in Kubo sources (`.WithDefault(\"value\")`)\n- a string\n\n### `optionalDuration`\n\nOptional durations allow specifying some duration value which has\nan implicit default when missing from the config file:\n\n- `null`/missing will apply the default value defined in Kubo sources (`.WithDefault(\"1h2m3s\")`)\n- a string with a valid [go duration](#duration) (e.g, `\"1d2h4m40.01s\"`).\n\n----\n\n[multiaddr]: https://docs.ipfs.tech/concepts/glossary/#multiaddr\n" }, { "path": "docs/content-blocking.md", "content": "

\n
\n \"content\n
\n Content Blocking in Kubo\n
\n

\n\nKubo ships with built-in support for denylist format from [IPIP-383](https://specs.ipfs.tech/ipips/ipip-0383/).\n\n## Default behavior\n\nOfficial Kubo build does not ship with any denylists enabled by default.\n\nContent blocking is an opt-in decision made by the operator of `ipfs daemon`.\n\n## How to enable blocking\n\nPlace a `*.deny` file in one of directories:\n\n- `$IPFS_PATH/denylists/` (`$HOME/.ipfs/denylists/` if `IPFS_PATH` is not set)\n- `$XDG_CONFIG_HOME/ipfs/denylists/` (`$HOME/.config/ipfs/denylists/` if `XDG_CONFIG_HOME` is not set)\n- `/etc/ipfs/denylists/` (global)\n\nFiles need to be present before starting the `ipfs daemon` in order to be watched for any new updates \nappended once started. Any other changes (such as removal of entries, prepending of entries, or \ninsertion of new entries before the EOF at time of daemon starting) will not be detected or processed\nafter boot; a restart of the daemon will be required for them to be factored in.\n\nIf an entire new denylist file is added, `ipfs daemon` also needs to be restarted to track it.\n\nCLI and Gateway users will receive errors in response to request impacted by a blocklist:\n\n```\nError: /ipfs/QmQvjk82hPkSaZsyJ8vNER5cmzKW7HyGX5XVusK7EAenCN is blocked and cannot be provided\n```\n\nEnd user is not informed about the exact reason, see [How to\ndebug](#how-to-debug) if you need to find out which line of which denylist\ncaused the request to be blocked.\n\n## Denylist file format\n\n[NOpfs](https://github.com/ipfs-shipyard/nopfs) supports the format from [IPIP-383](https://specs.ipfs.tech/ipips/ipip-0383/).\n\nClear-text rules are simple: just put content paths to block, one per line.\nPaths with unicode and whitespace need to be percent-encoded:\n\n```\n/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\n/ipfs/bafybeihfg3d7rdltd43u3tfvncx7n5loqofbsobojcadtmokrljfthuc7y/927%20-%20Standards/927%20-%20Standards.png\n```\n\nSensitive content paths can be double-hashed to block without revealing them.\nDouble-hashed list example: https://badbits.dwebops.pub/badbits.deny\n\nSee [IPIP-383](https://specs.ipfs.tech/ipips/ipip-0383/) for detailed format specification and more examples.\n\n## How to suspend blocking without removing denylists\n\nSet `IPFS_CONTENT_BLOCKING_DISABLE` environment variable to `true` and restart the daemon.\n\n\n## How to debug\n\nDebug logging of `nopfs` subsystem can be enabled with `GOLOG_LOG_LEVEL=\"nopfs=debug\"`\n\nAll block events are logged as warnings on a separate level named `nopfs-blocks`.\n\nTo only log requests for blocked content set `GOLOG_LOG_LEVEL=\"nopfs-blocks=warn\"`:\n\n```\nWARN (...) QmRFniDxwxoG2n4AcnGhRdjqDjCM5YeUcBE75K8WXmioH3: blocked (test.deny:9)\n```\n\n\n" }, { "path": "docs/customizing.md", "content": "# Customizing Kubo\n\nYou may want to customize Kubo if you want to reuse most of Kubo's machinery. This document discusses some approaches you may consider for customizing Kubo, and their tradeoffs.\n\nSome common use cases for customizing Kubo include:\n\n- Using a custom datastore for storing blocks, pins, or other Kubo metadata\n- Adding a custom data transfer protocol into Kubo\n- Customizing Kubo internals, such as adding allowlist/blocklist functionality to Bitswap\n- Adding new commands, interfaces, functionality, etc. to Kubo while reusing the libp2p swarm\n- Building on top of Kubo's configuration and config migration functionality\n\n## Summary\nThis table summarizes the tradeoffs between the approaches below:\n\n| | [Boxo](#boxo-build-your-own-binary) | [Kubo Plugin](#kubo-plugins) | [Bespoke Extension Point](#bespoke-extension-points) | [Go Plugin](#go-plugins) | [Fork](#fork-kubo) |\n|:-------------------:|:-----------------------------------:|:----------------------------:|:----------------------------------------------------:|:------------------------:|:------------------:|\n| Supported? | ✅ | ✅ | ✅ | ❌ | ❌ |\n| Future-proof? | ✅ | ❌ | ✅ | ❌ | ❌ |\n| Fully customizable? | ✅ | ✅ | ❌ | ✅ | ✅ |\n| Fast to implement? | ❌ | ✅ | ✅ | ✅ | ✅ |\n| Dynamic at runtime? | ❌ | ❌ | ✅ | ✅ | ❌ |\n| Add new commands? | ❌ | ✅ | ❌ | ✅ | ✅ |\n\n## Boxo: build your own binary\nThe best way to reuse Kubo functionality is to pick the functionality you need directly from [Boxo](https://github.com/ipfs/boxo) and compile your own binary.\n\nBoxo's raison d'etre is to be an IPFS component toolbox to support building custom-made implementations and applications. If your use case is not easy to implement with Boxo, you may want to consider adding whatever functionality is needed to Boxo instead of customizing Kubo, so that the community can benefit. If you are interested in this option, please reach out to Boxo maintainers, who will be happy to help you scope & plan the work. See [Boxo's FAQ](https://github.com/ipfs/boxo#help) for more info.\n\n## Kubo Plugins\nKubo plugins are a set of interfaces that may be implemented and injected into Kubo. Generally you should recompile the Kubo binary with your plugins added. A popular example of a Kubo plugin is [go-ds-s3](https://github.com/ipfs/go-ds-s3), which can be used to store blocks in Amazon S3.\n\nSome plugins, such as the `fx` plugin, allow deep customization of Kubo internals. As a result, Kubo maintainers can't guarantee backwards compatibility with these, so you may need to adapt to breaking changes when upgrading to new Kubo versions.\n\nFor more information about the different types of Kubo plugins, see [plugins.md](./plugins.md).\n\nKubo plugins can also be injected at runtime using Go plugins (see below), but these are hard to use and not well supported by Go, so we don't recommend them.\n\n### Kubo binary imports\n\nIt is possible to depend on the package `cmd/ipfs/kubo` as a way of using Kubo plugins that is an alternative to recompiling Kubo with additional preloaded plugins.\n\nThis gives a more Go-centric dependency updating flow to building a new binary with preloaded plugins by simply requiring updating a Kubo dependency rather than needing to update Kubo source code and recompile.\n\n## Bespoke Extension Points\nCertain Kubo functionality may have their own extension points. For example:\n\n* Kubo supports the [Routing v1](https://specs.ipfs.tech/routing/http-routing-v1/) API for delegating content routing to external processes\n* Kubo supports the [Pinning Service API](https://github.com/ipfs/pinning-services-api-spec) for delegating pinning to external processes\n* Kubo supports [DNSLink](https://dnslink.dev/) for delegating name->CID mappings to DNS\n\n(This list is not exhaustive.)\n\nThese can generally be developed and deployed as sidecars (or full external services) without modifying the Kubo binary.\n\n## Go Plugins\nGo provides [dynamic plugins](https://pkg.go.dev/plugin) which can be loaded at runtime into a Go binary.\n\nKubo currently works with Go plugins. But using Go plugins requires that you compile the plugin using the exact same version of the Go toolchain with the same configuration (build flags, environment variables, etc.). As a result, you likely need to build Kubo and the plugins together at the same time, and at that point you may as well just compile the functionality directly into Kubo and avoid Go plugins.\n\nAs a result, we don't recommend using Go plugins, and are likely to remove them in a future release of Kubo.\n\n## Fork Kubo\nThe \"nuclear option\" is to fork Kubo into your own repo, make your changes, and periodically sync your repo with the Kubo repo. This can be a good option if your changes are significant and you can commit to keeping your repo in sync with Kubo.\n\nKubo maintainers can't make any backwards compatibility guarantees about Kubo internals, so by choosing this option you're accepting the risk that you may need to spend more time adapting to breaking changes.\n" }, { "path": "docs/datastores.md", "content": "# Datastore Configuration Options\n\nThis document describes the different possible values for the `Datastore.Spec`\nfield in the ipfs configuration file.\n\n- [flatfs](#flatfs)\n- [levelds](#levelds)\n- [pebbleds](#pebbleds)\n- [badgerds](#badgerds)\n- [mount](#mount)\n- [measure](#measure)\n\n## flatfs\n\nStores each key-value pair as a file on the filesystem.\n\nThe shardFunc is prefixed with `/repo/flatfs/shard/v1` then followed by a descriptor of the sharding strategy. Some example values are:\n- `/repo/flatfs/shard/v1/next-to-last/2`\n - Shards on the two next to last characters of the key\n- `/repo/flatfs/shard/v1/prefix/2`\n - Shards based on the two-character prefix of the key\n\n```json\n{\n\t\"type\": \"flatfs\",\n\t\"path\": \"\",\n\t\"shardFunc\": \"\",\n\t\"sync\": true|false\n}\n```\n\n- `sync`: Flush every write to disk before continuing. Setting this to false is safe as kubo will automatically flush writes to disk before and after performing critical operations like pinning. However, you can set this to true to be extra-safe (at the cost of a slowdown when adding files).\n\nNOTE: flatfs must only be used as a block store (mounted at `/blocks`) as it only partially implements the datastore interface. You can mount flatfs for /blocks only using the mount datastore (described below).\n\n## levelds\n\nUses a [leveldb](https://github.com/syndtr/goleveldb) database to store key-value\npairs via [go-ds-leveldb](https://github.com/ipfs/go-ds-leveldb).\n\n```json\n{\n\t\"type\": \"levelds\",\n\t\"path\": \"\",\n\t\"compression\": \"none\" | \"snappy\",\n}\n```\n\n> [!NOTE]\n> LevelDB uses a log-structured merge-tree (LSM) storage engine. When keys are\n> deleted, the data is not removed immediately. Instead, a tombstone marker is\n> written, and the actual data is removed later by background compaction.\n>\n> LevelDB's compaction decides what to compact based on file counts (L0) and\n> total level size (L1+), without considering how many tombstones a file\n> contains. This means that after bulk deletions (such as pin removals or the\n> periodic provider keystore sync), disk space may not be reclaimed promptly.\n> The `datastore/` directory can grow significantly larger than the live data it\n> holds, especially on long-running nodes with many CIDs.\n>\n> Unlike flatfs (which deletes files immediately) or pebble (which has\n> tombstone-aware compaction), LevelDB has no way to prioritize reclaiming\n> space from deleted keys. Restarting the daemon may trigger some compaction,\n> but this is not guaranteed.\n>\n> If slow compaction is a problem, consider using the `pebbleds` datastore\n> instead (see below), which handles this workload more efficiently.\n\n## pebbleds\n\nUses [pebble](https://github.com/cockroachdb/pebble) as a key-value store.\n\n```json\n{\n\t\"type\": \"pebbleds\",\n\t\"path\": \"\",\n}\n```\n\nThe following options are available for tuning pebble.\nIf they are not configured (or assigned their zero-valued), then default values are used.\n\n* `bytesPerSync`: int, Sync sstables periodically in order to smooth out writes to disk. (default: 512KB)\n* `disableWAL`: true|false, Disable the write-ahead log (WAL) at expense of prohibiting crash recovery. (default: false)\n* `cacheSize`: Size of pebble's shared block cache. (default: 8MB)\n* `formatVersionMajor`: int, Sets the format of pebble on-disk files. If 0 or unset, automatically convert to latest format.\n* `l0CompactionThreshold`: int, Count of L0 files necessary to trigger an L0 compaction.\n* `l0StopWritesThreshold`: int, Limit on L0 read-amplification, computed as the number of L0 sublevels.\n* `lBaseMaxBytes`: int, Maximum number of bytes for LBase. The base level is the level which L0 is compacted into.\n* `maxConcurrentCompactions`: int, Maximum number of concurrent compactions. (default: 1)\n* `memTableSize`: int, Size of a MemTable in steady state. The actual MemTable size starts at min(256KB, MemTableSize) and doubles for each subsequent MemTable up to MemTableSize (default: 4MB)\n* `memTableStopWritesThreshold`: int, Limit on the number of queued of MemTables. (default: 2)\n* `walBytesPerSync`: int: Sets the number of bytes to write to a WAL before calling Sync on it in the background. (default: 0, no background syncing)\n* `walMinSyncSeconds`: int: Sets the minimum duration between syncs of the WAL. (default: 0)\n\n> [!TIP]\n> Start using pebble with only default values and configure tuning items are needed for your needs. For a more complete description of these values, see: `https://pkg.go.dev/github.com/cockroachdb/pebble@vA.B.C#Options` (where `A.B.C` is pebble version from Kubo's `go.mod`).\n\nUsing a pebble datastore can be set when initializing kubo `ipfs init --profile pebbleds`.\n\n#### Use of `formatMajorVersion`\n\n[Pebble's `FormatMajorVersion`](https://github.com/cockroachdb/pebble/tree/master?tab=readme-ov-file#format-major-versions) is a constant controlling the format of persisted data. Backwards incompatible changes to durable formats are gated behind new format major versions.\n\nAt any point, a database's format major version may be bumped. However, once a database's format major version is increased, previous versions of Pebble will refuse to open the database.\n\nWhen IPFS is initialized to use the pebbleds datastore (`ipfs init --profile=pebbleds`), the latest pebble database format is configured in the pebble datastore config as `\"formatMajorVersion\"`. Setting this in the datastore config prevents automatically upgrading to the latest available version when kubo is upgraded. If a later version becomes available, the kubo daemon prints a startup message to indicate this. The user can them update the config to use the latest format when they are certain a downgrade will not be necessary.\n\nWithout the `\"formatMajorVersion\"` in the pebble datastore config, the database format is automatically upgraded to the latest version. If this happens, then it is possible a downgrade back to the previous version of kubo will not work if new format is not compatible with the pebble datastore in the previous version of kubo.\n\nWhen installing a new version of kubo when `\"formatMajorVersion\"` is configured, migration does not upgrade this to the latest available version. This is done because a user may have reasons not to upgrade the pebble database format, and may want to be able to downgrade kubo if something else is not working in the new version. If the configured pebble database format in the old kubo is not supported in the new kubo, then the configured version must be updated and the old kubo run, before installing the new kubo.\n\n## badgerds\n\nUses [badger](https://github.com/dgraph-io/badger) as a key-value store.\n\n> [!CAUTION]\n> **Badger v1 datastore is deprecated and will be removed in a future Kubo release.**\n>\n> This is based on very old badger 1.x, which has not been maintained by its\n> upstream maintainers for years and has known bugs (startup timeouts, shutdown\n> hangs, file descriptor\n> exhaustion, and more). Do not use it for new deployments.\n>\n> **To migrate:** create a new `IPFS_PATH` with `flatfs`\n> (`ipfs init --profile=flatfs`), move pinned data via\n> `ipfs dag export/import` or `ipfs pin ls -t recursive|add`, and decommission the\n> old badger-based node. When it comes to block storage, use experimental\n> `pebbleds` only if you are sure modern `flatfs` does not serve your use case\n> (most users will be perfectly fine with `flatfs`, it is also possible to keep\n> `flatfs` for blocks and replace `leveldb` with `pebble` if preferred over\n> `leveldb`).\n\n- `syncWrites`: Flush every write to disk before continuing. Setting this to false is safe as kubo will automatically flush writes to disk before and after performing critical operations like pinning. However, you can set this to true to be extra-safe (at the cost of a 2-3x slowdown when adding files).\n- `truncate`: Truncate the DB if a partially written sector is found (defaults to true). There is no good reason to set this to false unless you want to manually recover partially written (and unpinned) blocks if kubo crashes half-way through a write operation.\n\n```json\n{\n\t\"type\": \"badgerds\",\n\t\"path\": \"\",\n\t\"syncWrites\": true|false,\n\t\"truncate\": true|false,\n}\n```\n\n## mount\n\nAllows specified datastores to handle keys prefixed with a given path.\nThe mountpoints are added as keys within the child datastore definitions.\n\n```json\n{\n\t\"type\": \"mount\",\n\t\"mounts\": [\n\t\t{\n\t\t\t// Insert other datastore definition here, but add the following key:\n\t\t\t\"mountpoint\": \"/path/to/handle\"\n\t\t},\n\t\t{\n\t\t\t// Insert other datastore definition here, but add the following key:\n\t\t\t\"mountpoint\": \"/path/to/handle\"\n\t\t},\n\t]\n}\n```\n\n## measure\n\nThis datastore is a wrapper that adds metrics tracking to any datastore.\n\n```json\n{\n\t\"type\": \"measure\",\n\t\"prefix\": \"sometag.datastore\",\n\t\"child\": { datastore being wrapped }\n}\n```\n\n" }, { "path": "docs/debug-guide.md", "content": "# General performance debugging guidelines\n\nThis is a document for helping debug Kubo. Please add to it if you can!\n\n# Table of Contents\n\n- [General performance debugging guidelines](#general-performance-debugging-guidelines)\n- [Table of Contents](#table-of-contents)\n - [Beginning](#beginning)\n - [Analyzing the stack dump](#analyzing-the-stack-dump)\n - [Analyzing the CPU Profile](#analyzing-the-cpu-profile)\n - [Analyzing vars and memory statistics](#analyzing-vars-and-memory-statistics)\n - [Tracing](#tracing)\n - [Other](#other)\n\n### Beginning\n\n> **Note:** Enable more logs by setting `GOLOG_LOG_LEVEL` env variable when troubleshooting. See [go-log documentation](https://github.com/ipfs/go-log#golog_log_level) for configuration options and available log levels.\n\nWhen you see ipfs doing something (using lots of CPU, memory, or otherwise\nbeing weird), the first thing you want to do is gather all the relevant\nprofiling information.\n\nThere's a command (`ipfs diag profile`) that will do this for you and\nbundle the results up into a zip file, ready to be attached to a bug report.\n\nIf you feel intrepid, you can dump this information and investigate it yourself:\n\n- goroutine dump\n - `curl localhost:5001/debug/pprof/goroutine\\?debug=2 > ipfs.stacks`\n- 30 second cpu profile\n - `curl localhost:5001/debug/pprof/profile > ipfs.cpuprof`\n- heap trace dump\n - `curl localhost:5001/debug/pprof/heap > ipfs.heap`\n- memory statistics (in json, see \"memstats\" object)\n - `curl localhost:5001/debug/vars > ipfs.vars`\n- system information\n - `ipfs diag sys > ipfs.sysinfo`\n\n\n### Analyzing the stack dump\n\nThe first thing to look for is hung goroutines -- any goroutine that's been stuck\nfor over a minute will note that in the trace. It looks something like:\n\n```\ngoroutine 2306090 [semacquire, 458 minutes]:\nsync.runtime_Semacquire(0xc8222fd3e4)\n /home/whyrusleeping/go/src/runtime/sema.go:47 +0x26\nsync.(*Mutex).Lock(0xc8222fd3e0)\n /home/whyrusleeping/go/src/sync/mutex.go:83 +0x1c4\ngx/ipfs/QmedFDs1WHcv3bcknfo64dw4mT1112yptW1H65Y2Wc7KTV/yamux.(*Session).Close(0xc8222fd340, 0x0, 0x0)\n /home/whyrusleeping/gopkg/src/gx/ipfs/QmedFDs1WHcv3bcknfo64dw4mT1112yptW1H65Y2Wc7KTV/yamux/session.go:205 +0x55\ngx/ipfs/QmWSJzRkCMJFHYUQZxKwPX8WA7XipaPtfiwMPARP51ymfn/go-stream-muxer/yamux.(*conn).Close(0xc8222fd340, 0x0, 0x0)\n /home/whyrusleeping/gopkg/src/gx/ipfs/QmWSJzRkCMJFHYUQZxKwPX8WA7XipaPtfiwMPARP51ymfn/go-stream-muxer/yamux/yamux.go:39 +0x2d\ngx/ipfs/QmZK81vcgMhpb2t7GNbozk7qzt6Rj4zFqitpvsWT9mduW8/go-peerstream.(*Conn).Close(0xc8257a2000, 0x0, 0x0)\n /home/whyrusleeping/gopkg/src/gx/ipfs/QmZK81vcgMhpb2t7GNbozk7qzt6Rj4zFqitpvsWT9mduW8/go-peerstream/conn.go:156 +0x1f2\ncreated by gx/ipfs/QmZK81vcgMhpb2t7GNbozk7qzt6Rj4zFqitpvsWT9mduW8/go-peerstream.(*Conn).GoClose\n /home/whyrusleeping/gopkg/src/gx/ipfs/QmZK81vcgMhpb2t7GNbozk7qzt6Rj4zFqitpvsWT9mduW8/go-peerstream/conn.go:131 +0xab\n```\n\nAt the top, you can see that this goroutine (number 2306090) has been waiting\nto acquire a semaphore for 458 minutes. That seems bad. Looking at the rest of\nthe trace, we see the exact line it's waiting on is line 47 of runtime/sema.go.\nThat's not particularly helpful, so we move on. Next, we see that call was made\nby line 205 of yamux/session.go in the `Close` method of `yamux.Session`. This\none appears to be the issue.\n\nGiven that information, look for another goroutine that might be\nholding the semaphore in question in the rest of the stack dump.\n(If you need help doing this, ping and we'll stub this out.)\n\nThere are a few different reasons that goroutines can be hung:\n- `semacquire` means we're waiting to take a lock or semaphore.\n- `select` means that the goroutine is hanging in a select statement and none of\n the cases are yielding anything.\n- `chan receive` and `chan send` are waiting for a channel to be received from\n or sent on, respectively.\n- `IO wait` generally means that we are waiting on a socket to read or write\n data, although it *can* mean we are waiting on a very slow filesystem.\n\nIf you see any of those tags _without_ a `,\nX minutes` suffix, that generally means there isn't a problem -- you just caught\nthat goroutine in the middle of a short wait for something. If the wait time is\nover a few minutes, that either means that goroutine doesn't do much, or\nsomething is pretty wrong.\n\nIf you're seeing a lot of goroutines, consider using\n[stackparse](https://github.com/whyrusleeping/stackparse) to filter, sort, and summarize them.\n\n### Analyzing the CPU Profile\n\nThe go team wrote an [excellent article on profiling go\nprograms](http://blog.golang.org/profiling-go-programs). If you've already\ngathered the above information, you can skip down to where they start talking\nabout `go tool pprof`. My go-to method of analyzing these is to run the `web`\ncommand, which generates an SVG dotgraph and opens it in your browser. This is\nthe quickest way to easily point out where the hot spots in the code are.\n\n### Analyzing vars and memory statistics\n\nThe output is JSON formatted and includes badger store statistics, the command line run, and the output from Go's [runtime.ReadMemStats](https://golang.org/pkg/runtime/#ReadMemStats). The [MemStats](https://golang.org/pkg/runtime/#MemStats) has useful information about memory allocation and garbage collection.\n\n### Tracing\n\nExperimental tracing via OpenTelemetry suite of tools is available.\nSee `tracing/doc.go` for more details.\n\n### Other\n\nIf you have any questions, or want us to analyze some weird kubo behavior,\njust let us know, and be sure to include all the profiling information\nmentioned at the top.\n" }, { "path": "docs/delegated-routing.md", "content": "# Delegated Routing Notes\n\n- Status Date: 2025-12\n\n> [!IMPORTANT]\n> Most users are best served by setting delegated HTTP router URLs in [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters) and `Routing.Type` to `auto` or `autoclient`, rather than using custom routing with `Routing.Routers` and `Routing.Methods` directly.\n>\n> The rest of this documentation describes experimental features intended only for researchers and advanced users.\n\n----\n\n# Custom Multi-Router Configuration (Experimental)\n\n- Start Date: 2022-08-15\n- Related Issues:\n - https://github.com/ipfs/kubo/issues/9188\n - https://github.com/ipfs/kubo/issues/9079\n - https://github.com/ipfs/kubo/pull/9877\n\n> [!CAUTION]\n> **`Routing.Type=custom` with `Routing.Routers` and `Routing.Methods` is EXPERIMENTAL.**\n>\n> This feature is provided for **research and testing purposes only**. It is **not suitable for production use**.\n>\n> - The configuration format and behavior may change without notice between Kubo releases.\n> - Bugs and regressions affecting custom routing may not be prioritized or fixed promptly.\n> - HTTP-only routing configurations (without DHT) cannot reliably provide content to the network (👉️ see [Limitations](#limitations) below).\n>\n> **For production deployments**, use `Routing.Type=auto` (default) or `Routing.Type=autoclient` with [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters).\n\n## Motivation\n\nThe actual routing implementation is not enough. Some users need to have more options when configuring the routing system. The new implementations should be able to:\n\n- [x] Be user-friendly and easy enough to configure, but also versatile\n- [x] Configurable Router execution order\n - [x] Delay some of the Router methods execution when they will be executed on parallel\n- [x] Configure which method of a giving router will be used\n- [x] Mark some router methods as mandatory to make the execution fails if that method fails\n\n## Detailed design\n\n### Configuration file description\n\nThe `Routing` configuration section will contain the following keys:\n\n#### Type\n\n`Type` will be still in use to avoid complexity for the user that only wants to use Kubo with the default behavior. We are going to add a new type, `custom`, that will use the new router systems. `none` type will deactivate **all** routers, default dht and delegated ones.\n\n#### Routers\n\n`Routers` will be a key-value list of routers that will be available to use. The key is the router name and the value is all the needed configurations for that router. the `Type` will define the routing kind. The main router types will be `http` and `dht`, but we will implement two special routers used to execute a set of routers in parallel or sequentially: `parallel` router and `sequential` router.\n\nDepending on the routing type, it will use different parameters:\n\n##### HTTP\n\nParams:\n\n- `\"Endpoint\"`: URL of HTTP server with endpoints that implement [Delegated Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) protocol.\n\n##### Amino DHT\n\nParams:\n- `\"Mode\"`: Mode used by the Amino DHT. Possible values: \"server\", \"client\", \"auto\"\n- `\"AcceleratedDHTClient\"`: Set to `true` if you want to use the experimentalDHT.\n- `\"PublicIPNetwork\"`: Set to `true` to create a `WAN` Amino DHT. Set to `false` to create a `LAN` DHT.\n\n##### Parallel\n\nParams:\n- `Routers`: A list of routers that will be executed in parallel:\n - `Name:string`: Name of the router. It should be one of the previously added to `Routers` list.\n - `Timeout:duration`: Local timeout. It accepts strings compatible with Go `time.ParseDuration(string)`. Time will start counting when this specific router is called, and it will stop when the router returns, or we reach the specified timeout.\n - `ExecuteAfter:duration`: Providing this param will delay the execution of that router at the specified time. It accepts strings compatible with Go `time.ParseDuration(string)`.\n - `IgnoreErrors:bool`: It will specify if that router should be ignored if an error occurred.\n- `Timeout:duration`: Global timeout. It accepts strings compatible with Go `time.ParseDuration(string)`.\n##### Sequential\n\nParams:\n- `Routers`: A list of routers that will be executed in order:\n - `Name:string`: Name of the router. It should be one of the previously added to `Routers` list.\n - `Timeout:duration`: Local timeout. It accepts strings compatible with Go `time.ParseDuration(string)`. Time will start counting when this specific router is called, and it will stop when the router returns, or we reach the specified timeout.\n - `IgnoreErrors:bool`: It will specify if that router should be ignored if an error occurred.\n- `Timeout:duration`: Global timeout. It accepts strings compatible with Go `time.ParseDuration(string)`.\n#### Methods\n\n`Methods:map` will define which routers will be executed per method. The key will be the name of the method: `\"provide\"`, `\"find-providers\"`, `\"find-peers\"`, `\"put-ipns\"`, `\"get-ipns\"`. All methods must be added to the list. This will make configuration discoverable giving good errors to the user if a method is missing.\n\nThe value will contain:\n- `RouterName:string`: Name of the router. It should be one of the previously added to `Routers` list.\n\n#### Configuration file example:\n\n```json\n\"Routing\": {\n \"Type\": \"custom\",\n \"Routers\": {\n \"http-delegated\": {\n \"Type\": \"http\",\n \"Parameters\": {\n \"Endpoint\": \"https://delegated-ipfs.dev\" // /routing/v1 (https://specs.ipfs.tech/routing/http-routing-v1/)\n }\n },\n \"dht-lan\": {\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"server\",\n \"PublicIPNetwork\": false,\n \"AcceleratedDHTClient\": false\n }\n },\n \"dht-wan\": {\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"auto\",\n \"PublicIPNetwork\": true,\n \"AcceleratedDHTClient\": false\n }\n },\n \"find-providers-router\": {\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\": \"dht-lan\",\n \"IgnoreErrors\": true\n },\n {\n \"RouterName\": \"dht-wan\"\n },\n {\n \"RouterName\": \"http-delegated\"\n }\n ]\n }\n },\n \"provide-router\": {\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\": \"dht-lan\",\n \"IgnoreErrors\": true\n },\n {\n \"RouterName\": \"dht-wan\",\n \"ExecuteAfter\": \"100ms\",\n \"Timeout\": \"100ms\"\n },\n {\n \"RouterName\": \"http-delegated\",\n \"ExecuteAfter\": \"100ms\"\n }\n ]\n }\n },\n \"get-ipns-router\": {\n \"Type\": \"sequential\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\": \"dht-lan\",\n \"IgnoreErrors\": true\n },\n {\n \"RouterName\": \"dht-wan\",\n \"Timeout\": \"300ms\"\n },\n {\n \"RouterName\": \"http-delegated\",\n \"Timeout\": \"300ms\"\n }\n ]\n }\n },\n \"put-ipns-router\": {\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\": \"dht-lan\"\n },\n {\n \"RouterName\": \"dht-wan\"\n },\n {\n \"RouterName\": \"http-delegated\"\n }\n ]\n }\n }\n },\n \"Methods\": {\n \"find-providers\": {\n \"RouterName\": \"find-providers-router\"\n },\n \"provide\": {\n \"RouterName\": \"provide-router\"\n },\n \"get-ipns\": {\n \"RouterName\": \"get-ipns-router\"\n },\n \"put-ipns\": {\n \"RouterName\": \"put-ipns-router\"\n }\n }\n}\n```\n\n### Error cases\n - If any of the routers fails, the output will be an error by default.\n - You can use `IgnoreErrors:true` to ignore errors for a specific router output\n - To avoid any error at the output, you must ignore all router errors.\n\n### Implementation Details\n\n#### Methods\n\nAll routers must implement the `routing.Routing` interface:\n\n```go=\ntype Routing interface {\n ContentRouting\n PeerRouting\n ValueStore\n\n Bootstrap(context.Context) error\n}\n```\n\nAll methods involved:\n\n```go=\ntype Routing interface {\n Provide(context.Context, cid.Cid, bool) error\n FindProvidersAsync(context.Context, cid.Cid, int) <-chan peer.AddrInfo\n \n FindPeer(context.Context, peer.ID) (peer.AddrInfo, error)\n\n PutValue(context.Context, string, []byte, ...Option) error\n GetValue(context.Context, string, ...Option) ([]byte, error)\n SearchValue(context.Context, string, ...Option) (<-chan []byte, error)\n\n Bootstrap(context.Context) error\n}\n```\nWe can configure which methods will be used per routing implementation. Methods names used in the configuration file will be:\n\n- `Provide`: `\"provide\"`\n- `FindProvidersAsync`: `\"find-providers\"`\n- `FindPeer`: `\"find-peers\"`\n- `PutValue`: `\"put-ipns\"`\n- `GetValue`, `SearchValue`: `\"get-ipns\"`\n- `Bootstrap`: It will be always executed when needed.\n\n#### Routers\n\nWe need to implement the `parallel` and `sequential` routers and stop using `routinghelpers.Tiered` router implementation.\n\nAdd cycle detection to avoid to user some headaches.\n\nAlso we need to implement an internal router, that will define the router used per method.\n\n#### Other considerations\n\n- We need to refactor how DHT routers are created to be able to use and add any amount of custom DHT routers.\n- We need to add a new `custom` router type to be able to use the new routing system.\n- Bitswap WANT broadcasting is not included on this document, but it can be added in next iterations.\n- This document will live in docs/design-notes for historical reasons and future reference.\n\n## Test fixtures\n\nAs test fixtures we can add different use cases here and see how the configuration will look like.\n\n### Mimic previous dual DHT config\n\n```json\n\"Routing\": {\n \"Type\": \"custom\",\n \"Routers\": {\n \"dht-lan\": {\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"server\",\n \"PublicIPNetwork\": false\n }\n },\n \"dht-wan\": {\n \"Type\": \"dht\",\n \"Parameters\": {\n \"Mode\": \"auto\",\n \"PublicIPNetwork\": true\n }\n },\n \"parallel-dht-strict\": {\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\": \"dht-lan\"\n },\n {\n \"RouterName\": \"dht-wan\"\n }\n ]\n }\n },\n \"parallel-dht\": {\n \"Type\": \"parallel\",\n \"Parameters\": {\n \"Routers\": [\n {\n \"RouterName\": \"dht-lan\",\n \"IgnoreError\": true\n },\n {\n \"RouterName\": \"dht-wan\"\n }\n ]\n }\n }\n },\n \"Methods\": {\n \"provide\": {\n \"RouterName\": \"dht-wan\"\n },\n \"find-providers\": {\n \"RouterName\": \"parallel-dht-strict\"\n },\n \"find-peers\": {\n \"RouterName\": \"parallel-dht-strict\"\n },\n \"get-ipns\": {\n \"RouterName\": \"parallel-dht\"\n },\n \"put-ipns\": {\n \"RouterName\": \"parallel-dht\"\n }\n }\n}\n```\n\n### Compatibility\n\n~~We need to create a config migration using [fs-repo-migrations](https://github.com/ipfs/fs-repo-migrations). We should remove the `Routing.Type` param and add the configuration specified [previously](#Mimic-previous-dual-DHT-config).~~\n\nWe don't need to create any config migration! To avoid to the users the hassle of understanding how the new routing system works, we are going to keep the old behavior. We will add the Type `custom` to make available the new Routing system.\n\n### Security\n\nNo new security implications or considerations were found.\n\n### Alternatives\n\nI got ideas from all of the following links to create this design document:\n\n- https://github.com/ipfs/kubo/issues/9079#issuecomment-1211288268\n- https://github.com/ipfs/kubo/issues/9157\n- https://github.com/ipfs/kubo/issues/9079#issuecomment-1205000253\n- https://www.notion.so/pl-strflt/Delegated-Routing-Thoughts-very-very-WIP-0543bc51b1bd4d63a061b0f28e195d38\n- https://gist.github.com/guseggert/effa027ff4cbadd7f67598efb6704d12\n\n### Limitations\n\n#### HTTP-only routing cannot reliably provide content\n\nConfigurations that use only HTTP routers (without any DHT router) are unable to reliably announce content (provider records) to the network.\n\nThis limitation exists because:\n\n1. **No standardized HTTP API for providing**: The [Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) spec only defines read operations (`GET /routing/v1/providers/{cid}`). The write operation (`PUT /routing/v1/providers`) was never standardized.\n\n2. **Legacy experimental API**: The only available HTTP providing mechanism is an undocumented `PUT /routing/v1/providers` request format called `ProvideBitswap`, which is a historical experiment. See [IPIP-526](https://github.com/ipfs/specs/pull/526) for ongoing discussion about formalizing HTTP-based provider announcements.\n\n3. **Provider system integration**: Kubo's default provider system (`Provide.DHT.SweepEnabled=true` since v0.38) is designed for DHT-based providing. When no DHT is configured, the provider system may silently skip HTTP routers or behave unexpectedly.\n\n**Workarounds for testing:**\n\nIf you need to test HTTP providing, you can try:\n\n- Setting `Provide.DHT.SweepEnabled=false` to use the legacy provider system\n- Including at least one DHT router in your custom configuration alongside HTTP routers\n\nThese workarounds are not guaranteed to work across Kubo versions and should not be relied upon for production use.\n\n### Copyright\n\nCopyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).\n" }, { "path": "docs/developer-certificate-of-origin", "content": "Developer Certificate of Origin\nVersion 1.1\n\nCopyright (C) 2004, 2006 The Linux Foundation and its contributors.\n1 Letterman Drive\nSuite D4700\nSan Francisco, CA, 94129\n\nEveryone is permitted to copy and distribute verbatim copies of this\nlicense document, but changing it is not allowed.\n\n\nDeveloper's Certificate of Origin 1.1\n\nBy making a contribution to this project, I certify that:\n\n(a) The contribution was created in whole or in part by me and I\n have the right to submit it under the open source license\n indicated in the file; or\n\n(b) The contribution is based upon previous work that, to the best\n of my knowledge, is covered under an appropriate open source\n license and I have the right under that license to submit that\n work with modifications, whether created in whole or in part\n by me, under the same open source license (unless I am\n permitted to submit under a different license), as indicated\n in the file; or\n\n(c) The contribution was provided directly to me by some other\n person who certified (a), (b) or (c) and I have not modified\n it.\n\n(d) I understand and agree that this project and the contribution\n are public and that a record of the contribution (including all\n personal information I submit with it, including my sign-off) is\n maintained indefinitely and may be redistributed consistent with\n this project or the open source license(s) involved.\n" }, { "path": "docs/developer-guide.md", "content": "# Developer Guide\n\nBy the end of this guide, you will be able to:\n\n- Build Kubo from source\n- Run the test suites\n- Make and verify code changes\n\nThis guide covers the local development workflow. For user documentation, see [docs.ipfs.tech](https://docs.ipfs.tech/).\n\n## Table of Contents\n\n- [Prerequisites](#prerequisites)\n- [Quick Start](#quick-start)\n- [Building](#building)\n- [Running Tests](#running-tests)\n- [Running the Linter](#running-the-linter)\n- [Common Development Tasks](#common-development-tasks)\n- [Code Organization](#code-organization)\n- [Architecture](#architecture)\n- [Troubleshooting](#troubleshooting)\n- [Development Dependencies](#development-dependencies)\n- [Further Reading](#further-reading)\n\n## Prerequisites\n\nBefore you begin, ensure you have:\n\n- **Go** - see `go.mod` for the minimum required version\n- **Git**\n- **GNU Make**\n- **GCC** (optional) - required for CGO (Go's C interop); without it, build with `CGO_ENABLED=0`\n\n## Quick Start\n\n```bash\ngit clone https://github.com/ipfs/kubo.git\ncd kubo\nmake build\n./cmd/ipfs/ipfs version\n```\n\nYou should see output like:\n\n```\nipfs version 0.34.0-dev\n```\n\nThe binary is built to `cmd/ipfs/ipfs`. To install it system-wide:\n\n```bash\nmake install\n```\n\nThis installs the binary to `$GOPATH/bin`.\n\n## Building\n\n| Command | Description |\n|---------|-------------|\n| `make build` | build the `ipfs` binary to `cmd/ipfs/ipfs` |\n| `make install` | install to `$GOPATH/bin` |\n| `make nofuse` | build without FUSE support |\n| `make build CGO_ENABLED=0` | build without CGO (no C compiler needed) |\n\nFor Windows-specific instructions, see [windows.md](windows.md).\n\n## Running Tests\n\nKubo has two types of tests:\n\n- **Unit tests** - test individual packages in isolation. Fast and don't require a running daemon.\n- **End-to-end tests** - spawn real `ipfs` nodes, run actual CLI commands, and test the full system. Slower but catch integration issues.\n\nNote that `go test ./...` runs both unit and end-to-end tests. Use `make test` to run all tests. CI runs unit and end-to-end tests in separate jobs for faster feedback.\n\n\n\nFor end-to-end tests, Kubo has two suites:\n\n- **`test/cli`** - modern Go-based test harness that spawns real `ipfs` nodes and runs actual CLI commands. All new tests should be added here.\n- **`test/sharness`** - legacy bash-based tests. We are slowly migrating these to `test/cli`.\n\nWhen modifying tests: cosmetic changes to `test/sharness` are fine, but if significant rewrites are needed, remove the outdated sharness test and add a modern one to `test/cli` instead.\n\n### Before Running Tests\n\n**Environment requirements**: some legacy tests expect default ports (8080, 5001, 4001) to be free and no mDNS (local network discovery) Kubo service on the LAN. Tests may fail if you have a local Kubo instance running. Before running the full test suite, stop any running `ipfs daemon`.\n\nTwo critical setup steps:\n\n1. **Rebuild after code changes**: if you modify any `.go` files outside of `test/`, you must run `make build` before running integration tests.\n\n2. **Set environment variables**: integration tests use the `ipfs` binary from `PATH` and need an isolated `IPFS_PATH`. Run these commands from the repository root:\n\n```bash\nexport PATH=\"$PWD/cmd/ipfs:$PATH\"\nexport IPFS_PATH=\"$(mktemp -d)\"\n```\n\n### Unit Tests\n\n```bash\ngo test ./...\n```\n\n### CLI Integration Tests (`test/cli`)\n\nThese are Go-based integration tests that invoke the `ipfs` CLI.\n\nInstead of running the entire test suite, you can run a specific test to get faster feedback during development.\n\nRun a specific test (recommended during development):\n\n```bash\ngo test ./test/cli/... -run TestAdd -v\n```\n\nRun all CLI tests:\n\n```bash\ngo test ./test/cli/...\n```\n\nRun a specific test:\n\n```bash\ngo test ./test/cli/... -run TestAdd\n```\n\nRun with verbose output:\n\n```bash\ngo test ./test/cli/... -v\n```\n\n**Common error**: \"version (16) is lower than repos (17)\" means your `PATH` points to an old binary. Check `which ipfs` and rebuild with `make build`.\n\n### Sharness Tests (`test/sharness`)\n\nShell-based integration tests using [sharness](https://github.com/chriscool/sharness) (a portable shell testing framework).\n\n```bash\ncd test/sharness\n```\n\nRun a specific test:\n\n```bash\ntimeout 60s ./t0080-repo.sh\n```\n\nRun with verbose output (this disables automatic cleanup):\n\n```bash\n./t0080-repo.sh -v\n```\n\n**Cleanup**: the `-v` flag disables automatic cleanup. Before re-running tests, kill any dangling `ipfs daemon` processes:\n\n```bash\npkill -f \"ipfs daemon\"\n```\n\n### Full Test Suite\n\n```bash\nmake test # run all tests\nmake test_short # run shorter test suite\n```\n\n## Running the Linter\n\nRun the linter using the Makefile target (not `golangci-lint` directly):\n\n```bash\nmake -O test_go_lint\n```\n\n## Common Development Tasks\n\n### Modifying CLI Commands\n\nAfter editing help text in `core/commands/`, verify the output width:\n\n```bash\ngo test ./test/cli/... -run TestCommandDocsWidth\n```\n\n### Updating Dependencies\n\nUse the Makefile target (not `go mod tidy` directly):\n\n```bash\nmake mod_tidy\n```\n\n### Editing the Changelog\n\nWhen modifying `docs/changelogs/`:\n\n- update the Table of Contents when adding sections\n- add user-facing changes to the Highlights section (the Changelog section is auto-generated)\n\n### Running the Daemon\n\nAlways run the daemon with a timeout or shut it down promptly.\n\nWith timeout:\n\n```bash\ntimeout 60s ipfs daemon\n```\n\nOr shut down via API:\n\n```bash\nipfs shutdown\n```\n\nFor multi-step experiments, store `IPFS_PATH` in a file to ensure consistency.\n\n## Code Organization\n\n| Directory | Description |\n|-----------|-------------|\n| `cmd/ipfs/` | CLI entry point and binary |\n| `core/` | core IPFS node implementation |\n| `core/commands/` | CLI command definitions |\n| `core/coreapi/` | Go API implementation |\n| `client/rpc/` | HTTP RPC client |\n| `plugin/` | plugin system |\n| `repo/` | repository management |\n| `test/cli/` | Go-based CLI integration tests |\n| `test/sharness/` | legacy shell-based integration tests |\n| `docs/` | documentation |\n\nKey external dependencies:\n\n- [go-libp2p](https://github.com/libp2p/go-libp2p) - networking stack\n- [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) - distributed hash table\n- [boxo](https://github.com/ipfs/boxo) - IPFS SDK (including Bitswap, the data exchange engine)\n\nFor a deep dive into how code flows through Kubo, see [The `Add` command demystified](add-code-flow.md).\n\n## Architecture\n\n**Map of Implemented Subsystems** ([editable source](https://docs.google.com/drawings/d/1OVpBT2q-NtSJqlPX3buvjYhOnWfdzb85YEsM_njesME/edit)):\n\n\n\n**CLI, HTTP-API, Core Diagram**:\n\n![](./cli-http-api-core-diagram.png)\n\n## Troubleshooting\n\n### \"version (N) is lower than repos (M)\" Error\n\nThis means the `ipfs` binary in your `PATH` is older than expected.\n\nCheck which binary is being used:\n\n```bash\nwhich ipfs\n```\n\nRebuild and verify PATH:\n\n```bash\nmake build\nexport PATH=\"$PWD/cmd/ipfs:$PATH\"\n./cmd/ipfs/ipfs version\n```\n\n### FUSE Issues\n\nIf you don't need FUSE support, build without it:\n\n```bash\nmake nofuse\n```\n\nOr set the `TEST_FUSE=0` environment variable when running tests.\n\n### Build Fails with \"No such file: stdlib.h\"\n\nYou're missing a C compiler. Either install GCC or build without CGO:\n\n```bash\nmake build CGO_ENABLED=0\n```\n\n## Development Dependencies\n\nIf you make changes to the protocol buffers, you will need to install the [protoc compiler](https://github.com/google/protobuf).\n\n## Further Reading\n\n- [The `Add` command demystified](add-code-flow.md) - deep dive into code flow\n- [Configuration reference](config.md)\n- [Performance debugging](debug-guide.md)\n- [Experimental features](experimental-features.md)\n- [Release process](releases.md)\n- [Contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)\n\n## Source Code\n\nThe complete source code is at [github.com/ipfs/kubo](https://github.com/ipfs/kubo).\n" }, { "path": "docs/environment-variables.md", "content": "# Kubo environment variables\n\n- [Variables](#variables)\n - [`IPFS_PATH`](#ipfs_path)\n - [`IPFS_LOGGING`](#ipfs_logging)\n - [`IPFS_LOGGING_FMT`](#ipfs_logging_fmt)\n - [`GOLOG_LOG_LEVEL`](#golog_log_level)\n - [`GOLOG_LOG_FMT`](#golog_log_fmt)\n - [`GOLOG_FILE`](#golog_file)\n - [`GOLOG_OUTPUT`](#golog_output)\n - [`GOLOG_TRACING_FILE`](#golog_tracing_file)\n - [`IPFS_FUSE_DEBUG`](#ipfs_fuse_debug)\n - [`YAMUX_DEBUG`](#yamux_debug)\n - [`IPFS_FD_MAX`](#ipfs_fd_max)\n - [`IPFS_DIST_PATH`](#ipfs_dist_path)\n - [`IPFS_NS_MAP`](#ipfs_ns_map)\n - [`IPFS_HTTP_ROUTERS`](#ipfs_http_routers)\n - [`IPFS_HTTP_ROUTERS_FILTER_PROTOCOLS`](#ipfs_http_routers_filter_protocols)\n - [`IPFS_CONTENT_BLOCKING_DISABLE`](#ipfs_content_blocking_disable)\n - [`IPFS_WAIT_REPO_LOCK`](#ipfs_wait_repo_lock)\n - [`IPFS_TELEMETRY`](#ipfs_telemetry)\n - [`LIBP2P_TCP_REUSEPORT`](#libp2p_tcp_reuseport)\n - [`LIBP2P_TCP_MUX`](#libp2p_tcp_mux)\n - [`LIBP2P_MUX_PREFS`](#libp2p_mux_prefs)\n - [`LIBP2P_RCMGR`](#libp2p_rcmgr)\n - [`LIBP2P_DEBUG_RCMGR`](#libp2p_debug_rcmgr)\n - [`LIBP2P_SWARM_FD_LIMIT`](#libp2p_swarm_fd_limit)\n- [Tracing](#tracing)\n\n# Variables\n\n## `IPFS_PATH`\n\nSets the location of the IPFS repo (where the config, blocks, etc.\nare stored).\n\nDefault: ~/.ipfs\n\n## `IPFS_LOGGING`\n\nSpecifies the log level for Kubo.\n\n`IPFS_LOGGING` is a deprecated alias for the `GOLOG_LOG_LEVEL` environment variable. See below.\n\n## `IPFS_LOGGING_FMT`\n\nSpecifies the log message format.\n\n`IPFS_LOGGING_FMT` is a deprecated alias for the `GOLOG_LOG_FMT` environment variable. See below.\n\n## `GOLOG_LOG_LEVEL`\n\nSpecifies the log-level, both globally and on a per-subsystem basis. Level can be one of:\n\n* `debug`\n* `info`\n* `warn`\n* `error`\n* `dpanic`\n* `panic`\n* `fatal`\n\nPer-subsystem levels can be specified with `subsystem=level`. One global level and one or more per-subsystem levels\ncan be specified by separating them with commas.\n\nDefault: `error`\n\nExample:\n\n```console\nGOLOG_LOG_LEVEL=\"error,core/server=debug\" ipfs daemon\n```\n\nLogging can also be configured at runtime, both globally and on a per-subsystem basis, with the `ipfs log` command.\n\n## `GOLOG_LOG_FMT`\n\nSpecifies the log message format. It supports the following values:\n\n- `color` -- human readable, colorized (ANSI) output\n- `nocolor` -- human readable, plain-text output.\n- `json` -- structured JSON.\n\nFor example, to log structured JSON (for easier parsing):\n\n```bash\nexport GOLOG_LOG_FMT=\"json\"\n```\nThe logging format defaults to `color` when the output is a terminal, and `nocolor` otherwise.\n\n## `GOLOG_FILE`\n\nSets the file to which Kubo logs. By default, Kubo logs to standard error.\n\n## `GOLOG_OUTPUT`\n\nWhen stderr and/or stdout options are configured or specified by the `GOLOG_OUTPUT` environ variable, log only to the output(s) specified. For example:\n\n- `GOLOG_OUTPUT=\"stderr\"` logs only to stderr\n- `GOLOG_OUTPUT=\"stdout\"` logs only to stdout\n- `GOLOG_OUTPUT=\"stderr+stdout\"` logs to both stderr and stdout\n\n## `GOLOG_TRACING_FILE`\n\nSets the file to which Kubo sends tracing events. By default, tracing is\ndisabled.\n\nThis log can be read at runtime (without writing it to a file) using the `ipfs\nlog tail` command.\n\nWarning: Enabling tracing will likely affect performance.\n\n## `IPFS_FUSE_DEBUG`\n\nIf SET, enables fuse debug logging.\n\nDefault: false\n\n## `YAMUX_DEBUG`\n\nIf SET, enables debug logging for the yamux stream muxer.\n\nDefault: false\n\n## `IPFS_FD_MAX`\n\nSets the file descriptor limit for Kubo. If Kubo fails to set the file\ndescriptor limit, it will log an error.\n\nDefaults: 2048\n\n## `IPFS_DIST_PATH`\n\nIPFS Content Path from which Kubo fetches repo migrations (when the daemon\nis launched with the `--migrate` flag).\n\nDefault: `/ipfs/` (the exact path is hardcoded in\n`migrations.CurrentIpfsDist`, depends on the IPFS version)\n\n## `IPFS_NS_MAP`\n\nAdds static namesys records for deterministic tests and debugging.\nUseful for testing things like DNSLink without real DNS lookup.\n\nExample:\n\n```console\n$ IPFS_NS_MAP=\"dnslink-test1.example.com:/ipfs/bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am,dnslink-test2.example.com:/ipns/dnslink-test1.example.com\" ipfs daemon\n...\n$ ipfs resolve -r /ipns/dnslink-test2.example.com\n/ipfs/bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am\n```\n\n## `IPFS_HTTP_ROUTERS`\n\nOverrides AutoConf and all other HTTP routers when set.\nWhen `Routing.Type=auto`, this environment variable takes precedence over\nboth AutoConf-provided endpoints and any manually configured delegated routers.\nThe value should be a space or comma-separated list of HTTP routing endpoint URLs.\n\nThis is useful for:\n- Testing and debugging in offline contexts\n- Overriding AutoConf endpoints temporarily\n- Using custom or private HTTP routing services\n\nExample:\n\n```console\n$ ipfs config Routing.Type auto\n$ IPFS_HTTP_ROUTERS=\"http://127.0.0.1:7423\" ipfs daemon\n```\n\nThe above will replace all AutoConf endpoints with a single local one, allowing for\ninspection/debug of HTTP requests sent by Kubo via `while true ; do nc -l 7423; done`\nor more advanced tools like [mitmproxy](https://docs.mitmproxy.org/stable/#mitmproxy).\n\nWhen not set, Kubo uses endpoints from AutoConf (when enabled) or manually configured `Routing.DelegatedRouters`.\n\n## `IPFS_HTTP_ROUTERS_FILTER_PROTOCOLS`\n\nOverrides values passed with `filter-protocols` parameter defined in IPIP-484.\nValue is space-separated.\n\n```console\n$ IPFS_HTTP_ROUTERS_FILTER_PROTOCOLS=\"unknown transport-bitswap transport-foo\" ipfs daemon\n```\n\nDefault: `config.DefaultHTTPRoutersFilterProtocols`\n\n## `IPFS_CONTENT_BLOCKING_DISABLE`\n\nDisables the content-blocking subsystem. No denylists will be watched and no\ncontent will be blocked.\n\n## `IPFS_WAIT_REPO_LOCK`\n\nSpecifies the amount of time to wait for the repo lock. Set the value of this variable to a string that can be [parsed](https://pkg.go.dev/time@go1.24.3#ParseDuration) as a golang `time.Duration`. For example:\n```\nIPFS_WAIT_REPO_LOCK=\"15s\"\n```\n\nIf the lock cannot be acquired because someone else has the lock, and `IPFS_WAIT_REPO_LOCK` is set to a valid value, then acquiring the lock is retried every second until the lock is acquired or the specified wait time has elapsed.\n\n## `IPFS_TELEMETRY`\n\nControls the behavior of the [telemetry plugin](telemetry.md). Valid values are:\n\n- `on`: Enables telemetry.\n- `off`: Disables telemetry.\n- `auto`: Like `on`, but logs an informative message about telemetry and gives user 15 minutes to opt-out before first collection. Used automatically on first run and when `IPFS_TELEMETRY` is not set.\n\nThe mode can also be set in the config file under `Plugins.Plugins.telemetry.Config.Mode`.\n\nExample:\n\n```bash\nexport IPFS_TELEMETRY=\"off\"\n```\n\n## `LIBP2P_TCP_REUSEPORT`\n\nKubo tries to reuse the same source port for all connections to improve NAT\ntraversal. If this is an issue, you can disable it by setting\n`LIBP2P_TCP_REUSEPORT` to false.\n\nDefault: `true`\n\n## `LIBP2P_TCP_MUX`\n\nBy default Kubo tries to reuse the same listener port for raw TCP and WebSockets transports via experimental `libp2p.ShareTCPListener()` feature introduced in [go-libp2p#2984](https://github.com/libp2p/go-libp2p/pull/2984).\nIf this is an issue, you can disable it by setting `LIBP2P_TCP_MUX` to `false` and use separate ports for each TCP transport.\n\n> [!CAUTION]\n> This configuration option may be removed once `libp2p.ShareTCPListener()` becomes default in go-libp2p.\n\nDefault: `true`\n\n## `LIBP2P_MUX_PREFS`\n\nDeprecated: Use the `Swarm.Transports.Multiplexers` config field.\n\nTells Kubo which multiplexers to use in which order.\n\nDefault: \"/yamux/1.0.0 /mplex/6.7.0\"\n\n## `LIBP2P_RCMGR`\n\nForces [libp2p Network Resource Manager](https://github.com/libp2p/go-libp2p-resource-manager#readme)\nto be enabled (`1`) or disabled (`0`).\nWhen set, overrides [`Swarm.ResourceMgr.Enabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgrenabled) from the config.\n\nDefault: use config (not set)\n\n## `LIBP2P_DEBUG_RCMGR`\n\nEnables tracing of [libp2p Network Resource Manager](https://github.com/libp2p/go-libp2p-resource-manager#readme)\nand outputs it to `rcmgr.json.gz`\n\n\nDefault: disabled (not set)\n\n## `LIBP2P_SWARM_FD_LIMIT`\n\nThis variable controls the number of concurrent outbound dials (except dials to relay addresses which have their own limiting logic).\n\nReducing it slows down connection ballooning but might affect performance negatively.\n\nDefault: [160](https://github.com/libp2p/go-libp2p/blob/master/p2p/net/swarm/swarm_dial.go#L91) (not set)\n\n# Tracing\n\nFor tracing configuration, please check: https://github.com/ipfs/boxo/blob/main/docs/tracing.md\n" }, { "path": "docs/examples/kubo-as-a-library/README.md", "content": "# Use Kubo (go-ipfs) as a library to spawn a node and add a file\n\n> Note: if you are trying to customize or extend Kubo, you should read the [Customizing Kubo](../../customizing.md) doc\n\nBy the end of this tutorial, you will learn how to:\n\n- Spawn an IPFS node that runs in process (no separate daemon process)\n- Create an IPFS repo\n- Add files and directories to IPFS\n- Retrieve those files and directories using ``cat`` and ``get``\n- Connect to other nodes in the network\n- Retrieve a file that only exists on the network\n- The difference between a node in DHT client mode and full DHT mode\n\nAll of this using only golang!\n\nIn order to complete this tutorial, you will need:\n- golang installed on your machine. See how at https://golang.org/doc/install\n- git installed on your machine (so that go can download the repo and the necessary dependencies). See how at https://git-scm.com/downloads\n- IPFS Desktop (for convenience) installed and running on your machine. See how at https://github.com/ipfs-shipyard/ipfs-desktop#ipfs-desktop\n\n\n**Disclaimer**: The example code is quite large (more than 300 lines of code) and it has been a great way to understand the scope of the [go-ipfs Core API](https://godoc.org/github.com/ipfs/interface-go-ipfs-core), and how it can be improved to further the user experience. You can expect to be able to come back to this example in the future and see how the number of lines of code have decreased and how the example have become simpler, making other go-ipfs programs simpler as well.\n\n## Getting started\n\n**Note:** Make sure you have [![](https://img.shields.io/badge/go-%3E%3D1.13.0-blue.svg?style=flat-square)](https://golang.org/dl/) installed.\n\nDownload Kubo and jump into the example folder:\n\n```console\n$ git clone https://github.com/ipfs/kubo.git\n$ cd kubo/docs/examples/kubo-as-a-library\n```\n\n## Running the example as-is\n\nTo run the example, simply do:\n\n```console\n$ go run main.go\n```\n\nYou should see the following as output:\n\n```\n-- Getting an IPFS node running --\nSpawning Kubo node on a temporary repo\nIPFS node is running\n\n-- Adding and getting back files & directories --\nAdded file to IPFS with CID /ipfs/QmV9tSDx9UiPeWExXEeH6aoDvmihvx6jD5eLb4jbTaKGps\nAdded directory to IPFS with CID /ipfs/QmdQdu1fkaAUokmkfpWrmPHK78F9Eo9K2nnuWuizUjmhyn\nGot file back from IPFS (IPFS path: /ipfs/QmV9tSDx9UiPeWExXEeH6aoDvmihvx6jD5eLb4jbTaKGps) and wrote it to ./example-folder/QmV9tSDx9UiPeWExXEeH6aoDvmihvx6jD5eLb4jbTaKGps\nGot directory back from IPFS (IPFS path: /ipfs/QmdQdu1fkaAUokmkfpWrmPHK78F9Eo9K2nnuWuizUjmhyn) and wrote it to ./example-folder/QmdQdu1fkaAUokmkfpWrmPHK78F9Eo9K2nnuWuizUjmhyn\n\n-- Going to connect to a few nodes in the Network as bootstrappers --\nFetching a file from the network with CID QmUaoioqU7bxezBQZkUcgcSyokatMY71sxsALxQmRRrHrj\nWrote the file to ./example-folder/QmUaoioqU7bxezBQZkUcgcSyokatMY71sxsALxQmRRrHrj\n\nAll done! You just finalized your first tutorial on how to use Kubo as a library\n```\n\n## Understanding the example\n\nIn this example, we add a file and a directory with files; we get them back from IPFS; and then we use the IPFS network to fetch a file that we didn't have in our machines before.\n\nEach section below has links to lines of code in the file [main.go](./main.go). The code itself will have comments explaining what is happening for you.\n\n### The `func main() {}`\n\nThe [main function](./main.go#L202-L331) is where the magic starts, and it is the best place to follow the path of what is happening in the tutorial.\n\n### Part 1: Getting an IPFS node running\n\nTo get [get a node running](./main.go#L218-L223) as an [ephemeral node](./main.go#L114-L128) (that will cease to exist when the run ends), you will need to:\n\n- [Prepare and set up the plugins](./main.go#L30-L47)\n- [Create an IPFS repo](./main.go#L49-L68)\n- [Construct the IPFS node instance itself](./main.go#L72-L96)\n\nAs soon as you construct the IPFS node instance, the node will be running.\n\n### Part 2: Adding a file and a directory to IPFS\n\n- [Prepare the file to be added to IPFS](./main.go#L166-L184)\n- [Add the file to IPFS](./main.go#L240-L243)\n- [Prepare the directory to be added to IPFS](./main.go#L186-L198)\n- [Add the directory to IPFS](./main.go#L252-L255)\n\n### Part 3: Getting the file and directory you added back\n\n- [Get the file back](./main.go#L265-L268)\n- [Write the file to your local filesystem](./main.go#L270-L273)\n- [Get the directory back](./main.go#L277-L280)\n- [Write the directory to your local filesystem](./main.go#L282-L285)\n\n### Part 4: Getting a file from the IPFS network\n\n- [Connect to nodes in the network](./main.go#L293-L310)\n- [Get the file from the network](./main.go#L318-L321)\n- [Write the file to your local filesystem](./main.go#L323-L326)\n\n### Bonus: Spawn a daemon on your existing IPFS repo (on the default path ~/.ipfs)\n\nAs a bonus, you can also find lines that show you how to spawn a node over your default path (~/.ipfs) in case you had already started a node there before. To try it:\n\n- [Comment these lines](./main.go#L219-L223)\n- [Uncomment these lines](./main.go#L209-L216)\n\n## Voilá! You are now a Kubo hacker\n\nYou've learned how to spawn a Kubo node using the Core API. There are many more [methods to experiment next](https://godoc.org/github.com/ipfs/interface-go-ipfs-core). Happy hacking!\n" }, { "path": "docs/examples/kubo-as-a-library/go.mod", "content": "module github.com/ipfs/kubo/examples/kubo-as-a-library\n\ngo 1.25.7\n\n// Used to keep this in sync with the current version of kubo. You should remove\n// this if you copy this example.\nreplace github.com/ipfs/kubo => ./../../..\n\nrequire (\n\tgithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422\n\tgithub.com/ipfs/kubo v0.0.0-00010101000000-000000000000\n\tgithub.com/libp2p/go-libp2p v0.48.0\n\tgithub.com/multiformats/go-multiaddr v0.16.1\n)\n\nrequire (\n\tbazil.org/fuse v0.0.0-20200117225306-7b5117fecadc // indirect\n\tfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5 // indirect\n\tfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b // indirect\n\tgithub.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect\n\tgithub.com/DataDog/zstd v1.5.7 // indirect\n\tgithub.com/Jorropo/jsync v1.0.1 // indirect\n\tgithub.com/RaduBerinde/axisds v0.1.0 // indirect\n\tgithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 // indirect\n\tgithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect\n\tgithub.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 // indirect\n\tgithub.com/benbjohnson/clock v1.3.5 // indirect\n\tgithub.com/beorn7/perks v1.0.1 // indirect\n\tgithub.com/blang/semver/v4 v4.0.0 // indirect\n\tgithub.com/caddyserver/certmagic v0.23.0 // indirect\n\tgithub.com/caddyserver/zerossl v0.1.3 // indirect\n\tgithub.com/cenkalti/backoff/v4 v4.3.0 // indirect\n\tgithub.com/cenkalti/backoff/v5 v5.0.3 // indirect\n\tgithub.com/ceramicnetwork/go-dag-jose v0.1.1 // indirect\n\tgithub.com/cespare/xxhash/v2 v2.3.0 // indirect\n\tgithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b // indirect\n\tgithub.com/cockroachdb/errors v1.11.3 // indirect\n\tgithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect\n\tgithub.com/cockroachdb/pebble/v2 v2.1.4 // indirect\n\tgithub.com/cockroachdb/redact v1.1.5 // indirect\n\tgithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b // indirect\n\tgithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect\n\tgithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf // indirect\n\tgithub.com/cskr/pubsub v1.0.2 // indirect\n\tgithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect\n\tgithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect\n\tgithub.com/dgraph-io/badger v1.6.2 // indirect\n\tgithub.com/dgraph-io/ristretto v0.1.1 // indirect\n\tgithub.com/dunglas/httpsfv v1.1.0 // indirect\n\tgithub.com/dustin/go-humanize v1.0.1 // indirect\n\tgithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5 // indirect\n\tgithub.com/felixge/httpsnoop v1.0.4 // indirect\n\tgithub.com/filecoin-project/go-clock v0.1.0 // indirect\n\tgithub.com/flynn/noise v1.1.0 // indirect\n\tgithub.com/fsnotify/fsnotify v1.9.0 // indirect\n\tgithub.com/gabriel-vasile/mimetype v1.4.13 // indirect\n\tgithub.com/gammazero/chanqueue v1.1.2 // indirect\n\tgithub.com/gammazero/deque v1.2.1 // indirect\n\tgithub.com/getsentry/sentry-go v0.27.0 // indirect\n\tgithub.com/go-jose/go-jose/v4 v4.1.3 // indirect\n\tgithub.com/go-logr/logr v1.4.3 // indirect\n\tgithub.com/go-logr/stdr v1.2.2 // indirect\n\tgithub.com/gogo/protobuf v1.3.2 // indirect\n\tgithub.com/golang/glog v1.2.5 // indirect\n\tgithub.com/golang/protobuf v1.5.4 // indirect\n\tgithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e // indirect\n\tgithub.com/google/gopacket v1.1.19 // indirect\n\tgithub.com/google/uuid v1.6.0 // indirect\n\tgithub.com/gorilla/websocket v1.5.3 // indirect\n\tgithub.com/grpc-ecosystem/grpc-gateway/v2 v2.27.7 // indirect\n\tgithub.com/guillaumemichel/reservedpool v0.3.0 // indirect\n\tgithub.com/hashicorp/golang-lru v1.0.2 // indirect\n\tgithub.com/hashicorp/golang-lru/v2 v2.0.7 // indirect\n\tgithub.com/huin/goupnp v1.3.0 // indirect\n\tgithub.com/ipfs-shipyard/nopfs v0.0.14 // indirect\n\tgithub.com/ipfs-shipyard/nopfs/ipfs v0.25.0 // indirect\n\tgithub.com/ipfs/bbloom v0.0.4 // indirect\n\tgithub.com/ipfs/go-bitfield v1.1.0 // indirect\n\tgithub.com/ipfs/go-block-format v0.2.3 // indirect\n\tgithub.com/ipfs/go-cid v0.6.0 // indirect\n\tgithub.com/ipfs/go-cidutil v0.1.1 // indirect\n\tgithub.com/ipfs/go-datastore v0.9.1 // indirect\n\tgithub.com/ipfs/go-ds-badger v0.3.4 // indirect\n\tgithub.com/ipfs/go-ds-flatfs v0.6.0 // indirect\n\tgithub.com/ipfs/go-ds-leveldb v0.5.2 // indirect\n\tgithub.com/ipfs/go-ds-measure v0.2.2 // indirect\n\tgithub.com/ipfs/go-ds-pebble v0.5.9 // indirect\n\tgithub.com/ipfs/go-dsqueue v0.2.0 // indirect\n\tgithub.com/ipfs/go-fs-lock v0.1.1 // indirect\n\tgithub.com/ipfs/go-ipfs-cmds v0.16.0 // indirect\n\tgithub.com/ipfs/go-ipfs-ds-help v1.1.1 // indirect\n\tgithub.com/ipfs/go-ipfs-pq v0.0.4 // indirect\n\tgithub.com/ipfs/go-ipfs-redirects-file v0.1.2 // indirect\n\tgithub.com/ipfs/go-ipld-cbor v0.2.1 // indirect\n\tgithub.com/ipfs/go-ipld-format v0.6.3 // indirect\n\tgithub.com/ipfs/go-ipld-git v0.1.1 // indirect\n\tgithub.com/ipfs/go-ipld-legacy v0.3.0 // indirect\n\tgithub.com/ipfs/go-libdht v0.5.0 // indirect\n\tgithub.com/ipfs/go-log/v2 v2.9.1 // indirect\n\tgithub.com/ipfs/go-metrics-interface v0.3.0 // indirect\n\tgithub.com/ipfs/go-peertaskqueue v0.8.3 // indirect\n\tgithub.com/ipfs/go-test v0.2.3 // indirect\n\tgithub.com/ipfs/go-unixfsnode v1.10.3 // indirect\n\tgithub.com/ipld/go-car/v2 v2.16.0 // indirect\n\tgithub.com/ipld/go-codec-dagpb v1.7.0 // indirect\n\tgithub.com/ipld/go-ipld-prime v0.22.0 // indirect\n\tgithub.com/ipshipyard/p2p-forge v0.7.0 // indirect\n\tgithub.com/jackpal/go-nat-pmp v1.0.2 // indirect\n\tgithub.com/jbenet/go-temp-err-catcher v0.1.0 // indirect\n\tgithub.com/klauspost/compress v1.18.0 // indirect\n\tgithub.com/klauspost/cpuid/v2 v2.3.0 // indirect\n\tgithub.com/koron/go-ssdp v0.0.6 // indirect\n\tgithub.com/kr/pretty v0.3.1 // indirect\n\tgithub.com/kr/text v0.2.0 // indirect\n\tgithub.com/libdns/libdns v1.0.0-beta.1 // indirect\n\tgithub.com/libp2p/go-buffer-pool v0.1.0 // indirect\n\tgithub.com/libp2p/go-cidranger v1.1.0 // indirect\n\tgithub.com/libp2p/go-doh-resolver v0.5.0 // indirect\n\tgithub.com/libp2p/go-flow-metrics v0.3.0 // indirect\n\tgithub.com/libp2p/go-libp2p-asn-util v0.4.1 // indirect\n\tgithub.com/libp2p/go-libp2p-kad-dht v0.39.0 // indirect\n\tgithub.com/libp2p/go-libp2p-kbucket v0.8.0 // indirect\n\tgithub.com/libp2p/go-libp2p-pubsub v0.15.0 // indirect\n\tgithub.com/libp2p/go-libp2p-pubsub-router v0.6.0 // indirect\n\tgithub.com/libp2p/go-libp2p-record v0.3.1 // indirect\n\tgithub.com/libp2p/go-libp2p-routing-helpers v0.7.5 // indirect\n\tgithub.com/libp2p/go-libp2p-xor v0.1.0 // indirect\n\tgithub.com/libp2p/go-msgio v0.3.0 // indirect\n\tgithub.com/libp2p/go-netroute v0.4.0 // indirect\n\tgithub.com/libp2p/go-reuseport v0.4.0 // indirect\n\tgithub.com/libp2p/go-yamux/v5 v5.0.1 // indirect\n\tgithub.com/libp2p/zeroconf/v2 v2.2.0 // indirect\n\tgithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd // indirect\n\tgithub.com/mattn/go-isatty v0.0.20 // indirect\n\tgithub.com/mholt/acmez/v3 v3.1.2 // indirect\n\tgithub.com/miekg/dns v1.1.72 // indirect\n\tgithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b // indirect\n\tgithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect\n\tgithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 // indirect\n\tgithub.com/minio/sha256-simd v1.0.1 // indirect\n\tgithub.com/mr-tron/base58 v1.2.0 // indirect\n\tgithub.com/multiformats/go-base32 v0.1.0 // indirect\n\tgithub.com/multiformats/go-base36 v0.2.0 // indirect\n\tgithub.com/multiformats/go-multiaddr-dns v0.5.0 // indirect\n\tgithub.com/multiformats/go-multiaddr-fmt v0.1.0 // indirect\n\tgithub.com/multiformats/go-multibase v0.2.0 // indirect\n\tgithub.com/multiformats/go-multicodec v0.10.0 // indirect\n\tgithub.com/multiformats/go-multihash v0.2.3 // indirect\n\tgithub.com/multiformats/go-multistream v0.6.1 // indirect\n\tgithub.com/multiformats/go-varint v0.1.0 // indirect\n\tgithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect\n\tgithub.com/opentracing/opentracing-go v1.2.0 // indirect\n\tgithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect\n\tgithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 // indirect\n\tgithub.com/pion/datachannel v1.5.10 // indirect\n\tgithub.com/pion/dtls/v3 v3.1.2 // indirect\n\tgithub.com/pion/ice/v4 v4.0.10 // indirect\n\tgithub.com/pion/interceptor v0.1.40 // indirect\n\tgithub.com/pion/logging v0.2.4 // indirect\n\tgithub.com/pion/mdns/v2 v2.0.7 // indirect\n\tgithub.com/pion/randutil v0.1.0 // indirect\n\tgithub.com/pion/rtcp v1.2.16 // indirect\n\tgithub.com/pion/rtp v1.8.19 // indirect\n\tgithub.com/pion/sctp v1.8.39 // indirect\n\tgithub.com/pion/sdp/v3 v3.0.18 // indirect\n\tgithub.com/pion/srtp/v3 v3.0.6 // indirect\n\tgithub.com/pion/stun/v3 v3.1.1 // indirect\n\tgithub.com/pion/transport/v3 v3.0.7 // indirect\n\tgithub.com/pion/transport/v4 v4.0.1 // indirect\n\tgithub.com/pion/turn/v4 v4.0.2 // indirect\n\tgithub.com/pion/webrtc/v4 v4.1.2 // indirect\n\tgithub.com/pkg/errors v0.9.1 // indirect\n\tgithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a // indirect\n\tgithub.com/prometheus/client_golang v1.23.2 // indirect\n\tgithub.com/prometheus/client_model v0.6.2 // indirect\n\tgithub.com/prometheus/common v0.67.5 // indirect\n\tgithub.com/prometheus/procfs v0.20.1 // indirect\n\tgithub.com/quic-go/qpack v0.6.0 // indirect\n\tgithub.com/quic-go/quic-go v0.59.0 // indirect\n\tgithub.com/quic-go/webtransport-go v0.10.0 // indirect\n\tgithub.com/rogpeppe/go-internal v1.14.1 // indirect\n\tgithub.com/spaolacci/murmur3 v1.1.0 // indirect\n\tgithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d // indirect\n\tgithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb // indirect\n\tgithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc // indirect\n\tgithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11 // indirect\n\tgithub.com/whyrusleeping/cbor-gen v0.3.1 // indirect\n\tgithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f // indirect\n\tgithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 // indirect\n\tgithub.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7 // indirect\n\tgithub.com/wlynxg/anet v0.0.5 // indirect\n\tgithub.com/zeebo/blake3 v0.2.4 // indirect\n\tgo.opencensus.io v0.24.0 // indirect\n\tgo.opentelemetry.io/auto/sdk v1.2.1 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 // indirect\n\tgo.opentelemetry.io/otel v1.42.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/otlp/otlptrace v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/metric v1.42.0 // indirect\n\tgo.opentelemetry.io/otel/sdk v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/trace v1.42.0 // indirect\n\tgo.opentelemetry.io/proto/otlp v1.9.0 // indirect\n\tgo.uber.org/dig v1.19.0 // indirect\n\tgo.uber.org/fx v1.24.0 // indirect\n\tgo.uber.org/mock v0.5.2 // indirect\n\tgo.uber.org/multierr v1.11.0 // indirect\n\tgo.uber.org/zap v1.27.1 // indirect\n\tgo.uber.org/zap/exp v0.3.0 // indirect\n\tgo.yaml.in/yaml/v2 v2.4.4 // indirect\n\tgo4.org v0.0.0-20230225012048-214862532bf5 // indirect\n\tgolang.org/x/crypto v0.49.0 // indirect\n\tgolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90 // indirect\n\tgolang.org/x/mod v0.34.0 // indirect\n\tgolang.org/x/net v0.52.0 // indirect\n\tgolang.org/x/sync v0.20.0 // indirect\n\tgolang.org/x/sys v0.42.0 // indirect\n\tgolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c // indirect\n\tgolang.org/x/text v0.35.0 // indirect\n\tgolang.org/x/time v0.12.0 // indirect\n\tgolang.org/x/tools v0.43.0 // indirect\n\tgolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect\n\tgonum.org/v1/gonum v0.17.0 // indirect\n\tgoogle.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 // indirect\n\tgoogle.golang.org/genproto/googleapis/rpc v0.0.0-20260128011058-8636f8732409 // indirect\n\tgoogle.golang.org/grpc v1.78.0 // indirect\n\tgoogle.golang.org/protobuf v1.36.11 // indirect\n\tgopkg.in/yaml.v3 v3.0.1 // indirect\n\tlukechampine.com/blake3 v1.4.1 // indirect\n)\n" }, { "path": "docs/examples/kubo-as-a-library/go.sum", "content": "bazil.org/fuse v0.0.0-20200117225306-7b5117fecadc h1:utDghgcjE8u+EBjHOgYT+dJPcnDF05KqWMBcjuJy510=\nbazil.org/fuse v0.0.0-20200117225306-7b5117fecadc/go.mod h1:FbcW6z/2VytnFDhZfumh8Ss8zxHE6qpMP5sHTRe0EaM=\ncloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=\ncloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=\ncloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=\ncloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=\ncloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=\ncloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=\ncloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=\ncloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=\ncloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=\ncloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=\ncloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=\ncloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=\ncloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=\ncloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=\ncloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=\ncloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=\ncloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=\ncloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=\ncloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=\ncloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=\ncloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=\ncloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=\ncloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=\ncloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=\ncloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=\ncloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=\ncloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk=\ncloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=\ncloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=\ncloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=\ncloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=\ncloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=\ncloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=\ncloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=\ncloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=\ncloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=\ndmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=\nfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5 h1:JA0fFr+kxpqTdxR9LOBiTWpGNchqmkcsgmdeJZRclZ0=\nfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5/go.mod h1:OjOXDNlClLblvXdwgFFOQFJEocLhhtai8vGLy0JCZlI=\nfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b h1:REI1FbdW71yO56Are4XAxD+OS/e+BQsB3gE4mZRQEXY=\nfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b/go.mod h1:9nnw1SlYHYuPSo/3wjQzNjSbeHlq2NsKo5iEtfJPWP0=\ngithub.com/AndreasBriese/bbloom v0.0.0-20190306092124-e2d15f34fcf9/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=\ngithub.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 h1:cTp8I5+VIoKjsnZuH8vjyaysT/ses3EvZeaV/1UkF2M=\ngithub.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=\ngithub.com/DataDog/zstd v1.5.7 h1:ybO8RBeh29qrxIhCA9E8gKY6xfONU9T6G6aP9DTKfLE=\ngithub.com/DataDog/zstd v1.5.7/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=\ngithub.com/Jorropo/jsync v1.0.1 h1:6HgRolFZnsdfzRUj+ImB9og1JYOxQoReSywkHOGSaUU=\ngithub.com/Jorropo/jsync v1.0.1/go.mod h1:jCOZj3vrBCri3bSU3ErUYvevKlnbssrXeCivybS5ABQ=\ngithub.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=\ngithub.com/RaduBerinde/axisds v0.1.0 h1:YItk/RmU5nvlsv/awo2Fjx97Mfpt4JfgtEVAGPrLdz8=\ngithub.com/RaduBerinde/axisds v0.1.0/go.mod h1:UHGJonU9z4YYGKJxSaC6/TNcLOBptpmM5m2Cksbnw0Y=\ngithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 h1:bsU8Tzxr/PNz75ayvCnxKZWEYdLMPDkUgticP4a4Bvk=\ngithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54/go.mod h1:0tr7FllbE9gJkHq7CVeeDDFAFKQVy5RnCSSNBOvdqbc=\ngithub.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f h1:JjxwchlOepwsUWcQwD2mLUAGE9aCp0/ehy6yCHFBOvo=\ngithub.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f/go.mod h1:tMDTce/yLLN/SK8gMOxQfnyeMeCg8KGzp0D1cbECEeo=\ngithub.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII=\ngithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0=\ngithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs=\ngithub.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 h1:iW0a5ljuFxkLGPNem5Ui+KBjFJzKg4Fv2fnxe4dvzpM=\ngithub.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5/go.mod h1:Y2QMoi1vgtOIfc+6DhrMOGkLoGzqSV2rKp4Sm+opsyA=\ngithub.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=\ngithub.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=\ngithub.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=\ngithub.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=\ngithub.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=\ngithub.com/benbjohnson/clock v1.3.5 h1:VvXlSJBzZpA/zum6Sj74hxwYI2DIxRWuNIoXAzHZz5o=\ngithub.com/benbjohnson/clock v1.3.5/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=\ngithub.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM=\ngithub.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM=\ngithub.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ=\ngithub.com/btcsuite/btcd v0.0.0-20190824003749-130ea5bddde3/go.mod h1:3J08xEfcugPacsc34/LKRU2yO7YmuT8yt28J8k2+rrI=\ngithub.com/btcsuite/btcd v0.20.1-beta/go.mod h1:wVuoA8VJLEcwgqHBwHmzLRazpKxTv13Px/pDuV7OomQ=\ngithub.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA=\ngithub.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg=\ngithub.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg=\ngithub.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVaaLLH7j4eDXPRvw78tMflu7Ie2bzYOH4Y8rRKBY=\ngithub.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc=\ngithub.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY=\ngithub.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs=\ngithub.com/caddyserver/certmagic v0.23.0 h1:CfpZ/50jMfG4+1J/u2LV6piJq4HOfO6ppOnOf7DkFEU=\ngithub.com/caddyserver/certmagic v0.23.0/go.mod h1:9mEZIWqqWoI+Gf+4Trh04MOVPD0tGSxtqsxg87hAIH4=\ngithub.com/caddyserver/zerossl v0.1.3 h1:onS+pxp3M8HnHpN5MMbOMyNjmTheJyWRaZYwn+YTAyA=\ngithub.com/caddyserver/zerossl v0.1.3/go.mod h1:CxA0acn7oEGO6//4rtrRjYgEoa4MFw/XofZnrYwGqG4=\ngithub.com/canonical/go-sp800.90a-drbg v0.0.0-20210314144037-6eeb1040d6c3 h1:oe6fCvaEpkhyW3qAicT0TnGtyht/UrgvOwMcEgLb7Aw=\ngithub.com/canonical/go-sp800.90a-drbg v0.0.0-20210314144037-6eeb1040d6c3/go.mod h1:qdP0gaj0QtgX2RUZhnlVrceJ+Qln8aSlDyJwelLLFeM=\ngithub.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=\ngithub.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=\ngithub.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM=\ngithub.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/ceramicnetwork/go-dag-jose v0.1.1 h1:7pObs22egc14vSS3AfCFfS1VmaL4lQUsAK7OGC3PlKk=\ngithub.com/ceramicnetwork/go-dag-jose v0.1.1/go.mod h1:8ptnYwY2Z2y/s5oJnNBn/UCxLg6CpramNJ2ZXF/5aNY=\ngithub.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=\ngithub.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=\ngithub.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=\ngithub.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=\ngithub.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b h1:SHlYZ/bMx7frnmeqCu+xm0TCxXLzX3jQIVuFbnFGtFU=\ngithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b/go.mod h1:Gq51ZeKaFCXk6QwuGM0w1dnaOqc/F5zKT2zA9D6Xeac=\ngithub.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5 h1:UycK/E0TkisVrQbSoxvU827FwgBBcZ95nRRmpj/12QI=\ngithub.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5/go.mod h1:jsaKMvD3RBCATk1/jbUZM8C9idWBJME9+VRZ5+Liq1g=\ngithub.com/cockroachdb/errors v1.11.3 h1:5bA+k2Y6r+oz/6Z/RFlNeVCesGARKuC6YymtcDrbC/I=\ngithub.com/cockroachdb/errors v1.11.3/go.mod h1:m4UIW4CDjx+R5cybPsNrRbreomiFqt8o1h1wUVazSd8=\ngithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZeQy818SGhaone5OnYfxFR/+AzdY3sf5aE=\ngithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=\ngithub.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895 h1:XANOgPYtvELQ/h4IrmPAohXqe2pWA8Bwhejr3VQoZsA=\ngithub.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895/go.mod h1:aPd7gM9ov9M8v32Yy5NJrDyOcD8z642dqs+F0CeNXfA=\ngithub.com/cockroachdb/pebble/v2 v2.1.4 h1:j9wPgMDbkErFdAKYFGhsoCcvzcjR+6zrJ4jhKtJ6bOk=\ngithub.com/cockroachdb/pebble/v2 v2.1.4/go.mod h1:Reo1RTniv1UjVTAu/Fv74y5i3kJ5gmVrPhO9UtFiKn8=\ngithub.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30=\ngithub.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=\ngithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b h1:VXvSNzmr8hMj8XTuY0PT9Ane9qZGul/p67vGYwl9BFI=\ngithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b/go.mod h1:yBRu/cnL4ks9bgy4vAASdjIW+/xMlFwuHKqtmh3GZQg=\ngithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo=\ngithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06/go.mod h1:7nc4anLGjupUW/PeY5qiNYsdNXj7zopG+eqsS7To5IQ=\ngithub.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=\ngithub.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=\ngithub.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=\ngithub.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf h1:dwGgBWn84wUS1pVikGiruW+x5XM4amhjaZO20vCjay4=\ngithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf/go.mod h1:p1d6YEZWvFzEh4KLyvBcVSnrfNDDvK2zfK/4x2v/4pE=\ngithub.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=\ngithub.com/cskr/pubsub v1.0.2 h1:vlOzMhl6PFn60gRlTQQsIfVwaPB/B/8MziK8FhEPt/0=\ngithub.com/cskr/pubsub v1.0.2/go.mod h1:/8MzYXk/NJAz782G8RPkFzXTZVu63VotefPnR9TIRis=\ngithub.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c h1:pFUpOrbxDR6AkioZ1ySsx5yxlDQZ8stG2b88gTPxgJU=\ngithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c/go.mod h1:6UhI8N9EjYm1c2odKpFpAYeR8dsBeM7PtzQhRgxRr9U=\ngithub.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U0x++OzVrdms8=\ngithub.com/decred/dcrd/crypto/blake256 v1.1.0/go.mod h1:2OfgNZ5wDpcsFmHmCK5gZTPcCXqlm2ArzUIkw9czNJo=\ngithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 h1:5RVFMOWjMyRy8cARdy79nAmgYw3hK/4HUq48LQ6Wwqo=\ngithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=\ngithub.com/dgraph-io/badger v1.6.0/go.mod h1:zwt7syl517jmP8s94KqSxTlM6IMsdhYy6psNgSztDR4=\ngithub.com/dgraph-io/badger v1.6.2 h1:mNw0qs90GVgGGWylh0umH5iag1j6n/PeJtNvL6KY/x8=\ngithub.com/dgraph-io/badger v1.6.2/go.mod h1:JW2yswe3V058sS0kZ2h/AXeDSqFjxnZcRrVH//y2UQE=\ngithub.com/dgraph-io/ristretto v0.0.2/go.mod h1:KPxhHT9ZxKefz+PCeOGsrHpl1qZ7i70dGTu2u+Ahh6E=\ngithub.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWajOK8=\ngithub.com/dgraph-io/ristretto v0.1.1/go.mod h1:S1GPSBCYCIhmVNfcth17y2zZtQT6wzkzgwUve0VDWWA=\ngithub.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=\ngithub.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 h1:fAjc9m62+UWV/WAFKLNi6ZS0675eEUC9y3AlwSbQu1Y=\ngithub.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=\ngithub.com/dunglas/httpsfv v1.1.0 h1:Jw76nAyKWKZKFrpMMcL76y35tOpYHqQPzHQiwDvpe54=\ngithub.com/dunglas/httpsfv v1.1.0/go.mod h1:zID2mqw9mFsnt7YC3vYQ9/cjq30q41W+1AnDwH8TiMg=\ngithub.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=\ngithub.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=\ngithub.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5 h1:BBso6MBKW8ncyZLv37o+KNyy0HrrHgfnOaGQC2qvN+A=\ngithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5/go.mod h1:JpoxHjuQauoxiFMl1ie8Xc/7TfLuMZ5eOCONd1sUBHg=\ngithub.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=\ngithub.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=\ngithub.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/filecoin-project/go-clock v0.1.0 h1:SFbYIM75M8NnFm1yMHhN9Ahy3W5bEZV9gd6MPfXbKVU=\ngithub.com/filecoin-project/go-clock v0.1.0/go.mod h1:4uB/O4PvOjlx1VCMdZ9MyDZXRm//gkj1ELEbxfI1AZs=\ngithub.com/flynn/noise v1.1.0 h1:KjPQoQCEFdZDiP03phOvGi11+SVVhBG2wOWAorLsstg=\ngithub.com/flynn/noise v1.1.0/go.mod h1:xbMo+0i6+IGbYdJhF31t2eR1BIU0CYc12+BNAKwUTag=\ngithub.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k=\ngithub.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=\ngithub.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=\ngithub.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=\ngithub.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=\ngithub.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU=\ngithub.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=\ngithub.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=\ngithub.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=\ngithub.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=\ngithub.com/gammazero/chanqueue v1.1.2 h1:dZEsxlyANZMyeTRemABqZF8QM9BnE4NBI43Oh3y5fIU=\ngithub.com/gammazero/chanqueue v1.1.2/go.mod h1:XDN1X/jjAbmSceNFOQbtKToeSkxtdVdpKu90LiEdBEE=\ngithub.com/gammazero/deque v1.2.1 h1:9fnQVFCCZ9/NOc7ccTNqzoKd1tCWOqeI05/lPqFPMGQ=\ngithub.com/gammazero/deque v1.2.1/go.mod h1:5nSFkzVm+afG9+gy0VIowlqVAW4N8zNcMne+CMQVD2g=\ngithub.com/getsentry/sentry-go v0.27.0 h1:Pv98CIbtB3LkMWmXi4Joa5OOcwbmnX88sF5qbK3r3Ps=\ngithub.com/getsentry/sentry-go v0.27.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY=\ngithub.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9 h1:r5GgOLGbza2wVHRzK7aAj6lWZjfbAwiu/RDCVOKjRyM=\ngithub.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9/go.mod h1:106OIgooyS7OzLDOpUGgm9fA3bQENb/cFSyyBmMoJDs=\ngithub.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=\ngithub.com/go-check/check v0.0.0-20180628173108-788fd7840127/go.mod h1:9ES+weclKsC9YodN5RgxqK/VD9HM9JsCSh7rNhMZE98=\ngithub.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=\ngithub.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=\ngithub.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-jose/go-jose/v4 v4.1.3 h1:CVLmWDhDVRa6Mi/IgCgaopNosCaHz7zrMeF9MlZRkrs=\ngithub.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=\ngithub.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=\ngithub.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=\ngithub.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=\ngithub.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=\ngithub.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=\ngithub.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=\ngithub.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0=\ngithub.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=\ngithub.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=\ngithub.com/gogo/protobuf v1.3.0/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=\ngithub.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=\ngithub.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=\ngithub.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/glog v1.2.5 h1:DrW6hGnjIhtvhOIiAKT6Psh/Kd/ldepEa81DKeiRJ5I=\ngithub.com/golang/glog v1.2.5/go.mod h1:6AhwSGph0fcJtXVM/PEHPqZlFeoLxhs7/t5UDAwmO+w=\ngithub.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=\ngithub.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=\ngithub.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=\ngithub.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=\ngithub.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\ngithub.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=\ngithub.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=\ngithub.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e h1:4bw4WeyTYPp0smaXiJZCNnLrvVBqirQVreixayXezGc=\ngithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=\ngithub.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=\ngithub.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=\ngithub.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo=\ngithub.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=\ngithub.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=\ngithub.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=\ngithub.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=\ngithub.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=\ngithub.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=\ngithub.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g=\ngithub.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k=\ngithub.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=\ngithub.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=\ngithub.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=\ngithub.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=\ngithub.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=\ngithub.com/grpc-ecosystem/grpc-gateway/v2 v2.27.7 h1:X+2YciYSxvMQK0UZ7sg45ZVabVZBeBuvMkmuI2V3Fak=\ngithub.com/grpc-ecosystem/grpc-gateway/v2 v2.27.7/go.mod h1:lW34nIZuQ8UDPdkon5fmfp2l3+ZkQ2me/+oecHYLOII=\ngithub.com/guillaumemichel/reservedpool v0.3.0 h1:eqqO/QvTllLBrit7LVtVJBqw4cD0WdV9ajUe7WNTajw=\ngithub.com/guillaumemichel/reservedpool v0.3.0/go.mod h1:sXSDIaef81TFdAJglsCFCMfgF5E5Z5xK1tFhjDhvbUc=\ngithub.com/gxed/hashland/keccakpg v0.0.1/go.mod h1:kRzw3HkwxFU1mpmPP8v1WyQzwdGfmKFJ6tItnhQ67kU=\ngithub.com/gxed/hashland/murmur3 v0.0.1/go.mod h1:KjXop02n4/ckmZSnY2+HKcLud/tcmvhST0bie/0lS48=\ngithub.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=\ngithub.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=\ngithub.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=\ngithub.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=\ngithub.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=\ngithub.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=\ngithub.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=\ngithub.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=\ngithub.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=\ngithub.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=\ngithub.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=\ngithub.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=\ngithub.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=\ngithub.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=\ngithub.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=\ngithub.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=\ngithub.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=\ngithub.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=\ngithub.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=\ngithub.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc=\ngithub.com/huin/goupnp v1.3.0/go.mod h1:gnGPsThkYa7bFi/KWmEysQRf48l2dvR5bxr2OFckNX8=\ngithub.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=\ngithub.com/ipfs-shipyard/nopfs v0.0.14 h1:HFepJt/MxhZ3/GsLZkkAPzIPdNYKaLO1Qb7YmPbWIKk=\ngithub.com/ipfs-shipyard/nopfs v0.0.14/go.mod h1:mQyd0BElYI2gB/kq/Oue97obP4B3os4eBmgfPZ+hnrE=\ngithub.com/ipfs-shipyard/nopfs/ipfs v0.25.0 h1:OqNqsGZPX8zh3eFMO8Lf8EHRRnSGBMqcdHUd7SDsUOY=\ngithub.com/ipfs-shipyard/nopfs/ipfs v0.25.0/go.mod h1:BxhUdtBgOXg1B+gAPEplkg/GpyTZY+kCMSfsJvvydqU=\ngithub.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=\ngithub.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=\ngithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422 h1:yY3ot/DU1bqTzHDBARACM76Tbx9s4xzcRbzifG1e/es=\ngithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422/go.mod h1:8yyiRn54F2CsW13n0zwXEPrVsZix/gFj9SYIRYMZ6KE=\ngithub.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=\ngithub.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=\ngithub.com/ipfs/go-block-format v0.0.3/go.mod h1:4LmD4ZUw0mhO+JSKdpWwrzATiEfM7WWgQ8H5l6P8MVk=\ngithub.com/ipfs/go-block-format v0.2.3 h1:mpCuDaNXJ4wrBJLrtEaGFGXkferrw5eqVvzaHhtFKQk=\ngithub.com/ipfs/go-block-format v0.2.3/go.mod h1:WJaQmPAKhD3LspLixqlqNFxiZ3BZ3xgqxxoSR/76pnA=\ngithub.com/ipfs/go-cid v0.0.3/go.mod h1:GHWU/WuQdMPmIosc4Yn1bcCT7dSeX4lBafM7iqUPQvM=\ngithub.com/ipfs/go-cid v0.0.4/go.mod h1:4LLaPOQwmk5z9LBgQnpkivrx8BJjUyGwTXCd5Xfj6+M=\ngithub.com/ipfs/go-cid v0.0.7/go.mod h1:6Ux9z5e+HpkQdckYoX1PG/6xqKspzlEIR5SDmgqgC/I=\ngithub.com/ipfs/go-cid v0.6.0 h1:DlOReBV1xhHBhhfy/gBNNTSyfOM6rLiIx9J7A4DGf30=\ngithub.com/ipfs/go-cid v0.6.0/go.mod h1:NC4kS1LZjzfhK40UGmpXv5/qD2kcMzACYJNntCUiDhQ=\ngithub.com/ipfs/go-cidutil v0.1.1 h1:COuby6H8C2ml0alvHYX3WdbFM4F07YtbY0UlT5j+sgI=\ngithub.com/ipfs/go-cidutil v0.1.1/go.mod h1:SCoUftGEUgoXe5Hjeyw5CiLZF8cwYn/TbtpFQXJCP6k=\ngithub.com/ipfs/go-datastore v0.1.0/go.mod h1:d4KVXhMt913cLBEI/PXAy6ko+W7e9AhyAKBGh803qeE=\ngithub.com/ipfs/go-datastore v0.1.1/go.mod h1:w38XXW9kVFNp57Zj5knbKWM2T+KOZCGDRVNdgPHtbHw=\ngithub.com/ipfs/go-datastore v0.9.1 h1:67Po2epre/o0UxrmkzdS9ZTe2GFGODgTd2odx8Wh6Yo=\ngithub.com/ipfs/go-datastore v0.9.1/go.mod h1:zi07Nvrpq1bQwSkEnx3bfjz+SQZbdbWyCNvyxMh9pN0=\ngithub.com/ipfs/go-detect-race v0.0.1 h1:qX/xay2W3E4Q1U7d9lNs1sU9nvguX0a7319XbyQ6cOk=\ngithub.com/ipfs/go-detect-race v0.0.1/go.mod h1:8BNT7shDZPo99Q74BpGMK+4D8Mn4j46UU0LZ723meps=\ngithub.com/ipfs/go-ds-badger v0.0.7/go.mod h1:qt0/fWzZDoPW6jpQeqUjR5kBfhDNB65jd9YlmAvpQBk=\ngithub.com/ipfs/go-ds-badger v0.3.4 h1:MmqFicftE0KrwMC77WjXTrPuoUxhwyFsjKONSeWrlOo=\ngithub.com/ipfs/go-ds-badger v0.3.4/go.mod h1:HfqsKJcNnIr9ZhZ+rkwS1J5PpaWjJjg6Ipmxd7KPfZ8=\ngithub.com/ipfs/go-ds-flatfs v0.6.0 h1:olAEnDNBK1VMoTRZvfzgo90H5kBP4qIZPpYMtNlBBws=\ngithub.com/ipfs/go-ds-flatfs v0.6.0/go.mod h1:p8a/YhmAFYyuonxDbvuIANlDCgS69uqVv+iH5f8fAxY=\ngithub.com/ipfs/go-ds-leveldb v0.1.0/go.mod h1:hqAW8y4bwX5LWcCtku2rFNX3vjDZCy5LZCg+cSZvYb8=\ngithub.com/ipfs/go-ds-leveldb v0.5.2 h1:6nmxlQ2zbp4LCNdJVsmHfs9GP0eylfBNxpmY1csp0x0=\ngithub.com/ipfs/go-ds-leveldb v0.5.2/go.mod h1:2fAwmcvD3WoRT72PzEekHBkQmBDhc39DJGoREiuGmYo=\ngithub.com/ipfs/go-ds-measure v0.2.2 h1:4kwvBGbbSXNYe4ANlg7qTIYoZU6mNlqzQHdVqICkqGI=\ngithub.com/ipfs/go-ds-measure v0.2.2/go.mod h1:b/87ak0jMgH9Ylt7oH0+XGy4P8jHx9KG09Qz+pOeTIs=\ngithub.com/ipfs/go-ds-pebble v0.5.9 h1:D1FEuMxjbEmDADNqsyT74n9QHVAn12nv9i9Qa15AFYc=\ngithub.com/ipfs/go-ds-pebble v0.5.9/go.mod h1:XmUBN05l6B+tMg7mpMS75ZcKW/CX01uZMhhWw85imQA=\ngithub.com/ipfs/go-dsqueue v0.2.0 h1:MBi9w3oSiX98Xc+Y7NuJ9G8MI6mAT4IGdO9dHEMCZzU=\ngithub.com/ipfs/go-dsqueue v0.2.0/go.mod h1:8FfNQC4DMF/KkzBXRNB9Rb3MKDW0Sh98HMtXYl1mLQE=\ngithub.com/ipfs/go-fs-lock v0.1.1 h1:TecsP/Uc7WqYYatasreZQiP9EGRy4ZnKoG4yXxR33nw=\ngithub.com/ipfs/go-fs-lock v0.1.1/go.mod h1:2goSXMCw7QfscHmSe09oXiR34DQeUdm+ei+dhonqly0=\ngithub.com/ipfs/go-ipfs-cmds v0.16.0 h1:Oq39Gzz3pWrPwP25SjbfBQugFjKyjFdsHlAMIXdzYzM=\ngithub.com/ipfs/go-ipfs-cmds v0.16.0/go.mod h1:iRNtY9ipM/vH0Yr+/0FY+JfT+trZDQIztDoesmmoTo4=\ngithub.com/ipfs/go-ipfs-delay v0.0.0-20181109222059-70721b86a9a8/go.mod h1:8SP1YXK1M1kXuc4KJZINY3TQQ03J2rwBG9QfXmbRPrw=\ngithub.com/ipfs/go-ipfs-delay v0.0.1 h1:r/UXYyRcddO6thwOnhiznIAiSvxMECGgtv35Xs1IeRQ=\ngithub.com/ipfs/go-ipfs-delay v0.0.1/go.mod h1:8SP1YXK1M1kXuc4KJZINY3TQQ03J2rwBG9QfXmbRPrw=\ngithub.com/ipfs/go-ipfs-ds-help v1.1.1 h1:B5UJOH52IbcfS56+Ul+sv8jnIV10lbjLF5eOO0C66Nw=\ngithub.com/ipfs/go-ipfs-ds-help v1.1.1/go.mod h1:75vrVCkSdSFidJscs8n4W+77AtTpCIAdDGAwjitJMIo=\ngithub.com/ipfs/go-ipfs-pq v0.0.4 h1:U7jjENWJd1jhcrR8X/xHTaph14PTAK9O+yaLJbjqgOw=\ngithub.com/ipfs/go-ipfs-pq v0.0.4/go.mod h1:9UdLOIIb99IFrgT0Fc53pvbvlJBhpUb4GJuAQf3+O2A=\ngithub.com/ipfs/go-ipfs-redirects-file v0.1.2 h1:QCK7VtL91FH17KROVVy5KrzDx2hu68QvB2FTWk08ZQk=\ngithub.com/ipfs/go-ipfs-redirects-file v0.1.2/go.mod h1:yIiTlLcDEM/8lS6T3FlCEXZktPPqSOyuY6dEzVqw7Fw=\ngithub.com/ipfs/go-ipfs-util v0.0.1/go.mod h1:spsl5z8KUnrve+73pOhSVZND1SIxPW5RyBCNzQxlJBc=\ngithub.com/ipfs/go-ipfs-util v0.0.2/go.mod h1:CbPtkWJzjLdEcezDns2XYaehFVNXG9zrdrtMecczcsQ=\ngithub.com/ipfs/go-ipld-cbor v0.2.1 h1:H05yEJbK/hxg0uf2AJhyerBDbjOuHX4yi+1U/ogRa7E=\ngithub.com/ipfs/go-ipld-cbor v0.2.1/go.mod h1:x9Zbeq8CoE5R2WicYgBMcr/9mnkQ0lHddYWJP2sMV3A=\ngithub.com/ipfs/go-ipld-format v0.6.3 h1:9/lurLDTotJpZSuL++gh3sTdmcFhVkCwsgx2+rAh4j8=\ngithub.com/ipfs/go-ipld-format v0.6.3/go.mod h1:74ilVN12NXVMIV+SrBAyC05UJRk0jVvGqdmrcYZvCBk=\ngithub.com/ipfs/go-ipld-git v0.1.1 h1:TWGnZjS0htmEmlMFEkA3ogrNCqWjIxwr16x1OsdhG+Y=\ngithub.com/ipfs/go-ipld-git v0.1.1/go.mod h1:+VyMqF5lMcJh4rwEppV0e6g4nCCHXThLYYDpKUkJubI=\ngithub.com/ipfs/go-ipld-legacy v0.3.0 h1:7XhFKkRyCvP5upOlQfKUFIqL3S5DEZnbUE4bQmQ/tNE=\ngithub.com/ipfs/go-ipld-legacy v0.3.0/go.mod h1:Ukef9ARQiX+RVetwH2XiReLgJvQDEXcUPszrZ1KRjKI=\ngithub.com/ipfs/go-libdht v0.5.0 h1:ZN+eCqwahZvUeT0e4DsIxRtm78Mc9UR5tmZUiMsrGjQ=\ngithub.com/ipfs/go-libdht v0.5.0/go.mod h1:L3YiuFXecLeZZFuuVRM0hjg1GgVhARzUdahFsuqSa7w=\ngithub.com/ipfs/go-log v0.0.1/go.mod h1:kL1d2/hzSpI0thNYjiKfjanbVNU+IIGA/WnNESY9leM=\ngithub.com/ipfs/go-log/v2 v2.9.1 h1:3JXwHWU31dsCpvQ+7asz6/QsFJHqFr4gLgQ0FWteujk=\ngithub.com/ipfs/go-log/v2 v2.9.1/go.mod h1:evFx7sBiohUN3AG12mXlZBw5hacBQld3ZPHrowlJYoo=\ngithub.com/ipfs/go-metrics-interface v0.3.0 h1:YwG7/Cy4R94mYDUuwsBfeziJCVm9pBMJ6q/JR9V40TU=\ngithub.com/ipfs/go-metrics-interface v0.3.0/go.mod h1:OxxQjZDGocXVdyTPocns6cOLwHieqej/jos7H4POwoY=\ngithub.com/ipfs/go-peertaskqueue v0.8.3 h1:tBPpGJy+A92RqtRFq5amJn0Uuj8Pw8tXi0X3eHfHM8w=\ngithub.com/ipfs/go-peertaskqueue v0.8.3/go.mod h1:OqVync4kPOcXEGdj/LKvox9DCB5mkSBeXsPczCxLtYA=\ngithub.com/ipfs/go-test v0.2.3 h1:Z/jXNAReQFtCYyn7bsv/ZqUwS6E7iIcSpJ2CuzCvnrc=\ngithub.com/ipfs/go-test v0.2.3/go.mod h1:QW8vSKkwYvWFwIZQLGQXdkt9Ud76eQXRQ9Ao2H+cA1o=\ngithub.com/ipfs/go-unixfsnode v1.10.3 h1:c8sJjuGNkxXAQH75P+f5ngPda/9T+DrboVA0TcDGvGI=\ngithub.com/ipfs/go-unixfsnode v1.10.3/go.mod h1:2Jlc7DoEwr12W+7l8Hr6C7XF4NHST3gIkqSArLhGSxU=\ngithub.com/ipld/go-car/v2 v2.16.0 h1:LWe0vmN/QcQmUU4tr34W5Nv5mNraW+G6jfN2s+ndBco=\ngithub.com/ipld/go-car/v2 v2.16.0/go.mod h1:RqFGWN9ifcXVmCrTAVnfnxiWZk1+jIx67SYhenlmL34=\ngithub.com/ipld/go-codec-dagpb v1.7.0 h1:hpuvQjCSVSLnTnHXn+QAMR0mLmb1gA6wl10LExo2Ts0=\ngithub.com/ipld/go-codec-dagpb v1.7.0/go.mod h1:rD3Zg+zub9ZnxcLwfol/OTQRVjaLzXypgy4UqHQvilM=\ngithub.com/ipld/go-ipld-prime v0.11.0/go.mod h1:+WIAkokurHmZ/KwzDOMUuoeJgaRQktHtEaLglS3ZeV8=\ngithub.com/ipld/go-ipld-prime v0.22.0 h1:YJhDhjEOvOYaqshd3b4atIWUoRg/rKrgmwCyUHwlbuY=\ngithub.com/ipld/go-ipld-prime v0.22.0/go.mod h1:ol7vKxOOVgEh0iAPuiDalM+0gScXVMA5ZZa4DVrTnEA=\ngithub.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20250821084354-a425e60cd714 h1:cqNk8PEwHnK0vqWln+U/YZhQc9h2NB3KjUjDPZo5Q2s=\ngithub.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20250821084354-a425e60cd714/go.mod h1:ZEUdra3CoqRVRYgAX/jAJO9aZGz6SKtKEG628fHHktY=\ngithub.com/ipshipyard/p2p-forge v0.7.0 h1:PQayexxZC1FR2Vx0XOSbmZ6wDPliidS48I+xXWuF+YU=\ngithub.com/ipshipyard/p2p-forge v0.7.0/go.mod h1:i2wg0p7WmHGyo5vYaK9COZBp8BN5Drncfu3WoQNZlQY=\ngithub.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=\ngithub.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=\ngithub.com/jbenet/go-cienv v0.1.0/go.mod h1:TqNnHUmJgXau0nCzC7kXWeotg3J9W34CUv5Djy1+FlA=\ngithub.com/jbenet/go-temp-err-catcher v0.1.0 h1:zpb3ZH6wIE8Shj2sKS+khgRvf7T7RABoLk/+KKHggpk=\ngithub.com/jbenet/go-temp-err-catcher v0.1.0/go.mod h1:0kJRvmDZXNMIiJirNPEYfhpPwbGVtZVWC34vc5WLsDk=\ngithub.com/jbenet/goprocess v0.0.0-20160826012719-b497e2f366b8/go.mod h1:Ly/wlsjFq/qrU3Rar62tu1gASgGw6chQbSh/XgIIXCY=\ngithub.com/jbenet/goprocess v0.1.3/go.mod h1:5yspPrukOVuOLORacaBi858NqyClJPQxYZlqdZVfqY4=\ngithub.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=\ngithub.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=\ngithub.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ=\ngithub.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=\ngithub.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=\ngithub.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=\ngithub.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=\ngithub.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=\ngithub.com/kami-zh/go-capturer v0.0.0-20171211120116-e492ea43421d/go.mod h1:P2viExyCEfeWGU259JnaQ34Inuec4R38JCyBx2edgD0=\ngithub.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=\ngithub.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=\ngithub.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=\ngithub.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=\ngithub.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=\ngithub.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=\ngithub.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=\ngithub.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=\ngithub.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=\ngithub.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=\ngithub.com/koron/go-ssdp v0.0.6 h1:Jb0h04599eq/CY7rB5YEqPS83HmRfHP2azkxMN2rFtU=\ngithub.com/koron/go-ssdp v0.0.6/go.mod h1:0R9LfRJGek1zWTjN3JUNlm5INCDYGpRDfAptnct63fI=\ngithub.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=\ngithub.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=\ngithub.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=\ngithub.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=\ngithub.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=\ngithub.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=\ngithub.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=\ngithub.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=\ngithub.com/libdns/libdns v1.0.0-beta.1 h1:KIf4wLfsrEpXpZ3vmc/poM8zCATXT2klbdPe6hyOBjQ=\ngithub.com/libdns/libdns v1.0.0-beta.1/go.mod h1:4Bj9+5CQiNMVGf87wjX4CY3HQJypUHRuLvlsfsZqLWQ=\ngithub.com/libp2p/go-buffer-pool v0.0.1/go.mod h1:xtyIz9PMobb13WaxR6Zo1Pd1zXJKYg0a8KiIvDp3TzQ=\ngithub.com/libp2p/go-buffer-pool v0.0.2/go.mod h1:MvaB6xw5vOrDl8rYZGLFdKAuk/hRoRZd1Vi32+RXyFM=\ngithub.com/libp2p/go-buffer-pool v0.1.0 h1:oK4mSFcQz7cTQIfqbe4MIj9gLW+mnanjyFtc6cdF0Y8=\ngithub.com/libp2p/go-buffer-pool v0.1.0/go.mod h1:N+vh8gMqimBzdKkSMVuydVDq+UV5QTWy5HSiZacSbPg=\ngithub.com/libp2p/go-cidranger v1.1.0 h1:ewPN8EZ0dd1LSnrtuwd4709PXVcITVeuwbag38yPW7c=\ngithub.com/libp2p/go-cidranger v1.1.0/go.mod h1:KWZTfSr+r9qEo9OkI9/SIEeAtw+NNoU0dXIXt15Okic=\ngithub.com/libp2p/go-doh-resolver v0.5.0 h1:4h7plVVW+XTS+oUBw2+8KfoM1jF6w8XmO7+skhePFdE=\ngithub.com/libp2p/go-doh-resolver v0.5.0/go.mod h1:aPDxfiD2hNURgd13+hfo29z9IC22fv30ee5iM31RzxU=\ngithub.com/libp2p/go-flow-metrics v0.0.1/go.mod h1:Iv1GH0sG8DtYN3SVJ2eG221wMiNpZxBdp967ls1g+k8=\ngithub.com/libp2p/go-flow-metrics v0.0.3/go.mod h1:HeoSNUrOJVK1jEpDqVEiUOIXqhbnS27omG0uWU5slZs=\ngithub.com/libp2p/go-flow-metrics v0.3.0 h1:q31zcHUvHnwDO0SHaukewPYgwOBSxtt830uJtUx6784=\ngithub.com/libp2p/go-flow-metrics v0.3.0/go.mod h1:nuhlreIwEguM1IvHAew3ij7A8BMlyHQJ279ao24eZZo=\ngithub.com/libp2p/go-libp2p v0.48.0 h1:h2BrLAgrj7X8bEN05K7qmrjpNHYA+6tnsGRdprjTnvo=\ngithub.com/libp2p/go-libp2p v0.48.0/go.mod h1:Q1fBZNdmC2Hf82husCTfkKJVfHm2we5zk+NWmOGEmWk=\ngithub.com/libp2p/go-libp2p-asn-util v0.4.1 h1:xqL7++IKD9TBFMgnLPZR6/6iYhawHKHl950SO9L6n94=\ngithub.com/libp2p/go-libp2p-asn-util v0.4.1/go.mod h1:d/NI6XZ9qxw67b4e+NgpQexCIiFYJjErASrYW4PFDN8=\ngithub.com/libp2p/go-libp2p-core v0.2.4/go.mod h1:STh4fdfa5vDYr0/SzYYeqnt+E6KfEV5VxfIrm0bcI0g=\ngithub.com/libp2p/go-libp2p-core v0.3.0/go.mod h1:ACp3DmS3/N64c2jDzcV429ukDpicbL6+TrrxANBjPGw=\ngithub.com/libp2p/go-libp2p-kad-dht v0.39.0 h1:mww38eBYiUvdsu+Xl/GLlBC0Aa8M+5HAwvafkFOygAM=\ngithub.com/libp2p/go-libp2p-kad-dht v0.39.0/go.mod h1:Po2JugFEkDq9Vig/JXtc153ntOi0q58o4j7IuITCOVs=\ngithub.com/libp2p/go-libp2p-kbucket v0.3.1/go.mod h1:oyjT5O7tS9CQurok++ERgc46YLwEpuGoFq9ubvoUOio=\ngithub.com/libp2p/go-libp2p-kbucket v0.8.0 h1:QAK7RzKJpYe+EuSEATAaaHYMYLkPDGC18m9jxPLnU8s=\ngithub.com/libp2p/go-libp2p-kbucket v0.8.0/go.mod h1:JMlxqcEyKwO6ox716eyC0hmiduSWZZl6JY93mGaaqc4=\ngithub.com/libp2p/go-libp2p-peerstore v0.1.4/go.mod h1:+4BDbDiiKf4PzpANZDAT+knVdLxvqh7hXOujessqdzs=\ngithub.com/libp2p/go-libp2p-pubsub v0.15.0 h1:cG7Cng2BT82WttmPFMi50gDNV+58K626m/wR00vGL1o=\ngithub.com/libp2p/go-libp2p-pubsub v0.15.0/go.mod h1:lr4oE8bFgQaifRcoc2uWhWWiK6tPdOEKpUuR408GFN4=\ngithub.com/libp2p/go-libp2p-pubsub-router v0.6.0 h1:D30iKdlqDt5ZmLEYhHELCMRj8b4sFAqrUcshIUvVP/s=\ngithub.com/libp2p/go-libp2p-pubsub-router v0.6.0/go.mod h1:FY/q0/RBTKsLA7l4vqC2cbRbOvyDotg8PJQ7j8FDudE=\ngithub.com/libp2p/go-libp2p-record v0.3.1 h1:cly48Xi5GjNw5Wq+7gmjfBiG9HCzQVkiZOUZ8kUl+Fg=\ngithub.com/libp2p/go-libp2p-record v0.3.1/go.mod h1:T8itUkLcWQLCYMqtX7Th6r7SexyUJpIyPgks757td/E=\ngithub.com/libp2p/go-libp2p-routing-helpers v0.7.5 h1:HdwZj9NKovMx0vqq6YNPTh6aaNzey5zHD7HeLJtq6fI=\ngithub.com/libp2p/go-libp2p-routing-helpers v0.7.5/go.mod h1:3YaxrwP0OBPDD7my3D0KxfR89FlcX/IEbxDEDfAmj98=\ngithub.com/libp2p/go-libp2p-testing v0.12.0 h1:EPvBb4kKMWO29qP4mZGyhVzUyR25dvfUIK5WDu6iPUA=\ngithub.com/libp2p/go-libp2p-testing v0.12.0/go.mod h1:KcGDRXyN7sQCllucn1cOOS+Dmm7ujhfEyXQL5lvkcPg=\ngithub.com/libp2p/go-libp2p-xor v0.1.0 h1:hhQwT4uGrBcuAkUGXADuPltalOdpf9aag9kaYNT2tLA=\ngithub.com/libp2p/go-libp2p-xor v0.1.0/go.mod h1:LSTM5yRnjGZbWNTA/hRwq2gGFrvRIbQJscoIL/u6InY=\ngithub.com/libp2p/go-msgio v0.0.4/go.mod h1:63lBBgOTDKQL6EWazRMCwXsEeEeK9O2Cd+0+6OOuipQ=\ngithub.com/libp2p/go-msgio v0.3.0 h1:mf3Z8B1xcFN314sWX+2vOTShIE0Mmn2TXn3YCUQGNj0=\ngithub.com/libp2p/go-msgio v0.3.0/go.mod h1:nyRM819GmVaF9LX3l03RMh10QdOroF++NBbxAb0mmDM=\ngithub.com/libp2p/go-netroute v0.4.0 h1:sZZx9hyANYUx9PZyqcgE/E1GUG3iEtTZHUEvdtXT7/Q=\ngithub.com/libp2p/go-netroute v0.4.0/go.mod h1:Nkd5ShYgSMS5MUKy/MU2T57xFoOKvvLR92Lic48LEyA=\ngithub.com/libp2p/go-openssl v0.0.3/go.mod h1:unDrJpgy3oFr+rqXsarWifmJuNnJR4chtO1HmaZjggc=\ngithub.com/libp2p/go-openssl v0.0.4/go.mod h1:unDrJpgy3oFr+rqXsarWifmJuNnJR4chtO1HmaZjggc=\ngithub.com/libp2p/go-reuseport v0.4.0 h1:nR5KU7hD0WxXCJbmw7r2rhRYruNRl2koHw8fQscQm2s=\ngithub.com/libp2p/go-reuseport v0.4.0/go.mod h1:ZtI03j/wO5hZVDFo2jKywN6bYKWLOy8Se6DrI2E1cLU=\ngithub.com/libp2p/go-yamux/v5 v5.0.1 h1:f0WoX/bEF2E8SbE4c/k1Mo+/9z0O4oC/hWEA+nfYRSg=\ngithub.com/libp2p/go-yamux/v5 v5.0.1/go.mod h1:en+3cdX51U0ZslwRdRLrvQsdayFt3TSUKvBGErzpWbU=\ngithub.com/libp2p/zeroconf/v2 v2.2.0 h1:Cup06Jv6u81HLhIj1KasuNM/RHHrJ8T7wOTS4+Tv53Q=\ngithub.com/libp2p/zeroconf/v2 v2.2.0/go.mod h1:fuJqLnUwZTshS3U/bMRJ3+ow/v9oid1n0DmyYyNO1Xs=\ngithub.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=\ngithub.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=\ngithub.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=\ngithub.com/marcopolo/simnet v0.0.4 h1:50Kx4hS9kFGSRIbrt9xUS3NJX33EyPqHVmpXvaKLqrY=\ngithub.com/marcopolo/simnet v0.0.4/go.mod h1:tfQF1u2DmaB6WHODMtQaLtClEf3a296CKQLq5gAsIS0=\ngithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd h1:br0buuQ854V8u83wA0rVZ8ttrq5CpaPZdvrK0LP2lOk=\ngithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd/go.mod h1:QuCEs1Nt24+FYQEqAAncTDPJIuGs+LxK1MCiFL25pMU=\ngithub.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=\ngithub.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ=\ngithub.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=\ngithub.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=\ngithub.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=\ngithub.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=\ngithub.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=\ngithub.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=\ngithub.com/mholt/acmez/v3 v3.1.2 h1:auob8J/0FhmdClQicvJvuDavgd5ezwLBfKuYmynhYzc=\ngithub.com/mholt/acmez/v3 v3.1.2/go.mod h1:L1wOU06KKvq7tswuMDwKdcHeKpFFgkppZy/y0DFxagQ=\ngithub.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=\ngithub.com/miekg/dns v1.1.43/go.mod h1:+evo5L0630/F6ca/Z9+GAqzhjGyn8/c+TBaOyfEl0V4=\ngithub.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=\ngithub.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=\ngithub.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c h1:bzE/A84HN25pxAuk9Eej1Kz9OUelF97nAc82bDquQI8=\ngithub.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c/go.mod h1:0SQS9kMwD2VsyFEB++InYyBJroV/FRmBgcydeSUcJms=\ngithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b h1:z78hV3sbSMAUoyUMM0I83AUIT6Hu17AWfgjzIbtrYFc=\ngithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b/go.mod h1:lxPUiZwKoFL8DUUmalo2yJJUCxbPKtm8OKfqr2/FTNU=\ngithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc h1:PTfri+PuQmWDqERdnNMiD9ZejrlswWrCpBEZgWOiTrc=\ngithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc/go.mod h1:cGKTAVKx4SxOuR/czcZ/E2RSJ3sfHs8FpHhQ5CWMf9s=\ngithub.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1/go.mod h1:pD8RvIylQ358TN4wwqatJ8rNavkEINozVn9DtGI3dfQ=\ngithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 h1:0lgqHvJWHLGW5TuObJrfyEi6+ASTKDBWikGvPqy9Yiw=\ngithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882/go.mod h1:qT0aEB35q79LLornSzeDH75LBf3aH1MV+jB5w9Wasec=\ngithub.com/minio/sha256-simd v0.0.0-20190131020904-2d45a736cd16/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U=\ngithub.com/minio/sha256-simd v0.1.1-0.20190913151208-6de447530771/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=\ngithub.com/minio/sha256-simd v0.1.1/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=\ngithub.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=\ngithub.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=\ngithub.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=\ngithub.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=\ngithub.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=\ngithub.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=\ngithub.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=\ngithub.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=\ngithub.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/mr-tron/base58 v1.1.0/go.mod h1:xcD2VGqlgYjBdcBLw+TuYLr8afG+Hj8g2eTVqeSzSU8=\ngithub.com/mr-tron/base58 v1.1.2/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/mr-tron/base58 v1.1.3/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=\ngithub.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/multiformats/go-base32 v0.0.3/go.mod h1:pLiuGC8y0QR3Ue4Zug5UzK9LjgbkL8NSQj0zQ5Nz/AA=\ngithub.com/multiformats/go-base32 v0.1.0 h1:pVx9xoSPqEIQG8o+UbAe7DNi51oej1NtK+aGkbLYxPE=\ngithub.com/multiformats/go-base32 v0.1.0/go.mod h1:Kj3tFY6zNr+ABYMqeUNeGvkIC/UYgtWibDcT0rExnbI=\ngithub.com/multiformats/go-base36 v0.1.0/go.mod h1:kFGE83c6s80PklsHO9sRn2NCoffoRdUUOENyW/Vv6sM=\ngithub.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0=\ngithub.com/multiformats/go-base36 v0.2.0/go.mod h1:qvnKE++v+2MWCfePClUEjE78Z7P2a1UV0xHgWc0hkp4=\ngithub.com/multiformats/go-multiaddr v0.1.0/go.mod h1:xKVEak1K9cS1VdmPZW3LSIb6lgmoS58qz/pzqmAxV44=\ngithub.com/multiformats/go-multiaddr v0.1.1/go.mod h1:aMKBKNEYmzmDmxfX88/vz+J5IU55txyt0p4aiWVohjo=\ngithub.com/multiformats/go-multiaddr v0.2.0/go.mod h1:0nO36NvPpyV4QzvTLi/lafl2y95ncPj0vFwVF6k6wJ4=\ngithub.com/multiformats/go-multiaddr v0.16.1 h1:fgJ0Pitow+wWXzN9do+1b8Pyjmo8m5WhGfzpL82MpCw=\ngithub.com/multiformats/go-multiaddr v0.16.1/go.mod h1:JSVUmXDjsVFiW7RjIFMP7+Ev+h1DTbiJgVeTV/tcmP0=\ngithub.com/multiformats/go-multiaddr-dns v0.5.0 h1:p/FTyHKX0nl59f+S+dEUe8HRK+i5Ow/QHMw8Nh3gPCo=\ngithub.com/multiformats/go-multiaddr-dns v0.5.0/go.mod h1:yJ349b8TPIAANUyuOzn1oz9o22tV9f+06L+cCeMxC14=\ngithub.com/multiformats/go-multiaddr-fmt v0.1.0 h1:WLEFClPycPkp4fnIzoFoV9FVd49/eQsuaL3/CWe167E=\ngithub.com/multiformats/go-multiaddr-fmt v0.1.0/go.mod h1:hGtDIW4PU4BqJ50gW2quDuPVjyWNZxToGUh/HwTZYJo=\ngithub.com/multiformats/go-multiaddr-net v0.1.1/go.mod h1:5JNbcfBOP4dnhoZOv10JJVkJO0pCCEf8mTnipAo2UZQ=\ngithub.com/multiformats/go-multibase v0.0.1/go.mod h1:bja2MqRZ3ggyXtZSEDKpl0uO/gviWFaSteVbWT51qgs=\ngithub.com/multiformats/go-multibase v0.0.3/go.mod h1:5+1R4eQrT3PkYZ24C3W2Ue2tPwIdYQD509ZjSb5y9Oc=\ngithub.com/multiformats/go-multibase v0.2.0 h1:isdYCVLvksgWlMW9OZRYJEa9pZETFivncJHmHnnd87g=\ngithub.com/multiformats/go-multibase v0.2.0/go.mod h1:bFBZX4lKCA/2lyOFSAoKH5SS6oPyjtnzK/XTFDPkNuk=\ngithub.com/multiformats/go-multicodec v0.10.0 h1:UpP223cig/Cx8J76jWt91njpK3GTAO1w02sdcjZDSuc=\ngithub.com/multiformats/go-multicodec v0.10.0/go.mod h1:wg88pM+s2kZJEQfRCKBNU+g32F5aWBEjyFHXvZLTcLI=\ngithub.com/multiformats/go-multihash v0.0.1/go.mod h1:w/5tugSrLEbWqlcgJabL3oHFKTwfvkofsjW2Qa1ct4U=\ngithub.com/multiformats/go-multihash v0.0.8/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa63epEDmG8nTduyAew=\ngithub.com/multiformats/go-multihash v0.0.10/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa63epEDmG8nTduyAew=\ngithub.com/multiformats/go-multihash v0.0.13/go.mod h1:VdAWLKTwram9oKAatUcLxBNUjdtcVwxObEQBtRfuyjc=\ngithub.com/multiformats/go-multihash v0.0.14/go.mod h1:VdAWLKTwram9oKAatUcLxBNUjdtcVwxObEQBtRfuyjc=\ngithub.com/multiformats/go-multihash v0.0.15/go.mod h1:D6aZrWNLFTV/ynMpKsNtB40mJzmCl4jb1alC0OvHiHg=\ngithub.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=\ngithub.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM=\ngithub.com/multiformats/go-multistream v0.6.1 h1:4aoX5v6T+yWmc2raBHsTvzmFhOI8WVOer28DeBBEYdQ=\ngithub.com/multiformats/go-multistream v0.6.1/go.mod h1:ksQf6kqHAb6zIsyw7Zm+gAuVo57Qbq84E27YlYqavqw=\ngithub.com/multiformats/go-varint v0.0.1/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=\ngithub.com/multiformats/go-varint v0.0.5/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=\ngithub.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=\ngithub.com/multiformats/go-varint v0.1.0 h1:i2wqFp4sdl3IcIxfAonHQV9qU5OsZ4Ts9IOoETFs5dI=\ngithub.com/multiformats/go-varint v0.1.0/go.mod h1:5KVAVXegtfmNQQm/lCY+ATvDzvJJhSkUlGQV9wgObdI=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=\ngithub.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo=\ngithub.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM=\ngithub.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=\ngithub.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=\ngithub.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=\ngithub.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=\ngithub.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=\ngithub.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=\ngithub.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=\ngithub.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=\ngithub.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=\ngithub.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c=\ngithub.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=\ngithub.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=\ngithub.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=\ngithub.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=\ngithub.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=\ngithub.com/onsi/gomega v1.36.3 h1:hID7cr8t3Wp26+cYnfcjR6HpJ00fdogN6dqZ1t6IylU=\ngithub.com/onsi/gomega v1.36.3/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=\ngithub.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=\ngithub.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=\ngithub.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=\ngithub.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=\ngithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0=\ngithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y=\ngithub.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=\ngithub.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 h1:1/WtZae0yGtPq+TI6+Tv1WTxkukpXeMlviSxvL7SRgk=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9/go.mod h1:x3N5drFsm2uilKKuuYo6LdyD8vZAW55sH/9w+pbo1sw=\ngithub.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=\ngithub.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=\ngithub.com/pion/datachannel v1.5.10 h1:ly0Q26K1i6ZkGf42W7D4hQYR90pZwzFOjTq5AuCKk4o=\ngithub.com/pion/datachannel v1.5.10/go.mod h1:p/jJfC9arb29W7WrxyKbepTU20CFgyx5oLo8Rs4Py/M=\ngithub.com/pion/dtls/v3 v3.1.2 h1:gqEdOUXLtCGW+afsBLO0LtDD8GnuBBjEy6HRtyofZTc=\ngithub.com/pion/dtls/v3 v3.1.2/go.mod h1:Hw/igcX4pdY69z1Hgv5x7wJFrUkdgHwAn/Q/uo7YHRo=\ngithub.com/pion/ice/v4 v4.0.10 h1:P59w1iauC/wPk9PdY8Vjl4fOFL5B+USq1+xbDcN6gT4=\ngithub.com/pion/ice/v4 v4.0.10/go.mod h1:y3M18aPhIxLlcO/4dn9X8LzLLSma84cx6emMSu14FGw=\ngithub.com/pion/interceptor v0.1.40 h1:e0BjnPcGpr2CFQgKhrQisBU7V3GXK6wrfYrGYaU6Jq4=\ngithub.com/pion/interceptor v0.1.40/go.mod h1:Z6kqH7M/FYirg3frjGJ21VLSRJGBXB/KqaTIrdqnOic=\ngithub.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=\ngithub.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so=\ngithub.com/pion/mdns/v2 v2.0.7 h1:c9kM8ewCgjslaAmicYMFQIde2H9/lrZpjBkN8VwoVtM=\ngithub.com/pion/mdns/v2 v2.0.7/go.mod h1:vAdSYNAT0Jy3Ru0zl2YiW3Rm/fJCwIeM0nToenfOJKA=\ngithub.com/pion/randutil v0.1.0 h1:CFG1UdESneORglEsnimhUjf33Rwjubwj6xfiOXBa3mA=\ngithub.com/pion/randutil v0.1.0/go.mod h1:XcJrSMMbbMRhASFVOlj/5hQial/Y8oH/HVo7TBZq+j8=\ngithub.com/pion/rtcp v1.2.16 h1:fk1B1dNW4hsI78XUCljZJlC4kZOPk67mNRuQ0fcEkSo=\ngithub.com/pion/rtcp v1.2.16/go.mod h1:/as7VKfYbs5NIb4h6muQ35kQF/J0ZVNz2Z3xKoCBYOo=\ngithub.com/pion/rtp v1.8.19 h1:jhdO/3XhL/aKm/wARFVmvTfq0lC/CvN1xwYKmduly3c=\ngithub.com/pion/rtp v1.8.19/go.mod h1:bAu2UFKScgzyFqvUKmbvzSdPr+NGbZtv6UB2hesqXBk=\ngithub.com/pion/sctp v1.8.39 h1:PJma40vRHa3UTO3C4MyeJDQ+KIobVYRZQZ0Nt7SjQnE=\ngithub.com/pion/sctp v1.8.39/go.mod h1:cNiLdchXra8fHQwmIoqw0MbLLMs+f7uQ+dGMG2gWebE=\ngithub.com/pion/sdp/v3 v3.0.18 h1:l0bAXazKHpepazVdp+tPYnrsy9dfh7ZbT8DxesH5ZnI=\ngithub.com/pion/sdp/v3 v3.0.18/go.mod h1:ZREGo6A9ZygQ9XkqAj5xYCQtQpif0i6Pa81HOiAdqQ8=\ngithub.com/pion/srtp/v3 v3.0.6 h1:E2gyj1f5X10sB/qILUGIkL4C2CqK269Xq167PbGCc/4=\ngithub.com/pion/srtp/v3 v3.0.6/go.mod h1:BxvziG3v/armJHAaJ87euvkhHqWe9I7iiOy50K2QkhY=\ngithub.com/pion/stun/v3 v3.1.1 h1:CkQxveJ4xGQjulGSROXbXq94TAWu8gIX2dT+ePhUkqw=\ngithub.com/pion/stun/v3 v3.1.1/go.mod h1:qC1DfmcCTQjl9PBaMa5wSn3x9IPmKxSdcCsxBcDBndM=\ngithub.com/pion/transport/v3 v3.0.7 h1:iRbMH05BzSNwhILHoBoAPxoB9xQgOaJk+591KC9P1o0=\ngithub.com/pion/transport/v3 v3.0.7/go.mod h1:YleKiTZ4vqNxVwh77Z0zytYi7rXHl7j6uPLGhhz9rwo=\ngithub.com/pion/transport/v4 v4.0.1 h1:sdROELU6BZ63Ab7FrOLn13M6YdJLY20wldXW2Cu2k8o=\ngithub.com/pion/transport/v4 v4.0.1/go.mod h1:nEuEA4AD5lPdcIegQDpVLgNoDGreqM/YqmEx3ovP4jM=\ngithub.com/pion/turn/v4 v4.0.2 h1:ZqgQ3+MjP32ug30xAbD6Mn+/K4Sxi3SdNOTFf+7mpps=\ngithub.com/pion/turn/v4 v4.0.2/go.mod h1:pMMKP/ieNAG/fN5cZiN4SDuyKsXtNTr0ccN7IToA1zs=\ngithub.com/pion/webrtc/v4 v4.1.2 h1:mpuUo/EJ1zMNKGE79fAdYNFZBX790KE7kQQpLMjjR54=\ngithub.com/pion/webrtc/v4 v4.1.2/go.mod h1:xsCXiNAmMEjIdFxAYU0MbB3RwRieJsegSB2JZsGN+8U=\ngithub.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=\ngithub.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=\ngithub.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/polydawn/refmt v0.0.0-20201211092308-30ac6d18308e/go.mod h1:uIp+gprXxxrWSjjklXD+mN4wed/tMfjMMmN/9+JsA9o=\ngithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a h1:cgqrm0F3zwf9IPzca7xN4w+Zy6MC9ZkPvAC8QEWa/iQ=\ngithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a/go.mod h1:ocZfO/tLSHqfScRDNTJbAJR1by4D1lewauX9OwTaPuY=\ngithub.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=\ngithub.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=\ngithub.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=\ngithub.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=\ngithub.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=\ngithub.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=\ngithub.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=\ngithub.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=\ngithub.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=\ngithub.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=\ngithub.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=\ngithub.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=\ngithub.com/quic-go/webtransport-go v0.10.0 h1:LqXXPOXuETY5Xe8ITdGisBzTYmUOy5eSj+9n4hLTjHI=\ngithub.com/quic-go/webtransport-go v0.10.0/go.mod h1:LeGIXr5BQKE3UsynwVBeQrU1TPrbh73MGoC6jd+V7ow=\ngithub.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=\ngithub.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=\ngithub.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=\ngithub.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=\ngithub.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=\ngithub.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=\ngithub.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=\ngithub.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk=\ngithub.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=\ngithub.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=\ngithub.com/shurcooL/go v0.0.0-20200502201357-93f07166e636/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk=\ngithub.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg=\ngithub.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=\ngithub.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546/go.mod h1:TrYk7fJVaAttu97ZZKrO9UbRa8izdowaMIZcxYMbVaw=\ngithub.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=\ngithub.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=\ngithub.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=\ngithub.com/slok/go-http-metrics v0.13.0/go.mod h1:HIr7t/HbN2sJaunvnt9wKP9xoBBVZFo1/KiHU3b0w+4=\ngithub.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY=\ngithub.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=\ngithub.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=\ngithub.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=\ngithub.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=\ngithub.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=\ngithub.com/smola/gocompat v0.2.0/go.mod h1:1B0MlxbmoZNo3h8guHp8HztB3BSYR5itql9qtVc0ypY=\ngithub.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572/go.mod h1:w0SWMsp6j9O/dk4/ZpIhL+3CkG8ofA2vuv7k+ltqUMc=\ngithub.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=\ngithub.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=\ngithub.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=\ngithub.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=\ngithub.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=\ngithub.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=\ngithub.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk=\ngithub.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=\ngithub.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=\ngithub.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=\ngithub.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=\ngithub.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=\ngithub.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns=\ngithub.com/src-d/envconfig v1.0.0/go.mod h1:Q9YQZ7BKITldTBnoxsE5gOeB5y66RyPXeue/R4aaNBc=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=\ngithub.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=\ngithub.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=\ngithub.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=\ngithub.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=\ngithub.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngithub.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=\ngithub.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=\ngithub.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=\ngithub.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=\ngithub.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=\ngithub.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=\ngithub.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=\ngithub.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=\ngithub.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=\ngithub.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=\ngithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d h1:vfofYNRScrDdvS342BElfbETmL1Aiz3i2t0zfRj16Hs=\ngithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d/go.mod h1:RRCYJbIwD5jmqPI9XoAFR0OcDxqUctll6zUj/+B4S48=\ngithub.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c h1:u6SKchux2yDvFQnDHS3lPnIRmfVJ5Sxy3ao2SIdysLQ=\ngithub.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c/go.mod h1:hzIxponao9Kjc7aWznkXaL4U4TWaDSs8zcsY4Ka08nM=\ngithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb h1:Ywfo8sUltxogBpFuMOFRrrSifO788kAFxmvVw31PtQQ=\ngithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb/go.mod h1:ikPs9bRWicNw3S7XpJ8sK/smGwU9WcSVU3dy9qahYBM=\ngithub.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=\ngithub.com/urfave/cli v1.22.10/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=\ngithub.com/wangjia184/sortedset v0.0.0-20160527075905-f5d03557ba30/go.mod h1:YkocrP2K2tcw938x9gCOmT5G5eCD6jsTz0SZuyAqwIE=\ngithub.com/warpfork/go-testmark v0.12.1 h1:rMgCpJfwy1sJ50x0M0NgyphxYYPMOODIJHhsXyEHU0s=\ngithub.com/warpfork/go-testmark v0.12.1/go.mod h1:kHwy7wfvGSPh1rQJYKayD4AbtNaeyZdcGi9tNJTaa5Y=\ngithub.com/warpfork/go-wish v0.0.0-20200122115046-b9ea61034e4a/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw=\ngithub.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0 h1:GDDkbFiaK8jsSDJfjId/PEGEShv6ugrt4kYsC5UIDaQ=\ngithub.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw=\ngithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc h1:BCPnHtcboadS0DvysUuJXZ4lWVv5Bh5i7+tbIyi+ck4=\ngithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc/go.mod h1:r45hJU7yEoA81k6MWNhpMj/kms0n14dkzkxYHoB96UM=\ngithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11 h1:5HZfQkwe0mIfyDmc1Em5GqlNRzcdtlv4HTNmdpt7XH0=\ngithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11/go.mod h1:Wlo/SzPmxVp6vXpGt/zaXhHH0fn4IxgqZc82aKg6bpQ=\ngithub.com/whyrusleeping/cbor-gen v0.3.1 h1:82ioxmhEYut7LBVGhGq8xoRkXPLElVuh5mV67AFfdv0=\ngithub.com/whyrusleeping/cbor-gen v0.3.1/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so=\ngithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f h1:jQa4QT2UP9WYv2nzyawpKMOCl+Z/jW7djv2/J50lj9E=\ngithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f/go.mod h1:p9UJB6dDgdPgMJZs7UjUOdulKyRr9fqkS+6JKAInPy8=\ngithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 h1:EKhdznlJHPMoKr0XTrX+IlJs1LH3lyx2nfr1dOlZ79k=\ngithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1/go.mod h1:8UvriyWtv5Q5EOgjHaSseUEdkQfvwFv1I/In/O2M9gc=\ngithub.com/whyrusleeping/go-logging v0.0.0-20170515211332-0457bb6b88fc/go.mod h1:bopw91TMyo8J3tvftk8xmU2kPmlrt4nScJQZU2hE5EM=\ngithub.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7 h1:E9S12nwJwEOXe2d6gT6qxdvqMnNq+VnSsKPgm2ZZNds=\ngithub.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7/go.mod h1:X2c0RVCI1eSUFI8eLcY3c0423ykwiUdxLJtkDvruhjI=\ngithub.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU=\ngithub.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA=\ngithub.com/x-cray/logrus-prefixed-formatter v0.5.2/go.mod h1:2duySbKsL6M18s5GU7VPsoEPHyzalCE06qoARUCeBBE=\ngithub.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=\ngithub.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=\ngithub.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=\ngithub.com/zeebo/assert v1.1.0 h1:hU1L1vLTHsnO8x8c9KAR5GmM5QscxHg5RNU5z5qbUWY=\ngithub.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=\ngithub.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI=\ngithub.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE=\ngithub.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo=\ngithub.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4=\ngo.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=\ngo.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=\ngo.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ=\ngo.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=\ngo.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=\ngo.opencensus.io v0.22.1/go.mod h1:Ap50jQcDJrx6rB6VgeeFPtuPIf3wMRvRfrfYDO6+BmA=\ngo.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=\ngo.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=\ngo.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=\ngo.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=\ngo.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=\ngo.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 h1:7iP2uCb7sGddAr30RRS6xjKy7AZ2JtTOPA3oolgVSw8=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0/go.mod h1:c7hN3ddxs/z6q9xwvfLPk+UHlWRQyaeR1LdgfL/66l0=\ngo.opentelemetry.io/otel v1.42.0 h1:lSQGzTgVR3+sgJDAU/7/ZMjN9Z+vUip7leaqBKy4sho=\ngo.opentelemetry.io/otel v1.42.0/go.mod h1:lJNsdRMxCUIWuMlVJWzecSMuNjE7dOYyWlqOXWkdqCc=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace v1.40.0 h1:QKdN8ly8zEMrByybbQgv8cWBcdAarwmIPZ6FThrWXJs=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace v1.40.0/go.mod h1:bTdK1nhqF76qiPoCCdyFIV+N/sRHYXYCTQc+3VCi3MI=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.40.0 h1:DvJDOPmSWQHWywQS6lKL+pb8s3gBLOZUtw4N+mavW1I=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.40.0/go.mod h1:EtekO9DEJb4/jRyN4v4Qjc2yA7AtfCBuz2FynRUWTXs=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.40.0 h1:wVZXIWjQSeSmMoxF74LzAnpVQOAFDo3pPji9Y4SOFKc=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.40.0/go.mod h1:khvBS2IggMFNwZK/6lEeHg/W57h/IX6J4URh57fuI40=\ngo.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.40.0 h1:MzfofMZN8ulNqobCmCAVbqVL5syHw+eB2qPRkCMA/fQ=\ngo.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.40.0/go.mod h1:E73G9UFtKRXrxhBsHtG00TB5WxX57lpsQzogDkqBTz8=\ngo.opentelemetry.io/otel/metric v1.42.0 h1:2jXG+3oZLNXEPfNmnpxKDeZsFI5o4J+nz6xUlaFdF/4=\ngo.opentelemetry.io/otel/metric v1.42.0/go.mod h1:RlUN/7vTU7Ao/diDkEpQpnz3/92J9ko05BIwxYa2SSI=\ngo.opentelemetry.io/otel/sdk v1.40.0 h1:KHW/jUzgo6wsPh9At46+h4upjtccTmuZCFAc9OJ71f8=\ngo.opentelemetry.io/otel/sdk v1.40.0/go.mod h1:Ph7EFdYvxq72Y8Li9q8KebuYUr2KoeyHx0DRMKrYBUE=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0 h1:mtmdVqgQkeRxHgRv4qhyJduP3fYJRMX4AtAlbuWdCYw=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0/go.mod h1:4Z2bGMf0KSK3uRjlczMOeMhKU2rhUqdWNoKcYrtcBPg=\ngo.opentelemetry.io/otel/trace v1.42.0 h1:OUCgIPt+mzOnaUTpOQcBiM/PLQ/Op7oq6g4LenLmOYY=\ngo.opentelemetry.io/otel/trace v1.42.0/go.mod h1:f3K9S+IFqnumBkKhRJMeaZeNk9epyhnCmQh/EysQCdc=\ngo.opentelemetry.io/proto/otlp v1.9.0 h1:l706jCMITVouPOqEnii2fIAuO3IVGBRPV5ICjceRb/A=\ngo.opentelemetry.io/proto/otlp v1.9.0/go.mod h1:xE+Cx5E/eEHw+ISFkwPLwCZefwVjY+pqKg1qcK03+/4=\ngo.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=\ngo.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4=\ngo.uber.org/dig v1.19.0/go.mod h1:Us0rSJiThwCv2GteUN0Q7OKvU7n5J4dxZ9JKUXozFdE=\ngo.uber.org/fx v1.24.0 h1:wE8mruvpg2kiiL1Vqd0CC+tr0/24XIB10Iwp2lLWzkg=\ngo.uber.org/fx v1.24.0/go.mod h1:AmDeGyS+ZARGKM4tlH4FY2Jr63VjbEDJHtqXTGP5hbo=\ngo.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=\ngo.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=\ngo.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=\ngo.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=\ngo.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=\ngo.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=\ngo.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=\ngo.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=\ngo.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=\ngo.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=\ngo.uber.org/zap/exp v0.3.0 h1:6JYzdifzYkGmTdRR59oYH+Ng7k49H9qVpWwNSsGJj3U=\ngo.uber.org/zap/exp v0.3.0/go.mod h1:5I384qq7XGxYyByIhHm6jg5CHkGY0nsTfbDLgDDlgJQ=\ngo.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=\ngo.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=\ngo4.org v0.0.0-20230225012048-214862532bf5 h1:nifaUDeh+rPaBCMPMQHZmvJf+QdpLFnuQPwx+LxVmtc=\ngo4.org v0.0.0-20230225012048-214862532bf5/go.mod h1:F57wTi5Lrj6WLyswp5EYV1ncrEbFGHD4hhz6S1ZYeaU=\ngolang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20200602180216-279210d13fed/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=\ngolang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=\ngolang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=\ngolang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=\ngolang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=\ngolang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=\ngolang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=\ngolang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=\ngolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90 h1:jiDhWWeC7jfWqR9c/uplMOqJ0sbNlNWv0UkzE0vX1MA=\ngolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90/go.mod h1:xE1HEv6b+1SCZ5/uscMRjUBKtIxworgEcEi+/n9NQDQ=\ngolang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=\ngolang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=\ngolang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=\ngolang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=\ngolang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=\ngolang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=\ngolang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=\ngolang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=\ngolang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190227160552-c95aed5357e7/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=\ngolang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=\ngolang.org/x/net v0.0.0-20210423184538-5f58ad60dda6/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=\ngolang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=\ngolang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=\ngolang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=\ngolang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=\ngolang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=\ngolang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=\ngolang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190219092855-153ac476189d/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190626221950-04f50cda93cb/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210303074136-134d130e1a04/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210309074719-68d13333faf2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210426080607-c94f62235c83/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=\ngolang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=\ngolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c h1:6a8FdnNk6bTXBjR4AGKFgUKuo+7GnR3FX5L7CbveeZc=\ngolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c/go.mod h1:TpUTTEp9frx7rTdLpC9gFG9kdI7zVLFTFFlqaH2Cncw=\ngolang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=\ngolang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=\ngolang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=\ngolang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=\ngolang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\ngolang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=\ngolang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=\ngolang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=\ngolang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=\ngolang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=\ngolang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20181130052023-1c3d964395ce/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=\ngolang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=\ngolang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=\ngolang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=\ngolang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=\ngolang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s=\ngolang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=\ngolang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=\ngolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da h1:noIWHXmPHxILtqtCOPIhSt0ABwskkZKjD3bXGnZGpNY=\ngolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=\ngonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=\ngonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=\ngoogle.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=\ngoogle.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=\ngoogle.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=\ngoogle.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=\ngoogle.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=\ngoogle.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=\ngoogle.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=\ngoogle.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=\ngoogle.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=\ngoogle.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=\ngoogle.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=\ngoogle.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=\ngoogle.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=\ngoogle.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\ngoogle.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=\ngoogle.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=\ngoogle.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 h1:merA0rdPeUV3YIIfHHcH4qBkiQAc1nfCKSI7lB4cV2M=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409/go.mod h1:fl8J1IvUjCilwZzQowmw2b7HQB2eAuYBabMXzWurF+I=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20260128011058-8636f8732409 h1:H86B94AW+VfJWDqFeEbBPhEtHzJwJfTbgE2lZa54ZAQ=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20260128011058-8636f8732409/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=\ngoogle.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=\ngoogle.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=\ngoogle.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=\ngoogle.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\ngoogle.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=\ngoogle.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=\ngoogle.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc=\ngoogle.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=\ngoogle.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\ngoogle.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=\ngoogle.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\ngoogle.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=\ngoogle.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=\ngopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=\ngopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=\ngopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/src-d/go-cli.v0 v0.0.0-20181105080154-d492247bbc0d/go.mod h1:z+K8VcOYVYcSwSjGebuDL6176A1XskgbtNl64NSg+n8=\ngopkg.in/src-d/go-log.v1 v1.0.1/go.mod h1:GN34hKP0g305ysm2/hctJ0Y8nWP3zxXXJ8GFabTyABE=\ngopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=\ngopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=\ngopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\ngopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\ngopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\ngopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=\nhonnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nhonnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nlukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg=\nlukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo=\npgregory.net/rapid v1.1.0 h1:CMa0sjHSru3puNx+J0MIAuiiEV4N0qj8/cMWGBBCsjw=\npgregory.net/rapid v1.1.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04=\nrsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=\nrsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=\nrsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=\n" }, { "path": "docs/examples/kubo-as-a-library/main.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/path\"\n\ticore \"github.com/ipfs/kubo/core/coreiface\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/plugin/loader\" // This package is needed so that all the preloaded plugins are loaded automatically\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n)\n\n/// ------ Setting up the IPFS Repo\n\nfunc setupPlugins(externalPluginsPath string) error {\n\t// Load any external plugins if available on externalPluginsPath\n\tplugins, err := loader.NewPluginLoader(filepath.Join(externalPluginsPath, \"plugins\"))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error loading plugins: %s\", err)\n\t}\n\n\t// Load preloaded and external plugins\n\tif err := plugins.Initialize(); err != nil {\n\t\treturn fmt.Errorf(\"error initializing plugins: %s\", err)\n\t}\n\n\tif err := plugins.Inject(); err != nil {\n\t\treturn fmt.Errorf(\"error initializing plugins: %s\", err)\n\t}\n\n\treturn nil\n}\n\nfunc createTempRepo() (string, error) {\n\trepoPath, err := os.MkdirTemp(\"\", \"ipfs-shell\")\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to get temp dir: %s\", err)\n\t}\n\n\t// Create a config with default options and a 2048 bit key\n\tcfg, err := config.Init(io.Discard, 2048)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Use TCP-only on loopback with random port for reliable local testing.\n\t// This matches what kubo's test harness uses (test/cli/transports_test.go).\n\t// QUIC/UDP transports are avoided because they may be throttled on CI.\n\tcfg.Addresses.Swarm = []string{\n\t\t\"/ip4/127.0.0.1/tcp/0\",\n\t}\n\n\t// Explicitly disable non-TCP transports for reliability.\n\tcfg.Swarm.Transports.Network.QUIC = config.False\n\tcfg.Swarm.Transports.Network.Relay = config.False\n\tcfg.Swarm.Transports.Network.WebTransport = config.False\n\tcfg.Swarm.Transports.Network.WebRTCDirect = config.False\n\tcfg.Swarm.Transports.Network.Websocket = config.False\n\tcfg.AutoTLS.Enabled = config.False\n\n\t// Disable routing - we don't need DHT for direct peer connections.\n\t// Bitswap works with directly connected peers without needing DHT lookups.\n\tcfg.Routing.Type = config.NewOptionalString(\"none\")\n\n\t// Disable bootstrap for this example - we manually connect only the peers we need.\n\tcfg.Bootstrap = []string{}\n\n\t// When creating the repository, you can define custom settings on the repository, such as enabling experimental\n\t// features (See experimental-features.md) or customizing the gateway endpoint.\n\t// To do such things, you should modify the variable `cfg`. For example:\n\tif *flagExp {\n\t\t// https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-filestore\n\t\tcfg.Experimental.FilestoreEnabled = true\n\t\t// https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-urlstore\n\t\tcfg.Experimental.UrlstoreEnabled = true\n\t\t// https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-p2p\n\t\tcfg.Experimental.Libp2pStreamMounting = true\n\t\t// https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#p2p-http-proxy\n\t\tcfg.Experimental.P2pHttpProxy = true\n\t\t// See also: https://github.com/ipfs/kubo/blob/master/docs/config.md\n\t\t// And: https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md\n\t}\n\n\t// Create the repo with the config\n\terr = fsrepo.Init(repoPath, cfg)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to init ephemeral node: %s\", err)\n\t}\n\n\treturn repoPath, nil\n}\n\n/// ------ Spawning the node\n\n// Creates an IPFS node and returns its coreAPI.\nfunc createNode(ctx context.Context, repoPath string) (*core.IpfsNode, error) {\n\t// Open the repo\n\trepo, err := fsrepo.Open(repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Construct the node\n\n\tnodeOptions := &core.BuildCfg{\n\t\tOnline: true,\n\t\t// For this example, we use NilRouterOption (no routing) since we connect peers directly.\n\t\t// Bitswap works with directly connected peers without needing DHT lookups.\n\t\t// In production, you would typically use:\n\t\t// Routing: libp2p.DHTOption, // Full DHT node (stores and fetches records)\n\t\t// Routing: libp2p.DHTClientOption, // DHT client (only fetches records)\n\t\tRouting: libp2p.NilRouterOption,\n\t\tRepo: repo,\n\t}\n\n\treturn core.NewNode(ctx, nodeOptions)\n}\n\nvar loadPluginsOnce sync.Once\n\n// Spawns a node to be used just for this run (i.e. creates a tmp repo).\nfunc spawnEphemeral(ctx context.Context) (icore.CoreAPI, *core.IpfsNode, error) {\n\tvar onceErr error\n\tloadPluginsOnce.Do(func() {\n\t\tonceErr = setupPlugins(\"\")\n\t})\n\tif onceErr != nil {\n\t\treturn nil, nil, onceErr\n\t}\n\n\t// Create a Temporary Repo\n\trepoPath, err := createTempRepo()\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"failed to create temp repo: %s\", err)\n\t}\n\n\tnode, err := createNode(ctx, repoPath)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tapi, err := coreapi.NewCoreAPI(node)\n\n\treturn api, node, err\n}\n\nfunc connectToPeers(ctx context.Context, ipfs icore.CoreAPI, peers []string) error {\n\tvar wg sync.WaitGroup\n\tpeerInfos := make(map[peer.ID]*peer.AddrInfo, len(peers))\n\tfor _, addrStr := range peers {\n\t\taddr, err := ma.NewMultiaddr(addrStr)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tpii, err := peer.AddrInfoFromP2pAddr(addr)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tpi, ok := peerInfos[pii.ID]\n\t\tif !ok {\n\t\t\tpi = &peer.AddrInfo{ID: pii.ID}\n\t\t\tpeerInfos[pi.ID] = pi\n\t\t}\n\t\tpi.Addrs = append(pi.Addrs, pii.Addrs...)\n\t}\n\n\twg.Add(len(peerInfos))\n\tfor _, peerInfo := range peerInfos {\n\t\tgo func(peerInfo *peer.AddrInfo) {\n\t\t\tdefer wg.Done()\n\t\t\terr := ipfs.Swarm().Connect(ctx, *peerInfo)\n\t\t\tif err != nil {\n\t\t\t\tlog.Printf(\"failed to connect to %s: %s\", peerInfo.ID, err)\n\t\t\t}\n\t\t}(peerInfo)\n\t}\n\twg.Wait()\n\treturn nil\n}\n\nfunc getUnixfsNode(path string) (files.Node, error) {\n\tst, err := os.Stat(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tf, err := files.NewSerialFile(path, false, st)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn f, nil\n}\n\n/// -------\n\nvar flagExp = flag.Bool(\"experimental\", false, \"enable experimental features\")\n\nfunc main() {\n\tflag.Parse()\n\n\t/// --- Part I: Getting a IPFS node running\n\n\tfmt.Println(\"-- Getting an IPFS node running -- \")\n\n\tctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)\n\tdefer cancel()\n\n\t// Spawn a local peer using a temporary path, for testing purposes\n\tipfsA, nodeA, err := spawnEphemeral(ctx)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"failed to spawn peer node: %s\", err))\n\t}\n\n\tpeerCidFile, err := ipfsA.Unixfs().Add(ctx,\n\t\tfiles.NewBytesFile([]byte(\"hello from ipfs 101 in Kubo\")))\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not add File: %s\", err))\n\t}\n\n\tfmt.Printf(\"Added file to peer with CID %s\\n\", peerCidFile.String())\n\n\t// Spawn a node using a temporary path, creating a temporary repo for the run\n\tfmt.Println(\"Spawning Kubo node on a temporary repo\")\n\tipfsB, _, err := spawnEphemeral(ctx)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"failed to spawn ephemeral node: %s\", err))\n\t}\n\n\tfmt.Println(\"IPFS node is running\")\n\n\t/// --- Part II: Adding a file and a directory to IPFS\n\n\tfmt.Println(\"\\n-- Adding and getting back files & directories --\")\n\n\tinputBasePath := \"../example-folder/\"\n\tinputPathFile := inputBasePath + \"ipfs.paper.draft3.pdf\"\n\tinputPathDirectory := inputBasePath + \"test-dir\"\n\n\tsomeFile, err := getUnixfsNode(inputPathFile)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not get File: %s\", err))\n\t}\n\n\tcidFile, err := ipfsB.Unixfs().Add(ctx, someFile)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not add File: %s\", err))\n\t}\n\n\tfmt.Printf(\"Added file to IPFS with CID %s\\n\", cidFile.String())\n\n\tsomeDirectory, err := getUnixfsNode(inputPathDirectory)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not get File: %s\", err))\n\t}\n\n\tcidDirectory, err := ipfsB.Unixfs().Add(ctx, someDirectory)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not add Directory: %s\", err))\n\t}\n\n\tfmt.Printf(\"Added directory to IPFS with CID %s\\n\", cidDirectory.String())\n\n\t/// --- Part III: Getting the file and directory you added back\n\n\toutputBasePath, err := os.MkdirTemp(\"\", \"example\")\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not create output dir (%v)\", err))\n\t}\n\tfmt.Printf(\"output folder: %s\\n\", outputBasePath)\n\toutputPathFile := outputBasePath + strings.Split(cidFile.String(), \"/\")[2]\n\toutputPathDirectory := outputBasePath + strings.Split(cidDirectory.String(), \"/\")[2]\n\n\trootNodeFile, err := ipfsB.Unixfs().Get(ctx, cidFile)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not get file with CID: %s\", err))\n\t}\n\n\terr = files.WriteTo(rootNodeFile, outputPathFile)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not write out the fetched CID: %s\", err))\n\t}\n\n\tfmt.Printf(\"got file back from IPFS (IPFS path: %s) and wrote it to %s\\n\", cidFile.String(), outputPathFile)\n\n\trootNodeDirectory, err := ipfsB.Unixfs().Get(ctx, cidDirectory)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not get file with CID: %s\", err))\n\t}\n\n\terr = files.WriteTo(rootNodeDirectory, outputPathDirectory)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not write out the fetched CID: %s\", err))\n\t}\n\n\tfmt.Printf(\"Got directory back from IPFS (IPFS path: %s) and wrote it to %s\\n\", cidDirectory.String(), outputPathDirectory)\n\n\t/// --- Part IV: Getting a file from another IPFS node\n\n\tfmt.Println(\"\\n-- Connecting to nodeA and fetching content via bitswap --\")\n\n\t// Get nodeA's actual listening address dynamically.\n\t// We configured TCP-only on 127.0.0.1 with random port, so this will be a TCP address.\n\tpeerAddrs, err := ipfsA.Swarm().LocalAddrs(ctx)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not get peer addresses: %s\", err))\n\t}\n\tpeerMa := peerAddrs[0].String() + \"/p2p/\" + nodeA.Identity.String()\n\n\tbootstrapNodes := []string{\n\t\t// In production, use real bootstrap peers like:\n\t\t// \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t// For this example, we only connect to nodeA which has our test content.\n\t\tpeerMa,\n\t}\n\n\tfmt.Println(\"Connecting to peer...\")\n\terr = connectToPeers(ctx, ipfsB, bootstrapNodes)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"failed to connect to peers: %s\", err))\n\t}\n\tfmt.Println(\"Connected to peer\")\n\n\texampleCIDStr := peerCidFile.RootCid().String()\n\n\tfmt.Printf(\"Fetching a file from the network with CID %s\\n\", exampleCIDStr)\n\toutputPath := outputBasePath + exampleCIDStr\n\ttestCID := path.FromCid(peerCidFile.RootCid())\n\n\trootNode, err := ipfsB.Unixfs().Get(ctx, testCID)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not get file with CID: %s\", err))\n\t}\n\n\terr = files.WriteTo(rootNode, outputPath)\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"could not write out the fetched CID: %s\", err))\n\t}\n\n\tfmt.Printf(\"Wrote the file to %s\\n\", outputPath)\n\n\tfmt.Println(\"\\nAll done! You just finalized your first tutorial on how to use Kubo as a library\")\n}\n" }, { "path": "docs/examples/kubo-as-a-library/main_test.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc TestExample(t *testing.T) {\n\tt.Log(\"Starting go run main.go...\")\n\tstart := time.Now()\n\n\tcmd := exec.Command(\"go\", \"run\", \"main.go\")\n\tcmd.Env = append(os.Environ(), \"GOLOG_LOG_LEVEL=error\") // reduce libp2p noise\n\n\t// Stream output to both test log and capture buffer for verification\n\t// This ensures we see progress even if the process is killed\n\tvar buf bytes.Buffer\n\tcmd.Stdout = io.MultiWriter(os.Stdout, &buf)\n\tcmd.Stderr = io.MultiWriter(os.Stderr, &buf)\n\n\terr := cmd.Run()\n\n\telapsed := time.Since(start)\n\tt.Logf(\"Command completed in %v\", elapsed)\n\n\tout := buf.String()\n\tif err != nil {\n\t\tt.Fatalf(\"running example (%v):\\n%s\", err, out)\n\t}\n\n\tif !strings.Contains(out, \"All done!\") {\n\t\tt.Errorf(\"example did not complete successfully, output:\\n%s\", out)\n\t}\n}\n" }, { "path": "docs/experimental-features.md", "content": "# Experimental features of Kubo\n\nThis document contains a list of experimental features in Kubo.\nThese features, commands, and APIs aren't mature, and you shouldn't rely on them.\nOnce they reach maturity, there's going to be mention in the changelog and\nrelease posts. If they don't reach maturity, the same applies, and their code is\nremoved.\n\nSubscribe to https://github.com/ipfs/kubo/issues/3397 to get updates.\n\nWhen you add a new experimental feature to kubo or change an experimental\nfeature, you MUST please make a PR updating this document, and link the PR in\nthe above issue.\n\n- [Raw leaves for unixfs files](#raw-leaves-for-unixfs-files)\n- [ipfs filestore](#ipfs-filestore)\n- [ipfs urlstore](#ipfs-urlstore)\n- [Private Networks](#private-networks)\n- [ipfs p2p](#ipfs-p2p)\n- [p2p http proxy](#p2p-http-proxy)\n- [FUSE](#fuse)\n- [Plugins](#plugins)\n- [Directory Sharding / HAMT](#directory-sharding--hamt)\n- [IPNS PubSub](#ipns-pubsub)\n- [AutoRelay](#autorelay)\n- [Strategic Providing](#strategic-providing)\n- [Graphsync](#graphsync)\n- [Noise](#noise)\n- [Optimistic Provide](#optimistic-provide)\n- [HTTP Gateway over Libp2p](#http-gateway-over-libp2p)\n\n---\n\n## Raw Leaves for unixfs files\n\nAllows files to be added with no formatting in the leaf nodes of the graph.\n\n### State\n\nStable but not used by default.\n\n### In Version\n\n0.4.5\n\n### How to enable\n\nUse `--raw-leaves` flag when calling `ipfs add`. This will save some space when adding files.\n\n### Road to being a real feature\n\nEnabling this feature _by default_ will change the CIDs (hashes) of all newly imported files and will prevent newly imported files from deduplicating against previously imported files. While we do intend on enabling this by default, we plan on doing so once we have a large batch of \"hash-changing\" features we can enable all at once.\n\n## ipfs filestore\n\nAllows files to be added without duplicating the space they take up on disk.\n\n### State\n\nExperimental.\n\n### In Version\n\n0.4.7\n\n### How to enable\n\n> [!WARNING]\n> **SECURITY CONSIDERATION**\n>\n> This feature provides the IPFS [`add` command](https://docs.ipfs.tech/reference/kubo/cli/#ipfs-add) with access to\n> the local filesystem. Consequently, any user with access to CLI or the HTTP [`/v0/add` RPC API](https://docs.ipfs.tech/reference/kubo/rpc/#api-v0-add) can read\n> files from the local filesystem with the same permissions as the Kubo daemon.\n> If you enable this, secure your RPC API using [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations) or custom auth middleware.\n\nModify your ipfs config:\n```\nipfs config --json Experimental.FilestoreEnabled true\n```\n\nThen restart your IPFS node to reload your config.\n\nFinally, when adding files with ipfs add, pass the --nocopy flag to use the\nfilestore instead of copying the files into your local IPFS repo.\n\n### Road to being a real feature\n\n- [ ] Needs more people to use and report on how well it works.\n- [ ] Need to address error states and failure conditions\n - [ ] cleanup of broken filesystem references (if file is deleted)\n - [ ] tests that confirm ability to override preexisting filesystem links (allowing user to fix broken link)\n - [ ] support for a single block having more than one sources in filesystem (blocks can be shared by unrelated files, and not be broken when some files are unpinned / gc'd)\n - [ ] [other known issues](https://github.com/ipfs/kubo/issues/7161)\n- [ ] Need to write docs on usage, advantages, disadvantages\n- [ ] Need to merge utility commands to aid in maintenance and repair of filestore\n\n## ipfs urlstore\n\nAllows ipfs to retrieve blocks contents via a URL instead of storing it in the datastore\n\n### State\n\nExperimental.\n\n### In Version\n\nv0.4.17\n\n### How to enable\n\n> [!WARNING]\n> **SECURITY CONSIDERATION**\n>\n> This feature provides the IPFS [`add` CLI command](https://docs.ipfs.tech/reference/kubo/cli/#ipfs-add) with access to\n> the local filesystem. Consequently, any user with access to the CLI or HTTP [`/v0/add` RPC API](https://docs.ipfs.tech/reference/kubo/rpc/#api-v0-add) can read\n> files from the local filesystem with the same permissions as the Kubo daemon.\n> If you enable this, secure your RPC API using [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations) or custom auth middleware.\n\nModify your ipfs config:\n```\nipfs config --json Experimental.UrlstoreEnabled true\n```\n\nAnd then add a file at a specific URL using `ipfs urlstore add `\n\n### Road to being a real feature\n- [ ] Needs more people to use and report on how well it works.\n- [ ] Need to address error states and failure conditions\n - [ ] cleanup of broken URL+range references (if URL starts returning 404 or error)\n - [ ] tests that confirm ability to override preexisting URL+range links (allowing user to fix broken link)\n - [ ] support for a single block having more than one URL+range (blocks can be shared by unrelated URLs)\n- [ ] Need to write docs on usage, advantages, disadvantages\n- [ ] Need to implement caching\n- [ ] Need to add metrics to monitor performance\n\n## Private Networks\n\nIt allows ipfs to only connect to other peers who have a shared secret key.\n\n### State\n\nStable but not quite ready for prime-time.\n\n> [!WARNING]\n> Limited to TCP transport, comes with overhead of double-encryption. See details below.\n\n### In Version\n\n0.4.7\n\n### How to enable\n\nGenerate a pre-shared-key using [ipfs-swarm-key-gen](https://github.com/Kubuxu/go-ipfs-swarm-key-gen)):\n```\ngo install github.com/Kubuxu/go-ipfs-swarm-key-gen/ipfs-swarm-key-gen@latest\nipfs-swarm-key-gen > ~/.ipfs/swarm.key\n```\n\nTo join a given private network, get the key file from someone in the network\nand save it to `~/.ipfs/swarm.key` (If you are using a custom `$IPFS_PATH`, put\nit in there instead).\n\nWhen using this feature, you will not be able to connect to the default bootstrap\nnodes (Since we aren't part of your private network) so you will need to set up\nyour own bootstrap nodes.\n\nFirst, to prevent your node from even trying to connect to the default bootstrap nodes, run:\n```bash\nipfs bootstrap rm --all\n```\n\nThen add your own bootstrap peers with:\n```bash\nipfs bootstrap add \n```\n\nFor example:\n```\nipfs bootstrap add /ip4/104.236.76.40/tcp/4001/p2p/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64\n```\n\nBootstrap nodes are no different from all other nodes in the network apart from\nthe function they serve.\n\nTo be extra cautious, You can also set the `LIBP2P_FORCE_PNET` environment\nvariable to `1` to force the usage of private networks. If no private network is\nconfigured, the daemon will fail to start.\n\n### Road to being a real feature\n\n- [x] Needs more people to use and report on how well it works\n- [ ] More documentation\n- [ ] Improve / future proof libp2p support (see [libp2p/specs#489](https://github.com/libp2p/specs/issues/489))\n - [ ] Currently limited to TCP-only, and double-encrypts all data sent on TCP. This is slow.\n - [ ] Does not work with QUIC: [go-libp2p#1432](https://github.com/libp2p/go-libp2p/issues/1432)\n- [ ] Needs better tooling/UX\n - [ ] Detect lack of peers when swarm key is present and prompt user to set up bootstrappers/peering\n - [ ] ipfs-webui will not load unless blocks are present in private swarm. Detect it and prompt user to import CAR with webui.\n\n## ipfs p2p\n\nAllows tunneling of TCP connections through libp2p streams, similar to SSH port\nforwarding (`ssh -L`).\n\n### State\n\nExperimental, will be stabilized in 0.6.0\n\n### In Version\n\n0.4.10\n\n### How to enable\n\n> [!WARNING]\n> **SECURITY CONSIDERATION**\n>\n> This feature provides CLI and HTTP RPC user with ability to set up port forwarding for all localhost and LAN ports.\n> If you enable this and plan to expose CLI or HTTP RPC to other users or machines,\n> secure RPC API using [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations) or custom auth middleware.\n\n```sh\n> ipfs config --json Experimental.Libp2pStreamMounting true\n```\n\n### How to use\n\nSee [docs/p2p-tunnels.md](p2p-tunnels.md) for usage examples, foreground mode,\nand systemd integration.\n\n### Road to being a real feature\n\n- [x] More documentation\n- [x] `ipfs p2p forward` mode\n- [ ] Ability to define tunnels via JSON config, similar to [`Peering.Peers`](https://github.com/ipfs/kubo/blob/master/docs/config.md#peeringpeers), see [kubo#5460](https://github.com/ipfs/kubo/issues/5460)\n\n## p2p http proxy\n\nAllows proxying of HTTP requests over p2p streams. This allows serving any standard HTTP app over p2p streams.\n\n### State\n\nExperimental\n\n### In Version\n\n0.4.19\n\n### How to enable\n\n> [!WARNING]\n> **SECURITY CONSIDERATION**\n>\n> This feature provides CLI and HTTP RPC user with ability to set up HTTP forwarding for all localhost and LAN ports.\n> If you enable this and plan to expose CLI or HTTP RPC to other users or machines,\n> secure RPC API using [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations) or custom auth middleware.\n\nThe `p2p` command needs to be enabled in the config:\n\n```sh\n> ipfs config --json Experimental.Libp2pStreamMounting true\n```\n\nOn the client, the p2p HTTP proxy needs to be enabled in the config:\n\n```sh\n> ipfs config --json Experimental.P2pHttpProxy true\n```\n\n### How to use\n\n**Netcat example:**\n\nFirst, pick a protocol name for your application. Think of the protocol name as\na port number, just significantly more user-friendly. In this example, we're\ngoing to use `/http`.\n\n***Setup:***\n\n1. A \"server\" node with peer ID `$SERVER_ID`\n2. A \"client\" node.\n\n***On the \"server\" node:***\n\nFirst, start your application and have it listen for TCP connections on\nport `$APP_PORT`.\n\nThen, configure the p2p listener by running:\n\n```sh\n> ipfs p2p listen --allow-custom-protocol /http /ip4/127.0.0.1/tcp/$APP_PORT\n```\n\nThis will configure IPFS to forward all incoming `/http` streams to\n`127.0.0.1:$APP_PORT` (opening a new connection to `127.0.0.1:$APP_PORT` per incoming stream.\n\n***On the \"client\" node:***\n\nNext, have your application make a http request to `127.0.0.1:8080/p2p/$SERVER_ID/http/$FORWARDED_PATH`. This\nconnection will be forwarded to the service running on `127.0.0.1:$APP_PORT` on\nthe remote machine (which needs to be a http server!) with path `$FORWARDED_PATH`. You can test it with netcat:\n\n***On \"server\" node:***\n```sh\n> echo -e \"HTTP/1.1 200\\nContent-length: 11\\n\\nIPFS rocks!\" | nc -l -p $APP_PORT\n```\n\n***On \"client\" node:***\n```sh\n> curl http://localhost:8080/p2p/$SERVER_ID/http/\n```\n\nYou should now see the resulting HTTP response: IPFS rocks!\n\n### Custom protocol names\n\nWe also support the use of protocol names of the form /x/$NAME/http where $NAME doesn't contain any \"/\"'s\n\n### Road to being a real feature\n\n- [ ] Needs p2p streams to graduate from experiments\n- [ ] Needs more people to use and report on how well it works and fits use cases\n- [ ] More documentation\n- [ ] Need better integration with the subdomain gateway feature.\n\n## FUSE\n\nFUSE makes it possible to mount `/ipfs`, `/ipns` and `/mfs` namespaces in your OS,\nallowing arbitrary apps access to IPFS using a subset of filesystem abstractions.\n\nIt is considered EXPERIMENTAL due to limited (and buggy) support on some platforms.\n\nSee [fuse.md](./fuse.md) for more details.\n\n## Plugins\n\n### In Version\n0.4.11\n\n### State\nExperimental\n\nPlugins allow adding functionality without the need to recompile the daemon.\n\n### Basic Usage:\n\nSee [Plugin docs](./plugins.md)\n\n### Road to being a real feature\n\n- [x] More plugins and plugin types\n- [ ] A way to reliably build and distribute plugins.\n- [ ] Better support for platforms other than Linux & MacOS\n- [ ] Feedback on stability\n\n## Directory Sharding / HAMT\n\n### In Version\n\n- 0.4.8:\n - Introduced `Experimental.ShardingEnabled` which enabled sharding globally.\n - All-or-nothing, unnecessary sharding of small directories.\n\n- 0.11.0 :\n - Removed support for `Experimental.ShardingEnabled`\n - Replaced with automatic sharding based on the block size\n\n### State\n\nReplaced by autosharding.\n\nThe `Experimental.ShardingEnabled` config field is no longer used, please remove it from your configs.\n\nkubo now automatically shards when directory block is bigger than 256KB, ensuring every block is small enough to be exchanged with other peers\n\n## IPNS pubsub\n\nSpecification: [IPNS PubSub Router](https://specs.ipfs.tech/ipns/ipns-pubsub-router/)\n\n### In Version\n\n0.4.14 :\n - Introduced\n\n0.5.0 :\n - No longer needs to use the DHT for the first resolution\n - When discovering PubSub peers via the DHT, the DHT key is different from previous versions\n - This leads to 0.5 IPNS pubsub peers and 0.4 IPNS pubsub peers not being able to find each other in the DHT\n - Robustness improvements\n\n0.11.0 :\n - Can be enabled via `Ipns.UsePubsub` flag in config\n\n0.40.0 :\n - Persistent message sequence number validation to prevent message cycles\n in large networks\n\n### State\n\nExperimental, default-disabled.\n\nUtilizes pubsub for publishing IPNS records in real time.\n\nWhen it is enabled:\n\n- IPNS publishers push records to a name-specific pubsub topic,\n in addition to publishing to the DHT.\n- IPNS resolvers subscribe to the name-specific topic on first\n resolution and receive subsequently published records through pubsub in real time.\n This makes subsequent resolutions instant, as they are resolved through the local cache.\n\nBoth the publisher and the resolver nodes need to have the feature enabled for it to work effectively.\n\n### How to enable\n\nRun your daemon with the `--enable-namesys-pubsub` flag\nor modify your ipfs config and restart the daemon:\n```\nipfs config --json Ipns.UsePubsub true\n```\n\nNOTE:\n- This feature implicitly enables pubsub.\n- Passing `--enable-namesys-pubsub` CLI flag overrides `Ipns.UsePubsub` config.\n\n### Road to being a real feature\n\n- [ ] Needs more people to use and report on how well it works\n\n## AutoRelay\n\n### In Version\n\n- 0.4.19 :\n - Introduced Circuit Relay v1\n- 0.11.0 :\n - Deprecated v1\n - Introduced [Circuit Relay v2](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md)\n\n### State\n\nExperimental, disabled by default.\n\nAutomatically discovers relays and advertises relay addresses when the node is behind an impenetrable NAT.\n\n### How to enable\n\nModify your ipfs config:\n\n```\nipfs config --json Swarm.RelayClient.Enabled true\n```\n\n### Road to being a real feature\n\n- [ ] needs testing\n- [ ] needs to be automatically enabled when AutoNAT detects node is behind an impenetrable NAT.\n\n\n## Strategic Providing\n\n### State\n\n`Experimental.StrategicProviding` was removed in Kubo v0.35.\n\nReplaced by [`Provide.Enabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#provideenabled) and [`Provide.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providestrategy).\n\n## GraphSync\n\n### State\n\nRemoved, no plans to reintegrate either as experimental or stable feature.\n\n[Trustless Gateway over Libp2p](#http-gateway-over-libp2p) should be easier to use for unixfs usecases and support basic wildcard car streams for non unixfs.\n\nSee https://github.com/ipfs/kubo/pull/9747 for more information.\n\n## Noise\n\n### State\n\nStable, enabled by default\n\n[Noise](https://github.com/libp2p/specs/tree/master/noise) libp2p transport based on the [Noise Protocol Framework](https://noiseprotocol.org/noise.html). While TLS remains the default transport in Kubo, Noise is easier to implement and is thus the \"interop\" transport between IPFS and libp2p implementations.\n\n## Optimistic Provide\n\n### In Version\n\n0.20.0\n\n### State\n\nExperimental, disabled by default.\n\nWhen the Amino DHT client tries to store a provider in the DHT, it typically searches for the 20 peers that are closest to the\ntarget key. However, this process can be time-consuming, as the search terminates only after no closer peers are found\namong the three currently (during the query) known closest ones. In cases where these closest peers are slow to respond\n(which often happens if they are located at the edge of the DHT network), the query gets blocked by the slowest peer.\n\nTo address this issue, the `OptimisticProvide` feature can be enabled. This feature allows the client to estimate the\nnetwork size and determine how close a peer _likely_ needs to be to the target key to be within the 20 closest peers.\nWhile searching for the closest peers in the DHT, the client will _optimistically_ store the provider record with peers\nand abort the query completely when the set of currently known 20 closest peers are also _likely_ the actual 20 closest\nones. This heuristic approach can significantly speed up the process, resulting in a speed improvement of 2x to >10x.\n\nWhen it is enabled:\n\n- Amino DHT provide operations should complete much faster than with it disabled\n- This can be tested with commands such as `ipfs routing provide`\n\n**Tradeoffs**\n\nThere are now the classic client, the accelerated DHT client, and optimistic provide that improve the provider process.\nThere are different trade-offs with all of them. The accelerated DHT client is still faster to provide large amounts\nof provider records at the cost of high resource requirements. Optimistic provide doesn't have the high resource\nrequirements but might not choose optimal peers and is not as fast as the accelerated client, but still much faster\nthan the classic client.\n\n**Caveats:**\n\n1. Providing optimistically requires a current network size estimation. This estimation is calculated through routing\n table refresh queries and is only available after the daemon has been running for some time. If there is no network\n size estimation available the client will transparently fall back to the classic approach.\n2. The chosen peers to store the provider records might not be the actual closest ones. Measurements showed that this\n is not a problem.\n3. The optimistic provide process returns already after 15 out of the 20 provider records were stored with peers. The\n reasoning here is that one out of the remaining 5 peers are very likely to time out and delay the whole process. To\n limit the number of in-flight async requests there is the second `OptimisticProvideJobsPoolSize` setting. Currently,\n this is set to 60. This means that at most 60 parallel background requests are allowed to be in-flight. If this\n limit is exceeded optimistic provide will block until all 20 provider records are written. This is still 2x faster\n than the classic approach but not as fast as returning early which yields >10x speed-ups.\n4. Since the in-flight background requests are likely to time out, they are not consuming many resources and the job\n pool size could probably be much higher.\n\nFor more information, see:\n\n- Project doc: https://protocollabs.notion.site/Optimistic-Provide-2c79745820fa45649d48de038516b814\n- go-libp2p-kad-dht: https://github.com/libp2p/go-libp2p-kad-dht/pull/783\n\n### Configuring\nTo enable:\n\n```\nipfs config --json Experimental.OptimisticProvide true\n```\n\nIf you want to change the `OptimisticProvideJobsPoolSize` setting from its default of 60:\n\n```\nipfs config --json Experimental.OptimisticProvideJobsPoolSize 120\n```\n\n### Road to being a real feature\n\n- [ ] Needs more people to use and report on how well it works\n- [ ] Should prove at least equivalent availability of provider records as the classic approach\n\n## HTTP Gateway over Libp2p\n\n### In Version\n\n0.23.0\n\n### State\n\nExperimental, disabled by default.\n\nEnables serving a subset of the [IPFS HTTP Gateway](https://specs.ipfs.tech/http-gateways/) semantics over libp2p `/http/1.1` protocol.\n\nNotes:\n- This feature only about serving verifiable gateway requests over libp2p:\n - Deserialized responses are not supported.\n - Only operate on `/ipfs` resources (no `/ipns` atm)\n - Only support requests for `application/vnd.ipld.raw` and\n `application/vnd.ipld.car` (from [Trustless Gateway Specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/),\n where data integrity can be verified).\n - Only serve data that is already local to the node (i.e. similar to a\n [`Gateway.NoFetch`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaynofetch))\n- While Kubo currently mounts the gateway API at the root (i.e. `/`) of the\n libp2p `/http/1.1` protocol, that is subject to change.\n - The way to reliably discover where a given HTTP protocol is mounted on a\n libp2p endpoint is via the `.well-known/libp2p` resource specified in the\n [http+libp2p specification](https://github.com/libp2p/specs/pull/508)\n - The identifier of the protocol mount point under `/http/1.1` listener is\n `/ipfs/gateway`, as noted in\n [ipfs/specs#434](https://github.com/ipfs/specs/pull/434).\n\n### How to enable\n\nModify your ipfs config:\n\n```\nipfs config --json Experimental.GatewayOverLibp2p true\n```\n\n### Road to being a real feature\n\n- [ ] Needs more people to use and report on how well it works\n- [ ] Needs UX work for exposing non-recursive \"HTTP transport\" (NoFetch) over both libp2p and plain TCP (and sharing the configuration)\n- [ ] Needs a mechanism for HTTP handler to signal supported features ([IPIP-425](https://github.com/ipfs/specs/pull/425))\n- [ ] Needs an option for Kubo to detect peers that have it enabled and prefer HTTP transport before falling back to bitswap (and use CAR if peer supports dag-scope=entity from [IPIP-402](https://specs.ipfs.tech/ipips/ipip-0402/))\n\n## Accelerated DHT Client\n\nThis feature now lives at [`Routing.AcceleratedDHTClient`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingaccelerateddhtclient).\n\n" }, { "path": "docs/file-transfer.md", "content": "# Transferring a file with ipfs\nThis document is a guide to help troubleshoot transferring a file between two\nmachines using ipfs.\n\n## A file transfer\nTo start, make sure that ipfs is running on both machines. To verify, run `ipfs\nid` on each machine and check if the `Addresses` field has anything in it. If\nit says `null`, then your node is not online and you will need to run `ipfs\ndaemon`.\n\nNow, lets call the node with the file you want to transfer node 'A' and the\nnode you want to get the file to node 'B'. On node A, add the file to ipfs\nusing the `ipfs add` command. This will print out the multihash of the content\nyou added. Now, on node B, you can fetch the content using `ipfs get `.\n\n```\n# On A\n> ipfs add myfile.txt\nadded QmZJ1xT1T9KYkHhgRhbv8D7mYrbemaXwYUkg7CeHdrk1Ye myfile.txt\n\n# On B\n> ipfs get QmZJ1xT1T9KYkHhgRhbv8D7mYrbemaXwYUkg7CeHdrk1Ye\nSaving file(s) to QmZJ1xT1T9KYkHhgRhbv8D7mYrbemaXwYUkg7CeHdrk1Ye\n 13 B / 13 B [=====================================================] 100.00% 1s\n ```\n\nIf that worked, and downloaded the file, then congratulations! You just used\nipfs to move files across the internet! But, if that `ipfs get` command is\nhanging, with no output, read onwards.\n\n## Troubleshooting\n\nSo your ipfs file transfer appears to not be working. The primary reason this\nhappens is because node B cannot figure out how to connect to node A, or node B\ndoesn't even know it has to connect to node A. \n\n### Checking for existing connections \n\nThe first thing to do is to double-check that both nodes are in fact running\nand online. To do this, run `ipfs id` on each machine. If both nodes show some\naddresses (like the example below), then your nodes are online.\n\n```json\n{\n \"ID\": \"QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8\",\n \"PublicKey\": \"CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDZb6znj3LQZKP1+X81exf+vbnqNCMtHjZ5RKTCm7Fytnfe+AI1fhs9YbZdkgFkM1HLxmIOLQj2bMXPIGxUM+EnewN8tWurx4B3+lR/LWNwNYcCFL+jF2ltc6SE6BC8kMLEZd4zidOLPZ8lIRpd0x3qmsjhGefuRwrKeKlR4tQ3C76ziOms47uLdiVVkl5LyJ5+mn4rXOjNKt/oy2O4m1St7X7/yNt8qQgYsPfe/hCOywxCEIHEkqmil+vn7bu4RpAtsUzCcBDoLUIWuU3i6qfytD05hP8Clo+at+l//ctjMxylf3IQ5qyP+yfvazk+WHcsB0tWueEmiU5P2nfUUIR3AgMBAAE=\",\n \"Addresses\": [\n \"/ip4/127.0.0.1/tcp/4001/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8\",\n \"/ip4/127.0.0.1/udp/4001/quic-v1/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8\",\n \"/ip4/192.168.2.131/tcp/4001/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8\",\n \"/ip4/192.168.2.131/udp/4001/quic-v1/p2p/QmTNwsFkLAed15kQEC1ZJWPfoNbBQnMFojfJKQ9sZj1dk8\",\n ],\n \"AgentVersion\": \"go-ipfs/0.4.11-dev/\",\n \"ProtocolVersion\": \"ipfs/0.1.0\"\n}\n```\n\nNext, check to see if the nodes have a connection to each other. You can do this\nby running `ipfs swarm peers` on one node, and checking for the other nodes\npeer ID in the output. If the two nodes *are* connected, and the `ipfs get`\ncommand is still hanging, then something unexpected is going on, and I\nrecommend filing an issue about it. If they are not connected, then let's try\nand debug why. (Note: you can skip to 'Manually connecting node A to node B' if\nyou just want things to work. Going through the debugging process and reporting\nwhat happened to the ipfs team on IRC is helpful to us to understand common\npitfalls that people run into)\n\n### Checking providers\nWhen requesting content on ipfs, nodes search the DHT for 'provider records' to\nsee who has what content. Let's manually do that on node B to make sure that\nnode B is able to determine that node A has the data. Run `ipfs routing findprovs\n`. We expect to see the peer ID of node A printed out. If this command\nreturns nothing (or returns IDs that are not node A), then no record of A\nhaving the data exists on the network. This can happen if the data is added\nwhile node A does not have a daemon running. If this happens, you can run `ipfs\nrouting provide ` on node A to announce to the network that you have that\nhash. Then if you restart the `ipfs get` command, node B should now be able\nto tell that node A has the content it wants. If node A's peer ID showed up in\nthe initial `findprovs` call, or manually providing the hash didn't resolve the\nproblem, then it's likely that node B is unable to make a connection to node A.\n\n### Checking addresses\n\nIn the case where node B simply cannot form a connection to node A, despite\nknowing that it needs to, the likely culprit is a bad NAT. When node B learns\nthat it needs to connect to node A, it checks the DHT for addresses for node A,\nand then starts trying to connect to them. We can check those addresses by\nrunning `ipfs routing findpeer ` on node B. This command should\nreturn a list of addresses for node A. If it doesn't return any addresses, then\nyou should try running the manual providing command from the previous steps. \nExample output of addresses might look something like this:\n\n```\n/ip4/127.0.0.1/tcp/4001\n/ip4/127.0.0.1/udp/4001/quic-v1\n/ip4/192.168.2.133/tcp/4001\n/ip4/192.168.2.133/udp/4001/quic-v1\n/ip4/88.157.217.196/tcp/63674\n/ip4/88.157.217.196/udp/63674/quic-v1\n```\n\nIn this case, we can see a localhost (127.0.0.1) address, a LAN address (the\n192.168.*.* one) and another address. If this third address matches your\nexternal IP, then the network knows a valid external address for your node. At\nthis point, its safe to assume that your node has a difficult to traverse NAT\nsituation. If this is the case, you can try to enable UPnP or NAT-PMP on the\nrouter of node A and retry the process. Otherwise, you can try manually\nconnecting node A to node B.\n\n### Manually connecting node A to B \n\nOn node B run `ipfs id` and take one of the multiaddrs that contains its public\nip address, and then on node A run `ipfs swarm connect `. You can\nalso try using a relayed connection, for more information [read this\ndoc](./experimental-features.md#circuit-relay). If that *still* doesn't work,\nthen you should either join IRC and ask for help there, or file an issue on\ngithub.\n\nIf this manual step *did* work, then you likely have an issue with NAT\ntraversal, and ipfs cannot figure out how to make it through. Please report\nsituations like this to us so we can work on fixing them.\n" }, { "path": "docs/fuse.md", "content": "# FUSE\n\n**EXPERIMENTAL:** FUSE support is limited, YMMV.\n\nKubo makes it possible to mount `/ipfs`, `/ipns` and `/mfs` namespaces in your OS,\nallowing arbitrary apps access to IPFS.\n\n## Install FUSE\n\nYou will need to install and configure fuse before you can mount IPFS\n\n#### Linux\n\nNote: while this guide should work for most distributions, you may need to refer\nto your distribution manual to get things working.\n\nInstall `fuse` with your favorite package manager:\n```\nsudo apt-get install fuse3\n```\n\nOn some older Linux distributions, you may need to add yourself to the `fuse` group. \n(If no such group exists, you can probably skip this step)\n```sh\nsudo usermod -a -G fuse \n```\n\nRestart user session, if active, for the change to apply, either by restarting\nssh connection or by re-logging to the system.\n\n#### Mac OSX -- OSXFUSE\n\nIt has been discovered that versions of `osxfuse` prior to `2.7.0` will cause a\nkernel panic. For everyone's sake, please upgrade (latest at time of writing is\n`2.7.4`). The installer can be found at https://osxfuse.github.io/. There is\nalso a homebrew formula (`brew cask install osxfuse`) but users report best results\ninstalling from the official OSXFUSE installer package.\n\nNote that `ipfs` attempts an automatic version check on `osxfuse` to prevent you\nfrom shooting yourself in the foot if you have pre `2.7.0`. Since checking the\nOSXFUSE version [is more complicated than it should be], running `ipfs mount`\nmay require you to install another binary:\n\n```sh\ngo get github.com/jbenet/go-fuse-version/fuse-version\n```\n\nIf you run into any problems installing FUSE or mounting IPFS, hop on IRC and\nspeak with us, or if you figure something new out, please add to this document!\n\n#### FreeBSD\n```sh\nsudo pkg install fusefs-ext2\n```\n\nLoad the fuse kernel module:\n```sh\nsudo kldload fusefs\n```\n\nTo load automatically on boot:\n```sh\nsudo echo fusefs_load=\"YES\" >> /boot/loader.conf\n```\n\n## Prepare mountpoints\n\nBy default ipfs uses `/ipfs`, `/ipns` and `/mfs` directories for mounting, this can be\nchanged in config. You will have to create the `/ipfs`, `/ipns` and `/mfs` directories\nexplicitly. Note that modifying root requires sudo permissions.\n\n```sh\n# make the directories\nsudo mkdir /ipfs\nsudo mkdir /ipns\nsudo mkdir /mfs\n\n# chown them so ipfs can use them without root permissions\nsudo chown /ipfs\nsudo chown /ipns\nsudo chown /mfs\n```\n\nDepending on whether you are using OSX or Linux, follow the proceeding instructions. \n\n## Make sure IPFS daemon is not running\n\nYou'll need to stop the IPFS daemon if you have it started, otherwise the mount will complain. \n\n```\n# Check to see if IPFS daemon is running\nps aux | grep ipfs\n\n# Kill the IPFS daemon \npkill -f ipfs\n\n# Verify that it has been killed\n```\n\n## Mounting IPFS\n\n```sh\nipfs daemon --mount\n```\n\nIf you wish to allow other users to use the mount points, edit `/etc/fuse.conf`\nto enable non-root users, i.e.:\n```sh\n# /etc/fuse.conf - Configuration file for Filesystem in Userspace (FUSE)\n\n# Set the maximum number of FUSE mounts allowed to non-root users.\n# The default is 1000.\n#mount_max = 1000\n\n# Allow non-root users to specify the allow_other or allow_root mount options.\nuser_allow_other\n```\n\nNext set `Mounts.FuseAllowOther` config option to `true`:\n```sh\nipfs config --json Mounts.FuseAllowOther true\nipfs daemon --mount\n```\n\nIf using FreeBSD, it is necessary to run `ipfs` as root:\n```sh\nsudo HOME=$HOME ipfs daemon --mount\n```\n\n## MFS mountpoint\n\nKubo v0.35.0 and later supports mounting the MFS (Mutable File System) root as\na FUSE filesystem, enabling manipulation of content-addressed data like regular\nfiles. The CID for any file or directory is retrievable via the `ipfs_cid`\nextended attribute.\n\n```sh\ngetfattr -n ipfs_cid /mfs/welcome-to-IPFS.jpg \ngetfattr: Removing leading '/' from absolute path names\n# file: mfs/welcome-to-IPFS.jpg\nipfs_cid=\"QmaeXDdwpUeKQcMy7d5SFBfVB4y7LtREbhm5KizawPsBSH\"\n```\n\nPlease note that the operations supported by the MFS FUSE mountpoint are\nlimited. Since the MFS wasn't designed to store file attributes like ownership\ninformation, permissions and creation date, some applications like `vim` and\n`sed` may misbehave due to missing functionality.\n\n## Troubleshooting\n\n#### `Permission denied` or `fusermount: user has no write access to mountpoint` error in Linux\n\nVerify that the config file can be read by your user:\n```sh\nsudo ls -l /etc/fuse.conf\n-rw-r----- 1 root fuse 216 Jan 2 2013 /etc/fuse.conf\n```\nIn most distributions, the group named `fuse` will be created during fuse\ninstallation. You can check this with:\n\n```sh\nsudo grep -q fuse /etc/group && echo fuse_group_present || echo fuse_group_missing\n```\n\nIf the group is present, just add your regular user to the `fuse` group:\n```sh\nsudo usermod -G fuse -a \n```\n\nIf the group didn't exist, create `fuse` group (add your regular user to it) and\nset necessary permissions, for example:\n```sh\nsudo chgrp fuse /etc/fuse.conf\nsudo chmod g+r /etc/fuse.conf\n```\n\n\nNote that the use of `fuse` group is optional and may depend on your operating\nsystem. It is okay to use a different group as long as proper permissions are\nset for user running `ipfs mount` command.\n\n#### Mount command crashes and mountpoint gets stuck\n\n```\nsudo umount /ipfs\nsudo umount /ipns\nsudo umount /mfs\n```\n\n#### Mounting fails with \"error mounting: could not resolve name\"\n\nMake sure your node's IPNS address has a directory published:\n```\n$ mkdir hello/; echo 'hello' > hello/hello.txt; ipfs add -rQ ./hello/\nQmU5PLEGqjetW4RAmXgHpEFL7nVCL3vFnEyrCKUfRk4MSq\n\n$ ipfs name publish QmU5PLEGqjetW4RAmXgHpEFL7nVCL3vFnEyrCKUfRk4MSq\n```\n\nIf you manage to mount on other systems (or followed an alternative path to one\nabove), please contribute to these docs :D\n" }, { "path": "docs/gateway.md", "content": "# Gateway\n\nAn IPFS Gateway acts as a bridge between traditional web browsers and IPFS.\nThrough the gateway, users can browse files and websites stored in IPFS as if\nthey were stored in a traditional web server. \n\n[More about Gateways](https://docs.ipfs.tech/concepts/ipfs-gateway/) and [addressing IPFS on the web](https://docs.ipfs.tech/how-to/address-ipfs-on-web/).\n\nKubo's Gateway implementation follows [IPFS Gateway Specifications](https://specs.ipfs.tech/http-gateways/) and is tested with [Gateway Conformance Test Suite](https://github.com/ipfs/gateway-conformance).\n\n### Local gateway\n\nBy default, Kubo nodes run\na [path gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#path-gateway) at `http://127.0.0.1:8080/`\nand a [subdomain gateway](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://localhost:8080/`.\n\n> [!CAUTION]\n> **For browsing websites, web apps, and dapps in a browser, use the subdomain\n> gateway** (`localhost`). Each content root gets its own\n> [web origin](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy),\n> isolating localStorage, cookies, and session data between sites.\n>\n> **For file retrieval, use the path gateway** (`127.0.0.1`). Path gateways are\n> suited for downloading files or fetching [verifiable](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval)\n> content, but lack origin isolation (all content shares the same origin).\n\nAdditional listening addresses and gateway behaviors can be set in the [config](#configuration) file.\n\n### Public gateways\n\nIPFS Foundation [provides public gateways](https://docs.ipfs.tech/concepts/public-utilities/) at\n`https://ipfs.io` ([path](https://specs.ipfs.tech/http-gateways/path-gateway/)),\n`https://dweb.link` ([subdomain](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway)),\nand `https://trustless-gateway.link` ([trustless](https://specs.ipfs.tech/http-gateways/trustless-gateway/) only).\nIf you've ever seen a link in the form `https://ipfs.io/ipfs/Qm...`, that's being served from a *public goods* gateway.\n\nThere is a list of third-party public gateways provided by the IPFS community at https://ipfs.github.io/public-gateway-checker/\n\n## Configuration\n\nThe `Gateway.*` configuration options are (briefly) described in the\n[config](https://github.com/ipfs/kubo/blob/master/docs/config.md#gateway)\ndocumentation, including a list of common [gateway recipes](https://github.com/ipfs/kubo/blob/master/docs/config.md#gateway-recipes).\n\n### Debug\nThe gateway's log level can be changed with this command:\n```\n> ipfs log level core/server debug\n```\n\n## Running in Production\n\nWhen deploying Kubo's gateway in production, be aware of these important considerations:\n\n\n> [!IMPORTANT]\n> **Reverse Proxy:** When running Kubo behind a reverse proxy (such as nginx),\n> the original `Host` header **must** be forwarded to Kubo for\n> [`Gateway.PublicGateways`](config.md#gatewaypublicgateways) to work.\n> Kubo uses the `Host` header to match configured hostnames and detect\n> subdomain gateway patterns like `{cid}.ipfs.example.org` or DNSLink hostnames.\n>\n> If the `Host` header is not forwarded correctly, Kubo will not recognize\n> the configured gateway hostnames and requests may be handled incorrectly.\n>\n> If `X-Forwarded-Proto` is not set, redirects over HTTPS will use wrong protocol\n> and DNSLink names will not be inlined for subdomain gateways.\n>\n> Example: minimal nginx configuration for `example.org`\n>\n> ```nginx\n> server {\n> listen 80;\n> listen [::]:80;\n>\n> # IMPORTANT: Include wildcard to match subdomain gateway requests.\n> # The dot prefix matches both apex domain and all subdomains.\n> server_name .example.org;\n>\n> location / {\n> proxy_pass http://127.0.0.1:8080;\n>\n> # IMPORTANT: Forward the original Host header to Kubo.\n> # Without this, PublicGateways configuration will not work.\n> proxy_set_header Host $host;\n>\n> # IMPORTANT: X-Forwarded-Proto is required for correct behavior:\n> # - Redirects will use https:// URLs when set to \"https\"\n> # - DNSLink names will be inlined for subdomain gateways\n> # (e.g., /ipns/en.wikipedia-on-ipfs.org → en-wikipedia--on--ipfs-org.ipns.example.org)\n> proxy_set_header X-Forwarded-Proto $scheme;\n> proxy_set_header X-Forwarded-Host $host;\n> }\n> }\n> ```\n>\n> Common mistakes to avoid:\n>\n> - **Missing wildcard in `server_name`:** Using only `server_name example.org;`\n> will not match subdomain requests like `{cid}.ipfs.example.org`. Always\n> include `*.example.org` or use the dot prefix `.example.org`.\n>\n> - **Wrong `Host` header value:** Using `proxy_set_header Host $proxy_host;`\n> sends the backend's hostname (e.g., `127.0.0.1:8080`) instead of the\n> original `Host` header. Always use `$host` or `$http_host`.\n>\n> - **Missing `Host` header entirely:** If `proxy_set_header Host` is not\n> specified, nginx defaults to `$proxy_host`, which breaks gateway routing.\n\n> [!IMPORTANT]\n> **Timeouts:** Configure [`Gateway.RetrievalTimeout`](config.md#gatewayretrievaltimeout)\n> to terminate stalled transfers (resets on each data write, catches unresponsive operations),\n> and [`Gateway.MaxRequestDuration`](config.md#gatewaymaxrequestduration) as a fallback\n> deadline (default: 1 hour, catches cases when other timeouts are misconfigured or fail to fire).\n\n> [!IMPORTANT]\n> **Rate Limiting:** Use [`Gateway.MaxConcurrentRequests`](config.md#gatewaymaxconcurrentrequests)\n> to protect against traffic spikes.\n\n> [!IMPORTANT]\n> **CDN/Cloudflare:** If using Cloudflare or other CDNs with\n> [deserialized responses](config.md#gatewaydeserializedresponses) enabled, review\n> [`Gateway.MaxRangeRequestFileSize`](config.md#gatewaymaxrangerequestfilesize) to avoid\n> excess bandwidth billing from range request bugs. Cloudflare users may need additional\n> protection via [Cloudflare Snippets](https://github.com/ipfs/boxo/issues/856#issuecomment-3523944976).\n\n## Directories\n\nFor convenience, the gateway (mostly) acts like a normal web-server when serving\na directory:\n\n1. If the directory contains an `index.html` file:\n 1. If the path does not end in a `/`, append a `/` and redirect. This helps\n avoid serving duplicate content from different paths.†\n 2. Otherwise, serve the `index.html` file.\n2. Dynamically build and serve a listing of the contents of the directory.\n\n†This redirect is skipped if the query string contains a\n`go-get=1` parameter. See [PR#3963](https://github.com/ipfs/kubo/pull/3963)\nfor details\n\n## Static Websites\n\nYou can use an IPFS gateway to serve static websites at a custom domain using\n[DNSLink](https://docs.ipfs.tech/concepts/glossary/#dnslink). See [Example: IPFS\nGateway](https://dnslink.dev/#example-ipfs-gateway) for instructions.\n\n## Filenames\n\nWhen downloading files, browsers will usually guess a file's filename by looking\nat the last component of the path. Unfortunately, when linking *directly* to a\nfile (with no containing directory), the final component is just a CID\n(`bafy..` or `Qm...`). This isn't exactly user-friendly.\n\nTo work around this issue, you can add a `filename=some_filename` parameter to\nyour query string to explicitly specify the filename. For example:\n\n> https://ipfs.io/ipfs/QmfM2r8seH2GiRaC4esTjeraXEachRt8ZsSeGaWTPLyMoG?filename=hello_world.txt\n\nWhen you try to save above page, you browser will use passed `filename` instead of a CID.\n\n## Downloads\n\nIt is possible to skip browser rendering of supported filetypes (plain text,\nimages, audio, video, PDF) and trigger immediate \"save as\" dialog by appending\n`&download=true`:\n\n> https://ipfs.io/ipfs/QmfM2r8seH2GiRaC4esTjeraXEachRt8ZsSeGaWTPLyMoG?filename=hello_world.txt&download=true\n\n## Response Format\n\nAn explicit response format can be requested using `?format=raw|car|..` URL parameter,\nor by sending `Accept: application/vnd.ipld.{format}` HTTP header with one of supported content types.\n\n## Content-Types\n\nMajority of resources can be retrieved trustlessly by requesting specific content type via `Accept` header or `?format=raw|car|ipns-record` URL query parameter.\n\nSee [trustless gateway specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/)\nand [verifiable retrieval documentation](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval) for more details.\n\n### `application/vnd.ipld.raw`\n\nReturns a byte array for a single `raw` block.\n\nSending such requests for `/ipfs/{cid}` allows for efficient fetch of blocks with data\nencoded in custom format, without the need for deserialization and traversal on the gateway.\n\nThis is equivalent of `ipfs block get`.\n\n### `application/vnd.ipld.car`\n\nReturns a [CAR](https://ipld.io/specs/transport/car/) stream for a DAG or a subset of it.\n\nThe `dag-scope` parameter controls which blocks are included: `all` (default, entire DAG),\n`entity` (logical unit like a file), or `block` (single block). For [UnixFS](https://specs.ipfs.tech/unixfs/) files,\n`entity-bytes` enables byte range requests. See [IPIP-402](https://specs.ipfs.tech/ipips/ipip-0402/)\nfor details.\n\nThis is a rough equivalent of `ipfs dag export`.\n\n### `application/vnd.ipfs.ipns-record`\n\nOnly works on `/ipns/{ipns-name}` content paths that use cryptographically signed [IPNS Records](https://specs.ipfs.tech/ipns/ipns-record/).\n\nReturns [IPNS Record in Protobuf Serialization Format](https://specs.ipfs.tech/ipns/ipns-record/#record-serialization-format)\nwhich can be verified on end client, without trusting gateway.\n" }, { "path": "docs/http-rpc-clients.md", "content": "# HTTP/RPC Clients\n\nKubo provides official HTTP RPC (`/api/v0`) clients for selected languages:\n\n| Language | Package Name | Github Repository |\n|:--------:|:-------------------:|--------------------------------------------|\n| JS | kubo-rpc-client | https://github.com/ipfs/js-kubo-rpc-client |\n| Go | `rpc` | [`../client/rpc`](../client/rpc) |\n\nThere are community-maintained libraries for other languages,\nbut the Kubo team does provide support for them, YMMV:\n\n- https://docs.ipfs.tech/reference/kubo-rpc-cli/\n" }, { "path": "docs/implement-api-bindings.md", "content": "# IPFS API Implementation Doc\n\nThis short document aims to give a quick guide to anyone implementing API\nbindings for IPFS implementations-- in particular kubo.\n\nSections:\n- IPFS Types\n- API Transports\n- API Commands\n- Implementing bindings for the HTTP API\n\n## IPFS Types\n\nIPFS uses a set of value type that is useful to enumerate up front:\n\n- `` is unix-style path, beginning with `/ipfs//...` or\n `/ipns//...` or `/ipns//...`.\n- `` is a base58 encoded [multihash](https://github.com/multiformats/multihash)\n- `cid` is a [multibase](https://github.com/multiformats/multibase) encoded\n [CID](https://github.com/ipld/cid) - a self-describing content-addressing identifier\n\nA note on streams: IPFS is a streaming protocol. Everything about it can be\nstreamed. When importing files, API requests should aim to stream the data in,\nand handle back-pressure correctly, so that the IPFS node can handle it\nsequentially without too much memory pressure. (If using HTTP, this is typically\nhandled for you by writes to the request body blocking.)\n\n## API Transports\n\nLike with everything else, IPFS aims to be flexible regarding the API transports.\nCurrently, the [kubo](https://github.com/ipfs/kubo) implementation supports\nboth an in-process API and an HTTP API. More can be added easily, by mapping the\nAPI functions over a transport. (This is similar to how gRPC is also _mapped on\ntop of transports_, like HTTP).\n\nMapping to a transport involves leveraging the transport's features to express\nfunction calls. For example:\n\n#### CLI API Transport\n\nIn the commandline, IPFS uses a traditional flag and arg-based mapping, where:\n- the first arguments select the command, as in git - e.g. `ipfs dag get`\n- the flags specify options - e.g. `--enc=protobuf -q`\n- the rest are positional arguments - e.g. `ipfs key rename `\n- files are specified by filename, or through stdin\n\n(NOTE: When kubo runs the daemon, the CLI API is converted to HTTP\ncalls. otherwise, they execute in the same process)\n\n#### HTTP API Transport\n\nIn HTTP, our API layering uses a REST-like mapping, where:\n- the URL path selects the command - e.g `/object/get`\n- the URL query string implements option arguments - e.g. `&enc=protobuf&q=true`\n- the URL query also implements positional arguments - e.g.\n `&arg=&arg=add-link&arg=foo&arg=`\n- the request body streams file data - reads files or stdin\n - multiple streams are muxed with multipart (todo: add tar stream support)\n\n\n## API Commands\n\nThere is a \"standard IPFS API\" which is currently defined as \"all the commands\nexposed by the kubo implementation\". There are auto-generated [API Docs](https://ipfs.io/docs/api/).\nYou can Also see [a listing here](https://github.com/ipfs/kubo/blob/94b832df861728c65e912935641d08880c341e0a/core/commands/root.go#L96-L130), or get a list of\ncommands by running `ipfs commands` locally.\n\n## Implementing bindings for the HTTP API\n\nAs mentioned above, the API commands map to HTTP with:\n- the URL path selects the command - e.g `/object/get`\n- the URL query string implements option arguments - e.g. `&enc=protobuf&q=true`\n- the URL query also implements positional arguments - e.g.\n `&arg=&arg=add-link&arg=foo&arg=`\n- the request body streams file data - reads files or stdin\n - multiple streams are muxed with multipart (todo: add tar stream support)\n\nYou can see the latest [list of our HTTP RPC clients here](http-rpc-clients.md)\n\nThe Go implementation is good to answer harder questions, like how is multipart\nhandled, or what headers should be set in edge conditions. But the javascript\nimplementation is very concise, and easy to follow.\n\n## Note on multipart + inspecting requests\n\nDespite all the generalization spoken about above, the IPFS API is actually very\nsimple. You can inspect all the requests made with `nc` and the `--api` option\n(as of [this PR](https://github.com/ipfs/kubo/pull/1598), or `0.3.8`):\n\n```sh\n> nc -l 5002 &\n> ipfs --api /ip4/127.0.0.1/tcp/5002 swarm addrs local --enc=json\nPOST /api/v0/version?enc=json&stream-channels=true HTTP/1.1\nHost: 127.0.0.1:5002\nUser-Agent: /kubo/0.14.0/\nContent-Length: 0\nContent-Type: application/octet-stream\nAccept-Encoding: gzip\n\n\n```\n\nThe only hard part is getting the file streaming right. It is (now) fairly easy\nto stream files to kubo using multipart. Basically, we end up with HTTP\nrequests like this:\n\n```sh\n> nc -l 5002 &\n> ipfs --api /ip4/127.0.0.1/tcp/5002 add -r ~/demo/basic/test\nPOST /api/v0/add?encoding=json&progress=true&r=true&stream-channels=true HTTP/1.1\nHost: 127.0.0.1:5002\nUser-Agent: /kubo/0.14.0/\nTransfer-Encoding: chunked\nContent-Disposition: form-data: name=\"files\"\nContent-Type: multipart/form-data; boundary=2186ef15d8f2c4f100af72d6d345afe36a4d17ef11264ec5b8ec4436447f\nAccept-Encoding: gzip\n\n1\n-\ne5\n-2186ef15d8f2c4f100af72d6d345afe36a4d17ef11264ec5b8ec4436447f\nContent-Disposition: form-data; name=\"file\"; filename=\"test\"\nContent-Type: multipart/mixed; boundary=acdb172fe12f25e8ffae9981ce6f4580abdefb0cae3ceebe464d802866be\n\n\n9c\n--acdb172fe12f25e8ffae9981ce6f4580abdefb0cae3ceebe464d802866be\nContent-Disposition: file; filename=\"test%2Fbar\"\nContent-Type: application/octet-stream\n\n\n4\nbar\n\ndc\n\n--acdb172fe12f25e8ffae9981ce6f4580abdefb0cae3ceebe464d802866be\nContent-Disposition: file; filename=\"test%2Fbaz\"\nContent-Type: multipart/mixed; boundary=2799ac77a72ef7b8a0281945806b9f9a28f7681145aa8e91b052d599b2dd\n\n\na0\n--2799ac77a72ef7b8a0281945806b9f9a28f7681145aa8e91b052d599b2dd\nContent-Type: application/octet-stream\nContent-Disposition: file; filename=\"test%2Fbaz%2Fb\"\n\n\n4\nbar\n\na2\n\n--2799ac77a72ef7b8a0281945806b9f9a28f7681145aa8e91b052d599b2dd\nContent-Disposition: file; filename=\"test%2Fbaz%2Ff\"\nContent-Type: application/octet-stream\n\n\n4\nfoo\n\n44\n\n--2799ac77a72ef7b8a0281945806b9f9a28f7681145aa8e91b052d599b2dd--\n\n9e\n\n--acdb172fe12f25e8ffae9981ce6f4580abdefb0cae3ceebe464d802866be\nContent-Disposition: file; filename=\"test%2Ffoo\"\nContent-Type: application/octet-stream\n\n\n4\nfoo\n\n44\n\n--acdb172fe12f25e8ffae9981ce6f4580abdefb0cae3ceebe464d802866be--\n\n44\n\n--2186ef15d8f2c4f100af72d6d345afe36a4d17ef11264ec5b8ec4436447f--\n\n0\n\n```\n\nWhich produces: http://gateway.ipfs.io/ipfs/QmNtpA5TBNqHrKf3cLQ1AiUKXiE4JmUodbG5gXrajg8wdv\n\n" }, { "path": "docs/libp2p-resource-management.md", "content": "\n# libp2p Network Resource Manager (`Swarm.ResourceMgr`)\n\n## Purpose\nThe purpose of this document is to provide more information about the [libp2p Network Resource Manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#readme) and how it's integrated into Kubo so that Kubo users can understand and configure it appropriately.\n\n## 🙋 Help! The resource manager is protecting my node but I want to understand more\nThe resource manager is generally a *feature* to bound libp2p's resources, whether from bugs, unintentionally misbehaving peers, or intentional Denial of Service attacks.\n\nGood places to start are:\n1. Understand [how the resource manager is configured](#levels-of-configuration).\n2. Understand [how to read the log message](#what-do-these-protected-from-exceeding-resource-limits-log-messages-mean)\n3. Understand [how to inspect and change limits](#user-supplied-override-limits)\n\n## Table of Contents\n\n- [Purpose](#purpose)\n- [🙋 Help! The resource manager is protecting my node but I want to understand more](#-help--the-resource-manager-is-protecting-my-node-but-i-want-to-understand-more)\n- [Table of Contents](#table-of-contents)\n- [Levels of Configuration](#levels-of-configuration)\n - [Approach](#approach)\n - [Computed Default Limits](#computed-default-limits)\n - [User Supplied Override Limits](#user-supplied-override-limits)\n- [FAQ](#faq)\n - [What do these \"Protected from exceeding resource limits\" log messages mean?](#what-do-these-protected-from-exceeding-resource-limits-log-messages-mean)\n - [How does one see the Active Limits?](#how-does-one-see-the-active-limits)\n - [How does one see the Computed Default Limits?](#how-does-one-see-the-computed-default-limits)\n - [How does one monitor libp2p resource usage?](#how-does-one-monitor-libp2p-resource-usage)\n - [How does the resource manager (ResourceMgr) relate to the connection manager (ConnMgr)?](#how-does-the-resource-manager-resourcemgr-relate-to-the-connection-manager-connmgr)\n - [What are the \"Application error 0x0 (remote) ... cannot reserve ...\" messages?](#what-are-the-application-error-0x0-remote--cannot-reserve--messages)\n- [History](#history)\n\n## Levels of Configuration\n\nSee also the [`Swarm.ResourceMgr` config docs](./config.md#swarmresourcemgr).\n\n### Approach\nlibp2p's resource manager provides tremendous flexibility but also adds complexity. There are these levels of limit configuration for resource management protection:\n\n1. \"The user who does nothing\" - In this case Kubo attempts to give some sane defaults discussed below\n based on the amount of memory and file descriptors their system has.\n This should protect the node from many attacks.\n\n2. \"Slightly more advanced user\" - Where the defaults aren't good enough, a good set of higher-level \"knobs\" are exposed to satisfy most use cases\n without requiring users to wade into all the intricacies of libp2p's resource manager.\n The \"knobs\"/inputs are `Swarm.ResourceMgr.MaxMemory` and `Swarm.ResourceMgr.MaxFileDescriptors` as described below.\n\n3. \"Power user\" - They [specify override limits](#user-supplied-override-limits) and own their own destiny without Kubo getting in the way.\n\n### Computed Default Limits\nWith the `Swarm.ResourceMgr.MaxMemory` and `Swarm.ResourceMgr.MaxFileDescriptors` inputs defined,\n[resource manager limits](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#limits) are created at the\n[system](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#the-system-scope),\n[transient](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#the-transient-scope),\nand [peer](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#peer-scopes) scopes.\nOther scopes are ignored (by being set to \"unlimited\").\n\nThe reason these scopes are chosen is because:\n- `system` - This gives us the coarse-grained control we want so we can reason about the system as a whole.\n It is the backstop, and allows us to reason about resource consumption more easily\n since don't have think about the interaction of many other scopes.\n- `transient` - Limiting connections that are in process of being established provides backpressure so not too much work queues up.\n- `peer` - The peer scope doesn't protect us against intentional DoS attacks.\n It's just as easy for an attacker to send 100 requests/second with 1 peerId vs. 10 requests/second with 10 peers.\n We are reliant on the system scope for protection here in the malicious case.\n The reason for having a peer scope is to protect against unintentional DoS attacks\n (e.g., bug in a peer which is causing it to \"misbehave\").\n In the unintentional case, we want to make sure a \"misbehaving\" node doesn't consume more resources than necessary.\n\nWithin these scopes, limits are set on:\n1. [memory](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#memory)\n2. [file descriptors (FD)](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#file-descriptors)\n3. [*inbound* connections](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#connections).\nLimits are set based on the `Swarm.ResourceMgr.MaxMemory` and `Swarm.ResourceMgr.MaxFileDescriptors` inputs above.\n\nThere are also some special cases where minimum values are enforced.\nFor example, Kubo maintainers have found in practice that it's a footgun to have too low of a value for `System.ConnsInbound` and a default minimum is used. (See [core/node/libp2p/rcmgr_defaults.go](https://github.com/ipfs/kubo/blob/master/core/node/libp2p/rcmgr_defaults.go) for specifics.)\n\nWe trust this node to behave properly and thus don't limit *outbound* connection/stream limits.\nWe apply any limits that libp2p has for its protocols/services\nsince we assume libp2p knows best here.\n\nSource: [core/node/libp2p/rcmgr_defaults.go](https://github.com/ipfs/kubo/blob/master/core/node/libp2p/rcmgr_defaults.go)\n\n### User Supplied Override Limits\nA user who wants fine control over the limits used by the go-libp2p resource manager can specify overrides to the [computed default limits](#computed-default-limits).\nThis is done by defining limits in ``$IPFS_PATH/libp2p-resource-limit-overrides.json``.\nThese values trump anything else and are parsed directly by go-libp2p.\n(See the [go-libp2p Resource Manager README](https://github.com/libp2p/go-libp2p/blob/master/p2p/host/resource-manager/README.md) for formatting.) \n\n## FAQ\n\n### What do these \"Protected from exceeding resource limits\" log messages mean?\n\"Protected from exceeding resource limits\" log messages denote that the resource manager is working and that it prevented additional resources from being used beyond the set limits. Per [libp2p code](https://github.com/libp2p/go-libp2p/blob/master/p2p/host/resource-manager/scope.go), these messages take the form of \"$scope: cannot reserve $limitKey\". \n\nAs an example:\n\n> Protected from exceeding resource limits 2 times: \"system: cannot reserve inbound connection: resource limit exceeded\"\n\nThis means that there were 2 recent occurrences where the libp2p resource manager prevented an inbound connection at the \"system\" [scope](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#resource-scopes). \nSpecifically the ``System.ConnsInbound`` limit was hit. \n\nThis can be analyzed by viewing the limit and current usage with `ipfs swarm resources`.\n`System.ConnsInbound` is likely close or at the limit value.\n\nThe simplest way to identify all resources across all scopes that are close to exceeding their limit (>90% usage) is with a command like `ipfs swarm resources | egrep \"9.\\..%\"` \n\nSources:\n* [kubo resource manager logging](https://github.com/ipfs/kubo/blob/master/core/node/libp2p/rcmgr_logging.go)\n* [libp2p resource manager messages](https://github.com/libp2p/go-libp2p/blob/master/p2p/host/resource-manager/scope.go)\n\n### How does one see the Active Limits?\nA dump of what limits are actually being used by the resource manager ([Computed Default Limits](#computed-default-limits) + [User Supplied Override Limits](#user-supplied-override-limits))\ncan be obtained by `ipfs swarm resources`.\n\n### How does one see the Computed Default Limits?\nThis can be observed [seeing the active limits](#how-does-one-see-the-active-limits) assuming one hasn't detoured into \"power user\" mode with [User Supplied Override Limits](#user-supplied-override-limits).\n\n### How does one monitor libp2p resource usage?\n\nFor [monitoring libp2p resource usage](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#monitoring), \nvarious `*rcmgr_*` metrics can be accessed as the Prometheus endpoint at `{Addresses.API}/debug/metrics/prometheus` (default: `http://127.0.0.1:5001/debug/metrics/prometheus`). \nThere are also [pre-built Grafana dashboards](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager/obs/grafana-dashboards) that can be added to a Grafana instance. \n\nA textual view of current resource usage and a list of services, protocols, and peers can be\nobtained via `ipfs swarm stats --help`\n\n### How does the resource manager (ResourceMgr) relate to the connection manager (ConnMgr)?\nAs discussed [here](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#connmanager-vs-resource-manager)\nthese are separate systems in go-libp2p.\nKubo performs sanity checks to ensure that some of the hard limits of the ResourceMgr are sufficiently greater than the soft limits of the ConnMgr.\n\nThe soft limit of `Swarm.ConnMgr.HighWater` needs to be less than the resource manager hard limit `System.ConnsInbound` for the configuration to make sense.\nThis ensures the ConnMgr cleans up connections based on connection priorities before the hard limits of the ResourceMgr are applied.\nIf `Swarm.ConnMgr.HighWater` is greater than resource manager's `System.ConnsInbound`,\nexisting low-priority idle connections can prevent new high-priority connections from being established.\nThe ResourceMgr doesn't know that the new connection is high priority and simply blocks it because of the limit its enforcing.\n\nTo ensure the ConnMgr and ResourceMgr are congruent, the ResourceMgr [computed default limits](#computed-default-limits) are adjusted such that:\n1. `System.ConnsInbound` >= `max(Swarm.ConnMgr.HighWater * 2, DefaultResourceMgrMinInboundConns)` AND\n2. `System.StreamsInbound` is greater than any new/adjusted `Swarm.ResourceMgr.Limits.System.ConnsInbound` value so that there's enough streams per connection.\n\nSource: [core/node/libp2p/rcmgr_defaults.go](https://github.com/ipfs/kubo/blob/master/core/node/libp2p/rcmgr_defaults.go)\n\n### What are the \"Application error 0x0 (remote) ... cannot reserve ...\" messages?\nThese are messages coming from old (pre go-libp2p 0.26) *remote* go-libp2p peers (likely another older Kubo node) with the resource manager enabled on why it failed to establish a connection. \n\nThis can be confusing, but these `Application error 0x0 (remote) ... cannot reserve ...` messages can occur even if your local node has the resource manager disabled.\n\nYou can distinguish resource manager messages originating from your local node if they're from the `resourcemanager` / `libp2p/rcmgr_logging.go` logger\nor you see the string that is unique to Kubo (and not in go-libp2p): \"Protected from exceeding resource limits\".\n\nSee more info in this go-libp2p issue ([#1928](https://github.com/libp2p/go-libp2p/issues/1928)). go-libp2p 0.26 / Kubo 0.19 onwards this confusing error message was removed.\n\n\n## History\nKubo first [exposed this functionality in Kubo 0.13](./changelogs/v0.13.md#-libp2p-network-resource-manager-swarmresourcemgr), but it was disabled by default. It was then enabled by default in [Kubo 0.17](./changelogs/v0.17.md#libp2p-resource-management-enabled-by-default). Until that point, Kubo was vulnerable to unbound resource usage which could bring down nodes. Introducing limits like this by default after the fact is tricky, which is why there have been changes and improvements afterwards. The general trend since 0.17 with (0.18)[./changeloges/v0.18.md#improving-libp2p-resource-management-integration] and 0.19 has been to simplify and provide less options (and footguns!) for users and better documentation.\n" }, { "path": "docs/metrics.md", "content": "## Kubo metrics\n\nBy default, a Prometheus endpoint is exposed by Kubo at `http://127.0.0.1:5001/debug/metrics/prometheus`.\n\nIt includes default [Prometheus Go client metrics](https://prometheus.io/docs/guides/go-application/) + Kubo-specific metrics listed below.\n\n### Table of Contents\n\n- [DHT RPC](#dht-rpc)\n - [Inbound RPC metrics](#inbound-rpc-metrics)\n - [Outbound RPC metrics](#outbound-rpc-metrics)\n- [Provide](#provide)\n - [Legacy Provider](#legacy-provider)\n - [DHT Provider](#dht-provider)\n- [Gateway (`boxo/gateway`)](#gateway-boxogateway)\n - [HTTP metrics](#http-metrics)\n - [Blockstore cache metrics](#blockstore-cache-metrics)\n - [Backend metrics](#backend-metrics)\n- [Generic HTTP Servers](#generic-http-servers)\n - [Core HTTP metrics](#core-http-metrics-ipfs_http_)\n - [HTTP Server metrics](#http-server-metrics-http_server_)\n- [OpenTelemetry Metadata](#opentelemetry-metadata)\n\n> [!WARNING]\n> This documentation is incomplete. For an up-to-date list of metrics available at daemon startup, see [test/sharness/t0119-prometheus-data/prometheus_metrics_added_by_measure_profile](https://github.com/ipfs/kubo/blob/master/test/sharness/t0119-prometheus-data/prometheus_metrics_added_by_measure_profile).\n>\n> Additional metrics may appear during runtime as some components (like boxo/gateway) register metrics only after their first event occurs (e.g., HTTP request/response).\n\n## DHT RPC\n\nMetrics from `go-libp2p-kad-dht` for DHT RPC operations:\n\n### Inbound RPC metrics\n\n- `rpc_inbound_messages_total` - Counter: total messages received per RPC\n- `rpc_inbound_message_errors_total` - Counter: total errors for received messages\n- `rpc_inbound_bytes_[bucket|sum|count]` - Histogram: distribution of received bytes per RPC\n- `rpc_inbound_request_latency_[bucket|sum|count]` - Histogram: latency distribution for inbound RPCs\n\n### Outbound RPC metrics\n\n- `rpc_outbound_messages_total` - Counter: total messages sent per RPC\n- `rpc_outbound_message_errors_total` - Counter: total errors for sent messages\n- `rpc_outbound_requests_total` - Counter: total requests sent\n- `rpc_outbound_request_errors_total` - Counter: total errors for sent requests\n- `rpc_outbound_bytes_[bucket|sum|count]` - Histogram: distribution of sent bytes per RPC\n- `rpc_outbound_request_latency_[bucket|sum|count]` - Histogram: latency distribution for outbound RPCs\n\n## Provide\n\n### Legacy Provider\n\nMetrics for the legacy provider system when `Provide.DHT.SweepEnabled=false`:\n\n- `provider_reprovider_provide_count` - Counter: total successful provide operations since node startup\n- `provider_reprovider_reprovide_count` - Counter: total reprovide sweep operations since node startup\n\n### DHT Provider\n\nMetrics for the DHT provider system when `Provide.DHT.SweepEnabled=true`:\n\n- `provider_provides_total` - Counter: total successful provide operations since node startup (includes both one-time provides and periodic provides done on `Provide.DHT.Interval`)\n\n> [!NOTE]\n> These metrics are exposed by [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht/). You can enable debug logging for DHT provider activity with `GOLOG_LOG_LEVEL=dht/provider=debug`.\n\n## Gateway (`boxo/gateway`)\n\n> [!TIP]\n> These metrics are limited to [IPFS Gateway](https://specs.ipfs.tech/http-gateways/) endpoints. For general HTTP metrics across all endpoints, consider using a reverse proxy.\n\nGateway metrics appear after the first HTTP request is processed:\n\n### HTTP metrics\n\n- `ipfs_http_gw_responses_total{code}` - Counter: total HTTP responses by status code\n- `ipfs_http_gw_retrieval_timeouts_total{code,truncated}` - Counter: requests that timed out during content retrieval\n- `ipfs_http_gw_concurrent_requests` - Gauge: number of requests currently being processed\n\n### Blockstore cache metrics\n\n- `ipfs_http_blockstore_cache_hit` - Counter: global block cache hits\n- `ipfs_http_blockstore_cache_requests` - Counter: global block cache requests\n\n### Backend metrics\n\n- `ipfs_gw_backend_api_call_duration_seconds_[bucket|sum|count]{backend_method}` - Histogram: time spent in IPFSBackend API calls\n\n## Generic HTTP Servers\n\n> [!TIP]\n> The metrics below are not very useful and exist mostly for historical reasons. If you need non-gateway HTTP metrics, it's better to put a reverse proxy in front of Kubo and use its metrics.\n\n### Core HTTP metrics (`ipfs_http_*`)\n\nPrometheus metrics for the HTTP API exposed at port 5001:\n\n- `ipfs_http_requests_total{method,code,handler}` - Counter: total HTTP requests (Legacy - new metrics are provided by boxo/gateway for gateway traffic)\n- `ipfs_http_request_duration_seconds[_sum|_count]{handler}` - Summary: request processing duration\n- `ipfs_http_request_size_bytes[_sum|_count]{handler}` - Summary: request body sizes\n- `ipfs_http_response_size_bytes[_sum|_count]{handler}` - Summary: response body sizes\n\n### HTTP Server metrics (`http_server_*`)\n\nAdditional HTTP instrumentation for all handlers (Gateway, API commands, etc.):\n\n- `http_server_request_body_size_bytes_[bucket|count|sum]` - Histogram: distribution of request body sizes\n- `http_server_request_duration_seconds_[bucket|count|sum]` - Histogram: distribution of request processing times\n- `http_server_response_body_size_bytes_[bucket|count|sum]` - Histogram: distribution of response body sizes\n\nThese metrics are automatically added to Gateway handlers, Hostname Gateway, Libp2p Gateway, and API command handlers.\n\n> [!NOTE]\n> The `server_address` label from `otelhttp` is dropped via an OTel SDK View to prevent cardinality explosion on subdomain gateways (where each unique `Host` header creates a new time series). All handlers include a `server_domain` label instead:\n>\n> - Gateway and Hostname Gateway handlers group requests by their matching [`Gateway.PublicGateways`](config.md#gatewaypublicgateways) domain suffix (e.g., `dweb.link`, `ipfs.io`). Unmatched hosts are labeled `localhost`, `loopback`, or `other`.\n> - The RPC API handler uses `api`.\n> - The Libp2p Gateway handler uses `libp2p`.\n\n## OpenTelemetry Metadata\n\nKubo uses Prometheus for metrics collection for historical reasons, but OpenTelemetry metrics are automatically exposed through the same Prometheus endpoint. These metadata metrics provide context about the instrumentation:\n\n- `otel_scope_info` - Information about instrumentation libraries producing metrics\n- `target_info` - Service metadata including version and instance information" }, { "path": "docs/p2p-tunnels.md", "content": "# P2P Tunnels\n\nKubo supports tunneling TCP connections through libp2p streams, similar to SSH\nport forwarding (`ssh -L`). This allows exposing local services to remote peers\nand forwarding remote services to local ports.\n\n- [Why P2P Tunnels?](#why-p2p-tunnels)\n- [Quick Start](#quick-start)\n- [Background Mode](#background-mode)\n- [Foreground Mode](#foreground-mode)\n - [systemd Integration](#systemd-integration)\n- [Security Considerations](#security-considerations)\n- [Troubleshooting](#troubleshooting)\n\n## Why P2P Tunnels?\n\nUnlike traditional SSH tunnels, libp2p-based tunnels do not require:\n\n- **No public IP or open ports**: The server does not need a static IP address\n or port forwarding configured on the router. Connectivity to peers behind NAT\n is facilitated by [Direct Connection Upgrade through Relay (DCUtR)](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md),\n which enables NAT hole-punching.\n\n- **No DNS or IP address management**: All you need is the server's PeerID and\n an agreed-upon protocol name (e.g., `/x/ssh`). Kubo handles peer discovery\n and routing via the [Amino DHT](https://specs.ipfs.tech/routing/kad-dht/).\n\n- **Simplified firewall rules**: Since connections are established through\n libp2p's existing swarm connections, no additional firewall configuration is\n needed beyond what Kubo already requires.\n\nThis makes p2p tunnels useful for connecting to machines on home networks,\nbehind corporate firewalls, or in environments where traditional port forwarding\nis not available.\n\n## Quick Start\n\nEnable the experimental feature:\n\n```console\n$ ipfs config --json Experimental.Libp2pStreamMounting true\n```\n\nTest with netcat (`nc`) - no services required:\n\n**On the server:**\n\n```console\n$ ipfs p2p listen /x/test /ip4/127.0.0.1/tcp/9999\n$ nc -l -p 9999\n```\n\n**On the client:**\n\nReplace `$SERVER_ID` with the server's peer ID (get it with `ipfs id -f \"\\n\"`\non the server).\n\n```console\n$ ipfs p2p forward /x/test /ip4/127.0.0.1/tcp/9998 /p2p/$SERVER_ID\n$ nc 127.0.0.1 9998\n```\n\nType in either terminal and the text appears in the other. Use Ctrl+C to exit.\n\n## Background Mode\n\nBy default, `ipfs p2p listen` and `ipfs p2p forward` register the tunnel with\nthe daemon and return immediately. The tunnel persists until explicitly closed\nwith `ipfs p2p close` or the daemon shuts down.\n\nThis example exposes a local SSH server (listening on `localhost:22`) to a\nremote peer. The same pattern works for any TCP service.\n\n**On the server** (the machine running SSH):\n\nRegister a p2p listener that forwards incoming connections to the local SSH\nserver. The protocol name `/x/ssh` is an arbitrary identifier that both peers\nmust agree on (the `/x/` prefix is required for custom protocols).\n\n```console\n$ ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22\n```\n\n**On the client:**\n\nCreate a local port (`2222`) that tunnels through libp2p to the server's SSH\nservice.\n\n```console\n$ ipfs p2p forward /x/ssh /ip4/127.0.0.1/tcp/2222 /p2p/$SERVER_ID\n```\n\nNow connect to SSH through the tunnel:\n\n```console\n$ ssh user@127.0.0.1 -p 2222\n```\n\n**Other services:** To tunnel a different service, change the port and protocol\nname. For example, to expose a web server on port 8080, use `/x/mywebapp` and\n`/ip4/127.0.0.1/tcp/8080`.\n\n## Foreground Mode\n\nUse `--foreground` (`-f`) to block until interrupted. The tunnel is\nautomatically removed when the command exits:\n\n```console\n$ ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 --foreground\nListening on /x/ssh, forwarding to /ip4/127.0.0.1/tcp/22, waiting for interrupt...\n^C\nReceived interrupt, removing listener for /x/ssh\n```\n\nThe listener/forwarder is automatically removed when:\n\n- The command receives Ctrl+C or SIGTERM\n- `ipfs p2p close` is called\n- The daemon shuts down\n\nThis mode is useful for systemd services and scripts that need cleanup on exit.\n\n### systemd Integration\n\nThe `--foreground` flag enables clean integration with systemd. The examples\nbelow show how to run `ipfs p2p listen` as a user service that starts\nautomatically when the IPFS daemon is ready.\n\nEnsure IPFS daemon runs as a systemd user service. See\n[misc/README.md](https://github.com/ipfs/kubo/blob/master/misc/README.md#systemd)\nfor setup instructions and where to place unit files.\n\n#### P2P listener with path-based activation\n\nUse a `.path` unit to wait for the daemon's RPC API to be ready before starting\nthe p2p listener.\n\n**`ipfs-p2p-tunnel.path`**:\n\n```systemd\n[Unit]\nDescription=Monitor for IPFS daemon startup\nAfter=ipfs.service\nRequires=ipfs.service\n\n[Path]\nPathExists=%h/.ipfs/api\nUnit=ipfs-p2p-tunnel.service\n\n[Install]\nWantedBy=default.target\n```\n\nThe `%h` specifier expands to the user's home directory. If you use a custom\n`IPFS_PATH`, adjust accordingly.\n\n**`ipfs-p2p-tunnel.service`**:\n\n```systemd\n[Unit]\nDescription=IPFS p2p tunnel\nRequires=ipfs.service\n\n[Service]\nExecStart=ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 -f\nRestart=on-failure\nRestartSec=10\n\n[Install]\nWantedBy=default.target\n```\n\n#### Enabling the services\n\n```console\n$ systemctl --user enable ipfs.service\n$ systemctl --user enable ipfs-p2p-tunnel.path\n$ systemctl --user start ipfs.service\n```\n\nThe path unit monitors `~/.ipfs/api` and starts `ipfs-p2p-tunnel.service`\nonce the file exists.\n\n## Security Considerations\n\n> [!WARNING]\n> This feature provides CLI and HTTP RPC users with the ability to set up port\n> forwarding for localhost and LAN ports. If you enable this and plan to expose\n> CLI or HTTP RPC to other users or machines, secure the RPC API using\n> [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations)\n> or custom auth middleware.\n\n## Troubleshooting\n\n### Foreground listener stops when terminal closes\n\nWhen using `--foreground`, the listener stops if the terminal closes. For\npersistent foreground listeners, use a systemd service, `nohup`, `tmux`, or\n`screen`. Without `--foreground`, the listener persists in the daemon regardless\nof terminal state.\n\n### Connection refused errors\n\nVerify:\n\n1. The experimental feature is enabled: `ipfs config Experimental.Libp2pStreamMounting`\n2. The listener is active: `ipfs p2p ls`\n3. Both peers can connect: `ipfs swarm connect /p2p/$PEER_ID`\n\n### Persistent tunnel configuration\n\nThere is currently no way to define tunnels in the Kubo JSON config file. Use\n`--foreground` mode with a systemd service for persistent tunnels. Support for\nconfiguring tunnels via JSON config may be added in the future (see [kubo#5460](https://github.com/ipfs/kubo/issues/5460) - PRs welcome!).\n" }, { "path": "docs/plugins.md", "content": "# Plugins\n\nSince 0.4.11 Kubo has an experimental plugin system that allows augmenting\nthe daemons functionality without recompiling.\n\nWhen an IPFS node is started, it will load plugins from the `$IPFS_PATH/plugins`\ndirectory (by default `~/.ipfs/plugins`).\n\n**Table of Contents**\n\n- [Plugin Types](#plugin-types)\n - [IPLD](#ipld)\n - [Datastore](#datastore)\n- [Available Plugins](#available-plugins)\n- [Installing Plugins](#installing-plugins)\n - [External Plugin](#external-plugin)\n - [In-tree](#in-tree)\n - [Out-of-tree](#out-of-tree)\n - [Preloaded Plugins](#preloaded-plugins)\n- [Creating A Plugin](#creating-a-plugin)\n\n## Plugin Types\n\nPlugins can implement one or more plugin types, defined in the\n[plugin](https://godoc.org/github.com/ipfs/kubo/plugin) package.\n\n### IPLD\n\nIPLD plugins add support for additional formats to `ipfs dag` and other IPLD\nrelated commands.\n\n### Datastore\n\nDatastore plugins add support for additional datastore backends.\n\n### Tracer\n\n(experimental)\n\nTracer plugins allow injecting an opentracing backend into Kubo.\n\n### Daemon\n\nDaemon plugins are started when the Kubo daemon is started and are given an\ninstance of the CoreAPI. This should make it possible to build an ipfs-based\napplication without IPC and without forking Kubo.\n\nNote: We eventually plan to make Kubo usable as a library. However, this\nplugin type is likely the best interim solution.\n\n### fx (experimental)\n\nFx plugins let you customize the [fx](https://pkg.go.dev/go.uber.org/fx) dependency graph and configuration,\nby customizing the`fx.Option`s that are passed to `fx` when the Kubo node is initialized.\n\nFor example, you can override an interface such as [exchange.Interface](https://github.com/ipfs/go-ipfs-exchange-interface)\nor [pin.Pinner](https://github.com/ipfs/go-ipfs-pinner) with a custom implementation by appending an option like\n`fx.Decorate(func() exchange.Interface { return customExchange })`.\n\nFx supports some advanced customization. Simple interface replacements like above are unlikely to break in the future, \nbut the more invasive your changes, the more likely they are to break between releases. Kubo cannot guarantee backwards\ncompatibility for `fx` customizations.\n\nFx options are applied across every execution of the `ipfs` binary, including:\n\n- Repo initialization\n- Daemon\n- Applying migrations\n- etc.\n\nSo if you plug in a blockservice that disallows non-allowlisted CIDs, then this may break migrations\nthat fetch migration code over the IPFS network.\n\n### Internal\n\n(never stable)\n\nInternal plugins are like daemon plugins _except_ that they can access, replace,\nand modify all internal state. Use this plugin type to extend Kubo in\narbitrary ways. However, be aware that your plugin will likely break every time\nKubo updated.\n\n## Configuration\n\nPlugins can be configured in the `Plugins` section of the config file. Here,\nplugins can be:\n\n1. Passed an arbitrary config object via the `Config` field.\n2. Disabled via the `Disabled` field.\n\nExample:\n\n```js\n{\n // ...\n \"Plugins\": {\n \"Plugins\": {\n // plugin named \"plugin-foo\"\n \"plugin-foo\": {\n \"Config\": { /* arbitrary json */ }\n },\n // plugin named \"plugin-bar\"\n \"plugin-bar\": {\n \"Disabled\": true // \"plugin-bar\" will not be loaded\n }\n }\n }\n}\n```\n\n## Available Plugins\n\n| Name | Type | Preloaded | Description |\n|---------------------------------------------------------------------------------|-----------|-----------|------------------------------------------------|\n| [git](https://github.com/ipfs/kubo/tree/master/plugin/plugins/git) | IPLD | x | An IPLD format for git objects. |\n| [badgerds](https://github.com/ipfs/kubo/tree/master/plugin/plugins/badgerds) | Datastore | x | A high performance but experimental datastore. |\n| [flatfs](https://github.com/ipfs/kubo/tree/master/plugin/plugins/flatfs) | Datastore | x | A stable filesystem-based datastore. |\n| [levelds](https://github.com/ipfs/kubo/tree/master/plugin/plugins/levelds) | Datastore | x | A stable, flexible datastore backend. |\n| [jaeger](https://github.com/ipfs/go-jaeger-plugin) | Tracing | | An opentracing backend. |\n| [telemetry](https://github.com/ipfs/kubo/tree/master/plugin/plugins/telemetry) | Telemetry | x | Collects anonymized usage data for Kubo development. |\n\n* **Preloaded** plugins are built into the Kubo binary and do not need to be\n installed separately. At the moment, all in-tree plugins are preloaded.\n\n## Installing Plugins\n\nKubo supports two types of plugins: External and Preloaded.\n\n* External plugins must be installed in `$IPFS_PATH/plugins/` (usually\n`~/.ipfs/plugins/`).\n* Preloaded plugins are built-into the Kubo when it's compiled.\n\n### External Plugin\n\nThe advantage of an external plugin is that it can be built, packaged, and\ninstalled independently of Kubo. Unfortunately, this method is only supported\non Linux and MacOS at the moment. Users of other operating systems should follow\nthe instructions for preloaded plugins.\n\n#### In-tree\n\nTo build plugins included in\n[plugin/plugins](https://github.com/ipfs/kubo/tree/master/plugin/plugins),\nrun:\n\n```bash\nkubo$ make build_plugins\nkubo$ ls plugin/plugins/*.so\n```\n\nTo install, copy desired plugins to `$IPFS_PATH/plugins`. For example:\n\n```bash\nkubo$ mkdir -p ~/.ipfs/plugins/\nkubo$ cp plugin/plugins/git.so ~/.ipfs/plugins/\nkubo$ chmod +x ~/.ipfs/plugins/git.so # ensure plugin is executable\n```\n\nFinally, restart daemon if it is running.\n\n#### Out-of-tree\n\nTo build out-of-tree plugins, use the plugin's Makefile if provided. Otherwise,\nyou can manually build the plugin by running:\n\n```bash\nmyplugin$ go build -buildmode=plugin -o myplugin.so myplugin.go\n```\n\nFinally, as with in-tree plugins:\n\n1. Install the plugin in `$IPFS_PATH/plugins`.\n2. Mark the plugin as executable (`chmod +x $IPFS_PATH/plugins/myplugin.so`).\n3. Restart your IPFS daemon (if running).\n\n### Preloaded Plugins\n\nThe advantages of preloaded plugins are:\n\n1. They're bundled with the Kubo binary.\n2. They work on all platforms.\n\nTo preload a Kubo plugin:\n\n1. Add the plugin to the preload list: `plugin/loader/preload_list`\n2. Build ipfs\n```bash\nkubo$ make build\n```\n\nYou can also preload an in-tree but disabled-by-default plugin by adding it to\nthe IPFS_PLUGINS variable. For example, to enable plugins foo, bar, and baz:\n\n```bash\nkubo$ make build IPFS_PLUGINS=\"foo bar baz\"\n```\n\n## Creating A Plugin\n\nTo create your own out-of-tree plugin, use the [example\nplugin](https://github.com/ipfs/go-ipfs-example-plugin/) as a starting point.\nWhen you're ready, submit a PR adding it to the list of [available\nplugins](#available-plugins).\n" }, { "path": "docs/production/reverse-proxy.md", "content": "# IPFS & Reverse HTTP Proxies\n\nWhen run in production environments, go-ipfs should generally be run behind a\nreverse HTTP proxy (usually NGINX). You may need a reverse proxy to:\n\n* Load balance requests across multiple go-ipfs daemons.\n* Cache responses.\n* Buffer requests, only releasing them to go-ipfs when complete. This can help\n protect go-ipfs from the\n [slowloris](https://en.wikipedia.org/wiki/Slowloris_(computer_security)\n attack.\n* Block content.\n* Rate limit and timeout requests.\n* Apply QoS rules (e.g., prioritize traffic for certain important IPFS resources).\n\nThis document contains a collection of tips, tricks, and pitfalls when running a\ngo-ipfs node behind a reverse HTTP proxy.\n\n**WARNING:** Due to\n[nginx#1293](https://trac.nginx.org/nginx/ticket/1293)/[go-ipfs#6402](https://github.com/ipfs/go-ipfs/issues/6402),\nparts of the go-ipfs API will not work correctly behind an NGINX reverse proxy\nas go-ipfs starts sending back a response before it finishes reading the request\nbody. The gateway itself is unaffected.\n\n## Peering\n\nGo-ipfs gateways behind a single load balancing reverse proxy should use the\n[peering](../config.md#peering) subsystem to peer with each other. That way, as\nlong as one go-ipfs daemon has the content being requested, the others will be\nable to serve it.\n\n# Garbage Collection\n\nGateways rarely store content permanently. However, running garbage collection\ncan slow down a go-ipfs node significantly. If you've noticed this issue in\nproduction, consider \"garbage collecting\" by resetting the go-ipfs repo whenever\nyou run out of space, instead of garbage collecting.\n\n1. Initialize your gateways repo to some known-good state (possibly pre-seeding\n it with some content, a config, etc.).\n2. When you start running low on space, for each load-balanced go-ipfs node:\n 1. Use the nginx API to set one of the upstream go-ipfs node's to \"down\".\n 2. Wait a minute to let go-ipfs finish processing any in-progress requests\n (or the short-lived ones, at least).\n 3. Take the go-ipfs node down.\n 4. Rollback the go-ipfs repo to the seed state.\n 5. Restart the go-ipfs daemon.\n 6. Update the nginx config, removing the \"down\" status from the node.\n\nThis will effectively \"garbage collect\" without actually running the garbage\ncollector.\n\n# Content Blocking\n\nTODO:\n\n* Filtering requests\n* Checking the X-IPFS-Path header in responses to filter again after resolving.\n\n# Subdomain Gateway\n\nTODO: Reverse proxies and the subdomain gateway.\n\n# Load balancing\n\nTODO: discuss load balancing based on the CID versus the source IP.\n" }, { "path": "docs/provide-stats.md", "content": "# Provide Stats\n\nThe `ipfs provide stat` command gives you statistics about your local provide\nsystem. This file provides a detailed explanation of the metrics reported by\nthis command.\n\n## Understanding the Metrics\n\nThe statistics are organized into three types of measurements:\n\n### Per-worker rates\n\nMetrics like \"CIDs reprovided/min/worker\" measure the throughput of a single\nworker processing one region. To estimate total system throughput, multiply by\nthe number of active workers of that type (see [Workers stats](#workers-stats)).\n\nExample: If \"CIDs reprovided/min/worker\" shows 100 and you have 10 active\nperiodic workers, your total reprovide throughput is approximately 1,000\nCIDs/min.\n\n### Per-region averages\n\nMetrics like \"Avg CIDs/reprovide\" measure properties of the work units (keyspace\nregions). These represent the average size or characteristics of a region, not a\nrate. Do NOT multiply these by worker count.\n\nExample: \"Avg CIDs/reprovide: 250,000\" means each region contains an average of\n250,000 CIDs that get reprovided together as a batch.\n\n### System totals\n\nMetrics like \"Total CIDs provided\" are cumulative counts since node startup.\nThese aggregate all work across all workers over time.\n\n## Connectivity\n\n### Status\n\nCurrent connectivity status (`online`, `disconnected`, or `offline`) and when\nit last changed (see [provide connectivity\nstatus](./config.md#providedhtofflinedelay)).\n\n## Queues\n\n### Provide queue\n\nNumber of CIDs waiting for initial provide, and the number of keyspace regions\nthey're grouped into.\n\n### Reprovide queue\n\nNumber of regions with overdue reprovides. These regions missed their scheduled\nreprovide time and will be processed as soon as possible. If decreasing, the\nnode is recovering from downtime. If increasing, either the node is offline or\nthe provide system needs more workers (see\n[`Provide.DHT.MaxWorkers`](./config.md#providedhtmaxworkers)\nand\n[`Provide.DHT.DedicatedPeriodicWorkers`](./config.md#providedhtdedicatedperiodicworkers)).\n\n## Schedule\n\n### CIDs scheduled\n\nTotal CIDs scheduled for reprovide.\n\n### Regions scheduled\n\nNumber of keyspace regions scheduled for reprovide. Each CID is mapped to a\nspecific region, and all CIDs within the same region are reprovided together as\na batch for efficient processing.\n\n### Avg prefix length\n\nAverage length of binary prefixes identifying the scheduled regions. Each\nkeyspace region is identified by a binary prefix, and this shows the average\nprefix length across all regions in the schedule. Longer prefixes indicate the\nkeyspace is divided into more regions (because there are more DHT servers in the\nswarm to distribute records across).\n\n### Next region prefix\n\nKeyspace prefix of the next region to be reprovided.\n\n### Next region reprovide\n\nWhen the next region is scheduled to be reprovided.\n\n## Timings\n\n### Uptime\n\nHow long the provide system has been running since Kubo started, along with the\nstart timestamp.\n\n### Current time offset\n\nElapsed time in the current reprovide cycle, showing cycle progress (e.g., '11h'\nmeans 11 hours into a 22-hour cycle, roughly halfway through).\n\n### Cycle started\n\nWhen the current reprovide cycle began.\n\n### Reprovide interval\n\nHow often each CID is reprovided (the complete cycle duration).\n\n## Network\n\n### Avg record holders\n\nAverage number of provider records successfully sent for each CID to distinct\nDHT servers. In practice, this is often lower than the [replication\nfactor](#replication-factor) due to unreachable peers or timeouts. Matching the\nreplication factor would indicate all DHT servers are reachable.\n\nNote: this counts successful sends; some DHT servers may have gone offline\nafterward, so actual availability may be lower.\n\n### Peers swept\n\nNumber of DHT servers to which we tried to send provider records in the last\nreprovide cycle (sweep). Excludes peers contacted during initial provides or\nDHT lookups.\n\n### Full keyspace coverage\n\nWhether provider records were sent to all DHT servers in the swarm during the\nlast reprovide cycle. If true, [peers swept](#peers-swept) approximates the\ntotal DHT swarm size over the last [reprovide interval](#reprovide-interval).\n\n### Reachable peers\n\nNumber and percentage of peers to which we successfully sent all provider\nrecords assigned to them during the last reprovide cycle.\n\n### Avg region size\n\nAverage number of DHT servers per keyspace region.\n\n### Replication factor\n\nTarget number of DHT servers to receive each provider record.\n\n## Operations\n\n### Ongoing provides\n\nNumber of CIDs and regions currently being provided for the first time. More\nCIDs than regions indicates efficient batching. Each region provide uses a\n[burst\nworker](./config.md#providedhtdedicatedburstworkers).\n\n### Ongoing reprovides\n\nNumber of CIDs and regions currently being reprovided. Each region reprovide\nuses a [periodic\nworker](./config.md#providedhtdedicatedperiodicworkers).\n\n### Total CIDs provided\n\nTotal number of provide operations since node startup (includes both provides\nand reprovides).\n\n### Total records provided\n\nTotal provider records successfully sent to DHT servers since startup (includes\nreprovides).\n\n### Total provide errors\n\nNumber of failed region provide/reprovide operations since startup. Failed\nregions are automatically retried unless the node is offline.\n\n### CIDs provided/min/worker\n\nAverage rate of initial provides per minute per worker during the last\nreprovide cycle (excludes reprovides). Each worker handles one keyspace region\nat a time, providing all CIDs in that region. This measures the throughput of a\nsingle worker only.\n\nTo estimate total system provide throughput, multiply by the number of active\nburst workers shown in [Workers stats](#workers-stats) (Burst > Active).\n\nNote: This rate only counts active time when initial provides are being\nprocessed. If workers are idle, actual throughput may be lower.\n\n### CIDs reprovided/min/worker\n\nAverage rate of reprovides per minute per worker during the last reprovide\ncycle (excludes initial provides). Each worker handles one keyspace region at a\ntime, reproviding all CIDs in that region. This measures the throughput of a\nsingle worker only.\n\nTo estimate total system reprovide throughput, multiply by the number of active\nperiodic workers shown in [Workers stats](#workers-stats) (Periodic > Active).\n\nExample: If this shows 100 CIDs/min and you have 10 active periodic workers,\nyour total reprovide throughput is approximately 1,000 CIDs/min.\n\nNote: This rate only counts active time when regions are being reprovided. If\nworkers are idle due to network issues or queue exhaustion, actual throughput\nmay be lower.\n\n### Region reprovide duration\n\nAverage time to reprovide all CIDs in a region during the last cycle.\n\n### Avg CIDs/reprovide\n\nAverage number of CIDs per region during the last reprovide cycle.\n\nThis measures the average size of a region (how many CIDs are batched together),\nnot a throughput rate. Do NOT multiply this by worker count.\n\nCombined with [Region reprovide duration](#region-reprovide-duration), this\nhelps estimate per-worker throughput: dividing Avg CIDs/reprovide by Region\nreprovide duration gives CIDs/min/worker.\n\n### Regions reprovided (last cycle)\n\nNumber of regions reprovided in the last cycle.\n\n> [!NOTE]\n> (⚠️ 0.39 limitation) If this shows 1 region while using\n> [`Routing.AcceleratedDHTClient`](./config.md#routingaccelerateddhtclient), sweep mode lost\n> efficiency gains. Consider disabling the accelerated client. See [caveat 4](./config.md#routingaccelerateddhtclient).\n\n## Workers\n\n### Active workers\n\nNumber of workers currently processing provide or reprovide operations.\n\n### Free workers\n\nNumber of idle workers not reserved for periodic or burst tasks.\n\n### Workers stats\n\nBreakdown of worker status by type (periodic for scheduled reprovides, burst for\ninitial provides). For each type:\n\n- **Active**: Currently processing operations (use this count when calculating total throughput from per-worker rates)\n- **Dedicated**: Reserved for this type\n- **Available**: Idle dedicated workers + [free workers](#free-workers)\n- **Queued**: 0 or 1 (workers acquired only when needed)\n\nThe number of active workers determines your total system throughput. For\nexample, if you have 10 active periodic workers, multiply\n[CIDs reprovided/min/worker](#cids-reprovidedminworker) by 10 to estimate total\nreprovide throughput.\n\nSee [provide queue](#provide-queue) and [reprovide queue](#reprovide-queue) for\nregions waiting to be processed.\n\n### Max connections/worker\n\nMaximum concurrent DHT server connections per worker when sending provider\nrecords for a region.\n\n## Capacity Planning\n\n### Estimating if your system can keep up with the reprovide schedule\n\nTo check if your provide system has sufficient capacity:\n\n1. Calculate required throughput:\n - Required CIDs/min = [CIDs scheduled](#cids-scheduled) / ([Reprovide interval](#reprovide-interval) in minutes)\n - Example: 67M CIDs / (22 hours × 60 min) = 50,758 CIDs/min needed\n\n2. Calculate actual throughput:\n - Actual CIDs/min = [CIDs reprovided/min/worker](#cids-reprovidedminworker) × Active periodic workers\n - Example: 100 CIDs/min/worker × 256 active workers = 25,600 CIDs/min\n\n3. Compare:\n - If actual < required: System is underprovisioned, increase [MaxWorkers](./config.md#providedhtmaxworkers) or [DedicatedPeriodicWorkers](./config.md#providedhtdedicatedperiodicworkers)\n - If actual > required: System has excess capacity\n - If [Reprovide queue](#reprovide-queue) is growing: System is falling behind\n\n### Understanding worker utilization\n\n- High active workers with growing reprovide queue: Need more workers or network connectivity is limiting throughput\n- Low active workers with non-empty reprovide queue: Workers may be waiting for network or DHT operations\n- Check [Reachable peers](#reachable-peers) to diagnose network connectivity issues\n- (⚠️ 0.39 limitation) If [Regions scheduled](#regions-scheduled) shows 1 while using\n [`Routing.AcceleratedDHTClient`](./config.md#routingaccelerateddhtclient), consider disabling\n the accelerated client to restore sweep efficiency. See [caveat 4](./config.md#routingaccelerateddhtclient).\n\n## See Also\n\n- [Provide configuration reference](./config.md#provide)\n- [Provide metrics for Prometheus](./metrics.md#provide)\n" }, { "path": "docs/releases.md", "content": "# `kubo` Release Flow\n\n# Table of Contents\n\n- [`kubo` Release Flow](#kubo-release-flow)\n- [Table of Contents](#table-of-contents)\n - [Release Philosophy](#release-philosophy)\n - [Release Flow](#release-flow)\n - [Stage 0 - Automated Testing](#stage-0---automated-testing)\n - [Stage 1 - Internal Testing](#stage-1---internal-testing)\n - [Stage 2 - Community Dev Testing](#stage-2---community-dev-testing)\n - [Stage 3 - Community Prod Testing](#stage-3---community-prod-testing)\n - [Stage 4 - Release](#stage-4---release)\n - [Release Cycle](#release-cycle)\n - [Patch Releases](#patch-releases)\n - [Security Fix Policy](#security-fix-policy)\n - [Performing a Release](#performing-a-release)\n - [Release Version Numbers (aka semver)](#release-version-numbers-aka-semver)\n - [_Footnotes_](#footnotes)\n\n## Release Philosophy\n\n`kubo` aims to have a release every six weeks, two releases per quarter. During these 6 week releases, we go through 4 different stages that allow us to test the new version against our test environments (unit, interop, integration), QA in our current production environment, IPFS apps (e.g. Desktop and WebUI) and with our community and _early testers_[1] that have IPFS running in production.\n\nWe might expand the six-week release schedule in case of:\n\n- No new updates to be added\n- In case of a large community event that takes the core team availability away (e.g. IPFS Conf, Dev Meetings, IPFS Camp, etc.)\n\n## Release Flow\n\n`kubo` releases come in 5 stages designed to gradually roll out changes and reduce the impact of any regressions that may have been introduced. If we need to merge non-trivial[2] changes during the process, we start over at stage 0.\n\n![kubo-release-process-illustration](https://user-images.githubusercontent.com/618519/62986422-653fee00-bdf0-11e9-8f61-197117b61da2.png)\n\n### Stage 0 - Automated Testing\n\nAt this stage, we expect _all_ automated tests (interop, testlab, performance, etc.) to pass.\n\n### Stage 1 - Internal Testing\n\nAt this stage, we'll:\n\n1. Start a partial-rollout to our own infrastructure.\n2. Test against ipfs and ipfs-shipyard applications.\n\n**Goals:**\n\n1. Make sure we haven't introduced any obvious regressions.\n2. Test the release in an environment we can monitor and easily roll back (i.e. our own infra).\n\n### Stage 2 - Community Dev Testing\n\nAt this stage, we'll announce the impending release to the community and ask for beta testers.\n\n**Goal:**\n\nTest the release in as many non-production environments as possible. This is relatively low-risk but gives us a _breadth_ of testing internal testing can't.\n\n### Stage 3 - Community Prod Testing\n\nAt this stage, we consider the release to be \"production-ready\" and will ask the community and our early testers to (partially) deploy the release to their production infrastructure.\n\n**Goals:**\n\n1. Test the release in some production environments with heavy workloads.\n2. Partially roll-out an upgrade to see how it affects the network.\n3. Retain the ability to ship last-minute fixes before the final release.\n\n### Stage 4 - Release\n\nAt this stage, the release is \"battle-hardened\" and ready for wide deployment.\n\n## Release Cycle\n\nA full release process should take about 3 weeks, a week per stage 1-3. We will start a new process every 6 weeks, regardless of when the previous release landed unless it's still ongoing.\n\n### Patch Releases\n\nIf we encounter a serious bug in the stable latest release, we will create a patch release based on this release. For now, bug fixes will _not_ be backported to previous releases.\n\nPatch releases will usually follow a compressed release cycle and should take 2-3 days. In a patch release:\n\n1. Automated and internal testing (stage 0 and 1) will be compressed into a few hours - ideally less than a day.\n2. Stage 2 will be skipped.\n3. Community production testing will be shortened to 1-2 days of opt-in testing in production (early testers can choose to pass).\n\nSome patch releases, especially ones fixing one or more complex bugs, may undergo the full release process.\n\n## Security Fix Policy\n\nAny release may contain security fixes. Unless the fix addresses a bug being exploited in the wild, the fix will _not_ be called out in the release notes. Please make sure to update ASAP.\n\nBy policy, the team will usually wait until about 2 weeks after the final release to announce any fixed security issues. However, depending on the impact and ease of discovery of the issue, the team may wait more or less time. It is important to always update to the latest version ASAP and file issues if you're unable to update for some reason.\n\nFinally, unless a security issue is actively being exploited or a significant number of users are unable to update to the latest version (e.g., due to a difficult migration, breaking changes, etc.), security fixes will _not_ be backported to previous releases.\n\n## Performing a Release\n\nThe release is managed by the `Lead Maintainer` for `kubo`. It starts with the opening of an issue containing the content available on the [RELEASE_ISSUE_TEMPLATE](./RELEASE_ISSUE_TEMPLATE.md) not more than **48 hours** after the previous release.\n\nThis issue is pinned and labeled [\"release\"](https://github.com/ipfs/kubo/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3Arelease). When the cycle is due to begin the 5 stages will be followed until the release is done.\n\n## Release Version Numbers (aka semver)\n\nUntil `kubo` 0.4.X, `kubo` was not using semver to communicate the type of release\n\nPost `kubo` 0.5.X, `kubo` will use semver. This means that patch releases will not contain any breaking changes nor new features. Minor releases might contain breaking changes and always contain some new feature\n\nPost `kubo` 1.X.X (future), `kubo` will use semver. This means that only major releases will contain breaking changes, minors will be reserved for new features and patches for bug fixes.\n\nWe do not yet retroactively apply fixes to older releases (no Long Term Support releases for now), which means that we always recommend users to update to the latest, whenever possible.\n\n----------------------------\n\n## _Footnotes_\n\n- **[1]** - _early testers_ is an IPFS programme in which members of the community can self-volunteer to help test `kubo` Release Candidates. You find more info about it at [EARLY_TESTERS.md](./EARLY_TESTERS.md)\n- **[2]** - A non-trivial change is any change that could potentially introduce an issue not trivially caught by automated testing. This is up to the discretion of the Lead Maintainer but the assumption is that every change is non-trivial unless proven otherwise.\n" }, { "path": "docs/releases_thunderdome.md", "content": "# Testing Kubo releases with Thunderdome\nThis document is for running Thunderdome tests by release engineers as part of releasing Kubo.\n\nWe use Thunderdome to replay ipfs.io gateway traffic in a controlled environment against two different versions of Kubo, and we record metrics and compare them to look for logic or performance regressions before releasing a new Kubo version.\n\nFor background information about how Thunderdome works, see: https://github.com/ipfs-shipyard/thunderdome\n\n## Prerequisites\n\n* Ensure you have access to the \"IPFS Stewards\" vault in 1Password, which contains the requisite AWS Console and API credentials\n* Ensure you have Docker and the Docker CLI installed\n* Checkout the Thunderdome repo locally (or `git pull` to ensure it's up-to-date)\n* Install AWS CLI v2: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html\n* Configure the AWS CLI\n * Configure the credentials as described in the [Thunderdome documentation](https://github.com/ipfs-shipyard/thunderdome/blob/main/cmd/thunderdome/README.md#credentials), using the credentials from 1Password\n* Make sure the `thunderdome` binary is up-to-date: `go build ./cmd/thunderdome`\n \n## Add & run an experiment\n\nCreate a new release configuration JSON in the `experiments/` directory, based on the most recent `kubo-release` configuration, and tweak as necessary. Generally we setup the targets to run a commit against the tag of the last release, such as:\n\n```json\n\t\"targets\": [\n\t\t{\n\t\t\t\"name\": \"kubo190-4283b9\",\n\t\t\t\"description\": \"kubo 0.19.0-rc1\",\n\t\t\t\"build_from_git\": {\n\t\t\t\t\"repo\": \"https://github.com/ipfs/kubo.git\",\n\t\t\t\t\"commit\":\"4283b9d98f8438fc8751ccc840d8fc24eeae6f13\"\n\t\t\t}\n\t\t},\n\t\t{\n\t\t\t\"name\": \"kubo181\",\n\t\t\t\"description\": \"kubo 0.18.\",\n\t\t\t\"build_from_git\": {\n\t\t\t\t\"repo\": \"https://github.com/ipfs/kubo.git\",\n\t\t\t\t\"tag\":\"v0.18.1\"\n\t\t\t}\n\t\t}\n\t]\n```\n \nRun the experiment (where `$EXPERIMENT_CONFIG_JSON` is a path to the config JSON created above):\n\n```shell\nAWS_PROFILE=thunderdome ./thunderdome deploy --verbose --duration 120 $EXPERIMENT_CONFIG_JSON\n```\n\nThis will build the Docker images, upload them to ECR, and then launch the experiment in Thunderdome. Once the experiment starts, the CLI will exit and the experiment will continue to run for the duration.\n\n## Analyze Results\n\nAdd a log entry in https://www.notion.so/ceb2047e79f2498494077a2739a6c493 and link to it from the release issue, so that experiment results are publicly visible.\n\nThe `deploy` command will output a link to the Grafana dashboard for the experiment. We don't currently have rigorous acceptance criteria, so you should look for anomalies or changes in the metrics and make sure they are tolerable and explainable. Unexplainable anomalies should be noted in the log with a screenshot, and then root caused.\n\n\n## Open a PR to merge the experiment config into Thunderdome\n\nThis is important for both posterity, and so that someone else can sanity-check the test parameters.\n" }, { "path": "docs/specifications/keystore.md", "content": "# ![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) Keystore\n\n**Authors(s):**\n- [whyrusleeping](github.com/whyrusleeping)\n- [Hector Sanjuan](github.com/hsanjuan)\n\n**Abstract**\n\nThis spec provides definitions and operations for the keystore feature in IPFS.\n\n# Table of Contents\n\n- [Goals](#goals)\n- [Planned Implementation](#planned-implementation)\n - [Key storage](#key-storage)\n - [Interface](#interface)\n - [Code changes and additions](#code-changes-and-additions)\n - [Structures](#structures)\n\n## Goals\n\nTo have a secure, simple and user-friendly way of storing and managing keys\nfor use by ipfs. As well as the ability to share these keys, encrypt, decrypt,\nsign and verify data.\n\n## Planned Implementation\n\n### Key storage\n\nStorage layout and format is defined in the [`repository_fs`](repository_fs.md) part of the spec.\n\n### Interface\n\n#### ipfs key\n\n```\nUSAGE\n ipfs key - Create and list IPNS name keypairs\n\n ipfs key\n\n 'ipfs key gen' generates a new keypair for usage with IPNS and 'ipfs name\n publish'.\n \n > ipfs key gen --type=rsa --size=2048 mykey\n > ipfs name publish --key=mykey QmSomeHash\n \n 'ipfs key list' lists the available keys.\n \n > ipfs key list\n self\n mykey\n \t\t\n\nSUBCOMMANDS\n ipfs key export - Export a keypair\n ipfs key gen - Create a new keypair\n ipfs key import - Import a key and prints imported key id\n ipfs key list - List all local keypairs.\n ipfs key rename - Rename a keypair.\n ipfs key rm ... - Remove a keypair.\n ipfs key rotate - Rotates the IPFS identity.\n\n For more information about each command, use:\n 'ipfs key --help'\n```\n\n#### ipfs crypt\n\n**NOTE:** as of 2023 Q4, `ipfs crypt` commands are not implemented yet.\n\n```\n ipfs crypt - Perform cryptographic operations using ipfs keypairs\n\nSUBCOMMANDS:\n\n ipfs crypt sign - Generates a signature for the given data with a specified key\n ipfs crypt verify - Verify that the given data and signature match\n ipfs crypt encrypt - Encrypt the given data\n ipfs crypt decrypt - Decrypt the given data\n\nDESCRIPTION:\n\n `ipfs crypt` is a command used to perform various cryptographic operations\n using ipfs keypairs, including: signing, verifying, encrypting and decrypting.\n```\n\n#### Some subcommands:\n\n##### ipfs key Gen\n\n\n```\nUSAGE\n ipfs key gen - Create a new keypair\n\nSYNOPSIS\n ipfs key gen [--type= | -t] [--size= | -s]\n [--ipns-base=] [--] \n\nARGUMENTS\n\n - name of key to create\n\nOPTIONS\n\n -t, --type string - type of the key to create: rsa, ed25519. Default:\n ed25519.\n -s, --size int - size of the key to generate.\n --ipns-base string - Encoding used for keys: Can either be a multibase\n encoded CID or a base58btc encoded multihash. Takes\n {b58mh|base36|k|base32|b...}. Default: base36.\n```\n\n* * *\n\n##### Key Send\n\n```\nUSAGE\n ipfs key - Create and list IPNS name keypairs\n\nSYNOPSIS\n ipfs key\n\nDESCRIPTION\n\n 'ipfs key gen' generates a new keypair for usage with IPNS and 'ipfs name\n publish'.\n \n > ipfs key gen --type=rsa --size=2048 mykey\n > ipfs name publish --key=mykey QmSomeHash\n \n 'ipfs key list' lists the available keys.\n \n > ipfs key list\n self\n mykey\n \t\t\n\nSUBCOMMANDS\n ipfs key export - Export a keypair\n ipfs key gen - Create a new keypair\n ipfs key import - Import a key and prints imported key id\n ipfs key list - List all local keypairs.\n ipfs key rename - Rename a keypair.\n ipfs key rm ... - Remove a keypair.\n ipfs key rotate - Rotates the IPFS identity.\n\n For more information about each command, use:\n 'ipfs key --help'\n```\n\n##### Comments:\n\nEnsure that the user knows the implications of sending a key.\n\n* * *\n\n##### Crypt Encrypt\n\n```\n ipfs crypt encrypt - Encrypt the given data with a specified key\n\nARGUMENTS:\n\n data - The filename of the data to be encrypted (\"-\" for stdin)\n\nOPTIONS:\n\n -k, -key string - The name of the key to use for encryption (default: localkey)\n -o, -output string - The name of the output file (default: stdout)\n -c, -cipher string - The cipher to use for the operation\n -m, -mode string - The block cipher mode to use for the operation\n\nDESCRIPTION:\n\n 'ipfs crypt encrypt' is a command used to encrypt data so that only holders of a certain\n key can read it.\n```\n\n##### Comments:\n\nThis should probably just operate on raw data and not on DAGs.\n\n* * *\n\n##### Other Interface Changes\n\nWe will also need to make additions to support keys in other commands, these changes are as follows:\n\n- `ipfs add`\n - Support for a `-encrypt-key` option, for block encrypting the file being added with the key\n - also adds an 'encrypted' node above the root unixfs node\n - Support for a `-sign-key` option to attach a signature node above the root unixfs node\n\n- `ipfs block put`\n - Support for a `-encrypt-key` option, for encrypting the block before hashing and storing\n\n- `ipfs object put`\n - Support for a `-encrypt-key` option, for encrypting the object before hashing and storing\n\n- `ipfs name publish`\n - Support for a `-key` option to select which keyspace to publish to\n\n### Code changes and additions\n\nThis sections outlines code organization around this feature.\n\n#### Keystore package\n\nThe fsrepo carries a `keystore` that can be used to load/store keys. The keystore is implemented following this interface:\n\n```go\n// Keystore provides a key management interface\ntype Keystore interface {\n\t// Has returns whether or not a key exist in the Keystore\n\tHas(string) (bool, error)\n\t// Put stores a key in the Keystore, if a key with the same name already exists, returns ErrKeyExists\n\tPut(string, ci.PrivKey) error\n\t// Get retrieves a key from the Keystore if it exists, and returns ErrNoSuchKey\n\t// otherwise.\n\tGet(string) (ci.PrivKey, error)\n\t// Delete removes a key from the Keystore\n\tDelete(string) error\n\t// List returns a list of key identifier\n\tList() ([]string, error)\n}\n```\n\nNote: Never store passwords as strings, strings cannot be zeroed out after they are used.\nusing a byte array allows you to write zeroes over the memory so that the users password\ndoes not linger in memory.\n\n#### Unixfs\n\n- new node types, 'encrypted' and 'signed', probably shouldn't be in unixfs, just understood by it\n- if new node types are not unixfs nodes, special consideration must be given to the interop\n\n- DagReader needs to be able to access keystore to seamlessly stream encrypted data we have keys for\n - also needs to be able to verify signatures\n\n#### Importer\n\n- DagBuilderHelper needs to be able to encrypt blocks\n - Dag Nodes should be generated like normal, then encrypted, and their parents should\n link to the hash of the encrypted node\n- DagBuilderParams should have extra parameters to accommodate creating a DBH that encrypts the blocks\n\n#### New 'Encrypt' package\n\nShould contain code for crypto operations on dags.\n\nEncryption of dags should work by first generating a symmetric key, and using\nthat key to encrypt all the data. That key should then be encrypted with the\npublic key chosen and stored in the Encrypted DAG structure.\n\nNote: One option is to simply add it to the key interface.\n\n### Structures\nSome tentative mockups (in json) of the new DAG structures for signing and encrypting\n\nSigned DAG:\n```\n{\n \"Links\" : [\n {\n \"Name\":\"@content\",\n \"Hash\":\"QmTheContent\",\n }\n ],\n \"Data\": protobuf{\n \"Type\":\"Signed DAG\",\n \"Signature\": \"thesignature\",\n \"PubKeyID\": \"QmPubKeyHash\",\n }\n}\n```\n\nEncrypted DAG:\n```\n{\n \"Links\" : [\n {\n \"Name\":\"@content\",\n \"Hash\":\"QmRawEncryptedDag\",\n }\n ],\n \"Data\": protobuf{\n \"Type\":\"Encrypted DAG\",\n \"PubKeyID\": \"QmPubKeyHash\",\n \"Key\": \"ephemeral symmetric key, encrypted with public key\",\n }\n}\n```\n" }, { "path": "docs/specifications/repository.md", "content": "# ![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) IPFS Repo Spec\n\n**Author(s)**:\n- [Juan Benet](github.com/jbenet)\n\n**Abstract**\n\nThis spec defines an IPFS Repo, its contents, and its interface. It does not specify how the repo data is actually stored, as that is done via swappable implementations.\n\n# Table of Contents\n\n- [Definition](#definition)\n- [Repo Contents](#repo-contents)\n - [version](#version)\n - [datastore](#datastore)\n - [keystore](#keystore)\n - [config (state)](#config-state)\n - [locks](#locks)\n - [datastore\\_spec](#datastore_spec)\n - [hooks (TODO)](#hooks-todo)\n- [Notes](#notes)\n\n## Definition\n\nA `repo` is the storage repository of an IPFS node. It is the subsystem that\nactually stores the data IPFS nodes use. All IPFS objects are stored\nin a repo (similar to git).\n\nThere are many possible repo implementations, depending on the storage media\nused. Most commonly, IPFS nodes use an [fs-repo](repository_fs.md).\n\nRepo Implementations:\n- [fs-repo](repository_fs.md) - stored in the os filesystem\n- mem-repo - stored in process memory\n- s3-repo - stored in amazon s3\n\n## Repo Contents\n\nThe Repo stores a collection of [IPLD](https://github.com/ipld/specs#readme) objects that represent:\n\n- **config** - node configuration and settings\n- **datastore** - content stored locally, and indexing data\n- **keystore** - cryptographic keys, including node's identity\n- **hooks** - scripts to run at predefined times (not yet implemented)\n\nNote that the IPLD objects a repo stores are divided into:\n- **state** (system, control plane) used for the node's internal state\n- **content** (userland, data plane) which represent the user's cached and pinned data.\n\nAdditionally, the repo state must determine the following. These need not be IPLD objects, though it is of course encouraged:\n\n- **version** - the repo version, required for safe migrations\n- **locks** - process semaphores for correct concurrent access\n- **datastore_spec** - array of mounting points and their properties\n\nFinally, the repo also stores the blocks with blobs containing binary data.\n\n![](./ipfs-repo-contents.png)\n\n### version\n\nRepo implementations may change over time, thus they MUST include a `version` recognizable across versions. Meaning that a tool MUST be able to read the `version` of a given repo type.\n\nFor example, the `fs-repo` simply includes a `version` file with the version number. This way, the repo contents can evolve over time but the version remains readable the same way across versions.\n\n### datastore\n\nIPFS nodes store some IPLD objects locally. These are either (a) **state objects** required for local operation -- such as the `config` and `keys` -- or (b) **content objects** used to represent data locally available. **Content objects** are either _pinned_ (stored until they are unpinned) or _cached_ (stored until the next repo garbage collection).\n\nThe name \"datastore\" comes from [go-datastore](https://github.com/jbenet/go-datastore), a library for swappable key-value stores. Like its name-sake, some repo implementations feature swappable datastores, for example:\n- an fs-repo with a leveldb datastore\n- an fs-repo with a boltdb datastore\n- an fs-repo with a union fs and leveldb datastore\n- an fs-repo with an s3 datastore\n- an s3-repo with a cached fs and s3 datastore\n\nThis makes it easy to change properties or performance characteristics of a repo without an entirely new implementation.\n\n### keystore\n\nA Repo typically holds the keys a node has access to, for signing and for encryption.\n\nDetails on operation and storage of the keystore can be found in [`repository_fs.md`](repository_fs.md) and [`keystore.md`](keystore.md).\n\n### config (state)\n\nThe node's `config` (configuration) is a tree of variables, used to configure various aspects of operation. For example:\n- the set of bootstrap peers IPFS uses to connect to the network\n- the Swarm, API, and Gateway network listen addresses\n- the Datastore configuration regarding the construction and operation of the on-disk storage system.\n\nThere is a set of properties, which are mandatory for the repo usage. Those are `Addresses`, `Discovery`, `Bootstrap`, `Identity`, `Datastore` and `Keychain`.\n\nIt is recommended that `config` files avoid identifying information, so that they may be re-shared across multiple nodes.\n\n**CHANGES**: today, implementations like js-ipfs and go-ipfs store the peer-id and private key directly in the config. These will be removed and moved out.\n\n### locks\n\nIPFS implementations may use multiple processes, or may disallow multiple processes from using the same repo simultaneously. Others may disallow using the same repo but may allow sharing _datastores_ simultaneously. This synchronization is accomplished via _locks_.\n\nAll repos contain the following standard locks:\n- `repo.lock` - prevents concurrent access to the repo. Must be held to _read_ or _write_.\n\n### datastore_spec\n\nThis file is created according to the Datastore configuration specified in the `config` file. It contains an array with all the mounting points that the repo is using, as well as its properties. This way, the `datastore_spec` file must have the same mounting points as defined in the Datastore configuration.\n\nIt is important pointing out that the `Datastore` in config must have a `Spec` property, which defines the structure of the ipfs datastore. It is a composable structure, where each datastore is represented by a json object.\n\n### hooks (TODO)\n\nLike git, IPFS nodes will allow `hooks`, a set of user configurable scripts to run at predefined moments in IPFS operations. This makes it easy to customize the behavior of IPFS nodes without changing the implementations themselves.\n\n## Notes\n\n#### A Repo uniquely identifies an IPFS Node\n\nA repository uniquely identifies a node. Running two different IPFS programs with identical repositories -- and thus identical identities -- WILL cause problems.\n\nDatastores MAY be shared -- with proper synchronization -- though note that sharing datastore access MAY erode privacy.\n\n#### Repo implementation changes MUST include migrations\n\n**DO NOT BREAK USERS' DATA.** This is critical. Thus, any changes to a repo's implementation **MUST** be accompanied by a **SAFE** migration tool.\n\nSee https://github.com/jbenet/go-ipfs/issues/537 and https://github.com/jbenet/random-ideas/issues/33\n\n#### Repo Versioning\n\nA repo version is a single incrementing integer. All versions are considered non-compatible. Repos of different versions MUST be run through the appropriate migration tools before use.\n" }, { "path": "docs/specifications/repository_fs.md", "content": "# ![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) fs-repo\n\n**Author(s)**:\n- [Juan Benet](github.com/jbenet)\n- [David Dias](github.com/daviddias)\n- [Hector Sanjuan](github.com/hsanjuan)\n\n**Abstract**\n\nThis spec defines `fs-repo` version `1`, its formats, and semantics.\n\n# Table of Contents\n\n- [Definition](#definition)\n- [Contents](#contents)\n - [api](#api)\n - [blocks/](#blocks)\n - [config](#config)\n - [hooks/](#hooks)\n - [keystore/](#keystore)\n - [datastore/](#datastore)\n - [logs/](#logs)\n - [repo.lock](#repolock)\n - [version](#version)\n- [Datastore](#datastore-1)\n- [Notes](#notes)\n - [Location](#location)\n - [blocks/ with an fs-datastore](#blocks-with-an-fs-datastore)\n - [Reading without the `repo.lock`](#reading-without-the-repolock)\n\n## Definition\n\n`fs-repo` is a filesystem implementation of the IPFS [repo](repository.md).\n\n\n## Contents\n\n![](img/ipfs-repo-contents.png?)\n\n```\n.ipfs/\n├── api <--- running daemon api addr\n├── blocks/ <--- objects stored directly on disk\n│ └── aa <--- prefix namespacing like git\n│ └── aa <--- N tiers\n├── config <--- config file (json or toml)\n├── hooks/ <--- hook scripts\n├── keystore/ <--- cryptographic keys\n│ ├── key_b32name <--- private key with base32-encoded name\n├── datastore/ <--- datastore\n├── logs/ <--- 1 or more files (log rotate)\n│ └── events.log <--- can be tailed\n├── repo.lock <--- mutex for repo\n└── version <--- version file\n```\n\n### api\n\n`./api` is a file that exists to denote an API endpoint to listen to.\n- It MAY exist even if the endpoint is no longer live (i.e. it is a _stale_ or left-over `./api` file).\n\nIn the presence of an `./api` file, ipfs tools (e.g. go-ipfs `ipfs daemon`) MUST attempt to delegate to the endpoint, and MAY remove the file if reasonably certain the file is stale. (e.g. endpoint is local, but no process is live)\n\nThe `./api` file is used in conjunction with the `repo.lock`. Clients may opt to use the api service, or wait until the process holding `repo.lock` exits. The file's content is the api endpoint as a [multiaddr](https://github.com/jbenet/multiaddr)\n\n```\n> cat .ipfs/api\n/ip4/127.0.0.1/tcp/5001\n```\n\nNotes:\n- The API server must remove the api file before releasing the `repo.lock`.\n- It is not enough to use the `config` file, as the API addr of a daemon may\n have been overridden via ENV or flag.\n\n#### api file for remote control\n\nOne use case of the `api` file is to have a repo directory like:\n\n```\n> tree $IPFS_PATH\n/Users/jbenet/.ipfs\n└── api\n\n0 directories, 1 files\n\n> cat $IPFS_PATH/api\n/ip4/1.2.3.4/tcp/5001\n```\n\nIn go-ipfs, this has the same effect as:\n\n```\nipfs --api /ip4/1.2.3.4/tcp/5001 \n```\n\nMeaning that it makes ipfs tools use an ipfs node at the given endpoint, instead of the local directory as a repo.\n\nIn this use case, the rest of the `$IPFS_PATH` may be completely empty, and no other information is necessary. It cannot be said it is a _repo_ per-se. (TODO: come up with a good name for this).\n\n### blocks/\n\nThe `block/` component contains the raw data representing all IPFS objects\nstored locally, whether pinned or cached. This component is controlled by the `\ndatastore`. For example, it may be stored within a leveldb instance in `\ndatastore/`, or it may be stored entirely with independent files, like git.\n\nIn the default case, the user uses fs-datastore for all `/blocks` so the\nobjects are stored in individual files. In other cases, `/blocks` may even be\nstored remotely\n\n- [blocks/ with an fs-datastore](#blocks-with-an-fs-datastore)\n\n### config\n\nThe `config` file is a JSON or TOML file that contains the tree of\nconfiguration variables. It MUST only be changed while holding the\n`repo.lock`, or potentially lose edits.\n\n### hooks/\n\nThe `hooks` directory contains executable scripts to be called on specific\nevents to alter ipfs node behavior.\n\nCurrently available hooks:\n\n```\nnone\n```\n\n### keystore/\n\n\nThe `keystore` directory holds additional private keys that the node has\naccess to (the public keys can be derived from them).\n\nThe keystore repository should have `0700` permissions (readable, writable by\nthe owner only).\n\nThe key files are named as `key_base32encodedNameNoPadding` where `key_` is a\nfixed prefix followed by a base32 encoded identifier, **without padding and\ndowncased**. The identifier usually corresponds to a human-friendly name given\nby the user.\n\nThe key files should have '0400' permissions (read-only, by the owner only).\n\nThe `self` key identifier is reserved for the peer's main key, and therefore key named\n`key_onswyzq` is allowed in this folder.\n\nThe key files themselves contain a serialized representation of the keys as\ndefined in the\n[libp2p specification](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#keys).\n\n### datastore/\n\nThe `datastore` directory contains the data for a leveldb instance used to\nstore operation data for the IPFS node. If the user uses a `boltdb` datastore\ninstead, the directory will be named `boltdb`. Thus the data files of each\ndatabase will not clash.\n\nTODO: consider whether all should just be named `leveldb/`\n\n### logs/\n\nIPFS implementations put event log files inside the `logs/` directory. The\nlatest log file is `logs/events`. Others, rotated out may exist, with a\ntimestamp of their creation. For example:\n\n\n\n### repo.lock\n\n`repo.lock` prevents concurrent access to the repo. Its content SHOULD BE the\nPID of the process currently holding the lock. This allows clients to detect\na failed lock and cleanup.\n\n```\n> cat .ipfs/repo.lock\n42\n> ps | grep \"ipfs daemon\"\n42 ttys000 79:05.83 ipfs daemon\n```\n\n**TODO, ADDRESS DISCREPANCY:** the go-ipfs implementation does not currently store the PID in the file, which in some systems causes failures after a failure or a teardown. This SHOULD NOT require any manual intervention-- a present lock should give new processes enough information to recover. Doing this correctly in a portable, safe way, with good UX is very tricky. We must be careful with TOCTTOU bugs, and multiple concurrent processes capable of running at any moment. The goal is for all processes to operate safely, to avoid bothering the user, and for the repo to always remain in a correct, consistent state.\n\n### version\n\nThe `version` file contains the repo implementation name and version. This format has changed over time:\n\n```\n# in version 0\n> cat $repo-at-version-0/version\ncat: /Users/jbenet/.ipfs/version: No such file or directory\n\n# in versions 1 and 2\n> cat $repo-at-version-1/version\n1\n> cat $repo-at-version-2/version\n2\n\n# in versions >3\n> cat $repo-at-version-3/version\nfs-repo/3\n```\n\n_Any_ fs-repo implementation of _any_ versions `>0` MUST be able to read the\n`version` file. It MUST NOT change format between versions. The sole exception is version 0, which had no file.\n\n**TODO: ADDRESS DISCREPANCY:** versions 1 and 2 of the go-ipfs implementation use just the integer number. It SHOULD have used `fs-repo/`. We could either change the spec and always just use the int, or change go-ipfs in version `>3`. we will have to be backwards compatible.\n\n## Datastore\n\nBoth the `/blocks` and `/datastore` directories are controlled by the\n`datastore` component of the repo.\n\n## Notes\n\n### Location\n\nThe `fs-repo` can be located anywhere on the filesystem. By default\nclients should search for a repo in:\n\n```\n~/.ipfs\n```\n\nUsers can tell IPFS programs to look elsewhere with the env var:\n\n```\nIPFS_PATH=/path/to/repo\n```\n\n### blocks/ with an fs-datastore\n\n![](fs-datastore.png)\n\nEach object is stored in its own file. The filename is the hash of the object.\nThe files are nested in directories whose names are prefixes of the hash, as\nin `.git/objects`.\n\nFor example:\n```sh\n# multihashes\n1220fe389b55ea958590769f9046b0f7268bca90a92e4a9f45cbb30930f4bf89269d # sha2\n1114f623e0ec7f8719fb14a18838d2a3ef4e550b5e53 # sha1\n\n# locations of the blocks\n.ipfs/blocks/1114/f6/23/e0ec7f8719fb14a18838d2a3ef4e550b5e53\n.ipfs/blocks/1220/fe/38/9b55ea958590769f9046b0f7268bca90a92e4a9f45cbb30930f4bf89269d\n```\n\n**Important Notes:**\n- the hashes are encoded in hex, not the usual base58, because some\n filesystems are case insensitive.\n- the multihash prefix is two bytes, which would waste two directory levels,\n thus these are combined into one.\n- the git `idx` and `pack` file formats could be used to coalesce objects\n\n**TODO: ADDRESS DISCREPANCY:**\n\nthe go-ipfs fs-repo in version 2 uses a different `blocks/` dir layout:\n\n```\n/Users/jbenet/.ipfs/blocks\n├── 12200007\n│ └── 12200007d4e3a319cd8c7c9979280e150fc5dbaae1ce54e790f84ae5fd3c3c1a0475.data\n├── 1220000f\n│ └── 1220000fadd95a98f3a47c1ba54a26c77e15c1a175a975d88cf198cc505a06295b12.data\n```\n\nWe MUST address whether we should change the fs-repo spec to match go-ipfs in version 2, or we should change go-ipfs to match the fs-repo spec (more tiers). We MUST also address whether the levels are a repo version parameter or a config parameter. There are filesystems in which a different fanout will have wildly different performance. These are mostly networked and legacy filesystems.\n\n### Reading without the `repo.lock`\n\nPrograms MUST hold the `repo.lock` while reading and writing most files in the\nrepo. The only two exceptions are:\n\n- `repo.lock` - so clients may check for it\n- `api` - so clients may use the API\n" }, { "path": "docs/telemetry.md", "content": "# Telemetry Plugin Documentation\n\nThe **Telemetry plugin** is a feature in Kubo that collects **anonymized usage data** to help the development team better understand how the software is used, identify areas for improvement, and guide future feature development.\n\nThis data is not personally identifiable and is used solely for the purpose of improving the Kubo project.\n\n---\n\n## 🛡️ How to Control Telemetry\n\nThe behavior of the Telemetry plugin is controlled via the environment variable [`IPFS_TELEMETRY`](environment-variables.md#ipfs_telemetry) and optionally via the `Plugins.Plugins.telemetry.Config.Mode` in the IPFS config file.\n\n### Available Modes\n\n| Mode | Description |\n|----------|-----------------------------------------------------------------------------|\n| `on` | **Default**. Telemetry is enabled. Data is sent periodically. |\n| `off` | Telemetry is disabled. No data is sent. Any existing telemetry UUID file is removed. |\n| `auto` | Like `on`, but logs an informative message about the telemetry and gives user 15 minutes to opt-out before first collection. This mode is automatically used on the first run when `IPFS_TELEMETRY` is not set and telemetry UUID is not found (not generated yet). The informative message is only shown once. |\n\nYou can set the mode in your environment:\n\n```bash\nexport IPFS_TELEMETRY=\"off\"\n```\n\nOr in your IPFS config file:\n\n```json\n{\n \"Plugins\": {\n \"Plugins\": {\n \"telemetry\": {\n \"Config\": {\n \"Mode\": \"off\"\n }\n }\n }\n }\n}\n```\n\n---\n\n## 📦 What Data is Collected?\n\nThe telemetry plugin collects the following anonymized data:\n\n### General Information\n\n- **UUID**: Anonymous identifier for this node\n- **Agent version**: Kubo version string\n- **Private network**: Whether running in a private IPFS network\n- **Repository size**: Categorized into privacy-preserving buckets (1GB, 5GB, 10GB, 100GB, 500GB, 1TB, 10TB, >10TB)\n- **Uptime**: Categorized into privacy-preserving buckets (1d, 2d, 3d, 7d, 14d, 30d, >30d)\n\n### Routing & Discovery\n\n- **Custom bootstrap peers**: Whether custom `Bootstrap` peers are configured\n- **Routing type**: The `Routing.Type` configured for the node\n- **Accelerated DHT client**: Whether `Routing.AcceleratedDHTClient` is enabled\n- **Delegated routing count**: Number of `Routing.DelegatedRouters` configured\n- **AutoConf enabled**: Whether `AutoConf.Enabled` is set\n- **Custom AutoConf URL**: Whether custom `AutoConf.URL` is configured\n- **mDNS**: Whether `Discovery.MDNS.Enabled` is set\n\n### Content Providing\n\n- **Provide and Reprovide strategy**: The `Provide.Strategy` configured\n- **Sweep-based provider**: Whether `Provide.DHT.SweepEnabled` is set\n- **Custom Interval**: Whether custom `Provide.DHT.Interval` is configured\n- **Custom MaxWorkers**: Whether custom `Provide.DHT.MaxWorkers` is configured\n\n### Network Configuration\n\n- **AutoNAT service mode**: The `AutoNAT.ServiceMode` configured\n- **AutoNAT reachability**: Current reachability status determined by AutoNAT\n- **Hole punching**: Whether `Swarm.EnableHolePunching` is enabled\n- **Circuit relay addresses**: Whether the node advertises circuit relay addresses\n- **Public IPv4 addresses**: Whether the node has public IPv4 addresses\n- **Public IPv6 addresses**: Whether the node has public IPv6 addresses\n- **AutoWSS**: Whether `AutoTLS.AutoWSS` is enabled\n- **Custom domain suffix**: Whether custom `AutoTLS.DomainSuffix` is configured\n\n### Platform Information\n\n- **Operating system**: The OS the node is running on\n- **CPU architecture**: The architecture the node is running on\n- **Container detection**: Whether the node is running inside a container\n- **VM detection**: Whether the node is running inside a virtual machine\n\n### Code Reference\n\nData is organized in the `LogEvent` struct at [`plugin/plugins/telemetry/telemetry.go`](https://github.com/ipfs/kubo/blob/master/plugin/plugins/telemetry/telemetry.go). This struct is the authoritative source of truth for all telemetry data, including privacy-preserving buckets for repository size and uptime. Note that this documentation may not always be up-to-date - refer to the code for the current implementation.\n\n---\n\n## 🧑‍🤝‍🧑 Privacy and Anonymization\n\nAll data collected is:\n- **Anonymized**: No personally identifiable information (PII) is sent.\n- **Optional**: Users can choose to opt out at any time.\n- **Secure**: Data is sent over HTTPS to a trusted endpoint.\n\nThe telemetry UUID is stored in the IPFS repo folder and is used to identify the node across runs, but it does not contain any personal information. When you opt-out, this UUID file is automatically removed to ensure complete privacy.\n\n---\n\n## 📦 Contributing to the Project\n\nBy enabling telemetry, you are helping the Kubo team improve the software for the entire community. The data is used to:\n\n- Prioritize feature development\n- Identify performance bottlenecks\n- Improve user experience\n\nYou can always disable telemetry at any time if you change your mind.\n\n---\n\n## 🧪 Testing Telemetry\n\nIf you're testing telemetry locally, you can change the endpoint by setting the `Endpoint` field in the config:\n\n```json\n{\n \"Plugins\": {\n \"Plugins\": {\n \"telemetry\": {\n \"Config\": {\n \"Mode\": \"on\",\n \"Endpoint\": \"http://localhost:8080\"\n }\n }\n }\n }\n}\n```\n\nThis allows you to capture and inspect telemetry data locally.\n\n---\n\n## 📦 Further Reading\n\nFor more information, see:\n- [IPFS Environment Variables](docs/environment-variables.md)\n- [IPFS Plugins](docs/plugins.md)\n- [IPFS Configuration](docs/config.md)\n" }, { "path": "docs/transports.md", "content": "## /ws and /wss -- websockets\n\nIf you want browsers to connect to e.g. `/dns4/example.com/tcp/443/wss/p2p/QmFoo`\n\n- [ ] An SSL cert matching the `/dns4` or `/dns6` name\n- [ ] Kubo listening on `/ip4/127.0.0.1/tcp/8081/ws`\n - 8081 is just an example\n - note that it's `/ws` here, not `/wss` -- Kubo can't currently do SSL, see the next point\n- [ ] nginx\n - configured with the SSL cert\n - listening on port 443\n - forwarding to 127.0.0.1:8081\n" }, { "path": "docs/windows.md", "content": "# Building on Windows\n![](https://ipfs.io/ipfs/QmccXW7JSZMVXidSc7tHsU6aktuaiV923q4yBGHUsdymYo/build.gif)\n\nIf you just want to install kubo, please download it from https://dist.ipfs.tech/#kubo. This document explains how to build it from source.\n\n## Install Go\n`kubo` is built on Golang and thus depends on it for all building methods. \nhttps://golang.org/doc/install \nThe `GOPATH` environment variable must be set as well. \nhttps://golang.org/doc/code.html#GOPATH\n\n## Choose the way you want to proceed\n`kubo` utilizes `make` to automate builds and run tests, but can be built without it using only `git` and `go`. \nNo matter which method you choose, if you encounter issues, please see the [Troubleshooting](#troubleshooting) section. \n\n**Using `make`:** \nMSYS2 and Cygwin provide the Unix tools we need to build `kubo`. You may use either, but if you don't already have one installed, we recommend MSYS2. \n[MSYS2→](#msys2) \n[Cygwin→](#cygwin) \n\n**Using build tools manually:** \nThis section assumes you have a working version of `go` and `git` already setup. You may want to build this way if your environment restricts installing additional software, or if you're integrating IPFS into your own build system. \n[Minimal→](#minimal) \n\n## MSYS2\n1. Install msys2 (http://www.msys2.org) \n2. Run the following inside a normal `cmd` prompt (Not the MSYS2 prompt, we only need MSYS2's tools). \nAn explanation of this block is below.\n```\nSET PATH=%PATH%;\\msys64\\usr\\bin\npacman --noconfirm -S git make unzip\ngo get -u github.com/ipfs/kubo\ncd %GOPATH%\\src\\github.com\\ipfs\\kubo\nmake install\n%GOPATH%\\bin\\ipfs.exe version --all\n```\n\nIf there were no errors, the final command should output version information similar to \"`ipfs version 0.4.14-dev-XXXXXXX`\" where \"XXXXXXX\" should match the current short-hash of the `kubo` repo. You can retrieve said hash via this command: `git rev-parse --short HEAD`. \nIf `ipfs.exe` executes and the version string matches, then building was successful.\n\n|Command|Explanation|\n| ---: | :--- |\n|`SET PATH=%PATH%;\\msys64\\usr\\bin` |Add msys2's tools to our [`PATH`](https://ss64.com/nt/path.html); Defaults to: (\\msys64\\usr\\bin)|\n|`pacman --noconfirm -S git make unzip` |Install `kubo` build dependencies|\n|`go get -u github.com/ipfs/kubo` |Fetch / Update `kubo` source|\n|`cd %GOPATH%\\src\\github.com\\ipfs\\kubo` |Change to `kubo` source directory|\n|`make install` |Build and install to `%GOPATH%\\bin\\ipfs.exe`|\n|`%GOPATH%\\bin\\ipfs.exe version --all` |Test the built binary|\n\nTo build again after making changes to the source, run:\n```\nSET PATH=%PATH%;\\msys64\\usr\\bin\ncd %GOPATH%\\src\\github.com\\ipfs\\kubo\nmake install\n```\n\n**Tip:** To avoid setting `PATH` every time (`SET PATH=%PATH%;\\msys64\\usr\\bin`), you can lock it in permanently using `setx` after it's been set once:\n```\nSETX PATH %PATH%\n```\n\n## Cygwin\n1. Install Cygwin (https://www.cygwin.com) \n2. During the install, select the following packages. (If you already have Cygwin installed, run the setup file again to install additional packages.) A fresh install should look something like [this reference image](https://ipfs.io/ipfs/QmaYFSQa4iHDafcebiKjm1WwuKhosoXr45HPpfaeMbCRpb/cygwin%20-%20install.png).\n - devel packages\n - `git`\n - `make`\n - archive packages\n - `unzip`\n - net packages\n - `curl` \n3. Run the following inside a normal `cmd` prompt (Not the Cygwin prompt, we only need Cygwin's tools) \nAn explanation of this block is below.\n```\nSET PATH=%PATH%;\\cygwin64\\bin\nmkdir %GOPATH%\\src\\github.com\\ipfs\ncd %GOPATH%\\src\\github.com\\ipfs\ngit clone https://github.com/ipfs/kubo.git\ncd %GOPATH%\\src\\github.com\\ipfs\\kubo\nmake install\n%GOPATH%\\bin\\ipfs.exe version --all\n```\n\nIf there were no errors, the final command should output version information similar to \"`ipfs version 0.4.14-dev-XXXXXXX`\" where \"XXXXXXX\" should match the current short-hash of the `kubo` repo. You can retrieve said hash via this command: `git rev-parse --short HEAD`. \nIf `ipfs.exe` executes and the version string matches, then building was successful.\n\n|Command|Explanation|\n| ---: | :--- |\n|`SET PATH=%PATH%;\\cygwin64\\bin` |Add Cygwin's tools to our [`PATH`](https://ss64.com/nt/path.html); Defaults to: (\\cygwin64\\bin)|\n|`mkdir %GOPATH%\\src\\github.com\\ipfs`
`cd %GOPATH%\\src\\github.com\\ipfs`
`git clone https://github.com/ipfs/kubo.git` |Fetch / Update `kubo` source|\n|`cd %GOPATH%\\src\\github.com\\ipfs\\kubo` |Change to `kubo` source directory|\n|`make install` |Build and install to `%GOPATH%\\bin\\ipfs.exe`|\n|`%GOPATH%\\bin\\ipfs.exe version --all` |Test the built binary|\n\nTo build again after making changes to the source, run:\n```\nSET PATH=%PATH%;\\cygwin64\\bin\ncd %GOPATH%\\src\\github.com\\ipfs\\kubo\nmake install\n```\n\n**Tip:** To avoid setting `PATH` every time (`SET PATH=%PATH%;\\cygwin64\\bin`), you can lock it in permanently using `setx` after it's been set once:\n```\nSETX PATH %PATH%\n```\n\n## Minimal\n\nWhile it's possible to build `kubo` with `go` alone, we'll be using `git` to fetch the source.\n\nYou can use whichever version of `git` you wish but we recommend the Windows builds at . `git` must be in your [`PATH`](https://ss64.com/nt/path.html) for `go get` to recognize and use it.\n\n### kubo\n\nClone and change directory to the source code, if you haven't already:\n\nCMD:\n```bat\ngit clone https://github.com/ipfs/kubo %GOPATH%/src/github.com/ipfs/kubo\ncd %GOPATH%/src/github.com/ipfs/kubo/cmd/ipfs\n```\n\nPowerShell:\n```powershell\ngit clone https://github.com/ipfs/kubo $env:GOPATH/src/github.com/ipfs/kubo\ncd $env:GOPATH/src/github.com/ipfs/kubo/cmd/ipfs\n```\n\nWe need the `git` commit hash to be included in our build so that in the extremely rare event a bug is found, we have a reference point later for tracking it. We'll ask `git` for it and store it in a variable. The syntax for the next command is different depending on whether you're using the interactive command line or writing a batch file. Use the one that applies to you. \n- interactive: `FOR /F %V IN ('git rev-parse --short HEAD') do set SHA=%V` \n- interpreter: `FOR /F %%V IN ('git rev-parse --short HEAD') do set SHA=%%V` \n\nFinally, we'll build and test `ipfs` itself.\n\nCMD:\n```bat\ngo install -ldflags=\"-X \"github.com/ipfs/kubo\".CurrentCommit=%SHA%\"\n%GOPATH%\\bin\\ipfs.exe version --all\n```\n\nPowerShell:\n```powershell\ngo install -ldflags=\"-X \"github.com/ipfs/kubo\".CurrentCommit=$env:SHA\"\ncp ./ipfs.exe $env:GOPATH/bin/ipfs.exe -force\n. $env:GOPATH/bin/ipfs.exe version --all\n```\nYou can check that the ipfs output versions match with `go version` and `git rev-parse --short HEAD`. \nIf `ipfs.exe` executes and everything matches, then building was successful.\n\n## Troubleshooting\n- **Git auth**\nIf you get authentication problems with Git, you might want to take a look at https://help.github.com/articles/caching-your-github-password-in-git/ and use the suggested solution: \n`git config --global credential.helper wincred`\n\n- **Anything else** \nPlease search [https://discuss.ipfs.tech](https://discuss.ipfs.tech/search?q=windows%20category%3A13) for any additional issues you may encounter. If you can't find any existing resolution, feel free to post a question asking for help.\n\nIf you encounter a bug with `kubo` itself (not related to building) please use the [issue tracker](https://github.com/ipfs/kubo/issues) to report it.\n" }, { "path": "fuse/ipns/common.go", "content": "package ipns\n\nimport (\n\t\"context\"\n\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/core\"\n\tci \"github.com/libp2p/go-libp2p/core/crypto\"\n)\n\n// InitializeKeyspace sets the ipns record for the given key to\n// point to an empty directory.\nfunc InitializeKeyspace(n *core.IpfsNode, key ci.PrivKey) error {\n\tctx, cancel := context.WithCancel(n.Context())\n\tdefer cancel()\n\n\temptyDir := ft.EmptyDirNode()\n\n\terr := n.Pinning.Pin(ctx, emptyDir, false, \"\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = n.Pinning.Flush(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tpub := namesys.NewIPNSPublisher(n.Routing, n.Repo.Datastore())\n\n\treturn pub.Publish(ctx, key, path.FromCid(emptyDir.Cid()))\n}\n" }, { "path": "fuse/ipns/ipns_test.go", "content": "//go:build !nofuse && !openbsd && !netbsd && !plan9\n\npackage ipns\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\tmrand \"math/rand\"\n\t\"os\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"bazil.org/fuse\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tcoreapi \"github.com/ipfs/kubo/core/coreapi\"\n\n\tfstest \"bazil.org/fuse/fs/fstestutil\"\n\tracedet \"github.com/ipfs/go-detect-race\"\n\t\"github.com/ipfs/go-test/random\"\n\tci \"github.com/libp2p/go-libp2p-testing/ci\"\n)\n\nfunc maybeSkipFuseTests(t *testing.T) {\n\tif ci.NoFuse() {\n\t\tt.Skip(\"Skipping FUSE tests\")\n\t}\n}\n\nfunc randBytes(size int) []byte {\n\tb := make([]byte, size)\n\t_, err := io.ReadFull(random.NewRand(), b)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn b\n}\n\nfunc mkdir(t *testing.T, path string) {\n\terr := os.Mkdir(path, os.ModeDir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc writeFileOrFail(t *testing.T, size int, path string) []byte {\n\tdata, err := writeFile(size, path)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn data\n}\n\nfunc writeFile(size int, path string) ([]byte, error) {\n\tdata := randBytes(size)\n\terr := os.WriteFile(path, data, 0o666)\n\treturn data, err\n}\n\nfunc verifyFile(t *testing.T, path string, wantData []byte) {\n\tisData, err := os.ReadFile(path)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(isData) != len(wantData) {\n\t\tt.Fatal(\"Data not equal - length check failed\")\n\t}\n\tif !bytes.Equal(isData, wantData) {\n\t\tt.Fatal(\"Data not equal\")\n\t}\n}\n\nfunc checkExists(t *testing.T, path string) {\n\t_, err := os.Stat(path)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc closeMount(mnt *mountWrap) {\n\tif err := recover(); err != nil {\n\t\tlog.Error(\"Recovered panic\")\n\t\tlog.Error(err)\n\t}\n\tmnt.Close()\n}\n\ntype mountWrap struct {\n\t*fstest.Mount\n\tFs *FileSystem\n}\n\nfunc (m *mountWrap) Close() error {\n\tm.Fs.Destroy()\n\tm.Mount.Close()\n\treturn nil\n}\n\nfunc setupIpnsTest(t *testing.T, node *core.IpfsNode) (*core.IpfsNode, *mountWrap) {\n\tt.Helper()\n\tmaybeSkipFuseTests(t)\n\n\tvar err error\n\tif node == nil {\n\t\tnode, err = core.NewNode(context.Background(), &core.BuildCfg{})\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\terr = InitializeKeyspace(node, node.PrivateKey)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tcoreAPI, err := coreapi.NewCoreAPI(node)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfs, err := NewFileSystem(node.Context(), coreAPI, \"\", \"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tmnt, err := fstest.MountedT(t, fs, nil)\n\tif err == fuse.ErrOSXFUSENotFound {\n\t\tt.Skip(err)\n\t}\n\tif err != nil {\n\t\tt.Fatalf(\"error mounting at temporary directory: %v\", err)\n\t}\n\n\treturn node, &mountWrap{\n\t\tMount: mnt,\n\t\tFs: fs,\n\t}\n}\n\nfunc TestIpnsLocalLink(t *testing.T) {\n\tnd, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\tname := mnt.Dir + \"/local\"\n\n\tcheckExists(t, name)\n\n\tlinksto, err := os.Readlink(name)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif linksto != nd.Identity.String() {\n\t\tt.Fatal(\"Link invalid\")\n\t}\n}\n\n// Test writing a file and reading it back.\nfunc TestIpnsBasicIO(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\tnd, mnt := setupIpnsTest(t, nil)\n\tdefer closeMount(mnt)\n\n\tfname := mnt.Dir + \"/local/testfile\"\n\tdata := writeFileOrFail(t, 10, fname)\n\n\trbuf, err := os.ReadFile(fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatal(\"Incorrect Read!\")\n\t}\n\n\tfname2 := mnt.Dir + \"/\" + nd.Identity.String() + \"/testfile\"\n\trbuf, err = os.ReadFile(fname2)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatal(\"Incorrect Read!\")\n\t}\n}\n\n// Test to make sure file changes persist over mounts of ipns.\nfunc TestFilePersistence(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\tnode, mnt := setupIpnsTest(t, nil)\n\n\tfname := \"/local/atestfile\"\n\tdata := writeFileOrFail(t, 127, mnt.Dir+fname)\n\n\tmnt.Close()\n\n\tt.Log(\"Closed, opening new fs\")\n\t_, mnt = setupIpnsTest(t, node)\n\tdefer mnt.Close()\n\n\trbuf, err := os.ReadFile(mnt.Dir + fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatalf(\"File data changed between mounts! sizes differ: %d != %d\", len(data), len(rbuf))\n\t}\n}\n\nfunc TestMultipleDirs(t *testing.T) {\n\tnode, mnt := setupIpnsTest(t, nil)\n\n\tt.Log(\"make a top level dir\")\n\tdir1 := \"/local/test1\"\n\tmkdir(t, mnt.Dir+dir1)\n\n\tcheckExists(t, mnt.Dir+dir1)\n\n\tt.Log(\"write a file in it\")\n\tdata1 := writeFileOrFail(t, 4000, mnt.Dir+dir1+\"/file1\")\n\n\tverifyFile(t, mnt.Dir+dir1+\"/file1\", data1)\n\n\tt.Log(\"sub directory\")\n\tmkdir(t, mnt.Dir+dir1+\"/dir2\")\n\n\tcheckExists(t, mnt.Dir+dir1+\"/dir2\")\n\n\tt.Log(\"file in that subdirectory\")\n\tdata2 := writeFileOrFail(t, 5000, mnt.Dir+dir1+\"/dir2/file2\")\n\n\tverifyFile(t, mnt.Dir+dir1+\"/dir2/file2\", data2)\n\n\tmnt.Close()\n\tt.Log(\"closing mount, then restarting\")\n\n\t_, mnt = setupIpnsTest(t, node)\n\n\tcheckExists(t, mnt.Dir+dir1)\n\n\tverifyFile(t, mnt.Dir+dir1+\"/file1\", data1)\n\n\tverifyFile(t, mnt.Dir+dir1+\"/dir2/file2\", data2)\n\tmnt.Close()\n}\n\n// Test to make sure the filesystem reports file sizes correctly.\nfunc TestFileSizeReporting(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\t_, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\n\tfname := mnt.Dir + \"/local/sizecheck\"\n\tdata := writeFileOrFail(t, 5555, fname)\n\n\tfinfo, err := os.Stat(fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif finfo.Size() != int64(len(data)) {\n\t\tt.Fatal(\"Read incorrect size from stat!\")\n\t}\n}\n\n// Test to make sure you can't create multiple entries with the same name.\nfunc TestDoubleEntryFailure(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\t_, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\n\tdname := mnt.Dir + \"/local/thisisadir\"\n\terr := os.Mkdir(dname, 0o777)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = os.Mkdir(dname, 0o777)\n\tif err == nil {\n\t\tt.Fatal(\"Should have gotten error one creating new directory.\")\n\t}\n}\n\nfunc TestAppendFile(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\t_, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\n\tfname := mnt.Dir + \"/local/file\"\n\tdata := writeFileOrFail(t, 1300, fname)\n\n\tfi, err := os.OpenFile(fname, os.O_RDWR|os.O_APPEND, 0o666)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnudata := randBytes(500)\n\n\tn, err := fi.Write(nudata)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\terr = fi.Close()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif n != len(nudata) {\n\t\tt.Fatal(\"Failed to write enough bytes.\")\n\t}\n\n\tdata = append(data, nudata...)\n\n\trbuf, err := os.ReadFile(fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatal(\"Data inconsistent!\")\n\t}\n}\n\nfunc TestConcurrentWrites(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\t_, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\n\tnactors := 4\n\tfilesPerActor := 400\n\tfileSize := 2000\n\n\tdata := make([][][]byte, nactors)\n\n\tif racedet.WithRace() {\n\t\tnactors = 2\n\t\tfilesPerActor = 50\n\t}\n\n\twg := sync.WaitGroup{}\n\tfor i := 0; i < nactors; i++ {\n\t\tdata[i] = make([][]byte, filesPerActor)\n\t\twg.Add(1)\n\t\tgo func(n int) {\n\t\t\tdefer wg.Done()\n\t\t\tfor j := 0; j < filesPerActor; j++ {\n\t\t\t\tout, err := writeFile(fileSize, mnt.Dir+fmt.Sprintf(\"/local/%dFILE%d\", n, j))\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Error(err)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tdata[n][j] = out\n\t\t\t}\n\t\t}(i)\n\t}\n\twg.Wait()\n\n\tfor i := 0; i < nactors; i++ {\n\t\tfor j := 0; j < filesPerActor; j++ {\n\t\t\tif data[i][j] == nil {\n\t\t\t\t// Error already reported.\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tverifyFile(t, mnt.Dir+fmt.Sprintf(\"/local/%dFILE%d\", i, j), data[i][j])\n\t\t}\n\t}\n}\n\nfunc TestFSThrash(t *testing.T) {\n\tfiles := make(map[string][]byte)\n\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\t_, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\n\tbase := mnt.Dir + \"/local\"\n\tdirs := []string{base}\n\tdirlock := sync.RWMutex{}\n\tfilelock := sync.Mutex{}\n\n\tndirWorkers := 2\n\tnfileWorkers := 2\n\n\tndirs := 100\n\tnfiles := 200\n\n\twg := sync.WaitGroup{}\n\n\t// Spawn off workers to make directories\n\tfor i := range ndirWorkers {\n\t\twg.Add(1)\n\t\tgo func(worker int) {\n\t\t\tdefer wg.Done()\n\t\t\tfor j := range ndirs {\n\t\t\t\tdirlock.RLock()\n\t\t\t\tn := mrand.Intn(len(dirs))\n\t\t\t\tdir := dirs[n]\n\t\t\t\tdirlock.RUnlock()\n\n\t\t\t\tnewDir := fmt.Sprintf(\"%s/dir%d-%d\", dir, worker, j)\n\t\t\t\terr := os.Mkdir(newDir, os.ModeDir)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Error(err)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tdirlock.Lock()\n\t\t\t\tdirs = append(dirs, newDir)\n\t\t\t\tdirlock.Unlock()\n\t\t\t}\n\t\t}(i)\n\t}\n\n\t// Spawn off workers to make files\n\tfor i := range nfileWorkers {\n\t\twg.Add(1)\n\t\tgo func(worker int) {\n\t\t\tdefer wg.Done()\n\t\t\tfor j := range nfiles {\n\t\t\t\tdirlock.RLock()\n\t\t\t\tn := mrand.Intn(len(dirs))\n\t\t\t\tdir := dirs[n]\n\t\t\t\tdirlock.RUnlock()\n\n\t\t\t\tnewFileName := fmt.Sprintf(\"%s/file%d-%d\", dir, worker, j)\n\n\t\t\t\tdata, err := writeFile(2000+mrand.Intn(5000), newFileName)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Error(err)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tfilelock.Lock()\n\t\t\t\tfiles[newFileName] = data\n\t\t\t\tfilelock.Unlock()\n\t\t\t}\n\t\t}(i)\n\t}\n\n\twg.Wait()\n\tfor name, data := range files {\n\t\tout, err := os.ReadFile(name)\n\t\tif err != nil {\n\t\t\tt.Error(err)\n\t\t}\n\n\t\tif !bytes.Equal(data, out) {\n\t\t\tt.Errorf(\"Data didn't match in %s: expected %v, got %v\", name, data, out)\n\t\t}\n\t}\n}\n\n// Test writing a medium sized file one byte at a time.\nfunc TestMultiWrite(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\n\t_, mnt := setupIpnsTest(t, nil)\n\tdefer mnt.Close()\n\n\tfpath := mnt.Dir + \"/local/file\"\n\tfi, err := os.Create(fpath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdata := randBytes(1001)\n\tfor i := range data {\n\t\tn, err := fi.Write(data[i : i+1])\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif n != 1 {\n\t\t\tt.Fatal(\"Somehow wrote the wrong number of bytes! (n != 1)\")\n\t\t}\n\t}\n\tfi.Close()\n\n\trbuf, err := os.ReadFile(fpath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatal(\"File on disk did not match bytes written\")\n\t}\n}\n" }, { "path": "fuse/ipns/ipns_unix.go", "content": "//go:build !nofuse && !openbsd && !netbsd && !plan9\n\n// package fuse/ipns implements a fuse filesystem that interfaces\n// with ipns, the naming system for ipfs.\npackage ipns\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"strings\"\n\t\"syscall\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/path\"\n\n\tfuse \"bazil.org/fuse\"\n\tfs \"bazil.org/fuse/fs\"\n\tmfs \"github.com/ipfs/boxo/mfs\"\n\tcid \"github.com/ipfs/go-cid\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\toptions \"github.com/ipfs/kubo/core/coreiface/options\"\n)\n\nfunc init() {\n\tif os.Getenv(\"IPFS_FUSE_DEBUG\") != \"\" {\n\t\tfuse.Debug = func(msg any) {\n\t\t\tfmt.Println(msg)\n\t\t}\n\t}\n}\n\nvar log = logging.Logger(\"fuse/ipns\")\n\n// FileSystem is the readwrite IPNS Fuse Filesystem.\ntype FileSystem struct {\n\tIpfs iface.CoreAPI\n\tRootNode *Root\n}\n\n// NewFileSystem constructs new fs using given core.IpfsNode instance.\nfunc NewFileSystem(ctx context.Context, ipfs iface.CoreAPI, ipfspath, ipnspath string) (*FileSystem, error) {\n\tkey, err := ipfs.Key().Self(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\troot, err := CreateRoot(ctx, ipfs, map[string]iface.Key{\"local\": key}, ipfspath, ipnspath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &FileSystem{Ipfs: ipfs, RootNode: root}, nil\n}\n\n// Root constructs the Root of the filesystem, a Root object.\nfunc (f *FileSystem) Root() (fs.Node, error) {\n\tlog.Debug(\"filesystem, get root\")\n\treturn f.RootNode, nil\n}\n\nfunc (f *FileSystem) Destroy() {\n\terr := f.RootNode.Close()\n\tif err != nil {\n\t\tlog.Errorf(\"Error Shutting Down Filesystem: %s\\n\", err)\n\t}\n}\n\n// Root is the root object of the filesystem tree.\ntype Root struct {\n\tIpfs iface.CoreAPI\n\tKeys map[string]iface.Key\n\n\t// Used for symlinking into ipfs\n\tIpfsRoot string\n\tIpnsRoot string\n\tLocalDirs map[string]fs.Node\n\tRoots map[string]*mfs.Root\n\n\tLocalLinks map[string]*Link\n}\n\nfunc ipnsPubFunc(ipfs iface.CoreAPI, key iface.Key) mfs.PubFunc {\n\treturn func(ctx context.Context, c cid.Cid) error {\n\t\t_, err := ipfs.Name().Publish(ctx, path.FromCid(c), options.Name.Key(key.Name()))\n\t\treturn err\n\t}\n}\n\nfunc loadRoot(ctx context.Context, ipfs iface.CoreAPI, key iface.Key) (*mfs.Root, fs.Node, error) {\n\tnode, err := ipfs.ResolveNode(ctx, key.Path())\n\tswitch err {\n\tcase nil:\n\tcase namesys.ErrResolveFailed:\n\t\tnode = ft.EmptyDirNode()\n\tdefault:\n\t\tlog.Errorf(\"looking up %s: %s\", key.Path(), err)\n\t\treturn nil, nil, err\n\t}\n\n\tpbnode, ok := node.(*dag.ProtoNode)\n\tif !ok {\n\t\treturn nil, nil, dag.ErrNotProtobuf\n\t}\n\n\t// We have no access to provider.System from the CoreAPI. The Routing\n\t// part offers Provide through the router so it may be slow/risky\n\t// to give that here to MFS. Therefore we leave as nil.\n\troot, err := mfs.NewRoot(ctx, ipfs.Dag(), pbnode, ipnsPubFunc(ipfs, key), nil)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\treturn root, &Directory{dir: root.GetDirectory()}, nil\n}\n\nfunc CreateRoot(ctx context.Context, ipfs iface.CoreAPI, keys map[string]iface.Key, ipfspath, ipnspath string) (*Root, error) {\n\tldirs := make(map[string]fs.Node)\n\troots := make(map[string]*mfs.Root)\n\tlinks := make(map[string]*Link)\n\tfor alias, k := range keys {\n\t\troot, fsn, err := loadRoot(ctx, ipfs, k)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tname := k.ID().String()\n\n\t\troots[name] = root\n\t\tldirs[name] = fsn\n\n\t\t// set up alias symlink\n\t\tlinks[alias] = &Link{\n\t\t\tTarget: name,\n\t\t}\n\t}\n\n\treturn &Root{\n\t\tIpfs: ipfs,\n\t\tIpfsRoot: ipfspath,\n\t\tIpnsRoot: ipnspath,\n\t\tKeys: keys,\n\t\tLocalDirs: ldirs,\n\t\tLocalLinks: links,\n\t\tRoots: roots,\n\t}, nil\n}\n\n// Attr returns file attributes.\nfunc (r *Root) Attr(ctx context.Context, a *fuse.Attr) error {\n\tlog.Debug(\"Root Attr\")\n\ta.Mode = os.ModeDir | 0o111 // -rw+x\n\treturn nil\n}\n\n// Lookup performs a lookup under this node.\nfunc (r *Root) Lookup(ctx context.Context, name string) (fs.Node, error) {\n\tswitch name {\n\tcase \"mach_kernel\", \".hidden\", \"._.\":\n\t\t// Just quiet some log noise on OS X.\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tif lnk, ok := r.LocalLinks[name]; ok {\n\t\treturn lnk, nil\n\t}\n\n\tnd, ok := r.LocalDirs[name]\n\tif ok {\n\t\tswitch nd := nd.(type) {\n\t\tcase *Directory:\n\t\t\treturn nd, nil\n\t\tcase *FileNode:\n\t\t\treturn nd, nil\n\t\tdefault:\n\t\t\treturn nil, syscall.Errno(syscall.EIO)\n\t\t}\n\t}\n\n\t// other links go through ipns resolution and are symlinked into the ipfs mountpoint\n\tipnsName := \"/ipns/\" + name\n\tresolved, err := r.Ipfs.Name().Resolve(ctx, ipnsName)\n\tif err != nil {\n\t\tlog.Warnf(\"ipns: namesys resolve error: %s\", err)\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tif resolved.Namespace() != path.IPFSNamespace {\n\t\treturn nil, errors.New(\"invalid path from ipns record\")\n\t}\n\n\treturn &Link{r.IpfsRoot + \"/\" + strings.TrimPrefix(resolved.String(), \"/ipfs/\")}, nil\n}\n\nfunc (r *Root) Close() error {\n\tfor _, mr := range r.Roots {\n\t\terr := mr.Close()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\n// Forget is called when the filesystem is unmounted. probably.\n// see comments here: http://godoc.org/bazil.org/fuse/fs#FSDestroyer\nfunc (r *Root) Forget() {\n\terr := r.Close()\n\tif err != nil {\n\t\tlog.Error(err)\n\t}\n}\n\n// ReadDirAll reads a particular directory. Will show locally available keys\n// as well as a symlink to the peerID key.\nfunc (r *Root) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {\n\tlog.Debug(\"Root ReadDirAll\")\n\n\tlisting := make([]fuse.Dirent, 0, len(r.Keys)*2)\n\tfor alias, k := range r.Keys {\n\t\tent := fuse.Dirent{\n\t\t\tName: k.ID().String(),\n\t\t\tType: fuse.DT_Dir,\n\t\t}\n\t\tlink := fuse.Dirent{\n\t\t\tName: alias,\n\t\t\tType: fuse.DT_Link,\n\t\t}\n\t\tlisting = append(listing, ent, link)\n\t}\n\treturn listing, nil\n}\n\n// Directory is wrapper over an mfs directory to satisfy the fuse fs interface.\ntype Directory struct {\n\tdir *mfs.Directory\n}\n\ntype FileNode struct {\n\tfi *mfs.File\n}\n\n// File is wrapper over an mfs file to satisfy the fuse fs interface.\ntype File struct {\n\tfi mfs.FileDescriptor\n}\n\n// Attr returns the attributes of a given node.\nfunc (d *Directory) Attr(ctx context.Context, a *fuse.Attr) error {\n\tlog.Debug(\"Directory Attr\")\n\ta.Mode = os.ModeDir | 0o555\n\ta.Uid = uint32(os.Getuid())\n\ta.Gid = uint32(os.Getgid())\n\treturn nil\n}\n\n// Attr returns the attributes of a given node.\nfunc (fi *FileNode) Attr(ctx context.Context, a *fuse.Attr) error {\n\tlog.Debug(\"File Attr\")\n\tsize, err := fi.fi.Size()\n\tif err != nil {\n\t\t// In this case, the dag node in question may not be unixfs\n\t\treturn fmt.Errorf(\"fuse/ipns: failed to get file.Size(): %s\", err)\n\t}\n\ta.Mode = os.FileMode(0o666)\n\ta.Size = uint64(size)\n\ta.Uid = uint32(os.Getuid())\n\ta.Gid = uint32(os.Getgid())\n\treturn nil\n}\n\n// Lookup performs a lookup under this node.\nfunc (d *Directory) Lookup(ctx context.Context, name string) (fs.Node, error) {\n\tchild, err := d.dir.Child(name)\n\tif err != nil {\n\t\t// todo: make this error more versatile.\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tswitch child := child.(type) {\n\tcase *mfs.Directory:\n\t\treturn &Directory{dir: child}, nil\n\tcase *mfs.File:\n\t\treturn &FileNode{fi: child}, nil\n\tdefault:\n\t\t// NB: if this happens, we do not want to continue, unpredictable behaviour\n\t\t// may occur.\n\t\tpanic(\"invalid type found under directory. programmer error.\")\n\t}\n}\n\n// ReadDirAll reads the link structure as directory entries.\nfunc (d *Directory) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {\n\tlisting, err := d.dir.List(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tentries := make([]fuse.Dirent, len(listing))\n\tfor i, entry := range listing {\n\t\tdirent := fuse.Dirent{Name: entry.Name}\n\n\t\tswitch mfs.NodeType(entry.Type) {\n\t\tcase mfs.TDir:\n\t\t\tdirent.Type = fuse.DT_Dir\n\t\tcase mfs.TFile:\n\t\t\tdirent.Type = fuse.DT_File\n\t\t}\n\n\t\tentries[i] = dirent\n\t}\n\n\tif len(entries) > 0 {\n\t\treturn entries, nil\n\t}\n\treturn nil, syscall.Errno(syscall.ENOENT)\n}\n\nfunc (fi *File) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {\n\t_, err := fi.fi.Seek(req.Offset, io.SeekStart)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfisize, err := fi.fi.Size()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tselect {\n\tcase <-ctx.Done():\n\t\treturn ctx.Err()\n\tdefault:\n\t}\n\n\treadsize := min(req.Size, int(fisize-req.Offset))\n\tn, err := fi.fi.CtxReadFull(ctx, resp.Data[:readsize])\n\tresp.Data = resp.Data[:n]\n\treturn err\n}\n\nfunc (fi *File) Write(ctx context.Context, req *fuse.WriteRequest, resp *fuse.WriteResponse) error {\n\t// TODO: at some point, ensure that WriteAt here respects the context\n\twrote, err := fi.fi.WriteAt(req.Data, req.Offset)\n\tif err != nil {\n\t\treturn err\n\t}\n\tresp.Size = wrote\n\treturn nil\n}\n\nfunc (fi *File) Flush(ctx context.Context, req *fuse.FlushRequest) error {\n\terrs := make(chan error, 1)\n\tgo func() {\n\t\terrs <- fi.fi.Flush()\n\t}()\n\tselect {\n\tcase err := <-errs:\n\t\treturn err\n\tcase <-ctx.Done():\n\t\treturn ctx.Err()\n\t}\n}\n\nfunc (fi *File) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fuse.SetattrResponse) error {\n\tif req.Valid.Size() {\n\t\tcursize, err := fi.fi.Size()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif cursize != int64(req.Size) {\n\t\t\terr := fi.fi.Truncate(int64(req.Size))\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}\n\n// Fsync flushes the content in the file to disk.\nfunc (fi *FileNode) Fsync(ctx context.Context, req *fuse.FsyncRequest) error {\n\t// This needs to perform a *full* flush because, in MFS, a write isn't\n\t// persisted until the root is updated.\n\terrs := make(chan error, 1)\n\tgo func() {\n\t\terrs <- fi.fi.Flush()\n\t}()\n\tselect {\n\tcase err := <-errs:\n\t\treturn err\n\tcase <-ctx.Done():\n\t\treturn ctx.Err()\n\t}\n}\n\nfunc (fi *File) Forget() {\n\t// TODO(steb): this seems like a place where we should be *uncaching*, not flushing.\n\terr := fi.fi.Flush()\n\tif err != nil {\n\t\tlog.Debug(\"forget file error: \", err)\n\t}\n}\n\nfunc (d *Directory) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) {\n\tchild, err := d.dir.Mkdir(req.Name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &Directory{dir: child}, nil\n}\n\nfunc (fi *FileNode) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.OpenResponse) (fs.Handle, error) {\n\tfd, err := fi.fi.Open(mfs.Flags{\n\t\tRead: req.Flags.IsReadOnly() || req.Flags.IsReadWrite(),\n\t\tWrite: req.Flags.IsWriteOnly() || req.Flags.IsReadWrite(),\n\t\tSync: true,\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif req.Flags&fuse.OpenTruncate != 0 {\n\t\tif req.Flags.IsReadOnly() {\n\t\t\tlog.Error(\"tried to open a readonly file with truncate\")\n\t\t\treturn nil, syscall.Errno(syscall.ENOTSUP)\n\t\t}\n\t\tlog.Info(\"Need to truncate file!\")\n\t\terr := fd.Truncate(0)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t} else if req.Flags&fuse.OpenAppend != 0 {\n\t\tlog.Info(\"Need to append to file!\")\n\t\tif req.Flags.IsReadOnly() {\n\t\t\tlog.Error(\"tried to open a readonly file with append\")\n\t\t\treturn nil, syscall.Errno(syscall.ENOTSUP)\n\t\t}\n\n\t\t_, err := fd.Seek(0, io.SeekEnd)\n\t\tif err != nil {\n\t\t\tlog.Error(\"seek reset failed: \", err)\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn &File{fi: fd}, nil\n}\n\nfunc (fi *File) Release(ctx context.Context, req *fuse.ReleaseRequest) error {\n\treturn fi.fi.Close()\n}\n\nfunc (d *Directory) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error) {\n\t// New 'empty' file\n\tnd := dag.NodeWithData(ft.FilePBData(nil, 0))\n\terr := d.dir.AddChild(req.Name, nd)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tchild, err := d.dir.Child(req.Name)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tfi, ok := child.(*mfs.File)\n\tif !ok {\n\t\treturn nil, nil, errors.New(\"child creation failed\")\n\t}\n\n\tnodechild := &FileNode{fi: fi}\n\n\tfd, err := fi.Open(mfs.Flags{\n\t\tRead: req.Flags.IsReadOnly() || req.Flags.IsReadWrite(),\n\t\tWrite: req.Flags.IsWriteOnly() || req.Flags.IsReadWrite(),\n\t\tSync: true,\n\t})\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\treturn nodechild, &File{fi: fd}, nil\n}\n\nfunc (d *Directory) Remove(ctx context.Context, req *fuse.RemoveRequest) error {\n\terr := d.dir.Unlink(req.Name)\n\tif err != nil {\n\t\treturn syscall.Errno(syscall.ENOENT)\n\t}\n\treturn nil\n}\n\n// Rename implements NodeRenamer.\nfunc (d *Directory) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error {\n\tcur, err := d.dir.Child(req.OldName)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = d.dir.Unlink(req.OldName)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tswitch newDir := newDir.(type) {\n\tcase *Directory:\n\t\tnd, err := cur.GetNode()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\terr = newDir.dir.AddChild(req.NewName, nd)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\tcase *FileNode:\n\t\tlog.Error(\"Cannot move node into a file!\")\n\t\treturn syscall.Errno(syscall.EPERM)\n\tdefault:\n\t\tlog.Error(\"Unknown node type for rename target dir!\")\n\t\treturn errors.New(\"unknown fs node type\")\n\t}\n\treturn nil\n}\n\n// to check that out Node implements all the interfaces we want.\ntype ipnsRoot interface {\n\tfs.Node\n\tfs.HandleReadDirAller\n\tfs.NodeStringLookuper\n}\n\nvar _ ipnsRoot = (*Root)(nil)\n\ntype ipnsDirectory interface {\n\tfs.HandleReadDirAller\n\tfs.Node\n\tfs.NodeCreater\n\tfs.NodeMkdirer\n\tfs.NodeRemover\n\tfs.NodeRenamer\n\tfs.NodeStringLookuper\n}\n\nvar _ ipnsDirectory = (*Directory)(nil)\n\ntype ipnsFile interface {\n\tfs.HandleFlusher\n\tfs.HandleReader\n\tfs.HandleWriter\n\tfs.HandleReleaser\n}\n\ntype ipnsFileNode interface {\n\tfs.Node\n\tfs.NodeFsyncer\n\tfs.NodeOpener\n}\n\nvar (\n\t_ ipnsFileNode = (*FileNode)(nil)\n\t_ ipnsFile = (*File)(nil)\n)\n" }, { "path": "fuse/ipns/link_unix.go", "content": "//go:build !nofuse && !openbsd && !netbsd && !plan9\n\npackage ipns\n\nimport (\n\t\"context\"\n\t\"os\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n)\n\ntype Link struct {\n\tTarget string\n}\n\nfunc (l *Link) Attr(ctx context.Context, a *fuse.Attr) error {\n\tlog.Debug(\"Link attr.\")\n\ta.Mode = os.ModeSymlink | 0o555\n\treturn nil\n}\n\nfunc (l *Link) Readlink(ctx context.Context, req *fuse.ReadlinkRequest) (string, error) {\n\tlog.Debugf(\"ReadLink: %s\", l.Target)\n\treturn l.Target, nil\n}\n\nvar _ fs.NodeReadlinker = (*Link)(nil)\n" }, { "path": "fuse/ipns/mount_unix.go", "content": "//go:build (linux || darwin || freebsd || netbsd || openbsd) && !nofuse\n\npackage ipns\n\nimport (\n\tcore \"github.com/ipfs/kubo/core\"\n\tcoreapi \"github.com/ipfs/kubo/core/coreapi\"\n\tmount \"github.com/ipfs/kubo/fuse/mount\"\n)\n\n// Mount mounts ipns at a given location, and returns a mount.Mount instance.\nfunc Mount(ipfs *core.IpfsNode, ipnsmp, ipfsmp string) (mount.Mount, error) {\n\tcoreAPI, err := coreapi.NewCoreAPI(ipfs)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcfg, err := ipfs.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tallowOther := cfg.Mounts.FuseAllowOther\n\n\tfsys, err := NewFileSystem(ipfs.Context(), coreAPI, ipfsmp, ipnsmp)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn mount.NewMount(fsys, ipnsmp, allowOther)\n}\n" }, { "path": "fuse/mfs/mfs_test.go", "content": "//go:build !nofuse && !openbsd && !netbsd && !plan9\n\npackage mfs\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/rand\"\n\t\"errors\"\n\tiofs \"io/fs\"\n\t\"os\"\n\t\"slices\"\n\t\"strconv\"\n\t\"testing\"\n\t\"time\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n\t\"bazil.org/fuse/fs/fstestutil\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/node\"\n\t\"github.com/libp2p/go-libp2p-testing/ci\"\n)\n\n// Create an Ipfs.Node, a filesystem and a mount point.\nfunc setUp(t *testing.T, ipfs *core.IpfsNode) (fs.FS, *fstestutil.Mount) {\n\tif ci.NoFuse() {\n\t\tt.Skip(\"Skipping FUSE tests\")\n\t}\n\n\tif ipfs == nil {\n\t\tvar err error\n\t\tipfs, err = core.NewNode(context.Background(), &node.BuildCfg{})\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tfs := NewFileSystem(ipfs)\n\tmnt, err := fstestutil.MountedT(t, fs, nil)\n\tif err == fuse.ErrOSXFUSENotFound {\n\t\tt.Skip(err)\n\t}\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treturn fs, mnt\n}\n\n// Test reading and writing a file.\nfunc TestReadWrite(t *testing.T) {\n\t_, mnt := setUp(t, nil)\n\tdefer mnt.Close()\n\n\tpath := mnt.Dir + \"/testrw\"\n\tcontent := make([]byte, 8196)\n\t_, err := rand.Read(content)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tt.Run(\"write\", func(t *testing.T) {\n\t\tf, err := os.Create(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer f.Close()\n\n\t\t_, err = f.Write(content)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\tt.Run(\"read\", func(t *testing.T) {\n\t\tf, err := os.Open(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer f.Close()\n\n\t\tbuf := make([]byte, 8196)\n\t\tl, err := f.Read(buf)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif bytes.Equal(content, buf[:l]) != true {\n\t\t\tt.Fatal(\"read and write not equal\")\n\t\t}\n\t})\n}\n\n// Test creating a directory.\nfunc TestMkdir(t *testing.T) {\n\t_, mnt := setUp(t, nil)\n\tdefer mnt.Close()\n\n\tpath := mnt.Dir + \"/foo/bar/baz/qux/quux\"\n\n\tt.Run(\"write\", func(t *testing.T) {\n\t\terr := os.MkdirAll(path, iofs.ModeDir)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\tt.Run(\"read\", func(t *testing.T) {\n\t\tstat, err := os.Stat(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !stat.IsDir() {\n\t\t\tt.Fatal(\"not dir\")\n\t\t}\n\t})\n}\n\n// Test file persistence across mounts.\nfunc TestPersistence(t *testing.T) {\n\tipfs, err := core.NewNode(context.Background(), &node.BuildCfg{})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tcontent := make([]byte, 8196)\n\t_, err = rand.Read(content)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tt.Run(\"write\", func(t *testing.T) {\n\t\t_, mnt := setUp(t, ipfs)\n\t\tdefer mnt.Close()\n\t\tpath := mnt.Dir + \"/testpersistence\"\n\n\t\tf, err := os.Create(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer f.Close()\n\n\t\t_, err = f.Write(content)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\tt.Run(\"read\", func(t *testing.T) {\n\t\t_, mnt := setUp(t, ipfs)\n\t\tdefer mnt.Close()\n\t\tpath := mnt.Dir + \"/testpersistence\"\n\n\t\tf, err := os.Open(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer f.Close()\n\n\t\tbuf := make([]byte, 8196)\n\t\tl, err := f.Read(buf)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif bytes.Equal(content, buf[:l]) != true {\n\t\t\tt.Fatal(\"read and write not equal\")\n\t\t}\n\t})\n}\n\n// Test getting the file attributes.\nfunc TestAttr(t *testing.T) {\n\t_, mnt := setUp(t, nil)\n\tdefer mnt.Close()\n\n\tpath := mnt.Dir + \"/testattr\"\n\tcontent := make([]byte, 8196)\n\t_, err := rand.Read(content)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tt.Run(\"write\", func(t *testing.T) {\n\t\tf, err := os.Create(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer f.Close()\n\n\t\t_, err = f.Write(content)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\tt.Run(\"read\", func(t *testing.T) {\n\t\tfi, err := os.Stat(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif fi.IsDir() {\n\t\t\tt.Fatal(\"file is a directory\")\n\t\t}\n\n\t\tif fi.ModTime().After(time.Now()) {\n\t\t\tt.Fatal(\"future modtime\")\n\t\t}\n\t\tif time.Since(fi.ModTime()) > time.Second {\n\t\t\tt.Fatal(\"past modtime\")\n\t\t}\n\n\t\tif fi.Name() != \"testattr\" {\n\t\t\tt.Fatal(\"invalid filename\")\n\t\t}\n\n\t\tif fi.Size() != 8196 {\n\t\t\tt.Fatal(\"invalid size\")\n\t\t}\n\t})\n}\n\n// Test concurrent access to the filesystem.\nfunc TestConcurrentRW(t *testing.T) {\n\t_, mnt := setUp(t, nil)\n\tdefer mnt.Close()\n\n\tfiles := 5\n\tfileWorkers := 5\n\n\tpath := mnt.Dir + \"/testconcurrent\"\n\tcontent := make([][]byte, files)\n\n\tfor i := range content {\n\t\tcontent[i] = make([]byte, 8196)\n\t\t_, err := rand.Read(content[i])\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tt.Run(\"write\", func(t *testing.T) {\n\t\terrs := make(chan (error), 1)\n\t\tfor i := range files {\n\t\t\tgo func() {\n\t\t\t\tvar err error\n\t\t\t\tdefer func() { errs <- err }()\n\n\t\t\t\tf, err := os.Create(path + strconv.Itoa(i))\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tdefer f.Close()\n\n\t\t\t\t_, err = f.Write(content[i])\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\tfor range files {\n\t\t\terr := <-errs\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t})\n\tt.Run(\"read\", func(t *testing.T) {\n\t\terrs := make(chan (error), 1)\n\t\tfor i := 0; i < files*fileWorkers; i++ {\n\t\t\tgo func() {\n\t\t\t\tvar err error\n\t\t\t\tdefer func() { errs <- err }()\n\n\t\t\t\tf, err := os.Open(path + strconv.Itoa(i/fileWorkers))\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tdefer f.Close()\n\n\t\t\t\tbuf := make([]byte, 8196)\n\t\t\t\tl, err := f.Read(buf)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif bytes.Equal(content[i/fileWorkers], buf[:l]) != true {\n\t\t\t\t\terr = errors.New(\"read and write not equal\")\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\tfor range files {\n\t\t\terr := <-errs\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t})\n}\n\n// Test ipfs_cid extended attribute\nfunc TestMFSRootXattr(t *testing.T) {\n\tipfs, err := core.NewNode(context.Background(), &node.BuildCfg{})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfs, mnt := setUp(t, ipfs)\n\tdefer mnt.Close()\n\n\tnode, err := fs.Root()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\troot := node.(*Dir)\n\n\tlistReq := fuse.ListxattrRequest{}\n\tlistRes := fuse.ListxattrResponse{}\n\terr = root.Listxattr(context.Background(), &listReq, &listRes)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif slices.Compare(listRes.Xattr, []byte(\"ipfs_cid\\x00\")) != 0 {\n\t\tt.Fatal(\"list xattr returns invalid value\")\n\t}\n\n\tgetReq := fuse.GetxattrRequest{\n\t\tName: \"ipfs_cid\",\n\t}\n\tgetRes := fuse.GetxattrResponse{}\n\terr = root.Getxattr(context.Background(), &getReq, &getRes)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tipldNode, err := ipfs.FilesRoot.GetDirectory().GetNode()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif slices.Compare(getRes.Xattr, []byte(ipldNode.Cid().String())) != 0 {\n\t\tt.Fatal(\"xattr cid not equal to mfs root cid\")\n\t}\n}\n" }, { "path": "fuse/mfs/mfs_unix.go", "content": "//go:build (linux || darwin || freebsd || netbsd || openbsd) && !nofuse\n\npackage mfs\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"os\"\n\t\"sync\"\n\t\"syscall\"\n\t\"time\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/boxo/mfs\"\n\t\"github.com/ipfs/kubo/core\"\n)\n\nconst (\n\tipfsCIDXattr = \"ipfs_cid\"\n\tmfsDirMode = os.ModeDir | 0755\n\tmfsFileMode = 0644\n\tblockSize = 512\n\tdirSize = 8\n)\n\n// FUSE filesystem mounted at /mfs.\ntype FileSystem struct {\n\troot Dir\n}\n\n// Get filesystem root.\nfunc (fs *FileSystem) Root() (fs.Node, error) {\n\treturn &fs.root, nil\n}\n\n// FUSE Adapter for MFS directories.\ntype Dir struct {\n\tmfsDir *mfs.Directory\n}\n\n// Directory attributes (stat).\nfunc (dir *Dir) Attr(ctx context.Context, attr *fuse.Attr) error {\n\tattr.Mode = mfsDirMode\n\tattr.Size = dirSize * blockSize\n\tattr.Blocks = dirSize\n\treturn nil\n}\n\n// Access files in a directory.\nfunc (dir *Dir) Lookup(ctx context.Context, req *fuse.LookupRequest, resp *fuse.LookupResponse) (fs.Node, error) {\n\tmfsNode, err := dir.mfsDir.Child(req.Name)\n\tswitch err {\n\tcase os.ErrNotExist:\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\tcase nil:\n\tdefault:\n\t\treturn nil, err\n\t}\n\n\tswitch mfsNode.Type() {\n\tcase mfs.TDir:\n\t\tresult := Dir{\n\t\t\tmfsDir: mfsNode.(*mfs.Directory),\n\t\t}\n\t\treturn &result, nil\n\tcase mfs.TFile:\n\t\tresult := File{\n\t\t\tmfsFile: mfsNode.(*mfs.File),\n\t\t}\n\t\treturn &result, nil\n\t}\n\n\treturn nil, syscall.Errno(syscall.ENOENT)\n}\n\n// List (ls) MFS directory.\nfunc (dir *Dir) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {\n\tvar res []fuse.Dirent\n\tnodes, err := dir.mfsDir.List(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, node := range nodes {\n\t\tnodeType := fuse.DT_File\n\t\tif node.Type == 1 {\n\t\t\tnodeType = fuse.DT_Dir\n\t\t}\n\t\tres = append(res, fuse.Dirent{\n\t\t\tType: nodeType,\n\t\t\tName: node.Name,\n\t\t})\n\t}\n\treturn res, nil\n}\n\n// Mkdir (mkdir) in MFS.\nfunc (dir *Dir) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) {\n\tmfsDir, err := dir.mfsDir.Mkdir(req.Name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn &Dir{\n\t\tmfsDir: mfsDir,\n\t}, nil\n}\n\n// Remove (rm/rmdir) an MFS file.\nfunc (dir *Dir) Remove(ctx context.Context, req *fuse.RemoveRequest) error {\n\t// Check for empty directory.\n\tif req.Dir {\n\t\ttargetNode, err := dir.mfsDir.Child(req.Name)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\ttarget := targetNode.(*mfs.Directory)\n\n\t\tchildren, err := target.ListNames(ctx)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif len(children) > 0 {\n\t\t\treturn os.ErrExist\n\t\t}\n\t}\n\terr := dir.mfsDir.Unlink(req.Name)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn dir.mfsDir.Flush()\n}\n\n// Move (mv) an MFS file.\nfunc (dir *Dir) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error {\n\tfile, err := dir.mfsDir.Child(req.OldName)\n\tif err != nil {\n\t\treturn err\n\t}\n\tnode, err := file.GetNode()\n\tif err != nil {\n\t\treturn err\n\t}\n\ttargetDir := newDir.(*Dir)\n\n\t// Remove file if exists\n\terr = targetDir.mfsDir.Unlink(req.NewName)\n\tif err != nil && err != os.ErrNotExist {\n\t\treturn err\n\t}\n\n\terr = targetDir.mfsDir.AddChild(req.NewName, node)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = dir.mfsDir.Unlink(req.OldName)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn dir.mfsDir.Flush()\n}\n\n// Create (touch) an MFS file.\nfunc (dir *Dir) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error) {\n\tnode := dag.NodeWithData(ft.FilePBData(nil, 0))\n\tif err := node.SetCidBuilder(dir.mfsDir.GetCidBuilder()); err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tif err := dir.mfsDir.AddChild(req.Name, node); err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tif err := dir.mfsDir.Flush(); err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tmfsNode, err := dir.mfsDir.Child(req.Name)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\tif err := mfsNode.SetModTime(time.Now()); err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tmfsFile := mfsNode.(*mfs.File)\n\n\tfile := File{\n\t\tmfsFile: mfsFile,\n\t}\n\n\t// Read access flags and create a handler.\n\taccessMode := req.Flags & fuse.OpenAccessModeMask\n\tflags := mfs.Flags{\n\t\tRead: accessMode == fuse.OpenReadOnly || accessMode == fuse.OpenReadWrite,\n\t\tWrite: accessMode == fuse.OpenWriteOnly || accessMode == fuse.OpenReadWrite,\n\t\tSync: req.Flags|fuse.OpenSync > 0,\n\t}\n\n\tfd, err := mfsFile.Open(flags)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\thandler := FileHandler{\n\t\tmfsFD: fd,\n\t}\n\n\treturn &file, &handler, nil\n}\n\n// List dir xattr.\nfunc (dir *Dir) Listxattr(ctx context.Context, req *fuse.ListxattrRequest, resp *fuse.ListxattrResponse) error {\n\tresp.Append(ipfsCIDXattr)\n\treturn nil\n}\n\n// Get dir xattr.\nfunc (dir *Dir) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error {\n\tswitch req.Name {\n\tcase ipfsCIDXattr:\n\t\tnode, err := dir.mfsDir.GetNode()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tresp.Xattr = []byte(node.Cid().String())\n\t\treturn nil\n\tdefault:\n\t\treturn fuse.ErrNoXattr\n\t}\n}\n\n// FUSE adapter for MFS files.\ntype File struct {\n\tmfsFile *mfs.File\n}\n\n// File attributes.\nfunc (file *File) Attr(ctx context.Context, attr *fuse.Attr) error {\n\tsize, _ := file.mfsFile.Size()\n\n\tattr.Size = uint64(size)\n\tif size%blockSize == 0 {\n\t\tattr.Blocks = uint64(size / blockSize)\n\t} else {\n\t\tattr.Blocks = uint64(size/blockSize + 1)\n\t}\n\n\tmtime, _ := file.mfsFile.ModTime()\n\tattr.Mtime = mtime\n\n\tattr.Mode = mfsFileMode\n\treturn nil\n}\n\n// Open an MFS file.\nfunc (file *File) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.OpenResponse) (fs.Handle, error) {\n\taccessMode := req.Flags & fuse.OpenAccessModeMask\n\tflags := mfs.Flags{\n\t\tRead: accessMode == fuse.OpenReadOnly || accessMode == fuse.OpenReadWrite,\n\t\tWrite: accessMode == fuse.OpenWriteOnly || accessMode == fuse.OpenReadWrite,\n\t\tSync: req.Flags|fuse.OpenSync > 0,\n\t}\n\tfd, err := file.mfsFile.Open(flags)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif flags.Write {\n\t\tif err := file.mfsFile.SetModTime(time.Now()); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn &FileHandler{\n\t\tmfsFD: fd,\n\t}, nil\n}\n\n// Sync the file's contents to MFS.\nfunc (file *File) Fsync(ctx context.Context, req *fuse.FsyncRequest) error {\n\treturn file.mfsFile.Sync()\n}\n\n// List file xattr.\nfunc (file *File) Listxattr(ctx context.Context, req *fuse.ListxattrRequest, resp *fuse.ListxattrResponse) error {\n\tresp.Append(ipfsCIDXattr)\n\treturn nil\n}\n\n// Get file xattr.\nfunc (file *File) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error {\n\tswitch req.Name {\n\tcase ipfsCIDXattr:\n\t\tnode, err := file.mfsFile.GetNode()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tresp.Xattr = []byte(node.Cid().String())\n\t\treturn nil\n\tdefault:\n\t\treturn fuse.ErrNoXattr\n\t}\n}\n\n// Wrapper for MFS's file descriptor that conforms to the FUSE fs.Handler\n// interface.\ntype FileHandler struct {\n\tmfsFD mfs.FileDescriptor\n\tmu sync.Mutex\n}\n\n// Read a opened MFS file.\nfunc (fh *FileHandler) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {\n\tfh.mu.Lock()\n\tdefer fh.mu.Unlock()\n\n\t_, err := fh.mfsFD.Seek(req.Offset, io.SeekStart)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbuf := make([]byte, req.Size)\n\tl, err := fh.mfsFD.Read(buf)\n\n\tresp.Data = buf[:l]\n\n\tswitch err {\n\tcase nil, io.EOF, io.ErrUnexpectedEOF:\n\t\treturn nil\n\tdefault:\n\t\treturn err\n\t}\n}\n\n// Write writes to an opened MFS file.\nfunc (fh *FileHandler) Write(ctx context.Context, req *fuse.WriteRequest, resp *fuse.WriteResponse) error {\n\tfh.mu.Lock()\n\tdefer fh.mu.Unlock()\n\n\tl, err := fh.mfsFD.WriteAt(req.Data, req.Offset)\n\tif err != nil {\n\t\treturn err\n\t}\n\tresp.Size = l\n\n\treturn nil\n}\n\n// Flushes the file's buffer.\nfunc (fh *FileHandler) Flush(ctx context.Context, req *fuse.FlushRequest) error {\n\tfh.mu.Lock()\n\tdefer fh.mu.Unlock()\n\n\treturn fh.mfsFD.Flush()\n}\n\n// Closes the file.\nfunc (fh *FileHandler) Release(ctx context.Context, req *fuse.ReleaseRequest) error {\n\tfh.mu.Lock()\n\tdefer fh.mu.Unlock()\n\n\treturn fh.mfsFD.Close()\n}\n\n// Create new filesystem.\nfunc NewFileSystem(ipfs *core.IpfsNode) fs.FS {\n\treturn &FileSystem{\n\t\troot: Dir{\n\t\t\tmfsDir: ipfs.FilesRoot.GetDirectory(),\n\t\t},\n\t}\n}\n\n// Check that our structs implement all the interfaces we want.\ntype mfsDir interface {\n\tfs.Node\n\tfs.NodeGetxattrer\n\tfs.NodeListxattrer\n\tfs.HandleReadDirAller\n\tfs.NodeRequestLookuper\n\tfs.NodeMkdirer\n\tfs.NodeRenamer\n\tfs.NodeRemover\n\tfs.NodeCreater\n}\n\nvar _ mfsDir = (*Dir)(nil)\n\ntype mfsFile interface {\n\tfs.Node\n\tfs.NodeGetxattrer\n\tfs.NodeListxattrer\n\tfs.NodeOpener\n\tfs.NodeFsyncer\n}\n\nvar _ mfsFile = (*File)(nil)\n\ntype mfsHandler interface {\n\tfs.Handle\n\tfs.HandleReader\n\tfs.HandleWriter\n\tfs.HandleFlusher\n\tfs.HandleReleaser\n}\n\nvar _ mfsHandler = (*FileHandler)(nil)\n" }, { "path": "fuse/mfs/mount_unix.go", "content": "//go:build (linux || darwin || freebsd || netbsd || openbsd) && !nofuse\n\npackage mfs\n\nimport (\n\tcore \"github.com/ipfs/kubo/core\"\n\tmount \"github.com/ipfs/kubo/fuse/mount\"\n)\n\n// Mount mounts MFS at a given location, and returns a mount.Mount instance.\nfunc Mount(ipfs *core.IpfsNode, mountpoint string) (mount.Mount, error) {\n\tcfg, err := ipfs.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tallowOther := cfg.Mounts.FuseAllowOther\n\tfsys := NewFileSystem(ipfs)\n\treturn mount.NewMount(fsys, mountpoint, allowOther)\n}\n" }, { "path": "fuse/mount/fuse.go", "content": "//go:build !nofuse && !windows && !openbsd && !netbsd && !plan9\n\npackage mount\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n)\n\nvar ErrNotMounted = errors.New(\"not mounted\")\n\n// mount implements go-ipfs/fuse/mount.\ntype mount struct {\n\tmpoint string\n\tfilesys fs.FS\n\tfuseConn *fuse.Conn\n\n\tactive bool\n\tactiveLock *sync.RWMutex\n\n\tunmountOnce sync.Once\n}\n\n// Mount mounts a fuse fs.FS at a given location, and returns a Mount instance.\n// ctx is parent is a ContextGroup to bind the mount's ContextGroup to.\nfunc NewMount(fsys fs.FS, mountpoint string, allowOther bool) (Mount, error) {\n\tvar conn *fuse.Conn\n\tvar err error\n\n\tmountOpts := []fuse.MountOption{\n\t\tfuse.MaxReadahead(64 * 1024 * 1024),\n\t\tfuse.AsyncRead(),\n\t}\n\n\tif allowOther {\n\t\tmountOpts = append(mountOpts, fuse.AllowOther())\n\t}\n\tconn, err = fuse.Mount(mountpoint, mountOpts...)\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tm := &mount{\n\t\tmpoint: mountpoint,\n\t\tfuseConn: conn,\n\t\tfilesys: fsys,\n\t\tactive: false,\n\t\tactiveLock: &sync.RWMutex{},\n\t}\n\n\t// launch the mounting process.\n\tif err = m.mount(); err != nil {\n\t\t_ = m.Unmount() // just in case.\n\t\treturn nil, err\n\t}\n\n\treturn m, nil\n}\n\nfunc (m *mount) mount() error {\n\tlog.Infof(\"Mounting %s\", m.MountPoint())\n\n\terrs := make(chan error, 1)\n\tgo func() {\n\t\t// fs.Serve blocks until the filesystem is unmounted.\n\t\terr := fs.Serve(m.fuseConn, m.filesys)\n\t\tlog.Debugf(\"%s is unmounted\", m.MountPoint())\n\t\tif err != nil {\n\t\t\tlog.Debugf(\"fs.Serve returned (%s)\", err)\n\t\t\terrs <- err\n\t\t}\n\t\tm.setActive(false)\n\t}()\n\n\t// wait for the mount process to be done, or timed out.\n\tselect {\n\tcase <-time.After(MountTimeout):\n\t\treturn fmt.Errorf(\"mounting %s timed out\", m.MountPoint())\n\tcase err := <-errs:\n\t\treturn err\n\tcase <-m.fuseConn.Ready:\n\t}\n\n\t// check if the mount process has an error to report\n\tif err := m.fuseConn.MountError; err != nil {\n\t\treturn err\n\t}\n\n\tm.setActive(true)\n\n\tlog.Infof(\"Mounted %s\", m.MountPoint())\n\treturn nil\n}\n\n// unmount is called exactly once to unmount this service.\n// note that closing the connection will not always unmount\n// properly. If that happens, we bring out the big guns\n// (mount.ForceUnmountManyTimes, exec unmount).\nfunc (m *mount) unmount() error {\n\tlog.Infof(\"Unmounting %s\", m.MountPoint())\n\n\t// try unmounting with fuse lib\n\terr := fuse.Unmount(m.MountPoint())\n\tif err == nil {\n\t\tm.setActive(false)\n\t\treturn nil\n\t}\n\tlog.Warnf(\"fuse unmount err: %s\", err)\n\n\t// try closing the fuseConn\n\terr = m.fuseConn.Close()\n\tif err == nil {\n\t\tm.setActive(false)\n\t\treturn nil\n\t}\n\tlog.Warnf(\"fuse conn error: %s\", err)\n\n\t// try mount.ForceUnmountManyTimes\n\tif err := ForceUnmountManyTimes(m, 10); err != nil {\n\t\treturn err\n\t}\n\n\tlog.Infof(\"Seemingly unmounted %s\", m.MountPoint())\n\tm.setActive(false)\n\treturn nil\n}\n\nfunc (m *mount) MountPoint() string {\n\treturn m.mpoint\n}\n\nfunc (m *mount) Unmount() error {\n\tif !m.IsActive() {\n\t\treturn ErrNotMounted\n\t}\n\n\tvar err error\n\tm.unmountOnce.Do(func() {\n\t\terr = m.unmount()\n\t})\n\n\treturn err\n}\n\nfunc (m *mount) IsActive() bool {\n\tm.activeLock.RLock()\n\tdefer m.activeLock.RUnlock()\n\n\treturn m.active\n}\n\nfunc (m *mount) setActive(a bool) {\n\tm.activeLock.Lock()\n\tm.active = a\n\tm.activeLock.Unlock()\n}\n" }, { "path": "fuse/mount/mount.go", "content": "// package mount provides a simple abstraction around a mount point\npackage mount\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os/exec\"\n\t\"runtime\"\n\t\"time\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n)\n\nvar log = logging.Logger(\"mount\")\n\nvar MountTimeout = time.Second * 5\n\n// Mount represents a filesystem mount.\ntype Mount interface {\n\t// MountPoint is the path at which this mount is mounted\n\tMountPoint() string\n\n\t// Unmounts the mount\n\tUnmount() error\n\n\t// Checks if the mount is still active.\n\tIsActive() bool\n}\n\n// ForceUnmount attempts to forcibly unmount a given mount.\n// It does so by calling diskutil or fusermount directly.\nfunc ForceUnmount(m Mount) error {\n\tpoint := m.MountPoint()\n\tlog.Warnf(\"Force-Unmounting %s...\", point)\n\n\tcmd, err := UnmountCmd(point)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terrc := make(chan error, 1)\n\tgo func() {\n\t\tdefer close(errc)\n\n\t\t// try vanilla unmount first.\n\t\tif err := exec.Command(\"umount\", point).Run(); err == nil {\n\t\t\treturn\n\t\t}\n\n\t\t// retry to unmount with the fallback cmd\n\t\terrc <- cmd.Run()\n\t}()\n\n\tselect {\n\tcase <-time.After(7 * time.Second):\n\t\treturn fmt.Errorf(\"umount timeout\")\n\tcase err := <-errc:\n\t\treturn err\n\t}\n}\n\n// UnmountCmd creates an exec.Cmd that is GOOS-specific\n// for unmount a FUSE mount.\nfunc UnmountCmd(point string) (*exec.Cmd, error) {\n\tswitch runtime.GOOS {\n\tcase \"darwin\":\n\t\treturn exec.Command(\"diskutil\", \"umount\", \"force\", point), nil\n\tcase \"linux\":\n\t\treturn exec.Command(\"fusermount\", \"-u\", point), nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unmount: unimplemented\")\n\t}\n}\n\n// ForceUnmountManyTimes attempts to forcibly unmount a given mount,\n// many times. It does so by calling diskutil or fusermount directly.\n// Attempts a given number of times.\nfunc ForceUnmountManyTimes(m Mount, attempts int) error {\n\tvar err error\n\tfor range attempts {\n\t\terr = ForceUnmount(m)\n\t\tif err == nil {\n\t\t\treturn err\n\t\t}\n\n\t\t<-time.After(time.Millisecond * 500)\n\t}\n\treturn fmt.Errorf(\"unmount %s failed after 10 seconds of trying\", m.MountPoint())\n}\n\ntype closer struct {\n\tM Mount\n}\n\nfunc (c *closer) Close() error {\n\tlog.Warn(\" (c *closer) Close(),\", c.M.MountPoint())\n\treturn c.M.Unmount()\n}\n\nfunc Closer(m Mount) io.Closer {\n\treturn &closer{m}\n}\n" }, { "path": "fuse/node/mount_darwin.go", "content": "//go:build !nofuse && darwin\n\npackage node\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"os/exec\"\n\t\"runtime\"\n\t\"strings\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\n\t\"github.com/blang/semver/v4\"\n\tunix \"golang.org/x/sys/unix\"\n)\n\nfunc init() {\n\t// this is a hack, but until we need to do it another way, this works.\n\tplatformFuseChecks = darwinFuseCheckVersion\n}\n\n// dontCheckOSXFUSEConfigKey is a key used to let the user tell us to\n// skip fuse checks.\nconst dontCheckOSXFUSEConfigKey = \"DontCheckOSXFUSE\"\n\n// fuseVersionPkg is the go pkg url for fuse-version.\nconst fuseVersionPkg = \"github.com/jbenet/go-fuse-version/fuse-version\"\n\n// errStrFuseRequired is returned when we're sure the user does not have fuse.\nconst errStrFuseRequired = `OSXFUSE not found.\n\nOSXFUSE is required to mount, please install it.\nNOTE: Version 2.7.2 or higher required; prior versions are known to kernel panic!\nIt is recommended you install it from the OSXFUSE website:\n\n\thttp://osxfuse.github.io/\n\nFor more help, see:\n\n\thttps://github.com/ipfs/kubo/issues/177\n`\n\n// errStrNoFuseHeaders is included in the output of `go get ` if there\n// are no fuse headers. this means they don't have OSXFUSE installed.\nvar errStrNoFuseHeaders = \"no such file or directory: '/usr/local/lib/libosxfuse.dylib'\"\n\nvar errStrUpgradeFuse = `OSXFUSE version %s not supported.\n\nOSXFUSE versions <2.7.2 are known to cause kernel panics!\nPlease upgrade to the latest OSXFUSE version.\nIt is recommended you install it from the OSXFUSE website:\n\n\thttp://osxfuse.github.io/\n\nFor more help, see:\n\n\thttps://github.com/ipfs/kubo/issues/177\n`\n\ntype errNeedFuseVersion struct {\n\tcause string\n}\n\nfunc (me errNeedFuseVersion) Error() string {\n\treturn fmt.Sprintf(`unable to check fuse version.\n\nDear User,\n\nBefore mounting, we must check your version of OSXFUSE. We are protecting\nyou from a nasty kernel panic we found in OSXFUSE versions <2.7.2.[1]. To\nmake matters worse, it's harder than it should be to check whether you have\nthe right version installed...[2]. We've automated the process with the\nhelp of a little tool. We tried to install it, but something went wrong[3].\nPlease install it yourself by running:\n\n\tgo get %s\n\nYou can also stop ipfs from running these checks and use whatever OSXFUSE\nversion you have by running:\n\n\tipfs config --bool %s true\n\n[1]: https://github.com/ipfs/kubo/issues/177\n[2]: https://github.com/ipfs/kubo/pull/533\n[3]: %s\n`, fuseVersionPkg, dontCheckOSXFUSEConfigKey, me.cause)\n}\n\nvar errStrFailedToRunFuseVersion = `unable to check fuse version.\n\nDear User,\n\nBefore mounting, we must check your version of OSXFUSE. We are protecting\nyou from a nasty kernel panic we found in OSXFUSE versions <2.7.2.[1]. To\nmake matters worse, it's harder than it should be to check whether you have\nthe right version installed...[2]. We've automated the process with the\nhelp of a little tool. We tried to run it, but something went wrong[3].\nPlease, try to run it yourself with:\n\n\tgo get %s\n\tfuse-version\n\nYou should see something like this:\n\n\t> fuse-version\n\tfuse-version -only agent\n\tOSXFUSE.AgentVersion: 2.7.3\n\nJust make sure the number is 2.7.2 or higher. You can then stop ipfs from\ntrying to run these checks with:\n\n\tipfs config --bool %s true\n\n[1]: https://github.com/ipfs/kubo/issues/177\n[2]: https://github.com/ipfs/kubo/pull/533\n[3]: %s\n`\n\nvar errStrFixConfig = `config key invalid: %s %v\nYou may be able to get this error to go away by setting it again:\n\n\tipfs config --bool %s true\n\nEither way, please tell us at: http://github.com/ipfs/kubo/issues\n`\n\nfunc darwinFuseCheckVersion(node *core.IpfsNode) error {\n\t// on OSX, check FUSE version.\n\tif runtime.GOOS != \"darwin\" {\n\t\treturn nil\n\t}\n\n\tov, errGFV := tryGFV()\n\tif errGFV != nil {\n\t\t// if we failed AND the user has told us to ignore the check we\n\t\t// continue. this is in case fuse-version breaks or the user cannot\n\t\t// install it, but is sure their fuse version will work.\n\t\tif skip, err := userAskedToSkipFuseCheck(node); err != nil {\n\t\t\treturn err\n\t\t} else if skip {\n\t\t\treturn nil // user told us not to check version... ok....\n\t\t}\n\t\treturn errGFV\n\t}\n\n\tlog.Debug(\"mount: osxfuse version:\", ov)\n\n\tmin := semver.MustParse(\"2.7.2\")\n\tcurr, err := semver.Make(ov)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif curr.LT(min) {\n\t\treturn fmt.Errorf(errStrUpgradeFuse, ov)\n\t}\n\treturn nil\n}\n\nfunc tryGFV() (string, error) {\n\t// first try sysctl. it may work!\n\tov, err := trySysctl()\n\tif err == nil {\n\t\treturn ov, nil\n\t}\n\tlog.Debug(err)\n\n\treturn tryGFVFromFuseVersion()\n}\n\nfunc trySysctl() (string, error) {\n\tv, err := unix.Sysctl(\"osxfuse.version.number\")\n\tif err != nil {\n\t\tlog.Debug(\"mount: sysctl osxfuse.version.number:\", \"failed\")\n\t\treturn \"\", err\n\t}\n\tlog.Debug(\"mount: sysctl osxfuse.version.number:\", v)\n\treturn v, nil\n}\n\nfunc tryGFVFromFuseVersion() (string, error) {\n\tif err := ensureFuseVersionIsInstalled(); err != nil {\n\t\treturn \"\", err\n\t}\n\n\tcmd := exec.Command(\"fuse-version\", \"-q\", \"-only\", \"agent\", \"-s\", \"OSXFUSE\")\n\tout := new(bytes.Buffer)\n\tcmd.Stdout = out\n\tif err := cmd.Run(); err != nil {\n\t\treturn \"\", fmt.Errorf(errStrFailedToRunFuseVersion, fuseVersionPkg, dontCheckOSXFUSEConfigKey, err)\n\t}\n\n\treturn out.String(), nil\n}\n\nfunc ensureFuseVersionIsInstalled() error {\n\t// see if fuse-version is there\n\tif _, err := exec.LookPath(\"fuse-version\"); err == nil {\n\t\treturn nil // got it!\n\t}\n\n\t// try installing it...\n\tlog.Debug(\"fuse-version: no fuse-version. attempting to install.\")\n\tcmd := exec.Command(\"go\", \"install\", \"github.com/jbenet/go-fuse-version/fuse-version\")\n\tcmdout := new(bytes.Buffer)\n\tcmd.Stdout = cmdout\n\tcmd.Stderr = cmdout\n\tif err := cmd.Run(); err != nil {\n\t\t// Ok, install fuse-version failed. is it they don't have fuse?\n\t\tcmdoutstr := cmdout.String()\n\t\tif strings.Contains(cmdoutstr, errStrNoFuseHeaders) {\n\t\t\t// yes! it is! they don't have fuse!\n\t\t\treturn fmt.Errorf(errStrFuseRequired)\n\t\t}\n\n\t\tlog.Debug(\"fuse-version: failed to install.\")\n\t\ts := err.Error() + \"\\n\" + cmdoutstr\n\t\treturn errNeedFuseVersion{s}\n\t}\n\n\t// ok, try again...\n\tif _, err := exec.LookPath(\"fuse-version\"); err != nil {\n\t\tlog.Debug(\"fuse-version: failed to install?\")\n\t\treturn errNeedFuseVersion{err.Error()}\n\t}\n\n\tlog.Debug(\"fuse-version: install success\")\n\treturn nil\n}\n\nfunc userAskedToSkipFuseCheck(node *core.IpfsNode) (skip bool, err error) {\n\tval, err := node.Repo.GetConfigKey(dontCheckOSXFUSEConfigKey)\n\tif err != nil {\n\t\treturn false, nil // failed to get config value. don't skip check.\n\t}\n\n\tswitch val := val.(type) {\n\tcase string:\n\t\treturn val == \"true\", nil\n\tcase bool:\n\t\treturn val, nil\n\tdefault:\n\t\t// got config value, but it's invalid... don't skip check, ask the user to fix it...\n\t\treturn false, fmt.Errorf(errStrFixConfig, dontCheckOSXFUSEConfigKey, val,\n\t\t\tdontCheckOSXFUSEConfigKey)\n\t}\n}\n" }, { "path": "fuse/node/mount_nofuse.go", "content": "//go:build !windows && nofuse\n\npackage node\n\nimport (\n\t\"errors\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\nfunc Mount(node *core.IpfsNode, fsdir, nsdir, mfsdir string) error {\n\treturn errors.New(\"not compiled in\")\n}\n\nfunc Unmount(node *core.IpfsNode) {\n\treturn\n}\n" }, { "path": "fuse/node/mount_notsupp.go", "content": "//go:build (!nofuse && openbsd) || (!nofuse && netbsd) || (!nofuse && plan9)\n\npackage node\n\nimport (\n\t\"errors\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n)\n\nfunc Mount(node *core.IpfsNode, fsdir, nsdir, mfsdir string) error {\n\treturn errors.New(\"FUSE not supported on OpenBSD or NetBSD. See #5334 (https://github.com/ipfs/kubo/issues/5334).\")\n}\n\nfunc Unmount(node *core.IpfsNode) {\n\treturn\n}\n" }, { "path": "fuse/node/mount_test.go", "content": "//go:build !openbsd && !nofuse && !netbsd && !plan9\n\npackage node\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"bazil.org/fuse\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tipns \"github.com/ipfs/kubo/fuse/ipns\"\n\tmount \"github.com/ipfs/kubo/fuse/mount\"\n\n\tci \"github.com/libp2p/go-libp2p-testing/ci\"\n)\n\nfunc maybeSkipFuseTests(t *testing.T) {\n\tif ci.NoFuse() {\n\t\tt.Skip(\"Skipping FUSE tests\")\n\t}\n}\n\nfunc mkdir(t *testing.T, path string) {\n\terr := os.Mkdir(path, os.ModeDir|os.ModePerm)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\n// Test externally unmounting, then trying to unmount in code.\nfunc TestExternalUnmount(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\n\t// TODO: needed?\n\tmaybeSkipFuseTests(t)\n\n\tnode, err := core.NewNode(context.Background(), &core.BuildCfg{})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = ipns.InitializeKeyspace(node, node.PrivateKey)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// get the test dir paths (/tmp/TestExternalUnmount)\n\tdir := t.TempDir()\n\n\tipfsDir := dir + \"/ipfs\"\n\tipnsDir := dir + \"/ipns\"\n\tmfsDir := dir + \"/mfs\"\n\tmkdir(t, ipfsDir)\n\tmkdir(t, ipnsDir)\n\tmkdir(t, mfsDir)\n\n\terr = Mount(node, ipfsDir, ipnsDir, mfsDir)\n\tif err != nil {\n\t\tif strings.Contains(err.Error(), \"unable to check fuse version\") || err == fuse.ErrOSXFUSENotFound {\n\t\t\tt.Skip(err)\n\t\t}\n\t}\n\n\tif err != nil {\n\t\tt.Fatalf(\"error mounting: %v\", err)\n\t}\n\n\t// Run shell command to externally unmount the directory\n\tcmd, err := mount.UnmountCmd(ipfsDir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := cmd.Run(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// TODO(noffle): it takes a moment for the goroutine that's running fs.Serve to be notified and do its cleanup.\n\ttime.Sleep(time.Millisecond * 100)\n\n\t// Attempt to unmount IPFS; it should unmount successfully.\n\terr = node.Mounts.Ipfs.Unmount()\n\tif err != mount.ErrNotMounted {\n\t\tt.Fatal(\"Unmount should have failed\")\n\t}\n}\n" }, { "path": "fuse/node/mount_unix.go", "content": "//go:build !windows && !openbsd && !netbsd && !plan9 && !nofuse\n\npackage node\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\t\"sync\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tipns \"github.com/ipfs/kubo/fuse/ipns\"\n\tmfs \"github.com/ipfs/kubo/fuse/mfs\"\n\tmount \"github.com/ipfs/kubo/fuse/mount\"\n\trofs \"github.com/ipfs/kubo/fuse/readonly\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n)\n\nvar log = logging.Logger(\"node\")\n\n// fuseNoDirectory used to check the returning fuse error.\nconst fuseNoDirectory = \"fusermount: failed to access mountpoint\"\n\n// fuseExitStatus1 used to check the returning fuse error.\nconst fuseExitStatus1 = \"fusermount: exit status 1\"\n\n// platformFuseChecks can get overridden by arch-specific files\n// to run fuse checks (like checking the OSXFUSE version).\nvar platformFuseChecks = func(*core.IpfsNode) error {\n\treturn nil\n}\n\nfunc Mount(node *core.IpfsNode, fsdir, nsdir, mfsdir string) error {\n\t// check if we already have live mounts.\n\t// if the user said \"Mount\", then there must be something wrong.\n\t// so, close them and try again.\n\tUnmount(node)\n\n\tif err := platformFuseChecks(node); err != nil {\n\t\treturn err\n\t}\n\n\treturn doMount(node, fsdir, nsdir, mfsdir)\n}\n\nfunc Unmount(node *core.IpfsNode) {\n\tif node.Mounts.Ipfs != nil && node.Mounts.Ipfs.IsActive() {\n\t\t// best effort\n\t\tif err := node.Mounts.Ipfs.Unmount(); err != nil {\n\t\t\tlog.Errorf(\"error unmounting IPFS: %s\", err)\n\t\t}\n\t}\n\tif node.Mounts.Ipns != nil && node.Mounts.Ipns.IsActive() {\n\t\t// best effort\n\t\tif err := node.Mounts.Ipns.Unmount(); err != nil {\n\t\t\tlog.Errorf(\"error unmounting IPNS: %s\", err)\n\t\t}\n\t}\n\tif node.Mounts.Mfs != nil && node.Mounts.Mfs.IsActive() {\n\t\t// best effort\n\t\tif err := node.Mounts.Mfs.Unmount(); err != nil {\n\t\t\tlog.Errorf(\"error unmounting MFS: %s\", err)\n\t\t}\n\t}\n}\n\nfunc doMount(node *core.IpfsNode, fsdir, nsdir, mfsdir string) error {\n\tfmtFuseErr := func(err error, mountpoint string) error {\n\t\ts := err.Error()\n\t\tif strings.Contains(s, fuseNoDirectory) {\n\t\t\ts = strings.Replace(s, `fusermount: \"fusermount:`, \"\", -1)\n\t\t\ts = strings.Replace(s, `\\n\", exit status 1`, \"\", -1)\n\t\t\treturn errors.New(s)\n\t\t}\n\t\tif s == fuseExitStatus1 {\n\t\t\ts = fmt.Sprintf(\"fuse failed to access mountpoint %s\", mountpoint)\n\t\t\treturn errors.New(s)\n\t\t}\n\t\treturn err\n\t}\n\n\t// this sync stuff is so that both can be mounted simultaneously.\n\tvar fsmount, nsmount, mfmount mount.Mount\n\tvar err1, err2, err3 error\n\n\tvar wg sync.WaitGroup\n\n\twg.Go(func() {\n\t\tfsmount, err1 = rofs.Mount(node, fsdir)\n\t})\n\n\tif node.IsOnline {\n\t\twg.Go(func() {\n\t\t\tnsmount, err2 = ipns.Mount(node, nsdir, fsdir)\n\t\t})\n\t}\n\n\twg.Go(func() {\n\t\tmfmount, err3 = mfs.Mount(node, mfsdir)\n\t})\n\n\twg.Wait()\n\n\tif err1 != nil {\n\t\tlog.Errorf(\"error mounting IPFS %s: %s\", fsdir, err1)\n\t}\n\n\tif err2 != nil {\n\t\tlog.Errorf(\"error mounting IPNS %s for IPFS %s: %s\", nsdir, fsdir, err2)\n\t}\n\n\tif err3 != nil {\n\t\tlog.Errorf(\"error mounting MFS %s: %s\", mfsdir, err3)\n\t}\n\n\tif err1 != nil || err2 != nil || err3 != nil {\n\t\tif fsmount != nil {\n\t\t\t_ = fsmount.Unmount()\n\t\t}\n\t\tif nsmount != nil {\n\t\t\t_ = nsmount.Unmount()\n\t\t}\n\t\tif mfmount != nil {\n\t\t\t_ = mfmount.Unmount()\n\t\t}\n\n\t\tif err1 != nil {\n\t\t\treturn fmtFuseErr(err1, fsdir)\n\t\t}\n\t\tif err2 != nil {\n\t\t\treturn fmtFuseErr(err2, nsdir)\n\t\t}\n\t\treturn fmtFuseErr(err3, mfsdir)\n\t}\n\n\t// setup node state, so that it can be canceled\n\tnode.Mounts.Ipfs = fsmount\n\tnode.Mounts.Ipns = nsmount\n\tnode.Mounts.Mfs = mfmount\n\treturn nil\n}\n" }, { "path": "fuse/node/mount_windows.go", "content": "package node\n\nimport (\n\t\"github.com/ipfs/kubo/core\"\n)\n\nfunc Mount(node *core.IpfsNode, fsdir, nsdir, mfsdir string) error {\n\t// TODO\n\t// currently a no-op, but we don't want to return an error\n\treturn nil\n}\n\nfunc Unmount(node *core.IpfsNode) {\n\t// TODO\n\t// currently a no-op\n\treturn\n}\n" }, { "path": "fuse/readonly/doc.go", "content": "// package fuse/readonly implements a fuse filesystem to access files\n// stored inside of ipfs.\npackage readonly\n" }, { "path": "fuse/readonly/ipfs_test.go", "content": "//go:build !nofuse && !openbsd && !netbsd && !plan9\n\npackage readonly\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"math/rand\"\n\t\"os\"\n\tgopath \"path\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"bazil.org/fuse\"\n\n\tcore \"github.com/ipfs/kubo/core\"\n\tcoreapi \"github.com/ipfs/kubo/core/coreapi\"\n\tcoremock \"github.com/ipfs/kubo/core/mock\"\n\n\tfstest \"bazil.org/fuse/fs/fstestutil\"\n\tchunker \"github.com/ipfs/boxo/chunker\"\n\t\"github.com/ipfs/boxo/files\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\timporter \"github.com/ipfs/boxo/ipld/unixfs/importer\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\t\"github.com/ipfs/boxo/path\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\t\"github.com/ipfs/go-test/random\"\n\tci \"github.com/libp2p/go-libp2p-testing/ci\"\n)\n\nfunc maybeSkipFuseTests(t *testing.T) {\n\tif ci.NoFuse() {\n\t\tt.Skip(\"Skipping FUSE tests\")\n\t}\n}\n\nfunc randObj(t *testing.T, nd *core.IpfsNode, size int64) (ipld.Node, []byte) {\n\tbuf := make([]byte, size)\n\t_, err := io.ReadFull(random.NewRand(), buf)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tread := bytes.NewReader(buf)\n\tobj, err := importer.BuildTrickleDagFromReader(nd.DAG, chunker.DefaultSplitter(read))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treturn obj, buf\n}\n\nfunc setupIpfsTest(t *testing.T, node *core.IpfsNode) (*core.IpfsNode, *fstest.Mount) {\n\tt.Helper()\n\tmaybeSkipFuseTests(t)\n\n\tvar err error\n\tif node == nil {\n\t\tnode, err = coremock.NewMockNode()\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tfs := NewFileSystem(node)\n\tmnt, err := fstest.MountedT(t, fs, nil)\n\tif err == fuse.ErrOSXFUSENotFound {\n\t\tt.Skip(err)\n\t}\n\tif err != nil {\n\t\tt.Fatalf(\"error mounting temporary directory: %v\", err)\n\t}\n\n\treturn node, mnt\n}\n\n// Test writing an object and reading it back through fuse.\nfunc TestIpfsBasicRead(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\tnd, mnt := setupIpfsTest(t, nil)\n\tdefer mnt.Close()\n\n\tfi, data := randObj(t, nd, 10000)\n\tk := fi.Cid()\n\tfname := gopath.Join(mnt.Dir, k.String())\n\trbuf, err := os.ReadFile(fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatal(\"Incorrect Read!\")\n\t}\n}\n\nfunc getPaths(t *testing.T, ipfs *core.IpfsNode, name string, n *dag.ProtoNode) []string {\n\tif len(n.Links()) == 0 {\n\t\treturn []string{name}\n\t}\n\tvar out []string\n\tfor _, lnk := range n.Links() {\n\t\tchild, err := lnk.GetNode(ipfs.Context(), ipfs.DAG)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tchildpb, ok := child.(*dag.ProtoNode)\n\t\tif !ok {\n\t\t\tt.Fatal(dag.ErrNotProtobuf)\n\t\t}\n\n\t\tsub := getPaths(t, ipfs, gopath.Join(name, lnk.Name), childpb)\n\t\tout = append(out, sub...)\n\t}\n\treturn out\n}\n\n// Perform a large number of concurrent reads to stress the system.\nfunc TestIpfsStressRead(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\tnd, mnt := setupIpfsTest(t, nil)\n\tdefer mnt.Close()\n\n\tapi, err := coreapi.NewCoreAPI(nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar nodes []ipld.Node\n\tvar paths []string\n\n\tnobj := 50\n\tndiriter := 50\n\n\t// Make a bunch of objects\n\tfor range nobj {\n\t\tfi, _ := randObj(t, nd, rand.Int63n(50000))\n\t\tnodes = append(nodes, fi)\n\t\tpaths = append(paths, fi.Cid().String())\n\t}\n\n\t// Now make a bunch of dirs\n\tfor range ndiriter {\n\t\tdb, err := uio.NewDirectory(nd.DAG)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tfor j := 0; j < 1+rand.Intn(10); j++ {\n\t\t\tname := fmt.Sprintf(\"child%d\", j)\n\n\t\t\terr := db.AddChild(nd.Context(), name, nodes[rand.Intn(len(nodes))])\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tnewdir, err := db.GetNode()\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\terr = nd.DAG.Add(nd.Context(), newdir)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tnodes = append(nodes, newdir)\n\t\tnpaths := getPaths(t, nd, newdir.Cid().String(), newdir.(*dag.ProtoNode))\n\t\tpaths = append(paths, npaths...)\n\t}\n\n\t// Now read a bunch, concurrently\n\twg := sync.WaitGroup{}\n\terrs := make(chan error)\n\n\tfor range 4 {\n\t\twg.Go(func() {\n\n\t\t\tfor range 2000 {\n\t\t\t\titem, err := path.NewPath(\"/ipfs/\" + paths[rand.Intn(len(paths))])\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs <- err\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\trelpath := strings.Replace(item.String(), item.Namespace(), \"\", 1)\n\t\t\t\tfname := gopath.Join(mnt.Dir, relpath)\n\n\t\t\t\trbuf, err := os.ReadFile(fname)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs <- err\n\t\t\t\t}\n\n\t\t\t\t// nd.Context() is never closed which leads to\n\t\t\t\t// hitting 8128 goroutine limit in go test -race mode\n\t\t\t\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\t\t\t\tread, err := api.Unixfs().Get(ctx, item)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs <- err\n\t\t\t\t}\n\n\t\t\t\tdata, err := io.ReadAll(read.(files.File))\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs <- err\n\t\t\t\t}\n\n\t\t\t\tcancelFunc()\n\n\t\t\t\tif !bytes.Equal(rbuf, data) {\n\t\t\t\t\terrs <- errors.New(\"incorrect read\")\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n\n\tgo func() {\n\t\twg.Wait()\n\t\tclose(errs)\n\t}()\n\n\tfor err := range errs {\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n}\n\n// Test writing a file and reading it back.\nfunc TestIpfsBasicDirRead(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\tnd, mnt := setupIpfsTest(t, nil)\n\tdefer mnt.Close()\n\n\t// Make a 'file'\n\tfi, data := randObj(t, nd, 10000)\n\n\t// Make a directory and put that file in it\n\tdb, err := uio.NewDirectory(nd.DAG)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\terr = db.AddChild(nd.Context(), \"actual\", fi)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\td1nd, err := db.GetNode()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = nd.DAG.Add(nd.Context(), d1nd)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdirname := gopath.Join(mnt.Dir, d1nd.Cid().String())\n\tfname := gopath.Join(dirname, \"actual\")\n\trbuf, err := os.ReadFile(fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdirents, err := os.ReadDir(dirname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(dirents) != 1 {\n\t\tt.Fatal(\"Bad directory entry count\")\n\t}\n\tif dirents[0].Name() != \"actual\" {\n\t\tt.Fatal(\"Bad directory entry\")\n\t}\n\n\tif !bytes.Equal(rbuf, data) {\n\t\tt.Fatal(\"Incorrect Read!\")\n\t}\n}\n\n// Test to make sure the filesystem reports file sizes correctly.\nfunc TestFileSizeReporting(t *testing.T) {\n\tif testing.Short() {\n\t\tt.SkipNow()\n\t}\n\tnd, mnt := setupIpfsTest(t, nil)\n\tdefer mnt.Close()\n\n\tfi, data := randObj(t, nd, 10000)\n\tk := fi.Cid()\n\n\tfname := gopath.Join(mnt.Dir, k.String())\n\n\tfinfo, err := os.Stat(fname)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif finfo.Size() != int64(len(data)) {\n\t\tt.Fatal(\"Read incorrect size from stat!\")\n\t}\n}\n" }, { "path": "fuse/readonly/mount_unix.go", "content": "//go:build (linux || darwin || freebsd) && !nofuse\n\npackage readonly\n\nimport (\n\tcore \"github.com/ipfs/kubo/core\"\n\tmount \"github.com/ipfs/kubo/fuse/mount\"\n)\n\n// Mount mounts IPFS at a given location, and returns a mount.Mount instance.\nfunc Mount(ipfs *core.IpfsNode, mountpoint string) (mount.Mount, error) {\n\tcfg, err := ipfs.Repo.Config()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tallowOther := cfg.Mounts.FuseAllowOther\n\tfsys := NewFileSystem(ipfs)\n\treturn mount.NewMount(fsys, mountpoint, allowOther)\n}\n" }, { "path": "fuse/readonly/readonly_unix.go", "content": "//go:build (linux || darwin || freebsd) && !nofuse\n\npackage readonly\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"syscall\"\n\n\tfuse \"bazil.org/fuse\"\n\tfs \"bazil.org/fuse/fs\"\n\tmdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\tuio \"github.com/ipfs/boxo/ipld/unixfs/io\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/go-cid\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tcore \"github.com/ipfs/kubo/core\"\n\tipldprime \"github.com/ipld/go-ipld-prime\"\n\tcidlink \"github.com/ipld/go-ipld-prime/linking/cid\"\n)\n\nvar log = logging.Logger(\"fuse/ipfs\")\n\n// FileSystem is the readonly IPFS Fuse Filesystem.\ntype FileSystem struct {\n\tIpfs *core.IpfsNode\n}\n\n// NewFileSystem constructs new fs using given core.IpfsNode instance.\nfunc NewFileSystem(ipfs *core.IpfsNode) *FileSystem {\n\treturn &FileSystem{Ipfs: ipfs}\n}\n\n// Root constructs the Root of the filesystem, a Root object.\nfunc (f FileSystem) Root() (fs.Node, error) {\n\treturn &Root{Ipfs: f.Ipfs}, nil\n}\n\n// Root is the root object of the filesystem tree.\ntype Root struct {\n\tIpfs *core.IpfsNode\n}\n\n// Attr returns file attributes.\nfunc (*Root) Attr(ctx context.Context, a *fuse.Attr) error {\n\ta.Mode = os.ModeDir | 0o111 // -rw+x\n\treturn nil\n}\n\n// Lookup performs a lookup under this node.\nfunc (s *Root) Lookup(ctx context.Context, name string) (fs.Node, error) {\n\tlog.Debugf(\"Root Lookup: '%s'\", name)\n\tswitch name {\n\tcase \"mach_kernel\", \".hidden\", \"._.\":\n\t\t// Just quiet some log noise on OS X.\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tp, err := path.NewPath(\"/ipfs/\" + name)\n\tif err != nil {\n\t\tlog.Debugf(\"fuse failed to parse path: %q: %s\", name, err)\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\timPath, err := path.NewImmutablePath(p)\n\tif err != nil {\n\t\tlog.Debugf(\"fuse failed to convert path: %q: %s\", name, err)\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tnd, ndLnk, err := s.Ipfs.UnixFSPathResolver.ResolvePath(ctx, imPath)\n\tif err != nil {\n\t\t// todo: make this error more versatile.\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tcidLnk, ok := ndLnk.(cidlink.Link)\n\tif !ok {\n\t\tlog.Debugf(\"non-cidlink returned from ResolvePath: %v\", ndLnk)\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\t// convert ipld-prime node to universal node\n\tblk, err := s.Ipfs.Blockstore.Get(ctx, cidLnk.Cid)\n\tif err != nil {\n\t\tlog.Debugf(\"fuse failed to retrieve block: %v: %s\", cidLnk, err)\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\tvar fnd ipld.Node\n\tswitch cidLnk.Cid.Prefix().Codec {\n\tcase cid.DagProtobuf:\n\t\tadl, ok := nd.(ipldprime.ADL)\n\t\tif ok {\n\t\t\tsubstrate := adl.Substrate()\n\t\t\tfnd, err = mdag.ProtoNodeConverter(blk, substrate)\n\t\t} else {\n\t\t\tfnd, err = mdag.ProtoNodeConverter(blk, nd)\n\t\t}\n\tcase cid.Raw:\n\t\tfnd, err = mdag.RawNodeConverter(blk, nd)\n\tdefault:\n\t\tlog.Error(\"fuse node was not a supported type\")\n\t\treturn nil, syscall.Errno(syscall.ENOTSUP)\n\t}\n\tif err != nil {\n\t\tlog.Errorf(\"could not convert protobuf or raw node: %s\", err)\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\t}\n\n\treturn &Node{Ipfs: s.Ipfs, Nd: fnd}, nil\n}\n\n// ReadDirAll reads a particular directory. Disallowed for root.\nfunc (*Root) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {\n\tlog.Debug(\"read Root\")\n\treturn nil, syscall.Errno(syscall.EPERM)\n}\n\n// Node is the core object representing a filesystem tree node.\ntype Node struct {\n\tIpfs *core.IpfsNode\n\tNd ipld.Node\n\tcached *ft.FSNode\n}\n\nfunc (s *Node) loadData() error {\n\tif pbnd, ok := s.Nd.(*mdag.ProtoNode); ok {\n\t\tfsn, err := ft.FSNodeFromBytes(pbnd.Data())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\ts.cached = fsn\n\t}\n\treturn nil\n}\n\n// Attr returns the attributes of a given node.\nfunc (s *Node) Attr(ctx context.Context, a *fuse.Attr) error {\n\tlog.Debug(\"Node attr\")\n\tif rawnd, ok := s.Nd.(*mdag.RawNode); ok {\n\t\ta.Mode = 0o444\n\t\ta.Size = uint64(len(rawnd.RawData()))\n\t\ta.Blocks = 1\n\t\treturn nil\n\t}\n\n\tif s.cached == nil {\n\t\tif err := s.loadData(); err != nil {\n\t\t\treturn fmt.Errorf(\"readonly: loadData() failed: %s\", err)\n\t\t}\n\t}\n\tswitch s.cached.Type() {\n\tcase ft.TDirectory, ft.THAMTShard:\n\t\ta.Mode = os.ModeDir | 0o555\n\tcase ft.TFile:\n\t\tsize := s.cached.FileSize()\n\t\ta.Mode = 0o444\n\t\ta.Size = uint64(size)\n\t\ta.Blocks = uint64(len(s.Nd.Links()))\n\tcase ft.TRaw:\n\t\ta.Mode = 0o444\n\t\ta.Size = uint64(len(s.cached.Data()))\n\t\ta.Blocks = uint64(len(s.Nd.Links()))\n\tcase ft.TSymlink:\n\t\ta.Mode = 0o777 | os.ModeSymlink\n\t\ta.Size = uint64(len(s.cached.Data()))\n\tdefault:\n\t\treturn fmt.Errorf(\"invalid data type - %s\", s.cached.Type())\n\t}\n\treturn nil\n}\n\n// Lookup performs a lookup under this node.\nfunc (s *Node) Lookup(ctx context.Context, name string) (fs.Node, error) {\n\tlog.Debugf(\"Lookup '%s'\", name)\n\tlink, _, err := uio.ResolveUnixfsOnce(ctx, s.Ipfs.DAG, s.Nd, []string{name})\n\tswitch err {\n\tcase os.ErrNotExist, mdag.ErrLinkNotFound:\n\t\t// todo: make this error more versatile.\n\t\treturn nil, syscall.Errno(syscall.ENOENT)\n\tcase nil:\n\t\t// noop\n\tdefault:\n\t\tlog.Errorf(\"fuse lookup %q: %s\", name, err)\n\t\treturn nil, syscall.Errno(syscall.EIO)\n\t}\n\n\tnd, err := s.Ipfs.DAG.Get(ctx, link.Cid)\n\tif err != nil && !ipld.IsNotFound(err) {\n\t\tlog.Errorf(\"fuse lookup %q: %s\", name, err)\n\t\treturn nil, err\n\t}\n\n\treturn &Node{Ipfs: s.Ipfs, Nd: nd}, nil\n}\n\n// ReadDirAll reads the link structure as directory entries.\nfunc (s *Node) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {\n\tlog.Debug(\"Node ReadDir\")\n\tdir, err := uio.NewDirectoryFromNode(s.Ipfs.DAG, s.Nd)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar entries []fuse.Dirent\n\terr = dir.ForEachLink(ctx, func(lnk *ipld.Link) error {\n\t\tn := lnk.Name\n\t\tif len(n) == 0 {\n\t\t\tn = lnk.Cid.String()\n\t\t}\n\t\tnd, err := s.Ipfs.DAG.Get(ctx, lnk.Cid)\n\t\tif err != nil {\n\t\t\tlog.Warn(\"error fetching directory child node: \", err)\n\t\t}\n\n\t\tt := fuse.DT_Unknown\n\t\tswitch nd := nd.(type) {\n\t\tcase *mdag.RawNode:\n\t\t\tt = fuse.DT_File\n\t\tcase *mdag.ProtoNode:\n\t\t\tif fsn, err := ft.FSNodeFromBytes(nd.Data()); err != nil {\n\t\t\t\tlog.Warn(\"failed to unmarshal protonode data field:\", err)\n\t\t\t} else {\n\t\t\t\tswitch fsn.Type() {\n\t\t\t\tcase ft.TDirectory, ft.THAMTShard:\n\t\t\t\t\tt = fuse.DT_Dir\n\t\t\t\tcase ft.TFile, ft.TRaw:\n\t\t\t\t\tt = fuse.DT_File\n\t\t\t\tcase ft.TSymlink:\n\t\t\t\t\tt = fuse.DT_Link\n\t\t\t\tcase ft.TMetadata:\n\t\t\t\t\tlog.Error(\"metadata object in fuse should contain its wrapped type\")\n\t\t\t\tdefault:\n\t\t\t\t\tlog.Error(\"unrecognized protonode data type: \", fsn.Type())\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tentries = append(entries, fuse.Dirent{Name: n, Type: t})\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif len(entries) > 0 {\n\t\treturn entries, nil\n\t}\n\treturn nil, syscall.Errno(syscall.ENOENT)\n}\n\nfunc (s *Node) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error {\n\t// TODO: is nil the right response for 'bug off, we ain't got none' ?\n\tresp.Xattr = nil\n\treturn nil\n}\n\nfunc (s *Node) Readlink(ctx context.Context, req *fuse.ReadlinkRequest) (string, error) {\n\tif s.cached == nil || s.cached.Type() != ft.TSymlink {\n\t\treturn \"\", fuse.Errno(syscall.EINVAL)\n\t}\n\treturn string(s.cached.Data()), nil\n}\n\nfunc (s *Node) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {\n\tr, err := uio.NewDagReader(ctx, s.Nd, s.Ipfs.DAG)\n\tif err != nil {\n\t\treturn err\n\t}\n\t_, err = r.Seek(req.Offset, io.SeekStart)\n\tif err != nil {\n\t\treturn err\n\t}\n\t// Data has a capacity of Size\n\tbuf := resp.Data[:int(req.Size)]\n\tn, err := io.ReadFull(r, buf)\n\tresp.Data = buf[:n]\n\tswitch err {\n\tcase nil, io.EOF, io.ErrUnexpectedEOF:\n\tdefault:\n\t\treturn err\n\t}\n\tresp.Data = resp.Data[:n]\n\treturn nil // may be non-nil / not succeeded\n}\n\n// to check that our Node implements all the interfaces we want.\ntype roRoot interface {\n\tfs.Node\n\tfs.HandleReadDirAller\n\tfs.NodeStringLookuper\n}\n\nvar _ roRoot = (*Root)(nil)\n\ntype roNode interface {\n\tfs.HandleReadDirAller\n\tfs.HandleReader\n\tfs.Node\n\tfs.NodeStringLookuper\n\tfs.NodeReadlinker\n\tfs.NodeGetxattrer\n}\n\nvar _ roNode = (*Node)(nil)\n" }, { "path": "gc/gc.go", "content": "// Package gc provides garbage collection for go-ipfs.\npackage gc\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\n\tbserv \"github.com/ipfs/boxo/blockservice\"\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\toffline \"github.com/ipfs/boxo/exchange/offline\"\n\tdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/boxo/verifcid\"\n\tcid \"github.com/ipfs/go-cid\"\n\tdstore \"github.com/ipfs/go-datastore\"\n\tipld \"github.com/ipfs/go-ipld-format\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n)\n\nvar log = logging.Logger(\"gc\")\n\n// Result represents an incremental output from a garbage collection\n// run. It contains either an error, or the cid of a removed object.\ntype Result struct {\n\tKeyRemoved cid.Cid\n\tError error\n}\n\n// converts a set of CIDs with different codecs to a set of CIDs with the raw codec.\nfunc toRawCids(set *cid.Set) (*cid.Set, error) {\n\tnewSet := cid.NewSet()\n\terr := set.ForEach(func(c cid.Cid) error {\n\t\tnewSet.Add(cid.NewCidV1(cid.Raw, c.Hash()))\n\t\treturn nil\n\t})\n\treturn newSet, err\n}\n\n// GC performs a mark and sweep garbage collection of the blocks in the blockstore\n// first, it creates a 'marked' set and adds to it the following:\n// - all recursively pinned blocks, plus all of their descendants (recursively)\n// - bestEffortRoots, plus all of its descendants (recursively)\n// - all directly pinned blocks\n// - all blocks utilized internally by the pinner\n//\n// The routine then iterates over every block in the blockstore and\n// deletes any block that is not found in the marked set.\nfunc GC(ctx context.Context, bs bstore.GCBlockstore, dstor dstore.Datastore, pn pin.Pinner, bestEffortRoots []cid.Cid) <-chan Result {\n\tctx, cancel := context.WithCancel(ctx)\n\n\tunlocker := bs.GCLock(ctx)\n\n\tbsrv := bserv.New(bs, offline.Exchange(bs))\n\tds := dag.NewDAGService(bsrv)\n\n\toutput := make(chan Result, 128)\n\n\tgo func() {\n\t\tdefer cancel()\n\t\tdefer close(output)\n\t\tdefer unlocker.Unlock(ctx)\n\n\t\tgcs, err := ColoredSet(ctx, pn, ds, bestEffortRoots, output)\n\t\tif err != nil {\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: err}:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t\treturn\n\t\t}\n\n\t\t// The blockstore reports raw blocks. We need to remove the codecs from the CIDs.\n\t\tgcs, err = toRawCids(gcs)\n\t\tif err != nil {\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: err}:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t\treturn\n\t\t}\n\n\t\tkeychain, err := bs.AllKeysChan(ctx)\n\t\tif err != nil {\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: err}:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t\treturn\n\t\t}\n\n\t\terrors := false\n\t\tvar removed uint64\n\n\tloop:\n\t\tfor ctx.Err() == nil { // select may not notice that we're \"done\".\n\t\t\tselect {\n\t\t\tcase k, ok := <-keychain:\n\t\t\t\tif !ok {\n\t\t\t\t\tbreak loop\n\t\t\t\t}\n\t\t\t\t// NOTE: assumes that all CIDs returned by the keychain are _raw_ CIDv1 CIDs.\n\t\t\t\t// This means we keep the block as long as we want it somewhere (CIDv1, CIDv0, Raw, other...).\n\t\t\t\tif !gcs.Has(k) {\n\t\t\t\t\terr := bs.DeleteBlock(ctx, k)\n\t\t\t\t\tremoved++\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrors = true\n\t\t\t\t\t\tselect {\n\t\t\t\t\t\tcase output <- Result{Error: &CannotDeleteBlockError{k, err}}:\n\t\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\t\tbreak loop\n\t\t\t\t\t\t}\n\t\t\t\t\t\t// continue as error is non-fatal\n\t\t\t\t\t\tcontinue loop\n\t\t\t\t\t}\n\t\t\t\t\tselect {\n\t\t\t\t\tcase output <- Result{KeyRemoved: k}:\n\t\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\t\tbreak loop\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\tcase <-ctx.Done():\n\t\t\t\tbreak loop\n\t\t\t}\n\t\t}\n\t\tif errors {\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: ErrCannotDeleteSomeBlocks}:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\n\t\tgds, ok := dstor.(dstore.GCDatastore)\n\t\tif !ok {\n\t\t\treturn\n\t\t}\n\n\t\terr = gds.CollectGarbage(ctx)\n\t\tif err != nil {\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: err}:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t\treturn\n\t\t}\n\t}()\n\n\treturn output\n}\n\n// Descendants recursively finds all the descendants of the given roots and\n// adds them to the given cid.Set, using the provided dag.GetLinks function\n// to walk the tree.\nfunc Descendants(ctx context.Context, getLinks dag.GetLinks, set *cid.Set, roots <-chan pin.StreamedPin) error {\n\tverifyGetLinks := func(ctx context.Context, c cid.Cid) ([]*ipld.Link, error) {\n\t\terr := verifcid.ValidateCid(verifcid.DefaultAllowlist, c)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn getLinks(ctx, c)\n\t}\n\n\tverboseCidError := func(err error) error {\n\t\tif strings.Contains(err.Error(), verifcid.ErrDigestTooSmall.Error()) ||\n\t\t\tstrings.Contains(err.Error(), verifcid.ErrPossiblyInsecureHashFunction.Error()) {\n\t\t\terr = fmt.Errorf(\"\\\"%s\\\"\\nPlease run 'ipfs pin verify'\"+ // nolint\n\t\t\t\t\" to list insecure hashes. If you want to read them,\"+\n\t\t\t\t\" please downgrade your go-ipfs to 0.4.13\\n\", err)\n\t\t\tlog.Error(err)\n\t\t}\n\t\treturn err\n\t}\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\tcase wrapper, ok := <-roots:\n\t\t\tif !ok {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tif wrapper.Err != nil {\n\t\t\t\treturn wrapper.Err\n\t\t\t}\n\n\t\t\t// Walk recursively walks the dag and adds the keys to the given set\n\t\t\terr := dag.Walk(ctx, verifyGetLinks, wrapper.Pin.Key, func(k cid.Cid) bool {\n\t\t\t\treturn set.Visit(toCidV1(k))\n\t\t\t}, dag.Concurrent())\n\t\t\tif err != nil {\n\t\t\t\terr = verboseCidError(err)\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n}\n\n// toCidV1 converts any CIDv0s to CIDv1s.\nfunc toCidV1(c cid.Cid) cid.Cid {\n\tif c.Version() == 0 {\n\t\treturn cid.NewCidV1(c.Type(), c.Hash())\n\t}\n\treturn c\n}\n\n// ColoredSet computes the set of nodes in the graph that are pinned by the\n// pins in the given pinner.\nfunc ColoredSet(ctx context.Context, pn pin.Pinner, ng ipld.NodeGetter, bestEffortRoots []cid.Cid, output chan<- Result) (*cid.Set, error) {\n\t// KeySet currently implemented in memory, in the future, may be bloom filter or\n\t// disk backed to conserve memory.\n\terrors := false\n\tgcs := cid.NewSet()\n\tgetLinks := func(ctx context.Context, cid cid.Cid) ([]*ipld.Link, error) {\n\t\tlinks, err := ipld.GetLinks(ctx, ng, cid)\n\t\tif err != nil {\n\t\t\terrors = true\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: &CannotFetchLinksError{cid, err}}:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn nil, ctx.Err()\n\t\t\t}\n\t\t}\n\t\treturn links, nil\n\t}\n\trkeys := pn.RecursiveKeys(ctx, false)\n\terr := Descendants(ctx, getLinks, gcs, rkeys)\n\tif err != nil {\n\t\terrors = true\n\t\tselect {\n\t\tcase output <- Result{Error: err}:\n\t\tcase <-ctx.Done():\n\t\t\treturn nil, ctx.Err()\n\t\t}\n\t}\n\n\tbestEffortGetLinks := func(ctx context.Context, cid cid.Cid) ([]*ipld.Link, error) {\n\t\tlinks, err := ipld.GetLinks(ctx, ng, cid)\n\t\tif err != nil && !ipld.IsNotFound(err) {\n\t\t\terrors = true\n\t\t\tselect {\n\t\t\tcase output <- Result{Error: &CannotFetchLinksError{cid, err}}:\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn nil, ctx.Err()\n\t\t\t}\n\t\t}\n\t\treturn links, nil\n\t}\n\tbestEffortRootsChan := make(chan pin.StreamedPin)\n\tgo func() {\n\t\tdefer close(bestEffortRootsChan)\n\t\tfor _, root := range bestEffortRoots {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase bestEffortRootsChan <- pin.StreamedPin{Pin: pin.Pinned{Key: root}}:\n\t\t\t}\n\t\t}\n\t}()\n\terr = Descendants(ctx, bestEffortGetLinks, gcs, bestEffortRootsChan)\n\tif err != nil {\n\t\terrors = true\n\t\tselect {\n\t\tcase output <- Result{Error: err}:\n\t\tcase <-ctx.Done():\n\t\t\treturn nil, ctx.Err()\n\t\t}\n\t}\n\n\tdkeys := pn.DirectKeys(ctx, false)\n\tfor k := range dkeys {\n\t\tif k.Err != nil {\n\t\t\treturn nil, k.Err\n\t\t}\n\t\tgcs.Add(toCidV1(k.Pin.Key))\n\t}\n\n\tikeys := pn.InternalPins(ctx, false)\n\terr = Descendants(ctx, getLinks, gcs, ikeys)\n\tif err != nil {\n\t\terrors = true\n\t\tselect {\n\t\tcase output <- Result{Error: err}:\n\t\tcase <-ctx.Done():\n\t\t\treturn nil, ctx.Err()\n\t\t}\n\t}\n\n\tif errors {\n\t\treturn nil, ErrCannotFetchAllLinks\n\t}\n\n\treturn gcs, nil\n}\n\n// ErrCannotFetchAllLinks is returned as the last Result in the GC output\n// channel when there was an error creating the marked set because of a\n// problem when finding descendants.\nvar ErrCannotFetchAllLinks = errors.New(\"garbage collection aborted: could not retrieve some links\")\n\n// ErrCannotDeleteSomeBlocks is returned when removing blocks marked for\n// deletion fails as the last Result in GC output channel.\nvar ErrCannotDeleteSomeBlocks = errors.New(\"garbage collection incomplete: could not delete some blocks\")\n\n// CannotFetchLinksError provides detailed information about which links\n// could not be fetched and can appear as a Result in the GC output channel.\ntype CannotFetchLinksError struct {\n\tKey cid.Cid\n\tErr error\n}\n\n// Error implements the error interface for this type with a useful\n// message.\nfunc (e *CannotFetchLinksError) Error() string {\n\treturn fmt.Sprintf(\"could not retrieve links for %s: %s\", e.Key, e.Err)\n}\n\n// CannotDeleteBlockError provides detailed information about which\n// blocks could not be deleted and can appear as a Result in the GC output\n// channel.\ntype CannotDeleteBlockError struct {\n\tKey cid.Cid\n\tErr error\n}\n\n// Error implements the error interface for this type with a\n// useful message.\nfunc (e *CannotDeleteBlockError) Error() string {\n\treturn fmt.Sprintf(\"could not remove %s: %s\", e.Key, e.Err)\n}\n" }, { "path": "gc/gc_test.go", "content": "package gc\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/blockservice\"\n\t\"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/exchange/offline\"\n\t\"github.com/ipfs/boxo/ipld/merkledag\"\n\tmdutils \"github.com/ipfs/boxo/ipld/merkledag/test\"\n\tpin \"github.com/ipfs/boxo/pinning/pinner\"\n\t\"github.com/ipfs/boxo/pinning/pinner/dspinner\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-datastore\"\n\tdssync \"github.com/ipfs/go-datastore/sync\"\n\t\"github.com/multiformats/go-multihash\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGC(t *testing.T) {\n\tctx := context.Background()\n\n\tds := dssync.MutexWrap(datastore.NewMapDatastore())\n\tbs := blockstore.NewGCBlockstore(blockstore.NewBlockstore(ds), blockstore.NewGCLocker())\n\tbserv := blockservice.New(bs, offline.Exchange(bs))\n\tdserv := merkledag.NewDAGService(bserv)\n\tpinner, err := dspinner.New(ctx, ds, dserv)\n\trequire.NoError(t, err)\n\n\tdaggen := mdutils.NewDAGGenerator()\n\n\tvar expectedKept []multihash.Multihash\n\tvar expectedDiscarded []multihash.Multihash\n\n\t// add some pins\n\tfor range 5 {\n\t\t// direct\n\t\troot, _, err := daggen.MakeDagNode(dserv.Add, 0, 1)\n\t\trequire.NoError(t, err)\n\t\terr = pinner.PinWithMode(ctx, root, pin.Direct, \"\")\n\t\trequire.NoError(t, err)\n\t\texpectedKept = append(expectedKept, root.Hash())\n\n\t\t// recursive\n\t\troot, allCids, err := daggen.MakeDagNode(dserv.Add, 5, 2)\n\t\trequire.NoError(t, err)\n\t\terr = pinner.PinWithMode(ctx, root, pin.Recursive, \"\")\n\t\trequire.NoError(t, err)\n\t\texpectedKept = append(expectedKept, toMHs(allCids)...)\n\t}\n\n\terr = pinner.Flush(ctx)\n\trequire.NoError(t, err)\n\n\t// add more dags to be GCed\n\tfor range 5 {\n\t\t_, allCids, err := daggen.MakeDagNode(dserv.Add, 5, 2)\n\t\trequire.NoError(t, err)\n\t\texpectedDiscarded = append(expectedDiscarded, toMHs(allCids)...)\n\t}\n\n\t// and some other as \"best effort roots\"\n\tvar bestEffortRoots []cid.Cid\n\tfor range 5 {\n\t\troot, allCids, err := daggen.MakeDagNode(dserv.Add, 5, 2)\n\t\trequire.NoError(t, err)\n\t\tbestEffortRoots = append(bestEffortRoots, root)\n\t\texpectedKept = append(expectedKept, toMHs(allCids)...)\n\t}\n\n\tch := GC(ctx, bs, ds, pinner, bestEffortRoots)\n\tvar discarded []multihash.Multihash\n\tfor res := range ch {\n\t\trequire.NoError(t, res.Error)\n\t\tdiscarded = append(discarded, res.KeyRemoved.Hash())\n\t}\n\n\tallKeys, err := bs.AllKeysChan(ctx)\n\trequire.NoError(t, err)\n\tvar kept []multihash.Multihash\n\tfor key := range allKeys {\n\t\tkept = append(kept, key.Hash())\n\t}\n\n\trequire.ElementsMatch(t, expectedDiscarded, discarded)\n\trequire.ElementsMatch(t, expectedKept, kept)\n}\n\nfunc toMHs(cids []cid.Cid) []multihash.Multihash {\n\tres := make([]multihash.Multihash, len(cids))\n\tfor i, c := range cids {\n\t\tres[i] = c.Hash()\n\t}\n\treturn res\n}\n" }, { "path": "go.mod", "content": "module github.com/ipfs/kubo\n\ngo 1.25.7\n\nrequire (\n\tbazil.org/fuse v0.0.0-20200117225306-7b5117fecadc\n\tcontrib.go.opencensus.io/exporter/prometheus v0.4.2\n\tgithub.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239\n\tgithub.com/blang/semver/v4 v4.0.0\n\tgithub.com/caddyserver/certmagic v0.23.0\n\tgithub.com/cenkalti/backoff/v4 v4.3.0\n\tgithub.com/ceramicnetwork/go-dag-jose v0.1.1\n\tgithub.com/cheggaaa/pb v1.0.29\n\tgithub.com/cockroachdb/pebble/v2 v2.1.4\n\tgithub.com/coreos/go-systemd/v22 v22.7.0\n\tgithub.com/dustin/go-humanize v1.0.1\n\tgithub.com/elgris/jsondiff v0.0.0-20160530203242-765b5c24c302\n\tgithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5\n\tgithub.com/fsnotify/fsnotify v1.9.0\n\tgithub.com/google/uuid v1.6.0\n\tgithub.com/hashicorp/go-version v1.8.0\n\tgithub.com/ipfs-shipyard/nopfs v0.0.14\n\tgithub.com/ipfs-shipyard/nopfs/ipfs v0.25.0\n\tgithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422\n\tgithub.com/ipfs/go-block-format v0.2.3\n\tgithub.com/ipfs/go-cid v0.6.0\n\tgithub.com/ipfs/go-cidutil v0.1.1\n\tgithub.com/ipfs/go-datastore v0.9.1\n\tgithub.com/ipfs/go-detect-race v0.0.1\n\tgithub.com/ipfs/go-ds-badger v0.3.4\n\tgithub.com/ipfs/go-ds-flatfs v0.6.0\n\tgithub.com/ipfs/go-ds-leveldb v0.5.2\n\tgithub.com/ipfs/go-ds-measure v0.2.2\n\tgithub.com/ipfs/go-ds-pebble v0.5.9\n\tgithub.com/ipfs/go-fs-lock v0.1.1\n\tgithub.com/ipfs/go-ipfs-cmds v0.16.0\n\tgithub.com/ipfs/go-ipld-cbor v0.2.1\n\tgithub.com/ipfs/go-ipld-format v0.6.3\n\tgithub.com/ipfs/go-ipld-git v0.1.1\n\tgithub.com/ipfs/go-ipld-legacy v0.3.0\n\tgithub.com/ipfs/go-log/v2 v2.9.1\n\tgithub.com/ipfs/go-metrics-interface v0.3.0\n\tgithub.com/ipfs/go-metrics-prometheus v0.1.0\n\tgithub.com/ipfs/go-test v0.2.3\n\tgithub.com/ipfs/go-unixfsnode v1.10.3\n\tgithub.com/ipld/go-car/v2 v2.16.0\n\tgithub.com/ipld/go-codec-dagpb v1.7.0\n\tgithub.com/ipld/go-ipld-prime v0.22.0\n\tgithub.com/ipshipyard/p2p-forge v0.7.0\n\tgithub.com/jbenet/go-temp-err-catcher v0.1.0\n\tgithub.com/julienschmidt/httprouter v1.3.0\n\tgithub.com/libp2p/go-doh-resolver v0.5.0\n\tgithub.com/libp2p/go-libp2p v0.48.0\n\tgithub.com/libp2p/go-libp2p-http v0.5.0\n\tgithub.com/libp2p/go-libp2p-kad-dht v0.39.0\n\tgithub.com/libp2p/go-libp2p-kbucket v0.8.0\n\tgithub.com/libp2p/go-libp2p-pubsub v0.15.0\n\tgithub.com/libp2p/go-libp2p-pubsub-router v0.6.0\n\tgithub.com/libp2p/go-libp2p-record v0.3.1\n\tgithub.com/libp2p/go-libp2p-routing-helpers v0.7.5\n\tgithub.com/libp2p/go-libp2p-testing v0.12.0\n\tgithub.com/libp2p/go-socket-activation v0.1.1\n\tgithub.com/miekg/dns v1.1.72\n\tgithub.com/multiformats/go-multiaddr v0.16.1\n\tgithub.com/multiformats/go-multiaddr-dns v0.5.0\n\tgithub.com/multiformats/go-multibase v0.2.0\n\tgithub.com/multiformats/go-multicodec v0.10.0\n\tgithub.com/multiformats/go-multihash v0.2.3\n\tgithub.com/opentracing/opentracing-go v1.2.0\n\tgithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58\n\tgithub.com/probe-lab/go-libdht v0.4.0\n\tgithub.com/prometheus/client_golang v1.23.2\n\tgithub.com/stretchr/testify v1.11.1\n\tgithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d\n\tgithub.com/tidwall/gjson v1.18.0\n\tgithub.com/tidwall/sjson v1.2.5\n\tgithub.com/whyrusleeping/go-sysinfo v0.0.0-20190219211824-4a357d4b90b1\n\tgithub.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7\n\tgo.opencensus.io v0.24.0\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0\n\tgo.opentelemetry.io/contrib/propagators/autoprop v0.46.1\n\tgo.opentelemetry.io/otel v1.42.0\n\tgo.opentelemetry.io/otel/exporters/prometheus v0.56.0\n\tgo.opentelemetry.io/otel/sdk v1.40.0\n\tgo.opentelemetry.io/otel/sdk/metric v1.40.0\n\tgo.opentelemetry.io/otel/trace v1.42.0\n\tgo.uber.org/dig v1.19.0\n\tgo.uber.org/fx v1.24.0\n\tgo.uber.org/zap v1.27.1\n\tgolang.org/x/crypto v0.49.0\n\tgolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90\n\tgolang.org/x/mod v0.34.0\n\tgolang.org/x/sync v0.20.0\n\tgolang.org/x/sys v0.42.0\n\tgoogle.golang.org/protobuf v1.36.11\n)\n\nrequire (\n\tfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5 // indirect\n\tfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b // indirect\n\tgithub.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect\n\tgithub.com/DataDog/zstd v1.5.7 // indirect\n\tgithub.com/Jorropo/jsync v1.0.1 // indirect\n\tgithub.com/RaduBerinde/axisds v0.1.0 // indirect\n\tgithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 // indirect\n\tgithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect\n\tgithub.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 // indirect\n\tgithub.com/benbjohnson/clock v1.3.5 // indirect\n\tgithub.com/beorn7/perks v1.0.1 // indirect\n\tgithub.com/caddyserver/zerossl v0.1.3 // indirect\n\tgithub.com/cenkalti/backoff/v5 v5.0.3 // indirect\n\tgithub.com/cespare/xxhash v1.1.0 // indirect\n\tgithub.com/cespare/xxhash/v2 v2.3.0 // indirect\n\tgithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b // indirect\n\tgithub.com/cockroachdb/errors v1.11.3 // indirect\n\tgithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect\n\tgithub.com/cockroachdb/redact v1.1.5 // indirect\n\tgithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b // indirect\n\tgithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect\n\tgithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf // indirect\n\tgithub.com/cskr/pubsub v1.0.2 // indirect\n\tgithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect\n\tgithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect\n\tgithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect\n\tgithub.com/dgraph-io/badger v1.6.2 // indirect\n\tgithub.com/dgraph-io/ristretto v0.0.2 // indirect\n\tgithub.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect\n\tgithub.com/dunglas/httpsfv v1.1.0 // indirect\n\tgithub.com/fatih/color v1.15.0 // indirect\n\tgithub.com/felixge/httpsnoop v1.0.4 // indirect\n\tgithub.com/filecoin-project/go-clock v0.1.0 // indirect\n\tgithub.com/flynn/noise v1.1.0 // indirect\n\tgithub.com/gabriel-vasile/mimetype v1.4.13 // indirect\n\tgithub.com/gammazero/chanqueue v1.1.2 // indirect\n\tgithub.com/gammazero/deque v1.2.1 // indirect\n\tgithub.com/getsentry/sentry-go v0.27.0 // indirect\n\tgithub.com/go-jose/go-jose/v4 v4.1.3 // indirect\n\tgithub.com/go-kit/log v0.2.1 // indirect\n\tgithub.com/go-logfmt/logfmt v0.6.0 // indirect\n\tgithub.com/go-logr/logr v1.4.3 // indirect\n\tgithub.com/go-logr/stdr v1.2.2 // indirect\n\tgithub.com/gogo/protobuf v1.3.2 // indirect\n\tgithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect\n\tgithub.com/golang/protobuf v1.5.4 // indirect\n\tgithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e // indirect\n\tgithub.com/google/gopacket v1.1.19 // indirect\n\tgithub.com/gorilla/mux v1.8.1 // indirect\n\tgithub.com/gorilla/websocket v1.5.3 // indirect\n\tgithub.com/grpc-ecosystem/grpc-gateway/v2 v2.27.7 // indirect\n\tgithub.com/guillaumemichel/reservedpool v0.3.0 // indirect\n\tgithub.com/hashicorp/golang-lru v1.0.2 // indirect\n\tgithub.com/hashicorp/golang-lru/v2 v2.0.7 // indirect\n\tgithub.com/huin/goupnp v1.3.0 // indirect\n\tgithub.com/ipfs/bbloom v0.0.4 // indirect\n\tgithub.com/ipfs/go-bitfield v1.1.0 // indirect\n\tgithub.com/ipfs/go-dsqueue v0.2.0 // indirect\n\tgithub.com/ipfs/go-ipfs-ds-help v1.1.1 // indirect\n\tgithub.com/ipfs/go-ipfs-pq v0.0.4 // indirect\n\tgithub.com/ipfs/go-ipfs-redirects-file v0.1.2 // indirect\n\tgithub.com/ipfs/go-libdht v0.5.0 // indirect\n\tgithub.com/ipfs/go-peertaskqueue v0.8.3 // indirect\n\tgithub.com/jackpal/go-nat-pmp v1.0.2 // indirect\n\tgithub.com/klauspost/compress v1.18.0 // indirect\n\tgithub.com/klauspost/cpuid/v2 v2.3.0 // indirect\n\tgithub.com/koron/go-ssdp v0.0.6 // indirect\n\tgithub.com/kr/pretty v0.3.1 // indirect\n\tgithub.com/kr/text v0.2.0 // indirect\n\tgithub.com/libdns/libdns v1.0.0-beta.1 // indirect\n\tgithub.com/libp2p/go-buffer-pool v0.1.0 // indirect\n\tgithub.com/libp2p/go-cidranger v1.1.0 // indirect\n\tgithub.com/libp2p/go-flow-metrics v0.3.0 // indirect\n\tgithub.com/libp2p/go-libp2p-asn-util v0.4.1 // indirect\n\tgithub.com/libp2p/go-libp2p-gostream v0.6.0 // indirect\n\tgithub.com/libp2p/go-libp2p-xor v0.1.0 // indirect\n\tgithub.com/libp2p/go-msgio v0.3.0 // indirect\n\tgithub.com/libp2p/go-netroute v0.4.0 // indirect\n\tgithub.com/libp2p/go-reuseport v0.4.0 // indirect\n\tgithub.com/libp2p/go-yamux/v5 v5.0.1 // indirect\n\tgithub.com/libp2p/zeroconf/v2 v2.2.0 // indirect\n\tgithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd // indirect\n\tgithub.com/mattn/go-colorable v0.1.13 // indirect\n\tgithub.com/mattn/go-isatty v0.0.20 // indirect\n\tgithub.com/mattn/go-runewidth v0.0.15 // indirect\n\tgithub.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect\n\tgithub.com/mholt/acmez/v3 v3.1.2 // indirect\n\tgithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b // indirect\n\tgithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect\n\tgithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 // indirect\n\tgithub.com/minio/sha256-simd v1.0.1 // indirect\n\tgithub.com/mr-tron/base58 v1.2.0 // indirect\n\tgithub.com/multiformats/go-base32 v0.1.0 // indirect\n\tgithub.com/multiformats/go-base36 v0.2.0 // indirect\n\tgithub.com/multiformats/go-multiaddr-fmt v0.1.0 // indirect\n\tgithub.com/multiformats/go-multistream v0.6.1 // indirect\n\tgithub.com/multiformats/go-varint v0.1.0 // indirect\n\tgithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect\n\tgithub.com/onsi/gomega v1.36.3 // indirect\n\tgithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 // indirect\n\tgithub.com/pion/datachannel v1.5.10 // indirect\n\tgithub.com/pion/dtls/v3 v3.1.2 // indirect\n\tgithub.com/pion/ice/v4 v4.0.10 // indirect\n\tgithub.com/pion/interceptor v0.1.40 // indirect\n\tgithub.com/pion/logging v0.2.4 // indirect\n\tgithub.com/pion/mdns/v2 v2.0.7 // indirect\n\tgithub.com/pion/randutil v0.1.0 // indirect\n\tgithub.com/pion/rtcp v1.2.16 // indirect\n\tgithub.com/pion/rtp v1.8.19 // indirect\n\tgithub.com/pion/sctp v1.8.39 // indirect\n\tgithub.com/pion/sdp/v3 v3.0.18 // indirect\n\tgithub.com/pion/srtp/v3 v3.0.6 // indirect\n\tgithub.com/pion/stun/v3 v3.1.1 // indirect\n\tgithub.com/pion/transport/v3 v3.0.7 // indirect\n\tgithub.com/pion/transport/v4 v4.0.1 // indirect\n\tgithub.com/pion/turn/v4 v4.0.2 // indirect\n\tgithub.com/pion/webrtc/v4 v4.1.2 // indirect\n\tgithub.com/pkg/errors v0.9.1 // indirect\n\tgithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect\n\tgithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a // indirect\n\tgithub.com/prometheus/client_model v0.6.2 // indirect\n\tgithub.com/prometheus/common v0.67.5 // indirect\n\tgithub.com/prometheus/procfs v0.20.1 // indirect\n\tgithub.com/prometheus/statsd_exporter v0.27.1 // indirect\n\tgithub.com/quic-go/qpack v0.6.0 // indirect\n\tgithub.com/quic-go/quic-go v0.59.0 // indirect\n\tgithub.com/quic-go/webtransport-go v0.10.0 // indirect\n\tgithub.com/rivo/uniseg v0.4.4 // indirect\n\tgithub.com/rogpeppe/go-internal v1.14.1 // indirect\n\tgithub.com/rs/cors v1.11.1 // indirect\n\tgithub.com/slok/go-http-metrics v0.13.0 // indirect\n\tgithub.com/spaolacci/murmur3 v1.1.0 // indirect\n\tgithub.com/texttheater/golang-levenshtein v1.0.1 // indirect\n\tgithub.com/tidwall/match v1.1.1 // indirect\n\tgithub.com/tidwall/pretty v1.2.1 // indirect\n\tgithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb // indirect\n\tgithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc // indirect\n\tgithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11 // indirect\n\tgithub.com/whyrusleeping/cbor-gen v0.3.1 // indirect\n\tgithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f // indirect\n\tgithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 // indirect\n\tgithub.com/wlynxg/anet v0.0.5 // indirect\n\tgithub.com/zeebo/blake3 v0.2.4 // indirect\n\tgo.opentelemetry.io/auto/sdk v1.2.1 // indirect\n\tgo.opentelemetry.io/contrib/propagators/aws v1.21.1 // indirect\n\tgo.opentelemetry.io/contrib/propagators/b3 v1.21.1 // indirect\n\tgo.opentelemetry.io/contrib/propagators/jaeger v1.21.1 // indirect\n\tgo.opentelemetry.io/contrib/propagators/ot v1.21.1 // indirect\n\tgo.opentelemetry.io/otel/exporters/otlp/otlptrace v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/metric v1.42.0 // indirect\n\tgo.opentelemetry.io/proto/otlp v1.9.0 // indirect\n\tgo.uber.org/mock v0.5.2 // indirect\n\tgo.uber.org/multierr v1.11.0 // indirect\n\tgo.uber.org/zap/exp v0.3.0 // indirect\n\tgo.yaml.in/yaml/v2 v2.4.4 // indirect\n\tgo4.org v0.0.0-20230225012048-214862532bf5 // indirect\n\tgolang.org/x/net v0.52.0 // indirect\n\tgolang.org/x/oauth2 v0.35.0 // indirect\n\tgolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c // indirect\n\tgolang.org/x/term v0.41.0 // indirect\n\tgolang.org/x/text v0.35.0 // indirect\n\tgolang.org/x/time v0.12.0 // indirect\n\tgolang.org/x/tools v0.43.0 // indirect\n\tgolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect\n\tgonum.org/v1/gonum v0.17.0 // indirect\n\tgoogle.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 // indirect\n\tgoogle.golang.org/genproto/googleapis/rpc v0.0.0-20260128011058-8636f8732409 // indirect\n\tgoogle.golang.org/grpc v1.78.0 // indirect\n\tgopkg.in/yaml.v2 v2.4.0 // indirect\n\tgopkg.in/yaml.v3 v3.0.1 // indirect\n\tlukechampine.com/blake3 v1.4.1 // indirect\n)\n\n// Exclude ancient +incompatible versions that confuse Dependabot.\n\n// These pre-Go-modules versions reference packages that no longer exist.\nexclude (\n\tgithub.com/ipfs/go-ipfs-cmds v2.0.1+incompatible\n\tgithub.com/libp2p/go-libp2p v6.0.23+incompatible\n)\n" }, { "path": "go.sum", "content": "bazil.org/fuse v0.0.0-20200117225306-7b5117fecadc h1:utDghgcjE8u+EBjHOgYT+dJPcnDF05KqWMBcjuJy510=\nbazil.org/fuse v0.0.0-20200117225306-7b5117fecadc/go.mod h1:FbcW6z/2VytnFDhZfumh8Ss8zxHE6qpMP5sHTRe0EaM=\ncloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=\ncloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=\ncloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=\ncloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=\ncloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=\ncloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=\ncloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=\ncloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=\ncloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=\ncloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=\ncloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=\ncloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=\ncloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=\ncloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=\ncloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=\ncloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=\ncloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=\ncloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=\ncloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=\ncloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=\ncloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=\ncloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=\ncloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=\ncloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=\ncloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=\ncloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=\ncloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk=\ncloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=\ncloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=\ncloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=\ncloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=\ncloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=\ncloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=\ncloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=\ncloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=\ncloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=\ncontrib.go.opencensus.io/exporter/prometheus v0.4.2 h1:sqfsYl5GIY/L570iT+l93ehxaWJs2/OwXtiWwew3oAg=\ncontrib.go.opencensus.io/exporter/prometheus v0.4.2/go.mod h1:dvEHbiKmgvbr5pjaF9fpw1KeYcjrnC1J8B+JKjsZyRQ=\ndmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=\nfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5 h1:JA0fFr+kxpqTdxR9LOBiTWpGNchqmkcsgmdeJZRclZ0=\nfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5/go.mod h1:OjOXDNlClLblvXdwgFFOQFJEocLhhtai8vGLy0JCZlI=\nfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b h1:REI1FbdW71yO56Are4XAxD+OS/e+BQsB3gE4mZRQEXY=\nfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b/go.mod h1:9nnw1SlYHYuPSo/3wjQzNjSbeHlq2NsKo5iEtfJPWP0=\ngithub.com/AndreasBriese/bbloom v0.0.0-20190306092124-e2d15f34fcf9/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=\ngithub.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 h1:cTp8I5+VIoKjsnZuH8vjyaysT/ses3EvZeaV/1UkF2M=\ngithub.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=\ngithub.com/DataDog/zstd v1.5.7 h1:ybO8RBeh29qrxIhCA9E8gKY6xfONU9T6G6aP9DTKfLE=\ngithub.com/DataDog/zstd v1.5.7/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=\ngithub.com/Jorropo/jsync v1.0.1 h1:6HgRolFZnsdfzRUj+ImB9og1JYOxQoReSywkHOGSaUU=\ngithub.com/Jorropo/jsync v1.0.1/go.mod h1:jCOZj3vrBCri3bSU3ErUYvevKlnbssrXeCivybS5ABQ=\ngithub.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=\ngithub.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=\ngithub.com/RaduBerinde/axisds v0.1.0 h1:YItk/RmU5nvlsv/awo2Fjx97Mfpt4JfgtEVAGPrLdz8=\ngithub.com/RaduBerinde/axisds v0.1.0/go.mod h1:UHGJonU9z4YYGKJxSaC6/TNcLOBptpmM5m2Cksbnw0Y=\ngithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 h1:bsU8Tzxr/PNz75ayvCnxKZWEYdLMPDkUgticP4a4Bvk=\ngithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54/go.mod h1:0tr7FllbE9gJkHq7CVeeDDFAFKQVy5RnCSSNBOvdqbc=\ngithub.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f h1:JjxwchlOepwsUWcQwD2mLUAGE9aCp0/ehy6yCHFBOvo=\ngithub.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f/go.mod h1:tMDTce/yLLN/SK8gMOxQfnyeMeCg8KGzp0D1cbECEeo=\ngithub.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII=\ngithub.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=\ngithub.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=\ngithub.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=\ngithub.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=\ngithub.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=\ngithub.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137/go.mod h1:OMCwj8VM1Kc9e19TLln2VL61YJF0x1XFtfdL4JdbSyE=\ngithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0=\ngithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs=\ngithub.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 h1:iW0a5ljuFxkLGPNem5Ui+KBjFJzKg4Fv2fnxe4dvzpM=\ngithub.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5/go.mod h1:Y2QMoi1vgtOIfc+6DhrMOGkLoGzqSV2rKp4Sm+opsyA=\ngithub.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 h1:kFOfPq6dUM1hTo4JG6LR5AXSUEsOjtdm0kw0FtQtMJA=\ngithub.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=\ngithub.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=\ngithub.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=\ngithub.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=\ngithub.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=\ngithub.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=\ngithub.com/benbjohnson/clock v1.3.5 h1:VvXlSJBzZpA/zum6Sj74hxwYI2DIxRWuNIoXAzHZz5o=\ngithub.com/benbjohnson/clock v1.3.5/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=\ngithub.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=\ngithub.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=\ngithub.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM=\ngithub.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM=\ngithub.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ=\ngithub.com/btcsuite/btcd v0.0.0-20190824003749-130ea5bddde3/go.mod h1:3J08xEfcugPacsc34/LKRU2yO7YmuT8yt28J8k2+rrI=\ngithub.com/btcsuite/btcd v0.20.1-beta/go.mod h1:wVuoA8VJLEcwgqHBwHmzLRazpKxTv13Px/pDuV7OomQ=\ngithub.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA=\ngithub.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg=\ngithub.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg=\ngithub.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVaaLLH7j4eDXPRvw78tMflu7Ie2bzYOH4Y8rRKBY=\ngithub.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc=\ngithub.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY=\ngithub.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs=\ngithub.com/caddyserver/certmagic v0.23.0 h1:CfpZ/50jMfG4+1J/u2LV6piJq4HOfO6ppOnOf7DkFEU=\ngithub.com/caddyserver/certmagic v0.23.0/go.mod h1:9mEZIWqqWoI+Gf+4Trh04MOVPD0tGSxtqsxg87hAIH4=\ngithub.com/caddyserver/zerossl v0.1.3 h1:onS+pxp3M8HnHpN5MMbOMyNjmTheJyWRaZYwn+YTAyA=\ngithub.com/caddyserver/zerossl v0.1.3/go.mod h1:CxA0acn7oEGO6//4rtrRjYgEoa4MFw/XofZnrYwGqG4=\ngithub.com/canonical/go-sp800.90a-drbg v0.0.0-20210314144037-6eeb1040d6c3 h1:oe6fCvaEpkhyW3qAicT0TnGtyht/UrgvOwMcEgLb7Aw=\ngithub.com/canonical/go-sp800.90a-drbg v0.0.0-20210314144037-6eeb1040d6c3/go.mod h1:qdP0gaj0QtgX2RUZhnlVrceJ+Qln8aSlDyJwelLLFeM=\ngithub.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=\ngithub.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=\ngithub.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM=\ngithub.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/ceramicnetwork/go-dag-jose v0.1.1 h1:7pObs22egc14vSS3AfCFfS1VmaL4lQUsAK7OGC3PlKk=\ngithub.com/ceramicnetwork/go-dag-jose v0.1.1/go.mod h1:8ptnYwY2Z2y/s5oJnNBn/UCxLg6CpramNJ2ZXF/5aNY=\ngithub.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko=\ngithub.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=\ngithub.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=\ngithub.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/cheggaaa/pb v1.0.29 h1:FckUN5ngEk2LpvuG0fw1GEFx6LtyY2pWI/Z2QgCnEYo=\ngithub.com/cheggaaa/pb v1.0.29/go.mod h1:W40334L7FMC5JKWldsTWbdGjLo0RxUKK73K+TuPxX30=\ngithub.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=\ngithub.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=\ngithub.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b h1:SHlYZ/bMx7frnmeqCu+xm0TCxXLzX3jQIVuFbnFGtFU=\ngithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b/go.mod h1:Gq51ZeKaFCXk6QwuGM0w1dnaOqc/F5zKT2zA9D6Xeac=\ngithub.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5 h1:UycK/E0TkisVrQbSoxvU827FwgBBcZ95nRRmpj/12QI=\ngithub.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5/go.mod h1:jsaKMvD3RBCATk1/jbUZM8C9idWBJME9+VRZ5+Liq1g=\ngithub.com/cockroachdb/errors v1.11.3 h1:5bA+k2Y6r+oz/6Z/RFlNeVCesGARKuC6YymtcDrbC/I=\ngithub.com/cockroachdb/errors v1.11.3/go.mod h1:m4UIW4CDjx+R5cybPsNrRbreomiFqt8o1h1wUVazSd8=\ngithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZeQy818SGhaone5OnYfxFR/+AzdY3sf5aE=\ngithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=\ngithub.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895 h1:XANOgPYtvELQ/h4IrmPAohXqe2pWA8Bwhejr3VQoZsA=\ngithub.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895/go.mod h1:aPd7gM9ov9M8v32Yy5NJrDyOcD8z642dqs+F0CeNXfA=\ngithub.com/cockroachdb/pebble/v2 v2.1.4 h1:j9wPgMDbkErFdAKYFGhsoCcvzcjR+6zrJ4jhKtJ6bOk=\ngithub.com/cockroachdb/pebble/v2 v2.1.4/go.mod h1:Reo1RTniv1UjVTAu/Fv74y5i3kJ5gmVrPhO9UtFiKn8=\ngithub.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30=\ngithub.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=\ngithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b h1:VXvSNzmr8hMj8XTuY0PT9Ane9qZGul/p67vGYwl9BFI=\ngithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b/go.mod h1:yBRu/cnL4ks9bgy4vAASdjIW+/xMlFwuHKqtmh3GZQg=\ngithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo=\ngithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06/go.mod h1:7nc4anLGjupUW/PeY5qiNYsdNXj7zopG+eqsS7To5IQ=\ngithub.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=\ngithub.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=\ngithub.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=\ngithub.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA=\ngithub.com/coreos/go-systemd/v22 v22.7.0/go.mod h1:xNUYtjHu2EDXbsxz1i41wouACIwT7Ybq9o0BQhMwD0w=\ngithub.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf h1:dwGgBWn84wUS1pVikGiruW+x5XM4amhjaZO20vCjay4=\ngithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf/go.mod h1:p1d6YEZWvFzEh4KLyvBcVSnrfNDDvK2zfK/4x2v/4pE=\ngithub.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=\ngithub.com/cskr/pubsub v1.0.2 h1:vlOzMhl6PFn60gRlTQQsIfVwaPB/B/8MziK8FhEPt/0=\ngithub.com/cskr/pubsub v1.0.2/go.mod h1:/8MzYXk/NJAz782G8RPkFzXTZVu63VotefPnR9TIRis=\ngithub.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c h1:pFUpOrbxDR6AkioZ1ySsx5yxlDQZ8stG2b88gTPxgJU=\ngithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c/go.mod h1:6UhI8N9EjYm1c2odKpFpAYeR8dsBeM7PtzQhRgxRr9U=\ngithub.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U0x++OzVrdms8=\ngithub.com/decred/dcrd/crypto/blake256 v1.1.0/go.mod h1:2OfgNZ5wDpcsFmHmCK5gZTPcCXqlm2ArzUIkw9czNJo=\ngithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 h1:5RVFMOWjMyRy8cARdy79nAmgYw3hK/4HUq48LQ6Wwqo=\ngithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=\ngithub.com/dgraph-io/badger v1.6.0/go.mod h1:zwt7syl517jmP8s94KqSxTlM6IMsdhYy6psNgSztDR4=\ngithub.com/dgraph-io/badger v1.6.2 h1:mNw0qs90GVgGGWylh0umH5iag1j6n/PeJtNvL6KY/x8=\ngithub.com/dgraph-io/badger v1.6.2/go.mod h1:JW2yswe3V058sS0kZ2h/AXeDSqFjxnZcRrVH//y2UQE=\ngithub.com/dgraph-io/ristretto v0.0.2 h1:a5WaUrDa0qm0YrAAS1tUykT5El3kt62KNZZeMxQn3po=\ngithub.com/dgraph-io/ristretto v0.0.2/go.mod h1:KPxhHT9ZxKefz+PCeOGsrHpl1qZ7i70dGTu2u+Ahh6E=\ngithub.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=\ngithub.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 h1:fAjc9m62+UWV/WAFKLNi6ZS0675eEUC9y3AlwSbQu1Y=\ngithub.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=\ngithub.com/dunglas/httpsfv v1.1.0 h1:Jw76nAyKWKZKFrpMMcL76y35tOpYHqQPzHQiwDvpe54=\ngithub.com/dunglas/httpsfv v1.1.0/go.mod h1:zID2mqw9mFsnt7YC3vYQ9/cjq30q41W+1AnDwH8TiMg=\ngithub.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=\ngithub.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=\ngithub.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=\ngithub.com/elgris/jsondiff v0.0.0-20160530203242-765b5c24c302 h1:QV0ZrfBLpFc2KDk+a4LJefDczXnonRwrYrQJY/9L4dA=\ngithub.com/elgris/jsondiff v0.0.0-20160530203242-765b5c24c302/go.mod h1:qBlWZqWeVx9BjvqBsnC/8RUlAYpIFmPvgROcw0n1scE=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5 h1:BBso6MBKW8ncyZLv37o+KNyy0HrrHgfnOaGQC2qvN+A=\ngithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5/go.mod h1:JpoxHjuQauoxiFMl1ie8Xc/7TfLuMZ5eOCONd1sUBHg=\ngithub.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=\ngithub.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=\ngithub.com/fatih/color v1.15.0 h1:kOqh6YHBtK8aywxGerMG2Eq3H6Qgoqeo13Bk2Mv/nBs=\ngithub.com/fatih/color v1.15.0/go.mod h1:0h5ZqXfHYED7Bhv2ZJamyIOUej9KtShiJESRwBDUSsw=\ngithub.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=\ngithub.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/filecoin-project/go-clock v0.1.0 h1:SFbYIM75M8NnFm1yMHhN9Ahy3W5bEZV9gd6MPfXbKVU=\ngithub.com/filecoin-project/go-clock v0.1.0/go.mod h1:4uB/O4PvOjlx1VCMdZ9MyDZXRm//gkj1ELEbxfI1AZs=\ngithub.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 h1:BHsljHzVlRcyQhjrss6TZTdY2VfCqZPbv5k3iBFa2ZQ=\ngithub.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=\ngithub.com/flynn/noise v1.1.0 h1:KjPQoQCEFdZDiP03phOvGi11+SVVhBG2wOWAorLsstg=\ngithub.com/flynn/noise v1.1.0/go.mod h1:xbMo+0i6+IGbYdJhF31t2eR1BIU0CYc12+BNAKwUTag=\ngithub.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k=\ngithub.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=\ngithub.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=\ngithub.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=\ngithub.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=\ngithub.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU=\ngithub.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=\ngithub.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=\ngithub.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=\ngithub.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=\ngithub.com/gammazero/chanqueue v1.1.2 h1:dZEsxlyANZMyeTRemABqZF8QM9BnE4NBI43Oh3y5fIU=\ngithub.com/gammazero/chanqueue v1.1.2/go.mod h1:XDN1X/jjAbmSceNFOQbtKToeSkxtdVdpKu90LiEdBEE=\ngithub.com/gammazero/deque v1.2.1 h1:9fnQVFCCZ9/NOc7ccTNqzoKd1tCWOqeI05/lPqFPMGQ=\ngithub.com/gammazero/deque v1.2.1/go.mod h1:5nSFkzVm+afG9+gy0VIowlqVAW4N8zNcMne+CMQVD2g=\ngithub.com/getsentry/sentry-go v0.27.0 h1:Pv98CIbtB3LkMWmXi4Joa5OOcwbmnX88sF5qbK3r3Ps=\ngithub.com/getsentry/sentry-go v0.27.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY=\ngithub.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9 h1:r5GgOLGbza2wVHRzK7aAj6lWZjfbAwiu/RDCVOKjRyM=\ngithub.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9/go.mod h1:106OIgooyS7OzLDOpUGgm9fA3bQENb/cFSyyBmMoJDs=\ngithub.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=\ngithub.com/go-check/check v0.0.0-20180628173108-788fd7840127/go.mod h1:9ES+weclKsC9YodN5RgxqK/VD9HM9JsCSh7rNhMZE98=\ngithub.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=\ngithub.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=\ngithub.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-jose/go-jose/v4 v4.1.3 h1:CVLmWDhDVRa6Mi/IgCgaopNosCaHz7zrMeF9MlZRkrs=\ngithub.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=\ngithub.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=\ngithub.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=\ngithub.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=\ngithub.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0=\ngithub.com/go-kit/log v0.2.1 h1:MRVx0/zhvdseW+Gza6N9rVzU/IVzaeE1SFI4raAhmBU=\ngithub.com/go-kit/log v0.2.1/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0=\ngithub.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=\ngithub.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=\ngithub.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=\ngithub.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=\ngithub.com/go-logfmt/logfmt v0.6.0 h1:wGYYu3uicYdqXVgoYbvnkrPVXkuLM1p1ifugDMEdRi4=\ngithub.com/go-logfmt/logfmt v0.6.0/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=\ngithub.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=\ngithub.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=\ngithub.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=\ngithub.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=\ngithub.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=\ngithub.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=\ngithub.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=\ngithub.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0=\ngithub.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=\ngithub.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=\ngithub.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=\ngithub.com/gogo/protobuf v1.3.0/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=\ngithub.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=\ngithub.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=\ngithub.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=\ngithub.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=\ngithub.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=\ngithub.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=\ngithub.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\ngithub.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=\ngithub.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=\ngithub.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e h1:4bw4WeyTYPp0smaXiJZCNnLrvVBqirQVreixayXezGc=\ngithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=\ngithub.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=\ngithub.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=\ngithub.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=\ngithub.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo=\ngithub.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=\ngithub.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=\ngithub.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=\ngithub.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=\ngithub.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=\ngithub.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=\ngithub.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g=\ngithub.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k=\ngithub.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=\ngithub.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=\ngithub.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=\ngithub.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=\ngithub.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=\ngithub.com/grpc-ecosystem/grpc-gateway/v2 v2.27.7 h1:X+2YciYSxvMQK0UZ7sg45ZVabVZBeBuvMkmuI2V3Fak=\ngithub.com/grpc-ecosystem/grpc-gateway/v2 v2.27.7/go.mod h1:lW34nIZuQ8UDPdkon5fmfp2l3+ZkQ2me/+oecHYLOII=\ngithub.com/guillaumemichel/reservedpool v0.3.0 h1:eqqO/QvTllLBrit7LVtVJBqw4cD0WdV9ajUe7WNTajw=\ngithub.com/guillaumemichel/reservedpool v0.3.0/go.mod h1:sXSDIaef81TFdAJglsCFCMfgF5E5Z5xK1tFhjDhvbUc=\ngithub.com/gxed/hashland/keccakpg v0.0.1/go.mod h1:kRzw3HkwxFU1mpmPP8v1WyQzwdGfmKFJ6tItnhQ67kU=\ngithub.com/gxed/hashland/murmur3 v0.0.1/go.mod h1:KjXop02n4/ckmZSnY2+HKcLud/tcmvhST0bie/0lS48=\ngithub.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=\ngithub.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=\ngithub.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=\ngithub.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=\ngithub.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=\ngithub.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=\ngithub.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=\ngithub.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=\ngithub.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=\ngithub.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=\ngithub.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4=\ngithub.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=\ngithub.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=\ngithub.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=\ngithub.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=\ngithub.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=\ngithub.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=\ngithub.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=\ngithub.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=\ngithub.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=\ngithub.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=\ngithub.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc=\ngithub.com/huin/goupnp v1.3.0/go.mod h1:gnGPsThkYa7bFi/KWmEysQRf48l2dvR5bxr2OFckNX8=\ngithub.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=\ngithub.com/ipfs-shipyard/nopfs v0.0.14 h1:HFepJt/MxhZ3/GsLZkkAPzIPdNYKaLO1Qb7YmPbWIKk=\ngithub.com/ipfs-shipyard/nopfs v0.0.14/go.mod h1:mQyd0BElYI2gB/kq/Oue97obP4B3os4eBmgfPZ+hnrE=\ngithub.com/ipfs-shipyard/nopfs/ipfs v0.25.0 h1:OqNqsGZPX8zh3eFMO8Lf8EHRRnSGBMqcdHUd7SDsUOY=\ngithub.com/ipfs-shipyard/nopfs/ipfs v0.25.0/go.mod h1:BxhUdtBgOXg1B+gAPEplkg/GpyTZY+kCMSfsJvvydqU=\ngithub.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=\ngithub.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=\ngithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422 h1:yY3ot/DU1bqTzHDBARACM76Tbx9s4xzcRbzifG1e/es=\ngithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422/go.mod h1:8yyiRn54F2CsW13n0zwXEPrVsZix/gFj9SYIRYMZ6KE=\ngithub.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=\ngithub.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=\ngithub.com/ipfs/go-block-format v0.0.3/go.mod h1:4LmD4ZUw0mhO+JSKdpWwrzATiEfM7WWgQ8H5l6P8MVk=\ngithub.com/ipfs/go-block-format v0.2.3 h1:mpCuDaNXJ4wrBJLrtEaGFGXkferrw5eqVvzaHhtFKQk=\ngithub.com/ipfs/go-block-format v0.2.3/go.mod h1:WJaQmPAKhD3LspLixqlqNFxiZ3BZ3xgqxxoSR/76pnA=\ngithub.com/ipfs/go-cid v0.0.3/go.mod h1:GHWU/WuQdMPmIosc4Yn1bcCT7dSeX4lBafM7iqUPQvM=\ngithub.com/ipfs/go-cid v0.0.4/go.mod h1:4LLaPOQwmk5z9LBgQnpkivrx8BJjUyGwTXCd5Xfj6+M=\ngithub.com/ipfs/go-cid v0.0.7/go.mod h1:6Ux9z5e+HpkQdckYoX1PG/6xqKspzlEIR5SDmgqgC/I=\ngithub.com/ipfs/go-cid v0.6.0 h1:DlOReBV1xhHBhhfy/gBNNTSyfOM6rLiIx9J7A4DGf30=\ngithub.com/ipfs/go-cid v0.6.0/go.mod h1:NC4kS1LZjzfhK40UGmpXv5/qD2kcMzACYJNntCUiDhQ=\ngithub.com/ipfs/go-cidutil v0.1.1 h1:COuby6H8C2ml0alvHYX3WdbFM4F07YtbY0UlT5j+sgI=\ngithub.com/ipfs/go-cidutil v0.1.1/go.mod h1:SCoUftGEUgoXe5Hjeyw5CiLZF8cwYn/TbtpFQXJCP6k=\ngithub.com/ipfs/go-datastore v0.1.0/go.mod h1:d4KVXhMt913cLBEI/PXAy6ko+W7e9AhyAKBGh803qeE=\ngithub.com/ipfs/go-datastore v0.1.1/go.mod h1:w38XXW9kVFNp57Zj5knbKWM2T+KOZCGDRVNdgPHtbHw=\ngithub.com/ipfs/go-datastore v0.9.1 h1:67Po2epre/o0UxrmkzdS9ZTe2GFGODgTd2odx8Wh6Yo=\ngithub.com/ipfs/go-datastore v0.9.1/go.mod h1:zi07Nvrpq1bQwSkEnx3bfjz+SQZbdbWyCNvyxMh9pN0=\ngithub.com/ipfs/go-detect-race v0.0.1 h1:qX/xay2W3E4Q1U7d9lNs1sU9nvguX0a7319XbyQ6cOk=\ngithub.com/ipfs/go-detect-race v0.0.1/go.mod h1:8BNT7shDZPo99Q74BpGMK+4D8Mn4j46UU0LZ723meps=\ngithub.com/ipfs/go-ds-badger v0.0.7/go.mod h1:qt0/fWzZDoPW6jpQeqUjR5kBfhDNB65jd9YlmAvpQBk=\ngithub.com/ipfs/go-ds-badger v0.3.4 h1:MmqFicftE0KrwMC77WjXTrPuoUxhwyFsjKONSeWrlOo=\ngithub.com/ipfs/go-ds-badger v0.3.4/go.mod h1:HfqsKJcNnIr9ZhZ+rkwS1J5PpaWjJjg6Ipmxd7KPfZ8=\ngithub.com/ipfs/go-ds-flatfs v0.6.0 h1:olAEnDNBK1VMoTRZvfzgo90H5kBP4qIZPpYMtNlBBws=\ngithub.com/ipfs/go-ds-flatfs v0.6.0/go.mod h1:p8a/YhmAFYyuonxDbvuIANlDCgS69uqVv+iH5f8fAxY=\ngithub.com/ipfs/go-ds-leveldb v0.1.0/go.mod h1:hqAW8y4bwX5LWcCtku2rFNX3vjDZCy5LZCg+cSZvYb8=\ngithub.com/ipfs/go-ds-leveldb v0.5.2 h1:6nmxlQ2zbp4LCNdJVsmHfs9GP0eylfBNxpmY1csp0x0=\ngithub.com/ipfs/go-ds-leveldb v0.5.2/go.mod h1:2fAwmcvD3WoRT72PzEekHBkQmBDhc39DJGoREiuGmYo=\ngithub.com/ipfs/go-ds-measure v0.2.2 h1:4kwvBGbbSXNYe4ANlg7qTIYoZU6mNlqzQHdVqICkqGI=\ngithub.com/ipfs/go-ds-measure v0.2.2/go.mod h1:b/87ak0jMgH9Ylt7oH0+XGy4P8jHx9KG09Qz+pOeTIs=\ngithub.com/ipfs/go-ds-pebble v0.5.9 h1:D1FEuMxjbEmDADNqsyT74n9QHVAn12nv9i9Qa15AFYc=\ngithub.com/ipfs/go-ds-pebble v0.5.9/go.mod h1:XmUBN05l6B+tMg7mpMS75ZcKW/CX01uZMhhWw85imQA=\ngithub.com/ipfs/go-dsqueue v0.2.0 h1:MBi9w3oSiX98Xc+Y7NuJ9G8MI6mAT4IGdO9dHEMCZzU=\ngithub.com/ipfs/go-dsqueue v0.2.0/go.mod h1:8FfNQC4DMF/KkzBXRNB9Rb3MKDW0Sh98HMtXYl1mLQE=\ngithub.com/ipfs/go-fs-lock v0.1.1 h1:TecsP/Uc7WqYYatasreZQiP9EGRy4ZnKoG4yXxR33nw=\ngithub.com/ipfs/go-fs-lock v0.1.1/go.mod h1:2goSXMCw7QfscHmSe09oXiR34DQeUdm+ei+dhonqly0=\ngithub.com/ipfs/go-ipfs-cmds v0.16.0 h1:Oq39Gzz3pWrPwP25SjbfBQugFjKyjFdsHlAMIXdzYzM=\ngithub.com/ipfs/go-ipfs-cmds v0.16.0/go.mod h1:iRNtY9ipM/vH0Yr+/0FY+JfT+trZDQIztDoesmmoTo4=\ngithub.com/ipfs/go-ipfs-delay v0.0.0-20181109222059-70721b86a9a8/go.mod h1:8SP1YXK1M1kXuc4KJZINY3TQQ03J2rwBG9QfXmbRPrw=\ngithub.com/ipfs/go-ipfs-delay v0.0.1 h1:r/UXYyRcddO6thwOnhiznIAiSvxMECGgtv35Xs1IeRQ=\ngithub.com/ipfs/go-ipfs-delay v0.0.1/go.mod h1:8SP1YXK1M1kXuc4KJZINY3TQQ03J2rwBG9QfXmbRPrw=\ngithub.com/ipfs/go-ipfs-ds-help v1.1.1 h1:B5UJOH52IbcfS56+Ul+sv8jnIV10lbjLF5eOO0C66Nw=\ngithub.com/ipfs/go-ipfs-ds-help v1.1.1/go.mod h1:75vrVCkSdSFidJscs8n4W+77AtTpCIAdDGAwjitJMIo=\ngithub.com/ipfs/go-ipfs-pq v0.0.4 h1:U7jjENWJd1jhcrR8X/xHTaph14PTAK9O+yaLJbjqgOw=\ngithub.com/ipfs/go-ipfs-pq v0.0.4/go.mod h1:9UdLOIIb99IFrgT0Fc53pvbvlJBhpUb4GJuAQf3+O2A=\ngithub.com/ipfs/go-ipfs-redirects-file v0.1.2 h1:QCK7VtL91FH17KROVVy5KrzDx2hu68QvB2FTWk08ZQk=\ngithub.com/ipfs/go-ipfs-redirects-file v0.1.2/go.mod h1:yIiTlLcDEM/8lS6T3FlCEXZktPPqSOyuY6dEzVqw7Fw=\ngithub.com/ipfs/go-ipfs-util v0.0.1/go.mod h1:spsl5z8KUnrve+73pOhSVZND1SIxPW5RyBCNzQxlJBc=\ngithub.com/ipfs/go-ipfs-util v0.0.2/go.mod h1:CbPtkWJzjLdEcezDns2XYaehFVNXG9zrdrtMecczcsQ=\ngithub.com/ipfs/go-ipld-cbor v0.2.1 h1:H05yEJbK/hxg0uf2AJhyerBDbjOuHX4yi+1U/ogRa7E=\ngithub.com/ipfs/go-ipld-cbor v0.2.1/go.mod h1:x9Zbeq8CoE5R2WicYgBMcr/9mnkQ0lHddYWJP2sMV3A=\ngithub.com/ipfs/go-ipld-format v0.6.3 h1:9/lurLDTotJpZSuL++gh3sTdmcFhVkCwsgx2+rAh4j8=\ngithub.com/ipfs/go-ipld-format v0.6.3/go.mod h1:74ilVN12NXVMIV+SrBAyC05UJRk0jVvGqdmrcYZvCBk=\ngithub.com/ipfs/go-ipld-git v0.1.1 h1:TWGnZjS0htmEmlMFEkA3ogrNCqWjIxwr16x1OsdhG+Y=\ngithub.com/ipfs/go-ipld-git v0.1.1/go.mod h1:+VyMqF5lMcJh4rwEppV0e6g4nCCHXThLYYDpKUkJubI=\ngithub.com/ipfs/go-ipld-legacy v0.3.0 h1:7XhFKkRyCvP5upOlQfKUFIqL3S5DEZnbUE4bQmQ/tNE=\ngithub.com/ipfs/go-ipld-legacy v0.3.0/go.mod h1:Ukef9ARQiX+RVetwH2XiReLgJvQDEXcUPszrZ1KRjKI=\ngithub.com/ipfs/go-libdht v0.5.0 h1:ZN+eCqwahZvUeT0e4DsIxRtm78Mc9UR5tmZUiMsrGjQ=\ngithub.com/ipfs/go-libdht v0.5.0/go.mod h1:L3YiuFXecLeZZFuuVRM0hjg1GgVhARzUdahFsuqSa7w=\ngithub.com/ipfs/go-log v0.0.1/go.mod h1:kL1d2/hzSpI0thNYjiKfjanbVNU+IIGA/WnNESY9leM=\ngithub.com/ipfs/go-log/v2 v2.9.1 h1:3JXwHWU31dsCpvQ+7asz6/QsFJHqFr4gLgQ0FWteujk=\ngithub.com/ipfs/go-log/v2 v2.9.1/go.mod h1:evFx7sBiohUN3AG12mXlZBw5hacBQld3ZPHrowlJYoo=\ngithub.com/ipfs/go-metrics-interface v0.3.0 h1:YwG7/Cy4R94mYDUuwsBfeziJCVm9pBMJ6q/JR9V40TU=\ngithub.com/ipfs/go-metrics-interface v0.3.0/go.mod h1:OxxQjZDGocXVdyTPocns6cOLwHieqej/jos7H4POwoY=\ngithub.com/ipfs/go-metrics-prometheus v0.1.0 h1:bApWOHkrH3VTBHzTHrZSfq4n4weOZDzZFxUXv+HyKcA=\ngithub.com/ipfs/go-metrics-prometheus v0.1.0/go.mod h1:2GtL525C/4yxtvSXpRJ4dnE45mCX9AS0XRa03vHx7G0=\ngithub.com/ipfs/go-peertaskqueue v0.8.3 h1:tBPpGJy+A92RqtRFq5amJn0Uuj8Pw8tXi0X3eHfHM8w=\ngithub.com/ipfs/go-peertaskqueue v0.8.3/go.mod h1:OqVync4kPOcXEGdj/LKvox9DCB5mkSBeXsPczCxLtYA=\ngithub.com/ipfs/go-test v0.2.3 h1:Z/jXNAReQFtCYyn7bsv/ZqUwS6E7iIcSpJ2CuzCvnrc=\ngithub.com/ipfs/go-test v0.2.3/go.mod h1:QW8vSKkwYvWFwIZQLGQXdkt9Ud76eQXRQ9Ao2H+cA1o=\ngithub.com/ipfs/go-unixfsnode v1.10.3 h1:c8sJjuGNkxXAQH75P+f5ngPda/9T+DrboVA0TcDGvGI=\ngithub.com/ipfs/go-unixfsnode v1.10.3/go.mod h1:2Jlc7DoEwr12W+7l8Hr6C7XF4NHST3gIkqSArLhGSxU=\ngithub.com/ipld/go-car/v2 v2.16.0 h1:LWe0vmN/QcQmUU4tr34W5Nv5mNraW+G6jfN2s+ndBco=\ngithub.com/ipld/go-car/v2 v2.16.0/go.mod h1:RqFGWN9ifcXVmCrTAVnfnxiWZk1+jIx67SYhenlmL34=\ngithub.com/ipld/go-codec-dagpb v1.7.0 h1:hpuvQjCSVSLnTnHXn+QAMR0mLmb1gA6wl10LExo2Ts0=\ngithub.com/ipld/go-codec-dagpb v1.7.0/go.mod h1:rD3Zg+zub9ZnxcLwfol/OTQRVjaLzXypgy4UqHQvilM=\ngithub.com/ipld/go-ipld-prime v0.11.0/go.mod h1:+WIAkokurHmZ/KwzDOMUuoeJgaRQktHtEaLglS3ZeV8=\ngithub.com/ipld/go-ipld-prime v0.22.0 h1:YJhDhjEOvOYaqshd3b4atIWUoRg/rKrgmwCyUHwlbuY=\ngithub.com/ipld/go-ipld-prime v0.22.0/go.mod h1:ol7vKxOOVgEh0iAPuiDalM+0gScXVMA5ZZa4DVrTnEA=\ngithub.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20250821084354-a425e60cd714 h1:cqNk8PEwHnK0vqWln+U/YZhQc9h2NB3KjUjDPZo5Q2s=\ngithub.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20250821084354-a425e60cd714/go.mod h1:ZEUdra3CoqRVRYgAX/jAJO9aZGz6SKtKEG628fHHktY=\ngithub.com/ipshipyard/p2p-forge v0.7.0 h1:PQayexxZC1FR2Vx0XOSbmZ6wDPliidS48I+xXWuF+YU=\ngithub.com/ipshipyard/p2p-forge v0.7.0/go.mod h1:i2wg0p7WmHGyo5vYaK9COZBp8BN5Drncfu3WoQNZlQY=\ngithub.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=\ngithub.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=\ngithub.com/jbenet/go-cienv v0.1.0/go.mod h1:TqNnHUmJgXau0nCzC7kXWeotg3J9W34CUv5Djy1+FlA=\ngithub.com/jbenet/go-temp-err-catcher v0.1.0 h1:zpb3ZH6wIE8Shj2sKS+khgRvf7T7RABoLk/+KKHggpk=\ngithub.com/jbenet/go-temp-err-catcher v0.1.0/go.mod h1:0kJRvmDZXNMIiJirNPEYfhpPwbGVtZVWC34vc5WLsDk=\ngithub.com/jbenet/goprocess v0.0.0-20160826012719-b497e2f366b8/go.mod h1:Ly/wlsjFq/qrU3Rar62tu1gASgGw6chQbSh/XgIIXCY=\ngithub.com/jbenet/goprocess v0.1.3/go.mod h1:5yspPrukOVuOLORacaBi858NqyClJPQxYZlqdZVfqY4=\ngithub.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=\ngithub.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=\ngithub.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=\ngithub.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ=\ngithub.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=\ngithub.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=\ngithub.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=\ngithub.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=\ngithub.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=\ngithub.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=\ngithub.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=\ngithub.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=\ngithub.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=\ngithub.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U=\ngithub.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=\ngithub.com/kami-zh/go-capturer v0.0.0-20171211120116-e492ea43421d/go.mod h1:P2viExyCEfeWGU259JnaQ34Inuec4R38JCyBx2edgD0=\ngithub.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=\ngithub.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=\ngithub.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=\ngithub.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=\ngithub.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=\ngithub.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=\ngithub.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=\ngithub.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=\ngithub.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=\ngithub.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=\ngithub.com/koron/go-ssdp v0.0.6 h1:Jb0h04599eq/CY7rB5YEqPS83HmRfHP2azkxMN2rFtU=\ngithub.com/koron/go-ssdp v0.0.6/go.mod h1:0R9LfRJGek1zWTjN3JUNlm5INCDYGpRDfAptnct63fI=\ngithub.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=\ngithub.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=\ngithub.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=\ngithub.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=\ngithub.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=\ngithub.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=\ngithub.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=\ngithub.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=\ngithub.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=\ngithub.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=\ngithub.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=\ngithub.com/libdns/libdns v1.0.0-beta.1 h1:KIf4wLfsrEpXpZ3vmc/poM8zCATXT2klbdPe6hyOBjQ=\ngithub.com/libdns/libdns v1.0.0-beta.1/go.mod h1:4Bj9+5CQiNMVGf87wjX4CY3HQJypUHRuLvlsfsZqLWQ=\ngithub.com/libp2p/go-buffer-pool v0.0.1/go.mod h1:xtyIz9PMobb13WaxR6Zo1Pd1zXJKYg0a8KiIvDp3TzQ=\ngithub.com/libp2p/go-buffer-pool v0.0.2/go.mod h1:MvaB6xw5vOrDl8rYZGLFdKAuk/hRoRZd1Vi32+RXyFM=\ngithub.com/libp2p/go-buffer-pool v0.1.0 h1:oK4mSFcQz7cTQIfqbe4MIj9gLW+mnanjyFtc6cdF0Y8=\ngithub.com/libp2p/go-buffer-pool v0.1.0/go.mod h1:N+vh8gMqimBzdKkSMVuydVDq+UV5QTWy5HSiZacSbPg=\ngithub.com/libp2p/go-cidranger v1.1.0 h1:ewPN8EZ0dd1LSnrtuwd4709PXVcITVeuwbag38yPW7c=\ngithub.com/libp2p/go-cidranger v1.1.0/go.mod h1:KWZTfSr+r9qEo9OkI9/SIEeAtw+NNoU0dXIXt15Okic=\ngithub.com/libp2p/go-doh-resolver v0.5.0 h1:4h7plVVW+XTS+oUBw2+8KfoM1jF6w8XmO7+skhePFdE=\ngithub.com/libp2p/go-doh-resolver v0.5.0/go.mod h1:aPDxfiD2hNURgd13+hfo29z9IC22fv30ee5iM31RzxU=\ngithub.com/libp2p/go-flow-metrics v0.0.1/go.mod h1:Iv1GH0sG8DtYN3SVJ2eG221wMiNpZxBdp967ls1g+k8=\ngithub.com/libp2p/go-flow-metrics v0.0.3/go.mod h1:HeoSNUrOJVK1jEpDqVEiUOIXqhbnS27omG0uWU5slZs=\ngithub.com/libp2p/go-flow-metrics v0.3.0 h1:q31zcHUvHnwDO0SHaukewPYgwOBSxtt830uJtUx6784=\ngithub.com/libp2p/go-flow-metrics v0.3.0/go.mod h1:nuhlreIwEguM1IvHAew3ij7A8BMlyHQJ279ao24eZZo=\ngithub.com/libp2p/go-libp2p v0.48.0 h1:h2BrLAgrj7X8bEN05K7qmrjpNHYA+6tnsGRdprjTnvo=\ngithub.com/libp2p/go-libp2p v0.48.0/go.mod h1:Q1fBZNdmC2Hf82husCTfkKJVfHm2we5zk+NWmOGEmWk=\ngithub.com/libp2p/go-libp2p-asn-util v0.4.1 h1:xqL7++IKD9TBFMgnLPZR6/6iYhawHKHl950SO9L6n94=\ngithub.com/libp2p/go-libp2p-asn-util v0.4.1/go.mod h1:d/NI6XZ9qxw67b4e+NgpQexCIiFYJjErASrYW4PFDN8=\ngithub.com/libp2p/go-libp2p-core v0.2.4/go.mod h1:STh4fdfa5vDYr0/SzYYeqnt+E6KfEV5VxfIrm0bcI0g=\ngithub.com/libp2p/go-libp2p-core v0.3.0/go.mod h1:ACp3DmS3/N64c2jDzcV429ukDpicbL6+TrrxANBjPGw=\ngithub.com/libp2p/go-libp2p-gostream v0.6.0 h1:QfAiWeQRce6pqnYfmIVWJFXNdDyfiR/qkCnjyaZUPYU=\ngithub.com/libp2p/go-libp2p-gostream v0.6.0/go.mod h1:Nywu0gYZwfj7Jc91PQvbGU8dIpqbQQkjWgDuOrFaRdA=\ngithub.com/libp2p/go-libp2p-http v0.5.0 h1:+x0AbLaUuLBArHubbbNRTsgWz0RjNTy6DJLOxQ3/QBc=\ngithub.com/libp2p/go-libp2p-http v0.5.0/go.mod h1:glh87nZ35XCQyFsdzZps6+F4HYI6DctVFY5u1fehwSg=\ngithub.com/libp2p/go-libp2p-kad-dht v0.39.0 h1:mww38eBYiUvdsu+Xl/GLlBC0Aa8M+5HAwvafkFOygAM=\ngithub.com/libp2p/go-libp2p-kad-dht v0.39.0/go.mod h1:Po2JugFEkDq9Vig/JXtc153ntOi0q58o4j7IuITCOVs=\ngithub.com/libp2p/go-libp2p-kbucket v0.3.1/go.mod h1:oyjT5O7tS9CQurok++ERgc46YLwEpuGoFq9ubvoUOio=\ngithub.com/libp2p/go-libp2p-kbucket v0.8.0 h1:QAK7RzKJpYe+EuSEATAaaHYMYLkPDGC18m9jxPLnU8s=\ngithub.com/libp2p/go-libp2p-kbucket v0.8.0/go.mod h1:JMlxqcEyKwO6ox716eyC0hmiduSWZZl6JY93mGaaqc4=\ngithub.com/libp2p/go-libp2p-peerstore v0.1.4/go.mod h1:+4BDbDiiKf4PzpANZDAT+knVdLxvqh7hXOujessqdzs=\ngithub.com/libp2p/go-libp2p-pubsub v0.15.0 h1:cG7Cng2BT82WttmPFMi50gDNV+58K626m/wR00vGL1o=\ngithub.com/libp2p/go-libp2p-pubsub v0.15.0/go.mod h1:lr4oE8bFgQaifRcoc2uWhWWiK6tPdOEKpUuR408GFN4=\ngithub.com/libp2p/go-libp2p-pubsub-router v0.6.0 h1:D30iKdlqDt5ZmLEYhHELCMRj8b4sFAqrUcshIUvVP/s=\ngithub.com/libp2p/go-libp2p-pubsub-router v0.6.0/go.mod h1:FY/q0/RBTKsLA7l4vqC2cbRbOvyDotg8PJQ7j8FDudE=\ngithub.com/libp2p/go-libp2p-record v0.3.1 h1:cly48Xi5GjNw5Wq+7gmjfBiG9HCzQVkiZOUZ8kUl+Fg=\ngithub.com/libp2p/go-libp2p-record v0.3.1/go.mod h1:T8itUkLcWQLCYMqtX7Th6r7SexyUJpIyPgks757td/E=\ngithub.com/libp2p/go-libp2p-routing-helpers v0.7.5 h1:HdwZj9NKovMx0vqq6YNPTh6aaNzey5zHD7HeLJtq6fI=\ngithub.com/libp2p/go-libp2p-routing-helpers v0.7.5/go.mod h1:3YaxrwP0OBPDD7my3D0KxfR89FlcX/IEbxDEDfAmj98=\ngithub.com/libp2p/go-libp2p-testing v0.12.0 h1:EPvBb4kKMWO29qP4mZGyhVzUyR25dvfUIK5WDu6iPUA=\ngithub.com/libp2p/go-libp2p-testing v0.12.0/go.mod h1:KcGDRXyN7sQCllucn1cOOS+Dmm7ujhfEyXQL5lvkcPg=\ngithub.com/libp2p/go-libp2p-xor v0.1.0 h1:hhQwT4uGrBcuAkUGXADuPltalOdpf9aag9kaYNT2tLA=\ngithub.com/libp2p/go-libp2p-xor v0.1.0/go.mod h1:LSTM5yRnjGZbWNTA/hRwq2gGFrvRIbQJscoIL/u6InY=\ngithub.com/libp2p/go-msgio v0.0.4/go.mod h1:63lBBgOTDKQL6EWazRMCwXsEeEeK9O2Cd+0+6OOuipQ=\ngithub.com/libp2p/go-msgio v0.3.0 h1:mf3Z8B1xcFN314sWX+2vOTShIE0Mmn2TXn3YCUQGNj0=\ngithub.com/libp2p/go-msgio v0.3.0/go.mod h1:nyRM819GmVaF9LX3l03RMh10QdOroF++NBbxAb0mmDM=\ngithub.com/libp2p/go-netroute v0.4.0 h1:sZZx9hyANYUx9PZyqcgE/E1GUG3iEtTZHUEvdtXT7/Q=\ngithub.com/libp2p/go-netroute v0.4.0/go.mod h1:Nkd5ShYgSMS5MUKy/MU2T57xFoOKvvLR92Lic48LEyA=\ngithub.com/libp2p/go-openssl v0.0.3/go.mod h1:unDrJpgy3oFr+rqXsarWifmJuNnJR4chtO1HmaZjggc=\ngithub.com/libp2p/go-openssl v0.0.4/go.mod h1:unDrJpgy3oFr+rqXsarWifmJuNnJR4chtO1HmaZjggc=\ngithub.com/libp2p/go-reuseport v0.4.0 h1:nR5KU7hD0WxXCJbmw7r2rhRYruNRl2koHw8fQscQm2s=\ngithub.com/libp2p/go-reuseport v0.4.0/go.mod h1:ZtI03j/wO5hZVDFo2jKywN6bYKWLOy8Se6DrI2E1cLU=\ngithub.com/libp2p/go-socket-activation v0.1.1 h1:wkLBj6RqKffjt7BI794ewoSt241UV52NKYvIbpzhn4Q=\ngithub.com/libp2p/go-socket-activation v0.1.1/go.mod h1:NBfVUPXTRL/FU6UmSOM+1O7/vJkpS523sQiriw0Qln8=\ngithub.com/libp2p/go-yamux/v5 v5.0.1 h1:f0WoX/bEF2E8SbE4c/k1Mo+/9z0O4oC/hWEA+nfYRSg=\ngithub.com/libp2p/go-yamux/v5 v5.0.1/go.mod h1:en+3cdX51U0ZslwRdRLrvQsdayFt3TSUKvBGErzpWbU=\ngithub.com/libp2p/zeroconf/v2 v2.2.0 h1:Cup06Jv6u81HLhIj1KasuNM/RHHrJ8T7wOTS4+Tv53Q=\ngithub.com/libp2p/zeroconf/v2 v2.2.0/go.mod h1:fuJqLnUwZTshS3U/bMRJ3+ow/v9oid1n0DmyYyNO1Xs=\ngithub.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=\ngithub.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=\ngithub.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=\ngithub.com/marcopolo/simnet v0.0.4 h1:50Kx4hS9kFGSRIbrt9xUS3NJX33EyPqHVmpXvaKLqrY=\ngithub.com/marcopolo/simnet v0.0.4/go.mod h1:tfQF1u2DmaB6WHODMtQaLtClEf3a296CKQLq5gAsIS0=\ngithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd h1:br0buuQ854V8u83wA0rVZ8ttrq5CpaPZdvrK0LP2lOk=\ngithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd/go.mod h1:QuCEs1Nt24+FYQEqAAncTDPJIuGs+LxK1MCiFL25pMU=\ngithub.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=\ngithub.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ=\ngithub.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=\ngithub.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=\ngithub.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=\ngithub.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=\ngithub.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=\ngithub.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=\ngithub.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=\ngithub.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=\ngithub.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=\ngithub.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=\ngithub.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=\ngithub.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=\ngithub.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=\ngithub.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=\ngithub.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=\ngithub.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4=\ngithub.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=\ngithub.com/mholt/acmez/v3 v3.1.2 h1:auob8J/0FhmdClQicvJvuDavgd5ezwLBfKuYmynhYzc=\ngithub.com/mholt/acmez/v3 v3.1.2/go.mod h1:L1wOU06KKvq7tswuMDwKdcHeKpFFgkppZy/y0DFxagQ=\ngithub.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=\ngithub.com/miekg/dns v1.1.43/go.mod h1:+evo5L0630/F6ca/Z9+GAqzhjGyn8/c+TBaOyfEl0V4=\ngithub.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=\ngithub.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=\ngithub.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c h1:bzE/A84HN25pxAuk9Eej1Kz9OUelF97nAc82bDquQI8=\ngithub.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c/go.mod h1:0SQS9kMwD2VsyFEB++InYyBJroV/FRmBgcydeSUcJms=\ngithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b h1:z78hV3sbSMAUoyUMM0I83AUIT6Hu17AWfgjzIbtrYFc=\ngithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b/go.mod h1:lxPUiZwKoFL8DUUmalo2yJJUCxbPKtm8OKfqr2/FTNU=\ngithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc h1:PTfri+PuQmWDqERdnNMiD9ZejrlswWrCpBEZgWOiTrc=\ngithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc/go.mod h1:cGKTAVKx4SxOuR/czcZ/E2RSJ3sfHs8FpHhQ5CWMf9s=\ngithub.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1/go.mod h1:pD8RvIylQ358TN4wwqatJ8rNavkEINozVn9DtGI3dfQ=\ngithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 h1:0lgqHvJWHLGW5TuObJrfyEi6+ASTKDBWikGvPqy9Yiw=\ngithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882/go.mod h1:qT0aEB35q79LLornSzeDH75LBf3aH1MV+jB5w9Wasec=\ngithub.com/minio/sha256-simd v0.0.0-20190131020904-2d45a736cd16/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U=\ngithub.com/minio/sha256-simd v0.1.1-0.20190913151208-6de447530771/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=\ngithub.com/minio/sha256-simd v0.1.1/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=\ngithub.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=\ngithub.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=\ngithub.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=\ngithub.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=\ngithub.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=\ngithub.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=\ngithub.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=\ngithub.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=\ngithub.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=\ngithub.com/mr-tron/base58 v1.1.0/go.mod h1:xcD2VGqlgYjBdcBLw+TuYLr8afG+Hj8g2eTVqeSzSU8=\ngithub.com/mr-tron/base58 v1.1.2/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/mr-tron/base58 v1.1.3/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=\ngithub.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/multiformats/go-base32 v0.0.3/go.mod h1:pLiuGC8y0QR3Ue4Zug5UzK9LjgbkL8NSQj0zQ5Nz/AA=\ngithub.com/multiformats/go-base32 v0.1.0 h1:pVx9xoSPqEIQG8o+UbAe7DNi51oej1NtK+aGkbLYxPE=\ngithub.com/multiformats/go-base32 v0.1.0/go.mod h1:Kj3tFY6zNr+ABYMqeUNeGvkIC/UYgtWibDcT0rExnbI=\ngithub.com/multiformats/go-base36 v0.1.0/go.mod h1:kFGE83c6s80PklsHO9sRn2NCoffoRdUUOENyW/Vv6sM=\ngithub.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0=\ngithub.com/multiformats/go-base36 v0.2.0/go.mod h1:qvnKE++v+2MWCfePClUEjE78Z7P2a1UV0xHgWc0hkp4=\ngithub.com/multiformats/go-multiaddr v0.1.0/go.mod h1:xKVEak1K9cS1VdmPZW3LSIb6lgmoS58qz/pzqmAxV44=\ngithub.com/multiformats/go-multiaddr v0.1.1/go.mod h1:aMKBKNEYmzmDmxfX88/vz+J5IU55txyt0p4aiWVohjo=\ngithub.com/multiformats/go-multiaddr v0.2.0/go.mod h1:0nO36NvPpyV4QzvTLi/lafl2y95ncPj0vFwVF6k6wJ4=\ngithub.com/multiformats/go-multiaddr v0.16.1 h1:fgJ0Pitow+wWXzN9do+1b8Pyjmo8m5WhGfzpL82MpCw=\ngithub.com/multiformats/go-multiaddr v0.16.1/go.mod h1:JSVUmXDjsVFiW7RjIFMP7+Ev+h1DTbiJgVeTV/tcmP0=\ngithub.com/multiformats/go-multiaddr-dns v0.5.0 h1:p/FTyHKX0nl59f+S+dEUe8HRK+i5Ow/QHMw8Nh3gPCo=\ngithub.com/multiformats/go-multiaddr-dns v0.5.0/go.mod h1:yJ349b8TPIAANUyuOzn1oz9o22tV9f+06L+cCeMxC14=\ngithub.com/multiformats/go-multiaddr-fmt v0.1.0 h1:WLEFClPycPkp4fnIzoFoV9FVd49/eQsuaL3/CWe167E=\ngithub.com/multiformats/go-multiaddr-fmt v0.1.0/go.mod h1:hGtDIW4PU4BqJ50gW2quDuPVjyWNZxToGUh/HwTZYJo=\ngithub.com/multiformats/go-multiaddr-net v0.1.1/go.mod h1:5JNbcfBOP4dnhoZOv10JJVkJO0pCCEf8mTnipAo2UZQ=\ngithub.com/multiformats/go-multibase v0.0.1/go.mod h1:bja2MqRZ3ggyXtZSEDKpl0uO/gviWFaSteVbWT51qgs=\ngithub.com/multiformats/go-multibase v0.0.3/go.mod h1:5+1R4eQrT3PkYZ24C3W2Ue2tPwIdYQD509ZjSb5y9Oc=\ngithub.com/multiformats/go-multibase v0.2.0 h1:isdYCVLvksgWlMW9OZRYJEa9pZETFivncJHmHnnd87g=\ngithub.com/multiformats/go-multibase v0.2.0/go.mod h1:bFBZX4lKCA/2lyOFSAoKH5SS6oPyjtnzK/XTFDPkNuk=\ngithub.com/multiformats/go-multicodec v0.10.0 h1:UpP223cig/Cx8J76jWt91njpK3GTAO1w02sdcjZDSuc=\ngithub.com/multiformats/go-multicodec v0.10.0/go.mod h1:wg88pM+s2kZJEQfRCKBNU+g32F5aWBEjyFHXvZLTcLI=\ngithub.com/multiformats/go-multihash v0.0.1/go.mod h1:w/5tugSrLEbWqlcgJabL3oHFKTwfvkofsjW2Qa1ct4U=\ngithub.com/multiformats/go-multihash v0.0.8/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa63epEDmG8nTduyAew=\ngithub.com/multiformats/go-multihash v0.0.10/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa63epEDmG8nTduyAew=\ngithub.com/multiformats/go-multihash v0.0.13/go.mod h1:VdAWLKTwram9oKAatUcLxBNUjdtcVwxObEQBtRfuyjc=\ngithub.com/multiformats/go-multihash v0.0.14/go.mod h1:VdAWLKTwram9oKAatUcLxBNUjdtcVwxObEQBtRfuyjc=\ngithub.com/multiformats/go-multihash v0.0.15/go.mod h1:D6aZrWNLFTV/ynMpKsNtB40mJzmCl4jb1alC0OvHiHg=\ngithub.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=\ngithub.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM=\ngithub.com/multiformats/go-multistream v0.6.1 h1:4aoX5v6T+yWmc2raBHsTvzmFhOI8WVOer28DeBBEYdQ=\ngithub.com/multiformats/go-multistream v0.6.1/go.mod h1:ksQf6kqHAb6zIsyw7Zm+gAuVo57Qbq84E27YlYqavqw=\ngithub.com/multiformats/go-varint v0.0.1/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=\ngithub.com/multiformats/go-varint v0.0.5/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=\ngithub.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=\ngithub.com/multiformats/go-varint v0.1.0 h1:i2wqFp4sdl3IcIxfAonHQV9qU5OsZ4Ts9IOoETFs5dI=\ngithub.com/multiformats/go-varint v0.1.0/go.mod h1:5KVAVXegtfmNQQm/lCY+ATvDzvJJhSkUlGQV9wgObdI=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=\ngithub.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=\ngithub.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=\ngithub.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo=\ngithub.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM=\ngithub.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=\ngithub.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=\ngithub.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=\ngithub.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=\ngithub.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=\ngithub.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=\ngithub.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=\ngithub.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=\ngithub.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=\ngithub.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c=\ngithub.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=\ngithub.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=\ngithub.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=\ngithub.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=\ngithub.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=\ngithub.com/onsi/gomega v1.36.3 h1:hID7cr8t3Wp26+cYnfcjR6HpJ00fdogN6dqZ1t6IylU=\ngithub.com/onsi/gomega v1.36.3/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=\ngithub.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=\ngithub.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=\ngithub.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=\ngithub.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=\ngithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0=\ngithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y=\ngithub.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=\ngithub.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 h1:1/WtZae0yGtPq+TI6+Tv1WTxkukpXeMlviSxvL7SRgk=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9/go.mod h1:x3N5drFsm2uilKKuuYo6LdyD8vZAW55sH/9w+pbo1sw=\ngithub.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=\ngithub.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=\ngithub.com/pion/datachannel v1.5.10 h1:ly0Q26K1i6ZkGf42W7D4hQYR90pZwzFOjTq5AuCKk4o=\ngithub.com/pion/datachannel v1.5.10/go.mod h1:p/jJfC9arb29W7WrxyKbepTU20CFgyx5oLo8Rs4Py/M=\ngithub.com/pion/dtls/v3 v3.1.2 h1:gqEdOUXLtCGW+afsBLO0LtDD8GnuBBjEy6HRtyofZTc=\ngithub.com/pion/dtls/v3 v3.1.2/go.mod h1:Hw/igcX4pdY69z1Hgv5x7wJFrUkdgHwAn/Q/uo7YHRo=\ngithub.com/pion/ice/v4 v4.0.10 h1:P59w1iauC/wPk9PdY8Vjl4fOFL5B+USq1+xbDcN6gT4=\ngithub.com/pion/ice/v4 v4.0.10/go.mod h1:y3M18aPhIxLlcO/4dn9X8LzLLSma84cx6emMSu14FGw=\ngithub.com/pion/interceptor v0.1.40 h1:e0BjnPcGpr2CFQgKhrQisBU7V3GXK6wrfYrGYaU6Jq4=\ngithub.com/pion/interceptor v0.1.40/go.mod h1:Z6kqH7M/FYirg3frjGJ21VLSRJGBXB/KqaTIrdqnOic=\ngithub.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=\ngithub.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so=\ngithub.com/pion/mdns/v2 v2.0.7 h1:c9kM8ewCgjslaAmicYMFQIde2H9/lrZpjBkN8VwoVtM=\ngithub.com/pion/mdns/v2 v2.0.7/go.mod h1:vAdSYNAT0Jy3Ru0zl2YiW3Rm/fJCwIeM0nToenfOJKA=\ngithub.com/pion/randutil v0.1.0 h1:CFG1UdESneORglEsnimhUjf33Rwjubwj6xfiOXBa3mA=\ngithub.com/pion/randutil v0.1.0/go.mod h1:XcJrSMMbbMRhASFVOlj/5hQial/Y8oH/HVo7TBZq+j8=\ngithub.com/pion/rtcp v1.2.16 h1:fk1B1dNW4hsI78XUCljZJlC4kZOPk67mNRuQ0fcEkSo=\ngithub.com/pion/rtcp v1.2.16/go.mod h1:/as7VKfYbs5NIb4h6muQ35kQF/J0ZVNz2Z3xKoCBYOo=\ngithub.com/pion/rtp v1.8.19 h1:jhdO/3XhL/aKm/wARFVmvTfq0lC/CvN1xwYKmduly3c=\ngithub.com/pion/rtp v1.8.19/go.mod h1:bAu2UFKScgzyFqvUKmbvzSdPr+NGbZtv6UB2hesqXBk=\ngithub.com/pion/sctp v1.8.39 h1:PJma40vRHa3UTO3C4MyeJDQ+KIobVYRZQZ0Nt7SjQnE=\ngithub.com/pion/sctp v1.8.39/go.mod h1:cNiLdchXra8fHQwmIoqw0MbLLMs+f7uQ+dGMG2gWebE=\ngithub.com/pion/sdp/v3 v3.0.18 h1:l0bAXazKHpepazVdp+tPYnrsy9dfh7ZbT8DxesH5ZnI=\ngithub.com/pion/sdp/v3 v3.0.18/go.mod h1:ZREGo6A9ZygQ9XkqAj5xYCQtQpif0i6Pa81HOiAdqQ8=\ngithub.com/pion/srtp/v3 v3.0.6 h1:E2gyj1f5X10sB/qILUGIkL4C2CqK269Xq167PbGCc/4=\ngithub.com/pion/srtp/v3 v3.0.6/go.mod h1:BxvziG3v/armJHAaJ87euvkhHqWe9I7iiOy50K2QkhY=\ngithub.com/pion/stun/v3 v3.1.1 h1:CkQxveJ4xGQjulGSROXbXq94TAWu8gIX2dT+ePhUkqw=\ngithub.com/pion/stun/v3 v3.1.1/go.mod h1:qC1DfmcCTQjl9PBaMa5wSn3x9IPmKxSdcCsxBcDBndM=\ngithub.com/pion/transport/v3 v3.0.7 h1:iRbMH05BzSNwhILHoBoAPxoB9xQgOaJk+591KC9P1o0=\ngithub.com/pion/transport/v3 v3.0.7/go.mod h1:YleKiTZ4vqNxVwh77Z0zytYi7rXHl7j6uPLGhhz9rwo=\ngithub.com/pion/transport/v4 v4.0.1 h1:sdROELU6BZ63Ab7FrOLn13M6YdJLY20wldXW2Cu2k8o=\ngithub.com/pion/transport/v4 v4.0.1/go.mod h1:nEuEA4AD5lPdcIegQDpVLgNoDGreqM/YqmEx3ovP4jM=\ngithub.com/pion/turn/v4 v4.0.2 h1:ZqgQ3+MjP32ug30xAbD6Mn+/K4Sxi3SdNOTFf+7mpps=\ngithub.com/pion/turn/v4 v4.0.2/go.mod h1:pMMKP/ieNAG/fN5cZiN4SDuyKsXtNTr0ccN7IToA1zs=\ngithub.com/pion/webrtc/v4 v4.1.2 h1:mpuUo/EJ1zMNKGE79fAdYNFZBX790KE7kQQpLMjjR54=\ngithub.com/pion/webrtc/v4 v4.1.2/go.mod h1:xsCXiNAmMEjIdFxAYU0MbB3RwRieJsegSB2JZsGN+8U=\ngithub.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=\ngithub.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=\ngithub.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/polydawn/refmt v0.0.0-20201211092308-30ac6d18308e/go.mod h1:uIp+gprXxxrWSjjklXD+mN4wed/tMfjMMmN/9+JsA9o=\ngithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a h1:cgqrm0F3zwf9IPzca7xN4w+Zy6MC9ZkPvAC8QEWa/iQ=\ngithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a/go.mod h1:ocZfO/tLSHqfScRDNTJbAJR1by4D1lewauX9OwTaPuY=\ngithub.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=\ngithub.com/probe-lab/go-libdht v0.4.0 h1:LAqHuko/owRW6+0cs5wmJXbHzg09EUMJEh5DI37yXqo=\ngithub.com/probe-lab/go-libdht v0.4.0/go.mod h1:hamw22kI6YkPQFGy5P6BrWWDrgE9ety5Si8iWAyuDvc=\ngithub.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=\ngithub.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=\ngithub.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=\ngithub.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0=\ngithub.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=\ngithub.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=\ngithub.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ=\ngithub.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=\ngithub.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=\ngithub.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=\ngithub.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=\ngithub.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=\ngithub.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=\ngithub.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=\ngithub.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc=\ngithub.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls=\ngithub.com/prometheus/common v0.35.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA=\ngithub.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA=\ngithub.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=\ngithub.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=\ngithub.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=\ngithub.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=\ngithub.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=\ngithub.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=\ngithub.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=\ngithub.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4=\ngithub.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=\ngithub.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=\ngithub.com/prometheus/statsd_exporter v0.22.7/go.mod h1:N/TevpjkIh9ccs6nuzY3jQn9dFqnUakOjnEuMPJJJnI=\ngithub.com/prometheus/statsd_exporter v0.27.1 h1:tcRJOmwlA83HPfWzosAgr2+zEN5XDFv+M2mn/uYkn5Y=\ngithub.com/prometheus/statsd_exporter v0.27.1/go.mod h1:vA6ryDfsN7py/3JApEst6nLTJboq66XsNcJGNmC88NQ=\ngithub.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=\ngithub.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=\ngithub.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=\ngithub.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=\ngithub.com/quic-go/webtransport-go v0.10.0 h1:LqXXPOXuETY5Xe8ITdGisBzTYmUOy5eSj+9n4hLTjHI=\ngithub.com/quic-go/webtransport-go v0.10.0/go.mod h1:LeGIXr5BQKE3UsynwVBeQrU1TPrbh73MGoC6jd+V7ow=\ngithub.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=\ngithub.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=\ngithub.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=\ngithub.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=\ngithub.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=\ngithub.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=\ngithub.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=\ngithub.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=\ngithub.com/rs/cors v1.11.1 h1:eU3gRzXLRK57F5rKMGMZURNdIG4EoAmX8k94r9wXWHA=\ngithub.com/rs/cors v1.11.1/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU=\ngithub.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=\ngithub.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=\ngithub.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk=\ngithub.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=\ngithub.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=\ngithub.com/shurcooL/go v0.0.0-20200502201357-93f07166e636/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk=\ngithub.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg=\ngithub.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=\ngithub.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546/go.mod h1:TrYk7fJVaAttu97ZZKrO9UbRa8izdowaMIZcxYMbVaw=\ngithub.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=\ngithub.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=\ngithub.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=\ngithub.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=\ngithub.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=\ngithub.com/slok/go-http-metrics v0.13.0/go.mod h1:HIr7t/HbN2sJaunvnt9wKP9xoBBVZFo1/KiHU3b0w+4=\ngithub.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY=\ngithub.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=\ngithub.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=\ngithub.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=\ngithub.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=\ngithub.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=\ngithub.com/smola/gocompat v0.2.0/go.mod h1:1B0MlxbmoZNo3h8guHp8HztB3BSYR5itql9qtVc0ypY=\ngithub.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572/go.mod h1:w0SWMsp6j9O/dk4/ZpIhL+3CkG8ofA2vuv7k+ltqUMc=\ngithub.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=\ngithub.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=\ngithub.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=\ngithub.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=\ngithub.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=\ngithub.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=\ngithub.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk=\ngithub.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=\ngithub.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=\ngithub.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=\ngithub.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=\ngithub.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=\ngithub.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns=\ngithub.com/src-d/envconfig v1.0.0/go.mod h1:Q9YQZ7BKITldTBnoxsE5gOeB5y66RyPXeue/R4aaNBc=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=\ngithub.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=\ngithub.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=\ngithub.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=\ngithub.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=\ngithub.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngithub.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=\ngithub.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=\ngithub.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=\ngithub.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=\ngithub.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=\ngithub.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=\ngithub.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=\ngithub.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=\ngithub.com/stvp/go-udp-testing v0.0.0-20201019212854-469649b16807/go.mod h1:7jxmlfBCDBXRzr0eAQJ48XC1hBu1np4CS5+cHEYfwpc=\ngithub.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=\ngithub.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=\ngithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d h1:vfofYNRScrDdvS342BElfbETmL1Aiz3i2t0zfRj16Hs=\ngithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d/go.mod h1:RRCYJbIwD5jmqPI9XoAFR0OcDxqUctll6zUj/+B4S48=\ngithub.com/texttheater/golang-levenshtein v1.0.1 h1:+cRNoVrfiwufQPhoMzB6N0Yf/Mqajr6t1lOv8GyGE2U=\ngithub.com/texttheater/golang-levenshtein v1.0.1/go.mod h1:PYAKrbF5sAiq9wd+H82hs7gNaen0CplQ9uvm6+enD/8=\ngithub.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=\ngithub.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=\ngithub.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=\ngithub.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=\ngithub.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=\ngithub.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=\ngithub.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=\ngithub.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=\ngithub.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=\ngithub.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=\ngithub.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c h1:u6SKchux2yDvFQnDHS3lPnIRmfVJ5Sxy3ao2SIdysLQ=\ngithub.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c/go.mod h1:hzIxponao9Kjc7aWznkXaL4U4TWaDSs8zcsY4Ka08nM=\ngithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb h1:Ywfo8sUltxogBpFuMOFRrrSifO788kAFxmvVw31PtQQ=\ngithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb/go.mod h1:ikPs9bRWicNw3S7XpJ8sK/smGwU9WcSVU3dy9qahYBM=\ngithub.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=\ngithub.com/urfave/cli v1.22.10/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=\ngithub.com/wangjia184/sortedset v0.0.0-20160527075905-f5d03557ba30/go.mod h1:YkocrP2K2tcw938x9gCOmT5G5eCD6jsTz0SZuyAqwIE=\ngithub.com/warpfork/go-testmark v0.12.1 h1:rMgCpJfwy1sJ50x0M0NgyphxYYPMOODIJHhsXyEHU0s=\ngithub.com/warpfork/go-testmark v0.12.1/go.mod h1:kHwy7wfvGSPh1rQJYKayD4AbtNaeyZdcGi9tNJTaa5Y=\ngithub.com/warpfork/go-wish v0.0.0-20200122115046-b9ea61034e4a/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw=\ngithub.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0 h1:GDDkbFiaK8jsSDJfjId/PEGEShv6ugrt4kYsC5UIDaQ=\ngithub.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw=\ngithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc h1:BCPnHtcboadS0DvysUuJXZ4lWVv5Bh5i7+tbIyi+ck4=\ngithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc/go.mod h1:r45hJU7yEoA81k6MWNhpMj/kms0n14dkzkxYHoB96UM=\ngithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11 h1:5HZfQkwe0mIfyDmc1Em5GqlNRzcdtlv4HTNmdpt7XH0=\ngithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11/go.mod h1:Wlo/SzPmxVp6vXpGt/zaXhHH0fn4IxgqZc82aKg6bpQ=\ngithub.com/whyrusleeping/cbor-gen v0.3.1 h1:82ioxmhEYut7LBVGhGq8xoRkXPLElVuh5mV67AFfdv0=\ngithub.com/whyrusleeping/cbor-gen v0.3.1/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so=\ngithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f h1:jQa4QT2UP9WYv2nzyawpKMOCl+Z/jW7djv2/J50lj9E=\ngithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f/go.mod h1:p9UJB6dDgdPgMJZs7UjUOdulKyRr9fqkS+6JKAInPy8=\ngithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 h1:EKhdznlJHPMoKr0XTrX+IlJs1LH3lyx2nfr1dOlZ79k=\ngithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1/go.mod h1:8UvriyWtv5Q5EOgjHaSseUEdkQfvwFv1I/In/O2M9gc=\ngithub.com/whyrusleeping/go-logging v0.0.0-20170515211332-0457bb6b88fc/go.mod h1:bopw91TMyo8J3tvftk8xmU2kPmlrt4nScJQZU2hE5EM=\ngithub.com/whyrusleeping/go-sysinfo v0.0.0-20190219211824-4a357d4b90b1 h1:ctS9Anw/KozviCCtK6VWMz5kPL9nbQzbQY4yfqlIV4M=\ngithub.com/whyrusleeping/go-sysinfo v0.0.0-20190219211824-4a357d4b90b1/go.mod h1:tKH72zYNt/exx6/5IQO6L9LoQ0rEjd5SbbWaDTs9Zso=\ngithub.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7 h1:E9S12nwJwEOXe2d6gT6qxdvqMnNq+VnSsKPgm2ZZNds=\ngithub.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7/go.mod h1:X2c0RVCI1eSUFI8eLcY3c0423ykwiUdxLJtkDvruhjI=\ngithub.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU=\ngithub.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA=\ngithub.com/x-cray/logrus-prefixed-formatter v0.5.2/go.mod h1:2duySbKsL6M18s5GU7VPsoEPHyzalCE06qoARUCeBBE=\ngithub.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=\ngithub.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=\ngithub.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=\ngithub.com/zeebo/assert v1.1.0 h1:hU1L1vLTHsnO8x8c9KAR5GmM5QscxHg5RNU5z5qbUWY=\ngithub.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=\ngithub.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI=\ngithub.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE=\ngithub.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo=\ngithub.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4=\ngo.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=\ngo.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=\ngo.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ=\ngo.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=\ngo.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=\ngo.opencensus.io v0.22.1/go.mod h1:Ap50jQcDJrx6rB6VgeeFPtuPIf3wMRvRfrfYDO6+BmA=\ngo.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=\ngo.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=\ngo.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=\ngo.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=\ngo.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=\ngo.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 h1:7iP2uCb7sGddAr30RRS6xjKy7AZ2JtTOPA3oolgVSw8=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0/go.mod h1:c7hN3ddxs/z6q9xwvfLPk+UHlWRQyaeR1LdgfL/66l0=\ngo.opentelemetry.io/contrib/propagators/autoprop v0.46.1 h1:cXTYcMjY0dsYokAuo8LbNBQxpF8VgTHdiHJJ1zlIXl4=\ngo.opentelemetry.io/contrib/propagators/autoprop v0.46.1/go.mod h1:WZxgny1/6+j67B1s72PLJ4bGjidoWFzSmLNfJKVt2bo=\ngo.opentelemetry.io/contrib/propagators/aws v1.21.1 h1:uQIQIDWb0gzyvon2ICnghpLAf9w7ADOCUiIiwCQgR2o=\ngo.opentelemetry.io/contrib/propagators/aws v1.21.1/go.mod h1:kCcto3ACQxm+VrkQX/NK/TkDmAd99MQhvffzyTKhzL4=\ngo.opentelemetry.io/contrib/propagators/b3 v1.21.1 h1:WPYiUgmw3+b7b3sQ1bFBFAf0q+Di9dvNc3AtYfnT4RQ=\ngo.opentelemetry.io/contrib/propagators/b3 v1.21.1/go.mod h1:EmzokPoSqsYMBVK4nRnhsfm5mbn8J1eDuz/U1UaQaWg=\ngo.opentelemetry.io/contrib/propagators/jaeger v1.21.1 h1:f4beMGDKiVzg9IcX7/VuWVy+oGdjx3dNJ72YehmtY5k=\ngo.opentelemetry.io/contrib/propagators/jaeger v1.21.1/go.mod h1:U9jhkEl8d1LL+QXY7q3kneJWJugiN3kZJV2OWz3hkBY=\ngo.opentelemetry.io/contrib/propagators/ot v1.21.1 h1:3TN5vkXjKYWp0YdMcnUEC/A+pBPvqz9V3nCS2xmcurk=\ngo.opentelemetry.io/contrib/propagators/ot v1.21.1/go.mod h1:oy0MYCbS/b3cqUDW37wBWtlwBIsutngS++Lklpgh+fc=\ngo.opentelemetry.io/otel v1.42.0 h1:lSQGzTgVR3+sgJDAU/7/ZMjN9Z+vUip7leaqBKy4sho=\ngo.opentelemetry.io/otel v1.42.0/go.mod h1:lJNsdRMxCUIWuMlVJWzecSMuNjE7dOYyWlqOXWkdqCc=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace v1.40.0 h1:QKdN8ly8zEMrByybbQgv8cWBcdAarwmIPZ6FThrWXJs=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace v1.40.0/go.mod h1:bTdK1nhqF76qiPoCCdyFIV+N/sRHYXYCTQc+3VCi3MI=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.40.0 h1:DvJDOPmSWQHWywQS6lKL+pb8s3gBLOZUtw4N+mavW1I=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.40.0/go.mod h1:EtekO9DEJb4/jRyN4v4Qjc2yA7AtfCBuz2FynRUWTXs=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.40.0 h1:wVZXIWjQSeSmMoxF74LzAnpVQOAFDo3pPji9Y4SOFKc=\ngo.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.40.0/go.mod h1:khvBS2IggMFNwZK/6lEeHg/W57h/IX6J4URh57fuI40=\ngo.opentelemetry.io/otel/exporters/prometheus v0.56.0 h1:GnCIi0QyG0yy2MrJLzVrIM7laaJstj//flf1zEJCG+E=\ngo.opentelemetry.io/otel/exporters/prometheus v0.56.0/go.mod h1:JQcVZtbIIPM+7SWBB+T6FK+xunlyidwLp++fN0sUaOk=\ngo.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.40.0 h1:MzfofMZN8ulNqobCmCAVbqVL5syHw+eB2qPRkCMA/fQ=\ngo.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.40.0/go.mod h1:E73G9UFtKRXrxhBsHtG00TB5WxX57lpsQzogDkqBTz8=\ngo.opentelemetry.io/otel/metric v1.42.0 h1:2jXG+3oZLNXEPfNmnpxKDeZsFI5o4J+nz6xUlaFdF/4=\ngo.opentelemetry.io/otel/metric v1.42.0/go.mod h1:RlUN/7vTU7Ao/diDkEpQpnz3/92J9ko05BIwxYa2SSI=\ngo.opentelemetry.io/otel/sdk v1.40.0 h1:KHW/jUzgo6wsPh9At46+h4upjtccTmuZCFAc9OJ71f8=\ngo.opentelemetry.io/otel/sdk v1.40.0/go.mod h1:Ph7EFdYvxq72Y8Li9q8KebuYUr2KoeyHx0DRMKrYBUE=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0 h1:mtmdVqgQkeRxHgRv4qhyJduP3fYJRMX4AtAlbuWdCYw=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0/go.mod h1:4Z2bGMf0KSK3uRjlczMOeMhKU2rhUqdWNoKcYrtcBPg=\ngo.opentelemetry.io/otel/trace v1.42.0 h1:OUCgIPt+mzOnaUTpOQcBiM/PLQ/Op7oq6g4LenLmOYY=\ngo.opentelemetry.io/otel/trace v1.42.0/go.mod h1:f3K9S+IFqnumBkKhRJMeaZeNk9epyhnCmQh/EysQCdc=\ngo.opentelemetry.io/proto/otlp v1.9.0 h1:l706jCMITVouPOqEnii2fIAuO3IVGBRPV5ICjceRb/A=\ngo.opentelemetry.io/proto/otlp v1.9.0/go.mod h1:xE+Cx5E/eEHw+ISFkwPLwCZefwVjY+pqKg1qcK03+/4=\ngo.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=\ngo.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4=\ngo.uber.org/dig v1.19.0/go.mod h1:Us0rSJiThwCv2GteUN0Q7OKvU7n5J4dxZ9JKUXozFdE=\ngo.uber.org/fx v1.24.0 h1:wE8mruvpg2kiiL1Vqd0CC+tr0/24XIB10Iwp2lLWzkg=\ngo.uber.org/fx v1.24.0/go.mod h1:AmDeGyS+ZARGKM4tlH4FY2Jr63VjbEDJHtqXTGP5hbo=\ngo.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=\ngo.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=\ngo.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=\ngo.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=\ngo.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=\ngo.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=\ngo.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=\ngo.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=\ngo.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=\ngo.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=\ngo.uber.org/zap/exp v0.3.0 h1:6JYzdifzYkGmTdRR59oYH+Ng7k49H9qVpWwNSsGJj3U=\ngo.uber.org/zap/exp v0.3.0/go.mod h1:5I384qq7XGxYyByIhHm6jg5CHkGY0nsTfbDLgDDlgJQ=\ngo.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=\ngo.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=\ngo4.org v0.0.0-20230225012048-214862532bf5 h1:nifaUDeh+rPaBCMPMQHZmvJf+QdpLFnuQPwx+LxVmtc=\ngo4.org v0.0.0-20230225012048-214862532bf5/go.mod h1:F57wTi5Lrj6WLyswp5EYV1ncrEbFGHD4hhz6S1ZYeaU=\ngolang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20200602180216-279210d13fed/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=\ngolang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=\ngolang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=\ngolang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=\ngolang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=\ngolang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=\ngolang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=\ngolang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=\ngolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90 h1:jiDhWWeC7jfWqR9c/uplMOqJ0sbNlNWv0UkzE0vX1MA=\ngolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90/go.mod h1:xE1HEv6b+1SCZ5/uscMRjUBKtIxworgEcEi+/n9NQDQ=\ngolang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=\ngolang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=\ngolang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=\ngolang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=\ngolang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=\ngolang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=\ngolang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=\ngolang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=\ngolang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190227160552-c95aed5357e7/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=\ngolang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=\ngolang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=\ngolang.org/x/net v0.0.0-20210423184538-5f58ad60dda6/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=\ngolang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=\ngolang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=\ngolang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=\ngolang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=\ngolang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=\ngolang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=\ngolang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=\ngolang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ=\ngolang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=\ngolang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=\ngolang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190219092855-153ac476189d/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190626221950-04f50cda93cb/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210303074136-134d130e1a04/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210309074719-68d13333faf2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210426080607-c94f62235c83/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220708085239-5a0f0661e09d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=\ngolang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=\ngolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c h1:6a8FdnNk6bTXBjR4AGKFgUKuo+7GnR3FX5L7CbveeZc=\ngolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c/go.mod h1:TpUTTEp9frx7rTdLpC9gFG9kdI7zVLFTFFlqaH2Cncw=\ngolang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=\ngolang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=\ngolang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU=\ngolang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A=\ngolang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=\ngolang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=\ngolang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\ngolang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=\ngolang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=\ngolang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=\ngolang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=\ngolang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=\ngolang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20181130052023-1c3d964395ce/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=\ngolang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=\ngolang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=\ngolang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=\ngolang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=\ngolang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s=\ngolang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=\ngolang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=\ngolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da h1:noIWHXmPHxILtqtCOPIhSt0ABwskkZKjD3bXGnZGpNY=\ngolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=\ngonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=\ngonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=\ngoogle.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=\ngoogle.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=\ngoogle.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=\ngoogle.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=\ngoogle.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=\ngoogle.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=\ngoogle.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=\ngoogle.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=\ngoogle.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=\ngoogle.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=\ngoogle.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=\ngoogle.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=\ngoogle.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=\ngoogle.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\ngoogle.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=\ngoogle.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=\ngoogle.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 h1:merA0rdPeUV3YIIfHHcH4qBkiQAc1nfCKSI7lB4cV2M=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409/go.mod h1:fl8J1IvUjCilwZzQowmw2b7HQB2eAuYBabMXzWurF+I=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20260128011058-8636f8732409 h1:H86B94AW+VfJWDqFeEbBPhEtHzJwJfTbgE2lZa54ZAQ=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20260128011058-8636f8732409/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=\ngoogle.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=\ngoogle.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=\ngoogle.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=\ngoogle.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\ngoogle.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=\ngoogle.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=\ngoogle.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc=\ngoogle.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=\ngoogle.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\ngoogle.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=\ngoogle.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\ngoogle.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=\ngoogle.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=\ngoogle.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=\ngoogle.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=\ngopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=\ngopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=\ngopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=\ngopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/src-d/go-cli.v0 v0.0.0-20181105080154-d492247bbc0d/go.mod h1:z+K8VcOYVYcSwSjGebuDL6176A1XskgbtNl64NSg+n8=\ngopkg.in/src-d/go-log.v1 v1.0.1/go.mod h1:GN34hKP0g305ysm2/hctJ0Y8nWP3zxXXJ8GFabTyABE=\ngopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=\ngopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=\ngopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\ngopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\ngopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\ngopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=\nhonnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nhonnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nlukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg=\nlukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo=\npgregory.net/rapid v1.1.0 h1:CMa0sjHSru3puNx+J0MIAuiiEV4N0qj8/cMWGBBCsjw=\npgregory.net/rapid v1.1.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04=\nrsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=\nrsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=\nrsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=\n" }, { "path": "misc/README.md", "content": "## init system integration\n\ngo-ipfs can be started by your operating system's native init system.\n\n- [systemd](#systemd)\n- [LSB init script](#initd)\n- [Upstart/startup job](#upstart)\n- [launchd](#launchd)\n\n### systemd\n\nFor `systemd`, the best approach is to run the daemon in a user session. Here is a sample service file:\n\n```systemd\n[Unit]\nDescription=IPFS daemon\n\n[Service]\n# Environment=\"IPFS_PATH=/data/ipfs\" # optional path to ipfs init directory if not default ($HOME/.ipfs)\nExecStart=/usr/local/bin/ipfs daemon\nRestart=on-failure\n\n[Install]\nWantedBy=default.target\n```\n\nTo run this in your user session, save it as `~/.config/systemd/user/ipfs.service` (creating directories as necessary). Once you run `ipfs init` to create your IPFS settings, you can control the daemon using the following commands:\n\n* `systemctl --user start ipfs` - start the daemon\n* `systemctl --user stop ipfs` - stop the daemon\n* `systemctl --user status ipfs` - get status of the daemon\n* `systemctl --user enable ipfs` - enable starting the daemon at boot\n* `systemctl --user disable ipfs` - disable starting the daemon at boot\n\n*Note:* If you want this `--user` service to run at system boot, you must [`enable-linger`](http://www.freedesktop.org/software/systemd/man/loginctl.html) on the account that runs the service:\n\n```\n# loginctl enable-linger [user]\n```\nRead more about `--user` services here: [wiki.archlinux.org:Systemd ](https://wiki.archlinux.org/index.php/Systemd/User#Automatic_start-up_of_systemd_user_instances)\n\n#### P2P tunnel services\n\nFor running `ipfs p2p listen` or `ipfs p2p forward` as systemd services,\nsee [docs/p2p-tunnels.md](../docs/p2p-tunnels.md) for examples using the\n`--foreground` flag and path-based activation.\n\n### initd\n\n- Here is a full-featured sample service file: https://github.com/dylanPowers/ipfs-linux-service/blob/master/init.d/ipfs\n- Use `service` or your distribution's equivalent to control the service.\n\n## upstart\n\n- And below is a very basic sample upstart job. **Note the username jbenet**.\n\n```\ncat /etc/init/ipfs.conf\n```\n```\ndescription \"ipfs: interplanetary filesystem\"\n\nstart on (local-filesystems and net-device-up IFACE!=lo)\nstop on runlevel [!2345]\n\nlimit nofile 524288 1048576\nlimit nproc 524288 1048576\nsetuid jbenet\nchdir /home/jbenet\nrespawn\nexec ipfs daemon\n```\n\nAnother version is available here:\n\n```sh\nipfs cat /ipfs/QmbYCwVeA23vz6mzAiVQhJNa2JSiRH4ebef1v2e5EkDEZS/ipfs.conf >/etc/init/ipfs.conf\n```\n\nFor both, edit to replace occurrences of `jbenet` with whatever user you want it to run as:\n\n```sh\nsed -i s/jbenet// /etc/init/ipfs.conf\n```\n\nOnce you run `ipfs init` to create your IPFS settings, you can control the daemon using the `init.d` commands:\n\n```sh\nsudo service ipfs start\nsudo service ipfs stop\nsudo service ipfs restart\n...\n```\n\n## launchd\n\nSimilar to `systemd`, on macOS you can run `go-ipfs` via a user LaunchAgent.\n\n- Create `~/Library/LaunchAgents/io.ipfs.go-ipfs.plist`:\n\n```xml\n\n\n\n\n KeepAlive\n \n Label\n io.ipfs.go-ipfs\n ProcessType\n Background\n ProgramArguments\n \n /bin/sh\n -c\n ~/go/bin/ipfs daemon\n \n RunAtLoad\n \n\n\n```\nThe reason for running `ipfs` under a shell is to avoid needing to hard-code the user's home directory in the job.\n\n- To start the job, run `launchctl load ~/Library/LaunchAgents/io.ipfs.go-ipfs.plist`\n\nNotes:\n\n- To check that the job is running, run `launchctl list | grep ipfs`.\n- IPFS should now start whenever you log in (and exit when you log out).\n- [LaunchControl](http://www.soma-zone.com/LaunchControl/) is a GUI tool which simplifies management of LaunchAgents.\n" }, { "path": "misc/fsutil/fsutil.go", "content": "package fsutil\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n\t\"os\"\n\t\"path/filepath\"\n)\n\n// DirWritable checks if a directory is writable. If the directory does\n// not exist it is created with writable permission.\nfunc DirWritable(dir string) error {\n\tif dir == \"\" {\n\t\treturn errors.New(\"directory not specified\")\n\t}\n\n\tvar err error\n\tdir, err = ExpandHome(dir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfi, err := os.Stat(dir)\n\tif err != nil {\n\t\tif errors.Is(err, fs.ErrNotExist) {\n\t\t\t// Directory does not exist, so create it.\n\t\t\terr = os.Mkdir(dir, 0775)\n\t\t\tif err == nil {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tif errors.Is(err, fs.ErrPermission) {\n\t\t\terr = fs.ErrPermission\n\t\t}\n\t\treturn fmt.Errorf(\"directory not writable: %s: %w\", dir, err)\n\t}\n\tif !fi.IsDir() {\n\t\treturn fmt.Errorf(\"not a directory: %s\", dir)\n\t}\n\n\t// Directory exists, check that a file can be written.\n\tfile, err := os.CreateTemp(dir, \"writetest\")\n\tif err != nil {\n\t\tif errors.Is(err, fs.ErrPermission) {\n\t\t\terr = fs.ErrPermission\n\t\t}\n\t\treturn fmt.Errorf(\"directory not writable: %s: %w\", dir, err)\n\t}\n\tfile.Close()\n\treturn os.Remove(file.Name())\n}\n\n// ExpandHome expands the path to include the home directory if the path is\n// prefixed with `~`. If it isn't prefixed with `~`, the path is returned\n// as-is.\nfunc ExpandHome(path string) (string, error) {\n\tif path == \"\" {\n\t\treturn path, nil\n\t}\n\n\tif path[0] != '~' {\n\t\treturn path, nil\n\t}\n\n\tif len(path) > 1 && path[1] != '/' && path[1] != '\\\\' {\n\t\treturn \"\", errors.New(\"cannot expand user-specific home dir\")\n\t}\n\n\tdir, err := os.UserHomeDir()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn filepath.Join(dir, path[1:]), nil\n}\n\n// FileExists return true if the file exists\nfunc FileExists(filename string) bool {\n\t_, err := os.Lstat(filename)\n\treturn !errors.Is(err, os.ErrNotExist)\n}\n" }, { "path": "misc/fsutil/fsutil_test.go", "content": "package fsutil_test\n\nimport (\n\t\"io/fs\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/misc/fsutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestDirWritable(t *testing.T) {\n\terr := fsutil.DirWritable(\"\")\n\trequire.Error(t, err)\n\n\terr = fsutil.DirWritable(\"~nosuchuser/tmp\")\n\trequire.Error(t, err)\n\n\ttmpDir := t.TempDir()\n\n\twrDir := filepath.Join(tmpDir, \"readwrite\")\n\terr = fsutil.DirWritable(wrDir)\n\trequire.NoError(t, err)\n\n\t// Check that DirWritable created directory.\n\tfi, err := os.Stat(wrDir)\n\trequire.NoError(t, err)\n\trequire.True(t, fi.IsDir())\n\n\terr = fsutil.DirWritable(wrDir)\n\trequire.NoError(t, err)\n\n\t// If running on Windows, skip read-only directory tests.\n\tif runtime.GOOS == \"windows\" {\n\t\tt.SkipNow()\n\t}\n\n\troDir := filepath.Join(tmpDir, \"readonly\")\n\trequire.NoError(t, os.Mkdir(roDir, 0500))\n\terr = fsutil.DirWritable(roDir)\n\trequire.ErrorIs(t, err, fs.ErrPermission)\n\n\troChild := filepath.Join(roDir, \"child\")\n\terr = fsutil.DirWritable(roChild)\n\trequire.ErrorIs(t, err, fs.ErrPermission)\n}\n\nfunc TestFileExists(t *testing.T) {\n\tfileName := filepath.Join(t.TempDir(), \"somefile\")\n\trequire.False(t, fsutil.FileExists(fileName))\n\n\tfile, err := os.Create(fileName)\n\trequire.NoError(t, err)\n\tfile.Close()\n\n\trequire.True(t, fsutil.FileExists(fileName))\n}\n\nfunc TestExpandHome(t *testing.T) {\n\tdir, err := fsutil.ExpandHome(\"\")\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"\", dir)\n\n\torigDir := filepath.Join(\"somedir\", \"somesub\")\n\tdir, err = fsutil.ExpandHome(origDir)\n\trequire.NoError(t, err)\n\trequire.Equal(t, origDir, dir)\n\n\t_, err = fsutil.ExpandHome(filepath.FromSlash(\"~nosuchuser/somedir\"))\n\trequire.Error(t, err)\n\n\thomeEnv := \"HOME\"\n\tif runtime.GOOS == \"windows\" {\n\t\thomeEnv = \"USERPROFILE\"\n\t}\n\torigHome := os.Getenv(homeEnv)\n\tdefer os.Setenv(homeEnv, origHome)\n\thomeDir := filepath.Join(t.TempDir(), \"testhome\")\n\tos.Setenv(homeEnv, homeDir)\n\n\tconst subDir = \"mytmp\"\n\torigDir = filepath.Join(\"~\", subDir)\n\tdir, err = fsutil.ExpandHome(origDir)\n\trequire.NoError(t, err)\n\trequire.Equal(t, filepath.Join(homeDir, subDir), dir)\n\n\tos.Unsetenv(homeEnv)\n\t_, err = fsutil.ExpandHome(origDir)\n\trequire.Error(t, err)\n}\n" }, { "path": "misc/launchd/README.md", "content": "# ipfs launchd agent\n\nA bare-bones launchd agent file for ipfs. To have launchd automatically run the ipfs daemon for you, run `./misc/launchd/install.sh`\n\n" }, { "path": "misc/launchd/install.sh", "content": "#!/bin/bash\n\nsrc_dir=$( cd \"$( dirname \"${BASH_SOURCE[0]}\" )\" && pwd )\nplist=io.ipfs.ipfs-daemon.plist\ndest_dir=\"$HOME/Library/LaunchAgents\"\nIPFS_PATH=\"${IPFS_PATH:-$HOME/.ipfs}\"\nescaped_ipfs_path=$(echo $IPFS_PATH|sed 's/\\//\\\\\\//g')\n\nIPFS_BIN=$(which ipfs || echo ipfs)\nescaped_ipfs_bin=$(echo $IPFS_BIN|sed 's/\\//\\\\\\//g')\n\nmkdir -p \"$dest_dir\"\n\nsed -e 's/{{IPFS_PATH}}/'\"$escaped_ipfs_path\"'/g' \\\n -e 's/{{IPFS_BIN}}/'\"$escaped_ipfs_bin\"'/g' \\\n \"$src_dir/$plist\" \\\n > \"$dest_dir/$plist\"\n\nlaunchctl list | grep ipfs-daemon >/dev/null\nif [ $? ]; then\n echo Unloading existing ipfs-daemon\n launchctl unload \"$dest_dir/$plist\"\nfi\n\necho Loading ipfs-daemon\nif (( `sw_vers -productVersion | cut -d'.' -f2` > 9 )); then\n sudo chown root \"$dest_dir/$plist\"\n sudo launchctl bootstrap system \"$dest_dir/$plist\"\nelse\n launchctl load \"$dest_dir/$plist\"\nfi\nlaunchctl list | grep ipfs-daemon\n" }, { "path": "misc/launchd/io.ipfs.ipfs-daemon.plist", "content": "\n\n\n \n KeepAlive\n \n Label\n io.ipfs.ipfs-daemon\n ProgramArguments\n \n {{IPFS_BIN}}\n daemon\n \n EnvironmentVariables\n \n IPFS_PATH\n {{IPFS_PATH}}\n \n RunAtLoad\n \n \n\n\n" }, { "path": "misc/systemd/ipfs-api.socket", "content": "# Enabling this will *completely override* any API listeners configured in your\n# config.\n\n[Unit]\nDescription=IPFS API Socket\n\n[Socket]\nService=ipfs.service\nFileDescriptorName=io.ipfs.api\nBindIPv6Only=true\nListenStream=127.0.0.1:5001\nListenStream=[::1]:5001\n\n[Install]\nWantedBy=sockets.target\n" }, { "path": "misc/systemd/ipfs-gateway.socket", "content": "# Enabling this will *completely override* any Gateway listeners configured in\n# your config.\n\n[Unit]\nDescription=IPFS Gateway Socket\n\n[Socket]\nService=ipfs.service\nFileDescriptorName=io.ipfs.gateway\nBindIPv6Only=true\nListenStream=127.0.0.1:8080\nListenStream=[::1]:8080\n\n[Install]\nWantedBy=sockets.target\n" }, { "path": "misc/systemd/ipfs-hardened.service", "content": "# This file will be overwritten on package upgrades, avoid customizations here.\n#\n# To make persistent changes, create file in \n# \"/etc/systemd/system/ipfs.service.d/overwrite.conf\" with \n# `systemctl edit ipfs.service`. This file will be parsed after this \n# file has been parsed.\n#\n# To overwrite a variable, like ExecStart you have to specify it once\n# blank and a second time with a new value, like:\n# ExecStart=\n# ExecStart=/usr/local/bin/ipfs daemon --flag1 --flag2\n#\n# For more info about custom unit files see systemd.unit(5).\n\n# This service file enables systemd-hardening features compatible with IPFS,\n# while breaking compatibility with the fuse-mount function. Use this one only \n# if you don't need the fuse-mount functionality.\n\n[Unit]\nDescription=InterPlanetary File System (IPFS) daemon\nDocumentation=https://docs.ipfs.tech/\nAfter=network.target\n\n[Service]\n# hardening\nReadWritePaths=\"/var/lib/ipfs/\"\nNoNewPrivileges=true\nProtectSystem=strict\nProtectKernelTunables=true\nProtectKernelModules=true\nProtectKernelLogs=true\nPrivateDevices=true\nDevicePolicy=closed\nProtectControlGroups=true\nRestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK\nProtectHostname=true\nPrivateTmp=true\nProtectClock=true\nLockPersonality=true\nRestrictNamespaces=true\nRestrictRealtime=true\nMemoryDenyWriteExecute=true\nSystemCallArchitectures=native\nSystemCallFilter=@system-service\nSystemCallFilter=~@privileged\nProtectHome=true\nRemoveIPC=true\nRestrictSUIDSGID=true\nCapabilityBoundingSet=CAP_NET_BIND_SERVICE\n\n# enable for 1-1024 port listening\n#AmbientCapabilities=CAP_NET_BIND_SERVICE \n# enable to specify a custom path see docs/environment-variables.md for further documentations\n#Environment=IPFS_PATH=/custom/ipfs/path\n# enable to specify a higher limit for open files/connections\n#LimitNOFILE=1000000\n\n#don't use swap\nMemorySwapMax=0\n\n# Don't timeout on startup. Opening the IPFS repo can take a long time in some cases (e.g., when\n# badger is recovering) and migrations can delay startup.\n#\n# Ideally, we'd be a bit smarter about this but there's no good way to do that without hooking\n# systemd dependencies deeper into go-ipfs.\nTimeoutStartSec=infinity\n\nType=notify\nUser=ipfs\nGroup=ipfs\nStateDirectory=ipfs\nEnvironment=IPFS_PATH=\"${HOME}\"\nExecStart=/usr/local/bin/ipfs daemon --init --migrate\nRestart=on-failure\nKillSignal=SIGINT\n\n[Install]\nWantedBy=default.target\n" }, { "path": "misc/systemd/ipfs-sysusers.conf", "content": "u ipfs - \"IPFS daemon\" /var/lib/ipfs\ng ipfs -\nm ipfs ipfs\n" }, { "path": "misc/systemd/ipfs.service", "content": "# This file will be overwritten on package upgrades, avoid customizations here.\n#\n# To make persistent changes, create file in \n# \"/etc/systemd/system/ipfs.service.d/overwrite.conf\" with \n# `systemctl edit ipfs.service`. This file will be parsed after this \n# file has been parsed.\n#\n# To overwrite a variable, like ExecStart you have to specify it once\n# blank and a second time with a new value, like:\n# ExecStart=\n# ExecStart=/usr/local/bin/ipfs daemon --flag1 --flag2\n#\n# For more info about custom unit files see systemd.unit(5).\n\n[Unit]\nDescription=InterPlanetary File System (IPFS) daemon\nDocumentation=https://docs.ipfs.tech/\nAfter=network.target\n\n[Service]\n\n# enable for 1-1024 port listening\n#AmbientCapabilities=CAP_NET_BIND_SERVICE \n# enable to specify a custom path see docs/environment-variables.md for further documentations\n#Environment=IPFS_PATH=/custom/ipfs/path\n# enable to specify a higher limit for open files/connections\n#LimitNOFILE=1000000\n\n#don't use swap\nMemorySwapMax=0\n\n# Don't timeout on startup. Opening the IPFS repo can take a long time in some cases (e.g., when\n# badger is recovering) and migrations can delay startup.\n#\n# Ideally, we'd be a bit smarter about this but there's no good way to do that without hooking\n# systemd dependencies deeper into go-ipfs.\nTimeoutStartSec=infinity\n\nType=notify\nUser=ipfs\nGroup=ipfs\nStateDirectory=ipfs\nEnvironment=IPFS_PATH=\"${HOME}\"\nExecStart=/usr/local/bin/ipfs daemon --init --migrate\nRestart=on-failure\nKillSignal=SIGINT\n\n[Install]\nWantedBy=default.target\n" }, { "path": "mk/footer.mk", "content": "# standard NR-make boilerplate, to be included at the end of a file\nd := $(dirstack_$(sp))\nsp := $(basename $(sp))\n" }, { "path": "mk/git.mk", "content": "# First try to \"describe\" the state. This tells us if the state is dirty.\n# If that fails (e.g., we're building a docker image and have an empty objects\n# directory), assume the source isn't dirty and build anyways.\ngit-hash:=$(shell git describe --always --match=NeVeRmAtCh --dirty 2>/dev/null || git rev-parse --short HEAD 2>/dev/null)\n\n# Detect if HEAD is a clean, tagged release. Used to omit redundant commit\n# hash from the libp2p user agent (the version number suffices).\nifeq ($(findstring dirty,$(git-hash)),)\n git-tag:=$(shell git tag --points-at HEAD 2>/dev/null | grep '^v' | head -1)\nelse\n git-tag:=\nendif\n" }, { "path": "mk/golang.mk", "content": "# golang utilities\nexport GO111MODULE=on\n\n\n# pre-definitions\nGOCC ?= go\nGOTAGS ?=\nGOTFLAGS ?=\n\n# Unexport GOFLAGS so we only apply it where we actually want it.\nunexport GOFLAGS\n# Override so we can combine with the user's go flags.\n# Try to make building as reproducible as possible by stripping the go path.\noverride GOFLAGS += \"-trimpath\"\n\nifeq ($(tarball-is),1)\n\tGOFLAGS += -mod=vendor\nendif\n\n# match Go's default GOPATH behaviour\nexport GOPATH ?= $(shell $(GOCC) env GOPATH)\n\nDEPS_GO :=\nTEST_GO :=\nTEST_GO_BUILD :=\nCHECK_GO :=\n\ngo-pkg-name=$(shell GOFLAGS=-buildvcs=false $(GOCC) list $(go-tags) github.com/ipfs/kubo/$(1))\ngo-main-name=$(notdir $(call go-pkg-name,$(1)))$(?exe)\ngo-curr-pkg-tgt=$(d)/$(call go-main-name,$(d))\ngo-pkgs=$(shell GOFLAGS=-buildvcs=false $(GOCC) list github.com/ipfs/kubo/...)\n\ngo-tags=$(if $(GOTAGS), -tags=\"$(call join-with,$(space),$(GOTAGS))\")\ngo-flags-with-tags=$(GOFLAGS)$(go-tags)\n\ndefine go-build-relative\n$(GOCC) build $(go-flags-with-tags) -o \"$@\" \"$(call go-pkg-name,$<)\"\nendef\n\ndefine go-build\n$(GOCC) build $(go-flags-with-tags) -o \"$@\" \"$(1)\"\nendef\n\n# Only disable colors when running in CI (non-interactive terminal)\nGOTESTSUM_NOCOLOR := $(if $(CI),--no-color,)\n\n# Packages excluded from coverage (test code and examples are not production code)\nCOVERPKG_EXCLUDE := /(test|docs/examples)/\n\n# Packages excluded from unit tests: coverage exclusions + client/rpc (tested by test_cli)\nUNIT_EXCLUDE := /(test|docs/examples)/|/client/rpc$$\n\n# Unit tests with coverage\n# Produces JSON for CI reporting and coverage profile for Codecov\ntest_unit: test/bin/gotestsum $$(DEPS_GO)\n\tmkdir -p test/unit coverage\n\trm -f test/unit/gotest.json coverage/unit_tests.coverprofile\n\tgotestsum $(GOTESTSUM_NOCOLOR) --jsonfile test/unit/gotest.json -- $(go-flags-with-tags) $(GOTFLAGS) -covermode=atomic -coverprofile=coverage/unit_tests.coverprofile -coverpkg=$$($(GOCC) list $(go-tags) ./... | grep -vE '$(COVERPKG_EXCLUDE)' | tr '\\n' ',' | sed 's/,$$//') $$($(GOCC) list $(go-tags) ./... | grep -vE '$(UNIT_EXCLUDE)')\n.PHONY: test_unit\n\n# CLI/integration tests (requires built binary in PATH)\n# Includes test/cli, test/integration, and client/rpc\n# Produces JSON for CI reporting\n# Override TEST_CLI_TIMEOUT for local development: make test_cli TEST_CLI_TIMEOUT=5m\nTEST_CLI_TIMEOUT ?= 10m\ntest_cli: cmd/ipfs/ipfs test/bin/gotestsum $$(DEPS_GO)\n\tmkdir -p test/cli\n\trm -f test/cli/cli-tests.json\n\tPATH=\"$(CURDIR)/cmd/ipfs:$(CURDIR)/test/bin:$$PATH\" gotestsum $(GOTESTSUM_NOCOLOR) --jsonfile test/cli/cli-tests.json -- -v -timeout=$(TEST_CLI_TIMEOUT) ./test/cli/... ./test/integration/... ./client/rpc/...\n.PHONY: test_cli\n\n# Example tests (docs/examples/kubo-as-a-library)\n# Tests against both published and current kubo versions\n# Uses timeout to ensure CI gets output before job-level timeout kills everything\nTEST_EXAMPLES_TIMEOUT ?= 2m\ntest_examples:\n\tcd docs/examples/kubo-as-a-library && go test -v -timeout=$(TEST_EXAMPLES_TIMEOUT) ./... && cp go.mod go.mod.bak && cp go.sum go.sum.bak && (go mod edit -replace github.com/ipfs/kubo=./../../.. && go mod tidy && go test -v -timeout=$(TEST_EXAMPLES_TIMEOUT) ./...; ret=$$?; mv go.mod.bak go.mod; mv go.sum.bak go.sum; exit $$ret)\n.PHONY: test_examples\n\n# Build kubo for all platforms from .github/build-platforms.yml\ntest_go_build:\n\tbin/test-go-build-platforms\n.PHONY: test_go_build\n\n# Check Go source formatting\ntest_go_fmt:\n\tbin/test-go-fmt\n.PHONY: test_go_fmt\n\n# Run golangci-lint (used by CI)\ntest_go_lint: test/bin/golangci-lint\n\tgolangci-lint run --timeout=3m ./...\n.PHONY: test_go_lint\n\nTEST_GO := test_go_fmt test_unit test_cli test_examples\nTEST += $(TEST_GO)\nTEST_SHORT += test_go_fmt test_unit\n" }, { "path": "mk/header.mk", "content": "# keep track of dirs\n# standard NR-make boilerplate, to be included at the beginning of a file\np := $(sp).x\ndirstack_$(sp) := $(d)\nd := $(dir)\n" }, { "path": "mk/tarball.mk", "content": "\n\nifeq (,$(wildcard .tarball))\ntarball-is:=0\nelse\ntarball-is:=1\n# override git hash\ngit-hash:=$(shell cat .tarball)\nendif\n\nGOCC ?= go\n\ngo-ipfs-source.tar.gz: distclean\n\tGOCC=$(GOCC) bin/maketarball.sh $@\n\nkubo-source.tar.gz: distclean\n\tGOCC=$(GOCC) bin/maketarball.sh $@\n" }, { "path": "mk/util.mk", "content": "# util functions\nOS ?= $(shell sh -c 'uname -s 2>/dev/null || echo not')\nifeq ($(OS),Windows_NT)\n\tWINDOWS :=1\n\t?exe :=.exe # windows compat\n\tPATH_SEP :=;\nelse\n\t?exe :=\n\tPATH_SEP :=:\nendif\n\n# Platforms are now defined in .github/build-platforms.yml\n# The cmd/ipfs-try-build target is deprecated in favor of GitHub Actions\n# Use 'make supported' to see the list of platforms\n\nspace:=$() $()\ncomma:=,\njoin-with=$(subst $(space),$1,$(strip $2))\n\n# debug target, prints variable. Example: `make print-GOFLAGS`\nprint-%:\n\t@echo $*=$($*)\n\n# phony target that will mean that recipe is always executed\nALWAYS:\n.PHONY: ALWAYS\n" }, { "path": "p2p/listener.go", "content": "package p2p\n\nimport (\n\t\"errors\"\n\t\"sync\"\n\n\tp2phost \"github.com/libp2p/go-libp2p/core/host\"\n\tnet \"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\n// Listener listens for connections and proxies them to a target.\ntype Listener interface {\n\tProtocol() protocol.ID\n\tListenAddress() ma.Multiaddr\n\tTargetAddress() ma.Multiaddr\n\n\tkey() protocol.ID\n\n\t// close closes the listener. Does not affect child streams\n\tclose()\n\n\t// Done returns a channel that is closed when the listener is closed.\n\t// This allows callers to detect when a listener has been removed.\n\tDone() <-chan struct{}\n}\n\n// Listeners manages a group of Listener implementations,\n// checking for conflicts and optionally dispatching connections.\ntype Listeners struct {\n\tsync.RWMutex\n\n\tListeners map[protocol.ID]Listener\n}\n\nfunc newListenersLocal() *Listeners {\n\treturn &Listeners{\n\t\tListeners: map[protocol.ID]Listener{},\n\t}\n}\n\nfunc newListenersP2P(host p2phost.Host) *Listeners {\n\treg := &Listeners{\n\t\tListeners: map[protocol.ID]Listener{},\n\t}\n\n\thost.SetStreamHandlerMatch(\"/x/\", func(p protocol.ID) bool {\n\t\treg.RLock()\n\t\tdefer reg.RUnlock()\n\n\t\t_, ok := reg.Listeners[p]\n\t\treturn ok\n\t}, func(stream net.Stream) {\n\t\treg.RLock()\n\t\tdefer reg.RUnlock()\n\n\t\tl := reg.Listeners[stream.Protocol()]\n\t\tif l != nil {\n\t\t\tgo l.(*remoteListener).handleStream(stream)\n\t\t}\n\t})\n\n\treturn reg\n}\n\n// Register registers listenerInfo into this registry and starts it.\nfunc (r *Listeners) Register(l Listener) error {\n\tr.Lock()\n\tdefer r.Unlock()\n\n\tif _, ok := r.Listeners[l.key()]; ok {\n\t\treturn errors.New(\"listener already registered\")\n\t}\n\n\tr.Listeners[l.key()] = l\n\treturn nil\n}\n\n// Close removes and closes all listeners for which matchFunc returns true.\n// Returns the number of listeners closed.\nfunc (r *Listeners) Close(matchFunc func(listener Listener) bool) int {\n\tvar todo []Listener\n\tr.Lock()\n\tfor _, l := range r.Listeners {\n\t\tif matchFunc(l) {\n\t\t\tdelete(r.Listeners, l.key())\n\t\t\ttodo = append(todo, l)\n\t\t}\n\t}\n\tr.Unlock()\n\n\tfor _, l := range todo {\n\t\tl.close()\n\t}\n\n\treturn len(todo)\n}\n" }, { "path": "p2p/local.go", "content": "package p2p\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\ttec \"github.com/jbenet/go-temp-err-catcher\"\n\tnet \"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\n// localListener manet streams and proxies them to libp2p services.\ntype localListener struct {\n\tctx context.Context\n\n\tp2p *P2P\n\n\tproto protocol.ID\n\tladdr ma.Multiaddr\n\tpeer peer.ID\n\n\tlistener manet.Listener\n\tdone chan struct{}\n}\n\n// ForwardLocal creates new P2P stream to a remote listener.\nfunc (p2p *P2P) ForwardLocal(ctx context.Context, peer peer.ID, proto protocol.ID, bindAddr ma.Multiaddr) (Listener, error) {\n\tlistener := &localListener{\n\t\tctx: ctx,\n\t\tp2p: p2p,\n\t\tproto: proto,\n\t\tpeer: peer,\n\t\tdone: make(chan struct{}),\n\t}\n\n\tmaListener, err := manet.Listen(bindAddr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlistener.listener = maListener\n\tlistener.laddr = maListener.Multiaddr()\n\n\tif err := p2p.ListenersLocal.Register(listener); err != nil {\n\t\treturn nil, err\n\t}\n\n\tgo listener.acceptConns()\n\n\treturn listener, nil\n}\n\nfunc (l *localListener) dial(ctx context.Context) (net.Stream, error) {\n\tcctx, cancel := context.WithTimeout(ctx, time.Second*30) // TODO: configurable?\n\tdefer cancel()\n\n\treturn l.p2p.peerHost.NewStream(cctx, l.peer, l.proto)\n}\n\nfunc (l *localListener) acceptConns() {\n\tfor {\n\t\tlocal, err := l.listener.Accept()\n\t\tif err != nil {\n\t\t\tif tec.ErrIsTemporary(err) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn\n\t\t}\n\n\t\tgo l.setupStream(local)\n\t}\n}\n\nfunc (l *localListener) setupStream(local manet.Conn) {\n\tremote, err := l.dial(l.ctx)\n\tif err != nil {\n\t\tlocal.Close()\n\t\tlog.Warnf(\"failed to dial to remote %s/%s\", l.peer, l.proto)\n\t\treturn\n\t}\n\n\tstream := &Stream{\n\t\tProtocol: l.proto,\n\n\t\tOriginAddr: local.RemoteMultiaddr(),\n\t\tTargetAddr: l.TargetAddress(),\n\t\tpeer: l.peer,\n\n\t\tLocal: local,\n\t\tRemote: remote,\n\n\t\tRegistry: l.p2p.Streams,\n\t}\n\n\tl.p2p.Streams.Register(stream)\n}\n\nfunc (l *localListener) close() {\n\tl.listener.Close()\n\tclose(l.done)\n}\n\nfunc (l *localListener) Done() <-chan struct{} {\n\treturn l.done\n}\n\nfunc (l *localListener) Protocol() protocol.ID {\n\treturn l.proto\n}\n\nfunc (l *localListener) ListenAddress() ma.Multiaddr {\n\treturn l.laddr\n}\n\nfunc (l *localListener) TargetAddress() ma.Multiaddr {\n\taddr, err := ma.NewMultiaddr(maPrefix + l.peer.String())\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn addr\n}\n\nfunc (l *localListener) key() protocol.ID {\n\treturn protocol.ID(l.ListenAddress().String())\n}\n" }, { "path": "p2p/p2p.go", "content": "package p2p\n\nimport (\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tp2phost \"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tpstore \"github.com/libp2p/go-libp2p/core/peerstore\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n)\n\nvar log = logging.Logger(\"p2p-mount\")\n\n// P2P structure holds information on currently running streams/Listeners.\ntype P2P struct {\n\tListenersLocal *Listeners\n\tListenersP2P *Listeners\n\tStreams *StreamRegistry\n\n\tidentity peer.ID\n\tpeerHost p2phost.Host\n\tpeerstore pstore.Peerstore\n}\n\n// New creates new P2P struct.\nfunc New(identity peer.ID, peerHost p2phost.Host, peerstore pstore.Peerstore) *P2P {\n\treturn &P2P{\n\t\tidentity: identity,\n\t\tpeerHost: peerHost,\n\t\tpeerstore: peerstore,\n\n\t\tListenersLocal: newListenersLocal(),\n\t\tListenersP2P: newListenersP2P(peerHost),\n\n\t\tStreams: &StreamRegistry{\n\t\t\tStreams: map[uint64]*Stream{},\n\t\t\tConnManager: peerHost.ConnManager(),\n\t\t\tconns: map[peer.ID]int{},\n\t\t},\n\t}\n}\n\n// CheckProtoExists checks whether a proto handler is registered to\n// mux handler.\nfunc (p2p *P2P) CheckProtoExists(proto protocol.ID) bool {\n\tprotos := p2p.peerHost.Mux().Protocols()\n\n\tfor _, p := range protos {\n\t\tif p != proto {\n\t\t\tcontinue\n\t\t}\n\t\treturn true\n\t}\n\treturn false\n}\n" }, { "path": "p2p/remote.go", "content": "package p2p\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\tnet \"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nvar maPrefix = \"/\" + ma.ProtocolWithCode(ma.P_IPFS).Name + \"/\"\n\n// remoteListener accepts libp2p streams and proxies them to a manet host.\ntype remoteListener struct {\n\tp2p *P2P\n\n\t// Application proto identifier.\n\tproto protocol.ID\n\n\t// Address to proxy the incoming connections to\n\taddr ma.Multiaddr\n\n\t// reportRemote if set to true makes the handler send '\\n'\n\t// to target before any data is forwarded\n\treportRemote bool\n\n\tdone chan struct{}\n}\n\n// ForwardRemote creates new p2p listener.\nfunc (p2p *P2P) ForwardRemote(ctx context.Context, proto protocol.ID, addr ma.Multiaddr, reportRemote bool) (Listener, error) {\n\tlistener := &remoteListener{\n\t\tp2p: p2p,\n\n\t\tproto: proto,\n\t\taddr: addr,\n\n\t\treportRemote: reportRemote,\n\t\tdone: make(chan struct{}),\n\t}\n\n\tif err := p2p.ListenersP2P.Register(listener); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn listener, nil\n}\n\nfunc (l *remoteListener) handleStream(remote net.Stream) {\n\tlocal, err := manet.Dial(l.addr)\n\tif err != nil {\n\t\t_ = remote.Reset()\n\t\treturn\n\t}\n\n\tpeer := remote.Conn().RemotePeer()\n\n\tif l.reportRemote {\n\t\tif _, err := fmt.Fprintf(local, \"%s\\n\", peer); err != nil {\n\t\t\t_ = remote.Reset()\n\t\t\treturn\n\t\t}\n\t}\n\n\tpeerMa, err := ma.NewMultiaddr(maPrefix + peer.String())\n\tif err != nil {\n\t\t_ = remote.Reset()\n\t\treturn\n\t}\n\n\tstream := &Stream{\n\t\tProtocol: l.proto,\n\n\t\tOriginAddr: peerMa,\n\t\tTargetAddr: l.addr,\n\t\tpeer: peer,\n\n\t\tLocal: local,\n\t\tRemote: remote,\n\n\t\tRegistry: l.p2p.Streams,\n\t}\n\n\tl.p2p.Streams.Register(stream)\n}\n\nfunc (l *remoteListener) Protocol() protocol.ID {\n\treturn l.proto\n}\n\nfunc (l *remoteListener) ListenAddress() ma.Multiaddr {\n\taddr, err := ma.NewMultiaddr(maPrefix + l.p2p.identity.String())\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn addr\n}\n\nfunc (l *remoteListener) TargetAddress() ma.Multiaddr {\n\treturn l.addr\n}\n\nfunc (l *remoteListener) close() {\n\tclose(l.done)\n}\n\nfunc (l *remoteListener) Done() <-chan struct{} {\n\treturn l.done\n}\n\nfunc (l *remoteListener) key() protocol.ID {\n\treturn l.proto\n}\n" }, { "path": "p2p/stream.go", "content": "package p2p\n\nimport (\n\t\"io\"\n\t\"sync\"\n\n\tifconnmgr \"github.com/libp2p/go-libp2p/core/connmgr\"\n\tnet \"github.com/libp2p/go-libp2p/core/network\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tprotocol \"github.com/libp2p/go-libp2p/core/protocol\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nconst cmgrTag = \"stream-fwd\"\n\n// Stream holds information on active incoming and outgoing p2p streams.\ntype Stream struct {\n\tid uint64\n\n\tProtocol protocol.ID\n\n\tOriginAddr ma.Multiaddr\n\tTargetAddr ma.Multiaddr\n\tpeer peer.ID\n\n\tLocal manet.Conn\n\tRemote net.Stream\n\n\tRegistry *StreamRegistry\n}\n\n// close stream endpoints and deregister it.\nfunc (s *Stream) close() {\n\ts.Registry.Close(s)\n}\n\n// reset closes stream endpoints and deregisters it.\nfunc (s *Stream) reset() {\n\ts.Registry.Reset(s)\n}\n\nfunc (s *Stream) startStreaming() {\n\tgo func() {\n\t\t_, err := io.Copy(s.Local, s.Remote)\n\t\tif err != nil {\n\t\t\ts.reset()\n\t\t} else {\n\t\t\ts.close()\n\t\t}\n\t}()\n\n\tgo func() {\n\t\t_, err := io.Copy(s.Remote, s.Local)\n\t\tif err != nil {\n\t\t\ts.reset()\n\t\t} else {\n\t\t\ts.close()\n\t\t}\n\t}()\n}\n\n// StreamRegistry is a collection of active incoming and outgoing proto app streams.\ntype StreamRegistry struct {\n\tsync.Mutex\n\n\tStreams map[uint64]*Stream\n\tconns map[peer.ID]int\n\tnextID uint64\n\n\tifconnmgr.ConnManager\n}\n\n// Register registers a stream to the registry.\nfunc (r *StreamRegistry) Register(streamInfo *Stream) {\n\tr.Lock()\n\tdefer r.Unlock()\n\n\tr.ConnManager.TagPeer(streamInfo.peer, cmgrTag, 20)\n\tr.conns[streamInfo.peer]++\n\n\tstreamInfo.id = r.nextID\n\tr.Streams[r.nextID] = streamInfo\n\tr.nextID++\n\n\tstreamInfo.startStreaming()\n}\n\n// Deregister deregisters stream from the registry.\nfunc (r *StreamRegistry) Deregister(streamID uint64) {\n\tr.Lock()\n\tdefer r.Unlock()\n\n\ts, ok := r.Streams[streamID]\n\tif !ok {\n\t\treturn\n\t}\n\tp := s.peer\n\tr.conns[p]--\n\tif r.conns[p] < 1 {\n\t\tdelete(r.conns, p)\n\t\tr.ConnManager.UntagPeer(p, cmgrTag)\n\t}\n\n\tdelete(r.Streams, streamID)\n}\n\n// Close stream endpoints and deregister it.\nfunc (r *StreamRegistry) Close(s *Stream) {\n\t_ = s.Local.Close()\n\t_ = s.Remote.Close()\n\ts.Registry.Deregister(s.id)\n}\n\n// Reset closes stream endpoints and deregisters it.\nfunc (r *StreamRegistry) Reset(s *Stream) {\n\t_ = s.Local.Close()\n\t_ = s.Remote.Reset()\n\ts.Registry.Deregister(s.id)\n}\n" }, { "path": "plugin/Rules.mk", "content": "include mk/header.mk\n\ndir := $(d)/loader\ninclude $(dir)/Rules.mk\n\ndir := $(d)/plugins\ninclude $(dir)/Rules.mk\n\ninclude mk/footer.mk\n" }, { "path": "plugin/daemon.go", "content": "package plugin\n\nimport (\n\tcoreiface \"github.com/ipfs/kubo/core/coreiface\"\n)\n\n// PluginDaemon is an interface for daemon plugins. These plugins will be run on\n// the daemon and will be given access to an implementation of the CoreAPI.\ntype PluginDaemon interface {\n\tPlugin\n\n\tStart(coreiface.CoreAPI) error\n}\n" }, { "path": "plugin/daemoninternal.go", "content": "package plugin\n\nimport \"github.com/ipfs/kubo/core\"\n\n// PluginDaemonInternal is an interface for daemon plugins. These plugins will be run on\n// the daemon and will be given a direct access to the IpfsNode.\n//\n// Note: PluginDaemonInternal is considered internal and no guarantee is made concerning\n// the stability of its API. If you can, use PluginAPI instead.\ntype PluginDaemonInternal interface {\n\tPlugin\n\n\tStart(*core.IpfsNode) error\n}\n" }, { "path": "plugin/datastore.go", "content": "package plugin\n\nimport (\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n)\n\n// PluginDatastore is an interface that can be implemented to add handlers for\n// for different datastores.\ntype PluginDatastore interface {\n\tPlugin\n\n\tDatastoreTypeName() string\n\tDatastoreConfigParser() fsrepo.ConfigFromMap\n}\n" }, { "path": "plugin/fx.go", "content": "package plugin\n\nimport (\n\t\"github.com/ipfs/kubo/core\"\n\t\"go.uber.org/fx\"\n)\n\n// PluginFx can be used to customize the fx options passed to the go-ipfs app when it is initialized.\n//\n// This is invasive and depends on internal details such as the structure of the dependency graph,\n// so breaking changes might occur between releases.\n// So it's recommended to keep this as simple as possible, and stick to overriding interfaces\n// with fx.Replace() or fx.Decorate().\n//\n// The returned options become the complete array of options passed to fx.\n// Generally you'll want to append additional options to NodeInfo.FXOptions and return that.\ntype PluginFx interface {\n\tPlugin\n\tOptions(core.FXNodeInfo) ([]fx.Option, error)\n}\n" }, { "path": "plugin/ipld.go", "content": "package plugin\n\nimport (\n\tmulticodec \"github.com/ipld/go-ipld-prime/multicodec\"\n)\n\n// PluginIPLD is an interface that can be implemented to add handlers for\n// for different IPLD codecs.\ntype PluginIPLD interface {\n\tPlugin\n\n\tRegister(multicodec.Registry) error\n}\n" }, { "path": "plugin/loader/Rules.mk", "content": "include mk/header.mk\n\nIPFS_PLUGINS ?= \nexport IPFS_PLUGINS\n\n$(d)/preload.go: d:=$(d)\n$(d)/preload.go: $(d)/preload_list $(d)/preload.sh ALWAYS\n\t$(d)/preload.sh > $@\n\tgo fmt $@ >/dev/null\n\nDEPS_GO += $(d)/preload.go\n\ninclude mk/footer.mk\n" }, { "path": "plugin/loader/load_nocgo.go", "content": "//go:build !cgo && !noplugin && (linux || darwin || freebsd)\n\npackage loader\n\nimport (\n\t\"errors\"\n\n\tiplugin \"github.com/ipfs/kubo/plugin\"\n)\n\nfunc init() {\n\tloadPluginFunc = nocgoLoadPlugin\n}\n\nfunc nocgoLoadPlugin(fi string) ([]iplugin.Plugin, error) {\n\treturn nil, errors.New(\"not built with cgo support\")\n}\n" }, { "path": "plugin/loader/load_noplugin.go", "content": "//go:build noplugin\n\npackage loader\n\nimport (\n\t\"errors\"\n\n\tiplugin \"github.com/ipfs/kubo/plugin\"\n)\n\nfunc init() {\n\tloadPluginFunc = nopluginLoadPlugin\n}\n\nfunc nopluginLoadPlugin(string) ([]iplugin.Plugin, error) {\n\treturn nil, errors.New(\"not built with plugin support\")\n}\n" }, { "path": "plugin/loader/load_unix.go", "content": "//go:build cgo && !noplugin && (linux || darwin || freebsd)\n\npackage loader\n\nimport (\n\t\"errors\"\n\t\"plugin\"\n\n\tiplugin \"github.com/ipfs/kubo/plugin\"\n)\n\nfunc init() {\n\tloadPluginFunc = unixLoadPlugin\n}\n\nfunc unixLoadPlugin(fi string) ([]iplugin.Plugin, error) {\n\tpl, err := plugin.Open(fi)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tpls, err := pl.Lookup(\"Plugins\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\ttypePls, ok := pls.(*[]iplugin.Plugin)\n\tif !ok {\n\t\treturn nil, errors.New(\"filed 'Plugins' didn't contain correct type\")\n\t}\n\n\treturn *typePls, nil\n}\n" }, { "path": "plugin/loader/loader.go", "content": "package loader\n\nimport (\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipld/go-ipld-prime/multicodec\"\n\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\tplugin \"github.com/ipfs/kubo/plugin\"\n\tfsrepo \"github.com/ipfs/kubo/repo/fsrepo\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\topentracing \"github.com/opentracing/opentracing-go\"\n)\n\nvar preloadPlugins []plugin.Plugin\n\n// Preload adds one or more plugins to the preload list. This should _only_ be called during init.\nfunc Preload(plugins ...plugin.Plugin) {\n\tpreloadPlugins = append(preloadPlugins, plugins...)\n}\n\nvar log = logging.Logger(\"plugin/loader\")\n\nvar loadPluginFunc = func(string) ([]plugin.Plugin, error) {\n\treturn nil, fmt.Errorf(\"unsupported platform %s\", runtime.GOOS)\n}\n\ntype loaderState int\n\nconst (\n\tloaderLoading loaderState = iota\n\tloaderInitializing\n\tloaderInitialized\n\tloaderInjecting\n\tloaderInjected\n\tloaderStarting\n\tloaderStarted\n\tloaderClosing\n\tloaderClosed\n\tloaderFailed\n)\n\nfunc (ls loaderState) String() string {\n\tswitch ls {\n\tcase loaderLoading:\n\t\treturn \"Loading\"\n\tcase loaderInitializing:\n\t\treturn \"Initializing\"\n\tcase loaderInitialized:\n\t\treturn \"Initialized\"\n\tcase loaderInjecting:\n\t\treturn \"Injecting\"\n\tcase loaderInjected:\n\t\treturn \"Injected\"\n\tcase loaderStarting:\n\t\treturn \"Starting\"\n\tcase loaderStarted:\n\t\treturn \"Started\"\n\tcase loaderClosing:\n\t\treturn \"Closing\"\n\tcase loaderClosed:\n\t\treturn \"Closed\"\n\tcase loaderFailed:\n\t\treturn \"Failed\"\n\tdefault:\n\t\treturn \"Unknown\"\n\t}\n}\n\n// PluginLoader keeps track of loaded plugins.\n//\n// To use:\n// 1. Load any desired plugins with Load and LoadDirectory. Preloaded plugins\n// will automatically be loaded.\n// 2. Call Initialize to run all initialization logic.\n// 3. Call Inject to register the plugins.\n// 4. Optionally call Start to start plugins.\n// 5. Call Close to close all plugins.\ntype PluginLoader struct {\n\tstate loaderState\n\tplugins []plugin.Plugin\n\tstarted []plugin.Plugin\n\tconfig config.Plugins\n\trepo string\n}\n\n// NewPluginLoader creates new plugin loader.\nfunc NewPluginLoader(repo string) (*PluginLoader, error) {\n\tloader := &PluginLoader{plugins: make([]plugin.Plugin, 0, len(preloadPlugins)), repo: repo}\n\tif repo != \"\" {\n\t\tswitch plugins, err := readPluginsConfig(repo, config.DefaultConfigFile); {\n\t\tcase err == nil:\n\t\t\tloader.config = plugins\n\t\tcase os.IsNotExist(err):\n\t\tdefault:\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tfor _, v := range preloadPlugins {\n\t\tif err := loader.Load(v); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif err := loader.LoadDirectory(filepath.Join(repo, \"plugins\")); err != nil {\n\t\treturn nil, err\n\t}\n\treturn loader, nil\n}\n\n// readPluginsConfig reads the Plugins section of the IPFS config, avoiding\n// reading anything other than the Plugin section. That way, we're free to\n// make arbitrary changes to all _other_ sections in migrations.\nfunc readPluginsConfig(repoRoot string, userConfigFile string) (config.Plugins, error) {\n\tvar cfg struct {\n\t\tPlugins config.Plugins\n\t}\n\n\tcfgPath, err := config.Filename(repoRoot, userConfigFile)\n\tif err != nil {\n\t\treturn config.Plugins{}, err\n\t}\n\n\tcfgFile, err := os.Open(cfgPath)\n\tif err != nil {\n\t\treturn config.Plugins{}, err\n\t}\n\tdefer cfgFile.Close()\n\n\terr = json.NewDecoder(cfgFile).Decode(&cfg)\n\tif err != nil {\n\t\treturn config.Plugins{}, err\n\t}\n\n\treturn cfg.Plugins, nil\n}\n\nfunc (loader *PluginLoader) assertState(state loaderState) error {\n\tif loader.state != state {\n\t\treturn fmt.Errorf(\"loader state must be %s, was %s\", state, loader.state)\n\t}\n\treturn nil\n}\n\nfunc (loader *PluginLoader) transition(from, to loaderState) error {\n\tif err := loader.assertState(from); err != nil {\n\t\treturn err\n\t}\n\tloader.state = to\n\treturn nil\n}\n\n// Load loads a plugin into the plugin loader.\nfunc (loader *PluginLoader) Load(pl plugin.Plugin) error {\n\tif err := loader.assertState(loaderLoading); err != nil {\n\t\treturn err\n\t}\n\n\tname := pl.Name()\n\n\tfor _, p := range loader.plugins {\n\t\tif p.Name() == name {\n\t\t\t// plugin is already loaded\n\t\t\treturn fmt.Errorf(\n\t\t\t\t\"plugin: %s, is duplicated in version: %s, \"+\n\t\t\t\t\t\"while trying to load dynamically: %s\",\n\t\t\t\tname, p.Version(), pl.Version())\n\t\t}\n\t}\n\n\tif loader.config.Plugins[name].Disabled {\n\t\tlog.Infof(\"not loading disabled plugin %s\", name)\n\t\treturn nil\n\t}\n\tloader.plugins = append(loader.plugins, pl)\n\treturn nil\n}\n\n// LoadDirectory loads a directory of plugins into the plugin loader.\nfunc (loader *PluginLoader) LoadDirectory(pluginDir string) error {\n\tif err := loader.assertState(loaderLoading); err != nil {\n\t\treturn err\n\t}\n\tnewPls, err := loadDynamicPlugins(pluginDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, pl := range newPls {\n\t\tif err := loader.Load(pl); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc loadDynamicPlugins(pluginDir string) ([]plugin.Plugin, error) {\n\t_, err := os.Stat(pluginDir)\n\tif os.IsNotExist(err) {\n\t\treturn nil, nil\n\t}\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar plugins []plugin.Plugin\n\n\terr = filepath.Walk(pluginDir, func(fi string, info os.FileInfo, err error) error {\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif info.IsDir() {\n\t\t\tif fi != pluginDir {\n\t\t\t\tlog.Warnf(\"found directory inside plugins directory: %s\", fi)\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tif info.Mode().Perm()&0o111 == 0 {\n\t\t\t// file is not executable let's not load it\n\t\t\t// this is to prevent loading plugins from for example non-executable\n\t\t\t// mounts, some /tmp mounts are marked as such for security\n\t\t\tlog.Errorf(\"non-executable file in plugins directory: %s\", fi)\n\t\t\treturn nil\n\t\t}\n\n\t\tif newPlugins, err := loadPluginFunc(fi); err == nil {\n\t\t\tplugins = append(plugins, newPlugins...)\n\t\t} else {\n\t\t\treturn fmt.Errorf(\"loading plugin %s: %s\", fi, err)\n\t\t}\n\t\treturn nil\n\t})\n\n\treturn plugins, err\n}\n\n// Initialize initializes all loaded plugins.\nfunc (loader *PluginLoader) Initialize() error {\n\tif err := loader.transition(loaderLoading, loaderInitializing); err != nil {\n\t\treturn err\n\t}\n\tfor _, p := range loader.plugins {\n\t\terr := p.Init(&plugin.Environment{\n\t\t\tRepo: loader.repo,\n\t\t\tConfig: loader.config.Plugins[p.Name()].Config,\n\t\t})\n\t\tif err != nil {\n\t\t\tloader.state = loaderFailed\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn loader.transition(loaderInitializing, loaderInitialized)\n}\n\n// Inject hooks all the plugins into the appropriate subsystems.\nfunc (loader *PluginLoader) Inject() error {\n\tif err := loader.transition(loaderInitialized, loaderInjecting); err != nil {\n\t\treturn err\n\t}\n\n\tfor _, pl := range loader.plugins {\n\t\tif pl, ok := pl.(plugin.PluginIPLD); ok {\n\t\t\terr := injectIPLDPlugin(pl)\n\t\t\tif err != nil {\n\t\t\t\tloader.state = loaderFailed\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tif pl, ok := pl.(plugin.PluginTracer); ok {\n\t\t\terr := injectTracerPlugin(pl)\n\t\t\tif err != nil {\n\t\t\t\tloader.state = loaderFailed\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tif pl, ok := pl.(plugin.PluginDatastore); ok {\n\t\t\terr := injectDatastorePlugin(pl)\n\t\t\tif err != nil {\n\t\t\t\tloader.state = loaderFailed\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tif pl, ok := pl.(plugin.PluginFx); ok {\n\t\t\terr := injectFxPlugin(pl)\n\t\t\tif err != nil {\n\t\t\t\tloader.state = loaderFailed\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn loader.transition(loaderInjecting, loaderInjected)\n}\n\n// Start starts all long-running plugins.\nfunc (loader *PluginLoader) Start(node *core.IpfsNode) error {\n\tif err := loader.transition(loaderInjected, loaderStarting); err != nil {\n\t\treturn err\n\t}\n\tiface, err := coreapi.NewCoreAPI(node)\n\tif err != nil {\n\t\treturn err\n\t}\n\tfor _, pl := range loader.plugins {\n\t\tif pl, ok := pl.(plugin.PluginDaemon); ok {\n\t\t\terr := pl.Start(iface)\n\t\t\tif err != nil {\n\t\t\t\t_ = loader.Close()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tloader.started = append(loader.started, pl)\n\t\t}\n\t\tif pl, ok := pl.(plugin.PluginDaemonInternal); ok {\n\t\t\terr := pl.Start(node)\n\t\t\tif err != nil {\n\t\t\t\t_ = loader.Close()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tloader.started = append(loader.started, pl)\n\t\t}\n\t}\n\n\treturn loader.transition(loaderStarting, loaderStarted)\n}\n\n// Close stops all long-running plugins.\nfunc (loader *PluginLoader) Close() error {\n\tswitch loader.state {\n\tcase loaderClosing, loaderFailed, loaderClosed:\n\t\t// nothing to do.\n\t\treturn nil\n\t}\n\tloader.state = loaderClosing\n\n\tvar errs []string\n\tstarted := loader.started\n\tloader.started = nil\n\tfor _, pl := range started {\n\t\tif closer, ok := pl.(io.Closer); ok {\n\t\t\terr := closer.Close()\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, fmt.Sprintf(\n\t\t\t\t\t\"error closing plugin %s: %s\",\n\t\t\t\t\tpl.Name(),\n\t\t\t\t\terr.Error(),\n\t\t\t\t))\n\t\t\t}\n\t\t}\n\t}\n\tif errs != nil {\n\t\tloader.state = loaderFailed\n\t\treturn errors.New(strings.Join(errs, \"\\n\"))\n\t}\n\tloader.state = loaderClosed\n\treturn nil\n}\n\nfunc injectDatastorePlugin(pl plugin.PluginDatastore) error {\n\treturn fsrepo.AddDatastoreConfigHandler(pl.DatastoreTypeName(), pl.DatastoreConfigParser())\n}\n\nfunc injectIPLDPlugin(pl plugin.PluginIPLD) error {\n\treturn pl.Register(multicodec.DefaultRegistry)\n}\n\nfunc injectTracerPlugin(pl plugin.PluginTracer) error {\n\tlog.Warn(\"Tracer plugins are deprecated, it's recommended to configure an OpenTelemetry collector instead.\")\n\ttracer, err := pl.InitTracer()\n\tif err != nil {\n\t\treturn err\n\t}\n\topentracing.SetGlobalTracer(tracer)\n\treturn nil\n}\n\nfunc injectFxPlugin(pl plugin.PluginFx) error {\n\tcore.RegisterFXOptionFunc(pl.Options)\n\treturn nil\n}\n" }, { "path": "plugin/loader/preload.go", "content": "package loader\n\nimport (\n\tpluginbadgerds \"github.com/ipfs/kubo/plugin/plugins/badgerds\"\n\tpluginiplddagjose \"github.com/ipfs/kubo/plugin/plugins/dagjose\"\n\tpluginflatfs \"github.com/ipfs/kubo/plugin/plugins/flatfs\"\n\tpluginfxtest \"github.com/ipfs/kubo/plugin/plugins/fxtest\"\n\tpluginipldgit \"github.com/ipfs/kubo/plugin/plugins/git\"\n\tpluginlevelds \"github.com/ipfs/kubo/plugin/plugins/levelds\"\n\tpluginnopfs \"github.com/ipfs/kubo/plugin/plugins/nopfs\"\n\tpluginpebbleds \"github.com/ipfs/kubo/plugin/plugins/pebbleds\"\n\tpluginpeerlog \"github.com/ipfs/kubo/plugin/plugins/peerlog\"\n\tplugintelemetry \"github.com/ipfs/kubo/plugin/plugins/telemetry\"\n)\n\n// DO NOT EDIT THIS FILE\n// This file is being generated as part of plugin build process\n// To change it, modify the plugin/loader/preload.sh\n\nfunc init() {\n\tPreload(pluginipldgit.Plugins...)\n\tPreload(pluginiplddagjose.Plugins...)\n\tPreload(pluginbadgerds.Plugins...)\n\tPreload(pluginflatfs.Plugins...)\n\tPreload(pluginlevelds.Plugins...)\n\tPreload(pluginpebbleds.Plugins...)\n\tPreload(pluginpeerlog.Plugins...)\n\tPreload(pluginfxtest.Plugins...)\n\tPreload(pluginnopfs.Plugins...)\n\tPreload(plugintelemetry.Plugins...)\n}\n" }, { "path": "plugin/loader/preload.sh", "content": "#!/usr/bin/env bash\n\nDIR=\"$(cd \"$(dirname \"${BASH_SOURCE[0]}\")\" && pwd)\"\n\nLIST=\"$DIR/preload_list\"\n\nto_preload() {\n awk 'NF' $LIST | sed '/^#/d'\n if [[ -n \"$IPFS_PLUGINS\" ]]; then\n for plugin in $IPFS_PLUGINS; do\n echo \"$plugin github.com/ipfs/kubo/plugin/plugins/$plugin *\"\n done\n fi\n}\n\ncat </dev/null\n\n$($(d)_plugins_so): %.so : %/main/main.go\n$($(d)_plugins_so): $$(DEPS_GO) ALWAYS\n\t$(GOCC) build -buildmode=plugin -pkgdir \"$(GOPATH)/pkg/linux_amd64_dynlink\" $(go-flags-with-tags) -o \"$@\" \"$(call go-pkg-name,$(basename $@))/main\"\n\tchmod +x \"$@\"\n\nCLEAN += $($(d)_plugins_so)\nCLEAN += $(foreach main_dir,$($(d)_plugins_main),$(dir $(main_dir)))\n\nbuild_plugins: $($(d)_plugins_so)\n\n\ninclude mk/footer.mk\n" }, { "path": "plugin/plugins/badgerds/badgerds.go", "content": "package badgerds\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/plugin\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\n\thumanize \"github.com/dustin/go-humanize\"\n\tbadgerds \"github.com/ipfs/go-ds-badger\"\n)\n\nvar log = logging.Logger(\"plugin/badgerds\")\n\n// Plugins is exported list of plugins that will be loaded.\nvar Plugins = []plugin.Plugin{\n\t&badgerdsPlugin{},\n}\n\ntype badgerdsPlugin struct{}\n\nvar _ plugin.PluginDatastore = (*badgerdsPlugin)(nil)\n\nfunc (*badgerdsPlugin) Name() string {\n\treturn \"ds-badgerds\"\n}\n\nfunc (*badgerdsPlugin) Version() string {\n\treturn \"0.1.0\"\n}\n\nfunc (*badgerdsPlugin) Init(_ *plugin.Environment) error {\n\treturn nil\n}\n\nfunc (*badgerdsPlugin) DatastoreTypeName() string {\n\treturn \"badgerds\"\n}\n\ntype datastoreConfig struct {\n\tpath string\n\tsyncWrites bool\n\ttruncate bool\n\n\tvlogFileSize int64\n}\n\n// BadgerdsDatastoreConfig returns a configuration stub for a badger datastore\n// from the given parameters.\nfunc (*badgerdsPlugin) DatastoreConfigParser() fsrepo.ConfigFromMap {\n\treturn func(params map[string]any) (fsrepo.DatastoreConfig, error) {\n\t\tvar c datastoreConfig\n\t\tvar ok bool\n\n\t\tc.path, ok = params[\"path\"].(string)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"'path' field is missing or not string\")\n\t\t}\n\n\t\tsw, ok := params[\"syncWrites\"]\n\t\tif !ok {\n\t\t\tc.syncWrites = false\n\t\t} else {\n\t\t\tif swb, ok := sw.(bool); ok {\n\t\t\t\tc.syncWrites = swb\n\t\t\t} else {\n\t\t\t\treturn nil, fmt.Errorf(\"'syncWrites' field was not a boolean\")\n\t\t\t}\n\t\t}\n\n\t\ttruncate, ok := params[\"truncate\"]\n\t\tif !ok {\n\t\t\tc.truncate = true\n\t\t} else {\n\t\t\tif truncate, ok := truncate.(bool); ok {\n\t\t\t\tc.truncate = truncate\n\t\t\t} else {\n\t\t\t\treturn nil, fmt.Errorf(\"'truncate' field was not a boolean\")\n\t\t\t}\n\t\t}\n\n\t\tvls, ok := params[\"vlogFileSize\"]\n\t\tif !ok {\n\t\t\t// default to 1GiB\n\t\t\tc.vlogFileSize = badgerds.DefaultOptions.ValueLogFileSize\n\t\t} else {\n\t\t\tif vlogSize, ok := vls.(string); ok {\n\t\t\t\ts, err := humanize.ParseBytes(vlogSize)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, err\n\t\t\t\t}\n\t\t\t\tc.vlogFileSize = int64(s)\n\t\t\t} else {\n\t\t\t\treturn nil, fmt.Errorf(\"'vlogFileSize' field was not a string\")\n\t\t\t}\n\t\t}\n\n\t\treturn &c, nil\n\t}\n}\n\nfunc (c *datastoreConfig) DiskSpec() fsrepo.DiskSpec {\n\treturn map[string]any{\n\t\t\"type\": \"badgerds\",\n\t\t\"path\": c.path,\n\t}\n}\n\nfunc (c *datastoreConfig) Create(path string) (repo.Datastore, error) {\n\tlog.Error(\"badger v1 datastore is deprecated and will be removed later in 2026, migrate to flatfs or experimental pebbleds: https://github.com/ipfs/kubo/issues/11186\")\n\tfmt.Fprintf(os.Stderr, `\n╔════════════════════════════════════════════════════════════════════════════╗\n║ ║\n║ ERROR: BADGER v1 DATASTORE IS DEPRECATED ║\n║ ║\n║ This datastore is based on badger 1.x which has not been maintained ║\n║ by its upstream maintainers for years and has known bugs (startup ║\n║ timeouts, shutdown hangs, file descriptor exhaustion, and more). ║\n║ ║\n║ Badger v1 support will be REMOVED later in 2026. ║\n║ ║\n║ To migrate: ║\n║ 1. Create a new IPFS_PATH with flatfs (or experimental pebbleds ║\n║ if flatfs does not serve your use case): ║\n║ export IPFS_PATH=/path/to/new/repo ║\n║ ipfs init --profile=flatfs ║\n║ 2. Move pinned data via ipfs dag export/import ║\n║ or ipfs pin ls -t recursive|add ║\n║ 3. Decommission the old badger-based node ║\n║ ║\n║ See https://github.com/ipfs/kubo/blob/master/docs/datastores.md ║\n║ https://github.com/ipfs/kubo/issues/11186 ║\n║ ║\n╚════════════════════════════════════════════════════════════════════════════╝\n`)\n\tp := c.path\n\tif !filepath.IsAbs(p) {\n\t\tp = filepath.Join(path, p)\n\t}\n\n\terr := os.MkdirAll(p, 0o755)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefopts := badgerds.DefaultOptions\n\tdefopts.SyncWrites = c.syncWrites\n\tdefopts.Truncate = c.truncate\n\tdefopts.ValueLogFileSize = c.vlogFileSize\n\n\treturn badgerds.NewDatastore(p, &defopts)\n}\n" }, { "path": "plugin/plugins/dagjose/dagjose.go", "content": "package dagjose\n\nimport (\n\t\"github.com/ipfs/kubo/plugin\"\n\n\t\"github.com/ceramicnetwork/go-dag-jose/dagjose\"\n\t\"github.com/ipld/go-ipld-prime/multicodec\"\n\tmc \"github.com/multiformats/go-multicodec\"\n)\n\n// Plugins is exported list of plugins that will be loaded.\nvar Plugins = []plugin.Plugin{\n\t&dagjosePlugin{},\n}\n\ntype dagjosePlugin struct{}\n\nvar _ plugin.PluginIPLD = (*dagjosePlugin)(nil)\n\nfunc (*dagjosePlugin) Name() string {\n\treturn \"ipld-codec-dagjose\"\n}\n\nfunc (*dagjosePlugin) Version() string {\n\treturn \"0.0.1\"\n}\n\nfunc (*dagjosePlugin) Init(_ *plugin.Environment) error {\n\treturn nil\n}\n\nfunc (*dagjosePlugin) Register(reg multicodec.Registry) error {\n\treg.RegisterEncoder(uint64(mc.DagJose), dagjose.Encode)\n\treg.RegisterDecoder(uint64(mc.DagJose), dagjose.Decode)\n\treturn nil\n}\n" }, { "path": "plugin/plugins/flatfs/flatfs.go", "content": "package flatfs\n\nimport (\n\t\"fmt\"\n\t\"path/filepath\"\n\n\t\"github.com/ipfs/kubo/plugin\"\n\t\"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\n\tflatfs \"github.com/ipfs/go-ds-flatfs\"\n)\n\n// Plugins is exported list of plugins that will be loaded.\nvar Plugins = []plugin.Plugin{\n\t&flatfsPlugin{},\n}\n\ntype flatfsPlugin struct{}\n\nvar _ plugin.PluginDatastore = (*flatfsPlugin)(nil)\n\nfunc (*flatfsPlugin) Name() string {\n\treturn \"ds-flatfs\"\n}\n\nfunc (*flatfsPlugin) Version() string {\n\treturn \"0.1.0\"\n}\n\nfunc (*flatfsPlugin) Init(_ *plugin.Environment) error {\n\treturn nil\n}\n\nfunc (*flatfsPlugin) DatastoreTypeName() string {\n\treturn \"flatfs\"\n}\n\ntype datastoreConfig struct {\n\tpath string\n\tshardFun *flatfs.ShardIdV1\n\tsyncField bool\n}\n\n// DatastoreConfigParser returns a configuration stub for a flatfs datastore\n// from the given parameters.\nfunc (*flatfsPlugin) DatastoreConfigParser() fsrepo.ConfigFromMap {\n\treturn func(params map[string]any) (fsrepo.DatastoreConfig, error) {\n\t\tvar c datastoreConfig\n\t\tvar ok bool\n\t\tvar err error\n\n\t\tc.path, ok = params[\"path\"].(string)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"'path' field is missing or not boolean\")\n\t\t}\n\n\t\tsshardFun, ok := params[\"shardFunc\"].(string)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"'shardFunc' field is missing or not a string\")\n\t\t}\n\t\tc.shardFun, err = flatfs.ParseShardFunc(sshardFun)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tc.syncField, ok = params[\"sync\"].(bool)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"'sync' field is missing or not boolean\")\n\t\t}\n\t\treturn &c, nil\n\t}\n}\n\nfunc (c *datastoreConfig) DiskSpec() fsrepo.DiskSpec {\n\treturn map[string]any{\n\t\t\"type\": \"flatfs\",\n\t\t\"path\": c.path,\n\t\t\"shardFunc\": c.shardFun.String(),\n\t}\n}\n\nfunc (c *datastoreConfig) Create(path string) (repo.Datastore, error) {\n\tp := c.path\n\tif !filepath.IsAbs(p) {\n\t\tp = filepath.Join(path, p)\n\t}\n\n\treturn flatfs.CreateOrOpen(p, c.shardFun, c.syncField)\n}\n" }, { "path": "plugin/plugins/fxtest/fxtest.go", "content": "package fxtest\n\nimport (\n\t\"os\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/plugin\"\n\t\"go.uber.org/fx\"\n)\n\nvar log = logging.Logger(\"fxtestplugin\")\n\nvar Plugins = []plugin.Plugin{\n\t&fxtestPlugin{},\n}\n\n// fxtestPlugin is used for testing the fx plugin.\n// It merely adds an fx option that logs a debug statement, so we can verify that it works in tests.\ntype fxtestPlugin struct{}\n\nvar _ plugin.PluginFx = (*fxtestPlugin)(nil)\n\nfunc (p *fxtestPlugin) Name() string {\n\treturn \"fx-test\"\n}\n\nfunc (p *fxtestPlugin) Version() string {\n\treturn \"0.1.0\"\n}\n\nfunc (p *fxtestPlugin) Init(env *plugin.Environment) error {\n\treturn nil\n}\n\nfunc (p *fxtestPlugin) Options(info core.FXNodeInfo) ([]fx.Option, error) {\n\topts := info.FXOptions\n\tif os.Getenv(\"TEST_FX_PLUGIN\") != \"\" {\n\t\topts = append(opts, fx.Invoke(func() {\n\t\t\tlog.Debug(\"invoked test fx function\")\n\t\t}))\n\t}\n\treturn opts, nil\n}\n" }, { "path": "plugin/plugins/gen_main.sh", "content": "#!/usr/bin/env bash\n\ndir=${1:?first parameter with dir to work in is required}\npkg=${2:?second parameter with full name of the package is required}\nmain_pkg=\"$dir/main\"\n\nshortpkg=\"uniquepkgname\"\n\nmkdir -p \"$main_pkg\"\n\ncat > \"$main_pkg/main.go\" < 0 {\n\t\t\tbusyCounter++\n\n\t\t\t// sleep a bit to give the system a chance to catch up with logging.\n\t\t\tselect {\n\t\t\tcase <-time.After(time.Duration(busyCounter) * time.Second):\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t// drain 1/8th of the backlog backlog so we\n\t\t\t// don't immediately run into this situation\n\t\t\t// again.\n\t\tloop:\n\t\t\tfor range busyDropAmount {\n\t\t\t\tselect {\n\t\t\t\tcase <-pl.events:\n\t\t\t\t\tdropped++\n\t\t\t\tdefault:\n\t\t\t\t\tbreak loop\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Add in any events we've dropped in the mean-time.\n\t\t\tdropped += atomic.SwapUint64(&pl.droppedCount, 0)\n\n\t\t\t// Report that we've dropped events.\n\t\t\tdlog.Error(\"dropped events\", zap.Uint64(\"count\", dropped))\n\t\t} else {\n\t\t\tbusyCounter = 0\n\t\t}\n\n\t\tvar e plEvent\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase e = <-pl.events:\n\t\t}\n\n\t\tpeerID := zap.String(\"peer\", e.peer.String())\n\n\t\tswitch e.kind {\n\t\tcase eventConnect:\n\t\t\tdlog.Info(\"connected\", peerID)\n\t\tcase eventIdentify:\n\t\t\tagent, err := node.Peerstore.Get(e.peer, \"AgentVersion\")\n\t\t\tswitch err {\n\t\t\tcase nil:\n\t\t\tcase peerstore.ErrNotFound:\n\t\t\t\tcontinue\n\t\t\tdefault:\n\t\t\t\tdlog.Error(\"failed to get agent version\", zap.Error(err))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tagentS, ok := agent.(string)\n\t\t\tif !ok {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tdlog.Info(\"identified\", peerID, zap.String(\"agent\", agentS))\n\t\t}\n\t}\n}\n\nfunc (pl *peerLogPlugin) emit(evt eventType, p peer.ID) {\n\tselect {\n\tcase pl.events <- plEvent{kind: evt, peer: p}:\n\tdefault:\n\t\tatomic.AddUint64(&pl.droppedCount, 1)\n\t}\n}\n\nfunc (pl *peerLogPlugin) Start(node *core.IpfsNode) error {\n\tif !pl.enabled {\n\t\treturn nil\n\t}\n\n\t// Ensure logs from this plugin get printed regardless of global GOLOG_LOG_LEVEL value\n\tif err := logging.SetLogLevel(\"plugin/peerlog\", \"info\"); err != nil {\n\t\treturn fmt.Errorf(\"failed to set log level: %w\", err)\n\t}\n\n\tsub, err := node.PeerHost.EventBus().Subscribe(new(event.EvtPeerIdentificationCompleted))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to subscribe to identify notifications\")\n\t}\n\n\tvar notifee network.NotifyBundle\n\tnotifee.ConnectedF = func(net network.Network, conn network.Conn) {\n\t\tpl.emit(eventConnect, conn.RemotePeer())\n\t}\n\tnode.PeerHost.Network().Notify(¬ifee)\n\n\tgo func() {\n\t\tdefer sub.Close()\n\t\tfor e := range sub.Out() {\n\t\t\tswitch e := e.(type) {\n\t\t\tcase event.EvtPeerIdentificationCompleted:\n\t\t\t\tpl.emit(eventIdentify, e.Peer)\n\t\t\t}\n\t\t}\n\t}()\n\n\tgo pl.collectEvents(node)\n\n\treturn nil\n}\n\nfunc (*peerLogPlugin) Close() error {\n\treturn nil\n}\n" }, { "path": "plugin/plugins/peerlog/peerlog_test.go", "content": "package peerlog\n\nimport \"testing\"\n\nfunc TestExtractEnabled(t *testing.T) {\n\tfor _, c := range []struct {\n\t\tname string\n\t\tconfig any\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"nil config returns false\",\n\t\t\tconfig: nil,\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"returns false when config is not a string map\",\n\t\t\tconfig: 1,\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"returns false when config has no Enabled field\",\n\t\t\tconfig: map[string]any{},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"returns false when config has a null Enabled field\",\n\t\t\tconfig: map[string]any{\"Enabled\": nil},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"returns false when config has a non-boolean Enabled field\",\n\t\t\tconfig: map[string]any{\"Enabled\": 1},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"returns the value of the Enabled field\",\n\t\t\tconfig: map[string]any{\"Enabled\": true},\n\t\t\texpected: true,\n\t\t},\n\t} {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tisEnabled := extractEnabled(c.config)\n\t\t\tif isEnabled != c.expected {\n\t\t\t\tt.Fatalf(\"expected %v, got %v\", c.expected, isEnabled)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "plugin/plugins/telemetry/telemetry.go", "content": "package telemetry\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"os\"\n\t\"path\"\n\t\"runtime\"\n\t\"slices\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tipfs \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/corerepo\"\n\t\"github.com/ipfs/kubo/plugin\"\n\t\"github.com/libp2p/go-libp2p/core/network\"\n\t\"github.com/libp2p/go-libp2p/core/pnet\"\n\tmultiaddr \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nvar log = logging.Logger(\"telemetry\")\n\n// Caching for virtualization detection - these values never change during process lifetime\nvar (\n\tcontainerDetectionOnce sync.Once\n\tvmDetectionOnce sync.Once\n\tisContainerCached bool\n\tisVMCached bool\n)\n\nconst (\n\tmodeEnvVar = \"IPFS_TELEMETRY\"\n\tuuidFilename = \"telemetry_uuid\"\n\tendpoint = \"https://telemetry.ipshipyard.dev\"\n\tsendDelay = 15 * time.Minute // delay before first telemetry collection after daemon start\n\tsendInterval = 24 * time.Hour // interval between telemetry collections after the first one\n\thttpTimeout = 30 * time.Second // timeout for telemetry HTTP requests\n)\n\ntype pluginMode int\n\nconst (\n\tmodeAuto pluginMode = iota\n\tmodeOn\n\tmodeOff\n)\n\n// repoSizeBuckets defines size thresholds for categorizing repository sizes.\n// Each value represents the upper limit of a bucket in bytes (except the last)\nvar repoSizeBuckets = []uint64{\n\t1 << 30, // 1 GB\n\t5 << 30, // 5 GB\n\t10 << 30, // 10 GB\n\t100 << 30, // 100 GB\n\t500 << 30, // 500 GB\n\t1 << 40, // 1 TB\n\t10 << 40, // 10 TB\n\t11 << 40, // + anything more than 10TB falls here.\n}\n\nvar uptimeBuckets = []time.Duration{\n\t1 * 24 * time.Hour,\n\t2 * 24 * time.Hour,\n\t3 * 24 * time.Hour,\n\t7 * 24 * time.Hour,\n\t14 * 24 * time.Hour,\n\t30 * 24 * time.Hour,\n\t31 * 24 * time.Hour, // + anything more than 30 days falls here.\n}\n\n// A LogEvent is the object sent to the telemetry endpoint.\n// See https://github.com/ipfs/kubo/blob/master/docs/telemetry.md for details.\ntype LogEvent struct {\n\tUUID string `json:\"uuid\"`\n\n\tAgentVersion string `json:\"agent_version\"`\n\n\tPrivateNetwork bool `json:\"private_network\"`\n\n\tBootstrappersCustom bool `json:\"bootstrappers_custom\"`\n\n\tRepoSizeBucket uint64 `json:\"repo_size_bucket\"`\n\n\tUptimeBucket time.Duration `json:\"uptime_bucket\"`\n\n\tReproviderStrategy string `json:\"reprovider_strategy\"`\n\tProvideDHTSweepEnabled bool `json:\"provide_dht_sweep_enabled\"`\n\tProvideDHTIntervalCustom bool `json:\"provide_dht_interval_custom\"`\n\tProvideDHTMaxWorkersCustom bool `json:\"provide_dht_max_workers_custom\"`\n\n\tRoutingType string `json:\"routing_type\"`\n\tRoutingAcceleratedDHTClient bool `json:\"routing_accelerated_dht_client\"`\n\tRoutingDelegatedCount int `json:\"routing_delegated_count\"`\n\n\tAutoNATServiceMode string `json:\"autonat_service_mode\"`\n\tAutoNATReachability string `json:\"autonat_reachability\"`\n\n\tAutoConf bool `json:\"autoconf\"`\n\tAutoConfCustom bool `json:\"autoconf_custom\"`\n\n\tSwarmEnableHolePunching bool `json:\"swarm_enable_hole_punching\"`\n\tSwarmCircuitAddresses bool `json:\"swarm_circuit_addresses\"`\n\tSwarmIPv4PublicAddresses bool `json:\"swarm_ipv4_public_addresses\"`\n\tSwarmIPv6PublicAddresses bool `json:\"swarm_ipv6_public_addresses\"`\n\n\tAutoTLSAutoWSS bool `json:\"auto_tls_auto_wss\"`\n\tAutoTLSDomainSuffixCustom bool `json:\"auto_tls_domain_suffix_custom\"`\n\n\tDiscoveryMDNSEnabled bool `json:\"discovery_mdns_enabled\"`\n\n\tPlatformOS string `json:\"platform_os\"`\n\tPlatformArch string `json:\"platform_arch\"`\n\tPlatformContainerized bool `json:\"platform_containerized\"`\n\tPlatformVM bool `json:\"platform_vm\"`\n}\n\nvar Plugins = []plugin.Plugin{\n\t&telemetryPlugin{},\n}\n\ntype telemetryPlugin struct {\n\tuuidFilename string\n\tmode pluginMode\n\tendpoint string\n\trunOnce bool // test-only flag: when true, sends telemetry immediately without delay\n\tsendDelay time.Duration\n\n\tnode *core.IpfsNode\n\tconfig *config.Config\n\tevent *LogEvent\n\tstartTime time.Time\n}\n\nfunc (p *telemetryPlugin) Name() string {\n\treturn \"telemetry\"\n}\n\nfunc (p *telemetryPlugin) Version() string {\n\treturn \"0.0.1\"\n}\n\nfunc readFromConfig(cfg any, key string) string {\n\tif cfg == nil {\n\t\treturn \"\"\n\t}\n\n\tpcfg, ok := cfg.(map[string]any)\n\tif !ok {\n\t\treturn \"\"\n\t}\n\n\tval, ok := pcfg[key].(string)\n\tif !ok {\n\t\treturn \"\"\n\t}\n\treturn val\n}\n\nfunc (p *telemetryPlugin) Init(env *plugin.Environment) error {\n\t// logging.SetLogLevel(\"telemetry\", \"DEBUG\")\n\tlog.Debug(\"telemetry plugin Init()\")\n\tp.event = &LogEvent{}\n\tp.startTime = time.Now()\n\n\trepoPath := env.Repo\n\tp.uuidFilename = path.Join(repoPath, uuidFilename)\n\n\tv := os.Getenv(modeEnvVar)\n\tif v != \"\" {\n\t\tlog.Debug(\"mode set from env-var\")\n\t} else if pmode := readFromConfig(env.Config, \"Mode\"); pmode != \"\" {\n\t\tv = pmode\n\t\tlog.Debug(\"mode set from config\")\n\t}\n\n\t// read \"Delay\" from the config. Parse as duration. Set p.sendDelay to it\n\t// or set default.\n\tif delayStr := readFromConfig(env.Config, \"Delay\"); delayStr != \"\" {\n\t\tdelay, err := time.ParseDuration(delayStr)\n\t\tif err != nil {\n\t\t\tlog.Debug(\"sendDelay set from default\")\n\t\t\tp.sendDelay = sendDelay\n\t\t} else {\n\t\t\tlog.Debug(\"sendDelay set from config\")\n\t\t\tp.sendDelay = delay\n\t\t}\n\t} else {\n\t\tlog.Debug(\"sendDelay set from default\")\n\t\tp.sendDelay = sendDelay\n\t}\n\n\tp.endpoint = endpoint\n\tif ep := readFromConfig(env.Config, \"Endpoint\"); ep != \"\" {\n\t\tlog.Debug(\"endpoint set from config\", ep)\n\t\tp.endpoint = ep\n\t}\n\n\tswitch v {\n\tcase \"off\":\n\t\tp.mode = modeOff\n\t\tlog.Debug(\"telemetry disabled via opt-out\")\n\t\t// Remove UUID file if it exists when user opts out\n\t\tif _, err := os.Stat(p.uuidFilename); err == nil {\n\t\t\tif err := os.Remove(p.uuidFilename); err != nil {\n\t\t\t\tlog.Debugf(\"failed to remove telemetry UUID file: %s\", err)\n\t\t\t} else {\n\t\t\t\tlog.Debug(\"removed existing telemetry UUID file due to opt-out\")\n\t\t\t}\n\t\t}\n\t\treturn nil\n\tcase \"auto\":\n\t\tp.mode = modeAuto\n\tdefault:\n\t\tp.mode = modeOn\n\t}\n\tlog.Debug(\"telemetry mode: \", p.mode)\n\treturn nil\n}\n\nfunc (p *telemetryPlugin) loadUUID() error {\n\t// Generate or read our UUID from disk\n\tb, err := os.ReadFile(p.uuidFilename)\n\tif err != nil {\n\t\tif !os.IsNotExist(err) {\n\t\t\tlog.Errorf(\"error reading telemetry uuid from disk: %s\", err)\n\t\t\treturn err\n\t\t}\n\t\tuid, err := uuid.NewRandom()\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"cannot generate telemetry uuid: %s\", err)\n\t\t\treturn err\n\t\t}\n\t\tp.event.UUID = uid.String()\n\t\tp.mode = modeAuto\n\t\tlog.Debugf(\"new telemetry UUID %s. Mode set to Auto\", uid)\n\n\t\t// Write the UUID to disk\n\t\tif err := os.WriteFile(p.uuidFilename, []byte(p.event.UUID), 0600); err != nil {\n\t\t\tlog.Errorf(\"cannot write telemetry uuid: %s\", err)\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t}\n\n\tv := string(b)\n\tv = strings.TrimSpace(v)\n\tuid, err := uuid.Parse(v)\n\tif err != nil {\n\t\tlog.Errorf(\"cannot parse telemetry uuid: %s\", err)\n\t\treturn err\n\t}\n\tlog.Debugf(\"uuid read from disk %s\", uid)\n\tp.event.UUID = uid.String()\n\treturn nil\n}\n\nfunc (p *telemetryPlugin) hasDefaultBootstrapPeers() bool {\n\t// With autoconf, default bootstrap is represented as [\"auto\"]\n\tcurrentPeers := p.config.Bootstrap\n\treturn len(currentPeers) == 1 && currentPeers[0] == \"auto\"\n}\n\nfunc (p *telemetryPlugin) showInfo() {\n\tfmt.Printf(`\n\nℹ️ Anonymous telemetry will be enabled in %s\n\nKubo will collect anonymous usage data to help improve the software:\n• What: Feature usage and configuration (no personal data)\n Use GOLOG_LOG_LEVEL=\"telemetry=debug\" to inspect collected data\n• When: First collection in %s, then every 24h\n• How: HTTP POST to %s\n Anonymous ID: %s\n\nNo data sent yet. To opt-out before collection starts:\n• Set environment: %s=off\n• Or run: ipfs config Plugins.Plugins.telemetry.Config.Mode off\n• Then restart daemon\n\nThis message is shown only once.\nLearn more: https://github.com/ipfs/kubo/blob/master/docs/telemetry.md\n\n\n`, p.sendDelay, p.sendDelay, endpoint, p.event.UUID, modeEnvVar)\n}\n\n// Start finishes telemetry initialization once the IpfsNode is ready,\n// collects telemetry data and sends it to the endpoint.\nfunc (p *telemetryPlugin) Start(n *core.IpfsNode) error {\n\t// We should not be crashing the daemon due to problems with telemetry\n\t// so this is always going to return nil and panics are going to be\n\t// handled.\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\tlog.Errorf(\"telemetry plugin panicked: %v\", r)\n\t\t}\n\t}()\n\n\tp.node = n\n\tcfg, err := n.Repo.Config()\n\tif err != nil {\n\t\tlog.Error(\"error getting the repo.Config: %s\", err)\n\t\treturn nil\n\t}\n\tp.config = cfg\n\tif p.mode == modeOff {\n\t\tlog.Debug(\"telemetry collection skipped: opted out\")\n\t\treturn nil\n\t}\n\n\tif !n.IsDaemon || !n.IsOnline {\n\t\tlog.Debugf(\"skipping telemetry. Daemon: %t. Online: %t\", n.IsDaemon, n.IsOnline)\n\t\treturn nil\n\t}\n\n\t// loadUUID might switch to modeAuto when generating a new uuid\n\tif err := p.loadUUID(); err != nil {\n\t\tp.mode = modeOff\n\t\treturn nil\n\t}\n\n\tif p.mode == modeAuto {\n\t\tp.showInfo()\n\t}\n\n\t// runOnce is only used in tests to send telemetry immediately.\n\t// In production, this is always false, ensuring users get the 15-minute delay.\n\tif p.runOnce {\n\t\tp.prepareEvent()\n\t\treturn p.sendTelemetry()\n\t}\n\n\tgo func() {\n\t\ttimer := time.NewTimer(p.sendDelay)\n\t\tfor range timer.C {\n\t\t\tp.prepareEvent()\n\t\t\tif err := p.sendTelemetry(); err != nil {\n\t\t\t\tlog.Warnf(\"telemetry submission failed: %s (will retry in %s)\", err, sendInterval)\n\t\t\t}\n\t\t\ttimer.Reset(sendInterval)\n\t\t}\n\t}()\n\n\treturn nil\n}\n\nfunc (p *telemetryPlugin) prepareEvent() {\n\tp.collectBasicInfo()\n\tp.collectRoutingInfo()\n\tp.collectProvideInfo()\n\tp.collectAutoNATInfo()\n\tp.collectAutoConfInfo()\n\tp.collectSwarmInfo()\n\tp.collectAutoTLSInfo()\n\tp.collectDiscoveryInfo()\n\tp.collectPlatformInfo()\n}\n\nfunc (p *telemetryPlugin) collectBasicInfo() {\n\tp.event.AgentVersion = ipfs.GetUserAgentVersion()\n\n\tprivNet := false\n\tif pnet.ForcePrivateNetwork {\n\t\tprivNet = true\n\t} else if key, _ := p.node.Repo.SwarmKey(); key != nil {\n\t\tprivNet = true\n\t}\n\tp.event.PrivateNetwork = privNet\n\n\tp.event.BootstrappersCustom = !p.hasDefaultBootstrapPeers()\n\n\trepoSizeBucket := repoSizeBuckets[len(repoSizeBuckets)-1]\n\tsizeStat, err := corerepo.RepoSize(context.Background(), p.node)\n\tif err == nil {\n\t\tfor _, b := range repoSizeBuckets {\n\t\t\tif sizeStat.RepoSize > b {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\trepoSizeBucket = b\n\t\t\tbreak\n\t\t}\n\t\tp.event.RepoSizeBucket = repoSizeBucket\n\t} else {\n\t\tlog.Debugf(\"error setting sizeStat: %s\", err)\n\t}\n\n\tuptime := time.Since(p.startTime)\n\tuptimeBucket := uptimeBuckets[len(uptimeBuckets)-1]\n\tfor _, bucket := range uptimeBuckets {\n\t\tif uptime > bucket {\n\t\t\tcontinue\n\n\t\t}\n\t\tuptimeBucket = bucket\n\t\tbreak\n\t}\n\tp.event.UptimeBucket = uptimeBucket\n}\n\nfunc (p *telemetryPlugin) collectRoutingInfo() {\n\tp.event.RoutingType = p.config.Routing.Type.WithDefault(\"auto\")\n\tp.event.RoutingAcceleratedDHTClient = p.config.Routing.AcceleratedDHTClient.WithDefault(false)\n\tp.event.RoutingDelegatedCount = len(p.config.Routing.DelegatedRouters)\n}\n\nfunc (p *telemetryPlugin) collectProvideInfo() {\n\tp.event.ReproviderStrategy = p.config.Provide.Strategy.WithDefault(config.DefaultProvideStrategy)\n\tp.event.ProvideDHTSweepEnabled = p.config.Provide.DHT.SweepEnabled.WithDefault(config.DefaultProvideDHTSweepEnabled)\n\tp.event.ProvideDHTIntervalCustom = !p.config.Provide.DHT.Interval.IsDefault()\n\tp.event.ProvideDHTMaxWorkersCustom = !p.config.Provide.DHT.MaxWorkers.IsDefault()\n}\n\ntype reachabilityHost interface {\n\tReachability() network.Reachability\n}\n\nfunc (p *telemetryPlugin) collectAutoNATInfo() {\n\tautonat := p.config.AutoNAT.ServiceMode\n\tif autonat == config.AutoNATServiceUnset {\n\t\tautonat = config.AutoNATServiceEnabled\n\t}\n\tautoNATSvcModeB, err := autonat.MarshalText()\n\tif err == nil {\n\t\tautoNATSvcMode := string(autoNATSvcModeB)\n\t\tif autoNATSvcMode == \"\" {\n\t\t\tautoNATSvcMode = \"unset\"\n\t\t}\n\t\tp.event.AutoNATServiceMode = autoNATSvcMode\n\t}\n\n\th := p.node.PeerHost\n\treachHost, ok := h.(reachabilityHost)\n\tif ok {\n\t\tp.event.AutoNATReachability = reachHost.Reachability().String()\n\t}\n}\n\nfunc (p *telemetryPlugin) collectSwarmInfo() {\n\tp.event.SwarmEnableHolePunching = p.config.Swarm.EnableHolePunching.WithDefault(true)\n\n\tvar circuitAddrs, publicIP4Addrs, publicIP6Addrs bool\n\tfor _, addr := range p.node.PeerHost.Addrs() {\n\t\tif manet.IsPublicAddr(addr) {\n\t\t\tif _, err := addr.ValueForProtocol(multiaddr.P_IP4); err == nil {\n\t\t\t\tpublicIP4Addrs = true\n\t\t\t} else if _, err := addr.ValueForProtocol(multiaddr.P_IP6); err == nil {\n\t\t\t\tpublicIP6Addrs = true\n\t\t\t}\n\t\t}\n\t\tif _, err := addr.ValueForProtocol(multiaddr.P_CIRCUIT); err == nil {\n\t\t\tcircuitAddrs = true\n\t\t}\n\t}\n\n\tp.event.SwarmCircuitAddresses = circuitAddrs\n\tp.event.SwarmIPv4PublicAddresses = publicIP4Addrs\n\tp.event.SwarmIPv6PublicAddresses = publicIP6Addrs\n}\n\nfunc (p *telemetryPlugin) collectAutoTLSInfo() {\n\tp.event.AutoTLSAutoWSS = p.config.AutoTLS.AutoWSS.WithDefault(config.DefaultAutoWSS)\n\tdomainSuffix := p.config.AutoTLS.DomainSuffix.WithDefault(config.DefaultDomainSuffix)\n\tp.event.AutoTLSDomainSuffixCustom = domainSuffix != config.DefaultDomainSuffix\n}\n\nfunc (p *telemetryPlugin) collectAutoConfInfo() {\n\tp.event.AutoConf = p.config.AutoConf.Enabled.WithDefault(config.DefaultAutoConfEnabled)\n\tp.event.AutoConfCustom = p.config.AutoConf.URL.WithDefault(config.DefaultAutoConfURL) != config.DefaultAutoConfURL\n}\n\nfunc (p *telemetryPlugin) collectDiscoveryInfo() {\n\tp.event.DiscoveryMDNSEnabled = p.config.Discovery.MDNS.Enabled\n}\n\nfunc (p *telemetryPlugin) collectPlatformInfo() {\n\tp.event.PlatformOS = runtime.GOOS\n\tp.event.PlatformArch = runtime.GOARCH\n\tp.event.PlatformContainerized = isRunningInContainer()\n\tp.event.PlatformVM = isRunningInVM()\n}\n\nfunc isRunningInContainer() bool {\n\tcontainerDetectionOnce.Do(func() {\n\t\tisContainerCached = detectContainer()\n\t})\n\treturn isContainerCached\n}\n\nfunc detectContainer() bool {\n\t// Docker creates /.dockerenv inside containers\n\tif _, err := os.Stat(\"/.dockerenv\"); err == nil {\n\t\treturn true\n\t}\n\n\t// Kubernetes mounts service account tokens inside pods\n\tif _, err := os.Stat(\"/var/run/secrets/kubernetes.io\"); err == nil {\n\t\treturn true\n\t}\n\n\t// systemd-nspawn creates this file inside containers\n\tif _, err := os.Stat(\"/run/systemd/container\"); err == nil {\n\t\treturn true\n\t}\n\n\t// Check if our process is running inside a container cgroup\n\t// Look for container-specific patterns in the cgroup path after \"::/\"\n\tif content, err := os.ReadFile(\"/proc/self/cgroup\"); err == nil {\n\t\tfor line := range strings.Lines(string(content)) {\n\t\t\t// cgroup lines format: \"ID:subsystem:/path\"\n\t\t\t// We want to check the path part after the last \":\"\n\t\t\tparts := strings.SplitN(line, \":\", 3)\n\t\t\tif len(parts) == 3 {\n\t\t\t\tcgroupPath := parts[2]\n\t\t\t\t// Check for container-specific paths\n\t\t\t\tcontainerIndicators := []string{\n\t\t\t\t\t\"/docker/\", // Docker containers\n\t\t\t\t\t\"/containerd/\", // containerd runtime\n\t\t\t\t\t\"/cri-o/\", // CRI-O runtime\n\t\t\t\t\t\"/lxc/\", // LXC containers\n\t\t\t\t\t\"/podman/\", // Podman containers\n\t\t\t\t\t\"/kubepods/\", // Kubernetes pods\n\t\t\t\t}\n\t\t\t\tfor _, indicator := range containerIndicators {\n\t\t\t\t\tif strings.Contains(cgroupPath, indicator) {\n\t\t\t\t\t\treturn true\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// WSL is technically a container-like environment\n\tif runtime.GOOS == \"linux\" {\n\t\tif content, err := os.ReadFile(\"/proc/sys/kernel/osrelease\"); err == nil {\n\t\t\tosrelease := strings.ToLower(string(content))\n\t\t\tif strings.Contains(osrelease, \"microsoft\") || strings.Contains(osrelease, \"wsl\") {\n\t\t\t\treturn true\n\t\t\t}\n\t\t}\n\t}\n\n\t// LXC sets container environment variable\n\tif content, err := os.ReadFile(\"/proc/1/environ\"); err == nil {\n\t\tif strings.Contains(string(content), \"container=lxc\") {\n\t\t\treturn true\n\t\t}\n\t}\n\n\t// Additional check: In containers, PID 1 is often not systemd/init\n\tif content, err := os.ReadFile(\"/proc/1/comm\"); err == nil {\n\t\tpid1 := strings.TrimSpace(string(content))\n\t\t// Common container init processes\n\t\tcontainerInits := []string{\"tini\", \"dumb-init\", \"s6-svscan\", \"runit\"}\n\t\tif slices.Contains(containerInits, pid1) {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc isRunningInVM() bool {\n\tvmDetectionOnce.Do(func() {\n\t\tisVMCached = detectVM()\n\t})\n\treturn isVMCached\n}\n\nfunc detectVM() bool {\n\t// Check for VM-specific files and drivers that only exist inside VMs\n\tvmIndicators := []string{\n\t\t\"/proc/xen\", // Xen hypervisor guest\n\t\t\"/sys/hypervisor/uuid\", // KVM/Xen hypervisor guest\n\t\t\"/dev/vboxguest\", // VirtualBox guest additions\n\t\t\"/sys/module/vmw_balloon\", // VMware balloon driver (guest only)\n\t\t\"/sys/module/hv_vmbus\", // Hyper-V VM bus driver (guest only)\n\t}\n\n\tfor _, path := range vmIndicators {\n\t\tif _, err := os.Stat(path); err == nil {\n\t\t\treturn true\n\t\t}\n\t}\n\n\t// Check DMI for VM vendors - these strings only appear inside VMs\n\t// DMI (Desktop Management Interface) is populated by the hypervisor\n\tdmiFiles := map[string][]string{\n\t\t\"/sys/class/dmi/id/sys_vendor\": {\n\t\t\t\"qemu\", \"kvm\", \"vmware\", \"virtualbox\", \"xen\",\n\t\t\t\"parallels\", // Parallels Desktop\n\t\t\t// Note: Removed \"microsoft corporation\" as it can match Surface devices\n\t\t},\n\t\t\"/sys/class/dmi/id/product_name\": {\n\t\t\t\"virtualbox\", \"vmware\", \"kvm\", \"qemu\",\n\t\t\t\"hvm domu\", // Xen HVM guest\n\t\t\t// Note: Removed generic \"virtual machine\" to avoid false positives\n\t\t},\n\t\t\"/sys/class/dmi/id/chassis_vendor\": {\n\t\t\t\"qemu\", \"oracle\", // Oracle for VirtualBox\n\t\t},\n\t}\n\n\tfor path, signatures := range dmiFiles {\n\t\tif content, err := os.ReadFile(path); err == nil {\n\t\t\tcontentStr := strings.ToLower(strings.TrimSpace(string(content)))\n\t\t\tfor _, sig := range signatures {\n\t\t\t\tif strings.Contains(contentStr, sig) {\n\t\t\t\t\treturn true\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc (p *telemetryPlugin) sendTelemetry() error {\n\tdata, err := json.MarshalIndent(p.event, \"\", \" \")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlog.Debugf(\"sending telemetry:\\n %s\", data)\n\n\treq, err := http.NewRequest(\"POST\", p.endpoint, bytes.NewBuffer(data))\n\tif err != nil {\n\t\treturn err\n\t}\n\treq.Header.Set(\"Content-Type\", \"application/json\")\n\treq.Header.Set(\"User-Agent\", ipfs.GetUserAgentVersion())\n\treq.Close = true\n\n\t// Use client with timeout to prevent hanging\n\tclient := &http.Client{\n\t\tTimeout: httpTimeout,\n\t}\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\tlog.Debugf(\"failed to send telemetry: %s\", err)\n\t\treturn err\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode >= 400 {\n\t\terr := fmt.Errorf(\"telemetry endpoint returned HTTP %d\", resp.StatusCode)\n\t\tlog.Debug(err)\n\t\treturn err\n\t}\n\tlog.Debugf(\"telemetry sent successfully (%d)\", resp.StatusCode)\n\treturn nil\n}\n" }, { "path": "plugin/plugins/telemetry/telemetry_test.go", "content": "package telemetry\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/cockroachdb/pebble/v2\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/plugin\"\n\t\"github.com/ipfs/kubo/plugin/plugins/pebbleds\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n)\n\nfunc mockServer(t *testing.T) (*httptest.Server, func() LogEvent) {\n\tt.Helper()\n\n\tvar e LogEvent\n\n\t// Create a mock HTTP test server\n\treturn httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t// Check if the request is POST to the correct endpoint\n\t\tif r.Method != \"POST\" || r.URL.Path != \"/\" {\n\t\t\tt.Log(\"invalid request\")\n\t\t\thttp.Error(w, \"invalid request\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\t// Check content type\n\t\tif r.Header.Get(\"Content-Type\") != \"application/json\" {\n\t\t\tt.Log(\"invalid content type\")\n\t\t\thttp.Error(w, \"invalid content type\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\t// Check if the body is not empty\n\t\tif r.Body == nil {\n\t\t\tt.Log(\"empty body\")\n\t\t\thttp.Error(w, \"empty body\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\t// Read the body\n\t\tbody, _ := io.ReadAll(r.Body)\n\t\tif len(body) == 0 {\n\t\t\tt.Log(\"zero-length body\")\n\t\t\thttp.Error(w, \"empty body\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\tt.Logf(\"Received telemetry:\\n %s\", string(body))\n\n\t\terr := json.Unmarshal(body, &e)\n\t\tif err != nil {\n\t\t\tt.Log(\"error unmarshaling event\", err)\n\t\t\thttp.Error(w, err.Error(), http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\t// Return success\n\t\tw.WriteHeader(http.StatusOK)\n\t})), func() LogEvent { return e }\n}\n\nfunc makeNode(t *testing.T) (node *core.IpfsNode, repopath string) {\n\tt.Helper()\n\n\t// Create a Temporary Repo\n\trepoPath, err := os.MkdirTemp(\"\", \"ipfs-shell\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpebbledspli := pebbleds.Plugins[0]\n\tpebbledspl, ok := pebbledspli.(plugin.PluginDatastore)\n\tif !ok {\n\t\tt.Fatal(\"bad datastore plugin\")\n\t}\n\n\terr = fsrepo.AddDatastoreConfigHandler(pebbledspl.DatastoreTypeName(), pebbledspl.DatastoreConfigParser())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create a config with default options and a 2048 bit key\n\tcfg, err := config.Init(io.Discard, 2048)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tcfg.Datastore.Spec = map[string]any{\n\t\t\"type\": \"pebbleds\",\n\t\t\"prefix\": \"pebble.datastore\",\n\t\t\"path\": \"pebbleds\",\n\t\t\"formatMajorVersion\": int(pebble.FormatNewest),\n\t}\n\n\t// Create the repo with the config\n\terr = fsrepo.Init(repoPath, cfg)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Open the repo\n\trepo, err := fsrepo.Open(repoPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Construct the node\n\n\tnodeOptions := &core.BuildCfg{\n\t\tOnline: true,\n\t\tRouting: libp2p.NilRouterOption,\n\t\tRepo: repo,\n\t}\n\n\tnode, err = core.NewNode(context.Background(), nodeOptions)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tnode.IsDaemon = true\n\treturn\n}\n\nfunc TestSendTelemetry(t *testing.T) {\n\tif err := logging.SetLogLevel(\"telemetry\", \"DEBUG\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tts, eventGetter := mockServer(t)\n\tdefer ts.Close()\n\n\tnode, repoPath := makeNode(t)\n\n\t// Create a plugin instance\n\tp := &telemetryPlugin{\n\t\trunOnce: true,\n\t}\n\n\t// Initialize the plugin\n\tpe := &plugin.Environment{\n\t\tRepo: repoPath,\n\t\tConfig: nil,\n\t}\n\terr := p.Init(pe)\n\tif err != nil {\n\t\tt.Fatalf(\"Init() failed: %v\", err)\n\t}\n\n\tp.endpoint = ts.URL\n\n\t// Start the plugin\n\terr = p.Start(node)\n\tif err != nil {\n\t\tt.Fatalf(\"Start() failed: %v\", err)\n\t}\n\n\te := eventGetter()\n\tif e.UUID != p.event.UUID {\n\t\tt.Fatal(\"uuid mismatch\")\n\t}\n}\n" }, { "path": "plugin/plugins/telemetry/telemetry_uuid", "content": "289ffed8-c770-49ae-922f-b020c8f776f2" }, { "path": "plugin/tracer.go", "content": "package plugin\n\nimport (\n\t\"github.com/opentracing/opentracing-go\"\n)\n\n// PluginTracer is an interface that can be implemented to add a tracer.\ntype PluginTracer interface {\n\tPlugin\n\n\tInitTracer() (opentracing.Tracer, error)\n}\n" }, { "path": "profile/goroutines.go", "content": "package profile\n\nimport (\n\t\"io\"\n\t\"runtime\"\n)\n\n// WriteAllGoroutineStacks writes a stack trace to the given writer.\n// This is distinct from the Go-provided method because it does not truncate after 64 MB.\nfunc WriteAllGoroutineStacks(w io.Writer) error {\n\t// this is based on pprof.writeGoroutineStacks, and removes the 64 MB limit\n\tbuf := make([]byte, 1<<20)\n\tfor i := 0; ; i++ {\n\t\tn := runtime.Stack(buf, true)\n\t\tif n < len(buf) {\n\t\t\tbuf = buf[:n]\n\t\t\tbreak\n\t\t}\n\t\t// if len(buf) >= 64<<20 {\n\t\t// \t// Filled 64 MB - stop there.\n\t\t// \tbreak\n\t\t// }\n\t\tbuf = make([]byte, 2*len(buf))\n\t}\n\t_, err := w.Write(buf)\n\treturn err\n}\n" }, { "path": "profile/profile.go", "content": "package profile\n\nimport (\n\t\"archive/zip\"\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"runtime\"\n\t\"runtime/pprof\"\n\t\"runtime/trace\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/ipfs/go-log/v2\"\n\tversion \"github.com/ipfs/kubo\"\n)\n\nconst (\n\tCollectorGoroutinesStack = \"goroutines-stack\"\n\tCollectorGoroutinesPprof = \"goroutines-pprof\"\n\tCollectorVersion = \"version\"\n\tCollectorHeap = \"heap\"\n\tCollectorAllocs = \"allocs\"\n\tCollectorBin = \"bin\"\n\tCollectorCPU = \"cpu\"\n\tCollectorMutex = \"mutex\"\n\tCollectorBlock = \"block\"\n\tCollectorTrace = \"trace\"\n)\n\nvar (\n\tlogger = log.Logger(\"profile\")\n\tgoos = runtime.GOOS\n)\n\ntype collector struct {\n\toutputFile string\n\tisExecutable bool\n\tcollectFunc func(ctx context.Context, opts Options, writer io.Writer) error\n\tenabledFunc func(opts Options) bool\n}\n\nfunc (p *collector) outputFileName() string {\n\tfName := p.outputFile\n\tif p.isExecutable {\n\t\tif goos == \"windows\" {\n\t\t\tfName += \".exe\"\n\t\t}\n\t}\n\treturn fName\n}\n\nvar collectors = map[string]collector{\n\tCollectorGoroutinesStack: {\n\t\toutputFile: \"goroutines.stacks\",\n\t\tcollectFunc: goroutineStacksText,\n\t\tenabledFunc: func(opts Options) bool { return true },\n\t},\n\tCollectorGoroutinesPprof: {\n\t\toutputFile: \"goroutines.pprof\",\n\t\tcollectFunc: goroutineStacksProto,\n\t\tenabledFunc: func(opts Options) bool { return true },\n\t},\n\tCollectorVersion: {\n\t\toutputFile: \"version.json\",\n\t\tcollectFunc: versionInfo,\n\t\tenabledFunc: func(opts Options) bool { return true },\n\t},\n\tCollectorHeap: {\n\t\toutputFile: \"heap.pprof\",\n\t\tcollectFunc: heapProfile,\n\t\tenabledFunc: func(opts Options) bool { return true },\n\t},\n\tCollectorAllocs: {\n\t\toutputFile: \"allocs.pprof\",\n\t\tcollectFunc: allocsProfile,\n\t\tenabledFunc: func(opts Options) bool { return true },\n\t},\n\tCollectorBin: {\n\t\toutputFile: \"ipfs\",\n\t\tisExecutable: true,\n\t\tcollectFunc: binary,\n\t\tenabledFunc: func(opts Options) bool { return true },\n\t},\n\tCollectorCPU: {\n\t\toutputFile: \"cpu.pprof\",\n\t\tcollectFunc: profileCPU,\n\t\tenabledFunc: func(opts Options) bool { return opts.ProfileDuration > 0 },\n\t},\n\tCollectorMutex: {\n\t\toutputFile: \"mutex.pprof\",\n\t\tcollectFunc: mutexProfile,\n\t\tenabledFunc: func(opts Options) bool { return opts.ProfileDuration > 0 && opts.MutexProfileFraction > 0 },\n\t},\n\tCollectorBlock: {\n\t\toutputFile: \"block.pprof\",\n\t\tcollectFunc: blockProfile,\n\t\tenabledFunc: func(opts Options) bool { return opts.ProfileDuration > 0 && opts.BlockProfileRate > 0 },\n\t},\n\tCollectorTrace: {\n\t\toutputFile: \"trace\",\n\t\tcollectFunc: captureTrace,\n\t\tenabledFunc: func(opts Options) bool { return opts.ProfileDuration > 0 },\n\t},\n}\n\ntype Options struct {\n\tCollectors []string\n\tProfileDuration time.Duration\n\tMutexProfileFraction int\n\tBlockProfileRate time.Duration\n}\n\nfunc WriteProfiles(ctx context.Context, archive *zip.Writer, opts Options) error {\n\tp := profiler{\n\t\tarchive: archive,\n\t\topts: opts,\n\t}\n\treturn p.runProfile(ctx)\n}\n\n// profiler runs the collectors concurrently and writes the results to the zip archive.\ntype profiler struct {\n\tarchive *zip.Writer\n\topts Options\n}\n\nfunc (p *profiler) runProfile(ctx context.Context) error {\n\ttype profileResult struct {\n\t\tfName string\n\t\tbuf *bytes.Buffer\n\t\terr error\n\t}\n\n\tctx, cancelFn := context.WithCancel(ctx)\n\tdefer cancelFn()\n\n\tcollectorsToRun := make([]collector, len(p.opts.Collectors))\n\tfor i, name := range p.opts.Collectors {\n\t\tc, ok := collectors[name]\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"unknown collector '%s'\", name)\n\t\t}\n\t\tcollectorsToRun[i] = c\n\t}\n\n\tresults := make(chan profileResult, len(p.opts.Collectors))\n\twg := sync.WaitGroup{}\n\tfor _, c := range collectorsToRun {\n\t\tif !c.enabledFunc(p.opts) {\n\t\t\tcontinue\n\t\t}\n\n\t\tfName := c.outputFileName()\n\n\t\twg.Add(1)\n\t\tgo func(c collector) {\n\t\t\tdefer wg.Done()\n\t\t\tlogger.Infow(\"collecting profile\", \"File\", fName)\n\t\t\tdefer logger.Infow(\"profile done\", \"File\", fName)\n\t\t\tb := bytes.Buffer{}\n\t\t\terr := c.collectFunc(ctx, p.opts, &b)\n\t\t\tif err != nil {\n\t\t\t\tselect {\n\t\t\t\tcase results <- profileResult{err: fmt.Errorf(\"generating profile data for %q: %w\", fName, err)}:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase results <- profileResult{buf: &b, fName: fName}:\n\t\t\tcase <-ctx.Done():\n\t\t\t}\n\t\t}(c)\n\t}\n\tgo func() {\n\t\twg.Wait()\n\t\tclose(results)\n\t}()\n\n\tfor res := range results {\n\t\tif res.err != nil {\n\t\t\treturn res.err\n\t\t}\n\t\tout, err := p.archive.Create(res.fName)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"creating output file %q: %w\", res.fName, err)\n\t\t}\n\t\t_, err = io.Copy(out, res.buf)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"compressing result %q: %w\", res.fName, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc goroutineStacksText(ctx context.Context, _ Options, w io.Writer) error {\n\treturn WriteAllGoroutineStacks(w)\n}\n\nfunc goroutineStacksProto(ctx context.Context, _ Options, w io.Writer) error {\n\treturn pprof.Lookup(\"goroutine\").WriteTo(w, 0)\n}\n\nfunc heapProfile(ctx context.Context, _ Options, w io.Writer) error {\n\treturn pprof.Lookup(\"heap\").WriteTo(w, 0)\n}\n\nfunc allocsProfile(ctx context.Context, _ Options, w io.Writer) error {\n\treturn pprof.Lookup(\"allocs\").WriteTo(w, 0)\n}\n\nfunc versionInfo(ctx context.Context, _ Options, w io.Writer) error {\n\treturn json.NewEncoder(w).Encode(version.GetVersionInfo())\n}\n\nfunc binary(ctx context.Context, _ Options, w io.Writer) error {\n\tvar (\n\t\tpath string\n\t\terr error\n\t)\n\tif goos == \"linux\" {\n\t\tpid := os.Getpid()\n\t\tpath = fmt.Sprintf(\"/proc/%d/exe\", pid)\n\t} else {\n\t\tpath, err = os.Executable()\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"finding binary path: %w\", err)\n\t\t}\n\t}\n\tfi, err := os.Open(path)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"opening binary %q: %w\", path, err)\n\t}\n\t_, err = io.Copy(w, fi)\n\t_ = fi.Close()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"copying binary %q: %w\", path, err)\n\t}\n\treturn nil\n}\n\nfunc mutexProfile(ctx context.Context, opts Options, w io.Writer) error {\n\tprev := runtime.SetMutexProfileFraction(opts.MutexProfileFraction)\n\tdefer runtime.SetMutexProfileFraction(prev)\n\terr := waitOrCancel(ctx, opts.ProfileDuration)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn pprof.Lookup(\"mutex\").WriteTo(w, 2)\n}\n\nfunc blockProfile(ctx context.Context, opts Options, w io.Writer) error {\n\truntime.SetBlockProfileRate(int(opts.BlockProfileRate.Nanoseconds()))\n\tdefer runtime.SetBlockProfileRate(0)\n\terr := waitOrCancel(ctx, opts.ProfileDuration)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn pprof.Lookup(\"block\").WriteTo(w, 2)\n}\n\nfunc profileCPU(ctx context.Context, opts Options, w io.Writer) error {\n\terr := pprof.StartCPUProfile(w)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer pprof.StopCPUProfile()\n\treturn waitOrCancel(ctx, opts.ProfileDuration)\n}\n\nfunc captureTrace(ctx context.Context, opts Options, w io.Writer) error {\n\terr := trace.Start(w)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer trace.Stop()\n\treturn waitOrCancel(ctx, opts.ProfileDuration)\n}\n\nfunc waitOrCancel(ctx context.Context, d time.Duration) error {\n\ttimer := time.NewTimer(d)\n\tdefer timer.Stop()\n\tselect {\n\tcase <-timer.C:\n\t\treturn nil\n\tcase <-ctx.Done():\n\t\treturn ctx.Err()\n\t}\n}\n" }, { "path": "profile/profile_test.go", "content": "package profile\n\nimport (\n\t\"archive/zip\"\n\t\"bytes\"\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestProfiler(t *testing.T) {\n\tallCollectors := []string{\n\t\tCollectorGoroutinesStack,\n\t\tCollectorGoroutinesPprof,\n\t\tCollectorVersion,\n\t\tCollectorHeap,\n\t\tCollectorAllocs,\n\t\tCollectorBin,\n\t\tCollectorCPU,\n\t\tCollectorMutex,\n\t\tCollectorBlock,\n\t\tCollectorTrace,\n\t}\n\n\tcases := []struct {\n\t\tname string\n\t\topts Options\n\t\tgoos string\n\n\t\texpectFiles []string\n\t}{\n\t\t{\n\t\t\tname: \"happy case\",\n\t\t\topts: Options{\n\t\t\t\tCollectors: allCollectors,\n\t\t\t\tProfileDuration: 1 * time.Millisecond,\n\t\t\t\tMutexProfileFraction: 4,\n\t\t\t\tBlockProfileRate: 50 * time.Nanosecond,\n\t\t\t},\n\t\t\texpectFiles: []string{\n\t\t\t\t\"goroutines.stacks\",\n\t\t\t\t\"goroutines.pprof\",\n\t\t\t\t\"version.json\",\n\t\t\t\t\"heap.pprof\",\n\t\t\t\t\"allocs.pprof\",\n\t\t\t\t\"ipfs\",\n\t\t\t\t\"cpu.pprof\",\n\t\t\t\t\"mutex.pprof\",\n\t\t\t\t\"block.pprof\",\n\t\t\t\t\"trace\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"windows\",\n\t\t\topts: Options{\n\t\t\t\tCollectors: allCollectors,\n\t\t\t\tProfileDuration: 1 * time.Millisecond,\n\t\t\t\tMutexProfileFraction: 4,\n\t\t\t\tBlockProfileRate: 50 * time.Nanosecond,\n\t\t\t},\n\t\t\tgoos: \"windows\",\n\t\t\texpectFiles: []string{\n\t\t\t\t\"goroutines.stacks\",\n\t\t\t\t\"goroutines.pprof\",\n\t\t\t\t\"version.json\",\n\t\t\t\t\"heap.pprof\",\n\t\t\t\t\"allocs.pprof\",\n\t\t\t\t\"ipfs.exe\",\n\t\t\t\t\"cpu.pprof\",\n\t\t\t\t\"mutex.pprof\",\n\t\t\t\t\"block.pprof\",\n\t\t\t\t\"trace\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"sampling profiling disabled\",\n\t\t\topts: Options{\n\t\t\t\tCollectors: allCollectors,\n\t\t\t\tMutexProfileFraction: 4,\n\t\t\t\tBlockProfileRate: 50 * time.Nanosecond,\n\t\t\t},\n\t\t\texpectFiles: []string{\n\t\t\t\t\"goroutines.stacks\",\n\t\t\t\t\"goroutines.pprof\",\n\t\t\t\t\"version.json\",\n\t\t\t\t\"heap.pprof\",\n\t\t\t\t\"allocs.pprof\",\n\t\t\t\t\"ipfs\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Mutex profiling disabled\",\n\t\t\topts: Options{\n\t\t\t\tCollectors: allCollectors,\n\t\t\t\tProfileDuration: 1 * time.Millisecond,\n\t\t\t\tBlockProfileRate: 50 * time.Nanosecond,\n\t\t\t},\n\t\t\texpectFiles: []string{\n\t\t\t\t\"goroutines.stacks\",\n\t\t\t\t\"goroutines.pprof\",\n\t\t\t\t\"version.json\",\n\t\t\t\t\"heap.pprof\",\n\t\t\t\t\"allocs.pprof\",\n\t\t\t\t\"ipfs\",\n\t\t\t\t\"cpu.pprof\",\n\t\t\t\t\"block.pprof\",\n\t\t\t\t\"trace\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"block profiling disabled\",\n\t\t\topts: Options{\n\t\t\t\tCollectors: allCollectors,\n\t\t\t\tProfileDuration: 1 * time.Millisecond,\n\t\t\t\tMutexProfileFraction: 4,\n\t\t\t\tBlockProfileRate: 0,\n\t\t\t},\n\t\t\texpectFiles: []string{\n\t\t\t\t\"goroutines.stacks\",\n\t\t\t\t\"goroutines.pprof\",\n\t\t\t\t\"version.json\",\n\t\t\t\t\"heap.pprof\",\n\t\t\t\t\"allocs.pprof\",\n\t\t\t\t\"ipfs\",\n\t\t\t\t\"cpu.pprof\",\n\t\t\t\t\"mutex.pprof\",\n\t\t\t\t\"trace\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single collector\",\n\t\t\topts: Options{\n\t\t\t\tCollectors: []string{CollectorVersion},\n\t\t\t\tProfileDuration: 1 * time.Millisecond,\n\t\t\t\tMutexProfileFraction: 4,\n\t\t\t\tBlockProfileRate: 0,\n\t\t\t},\n\t\t\texpectFiles: []string{\n\t\t\t\t\"version.json\",\n\t\t\t},\n\t\t},\n\t}\n\tfor _, c := range cases {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tif c.goos != \"\" {\n\t\t\t\toldGOOS := goos\n\t\t\t\tgoos = c.goos\n\t\t\t\tdefer func() { goos = oldGOOS }()\n\t\t\t}\n\n\t\t\tbuf := &bytes.Buffer{}\n\t\t\tarchive := zip.NewWriter(buf)\n\t\t\terr := WriteProfiles(context.Background(), archive, c.opts)\n\t\t\trequire.NoError(t, err)\n\n\t\t\terr = archive.Close()\n\t\t\trequire.NoError(t, err)\n\n\t\t\tzr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), int64(buf.Len()))\n\t\t\trequire.NoError(t, err)\n\n\t\t\tfor _, f := range zr.File {\n\t\t\t\tlogger.Info(\"zip file: \", f.Name)\n\t\t\t}\n\n\t\t\trequire.Equal(t, len(c.expectFiles), len(zr.File))\n\n\t\t\tfor _, expectedFile := range c.expectFiles {\n\t\t\t\tfunc() {\n\t\t\t\t\tf, err := zr.Open(expectedFile)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tdefer f.Close()\n\t\t\t\t\tfi, err := f.Stat()\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.NotZero(t, fi.Size())\n\t\t\t\t}()\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "repo/common/common.go", "content": "package common\n\nimport (\n\t\"fmt\"\n\t\"maps\"\n\t\"strings\"\n)\n\nfunc MapGetKV(v map[string]any, key string) (any, error) {\n\tvar ok bool\n\tvar mcursor map[string]any\n\tvar cursor any = v\n\n\tparts := strings.Split(key, \".\")\n\tfor i, part := range parts {\n\t\tsofar := strings.Join(parts[:i], \".\")\n\n\t\tmcursor, ok = cursor.(map[string]any)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"%s key is not a map\", sofar)\n\t\t}\n\n\t\tcursor, ok = mcursor[part]\n\t\tif !ok {\n\t\t\t// Construct the current path traversed to print a nice error message\n\t\t\tvar path string\n\t\t\tif len(sofar) > 0 {\n\t\t\t\tpath += sofar + \".\"\n\t\t\t}\n\t\t\tpath += part\n\t\t\treturn nil, fmt.Errorf(\"%s not found\", path)\n\t\t}\n\t}\n\treturn cursor, nil\n}\n\nfunc MapSetKV(v map[string]any, key string, value any) error {\n\tvar ok bool\n\tvar mcursor map[string]any\n\tvar cursor any = v\n\n\tparts := strings.Split(key, \".\")\n\tfor i, part := range parts {\n\t\tmcursor, ok = cursor.(map[string]any)\n\t\tif !ok {\n\t\t\tsofar := strings.Join(parts[:i], \".\")\n\t\t\treturn fmt.Errorf(\"%s key is not a map\", sofar)\n\t\t}\n\n\t\t// last part? set here\n\t\tif i == (len(parts) - 1) {\n\t\t\tmcursor[part] = value\n\t\t\tbreak\n\t\t}\n\n\t\tcursor, ok = mcursor[part]\n\t\tif !ok || cursor == nil { // create map if this is empty or is null\n\t\t\tmcursor[part] = map[string]any{}\n\t\t\tcursor = mcursor[part]\n\t\t}\n\t}\n\treturn nil\n}\n\n// MapMergeDeep merges the right map into the left map, recursively traversing\n// child maps until a non-map value is found.\nfunc MapMergeDeep(left, right map[string]any) map[string]any {\n\t// We want to alter a copy of the map, not the original\n\tresult := maps.Clone(left)\n\tif result == nil {\n\t\tresult = make(map[string]any)\n\t}\n\n\tfor key, rightVal := range right {\n\t\t// If right value is a map\n\t\tif rightMap, ok := rightVal.(map[string]any); ok {\n\t\t\t// If key is in left\n\t\t\tif leftVal, found := result[key]; found {\n\t\t\t\t// If left value is also a map\n\t\t\t\tif leftMap, ok := leftVal.(map[string]any); ok {\n\t\t\t\t\t// Merge nested map\n\t\t\t\t\tresult[key] = MapMergeDeep(leftMap, rightMap)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Otherwise set new value to result\n\t\tresult[key] = rightVal\n\t}\n\n\treturn result\n}\n" }, { "path": "repo/common/common_test.go", "content": "package common\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestMapMergeDeepReturnsNew(t *testing.T) {\n\tleftMap := make(map[string]any)\n\tleftMap[\"A\"] = \"Hello World\"\n\n\trightMap := make(map[string]any)\n\trightMap[\"A\"] = \"Foo\"\n\n\tMapMergeDeep(leftMap, rightMap)\n\n\trequire.Equal(t, \"Hello World\", leftMap[\"A\"], \"MapMergeDeep should return a new map instance\")\n}\n\nfunc TestMapMergeDeepNewKey(t *testing.T) {\n\tleftMap := make(map[string]any)\n\tleftMap[\"A\"] = \"Hello World\"\n\t/*\n\t\tleftMap\n\t\t{\n\t\t\tA: \"Hello World\"\n\t\t}\n\t*/\n\n\trightMap := make(map[string]any)\n\trightMap[\"B\"] = \"Bar\"\n\t/*\n\t\trightMap\n\t\t{\n\t\t\tB: \"Bar\"\n\t\t}\n\t*/\n\n\tresult := MapMergeDeep(leftMap, rightMap)\n\t/*\n\t\texpected\n\t\t{\n\t\t\tA: \"Hello World\"\n\t\t\tB: \"Bar\"\n\t\t}\n\t*/\n\n\trequire.Equal(t, \"Bar\", result[\"B\"], \"New keys in right map should exist in resulting map\")\n}\n\nfunc TestMapMergeDeepRecursesOnMaps(t *testing.T) {\n\tleftMapA := make(map[string]any)\n\tleftMapA[\"B\"] = \"A value!\"\n\tleftMapA[\"C\"] = \"Another value!\"\n\n\tleftMap := make(map[string]any)\n\tleftMap[\"A\"] = leftMapA\n\t/*\n\t\tleftMap\n\t\t{\n\t\t\tA: {\n\t\t\t\tB: \"A value!\"\n\t\t\t\tC: \"Another value!\"\n\t\t\t}\n\t\t}\n\t*/\n\n\trightMapA := make(map[string]any)\n\trightMapA[\"C\"] = \"A different value!\"\n\n\trightMap := make(map[string]any)\n\trightMap[\"A\"] = rightMapA\n\t/*\n\t\trightMap\n\t\t{\n\t\t\tA: {\n\t\t\t\tC: \"A different value!\"\n\t\t\t}\n\t\t}\n\t*/\n\n\tresult := MapMergeDeep(leftMap, rightMap)\n\t/*\n\t\texpected\n\t\t{\n\t\t\tA: {\n\t\t\t\tB: \"A value!\"\n\t\t\t\tC: \"A different value!\"\n\t\t\t}\n\t\t}\n\t*/\n\n\tresultA := result[\"A\"].(map[string]any)\n\trequire.Equal(t, \"A value!\", resultA[\"B\"], \"Unaltered values should not change\")\n\trequire.Equal(t, \"A different value!\", resultA[\"C\"], \"Nested values should be altered\")\n}\n\nfunc TestMapMergeDeepRightNotAMap(t *testing.T) {\n\tleftMapA := make(map[string]any)\n\tleftMapA[\"B\"] = \"A value!\"\n\n\tleftMap := make(map[string]any)\n\tleftMap[\"A\"] = leftMapA\n\t/*\n\t\torigMap\n\t\t{\n\t\t\tA: {\n\t\t\t\tB: \"A value!\"\n\t\t\t}\n\t\t}\n\t*/\n\n\trightMap := make(map[string]any)\n\trightMap[\"A\"] = \"Not a map!\"\n\t/*\n\t\tnewMap\n\t\t{\n\t\t\tA: \"Not a map!\"\n\t\t}\n\t*/\n\n\tresult := MapMergeDeep(leftMap, rightMap)\n\t/*\n\t\texpected\n\t\t{\n\t\t\tA: \"Not a map!\"\n\t\t}\n\t*/\n\n\trequire.Equal(t, \"Not a map!\", result[\"A\"], \"Right values that are not a map should be set on the result\")\n}\n" }, { "path": "repo/fsrepo/config_test.go", "content": "package fsrepo_test\n\nimport (\n\t\"encoding/json\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/plugin/loader\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\n\t\"github.com/ipfs/kubo/config\"\n)\n\n// note: to test sorting of the mountpoints in the disk spec they are\n// specified out of order in the test config.\nvar defaultConfig = []byte(`{\n \"StorageMax\": \"10GB\",\n \"StorageGCWatermark\": 90,\n \"GCPeriod\": \"1h\",\n \"Spec\": {\n \"mounts\": [\n {\n \"child\": {\n \"compression\": \"none\",\n \"path\": \"datastore\",\n \"type\": \"levelds\"\n },\n \"mountpoint\": \"/\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"measure\"\n },\n {\n \"child\": {\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": true,\n \"type\": \"flatfs\"\n },\n \"mountpoint\": \"/blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"type\": \"measure\"\n }\n ],\n \"type\": \"mount\"\n },\n \"HashOnRead\": false,\n \"BloomFilterSize\": 0\n}`)\n\nvar leveldbConfig = []byte(`{\n \"compression\": \"none\",\n \"path\": \"datastore\",\n \"type\": \"levelds\"\n}`)\n\nvar flatfsConfig = []byte(`{\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": true,\n \"type\": \"flatfs\"\n}`)\n\nvar measureConfig = []byte(`{\n \"child\": {\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": true,\n \"type\": \"flatfs\"\n },\n \"mountpoint\": \"/blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"type\": \"measure\"\n}`)\n\nfunc TestDefaultDatastoreConfig(t *testing.T) {\n\tloader, err := loader.NewPluginLoader(\"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\terr = loader.Initialize()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr = loader.Inject()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdir := t.TempDir()\n\n\tconfig := new(config.Datastore)\n\terr = json.Unmarshal(defaultConfig, config)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdsc, err := fsrepo.AnyDatastoreConfig(config.Spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\texpected := `{\"mounts\":[{\"mountpoint\":\"/blocks\",\"path\":\"blocks\",\"shardFunc\":\"/repo/flatfs/shard/v1/next-to-last/2\",\"type\":\"flatfs\"},{\"mountpoint\":\"/\",\"path\":\"datastore\",\"type\":\"levelds\"}],\"type\":\"mount\"}`\n\tif dsc.DiskSpec().String() != expected {\n\t\tt.Errorf(\"expected '%s' got '%s' as DiskId\", expected, dsc.DiskSpec().String())\n\t}\n\n\tds, err := dsc.Create(dir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif typ := reflect.TypeOf(ds).String(); typ != \"*mount.Datastore\" {\n\t\tt.Errorf(\"expected '*mount.Datastore' got '%s'\", typ)\n\t}\n}\n\nfunc TestLevelDbConfig(t *testing.T) {\n\tconfig := new(config.Datastore)\n\terr := json.Unmarshal(defaultConfig, config)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdir := t.TempDir()\n\n\tspec := make(map[string]any)\n\terr = json.Unmarshal(leveldbConfig, &spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdsc, err := fsrepo.AnyDatastoreConfig(spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\texpected := `{\"path\":\"datastore\",\"type\":\"levelds\"}`\n\tif dsc.DiskSpec().String() != expected {\n\t\tt.Errorf(\"expected '%s' got '%s' as DiskId\", expected, dsc.DiskSpec().String())\n\t}\n\n\tds, err := dsc.Create(dir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif typ := reflect.TypeOf(ds).String(); typ != \"*leveldb.Datastore\" {\n\t\tt.Errorf(\"expected '*leveldb.datastore' got '%s'\", typ)\n\t}\n}\n\nfunc TestFlatfsConfig(t *testing.T) {\n\tconfig := new(config.Datastore)\n\terr := json.Unmarshal(defaultConfig, config)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdir := t.TempDir()\n\n\tspec := make(map[string]any)\n\terr = json.Unmarshal(flatfsConfig, &spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdsc, err := fsrepo.AnyDatastoreConfig(spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\texpected := `{\"path\":\"blocks\",\"shardFunc\":\"/repo/flatfs/shard/v1/next-to-last/2\",\"type\":\"flatfs\"}`\n\tif dsc.DiskSpec().String() != expected {\n\t\tt.Errorf(\"expected '%s' got '%s' as DiskId\", expected, dsc.DiskSpec().String())\n\t}\n\n\tds, err := dsc.Create(dir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif typ := reflect.TypeOf(ds).String(); typ != \"*flatfs.Datastore\" {\n\t\tt.Errorf(\"expected '*flatfs.Datastore' got '%s'\", typ)\n\t}\n}\n\nfunc TestMeasureConfig(t *testing.T) {\n\tconfig := new(config.Datastore)\n\terr := json.Unmarshal(defaultConfig, config)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdir := t.TempDir()\n\n\tspec := make(map[string]any)\n\terr = json.Unmarshal(measureConfig, &spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdsc, err := fsrepo.AnyDatastoreConfig(spec)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\texpected := `{\"path\":\"blocks\",\"shardFunc\":\"/repo/flatfs/shard/v1/next-to-last/2\",\"type\":\"flatfs\"}`\n\tif dsc.DiskSpec().String() != expected {\n\t\tt.Errorf(\"expected '%s' got '%s' as DiskId\", expected, dsc.DiskSpec().String())\n\t}\n\n\tds, err := dsc.Create(dir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif typ := reflect.TypeOf(ds).String(); typ != \"*measure.measure\" {\n\t\tt.Errorf(\"expected '*measure.measure' got '%s'\", typ)\n\t}\n}\n" }, { "path": "repo/fsrepo/datastores.go", "content": "package fsrepo\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"sort\"\n\n\t\"github.com/ipfs/kubo/repo\"\n\n\tds \"github.com/ipfs/go-datastore\"\n\t\"github.com/ipfs/go-datastore/mount\"\n\tdssync \"github.com/ipfs/go-datastore/sync\"\n\t\"github.com/ipfs/go-ds-measure\"\n)\n\n// ConfigFromMap creates a new datastore config from a map.\ntype ConfigFromMap func(map[string]any) (DatastoreConfig, error)\n\n// DatastoreConfig is an abstraction of a datastore config. A \"spec\" is first\n// converted to a DatastoreConfig and then Create() is called to instantiate a\n// new datastore.\ntype DatastoreConfig interface {\n\t// DiskSpec returns a minimal configuration of the datastore representing\n\t// what is stored on disk. Run time values are excluded.\n\tDiskSpec() DiskSpec\n\n\t// Create instantiates a new datastore from this config.\n\tCreate(path string) (repo.Datastore, error)\n}\n\n// DiskSpec is a minimal representation of the characteristic values of the\n// datastore. If two diskspecs are the same, the loader assumes that they refer\n// to exactly the same datastore. If they differ at all, it is assumed they are\n// completely different datastores and a migration will be performed. Runtime\n// values such as cache options or concurrency options should not be added\n// here.\ntype DiskSpec map[string]any\n\n// Bytes returns a minimal JSON encoding of the DiskSpec.\nfunc (spec DiskSpec) Bytes() []byte {\n\tb, err := json.Marshal(spec)\n\tif err != nil {\n\t\t// should not happen\n\t\tpanic(err)\n\t}\n\treturn bytes.TrimSpace(b)\n}\n\n// String returns a minimal JSON encoding of the DiskSpec.\nfunc (spec DiskSpec) String() string {\n\treturn string(spec.Bytes())\n}\n\nvar datastores map[string]ConfigFromMap\n\nfunc init() {\n\tdatastores = map[string]ConfigFromMap{\n\t\t\"mount\": MountDatastoreConfig,\n\t\t\"mem\": MemDatastoreConfig,\n\t\t\"log\": LogDatastoreConfig,\n\t\t\"measure\": MeasureDatastoreConfig,\n\t}\n}\n\nfunc AddDatastoreConfigHandler(name string, dsc ConfigFromMap) error {\n\t_, ok := datastores[name]\n\tif ok {\n\t\treturn fmt.Errorf(\"already have a datastore named %q\", name)\n\t}\n\n\tdatastores[name] = dsc\n\treturn nil\n}\n\n// AnyDatastoreConfig returns a DatastoreConfig from a spec based on\n// the \"type\" parameter.\nfunc AnyDatastoreConfig(params map[string]any) (DatastoreConfig, error) {\n\twhich, ok := params[\"type\"].(string)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"'type' field missing or not a string\")\n\t}\n\tfun, ok := datastores[which]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"unknown datastore type: %s\", which)\n\t}\n\treturn fun(params)\n}\n\ntype mountDatastoreConfig struct {\n\tmounts []premount\n}\n\ntype premount struct {\n\tds DatastoreConfig\n\tprefix ds.Key\n}\n\n// MountDatastoreConfig returns a mount DatastoreConfig from a spec.\nfunc MountDatastoreConfig(params map[string]any) (DatastoreConfig, error) {\n\tvar res mountDatastoreConfig\n\tmounts, ok := params[\"mounts\"].([]any)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"'mounts' field is missing or not an array\")\n\t}\n\tfor _, iface := range mounts {\n\t\tcfg, ok := iface.(map[string]any)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"expected map for mountpoint\")\n\t\t}\n\n\t\tchild, err := AnyDatastoreConfig(cfg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tprefix, found := cfg[\"mountpoint\"]\n\t\tif !found {\n\t\t\treturn nil, fmt.Errorf(\"no 'mountpoint' on mount\")\n\t\t}\n\n\t\tres.mounts = append(res.mounts, premount{\n\t\t\tds: child,\n\t\t\tprefix: ds.NewKey(prefix.(string)),\n\t\t})\n\t}\n\tsort.Slice(res.mounts,\n\t\tfunc(i, j int) bool {\n\t\t\treturn res.mounts[i].prefix.String() > res.mounts[j].prefix.String()\n\t\t})\n\n\treturn &res, nil\n}\n\nfunc (c *mountDatastoreConfig) DiskSpec() DiskSpec {\n\tcfg := map[string]any{\"type\": \"mount\"}\n\tmounts := make([]any, len(c.mounts))\n\tfor i, m := range c.mounts {\n\t\tc := m.ds.DiskSpec()\n\t\tif c == nil {\n\t\t\tc = make(map[string]any)\n\t\t}\n\t\tc[\"mountpoint\"] = m.prefix.String()\n\t\tmounts[i] = c\n\t}\n\tcfg[\"mounts\"] = mounts\n\treturn cfg\n}\n\nfunc (c *mountDatastoreConfig) Create(path string) (repo.Datastore, error) {\n\tmounts := make([]mount.Mount, len(c.mounts))\n\tfor i, m := range c.mounts {\n\t\tds, err := m.ds.Create(path)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tmounts[i].Datastore = ds\n\t\tmounts[i].Prefix = m.prefix\n\t}\n\treturn mount.New(mounts), nil\n}\n\ntype memDatastoreConfig struct {\n\tcfg map[string]any\n}\n\n// MemDatastoreConfig returns a memory DatastoreConfig from a spec.\nfunc MemDatastoreConfig(params map[string]any) (DatastoreConfig, error) {\n\treturn &memDatastoreConfig{params}, nil\n}\n\nfunc (c *memDatastoreConfig) DiskSpec() DiskSpec {\n\treturn nil\n}\n\nfunc (c *memDatastoreConfig) Create(string) (repo.Datastore, error) {\n\treturn dssync.MutexWrap(ds.NewMapDatastore()), nil\n}\n\ntype logDatastoreConfig struct {\n\tchild DatastoreConfig\n\tname string\n}\n\n// LogDatastoreConfig returns a log DatastoreConfig from a spec.\nfunc LogDatastoreConfig(params map[string]any) (DatastoreConfig, error) {\n\tchildField, ok := params[\"child\"].(map[string]any)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"'child' field is missing or not a map\")\n\t}\n\tchild, err := AnyDatastoreConfig(childField)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tname, ok := params[\"name\"].(string)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"'name' field was missing or not a string\")\n\t}\n\treturn &logDatastoreConfig{child, name}, nil\n}\n\nfunc (c *logDatastoreConfig) Create(path string) (repo.Datastore, error) {\n\tchild, err := c.child.Create(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn ds.NewLogDatastore(child, c.name), nil\n}\n\nfunc (c *logDatastoreConfig) DiskSpec() DiskSpec {\n\treturn c.child.DiskSpec()\n}\n\ntype measureDatastoreConfig struct {\n\tchild DatastoreConfig\n\tprefix string\n}\n\n// MeasureDatastoreConfig returns a measure DatastoreConfig from a spec.\nfunc MeasureDatastoreConfig(params map[string]any) (DatastoreConfig, error) {\n\tchildField, ok := params[\"child\"].(map[string]any)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"'child' field is missing or not a map\")\n\t}\n\tchild, err := AnyDatastoreConfig(childField)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tprefix, ok := params[\"prefix\"].(string)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"'prefix' field was missing or not a string\")\n\t}\n\treturn &measureDatastoreConfig{child, prefix}, nil\n}\n\nfunc (c *measureDatastoreConfig) DiskSpec() DiskSpec {\n\treturn c.child.DiskSpec()\n}\n\nfunc (c measureDatastoreConfig) Create(path string) (repo.Datastore, error) {\n\tchild, err := c.child.Create(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn measure.New(c.prefix, child), nil\n}\n" }, { "path": "repo/fsrepo/doc.go", "content": "// package fsrepo\n//\n// TODO explain the package roadmap...\n//\n//\t.ipfs/\n//\t├── client/\n//\t| ├── client.lock <------ protects client/ + signals its own pid\n//\t│ ├── ipfs-client.cpuprof\n//\t│ └── ipfs-client.memprof\n//\t├── config\n//\t├── daemon/\n//\t│ ├── daemon.lock <------ protects daemon/ + signals its own address\n//\t│ ├── ipfs-daemon.cpuprof\n//\t│ └── ipfs-daemon.memprof\n//\t├── datastore/\n//\t├── repo.lock <------ protects datastore/ and config\n//\t└── version\npackage fsrepo\n\n// TODO prevent multiple daemons from running\n" }, { "path": "repo/fsrepo/fsrepo.go", "content": "package fsrepo\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\tfilestore \"github.com/ipfs/boxo/filestore\"\n\tkeystore \"github.com/ipfs/boxo/keystore\"\n\tversion \"github.com/ipfs/kubo\"\n\trepo \"github.com/ipfs/kubo/repo\"\n\t\"github.com/ipfs/kubo/repo/common\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\n\tds \"github.com/ipfs/go-datastore\"\n\tmeasure \"github.com/ipfs/go-ds-measure\"\n\tlockfile \"github.com/ipfs/go-fs-lock\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tserialize \"github.com/ipfs/kubo/config/serialize\"\n\t\"github.com/ipfs/kubo/misc/fsutil\"\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\n// LockFile is the filename of the repo lock, relative to config dir\n// TODO rename repo lock and hide name.\nconst LockFile = \"repo.lock\"\n\nvar log = logging.Logger(\"fsrepo\")\n\n// RepoVersion is the version number that we are currently expecting to see.\nvar RepoVersion = version.RepoVersion\n\nvar migrationInstructions = `See https://github.com/ipfs/fs-repo-migrations/blob/master/run.md\nSorry for the inconvenience. In the future, these will run automatically.`\n\nvar programTooLowMessage = `Your programs version (%d) is lower than your repos (%d).\nPlease update ipfs to a version that supports the existing repo, or run\na migration in reverse.\n\nSee https://github.com/ipfs/fs-repo-migrations/blob/master/run.md for details.`\n\nvar (\n\tErrNoVersion = errors.New(\"no version file found, please run 0-to-1 migration tool.\\n\" + migrationInstructions)\n\tErrOldRepo = errors.New(\"ipfs repo found in old '~/.go-ipfs' location, please run migration tool.\\n\" + migrationInstructions)\n\tErrNeedMigration = errors.New(\"ipfs repo needs migration, please run migration tool.\\n\" + migrationInstructions)\n)\n\ntype NoRepoError struct {\n\tPath string\n}\n\nvar _ error = NoRepoError{}\n\nfunc (err NoRepoError) Error() string {\n\treturn fmt.Sprintf(\"no IPFS repo found in %s.\\nplease run: 'ipfs init'\", err.Path)\n}\n\nconst (\n\tapiFile = \"api\"\n\tgatewayFile = \"gateway\"\n\tswarmKeyFile = \"swarm.key\"\n)\n\nconst specFn = \"datastore_spec\"\n\nvar (\n\n\t// packageLock must be held to while performing any operation that modifies an\n\t// FSRepo's state field. This includes Init, Open, Close, and Remove.\n\tpackageLock sync.Mutex\n\n\t// onlyOne keeps track of open FSRepo instances.\n\t//\n\t// TODO: once command Context / Repo integration is cleaned up,\n\t// this can be removed. Right now, this makes ConfigCmd.Run\n\t// function try to open the repo twice:\n\t//\n\t// $ ipfs daemon &\n\t// $ ipfs config foo\n\t//\n\t// The reason for the above is that in standalone mode without the\n\t// daemon, `ipfs config` tries to save work by not building the\n\t// full IpfsNode, but accessing the Repo directly.\n\tonlyOne repo.OnlyOne\n)\n\n// FSRepo represents an IPFS FileSystem Repo. It is safe for use by multiple\n// callers.\ntype FSRepo struct {\n\t// has Close been called already\n\tclosed bool\n\t// path is the file-system path\n\tpath string\n\t// Path to the configuration file that may or may not be inside the FSRepo\n\t// path (see config.Filename for more details).\n\tconfigFilePath string\n\t// lockfile is the file system lock to prevent others from opening\n\t// the same fsrepo path concurrently\n\tlockfile io.Closer\n\tconfig *config.Config\n\tuserResourceOverrides rcmgr.PartialLimitConfig\n\tds repo.Datastore\n\tkeystore keystore.Keystore\n\tfilemgr *filestore.FileManager\n}\n\nvar _ repo.Repo = (*FSRepo)(nil)\n\n// Open the FSRepo at path. Returns an error if the repo is not\n// initialized.\nfunc Open(repoPath string) (repo.Repo, error) {\n\tfn := func() (repo.Repo, error) {\n\t\treturn open(repoPath, \"\")\n\t}\n\treturn onlyOne.Open(repoPath, fn)\n}\n\n// OpenWithUserConfig is the equivalent to the Open function above but with the\n// option to set the configuration file path instead of using the default.\nfunc OpenWithUserConfig(repoPath string, userConfigFilePath string) (repo.Repo, error) {\n\tfn := func() (repo.Repo, error) {\n\t\treturn open(repoPath, userConfigFilePath)\n\t}\n\treturn onlyOne.Open(repoPath, fn)\n}\n\nfunc open(repoPath string, userConfigFilePath string) (repo.Repo, error) {\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tr, err := newFSRepo(repoPath, userConfigFilePath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Check if its initialized\n\tif err := checkInitialized(r.path); err != nil {\n\t\treturn nil, err\n\t}\n\n\ttext := os.Getenv(\"IPFS_WAIT_REPO_LOCK\")\n\tif text != \"\" {\n\t\tvar lockWaitTime time.Duration\n\t\tlockWaitTime, err = time.ParseDuration(text)\n\t\tif err != nil {\n\t\t\tlog.Errorw(\"Cannot parse value of IPFS_WAIT_REPO_LOCK as duration, not waiting for repo lock\", \"err\", err, \"value\", text)\n\t\t\tr.lockfile, err = lockfile.Lock(r.path, LockFile)\n\t\t} else if lockWaitTime <= 0 {\n\t\t\tr.lockfile, err = lockfile.WaitLock(context.Background(), r.path, LockFile)\n\t\t} else {\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), lockWaitTime)\n\t\t\tr.lockfile, err = lockfile.WaitLock(ctx, r.path, LockFile)\n\t\t\tcancel()\n\t\t}\n\t} else {\n\t\tr.lockfile, err = lockfile.Lock(r.path, LockFile)\n\t}\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tkeepLocked := false\n\tdefer func() {\n\t\t// unlock on error, leave it locked on success\n\t\tif !keepLocked {\n\t\t\tr.lockfile.Close()\n\t\t}\n\t}()\n\n\t// Check version, and error out if not matching\n\tver, err := migrations.RepoVersion(r.path)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn nil, ErrNoVersion\n\t\t}\n\t\treturn nil, err\n\t}\n\n\tif RepoVersion > ver {\n\t\treturn nil, ErrNeedMigration\n\t} else if ver > RepoVersion {\n\t\t// program version too low for existing repo\n\t\treturn nil, fmt.Errorf(programTooLowMessage, RepoVersion, ver)\n\t}\n\n\t// check repo path, then check all constituent parts.\n\tif err := fsutil.DirWritable(r.path); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := r.openConfig(); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := r.openUserResourceOverrides(); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := r.openDatastore(); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := r.openKeystore(); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif r.config.Experimental.FilestoreEnabled || r.config.Experimental.UrlstoreEnabled {\n\t\tr.filemgr = filestore.NewFileManager(r.ds, filepath.Dir(r.path))\n\t\tr.filemgr.AllowFiles = r.config.Experimental.FilestoreEnabled\n\t\tr.filemgr.AllowUrls = r.config.Experimental.UrlstoreEnabled\n\t}\n\n\tkeepLocked = true\n\treturn r, nil\n}\n\nfunc newFSRepo(rpath string, userConfigFilePath string) (*FSRepo, error) {\n\texpPath, err := fsutil.ExpandHome(filepath.Clean(rpath))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tconfigFilePath, err := config.Filename(rpath, userConfigFilePath)\n\tif err != nil {\n\t\t// FIXME: Personalize this when the user config path is \"\".\n\t\treturn nil, fmt.Errorf(\"finding config filepath from repo %s and user config %s: %w\",\n\t\t\trpath, userConfigFilePath, err)\n\t}\n\treturn &FSRepo{path: expPath, configFilePath: configFilePath}, nil\n}\n\nfunc checkInitialized(path string) error {\n\tif !isInitializedUnsynced(path) {\n\t\talt := strings.Replace(path, \".ipfs\", \".go-ipfs\", 1)\n\t\tif isInitializedUnsynced(alt) {\n\t\t\treturn ErrOldRepo\n\t\t}\n\t\treturn NoRepoError{Path: path}\n\t}\n\treturn nil\n}\n\n// configIsInitialized returns true if the repo is initialized at\n// provided |path|.\nfunc configIsInitialized(path string) bool {\n\tconfigFilename, err := config.Filename(path, \"\")\n\tif err != nil {\n\t\treturn false\n\t}\n\tif !fsutil.FileExists(configFilename) {\n\t\treturn false\n\t}\n\treturn true\n}\n\nfunc initConfig(path string, conf *config.Config) error {\n\tif configIsInitialized(path) {\n\t\treturn nil\n\t}\n\tconfigFilename, err := config.Filename(path, \"\")\n\tif err != nil {\n\t\treturn err\n\t}\n\t// initialization is the one time when it's okay to write to the config\n\t// without reading the config from disk and merging any user-provided keys\n\t// that may exist.\n\tif err := serialize.WriteConfigFile(configFilename, conf); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc initSpec(path string, conf map[string]any) error {\n\tfn, err := config.Path(path, specFn)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif fsutil.FileExists(fn) {\n\t\treturn nil\n\t}\n\n\tdsc, err := AnyDatastoreConfig(conf)\n\tif err != nil {\n\t\treturn err\n\t}\n\tbytes := dsc.DiskSpec().Bytes()\n\n\treturn os.WriteFile(fn, bytes, 0o600)\n}\n\n// Init initializes a new FSRepo at the given path with the provided config.\n// TODO add support for custom datastores.\nfunc Init(repoPath string, conf *config.Config) error {\n\t// packageLock must be held to ensure that the repo is not initialized more\n\t// than once.\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tif isInitializedUnsynced(repoPath) {\n\t\treturn nil\n\t}\n\n\tif err := initConfig(repoPath, conf); err != nil {\n\t\treturn err\n\t}\n\n\tif err := initSpec(repoPath, conf.Datastore.Spec); err != nil {\n\t\treturn err\n\t}\n\n\tif err := migrations.WriteRepoVersion(repoPath, RepoVersion); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// LockedByOtherProcess returns true if the FSRepo is locked by another\n// process. If true, then the repo cannot be opened by this process.\nfunc LockedByOtherProcess(repoPath string) (bool, error) {\n\trepoPath = filepath.Clean(repoPath)\n\tlocked, err := lockfile.Locked(repoPath, LockFile)\n\tif locked {\n\t\tlog.Debugf(\"(%t)<->Lock is held at %s\", locked, repoPath)\n\t}\n\treturn locked, err\n}\n\n// APIAddr returns the registered API addr, according to the api file\n// in the fsrepo. This is a concurrent operation, meaning that any\n// process may read this file. modifying this file, therefore, should\n// use \"mv\" to replace the whole file and avoid interleaved read/writes.\nfunc APIAddr(repoPath string) (ma.Multiaddr, error) {\n\trepoPath = filepath.Clean(repoPath)\n\tapiFilePath := filepath.Join(repoPath, apiFile)\n\n\t// if there is no file, assume there is no api addr.\n\tf, err := os.Open(apiFilePath)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn nil, repo.ErrApiNotRunning\n\t\t}\n\t\treturn nil, err\n\t}\n\tdefer f.Close()\n\n\t// read up to 2048 bytes. io.ReadAll is a vulnerability, as\n\t// someone could hose the process by putting a massive file there.\n\t//\n\t// NOTE(@stebalien): @jbenet probably wasn't thinking straight when he\n\t// wrote that comment but I'm leaving the limit here in case there was\n\t// some hidden wisdom. However, I'm fixing it such that:\n\t// 1. We don't read too little.\n\t// 2. We don't truncate and succeed.\n\tbuf, err := io.ReadAll(io.LimitReader(f, 2048))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif len(buf) == 2048 {\n\t\treturn nil, fmt.Errorf(\"API file too large, must be <2048 bytes long: %s\", apiFilePath)\n\t}\n\n\ts := string(buf)\n\ts = strings.TrimSpace(s)\n\treturn ma.NewMultiaddr(s)\n}\n\nfunc (r *FSRepo) Keystore() keystore.Keystore {\n\treturn r.keystore\n}\n\nfunc (r *FSRepo) Path() string {\n\treturn r.path\n}\n\n// SetAPIAddr writes the API Addr to the /api file.\nfunc (r *FSRepo) SetAPIAddr(addr ma.Multiaddr) error {\n\t// Create a temp file to write the address, so that we don't leave empty file when the\n\t// program crashes after creating the file.\n\tf, err := os.Create(filepath.Join(r.path, \".\"+apiFile+\".tmp\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif _, err = f.WriteString(addr.String()); err != nil {\n\t\tf.Close()\n\t\treturn err\n\t}\n\tif err = f.Close(); err != nil {\n\t\treturn err\n\t}\n\n\t// Atomically rename the temp file to the correct file name.\n\tif err = os.Rename(filepath.Join(r.path, \".\"+apiFile+\".tmp\"), filepath.Join(r.path,\n\t\tapiFile)); err == nil {\n\t\treturn nil\n\t}\n\t// Remove the temp file when rename return error\n\tif err1 := os.Remove(filepath.Join(r.path, \".\"+apiFile+\".tmp\")); err1 != nil {\n\t\treturn fmt.Errorf(\"file Rename error: %s, file remove error: %s\", err.Error(),\n\t\t\terr1.Error())\n\t}\n\treturn err\n}\n\n// SetGatewayAddr writes the Gateway Addr to the /gateway file.\nfunc (r *FSRepo) SetGatewayAddr(addr net.Addr) error {\n\t// Create a temp file to write the address, so that we don't leave empty file when the\n\t// program crashes after creating the file.\n\ttmpPath := filepath.Join(r.path, \".\"+gatewayFile+\".tmp\")\n\tf, err := os.Create(tmpPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tvar good bool\n\t// Silently remove as worst last case with defers.\n\tdefer func() {\n\t\tif !good {\n\t\t\tos.Remove(tmpPath)\n\t\t}\n\t}()\n\tdefer f.Close()\n\n\tif _, err := fmt.Fprintf(f, \"http://%s\", addr.String()); err != nil {\n\t\treturn err\n\t}\n\tif err := f.Close(); err != nil {\n\t\treturn err\n\t}\n\n\t// Atomically rename the temp file to the correct file name.\n\terr = os.Rename(tmpPath, filepath.Join(r.path, gatewayFile))\n\tgood = err == nil\n\tif good {\n\t\treturn nil\n\t}\n\t// Remove the temp file when rename return error\n\tif err1 := os.Remove(tmpPath); err1 != nil {\n\t\treturn fmt.Errorf(\"file Rename error: %w, file remove error: %s\", err, err1.Error())\n\t}\n\treturn err\n}\n\n// openConfig returns an error if the config file is not present.\nfunc (r *FSRepo) openConfig() error {\n\tconf, err := serialize.Load(r.configFilePath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tr.config = conf\n\treturn nil\n}\n\n// openUserResourceOverrides will remove all overrides if the file is not present.\n// It will error if the decoding fails.\nfunc (r *FSRepo) openUserResourceOverrides() error {\n\t// This filepath is documented in docs/libp2p-resource-management.md and be kept in sync.\n\terr := serialize.ReadConfigFile(filepath.Join(r.path, \"libp2p-resource-limit-overrides.json\"), &r.userResourceOverrides)\n\tif errors.Is(err, serialize.ErrNotInitialized) {\n\t\terr = nil\n\t}\n\treturn err\n}\n\nfunc (r *FSRepo) openKeystore() error {\n\tksp := filepath.Join(r.path, \"keystore\")\n\tks, err := keystore.NewFSKeystore(ksp)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tr.keystore = ks\n\n\treturn nil\n}\n\n// openDatastore returns an error if the config file is not present.\nfunc (r *FSRepo) openDatastore() error {\n\tif r.config.Datastore.Type != \"\" || r.config.Datastore.Path != \"\" {\n\t\treturn fmt.Errorf(\"old style datatstore config detected\")\n\t} else if r.config.Datastore.Spec == nil {\n\t\treturn fmt.Errorf(\"required Datastore.Spec entry missing from config file\")\n\t}\n\tif r.config.Datastore.NoSync {\n\t\tlog.Warn(\"NoSync is now deprecated in favor of datastore specific settings. If you want to disable fsync on flatfs set 'sync' to false. See https://github.com/ipfs/kubo/blob/master/docs/datastores.md#flatfs.\")\n\t}\n\n\tdsc, err := AnyDatastoreConfig(r.config.Datastore.Spec)\n\tif err != nil {\n\t\treturn err\n\t}\n\tspec := dsc.DiskSpec()\n\n\toldSpec, err := r.readSpec()\n\tif err != nil {\n\t\treturn err\n\t}\n\tif oldSpec != spec.String() {\n\t\treturn fmt.Errorf(\"datastore configuration of '%s' does not match what is on disk '%s'\",\n\t\t\toldSpec, spec.String())\n\t}\n\n\td, err := dsc.Create(r.path)\n\tif err != nil {\n\t\treturn err\n\t}\n\tr.ds = d\n\n\t// Wrap it with metrics gathering\n\tprefix := \"ipfs.fsrepo.datastore\"\n\tr.ds = measure.New(prefix, r.ds)\n\n\treturn nil\n}\n\nfunc (r *FSRepo) readSpec() (string, error) {\n\tfn, err := config.Path(r.path, specFn)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tb, err := os.ReadFile(fn)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn strings.TrimSpace(string(b)), nil\n}\n\n// Close closes the FSRepo, releasing held resources.\nfunc (r *FSRepo) Close() error {\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tif r.closed {\n\t\treturn errors.New(\"repo is closed\")\n\t}\n\n\terr := os.Remove(filepath.Join(r.path, apiFile))\n\tif err != nil && !os.IsNotExist(err) {\n\t\tlog.Warn(\"error removing api file: \", err)\n\t}\n\n\terr = os.Remove(filepath.Join(r.path, gatewayFile))\n\tif err != nil && !os.IsNotExist(err) {\n\t\tlog.Warn(\"error removing gateway file: \", err)\n\t}\n\n\tif err := r.ds.Close(); err != nil {\n\t\treturn err\n\t}\n\n\t// This code existed in the previous versions, but\n\t// EventlogComponent.Close was never called. Preserving here\n\t// pending further discussion.\n\t//\n\t// TODO It isn't part of the current contract, but callers may like for us\n\t// to disable logging once the component is closed.\n\t// logging.Configure(logging.Output(os.Stderr))\n\n\tr.closed = true\n\treturn r.lockfile.Close()\n}\n\n// Config the current config. This function DOES NOT copy the config. The caller\n// MUST NOT modify it without first calling `Clone`.\n//\n// Result when not Open is undefined. The method may panic if it pleases.\nfunc (r *FSRepo) Config() (*config.Config, error) {\n\t// It is not necessary to hold the package lock since the repo is in an\n\t// opened state. The package lock is _not_ meant to ensure that the repo is\n\t// thread-safe. The package lock is only meant to guard against removal and\n\t// coordinate the lockfile. However, we provide thread-safety to keep\n\t// things simple.\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tif r.closed {\n\t\treturn nil, errors.New(\"cannot access config, repo not open\")\n\t}\n\treturn r.config, nil\n}\n\nfunc (r *FSRepo) UserResourceOverrides() (rcmgr.PartialLimitConfig, error) {\n\t// It is not necessary to hold the package lock since the repo is in an\n\t// opened state. The package lock is _not_ meant to ensure that the repo is\n\t// thread-safe. The package lock is only meant to guard against removal and\n\t// coordinate the lockfile. However, we provide thread-safety to keep\n\t// things simple.\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tif r.closed {\n\t\treturn rcmgr.PartialLimitConfig{}, errors.New(\"cannot access config, repo not open\")\n\t}\n\treturn r.userResourceOverrides, nil\n}\n\nfunc (r *FSRepo) FileManager() *filestore.FileManager {\n\treturn r.filemgr\n}\n\nfunc (r *FSRepo) BackupConfig(prefix string) (string, error) {\n\ttemp, err := os.CreateTemp(r.path, \"config-\"+prefix)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tdefer temp.Close()\n\n\torig, err := os.OpenFile(r.configFilePath, os.O_RDONLY, 0o600)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tdefer orig.Close()\n\n\t_, err = io.Copy(temp, orig)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn orig.Name(), nil\n}\n\n// SetConfig updates the FSRepo's config. The user must not modify the config\n// object after calling this method.\n// FIXME: There is an inherent contradiction with storing non-user-generated\n// Go config.Config structures as user-generated JSON nested maps. This is\n// evidenced by the issue of `omitempty` property of fields that aren't defined\n// by the user and Go still needs to initialize them to its default (which\n// is not reflected in the repo's config file, see\n// https://github.com/ipfs/kubo/issues/8088 for more details).\n// In general we should call this API with a JSON nested maps as argument\n// (`map[string]interface{}`). Many calls to this function are forced to\n// synthesize the config.Config struct from their available JSON map just to\n// satisfy this (causing incompatibilities like the `omitempty` one above).\n// We need to comb SetConfig calls and replace them when possible with a\n// JSON map variant.\nfunc (r *FSRepo) SetConfig(updated *config.Config) error {\n\t// packageLock is held to provide thread-safety.\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\t// to avoid clobbering user-provided keys, must read the config from disk\n\t// as a map, write the updated struct values to the map and write the map\n\t// to disk.\n\tvar mapconf map[string]any\n\tif err := serialize.ReadConfigFile(r.configFilePath, &mapconf); err != nil {\n\t\treturn err\n\t}\n\tm, err := config.ToMap(updated)\n\tif err != nil {\n\t\treturn err\n\t}\n\tmergedMap := common.MapMergeDeep(mapconf, m)\n\tif err := serialize.WriteConfigFile(r.configFilePath, mergedMap); err != nil {\n\t\treturn err\n\t}\n\t// Do not use `*r.config = ...`. This will modify the *shared* config\n\t// returned by `r.Config`.\n\tr.config = updated\n\treturn nil\n}\n\n// GetConfigKey retrieves only the value of a particular key.\nfunc (r *FSRepo) GetConfigKey(key string) (any, error) {\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tif r.closed {\n\t\treturn nil, errors.New(\"repo is closed\")\n\t}\n\n\tvar cfg map[string]any\n\tif err := serialize.ReadConfigFile(r.configFilePath, &cfg); err != nil {\n\t\treturn nil, err\n\t}\n\treturn common.MapGetKV(cfg, key)\n}\n\n// SetConfigKey writes the value of a particular key.\nfunc (r *FSRepo) SetConfigKey(key string, value any) error {\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\tif r.closed {\n\t\treturn errors.New(\"repo is closed\")\n\t}\n\n\t// Validate the key's presence in the config structure.\n\terr := config.CheckKey(key)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Load into a map so we don't end up writing any additional defaults to the config file.\n\tvar mapconf map[string]any\n\tif err := serialize.ReadConfigFile(r.configFilePath, &mapconf); err != nil {\n\t\treturn err\n\t}\n\n\t// Load private key to guard against it being overwritten.\n\t// NOTE: this is a temporary measure to secure this field until we move\n\t// keys out of the config file.\n\tpkval, err := common.MapGetKV(mapconf, config.PrivKeySelector)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Set the key in the map.\n\tif err := common.MapSetKV(mapconf, key, value); err != nil {\n\t\treturn err\n\t}\n\n\t// replace private key, in case it was overwritten.\n\tif err := common.MapSetKV(mapconf, config.PrivKeySelector, pkval); err != nil {\n\t\treturn err\n\t}\n\n\t// This step doubles as to validate the map against the struct\n\t// before serialization\n\tconf, err := config.FromMap(mapconf)\n\tif err != nil {\n\t\treturn err\n\t}\n\tr.config = conf\n\n\tif err := serialize.WriteConfigFile(r.configFilePath, mapconf); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// Datastore returns a repo-owned datastore. If FSRepo is Closed, return value\n// is undefined.\nfunc (r *FSRepo) Datastore() repo.Datastore {\n\tpackageLock.Lock()\n\td := r.ds\n\tpackageLock.Unlock()\n\treturn d\n}\n\n// GetStorageUsage computes the storage space taken by the repo in bytes.\nfunc (r *FSRepo) GetStorageUsage(ctx context.Context) (uint64, error) {\n\treturn ds.DiskUsage(ctx, r.Datastore())\n}\n\nfunc (r *FSRepo) SwarmKey() ([]byte, error) {\n\trepoPath := filepath.Clean(r.path)\n\tspath := filepath.Join(repoPath, swarmKeyFile)\n\n\tf, err := os.Open(spath)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\terr = nil\n\t\t}\n\t\treturn nil, err\n\t}\n\tdefer f.Close()\n\n\treturn io.ReadAll(f)\n}\n\nvar (\n\t_ io.Closer = &FSRepo{}\n\t_ repo.Repo = &FSRepo{}\n)\n\n// IsInitialized returns true if the repo is initialized at provided |path|.\nfunc IsInitialized(path string) bool {\n\t// packageLock is held to ensure that another caller doesn't attempt to\n\t// Init or Remove the repo while this call is in progress.\n\tpackageLock.Lock()\n\tdefer packageLock.Unlock()\n\n\treturn isInitializedUnsynced(path)\n}\n\n// private methods below this point. NB: packageLock must held by caller.\n\n// isInitializedUnsynced reports whether the repo is initialized. Caller must\n// hold the packageLock.\nfunc isInitializedUnsynced(repoPath string) bool {\n\treturn configIsInitialized(repoPath)\n}\n" }, { "path": "repo/fsrepo/fsrepo_test.go", "content": "package fsrepo\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\tdatastore \"github.com/ipfs/go-datastore\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestInitIdempotence(t *testing.T) {\n\tt.Parallel()\n\tpath := t.TempDir()\n\tfor range 10 {\n\t\trequire.NoError(t, Init(path, &config.Config{Datastore: config.DefaultDatastoreConfig()}), \"multiple calls to init should succeed\")\n\t}\n}\n\nfunc Remove(repoPath string) error {\n\trepoPath = filepath.Clean(repoPath)\n\treturn os.RemoveAll(repoPath)\n}\n\nfunc TestCanManageReposIndependently(t *testing.T) {\n\tt.Parallel()\n\tpathA := t.TempDir()\n\tpathB := t.TempDir()\n\n\tt.Log(\"initialize two repos\")\n\trequire.NoError(t, Init(pathA, &config.Config{Datastore: config.DefaultDatastoreConfig()}), \"a\", \"should initialize successfully\")\n\trequire.NoError(t, Init(pathB, &config.Config{Datastore: config.DefaultDatastoreConfig()}), \"b\", \"should initialize successfully\")\n\n\tt.Log(\"ensure repos initialized\")\n\trequire.True(t, IsInitialized(pathA), \"a should be initialized\")\n\trequire.True(t, IsInitialized(pathB), \"b should be initialized\")\n\n\tt.Log(\"open the two repos\")\n\trepoA, err := Open(pathA)\n\trequire.NoError(t, err, \"a\")\n\trepoB, err := Open(pathB)\n\trequire.NoError(t, err, \"b\")\n\n\tt.Log(\"close and remove b while a is open\")\n\trequire.NoError(t, repoB.Close(), \"close b\")\n\trequire.NoError(t, Remove(pathB), \"remove b\")\n\n\tt.Log(\"close and remove a\")\n\trequire.NoError(t, repoA.Close())\n\trequire.NoError(t, Remove(pathA))\n}\n\nfunc TestDatastoreGetNotAllowedAfterClose(t *testing.T) {\n\tt.Parallel()\n\tpath := t.TempDir()\n\n\trequire.False(t, IsInitialized(path), \"should NOT be initialized\")\n\trequire.NoError(t, Init(path, &config.Config{Datastore: config.DefaultDatastoreConfig()}), \"should initialize successfully\")\n\tr, err := Open(path)\n\trequire.NoError(t, err, \"should open successfully\")\n\n\tk := \"key\"\n\tdata := []byte(k)\n\trequire.NoError(t, r.Datastore().Put(context.Background(), datastore.NewKey(k), data), \"Put should be successful\")\n\n\trequire.NoError(t, r.Close())\n\t_, err = r.Datastore().Get(context.Background(), datastore.NewKey(k))\n\trequire.Error(t, err, \"after closer, Get should be fail\")\n}\n\nfunc TestDatastorePersistsFromRepoToRepo(t *testing.T) {\n\tt.Parallel()\n\tpath := t.TempDir()\n\n\trequire.NoError(t, Init(path, &config.Config{Datastore: config.DefaultDatastoreConfig()}))\n\tr1, err := Open(path)\n\trequire.NoError(t, err)\n\n\tk := \"key\"\n\texpected := []byte(k)\n\trequire.NoError(t, r1.Datastore().Put(context.Background(), datastore.NewKey(k), expected), \"using first repo, Put should be successful\")\n\trequire.NoError(t, r1.Close())\n\n\tr2, err := Open(path)\n\trequire.NoError(t, err)\n\tactual, err := r2.Datastore().Get(context.Background(), datastore.NewKey(k))\n\trequire.NoError(t, err, \"using second repo, Get should be successful\")\n\trequire.NoError(t, r2.Close())\n\trequire.True(t, bytes.Equal(expected, actual), \"data should match\")\n}\n\nfunc TestOpenMoreThanOnceInSameProcess(t *testing.T) {\n\tt.Parallel()\n\tpath := t.TempDir()\n\trequire.NoError(t, Init(path, &config.Config{Datastore: config.DefaultDatastoreConfig()}))\n\n\tr1, err := Open(path)\n\trequire.NoError(t, err, \"first repo should open successfully\")\n\tr2, err := Open(path)\n\trequire.NoError(t, err, \"second repo should open successfully\")\n\trequire.Equal(t, r1, r2, \"second open returns same value\")\n\n\trequire.NoError(t, r1.Close())\n\trequire.NoError(t, r2.Close())\n}\n" }, { "path": "repo/fsrepo/migrations/README.md", "content": "# IPFS Repository Migrations\n\nThis directory contains the migration system for IPFS repositories, handling both embedded and external migrations.\n\n## Migration System Overview\n\n### Embedded vs External Migrations\n\nStarting from **repo version 17**, Kubo uses **embedded migrations** that are built into the binary, eliminating the need to download external migration tools.\n\n- **Repo versions <17**: Use external binary migrations downloaded from fs-repo-migrations\n- **Repo version 17+**: Use embedded migrations built into Kubo\n\n### Migration Functions\n\n#### `migrations.RunEmbeddedMigrations()`\n- **Purpose**: Runs migrations that are embedded directly in the Kubo binary\n- **Scope**: Handles repo version 17+ migrations\n- **Performance**: Fast execution, no network downloads required\n- **Dependencies**: Self-contained, uses only Kubo's internal dependencies\n- **Usage**: Primary migration method for modern repo versions\n\n**Parameters**:\n- `ctx`: Context for cancellation and timeouts\n- `targetVersion`: Target repository version to migrate to\n- `repoPath`: Path to the IPFS repository directory\n- `allowDowngrade`: Whether to allow downgrade migrations\n\n```go\nerr = migrations.RunEmbeddedMigrations(ctx, targetVersion, repoPath, allowDowngrade)\nif err != nil {\n // Handle migration failure, may fall back to external migrations\n}\n```\n\n#### `migrations.RunMigration()` with `migrations.ReadMigrationConfig()`\n- **Purpose**: Runs external binary migrations downloaded from fs-repo-migrations\n- **Scope**: Handles legacy repo versions <17 and serves as fallback\n- **Performance**: Slower due to network downloads and external process execution\n- **Dependencies**: Requires fs-repo-migrations binaries and network access\n- **Usage**: Fallback method for legacy migrations\n\n```go\n// Read migration configuration for external migrations\nmigrationCfg, err := migrations.ReadMigrationConfig(repoPath, configFile)\nfetcher, err := migrations.GetMigrationFetcher(migrationCfg.DownloadSources, ...)\nerr = migrations.RunMigration(ctx, fetcher, targetVersion, repoPath, allowDowngrade)\n```\n\n## Migration Flow in Daemon Startup\n\n1. **Primary**: Try embedded migrations first (`RunEmbeddedMigrations`)\n2. **Fallback**: If embedded migration fails, fall back to external migrations (`RunMigration`)\n3. **Legacy Support**: External migrations ensure compatibility with older repo versions\n\n## Directory Structure\n\n```\nrepo/fsrepo/migrations/\n├── README.md # This file\n├── embedded.go # Embedded migration system\n├── embedded_test.go # Tests for embedded migrations\n├── migrations.go # External migration system\n├── fs-repo-16-to-17/ # First embedded migration (16→17)\n│ ├── migration/\n│ │ ├── migration.go # Migration logic\n│ │ └── migration_test.go # Migration tests\n│ ├── atomicfile/\n│ │ └── atomicfile.go # Atomic file operations\n│ ├── main.go # Standalone migration binary\n│ └── README.md # Migration-specific documentation\n└── [other migration utilities]\n```\n\n## Adding New Embedded Migrations\n\nTo add a new embedded migration (e.g., fs-repo-17-to-18):\n\n1. **Create migration package**: `fs-repo-17-to-18/migration/migration.go`\n2. **Implement interface**: Ensure your migration implements the `EmbeddedMigration` interface\n3. **Register migration**: Add to `embeddedMigrations` map in `embedded.go`\n4. **Add tests**: Create comprehensive tests for your migration logic\n5. **Update repo version**: Increment `RepoVersion` in `fsrepo.go`\n\n```go\n// In embedded.go\nvar embeddedMigrations = map[string]EmbeddedMigration{\n \"fs-repo-16-to-17\": &mg16.Migration{},\n \"fs-repo-17-to-18\": &mg17.Migration{}, // Add new migration\n}\n```\n\n## Migration Requirements\n\nEach embedded migration must:\n- Implement the `EmbeddedMigration` interface\n- Be reversible with proper backup handling\n- Use atomic file operations to prevent corruption\n- Preserve user customizations\n- Include comprehensive tests\n- Follow the established naming pattern\n\n## External Migration Support\n\nExternal migrations are maintained for:\n- **Backward compatibility** with repo versions <17\n- **Fallback mechanism** if embedded migrations fail\n- **Legacy installations** that cannot be upgraded directly\n\nThe external migration system will continue to work but is not the preferred method for new migrations.\n\n## Security and Safety\n\nAll migrations (embedded and external) include:\n- **Atomic operations**: Prevent repository corruption\n- **Backup creation**: Allow rollback if migration fails\n- **Version validation**: Ensure migrations run on correct repo versions\n- **Error handling**: Graceful failure with informative messages\n- **User preservation**: Maintain custom configurations during migration\n\n## Testing\n\nTest both embedded and external migration systems:\n\n```bash\n# Test embedded migrations\ngo test ./repo/fsrepo/migrations/ -run TestEmbedded\n\n# Test specific migration\ngo test ./repo/fsrepo/migrations/fs-repo-16-to-17/migration/\n\n# Test migration registration\ngo test ./repo/fsrepo/migrations/ -run TestHasEmbedded\n```" }, { "path": "repo/fsrepo/migrations/atomicfile/atomicfile.go", "content": "package atomicfile\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n)\n\n// File represents an atomic file writer\ntype File struct {\n\t*os.File\n\tpath string\n}\n\n// New creates a new atomic file writer\nfunc New(path string, mode os.FileMode) (*File, error) {\n\tdir := filepath.Dir(path)\n\ttempFile, err := os.CreateTemp(dir, \".tmp-\"+filepath.Base(path))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := tempFile.Chmod(mode); err != nil {\n\t\ttempFile.Close()\n\t\tos.Remove(tempFile.Name())\n\t\treturn nil, err\n\t}\n\n\treturn &File{\n\t\tFile: tempFile,\n\t\tpath: path,\n\t}, nil\n}\n\n// Close atomically replaces the target file with the temporary file\nfunc (f *File) Close() error {\n\tcloseErr := f.File.Close()\n\tif closeErr != nil {\n\t\t// Try to cleanup temp file, but prioritize close error\n\t\t_ = os.Remove(f.File.Name())\n\t\treturn closeErr\n\t}\n\treturn os.Rename(f.File.Name(), f.path)\n}\n\n// Abort removes the temporary file without replacing the target\nfunc (f *File) Abort() error {\n\tcloseErr := f.File.Close()\n\tremoveErr := os.Remove(f.File.Name())\n\n\tif closeErr != nil && removeErr != nil {\n\t\treturn fmt.Errorf(\"abort failed: close: %w, remove: %v\", closeErr, removeErr)\n\t}\n\tif closeErr != nil {\n\t\treturn closeErr\n\t}\n\treturn removeErr\n}\n\n// ReadFrom reads from the given reader into the atomic file\nfunc (f *File) ReadFrom(r io.Reader) (int64, error) {\n\treturn io.Copy(f.File, r)\n}\n" }, { "path": "repo/fsrepo/migrations/atomicfile/atomicfile_test.go", "content": "package atomicfile\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestNew_Success verifies atomic file creation\nfunc TestNew_Success(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\tdefer func() { _ = af.Abort() }()\n\n\t// Verify temp file exists\n\tassert.FileExists(t, af.File.Name())\n\n\t// Verify temp file is in same directory\n\tassert.Equal(t, dir, filepath.Dir(af.File.Name()))\n}\n\n// TestClose_Success verifies atomic replacement\nfunc TestClose_Success(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\n\tcontent := []byte(\"test content\")\n\t_, err = af.Write(content)\n\trequire.NoError(t, err)\n\n\ttempName := af.File.Name()\n\n\trequire.NoError(t, af.Close())\n\n\t// Verify target file exists with correct content\n\tdata, err := os.ReadFile(path)\n\trequire.NoError(t, err)\n\tassert.Equal(t, content, data)\n\n\t// Verify temp file removed\n\tassert.NoFileExists(t, tempName)\n}\n\n// TestAbort_Success verifies cleanup\nfunc TestAbort_Success(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\n\ttempName := af.File.Name()\n\n\trequire.NoError(t, af.Abort())\n\n\t// Verify temp file removed\n\tassert.NoFileExists(t, tempName)\n\n\t// Verify target not created\n\tassert.NoFileExists(t, path)\n}\n\n// TestAbort_ErrorHandling tests error capture\nfunc TestAbort_ErrorHandling(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\n\t// Close file to force close error\n\taf.File.Close()\n\n\t// Remove temp file to force remove error\n\tos.Remove(af.File.Name())\n\n\terr = af.Abort()\n\t// Should get both errors\n\trequire.Error(t, err)\n\tassert.Contains(t, err.Error(), \"abort failed\")\n}\n\n// TestClose_CloseError verifies cleanup on close failure\nfunc TestClose_CloseError(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\n\ttempName := af.File.Name()\n\n\t// Close file to force close error\n\taf.File.Close()\n\n\terr = af.Close()\n\trequire.Error(t, err)\n\n\t// Verify temp file cleaned up even on error\n\tassert.NoFileExists(t, tempName)\n}\n\n// TestReadFrom verifies io.Copy integration\nfunc TestReadFrom(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\tdefer func() { _ = af.Abort() }()\n\n\tcontent := []byte(\"test content from reader\")\n\tn, err := af.ReadFrom(bytes.NewReader(content))\n\trequire.NoError(t, err)\n\tassert.Equal(t, int64(len(content)), n)\n}\n\n// TestFilePermissions verifies mode is set correctly\nfunc TestFilePermissions(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0600)\n\trequire.NoError(t, err)\n\n\t_, err = af.Write([]byte(\"test\"))\n\trequire.NoError(t, err)\n\n\trequire.NoError(t, af.Close())\n\n\tinfo, err := os.Stat(path)\n\trequire.NoError(t, err)\n\n\t// On Unix, check exact permissions\n\tif runtime.GOOS != \"windows\" {\n\t\tmode := info.Mode().Perm()\n\t\tassert.Equal(t, os.FileMode(0600), mode)\n\t}\n}\n\n// TestMultipleAbortsSafe verifies calling Abort multiple times is safe\nfunc TestMultipleAbortsSafe(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"test.txt\")\n\n\taf, err := New(path, 0644)\n\trequire.NoError(t, err)\n\n\ttempName := af.File.Name()\n\n\t// First abort should succeed\n\trequire.NoError(t, af.Abort())\n\tassert.NoFileExists(t, tempName, \"temp file should be removed after first abort\")\n\n\t// Second abort should handle gracefully (file already gone)\n\terr = af.Abort()\n\t// Error is acceptable since file is already removed, but it should not panic\n\tt.Logf(\"Second Abort() returned: %v\", err)\n}\n\n// TestNoTempFilesAfterOperations verifies no .tmp-* files remain after operations\nfunc TestNoTempFilesAfterOperations(t *testing.T) {\n\tconst testIterations = 5\n\n\ttests := []struct {\n\t\tname string\n\t\toperation func(*File) error\n\t}{\n\t\t{\"close\", (*File).Close},\n\t\t{\"abort\", (*File).Abort},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tdir := t.TempDir()\n\n\t\t\t// Perform multiple operations\n\t\t\tfor i := range testIterations {\n\t\t\t\tpath := filepath.Join(dir, fmt.Sprintf(\"test%d.txt\", i))\n\n\t\t\t\taf, err := New(path, 0644)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t_, err = af.Write([]byte(\"test data\"))\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\trequire.NoError(t, tt.operation(af))\n\t\t\t}\n\n\t\t\t// Check for any .tmp-* files\n\t\t\ttmpFiles, err := filepath.Glob(filepath.Join(dir, \".tmp-*\"))\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Empty(t, tmpFiles, \"should be no temp files after %s\", tt.name)\n\t\t})\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/common/base.go", "content": "package common\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"path/filepath\"\n)\n\n// BaseMigration provides common functionality for migrations\ntype BaseMigration struct {\n\tFromVersion string\n\tToVersion string\n\tDescription string\n\tConvert func(in io.ReadSeeker, out io.Writer) error\n}\n\n// Versions returns the version string for this migration\nfunc (m *BaseMigration) Versions() string {\n\treturn fmt.Sprintf(\"%s-to-%s\", m.FromVersion, m.ToVersion)\n}\n\n// configBackupSuffix returns the backup suffix for the config file\n// e.g. \".16-to-17.bak\" results in \"config.16-to-17.bak\"\nfunc (m *BaseMigration) configBackupSuffix() string {\n\treturn fmt.Sprintf(\".%s-to-%s.bak\", m.FromVersion, m.ToVersion)\n}\n\n// Reversible returns true as we keep backups\nfunc (m *BaseMigration) Reversible() bool {\n\treturn true\n}\n\n// Apply performs the migration\nfunc (m *BaseMigration) Apply(opts Options) error {\n\tif opts.Verbose {\n\t\tfmt.Printf(\"applying %s repo migration\\n\", m.Versions())\n\t\tif m.Description != \"\" {\n\t\t\tfmt.Printf(\"> %s\\n\", m.Description)\n\t\t}\n\t}\n\n\t// Check version\n\tif err := CheckVersion(opts.Path, m.FromVersion); err != nil {\n\t\treturn err\n\t}\n\n\tconfigPath := filepath.Join(opts.Path, \"config\")\n\n\t// Perform migration with backup\n\tif err := WithBackup(configPath, m.configBackupSuffix(), m.Convert); err != nil {\n\t\treturn err\n\t}\n\n\t// Update version\n\tif err := WriteVersion(opts.Path, m.ToVersion); err != nil {\n\t\tif opts.Verbose {\n\t\t\tfmt.Printf(\"failed to update version file to %s\\n\", m.ToVersion)\n\t\t}\n\t\treturn err\n\t}\n\n\tif opts.Verbose {\n\t\tfmt.Println(\"updated version file\")\n\t\tfmt.Printf(\"Migration %s succeeded\\n\", m.Versions())\n\t}\n\n\treturn nil\n}\n\n// Revert reverts the migration\nfunc (m *BaseMigration) Revert(opts Options) error {\n\tif opts.Verbose {\n\t\tfmt.Println(\"reverting migration\")\n\t}\n\n\t// Check we're at the expected version\n\tif err := CheckVersion(opts.Path, m.ToVersion); err != nil {\n\t\treturn err\n\t}\n\n\t// Restore backup\n\tconfigPath := filepath.Join(opts.Path, \"config\")\n\tif err := RevertBackup(configPath, m.configBackupSuffix()); err != nil {\n\t\treturn err\n\t}\n\n\t// Revert version\n\tif err := WriteVersion(opts.Path, m.FromVersion); err != nil {\n\t\treturn err\n\t}\n\n\tif opts.Verbose {\n\t\tfmt.Printf(\"lowered version number to %s\\n\", m.FromVersion)\n\t}\n\n\treturn nil\n}\n" }, { "path": "repo/fsrepo/migrations/common/config_helpers.go", "content": "package common\n\nimport (\n\t\"fmt\"\n\t\"maps\"\n\t\"slices\"\n\t\"strings\"\n)\n\n// GetField retrieves a field from a nested config structure using a dot-separated path\n// Example: GetField(config, \"DNS.Resolvers\") returns config[\"DNS\"][\"Resolvers\"]\nfunc GetField(config map[string]any, path string) (any, bool) {\n\tparts := strings.Split(path, \".\")\n\tcurrent := config\n\n\tfor i, part := range parts {\n\t\t// Last part - return the value\n\t\tif i == len(parts)-1 {\n\t\t\tval, exists := current[part]\n\t\t\treturn val, exists\n\t\t}\n\n\t\t// Navigate deeper\n\t\tnext, exists := current[part]\n\t\tif !exists {\n\t\t\treturn nil, false\n\t\t}\n\n\t\t// Ensure it's a map\n\t\tnextMap, ok := next.(map[string]any)\n\t\tif !ok {\n\t\t\treturn nil, false\n\t\t}\n\t\tcurrent = nextMap\n\t}\n\n\treturn nil, false\n}\n\n// SetField sets a field in a nested config structure using a dot-separated path\n// It creates intermediate maps as needed\nfunc SetField(config map[string]any, path string, value any) {\n\tparts := strings.Split(path, \".\")\n\tcurrent := config\n\n\tfor i, part := range parts {\n\t\t// Last part - set the value\n\t\tif i == len(parts)-1 {\n\t\t\tcurrent[part] = value\n\t\t\treturn\n\t\t}\n\n\t\t// Navigate or create intermediate maps\n\t\tnext, exists := current[part]\n\t\tif !exists {\n\t\t\t// Create new intermediate map\n\t\t\tnewMap := make(map[string]any)\n\t\t\tcurrent[part] = newMap\n\t\t\tcurrent = newMap\n\t\t} else {\n\t\t\t// Ensure it's a map\n\t\t\tnextMap, ok := next.(map[string]any)\n\t\t\tif !ok {\n\t\t\t\t// Can't navigate further, replace with new map\n\t\t\t\tnewMap := make(map[string]any)\n\t\t\t\tcurrent[part] = newMap\n\t\t\t\tcurrent = newMap\n\t\t\t} else {\n\t\t\t\tcurrent = nextMap\n\t\t\t}\n\t\t}\n\t}\n}\n\n// DeleteField removes a field from a nested config structure\nfunc DeleteField(config map[string]any, path string) bool {\n\tparts := strings.Split(path, \".\")\n\n\t// Handle simple case\n\tif len(parts) == 1 {\n\t\t_, exists := config[parts[0]]\n\t\tdelete(config, parts[0])\n\t\treturn exists\n\t}\n\n\t// Navigate to parent\n\tparentPath := strings.Join(parts[:len(parts)-1], \".\")\n\tparent, exists := GetField(config, parentPath)\n\tif !exists {\n\t\treturn false\n\t}\n\n\tparentMap, ok := parent.(map[string]any)\n\tif !ok {\n\t\treturn false\n\t}\n\n\tfieldName := parts[len(parts)-1]\n\t_, exists = parentMap[fieldName]\n\tdelete(parentMap, fieldName)\n\treturn exists\n}\n\n// MoveField moves a field from one location to another\nfunc MoveField(config map[string]any, from, to string) error {\n\tvalue, exists := GetField(config, from)\n\tif !exists {\n\t\treturn fmt.Errorf(\"source field %s does not exist\", from)\n\t}\n\n\tSetField(config, to, value)\n\tDeleteField(config, from)\n\treturn nil\n}\n\n// RenameField renames a field within the same parent\nfunc RenameField(config map[string]any, path, oldName, newName string) error {\n\tvar parent map[string]any\n\tif path == \"\" {\n\t\tparent = config\n\t} else {\n\t\tp, exists := GetField(config, path)\n\t\tif !exists {\n\t\t\treturn fmt.Errorf(\"parent path %s does not exist\", path)\n\t\t}\n\t\tvar ok bool\n\t\tparent, ok = p.(map[string]any)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"parent path %s is not a map\", path)\n\t\t}\n\t}\n\n\tvalue, exists := parent[oldName]\n\tif !exists {\n\t\treturn fmt.Errorf(\"field %s does not exist\", oldName)\n\t}\n\n\tparent[newName] = value\n\tdelete(parent, oldName)\n\treturn nil\n}\n\n// SetDefault sets a field value only if it doesn't already exist\nfunc SetDefault(config map[string]any, path string, value any) {\n\tif _, exists := GetField(config, path); !exists {\n\t\tSetField(config, path, value)\n\t}\n}\n\n// TransformField applies a transformation function to a field value\nfunc TransformField(config map[string]any, path string, transformer func(any) any) error {\n\tvalue, exists := GetField(config, path)\n\tif !exists {\n\t\treturn fmt.Errorf(\"field %s does not exist\", path)\n\t}\n\n\tnewValue := transformer(value)\n\tSetField(config, path, newValue)\n\treturn nil\n}\n\n// EnsureFieldIs checks if a field equals expected value, sets it if missing\nfunc EnsureFieldIs(config map[string]any, path string, expected any) {\n\tcurrent, exists := GetField(config, path)\n\tif !exists || current != expected {\n\t\tSetField(config, path, expected)\n\t}\n}\n\n// MergeInto merges multiple source fields into a destination map\nfunc MergeInto(config map[string]any, destination string, sources ...string) {\n\tvar destMap map[string]any\n\n\t// Get existing destination if it exists\n\tif existing, exists := GetField(config, destination); exists {\n\t\tif m, ok := existing.(map[string]any); ok {\n\t\t\tdestMap = m\n\t\t}\n\t}\n\n\t// Merge each source\n\tfor _, source := range sources {\n\t\tif value, exists := GetField(config, source); exists {\n\t\t\tif sourceMap, ok := value.(map[string]any); ok {\n\t\t\t\tif destMap == nil {\n\t\t\t\t\tdestMap = make(map[string]any)\n\t\t\t\t}\n\t\t\t\tmaps.Copy(destMap, sourceMap)\n\t\t\t}\n\t\t}\n\t}\n\n\tif destMap != nil {\n\t\tSetField(config, destination, destMap)\n\t}\n}\n\n// CopyField copies a field value to a new location (keeps original)\nfunc CopyField(config map[string]any, from, to string) error {\n\tvalue, exists := GetField(config, from)\n\tif !exists {\n\t\treturn fmt.Errorf(\"source field %s does not exist\", from)\n\t}\n\n\tSetField(config, to, value)\n\treturn nil\n}\n\n// ConvertInterfaceSlice converts []interface{} to []string\nfunc ConvertInterfaceSlice(slice []any) []string {\n\tresult := make([]string, 0, len(slice))\n\tfor _, item := range slice {\n\t\tif str, ok := item.(string); ok {\n\t\t\tresult = append(result, str)\n\t\t}\n\t}\n\treturn result\n}\n\n// GetOrCreateSection gets or creates a map section in config\nfunc GetOrCreateSection(config map[string]any, path string) map[string]any {\n\texisting, exists := GetField(config, path)\n\tif exists {\n\t\tif section, ok := existing.(map[string]any); ok {\n\t\t\treturn section\n\t\t}\n\t}\n\n\t// Create new section\n\tsection := make(map[string]any)\n\tSetField(config, path, section)\n\treturn section\n}\n\n// SafeCastMap safely casts to map[string]any with fallback to empty map\nfunc SafeCastMap(value any) map[string]any {\n\tif m, ok := value.(map[string]any); ok {\n\t\treturn m\n\t}\n\treturn make(map[string]any)\n}\n\n// SafeCastSlice safely casts to []interface{} with fallback to empty slice\nfunc SafeCastSlice(value any) []any {\n\tif s, ok := value.([]any); ok {\n\t\treturn s\n\t}\n\treturn []any{}\n}\n\n// ReplaceDefaultsWithAuto replaces default values with \"auto\" in a map\nfunc ReplaceDefaultsWithAuto(values map[string]any, defaults map[string]string) map[string]string {\n\tresult := make(map[string]string)\n\tfor k, v := range values {\n\t\tif vStr, ok := v.(string); ok {\n\t\t\tif replacement, isDefault := defaults[vStr]; isDefault {\n\t\t\t\tresult[k] = replacement\n\t\t\t} else {\n\t\t\t\tresult[k] = vStr\n\t\t\t}\n\t\t}\n\t}\n\treturn result\n}\n\n// EnsureSliceContains ensures a slice field contains a value\nfunc EnsureSliceContains(config map[string]any, path string, value string) {\n\texisting, exists := GetField(config, path)\n\tif !exists {\n\t\tSetField(config, path, []string{value})\n\t\treturn\n\t}\n\n\tif slice, ok := existing.([]any); ok {\n\t\t// Check if value already exists\n\t\tfor _, item := range slice {\n\t\t\tif str, ok := item.(string); ok && str == value {\n\t\t\t\treturn // Already contains value\n\t\t\t}\n\t\t}\n\t\t// Add value\n\t\tSetField(config, path, append(slice, value))\n\t} else if strSlice, ok := existing.([]string); ok {\n\t\tif !slices.Contains(strSlice, value) {\n\t\t\tSetField(config, path, append(strSlice, value))\n\t\t}\n\t} else {\n\t\t// Replace with new slice containing value\n\t\tSetField(config, path, []string{value})\n\t}\n}\n\n// ReplaceInSlice replaces old values with new in a slice field\nfunc ReplaceInSlice(config map[string]any, path string, oldValue, newValue string) {\n\texisting, exists := GetField(config, path)\n\tif !exists {\n\t\treturn\n\t}\n\n\tif slice, ok := existing.([]any); ok {\n\t\tresult := make([]string, 0, len(slice))\n\t\tfor _, item := range slice {\n\t\t\tif str, ok := item.(string); ok {\n\t\t\t\tif str == oldValue {\n\t\t\t\t\tresult = append(result, newValue)\n\t\t\t\t} else {\n\t\t\t\t\tresult = append(result, str)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tSetField(config, path, result)\n\t}\n}\n\n// GetMapSection gets a map section with error handling\nfunc GetMapSection(config map[string]any, path string) (map[string]any, error) {\n\tvalue, exists := GetField(config, path)\n\tif !exists {\n\t\treturn nil, fmt.Errorf(\"section %s does not exist\", path)\n\t}\n\n\tsection, ok := value.(map[string]any)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"section %s is not a map\", path)\n\t}\n\n\treturn section, nil\n}\n\n// CloneStringMap clones a map[string]any to map[string]string\nfunc CloneStringMap(m map[string]any) map[string]string {\n\tresult := make(map[string]string, len(m))\n\tfor k, v := range m {\n\t\tif str, ok := v.(string); ok {\n\t\t\tresult[k] = str\n\t\t}\n\t}\n\treturn result\n}\n\n// IsEmptySlice checks if a value is an empty slice\nfunc IsEmptySlice(value any) bool {\n\tif value == nil {\n\t\treturn true\n\t}\n\tif slice, ok := value.([]any); ok {\n\t\treturn len(slice) == 0\n\t}\n\tif slice, ok := value.([]string); ok {\n\t\treturn len(slice) == 0\n\t}\n\treturn false\n}\n" }, { "path": "repo/fsrepo/migrations/common/migration.go", "content": "// Package common contains common types and interfaces for file system repository migrations\npackage common\n\n// Options contains migration options for embedded migrations\ntype Options struct {\n\tPath string\n\tVerbose bool\n}\n\n// Migration is the interface that all migrations must implement\ntype Migration interface {\n\tVersions() string\n\tApply(opts Options) error\n\tRevert(opts Options) error\n\tReversible() bool\n}\n" }, { "path": "repo/fsrepo/migrations/common/testing_helpers.go", "content": "package common\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"maps\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"reflect\"\n\t\"testing\"\n)\n\n// TestCase represents a single migration test case\ntype TestCase struct {\n\tName string\n\tInputConfig map[string]any\n\tAssertions []ConfigAssertion\n}\n\n// ConfigAssertion represents an assertion about the migrated config\ntype ConfigAssertion struct {\n\tPath string\n\tExpected any\n}\n\n// RunMigrationTest runs a migration test with the given test case\nfunc RunMigrationTest(t *testing.T, migration Migration, tc TestCase) {\n\tt.Helper()\n\n\t// Convert input to JSON\n\tinputJSON, err := json.MarshalIndent(tc.InputConfig, \"\", \" \")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to marshal input config: %v\", err)\n\t}\n\n\t// Run the migration's convert function\n\tvar output bytes.Buffer\n\tif baseMig, ok := migration.(*BaseMigration); ok {\n\t\terr = baseMig.Convert(bytes.NewReader(inputJSON), &output)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"migration failed: %v\", err)\n\t\t}\n\t} else {\n\t\tt.Skip(\"migration is not a BaseMigration\")\n\t}\n\n\t// Parse output\n\tvar result map[string]any\n\terr = json.Unmarshal(output.Bytes(), &result)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to unmarshal output: %v\", err)\n\t}\n\n\t// Run assertions\n\tfor _, assertion := range tc.Assertions {\n\t\tAssertConfigField(t, result, assertion.Path, assertion.Expected)\n\t}\n}\n\n// AssertConfigField asserts that a field in the config has the expected value\nfunc AssertConfigField(t *testing.T, config map[string]any, path string, expected any) {\n\tt.Helper()\n\n\tactual, exists := GetField(config, path)\n\tif expected == nil {\n\t\tif exists {\n\t\t\tt.Errorf(\"expected field %s to not exist, but it has value: %v\", path, actual)\n\t\t}\n\t\treturn\n\t}\n\n\tif !exists {\n\t\tt.Errorf(\"expected field %s to exist with value %v, but it doesn't exist\", path, expected)\n\t\treturn\n\t}\n\n\t// Handle different types of comparisons\n\tswitch exp := expected.(type) {\n\tcase []string:\n\t\tactualSlice, ok := actual.([]any)\n\t\tif !ok {\n\t\t\tt.Errorf(\"field %s: expected []string, got %T\", path, actual)\n\t\t\treturn\n\t\t}\n\t\tif len(exp) != len(actualSlice) {\n\t\t\tt.Errorf(\"field %s: expected slice of length %d, got %d\", path, len(exp), len(actualSlice))\n\t\t\treturn\n\t\t}\n\t\tfor i, expVal := range exp {\n\t\t\tif actualSlice[i] != expVal {\n\t\t\t\tt.Errorf(\"field %s[%d]: expected %v, got %v\", path, i, expVal, actualSlice[i])\n\t\t\t}\n\t\t}\n\tcase map[string]string:\n\t\tactualMap, ok := actual.(map[string]any)\n\t\tif !ok {\n\t\t\tt.Errorf(\"field %s: expected map, got %T\", path, actual)\n\t\t\treturn\n\t\t}\n\t\tfor k, v := range exp {\n\t\t\tif actualMap[k] != v {\n\t\t\t\tt.Errorf(\"field %s[%s]: expected %v, got %v\", path, k, v, actualMap[k])\n\t\t\t}\n\t\t}\n\tdefault:\n\t\tif actual != expected {\n\t\t\tt.Errorf(\"field %s: expected %v, got %v\", path, expected, actual)\n\t\t}\n\t}\n}\n\n// GenerateTestConfig creates a basic test config with the given fields\nfunc GenerateTestConfig(fields map[string]any) map[string]any {\n\t// Start with a minimal valid config\n\tconfig := map[string]any{\n\t\t\"Identity\": map[string]any{\n\t\t\t\"PeerID\": \"QmTest\",\n\t\t},\n\t}\n\n\t// Merge in the provided fields\n\tmaps.Copy(config, fields)\n\n\treturn config\n}\n\n// CreateTestRepo creates a temporary test repository with the given version and config\nfunc CreateTestRepo(t *testing.T, version int, config map[string]any) string {\n\tt.Helper()\n\n\ttempDir := t.TempDir()\n\n\t// Write version file\n\tversionPath := filepath.Join(tempDir, \"version\")\n\terr := os.WriteFile(versionPath, fmt.Appendf(nil, \"%d\", version), 0644)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to write version file: %v\", err)\n\t}\n\n\t// Write config file\n\tconfigPath := filepath.Join(tempDir, \"config\")\n\tconfigData, err := json.MarshalIndent(config, \"\", \" \")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to marshal config: %v\", err)\n\t}\n\terr = os.WriteFile(configPath, configData, 0644)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to write config file: %v\", err)\n\t}\n\n\treturn tempDir\n}\n\n// AssertMigrationSuccess runs a full migration and checks that it succeeds\nfunc AssertMigrationSuccess(t *testing.T, migration Migration, fromVersion, toVersion int, inputConfig map[string]any) map[string]any {\n\tt.Helper()\n\n\t// Create test repo\n\trepoPath := CreateTestRepo(t, fromVersion, inputConfig)\n\n\t// Run migration\n\topts := Options{\n\t\tPath: repoPath,\n\t\tVerbose: false,\n\t}\n\n\terr := migration.Apply(opts)\n\tif err != nil {\n\t\tt.Fatalf(\"migration failed: %v\", err)\n\t}\n\n\t// Check version was updated\n\tversionBytes, err := os.ReadFile(filepath.Join(repoPath, \"version\"))\n\tif err != nil {\n\t\tt.Fatalf(\"failed to read version file: %v\", err)\n\t}\n\tactualVersion := string(versionBytes)\n\tif actualVersion != fmt.Sprintf(\"%d\", toVersion) {\n\t\tt.Errorf(\"expected version %d, got %s\", toVersion, actualVersion)\n\t}\n\n\t// Read and return the migrated config\n\tconfigBytes, err := os.ReadFile(filepath.Join(repoPath, \"config\"))\n\tif err != nil {\n\t\tt.Fatalf(\"failed to read config file: %v\", err)\n\t}\n\n\tvar result map[string]any\n\terr = json.Unmarshal(configBytes, &result)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to unmarshal config: %v\", err)\n\t}\n\n\treturn result\n}\n\n// AssertMigrationReversible checks that a migration can be reverted\nfunc AssertMigrationReversible(t *testing.T, migration Migration, fromVersion, toVersion int, inputConfig map[string]any) {\n\tt.Helper()\n\n\t// Create test repo at target version\n\trepoPath := CreateTestRepo(t, toVersion, inputConfig)\n\n\t// Create backup file (simulating a previous migration)\n\tbackupPath := filepath.Join(repoPath, fmt.Sprintf(\"config.%d-to-%d.bak\", fromVersion, toVersion))\n\toriginalConfig, err := json.MarshalIndent(inputConfig, \"\", \" \")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to marshal original config: %v\", err)\n\t}\n\n\tif err := os.WriteFile(backupPath, originalConfig, 0644); err != nil {\n\t\tt.Fatalf(\"failed to write backup file: %v\", err)\n\t}\n\n\t// Run revert\n\tif err := migration.Revert(Options{Path: repoPath}); err != nil {\n\t\tt.Fatalf(\"revert failed: %v\", err)\n\t}\n\n\t// Verify version was reverted\n\tversionBytes, err := os.ReadFile(filepath.Join(repoPath, \"version\"))\n\tif err != nil {\n\t\tt.Fatalf(\"failed to read version file: %v\", err)\n\t}\n\n\tif actualVersion := string(versionBytes); actualVersion != fmt.Sprintf(\"%d\", fromVersion) {\n\t\tt.Errorf(\"expected version %d after revert, got %s\", fromVersion, actualVersion)\n\t}\n\n\t// Verify config was reverted\n\tconfigBytes, err := os.ReadFile(filepath.Join(repoPath, \"config\"))\n\tif err != nil {\n\t\tt.Fatalf(\"failed to read reverted config file: %v\", err)\n\t}\n\n\tvar revertedConfig map[string]any\n\tif err := json.Unmarshal(configBytes, &revertedConfig); err != nil {\n\t\tt.Fatalf(\"failed to unmarshal reverted config: %v\", err)\n\t}\n\n\t// Compare reverted config with original\n\tcompareConfigs(t, inputConfig, revertedConfig, \"\")\n}\n\n// compareConfigs recursively compares two config maps and reports differences\nfunc compareConfigs(t *testing.T, expected, actual map[string]any, path string) {\n\tt.Helper()\n\n\t// Build current path helper\n\tbuildPath := func(key string) string {\n\t\tif path == \"\" {\n\t\t\treturn key\n\t\t}\n\t\treturn path + \".\" + key\n\t}\n\n\t// Check all expected fields exist and match\n\tfor key, expectedValue := range expected {\n\t\tcurrentPath := buildPath(key)\n\n\t\tactualValue, exists := actual[key]\n\t\tif !exists {\n\t\t\tt.Errorf(\"reverted config missing field %s\", currentPath)\n\t\t\tcontinue\n\t\t}\n\n\t\tswitch exp := expectedValue.(type) {\n\t\tcase map[string]any:\n\t\t\tact, ok := actualValue.(map[string]any)\n\t\t\tif !ok {\n\t\t\t\tt.Errorf(\"field %s: expected map, got %T\", currentPath, actualValue)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tcompareConfigs(t, exp, act, currentPath)\n\t\tdefault:\n\t\t\tif !reflect.DeepEqual(expectedValue, actualValue) {\n\t\t\t\tt.Errorf(\"field %s: expected %v, got %v after revert\",\n\t\t\t\t\tcurrentPath, expectedValue, actualValue)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Check for unexpected fields using maps.Keys (Go 1.23+)\n\tfor key := range actual {\n\t\tif _, exists := expected[key]; !exists {\n\t\t\tt.Errorf(\"reverted config has unexpected field %s\", buildPath(key))\n\t\t}\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/common/utils.go", "content": "package common\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/atomicfile\"\n)\n\n// CheckVersion verifies the repo is at the expected version\nfunc CheckVersion(repoPath string, expectedVersion string) error {\n\tversionPath := filepath.Join(repoPath, \"version\")\n\tversionBytes, err := os.ReadFile(versionPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not read version file: %w\", err)\n\t}\n\tversion := strings.TrimSpace(string(versionBytes))\n\tif version != expectedVersion {\n\t\treturn fmt.Errorf(\"expected version %s, got %s\", expectedVersion, version)\n\t}\n\treturn nil\n}\n\n// WriteVersion writes the version to the repo\nfunc WriteVersion(repoPath string, version string) error {\n\tversionPath := filepath.Join(repoPath, \"version\")\n\treturn os.WriteFile(versionPath, []byte(version), 0644)\n}\n\n// Must panics if the error is not nil. Use only for errors that cannot be handled gracefully.\nfunc Must(err error) {\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"error can't be dealt with transactionally: %w\", err))\n\t}\n}\n\n// WithBackup performs a config file operation with automatic backup and rollback on error\nfunc WithBackup(configPath string, backupSuffix string, fn func(in io.ReadSeeker, out io.Writer) error) error {\n\t// Read the entire file into memory first\n\t// This allows us to close the file before doing atomic operations,\n\t// which is necessary on Windows where open files can't be renamed\n\tdata, err := os.ReadFile(configPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read config file %s: %w\", configPath, err)\n\t}\n\n\t// Create an in-memory reader for the data\n\tin := bytes.NewReader(data)\n\n\t// Create backup atomically to prevent partial backup on interruption\n\tbackupPath := configPath + backupSuffix\n\tbackup, err := atomicfile.New(backupPath, 0600)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to create backup file for %s: %w\", backupPath, err)\n\t}\n\tif _, err := backup.Write(data); err != nil {\n\t\tMust(backup.Abort())\n\t\treturn fmt.Errorf(\"failed to write backup data: %w\", err)\n\t}\n\tif err := backup.Close(); err != nil {\n\t\tMust(backup.Abort())\n\t\treturn fmt.Errorf(\"failed to finalize backup: %w\", err)\n\t}\n\n\t// Create output file atomically\n\tout, err := atomicfile.New(configPath, 0600)\n\tif err != nil {\n\t\t// Clean up backup on error\n\t\tos.Remove(backupPath)\n\t\treturn fmt.Errorf(\"failed to create atomic file for %s: %w\", configPath, err)\n\t}\n\n\t// Run the conversion function\n\tif err := fn(in, out); err != nil {\n\t\tMust(out.Abort())\n\t\t// Clean up backup on error\n\t\tos.Remove(backupPath)\n\t\treturn fmt.Errorf(\"config conversion failed: %w\", err)\n\t}\n\n\t// Close the output file atomically\n\tMust(out.Close())\n\t// Backup remains for potential revert\n\n\treturn nil\n}\n\n// RevertBackup restores a backup file\nfunc RevertBackup(configPath string, backupSuffix string) error {\n\treturn os.Rename(configPath+backupSuffix, configPath)\n}\n\n// ReadConfig reads and unmarshals a JSON config file into a map\nfunc ReadConfig(r io.Reader) (map[string]any, error) {\n\tconfMap := make(map[string]any)\n\tif err := json.NewDecoder(r).Decode(&confMap); err != nil {\n\t\treturn nil, err\n\t}\n\treturn confMap, nil\n}\n\n// WriteConfig marshals and writes a config map as indented JSON\nfunc WriteConfig(w io.Writer, config map[string]any) error {\n\tenc := json.NewEncoder(w)\n\tenc.SetIndent(\"\", \" \")\n\treturn enc.Encode(config)\n}\n" }, { "path": "repo/fsrepo/migrations/embedded.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\n\tlockfile \"github.com/ipfs/go-fs-lock\"\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n\tmg16 \"github.com/ipfs/kubo/repo/fsrepo/migrations/fs-repo-16-to-17/migration\"\n\tmg17 \"github.com/ipfs/kubo/repo/fsrepo/migrations/fs-repo-17-to-18/migration\"\n)\n\n// embeddedMigrations contains all embedded migrations\n// Using a slice to maintain order and allow for future range-based operations\nvar embeddedMigrations = []common.Migration{\n\tmg16.Migration,\n\tmg17.Migration,\n}\n\n// migrationsByName provides quick lookup by name\nvar migrationsByName = make(map[string]common.Migration)\n\nfunc init() {\n\tfor _, m := range embeddedMigrations {\n\t\tmigrationsByName[\"fs-repo-\"+m.Versions()] = m\n\t}\n}\n\n// RunEmbeddedMigration runs an embedded migration if available\nfunc RunEmbeddedMigration(ctx context.Context, migrationName string, ipfsDir string, revert bool) error {\n\tmigration, exists := migrationsByName[migrationName]\n\tif !exists {\n\t\treturn fmt.Errorf(\"embedded migration %s not found\", migrationName)\n\t}\n\n\tif revert && !migration.Reversible() {\n\t\treturn fmt.Errorf(\"migration %s is not reversible\", migrationName)\n\t}\n\n\tlogger := log.New(os.Stdout, \"\", 0)\n\tlogger.Printf(\"Running embedded migration %s...\", migrationName)\n\n\topts := common.Options{\n\t\tPath: ipfsDir,\n\t\tVerbose: true,\n\t}\n\n\tvar err error\n\tif revert {\n\t\terr = migration.Revert(opts)\n\t} else {\n\t\terr = migration.Apply(opts)\n\t}\n\n\tif err != nil {\n\t\treturn fmt.Errorf(\"embedded migration %s failed: %w\", migrationName, err)\n\t}\n\n\tlogger.Printf(\"Embedded migration %s completed successfully\", migrationName)\n\treturn nil\n}\n\n// HasEmbeddedMigration checks if a migration is available as embedded\nfunc HasEmbeddedMigration(migrationName string) bool {\n\t_, exists := migrationsByName[migrationName]\n\treturn exists\n}\n\n// RunEmbeddedMigrations runs all needed embedded migrations from current version to target version.\n//\n// This function migrates an IPFS repository using embedded migrations that are built into the Kubo binary.\n// Embedded migrations are available for repo version 17+ and provide fast, network-free migration execution.\n//\n// Parameters:\n// - ctx: Context for cancellation and deadlines\n// - targetVer: Target repository version to migrate to\n// - ipfsDir: Path to the IPFS repository directory\n// - allowDowngrade: Whether to allow downgrade migrations (reduces target version)\n//\n// Returns:\n// - nil on successful migration\n// - error if migration fails, repo path is invalid, or no embedded migrations are available\n//\n// Behavior:\n// - Validates that ipfsDir contains a valid IPFS repository\n// - Determines current repository version automatically\n// - Returns immediately if already at target version\n// - Prevents downgrades unless allowDowngrade is true\n// - Runs all necessary migrations in sequence (e.g., 16→17→18 if going from 16 to 18)\n// - Creates backups and uses atomic operations to prevent corruption\n//\n// Error conditions:\n// - Repository path is invalid or inaccessible\n// - Current version cannot be determined\n// - Downgrade attempted with allowDowngrade=false\n// - No embedded migrations available for the version range\n// - Individual migration fails during execution\n//\n// Example:\n//\n//\terr := RunEmbeddedMigrations(ctx, 17, \"/path/to/.ipfs\", false)\n//\tif err != nil {\n//\t // Handle migration failure, may need to fall back to external migrations\n//\t}\nfunc RunEmbeddedMigrations(ctx context.Context, targetVer int, ipfsDir string, allowDowngrade bool) error {\n\tipfsDir, err := CheckIpfsDir(ipfsDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Acquire lock once for all embedded migrations to prevent concurrent access\n\tlk, err := lockfile.Lock(ipfsDir, \"repo.lock\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to acquire repo lock: %w\", err)\n\t}\n\tdefer lk.Close()\n\n\tfromVer, err := RepoVersion(ipfsDir)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not get repo version: %w\", err)\n\t}\n\n\tif fromVer == targetVer {\n\t\treturn nil\n\t}\n\n\trevert := fromVer > targetVer\n\tif revert && !allowDowngrade {\n\t\treturn fmt.Errorf(\"downgrade not allowed from %d to %d\", fromVer, targetVer)\n\t}\n\n\tlogger := log.New(os.Stdout, \"\", 0)\n\tlogger.Print(\"Looking for embedded migrations.\")\n\n\tmigrations, _, err := findMigrations(ctx, fromVer, targetVer)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tembeddedCount := 0\n\tfor _, migrationName := range migrations {\n\t\tif HasEmbeddedMigration(migrationName) {\n\t\t\terr = RunEmbeddedMigration(ctx, migrationName, ipfsDir, revert)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tembeddedCount++\n\t\t}\n\t}\n\n\tif embeddedCount == 0 {\n\t\treturn fmt.Errorf(\"no embedded migrations found for version %d to %d\", fromVer, targetVer)\n\t}\n\n\tlogger.Printf(\"Success: fs-repo migrated to version %d using embedded migrations.\\n\", targetVer)\n\treturn nil\n}\n" }, { "path": "repo/fsrepo/migrations/embedded_test.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestHasEmbeddedMigration(t *testing.T) {\n\t// Test that the 16-to-17 migration is registered\n\tassert.True(t, HasEmbeddedMigration(\"fs-repo-16-to-17\"),\n\t\t\"fs-repo-16-to-17 migration should be registered\")\n\n\t// Test that a non-existent migration is not found\n\tassert.False(t, HasEmbeddedMigration(\"fs-repo-99-to-100\"),\n\t\t\"fs-repo-99-to-100 migration should not be registered\")\n}\n\nfunc TestEmbeddedMigrations(t *testing.T) {\n\t// Test that we have at least one embedded migration\n\tassert.NotEmpty(t, embeddedMigrations, \"No embedded migrations found\")\n\n\t// Test that all registered migrations implement the interface\n\tfor name, migration := range embeddedMigrations {\n\t\tassert.NotEmpty(t, migration.Versions(),\n\t\t\t\"Migration %s has empty versions\", name)\n\t}\n}\n\nfunc TestRunEmbeddedMigration(t *testing.T) {\n\t// Test that running a non-existent migration returns an error\n\terr := RunEmbeddedMigration(context.Background(), \"non-existent\", \"/tmp\", false)\n\trequire.Error(t, err, \"Expected error for non-existent migration\")\n}\n" }, { "path": "repo/fsrepo/migrations/fetch.go", "content": "package migrations\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n)\n\n// DownloadDirectory can be set as the location for FetchBinary to save the\n// downloaded archive file in. If not set, then FetchBinary saves the archive\n// in a temporary directory that is removed after the contents of the archive\n// is extracted.\nvar DownloadDirectory string\n\n// FetchBinary downloads an archive from the distribution site and unpacks it.\n//\n// The base name of the binary inside the archive may differ from the base\n// archive name. If it does, then specify binName. For example, the following\n// is needed because the archive \"go-ipfs_v0.7.0_linux-amd64.tar.gz\" contains a\n// binary named \"ipfs\"\n//\n//\tFetchBinary(ctx, fetcher, \"go-ipfs\", \"v0.7.0\", \"ipfs\", tmpDir)\n//\n// If out is a directory, then the binary is written to that directory with the\n// same name it has inside the archive. Otherwise, the binary file is written\n// to the file named by out.\nfunc FetchBinary(ctx context.Context, fetcher Fetcher, dist, ver, binName, out string) (string, error) {\n\t// The archive file name is the base of dist. This is to support a possible subdir in\n\t// dist, for example: \"ipfs-repo-migrations/fs-repo-11-to-12\"\n\tarcName := filepath.Base(dist)\n\t// If binary base name is not specified, then it is same as archive base name.\n\tif binName == \"\" {\n\t\tbinName = arcName\n\t}\n\n\t// Name of binary that exists inside archive\n\tbinName = ExeName(binName)\n\n\t// Return error if file exists or stat fails for reason other than not\n\t// exists. If out is a directory, then write extracted binary to that dir.\n\tfi, err := os.Stat(out)\n\tif !os.IsNotExist(err) {\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t\tif !fi.IsDir() {\n\t\t\treturn \"\", &os.PathError{\n\t\t\t\tOp: \"FetchBinary\",\n\t\t\t\tPath: out,\n\t\t\t\tErr: os.ErrExist,\n\t\t\t}\n\t\t}\n\t\t// out exists and is a directory, so compose final name\n\t\tout = filepath.Join(out, binName)\n\t\t// Check if the binary already exists in the directory\n\t\t_, err = os.Stat(out)\n\t\tif !os.IsNotExist(err) {\n\t\t\tif err != nil {\n\t\t\t\treturn \"\", err\n\t\t\t}\n\t\t\treturn \"\", &os.PathError{\n\t\t\t\tOp: \"FetchBinary\",\n\t\t\t\tPath: out,\n\t\t\t\tErr: os.ErrExist,\n\t\t\t}\n\t\t}\n\t}\n\n\ttmpDir := DownloadDirectory\n\tif tmpDir != \"\" {\n\t\tfi, err = os.Stat(tmpDir)\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t\tif !fi.IsDir() {\n\t\t\treturn \"\", &os.PathError{\n\t\t\t\tOp: \"FetchBinary\",\n\t\t\t\tPath: tmpDir,\n\t\t\t\tErr: os.ErrExist,\n\t\t\t}\n\t\t}\n\t} else {\n\t\t// Create temp directory to store download\n\t\ttmpDir, err = os.MkdirTemp(\"\", arcName)\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t\tdefer os.RemoveAll(tmpDir)\n\t}\n\n\tatype := \"tar.gz\"\n\tif runtime.GOOS == \"windows\" {\n\t\tatype = \"zip\"\n\t}\n\n\tarcDistPath, arcFullName := makeArchivePath(dist, arcName, ver, atype)\n\n\t// Create a file to write the archive data to\n\tarcPath := filepath.Join(tmpDir, arcFullName)\n\tarcFile, err := os.Create(arcPath)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tdefer arcFile.Close()\n\n\t// Open connection to download archive from ipfs path and write to file\n\tarcBytes, err := fetcher.Fetch(ctx, arcDistPath)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Write download data\n\t_, err = io.Copy(arcFile, bytes.NewReader(arcBytes))\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tarcFile.Close()\n\n\t// Unpack the archive and write binary to out\n\terr = unpackArchive(arcPath, atype, dist, binName, out)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Set mode of binary to executable\n\terr = os.Chmod(out, 0o755)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn out, nil\n}\n\n// osWithVariant returns the OS name with optional variant.\n// Currently returns either runtime.GOOS, or \"linux-musl\".\nfunc osWithVariant() (string, error) {\n\tif runtime.GOOS != \"linux\" {\n\t\treturn runtime.GOOS, nil\n\t}\n\n\t// ldd outputs the system's kind of libc.\n\t// - on standard ubuntu: ldd (Ubuntu GLIBC 2.23-0ubuntu5) 2.23\n\t// - on alpine: musl libc (x86_64)\n\t//\n\t// we use the combined stdout+stderr,\n\t// because ldd --version prints differently on different OSes.\n\t// - on standard ubuntu: stdout\n\t// - on alpine: stderr (it probably doesn't know the --version flag)\n\t//\n\t// we suppress non-zero exit codes (see last point about alpine).\n\tout, err := exec.Command(\"sh\", \"-c\", \"ldd --version || true\").CombinedOutput()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// now just see if we can find \"musl\" somewhere in the output\n\tscan := bufio.NewScanner(bytes.NewBuffer(out))\n\tfor scan.Scan() {\n\t\tif strings.Contains(scan.Text(), \"musl\") {\n\t\t\treturn \"linux-musl\", nil\n\t\t}\n\t}\n\n\treturn \"linux\", nil\n}\n\n// makeArchivePath composes the path, relative to the distribution site, from which to\n// download a binary. The path returned does not contain the distribution site path,\n// e.g. \"/ipns/dist.ipfs.tech/\", since that is know to the fetcher.\n//\n// Returns the archive path and the base name.\n//\n// The ipfs path format is: distribution/version/archiveName\n// - distribution is the name of a distribution, such as \"go-ipfs\"\n// - version is the version to fetch, such as \"v0.8.0-rc2\"\n// - archiveName is formatted as name_version_osv-GOARCH.atype, such as\n// \"go-ipfs_v0.8.0-rc2_linux-amd64.tar.gz\"\n//\n// This would form the path:\n// go-ipfs/v0.8.0/go-ipfs_v0.8.0_linux-amd64.tar.gz\nfunc makeArchivePath(dist, name, ver, atype string) (string, string) {\n\tarcName := fmt.Sprintf(\"%s_%s_%s-%s.%s\", name, ver, runtime.GOOS, runtime.GOARCH, atype)\n\treturn fmt.Sprintf(\"%s/%s/%s\", dist, ver, arcName), arcName\n}\n" }, { "path": "repo/fsrepo/migrations/fetch_test.go", "content": "package migrations\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc TestGetDistPath(t *testing.T) {\n\tos.Unsetenv(envIpfsDistPath)\n\tdistPath := GetDistPathEnv(\"\")\n\tif distPath != LatestIpfsDist {\n\t\tt.Error(\"did not set default dist path\")\n\t}\n\n\ttestDist := \"/unit/test/dist\"\n\tt.Setenv(envIpfsDistPath, testDist)\n\tdefer func() {\n\t\tos.Unsetenv(envIpfsDistPath)\n\t}()\n\n\tdistPath = GetDistPathEnv(\"\")\n\tif distPath != testDist {\n\t\tt.Error(\"did not set dist path from environ\")\n\t}\n\tdistPath = GetDistPathEnv(\"ignored\")\n\tif distPath != testDist {\n\t\tt.Error(\"did not set dist path from environ\")\n\t}\n\n\ttestDist = \"/unit/test/dist2\"\n\tfetcher := NewHttpFetcher(testDist, \"\", \"\", 0)\n\tif fetcher.distPath != testDist {\n\t\tt.Error(\"did not set dist path\")\n\t}\n}\n\nfunc TestHttpFetch(t *testing.T) {\n\tctx := t.Context()\n\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\tout, err := fetcher.Fetch(ctx, \"/kubo/versions\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar lines []string\n\tscan := bufio.NewScanner(bytes.NewReader(out))\n\tfor scan.Scan() {\n\t\tlines = append(lines, scan.Text())\n\t}\n\terr = scan.Err()\n\tif err != nil {\n\t\tt.Fatal(\"could not read versions:\", err)\n\t}\n\n\tif len(lines) < 6 {\n\t\tt.Fatal(\"do not get all expected data\")\n\t}\n\tif lines[0] != \"v1.0.0\" {\n\t\tt.Fatal(\"expected v1.0.0 as first line, got\", lines[0])\n\t}\n\n\t// Check not found\n\t_, err = fetcher.Fetch(ctx, \"/no_such_file\")\n\tif err == nil || !strings.Contains(err.Error(), \"no link\") {\n\t\tt.Fatal(\"expected error 404\")\n\t}\n}\n\nfunc TestFetchBinary(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tctx := t.Context()\n\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\tvers, err := DistVersions(ctx, fetcher, distFSRM, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Log(\"latest version of\", distFSRM, \"is\", vers[len(vers)-1])\n\n\tbin, err := FetchBinary(ctx, fetcher, distFSRM, vers[0], \"\", tmpDir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfi, err := os.Stat(bin)\n\tif os.IsNotExist(err) {\n\t\tt.Error(\"expected file to exist:\", bin)\n\t}\n\n\tt.Log(\"downloaded and unpacked\", fi.Size(), \"byte file:\", fi.Name())\n\n\tbin, err = FetchBinary(ctx, fetcher, \"go-ipfs\", \"v1.0.0\", \"ipfs\", tmpDir)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfi, err = os.Stat(bin)\n\tif os.IsNotExist(err) {\n\t\tt.Error(\"expected file to exist:\", bin)\n\t}\n\n\tt.Log(\"downloaded and unpacked\", fi.Size(), \"byte file:\", fi.Name())\n\n\t// Check error is destination already exists and is not directory\n\t_, err = FetchBinary(ctx, fetcher, \"go-ipfs\", \"v1.0.0\", \"ipfs\", bin)\n\tif !os.IsExist(err) {\n\t\tt.Fatal(\"expected 'exists' error, got\", err)\n\t}\n\n\t_, err = FetchBinary(ctx, fetcher, \"go-ipfs\", \"v1.0.0\", \"ipfs\", tmpDir)\n\tif !os.IsExist(err) {\n\t\tt.Error(\"expected 'exists' error, got:\", err)\n\t}\n\n\tos.Remove(filepath.Join(tmpDir, ExeName(\"ipfs\")))\n\n\t// Check error creating temp download directory\n\t//\n\t// Windows doesn't have read-only directories https://github.com/golang/go/issues/35042 this would need to be\n\t// tested another way\n\tif runtime.GOOS != \"windows\" {\n\t\terr = os.Chmod(tmpDir, 0o555)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tt.Setenv(\"TMPDIR\", tmpDir)\n\t\t_, err = FetchBinary(ctx, fetcher, \"go-ipfs\", \"v1.0.0\", \"ipfs\", tmpDir)\n\t\tif !os.IsPermission(err) {\n\t\t\tt.Error(\"expected 'permission' error, got:\", err)\n\t\t}\n\t\tt.Setenv(\"TMPDIR\", \"/tmp\")\n\t\terr = os.Chmod(tmpDir, 0o755)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n\n\t// Check error if failure to fetch due to bad dist\n\t_, err = FetchBinary(ctx, fetcher, \"not-here\", \"v1.0.0\", \"ipfs\", tmpDir)\n\tif err == nil || !strings.Contains(err.Error(), \"no link\") {\n\t\tt.Error(\"expected 'Not Found' error, got:\", err)\n\t}\n\n\t// Check error if failure to unpack archive\n\t_, err = FetchBinary(ctx, fetcher, \"go-ipfs\", \"v1.0.0\", \"not-such-bin\", tmpDir)\n\tif err == nil || err.Error() != \"no binary found in archive\" {\n\t\tt.Error(\"expected 'no binary found in archive' error\")\n\t}\n}\n\nfunc TestMultiFetcher(t *testing.T) {\n\tctx := t.Context()\n\n\tbadFetcher := NewHttpFetcher(\"\", \"bad-url\", \"\", 0)\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\tmf := NewMultiFetcher(badFetcher, fetcher)\n\n\tvers, err := mf.Fetch(ctx, \"/kubo/versions\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(vers) < 45 {\n\t\tfmt.Println(\"unexpected more data\")\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/fetcher.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n)\n\nconst (\n\t// Current distribution to fetch migrations from.\n\tCurrentIpfsDist = \"/ipfs/QmRzRGJEjYDfbHHaALnHBuhzzrkXGdwcPMrgd5fgM7hqbe\" // fs-repo-15-to-16 v1.0.1\n\t// Latest distribution path. Default for fetchers.\n\tLatestIpfsDist = \"/ipns/dist.ipfs.tech\"\n\n\t// Distribution environ variable.\n\tenvIpfsDistPath = \"IPFS_DIST_PATH\"\n)\n\ntype Fetcher interface {\n\t// Fetch attempts to fetch the file at the given ipfs path.\n\tFetch(ctx context.Context, filePath string) ([]byte, error)\n\t// Close performs any cleanup after the fetcher is not longer needed.\n\tClose() error\n}\n\n// MultiFetcher holds multiple Fetchers and provides a Fetch that tries each\n// until one succeeds.\ntype MultiFetcher struct {\n\tfetchers []Fetcher\n}\n\ntype limitReadCloser struct {\n\tio.Reader\n\tio.Closer\n}\n\n// NewMultiFetcher creates a MultiFetcher with the given Fetchers. The\n// Fetchers are tried in order, then passed to this function.\nfunc NewMultiFetcher(f ...Fetcher) *MultiFetcher {\n\tmf := &MultiFetcher{\n\t\tfetchers: make([]Fetcher, len(f)),\n\t}\n\tcopy(mf.fetchers, f)\n\treturn mf\n}\n\n// Fetch attempts to fetch the file at each of its fetchers until one succeeds.\nfunc (f *MultiFetcher) Fetch(ctx context.Context, ipfsPath string) ([]byte, error) {\n\tvar errs []error\n\tfor _, fetcher := range f.fetchers {\n\t\tout, err := fetcher.Fetch(ctx, ipfsPath)\n\t\tif err == nil {\n\t\t\treturn out, nil\n\t\t}\n\t\tfmt.Printf(\"Error fetching: %s\\n\", err.Error())\n\t\terrs = append(errs, err)\n\t}\n\treturn nil, errors.Join(errs...)\n}\n\nfunc (f *MultiFetcher) Close() error {\n\tvar errs error\n\tfor _, fetcher := range f.fetchers {\n\t\tif err := fetcher.Close(); err != nil {\n\t\t\terrs = errors.Join(errs, err)\n\t\t}\n\t}\n\treturn errs\n}\n\nfunc (f *MultiFetcher) Len() int {\n\treturn len(f.fetchers)\n}\n\nfunc (f *MultiFetcher) Fetchers() []Fetcher {\n\treturn f.fetchers\n}\n\n// NewLimitReadCloser returns a new io.ReadCloser with the reader wrapped in a\n// io.LimitedReader limited to reading the amount specified.\nfunc NewLimitReadCloser(rc io.ReadCloser, limit int64) io.ReadCloser {\n\treturn limitReadCloser{\n\t\tReader: io.LimitReader(rc, limit),\n\t\tCloser: rc,\n\t}\n}\n\n// GetDistPathEnv returns the IPFS path to the distribution site, using\n// the value of environ variable specified by envIpfsDistPath. If the environ\n// variable is not set, then returns the provided distPath, and if that is not set\n// then returns the IPNS path.\n//\n// To get the IPFS path of the latest distribution, if not overridden by the\n// environ variable: GetDistPathEnv(CurrentIpfsDist).\nfunc GetDistPathEnv(distPath string) string {\n\tif dist := os.Getenv(envIpfsDistPath); dist != \"\" {\n\t\treturn dist\n\t}\n\tif distPath == \"\" {\n\t\treturn LatestIpfsDist\n\t}\n\treturn distPath\n}\n" }, { "path": "repo/fsrepo/migrations/fs-repo-16-to-17/main.go", "content": "// Package main implements fs-repo-16-to-17 migration for IPFS repositories.\n//\n// This migration transitions repositories from version 16 to 17, introducing\n// the AutoConf system that replaces hardcoded network defaults with dynamic\n// configuration fetched from autoconf.json.\n//\n// Changes made:\n// - Enables AutoConf system with default settings\n// - Migrates default bootstrap peers to \"auto\" sentinel value\n// - Sets DNS.Resolvers[\".\"] to \"auto\" for dynamic DNS resolver configuration\n// - Migrates Routing.DelegatedRouters to [\"auto\"]\n// - Migrates Ipns.DelegatedPublishers to [\"auto\"]\n// - Preserves user customizations (custom bootstrap peers, DNS resolvers)\n//\n// The migration is reversible and creates config.16-to-17.bak for rollback.\n//\n// Usage:\n//\n//\tfs-repo-16-to-17 -path /path/to/ipfs/repo [-verbose] [-revert]\n//\n// This migration is embedded in Kubo starting from version 0.37 and runs\n// automatically during daemon startup. This standalone binary is provided\n// for manual migration scenarios.\npackage main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n\tmg16 \"github.com/ipfs/kubo/repo/fsrepo/migrations/fs-repo-16-to-17/migration\"\n)\n\nfunc main() {\n\tvar path = flag.String(\"path\", \"\", \"Path to IPFS repository\")\n\tvar verbose = flag.Bool(\"verbose\", false, \"Enable verbose output\")\n\tvar revert = flag.Bool(\"revert\", false, \"Revert migration\")\n\tflag.Parse()\n\n\tif *path == \"\" {\n\t\tfmt.Fprintf(os.Stderr, \"Error: -path flag is required\\n\")\n\t\tflag.Usage()\n\t\tos.Exit(1)\n\t}\n\n\topts := common.Options{\n\t\tPath: *path,\n\t\tVerbose: *verbose,\n\t}\n\n\tvar err error\n\tif *revert {\n\t\terr = mg16.Migration.Revert(opts)\n\t} else {\n\t\terr = mg16.Migration.Apply(opts)\n\t}\n\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"Migration failed: %v\\n\", err)\n\t\tos.Exit(1)\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/fs-repo-16-to-17/migration/migration.go", "content": "// package mg16 contains the code to perform 16-17 repository migration in Kubo.\n// This handles the following:\n// - Migrate default bootstrap peers to \"auto\"\n// - Migrate DNS resolvers to use \"auto\" for \".\" eTLD\n// - Enable AutoConf system with default settings\n// - Increment repo version to 17\npackage mg16\n\nimport (\n\t\"io\"\n\t\"slices\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n)\n\n// DefaultBootstrapAddresses are the hardcoded bootstrap addresses from Kubo 0.36\n// for IPFS. they are nodes run by the IPFS team. docs on these later.\n// As with all p2p networks, bootstrap is an important security concern.\n// This list is used during migration to detect which peers are defaults vs custom.\nvar DefaultBootstrapAddresses = []string{\n\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\", // rust-libp2p-server\n\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n\t\"/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8\", // js-libp2p-amino-dht-bootstrapper\n\t\"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\", // mars.i.ipfs.io\n\t\"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\", // mars.i.ipfs.io\n}\n\n// Migration is the main exported migration for 16-to-17\nvar Migration = &common.BaseMigration{\n\tFromVersion: \"16\",\n\tToVersion: \"17\",\n\tDescription: \"Upgrading config to use AutoConf system\",\n\tConvert: convert,\n}\n\n// NewMigration creates a new migration instance (for compatibility)\nfunc NewMigration() common.Migration {\n\treturn Migration\n}\n\n// convert converts the config from version 16 to 17\nfunc convert(in io.ReadSeeker, out io.Writer) error {\n\tconfMap, err := common.ReadConfig(in)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Enable AutoConf system\n\tif err := enableAutoConf(confMap); err != nil {\n\t\treturn err\n\t}\n\n\t// Migrate Bootstrap peers\n\tif err := migrateBootstrap(confMap); err != nil {\n\t\treturn err\n\t}\n\n\t// Migrate DNS resolvers\n\tif err := migrateDNSResolvers(confMap); err != nil {\n\t\treturn err\n\t}\n\n\t// Migrate DelegatedRouters\n\tif err := migrateDelegatedRouters(confMap); err != nil {\n\t\treturn err\n\t}\n\n\t// Migrate DelegatedPublishers\n\tif err := migrateDelegatedPublishers(confMap); err != nil {\n\t\treturn err\n\t}\n\n\t// Save new config\n\treturn common.WriteConfig(out, confMap)\n}\n\n// enableAutoConf adds AutoConf section to config\nfunc enableAutoConf(confMap map[string]any) error {\n\t// Add empty AutoConf section if it doesn't exist - all fields will use implicit defaults:\n\t// - Enabled defaults to true (via DefaultAutoConfEnabled)\n\t// - URL defaults to mainnet URL (via DefaultAutoConfURL)\n\t// - RefreshInterval defaults to 24h (via DefaultAutoConfRefreshInterval)\n\t// - TLSInsecureSkipVerify defaults to false (no WithDefault, but false is zero value)\n\tcommon.SetDefault(confMap, \"AutoConf\", map[string]any{})\n\treturn nil\n}\n\n// migrateBootstrap migrates bootstrap peers to use \"auto\"\nfunc migrateBootstrap(confMap map[string]any) error {\n\tbootstrap, exists := confMap[\"Bootstrap\"]\n\tif !exists {\n\t\t// No bootstrap section, add \"auto\"\n\t\tconfMap[\"Bootstrap\"] = []string{config.AutoPlaceholder}\n\t\treturn nil\n\t}\n\n\t// Convert to string slice using helper\n\tbootstrapPeers := common.ConvertInterfaceSlice(common.SafeCastSlice(bootstrap))\n\tif len(bootstrapPeers) == 0 && bootstrap != nil {\n\t\t// Invalid bootstrap format, replace with \"auto\"\n\t\tconfMap[\"Bootstrap\"] = []string{config.AutoPlaceholder}\n\t\treturn nil\n\t}\n\n\t// Process bootstrap peers according to migration rules\n\tnewBootstrap := processBootstrapPeers(bootstrapPeers)\n\tconfMap[\"Bootstrap\"] = newBootstrap\n\n\treturn nil\n}\n\n// processBootstrapPeers processes bootstrap peers according to migration rules\nfunc processBootstrapPeers(peers []string) []string {\n\t// If empty, use \"auto\"\n\tif len(peers) == 0 {\n\t\treturn []string{config.AutoPlaceholder}\n\t}\n\n\t// Filter out default peers to get only custom ones\n\tcustomPeers := slices.DeleteFunc(slices.Clone(peers), func(peer string) bool {\n\t\treturn slices.Contains(DefaultBootstrapAddresses, peer)\n\t})\n\n\t// Check if any default peers were removed\n\thasDefaultPeers := len(customPeers) < len(peers)\n\n\t// If we have default peers, replace them with \"auto\"\n\tif hasDefaultPeers {\n\t\treturn append([]string{config.AutoPlaceholder}, customPeers...)\n\t}\n\n\t// No default peers found, keep as is\n\treturn peers\n}\n\n// migrateDNSResolvers migrates DNS resolvers to use \"auto\" for \".\" eTLD\nfunc migrateDNSResolvers(confMap map[string]any) error {\n\t// Get or create DNS section\n\tdns := common.GetOrCreateSection(confMap, \"DNS\")\n\n\t// Get existing resolvers or create empty map\n\tresolvers := common.SafeCastMap(dns[\"Resolvers\"])\n\n\t// Define default resolvers that should be replaced with \"auto\"\n\tdefaultResolvers := map[string]string{\n\t\t\"https://dns.eth.limo/dns-query\": config.AutoPlaceholder,\n\t\t\"https://dns.eth.link/dns-query\": config.AutoPlaceholder,\n\t\t\"https://resolver.cloudflare-eth.com/dns-query\": config.AutoPlaceholder,\n\t}\n\n\t// Replace default resolvers with \"auto\"\n\tstringResolvers := common.ReplaceDefaultsWithAuto(resolvers, defaultResolvers)\n\n\t// Ensure \".\" is set to \"auto\" if not already set\n\tif _, exists := stringResolvers[\".\"]; !exists {\n\t\tstringResolvers[\".\"] = config.AutoPlaceholder\n\t}\n\n\tdns[\"Resolvers\"] = stringResolvers\n\treturn nil\n}\n\n// migrateDelegatedRouters migrates DelegatedRouters to use \"auto\"\nfunc migrateDelegatedRouters(confMap map[string]any) error {\n\t// Get or create Routing section\n\trouting := common.GetOrCreateSection(confMap, \"Routing\")\n\n\t// Get existing delegated routers\n\tdelegatedRouters, exists := routing[\"DelegatedRouters\"]\n\n\t// Check if it's empty or nil\n\tif !exists || common.IsEmptySlice(delegatedRouters) {\n\t\trouting[\"DelegatedRouters\"] = []string{config.AutoPlaceholder}\n\t\treturn nil\n\t}\n\n\t// Process the list to replace cid.contact with \"auto\" and preserve others\n\trouters := common.ConvertInterfaceSlice(common.SafeCastSlice(delegatedRouters))\n\tvar newRouters []string\n\thasAuto := false\n\n\tfor _, router := range routers {\n\t\tif router == \"https://cid.contact\" {\n\t\t\tif !hasAuto {\n\t\t\t\tnewRouters = append(newRouters, config.AutoPlaceholder)\n\t\t\t\thasAuto = true\n\t\t\t}\n\t\t} else {\n\t\t\tnewRouters = append(newRouters, router)\n\t\t}\n\t}\n\n\t// If empty after processing, add \"auto\"\n\tif len(newRouters) == 0 {\n\t\tnewRouters = []string{config.AutoPlaceholder}\n\t}\n\n\trouting[\"DelegatedRouters\"] = newRouters\n\treturn nil\n}\n\n// migrateDelegatedPublishers migrates DelegatedPublishers to use \"auto\"\nfunc migrateDelegatedPublishers(confMap map[string]any) error {\n\t// Get or create Ipns section\n\tipns := common.GetOrCreateSection(confMap, \"Ipns\")\n\n\t// Get existing delegated publishers\n\tdelegatedPublishers, exists := ipns[\"DelegatedPublishers\"]\n\n\t// Check if it's empty or nil - only then replace with \"auto\"\n\t// Otherwise preserve custom publishers\n\tif !exists || common.IsEmptySlice(delegatedPublishers) {\n\t\tipns[\"DelegatedPublishers\"] = []string{config.AutoPlaceholder}\n\t}\n\t// If there are custom publishers, leave them as is\n\n\treturn nil\n}\n" }, { "path": "repo/fsrepo/migrations/fs-repo-16-to-17/migration/migration_test.go", "content": "package mg16\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"maps\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// Helper function to run migration on JSON input and return result\nfunc runMigrationOnJSON(t *testing.T, input string) map[string]any {\n\tt.Helper()\n\tvar output bytes.Buffer\n\terr := convert(bytes.NewReader([]byte(input)), &output)\n\trequire.NoError(t, err)\n\n\tvar result map[string]any\n\terr = json.Unmarshal(output.Bytes(), &result)\n\trequire.NoError(t, err)\n\n\treturn result\n}\n\n// Helper function to assert nested map key has expected value\nfunc assertMapKeyEquals(t *testing.T, result map[string]any, path []string, key string, expected any) {\n\tt.Helper()\n\tcurrent := result\n\tfor _, p := range path {\n\t\tsection, exists := current[p]\n\t\trequire.True(t, exists, \"Section %s not found in path %v\", p, path)\n\t\tcurrent = section.(map[string]any)\n\t}\n\n\tassert.Equal(t, expected, current[key], \"Expected %s to be %v\", key, expected)\n}\n\n// Helper function to assert slice contains expected values\nfunc assertSliceEquals(t *testing.T, result map[string]any, path []string, expected []string) {\n\tt.Helper()\n\tcurrent := result\n\tfor i, p := range path[:len(path)-1] {\n\t\tsection, exists := current[p]\n\t\trequire.True(t, exists, \"Section %s not found in path %v at index %d\", p, path, i)\n\t\tcurrent = section.(map[string]any)\n\t}\n\n\tsliceKey := path[len(path)-1]\n\tslice, exists := current[sliceKey]\n\trequire.True(t, exists, \"Slice %s not found\", sliceKey)\n\n\tactualSlice := slice.([]any)\n\trequire.Equal(t, len(expected), len(actualSlice), \"Expected slice length %d, got %d\", len(expected), len(actualSlice))\n\n\tfor i, exp := range expected {\n\t\tassert.Equal(t, exp, actualSlice[i], \"Expected slice[%d] to be %s\", i, exp)\n\t}\n}\n\n// Helper to build test config JSON with specified fields\nfunc buildTestConfig(fields map[string]any) string {\n\tconfig := map[string]any{\n\t\t\"Identity\": map[string]any{\"PeerID\": \"QmTest\"},\n\t}\n\tmaps.Copy(config, fields)\n\tdata, _ := json.MarshalIndent(config, \"\", \" \")\n\treturn string(data)\n}\n\n// Helper to run migration and get DNS resolvers\nfunc runMigrationAndGetDNSResolvers(t *testing.T, input string) map[string]any {\n\tt.Helper()\n\tresult := runMigrationOnJSON(t, input)\n\tdns := result[\"DNS\"].(map[string]any)\n\treturn dns[\"Resolvers\"].(map[string]any)\n}\n\n// Helper to assert multiple resolver values\nfunc assertResolvers(t *testing.T, resolvers map[string]any, expected map[string]string) {\n\tt.Helper()\n\tfor key, expectedValue := range expected {\n\t\tassert.Equal(t, expectedValue, resolvers[key], \"Expected %s resolver to be %v\", key, expectedValue)\n\t}\n}\n\n// =============================================================================\n// End-to-End Migration Tests\n// =============================================================================\n\nfunc TestMigration(t *testing.T) {\n\t// Create a temporary directory for testing\n\ttempDir, err := os.MkdirTemp(\"\", \"migration-test-16-to-17\")\n\trequire.NoError(t, err)\n\tdefer os.RemoveAll(tempDir)\n\n\t// Create a test config with default bootstrap peers\n\ttestConfig := map[string]any{\n\t\t\"Bootstrap\": []string{\n\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\t\"/ip4/192.168.1.1/tcp/4001/p2p/QmCustomPeer\", // Custom peer\n\t\t},\n\t\t\"DNS\": map[string]any{\n\t\t\t\"Resolvers\": map[string]string{},\n\t\t},\n\t\t\"Routing\": map[string]any{\n\t\t\t\"DelegatedRouters\": []string{},\n\t\t},\n\t\t\"Ipns\": map[string]any{\n\t\t\t\"ResolveCacheSize\": 128,\n\t\t},\n\t\t\"Identity\": map[string]any{\n\t\t\t\"PeerID\": \"QmTest\",\n\t\t},\n\t\t\"Version\": map[string]any{\n\t\t\t\"Current\": \"0.36.0\",\n\t\t},\n\t}\n\n\t// Write test config\n\tconfigPath := filepath.Join(tempDir, \"config\")\n\tconfigData, err := json.MarshalIndent(testConfig, \"\", \" \")\n\trequire.NoError(t, err)\n\terr = os.WriteFile(configPath, configData, 0644)\n\trequire.NoError(t, err)\n\n\t// Create version file\n\tversionPath := filepath.Join(tempDir, \"version\")\n\terr = os.WriteFile(versionPath, []byte(\"16\"), 0644)\n\trequire.NoError(t, err)\n\n\t// Run migration\n\topts := common.Options{\n\t\tPath: tempDir,\n\t\tVerbose: true,\n\t}\n\n\terr = Migration.Apply(opts)\n\trequire.NoError(t, err)\n\n\t// Verify version was updated\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"17\", string(versionData), \"Expected version 17\")\n\n\t// Verify config was updated\n\tconfigData, err = os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\n\tvar updatedConfig map[string]any\n\terr = json.Unmarshal(configData, &updatedConfig)\n\trequire.NoError(t, err)\n\n\t// Check AutoConf was added\n\tautoConf, exists := updatedConfig[\"AutoConf\"]\n\tassert.True(t, exists, \"AutoConf section not added\")\n\tautoConfMap := autoConf.(map[string]any)\n\t// URL is not set explicitly in migration (uses implicit default)\n\t_, hasURL := autoConfMap[\"URL\"]\n\tassert.False(t, hasURL, \"AutoConf URL should not be explicitly set in migration\")\n\n\t// Check Bootstrap was updated\n\tbootstrap := updatedConfig[\"Bootstrap\"].([]any)\n\tassert.Equal(t, 2, len(bootstrap), \"Expected 2 bootstrap entries\")\n\tassert.Equal(t, \"auto\", bootstrap[0], \"Expected first bootstrap entry to be 'auto'\")\n\tassert.Equal(t, \"/ip4/192.168.1.1/tcp/4001/p2p/QmCustomPeer\", bootstrap[1], \"Expected custom peer to be preserved\")\n\n\t// Check DNS.Resolvers was updated\n\tdns := updatedConfig[\"DNS\"].(map[string]any)\n\tresolvers := dns[\"Resolvers\"].(map[string]any)\n\tassert.Equal(t, \"auto\", resolvers[\".\"], \"Expected DNS resolver for '.' to be 'auto'\")\n\n\t// Check Routing.DelegatedRouters was updated\n\trouting := updatedConfig[\"Routing\"].(map[string]any)\n\tdelegatedRouters := routing[\"DelegatedRouters\"].([]any)\n\tassert.Equal(t, 1, len(delegatedRouters))\n\tassert.Equal(t, \"auto\", delegatedRouters[0], \"Expected DelegatedRouters to be ['auto']\")\n\n\t// Check Ipns.DelegatedPublishers was updated\n\tipns := updatedConfig[\"Ipns\"].(map[string]any)\n\tdelegatedPublishers := ipns[\"DelegatedPublishers\"].([]any)\n\tassert.Equal(t, 1, len(delegatedPublishers))\n\tassert.Equal(t, \"auto\", delegatedPublishers[0], \"Expected DelegatedPublishers to be ['auto']\")\n\n\t// Test revert\n\terr = Migration.Revert(opts)\n\trequire.NoError(t, err)\n\n\t// Verify version was reverted\n\tversionData, err = os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"16\", string(versionData), \"Expected version 16 after revert\")\n}\n\nfunc TestConvert(t *testing.T) {\n\tt.Parallel()\n\tinput := buildTestConfig(map[string]any{\n\t\t\"Bootstrap\": []string{\n\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t},\n\t})\n\n\tresult := runMigrationOnJSON(t, input)\n\n\t// Check that AutoConf section was added but is empty (using implicit defaults)\n\tautoConf, exists := result[\"AutoConf\"]\n\trequire.True(t, exists, \"AutoConf section should exist\")\n\tautoConfMap, ok := autoConf.(map[string]any)\n\trequire.True(t, ok, \"AutoConf should be a map\")\n\trequire.Empty(t, autoConfMap, \"AutoConf should be empty (using implicit defaults)\")\n\n\t// Check that Bootstrap was updated to \"auto\"\n\tassertSliceEquals(t, result, []string{\"Bootstrap\"}, []string{\"auto\"})\n}\n\n// =============================================================================\n// Bootstrap Migration Tests\n// =============================================================================\n\nfunc TestBootstrapMigration(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"process bootstrap peers logic verification\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tpeers []string\n\t\t\texpected []string\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"empty peers\",\n\t\t\t\tpeers: []string{},\n\t\t\t\texpected: []string{\"auto\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"only default peers\",\n\t\t\t\tpeers: []string{\n\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\t\t},\n\t\t\t\texpected: []string{\"auto\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"mixed default and custom peers\",\n\t\t\t\tpeers: []string{\n\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\"/ip4/192.168.1.1/tcp/4001/p2p/QmCustomPeer\",\n\t\t\t\t},\n\t\t\t\texpected: []string{\"auto\", \"/ip4/192.168.1.1/tcp/4001/p2p/QmCustomPeer\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"only custom peers\",\n\t\t\t\tpeers: []string{\n\t\t\t\t\t\"/ip4/192.168.1.1/tcp/4001/p2p/QmCustomPeer1\",\n\t\t\t\t\t\"/ip4/192.168.1.2/tcp/4001/p2p/QmCustomPeer2\",\n\t\t\t\t},\n\t\t\t\texpected: []string{\n\t\t\t\t\t\"/ip4/192.168.1.1/tcp/4001/p2p/QmCustomPeer1\",\n\t\t\t\t\t\"/ip4/192.168.1.2/tcp/4001/p2p/QmCustomPeer2\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tfor _, tt := range tests {\n\t\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresult := processBootstrapPeers(tt.peers)\n\t\t\t\trequire.Equal(t, len(tt.expected), len(result), \"Expected %d peers, got %d\", len(tt.expected), len(result))\n\t\t\t\tfor i, expected := range tt.expected {\n\t\t\t\t\tassert.Equal(t, expected, result[i], \"Expected peer %d to be %s\", i, expected)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"replaces all old default bootstrapper peers with auto entry\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n\t\t\t\t\"/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8\",\n\t\t\t\t\"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n\t\t\t\t\"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n\t\t\t},\n\t\t})\n\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertSliceEquals(t, result, []string{\"Bootstrap\"}, []string{\"auto\"})\n\t})\n\n\tt.Run(\"creates Bootstrap section with auto when missing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := `{\"Identity\": {\"PeerID\": \"QmTest\"}}`\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertSliceEquals(t, result, []string{\"Bootstrap\"}, []string{\"auto\"})\n\t})\n}\n\n// =============================================================================\n// DNS Migration Tests\n// =============================================================================\n\nfunc TestDNSMigration(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"creates DNS section with auto resolver when missing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := `{\"Identity\": {\"PeerID\": \"QmTest\"}}`\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertMapKeyEquals(t, result, []string{\"DNS\", \"Resolvers\"}, \".\", \"auto\")\n\t})\n\n\tt.Run(\"preserves all custom DNS resolvers unchanged\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"DNS\": map[string]any{\n\t\t\t\t\"Resolvers\": map[string]string{\n\t\t\t\t\t\".\": \"https://my-custom-resolver.com\",\n\t\t\t\t\t\".eth\": \"https://eth.resolver\",\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tresolvers := runMigrationAndGetDNSResolvers(t, input)\n\t\tassertResolvers(t, resolvers, map[string]string{\n\t\t\t\".\": \"https://my-custom-resolver.com\",\n\t\t\t\".eth\": \"https://eth.resolver\",\n\t\t})\n\t})\n\n\tt.Run(\"preserves custom dot and eth resolvers unchanged\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"DNS\": map[string]any{\n\t\t\t\t\"Resolvers\": map[string]string{\n\t\t\t\t\t\".\": \"https://cloudflare-dns.com/dns-query\",\n\t\t\t\t\t\".eth\": \"https://example.com/dns-query\",\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tresolvers := runMigrationAndGetDNSResolvers(t, input)\n\t\tassertResolvers(t, resolvers, map[string]string{\n\t\t\t\".\": \"https://cloudflare-dns.com/dns-query\",\n\t\t\t\".eth\": \"https://example.com/dns-query\",\n\t\t})\n\t})\n\n\tt.Run(\"replaces old default eth resolver with auto\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"DNS\": map[string]any{\n\t\t\t\t\"Resolvers\": map[string]string{\n\t\t\t\t\t\".\": \"https://cloudflare-dns.com/dns-query\",\n\t\t\t\t\t\".eth\": \"https://dns.eth.limo/dns-query\", // should be replaced\n\t\t\t\t\t\".crypto\": \"https://resolver.cloudflare-eth.com/dns-query\", // should be replaced\n\t\t\t\t\t\".link\": \"https://dns.eth.link/dns-query\", // should be replaced\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tresolvers := runMigrationAndGetDNSResolvers(t, input)\n\t\tassertResolvers(t, resolvers, map[string]string{\n\t\t\t\".\": \"https://cloudflare-dns.com/dns-query\", // preserved\n\t\t\t\".eth\": \"auto\", // replaced\n\t\t\t\".crypto\": \"auto\", // replaced\n\t\t\t\".link\": \"auto\", // replaced\n\t\t})\n\t})\n}\n\n// =============================================================================\n// Routing Migration Tests\n// =============================================================================\n\nfunc TestRoutingMigration(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"creates Routing section with auto DelegatedRouters when missing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := `{\"Identity\": {\"PeerID\": \"QmTest\"}}`\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertSliceEquals(t, result, []string{\"Routing\", \"DelegatedRouters\"}, []string{\"auto\"})\n\t})\n\n\tt.Run(\"replaces cid.contact with auto while preserving custom routers added by user\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"Routing\": map[string]any{\n\t\t\t\t\"DelegatedRouters\": []string{\n\t\t\t\t\t\"https://cid.contact\",\n\t\t\t\t\t\"https://my-custom-router.com\",\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertSliceEquals(t, result, []string{\"Routing\", \"DelegatedRouters\"}, []string{\"auto\", \"https://my-custom-router.com\"})\n\t})\n}\n\n// =============================================================================\n// IPNS Migration Tests\n// =============================================================================\n\nfunc TestIpnsMigration(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"creates Ipns section with auto DelegatedPublishers when missing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := `{\"Identity\": {\"PeerID\": \"QmTest\"}}`\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertSliceEquals(t, result, []string{\"Ipns\", \"DelegatedPublishers\"}, []string{\"auto\"})\n\t})\n\n\tt.Run(\"preserves existing custom DelegatedPublishers unchanged\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"Ipns\": map[string]any{\n\t\t\t\t\"DelegatedPublishers\": []string{\n\t\t\t\t\t\"https://my-publisher.com\",\n\t\t\t\t\t\"https://another-publisher.com\",\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertSliceEquals(t, result, []string{\"Ipns\", \"DelegatedPublishers\"}, []string{\"https://my-publisher.com\", \"https://another-publisher.com\"})\n\t})\n\n\tt.Run(\"adds auto DelegatedPublishers to existing Ipns section\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"Ipns\": map[string]any{\n\t\t\t\t\"ResolveCacheSize\": 128,\n\t\t\t},\n\t\t})\n\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertMapKeyEquals(t, result, []string{\"Ipns\"}, \"ResolveCacheSize\", float64(128))\n\t\tassertSliceEquals(t, result, []string{\"Ipns\", \"DelegatedPublishers\"}, []string{\"auto\"})\n\t})\n}\n\n// =============================================================================\n// AutoConf Migration Tests\n// =============================================================================\n\nfunc TestAutoConfMigration(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"preserves existing AutoConf fields unchanged\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tinput := buildTestConfig(map[string]any{\n\t\t\t\"AutoConf\": map[string]any{\n\t\t\t\t\"URL\": \"https://custom.example.com/autoconf.json\",\n\t\t\t\t\"Enabled\": false,\n\t\t\t\t\"CustomField\": \"preserved\",\n\t\t\t},\n\t\t})\n\n\t\tresult := runMigrationOnJSON(t, input)\n\t\tassertMapKeyEquals(t, result, []string{\"AutoConf\"}, \"URL\", \"https://custom.example.com/autoconf.json\")\n\t\tassertMapKeyEquals(t, result, []string{\"AutoConf\"}, \"Enabled\", false)\n\t\tassertMapKeyEquals(t, result, []string{\"AutoConf\"}, \"CustomField\", \"preserved\")\n\t})\n}\n" }, { "path": "repo/fsrepo/migrations/fs-repo-17-to-18/main.go", "content": "// Package main implements fs-repo-17-to-18 migration for IPFS repositories.\n//\n// This migration consolidates the Provider and Reprovider configurations into\n// a unified Provide configuration section.\n//\n// Changes made:\n// - Migrates Provider.Enabled to Provide.Enabled\n// - Migrates Provider.WorkerCount to Provide.DHT.MaxWorkers\n// - Migrates Reprovider.Strategy to Provide.Strategy (converts \"flat\" to \"all\")\n// - Migrates Reprovider.Interval to Provide.DHT.Interval\n// - Removes deprecated Provider and Reprovider sections\n//\n// The migration is reversible and creates config.17-to-18.bak for rollback.\n//\n// Usage:\n//\n//\tfs-repo-17-to-18 -path /path/to/ipfs/repo [-verbose] [-revert]\n//\n// This migration is embedded in Kubo and runs automatically during daemon startup.\n// This standalone binary is provided for manual migration scenarios.\npackage main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n\tmg17 \"github.com/ipfs/kubo/repo/fsrepo/migrations/fs-repo-17-to-18/migration\"\n)\n\nfunc main() {\n\tvar path = flag.String(\"path\", \"\", \"Path to IPFS repository\")\n\tvar verbose = flag.Bool(\"verbose\", false, \"Enable verbose output\")\n\tvar revert = flag.Bool(\"revert\", false, \"Revert migration\")\n\tflag.Parse()\n\n\tif *path == \"\" {\n\t\tfmt.Fprintf(os.Stderr, \"Error: -path flag is required\\n\")\n\t\tflag.Usage()\n\t\tos.Exit(1)\n\t}\n\n\topts := common.Options{\n\t\tPath: *path,\n\t\tVerbose: *verbose,\n\t}\n\n\tvar err error\n\tif *revert {\n\t\terr = mg17.Migration.Revert(opts)\n\t} else {\n\t\terr = mg17.Migration.Apply(opts)\n\t}\n\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"Migration failed: %v\\n\", err)\n\t\tos.Exit(1)\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/fs-repo-17-to-18/migration/migration.go", "content": "// package mg17 contains the code to perform 17-18 repository migration in Kubo.\n// This handles the following:\n// - Migrate Provider and Reprovider configs to unified Provide config\n// - Clear deprecated Provider and Reprovider fields\n// - Increment repo version to 18\npackage mg17\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n)\n\n// Migration is the main exported migration for 17-to-18\nvar Migration = &common.BaseMigration{\n\tFromVersion: \"17\",\n\tToVersion: \"18\",\n\tDescription: \"Migrating Provider and Reprovider configuration to unified Provide configuration\",\n\tConvert: convert,\n}\n\n// NewMigration creates a new migration instance (for compatibility)\nfunc NewMigration() common.Migration {\n\treturn Migration\n}\n\n// convert performs the actual configuration transformation\nfunc convert(in io.ReadSeeker, out io.Writer) error {\n\t// Read the configuration\n\tconfMap, err := common.ReadConfig(in)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Create new Provide section with DHT subsection from Provider and Reprovider\n\tprovide := make(map[string]any)\n\tdht := make(map[string]any)\n\thasNonDefaultValues := false\n\n\t// Migrate Provider fields if they exist\n\tprovider := common.SafeCastMap(confMap[\"Provider\"])\n\tif enabled, exists := provider[\"Enabled\"]; exists {\n\t\tprovide[\"Enabled\"] = enabled\n\t\t// Log migration for non-default values\n\t\tif enabledBool, ok := enabled.(bool); ok && !enabledBool {\n\t\t\tfmt.Printf(\" Migrated Provider.Enabled=%v to Provide.Enabled=%v\\n\", enabledBool, enabledBool)\n\t\t\thasNonDefaultValues = true\n\t\t}\n\t}\n\tif workerCount, exists := provider[\"WorkerCount\"]; exists {\n\t\tdht[\"MaxWorkers\"] = workerCount\n\t\t// Log migration for all worker count values\n\t\tif count, ok := workerCount.(float64); ok {\n\t\t\tfmt.Printf(\" Migrated Provider.WorkerCount=%v to Provide.DHT.MaxWorkers=%v\\n\", int(count), int(count))\n\t\t\thasNonDefaultValues = true\n\n\t\t\t// Additional guidance for high WorkerCount\n\t\t\tif count > 5 {\n\t\t\t\tfmt.Printf(\" ⚠️ For better resource utilization, consider enabling Provide.DHT.SweepEnabled=true\\n\")\n\t\t\t\tfmt.Printf(\" and adjusting Provide.DHT.DedicatedBurstWorkers if announcement of new CIDs\\n\")\n\t\t\t\tfmt.Printf(\" should take priority over periodic reprovide interval.\\n\")\n\t\t\t}\n\t\t}\n\t}\n\t// Note: Skip Provider.Strategy as it was unused\n\n\t// Migrate Reprovider fields if they exist\n\treprovider := common.SafeCastMap(confMap[\"Reprovider\"])\n\tif strategy, exists := reprovider[\"Strategy\"]; exists {\n\t\tif strategyStr, ok := strategy.(string); ok {\n\t\t\t// Convert deprecated \"flat\" strategy to \"all\"\n\t\t\tif strategyStr == \"flat\" {\n\t\t\t\tprovide[\"Strategy\"] = \"all\"\n\t\t\t\tfmt.Printf(\" Migrated deprecated Reprovider.Strategy=\\\"flat\\\" to Provide.Strategy=\\\"all\\\"\\n\")\n\t\t\t} else {\n\t\t\t\t// Migrate any other strategy value as-is\n\t\t\t\tprovide[\"Strategy\"] = strategyStr\n\t\t\t\tfmt.Printf(\" Migrated Reprovider.Strategy=\\\"%s\\\" to Provide.Strategy=\\\"%s\\\"\\n\", strategyStr, strategyStr)\n\t\t\t}\n\t\t\thasNonDefaultValues = true\n\t\t} else {\n\t\t\t// Not a string, set to default \"all\" to ensure valid config\n\t\t\tprovide[\"Strategy\"] = \"all\"\n\t\t\tfmt.Printf(\" Warning: Reprovider.Strategy was not a string, setting Provide.Strategy=\\\"all\\\"\\n\")\n\t\t\thasNonDefaultValues = true\n\t\t}\n\t}\n\tif interval, exists := reprovider[\"Interval\"]; exists {\n\t\tdht[\"Interval\"] = interval\n\t\t// Log migration for non-default intervals\n\t\tif intervalStr, ok := interval.(string); ok && intervalStr != \"22h\" && intervalStr != \"\" {\n\t\t\tfmt.Printf(\" Migrated Reprovider.Interval=\\\"%s\\\" to Provide.DHT.Interval=\\\"%s\\\"\\n\", intervalStr, intervalStr)\n\t\t\thasNonDefaultValues = true\n\t\t}\n\t}\n\t// Note: Sweep is a new field introduced in v0.38, not present in v0.37\n\t// So we don't need to migrate it from Reprovider\n\n\t// Set the DHT section if we have any DHT fields to migrate\n\tif len(dht) > 0 {\n\t\tprovide[\"DHT\"] = dht\n\t}\n\n\t// Set the new Provide section if we have any fields to migrate\n\tif len(provide) > 0 {\n\t\tconfMap[\"Provide\"] = provide\n\t}\n\n\t// Clear old Provider and Reprovider sections\n\tdelete(confMap, \"Provider\")\n\tdelete(confMap, \"Reprovider\")\n\n\t// Print documentation link if we migrated any non-default values\n\tif hasNonDefaultValues {\n\t\tfmt.Printf(\" See: https://github.com/ipfs/kubo/blob/master/docs/config.md#provide\\n\")\n\t}\n\n\t// Write the updated config\n\treturn common.WriteConfig(out, confMap)\n}\n" }, { "path": "repo/fsrepo/migrations/fs-repo-17-to-18/migration/migration_test.go", "content": "package mg17\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations/common\"\n)\n\nfunc TestMigration17to18(t *testing.T) {\n\tmigration := NewMigration()\n\n\ttestCases := []common.TestCase{\n\t\t{\n\t\t\tName: \"Migrate Provider and Reprovider to Provide\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Provider\": map[string]any{\n\t\t\t\t\t\"Enabled\": true,\n\t\t\t\t\t\"WorkerCount\": 8,\n\t\t\t\t\t\"Strategy\": \"unused\", // This field was unused and should be ignored\n\t\t\t\t},\n\t\t\t\t\"Reprovider\": map[string]any{\n\t\t\t\t\t\"Strategy\": \"pinned\",\n\t\t\t\t\t\"Interval\": \"12h\",\n\t\t\t\t},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide.Enabled\", Expected: true},\n\t\t\t\t{Path: \"Provide.DHT.MaxWorkers\", Expected: float64(8)}, // JSON unmarshals to float64\n\t\t\t\t{Path: \"Provide.Strategy\", Expected: \"pinned\"},\n\t\t\t\t{Path: \"Provide.DHT.Interval\", Expected: \"12h\"},\n\t\t\t\t{Path: \"Provider\", Expected: nil}, // Should be deleted\n\t\t\t\t{Path: \"Reprovider\", Expected: nil}, // Should be deleted\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"Convert flat strategy to all\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Provider\": map[string]any{\n\t\t\t\t\t\"Enabled\": false,\n\t\t\t\t},\n\t\t\t\t\"Reprovider\": map[string]any{\n\t\t\t\t\t\"Strategy\": \"flat\", // Deprecated, should be converted to \"all\"\n\t\t\t\t\t\"Interval\": \"24h\",\n\t\t\t\t},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide.Enabled\", Expected: false},\n\t\t\t\t{Path: \"Provide.Strategy\", Expected: \"all\"}, // \"flat\" converted to \"all\"\n\t\t\t\t{Path: \"Provide.DHT.Interval\", Expected: \"24h\"},\n\t\t\t\t{Path: \"Provider\", Expected: nil},\n\t\t\t\t{Path: \"Reprovider\", Expected: nil},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"Handle missing Provider section\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Reprovider\": map[string]any{\n\t\t\t\t\t\"Strategy\": \"roots\",\n\t\t\t\t\t\"Interval\": \"6h\",\n\t\t\t\t},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide.Strategy\", Expected: \"roots\"},\n\t\t\t\t{Path: \"Provide.DHT.Interval\", Expected: \"6h\"},\n\t\t\t\t{Path: \"Provider\", Expected: nil},\n\t\t\t\t{Path: \"Reprovider\", Expected: nil},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"Handle missing Reprovider section\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Provider\": map[string]any{\n\t\t\t\t\t\"Enabled\": true,\n\t\t\t\t\t\"WorkerCount\": 16,\n\t\t\t\t},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide.Enabled\", Expected: true},\n\t\t\t\t{Path: \"Provide.DHT.MaxWorkers\", Expected: float64(16)},\n\t\t\t\t{Path: \"Provider\", Expected: nil},\n\t\t\t\t{Path: \"Reprovider\", Expected: nil},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"Handle empty Provider and Reprovider sections\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Provider\": map[string]any{},\n\t\t\t\t\"Reprovider\": map[string]any{},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide\", Expected: nil}, // No fields to migrate\n\t\t\t\t{Path: \"Provider\", Expected: nil},\n\t\t\t\t{Path: \"Reprovider\", Expected: nil},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"Handle missing both sections\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Datastore\": map[string]any{\n\t\t\t\t\t\"StorageMax\": \"10GB\",\n\t\t\t\t},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide\", Expected: nil}, // No Provider/Reprovider to migrate\n\t\t\t\t{Path: \"Provider\", Expected: nil},\n\t\t\t\t{Path: \"Reprovider\", Expected: nil},\n\t\t\t\t{Path: \"Datastore.StorageMax\", Expected: \"10GB\"}, // Other config preserved\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"Preserve other config sections\",\n\t\t\tInputConfig: common.GenerateTestConfig(map[string]any{\n\t\t\t\t\"Provider\": map[string]any{\n\t\t\t\t\t\"Enabled\": true,\n\t\t\t\t},\n\t\t\t\t\"Reprovider\": map[string]any{\n\t\t\t\t\t\"Strategy\": \"all\",\n\t\t\t\t},\n\t\t\t\t\"Swarm\": map[string]any{\n\t\t\t\t\t\"ConnMgr\": map[string]any{\n\t\t\t\t\t\t\"Type\": \"basic\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}),\n\t\t\tAssertions: []common.ConfigAssertion{\n\t\t\t\t{Path: \"Provide.Enabled\", Expected: true},\n\t\t\t\t{Path: \"Provide.Strategy\", Expected: \"all\"},\n\t\t\t\t{Path: \"Swarm.ConnMgr.Type\", Expected: \"basic\"}, // Other config preserved\n\t\t\t\t{Path: \"Provider\", Expected: nil},\n\t\t\t\t{Path: \"Reprovider\", Expected: nil},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.Name, func(t *testing.T) {\n\t\t\tcommon.RunMigrationTest(t, migration, tc)\n\t\t})\n\t}\n}\n\nfunc TestMigration17to18Reversible(t *testing.T) {\n\tmigration := NewMigration()\n\n\t// Test that migration is reversible\n\tinputConfig := common.GenerateTestConfig(map[string]any{\n\t\t\"Provide\": map[string]any{\n\t\t\t\"Enabled\": true,\n\t\t\t\"WorkerCount\": 8,\n\t\t\t\"Strategy\": \"pinned\",\n\t\t\t\"Interval\": \"12h\",\n\t\t},\n\t})\n\n\t// Test full migration and revert\n\tmigratedConfig := common.AssertMigrationSuccess(t, migration, 17, 18, inputConfig)\n\n\t// Check that Provide section exists after migration\n\tcommon.AssertConfigField(t, migratedConfig, \"Provide.Enabled\", true)\n\n\t// Test revert\n\tcommon.AssertMigrationReversible(t, migration, 17, 18, migratedConfig)\n}\n\nfunc TestMigration17to18Integration(t *testing.T) {\n\tmigration := NewMigration()\n\n\t// Test that the migration properly integrates with the common framework\n\tif migration.Versions() != \"17-to-18\" {\n\t\tt.Errorf(\"expected versions '17-to-18', got '%s'\", migration.Versions())\n\t}\n\n\tif !migration.Reversible() {\n\t\tt.Error(\"migration should be reversible\")\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/httpfetcher.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\tgopath \"path\"\n\t\"strings\"\n\n\t\"github.com/ipfs/boxo/blockservice\"\n\t\"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/exchange/offline\"\n\tbsfetcher \"github.com/ipfs/boxo/fetcher/impl/blockservice\"\n\tfiles \"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/ipld/merkledag\"\n\tunixfile \"github.com/ipfs/boxo/ipld/unixfs/file\"\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/namesys\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/boxo/path/resolver\"\n\t\"github.com/ipfs/go-datastore\"\n\tdssync \"github.com/ipfs/go-datastore/sync\"\n\t\"github.com/ipfs/go-unixfsnode\"\n\tgocarv2 \"github.com/ipld/go-car/v2\"\n\tdagpb \"github.com/ipld/go-codec-dagpb\"\n\tmadns \"github.com/multiformats/go-multiaddr-dns\"\n)\n\nconst (\n\t// default is different name than ipfs.io which is being blocked by some ISPs\n\tdefaultGatewayURL = \"https://trustless-gateway.link\"\n\t// Default maximum download size.\n\tdefaultFetchLimit = 1024 * 1024 * 512\n)\n\n// HttpFetcher fetches files over HTTP using verifiable CAR archives.\ntype HttpFetcher struct { //nolint\n\tdistPath string\n\tgateway string\n\tlimit int64\n\tuserAgent string\n}\n\nvar _ Fetcher = (*HttpFetcher)(nil)\n\n// NewHttpFetcher creates a new [HttpFetcher].\n//\n// Specifying \"\" for distPath sets the default IPNS path.\n// Specifying \"\" for gateway sets the default.\n// Specifying 0 for fetchLimit sets the default, -1 means no limit.\nfunc NewHttpFetcher(distPath, gateway, userAgent string, fetchLimit int64) *HttpFetcher { //nolint\n\tf := &HttpFetcher{\n\t\tdistPath: LatestIpfsDist,\n\t\tgateway: defaultGatewayURL,\n\t\tlimit: defaultFetchLimit,\n\t}\n\n\tif distPath != \"\" {\n\t\tif !strings.HasPrefix(distPath, \"/\") {\n\t\t\tdistPath = \"/\" + distPath\n\t\t}\n\t\tf.distPath = distPath\n\t}\n\n\tif gateway != \"\" {\n\t\tf.gateway = strings.TrimRight(gateway, \"/\")\n\t}\n\n\tif fetchLimit != 0 {\n\t\tif fetchLimit < 0 {\n\t\t\tfetchLimit = 0\n\t\t}\n\t\tf.limit = fetchLimit\n\t}\n\n\treturn f\n}\n\n// Fetch attempts to fetch the file at the given path, from the distribution\n// site configured for this HttpFetcher.\nfunc (f *HttpFetcher) Fetch(ctx context.Context, filePath string) ([]byte, error) {\n\timPath, err := f.resolvePath(ctx, gopath.Join(f.distPath, filePath))\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"path could not be resolved: %w\", err)\n\t}\n\n\trc, err := f.httpRequest(ctx, imPath, \"application/vnd.ipld.car\")\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to fetch CAR: %w\", err)\n\t}\n\n\treturn carStreamToFileBytes(ctx, rc, imPath)\n}\n\nfunc (f *HttpFetcher) Close() error {\n\treturn nil\n}\n\nfunc (f *HttpFetcher) resolvePath(ctx context.Context, pathStr string) (path.ImmutablePath, error) {\n\tp, err := path.NewPath(pathStr)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, fmt.Errorf(\"path is invalid: %w\", err)\n\t}\n\n\tfor p.Mutable() {\n\t\t// Download IPNS record and verify through the gateway, or resolve the\n\t\t// DNSLink with the default DNS resolver.\n\t\tname, err := ipns.NameFromString(p.Segments()[1])\n\t\tif err == nil {\n\t\t\tp, err = f.resolveIPNS(ctx, name)\n\t\t} else {\n\t\t\tp, err = f.resolveDNSLink(ctx, p)\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn path.ImmutablePath{}, err\n\t\t}\n\t}\n\n\treturn path.NewImmutablePath(p)\n}\n\nfunc (f *HttpFetcher) resolveIPNS(ctx context.Context, name ipns.Name) (path.Path, error) {\n\trc, err := f.httpRequest(ctx, name.AsPath(), \"application/vnd.ipfs.ipns-record\")\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\trc = NewLimitReadCloser(rc, int64(ipns.MaxRecordSize))\n\trawRecord, err := io.ReadAll(rc)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\trec, err := ipns.UnmarshalRecord(rawRecord)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\terr = ipns.ValidateWithName(rec, name)\n\tif err != nil {\n\t\treturn path.ImmutablePath{}, err\n\t}\n\n\treturn rec.Value()\n}\n\nfunc (f *HttpFetcher) resolveDNSLink(ctx context.Context, p path.Path) (path.Path, error) {\n\tdnsResolver := namesys.NewDNSResolver(madns.DefaultResolver.LookupTXT)\n\tres, err := dnsResolver.Resolve(ctx, p)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn res.Path, nil\n}\n\nfunc (f *HttpFetcher) httpRequest(ctx context.Context, p path.Path, accept string) (io.ReadCloser, error) {\n\turl := f.gateway + p.String()\n\tfmt.Printf(\"Fetching with HTTP: %q\\n\", url)\n\treq, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"http.NewRequest error: %w\", err)\n\t}\n\treq.Header.Set(\"Accept\", accept)\n\n\tif f.userAgent != \"\" {\n\t\treq.Header.Set(\"User-Agent\", f.userAgent)\n\t}\n\n\tresp, err := http.DefaultClient.Do(req)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"http.DefaultClient.Do error: %w\", err)\n\t}\n\n\tif resp.StatusCode >= 400 {\n\t\tdefer resp.Body.Close()\n\t\tmes, err := io.ReadAll(resp.Body)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error reading error body: %w\", err)\n\t\t}\n\t\treturn nil, fmt.Errorf(\"GET %s error: %s: %s\", url, resp.Status, string(mes))\n\t}\n\n\tvar rc io.ReadCloser\n\tif f.limit != 0 {\n\t\trc = NewLimitReadCloser(resp.Body, f.limit)\n\t} else {\n\t\trc = resp.Body\n\t}\n\n\treturn rc, nil\n}\n\nfunc carStreamToFileBytes(ctx context.Context, r io.ReadCloser, imPath path.ImmutablePath) ([]byte, error) {\n\tdefer r.Close()\n\n\t// Create temporary block datastore and dag service.\n\tdataStore := dssync.MutexWrap(datastore.NewMapDatastore())\n\tblockStore := blockstore.NewBlockstore(dataStore)\n\tblockService := blockservice.New(blockStore, offline.Exchange(blockStore))\n\tdagService := merkledag.NewDAGService(blockService)\n\n\tdefer dagService.Blocks.Close()\n\tdefer dataStore.Close()\n\n\t// Create CAR reader\n\tcar, err := gocarv2.NewBlockReader(r)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn nil, fmt.Errorf(\"error creating car reader: %s\", err)\n\t}\n\n\t// Add all blocks to the blockstore.\n\tfor {\n\t\tblock, err := car.Next()\n\t\tif err != nil && err != io.EOF {\n\t\t\treturn nil, fmt.Errorf(\"error reading block from car: %s\", err)\n\t\t} else if block == nil {\n\t\t\tbreak\n\t\t}\n\n\t\terr = blockStore.Put(ctx, block)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error putting block in blockstore: %s\", err)\n\t\t}\n\t}\n\n\tfetcherCfg := bsfetcher.NewFetcherConfig(blockService)\n\tfetcherCfg.PrototypeChooser = dagpb.AddSupportToChooser(bsfetcher.DefaultPrototypeChooser)\n\tfetcher := fetcherCfg.WithReifier(unixfsnode.Reify)\n\tresolver := resolver.NewBasicResolver(fetcher)\n\n\tcid, _, err := resolver.ResolveToLastNode(ctx, imPath)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to resolve: %w\", err)\n\t}\n\n\tnd, err := dagService.Get(ctx, cid)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to resolve: %w\", err)\n\t}\n\n\t// Make UnixFS file out of the node.\n\tuf, err := unixfile.NewUnixfsFile(ctx, dagService, nd)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error building unixfs file: %s\", err)\n\t}\n\n\t// Check if it's a file and return.\n\tif f, ok := uf.(files.File); ok {\n\t\treturn io.ReadAll(f)\n\t}\n\n\treturn nil, errors.New(\"unexpected unixfs node type\")\n}\n" }, { "path": "repo/fsrepo/migrations/ipfsdir.go", "content": "package migrations\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/misc/fsutil\"\n)\n\nconst (\n\tversionFile = \"version\"\n)\n\n// IpfsDir returns the path of the ipfs directory. If dir specified, then\n// returns the expanded version dir. If dir is \"\", then return the directory\n// set by IPFS_PATH, or if IPFS_PATH is not set, then return the default\n// location in the home directory.\nfunc IpfsDir(dir string) (string, error) {\n\tvar err error\n\tif dir == \"\" {\n\t\tdir, err = config.PathRoot()\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t}\n\tdir, err = fsutil.ExpandHome(dir)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn dir, nil\n}\n\n// CheckIpfsDir gets the ipfs directory and checks that the directory exists.\nfunc CheckIpfsDir(dir string) (string, error) {\n\tvar err error\n\tdir, err = IpfsDir(dir)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t_, err = os.Stat(dir)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn dir, nil\n}\n\n// RepoVersion returns the version of the repo in the ipfs directory. If the\n// ipfs directory is not specified then the default location is used.\nfunc RepoVersion(ipfsDir string) (int, error) {\n\tipfsDir, err := CheckIpfsDir(ipfsDir)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\treturn repoVersion(ipfsDir)\n}\n\n// WriteRepoVersion writes the specified repo version to the repo located in\n// ipfsDir. If ipfsDir is not specified, then the default location is used.\nfunc WriteRepoVersion(ipfsDir string, version int) error {\n\tipfsDir, err := IpfsDir(ipfsDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvFilePath := filepath.Join(ipfsDir, versionFile)\n\treturn os.WriteFile(vFilePath, fmt.Appendf(nil, \"%d\\n\", version), 0o644)\n}\n\nfunc repoVersion(ipfsDir string) (int, error) {\n\tc, err := os.ReadFile(filepath.Join(ipfsDir, versionFile))\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tver, err := strconv.Atoi(strings.TrimSpace(string(c)))\n\tif err != nil {\n\t\treturn 0, errors.New(\"invalid data in repo version file\")\n\t}\n\treturn ver, nil\n}\n" }, { "path": "repo/fsrepo/migrations/ipfsdir_test.go", "content": "package migrations\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/config\"\n)\n\nfunc TestRepoDir(t *testing.T) {\n\tfakeHome := t.TempDir()\n\tt.Setenv(\"HOME\", fakeHome)\n\t// On Windows, os.UserHomeDir() uses USERPROFILE, not HOME\n\tt.Setenv(\"USERPROFILE\", fakeHome)\n\tfakeIpfs := filepath.Join(fakeHome, \".ipfs\")\n\tt.Setenv(config.EnvDir, fakeIpfs)\n\n\tt.Run(\"testIpfsDir\", func(t *testing.T) {\n\t\ttestIpfsDir(t, fakeIpfs)\n\t})\n\tt.Run(\"testCheckIpfsDir\", func(t *testing.T) {\n\t\ttestCheckIpfsDir(t, fakeIpfs)\n\t})\n\tt.Run(\"testRepoVersion\", func(t *testing.T) {\n\t\ttestRepoVersion(t, fakeIpfs)\n\t})\n}\n\nfunc testIpfsDir(t *testing.T, fakeIpfs string) {\n\t_, err := CheckIpfsDir(\"\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error when no .ipfs directory to find\")\n\t}\n\n\terr = os.Mkdir(fakeIpfs, os.ModePerm)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tdir, err := IpfsDir(\"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif dir != fakeIpfs {\n\t\tt.Fatalf(\"wrong ipfs directory: got %s, expected %s\", dir, fakeIpfs)\n\t}\n\n\tt.Setenv(config.EnvDir, \"~/.ipfs\")\n\tdir, err = IpfsDir(\"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif dir != fakeIpfs {\n\t\tt.Fatalf(\"wrong ipfs directory: got %s, expected %s\", dir, fakeIpfs)\n\t}\n\n\t_, err = IpfsDir(\"~somesuer/foo\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error with user-specific home dir\")\n\t}\n\n\tt.Setenv(config.EnvDir, \"~somesuer/foo\")\n\t_, err = IpfsDir(\"~somesuer/foo\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error with user-specific home dir\")\n\t}\n\terr = os.Unsetenv(config.EnvDir)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tdir, err = IpfsDir(\"~/.ipfs\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif dir != fakeIpfs {\n\t\tt.Fatalf(\"wrong ipfs directory: got %s, expected %s\", dir, fakeIpfs)\n\t}\n\n\t_, err = IpfsDir(\"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc testCheckIpfsDir(t *testing.T, fakeIpfs string) {\n\t_, err := CheckIpfsDir(\"~somesuer/foo\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error with user-specific home dir\")\n\t}\n\n\t_, err = CheckIpfsDir(\"~/no_such_dir\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error from nonexistent directory\")\n\t}\n\n\tdir, err := CheckIpfsDir(\"~/.ipfs\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif dir != fakeIpfs {\n\t\tt.Fatal(\"wrong ipfs directory:\", dir)\n\t}\n}\n\nfunc testRepoVersion(t *testing.T, fakeIpfs string) {\n\tbadDir := \"~somesuer/foo\"\n\t_, err := RepoVersion(badDir)\n\tif err == nil {\n\t\tt.Fatal(\"expected error with user-specific home dir\")\n\t}\n\n\t_, err = RepoVersion(fakeIpfs)\n\tif !os.IsNotExist(err) {\n\t\tt.Fatal(\"expected not-exist error\")\n\t}\n\n\ttestVer := 42\n\terr = WriteRepoVersion(fakeIpfs, testVer)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar ver int\n\tver, err = RepoVersion(fakeIpfs)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif ver != testVer {\n\t\tt.Fatalf(\"expected version %d, got %d\", testVer, ver)\n\t}\n\n\terr = WriteRepoVersion(badDir, testVer)\n\tif err == nil {\n\t\tt.Fatal(\"expected error with user-specific home dir\")\n\t}\n\n\tipfsDir, err := IpfsDir(fakeIpfs)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tvFilePath := filepath.Join(ipfsDir, versionFile)\n\terr = os.WriteFile(vFilePath, []byte(\"bad-version-data\\n\"), 0o644)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\t_, err = RepoVersion(fakeIpfs)\n\tif err == nil || err.Error() != \"invalid data in repo version file\" {\n\t\tt.Fatal(\"expected 'invalid data' error\")\n\t}\n\terr = WriteRepoVersion(fakeIpfs, testVer)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n" }, { "path": "repo/fsrepo/migrations/ipfsfetcher/ipfsfetcher.go", "content": "package ipfsfetcher\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/url\"\n\t\"os\"\n\tgopath \"path\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/boxo/path\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\tiface \"github.com/ipfs/kubo/core/coreiface\"\n\t\"github.com/ipfs/kubo/core/coreiface/options\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/repo/fsrepo\"\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n)\n\nconst (\n\t// Default maximum download size.\n\tdefaultFetchLimit = 1024 * 1024 * 512\n\n\ttempNodeTCPAddr = \"/ip4/127.0.0.1/tcp/0\"\n)\n\ntype IpfsFetcher struct {\n\tdistPath string\n\tlimit int64\n\trepoRoot *string\n\tuserConfigFile string\n\n\topenOnce sync.Once\n\topenErr error\n\tcloseOnce sync.Once\n\tcloseErr error\n\n\tipfs iface.CoreAPI\n\tipfsTmpDir string\n\tipfsStopFunc func()\n\n\tfetched []path.Path\n\tmutex sync.Mutex\n\n\taddrInfo peer.AddrInfo\n}\n\nvar _ migrations.Fetcher = (*IpfsFetcher)(nil)\n\n// NewIpfsFetcher creates a new IpfsFetcher\n//\n// Specifying \"\" for distPath sets the default IPNS path.\n// Specifying 0 for fetchLimit sets the default, -1 means no limit.\n//\n// Bootstrap and peer information in read from the IPFS config file in\n// repoRoot, unless repoRoot is nil. If repoRoot is empty (\"\"), then read the\n// config from the default IPFS directory.\nfunc NewIpfsFetcher(distPath string, fetchLimit int64, repoRoot *string, userConfigFile string) *IpfsFetcher {\n\tf := &IpfsFetcher{\n\t\tlimit: defaultFetchLimit,\n\t\tdistPath: migrations.LatestIpfsDist,\n\t\trepoRoot: repoRoot,\n\t\tuserConfigFile: userConfigFile,\n\t}\n\n\tif distPath != \"\" {\n\t\tif !strings.HasPrefix(distPath, \"/\") {\n\t\t\tdistPath = \"/\" + distPath\n\t\t}\n\t\tf.distPath = distPath\n\t}\n\n\tif fetchLimit != 0 {\n\t\tif fetchLimit < 0 {\n\t\t\tfetchLimit = 0\n\t\t}\n\t\tf.limit = fetchLimit\n\t}\n\n\treturn f\n}\n\n// Fetch attempts to fetch the file at the given path, from the distribution\n// site configured for this HttpFetcher.\nfunc (f *IpfsFetcher) Fetch(ctx context.Context, filePath string) ([]byte, error) {\n\t// Initialize and start IPFS node on first call to Fetch, since the fetcher\n\t// may be created by not used.\n\tf.openOnce.Do(func() {\n\t\tbootstrap, peers := readIpfsConfig(f.repoRoot, f.userConfigFile)\n\t\tf.ipfsTmpDir, f.openErr = initTempNode(ctx, bootstrap, peers)\n\t\tif f.openErr != nil {\n\t\t\treturn\n\t\t}\n\n\t\tf.openErr = f.startTempNode(ctx)\n\t})\n\n\tfmt.Printf(\"Fetching with IPFS: %q\\n\", filePath)\n\n\tif f.openErr != nil {\n\t\treturn nil, f.openErr\n\t}\n\n\tiPath, err := parsePath(gopath.Join(f.distPath, filePath))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnd, err := f.ipfs.Unixfs().Get(ctx, iPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tf.recordFetched(iPath)\n\n\tfileNode, ok := nd.(files.File)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"%q is not a file\", filePath)\n\t}\n\n\tvar rc io.ReadCloser\n\tif f.limit != 0 {\n\t\trc = migrations.NewLimitReadCloser(fileNode, f.limit)\n\t} else {\n\t\trc = fileNode\n\t}\n\tdefer rc.Close()\n\n\treturn io.ReadAll(rc)\n}\n\nfunc (f *IpfsFetcher) Close() error {\n\tf.closeOnce.Do(func() {\n\t\tif f.ipfsStopFunc != nil {\n\t\t\t// Tell ipfs node to stop and wait for it to stop\n\t\t\tf.ipfsStopFunc()\n\t\t}\n\n\t\tif f.ipfsTmpDir != \"\" {\n\t\t\t// Remove the temp ipfs dir\n\t\t\tf.closeErr = os.RemoveAll(f.ipfsTmpDir)\n\t\t}\n\t})\n\treturn f.closeErr\n}\n\nfunc (f *IpfsFetcher) AddrInfo() peer.AddrInfo {\n\treturn f.addrInfo\n}\n\n// FetchedPaths returns the IPFS paths of all items fetched by this fetcher.\nfunc (f *IpfsFetcher) FetchedPaths() []path.Path {\n\tf.mutex.Lock()\n\tdefer f.mutex.Unlock()\n\treturn f.fetched\n}\n\nfunc (f *IpfsFetcher) recordFetched(fetchedPath path.Path) {\n\t// Mutex protects against update by concurrent calls to Fetch\n\tf.mutex.Lock()\n\tdefer f.mutex.Unlock()\n\tf.fetched = append(f.fetched, fetchedPath)\n}\n\nfunc initTempNode(ctx context.Context, bootstrap []string, peers []peer.AddrInfo) (string, error) {\n\tidentity, err := config.CreateIdentity(io.Discard, []options.KeyGenerateOption{\n\t\toptions.Key.Type(options.Ed25519Key),\n\t})\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tcfg, err := config.InitWithIdentity(identity)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// create temporary ipfs directory\n\tdir, err := os.MkdirTemp(\"\", \"ipfs-temp\")\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to get temp dir: %s\", err)\n\t}\n\n\t// configure the temporary node\n\tcfg.Routing.Type = config.NewOptionalString(\"dhtclient\")\n\n\t// Disable listening for inbound connections\n\tcfg.Addresses.Gateway = []string{}\n\tcfg.Addresses.API = []string{}\n\tcfg.Addresses.Swarm = []string{tempNodeTCPAddr}\n\n\tif len(bootstrap) != 0 {\n\t\tcfg.Bootstrap = bootstrap\n\t}\n\n\tif len(peers) != 0 {\n\t\tcfg.Peering.Peers = peers\n\t}\n\n\t// Assumes that repo plugins are already loaded\n\terr = fsrepo.Init(dir, cfg)\n\tif err != nil {\n\t\tos.RemoveAll(dir)\n\t\treturn \"\", fmt.Errorf(\"failed to initialize ephemeral node: %s\", err)\n\t}\n\n\treturn dir, nil\n}\n\nfunc (f *IpfsFetcher) startTempNode(ctx context.Context) error {\n\t// Open the repo\n\tr, err := fsrepo.Open(f.ipfsTmpDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Create a new lifetime context that is used to stop the temp ipfs node\n\tctxIpfsLife, cancel := context.WithCancel(context.Background())\n\n\t// Construct the node\n\tnode, err := core.NewNode(ctxIpfsLife, &core.BuildCfg{\n\t\tOnline: true,\n\t\tRouting: libp2p.DHTClientOption,\n\t\tRepo: r,\n\t})\n\tif err != nil {\n\t\tcancel()\n\t\tr.Close()\n\t\treturn err\n\t}\n\n\tipfs, err := coreapi.NewCoreAPI(node)\n\tif err != nil {\n\t\tcancel()\n\t\treturn err\n\t}\n\n\tstopFunc := func() {\n\t\t// Tell ipfs to stop\n\t\tcancel()\n\t\t// Wait until ipfs is stopped\n\t\t<-node.Context().Done()\n\t}\n\n\taddrs, err := ipfs.Swarm().LocalAddrs(ctx)\n\tif err != nil {\n\t\t// Failure to get the local swarm address only means that the\n\t\t// downloaded migrations cannot be fetched through the temporary node.\n\t\t// So, print the error message and keep going.\n\t\tfmt.Fprintln(os.Stderr, \"cannot get local swarm address:\", err)\n\t}\n\n\tf.addrInfo = peer.AddrInfo{\n\t\tID: node.Identity,\n\t\tAddrs: addrs,\n\t}\n\n\tf.ipfs = ipfs\n\tf.ipfsStopFunc = stopFunc\n\n\treturn nil\n}\n\nfunc parsePath(fetchPath string) (path.Path, error) {\n\tif ipfsPath, err := path.NewPath(fetchPath); err == nil {\n\t\treturn ipfsPath, nil\n\t}\n\n\tu, err := url.Parse(fetchPath)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"%q could not be parsed: %s\", fetchPath, err)\n\t}\n\n\tswitch proto := u.Scheme; proto {\n\tcase \"ipfs\", \"ipld\", \"ipns\":\n\t\treturn path.NewPath(gopath.Join(\"/\", proto, u.Host, u.Path))\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"%q is not an IPFS path\", fetchPath)\n\t}\n}\n\nfunc readIpfsConfig(repoRoot *string, userConfigFile string) (bootstrap []string, peers []peer.AddrInfo) {\n\tif repoRoot == nil {\n\t\treturn\n\t}\n\n\tcfgPath, err := config.Filename(*repoRoot, userConfigFile)\n\tif err != nil {\n\t\tfmt.Fprintln(os.Stderr, err)\n\t\treturn\n\t}\n\n\tcfgFile, err := os.Open(cfgPath)\n\tif err != nil {\n\t\tfmt.Fprintln(os.Stderr, err)\n\t\treturn\n\t}\n\tdefer cfgFile.Close()\n\n\t// Attempt to read bootstrap addresses\n\tvar bootstrapCfg struct {\n\t\tBootstrap []string\n\t}\n\terr = json.NewDecoder(cfgFile).Decode(&bootstrapCfg)\n\tif err != nil {\n\t\tfmt.Fprintln(os.Stderr, \"cannot read bootstrap peers from config\")\n\t} else {\n\t\tbootstrap = bootstrapCfg.Bootstrap\n\t}\n\n\tif _, err = cfgFile.Seek(0, 0); err != nil {\n\t\t// If Seek fails, only log the error and continue on to try to read the\n\t\t// peering config anyway as it might still be readable\n\t\tfmt.Fprintln(os.Stderr, err)\n\t}\n\n\t// Attempt to read peers\n\tvar peeringCfg struct {\n\t\tPeering config.Peering\n\t}\n\terr = json.NewDecoder(cfgFile).Decode(&peeringCfg)\n\tif err != nil {\n\t\tfmt.Fprintln(os.Stderr, \"cannot read peering from config\")\n\t} else {\n\t\tpeers = peeringCfg.Peering.Peers\n\t}\n\n\treturn\n}\n" }, { "path": "repo/fsrepo/migrations/ipfsfetcher/ipfsfetcher_test.go", "content": "package ipfsfetcher\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/plugin/loader\"\n\t\"github.com/ipfs/kubo/repo/fsrepo/migrations\"\n)\n\nfunc init() {\n\terr := setupPlugins()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\nfunc TestIpfsFetcher(t *testing.T) {\n\tskipUnlessEpic(t)\n\n\tctx := t.Context()\n\n\tfetcher := NewIpfsFetcher(\"\", 0, nil, \"\")\n\tdefer fetcher.Close()\n\n\tout, err := fetcher.Fetch(ctx, \"go-ipfs/versions\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar lines []string\n\tscan := bufio.NewScanner(bytes.NewReader(out))\n\tfor scan.Scan() {\n\t\tlines = append(lines, scan.Text())\n\t}\n\terr = scan.Err()\n\tif err != nil {\n\t\tt.Fatal(\"could not read versions:\", err)\n\t}\n\n\tif len(lines) < 6 {\n\t\tt.Fatal(\"do not get all expected data\")\n\t}\n\tif lines[0] != \"v0.3.2\" {\n\t\tt.Fatal(\"expected v1.0.0 as first line, got\", lines[0])\n\t}\n\n\t// Check not found\n\tif _, err = fetcher.Fetch(ctx, \"/no_such_file\"); err == nil {\n\t\tt.Fatal(\"expected error 404\")\n\t}\n}\n\nfunc TestInitIpfsFetcher(t *testing.T) {\n\tctx := t.Context()\n\n\tf := NewIpfsFetcher(\"\", 0, nil, \"\")\n\tdefer f.Close()\n\n\t// Init ipfs repo\n\tf.ipfsTmpDir, f.openErr = initTempNode(ctx, nil, nil)\n\tif f.openErr != nil {\n\t\tt.Fatalf(\"failed to initialize ipfs node: %s\", f.openErr)\n\t}\n\n\t// Start ipfs node\n\tf.openErr = f.startTempNode(ctx)\n\tif f.openErr != nil {\n\t\tt.Errorf(\"failed to start ipfs node: %s\", f.openErr)\n\t\treturn\n\t}\n\n\tvar stopFuncCalled bool\n\tstopFunc := f.ipfsStopFunc\n\tf.ipfsStopFunc = func() {\n\t\tstopFuncCalled = true\n\t\tstopFunc()\n\t}\n\n\taddrInfo := f.AddrInfo()\n\tif string(addrInfo.ID) == \"\" {\n\t\tt.Error(\"AddInfo ID not set\")\n\t}\n\tif len(addrInfo.Addrs) == 0 {\n\t\tt.Error(\"AddInfo Addrs not set\")\n\t}\n\tt.Log(\"Temp node listening on:\", addrInfo.Addrs)\n\n\terr := f.Close()\n\tif err != nil {\n\t\tt.Fatalf(\"failed to close fetcher: %s\", err)\n\t}\n\n\tif stopFunc != nil && !stopFuncCalled {\n\t\tt.Error(\"Close did not call stop function\")\n\t}\n\n\terr = f.Close()\n\tif err != nil {\n\t\tt.Fatalf(\"failed to close fetcher 2nd time: %s\", err)\n\t}\n}\n\nfunc TestReadIpfsConfig(t *testing.T) {\n\ttestConfig := `\n{\n\t\"Bootstrap\": [\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n\t\t\"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n\t],\n\t\"Migration\": {\n\t\t\"DownloadSources\": [\"IPFS\", \"HTTP\", \"127.0.0.1\", \"https://127.0.1.1\"],\n\t\t\"Keep\": \"cache\"\n\t},\n\t\"Peering\": {\n\t\t\"Peers\": [\n\t\t\t{\n\t\t\t\t\"ID\": \"12D3KooWGC6TvWhfapngX6wvJHMYvKpDMXPb3ZnCZ6dMoaMtimQ5\",\n\t\t\t\t\"Addrs\": [\"/ip4/127.0.0.1/tcp/4001\", \"/ip4/127.0.0.1/udp/4001/quic\"]\n\t\t\t}\n\t\t]\n\t}\n}\n`\n\n\tnoSuchDir := \"no_such_dir-5953aa51-1145-4efd-afd1-a069075fcf76\"\n\tbootstrap, peers := readIpfsConfig(&noSuchDir, \"\")\n\tif bootstrap != nil {\n\t\tt.Error(\"expected nil bootstrap\")\n\t}\n\tif peers != nil {\n\t\tt.Error(\"expected nil peers\")\n\t}\n\n\ttmpDir := makeConfig(t, testConfig)\n\n\tbootstrap, peers = readIpfsConfig(nil, \"\")\n\tif bootstrap != nil || peers != nil {\n\t\tt.Fatal(\"expected nil ipfs config items\")\n\t}\n\n\tbootstrap, peers = readIpfsConfig(&tmpDir, \"\")\n\tif len(bootstrap) != 2 {\n\t\tt.Fatal(\"wrong number of bootstrap addresses\")\n\t}\n\tif bootstrap[0] != \"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\" {\n\t\tt.Fatal(\"wrong bootstrap address\")\n\t}\n\n\tif len(peers) != 1 {\n\t\tt.Fatal(\"wrong number of peers\")\n\t}\n\n\tpeer := peers[0]\n\tif peer.ID.String() != \"12D3KooWGC6TvWhfapngX6wvJHMYvKpDMXPb3ZnCZ6dMoaMtimQ5\" {\n\t\tt.Errorf(\"wrong ID for first peer\")\n\t}\n\tif len(peer.Addrs) != 2 {\n\t\tt.Error(\"wrong number of addrs for first peer\")\n\t}\n}\n\nfunc TestBadBootstrappingIpfsConfig(t *testing.T) {\n\tconst configBadBootstrap = `\n{\n\t\"Bootstrap\": \"unreadable\",\n\t\"Migration\": {\n\t\t\"DownloadSources\": [\"IPFS\", \"HTTP\", \"127.0.0.1\"],\n\t\t\"Keep\": \"cache\"\n\t},\n\t\"Peering\": {\n\t\t\"Peers\": [\n\t\t\t{\n\t\t\t\t\"ID\": \"12D3KooWGC6TvWhfapngX6wvJHMYvKpDMXPb3ZnCZ6dMoaMtimQ5\",\n\t\t\t\t\"Addrs\": [\"/ip4/127.0.0.1/tcp/4001\", \"/ip4/127.0.0.1/udp/4001/quic\"]\n\t\t\t}\n\t\t]\n\t}\n}\n`\n\n\ttmpDir := makeConfig(t, configBadBootstrap)\n\n\tbootstrap, peers := readIpfsConfig(&tmpDir, \"\")\n\tif bootstrap != nil {\n\t\tt.Fatal(\"expected nil bootstrap\")\n\t}\n\tif len(peers) != 1 {\n\t\tt.Fatal(\"wrong number of peers\")\n\t}\n\tif len(peers[0].Addrs) != 2 {\n\t\tt.Error(\"wrong number of addrs for first peer\")\n\t}\n\tos.RemoveAll(tmpDir)\n}\n\nfunc TestBadPeersIpfsConfig(t *testing.T) {\n\tconst configBadPeers = `\n{\n\t\"Bootstrap\": [\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n\t\t\"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n\t],\n\t\"Migration\": {\n\t\t\"DownloadSources\": [\"IPFS\", \"HTTP\", \"127.0.0.1\"],\n\t\t\"Keep\": \"cache\"\n\t},\n\t\"Peering\": \"Unreadable-data\"\n}\n`\n\n\ttmpDir := makeConfig(t, configBadPeers)\n\n\tbootstrap, peers := readIpfsConfig(&tmpDir, \"\")\n\tif peers != nil {\n\t\tt.Fatal(\"expected nil peers\")\n\t}\n\tif len(bootstrap) != 2 {\n\t\tt.Fatal(\"wrong number of bootstrap addresses\")\n\t}\n\tif bootstrap[0] != \"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\" {\n\t\tt.Fatal(\"wrong bootstrap address\")\n\t}\n}\n\nfunc makeConfig(t *testing.T, configData string) string {\n\ttmpDir := t.TempDir()\n\n\tcfgFile, err := os.Create(filepath.Join(tmpDir, \"config\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err = cfgFile.Write([]byte(configData)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err = cfgFile.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn tmpDir\n}\n\nfunc skipUnlessEpic(t *testing.T) {\n\tif os.Getenv(\"IPFS_EPIC_TEST\") == \"\" {\n\t\tt.SkipNow()\n\t}\n}\n\nfunc setupPlugins() error {\n\tdefaultPath, err := migrations.IpfsDir(\"\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Load plugins. This will skip the repo if not available.\n\tplugins, err := loader.NewPluginLoader(filepath.Join(defaultPath, \"plugins\"))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error loading plugins: %w\", err)\n\t}\n\n\tif err := plugins.Initialize(); err != nil {\n\t\t// Need to ignore errors here because plugins may already be loaded when\n\t\t// run from ipfs daemon.\n\t\treturn fmt.Errorf(\"error initializing plugins: %w\", err)\n\t}\n\n\tif err := plugins.Inject(); err != nil {\n\t\t// Need to ignore errors here because plugins may already be loaded when\n\t\t// run from ipfs daemon.\n\t\treturn fmt.Errorf(\"error injecting plugins: %w\", err)\n\t}\n\n\treturn nil\n}\n" }, { "path": "repo/fsrepo/migrations/migrations.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"log\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path\"\n\t\"runtime\"\n\t\"strings\"\n\t\"sync\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n)\n\nconst (\n\t// Migrations subdirectory in distribution. Empty for root (no subdir).\n\tdistMigsRoot = \"\"\n\tdistFSRM = \"fs-repo-migrations\"\n)\n\n// RunMigration finds, downloads, and runs the individual migrations needed to\n// migrate the repo from its current version to the target version.\n//\n// Deprecated: This function downloads migration binaries from the internet and will be removed\n// in a future version. Use RunHybridMigrations for modern migrations with embedded support,\n// or RunEmbeddedMigrations for repo versions ≥16.\nfunc RunMigration(ctx context.Context, fetcher Fetcher, targetVer int, ipfsDir string, allowDowngrade bool) error {\n\tipfsDir, err := CheckIpfsDir(ipfsDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\tfromVer, err := RepoVersion(ipfsDir)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not get repo version: %w\", err)\n\t}\n\tif fromVer == targetVer {\n\t\t// repo already at target version number\n\t\treturn nil\n\t}\n\tif fromVer > targetVer && !allowDowngrade {\n\t\treturn fmt.Errorf(\"downgrade not allowed from %d to %d\", fromVer, targetVer)\n\t}\n\n\tlogger := log.New(os.Stdout, \"\", 0)\n\n\tlogger.Print(\"Looking for suitable migration binaries.\")\n\n\tmigrations, binPaths, err := findMigrations(ctx, fromVer, targetVer)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Download migrations that were not found\n\tif len(binPaths) < len(migrations) {\n\t\tmissing := make([]string, 0, len(migrations)-len(binPaths))\n\t\tfor _, mig := range migrations {\n\t\t\tif _, ok := binPaths[mig]; !ok {\n\t\t\t\tmissing = append(missing, mig)\n\t\t\t}\n\t\t}\n\n\t\tlogger.Println(\"Need\", len(missing), \"migrations, downloading.\")\n\n\t\ttmpDir, err := os.MkdirTemp(\"\", \"migrations\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer os.RemoveAll(tmpDir)\n\n\t\tfetched, err := fetchMigrations(ctx, fetcher, missing, tmpDir, logger)\n\t\tif err != nil {\n\t\t\tlogger.Print(\"Failed to download migrations.\")\n\t\t\treturn err\n\t\t}\n\n\t\tfor i := range missing {\n\t\t\tbinPaths[missing[i]] = fetched[i]\n\t\t}\n\t}\n\n\tvar revert bool\n\tif fromVer > targetVer {\n\t\trevert = true\n\t}\n\tfor _, migration := range migrations {\n\t\tlogger.Println(\"Running migration\", migration, \"...\")\n\t\terr = runMigration(ctx, binPaths[migration], ipfsDir, revert, logger)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"migration %s failed: %w\", migration, err)\n\t\t}\n\t}\n\tlogger.Printf(\"Success: fs-repo migrated to version %d.\\n\", targetVer)\n\n\treturn nil\n}\n\nfunc NeedMigration(target int) (bool, error) {\n\tvnum, err := RepoVersion(\"\")\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"could not get repo version: %w\", err)\n\t}\n\n\treturn vnum != target, nil\n}\n\nfunc ExeName(name string) string {\n\tif runtime.GOOS == \"windows\" {\n\t\treturn name + \".exe\"\n\t}\n\treturn name\n}\n\n// ReadMigrationConfig reads the Migration section of the IPFS config, avoiding\n// reading anything other than the Migration section. That way, we're free to\n// make arbitrary changes to all _other_ sections in migrations.\n//\n// Deprecated: This function is used by legacy migration downloads and will be removed\n// in a future version. Use RunHybridMigrations or RunEmbeddedMigrations instead.\nfunc ReadMigrationConfig(repoRoot string, userConfigFile string) (*config.Migration, error) {\n\tvar cfg struct {\n\t\tMigration config.Migration\n\t}\n\n\tcfgPath, err := config.Filename(repoRoot, userConfigFile)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcfgFile, err := os.Open(cfgPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer cfgFile.Close()\n\n\terr = json.NewDecoder(cfgFile).Decode(&cfg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tswitch cfg.Migration.Keep {\n\tcase \"\":\n\t\tcfg.Migration.Keep = config.DefaultMigrationKeep\n\tcase \"discard\", \"cache\", \"keep\":\n\tdefault:\n\t\treturn nil, errors.New(\"unknown config value, Migrations.Keep must be 'cache', 'pin', or 'discard'\")\n\t}\n\n\tif len(cfg.Migration.DownloadSources) == 0 {\n\t\tcfg.Migration.DownloadSources = config.DefaultMigrationDownloadSources\n\t}\n\n\treturn &cfg.Migration, nil\n}\n\n// GetMigrationFetcher creates one or more fetchers according to\n// downloadSources.\n//\n// Deprecated: This function is used by legacy migration downloads and will be removed\n// in a future version. Use RunHybridMigrations or RunEmbeddedMigrations instead.\nfunc GetMigrationFetcher(downloadSources []string, distPath string, newIpfsFetcher func(string) Fetcher) (Fetcher, error) {\n\tconst httpUserAgent = \"kubo/migration\"\n\tconst numTriesPerHTTP = 3\n\n\tvar fetchers []Fetcher\n\tfor _, src := range downloadSources {\n\t\tsrc := strings.TrimSpace(src)\n\t\tswitch src {\n\t\tcase \"HTTPS\", \"https\", \"HTTP\", \"http\":\n\t\t\tfetchers = append(fetchers, &RetryFetcher{NewHttpFetcher(distPath, \"\", httpUserAgent, 0), numTriesPerHTTP})\n\t\tcase \"IPFS\", \"ipfs\":\n\t\t\treturn nil, errors.New(\"IPFS downloads are not supported for legacy migrations (repo versions <16). Please use only HTTPS in Migration.DownloadSources\")\n\t\tcase \"\":\n\t\t\t// Ignore empty string\n\t\tdefault:\n\t\t\tu, err := url.Parse(src)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"bad gateway address: %w\", err)\n\t\t\t}\n\t\t\tswitch u.Scheme {\n\t\t\tcase \"\":\n\t\t\t\tu.Scheme = \"https\"\n\t\t\tcase \"https\", \"http\":\n\t\t\tdefault:\n\t\t\t\treturn nil, errors.New(\"bad gateway address: url scheme must be http or https\")\n\t\t\t}\n\t\t\tfetchers = append(fetchers, &RetryFetcher{NewHttpFetcher(distPath, u.String(), httpUserAgent, 0), numTriesPerHTTP})\n\t\t}\n\t}\n\n\tswitch len(fetchers) {\n\tcase 0:\n\t\treturn nil, errors.New(\"no sources specified\")\n\tcase 1:\n\t\treturn fetchers[0], nil\n\t}\n\n\t// Wrap fetchers in a MultiFetcher to try them in order\n\treturn NewMultiFetcher(fetchers...), nil\n}\n\nfunc migrationName(from, to int) string {\n\treturn fmt.Sprintf(\"fs-repo-%d-to-%d\", from, to)\n}\n\n// findMigrations returns a list of migrations, ordered from first to last\n// migration to apply, and a map of locations of migration binaries of any\n// migrations that were found.\n//\n// Deprecated: This function is used by legacy migration downloads and will be removed\n// in a future version.\nfunc findMigrations(ctx context.Context, from, to int) ([]string, map[string]string, error) {\n\tstep := 1\n\tcount := to - from\n\tif from > to {\n\t\tstep = -1\n\t\tcount = from - to\n\t}\n\n\tmigrations := make([]string, 0, count)\n\tbinPaths := make(map[string]string, count)\n\n\tfor cur := from; cur != to; cur += step {\n\t\tif ctx.Err() != nil {\n\t\t\treturn nil, nil, ctx.Err()\n\t\t}\n\t\tvar migName string\n\t\tif step == -1 {\n\t\t\tmigName = migrationName(cur+step, cur)\n\t\t} else {\n\t\t\tmigName = migrationName(cur, cur+step)\n\t\t}\n\t\tmigrations = append(migrations, migName)\n\t\tbin, err := exec.LookPath(migName)\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tbinPaths[migName] = bin\n\t}\n\treturn migrations, binPaths, nil\n}\n\nfunc runMigration(ctx context.Context, binPath, ipfsDir string, revert bool, logger *log.Logger) error {\n\tpathArg := fmt.Sprintf(\"-path=%s\", ipfsDir)\n\tvar cmd *exec.Cmd\n\tif revert {\n\t\tlogger.Println(\" => Running:\", binPath, pathArg, \"-verbose=true -revert\")\n\t\tcmd = exec.CommandContext(ctx, binPath, pathArg, \"-verbose=true\", \"-revert\")\n\t} else {\n\t\tlogger.Println(\" => Running:\", binPath, pathArg, \"-verbose=true\")\n\t\tcmd = exec.CommandContext(ctx, binPath, pathArg, \"-verbose=true\")\n\t}\n\tcmd.Stdout = os.Stdout\n\tcmd.Stderr = os.Stderr\n\treturn cmd.Run()\n}\n\n// fetchMigrations downloads the requested migrations, and returns a slice with\n// the paths of each binary, in the same order specified by needed.\n//\n// Deprecated: This function downloads migration binaries from the internet and will be removed\n// in a future version. Use RunHybridMigrations or RunEmbeddedMigrations instead.\nfunc fetchMigrations(ctx context.Context, fetcher Fetcher, needed []string, destDir string, logger *log.Logger) ([]string, error) {\n\tosv, err := osWithVariant()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif osv == \"linux-musl\" {\n\t\treturn nil, fmt.Errorf(\"linux-musl not supported, you must build the binary from source for your platform\")\n\t}\n\n\tvar wg sync.WaitGroup\n\twg.Add(len(needed))\n\tbins := make([]string, len(needed))\n\t// Download and unpack all requested migrations concurrently.\n\tfor i, name := range needed {\n\t\tlogger.Printf(\"Downloading migration: %s...\", name)\n\t\tgo func(i int, name string) {\n\t\t\tdefer wg.Done()\n\t\t\tdist := path.Join(distMigsRoot, name)\n\t\t\tver, err := LatestDistVersion(ctx, fetcher, dist, false)\n\t\t\tif err != nil {\n\t\t\t\tlogger.Printf(\"could not get latest version of migration %s: %s\", name, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tloc, err := FetchBinary(ctx, fetcher, dist, ver, name, destDir)\n\t\t\tif err != nil {\n\t\t\t\tlogger.Printf(\"could not download %s: %s\", name, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tlogger.Printf(\"Downloaded and unpacked migration: %s (%s)\", loc, ver)\n\t\t\tbins[i] = loc\n\t\t}(i, name)\n\t}\n\twg.Wait()\n\n\tvar fails []string\n\tfor i := range bins {\n\t\tif bins[i] == \"\" {\n\t\t\tfails = append(fails, needed[i])\n\t\t}\n\t}\n\tif len(fails) != 0 {\n\t\terr = fmt.Errorf(\"failed to download migrations: %s\", strings.Join(fails, \" \"))\n\t\tif ctx.Err() != nil {\n\t\t\terr = fmt.Errorf(\"%s, %w\", ctx.Err(), err)\n\t\t}\n\t\treturn nil, err\n\t}\n\n\treturn bins, nil\n}\n\n// RunHybridMigrations intelligently runs migrations using external tools for legacy versions\n// and embedded migrations for modern versions. This handles the transition from external\n// fs-repo-migrations binaries (for repo versions <16) to embedded migrations (for repo versions ≥16).\n//\n// The function automatically:\n// 1. Uses external migrations to get from current version to v16 (if needed)\n// 2. Uses embedded migrations for v16+ steps\n// 3. Handles pure external, pure embedded, or mixed migration scenarios\n//\n// Legacy external migrations (repo versions <16) only support HTTPS downloads.\n//\n// Parameters:\n// - ctx: Context for cancellation and timeouts\n// - targetVer: Target repository version to migrate to\n// - ipfsDir: Path to the IPFS repository directory\n// - allowDowngrade: Whether to allow downgrade migrations\n//\n// Returns error if migration fails at any step.\nfunc RunHybridMigrations(ctx context.Context, targetVer int, ipfsDir string, allowDowngrade bool) error {\n\tconst embeddedMigrationsMinVersion = 16\n\n\t// Get current repo version\n\tcurrentVer, err := RepoVersion(ipfsDir)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not get current repo version: %w\", err)\n\t}\n\n\tvar logger = log.New(os.Stdout, \"\", 0)\n\n\t// Check if migration is needed\n\tif currentVer == targetVer {\n\t\tlogger.Printf(\"Repository is already at version %d\", targetVer)\n\t\treturn nil\n\t}\n\n\t// Validate downgrade request\n\tif targetVer < currentVer && !allowDowngrade {\n\t\treturn fmt.Errorf(\"downgrade from version %d to %d requires allowDowngrade=true\", currentVer, targetVer)\n\t}\n\n\t// Determine migration strategy based on version ranges\n\tneedsExternal := currentVer < embeddedMigrationsMinVersion\n\tneedsEmbedded := targetVer >= embeddedMigrationsMinVersion\n\n\t// Case 1: Pure embedded migration (both current and target ≥ 16)\n\tif !needsExternal && needsEmbedded {\n\t\treturn RunEmbeddedMigrations(ctx, targetVer, ipfsDir, allowDowngrade)\n\t}\n\n\t// For cases requiring external migrations, we check if migration binaries\n\t// are available in PATH before attempting network downloads\n\n\t// Case 2: Pure external migration (target < 16)\n\tif needsExternal && !needsEmbedded {\n\n\t\t// Check for migration binaries in PATH first (for testing/local development)\n\t\tmigrations, binPaths, err := findMigrations(ctx, currentVer, targetVer)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not determine migration paths: %w\", err)\n\t\t}\n\n\t\tfoundAll := true\n\t\tfor _, migName := range migrations {\n\t\t\tif _, exists := binPaths[migName]; !exists {\n\t\t\t\tfoundAll = false\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif foundAll {\n\t\t\treturn runMigrationsFromPath(ctx, migrations, binPaths, ipfsDir, logger, false)\n\t\t}\n\n\t\t// Fall back to network download (original behavior)\n\t\tmigrationCfg, err := ReadMigrationConfig(ipfsDir, \"\")\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not read migration config: %w\", err)\n\t\t}\n\n\t\t// Use existing RunMigration which handles network downloads properly (HTTPS only for legacy migrations)\n\t\tfetcher, err := GetMigrationFetcher(migrationCfg.DownloadSources, GetDistPathEnv(CurrentIpfsDist), nil)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to get migration fetcher: %w\", err)\n\t\t}\n\t\tdefer fetcher.Close()\n\t\treturn RunMigration(ctx, fetcher, targetVer, ipfsDir, allowDowngrade)\n\t}\n\n\t// Case 3: Hybrid migration (current < 16, target ≥ 16)\n\tif needsExternal && needsEmbedded {\n\t\tlogger.Printf(\"Starting hybrid migration from version %d to %d\", currentVer, targetVer)\n\t\tlogger.Print(\"Using hybrid migration strategy: external to v16, then embedded\")\n\n\t\t// Phase 1: Use external migrations to get to v16\n\t\tlogger.Printf(\"Phase 1: External migration from v%d to v%d\", currentVer, embeddedMigrationsMinVersion)\n\n\t\t// Check for external migration binaries in PATH first\n\t\tmigrations, binPaths, err := findMigrations(ctx, currentVer, embeddedMigrationsMinVersion)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not determine external migration paths: %w\", err)\n\t\t}\n\n\t\tfoundAll := true\n\t\tfor _, migName := range migrations {\n\t\t\tif _, exists := binPaths[migName]; !exists {\n\t\t\t\tfoundAll = false\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif foundAll {\n\t\t\tif err = runMigrationsFromPath(ctx, migrations, binPaths, ipfsDir, logger, false); err != nil {\n\t\t\t\treturn fmt.Errorf(\"external migration phase failed: %w\", err)\n\t\t\t}\n\t\t} else {\n\t\t\tmigrationCfg, err := ReadMigrationConfig(ipfsDir, \"\")\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"could not read migration config: %w\", err)\n\t\t\t}\n\n\t\t\t// Legacy migrations only support HTTPS downloads\n\t\t\tfetcher, err := GetMigrationFetcher(migrationCfg.DownloadSources, GetDistPathEnv(CurrentIpfsDist), nil)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to get migration fetcher: %w\", err)\n\t\t\t}\n\t\t\tdefer fetcher.Close()\n\n\t\t\tif err = RunMigration(ctx, fetcher, embeddedMigrationsMinVersion, ipfsDir, allowDowngrade); err != nil {\n\t\t\t\treturn fmt.Errorf(\"external migration phase failed: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\t// Phase 2: Use embedded migrations for v16+\n\t\tlogger.Printf(\"Phase 2: Embedded migration from v%d to v%d\", embeddedMigrationsMinVersion, targetVer)\n\t\terr = RunEmbeddedMigrations(ctx, targetVer, ipfsDir, allowDowngrade)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"embedded migration phase failed: %w\", err)\n\t\t}\n\n\t\tlogger.Printf(\"Hybrid migration completed successfully: v%d → v%d\", currentVer, targetVer)\n\t\treturn nil\n\t}\n\n\t// Case 4: Reverse hybrid migration (≥16 to <16)\n\t// Use embedded migrations for ≥16 steps, then external migrations for <16 steps\n\tlogger.Printf(\"Starting reverse hybrid migration from version %d to %d\", currentVer, targetVer)\n\tlogger.Print(\"Using reverse hybrid migration strategy: embedded to v16, then external\")\n\n\t// Phase 1: Use embedded migrations from current version down to v16 (if needed)\n\tif currentVer > embeddedMigrationsMinVersion {\n\t\tlogger.Printf(\"Phase 1: Embedded downgrade from v%d to v%d\", currentVer, embeddedMigrationsMinVersion)\n\t\terr = RunEmbeddedMigrations(ctx, embeddedMigrationsMinVersion, ipfsDir, allowDowngrade)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"embedded downgrade phase failed: %w\", err)\n\t\t}\n\t}\n\n\t// Phase 2: Use external migrations from v16 to target (if needed)\n\tif embeddedMigrationsMinVersion > targetVer {\n\t\tlogger.Printf(\"Phase 2: External downgrade from v%d to v%d\", embeddedMigrationsMinVersion, targetVer)\n\n\t\t// Check for external migration binaries in PATH first\n\t\tmigrations, binPaths, err := findMigrations(ctx, embeddedMigrationsMinVersion, targetVer)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not determine external migration paths: %w\", err)\n\t\t}\n\n\t\tfoundAll := true\n\t\tfor _, migName := range migrations {\n\t\t\tif _, exists := binPaths[migName]; !exists {\n\t\t\t\tfoundAll = false\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif foundAll {\n\t\t\tif err = runMigrationsFromPath(ctx, migrations, binPaths, ipfsDir, logger, true); err != nil {\n\t\t\t\treturn fmt.Errorf(\"external downgrade phase failed: %w\", err)\n\t\t\t}\n\t\t} else {\n\t\t\tmigrationCfg, err := ReadMigrationConfig(ipfsDir, \"\")\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"could not read migration config: %w\", err)\n\t\t\t}\n\n\t\t\t// Legacy migrations only support HTTPS downloads\n\t\t\tfetcher, err := GetMigrationFetcher(migrationCfg.DownloadSources, GetDistPathEnv(CurrentIpfsDist), nil)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to get migration fetcher: %w\", err)\n\t\t\t}\n\t\t\tdefer fetcher.Close()\n\n\t\t\tif err = RunMigration(ctx, fetcher, targetVer, ipfsDir, allowDowngrade); err != nil {\n\t\t\t\treturn fmt.Errorf(\"external downgrade phase failed: %w\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\tlogger.Printf(\"Reverse hybrid migration completed successfully: v%d → v%d\", currentVer, targetVer)\n\treturn nil\n}\n\n// runMigrationsFromPath runs migrations using binaries found in PATH\nfunc runMigrationsFromPath(ctx context.Context, migrations []string, binPaths map[string]string, ipfsDir string, logger *log.Logger, revert bool) error {\n\tfor _, migName := range migrations {\n\t\tbinPath, exists := binPaths[migName]\n\t\tif !exists {\n\t\t\treturn fmt.Errorf(\"migration binary %s not found in PATH\", migName)\n\t\t}\n\n\t\tlogger.Printf(\"Running migration %s using binary from PATH: %s\", migName, binPath)\n\n\t\t// Run the migration binary directly\n\t\terr := runMigration(ctx, binPath, ipfsDir, revert, logger)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"migration %s failed: %w\", migName, err)\n\t\t}\n\t}\n\treturn nil\n}\n" }, { "path": "repo/fsrepo/migrations/migrations_test.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n)\n\nfunc TestFindMigrations(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tctx := t.Context()\n\n\tmigs, bins, err := findMigrations(ctx, 0, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(migs) != 5 {\n\t\tt.Fatal(\"expected 5 migrations\")\n\t}\n\tif len(bins) != 0 {\n\t\tt.Fatal(\"should not have found migrations\")\n\t}\n\n\tfor i := 1; i < 6; i++ {\n\t\tcreateFakeBin(i-1, i, tmpDir)\n\t}\n\n\tt.Setenv(\"PATH\", tmpDir)\n\n\tmigs, bins, err = findMigrations(ctx, 0, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(migs) != 5 {\n\t\tt.Fatal(\"expected 5 migrations\")\n\t}\n\tif len(bins) != len(migs) {\n\t\tt.Fatal(\"missing\", len(migs)-len(bins), \"migrations\")\n\t}\n\n\tos.Remove(bins[migs[2]])\n\n\tmigs, bins, err = findMigrations(ctx, 0, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(bins) != len(migs)-1 {\n\t\tt.Fatal(\"should be missing one migration bin\")\n\t}\n}\n\nfunc TestFindMigrationsReverse(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tctx := t.Context()\n\n\tmigs, bins, err := findMigrations(ctx, 5, 0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(migs) != 5 {\n\t\tt.Fatal(\"expected 5 migrations\")\n\t}\n\tif len(bins) != 0 {\n\t\tt.Fatal(\"should not have found migrations\")\n\t}\n\n\tfor i := 1; i < 6; i++ {\n\t\tcreateFakeBin(i-1, i, tmpDir)\n\t}\n\n\tt.Setenv(\"PATH\", tmpDir)\n\n\tmigs, bins, err = findMigrations(ctx, 5, 0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(migs) != 5 {\n\t\tt.Fatal(\"expected 5 migrations\")\n\t}\n\tif len(bins) != len(migs) {\n\t\tt.Fatal(\"missing\", len(migs)-len(bins), \"migrations:\", migs)\n\t}\n\n\tos.Remove(bins[migs[2]])\n\n\tmigs, bins, err = findMigrations(ctx, 5, 0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(bins) != len(migs)-1 {\n\t\tt.Fatal(\"should be missing one migration bin\")\n\t}\n}\n\nfunc TestFetchMigrations(t *testing.T) {\n\tctx := t.Context()\n\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\ttmpDir := t.TempDir()\n\n\tneeded := []string{\"fs-repo-1-to-2\", \"fs-repo-2-to-3\"}\n\tbuf := new(strings.Builder)\n\tbuf.Grow(256)\n\tlogger := log.New(buf, \"\", 0)\n\tfetched, err := fetchMigrations(ctx, fetcher, needed, tmpDir, logger)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfor _, bin := range fetched {\n\t\t_, err = os.Stat(bin)\n\t\tif os.IsNotExist(err) {\n\t\t\tt.Error(\"expected file to exist:\", bin)\n\t\t}\n\t}\n\n\t// Check expected log output\n\tfor _, mig := range needed {\n\t\tlogOut := fmt.Sprintf(\"Downloading migration: %s\", mig)\n\t\tif !strings.Contains(buf.String(), logOut) {\n\t\t\tt.Fatalf(\"did not find expected log output %q\", logOut)\n\t\t}\n\t\tlogOut = fmt.Sprintf(\"Downloaded and unpacked migration: %s\", filepath.Join(tmpDir, mig))\n\t\tif !strings.Contains(buf.String(), logOut) {\n\t\t\tt.Fatalf(\"did not find expected log output %q\", logOut)\n\t\t}\n\t}\n}\n\nfunc TestRunMigrations(t *testing.T) {\n\tfakeIpfs := filepath.Join(t.TempDir(), \".ipfs\")\n\tt.Setenv(config.EnvDir, fakeIpfs)\n\n\terr := os.Mkdir(fakeIpfs, os.ModePerm)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\ttestVer := 11\n\terr = WriteRepoVersion(fakeIpfs, testVer)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\tctx := t.Context()\n\n\ttargetVer := 9\n\n\terr = RunMigration(ctx, fetcher, targetVer, fakeIpfs, false)\n\tif err == nil || !strings.HasPrefix(err.Error(), \"downgrade not allowed\") {\n\t\tt.Fatal(\"expected 'downgrade not allowed' error\")\n\t}\n\n\terr = RunMigration(ctx, fetcher, targetVer, fakeIpfs, true)\n\tif err != nil {\n\t\tif !strings.HasPrefix(err.Error(), \"migration fs-repo-10-to-11 failed\") {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n}\n\nfunc createFakeBin(from, to int, tmpDir string) {\n\tmigPath := filepath.Join(tmpDir, ExeName(migrationName(from, to)))\n\temptyFile, err := os.Create(migPath)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\temptyFile.Close()\n\terr = os.Chmod(migPath, 0o755)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\nvar testConfig = `\n{\n\t\"Bootstrap\": [\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n\t\t\"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n\t],\n\t\"Migration\": {\n\t\t\"DownloadSources\": [\"IPFS\", \"HTTP\", \"127.0.0.1\", \"https://127.0.1.1\"],\n\t\t\"Keep\": \"cache\"\n\t},\n\t\"Peering\": {\n\t\t\"Peers\": [\n\t\t\t{\n\t\t\t\t\"ID\": \"12D3KooWGC6TvWhfapngX6wvJHMYvKpDMXPb3ZnCZ6dMoaMtimQ5\",\n\t\t\t\t\"Addrs\": [\"/ip4/127.0.0.1/tcp/4001\", \"/ip4/127.0.0.1/udp/4001/quic\"]\n\t\t\t}\n\t\t]\n\t}\n}\n`\n\nfunc TestReadMigrationConfigDefaults(t *testing.T) {\n\ttmpDir := makeConfig(t, \"{}\")\n\n\tcfg, err := ReadMigrationConfig(tmpDir, \"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif cfg.Keep != config.DefaultMigrationKeep {\n\t\tt.Error(\"expected default value for Keep\")\n\t}\n\n\tif len(cfg.DownloadSources) != len(config.DefaultMigrationDownloadSources) {\n\t\tt.Fatal(\"expected default number of download sources\")\n\t}\n\tfor i, src := range config.DefaultMigrationDownloadSources {\n\t\tif cfg.DownloadSources[i] != src {\n\t\t\tt.Errorf(\"wrong DownloadSource: %s\", cfg.DownloadSources[i])\n\t\t}\n\t}\n}\n\nfunc TestReadMigrationConfigErrors(t *testing.T) {\n\ttmpDir := makeConfig(t, `{\"Migration\": {\"Keep\": \"badvalue\"}}`)\n\n\t_, err := ReadMigrationConfig(tmpDir, \"\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error\")\n\t}\n\tif !strings.HasPrefix(err.Error(), \"unknown\") {\n\t\tt.Fatal(\"did not get expected error:\", err)\n\t}\n\n\tos.RemoveAll(tmpDir)\n\t_, err = ReadMigrationConfig(tmpDir, \"\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error\")\n\t}\n\n\ttmpDir = makeConfig(t, `}{`)\n\t_, err = ReadMigrationConfig(tmpDir, \"\")\n\tif err == nil {\n\t\tt.Fatal(\"expected error\")\n\t}\n}\n\nfunc TestReadMigrationConfig(t *testing.T) {\n\ttmpDir := makeConfig(t, testConfig)\n\n\tcfg, err := ReadMigrationConfig(tmpDir, \"\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(cfg.DownloadSources) != 4 {\n\t\tt.Fatal(\"wrong number of DownloadSources\")\n\t}\n\texpect := []string{\"IPFS\", \"HTTP\", \"127.0.0.1\", \"https://127.0.1.1\"}\n\tfor i := range expect {\n\t\tif cfg.DownloadSources[i] != expect[i] {\n\t\t\tt.Errorf(\"wrong DownloadSource at %d\", i)\n\t\t}\n\t}\n\n\tif cfg.Keep != \"cache\" {\n\t\tt.Error(\"wrong value for Keep\")\n\t}\n}\n\ntype mockIpfsFetcher struct{}\n\nvar _ Fetcher = (*mockIpfsFetcher)(nil)\n\nfunc (m *mockIpfsFetcher) Fetch(ctx context.Context, filePath string) ([]byte, error) {\n\treturn nil, nil\n}\n\nfunc (m *mockIpfsFetcher) Close() error {\n\treturn nil\n}\n\nfunc TestGetMigrationFetcher(t *testing.T) {\n\tvar f Fetcher\n\tvar err error\n\n\tnewIpfsFetcher := func(distPath string) Fetcher {\n\t\treturn &mockIpfsFetcher{}\n\t}\n\n\tdownloadSources := []string{\"ftp://bad.gateway.io\"}\n\t_, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err == nil || !strings.HasPrefix(err.Error(), \"bad gateway addr\") {\n\t\tt.Fatal(\"Expected bad gateway address error, got:\", err)\n\t}\n\n\tdownloadSources = []string{\"::bad.gateway.io\"}\n\t_, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err == nil || !strings.HasPrefix(err.Error(), \"bad gateway addr\") {\n\t\tt.Fatal(\"Expected bad gateway address error, got:\", err)\n\t}\n\n\tdownloadSources = []string{\"http://localhost\"}\n\tf, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif rf, ok := f.(*RetryFetcher); !ok {\n\t\tt.Fatal(\"expected RetryFetcher\")\n\t} else if _, ok := rf.Fetcher.(*HttpFetcher); !ok {\n\t\tt.Fatal(\"expected HttpFetcher\")\n\t}\n\n\tdownloadSources = []string{\"ipfs\"}\n\t_, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err == nil || !strings.Contains(err.Error(), \"IPFS downloads are not supported for legacy migrations\") {\n\t\tt.Fatal(\"Expected IPFS downloads error, got:\", err)\n\t}\n\n\tdownloadSources = []string{\"http\"}\n\tf, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif rf, ok := f.(*RetryFetcher); !ok {\n\t\tt.Fatal(\"expected RetryFetcher\")\n\t} else if _, ok := rf.Fetcher.(*HttpFetcher); !ok {\n\t\tt.Fatal(\"expected HttpFetcher\")\n\t}\n\n\tdownloadSources = []string{\"IPFS\", \"HTTPS\"}\n\t_, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err == nil || !strings.Contains(err.Error(), \"IPFS downloads are not supported for legacy migrations\") {\n\t\tt.Fatal(\"Expected IPFS downloads error, got:\", err)\n\t}\n\n\tdownloadSources = []string{\"https\", \"some.domain.io\"}\n\tf, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tmf, ok := f.(*MultiFetcher)\n\tif !ok {\n\t\tt.Fatal(\"expected MultiFetcher\")\n\t}\n\tif mf.Len() != 2 {\n\t\tt.Fatal(\"expected 2 fetchers in MultiFetcher\")\n\t}\n\n\tdownloadSources = nil\n\t_, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err == nil {\n\t\tt.Fatal(\"expected error when no sources specified\")\n\t}\n\n\tdownloadSources = []string{\"\", \"\"}\n\t_, err = GetMigrationFetcher(downloadSources, \"\", newIpfsFetcher)\n\tif err == nil {\n\t\tt.Fatal(\"expected error when empty string fetchers specified\")\n\t}\n}\n\nfunc makeConfig(t *testing.T, configData string) string {\n\ttmpDir := t.TempDir()\n\n\tcfgFile, err := os.Create(filepath.Join(tmpDir, \"config\"))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err = cfgFile.Write([]byte(configData)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err = cfgFile.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn tmpDir\n}\n" }, { "path": "repo/fsrepo/migrations/retryfetcher.go", "content": "package migrations\n\nimport (\n\t\"context\"\n\t\"fmt\"\n)\n\ntype RetryFetcher struct {\n\tFetcher\n\tMaxTries int\n}\n\nvar _ Fetcher = (*RetryFetcher)(nil)\n\nfunc (r *RetryFetcher) Fetch(ctx context.Context, filePath string) ([]byte, error) {\n\tvar lastErr error\n\tfor i := 0; i < r.MaxTries; i++ {\n\t\tout, err := r.Fetcher.Fetch(ctx, filePath)\n\t\tif err == nil {\n\t\t\treturn out, nil\n\t\t}\n\n\t\tif ctx.Err() != nil {\n\t\t\treturn nil, ctx.Err()\n\t\t}\n\t\tlastErr = err\n\t}\n\treturn nil, fmt.Errorf(\"exceeded number of retries. last error was %w\", lastErr)\n}\n\nfunc (r *RetryFetcher) Close() error {\n\treturn r.Fetcher.Close()\n}\n" }, { "path": "repo/fsrepo/migrations/setup_test.go", "content": "package migrations\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/blockservice\"\n\t\"github.com/ipfs/boxo/exchange/offline\"\n\t\"github.com/ipfs/boxo/gateway\"\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-unixfsnode/data/builder\"\n\t\"github.com/ipld/go-car/v2\"\n\tcarblockstore \"github.com/ipld/go-car/v2/blockstore\"\n\t\"github.com/ipld/go-ipld-prime\"\n\tcidlink \"github.com/ipld/go-ipld-prime/linking/cid\"\n\t\"github.com/multiformats/go-multicodec\"\n\t\"github.com/multiformats/go-multihash\"\n)\n\nvar (\n\ttestIpfsDist string\n\ttestServer *httptest.Server\n)\n\nfunc TestMain(m *testing.M) {\n\tt := &testing.T{}\n\n\t// Setup test data\n\ttestDataDir := makeTestData(t)\n\n\ttestCar := makeTestCar(testDataDir)\n\tdefer os.RemoveAll(testCar)\n\n\t// Setup test gateway\n\tfd := setupTestGateway(testCar)\n\tdefer fd.Close()\n\n\t// Run tests\n\tos.Exit(m.Run())\n}\n\nfunc makeTestData(t testing.TB) string {\n\ttempDir := t.TempDir()\n\n\tversions := []string{\"v1.0.0\", \"v1.1.0\", \"v1.1.2\", \"v2.0.0-rc1\", \"2.0.0\", \"v2.0.1\"}\n\tpackages := []string{\"kubo\", \"go-ipfs\", \"fs-repo-migrations\", \"fs-repo-1-to-2\", \"fs-repo-2-to-3\", \"fs-repo-9-to-10\", \"fs-repo-10-to-11\"}\n\n\t// Generate fake data\n\tfor _, name := range packages {\n\t\terr := os.MkdirAll(filepath.Join(tempDir, name), 0777)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\terr = os.WriteFile(filepath.Join(tempDir, name, \"versions\"), []byte(strings.Join(versions, \"\\n\")+\"\\n\"), 0666)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tfor _, version := range versions {\n\t\t\tfilename, archName := makeArchivePath(name, name, version, \"tar.gz\")\n\t\t\tcreateFakeArchive(filepath.Join(tempDir, filename), archName, false)\n\n\t\t\tfilename, archName = makeArchivePath(name, name, version, \"zip\")\n\t\t\tcreateFakeArchive(filepath.Join(tempDir, filename), archName, true)\n\t\t}\n\t}\n\n\treturn tempDir\n}\n\nfunc createFakeArchive(archName, name string, archZip bool) {\n\terr := os.MkdirAll(filepath.Dir(archName), 0777)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tfileName := strings.Split(path.Base(name), \"_\")[0]\n\troot := fileName\n\n\t// Simulate fetching go-ipfs, which has \"ipfs\" as the name in the archive.\n\tif fileName == \"go-ipfs\" || fileName == \"kubo\" {\n\t\tfileName = \"ipfs\"\n\t}\n\tfileName = ExeName(fileName)\n\n\tif archZip {\n\t\terr = writeZipFile(archName, root, fileName, \"FAKE DATA\")\n\t} else {\n\t\terr = writeTarGzipFile(archName, root, fileName, \"FAKE DATA\")\n\t}\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\n// makeTestCar makes a CAR file with the directory [testData]. This code is mostly\n// sourced from https://github.com/ipld/go-car/blob/1e2f0bd2c44ee31f48a8f602b25b5671cc0c4687/cmd/car/create.go\nfunc makeTestCar(testData string) string {\n\t// make a cid with the right length that we eventually will patch with the root.\n\thasher, err := multihash.GetHasher(multihash.SHA2_256)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdigest := hasher.Sum([]byte{})\n\thash, err := multihash.Encode(digest, multihash.SHA2_256)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tproxyRoot := cid.NewCidV1(uint64(multicodec.DagPb), hash)\n\n\t// Make CAR file\n\tfd, err := os.CreateTemp(\"\", \"kubo-migrations-test-*.car\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer fd.Close()\n\tfilename := fd.Name()\n\n\trw, err := carblockstore.OpenReadWriteFile(fd, []cid.Cid{proxyRoot}, carblockstore.WriteAsCarV1(true))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer rw.Close()\n\n\tctx := context.Background()\n\n\tls := cidlink.DefaultLinkSystem()\n\tls.TrustedStorage = true\n\tls.StorageReadOpener = func(_ ipld.LinkContext, l ipld.Link) (io.Reader, error) {\n\t\tcl, ok := l.(cidlink.Link)\n\t\tif !ok {\n\t\t\treturn nil, fmt.Errorf(\"not a cidlink\")\n\t\t}\n\t\tblk, err := rw.Get(ctx, cl.Cid)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn bytes.NewBuffer(blk.RawData()), nil\n\t}\n\tls.StorageWriteOpener = func(_ ipld.LinkContext) (io.Writer, ipld.BlockWriteCommitter, error) {\n\t\tbuf := bytes.NewBuffer(nil)\n\t\treturn buf, func(l ipld.Link) error {\n\t\t\tcl, ok := l.(cidlink.Link)\n\t\t\tif !ok {\n\t\t\t\treturn fmt.Errorf(\"not a cidlink\")\n\t\t\t}\n\t\t\tblk, err := blocks.NewBlockWithCid(buf.Bytes(), cl.Cid)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn rw.Put(ctx, blk)\n\t\t}, nil\n\t}\n\n\tl, _, err := builder.BuildUnixFSRecursive(testData, &ls)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\trcl, ok := l.(cidlink.Link)\n\tif !ok {\n\t\tpanic(fmt.Errorf(\"could not interpret %s\", l))\n\t}\n\n\tif err := rw.Finalize(); err != nil {\n\t\tpanic(err)\n\t}\n\t// re-open/finalize with the final root.\n\terr = car.ReplaceRootsInFile(filename, []cid.Cid{rcl.Cid})\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\treturn filename\n}\n\nfunc setupTestGateway(testCar string) io.Closer {\n\tblockService, roots, fd, err := newBlockServiceFromCAR(testCar)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tif len(roots) != 1 {\n\t\tpanic(\"expected car with 1 root\")\n\t}\n\n\tbackend, err := gateway.NewBlocksBackend(blockService)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tconf := gateway.Config{\n\t\tNoDNSLink: false,\n\t\tDeserializedResponses: false,\n\t}\n\n\ttestIpfsDist = \"/ipfs/\" + roots[0].String()\n\ttestServer = httptest.NewServer(gateway.NewHandler(conf, backend))\n\n\treturn fd\n}\n\nfunc newBlockServiceFromCAR(filepath string) (blockservice.BlockService, []cid.Cid, io.Closer, error) {\n\tr, err := os.Open(filepath)\n\tif err != nil {\n\t\treturn nil, nil, nil, err\n\t}\n\n\tbs, err := carblockstore.NewReadOnly(r, nil)\n\tif err != nil {\n\t\t_ = r.Close()\n\t\treturn nil, nil, nil, err\n\t}\n\n\troots, err := bs.Roots()\n\tif err != nil {\n\t\treturn nil, nil, nil, err\n\t}\n\n\tblockService := blockservice.New(bs, offline.Exchange(bs))\n\treturn blockService, roots, r, nil\n}\n" }, { "path": "repo/fsrepo/migrations/unpack.go", "content": "package migrations\n\nimport (\n\t\"archive/tar\"\n\t\"archive/zip\"\n\t\"compress/gzip\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n)\n\nfunc unpackArchive(arcPath, atype, root, name, out string) error {\n\tvar err error\n\tswitch atype {\n\tcase \"tar.gz\":\n\t\terr = unpackTgz(arcPath, root, name, out)\n\tcase \"zip\":\n\t\terr = unpackZip(arcPath, root, name, out)\n\tdefault:\n\t\terr = fmt.Errorf(\"unrecognized archive type: %s\", atype)\n\t}\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc unpackTgz(arcPath, root, name, out string) error {\n\tfi, err := os.Open(arcPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot open archive file: %w\", err)\n\t}\n\tdefer fi.Close()\n\n\tgzr, err := gzip.NewReader(fi)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error opening gzip reader: %w\", err)\n\t}\n\tdefer gzr.Close()\n\n\tvar bin io.Reader\n\ttarr := tar.NewReader(gzr)\n\n\tlookFor := root + \"/\" + name\n\tfor {\n\t\tth, err := tarr.Next()\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"cannot read archive: %w\", err)\n\t\t}\n\n\t\tif th.Name == lookFor {\n\t\t\tbin = tarr\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif bin == nil {\n\t\treturn errors.New(\"no binary found in archive\")\n\t}\n\n\treturn writeToPath(bin, out)\n}\n\nfunc unpackZip(arcPath, root, name, out string) error {\n\tzipr, err := zip.OpenReader(arcPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error opening zip reader: %w\", err)\n\t}\n\tdefer zipr.Close()\n\n\tlookFor := root + \"/\" + name\n\tvar bin io.ReadCloser\n\tfor _, fis := range zipr.File {\n\t\tif fis.Name == lookFor {\n\t\t\trc, err := fis.Open()\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"error extracting binary from archive: %w\", err)\n\t\t\t}\n\n\t\t\tbin = rc\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif bin == nil {\n\t\treturn errors.New(\"no binary found in archive\")\n\t}\n\n\treturn writeToPath(bin, out)\n}\n\nfunc writeToPath(rc io.Reader, out string) error {\n\tbinfi, err := os.Create(out)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error creating output file '%s': %w\", out, err)\n\t}\n\tdefer binfi.Close()\n\n\t_, err = io.Copy(binfi, rc)\n\n\treturn err\n}\n" }, { "path": "repo/fsrepo/migrations/unpack_test.go", "content": "package migrations\n\nimport (\n\t\"archive/tar\"\n\t\"archive/zip\"\n\t\"bufio\"\n\t\"compress/gzip\"\n\t\"io\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc TestUnpackArchive(t *testing.T) {\n\t// Check unrecognized archive type\n\terr := unpackArchive(\"\", \"no-arch-type\", \"\", \"\", \"\")\n\tif err == nil || err.Error() != \"unrecognized archive type: no-arch-type\" {\n\t\tt.Fatal(\"expected 'unrecognized archive type' error\")\n\t}\n\n\t// Test cannot open errors\n\terr = unpackArchive(\"no-archive\", \"tar.gz\", \"\", \"\", \"\")\n\tif err == nil || !strings.HasPrefix(err.Error(), \"cannot open archive file\") {\n\t\tt.Fatal(\"expected 'cannot open' error, got:\", err)\n\t}\n\terr = unpackArchive(\"no-archive\", \"zip\", \"\", \"\", \"\")\n\tif err == nil || !strings.HasPrefix(err.Error(), \"error opening zip reader\") {\n\t\tt.Fatal(\"expected 'cannot open' error, got:\", err)\n\t}\n}\n\nfunc TestUnpackTgz(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tbadTarGzip := filepath.Join(tmpDir, \"bad.tar.gz\")\n\terr := os.WriteFile(badTarGzip, []byte(\"bad-data\\n\"), 0o644)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\terr = unpackTgz(badTarGzip, \"\", \"abc\", \"abc\")\n\tif err == nil || !strings.HasPrefix(err.Error(), \"error opening gzip reader\") {\n\t\tt.Fatal(\"expected error opening gzip reader, got:\", err)\n\t}\n\n\ttestTarGzip := filepath.Join(tmpDir, \"test.tar.gz\")\n\ttestData := \"some data\"\n\terr = writeTarGzipFile(testTarGzip, \"testroot\", \"testfile\", testData)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tout := filepath.Join(tmpDir, \"out.txt\")\n\n\t// Test looking for file that is not in archive\n\terr = unpackTgz(testTarGzip, \"testroot\", \"abc\", out)\n\tif err == nil || err.Error() != \"no binary found in archive\" {\n\t\tt.Fatal(\"expected 'no binary found in archive' error, got:\", err)\n\t}\n\n\t// Test that unpack works.\n\terr = unpackTgz(testTarGzip, \"testroot\", \"testfile\", out)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfi, err := os.Stat(out)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif fi.Size() != int64(len(testData)) {\n\t\tt.Fatal(\"unpacked file size is\", fi.Size(), \"expected\", len(testData))\n\t}\n}\n\nfunc TestUnpackZip(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tbadZip := filepath.Join(tmpDir, \"bad.zip\")\n\terr := os.WriteFile(badZip, []byte(\"bad-data\\n\"), 0o644)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\terr = unpackZip(badZip, \"\", \"abc\", \"abc\")\n\tif err == nil || !strings.HasPrefix(err.Error(), \"error opening zip reader\") {\n\t\tt.Fatal(\"expected error opening zip reader, got:\", err)\n\t}\n\n\ttestZip := filepath.Join(tmpDir, \"test.zip\")\n\ttestData := \"some data\"\n\terr = writeZipFile(testZip, \"testroot\", \"testfile\", testData)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tout := filepath.Join(tmpDir, \"out.txt\")\n\n\t// Test looking for file that is not in archive\n\terr = unpackZip(testZip, \"testroot\", \"abc\", out)\n\tif err == nil || err.Error() != \"no binary found in archive\" {\n\t\tt.Fatal(\"expected 'no binary found in archive' error, got:\", err)\n\t}\n\n\t// Test that unpack works.\n\terr = unpackZip(testZip, \"testroot\", \"testfile\", out)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfi, err := os.Stat(out)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif fi.Size() != int64(len(testData)) {\n\t\tt.Fatal(\"unpacked file size is\", fi.Size(), \"expected\", len(testData))\n\t}\n}\n\nfunc writeTarGzipFile(archName, root, fileName, data string) error {\n\tarchFile, err := os.Create(archName)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer archFile.Close()\n\tw := bufio.NewWriter(archFile)\n\n\terr = writeTarGzip(root, fileName, data, w)\n\tif err != nil {\n\t\treturn err\n\t}\n\t// Flush buffered data to file\n\tif err = w.Flush(); err != nil {\n\t\treturn err\n\t}\n\t// Close tar file\n\tif err = archFile.Close(); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc writeTarGzip(root, fileName, data string, w io.Writer) error {\n\t// gzip writer writes to buffer\n\tgzw := gzip.NewWriter(w)\n\tdefer gzw.Close()\n\t// tar writer writes to gzip\n\ttw := tar.NewWriter(gzw)\n\tdefer tw.Close()\n\n\tvar err error\n\tif fileName != \"\" {\n\t\thdr := &tar.Header{\n\t\t\tName: path.Join(root, fileName),\n\t\t\tMode: 0o600,\n\t\t\tSize: int64(len(data)),\n\t\t}\n\t\t// Write header\n\t\tif err = tw.WriteHeader(hdr); err != nil {\n\t\t\treturn err\n\t\t}\n\t\t// Write file body\n\t\tif _, err := tw.Write([]byte(data)); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif err = tw.Close(); err != nil {\n\t\treturn err\n\t}\n\t// Close gzip writer; finish writing gzip data to buffer\n\tif err = gzw.Close(); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc writeZipFile(archName, root, fileName, data string) error {\n\tarchFile, err := os.Create(archName)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer archFile.Close()\n\tw := bufio.NewWriter(archFile)\n\n\terr = writeZip(root, fileName, data, w)\n\tif err != nil {\n\t\treturn err\n\t}\n\t// Flush buffered data to file\n\tif err = w.Flush(); err != nil {\n\t\treturn err\n\t}\n\t// Close zip file\n\tif err = archFile.Close(); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc writeZip(root, fileName, data string, w io.Writer) error {\n\tzw := zip.NewWriter(w)\n\tdefer zw.Close()\n\n\t// Write file name\n\tf, err := zw.Create(path.Join(root, fileName))\n\tif err != nil {\n\t\treturn err\n\t}\n\t// Write file data\n\t_, err = f.Write([]byte(data))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Close zip writer\n\tif err = zw.Close(); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n" }, { "path": "repo/fsrepo/migrations/versions.go", "content": "package migrations\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"path\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/blang/semver/v4\"\n)\n\nconst distVersions = \"versions\"\n\n// LatestDistVersion returns the latest version, of the specified distribution,\n// that is available on the distribution site.\nfunc LatestDistVersion(ctx context.Context, fetcher Fetcher, dist string, stableOnly bool) (string, error) {\n\tvs, err := DistVersions(ctx, fetcher, dist, false)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tfor i := len(vs) - 1; i >= 0; i-- {\n\t\tver := vs[i]\n\t\tif stableOnly && strings.Contains(ver, \"-rc\") {\n\t\t\tcontinue\n\t\t}\n\t\tif strings.Contains(ver, \"-dev\") {\n\t\t\tcontinue\n\t\t}\n\t\treturn ver, nil\n\t}\n\treturn \"\", errors.New(\"could not find a non dev version\")\n}\n\n// DistVersions returns all versions of the specified distribution, that are\n// available on the distriburion site. List is in ascending order, unless\n// sortDesc is true.\nfunc DistVersions(ctx context.Context, fetcher Fetcher, dist string, sortDesc bool) ([]string, error) {\n\tversionBytes, err := fetcher.Fetch(ctx, path.Join(dist, distVersions))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tprefix := \"v\"\n\tvar vers []semver.Version\n\n\tscan := bufio.NewScanner(bytes.NewReader(versionBytes))\n\tfor scan.Scan() {\n\t\tver, err := semver.Make(strings.TrimLeft(scan.Text(), prefix))\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tvers = append(vers, ver)\n\t}\n\tif scan.Err() != nil {\n\t\treturn nil, fmt.Errorf(\"could not read versions: %w\", scan.Err())\n\t}\n\n\tif sortDesc {\n\t\tsort.Sort(sort.Reverse(semver.Versions(vers)))\n\t} else {\n\t\tsort.Sort(semver.Versions(vers))\n\t}\n\n\tout := make([]string, len(vers))\n\tfor i := range vers {\n\t\tout[i] = prefix + vers[i].String()\n\t}\n\n\treturn out, nil\n}\n" }, { "path": "repo/fsrepo/migrations/versions_test.go", "content": "package migrations\n\nimport (\n\t\"testing\"\n\n\t\"github.com/blang/semver/v4\"\n)\n\nconst testDist = \"go-ipfs\"\n\nfunc TestDistVersions(t *testing.T) {\n\tctx := t.Context()\n\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\tvers, err := DistVersions(ctx, fetcher, testDist, true)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(vers) == 0 {\n\t\tt.Fatal(\"no versions of\", testDist)\n\t}\n\tt.Log(\"There are\", len(vers), \"versions of\", testDist)\n\tt.Log(\"Latest 5 are:\", vers[:5])\n}\n\nfunc TestLatestDistVersion(t *testing.T) {\n\tctx := t.Context()\n\n\tfetcher := NewHttpFetcher(testIpfsDist, testServer.URL, \"\", 0)\n\n\tlatest, err := LatestDistVersion(ctx, fetcher, testDist, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(latest) < 6 {\n\t\tt.Fatal(\"latest version string too short\", latest)\n\t}\n\t_, err = semver.New(latest[1:])\n\tif err != nil {\n\t\tt.Fatal(\"latest version has invalid format:\", latest)\n\t}\n\tt.Log(\"Latest version of\", testDist, \"is\", latest)\n}\n" }, { "path": "repo/fsrepo/misc.go", "content": "package fsrepo\n\nimport (\n\t\"os\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/misc/fsutil\"\n)\n\n// BestKnownPath returns the best known fsrepo path. If the ENV override is\n// present, this function returns that value. Otherwise, it returns the default\n// repo path.\nfunc BestKnownPath() (string, error) {\n\tipfsPath := config.DefaultPathRoot\n\tif os.Getenv(config.EnvDir) != \"\" {\n\t\tipfsPath = os.Getenv(config.EnvDir)\n\t}\n\tipfsPath, err := fsutil.ExpandHome(ipfsPath)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn ipfsPath, nil\n}\n" }, { "path": "repo/mock.go", "content": "package repo\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"net\"\n\n\tfilestore \"github.com/ipfs/boxo/filestore\"\n\tkeystore \"github.com/ipfs/boxo/keystore\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\n\tconfig \"github.com/ipfs/kubo/config\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\nvar errTODO = errors.New(\"TODO: mock repo\")\n\n// Mock is not thread-safe.\ntype Mock struct {\n\tC config.Config\n\tD Datastore\n\tK keystore.Keystore\n\tF *filestore.FileManager\n}\n\nfunc (m *Mock) Config() (*config.Config, error) {\n\treturn &m.C, nil // FIXME threadsafety\n}\n\nfunc (m *Mock) Path() string {\n\treturn \"\"\n}\n\nfunc (m *Mock) UserResourceOverrides() (rcmgr.PartialLimitConfig, error) {\n\treturn rcmgr.PartialLimitConfig{}, nil\n}\n\nfunc (m *Mock) SetConfig(updated *config.Config) error {\n\tm.C = *updated // FIXME threadsafety\n\treturn nil\n}\n\nfunc (m *Mock) BackupConfig(prefix string) (string, error) {\n\treturn \"\", errTODO\n}\n\nfunc (m *Mock) SetConfigKey(key string, value any) error {\n\treturn errTODO\n}\n\nfunc (m *Mock) GetConfigKey(key string) (any, error) {\n\treturn nil, errTODO\n}\n\nfunc (m *Mock) Datastore() Datastore { return m.D }\n\nfunc (m *Mock) GetStorageUsage(_ context.Context) (uint64, error) { return 0, nil }\n\nfunc (m *Mock) Close() error { return m.D.Close() }\n\nfunc (m *Mock) SetAPIAddr(addr ma.Multiaddr) error { return errTODO }\n\nfunc (m *Mock) SetGatewayAddr(addr net.Addr) error { return errTODO }\n\nfunc (m *Mock) Keystore() keystore.Keystore { return m.K }\n\nfunc (m *Mock) SwarmKey() ([]byte, error) {\n\treturn nil, nil\n}\n\nfunc (m *Mock) FileManager() *filestore.FileManager { return m.F }\n" }, { "path": "repo/onlyone.go", "content": "package repo\n\nimport (\n\t\"sync\"\n)\n\n// OnlyOne tracks open Repos by arbitrary key and returns the already\n// open one.\ntype OnlyOne struct {\n\tmu sync.Mutex\n\tactive map[any]*ref\n}\n\n// Open a Repo identified by key. If Repo is not already open, the\n// open function is called, and the result is remembered for further\n// use.\n//\n// Key must be comparable, or Open will panic. Make sure to pick keys\n// that are unique across different concrete Repo implementations,\n// e.g. by creating a local type:\n//\n//\ttype repoKey string\n//\tr, err := o.Open(repoKey(path), open)\n//\n// Call Repo.Close when done.\nfunc (o *OnlyOne) Open(key any, open func() (Repo, error)) (Repo, error) {\n\to.mu.Lock()\n\tdefer o.mu.Unlock()\n\tif o.active == nil {\n\t\to.active = make(map[any]*ref)\n\t}\n\n\titem, found := o.active[key]\n\tif !found {\n\t\trepo, err := open()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\titem = &ref{\n\t\t\tparent: o,\n\t\t\tkey: key,\n\t\t\tRepo: repo,\n\t\t}\n\t\to.active[key] = item\n\t}\n\titem.refs++\n\treturn item, nil\n}\n\ntype ref struct {\n\tparent *OnlyOne\n\tkey any\n\trefs uint32\n\tRepo\n}\n\nvar _ Repo = (*ref)(nil)\n\nfunc (r *ref) Close() error {\n\tr.parent.mu.Lock()\n\tdefer r.parent.mu.Unlock()\n\n\tr.refs--\n\tif r.refs > 0 {\n\t\t// others are holding it open\n\t\treturn nil\n\t}\n\n\t// last one\n\tdelete(r.parent.active, r.key)\n\treturn r.Repo.Close()\n}\n" }, { "path": "repo/repo.go", "content": "package repo\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"net\"\n\n\tfilestore \"github.com/ipfs/boxo/filestore\"\n\tkeystore \"github.com/ipfs/boxo/keystore\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\n\tds \"github.com/ipfs/go-datastore\"\n\tconfig \"github.com/ipfs/kubo/config\"\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\nvar ErrApiNotRunning = errors.New(\"api not running\") //nolint\n\n// Repo represents all persistent data of a given ipfs node.\ntype Repo interface {\n\t// Config returns the ipfs configuration file from the repo. Changes made\n\t// to the returned config are not automatically persisted.\n\tConfig() (*config.Config, error)\n\n\t// Path is the repo file-system path\n\tPath() string\n\n\t// UserResourceOverrides returns optional user resource overrides for the\n\t// libp2p resource manager.\n\tUserResourceOverrides() (rcmgr.PartialLimitConfig, error)\n\n\t// BackupConfig creates a backup of the current configuration file using\n\t// the given prefix for naming.\n\tBackupConfig(prefix string) (string, error)\n\n\t// SetConfig persists the given configuration struct to storage.\n\tSetConfig(*config.Config) error\n\n\t// SetConfigKey sets the given key-value pair within the config and persists it to storage.\n\tSetConfigKey(key string, value any) error\n\n\t// GetConfigKey reads the value for the given key from the configuration in storage.\n\tGetConfigKey(key string) (any, error)\n\n\t// Datastore returns a reference to the configured data storage backend.\n\tDatastore() Datastore\n\n\t// GetStorageUsage returns the number of bytes stored.\n\tGetStorageUsage(context.Context) (uint64, error)\n\n\t// Keystore returns a reference to the key management interface.\n\tKeystore() keystore.Keystore\n\n\t// FileManager returns a reference to the filestore file manager.\n\tFileManager() *filestore.FileManager\n\n\t// SetAPIAddr sets the API address in the repo.\n\tSetAPIAddr(addr ma.Multiaddr) error\n\n\t// SetGatewayAddr sets the Gateway address in the repo.\n\tSetGatewayAddr(addr net.Addr) error\n\n\t// SwarmKey returns the configured shared symmetric key for the private networks feature.\n\tSwarmKey() ([]byte, error)\n\n\tio.Closer\n}\n\n// Datastore is the interface required from a datastore to be\n// acceptable to FSRepo.\ntype Datastore interface {\n\tds.Batching // must be thread-safe\n}\n" }, { "path": "routing/composer.go", "content": "package routing\n\nimport (\n\t\"context\"\n\t\"errors\"\n\n\t\"github.com/ipfs/go-cid\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\t\"github.com/multiformats/go-multihash\"\n)\n\nvar (\n\t_ routinghelpers.ProvideManyRouter = &Composer{}\n\t_ routing.Routing = &Composer{}\n)\n\ntype Composer struct {\n\tGetValueRouter routing.Routing\n\tPutValueRouter routing.Routing\n\tFindPeersRouter routing.Routing\n\tFindProvidersRouter routing.Routing\n\tProvideRouter routing.Routing\n}\n\nfunc (c *Composer) Provide(ctx context.Context, cid cid.Cid, provide bool) error {\n\tlog.Debug(\"composer: calling provide: \", cid)\n\terr := c.ProvideRouter.Provide(ctx, cid, provide)\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling provide: \", cid, \" error: \", err)\n\t}\n\n\treturn err\n}\n\nfunc (c *Composer) ProvideMany(ctx context.Context, keys []multihash.Multihash) error {\n\tlog.Debug(\"composer: calling provide many: \", len(keys))\n\tpmr, ok := c.ProvideRouter.(routinghelpers.ProvideManyRouter)\n\tif !ok {\n\t\tlog.Debug(\"composer: provide many is not implemented on the actual router\")\n\t\treturn nil\n\t}\n\n\terr := pmr.ProvideMany(ctx, keys)\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling provide many error: \", err)\n\t}\n\n\treturn err\n}\n\nfunc (c *Composer) Ready() bool {\n\tlog.Debug(\"composer: calling ready\")\n\tpmr, ok := c.ProvideRouter.(routinghelpers.ReadyAbleRouter)\n\tif !ok {\n\t\treturn true\n\t}\n\n\tready := pmr.Ready()\n\n\tlog.Debug(\"composer: calling ready result: \", ready)\n\n\treturn ready\n}\n\nfunc (c *Composer) FindProvidersAsync(ctx context.Context, cid cid.Cid, count int) <-chan peer.AddrInfo {\n\tlog.Debug(\"composer: calling findProvidersAsync: \", cid)\n\treturn c.FindProvidersRouter.FindProvidersAsync(ctx, cid, count)\n}\n\nfunc (c *Composer) FindPeer(ctx context.Context, pid peer.ID) (peer.AddrInfo, error) {\n\tlog.Debug(\"composer: calling findPeer: \", pid)\n\taddr, err := c.FindPeersRouter.FindPeer(ctx, pid)\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling findPeer error: \", pid, addr.String(), err)\n\t}\n\treturn addr, err\n}\n\nfunc (c *Composer) PutValue(ctx context.Context, key string, val []byte, opts ...routing.Option) error {\n\tlog.Debug(\"composer: calling putValue: \", key, len(val))\n\terr := c.PutValueRouter.PutValue(ctx, key, val, opts...)\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling putValue error: \", key, len(val), err)\n\t}\n\n\treturn err\n}\n\nfunc (c *Composer) GetValue(ctx context.Context, key string, opts ...routing.Option) ([]byte, error) {\n\tlog.Debug(\"composer: calling getValue: \", key)\n\tval, err := c.GetValueRouter.GetValue(ctx, key, opts...)\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling getValue error: \", key, len(val), err)\n\t}\n\n\treturn val, err\n}\n\nfunc (c *Composer) SearchValue(ctx context.Context, key string, opts ...routing.Option) (<-chan []byte, error) {\n\tlog.Debug(\"composer: calling searchValue: \", key)\n\tch, err := c.GetValueRouter.SearchValue(ctx, key, opts...)\n\n\t// avoid nil channels on implementations not supporting SearchValue method.\n\tif errors.Is(err, routing.ErrNotFound) && ch == nil {\n\t\tout := make(chan []byte)\n\t\tclose(out)\n\t\treturn out, err\n\t}\n\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling searchValue error: \", key, err)\n\t}\n\n\treturn ch, err\n}\n\nfunc (c *Composer) Bootstrap(ctx context.Context) error {\n\tlog.Debug(\"composer: calling bootstrap\")\n\terrfp := c.FindPeersRouter.Bootstrap(ctx)\n\terrfps := c.FindProvidersRouter.Bootstrap(ctx)\n\terrgv := c.GetValueRouter.Bootstrap(ctx)\n\terrpv := c.PutValueRouter.Bootstrap(ctx)\n\terrp := c.ProvideRouter.Bootstrap(ctx)\n\terr := errors.Join(errfp, errfps, errgv, errpv, errp)\n\tif err != nil {\n\t\tlog.Debug(\"composer: calling bootstrap error: \", err)\n\t}\n\treturn err\n}\n" }, { "path": "routing/delegated.go", "content": "package routing\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"errors\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"path\"\n\t\"strings\"\n\n\tdrclient \"github.com/ipfs/boxo/routing/http/client\"\n\t\"github.com/ipfs/boxo/routing/http/contentrouter\"\n\t\"github.com/ipfs/go-datastore\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/config\"\n\tdht \"github.com/libp2p/go-libp2p-kad-dht\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/dual\"\n\t\"github.com/libp2p/go-libp2p-kad-dht/fullrt\"\n\trecord \"github.com/libp2p/go-libp2p-record\"\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\tic \"github.com/libp2p/go-libp2p/core/crypto\"\n\thost \"github.com/libp2p/go-libp2p/core/host\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\t\"go.opencensus.io/stats/view\"\n\t\"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp\"\n)\n\nvar log = logging.Logger(\"routing/delegated\")\n\n// Parse creates a composed router from the custom routing configuration.\n//\n// EXPERIMENTAL: Custom routing (Routing.Type=custom with Routing.Routers and\n// Routing.Methods) is for research and testing only, not production use.\n// The configuration format and behavior may change without notice between\n// releases. HTTP-only configurations cannot reliably provide content.\n// See docs/delegated-routing.md for limitations.\nfunc Parse(routers config.Routers, methods config.Methods, extraDHT *ExtraDHTParams, extraHTTP *ExtraHTTPParams) (routing.Routing, error) {\n\tif err := methods.Check(); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcreatedRouters := make(map[string]routing.Routing)\n\tfinalRouter := &Composer{}\n\n\t// Create all needed routers from method names\n\tfor mn, m := range methods {\n\t\trouter, err := parse(make(map[string]bool), createdRouters, m.RouterName, routers, extraDHT, extraHTTP)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tswitch mn {\n\t\tcase config.MethodNamePutIPNS:\n\t\t\tfinalRouter.PutValueRouter = router\n\t\tcase config.MethodNameGetIPNS:\n\t\t\tfinalRouter.GetValueRouter = router\n\t\tcase config.MethodNameFindPeers:\n\t\t\tfinalRouter.FindPeersRouter = router\n\t\tcase config.MethodNameFindProviders:\n\t\t\tfinalRouter.FindProvidersRouter = router\n\t\tcase config.MethodNameProvide:\n\t\t\tfinalRouter.ProvideRouter = router\n\t\t}\n\n\t\tlog.Info(\"using method \", mn, \" with router \", m.RouterName)\n\t}\n\n\treturn finalRouter, nil\n}\n\nfunc parse(visited map[string]bool,\n\tcreatedRouters map[string]routing.Routing,\n\trouterName string,\n\troutersCfg config.Routers,\n\textraDHT *ExtraDHTParams,\n\textraHTTP *ExtraHTTPParams,\n) (routing.Routing, error) {\n\t// check if we already created it\n\tr, ok := createdRouters[routerName]\n\tif ok {\n\t\treturn r, nil\n\t}\n\n\t// check if we are in a dep loop\n\tif visited[routerName] {\n\t\treturn nil, fmt.Errorf(\"dependency loop creating router with name %q\", routerName)\n\t}\n\n\t// set node as visited\n\tvisited[routerName] = true\n\n\tcfg, ok := routersCfg[routerName]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"config for router with name %q not found\", routerName)\n\t}\n\n\tvar router routing.Routing\n\tvar err error\n\tswitch cfg.Type {\n\tcase config.RouterTypeHTTP:\n\t\trouter, err = httpRoutingFromConfig(cfg.Router, extraHTTP)\n\tcase config.RouterTypeDHT:\n\t\trouter, err = dhtRoutingFromConfig(cfg.Router, extraDHT)\n\tcase config.RouterTypeParallel:\n\t\tcrp := cfg.Parameters.(*config.ComposableRouterParams)\n\t\tvar pr []*routinghelpers.ParallelRouter\n\t\tfor _, cr := range crp.Routers {\n\t\t\tri, err := parse(visited, createdRouters, cr.RouterName, routersCfg, extraDHT, extraHTTP)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tpr = append(pr, &routinghelpers.ParallelRouter{\n\t\t\t\tRouter: ri,\n\t\t\t\tIgnoreError: cr.IgnoreErrors,\n\t\t\t\tDoNotWaitForSearchValue: true,\n\t\t\t\tTimeout: cr.Timeout.Duration,\n\t\t\t\tExecuteAfter: cr.ExecuteAfter.WithDefault(0),\n\t\t\t})\n\n\t\t}\n\n\t\trouter = routinghelpers.NewComposableParallel(pr)\n\tcase config.RouterTypeSequential:\n\t\tcrp := cfg.Parameters.(*config.ComposableRouterParams)\n\t\tvar sr []*routinghelpers.SequentialRouter\n\t\tfor _, cr := range crp.Routers {\n\t\t\tri, err := parse(visited, createdRouters, cr.RouterName, routersCfg, extraDHT, extraHTTP)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tsr = append(sr, &routinghelpers.SequentialRouter{\n\t\t\t\tRouter: ri,\n\t\t\t\tIgnoreError: cr.IgnoreErrors,\n\t\t\t\tTimeout: cr.Timeout.Duration,\n\t\t\t})\n\n\t\t}\n\n\t\trouter = routinghelpers.NewComposableSequential(sr)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unknown router type %q\", cfg.Type)\n\t}\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcreatedRouters[routerName] = router\n\n\tlog.Info(\"created router \", routerName, \" with params \", cfg.Parameters)\n\n\treturn router, nil\n}\n\ntype ExtraHTTPParams struct {\n\tPeerID string\n\tAddrFunc func() []ma.Multiaddr // dynamic address resolver for provider records\n\tPrivKeyB64 string\n\tHTTPRetrieval bool\n}\n\nfunc ConstructHTTPRouter(endpoint string, peerID string, addrFunc func() []ma.Multiaddr, privKey string, httpRetrieval bool) (routing.Routing, error) {\n\treturn httpRoutingFromConfig(\n\t\tconfig.Router{\n\t\t\tType: \"http\",\n\t\t\tParameters: &config.HTTPRouterParams{\n\t\t\t\tEndpoint: endpoint,\n\t\t\t},\n\t\t},\n\t\t&ExtraHTTPParams{\n\t\t\tPeerID: peerID,\n\t\t\tAddrFunc: addrFunc,\n\t\t\tPrivKeyB64: privKey,\n\t\t\tHTTPRetrieval: httpRetrieval,\n\t\t},\n\t)\n}\n\nfunc httpRoutingFromConfig(conf config.Router, extraHTTP *ExtraHTTPParams) (routing.Routing, error) {\n\tparams := conf.Parameters.(*config.HTTPRouterParams)\n\tif params.Endpoint == \"\" {\n\t\treturn nil, NewParamNeededErr(\"Endpoint\", conf.Type)\n\t}\n\n\tparams.FillDefaults()\n\n\t// Increase per-host connection pool since we are making lots of concurrent requests.\n\ttransport := http.DefaultTransport.(*http.Transport).Clone()\n\ttransport.MaxIdleConns = 500\n\ttransport.MaxIdleConnsPerHost = 100\n\n\tdelegateHTTPClient := &http.Client{\n\t\tTransport: &drclient.ResponseBodyLimitedTransport{\n\t\t\tRoundTripper: otelhttp.NewTransport(transport,\n\t\t\t\totelhttp.WithSpanNameFormatter(func(operation string, req *http.Request) string {\n\t\t\t\t\tif req.Method == http.MethodGet {\n\t\t\t\t\t\tswitch {\n\t\t\t\t\t\tcase strings.HasPrefix(req.URL.Path, \"/routing/v1/providers\"):\n\t\t\t\t\t\t\treturn \"DelegatedHTTPClient.FindProviders\"\n\t\t\t\t\t\tcase strings.HasPrefix(req.URL.Path, \"/routing/v1/peers\"):\n\t\t\t\t\t\t\treturn \"DelegatedHTTPClient.FindPeers\"\n\t\t\t\t\t\tcase strings.HasPrefix(req.URL.Path, \"/routing/v1/ipns\"):\n\t\t\t\t\t\t\treturn \"DelegatedHTTPClient.GetIPNS\"\n\t\t\t\t\t\t}\n\t\t\t\t\t} else if req.Method == http.MethodPut {\n\t\t\t\t\t\tswitch {\n\t\t\t\t\t\tcase strings.HasPrefix(req.URL.Path, \"/routing/v1/ipns\"):\n\t\t\t\t\t\t\treturn \"DelegatedHTTPClient.PutIPNS\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\treturn \"DelegatedHTTPClient.\" + path.Dir(req.URL.Path)\n\t\t\t\t}),\n\t\t\t),\n\t\t\tLimitBytes: 1 << 20,\n\t\t},\n\t}\n\n\tkey, err := decodePrivKey(extraHTTP.PrivKeyB64)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tprotocols := config.DefaultHTTPRoutersFilterProtocols\n\tif extraHTTP.HTTPRetrieval {\n\t\tprotocols = append(protocols, \"transport-ipfs-gateway-http\")\n\t}\n\n\tpeerID, err := peer.Decode(extraHTTP.PeerID)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar providerInfoOpt drclient.Option\n\tif extraHTTP.AddrFunc != nil {\n\t\tproviderInfoOpt = drclient.WithProviderInfoFunc(peerID, extraHTTP.AddrFunc)\n\t} else {\n\t\tproviderInfoOpt = drclient.WithProviderInfo(peerID, nil)\n\t}\n\n\tcli, err := drclient.New(\n\t\tparams.Endpoint,\n\t\tdrclient.WithHTTPClient(delegateHTTPClient),\n\t\tdrclient.WithIdentity(key),\n\t\tproviderInfoOpt,\n\t\tdrclient.WithUserAgent(version.GetUserAgentVersion()),\n\t\tdrclient.WithProtocolFilter(protocols),\n\t\tdrclient.WithStreamResultsRequired(), // https://specs.ipfs.tech/routing/http-routing-v1/#streaming\n\t\tdrclient.WithDisabledLocalFiltering(false), // force local filtering in case remote server does not support IPIP-484\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcr := contentrouter.NewContentRoutingClient(\n\t\tcli,\n\t\tcontentrouter.WithMaxProvideBatchSize(params.MaxProvideBatchSize),\n\t\tcontentrouter.WithMaxProvideConcurrency(params.MaxProvideConcurrency),\n\t)\n\n\terr = view.Register(drclient.OpenCensusViews...)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"registering HTTP delegated routing views: %w\", err)\n\t}\n\n\treturn &httpRoutingWrapper{\n\t\tContentRouting: cr,\n\t\tPeerRouting: cr,\n\t\tValueStore: cr,\n\t\tProvideManyRouter: cr,\n\t}, nil\n}\n\nfunc decodePrivKey(keyB64 string) (ic.PrivKey, error) {\n\tpk, err := base64.StdEncoding.DecodeString(keyB64)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ic.UnmarshalPrivateKey(pk)\n}\n\ntype ExtraDHTParams struct {\n\tBootstrapPeers []peer.AddrInfo\n\tHost host.Host\n\tValidator record.Validator\n\tDatastore datastore.Batching\n\tContext context.Context\n}\n\nfunc dhtRoutingFromConfig(conf config.Router, extra *ExtraDHTParams) (routing.Routing, error) {\n\tparams, ok := conf.Parameters.(*config.DHTRouterParams)\n\tif !ok {\n\t\treturn nil, errors.New(\"incorrect params for DHT router\")\n\t}\n\n\tif params.AcceleratedDHTClient {\n\t\treturn createFullRT(extra)\n\t}\n\n\tvar mode dht.ModeOpt\n\tswitch params.Mode {\n\tcase config.DHTModeAuto:\n\t\tmode = dht.ModeAuto\n\tcase config.DHTModeClient:\n\t\tmode = dht.ModeClient\n\tcase config.DHTModeServer:\n\t\tmode = dht.ModeServer\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"invalid DHT mode: %q\", params.Mode)\n\t}\n\n\treturn createDHT(extra, params.PublicIPNetwork, mode)\n}\n\nfunc createDHT(params *ExtraDHTParams, public bool, mode dht.ModeOpt) (routing.Routing, error) {\n\tvar opts []dht.Option\n\n\tif public {\n\t\topts = append(opts, dht.QueryFilter(dht.PublicQueryFilter),\n\t\t\tdht.RoutingTableFilter(dht.PublicRoutingTableFilter),\n\t\t\tdht.RoutingTablePeerDiversityFilter(dht.NewRTPeerDiversityFilter(params.Host, 2, 3)))\n\t} else {\n\t\topts = append(opts, dht.ProtocolExtension(dual.LanExtension),\n\t\t\tdht.QueryFilter(dht.PrivateQueryFilter),\n\t\t\tdht.RoutingTableFilter(dht.PrivateRoutingTableFilter))\n\t}\n\n\topts = append(opts,\n\t\tdht.Concurrency(10),\n\t\tdht.Mode(mode),\n\t\tdht.Datastore(params.Datastore),\n\t\tdht.Validator(params.Validator),\n\t\tdht.BootstrapPeers(params.BootstrapPeers...))\n\n\treturn dht.New(\n\t\tparams.Context, params.Host, opts...,\n\t)\n}\n\nfunc createFullRT(params *ExtraDHTParams) (routing.Routing, error) {\n\treturn fullrt.NewFullRT(params.Host,\n\t\tdht.DefaultPrefix,\n\t\tfullrt.DHTOption(\n\t\t\tdht.Validator(params.Validator),\n\t\t\tdht.Datastore(params.Datastore),\n\t\t\tdht.BootstrapPeers(params.BootstrapPeers...),\n\t\t\tdht.BucketSize(20),\n\t\t),\n\t)\n}\n" }, { "path": "routing/delegated_test.go", "content": "package routing\n\nimport (\n\t\"crypto/rand\"\n\t\"encoding/base64\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/libp2p/go-libp2p/core/crypto\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestParser(t *testing.T) {\n\trequire := require.New(t)\n\n\tpid, sk, err := generatePeerID()\n\trequire.NoError(err)\n\n\trouter, err := Parse(config.Routers{\n\t\t\"r1\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeHTTP,\n\t\t\t\tParameters: &config.HTTPRouterParams{\n\t\t\t\t\tEndpoint: \"http://testEndpoint\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"r2\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeSequential,\n\t\t\t\tParameters: &config.ComposableRouterParams{\n\t\t\t\t\tRouters: []config.ConfigRouter{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"r1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, config.Methods{\n\t\tconfig.MethodNameFindPeers: config.Method{\n\t\t\tRouterName: \"r1\",\n\t\t},\n\t\tconfig.MethodNameFindProviders: config.Method{\n\t\t\tRouterName: \"r1\",\n\t\t},\n\t\tconfig.MethodNameGetIPNS: config.Method{\n\t\t\tRouterName: \"r1\",\n\t\t},\n\t\tconfig.MethodNamePutIPNS: config.Method{\n\t\t\tRouterName: \"r2\",\n\t\t},\n\t\tconfig.MethodNameProvide: config.Method{\n\t\t\tRouterName: \"r2\",\n\t\t},\n\t}, &ExtraDHTParams{}, &ExtraHTTPParams{\n\t\tPeerID: string(pid),\n\t\tPrivKeyB64: sk,\n\t})\n\n\trequire.NoError(err)\n\n\tcomp, ok := router.(*Composer)\n\trequire.True(ok)\n\n\trequire.Equal(comp.FindPeersRouter, comp.FindProvidersRouter)\n\trequire.Equal(comp.ProvideRouter, comp.PutValueRouter)\n}\n\nfunc TestParserRecursive(t *testing.T) {\n\trequire := require.New(t)\n\n\tpid, sk, err := generatePeerID()\n\trequire.NoError(err)\n\n\trouter, err := Parse(config.Routers{\n\t\t\"http1\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeHTTP,\n\t\t\t\tParameters: &config.HTTPRouterParams{\n\t\t\t\t\tEndpoint: \"http://testEndpoint1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"http2\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeHTTP,\n\t\t\t\tParameters: &config.HTTPRouterParams{\n\t\t\t\t\tEndpoint: \"http://testEndpoint2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"http3\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeHTTP,\n\t\t\t\tParameters: &config.HTTPRouterParams{\n\t\t\t\t\tEndpoint: \"http://testEndpoint3\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"composable1\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeSequential,\n\t\t\t\tParameters: &config.ComposableRouterParams{\n\t\t\t\t\tRouters: []config.ConfigRouter{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"http1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"http2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"composable2\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeParallel,\n\t\t\t\tParameters: &config.ComposableRouterParams{\n\t\t\t\t\tRouters: []config.ConfigRouter{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"composable1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"http3\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, config.Methods{\n\t\tconfig.MethodNameFindPeers: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNameFindProviders: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNameGetIPNS: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNamePutIPNS: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNameProvide: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t}, &ExtraDHTParams{}, &ExtraHTTPParams{\n\t\tPeerID: string(pid),\n\t\tPrivKeyB64: sk,\n\t})\n\n\trequire.NoError(err)\n\n\t_, ok := router.(*Composer)\n\trequire.True(ok)\n}\n\nfunc TestParserRecursiveLoop(t *testing.T) {\n\trequire := require.New(t)\n\n\t_, err := Parse(config.Routers{\n\t\t\"composable1\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeSequential,\n\t\t\t\tParameters: &config.ComposableRouterParams{\n\t\t\t\t\tRouters: []config.ConfigRouter{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"composable2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"composable2\": config.RouterParser{\n\t\t\tRouter: config.Router{\n\t\t\t\tType: config.RouterTypeParallel,\n\t\t\t\tParameters: &config.ComposableRouterParams{\n\t\t\t\t\tRouters: []config.ConfigRouter{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tRouterName: \"composable1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, config.Methods{\n\t\tconfig.MethodNameFindPeers: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNameFindProviders: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNameGetIPNS: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNamePutIPNS: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t\tconfig.MethodNameProvide: config.Method{\n\t\t\tRouterName: \"composable2\",\n\t\t},\n\t}, &ExtraDHTParams{}, nil)\n\n\trequire.ErrorContains(err, \"dependency loop creating router with name \\\"composable2\\\"\")\n}\n\nfunc generatePeerID() (string, string, error) {\n\tsk, pk, err := crypto.GenerateEd25519Key(rand.Reader)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\n\tbytes, err := crypto.MarshalPrivateKey(sk)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\n\tenc := base64.StdEncoding.EncodeToString(bytes)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\n\tpid, err := peer.IDFromPublicKey(pk)\n\treturn pid.String(), enc, err\n}\n" }, { "path": "routing/error.go", "content": "package routing\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/ipfs/kubo/config\"\n)\n\ntype ParamNeededError struct {\n\tParamName string\n\tRouterType config.RouterType\n}\n\nfunc NewParamNeededErr(param string, routing config.RouterType) error {\n\treturn &ParamNeededError{\n\t\tParamName: param,\n\t\tRouterType: routing,\n\t}\n}\n\nfunc (e *ParamNeededError) Error() string {\n\treturn fmt.Sprintf(\"configuration param '%v' is needed for %v delegated routing types\", e.ParamName, e.RouterType)\n}\n" }, { "path": "routing/wrapper.go", "content": "package routing\n\nimport (\n\t\"context\"\n\n\troutinghelpers \"github.com/libp2p/go-libp2p-routing-helpers\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n)\n\ntype ProvideManyRouter interface {\n\troutinghelpers.ProvideManyRouter\n\trouting.Routing\n}\n\nvar (\n\t_ routing.Routing = &httpRoutingWrapper{}\n\t_ routinghelpers.ProvideManyRouter = &httpRoutingWrapper{}\n)\n\n// httpRoutingWrapper is a wrapper needed to construct the routing.Routing interface from\n// http delegated routing.\ntype httpRoutingWrapper struct {\n\trouting.ContentRouting\n\trouting.PeerRouting\n\trouting.ValueStore\n\troutinghelpers.ProvideManyRouter\n}\n\nfunc (c *httpRoutingWrapper) Bootstrap(ctx context.Context) error {\n\treturn nil\n}\n" }, { "path": "test/.gitignore", "content": "IPFS-BUILD-OPTIONS\n" }, { "path": "test/3nodetest/GNUmakefile", "content": "IMAGE_NAME = ipfs-test-latest\nIPFS_ROOT = ../..\n\ntest: clean setup\n\t./run-test-on-img.sh $(IMAGE_NAME)\n\nsetup: docker_ipfs_image data/filetiny data/filerand\n\nsave_logs:\n\tsh bin/save_logs.sh\n\nsave_profiling_data:\n\tsh bin/save_profiling_data.sh\n\ndata/filetiny: Makefile\n\tcp Makefile ./data/filetiny # simple\n\ndata/filerand: ../bin/random\n\t../bin/random 50000000 > ./data/filerand\n\n../bin/random:\n\tmake -C ./../../ test/bin/random\n\n# just build it every time... this part isn't\n# even the lengthy part, and it decreases pain.\ndocker_ipfs_image:\n\tdocker build -t $(IMAGE_NAME) -f Dockerfile .\n\tdocker images | grep $(IMAGE_NAME)\n\nclean:\n\tsh bin/clean.sh\n\tfig stop\n\tfig rm -v --force\n\trm -f bin/random\n\trm -f data/filetiny\n\trm -f data/filerand\n\trm -rf build/*\n\tdocker rmi $(docker images | grep \"^\" | awk \"{print \\$3}\") -f || true\n" }, { "path": "test/3nodetest/README.md", "content": "this is an ipfs integration test\n\n**requirements**\n\n* Docker\n* fig\n* Go\n\n* ipfs image named \"zaqwsx_ipfs-test-img\"\n\n```\nmake setup\nfig build \nfig up\n```\n" }, { "path": "test/3nodetest/bin/.gitignore", "content": "random\n" }, { "path": "test/3nodetest/bin/clean.sh", "content": "docker rm -f $( docker ps -q -a -f status=exited ) || true\n" }, { "path": "test/3nodetest/bin/save_logs.sh", "content": "# STRIP strips color from terminal output\nSTRIP=\"perl -pe 's/\\e\\[?.*?[\\@-~]//g'\"\n\n# TODO use a for loop like a grownup\ndocker logs 3nodetest_bootstrap_1 2>&1 | eval $STRIP > ./build/bootstrap.log\ndocker logs 3nodetest_client_1 2>&1 | eval $STRIP > ./build/client.log\ndocker logs 3nodetest_data_1 2>&1 | eval $STRIP > ./build/data.log\ndocker logs 3nodetest_server_1 2>&1 | eval $STRIP > ./build/server.log\n" }, { "path": "test/3nodetest/bin/save_profiling_data.sh", "content": "#!/bin/sh\n\nfor container in 3nodetest_bootstrap_1 3nodetest_client_1 3nodetest_server_1; do\n # ipfs binary is required by `go tool pprof`\n docker cp $container:/go/bin/ipfs build/profiling_data_$container\ndone\n\n# since the nodes are executed with the --debug flag, profiling data is written\n# to the working dir. by default, the working dir is /go.\n\nfor container in 3nodetest_bootstrap_1 3nodetest_client_1 3nodetest_server_1; do\n docker cp $container:/go/ipfs.cpuprof build/profiling_data_$container\ndone\n\n# TODO get memprof from client (client daemon isn't terminated, so memprof isn't retrieved)\nfor container in 3nodetest_bootstrap_1 3nodetest_server_1; do\n docker cp $container:/go/ipfs.memprof build/profiling_data_$container\ndone\n" }, { "path": "test/3nodetest/bootstrap/Dockerfile", "content": "FROM zaqwsx_ipfs-test-img\n\nRUN ipfs init -b=2048\nADD . /tmp/id\nRUN mv -f /tmp/id/config /root/.ipfs/config\nRUN ipfs id\n\nENV IPFS_PROF true\nENV GOLOG_LOG_FMT nocolor\n\nEXPOSE 4011 4012/udp\n" }, { "path": "test/3nodetest/bootstrap/README.md", "content": "this is a bootstrap peer with an empty bootstrap list\n\nit listens on 4011 and 4012\n" }, { "path": "test/3nodetest/bootstrap/config", "content": "{\n \"Identity\": {\n \"PeerID\": \"QmNXuBh8HFsWq68Fid8dMbGNQTh7eG6hV9rr1fQyfmfomE\",\n \"PrivKey\": \"CAAS4gQwggJeAgEAAoGBAL+E7A0fcQS9+CHO3YAHj+JzHnWyVA7qqtiAIYbTnp9UvHBb2VFj2Q8eeyKFZD5wHoq3AtOqmIb4TUOMEtWYqXnE8o0T9np8vyCRK5dPn5SVoUw9uax6o2X7OxO1HqTcXHNHGbracawJUdwsk4yuZUpzXLez03yocWwneR0JpVJPAgMBAAECgYAXsa4ygW1OFOKZ7CnjKQxYC738+a8EmWvBlTiQoaXCOI2HqRVdyGiWQkMhpjccsmpU5wdmgHiWWinU7YN3AYgV3cP3qAjyNLBFoxy2dKsS9AOWVwRuuRP12tD05kCCjG4rJAX0JEOClOOtzvQ7/bXarMc3/tMHW7TMLNV8MzcYwQJBAOP9aYSHp8VnsO5j32Ju5SjOQorSdcCweqeUxwlAnXz50KdbNSCMypP3TOt7VeiXTuSITtN44yh+eogF5c4ehycCQQDXDHVmPeBN7uqqqZxZwW5pdeJWvx+REiXXCLE6KEPWlcxbw1D9ublpCCFLYuM68rjq1sjsIVGtiV1tYoMdHJSZAkEA0ddMZ070fB0UHFaQJGktQoGVfXB4MQI94kBtcXanfX/xLBgmre7oBYh4o8TBLXMWigFri/iYG41N+iRzf2NZwQJBAIh8rMpufT2ZZLFaoxRIc4ZVvojmFufhR8j6CFnsElpQivq2tWHEDcx+z3rkUWopgXnzRmSwJQHqTDTPsH26lQkCQQCehmxqaQEwE/QKAI8L8YVolgY2cjUGi6qF/awnI584lDSCuJRU3R/c6nc9R8qljtlJYTtp9iUr8vuN+jt48j+6\"\n },\n \"Datastore\": {\n \"Type\": \"leveldb\",\n \"Path\": \"/root/.ipfs/datastore\"\n },\n \"Addresses\": {\n \"Swarm\": [\n \"/ip4/0.0.0.0/tcp/4011\"\n ],\n \"API\": \"/ip4/127.0.0.1/tcp/5001\"\n },\n \"Mounts\": {\n \"IPFS\": \"/ipfs\",\n \"IPNS\": \"/ipns\",\n \"MFS\": \"/mfs\"\n },\n \"Version\": {\n \"Current\": \"0.1.7\",\n \"Check\": \"error\",\n \"CheckDate\": \"0001-01-01T00:00:00Z\",\n \"CheckPeriod\": \"172800000000000\",\n \"AutoUpdate\": \"minor\"\n },\n \"Bootstrap\": [\n ]\n}\n" }, { "path": "test/3nodetest/build/.gitignore", "content": ".built_img\n*.log\nprofiling_data*\n" }, { "path": "test/3nodetest/build/.gitkeep", "content": "" }, { "path": "test/3nodetest/client/Dockerfile", "content": "FROM zaqwsx_ipfs-test-img\n\nRUN ipfs init -b=2048\nADD . /tmp/id\nRUN mv -f /tmp/id/config /root/.ipfs/config\nRUN ipfs id\n\nEXPOSE 4031 4032/udp\n\nENV IPFS_PROF true\nENV GOLOG_LOG_FMT nocolor\n\nENTRYPOINT [\"/bin/bash\"]\nCMD [\"/tmp/id/run.sh\"]\n" }, { "path": "test/3nodetest/client/config", "content": "{\n \"Addresses\": {\n \"API\": \"/ip4/127.0.0.1/tcp/5001\",\n \"Swarm\": [\n \"/ip4/0.0.0.0/tcp/4031\"\n ]\n },\n \"Bootstrap\": [\n ],\n \"Datastore\": {\n \"Path\": \"/root/.ipfs/datastore\",\n \"Type\": \"leveldb\"\n },\n \"Identity\": {\n \"PeerID\": \"Qmbtc35vdjVh5o9w2AaT2SgcWwigsaoZUpRfq2FSrFD6mb\",\n \"PrivKey\": \"CAAS4AQwggJcAgEAAoGBANlJUjOCbPXgYUfo1Pr6nlIjJDPNwN81ACamhaoEZ9VRHXI3fPe7RVAaaXrWLHb892mRqFi1ScE2lcMTLc7WGfyc7dwPqBOZqkVvT0KpCx3Mg246+WvnG8I3HCbWyjSP9tJflOBQxVq6qT2yZSXjNTtDdO4skd4PsPqBco53guYTAgMBAAECgYEAtIcYhrdMNBSSfp5RpZxnwbJ0t52xK0HruDEOSK2UX0Ufg+/aIjEza1QmYupi0xFltg5QojMs7hyd3Q+oNXro5tKsYVeiqrLsUh9jMjaQofzSlV9Oc+bhkkl48YWvF6Y8qx88UYAX+oJqB627H4S1gxLdNEJhPjEAD6n/jql3zUECQQDmHP75wJ7nC4TlxT1SHim5syMAqWNs/SOHnvX8yLrFV9FrMRzsD5qMlIEGBrAjaESzEck6XpbqkyxB8KKGo7OjAkEA8brtEh/AMoQ/yoSWdYT2MRbJxCAn+KG2c6Hi9AMMmJ+K779HxywpUIDYIa22hzLKYumYIuRa1X++1glOAFGq0QJAPQgXwFoMSy9M8jwcBXmmi3AtqnFCw5doIwJQL9l1X/3ot0txZlLFJOAGUHjZoqp2/h+LhYWs9U5PgLW4BYnJjQJAPydY/J0y93+5ss1FCdr8/wI3IHhOORT2t+sZgiqxxcYY5F4TAKQ2/wNKdDIQN+47FfB1gNgsKw8+6mhv6oFroQJACBF2yssNVXiXa2Na/a9tKYutGvxbm3lXzOvmpkW3FukbsObKYS344J1vdg0nzM6EWQCaiBweSA5TQ27iNW6BzQ==\"\n },\n \"Mounts\": {\n \"IPFS\": \"/ipfs\",\n \"IPNS\": \"/ipns\",\n \"MFS\": \"/mfs\"\n },\n \"Version\": {\n \"AutoUpdate\": \"minor\",\n \"Check\": \"error\",\n \"CheckDate\": \"0001-01-01T00:00:00Z\",\n \"CheckPeriod\": \"172800000000000\",\n \"Current\": \"0.1.7\"\n }\n}\n" }, { "path": "test/3nodetest/client/run.sh", "content": "ipfs bootstrap add /ip4/$BOOTSTRAP_PORT_4011_TCP_ADDR/tcp/$BOOTSTRAP_PORT_4011_TCP_PORT/p2p/QmNXuBh8HFsWq68Fid8dMbGNQTh7eG6hV9rr1fQyfmfomE\nipfs bootstrap # list bootstrap nodes for debugging\n\necho \"3nodetest> starting client daemon\"\n\nipfs daemon --debug &\nsleep 3\n\n# switch dirs so ipfs client profiling data doesn't overwrite the ipfs daemon\n# profiling data\ncd /tmp\n\nwhile [ ! -f /data/idtiny ]\ndo\n echo \"3nodetest> waiting for server to add the file...\"\n sleep 1\ndone\necho \"3nodetest> client found file with hash:\" $(cat /data/idtiny)\n\nipfs cat $(cat /data/idtiny) > filetiny\n\ncat filetiny\n\ndiff -u filetiny /data/filetiny\n\nif (($? > 0)); then\n printf '%s\\n' 'files did not match' >&2\n exit 1\nfi\n\nwhile [ ! -f /data/idrand ]\ndo\n echo \"3nodetest> waiting for server to add the file...\"\n sleep 1\ndone\necho \"3nodetest> client found file with hash:\" $(cat /data/idrand)\n\ncat /data/idrand\n\nipfs cat $(cat /data/idrand) > filerand\n\nif (($? > 0)); then\n printf '%s\\n' 'ipfs cat failed' >&2\n exit 1\nfi\n\ndiff -u filerand /data/filerand\n\nif (($? > 0)); then\n printf '%s\\n' 'files did not match' >&2\n exit 1\nfi\n\necho \"3nodetest> success\"\n" }, { "path": "test/3nodetest/data/.gitignore", "content": "file*\n" }, { "path": "test/3nodetest/data/Dockerfile", "content": "FROM ubuntu\n\nADD filetiny /data/filetiny\nADD filerand /data/filerand\n\nVOLUME [\"/data\"]\n" }, { "path": "test/3nodetest/fig.yml", "content": "data:\n build: ./data\n volumes:\n - /data\n command: sleep 1000000\n\nbootstrap:\n build: ./bootstrap\n command: daemon --debug --init\n expose:\n - \"4011\"\n - \"4012/udp\"\n environment:\n GOLOG_LOG_LEVEL: debug\n\nserver:\n build: ./server\n links:\n - bootstrap\n volumes_from:\n - data\n expose:\n - \"4021\"\n - \"4022/udp\"\n environment:\n GOLOG_LOG_LEVEL: debug\n\nclient:\n build: ./client\n links:\n - bootstrap\n volumes_from:\n - data\n expose:\n - \"4031\"\n - \"4032/udp\"\n environment:\n GOLOG_LOG_LEVEL: debug\n" }, { "path": "test/3nodetest/run-test-on-img.sh", "content": "#!/bin/sh\nif [ \"$#\" -ne 1 ]; then\n echo \"usage: $0 \"\n echo \"runs this test on image matching \"\n exit 1\nfi\n\n# this tag is used by the dockerfiles in\n# {data, server, client, bootstrap}\ntag=zaqwsx_ipfs-test-img\n\n# could use set -v, but i don't want to see the comments...\n\nimg=$(docker images | grep $1 | awk '{print $3}')\necho \"using docker image: $img ($1)\"\n\necho docker tag -f $img $tag\ndocker tag -f $img $tag\n\necho \"fig build --no-cache\"\nfig build --no-cache\n\necho \"fig up --no-color | tee build/fig.log\"\nfig up --no-color | tee build/fig.log\n\n# save the ipfs logs for inspection\necho \"make save_logs\"\nmake save_logs || true # don't fail\n\n# save the ipfs logs for inspection\necho \"make save_profiling_data\"\nmake save_profiling_data || true # don't fail\n\n# fig up won't report the error using an error code, so we grep the\n# fig.log file to find out whether the call succeeded\necho 'tail build/fig.log | grep \"exited with code 0\"'\ntail build/fig.log | grep \"exited with code 0\"\n" }, { "path": "test/3nodetest/server/Dockerfile", "content": "FROM zaqwsx_ipfs-test-img\n\nRUN ipfs init -b=2048\nADD . /tmp/test\nRUN mv -f /tmp/test/config /root/.ipfs/config\nRUN ipfs id\nRUN chmod +x /tmp/test/run.sh\n\nEXPOSE 4021 4022/udp\n\nENV IPFS_PROF true\nENV GOLOG_LOG_FMT nocolor\n\nENTRYPOINT [\"/bin/bash\"]\nCMD [\"/tmp/test/run.sh\"]\n" }, { "path": "test/3nodetest/server/README.md", "content": "**requirements**\n\n* docker container with bootstrap node linked as \"bootstrap\" with ID QmNXuBh8HFsWq68Fid8dMbGNQTh7eG6hV9rr1fQyfmfomE\n* file in data volume, internally mapped to /data/file\n" }, { "path": "test/3nodetest/server/config", "content": "{\n \"Addresses\": {\n \"API\": \"/ip4/127.0.0.1/tcp/5001\",\n \"Swarm\": [\n \"/ip4/0.0.0.0/tcp/4021\"\n ]\n },\n \"Bootstrap\": [\n ],\n \"Datastore\": {\n \"Path\": \"/root/.ipfs/datastore\",\n \"Type\": \"leveldb\"\n },\n \"Identity\": {\n \"PeerID\": \"Qmbtc2C7rqmAfdeMTM7FX4YF8CeBumMCfk5Z1GBCMbMTfY\",\n \"PrivKey\": \"CAAS4AQwggJcAgEAAoGBANW3mJMmDSJbdRyykO0Ze5t6WL6jeTtpOhklxePBIkJL/Uil78Va/tODx6Mvv3GMCkbGvzWslTZXpaHa9vBmjE3MVZSmd5fLRybKT0zZ3juABKcx+WIVNw8JlkpEORihJdwb+5tRUC5pUcMzxqHSmGX+d6e9KZqLnv7piNKg2+r7AgMBAAECgYAqc6+w+wv82SHoM2gqULeG6MScCajZLkvGFwS5+vEtLh7/wUZhc3PO3AxZ0/A5Q9H+wRfWN5PkGYDjJ7WJhzUzGfTbrQ821JV6B3IUR4UHo2IgJkZO4EUB5L9KBUqvYxDJigtGBopgQh0EeDSS+9X8vaGmit5l4zcAfi+UGYPgMQJBAOCJQU8N2HW5SawBo2QX0bnCAAnu5Ilk2QaqwDZbDQaM5JWFcpRpGnjBhsZihHwVWvKCbnq83JhAGRQvKAEepMUCQQDzqjvIyM+Au42nP7SFDHoMjEnHW8Nimvz8zPbyrSUEHe4l9/yS4+BeRPxpwI5xgzp8g1wEYfNeXt08buYwCsy/AkBXWg5mSuSjJ+pZWGnQTtPwiGCrfJy8NteXmGYev11Z5wYmhTwGML1zrRZZp4oTG9u97LA+X6sSMB2RlKbjiKBhAkEAgl/hoSshK+YugwCpHE9ytmgRyeOlhYscNj+NGofeOHezRwmLUSUwlgAfdo4bKU1n69t1TrsCNspXYdCMxcPhjQJAMNxkJ8t2tFMpucCQfWJ09wvFKZSHX1/iD9GKWL0Qk2FcMCg3NXiqei5NL3NYqCWpdC/IfjsAEGCJrTFwp/OoUw==\"\n },\n \"Mounts\": {\n \"IPFS\": \"/ipfs\",\n \"IPNS\": \"/ipns\",\n \"MFS\": \"/mfs\"\n },\n \"Version\": {\n \"AutoUpdate\": \"minor\",\n \"Check\": \"error\",\n \"CheckDate\": \"0001-01-01T00:00:00Z\",\n \"CheckPeriod\": \"172800000000000\",\n \"Current\": \"0.1.7\"\n }\n}\n" }, { "path": "test/3nodetest/server/run.sh", "content": "# must be connected to bootstrap node\nipfs bootstrap add /ip4/$BOOTSTRAP_PORT_4011_TCP_ADDR/tcp/$BOOTSTRAP_PORT_4011_TCP_PORT/p2p/QmNXuBh8HFsWq68Fid8dMbGNQTh7eG6hV9rr1fQyfmfomE\nipfs bootstrap # list bootstrap nodes for debugging\n\n# wait for daemon to start/bootstrap\n# alternatively use ipfs swarm connect\necho \"3nodetest> starting server daemon\"\n\n# run daemon in debug mode to collect profiling data\nipfs daemon --debug &\nsleep 3\n# TODO instead of bootstrapping: ipfs swarm connect /ip4/$BOOTSTRAP_PORT_4011_TCP_ADDR/tcp/$BOOTSTRAP_PORT_4011_TCP_PORT/p2p/QmNXuBh8HFsWq68Fid8dMbGNQTh7eG6hV9rr1fQyfmfomE\n\n# change dir before running add commands so ipfs client profiling data doesn't\n# overwrite the daemon profiling data\ncd /tmp\n\n# must mount this volume from data container\nipfs add -q /data/filetiny > tmptiny\nmv tmptiny /data/idtiny\necho \"3nodetest> added tiny file. hash is\" $(cat /data/idtiny)\n\nipfs add -q /data/filerand > tmprand\nmv tmprand /data/idrand\necho \"3nodetest> added rand file. hash is\" $(cat /data/idrand)\n\n# allow ample time for the client to pull the data\nsleep 10000000\n" }, { "path": "test/README.md", "content": "\n## Sharness test command coverage\n\nModule | Online Test | Offline Test |\n-----------|-----------------|-----------------|\nobject | t0051 | t0051\nls | t0045 | t0045\ncat | t0040 |\ndht | |\nbitswap | |\nblock | | t0050\ndaemon | t0030 | N/A\ninit | N/A | t0020\nadd | t0040 |\nconfig | t0021 | t0021\nversion | t0060 | t0010\nping | |\ndiag | |\nmount | t0030 |\nname | t0110 | t0100\npin | t0080 |\nget | t0090 | t0090\nrefs | t0080 |\nrepo gc | t0080 |\nid | |\nbootstrap | t0120 | t0120\nswarm | |\nupdate | |\ncommands | |\n" }, { "path": "test/Rules.mk", "content": "include mk/header.mk\n\ndir := $(d)/bin\ninclude $(dir)/Rules.mk\n\ndir := $(d)/sharness\ninclude $(dir)/Rules.mk\n\ndir := $(d)/unit\ninclude $(dir)/Rules.mk\n\ninclude mk/footer.mk\n" }, { "path": "test/api-startup/main.go", "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"log\"\n\t\"net/http\"\n\t\"sync\"\n\t\"time\"\n)\n\nfunc main() {\n\twhen := make(chan time.Time, 2)\n\tvar wg sync.WaitGroup\n\twg.Add(2)\n\tfor _, port := range []string{\"5001\", \"8080\"} {\n\t\tgo func(port string) {\n\t\t\tdefer wg.Done()\n\t\t\tfor {\n\t\t\t\tr, err := http.Get(fmt.Sprintf(\"http://127.0.0.1:%s\", port))\n\t\t\t\tif err != nil {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tt := time.Now()\n\t\t\t\twhen <- t\n\t\t\t\tlog.Println(port, t, r.StatusCode)\n\t\t\t\tbreak\n\t\t\t}\n\t\t}(port)\n\t}\n\twg.Wait()\n\tfirst := <-when\n\tsecond := <-when\n\tlog.Println(second.Sub(first))\n}\n" }, { "path": "test/bench/bench_cli_ipfs_add/main.go", "content": "package main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/thirdparty/unit\"\n\n\trandom \"github.com/ipfs/go-test/random\"\n\tconfig \"github.com/ipfs/kubo/config\"\n)\n\nvar (\n\tdebug = flag.Bool(\"debug\", false, \"direct ipfs output to console\")\n\tonline = flag.Bool(\"online\", false, \"run the benchmarks with a running daemon\")\n)\n\nfunc main() {\n\tflag.Parse()\n\tif err := compareResults(); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc compareResults() error {\n\tvar amount unit.Information\n\tfor amount = 10 * unit.MB; amount > 0; amount = amount * 2 {\n\t\tif results, err := benchmarkAdd(int64(amount)); err != nil { // TODO compare\n\t\t\treturn err\n\t\t} else {\n\t\t\tlog.Println(amount, \"\\t\", results)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc benchmarkAdd(amount int64) (*testing.BenchmarkResult, error) {\n\tvar benchmarkError error\n\tresults := testing.Benchmark(func(b *testing.B) {\n\t\tb.SetBytes(amount)\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tb.StopTimer()\n\t\t\ttmpDir := b.TempDir()\n\n\t\t\tenv := append(\n\t\t\t\t[]string{fmt.Sprintf(\"%s=%s\", config.EnvDir, path.Join(tmpDir, config.DefaultPathName))}, // first in order to override\n\t\t\t\tos.Environ()...,\n\t\t\t)\n\t\t\tsetupCmd := func(cmd *exec.Cmd) {\n\t\t\t\tcmd.Env = env\n\t\t\t\tif *debug {\n\t\t\t\t\tcmd.Stdout = os.Stdout\n\t\t\t\t\tcmd.Stderr = os.Stderr\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tinitCmd := exec.Command(\"ipfs\", \"init\")\n\t\t\tsetupCmd(initCmd)\n\t\t\tif err := initCmd.Run(); err != nil {\n\t\t\t\tbenchmarkError = err\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\n\t\t\tconst seed = 1\n\t\t\tf, err := os.CreateTemp(\"\", \"\")\n\t\t\tif err != nil {\n\t\t\t\tbenchmarkError = err\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\t\t\tdefer os.Remove(f.Name())\n\n\t\t\trandReader := &io.LimitedReader{\n\t\t\t\tR: random.NewSeededRand(seed),\n\t\t\t\tN: amount,\n\t\t\t}\n\t\t\tif _, err := io.Copy(f, randReader); err != nil {\n\t\t\t\tbenchmarkError = err\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\t\t\tif err := f.Close(); err != nil {\n\t\t\t\tbenchmarkError = err\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\n\t\t\tfunc() {\n\t\t\t\t// FIXME online mode isn't working. client complains that it cannot open leveldb\n\t\t\t\tif *online {\n\t\t\t\t\tdaemonCmd := exec.Command(\"ipfs\", \"daemon\")\n\t\t\t\t\tsetupCmd(daemonCmd)\n\t\t\t\t\tif err := daemonCmd.Start(); err != nil {\n\t\t\t\t\t\tbenchmarkError = err\n\t\t\t\t\t\tb.Fatal(err)\n\t\t\t\t\t}\n\t\t\t\t\tdefer func() {\n\t\t\t\t\t\t_ = daemonCmd.Process.Signal(os.Interrupt)\n\t\t\t\t\t\t_ = daemonCmd.Wait()\n\t\t\t\t\t}()\n\t\t\t\t}\n\n\t\t\t\tb.StartTimer()\n\t\t\t\taddCmd := exec.Command(\"ipfs\", \"add\", f.Name())\n\t\t\t\tsetupCmd(addCmd)\n\t\t\t\tif err := addCmd.Run(); err != nil {\n\t\t\t\t\tbenchmarkError = err\n\t\t\t\t\tb.Fatal(err)\n\t\t\t\t}\n\t\t\t\tb.StopTimer()\n\t\t\t}()\n\t\t}\n\t})\n\tif benchmarkError != nil {\n\t\treturn nil, benchmarkError\n\t}\n\treturn &results, nil\n}\n" }, { "path": "test/bench/offline_add/main.go", "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/thirdparty/unit\"\n\n\trandom \"github.com/ipfs/go-test/random\"\n\tconfig \"github.com/ipfs/kubo/config\"\n)\n\nfunc main() {\n\tif err := compareResults(); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc compareResults() error {\n\tvar amount unit.Information\n\tfor amount = 10 * unit.MB; amount > 0; amount = amount * 2 {\n\t\tif results, err := benchmarkAdd(int64(amount)); err != nil { // TODO compare\n\t\t\treturn err\n\t\t} else {\n\t\t\tlog.Println(amount, \"\\t\", results)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc benchmarkAdd(amount int64) (*testing.BenchmarkResult, error) {\n\tresults := testing.Benchmark(func(b *testing.B) {\n\t\tb.SetBytes(amount)\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tb.StopTimer()\n\t\t\ttmpDir := b.TempDir()\n\n\t\t\tenv := append(os.Environ(), fmt.Sprintf(\"%s=%s\", config.EnvDir, path.Join(tmpDir, config.DefaultPathName)))\n\t\t\tsetupCmd := func(cmd *exec.Cmd) {\n\t\t\t\tcmd.Env = env\n\t\t\t}\n\n\t\t\tcmd := exec.Command(\"ipfs\", \"init\")\n\t\t\tsetupCmd(cmd)\n\t\t\tif err := cmd.Run(); err != nil {\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\n\t\t\tconst seed = 1\n\t\t\tf, err := os.CreateTemp(\"\", \"\")\n\t\t\tif err != nil {\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\t\t\tdefer os.Remove(f.Name())\n\n\t\t\trandReader := &io.LimitedReader{\n\t\t\t\tR: random.NewSeededRand(seed),\n\t\t\t\tN: amount,\n\t\t\t}\n\t\t\t_, err = io.Copy(f, randReader)\n\t\t\tif err != nil {\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\t\t\tif err := f.Close(); err != nil {\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\n\t\t\tb.StartTimer()\n\t\t\tcmd = exec.Command(\"ipfs\", \"add\", f.Name())\n\t\t\tsetupCmd(cmd)\n\t\t\tif err := cmd.Run(); err != nil {\n\t\t\t\tb.Fatal(err)\n\t\t\t}\n\t\t\tb.StopTimer()\n\t\t}\n\t})\n\treturn &results, nil\n}\n" }, { "path": "test/bin/.gitignore", "content": "# Ignore everything in this directory by default\n/**\n\n# Do not ignore this file itself\n!.gitignore\n\n# Do not ignore the following special scripts\n!checkflags\n!continueyn\n!verify-go-fmt.sh\n!Rules.mk\n" }, { "path": "test/bin/Rules.mk", "content": "include mk/header.mk\n\nTGTS_$(d) :=\n\ndefine go-build-testdep\n\tOUT=\"$(CURDIR)/$@\" ; \\\n\tcd \"test/dependencies\" ; \\\n\t$(GOCC) build $(go-flags-with-tags) -o \"$${OUT}\" \"$<\" 2>&1\nendef\n\n.PHONY: github.com/ipfs/kubo/test/dependencies/pollEndpoint\n$(d)/pollEndpoint: github.com/ipfs/kubo/test/dependencies/pollEndpoint\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/pollEndpoint\n\n.PHONY: github.com/ipfs/kubo/test/dependencies/go-sleep\n$(d)/go-sleep: github.com/ipfs/kubo/test/dependencies/go-sleep\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/go-sleep\n\n.PHONY: github.com/ipfs/kubo/test/dependencies/go-timeout\n$(d)/go-timeout: github.com/ipfs/kubo/test/dependencies/go-timeout\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/go-timeout\n\n.PHONY: github.com/ipfs/kubo/test/dependencies/iptb\n$(d)/iptb: github.com/ipfs/kubo/test/dependencies/iptb\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/iptb\n\n.PHONY: github.com/ipfs/kubo/test/dependencies/ma-pipe-unidir\n$(d)/ma-pipe-unidir: github.com/ipfs/kubo/test/dependencies/ma-pipe-unidir\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/ma-pipe-unidir\n\n.PHONY: github.com/ipfs/kubo/test/dependencies/json-to-junit\n$(d)/json-to-junit: github.com/ipfs/kubo/test/dependencies/json-to-junit\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/json-to-junit\n\n.PHONY: gotest.tools/gotestsum\n$(d)/gotestsum: gotest.tools/gotestsum\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/gotestsum\n\n.PHONY: github.com/ipfs/hang-fds\n$(d)/hang-fds: github.com/ipfs/hang-fds\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/hang-fds\n\n.PHONY: github.com/multiformats/go-multihash/multihash\n$(d)/multihash: github.com/multiformats/go-multihash/multihash\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/multihash\n\n.PHONY: github.com/ipfs/go-cidutil/cid-fmt\n$(d)/cid-fmt: github.com/ipfs/go-cidutil/cid-fmt\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/cid-fmt\n\n.PHONY: github.com/ipfs/go-test/cli/random-data\n$(d)/random-data: github.com/ipfs/go-test/cli/random-data\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/random-data\n\n.PHONY: github.com/ipfs/go-test/cli/random-files\n$(d)/random-files: github.com/ipfs/go-test/cli/random-files\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/random-files\n\n.PHONY: github.com/Kubuxu/gocovmerge\n$(d)/gocovmerge: github.com/Kubuxu/gocovmerge\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/gocovmerge\n\n.PHONY: github.com/golangci/golangci-lint/cmd/golangci-lint\n$(d)/golangci-lint: github.com/golangci/golangci-lint/cmd/golangci-lint\n\t$(go-build-testdep)\nTGTS_$(d) += $(d)/golangci-lint\n\n$(TGTS_$(d)): $$(DEPS_GO)\n\nCLEAN += $(TGTS_$(d))\n\nPATH := $(realpath $(d)):$(PATH)\n\ninclude mk/footer.mk\n" }, { "path": "test/bin/checkflags", "content": "#!/bin/sh\n# Author: Christian Couder \n# MIT LICENSED\n\nif test \"$#\" -lt 3\nthen\n echo >&2 \"usage $0 FILE VALUES MSG...\"\n exit 1\nfi\n\nFLAG_FILE=\"$1\"\nFLAG_VALS=\"$2\"\nshift\nshift\nFLAG_MSGS=\"$@\"\n\ntest -f $FLAG_FILE || touch $FLAG_FILE\n\n# Use x in front of tested values as flags could be\n# interpreted by \"test\" to be for itself.\nif test x\"$FLAG_VALS\" != x\"$(cat \"$FLAG_FILE\")\"\nthen\n echo \"$FLAG_MSGS\"\n echo \"$FLAG_VALS\" >\"$FLAG_FILE\"\nfi\n" }, { "path": "test/bin/continueyn", "content": "#!/bin/sh\n# Author: Juan Batiz-Benet \n# MIT LICENSED\n\n# if not a terminal, exit 0 (yes!)\ntest -t 1 || exit 0\n\nread -p \"continue? [y/N] \" REPLY\necho\ncase \"$REPLY\" in\n [Yy]*) exit 0 ;;\n *) exit 1 ;;\nesac\n" }, { "path": "test/cli/add_test.go", "content": "package cli\n\nimport (\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// waitForLogMessage polls a buffer for a log message, waiting up to timeout duration.\n// Returns true if message found, false if timeout reached.\nfunc waitForLogMessage(buffer *harness.Buffer, message string, timeout time.Duration) bool {\n\tdeadline := time.Now().Add(timeout)\n\tfor time.Now().Before(deadline) {\n\t\tif strings.Contains(buffer.String(), message) {\n\t\t\treturn true\n\t\t}\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\treturn false\n}\n\nfunc TestAdd(t *testing.T) {\n\tt.Parallel()\n\n\tvar (\n\t\tshortString = \"hello world\"\n\t\tshortStringCidV0 = \"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD\" // cidv0 - dag-pb - sha2-256\n\t\tshortStringCidV1 = \"bafkreifzjut3te2nhyekklss27nh3k72ysco7y32koao5eei66wof36n5e\" // cidv1 - raw - sha2-256\n\t\tshortStringCidV1NoRawLeaves = \"bafybeihykld7uyxzogax6vgyvag42y7464eywpf55gxi5qpoisibh3c5wa\" // cidv1 - dag-pb - sha2-256\n\t\tshortStringCidV1Sha512 = \"bafkrgqbqt3gerhas23vuzrapkdeqf4vu2dwxp3srdj6hvg6nhsug2tgyn6mj3u23yx7utftq3i2ckw2fwdh5qmhid5qf3t35yvkc5e5ottlw6\"\n\t)\n\n\tt.Run(\"produced cid version: implicit default (CIDv0)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\t})\n\n\tt.Run(\"produced cid version: follows user-set configuration Import.CidVersion=0\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(0)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\t})\n\n\tt.Run(\"produced cid multihash: follows user-set configuration in Import.HashFunction\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.HashFunction = *config.NewOptionalString(\"sha2-512\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV1Sha512, cidStr)\n\t})\n\n\tt.Run(\"produced cid version: follows user-set configuration Import.CidVersion=1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV1, cidStr)\n\t})\n\n\tt.Run(\"produced cid version: command flag overrides configuration in Import.CidVersion\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString, \"--cid-version\", \"0\")\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\t})\n\n\tt.Run(\"produced unixfs raw leaves: follows user-set configuration Import.UnixFSRawLeaves\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// CIDv1 defaults to raw-leaves=true\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\t// disable manually\n\t\t\tcfg.Import.UnixFSRawLeaves = config.False\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV1NoRawLeaves, cidStr)\n\t})\n\n\tt.Run(\"ipfs add --pin-name=foo\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tpinName := \"test-pin-name\"\n\t\tcidStr := node.IPFSAddStr(shortString, \"--pin-name\", pinName)\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\n\t\tpinList := node.IPFS(\"pin\", \"ls\", \"--names\").Stdout.Trimmed()\n\t\trequire.Contains(t, pinList, shortStringCidV0)\n\t\trequire.Contains(t, pinList, pinName)\n\t})\n\n\tt.Run(\"ipfs add --pin=false --pin-name=foo returns an error\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Use RunIPFS to allow for errors without assertion\n\t\tresult := node.RunIPFS(\"add\", \"--pin=false\", \"--pin-name=foo\")\n\t\trequire.Error(t, result.Err, \"Expected an error due to incompatible --pin and --pin-name\")\n\t\trequire.Contains(t, result.Stderr.String(), \"pin-name option requires pin to be set\")\n\t})\n\n\tt.Run(\"ipfs add --pin-name without value should fail\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// When --pin-name is passed without any value, it should fail\n\t\tresult := node.RunIPFS(\"add\", \"--pin-name\")\n\t\trequire.Error(t, result.Err, \"Expected an error when --pin-name has no value\")\n\t\trequire.Contains(t, result.Stderr.String(), \"missing argument for option \\\"pin-name\\\"\")\n\t})\n\n\tt.Run(\"produced unixfs max file links: command flag --max-file-links overrides configuration in Import.UnixFSFileMaxLinks\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t//\n\t\t// UnixFSChunker=size-262144 (256KiB)\n\t\t// Import.UnixFSFileMaxLinks=174\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=unixfs-v0-2015\") // unixfs-v0-2015 for determinism across all params\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.UnixFSChunker = *config.NewOptionalString(\"size-262144\") // 256 KiB chunks\n\t\t\tcfg.Import.UnixFSFileMaxLinks = *config.NewOptionalInteger(174) // max 174 per level\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Add 174MiB file:\n\t\t// 1024 * 256KiB should fit in single layer\n\t\tseed := shortString\n\t\tcidStr := node.IPFSAddDeterministic(\"262144KiB\", seed, \"--max-file-links\", \"1024\")\n\t\troot, err := node.InspectPBNode(cidStr)\n\t\tassert.NoError(t, err)\n\n\t\t// Expect 1024 links due to cli parameter raising link limit from 174 to 1024\n\t\trequire.Equal(t, 1024, len(root.Links))\n\t\t// expect same CID every time\n\t\trequire.Equal(t, \"QmbBftNHWmjSWKLC49dMVrfnY8pjrJYntiAXirFJ7oJrNk\", cidStr)\n\t})\n\n\t// Profile-specific threshold tests are in cid_profiles_test.go (TestCIDProfiles).\n\t// Tests here cover general ipfs add behavior not tied to specific profiles.\n\n\tt.Run(\"ipfs add --hidden\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Helper to create test directory with hidden file\n\t\tsetupTestDir := func(t *testing.T, node *harness.Node) string {\n\t\t\ttestDir, err := os.MkdirTemp(node.Dir, \"hidden-test\")\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.NoError(t, os.WriteFile(filepath.Join(testDir, \"visible.txt\"), []byte(\"visible\"), 0o644))\n\t\t\trequire.NoError(t, os.WriteFile(filepath.Join(testDir, \".hidden\"), []byte(\"hidden\"), 0o644))\n\t\t\treturn testDir\n\t\t}\n\n\t\tt.Run(\"default excludes hidden files\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", testDir).Stdout.Trimmed()\n\t\t\tlsOutput := node.IPFS(\"ls\", cidStr).Stdout.Trimmed()\n\t\t\trequire.Contains(t, lsOutput, \"visible.txt\")\n\t\t\trequire.NotContains(t, lsOutput, \".hidden\")\n\t\t})\n\n\t\tt.Run(\"--hidden includes hidden files\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", \"--hidden\", testDir).Stdout.Trimmed()\n\t\t\tlsOutput := node.IPFS(\"ls\", cidStr).Stdout.Trimmed()\n\t\t\trequire.Contains(t, lsOutput, \"visible.txt\")\n\t\t\trequire.Contains(t, lsOutput, \".hidden\")\n\t\t})\n\n\t\tt.Run(\"-H includes hidden files\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", \"-H\", testDir).Stdout.Trimmed()\n\t\t\tlsOutput := node.IPFS(\"ls\", cidStr).Stdout.Trimmed()\n\t\t\trequire.Contains(t, lsOutput, \"visible.txt\")\n\t\t\trequire.Contains(t, lsOutput, \".hidden\")\n\t\t})\n\t})\n\n\tt.Run(\"ipfs add --empty-dirs\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Helper to create test directory with empty subdirectory\n\t\tsetupTestDir := func(t *testing.T, node *harness.Node) string {\n\t\t\ttestDir, err := os.MkdirTemp(node.Dir, \"empty-dirs-test\")\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.NoError(t, os.Mkdir(filepath.Join(testDir, \"empty-subdir\"), 0o755))\n\t\t\trequire.NoError(t, os.WriteFile(filepath.Join(testDir, \"file.txt\"), []byte(\"content\"), 0o644))\n\t\t\treturn testDir\n\t\t}\n\n\t\tt.Run(\"default includes empty directories\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", testDir).Stdout.Trimmed()\n\t\t\trequire.Contains(t, node.IPFS(\"ls\", cidStr).Stdout.Trimmed(), \"empty-subdir\")\n\t\t})\n\n\t\tt.Run(\"--empty-dirs=true includes empty directories\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", \"--empty-dirs=true\", testDir).Stdout.Trimmed()\n\t\t\trequire.Contains(t, node.IPFS(\"ls\", cidStr).Stdout.Trimmed(), \"empty-subdir\")\n\t\t})\n\n\t\tt.Run(\"--empty-dirs=false excludes empty directories\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", \"--empty-dirs=false\", testDir).Stdout.Trimmed()\n\t\t\tlsOutput := node.IPFS(\"ls\", cidStr).Stdout.Trimmed()\n\t\t\trequire.NotContains(t, lsOutput, \"empty-subdir\")\n\t\t\trequire.Contains(t, lsOutput, \"file.txt\")\n\t\t})\n\t})\n\n\tt.Run(\"ipfs add symlink handling\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Helper to create test directory structure:\n\t\t// testDir/\n\t\t// target.txt (file with \"target content\")\n\t\t// link.txt -> target.txt (symlink at top level)\n\t\t// subdir/\n\t\t// subsubdir/\n\t\t// nested-target.txt (file with \"nested content\")\n\t\t// nested-link.txt -> nested-target.txt (symlink in sub-sub directory)\n\t\tsetupTestDir := func(t *testing.T, node *harness.Node) string {\n\t\t\ttestDir, err := os.MkdirTemp(node.Dir, \"deref-symlinks-test\")\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Top-level file and symlink\n\t\t\ttargetFile := filepath.Join(testDir, \"target.txt\")\n\t\t\trequire.NoError(t, os.WriteFile(targetFile, []byte(\"target content\"), 0o644))\n\t\t\trequire.NoError(t, os.Symlink(\"target.txt\", filepath.Join(testDir, \"link.txt\")))\n\n\t\t\t// Nested file and symlink in sub-sub directory\n\t\t\tsubsubdir := filepath.Join(testDir, \"subdir\", \"subsubdir\")\n\t\t\trequire.NoError(t, os.MkdirAll(subsubdir, 0o755))\n\t\t\tnestedTarget := filepath.Join(subsubdir, \"nested-target.txt\")\n\t\t\trequire.NoError(t, os.WriteFile(nestedTarget, []byte(\"nested content\"), 0o644))\n\t\t\trequire.NoError(t, os.Symlink(\"nested-target.txt\", filepath.Join(subsubdir, \"nested-link.txt\")))\n\n\t\t\treturn testDir\n\t\t}\n\n\t\tt.Run(\"default preserves symlinks\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\n\t\t\t// Add directory with symlink (default: preserve)\n\t\t\tdirCID := node.IPFS(\"add\", \"-r\", \"-Q\", testDir).Stdout.Trimmed()\n\n\t\t\t// Get and verify symlinks are preserved\n\t\t\toutDir, err := os.MkdirTemp(node.Dir, \"symlink-get-out\")\n\t\t\trequire.NoError(t, err)\n\t\t\tnode.IPFS(\"get\", \"-o\", outDir, dirCID)\n\n\t\t\t// Check top-level symlink is preserved\n\t\t\tlinkPath := filepath.Join(outDir, \"link.txt\")\n\t\t\tfi, err := os.Lstat(linkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.True(t, fi.Mode()&os.ModeSymlink != 0, \"link.txt should be a symlink\")\n\t\t\ttarget, err := os.Readlink(linkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, \"target.txt\", target)\n\n\t\t\t// Check nested symlink is preserved\n\t\t\tnestedLinkPath := filepath.Join(outDir, \"subdir\", \"subsubdir\", \"nested-link.txt\")\n\t\t\tfi, err = os.Lstat(nestedLinkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.True(t, fi.Mode()&os.ModeSymlink != 0, \"nested-link.txt should be a symlink\")\n\t\t})\n\n\t\t// --dereference-args is deprecated but still works for backwards compatibility.\n\t\t// It only resolves symlinks passed as CLI arguments, NOT symlinks found\n\t\t// during directory traversal. Use --dereference-symlinks instead.\n\t\tt.Run(\"--dereference-args resolves CLI args only\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tsymlinkPath := filepath.Join(testDir, \"link.txt\")\n\t\t\ttargetPath := filepath.Join(testDir, \"target.txt\")\n\n\t\t\tsymlinkCID := node.IPFS(\"add\", \"-Q\", \"--dereference-args\", symlinkPath).Stdout.Trimmed()\n\t\t\ttargetCID := node.IPFS(\"add\", \"-Q\", targetPath).Stdout.Trimmed()\n\n\t\t\t// CIDs should match because --dereference-args resolves the symlink\n\t\t\trequire.Equal(t, targetCID, symlinkCID,\n\t\t\t\t\"--dereference-args should resolve CLI arg symlink to target content\")\n\n\t\t\t// Now add the directory recursively with --dereference-args\n\t\t\t// Nested symlinks should NOT be resolved (only CLI args are resolved)\n\t\t\tdirCID := node.IPFS(\"add\", \"-r\", \"-Q\", \"--dereference-args\", testDir).Stdout.Trimmed()\n\n\t\t\toutDir, err := os.MkdirTemp(node.Dir, \"deref-args-out\")\n\t\t\trequire.NoError(t, err)\n\t\t\tnode.IPFS(\"get\", \"-o\", outDir, dirCID)\n\n\t\t\t// Nested symlink should still be a symlink (not dereferenced)\n\t\t\tnestedLinkPath := filepath.Join(outDir, \"subdir\", \"subsubdir\", \"nested-link.txt\")\n\t\t\tfi, err := os.Lstat(nestedLinkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.True(t, fi.Mode()&os.ModeSymlink != 0,\n\t\t\t\t\"--dereference-args should NOT resolve nested symlinks, only CLI args\")\n\t\t})\n\n\t\t// --dereference-symlinks resolves ALL symlinks: both CLI arguments AND\n\t\t// symlinks found during directory traversal. This is a superset of\n\t\t// the deprecated --dereference-args behavior.\n\t\tt.Run(\"--dereference-symlinks resolves all symlinks\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\ttestDir := setupTestDir(t, node)\n\t\t\tsymlinkPath := filepath.Join(testDir, \"link.txt\")\n\t\t\ttargetPath := filepath.Join(testDir, \"target.txt\")\n\n\t\t\tsymlinkCID := node.IPFS(\"add\", \"-Q\", \"--dereference-symlinks\", symlinkPath).Stdout.Trimmed()\n\t\t\ttargetCID := node.IPFS(\"add\", \"-Q\", targetPath).Stdout.Trimmed()\n\n\t\t\trequire.Equal(t, targetCID, symlinkCID,\n\t\t\t\t\"--dereference-symlinks should resolve CLI arg symlink (like --dereference-args)\")\n\n\t\t\t// Test 2: Nested symlinks in sub-sub directory are ALSO resolved\n\t\t\tdirCID := node.IPFS(\"add\", \"-r\", \"-Q\", \"--dereference-symlinks\", testDir).Stdout.Trimmed()\n\n\t\t\toutDir, err := os.MkdirTemp(node.Dir, \"deref-symlinks-out\")\n\t\t\trequire.NoError(t, err)\n\t\t\tnode.IPFS(\"get\", \"-o\", outDir, dirCID)\n\n\t\t\t// Top-level symlink should be dereferenced to regular file\n\t\t\tlinkPath := filepath.Join(outDir, \"link.txt\")\n\t\t\tfi, err := os.Lstat(linkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.False(t, fi.Mode()&os.ModeSymlink != 0,\n\t\t\t\t\"link.txt should be dereferenced to regular file\")\n\t\t\tcontent, err := os.ReadFile(linkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, \"target content\", string(content))\n\n\t\t\t// Nested symlink in sub-sub directory should ALSO be dereferenced\n\t\t\tnestedLinkPath := filepath.Join(outDir, \"subdir\", \"subsubdir\", \"nested-link.txt\")\n\t\t\tfi, err = os.Lstat(nestedLinkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.False(t, fi.Mode()&os.ModeSymlink != 0,\n\t\t\t\t\"nested-link.txt should be dereferenced (--dereference-symlinks resolves ALL symlinks)\")\n\t\t\tnestedContent, err := os.ReadFile(nestedLinkPath)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, \"nested content\", string(nestedContent))\n\t\t})\n\t})\n}\n\nfunc TestAddFastProvide(t *testing.T) {\n\tt.Parallel()\n\n\tconst (\n\t\tshortString = \"hello world\"\n\t\tshortStringCidV0 = \"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD\" // cidv0 - dag-pb - sha2-256\n\t)\n\n\tt.Run(\"fast-provide-root disabled via config: verify skipped in logs\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.False\n\t\t})\n\n\t\t// Start daemon with debug logging\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\n\t\t// Verify fast-provide-root was disabled\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: skipped\")\n\t})\n\n\tt.Run(\"fast-provide-root enabled with wait=false: verify async provide\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t// Use default config (FastProvideRoot=true, FastProvideWait=false)\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\n\t\tdaemonLog := node.Daemon.Stderr\n\t\t// Should see async mode started\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: enabled\")\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: providing asynchronously\")\n\n\t\t// Wait for async completion or failure (up to 11 seconds - slightly more than fastProvideTimeout)\n\t\t// In test environment with no DHT peers, this will fail with \"failed to find any peer in table\"\n\t\tcompletedOrFailed := waitForLogMessage(daemonLog, \"async provide completed\", 11*time.Second) ||\n\t\t\twaitForLogMessage(daemonLog, \"async provide failed\", 11*time.Second)\n\t\trequire.True(t, completedOrFailed, \"async provide should complete or fail within timeout\")\n\t})\n\n\tt.Run(\"fast-provide-root enabled with wait=true: verify sync provide\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideWait = config.True\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Use Runner.Run with stdin to allow for expected errors\n\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"add\", \"-q\"},\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithStdin(strings.NewReader(shortString)),\n\t\t\t},\n\t\t})\n\n\t\t// In sync mode (wait=true), provide errors propagate and fail the command.\n\t\t// Test environment uses 'test' profile with no bootstrappers, and CI has\n\t\t// insufficient peers for proper DHT puts, so we expect this to fail with\n\t\t// \"failed to find any peer in table\" error from the DHT.\n\t\trequire.Equal(t, 1, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"Error: fast-provide: failed to find any peer in table\")\n\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\t// Should see sync mode started\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: enabled\")\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: providing synchronously\")\n\t\trequire.Contains(t, daemonLog, \"sync provide failed\") // Verify the failure was logged\n\t})\n\n\tt.Run(\"fast-provide-wait ignored when root disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.False\n\t\t\tcfg.Import.FastProvideWait = config.True\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString)\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: skipped\")\n\t\trequire.Contains(t, daemonLog, \"wait-flag-ignored\")\n\t})\n\n\tt.Run(\"CLI flag overrides config: flag=true overrides config=false\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.False\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString, \"--fast-provide-root=true\")\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\n\t\tdaemonLog := node.Daemon.Stderr\n\t\t// Flag should enable it despite config saying false\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: enabled\")\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: providing asynchronously\")\n\t})\n\n\tt.Run(\"CLI flag overrides config: flag=false overrides config=true\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.True\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(shortString, \"--fast-provide-root=false\")\n\t\trequire.Equal(t, shortStringCidV0, cidStr)\n\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\t// Flag should disable it despite config saying true\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: skipped\")\n\t})\n}\n\n// createDirectoryForHAMTLinksEstimation creates a directory with the specified number\n// of files for testing links-based size estimation (size = sum of nameLen + cidLen).\n// Used by legacy profiles (unixfs-v0-2015).\n//\n// The lastNameLen parameter allows the last file to have a different name length,\n// enabling exact +1 byte threshold tests.\nfunc createDirectoryForHAMTLinksEstimation(dirPath string, numFiles, nameLen, lastNameLen int, seed string) error {\n\treturn createDeterministicFiles(dirPath, numFiles, nameLen, lastNameLen, seed)\n}\n\n// createDirectoryForHAMTBlockEstimation creates a directory with the specified number\n// of files for testing block-based size estimation (LinkSerializedSize with protobuf overhead).\n// Used by modern profiles (unixfs-v1-2025).\n//\n// The lastNameLen parameter allows the last file to have a different name length,\n// enabling exact +1 byte threshold tests.\nfunc createDirectoryForHAMTBlockEstimation(dirPath string, numFiles, nameLen, lastNameLen int, seed string) error {\n\treturn createDeterministicFiles(dirPath, numFiles, nameLen, lastNameLen, seed)\n}\n\n// createDeterministicFiles creates numFiles files with deterministic names.\n// Files 0 to numFiles-2 have nameLen characters, and the last file has lastNameLen characters.\n// Each file contains \"x\" (1 byte) for non-zero tsize in directory links.\nfunc createDeterministicFiles(dirPath string, numFiles, nameLen, lastNameLen int, seed string) error {\n\talphabetLen := len(testutils.AlphabetEasy)\n\n\t// Deterministic pseudo-random bytes for static filenames\n\tdrand, err := testutils.DeterministicRandomReader(\"1MiB\", seed)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor i := range numFiles {\n\t\t// Use lastNameLen for the final file\n\t\tcurrentNameLen := nameLen\n\t\tif i == numFiles-1 {\n\t\t\tcurrentNameLen = lastNameLen\n\t\t}\n\n\t\tbuf := make([]byte, currentNameLen)\n\t\t_, err := io.ReadFull(drand, buf)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Convert deterministic pseudo-random bytes to ASCII\n\t\tvar sb strings.Builder\n\t\tfor _, b := range buf {\n\t\t\tchar := testutils.AlphabetEasy[int(b)%alphabetLen]\n\t\t\tsb.WriteRune(char)\n\t\t}\n\t\tfilename := sb.String()[:currentNameLen]\n\t\tfilePath := filepath.Join(dirPath, filename)\n\n\t\t// Create file with 1-byte content for non-zero tsize\n\t\tif err := os.WriteFile(filePath, []byte(\"x\"), 0o644); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n" }, { "path": "test/cli/agent_version_unicode_test.go", "content": "package cli\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestCleanAndTrimUnicode(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput string\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"Basic ASCII\",\n\t\t\tinput: \"kubo/1.0.0\",\n\t\t\texpected: \"kubo/1.0.0\",\n\t\t},\n\t\t{\n\t\t\tname: \"Polish characters preserved\",\n\t\t\tinput: \"test-ąęćłńóśźż\",\n\t\t\texpected: \"test-ąęćłńóśźż\",\n\t\t},\n\t\t{\n\t\t\tname: \"Chinese characters preserved\",\n\t\t\tinput: \"版本-中文测试\",\n\t\t\texpected: \"版本-中文测试\",\n\t\t},\n\t\t{\n\t\t\tname: \"Arabic text preserved\",\n\t\t\tinput: \"اختبار-العربية\",\n\t\t\texpected: \"اختبار-العربية\",\n\t\t},\n\t\t{\n\t\t\tname: \"Emojis preserved\",\n\t\t\tinput: \"version-1.0-🚀-🎉\",\n\t\t\texpected: \"version-1.0-🚀-🎉\",\n\t\t},\n\t\t{\n\t\t\tname: \"Complex Unicode with combining marks preserved\",\n\t\t\tinput: \"h̸̢̢̢̢̢̢̢̢̢̢e̵̵̵̵̵̵̵̵̵̵l̷̷̷̷̷̷̷̷̷̷l̶̶̶̶̶̶̶̶̶̶o̴̴̴̴̴̴̴̴̴̴\",\n\t\t\texpected: \"h̸̢̢̢̢̢̢̢̢̢̢e̵̵̵̵̵̵̵̵̵̵l̷̷̷̷̷̷̷̷̷̷l̶̶̶̶̶̶̶̶̶̶o̴̴̴̴̴̴̴̴̴̴\", // Preserved as-is (only 50 runes)\n\t\t},\n\t\t{\n\t\t\tname: \"Long text with combining marks truncated at 128\",\n\t\t\tinput: strings.Repeat(\"ẽ̸̢̛̖̬͈͉͖͇͈̭̥́̓̌̾͊̊̂̄̍̅̂͌́\", 10), // Very long text (260 runes)\n\t\t\texpected: \"ẽ̸̢̛̖̬͈͉͖͇͈̭̥́̓̌̾͊̊̂̄̍̅̂͌́ẽ̸̢̛̖̬͈͉͖͇͈̭̥́̓̌̾͊̊̂̄̍̅̂͌́ẽ̸̢̛̖̬͈͉͖͇͈̭̥́̓̌̾͊̊̂̄̍̅̂͌́ẽ̸̢̛̖̬͈͉͖͇͈̭̥́̓̌̾͊̊̂̄̍̅̂͌́ẽ̸̢̛̖̬͈͉͖͇͈̭̥́̓̌̾͊̊̂̄̍̅̂\", // Truncated at 128 runes\n\t\t},\n\t\t{\n\t\t\tname: \"Zero-width characters replaced with U+FFFD\",\n\t\t\tinput: \"test\\u200Bzero\\u200Cwidth\\u200D\\uFEFFchars\",\n\t\t\texpected: \"test�zero�width��chars\",\n\t\t},\n\t\t{\n\t\t\tname: \"RTL/LTR override replaced with U+FFFD\",\n\t\t\tinput: \"test\\u202Drtl\\u202Eltr\\u202Aoverride\",\n\t\t\texpected: \"test�rtl�ltr�override\",\n\t\t},\n\t\t{\n\t\t\tname: \"Bidi isolates replaced with U+FFFD\",\n\t\t\tinput: \"test\\u2066bidi\\u2067isolate\\u2068text\\u2069end\",\n\t\t\texpected: \"test�bidi�isolate�text�end\",\n\t\t},\n\t\t{\n\t\t\tname: \"Control characters replaced with U+FFFD\",\n\t\t\tinput: \"test\\x00null\\x1Fescape\\x7Fdelete\",\n\t\t\texpected: \"test�null�escape�delete\",\n\t\t},\n\t\t{\n\t\t\tname: \"Combining marks preserved\",\n\t\t\tinput: \"e\\u0301\\u0302\\u0303\\u0304\\u0305\", // e with 5 combining marks\n\t\t\texpected: \"e\\u0301\\u0302\\u0303\\u0304\\u0305\", // All preserved\n\t\t},\n\t\t{\n\t\t\tname: \"No truncation at 70 characters\",\n\t\t\tinput: \"123456789012345678901234567890123456789012345678901234567890123456789\",\n\t\t\texpected: \"123456789012345678901234567890123456789012345678901234567890123456789\",\n\t\t},\n\t\t{\n\t\t\tname: \"No truncation with Unicode - 70 rockets preserved\",\n\t\t\tinput: strings.Repeat(\"🚀\", 70),\n\t\t\texpected: strings.Repeat(\"🚀\", 70),\n\t\t},\n\t\t{\n\t\t\tname: \"Empty string\",\n\t\t\tinput: \"\",\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Only whitespace with control chars\",\n\t\t\tinput: \" \\t\\n \",\n\t\t\texpected: \"\\uFFFD\\uFFFD\", // Tab and newline become U+FFFD, spaces trimmed\n\t\t},\n\t\t{\n\t\t\tname: \"Leading and trailing whitespace\",\n\t\t\tinput: \" test \",\n\t\t\texpected: \"test\",\n\t\t},\n\t\t{\n\t\t\tname: \"Complex mix - invisible chars replaced with U+FFFD, Unicode preserved\",\n\t\t\tinput: \"kubo/1.0-🚀\\u200B h̸̢̏̔ḛ̶̽̀s̵t\\u202E-ąęł-中文\",\n\t\t\texpected: \"kubo/1.0-🚀� h̸̢̏̔ḛ̶̽̀s̵t�-ąęł-中文\",\n\t\t},\n\t\t{\n\t\t\tname: \"Emoji with skin tone preserved\",\n\t\t\tinput: \"👍🏽\", // Thumbs up with skin tone modifier\n\t\t\texpected: \"👍🏽\", // Preserved as-is\n\t\t},\n\t\t{\n\t\t\tname: \"Mixed scripts preserved\",\n\t\t\tinput: \"Hello-你好-مرحبا-Здравствуйте\",\n\t\t\texpected: \"Hello-你好-مرحبا-Здравствуйте\",\n\t\t},\n\t\t{\n\t\t\tname: \"Format characters replaced with U+FFFD\",\n\t\t\tinput: \"test\\u00ADsoft\\u2060word\\u206Fnom\\u200Ebreak\",\n\t\t\texpected: \"test�soft�word�nom�break\", // Soft hyphen, word joiner, etc replaced\n\t\t},\n\t\t{\n\t\t\tname: \"Complex Unicode text with many combining marks (91 runes, no truncation)\",\n\t\t\tinput: \"ț̸̢͙̞̖̏̔ȩ̶̰͓̪͎̱̠̥̳͔̽̀̃̿̌̾̀͗̕̕͜s̵̢̛̖̬͈͉͖͇͈̭̥̃́̓̌̾͊̊̂̄̍̅̂͌́ͅţ̴̯̹̪͖͓̘̊́̑̄̋̈́͐̈́̔̇̄̂́̎̓͛͠ͅ test\",\n\t\t\texpected: \"ț̸̢͙̞̖̏̔ȩ̶̰͓̪͎̱̠̥̳͔̽̀̃̿̌̾̀͗̕̕͜s̵̢̛̖̬͈͉͖͇͈̭̥̃́̓̌̾͊̊̂̄̍̅̂͌́ͅţ̴̯̹̪͖͓̘̊́̑̄̋̈́͐̈́̔̇̄̂́̎̓͛͠ͅ test\", // Not truncated (91 < 128)\n\t\t},\n\t\t{\n\t\t\tname: \"Truncation at 128 characters\",\n\t\t\tinput: strings.Repeat(\"a\", 150),\n\t\t\texpected: strings.Repeat(\"a\", 128),\n\t\t},\n\t\t{\n\t\t\tname: \"Truncation with Unicode at 128\",\n\t\t\tinput: strings.Repeat(\"🚀\", 150),\n\t\t\texpected: strings.Repeat(\"🚀\", 128),\n\t\t},\n\t\t{\n\t\t\tname: \"Private use characters preserved (per spec)\",\n\t\t\tinput: \"test\\uE000\\uF8FF\", // Private use area characters\n\t\t\texpected: \"test\\uE000\\uF8FF\", // Should be preserved\n\t\t},\n\t\t{\n\t\t\tname: \"U+FFFD replacement for multiple categories\",\n\t\t\tinput: \"a\\x00b\\u200Cc\\u202Ed\", // control, format chars\n\t\t\texpected: \"a\\uFFFDb\\uFFFDc\\uFFFDd\", // All replaced with U+FFFD\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := cmdutils.CleanAndTrim(tt.input)\n\t\t\tassert.Equal(t, tt.expected, result, \"CleanAndTrim(%q) = %q, want %q\", tt.input, result, tt.expected)\n\t\t})\n\t}\n}\n\nfunc TestCleanAndTrimIdempotent(t *testing.T) {\n\t// Test that applying CleanAndTrim twice gives the same result\n\tinputs := []string{\n\t\t\"test-ąęćłńóśźż\",\n\t\t\"版本-中文测试\",\n\t\t\"version-1.0-🚀-🎉\",\n\t\t\"h̸e̵l̷l̶o̴ w̸o̵r̷l̶d̴\",\n\t\t\"test\\u200Bzero\\u200Cwidth\",\n\t}\n\n\tfor _, input := range inputs {\n\t\tonce := cmdutils.CleanAndTrim(input)\n\t\ttwice := cmdutils.CleanAndTrim(once)\n\t\tassert.Equal(t, once, twice, \"CleanAndTrim should be idempotent for %q\", input)\n\t}\n}\n\nfunc TestCleanAndTrimSecurity(t *testing.T) {\n\t// Test that all invisible/dangerous characters are removed\n\ttests := []struct {\n\t\tname string\n\t\tinput string\n\t\tcheck func(string) bool\n\t}{\n\t\t{\n\t\t\tname: \"No zero-width spaces\",\n\t\t\tinput: \"test\\u200B\\u200C\\u200Dtest\",\n\t\t\tcheck: func(s string) bool {\n\t\t\t\treturn !strings.Contains(s, \"\\u200B\") && !strings.Contains(s, \"\\u200C\") && !strings.Contains(s, \"\\u200D\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"No bidi overrides\",\n\t\t\tinput: \"test\\u202A\\u202B\\u202C\\u202D\\u202Etest\",\n\t\t\tcheck: func(s string) bool {\n\t\t\t\tfor _, r := range []rune{0x202A, 0x202B, 0x202C, 0x202D, 0x202E} {\n\t\t\t\t\tif strings.ContainsRune(s, r) {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"No control characters\",\n\t\t\tinput: \"test\\x00\\x01\\x02\\x1F\\x7Ftest\",\n\t\t\tcheck: func(s string) bool {\n\t\t\t\tfor _, r := range s {\n\t\t\t\t\tif r < 0x20 || r == 0x7F {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := cmdutils.CleanAndTrim(tt.input)\n\t\t\tassert.True(t, tt.check(result), \"Security check failed for %q -> %q\", tt.input, result)\n\t\t})\n\t}\n}\n" }, { "path": "test/cli/api_file_test.go", "content": "package cli\n\nimport (\n\t\"net/http\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestAddressFileReady verifies that when address files ($IPFS_PATH/api and\n// $IPFS_PATH/gateway) are created, the corresponding HTTP servers are ready\n// to accept connections immediately. This prevents race conditions for tools\n// like systemd path units that start services when these files appear.\nfunc TestAddressFileReady(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"api file\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\n\t\t// Start daemon in background (don't use StartDaemon which waits for API)\n\t\tres := node.Runner.MustRun(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"daemon\"},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\tnode.Daemon = res\n\t\tdefer node.StopDaemon()\n\n\t\t// Poll for api file to appear\n\t\tapiFile := filepath.Join(node.Dir, \"api\")\n\t\tvar fileExists bool\n\t\tfor range 100 {\n\t\t\tif _, err := os.Stat(apiFile); err == nil {\n\t\t\t\tfileExists = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t}\n\t\trequire.True(t, fileExists, \"api file should be created\")\n\n\t\t// Read the api file to get the address\n\t\tapiAddr, err := node.TryAPIAddr()\n\t\trequire.NoError(t, err)\n\n\t\t// Extract IP and port from multiaddr\n\t\tip, err := apiAddr.ValueForProtocol(4) // P_IP4\n\t\trequire.NoError(t, err)\n\t\tport, err := apiAddr.ValueForProtocol(6) // P_TCP\n\t\trequire.NoError(t, err)\n\n\t\t// Immediately try to use the API - should work on first attempt\n\t\turl := \"http://\" + ip + \":\" + port + \"/api/v0/id\"\n\t\tresp, err := http.Post(url, \"\", nil)\n\t\trequire.NoError(t, err, \"RPC API should be ready immediately when api file exists\")\n\t\tdefer resp.Body.Close()\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t})\n\n\tt.Run(\"gateway file\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\n\t\t// Start daemon in background\n\t\tres := node.Runner.MustRun(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"daemon\"},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\tnode.Daemon = res\n\t\tdefer node.StopDaemon()\n\n\t\t// Poll for gateway file to appear\n\t\tgatewayFile := filepath.Join(node.Dir, \"gateway\")\n\t\tvar fileExists bool\n\t\tfor range 100 {\n\t\t\tif _, err := os.Stat(gatewayFile); err == nil {\n\t\t\t\tfileExists = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t}\n\t\trequire.True(t, fileExists, \"gateway file should be created\")\n\n\t\t// Read the gateway file to get the URL (already includes http:// prefix)\n\t\tgatewayURL, err := os.ReadFile(gatewayFile)\n\t\trequire.NoError(t, err)\n\n\t\t// Immediately try to use the Gateway - should work on first attempt\n\t\turl := strings.TrimSpace(string(gatewayURL)) + \"/ipfs/bafkqaaa\" // empty file CID\n\t\tresp, err := http.Get(url)\n\t\trequire.NoError(t, err, \"Gateway should be ready immediately when gateway file exists\")\n\t\tdefer resp.Body.Close()\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t})\n}\n" }, { "path": "test/cli/autoconf/autoconf_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAutoConf(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"basic functionality\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfBasicFunctionality(t)\n\t})\n\n\tt.Run(\"background service updates\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfBackgroundService(t)\n\t})\n\n\tt.Run(\"HTTP error scenarios\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfHTTPErrors(t)\n\t})\n\n\tt.Run(\"cache-based config expansion\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfCacheBasedExpansion(t)\n\t})\n\n\tt.Run(\"disabled autoconf\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfDisabled(t)\n\t})\n\n\tt.Run(\"bootstrap list shows auto as-is\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestBootstrapListResolved(t)\n\t})\n\n\tt.Run(\"daemon uses resolved bootstrap values\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestDaemonUsesResolvedBootstrap(t)\n\t})\n\n\tt.Run(\"empty cache uses fallback defaults\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestEmptyCacheUsesFallbacks(t)\n\t})\n\n\tt.Run(\"stale cache with unreachable server\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestStaleCacheWithUnreachableServer(t)\n\t})\n\n\tt.Run(\"autoconf disabled with auto values\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfDisabledWithAutoValues(t)\n\t})\n\n\tt.Run(\"network behavior - cached vs refresh\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfNetworkBehavior(t)\n\t})\n\n\tt.Run(\"HTTPS autoconf server\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAutoConfWithHTTPS(t)\n\t})\n}\n\nfunc testAutoConfBasicFunctionality(t *testing.T) {\n\t// Load test autoconf data\n\tautoConfData := loadTestData(t, \"valid_autoconf.json\")\n\n\t// Create HTTP server that serves autoconf.json\n\tetag := `\"test-etag-123\"`\n\trequestCount := 0\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\trequestCount++\n\t\tt.Logf(\"AutoConf server request #%d: %s %s\", requestCount, r.Method, r.URL.Path)\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", etag)\n\t\tw.Header().Set(\"Last-Modified\", \"Wed, 21 Oct 2015 07:28:00 GMT\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node and configure it to use our test server\n\t// Use test profile to avoid autoconf profile being applied by default\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t// Disable background updates to prevent multiple requests\n\tnode.SetIPFSConfig(\"AutoConf.RefreshInterval\", \"24h\")\n\n\t// Test with normal bootstrap peers (not \"auto\") to avoid multiaddr parsing issues\n\t// This tests that autoconf fetching works without complex auto replacement\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"})\n\n\t// Start daemon to trigger autoconf fetch\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Give autoconf some time to fetch\n\ttime.Sleep(2 * time.Second)\n\n\t// Verify that the autoconf system fetched data from our server\n\tt.Logf(\"Server request count: %d\", requestCount)\n\trequire.GreaterOrEqual(t, requestCount, 1, \"AutoConf server should have been called at least once\")\n\n\t// Test that daemon is functional\n\tresult := node.RunIPFS(\"id\")\n\tassert.Equal(t, 0, result.ExitCode(), \"IPFS daemon should be responsive\")\n\tassert.Contains(t, result.Stdout.String(), \"ID\", \"IPFS id command should return peer information\")\n\n\t// Success! AutoConf system is working:\n\t// 1. Server was called (proves fetch works)\n\t// 2. Daemon started successfully (proves DNS resolver validation is fixed)\n\t// 3. Daemon is functional (proves autoconf doesn't break core functionality)\n\t// Note: We skip checking metadata values due to JSON parsing complexity in test harness\n}\n\nfunc testAutoConfBackgroundService(t *testing.T) {\n\t// Test that the startAutoConfUpdater() goroutine makes network requests for background refresh\n\t// This is separate from daemon config operations which now use cache-first approach\n\n\t// Load initial and updated test data\n\tinitialData := loadTestData(t, \"valid_autoconf.json\")\n\tupdatedData := loadTestData(t, \"updated_autoconf.json\")\n\n\t// Track which config is being served\n\tcurrentData := initialData\n\tvar requestCount atomic.Int32\n\n\t// Create server that switches payload after first request\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tcount := requestCount.Add(1)\n\t\tt.Logf(\"Background service request #%d from %s\", count, r.UserAgent())\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", fmt.Sprintf(`\"background-test-etag-%d\"`, count))\n\t\tw.Header().Set(\"Last-Modified\", time.Now().Format(http.TimeFormat))\n\n\t\tif count > 1 {\n\t\t\t// After first request, serve updated config\n\t\t\tcurrentData = updatedData\n\t\t}\n\n\t\t_, _ = w.Write(currentData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with short refresh interval to trigger background service\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"AutoConf.RefreshInterval\", \"1s\") // Very short for testing background service\n\n\t// Use normal bootstrap values to avoid dependency on autoconf during initialization\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"})\n\n\t// Start daemon - this should start the background service via startAutoConfUpdater()\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Wait for initial request (daemon startup may trigger one)\n\ttime.Sleep(1 * time.Second)\n\tinitialCount := requestCount.Load()\n\tt.Logf(\"Initial request count after daemon start: %d\", initialCount)\n\n\t// Wait for background service to make additional requests\n\t// The background service should make requests at the RefreshInterval (1s)\n\ttime.Sleep(3 * time.Second)\n\n\tfinalCount := requestCount.Load()\n\tt.Logf(\"Final request count after background updates: %d\", finalCount)\n\n\t// Background service should have made multiple requests due to 1s refresh interval\n\tassert.Greater(t, finalCount, initialCount,\n\t\t\"Background service should have made additional requests beyond daemon startup\")\n\n\t// Verify that the service is actively making requests (not just relying on cache)\n\tassert.GreaterOrEqual(t, finalCount, int32(2),\n\t\t\"Should have at least 2 requests total (startup + background refresh)\")\n\n\tt.Logf(\"Successfully verified startAutoConfUpdater() background service makes network requests\")\n}\n\nfunc testAutoConfHTTPErrors(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tstatusCode int\n\t\tbody string\n\t}{\n\t\t{\"404 Not Found\", http.StatusNotFound, \"Not Found\"},\n\t\t{\"500 Internal Server Error\", http.StatusInternalServerError, \"Internal Server Error\"},\n\t\t{\"Invalid JSON\", http.StatusOK, \"invalid json content\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Create server that returns error\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.WriteHeader(tt.statusCode)\n\t\t\t\t_, _ = w.Write([]byte(tt.body))\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\t// Create node with failing AutoConf URL\n\t\t\t// Use test profile to avoid autoconf profile being applied by default\n\t\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\t\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\t\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t\t\t// Start daemon - it should start but autoconf should fail gracefully\n\t\t\tnode.StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Daemon should still be functional even with autoconf HTTP errors\n\t\t\tresult := node.RunIPFS(\"version\")\n\t\t\tassert.Equal(t, 0, result.ExitCode(), \"Daemon should start even with HTTP errors in autoconf\")\n\t\t})\n\t}\n}\n\nfunc testAutoConfCacheBasedExpansion(t *testing.T) {\n\t// Test that config expansion works correctly with cached autoconf data\n\t// without requiring active network requests during expansion operations\n\n\tautoConfData := loadTestData(t, \"valid_autoconf.json\")\n\n\t// Create server that serves autoconf data\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", `\"cache-test-etag\"`)\n\t\tw.Header().Set(\"Last-Modified\", \"Wed, 21 Oct 2015 07:28:00 GMT\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with autoconf enabled\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Set configuration with \"auto\" values to test expansion\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"test.\": \"auto\"})\n\n\t// Populate cache by running a command that triggers autoconf (without daemon)\n\tresult := node.RunIPFS(\"bootstrap\", \"list\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Initial bootstrap expansion should succeed\")\n\n\texpandedBootstrap := result.Stdout.String()\n\tassert.NotContains(t, expandedBootstrap, \"auto\", \"Expanded bootstrap should not contain 'auto' literal\")\n\tassert.Greater(t, len(strings.Fields(expandedBootstrap)), 0, \"Should have expanded bootstrap peers\")\n\n\t// Test that subsequent config operations work with cached data (no network required)\n\t// This simulates the cache-first behavior our architecture now uses\n\n\t// Test Bootstrap expansion\n\tresult = node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Cached bootstrap expansion should succeed\")\n\n\tvar expandedBootstrapList []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedBootstrapList)\n\trequire.NoError(t, err)\n\tassert.NotContains(t, expandedBootstrapList, \"auto\", \"Expanded bootstrap list should not contain 'auto'\")\n\tassert.Greater(t, len(expandedBootstrapList), 0, \"Should have expanded bootstrap peers from cache\")\n\n\t// Test Routing.DelegatedRouters expansion\n\tresult = node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Cached router expansion should succeed\")\n\n\tvar expandedRouters []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\tassert.NotContains(t, expandedRouters, \"auto\", \"Expanded routers should not contain 'auto'\")\n\n\t// Test DNS.Resolvers expansion\n\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Cached DNS resolver expansion should succeed\")\n\n\tvar expandedResolvers map[string]string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedResolvers)\n\trequire.NoError(t, err)\n\n\t// Should have expanded the \"auto\" value for test. domain, or removed it if no autoconf data available\n\ttestResolver, exists := expandedResolvers[\"test.\"]\n\tif exists {\n\t\tassert.NotEqual(t, \"auto\", testResolver, \"test. resolver should not be literal 'auto'\")\n\t\tt.Logf(\"Found expanded resolver for test.: %s\", testResolver)\n\t} else {\n\t\tt.Logf(\"No resolver found for test. domain (autoconf may not have DNS resolver data)\")\n\t}\n\n\t// Test full config expansion\n\tresult = node.RunIPFS(\"config\", \"show\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Full config expansion should succeed\")\n\n\texpandedConfig := result.Stdout.String()\n\t// Should not contain literal \"auto\" values after expansion\n\tassert.NotContains(t, expandedConfig, `\"auto\"`, \"Expanded config should not contain literal 'auto' values\")\n\tassert.Contains(t, expandedConfig, `\"Bootstrap\"`, \"Should contain Bootstrap section\")\n\tassert.Contains(t, expandedConfig, `\"DNS\"`, \"Should contain DNS section\")\n\n\tt.Logf(\"Successfully tested cache-based config expansion without active network requests\")\n}\n\nfunc testAutoConfDisabled(t *testing.T) {\n\t// Create node with AutoConf disabled but \"auto\" values\n\t// Use test profile to avoid autoconf profile being applied by default\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", false)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Test by trying to list bootstrap - when AutoConf is disabled, it should show literal \"auto\"\n\tresult := node.RunIPFS(\"bootstrap\", \"list\")\n\tif result.ExitCode() == 0 {\n\t\t// If command succeeds, it should show literal \"auto\" (no resolution)\n\t\toutput := result.Stdout.String()\n\t\tassert.Contains(t, output, \"auto\", \"Should show literal 'auto' when AutoConf is disabled\")\n\t} else {\n\t\t// If command fails, error should mention autoconf issue\n\t\tassert.Contains(t, result.Stderr.String(), \"auto\", \"Should mention 'auto' values in error\")\n\t}\n}\n\n// Helper function to load test data files\nfunc loadTestData(t *testing.T, filename string) []byte {\n\tt.Helper()\n\n\tdata, err := os.ReadFile(\"testdata/\" + filename)\n\trequire.NoError(t, err, \"Failed to read test data file: %s\", filename)\n\n\treturn data\n}\n\nfunc testBootstrapListResolved(t *testing.T) {\n\t// Test that bootstrap list shows \"auto\" as-is (not expanded)\n\n\t// Load test autoconf data\n\tautoConfData := loadTestData(t, \"valid_autoconf.json\")\n\n\t// Create HTTP server that serves autoconf.json\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with \"auto\" bootstrap value\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Test 1: bootstrap list (without --expand-auto) shows \"auto\" as-is - NO DAEMON NEEDED!\n\tresult := node.RunIPFS(\"bootstrap\", \"list\")\n\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap list command should succeed\")\n\n\toutput := result.Stdout.String()\n\tt.Logf(\"Bootstrap list output: %s\", output)\n\tassert.Contains(t, output, \"auto\", \"bootstrap list should show 'auto' value as-is\")\n\n\t// Should NOT contain expanded bootstrap peers without --expand-auto\n\tunexpectedPeers := []string{\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n\t}\n\n\tfor _, peer := range unexpectedPeers {\n\t\tassert.NotContains(t, output, peer, \"bootstrap list should not contain expanded peer: %s\", peer)\n\t}\n\n\t// Test 2: bootstrap list --expand-auto shows expanded values (no daemon needed!)\n\tresult = node.RunIPFS(\"bootstrap\", \"list\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap list --expand-auto command should succeed\")\n\n\texpandedOutput := result.Stdout.String()\n\tt.Logf(\"Bootstrap list --expand-auto output: %s\", expandedOutput)\n\n\t// Should NOT contain \"auto\" literal when expanded\n\tassert.NotContains(t, expandedOutput, \"auto\", \"bootstrap list --expand-auto should not show 'auto' literal\")\n\n\t// Should contain at least one expanded bootstrap peer\n\texpectedPeers := []string{\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n\t}\n\n\tfoundExpectedPeer := false\n\tfor _, peer := range expectedPeers {\n\t\tif strings.Contains(expandedOutput, peer) {\n\t\t\tfoundExpectedPeer = true\n\t\t\tt.Logf(\"Found expected expanded peer: %s\", peer)\n\t\t\tbreak\n\t\t}\n\t}\n\tassert.True(t, foundExpectedPeer, \"bootstrap list --expand-auto should contain at least one expanded bootstrap peer\")\n}\n\nfunc testDaemonUsesResolvedBootstrap(t *testing.T) {\n\t// Test that daemon actually uses expanded bootstrap values for P2P connections\n\t// even though bootstrap list shows \"auto\"\n\n\t// Step 1: Create bootstrap node (target for connections)\n\tbootstrapNode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t// Set a specific swarm port for the bootstrap node to avoid port 0 issues\n\tbootstrapNode.SetIPFSConfig(\"Addresses.Swarm\", []string{\"/ip4/127.0.0.1/tcp/14001\"})\n\t// Disable routing and discovery to ensure it's only discoverable via explicit multiaddr\n\tbootstrapNode.SetIPFSConfig(\"Routing.Type\", \"none\")\n\tbootstrapNode.SetIPFSConfig(\"Discovery.MDNS.Enabled\", false)\n\tbootstrapNode.SetIPFSConfig(\"Bootstrap\", []string{}) // No bootstrap peers\n\n\t// Start the bootstrap node first\n\tbootstrapNode.StartDaemon()\n\tdefer bootstrapNode.StopDaemon()\n\n\t// Get bootstrap node's peer ID and swarm address\n\tbootstrapPeerID := bootstrapNode.PeerID()\n\n\t// Use the configured swarm address (we set it to a specific port above)\n\tbootstrapMultiaddr := fmt.Sprintf(\"/ip4/127.0.0.1/tcp/14001/p2p/%s\", bootstrapPeerID.String())\n\tt.Logf(\"Bootstrap node configured at: %s\", bootstrapMultiaddr)\n\n\t// Step 2: Create autoconf server that returns bootstrap node's address\n\tautoConfData := fmt.Sprintf(`{\n\t\t\"AutoConfVersion\": 2025072301,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": [\"%s\"]\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t\t\"DNSResolvers\": {},\n\t\t\"DelegatedEndpoints\": {}\n\t}`, bootstrapMultiaddr)\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(autoConfData))\n\t}))\n\tdefer server.Close()\n\n\t// Step 3: Create autoconf-enabled node that should connect to bootstrap node\n\tautoconfNode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tautoconfNode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tautoconfNode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tautoconfNode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"}) // This should resolve to bootstrap node\n\t// Disable other discovery methods to force bootstrap-only connectivity\n\tautoconfNode.SetIPFSConfig(\"Routing.Type\", \"none\")\n\tautoconfNode.SetIPFSConfig(\"Discovery.MDNS.Enabled\", false)\n\n\t// Start the autoconf node\n\tautoconfNode.StartDaemon()\n\tdefer autoconfNode.StopDaemon()\n\n\t// Step 4: Give time for autoconf resolution and connection attempts\n\ttime.Sleep(8 * time.Second)\n\n\t// Step 5: Verify both nodes are responsive\n\tresult := bootstrapNode.RunIPFS(\"id\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Bootstrap node should be responsive: %s\", result.Stderr.String())\n\n\tresult = autoconfNode.RunIPFS(\"id\")\n\trequire.Equal(t, 0, result.ExitCode(), \"AutoConf node should be responsive: %s\", result.Stderr.String())\n\n\t// Step 6: Verify that autoconf node connected to bootstrap node\n\t// Check swarm peers on autoconf node - it should show bootstrap node's peer ID\n\tresult = autoconfNode.RunIPFS(\"swarm\", \"peers\")\n\tif result.ExitCode() == 0 {\n\t\tpeerOutput := result.Stdout.String()\n\t\tif strings.Contains(peerOutput, bootstrapPeerID.String()) {\n\t\t\tt.Logf(\"SUCCESS: AutoConf node connected to bootstrap peer %s\", bootstrapPeerID.String())\n\t\t} else {\n\t\t\tt.Logf(\"No active connection found. Peers output: %s\", peerOutput)\n\t\t\t// This might be OK if connection attempt was made but didn't persist\n\t\t}\n\t} else {\n\t\t// If swarm peers fails, try alternative verification via daemon logs\n\t\tt.Logf(\"Swarm peers command failed, checking daemon logs for connection attempts\")\n\t\tdaemonOutput := autoconfNode.Daemon.Stderr.String()\n\t\tif strings.Contains(daemonOutput, bootstrapPeerID.String()) {\n\t\t\tt.Logf(\"SUCCESS: Found bootstrap peer %s in daemon logs, connection attempted\", bootstrapPeerID.String())\n\t\t} else {\n\t\t\tt.Logf(\"Daemon stderr: %s\", daemonOutput)\n\t\t}\n\t}\n\n\t// Step 7: Verify bootstrap configuration still shows \"auto\" (not resolved values)\n\tresult = autoconfNode.RunIPFS(\"bootstrap\", \"list\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Bootstrap list command should work\")\n\tassert.Contains(t, result.Stdout.String(), \"auto\",\n\t\t\"Bootstrap list should still show 'auto' even though values were resolved for networking\")\n}\n\nfunc testEmptyCacheUsesFallbacks(t *testing.T) {\n\t// Test that daemon uses fallback defaults when no cache exists and server is unreachable\n\n\t// Create IPFS node with auto values and unreachable autoconf server\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", \"http://127.0.0.1:9999/nonexistent\")\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\n\t// Start daemon - should succeed using fallback values\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Verify daemon started successfully (uses fallback bootstrap)\n\tresult := node.RunIPFS(\"id\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Daemon should start successfully with fallback values\")\n\n\t// Verify config commands still show \"auto\"\n\tresult = node.RunIPFS(\"config\", \"Bootstrap\")\n\trequire.Equal(t, 0, result.ExitCode())\n\tassert.Contains(t, result.Stdout.String(), \"auto\", \"Bootstrap config should still show 'auto'\")\n\n\tresult = node.RunIPFS(\"config\", \"Routing.DelegatedRouters\")\n\trequire.Equal(t, 0, result.ExitCode())\n\tassert.Contains(t, result.Stdout.String(), \"auto\", \"DelegatedRouters config should still show 'auto'\")\n\n\t// Check daemon logs for error about failed autoconf fetch\n\tlogOutput := node.Daemon.Stderr.String()\n\t// The daemon should attempt to fetch autoconf but will use fallbacks on failure\n\t// We don't require specific log messages as long as the daemon starts successfully\n\tif logOutput != \"\" {\n\t\tt.Logf(\"Daemon logs: %s\", logOutput)\n\t}\n}\n\nfunc testStaleCacheWithUnreachableServer(t *testing.T) {\n\t// Test that daemon uses stale cache when server is unreachable\n\n\t// First create a working autoconf server and cache\n\tautoConfData := loadTestData(t, \"valid_autoconf.json\")\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\n\t// Create node and fetch autoconf to populate cache\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Start daemon briefly to populate cache\n\tnode.StartDaemon()\n\ttime.Sleep(1 * time.Second) // Allow cache population\n\tnode.StopDaemon()\n\n\t// Close the server to make it unreachable\n\tserver.Close()\n\n\t// Update config to point to unreachable server\n\tnode.SetIPFSConfig(\"AutoConf.URL\", \"http://127.0.0.1:9999/unreachable\")\n\n\t// Start daemon again - should use stale cache\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Verify daemon started successfully (uses cached autoconf)\n\tresult := node.RunIPFS(\"id\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Daemon should start successfully with cached autoconf\")\n\n\t// Check daemon logs for error about using stale config\n\tlogOutput := node.Daemon.Stderr.String()\n\t// The daemon should use cached config when server is unreachable\n\t// We don't require specific log messages as long as the daemon starts successfully\n\tif logOutput != \"\" {\n\t\tt.Logf(\"Daemon logs: %s\", logOutput)\n\t}\n}\n\nfunc testAutoConfDisabledWithAutoValues(t *testing.T) {\n\t// Test that daemon fails to start when AutoConf is disabled but \"auto\" values are present\n\n\t// Create IPFS node with AutoConf disabled but \"auto\" values configured\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", false)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Test by trying to list bootstrap - when AutoConf is disabled, it should show literal \"auto\"\n\tresult := node.RunIPFS(\"bootstrap\", \"list\")\n\tif result.ExitCode() == 0 {\n\t\t// If command succeeds, it should show literal \"auto\" (no resolution)\n\t\toutput := result.Stdout.String()\n\t\tassert.Contains(t, output, \"auto\", \"Should show literal 'auto' when AutoConf is disabled\")\n\t} else {\n\t\t// If command fails, error should mention autoconf issue\n\t\tlogOutput := result.Stderr.String()\n\t\tassert.Contains(t, logOutput, \"auto\", \"Error should mention 'auto' values\")\n\t\t// Check that the error message contains information about disabled state\n\t\tassert.True(t,\n\t\t\tstrings.Contains(logOutput, \"disabled\") || strings.Contains(logOutput, \"AutoConf.Enabled=false\"),\n\t\t\t\"Error should mention that AutoConf is disabled or show AutoConf.Enabled=false\")\n\t}\n}\n\nfunc testAutoConfNetworkBehavior(t *testing.T) {\n\t// Test the network behavior differences between MustGetConfigCached and MustGetConfigWithRefresh\n\t// This validates that our cache-first architecture works as expected\n\n\tautoConfData := loadTestData(t, \"valid_autoconf.json\")\n\tvar requestCount atomic.Int32\n\n\t// Create server that tracks all requests\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tcount := requestCount.Add(1)\n\t\tt.Logf(\"Network behavior test request #%d: %s %s\", count, r.Method, r.URL.Path)\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", fmt.Sprintf(`\"network-test-etag-%d\"`, count))\n\t\tw.Header().Set(\"Last-Modified\", time.Now().Format(http.TimeFormat))\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with autoconf\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Phase 1: Test cache-first behavior (no network requests expected)\n\tt.Logf(\"=== Phase 1: Testing cache-first behavior ===\")\n\tinitialCount := requestCount.Load()\n\n\t// Multiple config operations should NOT trigger network requests (cache-first)\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Bootstrap config read should succeed\")\n\n\tresult = node.RunIPFS(\"config\", \"show\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Config show should succeed\")\n\n\tresult = node.RunIPFS(\"bootstrap\", \"list\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Bootstrap list should succeed\")\n\n\t// Check that cache-first operations didn't trigger network requests\n\tafterCacheOpsCount := requestCount.Load()\n\tcachedRequestDiff := afterCacheOpsCount - initialCount\n\tt.Logf(\"Network requests during cache-first operations: %d\", cachedRequestDiff)\n\n\t// Phase 2: Test explicit expansion (may trigger cache population)\n\tt.Logf(\"=== Phase 2: Testing expansion operations ===\")\n\tbeforeExpansionCount := requestCount.Load()\n\n\t// Expansion operations may need to populate cache if empty\n\tresult = node.RunIPFS(\"bootstrap\", \"list\", \"--expand-auto\")\n\tif result.ExitCode() == 0 {\n\t\toutput := result.Stdout.String()\n\t\tassert.NotContains(t, output, \"auto\", \"Expanded bootstrap should not contain 'auto' literal\")\n\t\tt.Logf(\"Bootstrap expansion succeeded\")\n\t} else {\n\t\tt.Logf(\"Bootstrap expansion failed (may be due to network/cache issues): %s\", result.Stderr.String())\n\t}\n\n\tresult = node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\tif result.ExitCode() == 0 {\n\t\tt.Logf(\"Config Bootstrap expansion succeeded\")\n\t} else {\n\t\tt.Logf(\"Config Bootstrap expansion failed: %s\", result.Stderr.String())\n\t}\n\n\tafterExpansionCount := requestCount.Load()\n\texpansionRequestDiff := afterExpansionCount - beforeExpansionCount\n\tt.Logf(\"Network requests during expansion operations: %d\", expansionRequestDiff)\n\n\t// Phase 3: Test background service behavior (if daemon is started)\n\tt.Logf(\"=== Phase 3: Testing background service behavior ===\")\n\tbeforeDaemonCount := requestCount.Load()\n\n\t// Set short refresh interval to test background service\n\tnode.SetIPFSConfig(\"AutoConf.RefreshInterval\", \"1s\")\n\n\t// Start daemon - this triggers startAutoConfUpdater() which should make network requests\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Wait for background service to potentially make requests\n\ttime.Sleep(2 * time.Second)\n\n\tafterDaemonCount := requestCount.Load()\n\tdaemonRequestDiff := afterDaemonCount - beforeDaemonCount\n\tt.Logf(\"Network requests from background service: %d\", daemonRequestDiff)\n\n\t// Verify expected behavior patterns\n\tt.Logf(\"=== Summary ===\")\n\tt.Logf(\"Cache-first operations: %d requests\", cachedRequestDiff)\n\tt.Logf(\"Expansion operations: %d requests\", expansionRequestDiff)\n\tt.Logf(\"Background service: %d requests\", daemonRequestDiff)\n\n\t// Cache-first operations should minimize network requests\n\tassert.LessOrEqual(t, cachedRequestDiff, int32(1),\n\t\t\"Cache-first config operations should make minimal network requests\")\n\n\t// Background service should make requests for refresh\n\tif daemonRequestDiff > 0 {\n\t\tt.Logf(\"✓ Background service is making network requests as expected\")\n\t} else {\n\t\tt.Logf(\"⚠ Background service made no requests (may be using existing cache)\")\n\t}\n\n\tt.Logf(\"Successfully verified network behavior patterns in autoconf architecture\")\n}\n\nfunc testAutoConfWithHTTPS(t *testing.T) {\n\t// Test autoconf with HTTPS server and TLSInsecureSkipVerify enabled\n\tautoConfData := loadTestData(t, \"valid_autoconf.json\")\n\n\t// Create HTTPS server with self-signed certificate\n\tserver := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tt.Logf(\"HTTPS autoconf request from %s\", r.UserAgent())\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", `\"https-test-etag\"`)\n\t\tw.Header().Set(\"Last-Modified\", \"Wed, 21 Oct 2015 07:28:00 GMT\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\n\t// Enable HTTP/2 and start with TLS (self-signed certificate)\n\tserver.EnableHTTP2 = true\n\tserver.StartTLS()\n\tdefer server.Close()\n\n\t// Create IPFS node with HTTPS autoconf server and TLS skip verify\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"AutoConf.TLSInsecureSkipVerify\", true) // Allow self-signed cert\n\tnode.SetIPFSConfig(\"AutoConf.RefreshInterval\", \"24h\") // Disable background updates\n\n\t// Use normal bootstrap peers to test HTTPS fetching without complex auto replacement\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"})\n\n\t// Start daemon to trigger HTTPS autoconf fetch\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Give autoconf time to fetch over HTTPS\n\ttime.Sleep(2 * time.Second)\n\n\t// Verify daemon is functional with HTTPS autoconf\n\tresult := node.RunIPFS(\"id\")\n\tassert.Equal(t, 0, result.ExitCode(), \"IPFS daemon should be responsive with HTTPS autoconf\")\n\tassert.Contains(t, result.Stdout.String(), \"ID\", \"IPFS id command should return peer information\")\n\n\t// Test that config operations work with HTTPS-fetched autoconf cache\n\tresult = node.RunIPFS(\"config\", \"show\")\n\tassert.Equal(t, 0, result.ExitCode(), \"Config show should work with HTTPS autoconf\")\n\n\t// Test bootstrap list functionality\n\tresult = node.RunIPFS(\"bootstrap\", \"list\")\n\tassert.Equal(t, 0, result.ExitCode(), \"Bootstrap list should work with HTTPS autoconf\")\n\n\tt.Logf(\"Successfully tested AutoConf with HTTPS server and TLS skip verify\")\n}\n" }, { "path": "test/cli/autoconf/dns_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/miekg/dns\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAutoConfDNS(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"DNS resolution with auto DoH resolver\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestDNSResolutionWithAutoDoH(t)\n\t})\n\n\tt.Run(\"DNS errors are handled properly\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestDNSErrorHandling(t)\n\t})\n}\n\n// mockDoHServer implements a simple DNS-over-HTTPS server for testing\ntype mockDoHServer struct {\n\tt *testing.T\n\tserver *httptest.Server\n\tmu sync.Mutex\n\trequests []string\n\tresponseFunc func(name string) *dns.Msg\n}\n\nfunc newMockDoHServer(t *testing.T) *mockDoHServer {\n\tm := &mockDoHServer{\n\t\tt: t,\n\t\trequests: []string{},\n\t}\n\n\t// Default response function returns a dnslink TXT record\n\tm.responseFunc = func(name string) *dns.Msg {\n\t\tmsg := &dns.Msg{}\n\t\tmsg.SetReply(&dns.Msg{Question: []dns.Question{{Name: name, Qtype: dns.TypeTXT}}})\n\n\t\tif strings.HasPrefix(name, \"_dnslink.\") {\n\t\t\t// Return a valid dnslink record\n\t\t\trr := &dns.TXT{\n\t\t\t\tHdr: dns.RR_Header{\n\t\t\t\t\tName: name,\n\t\t\t\t\tRrtype: dns.TypeTXT,\n\t\t\t\t\tClass: dns.ClassINET,\n\t\t\t\t\tTtl: 300,\n\t\t\t\t},\n\t\t\t\tTxt: []string{\"dnslink=/ipfs/QmYNQJoKGNHTpPxCBPh9KkDpaExgd2duMa3aF6ytMpHdao\"},\n\t\t\t}\n\t\t\tmsg.Answer = append(msg.Answer, rr)\n\t\t}\n\n\t\treturn msg\n\t}\n\n\tmux := http.NewServeMux()\n\tmux.HandleFunc(\"/dns-query\", m.handleDNSQuery)\n\n\tm.server = httptest.NewServer(mux)\n\treturn m\n}\n\nfunc (m *mockDoHServer) handleDNSQuery(w http.ResponseWriter, r *http.Request) {\n\tm.mu.Lock()\n\tdefer m.mu.Unlock()\n\n\tvar dnsMsg *dns.Msg\n\n\tif r.Method == \"GET\" {\n\t\t// Handle GET with ?dns= parameter\n\t\tdnsParam := r.URL.Query().Get(\"dns\")\n\t\tif dnsParam == \"\" {\n\t\t\thttp.Error(w, \"missing dns parameter\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\tdata, err := base64.RawURLEncoding.DecodeString(dnsParam)\n\t\tif err != nil {\n\t\t\thttp.Error(w, \"invalid base64\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\tdnsMsg = &dns.Msg{}\n\t\tif err := dnsMsg.Unpack(data); err != nil {\n\t\t\thttp.Error(w, \"invalid DNS message\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\t} else if r.Method == \"POST\" {\n\t\t// Handle POST with DNS wire format\n\t\tdata, err := io.ReadAll(r.Body)\n\t\tif err != nil {\n\t\t\thttp.Error(w, \"failed to read body\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\tdnsMsg = &dns.Msg{}\n\t\tif err := dnsMsg.Unpack(data); err != nil {\n\t\t\thttp.Error(w, \"invalid DNS message\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\t} else {\n\t\thttp.Error(w, \"method not allowed\", http.StatusMethodNotAllowed)\n\t\treturn\n\t}\n\n\t// Log the DNS query\n\tif len(dnsMsg.Question) > 0 {\n\t\tqname := dnsMsg.Question[0].Name\n\t\tm.requests = append(m.requests, qname)\n\t\tm.t.Logf(\"DoH server received query for: %s\", qname)\n\t}\n\n\t// Generate response\n\tresponse := m.responseFunc(dnsMsg.Question[0].Name)\n\tresponseData, err := response.Pack()\n\tif err != nil {\n\t\thttp.Error(w, \"failed to pack response\", http.StatusInternalServerError)\n\t\treturn\n\t}\n\n\tw.Header().Set(\"Content-Type\", \"application/dns-message\")\n\t_, _ = w.Write(responseData)\n}\n\nfunc (m *mockDoHServer) getRequests() []string {\n\tm.mu.Lock()\n\tdefer m.mu.Unlock()\n\treturn append([]string{}, m.requests...)\n}\n\nfunc (m *mockDoHServer) close() {\n\tm.server.Close()\n}\n\nfunc testDNSResolutionWithAutoDoH(t *testing.T) {\n\t// Create mock DoH server\n\tdohServer := newMockDoHServer(t)\n\tdefer dohServer.close()\n\n\t// Create autoconf data with DoH resolver for \"foo.\" domain\n\tautoConfData := fmt.Sprintf(`{\n\t\t\"AutoConfVersion\": 2025072302,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": []\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t\t\"DNSResolvers\": {\n\t\t\t\"foo.\": [\"%s/dns-query\"]\n\t\t},\n\t\t\"DelegatedEndpoints\": {}\n\t}`, dohServer.server.URL)\n\n\t// Create autoconf server\n\tautoConfServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(autoConfData))\n\t}))\n\tdefer autoConfServer.Close()\n\n\t// Create IPFS node with auto DNS resolver\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", autoConfServer.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\n\t// Start daemon\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Verify config still shows \"auto\" for DNS resolvers\n\tresult := node.RunIPFS(\"config\", \"DNS.Resolvers\")\n\trequire.Equal(t, 0, result.ExitCode())\n\tdnsResolversOutput := result.Stdout.String()\n\tassert.Contains(t, dnsResolversOutput, \"foo.\", \"DNS resolvers should contain foo. domain\")\n\tassert.Contains(t, dnsResolversOutput, \"auto\", \"DNS resolver config should show 'auto'\")\n\n\t// Try to resolve a .foo domain\n\tresult = node.RunIPFS(\"resolve\", \"/ipns/example.foo\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\t// Should resolve to the IPFS path from our mock DoH server\n\toutput := strings.TrimSpace(result.Stdout.String())\n\tassert.Equal(t, \"/ipfs/QmYNQJoKGNHTpPxCBPh9KkDpaExgd2duMa3aF6ytMpHdao\", output,\n\t\t\"Should resolve to the path returned by DoH server\")\n\n\t// Verify DoH server received the DNS query\n\trequests := dohServer.getRequests()\n\trequire.Greater(t, len(requests), 0, \"DoH server should have received at least one request\")\n\n\tfoundDNSLink := false\n\tfor _, req := range requests {\n\t\tif strings.Contains(req, \"_dnslink.example.foo\") {\n\t\t\tfoundDNSLink = true\n\t\t\tbreak\n\t\t}\n\t}\n\tassert.True(t, foundDNSLink, \"DoH server should have received query for _dnslink.example.foo\")\n}\n\nfunc testDNSErrorHandling(t *testing.T) {\n\t// Create DoH server that returns NXDOMAIN\n\tdohServer := newMockDoHServer(t)\n\tdefer dohServer.close()\n\n\t// Configure to return NXDOMAIN\n\tdohServer.responseFunc = func(name string) *dns.Msg {\n\t\tmsg := &dns.Msg{}\n\t\tmsg.SetReply(&dns.Msg{Question: []dns.Question{{Name: name, Qtype: dns.TypeTXT}}})\n\t\tmsg.Rcode = dns.RcodeNameError // NXDOMAIN\n\t\treturn msg\n\t}\n\n\t// Create autoconf data with DoH resolver\n\tautoConfData := fmt.Sprintf(`{\n\t\t\"AutoConfVersion\": 2025072302,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": []\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t\t\"DNSResolvers\": {\n\t\t\t\"bar.\": [\"%s/dns-query\"]\n\t\t},\n\t\t\"DelegatedEndpoints\": {}\n\t}`, dohServer.server.URL)\n\n\t// Create autoconf server\n\tautoConfServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(autoConfData))\n\t}))\n\tdefer autoConfServer.Close()\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", autoConfServer.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"bar.\": \"auto\"})\n\n\t// Start daemon\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Try to resolve a non-existent domain\n\tresult := node.RunIPFS(\"resolve\", \"/ipns/nonexistent.bar\")\n\trequire.NotEqual(t, 0, result.ExitCode(), \"Resolution should fail for non-existent domain\")\n\n\t// Should contain appropriate error message\n\tstderr := result.Stderr.String()\n\tassert.Contains(t, stderr, \"could not resolve name\",\n\t\t\"Error should indicate DNS resolution failure\")\n\n\t// Verify DoH server received the query\n\trequests := dohServer.getRequests()\n\tfoundQuery := false\n\tfor _, req := range requests {\n\t\tif strings.Contains(req, \"_dnslink.nonexistent.bar\") {\n\t\t\tfoundQuery = true\n\t\t\tbreak\n\t\t}\n\t}\n\tassert.True(t, foundQuery, \"DoH server should have received query even for failed resolution\")\n}\n" }, { "path": "test/cli/autoconf/expand_comprehensive_test.go", "content": "// Package autoconf provides comprehensive tests for --expand-auto functionality.\n//\n// Test Scenarios:\n// 1. Tests WITH daemon: Most tests start a daemon to fetch and cache autoconf data,\n// then test CLI commands that read from that cache using MustGetConfigCached.\n// 2. Tests WITHOUT daemon: Error condition tests that don't need cached autoconf.\n//\n// The daemon setup uses startDaemonAndWaitForAutoConf() helper which:\n// - Starts the daemon\n// - Waits for HTTP request to mock server (not arbitrary timeout)\n// - Returns when autoconf is cached and ready for CLI commands\npackage autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestExpandAutoComprehensive(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"all autoconf fields resolve correctly\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestAllAutoConfFieldsResolve(t)\n\t})\n\n\tt.Run(\"bootstrap list --expand-auto matches config Bootstrap --expand-auto\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestBootstrapCommandConsistency(t)\n\t})\n\n\tt.Run(\"write operations fail with --expand-auto\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestWriteOperationsFailWithExpandAuto(t)\n\t})\n\n\tt.Run(\"config show --expand-auto provides complete expanded view\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestConfigShowExpandAutoComplete(t)\n\t})\n\n\tt.Run(\"multiple expand-auto calls use cache (single HTTP request)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestMultipleExpandAutoUsesCache(t)\n\t})\n\n\tt.Run(\"CLI uses cache only while daemon handles background updates\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestCLIUsesCacheOnlyDaemonUpdatesBackground(t)\n\t})\n}\n\n// testAllAutoConfFieldsResolve verifies that all autoconf fields (Bootstrap, DNS.Resolvers,\n// Routing.DelegatedRouters, and Ipns.DelegatedPublishers) can be resolved from \"auto\" values\n// to their actual configuration using --expand-auto flag with daemon-cached autoconf data.\n//\n// This test is critical because:\n// 1. It validates the core autoconf resolution functionality across all supported fields\n// 2. It ensures that \"auto\" placeholders are properly replaced with real configuration values\n// 3. It verifies that the autoconf JSON structure is correctly parsed and applied\n// 4. It tests the end-to-end flow from HTTP fetch to config field expansion\nfunc testAllAutoConfFieldsResolve(t *testing.T) {\n\t// Test scenario: CLI with daemon started and autoconf cached\n\t// This validates core autoconf resolution functionality across all supported fields\n\n\t// Track HTTP requests to verify mock server is being used\n\tvar requestCount atomic.Int32\n\tvar autoConfData []byte\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tcount := requestCount.Add(1)\n\t\tt.Logf(\"Mock autoconf server request #%d: %s %s\", count, r.Method, r.URL.Path)\n\n\t\t// Create comprehensive autoconf response matching Schema 4 format\n\t\t// Use server URLs to ensure they're reachable and valid\n\t\tserverURL := fmt.Sprintf(\"http://%s\", r.Host) // Get the server URL from the request\n\t\tautoConf := map[string]any{\n\t\t\t\"AutoConfVersion\": 2025072301,\n\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://github.com/ipfs/specs/pull/497\",\n\t\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\", \"/routing/v1/peers\", \"/routing/v1/ipns\"},\n\t\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"IPNI\": map[string]any{\n\t\t\t\t\t\"URL\": serverURL + \"/ipni-system\",\n\t\t\t\t\t\"Description\": \"Test IPNI system\",\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"CustomIPNS\": map[string]any{\n\t\t\t\t\t\"URL\": serverURL + \"/ipns-system\",\n\t\t\t\t\t\"Description\": \"Test IPNS system\",\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"DNSResolvers\": map[string][]string{\n\t\t\t\t\".\": {\"https://cloudflare-dns.com/dns-query\"},\n\t\t\t\t\"eth.\": {\"https://dns.google/dns-query\"},\n\t\t\t},\n\t\t\t\"DelegatedEndpoints\": map[string]any{\n\t\t\t\tserverURL: map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"IPNI\", \"CustomIPNS\"}, // Use non-AminoDHT systems to avoid filtering\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\", \"/routing/v1/ipns\"},\n\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tvar err error\n\t\tautoConfData, err = json.Marshal(autoConf)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to marshal autoConf: %v\", err)\n\t\t}\n\n\t\tt.Logf(\"Serving mock autoconf data: %s\", string(autoConfData))\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", `\"test-mock-config\"`)\n\t\tw.Header().Set(\"Last-Modified\", \"Wed, 21 Oct 2015 07:28:00 GMT\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with all auto values\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Clear any existing autoconf cache to prevent interference\n\tresult := node.RunIPFS(\"config\", \"show\")\n\tif result.ExitCode() == 0 {\n\t\tvar cfg map[string]any\n\t\tif json.Unmarshal([]byte(result.Stdout.String()), &cfg) == nil {\n\t\t\tif repoPath, exists := cfg[\"path\"]; exists {\n\t\t\t\tif pathStr, ok := repoPath.(string); ok {\n\t\t\t\t\tt.Logf(\"Clearing autoconf cache from %s/autoconf\", pathStr)\n\t\t\t\t\t// Note: We can't directly remove files, but clearing cache via config change should help\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"AutoConf.RefreshInterval\", \"1s\") // Force fresh fetches for testing\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\n\t\t\".\": \"auto\",\n\t\t\"eth.\": \"auto\",\n\t})\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Start daemon and wait for autoconf fetch\n\tdaemon := startDaemonAndWaitForAutoConf(t, node, &requestCount)\n\tdefer daemon.StopDaemon()\n\n\t// Test 1: Bootstrap resolution\n\tresult = node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Bootstrap expansion should succeed\")\n\n\tvar expandedBootstrap []string\n\tvar err error\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedBootstrap)\n\trequire.NoError(t, err)\n\n\tassert.NotContains(t, expandedBootstrap, \"auto\", \"Bootstrap should not contain 'auto'\")\n\tassert.Contains(t, expandedBootstrap, \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\")\n\tassert.Contains(t, expandedBootstrap, \"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\")\n\tt.Logf(\"Bootstrap expanded to: %v\", expandedBootstrap)\n\n\t// Test 2: DNS.Resolvers resolution\n\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"DNS.Resolvers expansion should succeed\")\n\n\tvar expandedResolvers map[string]string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedResolvers)\n\trequire.NoError(t, err)\n\n\tassert.NotContains(t, expandedResolvers, \"auto\", \"DNS.Resolvers should not contain 'auto'\")\n\tassert.Equal(t, \"https://cloudflare-dns.com/dns-query\", expandedResolvers[\".\"])\n\tassert.Equal(t, \"https://dns.google/dns-query\", expandedResolvers[\"eth.\"])\n\tt.Logf(\"DNS.Resolvers expanded to: %v\", expandedResolvers)\n\n\t// Test 3: Routing.DelegatedRouters resolution\n\tresult = node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Routing.DelegatedRouters expansion should succeed\")\n\n\tvar expandedRouters []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\tassert.NotContains(t, expandedRouters, \"auto\", \"DelegatedRouters should not contain 'auto'\")\n\n\t// Test should strictly require mock autoconf to work - no fallback acceptance\n\t// The mock endpoint has Read paths [\"/routing/v1/providers\", \"/routing/v1/ipns\"]\n\t// so we expect 2 URLs with those paths\n\texpectedMockURLs := []string{\n\t\tserver.URL + \"/routing/v1/providers\",\n\t\tserver.URL + \"/routing/v1/ipns\",\n\t}\n\trequire.Equal(t, 2, len(expandedRouters),\n\t\t\"Should have exactly 2 routers from mock autoconf (one for each Read path). Got %d routers: %v. \"+\n\t\t\t\"This indicates autoconf is not working properly - check if mock server data is being parsed and filtered correctly.\",\n\t\tlen(expandedRouters), expandedRouters)\n\n\t// Check that both expected URLs are present\n\tfor _, expectedURL := range expectedMockURLs {\n\t\tassert.Contains(t, expandedRouters, expectedURL,\n\t\t\t\"Should contain mock autoconf endpoint with path %s. Got: %v. \"+\n\t\t\t\t\"This indicates autoconf endpoint path generation is not working properly.\",\n\t\t\texpectedURL, expandedRouters)\n\t}\n\n\t// Test 4: Ipns.DelegatedPublishers resolution\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Ipns.DelegatedPublishers expansion should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\tassert.NotContains(t, expandedPublishers, \"auto\", \"DelegatedPublishers should not contain 'auto'\")\n\n\t// Test should require mock autoconf endpoint for IPNS publishing\n\t// The mock endpoint supports /routing/v1/ipns write operations, so it should be included with path\n\texpectedMockPublisherURL := server.URL + \"/routing/v1/ipns\"\n\trequire.Equal(t, 1, len(expandedPublishers),\n\t\t\"Should have exactly 1 IPNS publisher from mock autoconf. Got %d publishers: %v. \"+\n\t\t\t\"This indicates autoconf IPNS publisher filtering is not working properly.\",\n\t\tlen(expandedPublishers), expandedPublishers)\n\tassert.Equal(t, expectedMockPublisherURL, expandedPublishers[0],\n\t\t\"Should use mock autoconf endpoint %s for IPNS publishing, not fallback. Got: %s. \"+\n\t\t\t\"This indicates autoconf IPNS publisher resolution is not working properly.\",\n\t\texpectedMockPublisherURL, expandedPublishers[0])\n\n\t// CRITICAL: Verify that mock server was actually used\n\tfinalRequestCount := requestCount.Load()\n\trequire.Greater(t, finalRequestCount, int32(0),\n\t\t\"Mock autoconf server should have been called at least once. Got %d requests. \"+\n\t\t\t\"This indicates the test is using cached or fallback config instead of mock data.\", finalRequestCount)\n\tt.Logf(\"Mock server was called %d times - test is using mock data\", finalRequestCount)\n}\n\n// testBootstrapCommandConsistency verifies that `ipfs bootstrap list --expand-auto` and\n// `ipfs config Bootstrap --expand-auto` return identical results when both use autoconf.\n//\n// This test is important because:\n// 1. It ensures consistency between different CLI commands that access the same data\n// 2. It validates that both the bootstrap-specific command and generic config command\n// use the same underlying autoconf resolution mechanism\n// 3. It prevents regression where different commands might resolve \"auto\" differently\n// 4. It ensures users get consistent results regardless of which command they use\nfunc testBootstrapCommandConsistency(t *testing.T) {\n\t// Test scenario: CLI with daemon started and autoconf cached\n\t// This ensures both bootstrap commands read from the same cached autoconf data\n\n\t// Load test autoconf data\n\tautoConfData := loadTestDataComprehensive(t, \"valid_autoconf.json\")\n\n\t// Track HTTP requests to verify daemon fetches autoconf\n\tvar requestCount atomic.Int32\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\trequestCount.Add(1)\n\t\tt.Logf(\"Bootstrap consistency test request: %s %s\", r.Method, r.URL.Path)\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with auto bootstrap\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Start daemon and wait for autoconf fetch\n\tdaemon := startDaemonAndWaitForAutoConf(t, node, &requestCount)\n\tdefer daemon.StopDaemon()\n\n\t// Get bootstrap via config command\n\tconfigResult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, configResult.ExitCode(), \"config Bootstrap --expand-auto should succeed\")\n\n\t// Get bootstrap via bootstrap command\n\tbootstrapResult := node.RunIPFS(\"bootstrap\", \"list\", \"--expand-auto\")\n\trequire.Equal(t, 0, bootstrapResult.ExitCode(), \"bootstrap list --expand-auto should succeed\")\n\n\t// Parse both results\n\tvar configBootstrap, bootstrapBootstrap []string\n\terr := json.Unmarshal([]byte(configResult.Stdout.String()), &configBootstrap)\n\trequire.NoError(t, err)\n\n\t// Bootstrap command output is line-separated, not JSON\n\tbootstrapOutput := strings.TrimSpace(bootstrapResult.Stdout.String())\n\tif bootstrapOutput != \"\" {\n\t\tbootstrapBootstrap = strings.Split(bootstrapOutput, \"\\n\")\n\t}\n\n\t// Results should be equivalent\n\tassert.Equal(t, len(configBootstrap), len(bootstrapBootstrap), \"Both commands should return same number of peers\")\n\n\t// Both should contain same peers (order might differ due to different output formats)\n\tfor _, peer := range configBootstrap {\n\t\tfound := false\n\t\tfor _, bsPeer := range bootstrapBootstrap {\n\t\t\tif strings.TrimSpace(bsPeer) == peer {\n\t\t\t\tfound = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tassert.True(t, found, \"Peer %s should be in both results\", peer)\n\t}\n\n\tt.Logf(\"Config command result: %v\", configBootstrap)\n\tt.Logf(\"Bootstrap command result: %v\", bootstrapBootstrap)\n}\n\n// testWriteOperationsFailWithExpandAuto verifies that --expand-auto flag is properly\n// restricted to read-only operations and fails when used with config write operations.\n//\n// This test is essential because:\n// 1. It enforces the security principle that --expand-auto should only be used for reading\n// 2. It prevents users from accidentally overwriting config with expanded values\n// 3. It ensures that \"auto\" placeholders are preserved in the stored configuration\n// 4. It validates proper error handling and user guidance when misused\n// 5. It protects against accidental loss of the \"auto\" semantic meaning\nfunc testWriteOperationsFailWithExpandAuto(t *testing.T) {\n\t// Test scenario: CLI without daemon (tests error conditions)\n\t// This test doesn't need daemon setup since it's testing that write operations\n\t// with --expand-auto should fail with appropriate error messages\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Test that setting config with --expand-auto fails\n\ttestCases := []struct {\n\t\tname string\n\t\targs []string\n\t}{\n\t\t{\"config set with expand-auto\", []string{\"config\", \"Bootstrap\", \"[\\\"test\\\"]\", \"--expand-auto\"}},\n\t\t{\"config set JSON with expand-auto\", []string{\"config\", \"Bootstrap\", \"[\\\"test\\\"]\", \"--json\", \"--expand-auto\"}},\n\t\t{\"config set bool with expand-auto\", []string{\"config\", \"SomeField\", \"true\", \"--bool\", \"--expand-auto\"}},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := node.RunIPFS(tc.args...)\n\t\t\tassert.NotEqual(t, 0, result.ExitCode(), \"Write operation with --expand-auto should fail\")\n\n\t\t\tstderr := result.Stderr.String()\n\t\t\tassert.Contains(t, stderr, \"--expand-auto\", \"Error should mention --expand-auto\")\n\t\t\tassert.Contains(t, stderr, \"reading\", \"Error should mention reading limitation\")\n\t\t\tt.Logf(\"Expected error: %s\", stderr)\n\t\t})\n\t}\n}\n\n// testConfigShowExpandAutoComplete verifies that `ipfs config show --expand-auto`\n// produces a complete configuration with all \"auto\" values expanded to their resolved forms.\n//\n// This test is important because:\n// 1. It validates the full-config expansion functionality for comprehensive troubleshooting\n// 2. It ensures that users can see the complete resolved configuration state\n// 3. It verifies that all \"auto\" placeholders are replaced, not just individual fields\n// 4. It tests that the resulting JSON is valid and well-formed\n// 5. It provides a way to export/backup the fully expanded configuration\nfunc testConfigShowExpandAutoComplete(t *testing.T) {\n\t// Test scenario: CLI with daemon started and autoconf cached\n\n\t// Load test autoconf data\n\tautoConfData := loadTestDataComprehensive(t, \"valid_autoconf.json\")\n\n\t// Track HTTP requests to verify daemon fetches autoconf\n\tvar requestCount atomic.Int32\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\trequestCount.Add(1)\n\t\tt.Logf(\"Config show test request: %s %s\", r.Method, r.URL.Path)\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with multiple auto values\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\".\": \"auto\"})\n\n\t// Start daemon and wait for autoconf fetch\n\tdaemon := startDaemonAndWaitForAutoConf(t, node, &requestCount)\n\tdefer daemon.StopDaemon()\n\n\t// Test config show --expand-auto\n\tresult := node.RunIPFS(\"config\", \"show\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config show --expand-auto should succeed\")\n\n\texpandedConfig := result.Stdout.String()\n\n\t// Should not contain any literal \"auto\" values\n\tassert.NotContains(t, expandedConfig, `\"auto\"`, \"Expanded config should not contain literal 'auto' values\")\n\n\t// Should contain expected expanded sections\n\tassert.Contains(t, expandedConfig, `\"Bootstrap\"`, \"Should contain Bootstrap section\")\n\tassert.Contains(t, expandedConfig, `\"DNS\"`, \"Should contain DNS section\")\n\tassert.Contains(t, expandedConfig, `\"Resolvers\"`, \"Should contain Resolvers section\")\n\n\t// Should contain expanded peer addresses (not \"auto\")\n\tassert.Contains(t, expandedConfig, \"bootstrap.libp2p.io\", \"Should contain expanded bootstrap peers\")\n\n\t// Should be valid JSON\n\tvar configMap map[string]any\n\terr := json.Unmarshal([]byte(expandedConfig), &configMap)\n\trequire.NoError(t, err, \"Expanded config should be valid JSON\")\n\n\t// Verify specific fields were expanded\n\tif bootstrap, ok := configMap[\"Bootstrap\"].([]any); ok {\n\t\tassert.Greater(t, len(bootstrap), 0, \"Bootstrap should have expanded entries\")\n\t\tfor _, peer := range bootstrap {\n\t\t\tassert.NotEqual(t, \"auto\", peer, \"Bootstrap entries should not be 'auto'\")\n\t\t}\n\t}\n\n\tt.Logf(\"Config show --expand-auto produced %d characters of expanded config\", len(expandedConfig))\n}\n\n// testMultipleExpandAutoUsesCache verifies that multiple consecutive --expand-auto calls\n// efficiently use cached autoconf data instead of making repeated HTTP requests.\n//\n// This test is critical for performance because:\n// 1. It validates that the caching mechanism works correctly to reduce network overhead\n// 2. It ensures that users can make multiple config queries without causing excessive HTTP traffic\n// 3. It verifies that cached data is shared across different config fields and commands\n// 4. It tests that HTTP headers (ETag/Last-Modified) are properly used for cache validation\n// 5. It prevents regression where each --expand-auto call would trigger a new HTTP request\n// 6. It demonstrates the performance benefit: 5 operations with only 1 network request\nfunc testMultipleExpandAutoUsesCache(t *testing.T) {\n\t// Test scenario: CLI with daemon started and autoconf cached\n\n\t// Create comprehensive autoconf response\n\tautoConfData := loadTestDataComprehensive(t, \"valid_autoconf.json\")\n\n\t// Track HTTP requests to verify caching\n\tvar requestCount atomic.Int32\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tcount := requestCount.Add(1)\n\t\tt.Logf(\"AutoConf cache test request #%d: %s %s\", count, r.Method, r.URL.Path)\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", `\"cache-test-123\"`)\n\t\tw.Header().Set(\"Last-Modified\", \"Wed, 21 Oct 2015 07:28:00 GMT\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with all auto values\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t// Note: Using default RefreshInterval (24h) to ensure caching - explicit setting would require rebuilt binary\n\n\t// Set up auto values for multiple fields\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Start daemon and wait for autoconf fetch\n\tdaemon := startDaemonAndWaitForAutoConf(t, node, &requestCount)\n\tdefer daemon.StopDaemon()\n\n\t// Reset counter to only track our expand-auto calls\n\trequestCount.Store(0)\n\n\t// Make multiple --expand-auto calls on different fields\n\tt.Log(\"Testing multiple --expand-auto calls should use cache...\")\n\n\t// Call 1: Bootstrap --expand-auto (should trigger HTTP request)\n\tresult1 := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result1.ExitCode(), \"Bootstrap --expand-auto should succeed\")\n\n\tvar expandedBootstrap []string\n\terr := json.Unmarshal([]byte(result1.Stdout.String()), &expandedBootstrap)\n\trequire.NoError(t, err)\n\tassert.NotContains(t, expandedBootstrap, \"auto\", \"Bootstrap should be expanded\")\n\tassert.Greater(t, len(expandedBootstrap), 0, \"Bootstrap should have entries\")\n\n\t// Call 2: DNS.Resolvers --expand-auto (should use cache, no HTTP)\n\tresult2 := node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result2.ExitCode(), \"DNS.Resolvers --expand-auto should succeed\")\n\n\tvar expandedResolvers map[string]string\n\terr = json.Unmarshal([]byte(result2.Stdout.String()), &expandedResolvers)\n\trequire.NoError(t, err)\n\n\t// Call 3: Routing.DelegatedRouters --expand-auto (should use cache, no HTTP)\n\tresult3 := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result3.ExitCode(), \"Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr = json.Unmarshal([]byte(result3.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\tassert.NotContains(t, expandedRouters, \"auto\", \"Routers should be expanded\")\n\n\t// Call 4: Ipns.DelegatedPublishers --expand-auto (should use cache, no HTTP)\n\tresult4 := node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result4.ExitCode(), \"Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result4.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\tassert.NotContains(t, expandedPublishers, \"auto\", \"Publishers should be expanded\")\n\n\t// Call 5: config show --expand-auto (should use cache, no HTTP)\n\tresult5 := node.RunIPFS(\"config\", \"show\", \"--expand-auto\")\n\trequire.Equal(t, 0, result5.ExitCode(), \"config show --expand-auto should succeed\")\n\n\texpandedConfig := result5.Stdout.String()\n\tassert.NotContains(t, expandedConfig, `\"auto\"`, \"Full config should not contain 'auto' values\")\n\n\t// CRITICAL TEST: Verify NO HTTP requests were made for --expand-auto calls (using cache)\n\tfinalRequestCount := requestCount.Load()\n\tassert.Equal(t, int32(0), finalRequestCount,\n\t\t\"Multiple --expand-auto calls should result in 0 HTTP requests (using cache). Got %d requests\", finalRequestCount)\n\n\tt.Logf(\"Made 5 --expand-auto calls, resulted in %d HTTP request(s) - cache is being used!\", finalRequestCount)\n\n\t// Now simulate a manual cache refresh (what the background updater would do)\n\tt.Log(\"Simulating manual cache refresh...\")\n\n\t// Update the mock server to return different data\n\tautoConfData2 := loadTestDataComprehensive(t, \"updated_autoconf.json\")\n\tserver.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tcount := requestCount.Add(1)\n\t\tt.Logf(\"Manual refresh request #%d: %s %s\", count, r.Method, r.URL.Path)\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", `\"cache-test-456\"`)\n\t\tw.Header().Set(\"Last-Modified\", \"Thu, 22 Oct 2015 08:00:00 GMT\")\n\t\t_, _ = w.Write(autoConfData2)\n\t})\n\n\t// Note: In the actual daemon, the background updater would call MustGetConfigWithRefresh\n\t// For this test, we'll verify that subsequent --expand-auto calls still use cache\n\t// and don't trigger additional requests\n\n\t// Reset counter before manual refresh simulation\n\tbeforeRefresh := requestCount.Load()\n\n\t// Make another --expand-auto call - should still use cache\n\tresult6 := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result6.ExitCode(), \"Bootstrap --expand-auto after refresh should succeed\")\n\n\tafterRefresh := requestCount.Load()\n\tassert.Equal(t, beforeRefresh, afterRefresh,\n\t\t\"--expand-auto should continue using cache even after server update\")\n\n\tt.Logf(\"Cache continues to be used after server update - background updater pattern confirmed!\")\n}\n\n// testCLIUsesCacheOnlyDaemonUpdatesBackground verifies the correct autoconf behavior:\n// daemon makes exactly one HTTP request during startup to fetch and cache data, then\n// CLI commands always use cached data without making additional HTTP requests.\n//\n// This test is essential for correctness because:\n// 1. It validates that daemon startup makes exactly one HTTP request to fetch autoconf\n// 2. It verifies that CLI --expand-auto never makes HTTP requests (uses cache only)\n// 3. It ensures CLI commands remain fast by always using cached data\n// 4. It prevents regression where CLI commands might start making HTTP requests\n// 5. It confirms the correct separation between daemon (network) and CLI (cache-only) behavior\nfunc testCLIUsesCacheOnlyDaemonUpdatesBackground(t *testing.T) {\n\t// Test scenario: CLI with daemon and long RefreshInterval (no background updates during test)\n\n\t// Create autoconf response\n\tautoConfData := loadTestDataComprehensive(t, \"valid_autoconf.json\")\n\n\t// Track HTTP requests with timestamps\n\tvar requestCount atomic.Int32\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tcount := requestCount.Add(1)\n\t\tt.Logf(\"Cache expiry test request #%d at %s: %s %s\", count, time.Now().Format(\"15:04:05.000\"), r.Method, r.URL.Path)\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t// Use different ETag for each request to ensure we can detect new fetches\n\t\tw.Header().Set(\"ETag\", fmt.Sprintf(`\"expiry-test-%d\"`, count))\n\t\tw.Header().Set(\"Last-Modified\", time.Now().Format(http.TimeFormat))\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with long refresh interval\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t// Set long RefreshInterval to avoid background updates during test\n\tnode.SetIPFSConfig(\"AutoConf.RefreshInterval\", \"1h\")\n\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"test.\": \"auto\"})\n\n\t// Start daemon and wait for autoconf fetch\n\tdaemon := startDaemonAndWaitForAutoConf(t, node, &requestCount)\n\tdefer daemon.StopDaemon()\n\n\t// Confirm only one request was made during daemon startup\n\tinitialRequestCount := requestCount.Load()\n\tassert.Equal(t, int32(1), initialRequestCount, \"Expected exactly 1 HTTP request during daemon startup, got: %d\", initialRequestCount)\n\tt.Logf(\"Daemon startup made exactly 1 HTTP request\")\n\n\t// Test: CLI commands use cache only (no additional HTTP requests)\n\tt.Log(\"Testing that CLI --expand-auto commands use cache only...\")\n\n\t// Make several CLI calls - none should trigger HTTP requests\n\tresult1 := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result1.ExitCode(), \"Bootstrap --expand-auto should succeed\")\n\n\tresult2 := node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result2.ExitCode(), \"DNS.Resolvers --expand-auto should succeed\")\n\n\tresult3 := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result3.ExitCode(), \"Routing.DelegatedRouters --expand-auto should succeed\")\n\n\t// Verify the request count remains at 1 (no additional requests from CLI)\n\tfinalRequestCount := requestCount.Load()\n\tassert.Equal(t, int32(1), finalRequestCount, \"Request count should remain at 1 after CLI commands, got: %d\", finalRequestCount)\n\tt.Log(\"CLI commands use cache only - request count remains at 1\")\n\n\tt.Log(\"Test completed: Daemon makes 1 startup request, CLI commands use cache only\")\n}\n\n// loadTestDataComprehensive is a helper function that loads test autoconf JSON data files.\n// It locates the test data directory relative to the test file and reads the specified file.\n// This centralized helper ensures consistent test data loading across all comprehensive tests.\nfunc loadTestDataComprehensive(t *testing.T, filename string) []byte {\n\tt.Helper()\n\n\tdata, err := os.ReadFile(\"testdata/\" + filename)\n\trequire.NoError(t, err, \"Failed to read test data file: %s\", filename)\n\n\treturn data\n}\n\n// startDaemonAndWaitForAutoConf starts a daemon and waits for it to fetch autoconf data.\n// It returns the node with daemon running and ensures autoconf has been cached before returning.\n// This is a DRY helper to avoid repeating daemon setup and request waiting logic in every test.\nfunc startDaemonAndWaitForAutoConf(t *testing.T, node *harness.Node, requestCount *atomic.Int32) *harness.Node {\n\tt.Helper()\n\n\t// Start daemon to fetch and cache autoconf data\n\tt.Log(\"Starting daemon to fetch and cache autoconf data...\")\n\tdaemon := node.StartDaemon()\n\t// StartDaemon returns *Node, no error to check\n\n\t// Wait for daemon to fetch autoconf (wait for HTTP request to mock server)\n\tt.Log(\"Waiting for daemon to fetch autoconf from mock server...\")\n\ttimeout := time.After(10 * time.Second) // Safety timeout\n\tticker := time.NewTicker(10 * time.Millisecond)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-timeout:\n\t\t\tt.Fatal(\"Timeout waiting for autoconf fetch\")\n\t\tcase <-ticker.C:\n\t\t\tif requestCount.Load() > 0 {\n\t\t\t\tt.Logf(\"Daemon fetched autoconf (%d requests made)\", requestCount.Load())\n\t\t\t\tt.Log(\"AutoConf should now be cached by daemon\")\n\t\t\t\treturn daemon\n\t\t\t}\n\t\t}\n\t}\n}\n" }, { "path": "test/cli/autoconf/expand_fallback_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"slices\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestExpandAutoFallbacks(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"expand-auto with unreachable server shows fallbacks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithUnreachableServer(t)\n\t})\n\n\tt.Run(\"expand-auto with disabled autoconf shows error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithDisabledAutoConf(t)\n\t})\n\n\tt.Run(\"expand-auto with malformed response shows fallbacks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithMalformedResponse(t)\n\t})\n\n\tt.Run(\"expand-auto preserves static values in mixed config\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoMixedConfigPreservesStatic(t)\n\t})\n\n\tt.Run(\"daemon gracefully handles malformed autoconf and uses fallbacks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestDaemonWithMalformedAutoConf(t)\n\t})\n}\n\nfunc testExpandAutoWithUnreachableServer(t *testing.T) {\n\t// Create IPFS node with unreachable AutoConf server\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", \"http://127.0.0.1:99999/nonexistent\") // Unreachable\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\n\t// Test that --expand-auto falls back to defaults when server is unreachable\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Bootstrap --expand-auto should succeed even with unreachable server\")\n\n\tvar bootstrap []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\n\t// Should contain fallback bootstrap peers (not \"auto\" and not empty)\n\tassert.NotContains(t, bootstrap, \"auto\", \"Fallback bootstrap should not contain 'auto'\")\n\tassert.Greater(t, len(bootstrap), 0, \"Fallback bootstrap should not be empty\")\n\n\t// Should contain known default bootstrap peers\n\tfoundDefaultPeer := false\n\tfor _, peer := range bootstrap {\n\t\tif peer != \"\" && peer != \"auto\" {\n\t\t\tfoundDefaultPeer = true\n\t\t\tt.Logf(\"Found fallback bootstrap peer: %s\", peer)\n\t\t\tbreak\n\t\t}\n\t}\n\tassert.True(t, foundDefaultPeer, \"Should contain at least one fallback bootstrap peer\")\n\n\t// Test DNS resolvers fallback\n\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config DNS.Resolvers --expand-auto should succeed with unreachable server\")\n\n\tvar resolvers map[string]string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &resolvers)\n\trequire.NoError(t, err)\n\n\t// When autoconf server is unreachable, DNS resolvers should fall back to defaults\n\t// The \"foo.\" resolver should not exist in fallbacks (only \"eth.\" has fallback)\n\tfooResolver, fooExists := resolvers[\"foo.\"]\n\n\tif !fooExists {\n\t\tt.Log(\"DNS resolver for 'foo.' has no fallback - correct behavior (only eth. has fallbacks)\")\n\t} else {\n\t\tassert.NotEqual(t, \"auto\", fooResolver, \"DNS resolver should not be 'auto' after expansion\")\n\t\tt.Logf(\"Unexpected DNS resolver for foo.: %s\", fooResolver)\n\t}\n}\n\nfunc testExpandAutoWithDisabledAutoConf(t *testing.T) {\n\t// Create IPFS node with AutoConf disabled\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", false)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Test that --expand-auto with disabled AutoConf returns appropriate error or fallback\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\n\t// When AutoConf is disabled, expand-auto should show empty results\n\t// since \"auto\" values are not expanded when AutoConf.Enabled=false\n\tvar bootstrap []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\n\t// With AutoConf disabled, \"auto\" values are not expanded so we get empty result\n\tassert.NotContains(t, bootstrap, \"auto\", \"Should not contain 'auto' after expansion\")\n\tassert.Equal(t, 0, len(bootstrap), \"Should be empty when AutoConf disabled (auto values not expanded)\")\n\tt.Log(\"Bootstrap is empty when AutoConf disabled - correct behavior\")\n}\n\nfunc testExpandAutoWithMalformedResponse(t *testing.T) {\n\t// Create server that returns malformed JSON\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(`{\"invalid\": \"json\", \"Bootstrap\": [incomplete`)) // Malformed JSON\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with malformed autoconf server\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Test that --expand-auto handles malformed response gracefully\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Bootstrap --expand-auto should succeed even with malformed response\")\n\n\tvar bootstrap []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\n\t// Should fall back to defaults, not contain \"auto\"\n\tassert.NotContains(t, bootstrap, \"auto\", \"Should not contain 'auto' after fallback\")\n\tassert.Greater(t, len(bootstrap), 0, \"Should contain fallback peers after malformed response\")\n\tt.Logf(\"Bootstrap after malformed response: %v\", bootstrap)\n}\n\nfunc testExpandAutoMixedConfigPreservesStatic(t *testing.T) {\n\t// Load valid test autoconf data\n\tautoConfData := loadTestDataForFallback(t, \"valid_autoconf.json\")\n\n\t// Create HTTP server that serves autoconf.json\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with mixed auto and static values\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Set mixed configuration: static + auto + static\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\n\t\t\"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWTest\",\n\t\t\"auto\",\n\t\t\"/ip4/127.0.0.2/tcp/4001/p2p/12D3KooWTest2\",\n\t})\n\n\t// Test that --expand-auto only expands \"auto\" values, preserves static ones\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Bootstrap --expand-auto should succeed\")\n\n\tvar bootstrap []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\n\t// Should not contain literal \"auto\" anymore\n\tassert.NotContains(t, bootstrap, \"auto\", \"Expanded config should not contain literal 'auto'\")\n\n\t// Should preserve static values at original positions\n\tassert.Contains(t, bootstrap, \"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWTest\", \"Should preserve first static peer\")\n\tassert.Contains(t, bootstrap, \"/ip4/127.0.0.2/tcp/4001/p2p/12D3KooWTest2\", \"Should preserve third static peer\")\n\n\t// Should have more entries than just the static ones (auto got expanded)\n\tassert.Greater(t, len(bootstrap), 2, \"Should have more than just the 2 static peers\")\n\n\tt.Logf(\"Mixed config expansion result: %v\", bootstrap)\n\n\t// Verify order is preserved: static, expanded auto values, static\n\tassert.Equal(t, \"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWTest\", bootstrap[0], \"First peer should be preserved\")\n\tlastIndex := len(bootstrap) - 1\n\tassert.Equal(t, \"/ip4/127.0.0.2/tcp/4001/p2p/12D3KooWTest2\", bootstrap[lastIndex], \"Last peer should be preserved\")\n}\n\nfunc testDaemonWithMalformedAutoConf(t *testing.T) {\n\t// Test scenario: Daemon starts with AutoConf.URL pointing to server that returns malformed JSON\n\t// This tests that daemon gracefully handles malformed responses and falls back to hardcoded defaults\n\n\t// Create server that returns malformed JSON to simulate broken autoconf service\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t// Return malformed JSON that cannot be parsed\n\t\t_, _ = w.Write([]byte(`{\"Bootstrap\": [\"incomplete array\", \"missing closing bracket\"`))\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node with autoconf pointing to malformed server\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\n\t// Start daemon - this will attempt to fetch autoconf from malformed server\n\tt.Log(\"Starting daemon with malformed autoconf server...\")\n\tdaemon := node.StartDaemon()\n\tdefer daemon.StopDaemon()\n\n\t// Wait for daemon to attempt autoconf fetch and handle the error gracefully\n\ttime.Sleep(6 * time.Second) // defaultTimeout is 5s, add 1s buffer\n\tt.Log(\"Daemon should have attempted autoconf fetch and fallen back to defaults\")\n\n\t// Test that daemon is still running and CLI commands work with fallback values\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Bootstrap --expand-auto should succeed with daemon running\")\n\n\tvar bootstrap []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\n\t// Should fall back to hardcoded defaults from GetMainnetFallbackConfig()\n\t// NOTE: These values may change if autoconf library updates GetMainnetFallbackConfig()\n\tassert.NotContains(t, bootstrap, \"auto\", \"Should not contain 'auto' after fallback\")\n\tassert.Greater(t, len(bootstrap), 0, \"Should contain fallback bootstrap peers\")\n\n\t// Verify we got actual fallback bootstrap peers from GetMainnetFallbackConfig() AminoDHT NativeConfig\n\tfallbackConfig := autoconf.GetMainnetFallbackConfig()\n\taminoDHTSystem := fallbackConfig.SystemRegistry[\"AminoDHT\"]\n\texpectedBootstrapPeers := aminoDHTSystem.NativeConfig.Bootstrap\n\n\tfoundFallbackPeers := 0\n\tfor _, expectedPeer := range expectedBootstrapPeers {\n\t\tif slices.Contains(bootstrap, expectedPeer) {\n\t\t\tfoundFallbackPeers++\n\t\t}\n\t}\n\tassert.Greater(t, foundFallbackPeers, 0, \"Should contain bootstrap peers from GetMainnetFallbackConfig() AminoDHT NativeConfig\")\n\tassert.Equal(t, len(expectedBootstrapPeers), foundFallbackPeers, \"Should contain all bootstrap peers from GetMainnetFallbackConfig() AminoDHT NativeConfig\")\n\n\tt.Logf(\"Daemon fallback bootstrap peers after malformed response: %v\", bootstrap)\n\n\t// Test DNS resolvers also fall back correctly\n\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config DNS.Resolvers --expand-auto should succeed with daemon running\")\n\n\tvar resolvers map[string]string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &resolvers)\n\trequire.NoError(t, err)\n\n\t// Should not contain \"auto\" and should have fallback DNS resolvers\n\tassert.NotEqual(t, \"auto\", resolvers[\"foo.\"], \"DNS resolver should not be 'auto' after fallback\")\n\tif resolvers[\"foo.\"] != \"\" {\n\t\t// If resolver is populated, it should be a valid URL from fallbacks\n\t\tassert.Contains(t, resolvers[\"foo.\"], \"https://\", \"Fallback DNS resolver should be HTTPS URL\")\n\t}\n\n\tt.Logf(\"Daemon fallback DNS resolvers after malformed response: %v\", resolvers)\n\n\t// Verify daemon is still healthy and responsive\n\tversionResult := node.RunIPFS(\"version\")\n\trequire.Equal(t, 0, versionResult.ExitCode(), \"daemon should remain healthy after handling malformed autoconf\")\n\tt.Log(\"Daemon remains healthy after gracefully handling malformed autoconf response\")\n}\n\n// Helper function to load test data files for fallback tests\nfunc loadTestDataForFallback(t *testing.T, filename string) []byte {\n\tt.Helper()\n\n\tdata, err := os.ReadFile(\"testdata/\" + filename)\n\trequire.NoError(t, err, \"Failed to read test data file: %s\", filename)\n\n\treturn data\n}\n" }, { "path": "test/cli/autoconf/expand_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAutoConfExpand(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"config commands show auto values\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestConfigCommandsShowAutoValues(t)\n\t})\n\n\tt.Run(\"mixed configuration preserves both auto and static\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestMixedConfigurationPreserved(t)\n\t})\n\n\tt.Run(\"config replace preserves auto values\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestConfigReplacePreservesAuto(t)\n\t})\n\n\tt.Run(\"expand-auto filters unsupported URL paths with delegated routing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoFiltersUnsupportedPathsDelegated(t)\n\t})\n\n\tt.Run(\"expand-auto with auto routing uses NewRoutingSystem\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithAutoRouting(t)\n\t})\n\n\tt.Run(\"expand-auto with auto routing shows AminoDHT native vs IPNI delegated\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithMixedSystems(t)\n\t})\n\n\tt.Run(\"expand-auto filters paths with NewRoutingSystem and auto routing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithFiltering(t)\n\t})\n\n\tt.Run(\"expand-auto falls back to defaults without cache (delegated)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithoutCacheDelegated(t)\n\t})\n\n\tt.Run(\"expand-auto with auto routing without cache\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestExpandAutoWithoutCacheAuto(t)\n\t})\n}\n\nfunc testConfigCommandsShowAutoValues(t *testing.T) {\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Set all fields to \"auto\"\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Test individual field queries\n\tt.Run(\"Bootstrap shows auto\", func(t *testing.T) {\n\t\tresult := node.RunIPFS(\"config\", \"Bootstrap\")\n\t\trequire.Equal(t, 0, result.ExitCode())\n\n\t\tvar bootstrap []string\n\t\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, []string{\"auto\"}, bootstrap)\n\t})\n\n\tt.Run(\"DNS.Resolvers shows auto\", func(t *testing.T) {\n\t\tresult := node.RunIPFS(\"config\", \"DNS.Resolvers\")\n\t\trequire.Equal(t, 0, result.ExitCode())\n\n\t\tvar resolvers map[string]string\n\t\terr := json.Unmarshal([]byte(result.Stdout.String()), &resolvers)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, map[string]string{\"foo.\": \"auto\"}, resolvers)\n\t})\n\n\tt.Run(\"Routing.DelegatedRouters shows auto\", func(t *testing.T) {\n\t\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\")\n\t\trequire.Equal(t, 0, result.ExitCode())\n\n\t\tvar routers []string\n\t\terr := json.Unmarshal([]byte(result.Stdout.String()), &routers)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, []string{\"auto\"}, routers)\n\t})\n\n\tt.Run(\"Ipns.DelegatedPublishers shows auto\", func(t *testing.T) {\n\t\tresult := node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\")\n\t\trequire.Equal(t, 0, result.ExitCode())\n\n\t\tvar publishers []string\n\t\terr := json.Unmarshal([]byte(result.Stdout.String()), &publishers)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, []string{\"auto\"}, publishers)\n\t})\n\n\tt.Run(\"config show contains all auto values\", func(t *testing.T) {\n\t\tresult := node.RunIPFS(\"config\", \"show\")\n\t\trequire.Equal(t, 0, result.ExitCode())\n\n\t\toutput := result.Stdout.String()\n\n\t\t// Check that auto values are present in the full config\n\t\tassert.Contains(t, output, `\"Bootstrap\": [\n \"auto\"\n ]`, \"Bootstrap should contain auto\")\n\n\t\tassert.Contains(t, output, `\"DNS\": {\n \"Resolvers\": {\n \"foo.\": \"auto\"\n }\n }`, \"DNS.Resolvers should contain auto\")\n\n\t\tassert.Contains(t, output, `\"DelegatedRouters\": [\n \"auto\"\n ]`, \"Routing.DelegatedRouters should contain auto\")\n\n\t\tassert.Contains(t, output, `\"DelegatedPublishers\": [\n \"auto\"\n ]`, \"Ipns.DelegatedPublishers should contain auto\")\n\t})\n\n\t// Test with autoconf server for --expand-auto functionality\n\tt.Run(\"config with --expand-auto expands auto values\", func(t *testing.T) {\n\t\t// Load test autoconf data\n\t\tautoConfData := loadTestDataExpand(t, \"valid_autoconf.json\")\n\n\t\t// Create HTTP server that serves autoconf.json\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t_, _ = w.Write(autoConfData)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\t// Configure autoconf for the node\n\t\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t\t// Test Bootstrap field expansion\n\t\tresult := node.RunIPFS(\"config\", \"Bootstrap\", \"--expand-auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"config Bootstrap --expand-auto should succeed\")\n\n\t\tvar expandedBootstrap []string\n\t\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedBootstrap)\n\t\trequire.NoError(t, err)\n\t\tassert.NotContains(t, expandedBootstrap, \"auto\", \"Expanded bootstrap should not contain 'auto'\")\n\t\tassert.Greater(t, len(expandedBootstrap), 0, \"Expanded bootstrap should contain expanded peers\")\n\n\t\t// Test DNS.Resolvers field expansion\n\t\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\", \"--expand-auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"config DNS.Resolvers --expand-auto should succeed\")\n\n\t\tvar expandedResolvers map[string]string\n\t\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedResolvers)\n\t\trequire.NoError(t, err)\n\t\tassert.NotEqual(t, \"auto\", expandedResolvers[\"foo.\"], \"Expanded DNS resolver should not be 'auto'\")\n\n\t\t// Test Routing.DelegatedRouters field expansion\n\t\tresult = node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\t\tvar expandedRouters []string\n\t\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\t\trequire.NoError(t, err)\n\t\tassert.NotContains(t, expandedRouters, \"auto\", \"Expanded routers should not contain 'auto'\")\n\n\t\t// Test Ipns.DelegatedPublishers field expansion\n\t\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\t\tvar expandedPublishers []string\n\t\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\t\trequire.NoError(t, err)\n\t\tassert.NotContains(t, expandedPublishers, \"auto\", \"Expanded publishers should not contain 'auto'\")\n\n\t\t// Test config show --expand-auto (full config expansion)\n\t\tresult = node.RunIPFS(\"config\", \"show\", \"--expand-auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"config show --expand-auto should succeed\")\n\n\t\texpandedOutput := result.Stdout.String()\n\t\tt.Logf(\"Expanded config output contains: %d characters\", len(expandedOutput))\n\n\t\t// Verify that auto values are expanded in the full config\n\t\tassert.NotContains(t, expandedOutput, `\"auto\"`, \"Expanded config should not contain literal 'auto' values\")\n\t\tassert.Contains(t, expandedOutput, `\"Bootstrap\"`, \"Expanded config should contain Bootstrap section\")\n\t\tassert.Contains(t, expandedOutput, `\"DNS\"`, \"Expanded config should contain DNS section\")\n\t})\n}\n\nfunc testMixedConfigurationPreserved(t *testing.T) {\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Set mixed configuration\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\n\t\t\"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWTest\",\n\t\t\"auto\",\n\t\t\"/ip4/127.0.0.2/tcp/4001/p2p/12D3KooWTest2\",\n\t})\n\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\n\t\t\"eth.\": \"https://eth.resolver\",\n\t\t\"foo.\": \"auto\",\n\t\t\"bar.\": \"https://bar.resolver\",\n\t})\n\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\n\t\t\"https://static.router\",\n\t\t\"auto\",\n\t})\n\n\t// Verify Bootstrap preserves order and mixes auto with static\n\tresult := node.RunIPFS(\"config\", \"Bootstrap\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar bootstrap []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\tassert.Equal(t, []string{\n\t\t\"/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWTest\",\n\t\t\"auto\",\n\t\t\"/ip4/127.0.0.2/tcp/4001/p2p/12D3KooWTest2\",\n\t}, bootstrap)\n\n\t// Verify DNS.Resolvers preserves both auto and static\n\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar resolvers map[string]string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &resolvers)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"https://eth.resolver\", resolvers[\"eth.\"])\n\tassert.Equal(t, \"auto\", resolvers[\"foo.\"])\n\tassert.Equal(t, \"https://bar.resolver\", resolvers[\"bar.\"])\n\n\t// Verify Routing.DelegatedRouters preserves order\n\tresult = node.RunIPFS(\"config\", \"Routing.DelegatedRouters\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar routers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &routers)\n\trequire.NoError(t, err)\n\tassert.Equal(t, []string{\n\t\t\"https://static.router\",\n\t\t\"auto\",\n\t}, routers)\n}\n\nfunc testConfigReplacePreservesAuto(t *testing.T) {\n\t// Create IPFS node\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init(\"--profile=test\")\n\n\t// Set initial auto values\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\n\t// Export current config\n\tresult := node.RunIPFS(\"config\", \"show\")\n\trequire.Equal(t, 0, result.ExitCode())\n\toriginalConfig := result.Stdout.String()\n\n\t// Verify auto values are in the exported config\n\tassert.Contains(t, originalConfig, `\"Bootstrap\": [\n \"auto\"\n ]`)\n\tassert.Contains(t, originalConfig, `\"foo.\": \"auto\"`)\n\n\t// Modify the config string to add a new field but preserve auto values\n\tvar configMap map[string]any\n\terr := json.Unmarshal([]byte(originalConfig), &configMap)\n\trequire.NoError(t, err)\n\n\t// Add a new field\n\tconfigMap[\"NewTestField\"] = \"test-value\"\n\n\t// Marshal back to JSON\n\tmodifiedConfig, err := json.MarshalIndent(configMap, \"\", \" \")\n\trequire.NoError(t, err)\n\n\t// Write config to file and replace\n\tconfigFile := h.WriteToTemp(string(modifiedConfig))\n\treplaceResult := node.RunIPFS(\"config\", \"replace\", configFile)\n\tif replaceResult.ExitCode() != 0 {\n\t\tt.Logf(\"Config replace failed: stdout=%s, stderr=%s\", replaceResult.Stdout.String(), replaceResult.Stderr.String())\n\t}\n\trequire.Equal(t, 0, replaceResult.ExitCode())\n\n\t// Verify auto values are still present after replace\n\tresult = node.RunIPFS(\"config\", \"Bootstrap\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar bootstrap []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &bootstrap)\n\trequire.NoError(t, err)\n\tassert.Equal(t, []string{\"auto\"}, bootstrap, \"Bootstrap should still contain auto after config replace\")\n\n\t// Verify DNS resolver config is preserved after replace\n\tresult = node.RunIPFS(\"config\", \"DNS.Resolvers\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar resolvers map[string]string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &resolvers)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"auto\", resolvers[\"foo.\"], \"DNS resolver for foo. should still be auto after config replace\")\n}\n\nfunc testExpandAutoFiltersUnsupportedPathsDelegated(t *testing.T) {\n\t// Test scenario: CLI with daemon started and autoconf cached using delegated routing\n\t// This tests the production scenario where delegated routing is enabled and\n\t// daemon has fetched and cached autoconf data, and CLI commands read from that cache\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure delegated routing to use autoconf URLs\n\tnode.SetIPFSConfig(\"Routing.Type\", \"delegated\")\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\t// Disable content providing when using delegated routing\n\tnode.SetIPFSConfig(\"Provide.Enabled\", false)\n\tnode.SetIPFSConfig(\"Provide.DHT.Interval\", \"0\")\n\n\t// Load test autoconf data with unsupported paths\n\tautoConfData := loadTestDataExpand(t, \"autoconf_with_unsupported_paths.json\")\n\n\t// Create HTTP server that serves autoconf.json with unsupported paths\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Configure autoconf for the node\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Verify the autoconf URL is set correctly\n\tresult := node.RunIPFS(\"config\", \"AutoConf.URL\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config AutoConf.URL should succeed\")\n\tt.Logf(\"AutoConf URL is set to: %s\", result.Stdout.String())\n\tassert.Contains(t, result.Stdout.String(), \"127.0.0.1\", \"AutoConf URL should contain the test server address\")\n\n\t// Start daemon to fetch and cache autoconf data\n\tt.Log(\"Starting daemon to fetch and cache autoconf data...\")\n\tdaemon := node.StartDaemon()\n\tdefer daemon.StopDaemon()\n\n\t// Wait for autoconf fetch (use autoconf default timeout + buffer)\n\ttime.Sleep(6 * time.Second) // defaultTimeout is 5s, add 1s buffer\n\tt.Log(\"AutoConf should now be cached by daemon\")\n\n\t// Test Routing.DelegatedRouters field expansion filters unsupported paths\n\tresult = node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\t// After cache prewarming, should get URLs from autoconf that have supported paths\n\tassert.Contains(t, expandedRouters, \"https://supported.example.com/routing/v1/providers\", \"Should contain supported provider URL\")\n\tassert.Contains(t, expandedRouters, \"https://supported.example.com/routing/v1/peers\", \"Should contain supported peers URL\")\n\tassert.Contains(t, expandedRouters, \"https://mixed.example.com/routing/v1/providers\", \"Should contain mixed provider URL\")\n\tassert.Contains(t, expandedRouters, \"https://mixed.example.com/routing/v1/peers\", \"Should contain mixed peers URL\")\n\n\t// Verify unsupported URLs from autoconf are filtered out (not in result)\n\tassert.NotContains(t, expandedRouters, \"https://unsupported.example.com/example/v0/read\", \"Should filter out unsupported path /example/v0/read\")\n\tassert.NotContains(t, expandedRouters, \"https://unsupported.example.com/api/v1/custom\", \"Should filter out unsupported path /api/v1/custom\")\n\tassert.NotContains(t, expandedRouters, \"https://mixed.example.com/unsupported/path\", \"Should filter out unsupported path /unsupported/path\")\n\n\tt.Logf(\"Filtered routers: %v\", expandedRouters)\n\n\t// Test Ipns.DelegatedPublishers field expansion filters unsupported paths\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\t// After cache prewarming, should get URLs from autoconf that have supported paths\n\tassert.Contains(t, expandedPublishers, \"https://supported.example.com/routing/v1/ipns\", \"Should contain supported IPNS URL\")\n\tassert.Contains(t, expandedPublishers, \"https://mixed.example.com/routing/v1/ipns\", \"Should contain mixed IPNS URL\")\n\n\t// Verify unsupported URLs from autoconf are filtered out (not in result)\n\tassert.NotContains(t, expandedPublishers, \"https://unsupported.example.com/example/v0/write\", \"Should filter out unsupported write path\")\n\n\tt.Logf(\"Filtered publishers: %v\", expandedPublishers)\n}\n\nfunc testExpandAutoWithoutCacheDelegated(t *testing.T) {\n\t// Test scenario: CLI without daemon ever starting (no cached autoconf) using delegated routing\n\t// This tests the fallback scenario where delegated routing is configured but CLI commands\n\t// cannot read from cache and must fall back to hardcoded defaults\n\n\t// Create IPFS node but DO NOT start daemon\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure delegated routing to use autoconf URLs (but no daemon to fetch them)\n\tnode.SetIPFSConfig(\"Routing.Type\", \"delegated\")\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\t// Disable content providing when using delegated routing\n\tnode.SetIPFSConfig(\"Provide.Enabled\", false)\n\tnode.SetIPFSConfig(\"Provide.DHT.Interval\", \"0\")\n\n\t// Load test autoconf data with unsupported paths (this won't be used since no daemon)\n\tautoConfData := loadTestDataExpand(t, \"autoconf_with_unsupported_paths.json\")\n\n\t// Create HTTP server that serves autoconf.json with unsupported paths\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Configure autoconf for the node (but daemon never starts to fetch it)\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Test Routing.DelegatedRouters field expansion without cached autoconf\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\t// Without cached autoconf, should get fallback URLs from GetMainnetFallbackConfig()\n\t// NOTE: These values may change if autoconf library updates GetMainnetFallbackConfig()\n\tassert.Contains(t, expandedRouters, \"https://cid.contact/routing/v1/providers\", \"Should contain fallback provider URL from GetMainnetFallbackConfig()\")\n\n\tt.Logf(\"Fallback routers (no cache): %v\", expandedRouters)\n\n\t// Test Ipns.DelegatedPublishers field expansion without cached autoconf\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\t// Without cached autoconf, should get fallback IPNS publishers from GetMainnetFallbackConfig()\n\t// NOTE: These values may change if autoconf library updates GetMainnetFallbackConfig()\n\tassert.Contains(t, expandedPublishers, \"https://delegated-ipfs.dev/routing/v1/ipns\", \"Should contain fallback IPNS URL from GetMainnetFallbackConfig()\")\n\n\tt.Logf(\"Fallback publishers (no cache): %v\", expandedPublishers)\n}\n\nfunc testExpandAutoWithAutoRouting(t *testing.T) {\n\t// Test scenario: CLI with daemon started using auto routing with NewRoutingSystem\n\t// This tests that non-native systems (NewRoutingSystem) ARE delegated even with auto routing\n\t// Only native systems like AminoDHT are handled internally with auto routing\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure auto routing with non-native system\n\tnode.SetIPFSConfig(\"Routing.Type\", \"auto\")\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Load test autoconf data with NewRoutingSystem (non-native, will be delegated)\n\tautoConfData := loadTestDataExpand(t, \"autoconf_new_routing_system.json\")\n\n\t// Create HTTP server that serves autoconf.json\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Configure autoconf for the node\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Start daemon to fetch and cache autoconf data\n\tt.Log(\"Starting daemon to fetch and cache autoconf data...\")\n\tdaemon := node.StartDaemon()\n\tdefer daemon.StopDaemon()\n\n\t// Wait for autoconf fetch (use autoconf default timeout + buffer)\n\ttime.Sleep(6 * time.Second) // defaultTimeout is 5s, add 1s buffer\n\tt.Log(\"AutoConf should now be cached by daemon\")\n\n\t// Test Routing.DelegatedRouters field expansion with auto routing\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\t// With auto routing and NewRoutingSystem (non-native), delegated endpoints should be populated\n\tassert.Contains(t, expandedRouters, \"https://new-routing.example.com/routing/v1/providers\", \"Should contain NewRoutingSystem provider URL\")\n\tassert.Contains(t, expandedRouters, \"https://new-routing.example.com/routing/v1/peers\", \"Should contain NewRoutingSystem peers URL\")\n\n\tt.Logf(\"Auto routing routers (NewRoutingSystem delegated): %v\", expandedRouters)\n\n\t// Test Ipns.DelegatedPublishers field expansion with auto routing\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\t// With auto routing and NewRoutingSystem (non-native), delegated publishers should be populated\n\tassert.Contains(t, expandedPublishers, \"https://new-routing.example.com/routing/v1/ipns\", \"Should contain NewRoutingSystem IPNS URL\")\n\n\tt.Logf(\"Auto routing publishers (NewRoutingSystem delegated): %v\", expandedPublishers)\n}\n\nfunc testExpandAutoWithMixedSystems(t *testing.T) {\n\t// Test scenario: Auto routing with both AminoDHT (native) and IPNI (delegated) systems\n\t// This explicitly confirms that AminoDHT is NOT delegated but IPNI at cid.contact IS delegated\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure auto routing\n\tnode.SetIPFSConfig(\"Routing.Type\", \"auto\")\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Load test autoconf data with both AminoDHT and IPNI systems\n\tautoConfData := loadTestDataExpand(t, \"autoconf_amino_and_ipni.json\")\n\n\t// Create HTTP server that serves autoconf.json\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Configure autoconf for the node\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Start daemon to fetch and cache autoconf data\n\tt.Log(\"Starting daemon to fetch and cache autoconf data...\")\n\tdaemon := node.StartDaemon()\n\tdefer daemon.StopDaemon()\n\n\t// Wait for autoconf fetch (use autoconf default timeout + buffer)\n\ttime.Sleep(6 * time.Second) // defaultTimeout is 5s, add 1s buffer\n\tt.Log(\"AutoConf should now be cached by daemon\")\n\n\t// Test Routing.DelegatedRouters field expansion\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\t// With auto routing: AminoDHT (native) should NOT be delegated, IPNI should be delegated\n\tassert.Contains(t, expandedRouters, \"https://cid.contact/routing/v1/providers\", \"Should contain IPNI provider URL (delegated)\")\n\tassert.NotContains(t, expandedRouters, \"https://amino-dht.example.com\", \"Should NOT contain AminoDHT URLs (native)\")\n\n\tt.Logf(\"Mixed systems routers (IPNI delegated, AminoDHT native): %v\", expandedRouters)\n\n\t// Test Ipns.DelegatedPublishers field expansion\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\t// IPNI system doesn't have write endpoints, so publishers should be empty\n\t// (or contain other systems if they have write endpoints)\n\tt.Logf(\"Mixed systems publishers (IPNI has no write endpoints): %v\", expandedPublishers)\n}\n\nfunc testExpandAutoWithFiltering(t *testing.T) {\n\t// Test scenario: Auto routing with NewRoutingSystem and path filtering\n\t// This tests that path filtering works for delegated systems even with auto routing\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure auto routing\n\tnode.SetIPFSConfig(\"Routing.Type\", \"auto\")\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Load test autoconf data with NewRoutingSystem and mixed valid/invalid paths\n\tautoConfData := loadTestDataExpand(t, \"autoconf_new_routing_with_filtering.json\")\n\n\t// Create HTTP server that serves autoconf.json\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Configure autoconf for the node\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Start daemon to fetch and cache autoconf data\n\tt.Log(\"Starting daemon to fetch and cache autoconf data...\")\n\tdaemon := node.StartDaemon()\n\tdefer daemon.StopDaemon()\n\n\t// Wait for autoconf fetch (use autoconf default timeout + buffer)\n\ttime.Sleep(6 * time.Second) // defaultTimeout is 5s, add 1s buffer\n\tt.Log(\"AutoConf should now be cached by daemon\")\n\n\t// Test Routing.DelegatedRouters field expansion with filtering\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\t// Should contain supported paths from NewRoutingSystem\n\tassert.Contains(t, expandedRouters, \"https://supported-new.example.com/routing/v1/providers\", \"Should contain supported provider URL\")\n\tassert.Contains(t, expandedRouters, \"https://supported-new.example.com/routing/v1/peers\", \"Should contain supported peers URL\")\n\tassert.Contains(t, expandedRouters, \"https://mixed-new.example.com/routing/v1/providers\", \"Should contain mixed provider URL\")\n\tassert.Contains(t, expandedRouters, \"https://mixed-new.example.com/routing/v1/peers\", \"Should contain mixed peers URL\")\n\n\t// Should NOT contain unsupported paths\n\tassert.NotContains(t, expandedRouters, \"https://unsupported-new.example.com/custom/v0/read\", \"Should filter out unsupported path\")\n\tassert.NotContains(t, expandedRouters, \"https://unsupported-new.example.com/api/v1/nonstandard\", \"Should filter out unsupported path\")\n\tassert.NotContains(t, expandedRouters, \"https://mixed-new.example.com/invalid/path\", \"Should filter out invalid path from mixed endpoint\")\n\n\tt.Logf(\"Filtered routers (NewRoutingSystem with auto routing): %v\", expandedRouters)\n\n\t// Test Ipns.DelegatedPublishers field expansion with filtering\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\t// Should contain supported IPNS paths\n\tassert.Contains(t, expandedPublishers, \"https://supported-new.example.com/routing/v1/ipns\", \"Should contain supported IPNS URL\")\n\tassert.Contains(t, expandedPublishers, \"https://mixed-new.example.com/routing/v1/ipns\", \"Should contain mixed IPNS URL\")\n\n\t// Should NOT contain unsupported write paths\n\tassert.NotContains(t, expandedPublishers, \"https://unsupported-new.example.com/custom/v0/write\", \"Should filter out unsupported write path\")\n\n\tt.Logf(\"Filtered publishers (NewRoutingSystem with auto routing): %v\", expandedPublishers)\n}\n\nfunc testExpandAutoWithoutCacheAuto(t *testing.T) {\n\t// Test scenario: CLI without daemon ever starting using auto routing (default)\n\t// This tests the fallback scenario where auto routing is used but doesn't populate delegated config fields\n\n\t// Create IPFS node but DO NOT start daemon\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure auto routing - delegated fields are set to \"auto\" but won't be populated\n\t// because auto routing uses different internal mechanisms\n\tnode.SetIPFSConfig(\"Routing.Type\", \"auto\")\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\n\t// Load test autoconf data (this won't be used since no daemon and auto routing doesn't use these fields)\n\tautoConfData := loadTestDataExpand(t, \"autoconf_with_unsupported_paths.json\")\n\n\t// Create HTTP server (won't be contacted since no daemon)\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(autoConfData)\n\t}))\n\tdefer server.Close()\n\n\t// Configure autoconf for the node (but daemon never starts to fetch it)\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\n\t// Test Routing.DelegatedRouters field expansion without cached autoconf\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Routing.DelegatedRouters --expand-auto should succeed\")\n\n\tvar expandedRouters []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &expandedRouters)\n\trequire.NoError(t, err)\n\n\t// With auto routing, some fallback URLs are still populated from GetMainnetFallbackConfig()\n\t// NOTE: These values may change if autoconf library updates GetMainnetFallbackConfig()\n\tassert.Contains(t, expandedRouters, \"https://cid.contact/routing/v1/providers\", \"Should contain fallback provider URL from GetMainnetFallbackConfig()\")\n\n\tt.Logf(\"Auto routing fallback routers (with fallbacks): %v\", expandedRouters)\n\n\t// Test Ipns.DelegatedPublishers field expansion without cached autoconf\n\tresult = node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\trequire.Equal(t, 0, result.ExitCode(), \"config Ipns.DelegatedPublishers --expand-auto should succeed\")\n\n\tvar expandedPublishers []string\n\terr = json.Unmarshal([]byte(result.Stdout.String()), &expandedPublishers)\n\trequire.NoError(t, err)\n\n\t// With auto routing, delegated publishers may be empty for fallback scenario\n\t// This can vary based on which systems have write endpoints in the fallback config\n\tt.Logf(\"Auto routing fallback publishers: %v\", expandedPublishers)\n}\n\n// Helper function to load test data files\nfunc loadTestDataExpand(t *testing.T, filename string) []byte {\n\tt.Helper()\n\n\tdata, err := os.ReadFile(\"testdata/\" + filename)\n\trequire.NoError(t, err, \"Failed to read test data file: %s\", filename)\n\n\treturn data\n}\n" }, { "path": "test/cli/autoconf/extensibility_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"slices\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestAutoConfExtensibility_NewSystem verifies that the AutoConf system can be extended\n// with new routing systems beyond the default AminoDHT and IPNI.\n//\n// The test verifies that:\n// 1. New systems can be added via AutoConf's SystemRegistry\n// 2. Native vs delegated system filtering works correctly:\n// - Native systems (AminoDHT) provide bootstrap peers and are used for P2P routing\n// - Delegated systems (IPNI, NewSystem) provide HTTP endpoints for delegated routing\n//\n// 3. The system correctly filters endpoints based on routing type\n//\n// Note: Only native systems contribute bootstrap peers. Delegated systems like \"NewSystem\"\n// only provide HTTP routing endpoints, not P2P bootstrap peers.\nfunc TestAutoConfExtensibility_NewSystem(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping test in short mode\")\n\t}\n\n\t// Setup mock autoconf server with NewSystem\n\tvar mockServer *httptest.Server\n\tmockServer = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t// Create autoconf.json with NewSystem\n\t\tautoconfData := map[string]any{\n\t\t\t\"AutoConfVersion\": 2025072901,\n\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://github.com/ipfs/specs/pull/497\",\n\t\t\t\t\t\"Description\": \"Public DHT swarm\",\n\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\", \"/routing/v1/peers\", \"/routing/v1/ipns\"},\n\t\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"IPNI\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://ipni.example.com\",\n\t\t\t\t\t\"Description\": \"Network Indexer\",\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"NewSystem\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://example.com/newsystem\",\n\t\t\t\t\t\"Description\": \"Test system for extensibility verification\",\n\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\t\t\t\"/ip4/127.0.0.1/tcp/9999/p2p/12D3KooWPeQ4r3v6CmVmKXoFGtqEqcr3L8P6La9yH5oEWKtoLVVa\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"DNSResolvers\": map[string]any{\n\t\t\t\t\"eth.\": []string{\"https://dns.eth.limo/dns-query\"},\n\t\t\t},\n\t\t\t\"DelegatedEndpoints\": map[string]any{\n\t\t\t\t\"https://ipni.example.com\": map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"IPNI\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t},\n\t\t\t\tmockServer.URL + \"/newsystem\": map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"NewSystem\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"Cache-Control\", \"max-age=300\")\n\t\t_ = json.NewEncoder(w).Encode(autoconfData)\n\t}))\n\tdefer mockServer.Close()\n\n\t// NewSystem mock server URL will be dynamically assigned\n\tnewSystemServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t// Simple mock server for NewSystem endpoint\n\t\tresponse := map[string]any{\"Providers\": []any{}}\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_ = json.NewEncoder(w).Encode(response)\n\t}))\n\tdefer newSystemServer.Close()\n\n\t// Update the autoconf to point to the correct NewSystem endpoint\n\tmockServer.Close()\n\tmockServer = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tautoconfData := map[string]any{\n\t\t\t\"AutoConfVersion\": 2025072901,\n\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://github.com/ipfs/specs/pull/497\",\n\t\t\t\t\t\"Description\": \"Public DHT swarm\",\n\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\", \"/routing/v1/peers\", \"/routing/v1/ipns\"},\n\t\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"IPNI\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://ipni.example.com\",\n\t\t\t\t\t\"Description\": \"Network Indexer\",\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"NewSystem\": map[string]any{\n\t\t\t\t\t\"URL\": \"https://example.com/newsystem\",\n\t\t\t\t\t\"Description\": \"Test system for extensibility verification\",\n\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\t\t\t\"/ip4/127.0.0.1/tcp/9999/p2p/12D3KooWPeQ4r3v6CmVmKXoFGtqEqcr3L8P6La9yH5oEWKtoLVVa\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"DNSResolvers\": map[string]any{\n\t\t\t\t\"eth.\": []string{\"https://dns.eth.limo/dns-query\"},\n\t\t\t},\n\t\t\t\"DelegatedEndpoints\": map[string]any{\n\t\t\t\t\"https://ipni.example.com\": map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"IPNI\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t},\n\t\t\t\tnewSystemServer.URL: map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"NewSystem\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"Cache-Control\", \"max-age=300\")\n\t\t_ = json.NewEncoder(w).Encode(autoconfData)\n\t}))\n\tdefer mockServer.Close()\n\n\t// Create Kubo node with autoconf pointing to mock server\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init()\n\n\t// Update config to use mock autoconf server\n\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\tcfg.AutoConf.URL = config.NewOptionalString(mockServer.URL)\n\t\tcfg.AutoConf.Enabled = config.True\n\t\tcfg.AutoConf.RefreshInterval = config.NewOptionalDuration(1 * time.Second)\n\t\tcfg.Routing.Type = config.NewOptionalString(\"auto\") // Should enable native AminoDHT + delegated others\n\t\tcfg.Bootstrap = []string{\"auto\"}\n\t\tcfg.Routing.DelegatedRouters = []string{\"auto\"}\n\t})\n\n\t// Start the daemon\n\tdaemon := node.StartDaemon()\n\tdefer daemon.StopDaemon()\n\n\t// Give the daemon some time to initialize and make requests\n\ttime.Sleep(3 * time.Second)\n\n\t// Test 1: Verify bootstrap includes both AminoDHT and NewSystem peers (deduplicated)\n\tbootstrapResult := daemon.IPFS(\"bootstrap\", \"list\", \"--expand-auto\")\n\tbootstrapOutput := bootstrapResult.Stdout.String()\n\tt.Logf(\"Bootstrap output: %s\", bootstrapOutput)\n\n\t// Should contain original DHT bootstrap peer (AminoDHT is a native system)\n\trequire.Contains(t, bootstrapOutput, \"QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\", \"Should contain AminoDHT bootstrap peer\")\n\n\t// Note: NewSystem bootstrap peers are NOT included because only native systems\n\t// (AminoDHT for Routing.Type=\"auto\") contribute bootstrap peers.\n\t// Delegated systems like NewSystem only provide HTTP routing endpoints.\n\n\t// Test 2: Verify delegated endpoints are filtered correctly\n\t// For Routing.Type=auto, native systems=[AminoDHT], so:\n\t// - AminoDHT endpoints should be filtered out\n\t// - IPNI and NewSystem endpoints should be included\n\n\t// Get the expanded delegated routers using --expand-auto\n\trouterResult := daemon.IPFS(\"config\", \"Routing.DelegatedRouters\", \"--expand-auto\")\n\tvar expandedRouters []string\n\trequire.NoError(t, json.Unmarshal([]byte(routerResult.Stdout.String()), &expandedRouters))\n\n\tt.Logf(\"Expanded delegated routers: %v\", expandedRouters)\n\n\t// Verify we got exactly 2 delegated routers: IPNI and NewSystem\n\trequire.Equal(t, 2, len(expandedRouters), \"Should have exactly 2 delegated routers (IPNI and NewSystem). Got %d: %v\", len(expandedRouters), expandedRouters)\n\n\t// Convert to URLs for checking\n\trouterURLs := expandedRouters\n\n\t// Should contain NewSystem endpoint (not native) - now with routing path\n\tfoundNewSystem := false\n\texpectedNewSystemURL := newSystemServer.URL + \"/routing/v1/providers\" // Full URL with path, as returned by DelegatedRoutersWithAutoConf\n\tif slices.Contains(routerURLs, expectedNewSystemURL) {\n\t\tfoundNewSystem = true\n\t}\n\trequire.True(t, foundNewSystem, \"Should contain NewSystem endpoint (%s) for delegated routing, got: %v\", expectedNewSystemURL, routerURLs)\n\n\t// Should contain ipni.example.com (IPNI is not native)\n\tfoundIPNI := false\n\tfor _, url := range routerURLs {\n\t\tif strings.Contains(url, \"ipni.example.com\") {\n\t\t\tfoundIPNI = true\n\t\t\tbreak\n\t\t}\n\t}\n\trequire.True(t, foundIPNI, \"Should contain ipni.example.com endpoint for IPNI\")\n\n\t// Test passes - we've verified that:\n\t// 1. Bootstrap peers are correctly resolved from native systems only\n\t// 2. Delegated routers include both IPNI and NewSystem endpoints\n\t// 3. URL format is correct (base URLs with paths)\n\t// 4. AutoConf extensibility works for unknown systems\n\n\tt.Log(\"NewSystem extensibility test passed - Kubo successfully discovered and used unknown routing system\")\n}\n" }, { "path": "test/cli/autoconf/fuzz_test.go", "content": "package autoconf\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// testAutoConfWithFallback is a helper function that tests autoconf parsing with fallback detection\nfunc testAutoConfWithFallback(t *testing.T, serverURL string, expectError bool, expectErrorMsg string) (*autoconf.Config, bool) {\n\treturn testAutoConfWithFallbackAndTimeout(t, serverURL, expectError, expectErrorMsg, 10*time.Second)\n}\n\n// testAutoConfWithFallbackAndTimeout is a helper function that tests autoconf parsing with fallback detection and custom timeout\nfunc testAutoConfWithFallbackAndTimeout(t *testing.T, serverURL string, expectError bool, expectErrorMsg string, timeout time.Duration) (*autoconf.Config, bool) {\n\t// Use fallback detection to test error conditions with MustGetConfigWithRefresh\n\tfallbackUsed := false\n\tfallbackConfig := &autoconf.Config{\n\t\tAutoConfVersion: -999, // Special marker to detect fallback usage\n\t\tAutoConfSchema: -999,\n\t}\n\n\tclient, err := autoconf.NewClient(\n\t\tautoconf.WithUserAgent(\"test-agent\"),\n\t\tautoconf.WithURL(serverURL),\n\t\tautoconf.WithRefreshInterval(autoconf.DefaultRefreshInterval),\n\t\tautoconf.WithFallback(func() *autoconf.Config {\n\t\t\tfallbackUsed = true\n\t\t\treturn fallbackConfig\n\t\t}),\n\t)\n\trequire.NoError(t, err)\n\n\tctx, cancel := context.WithTimeout(context.Background(), timeout)\n\tdefer cancel()\n\tresult := client.GetCachedOrRefresh(ctx)\n\n\tif expectError {\n\t\trequire.True(t, fallbackUsed, expectErrorMsg)\n\t\trequire.Equal(t, int64(-999), result.AutoConfVersion, \"Should return fallback config for error case\")\n\t} else {\n\t\trequire.False(t, fallbackUsed, \"Expected no fallback to be used\")\n\t\trequire.NotEqual(t, int64(-999), result.AutoConfVersion, \"Should return fetched config for success case\")\n\t}\n\n\treturn result, fallbackUsed\n}\n\nfunc TestAutoConfFuzz(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"fuzz autoconf version\", testFuzzAutoConfVersion)\n\tt.Run(\"fuzz bootstrap arrays\", testFuzzBootstrapArrays)\n\tt.Run(\"fuzz dns resolvers\", testFuzzDNSResolvers)\n\tt.Run(\"fuzz delegated routers\", testFuzzDelegatedRouters)\n\tt.Run(\"fuzz delegated publishers\", testFuzzDelegatedPublishers)\n\tt.Run(\"fuzz malformed json\", testFuzzMalformedJSON)\n\tt.Run(\"fuzz large payloads\", testFuzzLargePayloads)\n}\n\nfunc testFuzzAutoConfVersion(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tversion any\n\t\texpectError bool\n\t}{\n\t\t{\"valid version\", 2025071801, false},\n\t\t{\"zero version\", 0, true}, // Should be invalid\n\t\t{\"negative version\", -1, false}, // Parser accepts negative versions\n\t\t{\"string version\", \"2025071801\", true}, // Should be number\n\t\t{\"float version\", 2025071801.5, true},\n\t\t{\"very large version\", 9999999999999999, false}, // Large but valid int64\n\t\t{\"null version\", nil, true},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tconfig := map[string]any{\n\t\t\t\t\"AutoConfVersion\": tc.version,\n\t\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\t\"Bootstrap\": []string{\n\t\t\t\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"DNSResolvers\": map[string]any{},\n\t\t\t\t\"DelegatedEndpoints\": map[string]any{},\n\t\t\t}\n\n\t\t\tjsonData, err := json.Marshal(config)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t\t_, _ = w.Write(jsonData)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\t// Test that our autoconf parser handles this gracefully\n\t\t\t_, _ = testAutoConfWithFallback(t, server.URL, tc.expectError, fmt.Sprintf(\"Expected fallback to be used for %s\", tc.name))\n\t\t})\n\t}\n}\n\nfunc testFuzzBootstrapArrays(t *testing.T) {\n\ttype testCase struct {\n\t\tname string\n\t\tbootstrap any\n\t\texpectError bool\n\t\tvalidate func(*testing.T, *autoconf.Response)\n\t}\n\n\ttestCases := []testCase{\n\t\t{\n\t\t\tname: \"valid bootstrap\",\n\t\t\tbootstrap: []string{\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\texpected := []string{\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"}\n\t\t\t\tbootstrapPeers := resp.Config.GetBootstrapPeers(\"AminoDHT\")\n\t\t\t\tassert.Equal(t, expected, bootstrapPeers, \"Bootstrap peers should match configured values\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty bootstrap\",\n\t\t\tbootstrap: []string{},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tbootstrapPeers := resp.Config.GetBootstrapPeers(\"AminoDHT\")\n\t\t\t\tassert.Empty(t, bootstrapPeers, \"Empty bootstrap should result in empty peers\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"null bootstrap\",\n\t\t\tbootstrap: nil,\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tbootstrapPeers := resp.Config.GetBootstrapPeers(\"AminoDHT\")\n\t\t\t\tassert.Empty(t, bootstrapPeers, \"Null bootstrap should result in empty peers\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid multiaddr\",\n\t\t\tbootstrap: []string{\"invalid-multiaddr\"},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"very long multiaddr\",\n\t\t\tbootstrap: []string{\"/dnsaddr/\" + strings.Repeat(\"a\", 100) + \".com/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\texpected := []string{\"/dnsaddr/\" + strings.Repeat(\"a\", 100) + \".com/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"}\n\t\t\t\tbootstrapPeers := resp.Config.GetBootstrapPeers(\"AminoDHT\")\n\t\t\t\tassert.Equal(t, expected, bootstrapPeers, \"Very long multiaddr should be preserved\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"bootstrap as string\",\n\t\t\tbootstrap: \"/dnsaddr/test\",\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"bootstrap as number\",\n\t\t\tbootstrap: 123,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"mixed types in array\",\n\t\t\tbootstrap: []any{\"/dnsaddr/test\", 123, nil},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"extremely large array\",\n\t\t\tbootstrap: make([]string, 1000),\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\t// Array will be filled in the loop below\n\t\t\t\tbootstrapPeers := resp.Config.GetBootstrapPeers(\"AminoDHT\")\n\t\t\t\tassert.Len(t, bootstrapPeers, 1000, \"Large bootstrap array should be preserved\")\n\t\t\t},\n\t\t},\n\t}\n\n\t// Fill the large array with valid multiaddrs\n\tlargeArray := testCases[len(testCases)-1].bootstrap.([]string)\n\tfor i := range largeArray {\n\t\tlargeArray[i] = fmt.Sprintf(\"/dnsaddr/bootstrap%d.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\", i)\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tconfig := map[string]any{\n\t\t\t\t\"AutoConfVersion\": 2025072301,\n\t\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\t\"Bootstrap\": tc.bootstrap,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"DNSResolvers\": map[string]any{},\n\t\t\t\t\"DelegatedEndpoints\": map[string]any{},\n\t\t\t}\n\n\t\t\tjsonData, err := json.Marshal(config)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t\t_, _ = w.Write(jsonData)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\tautoConf, fallbackUsed := testAutoConfWithFallback(t, server.URL, tc.expectError, fmt.Sprintf(\"Expected fallback to be used for %s\", tc.name))\n\n\t\t\tif !tc.expectError {\n\t\t\t\trequire.NotNil(t, autoConf, \"AutoConf should not be nil for successful parsing\")\n\n\t\t\t\t// Verify structure is reasonable\n\t\t\t\tbootstrapPeers := autoConf.GetBootstrapPeers(\"AminoDHT\")\n\t\t\t\trequire.IsType(t, []string{}, bootstrapPeers, \"Bootstrap should be []string\")\n\n\t\t\t\t// Run test-specific validation if provided (only for non-fallback cases)\n\t\t\t\tif tc.validate != nil && !fallbackUsed {\n\t\t\t\t\t// Create a mock Response for compatibility with validation functions\n\t\t\t\t\tmockResponse := &autoconf.Response{Config: autoConf}\n\t\t\t\t\ttc.validate(t, mockResponse)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc testFuzzDNSResolvers(t *testing.T) {\n\ttype testCase struct {\n\t\tname string\n\t\tresolvers any\n\t\texpectError bool\n\t\tvalidate func(*testing.T, *autoconf.Response)\n\t}\n\n\ttestCases := []testCase{\n\t\t{\n\t\t\tname: \"valid resolvers\",\n\t\t\tresolvers: map[string][]string{\".\": {\"https://dns.google/dns-query\"}},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\texpected := map[string][]string{\".\": {\"https://dns.google/dns-query\"}}\n\t\t\t\tassert.Equal(t, expected, resp.Config.DNSResolvers, \"DNS resolvers should match configured values\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty resolvers\",\n\t\t\tresolvers: map[string][]string{},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Empty(t, resp.Config.DNSResolvers, \"Empty resolvers should result in empty map\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"null resolvers\",\n\t\t\tresolvers: nil,\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Empty(t, resp.Config.DNSResolvers, \"Null resolvers should result in empty map\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"relative URL (missing scheme)\",\n\t\t\tresolvers: map[string][]string{\".\": {\"not-a-url\"}},\n\t\t\texpectError: true, // Should error due to strict HTTP/HTTPS validation\n\t\t},\n\t\t{\n\t\t\tname: \"invalid URL format\",\n\t\t\tresolvers: map[string][]string{\".\": {\"://invalid-missing-scheme\"}},\n\t\t\texpectError: true, // Should error because url.Parse() fails\n\t\t},\n\t\t{\n\t\t\tname: \"non-HTTP scheme\",\n\t\t\tresolvers: map[string][]string{\".\": {\"ftp://example.com/dns-query\"}},\n\t\t\texpectError: true, // Should error due to non-HTTP/HTTPS scheme\n\t\t},\n\t\t{\n\t\t\tname: \"very long domain\",\n\t\t\tresolvers: map[string][]string{strings.Repeat(\"a\", 1000) + \".com\": {\"https://dns.google/dns-query\"}},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\texpected := map[string][]string{strings.Repeat(\"a\", 1000) + \".com\": {\"https://dns.google/dns-query\"}}\n\t\t\t\tassert.Equal(t, expected, resp.Config.DNSResolvers, \"Very long domain should be preserved\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"many resolvers\",\n\t\t\tresolvers: generateManyResolvers(100),\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\texpected := generateManyResolvers(100)\n\t\t\t\tassert.Equal(t, expected, resp.Config.DNSResolvers, \"Many resolvers should be preserved\")\n\t\t\t\tassert.Equal(t, 100, len(resp.Config.DNSResolvers), \"Should have 100 resolvers\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"resolvers as array\",\n\t\t\tresolvers: []string{\"https://dns.google/dns-query\"},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"nested invalid structure\",\n\t\t\tresolvers: map[string]any{\".\": map[string]string{\"invalid\": \"structure\"}},\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tconfig := map[string]any{\n\t\t\t\t\"AutoConfVersion\": 2025072301,\n\t\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\t\"Bootstrap\": []string{\"/dnsaddr/test\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"DNSResolvers\": tc.resolvers,\n\t\t\t\t\"DelegatedEndpoints\": map[string]any{},\n\t\t\t}\n\n\t\t\tjsonData, err := json.Marshal(config)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t\t_, _ = w.Write(jsonData)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\tautoConf, fallbackUsed := testAutoConfWithFallback(t, server.URL, tc.expectError, fmt.Sprintf(\"Expected fallback to be used for %s\", tc.name))\n\n\t\t\tif !tc.expectError {\n\t\t\t\trequire.NotNil(t, autoConf, \"AutoConf should not be nil for successful parsing\")\n\n\t\t\t\t// Run test-specific validation if provided (only for non-fallback cases)\n\t\t\t\tif tc.validate != nil && !fallbackUsed {\n\t\t\t\t\t// Create a mock Response for compatibility with validation functions\n\t\t\t\t\tmockResponse := &autoconf.Response{Config: autoConf}\n\t\t\t\t\ttc.validate(t, mockResponse)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc testFuzzDelegatedRouters(t *testing.T) {\n\t// Test various malformed delegated router configurations\n\ttype testCase struct {\n\t\tname string\n\t\trouters any\n\t\texpectError bool\n\t\tvalidate func(*testing.T, *autoconf.Response)\n\t}\n\n\ttestCases := []testCase{\n\t\t{\n\t\t\tname: \"valid endpoints\",\n\t\t\trouters: map[string]any{\n\t\t\t\t\"https://ipni.example.com\": map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"IPNI\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Len(t, resp.Config.DelegatedEndpoints, 1, \"Should have 1 delegated endpoint\")\n\t\t\t\tfor url, config := range resp.Config.DelegatedEndpoints {\n\t\t\t\t\tassert.Contains(t, url, \"ipni.example.com\", \"Endpoint URL should contain expected domain\")\n\t\t\t\t\tassert.Contains(t, config.Systems, \"IPNI\", \"Endpoint should have IPNI system\")\n\t\t\t\t\tassert.Contains(t, config.Read, \"/routing/v1/providers\", \"Endpoint should have providers read path\")\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty routers\",\n\t\t\trouters: map[string]any{},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Empty(t, resp.Config.DelegatedEndpoints, \"Empty routers should result in empty endpoints\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"null routers\",\n\t\t\trouters: nil,\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Empty(t, resp.Config.DelegatedEndpoints, \"Null routers should result in empty endpoints\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid nested structure\",\n\t\t\trouters: map[string]string{\"invalid\": \"structure\"},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid endpoint URLs\",\n\t\t\trouters: map[string]any{\n\t\t\t\t\"not-a-url\": map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"IPNI\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/providers\"},\n\t\t\t\t\t\"Write\": []string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: true, // Should error due to URL validation\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tconfig := map[string]any{\n\t\t\t\t\"AutoConfVersion\": 2025072301,\n\t\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\t\t\"Bootstrap\": []string{\"/dnsaddr/test\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"DNSResolvers\": map[string]any{},\n\t\t\t\t\"DelegatedEndpoints\": tc.routers,\n\t\t\t}\n\n\t\t\tjsonData, err := json.Marshal(config)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t\t_, _ = w.Write(jsonData)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\tautoConf, fallbackUsed := testAutoConfWithFallback(t, server.URL, tc.expectError, fmt.Sprintf(\"Expected fallback to be used for %s\", tc.name))\n\n\t\t\tif !tc.expectError {\n\t\t\t\trequire.NotNil(t, autoConf, \"AutoConf should not be nil for successful parsing\")\n\n\t\t\t\t// Run test-specific validation if provided (only for non-fallback cases)\n\t\t\t\tif tc.validate != nil && !fallbackUsed {\n\t\t\t\t\t// Create a mock Response for compatibility with validation functions\n\t\t\t\t\tmockResponse := &autoconf.Response{Config: autoConf}\n\t\t\t\t\ttc.validate(t, mockResponse)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc testFuzzDelegatedPublishers(t *testing.T) {\n\t// DelegatedPublishers use the same autoclient library validation as DelegatedRouters\n\t// Test that URL validation works for delegated publishers\n\ttype testCase struct {\n\t\tname string\n\t\turls []string\n\t\texpectErr bool\n\t\tvalidate func(*testing.T, *autoconf.Response)\n\t}\n\n\ttestCases := []testCase{\n\t\t{\n\t\t\tname: \"valid HTTPS URLs\",\n\t\t\turls: []string{\"https://delegated-ipfs.dev\", \"https://another-publisher.com\"},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Len(t, resp.Config.DelegatedEndpoints, 2, \"Should have 2 delegated endpoints\")\n\t\t\t\tfoundURLs := make([]string, 0, len(resp.Config.DelegatedEndpoints))\n\t\t\t\tfor url := range resp.Config.DelegatedEndpoints {\n\t\t\t\t\tfoundURLs = append(foundURLs, url)\n\t\t\t\t}\n\t\t\t\texpectedURLs := []string{\"https://delegated-ipfs.dev\", \"https://another-publisher.com\"}\n\t\t\t\tfor _, expectedURL := range expectedURLs {\n\t\t\t\t\tassert.Contains(t, foundURLs, expectedURL, \"Should contain configured URL: %s\", expectedURL)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid URL\",\n\t\t\turls: []string{\"not-a-url\"},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"HTTP URL (accepted during parsing)\",\n\t\t\turls: []string{\"http://insecure-publisher.com\"},\n\t\t\tvalidate: func(t *testing.T, resp *autoconf.Response) {\n\t\t\t\tassert.Len(t, resp.Config.DelegatedEndpoints, 1, \"Should have 1 delegated endpoint\")\n\t\t\t\tfor url := range resp.Config.DelegatedEndpoints {\n\t\t\t\t\tassert.Equal(t, \"http://insecure-publisher.com\", url, \"HTTP URL should be preserved during parsing\")\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tautoConfData := map[string]any{\n\t\t\t\t\"AutoConfVersion\": 2025072301,\n\t\t\t\t\"AutoConfSchema\": 1,\n\t\t\t\t\"AutoConfTTL\": 86400,\n\t\t\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\t\t\"TestSystem\": map[string]any{\n\t\t\t\t\t\t\"Description\": \"Test system for fuzz testing\",\n\t\t\t\t\t\t\"DelegatedConfig\": map[string]any{\n\t\t\t\t\t\t\t\"Read\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"DNSResolvers\": map[string]any{},\n\t\t\t\t\"DelegatedEndpoints\": map[string]any{},\n\t\t\t}\n\n\t\t\t// Add test URLs as delegated endpoints\n\t\t\tfor _, url := range tc.urls {\n\t\t\t\tautoConfData[\"DelegatedEndpoints\"].(map[string]any)[url] = map[string]any{\n\t\t\t\t\t\"Systems\": []string{\"TestSystem\"},\n\t\t\t\t\t\"Read\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t\t\"Write\": []string{\"/routing/v1/ipns\"},\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tjsonData, err := json.Marshal(autoConfData)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t\t_, _ = w.Write(jsonData)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\t// Test that our autoconf parser handles this gracefully\n\t\t\tautoConf, fallbackUsed := testAutoConfWithFallback(t, server.URL, tc.expectErr, fmt.Sprintf(\"Expected fallback to be used for %s\", tc.name))\n\n\t\t\tif !tc.expectErr {\n\t\t\t\trequire.NotNil(t, autoConf, \"AutoConf should not be nil for successful parsing\")\n\n\t\t\t\t// Run test-specific validation if provided (only for non-fallback cases)\n\t\t\t\tif tc.validate != nil && !fallbackUsed {\n\t\t\t\t\t// Create a mock Response for compatibility with validation functions\n\t\t\t\t\tmockResponse := &autoconf.Response{Config: autoConf}\n\t\t\t\t\ttc.validate(t, mockResponse)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc testFuzzMalformedJSON(t *testing.T) {\n\tmalformedJSONs := []string{\n\t\t`{`, // Incomplete JSON\n\t\t`{\"AutoConfVersion\": }`, // Missing value\n\t\t`{\"AutoConfVersion\": 123,}`, // Trailing comma\n\t\t`{AutoConfVersion: 123}`, // Unquoted key\n\t\t`{\"Bootstrap\": [}`, // Incomplete array\n\t\t`{\"Bootstrap\": [\"/test\",]}`, // Trailing comma in array\n\t\t`invalid json`, // Not JSON at all\n\t\t`null`, // Just null\n\t\t`[]`, // Array instead of object\n\t\t`\"\"`, // String instead of object\n\t}\n\n\tfor i, malformedJSON := range malformedJSONs {\n\t\tt.Run(fmt.Sprintf(\"malformed_%d\", i), func(t *testing.T) {\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t\t\t_, _ = w.Write([]byte(malformedJSON))\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\t// All malformed JSON should result in fallback usage\n\t\t\t_, _ = testAutoConfWithFallback(t, server.URL, true, fmt.Sprintf(\"Expected fallback to be used for malformed JSON: %s\", malformedJSON))\n\t\t})\n\t}\n}\n\nfunc testFuzzLargePayloads(t *testing.T) {\n\t// Test with very large but valid JSON payloads\n\tlargeBootstrap := make([]string, 10000)\n\tfor i := range largeBootstrap {\n\t\tlargeBootstrap[i] = fmt.Sprintf(\"/dnsaddr/bootstrap%d.example.com/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\", i)\n\t}\n\n\tlargeDNSResolvers := make(map[string][]string)\n\tfor i := range 1000 {\n\t\tdomain := fmt.Sprintf(\"domain%d.example.com\", i)\n\t\tlargeDNSResolvers[domain] = []string{\n\t\t\tfmt.Sprintf(\"https://resolver%d.example.com/dns-query\", i),\n\t\t}\n\t}\n\n\tconfig := map[string]any{\n\t\t\"AutoConfVersion\": 2025072301,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": map[string]any{\n\t\t\t\"AminoDHT\": map[string]any{\n\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\"NativeConfig\": map[string]any{\n\t\t\t\t\t\"Bootstrap\": largeBootstrap,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"DNSResolvers\": largeDNSResolvers,\n\t\t\"DelegatedEndpoints\": map[string]any{},\n\t}\n\n\tjsonData, err := json.Marshal(config)\n\trequire.NoError(t, err)\n\n\tt.Logf(\"Large payload size: %d bytes\", len(jsonData))\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write(jsonData)\n\t}))\n\tdefer server.Close()\n\n\t// Should handle large payloads gracefully (up to reasonable limits)\n\tautoConf, _ := testAutoConfWithFallbackAndTimeout(t, server.URL, false, \"Large payload should not trigger fallback\", 30*time.Second)\n\trequire.NotNil(t, autoConf, \"Should return valid config\")\n\n\t// Verify bootstrap entries were preserved\n\tbootstrapPeers := autoConf.GetBootstrapPeers(\"AminoDHT\")\n\trequire.Len(t, bootstrapPeers, 10000, \"Should preserve all bootstrap entries\")\n}\n\n// Helper function to generate many DNS resolvers for testing\nfunc generateManyResolvers(count int) map[string][]string {\n\tresolvers := make(map[string][]string)\n\tfor i := range count {\n\t\tdomain := fmt.Sprintf(\"domain%d.example.com\", i)\n\t\tresolvers[domain] = []string{\n\t\t\tfmt.Sprintf(\"https://resolver%d.example.com/dns-query\", i),\n\t\t}\n\t}\n\treturn resolvers\n}\n" }, { "path": "test/cli/autoconf/ipns_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"maps\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestAutoConfIPNS tests IPNS publishing with autoconf-resolved delegated publishers\nfunc TestAutoConfIPNS(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"PublishingWithWorkingEndpoint\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestIPNSPublishingWithWorkingEndpoint(t)\n\t})\n\n\tt.Run(\"PublishingResilience\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestIPNSPublishingResilience(t)\n\t})\n}\n\n// testIPNSPublishingWithWorkingEndpoint verifies that IPNS delegated publishing works\n// correctly when the HTTP endpoint is functioning normally and accepts requests.\n// It also verifies that the PUT payload matches what can be retrieved via routing get.\nfunc testIPNSPublishingWithWorkingEndpoint(t *testing.T) {\n\t// Create mock IPNS publisher that accepts requests\n\tpublisher := newMockIPNSPublisher(t)\n\tdefer publisher.close()\n\n\t// Create node with delegated publisher\n\tnode := setupNodeWithAutoconf(t, publisher.server.URL, \"auto\")\n\tdefer node.StopDaemon()\n\n\t// Wait for daemon to be ready\n\ttime.Sleep(5 * time.Second)\n\n\t// Get node's peer ID\n\tidResult := node.RunIPFS(\"id\", \"-f\", \"\")\n\trequire.Equal(t, 0, idResult.ExitCode())\n\tpeerID := strings.TrimSpace(idResult.Stdout.String())\n\n\t// Get peer ID in base36 format (used for IPNS keys)\n\tidBase36Result := node.RunIPFS(\"id\", \"--peerid-base\", \"base36\", \"-f\", \"\")\n\trequire.Equal(t, 0, idBase36Result.ExitCode())\n\tpeerIDBase36 := strings.TrimSpace(idBase36Result.Stdout.String())\n\n\t// Verify autoconf resolved \"auto\" correctly\n\tresult := node.RunIPFS(\"config\", \"Ipns.DelegatedPublishers\", \"--expand-auto\")\n\tvar resolvedPublishers []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &resolvedPublishers)\n\trequire.NoError(t, err)\n\texpectedURL := publisher.server.URL + \"/routing/v1/ipns\"\n\tassert.Contains(t, resolvedPublishers, expectedURL, \"AutoConf should resolve 'auto' to mock publisher\")\n\n\t// Test publishing with --allow-delegated\n\ttestCID := \"bafkqablimvwgy3y\"\n\tresult = node.RunIPFS(\"name\", \"publish\", \"--allow-delegated\", \"/ipfs/\"+testCID)\n\trequire.Equal(t, 0, result.ExitCode(), \"Publishing should succeed\")\n\tassert.Contains(t, result.Stdout.String(), \"Published to\")\n\n\t// Wait for async HTTP request to delegated publisher\n\ttime.Sleep(2 * time.Second)\n\n\t// Verify HTTP PUT was made to delegated publisher\n\tpublishedKeys := publisher.getPublishedKeys()\n\tassert.NotEmpty(t, publishedKeys, \"HTTP PUT request should have been made to delegated publisher\")\n\n\t// Get the PUT payload that was sent to the delegated publisher\n\tputPayload := publisher.getRecordPayload(peerIDBase36)\n\trequire.NotNil(t, putPayload, \"Should have captured PUT payload\")\n\trequire.Greater(t, len(putPayload), 0, \"PUT payload should not be empty\")\n\n\t// Retrieve the IPNS record using routing get\n\tgetResult := node.RunIPFS(\"routing\", \"get\", \"/ipns/\"+peerID)\n\trequire.Equal(t, 0, getResult.ExitCode(), \"Should be able to retrieve IPNS record\")\n\tgetPayload := getResult.Stdout.Bytes()\n\n\t// Compare the payloads\n\tassert.Equal(t, putPayload, getPayload,\n\t\t\"PUT payload sent to delegated publisher should match what routing get returns\")\n\n\t// Also verify the record points to the expected content\n\tassert.Contains(t, getResult.Stdout.String(), testCID,\n\t\t\"Retrieved IPNS record should reference the published CID\")\n\n\t// Use ipfs name inspect to verify the IPNS record's value matches the published CID\n\t// First write the routing get result to a file for inspection\n\tnode.WriteBytes(\"ipns-record\", getPayload)\n\tinspectResult := node.RunIPFS(\"name\", \"inspect\", \"ipns-record\")\n\trequire.Equal(t, 0, inspectResult.ExitCode(), \"Should be able to inspect IPNS record\")\n\n\t// The inspect output should show the path we published\n\tinspectOutput := inspectResult.Stdout.String()\n\tassert.Contains(t, inspectOutput, \"/ipfs/\"+testCID,\n\t\t\"IPNS record value should match the published path\")\n\n\t// Also verify it's a valid record with proper fields\n\tassert.Contains(t, inspectOutput, \"Value:\", \"Should have Value field\")\n\tassert.Contains(t, inspectOutput, \"Validity:\", \"Should have Validity field\")\n\tassert.Contains(t, inspectOutput, \"Sequence:\", \"Should have Sequence field\")\n\n\tt.Log(\"Verified: PUT payload to delegated publisher matches routing get result and name inspect confirms correct path\")\n}\n\n// testIPNSPublishingResilience verifies that IPNS publishing is resilient by design.\n// Publishing succeeds as long as local storage works, even when all delegated endpoints fail.\n// This test documents the intentional resilient behavior, not bugs.\nfunc testIPNSPublishingResilience(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\troutingType string // \"auto\" or \"delegated\"\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"AutoRouting\",\n\t\t\troutingType: \"auto\",\n\t\t\tdescription: \"auto mode uses DHT + HTTP, tolerates HTTP failures\",\n\t\t},\n\t\t{\n\t\t\tname: \"DelegatedRouting\",\n\t\t\troutingType: \"delegated\",\n\t\t\tdescription: \"delegated mode uses HTTP only, tolerates HTTP failures\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create publisher that always fails\n\t\t\tpublisher := newMockIPNSPublisher(t)\n\t\t\tdefer publisher.close()\n\t\t\tpublisher.responseFunc = func(peerID string, record []byte) int {\n\t\t\t\treturn http.StatusInternalServerError\n\t\t\t}\n\n\t\t\t// Create node with failing endpoint\n\t\t\tnode := setupNodeWithAutoconf(t, publisher.server.URL, tc.routingType)\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Test different publishing modes - all should succeed due to resilient design\n\t\t\ttestCID := \"/ipfs/bafkqablimvwgy3y\"\n\n\t\t\t// Normal publishing (should succeed despite endpoint failures)\n\t\t\tresult := node.RunIPFS(\"name\", \"publish\", testCID)\n\t\t\tassert.Equal(t, 0, result.ExitCode(),\n\t\t\t\t\"%s: Normal publishing should succeed (local storage works)\", tc.description)\n\n\t\t\t// Publishing with --allow-offline (local only, no network)\n\t\t\tresult = node.RunIPFS(\"name\", \"publish\", \"--allow-offline\", testCID)\n\t\t\tassert.Equal(t, 0, result.ExitCode(),\n\t\t\t\t\"--allow-offline should succeed (local only)\")\n\n\t\t\t// Publishing with --allow-delegated (if using auto routing)\n\t\t\tif tc.routingType == \"auto\" {\n\t\t\t\tresult = node.RunIPFS(\"name\", \"publish\", \"--allow-delegated\", testCID)\n\t\t\t\tassert.Equal(t, 0, result.ExitCode(),\n\t\t\t\t\t\"--allow-delegated should succeed (no DHT required)\")\n\t\t\t}\n\n\t\t\tt.Logf(\"%s: All publishing modes succeeded despite endpoint failures (resilient design)\", tc.name)\n\t\t})\n\t}\n}\n\n// ============================================================================\n// Helper Functions\n// ============================================================================\n\n// setupNodeWithAutoconf creates an IPFS node with autoconf-configured delegated publishers\nfunc setupNodeWithAutoconf(t *testing.T, publisherURL string, routingType string) *harness.Node {\n\t// Create autoconf server with the publisher endpoint\n\tautoconfData := createAutoconfJSON(publisherURL)\n\tautoconfServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tfmt.Fprint(w, autoconfData)\n\t}))\n\tt.Cleanup(func() { autoconfServer.Close() })\n\n\t// Create and configure node\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init(\"--profile=test\")\n\n\t// Configure autoconf\n\tnode.SetIPFSConfig(\"AutoConf.URL\", autoconfServer.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Ipns.DelegatedPublishers\", []string{\"auto\"})\n\tnode.SetIPFSConfig(\"Routing.Type\", routingType)\n\n\t// Additional config for delegated routing mode\n\tif routingType == \"delegated\" {\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", false)\n\t\tnode.SetIPFSConfig(\"Provide.DHT.Interval\", \"0s\")\n\t}\n\n\t// Add bootstrap peers for connectivity\n\tnode.SetIPFSConfig(\"Bootstrap\", autoconf.FallbackBootstrapPeers)\n\n\t// Start daemon\n\tnode.StartDaemon()\n\n\treturn node\n}\n\n// createAutoconfJSON generates autoconf configuration with a delegated IPNS publisher\nfunc createAutoconfJSON(publisherURL string) string {\n\t// Use bootstrap peers from autoconf fallbacks for consistency\n\tbootstrapPeers, _ := json.Marshal(autoconf.FallbackBootstrapPeers)\n\n\treturn fmt.Sprintf(`{\n\t\t\"AutoConfVersion\": 2025072302,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": {\n\t\t\t\"TestSystem\": {\n\t\t\t\t\"Description\": \"Test system for IPNS publishing\",\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": %s\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t\t\"DNSResolvers\": {},\n\t\t\"DelegatedEndpoints\": {\n\t\t\t\"%s\": {\n\t\t\t\t\"Systems\": [\"TestSystem\"],\n\t\t\t\t\"Read\": [\"/routing/v1/ipns\"],\n\t\t\t\t\"Write\": [\"/routing/v1/ipns\"]\n\t\t\t}\n\t\t}\n\t}`, string(bootstrapPeers), publisherURL)\n}\n\n// ============================================================================\n// Mock IPNS Publisher\n// ============================================================================\n\n// mockIPNSPublisher implements a simple IPNS publishing HTTP API server\ntype mockIPNSPublisher struct {\n\tt *testing.T\n\tserver *httptest.Server\n\tmu sync.Mutex\n\tpublishedKeys map[string]string // peerID -> published CID\n\trecordPayloads map[string][]byte // peerID -> actual HTTP PUT record payload\n\tresponseFunc func(peerID string, record []byte) int // returns HTTP status code\n}\n\nfunc newMockIPNSPublisher(t *testing.T) *mockIPNSPublisher {\n\tm := &mockIPNSPublisher{\n\t\tt: t,\n\t\tpublishedKeys: make(map[string]string),\n\t\trecordPayloads: make(map[string][]byte),\n\t}\n\n\t// Default response function accepts all publishes\n\tm.responseFunc = func(peerID string, record []byte) int {\n\t\treturn http.StatusOK\n\t}\n\n\tmux := http.NewServeMux()\n\tmux.HandleFunc(\"/routing/v1/ipns/\", m.handleIPNS)\n\n\tm.server = httptest.NewServer(mux)\n\treturn m\n}\n\nfunc (m *mockIPNSPublisher) handleIPNS(w http.ResponseWriter, r *http.Request) {\n\tm.mu.Lock()\n\tdefer m.mu.Unlock()\n\n\t// Extract peer ID from path\n\tparts := strings.Split(r.URL.Path, \"/\")\n\tif len(parts) < 5 {\n\t\thttp.Error(w, \"invalid path\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\tpeerID := parts[4]\n\n\tif r.Method == \"PUT\" {\n\t\t// Handle IPNS record publication\n\t\tbody, err := io.ReadAll(r.Body)\n\t\tif err != nil {\n\t\t\thttp.Error(w, \"failed to read body\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\t// Get response status from response function\n\t\tstatus := m.responseFunc(peerID, body)\n\n\t\tif status == http.StatusOK {\n\t\t\tif len(body) > 0 {\n\t\t\t\t// Store the actual record payload\n\t\t\t\tm.recordPayloads[peerID] = make([]byte, len(body))\n\t\t\t\tcopy(m.recordPayloads[peerID], body)\n\t\t\t}\n\n\t\t\t// Mark as published\n\t\t\tm.publishedKeys[peerID] = fmt.Sprintf(\"published-%d\", time.Now().Unix())\n\t\t}\n\n\t\tw.WriteHeader(status)\n\t\tif status != http.StatusOK {\n\t\t\tfmt.Fprint(w, `{\"error\": \"publish failed\"}`)\n\t\t}\n\t} else if r.Method == \"GET\" {\n\t\t// Handle IPNS record retrieval\n\t\tif record, exists := m.publishedKeys[peerID]; exists {\n\t\t\tw.Header().Set(\"Content-Type\", \"application/vnd.ipfs.ipns-record\")\n\t\t\tfmt.Fprint(w, record)\n\t\t} else {\n\t\t\thttp.Error(w, \"record not found\", http.StatusNotFound)\n\t\t}\n\t} else {\n\t\thttp.Error(w, \"method not allowed\", http.StatusMethodNotAllowed)\n\t}\n}\n\nfunc (m *mockIPNSPublisher) getPublishedKeys() map[string]string {\n\tm.mu.Lock()\n\tdefer m.mu.Unlock()\n\tresult := make(map[string]string)\n\tmaps.Copy(result, m.publishedKeys)\n\treturn result\n}\n\nfunc (m *mockIPNSPublisher) getRecordPayload(peerID string) []byte {\n\tm.mu.Lock()\n\tdefer m.mu.Unlock()\n\tif payload, exists := m.recordPayloads[peerID]; exists {\n\t\tresult := make([]byte, len(payload))\n\t\tcopy(result, payload)\n\t\treturn result\n\t}\n\treturn nil\n}\n\nfunc (m *mockIPNSPublisher) close() {\n\tm.server.Close()\n}\n" }, { "path": "test/cli/autoconf/routing_test.go", "content": "package autoconf\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAutoConfDelegatedRouting(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"delegated routing with auto router\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestDelegatedRoutingWithAuto(t)\n\t})\n\n\tt.Run(\"routing errors are handled properly\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestRoutingErrorHandling(t)\n\t})\n}\n\n// mockRoutingServer implements a simple Delegated Routing HTTP API server\ntype mockRoutingServer struct {\n\tt *testing.T\n\tserver *httptest.Server\n\tmu sync.Mutex\n\trequests []string\n\tproviderFunc func(cid string) []map[string]any\n}\n\nfunc newMockRoutingServer(t *testing.T) *mockRoutingServer {\n\tm := &mockRoutingServer{\n\t\tt: t,\n\t\trequests: []string{},\n\t}\n\n\t// Default provider function returns mock provider records\n\tm.providerFunc = func(cid string) []map[string]any {\n\t\treturn []map[string]any{\n\t\t\t{\n\t\t\t\t\"Protocol\": \"transport-bitswap\",\n\t\t\t\t\"Schema\": \"bitswap\",\n\t\t\t\t\"ID\": \"12D3KooWMockProvider1\",\n\t\t\t\t\"Addrs\": []string{\"/ip4/192.168.1.100/tcp/4001\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\t\"Protocol\": \"transport-bitswap\",\n\t\t\t\t\"Schema\": \"bitswap\",\n\t\t\t\t\"ID\": \"12D3KooWMockProvider2\",\n\t\t\t\t\"Addrs\": []string{\"/ip4/192.168.1.101/tcp/4001\"},\n\t\t\t},\n\t\t}\n\t}\n\n\tmux := http.NewServeMux()\n\tmux.HandleFunc(\"/routing/v1/providers/\", m.handleProviders)\n\n\tm.server = httptest.NewServer(mux)\n\treturn m\n}\n\nfunc (m *mockRoutingServer) handleProviders(w http.ResponseWriter, r *http.Request) {\n\tm.mu.Lock()\n\tdefer m.mu.Unlock()\n\n\t// Extract CID from path\n\tparts := strings.Split(r.URL.Path, \"/\")\n\tif len(parts) < 5 {\n\t\thttp.Error(w, \"invalid path\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\tcid := parts[4]\n\tm.requests = append(m.requests, cid)\n\tm.t.Logf(\"Routing server received providers request for CID: %s\", cid)\n\n\t// Get provider records\n\tproviders := m.providerFunc(cid)\n\n\t// Return NDJSON response as per IPIP-378\n\tw.Header().Set(\"Content-Type\", \"application/x-ndjson\")\n\tencoder := json.NewEncoder(w)\n\n\tfor _, provider := range providers {\n\t\tif err := encoder.Encode(provider); err != nil {\n\t\t\tm.t.Logf(\"Failed to encode provider: %v\", err)\n\t\t\treturn\n\t\t}\n\t}\n}\n\nfunc (m *mockRoutingServer) close() {\n\tm.server.Close()\n}\n\nfunc testDelegatedRoutingWithAuto(t *testing.T) {\n\t// Create mock routing server\n\troutingServer := newMockRoutingServer(t)\n\tdefer routingServer.close()\n\n\t// Create autoconf data with delegated router\n\tautoConfData := fmt.Sprintf(`{\n\t\t\"AutoConfVersion\": 2025072302,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": []\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t\t\"DNSResolvers\": {},\n\t\t\"DelegatedEndpoints\": {\n\t\t\t\"%s\": {\n\t\t\t\t\"Systems\": [\"AminoDHT\", \"IPNI\"],\n\t\t\t\t\"Read\": [\"/routing/v1/providers\", \"/routing/v1/peers\", \"/routing/v1/ipns\"],\n\t\t\t\t\"Write\": []\n\t\t\t}\n\t\t}\n\t}`, routingServer.server.URL)\n\n\t// Create autoconf server\n\tautoConfServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(autoConfData))\n\t}))\n\tdefer autoConfServer.Close()\n\n\t// Create IPFS node with auto delegated router\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", autoConfServer.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\n\t// Test that daemon starts successfully with auto routing configuration\n\t// The actual routing functionality requires online mode, but we can test\n\t// that the configuration is expanded and daemon starts properly\n\tnode.StartDaemon(\"--offline\")\n\tdefer node.StopDaemon()\n\n\t// Verify config still shows \"auto\" (this tests that auto values are preserved in user-facing config)\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar routers []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &routers)\n\trequire.NoError(t, err)\n\tassert.Equal(t, []string{\"auto\"}, routers, \"Delegated routers config should show 'auto'\")\n\n\t// Test that daemon is running and accepting commands\n\tresult = node.RunIPFS(\"version\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Daemon should be running and accepting commands\")\n\n\t// Test that autoconf server was contacted (indicating successful resolution)\n\t// We can't test actual routing in offline mode, but we can verify that\n\t// the AutoConf system expanded the \"auto\" placeholder successfully\n\t// by checking that the daemon started without errors\n\tt.Log(\"AutoConf successfully expanded delegated router configuration and daemon started\")\n}\n\nfunc testRoutingErrorHandling(t *testing.T) {\n\t// Create routing server that returns no providers\n\troutingServer := newMockRoutingServer(t)\n\tdefer routingServer.close()\n\n\t// Configure to return no providers (empty response)\n\troutingServer.providerFunc = func(cid string) []map[string]any {\n\t\treturn []map[string]any{}\n\t}\n\n\t// Create autoconf data\n\tautoConfData := fmt.Sprintf(`{\n\t\t\"AutoConfVersion\": 2025072302,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"AutoConfTTL\": 86400,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"Description\": \"Test AminoDHT system\",\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": []\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t\t\"DNSResolvers\": {},\n\t\t\"DelegatedEndpoints\": {\n\t\t\t\"%s\": {\n\t\t\t\t\"Systems\": [\"AminoDHT\", \"IPNI\"],\n\t\t\t\t\"Read\": [\"/routing/v1/providers\", \"/routing/v1/peers\", \"/routing/v1/ipns\"],\n\t\t\t\t\"Write\": []\n\t\t\t}\n\t\t}\n\t}`, routingServer.server.URL)\n\n\t// Create autoconf server\n\tautoConfServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(autoConfData))\n\t}))\n\tdefer autoConfServer.Close()\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", autoConfServer.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Routing.DelegatedRouters\", []string{\"auto\"})\n\n\t// Test that daemon starts successfully even when no providers are available\n\tnode.StartDaemon(\"--offline\")\n\tdefer node.StopDaemon()\n\n\t// Verify config shows \"auto\"\n\tresult := node.RunIPFS(\"config\", \"Routing.DelegatedRouters\")\n\trequire.Equal(t, 0, result.ExitCode())\n\n\tvar routers []string\n\terr := json.Unmarshal([]byte(result.Stdout.String()), &routers)\n\trequire.NoError(t, err)\n\tassert.Equal(t, []string{\"auto\"}, routers, \"Delegated routers config should show 'auto'\")\n\n\t// Test that daemon is running and accepting commands\n\tresult = node.RunIPFS(\"version\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Daemon should be running even with empty routing config\")\n\n\tt.Log(\"AutoConf successfully handled routing configuration with empty providers\")\n}\n" }, { "path": "test/cli/autoconf/swarm_connect_test.go", "content": "package autoconf\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestSwarmConnectWithAutoConf tests that ipfs swarm connect works properly\n// when AutoConf is enabled and a daemon is running.\n//\n// This is a regression test for the issue where:\n// - AutoConf disabled: ipfs swarm connect works\n// - AutoConf enabled: ipfs swarm connect fails with \"Error: connect\"\n//\n// The issue affects CLI command fallback behavior when the HTTP API connection fails.\nfunc TestSwarmConnectWithAutoConf(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"AutoConf disabled - should work\", func(t *testing.T) {\n\t\ttestSwarmConnectWithAutoConfSetting(t, false, true) // expect success\n\t})\n\n\tt.Run(\"AutoConf enabled - should work\", func(t *testing.T) {\n\t\ttestSwarmConnectWithAutoConfSetting(t, true, true) // expect success (fix the bug!)\n\t})\n}\n\nfunc testSwarmConnectWithAutoConfSetting(t *testing.T, autoConfEnabled bool, expectSuccess bool) {\n\t// Create IPFS node with test profile\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t// Configure AutoConf\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", autoConfEnabled)\n\n\t// Set up bootstrap peers so the node has something to connect to\n\t// Use the same bootstrap peers from boxo/autoconf fallbacks\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n\t})\n\n\t// CRITICAL: Start the daemon first - this is the key requirement\n\t// The daemon must be running and working properly\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Give daemon time to start up completely\n\ttime.Sleep(3 * time.Second)\n\n\t// Verify daemon is responsive\n\tresult := node.RunIPFS(\"id\")\n\trequire.Equal(t, 0, result.ExitCode(), \"Daemon should be responsive before testing swarm connect\")\n\tt.Logf(\"Daemon is running and responsive. AutoConf enabled: %v\", autoConfEnabled)\n\n\t// Now test swarm connect to a bootstrap peer\n\t// This should work because:\n\t// 1. The daemon is running\n\t// 2. The CLI should connect to the daemon via API\n\t// 3. The daemon should handle the swarm connect request\n\tresult = node.RunIPFS(\"swarm\", \"connect\", \"/dnsaddr/bootstrap.libp2p.io\")\n\n\t// swarm connect should work regardless of AutoConf setting\n\tassert.Equal(t, 0, result.ExitCode(),\n\t\t\"swarm connect should succeed with AutoConf=%v. stderr: %s\",\n\t\tautoConfEnabled, result.Stderr.String())\n\n\t// Should contain success message\n\toutput := result.Stdout.String()\n\tassert.Contains(t, output, \"success\",\n\t\t\"swarm connect output should contain 'success' with AutoConf=%v. output: %s\",\n\t\tautoConfEnabled, output)\n\n\t// Additional diagnostic: Check if ipfs id shows addresses\n\t// Both AutoConf enabled and disabled should show proper addresses\n\tresult = node.RunIPFS(\"id\")\n\trequire.Equal(t, 0, result.ExitCode(), \"ipfs id should work with AutoConf=%v\", autoConfEnabled)\n\n\tidOutput := result.Stdout.String()\n\tt.Logf(\"ipfs id output with AutoConf=%v: %s\", autoConfEnabled, idOutput)\n\n\t// Addresses should not be null regardless of AutoConf setting\n\tassert.Contains(t, idOutput, `\"Addresses\"`, \"ipfs id should show Addresses field\")\n\tassert.NotContains(t, idOutput, `\"Addresses\": null`,\n\t\t\"ipfs id should not show null addresses with AutoConf=%v\", autoConfEnabled)\n}\n" }, { "path": "test/cli/autoconf/testdata/autoconf_amino_and_ipni.json", "content": "{\n \"AutoConfVersion\": 2025072901,\n \"AutoConfSchema\": 1,\n \"AutoConfTTL\": 86400,\n \"SystemRegistry\": {\n \"AminoDHT\": {\n \"URL\": \"https://github.com/ipfs/specs/pull/497\",\n \"Description\": \"Public DHT swarm that implements the IPFS Kademlia DHT specification under protocol identifier /ipfs/kad/1.0.0\",\n \"NativeConfig\": {\n \"Bootstrap\": [\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"\n ]\n },\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n },\n \"IPNI\": {\n \"URL\": \"https://cid.contact\",\n \"Description\": \"Network Indexer - content routing database for large storage providers\",\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n }\n }\n },\n \"DNSResolvers\": {\n \"eth.\": [\n \"https://dns.eth.limo/dns-query\"\n ]\n },\n \"DelegatedEndpoints\": {\n \"https://amino-dht.example.com\": {\n \"Systems\": [\"AminoDHT\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n },\n \"https://cid.contact\": {\n \"Systems\": [\"IPNI\"],\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n }\n }\n}" }, { "path": "test/cli/autoconf/testdata/autoconf_new_routing_system.json", "content": "{\n \"AutoConfVersion\": 2025072901,\n \"AutoConfSchema\": 1,\n \"AutoConfTTL\": 86400,\n \"SystemRegistry\": {\n \"NewRoutingSystem\": {\n \"URL\": \"https://new-routing.example.com\",\n \"Description\": \"New routing system for testing delegation with auto routing\",\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n },\n \"DNSResolvers\": {\n \"eth.\": [\n \"https://dns.eth.limo/dns-query\"\n ]\n },\n \"DelegatedEndpoints\": {\n \"https://new-routing.example.com\": {\n \"Systems\": [\"NewRoutingSystem\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n}" }, { "path": "test/cli/autoconf/testdata/autoconf_new_routing_with_filtering.json", "content": "{\n \"AutoConfVersion\": 2025072901,\n \"AutoConfSchema\": 1,\n \"AutoConfTTL\": 86400,\n \"SystemRegistry\": {\n \"NewRoutingSystem\": {\n \"URL\": \"https://new-routing.example.com\",\n \"Description\": \"New routing system for testing path filtering with auto routing\",\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n },\n \"DNSResolvers\": {\n \"eth.\": [\n \"https://dns.eth.limo/dns-query\"\n ]\n },\n \"DelegatedEndpoints\": {\n \"https://supported-new.example.com\": {\n \"Systems\": [\"NewRoutingSystem\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n },\n \"https://unsupported-new.example.com\": {\n \"Systems\": [\"NewRoutingSystem\"],\n \"Read\": [\n \"/custom/v0/read\",\n \"/api/v1/nonstandard\"\n ],\n \"Write\": [\n \"/custom/v0/write\"\n ]\n },\n \"https://mixed-new.example.com\": {\n \"Systems\": [\"NewRoutingSystem\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/invalid/path\",\n \"/routing/v1/peers\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n}" }, { "path": "test/cli/autoconf/testdata/autoconf_with_unsupported_paths.json", "content": "{\n \"AutoConfVersion\": 2025072901,\n \"AutoConfSchema\": 1,\n \"AutoConfTTL\": 86400,\n \"SystemRegistry\": {\n \"AminoDHT\": {\n \"URL\": \"https://github.com/ipfs/specs/pull/497\",\n \"Description\": \"Public DHT swarm that implements the IPFS Kademlia DHT specification under protocol identifier /ipfs/kad/1.0.0\",\n \"NativeConfig\": {\n \"Bootstrap\": [\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"\n ]\n },\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n },\n \"DNSResolvers\": {\n \"eth.\": [\n \"https://dns.eth.limo/dns-query\"\n ]\n },\n \"DelegatedEndpoints\": {\n \"https://supported.example.com\": {\n \"Systems\": [\"AminoDHT\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n },\n \"https://unsupported.example.com\": {\n \"Systems\": [\"AminoDHT\"],\n \"Read\": [\n \"/example/v0/read\",\n \"/api/v1/custom\"\n ],\n \"Write\": [\n \"/example/v0/write\"\n ]\n },\n \"https://mixed.example.com\": {\n \"Systems\": [\"AminoDHT\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/unsupported/path\",\n \"/routing/v1/peers\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n}\n" }, { "path": "test/cli/autoconf/testdata/updated_autoconf.json", "content": "{\n \"AutoConfVersion\": 2025072902,\n \"AutoConfSchema\": 1,\n \"AutoConfTTL\": 86400,\n \"SystemRegistry\": {\n \"AminoDHT\": {\n \"URL\": \"https://github.com/ipfs/specs/pull/497\",\n \"Description\": \"Public DHT swarm that implements the IPFS Kademlia DHT specification under protocol identifier /ipfs/kad/1.0.0\",\n \"NativeConfig\": {\n \"Bootstrap\": [\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n \"/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8\",\n \"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n \"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n ]\n },\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n },\n \"IPNI\": {\n \"URL\": \"https://ipni.example.com\",\n \"Description\": \"Network Indexer - content routing database for large storage providers\",\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n }\n }\n },\n \"DNSResolvers\": {\n \"eth.\": [\n \"https://dns.eth.limo/dns-query\",\n \"https://dns.eth.link/dns-query\"\n ],\n \"test.\": [\n \"https://test.resolver/dns-query\"\n ]\n },\n \"DelegatedEndpoints\": {\n \"https://ipni.example.com\": {\n \"Systems\": [\"IPNI\"],\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n },\n \"https://routing.example.com\": {\n \"Systems\": [\"IPNI\"],\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n },\n \"https://delegated-ipfs.dev\": {\n \"Systems\": [\"AminoDHT\", \"IPNI\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n },\n \"https://ipns.example.com\": {\n \"Systems\": [\"AminoDHT\"],\n \"Read\": [\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n}" }, { "path": "test/cli/autoconf/testdata/valid_autoconf.json", "content": "{\n \"AutoConfVersion\": 2025072901,\n \"AutoConfSchema\": 1,\n \"AutoConfTTL\": 86400,\n \"SystemRegistry\": {\n \"AminoDHT\": {\n \"URL\": \"https://github.com/ipfs/specs/pull/497\",\n \"Description\": \"Public DHT swarm that implements the IPFS Kademlia DHT specification under protocol identifier /ipfs/kad/1.0.0\",\n \"NativeConfig\": {\n \"Bootstrap\": [\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n \"/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8\",\n \"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n \"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n ]\n },\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n },\n \"IPNI\": {\n \"URL\": \"https://ipni.example.com\",\n \"Description\": \"Network Indexer - content routing database for large storage providers\",\n \"DelegatedConfig\": {\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n }\n }\n },\n \"DNSResolvers\": {\n \"eth.\": [\n \"https://dns.eth.limo/dns-query\",\n \"https://dns.eth.link/dns-query\"\n ]\n },\n \"DelegatedEndpoints\": {\n \"https://ipni.example.com\": {\n \"Systems\": [\"IPNI\"],\n \"Read\": [\n \"/routing/v1/providers\"\n ],\n \"Write\": []\n },\n \"https://delegated-ipfs.dev\": {\n \"Systems\": [\"AminoDHT\", \"IPNI\"],\n \"Read\": [\n \"/routing/v1/providers\",\n \"/routing/v1/peers\",\n \"/routing/v1/ipns\"\n ],\n \"Write\": [\n \"/routing/v1/ipns\"\n ]\n }\n }\n}" }, { "path": "test/cli/autoconf/validation_test.go", "content": "package autoconf\n\nimport (\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestAutoConfValidation(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"invalid autoconf JSON prevents caching\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestInvalidAutoConfJSONPreventsCaching(t)\n\t})\n\n\tt.Run(\"malformed multiaddr in autoconf\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestMalformedMultiaddrInAutoConf(t)\n\t})\n\n\tt.Run(\"malformed URL in autoconf\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestMalformedURLInAutoConf(t)\n\t})\n}\n\nfunc testInvalidAutoConfJSONPreventsCaching(t *testing.T) {\n\t// Create server that serves invalid autoconf JSON\n\tinvalidAutoConfData := `{\n\t\t\"AutoConfVersion\": 123,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": [\n\t\t\t\t\t\t\"invalid-multiaddr-that-should-fail\"\n\t\t\t\t\t]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}`\n\n\trequestCount := 0\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\trequestCount++\n\t\tt.Logf(\"Invalid autoconf server request #%d: %s %s\", requestCount, r.Method, r.URL.Path)\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tw.Header().Set(\"ETag\", `\"invalid-config-123\"`)\n\t\t_, _ = w.Write([]byte(invalidAutoConfData))\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node and try to start daemon with invalid autoconf\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Start daemon to trigger autoconf fetch - this should start but log validation errors\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Give autoconf some time to attempt fetch and fail validation\n\t// The daemon should still start but autoconf should fail\n\tresult := node.RunIPFS(\"version\")\n\tassert.Equal(t, 0, result.ExitCode(), \"Daemon should start even with invalid autoconf\")\n\n\t// Verify server was called (autoconf was attempted even though validation failed)\n\tassert.Greater(t, requestCount, 0, \"Invalid autoconf server should have been called\")\n}\n\nfunc testMalformedMultiaddrInAutoConf(t *testing.T) {\n\t// Create server that serves autoconf with malformed multiaddr\n\tinvalidAutoConfData := `{\n\t\t\"AutoConfVersion\": 456,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"SystemRegistry\": {\n\t\t\t\"AminoDHT\": {\n\t\t\t\t\"NativeConfig\": {\n\t\t\t\t\t\"Bootstrap\": [\n\t\t\t\t\t\t\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n\t\t\t\t\t\t\"not-a-valid-multiaddr\"\n\t\t\t\t\t]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}`\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(invalidAutoConfData))\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"})\n\n\t// Start daemon to trigger autoconf fetch - daemon should start but autoconf validation should fail\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Daemon should still be functional even with invalid autoconf\n\tresult := node.RunIPFS(\"version\")\n\tassert.Equal(t, 0, result.ExitCode(), \"Daemon should start even with invalid autoconf\")\n}\n\nfunc testMalformedURLInAutoConf(t *testing.T) {\n\t// Create server that serves autoconf with malformed URL\n\tinvalidAutoConfData := `{\n\t\t\"AutoConfVersion\": 789,\n\t\t\"AutoConfSchema\": 1,\n\t\t\"DNSResolvers\": {\n\t\t\t\"eth.\": [\"https://valid.example.com\"],\n\t\t\t\"bad.\": [\"://malformed-url-missing-scheme\"]\n\t\t}\n\t}`\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\t_, _ = w.Write([]byte(invalidAutoConfData))\n\t}))\n\tdefer server.Close()\n\n\t// Create IPFS node\n\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\tnode.SetIPFSConfig(\"AutoConf.URL\", server.URL)\n\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\"foo.\": \"auto\"})\n\n\t// Start daemon to trigger autoconf fetch - daemon should start but autoconf validation should fail\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Daemon should still be functional even with invalid autoconf\n\tresult := node.RunIPFS(\"version\")\n\tassert.Equal(t, 0, result.ExitCode(), \"Daemon should start even with invalid autoconf\")\n}\n" }, { "path": "test/cli/backup_bootstrap_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestBackupBootstrapPeers(t *testing.T) {\n\tnodes := harness.NewT(t).NewNodes(3).Init()\n\tnodes.ForEachPar(func(n *harness.Node) {\n\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Bootstrap = []string{}\n\t\t\tcfg.Addresses.Swarm = []string{fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", harness.NewRandPort())}\n\t\t\tcfg.Discovery.MDNS.Enabled = false\n\t\t\tcfg.Internal.BackupBootstrapInterval = config.NewOptionalDuration(250 * time.Millisecond)\n\t\t})\n\t})\n\n\t// Start all nodes and ensure they all have no peers.\n\tnodes.StartDaemons()\n\tnodes.ForEachPar(func(n *harness.Node) {\n\t\tassert.Len(t, n.Peers(), 0)\n\t})\n\n\t// Connect nodes 0 and 1, ensure they know each other.\n\tnodes[0].Connect(nodes[1])\n\tassert.Len(t, nodes[0].Peers(), 1)\n\tassert.Len(t, nodes[1].Peers(), 1)\n\tassert.Len(t, nodes[2].Peers(), 0)\n\n\t// Wait a bit to ensure that 0 and 1 saved their temporary bootstrap backups.\n\ttime.Sleep(time.Millisecond * 500)\n\tnodes.StopDaemons()\n\n\t// Start 1 and 2. 2 does not know anyone yet.\n\tnodes[1].StartDaemon()\n\tdefer nodes[1].StopDaemon()\n\tnodes[2].StartDaemon()\n\tdefer nodes[2].StopDaemon()\n\tassert.Len(t, nodes[1].Peers(), 0)\n\tassert.Len(t, nodes[2].Peers(), 0)\n\n\t// Connect 1 and 2, ensure they know each other.\n\tnodes[1].Connect(nodes[2])\n\tassert.Len(t, nodes[1].Peers(), 1)\n\tassert.Len(t, nodes[2].Peers(), 1)\n\n\t// Start 0, wait a bit. Should connect to 1, and then discover 2 via the\n\t// backup bootstrap peers.\n\tnodes[0].StartDaemon()\n\tdefer nodes[0].StopDaemon()\n\ttime.Sleep(time.Millisecond * 500)\n\n\t// Check if they're all connected.\n\tassert.Len(t, nodes[0].Peers(), 2)\n\tassert.Len(t, nodes[1].Peers(), 2)\n\tassert.Len(t, nodes[2].Peers(), 2)\n}\n" }, { "path": "test/cli/basic_commands_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"regexp\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/blang/semver/v4\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\tgomod \"golang.org/x/mod/module\"\n)\n\nvar versionRegexp = regexp.MustCompile(`^ipfs version (.+)$`)\n\nfunc parseVersionOutput(s string) semver.Version {\n\tversString := versionRegexp.FindStringSubmatch(s)[1]\n\tv, err := semver.Parse(versString)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn v\n}\n\nfunc TestCurDirIsWritable(t *testing.T) {\n\tt.Parallel()\n\th := harness.NewT(t)\n\th.WriteFile(\"test.txt\", \"It works!\")\n}\n\nfunc TestIPFSVersionCommandMatchesFlag(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tcommandVersionStr := node.IPFS(\"version\").Stdout.String()\n\tcommandVersionStr = strings.TrimSpace(commandVersionStr)\n\tcommandVersion := parseVersionOutput(commandVersionStr)\n\n\tflagVersionStr := node.IPFS(\"--version\").Stdout.String()\n\tflagVersionStr = strings.TrimSpace(flagVersionStr)\n\tflagVersion := parseVersionOutput(flagVersionStr)\n\n\tassert.Equal(t, commandVersion, flagVersion)\n}\n\nfunc TestIPFSVersionAll(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tres := node.IPFS(\"version\", \"--all\").Stdout.String()\n\tres = strings.TrimSpace(res)\n\tassert.Contains(t, res, \"Kubo version\")\n\tassert.Contains(t, res, \"Repo version\")\n\tassert.Contains(t, res, \"System version\")\n\tassert.Contains(t, res, \"Golang version\")\n}\n\nfunc TestIPFSVersionDeps(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tres := node.IPFS(\"version\", \"deps\").Stdout.String()\n\tres = strings.TrimSpace(res)\n\tlines := SplitLines(res)\n\n\tassert.True(t, strings.HasPrefix(lines[0], \"github.com/ipfs/kubo@v\"))\n\n\tfor _, depLine := range lines[1:] {\n\t\tsplit := strings.SplitSeq(depLine, \" => \")\n\t\tfor moduleVersion := range split {\n\t\t\tsplitModVers := strings.Split(moduleVersion, \"@\")\n\t\t\tmodPath := splitModVers[0]\n\t\t\tmodVers := splitModVers[1]\n\t\t\t// Skip local replace paths (starting with \"./\")\n\t\t\tif strings.HasPrefix(modPath, \"./\") {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tassert.NoError(t, gomod.Check(modPath, modVers), \"path: %s, version: %s\", modPath, modVers)\n\t\t}\n\t}\n}\n\nfunc TestIPFSCommands(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tcmds := node.IPFSCommands()\n\tassert.Contains(t, cmds, \"ipfs add\")\n\tassert.Contains(t, cmds, \"ipfs daemon\")\n\tassert.Contains(t, cmds, \"ipfs update\")\n}\n\nfunc TestAllSubcommandsAcceptHelp(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tfor _, cmd := range node.IPFSCommands() {\n\t\tt.Run(fmt.Sprintf(\"command %q accepts help\", cmd), func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tsplitCmd := strings.Split(cmd, \" \")[1:]\n\t\t\tnode.IPFS(StrCat(\"help\", splitCmd)...)\n\t\t\tnode.IPFS(StrCat(splitCmd, \"--help\")...)\n\t\t})\n\t}\n}\n\nfunc TestAllRootCommandsAreMentionedInHelpText(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tcmds := node.IPFSCommands()\n\tvar rootCmds []string\n\tfor _, cmd := range cmds {\n\t\tsplitCmd := strings.Split(cmd, \" \")\n\t\tif len(splitCmd) == 2 {\n\t\t\trootCmds = append(rootCmds, splitCmd[1])\n\t\t}\n\t}\n\n\t// a few base commands are not expected to be in the help message\n\t// but we default to requiring them to be in the help message, so that we\n\t// have to make a conscious decision to exclude them\n\tnotInHelp := map[string]bool{\n\t\t\"object\": true,\n\t\t\"shutdown\": true,\n\t}\n\n\thelpMsg := strings.TrimSpace(node.IPFS(\"--help\").Stdout.String())\n\tfor _, rootCmd := range rootCmds {\n\t\tif _, ok := notInHelp[rootCmd]; ok {\n\t\t\tcontinue\n\t\t}\n\t\tassert.Contains(t, helpMsg, fmt.Sprintf(\" %s\", rootCmd))\n\t}\n}\n\nfunc TestCommandDocsWidth(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\t// require new commands to explicitly opt in to longer lines\n\tallowList := map[string]bool{\n\t\t\"ipfs add\": true,\n\t\t\"ipfs block put\": true,\n\t\t\"ipfs daemon\": true,\n\t\t\"ipfs config profile\": true,\n\t\t\"ipfs pin remote service\": true,\n\t\t\"ipfs name pubsub\": true,\n\t\t\"ipfs object patch\": true,\n\t\t\"ipfs swarm connect\": true,\n\t\t\"ipfs p2p forward\": true,\n\t\t\"ipfs p2p close\": true,\n\t\t\"ipfs swarm disconnect\": true,\n\t\t\"ipfs swarm addrs listen\": true,\n\t\t\"ipfs dag resolve\": true,\n\t\t\"ipfs dag get\": true,\n\t\t\"ipfs pin remote add\": true,\n\t\t\"ipfs config show\": true,\n\t\t\"ipfs config edit\": true,\n\t\t\"ipfs pin remote rm\": true,\n\t\t\"ipfs pin remote ls\": true,\n\t\t\"ipfs pin verify\": true,\n\t\t\"ipfs pin remote service add\": true,\n\t\t\"ipfs pin update\": true,\n\t\t\"ipfs pin rm\": true,\n\t\t\"ipfs p2p\": true,\n\t\t\"ipfs resolve\": true,\n\t\t\"ipfs dag stat\": true,\n\t\t\"ipfs name publish\": true,\n\t\t\"ipfs object diff\": true,\n\t\t\"ipfs object patch add-link\": true,\n\t\t\"ipfs name\": true,\n\t\t\"ipfs diag profile\": true,\n\t\t\"ipfs diag cmds\": true,\n\t\t\"ipfs swarm addrs local\": true,\n\t\t\"ipfs files ls\": true,\n\t\t\"ipfs stats bw\": true,\n\t\t\"ipfs swarm peers\": true,\n\t\t\"ipfs pubsub sub\": true,\n\t\t\"ipfs files write\": true,\n\t\t\"ipfs swarm limit\": true,\n\t\t\"ipfs commands completion fish\": true,\n\t\t\"ipfs key export\": true,\n\t\t\"ipfs routing get\": true,\n\t\t\"ipfs refs\": true,\n\t\t\"ipfs refs local\": true,\n\t\t\"ipfs cid base32\": true,\n\t\t\"ipfs pubsub pub\": true,\n\t\t\"ipfs repo ls\": true,\n\t\t\"ipfs routing put\": true,\n\t\t\"ipfs key import\": true,\n\t\t\"ipfs swarm peering add\": true,\n\t\t\"ipfs swarm peering rm\": true,\n\t\t\"ipfs swarm peering ls\": true,\n\t\t\"ipfs update\": true,\n\t\t\"ipfs swarm stats\": true,\n\t}\n\tfor _, cmd := range node.IPFSCommands() {\n\t\tif _, ok := allowList[cmd]; ok {\n\t\t\tcontinue\n\t\t}\n\t\tt.Run(fmt.Sprintf(\"command %q conforms to docs width limit\", cmd), func(t *testing.T) {\n\t\t\tsplitCmd := strings.Split(cmd, \" \")\n\t\t\tresStr := node.IPFS(StrCat(splitCmd[1:], \"--help\")...)\n\t\t\tres := strings.TrimSpace(resStr.Stdout.String())\n\t\t\tfor _, line := range SplitLines(res) {\n\t\t\t\tassert.LessOrEqualf(t, len(line), 80, \"expected width %d < 80 for %q\", len(line), cmd)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestAllCommandsFailWhenPassedBadFlag(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tfor _, cmd := range node.IPFSCommands() {\n\t\tt.Run(fmt.Sprintf(\"command %q fails when passed a bad flag\", cmd), func(t *testing.T) {\n\t\t\tsplitCmd := strings.Split(cmd, \" \")\n\t\t\tres := node.RunIPFS(StrCat(splitCmd, \"--badflag\")...)\n\t\t\tassert.Equal(t, 1, res.Cmd.ProcessState.ExitCode())\n\t\t})\n\t}\n}\n\nfunc TestCommandsFlags(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\tresStr := node.IPFS(\"commands\", \"--flags\").Stdout.String()\n\tassert.Contains(t, resStr, \"ipfs pin add --recursive / ipfs pin add -r\")\n\tassert.Contains(t, resStr, \"ipfs id --format / ipfs id -f\")\n\tassert.Contains(t, resStr, \"ipfs repo gc --quiet / ipfs repo gc -q\")\n}\n" }, { "path": "test/cli/bitswap_config_test.go", "content": "package cli\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/bitswap/network/bsnet\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestBitswapConfig(t *testing.T) {\n\tt.Parallel()\n\n\t// Create test data that will be shared between nodes\n\ttestData := random.Bytes(100)\n\n\tt.Run(\"server enabled (default)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tprovider := h.NewNode().Init().StartDaemon()\n\t\tdefer provider.StopDaemon()\n\t\trequester := h.NewNode().Init().StartDaemon()\n\t\tdefer requester.StopDaemon()\n\n\t\thash := provider.IPFSAddStr(string(testData))\n\t\trequester.Connect(provider)\n\n\t\tres := requester.IPFS(\"cat\", hash)\n\t\tassert.Equal(t, testData, res.Stdout.Bytes(), \"retrieved data should match original\")\n\t})\n\n\tt.Run(\"server disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\tprovider := h.NewNode().Init()\n\t\tprovider.SetIPFSConfig(\"Bitswap.ServerEnabled\", false)\n\t\tprovider = provider.StartDaemon()\n\t\tdefer provider.StopDaemon()\n\n\t\trequester := h.NewNode().Init().StartDaemon()\n\t\tdefer requester.StopDaemon()\n\n\t\thash := provider.IPFSAddStr(string(testData))\n\t\trequester.Connect(provider)\n\n\t\t// If the data was available, it would be retrieved immediately.\n\t\t// Therefore, after the timeout, we can assume the data is not available\n\t\t// i.e. the server is disabled\n\t\ttimeout := time.After(3 * time.Second)\n\t\tdataChan := make(chan []byte)\n\n\t\tgo func() {\n\t\t\tres := requester.RunIPFS(\"cat\", hash)\n\t\t\tdataChan <- res.Stdout.Bytes()\n\t\t}()\n\n\t\tselect {\n\t\tcase data := <-dataChan:\n\t\t\tassert.NotEqual(t, testData, data, \"retrieved data should not match original\")\n\t\tcase <-timeout:\n\t\t\tt.Log(\"Test passed: operation timed out after 3 seconds as expected\")\n\t\t}\n\t})\n\n\tt.Run(\"client still works when server disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\trequester := h.NewNode().Init()\n\t\trequester.SetIPFSConfig(\"Bitswap.ServerEnabled\", false)\n\t\trequester.StartDaemon()\n\t\tdefer requester.StopDaemon()\n\n\t\tprovider := h.NewNode().Init().StartDaemon()\n\t\tdefer provider.StopDaemon()\n\t\thash := provider.IPFSAddStr(string(testData))\n\t\trequester.Connect(provider)\n\n\t\t// Even when the server is disabled, the client should be able to retrieve data\n\t\tres := requester.RunIPFS(\"cat\", hash)\n\t\tassert.Equal(t, testData, res.Stdout.Bytes(), \"retrieved data should match original\")\n\t})\n\n\tt.Run(\"bitswap over libp2p disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\trequester := h.NewNode().Init()\n\t\trequester.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Bitswap.Libp2pEnabled = config.False\n\t\t\tcfg.Bitswap.ServerEnabled = config.False\n\t\t\tcfg.HTTPRetrieval.Enabled = config.True\n\t\t})\n\t\trequester.StartDaemon()\n\t\tdefer requester.StopDaemon()\n\n\t\tprovider := h.NewNode().Init().StartDaemon()\n\t\tdefer provider.StopDaemon()\n\t\thash := provider.IPFSAddStr(string(testData))\n\n\t\trequester.Connect(provider)\n\t\tres := requester.RunIPFS(\"cat\", hash)\n\t\tassert.Equal(t, []byte{}, res.Stdout.Bytes(), \"cat should not return any data\")\n\t\tassert.Contains(t, res.Stderr.String(), \"Error: ipld: could not find\")\n\n\t\t// Verify that basic operations still work with bitswap disabled\n\t\tres = requester.IPFS(\"id\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"basic IPFS operations should work\")\n\t\tres = requester.IPFS(\"bitswap\", \"stat\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"bitswap stat should work even with bitswap disabled\")\n\t\tres = requester.IPFS(\"bitswap\", \"wantlist\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"bitswap wantlist should work even with bitswap disabled\")\n\n\t\t// Verify local operations still work\n\t\thashNew := requester.IPFSAddStr(\"random\")\n\t\tres = requester.IPFS(\"cat\", hashNew)\n\t\tassert.Equal(t, []byte(\"random\"), res.Stdout.Bytes(), \"cat should return the added data\")\n\t})\n\n\t// Disabling Bitswap.Libp2pEnabled should remove /ipfs/bitswap* protocols from `ipfs id`\n\tt.Run(\"disabling bitswap over libp2p removes it from identify protocol list\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\tprovider := h.NewNode().Init()\n\t\tprovider.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Bitswap.Libp2pEnabled = config.False\n\t\t\tcfg.Bitswap.ServerEnabled = config.False\n\t\t\tcfg.HTTPRetrieval.Enabled = config.True\n\t\t})\n\t\tprovider = provider.StartDaemon()\n\t\tdefer provider.StopDaemon()\n\t\trequester := h.NewNode().Init().StartDaemon()\n\t\tdefer requester.StopDaemon()\n\t\trequester.Connect(provider)\n\n\t\t// read libp2p identify from remote peer, and print protocols\n\t\tres := requester.IPFS(\"id\", \"-f\", \"\", provider.PeerID().String())\n\t\tprotocols := strings.SplitSeq(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\n\t\t// No bitswap protocols should be present\n\t\tfor proto := range protocols {\n\t\t\tassert.NotContains(t, proto, bsnet.ProtocolBitswap, \"bitswap protocol %s should not be advertised when server is disabled\", proto)\n\t\t\tassert.NotContains(t, proto, bsnet.ProtocolBitswapNoVers, \"bitswap protocol %s should not be advertised when server is disabled\", proto)\n\t\t\tassert.NotContains(t, proto, bsnet.ProtocolBitswapOneOne, \"bitswap protocol %s should not be advertised when server is disabled\", proto)\n\t\t\tassert.NotContains(t, proto, bsnet.ProtocolBitswapOneZero, \"bitswap protocol %s should not be advertised when server is disabled\", proto)\n\t\t}\n\t})\n\n\t// HTTPRetrieval uses bitswap engine, we need it\n\tt.Run(\"errors when both HTTP and libp2p are disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// init Kubo repo\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.HTTPRetrieval.Enabled = config.False\n\t\t\tcfg.Bitswap.Libp2pEnabled = config.False\n\t\t\tcfg.Bitswap.ServerEnabled = config.Default\n\t\t})\n\t\tres := node.RunIPFS(\"daemon\")\n\t\tassert.Contains(t, res.Stderr.Trimmed(), \"invalid configuration: Bitswap.Libp2pEnabled and HTTPRetrieval.Enabled are both disabled, unable to initialize Bitswap\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t})\n\n\t// HTTPRetrieval uses bitswap engine, we need it\n\tt.Run(\"errors when user set conflicting HTTP and libp2p flags\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// init Kubo repo\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.HTTPRetrieval.Enabled = config.False\n\t\t\tcfg.Bitswap.Libp2pEnabled = config.False\n\t\t\tcfg.Bitswap.ServerEnabled = config.True // bad user config: can't enable server when libp2p is down\n\t\t})\n\t\tres := node.RunIPFS(\"daemon\")\n\t\tassert.Contains(t, res.Stderr.Trimmed(), \"invalid configuration: Bitswap.Libp2pEnabled and HTTPRetrieval.Enabled are both disabled, unable to initialize Bitswap\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t})\n}\n" }, { "path": "test/cli/block_size_test.go", "content": "package cli\n\nimport (\n\t\"bytes\"\n\t\"crypto/rand\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\ttwoMiB = 2 * 1024 * 1024 // 2097152 - bitswap spec block size limit\n\ttwoMiBPlus = twoMiB + 1 // 2097153\n\tmaxChunkSize = twoMiB - 256 // 2096896 - max chunker value (overhead budget for protobuf framing)\n\toverMaxChunk = maxChunkSize + 1 // 2096897\n\n\t// go-libp2p v0.47.0 network.MessageSizeMax is 4194304 bytes (4MiB).\n\t// A bitswap message carrying a single block has a protobuf envelope\n\t// whose size depends on the CID used to represent the block. For\n\t// CIDv1 with raw codec and SHA2-256 multihash (4-byte CID prefix),\n\t// the envelope is 18 bytes: 2 bytes for the empty Wantlist submessage,\n\t// 6 bytes for the CID prefix field, 5 bytes for field tags and the\n\t// payload length varint, and 5 bytes for the data length varint and\n\t// block submessage length varint. The msgio varint reader rejects\n\t// messages strictly larger than MessageSizeMax, so the maximum block\n\t// that fits is 4194304 - 18 = 4194286 bytes.\n\t//\n\t// The hard limit varies slightly depending on the CID: a longer\n\t// multihash (e.g. SHA-512) increases the CID prefix and reduces the\n\t// maximum block payload by the same amount.\n\tlibp2pMsgMax = 4 * 1024 * 1024 // 4194304 - libp2p network.MessageSizeMax\n\tbsBlockEnvelope = 18 // protobuf overhead for CIDv1 + raw + SHA2-256\n\tmaxTransferBlock = libp2pMsgMax - bsBlockEnvelope // 4194286 - largest block transferable via bitswap\n\toverMaxTransfer = maxTransferBlock + 1 // 4194287\n)\n\n// blockSize returns the block size in bytes for a given CID by parsing\n// the JSON output of `ipfs block stat --enc=json `.\nfunc blockSize(t *testing.T, node *harness.Node, cid string) int {\n\tt.Helper()\n\tres := node.IPFS(\"block\", \"stat\", \"--enc=json\", cid)\n\tvar stat struct {\n\t\tKey string\n\t\tSize int\n\t}\n\trequire.NoError(t, json.Unmarshal(res.Stdout.Bytes(), &stat))\n\treturn stat.Size\n}\n\n// allBlockCIDs returns the root CID plus all recursive refs for a DAG.\nfunc allBlockCIDs(t *testing.T, node *harness.Node, root string) []string {\n\tt.Helper()\n\tcids := []string{root}\n\tres := node.IPFS(\"refs\", \"-r\", \"--unique\", root)\n\tfor line := range strings.SplitSeq(strings.TrimSpace(res.Stdout.String()), \"\\n\") {\n\t\tif line != \"\" {\n\t\t\tcids = append(cids, line)\n\t\t}\n\t}\n\treturn cids\n}\n\n// assertAllBlocksWithinLimit checks that every block in the DAG rooted at\n// root is at most twoMiB bytes.\nfunc assertAllBlocksWithinLimit(t *testing.T, node *harness.Node, root string) {\n\tt.Helper()\n\tfor _, c := range allBlockCIDs(t, node, root) {\n\t\tsize := blockSize(t, node, c)\n\t\tassert.LessOrEqual(t, size, twoMiB, fmt.Sprintf(\"block %s is %d bytes, exceeds 2MiB limit\", c, size))\n\t}\n}\n\nfunc TestBlockSizeBoundary(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"block put\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"exactly 2MiB succeeds\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiB)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tnode.PipeToIPFS(bytes.NewReader(data), \"block\", \"put\").Stdout.String(),\n\t\t\t)\n\t\t\tgot := node.IPFS(\"block\", \"get\", cid)\n\t\t\tassert.Len(t, got.Stdout.Bytes(), twoMiB)\n\t\t})\n\n\t\tt.Run(\"2MiB+1 fails without --allow-big-block\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiBPlus)\n\t\t\tres := node.RunPipeToIPFS(bytes.NewReader(data), \"block\", \"put\")\n\t\t\tassert.NotEqual(t, 0, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"produced block is over 2MiB: big blocks can't be exchanged with other peers. consider using UnixFS for automatic chunking of bigger files, or pass --allow-big-block to override\")\n\t\t})\n\n\t\tt.Run(\"2MiB+1 succeeds with --allow-big-block\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiBPlus)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tnode.PipeToIPFS(bytes.NewReader(data), \"block\", \"put\", \"--allow-big-block\").Stdout.String(),\n\t\t\t)\n\t\t\tgot := node.IPFS(\"block\", \"get\", cid)\n\t\t\tassert.Len(t, got.Stdout.Bytes(), twoMiBPlus)\n\t\t})\n\t})\n\n\tt.Run(\"dag put\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"exactly 2MiB succeeds\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiB)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tnode.PipeToIPFS(bytes.NewReader(data), \"dag\", \"put\", \"--input-codec=raw\", \"--store-codec=raw\").Stdout.String(),\n\t\t\t)\n\t\t\tgot := node.IPFS(\"block\", \"get\", cid)\n\t\t\tassert.Len(t, got.Stdout.Bytes(), twoMiB)\n\t\t})\n\n\t\tt.Run(\"2MiB+1 fails without --allow-big-block\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiBPlus)\n\t\t\tres := node.RunPipeToIPFS(bytes.NewReader(data), \"dag\", \"put\", \"--input-codec=raw\", \"--store-codec=raw\")\n\t\t\tassert.NotEqual(t, 0, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"produced block is over 2MiB: big blocks can't be exchanged with other peers. consider using UnixFS for automatic chunking of bigger files, or pass --allow-big-block to override\")\n\t\t})\n\n\t\tt.Run(\"2MiB+1 succeeds with --allow-big-block\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiBPlus)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tnode.PipeToIPFS(bytes.NewReader(data), \"dag\", \"put\", \"--input-codec=raw\", \"--store-codec=raw\", \"--allow-big-block\").Stdout.String(),\n\t\t\t)\n\t\t\tgot := node.IPFS(\"block\", \"get\", cid)\n\t\t\tassert.Len(t, got.Stdout.Bytes(), twoMiBPlus)\n\t\t})\n\t})\n\n\tt.Run(\"dag import and export\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"2MiB+1 block round-trips with --allow-big-block\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// put an oversized raw block with override\n\t\t\tdata := make([]byte, twoMiBPlus)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tnode.PipeToIPFS(bytes.NewReader(data), \"dag\", \"put\", \"--input-codec=raw\", \"--store-codec=raw\", \"--allow-big-block\").Stdout.String(),\n\t\t\t)\n\n\t\t\t// export to CAR\n\t\t\tcarPath := filepath.Join(node.Dir, \"oversized.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cid, carPath))\n\n\t\t\t// re-import without --allow-big-block should fail\n\t\t\tcarFile, err := os.Open(carPath)\n\t\t\trequire.NoError(t, err)\n\t\t\tres := node.RunPipeToIPFS(carFile, \"dag\", \"import\")\n\t\t\tcarFile.Close()\n\t\t\tassert.NotEqual(t, 0, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String()+res.Stdout.String(), \"produced block is over 2MiB: big blocks can't be exchanged with other peers. consider using UnixFS for automatic chunking of bigger files, or pass --allow-big-block to override\")\n\n\t\t\t// re-import with --allow-big-block should succeed\n\t\t\tcarFile, err = os.Open(carPath)\n\t\t\trequire.NoError(t, err)\n\t\t\tres = node.RunPipeToIPFS(carFile, \"dag\", \"import\", \"--allow-big-block\")\n\t\t\tcarFile.Close()\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t})\n\t})\n\n\tt.Run(\"ipfs add non-raw-leaves\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// The chunker enforces ChunkSizeLimit (maxChunkSize = 2MiB - 256\n\t\t// as of boxo 2026Q1) regardless of leaf type. It does not know at parse time whether\n\t\t// raw or wrapped leaves will be used, so the 256-byte overhead\n\t\t// budget is applied uniformly.\n\t\t//\n\t\t// With --raw-leaves=false each chunk is wrapped in protobuf,\n\t\t// adding ~14 bytes overhead that pushes blocks past the chunk size.\n\t\t// The overhead budget ensures the wrapped block stays within 2MiB.\n\t\t//\n\t\t// With --raw-leaves=true there is no protobuf wrapper, so the\n\t\t// block is exactly the chunk size (maxChunkSize). The 256-byte\n\t\t// budget is unused in this case but the chunker still enforces it.\n\t\t// A full 2MiB chunk (--chunker=size-2097152) is rejected even\n\t\t// though the resulting raw block would fit within BlockSizeLimit.\n\n\t\tt.Run(\"1MiB chunk with protobuf wrapping succeeds under 2MiB limit\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiB)\n\t\t\tres := node.RunPipeToIPFS(bytes.NewReader(data), \"add\", \"-q\", \"--chunker=size-1048576\", \"--raw-leaves=false\")\n\t\t\trequire.Equal(t, 0, res.ExitCode(), \"stderr: %s\", res.Stderr.String())\n\t\t\troot := strings.TrimSpace(res.Stdout.String())\n\t\t\t// the last line of `ipfs add -q` is the root CID\n\t\t\tlines := strings.Split(root, \"\\n\")\n\t\t\troot = lines[len(lines)-1]\n\t\t\tassertAllBlocksWithinLimit(t, node, root)\n\t\t})\n\n\t\tt.Run(\"max chunk with protobuf wrapping stays within block limit\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// maxChunkSize leaves room for protobuf framing overhead\n\t\t\tdata := make([]byte, maxChunkSize*2)\n\t\t\tres := node.RunPipeToIPFS(bytes.NewReader(data), \"add\", \"-q\",\n\t\t\t\tfmt.Sprintf(\"--chunker=size-%d\", maxChunkSize), \"--raw-leaves=false\")\n\t\t\trequire.Equal(t, 0, res.ExitCode(), \"stderr: %s\", res.Stderr.String())\n\t\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\t\troot := lines[len(lines)-1]\n\t\t\tassertAllBlocksWithinLimit(t, node, root)\n\t\t})\n\n\t\tt.Run(\"chunk size over limit is rejected by chunker\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiB+twoMiB)\n\t\t\tres := node.RunPipeToIPFS(bytes.NewReader(data), \"add\", \"-q\",\n\t\t\t\tfmt.Sprintf(\"--chunker=size-%d\", overMaxChunk), \"--raw-leaves=false\")\n\t\t\tassert.NotEqual(t, 0, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(),\n\t\t\t\tfmt.Sprintf(\"chunker parameters may not exceed the maximum chunk size of %d\", maxChunkSize))\n\t\t})\n\n\t\tt.Run(\"max chunk with raw leaves succeeds\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// raw leaves have no protobuf wrapper, so max chunk size fits easily\n\t\t\tdata := make([]byte, maxChunkSize*2)\n\t\t\tres := node.RunPipeToIPFS(bytes.NewReader(data), \"add\", \"-q\",\n\t\t\t\tfmt.Sprintf(\"--chunker=size-%d\", maxChunkSize), \"--raw-leaves=true\")\n\t\t\trequire.Equal(t, 0, res.ExitCode(), \"stderr: %s\", res.Stderr.String())\n\t\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\t\troot := lines[len(lines)-1]\n\t\t\tassertAllBlocksWithinLimit(t, node, root)\n\t\t})\n\t})\n\n\tt.Run(\"bitswap exchange\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"2MiB raw block transfers between peers\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\th := harness.NewT(t)\n\t\t\tprovider := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer provider.StopDaemon()\n\t\t\trequester := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer requester.StopDaemon()\n\n\t\t\tdata := make([]byte, twoMiB)\n\t\t\t_, err := rand.Read(data)\n\t\t\trequire.NoError(t, err)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tprovider.PipeToIPFS(bytes.NewReader(data), \"block\", \"put\").Stdout.String(),\n\t\t\t)\n\n\t\t\trequester.Connect(provider)\n\n\t\t\tres := requester.IPFS(\"block\", \"get\", cid)\n\t\t\tassert.Equal(t, data, res.Stdout.Bytes(), \"retrieved block should match original\")\n\t\t})\n\n\t\tt.Run(\"unixfs-v1-2025: 2MiB file transfers between peers\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\th := harness.NewT(t)\n\t\t\tprovider := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer provider.StopDaemon()\n\t\t\trequester := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer requester.StopDaemon()\n\n\t\t\t// unixfs-v1-2025 profile uses CIDv1, raw leaves, SHA2-256,\n\t\t\t// and 1MiB chunks. A 2MiB file produces two 1MiB raw leaf\n\t\t\t// blocks plus a root node, all within the 2MiB spec limit.\n\t\t\tdata := make([]byte, twoMiB)\n\t\t\t_, err := rand.Read(data)\n\t\t\trequire.NoError(t, err)\n\t\t\tres := provider.RunPipeToIPFS(bytes.NewReader(data), \"add\", \"-q\")\n\t\t\trequire.Equal(t, 0, res.ExitCode(), \"stderr: %s\", res.Stderr.String())\n\t\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\t\troot := lines[len(lines)-1]\n\n\t\t\trequester.Connect(provider)\n\n\t\t\tgot := requester.IPFS(\"cat\", root)\n\t\t\tassert.Equal(t, data, got.Stdout.Bytes(), \"retrieved file should match original\")\n\t\t})\n\n\t\t// The following two tests guard the physical hard limit of the\n\t\t// libp2p transport layer (network.MessageSizeMax = 4MiB). This is\n\t\t// the actual ceiling for bitswap block transfer, independent of the\n\t\t// 2MiB soft limit from the bitswap spec. Knowing the exact hard\n\t\t// limit is important for backward-compatible protocol and standards\n\t\t// evolution: any future increase to the bitswap spec block size\n\t\t// must stay within the libp2p message framing budget, or the\n\t\t// transport layer must be updated first.\n\n\t\tt.Run(\"bitswap-over-libp2p: largest block that fits in message transfers\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\th := harness.NewT(t)\n\t\t\tprovider := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer provider.StopDaemon()\n\t\t\trequester := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer requester.StopDaemon()\n\n\t\t\tdata := make([]byte, maxTransferBlock)\n\t\t\t_, err := rand.Read(data)\n\t\t\trequire.NoError(t, err)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tprovider.PipeToIPFS(bytes.NewReader(data), \"block\", \"put\", \"--allow-big-block\").Stdout.String(),\n\t\t\t)\n\n\t\t\trequester.Connect(provider)\n\n\t\t\t// successful transfers complete in ~1s\n\t\t\ttimeout := time.After(5 * time.Second)\n\t\t\tdataChan := make(chan []byte, 1)\n\n\t\t\tgo func() {\n\t\t\t\tres := requester.RunIPFS(\"block\", \"get\", cid)\n\t\t\t\tdataChan <- res.Stdout.Bytes()\n\t\t\t}()\n\n\t\t\tselect {\n\t\t\tcase got := <-dataChan:\n\t\t\t\tassert.Equal(t, data, got, \"retrieved block should match original\")\n\t\t\tcase <-timeout:\n\t\t\t\tt.Fatal(\"block get timed out: expected transfer to succeed at maxTransferBlock\")\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"bitswap-over-libp2p: one byte over message limit does not transfer\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\th := harness.NewT(t)\n\t\t\tprovider := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer provider.StopDaemon()\n\t\t\trequester := h.NewNode().Init(\"--profile=unixfs-v1-2025\").StartDaemon()\n\t\t\tdefer requester.StopDaemon()\n\n\t\t\tdata := make([]byte, overMaxTransfer)\n\t\t\t_, err := rand.Read(data)\n\t\t\trequire.NoError(t, err)\n\t\t\tcid := strings.TrimSpace(\n\t\t\t\tprovider.PipeToIPFS(bytes.NewReader(data), \"block\", \"put\", \"--allow-big-block\").Stdout.String(),\n\t\t\t)\n\n\t\t\trequester.Connect(provider)\n\n\t\t\ttimeout := time.After(5 * time.Second)\n\t\t\tdataChan := make(chan []byte, 1)\n\n\t\t\tgo func() {\n\t\t\t\tres := requester.RunIPFS(\"block\", \"get\", cid)\n\t\t\t\tdataChan <- res.Stdout.Bytes()\n\t\t\t}()\n\n\t\t\tselect {\n\t\t\tcase got := <-dataChan:\n\t\t\t\tt.Fatalf(\"expected timeout, but block was retrieved (%d bytes)\", len(got))\n\t\t\tcase <-timeout:\n\t\t\t\tt.Log(\"block get timed out as expected: block exceeds libp2p message size limit\")\n\t\t\t}\n\t\t})\n\t})\n}\n" }, { "path": "test/cli/bootstrap_auto_test.go", "content": "package cli\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestBootstrapCommandsWithAutoPlaceholder(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"bootstrap add default\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that 'ipfs bootstrap add default' works correctly\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{}) // Start with empty bootstrap\n\n\t\t// Add default bootstrap peers via \"auto\" placeholder\n\t\tresult := node.RunIPFS(\"bootstrap\", \"add\", \"default\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap add default should succeed\")\n\n\t\toutput := result.Stdout.String()\n\t\tt.Logf(\"Bootstrap add default output: %s\", output)\n\t\tassert.Contains(t, output, \"added auto\", \"bootstrap add default should report adding 'auto'\")\n\n\t\t// Verify bootstrap list shows \"auto\"\n\t\tlistResult := node.RunIPFS(\"bootstrap\", \"list\")\n\t\trequire.Equal(t, 0, listResult.ExitCode(), \"bootstrap list should succeed\")\n\n\t\tlistOutput := listResult.Stdout.String()\n\t\tt.Logf(\"Bootstrap list after add default: %s\", listOutput)\n\t\tassert.Contains(t, listOutput, \"auto\", \"bootstrap list should show 'auto' placeholder\")\n\t})\n\n\tt.Run(\"bootstrap add auto explicitly\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that 'ipfs bootstrap add auto' works correctly\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{}) // Start with empty bootstrap\n\n\t\t// Add \"auto\" placeholder explicitly\n\t\tresult := node.RunIPFS(\"bootstrap\", \"add\", \"auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap add auto should succeed\")\n\n\t\toutput := result.Stdout.String()\n\t\tt.Logf(\"Bootstrap add auto output: %s\", output)\n\t\tassert.Contains(t, output, \"added auto\", \"bootstrap add auto should report adding 'auto'\")\n\n\t\t// Verify bootstrap list shows \"auto\"\n\t\tlistResult := node.RunIPFS(\"bootstrap\", \"list\")\n\t\trequire.Equal(t, 0, listResult.ExitCode(), \"bootstrap list should succeed\")\n\n\t\tlistOutput := listResult.Stdout.String()\n\t\tt.Logf(\"Bootstrap list after add auto: %s\", listOutput)\n\t\tassert.Contains(t, listOutput, \"auto\", \"bootstrap list should show 'auto' placeholder\")\n\t})\n\n\tt.Run(\"bootstrap add default converts to auto\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that 'ipfs bootstrap add default' adds \"auto\" to the bootstrap list\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{}) // Start with empty bootstrap\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true) // Enable AutoConf to allow adding \"auto\"\n\n\t\t// Add default bootstrap peers\n\t\tresult := node.RunIPFS(\"bootstrap\", \"add\", \"default\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap add default should succeed\")\n\t\tassert.Contains(t, result.Stdout.String(), \"added auto\", \"should report adding 'auto'\")\n\n\t\t// Verify bootstrap list shows \"auto\"\n\t\tvar bootstrap []string\n\t\tnode.GetIPFSConfig(\"Bootstrap\", &bootstrap)\n\t\trequire.Equal(t, []string{\"auto\"}, bootstrap, \"Bootstrap should contain ['auto']\")\n\t})\n\n\tt.Run(\"bootstrap add default fails when AutoConf disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that adding default/auto fails when AutoConf is disabled\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{}) // Start with empty bootstrap\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", false) // Disable AutoConf\n\n\t\t// Try to add default - should fail\n\t\tresult := node.RunIPFS(\"bootstrap\", \"add\", \"default\")\n\t\trequire.NotEqual(t, 0, result.ExitCode(), \"bootstrap add default should fail when AutoConf disabled\")\n\t\tassert.Contains(t, result.Stderr.String(), \"AutoConf is disabled\", \"should mention AutoConf is disabled\")\n\n\t\t// Try to add auto - should also fail\n\t\tresult = node.RunIPFS(\"bootstrap\", \"add\", \"auto\")\n\t\trequire.NotEqual(t, 0, result.ExitCode(), \"bootstrap add auto should fail when AutoConf disabled\")\n\t\tassert.Contains(t, result.Stderr.String(), \"AutoConf is disabled\", \"should mention AutoConf is disabled\")\n\t})\n\n\tt.Run(\"bootstrap rm with auto placeholder\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that selective removal fails properly when \"auto\" is present\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"}) // Start with auto\n\n\t\t// Try to remove a specific peer - should fail with helpful error\n\t\tresult := node.RunIPFS(\"bootstrap\", \"rm\", \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\")\n\t\trequire.NotEqual(t, 0, result.ExitCode(), \"bootstrap rm of specific peer should fail when 'auto' is present\")\n\n\t\toutput := result.Stderr.String()\n\t\tt.Logf(\"Bootstrap rm error output: %s\", output)\n\t\tassert.Contains(t, output, \"cannot remove individual bootstrap peers when using 'auto' placeholder\",\n\t\t\t\"should provide helpful error message about auto placeholder\")\n\t\tassert.Contains(t, output, \"disable AutoConf\",\n\t\t\t\"should suggest disabling AutoConf as solution\")\n\t\tassert.Contains(t, output, \"ipfs bootstrap rm --all\",\n\t\t\t\"should suggest using rm --all as alternative\")\n\t})\n\n\tt.Run(\"bootstrap rm --all with auto placeholder\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that 'ipfs bootstrap rm --all' works with \"auto\" placeholder\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"}) // Start with auto\n\n\t\t// Remove all bootstrap peers\n\t\tresult := node.RunIPFS(\"bootstrap\", \"rm\", \"--all\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap rm --all should succeed with auto placeholder\")\n\n\t\toutput := result.Stdout.String()\n\t\tt.Logf(\"Bootstrap rm --all output: %s\", output)\n\t\tassert.Contains(t, output, \"removed auto\", \"bootstrap rm --all should report removing 'auto'\")\n\n\t\t// Verify bootstrap list is now empty\n\t\tlistResult := node.RunIPFS(\"bootstrap\", \"list\")\n\t\trequire.Equal(t, 0, listResult.ExitCode(), \"bootstrap list should succeed\")\n\n\t\tlistOutput := listResult.Stdout.String()\n\t\tt.Logf(\"Bootstrap list after rm --all: %s\", listOutput)\n\t\tassert.Empty(t, listOutput, \"bootstrap list should be empty after rm --all\")\n\n\t\t// Test the rm all subcommand too\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{\"auto\"}) // Reset to auto\n\n\t\tresult = node.RunIPFS(\"bootstrap\", \"rm\", \"all\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap rm all should succeed with auto placeholder\")\n\n\t\toutput = result.Stdout.String()\n\t\tt.Logf(\"Bootstrap rm all output: %s\", output)\n\t\tassert.Contains(t, output, \"removed auto\", \"bootstrap rm all should report removing 'auto'\")\n\t})\n\n\tt.Run(\"bootstrap mixed auto and specific peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Test that bootstrap commands work when mixing \"auto\" with specific peers\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tnode.SetIPFSConfig(\"AutoConf.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{}) // Start with empty bootstrap\n\n\t\t// Add a specific peer first\n\t\tspecificPeer := \"/ip4/127.0.0.1/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n\t\tresult := node.RunIPFS(\"bootstrap\", \"add\", specificPeer)\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap add specific peer should succeed\")\n\n\t\t// Add auto placeholder\n\t\tresult = node.RunIPFS(\"bootstrap\", \"add\", \"auto\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap add auto should succeed\")\n\n\t\t// Verify bootstrap list shows both\n\t\tlistResult := node.RunIPFS(\"bootstrap\", \"list\")\n\t\trequire.Equal(t, 0, listResult.ExitCode(), \"bootstrap list should succeed\")\n\n\t\tlistOutput := listResult.Stdout.String()\n\t\tt.Logf(\"Bootstrap list with mixed peers: %s\", listOutput)\n\t\tassert.Contains(t, listOutput, \"auto\", \"bootstrap list should contain 'auto' placeholder\")\n\t\tassert.Contains(t, listOutput, specificPeer, \"bootstrap list should contain specific peer\")\n\n\t\t// Try to remove the specific peer - should fail because auto is present\n\t\tresult = node.RunIPFS(\"bootstrap\", \"rm\", specificPeer)\n\t\trequire.NotEqual(t, 0, result.ExitCode(), \"bootstrap rm of specific peer should fail when 'auto' is present\")\n\n\t\toutput := result.Stderr.String()\n\t\tassert.Contains(t, output, \"cannot remove individual bootstrap peers when using 'auto' placeholder\",\n\t\t\t\"should provide helpful error message about auto placeholder\")\n\n\t\t// Remove all should work and remove both auto and specific peer\n\t\tresult = node.RunIPFS(\"bootstrap\", \"rm\", \"--all\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"bootstrap rm --all should succeed\")\n\n\t\toutput = result.Stdout.String()\n\t\tt.Logf(\"Bootstrap rm --all output with mixed peers: %s\", output)\n\t\t// Should report removing both the specific peer and auto\n\t\tassert.Contains(t, output, \"removed\", \"should report removing peers\")\n\n\t\t// Verify bootstrap list is now empty\n\t\tlistResult = node.RunIPFS(\"bootstrap\", \"list\")\n\t\trequire.Equal(t, 0, listResult.ExitCode(), \"bootstrap list should succeed\")\n\n\t\tlistOutput = listResult.Stdout.String()\n\t\tassert.Empty(t, listOutput, \"bootstrap list should be empty after rm --all\")\n\t})\n}\n" }, { "path": "test/cli/cid_profiles_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// cidProfileExpectations defines expected behaviors for a UnixFS import profile.\n// This allows DRY testing of multiple profiles with the same test logic.\n//\n// Each profile is tested against threshold boundaries to verify:\n// - CID format (version, hash function, raw leaves vs dag-pb wrapped)\n// - File chunking (UnixFSChunker size threshold)\n// - DAG structure (UnixFSFileMaxLinks rebalancing threshold)\n// - Directory sharding (HAMTThreshold for flat vs HAMT directories)\ntype cidProfileExpectations struct {\n\t// Profile identification\n\tName string // canonical profile name from IPIP-499\n\tProfileArgs []string // args to pass to ipfs init (empty for default behavior)\n\n\t// CID format expectations\n\tCIDVersion int // 0 or 1\n\tHashFunc string // e.g., \"sha2-256\"\n\tRawLeaves bool // true = raw codec for small files, false = dag-pb wrapped\n\n\t// File chunking expectations (UnixFSChunker config)\n\tChunkSize int // chunk size in bytes (e.g., 262144 for 256KiB, 1048576 for 1MiB)\n\tChunkSizeHuman string // human-readable chunk size (e.g., \"256KiB\", \"1MiB\")\n\tFileMaxLinks int // max links before DAG rebalancing (UnixFSFileMaxLinks config)\n\n\t// HAMT directory sharding expectations (UnixFSHAMTDirectory* config).\n\t// Threshold behavior: boxo converts to HAMT when size > HAMTThreshold (not >=).\n\t// This means a directory exactly at the threshold stays as a basic (flat) directory.\n\tHAMTFanout int // max links per HAMT shard bucket (256)\n\tHAMTThreshold int // sharding threshold in bytes (262144 = 256 KiB)\n\tHAMTSizeEstimation string // \"block\" (protobuf size) or \"links\" (legacy name+cid)\n\n\t// Test vector parameters for threshold boundary tests.\n\t// - DirBasic: size == threshold (stays basic)\n\t// - DirHAMT: size > threshold (converts to HAMT)\n\t// For block estimation, last filename length is adjusted to hit exact thresholds.\n\tDirBasicNameLen int // filename length for basic directory (files 0 to N-2)\n\tDirBasicLastNameLen int // filename length for last file (0 = same as DirBasicNameLen)\n\tDirBasicFiles int // file count for basic directory (at exact threshold)\n\tDirHAMTNameLen int // filename length for HAMT directory (files 0 to N-2)\n\tDirHAMTLastNameLen int // filename length for last file (0 = same as DirHAMTNameLen)\n\tDirHAMTFiles int // total file count for HAMT directory (over threshold)\n\n\t// Expected deterministic CIDs for test vectors.\n\t// These serve as regression tests to detect unintended changes in CID generation.\n\n\t// SmallFileCID is the deterministic CID for \"hello world\" string.\n\t// Tests basic CID format (version, codec, hash).\n\tSmallFileCID string\n\n\t// FileAtChunkSizeCID is the deterministic CID for a file exactly at chunk size.\n\t// This file fits in a single block with no links:\n\t// - v0-2015: dag-pb wrapped TFile node (CIDv0)\n\t// - v1-2025: raw leaf block (CIDv1)\n\tFileAtChunkSizeCID string\n\n\t// FileOverChunkSizeCID is the deterministic CID for a file 1 byte over chunk size.\n\t// This file requires 2 chunks, producing a root dag-pb node with 2 links:\n\t// - v0-2015: links point to dag-pb wrapped TFile leaf nodes\n\t// - v1-2025: links point to raw leaf blocks\n\tFileOverChunkSizeCID string\n\n\t// FileAtMaxLinksCID is the deterministic CID for a file at UnixFSFileMaxLinks threshold.\n\t// File size = maxLinks * chunkSize, producing a single-layer DAG with exactly maxLinks children.\n\tFileAtMaxLinksCID string\n\n\t// FileOverMaxLinksCID is the deterministic CID for a file 1 byte over max links threshold.\n\t// The +1 byte requires an additional chunk, forcing DAG rebalancing to 2 layers.\n\tFileOverMaxLinksCID string\n\n\t// DirBasicCID is the deterministic CID for a directory exactly at HAMTThreshold.\n\t// With > comparison (not >=), directory at exact threshold stays as basic (flat) directory.\n\tDirBasicCID string\n\n\t// DirHAMTCID is the deterministic CID for a directory 1 byte over HAMTThreshold.\n\t// Crossing the threshold converts the directory to a HAMT sharded structure.\n\tDirHAMTCID string\n}\n\n// unixfsV02015 is the legacy profile for backward-compatible CID generation.\n// Alias: legacy-cid-v0\nvar unixfsV02015 = cidProfileExpectations{\n\tName: \"unixfs-v0-2015\",\n\tProfileArgs: []string{\"--profile=unixfs-v0-2015\"},\n\n\tCIDVersion: 0,\n\tHashFunc: \"sha2-256\",\n\tRawLeaves: false,\n\n\tChunkSize: 262144, // 256 KiB\n\tChunkSizeHuman: \"256KiB\",\n\tFileMaxLinks: 174,\n\n\tHAMTFanout: 256,\n\tHAMTThreshold: 262144, // 256 KiB\n\tHAMTSizeEstimation: \"links\",\n\tDirBasicNameLen: 30, // 4096 * (30 + 34) = 262144 exactly at threshold\n\tDirBasicFiles: 4096, // 4096 * 64 = 262144 (stays basic with >)\n\tDirHAMTNameLen: 31, // 4033 * (31 + 34) = 262145 exactly +1 over threshold\n\tDirHAMTLastNameLen: 0, // 0 = same as DirHAMTNameLen (uniform filenames)\n\tDirHAMTFiles: 4033, // 4033 * 65 = 262145 (becomes HAMT)\n\n\tSmallFileCID: \"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD\", // \"hello world\" dag-pb wrapped\n\tFileAtChunkSizeCID: \"QmWmRj3dFDZdb6ABvbmKhEL6TmPbAfBZ1t5BxsEyJrcZhE\", // 262144 bytes with seed \"chunk-v0-seed\"\n\tFileOverChunkSizeCID: \"QmYyLxtzZyW22zpoVAtKANLRHpDjZtNeDjQdJrcQNWoRkJ\", // 262145 bytes with seed \"chunk-v0-seed\"\n\tFileAtMaxLinksCID: \"QmUbBALi174SnogsUzLpYbD4xPiBSFANF4iztWCsHbMKh2\", // 174*256KiB bytes with seed \"v0-seed\"\n\tFileOverMaxLinksCID: \"QmV81WL765sC8DXsRhE5fJv2rwhS4icHRaf3J9Zk5FdRnW\", // 174*256KiB+1 bytes with seed \"v0-seed\"\n\tDirBasicCID: \"QmX5GtRk3TSSEHtdrykgqm4eqMEn3n2XhfkFAis5fjyZmN\", // 4096 files at threshold\n\tDirHAMTCID: \"QmeMiJzmhpJAUgynAcxTQYek5PPKgdv3qEvFsdV3XpVnvP\", // 4033 files +1 over threshold\n}\n\n// unixfsV12025 is the recommended profile for cross-implementation CID determinism.\nvar unixfsV12025 = cidProfileExpectations{\n\tName: \"unixfs-v1-2025\",\n\tProfileArgs: []string{\"--profile=unixfs-v1-2025\"},\n\n\tCIDVersion: 1,\n\tHashFunc: \"sha2-256\",\n\tRawLeaves: true,\n\n\tChunkSize: 1048576, // 1 MiB\n\tChunkSizeHuman: \"1MiB\",\n\tFileMaxLinks: 1024,\n\n\tHAMTFanout: 256,\n\tHAMTThreshold: 262144, // 256 KiB\n\tHAMTSizeEstimation: \"block\",\n\t// Block size = numFiles * linkSize + 4 bytes overhead\n\t// LinkSerializedSize(11, 36, 1) = 55, LinkSerializedSize(21, 36, 1) = 65, LinkSerializedSize(22, 36, 1) = 66\n\tDirBasicNameLen: 11, // 4765 files * 55 bytes\n\tDirBasicLastNameLen: 21, // last file: 65 bytes; total: 4765*55 + 65 + 4 = 262144 (at threshold)\n\tDirBasicFiles: 4766, // stays basic with > comparison\n\tDirHAMTNameLen: 11, // 4765 files * 55 bytes\n\tDirHAMTLastNameLen: 22, // last file: 66 bytes; total: 4765*55 + 66 + 4 = 262145 (+1 over threshold)\n\tDirHAMTFiles: 4766, // becomes HAMT\n\n\tSmallFileCID: \"bafkreifzjut3te2nhyekklss27nh3k72ysco7y32koao5eei66wof36n5e\", // \"hello world\" raw leaf\n\tFileAtChunkSizeCID: \"bafkreiacndfy443ter6qr2tmbbdhadvxxheowwf75s6zehscklu6ezxmta\", // 1048576 bytes with seed \"chunk-v1-seed\"\n\tFileOverChunkSizeCID: \"bafybeigmix7t42i6jacydtquhet7srwvgpizfg7gjbq7627d35mjomtu64\", // 1048577 bytes with seed \"chunk-v1-seed\"\n\tFileAtMaxLinksCID: \"bafybeihmf37wcuvtx4hpu7he5zl5qaf2ineo2lqlfrapokkm5zzw7zyhvm\", // 1024*1MiB bytes with seed \"v1-2025-seed\"\n\tFileOverMaxLinksCID: \"bafybeibdsi225ugbkmpbdohnxioyab6jsqrmkts3twhpvfnzp77xtzpyhe\", // 1024*1MiB+1 bytes with seed \"v1-2025-seed\"\n\tDirBasicCID: \"bafybeic3h7rwruealwxkacabdy45jivq2crwz6bufb5ljwupn36gicplx4\", // 4766 files at 262144 bytes (threshold)\n\tDirHAMTCID: \"bafybeiegvuterwurhdtkikfhbxcldohmxp566vpjdofhzmnhv6o4freidu\", // 4766 files at 262145 bytes (+1 over)\n}\n\n// defaultProfile points to the profile that matches Kubo's implicit default behavior.\n// Today this is unixfs-v0-2015. When Kubo changes defaults, update this pointer.\nvar defaultProfile = unixfsV02015\n\nconst (\n\tcidV0Length = 34 // CIDv0 sha2-256\n\tcidV1Length = 36 // CIDv1 sha2-256\n)\n\n// TestCIDProfiles generates deterministic test vectors for CID profile verification.\n// Set CID_PROFILES_CAR_OUTPUT environment variable to export CAR files.\n// Example: CID_PROFILES_CAR_OUTPUT=/tmp/cid-profiles go test -run TestCIDProfiles -v\nfunc TestCIDProfiles(t *testing.T) {\n\tt.Parallel()\n\n\tcarOutputDir := os.Getenv(\"CID_PROFILES_CAR_OUTPUT\")\n\texportCARs := carOutputDir != \"\"\n\tif exportCARs {\n\t\tif err := os.MkdirAll(carOutputDir, 0o755); err != nil {\n\t\t\tt.Fatalf(\"failed to create CAR output directory: %v\", err)\n\t\t}\n\t\tt.Logf(\"CAR export enabled, writing to: %s\", carOutputDir)\n\t}\n\n\t// Test both IPIP-499 profiles\n\tfor _, profile := range []cidProfileExpectations{unixfsV02015, unixfsV12025} {\n\t\tt.Run(profile.Name, func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\trunProfileTests(t, profile, carOutputDir, exportCARs)\n\t\t})\n\t}\n\n\t// Test default behavior (no profile specified)\n\tt.Run(\"default\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\t// Default behavior should match defaultProfile (currently unixfs-v0-2015)\n\t\tdefaultExp := defaultProfile\n\t\tdefaultExp.Name = \"default\"\n\t\tdefaultExp.ProfileArgs = nil // no profile args = default behavior\n\t\trunProfileTests(t, defaultExp, carOutputDir, exportCARs)\n\t})\n}\n\n// runProfileTests runs all test vectors for a given profile.\n// Tests verify threshold behaviors for:\n// - Small files (CID format verification)\n// - UnixFSChunker threshold (single block vs multi-block)\n// - UnixFSFileMaxLinks threshold (single-layer vs rebalanced DAG)\n// - HAMTThreshold (basic flat directory vs HAMT sharded)\nfunc runProfileTests(t *testing.T, exp cidProfileExpectations, carOutputDir string, exportCARs bool) {\n\tcidLen := cidV0Length\n\tif exp.CIDVersion == 1 {\n\t\tcidLen = cidV1Length\n\t}\n\n\t// Test: small file produces correct CID format\n\t// Verifies the profile sets the expected CID version, hash function, and leaf encoding.\n\tt.Run(\"small file produces correct CID format\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Use \"hello world\" for determinism\n\t\tcidStr := node.IPFSAddStr(\"hello world\")\n\n\t\t// Verify CID version (v0 starts with \"Qm\", v1 with \"b\")\n\t\tverifyCIDVersion(t, node, cidStr, exp.CIDVersion)\n\n\t\t// Verify hash function (sha2-256 for both profiles)\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\t// Verify raw leaves vs dag-pb wrapped\n\t\t// - v0-2015: dag-pb codec (wrapped)\n\t\t// - v1-2025: raw codec (raw leaves)\n\t\tverifyRawLeaves(t, node, cidStr, exp.RawLeaves)\n\n\t\t// Verify deterministic CID matches expected value\n\t\tif exp.SmallFileCID != \"\" {\n\t\t\trequire.Equal(t, exp.SmallFileCID, cidStr, \"expected deterministic CID for small file\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_small-file.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n\n\t// Test: file at UnixFSChunker threshold (single block)\n\t// A file exactly at chunk size fits in one block with no links.\n\t// - v0-2015 (256KiB): produces dag-pb wrapped TFile node\n\t// - v1-2025 (1MiB): produces raw leaf block\n\tt.Run(\"file at UnixFSChunker threshold (single block)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// File exactly at chunk size = single block (no links)\n\t\tseed := chunkSeedForProfile(exp)\n\t\tcidStr := node.IPFSAddDeterministicBytes(int64(exp.ChunkSize), seed)\n\n\t\t// Verify block structure based on raw leaves setting\n\t\tif exp.RawLeaves {\n\t\t\t// v1-2025: single block is a raw leaf (no dag-pb structure)\n\t\t\tcodec := node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", cidStr).Stdout.Trimmed()\n\t\t\trequire.Equal(t, \"raw\", codec, \"single block file is raw leaf\")\n\t\t} else {\n\t\t\t// v0-2015: single block is a dag-pb node with no links (TFile type)\n\t\t\troot, err := node.InspectPBNode(cidStr)\n\t\t\tassert.NoError(t, err)\n\t\t\trequire.Equal(t, 0, len(root.Links), \"single block file has no links\")\n\t\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, ft.TFile, fsType, \"single block file is dag-pb wrapped (TFile)\")\n\t\t}\n\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\tif exp.FileAtChunkSizeCID != \"\" {\n\t\t\trequire.Equal(t, exp.FileAtChunkSizeCID, cidStr, \"expected deterministic CID for file at chunk size\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_file-at-chunk-size.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n\n\t// Test: file 1 byte over UnixFSChunker threshold (2 blocks)\n\t// A file 1 byte over chunk size requires 2 chunks.\n\t// Root is a dag-pb node with 2 links. Leaf encoding depends on profile:\n\t// - v0-2015: leaf blocks are dag-pb wrapped TFile nodes\n\t// - v1-2025: leaf blocks are raw codec blocks\n\tt.Run(\"file 1 byte over UnixFSChunker threshold (2 blocks)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// File +1 byte over chunk size = 2 blocks\n\t\tseed := chunkSeedForProfile(exp)\n\t\tcidStr := node.IPFSAddDeterministicBytes(int64(exp.ChunkSize)+1, seed)\n\n\t\troot, err := node.InspectPBNode(cidStr)\n\t\tassert.NoError(t, err)\n\t\trequire.Equal(t, 2, len(root.Links), \"file over chunk size has 2 links\")\n\n\t\t// Verify leaf block encoding\n\t\tfor _, link := range root.Links {\n\t\t\tif exp.RawLeaves {\n\t\t\t\t// v1-2025: leaves are raw blocks\n\t\t\t\tleafCodec := node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", link.Hash.Slash).Stdout.Trimmed()\n\t\t\t\trequire.Equal(t, \"raw\", leafCodec, \"leaf blocks are raw, not dag-pb\")\n\t\t\t} else {\n\t\t\t\t// v0-2015: leaves are dag-pb wrapped (TFile type)\n\t\t\t\tleafType, err := node.UnixFSDataType(link.Hash.Slash)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, ft.TFile, leafType, \"leaf blocks are dag-pb wrapped (TFile)\")\n\t\t\t}\n\t\t}\n\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\tif exp.FileOverChunkSizeCID != \"\" {\n\t\t\trequire.Equal(t, exp.FileOverChunkSizeCID, cidStr, \"expected deterministic CID for file over chunk size\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_file-over-chunk-size.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n\n\t// Test: file at UnixFSFileMaxLinks threshold (single layer)\n\t// A file of exactly maxLinks * chunkSize bytes fits in a single DAG layer.\n\t// - v0-2015: 174 links (174 * 256KiB = ~44.6MiB)\n\t// - v1-2025: 1024 links (1024 * 1MiB = 1GiB)\n\tt.Run(\"file at UnixFSFileMaxLinks threshold (single layer)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// File size = maxLinks * chunkSize (exactly at threshold)\n\t\tfileSize := fileAtMaxLinksBytes(exp)\n\t\tseed := seedForProfile(exp)\n\t\tcidStr := node.IPFSAddDeterministicBytes(fileSize, seed)\n\n\t\troot, err := node.InspectPBNode(cidStr)\n\t\tassert.NoError(t, err)\n\t\trequire.Equal(t, exp.FileMaxLinks, len(root.Links),\n\t\t\t\"expected exactly %d links at max\", exp.FileMaxLinks)\n\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\tif exp.FileAtMaxLinksCID != \"\" {\n\t\t\trequire.Equal(t, exp.FileAtMaxLinksCID, cidStr, \"expected deterministic CID for file at max links\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_file-at-max-links.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n\n\t// Test: file 1 byte over UnixFSFileMaxLinks threshold (rebalanced DAG)\n\t// Adding 1 byte requires an additional chunk, exceeding maxLinks.\n\t// This triggers DAG rebalancing: chunks are grouped into intermediate nodes,\n\t// producing a 2-layer DAG with 2 links at the root.\n\tt.Run(\"file 1 byte over UnixFSFileMaxLinks threshold (rebalanced DAG)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// +1 byte over max links threshold triggers DAG rebalancing\n\t\tfileSize := fileOverMaxLinksBytes(exp)\n\t\tseed := seedForProfile(exp)\n\t\tcidStr := node.IPFSAddDeterministicBytes(fileSize, seed)\n\n\t\troot, err := node.InspectPBNode(cidStr)\n\t\tassert.NoError(t, err)\n\t\trequire.Equal(t, 2, len(root.Links), \"expected 2 links after DAG rebalancing\")\n\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\tif exp.FileOverMaxLinksCID != \"\" {\n\t\t\trequire.Equal(t, exp.FileOverMaxLinksCID, cidStr, \"expected deterministic CID for rebalanced file\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_file-over-max-links.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n\n\t// Test: directory at HAMTThreshold (basic flat dir)\n\t// A directory exactly at HAMTThreshold stays as a basic (flat) UnixFS directory.\n\t// Threshold uses > comparison (not >=), so size == threshold stays basic.\n\t// Size estimation method depends on profile:\n\t// - v0-2015 \"links\": size = sum(nameLen + cidLen)\n\t// - v1-2025 \"block\": size = serialized protobuf block size\n\tt.Run(\"directory at HAMTThreshold (basic flat dir)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Use consistent seed for deterministic CIDs\n\t\tseed := hamtSeedForProfile(exp)\n\t\trandDir, err := os.MkdirTemp(node.Dir, seed)\n\t\trequire.NoError(t, err)\n\n\t\t// Create basic (flat) directory exactly at threshold\n\t\tbasicLastNameLen := exp.DirBasicLastNameLen\n\t\tif basicLastNameLen == 0 {\n\t\t\tbasicLastNameLen = exp.DirBasicNameLen\n\t\t}\n\t\tif exp.HAMTSizeEstimation == \"block\" {\n\t\t\terr = createDirectoryForHAMTBlockEstimation(randDir, exp.DirBasicFiles, exp.DirBasicNameLen, basicLastNameLen, seed)\n\t\t} else {\n\t\t\terr = createDirectoryForHAMTLinksEstimation(randDir, exp.DirBasicFiles, exp.DirBasicNameLen, basicLastNameLen, seed)\n\t\t}\n\t\trequire.NoError(t, err)\n\n\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", randDir).Stdout.Trimmed()\n\n\t\t// Verify UnixFS type is TDirectory (1), not THAMTShard (5)\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.TDirectory, fsType, \"expected basic directory (type=1) at exact threshold\")\n\n\t\troot, err := node.InspectPBNode(cidStr)\n\t\tassert.NoError(t, err)\n\t\trequire.Equal(t, exp.DirBasicFiles, len(root.Links),\n\t\t\t\"expected basic directory with %d links\", exp.DirBasicFiles)\n\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\t// Verify size is exactly at threshold\n\t\tif exp.HAMTSizeEstimation == \"block\" {\n\t\t\tblockSize := getBlockSize(t, node, cidStr)\n\t\t\trequire.Equal(t, exp.HAMTThreshold, blockSize,\n\t\t\t\t\"expected basic directory block size to be exactly at threshold (%d), got %d\", exp.HAMTThreshold, blockSize)\n\t\t}\n\t\tif exp.HAMTSizeEstimation == \"links\" {\n\t\t\tlinksSize := 0\n\t\t\tfor _, link := range root.Links {\n\t\t\t\tlinksSize += len(link.Name) + cidLen\n\t\t\t}\n\t\t\trequire.Equal(t, exp.HAMTThreshold, linksSize,\n\t\t\t\t\"expected basic directory links size to be exactly at threshold (%d), got %d\", exp.HAMTThreshold, linksSize)\n\t\t}\n\n\t\tif exp.DirBasicCID != \"\" {\n\t\t\trequire.Equal(t, exp.DirBasicCID, cidStr, \"expected deterministic CID for basic directory\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_dir-basic.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s (%d files) -> %s\", cidStr, exp.DirBasicFiles, carPath)\n\t\t}\n\t})\n\n\t// Test: directory 1 byte over HAMTThreshold (HAMT sharded)\n\t// A directory 1 byte over HAMTThreshold is converted to a HAMT sharded structure.\n\t// HAMT distributes entries across buckets using consistent hashing.\n\t// Root has at most HAMTFanout links (256), with entries distributed across buckets.\n\tt.Run(\"directory 1 byte over HAMTThreshold (HAMT sharded)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(exp.ProfileArgs...)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Use consistent seed for deterministic CIDs\n\t\tseed := hamtSeedForProfile(exp)\n\t\trandDir, err := os.MkdirTemp(node.Dir, seed)\n\t\trequire.NoError(t, err)\n\n\t\t// Create HAMT (sharded) directory exactly +1 byte over threshold\n\t\tlastNameLen := exp.DirHAMTLastNameLen\n\t\tif lastNameLen == 0 {\n\t\t\tlastNameLen = exp.DirHAMTNameLen\n\t\t}\n\t\tif exp.HAMTSizeEstimation == \"block\" {\n\t\t\terr = createDirectoryForHAMTBlockEstimation(randDir, exp.DirHAMTFiles, exp.DirHAMTNameLen, lastNameLen, seed)\n\t\t} else {\n\t\t\terr = createDirectoryForHAMTLinksEstimation(randDir, exp.DirHAMTFiles, exp.DirHAMTNameLen, lastNameLen, seed)\n\t\t}\n\t\trequire.NoError(t, err)\n\n\t\tcidStr := node.IPFS(\"add\", \"-r\", \"-Q\", randDir).Stdout.Trimmed()\n\n\t\t// Verify UnixFS type is THAMTShard (5), not TDirectory (1)\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"expected HAMT directory (type=5) when over threshold\")\n\n\t\t// HAMT root has at most fanout links (actual count depends on hash distribution)\n\t\troot, err := node.InspectPBNode(cidStr)\n\t\tassert.NoError(t, err)\n\t\trequire.LessOrEqual(t, len(root.Links), exp.HAMTFanout,\n\t\t\t\"expected HAMT directory root to have <= %d links\", exp.HAMTFanout)\n\n\t\tverifyHashFunction(t, node, cidStr, exp.HashFunc)\n\n\t\tif exp.DirHAMTCID != \"\" {\n\t\t\trequire.Equal(t, exp.DirHAMTCID, cidStr, \"expected deterministic CID for HAMT directory\")\n\t\t}\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, exp.Name+\"_dir-hamt.car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s (%d files, HAMT root links: %d) -> %s\",\n\t\t\t\tcidStr, exp.DirHAMTFiles, len(root.Links), carPath)\n\t\t}\n\t})\n}\n\n// verifyCIDVersion checks that the CID has the expected version.\nfunc verifyCIDVersion(t *testing.T, _ *harness.Node, cidStr string, expectedVersion int) {\n\tt.Helper()\n\tif expectedVersion == 0 {\n\t\trequire.True(t, strings.HasPrefix(cidStr, \"Qm\"),\n\t\t\t\"expected CIDv0 (starts with Qm), got: %s\", cidStr)\n\t} else {\n\t\trequire.True(t, strings.HasPrefix(cidStr, \"b\"),\n\t\t\t\"expected CIDv1 (base32, starts with b), got: %s\", cidStr)\n\t}\n}\n\n// verifyHashFunction checks that the CID uses the expected hash function.\nfunc verifyHashFunction(t *testing.T, node *harness.Node, cidStr, expectedHash string) {\n\tt.Helper()\n\t// Use ipfs cid format to get hash function info\n\t// Format string %h gives the hash function name\n\tres := node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cidStr)\n\thashFunc := strings.TrimSpace(res.Stdout.String())\n\trequire.Equal(t, expectedHash, hashFunc,\n\t\t\"expected hash function %s, got %s for CID %s\", expectedHash, hashFunc, cidStr)\n}\n\n// verifyRawLeaves checks whether the CID represents a raw leaf or dag-pb wrapped block.\n// For CIDv1: raw leaves have codec 0x55 (raw), wrapped have codec 0x70 (dag-pb).\n// For CIDv0: always dag-pb (no raw leaves possible).\nfunc verifyRawLeaves(t *testing.T, node *harness.Node, cidStr string, expectRaw bool) {\n\tt.Helper()\n\t// Use ipfs cid format to get codec info\n\t// Format string %c gives the codec name\n\tres := node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", cidStr)\n\tcodec := strings.TrimSpace(res.Stdout.String())\n\n\tif expectRaw {\n\t\trequire.Equal(t, \"raw\", codec,\n\t\t\t\"expected raw codec for raw leaves, got %s for CID %s\", codec, cidStr)\n\t} else {\n\t\trequire.Equal(t, \"dag-pb\", codec,\n\t\t\t\"expected dag-pb codec for wrapped leaves, got %s for CID %s\", codec, cidStr)\n\t}\n}\n\n// getBlockSize returns the size of a block in bytes using ipfs block stat.\nfunc getBlockSize(t *testing.T, node *harness.Node, cidStr string) int {\n\tt.Helper()\n\tres := node.IPFS(\"block\", \"stat\", \"--enc=json\", cidStr)\n\tvar stat struct {\n\t\tSize int `json:\"Size\"`\n\t}\n\trequire.NoError(t, json.Unmarshal(res.Stdout.Bytes(), &stat))\n\treturn stat.Size\n}\n\n// fileAtMaxLinksBytes returns the file size in bytes that produces exactly FileMaxLinks chunks.\nfunc fileAtMaxLinksBytes(exp cidProfileExpectations) int64 {\n\treturn int64(exp.FileMaxLinks) * int64(exp.ChunkSize)\n}\n\n// fileOverMaxLinksBytes returns the file size in bytes that triggers DAG rebalancing (+1 byte over max links threshold).\nfunc fileOverMaxLinksBytes(exp cidProfileExpectations) int64 {\n\treturn int64(exp.FileMaxLinks)*int64(exp.ChunkSize) + 1\n}\n\n// seedForProfile returns the deterministic seed used in add_test.go for file max links tests.\nfunc seedForProfile(exp cidProfileExpectations) string {\n\tswitch exp.Name {\n\tcase \"unixfs-v0-2015\", \"default\":\n\t\treturn \"v0-seed\"\n\tcase \"unixfs-v1-2025\":\n\t\treturn \"v1-2025-seed\"\n\tdefault:\n\t\treturn exp.Name + \"-seed\"\n\t}\n}\n\n// chunkSeedForProfile returns the deterministic seed for chunk threshold tests.\nfunc chunkSeedForProfile(exp cidProfileExpectations) string {\n\tswitch exp.Name {\n\tcase \"unixfs-v0-2015\", \"default\":\n\t\treturn \"chunk-v0-seed\"\n\tcase \"unixfs-v1-2025\":\n\t\treturn \"chunk-v1-seed\"\n\tdefault:\n\t\treturn \"chunk-\" + exp.Name + \"-seed\"\n\t}\n}\n\n// hamtSeedForProfile returns the deterministic seed for HAMT directory tests.\n// Uses the same seed for both under/at threshold tests to ensure consistency.\nfunc hamtSeedForProfile(exp cidProfileExpectations) string {\n\tswitch exp.Name {\n\tcase \"unixfs-v0-2015\", \"default\":\n\t\treturn \"hamt-unixfs-v0-2015\"\n\tcase \"unixfs-v1-2025\":\n\t\treturn \"hamt-unixfs-v1-2025\"\n\tdefault:\n\t\treturn \"hamt-\" + exp.Name\n\t}\n}\n\n// TestDefaultMatchesExpectedProfile verifies that default ipfs add behavior\n// matches the expected profile (currently unixfs-v0-2015).\nfunc TestDefaultMatchesExpectedProfile(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Small file test\n\tcidDefault := node.IPFSAddStr(\"x\")\n\n\t// Same file with explicit profile\n\tnodeWithProfile := harness.NewT(t).NewNode().Init(defaultProfile.ProfileArgs...)\n\tnodeWithProfile.StartDaemon()\n\tdefer nodeWithProfile.StopDaemon()\n\n\tcidWithProfile := nodeWithProfile.IPFSAddStr(\"x\")\n\n\trequire.Equal(t, cidWithProfile, cidDefault,\n\t\t\"default behavior should match %s profile\", defaultProfile.Name)\n}\n\n// TestProtobufHelpers verifies the protobuf size calculation helpers.\nfunc TestProtobufHelpers(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"VarintLen\", func(t *testing.T) {\n\t\t// Varint encoding: 7 bits per byte, MSB indicates continuation\n\t\tcases := []struct {\n\t\t\tvalue uint64\n\t\t\texpected int\n\t\t}{\n\t\t\t{0, 1},\n\t\t\t{127, 1}, // 0x7F - max 1-byte varint\n\t\t\t{128, 2}, // 0x80 - min 2-byte varint\n\t\t\t{16383, 2}, // 0x3FFF - max 2-byte varint\n\t\t\t{16384, 3}, // 0x4000 - min 3-byte varint\n\t\t\t{2097151, 3}, // 0x1FFFFF - max 3-byte varint\n\t\t\t{2097152, 4}, // 0x200000 - min 4-byte varint\n\t\t\t{268435455, 4}, // 0xFFFFFFF - max 4-byte varint\n\t\t\t{268435456, 5}, // 0x10000000 - min 5-byte varint\n\t\t\t{34359738367, 5}, // 0x7FFFFFFFF - max 5-byte varint\n\t\t}\n\n\t\tfor _, tc := range cases {\n\t\t\tgot := testutils.VarintLen(tc.value)\n\t\t\trequire.Equal(t, tc.expected, got, \"VarintLen(%d)\", tc.value)\n\t\t}\n\t})\n\n\tt.Run(\"LinkSerializedSize\", func(t *testing.T) {\n\t\t// Test typical cases for directory links\n\t\tcases := []struct {\n\t\t\tnameLen int\n\t\t\tcidLen int\n\t\t\ttsize uint64\n\t\t\texpected int\n\t\t}{\n\t\t\t// 255-char name, CIDv0 (34 bytes), tsize=0\n\t\t\t// Inner: 1+1+34 + 1+2+255 + 1+1 = 296\n\t\t\t// Outer: 1 + 2 + 296 = 299\n\t\t\t{255, 34, 0, 299},\n\t\t\t// 255-char name, CIDv1 (36 bytes), tsize=0\n\t\t\t// Inner: 1+1+36 + 1+2+255 + 1+1 = 298\n\t\t\t// Outer: 1 + 2 + 298 = 301\n\t\t\t{255, 36, 0, 301},\n\t\t\t// Short name (10 chars), CIDv1, tsize=0\n\t\t\t// Inner: 1+1+36 + 1+1+10 + 1+1 = 52\n\t\t\t// Outer: 1 + 1 + 52 = 54\n\t\t\t{10, 36, 0, 54},\n\t\t\t// 255-char name, CIDv1, large tsize\n\t\t\t// Inner: 1+1+36 + 1+2+255 + 1+5 = 302 (tsize uses 5-byte varint)\n\t\t\t// Outer: 1 + 2 + 302 = 305\n\t\t\t{255, 36, 34359738367, 305},\n\t\t}\n\n\t\tfor _, tc := range cases {\n\t\t\tgot := testutils.LinkSerializedSize(tc.nameLen, tc.cidLen, tc.tsize)\n\t\t\trequire.Equal(t, tc.expected, got, \"LinkSerializedSize(%d, %d, %d)\", tc.nameLen, tc.cidLen, tc.tsize)\n\t\t}\n\t})\n\n\tt.Run(\"EstimateFilesForBlockThreshold\", func(t *testing.T) {\n\t\tthreshold := 262144\n\t\tnameLen := 255\n\t\tcidLen := 36\n\t\tvar tsize uint64 = 0\n\n\t\tnumFiles := testutils.EstimateFilesForBlockThreshold(threshold, nameLen, cidLen, tsize)\n\t\trequire.Equal(t, 870, numFiles, \"expected 870 files for threshold 262144\")\n\n\t\tnumFilesUnder := testutils.EstimateFilesForBlockThreshold(threshold-1, nameLen, cidLen, tsize)\n\t\trequire.Equal(t, 870, numFilesUnder, \"expected 870 files for threshold 262143\")\n\n\t\tnumFilesOver := testutils.EstimateFilesForBlockThreshold(262185, nameLen, cidLen, tsize)\n\t\trequire.Equal(t, 871, numFilesOver, \"expected 871 files for threshold 262185\")\n\t})\n}\n" }, { "path": "test/cli/cid_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\tcid \"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\tpeer \"github.com/libp2p/go-libp2p/core/peer\"\n\tmhash \"github.com/multiformats/go-multihash\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestCidCommands(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"inspect\", testCidInspect)\n\tt.Run(\"base32\", testCidBase32)\n\tt.Run(\"format\", testCidFormat)\n\tt.Run(\"bases\", testCidBases)\n\tt.Run(\"codecs\", testCidCodecs)\n\tt.Run(\"hashes\", testCidHashes)\n}\n\n// testCidInspect tests 'ipfs cid inspect' subcommand\nfunc testCidInspect(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tt.Run(\"CIDv0\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"CID: QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\")\n\t\tassert.Contains(t, out, \"Version: 0\")\n\t\tassert.Contains(t, out, \"Multibase: base58btc (implicit)\")\n\t\tassert.Contains(t, out, \"Multicodec: dag-pb (0x70, implicit)\")\n\t\tassert.Contains(t, out, \"Multihash: sha2-256 (0x12, implicit)\")\n\t\tassert.Contains(t, out, \" Length: 32 bytes\")\n\t\tassert.Contains(t, out, \" Digest: c3c4733ec8affd06cf9e9ff50ffc6bcd2ec85a6170004bb709669c31de94391a\")\n\t\tassert.Contains(t, out, \"CIDv0: QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\")\n\t\tassert.Contains(t, out, \"CIDv1: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t})\n\n\tt.Run(\"CIDv1 base32 dag-pb\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"CID: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\tassert.Contains(t, out, \"Version: 1\")\n\t\tassert.Contains(t, out, \"Multibase: base32 (b)\")\n\t\tassert.Contains(t, out, \"Multicodec: dag-pb (0x70)\")\n\t\tassert.Contains(t, out, \"Multihash: sha2-256 (0x12)\")\n\t\tassert.Contains(t, out, \" Length: 32 bytes\")\n\t\tassert.Contains(t, out, \" Digest: c3c4733ec8affd06cf9e9ff50ffc6bcd2ec85a6170004bb709669c31de94391a\")\n\t\tassert.NotContains(t, out, \"implicit\")\n\t\tassert.Contains(t, out, \"CIDv0: QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\")\n\t\tassert.Contains(t, out, \"CIDv1: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t})\n\n\tt.Run(\"CIDv1 raw codec\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"bafkreigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"CID: bafkreigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\tassert.Contains(t, out, \"Multibase: base32 (b)\")\n\t\tassert.Contains(t, out, \"Multicodec: raw (0x55)\")\n\t\tassert.Contains(t, out, \"Multihash: sha2-256 (0x12)\")\n\t\tassert.Contains(t, out, \" Length: 32 bytes\")\n\t\tassert.Contains(t, out, \" Digest: c3c4733ec8affd06cf9e9ff50ffc6bcd2ec85a6170004bb709669c31de94391a\")\n\t\tassert.Contains(t, out, \"CIDv0: not possible, requires dag-pb (0x70), got raw (0x55)\")\n\t\tassert.Contains(t, out, \"CIDv1: bafkreigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t})\n\n\tt.Run(\"CIDv1 base36\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"k2jmtxw8rjh1z69c6not3wtdxb0u3urbzhyll1t9jg6ox26dhi5sfi1m\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"CID: k2jmtxw8rjh1z69c6not3wtdxb0u3urbzhyll1t9jg6ox26dhi5sfi1m\")\n\t\tassert.Contains(t, out, \"Multibase: base36 (k)\")\n\t\tassert.Contains(t, out, \"Multicodec: dag-pb (0x70)\")\n\t\tassert.Contains(t, out, \" Digest: c3c4733ec8affd06cf9e9ff50ffc6bcd2ec85a6170004bb709669c31de94391a\")\n\t\tassert.Contains(t, out, \"CIDv0: QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR\")\n\t\tassert.Contains(t, out, \"CIDv1: bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t})\n\n\tt.Run(\"invalid CID\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"garbage\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"invalid CID\")\n\t})\n\n\tt.Run(\"PeerID as input\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"12D3KooWD3eckifWpRn9wQpMG9R9hX3sD158z7EqHWmweQAJU5SA\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tstderr := res.Stderr.String()\n\t\tassert.Contains(t, stderr, \"PeerID\")\n\t\tassert.Contains(t, stderr, \"inspect its CID representation instead\")\n\t\t// suggested CID should use base36 (k prefix)\n\t\tassert.Contains(t, stderr, \"\\n k\")\n\t})\n\n\tt.Run(\"libp2p-key CID uses base36\", func(t *testing.T) {\n\t\t// Construct a libp2p-key CIDv1 from a known PeerID\n\t\tpid, err := peer.Decode(\"12D3KooWD3eckifWpRn9wQpMG9R9hX3sD158z7EqHWmweQAJU5SA\")\n\t\trequire.NoError(t, err)\n\t\tpidCid := peer.ToCid(pid)\n\t\tcidStr := pidCid.String()\n\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", cidStr)\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"Multicodec: libp2p-key (0x72)\")\n\t\t// CIDv1 should use base36 (k prefix)\n\t\tassert.Contains(t, out, \"CIDv1: k\")\n\t})\n\n\tt.Run(\"identity multihash CID\", func(t *testing.T) {\n\t\t// raw codec + identity multihash: digest is the raw content (\"test\" = 74657374)\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"bafkqabdumvzxi\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"CID: bafkqabdumvzxi\")\n\t\tassert.Contains(t, out, \"Multicodec: raw (0x55)\")\n\t\tassert.Contains(t, out, \"Multihash: identity (0x0)\")\n\t\tassert.Contains(t, out, \" Length: 4 bytes\")\n\t\tassert.Contains(t, out, \" Digest: 74657374\")\n\t})\n\n\tt.Run(\"unknown codec\", func(t *testing.T) {\n\t\t// Construct a CID with unknown codec 0x9999\n\t\tmh, err := mhash.Sum([]byte(\"test\"), mhash.SHA2_256, -1)\n\t\trequire.NoError(t, err)\n\t\tunknownCID := cid.NewCidV1(0x9999, mh)\n\t\tcidStr := unknownCID.String()\n\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", cidStr)\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tout := res.Stdout.String()\n\t\tassert.Contains(t, out, \"Multicodec: unknown (0x9999)\")\n\t\tassert.Contains(t, out, \"not possible, requires dag-pb (0x70), got unknown (0x9999)\")\n\t})\n\n\tt.Run(\"JSON output\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"--enc=json\", \"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tvar result map[string]any\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &result)\n\t\trequire.NoError(t, err)\n\n\t\t// multibase.prefix should be a string, not a number\n\t\tmb := result[\"multibase\"].(map[string]any)\n\t\tassert.IsType(t, \"\", mb[\"prefix\"])\n\t\tassert.Equal(t, \"b\", mb[\"prefix\"])\n\n\t\t// multihash.length should be a number (bytes)\n\t\tmh := result[\"multihash\"].(map[string]any)\n\t\tassert.Equal(t, float64(32), mh[\"length\"])\n\n\t\t// cidV0 should be a clean CID string, no explanatory text\n\t\tcidV0 := result[\"cidV0\"].(string)\n\t\tassert.True(t, strings.HasPrefix(cidV0, \"Qm\"), \"cidV0 should be a valid CIDv0\")\n\n\t\t// cidV1 should be a clean CID string\n\t\tcidV1 := result[\"cidV1\"].(string)\n\t\tassert.True(t, strings.HasPrefix(cidV1, \"b\"), \"cidV1 should be base32 encoded\")\n\t})\n\n\tt.Run(\"JSON output with empty CIDv0\", func(t *testing.T) {\n\t\t// raw codec can't be CIDv0\n\t\tres := node.RunIPFS(\"cid\", \"inspect\", \"--enc=json\", \"bafkreigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tvar result map[string]any\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &result)\n\t\trequire.NoError(t, err)\n\n\t\t// cidV0 should not be present (omitempty)\n\t\t_, hasCidV0 := result[\"cidV0\"]\n\t\tassert.False(t, hasCidV0, \"cidV0 should be omitted when not possible\")\n\t})\n}\n\n// testCidBase32 tests 'ipfs cid base32' subcommand\n// Includes regression tests for https://github.com/ipfs/kubo/issues/9007\nfunc testCidBase32(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tt.Run(\"converts valid CIDs to base32\", func(t *testing.T) {\n\t\tt.Run(\"CIDv0 to base32\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"base32\", \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"CIDv1 base58 to base32\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"base32\", \"zdj7WgefqQm5HogBQ2bckZuTYYDarRTUZi51GYCnerHD2G86j\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"already base32 CID remains unchanged\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"base32\", \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"multiple valid CIDs\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"base32\",\n\t\t\t\t\"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\",\n\t\t\t\t\"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Empty(t, res.Stderr.String())\n\t\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\t\tassert.Equal(t, 2, len(lines))\n\t\t\tassert.Equal(t, \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\", lines[0])\n\t\t\tassert.Equal(t, \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\", lines[1])\n\t\t})\n\t})\n\n\tt.Run(\"error handling\", func(t *testing.T) {\n\t\t// Regression tests for https://github.com/ipfs/kubo/issues/9007\n\t\tt.Run(\"returns error code 1 for single invalid CID\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"base32\", \"invalid-cid\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"invalid-cid: invalid cid\")\n\t\t\tassert.Contains(t, res.Stderr.String(), \"Error: errors while displaying some entries\")\n\t\t})\n\n\t\tt.Run(\"returns error code 1 for mixed valid and invalid CIDs\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"base32\", \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\", \"invalid-cid\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t// Valid CID should be converted and printed to stdout\n\t\t\tassert.Contains(t, res.Stdout.String(), \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\")\n\t\t\t// Invalid CID error should be printed to stderr\n\t\t\tassert.Contains(t, res.Stderr.String(), \"invalid-cid: invalid cid\")\n\t\t\tassert.Contains(t, res.Stderr.String(), \"Error: errors while displaying some entries\")\n\t\t})\n\n\t\tt.Run(\"returns error code 1 for stdin with invalid CIDs\", func(t *testing.T) {\n\t\t\tinput := \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\\nbad-cid\\nbafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\"\n\t\t\tres := node.RunPipeToIPFS(strings.NewReader(input), \"cid\", \"base32\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t// Valid CIDs should be converted\n\t\t\tassert.Contains(t, res.Stdout.String(), \"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\")\n\t\t\t// Invalid CID error should be in stderr\n\t\t\tassert.Contains(t, res.Stderr.String(), \"bad-cid: invalid cid\")\n\t\t})\n\t})\n}\n\n// testCidFormat tests 'ipfs cid format' subcommand\n// Includes regression tests for https://github.com/ipfs/kubo/issues/9007\nfunc testCidFormat(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tt.Run(\"formats CIDs with various options\", func(t *testing.T) {\n\t\tt.Run(\"default format preserves CID\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"convert to CIDv1 with base58btc\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"-v\", \"1\", \"-b\", \"base58btc\",\n\t\t\t\t\"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"zdj7WgefqQm5HogBQ2bckZuTYYDarRTUZi51GYCnerHD2G86j\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"convert to CIDv0\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"-v\", \"0\",\n\t\t\t\t\"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"change codec to raw\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"--mc\", \"raw\", \"-b\", \"base32\",\n\t\t\t\t\"bafybeievd6mwe6vcwnkwo3eizs3h7w3a34opszbyfxziqdxguhjw7imdve\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Equal(t, \"bafkreievd6mwe6vcwnkwo3eizs3h7w3a34opszbyfxziqdxguhjw7imdve\\n\", res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"multiple valid CIDs with format options\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"-v\", \"1\", \"-b\", \"base58btc\",\n\t\t\t\t\"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\",\n\t\t\t\t\"bafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Empty(t, res.Stderr.String())\n\t\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\t\tassert.Equal(t, 2, len(lines))\n\t\t\tassert.Equal(t, \"zdj7WgefqQm5HogBQ2bckZuTYYDarRTUZi51GYCnerHD2G86j\", lines[0])\n\t\t\tassert.Equal(t, \"zdj7WgefqQm5HogBQ2bckZuTYYDarRTUZi51GYCnerHD2G86j\", lines[1])\n\t\t})\n\t})\n\n\tt.Run(\"error handling\", func(t *testing.T) {\n\t\t// Regression tests for https://github.com/ipfs/kubo/issues/9007\n\t\tt.Run(\"returns error code 1 for single invalid CID\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"not-a-cid\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"not-a-cid: invalid cid\")\n\t\t\tassert.Contains(t, res.Stderr.String(), \"Error: errors while displaying some entries\")\n\t\t})\n\n\t\tt.Run(\"returns error code 1 for mixed valid and invalid CIDs\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"cid\", \"format\", \"not-a-cid\", \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t// Valid CID should be printed to stdout\n\t\t\tassert.Contains(t, res.Stdout.String(), \"QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\")\n\t\t\t// Invalid CID error should be printed to stderr\n\t\t\tassert.Contains(t, res.Stderr.String(), \"not-a-cid: invalid cid\")\n\t\t\tassert.Contains(t, res.Stderr.String(), \"Error: errors while displaying some entries\")\n\t\t})\n\n\t\tt.Run(\"returns error code 1 for stdin with invalid CIDs\", func(t *testing.T) {\n\t\t\tinput := \"invalid\\nQmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\"\n\t\t\tres := node.RunPipeToIPFS(strings.NewReader(input), \"cid\", \"format\", \"-v\", \"1\", \"-b\", \"base58btc\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t// Valid CID should be converted\n\t\t\tassert.Contains(t, res.Stdout.String(), \"zdj7WgefqQm5HogBQ2bckZuTYYDarRTUZi51GYCnerHD2G86j\")\n\t\t\t// Invalid CID error should be in stderr\n\t\t\tassert.Contains(t, res.Stderr.String(), \"invalid: invalid cid\")\n\t\t})\n\t})\n}\n\n// testCidBases tests 'ipfs cid bases' subcommand\nfunc testCidBases(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tt.Run(\"lists available bases\", func(t *testing.T) {\n\t\t// This is a regression test to ensure we don't accidentally add or remove support\n\t\t// for multibase encodings. If a new base is intentionally added or removed,\n\t\t// this test should be updated accordingly.\n\t\texpectedBases := []string{\n\t\t\t\"identity\",\n\t\t\t\"base2\",\n\t\t\t\"base16\",\n\t\t\t\"base16upper\",\n\t\t\t\"base32\",\n\t\t\t\"base32upper\",\n\t\t\t\"base32pad\",\n\t\t\t\"base32padupper\",\n\t\t\t\"base32hex\",\n\t\t\t\"base32hexupper\",\n\t\t\t\"base32hexpad\",\n\t\t\t\"base32hexpadupper\",\n\t\t\t\"base36\",\n\t\t\t\"base36upper\",\n\t\t\t\"base58btc\",\n\t\t\t\"base58flickr\",\n\t\t\t\"base64\",\n\t\t\t\"base64pad\",\n\t\t\t\"base64url\",\n\t\t\t\"base64urlpad\",\n\t\t\t\"base256emoji\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"bases\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"bases\", expectedBases, lines)\n\t})\n\n\tt.Run(\"with --prefix flag shows single letter prefixes\", func(t *testing.T) {\n\t\t// Regression test to catch any changes to the output format or supported bases\n\t\texpectedLines := []string{\n\t\t\t\"identity\",\n\t\t\t\"0 base2\",\n\t\t\t\"b base32\",\n\t\t\t\"B base32upper\",\n\t\t\t\"c base32pad\",\n\t\t\t\"C base32padupper\",\n\t\t\t\"f base16\",\n\t\t\t\"F base16upper\",\n\t\t\t\"k base36\",\n\t\t\t\"K base36upper\",\n\t\t\t\"m base64\",\n\t\t\t\"M base64pad\",\n\t\t\t\"t base32hexpad\",\n\t\t\t\"T base32hexpadupper\",\n\t\t\t\"u base64url\",\n\t\t\t\"U base64urlpad\",\n\t\t\t\"v base32hex\",\n\t\t\t\"V base32hexupper\",\n\t\t\t\"z base58btc\",\n\t\t\t\"Z base58flickr\",\n\t\t\t\"🚀 base256emoji\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"bases\", \"--prefix\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"bases --prefix output\", expectedLines, lines)\n\t})\n\n\tt.Run(\"with --numeric flag shows numeric codes\", func(t *testing.T) {\n\t\t// Regression test to catch any changes to the output format or supported bases\n\t\texpectedLines := []string{\n\t\t\t\"0 identity\",\n\t\t\t\"48 base2\",\n\t\t\t\"98 base32\",\n\t\t\t\"66 base32upper\",\n\t\t\t\"99 base32pad\",\n\t\t\t\"67 base32padupper\",\n\t\t\t\"102 base16\",\n\t\t\t\"70 base16upper\",\n\t\t\t\"107 base36\",\n\t\t\t\"75 base36upper\",\n\t\t\t\"109 base64\",\n\t\t\t\"77 base64pad\",\n\t\t\t\"116 base32hexpad\",\n\t\t\t\"84 base32hexpadupper\",\n\t\t\t\"117 base64url\",\n\t\t\t\"85 base64urlpad\",\n\t\t\t\"118 base32hex\",\n\t\t\t\"86 base32hexupper\",\n\t\t\t\"122 base58btc\",\n\t\t\t\"90 base58flickr\",\n\t\t\t\"128640 base256emoji\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"bases\", \"--numeric\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"bases --numeric output\", expectedLines, lines)\n\t})\n\n\tt.Run(\"with both --prefix and --numeric flags\", func(t *testing.T) {\n\t\t// Regression test to catch any changes to the output format or supported bases\n\t\texpectedLines := []string{\n\t\t\t\"0 identity\",\n\t\t\t\"0 48 base2\",\n\t\t\t\"b 98 base32\",\n\t\t\t\"B 66 base32upper\",\n\t\t\t\"c 99 base32pad\",\n\t\t\t\"C 67 base32padupper\",\n\t\t\t\"f 102 base16\",\n\t\t\t\"F 70 base16upper\",\n\t\t\t\"k 107 base36\",\n\t\t\t\"K 75 base36upper\",\n\t\t\t\"m 109 base64\",\n\t\t\t\"M 77 base64pad\",\n\t\t\t\"t 116 base32hexpad\",\n\t\t\t\"T 84 base32hexpadupper\",\n\t\t\t\"u 117 base64url\",\n\t\t\t\"U 85 base64urlpad\",\n\t\t\t\"v 118 base32hex\",\n\t\t\t\"V 86 base32hexupper\",\n\t\t\t\"z 122 base58btc\",\n\t\t\t\"Z 90 base58flickr\",\n\t\t\t\"🚀 128640 base256emoji\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"bases\", \"--prefix\", \"--numeric\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"bases --prefix --numeric output\", expectedLines, lines)\n\t})\n}\n\n// testCidCodecs tests 'ipfs cid codecs' subcommand\nfunc testCidCodecs(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tt.Run(\"lists available codecs\", func(t *testing.T) {\n\t\t// This is a regression test to ensure we don't accidentally add or remove\n\t\t// IPLD codecs. If a codec is intentionally added or removed,\n\t\t// this test should be updated accordingly.\n\t\texpectedCodecs := []string{\n\t\t\t\"cbor\",\n\t\t\t\"raw\",\n\t\t\t\"dag-pb\",\n\t\t\t\"dag-cbor\",\n\t\t\t\"libp2p-key\",\n\t\t\t\"git-raw\",\n\t\t\t\"torrent-info\",\n\t\t\t\"torrent-file\",\n\t\t\t\"blake3-hashseq\",\n\t\t\t\"leofcoin-block\",\n\t\t\t\"leofcoin-tx\",\n\t\t\t\"leofcoin-pr\",\n\t\t\t\"dag-jose\",\n\t\t\t\"dag-cose\",\n\t\t\t\"eth-block\",\n\t\t\t\"eth-block-list\",\n\t\t\t\"eth-tx-trie\",\n\t\t\t\"eth-tx\",\n\t\t\t\"eth-tx-receipt-trie\",\n\t\t\t\"eth-tx-receipt\",\n\t\t\t\"eth-state-trie\",\n\t\t\t\"eth-account-snapshot\",\n\t\t\t\"eth-storage-trie\",\n\t\t\t\"eth-receipt-log-trie\",\n\t\t\t\"eth-receipt-log\",\n\t\t\t\"bitcoin-block\",\n\t\t\t\"bitcoin-tx\",\n\t\t\t\"bitcoin-witness-commitment\",\n\t\t\t\"zcash-block\",\n\t\t\t\"zcash-tx\",\n\t\t\t\"stellar-block\",\n\t\t\t\"stellar-tx\",\n\t\t\t\"decred-block\",\n\t\t\t\"decred-tx\",\n\t\t\t\"dash-block\",\n\t\t\t\"dash-tx\",\n\t\t\t\"swarm-manifest\",\n\t\t\t\"swarm-feed\",\n\t\t\t\"beeson\",\n\t\t\t\"dag-json\",\n\t\t\t\"swhid-1-snp\",\n\t\t\t\"json\",\n\t\t\t\"rdfc-1\",\n\t\t\t\"json-jcs\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"codecs\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"codecs\", expectedCodecs, lines)\n\t})\n\n\tt.Run(\"with --numeric flag shows codec numbers\", func(t *testing.T) {\n\t\t// This is a regression test to ensure we don't accidentally add or remove\n\t\t// IPLD codecs. If a codec is intentionally added or removed,\n\t\t// this test should be updated accordingly.\n\t\texpectedLines := []string{\n\t\t\t\"81 cbor\",\n\t\t\t\"85 raw\",\n\t\t\t\"112 dag-pb\",\n\t\t\t\"113 dag-cbor\",\n\t\t\t\"114 libp2p-key\",\n\t\t\t\"120 git-raw\",\n\t\t\t\"123 torrent-info\",\n\t\t\t\"124 torrent-file\",\n\t\t\t\"128 blake3-hashseq\",\n\t\t\t\"129 leofcoin-block\",\n\t\t\t\"130 leofcoin-tx\",\n\t\t\t\"131 leofcoin-pr\",\n\t\t\t\"133 dag-jose\",\n\t\t\t\"134 dag-cose\",\n\t\t\t\"144 eth-block\",\n\t\t\t\"145 eth-block-list\",\n\t\t\t\"146 eth-tx-trie\",\n\t\t\t\"147 eth-tx\",\n\t\t\t\"148 eth-tx-receipt-trie\",\n\t\t\t\"149 eth-tx-receipt\",\n\t\t\t\"150 eth-state-trie\",\n\t\t\t\"151 eth-account-snapshot\",\n\t\t\t\"152 eth-storage-trie\",\n\t\t\t\"153 eth-receipt-log-trie\",\n\t\t\t\"154 eth-receipt-log\",\n\t\t\t\"176 bitcoin-block\",\n\t\t\t\"177 bitcoin-tx\",\n\t\t\t\"178 bitcoin-witness-commitment\",\n\t\t\t\"192 zcash-block\",\n\t\t\t\"193 zcash-tx\",\n\t\t\t\"208 stellar-block\",\n\t\t\t\"209 stellar-tx\",\n\t\t\t\"224 decred-block\",\n\t\t\t\"225 decred-tx\",\n\t\t\t\"240 dash-block\",\n\t\t\t\"241 dash-tx\",\n\t\t\t\"250 swarm-manifest\",\n\t\t\t\"251 swarm-feed\",\n\t\t\t\"252 beeson\",\n\t\t\t\"297 dag-json\",\n\t\t\t\"496 swhid-1-snp\",\n\t\t\t\"512 json\",\n\t\t\t\"46083 rdfc-1\",\n\t\t\t\"46593 json-jcs\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"codecs\", \"--numeric\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"codecs --numeric output\", expectedLines, lines)\n\t})\n\n\tt.Run(\"with --supported flag lists only supported codecs\", func(t *testing.T) {\n\t\t// This is a regression test to ensure we don't accidentally change the list\n\t\t// of supported codecs. If a codec is intentionally added or removed from\n\t\t// support, this test should be updated accordingly.\n\t\texpectedSupportedCodecs := []string{\n\t\t\t\"cbor\",\n\t\t\t\"dag-cbor\",\n\t\t\t\"dag-jose\",\n\t\t\t\"dag-json\",\n\t\t\t\"dag-pb\",\n\t\t\t\"git-raw\",\n\t\t\t\"json\",\n\t\t\t\"libp2p-key\",\n\t\t\t\"raw\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"codecs\", \"--supported\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"supported codecs\", expectedSupportedCodecs, lines)\n\t})\n\n\tt.Run(\"with both --supported and --numeric flags\", func(t *testing.T) {\n\t\t// Regression test to catch any changes to supported codecs or output format\n\t\texpectedLines := []string{\n\t\t\t\"81 cbor\",\n\t\t\t\"85 raw\",\n\t\t\t\"112 dag-pb\",\n\t\t\t\"113 dag-cbor\",\n\t\t\t\"114 libp2p-key\",\n\t\t\t\"120 git-raw\",\n\t\t\t\"133 dag-jose\",\n\t\t\t\"297 dag-json\",\n\t\t\t\"512 json\",\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"codecs\", \"--supported\", \"--numeric\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"codecs --supported --numeric output\", expectedLines, lines)\n\t})\n}\n\n// testCidHashes tests 'ipfs cid hashes' subcommand\nfunc testCidHashes(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode()\n\n\tt.Run(\"lists available hashes\", func(t *testing.T) {\n\t\t// This is a regression test to ensure we don't accidentally add or remove\n\t\t// support for hash functions. If a hash function is intentionally added\n\t\t// or removed, this test should be updated accordingly.\n\t\texpectedHashes := []string{\n\t\t\t\"identity\",\n\t\t\t\"sha1\",\n\t\t\t\"sha2-256\",\n\t\t\t\"sha2-512\",\n\t\t\t\"sha3-512\",\n\t\t\t\"sha3-384\",\n\t\t\t\"sha3-256\",\n\t\t\t\"sha3-224\",\n\t\t\t\"shake-256\",\n\t\t\t\"keccak-224\",\n\t\t\t\"keccak-256\",\n\t\t\t\"keccak-384\",\n\t\t\t\"keccak-512\",\n\t\t\t\"blake3\",\n\t\t\t\"dbl-sha2-256\",\n\t\t}\n\n\t\t// Also expect all blake2b variants (160-512 in steps of 8)\n\t\tfor i := 160; i <= 512; i += 8 {\n\t\t\texpectedHashes = append(expectedHashes, fmt.Sprintf(\"blake2b-%d\", i))\n\t\t}\n\n\t\t// Also expect all blake2s variants (160-256 in steps of 8)\n\t\tfor i := 160; i <= 256; i += 8 {\n\t\t\texpectedHashes = append(expectedHashes, fmt.Sprintf(\"blake2s-%d\", i))\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"hashes\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"hash functions\", expectedHashes, lines)\n\t})\n\n\tt.Run(\"with --numeric flag shows hash function codes\", func(t *testing.T) {\n\t\t// This is a regression test to ensure we don't accidentally add or remove\n\t\t// support for hash functions. If a hash function is intentionally added\n\t\t// or removed, this test should be updated accordingly.\n\t\texpectedLines := []string{\n\t\t\t\"0 identity\",\n\t\t\t\"17 sha1\",\n\t\t\t\"18 sha2-256\",\n\t\t\t\"19 sha2-512\",\n\t\t\t\"20 sha3-512\",\n\t\t\t\"21 sha3-384\",\n\t\t\t\"22 sha3-256\",\n\t\t\t\"23 sha3-224\",\n\t\t\t\"25 shake-256\",\n\t\t\t\"26 keccak-224\",\n\t\t\t\"27 keccak-256\",\n\t\t\t\"28 keccak-384\",\n\t\t\t\"29 keccak-512\",\n\t\t\t\"30 blake3\",\n\t\t\t\"86 dbl-sha2-256\",\n\t\t}\n\n\t\t// Add all blake2b variants (160-512 in steps of 8)\n\t\tfor i := 160; i <= 512; i += 8 {\n\t\t\texpectedLines = append(expectedLines, fmt.Sprintf(\"%d blake2b-%d\", 45568+i/8, i))\n\t\t}\n\n\t\t// Add all blake2s variants (160-256 in steps of 8)\n\t\tfor i := 160; i <= 256; i += 8 {\n\t\t\texpectedLines = append(expectedLines, fmt.Sprintf(\"%d blake2s-%d\", 45632+i/8, i))\n\t\t}\n\n\t\tres := node.RunIPFS(\"cid\", \"hashes\", \"--numeric\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\tlines := strings.Split(strings.TrimSpace(res.Stdout.String()), \"\\n\")\n\t\tassertExactSet(t, \"hashes --numeric output\", expectedLines, lines)\n\t})\n}\n\n// assertExactSet compares expected vs actual items and reports clear errors for any differences.\n// This is used as a regression test to ensure we don't accidentally add or remove support.\n// Both expected and actual strings are trimmed of whitespace before comparison for maintainability.\nfunc assertExactSet(t *testing.T, itemType string, expected []string, actual []string) {\n\tt.Helper()\n\n\t// Normalize by trimming whitespace\n\tnormalizedExpected := make([]string, len(expected))\n\tfor i, item := range expected {\n\t\tnormalizedExpected[i] = strings.TrimSpace(item)\n\t}\n\n\tnormalizedActual := make([]string, len(actual))\n\tfor i, item := range actual {\n\t\tnormalizedActual[i] = strings.TrimSpace(item)\n\t}\n\n\texpectedSet := make(map[string]bool)\n\tfor _, item := range normalizedExpected {\n\t\texpectedSet[item] = true\n\t}\n\n\tactualSet := make(map[string]bool)\n\tfor _, item := range normalizedActual {\n\t\tactualSet[item] = true\n\t}\n\n\tvar missing []string\n\tfor _, item := range normalizedExpected {\n\t\tif !actualSet[item] {\n\t\t\tmissing = append(missing, item)\n\t\t}\n\t}\n\n\tvar unexpected []string\n\tfor _, item := range normalizedActual {\n\t\tif !expectedSet[item] {\n\t\t\tunexpected = append(unexpected, item)\n\t\t}\n\t}\n\n\tif len(missing) > 0 {\n\t\tt.Errorf(\"Missing expected %s: %q\", itemType, missing)\n\t}\n\tif len(unexpected) > 0 {\n\t\tt.Errorf(\"Unexpected %s found: %q\", itemType, unexpected)\n\t}\n\n\tassert.Equal(t, len(expected), len(actual),\n\t\t\"Expected %d %s but got %d\", len(expected), itemType, len(actual))\n}\n" }, { "path": "test/cli/cli_https_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"net/url\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestCLIWithRemoteHTTPS(t *testing.T) {\n\ttests := []struct{ addrSuffix string }{{\"https\"}, {\"tls/http\"}}\n\tfor _, tt := range tests {\n\t\tt.Run(\"with \"+tt.addrSuffix+\" multiaddr\", func(t *testing.T) {\n\n\t\t\t// Create HTTPS test server\n\t\t\tserver := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tif r.TLS == nil {\n\t\t\t\t\tt.Error(\"Mocked Kubo RPC received plain HTTP request instead of HTTPS TLS Handshake\")\n\t\t\t\t}\n\t\t\t\t_, _ = w.Write([]byte(\"OK\"))\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\tserverURL, _ := url.Parse(server.URL)\n\t\t\t_, port, _ := net.SplitHostPort(serverURL.Host)\n\n\t\t\t// Create Kubo repo\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Attempt to talk to remote Kubo RPC endpoint over HTTPS\n\t\t\tresp := node.RunIPFS(\"id\", \"--api\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%s/%s\", port, tt.addrSuffix))\n\n\t\t\t// Expect HTTPS error (confirming TLS and https:// were used, and not Cleartext HTTP)\n\t\t\trequire.Error(t, resp.Err)\n\t\t\trequire.Contains(t, resp.Stderr.String(), \"Error: tls: failed to verify certificate: x509: certificate signed by unknown authority\")\n\n\t\t\tnode.StopDaemon()\n\n\t\t})\n\t}\n}\n" }, { "path": "test/cli/commands_without_repo_test.go", "content": "package cli\n\nimport (\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc TestCommandsWithoutRepo(t *testing.T) {\n\tt.Run(\"cid\", func(t *testing.T) {\n\t\tt.Run(\"base32\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"cid\", \"base32\", \"QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\texpected := \"bafybeibxm2nsadl3fnxv2sxcxmxaco2jl53wpeorjdzidjwf5aqdg7wa6u\\n\"\n\t\t\tif string(stdout) != expected {\n\t\t\t\tt.Fatalf(\"expected %q, got: %q\", expected, stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"format\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"cid\", \"format\", \"-v\", \"1\", \"QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\texpected := \"zdj7WZAAFKPvYPPzyJLso2hhxo8a7ZACFQ4DvvfrNXTHidofr\\n\"\n\t\t\tif string(stdout) != expected {\n\t\t\t\tt.Fatalf(\"expected %q, got: %q\", expected, stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"bases\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"cid\", \"bases\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\tif !strings.Contains(string(stdout), \"base32\") {\n\t\t\t\tt.Fatalf(\"expected base32 in output, got: %s\", stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"codecs\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"cid\", \"codecs\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\tif !strings.Contains(string(stdout), \"dag-pb\") {\n\t\t\t\tt.Fatalf(\"expected dag-pb in output, got: %s\", stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"hashes\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"cid\", \"hashes\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\tif !strings.Contains(string(stdout), \"sha2-256\") {\n\t\t\t\tt.Fatalf(\"expected sha2-256 in output, got: %s\", stdout)\n\t\t\t}\n\t\t})\n\t})\n\n\tt.Run(\"multibase\", func(t *testing.T) {\n\t\tt.Run(\"list\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"multibase\", \"list\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\tif !strings.Contains(string(stdout), \"base32\") {\n\t\t\t\tt.Fatalf(\"expected base32 in output, got: %s\", stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"encode\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"multibase\", \"encode\", \"-b\", \"base32\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tcmd.Stdin = strings.NewReader(\"hello\\n\")\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\texpected := \"bnbswy3dpbi\"\n\t\t\tif string(stdout) != expected {\n\t\t\t\tt.Fatalf(\"expected %q, got: %q\", expected, stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"decode\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"multibase\", \"decode\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tcmd.Stdin = strings.NewReader(\"bnbswy3dpbi\")\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\texpected := \"hello\\n\"\n\t\t\tif string(stdout) != expected {\n\t\t\t\tt.Fatalf(\"expected %q, got: %q\", expected, stdout)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"transcode\", func(t *testing.T) {\n\t\t\tcmd := exec.Command(\"ipfs\", \"multibase\", \"transcode\", \"-b\", \"base64\")\n\t\t\tcmd.Env = append(os.Environ(), \"IPFS_PATH=\"+t.TempDir())\n\t\t\tcmd.Stdin = strings.NewReader(\"bnbswy3dpbi\")\n\t\t\tstdout, err := cmd.Output()\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\texpected := \"maGVsbG8K\"\n\t\t\tif string(stdout) != expected {\n\t\t\t\tt.Fatalf(\"expected %q, got: %q\", expected, stdout)\n\t\t\t}\n\t\t})\n\t})\n}\n" }, { "path": "test/cli/completion_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestBashCompletion(t *testing.T) {\n\tt.Parallel()\n\th := harness.NewT(t)\n\tnode := h.NewNode()\n\n\tres := node.IPFS(\"commands\", \"completion\", \"bash\")\n\n\tlength := len(res.Stdout.String())\n\tif length < 100 {\n\t\tt.Fatalf(\"expected a long Bash completion file, but got one of length %d\", length)\n\t}\n\n\tt.Run(\"completion file can be loaded in bash\", func(t *testing.T) {\n\t\tRequiresLinux(t)\n\n\t\tcompletionFile := h.WriteToTemp(res.Stdout.String())\n\t\tres = h.Sh(fmt.Sprintf(\"source %s && type -t _ipfs\", completionFile))\n\t\tassert.NoError(t, res.Err)\n\t})\n}\n\nfunc TestZshCompletion(t *testing.T) {\n\tt.Parallel()\n\th := harness.NewT(t)\n\tnode := h.NewNode()\n\n\tres := node.IPFS(\"commands\", \"completion\", \"zsh\")\n\n\tlength := len(res.Stdout.String())\n\tif length < 100 {\n\t\tt.Fatalf(\"expected a long Bash completion file, but got one of length %d\", length)\n\t}\n\n\tt.Run(\"completion file can be loaded in bash\", func(t *testing.T) {\n\t\tRequiresLinux(t)\n\n\t\tcompletionFile := h.WriteToTemp(res.Stdout.String())\n\t\tres = h.Runner.Run(harness.RunRequest{\n\t\t\tPath: \"zsh\",\n\t\t\tArgs: []string{\"-c\", fmt.Sprintf(\"autoload -Uz compinit && compinit && source %s && echo -E $_comps[ipfs]\", completionFile)},\n\t\t})\n\n\t\tassert.NoError(t, res.Err)\n\t\tassert.NotEmpty(t, res.Stdout.String())\n\t})\n}\n" }, { "path": "test/cli/config_secrets_test.go", "content": "package cli\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/tidwall/sjson\"\n)\n\nfunc TestConfigSecrets(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"Identity.PrivKey protection\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"Identity.PrivKey is concealed in config show\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Read the actual config file to get the real PrivKey\n\t\t\tconfigFile := node.ReadFile(node.ConfigFile())\n\t\t\tassert.Contains(t, configFile, \"PrivKey\")\n\n\t\t\t// config show should NOT contain the PrivKey\n\t\t\tconfigShow := node.RunIPFS(\"config\", \"show\").Stdout.String()\n\t\t\tassert.NotContains(t, configShow, \"PrivKey\")\n\t\t})\n\n\t\tt.Run(\"Identity.PrivKey cannot be read via ipfs config\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Attempting to read Identity.PrivKey should fail\n\t\t\tres := node.RunIPFS(\"config\", \"Identity.PrivKey\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"cannot show or change private key\")\n\t\t})\n\n\t\tt.Run(\"Identity.PrivKey cannot be read via ipfs config Identity\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Attempting to read Identity section should fail (it contains PrivKey)\n\t\t\tres := node.RunIPFS(\"config\", \"Identity\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"cannot show or change private key\")\n\t\t})\n\n\t\tt.Run(\"Identity.PrivKey cannot be set via config replace\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\t// Key rotation must be done in offline mode via the dedicated `ipfs key rotate` command.\n\t\t\t// This test ensures PrivKey cannot be changed via config replace.\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\tconfigShow := node.RunIPFS(\"config\", \"show\").Stdout.String()\n\n\t\t\t// Try to inject a PrivKey via config replace\n\t\t\tconfigJSON := MustVal(sjson.Set(configShow, \"Identity.PrivKey\", \"CAASqAkwggSkAgEAAo\"))\n\t\t\tnode.WriteBytes(\"new-config\", []byte(configJSON))\n\t\t\tres := node.RunIPFS(\"config\", \"replace\", \"new-config\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"setting private key\")\n\t\t})\n\n\t\tt.Run(\"Identity.PrivKey is preserved when re-injecting config\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Read the original config file\n\t\t\toriginalConfig := node.ReadFile(node.ConfigFile())\n\t\t\tassert.Contains(t, originalConfig, \"PrivKey\")\n\n\t\t\t// Extract the PrivKey value for comparison\n\t\t\tvar origPrivKey string\n\t\t\tassert.Contains(t, originalConfig, \"PrivKey\")\n\t\t\t// Simple extraction - find the PrivKey line\n\t\t\tfor line := range strings.SplitSeq(originalConfig, \"\\n\") {\n\t\t\t\tif strings.Contains(line, \"\\\"PrivKey\\\":\") {\n\t\t\t\t\torigPrivKey = line\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tassert.NotEmpty(t, origPrivKey)\n\n\t\t\t// Get config show output (which should NOT contain PrivKey)\n\t\t\tconfigShow := node.RunIPFS(\"config\", \"show\").Stdout.String()\n\t\t\tassert.NotContains(t, configShow, \"PrivKey\")\n\n\t\t\t// Re-inject the config via config replace\n\t\t\tnode.WriteBytes(\"config-show\", []byte(configShow))\n\t\t\tnode.IPFS(\"config\", \"replace\", \"config-show\")\n\n\t\t\t// The PrivKey should still be in the config file\n\t\t\tnewConfig := node.ReadFile(node.ConfigFile())\n\t\t\tassert.Contains(t, newConfig, \"PrivKey\")\n\n\t\t\t// Verify the PrivKey line is the same\n\t\t\tvar newPrivKey string\n\t\t\tfor line := range strings.SplitSeq(newConfig, \"\\n\") {\n\t\t\t\tif strings.Contains(line, \"\\\"PrivKey\\\":\") {\n\t\t\t\t\tnewPrivKey = line\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tassert.Equal(t, origPrivKey, newPrivKey, \"PrivKey should be preserved\")\n\t\t})\n\t})\n\n\tt.Run(\"TLS security validation\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"AutoConf.TLSInsecureSkipVerify defaults to false\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Check the default value in a fresh init\n\t\t\tres := node.RunIPFS(\"config\", \"AutoConf.TLSInsecureSkipVerify\")\n\t\t\t// Field may not exist (exit code 1) or be false/empty (exit code 0)\n\t\t\t// Both are acceptable as they mean \"not true\"\n\t\t\toutput := res.Stdout.String()\n\t\t\tassert.NotContains(t, output, \"true\", \"default should not be true\")\n\t\t})\n\n\t\tt.Run(\"AutoConf.TLSInsecureSkipVerify can be set to true\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Set to true\n\t\t\tnode.IPFS(\"config\", \"AutoConf.TLSInsecureSkipVerify\", \"true\", \"--json\")\n\n\t\t\t// Verify it was set\n\t\t\tres := node.RunIPFS(\"config\", \"AutoConf.TLSInsecureSkipVerify\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stdout.String(), \"true\")\n\t\t})\n\n\t\tt.Run(\"HTTPRetrieval.TLSInsecureSkipVerify defaults to false\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Check the default value in a fresh init\n\t\t\tres := node.RunIPFS(\"config\", \"HTTPRetrieval.TLSInsecureSkipVerify\")\n\t\t\t// Field may not exist (exit code 1) or be false/empty (exit code 0)\n\t\t\t// Both are acceptable as they mean \"not true\"\n\t\t\toutput := res.Stdout.String()\n\t\t\tassert.NotContains(t, output, \"true\", \"default should not be true\")\n\t\t})\n\n\t\tt.Run(\"HTTPRetrieval.TLSInsecureSkipVerify can be set to true\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// Set to true\n\t\t\tnode.IPFS(\"config\", \"HTTPRetrieval.TLSInsecureSkipVerify\", \"true\", \"--json\")\n\n\t\t\t// Verify it was set\n\t\t\tres := node.RunIPFS(\"config\", \"HTTPRetrieval.TLSInsecureSkipVerify\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stdout.String(), \"true\")\n\t\t})\n\t})\n}\n" }, { "path": "test/cli/content_blocking_test.go", "content": "package cli\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\tcarstore \"github.com/ipld/go-car/v2/blockstore\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tlibp2phttp \"github.com/libp2p/go-libp2p/p2p/http\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestContentBlocking(t *testing.T) {\n\t// NOTE: we can't run this with t.Parallel() because we set IPFS_NS_MAP\n\t// and running in parallel could impact other tests\n\n\tconst blockedMsg = \"blocked and cannot be provided\"\n\tconst statusExpl = \"specific HTTP error code is expected\"\n\tconst bodyExpl = \"Error message informing about content block is expected\"\n\n\th := harness.NewT(t)\n\n\t// Init IPFS_PATH\n\tnode := h.NewNode().Init(\"--empty-repo\", \"--profile=test\")\n\n\t// Create CIDs we use in test\n\th.WriteFile(\"parent-dir/blocked-subdir/indirectly-blocked-file.txt\", \"indirectly blocked file content\")\n\tallowedParentDirCID := node.IPFS(\"add\", \"--raw-leaves\", \"-Q\", \"-r\", \"--pin=false\", filepath.Join(h.Dir, \"parent-dir\")).Stdout.Trimmed()\n\tblockedSubDirCID := node.IPFS(\"add\", \"--raw-leaves\", \"-Q\", \"-r\", \"--pin=false\", filepath.Join(h.Dir, \"parent-dir\", \"blocked-subdir\")).Stdout.Trimmed()\n\tnode.IPFS(\"block\", \"rm\", blockedSubDirCID)\n\n\th.WriteFile(\"directly-blocked-file.txt\", \"directly blocked file content\")\n\tblockedCID := node.IPFS(\"add\", \"--raw-leaves\", \"-Q\", filepath.Join(h.Dir, \"directly-blocked-file.txt\")).Stdout.Trimmed()\n\n\th.WriteFile(\"not-blocked-file.txt\", \"not blocked file content\")\n\tallowedCID := node.IPFS(\"add\", \"--raw-leaves\", \"-Q\", filepath.Join(h.Dir, \"not-blocked-file.txt\")).Stdout.Trimmed()\n\n\t// Create denylist at $IPFS_PATH/denylists/test.deny\n\tdenylistTmp := h.WriteToTemp(\"name: test list\\n---\\n\" +\n\t\t\"//QmX9dhRcQcKUw3Ws8485T5a9dtjrSCQaUAHnG4iK9i4ceM\\n\" + // Double hash (sha256) CID block: base58btc(sha256-multihash(QmVTF1yEejXd9iMgoRTFDxBv7HAz9kuZcQNBzHrceuK9HR))\n\t\t\"//gW813G35CnLsy7gRYYHuf63hrz71U1xoLFDVeV7actx6oX\\n\" + // Double hash (blake3) Path block under blake3 root CID: base58btc(blake3-multihash(gW7Nhu4HrfDtphEivm3Z9NNE7gpdh5Tga8g6JNZc1S8E47/path))\n\t\t\"//8526ba05eec55e28f8db5974cc891d0d92c8af69d386fc6464f1e9f372caf549\\n\" + // Legacy CID double-hash block: sha256(bafkqahtcnrxwg23fmqqgi33vmjwgk2dbonuca3dfm5qwg6jamnuwicq/)\n\t\t\"//e5b7d2ce2594e2e09901596d8e1f29fa249b74c8c9e32ea01eda5111e4d33f07\\n\" + // Legacy Path double-hash block: sha256(bafyaagyscufaqalqaacauaqiaejao43vmjygc5didacauaqiae/subpath)\n\t\t\"/ipfs/\" + blockedCID + \"\\n\" + // block specific CID\n\t\t\"/ipfs/\" + allowedParentDirCID + \"/blocked-subdir*\\n\" + // block only specific subpath\n\t\t\"/ipns/blocked-cid.example.com\\n\" +\n\t\t\"/ipns/blocked-dnslink.example.com\\n\")\n\n\tif err := os.MkdirAll(filepath.Join(node.Dir, \"denylists\"), 0o777); err != nil {\n\t\tlog.Panicf(\"failed to create denylists dir: %s\", err.Error())\n\t}\n\tif err := os.Rename(denylistTmp, filepath.Join(node.Dir, \"denylists\", \"test.deny\")); err != nil {\n\t\tlog.Panicf(\"failed to create test denylist: %s\", err.Error())\n\t}\n\n\t// Add two entries to namesys resolution cache\n\t// /ipns/blocked-cid.example.com point at a blocked CID (to confirm blocking impacts /ipns resolution)\n\t// /ipns/blocked-dnslink.example.com with safe CID (to test blocking of /ipns/ paths)\n\tos.Setenv(\"IPFS_NS_MAP\", \"blocked-cid.example.com:/ipfs/\"+blockedCID+\",blocked-dnslink.example.com/ipns/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\")\n\tdefer os.Unsetenv(\"IPFS_NS_MAP\")\n\n\t// Enable GatewayOverLibp2p as we want to test denylist there too\n\tnode.IPFS(\"config\", \"--json\", \"Experimental.GatewayOverLibp2p\", \"true\")\n\n\t// Start daemon, it should pick up denylist from $IPFS_PATH/denylists/test.deny\n\tnode.StartDaemon() // we need online mode for GatewayOverLibp2p tests\n\tt.Cleanup(func() { node.StopDaemon() })\n\tclient := node.GatewayClient()\n\n\t// First, confirm gateway works\n\tt.Run(\"Gateway Allows CID that is not blocked\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/\" + allowedCID)\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tassert.Equal(t, \"not blocked file content\", resp.Body)\n\t})\n\n\t// Then, does the most basic blocking case work?\n\tt.Run(\"Gateway Denies directly blocked CID\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/\" + blockedCID)\n\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\tassert.NotEqual(t, \"directly blocked file content\", resp.Body)\n\t\tassert.Contains(t, resp.Body, blockedMsg, bodyExpl)\n\t})\n\n\t// Confirm parent of blocked subpath is not blocked\n\tt.Run(\"Gateway Allows parent Path that is not blocked\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/\" + allowedParentDirCID)\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t})\n\n\t// Confirm CAR responses skip blocked subpaths\n\tt.Run(\"Gateway returns CAR without blocked subpath\", func(t *testing.T) {\n\t\tresp := client.Get(\"/ipfs/\" + allowedParentDirCID + \"/subdir?format=car\")\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tbs, err := carstore.NewReadOnly(strings.NewReader(resp.Body), nil)\n\t\tassert.NoError(t, err)\n\n\t\thas, err := bs.Has(context.Background(), cid.MustParse(blockedSubDirCID))\n\t\tassert.NoError(t, err)\n\t\tassert.False(t, has)\n\t})\n\n\t/* TODO: this was already broken in 0.26, but we should fix it\n\tt.Run(\"Gateway returns CAR without directly blocked CID\", func(t *testing.T) {\n\t\tallowedDirWithDirectlyBlockedCID := node.IPFS(\"add\", \"--raw-leaves\", \"-Q\", \"-rw\", filepath.Join(h.Dir, \"directly-blocked-file.txt\")).Stdout.Trimmed()\n\t\tresp := client.Get(\"/ipfs/\" + allowedDirWithDirectlyBlockedCID + \"?format=car\")\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tbs, err := carstore.NewReadOnly(strings.NewReader(resp.Body), nil)\n\t\tassert.NoError(t, err)\n\n\t\thas, err := bs.Has(context.Background(), cid.MustParse(blockedCID))\n\t\tassert.NoError(t, err)\n\t\tassert.False(t, has, \"Returned CAR should not include blockedCID\")\n\t})\n\t*/\n\n\t// Confirm CAR responses skip blocked subpaths\n\tt.Run(\"Gateway returns CAR without blocked subpath\", func(t *testing.T) {\n\t\tresp := client.Get(\"/ipfs/\" + allowedParentDirCID + \"/subdir?format=car\")\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tbs, err := carstore.NewReadOnly(strings.NewReader(resp.Body), nil)\n\t\tassert.NoError(t, err)\n\n\t\thas, err := bs.Has(context.Background(), cid.MustParse(blockedSubDirCID))\n\t\tassert.NoError(t, err)\n\t\tassert.False(t, has, \"Returned CAR should not include blockedSubDirCID\")\n\t})\n\n\t// Ok, now the full list of test cases we want to cover in both CLI and Gateway\n\ttestCases := []struct {\n\t\tname string\n\t\tpath string\n\t}{\n\t\t{\n\t\t\tname: \"directly blocked file CID\",\n\t\t\tpath: \"/ipfs/\" + blockedCID,\n\t\t},\n\t\t{\n\t\t\tname: \"indirectly blocked file (on a blocked subpath)\",\n\t\t\tpath: \"/ipfs/\" + allowedParentDirCID + \"/blocked-subdir/indirectly-blocked-file.txt\",\n\t\t},\n\t\t{\n\t\t\tname: \"/ipns path that resolves to a blocked CID\",\n\t\t\tpath: \"/ipns/blocked-cid.example.com\",\n\t\t},\n\t\t{\n\t\t\tname: \"/ipns Path that is blocked by DNSLink name\",\n\t\t\tpath: \"/ipns/blocked-dnslink.example.com\",\n\t\t},\n\t\t{\n\t\t\tname: \"double-hash CID block (sha256-multihash)\",\n\t\t\tpath: \"/ipfs/QmVTF1yEejXd9iMgoRTFDxBv7HAz9kuZcQNBzHrceuK9HR\",\n\t\t},\n\t\t{\n\t\t\tname: \"double-hash Path block (blake3-multihash)\",\n\t\t\tpath: \"/ipfs/bafyb4ieqht3b2rssdmc7sjv2cy2gfdilxkfh7623nvndziyqnawkmo266a/path\",\n\t\t},\n\t\t{\n\t\t\tname: \"legacy CID double-hash block (sha256)\",\n\t\t\tpath: \"/ipfs/bafkqahtcnrxwg23fmqqgi33vmjwgk2dbonuca3dfm5qwg6jamnuwicq\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"legacy Path double-hash block (sha256)\",\n\t\t\tpath: \"/ipfs/bafyaagyscufaqalqaacauaqiaejao43vmjygc5didacauaqiae/subpath\",\n\t\t},\n\t}\n\n\t// Which specific cliCmds we test against testCases\n\tcliCmds := [][]string{\n\t\t{\"block\", \"get\"},\n\t\t{\"block\", \"stat\"},\n\t\t{\"dag\", \"get\"},\n\t\t{\"dag\", \"export\"},\n\t\t{\"dag\", \"stat\"},\n\t\t{\"cat\"},\n\t\t{\"ls\"},\n\t\t{\"get\"},\n\t\t{\"refs\"},\n\t}\n\n\texpectedMsg := blockedMsg\n\tfor _, testCase := range testCases {\n\n\t\t// Confirm that denylist is active for every command in 'cliCmds' x 'testCases'\n\t\tfor _, cmd := range cliCmds {\n\t\t\tcliTestName := fmt.Sprintf(\"CLI '%s' denies %s\", strings.Join(cmd, \" \"), testCase.name)\n\t\t\tt.Run(cliTestName, func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\targs := append(cmd, testCase.path)\n\t\t\t\tcmd := node.RunIPFS(args...)\n\t\t\t\tstdout := cmd.Stdout.Trimmed()\n\t\t\t\tstderr := cmd.Stderr.Trimmed()\n\t\t\t\tif !strings.Contains(stderr, expectedMsg) {\n\t\t\t\t\tt.Errorf(\"Expected STDERR error message %q, but got: %q\", expectedMsg, stderr)\n\t\t\t\t\tif stdout != \"\" {\n\t\t\t\t\t\tt.Errorf(\"Expected STDOUT to be empty, but got: %q\", stdout)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\n\t\t// Confirm that denylist is active for every content path in 'testCases'\n\t\tgwTestName := fmt.Sprintf(\"Gateway denies %s\", testCase.name)\n\t\tt.Run(gwTestName, func(t *testing.T) {\n\t\t\tresp := client.Get(testCase.path)\n\t\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\t\tassert.Contains(t, resp.Body, blockedMsg, bodyExpl)\n\t\t})\n\n\t}\n\n\t// Extra edge cases on subdomain gateway\n\n\tt.Run(\"Gateway Denies /ipns Path that is blocked by DNSLink name (subdomain redirect)\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tgwURL, _ := url.Parse(node.GatewayURL())\n\t\tresp := client.Get(\"/ipns/blocked-dnslink.example.com\", func(r *http.Request) {\n\t\t\tr.Host = \"localhost:\" + gwURL.Port()\n\t\t})\n\n\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\tassert.Contains(t, resp.Body, blockedMsg, bodyExpl)\n\t})\n\n\tt.Run(\"Gateway Denies /ipns Path that is blocked by DNSLink name (subdomain, no TLS)\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tgwURL, _ := url.Parse(node.GatewayURL())\n\t\tresp := client.Get(\"/\", func(r *http.Request) {\n\t\t\tr.Host = \"blocked-dnslink.example.com.ipns.localhost:\" + gwURL.Port()\n\t\t})\n\n\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\tassert.Contains(t, resp.Body, blockedMsg, bodyExpl)\n\t})\n\n\tt.Run(\"Gateway Denies /ipns Path that is blocked by DNSLink name (subdomain, inlined for TLS)\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tgwURL, _ := url.Parse(node.GatewayURL())\n\t\tresp := client.Get(\"/\", func(r *http.Request) {\n\t\t\t// Inlined DNSLink to fit in single DNS label for TLS interop:\n\t\t\t// https://specs.ipfs.tech/http-gateways/subdomain-gateway/#host-request-header\n\t\t\tr.Host = \"blocked--dnslink-example-com.ipns.localhost:\" + gwURL.Port()\n\t\t})\n\n\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\tassert.Contains(t, resp.Body, blockedMsg, bodyExpl)\n\t})\n\n\t// We need to confirm denylist is active when gateway is run in NoFetch\n\t// mode (which usually swaps blockservice to a read-only one, and that swap\n\t// may cause denylists to not be applied, as it is a separate code path)\n\tt.Run(\"GatewayNoFetch\", func(t *testing.T) {\n\t\t// NOTE: we don't run this in parallel, as it requires restart with different config\n\n\t\t// Switch gateway to NoFetch mode\n\t\tnode.StopDaemon()\n\t\tnode.IPFS(\"config\", \"--json\", \"Gateway.NoFetch\", \"true\")\n\t\tnode.StartDaemon()\n\n\t\t// update client, as the port of test node might've changed after restart\n\t\tclient = node.GatewayClient()\n\n\t\t// First, confirm gateway works\n\t\tt.Run(\"Allows CID that is not blocked\", func(t *testing.T) {\n\t\t\tresp := client.Get(\"/ipfs/\" + allowedCID)\n\t\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t\t\tassert.Equal(t, \"not blocked file content\", resp.Body)\n\t\t})\n\n\t\t// Then, does the most basic blocking case work?\n\t\tt.Run(\"Denies directly blocked CID\", func(t *testing.T) {\n\t\t\tresp := client.Get(\"/ipfs/\" + blockedCID)\n\t\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\t\tassert.NotEqual(t, \"directly blocked file content\", resp.Body)\n\t\t\tassert.Contains(t, resp.Body, blockedMsg, bodyExpl)\n\t\t})\n\n\t\t// Restore default\n\t\tnode.StopDaemon()\n\t\tnode.IPFS(\"config\", \"--json\", \"Gateway.NoFetch\", \"false\")\n\t\tnode.StartDaemon()\n\t\tclient = node.GatewayClient()\n\t})\n\n\t// We need to confirm denylist is active on the\n\t// trustless gateway exposed over libp2p\n\t// when Experimental.GatewayOverLibp2p=true\n\t// (https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#http-gateway-over-libp2p)\n\t// NOTE: this type of gateway is hardcoded to be NoFetch: it does not fetch\n\t// data that is not in local store, so we only need to run it once: a\n\t// simple smoke-test for allowed CID and blockedCID.\n\tt.Run(\"GatewayOverLibp2p\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create libp2p client that connects to our node over\n\t\t// /http1.1 and then talks gateway semantics over the /ipfs/gateway sub-protocol\n\t\tclientHost, err := libp2p.New(libp2p.NoListenAddrs)\n\t\trequire.NoError(t, err)\n\t\terr = clientHost.Connect(context.Background(), peer.AddrInfo{\n\t\t\tID: node.PeerID(),\n\t\t\tAddrs: node.SwarmAddrs(),\n\t\t})\n\t\trequire.NoError(t, err)\n\n\t\tlibp2pClient, err := (&libp2phttp.Host{StreamHost: clientHost}).NamespacedClient(\"/ipfs/gateway\", peer.AddrInfo{ID: node.PeerID()})\n\t\trequire.NoError(t, err)\n\n\t\tt.Run(\"Serves Allowed CID\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp, err := libp2pClient.Get(fmt.Sprintf(\"/ipfs/%s?format=raw\", allowedCID))\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\t\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t\t\tbody, err := io.ReadAll(resp.Body)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, string(body), \"not blocked file content\", bodyExpl)\n\t\t})\n\n\t\tt.Run(\"Denies Blocked CID\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp, err := libp2pClient.Get(fmt.Sprintf(\"/ipfs/%s?format=raw\", blockedCID))\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\t\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\t\tbody, err := io.ReadAll(resp.Body)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.NotEqual(t, string(body), \"directly blocked file content\")\n\t\t\tassert.Contains(t, string(body), blockedMsg, bodyExpl)\n\t\t})\n\n\t\tt.Run(\"Denies Blocked CID as CAR\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp, err := libp2pClient.Get(fmt.Sprintf(\"/ipfs/%s?format=car\", blockedCID))\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\t\t\tassert.Equal(t, http.StatusGone, resp.StatusCode, statusExpl)\n\t\t\tbody, err := io.ReadAll(resp.Body)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.NotContains(t, string(body), \"directly blocked file content\")\n\t\t\tassert.Contains(t, string(body), blockedMsg, bodyExpl)\n\t\t})\n\t})\n}\n" }, { "path": "test/cli/content_routing_http_test.go", "content": "package cli\n\nimport (\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os/exec\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/routing/http/server\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils/httprouting\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\n// userAgentRecorder records the user agent of every HTTP request\ntype userAgentRecorder struct {\n\tdelegate http.Handler\n\tuserAgents []string\n}\n\nfunc (r *userAgentRecorder) ServeHTTP(w http.ResponseWriter, req *http.Request) {\n\tr.userAgents = append(r.userAgents, req.UserAgent())\n\tr.delegate.ServeHTTP(w, req)\n}\n\nfunc TestContentRoutingHTTP(t *testing.T) {\n\tmockRouter := &httprouting.MockHTTPContentRouter{}\n\n\t// run the content routing HTTP server\n\tuserAgentRecorder := &userAgentRecorder{delegate: server.Handler(mockRouter)}\n\tserver := httptest.NewServer(userAgentRecorder)\n\tt.Cleanup(func() { server.Close() })\n\n\t// setup the node\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t// setup Kubo node to use mocked HTTP Router\n\t\tcfg.Routing.DelegatedRouters = []string{server.URL}\n\t})\n\tnode.StartDaemon()\n\n\t// compute a random CID\n\trandStr := string(random.Bytes(100))\n\tres := node.PipeStrToIPFS(randStr, \"add\", \"-qn\")\n\twantCIDStr := res.Stdout.Trimmed()\n\n\tt.Run(\"fetching an uncached block results in an HTTP lookup\", func(t *testing.T) {\n\t\tstatRes := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"block\", \"stat\", wantCIDStr},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\tdefer func() {\n\t\t\tif err := statRes.Cmd.Process.Kill(); err != nil {\n\t\t\t\tt.Logf(\"error killing 'block stat' cmd: %s\", err)\n\t\t\t}\n\t\t}()\n\n\t\t// verify the content router was called\n\t\tassert.Eventually(t, func() bool {\n\t\t\treturn mockRouter.NumFindProvidersCalls() > 0\n\t\t}, time.Minute, 10*time.Millisecond)\n\n\t\tassert.NotEmpty(t, userAgentRecorder.userAgents)\n\t\tversion := node.IPFS(\"id\", \"-f\", \"\").Stdout.Trimmed()\n\t\tfor _, userAgent := range userAgentRecorder.userAgents {\n\t\t\tassert.Equal(t, version, userAgent)\n\t\t}\n\t})\n}\n" }, { "path": "test/cli/daemon_test.go", "content": "package cli\n\nimport (\n\t\"bytes\"\n\t\"crypto/rand\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"os/exec\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestDaemon(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"daemon starts if api is set to null\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Addresses.API\", nil)\n\t\tnode.Runner.MustRun(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"daemon\"},\n\t\t\tRunFunc: (*exec.Cmd).Start, // Start without waiting for completion.\n\t\t})\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"daemon shuts down gracefully with active operations\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Start daemon with multiple components active via config\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Enable experimental features and pubsub via config\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Pubsub.Enabled = config.True // Instead of --enable-pubsub-experiment\n\t\t\tcfg.Experimental.P2pHttpProxy = true // Enable P2P HTTP proxy\n\t\t\tcfg.Experimental.GatewayOverLibp2p = true // Enable gateway over libp2p\n\t\t})\n\n\t\tnode.StartDaemon(\"--enable-gc\")\n\n\t\t// Start background operations to simulate real daemon workload:\n\t\t// 1. \"ipfs add\" simulates content onboarding/ingestion work\n\t\t// 2. Gateway request simulates content retrieval and gateway processing work\n\n\t\t// Background operation 1: Continuous add of random data to simulate onboarding\n\t\taddDone := make(chan struct{})\n\t\tgo func() {\n\t\t\tdefer close(addDone)\n\n\t\t\t// Start the add command asynchronously\n\t\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\t\tPath: node.IPFSBin,\n\t\t\t\tArgs: []string{\"add\", \"--progress=false\", \"-\"},\n\t\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\t\tharness.RunWithStdin(&infiniteReader{}),\n\t\t\t\t},\n\t\t\t})\n\n\t\t\t// Wait for command to finish (when daemon stops)\n\t\t\tif res.Cmd != nil {\n\t\t\t\t_ = res.Cmd.Wait() // Ignore error, expect command to be killed during shutdown\n\t\t\t}\n\t\t}()\n\n\t\t// Background operation 2: Gateway CAR request to simulate retrieval work\n\t\tgatewayDone := make(chan struct{})\n\t\tgo func() {\n\t\t\tdefer close(gatewayDone)\n\n\t\t\t// First add a file sized to ensure gateway request takes ~1 minute\n\t\t\tlargeData := make([]byte, 512*1024) // 512KB of data\n\t\t\t_, _ = rand.Read(largeData) // Always succeeds for crypto/rand\n\t\t\ttestCID := node.IPFSAdd(bytes.NewReader(largeData))\n\n\t\t\t// Get gateway address from config\n\t\t\tcfg := node.ReadConfig()\n\t\t\tgatewayMaddr, err := multiaddr.NewMultiaddr(cfg.Addresses.Gateway[0])\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tgatewayAddr, err := manet.ToNetAddr(gatewayMaddr)\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t// Request CAR but slow reading to simulate heavy gateway load\n\t\t\tgatewayURL := fmt.Sprintf(\"http://%s/ipfs/%s?format=car\", gatewayAddr, testCID)\n\n\t\t\tclient := &http.Client{Timeout: 90 * time.Second}\n\t\t\tresp, err := client.Get(gatewayURL)\n\t\t\tif err == nil {\n\t\t\t\tdefer resp.Body.Close()\n\t\t\t\t// Read response slowly: 512KB ÷ 1KB × 125ms = ~64 seconds (1+ minute) total\n\t\t\t\t// This ensures operation is still active when we shutdown at 2 seconds\n\t\t\t\tbuf := make([]byte, 1024) // 1KB buffer\n\t\t\t\tfor {\n\t\t\t\t\tif _, err := io.ReadFull(resp.Body, buf); err != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\ttime.Sleep(125 * time.Millisecond) // 125ms delay = ~64s total for 512KB\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\t// Let operations run for 2 seconds to ensure they're active\n\t\ttime.Sleep(2 * time.Second)\n\n\t\t// Trigger graceful shutdown\n\t\tshutdownStart := time.Now()\n\t\tnode.StopDaemon()\n\t\tshutdownDuration := time.Since(shutdownStart)\n\n\t\t// Verify clean shutdown:\n\t\t// - Daemon should stop within reasonable time (not hang)\n\t\trequire.Less(t, shutdownDuration, 10*time.Second, \"daemon should shut down within 10 seconds\")\n\n\t\t// Wait for background operations to complete (with timeout)\n\t\tselect {\n\t\tcase <-addDone:\n\t\t\t// Good, add operation terminated\n\t\tcase <-time.After(5 * time.Second):\n\t\t\tt.Error(\"add operation did not terminate within 5 seconds after daemon shutdown\")\n\t\t}\n\n\t\tselect {\n\t\tcase <-gatewayDone:\n\t\t\t// Good, gateway operation terminated\n\t\tcase <-time.After(5 * time.Second):\n\t\t\tt.Error(\"gateway operation did not terminate within 5 seconds after daemon shutdown\")\n\t\t}\n\n\t\t// Verify we can restart with same repo (no lock issues)\n\t\tnode.StartDaemon()\n\t\tnode.StopDaemon()\n\t})\n}\n\n// infiniteReader provides an infinite stream of random data\ntype infiniteReader struct{}\n\nfunc (r *infiniteReader) Read(p []byte) (n int, err error) {\n\t_, _ = rand.Read(p) // Always succeeds for crypto/rand\n\ttime.Sleep(50 * time.Millisecond) // Rate limit to simulate steady stream\n\treturn len(p), nil\n}\n" }, { "path": "test/cli/dag_layout_test.go", "content": "package cli\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestBalancedDAGLayout verifies that kubo uses the \"balanced\" DAG layout\n// (all leaves at same depth) rather than \"balanced-packed\" (varying leaf depths).\n//\n// DAG layout differences across implementations:\n//\n// - balanced: kubo, helia (all leaves at same depth, uniform traversal distance)\n// - balanced-packed: singularity (trailing leaves may be at different depths)\n// - trickle: kubo --trickle (varying depths, optimized for append-only/streaming)\n//\n// kubo does not implement balanced-packed. The trickle layout also produces\n// non-uniform leaf depths but with different trade-offs: trickle is optimized\n// for append-only and streaming reads (no seeking), while balanced-packed\n// minimizes node count.\n//\n// IPIP-499 documents the balanced vs balanced-packed distinction. Files larger\n// than dag_width × chunk_size will have different CIDs between implementations\n// using different layouts.\n//\n// Set DAG_LAYOUT_CAR_OUTPUT environment variable to export CAR files.\n// Example: DAG_LAYOUT_CAR_OUTPUT=/tmp/dag-layout go test -run TestBalancedDAGLayout -v\nfunc TestBalancedDAGLayout(t *testing.T) {\n\tt.Parallel()\n\n\tcarOutputDir := os.Getenv(\"DAG_LAYOUT_CAR_OUTPUT\")\n\texportCARs := carOutputDir != \"\"\n\tif exportCARs {\n\t\tif err := os.MkdirAll(carOutputDir, 0755); err != nil {\n\t\t\tt.Fatalf(\"failed to create CAR output directory: %v\", err)\n\t\t}\n\t\tt.Logf(\"CAR export enabled, writing to: %s\", carOutputDir)\n\t}\n\n\tt.Run(\"balanced layout has uniform leaf depth\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\n\t\t// Create file that triggers multi-level DAG.\n\t\t// For default v0: 175 chunks × 256KiB = ~44.8 MiB (just over 174 max links)\n\t\t// This creates a 2-level DAG where balanced layout ensures uniform depth.\n\t\tfileSize := \"45MiB\"\n\t\tseed := \"balanced-test\"\n\n\t\tcidStr := node.IPFSAddDeterministic(fileSize, seed)\n\n\t\t// Collect leaf depths by walking DAG\n\t\tdepths := collectLeafDepths(t, node, cidStr, 0)\n\n\t\t// All leaves must be at same depth for balanced layout\n\t\trequire.NotEmpty(t, depths, \"expected at least one leaf node\")\n\t\tfirstDepth := depths[0]\n\t\tfor i, d := range depths {\n\t\t\trequire.Equal(t, firstDepth, d,\n\t\t\t\t\"leaf %d at depth %d, expected %d (balanced layout requires uniform leaf depth)\",\n\t\t\t\ti, d, firstDepth)\n\t\t}\n\t\tt.Logf(\"verified %d leaves all at depth %d (CID: %s)\", len(depths), firstDepth, cidStr)\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, \"balanced_\"+fileSize+\".car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n\n\tt.Run(\"trickle layout has varying leaf depth\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\n\t\tfileSize := \"45MiB\"\n\t\tseed := \"trickle-test\"\n\n\t\t// Add with trickle layout (--trickle flag).\n\t\t// Trickle produces non-uniform leaf depths, optimized for append-only\n\t\t// and streaming reads (no seeking). This subtest validates the test\n\t\t// logic by confirming we can detect varying depths.\n\t\tcidStr := node.IPFSAddDeterministic(fileSize, seed, \"--trickle\")\n\n\t\tdepths := collectLeafDepths(t, node, cidStr, 0)\n\n\t\t// Trickle layout should have varying depths\n\t\trequire.NotEmpty(t, depths, \"expected at least one leaf node\")\n\t\tminDepth, maxDepth := depths[0], depths[0]\n\t\tfor _, d := range depths {\n\t\t\tif d < minDepth {\n\t\t\t\tminDepth = d\n\t\t\t}\n\t\t\tif d > maxDepth {\n\t\t\t\tmaxDepth = d\n\t\t\t}\n\t\t}\n\t\trequire.NotEqual(t, minDepth, maxDepth,\n\t\t\t\"trickle layout should have varying leaf depths, got uniform depth %d\", minDepth)\n\t\tt.Logf(\"verified %d leaves with depths ranging from %d to %d (CID: %s)\", len(depths), minDepth, maxDepth, cidStr)\n\n\t\tif exportCARs {\n\t\t\tcarPath := filepath.Join(carOutputDir, \"trickle_\"+fileSize+\".car\")\n\t\t\trequire.NoError(t, node.IPFSDagExport(cidStr, carPath))\n\t\t\tt.Logf(\"exported: %s -> %s\", cidStr, carPath)\n\t\t}\n\t})\n}\n\n// collectLeafDepths recursively walks DAG and returns depth of each leaf node.\n// A node is a leaf if it's a raw block or a dag-pb node with no links.\nfunc collectLeafDepths(t *testing.T, node *harness.Node, cid string, depth int) []int {\n\tt.Helper()\n\n\t// Check codec to see if this is a raw leaf\n\tres := node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", cid)\n\tcodec := strings.TrimSpace(res.Stdout.String())\n\tif codec == \"raw\" {\n\t\t// Raw blocks are always leaves\n\t\treturn []int{depth}\n\t}\n\n\t// Try to inspect as dag-pb node\n\tpbNode, err := node.InspectPBNode(cid)\n\tif err != nil {\n\t\t// Can't parse as dag-pb, treat as leaf\n\t\treturn []int{depth}\n\t}\n\n\t// No links = leaf node\n\tif len(pbNode.Links) == 0 {\n\t\treturn []int{depth}\n\t}\n\n\t// Recurse into children\n\tvar depths []int\n\tfor _, link := range pbNode.Links {\n\t\tchildDepths := collectLeafDepths(t, node, link.Hash.Slash, depth+1)\n\t\tdepths = append(depths, childDepths...)\n\t}\n\treturn depths\n}\n" }, { "path": "test/cli/dag_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"io\"\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\tfixtureFile = \"./fixtures/TestDagStat.car\"\n\ttextOutputPath = \"./fixtures/TestDagStatExpectedOutput.txt\"\n\tnode1Cid = \"bafyreibmdfd7c5db4kls4ty57zljfhqv36gi43l6txl44pi423wwmeskwy\"\n\tnode2Cid = \"bafyreie3njilzdi4ixumru4nzgecsnjtu7fzfcwhg7e6s4s5i7cnbslvn4\"\n\tfixtureCid = \"bafyreifrm6uf5o4dsaacuszf35zhibyojlqclabzrms7iak67pf62jygaq\"\n)\n\ntype DagStat struct {\n\tCid string `json:\"Cid\"`\n\tSize int `json:\"Size\"`\n\tNumBlocks int `json:\"NumBlocks\"`\n}\n\ntype Data struct {\n\tUniqueBlocks int `json:\"UniqueBlocks\"`\n\tTotalSize int `json:\"TotalSize\"`\n\tSharedSize int `json:\"SharedSize\"`\n\tRatio float64 `json:\"Ratio\"`\n\tDagStats []DagStat `json:\"DagStats\"`\n}\n\n// The Fixture file represents a dag where 2 nodes of size = 46B each, have a common child of 7B\n// when traversing the DAG from the root's children (node1 and node2) we count (46 + 7)x2 bytes (counting redundant bytes) = 106\n// since both nodes share a common child of 7 bytes we actually had to read (46)x2 + 7 = 99 bytes\n// we should get a dedup ratio of 106/99 that results in approximately 1.0707071\n\nfunc TestDag(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"ipfs dag stat --enc=json\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Import fixture\n\t\tr, err := os.Open(fixtureFile)\n\t\tassert.Nil(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\tassert.NoError(t, err)\n\t\tstat := node.RunIPFS(\"dag\", \"stat\", \"--progress=false\", \"--enc=json\", node1Cid, node2Cid)\n\t\tvar data Data\n\t\terr = json.Unmarshal(stat.Stdout.Bytes(), &data)\n\t\tassert.NoError(t, err)\n\n\t\texpectedUniqueBlocks := 3\n\t\texpectedSharedSize := 7\n\t\texpectedTotalSize := 99\n\t\texpectedRatio := float64(expectedSharedSize+expectedTotalSize) / float64(expectedTotalSize)\n\t\texpectedDagStatsLength := 2\n\t\t// Validate UniqueBlocks\n\t\tassert.Equal(t, expectedUniqueBlocks, data.UniqueBlocks)\n\t\tassert.Equal(t, expectedSharedSize, data.SharedSize)\n\t\tassert.Equal(t, expectedTotalSize, data.TotalSize)\n\t\tassert.Equal(t, testutils.FloatTruncate(expectedRatio, 4), testutils.FloatTruncate(data.Ratio, 4))\n\n\t\t// Validate DagStats\n\t\tassert.Equal(t, expectedDagStatsLength, len(data.DagStats))\n\t\tnode1Output := data.DagStats[0]\n\t\tnode2Output := data.DagStats[1]\n\n\t\tassert.Equal(t, node1Output.Cid, node1Cid)\n\t\tassert.Equal(t, node2Output.Cid, node2Cid)\n\n\t\texpectedNode1Size := (expectedTotalSize + expectedSharedSize) / 2\n\t\texpectedNode2Size := (expectedTotalSize + expectedSharedSize) / 2\n\t\tassert.Equal(t, expectedNode1Size, node1Output.Size)\n\t\tassert.Equal(t, expectedNode2Size, node2Output.Size)\n\n\t\texpectedNode1Blocks := 2\n\t\texpectedNode2Blocks := 2\n\t\tassert.Equal(t, expectedNode1Blocks, node1Output.NumBlocks)\n\t\tassert.Equal(t, expectedNode2Blocks, node2Output.NumBlocks)\n\t})\n\n\tt.Run(\"ipfs dag stat\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tr, err := os.Open(fixtureFile)\n\t\tassert.NoError(t, err)\n\t\tdefer r.Close()\n\t\tf, err := os.Open(textOutputPath)\n\t\tassert.NoError(t, err)\n\t\tdefer f.Close()\n\t\tcontent, err := io.ReadAll(f)\n\t\tassert.NoError(t, err)\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\tassert.NoError(t, err)\n\t\tstat := node.RunIPFS(\"dag\", \"stat\", \"--progress=false\", node1Cid, node2Cid)\n\t\tassert.Equal(t, content, stat.Stdout.Bytes())\n\t})\n}\n\nfunc TestDagImportFastProvide(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"fast-provide-root disabled via config: verify skipped in logs\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.False\n\t\t})\n\n\t\t// Start daemon with debug logging\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Import CAR file\n\t\tr, err := os.Open(fixtureFile)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify fast-provide-root was disabled\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: skipped\")\n\t})\n\n\tt.Run(\"fast-provide-root enabled with wait=false: verify async provide\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t// Use default config (FastProvideRoot=true, FastProvideWait=false)\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Import CAR file\n\t\tr, err := os.Open(fixtureFile)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\trequire.NoError(t, err)\n\n\t\tdaemonLog := node.Daemon.Stderr\n\t\t// Should see async mode started\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: enabled\")\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: providing asynchronously\")\n\t\trequire.Contains(t, daemonLog.String(), fixtureCid) // Should log the specific CID being provided\n\n\t\t// Wait for async completion or failure (slightly more than DefaultFastProvideTimeout)\n\t\t// In test environment with no DHT peers, this will fail with \"failed to find any peer in table\"\n\t\ttimeout := config.DefaultFastProvideTimeout + time.Second\n\t\tcompletedOrFailed := waitForLogMessage(daemonLog, \"async provide completed\", timeout) ||\n\t\t\twaitForLogMessage(daemonLog, \"async provide failed\", timeout)\n\t\trequire.True(t, completedOrFailed, \"async provide should complete or fail within timeout\")\n\t})\n\n\tt.Run(\"fast-provide-root enabled with wait=true: verify sync provide\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideWait = config.True\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Import CAR file - use Run instead of IPFSDagImport to handle expected error\n\t\tr, err := os.Open(fixtureFile)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"dag\", \"import\", \"--pin-roots=false\"},\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithStdin(r),\n\t\t\t},\n\t\t})\n\t\t// In sync mode (wait=true), provide errors propagate and fail the command.\n\t\t// Test environment uses 'test' profile with no bootstrappers, and CI has\n\t\t// insufficient peers for proper DHT puts, so we expect this to fail with\n\t\t// \"failed to find any peer in table\" error from the DHT.\n\t\trequire.Equal(t, 1, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"Error: fast-provide: failed to find any peer in table\")\n\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\t// Should see sync mode started\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: enabled\")\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: providing synchronously\")\n\t\trequire.Contains(t, daemonLog, fixtureCid) // Should log the specific CID being provided\n\t\trequire.Contains(t, daemonLog, \"sync provide failed\") // Verify the failure was logged\n\t})\n\n\tt.Run(\"fast-provide-wait ignored when root disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.False\n\t\t\tcfg.Import.FastProvideWait = config.True\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Import CAR file\n\t\tr, err := os.Open(fixtureFile)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\trequire.NoError(t, err)\n\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: skipped\")\n\t\t// Note: dag import doesn't log wait-flag-ignored like add does\n\t})\n\n\tt.Run(\"CLI flag overrides config: flag=true overrides config=false\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.False\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Import CAR file with flag override\n\t\tr, err := os.Open(fixtureFile)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid, \"--fast-provide-root=true\")\n\t\trequire.NoError(t, err)\n\n\t\tdaemonLog := node.Daemon.Stderr\n\t\t// Flag should enable it despite config saying false\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: enabled\")\n\t\trequire.Contains(t, daemonLog.String(), \"fast-provide-root: providing asynchronously\")\n\t\trequire.Contains(t, daemonLog.String(), fixtureCid) // Should log the specific CID being provided\n\t})\n\n\tt.Run(\"CLI flag overrides config: flag=false overrides config=true\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.FastProvideRoot = config.True\n\t\t})\n\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithEnv(map[string]string{\n\t\t\t\t\t\"GOLOG_LOG_LEVEL\": \"error,core/commands=debug,core/commands/cmdenv=debug\",\n\t\t\t\t}),\n\t\t\t},\n\t\t}, \"\")\n\t\tdefer node.StopDaemon()\n\n\t\t// Import CAR file with flag override\n\t\tr, err := os.Open(fixtureFile)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid, \"--fast-provide-root=false\")\n\t\trequire.NoError(t, err)\n\n\t\tdaemonLog := node.Daemon.Stderr.String()\n\t\t// Flag should disable it despite config saying true\n\t\trequire.Contains(t, daemonLog, \"fast-provide-root: skipped\")\n\t})\n}\n" }, { "path": "test/cli/delegated_routing_v1_http_client_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestHTTPDelegatedRouting(t *testing.T) {\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\n\tfakeServer := func(contentType string, resp ...string) *httptest.Server {\n\t\treturn httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tw.Header().Set(\"Content-Type\", contentType)\n\t\t\tfor _, r := range resp {\n\t\t\t\t_, err := w.Write([]byte(r))\n\t\t\t\tif err != nil {\n\t\t\t\t\tpanic(err)\n\t\t\t\t}\n\t\t\t}\n\t\t}))\n\t}\n\n\tfindProvsCID := \"baeabep4vu3ceru7nerjjbk37sxb7wmftteve4hcosmyolsbsiubw2vr6pqzj6mw7kv6tbn6nqkkldnklbjgm5tzbi4hkpkled4xlcr7xz4bq\"\n\tprovs := []string{\"12D3KooWAobjw92XDcnQ1rRmRJDA3zAQpdPYUpZKrJxH6yccSpje\", \"12D3KooWARYacCc6eoCqvsS9RW9MA2vo51CV75deoiqssx3YgyYJ\"}\n\n\tt.Run(\"default routing config has no routers defined\", func(t *testing.T) {\n\t\tassert.Nil(t, node.ReadConfig().Routing.Routers)\n\t})\n\n\tt.Run(\"no routers means findprovs returns no results\", func(t *testing.T) {\n\t\tres := node.IPFS(\"routing\", \"findprovs\", findProvsCID).Stdout.String()\n\t\tassert.Empty(t, res)\n\t})\n\n\tt.Run(\"no routers means findprovs returns no results\", func(t *testing.T) {\n\t\tres := node.IPFS(\"routing\", \"findprovs\", findProvsCID).Stdout.String()\n\t\tassert.Empty(t, res)\n\t})\n\n\tnode.StopDaemon()\n\n\tt.Run(\"missing method params make the daemon fail\", func(t *testing.T) {\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"custom\")\n\t\t\tcfg.Routing.Methods = config.Methods{\n\t\t\t\t\"find-peers\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"find-providers\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"get-ipns\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"provide\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t}\n\t\t})\n\t\tres := node.RunIPFS(\"daemon\")\n\t\tassert.Equal(t, 1, res.ExitErr.ProcessState.ExitCode())\n\t\tassert.Contains(\n\t\t\tt,\n\t\t\tres.Stderr.String(),\n\t\t\t`method name \"put-ipns\" is missing from Routing.Methods config param`,\n\t\t)\n\t})\n\n\tt.Run(\"having wrong methods makes daemon fail\", func(t *testing.T) {\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"custom\")\n\t\t\tcfg.Routing.Methods = config.Methods{\n\t\t\t\t\"find-peers\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"find-providers\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"get-ipns\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"provide\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"put-ipns\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t\t\"NOT_SUPPORTED\": {RouterName: \"TestDelegatedRouter\"},\n\t\t\t}\n\t\t})\n\t\tres := node.RunIPFS(\"daemon\")\n\t\tassert.Equal(t, 1, res.ExitErr.ProcessState.ExitCode())\n\t\tassert.Contains(\n\t\t\tt,\n\t\t\tres.Stderr.String(),\n\t\t\t`method name \"NOT_SUPPORTED\" is not a supported method on Routing.Methods config param`,\n\t\t)\n\t})\n\n\tt.Run(\"adding HTTP delegated routing endpoint to Routing.Routers config works\", func(t *testing.T) {\n\t\tserver := fakeServer(\"application/json\", ToJSONStr(JSONObj{\n\t\t\t\"Providers\": []JSONObj{\n\t\t\t\t{\n\t\t\t\t\t\"Schema\": \"bitswap\", // Legacy bitswap schema.\n\t\t\t\t\t\"Protocol\": \"transport-bitswap\",\n\t\t\t\t\t\"ID\": provs[1],\n\t\t\t\t\t\"Addrs\": []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/tcp/4002\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"Schema\": \"peer\",\n\t\t\t\t\t\"Protocols\": []string{\"transport-bitswap\"},\n\t\t\t\t\t\"ID\": provs[0],\n\t\t\t\t\t\"Addrs\": []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/tcp/4002\"},\n\t\t\t\t},\n\t\t\t},\n\t\t}))\n\t\tt.Cleanup(server.Close)\n\n\t\tnode.IPFS(\"config\", \"Routing.Type\", \"custom\")\n\t\tnode.IPFS(\"config\", \"Routing.Routers.TestDelegatedRouter\", \"--json\", ToJSONStr(JSONObj{\n\t\t\t\"Type\": \"http\",\n\t\t\t\"Parameters\": JSONObj{\n\t\t\t\t\"Endpoint\": server.URL,\n\t\t\t},\n\t\t}))\n\t\tnode.IPFS(\"config\", \"Routing.Methods\", \"--json\", ToJSONStr(JSONObj{\n\t\t\t\"find-peers\": JSONObj{\"RouterName\": \"TestDelegatedRouter\"},\n\t\t\t\"find-providers\": JSONObj{\"RouterName\": \"TestDelegatedRouter\"},\n\t\t\t\"get-ipns\": JSONObj{\"RouterName\": \"TestDelegatedRouter\"},\n\t\t\t\"provide\": JSONObj{\"RouterName\": \"TestDelegatedRouter\"},\n\t\t\t\"put-ipns\": JSONObj{\"RouterName\": \"TestDelegatedRouter\"},\n\t\t}))\n\n\t\tres := node.IPFS(\"config\", \"Routing.Routers.TestDelegatedRouter.Parameters.Endpoint\")\n\t\tassert.Equal(t, res.Stdout.Trimmed(), server.URL)\n\n\t\tnode.StartDaemon()\n\t\tres = node.IPFS(\"routing\", \"findprovs\", findProvsCID)\n\t\tassert.Equal(t, provs[1]+\"\\n\"+provs[0], res.Stdout.Trimmed())\n\t})\n\n\tnode.StopDaemon()\n\n\tt.Run(\"adding HTTP delegated routing endpoint to Routing.Routers config works (streaming)\", func(t *testing.T) {\n\t\tserver := fakeServer(\"application/x-ndjson\", ToJSONStr(JSONObj{\n\t\t\t\"Schema\": \"peer\",\n\t\t\t\"Protocols\": []string{\"transport-bitswap\"},\n\t\t\t\"ID\": provs[0],\n\t\t\t\"Addrs\": []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/tcp/4002\"},\n\t\t}), ToJSONStr(JSONObj{\n\t\t\t\"Schema\": \"bitswap\", // Legacy bitswap schema.\n\t\t\t\"Protocol\": \"transport-bitswap\",\n\t\t\t\"ID\": provs[1],\n\t\t\t\"Addrs\": []string{\"/ip4/0.0.0.0/tcp/4001\", \"/ip4/0.0.0.0/tcp/4002\"},\n\t\t}))\n\t\tt.Cleanup(server.Close)\n\n\t\tnode.IPFS(\"config\", \"Routing.Routers.TestDelegatedRouter\", \"--json\", ToJSONStr(JSONObj{\n\t\t\t\"Type\": \"http\",\n\t\t\t\"Parameters\": JSONObj{\n\t\t\t\t\"Endpoint\": server.URL,\n\t\t\t},\n\t\t}))\n\n\t\tres := node.IPFS(\"config\", \"Routing.Routers.TestDelegatedRouter.Parameters.Endpoint\")\n\t\tassert.Equal(t, res.Stdout.Trimmed(), server.URL)\n\n\t\tnode.StartDaemon()\n\t\tres = node.IPFS(\"routing\", \"findprovs\", findProvsCID)\n\t\tassert.Equal(t, provs[0]+\"\\n\"+provs[1], res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"HTTP client should emit OpenCensus metrics\", func(t *testing.T) {\n\t\tresp := node.APIClient().Get(\"/debug/metrics/prometheus\")\n\t\tassert.Contains(t, resp.Body, \"routing_http_client_length_count\")\n\t})\n}\n\n// TestHTTPDelegatedRoutingProviderAddrs verifies that provider records sent to\n// HTTP routers contain the expected addresses based on Addresses configuration.\n// See https://github.com/ipfs/kubo/issues/11213\nfunc TestHTTPDelegatedRoutingProviderAddrs(t *testing.T) {\n\tt.Parallel()\n\n\t// captureProviderAddrs returns a mock server and a function to retrieve captured addresses.\n\tcaptureProviderAddrs := func(t *testing.T) (*httptest.Server, func() []string) {\n\t\tt.Helper()\n\t\tvar mu sync.Mutex\n\t\tvar capturedAddrs []string\n\t\tsrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif (r.Method == http.MethodPut || r.Method == http.MethodPost) &&\n\t\t\t\tstrings.HasPrefix(r.URL.Path, \"/routing/v1/providers\") {\n\t\t\t\tbody, _ := io.ReadAll(r.Body)\n\t\t\t\tvar envelope struct {\n\t\t\t\t\tProviders []struct {\n\t\t\t\t\t\tPayload json.RawMessage `json:\"Payload\"`\n\t\t\t\t\t} `json:\"Providers\"`\n\t\t\t\t}\n\t\t\t\tif json.Unmarshal(body, &envelope) == nil {\n\t\t\t\t\tfor _, prov := range envelope.Providers {\n\t\t\t\t\t\tvar payload struct {\n\t\t\t\t\t\t\tAddrs []string `json:\"Addrs\"`\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif json.Unmarshal(prov.Payload, &payload) == nil && len(payload.Addrs) > 0 {\n\t\t\t\t\t\t\tmu.Lock()\n\t\t\t\t\t\t\tcapturedAddrs = payload.Addrs\n\t\t\t\t\t\t\tmu.Unlock()\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif strings.HasPrefix(r.URL.Path, \"/routing/v1/\") {\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t}))\n\t\tt.Cleanup(srv.Close)\n\t\treturn srv, func() []string {\n\t\t\tmu.Lock()\n\t\t\tdefer mu.Unlock()\n\t\t\treturn capturedAddrs\n\t\t}\n\t}\n\n\tcustomRoutingConf := func(endpoint string) map[string]any {\n\t\treturn map[string]any{\n\t\t\t\"Type\": \"custom\",\n\t\t\t\"Methods\": map[string]any{\n\t\t\t\t\"provide\": map[string]any{\"RouterName\": \"TestRouter\"},\n\t\t\t\t\"find-providers\": map[string]any{\"RouterName\": \"TestRouter\"},\n\t\t\t\t\"find-peers\": map[string]any{\"RouterName\": \"TestRouter\"},\n\t\t\t\t\"get-ipns\": map[string]any{\"RouterName\": \"TestRouter\"},\n\t\t\t\t\"put-ipns\": map[string]any{\"RouterName\": \"TestRouter\"},\n\t\t\t},\n\t\t\t\"Routers\": map[string]any{\n\t\t\t\t\"TestRouter\": map[string]any{\n\t\t\t\t\t\"Type\": \"http\",\n\t\t\t\t\t\"Parameters\": map[string]any{\"Endpoint\": endpoint},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t}\n\n\tt.Run(\"provider records respect user-provided Addresses.Announce override\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tsrv, getAddrs := captureProviderAddrs(t)\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Addresses.Announce\", []string{\"/ip4/1.2.3.4/tcp/4001\"})\n\t\tnode.SetIPFSConfig(\"Routing\", customRoutingConf(srv.URL))\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(time.Now().String())\n\t\tnode.IPFS(\"routing\", \"provide\", cidStr)\n\n\t\taddrs := getAddrs()\n\t\trequire.NotEmpty(t, addrs, \"provider record should contain addresses\")\n\t\tassert.Equal(t, []string{\"/ip4/1.2.3.4/tcp/4001\"}, addrs)\n\t})\n\n\tt.Run(\"provider records respect user-provided Addresses.AppendAnnounce\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tsrv, getAddrs := captureProviderAddrs(t)\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Addresses.AppendAnnounce\", []string{\"/ip4/5.6.7.8/tcp/4001\"})\n\t\tnode.SetIPFSConfig(\"Routing\", customRoutingConf(srv.URL))\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcidStr := node.IPFSAddStr(time.Now().String())\n\t\tnode.IPFS(\"routing\", \"provide\", cidStr)\n\n\t\taddrs := getAddrs()\n\t\trequire.NotEmpty(t, addrs, \"provider record should contain addresses\")\n\t\tassert.Contains(t, addrs, \"/ip4/5.6.7.8/tcp/4001\", \"AppendAnnounce address should be present\")\n\t})\n}\n" }, { "path": "test/cli/delegated_routing_v1_http_proxy_test.go", "content": "package cli\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRoutingV1Proxy(t *testing.T) {\n\tt.Parallel()\n\n\tsetupNodes := func(t *testing.T) harness.Nodes {\n\t\tnodes := harness.NewT(t).NewNodes(3).Init()\n\n\t\t// Node 0 uses DHT and exposes the Routing API. For the DHT\n\t\t// to actually work there will need to be another DHT-enabled\n\t\t// node.\n\t\tnodes[0].UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.ExposeRoutingAPI = config.True\n\t\t\tcfg.Discovery.MDNS.Enabled = false\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"dht\")\n\t\t})\n\t\tnodes[0].StartDaemon()\n\n\t\t// Node 1 uses Node 0 as Routing V1 source, no DHT.\n\t\tnodes[1].UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Discovery.MDNS.Enabled = false\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"custom\")\n\t\t\tcfg.Routing.Methods = config.Methods{\n\t\t\t\tconfig.MethodNameFindPeers: {RouterName: \"KuboA\"},\n\t\t\t\tconfig.MethodNameFindProviders: {RouterName: \"KuboA\"},\n\t\t\t\tconfig.MethodNameGetIPNS: {RouterName: \"KuboA\"},\n\t\t\t\tconfig.MethodNamePutIPNS: {RouterName: \"KuboA\"},\n\t\t\t\tconfig.MethodNameProvide: {RouterName: \"KuboA\"},\n\t\t\t}\n\t\t\tcfg.Routing.Routers = config.Routers{\n\t\t\t\t\"KuboA\": config.RouterParser{\n\t\t\t\t\tRouter: config.Router{\n\t\t\t\t\t\tType: config.RouterTypeHTTP,\n\t\t\t\t\t\tParameters: &config.HTTPRouterParams{\n\t\t\t\t\t\t\tEndpoint: nodes[0].GatewayURL(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t})\n\t\tnodes[1].StartDaemon()\n\n\t\t// This is the second DHT node. Only used so that the DHT is\n\t\t// operative.\n\t\tnodes[2].UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.ExposeRoutingAPI = config.True\n\t\t\tcfg.Discovery.MDNS.Enabled = false\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"dht\")\n\t\t})\n\t\tnodes[2].StartDaemon()\n\n\t\tt.Cleanup(func() {\n\t\t\tnodes.StopDaemons()\n\t\t})\n\n\t\t// Connect them.\n\t\tnodes.Connect()\n\n\t\treturn nodes\n\t}\n\n\tt.Run(\"Kubo can find provider for CID via Routing V1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\tcidStr := nodes[0].IPFSAddStr(string(random.Bytes(1000)))\n\t\t// Reprovide as initialProviderDelay still ongoing\n\t\twaitUntilProvidesComplete(t, nodes[0])\n\n\t\tres := nodes[1].IPFS(\"routing\", \"findprovs\", cidStr)\n\t\tassert.Equal(t, nodes[0].PeerID().String(), res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"Kubo can find peer via Routing V1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\t// Start lonely node that is not connected to other nodes.\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Discovery.MDNS.Enabled = false\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"dht\")\n\t\t})\n\t\tnode.StartDaemon()\n\n\t\t// Connect Node 0 to Lonely Node.\n\t\tnodes[0].Connect(node)\n\n\t\t// Node 1 must find Lonely Node through Node 0 Routing V1.\n\t\tres := nodes[1].IPFS(\"routing\", \"findpeer\", node.PeerID().String())\n\t\tassert.Equal(t, node.SwarmAddrs()[0].String(), res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"Kubo can retrieve IPNS record via Routing V1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\tnodeName := \"/ipns/\" + ipns.NameFromPeer(nodes[0].PeerID()).String()\n\n\t\t// Can't resolve the name as isn't published yet.\n\t\tres := nodes[1].RunIPFS(\"routing\", \"get\", nodeName)\n\t\trequire.Error(t, res.ExitErr)\n\n\t\t// Publish record on Node 0.\n\t\tpath := \"/ipfs/\" + nodes[0].IPFSAddStr(string(random.Bytes(1000)))\n\t\tnodes[0].IPFS(\"name\", \"publish\", \"--allow-offline\", path)\n\n\t\t// Get record on Node 1 (no DHT).\n\t\tres = nodes[1].IPFS(\"routing\", \"get\", nodeName)\n\t\trecord, err := ipns.UnmarshalRecord(res.Stdout.Bytes())\n\t\trequire.NoError(t, err)\n\t\tvalue, err := record.Value()\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, path, value.String())\n\t})\n\n\tt.Run(\"Kubo can resolve IPNS name via Routing V1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\tnodeName := \"/ipns/\" + ipns.NameFromPeer(nodes[0].PeerID()).String()\n\n\t\t// Can't resolve the name as isn't published yet.\n\t\tres := nodes[1].RunIPFS(\"routing\", \"get\", nodeName)\n\t\trequire.Error(t, res.ExitErr)\n\n\t\t// Publish name.\n\t\tpath := \"/ipfs/\" + nodes[0].IPFSAddStr(string(random.Bytes(1000)))\n\t\tnodes[0].IPFS(\"name\", \"publish\", \"--allow-offline\", path)\n\n\t\t// Resolve IPNS name\n\t\tres = nodes[1].IPFS(\"name\", \"resolve\", nodeName)\n\t\trequire.Equal(t, path, res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"Kubo can provide IPNS record via Routing V1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\t// Publish something on Node 1 (no DHT).\n\t\tnodeName := \"/ipns/\" + ipns.NameFromPeer(nodes[1].PeerID()).String()\n\t\tpath := \"/ipfs/\" + nodes[1].IPFSAddStr(string(random.Bytes(1000)))\n\t\tnodes[1].IPFS(\"name\", \"publish\", \"--allow-offline\", path)\n\n\t\t// Retrieve through Node 0.\n\t\tres := nodes[0].IPFS(\"routing\", \"get\", nodeName)\n\t\trecord, err := ipns.UnmarshalRecord(res.Stdout.Bytes())\n\t\trequire.NoError(t, err)\n\t\tvalue, err := record.Value()\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, path, value.String())\n\t})\n}\n" }, { "path": "test/cli/delegated_routing_v1_http_server_test.go", "content": "package cli\n\nimport (\n\t\"context\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/ipfs/boxo/autoconf\"\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/routing/http/client\"\n\t\"github.com/ipfs/boxo/routing/http/types\"\n\t\"github.com/ipfs/boxo/routing/http/types/iter\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRoutingV1Server(t *testing.T) {\n\tt.Parallel()\n\n\tsetupNodes := func(t *testing.T) harness.Nodes {\n\t\tnodes := harness.NewT(t).NewNodes(5).Init()\n\t\tnodes.ForEachPar(func(node *harness.Node) {\n\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Gateway.ExposeRoutingAPI = config.True\n\t\t\t\tcfg.Routing.Type = config.NewOptionalString(\"dht\")\n\t\t\t})\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\t\tt.Cleanup(func() { nodes.StopDaemons() })\n\t\treturn nodes\n\t}\n\n\tt.Run(\"Get Providers Responds With Correct Peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\ttext := \"hello world \" + uuid.New().String()\n\t\tcidStr := nodes[2].IPFSAddStr(text)\n\t\t_ = nodes[3].IPFSAddStr(text)\n\t\twaitUntilProvidesComplete(t, nodes[3])\n\n\t\tcid, err := cid.Decode(cidStr)\n\t\tassert.NoError(t, err)\n\n\t\tc, err := client.New(nodes[1].GatewayURL())\n\t\tassert.NoError(t, err)\n\n\t\tresultsIter, err := c.FindProviders(context.Background(), cid)\n\t\tassert.NoError(t, err)\n\n\t\trecords, err := iter.ReadAllResults(resultsIter)\n\t\tassert.NoError(t, err)\n\n\t\tvar peers []peer.ID\n\t\tfor _, record := range records {\n\t\t\tassert.Equal(t, types.SchemaPeer, record.GetSchema())\n\n\t\t\tpeer, ok := record.(*types.PeerRecord)\n\t\t\tassert.True(t, ok)\n\t\t\tpeers = append(peers, *peer.ID)\n\t\t}\n\n\t\tassert.Contains(t, peers, nodes[2].PeerID())\n\t\tassert.Contains(t, peers, nodes[3].PeerID())\n\t})\n\n\tt.Run(\"Get Peers Responds With Correct Peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\tc, err := client.New(nodes[1].GatewayURL())\n\t\tassert.NoError(t, err)\n\n\t\tresultsIter, err := c.FindPeers(context.Background(), nodes[2].PeerID())\n\t\tassert.NoError(t, err)\n\n\t\trecords, err := iter.ReadAllResults(resultsIter)\n\t\tassert.NoError(t, err)\n\t\tassert.Len(t, records, 1)\n\t\tassert.IsType(t, records[0].GetSchema(), records[0].GetSchema())\n\t\tassert.IsType(t, records[0], &types.PeerRecord{})\n\n\t\tpeer := records[0]\n\t\tassert.Equal(t, nodes[2].PeerID().String(), peer.ID.String())\n\t\tassert.NotEmpty(t, peer.Addrs)\n\t})\n\n\tt.Run(\"Get IPNS Record Responds With Correct Record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\ttext := \"hello ipns test \" + uuid.New().String()\n\t\tcidStr := nodes[0].IPFSAddStr(text)\n\t\tnodes[0].IPFS(\"name\", \"publish\", \"--allow-offline\", cidStr)\n\n\t\t// Ask for record from a different peer.\n\t\tc, err := client.New(nodes[1].GatewayURL())\n\t\tassert.NoError(t, err)\n\n\t\trecord, err := c.GetIPNS(context.Background(), ipns.NameFromPeer(nodes[0].PeerID()))\n\t\tassert.NoError(t, err)\n\n\t\tvalue, err := record.Value()\n\t\tassert.NoError(t, err)\n\t\tassert.Equal(t, \"/ipfs/\"+cidStr, value.String())\n\t})\n\n\tt.Run(\"Put IPNS Record Succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := setupNodes(t)\n\n\t\t// Publish a record and confirm the /routing/v1/ipns API exposes the IPNS record\n\t\ttext := \"hello ipns test \" + uuid.New().String()\n\t\tcidStr := nodes[0].IPFSAddStr(text)\n\t\tnodes[0].IPFS(\"name\", \"publish\", \"--allow-offline\", cidStr)\n\t\tc, err := client.New(nodes[0].GatewayURL())\n\t\tassert.NoError(t, err)\n\t\trecord, err := c.GetIPNS(context.Background(), ipns.NameFromPeer(nodes[0].PeerID()))\n\t\tassert.NoError(t, err)\n\t\tvalue, err := record.Value()\n\t\tassert.NoError(t, err)\n\t\tassert.Equal(t, \"/ipfs/\"+cidStr, value.String())\n\n\t\t// Start lonely node that is not connected to other nodes.\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.ExposeRoutingAPI = config.True\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"dht\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Put IPNS record in lonely node. It should be accepted as it is a valid record.\n\t\tc, err = client.New(node.GatewayURL())\n\t\tassert.NoError(t, err)\n\t\terr = c.PutIPNS(context.Background(), ipns.NameFromPeer(nodes[0].PeerID()), record)\n\t\tassert.NoError(t, err)\n\n\t\t// Get the record from lonely node and double check.\n\t\trecord, err = c.GetIPNS(context.Background(), ipns.NameFromPeer(nodes[0].PeerID()))\n\t\tassert.NoError(t, err)\n\t\tvalue, err = record.Value()\n\t\tassert.NoError(t, err)\n\t\tassert.Equal(t, \"/ipfs/\"+cidStr, value.String())\n\t})\n\n\tt.Run(\"GetClosestPeers returns error when DHT is disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Test various routing types that don't support DHT\n\t\troutingTypes := []string{\"none\", \"delegated\", \"custom\"}\n\t\tfor _, routingType := range routingTypes {\n\t\t\tt.Run(\"routing_type=\"+routingType, func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\n\t\t\t\t// Create node with specified routing type (DHT disabled)\n\t\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\t\tcfg.Gateway.ExposeRoutingAPI = config.True\n\t\t\t\t\tcfg.Routing.Type = config.NewOptionalString(routingType)\n\n\t\t\t\t\t// For custom routing type, we need to provide minimal valid config\n\t\t\t\t\t// otherwise daemon startup will fail\n\t\t\t\t\tif routingType == \"custom\" {\n\t\t\t\t\t\t// Configure a minimal HTTP router (no DHT)\n\t\t\t\t\t\tcfg.Routing.Routers = map[string]config.RouterParser{\n\t\t\t\t\t\t\t\"http-only\": {\n\t\t\t\t\t\t\t\tRouter: config.Router{\n\t\t\t\t\t\t\t\t\tType: config.RouterTypeHTTP,\n\t\t\t\t\t\t\t\t\tParameters: config.HTTPRouterParams{\n\t\t\t\t\t\t\t\t\t\tEndpoint: \"https://delegated-ipfs.dev\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}\n\t\t\t\t\t\tcfg.Routing.Methods = map[config.MethodName]config.Method{\n\t\t\t\t\t\t\tconfig.MethodNameProvide: {RouterName: \"http-only\"},\n\t\t\t\t\t\t\tconfig.MethodNameFindProviders: {RouterName: \"http-only\"},\n\t\t\t\t\t\t\tconfig.MethodNameFindPeers: {RouterName: \"http-only\"},\n\t\t\t\t\t\t\tconfig.MethodNameGetIPNS: {RouterName: \"http-only\"},\n\t\t\t\t\t\t\tconfig.MethodNamePutIPNS: {RouterName: \"http-only\"},\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\n\t\t\t\t\t// For delegated routing type, ensure we have at least one HTTP router\n\t\t\t\t\t// to avoid daemon startup failure\n\t\t\t\t\tif routingType == \"delegated\" {\n\t\t\t\t\t\t// Use a minimal delegated router configuration\n\t\t\t\t\t\tcfg.Routing.DelegatedRouters = []string{\"https://delegated-ipfs.dev\"}\n\t\t\t\t\t\t// Delegated routing doesn't support providing, must be disabled\n\t\t\t\t\t\tcfg.Provide.Enabled = config.False\n\t\t\t\t\t}\n\t\t\t\t})\n\t\t\t\tnode.StartDaemon()\n\t\t\t\tdefer node.StopDaemon()\n\n\t\t\t\tc, err := client.New(node.GatewayURL())\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t// Try to get closest peers - should fail gracefully with an error.\n\t\t\t\t// Use 60-second timeout (server has 30s routing timeout).\n\t\t\t\ttestCid, err := cid.Decode(\"QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\")\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)\n\t\t\t\tdefer cancel()\n\t\t\t\t_, err = c.GetClosestPeers(ctx, testCid)\n\t\t\t\trequire.Error(t, err)\n\t\t\t\t// All these routing types should indicate DHT is not available\n\t\t\t\t// The exact error message may vary based on implementation details\n\t\t\t\terrStr := err.Error()\n\t\t\t\tassert.True(t,\n\t\t\t\t\tstrings.Contains(errStr, \"not supported\") ||\n\t\t\t\t\t\tstrings.Contains(errStr, \"not available\") ||\n\t\t\t\t\t\tstrings.Contains(errStr, \"500\"),\n\t\t\t\t\t\"Expected error indicating DHT not available for routing type %s, got: %s\", routingType, errStr)\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"GetClosestPeers returns peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\troutingTypes := []string{\"auto\", \"autoclient\", \"dht\", \"dhtclient\"}\n\t\tfor _, routingType := range routingTypes {\n\t\t\tt.Run(\"routing_type=\"+routingType, func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\n\t\t\t\t// Single node with DHT and real bootstrap peers\n\t\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\t\tcfg.Gateway.ExposeRoutingAPI = config.True\n\t\t\t\t\tcfg.Routing.Type = config.NewOptionalString(routingType)\n\t\t\t\t\t// Set real bootstrap peers from boxo/autoconf\n\t\t\t\t\tcfg.Bootstrap = autoconf.FallbackBootstrapPeers\n\t\t\t\t})\n\t\t\t\tnode.StartDaemon()\n\t\t\t\tdefer node.StopDaemon()\n\n\t\t\t\tc, err := client.New(node.GatewayURL())\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t// Query for closest peers to our own peer ID\n\t\t\t\tkey := peer.ToCid(node.PeerID())\n\n\t\t\t\t// Wait for WAN DHT routing table to be populated.\n\t\t\t\t// The server has a 30-second routing timeout, so we use 60 seconds\n\t\t\t\t// per request to allow for network latency while preventing hangs.\n\t\t\t\t// Total wait time is 5 minutes to accommodate slow CI DHT bootstrapping.\n\t\t\t\t// Passing runs finish in 8-48s; failures are total bootstrap failures,\n\t\t\t\t// not slow convergence, so extra headroom doesn't waste time on success.\n\t\t\t\tvar records []*types.PeerRecord\n\t\t\t\trequire.EventuallyWithT(t, func(ct *assert.CollectT) {\n\t\t\t\t\tctx, cancel := context.WithTimeout(t.Context(), 60*time.Second)\n\t\t\t\t\tdefer cancel()\n\t\t\t\t\tresultsIter, err := c.GetClosestPeers(ctx, key)\n\t\t\t\t\tif !assert.NoError(ct, err) {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\trecords, err = iter.ReadAllResults(resultsIter)\n\t\t\t\t\tassert.NoError(ct, err)\n\t\t\t\t}, 5*time.Minute, 5*time.Second)\n\n\t\t\t\t// Verify we got some peers back from WAN DHT\n\t\t\t\trequire.NotEmpty(t, records, \"should return peers close to own peerid\")\n\n\t\t\t\t// Per IPIP-0476, GetClosestPeers returns at most 20 peers\n\t\t\t\tassert.LessOrEqual(t, len(records), 20, \"IPIP-0476 limits GetClosestPeers to 20 peers\")\n\n\t\t\t\t// Verify structure of returned records\n\t\t\t\tfor _, record := range records {\n\t\t\t\t\tassert.Equal(t, types.SchemaPeer, record.Schema)\n\t\t\t\t\tassert.NotNil(t, record.ID)\n\t\t\t\t\tassert.NotEmpty(t, record.Addrs, \"peer record should have addresses\")\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n}\n" }, { "path": "test/cli/dht_autoclient_test.go", "content": "package cli\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestDHTAutoclient(t *testing.T) {\n\tt.Parallel()\n\tnodes := harness.NewT(t).NewNodes(10).Init()\n\tharness.Nodes(nodes[8:]).ForEachPar(func(node *harness.Node) {\n\t\tnode.IPFS(\"config\", \"Routing.Type\", \"autoclient\")\n\t})\n\tnodes.StartDaemons().Connect()\n\tt.Cleanup(func() { nodes.StopDaemons() })\n\n\tt.Run(\"file added on node in client mode is retrievable from node in client mode\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\trandomBytes := random.Bytes(1000)\n\t\trandomBytes = append(randomBytes, '\\r')\n\t\thash := nodes[8].IPFSAdd(bytes.NewReader(randomBytes))\n\n\t\tres := nodes[9].IPFS(\"cat\", hash)\n\t\tassert.Equal(t, randomBytes, []byte(res.Stdout.Trimmed()))\n\t})\n\n\tt.Run(\"file added on node in server mode is retrievable from all nodes\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\trandomBytes := random.Bytes(1000)\n\t\thash := nodes[0].IPFSAdd(bytes.NewReader(randomBytes))\n\n\t\tfor i := range 10 {\n\t\t\tres := nodes[i].IPFS(\"cat\", hash)\n\t\t\tassert.Equal(t, randomBytes, []byte(res.Stdout.Trimmed()))\n\t\t}\n\t})\n}\n" }, { "path": "test/cli/dht_opt_prov_test.go", "content": "package cli\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestDHTOptimisticProvide(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"optimistic provide smoke test\", func(t *testing.T) {\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\n\t\tnodes[0].UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Experimental.OptimisticProvide = true\n\t\t\t// Optimistic provide only works with the legacy provider.\n\t\t\tcfg.Provide.DHT.SweepEnabled = config.False\n\t\t})\n\n\t\tnodes.StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\n\t\thash := nodes[0].IPFSAddStr(string(random.Bytes(100)))\n\t\tnodes[0].IPFS(\"routing\", \"provide\", hash)\n\n\t\tres := nodes[1].IPFS(\"routing\", \"findprovs\", \"--num-providers=1\", hash)\n\t\tassert.Equal(t, nodes[0].PeerID().String(), res.Stdout.Trimmed())\n\t})\n}\n" }, { "path": "test/cli/diag_datastore_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestDiagDatastore(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"diag datastore get returns error for non-existent key\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t// Don't start daemon - these commands require daemon to be stopped\n\n\t\tres := node.RunIPFS(\"diag\", \"datastore\", \"get\", \"/nonexistent/key\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"key not found\")\n\t})\n\n\tt.Run(\"diag datastore get returns raw bytes by default\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Add some data to create a known datastore key\n\t\t// We need daemon for add, then stop it\n\t\tnode.StartDaemon()\n\t\tcid := node.IPFSAddStr(\"test data for diag datastore\")\n\t\tnode.IPFS(\"pin\", \"add\", cid)\n\t\tnode.StopDaemon()\n\n\t\t// Test count to verify we have entries\n\t\tcount := node.DatastoreCount(\"/\")\n\t\tt.Logf(\"total datastore entries: %d\", count)\n\t\tassert.NotEqual(t, int64(0), count, \"should have datastore entries after pinning\")\n\t})\n\n\tt.Run(\"diag datastore get --hex returns hex dump\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Add and pin some data\n\t\tnode.StartDaemon()\n\t\tcid := node.IPFSAddStr(\"test data for hex dump\")\n\t\tnode.IPFS(\"pin\", \"add\", cid)\n\t\tnode.StopDaemon()\n\n\t\t// Test with existing keys in pins namespace\n\t\tcount := node.DatastoreCount(\"/pins/\")\n\t\tt.Logf(\"pins datastore entries: %d\", count)\n\n\t\tif count != 0 {\n\t\t\tt.Log(\"pins datastore has entries, hex dump format tested implicitly\")\n\t\t}\n\t})\n\n\tt.Run(\"diag datastore count returns 0 for empty prefix\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tcount := node.DatastoreCount(\"/definitely/nonexistent/prefix/\")\n\t\tassert.Equal(t, int64(0), count)\n\t})\n\n\tt.Run(\"diag datastore count returns JSON with --enc=json\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tres := node.IPFS(\"diag\", \"datastore\", \"count\", \"/pubsub/seqno/\", \"--enc=json\")\n\t\tassert.NoError(t, res.Err)\n\n\t\tvar result struct {\n\t\t\tPrefix string `json:\"prefix\"`\n\t\t\tCount int64 `json:\"count\"`\n\t\t}\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &result)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"/pubsub/seqno/\", result.Prefix)\n\t\tassert.Equal(t, int64(0), result.Count)\n\t})\n\n\tt.Run(\"diag datastore get returns JSON with --enc=json\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Test error case with JSON encoding\n\t\tres := node.RunIPFS(\"diag\", \"datastore\", \"get\", \"/nonexistent\", \"--enc=json\")\n\t\tassert.Error(t, res.Err)\n\t})\n\n\tt.Run(\"diag datastore count counts entries correctly\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Add multiple pins to create multiple entries\n\t\tnode.StartDaemon()\n\t\tcid1 := node.IPFSAddStr(\"data 1\")\n\t\tcid2 := node.IPFSAddStr(\"data 2\")\n\t\tcid3 := node.IPFSAddStr(\"data 3\")\n\n\t\tnode.IPFS(\"pin\", \"add\", cid1)\n\t\tnode.IPFS(\"pin\", \"add\", cid2)\n\t\tnode.IPFS(\"pin\", \"add\", cid3)\n\t\tnode.StopDaemon()\n\n\t\t// Count should reflect the pins (plus any system entries)\n\t\tcount := node.DatastoreCount(\"/\")\n\t\tt.Logf(\"total entries after adding 3 pins: %d\", count)\n\n\t\t// Should have more than 0 entries\n\t\tassert.NotEqual(t, int64(0), count)\n\t})\n\n\tt.Run(\"diag datastore commands work offline\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t// Don't start daemon - these commands require daemon to be stopped\n\n\t\t// Count should work offline\n\t\tcount := node.DatastoreCount(\"/pubsub/seqno/\")\n\t\tassert.Equal(t, int64(0), count)\n\n\t\t// Get should return error for missing key (but command should work)\n\t\tres := node.RunIPFS(\"diag\", \"datastore\", \"get\", \"/nonexistent/key\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"key not found\")\n\t})\n\n\tt.Run(\"diag datastore put and get roundtrip\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tnode.DatastorePut(\"/test/roundtrip\", \"hello world\")\n\t\tassert.True(t, node.DatastoreHasKey(\"/test/roundtrip\"))\n\t\tassert.Equal(t, []byte(\"hello world\"), node.DatastoreGet(\"/test/roundtrip\"))\n\n\t\tcount := node.DatastoreCount(\"/test/\")\n\t\tassert.Equal(t, int64(1), count)\n\t})\n\n\tt.Run(\"diag datastore commands require daemon to be stopped\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Both get and count require repo lock, which is held by the running daemon\n\t\tres := node.RunIPFS(\"diag\", \"datastore\", \"get\", \"/test\")\n\t\tassert.Error(t, res.Err, \"get should fail when daemon is running\")\n\t\tassert.Contains(t, res.Stderr.String(), \"ipfs daemon is running\")\n\n\t\tres = node.RunIPFS(\"diag\", \"datastore\", \"count\", \"/pubsub/seqno/\")\n\t\tassert.Error(t, res.Err, \"count should fail when daemon is running\")\n\t\tassert.Contains(t, res.Stderr.String(), \"ipfs daemon is running\")\n\t})\n\n\tt.Run(\"provider keystore datastores are visible in unified view\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\n\t\t// Start daemon to create the provider-keystore datastores, then add data\n\t\tnode.StartDaemon()\n\t\tcid := node.IPFSAddStr(\"data for provider keystore test\")\n\t\tnode.IPFS(\"pin\", \"add\", cid)\n\t\tnode.StopDaemon()\n\n\t\t// Verify the provider-keystore directory was created\n\t\tkeystorePath := filepath.Join(node.Dir, \"provider-keystore\")\n\t\t_, err := os.Stat(keystorePath)\n\t\trequire.NoError(t, err, \"provider-keystore directory should exist after sweep-enabled daemon ran\")\n\n\t\t// Count entries in each keystore namespace via the unified view\n\t\tfor _, prefix := range []string{\"/provider/keystore/0/\", \"/provider/keystore/1/\"} {\n\t\t\tres := node.IPFS(\"diag\", \"datastore\", \"count\", prefix)\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tt.Logf(\"count %s: %s\", prefix, res.Stdout.String())\n\t\t}\n\n\t\t// The total count under /provider/keystore/ should include entries\n\t\t// from both keystore instances (0 and 1)\n\t\tcount := node.DatastoreCount(\"/provider/keystore/\")\n\t\tt.Logf(\"total /provider/keystore/ entries: %d\", count)\n\t\tassert.Greater(t, count, int64(0), \"should have provider keystore entries\")\n\t})\n\n\tt.Run(\"provider keystore count JSON output\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\n\t\tnode.StartDaemon()\n\t\tnode.StopDaemon()\n\n\t\tres := node.IPFS(\"diag\", \"datastore\", \"count\", \"/provider/keystore/0/\", \"--enc=json\")\n\t\tassert.NoError(t, res.Err)\n\n\t\tvar result struct {\n\t\t\tPrefix string `json:\"prefix\"`\n\t\t\tCount int64 `json:\"count\"`\n\t\t}\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &result)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"/provider/keystore/0/\", result.Prefix)\n\t\tassert.GreaterOrEqual(t, result.Count, int64(0), \"count should be non-negative\")\n\t})\n\n\tt.Run(\"works without provider keystore\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// No sweep enabled, no provider-keystore dirs — should still work fine\n\t\tcount := node.DatastoreCount(\"/provider/keystore/0/\")\n\t\tassert.Zero(t, count)\n\n\t\tcount = node.DatastoreCount(\"/\")\n\t\tassert.Greater(t, count, int64(0))\n\t})\n}\n" }, { "path": "test/cli/dns_resolvers_multiaddr_test.go", "content": "package cli\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// testDomainSuffix is the default p2p-forge domain used in tests\nconst testDomainSuffix = config.DefaultDomainSuffix // libp2p.direct\n\n// TestDNSResolversApplyToMultiaddr is a regression test for:\n// https://github.com/ipfs/kubo/issues/9199\n//\n// It verifies that DNS.Resolvers config is used when resolving /dnsaddr,\n// /dns, /dns4, /dns6 multiaddrs during peer connections, not just for\n// DNSLink resolution.\nfunc TestDNSResolversApplyToMultiaddr(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"invalid DoH resolver causes multiaddr resolution to fail\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\n\t\t// Set an invalid DoH resolver that will fail when used.\n\t\t// If DNS.Resolvers is properly wired to multiaddr resolution,\n\t\t// swarm connect to a /dnsaddr will fail with an error mentioning\n\t\t// the invalid resolver URL.\n\t\tinvalidResolver := \"https://invalid.broken.resolver.test/dns-query\"\n\t\tnode.SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\n\t\t\t\".\": invalidResolver,\n\t\t})\n\n\t\t// Clear bootstrap peers to prevent background connection attempts\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Give daemon time to fully start\n\t\ttime.Sleep(2 * time.Second)\n\n\t\t// Verify daemon is responsive\n\t\tresult := node.RunIPFS(\"id\")\n\t\trequire.Equal(t, 0, result.ExitCode(), \"daemon should be responsive\")\n\n\t\t// Try to connect to a /dnsaddr peer - this should fail because\n\t\t// the DNS.Resolvers config points to an invalid DoH server\n\t\tresult = node.RunIPFS(\"swarm\", \"connect\", \"/dnsaddr/bootstrap.libp2p.io\")\n\n\t\t// The connection should fail\n\t\trequire.NotEqual(t, 0, result.ExitCode(),\n\t\t\t\"swarm connect should fail when DNS.Resolvers points to invalid DoH server\")\n\n\t\t// The error should mention the invalid resolver, proving DNS.Resolvers\n\t\t// is being used for multiaddr resolution\n\t\tstderr := result.Stderr.String()\n\t\tassert.True(t,\n\t\t\tstrings.Contains(stderr, \"invalid.broken.resolver.test\") ||\n\t\t\t\tstrings.Contains(stderr, \"no such host\") ||\n\t\t\t\tstrings.Contains(stderr, \"lookup\") ||\n\t\t\t\tstrings.Contains(stderr, \"dial\"),\n\t\t\t\"error should indicate DNS resolution failure using custom resolver. got: %s\", stderr)\n\t})\n\n\tt.Run(\"libp2p.direct resolves locally even with broken DNS.Resolvers\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnodes := h.NewNodes(2).Init(\"--profile=test\")\n\n\t\t// Configure node0 with a broken DNS resolver\n\t\t// This would break all DNS resolution if libp2p.direct wasn't resolved locally\n\t\tinvalidResolver := \"https://invalid.broken.resolver.test/dns-query\"\n\t\tnodes[0].SetIPFSConfig(\"DNS.Resolvers\", map[string]string{\n\t\t\t\".\": invalidResolver,\n\t\t})\n\n\t\t// Clear bootstrap peers on both nodes\n\t\tfor _, n := range nodes {\n\t\t\tn.SetIPFSConfig(\"Bootstrap\", []string{})\n\t\t}\n\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Get node1's peer ID in base36 format (what p2p-forge uses in DNS hostnames)\n\t\t// DNS is case-insensitive, and base36 is lowercase-only, making it ideal for DNS\n\t\tidResult := nodes[1].RunIPFS(\"id\", \"--peerid-base\", \"base36\", \"-f\", \"\")\n\t\trequire.Equal(t, 0, idResult.ExitCode())\n\t\tnode1IDBase36 := strings.TrimSpace(idResult.Stdout.String())\n\t\tnode1ID := nodes[1].PeerID().String()\n\t\tnode1Addrs := nodes[1].SwarmAddrs()\n\n\t\t// Find a TCP address we can use\n\t\tvar tcpAddr string\n\t\tfor _, addr := range node1Addrs {\n\t\t\taddrStr := addr.String()\n\t\t\tif strings.Contains(addrStr, \"/tcp/\") && strings.Contains(addrStr, \"/ip4/127.0.0.1\") {\n\t\t\t\ttcpAddr = addrStr\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\trequire.NotEmpty(t, tcpAddr, \"node1 should have a local TCP address\")\n\n\t\t// Extract port from address like /ip4/127.0.0.1/tcp/12345/...\n\t\tparts := strings.Split(tcpAddr, \"/\")\n\t\tvar port string\n\t\tfor i, p := range parts {\n\t\t\tif p == \"tcp\" && i+1 < len(parts) {\n\t\t\t\tport = parts[i+1]\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\trequire.NotEmpty(t, port, \"should find TCP port in address\")\n\n\t\t// Construct a libp2p.direct hostname that encodes 127.0.0.1\n\t\t// Format: /dns4/..libp2p.direct/tcp//p2p/\n\t\t// p2p-forge uses base36 peerIDs in DNS hostnames (lowercase, DNS-safe)\n\t\tlibp2pDirectAddr := \"/dns4/127-0-0-1.\" + node1IDBase36 + \".\" + testDomainSuffix + \"/tcp/\" + port + \"/p2p/\" + node1ID\n\n\t\t// This connection should succeed because libp2p.direct is resolved locally\n\t\t// even though DNS.Resolvers points to a broken server\n\t\tresult := nodes[0].RunIPFS(\"swarm\", \"connect\", libp2pDirectAddr)\n\n\t\t// The connection should succeed - local resolution bypasses broken DNS\n\t\tassert.Equal(t, 0, result.ExitCode(),\n\t\t\t\"swarm connect to libp2p.direct should succeed with local resolution. stderr: %s\",\n\t\t\tresult.Stderr.String())\n\n\t\t// Verify the connection was actually established\n\t\tresult = nodes[0].RunIPFS(\"swarm\", \"peers\")\n\t\trequire.Equal(t, 0, result.ExitCode())\n\t\tassert.Contains(t, result.Stdout.String(), node1ID,\n\t\t\t\"node0 should be connected to node1\")\n\t})\n}\n" }, { "path": "test/cli/files_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestFilesCp(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"files cp with valid UnixFS succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create simple text file\n\t\tdata := \"testing files cp command\"\n\t\tcid := node.IPFSAddStr(data)\n\n\t\t// Copy form IPFS => MFS\n\t\tres := node.IPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", cid), \"/valid-file\")\n\t\tassert.NoError(t, res.Err)\n\n\t\t// verification\n\t\tcatRes := node.IPFS(\"files\", \"read\", \"/valid-file\")\n\t\tassert.Equal(t, data, catRes.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"files cp with unsupported DAG node type fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// MFS UnixFS is limited to dag-pb or raw, so we create a dag-cbor node to test this\n\t\tjsonData := `{\"data\": \"not a UnixFS node\"}`\n\t\ttempFile := filepath.Join(node.Dir, \"test.json\")\n\t\terr := os.WriteFile(tempFile, []byte(jsonData), 0644)\n\t\trequire.NoError(t, err)\n\t\tcid := node.IPFS(\"dag\", \"put\", \"--input-codec=json\", \"--store-codec=dag-cbor\", tempFile).Stdout.Trimmed()\n\n\t\t// copy without --force\n\t\tres := node.RunIPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", cid), \"/invalid-file\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"Error: cp: source must be a valid UnixFS (dag-pb or raw codec)\")\n\t})\n\n\tt.Run(\"files cp with invalid UnixFS data structure fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create an invalid proto file\n\t\tdata := []byte{0xDE, 0xAD, 0xBE, 0xEF} // Invalid protobuf data\n\t\ttempFile := filepath.Join(node.Dir, \"invalid-proto.bin\")\n\t\terr := os.WriteFile(tempFile, data, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.IPFS(\"block\", \"put\", \"--format=raw\", tempFile)\n\t\trequire.NoError(t, res.Err)\n\n\t\t// we manually changed codec from raw to dag-pb to test \"bad dag-pb\" scenario\n\t\tcid := \"bafybeic7pdbte5heh6u54vszezob3el6exadoiw4wc4ne7ny2x7kvajzkm\"\n\n\t\t// should fail because node cannot be read as a valid dag-pb\n\t\tcpResNoForce := node.RunIPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", cid), \"/invalid-proto\")\n\t\tassert.NotEqual(t, 0, cpResNoForce.ExitErr.ExitCode())\n\t\tassert.Contains(t, cpResNoForce.Stderr.String(), \"Error\")\n\t})\n\n\tt.Run(\"files cp with raw node succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a raw node\n\t\tdata := \"raw data\"\n\t\ttempFile := filepath.Join(node.Dir, \"raw.bin\")\n\t\terr := os.WriteFile(tempFile, []byte(data), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.IPFS(\"block\", \"put\", \"--format=raw\", tempFile)\n\t\trequire.NoError(t, res.Err)\n\t\tcid := res.Stdout.Trimmed()\n\n\t\t// Copy from IPFS to MFS (raw nodes should work without --force)\n\t\tcpRes := node.IPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", cid), \"/raw-file\")\n\t\tassert.NoError(t, cpRes.Err)\n\n\t\t// Verify the file was copied correctly\n\t\tcatRes := node.IPFS(\"files\", \"read\", \"/raw-file\")\n\t\tassert.Equal(t, data, catRes.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"files cp creates intermediate directories with -p\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a simple text file and add it to IPFS\n\t\tdata := \"hello parent directories\"\n\t\ttempFile := filepath.Join(node.Dir, \"parent-test.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(data), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tcid := node.IPFS(\"add\", \"-Q\", tempFile).Stdout.Trimmed()\n\n\t\t// Copy from IPFS to MFS with parent flag\n\t\tres := node.IPFS(\"files\", \"cp\", \"-p\", fmt.Sprintf(\"/ipfs/%s\", cid), \"/parent/dir/file\")\n\t\tassert.NoError(t, res.Err)\n\n\t\t// Verify the file and directories were created\n\t\tlsRes := node.IPFS(\"files\", \"ls\", \"/parent/dir\")\n\t\tassert.Contains(t, lsRes.Stdout.String(), \"file\")\n\n\t\tcatRes := node.IPFS(\"files\", \"read\", \"/parent/dir/file\")\n\t\tassert.Equal(t, data, catRes.Stdout.Trimmed())\n\t})\n}\n\nfunc TestFilesRm(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"files rm with --flush=false returns error\", func(t *testing.T) {\n\t\t// Test that files rm rejects --flush=false so user does not assume disabling flush works\n\t\t// (rm ignored it before, better to explicitly error)\n\t\t// See https://github.com/ipfs/kubo/issues/10842\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a file to remove\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/test-dir\")\n\n\t\t// Try to remove with --flush=false, should error\n\t\tres := node.RunIPFS(\"files\", \"rm\", \"-r\", \"--flush=false\", \"/test-dir\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"files rm always flushes for safety\")\n\t\tassert.Contains(t, res.Stderr.String(), \"cannot be set to false\")\n\n\t\t// Verify the directory still exists (wasn't removed due to error)\n\t\tlsRes := node.IPFS(\"files\", \"ls\", \"/\")\n\t\tassert.Contains(t, lsRes.Stdout.String(), \"test-dir\")\n\t})\n\n\tt.Run(\"files rm with --flush=true works\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a file to remove\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/test-dir\")\n\n\t\t// Remove with explicit --flush=true, should work\n\t\tres := node.IPFS(\"files\", \"rm\", \"-r\", \"--flush=true\", \"/test-dir\")\n\t\tassert.NoError(t, res.Err)\n\n\t\t// Verify the directory was removed\n\t\tlsRes := node.IPFS(\"files\", \"ls\", \"/\")\n\t\tassert.NotContains(t, lsRes.Stdout.String(), \"test-dir\")\n\t})\n\n\tt.Run(\"files rm without flush flag works (default behavior)\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a file to remove\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/test-dir\")\n\n\t\t// Remove without flush flag (should use default which is true)\n\t\tres := node.IPFS(\"files\", \"rm\", \"-r\", \"/test-dir\")\n\t\tassert.NoError(t, res.Err)\n\n\t\t// Verify the directory was removed\n\t\tlsRes := node.IPFS(\"files\", \"ls\", \"/\")\n\t\tassert.NotContains(t, lsRes.Stdout.String(), \"test-dir\")\n\t})\n}\n\nfunc TestFilesNoFlushLimit(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"reaches default limit of 256 operations\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Perform 256 operations with --flush=false (should succeed)\n\t\tfor i := range 256 {\n\t\t\tres := node.IPFS(\"files\", \"mkdir\", \"--flush=false\", fmt.Sprintf(\"/dir%d\", i))\n\t\t\tassert.NoError(t, res.Err, \"operation %d should succeed\", i+1)\n\t\t}\n\n\t\t// 257th operation should fail\n\t\tres := node.RunIPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir256\")\n\t\trequire.NotNil(t, res.ExitErr, \"command should have failed\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"reached limit of 256 unflushed MFS operations\")\n\t\tassert.Contains(t, res.Stderr.String(), \"run 'ipfs files flush'\")\n\t\tassert.Contains(t, res.Stderr.String(), \"use --flush=true\")\n\t\tassert.Contains(t, res.Stderr.String(), \"increase Internal.MFSNoFlushLimit\")\n\t})\n\n\tt.Run(\"custom limit via config\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Set custom limit to 5\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tlimit := config.NewOptionalInteger(5)\n\t\t\tcfg.Internal.MFSNoFlushLimit = limit\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Perform 5 operations (should succeed)\n\t\tfor i := range 5 {\n\t\t\tres := node.IPFS(\"files\", \"mkdir\", \"--flush=false\", fmt.Sprintf(\"/dir%d\", i))\n\t\t\tassert.NoError(t, res.Err, \"operation %d should succeed\", i+1)\n\t\t}\n\n\t\t// 6th operation should fail\n\t\tres := node.RunIPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir5\")\n\t\trequire.NotNil(t, res.ExitErr, \"command should have failed\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"reached limit of 5 unflushed MFS operations\")\n\t})\n\n\tt.Run(\"flush=true resets counter\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Set limit to 3 for faster testing\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tlimit := config.NewOptionalInteger(3)\n\t\t\tcfg.Internal.MFSNoFlushLimit = limit\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Do 2 operations with --flush=false\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir1\")\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir2\")\n\n\t\t// Operation with --flush=true should reset counter\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=true\", \"/dir3\")\n\n\t\t// Now we should be able to do 3 more operations with --flush=false\n\t\tfor i := 4; i <= 6; i++ {\n\t\t\tres := node.IPFS(\"files\", \"mkdir\", \"--flush=false\", fmt.Sprintf(\"/dir%d\", i))\n\t\t\tassert.NoError(t, res.Err, \"operation after flush should succeed\")\n\t\t}\n\n\t\t// 4th operation after reset should fail\n\t\tres := node.RunIPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir7\")\n\t\trequire.NotNil(t, res.ExitErr, \"command should have failed\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"reached limit of 3 unflushed MFS operations\")\n\t})\n\n\tt.Run(\"explicit flush command resets counter\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Set limit to 3 for faster testing\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tlimit := config.NewOptionalInteger(3)\n\t\t\tcfg.Internal.MFSNoFlushLimit = limit\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Do 2 operations with --flush=false\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir1\")\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir2\")\n\n\t\t// Explicit flush should reset counter\n\t\tnode.IPFS(\"files\", \"flush\")\n\n\t\t// Now we should be able to do 3 more operations\n\t\tfor i := 3; i <= 5; i++ {\n\t\t\tres := node.IPFS(\"files\", \"mkdir\", \"--flush=false\", fmt.Sprintf(\"/dir%d\", i))\n\t\t\tassert.NoError(t, res.Err, \"operation after flush should succeed\")\n\t\t}\n\n\t\t// 4th operation should fail\n\t\tres := node.RunIPFS(\"files\", \"mkdir\", \"--flush=false\", \"/dir6\")\n\t\trequire.NotNil(t, res.ExitErr, \"command should have failed\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"reached limit of 3 unflushed MFS operations\")\n\t})\n\n\tt.Run(\"limit=0 disables the feature\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Set limit to 0 (disabled)\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tlimit := config.NewOptionalInteger(0)\n\t\t\tcfg.Internal.MFSNoFlushLimit = limit\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Should be able to do many operations without error\n\t\tfor i := range 300 {\n\t\t\tres := node.IPFS(\"files\", \"mkdir\", \"--flush=false\", fmt.Sprintf(\"/dir%d\", i))\n\t\t\tassert.NoError(t, res.Err, \"operation %d should succeed with limit disabled\", i+1)\n\t\t}\n\t})\n\n\tt.Run(\"different MFS commands count towards limit\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Set limit to 5 for testing\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tlimit := config.NewOptionalInteger(5)\n\t\t\tcfg.Internal.MFSNoFlushLimit = limit\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Mix of different MFS operations (5 operations to hit the limit)\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=false\", \"/testdir\")\n\t\t// Create a file first, then copy it\n\t\ttestCid := node.IPFSAddStr(\"test content\")\n\t\tnode.IPFS(\"files\", \"cp\", \"--flush=false\", fmt.Sprintf(\"/ipfs/%s\", testCid), \"/testfile\")\n\t\tnode.IPFS(\"files\", \"cp\", \"--flush=false\", \"/testfile\", \"/testfile2\")\n\t\tnode.IPFS(\"files\", \"mv\", \"--flush=false\", \"/testfile2\", \"/testfile3\")\n\t\tnode.IPFS(\"files\", \"mkdir\", \"--flush=false\", \"/anotherdir\")\n\n\t\t// 6th operation should fail\n\t\tres := node.RunIPFS(\"files\", \"mkdir\", \"--flush=false\", \"/another\")\n\t\trequire.NotNil(t, res.ExitErr, \"command should have failed\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"reached limit of 5 unflushed MFS operations\")\n\t})\n}\n\nfunc TestFilesChroot(t *testing.T) {\n\tt.Parallel()\n\n\t// Known CIDs for testing\n\temptyDirCid := \"QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\"\n\n\tt.Run(\"requires --confirm flag\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t// Don't start daemon - chroot runs offline\n\n\t\tres := node.RunIPFS(\"files\", \"chroot\")\n\t\trequire.NotNil(t, res.ExitErr)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"pass --confirm to proceed\")\n\t})\n\n\tt.Run(\"resets to empty directory\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Start daemon to create MFS state\n\t\tnode.StartDaemon()\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\t\tnode.StopDaemon()\n\n\t\t// Reset MFS to empty - should exit 0\n\t\tres := node.RunIPFS(\"files\", \"chroot\", \"--confirm\")\n\t\tassert.Nil(t, res.ExitErr, \"expected exit code 0\")\n\t\tassert.Contains(t, res.Stdout.String(), emptyDirCid)\n\n\t\t// Verify daemon starts and MFS is empty\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tlsRes := node.IPFS(\"files\", \"ls\", \"/\")\n\t\tassert.Empty(t, lsRes.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"replaces with valid directory CID\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Start daemon to add content\n\t\tnode.StartDaemon()\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/mydir\")\n\t\t// Create a temp file for content\n\t\ttempFile := filepath.Join(node.Dir, \"testfile.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(\"hello\"), 0644))\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/mydir/file.txt\", tempFile)\n\t\tstatRes := node.IPFS(\"files\", \"stat\", \"--hash\", \"/mydir\")\n\t\tdirCid := statRes.Stdout.Trimmed()\n\t\tnode.StopDaemon()\n\n\t\t// Reset to empty first\n\t\tnode.IPFS(\"files\", \"chroot\", \"--confirm\")\n\n\t\t// Set root to the saved directory - should exit 0\n\t\tres := node.RunIPFS(\"files\", \"chroot\", \"--confirm\", dirCid)\n\t\tassert.Nil(t, res.ExitErr, \"expected exit code 0\")\n\t\tassert.Contains(t, res.Stdout.String(), dirCid)\n\n\t\t// Verify content\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\treadRes := node.IPFS(\"files\", \"read\", \"/file.txt\")\n\t\tassert.Equal(t, \"hello\", readRes.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"fails with non-existent CID\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tres := node.RunIPFS(\"files\", \"chroot\", \"--confirm\", \"bafybeibdxtd5thfoitjmnfhxhywokebwdmwnuqgkzjjdjhwjz7qh77777a\")\n\t\trequire.NotNil(t, res.ExitErr)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"does not exist locally\")\n\t})\n\n\tt.Run(\"fails with file CID\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Add a file to get a file CID\n\t\tnode.StartDaemon()\n\t\tfileCid := node.IPFSAddStr(\"hello world\")\n\t\tnode.StopDaemon()\n\n\t\t// Try to set file as root - should fail with non-zero exit\n\t\tres := node.RunIPFS(\"files\", \"chroot\", \"--confirm\", fileCid)\n\t\trequire.NotNil(t, res.ExitErr)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"must be a directory\")\n\t})\n\n\tt.Run(\"fails while daemon is running\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"files\", \"chroot\", \"--confirm\")\n\t\trequire.NotNil(t, res.ExitErr)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"opening repo\")\n\t})\n}\n\n// TestFilesMFSImportConfig tests that MFS operations respect Import.* configuration settings.\n// These tests verify that `ipfs files` commands use the same import settings as `ipfs add`.\nfunc TestFilesMFSImportConfig(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"files write respects Import.CidVersion=1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Write file via MFS\n\t\ttempFile := filepath.Join(node.Dir, \"test.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(\"hello\"), 0644))\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/test.txt\", tempFile)\n\n\t\t// Get CID of written file\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/test.txt\").Stdout.Trimmed()\n\n\t\t// Verify CIDv1 format (base32, starts with \"b\")\n\t\trequire.True(t, strings.HasPrefix(cidStr, \"b\"), \"expected CIDv1 (starts with b), got: %s\", cidStr)\n\t})\n\n\tt.Run(\"files write respects Import.UnixFSRawLeaves=true\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\tcfg.Import.UnixFSRawLeaves = config.True\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\ttempFile := filepath.Join(node.Dir, \"test.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(\"hello world\"), 0644))\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/test.txt\", tempFile)\n\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/test.txt\").Stdout.Trimmed()\n\t\tcodec := node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", cidStr).Stdout.Trimmed()\n\t\trequire.Equal(t, \"raw\", codec, \"expected raw codec for small file with raw leaves\")\n\t})\n\n\t// This test verifies CID parity for single-block files only.\n\t// Multi-block files will have different CIDs because MFS uses trickle DAG layout\n\t// while 'ipfs add' uses balanced DAG layout. See \"files write vs add for multi-block\" test.\n\tt.Run(\"single-block file: files write produces same CID as ipfs add\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\tcfg.Import.UnixFSRawLeaves = config.True\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\ttempFile := filepath.Join(node.Dir, \"test.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(\"hello world\"), 0644))\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/test.txt\", tempFile)\n\n\t\tmfsCid := node.IPFS(\"files\", \"stat\", \"--hash\", \"/test.txt\").Stdout.Trimmed()\n\t\taddCid := node.IPFSAddStr(\"hello world\")\n\t\trequire.Equal(t, addCid, mfsCid, \"MFS write should produce same CID as ipfs add for single-block files\")\n\t})\n\n\tt.Run(\"files mkdir respects Import.CidVersion=1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\n\t\t// Verify CIDv1 format\n\t\trequire.True(t, strings.HasPrefix(cidStr, \"b\"), \"expected CIDv1 (starts with b), got: %s\", cidStr)\n\t})\n\n\tt.Run(\"MFS subdirectory becomes HAMT when exceeding threshold\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// Use small threshold for faster testing\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = *config.NewOptionalBytes(\"1KiB\")\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"block\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/bigdir\")\n\n\t\tcontent := \"x\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\n\t\t// Add enough files to exceed 1KiB threshold\n\t\tfor i := range 25 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/bigdir/file%02d\", i), tempFile)\n\t\t}\n\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/bigdir\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"expected HAMT directory\")\n\t})\n\n\tt.Run(\"MFS root directory becomes HAMT when exceeding threshold\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = *config.NewOptionalBytes(\"1KiB\")\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"block\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcontent := \"x\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\n\t\t// Add files directly to root /\n\t\tfor i := range 25 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/file%02d\", i), tempFile)\n\t\t}\n\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"expected MFS root to become HAMT\")\n\t})\n\n\tt.Run(\"MFS directory reverts from HAMT to basic when items removed\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = *config.NewOptionalBytes(\"1KiB\")\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"block\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\n\t\tcontent := \"x\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\n\t\t// Add files to exceed threshold\n\t\tfor i := range 25 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/testdir/file%02d\", i), tempFile)\n\t\t}\n\n\t\t// Verify it became HAMT\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"should be HAMT after adding many files\")\n\n\t\t// Remove files to get back below threshold\n\t\tfor i := range 20 {\n\t\t\tnode.IPFS(\"files\", \"rm\", fmt.Sprintf(\"/testdir/file%02d\", i))\n\t\t}\n\n\t\t// Verify it reverted to basic directory\n\t\tcidStr = node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err = node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.TDirectory, fsType, \"should revert to basic directory after removing files\")\n\t})\n\n\t// Note: 'files write' produces DIFFERENT CIDs than 'ipfs add' for multi-block files because\n\t// MFS uses trickle DAG layout while 'ipfs add' uses balanced DAG layout.\n\t// Single-block files produce the same CID (tested above in \"single-block file: files write...\").\n\t// For multi-block CID compatibility with 'ipfs add', use 'ipfs add --to-files' instead.\n\n\tt.Run(\"files cp preserves original CID\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\tcfg.Import.UnixFSRawLeaves = config.True\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Add file via ipfs add\n\t\toriginalCid := node.IPFSAddStr(\"hello world\")\n\n\t\t// Copy to MFS\n\t\tnode.IPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", originalCid), \"/copied.txt\")\n\n\t\t// Verify CID is preserved\n\t\tmfsCid := node.IPFS(\"files\", \"stat\", \"--hash\", \"/copied.txt\").Stdout.Trimmed()\n\t\trequire.Equal(t, originalCid, mfsCid, \"files cp should preserve original CID\")\n\t})\n\n\tt.Run(\"add --to-files respects Import config\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\tcfg.Import.UnixFSRawLeaves = config.True\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create temp file\n\t\ttempFile := filepath.Join(node.Dir, \"test.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(\"hello world\"), 0644))\n\n\t\t// Add with --to-files\n\t\taddCid := node.IPFS(\"add\", \"-Q\", \"--to-files=/added.txt\", tempFile).Stdout.Trimmed()\n\n\t\t// Verify MFS file has same CID\n\t\tmfsCid := node.IPFS(\"files\", \"stat\", \"--hash\", \"/added.txt\").Stdout.Trimmed()\n\t\trequire.Equal(t, addCid, mfsCid)\n\n\t\t// Should be CIDv1 raw leaf\n\t\tcodec := node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", mfsCid).Stdout.Trimmed()\n\t\trequire.Equal(t, \"raw\", codec)\n\t})\n\n\tt.Run(\"files mkdir respects Import.UnixFSDirectoryMaxLinks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\t// Set low link threshold to trigger HAMT sharding at 5 links\n\t\t\tcfg.Import.UnixFSDirectoryMaxLinks = *config.NewOptionalInteger(5)\n\t\t\t// Also need size estimation enabled for switching to work\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"block\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create directory with 6 files (exceeds max 5 links)\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\n\t\tcontent := \"x\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\n\t\tfor i := range 6 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/testdir/file%d.txt\", i), tempFile)\n\t\t}\n\n\t\t// Verify directory became HAMT sharded\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"expected HAMT directory after exceeding UnixFSDirectoryMaxLinks\")\n\t})\n\n\tt.Run(\"files write respects Import.UnixFSChunker\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\tcfg.Import.UnixFSRawLeaves = config.True\n\t\t\tcfg.Import.UnixFSChunker = *config.NewOptionalString(\"size-1024\") // 1KB chunks\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create file larger than chunk size (3KB)\n\t\tdata := make([]byte, 3*1024)\n\t\tfor i := range data {\n\t\t\tdata[i] = byte(i % 256)\n\t\t}\n\t\ttempFile := filepath.Join(node.Dir, \"large.bin\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, data, 0644))\n\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/large.bin\", tempFile)\n\n\t\t// Verify chunking: 3KB file with 1KB chunks should have multiple child blocks\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/large.bin\").Stdout.Trimmed()\n\t\tdagStatJSON := node.IPFS(\"dag\", \"stat\", \"--enc=json\", cidStr).Stdout.Trimmed()\n\t\tvar dagStat struct {\n\t\t\tUniqueBlocks int `json:\"UniqueBlocks\"`\n\t\t}\n\t\trequire.NoError(t, json.Unmarshal([]byte(dagStatJSON), &dagStat))\n\t\t// With 1KB chunks on a 3KB file, we expect 4 blocks (3 leaf + 1 root)\n\t\tassert.Greater(t, dagStat.UniqueBlocks, 1, \"expected more than 1 block with 1KB chunker on 3KB file\")\n\t})\n\n\tt.Run(\"files write with custom chunker produces same CID as ipfs add --trickle\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.CidVersion = *config.NewOptionalInteger(1)\n\t\t\tcfg.Import.UnixFSRawLeaves = config.True\n\t\t\tcfg.Import.UnixFSChunker = *config.NewOptionalString(\"size-512\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create test data (2KB to get multiple chunks)\n\t\tdata := make([]byte, 2048)\n\t\tfor i := range data {\n\t\t\tdata[i] = byte(i % 256)\n\t\t}\n\t\ttempFile := filepath.Join(node.Dir, \"test.bin\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, data, 0644))\n\n\t\t// Add via MFS\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/test.bin\", tempFile)\n\t\tmfsCid := node.IPFS(\"files\", \"stat\", \"--hash\", \"/test.bin\").Stdout.Trimmed()\n\n\t\t// Add via ipfs add with same chunker and trickle (MFS always uses trickle)\n\t\taddCid := node.IPFS(\"add\", \"-Q\", \"--chunker=size-512\", \"--trickle\", tempFile).Stdout.Trimmed()\n\n\t\t// CIDs should match when using same chunker + trickle layout\n\t\trequire.Equal(t, addCid, mfsCid, \"MFS and add --trickle should produce same CID with matching chunker\")\n\t})\n\n\tt.Run(\"files mkdir respects Import.UnixFSHAMTDirectoryMaxFanout\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// Use non-default fanout of 64 (default is 256)\n\t\t\tcfg.Import.UnixFSHAMTDirectoryMaxFanout = *config.NewOptionalInteger(64)\n\t\t\t// Set low link threshold to trigger HAMT at 5 links\n\t\t\tcfg.Import.UnixFSDirectoryMaxLinks = *config.NewOptionalInteger(5)\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"disabled\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\n\t\tcontent := \"x\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\n\t\t// Add 6 files (exceeds MaxLinks=5) to trigger HAMT\n\t\tfor i := range 6 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/testdir/file%d.txt\", i), tempFile)\n\t\t}\n\n\t\t// Verify directory became HAMT\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"expected HAMT directory\")\n\n\t\t// Verify the HAMT uses the custom fanout (64) by inspecting the UnixFS Data field.\n\t\tfanout, err := node.UnixFSHAMTFanout(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, uint64(64), fanout, \"expected HAMT fanout 64\")\n\t})\n\n\tt.Run(\"files mkdir respects Import.UnixFSHAMTDirectorySizeThreshold\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// Use very small threshold (100 bytes) to trigger HAMT quickly\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = *config.NewOptionalBytes(\"100B\")\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"block\")\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\n\t\tcontent := \"test content\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\n\t\t// Add 3 files - each link adds ~40-50 bytes, so 3 should exceed 100B threshold\n\t\tfor i := range 3 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/testdir/file%d.txt\", i), tempFile)\n\t\t}\n\n\t\t// Verify directory became HAMT due to size threshold\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"expected HAMT directory after exceeding size threshold\")\n\t})\n\n\tt.Run(\"config change takes effect after daemon restart\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Start with high threshold (won't trigger HAMT)\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = *config.NewOptionalBytes(\"256KiB\")\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeEstimation = *config.NewOptionalString(\"block\")\n\t\t})\n\t\tnode.StartDaemon()\n\n\t\t// Create directory with some files\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\t\tcontent := \"test\"\n\t\ttempFile := filepath.Join(node.Dir, \"content.txt\")\n\t\trequire.NoError(t, os.WriteFile(tempFile, []byte(content), 0644))\n\t\tfor i := range 3 {\n\t\t\tnode.IPFS(\"files\", \"write\", \"--create\", fmt.Sprintf(\"/testdir/file%d.txt\", i), tempFile)\n\t\t}\n\n\t\t// Verify it's still a basic directory (threshold not exceeded)\n\t\tcidStr := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err := node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.TDirectory, fsType, \"should be basic directory with high threshold\")\n\n\t\t// Stop daemon\n\t\tnode.StopDaemon()\n\n\t\t// Change config to use very low threshold\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Import.UnixFSHAMTDirectorySizeThreshold = *config.NewOptionalBytes(\"100B\")\n\t\t})\n\n\t\t// Restart daemon\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Add one more file - this should trigger HAMT conversion with new threshold\n\t\tnode.IPFS(\"files\", \"write\", \"--create\", \"/testdir/file3.txt\", tempFile)\n\n\t\t// Verify it became HAMT (new threshold applied)\n\t\tcidStr = node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\").Stdout.Trimmed()\n\t\tfsType, err = node.UnixFSDataType(cidStr)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, ft.THAMTShard, fsType, \"should be HAMT after daemon restart with lower threshold\")\n\t})\n}\n" }, { "path": "test/cli/fixtures/README.md", "content": "# Dataset Description / Sources\n\nTestGatewayHAMTDirectory.car generated with:\n\n```bash\nipfs version\n# ipfs version 0.19.0\n\nexport HAMT_DIR=bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm\nexport IPFS_PATH=$(mktemp -d)\n\n# Init and start daemon, ensure we have an empty repository.\nipfs init --empty-repo\nipfs daemon &> /dev/null &\nexport IPFS_PID=$!\n\n# Retrieve the directory listing, forcing the daemon to download all required DAGs. Kill daemon.\ncurl -o dir.html http://127.0.0.1:8080/ipfs/$HAMT_DIR/\nkill $IPFS_PID\n\n# Get the list with all the downloaded refs and sanity check.\nipfs refs local > required_refs\ncat required_refs | wc -l\n# 962\n\n# Get the list of all the files CIDs inside the directory and sanity check.\ncat dir.html| pup '#content tbody .ipfs-hash attr{href}' | sed 's/\\/ipfs\\///g;s/\\?filename=.*//g' > files_refs\ncat files_refs | wc -l\n# 10100\n\n# Make and export our fixture.\nipfs files mkdir --cid-version 1 /fixtures\ncat required_refs | xargs -I {} ipfs files cp /ipfs/{} /fixtures/{}\ncat files_refs | ipfs files write --create /fixtures/files_refs\nexport FIXTURE_CID=$(ipfs files stat --hash /fixtures/)\necho $FIXTURE_CID\n# bafybeig3yoibxe56aolixqa4zk55gp5sug3qgaztkakpndzk2b2ynobd4i\nipfs dag export $FIXTURE_CID > TestGatewayHAMTDirectory.car\n```\n\nTestGatewayMultiRange.car generated with:\n\n\n```sh\nipfs version\n# ipfs version 0.19.0\n\nexport FILE_CID=bafybeiae5abzv6j3ucqbzlpnx3pcqbr2otbnpot7d2k5pckmpymin4guau\nexport IPFS_PATH=$(mktemp -d)\n\n# Init and start daemon, ensure we have an empty repository.\nipfs init --empty-repo\nipfs daemon &> /dev/null &\nexport IPFS_PID=$!\n\n# Get a specific byte range from the file. \ncurl http://127.0.0.1:8080/ipfs/$FILE_CID -i -H \"Range: bytes=1276-1279, 29839070-29839080\"\nkill $IPFS_PID\n\n# Get the list with all the downloaded refs and sanity check.\nipfs refs local > required_refs\ncat required_refs | wc -l\n# 19\n\n# Make and export our fixture.\nipfs files mkdir --cid-version 1 /fixtures\ncat required_refs | xargs -I {} ipfs files cp /ipfs/{} /fixtures/{}\nexport FIXTURE_CID=$(ipfs files stat --hash /fixtures/)\necho $FIXTURE_CID\n# bafybeicgsg3lwyn3yl75lw7sn4zhyj5dxtb7wfxwscpq6yzippetmr2w3y\nipfs dag export $FIXTURE_CID > TestGatewayMultiRange.car\n```\n" }, { "path": "test/cli/fixtures/TestDagStatExpectedOutput.txt", "content": "\nCID \tBlocks \tSize\nbafyreibmdfd7c5db4kls4ty57zljfhqv36gi43l6txl44pi423wwmeskwy\t2 \t53\nbafyreie3njilzdi4ixumru4nzgecsnjtu7fzfcwhg7e6s4s5i7cnbslvn4\t2 \t53\n\nSummary\nTotal Size: 99 (99 B)\nUnique Blocks: 3\nShared Size: 7 (7 B)\nRatio: 1.070707\n\n\n" }, { "path": "test/cli/fuse_test.go", "content": "package cli\n\nimport (\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestFUSE(t *testing.T) {\n\ttestutils.RequiresFUSE(t)\n\tt.Parallel()\n\n\tt.Run(\"mount and unmount work correctly\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a node and start daemon\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.StartDaemon()\n\n\t\t// Create mount directories in the node's working directory\n\t\tnodeDir := node.Dir\n\t\tipfsMount := filepath.Join(nodeDir, \"ipfs\")\n\t\tipnsMount := filepath.Join(nodeDir, \"ipns\")\n\t\tmfsMount := filepath.Join(nodeDir, \"mfs\")\n\n\t\terr := os.MkdirAll(ipfsMount, 0755)\n\t\trequire.NoError(t, err)\n\t\terr = os.MkdirAll(ipnsMount, 0755)\n\t\trequire.NoError(t, err)\n\t\terr = os.MkdirAll(mfsMount, 0755)\n\t\trequire.NoError(t, err)\n\n\t\t// Ensure any existing mounts are cleaned up first\n\t\tfailOnError := false // mount points might not exist from previous runs\n\t\tdoUnmount(t, ipfsMount, failOnError)\n\t\tdoUnmount(t, ipnsMount, failOnError)\n\t\tdoUnmount(t, mfsMount, failOnError)\n\n\t\t// Test mount operation\n\t\tresult := node.IPFS(\"mount\", \"-f\", ipfsMount, \"-n\", ipnsMount, \"-m\", mfsMount)\n\n\t\t// Verify mount output\n\t\texpectedOutput := \"IPFS mounted at: \" + ipfsMount + \"\\n\" +\n\t\t\t\"IPNS mounted at: \" + ipnsMount + \"\\n\" +\n\t\t\t\"MFS mounted at: \" + mfsMount + \"\\n\"\n\t\trequire.Equal(t, expectedOutput, result.Stdout.String())\n\n\t\t// Test basic MFS functionality via FUSE mount\n\t\ttestFile := filepath.Join(mfsMount, \"testfile\")\n\t\ttestContent := \"hello fuse world\"\n\n\t\t// Create file via FUSE mount\n\t\terr = os.WriteFile(testFile, []byte(testContent), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify file appears in MFS via IPFS commands\n\t\tresult = node.IPFS(\"files\", \"ls\", \"/\")\n\t\trequire.Contains(t, result.Stdout.String(), \"testfile\")\n\n\t\t// Read content back via MFS FUSE mount\n\t\treadContent, err := os.ReadFile(testFile)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, testContent, string(readContent))\n\n\t\t// Get the CID of the MFS file\n\t\tresult = node.IPFS(\"files\", \"stat\", \"/testfile\", \"--format=\")\n\t\tfileCID := strings.TrimSpace(result.Stdout.String())\n\t\trequire.NotEmpty(t, fileCID, \"should have a CID for the MFS file\")\n\n\t\t// Read the same content via IPFS FUSE mount using the CID\n\t\tipfsFile := filepath.Join(ipfsMount, fileCID)\n\t\tipfsContent, err := os.ReadFile(ipfsFile)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, testContent, string(ipfsContent), \"content should match between MFS and IPFS mounts\")\n\n\t\t// Verify both FUSE mounts return identical data\n\t\trequire.Equal(t, readContent, ipfsContent, \"MFS and IPFS FUSE mounts should return identical data\")\n\n\t\t// Test that mount directories cannot be removed while mounted\n\t\terr = os.Remove(ipfsMount)\n\t\trequire.Error(t, err, \"should not be able to remove mounted directory\")\n\n\t\t// Stop daemon - this should trigger automatic unmount via context cancellation\n\t\tnode.StopDaemon()\n\n\t\t// Daemon shutdown should handle unmount synchronously via context.AfterFunc\n\n\t\t// Verify directories can now be removed (indicating successful unmount)\n\t\terr = os.Remove(ipfsMount)\n\t\trequire.NoError(t, err, \"should be able to remove directory after unmount\")\n\t\terr = os.Remove(ipnsMount)\n\t\trequire.NoError(t, err, \"should be able to remove directory after unmount\")\n\t\terr = os.Remove(mfsMount)\n\t\trequire.NoError(t, err, \"should be able to remove directory after unmount\")\n\t})\n\n\tt.Run(\"explicit unmount works\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.StartDaemon()\n\n\t\t// Create mount directories\n\t\tnodeDir := node.Dir\n\t\tipfsMount := filepath.Join(nodeDir, \"ipfs\")\n\t\tipnsMount := filepath.Join(nodeDir, \"ipns\")\n\t\tmfsMount := filepath.Join(nodeDir, \"mfs\")\n\n\t\terr := os.MkdirAll(ipfsMount, 0755)\n\t\trequire.NoError(t, err)\n\t\terr = os.MkdirAll(ipnsMount, 0755)\n\t\trequire.NoError(t, err)\n\t\terr = os.MkdirAll(mfsMount, 0755)\n\t\trequire.NoError(t, err)\n\n\t\t// Clean up any existing mounts\n\t\tfailOnError := false // mount points might not exist from previous runs\n\t\tdoUnmount(t, ipfsMount, failOnError)\n\t\tdoUnmount(t, ipnsMount, failOnError)\n\t\tdoUnmount(t, mfsMount, failOnError)\n\n\t\t// Mount\n\t\tnode.IPFS(\"mount\", \"-f\", ipfsMount, \"-n\", ipnsMount, \"-m\", mfsMount)\n\n\t\t// Explicit unmount via platform-specific command\n\t\tfailOnError = true // test that explicit unmount works correctly\n\t\tdoUnmount(t, ipfsMount, failOnError)\n\t\tdoUnmount(t, ipnsMount, failOnError)\n\t\tdoUnmount(t, mfsMount, failOnError)\n\n\t\t// Verify directories can be removed after explicit unmount\n\t\terr = os.Remove(ipfsMount)\n\t\trequire.NoError(t, err)\n\t\terr = os.Remove(ipnsMount)\n\t\trequire.NoError(t, err)\n\t\terr = os.Remove(mfsMount)\n\t\trequire.NoError(t, err)\n\n\t\tnode.StopDaemon()\n\t})\n}\n\n// doUnmount performs platform-specific unmount, similar to sharness do_umount\n// failOnError: if true, unmount errors cause test failure; if false, errors are ignored (useful for cleanup)\nfunc doUnmount(t *testing.T, mountPoint string, failOnError bool) {\n\tt.Helper()\n\tvar cmd *exec.Cmd\n\tif runtime.GOOS == \"linux\" {\n\t\t// fusermount -u: unmount filesystem (strict - fails if busy)\n\t\tcmd = exec.Command(\"fusermount\", \"-u\", mountPoint)\n\t} else {\n\t\tcmd = exec.Command(\"umount\", mountPoint)\n\t}\n\n\terr := cmd.Run()\n\tif err != nil && failOnError {\n\t\tt.Fatalf(\"failed to unmount %s: %v\", mountPoint, err)\n\t}\n}\n" }, { "path": "test/cli/gateway_limits_test.go", "content": "package cli\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\n// TestGatewayLimits tests the gateway request limiting and timeout features.\n// These are basic integration tests that verify the configuration works.\n// For comprehensive tests, see:\n// - github.com/ipfs/boxo/gateway/middleware_retrieval_timeout_test.go\n// - github.com/ipfs/boxo/gateway/middleware_ratelimit_test.go\nfunc TestGatewayLimits(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"RetrievalTimeout\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a node with a short retrieval timeout\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// Set a 1 second timeout for retrieval\n\t\t\tcfg.Gateway.RetrievalTimeout = config.NewOptionalDuration(1 * time.Second)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Add content that can be retrieved quickly\n\t\tcid := node.IPFSAddStr(\"test content\")\n\n\t\tclient := node.GatewayClient()\n\n\t\t// Normal request should succeed (content is local)\n\t\tresp := client.Get(\"/ipfs/\" + cid)\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tassert.Equal(t, \"test content\", resp.Body)\n\n\t\t// Request for non-existent content should timeout\n\t\t// Using a CID that has no providers (generated with ipfs add -n)\n\t\tnonExistentCID := \"bafkreif6lrhgz3fpiwypdk65qrqiey7svgpggruhbylrgv32l3izkqpsc4\"\n\n\t\t// Create a client with longer timeout than the gateway's retrieval timeout\n\t\t// to ensure we get the gateway's 504 response\n\t\tclientWithTimeout := &harness.HTTPClient{\n\t\t\tClient: &http.Client{\n\t\t\t\tTimeout: 5 * time.Second,\n\t\t\t},\n\t\t\tBaseURL: client.BaseURL,\n\t\t}\n\n\t\tresp = clientWithTimeout.Get(\"/ipfs/\" + nonExistentCID)\n\t\tassert.Equal(t, http.StatusGatewayTimeout, resp.StatusCode, \"Expected 504 Gateway Timeout for stuck retrieval\")\n\t\tassert.Contains(t, resp.Body, \"Unable to retrieve content within timeout period\")\n\t})\n\n\tt.Run(\"MaxRequestDuration\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a node with a short max request duration\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// Set a short absolute deadline (500ms) for the entire request\n\t\t\tcfg.Gateway.MaxRequestDuration = config.NewOptionalDuration(500 * time.Millisecond)\n\t\t\t// Set retrieval timeout much longer so MaxRequestDuration fires first\n\t\t\tcfg.Gateway.RetrievalTimeout = config.NewOptionalDuration(30 * time.Second)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Add content that can be retrieved quickly\n\t\tcid := node.IPFSAddStr(\"test content for max request duration\")\n\n\t\tclient := node.GatewayClient()\n\n\t\t// Fast request for local content should succeed (well within 500ms)\n\t\tresp := client.Get(\"/ipfs/\" + cid)\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tassert.Equal(t, \"test content for max request duration\", resp.Body)\n\n\t\t// Request for non-existent content should timeout due to MaxRequestDuration\n\t\t// This CID has no providers and will block during content routing\n\t\tnonExistentCID := \"bafkreif6lrhgz3fpiwypdk65qrqiey7svgpggruhbylrgv32l3izkqpsc4\"\n\n\t\t// Create a client with a longer timeout than MaxRequestDuration\n\t\t// to ensure we receive the gateway's 504 response\n\t\tclientWithTimeout := &harness.HTTPClient{\n\t\t\tClient: &http.Client{\n\t\t\t\tTimeout: 5 * time.Second,\n\t\t\t},\n\t\t\tBaseURL: client.BaseURL,\n\t\t}\n\n\t\tresp = clientWithTimeout.Get(\"/ipfs/\" + nonExistentCID)\n\t\tassert.Equal(t, http.StatusGatewayTimeout, resp.StatusCode, \"Expected 504 when request exceeds MaxRequestDuration\")\n\t})\n\n\tt.Run(\"MaxConcurrentRequests\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a node with a low concurrent request limit\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// Allow only 1 concurrent request to make test deterministic\n\t\t\tcfg.Gateway.MaxConcurrentRequests = config.NewOptionalInteger(1)\n\t\t\t// Set retrieval timeout so blocking requests don't hang forever\n\t\t\tcfg.Gateway.RetrievalTimeout = config.NewOptionalDuration(2 * time.Second)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Add some content - use a non-existent CID that will block during retrieval\n\t\t// to ensure we can control timing\n\t\tblockingCID := \"bafkreif6lrhgz3fpiwypdk65qrqiey7svgpggruhbylrgv32l3izkqpsc4\"\n\t\tnormalCID := node.IPFSAddStr(\"test content for concurrent request limiting\")\n\n\t\tclient := node.GatewayClient()\n\n\t\t// First, verify single request succeeds\n\t\tresp := client.Get(\"/ipfs/\" + normalCID)\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\t// Now test deterministic 429 response:\n\t\t// Start a blocking request that will occupy the single slot,\n\t\t// then make another request that MUST get 429\n\n\t\tblockingStarted := make(chan bool)\n\t\tblockingDone := make(chan bool)\n\n\t\t// Start a request that will block (searching for non-existent content)\n\t\tgo func() {\n\t\t\tblockingStarted <- true\n\t\t\t// This will block until timeout looking for providers\n\t\t\tclient.Get(\"/ipfs/\" + blockingCID)\n\t\t\tblockingDone <- true\n\t\t}()\n\n\t\t// Wait for blocking request to start and occupy the slot\n\t\t<-blockingStarted\n\t\ttime.Sleep(1 * time.Second) // Ensure it has acquired the semaphore\n\n\t\t// This request MUST get 429 because the slot is occupied\n\t\tresp = client.Get(\"/ipfs/\" + normalCID + \"?must-get-429=true\")\n\t\tassert.Equal(t, http.StatusTooManyRequests, resp.StatusCode, \"Second request must get 429 when slot is occupied\")\n\n\t\t// Verify 429 response headers\n\t\tretryAfter := resp.Headers.Get(\"Retry-After\")\n\t\tassert.NotEmpty(t, retryAfter, \"Retry-After header must be set on 429 response\")\n\t\tassert.Equal(t, \"60\", retryAfter, \"Retry-After must be 60 seconds\")\n\n\t\tcacheControl := resp.Headers.Get(\"Cache-Control\")\n\t\tassert.Equal(t, \"no-store\", cacheControl, \"Cache-Control must be no-store on 429 response\")\n\n\t\tassert.Contains(t, resp.Body, \"Too many requests\", \"429 response must contain error message\")\n\n\t\t// Clean up: wait for blocking request to timeout (it will timeout due to gateway retrieval timeout)\n\t\tselect {\n\t\tcase <-blockingDone:\n\t\t\t// Good, it completed\n\t\tcase <-time.After(10 * time.Second):\n\t\t\t// Give it more time if needed\n\t\t}\n\n\t\t// Wait a bit more to ensure slot is fully released\n\t\ttime.Sleep(1 * time.Second)\n\n\t\t// After blocking request completes, new request should succeed\n\t\tresp = client.Get(\"/ipfs/\" + normalCID + \"?after-limit-cleared=true\")\n\t\tassert.Equal(t, http.StatusOK, resp.StatusCode, \"Request must succeed after slot is freed\")\n\t})\n}\n" }, { "path": "test/cli/gateway_range_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"net/http\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestGatewayHAMTDirectory(t *testing.T) {\n\tt.Parallel()\n\n\tconst (\n\t\t// The CID of the HAMT-sharded directory that has 10k items\n\t\thamtCid = \"bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm\"\n\n\t\t// fixtureCid is the CID of root of the DAG that is a subset of hamtCid DAG\n\t\t// representing the minimal set of blocks necessary for directory listing.\n\t\t// It also includes a \"files_refs\" file with the list of the references\n\t\t// we do NOT needs to fetch (files inside the directory)\n\t\tfixtureCid = \"bafybeig3yoibxe56aolixqa4zk55gp5sug3qgaztkakpndzk2b2ynobd4i\"\n\t)\n\n\t// Start node\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init(\"--empty-repo\", \"--profile=test\").StartDaemon(\"--offline\")\n\tdefer node.StopDaemon()\n\tclient := node.GatewayClient()\n\n\t// Import fixtures\n\tr, err := os.Open(\"./fixtures/TestGatewayHAMTDirectory.car\")\n\tassert.NoError(t, err)\n\tdefer r.Close()\n\terr = node.IPFSDagImport(r, fixtureCid)\n\tassert.NoError(t, err)\n\n\t// Fetch HAMT directory succeeds with minimal refs\n\tresp := client.Get(fmt.Sprintf(\"/ipfs/%s/\", hamtCid))\n\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n}\n\nfunc TestGatewayHAMTRanges(t *testing.T) {\n\tt.Parallel()\n\n\tconst (\n\t\t// fileCid is the CID of the large HAMT-sharded file.\n\t\tfileCid = \"bafybeiae5abzv6j3ucqbzlpnx3pcqbr2otbnpot7d2k5pckmpymin4guau\"\n\n\t\t// fixtureCid is the CID of root of the DAG that is a subset of fileCid DAG\n\t\t// representing the minimal set of blocks necessary for a simple byte range request.\n\t\tfixtureCid = \"bafybeicgsg3lwyn3yl75lw7sn4zhyj5dxtb7wfxwscpq6yzippetmr2w3y\"\n\t)\n\n\t// Start node\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init(\"--empty-repo\", \"--profile=test\").StartDaemon(\"--offline\")\n\tt.Cleanup(func() { node.StopDaemon() })\n\tclient := node.GatewayClient()\n\n\t// Import fixtures\n\tr, err := os.Open(\"./fixtures/TestGatewayMultiRange.car\")\n\tassert.NoError(t, err)\n\tdefer r.Close()\n\terr = node.IPFSDagImport(r, fixtureCid)\n\tassert.NoError(t, err)\n\n\tt.Run(\"Succeeds Fetching Range\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tresp := client.Get(fmt.Sprintf(\"/ipfs/%s\", fileCid), func(r *http.Request) {\n\t\t\tr.Header.Set(\"Range\", \"bytes=1276-1279\")\n\t\t})\n\t\tassert.Equal(t, http.StatusPartialContent, resp.StatusCode)\n\t\tassert.Equal(t, \"bytes 1276-1279/109266405\", resp.Headers.Get(\"Content-Range\"))\n\t\tassert.Equal(t, \"iana\", resp.Body)\n\t})\n\n\tt.Run(\"Succeeds Fetching Second Range\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tresp := client.Get(fmt.Sprintf(\"/ipfs/%s\", fileCid), func(r *http.Request) {\n\t\t\tr.Header.Set(\"Range\", \"bytes=29839070-29839080\")\n\t\t})\n\t\tassert.Equal(t, http.StatusPartialContent, resp.StatusCode)\n\t\tassert.Equal(t, \"bytes 29839070-29839080/109266405\", resp.Headers.Get(\"Content-Range\"))\n\t\tassert.Equal(t, \"EXAMPLE.COM\", resp.Body)\n\t})\n\n\tt.Run(\"Succeeds Fetching First Range of Multi-range Request\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tresp := client.Get(fmt.Sprintf(\"/ipfs/%s\", fileCid), func(r *http.Request) {\n\t\t\tr.Header.Set(\"Range\", \"bytes=1276-1279, 29839070-29839080\")\n\t\t})\n\t\tassert.Equal(t, http.StatusPartialContent, resp.StatusCode)\n\t\tassert.Equal(t, \"bytes 1276-1279/109266405\", resp.Headers.Get(\"Content-Range\"))\n\t\tassert.Equal(t, \"iana\", resp.Body)\n\t})\n}\n" }, { "path": "test/cli/gateway_test.go", "content": "package cli\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n\t\"github.com/multiformats/go-multibase\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGateway(t *testing.T) {\n\tt.Parallel()\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init().StartDaemon(\"--offline\")\n\tt.Cleanup(func() { node.StopDaemon() })\n\tcid := node.IPFSAddStr(\"Hello Worlds!\")\n\n\tpeerID, err := peer.ToCid(node.PeerID()).StringOfBase(multibase.Base36)\n\tassert.NoError(t, err)\n\n\tclient := node.GatewayClient()\n\tclient.TemplateData = map[string]string{\n\t\t\"CID\": cid,\n\t\t\"PeerID\": peerID,\n\t}\n\n\tt.Run(\"GET IPFS path succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/{{.CID}}\")\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\t})\n\n\tt.Run(\"GET IPFS path with explicit ?filename succeeds with proper header\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/{{.CID}}?filename=testтест.pdf\")\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\tassert.Equal(t,\n\t\t\t`inline; filename=\"test____.pdf\"; filename*=UTF-8''test%D1%82%D0%B5%D1%81%D1%82.pdf`,\n\t\t\tresp.Headers.Get(\"Content-Disposition\"),\n\t\t)\n\t})\n\n\tt.Run(\"GET IPFS path with explicit ?filename and &download=true succeeds with proper header\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/{{.CID}}?filename=testтест.mp4&download=true\")\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\tassert.Equal(t,\n\t\t\t`attachment; filename=\"test____.mp4\"; filename*=UTF-8''test%D1%82%D0%B5%D1%81%D1%82.mp4`,\n\t\t\tresp.Headers.Get(\"Content-Disposition\"),\n\t\t)\n\t})\n\n\t// https://github.com/ipfs/go-ipfs/issues/4025#issuecomment-342250616\n\tt.Run(\"GET for Server Worker registration outside of an IPFS content root errors\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := client.Get(\"/ipfs/{{.CID}}?filename=sw.js\", client.WithHeader(\"Service-Worker\", \"script\"))\n\t\tassert.Equal(t, 400, resp.StatusCode)\n\t\tassert.Contains(t, resp.Body, \"navigator.serviceWorker: registration is not allowed for this scope\")\n\t})\n\n\tt.Run(\"GET IPFS directory path succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tclient := node.GatewayClient().DisableRedirects()\n\n\t\tpageContents := \"hello i am a webpage\"\n\t\tfileContents := \"12345\"\n\t\th.WriteFile(\"dir/test\", fileContents)\n\t\th.WriteFile(\"dir/dirwithindex/index.html\", pageContents)\n\t\tcids := node.IPFS(\"add\", \"-r\", \"-q\", filepath.Join(h.Dir, \"dir\")).Stdout.Lines()\n\n\t\trootCID := cids[len(cids)-1]\n\t\tclient.TemplateData = map[string]string{\n\t\t\t\"IndexFileCID\": cids[0],\n\t\t\t\"TestFileCID\": cids[1],\n\t\t\t\"RootCID\": rootCID,\n\t\t}\n\n\t\tt.Run(\"GET IPFS the index file CID\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/{{.IndexFileCID}}\")\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\tassert.Equal(t, pageContents, resp.Body)\n\t\t})\n\n\t\tt.Run(\"GET IPFS the test file CID\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/{{.TestFileCID}}\")\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\tassert.Equal(t, fileContents, resp.Body)\n\t\t})\n\n\t\tt.Run(\"GET IPFS directory with index.html returns redirect to add trailing slash\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Head(\"/ipfs/{{.RootCID}}/dirwithindex?query=to-remember\")\n\t\t\tassert.Equal(t, 301, resp.StatusCode)\n\t\t\tassert.Equal(t,\n\t\t\t\tfmt.Sprintf(\"/ipfs/%s/dirwithindex/?query=to-remember\", rootCID),\n\t\t\t\tresp.Headers.Get(\"Location\"),\n\t\t\t)\n\t\t})\n\n\t\t// This enables go get to parse go-import meta tags from index.html files stored in IPFS\n\t\t// https://github.com/ipfs/kubo/pull/3963\n\t\tt.Run(\"GET IPFS directory with index.html and no trailing slash returns expected output when go-get is passed\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/{{.RootCID}}/dirwithindex?go-get=1\")\n\t\t\tassert.Equal(t, pageContents, resp.Body)\n\t\t})\n\n\t\tt.Run(\"GET IPFS directory with index.html and trailing slash returns expected output\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/{{.RootCID}}/dirwithindex/?query=to-remember\")\n\t\t\tassert.Equal(t, pageContents, resp.Body)\n\t\t})\n\n\t\tt.Run(\"GET IPFS nonexistent file returns 404 (Not Found)\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/{{.RootCID}}/pleaseDontAddMe\")\n\t\t\tassert.Equal(t, 404, resp.StatusCode)\n\t\t})\n\n\t\tt.Run(\"GET IPFS invalid CID returns 400 (Bad Request)\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/QmInvalid/pleaseDontAddMe\")\n\t\t\tassert.Equal(t, 400, resp.StatusCode)\n\t\t})\n\n\t\tt.Run(\"GET IPFS inlined zero-length data object returns ok code (200)\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/bafkqaaa\")\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\tassert.Equal(t, \"0\", resp.Resp.Header.Get(\"Content-Length\"))\n\t\t\tassert.Equal(t, \"\", resp.Body)\n\t\t})\n\n\t\tt.Run(\"GET IPFS inlined zero-length data object with byte range returns ok code (200)\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/bafkqaaa\", client.WithHeader(\"Range\", \"bytes=0-1048575\"))\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\tassert.Equal(t, \"0\", resp.Resp.Header.Get(\"Content-Length\"))\n\t\t\tassert.Equal(t, \"text/plain\", resp.Resp.Header.Get(\"Content-Type\"))\n\t\t})\n\n\t\tt.Run(\"GET /ipfs/ipfs/{cid} returns redirect to the valid path\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/ipfs/bafkqaaa?query=to-remember\")\n\t\t\tassert.Equal(t, 301, resp.StatusCode)\n\t\t\tassert.Equal(t, \"/ipfs/bafkqaaa?query=to-remember\", resp.Resp.Header.Get(\"Location\"))\n\t\t})\n\t})\n\n\tt.Run(\"IPNS\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode.IPFS(\"name\", \"publish\", \"--allow-offline\", \"--ttl\", \"42h\", cid)\n\n\t\tt.Run(\"GET invalid IPNS root returns 500 (Internal Server Error)\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipns/QmInvalid/pleaseDontAddMe\")\n\t\t\tassert.Equal(t, 500, resp.StatusCode)\n\t\t})\n\n\t\tt.Run(\"GET IPNS path succeeds\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipns/{{.PeerID}}\")\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\tassert.Equal(t, \"Hello Worlds!\", resp.Body)\n\t\t})\n\n\t\tt.Run(\"GET IPNS path has correct Cache-Control\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipns/{{.PeerID}}\")\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\tcacheControl := resp.Headers.Get(\"Cache-Control\")\n\t\t\tassert.True(t, strings.HasPrefix(cacheControl, \"public, max-age=\"))\n\t\t\tmaxAge, err := strconv.Atoi(strings.TrimPrefix(cacheControl, \"public, max-age=\"))\n\t\t\tassert.NoError(t, err)\n\t\t\tassert.True(t, maxAge-151200 < 60) // MaxAge within 42h and 42h-1m\n\t\t})\n\n\t\tt.Run(\"GET /ipfs/ipns/{peerid} returns redirect to the valid path\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tresp := client.Get(\"/ipfs/ipns/{{.PeerID}}?query=to-remember\")\n\t\t\tassert.Equal(t, 301, resp.StatusCode)\n\t\t\tassert.Equal(t, fmt.Sprintf(\"/ipns/%s?query=to-remember\", peerID), resp.Resp.Header.Get(\"Location\"))\n\t\t})\n\t})\n\n\tt.Run(\"GET invalid IPFS path errors\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tassert.Equal(t, 400, client.Get(\"/ipfs/12345\").StatusCode)\n\t})\n\n\tt.Run(\"GET invalid path errors\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tassert.Equal(t, 404, client.Get(\"/12345\").StatusCode)\n\t})\n\n\t// TODO: these tests that use the API URL shouldn't be part of gateway tests...\n\tt.Run(\"GET /webui returns 301 or 302\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := node.APIClient().DisableRedirects().Get(\"/webui\")\n\t\tassert.Contains(t, []int{302, 301, 307, 308}, resp.StatusCode)\n\t})\n\n\tt.Run(\"GET /webui/ returns 301 or 302\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := node.APIClient().DisableRedirects().Get(\"/webui/\")\n\t\tassert.Contains(t, []int{302, 301, 307, 308}, resp.StatusCode)\n\t})\n\n\tt.Run(\"GET /webui/ returns user-specified headers\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\theader := \"Access-Control-Allow-Origin\"\n\t\tvalues := []string{\"http://localhost:3000\", \"https://webui.ipfs.io\"}\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.API.HTTPHeaders = map[string][]string{header: values}\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tresp := node.APIClient().DisableRedirects().Get(\"/webui/\")\n\t\tassert.Equal(t, resp.Headers.Values(header), values)\n\t\tassert.Contains(t, []int{302, 301}, resp.StatusCode)\n\t})\n\n\tt.Run(\"POST /api/v0/version succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tresp := node.APIClient().Post(\"/api/v0/version\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\tassert.Len(t, resp.Resp.TransferEncoding, 1)\n\t\tassert.Equal(t, \"chunked\", resp.Resp.TransferEncoding[0])\n\n\t\tvers := struct{ Version string }{}\n\t\terr := json.Unmarshal([]byte(resp.Body), &vers)\n\t\trequire.NoError(t, err)\n\t\tassert.NotEmpty(t, vers.Version)\n\t})\n\n\tt.Run(\"pprof\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tt.Cleanup(func() { node.StopDaemon() })\n\t\tapiClient := node.APIClient()\n\t\tt.Run(\"mutex\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tt.Run(\"setting the mutex fraction works (negative so it doesn't enable)\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresp := apiClient.Post(\"/debug/pprof-mutex/?fraction=-1\", nil)\n\t\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\t})\n\t\t\tt.Run(\"mutex endpoint doesn't accept a string as an argument\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresp := apiClient.Post(\"/debug/pprof-mutex/?fraction=that_is_a_string\", nil)\n\t\t\t\tassert.Equal(t, 400, resp.StatusCode)\n\t\t\t})\n\t\t\tt.Run(\"mutex endpoint returns 405 on GET\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresp := apiClient.Get(\"/debug/pprof-mutex/?fraction=-1\")\n\t\t\t\tassert.Equal(t, 405, resp.StatusCode)\n\t\t\t})\n\t\t})\n\t\tt.Run(\"block\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tt.Run(\"setting the block profiler rate works (0 so it doesn't enable)\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresp := apiClient.Post(\"/debug/pprof-block/?rate=0\", nil)\n\t\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\t\t})\n\t\t\tt.Run(\"block profiler endpoint doesn't accept a string as an argument\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresp := apiClient.Post(\"/debug/pprof-block/?rate=that_is_a_string\", nil)\n\t\t\t\tassert.Equal(t, 400, resp.StatusCode)\n\t\t\t})\n\t\t\tt.Run(\"block profiler endpoint returns 405 on GET\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tresp := apiClient.Get(\"/debug/pprof-block/?rate=0\")\n\t\t\t\tassert.Equal(t, 405, resp.StatusCode)\n\t\t\t})\n\t\t})\n\t})\n\n\tt.Run(\"index content types\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init().StartDaemon()\n\t\tt.Cleanup(func() { node.StopDaemon() })\n\n\t\th.WriteFile(\"index/index.html\", \"

\")\n\t\tcid := node.IPFS(\"add\", \"-Q\", \"-r\", filepath.Join(h.Dir, \"index\")).Stderr.Trimmed()\n\n\t\tapiClient := node.APIClient()\n\t\tapiClient.TemplateData = map[string]string{\"CID\": cid}\n\n\t\tt.Run(\"GET index.html has correct content type\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := apiClient.Get(\"/ipfs/{{.CID}}/\")\n\t\t\tassert.Equal(t, \"text/html; charset=utf-8\", res.Resp.Header.Get(\"Content-Type\"))\n\t\t})\n\n\t\tt.Run(\"HEAD index.html has no content\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := apiClient.Head(\"/ipfs/{{.CID}}/\")\n\t\t\tassert.Equal(t, \"\", res.Body)\n\t\t\tassert.Equal(t, \"\", res.Resp.Header.Get(\"Content-Length\"))\n\t\t})\n\t})\n\n\tt.Run(\"raw leaves node\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tcontents := \"This is RAW!\"\n\t\tcid := node.IPFSAddStr(contents, \"--raw-leaves\")\n\t\tassert.Equal(t, contents, client.Get(\"/ipfs/\"+cid).Body)\n\t})\n\n\tt.Run(\"compact blocks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tblock1 := \"\\x0a\\x09\\x08\\x02\\x12\\x03\\x66\\x6f\\x6f\\x18\\x03\"\n\t\tblock2 := \"\\x0a\\x04\\x08\\x02\\x18\\x06\\x12\\x24\\x0a\\x22\\x12\\x20\\xcf\\x92\\xfd\\xef\\xcd\\xc3\\x4c\\xac\\x00\\x9c\" +\n\t\t\t\"\\x8b\\x05\\xeb\\x66\\x2b\\xe0\\x61\\x8d\\xb9\\xde\\x55\\xec\\xd4\\x27\\x85\\xe9\\xec\\x67\\x12\\xf8\\xdf\\x65\" +\n\t\t\t\"\\x12\\x24\\x0a\\x22\\x12\\x20\\xcf\\x92\\xfd\\xef\\xcd\\xc3\\x4c\\xac\\x00\\x9c\\x8b\\x05\\xeb\\x66\\x2b\\xe0\" +\n\t\t\t\"\\x61\\x8d\\xb9\\xde\\x55\\xec\\xd4\\x27\\x85\\xe9\\xec\\x67\\x12\\xf8\\xdf\\x65\"\n\n\t\tnode.PipeStrToIPFS(block1, \"block\", \"put\")\n\t\tblock2CID := node.PipeStrToIPFS(block2, \"block\", \"put\", \"--cid-codec=dag-pb\").Stdout.Trimmed()\n\n\t\tresp := client.Get(\"/ipfs/\" + block2CID)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\t\tassert.Equal(t, \"foofoo\", resp.Body)\n\t})\n\n\tt.Run(\"verify gateway file\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tr := regexp.MustCompile(`Gateway server listening on (?P.+)\\s`)\n\t\tmatches := r.FindStringSubmatch(node.Daemon.Stdout.String())\n\t\tma, err := multiaddr.NewMultiaddr(matches[1])\n\t\trequire.NoError(t, err)\n\t\tnetAddr, err := manet.ToNetAddr(ma)\n\t\trequire.NoError(t, err)\n\t\texpURL := \"http://\" + netAddr.String()\n\n\t\tb, err := os.ReadFile(filepath.Join(node.Dir, \"gateway\"))\n\t\trequire.NoError(t, err)\n\n\t\tassert.Equal(t, expURL, string(b))\n\t})\n\n\tt.Run(\"verify gateway file diallable while on unspecified\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Addresses.Gateway = config.Strings{\"/ip4/127.0.0.1/tcp/32563\"}\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tb, err := os.ReadFile(filepath.Join(node.Dir, \"gateway\"))\n\t\trequire.NoError(t, err)\n\n\t\tassert.Equal(t, \"http://127.0.0.1:32563\", string(b))\n\t})\n\n\tt.Run(\"NoFetch\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tnode1.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.NoFetch = true\n\t\t})\n\n\t\tnode2PeerID, err := peer.ToCid(node2.PeerID()).StringOfBase(multibase.Base36)\n\t\tassert.NoError(t, err)\n\n\t\tnodes.StartDaemons().Connect()\n\t\tt.Cleanup(func() { nodes.StopDaemons() })\n\n\t\tt.Run(\"not present\", func(t *testing.T) {\n\t\t\tcidFoo := node2.IPFSAddStr(\"foo\")\n\n\t\t\tt.Run(\"not present CID from node 1\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tassert.Equal(t, 404, node1.GatewayClient().Get(\"/ipfs/\"+cidFoo).StatusCode)\n\t\t\t})\n\n\t\t\tt.Run(\"not present IPNS Record from node 1\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tassert.Equal(t, 500, node1.GatewayClient().Get(\"/ipns/\"+node2PeerID).StatusCode)\n\t\t\t})\n\t\t})\n\n\t\tt.Run(\"present\", func(t *testing.T) {\n\t\t\tcidBar := node1.IPFSAddStr(\"bar\")\n\n\t\t\tt.Run(\"present CID from node 1\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tassert.Equal(t, 200, node1.GatewayClient().Get(\"/ipfs/\"+cidBar).StatusCode)\n\t\t\t})\n\n\t\t\tt.Run(\"present IPNS Record from node 1\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tnode2.IPFS(\"name\", \"publish\", \"/ipfs/\"+cidBar)\n\t\t\t\tassert.Equal(t, 200, node1.GatewayClient().Get(\"/ipns/\"+node2PeerID).StatusCode)\n\t\t\t})\n\t\t})\n\t})\n\n\tt.Run(\"DeserializedResponses\", func(t *testing.T) {\n\t\ttype testCase struct {\n\t\t\tglobalValue config.Flag\n\t\t\tgatewayValue config.Flag\n\t\t\tdeserializedGlobalStatusCode int\n\t\t\tdeserializedGatewayStaticCode int\n\t\t\tmessage string\n\t\t}\n\n\t\tsetHost := func(r *http.Request) {\n\t\t\tr.Host = \"example.com\"\n\t\t}\n\n\t\twithAccept := func(accept string) func(r *http.Request) {\n\t\t\treturn func(r *http.Request) {\n\t\t\t\tr.Header.Set(\"Accept\", accept)\n\t\t\t}\n\t\t}\n\n\t\twithHostAndAccept := func(accept string) func(r *http.Request) {\n\t\t\treturn func(r *http.Request) {\n\t\t\t\tsetHost(r)\n\t\t\t\twithAccept(accept)(r)\n\t\t\t}\n\t\t}\n\n\t\tmakeTest := func(test *testCase) func(t *testing.T) {\n\t\t\treturn func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\n\t\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\t\tcfg.Gateway.DeserializedResponses = test.globalValue\n\t\t\t\t\tcfg.Gateway.PublicGateways = map[string]*config.GatewaySpec{\n\t\t\t\t\t\t\"example.com\": {\n\t\t\t\t\t\t\tPaths: []string{\"/ipfs\", \"/ipns\"},\n\t\t\t\t\t\t\tDeserializedResponses: test.gatewayValue,\n\t\t\t\t\t\t},\n\t\t\t\t\t}\n\t\t\t\t})\n\t\t\t\tnode.StartDaemon()\n\t\t\t\tdefer node.StopDaemon()\n\n\t\t\t\tcidFoo := node.IPFSAddStr(\"foo\")\n\t\t\t\tclient := node.GatewayClient()\n\n\t\t\t\tdeserializedPath := \"/ipfs/\" + cidFoo\n\n\t\t\t\tblockPath := deserializedPath + \"?format=raw\"\n\t\t\t\tcarPath := deserializedPath + \"?format=car\"\n\n\t\t\t\t// Global Check (Gateway.DeserializedResponses)\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(blockPath).StatusCode)\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(deserializedPath, withAccept(\"application/vnd.ipld.raw\")).StatusCode)\n\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(carPath).StatusCode)\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(deserializedPath, withAccept(\"application/vnd.ipld.car\")).StatusCode)\n\n\t\t\t\tassert.Equal(t, test.deserializedGlobalStatusCode, client.Get(deserializedPath).StatusCode)\n\t\t\t\tassert.Equal(t, test.deserializedGlobalStatusCode, client.Get(deserializedPath, withAccept(\"application/json\")).StatusCode)\n\n\t\t\t\t// Public Gateway (example.com) Check (Gateway.PublicGateways[example.com].DeserializedResponses)\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(blockPath, setHost).StatusCode)\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(deserializedPath, withHostAndAccept(\"application/vnd.ipld.raw\")).StatusCode)\n\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(carPath, setHost).StatusCode)\n\t\t\t\tassert.Equal(t, http.StatusOK, client.Get(deserializedPath, withHostAndAccept(\"application/vnd.ipld.car\")).StatusCode)\n\n\t\t\t\tassert.Equal(t, test.deserializedGatewayStaticCode, client.Get(deserializedPath, setHost).StatusCode)\n\t\t\t\tassert.Equal(t, test.deserializedGatewayStaticCode, client.Get(deserializedPath, withHostAndAccept(\"application/json\")).StatusCode)\n\t\t\t}\n\t\t}\n\n\t\tfor _, test := range []*testCase{\n\t\t\t{config.True, config.Default, http.StatusOK, http.StatusOK, \"when Gateway.DeserializedResponses is globally enabled, leaving implicit default for Gateway.PublicGateways[example.com] should inherit the global setting (enabled)\"},\n\t\t\t{config.False, config.Default, http.StatusNotAcceptable, http.StatusNotAcceptable, \"when Gateway.DeserializedResponses is globally disabled, leaving implicit default on Gateway.PublicGateways[example.com] should inherit the global setting (disabled)\"},\n\t\t\t{config.False, config.True, http.StatusNotAcceptable, http.StatusOK, \"when Gateway.DeserializedResponses is globally disabled, explicitly enabling on Gateway.PublicGateways[example.com] should override global (enabled)\"},\n\t\t\t{config.True, config.False, http.StatusOK, http.StatusNotAcceptable, \"when Gateway.DeserializedResponses is globally enabled, explicitly disabling on Gateway.PublicGateways[example.com] should override global (disabled)\"},\n\t\t} {\n\t\t\tt.Run(test.message, makeTest(test))\n\t\t}\n\t})\n\n\tt.Run(\"DisableHTMLErrors\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tt.Run(\"Returns HTML error without DisableHTMLErrors, Accept contains text/html\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\tnode.StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tclient := node.GatewayClient()\n\n\t\t\tres := client.Get(\"/ipfs/invalid-thing\", func(r *http.Request) {\n\t\t\t\tr.Header.Set(\"Accept\", \"text/html\")\n\t\t\t})\n\t\t\tassert.NotEqual(t, http.StatusOK, res.StatusCode)\n\t\t\tassert.Contains(t, res.Resp.Header.Get(\"Content-Type\"), \"text/html\")\n\t\t})\n\n\t\tt.Run(\"Does not return HTML error with DisableHTMLErrors enabled, and Accept contains text/html\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Gateway.DisableHTMLErrors = config.True\n\t\t\t})\n\t\t\tnode.StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tclient := node.GatewayClient()\n\n\t\t\tres := client.Get(\"/ipfs/invalid-thing\", func(r *http.Request) {\n\t\t\t\tr.Header.Set(\"Accept\", \"text/html\")\n\t\t\t})\n\t\t\tassert.NotEqual(t, http.StatusOK, res.StatusCode)\n\t\t\tassert.NotContains(t, res.Resp.Header.Get(\"Content-Type\"), \"text/html\")\n\t\t})\n\t})\n}\n\n// TestLogs tests that GET /logs returns log messages. This test is separate\n// because it requires setting the server's log level to \"info\" which may\n// change the output expected by other tests.\nfunc TestLogs(t *testing.T) {\n\th := harness.NewT(t)\n\n\tt.Setenv(\"GOLOG_LOG_LEVEL\", \"info\")\n\n\tnode := h.NewNode().Init().StartDaemon(\"--offline\")\n\tdefer node.StopDaemon()\n\tcid := node.IPFSAddStr(\"Hello Worlds!\")\n\n\tpeerID, err := peer.ToCid(node.PeerID()).StringOfBase(multibase.Base36)\n\tassert.NoError(t, err)\n\n\tclient := node.GatewayClient()\n\tclient.TemplateData = map[string]string{\n\t\t\"CID\": cid,\n\t\t\"PeerID\": peerID,\n\t}\n\n\tapiClient := node.APIClient()\n\treqURL := apiClient.BuildURL(\"/logs\")\n\n\tctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)\n\tdefer cancel()\n\n\treq, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)\n\trequire.NoError(t, err)\n\n\tresp, err := apiClient.Client.Do(req)\n\trequire.NoError(t, err)\n\tdefer resp.Body.Close()\n\n\tvar found bool\n\tscanner := bufio.NewScanner(resp.Body)\n\tfor scanner.Scan() {\n\t\tif strings.Contains(scanner.Text(), \"log API client connected\") {\n\t\t\tfound = true\n\t\t\tbreak\n\t\t}\n\t}\n\tassert.True(t, found)\n}\n" }, { "path": "test/cli/harness/buffer.go", "content": "package harness\n\nimport (\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n)\n\n// Buffer is a thread-safe byte buffer.\ntype Buffer struct {\n\tb strings.Builder\n\tm sync.Mutex\n}\n\nfunc (b *Buffer) Write(p []byte) (n int, err error) {\n\tb.m.Lock()\n\tdefer b.m.Unlock()\n\treturn b.b.Write(p)\n}\n\nfunc (b *Buffer) String() string {\n\tb.m.Lock()\n\tdefer b.m.Unlock()\n\treturn b.b.String()\n}\n\n// Trimmed returns the bytes as a string, but with the trailing newline removed.\n// This only removes a single trailing newline, not all whitespace.\nfunc (b *Buffer) Trimmed() string {\n\tb.m.Lock()\n\tdefer b.m.Unlock()\n\ts := b.b.String()\n\tif len(s) == 0 {\n\t\treturn s\n\t}\n\tif s[len(s)-1] == '\\n' {\n\t\treturn s[:len(s)-1]\n\t}\n\treturn s\n}\n\nfunc (b *Buffer) Bytes() []byte {\n\tb.m.Lock()\n\tdefer b.m.Unlock()\n\treturn []byte(b.b.String())\n}\n\nfunc (b *Buffer) Lines() []string {\n\tb.m.Lock()\n\tdefer b.m.Unlock()\n\treturn testutils.SplitLines(b.b.String())\n}\n" }, { "path": "test/cli/harness/harness.go", "content": "package harness\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/multiformats/go-multiaddr\"\n)\n\n// Harness tracks state for a test, such as temp dirs and IFPS nodes, and cleans them up after the test.\ntype Harness struct {\n\tDir string\n\tIPFSBin string\n\tRunner *Runner\n\tNodesRoot string\n\tNodes Nodes\n}\n\n// TODO: use zaptest.NewLogger(t) instead\nfunc EnableDebugLogging() {\n\terr := logging.SetLogLevel(\"testharness\", \"DEBUG\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\n// NewT constructs a harness that cleans up after the given test is done.\nfunc NewT(t *testing.T, options ...func(h *Harness)) *Harness {\n\th := New(options...)\n\tt.Cleanup(h.Cleanup)\n\treturn h\n}\n\nfunc New(options ...func(h *Harness)) *Harness {\n\th := &Harness{Runner: &Runner{Env: osEnviron()}}\n\n\t// walk up to find the root dir, from which we can locate the binary\n\twd, err := os.Getwd()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tgoMod := FindUp(\"go.mod\", wd)\n\tif goMod == \"\" {\n\t\tpanic(\"unable to find root dir\")\n\t}\n\trootDir := filepath.Dir(goMod)\n\th.IPFSBin = filepath.Join(rootDir, \"cmd\", \"ipfs\", \"ipfs\")\n\n\t// setup working dir\n\ttmpDir, err := os.MkdirTemp(\"\", \"\")\n\tif err != nil {\n\t\tlog.Panicf(\"error creating temp dir: %s\", err)\n\t}\n\th.Dir = tmpDir\n\th.Runner.Dir = h.Dir\n\n\th.NodesRoot = filepath.Join(h.Dir, \".nodes\")\n\n\t// apply any customizations\n\t// this should happen after all initialization\n\tfor _, o := range options {\n\t\to(h)\n\t}\n\n\treturn h\n}\n\nfunc osEnviron() map[string]string {\n\tm := map[string]string{}\n\tfor _, entry := range os.Environ() {\n\t\tsplit := strings.Split(entry, \"=\")\n\t\tm[split[0]] = split[1]\n\t}\n\treturn m\n}\n\nfunc (h *Harness) NewNode() *Node {\n\tnodeID := len(h.Nodes)\n\tnode := BuildNode(h.IPFSBin, h.NodesRoot, nodeID)\n\th.Nodes = append(h.Nodes, node)\n\treturn node\n}\n\nfunc (h *Harness) NewNodes(count int) Nodes {\n\tvar newNodes []*Node\n\tfor range count {\n\t\tnewNodes = append(newNodes, h.NewNode())\n\t}\n\treturn newNodes\n}\n\n// WriteToTemp writes the given contents to a guaranteed-unique temp file, returning its path.\nfunc (h *Harness) WriteToTemp(contents string) string {\n\tf := h.TempFile()\n\t_, err := f.WriteString(contents)\n\tif err != nil {\n\t\tlog.Panicf(\"writing to temp file: %s\", err.Error())\n\t}\n\terr = f.Close()\n\tif err != nil {\n\t\tlog.Panicf(\"closing temp file: %s\", err.Error())\n\t}\n\treturn f.Name()\n}\n\n// TempFile creates a new unique temp file.\nfunc (h *Harness) TempFile() *os.File {\n\tf, err := os.CreateTemp(h.Dir, \"\")\n\tif err != nil {\n\t\tlog.Panicf(\"creating temp file: %s\", err.Error())\n\t}\n\treturn f\n}\n\n// WriteFile writes a file given a filename and its contents.\n// The filename must be a relative path, or this panics.\nfunc (h *Harness) WriteFile(filename, contents string) {\n\tif filepath.IsAbs(filename) {\n\t\tlog.Panicf(\"%s must be a relative path\", filename)\n\t}\n\tabsPath := filepath.Join(h.Runner.Dir, filename)\n\terr := os.MkdirAll(filepath.Dir(absPath), 0o777)\n\tif err != nil {\n\t\tlog.Panicf(\"creating intermediate dirs for %q: %s\", filename, err.Error())\n\t}\n\terr = os.WriteFile(absPath, []byte(contents), 0o644)\n\tif err != nil {\n\t\tlog.Panicf(\"writing %q (%q): %s\", filename, absPath, err.Error())\n\t}\n}\n\nfunc WaitForFile(path string, timeout time.Duration) error {\n\tstart := time.Now()\n\ttimer := time.NewTimer(timeout)\n\tticker := time.NewTicker(1 * time.Millisecond)\n\tdefer timer.Stop()\n\tdefer ticker.Stop()\n\tfor {\n\t\tselect {\n\t\tcase <-timer.C:\n\t\t\treturn fmt.Errorf(\"timeout waiting for %s after %v\", path, time.Since(start))\n\t\tcase <-ticker.C:\n\t\t\t_, err := os.Stat(path)\n\t\t\tif err == nil {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tif errors.Is(err, os.ErrNotExist) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"error waiting for %s: %w\", path, err)\n\t\t}\n\t}\n}\n\nfunc (h *Harness) Mkdirs(paths ...string) {\n\tfor _, path := range paths {\n\t\tif filepath.IsAbs(path) {\n\t\t\tlog.Panicf(\"%s must be a relative path when making dirs\", path)\n\t\t}\n\t\tabsPath := filepath.Join(h.Runner.Dir, path)\n\t\terr := os.MkdirAll(absPath, 0o777)\n\t\tif err != nil {\n\t\t\tlog.Panicf(\"recursively making dirs under %s: %s\", absPath, err)\n\t\t}\n\t}\n}\n\nfunc (h *Harness) Sh(expr string) *RunResult {\n\treturn h.Runner.Run(RunRequest{\n\t\tPath: \"bash\",\n\t\tArgs: []string{\"-c\", expr},\n\t})\n}\n\nfunc (h *Harness) Cleanup() {\n\tlog.Debugf(\"cleaning up cluster\")\n\th.Nodes.StopDaemons()\n\t// TODO: don't do this if test fails, not sure how?\n\tlog.Debugf(\"removing harness dir\")\n\terr := os.RemoveAll(h.Dir)\n\tif err != nil {\n\t\tlog.Panicf(\"removing temp dir %s: %s\", h.Dir, err)\n\t}\n}\n\n// ExtractPeerID extracts a peer ID from the given multiaddr, and fatals if it does not contain a peer ID.\nfunc (h *Harness) ExtractPeerID(m multiaddr.Multiaddr) peer.ID {\n\tvar peerIDStr string\n\tmultiaddr.ForEach(m, func(c multiaddr.Component) bool {\n\t\tif c.Protocol().Code == multiaddr.P_P2P {\n\t\t\tpeerIDStr = c.Value()\n\t\t}\n\t\treturn true\n\t})\n\tif peerIDStr == \"\" {\n\t\tpanic(multiaddr.ErrProtocolNotFound)\n\t}\n\tpeerID, err := peer.Decode(peerIDStr)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn peerID\n}\n" }, { "path": "test/cli/harness/http_client.go", "content": "package harness\n\nimport (\n\t\"io\"\n\t\"net/http\"\n\t\"strings\"\n\t\"text/template\"\n\t\"time\"\n)\n\n// HTTPClient is an HTTP client with some conveniences for testing.\n// URLs are constructed from a base URL.\n// The response body is buffered into a string.\n// Internal errors cause panics so that tests don't need to check errors.\n// The paths are evaluated as Go templates for readable string interpolation.\ntype HTTPClient struct {\n\tClient *http.Client\n\tBaseURL string\n\n\tTimeout time.Duration\n\tTemplateData any\n}\n\ntype HTTPResponse struct {\n\tBody string\n\tStatusCode int\n\tHeaders http.Header\n\n\t// The raw response. The body will be closed on this response.\n\tResp *http.Response\n}\n\nfunc (c *HTTPClient) WithHeader(k, v string) func(h *http.Request) {\n\treturn func(h *http.Request) {\n\t\th.Header.Add(k, v)\n\t}\n}\n\nfunc (c *HTTPClient) DisableRedirects() *HTTPClient {\n\tc.Client.CheckRedirect = func(req *http.Request, via []*http.Request) error {\n\t\treturn http.ErrUseLastResponse\n\t}\n\treturn c\n}\n\n// Do executes the request unchanged.\nfunc (c *HTTPClient) Do(req *http.Request) *HTTPResponse {\n\tlog.Debugf(\"making HTTP req %s to %q with headers %+v\", req.Method, req.URL.String(), req.Header)\n\tresp, err := c.Client.Do(req)\n\tif resp != nil && resp.Body != nil {\n\t\tdefer resp.Body.Close()\n\t}\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tbodyStr, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\treturn &HTTPResponse{\n\t\tBody: string(bodyStr),\n\t\tStatusCode: resp.StatusCode,\n\t\tHeaders: resp.Header,\n\t\tResp: resp,\n\t}\n}\n\n// BuildURL constructs a request URL from the given path by interpolating the string and then appending it to the base URL.\nfunc (c *HTTPClient) BuildURL(urlPath string) string {\n\tsb := &strings.Builder{}\n\terr := template.Must(template.New(\"test\").Parse(urlPath)).Execute(sb, c.TemplateData)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\trenderedPath := sb.String()\n\treturn c.BaseURL + renderedPath\n}\n\nfunc (c *HTTPClient) Get(urlPath string, opts ...func(*http.Request)) *HTTPResponse {\n\treq, err := http.NewRequest(http.MethodGet, c.BuildURL(urlPath), nil)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfor _, o := range opts {\n\t\to(req)\n\t}\n\treturn c.Do(req)\n}\n\nfunc (c *HTTPClient) Post(urlPath string, body io.Reader, opts ...func(*http.Request)) *HTTPResponse {\n\treq, err := http.NewRequest(http.MethodPost, c.BuildURL(urlPath), body)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfor _, o := range opts {\n\t\to(req)\n\t}\n\treturn c.Do(req)\n}\n\nfunc (c *HTTPClient) PostStr(urlpath, body string, opts ...func(*http.Request)) *HTTPResponse {\n\tr := strings.NewReader(body)\n\treturn c.Post(urlpath, r, opts...)\n}\n\nfunc (c *HTTPClient) Head(urlPath string, opts ...func(*http.Request)) *HTTPResponse {\n\treq, err := http.NewRequest(http.MethodHead, c.BuildURL(urlPath), nil)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfor _, o := range opts {\n\t\to(req)\n\t}\n\treturn c.Do(req)\n}\n" }, { "path": "test/cli/harness/ipfs.go", "content": "package harness\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"reflect\"\n\t\"strings\"\n\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n)\n\nfunc (n *Node) IPFSCommands() []string {\n\tres := n.IPFS(\"commands\").Stdout.String()\n\tres = strings.TrimSpace(res)\n\tsplit := SplitLines(res)\n\tvar cmds []string\n\tfor _, line := range split {\n\t\ttrimmed := strings.TrimSpace(line)\n\t\tif trimmed == \"ipfs\" {\n\t\t\tcontinue\n\t\t}\n\t\tcmds = append(cmds, trimmed)\n\t}\n\treturn cmds\n}\n\nfunc (n *Node) SetIPFSConfig(key string, val any, flags ...string) {\n\tvalBytes, err := json.Marshal(val)\n\tif err != nil {\n\t\tlog.Panicf(\"marshling config for key '%s': %s\", key, err)\n\t}\n\tvalStr := string(valBytes)\n\n\targs := []string{\"config\", \"--json\"}\n\targs = append(args, flags...)\n\targs = append(args, key, valStr)\n\tn.IPFS(args...)\n\n\t// validate the config was set correctly\n\n\t// Create a new value which is a pointer to the same type as the source.\n\tvar newVal any\n\tif val != nil {\n\t\t// If it is not nil grab the type with reflect.\n\t\tnewVal = reflect.New(reflect.TypeOf(val)).Interface()\n\t} else {\n\t\t// else just set a pointer to an any.\n\t\tvar anything any\n\t\tnewVal = &anything\n\t}\n\tn.GetIPFSConfig(key, newVal)\n\t// dereference newVal using reflect to load the resulting value\n\tif !reflect.DeepEqual(val, reflect.ValueOf(newVal).Elem().Interface()) {\n\t\tlog.Panicf(\"key '%s' did not retain value '%s' after it was set, got '%s'\", key, val, newVal)\n\t}\n}\n\nfunc (n *Node) GetIPFSConfig(key string, val any) {\n\tres := n.IPFS(\"config\", key)\n\tvalStr := strings.TrimSpace(res.Stdout.String())\n\t// only when the result is a string is the result not well-formed JSON,\n\t// so check the value type and add quotes if it's expected to be a string\n\treflectVal := reflect.ValueOf(val)\n\tif reflectVal.Kind() == reflect.Pointer && reflectVal.Elem().Kind() == reflect.String {\n\t\tvalStr = fmt.Sprintf(`\"%s\"`, valStr)\n\t}\n\terr := json.Unmarshal([]byte(valStr), val)\n\tif err != nil {\n\t\tlog.Fatalf(\"unmarshaling config for key '%s', value '%s': %s\", key, valStr, err)\n\t}\n}\n\nfunc (n *Node) IPFSAddStr(content string, args ...string) string {\n\tlog.Debugf(\"node %d adding content '%s' with args: %v\", n.ID, PreviewStr(content), args)\n\treturn n.IPFSAdd(strings.NewReader(content), args...)\n}\n\n// IPFSAddDeterministic produces a CID of a file of a certain size, filled with deterministically generated bytes based on some seed.\n// Size is specified as a humanize string (e.g., \"256KiB\", \"1MiB\").\n// This ensures deterministic CID on the other end, that can be used in tests.\nfunc (n *Node) IPFSAddDeterministic(size string, seed string, args ...string) string {\n\tlog.Debugf(\"node %d adding %s of deterministic pseudo-random data with seed %q and args: %v\", n.ID, size, seed, args)\n\treader, err := DeterministicRandomReader(size, seed)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn n.IPFSAdd(reader, args...)\n}\n\n// IPFSAddDeterministicBytes produces a CID of a file of exactly `size` bytes, filled with deterministically generated bytes based on some seed.\n// Use this when exact byte precision is needed (e.g., threshold tests at T and T+1 bytes).\nfunc (n *Node) IPFSAddDeterministicBytes(size int64, seed string, args ...string) string {\n\tlog.Debugf(\"node %d adding %d bytes of deterministic pseudo-random data with seed %q and args: %v\", n.ID, size, seed, args)\n\treader, err := DeterministicRandomReaderBytes(size, seed)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn n.IPFSAdd(reader, args...)\n}\n\nfunc (n *Node) IPFSAdd(content io.Reader, args ...string) string {\n\tlog.Debugf(\"node %d adding with args: %v\", n.ID, args)\n\tfullArgs := []string{\"add\", \"-q\"}\n\tfullArgs = append(fullArgs, args...)\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: fullArgs,\n\t\tCmdOpts: []CmdOpt{RunWithStdin(content)},\n\t})\n\tout := strings.TrimSpace(res.Stdout.String())\n\tlog.Debugf(\"add result: %q\", out)\n\treturn out\n}\n\nfunc (n *Node) IPFSBlockPut(content io.Reader, args ...string) string {\n\tlog.Debugf(\"node %d block put with args: %v\", n.ID, args)\n\tfullArgs := []string{\"block\", \"put\"}\n\tfullArgs = append(fullArgs, args...)\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: fullArgs,\n\t\tCmdOpts: []CmdOpt{RunWithStdin(content)},\n\t})\n\tout := strings.TrimSpace(res.Stdout.String())\n\tlog.Debugf(\"block put result: %q\", out)\n\treturn out\n}\n\nfunc (n *Node) IPFSDAGPut(content io.Reader, args ...string) string {\n\tlog.Debugf(\"node %d dag put with args: %v\", n.ID, args)\n\tfullArgs := []string{\"dag\", \"put\"}\n\tfullArgs = append(fullArgs, args...)\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: fullArgs,\n\t\tCmdOpts: []CmdOpt{RunWithStdin(content)},\n\t})\n\tout := strings.TrimSpace(res.Stdout.String())\n\tlog.Debugf(\"dag put result: %q\", out)\n\treturn out\n}\n\nfunc (n *Node) IPFSDagImport(content io.Reader, cid string, args ...string) error {\n\tlog.Debugf(\"node %d dag import with args: %v\", n.ID, args)\n\tfullArgs := []string{\"dag\", \"import\", \"--pin-roots=false\"}\n\tfullArgs = append(fullArgs, args...)\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: fullArgs,\n\t\tCmdOpts: []CmdOpt{RunWithStdin(content)},\n\t})\n\tif res.Err != nil {\n\t\treturn res.Err\n\t}\n\tres = n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"block\", \"stat\", \"--offline\", cid},\n\t})\n\treturn res.Err\n}\n\n// IPFSDagExport exports a DAG rooted at cid to a CAR file at carPath.\nfunc (n *Node) IPFSDagExport(cid string, carPath string) error {\n\tlog.Debugf(\"node %d dag export of %s to %q\", n.ID, cid, carPath)\n\tcar, err := os.Create(carPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer car.Close()\n\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"dag\", \"export\", cid},\n\t\tCmdOpts: []CmdOpt{RunWithStdout(car)},\n\t})\n\treturn res.Err\n}\n" }, { "path": "test/cli/harness/log.go", "content": "package harness\n\nimport (\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n)\n\ntype event struct {\n\ttimestamp time.Time\n\tmsg string\n}\n\ntype events []*event\n\nfunc (e events) Len() int { return len(e) }\nfunc (e events) Less(i, j int) bool { return e[i].timestamp.Before(e[j].timestamp) }\nfunc (e events) Swap(i, j int) { e[i], e[j] = e[j], e[i] }\n\n// TestLogger is a logger for tests.\n// It buffers output and only writes the output if the test fails or output is explicitly turned on.\n// The purpose of this logger is to allow Go test to run with the verbose flag without printing logs.\n// The verbose flag is useful since it streams test progress, but also printing logs makes the output too verbose.\n//\n// You can also add prefixes that are prepended to each log message, for extra logging context.\n//\n// This is implemented as a hierarchy of loggers, with children flushing log entries back to parents.\n// This works because t.Cleanup() processes entries in LIFO order, so children always flush first.\n//\n// Obviously this logger should never be used in production systems.\ntype TestLogger struct {\n\tparent *TestLogger\n\tchildren []*TestLogger\n\tprefixes []string\n\tprefixesIface []any\n\tt *testing.T\n\tbuf events\n\tm sync.Mutex\n\tlogsEnabled bool\n}\n\nfunc NewTestLogger(t *testing.T) *TestLogger {\n\tl := &TestLogger{t: t, buf: make(events, 0)}\n\tt.Cleanup(l.flush)\n\treturn l\n}\n\nfunc (t *TestLogger) buildPrefix(timestamp time.Time) string {\n\td := timestamp.Format(\"2006-01-02T15:04:05.999999\")\n\t_, file, lineno, _ := runtime.Caller(2)\n\tfile = filepath.Base(file)\n\tcaller := fmt.Sprintf(\"%s:%d\", file, lineno)\n\n\tif len(t.prefixes) == 0 {\n\t\treturn fmt.Sprintf(\"%s\\t%s\\t\", d, caller)\n\t}\n\n\tprefixes := strings.Join(t.prefixes, \":\")\n\treturn fmt.Sprintf(\"%s\\t%s\\t%s: \", d, caller, prefixes)\n}\n\nfunc (t *TestLogger) Log(args ...any) {\n\ttimestamp := time.Now()\n\te := t.buildPrefix(timestamp) + fmt.Sprint(args...)\n\tt.add(&event{timestamp: timestamp, msg: e})\n}\n\nfunc (t *TestLogger) Logf(format string, args ...any) {\n\ttimestamp := time.Now()\n\te := t.buildPrefix(timestamp) + fmt.Sprintf(format, args...)\n\tt.add(&event{timestamp: timestamp, msg: e})\n}\n\nfunc (t *TestLogger) Fatal(args ...any) {\n\ttimestamp := time.Now()\n\te := t.buildPrefix(timestamp) + fmt.Sprint(append([]any{\"fatal: \"}, args...)...)\n\tt.add(&event{timestamp: timestamp, msg: e})\n\tt.t.FailNow()\n}\n\nfunc (t *TestLogger) Fatalf(format string, args ...any) {\n\ttimestamp := time.Now()\n\te := t.buildPrefix(timestamp) + fmt.Sprintf(fmt.Sprintf(\"fatal: %s\", format), args...)\n\tt.add(&event{timestamp: timestamp, msg: e})\n\tt.t.FailNow()\n}\n\nfunc (t *TestLogger) add(e *event) {\n\tt.m.Lock()\n\tdefer t.m.Unlock()\n\tt.buf = append(t.buf, e)\n}\n\nfunc (t *TestLogger) AddPrefix(prefix string) *TestLogger {\n\tl := &TestLogger{\n\t\tprefixes: append(t.prefixes, prefix),\n\t\tprefixesIface: append(t.prefixesIface, prefix),\n\t\tt: t.t,\n\t\tparent: t,\n\t\tlogsEnabled: t.logsEnabled,\n\t}\n\tt.m.Lock()\n\tdefer t.m.Unlock()\n\n\tt.children = append(t.children, l)\n\tt.t.Cleanup(l.flush)\n\n\treturn l\n}\n\nfunc (t *TestLogger) EnableLogs() {\n\tt.m.Lock()\n\tdefer t.m.Unlock()\n\tt.logsEnabled = true\n\tif t.parent != nil {\n\t\tif t.parent.logsEnabled {\n\t\t\tt.parent.EnableLogs()\n\t\t}\n\t}\n\tfmt.Printf(\"enabling %d children\\n\", len(t.children))\n\tfor _, c := range t.children {\n\t\tif !c.logsEnabled {\n\t\t\tc.EnableLogs()\n\t\t}\n\t}\n}\n\nfunc (t *TestLogger) flush() {\n\tif t.t.Failed() || t.logsEnabled {\n\t\tt.m.Lock()\n\t\tdefer t.m.Unlock()\n\t\t// if this is a child, send the events to the parent\n\t\t// the root parent will print all the events in sorted order\n\t\tif t.parent != nil {\n\t\t\tfor _, e := range t.buf {\n\t\t\t\tt.parent.add(e)\n\t\t\t}\n\t\t} else {\n\t\t\t// we're the root, sort all the events and then print them\n\t\t\tsort.Sort(t.buf)\n\t\t\tfmt.Println()\n\t\t\tfmt.Printf(\"Logs for test %q:\\n\\n\", t.t.Name())\n\t\t\tfor _, e := range t.buf {\n\t\t\t\tfmt.Println(e.msg)\n\t\t\t}\n\t\t\tfmt.Println()\n\t\t}\n\t\tt.buf = nil\n\t}\n}\n" }, { "path": "test/cli/harness/node.go", "content": "package harness\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/fs\"\n\t\"net/http\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strconv\"\n\t\"strings\"\n\t\"syscall\"\n\t\"time\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/kubo/config\"\n\tserial \"github.com/ipfs/kubo/config/serialize\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\t\"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nvar log = logging.Logger(\"testharness\")\n\n// Node is a single Kubo node.\n// Each node has its own config and can run its own Kubo daemon.\ntype Node struct {\n\tID int\n\tDir string\n\n\tAPIListenAddr multiaddr.Multiaddr\n\tGatewayListenAddr multiaddr.Multiaddr\n\tSwarmAddr multiaddr.Multiaddr\n\tEnableMDNS bool\n\n\tIPFSBin string\n\tRunner *Runner\n\n\tDaemon *RunResult\n}\n\nfunc BuildNode(ipfsBin, baseDir string, id int) *Node {\n\tdir := filepath.Join(baseDir, strconv.Itoa(id))\n\tif err := os.MkdirAll(dir, 0o755); err != nil {\n\t\tpanic(err)\n\t}\n\n\tenv := environToMap(os.Environ())\n\tenv[\"IPFS_PATH\"] = dir\n\n\t// If using \"ipfs\" binary name, provide helpful binary information\n\tif ipfsBin == \"ipfs\" {\n\t\t// Check if cmd/ipfs/ipfs exists (simple relative path check)\n\t\tlocalBinary := \"cmd/ipfs/ipfs\"\n\t\tlocalExists := false\n\t\tif _, err := os.Stat(localBinary); err == nil {\n\t\t\tlocalExists = true\n\t\t\tif abs, err := filepath.Abs(localBinary); err == nil {\n\t\t\t\tlocalBinary = abs\n\t\t\t}\n\t\t}\n\n\t\t// Check if ipfs is available in PATH\n\t\tpathBinary, pathErr := exec.LookPath(\"ipfs\")\n\n\t\t// Handle different scenarios\n\t\tif pathErr != nil {\n\t\t\t// No ipfs in PATH\n\t\t\tif localExists {\n\t\t\t\tfmt.Printf(\"WARNING: No 'ipfs' found in PATH, but local binary exists at %s\\n\", localBinary)\n\t\t\t\tfmt.Printf(\"Consider adding it to PATH or run: export PATH=\\\"$(pwd)/cmd/ipfs:$PATH\\\"\\n\")\n\t\t\t} else {\n\t\t\t\tfmt.Printf(\"ERROR: No 'ipfs' binary found in PATH and no local build at cmd/ipfs/ipfs\\n\")\n\t\t\t\tfmt.Printf(\"Run 'make build' first or install ipfs and add it to PATH\\n\")\n\t\t\t\tpanic(\"ipfs binary not available\")\n\t\t\t}\n\t\t} else {\n\t\t\t// ipfs found in PATH\n\t\t\tif localExists && localBinary != pathBinary {\n\t\t\t\tfmt.Printf(\"NOTE: Local binary at %s differs from PATH binary at %s\\n\", localBinary, pathBinary)\n\t\t\t\tfmt.Printf(\"Consider adding the local binary to PATH if you want to use the version built by 'make build'\\n\")\n\t\t\t}\n\t\t\t// If they match or no local binary, no message needed\n\t\t}\n\t}\n\n\treturn &Node{\n\t\tID: id,\n\t\tDir: dir,\n\t\tIPFSBin: ipfsBin,\n\t\tRunner: &Runner{\n\t\t\tEnv: env,\n\t\t\tDir: dir,\n\t\t},\n\t}\n}\n\nfunc (n *Node) WriteBytes(filename string, b []byte) {\n\tf, err := os.Create(filepath.Join(n.Dir, filename))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer f.Close()\n\t_, err = io.Copy(f, bytes.NewReader(b))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\n// ReadFile reads the specific file. If it is relative, it is relative the node's root dir.\nfunc (n *Node) ReadFile(filename string) string {\n\tf := filename\n\tif !filepath.IsAbs(filename) {\n\t\tf = filepath.Join(n.Dir, filename)\n\t}\n\tb, err := os.ReadFile(f)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn string(b)\n}\n\nfunc (n *Node) ConfigFile() string {\n\treturn filepath.Join(n.Dir, \"config\")\n}\n\nfunc (n *Node) ReadConfig() *config.Config {\n\tcfg, err := serial.Load(filepath.Join(n.Dir, \"config\"))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn cfg\n}\n\nfunc (n *Node) WriteConfig(c *config.Config) {\n\terr := serial.WriteConfigFile(filepath.Join(n.Dir, \"config\"), c)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\nfunc (n *Node) UpdateConfig(f func(cfg *config.Config)) {\n\tcfg := n.ReadConfig()\n\tf(cfg)\n\tn.WriteConfig(cfg)\n}\n\nfunc (n *Node) ReadUserResourceOverrides() *rcmgr.PartialLimitConfig {\n\tvar r rcmgr.PartialLimitConfig\n\terr := serial.ReadConfigFile(filepath.Join(n.Dir, \"libp2p-resource-limit-overrides.json\"), &r)\n\tswitch err {\n\tcase nil, serial.ErrNotInitialized:\n\t\treturn &r\n\tdefault:\n\t\tpanic(err)\n\t}\n}\n\nfunc (n *Node) WriteUserSuppliedResourceOverrides(c *rcmgr.PartialLimitConfig) {\n\terr := serial.WriteConfigFile(filepath.Join(n.Dir, \"libp2p-resource-limit-overrides.json\"), c)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\nfunc (n *Node) UpdateUserSuppliedResourceManagerOverrides(f func(overrides *rcmgr.PartialLimitConfig)) {\n\toverrides := n.ReadUserResourceOverrides()\n\tf(overrides)\n\tn.WriteUserSuppliedResourceOverrides(overrides)\n}\n\nfunc (n *Node) IPFS(args ...string) *RunResult {\n\tres := n.RunIPFS(args...)\n\tn.Runner.AssertNoError(res)\n\treturn res\n}\n\nfunc (n *Node) PipeStrToIPFS(s string, args ...string) *RunResult {\n\treturn n.PipeToIPFS(strings.NewReader(s), args...)\n}\n\nfunc (n *Node) PipeToIPFS(reader io.Reader, args ...string) *RunResult {\n\tres := n.RunPipeToIPFS(reader, args...)\n\tn.Runner.AssertNoError(res)\n\treturn res\n}\n\nfunc (n *Node) RunPipeToIPFS(reader io.Reader, args ...string) *RunResult {\n\treturn n.Runner.Run(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: args,\n\t\tCmdOpts: []CmdOpt{RunWithStdin(reader)},\n\t})\n}\n\nfunc (n *Node) RunIPFS(args ...string) *RunResult {\n\treturn n.Runner.Run(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: args,\n\t})\n}\n\n// Init initializes and configures the IPFS node, after which it is ready to run.\nfunc (n *Node) Init(ipfsArgs ...string) *Node {\n\tn.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: append([]string{\"init\"}, ipfsArgs...),\n\t})\n\n\tif n.SwarmAddr == nil {\n\t\tswarmAddr, err := multiaddr.NewMultiaddr(\"/ip4/127.0.0.1/tcp/0\")\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tn.SwarmAddr = swarmAddr\n\t}\n\n\tif n.APIListenAddr == nil {\n\t\tapiAddr, err := multiaddr.NewMultiaddr(\"/ip4/127.0.0.1/tcp/0\")\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tn.APIListenAddr = apiAddr\n\t}\n\n\tif n.GatewayListenAddr == nil {\n\t\tgatewayAddr, err := multiaddr.NewMultiaddr(\"/ip4/127.0.0.1/tcp/0\")\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tn.GatewayListenAddr = gatewayAddr\n\t}\n\n\tn.UpdateConfig(func(cfg *config.Config) {\n\t\tcfg.Bootstrap = []string{}\n\t\tcfg.Addresses.Swarm = []string{n.SwarmAddr.String()}\n\t\tcfg.Addresses.API = []string{n.APIListenAddr.String()}\n\t\tcfg.Addresses.Gateway = []string{n.GatewayListenAddr.String()}\n\t\tcfg.Swarm.DisableNatPortMap = true\n\t\tcfg.Discovery.MDNS.Enabled = n.EnableMDNS\n\t\tcfg.Routing.LoopbackAddressesOnLanDHT = config.True\n\t\t// Telemetry disabled by default in tests.\n\t\tcfg.Plugins = config.Plugins{\n\t\t\tPlugins: map[string]config.Plugin{\n\t\t\t\t\"telemetry\": config.Plugin{\n\t\t\t\t\tDisabled: true,\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t})\n\treturn n\n}\n\n// StartDaemonWithReq runs a Kubo daemon with the given request.\n// This overwrites the request Path with the Kubo bin path.\n//\n// For example, if you want to run the daemon and see stderr and stdout to debug:\n//\n//\t node.StartDaemonWithReq(harness.RunRequest{\n//\t \t CmdOpts: []harness.CmdOpt{\n//\t\t\tharness.RunWithStderr(os.Stdout),\n//\t\t\tharness.RunWithStdout(os.Stdout),\n//\t\t },\n//\t })\nfunc (n *Node) StartDaemonWithReq(req RunRequest, authorization string) *Node {\n\talive := n.IsAlive()\n\tif alive {\n\t\tlog.Panicf(\"node %d is already running\", n.ID)\n\t}\n\tnewReq := req\n\tnewReq.Path = n.IPFSBin\n\tnewReq.Args = append([]string{\"daemon\"}, req.Args...)\n\tnewReq.RunFunc = (*exec.Cmd).Start\n\n\tlog.Debugf(\"starting node %d\", n.ID)\n\tres := n.Runner.MustRun(newReq)\n\n\tn.Daemon = res\n\n\tlog.Debugf(\"node %d started, checking API\", n.ID)\n\tn.WaitOnAPI(authorization)\n\treturn n\n}\n\nfunc (n *Node) StartDaemon(ipfsArgs ...string) *Node {\n\treturn n.StartDaemonWithReq(RunRequest{\n\t\tArgs: ipfsArgs,\n\t}, \"\")\n}\n\nfunc (n *Node) StartDaemonWithAuthorization(secret string, ipfsArgs ...string) *Node {\n\treturn n.StartDaemonWithReq(RunRequest{\n\t\tArgs: ipfsArgs,\n\t}, secret)\n}\n\nfunc (n *Node) signalAndWait(watch <-chan struct{}, signal os.Signal, t time.Duration) bool {\n\terr := n.Daemon.Cmd.Process.Signal(signal)\n\tif err != nil {\n\t\tif errors.Is(err, os.ErrProcessDone) {\n\t\t\tlog.Debugf(\"process for node %d has already finished\", n.ID)\n\t\t\treturn true\n\t\t}\n\t\tlog.Panicf(\"error killing daemon for node %d with peer ID %s: %s\", n.ID, n.PeerID(), err.Error())\n\t}\n\ttimer := time.NewTimer(t)\n\tdefer timer.Stop()\n\tselect {\n\tcase <-watch:\n\t\treturn true\n\tcase <-timer.C:\n\t\treturn false\n\t}\n}\n\nfunc (n *Node) StopDaemon() *Node {\n\tlog.Debugf(\"stopping node %d\", n.ID)\n\tif n.Daemon == nil {\n\t\tlog.Debugf(\"didn't stop node %d since no daemon present\", n.ID)\n\t\treturn n\n\t}\n\twatch := make(chan struct{}, 1)\n\tgo func() {\n\t\t_, _ = n.Daemon.Cmd.Process.Wait()\n\t\twatch <- struct{}{}\n\t}()\n\n\t// os.Interrupt does not support interrupts on Windows https://github.com/golang/go/issues/46345\n\tif runtime.GOOS == \"windows\" {\n\t\tif n.signalAndWait(watch, syscall.SIGKILL, 5*time.Second) {\n\t\t\treturn n\n\t\t}\n\t\tlog.Panicf(\"timed out stopping node %d with peer ID %s\", n.ID, n.PeerID())\n\t}\n\n\tlog.Debugf(\"signaling node %d with SIGTERM\", n.ID)\n\tif n.signalAndWait(watch, syscall.SIGTERM, 1*time.Second) {\n\t\treturn n\n\t}\n\tlog.Debugf(\"signaling node %d with SIGTERM\", n.ID)\n\tif n.signalAndWait(watch, syscall.SIGTERM, 2*time.Second) {\n\t\treturn n\n\t}\n\tlog.Debugf(\"signaling node %d with SIGQUIT\", n.ID)\n\tif n.signalAndWait(watch, syscall.SIGQUIT, 5*time.Second) {\n\t\treturn n\n\t}\n\tlog.Debugf(\"signaling node %d with SIGKILL\", n.ID)\n\tif n.signalAndWait(watch, syscall.SIGKILL, 5*time.Second) {\n\t\treturn n\n\t}\n\tlog.Panicf(\"timed out stopping node %d with peer ID %s\", n.ID, n.PeerID())\n\treturn n\n}\n\nfunc (n *Node) APIAddr() multiaddr.Multiaddr {\n\tma, err := n.TryAPIAddr()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn ma\n}\n\nfunc (n *Node) APIURL() string {\n\tapiAddr := n.APIAddr()\n\tnetAddr, err := manet.ToNetAddr(apiAddr)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn \"http://\" + netAddr.String()\n}\n\nfunc (n *Node) TryAPIAddr() (multiaddr.Multiaddr, error) {\n\tb, err := os.ReadFile(filepath.Join(n.Dir, \"api\"))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tma, err := multiaddr.NewMultiaddr(string(b))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn ma, nil\n}\n\nfunc (n *Node) checkAPI(authorization string) bool {\n\tapiAddr, err := n.TryAPIAddr()\n\tif err != nil {\n\t\tlog.Debugf(\"node %d API addr not available yet: %s\", n.ID, err.Error())\n\t\treturn false\n\t}\n\n\tif unixAddr, err := apiAddr.ValueForProtocol(multiaddr.P_UNIX); err == nil {\n\t\tparts := strings.SplitN(unixAddr, \"/\", 2)\n\t\tif len(parts) < 1 {\n\t\t\tpanic(\"malformed unix socket address\")\n\t\t}\n\t\tfileName := \"/\" + parts[1]\n\t\t_, err := os.Stat(fileName)\n\t\treturn !errors.Is(err, fs.ErrNotExist)\n\t}\n\n\tip, err := apiAddr.ValueForProtocol(multiaddr.P_IP4)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tport, err := apiAddr.ValueForProtocol(multiaddr.P_TCP)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\turl := fmt.Sprintf(\"http://%s:%s/api/v0/id\", ip, port)\n\tlog.Debugf(\"checking API for node %d at %s\", n.ID, url)\n\n\treq, err := http.NewRequest(http.MethodPost, url, nil)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tif authorization != \"\" {\n\t\treq.Header.Set(\"Authorization\", authorization)\n\t}\n\n\thttpResp, err := http.DefaultClient.Do(req)\n\tif err != nil {\n\t\tlog.Debugf(\"node %d API check error: %s\", err.Error())\n\t\treturn false\n\t}\n\tdefer httpResp.Body.Close()\n\tresp := struct {\n\t\tID string\n\t}{}\n\n\trespBytes, err := io.ReadAll(httpResp.Body)\n\tif err != nil {\n\t\tlog.Debugf(\"error reading API check response for node %d: %s\", n.ID, err.Error())\n\t\treturn false\n\t}\n\tlog.Debugf(\"got API check response for node %d: %s\", n.ID, string(respBytes))\n\n\terr = json.Unmarshal(respBytes, &resp)\n\tif err != nil {\n\t\tlog.Debugf(\"error decoding API check response for node %d: %s\", n.ID, err.Error())\n\t\treturn false\n\t}\n\tif resp.ID == \"\" {\n\t\tlog.Debugf(\"API check response for node %d did not contain a Peer ID\", n.ID)\n\t\treturn false\n\t}\n\trespPeerID, err := peer.Decode(resp.ID)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tpeerID := n.PeerID()\n\tif respPeerID != peerID {\n\t\tlog.Panicf(\"expected peer ID %s but got %s\", peerID, resp.ID)\n\t}\n\n\tlog.Debugf(\"API check for node %d successful\", n.ID)\n\treturn true\n}\n\nfunc (n *Node) PeerID() peer.ID {\n\tcfg := n.ReadConfig()\n\tid, err := peer.Decode(cfg.Identity.PeerID)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn id\n}\n\nfunc (n *Node) WaitOnAPI(authorization string) *Node {\n\tlog.Debugf(\"waiting on API for node %d\", n.ID)\n\tfor range 50 {\n\t\tif n.checkAPI(authorization) {\n\t\t\tlog.Debugf(\"daemon API found, daemon stdout: %s\", n.Daemon.Stdout.String())\n\t\t\treturn n\n\t\t}\n\t\ttime.Sleep(400 * time.Millisecond)\n\t}\n\tlog.Panicf(\"node %d with peer ID %s failed to come online: \\n%s\\n\\n%s\", n.ID, n.PeerID(), n.Daemon.Stderr.String(), n.Daemon.Stdout.String())\n\treturn n\n}\n\nfunc (n *Node) IsAlive() bool {\n\tif n.Daemon == nil || n.Daemon.Cmd == nil || n.Daemon.Cmd.Process == nil {\n\t\treturn false\n\t}\n\tlog.Debugf(\"signaling node %d daemon process for liveness check\", n.ID)\n\terr := n.Daemon.Cmd.Process.Signal(syscall.Signal(0))\n\tif err == nil {\n\t\tlog.Debugf(\"node %d daemon is alive\", n.ID)\n\t\treturn true\n\t}\n\tlog.Debugf(\"node %d daemon not alive: %s\", err.Error())\n\treturn false\n}\n\nfunc (n *Node) SwarmAddrs() []multiaddr.Multiaddr {\n\tres := n.Runner.Run(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"swarm\", \"addrs\", \"local\"},\n\t})\n\tif res.ExitCode() != 0 {\n\t\t// If swarm command fails (e.g., daemon not online), return empty slice\n\t\tlog.Debugf(\"Node %d: swarm addrs local failed (exit %d): %s\", n.ID, res.ExitCode(), res.Stderr.String())\n\t\treturn []multiaddr.Multiaddr{}\n\t}\n\tout := strings.TrimSpace(res.Stdout.String())\n\tif out == \"\" {\n\t\tlog.Debugf(\"Node %d: swarm addrs local returned empty output\", n.ID)\n\t\treturn []multiaddr.Multiaddr{}\n\t}\n\tlog.Debugf(\"Node %d: swarm addrs local output: %s\", n.ID, out)\n\toutLines := strings.Split(out, \"\\n\")\n\tvar addrs []multiaddr.Multiaddr\n\tfor _, addrStr := range outLines {\n\t\taddrStr = strings.TrimSpace(addrStr)\n\t\tif addrStr == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tma, err := multiaddr.NewMultiaddr(addrStr)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\taddrs = append(addrs, ma)\n\t}\n\tlog.Debugf(\"Node %d: parsed %d swarm addresses\", n.ID, len(addrs))\n\treturn addrs\n}\n\n// SwarmAddrsWithTimeout waits for swarm addresses to be available\nfunc (n *Node) SwarmAddrsWithTimeout(timeout time.Duration) []multiaddr.Multiaddr {\n\tstart := time.Now()\n\tfor time.Since(start) < timeout {\n\t\taddrs := n.SwarmAddrs()\n\t\tif len(addrs) > 0 {\n\t\t\treturn addrs\n\t\t}\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\treturn []multiaddr.Multiaddr{}\n}\n\nfunc (n *Node) SwarmAddrsWithPeerIDs() []multiaddr.Multiaddr {\n\treturn n.SwarmAddrsWithPeerIDsTimeout(5 * time.Second)\n}\n\nfunc (n *Node) SwarmAddrsWithPeerIDsTimeout(timeout time.Duration) []multiaddr.Multiaddr {\n\tipfsProtocol := multiaddr.ProtocolWithCode(multiaddr.P_IPFS).Name\n\tpeerID := n.PeerID()\n\tvar addrs []multiaddr.Multiaddr\n\tfor _, ma := range n.SwarmAddrsWithTimeout(timeout) {\n\t\t// add the peer ID to the multiaddr if it doesn't have it\n\t\t_, err := ma.ValueForProtocol(multiaddr.P_IPFS)\n\t\tif errors.Is(err, multiaddr.ErrProtocolNotFound) {\n\t\t\tcomp, err := multiaddr.NewComponent(ipfsProtocol, peerID.String())\n\t\t\tif err != nil {\n\t\t\t\tpanic(err)\n\t\t\t}\n\t\t\tma = ma.Encapsulate(comp)\n\t\t}\n\t\taddrs = append(addrs, ma)\n\t}\n\treturn addrs\n}\n\nfunc (n *Node) SwarmAddrsWithoutPeerIDs() []multiaddr.Multiaddr {\n\tvar addrs []multiaddr.Multiaddr\n\tfor _, ma := range n.SwarmAddrs() {\n\t\ti := 0\n\t\tfor _, c := range ma {\n\t\t\tif c.Protocol().Code == multiaddr.P_IPFS {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tma[i] = c\n\t\t\ti++\n\t\t}\n\t\tma = ma[:i]\n\t\tif len(ma) > 0 {\n\t\t\taddrs = append(addrs, ma)\n\t\t}\n\t}\n\treturn addrs\n}\n\nfunc (n *Node) Connect(other *Node) *Node {\n\t// Get the peer addresses to connect to\n\taddrs := other.SwarmAddrsWithPeerIDs()\n\tif len(addrs) == 0 {\n\t\t// If no addresses available, skip connection\n\t\tlog.Debugf(\"No swarm addresses available for connection\")\n\t\treturn n\n\t}\n\t// Use Run instead of MustRun to avoid panics on connection failures\n\tres := n.Runner.Run(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"swarm\", \"connect\", addrs[0].String()},\n\t})\n\tif res.ExitCode() != 0 {\n\t\tlog.Debugf(\"swarm connect failed: %s\", res.Stderr.String())\n\t}\n\treturn n\n}\n\n// ConnectAndWait connects to another node and waits for the connection to be established\nfunc (n *Node) ConnectAndWait(other *Node, timeout time.Duration) error {\n\t// Get the peer addresses to connect to - wait up to half the timeout for addresses\n\taddrs := other.SwarmAddrsWithPeerIDsTimeout(timeout / 2)\n\tif len(addrs) == 0 {\n\t\treturn fmt.Errorf(\"no swarm addresses available for node %d after waiting %v\", other.ID, timeout/2)\n\t}\n\n\totherPeerID := other.PeerID()\n\n\t// Try to connect\n\tres := n.Runner.Run(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"swarm\", \"connect\", addrs[0].String()},\n\t})\n\tif res.ExitCode() != 0 {\n\t\treturn fmt.Errorf(\"swarm connect failed: %s\", res.Stderr.String())\n\t}\n\n\t// Wait for connection to be established\n\tstart := time.Now()\n\tfor time.Since(start) < timeout {\n\t\tpeers := n.Peers()\n\t\tfor _, peerAddr := range peers {\n\t\t\tif peerID, err := peerAddr.ValueForProtocol(multiaddr.P_P2P); err == nil {\n\t\t\t\tif peerID == otherPeerID.String() {\n\t\t\t\t\treturn nil // Connection established\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\n\treturn fmt.Errorf(\"timeout waiting for connection to node %d (peer %s)\", other.ID, otherPeerID)\n}\n\nfunc (n *Node) Peers() []multiaddr.Multiaddr {\n\t// Wait for daemon to be ready if it's supposed to be running\n\tif n.Daemon != nil && n.Daemon.Cmd != nil && n.Daemon.Cmd.Process != nil {\n\t\t// Give daemon a short time to become ready\n\t\tfor range 10 {\n\t\t\tif n.IsAlive() {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t}\n\t}\n\tres := n.Runner.Run(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"swarm\", \"peers\"},\n\t})\n\tif res.ExitCode() != 0 {\n\t\t// If swarm peers fails (e.g., daemon not online), return empty slice\n\t\tlog.Debugf(\"swarm peers failed: %s\", res.Stderr.String())\n\t\treturn []multiaddr.Multiaddr{}\n\t}\n\tvar addrs []multiaddr.Multiaddr\n\tfor _, line := range res.Stdout.Lines() {\n\t\tma, err := multiaddr.NewMultiaddr(line)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\taddrs = append(addrs, ma)\n\t}\n\treturn addrs\n}\n\nfunc (n *Node) PeerWith(other *Node) {\n\tn.UpdateConfig(func(cfg *config.Config) {\n\t\tvar addrs []multiaddr.Multiaddr\n\t\tfor _, addrStr := range other.ReadConfig().Addresses.Swarm {\n\t\t\tma, err := multiaddr.NewMultiaddr(addrStr)\n\t\t\tif err != nil {\n\t\t\t\tpanic(err)\n\t\t\t}\n\t\t\taddrs = append(addrs, ma)\n\t\t}\n\n\t\tcfg.Peering.Peers = append(cfg.Peering.Peers, peer.AddrInfo{\n\t\t\tID: other.PeerID(),\n\t\t\tAddrs: addrs,\n\t\t})\n\t})\n}\n\nfunc (n *Node) Disconnect(other *Node) {\n\tn.IPFS(\"swarm\", \"disconnect\", \"/p2p/\"+other.PeerID().String())\n}\n\n// GatewayURL waits for the gateway file and then returns its contents or times out.\nfunc (n *Node) GatewayURL() string {\n\ttimer := time.NewTimer(1 * time.Second)\n\tdefer timer.Stop()\n\tfor {\n\t\tselect {\n\t\tcase <-timer.C:\n\t\t\tpanic(\"timeout waiting for gateway file\")\n\t\tdefault:\n\t\t\tb, err := os.ReadFile(filepath.Join(n.Dir, \"gateway\"))\n\t\t\tif err == nil {\n\t\t\t\treturn strings.TrimSpace(string(b))\n\t\t\t}\n\t\t\tif !errors.Is(err, fs.ErrNotExist) {\n\t\t\t\tpanic(err)\n\t\t\t}\n\t\t\ttime.Sleep(1 * time.Millisecond)\n\t\t}\n\t}\n}\n\nfunc (n *Node) GatewayClient() *HTTPClient {\n\treturn &HTTPClient{\n\t\tClient: http.DefaultClient,\n\t\tBaseURL: n.GatewayURL(),\n\t}\n}\n\nfunc (n *Node) APIClient() *HTTPClient {\n\treturn &HTTPClient{\n\t\tClient: http.DefaultClient,\n\t\tBaseURL: n.APIURL(),\n\t}\n}\n\n// DatastoreCount returns the count of entries matching the given prefix.\n// Requires the daemon to be stopped.\nfunc (n *Node) DatastoreCount(prefix string) int64 {\n\tres := n.IPFS(\"diag\", \"datastore\", \"count\", prefix)\n\tcount, _ := strconv.ParseInt(strings.TrimSpace(res.Stdout.String()), 10, 64)\n\treturn count\n}\n\n// DatastorePut writes a key-value pair to the datastore.\n// Requires the daemon to be stopped.\nfunc (n *Node) DatastorePut(key, value string) {\n\tn.IPFS(\"diag\", \"datastore\", \"put\", key, value)\n}\n\n// DatastoreGet retrieves the value at the given key.\n// Requires the daemon to be stopped. Returns nil if key not found.\nfunc (n *Node) DatastoreGet(key string) []byte {\n\tres := n.RunIPFS(\"diag\", \"datastore\", \"get\", key)\n\tif res.Err != nil {\n\t\treturn nil\n\t}\n\treturn res.Stdout.Bytes()\n}\n\n// DatastoreHasKey checks if a key exists in the datastore.\n// Requires the daemon to be stopped.\nfunc (n *Node) DatastoreHasKey(key string) bool {\n\tres := n.RunIPFS(\"diag\", \"datastore\", \"get\", key)\n\treturn res.Err == nil\n}\n" }, { "path": "test/cli/harness/nodes.go", "content": "package harness\n\nimport (\n\t\"sync\"\n\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/multiformats/go-multiaddr\"\n)\n\n// Nodes is a collection of Kubo nodes along with operations on groups of nodes.\ntype Nodes []*Node\n\nfunc (n Nodes) Init(args ...string) Nodes {\n\tForEachPar(n, func(node *Node) { node.Init(args...) })\n\treturn n\n}\n\nfunc (n Nodes) ForEachPar(f func(*Node)) {\n\tvar wg sync.WaitGroup\n\tfor _, node := range n {\n\t\twg.Add(1)\n\t\tnode := node\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tf(node)\n\t\t}()\n\t}\n\twg.Wait()\n}\n\nfunc (n Nodes) Connect() Nodes {\n\tfor i, node := range n {\n\t\tfor j, otherNode := range n {\n\t\t\tif i == j {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// Do not connect in parallel, because that can cause TLS handshake problems on some platforms.\n\t\t\tnode.Connect(otherNode)\n\t\t}\n\t}\n\tfor _, node := range n {\n\t\tfirstPeer := node.Peers()[0]\n\t\tif _, err := firstPeer.ValueForProtocol(multiaddr.P_P2P); err != nil {\n\t\t\tlog.Panicf(\"unexpected state for node %d with peer ID %s: %s\", node.ID, node.PeerID(), err)\n\t\t}\n\t}\n\treturn n\n}\n\nfunc (n Nodes) StartDaemons(args ...string) Nodes {\n\tForEachPar(n, func(node *Node) { node.StartDaemon(args...) })\n\treturn n\n}\n\nfunc (n Nodes) StopDaemons() Nodes {\n\tForEachPar(n, func(node *Node) { node.StopDaemon() })\n\treturn n\n}\n" }, { "path": "test/cli/harness/pbinspect.go", "content": "package harness\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\n\tmdag \"github.com/ipfs/boxo/ipld/merkledag\"\n\tft \"github.com/ipfs/boxo/ipld/unixfs\"\n\tpb \"github.com/ipfs/boxo/ipld/unixfs/pb\"\n)\n\n// UnixFSDataType returns the UnixFS DataType for the given CID by fetching the\n// raw block and parsing the protobuf. This directly checks the Type field in\n// the UnixFS Data message (https://specs.ipfs.tech/unixfs/#data).\n//\n// Common types:\n// - ft.TDirectory (1) = basic flat directory\n// - ft.THAMTShard (5) = HAMT sharded directory\nfunc (n *Node) UnixFSDataType(cid string) (pb.Data_DataType, error) {\n\tlog.Debugf(\"node %d block get %s\", n.ID, cid)\n\n\tvar blockData bytes.Buffer\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"block\", \"get\", cid},\n\t\tCmdOpts: []CmdOpt{RunWithStdout(&blockData)},\n\t})\n\tif res.Err != nil {\n\t\treturn 0, res.Err\n\t}\n\n\t// Parse dag-pb block\n\tprotoNode, err := mdag.DecodeProtobuf(blockData.Bytes())\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\t// Parse UnixFS data\n\tfsNode, err := ft.FSNodeFromBytes(protoNode.Data())\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn fsNode.Type(), nil\n}\n\n// UnixFSHAMTFanout returns the fanout value for a HAMT shard directory.\n// This is only valid for HAMT shards (THAMTShard type).\nfunc (n *Node) UnixFSHAMTFanout(cid string) (uint64, error) {\n\tlog.Debugf(\"node %d block get %s for fanout\", n.ID, cid)\n\n\tvar blockData bytes.Buffer\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"block\", \"get\", cid},\n\t\tCmdOpts: []CmdOpt{RunWithStdout(&blockData)},\n\t})\n\tif res.Err != nil {\n\t\treturn 0, res.Err\n\t}\n\n\t// Parse dag-pb block\n\tprotoNode, err := mdag.DecodeProtobuf(blockData.Bytes())\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\t// Parse UnixFS data\n\tfsNode, err := ft.FSNodeFromBytes(protoNode.Data())\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn fsNode.Fanout(), nil\n}\n\n// InspectPBNode uses dag-json output of 'ipfs dag get' to inspect\n// \"Logical Format\" of DAG-PB as defined in\n// https://web.archive.org/web/20250403194752/https://ipld.io/specs/codecs/dag-pb/spec/#logical-format\n// (mainly used for inspecting Links without depending on any libraries)\nfunc (n *Node) InspectPBNode(cid string) (PBNode, error) {\n\tlog.Debugf(\"node %d dag get %s as dag-json\", n.ID, cid)\n\n\tvar root PBNode\n\tvar dagJsonOutput bytes.Buffer\n\tres := n.Runner.MustRun(RunRequest{\n\t\tPath: n.IPFSBin,\n\t\tArgs: []string{\"dag\", \"get\", \"--output-codec=dag-json\", cid},\n\t\tCmdOpts: []CmdOpt{RunWithStdout(&dagJsonOutput)},\n\t})\n\tif res.Err != nil {\n\t\treturn root, res.Err\n\t}\n\n\terr := json.Unmarshal(dagJsonOutput.Bytes(), &root)\n\tif err != nil {\n\t\treturn root, err\n\t}\n\treturn root, nil\n}\n\n// Define structs to match the JSON for\ntype PBHash struct {\n\tSlash string `json:\"/\"`\n}\n\ntype PBLink struct {\n\tHash PBHash `json:\"Hash\"`\n\tName string `json:\"Name\"`\n\tTsize int `json:\"Tsize\"`\n}\n\ntype PBData struct {\n\tSlash struct {\n\t\tBytes string `json:\"bytes\"`\n\t} `json:\"/\"`\n}\n\ntype PBNode struct {\n\tData PBData `json:\"Data\"`\n\tLinks []PBLink `json:\"Links\"`\n}\n" }, { "path": "test/cli/harness/peering.go", "content": "package harness\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"net\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/config\"\n)\n\ntype Peering struct {\n\tFrom int\n\tTo int\n}\n\nvar (\n\tallocatedPorts = make(map[int]struct{})\n\tportMutex sync.Mutex\n)\n\nfunc NewRandPort() int {\n\tportMutex.Lock()\n\tdefer portMutex.Unlock()\n\n\tfor range 100 {\n\t\tl, err := net.Listen(\"tcp\", \"localhost:0\")\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tport := l.Addr().(*net.TCPAddr).Port\n\t\tl.Close()\n\n\t\tif _, used := allocatedPorts[port]; !used {\n\t\t\tallocatedPorts[port] = struct{}{}\n\t\t\treturn port\n\t\t}\n\t}\n\n\t// Fallback to random port if we can't get a unique one from the OS\n\tfor range 1000 {\n\t\tport := 30000 + rand.Intn(10000)\n\t\tif _, used := allocatedPorts[port]; !used {\n\t\t\tallocatedPorts[port] = struct{}{}\n\t\t\treturn port\n\t\t}\n\t}\n\n\tpanic(\"failed to allocate unique port after 1100 attempts\")\n}\n\nfunc CreatePeerNodes(t *testing.T, n int, peerings []Peering) (*Harness, Nodes) {\n\th := NewT(t)\n\tnodes := h.NewNodes(n).Init()\n\tnodes.ForEachPar(func(node *Node) {\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Routing.Type = config.NewOptionalString(\"none\")\n\t\t\tcfg.Addresses.Swarm = []string{fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", NewRandPort())}\n\t\t})\n\t})\n\n\tfor _, peering := range peerings {\n\t\tnodes[peering.From].PeerWith(nodes[peering.To])\n\t}\n\n\treturn h, nodes\n}\n" }, { "path": "test/cli/harness/run.go", "content": "package harness\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n)\n\n// Runner is a process runner which can run subprocesses and aggregate output.\ntype Runner struct {\n\tEnv map[string]string\n\tDir string\n\tVerbose bool\n}\n\ntype (\n\tCmdOpt func(*exec.Cmd)\n\tRunFunc func(*exec.Cmd) error\n)\n\nvar RunFuncStart = (*exec.Cmd).Start\n\ntype RunRequest struct {\n\tPath string\n\tArgs []string\n\t// Options that are applied to the exec.Cmd just before running it\n\tCmdOpts []CmdOpt\n\t// Function to use to run the command.\n\t// If not specified, defaults to cmd.Run\n\tRunFunc func(*exec.Cmd) error\n\tVerbose bool\n}\n\ntype RunResult struct {\n\tStdout *Buffer\n\tStderr *Buffer\n\tErr error\n\tExitErr *exec.ExitError\n\tCmd *exec.Cmd\n}\n\nfunc (r *RunResult) ExitCode() int {\n\treturn r.Cmd.ProcessState.ExitCode()\n}\n\nfunc environToMap(environ []string) map[string]string {\n\tm := map[string]string{}\n\tfor _, e := range environ {\n\t\tkv := strings.Split(e, \"=\")\n\t\t// Skip environment variables that start with =\n\t\t// These can occur in Windows https://github.com/golang/go/issues/61956\n\t\tif kv[0] == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tm[kv[0]] = kv[1]\n\t}\n\treturn m\n}\n\nfunc (r *Runner) Run(req RunRequest) *RunResult {\n\tcmd := exec.Command(req.Path, req.Args...)\n\tvar stdout io.Writer\n\tvar stderr io.Writer\n\toutbuf := &Buffer{}\n\terrbuf := &Buffer{}\n\n\tif r.Verbose {\n\t\tor, ow := io.Pipe()\n\t\terrr, errw := io.Pipe()\n\t\tstdout = io.MultiWriter(outbuf, ow)\n\t\tstderr = io.MultiWriter(errbuf, errw)\n\t\tgo func() {\n\t\t\t_, _ = io.Copy(os.Stdout, or)\n\t\t}()\n\t\tgo func() {\n\t\t\t_, _ = io.Copy(os.Stderr, errr)\n\t\t}()\n\t} else {\n\t\tstdout = outbuf\n\t\tstderr = errbuf\n\t}\n\n\tcmd.Stdout = stdout\n\tcmd.Stderr = stderr\n\tcmd.Dir = r.Dir\n\n\tfor k, v := range r.Env {\n\t\tcmd.Env = append(cmd.Env, fmt.Sprintf(\"%s=%s\", k, v))\n\t}\n\n\tfor _, o := range req.CmdOpts {\n\t\to(cmd)\n\t}\n\n\tif req.RunFunc == nil {\n\t\treq.RunFunc = (*exec.Cmd).Run\n\t}\n\n\tlog.Debugf(\"running %v\", cmd.Args)\n\n\terr := req.RunFunc(cmd)\n\n\tresult := RunResult{\n\t\tStdout: outbuf,\n\t\tStderr: errbuf,\n\t\tCmd: cmd,\n\t\tErr: err,\n\t}\n\n\tif exitErr, ok := err.(*exec.ExitError); ok {\n\t\tresult.ExitErr = exitErr\n\t}\n\n\treturn &result\n}\n\n// MustRun runs the command and fails the test if the command fails.\nfunc (r *Runner) MustRun(req RunRequest) *RunResult {\n\tresult := r.Run(req)\n\tr.AssertNoError(result)\n\treturn result\n}\n\nfunc (r *Runner) AssertNoError(result *RunResult) {\n\tif result.ExitErr != nil {\n\t\tlog.Panicf(\"'%s' returned error, code: %d, err: %s\\nstdout:%s\\nstderr:%s\\n\",\n\t\t\tresult.Cmd.Args, result.ExitErr.ExitCode(), result.ExitErr.Error(), result.Stdout.String(), result.Stderr.String())\n\t}\n\tif result.Err != nil {\n\t\tlog.Panicf(\"unable to run %s: %s\", result.Cmd.Path, result.Err)\n\t}\n}\n\nfunc RunWithEnv(env map[string]string) CmdOpt {\n\treturn func(cmd *exec.Cmd) {\n\t\tfor k, v := range env {\n\t\t\tcmd.Env = append(cmd.Env, fmt.Sprintf(\"%s=%s\", k, v))\n\t\t}\n\t}\n}\n\nfunc RunWithPath(path string) CmdOpt {\n\treturn func(cmd *exec.Cmd) {\n\t\tvar newEnv []string\n\t\tfor _, env := range cmd.Env {\n\t\t\te := strings.Split(env, \"=\")\n\t\t\tif e[0] == \"PATH\" {\n\t\t\t\tpaths := strings.Split(e[1], \":\")\n\t\t\t\tpaths = append(paths, path)\n\t\t\t\te[1] = strings.Join(paths, \":\")\n\t\t\t\tfmt.Printf(\"path: %s\\n\", strings.Join(e, \"=\"))\n\t\t\t}\n\t\t\tnewEnv = append(newEnv, strings.Join(e, \"=\"))\n\t\t}\n\t\tcmd.Env = newEnv\n\t}\n}\n\nfunc RunWithStdin(reader io.Reader) CmdOpt {\n\treturn func(cmd *exec.Cmd) {\n\t\tcmd.Stdin = reader\n\t}\n}\n\nfunc RunWithStdinStr(s string) CmdOpt {\n\treturn RunWithStdin(strings.NewReader(s))\n}\n\nfunc RunWithStdout(writer io.Writer) CmdOpt {\n\treturn func(cmd *exec.Cmd) {\n\t\tcmd.Stdout = io.MultiWriter(writer, cmd.Stdout)\n\t}\n}\n\nfunc RunWithStderr(writer io.Writer) CmdOpt {\n\treturn func(cmd *exec.Cmd) {\n\t\tcmd.Stderr = io.MultiWriter(writer, cmd.Stdout)\n\t}\n}\n" }, { "path": "test/cli/http_gateway_over_libp2p_test.go", "content": "package cli\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/core/commands\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/libp2p/go-libp2p\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tlibp2phttp \"github.com/libp2p/go-libp2p/p2p/http\"\n\t\"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGatewayOverLibp2p(t *testing.T) {\n\tt.Parallel()\n\tnodes := harness.NewT(t).NewNodes(2).Init()\n\n\t// Setup streaming functionality\n\tnodes.ForEachPar(func(node *harness.Node) {\n\t\tnode.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t})\n\n\tgwNode := nodes[0]\n\tp2pProxyNode := nodes[1]\n\n\tnodes.StartDaemons().Connect()\n\tdefer nodes.StopDaemons()\n\n\t// Add data to the gateway node\n\tcidDataOnGatewayNode := cid.MustParse(gwNode.IPFSAddStr(\"Hello Worlds2!\"))\n\tr := gwNode.GatewayClient().Get(fmt.Sprintf(\"/ipfs/%s?format=raw\", cidDataOnGatewayNode))\n\tblockDataOnGatewayNode := []byte(r.Body)\n\n\t// Add data to the non-gateway node\n\tcidDataNotOnGatewayNode := cid.MustParse(p2pProxyNode.IPFSAddStr(\"Hello Worlds!\"))\n\tr = p2pProxyNode.GatewayClient().Get(fmt.Sprintf(\"/ipfs/%s?format=raw\", cidDataNotOnGatewayNode))\n\tblockDataNotOnGatewayNode := []byte(r.Body)\n\t_ = blockDataNotOnGatewayNode\n\n\t// Setup one of the nodes as http to http-over-libp2p proxy\n\tp2pProxyNode.IPFS(\"p2p\", \"forward\", \"--allow-custom-protocol\", \"/http/1.1\", \"/ip4/127.0.0.1/tcp/0\", fmt.Sprintf(\"/p2p/%s\", gwNode.PeerID()))\n\tlsOutput := commands.P2PLsOutput{}\n\tif err := json.Unmarshal(p2pProxyNode.IPFS(\"p2p\", \"ls\", \"--enc=json\").Stdout.Bytes(), &lsOutput); err != nil {\n\t\tt.Fatal(err)\n\t}\n\trequire.Len(t, lsOutput.Listeners, 1)\n\tp2pProxyNodeHTTPListenMA, err := multiaddr.NewMultiaddr(lsOutput.Listeners[0].ListenAddress)\n\trequire.NoError(t, err)\n\n\tp2pProxyNodeHTTPListenAddr, err := manet.ToNetAddr(p2pProxyNodeHTTPListenMA)\n\trequire.NoError(t, err)\n\n\tt.Run(\"DoesNotWorkWithoutExperimentalConfig\", func(t *testing.T) {\n\t\t_, err := http.Get(fmt.Sprintf(\"http://%s/ipfs/%s?format=raw\", p2pProxyNodeHTTPListenAddr, cidDataOnGatewayNode))\n\t\trequire.Error(t, err)\n\t})\n\n\t// Enable the experimental feature and reconnect the nodes\n\tgwNode.IPFS(\"config\", \"--json\", \"Experimental.GatewayOverLibp2p\", \"true\")\n\tgwNode.StopDaemon().StartDaemon()\n\tt.Cleanup(func() { gwNode.StopDaemon() })\n\tnodes.Connect()\n\n\t// Note: the bare HTTP requests here assume that the gateway is mounted at `/`\n\tt.Run(\"WillNotServeRemoteContent\", func(t *testing.T) {\n\t\tresp, err := http.Get(fmt.Sprintf(\"http://%s/ipfs/%s?format=raw\", p2pProxyNodeHTTPListenAddr, cidDataNotOnGatewayNode))\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, http.StatusNotFound, resp.StatusCode)\n\t})\n\n\tt.Run(\"WillNotServeDeserializedResponses\", func(t *testing.T) {\n\t\tresp, err := http.Get(fmt.Sprintf(\"http://%s/ipfs/%s\", p2pProxyNodeHTTPListenAddr, cidDataOnGatewayNode))\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, http.StatusNotAcceptable, resp.StatusCode)\n\t})\n\n\tt.Run(\"ServeBlock\", func(t *testing.T) {\n\t\tt.Run(\"UsingKuboProxy\", func(t *testing.T) {\n\t\t\tresp, err := http.Get(fmt.Sprintf(\"http://%s/ipfs/%s?format=raw\", p2pProxyNodeHTTPListenAddr, cidDataOnGatewayNode))\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\t\t\trequire.Equal(t, 200, resp.StatusCode)\n\t\t\tbody, err := io.ReadAll(resp.Body)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, blockDataOnGatewayNode, body)\n\t\t})\n\t\tt.Run(\"UsingLibp2pClientWithPathDiscovery\", func(t *testing.T) {\n\t\t\tclientHost, err := libp2p.New(libp2p.NoListenAddrs)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = clientHost.Connect(context.Background(), peer.AddrInfo{\n\t\t\t\tID: gwNode.PeerID(),\n\t\t\t\tAddrs: gwNode.SwarmAddrs(),\n\t\t\t})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tclient, err := (&libp2phttp.Host{StreamHost: clientHost}).NamespacedClient(\"/ipfs/gateway\", peer.AddrInfo{ID: gwNode.PeerID()})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tresp, err := client.Get(fmt.Sprintf(\"/ipfs/%s?format=raw\", cidDataOnGatewayNode))\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\t\t\trequire.Equal(t, 200, resp.StatusCode)\n\n\t\t\tbody, err := io.ReadAll(resp.Body)\n\t\t\trequire.NoError(t, err)\n\n\t\t\trequire.Equal(t, blockDataOnGatewayNode, body)\n\t\t})\n\t})\n}\n" }, { "path": "test/cli/http_retrieval_client_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"net/url\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/routing/http/server\"\n\t\"github.com/ipfs/boxo/routing/http/types\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils/httprouting\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/multiformats/go-multiaddr\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestHTTPRetrievalClient(t *testing.T) {\n\tt.Parallel()\n\n\t// many moving pieces here, show more when debug is needed\n\tdebug := os.Getenv(\"DEBUG\") == \"true\"\n\n\t// usee local /routing/v1/providers/{cid} and\n\t// /ipfs/{cid} HTTP servers to confirm HTTP-only retrieval works end-to-end.\n\tt.Run(\"works end-to-end with an HTTP-only provider\", func(t *testing.T) {\n\t\t// setup mocked HTTP Router to handle /routing/v1/providers/cid\n\t\tmockRouter := &httprouting.MockHTTPContentRouter{Debug: debug}\n\t\tdelegatedRoutingServer := httptest.NewServer(server.Handler(mockRouter))\n\t\tt.Cleanup(func() { delegatedRoutingServer.Close() })\n\n\t\t// init Kubo repo\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t// explicitly enable http client\n\t\t\tcfg.HTTPRetrieval.Enabled = config.True\n\t\t\t// allow NewMockHTTPProviderServer to use self-signed TLS cert\n\t\t\tcfg.HTTPRetrieval.TLSInsecureSkipVerify = config.True\n\t\t\t// setup client-only routing which asks both HTTP + DHT\n\t\t\t// cfg.Routing.Type = config.NewOptionalString(\"autoclient\")\n\t\t\t// setup Kubo node to use mocked HTTP Router\n\t\t\tcfg.Routing.DelegatedRouters = []string{delegatedRoutingServer.URL}\n\t\t})\n\n\t\t// compute a random CID\n\t\trandStr := string(random.Bytes(100))\n\t\tres := node.PipeStrToIPFS(randStr, \"add\", \"-qn\", \"--cid-version\", \"1\") // -n means dont add to local repo, just produce CID\n\t\twantCIDStr := res.Stdout.Trimmed()\n\t\ttestCid := cid.MustParse(wantCIDStr)\n\n\t\t// setup mock HTTP provider\n\t\thttpProviderServer := NewMockHTTPProviderServer(testCid, randStr, debug)\n\t\tt.Cleanup(func() { httpProviderServer.Close() })\n\t\thttpHost, httpPort, err := splitHostPort(httpProviderServer.URL)\n\t\tassert.NoError(t, err)\n\n\t\t// setup /routing/v1/providers/cid result that points at our mocked HTTP provider\n\t\tmockHTTPProviderPeerID := \"12D3KooWCjfPiojcCUmv78Wd1NJzi4Mraj1moxigp7AfQVQvGLwH\" // static, it does not matter, we only care about multiaddr\n\t\tmockHTTPMultiaddr, _ := multiaddr.NewMultiaddr(fmt.Sprintf(\"/ip4/%s/tcp/%s/tls/http\", httpHost, httpPort))\n\t\tmpid, _ := peer.Decode(mockHTTPProviderPeerID)\n\t\tmockRouter.AddProvider(testCid, &types.PeerRecord{\n\t\t\tSchema: types.SchemaPeer,\n\t\t\tID: &mpid,\n\t\t\tAddrs: []types.Multiaddr{{Multiaddr: mockHTTPMultiaddr}},\n\t\t\t// no explicit Protocols, ensure multiaddr alone is enough\n\t\t})\n\n\t\t// Start Kubo\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tif debug {\n\t\t\tfmt.Printf(\"delegatedRoutingServer.URL: %s\\n\", delegatedRoutingServer.URL)\n\t\t\tfmt.Printf(\"httpProviderServer.URL: %s\\n\", httpProviderServer.URL)\n\t\t\tfmt.Printf(\"httpProviderServer.Multiaddr: %s\\n\", mockHTTPMultiaddr)\n\t\t\tfmt.Printf(\"testCid: %s\\n\", testCid)\n\t\t}\n\n\t\t// Now, make Kubo to read testCid. it was not added to local blockstore, so it has only one provider -- a HTTP server.\n\n\t\t// First, confirm delegatedRoutingServer returned HTTP provider\n\t\tfindprovsRes := node.IPFS(\"routing\", \"findprovs\", testCid.String())\n\t\tassert.Equal(t, mockHTTPProviderPeerID, findprovsRes.Stdout.Trimmed())\n\n\t\t// Ok, now attempt retrieval.\n\t\t// If there was no timeout and returned bytes match expected body, HTTP routing and retrieval worked end-to-end.\n\t\tcatRes := node.IPFS(\"cat\", testCid.String())\n\t\tassert.Equal(t, randStr, catRes.Stdout.Trimmed())\n\t})\n}\n\n// NewMockHTTPProviderServer pretends to be http provider that supports\n// block response https://specs.ipfs.tech/http-gateways/trustless-gateway/#block-responses-application-vnd-ipld-raw\nfunc NewMockHTTPProviderServer(c cid.Cid, body string, debug bool) *httptest.Server {\n\texpectedPathPrefix := \"/ipfs/\" + c.String()\n\thandler := http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {\n\t\tif debug {\n\t\t\tfmt.Printf(\"NewMockHTTPProviderServer GET %s\\n\", req.URL.Path)\n\t\t}\n\t\tif strings.HasPrefix(req.URL.Path, expectedPathPrefix) {\n\t\t\tw.Header().Set(\"Content-Type\", \"application/vnd.ipld.raw\")\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\tif req.Method == \"GET\" {\n\t\t\t\t_, err := w.Write([]byte(body))\n\t\t\t\tif err != nil {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"NewMockHTTPProviderServer GET %s error: %v\\n\", req.URL.Path, err)\n\t\t\t\t}\n\t\t\t}\n\t\t} else if strings.HasPrefix(req.URL.Path, \"/ipfs/bafkqaaa\") {\n\t\t\t// This is probe from https://specs.ipfs.tech/http-gateways/trustless-gateway/#dedicated-probe-paths\n\t\t\tw.Header().Set(\"Content-Type\", \"application/vnd.ipld.raw\")\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t} else {\n\t\t\thttp.Error(w, \"Not Found\", http.StatusNotFound)\n\t\t}\n\t})\n\n\t// Make it HTTP/2 with self-signed TLS cert\n\tsrv := httptest.NewUnstartedServer(handler)\n\tsrv.EnableHTTP2 = true\n\tsrv.StartTLS()\n\treturn srv\n}\n\nfunc splitHostPort(httpUrl string) (ipAddr string, port string, err error) {\n\tu, err := url.Parse(httpUrl)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\tif u.Scheme == \"\" || u.Host == \"\" {\n\t\treturn \"\", \"\", fmt.Errorf(\"invalid URL format: missing scheme or host\")\n\t}\n\tipAddr, port, err = net.SplitHostPort(u.Host)\n\tif err != nil {\n\t\treturn \"\", \"\", fmt.Errorf(\"failed to split host and port from %q: %w\", u.Host, err)\n\t}\n\treturn ipAddr, port, nil\n}\n" }, { "path": "test/cli/identity_cid_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/verifcid\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestIdentityCIDOverflowProtection(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"ipfs add --hash=identity with small data succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// small data that fits in identity CID\n\t\tsmallData := \"small data\"\n\t\ttempFile := filepath.Join(node.Dir, \"small.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(smallData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.IPFS(\"add\", \"--hash=identity\", tempFile)\n\t\tassert.NoError(t, res.Err)\n\t\tcid := strings.Fields(res.Stdout.String())[1]\n\n\t\t// verify it's actually using identity hash\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"identity\", res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"ipfs add --hash=identity with large data fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// data larger than verifcid.DefaultMaxIdentityDigestSize\n\t\tlargeData := strings.Repeat(\"x\", verifcid.DefaultMaxIdentityDigestSize+50)\n\t\ttempFile := filepath.Join(node.Dir, \"large.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(largeData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"add\", \"--hash=identity\", tempFile)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\t// should error with digest too large message\n\t\tassert.Contains(t, res.Stderr.String(), \"digest too large\")\n\t})\n\n\tt.Run(\"ipfs add --inline with valid --inline-limit succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tsmallData := \"small inline data\"\n\t\ttempFile := filepath.Join(node.Dir, \"inline.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(smallData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// use limit just under the maximum\n\t\tlimit := verifcid.DefaultMaxIdentityDigestSize - 10\n\t\tres := node.IPFS(\"add\", \"--inline\", fmt.Sprintf(\"--inline-limit=%d\", limit), tempFile)\n\t\tassert.NoError(t, res.Err)\n\t\tcid := strings.Fields(res.Stdout.String())[1]\n\n\t\t// verify the CID is using identity hash (inline)\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"identity\", res.Stdout.Trimmed())\n\n\t\t// verify the codec (may be dag-pb or raw depending on kubo version)\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", cid)\n\t\tassert.NoError(t, res.Err)\n\t\t// Accept either raw or dag-pb as both are valid for inline data\n\t\tcodec := res.Stdout.Trimmed()\n\t\tassert.True(t, codec == \"raw\" || codec == \"dag-pb\", \"expected raw or dag-pb codec, got %s\", codec)\n\t})\n\n\tt.Run(\"ipfs add --inline with excessive --inline-limit fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tsmallData := \"data\"\n\t\ttempFile := filepath.Join(node.Dir, \"inline2.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(smallData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\texcessiveLimit := verifcid.DefaultMaxIdentityDigestSize + 50\n\t\tres := node.RunIPFS(\"add\", \"--inline\", fmt.Sprintf(\"--inline-limit=%d\", excessiveLimit), tempFile)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), fmt.Sprintf(\"inline-limit %d exceeds maximum allowed size of %d bytes\", excessiveLimit, verifcid.DefaultMaxIdentityDigestSize))\n\t})\n\n\tt.Run(\"ipfs files write --hash=identity appending to identity CID switches to configured hash\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// create initial small file with identity CID\n\t\tinitialData := \"initial\"\n\t\ttempFile := filepath.Join(node.Dir, \"initial.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(initialData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.IPFS(\"add\", \"--hash=identity\", tempFile)\n\t\tassert.NoError(t, res.Err)\n\t\tcid1 := strings.Fields(res.Stdout.String())[1]\n\n\t\t// verify initial CID uses identity\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cid1)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"identity\", res.Stdout.Trimmed())\n\n\t\t// copy to MFS\n\t\tres = node.IPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", cid1), \"/identity-file\")\n\t\tassert.NoError(t, res.Err)\n\n\t\t// append data that would exceed identity CID limit\n\t\tappendData := strings.Repeat(\"a\", verifcid.DefaultMaxIdentityDigestSize)\n\t\tappendFile := filepath.Join(node.Dir, \"append.txt\")\n\t\terr = os.WriteFile(appendFile, []byte(appendData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// append to the end of the file\n\t\t// get the current data size\n\t\tres = node.IPFS(\"files\", \"stat\", \"--format\", \"\", \"/identity-file\")\n\t\tassert.NoError(t, res.Err)\n\t\tsize := res.Stdout.Trimmed()\n\t\t// this should succeed because DagModifier in boxo handles the overflow\n\t\tres = node.IPFS(\"files\", \"write\", \"--hash=identity\", \"--offset=\"+size, \"/identity-file\", appendFile)\n\t\tassert.NoError(t, res.Err)\n\n\t\t// check that the file now uses non-identity hash\n\t\tres = node.IPFS(\"files\", \"stat\", \"--hash\", \"/identity-file\")\n\t\tassert.NoError(t, res.Err)\n\t\tnewCid := res.Stdout.Trimmed()\n\n\t\t// verify new CID does NOT use identity\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", newCid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.NotEqual(t, \"identity\", res.Stdout.Trimmed())\n\n\t\t// verify it switched to a cryptographic hash\n\t\tassert.Equal(t, config.DefaultHashFunction, res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"ipfs files write --hash=identity with small write creates identity CID\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// create a small file with identity hash directly in MFS\n\t\tsmallData := \"small\"\n\t\ttempFile := filepath.Join(node.Dir, \"small.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(smallData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// write to MFS with identity hash\n\t\tres := node.IPFS(\"files\", \"write\", \"--create\", \"--hash=identity\", \"/mfs-identity\", tempFile)\n\t\tassert.NoError(t, res.Err)\n\n\t\t// verify using identity CID\n\t\tres = node.IPFS(\"files\", \"stat\", \"--hash\", \"/mfs-identity\")\n\t\tassert.NoError(t, res.Err)\n\t\tcid := res.Stdout.Trimmed()\n\n\t\t// verify CID uses identity hash\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"identity\", res.Stdout.Trimmed())\n\n\t\t// verify content\n\t\tres = node.IPFS(\"files\", \"read\", \"/mfs-identity\")\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, smallData, res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"raw node with identity CID converts to UnixFS when appending\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// create raw block with identity CID\n\t\trawData := \"raw\"\n\t\ttempFile := filepath.Join(node.Dir, \"raw.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(rawData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.IPFS(\"block\", \"put\", \"--format=raw\", \"--mhtype=identity\", tempFile)\n\t\tassert.NoError(t, res.Err)\n\t\trawCid := res.Stdout.Trimmed()\n\n\t\t// verify initial CID uses identity hash and raw codec\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", rawCid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"identity\", res.Stdout.Trimmed())\n\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", rawCid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"raw\", res.Stdout.Trimmed())\n\n\t\t// copy to MFS\n\t\tres = node.IPFS(\"files\", \"cp\", fmt.Sprintf(\"/ipfs/%s\", rawCid), \"/raw-identity\")\n\t\tassert.NoError(t, res.Err)\n\n\t\t// append data\n\t\tappendData := \"appended\"\n\t\tappendFile := filepath.Join(node.Dir, \"append-raw.txt\")\n\t\terr = os.WriteFile(appendFile, []byte(appendData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// get current data size for appending\n\t\tres = node.IPFS(\"files\", \"stat\", \"--format\", \"\", \"/raw-identity\")\n\t\tassert.NoError(t, res.Err)\n\t\tsize := res.Stdout.Trimmed()\n\t\tres = node.IPFS(\"files\", \"write\", \"--hash=identity\", \"--offset=\"+size, \"/raw-identity\", appendFile)\n\t\tassert.NoError(t, res.Err)\n\n\t\t// verify content\n\t\tres = node.IPFS(\"files\", \"read\", \"/raw-identity\")\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, rawData+appendData, res.Stdout.Trimmed())\n\n\t\t// check that it's now a UnixFS structure (dag-pb)\n\t\tres = node.IPFS(\"files\", \"stat\", \"--hash\", \"/raw-identity\")\n\t\tassert.NoError(t, res.Err)\n\t\tnewCid := res.Stdout.Trimmed()\n\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%c\", newCid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"dag-pb\", res.Stdout.Trimmed())\n\n\t\tres = node.IPFS(\"files\", \"stat\", \"/raw-identity\")\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Contains(t, res.Stdout.String(), \"Type: file\")\n\t})\n\n\tt.Run(\"ipfs add --inline-limit at exactly max size succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// create small data that will be inlined\n\t\tsmallData := \"test data for inline\"\n\t\ttempFile := filepath.Join(node.Dir, \"exact.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(smallData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// exactly at the limit should succeed\n\t\tres := node.IPFS(\"add\", \"--inline\", fmt.Sprintf(\"--inline-limit=%d\", verifcid.DefaultMaxIdentityDigestSize), tempFile)\n\t\tassert.NoError(t, res.Err)\n\t\tcid := strings.Fields(res.Stdout.String())[1]\n\n\t\t// verify it uses identity hash (inline) since data is small enough\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, \"identity\", res.Stdout.Trimmed())\n\t})\n\n\tt.Run(\"ipfs add --inline-limit one byte over max fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tsmallData := \"test\"\n\t\ttempFile := filepath.Join(node.Dir, \"oneover.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(smallData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// one byte over should fail\n\t\toverLimit := verifcid.DefaultMaxIdentityDigestSize + 1\n\t\tres := node.RunIPFS(\"add\", \"--inline\", fmt.Sprintf(\"--inline-limit=%d\", overLimit), tempFile)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), fmt.Sprintf(\"inline-limit %d exceeds maximum allowed size of %d bytes\", overLimit, verifcid.DefaultMaxIdentityDigestSize))\n\t})\n\n\tt.Run(\"ipfs add --inline with data larger than limit uses configured hash\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// data larger than inline limit\n\t\tlargeData := strings.Repeat(\"y\", 100)\n\t\ttempFile := filepath.Join(node.Dir, \"toolarge.txt\")\n\t\terr := os.WriteFile(tempFile, []byte(largeData), 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// set inline limit smaller than data\n\t\tres := node.IPFS(\"add\", \"--inline\", \"--inline-limit=50\", tempFile)\n\t\tassert.NoError(t, res.Err)\n\t\tcid := strings.Fields(res.Stdout.String())[1]\n\n\t\t// verify it's NOT using identity hash (data too large for inline)\n\t\tres = node.IPFS(\"cid\", \"format\", \"-f\", \"%h\", cid)\n\t\tassert.NoError(t, res.Err)\n\t\tassert.NotEqual(t, \"identity\", res.Stdout.Trimmed())\n\n\t\t// should use configured hash\n\t\tassert.Equal(t, config.DefaultHashFunction, res.Stdout.Trimmed())\n\t})\n}\n" }, { "path": "test/cli/init_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\tfp \"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\tpb \"github.com/libp2p/go-libp2p/core/crypto/pb\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc validatePeerID(t *testing.T, peerID peer.ID, expErr error, expAlgo pb.KeyType) {\n\tassert.NoError(t, peerID.Validate())\n\tpub, err := peerID.ExtractPublicKey()\n\tassert.ErrorIs(t, expErr, err)\n\tif expAlgo != 0 {\n\t\tassert.Equal(t, expAlgo, pub.Type())\n\t}\n}\n\nfunc testInitAlgo(t *testing.T, initFlags []string, expOutputName string, expPeerIDPubKeyErr error, expPeerIDPubKeyType pb.KeyType) {\n\tt.Run(\"init\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode()\n\t\tinitRes := node.IPFS(StrCat(\"init\", initFlags)...)\n\n\t\tlines := []string{\n\t\t\tfmt.Sprintf(\"generating %s keypair...done\", expOutputName),\n\t\t\tfmt.Sprintf(\"peer identity: %s\", node.PeerID().String()),\n\t\t\tfmt.Sprintf(\"initializing IPFS node at %s\\n\", node.Dir),\n\t\t}\n\t\texpectedInitOutput := strings.Join(lines, \"\\n\")\n\t\tassert.Equal(t, expectedInitOutput, initRes.Stdout.String())\n\n\t\tassert.DirExists(t, node.Dir)\n\t\tassert.FileExists(t, fp.Join(node.Dir, \"config\"))\n\t\tassert.DirExists(t, fp.Join(node.Dir, \"datastore\"))\n\t\tassert.DirExists(t, fp.Join(node.Dir, \"blocks\"))\n\t\tassert.NoFileExists(t, fp.Join(node.Dir, \"._check_writeable\"))\n\n\t\t_, err := os.ReadDir(node.Dir)\n\t\tassert.NoError(t, err, \"ipfs dir should be listable\")\n\n\t\tvalidatePeerID(t, node.PeerID(), expPeerIDPubKeyErr, expPeerIDPubKeyType)\n\n\t\tres := node.IPFS(\"config\", \"Mounts.IPFS\")\n\t\tassert.Equal(t, \"/ipfs\", res.Stdout.Trimmed())\n\n\t\tcatRes := node.RunIPFS(\"cat\", fmt.Sprintf(\"/ipfs/%s/readme\", CIDWelcomeDocs))\n\t\tassert.NotEqual(t, 0, catRes.ExitErr.ExitCode(), \"welcome readme doesn't exist\")\n\t})\n\n\tt.Run(\"init without empty repo\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode()\n\t\tinitRes := node.IPFS(StrCat(\"init\", \"--empty-repo=false\", initFlags)...)\n\n\t\tvalidatePeerID(t, node.PeerID(), expPeerIDPubKeyErr, expPeerIDPubKeyType)\n\n\t\tlines := []string{\n\t\t\tfmt.Sprintf(\"generating %s keypair...done\", expOutputName),\n\t\t\tfmt.Sprintf(\"peer identity: %s\", node.PeerID().String()),\n\t\t\tfmt.Sprintf(\"initializing IPFS node at %s\", node.Dir),\n\t\t\t\"to get started, enter:\",\n\t\t\tfmt.Sprintf(\"\\n\\tipfs cat /ipfs/%s/readme\\n\\n\", CIDWelcomeDocs),\n\t\t}\n\t\texpectedEmptyInitOutput := strings.Join(lines, \"\\n\")\n\t\tassert.Equal(t, expectedEmptyInitOutput, initRes.Stdout.String())\n\n\t\tnode.IPFS(\"cat\", fmt.Sprintf(\"/ipfs/%s/readme\", CIDWelcomeDocs))\n\n\t\tidRes := node.IPFS(\"id\", \"-f\", \"\")\n\t\tversion := node.IPFS(\"version\", \"-n\").Stdout.Trimmed()\n\t\tassert.Contains(t, idRes.Stdout.String(), version)\n\t})\n}\n\nfunc TestInit(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"init fails if the repo dir has no perms\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode()\n\t\tbadDir := fp.Join(node.Dir, \".badipfs\")\n\t\terr := os.Mkdir(badDir, 0o000)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"init\", \"--repo-dir\", badDir)\n\t\tassert.NotEqual(t, 0, res.Cmd.ProcessState.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"permission denied\")\n\t})\n\n\tt.Run(\"init with ed25519\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestInitAlgo(t, []string{\"--algorithm=ed25519\"}, \"ED25519\", nil, pb.KeyType_Ed25519)\n\t})\n\n\tt.Run(\"init with rsa\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestInitAlgo(t, []string{\"--bits=2048\", \"--algorithm=rsa\"}, \"2048-bit RSA\", peer.ErrNoPublicKey, 0)\n\t})\n\n\tt.Run(\"init with default algorithm\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestInitAlgo(t, []string{}, \"ED25519\", nil, pb.KeyType_Ed25519)\n\t})\n\n\tt.Run(\"ipfs init --profile with invalid profile fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode()\n\t\tres := node.RunIPFS(\"init\", \"--profile=invalid_profile\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Equal(t, \"Error: invalid configuration profile: invalid_profile\", res.Stderr.Trimmed())\n\t})\n\n\tt.Run(\"ipfs init --profile with valid profile succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode()\n\t\tnode.IPFS(\"init\", \"--profile=server\")\n\t})\n\n\tt.Run(\"ipfs config looks good\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=server\")\n\n\t\tlines := node.IPFS(\"config\", \"Swarm.AddrFilters\").Stdout.Lines()\n\t\tassert.Len(t, lines, 18)\n\n\t\tout := node.IPFS(\"config\", \"Bootstrap\").Stdout.Trimmed()\n\t\tassert.Equal(t, \"[]\", out)\n\n\t\tout = node.IPFS(\"config\", \"Addresses.API\").Stdout.Trimmed()\n\t\tassert.Equal(t, \"/ip4/127.0.0.1/tcp/0\", out)\n\t})\n\n\tt.Run(\"ipfs init from existing config succeeds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2)\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tnode1.Init(\"--profile=server\")\n\n\t\tnode2.IPFS(\"init\", fp.Join(node1.Dir, \"config\"))\n\t\tout := node2.IPFS(\"config\", \"Addresses.API\").Stdout.Trimmed()\n\t\tassert.Equal(t, \"/ip4/127.0.0.1/tcp/0\", out)\n\t})\n\n\tt.Run(\"ipfs init should not run while daemon is running\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tres := node.RunIPFS(\"init\")\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"Error: ipfs daemon is running. please stop it to run this command\")\n\t})\n}\n" }, { "path": "test/cli/ipfswatch_test.go", "content": "//go:build !plan9\n\npackage cli\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestIPFSWatch(t *testing.T) {\n\tt.Parallel()\n\n\t// Build ipfswatch binary once before running parallel subtests.\n\t// This avoids race conditions and duplicate builds.\n\th := harness.NewT(t)\n\trepoRoot := filepath.Dir(filepath.Dir(filepath.Dir(h.IPFSBin)))\n\tipfswatchBin := filepath.Join(repoRoot, \"cmd\", \"ipfswatch\", \"ipfswatch\")\n\n\tif _, err := os.Stat(ipfswatchBin); os.IsNotExist(err) {\n\t\t// -C changes to repo root so go.mod is found\n\t\tcmd := exec.Command(\"go\", \"build\", \"-C\", repoRoot, \"-o\", ipfswatchBin, \"./cmd/ipfswatch\")\n\t\tout, err := cmd.CombinedOutput()\n\t\trequire.NoError(t, err, \"failed to build ipfswatch: %s\", string(out))\n\t}\n\n\tt.Run(\"ipfswatch adds watched files to IPFS\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\n\t\t// Create a temp directory to watch\n\t\twatchDir := filepath.Join(h.Dir, \"watch\")\n\t\terr := os.MkdirAll(watchDir, 0o755)\n\t\trequire.NoError(t, err)\n\n\t\t// Start ipfswatch in background\n\t\tresult := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: ipfswatchBin,\n\t\t\tArgs: []string{\"--repo\", node.Dir, \"--path\", watchDir},\n\t\t\tRunFunc: harness.RunFuncStart,\n\t\t})\n\t\trequire.NoError(t, result.Err, \"ipfswatch should start without error\")\n\t\tdefer func() {\n\t\t\tif result.Cmd.Process != nil {\n\t\t\t\t_ = result.Cmd.Process.Kill()\n\t\t\t\t_, _ = result.Cmd.Process.Wait()\n\t\t\t}\n\t\t}()\n\n\t\t// Wait for ipfswatch to initialize\n\t\ttime.Sleep(2 * time.Second)\n\n\t\t// Check for startup errors\n\t\tstderrStr := result.Stderr.String()\n\t\trequire.NotContains(t, stderrStr, \"unknown datastore type\", \"ipfswatch should recognize datastore plugins\")\n\n\t\t// Create a test file with unique content based on timestamp\n\t\ttestContent := fmt.Sprintf(\"ipfswatch test content generated at %s\", time.Now().Format(time.RFC3339Nano))\n\t\ttestFile := filepath.Join(watchDir, \"test.txt\")\n\t\terr = os.WriteFile(testFile, []byte(testContent), 0o644)\n\t\trequire.NoError(t, err)\n\n\t\t// Wait for ipfswatch to process the file and extract CID from log\n\t\t// Log format: \"added %s... key: %s\"\n\t\tcidPattern := regexp.MustCompile(`added .*/test\\.txt\\.\\.\\. key: (\\S+)`)\n\t\tvar cid string\n\t\tdeadline := time.Now().Add(10 * time.Second)\n\t\tfor time.Now().Before(deadline) {\n\t\t\tstderrStr = result.Stderr.String()\n\t\t\tif matches := cidPattern.FindStringSubmatch(stderrStr); len(matches) > 1 {\n\t\t\t\tcid = matches[1]\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t}\n\t\trequire.NotEmpty(t, cid, \"ipfswatch should have added test.txt and logged the CID, got stderr: %s\", stderrStr)\n\n\t\t// Kill ipfswatch to release the repo lock\n\t\tif result.Cmd.Process != nil {\n\t\t\tif err = result.Cmd.Process.Signal(os.Interrupt); err != nil {\n\t\t\t\t_ = result.Cmd.Process.Kill()\n\t\t\t}\n\t\t\t_, _ = result.Cmd.Process.Wait()\n\t\t}\n\n\t\t// Verify the content matches by reading it back via ipfs cat\n\t\tcatRes := node.RunIPFS(\"cat\", \"--offline\", cid)\n\t\trequire.Equal(t, 0, catRes.Cmd.ProcessState.ExitCode(),\n\t\t\t\"ipfs cat should succeed, cid=%s, stderr: %s\", cid, catRes.Stderr.String())\n\t\trequire.Equal(t, testContent, catRes.Stdout.String(),\n\t\t\t\"content read from IPFS should match what was written\")\n\t})\n\n\tt.Run(\"ipfswatch loads datastore plugins for pebbleds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\n\t\t// Configure pebbleds as the datastore\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Datastore.Spec = map[string]any{\n\t\t\t\t\"type\": \"mount\",\n\t\t\t\t\"mounts\": []any{\n\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\"mountpoint\": \"/blocks\",\n\t\t\t\t\t\t\"path\": \"blocks\",\n\t\t\t\t\t\t\"prefix\": \"flatfs.datastore\",\n\t\t\t\t\t\t\"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n\t\t\t\t\t\t\"sync\": true,\n\t\t\t\t\t\t\"type\": \"flatfs\",\n\t\t\t\t\t},\n\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\"mountpoint\": \"/\",\n\t\t\t\t\t\t\"path\": \"datastore\",\n\t\t\t\t\t\t\"prefix\": \"pebble.datastore\",\n\t\t\t\t\t\t\"type\": \"pebbleds\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t})\n\n\t\t// Re-initialize datastore directory for pebbleds\n\t\t// (the repo was initialized with levelds, need to remove it)\n\t\tdsPath := filepath.Join(node.Dir, \"datastore\")\n\t\terr := os.RemoveAll(dsPath)\n\t\trequire.NoError(t, err)\n\t\terr = os.MkdirAll(dsPath, 0o755)\n\t\trequire.NoError(t, err)\n\n\t\t// Create a temp directory to watch\n\t\twatchDir := filepath.Join(h.Dir, \"watch\")\n\t\terr = os.MkdirAll(watchDir, 0o755)\n\t\trequire.NoError(t, err)\n\n\t\t// Start ipfswatch in background\n\t\tresult := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: ipfswatchBin,\n\t\t\tArgs: []string{\"--repo\", node.Dir, \"--path\", watchDir},\n\t\t\tRunFunc: harness.RunFuncStart,\n\t\t})\n\t\trequire.NoError(t, result.Err, \"ipfswatch should start without error\")\n\t\tdefer func() {\n\t\t\tif result.Cmd.Process != nil {\n\t\t\t\t_ = result.Cmd.Process.Kill()\n\t\t\t\t_, _ = result.Cmd.Process.Wait()\n\t\t\t}\n\t\t}()\n\n\t\t// Wait for ipfswatch to initialize and check for errors\n\t\ttime.Sleep(3 * time.Second)\n\n\t\tstderrStr := result.Stderr.String()\n\t\trequire.NotContains(t, stderrStr, \"unknown datastore type\", \"ipfswatch should recognize pebbleds datastore plugin\")\n\t})\n}\n" }, { "path": "test/cli/log_level_test.go", "content": "package cli\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestLogLevel(t *testing.T) {\n\n\tt.Run(\"CLI\", func(t *testing.T) {\n\t\tt.Run(\"level '*' shows all subsystems\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\texpectedSubsystems := getExpectedSubsystems(t, node)\n\n\t\t\tres := node.IPFS(\"log\", \"level\", \"*\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Empty(t, res.Stderr.Lines())\n\n\t\t\tactualSubsystems := parseCLIOutput(t, res.Stdout.String())\n\n\t\t\t// Should show all subsystems plus the (default) entry\n\t\t\tassert.GreaterOrEqual(t, len(actualSubsystems), len(expectedSubsystems))\n\n\t\t\tvalidateAllSubsystemsPresentCLI(t, expectedSubsystems, actualSubsystems, \"CLI output\")\n\n\t\t\t// Should have the (default) entry\n\t\t\t_, hasDefault := actualSubsystems[\"(default)\"]\n\t\t\tassert.True(t, hasDefault, \"Should have '(default)' entry\")\n\t\t})\n\n\t\tt.Run(\"level 'all' shows all subsystems (alias for '*')\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\texpectedSubsystems := getExpectedSubsystems(t, node)\n\n\t\t\tres := node.IPFS(\"log\", \"level\", \"all\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Empty(t, res.Stderr.Lines())\n\n\t\t\tactualSubsystems := parseCLIOutput(t, res.Stdout.String())\n\n\t\t\t// Should show all subsystems plus the (default) entry\n\t\t\tassert.GreaterOrEqual(t, len(actualSubsystems), len(expectedSubsystems))\n\n\t\t\tvalidateAllSubsystemsPresentCLI(t, expectedSubsystems, actualSubsystems, \"CLI output\")\n\n\t\t\t// Should have the (default) entry\n\t\t\t_, hasDefault := actualSubsystems[\"(default)\"]\n\t\t\tassert.True(t, hasDefault, \"Should have '(default)' entry\")\n\t\t})\n\n\t\tt.Run(\"get level for specific subsystem\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tnode.IPFS(\"log\", \"level\", \"core\", \"debug\")\n\t\t\tres := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Empty(t, res.Stderr.Lines())\n\n\t\t\toutput := res.Stdout.String()\n\t\t\tlines := SplitLines(output)\n\n\t\t\tassert.Equal(t, 1, len(lines))\n\n\t\t\tline := strings.TrimSpace(lines[0])\n\t\t\tassert.Equal(t, \"debug\", line)\n\t\t})\n\n\t\tt.Run(\"get level with no args returns default level\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tres1 := node.IPFS(\"log\", \"level\", \"*\", \"fatal\")\n\t\t\tassert.NoError(t, res1.Err)\n\t\t\tassert.Empty(t, res1.Stderr.Lines())\n\n\t\t\tres := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, 0, len(res.Stderr.Lines()))\n\n\t\t\toutput := res.Stdout.String()\n\t\t\tlines := SplitLines(output)\n\n\t\t\tassert.Equal(t, 1, len(lines))\n\n\t\t\tline := strings.TrimSpace(lines[0])\n\t\t\tassert.Equal(t, \"fatal\", line)\n\t\t})\n\n\t\tt.Run(\"get level reflects runtime log level changes\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tnode.IPFS(\"log\", \"level\", \"core\", \"debug\")\n\t\t\tres := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res.Err)\n\n\t\t\toutput := res.Stdout.String()\n\t\t\tlines := SplitLines(output)\n\n\t\t\tassert.Equal(t, 1, len(lines))\n\n\t\t\tline := strings.TrimSpace(lines[0])\n\t\t\tassert.Equal(t, \"debug\", line)\n\t\t})\n\n\t\tt.Run(\"get level with non-existent subsystem returns error\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tres := node.RunIPFS(\"log\", \"level\", \"non-existent-subsystem\")\n\t\t\tassert.Error(t, res.Err)\n\t\t\tassert.NotEqual(t, 0, len(res.Stderr.Lines()))\n\t\t})\n\n\t\tt.Run(\"set level to 'default' keyword\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// First set a specific subsystem to a different level\n\t\t\tres1 := node.IPFS(\"log\", \"level\", \"core\", \"debug\")\n\t\t\tassert.NoError(t, res1.Err)\n\t\t\tassert.Contains(t, res1.Stdout.String(), \"Changed log level of 'core' to 'debug'\")\n\n\t\t\t// Verify it was set to debug\n\t\t\tres2 := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res2.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res2.Stdout.String()))\n\n\t\t\t// Get the current default level (should be 'error' since unchanged)\n\t\t\tres3 := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res3.Err)\n\t\t\tdefaultLevel := strings.TrimSpace(res3.Stdout.String())\n\t\t\tassert.Equal(t, \"error\", defaultLevel, \"Default level should be 'error' when unchanged\")\n\n\t\t\t// Now set the subsystem back to default\n\t\t\tres4 := node.IPFS(\"log\", \"level\", \"core\", \"default\")\n\t\t\tassert.NoError(t, res4.Err)\n\t\t\tassert.Contains(t, res4.Stdout.String(), \"Changed log level of 'core' to\")\n\n\t\t\t// Verify it's now at the default level (should be 'error')\n\t\t\tres5 := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res5.Err)\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res5.Stdout.String()))\n\t\t})\n\n\t\tt.Run(\"set all subsystems with 'all' changes default (alias for '*')\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Initial state - default should be 'error'\n\t\t\tres := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Set one subsystem to a different level\n\t\t\tres = node.IPFS(\"log\", \"level\", \"core\", \"debug\")\n\t\t\tassert.NoError(t, res.Err)\n\n\t\t\t// Default should still be 'error'\n\t\t\tres = node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Now use 'all' to set everything to 'info'\n\t\t\tres = node.IPFS(\"log\", \"level\", \"all\", \"info\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Contains(t, res.Stdout.String(), \"Changed log level of '*' to 'info'\")\n\n\t\t\t// Default should now be 'info'\n\t\t\tres = node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"info\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Core should also be 'info' (overwritten by 'all')\n\t\t\tres = node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"info\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Any other subsystem should also be 'info'\n\t\t\tres = node.IPFS(\"log\", \"level\", \"dht\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"info\", strings.TrimSpace(res.Stdout.String()))\n\t\t})\n\n\t\tt.Run(\"set all subsystems with '*' changes default\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Initial state - default should be 'error'\n\t\t\tres := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Set one subsystem to a different level\n\t\t\tres = node.IPFS(\"log\", \"level\", \"core\", \"debug\")\n\t\t\tassert.NoError(t, res.Err)\n\n\t\t\t// Default should still be 'error'\n\t\t\tres = node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Now use '*' to set everything to 'info'\n\t\t\tres = node.IPFS(\"log\", \"level\", \"*\", \"info\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Contains(t, res.Stdout.String(), \"Changed log level of '*' to 'info'\")\n\n\t\t\t// Default should now be 'info'\n\t\t\tres = node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"info\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Core should also be 'info' (overwritten by '*')\n\t\t\tres = node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"info\", strings.TrimSpace(res.Stdout.String()))\n\n\t\t\t// Any other subsystem should also be 'info'\n\t\t\tres = node.IPFS(\"log\", \"level\", \"dht\")\n\t\t\tassert.NoError(t, res.Err)\n\t\t\tassert.Equal(t, \"info\", strings.TrimSpace(res.Stdout.String()))\n\t\t})\n\n\t\tt.Run(\"'all' in get mode shows (default) entry (alias for '*')\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Get all levels with 'all'\n\t\t\tres := node.IPFS(\"log\", \"level\", \"all\")\n\t\t\tassert.NoError(t, res.Err)\n\n\t\t\toutput := res.Stdout.String()\n\n\t\t\t// Should contain \"(default): error\" entry\n\t\t\tassert.Contains(t, output, \"(default): error\", \"Should show default level with (default) key\")\n\n\t\t\t// Should also contain various subsystems\n\t\t\tassert.Contains(t, output, \"core: error\")\n\t\t\tassert.Contains(t, output, \"dht: error\")\n\t\t})\n\n\t\tt.Run(\"'*' in get mode shows (default) entry\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Get all levels with '*'\n\t\t\tres := node.IPFS(\"log\", \"level\", \"*\")\n\t\t\tassert.NoError(t, res.Err)\n\n\t\t\toutput := res.Stdout.String()\n\n\t\t\t// Should contain \"(default): error\" entry\n\t\t\tassert.Contains(t, output, \"(default): error\", \"Should show default level with (default) key\")\n\n\t\t\t// Should also contain various subsystems\n\t\t\tassert.Contains(t, output, \"core: error\")\n\t\t\tassert.Contains(t, output, \"dht: error\")\n\t\t})\n\n\t\tt.Run(\"set all subsystems to 'default' using 'all' (alias for '*')\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Get the original default level (just for reference, it should be \"error\")\n\t\t\tres0 := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res0.Err)\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res0.Stdout.String()))\n\n\t\t\t// First set all subsystems to debug using 'all'\n\t\t\tres1 := node.IPFS(\"log\", \"level\", \"all\", \"debug\")\n\t\t\tassert.NoError(t, res1.Err)\n\t\t\tassert.Contains(t, res1.Stdout.String(), \"Changed log level of '*' to 'debug'\")\n\n\t\t\t// Verify a specific subsystem is at debug\n\t\t\tres2 := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res2.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res2.Stdout.String()))\n\n\t\t\t// Verify the default level is now debug\n\t\t\tres3 := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res3.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res3.Stdout.String()))\n\n\t\t\t// Now set all subsystems back to default (which is now \"debug\") using 'all'\n\t\t\tres4 := node.IPFS(\"log\", \"level\", \"all\", \"default\")\n\t\t\tassert.NoError(t, res4.Err)\n\t\t\tassert.Contains(t, res4.Stdout.String(), \"Changed log level of '*' to\")\n\n\t\t\t// The subsystem should still be at debug (because that's what default is now)\n\t\t\tres5 := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res5.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res5.Stdout.String()))\n\n\t\t\t// The behavior is correct: \"default\" uses the current default level,\n\t\t\t// which was changed to \"debug\" when we set \"all\" to \"debug\"\n\t\t})\n\n\t\tt.Run(\"set all subsystems to 'default' keyword\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Get the original default level (just for reference, it should be \"error\")\n\t\t\tres0 := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res0.Err)\n\t\t\t// originalDefault := strings.TrimSpace(res0.Stdout.String())\n\t\t\tassert.Equal(t, \"error\", strings.TrimSpace(res0.Stdout.String()))\n\n\t\t\t// First set all subsystems to debug\n\t\t\tres1 := node.IPFS(\"log\", \"level\", \"*\", \"debug\")\n\t\t\tassert.NoError(t, res1.Err)\n\t\t\tassert.Contains(t, res1.Stdout.String(), \"Changed log level of '*' to 'debug'\")\n\n\t\t\t// Verify a specific subsystem is at debug\n\t\t\tres2 := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res2.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res2.Stdout.String()))\n\n\t\t\t// Verify the default level is now debug\n\t\t\tres3 := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res3.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res3.Stdout.String()))\n\n\t\t\t// Now set all subsystems back to default (which is now \"debug\")\n\t\t\tres4 := node.IPFS(\"log\", \"level\", \"*\", \"default\")\n\t\t\tassert.NoError(t, res4.Err)\n\t\t\tassert.Contains(t, res4.Stdout.String(), \"Changed log level of '*' to\")\n\n\t\t\t// The subsystem should still be at debug (because that's what default is now)\n\t\t\tres5 := node.IPFS(\"log\", \"level\", \"core\")\n\t\t\tassert.NoError(t, res5.Err)\n\t\t\tassert.Equal(t, \"debug\", strings.TrimSpace(res5.Stdout.String()))\n\n\t\t\t// The behavior is correct: \"default\" uses the current default level,\n\t\t\t// which was changed to \"debug\" when we set \"*\" to \"debug\"\n\t\t})\n\n\t\tt.Run(\"shell escaping variants for '*' wildcard\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\th := harness.NewT(t)\n\t\t\tnode := h.NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Test different shell escaping methods work for '*'\n\t\t\t// This tests the behavior documented in help text: '*' or \"*\" or \\*\n\n\t\t\t// Test 1: Single quotes '*' (should work)\n\t\t\tcmd1 := fmt.Sprintf(\"IPFS_PATH='%s' %s --api='%s' log level '*' info\",\n\t\t\t\tnode.Dir, node.IPFSBin, node.APIAddr())\n\t\t\tres1 := h.Sh(cmd1)\n\t\t\tassert.NoError(t, res1.Err)\n\t\t\tassert.Contains(t, res1.Stdout.String(), \"Changed log level of '*' to 'info'\")\n\n\t\t\t// Test 2: Double quotes \"*\" (should work)\n\t\t\tcmd2 := fmt.Sprintf(\"IPFS_PATH='%s' %s --api='%s' log level \\\"*\\\" debug\",\n\t\t\t\tnode.Dir, node.IPFSBin, node.APIAddr())\n\t\t\tres2 := h.Sh(cmd2)\n\t\t\tassert.NoError(t, res2.Err)\n\t\t\tassert.Contains(t, res2.Stdout.String(), \"Changed log level of '*' to 'debug'\")\n\n\t\t\t// Test 3: Backslash escape \\* (should work)\n\t\t\tcmd3 := fmt.Sprintf(\"IPFS_PATH='%s' %s --api='%s' log level \\\\* warn\",\n\t\t\t\tnode.Dir, node.IPFSBin, node.APIAddr())\n\t\t\tres3 := h.Sh(cmd3)\n\t\t\tassert.NoError(t, res3.Err)\n\t\t\tassert.Contains(t, res3.Stdout.String(), \"Changed log level of '*' to 'warn'\")\n\n\t\t\t// Test 4: Verify the final state - should show 'warn' as default\n\t\t\tres4 := node.IPFS(\"log\", \"level\")\n\t\t\tassert.NoError(t, res4.Err)\n\t\t\tassert.Equal(t, \"warn\", strings.TrimSpace(res4.Stdout.String()))\n\n\t\t\t// Test 5: Get all levels using escaped '*' to verify it shows all subsystems\n\t\t\tcmd5 := fmt.Sprintf(\"IPFS_PATH='%s' %s --api='%s' log level \\\\*\",\n\t\t\t\tnode.Dir, node.IPFSBin, node.APIAddr())\n\t\t\tres5 := h.Sh(cmd5)\n\t\t\tassert.NoError(t, res5.Err)\n\t\t\toutput := res5.Stdout.String()\n\t\t\tassert.Contains(t, output, \"(default): warn\", \"Should show updated default level\")\n\t\t\tassert.Contains(t, output, \"core: warn\", \"Should show core subsystem at warn level\")\n\t\t})\n\t})\n\n\tt.Run(\"HTTP RPC\", func(t *testing.T) {\n\t\tt.Run(\"get default level returns JSON\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Make HTTP request to get default log level\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\t// Parse JSON response\n\t\t\tvar result map[string]any\n\t\t\terr = json.NewDecoder(resp.Body).Decode(&result)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Check that we have the Levels field\n\t\t\tlevels, ok := result[\"Levels\"].(map[string]any)\n\t\t\trequire.True(t, ok, \"Response should have 'Levels' field\")\n\n\t\t\t// Should have exactly one entry for the default level\n\t\t\tassert.Equal(t, 1, len(levels))\n\n\t\t\t// The default level should be present\n\t\t\tdefaultLevel, ok := levels[\"\"]\n\t\t\trequire.True(t, ok, \"Should have empty string key for default level\")\n\t\t\tassert.Equal(t, \"error\", defaultLevel, \"Default level should be 'error'\")\n\t\t})\n\n\t\tt.Run(\"get all levels using 'all' returns JSON (alias for '*')\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\texpectedSubsystems := getExpectedSubsystems(t, node)\n\n\t\t\t// Make HTTP request to get all log levels using 'all'\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level?arg=all\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\tlevels := parseHTTPResponse(t, resp)\n\t\t\tvalidateAllSubsystemsPresent(t, expectedSubsystems, levels, \"JSON response\")\n\n\t\t\t// Should have the (default) entry\n\t\t\tdefaultLevel, ok := levels[\"(default)\"]\n\t\t\trequire.True(t, ok, \"Should have '(default)' key\")\n\t\t\tassert.Equal(t, \"error\", defaultLevel, \"Default level should be 'error'\")\n\t\t})\n\n\t\tt.Run(\"get all levels returns JSON\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\texpectedSubsystems := getExpectedSubsystems(t, node)\n\n\t\t\t// Make HTTP request to get all log levels\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level?arg=*\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\tlevels := parseHTTPResponse(t, resp)\n\t\t\tvalidateAllSubsystemsPresent(t, expectedSubsystems, levels, \"JSON response\")\n\n\t\t\t// Should have the (default) entry\n\t\t\tdefaultLevel, ok := levels[\"(default)\"]\n\t\t\trequire.True(t, ok, \"Should have '(default)' key\")\n\t\t\tassert.Equal(t, \"error\", defaultLevel, \"Default level should be 'error'\")\n\t\t})\n\n\t\tt.Run(\"get specific subsystem level returns JSON\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// First set a specific level for a subsystem\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level?arg=core&arg=debug\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tresp.Body.Close()\n\n\t\t\t// Now get the level for that subsystem\n\t\t\tresp, err = http.Post(node.APIURL()+\"/api/v0/log/level?arg=core\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\t// Parse JSON response\n\t\t\tvar result map[string]any\n\t\t\terr = json.NewDecoder(resp.Body).Decode(&result)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Check that we have the Levels field\n\t\t\tlevels, ok := result[\"Levels\"].(map[string]any)\n\t\t\trequire.True(t, ok, \"Response should have 'Levels' field\")\n\n\t\t\t// Should have exactly one entry\n\t\t\tassert.Equal(t, 1, len(levels))\n\n\t\t\t// Check the level for 'core' subsystem\n\t\t\tcoreLevel, ok := levels[\"core\"]\n\t\t\trequire.True(t, ok, \"Should have 'core' key\")\n\t\t\tassert.Equal(t, \"debug\", coreLevel, \"Core level should be 'debug'\")\n\t\t})\n\n\t\tt.Run(\"set level using 'all' returns JSON message (alias for '*')\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Set a log level using 'all'\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level?arg=all&arg=info\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\t// Parse JSON response\n\t\t\tvar result map[string]any\n\t\t\terr = json.NewDecoder(resp.Body).Decode(&result)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Check that we have the Message field\n\t\t\tmessage, ok := result[\"Message\"].(string)\n\t\t\trequire.True(t, ok, \"Response should have 'Message' field\")\n\n\t\t\t// Check the message content (should show '*' in message even when 'all' was used)\n\t\t\tassert.Contains(t, message, \"Changed log level of '*' to 'info'\")\n\t\t})\n\n\t\tt.Run(\"set level returns JSON message\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// Set a log level\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level?arg=core&arg=info\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\t// Parse JSON response\n\t\t\tvar result map[string]any\n\t\t\terr = json.NewDecoder(resp.Body).Decode(&result)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Check that we have the Message field\n\t\t\tmessage, ok := result[\"Message\"].(string)\n\t\t\trequire.True(t, ok, \"Response should have 'Message' field\")\n\n\t\t\t// Check the message content\n\t\t\tassert.Contains(t, message, \"Changed log level of 'core' to 'info'\")\n\t\t})\n\n\t\tt.Run(\"set level to 'default' keyword\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\t// First set a subsystem to debug\n\t\t\tresp, err := http.Post(node.APIURL()+\"/api/v0/log/level?arg=core&arg=debug\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tresp.Body.Close()\n\n\t\t\t// Now set it back to default\n\t\t\tresp, err = http.Post(node.APIURL()+\"/api/v0/log/level?arg=core&arg=default\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\t// Parse JSON response\n\t\t\tvar result map[string]any\n\t\t\terr = json.NewDecoder(resp.Body).Decode(&result)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Check that we have the Message field\n\t\t\tmessage, ok := result[\"Message\"].(string)\n\t\t\trequire.True(t, ok, \"Response should have 'Message' field\")\n\n\t\t\t// The message should indicate the change\n\t\t\tassert.True(t, strings.Contains(message, \"Changed log level of 'core' to\"),\n\t\t\t\t\"Message should indicate level change\")\n\n\t\t\t// Verify the level is back to error (default)\n\t\t\tresp, err = http.Post(node.APIURL()+\"/api/v0/log/level?arg=core\", \"\", nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\tvar getResult map[string]any\n\t\t\terr = json.NewDecoder(resp.Body).Decode(&getResult)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tlevels, _ := getResult[\"Levels\"].(map[string]any)\n\t\t\tcoreLevel, _ := levels[\"core\"].(string)\n\t\t\tassert.Equal(t, \"error\", coreLevel, \"Core level should be back to 'error' (default)\")\n\t\t})\n\t})\n\n\t// Constants for slog interop tests\n\tconst (\n\t\tslogTestLogTailTimeout = 10 * time.Second\n\t\tslogTestLogWaitTimeout = 5 * time.Second\n\t\tslogTestLogStartupDelay = 1 * time.Second // Wait for log tail to start\n\t\tslogTestSubsystemCmdsHTTP = \"cmds/http\" // Native go-log subsystem\n\t\tslogTestSubsystemNetIdentify = \"net/identify\" // go-libp2p slog subsystem\n\t)\n\n\t// logMatch represents a matched log entry for slog interop tests\n\ttype logMatch struct {\n\t\tsubsystem string\n\t\tline string\n\t}\n\n\t// startLogMonitoring starts ipfs log tail and returns command and channel for matched logs.\n\tstartLogMonitoring := func(t *testing.T, node *harness.Node) (*exec.Cmd, chan logMatch) {\n\t\tt.Helper()\n\n\t\tctx, cancel := context.WithTimeout(context.Background(), slogTestLogTailTimeout)\n\t\tt.Cleanup(cancel)\n\n\t\tcmd := exec.CommandContext(ctx, node.IPFSBin, \"log\", \"tail\")\n\t\tcmd.Env = append([]string(nil), os.Environ()...)\n\t\tfor k, v := range node.Runner.Env {\n\t\t\tcmd.Env = append(cmd.Env, fmt.Sprintf(\"%s=%s\", k, v))\n\t\t}\n\t\tcmd.Dir = node.Runner.Dir\n\n\t\tstdout, err := cmd.StdoutPipe()\n\t\trequire.NoError(t, err)\n\t\trequire.NoError(t, cmd.Start())\n\n\t\tmatches := make(chan logMatch, 10)\n\n\t\tgo func() {\n\t\t\tscanner := bufio.NewScanner(stdout)\n\t\t\tfor scanner.Scan() {\n\t\t\t\tline := scanner.Text()\n\t\t\t\t// Check for actual logger field in JSON, not just substring match\n\t\t\t\tif strings.Contains(line, `\"logger\":\"cmds/http\"`) {\n\t\t\t\t\tmatches <- logMatch{slogTestSubsystemCmdsHTTP, line}\n\t\t\t\t}\n\t\t\t\tif strings.Contains(line, `\"logger\":\"net/identify\"`) {\n\t\t\t\t\tmatches <- logMatch{slogTestSubsystemNetIdentify, line}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\treturn cmd, matches\n\t}\n\n\t// waitForBothSubsystems waits for both native go-log and slog subsystems to appear in logs.\n\twaitForBothSubsystems := func(t *testing.T, matches chan logMatch, timeout time.Duration) {\n\t\tt.Helper()\n\n\t\tseen := make(map[string]struct{})\n\t\tdeadline := time.After(timeout)\n\n\t\tfor len(seen) < 2 {\n\t\t\tselect {\n\t\t\tcase match := <-matches:\n\t\t\t\tif _, exists := seen[match.subsystem]; !exists {\n\t\t\t\t\tt.Logf(\"Found %s log\", match.subsystem)\n\t\t\t\t\tseen[match.subsystem] = struct{}{}\n\t\t\t\t}\n\t\t\tcase <-deadline:\n\t\t\t\tt.Fatalf(\"Timeout waiting for logs. Seen: %v\", seen)\n\t\t\t}\n\t\t}\n\n\t\tassert.Contains(t, seen, slogTestSubsystemCmdsHTTP, \"should see cmds/http (native go-log)\")\n\t\tassert.Contains(t, seen, slogTestSubsystemNetIdentify, \"should see net/identify (slog from go-libp2p)\")\n\t}\n\n\t// triggerIdentifyProtocol connects node1 to node2, triggering net/identify logs.\n\ttriggerIdentifyProtocol := func(t *testing.T, node1, node2 *harness.Node) {\n\t\tt.Helper()\n\n\t\t// Get node2's peer ID and address\n\t\tnode2ID := node2.PeerID().String()\n\t\taddrsRes := node2.IPFS(\"id\", \"-f\", \"\")\n\t\trequire.NoError(t, addrsRes.Err)\n\n\t\taddrs := strings.Split(strings.TrimSpace(addrsRes.Stdout.String()), \"\\n\")\n\t\trequire.NotEmpty(t, addrs, \"node2 should have at least one address\")\n\n\t\t// Connect node1 to node2\n\t\tmultiaddr := fmt.Sprintf(\"%s/p2p/%s\", addrs[0], node2ID)\n\t\tres := node1.IPFS(\"swarm\", \"connect\", multiaddr)\n\t\trequire.NoError(t, res.Err)\n\t}\n\n\t// verifySlogInterop verifies that both native go-log and slog from go-libp2p\n\t// appear in ipfs log tail with correct formatting and level control.\n\tverifySlogInterop := func(t *testing.T, node1, node2 *harness.Node) {\n\t\tt.Helper()\n\n\t\tcmd, matches := startLogMonitoring(t, node1)\n\t\tdefer func() {\n\t\t\t_ = cmd.Process.Kill()\n\t\t}()\n\n\t\ttime.Sleep(slogTestLogStartupDelay)\n\n\t\t// Trigger cmds/http (native go-log)\n\t\tnode1.IPFS(\"version\")\n\n\t\t// Trigger net/identify (slog from go-libp2p)\n\t\ttriggerIdentifyProtocol(t, node1, node2)\n\n\t\twaitForBothSubsystems(t, matches, slogTestLogWaitTimeout)\n\t}\n\n\t// This test verifies that go-log's slog bridge works with go-libp2p's gologshim\n\t// when log levels are set via GOLOG_LOG_LEVEL environment variable.\n\t// It tests both native go-log loggers (cmds/http) and slog-based loggers from\n\t// go-libp2p (net/identify), ensuring both types appear in `ipfs log tail`.\n\tt.Run(\"slog interop via env var\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\tnode1 := h.NewNode().Init()\n\t\tnode1.Runner.Env[\"GOLOG_LOG_LEVEL\"] = \"error,cmds/http=debug,net/identify=debug\"\n\t\tnode1.StartDaemon()\n\t\tdefer node1.StopDaemon()\n\n\t\tnode2 := h.NewNode().Init().StartDaemon()\n\t\tdefer node2.StopDaemon()\n\n\t\tverifySlogInterop(t, node1, node2)\n\t})\n\n\t// This test verifies that go-log's slog bridge works with go-libp2p's gologshim\n\t// when log levels are set dynamically via `ipfs log level` CLI commands.\n\t// It tests the key feature that SetLogLevel auto-creates level entries for subsystems\n\t// that don't exist yet, enabling `ipfs log level net/identify debug` to work even\n\t// before the net/identify logger is created. This is critical for slog interop.\n\tt.Run(\"slog interop via CLI\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\tnode1 := h.NewNode().Init().StartDaemon()\n\t\tdefer node1.StopDaemon()\n\n\t\tnode2 := h.NewNode().Init().StartDaemon()\n\t\tdefer node2.StopDaemon()\n\n\t\t// Set levels via CLI for both subsystems BEFORE triggering events\n\t\tres := node1.IPFS(\"log\", \"level\", slogTestSubsystemCmdsHTTP, \"debug\")\n\t\trequire.NoError(t, res.Err)\n\n\t\tres = node1.IPFS(\"log\", \"level\", slogTestSubsystemNetIdentify, \"debug\")\n\t\trequire.NoError(t, res.Err) // Auto-creates level entry for slog subsystem\n\n\t\tverifySlogInterop(t, node1, node2)\n\t})\n\n}\n\nfunc getExpectedSubsystems(t *testing.T, node *harness.Node) []string {\n\tt.Helper()\n\tlsRes := node.IPFS(\"log\", \"ls\")\n\trequire.NoError(t, lsRes.Err)\n\texpectedSubsystems := SplitLines(lsRes.Stdout.String())\n\tassert.Greater(t, len(expectedSubsystems), 10, \"Should have many subsystems\")\n\treturn expectedSubsystems\n}\n\nfunc parseCLIOutput(t *testing.T, output string) map[string]string {\n\tt.Helper()\n\tlines := SplitLines(output)\n\tactualSubsystems := make(map[string]string)\n\tfor _, line := range lines {\n\t\tif strings.TrimSpace(line) == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tparts := strings.Split(line, \": \")\n\t\tassert.Equal(t, 2, len(parts), \"Line should have format 'subsystem: level', got: %s\", line)\n\t\tassert.NotEmpty(t, parts[0], \"Subsystem should not be empty\")\n\t\tassert.NotEmpty(t, parts[1], \"Level should not be empty\")\n\t\tactualSubsystems[parts[0]] = parts[1]\n\t}\n\treturn actualSubsystems\n}\n\nfunc parseHTTPResponse(t *testing.T, resp *http.Response) map[string]any {\n\tt.Helper()\n\tvar result map[string]any\n\terr := json.NewDecoder(resp.Body).Decode(&result)\n\trequire.NoError(t, err)\n\tlevels, ok := result[\"Levels\"].(map[string]any)\n\trequire.True(t, ok, \"Response should have 'Levels' field\")\n\tassert.Greater(t, len(levels), 10, \"Should have many subsystems\")\n\treturn levels\n}\n\nfunc validateAllSubsystemsPresent(t *testing.T, expectedSubsystems []string, actualLevels map[string]any, context string) {\n\tt.Helper()\n\tfor _, expectedSub := range expectedSubsystems {\n\t\texpectedSub = strings.TrimSpace(expectedSub)\n\t\tif expectedSub == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\t_, found := actualLevels[expectedSub]\n\t\tassert.True(t, found, \"Expected subsystem '%s' should be present in %s\", expectedSub, context)\n\t}\n}\n\nfunc validateAllSubsystemsPresentCLI(t *testing.T, expectedSubsystems []string, actualLevels map[string]string, context string) {\n\tt.Helper()\n\tfor _, expectedSub := range expectedSubsystems {\n\t\texpectedSub = strings.TrimSpace(expectedSub)\n\t\tif expectedSub == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\t_, found := actualLevels[expectedSub]\n\t\tassert.True(t, found, \"Expected subsystem '%s' should be present in %s\", expectedSub, context)\n\t}\n}\n" }, { "path": "test/cli/ls_test.go", "content": "package cli\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestLsLongFormat(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"long format shows mode and mtime when preserved\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a test directory structure with known permissions\n\t\ttestDir := filepath.Join(node.Dir, \"testdata\")\n\t\trequire.NoError(t, os.MkdirAll(testDir, 0755))\n\n\t\t// Create files with specific permissions\n\t\tfile1 := filepath.Join(testDir, \"readable.txt\")\n\t\trequire.NoError(t, os.WriteFile(file1, []byte(\"hello\"), 0644))\n\n\t\tfile2 := filepath.Join(testDir, \"executable.sh\")\n\t\trequire.NoError(t, os.WriteFile(file2, []byte(\"#!/bin/sh\\necho hi\"), 0755))\n\n\t\t// Set a known mtime in the past (to get year format, avoiding flaky time-based tests)\n\t\toldTime := time.Date(2020, time.June, 15, 10, 30, 0, 0, time.UTC)\n\t\trequire.NoError(t, os.Chtimes(file1, oldTime, oldTime))\n\t\trequire.NoError(t, os.Chtimes(file2, oldTime, oldTime))\n\n\t\t// Add with preserved mode and mtime\n\t\taddRes := node.IPFS(\"add\", \"-r\", \"--preserve-mode\", \"--preserve-mtime\", \"-Q\", testDir)\n\t\tdirCid := addRes.Stdout.Trimmed()\n\n\t\t// Run ls with --long flag\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", dirCid)\n\t\toutput := lsRes.Stdout.String()\n\n\t\t// Verify format: Mode Hash Size ModTime Name\n\t\tlines := strings.Split(strings.TrimSpace(output), \"\\n\")\n\t\trequire.Len(t, lines, 2, \"expected 2 files in output\")\n\n\t\t// Check executable.sh line (should be first alphabetically)\n\t\tassert.Contains(t, lines[0], \"-rwxr-xr-x\", \"executable should have 755 permissions\")\n\t\tassert.Contains(t, lines[0], \"Jun 15 2020\", \"should show mtime with year format\")\n\t\tassert.Contains(t, lines[0], \"executable.sh\", \"should show filename\")\n\n\t\t// Check readable.txt line\n\t\tassert.Contains(t, lines[1], \"-rw-r--r--\", \"readable file should have 644 permissions\")\n\t\tassert.Contains(t, lines[1], \"Jun 15 2020\", \"should show mtime with year format\")\n\t\tassert.Contains(t, lines[1], \"readable.txt\", \"should show filename\")\n\t})\n\n\tt.Run(\"long format shows dash for files without preserved mode or mtime\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create and add a file without --preserve-mode or --preserve-mtime\n\t\ttestFile := filepath.Join(node.Dir, \"nopreserve.txt\")\n\t\trequire.NoError(t, os.WriteFile(testFile, []byte(\"test content\"), 0644))\n\n\t\taddRes := node.IPFS(\"add\", \"-Q\", testFile)\n\t\tfileCid := addRes.Stdout.Trimmed()\n\n\t\t// Create a wrapper directory to list\n\t\tnode.IPFS(\"files\", \"mkdir\", \"/testdir\")\n\t\tnode.IPFS(\"files\", \"cp\", \"/ipfs/\"+fileCid, \"/testdir/file.txt\")\n\t\tstatRes := node.IPFS(\"files\", \"stat\", \"--hash\", \"/testdir\")\n\t\tdirCid := statRes.Stdout.Trimmed()\n\n\t\t// Run ls with --long flag\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", dirCid)\n\t\toutput := lsRes.Stdout.String()\n\n\t\t// Files without preserved mode or mtime should show \"-\" for both columns\n\t\t// Format: \"-\" (mode) \"-\" (mtime) \n\t\tassert.Regexp(t, `^-\\s+\\S+\\s+\\d+\\s+-\\s+`, output, \"missing mode and mtime should both show dash\")\n\t})\n\n\tt.Run(\"long format with headers shows correct column order\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create a simple test file\n\t\ttestDir := filepath.Join(node.Dir, \"headertest\")\n\t\trequire.NoError(t, os.MkdirAll(testDir, 0755))\n\t\ttestFile := filepath.Join(testDir, \"file.txt\")\n\t\trequire.NoError(t, os.WriteFile(testFile, []byte(\"hello\"), 0644))\n\n\t\toldTime := time.Date(2020, time.January, 1, 0, 0, 0, 0, time.UTC)\n\t\trequire.NoError(t, os.Chtimes(testFile, oldTime, oldTime))\n\n\t\taddRes := node.IPFS(\"add\", \"-r\", \"--preserve-mode\", \"--preserve-mtime\", \"-Q\", testDir)\n\t\tdirCid := addRes.Stdout.Trimmed()\n\n\t\t// Run ls with --long and --headers (--size defaults to true)\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", \"--headers\", dirCid)\n\t\toutput := lsRes.Stdout.String()\n\t\tlines := strings.Split(strings.TrimSpace(output), \"\\n\")\n\n\t\t// First line should be headers in correct order: Mode Hash Size ModTime Name\n\t\trequire.GreaterOrEqual(t, len(lines), 2)\n\t\theaderFields := strings.Fields(lines[0])\n\t\trequire.Len(t, headerFields, 5, \"header should have 5 columns\")\n\t\tassert.Equal(t, \"Mode\", headerFields[0])\n\t\tassert.Equal(t, \"Hash\", headerFields[1])\n\t\tassert.Equal(t, \"Size\", headerFields[2])\n\t\tassert.Equal(t, \"ModTime\", headerFields[3])\n\t\tassert.Equal(t, \"Name\", headerFields[4])\n\n\t\t// Data line should have matching columns\n\t\tdataFields := strings.Fields(lines[1])\n\t\trequire.GreaterOrEqual(t, len(dataFields), 5)\n\t\tassert.Regexp(t, `^-[rwx-]{9}$`, dataFields[0], \"first field should be mode\")\n\t\tassert.Regexp(t, `^Qm`, dataFields[1], \"second field should be CID\")\n\t\tassert.Regexp(t, `^\\d+$`, dataFields[2], \"third field should be size\")\n\t})\n\n\tt.Run(\"long format with headers and size=false\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\ttestDir := filepath.Join(node.Dir, \"headertest2\")\n\t\trequire.NoError(t, os.MkdirAll(testDir, 0755))\n\t\ttestFile := filepath.Join(testDir, \"file.txt\")\n\t\trequire.NoError(t, os.WriteFile(testFile, []byte(\"hello\"), 0644))\n\n\t\toldTime := time.Date(2020, time.January, 1, 0, 0, 0, 0, time.UTC)\n\t\trequire.NoError(t, os.Chtimes(testFile, oldTime, oldTime))\n\n\t\taddRes := node.IPFS(\"add\", \"-r\", \"--preserve-mode\", \"--preserve-mtime\", \"-Q\", testDir)\n\t\tdirCid := addRes.Stdout.Trimmed()\n\n\t\t// Run ls with --long --headers --size=false\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", \"--headers\", \"--size=false\", dirCid)\n\t\toutput := lsRes.Stdout.String()\n\t\tlines := strings.Split(strings.TrimSpace(output), \"\\n\")\n\n\t\t// Header should be: Mode Hash ModTime Name (no Size)\n\t\trequire.GreaterOrEqual(t, len(lines), 2)\n\t\theaderFields := strings.Fields(lines[0])\n\t\trequire.Len(t, headerFields, 4, \"header should have 4 columns without size\")\n\t\tassert.Equal(t, \"Mode\", headerFields[0])\n\t\tassert.Equal(t, \"Hash\", headerFields[1])\n\t\tassert.Equal(t, \"ModTime\", headerFields[2])\n\t\tassert.Equal(t, \"Name\", headerFields[3])\n\t})\n\n\tt.Run(\"long format for directories shows trailing slash\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Create nested directory structure\n\t\ttestDir := filepath.Join(node.Dir, \"dirtest\")\n\t\tsubDir := filepath.Join(testDir, \"subdir\")\n\t\trequire.NoError(t, os.MkdirAll(subDir, 0755))\n\t\trequire.NoError(t, os.WriteFile(filepath.Join(subDir, \"file.txt\"), []byte(\"hi\"), 0644))\n\n\t\taddRes := node.IPFS(\"add\", \"-r\", \"--preserve-mode\", \"-Q\", testDir)\n\t\tdirCid := addRes.Stdout.Trimmed()\n\n\t\t// Run ls with --long flag\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", dirCid)\n\t\toutput := lsRes.Stdout.String()\n\n\t\t// Directory should end with /\n\t\tassert.Contains(t, output, \"subdir/\", \"directory should have trailing slash\")\n\t\t// Directory should show 'd' in mode\n\t\tassert.Contains(t, output, \"drwxr-xr-x\", \"directory should show directory mode\")\n\t})\n\n\tt.Run(\"long format without size flag\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\ttestDir := filepath.Join(node.Dir, \"nosizetest\")\n\t\trequire.NoError(t, os.MkdirAll(testDir, 0755))\n\t\ttestFile := filepath.Join(testDir, \"file.txt\")\n\t\trequire.NoError(t, os.WriteFile(testFile, []byte(\"hello world\"), 0644))\n\n\t\toldTime := time.Date(2020, time.January, 1, 0, 0, 0, 0, time.UTC)\n\t\trequire.NoError(t, os.Chtimes(testFile, oldTime, oldTime))\n\n\t\taddRes := node.IPFS(\"add\", \"-r\", \"--preserve-mode\", \"--preserve-mtime\", \"-Q\", testDir)\n\t\tdirCid := addRes.Stdout.Trimmed()\n\n\t\t// Run ls with --long but --size=false\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", \"--size=false\", dirCid)\n\t\toutput := lsRes.Stdout.String()\n\n\t\t// Should still have mode and mtime, but format differs (no size column)\n\t\tassert.Contains(t, output, \"-rw-r--r--\")\n\t\tassert.Contains(t, output, \"Jan 01 2020\")\n\t\tassert.Contains(t, output, \"file.txt\")\n\t})\n\n\tt.Run(\"long format output is stable\", func(t *testing.T) {\n\t\t// This test ensures the output format doesn't change due to refactors\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\ttestDir := filepath.Join(node.Dir, \"stabletest\")\n\t\trequire.NoError(t, os.MkdirAll(testDir, 0755))\n\t\ttestFile := filepath.Join(testDir, \"test.txt\")\n\t\trequire.NoError(t, os.WriteFile(testFile, []byte(\"stable\"), 0644))\n\n\t\t// Use a fixed time for reproducibility\n\t\tfixedTime := time.Date(2020, time.December, 25, 12, 0, 0, 0, time.UTC)\n\t\trequire.NoError(t, os.Chtimes(testFile, fixedTime, fixedTime))\n\n\t\taddRes := node.IPFS(\"add\", \"-r\", \"--preserve-mode\", \"--preserve-mtime\", \"-Q\", testDir)\n\t\tdirCid := addRes.Stdout.Trimmed()\n\n\t\t// The CID should be deterministic given same content, mode, and mtime\n\t\t// This is the expected CID for this specific test data\n\t\tlsRes := node.IPFS(\"ls\", \"--long\", dirCid)\n\t\toutput := strings.TrimSpace(lsRes.Stdout.String())\n\n\t\t// Verify the format: ModeHashSizeModTimeName\n\t\tfields := strings.Fields(output)\n\t\trequire.GreaterOrEqual(t, len(fields), 5, \"output should have at least 5 fields\")\n\n\t\t// Field 0: mode (10 chars, starts with - for regular file)\n\t\tassert.Regexp(t, `^-[rwx-]{9}$`, fields[0], \"mode should be Unix permission format\")\n\n\t\t// Field 1: CID (starts with Qm or bafy)\n\t\tassert.Regexp(t, `^(Qm|bafy)`, fields[1], \"second field should be CID\")\n\n\t\t// Field 2: size (numeric)\n\t\tassert.Regexp(t, `^\\d+$`, fields[2], \"third field should be numeric size\")\n\n\t\t// Fields 3-4: date (e.g., \"Dec 25 2020\" or \"Dec 25 12:00\")\n\t\t// The date format is \"Mon DD YYYY\" for old files\n\t\tassert.Equal(t, \"Dec\", fields[3])\n\t\tassert.Equal(t, \"25\", fields[4])\n\n\t\t// Last field: filename\n\t\tassert.Equal(t, \"test.txt\", fields[len(fields)-1])\n\t})\n}\n" }, { "path": "test/cli/migrations/migration_16_to_latest_test.go", "content": "package migrations\n\n// NOTE: These migration tests require the local Kubo binary (built with 'make build') to be in PATH.\n//\n// To run these tests successfully:\n// export PATH=\"$(pwd)/cmd/ipfs:$PATH\"\n// go test ./test/cli/migrations/\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tipfs \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestMigration16ToLatest tests migration from repo version 16 to the latest version.\n//\n// This test uses a real IPFS repository snapshot from Kubo v0.36.0 (the last version that used repo v16).\n// The intention is to confirm that users can upgrade from Kubo v0.36.0 to the latest version by applying\n// all intermediate migrations successfully.\n//\n// NOTE: This test comprehensively tests all migration methods (daemon --migrate, repo migrate,\n// and reverse migration) because 16-to-17 was the first embedded migration that did not fetch\n// external files. It serves as a reference implementation for migration testing.\n//\n// Future migrations can have simplified tests (like 17-to-18 in migration_17_to_latest_test.go)\n// that focus on specific migration logic rather than testing all migration methods.\n//\n// If you need to test migration of configuration keys that appeared in later repo versions,\n// create a new test file migration_N_to_latest_test.go with a separate IPFS repository test vector\n// from the appropriate Kubo version.\nfunc TestMigration16ToLatest(t *testing.T) {\n\tt.Parallel()\n\n\t// Primary tests using 'ipfs daemon --migrate' command (default in Docker)\n\tt.Run(\"daemon migrate: forward migration with auto values\", testDaemonMigrationWithAuto)\n\tt.Run(\"daemon migrate: forward migration without auto values\", testDaemonMigrationWithoutAuto)\n\tt.Run(\"daemon migrate: corrupted config handling\", testDaemonCorruptedConfigHandling)\n\tt.Run(\"daemon migrate: missing fields handling\", testDaemonMissingFieldsHandling)\n\n\t// Comparison tests using 'ipfs repo migrate' command\n\tt.Run(\"repo migrate: forward migration with auto values\", testRepoMigrationWithAuto)\n\tt.Run(\"repo migrate: backward migration\", testRepoBackwardMigration)\n\n\t// Temp file and backup cleanup tests\n\tt.Run(\"daemon migrate: no temp files after successful migration\", testNoTempFilesAfterSuccessfulMigration)\n\tt.Run(\"daemon migrate: no temp files after failed migration\", testNoTempFilesAfterFailedMigration)\n\tt.Run(\"daemon migrate: backup files persist after successful migration\", testBackupFilesPersistAfterSuccessfulMigration)\n\tt.Run(\"repo migrate: backup files can revert migration\", testBackupFilesCanRevertMigration)\n\tt.Run(\"repo migrate: conversion failure cleans up temp files\", testConversionFailureCleanup)\n}\n\n// =============================================================================\n// PRIMARY TESTS: 'ipfs daemon --migrate' command (default in Docker)\n//\n// These tests exercise the primary migration path used in production Docker\n// containers where --migrate is enabled by default. This covers:\n// - Normal forward migration scenarios\n// - Error handling with corrupted configs\n// - Migration with minimal/missing config fields\n// =============================================================================\n\nfunc testDaemonMigrationWithAuto(t *testing.T) {\n\t// TEST: Forward migration using 'ipfs daemon --migrate' command (PRIMARY)\n\t// Use static v16 repo fixture from real Kubo 0.36 `ipfs init`\n\t// NOTE: This test may need to be revised/updated once repo version 18 is released,\n\t// at that point only keep tests that use 'ipfs repo migrate'\n\tnode := setupStaticV16Repo(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Static fixture already uses port 0 for random port assignment - no config update needed\n\n\t// Run migration using daemon --migrate (automatic during daemon startup)\n\t// This is the primary method used in Docker containers\n\t// Monitor output until daemon is ready, then shut it down gracefully\n\tstdoutOutput, migrationSuccess := runDaemonMigrationWithMonitoring(t, node)\n\n\t// Debug: Print the actual output\n\tt.Logf(\"Daemon output:\\n%s\", stdoutOutput)\n\n\t// Verify migration was successful based on monitoring\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"applying 16-to-17 repo migration\", \"Migration should have been triggered\")\n\trequire.Contains(t, stdoutOutput, \"Migration 16-to-17 succeeded\", \"Migration should have completed successfully\")\n\n\t// Verify version was updated to latest\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\texpectedVersion := fmt.Sprint(ipfs.RepoVersion)\n\trequire.Equal(t, expectedVersion, strings.TrimSpace(string(versionData)), \"Version should be updated to %s (latest)\", expectedVersion)\n\n\t// Verify migration results using DRY helper\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireAutoConfDefaults().\n\t\tRequireArrayContains(\"Bootstrap\", \"auto\").\n\t\tRequireArrayLength(\"Bootstrap\", 1). // Should only contain \"auto\" when all peers were defaults\n\t\tRequireArrayContains(\"Routing.DelegatedRouters\", \"auto\").\n\t\tRequireArrayContains(\"Ipns.DelegatedPublishers\", \"auto\")\n\n\t// DNS resolver in static fixture should be empty, so \".\" should be set to \"auto\"\n\thelper.RequireFieldEquals(\"DNS.Resolvers[.]\", \"auto\")\n}\n\nfunc testDaemonMigrationWithoutAuto(t *testing.T) {\n\t// TEST: Forward migration using 'ipfs daemon --migrate' command (PRIMARY)\n\t// Test migration of a config that already has some custom values\n\t// NOTE: This test may need to be revised/updated once repo version 18 is released,\n\t// at that point only keep tests that use 'ipfs repo migrate'\n\t// Should preserve existing settings and only add missing ones\n\tnode := setupStaticV16Repo(t)\n\n\t// Modify the static fixture to add some custom values for testing mixed scenarios\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Read existing config from static fixture\n\tvar v16Config map[string]any\n\tconfigData, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &v16Config))\n\n\t// Add custom DNS resolver that should be preserved\n\tif v16Config[\"DNS\"] == nil {\n\t\tv16Config[\"DNS\"] = map[string]any{}\n\t}\n\tdnsSection := v16Config[\"DNS\"].(map[string]any)\n\tdnsSection[\"Resolvers\"] = map[string]string{\n\t\t\".\": \"https://custom-dns.example.com/dns-query\",\n\t\t\"eth.\": \"https://dns.eth.limo/dns-query\", // This is a default that will be replaced with \"auto\"\n\t}\n\n\t// Write modified config back\n\tmodifiedConfigData, err := json.MarshalIndent(v16Config, \"\", \" \")\n\trequire.NoError(t, err)\n\trequire.NoError(t, os.WriteFile(configPath, modifiedConfigData, 0644))\n\n\t// Static fixture already uses port 0 for random port assignment - no config update needed\n\n\t// Run migration using daemon --migrate command (this is a daemon test)\n\t// Monitor output until daemon is ready, then shut it down gracefully\n\tstdoutOutput, migrationSuccess := runDaemonMigrationWithMonitoring(t, node)\n\n\t// Verify migration was successful based on monitoring\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"applying 16-to-17 repo migration\", \"Migration should have been triggered\")\n\trequire.Contains(t, stdoutOutput, \"Migration 16-to-17 succeeded\", \"Migration should have completed successfully\")\n\n\t// Verify migration results: custom values preserved alongside \"auto\"\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireAutoConfDefaults().\n\t\tRequireArrayContains(\"Bootstrap\", \"auto\").\n\t\tRequireFieldEquals(\"DNS.Resolvers[.]\", \"https://custom-dns.example.com/dns-query\")\n\n\t// Check that eth. resolver was replaced with \"auto\" since it uses a default URL\n\thelper.RequireFieldEquals(\"DNS.Resolvers[eth.]\", \"auto\").\n\t\tRequireFieldEquals(\"DNS.Resolvers[.]\", \"https://custom-dns.example.com/dns-query\")\n}\n\n// =============================================================================\n// Tests using 'ipfs daemon --migrate' command\n// =============================================================================\n\n// Test helper structs and functions for cleaner, more DRY tests\n\ntype ConfigField struct {\n\tPath string\n\tExpected any\n\tMessage string\n}\n\ntype MigrationTestHelper struct {\n\tt *testing.T\n\tconfig map[string]any\n}\n\nfunc NewMigrationTestHelper(t *testing.T, configPath string) *MigrationTestHelper {\n\tvar config map[string]any\n\tconfigData, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &config))\n\n\treturn &MigrationTestHelper{t: t, config: config}\n}\n\nfunc (h *MigrationTestHelper) RequireFieldExists(path string) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.NotNil(h.t, value, \"Field %s should exist\", path)\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireFieldEquals(path string, expected any) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.Equal(h.t, expected, value, \"Field %s should equal %v\", path, expected)\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireArrayContains(path string, expected any) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.IsType(h.t, []any{}, value, \"Field %s should be an array\", path)\n\tarray := value.([]any)\n\trequire.Contains(h.t, array, expected, \"Array %s should contain %v\", path, expected)\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireArrayLength(path string, expectedLen int) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.IsType(h.t, []any{}, value, \"Field %s should be an array\", path)\n\tarray := value.([]any)\n\trequire.Len(h.t, array, expectedLen, \"Array %s should have length %d\", path, expectedLen)\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireArrayDoesNotContain(path string, notExpected any) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.IsType(h.t, []any{}, value, \"Field %s should be an array\", path)\n\tarray := value.([]any)\n\trequire.NotContains(h.t, array, notExpected, \"Array %s should not contain %v\", path, notExpected)\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireFieldAbsent(path string) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.Nil(h.t, value, \"Field %s should not exist\", path)\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireAutoConfDefaults() *MigrationTestHelper {\n\t// AutoConf section should exist but be empty (using implicit defaults)\n\treturn h.RequireFieldExists(\"AutoConf\").\n\t\tRequireFieldAbsent(\"AutoConf.Enabled\"). // Should use implicit default (true)\n\t\tRequireFieldAbsent(\"AutoConf.URL\"). // Should use implicit default (mainnet URL)\n\t\tRequireFieldAbsent(\"AutoConf.RefreshInterval\"). // Should use implicit default (24h)\n\t\tRequireFieldAbsent(\"AutoConf.TLSInsecureSkipVerify\") // Should use implicit default (false)\n}\n\nfunc (h *MigrationTestHelper) RequireAutoFieldsSetToAuto() *MigrationTestHelper {\n\treturn h.RequireArrayContains(\"Bootstrap\", \"auto\").\n\t\tRequireFieldEquals(\"DNS.Resolvers[.]\", \"auto\").\n\t\tRequireArrayContains(\"Routing.DelegatedRouters\", \"auto\").\n\t\tRequireArrayContains(\"Ipns.DelegatedPublishers\", \"auto\")\n}\n\nfunc (h *MigrationTestHelper) RequireNoAutoValues() *MigrationTestHelper {\n\t// Check Bootstrap if it exists\n\tif h.getNestedValue(\"Bootstrap\") != nil {\n\t\th.RequireArrayDoesNotContain(\"Bootstrap\", \"auto\")\n\t}\n\n\t// Check DNS.Resolvers if it exists\n\tif h.getNestedValue(\"DNS.Resolvers\") != nil {\n\t\th.RequireMapDoesNotContainValue(\"DNS.Resolvers\", \"auto\")\n\t}\n\n\t// Check Routing.DelegatedRouters if it exists\n\tif h.getNestedValue(\"Routing.DelegatedRouters\") != nil {\n\t\th.RequireArrayDoesNotContain(\"Routing.DelegatedRouters\", \"auto\")\n\t}\n\n\t// Check Ipns.DelegatedPublishers if it exists\n\tif h.getNestedValue(\"Ipns.DelegatedPublishers\") != nil {\n\t\th.RequireArrayDoesNotContain(\"Ipns.DelegatedPublishers\", \"auto\")\n\t}\n\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) RequireMapDoesNotContainValue(path string, notExpected any) *MigrationTestHelper {\n\tvalue := h.getNestedValue(path)\n\trequire.IsType(h.t, map[string]any{}, value, \"Field %s should be a map\", path)\n\tmapValue := value.(map[string]any)\n\tfor k, v := range mapValue {\n\t\trequire.NotEqual(h.t, notExpected, v, \"Map %s[%s] should not equal %v\", path, k, notExpected)\n\t}\n\treturn h\n}\n\nfunc (h *MigrationTestHelper) getNestedValue(path string) any {\n\tsegments := h.parseKuboConfigPath(path)\n\tcurrent := any(h.config)\n\n\tfor _, segment := range segments {\n\t\tswitch segment.Type {\n\t\tcase \"field\":\n\t\t\tswitch v := current.(type) {\n\t\t\tcase map[string]any:\n\t\t\t\tcurrent = v[segment.Key]\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\tcase \"mapKey\":\n\t\t\tswitch v := current.(type) {\n\t\t\tcase map[string]any:\n\t\t\t\tcurrent = v[segment.Key]\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\tdefault:\n\t\t\treturn nil\n\t\t}\n\n\t\tif current == nil {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn current\n}\n\ntype PathSegment struct {\n\tType string // \"field\" or \"mapKey\"\n\tKey string\n}\n\nfunc (h *MigrationTestHelper) parseKuboConfigPath(path string) []PathSegment {\n\tvar segments []PathSegment\n\n\t// Split path into parts, respecting bracket boundaries\n\tparts := h.splitKuboConfigPath(path)\n\n\tfor _, part := range parts {\n\t\tif strings.Contains(part, \"[\") && strings.HasSuffix(part, \"]\") {\n\t\t\t// Handle field[key] notation\n\t\t\tbracketStart := strings.Index(part, \"[\")\n\t\t\tfieldName := part[:bracketStart]\n\t\t\tmapKey := part[bracketStart+1 : len(part)-1] // Remove [ and ]\n\n\t\t\t// Add field segment if present\n\t\t\tif fieldName != \"\" {\n\t\t\t\tsegments = append(segments, PathSegment{Type: \"field\", Key: fieldName})\n\t\t\t}\n\t\t\t// Add map key segment\n\t\t\tsegments = append(segments, PathSegment{Type: \"mapKey\", Key: mapKey})\n\t\t} else {\n\t\t\t// Regular field access\n\t\t\tif part != \"\" {\n\t\t\t\tsegments = append(segments, PathSegment{Type: \"field\", Key: part})\n\t\t\t}\n\t\t}\n\t}\n\n\treturn segments\n}\n\n// splitKuboConfigPath splits a path on dots, but preserves bracket sections intact\nfunc (h *MigrationTestHelper) splitKuboConfigPath(path string) []string {\n\tvar parts []string\n\tvar current strings.Builder\n\tinBrackets := false\n\n\tfor _, r := range path {\n\t\tswitch r {\n\t\tcase '[':\n\t\t\tinBrackets = true\n\t\t\tcurrent.WriteRune(r)\n\t\tcase ']':\n\t\t\tinBrackets = false\n\t\t\tcurrent.WriteRune(r)\n\t\tcase '.':\n\t\t\tif inBrackets {\n\t\t\t\t// Inside brackets, preserve the dot\n\t\t\t\tcurrent.WriteRune(r)\n\t\t\t} else {\n\t\t\t\t// Outside brackets, split here\n\t\t\t\tif current.Len() > 0 {\n\t\t\t\t\tparts = append(parts, current.String())\n\t\t\t\t\tcurrent.Reset()\n\t\t\t\t}\n\t\t\t}\n\t\tdefault:\n\t\t\tcurrent.WriteRune(r)\n\t\t}\n\t}\n\n\t// Add final part if any\n\tif current.Len() > 0 {\n\t\tparts = append(parts, current.String())\n\t}\n\n\treturn parts\n}\n\n// setupStaticV16Repo creates a test node using static v16 repo fixture from real Kubo 0.36 `ipfs init`\n// This ensures tests remain stable regardless of future changes to the IPFS binary\n// Each test gets its own copy in a temporary directory to allow modifications\nfunc setupStaticV16Repo(t *testing.T) *harness.Node {\n\t// Get absolute path to static v16 repo fixture\n\tv16FixturePath := \"testdata/v16-repo\"\n\n\t// Create a temporary test directory - each test gets its own copy\n\t// Sanitize test name for Windows - replace invalid characters\n\tsanitizedName := strings.Map(func(r rune) rune {\n\t\tif strings.ContainsRune(`<>:\"/\\|?*`, r) {\n\t\t\treturn '_'\n\t\t}\n\t\treturn r\n\t}, t.Name())\n\ttmpDir := filepath.Join(t.TempDir(), \"migration-test-\"+sanitizedName)\n\trequire.NoError(t, os.MkdirAll(tmpDir, 0755))\n\n\t// Convert to absolute path for harness\n\tabsTmpDir, err := filepath.Abs(tmpDir)\n\trequire.NoError(t, err)\n\n\t// Use the built binary (should be in PATH)\n\tnode := harness.BuildNode(\"ipfs\", absTmpDir, 0)\n\n\t// Replace IPFS_PATH with static fixture files to test directory (creates independent copy per test)\n\tcloneStaticRepoFixture(t, v16FixturePath, node.Dir)\n\n\treturn node\n}\n\n// cloneStaticRepoFixture recursively copies the v16 repo fixture to the target directory\n// It completely removes the target directory contents before copying to ensure no extra files remain\nfunc cloneStaticRepoFixture(t *testing.T, srcPath, dstPath string) {\n\tsrcInfo, err := os.Stat(srcPath)\n\trequire.NoError(t, err)\n\n\tif srcInfo.IsDir() {\n\t\t// Completely remove destination directory and all contents\n\t\trequire.NoError(t, os.RemoveAll(dstPath))\n\t\t// Create fresh destination directory\n\t\trequire.NoError(t, os.MkdirAll(dstPath, srcInfo.Mode()))\n\n\t\t// Read source directory\n\t\tentries, err := os.ReadDir(srcPath)\n\t\trequire.NoError(t, err)\n\n\t\t// Copy each entry recursively\n\t\tfor _, entry := range entries {\n\t\t\tsrcEntryPath := filepath.Join(srcPath, entry.Name())\n\t\t\tdstEntryPath := filepath.Join(dstPath, entry.Name())\n\t\t\tcloneStaticRepoFixture(t, srcEntryPath, dstEntryPath)\n\t\t}\n\t} else {\n\t\t// Copy file (destination directory should already be clean from parent call)\n\t\tsrcFile, err := os.Open(srcPath)\n\t\trequire.NoError(t, err)\n\t\tdefer srcFile.Close()\n\n\t\tdstFile, err := os.Create(dstPath)\n\t\trequire.NoError(t, err)\n\t\tdefer dstFile.Close()\n\n\t\t_, err = io.Copy(dstFile, srcFile)\n\t\trequire.NoError(t, err)\n\n\t\t// Copy file permissions\n\t\trequire.NoError(t, dstFile.Chmod(srcInfo.Mode()))\n\t}\n}\n\n// Placeholder stubs for new test functions - to be implemented\nfunc testDaemonCorruptedConfigHandling(t *testing.T) {\n\t// TEST: Error handling using 'ipfs daemon --migrate' command with corrupted config (PRIMARY)\n\t// Test what happens when config file is corrupted during migration\n\t// NOTE: This test may need to be revised/updated once repo version 18 is released,\n\t// at that point only keep tests that use 'ipfs repo migrate'\n\tnode := setupStaticV16Repo(t)\n\n\t// Create corrupted config\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tcorruptedJson := `{\"Bootstrap\": [invalid json}`\n\trequire.NoError(t, os.WriteFile(configPath, []byte(corruptedJson), 0644))\n\n\t// Write version file indicating v16\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\trequire.NoError(t, os.WriteFile(versionPath, []byte(\"16\"), 0644))\n\n\t// Run daemon with --migrate flag - this should fail gracefully\n\tresult := node.RunIPFS(\"daemon\", \"--migrate\")\n\n\t// Verify graceful failure handling\n\t// The daemon should fail but migration error should be clear\n\terrorOutput := result.Stderr.String() + result.Stdout.String()\n\trequire.True(t, strings.Contains(errorOutput, \"json\") || strings.Contains(errorOutput, \"invalid character\"), \"Error should mention JSON parsing issue\")\n\n\t// Verify atomic failure: version and config should remain unchanged\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"16\", strings.TrimSpace(string(versionData)), \"Version should remain unchanged after failed migration\")\n\n\toriginalContent, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.Equal(t, corruptedJson, string(originalContent), \"Original config should be unchanged after failed migration\")\n}\n\nfunc testDaemonMissingFieldsHandling(t *testing.T) {\n\t// TEST: Migration using 'ipfs daemon --migrate' command with minimal config (PRIMARY)\n\t// Test migration when config is missing expected fields\n\t// NOTE: This test may need to be revised/updated once repo version 18 is released,\n\t// at that point only keep tests that use 'ipfs repo migrate'\n\tnode := setupStaticV16Repo(t)\n\n\t// The static fixture already has all required fields, use it as-is\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Static fixture already uses port 0 for random port assignment - no config update needed\n\n\t// Run daemon migration\n\tstdoutOutput, migrationSuccess := runDaemonMigrationWithMonitoring(t, node)\n\n\t// Verify migration was successful\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"applying 16-to-17 repo migration\", \"Migration should have been triggered\")\n\trequire.Contains(t, stdoutOutput, \"Migration 16-to-17 succeeded\", \"Migration should have completed successfully\")\n\n\t// Verify version was updated to latest\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\texpectedVersion := fmt.Sprint(ipfs.RepoVersion)\n\trequire.Equal(t, expectedVersion, strings.TrimSpace(string(versionData)), \"Version should be updated to %s (latest)\", expectedVersion)\n\n\t// Verify migration adds all required fields to minimal config\n\tNewMigrationTestHelper(t, configPath).\n\t\tRequireAutoConfDefaults().\n\t\tRequireAutoFieldsSetToAuto().\n\t\tRequireFieldExists(\"Identity.PeerID\") // Original identity preserved from static fixture\n}\n\n// =============================================================================\n// COMPARISON TESTS: 'ipfs repo migrate' command\n//\n// These tests verify that repo migrate produces equivalent results to\n// daemon migrate, and test scenarios specific to repo migrate like\n// backward migration (which daemon doesn't support).\n// =============================================================================\n\nfunc testRepoMigrationWithAuto(t *testing.T) {\n\t// TEST: Forward migration using 'ipfs repo migrate' command (COMPARISON)\n\t// Simple comparison test to verify repo migrate produces same results as daemon migrate\n\tnode := setupStaticV16Repo(t)\n\n\t// Use static fixture as-is\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run migration using 'ipfs repo migrate' command\n\tresult := node.RunIPFS(\"repo\", \"migrate\")\n\trequire.Empty(t, result.Stderr.String(), \"Migration should succeed without errors\")\n\n\t// Verify same results as daemon migrate\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireAutoConfDefaults().\n\t\tRequireArrayContains(\"Bootstrap\", \"auto\").\n\t\tRequireArrayContains(\"Routing.DelegatedRouters\", \"auto\").\n\t\tRequireArrayContains(\"Ipns.DelegatedPublishers\", \"auto\").\n\t\tRequireFieldEquals(\"DNS.Resolvers[.]\", \"auto\")\n}\n\nfunc testRepoBackwardMigration(t *testing.T) {\n\t// TEST: Backward migration using 'ipfs repo migrate --to=16 --allow-downgrade' command\n\t// This is kept as repo migrate since daemon doesn't support backward migration\n\tnode := setupStaticV16Repo(t)\n\n\t// Use static fixture as-is\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// First run forward migration to get to v17\n\tresult := node.RunIPFS(\"repo\", \"migrate\")\n\tt.Logf(\"Forward migration stdout:\\n%s\", result.Stdout.String())\n\tt.Logf(\"Forward migration stderr:\\n%s\", result.Stderr.String())\n\trequire.Empty(t, result.Stderr.String(), \"Forward migration should succeed\")\n\n\t// Verify we're at the latest version\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\texpectedVersion := fmt.Sprint(ipfs.RepoVersion)\n\trequire.Equal(t, expectedVersion, strings.TrimSpace(string(versionData)), \"Should be at version %s (latest) after forward migration\", expectedVersion)\n\n\t// Now run reverse migration back to v16\n\tresult = node.RunIPFS(\"repo\", \"migrate\", \"--to=16\", \"--allow-downgrade\")\n\tt.Logf(\"Backward migration stdout:\\n%s\", result.Stdout.String())\n\tt.Logf(\"Backward migration stderr:\\n%s\", result.Stderr.String())\n\trequire.Empty(t, result.Stderr.String(), \"Reverse migration should succeed\")\n\n\t// Verify version was downgraded to 16\n\tversionData, err = os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"16\", strings.TrimSpace(string(versionData)), \"Version should be downgraded to 16\")\n\n\t// Verify backward migration results: AutoConf removed and no \"auto\" values remain\n\tNewMigrationTestHelper(t, configPath).\n\t\tRequireFieldAbsent(\"AutoConf\").\n\t\tRequireNoAutoValues()\n}\n\n// runDaemonMigrationWithMonitoring starts daemon --migrate, monitors output until \"Daemon is ready\",\n// then gracefully shuts down the daemon and returns the captured output and success status.\n// This monitors for all expected migrations from version 16 to latest.\nfunc runDaemonMigrationWithMonitoring(t *testing.T, node *harness.Node) (string, bool) {\n\t// Monitor migrations from repo v16 to latest\n\treturn runDaemonWithExpectedMigrations(t, node, 16, ipfs.RepoVersion)\n}\n\n// runDaemonWithExpectedMigrations monitors daemon startup for a sequence of migrations from startVersion to endVersion\nfunc runDaemonWithExpectedMigrations(t *testing.T, node *harness.Node, startVersion, endVersion int) (string, bool) {\n\t// Build list of expected migrations\n\tvar expectedMigrations []struct {\n\t\tpattern string\n\t\tsuccess string\n\t}\n\n\tfor v := startVersion; v < endVersion; v++ {\n\t\tfrom := v\n\t\tto := v + 1\n\t\texpectedMigrations = append(expectedMigrations, struct {\n\t\t\tpattern string\n\t\t\tsuccess string\n\t\t}{\n\t\t\tpattern: fmt.Sprintf(\"applying %d-to-%d repo migration\", from, to),\n\t\t\tsuccess: fmt.Sprintf(\"Migration %d-to-%d succeeded\", from, to),\n\t\t})\n\t}\n\n\treturn runDaemonWithMultipleMigrationMonitoring(t, node, expectedMigrations)\n}\n\n// runDaemonWithMultipleMigrationMonitoring monitors daemon startup for multiple sequential migrations\nfunc runDaemonWithMultipleMigrationMonitoring(t *testing.T, node *harness.Node, expectedMigrations []struct {\n\tpattern string\n\tsuccess string\n}) (string, bool) {\n\t// Create context with timeout as safety net\n\tctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)\n\tdefer cancel()\n\n\t// Set up daemon command with output monitoring\n\tcmd := exec.CommandContext(ctx, node.IPFSBin, \"daemon\", \"--migrate\")\n\tcmd.Dir = node.Dir\n\n\t// Set environment (especially IPFS_PATH)\n\tfor k, v := range node.Runner.Env {\n\t\tcmd.Env = append(cmd.Env, k+\"=\"+v)\n\t}\n\n\t// Set up pipes for output monitoring\n\tstdout, err := cmd.StdoutPipe()\n\trequire.NoError(t, err)\n\tstderr, err := cmd.StderrPipe()\n\trequire.NoError(t, err)\n\n\t// Start the daemon\n\terr = cmd.Start()\n\trequire.NoError(t, err)\n\n\tvar allOutput strings.Builder\n\tvar daemonReady bool\n\n\t// Track which migrations have been detected\n\tmigrationsDetected := make([]bool, len(expectedMigrations))\n\tmigrationsSucceeded := make([]bool, len(expectedMigrations))\n\n\t// Monitor stdout for completion signals\n\tscanner := bufio.NewScanner(stdout)\n\tgo func() {\n\t\tfor scanner.Scan() {\n\t\t\tline := scanner.Text()\n\t\t\tallOutput.WriteString(line + \"\\n\")\n\n\t\t\t// Check for migration messages\n\t\t\tfor i, migration := range expectedMigrations {\n\t\t\t\tif strings.Contains(line, migration.pattern) {\n\t\t\t\t\tmigrationsDetected[i] = true\n\t\t\t\t}\n\t\t\t\tif strings.Contains(line, migration.success) {\n\t\t\t\t\tmigrationsSucceeded[i] = true\n\t\t\t\t}\n\t\t\t}\n\t\t\tif strings.Contains(line, \"Daemon is ready\") {\n\t\t\t\tdaemonReady = true\n\t\t\t\tbreak // Exit monitoring loop\n\t\t\t}\n\t\t}\n\t}()\n\n\t// Also monitor stderr (but don't use it for completion detection)\n\tgo func() {\n\t\tstderrScanner := bufio.NewScanner(stderr)\n\t\tfor stderrScanner.Scan() {\n\t\t\tline := stderrScanner.Text()\n\t\t\tallOutput.WriteString(\"STDERR: \" + line + \"\\n\")\n\t\t}\n\t}()\n\n\t// Wait for daemon ready signal or timeout\n\tticker := time.NewTicker(100 * time.Millisecond)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\t// Timeout - kill the process\n\t\t\tif cmd.Process != nil {\n\t\t\t\t_ = cmd.Process.Kill()\n\t\t\t}\n\t\t\tt.Logf(\"Daemon migration timed out after 60 seconds\")\n\t\t\treturn allOutput.String(), false\n\n\t\tcase <-ticker.C:\n\t\t\tif daemonReady {\n\t\t\t\t// Daemon is ready - shut it down gracefully\n\t\t\t\tshutdownCmd := exec.Command(node.IPFSBin, \"shutdown\")\n\t\t\t\tshutdownCmd.Dir = node.Dir\n\t\t\t\tfor k, v := range node.Runner.Env {\n\t\t\t\t\tshutdownCmd.Env = append(shutdownCmd.Env, k+\"=\"+v)\n\t\t\t\t}\n\n\t\t\t\tif err := shutdownCmd.Run(); err != nil {\n\t\t\t\t\tt.Logf(\"Warning: ipfs shutdown failed: %v\", err)\n\t\t\t\t\t// Force kill if graceful shutdown fails\n\t\t\t\t\tif cmd.Process != nil {\n\t\t\t\t\t\t_ = cmd.Process.Kill()\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Wait for process to exit\n\t\t\t\t_ = cmd.Wait()\n\n\t\t\t\t// Check all migrations were detected and succeeded\n\t\t\t\tallDetected := true\n\t\t\t\tallSucceeded := true\n\t\t\t\tfor i := range expectedMigrations {\n\t\t\t\t\tif !migrationsDetected[i] {\n\t\t\t\t\t\tallDetected = false\n\t\t\t\t\t\tt.Logf(\"Migration %s was not detected\", expectedMigrations[i].pattern)\n\t\t\t\t\t}\n\t\t\t\t\tif !migrationsSucceeded[i] {\n\t\t\t\t\t\tallSucceeded = false\n\t\t\t\t\t\tt.Logf(\"Migration %s did not succeed\", expectedMigrations[i].success)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\treturn allOutput.String(), allDetected && allSucceeded\n\t\t\t}\n\n\t\t\t// Check if process has exited (e.g., due to startup failure after migration)\n\t\t\tif cmd.ProcessState != nil && cmd.ProcessState.Exited() {\n\t\t\t\t// Process exited - migration may have completed but daemon failed to start\n\t\t\t\t// This is expected for corrupted config tests\n\n\t\t\t\t// Check all migrations status\n\t\t\t\tallDetected := true\n\t\t\t\tallSucceeded := true\n\t\t\t\tfor i := range expectedMigrations {\n\t\t\t\t\tif !migrationsDetected[i] {\n\t\t\t\t\t\tallDetected = false\n\t\t\t\t\t}\n\t\t\t\t\tif !migrationsSucceeded[i] {\n\t\t\t\t\t\tallSucceeded = false\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\treturn allOutput.String(), allDetected && allSucceeded\n\t\t\t}\n\t\t}\n\t}\n}\n\n// =============================================================================\n// TEMP FILE AND BACKUP CLEANUP TESTS\n// =============================================================================\n\n// Helper functions for test cleanup assertions\nfunc assertNoTempFiles(t *testing.T, dir string, msgAndArgs ...any) {\n\tt.Helper()\n\ttmpFiles, err := filepath.Glob(filepath.Join(dir, \".tmp-*\"))\n\trequire.NoError(t, err)\n\tassert.Empty(t, tmpFiles, msgAndArgs...)\n}\n\nfunc backupPath(configPath string, fromVer, toVer int) string {\n\treturn fmt.Sprintf(\"%s.%d-to-%d.bak\", configPath, fromVer, toVer)\n}\n\nfunc setupDaemonCmd(ctx context.Context, node *harness.Node, args ...string) *exec.Cmd {\n\tcmd := exec.CommandContext(ctx, node.IPFSBin, args...)\n\tcmd.Dir = node.Dir\n\tfor k, v := range node.Runner.Env {\n\t\tcmd.Env = append(cmd.Env, k+\"=\"+v)\n\t}\n\treturn cmd\n}\n\nfunc testNoTempFilesAfterSuccessfulMigration(t *testing.T) {\n\tnode := setupStaticV16Repo(t)\n\n\t// Run successful migration\n\t_, migrationSuccess := runDaemonMigrationWithMonitoring(t, node)\n\trequire.True(t, migrationSuccess, \"migration should succeed\")\n\n\tassertNoTempFiles(t, node.Dir, \"no temp files should remain after successful migration\")\n}\n\nfunc testNoTempFilesAfterFailedMigration(t *testing.T) {\n\tnode := setupStaticV16Repo(t)\n\n\t// Corrupt config to force migration failure\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tcorruptedJson := `{\"Bootstrap\": [\"auto\",` // Invalid JSON\n\trequire.NoError(t, os.WriteFile(configPath, []byte(corruptedJson), 0644))\n\n\t// Attempt migration (should fail)\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tdefer cancel()\n\n\tcmd := setupDaemonCmd(ctx, node, \"daemon\", \"--migrate\")\n\toutput, _ := cmd.CombinedOutput()\n\tt.Logf(\"Failed migration output: %s\", output)\n\n\tassertNoTempFiles(t, node.Dir, \"no temp files should remain after failed migration\")\n}\n\nfunc testBackupFilesPersistAfterSuccessfulMigration(t *testing.T) {\n\tnode := setupStaticV16Repo(t)\n\n\t// Run migration from v16 to latest (v18)\n\t_, migrationSuccess := runDaemonMigrationWithMonitoring(t, node)\n\trequire.True(t, migrationSuccess, \"migration should succeed\")\n\n\t// Check for backup files from each migration step\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tbackup16to17 := backupPath(configPath, 16, 17)\n\tbackup17to18 := backupPath(configPath, 17, 18)\n\n\t// Both backup files should exist\n\tassert.FileExists(t, backup16to17, \"16-to-17 backup should exist\")\n\tassert.FileExists(t, backup17to18, \"17-to-18 backup should exist\")\n\n\t// Verify backup files contain valid JSON\n\tdata16to17, err := os.ReadFile(backup16to17)\n\trequire.NoError(t, err)\n\tvar config16to17 map[string]any\n\trequire.NoError(t, json.Unmarshal(data16to17, &config16to17), \"16-to-17 backup should be valid JSON\")\n\n\tdata17to18, err := os.ReadFile(backup17to18)\n\trequire.NoError(t, err)\n\tvar config17to18 map[string]any\n\trequire.NoError(t, json.Unmarshal(data17to18, &config17to18), \"17-to-18 backup should be valid JSON\")\n}\n\nfunc testBackupFilesCanRevertMigration(t *testing.T) {\n\tnode := setupStaticV16Repo(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Read original v16 config\n\toriginalConfig, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\n\t// Migrate to v17 only\n\tresult := node.RunIPFS(\"repo\", \"migrate\", \"--to=17\")\n\trequire.Empty(t, result.Stderr.String(), \"migration to v17 should succeed\")\n\n\t// Verify backup exists\n\tbackup16to17 := backupPath(configPath, 16, 17)\n\tassert.FileExists(t, backup16to17, \"16-to-17 backup should exist\")\n\n\t// Manually revert using backup\n\tbackupData, err := os.ReadFile(backup16to17)\n\trequire.NoError(t, err)\n\trequire.NoError(t, os.WriteFile(configPath, backupData, 0600))\n\trequire.NoError(t, os.WriteFile(versionPath, []byte(\"16\"), 0644))\n\n\t// Verify config matches original\n\trevertedConfig, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\tassert.JSONEq(t, string(originalConfig), string(revertedConfig), \"reverted config should match original\")\n\n\t// Verify version is back to 16\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"16\", strings.TrimSpace(string(versionData)), \"version should be reverted to 16\")\n}\n\nfunc testConversionFailureCleanup(t *testing.T) {\n\t// This test verifies that when a migration's conversion function fails,\n\t// all temporary files are cleaned up properly\n\tnode := setupStaticV16Repo(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Create a corrupted config that will cause conversion to fail during JSON parsing\n\t// The migration will read this, attempt to parse as JSON, and fail\n\tcorruptedJson := `{\"Bootstrap\": [\"auto\",` // Invalid JSON - missing closing bracket\n\trequire.NoError(t, os.WriteFile(configPath, []byte(corruptedJson), 0644))\n\n\t// Attempt migration (should fail during conversion)\n\tresult := node.RunIPFS(\"repo\", \"migrate\")\n\trequire.NotEmpty(t, result.Stderr.String(), \"migration should fail with error\")\n\n\tassertNoTempFiles(t, node.Dir, \"no temp files should remain after conversion failure\")\n\n\t// Verify no backup files were created (failure happened before backup)\n\tbackupFiles, err := filepath.Glob(filepath.Join(node.Dir, \"config.*.bak\"))\n\trequire.NoError(t, err)\n\tassert.Empty(t, backupFiles, \"no backup files should be created on conversion failure\")\n\n\t// Verify corrupted config is unchanged (atomic operations prevented overwrite)\n\tcurrentConfig, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\tassert.Equal(t, corruptedJson, string(currentConfig), \"corrupted config should remain unchanged\")\n}\n" }, { "path": "test/cli/migrations/migration_17_to_latest_test.go", "content": "package migrations\n\n// NOTE: These migration tests require the local Kubo binary (built with 'make build') to be in PATH.\n//\n// To run these tests successfully:\n// export PATH=\"$(pwd)/cmd/ipfs:$PATH\"\n// go test ./test/cli/migrations/\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tipfs \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestMigration17ToLatest tests migration from repo version 17 to the latest version.\n//\n// Since we don't have a v17 repo fixture, we start with v16 and migrate it to v17 first,\n// then test the 17-to-18 migration specifically.\n//\n// This test focuses on the Provider/Reprovider to Provide consolidation that happens in 17-to-18.\nfunc TestMigration17ToLatest(t *testing.T) {\n\tt.Parallel()\n\n\t// Tests for Provider/Reprovider to Provide migration (17-to-18)\n\tt.Run(\"daemon migrate: Provider/Reprovider to Provide consolidation\", testProviderReproviderMigration)\n\tt.Run(\"daemon migrate: flat strategy conversion\", testFlatStrategyConversion)\n\tt.Run(\"daemon migrate: empty Provider/Reprovider sections\", testEmptyProviderReproviderMigration)\n\tt.Run(\"daemon migrate: partial configuration (Provider only)\", testProviderOnlyMigration)\n\tt.Run(\"daemon migrate: partial configuration (Reprovider only)\", testReproviderOnlyMigration)\n\tt.Run(\"repo migrate: invalid strategy values preserved\", testInvalidStrategyMigration)\n\tt.Run(\"repo migrate: Provider/Reprovider to Provide consolidation\", testRepoProviderReproviderMigration)\n}\n\n// =============================================================================\n// MIGRATION 17-to-18 SPECIFIC TESTS: Provider/Reprovider to Provide consolidation\n// =============================================================================\n\nfunc testProviderReproviderMigration(t *testing.T) {\n\t// TEST: 17-to-18 migration with explicit Provider/Reprovider configuration\n\tnode := setupV17RepoWithProviderConfig(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Run migration using daemon --migrate command\n\tstdoutOutput, migrationSuccess := runDaemonMigrationFromV17(t, node)\n\n\t// Debug: Print the actual output\n\tt.Logf(\"Daemon output:\\n%s\", stdoutOutput)\n\n\t// Verify migration was successful\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"applying 17-to-18 repo migration\", \"Migration 17-to-18 should have been triggered\")\n\trequire.Contains(t, stdoutOutput, \"Migration 17-to-18 succeeded\", \"Migration 17-to-18 should have completed successfully\")\n\n\t// Verify version was updated to latest\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\texpectedVersion := fmt.Sprint(ipfs.RepoVersion)\n\trequire.Equal(t, expectedVersion, strings.TrimSpace(string(versionData)), \"Version should be updated to %s (latest)\", expectedVersion)\n\n\t// =============================================================================\n\t// MIGRATION 17-to-18 ASSERTIONS: Provider/Reprovider to Provide consolidation\n\t// =============================================================================\n\thelper := NewMigrationTestHelper(t, configPath)\n\n\t// Verify Provider/Reprovider migration to Provide\n\thelper.RequireProviderMigration().\n\t\tRequireFieldEquals(\"Provide.Enabled\", true). // Migrated from Provider.Enabled\n\t\tRequireFieldEquals(\"Provide.DHT.MaxWorkers\", float64(8)). // Migrated from Provider.WorkerCount\n\t\tRequireFieldEquals(\"Provide.Strategy\", \"roots\"). // Migrated from Reprovider.Strategy\n\t\tRequireFieldEquals(\"Provide.DHT.Interval\", \"24h\") // Migrated from Reprovider.Interval\n\n\t// Verify old sections are removed\n\thelper.RequireFieldAbsent(\"Provider\").\n\t\tRequireFieldAbsent(\"Reprovider\")\n}\n\nfunc testFlatStrategyConversion(t *testing.T) {\n\t// TEST: 17-to-18 migration with \"flat\" strategy that should convert to \"all\"\n\tnode := setupV17RepoWithFlatStrategy(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run migration using daemon --migrate command\n\tstdoutOutput, migrationSuccess := runDaemonMigrationFromV17(t, node)\n\n\t// Verify migration was successful\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"applying 17-to-18 repo migration\", \"Migration 17-to-18 should have been triggered\")\n\trequire.Contains(t, stdoutOutput, \"Migration 17-to-18 succeeded\", \"Migration 17-to-18 should have completed successfully\")\n\n\t// =============================================================================\n\t// MIGRATION 17-to-18 ASSERTIONS: \"flat\" to \"all\" strategy conversion\n\t// =============================================================================\n\thelper := NewMigrationTestHelper(t, configPath)\n\n\t// Verify \"flat\" was converted to \"all\"\n\thelper.RequireProviderMigration().\n\t\tRequireFieldEquals(\"Provide.Strategy\", \"all\"). // \"flat\" converted to \"all\"\n\t\tRequireFieldEquals(\"Provide.DHT.Interval\", \"12h\")\n}\n\nfunc testEmptyProviderReproviderMigration(t *testing.T) {\n\t// TEST: 17-to-18 migration with empty Provider and Reprovider sections\n\tnode := setupV17RepoWithEmptySections(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run migration\n\tstdoutOutput, migrationSuccess := runDaemonMigrationFromV17(t, node)\n\n\t// Verify migration was successful\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"Migration 17-to-18 succeeded\")\n\n\t// Verify empty sections are removed and no Provide section is created\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireFieldAbsent(\"Provider\").\n\t\tRequireFieldAbsent(\"Reprovider\").\n\t\tRequireFieldAbsent(\"Provide\") // No Provide section should be created for empty configs\n}\n\nfunc testProviderOnlyMigration(t *testing.T) {\n\t// TEST: 17-to-18 migration with only Provider configuration\n\tnode := setupV17RepoWithProviderOnly(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run migration\n\tstdoutOutput, migrationSuccess := runDaemonMigrationFromV17(t, node)\n\n\t// Verify migration was successful\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"Migration 17-to-18 succeeded\")\n\n\t// Verify only Provider fields are migrated\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireProviderMigration().\n\t\tRequireFieldEquals(\"Provide.Enabled\", false).\n\t\tRequireFieldEquals(\"Provide.DHT.MaxWorkers\", float64(32)).\n\t\tRequireFieldAbsent(\"Provide.Strategy\"). // No Reprovider.Strategy to migrate\n\t\tRequireFieldAbsent(\"Provide.DHT.Interval\") // No Reprovider.Interval to migrate\n}\n\nfunc testReproviderOnlyMigration(t *testing.T) {\n\t// TEST: 17-to-18 migration with only Reprovider configuration\n\tnode := setupV17RepoWithReproviderOnly(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run migration\n\tstdoutOutput, migrationSuccess := runDaemonMigrationFromV17(t, node)\n\n\t// Verify migration was successful\n\trequire.True(t, migrationSuccess, \"Migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"Migration 17-to-18 succeeded\")\n\n\t// Verify only Reprovider fields are migrated\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireProviderMigration().\n\t\tRequireFieldEquals(\"Provide.Strategy\", \"pinned\").\n\t\tRequireFieldEquals(\"Provide.DHT.Interval\", \"48h\").\n\t\tRequireFieldAbsent(\"Provide.Enabled\"). // No Provider.Enabled to migrate\n\t\tRequireFieldAbsent(\"Provide.DHT.MaxWorkers\") // No Provider.WorkerCount to migrate\n}\n\nfunc testInvalidStrategyMigration(t *testing.T) {\n\t// TEST: 17-to-18 migration with invalid strategy values (should be preserved as-is)\n\t// The migration itself should succeed, but daemon start will fail due to invalid strategy\n\tnode := setupV17RepoWithInvalidStrategy(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run the migration using 'ipfs repo migrate' (not daemon --migrate)\n\t// because daemon would fail to start with invalid strategy after migration\n\tresult := node.RunIPFS(\"repo\", \"migrate\")\n\trequire.Empty(t, result.Stderr.String(), \"Migration should succeed without errors\")\n\n\t// Verify invalid strategy is preserved as-is (not validated during migration)\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireProviderMigration().\n\t\tRequireFieldEquals(\"Provide.Strategy\", \"invalid-strategy\") // Should be preserved\n\n\t// Now verify that daemon fails to start with invalid strategy\n\t// Note: We cannot use --offline as it skips provider validation\n\t// Use a context with timeout to avoid hanging\n\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\tdefer cancel()\n\n\tcmd := exec.CommandContext(ctx, node.IPFSBin, \"daemon\")\n\tcmd.Dir = node.Dir\n\tfor k, v := range node.Runner.Env {\n\t\tcmd.Env = append(cmd.Env, k+\"=\"+v)\n\t}\n\n\toutput, err := cmd.CombinedOutput()\n\n\t// The daemon should fail (either with error or timeout if it's hanging)\n\trequire.Error(t, err, \"Daemon should fail to start with invalid strategy\")\n\n\t// Check if we got the expected error message\n\toutputStr := string(output)\n\tt.Logf(\"Daemon output with invalid strategy: %s\", outputStr)\n\n\t// The error should mention unknown strategy\n\trequire.Contains(t, outputStr, \"unknown strategy\", \"Should report unknown strategy error\")\n}\n\nfunc testRepoProviderReproviderMigration(t *testing.T) {\n\t// TEST: 17-to-18 migration using 'ipfs repo migrate' command\n\tnode := setupV17RepoWithProviderConfig(t)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\n\t// Run migration using 'ipfs repo migrate' command\n\tresult := node.RunIPFS(\"repo\", \"migrate\")\n\trequire.Empty(t, result.Stderr.String(), \"Migration should succeed without errors\")\n\n\t// Verify same results as daemon migrate\n\thelper := NewMigrationTestHelper(t, configPath)\n\thelper.RequireProviderMigration().\n\t\tRequireFieldEquals(\"Provide.Enabled\", true).\n\t\tRequireFieldEquals(\"Provide.DHT.MaxWorkers\", float64(8)).\n\t\tRequireFieldEquals(\"Provide.Strategy\", \"roots\").\n\t\tRequireFieldEquals(\"Provide.DHT.Interval\", \"24h\")\n}\n\n// =============================================================================\n// HELPER FUNCTIONS\n// =============================================================================\n\n// setupV17RepoWithProviderConfig creates a v17 repo with Provider/Reprovider configuration\nfunc setupV17RepoWithProviderConfig(t *testing.T) *harness.Node {\n\treturn setupV17RepoWithConfig(t,\n\t\tmap[string]any{\n\t\t\t\"Enabled\": true,\n\t\t\t\"WorkerCount\": 8,\n\t\t},\n\t\tmap[string]any{\n\t\t\t\"Strategy\": \"roots\",\n\t\t\t\"Interval\": \"24h\",\n\t\t})\n}\n\n// setupV17RepoWithFlatStrategy creates a v17 repo with \"flat\" strategy for testing conversion\nfunc setupV17RepoWithFlatStrategy(t *testing.T) *harness.Node {\n\treturn setupV17RepoWithConfig(t,\n\t\tmap[string]any{\n\t\t\t\"Enabled\": false,\n\t\t},\n\t\tmap[string]any{\n\t\t\t\"Strategy\": \"flat\", // This should be converted to \"all\"\n\t\t\t\"Interval\": \"12h\",\n\t\t})\n}\n\n// setupV17RepoWithConfig is a helper that creates a v17 repo with specified Provider/Reprovider config\nfunc setupV17RepoWithConfig(t *testing.T, providerConfig, reproviderConfig map[string]any) *harness.Node {\n\tnode := setupStaticV16Repo(t)\n\n\t// First migrate to v17\n\tresult := node.RunIPFS(\"repo\", \"migrate\", \"--to=17\")\n\trequire.Empty(t, result.Stderr.String(), \"Migration to v17 should succeed\")\n\n\t// Update config with specified Provider and Reprovider settings\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tvar config map[string]any\n\tconfigData, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &config))\n\n\tif providerConfig != nil {\n\t\tconfig[\"Provider\"] = providerConfig\n\t} else {\n\t\tconfig[\"Provider\"] = map[string]any{}\n\t}\n\n\tif reproviderConfig != nil {\n\t\tconfig[\"Reprovider\"] = reproviderConfig\n\t} else {\n\t\tconfig[\"Reprovider\"] = map[string]any{}\n\t}\n\n\tmodifiedConfigData, err := json.MarshalIndent(config, \"\", \" \")\n\trequire.NoError(t, err)\n\trequire.NoError(t, os.WriteFile(configPath, modifiedConfigData, 0644))\n\n\treturn node\n}\n\n// setupV17RepoWithEmptySections creates a v17 repo with empty Provider/Reprovider sections\nfunc setupV17RepoWithEmptySections(t *testing.T) *harness.Node {\n\treturn setupV17RepoWithConfig(t,\n\t\tmap[string]any{},\n\t\tmap[string]any{})\n}\n\n// setupV17RepoWithProviderOnly creates a v17 repo with only Provider configuration\nfunc setupV17RepoWithProviderOnly(t *testing.T) *harness.Node {\n\treturn setupV17RepoWithConfig(t,\n\t\tmap[string]any{\n\t\t\t\"Enabled\": false,\n\t\t\t\"WorkerCount\": 32,\n\t\t},\n\t\tmap[string]any{})\n}\n\n// setupV17RepoWithReproviderOnly creates a v17 repo with only Reprovider configuration\nfunc setupV17RepoWithReproviderOnly(t *testing.T) *harness.Node {\n\treturn setupV17RepoWithConfig(t,\n\t\tmap[string]any{},\n\t\tmap[string]any{\n\t\t\t\"Strategy\": \"pinned\",\n\t\t\t\"Interval\": \"48h\",\n\t\t})\n}\n\n// setupV17RepoWithInvalidStrategy creates a v17 repo with an invalid strategy value\nfunc setupV17RepoWithInvalidStrategy(t *testing.T) *harness.Node {\n\treturn setupV17RepoWithConfig(t,\n\t\tmap[string]any{},\n\t\tmap[string]any{\n\t\t\t\"Strategy\": \"invalid-strategy\", // This is not a valid strategy\n\t\t\t\"Interval\": \"24h\",\n\t\t})\n}\n\n// runDaemonMigrationFromV17 monitors daemon startup for 17-to-18 migration only\nfunc runDaemonMigrationFromV17(t *testing.T, node *harness.Node) (string, bool) {\n\t// Monitor only the 17-to-18 migration\n\texpectedMigrations := []struct {\n\t\tpattern string\n\t\tsuccess string\n\t}{\n\t\t{\n\t\t\tpattern: \"applying 17-to-18 repo migration\",\n\t\t\tsuccess: \"Migration 17-to-18 succeeded\",\n\t\t},\n\t}\n\n\treturn runDaemonWithMultipleMigrationMonitoring(t, node, expectedMigrations)\n}\n\n// RequireProviderMigration verifies that Provider/Reprovider have been migrated to Provide section\nfunc (h *MigrationTestHelper) RequireProviderMigration() *MigrationTestHelper {\n\treturn h.RequireFieldExists(\"Provide\").\n\t\tRequireFieldAbsent(\"Provider\").\n\t\tRequireFieldAbsent(\"Reprovider\")\n}\n" }, { "path": "test/cli/migrations/migration_concurrent_test.go", "content": "package migrations\n\n// NOTE: These concurrent migration tests require the local Kubo binary (built with 'make build') to be in PATH.\n//\n// To run these tests successfully:\n// export PATH=\"$(pwd)/cmd/ipfs:$PATH\"\n// go test ./test/cli/migrations/\n\nimport (\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst daemonStartupWait = 2 * time.Second\n\n// TestConcurrentMigrations tests concurrent daemon --migrate attempts\nfunc TestConcurrentMigrations(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"concurrent daemon migrations prevented by lock\", testConcurrentDaemonMigrations)\n}\n\nfunc testConcurrentDaemonMigrations(t *testing.T) {\n\tnode := setupStaticV16Repo(t)\n\n\t// Start first daemon --migrate in background (holds repo.lock)\n\tctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\n\tdefer cancel()\n\n\tfirstDaemon := setupDaemonCmd(ctx, node, \"daemon\", \"--migrate\")\n\trequire.NoError(t, firstDaemon.Start())\n\tdefer func() {\n\t\t// Shutdown first daemon\n\t\tshutdownCmd := setupDaemonCmd(context.Background(), node, \"shutdown\")\n\t\t_ = shutdownCmd.Run()\n\t\t_ = firstDaemon.Wait()\n\t}()\n\n\t// Wait for first daemon to start and acquire lock\n\ttime.Sleep(daemonStartupWait)\n\n\t// Attempt second daemon --migrate (should fail due to lock)\n\tsecondDaemon := setupDaemonCmd(context.Background(), node, \"daemon\", \"--migrate\")\n\toutput, err := secondDaemon.CombinedOutput()\n\tt.Logf(\"Second daemon output: %s\", output)\n\n\t// Should fail with lock error\n\trequire.Error(t, err, \"second daemon should fail when first daemon holds lock\")\n\trequire.Contains(t, string(output), \"lock\", \"error should mention lock\")\n\n\tassertNoTempFiles(t, node.Dir, \"no temp files should be created when lock fails\")\n}\n" }, { "path": "test/cli/migrations/migration_mixed_15_to_latest_test.go", "content": "package migrations\n\n// NOTE: These mixed migration tests validate the transition from old Kubo versions that used external\n// migration binaries to the latest version with embedded migrations. This ensures users can upgrade\n// from very old installations (v15) to the latest version seamlessly.\n//\n// The tests verify hybrid migration paths:\n// - Forward: external binary (15→16) + embedded migrations (16→latest)\n// - Backward: embedded migrations (latest→16) + external binary (16→15)\n//\n// This confirms compatibility between the old external migration system and the new embedded system.\n//\n// To run these tests successfully:\n// export PATH=\"$(pwd)/cmd/ipfs:$PATH\"\n// go test ./test/cli/migrations/\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"slices\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tipfs \"github.com/ipfs/kubo\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestMixedMigration15ToLatest tests migration from old Kubo (v15 with external migrations)\n// to the latest version using a hybrid approach: external binary for 15→16, then embedded\n// migrations for 16→latest. This ensures backward compatibility for users upgrading from\n// very old Kubo installations.\nfunc TestMixedMigration15ToLatest(t *testing.T) {\n\tt.Parallel()\n\n\t// Test mixed migration from v15 to latest (combines external 15→16 + embedded 16→latest)\n\tt.Run(\"daemon migrate: mixed 15 to latest\", testDaemonMigration15ToLatest)\n\tt.Run(\"repo migrate: mixed 15 to latest\", testRepoMigration15ToLatest)\n}\n\n// TestMixedMigrationLatestTo15Downgrade tests downgrading from the latest version back to v15\n// using a hybrid approach: embedded migrations for latest→16, then external binary for 16→15.\n// This ensures the migration system works bidirectionally for recovery scenarios.\nfunc TestMixedMigrationLatestTo15Downgrade(t *testing.T) {\n\tt.Parallel()\n\n\t// Test reverse hybrid migration from latest to v15 (embedded latest→16 + external 16→15)\n\tt.Run(\"repo migrate: reverse hybrid latest to 15\", testRepoReverseHybridMigrationLatestTo15)\n}\n\nfunc testDaemonMigration15ToLatest(t *testing.T) {\n\t// TEST: Migration from v15 to latest using 'ipfs daemon --migrate'\n\t// This tests the mixed migration path: external binary (15→16) + embedded (16→latest)\n\tnode := setupStaticV15Repo(t)\n\n\t// Create mock migration binary for 15→16 (16→17 will use embedded migration)\n\tmockBinDir := createMockMigrationBinary(t, \"15\", \"16\")\n\tcustomPath := buildCustomPath(mockBinDir)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Verify starting conditions\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"15\", strings.TrimSpace(string(versionData)), \"Should start at version 15\")\n\n\t// Read original config to verify preservation of key fields\n\tvar originalConfig map[string]any\n\tconfigData, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &originalConfig))\n\n\toriginalPeerID := getNestedValue(originalConfig, \"Identity.PeerID\")\n\n\t// Run dual migration using daemon --migrate\n\tstdoutOutput, migrationSuccess := runDaemonWithLegacyMigrationMonitoring(t, node, customPath)\n\n\t// Debug output\n\tt.Logf(\"Daemon output:\\n%s\", stdoutOutput)\n\n\t// Verify hybrid migration was successful\n\trequire.True(t, migrationSuccess, \"Hybrid migration should have been successful\")\n\trequire.Contains(t, stdoutOutput, \"Phase 1: External migration from v15 to v16\", \"Should detect external migration phase\")\n\t// Verify each embedded migration step from 16 to latest\n\tverifyMigrationSteps(t, stdoutOutput, 16, ipfs.RepoVersion, true)\n\trequire.Contains(t, stdoutOutput, fmt.Sprintf(\"Phase 2: Embedded migration from v16 to v%d\", ipfs.RepoVersion), \"Should detect embedded migration phase\")\n\trequire.Contains(t, stdoutOutput, \"Hybrid migration completed successfully\", \"Should confirm hybrid migration completion\")\n\n\t// Verify final version is latest\n\tversionData, err = os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\tlatestVersion := fmt.Sprintf(\"%d\", ipfs.RepoVersion)\n\trequire.Equal(t, latestVersion, strings.TrimSpace(string(versionData)), \"Version should be updated to latest\")\n\n\t// Verify config is still valid JSON and key fields preserved\n\tvar finalConfig map[string]any\n\tconfigData, err = os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &finalConfig), \"Config should remain valid JSON\")\n\n\t// Verify essential fields preserved\n\tfinalPeerID := getNestedValue(finalConfig, \"Identity.PeerID\")\n\trequire.Equal(t, originalPeerID, finalPeerID, \"Identity.PeerID should be preserved\")\n\n\t// Verify bootstrap exists (may be modified by 16→17 migration)\n\tfinalBootstrap := getNestedValue(finalConfig, \"Bootstrap\")\n\trequire.NotNil(t, finalBootstrap, \"Bootstrap should exist after migration\")\n\n\t// Verify AutoConf was added by 16→17 migration\n\tautoConf := getNestedValue(finalConfig, \"AutoConf\")\n\trequire.NotNil(t, autoConf, \"AutoConf should be added by 16→17 migration\")\n}\n\nfunc testRepoMigration15ToLatest(t *testing.T) {\n\t// TEST: Migration from v15 to latest using 'ipfs repo migrate'\n\t// Comparison test to verify repo migrate produces same results as daemon migrate\n\tnode := setupStaticV15Repo(t)\n\n\t// Create mock migration binary for 15→16 (16→17 will use embedded migration)\n\tmockBinDir := createMockMigrationBinary(t, \"15\", \"16\")\n\tcustomPath := buildCustomPath(mockBinDir)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Verify starting version\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"15\", strings.TrimSpace(string(versionData)), \"Should start at version 15\")\n\n\t// Run migration using 'ipfs repo migrate' with custom PATH\n\tresult := runMigrationWithCustomPath(node, customPath, \"repo\", \"migrate\")\n\trequire.Empty(t, result.Stderr.String(), \"Migration should succeed without errors\")\n\n\t// Verify final version is latest\n\tversionData, err = os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\tlatestVersion := fmt.Sprintf(\"%d\", ipfs.RepoVersion)\n\trequire.Equal(t, latestVersion, strings.TrimSpace(string(versionData)), \"Version should be updated to latest\")\n\n\t// Verify config is valid JSON\n\tvar finalConfig map[string]any\n\tconfigData, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &finalConfig), \"Config should remain valid JSON\")\n\n\t// Verify essential fields exist\n\trequire.NotNil(t, getNestedValue(finalConfig, \"Identity.PeerID\"), \"Identity.PeerID should exist\")\n\trequire.NotNil(t, getNestedValue(finalConfig, \"Bootstrap\"), \"Bootstrap should exist\")\n\trequire.NotNil(t, getNestedValue(finalConfig, \"AutoConf\"), \"AutoConf should be added\")\n}\n\n// setupStaticV15Repo creates a test node using static v15 repo fixture\n// This ensures tests remain stable and validates migration from very old repos\nfunc setupStaticV15Repo(t *testing.T) *harness.Node {\n\t// Get path to static v15 repo fixture\n\tv15FixturePath := \"testdata/v15-repo\"\n\n\t// Create temporary test directory using Go's testing temp dir\n\ttmpDir := t.TempDir()\n\n\t// Use the built binary (should be in PATH)\n\tnode := harness.BuildNode(\"ipfs\", tmpDir, 0)\n\n\t// Copy static fixture to test directory\n\tcloneStaticRepoFixture(t, v15FixturePath, node.Dir)\n\n\treturn node\n}\n\n// runDaemonWithLegacyMigrationMonitoring monitors for hybrid migration patterns\nfunc runDaemonWithLegacyMigrationMonitoring(t *testing.T, node *harness.Node, customPath string) (string, bool) {\n\t// Monitor for hybrid migration completion - use \"Hybrid migration completed successfully\" as success pattern\n\tstdoutOutput, daemonStarted := runDaemonWithMigrationMonitoringCustomEnv(t, node, \"Using hybrid migration strategy\", \"Hybrid migration completed successfully\", map[string]string{\n\t\t\"PATH\": customPath, // Pass custom PATH with our mock binaries\n\t})\n\n\t// Check for hybrid migration patterns in output\n\thasHybridStart := strings.Contains(stdoutOutput, \"Using hybrid migration strategy\")\n\thasPhase1 := strings.Contains(stdoutOutput, \"Phase 1: External migration from v15 to v16\")\n\thasPhase2 := strings.Contains(stdoutOutput, fmt.Sprintf(\"Phase 2: Embedded migration from v16 to v%d\", ipfs.RepoVersion))\n\thasHybridSuccess := strings.Contains(stdoutOutput, \"Hybrid migration completed successfully\")\n\n\t// Success requires daemon to start and hybrid migration patterns to be detected\n\thybridMigrationSuccess := daemonStarted && hasHybridStart && hasPhase1 && hasPhase2 && hasHybridSuccess\n\n\treturn stdoutOutput, hybridMigrationSuccess\n}\n\n// runDaemonWithMigrationMonitoringCustomEnv is like runDaemonWithMigrationMonitoring but allows custom environment\nfunc runDaemonWithMigrationMonitoringCustomEnv(t *testing.T, node *harness.Node, migrationPattern, successPattern string, extraEnv map[string]string) (string, bool) {\n\t// Create context with timeout as safety net\n\tctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)\n\tdefer cancel()\n\n\t// Set up daemon command with output monitoring\n\tcmd := exec.CommandContext(ctx, node.IPFSBin, \"daemon\", \"--migrate\")\n\tcmd.Dir = node.Dir\n\n\t// Set environment (especially IPFS_PATH)\n\tfor k, v := range node.Runner.Env {\n\t\tcmd.Env = append(cmd.Env, k+\"=\"+v)\n\t}\n\n\t// Add extra environment variables (like PATH with mock binaries)\n\tfor k, v := range extraEnv {\n\t\tcmd.Env = append(cmd.Env, k+\"=\"+v)\n\t}\n\n\t// Set up pipes for output monitoring\n\tstdout, err := cmd.StdoutPipe()\n\trequire.NoError(t, err)\n\tstderr, err := cmd.StderrPipe()\n\trequire.NoError(t, err)\n\n\t// Start the daemon\n\trequire.NoError(t, cmd.Start())\n\n\t// Monitor output from both streams\n\tvar outputBuffer strings.Builder\n\tdone := make(chan bool)\n\tmigrationStarted := false\n\tmigrationCompleted := false\n\n\tgo func() {\n\t\tscanner := bufio.NewScanner(io.MultiReader(stdout, stderr))\n\t\tfor scanner.Scan() {\n\t\t\tline := scanner.Text()\n\t\t\toutputBuffer.WriteString(line + \"\\n\")\n\n\t\t\t// Check for migration start\n\t\t\tif strings.Contains(line, migrationPattern) {\n\t\t\t\tmigrationStarted = true\n\t\t\t}\n\n\t\t\t// Check for migration completion\n\t\t\tif strings.Contains(line, successPattern) {\n\t\t\t\tmigrationCompleted = true\n\t\t\t}\n\n\t\t\t// Check for daemon ready\n\t\t\tif strings.Contains(line, \"Daemon is ready\") {\n\t\t\t\tdone <- true\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t\tdone <- false\n\t}()\n\n\t// Wait for daemon to be ready or timeout\n\tdaemonReady := false\n\tselect {\n\tcase ready := <-done:\n\t\tdaemonReady = ready\n\tcase <-ctx.Done():\n\t\tt.Log(\"Daemon startup timed out\")\n\t}\n\n\t// Stop the daemon using ipfs shutdown command for graceful shutdown\n\tif cmd.Process != nil {\n\t\tshutdownCmd := exec.Command(node.IPFSBin, \"shutdown\")\n\t\tshutdownCmd.Dir = node.Dir\n\t\tfor k, v := range node.Runner.Env {\n\t\t\tshutdownCmd.Env = append(shutdownCmd.Env, k+\"=\"+v)\n\t\t}\n\n\t\tif err := shutdownCmd.Run(); err != nil {\n\t\t\t// If graceful shutdown fails, force kill\n\t\t\t_ = cmd.Process.Kill()\n\t\t}\n\n\t\t// Wait for process to exit\n\t\t_ = cmd.Wait()\n\t}\n\n\treturn outputBuffer.String(), daemonReady && migrationStarted && migrationCompleted\n}\n\n// buildCustomPath creates a custom PATH with mock migration binaries prepended.\n// This is necessary for test isolation when running tests in parallel with t.Parallel().\n// Without isolated PATH handling, parallel tests can interfere with each other through\n// global PATH modifications, causing tests to download real migration binaries instead\n// of using the test mocks.\nfunc buildCustomPath(mockBinDirs ...string) string {\n\t// Prepend mock directories to ensure they're found first\n\tpathElements := append(mockBinDirs, os.Getenv(\"PATH\"))\n\treturn strings.Join(pathElements, string(filepath.ListSeparator))\n}\n\n// runMigrationWithCustomPath runs a migration command with a custom PATH environment.\n// This ensures the migration uses our mock binaries instead of downloading real ones.\nfunc runMigrationWithCustomPath(node *harness.Node, customPath string, args ...string) *harness.RunResult {\n\treturn node.Runner.Run(harness.RunRequest{\n\t\tPath: node.IPFSBin,\n\t\tArgs: args,\n\t\tCmdOpts: []harness.CmdOpt{\n\t\t\tfunc(cmd *exec.Cmd) {\n\t\t\t\t// Remove existing PATH entries using slices.DeleteFunc\n\t\t\t\tcmd.Env = slices.DeleteFunc(cmd.Env, func(s string) bool {\n\t\t\t\t\treturn strings.HasPrefix(s, \"PATH=\")\n\t\t\t\t})\n\t\t\t\t// Add custom PATH\n\t\t\t\tcmd.Env = append(cmd.Env, \"PATH=\"+customPath)\n\t\t\t},\n\t\t},\n\t})\n}\n\n// createMockMigrationBinary creates a platform-agnostic Go binary for migration testing.\n// Returns the directory containing the binary to be added to PATH.\nfunc createMockMigrationBinary(t *testing.T, fromVer, toVer string) string {\n\t// Create bin directory for migration binaries\n\tbinDir := t.TempDir()\n\n\t// Create Go source for mock migration binary\n\tscriptName := fmt.Sprintf(\"fs-repo-%s-to-%s\", fromVer, toVer)\n\tsourceFile := filepath.Join(binDir, scriptName+\".go\")\n\tbinaryPath := filepath.Join(binDir, scriptName)\n\tif runtime.GOOS == \"windows\" {\n\t\tbinaryPath += \".exe\"\n\t}\n\n\t// Generate minimal mock migration binary code\n\tgoSource := fmt.Sprintf(`package main\nimport (\"fmt\"; \"os\"; \"path/filepath\"; \"strings\"; \"time\")\nfunc main() {\n\tvar path string\n\tvar revert bool\n\tfor _, a := range os.Args[1:] {\n\t\tif strings.HasPrefix(a, \"-path=\") { path = a[6:] }\n\t\tif a == \"-revert\" { revert = true }\n\t}\n\tif path == \"\" { fmt.Fprintln(os.Stderr, \"missing -path=\"); os.Exit(1) }\n\n\tfrom, to := \"%s\", \"%s\"\n\tif revert { from, to = to, from }\n\tfmt.Printf(\"fake applying %%s-to-%%s repo migration\\n\", from, to)\n\n\t// Create and immediately remove lock file to simulate proper locking behavior\n\tlockPath := filepath.Join(path, \"repo.lock\")\n\tlockFile, err := os.Create(lockPath)\n\tif err != nil && !os.IsExist(err) {\n\t\tfmt.Fprintf(os.Stderr, \"Error creating lock: %%v\\n\", err)\n\t\tos.Exit(1)\n\t}\n\tif lockFile != nil {\n\t\tlockFile.Close()\n\t\tdefer os.Remove(lockPath)\n\t}\n\n\t// Small delay to simulate migration work\n\ttime.Sleep(10 * time.Millisecond)\n\n\tif err := os.WriteFile(filepath.Join(path, \"version\"), []byte(to), 0644); err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"Error: %%v\\n\", err)\n\t\tos.Exit(1)\n\t}\n}`, fromVer, toVer)\n\n\trequire.NoError(t, os.WriteFile(sourceFile, []byte(goSource), 0644))\n\n\t// Compile the Go binary\n\tcmd := exec.Command(\"go\", \"build\", \"-o\", binaryPath, sourceFile)\n\tcmd.Env = append(os.Environ(), \"CGO_ENABLED=0\") // Ensure static binary\n\trequire.NoError(t, cmd.Run())\n\n\t// Verify the binary exists and is executable\n\t_, err := os.Stat(binaryPath)\n\trequire.NoError(t, err, \"Mock binary should exist\")\n\n\t// Return the bin directory to be added to PATH\n\treturn binDir\n}\n\n// expectedMigrationSteps generates the expected migration step strings for a version range.\n// For forward migrations (from < to), it returns strings like \"Running embedded migration fs-repo-16-to-17\"\n// For reverse migrations (from > to), it returns strings for the reverse path.\nfunc expectedMigrationSteps(from, to int, forward bool) []string {\n\tvar steps []string\n\n\tif forward {\n\t\t// Forward migration: increment by 1 each step\n\t\tfor v := from; v < to; v++ {\n\t\t\tmigrationName := fmt.Sprintf(\"fs-repo-%d-to-%d\", v, v+1)\n\t\t\tsteps = append(steps, fmt.Sprintf(\"Running embedded migration %s\", migrationName))\n\t\t}\n\t} else {\n\t\t// Reverse migration: decrement by 1 each step\n\t\tfor v := from; v > to; v-- {\n\t\t\tmigrationName := fmt.Sprintf(\"fs-repo-%d-to-%d\", v, v-1)\n\t\t\tsteps = append(steps, fmt.Sprintf(\"Running reverse migration %s\", migrationName))\n\t\t}\n\t}\n\n\treturn steps\n}\n\n// verifyMigrationSteps checks that all expected migration steps appear in the output\nfunc verifyMigrationSteps(t *testing.T, output string, from, to int, forward bool) {\n\tsteps := expectedMigrationSteps(from, to, forward)\n\tfor _, step := range steps {\n\t\trequire.Contains(t, output, step, \"Migration output should contain: %s\", step)\n\t}\n}\n\n// getNestedValue retrieves a nested value from a config map using dot notation\nfunc getNestedValue(config map[string]any, path string) any {\n\tparts := strings.Split(path, \".\")\n\tcurrent := any(config)\n\n\tfor _, part := range parts {\n\t\tswitch v := current.(type) {\n\t\tcase map[string]any:\n\t\t\tcurrent = v[part]\n\t\tdefault:\n\t\t\treturn nil\n\t\t}\n\t\tif current == nil {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn current\n}\n\nfunc testRepoReverseHybridMigrationLatestTo15(t *testing.T) {\n\t// TEST: Reverse hybrid migration from latest to v15 using 'ipfs repo migrate --to=15 --allow-downgrade'\n\t// This tests reverse hybrid migration: embedded (17→16) + external (16→15)\n\n\t// Start with v15 fixture and migrate forward to latest to create proper backup files\n\tnode := setupStaticV15Repo(t)\n\n\t// Create mock migration binaries for both forward and reverse migrations\n\tmockBinDirs := []string{\n\t\tcreateMockMigrationBinary(t, \"15\", \"16\"), // for forward migration\n\t\tcreateMockMigrationBinary(t, \"16\", \"15\"), // for downgrade\n\t}\n\tcustomPath := buildCustomPath(mockBinDirs...)\n\n\tconfigPath := filepath.Join(node.Dir, \"config\")\n\tversionPath := filepath.Join(node.Dir, \"version\")\n\n\t// Step 1: Forward migration from v15 to latest to create backup files\n\tt.Logf(\"Step 1: Forward migration v15 → v%d\", ipfs.RepoVersion)\n\tresult := runMigrationWithCustomPath(node, customPath, \"repo\", \"migrate\")\n\n\t// Debug: print the output to see what happened\n\tt.Logf(\"Forward migration stdout:\\n%s\", result.Stdout.String())\n\tt.Logf(\"Forward migration stderr:\\n%s\", result.Stderr.String())\n\n\trequire.Empty(t, result.Stderr.String(), \"Forward migration should succeed without errors\")\n\n\t// Verify we're at latest version after forward migration\n\tversionData, err := os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\tlatestVersion := fmt.Sprintf(\"%d\", ipfs.RepoVersion)\n\trequire.Equal(t, latestVersion, strings.TrimSpace(string(versionData)), \"Should be at latest version after forward migration\")\n\n\t// Read config after forward migration to use as baseline for downgrade\n\tvar latestConfig map[string]any\n\tconfigData, err := os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &latestConfig))\n\n\toriginalPeerID := getNestedValue(latestConfig, \"Identity.PeerID\")\n\n\t// Step 2: Reverse hybrid migration from latest to v15\n\tt.Logf(\"Step 2: Reverse hybrid migration v%d → v15\", ipfs.RepoVersion)\n\tresult = runMigrationWithCustomPath(node, customPath, \"repo\", \"migrate\", \"--to=15\", \"--allow-downgrade\")\n\trequire.Empty(t, result.Stderr.String(), \"Reverse hybrid migration should succeed without errors\")\n\n\t// Debug output\n\tt.Logf(\"Downgrade migration output:\\n%s\", result.Stdout.String())\n\n\t// Verify final version is 15\n\tversionData, err = os.ReadFile(versionPath)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"15\", strings.TrimSpace(string(versionData)), \"Version should be updated to 15\")\n\n\t// Verify config is still valid JSON and key fields preserved\n\tvar finalConfig map[string]any\n\tconfigData, err = os.ReadFile(configPath)\n\trequire.NoError(t, err)\n\trequire.NoError(t, json.Unmarshal(configData, &finalConfig), \"Config should remain valid JSON\")\n\n\t// Verify essential fields preserved\n\tfinalPeerID := getNestedValue(finalConfig, \"Identity.PeerID\")\n\trequire.Equal(t, originalPeerID, finalPeerID, \"Identity.PeerID should be preserved\")\n\n\t// Verify bootstrap exists (may be modified by migrations)\n\tfinalBootstrap := getNestedValue(finalConfig, \"Bootstrap\")\n\trequire.NotNil(t, finalBootstrap, \"Bootstrap should exist after migration\")\n\n\t// AutoConf should be removed by the downgrade (was added in 16→17)\n\tautoConf := getNestedValue(finalConfig, \"AutoConf\")\n\trequire.Nil(t, autoConf, \"AutoConf should be removed by downgrade to v15\")\n}\n" }, { "path": "test/cli/migrations/testdata/v15-repo/blocks/SHARDING", "content": "/repo/flatfs/shard/v1/next-to-last/2\n" }, { "path": "test/cli/migrations/testdata/v15-repo/blocks/X3/CIQFTFEEHEDF6KLBT32BFAGLXEZL4UWFNWM4LFTLMXQBCERZ6CMLX3Y.data", "content": "\n\u0002\b\u0001" }, { "path": "test/cli/migrations/testdata/v15-repo/blocks/_README", "content": "This is a repository of IPLD objects. Each IPLD object is in a single file,\nnamed .data. Where is the\n\"base32\" encoding of the CID (as specified in\nhttps://github.com/multiformats/multibase) without the 'B' prefix.\nAll the object files are placed in a tree of directories, based on a\nfunction of the CID. This is a form of sharding similar to\nthe objects directory in git repositories. Previously, we used\nprefixes, we now use the next-to-last two characters.\n\n func NextToLast(base32cid string) {\n nextToLastLen := 2\n offset := len(base32cid) - nextToLastLen - 1\n return str[offset : offset+nextToLastLen]\n }\n\nFor example, an object with a base58 CIDv1 of\n\n zb2rhYSxw4ZjuzgCnWSt19Q94ERaeFhu9uSqRgjSdx9bsgM6f\n\nhas a base32 CIDv1 of\n\n BAFKREIA22FLID5AJ2KU7URG47MDLROZIH6YF2KALU2PWEFPVI37YLKRSCA\n\nand will be placed at\n\n SC/AFKREIA22FLID5AJ2KU7URG47MDLROZIH6YF2KALU2PWEFPVI37YLKRSCA.data\n\nwith 'SC' being the last-to-next two characters and the 'B' at the\nbeginning of the CIDv1 string is the multibase prefix that is not\nstored in the filename.\n" }, { "path": "test/cli/migrations/testdata/v15-repo/blocks/diskUsage.cache", "content": "{\"diskUsage\":13452,\"accuracy\":\"initial-exact\"}\n" }, { "path": "test/cli/migrations/testdata/v15-repo/config", "content": "{\n \"Identity\": {\n \"PeerID\": \"12D3KooWPeo9gaDV6URwwwyWWjEJsCaMeZ7PBE5vpqvR1KFnPv3B\",\n \"PrivKey\": \"CAESQGPAQlzI5P/KnsbQ3e7dPNbv5Ztw8YwLv9k1dtS3pkd1zZAOR2796fXBZSKyo8Lw/wOqFb9plijC0iW0vTDuxXI=\"\n },\n \"Datastore\": {\n \"StorageMax\": \"10GB\",\n \"StorageGCWatermark\": 90,\n \"GCPeriod\": \"1h\",\n \"Spec\": {\n \"mounts\": [\n {\n \"child\": {\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": true,\n \"type\": \"flatfs\"\n },\n \"mountpoint\": \"/blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"type\": \"measure\"\n },\n {\n \"child\": {\n \"compression\": \"none\",\n \"path\": \"datastore\",\n \"type\": \"levelds\"\n },\n \"mountpoint\": \"/\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"measure\"\n }\n ],\n \"type\": \"mount\"\n },\n \"HashOnRead\": false,\n \"BloomFilterSize\": 0\n },\n \"Addresses\": {\n \"Swarm\": [\n \"/ip4/0.0.0.0/tcp/4001\",\n \"/ip6/::/tcp/4001\",\n \"/ip4/0.0.0.0/udp/4001/quic-v1\",\n \"/ip4/0.0.0.0/udp/4001/quic-v1/webtransport\",\n \"/ip6/::/udp/4001/quic-v1\",\n \"/ip6/::/udp/4001/quic-v1/webtransport\"\n ],\n \"Announce\": [],\n \"AppendAnnounce\": [],\n \"NoAnnounce\": [],\n \"API\": \"/ip4/127.0.0.1/tcp/5001\",\n \"Gateway\": \"/ip4/127.0.0.1/tcp/8080\"\n },\n \"Mounts\": {\n \"IPFS\": \"/ipfs\",\n \"IPNS\": \"/ipns\",\n \"FuseAllowOther\": false\n },\n \"Discovery\": {\n \"MDNS\": {\n \"Enabled\": true\n }\n },\n \"Routing\": {\n \"Routers\": null,\n \"Methods\": null\n },\n \"Ipns\": {\n \"RepublishPeriod\": \"\",\n \"RecordLifetime\": \"\",\n \"ResolveCacheSize\": 128\n },\n \"Bootstrap\": [\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n \"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n \"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n ],\n \"Gateway\": {\n \"HTTPHeaders\": {},\n \"RootRedirect\": \"\",\n \"NoFetch\": false,\n \"NoDNSLink\": false,\n \"DeserializedResponses\": null,\n \"DisableHTMLErrors\": null,\n \"PublicGateways\": null,\n \"ExposeRoutingAPI\": null\n },\n \"API\": {\n \"HTTPHeaders\": {}\n },\n \"Swarm\": {\n \"AddrFilters\": null,\n \"DisableBandwidthMetrics\": false,\n \"DisableNatPortMap\": false,\n \"RelayClient\": {},\n \"RelayService\": {},\n \"Transports\": {\n \"Network\": {},\n \"Security\": {},\n \"Multiplexers\": {}\n },\n \"ConnMgr\": {},\n \"ResourceMgr\": {}\n },\n \"AutoNAT\": {},\n \"Pubsub\": {\n \"Router\": \"\",\n \"DisableSigning\": false\n },\n \"Peering\": {\n \"Peers\": null\n },\n \"DNS\": {\n \"Resolvers\": {}\n },\n \"Migration\": {\n \"DownloadSources\": [],\n \"Keep\": \"\"\n },\n \"Provider\": {\n \"Strategy\": \"\"\n },\n \"Reprovider\": {},\n \"Experimental\": {\n \"FilestoreEnabled\": false,\n \"UrlstoreEnabled\": false,\n \"Libp2pStreamMounting\": false,\n \"P2pHttpProxy\": false,\n \"StrategicProviding\": false,\n \"OptimisticProvide\": false,\n \"OptimisticProvideJobsPoolSize\": 0\n },\n \"Plugins\": {\n \"Plugins\": null\n },\n \"Pinning\": {\n \"RemoteServices\": {}\n },\n \"Import\": {\n \"CidVersion\": null,\n \"UnixFSRawLeaves\": null,\n \"UnixFSChunker\": null,\n \"HashFunction\": null\n },\n \"Internal\": {}\n}" }, { "path": "test/cli/migrations/testdata/v15-repo/datastore/CURRENT", "content": "MANIFEST-000000\n" }, { "path": "test/cli/migrations/testdata/v15-repo/datastore/LOCK", "content": "" }, { "path": "test/cli/migrations/testdata/v15-repo/datastore/LOG", "content": "=============== Aug 4, 2025 (CEST) ===============\n01:47:33.360920 log@legend F·NumFile S·FileSize N·Entry C·BadEntry B·BadBlock Ke·KeyError D·DroppedEntry L·Level Q·SeqNum T·TimeElapsed\n01:47:33.384586 db@open opening\n01:47:33.385359 version@stat F·[] S·0B[] Sc·[]\n01:47:33.397679 db@janitor F·2 G·0\n01:47:33.397725 db@open done T·13.097186ms\n01:47:33.460539 db@close closing\n01:47:33.460679 db@close done T·135.605µs\n" }, { "path": "test/cli/migrations/testdata/v15-repo/datastore_spec", "content": "{\"mounts\":[{\"mountpoint\":\"/blocks\",\"path\":\"blocks\",\"shardFunc\":\"/repo/flatfs/shard/v1/next-to-last/2\",\"type\":\"flatfs\"},{\"mountpoint\":\"/\",\"path\":\"datastore\",\"type\":\"levelds\"}],\"type\":\"mount\"}" }, { "path": "test/cli/migrations/testdata/v15-repo/version", "content": "15\n" }, { "path": "test/cli/migrations/testdata/v16-repo/blocks/SHARDING", "content": "/repo/flatfs/shard/v1/next-to-last/2\n" }, { "path": "test/cli/migrations/testdata/v16-repo/blocks/X3/CIQFTFEEHEDF6KLBT32BFAGLXEZL4UWFNWM4LFTLMXQBCERZ6CMLX3Y.data", "content": "\n\u0002\b\u0001" }, { "path": "test/cli/migrations/testdata/v16-repo/blocks/_README", "content": "This is a repository of IPLD objects. Each IPLD object is in a single file,\nnamed .data. Where is the\n\"base32\" encoding of the CID (as specified in\nhttps://github.com/multiformats/multibase) without the 'B' prefix.\nAll the object files are placed in a tree of directories, based on a\nfunction of the CID. This is a form of sharding similar to\nthe objects directory in git repositories. Previously, we used\nprefixes, we now use the next-to-last two characters.\n\n func NextToLast(base32cid string) {\n nextToLastLen := 2\n offset := len(base32cid) - nextToLastLen - 1\n return str[offset : offset+nextToLastLen]\n }\n\nFor example, an object with a base58 CIDv1 of\n\n zb2rhYSxw4ZjuzgCnWSt19Q94ERaeFhu9uSqRgjSdx9bsgM6f\n\nhas a base32 CIDv1 of\n\n BAFKREIA22FLID5AJ2KU7URG47MDLROZIH6YF2KALU2PWEFPVI37YLKRSCA\n\nand will be placed at\n\n SC/AFKREIA22FLID5AJ2KU7URG47MDLROZIH6YF2KALU2PWEFPVI37YLKRSCA.data\n\nwith 'SC' being the last-to-next two characters and the 'B' at the\nbeginning of the CIDv1 string is the multibase prefix that is not\nstored in the filename.\n" }, { "path": "test/cli/migrations/testdata/v16-repo/blocks/diskUsage.cache", "content": "{\"diskUsage\":13452,\"accuracy\":\"initial-exact\"}\n" }, { "path": "test/cli/migrations/testdata/v16-repo/config", "content": "{\n \"Identity\": {\n \"PeerID\": \"12D3KooWGU72UzYkzVAiTyNLugX72zoDPTGkRegoKcTfB8oWxSuu\",\n \"PrivKey\": \"CAESQNfpGWI4zS+x+HSggBd7qqBai+Je5fopjmBylaTo7uZZYtESGX1PLDr5HmS3NJmrK7glW5kGRuYDvpqwJ2hnC2g=\"\n },\n \"Datastore\": {\n \"StorageMax\": \"10GB\",\n \"StorageGCWatermark\": 90,\n \"GCPeriod\": \"1h\",\n \"Spec\": {\n \"mounts\": [\n {\n \"mountpoint\": \"/blocks\",\n \"path\": \"blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": false,\n \"type\": \"flatfs\"\n },\n {\n \"compression\": \"none\",\n \"mountpoint\": \"/\",\n \"path\": \"datastore\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"levelds\"\n }\n ],\n \"type\": \"mount\"\n },\n \"HashOnRead\": false,\n \"BloomFilterSize\": 0,\n \"BlockKeyCacheSize\": null\n },\n \"Addresses\": {\n \"Swarm\": [\n \"/ip4/0.0.0.0/tcp/0\"\n ],\n \"Announce\": [],\n \"AppendAnnounce\": [],\n \"NoAnnounce\": [],\n \"API\": \"/ip4/127.0.0.1/tcp/0\",\n \"Gateway\": \"/ip4/127.0.0.1/tcp/0\"\n },\n \"Mounts\": {\n \"IPFS\": \"/ipfs\",\n \"IPNS\": \"/ipns\",\n \"MFS\": \"/mfs\",\n \"FuseAllowOther\": false\n },\n \"Discovery\": {\n \"MDNS\": {\n \"Enabled\": true\n }\n },\n \"Routing\": {},\n \"Ipns\": {\n \"RepublishPeriod\": \"\",\n \"RecordLifetime\": \"\",\n \"ResolveCacheSize\": 128\n },\n \"Bootstrap\": [\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\",\n \"/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8\",\n \"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n \"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\",\n \"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"\n ],\n \"Gateway\": {\n \"HTTPHeaders\": {},\n \"RootRedirect\": \"\",\n \"NoFetch\": false,\n \"NoDNSLink\": false,\n \"DeserializedResponses\": null,\n \"DisableHTMLErrors\": null,\n \"PublicGateways\": null,\n \"ExposeRoutingAPI\": null\n },\n \"API\": {\n \"HTTPHeaders\": {}\n },\n \"Swarm\": {\n \"AddrFilters\": null,\n \"DisableBandwidthMetrics\": false,\n \"DisableNatPortMap\": false,\n \"RelayClient\": {},\n \"RelayService\": {},\n \"Transports\": {\n \"Network\": {},\n \"Security\": {},\n \"Multiplexers\": {}\n },\n \"ConnMgr\": {},\n \"ResourceMgr\": {}\n },\n \"AutoNAT\": {},\n \"AutoTLS\": {},\n \"Pubsub\": {\n \"Router\": \"\",\n \"DisableSigning\": false\n },\n \"Peering\": {\n \"Peers\": null\n },\n \"DNS\": {\n \"Resolvers\": {}\n },\n \"Migration\": {\n \"DownloadSources\": [],\n \"Keep\": \"\"\n },\n \"Provider\": {},\n \"Reprovider\": {},\n \"HTTPRetrieval\": {},\n \"Experimental\": {\n \"FilestoreEnabled\": false,\n \"UrlstoreEnabled\": false,\n \"Libp2pStreamMounting\": false,\n \"P2pHttpProxy\": false,\n \"OptimisticProvide\": false,\n \"OptimisticProvideJobsPoolSize\": 0\n },\n \"Plugins\": {\n \"Plugins\": null\n },\n \"Pinning\": {\n \"RemoteServices\": {}\n },\n \"Import\": {\n \"CidVersion\": null,\n \"UnixFSRawLeaves\": null,\n \"UnixFSChunker\": null,\n \"HashFunction\": null,\n \"UnixFSFileMaxLinks\": null,\n \"UnixFSDirectoryMaxLinks\": null,\n \"UnixFSHAMTDirectoryMaxFanout\": null,\n \"UnixFSHAMTDirectorySizeThreshold\": null,\n \"BatchMaxNodes\": null,\n \"BatchMaxSize\": null\n },\n \"Version\": {},\n \"Internal\": {},\n \"Bitswap\": {}\n}\n" }, { "path": "test/cli/migrations/testdata/v16-repo/datastore/CURRENT", "content": "MANIFEST-000000\n" }, { "path": "test/cli/migrations/testdata/v16-repo/datastore/LOCK", "content": "" }, { "path": "test/cli/migrations/testdata/v16-repo/datastore/LOG", "content": "=============== Jul 23, 2025 (CEST) ===============\n19:18:16.721510 log@legend F·NumFile S·FileSize N·Entry C·BadEntry B·BadBlock Ke·KeyError D·DroppedEntry L·Level Q·SeqNum T·TimeElapsed\n19:18:16.746720 db@open opening\n19:18:16.747562 version@stat F·[] S·0B[] Sc·[]\n19:18:16.763409 db@janitor F·2 G·0\n19:18:16.763468 db@open done T·16.722352ms\n19:18:16.831746 db@close closing\n19:18:16.831861 db@close done T·110.694µs\n" }, { "path": "test/cli/migrations/testdata/v16-repo/datastore_spec", "content": "{\"mounts\":[{\"mountpoint\":\"/blocks\",\"path\":\"blocks\",\"shardFunc\":\"/repo/flatfs/shard/v1/next-to-last/2\",\"type\":\"flatfs\"},{\"mountpoint\":\"/\",\"path\":\"datastore\",\"type\":\"levelds\"}],\"type\":\"mount\"}" }, { "path": "test/cli/migrations/testdata/v16-repo/version", "content": "16\n" }, { "path": "test/cli/must.go", "content": "package cli\n\nfunc MustVal[V any](val V, err error) V {\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn val\n}\n" }, { "path": "test/cli/name_test.go", "content": "// Tests for `ipfs name` CLI commands.\n// - TestName: tests name publish, resolve, and inspect\n// - TestNameGetPut: tests name get and put for raw IPNS record handling\n\npackage cli\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/kubo/core/commands/name\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestName(t *testing.T) {\n\tconst (\n\t\tfixturePath = \"fixtures/TestName.car\"\n\t\tfixtureCid = \"bafybeidg3uxibfrt7uqh7zd5yaodetik7wjwi4u7rwv2ndbgj6ec7lsv2a\"\n\t\tdagCid = \"bafyreidgts62p4rtg3rtmptmbv2dt46zjzin275fr763oku3wfod3quzay\"\n\t)\n\n\tmakeDaemon := func(t *testing.T, initArgs []string) *harness.Node {\n\t\tnode := harness.NewT(t).NewNode().Init(append([]string{\"--profile=test\"}, initArgs...)...)\n\t\tr, err := os.Open(fixturePath)\n\t\trequire.Nil(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\trequire.NoError(t, err)\n\t\treturn node\n\t}\n\n\ttestPublishingWithSelf := func(keyType string) {\n\t\tt.Run(\"Publishing with self (keyType = \"+keyType+\")\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\targs := []string{}\n\t\t\tif keyType != \"default\" {\n\t\t\t\targs = append(args, \"-a=\"+keyType)\n\t\t\t}\n\n\t\t\tnode := makeDaemon(t, args)\n\t\t\tname := ipns.NameFromPeer(node.PeerID())\n\n\t\t\tt.Run(\"Publishing a CID\", func(t *testing.T) {\n\t\t\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", publishPath)\n\t\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\t\t\t})\n\n\t\t\tt.Run(\"Publishing a CID with -Q option\", func(t *testing.T) {\n\t\t\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", \"-Q\", publishPath)\n\t\t\t\trequire.Equal(t, name.String()+\"\\n\", res.Stdout.String())\n\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\t\t\t})\n\n\t\t\tt.Run(\"Publishing a CID+subpath\", func(t *testing.T) {\n\t\t\t\tpublishPath := \"/ipfs/\" + fixtureCid + \"/hello\"\n\n\t\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", publishPath)\n\t\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\t\t\t})\n\n\t\t\tt.Run(\"Publishing nothing fails\", func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"name\", \"publish\")\n\t\t\t\trequire.Error(t, res.Err)\n\t\t\t\trequire.Equal(t, 1, res.ExitCode())\n\t\t\t\trequire.Contains(t, res.Stderr.String(), `argument \"ipfs-path\" is required`)\n\t\t\t})\n\n\t\t\tt.Run(\"Publishing with IPLD works\", func(t *testing.T) {\n\t\t\t\tpublishPath := \"/ipld/\" + dagCid + \"/thing\"\n\t\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", publishPath)\n\t\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\t\t\t})\n\n\t\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", publishPath)\n\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\n\t\t\tt.Run(\"Resolving self offline succeeds (daemon off)\", func(t *testing.T) {\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"--offline\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\n\t\t\t\t// Test without cache.\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"--offline\", \"-n\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\t\t\t})\n\n\t\t\tnode.StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\n\t\t\tt.Run(\"Resolving self offline succeeds (daemon on)\", func(t *testing.T) {\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"--offline\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\n\t\t\t\t// Test without cache.\n\t\t\t\tres = node.IPFS(\"name\", \"resolve\", \"--offline\", \"-n\", \"/ipns/\"+name.String())\n\t\t\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\t\t\t})\n\t\t})\n\t}\n\n\ttestPublishingWithSelf(\"default\")\n\ttestPublishingWithSelf(\"rsa\")\n\ttestPublishingWithSelf(\"ed25519\")\n\n\ttestPublishWithKey := func(name string, keyArgs ...string) {\n\t\tt.Run(name, func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := makeDaemon(t, nil)\n\n\t\t\tkeyGenArgs := []string{\"key\", \"gen\"}\n\t\t\tkeyGenArgs = append(keyGenArgs, keyArgs...)\n\t\t\tkeyGenArgs = append(keyGenArgs, \"key\")\n\n\t\t\tres := node.IPFS(keyGenArgs...)\n\t\t\tkey := strings.TrimSpace(res.Stdout.String())\n\n\t\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\t\tname, err := ipns.NameFromString(key)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tres = node.IPFS(\"name\", \"publish\", \"--allow-offline\", \"--key=\"+key, publishPath)\n\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\t\t})\n\t}\n\n\ttestPublishWithKey(\"Publishing with RSA (with b58mh) Key\", \"--ipns-base=b58mh\", \"--type=rsa\", \"--size=2048\")\n\ttestPublishWithKey(\"Publishing with ED25519 (with b58mh) Key\", \"--ipns-base=b58mh\", \"--type=ed25519\")\n\ttestPublishWithKey(\"Publishing with ED25519 (with base36) Key\", \"--ipns-base=base36\", \"--type=ed25519\")\n\n\tt.Run(\"Fails to publish in offline mode\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t, nil).StartDaemon(\"--offline\")\n\t\tdefer node.StopDaemon()\n\t\tres := node.RunIPFS(\"name\", \"publish\", \"/ipfs/\"+fixtureCid)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Equal(t, 1, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"can't publish while offline: pass `--allow-offline` to override or `--allow-delegated` if Ipns.DelegatedPublishers are set up\")\n\t})\n\n\tt.Run(\"Publish V2-only record\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeDaemon(t, nil).StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tipnsName := ipns.NameFromPeer(node.PeerID()).String()\n\t\tipnsPath := ipns.NamespacePrefix + ipnsName\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\tres := node.IPFS(\"name\", \"publish\", \"--ttl=30m\", \"--v1compat=false\", publishPath)\n\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", ipnsName, publishPath), res.Stdout.String())\n\n\t\tres = node.IPFS(\"name\", \"resolve\", ipnsPath)\n\t\trequire.Equal(t, publishPath+\"\\n\", res.Stdout.String())\n\n\t\tres = node.IPFS(\"routing\", \"get\", ipnsPath)\n\t\trecord := res.Stdout.Bytes()\n\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\")\n\t\tout := res.Stdout.String()\n\t\trequire.Contains(t, out, \"This record was not verified.\")\n\t\trequire.Contains(t, out, publishPath)\n\t\trequire.Contains(t, out, \"30m\")\n\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--verify=\"+ipnsPath)\n\t\tout = res.Stdout.String()\n\t\trequire.Contains(t, out, \"Valid: true\")\n\t\trequire.Contains(t, out, \"Signature Type: V2\")\n\t\trequire.Contains(t, out, fmt.Sprintf(\"Protobuf Size: %d\", len(record)))\n\t})\n\n\tt.Run(\"Publish with TTL and inspect record\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeDaemon(t, nil).StartDaemon()\n\t\tt.Cleanup(func() { node.StopDaemon() })\n\t\tipnsPath := ipns.NamespacePrefix + ipns.NameFromPeer(node.PeerID()).String()\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t_ = node.IPFS(\"name\", \"publish\", \"--ttl=30m\", publishPath)\n\t\tres := node.IPFS(\"routing\", \"get\", ipnsPath)\n\t\trecord := res.Stdout.Bytes()\n\n\t\tt.Run(\"Inspect record shows correct TTL and that it is not validated\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\")\n\t\t\tout := res.Stdout.String()\n\t\t\trequire.Contains(t, out, \"This record was not verified.\")\n\t\t\trequire.Contains(t, out, publishPath)\n\t\t\trequire.Contains(t, out, \"30m\")\n\t\t\trequire.Contains(t, out, \"Signature Type: V1+V2\")\n\t\t\trequire.Contains(t, out, fmt.Sprintf(\"Protobuf Size: %d\", len(record)))\n\t\t})\n\n\t\tt.Run(\"Inspect record shows valid with correct name\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--enc=json\", \"--verify=\"+ipnsPath)\n\t\t\tval := name.IpnsInspectResult{}\n\t\t\terr := json.Unmarshal(res.Stdout.Bytes(), &val)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.True(t, val.Validation.Valid)\n\t\t})\n\n\t\tt.Run(\"Inspect record shows invalid with wrong name\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--enc=json\", \"--verify=12D3KooWRirYjmmQATx2kgHBfky6DADsLP7ex1t7BRxJ6nqLs9WH\")\n\t\t\tval := name.IpnsInspectResult{}\n\t\t\terr := json.Unmarshal(res.Stdout.Bytes(), &val)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.False(t, val.Validation.Valid)\n\t\t})\n\t})\n\n\tt.Run(\"Inspect with verification using wrong RSA key errors\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t, nil).StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Prepare RSA Key 1\n\t\tres := node.IPFS(\"key\", \"gen\", \"--type=rsa\", \"--size=4096\", \"key1\")\n\t\tkey1 := strings.TrimSpace(res.Stdout.String())\n\t\tname1, err := ipns.NameFromString(key1)\n\t\trequire.NoError(t, err)\n\n\t\t// Prepare RSA Key 2\n\t\tres = node.IPFS(\"key\", \"gen\", \"--type=rsa\", \"--size=4096\", \"key2\")\n\t\tkey2 := strings.TrimSpace(res.Stdout.String())\n\t\tname2, err := ipns.NameFromString(key2)\n\t\trequire.NoError(t, err)\n\n\t\t// Publish using Key 1\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\tres = node.IPFS(\"name\", \"publish\", \"--allow-offline\", \"--key=\"+key1, publishPath)\n\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name1.String(), publishPath), res.Stdout.String())\n\n\t\t// Get IPNS Record\n\t\tres = node.IPFS(\"routing\", \"get\", ipns.NamespacePrefix+name1.String())\n\t\trecord := res.Stdout.Bytes()\n\n\t\t// Validate with correct key succeeds\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--verify=\"+name1.String(), \"--enc=json\")\n\t\tval := name.IpnsInspectResult{}\n\t\terr = json.Unmarshal(res.Stdout.Bytes(), &val)\n\t\trequire.NoError(t, err)\n\t\trequire.True(t, val.Validation.Valid)\n\n\t\t// Validate with wrong key fails\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--verify=\"+name2.String(), \"--enc=json\")\n\t\tval = name.IpnsInspectResult{}\n\t\terr = json.Unmarshal(res.Stdout.Bytes(), &val)\n\t\trequire.NoError(t, err)\n\t\trequire.False(t, val.Validation.Valid)\n\t})\n\n\tt.Run(\"Publishing with custom sequence number\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeDaemon(t, nil)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\tname := ipns.NameFromPeer(node.PeerID())\n\n\t\tt.Run(\"Publish with sequence=0 is not allowed\", func(t *testing.T) {\n\t\t\t// Sequence=0 is never valid, even on a fresh node\n\t\t\tres := node.RunIPFS(\"name\", \"publish\", \"--allow-offline\", \"--ttl=0\", \"--sequence=0\", publishPath)\n\t\t\trequire.NotEqual(t, 0, res.ExitCode(), \"Expected publish with sequence=0 to fail\")\n\t\t\trequire.Contains(t, res.Stderr.String(), \"sequence number must be greater than the current record sequence\")\n\t\t})\n\n\t\tt.Run(\"Publish with sequence=1 on fresh node\", func(t *testing.T) {\n\t\t\t// Sequence=1 is the minimum valid sequence number for first publish\n\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", \"--ttl=0\", \"--sequence=1\", publishPath)\n\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"Publish with sequence=42\", func(t *testing.T) {\n\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", \"--ttl=0\", \"--sequence=42\", publishPath)\n\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\t\t})\n\n\t\tt.Run(\"Publish with large sequence number\", func(t *testing.T) {\n\t\t\tres := node.IPFS(\"name\", \"publish\", \"--allow-offline\", \"--ttl=0\", \"--sequence=18446744073709551615\", publishPath) // Max uint64\n\t\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath), res.Stdout.String())\n\t\t})\n\t})\n\n\tt.Run(\"Sequence number monotonic check\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeDaemon(t, nil).StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tpublishPath1 := \"/ipfs/\" + fixtureCid\n\t\tpublishPath2 := \"/ipfs/\" + dagCid // Different content\n\t\tname := ipns.NameFromPeer(node.PeerID())\n\n\t\t// First, publish with a high sequence number (1000)\n\t\tres := node.IPFS(\"name\", \"publish\", \"--ttl=0\", \"--sequence=1000\", publishPath1)\n\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath1), res.Stdout.String())\n\n\t\t// Verify the record was published successfully\n\t\tres = node.IPFS(\"name\", \"resolve\", name.String())\n\t\trequire.Contains(t, res.Stdout.String(), publishPath1)\n\n\t\t// Now try to publish different content with a LOWER sequence number (500)\n\t\t// This should fail due to monotonic sequence check\n\t\tres = node.RunIPFS(\"name\", \"publish\", \"--ttl=0\", \"--sequence=500\", publishPath2)\n\t\trequire.NotEqual(t, 0, res.ExitCode(), \"Expected publish with lower sequence to fail\")\n\t\trequire.Contains(t, res.Stderr.String(), \"sequence number\", \"Expected error about sequence number\")\n\n\t\t// Verify the original content is still published (not overwritten)\n\t\tres = node.IPFS(\"name\", \"resolve\", name.String())\n\t\trequire.Contains(t, res.Stdout.String(), publishPath1, \"Original content should still be published\")\n\t\trequire.NotContains(t, res.Stdout.String(), publishPath2, \"New content should not have been published\")\n\n\t\t// Publishing with a HIGHER sequence number should succeed\n\t\tres = node.IPFS(\"name\", \"publish\", \"--ttl=0\", \"--sequence=2000\", publishPath2)\n\t\trequire.Equal(t, fmt.Sprintf(\"Published to %s: %s\\n\", name.String(), publishPath2), res.Stdout.String())\n\n\t\t// Verify the new content is now published\n\t\tres = node.IPFS(\"name\", \"resolve\", name.String())\n\t\trequire.Contains(t, res.Stdout.String(), publishPath2, \"New content should now be published\")\n\t})\n}\n\nfunc TestNameGetPut(t *testing.T) {\n\tt.Parallel()\n\n\tconst (\n\t\tfixturePath = \"fixtures/TestName.car\"\n\t\tfixtureCid = \"bafybeidg3uxibfrt7uqh7zd5yaodetik7wjwi4u7rwv2ndbgj6ec7lsv2a\"\n\t)\n\n\tmakeDaemon := func(t *testing.T, daemonArgs ...string) *harness.Node {\n\t\tnode := harness.NewT(t).NewNode().Init(\"--profile=test\")\n\t\tr, err := os.Open(fixturePath)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\trequire.NoError(t, err)\n\t\treturn node.StartDaemon(daemonArgs...)\n\t}\n\n\t// makeKey creates a unique IPNS key for a test and returns the IPNS name\n\tmakeKey := func(t *testing.T, node *harness.Node, keyName string) ipns.Name {\n\t\tres := node.IPFS(\"key\", \"gen\", \"--type=ed25519\", keyName)\n\t\tkeyID := strings.TrimSpace(res.Stdout.String())\n\t\tname, err := ipns.NameFromString(keyID)\n\t\trequire.NoError(t, err)\n\t\treturn name\n\t}\n\n\t// makeExternalRecord creates an IPNS record on an ephemeral node that is\n\t// shut down before returning. This ensures the test node has no local\n\t// knowledge of the record, properly testing put/get functionality.\n\t// We use short --lifetime so if IPNS records from tests get published on\n\t// the public DHT, they won't waste storage for long.\n\tmakeExternalRecord := func(t *testing.T, h *harness.Harness, publishPath string, publishArgs ...string) (ipns.Name, []byte) {\n\t\tnode := h.NewNode().Init(\"--profile=test\")\n\n\t\tr, err := os.Open(fixturePath)\n\t\trequire.NoError(t, err)\n\t\tdefer r.Close()\n\t\terr = node.IPFSDagImport(r, fixtureCid)\n\t\trequire.NoError(t, err)\n\n\t\tnode.StartDaemon()\n\n\t\tres := node.IPFS(\"key\", \"gen\", \"--type=ed25519\", \"ephemeral-key\")\n\t\tkeyID := strings.TrimSpace(res.Stdout.String())\n\t\tipnsName, err := ipns.NameFromString(keyID)\n\t\trequire.NoError(t, err)\n\n\t\targs := []string{\"name\", \"publish\", \"--key=ephemeral-key\", \"--lifetime=5m\"}\n\t\targs = append(args, publishArgs...)\n\t\targs = append(args, publishPath)\n\t\tnode.IPFS(args...)\n\n\t\tres = node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord := res.Stdout.Bytes()\n\t\trequire.NotEmpty(t, record)\n\n\t\tnode.StopDaemon()\n\n\t\treturn ipnsName, record\n\t}\n\n\tt.Run(\"name get retrieves IPNS record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\t// publish a record first\n\t\tnode.IPFS(\"name\", \"publish\", \"--key=testkey\", \"--lifetime=5m\", publishPath)\n\n\t\t// retrieve the record using name get\n\t\tres := node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord := res.Stdout.Bytes()\n\t\trequire.NotEmpty(t, record, \"expected non-empty IPNS record\")\n\n\t\t// verify the record is valid by inspecting it\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--verify=\"+ipnsName.String())\n\t\trequire.Contains(t, res.Stdout.String(), \"Valid: true\")\n\t\trequire.Contains(t, res.Stdout.String(), publishPath)\n\t})\n\n\tt.Run(\"name get accepts /ipns/ prefix\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\tnode.IPFS(\"name\", \"publish\", \"--key=testkey\", \"--lifetime=5m\", publishPath)\n\n\t\t// retrieve with /ipns/ prefix\n\t\tres := node.IPFS(\"name\", \"get\", \"/ipns/\"+ipnsName.String())\n\t\trecord := res.Stdout.Bytes()\n\t\trequire.NotEmpty(t, record)\n\n\t\t// verify the record\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--verify=\"+ipnsName.String())\n\t\trequire.Contains(t, res.Stdout.String(), \"Valid: true\")\n\t})\n\n\tt.Run(\"name get fails for non-existent name\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// try to get a record for a random peer ID that doesn't exist\n\t\tres := node.RunIPFS(\"name\", \"get\", \"12D3KooWRirYjmmQATx2kgHBfky6DADsLP7ex1t7BRxJ6nqLs9WH\")\n\t\trequire.Error(t, res.Err)\n\t\trequire.NotEqual(t, 0, res.ExitCode())\n\t})\n\n\tt.Run(\"name get fails for invalid name format\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"name\", \"get\", \"not-a-valid-ipns-name\")\n\t\trequire.Error(t, res.Err)\n\t\trequire.NotEqual(t, 0, res.ExitCode())\n\t})\n\n\tt.Run(\"name put accepts /ipns/ prefix\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\tnode.IPFS(\"name\", \"publish\", \"--key=testkey\", \"--lifetime=5m\", publishPath)\n\n\t\tres := node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord := res.Stdout.Bytes()\n\n\t\t// put with /ipns/ prefix\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"put\", \"--force\", \"/ipns/\"+ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\t})\n\n\tt.Run(\"name put fails for invalid name format\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// create a dummy file\n\t\trecordFile := filepath.Join(node.Dir, \"dummy.bin\")\n\t\terr := os.WriteFile(recordFile, []byte(\"dummy\"), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"name\", \"put\", \"not-a-valid-ipns-name\", recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"invalid IPNS name\")\n\t})\n\n\tt.Run(\"name put rejects oversized record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\t// create a file larger than 10 KiB\n\t\toversizedRecord := make([]byte, 11*1024)\n\t\trecordFile := filepath.Join(node.Dir, \"oversized.bin\")\n\t\terr := os.WriteFile(recordFile, oversizedRecord, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"exceeds maximum size\")\n\t})\n\n\tt.Run(\"name put --force skips size check\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\t// create a file larger than 10 KiB\n\t\toversizedRecord := make([]byte, 11*1024)\n\t\trecordFile := filepath.Join(node.Dir, \"oversized.bin\")\n\t\terr := os.WriteFile(recordFile, oversizedRecord, 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// with --force, size check is skipped (but routing will likely reject it)\n\t\tres := node.RunIPFS(\"name\", \"put\", \"--force\", ipnsName.String(), recordFile)\n\t\t// the command itself should not fail on size, but routing may reject\n\t\t// we just verify it doesn't fail with \"exceeds maximum size\"\n\t\tif res.Err != nil {\n\t\t\trequire.NotContains(t, res.Stderr.String(), \"exceeds maximum size\")\n\t\t}\n\t})\n\n\tt.Run(\"name put stores IPNS record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node (shut down before test node starts)\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// start test node (has no local knowledge of the record)\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// put the record (should succeed since no existing record)\n\t\trecordFile := filepath.Join(node.Dir, \"record.bin\")\n\t\terr := os.WriteFile(recordFile, record, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.NoError(t, res.Err)\n\n\t\t// verify the record was stored by getting it back\n\t\tres = node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\tretrievedRecord := res.Stdout.Bytes()\n\t\trequire.Equal(t, record, retrievedRecord, \"stored record should match original\")\n\t})\n\n\tt.Run(\"name put with --force overwrites existing record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// start test node\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// first put the record normally\n\t\trecordFile := filepath.Join(node.Dir, \"record.bin\")\n\t\terr := os.WriteFile(recordFile, record, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.NoError(t, res.Err)\n\n\t\t// put the same record again (identical record republishing is allowed)\n\t\tres = node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.NoError(t, res.Err)\n\n\t\t// put the record with --force (should succeed)\n\t\tres = node.RunIPFS(\"name\", \"put\", \"--force\", ipnsName.String(), recordFile)\n\t\trequire.NoError(t, res.Err)\n\t})\n\n\tt.Run(\"name put validates signature against name\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\t_, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// start test node\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// write the record to a file\n\t\trecordFile := filepath.Join(node.Dir, \"record.bin\")\n\t\terr := os.WriteFile(recordFile, record, 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// try to put with a wrong name (should fail validation)\n\t\twrongName := \"12D3KooWRirYjmmQATx2kgHBfky6DADsLP7ex1t7BRxJ6nqLs9WH\"\n\t\tres := node.RunIPFS(\"name\", \"put\", wrongName, recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"record validation failed\")\n\t})\n\n\tt.Run(\"name put with --force skips command validation\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// start test node\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// with --force the command skips its own validation (signature, sequence check)\n\t\t// and passes the record directly to the routing layer\n\t\tres := node.PipeToIPFS(bytes.NewReader(record), \"name\", \"put\", \"--force\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\t})\n\n\tt.Run(\"name put rejects empty record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\t// create an empty file\n\t\trecordFile := filepath.Join(node.Dir, \"empty.bin\")\n\t\terr := os.WriteFile(recordFile, []byte{}, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"record is empty\")\n\t})\n\n\tt.Run(\"name put rejects invalid record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\tipnsName := makeKey(t, node, \"testkey\")\n\n\t\t// create a file with garbage data\n\t\trecordFile := filepath.Join(node.Dir, \"garbage.bin\")\n\t\terr := os.WriteFile(recordFile, []byte(\"not a valid ipns record\"), 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres := node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"invalid IPNS record\")\n\t})\n\n\tt.Run(\"name put accepts stdin\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// start test node (has no local knowledge of the record)\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// put via stdin (no --force needed since no existing record)\n\t\tres := node.PipeToIPFS(bytes.NewReader(record), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\t})\n\n\tt.Run(\"name put fails when offline without --allow-offline\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// write the record to a file\n\t\trecordFile := filepath.Join(h.Dir, \"record.bin\")\n\t\terr := os.WriteFile(recordFile, record, 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// start test node in offline mode\n\t\tnode := h.NewNode().Init(\"--profile=test\")\n\t\tnode.StartDaemon(\"--offline\")\n\t\tdefer node.StopDaemon()\n\n\t\t// try to put without --allow-offline (should fail)\n\t\tres := node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.Error(t, res.Err)\n\t\t// error can come from our command or from the routing layer\n\t\tstderr := res.Stderr.String()\n\t\trequire.True(t, strings.Contains(stderr, \"offline\") || strings.Contains(stderr, \"online mode\"),\n\t\t\t\"expected offline-related error, got: %s\", stderr)\n\t})\n\n\tt.Run(\"name put succeeds with --allow-offline\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath)\n\n\t\t// write the record to a file\n\t\trecordFile := filepath.Join(h.Dir, \"record.bin\")\n\t\terr := os.WriteFile(recordFile, record, 0644)\n\t\trequire.NoError(t, err)\n\n\t\t// start test node in offline mode\n\t\tnode := h.NewNode().Init(\"--profile=test\")\n\t\tnode.StartDaemon(\"--offline\")\n\t\tdefer node.StopDaemon()\n\n\t\t// put with --allow-offline (should succeed, no --force needed since no existing record)\n\t\tres := node.RunIPFS(\"name\", \"put\", \"--allow-offline\", ipnsName.String(), recordFile)\n\t\trequire.NoError(t, res.Err)\n\t})\n\n\tt.Run(\"name get/put round trip preserves record bytes\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create a record on an ephemeral node\n\t\tipnsName, originalRecord := makeExternalRecord(t, h, publishPath)\n\n\t\t// start test node (has no local knowledge of the record)\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// put the record\n\t\tres := node.PipeToIPFS(bytes.NewReader(originalRecord), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\n\t\t// get the record back\n\t\tres = node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\tretrievedRecord := res.Stdout.Bytes()\n\n\t\t// the records should be byte-for-byte identical\n\t\trequire.Equal(t, originalRecord, retrievedRecord, \"record bytes should be preserved after get/put round trip\")\n\t})\n\n\tt.Run(\"name put --force allows storing lower sequence record\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create an ephemeral node to generate two records with different sequences\n\t\tephNode := h.NewNode().Init(\"--profile=test\")\n\n\t\tr, err := os.Open(fixturePath)\n\t\trequire.NoError(t, err)\n\t\terr = ephNode.IPFSDagImport(r, fixtureCid)\n\t\tr.Close()\n\t\trequire.NoError(t, err)\n\n\t\tephNode.StartDaemon()\n\n\t\tres := ephNode.IPFS(\"key\", \"gen\", \"--type=ed25519\", \"ephemeral-key\")\n\t\tkeyID := strings.TrimSpace(res.Stdout.String())\n\t\tipnsName, err := ipns.NameFromString(keyID)\n\t\trequire.NoError(t, err)\n\n\t\t// publish record with sequence 100\n\t\tephNode.IPFS(\"name\", \"publish\", \"--key=ephemeral-key\", \"--lifetime=5m\", \"--sequence=100\", publishPath)\n\t\tres = ephNode.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord100 := res.Stdout.Bytes()\n\n\t\t// publish record with sequence 200\n\t\tephNode.IPFS(\"name\", \"publish\", \"--key=ephemeral-key\", \"--lifetime=5m\", \"--sequence=200\", publishPath)\n\t\tres = ephNode.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord200 := res.Stdout.Bytes()\n\n\t\tephNode.StopDaemon()\n\n\t\t// start test node (has no local knowledge of the records)\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// helper to get sequence from record\n\t\tgetSequence := func(record []byte) uint64 {\n\t\t\tres := node.PipeToIPFS(bytes.NewReader(record), \"name\", \"inspect\", \"--enc=json\")\n\t\t\tvar result name.IpnsInspectResult\n\t\t\terr := json.Unmarshal(res.Stdout.Bytes(), &result)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.NotNil(t, result.Entry.Sequence)\n\t\t\treturn *result.Entry.Sequence\n\t\t}\n\n\t\t// verify we have the right records\n\t\trequire.Equal(t, uint64(100), getSequence(record100))\n\t\trequire.Equal(t, uint64(200), getSequence(record200))\n\n\t\t// put record with sequence 200 first\n\t\tres = node.PipeToIPFS(bytes.NewReader(record200), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\n\t\t// verify current record has sequence 200\n\t\tres = node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trequire.Equal(t, uint64(200), getSequence(res.Stdout.Bytes()))\n\n\t\t// now put the lower sequence record (100) with --force\n\t\t// this should succeed (--force bypasses our sequence check)\n\t\tres = node.PipeToIPFS(bytes.NewReader(record100), \"name\", \"put\", \"--force\", ipnsName.String())\n\t\trequire.NoError(t, res.Err, \"putting lower sequence record with --force should succeed\")\n\n\t\t// note: when we get the record, IPNS resolution returns the \"best\" record\n\t\t// (highest sequence), so we'll get the sequence 200 record back\n\t\t// this is expected IPNS behavior - the put succeeded, but get returns the best record\n\t\tres = node.IPFS(\"name\", \"get\", ipnsName.String())\n\t\tretrievedSeq := getSequence(res.Stdout.Bytes())\n\t\trequire.Equal(t, uint64(200), retrievedSeq, \"IPNS get returns the best (highest sequence) record\")\n\t})\n\n\tt.Run(\"name put sequence conflict detection\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\t// create an ephemeral node to generate two records with different sequences\n\t\tephNode := h.NewNode().Init(\"--profile=test\")\n\n\t\tr, err := os.Open(fixturePath)\n\t\trequire.NoError(t, err)\n\t\terr = ephNode.IPFSDagImport(r, fixtureCid)\n\t\tr.Close()\n\t\trequire.NoError(t, err)\n\n\t\tephNode.StartDaemon()\n\n\t\tres := ephNode.IPFS(\"key\", \"gen\", \"--type=ed25519\", \"ephemeral-key\")\n\t\tkeyID := strings.TrimSpace(res.Stdout.String())\n\t\tipnsName, err := ipns.NameFromString(keyID)\n\t\trequire.NoError(t, err)\n\n\t\t// publish record with sequence 100\n\t\tephNode.IPFS(\"name\", \"publish\", \"--key=ephemeral-key\", \"--lifetime=5m\", \"--sequence=100\", publishPath)\n\t\tres = ephNode.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord100 := res.Stdout.Bytes()\n\n\t\t// publish record with sequence 200\n\t\tephNode.IPFS(\"name\", \"publish\", \"--key=ephemeral-key\", \"--lifetime=5m\", \"--sequence=200\", publishPath)\n\t\tres = ephNode.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecord200 := res.Stdout.Bytes()\n\n\t\tephNode.StopDaemon()\n\n\t\t// start test node (has no local knowledge of the records)\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// put record with sequence 200 first\n\t\tres = node.PipeToIPFS(bytes.NewReader(record200), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\n\t\t// try to put record with sequence 100 (lower than current 200)\n\t\trecordFile := filepath.Join(node.Dir, \"record100.bin\")\n\t\terr = os.WriteFile(recordFile, record100, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres = node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"existing IPNS record has sequence 200 >= new record sequence 100\")\n\t})\n\n\tt.Run(\"name put allows identical record republishing\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tpublishPath := \"/ipfs/\" + fixtureCid\n\n\t\tipnsName, record := makeExternalRecord(t, h, publishPath, \"--sequence=100\")\n\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// put the record\n\t\tres := node.PipeToIPFS(bytes.NewReader(record), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\n\t\t// put the exact same record again (same bytes, same sequence)\n\t\t// this should succeed: republishing an identical record is a valid use case\n\t\tres = node.PipeToIPFS(bytes.NewReader(record), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\t\trequire.Contains(t, res.Stdout.String(), ipnsName.String())\n\t})\n\n\tt.Run(\"name put rejects different record with same sequence\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// create two different records signed by the same key with the same\n\t\t// sequence number by using two ephemeral nodes that share a key\n\t\tephNode1 := h.NewNode().Init(\"--profile=test\")\n\t\tr, err := os.Open(fixturePath)\n\t\trequire.NoError(t, err)\n\t\terr = ephNode1.IPFSDagImport(r, fixtureCid)\n\t\tr.Close()\n\t\trequire.NoError(t, err)\n\t\tephNode1.StartDaemon()\n\n\t\tres := ephNode1.IPFS(\"key\", \"gen\", \"--type=ed25519\", \"shared-key\")\n\t\tkeyID := strings.TrimSpace(res.Stdout.String())\n\t\tipnsName, err := ipns.NameFromString(keyID)\n\t\trequire.NoError(t, err)\n\n\t\t// publish record A (sequence=100, value=fixtureCid)\n\t\tephNode1.IPFS(\"name\", \"publish\", \"--key=shared-key\", \"--lifetime=5m\", \"--sequence=100\", \"/ipfs/\"+fixtureCid)\n\t\tres = ephNode1.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecordA := res.Stdout.Bytes()\n\n\t\t// export key and import into second ephemeral node\n\t\tkeyFile := filepath.Join(ephNode1.Dir, \"shared-key.key\")\n\t\tephNode1.IPFS(\"key\", \"export\", \"--output=\"+keyFile, \"shared-key\")\n\t\tephNode1.StopDaemon()\n\n\t\tephNode2 := h.NewNode().Init(\"--profile=test\")\n\t\tephNode2.StartDaemon()\n\t\tephNode2.IPFS(\"key\", \"import\", \"shared-key\", keyFile)\n\n\t\t// publish record B (sequence=100, different value)\n\t\tephNode2.IPFS(\"name\", \"publish\", \"--key=shared-key\", \"--lifetime=5m\", \"--sequence=100\", \"/ipfs/bafkqaaa\")\n\t\tres = ephNode2.IPFS(\"name\", \"get\", ipnsName.String())\n\t\trecordB := res.Stdout.Bytes()\n\t\tephNode2.StopDaemon()\n\n\t\t// verify records have same sequence but different bytes\n\t\trequire.NotEqual(t, recordA, recordB, \"records should have different bytes\")\n\n\t\t// start test node and try the put scenario\n\t\tnode := makeDaemon(t)\n\t\tdefer node.StopDaemon()\n\n\t\t// put record A\n\t\tres = node.PipeToIPFS(bytes.NewReader(recordA), \"name\", \"put\", ipnsName.String())\n\t\trequire.NoError(t, res.Err)\n\n\t\t// try to put record B (different bytes, same sequence=100)\n\t\trecordFile := filepath.Join(node.Dir, \"recordB.bin\")\n\t\terr = os.WriteFile(recordFile, recordB, 0644)\n\t\trequire.NoError(t, err)\n\n\t\tres = node.RunIPFS(\"name\", \"put\", ipnsName.String(), recordFile)\n\t\trequire.Error(t, res.Err)\n\t\trequire.Contains(t, res.Stderr.String(), \"existing IPNS record has sequence 100 >= new record sequence 100\")\n\t})\n}\n" }, { "path": "test/cli/p2p_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"os/exec\"\n\t\"slices\"\n\t\"syscall\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/core/commands\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// waitForListenerCount waits until the node has exactly the expected number of listeners.\nfunc waitForListenerCount(t *testing.T, node *harness.Node, expectedCount int) {\n\tt.Helper()\n\trequire.Eventually(t, func() bool {\n\t\tlsOut := node.IPFS(\"p2p\", \"ls\", \"--enc=json\")\n\t\tvar lsResult commands.P2PLsOutput\n\t\tif err := json.Unmarshal(lsOut.Stdout.Bytes(), &lsResult); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn len(lsResult.Listeners) == expectedCount\n\t}, 5*time.Second, 100*time.Millisecond, \"expected %d listeners\", expectedCount)\n}\n\n// waitForListenerProtocol waits until the node has a listener with the given protocol.\nfunc waitForListenerProtocol(t *testing.T, node *harness.Node, protocol string) {\n\tt.Helper()\n\trequire.Eventually(t, func() bool {\n\t\tlsOut := node.IPFS(\"p2p\", \"ls\", \"--enc=json\")\n\t\tvar lsResult commands.P2PLsOutput\n\t\tif err := json.Unmarshal(lsOut.Stdout.Bytes(), &lsResult); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn slices.ContainsFunc(lsResult.Listeners, func(l commands.P2PListenerInfoOutput) bool {\n\t\t\treturn l.Protocol == protocol\n\t\t})\n\t}, 5*time.Second, 100*time.Millisecond, \"expected listener with protocol %s\", protocol)\n}\n\nfunc TestP2PForeground(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"listen foreground creates listener and removes on interrupt\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\tnode.StartDaemon()\n\n\t\tlistenPort := harness.NewRandPort()\n\n\t\t// Start foreground listener asynchronously\n\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"listen\", \"--foreground\", \"/x/fgtest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", listenPort)},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Wait for listener to be created\n\t\twaitForListenerProtocol(t, node, \"/x/fgtest\")\n\n\t\t// Send SIGTERM\n\t\t_ = res.Cmd.Process.Signal(syscall.SIGTERM)\n\t\t_ = res.Cmd.Wait()\n\n\t\t// Wait for listener to be removed\n\t\twaitForListenerCount(t, node, 0)\n\t})\n\n\tt.Run(\"listen foreground text output on SIGTERM\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\tnode.StartDaemon()\n\n\t\tlistenPort := harness.NewRandPort()\n\n\t\t// Run without --enc=json to test actual text output users see\n\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"listen\", \"--foreground\", \"/x/sigterm\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", listenPort)},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\twaitForListenerProtocol(t, node, \"/x/sigterm\")\n\n\t\t_ = res.Cmd.Process.Signal(syscall.SIGTERM)\n\t\t_ = res.Cmd.Wait()\n\n\t\t// Verify stdout shows \"waiting for interrupt\" message\n\t\tstdout := res.Stdout.String()\n\t\trequire.Contains(t, stdout, \"waiting for interrupt\")\n\n\t\t// Note: \"Received interrupt, removing listener\" message is NOT visible to CLI on SIGTERM\n\t\t// because the command runs in the daemon via RPC and the response stream closes before\n\t\t// the message can be emitted. The important behavior is verified in the first test:\n\t\t// the listener IS removed when SIGTERM is sent.\n\t})\n\n\tt.Run(\"forward foreground creates forwarder and removes on interrupt\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\n\t\tforwardPort := harness.NewRandPort()\n\n\t\t// Start foreground forwarder asynchronously on node 0\n\t\tres := nodes[0].Runner.Run(harness.RunRequest{\n\t\t\tPath: nodes[0].IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"forward\", \"--foreground\", \"/x/fgfwd\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", forwardPort), \"/p2p/\" + nodes[1].PeerID().String()},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Wait for forwarder to be created\n\t\twaitForListenerCount(t, nodes[0], 1)\n\n\t\t// Send SIGTERM\n\t\t_ = res.Cmd.Process.Signal(syscall.SIGTERM)\n\t\t_ = res.Cmd.Wait()\n\n\t\t// Wait for forwarder to be removed\n\t\twaitForListenerCount(t, nodes[0], 0)\n\t})\n\n\tt.Run(\"forward foreground text output on SIGTERM\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\n\t\tforwardPort := harness.NewRandPort()\n\n\t\t// Run without --enc=json to test actual text output users see\n\t\tres := nodes[0].Runner.Run(harness.RunRequest{\n\t\t\tPath: nodes[0].IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"forward\", \"--foreground\", \"/x/fwdsigterm\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", forwardPort), \"/p2p/\" + nodes[1].PeerID().String()},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\twaitForListenerCount(t, nodes[0], 1)\n\n\t\t_ = res.Cmd.Process.Signal(syscall.SIGTERM)\n\t\t_ = res.Cmd.Wait()\n\n\t\t// Verify stdout shows \"waiting for interrupt\" message\n\t\tstdout := res.Stdout.String()\n\t\trequire.Contains(t, stdout, \"waiting for interrupt\")\n\n\t\t// Note: \"Received interrupt, removing forwarder\" message is NOT visible to CLI on SIGTERM\n\t\t// because the response stream closes before the message can be emitted.\n\t})\n\n\tt.Run(\"listen without foreground returns immediately and persists\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\tnode.StartDaemon()\n\n\t\tlistenPort := harness.NewRandPort()\n\n\t\t// This should return immediately (not block)\n\t\tnode.IPFS(\"p2p\", \"listen\", \"/x/nofg\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", listenPort))\n\n\t\t// Listener should still exist\n\t\twaitForListenerProtocol(t, node, \"/x/nofg\")\n\n\t\t// Clean up\n\t\tnode.IPFS(\"p2p\", \"close\", \"-p\", \"/x/nofg\")\n\t})\n\n\tt.Run(\"listen foreground text output on p2p close\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\tnode.StartDaemon()\n\n\t\tlistenPort := harness.NewRandPort()\n\n\t\t// Run without --enc=json to test actual text output users see\n\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"listen\", \"--foreground\", \"/x/closetest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", listenPort)},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Wait for listener to be created\n\t\twaitForListenerProtocol(t, node, \"/x/closetest\")\n\n\t\t// Close the listener via ipfs p2p close command\n\t\tnode.IPFS(\"p2p\", \"close\", \"-p\", \"/x/closetest\")\n\n\t\t// Wait for foreground command to exit (it should exit quickly after close)\n\t\tdone := make(chan error, 1)\n\t\tgo func() {\n\t\t\tdone <- res.Cmd.Wait()\n\t\t}()\n\n\t\tselect {\n\t\tcase <-done:\n\t\t\t// Good - command exited\n\t\tcase <-time.After(5 * time.Second):\n\t\t\t_ = res.Cmd.Process.Kill()\n\t\t\tt.Fatal(\"foreground command did not exit after listener was closed via ipfs p2p close\")\n\t\t}\n\n\t\t// Wait for listener to be removed\n\t\twaitForListenerCount(t, node, 0)\n\n\t\t// Verify text output shows BOTH messages when closed via p2p close\n\t\t// (unlike SIGTERM, the stream is still open so \"Received interrupt\" is emitted)\n\t\tout := res.Stdout.String()\n\t\trequire.Contains(t, out, \"waiting for interrupt\")\n\t\trequire.Contains(t, out, \"Received interrupt, removing listener\")\n\t})\n\n\tt.Run(\"forward foreground text output on p2p close\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\n\t\tforwardPort := harness.NewRandPort()\n\n\t\t// Run without --enc=json to test actual text output users see\n\t\tres := nodes[0].Runner.Run(harness.RunRequest{\n\t\t\tPath: nodes[0].IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"forward\", \"--foreground\", \"/x/fwdclose\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", forwardPort), \"/p2p/\" + nodes[1].PeerID().String()},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Wait for forwarder to be created\n\t\twaitForListenerCount(t, nodes[0], 1)\n\n\t\t// Close the forwarder via ipfs p2p close command\n\t\tnodes[0].IPFS(\"p2p\", \"close\", \"-a\")\n\n\t\t// Wait for foreground command to exit\n\t\tdone := make(chan error, 1)\n\t\tgo func() {\n\t\t\tdone <- res.Cmd.Wait()\n\t\t}()\n\n\t\tselect {\n\t\tcase <-done:\n\t\t\t// Good - command exited\n\t\tcase <-time.After(5 * time.Second):\n\t\t\t_ = res.Cmd.Process.Kill()\n\t\t\tt.Fatal(\"foreground command did not exit after forwarder was closed via ipfs p2p close\")\n\t\t}\n\n\t\t// Wait for forwarder to be removed\n\t\twaitForListenerCount(t, nodes[0], 0)\n\n\t\t// Verify text output shows BOTH messages when closed via p2p close\n\t\tout := res.Stdout.String()\n\t\trequire.Contains(t, out, \"waiting for interrupt\")\n\t\trequire.Contains(t, out, \"Received interrupt, removing forwarder\")\n\t})\n\n\tt.Run(\"listen foreground tunnel transfers data and cleans up on SIGTERM\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\n\t\thttpServerPort := harness.NewRandPort()\n\t\tforwardPort := harness.NewRandPort()\n\n\t\t// Start HTTP server\n\t\texpectedBody := \"Hello from p2p tunnel!\"\n\t\thttpServer := &http.Server{\n\t\t\tAddr: fmt.Sprintf(\"127.0.0.1:%d\", httpServerPort),\n\t\t\tHandler: http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {\n\t\t\t\t_, _ = w.Write([]byte(expectedBody))\n\t\t\t}),\n\t\t}\n\t\tlistener, err := net.Listen(\"tcp\", httpServer.Addr)\n\t\trequire.NoError(t, err)\n\t\tgo func() { _ = httpServer.Serve(listener) }()\n\t\tdefer httpServer.Close()\n\n\t\t// Node 0: listen --foreground\n\t\tlistenRes := nodes[0].Runner.Run(harness.RunRequest{\n\t\t\tPath: nodes[0].IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"listen\", \"--foreground\", \"/x/httptest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", httpServerPort)},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, listenRes.Err)\n\n\t\t// Wait for listener to be created\n\t\twaitForListenerProtocol(t, nodes[0], \"/x/httptest\")\n\n\t\t// Node 1: forward (non-foreground)\n\t\tnodes[1].IPFS(\"p2p\", \"forward\", \"/x/httptest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", forwardPort), \"/p2p/\"+nodes[0].PeerID().String())\n\n\t\t// Verify data flows through tunnel\n\t\tresp, err := http.Get(fmt.Sprintf(\"http://127.0.0.1:%d/\", forwardPort))\n\t\trequire.NoError(t, err)\n\t\tbody, err := io.ReadAll(resp.Body)\n\t\tresp.Body.Close()\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, expectedBody, string(body))\n\n\t\t// Clean up forwarder on node 1\n\t\tnodes[1].IPFS(\"p2p\", \"close\", \"-a\")\n\n\t\t// SIGTERM the listen --foreground command\n\t\t_ = listenRes.Cmd.Process.Signal(syscall.SIGTERM)\n\t\t_ = listenRes.Cmd.Wait()\n\n\t\t// Wait for listener to be removed on node 0\n\t\twaitForListenerCount(t, nodes[0], 0)\n\t})\n\n\tt.Run(\"forward foreground tunnel transfers data and cleans up on SIGTERM\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\n\t\thttpServerPort := harness.NewRandPort()\n\t\tforwardPort := harness.NewRandPort()\n\n\t\t// Start HTTP server\n\t\texpectedBody := \"Hello from forward foreground tunnel!\"\n\t\thttpServer := &http.Server{\n\t\t\tAddr: fmt.Sprintf(\"127.0.0.1:%d\", httpServerPort),\n\t\t\tHandler: http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {\n\t\t\t\t_, _ = w.Write([]byte(expectedBody))\n\t\t\t}),\n\t\t}\n\t\tlistener, err := net.Listen(\"tcp\", httpServer.Addr)\n\t\trequire.NoError(t, err)\n\t\tgo func() { _ = httpServer.Serve(listener) }()\n\t\tdefer httpServer.Close()\n\n\t\t// Node 0: listen (non-foreground)\n\t\tnodes[0].IPFS(\"p2p\", \"listen\", \"/x/httptest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", httpServerPort))\n\n\t\t// Node 1: forward --foreground\n\t\tforwardRes := nodes[1].Runner.Run(harness.RunRequest{\n\t\t\tPath: nodes[1].IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"forward\", \"--foreground\", \"/x/httptest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", forwardPort), \"/p2p/\" + nodes[0].PeerID().String()},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, forwardRes.Err)\n\n\t\t// Wait for forwarder to be created\n\t\twaitForListenerCount(t, nodes[1], 1)\n\n\t\t// Verify data flows through tunnel\n\t\tresp, err := http.Get(fmt.Sprintf(\"http://127.0.0.1:%d/\", forwardPort))\n\t\trequire.NoError(t, err)\n\t\tbody, err := io.ReadAll(resp.Body)\n\t\tresp.Body.Close()\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, expectedBody, string(body))\n\n\t\t// SIGTERM the forward --foreground command\n\t\t_ = forwardRes.Cmd.Process.Signal(syscall.SIGTERM)\n\t\t_ = forwardRes.Cmd.Wait()\n\n\t\t// Wait for forwarder to be removed on node 1\n\t\twaitForListenerCount(t, nodes[1], 0)\n\n\t\t// Clean up listener on node 0\n\t\tnodes[0].IPFS(\"p2p\", \"close\", \"-a\")\n\t})\n\n\tt.Run(\"foreground command exits when daemon shuts down\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\")\n\t\tnode.StartDaemon()\n\n\t\tlistenPort := harness.NewRandPort()\n\n\t\t// Start foreground listener\n\t\tres := node.Runner.Run(harness.RunRequest{\n\t\t\tPath: node.IPFSBin,\n\t\t\tArgs: []string{\"p2p\", \"listen\", \"--foreground\", \"/x/daemontest\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", listenPort)},\n\t\t\tRunFunc: (*exec.Cmd).Start,\n\t\t})\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Wait for listener to be created\n\t\twaitForListenerProtocol(t, node, \"/x/daemontest\")\n\n\t\t// Stop the daemon\n\t\tnode.StopDaemon()\n\n\t\t// Wait for foreground command to exit\n\t\tdone := make(chan error, 1)\n\t\tgo func() {\n\t\t\tdone <- res.Cmd.Wait()\n\t\t}()\n\n\t\tselect {\n\t\tcase <-done:\n\t\t\t// Good - foreground command exited when daemon stopped\n\t\tcase <-time.After(5 * time.Second):\n\t\t\t_ = res.Cmd.Process.Kill()\n\t\t\tt.Fatal(\"foreground command did not exit when daemon was stopped\")\n\t\t}\n\t})\n}\n" }, { "path": "test/cli/peering_test.go", "content": "package cli\n\nimport (\n\t\"slices\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestPeering(t *testing.T) {\n\tt.Parallel()\n\n\tcontainsPeerID := func(p peer.ID, peers []peer.ID) bool {\n\t\treturn slices.Contains(peers, p)\n\t}\n\n\tassertPeered := func(h *harness.Harness, from *harness.Node, to *harness.Node) {\n\t\tassert.Eventuallyf(t, func() bool {\n\t\t\tfromPeers := from.Peers()\n\t\t\tif len(fromPeers) == 0 {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tvar fromPeerIDs []peer.ID\n\t\t\tfor _, p := range fromPeers {\n\t\t\t\tfromPeerIDs = append(fromPeerIDs, h.ExtractPeerID(p))\n\t\t\t}\n\t\t\treturn containsPeerID(to.PeerID(), fromPeerIDs)\n\t\t}, time.Minute, 10*time.Millisecond, \"%d -> %d not peered\", from.ID, to.ID)\n\t}\n\n\tassertNotPeered := func(h *harness.Harness, from *harness.Node, to *harness.Node) {\n\t\tassert.Eventuallyf(t, func() bool {\n\t\t\tfromPeers := from.Peers()\n\t\t\tif len(fromPeers) == 0 {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tvar fromPeerIDs []peer.ID\n\t\t\tfor _, p := range fromPeers {\n\t\t\t\tfromPeerIDs = append(fromPeerIDs, h.ExtractPeerID(p))\n\t\t\t}\n\t\t\treturn !containsPeerID(to.PeerID(), fromPeerIDs)\n\t\t}, 20*time.Second, 10*time.Millisecond, \"%d -> %d peered\", from.ID, to.ID)\n\t}\n\n\tassertPeerings := func(h *harness.Harness, nodes []*harness.Node, peerings []harness.Peering) {\n\t\tForEachPar(peerings, func(peering harness.Peering) {\n\t\t\tassertPeered(h, nodes[peering.From], nodes[peering.To])\n\t\t})\n\t}\n\n\tt.Run(\"bidirectional peering should work (simultaneous connect)\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tpeerings := []harness.Peering{{From: 0, To: 1}, {From: 1, To: 0}, {From: 1, To: 2}}\n\t\th, nodes := harness.CreatePeerNodes(t, 3, peerings)\n\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\t\tassertPeerings(h, nodes, peerings)\n\n\t\tnodes[0].Disconnect(nodes[1])\n\t\tassertPeerings(h, nodes, peerings)\n\t})\n\n\tt.Run(\"1 should reconnect to 2 when 2 disconnects from 1\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tpeerings := []harness.Peering{{From: 0, To: 1}, {From: 1, To: 0}, {From: 1, To: 2}}\n\t\th, nodes := harness.CreatePeerNodes(t, 3, peerings)\n\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\t\tassertPeerings(h, nodes, peerings)\n\n\t\tnodes[2].Disconnect(nodes[1])\n\t\tassertPeerings(h, nodes, peerings)\n\t})\n\n\tt.Run(\"1 will peer with 2 when it comes online\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tpeerings := []harness.Peering{{From: 0, To: 1}, {From: 1, To: 0}, {From: 1, To: 2}}\n\t\th, nodes := harness.CreatePeerNodes(t, 3, peerings)\n\n\t\tdefer nodes.StopDaemons()\n\t\tnodes[0].StartDaemon()\n\t\tnodes[1].StartDaemon()\n\t\tassertPeerings(h, nodes, []harness.Peering{{From: 0, To: 1}, {From: 1, To: 0}})\n\n\t\tnodes[2].StartDaemon()\n\t\tassertPeerings(h, nodes, peerings)\n\t})\n\n\tt.Run(\"1 will re-peer with 2 when it disconnects and then comes back online\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tpeerings := []harness.Peering{{From: 0, To: 1}, {From: 1, To: 0}, {From: 1, To: 2}}\n\t\th, nodes := harness.CreatePeerNodes(t, 3, peerings)\n\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\t\tassertPeerings(h, nodes, peerings)\n\n\t\tnodes[2].StopDaemon()\n\t\tassertNotPeered(h, nodes[1], nodes[2])\n\n\t\tnodes[2].StartDaemon()\n\t\tassertPeerings(h, nodes, peerings)\n\t})\n}\n" }, { "path": "test/cli/pin_ls_names_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// pinInfo represents the JSON structure for pin ls output\ntype pinInfo struct {\n\tType string `json:\"Type\"`\n\tName string `json:\"Name\"`\n}\n\n// pinLsJSON represents the JSON output structure for pin ls command\ntype pinLsJSON struct {\n\tKeys map[string]pinInfo `json:\"Keys\"`\n}\n\n// Helper function to initialize a test node with daemon\nfunc setupTestNode(t *testing.T) *harness.Node {\n\tt.Helper()\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon(\"--offline\")\n\tt.Cleanup(func() {\n\t\tnode.StopDaemon()\n\t})\n\treturn node\n}\n\n// Helper function to assert pin name and CID are present in output\nfunc assertPinOutput(t *testing.T, output, cid, pinName string) {\n\tt.Helper()\n\trequire.Contains(t, output, pinName, \"pin name '%s' not found in output: %s\", pinName, output)\n\trequire.Contains(t, output, cid, \"CID %s not found in output: %s\", cid, output)\n}\n\n// Helper function to assert CID is present but name is not\nfunc assertCIDOnly(t *testing.T, output, cid string) {\n\tt.Helper()\n\trequire.Contains(t, output, cid, \"CID %s not found in output: %s\", cid, output)\n}\n\n// Helper function to assert neither CID nor name are present\nfunc assertNotPresent(t *testing.T, output, cid, pinName string) {\n\tt.Helper()\n\trequire.NotContains(t, output, cid, \"CID %s should not be present in output: %s\", cid, output)\n\trequire.NotContains(t, output, pinName, \"pin name '%s' should not be present in output: %s\", pinName, output)\n}\n\n// Test that pin ls returns names when querying specific CIDs with --names flag\nfunc TestPinLsWithNamesForSpecificCIDs(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"pin ls with specific CID returns name\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add content without pinning\n\t\tcidA := node.IPFSAddStr(\"content A\", \"--pin=false\")\n\t\tcidB := node.IPFSAddStr(\"content B\", \"--pin=false\")\n\t\tcidC := node.IPFSAddStr(\"content C\", \"--pin=false\")\n\n\t\t// Pin with names\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=pin-a\", cidA)\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=pin-b\", cidB)\n\t\tnode.IPFS(\"pin\", \"add\", cidC) // No name\n\n\t\t// Test: pin ls --names should return the name\n\t\tres := node.IPFS(\"pin\", \"ls\", cidA, \"--names\")\n\t\tassertPinOutput(t, res.Stdout.String(), cidA, \"pin-a\")\n\n\t\tres = node.IPFS(\"pin\", \"ls\", cidB, \"--names\")\n\t\tassertPinOutput(t, res.Stdout.String(), cidB, \"pin-b\")\n\n\t\t// Test: pin without name should work\n\t\tres = node.IPFS(\"pin\", \"ls\", cidC, \"--names\")\n\t\toutput := res.Stdout.String()\n\t\tassertCIDOnly(t, output, cidC)\n\t\trequire.Contains(t, output, \"recursive\", \"pin type 'recursive' not found for CID %s in output: %s\", cidC, output)\n\n\t\t// Test: without --names flag, no names returned\n\t\tres = node.IPFS(\"pin\", \"ls\", cidA)\n\t\toutput = res.Stdout.String()\n\t\trequire.NotContains(t, output, \"pin-a\", \"pin name 'pin-a' should not be present without --names flag, but found in: %s\", output)\n\t\tassertCIDOnly(t, output, cidA)\n\t})\n\n\tt.Run(\"pin ls with multiple CIDs returns names\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Create test content\n\t\tcidA := node.IPFSAddStr(\"multi A\", \"--pin=false\")\n\t\tcidB := node.IPFSAddStr(\"multi B\", \"--pin=false\")\n\n\t\t// Pin with names\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=multi-pin-a\", cidA)\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=multi-pin-b\", cidB)\n\n\t\t// Test multiple CIDs at once\n\t\tres := node.IPFS(\"pin\", \"ls\", cidA, cidB, \"--names\")\n\t\toutput := res.Stdout.String()\n\t\tassertPinOutput(t, output, cidA, \"multi-pin-a\")\n\t\tassertPinOutput(t, output, cidB, \"multi-pin-b\")\n\t})\n\n\tt.Run(\"pin ls without CID lists all pins with names\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Create and pin content with names\n\t\tcidA := node.IPFSAddStr(\"list all A\", \"--pin=false\")\n\t\tcidB := node.IPFSAddStr(\"list all B\", \"--pin=false\")\n\t\tcidC := node.IPFSAddStr(\"list all C\", \"--pin=false\")\n\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=all-pin-a\", cidA)\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=all-pin-b\", \"--recursive=false\", cidB)\n\t\tnode.IPFS(\"pin\", \"add\", cidC) // No name\n\n\t\t// Test: pin ls --names (without CID) should list all pins with their names\n\t\tres := node.IPFS(\"pin\", \"ls\", \"--names\")\n\t\toutput := res.Stdout.String()\n\n\t\t// Should contain all pins with their names\n\t\tassertPinOutput(t, output, cidA, \"all-pin-a\")\n\t\tassertPinOutput(t, output, cidB, \"all-pin-b\")\n\t\tassertCIDOnly(t, output, cidC)\n\n\t\t// Pin C should appear but without a name (just type)\n\t\tlines := strings.SplitSeq(output, \"\\n\")\n\t\tfor line := range lines {\n\t\t\tif strings.Contains(line, cidC) {\n\t\t\t\t// Should have CID and type but no name\n\t\t\t\trequire.Contains(t, line, \"recursive\", \"pin type 'recursive' not found for unnamed pin %s in line: %s\", cidC, line)\n\t\t\t\trequire.NotContains(t, line, \"all-pin\", \"pin name should not be present for unnamed pin %s, but found in line: %s\", cidC, line)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"pin ls --type with --names\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Create test content\n\t\tcidDirect := node.IPFSAddStr(\"direct content\", \"--pin=false\")\n\t\tcidRecursive := node.IPFSAddStr(\"recursive content\", \"--pin=false\")\n\n\t\t// Create a DAG for indirect testing\n\t\tchildCid := node.IPFSAddStr(\"child for indirect\", \"--pin=false\")\n\t\tparentContent := fmt.Sprintf(`{\"link\": \"/ipfs/%s\"}`, childCid)\n\t\tparentCid := node.PipeStrToIPFS(parentContent, \"dag\", \"put\", \"--input-codec=json\", \"--store-codec=dag-cbor\").Stdout.Trimmed()\n\n\t\t// Pin with different types and names\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=direct-pin\", \"--recursive=false\", cidDirect)\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=recursive-pin\", cidRecursive)\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=parent-pin\", parentCid)\n\n\t\t// Test: --type=direct --names\n\t\tres := node.IPFS(\"pin\", \"ls\", \"--type=direct\", \"--names\")\n\t\toutput := res.Stdout.String()\n\t\tassertPinOutput(t, output, cidDirect, \"direct-pin\")\n\t\tassertNotPresent(t, output, cidRecursive, \"recursive-pin\")\n\n\t\t// Test: --type=recursive --names\n\t\tres = node.IPFS(\"pin\", \"ls\", \"--type=recursive\", \"--names\")\n\t\toutput = res.Stdout.String()\n\t\tassertPinOutput(t, output, cidRecursive, \"recursive-pin\")\n\t\tassertPinOutput(t, output, parentCid, \"parent-pin\")\n\t\tassertNotPresent(t, output, cidDirect, \"direct-pin\")\n\n\t\t// Test: --type=indirect with proper directory structure\n\t\t// Create a directory with a file for indirect pin testing\n\t\tdirPath := t.TempDir()\n\t\trequire.NoError(t, os.WriteFile(filepath.Join(dirPath, \"file.txt\"), []byte(\"test content\"), 0644))\n\n\t\t// Add directory recursively\n\t\tdirAddRes := node.IPFS(\"add\", \"-r\", \"-q\", dirPath)\n\t\tdirCidStr := strings.TrimSpace(dirAddRes.Stdout.Lines()[len(dirAddRes.Stdout.Lines())-1])\n\n\t\t// Add file separately without pinning to get its CID\n\t\tfileAddRes := node.IPFS(\"add\", \"-q\", \"--pin=false\", filepath.Join(dirPath, \"file.txt\"))\n\t\tfileCidStr := strings.TrimSpace(fileAddRes.Stdout.String())\n\n\t\t// Check if file shows as indirect\n\t\tres = node.IPFS(\"pin\", \"ls\", \"--type=indirect\", fileCidStr)\n\t\toutput = res.Stdout.String()\n\t\trequire.Contains(t, output, fileCidStr, \"indirect pin CID %s not found in output: %s\", fileCidStr, output)\n\t\trequire.Contains(t, output, \"indirect through \"+dirCidStr, \"indirect relationship not found for CID %s through %s in output: %s\", fileCidStr, dirCidStr, output)\n\n\t\t// Test: --type=all --names\n\t\tres = node.IPFS(\"pin\", \"ls\", \"--type=all\", \"--names\")\n\t\toutput = res.Stdout.String()\n\t\tassertPinOutput(t, output, cidDirect, \"direct-pin\")\n\t\tassertPinOutput(t, output, cidRecursive, \"recursive-pin\")\n\t\tassertPinOutput(t, output, parentCid, \"parent-pin\")\n\t\t// Indirect pins are included in --type=all output\n\t})\n\n\tt.Run(\"pin ls JSON output with names\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add and pin content with name\n\t\tcidA := node.IPFSAddStr(\"json content\", \"--pin=false\")\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=json-pin\", cidA)\n\n\t\t// Test JSON output with specific CID\n\t\tres := node.IPFS(\"pin\", \"ls\", cidA, \"--names\", \"--enc=json\")\n\t\tvar pinOutput pinLsJSON\n\t\terr := json.Unmarshal([]byte(res.Stdout.String()), &pinOutput)\n\t\trequire.NoError(t, err, \"failed to unmarshal JSON output: %s\", res.Stdout.String())\n\n\t\tpinData, ok := pinOutput.Keys[cidA]\n\t\trequire.True(t, ok, \"CID %s should be in Keys map, got: %+v\", cidA, pinOutput.Keys)\n\t\trequire.Equal(t, \"recursive\", pinData.Type, \"expected pin type 'recursive', got '%s'\", pinData.Type)\n\t\trequire.Equal(t, \"json-pin\", pinData.Name, \"expected pin name 'json-pin', got '%s'\", pinData.Name)\n\n\t\t// Without names flag\n\t\tres = node.IPFS(\"pin\", \"ls\", cidA, \"--enc=json\")\n\t\terr = json.Unmarshal([]byte(res.Stdout.String()), &pinOutput)\n\t\trequire.NoError(t, err, \"failed to unmarshal JSON output: %s\", res.Stdout.String())\n\n\t\tpinData, ok = pinOutput.Keys[cidA]\n\t\trequire.True(t, ok, \"CID %s should be in Keys map, got: %+v\", cidA, pinOutput.Keys)\n\t\t// Name should be empty without --names flag\n\t\trequire.Equal(t, \"\", pinData.Name, \"pin name should be empty without --names flag, got '%s'\", pinData.Name)\n\n\t\t// Test JSON output without CID (list all)\n\t\tres = node.IPFS(\"pin\", \"ls\", \"--names\", \"--enc=json\")\n\t\tvar listOutput pinLsJSON\n\t\terr = json.Unmarshal([]byte(res.Stdout.String()), &listOutput)\n\t\trequire.NoError(t, err, \"failed to unmarshal JSON list output: %s\", res.Stdout.String())\n\t\t// Should have at least one pin (the one we just added)\n\t\trequire.NotEmpty(t, listOutput.Keys, \"pin list should not be empty\")\n\t\t// Check that our pin is in the list\n\t\tpinData, ok = listOutput.Keys[cidA]\n\t\trequire.True(t, ok, \"our pin with CID %s should be in the list, got: %+v\", cidA, listOutput.Keys)\n\t\trequire.Equal(t, \"json-pin\", pinData.Name, \"expected pin name 'json-pin' in list, got '%s'\", pinData.Name)\n\t})\n\n\tt.Run(\"direct and indirect pins with names\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Create a small DAG: parent -> child\n\t\tchildCid := node.IPFSAddStr(\"child content\", \"--pin=false\")\n\n\t\t// Create parent that references child\n\t\tparentContent := fmt.Sprintf(`{\"link\": \"/ipfs/%s\"}`, childCid)\n\t\tparentCid := node.PipeStrToIPFS(parentContent, \"dag\", \"put\", \"--input-codec=json\", \"--store-codec=dag-cbor\").Stdout.Trimmed()\n\n\t\t// Pin child directly with a name\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=direct-child\", \"--recursive=false\", childCid)\n\n\t\t// Pin parent recursively with a name\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=recursive-parent\", parentCid)\n\n\t\t// Check direct pin with specific CID\n\t\tres := node.IPFS(\"pin\", \"ls\", \"--type=direct\", childCid, \"--names\")\n\t\toutput := res.Stdout.String()\n\t\trequire.Contains(t, output, \"direct-child\", \"pin name 'direct-child' not found in output: %s\", output)\n\t\trequire.Contains(t, output, \"direct\", \"pin type 'direct' not found in output: %s\", output)\n\n\t\t// Check recursive pin with specific CID\n\t\tres = node.IPFS(\"pin\", \"ls\", \"--type=recursive\", parentCid, \"--names\")\n\t\toutput = res.Stdout.String()\n\t\trequire.Contains(t, output, \"recursive-parent\", \"pin name 'recursive-parent' not found in output: %s\", output)\n\t\trequire.Contains(t, output, \"recursive\", \"pin type 'recursive' not found in output: %s\", output)\n\n\t\t// Child is both directly pinned and indirectly pinned through parent\n\t\t// Both relationships are valid and can be checked\n\t})\n\n\tt.Run(\"pin update preserves name\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Create two pieces of content\n\t\tcidOld := node.IPFSAddStr(\"old content\", \"--pin=false\")\n\t\tcidNew := node.IPFSAddStr(\"new content\", \"--pin=false\")\n\n\t\t// Pin with name\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=my-pin\", cidOld)\n\n\t\t// Update pin\n\t\tnode.IPFS(\"pin\", \"update\", cidOld, cidNew)\n\n\t\t// Check that new pin has the same name\n\t\tres := node.IPFS(\"pin\", \"ls\", cidNew, \"--names\")\n\t\trequire.Contains(t, res.Stdout.String(), \"my-pin\", \"pin name 'my-pin' not preserved after update, output: %s\", res.Stdout.String())\n\n\t\t// Old pin should not exist\n\t\tres = node.RunIPFS(\"pin\", \"ls\", cidOld)\n\t\trequire.Equal(t, 1, res.ExitCode(), \"expected exit code 1 for unpinned CID, got %d\", res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"is not pinned\", \"expected 'is not pinned' error for old CID %s, got: %s\", cidOld, res.Stderr.String())\n\t})\n\n\tt.Run(\"pin ls with invalid CID returns error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tres := node.RunIPFS(\"pin\", \"ls\", \"invalid-cid\")\n\t\trequire.Equal(t, 1, res.ExitCode(), \"expected exit code 1 for invalid CID, got %d\", res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"invalid\", \"expected 'invalid' in error message, got: %s\", res.Stderr.String())\n\t})\n\n\tt.Run(\"pin ls with unpinned CID returns error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add content without pinning\n\t\tcid := node.IPFSAddStr(\"unpinned content\", \"--pin=false\")\n\n\t\tres := node.RunIPFS(\"pin\", \"ls\", cid)\n\t\trequire.Equal(t, 1, res.ExitCode(), \"expected exit code 1 for unpinned CID, got %d\", res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"is not pinned\", \"expected 'is not pinned' error for CID %s, got: %s\", cid, res.Stderr.String())\n\t})\n\n\tt.Run(\"pin with special characters in name\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\ttestCases := []struct {\n\t\t\tname string\n\t\t\tpinName string\n\t\t}{\n\t\t\t{\"unicode\", \"test-📌-pin\"},\n\t\t\t{\"spaces\", \"test pin name\"},\n\t\t\t{\"special chars\", \"test!@#$%\"},\n\t\t\t{\"path-like\", \"test/pin/name\"},\n\t\t\t{\"dots\", \"test.pin.name\"},\n\t\t\t{\"long name\", strings.Repeat(\"a\", 255)},\n\t\t\t{\"empty name\", \"\"},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\tcid := node.IPFSAddStr(\"content for \"+tc.name, \"--pin=false\")\n\t\t\t\tnode.IPFS(\"pin\", \"add\", \"--name=\"+tc.pinName, cid)\n\n\t\t\t\tres := node.IPFS(\"pin\", \"ls\", cid, \"--names\")\n\t\t\t\tif tc.pinName != \"\" {\n\t\t\t\t\trequire.Contains(t, res.Stdout.String(), tc.pinName,\n\t\t\t\t\t\t\"pin name '%s' not found in output for test case '%s'\", tc.pinName, tc.name)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"concurrent pin operations with names\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Create multiple goroutines adding pins with names\n\t\tnumPins := 10\n\t\tdone := make(chan struct{}, numPins)\n\n\t\tfor i := range numPins {\n\t\t\tgo func(idx int) {\n\t\t\t\tdefer func() { done <- struct{}{} }()\n\n\t\t\t\tcontent := fmt.Sprintf(\"concurrent content %d\", idx)\n\t\t\t\tcid := node.IPFSAddStr(content, \"--pin=false\")\n\t\t\t\tpinName := fmt.Sprintf(\"concurrent-pin-%d\", idx)\n\t\t\t\tnode.IPFS(\"pin\", \"add\", \"--name=\"+pinName, cid)\n\t\t\t}(i)\n\t\t}\n\n\t\t// Wait for all goroutines\n\t\tfor range numPins {\n\t\t\t<-done\n\t\t}\n\n\t\t// Verify all pins have correct names\n\t\tres := node.IPFS(\"pin\", \"ls\", \"--names\")\n\t\toutput := res.Stdout.String()\n\t\tfor i := range numPins {\n\t\t\tpinName := fmt.Sprintf(\"concurrent-pin-%d\", i)\n\t\t\trequire.Contains(t, output, pinName,\n\t\t\t\t\"concurrent pin name '%s' not found in output\", pinName)\n\t\t}\n\t})\n\n\tt.Run(\"pin rm removes name association\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add and pin with name\n\t\tcid := node.IPFSAddStr(\"content to remove\", \"--pin=false\")\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=to-be-removed\", cid)\n\n\t\t// Verify pin exists with name\n\t\tres := node.IPFS(\"pin\", \"ls\", cid, \"--names\")\n\t\trequire.Contains(t, res.Stdout.String(), \"to-be-removed\")\n\n\t\t// Remove pin\n\t\tnode.IPFS(\"pin\", \"rm\", cid)\n\n\t\t// Verify pin and name are gone\n\t\tres = node.RunIPFS(\"pin\", \"ls\", cid)\n\t\trequire.Equal(t, 1, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"is not pinned\")\n\t})\n\n\tt.Run(\"garbage collection preserves named pins\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add content with and without pin names\n\t\tcidNamed := node.IPFSAddStr(\"named content\", \"--pin=false\")\n\t\tcidUnnamed := node.IPFSAddStr(\"unnamed content\", \"--pin=false\")\n\t\tcidUnpinned := node.IPFSAddStr(\"unpinned content\", \"--pin=false\")\n\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=important-data\", cidNamed)\n\t\tnode.IPFS(\"pin\", \"add\", cidUnnamed)\n\n\t\t// Run garbage collection\n\t\tnode.IPFS(\"repo\", \"gc\")\n\n\t\t// Named and unnamed pins should still exist\n\t\tres := node.IPFS(\"pin\", \"ls\", cidNamed, \"--names\")\n\t\trequire.Contains(t, res.Stdout.String(), \"important-data\")\n\n\t\tres = node.IPFS(\"pin\", \"ls\", cidUnnamed)\n\t\trequire.Contains(t, res.Stdout.String(), cidUnnamed)\n\n\t\t// Unpinned content should be gone (cat should fail)\n\t\tres = node.RunIPFS(\"cat\", cidUnpinned)\n\t\trequire.NotEqual(t, 0, res.ExitCode(), \"unpinned content should be garbage collected\")\n\t})\n\n\tt.Run(\"pin add with same name can be used for multiple pins\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add two different pieces of content\n\t\tcid1 := node.IPFSAddStr(\"first content\", \"--pin=false\")\n\t\tcid2 := node.IPFSAddStr(\"second content\", \"--pin=false\")\n\n\t\t// Pin both with the same name - this is allowed\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=shared-name\", cid1)\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=shared-name\", cid2)\n\n\t\t// List all pins with names\n\t\tres := node.IPFS(\"pin\", \"ls\", \"--names\")\n\t\toutput := res.Stdout.String()\n\n\t\t// Both CIDs should be pinned\n\t\trequire.Contains(t, output, cid1)\n\t\trequire.Contains(t, output, cid2)\n\n\t\t// Both pins can have the same name\n\t\tlines := strings.Split(output, \"\\n\")\n\t\tfoundCid1WithName := false\n\t\tfoundCid2WithName := false\n\t\tfor _, line := range lines {\n\t\t\tif strings.Contains(line, cid1) && strings.Contains(line, \"shared-name\") {\n\t\t\t\tfoundCid1WithName = true\n\t\t\t}\n\t\t\tif strings.Contains(line, cid2) && strings.Contains(line, \"shared-name\") {\n\t\t\t\tfoundCid2WithName = true\n\t\t\t}\n\t\t}\n\t\trequire.True(t, foundCid1WithName, \"first pin should have the name\")\n\t\trequire.True(t, foundCid2WithName, \"second pin should have the name\")\n\t})\n\n\tt.Run(\"pin names persist across daemon restarts\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.StartDaemon(\"--offline\")\n\n\t\t// Add content with pin name\n\t\tcid := node.IPFSAddStr(\"persistent content\")\n\t\tnode.IPFS(\"pin\", \"add\", \"--name=persistent-pin\", cid)\n\n\t\t// Restart daemon\n\t\tnode.StopDaemon()\n\t\tnode.StartDaemon(\"--offline\")\n\n\t\t// Check pin name persisted\n\t\tres := node.IPFS(\"pin\", \"ls\", cid, \"--names\")\n\t\trequire.Contains(t, res.Stdout.String(), \"persistent-pin\",\n\t\t\t\"pin name should persist across daemon restarts\")\n\n\t\tnode.StopDaemon()\n\t})\n}\n\n// TestPinLsEdgeCases tests edge cases for pin ls command\nfunc TestPinLsEdgeCases(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"invalid pin type returns error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Try to list pins with invalid type\n\t\tres := node.RunIPFS(\"pin\", \"ls\", \"--type=invalid\")\n\t\trequire.NotEqual(t, 0, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"invalid type 'invalid'\")\n\t\trequire.Contains(t, res.Stderr.String(), \"must be one of {direct, indirect, recursive, all}\")\n\t})\n\n\tt.Run(\"known but non-listable pin type returns error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// \"internal\" is a valid pin.Mode in boxo but not a valid --type for pin ls.\n\t\t// Before the fix, this caused a panic instead of returning an error.\n\t\tres := node.RunIPFS(\"pin\", \"ls\", \"--type=internal\")\n\t\trequire.NotEqual(t, 0, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"invalid type 'internal'\")\n\t})\n\n\tt.Run(\"non-existent path returns proper error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Try to list a non-existent CID\n\t\tfakeCID := \"QmNonExistent123456789\"\n\t\tres := node.RunIPFS(\"pin\", \"ls\", fakeCID)\n\t\trequire.NotEqual(t, 0, res.ExitCode())\n\t})\n\n\tt.Run(\"unpinned CID returns not pinned error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := setupTestNode(t)\n\n\t\t// Add content but don't pin it explicitly (it's just in blockstore)\n\t\tunpinnedCID := node.IPFSAddStr(\"unpinned content\", \"--pin=false\")\n\n\t\t// Try to list specific unpinned CID\n\t\tres := node.RunIPFS(\"pin\", \"ls\", unpinnedCID)\n\t\trequire.NotEqual(t, 0, res.ExitCode())\n\t\trequire.Contains(t, res.Stderr.String(), \"is not pinned\")\n\t})\n}\n" }, { "path": "test/cli/pin_name_validation_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPinNameValidation(t *testing.T) {\n\tt.Parallel()\n\n\t// Create a test node and add a test file\n\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\tdefer node.StopDaemon()\n\n\t// Add a test file to get a CID\n\ttestContent := \"test content for pin name validation\"\n\ttestCID := node.IPFSAddStr(testContent, \"--pin=false\")\n\n\tt.Run(\"pin add accepts valid names\", func(t *testing.T) {\n\t\ttestCases := []struct {\n\t\t\tname string\n\t\t\tpinName string\n\t\t\tdescription string\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"empty_name\",\n\t\t\t\tpinName: \"\",\n\t\t\t\tdescription: \"Empty name should be allowed\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"short_name\",\n\t\t\t\tpinName: \"test\",\n\t\t\t\tdescription: \"Short ASCII name should be allowed\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"max_255_bytes\",\n\t\t\t\tpinName: strings.Repeat(\"a\", 255),\n\t\t\t\tdescription: \"Exactly 255 bytes should be allowed\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"unicode_within_limit\",\n\t\t\t\tpinName: \"测试名称🔥\", // Chinese characters and emoji\n\t\t\t\tdescription: \"Unicode characters within 255 bytes should be allowed\",\n\t\t\t},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\tvar args []string\n\t\t\t\tif tc.pinName != \"\" {\n\t\t\t\t\targs = []string{\"pin\", \"add\", \"--name\", tc.pinName, testCID}\n\t\t\t\t} else {\n\t\t\t\t\targs = []string{\"pin\", \"add\", testCID}\n\t\t\t\t}\n\n\t\t\t\tres := node.RunIPFS(args...)\n\t\t\t\trequire.Equal(t, 0, res.ExitCode(), tc.description)\n\n\t\t\t\t// Clean up - unpin\n\t\t\t\tnode.RunIPFS(\"pin\", \"rm\", testCID)\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"pin add rejects names exceeding 255 bytes\", func(t *testing.T) {\n\t\ttestCases := []struct {\n\t\t\tname string\n\t\t\tpinName string\n\t\t\tdescription string\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"256_bytes\",\n\t\t\t\tpinName: strings.Repeat(\"a\", 256),\n\t\t\t\tdescription: \"256 bytes should be rejected\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"300_bytes\",\n\t\t\t\tpinName: strings.Repeat(\"b\", 300),\n\t\t\t\tdescription: \"300 bytes should be rejected\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"unicode_exceeding_limit\",\n\t\t\t\tpinName: strings.Repeat(\"测\", 100), // Each Chinese character is 3 bytes, total 300 bytes\n\t\t\t\tdescription: \"Unicode string exceeding 255 bytes should be rejected\",\n\t\t\t},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"pin\", \"add\", \"--name\", tc.pinName, testCID)\n\t\t\t\trequire.NotEqual(t, 0, res.ExitCode(), tc.description)\n\t\t\t\trequire.Contains(t, res.Stderr.String(), \"max 255 bytes\", \"Error should mention the 255 byte limit\")\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"pin ls with name filter validates length\", func(t *testing.T) {\n\t\t// Test valid filter\n\t\tres := node.RunIPFS(\"pin\", \"ls\", \"--name\", strings.Repeat(\"a\", 255))\n\t\trequire.Equal(t, 0, res.ExitCode(), \"255-byte name filter should be accepted\")\n\n\t\t// Test invalid filter\n\t\tres = node.RunIPFS(\"pin\", \"ls\", \"--name\", strings.Repeat(\"a\", 256))\n\t\trequire.NotEqual(t, 0, res.ExitCode(), \"256-byte name filter should be rejected\")\n\t\trequire.Contains(t, res.Stderr.String(), \"max 255 bytes\", \"Error should mention the 255 byte limit\")\n\t})\n}\n\nfunc TestAddPinNameValidation(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init().StartDaemon(\"--offline\")\n\tdefer node.StopDaemon()\n\n\t// Create a test file\n\ttestFile := \"test.txt\"\n\tnode.WriteBytes(testFile, []byte(\"test content for add command\"))\n\n\tt.Run(\"ipfs add with --pin-name accepts valid names\", func(t *testing.T) {\n\t\ttestCases := []struct {\n\t\t\tname string\n\t\t\tpinName string\n\t\t\tdescription string\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"short_name\",\n\t\t\t\tpinName: \"test-add\",\n\t\t\t\tdescription: \"Short ASCII name should be allowed\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"max_255_bytes\",\n\t\t\t\tpinName: strings.Repeat(\"x\", 255),\n\t\t\t\tdescription: \"Exactly 255 bytes should be allowed\",\n\t\t\t},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"add\", fmt.Sprintf(\"--pin-name=%s\", tc.pinName), \"-q\", testFile)\n\t\t\t\trequire.Equal(t, 0, res.ExitCode(), tc.description)\n\t\t\t\tcid := strings.TrimSpace(res.Stdout.String())\n\n\t\t\t\t// Verify pin exists with name\n\t\t\t\tlsRes := node.RunIPFS(\"pin\", \"ls\", \"--names\", \"--type=recursive\", cid)\n\t\t\t\trequire.Equal(t, 0, lsRes.ExitCode())\n\t\t\t\trequire.Contains(t, lsRes.Stdout.String(), tc.pinName, \"Pin should have the specified name\")\n\n\t\t\t\t// Clean up\n\t\t\t\tnode.RunIPFS(\"pin\", \"rm\", cid)\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"ipfs add with --pin-name rejects names exceeding 255 bytes\", func(t *testing.T) {\n\t\ttestCases := []struct {\n\t\t\tname string\n\t\t\tpinName string\n\t\t\tdescription string\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"256_bytes\",\n\t\t\t\tpinName: strings.Repeat(\"y\", 256),\n\t\t\t\tdescription: \"256 bytes should be rejected\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"500_bytes\",\n\t\t\t\tpinName: strings.Repeat(\"z\", 500),\n\t\t\t\tdescription: \"500 bytes should be rejected\",\n\t\t\t},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"add\", fmt.Sprintf(\"--pin-name=%s\", tc.pinName), testFile)\n\t\t\t\trequire.NotEqual(t, 0, res.ExitCode(), tc.description)\n\t\t\t\trequire.Contains(t, res.Stderr.String(), \"max 255 bytes\", \"Error should mention the 255 byte limit\")\n\t\t\t})\n\t\t}\n\t})\n}\n" }, { "path": "test/cli/ping_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestPing(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"other\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init().StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tnode1.IPFS(\"ping\", \"-n\", \"2\", \"--\", node2.PeerID().String())\n\t\tnode2.IPFS(\"ping\", \"-n\", \"2\", \"--\", node1.PeerID().String())\n\t})\n\n\tt.Run(\"ping unreachable peer\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init().StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\t\tnode1 := nodes[0]\n\n\t\tbadPeer := \"QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJx\"\n\t\tres := node1.RunIPFS(\"ping\", \"-n\", \"2\", \"--\", badPeer)\n\t\tassert.Contains(t, res.Stdout.String(), fmt.Sprintf(\"Looking up peer %s\", badPeer))\n\t\tmsg := res.Stderr.String()\n\t\tassert.Truef(t, strings.HasPrefix(msg, \"Error:\"), \"should fail got this instead: %q\", msg)\n\t})\n\n\tt.Run(\"self\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init().StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tres := node1.RunIPFS(\"ping\", \"-n\", \"2\", \"--\", node1.PeerID().String())\n\t\tassert.Equal(t, 1, res.Cmd.ProcessState.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"can't ping self\")\n\n\t\tres = node2.RunIPFS(\"ping\", \"-n\", \"2\", \"--\", node2.PeerID().String())\n\t\tassert.Equal(t, 1, res.Cmd.ProcessState.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"can't ping self\")\n\t})\n\n\tt.Run(\"0\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init().StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tres := node1.RunIPFS(\"ping\", \"-n\", \"0\", \"--\", node2.PeerID().String())\n\t\tassert.Equal(t, 1, res.Cmd.ProcessState.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"ping count must be greater than 0\")\n\t})\n\n\tt.Run(\"offline\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init().StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tnode2.StopDaemon()\n\n\t\tres := node1.RunIPFS(\"ping\", \"-n\", \"2\", \"--\", node2.PeerID().String())\n\t\tassert.Equal(t, 1, res.Cmd.ProcessState.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"ping failed\")\n\t})\n}\n" }, { "path": "test/cli/pinning_remote_test.go", "content": "package cli\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils/pinningservice\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/sjson\"\n)\n\nfunc runPinningService(t *testing.T, authToken string) (*pinningservice.PinningService, string) {\n\tsvc := pinningservice.New()\n\trouter := pinningservice.NewRouter(authToken, svc)\n\tserver := &http.Server{Handler: router}\n\tlistener, err := net.Listen(\"tcp\", \"127.0.0.1:0\")\n\trequire.NoError(t, err)\n\tgo func() {\n\t\terr := server.Serve(listener)\n\t\tif err != nil && !errors.Is(err, net.ErrClosed) && !errors.Is(err, http.ErrServerClosed) {\n\t\t\tt.Logf(\"Serve error: %s\", err)\n\t\t}\n\t}()\n\tt.Cleanup(func() { listener.Close() })\n\n\treturn svc, fmt.Sprintf(\"http://%s/api/v1\", listener.Addr().String())\n}\n\nfunc TestRemotePinning(t *testing.T) {\n\tt.Parallel()\n\tauthToken := \"testauthtoken\"\n\n\tt.Run(\"MFS pinning\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.Runner.Env[\"MFS_PIN_POLL_INTERVAL\"] = \"10ms\"\n\n\t\t_, svcURL := runPinningService(t, authToken)\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\t\tnode.IPFS(\"config\", \"--json\", \"Pinning.RemoteServices.svc.Policies.MFS.RepinInterval\", `\"1s\"`)\n\t\tnode.IPFS(\"config\", \"--json\", \"Pinning.RemoteServices.svc.Policies.MFS.PinName\", `\"test_pin\"`)\n\t\tnode.IPFS(\"config\", \"--json\", \"Pinning.RemoteServices.svc.Policies.MFS.Enable\", \"true\")\n\n\t\tnode.StartDaemon()\n\t\tt.Cleanup(func() { node.StopDaemon() })\n\n\t\tnode.IPFS(\"files\", \"cp\", \"/ipfs/bafkqaaa\", \"/mfs-pinning-test-\"+uuid.NewString())\n\t\tnode.IPFS(\"files\", \"flush\")\n\t\tres := node.IPFS(\"files\", \"stat\", \"/\", \"--enc=json\")\n\t\thash := gjson.Get(res.Stdout.String(), \"Hash\").Str\n\n\t\tassert.Eventually(t,\n\t\t\tfunc() bool {\n\t\t\t\tres = node.IPFS(\"pin\", \"remote\", \"ls\",\n\t\t\t\t\t\"--service=svc\",\n\t\t\t\t\t\"--name=test_pin\",\n\t\t\t\t\t\"--status=queued,pinning,pinned,failed\",\n\t\t\t\t\t\"--enc=json\",\n\t\t\t\t)\n\t\t\t\tpinnedHash := gjson.Get(res.Stdout.String(), \"Cid\").Str\n\t\t\t\treturn hash == pinnedHash\n\t\t\t},\n\t\t\t10*time.Second,\n\t\t\t10*time.Millisecond,\n\t\t)\n\n\t\tt.Run(\"MFS root is repinned on CID change\", func(t *testing.T) {\n\t\t\tnode.IPFS(\"files\", \"cp\", \"/ipfs/bafkqaaa\", \"/mfs-pinning-repin-test-\"+uuid.NewString())\n\t\t\tnode.IPFS(\"files\", \"flush\")\n\t\t\tres = node.IPFS(\"files\", \"stat\", \"/\", \"--enc=json\")\n\t\t\thash := gjson.Get(res.Stdout.String(), \"Hash\").Str\n\t\t\tassert.Eventually(t,\n\t\t\t\tfunc() bool {\n\t\t\t\t\tres := node.IPFS(\"pin\", \"remote\", \"ls\",\n\t\t\t\t\t\t\"--service=svc\",\n\t\t\t\t\t\t\"--name=test_pin\",\n\t\t\t\t\t\t\"--status=queued,pinning,pinned,failed\",\n\t\t\t\t\t\t\"--enc=json\",\n\t\t\t\t\t)\n\t\t\t\t\tpinnedHash := gjson.Get(res.Stdout.String(), \"Cid\").Str\n\t\t\t\t\treturn hash == pinnedHash\n\t\t\t\t},\n\t\t\t\t10*time.Second,\n\t\t\t\t10*time.Millisecond,\n\t\t\t)\n\t\t})\n\t})\n\n\t// Pinning.RemoteServices includes API.Key, so we give it the same treatment\n\t// as Identity,PrivKey to prevent exposing it on the network\n\tt.Run(\"access token security\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"1\", \"http://example1.com\", \"testkey\")\n\t\tres := node.RunIPFS(\"config\", \"Pinning\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"cannot show or change pinning services credentials\")\n\t\tassert.NotContains(t, res.Stdout.String(), \"testkey\")\n\n\t\tres = node.RunIPFS(\"config\", \"Pinning.RemoteServices.1.API.Key\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"cannot show or change pinning services credentials\")\n\t\tassert.NotContains(t, res.Stdout.String(), \"testkey\")\n\n\t\tconfigShow := node.RunIPFS(\"config\", \"show\").Stdout.String()\n\t\tassert.NotContains(t, configShow, \"testkey\")\n\n\t\tt.Run(\"re-injecting config with 'ipfs config replace' preserves the API keys\", func(t *testing.T) {\n\t\t\tnode.WriteBytes(\"config-show\", []byte(configShow))\n\t\t\tnode.IPFS(\"config\", \"replace\", \"config-show\")\n\t\t\tassert.Contains(t, node.ReadFile(node.ConfigFile()), \"testkey\")\n\t\t})\n\n\t\tt.Run(\"injecting config with 'ipfs config replace' with API keys returns an error\", func(t *testing.T) {\n\t\t\t// remove Identity.PrivKey to ensure error is triggered by Pinning.RemoteServices\n\t\t\tconfigJSON := MustVal(sjson.Delete(configShow, \"Identity.PrivKey\"))\n\t\t\tconfigJSON = MustVal(sjson.Set(configJSON, \"Pinning.RemoteServices.1.API.Key\", \"testkey\"))\n\t\t\tnode.WriteBytes(\"new-config\", []byte(configJSON))\n\t\t\tres := node.RunIPFS(\"config\", \"replace\", \"new-config\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"cannot change remote pinning services api info with `config replace`\")\n\t\t})\n\t})\n\n\tt.Run(\"pin remote service ls --stat\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t_, svcURL := runPinningService(t, authToken)\n\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"invalid-svc\", svcURL+\"/invalidpath\", authToken)\n\n\t\tres := node.IPFS(\"pin\", \"remote\", \"service\", \"ls\", \"--stat\")\n\t\tassert.Contains(t, res.Stdout.String(), \" 0/0/0/0\")\n\n\t\tstats := node.IPFS(\"pin\", \"remote\", \"service\", \"ls\", \"--stat\", \"--enc=json\").Stdout.String()\n\t\tassert.Equal(t, \"valid\", gjson.Get(stats, `RemoteServices.#(Service == \"svc\").Stat.Status`).Str)\n\t\tassert.Equal(t, \"invalid\", gjson.Get(stats, `RemoteServices.#(Service == \"invalid-svc\").Stat.Status`).Str)\n\n\t\t// no --stat returns no stat obj\n\t\tt.Run(\"no --stat returns no stat obj\", func(t *testing.T) {\n\t\t\tres := node.IPFS(\"pin\", \"remote\", \"service\", \"ls\", \"--enc=json\")\n\t\t\tassert.False(t, gjson.Get(res.Stdout.String(), `RemoteServices.#(Service == \"svc\").Stat`).Exists())\n\t\t})\n\t})\n\n\tt.Run(\"adding service with invalid URL fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", \"invalid-service.example.com\", \"key\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"service endpoint must be a valid HTTP URL\")\n\n\t\tres = node.RunIPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", \"xyz://invalid-service.example.com\", \"key\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"service endpoint must be a valid HTTP URL\")\n\t})\n\n\tt.Run(\"unauthorized pinning service calls fail\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\t_, svcURL := runPinningService(t, authToken)\n\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, \"othertoken\")\n\n\t\tres := node.RunIPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"access denied\")\n\t})\n\n\tt.Run(\"pinning service calls fail when there is a wrong path\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\t_, svcURL := runPinningService(t, authToken)\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL+\"/invalid-path\", authToken)\n\n\t\tres := node.RunIPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"404\")\n\t})\n\n\tt.Run(\"pinning service calls fail when DNS resolution fails\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", \"https://invalid-service.example.com\", authToken)\n\n\t\tres := node.RunIPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"no such host\")\n\t})\n\n\tt.Run(\"pin remote service rm\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", \"https://example.com\", authToken)\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"rm\", \"svc\")\n\t\tres := node.IPFS(\"pin\", \"remote\", \"service\", \"ls\")\n\t\tassert.NotContains(t, res.Stdout.String(), \"svc\")\n\t})\n\n\tt.Run(\"remote pinning\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tverifyStatus := func(node *harness.Node, name, hash, status string) {\n\t\t\tresJSON := node.IPFS(\"pin\", \"remote\", \"ls\",\n\t\t\t\t\"--service=svc\",\n\t\t\t\t\"--enc=json\",\n\t\t\t\t\"--name=\"+name,\n\t\t\t\t\"--status=\"+status,\n\t\t\t).Stdout.String()\n\n\t\t\tassert.Equal(t, status, gjson.Get(resJSON, \"Status\").Str)\n\t\t\tassert.Equal(t, hash, gjson.Get(resJSON, \"Cid\").Str)\n\t\t\tassert.Equal(t, name, gjson.Get(resJSON, \"Name\").Str)\n\t\t}\n\n\t\tt.Run(\"'ipfs pin remote add --background=true'\", func(t *testing.T) {\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\t// retain a ptr to the pin that's in the DB so we can directly mutate its status\n\t\t\t// to simulate async work\n\t\t\tpinCh := make(chan *pinningservice.PinStatus, 1)\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpinCh <- pin\n\t\t\t}\n\n\t\t\thash := node.IPFSAddStr(\"foo\")\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\",\n\t\t\t\t\"--background=true\",\n\t\t\t\t\"--service=svc\",\n\t\t\t\t\"--name=pin1\",\n\t\t\t\thash,\n\t\t\t)\n\n\t\t\tpin := <-pinCh\n\n\t\t\ttransitionStatus := func(status string) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tpin.Status = status\n\t\t\t\tpin.M.Unlock()\n\t\t\t}\n\n\t\t\tverifyStatus(node, \"pin1\", hash, \"queued\")\n\n\t\t\ttransitionStatus(\"pinning\")\n\t\t\tverifyStatus(node, \"pin1\", hash, \"pinning\")\n\n\t\t\ttransitionStatus(\"pinned\")\n\t\t\tverifyStatus(node, \"pin1\", hash, \"pinned\")\n\n\t\t\ttransitionStatus(\"failed\")\n\t\t\tverifyStatus(node, \"pin1\", hash, \"failed\")\n\t\t})\n\n\t\tt.Run(\"'ipfs pin remote add --background=false'\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tdefer pin.M.Unlock()\n\t\t\t\tpin.Status = \"pinned\"\n\t\t\t}\n\t\t\thash := node.IPFSAddStr(\"foo\")\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\",\n\t\t\t\t\"--background=false\",\n\t\t\t\t\"--service=svc\",\n\t\t\t\t\"--name=pin2\",\n\t\t\t\thash,\n\t\t\t)\n\t\t\tverifyStatus(node, \"pin2\", hash, \"pinned\")\n\t\t})\n\n\t\tt.Run(\"'ipfs pin remote ls' with multiple statuses\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\thash := node.IPFSAddStr(\"foo\")\n\t\t\tdesiredStatuses := map[string]string{\n\t\t\t\t\"pin-queued\": \"queued\",\n\t\t\t\t\"pin-pinning\": \"pinning\",\n\t\t\t\t\"pin-pinned\": \"pinned\",\n\t\t\t\t\"pin-failed\": \"failed\",\n\t\t\t}\n\t\t\tvar pins []*pinningservice.PinStatus\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tdefer pin.M.Unlock()\n\t\t\t\tpins = append(pins, pin)\n\t\t\t\t// this must be \"pinned\" for the 'pin remote add' command to return\n\t\t\t\t// after 'pin remote add', we change the status to its real status\n\t\t\t\tpin.Status = \"pinned\"\n\t\t\t}\n\n\t\t\tfor pinName := range desiredStatuses {\n\t\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\",\n\t\t\t\t\t\"--service=svc\",\n\t\t\t\t\t\"--name=\"+pinName,\n\t\t\t\t\thash,\n\t\t\t\t)\n\t\t\t}\n\t\t\tfor _, pin := range pins {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tpin.Status = desiredStatuses[pin.Pin.Name]\n\t\t\t\tpin.M.Unlock()\n\t\t\t}\n\n\t\t\tres := node.IPFS(\"pin\", \"remote\", \"ls\",\n\t\t\t\t\"--service=svc\",\n\t\t\t\t\"--status=queued,pinning,pinned,failed\",\n\t\t\t\t\"--enc=json\",\n\t\t\t)\n\t\t\tactualStatuses := map[string]string{}\n\t\t\tfor _, line := range res.Stdout.Lines() {\n\t\t\t\tname := gjson.Get(line, \"Name\").Str\n\t\t\t\tstatus := gjson.Get(line, \"Status\").Str\n\t\t\t\t// drop statuses of other pins we didn't add\n\t\t\t\tif _, ok := desiredStatuses[name]; ok {\n\t\t\t\t\tactualStatuses[name] = status\n\t\t\t\t}\n\t\t\t}\n\t\t\tassert.Equal(t, desiredStatuses, actualStatuses)\n\t\t})\n\n\t\tt.Run(\"'ipfs pin remote ls' by CID\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\ttransitionedCh := make(chan struct{}, 1)\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tdefer pin.M.Unlock()\n\t\t\t\tpin.Status = \"pinned\"\n\t\t\t\ttransitionedCh <- struct{}{}\n\t\t\t}\n\t\t\thash := node.IPFSAddStr(string(random.Bytes(1000)))\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\", \"--background=false\", \"--service=svc\", hash)\n\t\t\t<-transitionedCh\n\t\t\tres := node.IPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\", \"--cid=\"+hash, \"--enc=json\").Stdout.String()\n\t\t\tassert.Contains(t, res, hash)\n\t\t})\n\n\t\tt.Run(\"'ipfs pin remote rm --name' without --force when multiple pins match\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tdefer pin.M.Unlock()\n\t\t\t\tpin.Status = \"pinned\"\n\t\t\t}\n\t\t\thash := node.IPFSAddStr(string(random.Bytes(1000)))\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\", \"--service=svc\", \"--name=force-test-name\", hash)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\", \"--service=svc\", \"--name=force-test-name\", hash)\n\n\t\t\tt.Run(\"fails\", func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"pin\", \"remote\", \"rm\", \"--service=svc\", \"--name=force-test-name\")\n\t\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t\tassert.Contains(t, res.Stderr.String(), \"Error: multiple remote pins are matching this query, add --force to confirm the bulk removal\")\n\t\t\t})\n\n\t\t\tt.Run(\"matching pins are not removed\", func(t *testing.T) {\n\t\t\t\tlines := node.IPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\", \"--name=force-test-name\").Stdout.Lines()\n\t\t\t\tassert.Contains(t, lines[0], \"force-test-name\")\n\t\t\t\tassert.Contains(t, lines[1], \"force-test-name\")\n\t\t\t})\n\t\t})\n\n\t\tt.Run(\"'ipfs pin remote rm --name --force' remove multiple pins\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tdefer pin.M.Unlock()\n\t\t\t\tpin.Status = \"pinned\"\n\t\t\t}\n\t\t\thash := node.IPFSAddStr(string(random.Bytes(1000)))\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\", \"--service=svc\", \"--name=force-test-name\", hash)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\", \"--service=svc\", \"--name=force-test-name\", hash)\n\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"rm\", \"--service=svc\", \"--name=force-test-name\", \"--force\")\n\t\t\tout := node.IPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\", \"--name=force-test-name\").Stdout.Trimmed()\n\t\t\tassert.Empty(t, out)\n\t\t})\n\n\t\tt.Run(\"'ipfs pin remote rm --force' removes all pins\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\t\tdefer node.StopDaemon()\n\t\t\tsvc, svcURL := runPinningService(t, authToken)\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\t\tsvc.PinAdded = func(req *pinningservice.AddPinRequest, pin *pinningservice.PinStatus) {\n\t\t\t\tpin.M.Lock()\n\t\t\t\tdefer pin.M.Unlock()\n\t\t\t\tpin.Status = \"pinned\"\n\t\t\t}\n\t\t\tfor i := range 4 {\n\t\t\t\thash := node.IPFSAddStr(string(random.Bytes(1000)))\n\t\t\t\tname := fmt.Sprintf(\"--name=%d\", i)\n\t\t\t\tnode.IPFS(\"pin\", \"remote\", \"add\", \"--service=svc\", \"--name=\"+name, hash)\n\t\t\t}\n\n\t\t\tlines := node.IPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\").Stdout.Lines()\n\t\t\tassert.Len(t, lines, 4)\n\n\t\t\tnode.IPFS(\"pin\", \"remote\", \"rm\", \"--service=svc\", \"--force\")\n\n\t\t\tlines = node.IPFS(\"pin\", \"remote\", \"ls\", \"--service=svc\").Stdout.Lines()\n\t\t\tassert.Len(t, lines, 0)\n\t\t})\n\t})\n\n\tt.Run(\"'ipfs pin remote add' shows a warning message when offline\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t_, svcURL := runPinningService(t, authToken)\n\t\tnode.IPFS(\"pin\", \"remote\", \"service\", \"add\", \"svc\", svcURL, authToken)\n\n\t\thash := node.IPFSAddStr(string(random.Bytes(1000)))\n\t\tres := node.IPFS(\"pin\", \"remote\", \"add\", \"--service=svc\", \"--background\", hash)\n\t\twarningMsg := \"WARNING: the local node is offline and remote pinning may fail if there is no other provider for this CID\"\n\t\tassert.Contains(t, res.Stdout.String(), warningMsg)\n\t})\n}\n" }, { "path": "test/cli/pins_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t. \"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype testPinsArgs struct {\n\trunDaemon bool\n\tpinArg string\n\tlsArg string\n\tbaseArg string\n}\n\nfunc testPins(t *testing.T, args testPinsArgs) {\n\tt.Run(fmt.Sprintf(\"test pins with args=%+v\", args), func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tif args.runDaemon {\n\t\t\tnode.StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\t\t}\n\n\t\tstrs := []string{\"a\", \"b\", \"c\", \"d\", \"e\", \"f\", \"g\"}\n\t\tdataToCid := map[string]string{}\n\t\tcids := []string{}\n\n\t\tipfsAdd := func(t *testing.T, content string) string {\n\t\t\tcidStr := node.IPFSAddStr(content, StrCat(args.baseArg, \"--pin=false\")...)\n\n\t\t\t_, err := cid.Decode(cidStr)\n\t\t\trequire.NoError(t, err)\n\t\t\tdataToCid[content] = cidStr\n\t\t\tcids = append(cids, cidStr)\n\t\t\treturn cidStr\n\t\t}\n\n\t\tipfsPinAdd := func(cids []string) []string {\n\t\t\tinput := strings.Join(cids, \"\\n\")\n\t\t\treturn node.PipeStrToIPFS(input, StrCat(\"pin\", \"add\", args.pinArg, args.baseArg)...).Stdout.Lines()\n\t\t}\n\n\t\tipfsPinLS := func() string {\n\t\t\treturn node.IPFS(StrCat(\"pin\", \"ls\", args.lsArg, args.baseArg)...).Stdout.Trimmed()\n\t\t}\n\n\t\tfor _, s := range strs {\n\t\t\tipfsAdd(t, s)\n\t\t}\n\n\t\t// these subtests run sequentially since they depend on state\n\n\t\tt.Run(\"check output of pin command\", func(t *testing.T) {\n\t\t\tresLines := ipfsPinAdd(cids)\n\n\t\t\tfor i, s := range resLines {\n\t\t\t\tassert.Equal(t,\n\t\t\t\t\tfmt.Sprintf(\"pinned %s recursively\", cids[i]),\n\t\t\t\t\ts,\n\t\t\t\t)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"pin verify should succeed\", func(t *testing.T) {\n\t\t\tnode.IPFS(\"pin\", \"verify\")\n\t\t})\n\n\t\tt.Run(\"'pin verify --verbose' should include all the cids\", func(t *testing.T) {\n\t\t\tverboseVerifyOut := node.IPFS(StrCat(\"pin\", \"verify\", \"--verbose\", args.baseArg)...).Stdout.String()\n\t\t\tfor _, cid := range cids {\n\t\t\t\tassert.Contains(t, verboseVerifyOut, fmt.Sprintf(\"%s ok\", cid))\n\t\t\t}\n\t\t})\n\t\tt.Run(\"ls output should contain the cids\", func(t *testing.T) {\n\t\t\tlsOut := ipfsPinLS()\n\t\t\tfor _, cid := range cids {\n\t\t\t\tassert.Contains(t, lsOut, cid)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"check 'pin ls hash' output\", func(t *testing.T) {\n\t\t\tlsHashOut := node.IPFS(StrCat(\"pin\", \"ls\", args.lsArg, args.baseArg, dataToCid[\"b\"])...)\n\t\t\tlsHashOutStr := lsHashOut.Stdout.String()\n\t\t\tassert.Equal(t, fmt.Sprintf(\"%s recursive\\n\", dataToCid[\"b\"]), lsHashOutStr)\n\t\t})\n\n\t\tt.Run(\"unpinning works\", func(t *testing.T) {\n\t\t\tnode.PipeStrToIPFS(strings.Join(cids, \"\\n\"), \"pin\", \"rm\")\n\t\t})\n\n\t\tt.Run(\"test pin update\", func(t *testing.T) {\n\t\t\tcidA := dataToCid[\"a\"]\n\t\t\tcidB := dataToCid[\"b\"]\n\n\t\t\tipfsPinAdd([]string{cidA})\n\t\t\tbeforeUpdate := ipfsPinLS()\n\n\t\t\tassert.Contains(t, beforeUpdate, cidA)\n\t\t\tassert.NotContains(t, beforeUpdate, cidB)\n\n\t\t\tnode.IPFS(\"pin\", \"update\", \"--unpin=true\", cidA, cidB)\n\t\t\tafterUpdate := ipfsPinLS()\n\n\t\t\tassert.NotContains(t, afterUpdate, cidA)\n\t\t\tassert.Contains(t, afterUpdate, cidB)\n\n\t\t\tnode.IPFS(\"pin\", \"update\", \"--unpin=true\", cidB, cidB)\n\t\t\tafterIdempotentUpdate := ipfsPinLS()\n\n\t\t\tassert.Contains(t, afterIdempotentUpdate, cidB)\n\n\t\t\tnode.IPFS(\"pin\", \"rm\", cidB)\n\t\t})\n\t})\n}\n\nfunc testPinsErrorReporting(t *testing.T, args testPinsArgs) {\n\tt.Run(fmt.Sprintf(\"test pins error reporting with args=%+v\", args), func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tif args.runDaemon {\n\t\t\tnode.StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\t\t}\n\t\trandomCID := \"Qme8uX5n9hn15pw9p6WcVKoziyyC9LXv4LEgvsmKMULjnV\"\n\t\tres := node.RunIPFS(StrCat(\"pin\", \"add\", args.pinArg, randomCID)...)\n\t\tassert.NotEqual(t, 0, res.ExitErr.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"ipld: could not find\")\n\t})\n}\n\nfunc testPinDAG(t *testing.T, args testPinsArgs) {\n\tt.Run(fmt.Sprintf(\"test pin DAG with args=%+v\", args), func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tif args.runDaemon {\n\t\t\tnode.StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\t\t}\n\t\tbytes := random.Bytes(1 << 20) // 1 MiB\n\t\ttmpFile := h.WriteToTemp(string(bytes))\n\t\tcid := node.IPFS(StrCat(\"add\", args.pinArg, \"--pin=false\", \"-q\", tmpFile)...).Stdout.Trimmed()\n\n\t\tnode.IPFS(\"pin\", \"add\", \"--recursive=true\", cid)\n\t\tnode.IPFS(\"pin\", \"rm\", cid)\n\n\t\t// remove part of the DAG\n\t\tpart := node.IPFS(\"refs\", cid).Stdout.Lines()[0]\n\t\tnode.IPFS(\"block\", \"rm\", part)\n\n\t\tres := node.RunIPFS(\"pin\", \"add\", \"--recursive=true\", cid)\n\t\tassert.NotEqual(t, 0, res)\n\t\tassert.Contains(t, res.Stderr.String(), \"ipld: could not find\")\n\t})\n}\n\nfunc testPinProgress(t *testing.T, args testPinsArgs) {\n\tt.Run(fmt.Sprintf(\"test pin progress with args=%+v\", args), func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\n\t\tif args.runDaemon {\n\t\t\tnode.StartDaemon(\"--offline\")\n\t\t\tdefer node.StopDaemon()\n\t\t}\n\n\t\tbytes := random.Bytes(1 << 20) // 1 MiB\n\t\ttmpFile := h.WriteToTemp(string(bytes))\n\t\tcid := node.IPFS(StrCat(\"add\", args.pinArg, \"--pin=false\", \"-q\", tmpFile)...).Stdout.Trimmed()\n\n\t\tres := node.RunIPFS(\"pin\", \"add\", \"--progress\", cid)\n\t\tnode.Runner.AssertNoError(res)\n\n\t\tassert.Contains(t, res.Stderr.String(), \" 5 nodes (1.0 MB)\")\n\t})\n}\n\nfunc TestPins(t *testing.T) {\n\tt.Parallel()\n\tt.Run(\"test pinning without daemon running\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestPinsErrorReporting(t, testPinsArgs{})\n\t\ttestPinsErrorReporting(t, testPinsArgs{pinArg: \"--progress\"})\n\t\ttestPinDAG(t, testPinsArgs{})\n\t\ttestPinDAG(t, testPinsArgs{pinArg: \"--raw-leaves\"})\n\t\ttestPinProgress(t, testPinsArgs{})\n\t\ttestPins(t, testPinsArgs{})\n\t\ttestPins(t, testPinsArgs{pinArg: \"--progress\"})\n\t\ttestPins(t, testPinsArgs{pinArg: \"--progress\", lsArg: \"--stream\"})\n\t\ttestPins(t, testPinsArgs{baseArg: \"--cid-base=base32\"})\n\t\ttestPins(t, testPinsArgs{lsArg: \"--stream\", baseArg: \"--cid-base=base32\"})\n\t})\n\n\tt.Run(\"test pinning with daemon running without network\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\ttestPinsErrorReporting(t, testPinsArgs{runDaemon: true})\n\t\ttestPinsErrorReporting(t, testPinsArgs{runDaemon: true, pinArg: \"--progress\"})\n\t\ttestPinDAG(t, testPinsArgs{runDaemon: true})\n\t\ttestPinDAG(t, testPinsArgs{runDaemon: true, pinArg: \"--raw-leaves\"})\n\t\ttestPinProgress(t, testPinsArgs{runDaemon: true})\n\t\ttestPins(t, testPinsArgs{runDaemon: true})\n\t\ttestPins(t, testPinsArgs{runDaemon: true, pinArg: \"--progress\"})\n\t\ttestPins(t, testPinsArgs{runDaemon: true, pinArg: \"--progress\", lsArg: \"--stream\"})\n\t\ttestPins(t, testPinsArgs{runDaemon: true, baseArg: \"--cid-base=base32\"})\n\t\ttestPins(t, testPinsArgs{runDaemon: true, lsArg: \"--stream\", baseArg: \"--cid-base=base32\"})\n\t})\n\n\tpinLs := func(node *harness.Node, args ...string) []string {\n\t\treturn strings.Split(node.IPFS(StrCat(\"pin\", \"ls\", args)...).Stdout.Trimmed(), \"\\n\")\n\t}\n\n\tt.Run(\"test pinning with names cli text output\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tcidAStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\t\tcidBStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\n\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"testPin\", cidAStr)\n\n\t\toutARegular := cidAStr + \" recursive\"\n\t\toutADetailed := outARegular + \" testPin\"\n\t\toutBRegular := cidBStr + \" recursive\"\n\t\toutBDetailed := outBRegular + \" testPin\"\n\n\t\tlsOut := pinLs(node, \"-t=recursive\")\n\t\trequire.Contains(t, lsOut, outARegular)\n\t\trequire.NotContains(t, lsOut, outADetailed)\n\n\t\tlsOut = pinLs(node, \"-t=recursive\", \"--names\")\n\t\trequire.Contains(t, lsOut, outADetailed)\n\t\trequire.NotContains(t, lsOut, outARegular)\n\n\t\t_ = node.IPFS(\"pin\", \"update\", cidAStr, cidBStr)\n\t\tlsOut = pinLs(node, \"-t=recursive\", \"--names\")\n\t\trequire.Contains(t, lsOut, outBDetailed)\n\t\trequire.NotContains(t, lsOut, outADetailed)\n\t})\n\n\tt.Run(\"test listing pins with names that contain specific string\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tcidAStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\t\tcidBStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\t\tcidCStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\n\t\toutA := cidAStr + \" recursive testPin\"\n\t\toutB := cidBStr + \" recursive testPin\"\n\t\toutC := cidCStr + \" recursive randPin\"\n\n\t\t// make sure both -n and --name work\n\t\tfor _, nameParam := range []string{\"--name\", \"-n\"} {\n\t\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"testPin\", cidAStr)\n\t\t\tlsOut := pinLs(node, \"-t=recursive\", nameParam+\"=test\")\n\t\t\trequire.Contains(t, lsOut, outA)\n\t\t\tlsOut = pinLs(node, \"-t=recursive\", nameParam+\"=randomLabel\")\n\t\t\trequire.NotContains(t, lsOut, outA)\n\n\t\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"testPin\", cidBStr)\n\t\t\tlsOut = pinLs(node, \"-t=recursive\", nameParam+\"=test\")\n\t\t\trequire.Contains(t, lsOut, outA)\n\t\t\trequire.Contains(t, lsOut, outB)\n\n\t\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"randPin\", cidCStr)\n\t\t\tlsOut = pinLs(node, \"-t=recursive\", nameParam+\"=rand\")\n\t\t\trequire.NotContains(t, lsOut, outA)\n\t\t\trequire.NotContains(t, lsOut, outB)\n\t\t\trequire.Contains(t, lsOut, outC)\n\n\t\t\tlsOut = pinLs(node, \"-t=recursive\", nameParam+\"=testPin\")\n\t\t\trequire.Contains(t, lsOut, outA)\n\t\t\trequire.Contains(t, lsOut, outB)\n\t\t\trequire.NotContains(t, lsOut, outC)\n\t\t}\n\t})\n\n\tt.Run(\"test overwriting pin with name\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tcidStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\n\t\toutBefore := cidStr + \" recursive A\"\n\t\toutAfter := cidStr + \" recursive B\"\n\n\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"A\", cidStr)\n\t\tlsOut := pinLs(node, \"-t=recursive\", \"--names\")\n\t\trequire.Contains(t, lsOut, outBefore)\n\t\trequire.NotContains(t, lsOut, outAfter)\n\n\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"B\", cidStr)\n\t\tlsOut = pinLs(node, \"-t=recursive\", \"--names\")\n\t\trequire.Contains(t, lsOut, outAfter)\n\t\trequire.NotContains(t, lsOut, outBefore)\n\t})\n\n\t// JSON that is also the wire format of /api/v0\n\tt.Run(\"test pinning with names json output\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tcidAStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\t\tcidBStr := node.IPFSAddStr(string(random.Bytes(1000)), \"--pin=false\")\n\n\t\t_ = node.IPFS(\"pin\", \"add\", \"--name\", \"testPinJson\", cidAStr)\n\n\t\toutARegular := `\"` + cidAStr + `\":{\"Type\":\"recursive\"`\n\t\toutADetailed := outARegular + `,\"Name\":\"testPinJson\"`\n\t\toutBRegular := `\"` + cidBStr + `\":{\"Type\":\"recursive\"`\n\t\toutBDetailed := outBRegular + `,\"Name\":\"testPinJson\"`\n\n\t\tpinLs := func(args ...string) string {\n\t\t\treturn node.IPFS(StrCat(\"pin\", \"ls\", \"--enc=json\", args)...).Stdout.Trimmed()\n\t\t}\n\n\t\tlsOut := pinLs(\"-t=recursive\")\n\t\trequire.Contains(t, lsOut, outARegular)\n\t\trequire.NotContains(t, lsOut, outADetailed)\n\n\t\tlsOut = pinLs(\"-t=recursive\", \"--names\")\n\t\trequire.Contains(t, lsOut, outADetailed)\n\n\t\t_ = node.IPFS(\"pin\", \"update\", cidAStr, cidBStr)\n\t\tlsOut = pinLs(\"-t=recursive\", \"--names\")\n\t\trequire.Contains(t, lsOut, outBDetailed)\n\t})\n}\n" }, { "path": "test/cli/provide_stats_test.go", "content": "package cli\n\nimport (\n\t\"bufio\"\n\t\"encoding/json\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\tprovideStatEventuallyTimeout = 15 * time.Second\n\tprovideStatEventuallyTick = 100 * time.Millisecond\n)\n\n// sweepStats mirrors the subset of JSON fields actually used by tests.\n// This type is intentionally independent from upstream types to detect breaking changes.\n// Only includes fields that tests actually access to keep it simple and maintainable.\ntype sweepStats struct {\n\tSweep struct {\n\t\tClosed bool `json:\"closed\"`\n\t\tConnectivity struct {\n\t\t\tStatus string `json:\"status\"`\n\t\t} `json:\"connectivity\"`\n\t\tQueues struct {\n\t\t\tPendingKeyProvides int `json:\"pending_key_provides\"`\n\t\t} `json:\"queues\"`\n\t\tSchedule struct {\n\t\t\tKeys int `json:\"keys\"`\n\t\t} `json:\"schedule\"`\n\t} `json:\"Sweep\"`\n}\n\n// parseSweepStats parses JSON output from ipfs provide stat command.\n// Tests will naturally fail if upstream removes/renames fields we depend on.\nfunc parseSweepStats(t *testing.T, jsonOutput string) sweepStats {\n\tt.Helper()\n\tvar stats sweepStats\n\terr := json.Unmarshal([]byte(jsonOutput), &stats)\n\trequire.NoError(t, err, \"failed to parse provide stat JSON output\")\n\treturn stats\n}\n\n// TestProvideStatAllMetricsDocumented verifies that all metrics output by\n// `ipfs provide stat --all` are documented in docs/provide-stats.md.\n//\n// The test works as follows:\n// 1. Starts an IPFS node with Provide.DHT.SweepEnabled=true\n// 2. Runs `ipfs provide stat --all` to get all metrics\n// 3. Parses the output and extracts all lines with exactly 2 spaces indent\n// (these are the actual metric lines)\n// 4. Reads docs/provide-stats.md and extracts all ### section headers\n// 5. Ensures every metric in the output has a corresponding ### section in the docs\nfunc TestProvideStatAllMetricsDocumented(t *testing.T) {\n\tt.Parallel()\n\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init()\n\n\t// Enable sweep provider\n\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Run `ipfs provide stat --all` to get all metrics\n\tres := node.IPFS(\"provide\", \"stat\", \"--all\")\n\trequire.NoError(t, res.Err)\n\n\t// Parse metrics from the command output\n\t// Only consider lines with exactly two spaces of padding (\" \")\n\t// These are the actual metric lines as shown in provide.go\n\toutputMetrics := make(map[string]bool)\n\tscanner := bufio.NewScanner(strings.NewReader(res.Stdout.String()))\n\t// Only consider lines that start with exactly two spaces\n\tindent := \" \"\n\tfor scanner.Scan() {\n\t\tline := scanner.Text()\n\t\tif !strings.HasPrefix(line, indent) || strings.HasPrefix(line, indent) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Remove the indent\n\t\tline = strings.TrimPrefix(line, indent)\n\n\t\t// Extract metric name - everything before the first ':'\n\t\tparts := strings.SplitN(line, \":\", 2)\n\t\tif len(parts) >= 1 {\n\t\t\tmetricName := strings.TrimSpace(parts[0])\n\t\t\tif metricName != \"\" {\n\t\t\t\toutputMetrics[metricName] = true\n\t\t\t}\n\t\t}\n\t}\n\trequire.NoError(t, scanner.Err())\n\n\t// Read docs/provide-stats.md\n\t// Find the repo root by looking for go.mod\n\trepoRoot := \"..\"\n\tfor range 6 {\n\t\tif _, err := os.Stat(filepath.Join(repoRoot, \"go.mod\")); err == nil {\n\t\t\tbreak\n\t\t}\n\t\trepoRoot = filepath.Join(\"..\", repoRoot)\n\t}\n\tdocsPath := filepath.Join(repoRoot, \"docs\", \"provide-stats.md\")\n\tdocsFile, err := os.Open(docsPath)\n\trequire.NoError(t, err, \"Failed to open provide-stats.md\")\n\tdefer docsFile.Close()\n\n\t// Parse all ### metric headers from the docs\n\tdocumentedMetrics := make(map[string]bool)\n\tdocsScanner := bufio.NewScanner(docsFile)\n\tfor docsScanner.Scan() {\n\t\tline := docsScanner.Text()\n\t\tif metricName, found := strings.CutPrefix(line, \"### \"); found {\n\t\t\tmetricName = strings.TrimSpace(metricName)\n\t\t\tdocumentedMetrics[metricName] = true\n\t\t}\n\t}\n\trequire.NoError(t, docsScanner.Err())\n\n\t// Check that all output metrics are documented\n\tvar undocumentedMetrics []string\n\tfor metric := range outputMetrics {\n\t\tif !documentedMetrics[metric] {\n\t\t\tundocumentedMetrics = append(undocumentedMetrics, metric)\n\t\t}\n\t}\n\n\trequire.Empty(t, undocumentedMetrics,\n\t\t\"The following metrics from 'ipfs provide stat --all' are not documented in docs/provide-stats.md: %v\\n\"+\n\t\t\t\"All output metrics: %v\\n\"+\n\t\t\t\"Documented metrics: %v\",\n\t\tundocumentedMetrics, outputMetrics, documentedMetrics)\n}\n\n// TestProvideStatBasic tests basic functionality of ipfs provide stat\nfunc TestProvideStatBasic(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"works with Sweep provider and shows brief output\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.IPFS(\"provide\", \"stat\")\n\t\trequire.NoError(t, res.Err)\n\t\tassert.Empty(t, res.Stderr.String())\n\n\t\toutput := res.Stdout.String()\n\t\t// Brief output should contain specific full labels\n\t\tassert.Contains(t, output, \"Provide queue:\")\n\t\tassert.Contains(t, output, \"Reprovide queue:\")\n\t\tassert.Contains(t, output, \"CIDs scheduled:\")\n\t\tassert.Contains(t, output, \"Regions scheduled:\")\n\t\tassert.Contains(t, output, \"Avg record holders:\")\n\t\tassert.Contains(t, output, \"Ongoing provides:\")\n\t\tassert.Contains(t, output, \"Ongoing reprovides:\")\n\t\tassert.Contains(t, output, \"Total CIDs provided:\")\n\t})\n\n\tt.Run(\"requires daemon to be online\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\n\t\tres := node.RunIPFS(\"provide\", \"stat\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"this command must be run in online mode\")\n\t})\n}\n\n// TestProvideStatFlags tests various command flags\nfunc TestProvideStatFlags(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"--all flag shows all sections with headings\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.IPFS(\"provide\", \"stat\", \"--all\")\n\t\trequire.NoError(t, res.Err)\n\n\t\toutput := res.Stdout.String()\n\t\t// Should contain section headings with colons\n\t\tassert.Contains(t, output, \"Connectivity:\")\n\t\tassert.Contains(t, output, \"Queues:\")\n\t\tassert.Contains(t, output, \"Schedule:\")\n\t\tassert.Contains(t, output, \"Timings:\")\n\t\tassert.Contains(t, output, \"Network:\")\n\t\tassert.Contains(t, output, \"Operations:\")\n\t\tassert.Contains(t, output, \"Workers:\")\n\n\t\t// Should contain detailed metrics not in brief mode\n\t\tassert.Contains(t, output, \"Uptime:\")\n\t\tassert.Contains(t, output, \"Cycle started:\")\n\t\tassert.Contains(t, output, \"Reprovide interval:\")\n\t\tassert.Contains(t, output, \"Peers swept:\")\n\t\tassert.Contains(t, output, \"Full keyspace coverage:\")\n\t})\n\n\tt.Run(\"--compact requires --all\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"provide\", \"stat\", \"--compact\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"--compact requires --all flag\")\n\t})\n\n\tt.Run(\"--compact with --all shows 2-column layout\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.IPFS(\"provide\", \"stat\", \"--all\", \"--compact\")\n\t\trequire.NoError(t, res.Err)\n\n\t\toutput := res.Stdout.String()\n\t\tlines := strings.Split(strings.TrimSpace(output), \"\\n\")\n\t\trequire.NotEmpty(t, lines)\n\n\t\t// In compact mode, find a line that has both Schedule and Connectivity metrics\n\t\t// This confirms 2-column layout is working\n\t\tfoundTwoColumns := false\n\t\tfor _, line := range lines {\n\t\t\tif strings.Contains(line, \"CIDs scheduled:\") && strings.Contains(line, \"Status:\") {\n\t\t\t\tfoundTwoColumns = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tassert.True(t, foundTwoColumns, \"Should have at least one line with both 'CIDs scheduled:' and 'Status:' confirming 2-column layout\")\n\t})\n\n\tt.Run(\"individual section flags work with full labels\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\ttestCases := []struct {\n\t\t\tflag string\n\t\t\tcontains []string\n\t\t}{\n\t\t\t{\n\t\t\t\tflag: \"--connectivity\",\n\t\t\t\tcontains: []string{\"Status:\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tflag: \"--queues\",\n\t\t\t\tcontains: []string{\"Provide queue:\", \"Reprovide queue:\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tflag: \"--schedule\",\n\t\t\t\tcontains: []string{\"CIDs scheduled:\", \"Regions scheduled:\", \"Avg prefix length:\", \"Next region prefix:\", \"Next region reprovide:\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tflag: \"--timings\",\n\t\t\t\tcontains: []string{\"Uptime:\", \"Current time offset:\", \"Cycle started:\", \"Reprovide interval:\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tflag: \"--network\",\n\t\t\t\tcontains: []string{\"Avg record holders:\", \"Peers swept:\", \"Full keyspace coverage:\", \"Reachable peers:\", \"Avg region size:\", \"Replication factor:\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tflag: \"--operations\",\n\t\t\t\tcontains: []string{\"Ongoing provides:\", \"Ongoing reprovides:\", \"Total CIDs provided:\", \"Total records provided:\", \"Total provide errors:\"},\n\t\t\t},\n\t\t\t{\n\t\t\t\tflag: \"--workers\",\n\t\t\t\tcontains: []string{\"Active workers:\", \"Free workers:\", \"Workers stats:\", \"Periodic\", \"Burst\"},\n\t\t\t},\n\t\t}\n\n\t\tfor _, tc := range testCases {\n\t\t\tres := node.IPFS(\"provide\", \"stat\", tc.flag)\n\t\t\trequire.NoError(t, res.Err, \"flag %s should work\", tc.flag)\n\t\t\toutput := res.Stdout.String()\n\t\t\tfor _, expected := range tc.contains {\n\t\t\t\tassert.Contains(t, output, expected, \"flag %s should contain '%s'\", tc.flag, expected)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"multiple section flags can be combined\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.IPFS(\"provide\", \"stat\", \"--network\", \"--operations\")\n\t\trequire.NoError(t, res.Err)\n\n\t\toutput := res.Stdout.String()\n\t\t// Should have section headings when multiple flags combined\n\t\tassert.Contains(t, output, \"Network:\")\n\t\tassert.Contains(t, output, \"Operations:\")\n\t\tassert.Contains(t, output, \"Avg record holders:\")\n\t\tassert.Contains(t, output, \"Ongoing provides:\")\n\t})\n}\n\n// TestProvideStatLegacyProvider tests Legacy provider specific behavior\nfunc TestProvideStatLegacyProvider(t *testing.T) {\n\tt.Parallel()\n\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init()\n\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", false)\n\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\tt.Run(\"shows legacy stats from old provider system\", func(t *testing.T) {\n\t\tres := node.IPFS(\"provide\", \"stat\")\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Legacy provider shows stats from the old reprovider system\n\t\toutput := res.Stdout.String()\n\t\tassert.Contains(t, output, \"TotalReprovides:\")\n\t\tassert.Contains(t, output, \"AvgReprovideDuration:\")\n\t\tassert.Contains(t, output, \"LastReprovideDuration:\")\n\t})\n\n\tt.Run(\"rejects flags with legacy provider\", func(t *testing.T) {\n\t\tflags := []string{\"--all\", \"--connectivity\", \"--queues\", \"--network\", \"--workers\"}\n\t\tfor _, flag := range flags {\n\t\t\tres := node.RunIPFS(\"provide\", \"stat\", flag)\n\t\t\tassert.Error(t, res.Err, \"flag %s should be rejected for legacy provider\", flag)\n\t\t\tassert.Contains(t, res.Stderr.String(), \"cannot use flags with legacy provide stats\")\n\t\t}\n\t})\n\n\tt.Run(\"rejects --lan flag with legacy provider\", func(t *testing.T) {\n\t\tres := node.RunIPFS(\"provide\", \"stat\", \"--lan\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"LAN stats only available for Sweep provider with Dual DHT\")\n\t})\n}\n\n// TestProvideStatOutputFormats tests different output formats\nfunc TestProvideStatOutputFormats(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"JSON output with Sweep provider\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Parse JSON to verify structure\n\t\tvar result struct {\n\t\t\tSweep map[string]any `json:\"Sweep\"`\n\t\t\tLegacy map[string]any `json:\"Legacy\"`\n\t\t}\n\t\terr := json.Unmarshal([]byte(res.Stdout.String()), &result)\n\t\trequire.NoError(t, err, \"Output should be valid JSON\")\n\t\tassert.NotNil(t, result.Sweep, \"Sweep stats should be present\")\n\t\tassert.Nil(t, result.Legacy, \"Legacy stats should not be present\")\n\t})\n\n\tt.Run(\"JSON output with Legacy provider\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", false)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Parse JSON to verify structure\n\t\tvar result struct {\n\t\t\tSweep map[string]any `json:\"Sweep\"`\n\t\t\tLegacy map[string]any `json:\"Legacy\"`\n\t\t}\n\t\terr := json.Unmarshal([]byte(res.Stdout.String()), &result)\n\t\trequire.NoError(t, err, \"Output should be valid JSON\")\n\t\tassert.Nil(t, result.Sweep, \"Sweep stats should not be present\")\n\t\tassert.NotNil(t, result.Legacy, \"Legacy stats should be present\")\n\t})\n}\n\n// TestProvideStatIntegration tests integration with provide operations\nfunc TestProvideStatIntegration(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"stats reflect content being added to schedule\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.DHT.Interval\", \"1h\")\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Get initial scheduled CID count\n\t\tres1 := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\trequire.NoError(t, res1.Err)\n\t\tinitialKeys := parseSweepStats(t, res1.Stdout.String()).Sweep.Schedule.Keys\n\n\t\t// Add content - this should increase CIDs scheduled\n\t\tnode.IPFSAddStr(\"test content for stats\")\n\n\t\t// Wait for content to appear in schedule (with timeout)\n\t\t// The buffered provider may take a moment to schedule items\n\t\trequire.Eventually(t, func() bool {\n\t\t\tres := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\t\trequire.NoError(t, res.Err)\n\t\t\tstats := parseSweepStats(t, res.Stdout.String())\n\t\t\treturn stats.Sweep.Schedule.Keys > initialKeys\n\t\t}, provideStatEventuallyTimeout, provideStatEventuallyTick, \"Content should appear in schedule after adding\")\n\t})\n\n\tt.Run(\"stats work with all documented strategies\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Test all strategies documented in docs/config.md#providestrategy\n\t\tstrategies := []string{\"all\", \"pinned\", \"roots\", \"mfs\", \"pinned+mfs\"}\n\t\tfor _, strategy := range strategies {\n\t\t\th := harness.NewT(t)\n\t\t\tnode := h.NewNode().Init()\n\t\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\tnode.SetIPFSConfig(\"Provide.Strategy\", strategy)\n\t\t\tnode.StartDaemon()\n\n\t\t\tres := node.IPFS(\"provide\", \"stat\")\n\t\t\trequire.NoError(t, res.Err, \"stats should work with strategy %s\", strategy)\n\t\t\toutput := res.Stdout.String()\n\t\t\tassert.NotEmpty(t, output)\n\t\t\tassert.Contains(t, output, \"CIDs scheduled:\")\n\n\t\t\tnode.StopDaemon()\n\t\t}\n\t})\n}\n\n// TestProvideStatDisabledConfig tests behavior when provide system is disabled\nfunc TestProvideStatDisabledConfig(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"Provide.Enabled=false returns error stats not available\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", false)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"provide\", \"stat\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"stats not available\")\n\t})\n\n\tt.Run(\"Provide.Enabled=true with Provide.DHT.Interval=0 returns error stats not available\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.SetIPFSConfig(\"Provide.DHT.Interval\", \"0\")\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"provide\", \"stat\")\n\t\tassert.Error(t, res.Err)\n\t\tassert.Contains(t, res.Stderr.String(), \"stats not available\")\n\t})\n}\n" }, { "path": "test/cli/provider_test.go", "content": "package cli\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\ttimeStep = 20 * time.Millisecond\n\ttimeout = time.Second\n)\n\ntype cfgApplier func(*harness.Node)\n\nfunc runProviderSuite(t *testing.T, reprovide bool, apply cfgApplier) {\n\tt.Helper()\n\n\tinitNodes := func(t *testing.T, n int, fn func(n *harness.Node)) harness.Nodes {\n\t\tnodes := harness.NewT(t).NewNodes(n).Init()\n\t\tnodes.ForEachPar(apply)\n\t\tnodes.ForEachPar(fn)\n\t\tnodes = nodes.StartDaemons().Connect()\n\t\ttime.Sleep(500 * time.Millisecond) // wait for DHT clients to be bootstrapped\n\t\treturn nodes\n\t}\n\n\tinitNodesWithoutStart := func(t *testing.T, n int, fn func(n *harness.Node)) harness.Nodes {\n\t\tnodes := harness.NewT(t).NewNodes(n).Init()\n\t\tnodes.ForEachPar(apply)\n\t\tnodes.ForEachPar(fn)\n\t\treturn nodes\n\t}\n\n\texpectNoProviders := func(t *testing.T, cid string, nodes ...*harness.Node) {\n\t\tfor _, node := range nodes {\n\t\t\tres := node.IPFS(\"routing\", \"findprovs\", \"-n=1\", cid)\n\t\t\trequire.Empty(t, res.Stdout.String())\n\t\t}\n\t}\n\n\texpectProviders := func(t *testing.T, cid, expectedProvider string, nodes ...*harness.Node) {\n\touterLoop:\n\t\tfor _, node := range nodes {\n\t\t\tfor i := time.Duration(0); i*timeStep < timeout; i++ {\n\t\t\t\tres := node.IPFS(\"routing\", \"findprovs\", \"-n=1\", cid)\n\t\t\t\tif res.Stdout.Trimmed() == expectedProvider {\n\t\t\t\t\tcontinue outerLoop\n\t\t\t\t}\n\t\t\t}\n\t\t\trequire.FailNowf(t, \"found no providers\", \"expected a provider for %s\", cid)\n\t\t}\n\t}\n\n\tt.Run(\"Provide.Enabled=true announces new CIDs created by ipfs add\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide.Enabled=true announces new CIDs created by ipfs add --pin=false with default strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\t// Default strategy is \"all\" which should provide even unpinned content\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String(), \"--pin=false\")\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide.Enabled=true announces new CIDs created by ipfs block put --pin=false with default strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\t// Default strategy is \"all\" which should provide unpinned content from block put\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tdata := random.Bytes(256)\n\t\tcid := nodes[0].IPFSBlockPut(bytes.NewReader(data), \"--pin=false\")\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide.Enabled=true announces new CIDs created by ipfs dag put --pin=false with default strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\t// Default strategy is \"all\" which should provide unpinned content from dag put\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tdagData := `{\"hello\": \"world\", \"timestamp\": \"` + time.Now().String() + `\"}`\n\t\tcid := nodes[0].IPFSDAGPut(bytes.NewReader([]byte(dagData)), \"--pin=false\")\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide.Enabled=false disables announcement of new CID from ipfs add\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", false)\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide.Enabled=false disables manual announcement via RPC command\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", false)\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\t\tres := nodes[0].RunIPFS(\"routing\", \"provide\", cid)\n\t\tassert.Contains(t, res.Stderr.Trimmed(), \"invalid configuration: Provide.Enabled is set to 'false'\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\t})\n\n\tt.Run(\"manual provide fails when no libp2p peers and no custom HTTP router\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tapply(node)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcid := node.IPFSAddStr(time.Now().String())\n\t\tres := node.RunIPFS(\"routing\", \"provide\", cid)\n\t\tassert.Contains(t, res.Stderr.Trimmed(), \"cannot provide, no connected peers\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t})\n\n\tt.Run(\"manual provide succeeds via custom HTTP router when no libp2p peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a mock HTTP server that accepts provide requests.\n\t\t// This simulates the undocumented API behavior described in\n\t\t// https://discuss.ipfs.tech/t/only-peers-found-from-dht-seem-to-be-getting-used-as-relays-so-cant-use-http-routers/19545/9\n\t\t// Note: This is NOT IPIP-378, which was not implemented.\n\t\tmockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t// Accept both PUT and POST requests to /routing/v1/providers and /routing/v1/ipns\n\t\t\tif (r.Method == http.MethodPut || r.Method == http.MethodPost) &&\n\t\t\t\t(strings.HasPrefix(r.URL.Path, \"/routing/v1/providers\") || strings.HasPrefix(r.URL.Path, \"/routing/v1/ipns\")) {\n\t\t\t\t// Return HTTP 200 to indicate successful publishing\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t} else {\n\t\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t\t}\n\t\t}))\n\t\tdefer mockServer.Close()\n\n\t\th := harness.NewT(t)\n\t\tnode := h.NewNode().Init()\n\t\tapply(node)\n\t\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t// Configure a custom HTTP router for providing.\n\t\t// Using our mock server that will accept the provide requests.\n\t\troutingConf := map[string]any{\n\t\t\t\"Type\": \"custom\", // https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md#configuration-file-example\n\t\t\t\"Methods\": map[string]any{\n\t\t\t\t\"provide\": map[string]any{\"RouterName\": \"MyCustomRouter\"},\n\t\t\t\t\"get-ipns\": map[string]any{\"RouterName\": \"MyCustomRouter\"},\n\t\t\t\t\"put-ipns\": map[string]any{\"RouterName\": \"MyCustomRouter\"},\n\t\t\t\t\"find-peers\": map[string]any{\"RouterName\": \"MyCustomRouter\"},\n\t\t\t\t\"find-providers\": map[string]any{\"RouterName\": \"MyCustomRouter\"},\n\t\t\t},\n\t\t\t\"Routers\": map[string]any{\n\t\t\t\t\"MyCustomRouter\": map[string]any{\n\t\t\t\t\t\"Type\": \"http\",\n\t\t\t\t\t\"Parameters\": map[string]any{\n\t\t\t\t\t\t// Use the mock server URL\n\t\t\t\t\t\t\"Endpoint\": mockServer.URL,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\tnode.SetIPFSConfig(\"Routing\", routingConf)\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tcid := node.IPFSAddStr(time.Now().String())\n\t\t// The command should successfully provide via HTTP even without libp2p peers\n\t\tres := node.RunIPFS(\"routing\", \"provide\", cid)\n\t\tassert.Empty(t, res.Stderr.String(), \"Should have no errors when providing via HTTP router\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"Should succeed with exit code 0\")\n\t})\n\n\t// Right now Provide and Reprovide are tied together\n\tt.Run(\"Reprovide.Interval=0 disables announcement of new CID too\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.DHT.Interval\", \"0\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\t})\n\n\t// It is a lesser evil - forces users to fix their config and have some sort of interval\n\tt.Run(\"Manual Reprovide trigger does not work when periodic reprovide is disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.DHT.Interval\", \"0\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\n\t\tres := nodes[0].RunIPFS(\"routing\", \"reprovide\")\n\t\tassert.Contains(t, res.Stderr.Trimmed(), \"invalid configuration: Provide.DHT.Interval is set to '0'\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\t})\n\n\t// It is a lesser evil - forces users to fix their config and have some sort of interval\n\tt.Run(\"Manual Reprovide trigger does not work when Provide system is disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", false)\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\n\t\tres := nodes[0].RunIPFS(\"routing\", \"reprovide\")\n\t\tassert.Contains(t, res.Stderr.Trimmed(), \"invalid configuration: Provide.Enabled is set to 'false'\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide with 'all' strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"all\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\tcid := nodes[0].IPFSAddStr(\"all strategy\")\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide with 'pinned' strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"pinned\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add a non-pinned CID (should not be provided)\n\t\tcid := nodes[0].IPFSAddStr(\"pinned strategy\", \"--pin=false\")\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\n\t\t// Pin the CID (should now be provided)\n\t\tnodes[0].IPFS(\"pin\", \"add\", cid)\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide with 'pinned+mfs' strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"pinned+mfs\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add a pinned CID (should be provided)\n\t\tcidPinned := nodes[0].IPFSAddStr(\"pinned content\")\n\t\tcidUnpinned := nodes[0].IPFSAddStr(\"unpinned content\", \"--pin=false\")\n\t\tcidMFS := nodes[0].IPFSAddStr(\"mfs content\", \"--pin=false\")\n\t\tnodes[0].IPFS(\"files\", \"cp\", \"/ipfs/\"+cidMFS, \"/myfile\")\n\n\t\tn0pid := nodes[0].PeerID().String()\n\t\texpectProviders(t, cidPinned, n0pid, nodes[1:]...)\n\t\texpectNoProviders(t, cidUnpinned, nodes[1:]...)\n\t\texpectProviders(t, cidMFS, n0pid, nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide with 'roots' strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"roots\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add a root CID (should be provided)\n\t\tcidRoot := nodes[0].IPFSAddStr(\"roots strategy\", \"-w\", \"-Q\")\n\t\t// the same without wrapping should give us a child node.\n\t\tcidChild := nodes[0].IPFSAddStr(\"root strategy\", \"--pin=false\")\n\n\t\texpectProviders(t, cidRoot, nodes[0].PeerID().String(), nodes[1:]...)\n\t\texpectNoProviders(t, cidChild, nodes[1:]...)\n\t})\n\n\tt.Run(\"Provide with 'mfs' strategy\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := initNodes(t, 2, func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"mfs\")\n\t\t})\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add a file to MFS (should be provided)\n\t\tdata := random.Bytes(1000)\n\t\tcid := nodes[0].IPFSAdd(bytes.NewReader(data), \"-Q\")\n\n\t\t// not yet in MFS\n\t\texpectNoProviders(t, cid, nodes[1:]...)\n\n\t\tnodes[0].IPFS(\"files\", \"cp\", \"/ipfs/\"+cid, \"/myfile\")\n\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t})\n\n\tif reprovide {\n\n\t\tt.Run(\"Reprovides with 'all' strategy when strategy is '' (empty)\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tnodes := initNodesWithoutStart(t, 2, func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"\")\n\t\t\t})\n\n\t\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\n\t\t\tnodes = nodes.StartDaemons().Connect()\n\t\t\tdefer nodes.StopDaemons()\n\t\t\texpectNoProviders(t, cid, nodes[1:]...)\n\n\t\t\tnodes[0].IPFS(\"routing\", \"reprovide\")\n\n\t\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t\t})\n\n\t\tt.Run(\"Reprovides with 'all' strategy\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tnodes := initNodesWithoutStart(t, 2, func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"all\")\n\t\t\t})\n\n\t\t\tcid := nodes[0].IPFSAddStr(time.Now().String())\n\n\t\t\tnodes = nodes.StartDaemons().Connect()\n\t\t\tdefer nodes.StopDaemons()\n\t\t\texpectNoProviders(t, cid, nodes[1:]...)\n\n\t\t\tnodes[0].IPFS(\"routing\", \"reprovide\")\n\n\t\t\texpectProviders(t, cid, nodes[0].PeerID().String(), nodes[1:]...)\n\t\t})\n\n\t\tt.Run(\"Reprovides with 'pinned' strategy\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tfoo := random.Bytes(1000)\n\t\t\tbar := random.Bytes(1000)\n\n\t\t\tnodes := initNodesWithoutStart(t, 2, func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"pinned\")\n\t\t\t})\n\n\t\t\t// Add a pin while offline so it cannot be provided\n\t\t\tcidBarDir := nodes[0].IPFSAdd(bytes.NewReader(bar), \"-Q\", \"-w\")\n\n\t\t\tnodes = nodes.StartDaemons().Connect()\n\t\t\tdefer nodes.StopDaemons()\n\n\t\t\t// Add content without pinning while daemon line\n\t\t\tcidFoo := nodes[0].IPFSAdd(bytes.NewReader(foo), \"--pin=false\")\n\t\t\tcidBar := nodes[0].IPFSAdd(bytes.NewReader(bar), \"--pin=false\")\n\n\t\t\t// Nothing should have been provided. The pin was offline, and\n\t\t\t// the others should not be provided per the strategy.\n\t\t\texpectNoProviders(t, cidFoo, nodes[1:]...)\n\t\t\texpectNoProviders(t, cidBar, nodes[1:]...)\n\t\t\texpectNoProviders(t, cidBarDir, nodes[1:]...)\n\n\t\t\tnodes[0].IPFS(\"routing\", \"reprovide\")\n\n\t\t\t// cidFoo is not pinned so should not be provided.\n\t\t\texpectNoProviders(t, cidFoo, nodes[1:]...)\n\t\t\t// cidBar gets provided by being a child from cidBarDir even though we added with pin=false.\n\t\t\texpectProviders(t, cidBar, nodes[0].PeerID().String(), nodes[1:]...)\n\t\t\texpectProviders(t, cidBarDir, nodes[0].PeerID().String(), nodes[1:]...)\n\t\t})\n\n\t\tt.Run(\"Reprovides with 'roots' strategy\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tfoo := random.Bytes(1000)\n\t\t\tbar := random.Bytes(1000)\n\n\t\t\tnodes := initNodesWithoutStart(t, 2, func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"roots\")\n\t\t\t})\n\t\t\tn0pid := nodes[0].PeerID().String()\n\n\t\t\t// Add a pin. Only root should get pinned but not provided\n\t\t\t// because node not started\n\t\t\tcidBarDir := nodes[0].IPFSAdd(bytes.NewReader(bar), \"-Q\", \"-w\")\n\n\t\t\tnodes = nodes.StartDaemons().Connect()\n\t\t\tdefer nodes.StopDaemons()\n\n\t\t\tcidFoo := nodes[0].IPFSAdd(bytes.NewReader(foo))\n\t\t\tcidBar := nodes[0].IPFSAdd(bytes.NewReader(bar), \"--pin=false\")\n\n\t\t\t// cidFoo will get provided per the strategy but cidBar will not.\n\t\t\texpectProviders(t, cidFoo, n0pid, nodes[1:]...)\n\t\t\texpectNoProviders(t, cidBar, nodes[1:]...)\n\n\t\t\tnodes[0].IPFS(\"routing\", \"reprovide\")\n\n\t\t\texpectProviders(t, cidFoo, n0pid, nodes[1:]...)\n\t\t\texpectNoProviders(t, cidBar, nodes[1:]...)\n\t\t\texpectProviders(t, cidBarDir, n0pid, nodes[1:]...)\n\t\t})\n\n\t\tt.Run(\"Reprovides with 'mfs' strategy\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tbar := random.Bytes(1000)\n\n\t\t\tnodes := initNodesWithoutStart(t, 2, func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"mfs\")\n\t\t\t})\n\t\t\tn0pid := nodes[0].PeerID().String()\n\n\t\t\t// add something and lets put it in MFS\n\t\t\tcidBar := nodes[0].IPFSAdd(bytes.NewReader(bar), \"--pin=false\", \"-Q\")\n\t\t\tnodes[0].IPFS(\"files\", \"cp\", \"/ipfs/\"+cidBar, \"/myfile\")\n\n\t\t\tnodes = nodes.StartDaemons().Connect()\n\t\t\tdefer nodes.StopDaemons()\n\n\t\t\t// cidBar is in MFS but not provided\n\t\t\texpectNoProviders(t, cidBar, nodes[1:]...)\n\n\t\t\tnodes[0].IPFS(\"routing\", \"reprovide\")\n\n\t\t\t// And now is provided\n\t\t\texpectProviders(t, cidBar, n0pid, nodes[1:]...)\n\t\t})\n\n\t\tt.Run(\"Reprovides with 'pinned+mfs' strategy\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tnodes := initNodesWithoutStart(t, 2, func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"pinned+mfs\")\n\t\t\t})\n\t\t\tn0pid := nodes[0].PeerID().String()\n\n\t\t\t// Add a pinned CID (should be provided)\n\t\t\tcidPinned := nodes[0].IPFSAddStr(\"pinned content\", \"--pin=true\")\n\t\t\t// Add a CID to MFS (should be provided)\n\t\t\tcidMFS := nodes[0].IPFSAddStr(\"mfs content\")\n\t\t\tnodes[0].IPFS(\"files\", \"cp\", \"/ipfs/\"+cidMFS, \"/myfile\")\n\t\t\t// Add a CID that is neither pinned nor in MFS (should not be provided)\n\t\t\tcidNeither := nodes[0].IPFSAddStr(\"neither content\", \"--pin=false\")\n\n\t\t\tnodes = nodes.StartDaemons().Connect()\n\t\t\tdefer nodes.StopDaemons()\n\n\t\t\t// Trigger reprovide\n\t\t\tnodes[0].IPFS(\"routing\", \"reprovide\")\n\n\t\t\t// Check that pinned CID is provided\n\t\t\texpectProviders(t, cidPinned, n0pid, nodes[1:]...)\n\t\t\t// Check that MFS CID is provided\n\t\t\texpectProviders(t, cidMFS, n0pid, nodes[1:]...)\n\t\t\t// Check that neither CID is not provided\n\t\t\texpectNoProviders(t, cidNeither, nodes[1:]...)\n\t\t})\n\t}\n\n\tt.Run(\"provide clear command removes items from provide queue\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := harness.NewT(t).NewNodes(1).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\tn.SetIPFSConfig(\"Provide.DHT.Interval\", \"22h\")\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"all\")\n\t\t})\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Clear the provide queue first time - works regardless of queue state\n\t\tres1 := nodes[0].IPFS(\"provide\", \"clear\")\n\t\trequire.NoError(t, res1.Err)\n\n\t\t// Should report cleared items and proper message format\n\t\tassert.Contains(t, res1.Stdout.String(), \"removed\")\n\t\tassert.Contains(t, res1.Stdout.String(), \"items from provide queue\")\n\n\t\t// Clear the provide queue second time - should definitely report 0 items\n\t\tres2 := nodes[0].IPFS(\"provide\", \"clear\")\n\t\trequire.NoError(t, res2.Err)\n\n\t\t// Should report 0 items cleared since queue was already cleared\n\t\tassert.Contains(t, res2.Stdout.String(), \"removed 0 items from provide queue\")\n\t})\n\n\tt.Run(\"provide clear command with quiet option\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := harness.NewT(t).NewNodes(1).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\tn.SetIPFSConfig(\"Provide.DHT.Interval\", \"22h\")\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"all\")\n\t\t})\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Clear the provide queue with quiet option\n\t\tres := nodes[0].IPFS(\"provide\", \"clear\", \"-q\")\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Should have no output when quiet\n\t\tassert.Empty(t, res.Stdout.String())\n\t})\n\n\tt.Run(\"provide clear command works when provider is disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := harness.NewT(t).NewNodes(1).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", false)\n\t\t\tn.SetIPFSConfig(\"Provide.DHT.Interval\", \"22h\")\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"all\")\n\t\t})\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Clear should succeed even when provider is disabled\n\t\tres := nodes[0].IPFS(\"provide\", \"clear\")\n\t\trequire.NoError(t, res.Err)\n\t})\n\n\tt.Run(\"provide clear command returns JSON with removed item count\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnodes := harness.NewT(t).NewNodes(1).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.SetIPFSConfig(\"Provide.Enabled\", true)\n\t\t\tn.SetIPFSConfig(\"Provide.DHT.Interval\", \"22h\")\n\t\t\tn.SetIPFSConfig(\"Provide.Strategy\", \"all\")\n\t\t})\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Clear the provide queue with JSON encoding\n\t\tres := nodes[0].IPFS(\"provide\", \"clear\", \"--enc=json\")\n\t\trequire.NoError(t, res.Err)\n\n\t\t// Should return valid JSON with the number of removed items\n\t\toutput := res.Stdout.String()\n\t\tassert.NotEmpty(t, output)\n\n\t\t// Parse JSON to verify structure\n\t\tvar result int\n\t\terr := json.Unmarshal([]byte(output), &result)\n\t\trequire.NoError(t, err, \"Output should be valid JSON\")\n\n\t\t// Should be a non-negative integer (0 or positive)\n\t\tassert.GreaterOrEqual(t, result, 0)\n\t})\n}\n\n// runResumeTests validates Provide.DHT.ResumeEnabled behavior for SweepingProvider.\n//\n// Background: The provider tracks current_time_offset = (now - cycleStart) % interval\n// where cycleStart is the timestamp marking the beginning of the reprovide cycle.\n// With ResumeEnabled=true, cycleStart persists in the datastore across restarts.\n// With ResumeEnabled=false, cycleStart resets to 'now' on each startup.\nfunc runResumeTests(t *testing.T, apply cfgApplier) {\n\tt.Helper()\n\n\tconst (\n\t\treprovideInterval = 30 * time.Second\n\t\tinitialRuntime = 10 * time.Second // Let cycle progress\n\t\tdowntime = 5 * time.Second // Simulated offline period\n\t\trestartTime = 2 * time.Second // Daemon restart stabilization\n\n\t\t// Thresholds account for timing jitter (~2-3s margin)\n\t\tminOffsetBeforeRestart = 8 * time.Second // Expect ~10s\n\t\tminOffsetAfterResume = 12 * time.Second // Expect ~17s (10s + 5s + 2s)\n\t\tmaxOffsetAfterReset = 5 * time.Second // Expect ~2s (fresh start)\n\t)\n\n\tsetupNode := func(t *testing.T, resumeEnabled bool) *harness.Node {\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tapply(node) // Sets Provide.DHT.SweepEnabled=true\n\t\tnode.SetIPFSConfig(\"Provide.DHT.ResumeEnabled\", resumeEnabled)\n\t\tnode.SetIPFSConfig(\"Provide.DHT.Interval\", reprovideInterval.String())\n\t\tnode.SetIPFSConfig(\"Bootstrap\", []string{})\n\t\tnode.StartDaemon()\n\t\treturn node\n\t}\n\n\tt.Run(\"preserves cycle state across restart\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := setupNode(t, true)\n\t\tdefer node.StopDaemon()\n\n\t\tfor i := range 10 {\n\t\t\tnode.IPFSAddStr(fmt.Sprintf(\"resume-test-%d-%d\", i, time.Now().UnixNano()))\n\t\t}\n\n\t\ttime.Sleep(initialRuntime)\n\n\t\tbeforeRestart := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\toffsetBeforeRestart, _, err := parseProvideStatJSON(beforeRestart.Stdout.String())\n\t\trequire.NoError(t, err)\n\t\trequire.Greater(t, offsetBeforeRestart, minOffsetBeforeRestart,\n\t\t\t\"cycle should have progressed\")\n\n\t\tnode.StopDaemon()\n\t\ttime.Sleep(downtime)\n\t\tnode.StartDaemon()\n\t\ttime.Sleep(restartTime)\n\n\t\tafterRestart := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\toffsetAfterRestart, _, err := parseProvideStatJSON(afterRestart.Stdout.String())\n\t\trequire.NoError(t, err)\n\n\t\tassert.GreaterOrEqual(t, offsetAfterRestart, minOffsetAfterResume,\n\t\t\t\"offset should account for downtime\")\n\t})\n\n\tt.Run(\"resets cycle when disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := setupNode(t, false)\n\t\tdefer node.StopDaemon()\n\n\t\tfor i := range 10 {\n\t\t\tnode.IPFSAddStr(fmt.Sprintf(\"no-resume-%d-%d\", i, time.Now().UnixNano()))\n\t\t}\n\n\t\ttime.Sleep(initialRuntime)\n\n\t\tbeforeRestart := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\toffsetBeforeRestart, _, err := parseProvideStatJSON(beforeRestart.Stdout.String())\n\t\trequire.NoError(t, err)\n\t\trequire.Greater(t, offsetBeforeRestart, minOffsetBeforeRestart,\n\t\t\t\"cycle should have progressed\")\n\n\t\tnode.StopDaemon()\n\t\ttime.Sleep(downtime)\n\t\tnode.StartDaemon()\n\t\ttime.Sleep(restartTime)\n\n\t\tafterRestart := node.IPFS(\"provide\", \"stat\", \"--enc=json\")\n\t\toffsetAfterRestart, _, err := parseProvideStatJSON(afterRestart.Stdout.String())\n\t\trequire.NoError(t, err)\n\n\t\tassert.Less(t, offsetAfterRestart, maxOffsetAfterReset,\n\t\t\t\"offset should reset to near zero\")\n\t})\n}\n\ntype provideStatJSON struct {\n\tSweep struct {\n\t\tTiming struct {\n\t\t\tCurrentTimeOffset int64 `json:\"current_time_offset\"` // nanoseconds\n\t\t} `json:\"timing\"`\n\t\tSchedule struct {\n\t\t\tNextReprovidePrefix string `json:\"next_reprovide_prefix\"`\n\t\t} `json:\"schedule\"`\n\t} `json:\"Sweep\"`\n}\n\n// parseProvideStatJSON extracts timing and schedule information from\n// the JSON output of 'ipfs provide stat --enc=json'.\n// Note: prefix is unused in current tests but kept for potential future use.\nfunc parseProvideStatJSON(output string) (offset time.Duration, prefix string, err error) {\n\tvar stat provideStatJSON\n\tif err := json.Unmarshal([]byte(output), &stat); err != nil {\n\t\treturn 0, \"\", err\n\t}\n\toffset = time.Duration(stat.Sweep.Timing.CurrentTimeOffset)\n\tprefix = stat.Sweep.Schedule.NextReprovidePrefix\n\treturn offset, prefix, nil\n}\n\nfunc TestProvider(t *testing.T) {\n\tt.Parallel()\n\n\tvariants := []struct {\n\t\tname string\n\t\treprovide bool\n\t\tapply cfgApplier\n\t}{\n\t\t{\n\t\t\tname: \"LegacyProvider\",\n\t\t\treprovide: true,\n\t\t\tapply: func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", false)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"SweepingProvider\",\n\t\t\treprovide: false,\n\t\t\tapply: func(n *harness.Node) {\n\t\t\t\tn.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, v := range variants {\n\t\tt.Run(v.name, func(t *testing.T) {\n\t\t\t// t.Parallel()\n\t\t\trunProviderSuite(t, v.reprovide, v.apply)\n\n\t\t\t// Resume tests only apply to SweepingProvider\n\t\t\tif v.name == \"SweepingProvider\" {\n\t\t\t\trunResumeTests(t, v.apply)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestHTTPOnlyProviderWithSweepEnabled tests that provider records are correctly\n// sent to HTTP routers when Routing.Type=\"custom\" with only HTTP routers configured,\n// even when Provide.DHT.SweepEnabled=true (the default since v0.39).\n//\n// This is a regression test for https://github.com/ipfs/kubo/issues/11089\nfunc TestHTTPOnlyProviderWithSweepEnabled(t *testing.T) {\n\tt.Parallel()\n\n\t// Track provide requests received by the mock HTTP router\n\tvar provideRequests atomic.Int32\n\tmockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif (r.Method == http.MethodPut || r.Method == http.MethodPost) &&\n\t\t\tstrings.HasPrefix(r.URL.Path, \"/routing/v1/providers\") {\n\t\t\tprovideRequests.Add(1)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t} else if strings.HasPrefix(r.URL.Path, \"/routing/v1/providers\") && r.Method == http.MethodGet {\n\t\t\t// Return empty providers for findprovs\n\t\t\tw.Header().Set(\"Content-Type\", \"application/x-ndjson\")\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t} else {\n\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t}\n\t}))\n\tdefer mockServer.Close()\n\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init()\n\n\t// Explicitly set SweepEnabled=true (the default since v0.39, but be explicit for test clarity)\n\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\n\t// Configure HTTP-only custom routing (no DHT) with explicit Routing.Type=custom\n\troutingConf := map[string]any{\n\t\t\"Type\": \"custom\", // Explicitly set Routing.Type=custom\n\t\t\"Methods\": map[string]any{\n\t\t\t\"provide\": map[string]any{\"RouterName\": \"HTTPRouter\"},\n\t\t\t\"get-ipns\": map[string]any{\"RouterName\": \"HTTPRouter\"},\n\t\t\t\"put-ipns\": map[string]any{\"RouterName\": \"HTTPRouter\"},\n\t\t\t\"find-peers\": map[string]any{\"RouterName\": \"HTTPRouter\"},\n\t\t\t\"find-providers\": map[string]any{\"RouterName\": \"HTTPRouter\"},\n\t\t},\n\t\t\"Routers\": map[string]any{\n\t\t\t\"HTTPRouter\": map[string]any{\n\t\t\t\t\"Type\": \"http\",\n\t\t\t\t\"Parameters\": map[string]any{\n\t\t\t\t\t\"Endpoint\": mockServer.URL,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tnode.SetIPFSConfig(\"Routing\", routingConf)\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\t// Add content and manually provide it\n\tcid := node.IPFSAddStr(time.Now().String())\n\n\t// Manual provide should succeed even without libp2p peers\n\tres := node.RunIPFS(\"routing\", \"provide\", cid)\n\t// Check that the command succeeded (exit code 0) and no provide-related errors\n\tassert.Equal(t, 0, res.ExitCode(), \"routing provide should succeed with HTTP-only routing and SweepEnabled=true\")\n\tassert.NotContains(t, res.Stderr.String(), \"cannot provide\", \"should not have provide errors\")\n\n\t// Verify HTTP router received at least one provide request\n\tassert.Greater(t, provideRequests.Load(), int32(0),\n\t\t\"HTTP router should have received provide requests\")\n\n\t// Verify 'provide stat' works with HTTP-only routing (regression test for stats)\n\tstatRes := node.RunIPFS(\"provide\", \"stat\")\n\tassert.Equal(t, 0, statRes.ExitCode(), \"provide stat should succeed with HTTP-only routing\")\n\tassert.NotContains(t, statRes.Stderr.String(), \"stats not available\",\n\t\t\"should not report stats unavailable\")\n\t// LegacyProvider outputs \"TotalReprovides:\" in its stats\n\tassert.Contains(t, statRes.Stdout.String(), \"TotalReprovides:\",\n\t\t\"should show legacy provider stats\")\n}\n\n// TestProviderKeystoreDatastoreCompaction verifies that the SweepingProvider's\n// keystore uses a datastore factory that creates separate physical datastores\n// and reclaims disk space by deleting old datastores after each reset cycle.\n//\n// The keystore uses two alternating namespaces (\"0\" and \"1\") plus a \"meta\"\n// namespace. The lifecycle is:\n// 1. First start: namespace \"0\" is created as the initial active datastore\n// 2. First reset (keystore sync at startup): \"1\" is created, data is written,\n// namespaces swap, \"0\" is destroyed from disk via os.RemoveAll\n// 3. Restart: \"1\" and \"meta\" survive on disk\n// 4. Second reset: \"0\" is recreated, namespaces swap, \"1\" is destroyed\nfunc TestProviderKeystoreDatastorePurge(t *testing.T) {\n\tt.Parallel()\n\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init()\n\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{})\n\n\t// Add content offline so the keystore has something to sync on startup.\n\tfor i := range 5 {\n\t\tnode.IPFSAddStr(fmt.Sprintf(\"keystore-compaction-test-%d\", i))\n\t}\n\n\tkeystoreBase := filepath.Join(node.Dir, \"provider-keystore\")\n\tns0 := filepath.Join(keystoreBase, \"0\")\n\tns1 := filepath.Join(keystoreBase, \"1\")\n\n\t// Directory should not exist before starting the daemon.\n\t_, err := os.Stat(keystoreBase)\n\trequire.True(t, os.IsNotExist(err), \"provider-keystore should not exist before daemon start\")\n\n\t// --- First start: triggers keystore sync (ResetCids) ---\n\t// Init creates \"0\", then reset swaps to \"1\" and destroys \"0\".\n\tnode.StartDaemon()\n\n\trequire.Eventually(t, func() bool {\n\t\treturn dirExists(ns1) && !dirExists(ns0)\n\t}, 30*time.Second, 200*time.Millisecond,\n\t\t\"after first reset: ns1 should exist, ns0 should be destroyed\")\n\n\t// --- Restart: triggers a second keystore sync (ResetCids) ---\n\t// Reset swaps back to \"0\" and destroys \"1\".\n\tnode.StopDaemon()\n\n\t// Between restarts: ns1 survives on disk, ns0 does not.\n\tassert.True(t, dirExists(ns1), \"ns1 should survive shutdown\")\n\tassert.False(t, dirExists(ns0), \"ns0 should not reappear between restarts\")\n\n\tnode.StartDaemon()\n\n\trequire.Eventually(t, func() bool {\n\t\treturn dirExists(ns0) && !dirExists(ns1)\n\t}, 30*time.Second, 200*time.Millisecond,\n\t\t\"after second reset: ns0 should exist, ns1 should be destroyed\")\n\n\tnode.StopDaemon()\n}\n\n// TestProviderKeystoreMigrationPurge verifies that orphaned keystore data\n// left in the shared repo datastore by older Kubo versions is purged on\n// the first sweep-enabled daemon start. The migration is triggered by the\n// absence of the /provider-keystore/ directory.\nfunc TestProviderKeystoreMigrationPurge(t *testing.T) {\n\tt.Parallel()\n\n\th := harness.NewT(t)\n\tnode := h.NewNode().Init()\n\tnode.SetIPFSConfig(\"Provide.DHT.SweepEnabled\", true)\n\tnode.SetIPFSConfig(\"Provide.Enabled\", true)\n\tnode.SetIPFSConfig(\"Bootstrap\", []string{})\n\n\tkeystoreBase := filepath.Join(node.Dir, \"provider-keystore\")\n\n\t// Pre-seed orphaned keystore data into the shared datastore, simulating\n\t// the layout produced by older Kubo that stored keystore entries inline.\n\tconst numOrphans = 10\n\tfor i := range numOrphans {\n\t\tnode.DatastorePut(\n\t\t\tfmt.Sprintf(\"/provider/keystore/%d/fake-key-%d\", i%2, i),\n\t\t\tfmt.Sprintf(\"orphan-%d\", i),\n\t\t)\n\t}\n\n\t// The orphaned keys should be visible via diag datastore.\n\tcount := node.DatastoreCount(\"/provider/keystore/\")\n\trequire.Equal(t, int64(numOrphans), count, \"orphaned keys should be present before migration\")\n\n\t// The provider-keystore directory must not exist yet (its absence\n\t// triggers the migration).\n\trequire.False(t, dirExists(keystoreBase),\n\t\t\"provider-keystore/ should not exist before first sweep-enabled start\")\n\n\t// Start the daemon: this triggers the one-time migration purge.\n\tnode.StartDaemon()\n\tnode.StopDaemon()\n\n\t// After migration the seeded orphaned keys should be gone from the\n\t// shared datastore. The diag datastore count command mounts the\n\t// separate provider-keystore datastores, so we check for the specific\n\t// fake keys we seeded to confirm they were purged.\n\tfor i := range numOrphans {\n\t\tkey := fmt.Sprintf(\"/provider/keystore/%d/fake-key-%d\", i%2, i)\n\t\tassert.False(t, node.DatastoreHasKey(key),\n\t\t\t\"orphaned key %s should be purged after migration\", key)\n\t}\n\n\t// The provider-keystore directory should now exist.\n\tassert.True(t, dirExists(keystoreBase),\n\t\t\"provider-keystore/ should exist after sweep-enabled daemon ran\")\n}\n\nfunc dirExists(path string) bool {\n\tinfo, err := os.Stat(path)\n\treturn err == nil && info.IsDir()\n}\n" }, { "path": "test/cli/pubsub_test.go", "content": "package cli\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"slices\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// waitForSubscription waits until the node has a subscription to the given topic.\nfunc waitForSubscription(t *testing.T, node *harness.Node, topic string) {\n\tt.Helper()\n\trequire.Eventually(t, func() bool {\n\t\tres := node.RunIPFS(\"pubsub\", \"ls\")\n\t\tif res.Err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn slices.Contains(res.Stdout.Lines(), topic)\n\t}, 5*time.Second, 100*time.Millisecond, \"expected subscription to topic %s\", topic)\n}\n\n// waitForMessagePropagation waits for pubsub messages to propagate through the network\n// and for seqno state to be persisted to the datastore.\nfunc waitForMessagePropagation(t *testing.T) {\n\tt.Helper()\n\ttime.Sleep(1 * time.Second)\n}\n\n// publishMessages publishes n messages from publisher to the given topic with\n// a small delay between each to allow for ordered delivery.\nfunc publishMessages(t *testing.T, publisher *harness.Node, topic string, n int) {\n\tt.Helper()\n\tfor range n {\n\t\tpublisher.PipeStrToIPFS(\"msg\", \"pubsub\", \"pub\", topic)\n\t\ttime.Sleep(50 * time.Millisecond)\n\t}\n}\n\n// TestPubsub tests pubsub functionality and the persistent seqno validator.\n//\n// Pubsub has two deduplication layers:\n//\n// Layer 1: MessageID-based TimeCache (in-memory)\n// - Controlled by Pubsub.SeenMessagesTTL config (default 120s)\n// - Tested in go-libp2p-pubsub (see timecache in github.com/libp2p/go-libp2p-pubsub)\n// - Only tested implicitly here via message delivery (timing-sensitive, not practical for CLI tests)\n//\n// Layer 2: Per-peer seqno validator (persistent in datastore)\n// - Stores max seen seqno per peer at /pubsub/seqno/\n// - Tested directly below: persistence, updates, reset, survives restart\n// - Validator: go-libp2p-pubsub BasicSeqnoValidator\nfunc TestPubsub(t *testing.T) {\n\tt.Parallel()\n\n\t// enablePubsub configures a node with pubsub enabled\n\tenablePubsub := func(n *harness.Node) {\n\t\tn.SetIPFSConfig(\"Pubsub.Enabled\", true)\n\t\tn.SetIPFSConfig(\"Routing.Type\", \"none\") // simplify test setup\n\t}\n\n\tt.Run(\"basic pub/sub message delivery\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// Create two connected nodes with pubsub enabled\n\t\tnodes := h.NewNodes(2).Init()\n\t\tnodes.ForEachPar(enablePubsub)\n\t\tnodes = nodes.StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\n\t\tsubscriber := nodes[0]\n\t\tpublisher := nodes[1]\n\n\t\tconst topic = \"test-topic\"\n\t\tconst message = \"hello pubsub\"\n\n\t\t// Start subscriber in background\n\t\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\t\tdefer cancel()\n\n\t\t// Use a channel to receive the message\n\t\tmsgChan := make(chan string, 1)\n\t\tgo func() {\n\t\t\t// Subscribe and wait for one message\n\t\t\tres := subscriber.RunIPFS(\"pubsub\", \"sub\", \"--enc=json\", topic)\n\t\t\tif res.Err == nil {\n\t\t\t\t// Parse JSON output to get message data\n\t\t\t\tlines := res.Stdout.Lines()\n\t\t\t\tif len(lines) > 0 {\n\t\t\t\t\tvar msg struct {\n\t\t\t\t\t\tData []byte `json:\"data\"`\n\t\t\t\t\t}\n\t\t\t\t\tif json.Unmarshal([]byte(lines[0]), &msg) == nil {\n\t\t\t\t\t\tmsgChan <- string(msg.Data)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\t// Wait for subscriber to be ready\n\t\twaitForSubscription(t, subscriber, topic)\n\n\t\t// Publish message\n\t\tpublisher.PipeStrToIPFS(message, \"pubsub\", \"pub\", topic)\n\n\t\t// Wait for message or timeout\n\t\tselect {\n\t\tcase received := <-msgChan:\n\t\t\tassert.Equal(t, message, received)\n\t\tcase <-ctx.Done():\n\t\t\t// Subscriber may not receive in time due to test timing - that's OK\n\t\t\t// The main goal is to test the seqno validator state persistence\n\t\t\tt.Log(\"subscriber did not receive message in time (this is acceptable)\")\n\t\t}\n\t})\n\n\tt.Run(\"seqno validator state is persisted\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// Create two connected nodes with pubsub\n\t\tnodes := h.NewNodes(2).Init()\n\t\tnodes.ForEachPar(enablePubsub)\n\t\tnodes = nodes.StartDaemons().Connect()\n\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\t\tnode2PeerID := node2.PeerID().String()\n\n\t\tconst topic = \"seqno-test\"\n\n\t\t// Start subscriber on node1\n\t\tgo func() {\n\t\t\tnode1.RunIPFS(\"pubsub\", \"sub\", topic)\n\t\t}()\n\t\twaitForSubscription(t, node1, topic)\n\n\t\t// Publish multiple messages from node2 to trigger seqno validation\n\t\tpublishMessages(t, node2, topic, 3)\n\n\t\t// Wait for messages to propagate and seqno to be stored\n\t\twaitForMessagePropagation(t)\n\n\t\t// Stop daemons to check datastore (diag datastore requires daemon to be stopped)\n\t\tnodes.StopDaemons()\n\n\t\t// Check that seqno state exists\n\t\tcount := node1.DatastoreCount(\"/pubsub/seqno/\")\n\t\tt.Logf(\"seqno entries count: %d\", count)\n\n\t\t// There should be at least one seqno entry (from node2)\n\t\tassert.NotEqual(t, int64(0), count, \"expected seqno state to be persisted\")\n\n\t\t// Verify the specific peer's key exists and test --hex output format\n\t\tkey := \"/pubsub/seqno/\" + node2PeerID\n\t\tres := node1.RunIPFS(\"diag\", \"datastore\", \"get\", \"--hex\", key)\n\t\tif res.Err == nil {\n\t\t\tt.Logf(\"seqno for peer %s:\\n%s\", node2PeerID, res.Stdout.String())\n\t\t\tassert.Contains(t, res.Stdout.String(), \"Hex Dump:\")\n\t\t} else {\n\t\t\t// Key might not exist if messages didn't propagate - log but don't fail\n\t\t\tt.Logf(\"seqno key not found for peer %s (messages may not have propagated)\", node2PeerID)\n\t\t}\n\t})\n\n\tt.Run(\"seqno updates when receiving multiple messages\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// Create two connected nodes with pubsub\n\t\tnodes := h.NewNodes(2).Init()\n\t\tnodes.ForEachPar(enablePubsub)\n\t\tnodes = nodes.StartDaemons().Connect()\n\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\t\tnode2PeerID := node2.PeerID().String()\n\n\t\tconst topic = \"seqno-update-test\"\n\t\tseqnoKey := \"/pubsub/seqno/\" + node2PeerID\n\n\t\t// Start subscriber on node1\n\t\tgo func() {\n\t\t\tnode1.RunIPFS(\"pubsub\", \"sub\", topic)\n\t\t}()\n\t\twaitForSubscription(t, node1, topic)\n\n\t\t// Send first message\n\t\tnode2.PipeStrToIPFS(\"msg1\", \"pubsub\", \"pub\", topic)\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Stop daemons to check seqno (diag datastore requires daemon to be stopped)\n\t\tnodes.StopDaemons()\n\n\t\t// Get seqno after first message\n\t\tres1 := node1.RunIPFS(\"diag\", \"datastore\", \"get\", seqnoKey)\n\t\tvar seqno1 []byte\n\t\tif res1.Err == nil {\n\t\t\tseqno1 = res1.Stdout.Bytes()\n\t\t\tt.Logf(\"seqno after first message: %d bytes\", len(seqno1))\n\t\t} else {\n\t\t\tt.Logf(\"seqno not found after first message (message may not have propagated)\")\n\t\t}\n\n\t\t// Restart daemons for second message\n\t\tnodes = nodes.StartDaemons().Connect()\n\n\t\t// Resubscribe\n\t\tgo func() {\n\t\t\tnode1.RunIPFS(\"pubsub\", \"sub\", topic)\n\t\t}()\n\t\twaitForSubscription(t, node1, topic)\n\n\t\t// Send second message\n\t\tnode2.PipeStrToIPFS(\"msg2\", \"pubsub\", \"pub\", topic)\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Stop daemons to check seqno\n\t\tnodes.StopDaemons()\n\n\t\t// Get seqno after second message\n\t\tres2 := node1.RunIPFS(\"diag\", \"datastore\", \"get\", seqnoKey)\n\t\tvar seqno2 []byte\n\t\tif res2.Err == nil {\n\t\t\tseqno2 = res2.Stdout.Bytes()\n\t\t\tt.Logf(\"seqno after second message: %d bytes\", len(seqno2))\n\t\t} else {\n\t\t\tt.Logf(\"seqno not found after second message\")\n\t\t}\n\n\t\t// If both messages were received, seqno should have been updated\n\t\t// The seqno is a uint64 that should increase with each message\n\t\tif len(seqno1) > 0 && len(seqno2) > 0 {\n\t\t\t// seqno2 should be >= seqno1 (it's the max seen seqno)\n\t\t\t// We just verify they're both non-empty and potentially different\n\t\t\tt.Logf(\"seqno1: %x\", seqno1)\n\t\t\tt.Logf(\"seqno2: %x\", seqno2)\n\t\t\t// The seqno validator stores the max seqno seen, so seqno2 >= seqno1\n\t\t\t// We can't do a simple byte comparison due to potential endianness\n\t\t\t// but both should be valid uint64 values (8 bytes)\n\t\t\tassert.Equal(t, 8, len(seqno2), \"seqno should be 8 bytes (uint64)\")\n\t\t}\n\t})\n\n\tt.Run(\"pubsub reset clears seqno state\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// Create two connected nodes\n\t\tnodes := h.NewNodes(2).Init()\n\t\tnodes.ForEachPar(enablePubsub)\n\t\tnodes = nodes.StartDaemons().Connect()\n\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\n\t\tconst topic = \"reset-test\"\n\n\t\t// Start subscriber and exchange messages\n\t\tgo func() {\n\t\t\tnode1.RunIPFS(\"pubsub\", \"sub\", topic)\n\t\t}()\n\t\twaitForSubscription(t, node1, topic)\n\n\t\tpublishMessages(t, node2, topic, 3)\n\t\twaitForMessagePropagation(t)\n\n\t\t// Stop daemons to check initial count\n\t\tnodes.StopDaemons()\n\n\t\t// Verify there is state before resetting\n\t\tinitialCount := node1.DatastoreCount(\"/pubsub/seqno/\")\n\t\tt.Logf(\"initial seqno count: %d\", initialCount)\n\n\t\t// Restart node1 to run pubsub reset\n\t\tnode1.StartDaemon()\n\n\t\t// Reset all seqno state (while daemon is running)\n\t\tres := node1.IPFS(\"pubsub\", \"reset\")\n\t\tassert.NoError(t, res.Err)\n\t\tt.Logf(\"reset output: %s\", res.Stdout.String())\n\n\t\t// Stop daemon to verify state was cleared\n\t\tnode1.StopDaemon()\n\n\t\t// Verify state was cleared\n\t\tfinalCount := node1.DatastoreCount(\"/pubsub/seqno/\")\n\t\tt.Logf(\"final seqno count: %d\", finalCount)\n\t\tassert.Equal(t, int64(0), finalCount, \"seqno state should be cleared after reset\")\n\t})\n\n\tt.Run(\"pubsub reset with peer flag\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// Create three connected nodes\n\t\tnodes := h.NewNodes(3).Init()\n\t\tnodes.ForEachPar(enablePubsub)\n\t\tnodes = nodes.StartDaemons().Connect()\n\n\t\tnode1 := nodes[0]\n\t\tnode2 := nodes[1]\n\t\tnode3 := nodes[2]\n\t\tnode2PeerID := node2.PeerID().String()\n\t\tnode3PeerID := node3.PeerID().String()\n\n\t\tconst topic = \"peer-reset-test\"\n\n\t\t// Start subscriber on node1\n\t\tgo func() {\n\t\t\tnode1.RunIPFS(\"pubsub\", \"sub\", topic)\n\t\t}()\n\t\twaitForSubscription(t, node1, topic)\n\n\t\t// Publish from both node2 and node3\n\t\tfor range 3 {\n\t\t\tnode2.PipeStrToIPFS(\"msg2\", \"pubsub\", \"pub\", topic)\n\t\t\tnode3.PipeStrToIPFS(\"msg3\", \"pubsub\", \"pub\", topic)\n\t\t\ttime.Sleep(50 * time.Millisecond)\n\t\t}\n\t\twaitForMessagePropagation(t)\n\n\t\t// Stop node2 and node3\n\t\tnode2.StopDaemon()\n\t\tnode3.StopDaemon()\n\n\t\t// Reset only node2's state (while node1 daemon is running)\n\t\tres := node1.IPFS(\"pubsub\", \"reset\", \"--peer\", node2PeerID)\n\t\trequire.NoError(t, res.Err)\n\t\tt.Logf(\"reset output: %s\", res.Stdout.String())\n\n\t\t// Stop node1 daemon to check datastore\n\t\tnode1.StopDaemon()\n\n\t\t// Check that node2's key is gone\n\t\tres = node1.RunIPFS(\"diag\", \"datastore\", \"get\", \"/pubsub/seqno/\"+node2PeerID)\n\t\tassert.Error(t, res.Err, \"node2's seqno key should be deleted\")\n\n\t\t// Check that node3's key still exists (if it was created)\n\t\tres = node1.RunIPFS(\"diag\", \"datastore\", \"get\", \"/pubsub/seqno/\"+node3PeerID)\n\t\t// Note: node3's key might not exist if messages didn't propagate\n\t\t// So we just log the result without asserting\n\t\tif res.Err == nil {\n\t\t\tt.Logf(\"node3's seqno key still exists (as expected)\")\n\t\t} else {\n\t\t\tt.Logf(\"node3's seqno key not found (messages may not have propagated)\")\n\t\t}\n\t})\n\n\tt.Run(\"seqno state survives daemon restart\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\th := harness.NewT(t)\n\n\t\t// Create and start single node\n\t\tnode := h.NewNode().Init()\n\t\tenablePubsub(node)\n\t\tnode.StartDaemon()\n\n\t\t// We need another node to publish messages\n\t\tnode2 := h.NewNode().Init()\n\t\tenablePubsub(node2)\n\t\tnode2.StartDaemon()\n\t\tnode.Connect(node2)\n\n\t\tconst topic = \"restart-test\"\n\n\t\t// Start subscriber and exchange messages\n\t\tgo func() {\n\t\t\tnode.RunIPFS(\"pubsub\", \"sub\", topic)\n\t\t}()\n\t\twaitForSubscription(t, node, topic)\n\n\t\tpublishMessages(t, node2, topic, 3)\n\t\twaitForMessagePropagation(t)\n\n\t\t// Stop daemons to check datastore\n\t\tnode.StopDaemon()\n\t\tnode2.StopDaemon()\n\n\t\t// Get count before restart\n\t\tbeforeCount := node.DatastoreCount(\"/pubsub/seqno/\")\n\t\tt.Logf(\"seqno count before restart: %d\", beforeCount)\n\n\t\t// Restart node (simulate restart scenario)\n\t\tnode.StartDaemon()\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Stop daemon to check datastore again\n\t\tnode.StopDaemon()\n\n\t\t// Get count after restart\n\t\tafterCount := node.DatastoreCount(\"/pubsub/seqno/\")\n\t\tt.Logf(\"seqno count after restart: %d\", afterCount)\n\n\t\t// Count should be the same (state persisted)\n\t\tassert.Equal(t, beforeCount, afterCount, \"seqno state should survive daemon restart\")\n\t})\n}\n" }, { "path": "test/cli/rcmgr_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/protocol\"\n\trcmgr \"github.com/libp2p/go-libp2p/p2p/host/resource-manager\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRcmgr(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"Resource manager disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Swarm.ResourceMgr.Enabled = config.False\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tt.Run(\"swarm resources should fail\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"swarm\", \"resources\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"missing ResourceMgr\")\n\t\t})\n\t})\n\n\tt.Run(\"Node with resource manager disabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Swarm.ResourceMgr.Enabled = config.False\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tt.Run(\"swarm resources should fail\", func(t *testing.T) {\n\t\t\tres := node.RunIPFS(\"swarm\", \"resources\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"missing ResourceMgr\")\n\t\t})\n\t})\n\n\tt.Run(\"Very high connmgr highwater\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Swarm.ConnMgr.HighWater = config.NewOptionalInteger(1000)\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\trequire.Equal(t, 0, res.ExitCode())\n\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\n\t\trl := limits.System.ToResourceLimits()\n\t\ts := rl.Build(rcmgr.BaseLimit{})\n\t\tassert.GreaterOrEqual(t, s.ConnsInbound, 2000)\n\t\tassert.GreaterOrEqual(t, s.StreamsInbound, 2000)\n\t})\n\n\tt.Run(\"default configuration\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Swarm.ConnMgr.HighWater = config.NewOptionalInteger(1000)\n\t\t})\n\n\t\tnode.StartDaemon()\n\t\tt.Cleanup(func() { node.StopDaemon() })\n\n\t\tt.Run(\"conns and streams are above 800 for default connmgr settings\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\t\trequire.Equal(t, 0, res.ExitCode())\n\t\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\n\t\t\tif limits.System.ConnsInbound > rcmgr.DefaultLimit {\n\t\t\t\tassert.GreaterOrEqual(t, limits.System.ConnsInbound, 800)\n\t\t\t}\n\t\t\tif limits.System.StreamsInbound > rcmgr.DefaultLimit {\n\t\t\t\tassert.GreaterOrEqual(t, limits.System.StreamsInbound, 800)\n\t\t\t}\n\t\t})\n\n\t\tt.Run(\"limits should succeed\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\t\tlimits := rcmgr.PartialLimitConfig{}\n\t\t\terr := json.Unmarshal(res.Stdout.Bytes(), &limits)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.NotEqual(t, limits.Transient.Memory, rcmgr.BlockAllLimit64)\n\t\t\tassert.NotEqual(t, limits.System.Memory, rcmgr.BlockAllLimit64)\n\t\t\tassert.NotEqual(t, limits.System.FD, rcmgr.BlockAllLimit)\n\t\t\tassert.NotEqual(t, limits.System.Conns, rcmgr.BlockAllLimit)\n\t\t\tassert.NotEqual(t, limits.System.ConnsInbound, rcmgr.BlockAllLimit)\n\t\t\tassert.NotEqual(t, limits.System.ConnsOutbound, rcmgr.BlockAllLimit)\n\t\t\tassert.NotEqual(t, limits.System.Streams, rcmgr.BlockAllLimit)\n\t\t\tassert.NotEqual(t, limits.System.StreamsInbound, rcmgr.BlockAllLimit)\n\t\t\tassert.NotEqual(t, limits.System.StreamsOutbound, rcmgr.BlockAllLimit)\n\t\t})\n\n\t\tt.Run(\"swarm stats works\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\t\trequire.Equal(t, 0, res.ExitCode())\n\n\t\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\n\t\t\t// every scope has the same fields, so we only inspect system\n\t\t\tassert.Zero(t, limits.System.MemoryUsage)\n\t\t\tassert.Zero(t, limits.System.FDUsage)\n\t\t\tassert.Zero(t, limits.System.ConnsInboundUsage)\n\t\t\tassert.Zero(t, limits.System.ConnsOutboundUsage)\n\t\t\tassert.Zero(t, limits.System.StreamsInboundUsage)\n\t\t\tassert.Zero(t, limits.System.StreamsOutboundUsage)\n\t\t\tassert.Zero(t, limits.Transient.MemoryUsage)\n\t\t})\n\t})\n\n\tt.Run(\"smoke test unlimited System inbounds\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\toverrides.System.StreamsInbound = rcmgr.Unlimited\n\t\t\toverrides.System.ConnsInbound = rcmgr.Unlimited\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\n\t\tassert.Equal(t, rcmgr.Unlimited, limits.System.ConnsInbound)\n\t\tassert.Equal(t, rcmgr.Unlimited, limits.System.StreamsInbound)\n\t})\n\n\tt.Run(\"smoke test transient scope\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\toverrides.Transient.Memory = 88888\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\t\tassert.Equal(t, rcmgr.LimitVal64(88888), limits.Transient.Memory)\n\t})\n\n\tt.Run(\"smoke test service scope\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\toverrides.Service = map[string]rcmgr.ResourceLimits{\"foo\": {Memory: 77777}}\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\t\tassert.Equal(t, rcmgr.LimitVal64(77777), limits.Services[\"foo\"].Memory)\n\t})\n\n\tt.Run(\"smoke test protocol scope\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\toverrides.Protocol = map[protocol.ID]rcmgr.ResourceLimits{\"foo\": {Memory: 66666}}\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\t\tassert.Equal(t, rcmgr.LimitVal64(66666), limits.Protocols[\"foo\"].Memory)\n\t})\n\n\tt.Run(\"smoke test peer scope\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tvalidPeerID, err := peer.Decode(\"QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\")\n\t\tassert.NoError(t, err)\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\toverrides.Peer = map[peer.ID]rcmgr.ResourceLimits{validPeerID: {Memory: 55555}}\n\t\t})\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"resources\", \"--enc=json\")\n\t\tlimits := unmarshalLimits(t, res.Stdout.Bytes())\n\t\tassert.Equal(t, rcmgr.LimitVal64(55555), limits.Peers[validPeerID].Memory)\n\t})\n\n\tt.Run(\"blocking and allowlists\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(3).Init()\n\t\tnode0, node1, node2 := nodes[0], nodes[1], nodes[2]\n\t\tpeerID1, peerID2 := node1.PeerID().String(), node2.PeerID().String()\n\n\t\tnode0.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Swarm.ResourceMgr.Enabled = config.True\n\t\t\tcfg.Swarm.ResourceMgr.Allowlist = []string{\"/ip4/0.0.0.0/ipcidr/0/p2p/\" + peerID2}\n\t\t})\n\t\tnode0.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\t*overrides = rcmgr.PartialLimitConfig{\n\t\t\t\tSystem: rcmgr.ResourceLimits{\n\t\t\t\t\tConns: rcmgr.BlockAllLimit,\n\t\t\t\t\tConnsInbound: rcmgr.BlockAllLimit,\n\t\t\t\t\tConnsOutbound: rcmgr.BlockAllLimit,\n\t\t\t\t},\n\t\t\t}\n\t\t})\n\n\t\tnodes.StartDaemons()\n\t\tt.Cleanup(func() { nodes.StopDaemons() })\n\n\t\tt.Run(\"node 0 should fail to connect to and ping node 1\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node0.Runner.Run(harness.RunRequest{\n\t\t\t\tPath: node0.IPFSBin,\n\t\t\t\tArgs: []string{\"swarm\", \"connect\", node1.SwarmAddrsWithPeerIDs()[0].String()},\n\t\t\t})\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\ttestutils.AssertStringContainsOneOf(t, res.Stderr.String(),\n\t\t\t\t\"failed to find any peer in table\",\n\t\t\t\t\"resource limit exceeded\",\n\t\t\t)\n\n\t\t\tres = node0.RunIPFS(\"ping\", \"-n2\", peerID1)\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\tassert.Contains(t, res.Stderr.String(), \"Error: ping failed\")\n\t\t})\n\n\t\tt.Run(\"node 0 should connect to and ping node 2 since it is allowlisted\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := node0.Runner.Run(harness.RunRequest{\n\t\t\t\tPath: node0.IPFSBin,\n\t\t\t\tArgs: []string{\"swarm\", \"connect\", node2.SwarmAddrsWithPeerIDs()[0].String()},\n\t\t\t})\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\t\tres = node0.RunIPFS(\"ping\", \"-n2\", peerID2)\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t})\n\t})\n\n\tt.Run(\"daemon should refuse to start if connmgr.highwater < resources inbound\", func(t *testing.T) {\n\t\tt.Run(\"system conns\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Swarm.ConnMgr.HighWater = config.NewOptionalInteger(128)\n\t\t\t\tcfg.Swarm.ConnMgr.LowWater = config.NewOptionalInteger(64)\n\t\t\t})\n\t\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\t\t*overrides = rcmgr.PartialLimitConfig{\n\t\t\t\t\tSystem: rcmgr.ResourceLimits{Conns: 128},\n\t\t\t\t}\n\t\t\t})\n\n\t\t\tres := node.RunIPFS(\"daemon\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t})\n\t\tt.Run(\"system conns inbound\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Swarm.ConnMgr.HighWater = config.NewOptionalInteger(128)\n\t\t\t\tcfg.Swarm.ConnMgr.LowWater = config.NewOptionalInteger(64)\n\t\t\t})\n\t\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\t\t*overrides = rcmgr.PartialLimitConfig{\n\t\t\t\t\tSystem: rcmgr.ResourceLimits{ConnsInbound: 128},\n\t\t\t\t}\n\t\t\t})\n\n\t\t\tres := node.RunIPFS(\"daemon\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t})\n\t\tt.Run(\"system streams\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Swarm.ConnMgr.HighWater = config.NewOptionalInteger(128)\n\t\t\t\tcfg.Swarm.ConnMgr.LowWater = config.NewOptionalInteger(64)\n\t\t\t})\n\t\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\t\t*overrides = rcmgr.PartialLimitConfig{\n\t\t\t\t\tSystem: rcmgr.ResourceLimits{Streams: 128},\n\t\t\t\t}\n\t\t\t})\n\n\t\t\tres := node.RunIPFS(\"daemon\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t})\n\t\tt.Run(\"system streams inbound\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\t\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Swarm.ConnMgr.HighWater = config.NewOptionalInteger(128)\n\t\t\t\tcfg.Swarm.ConnMgr.LowWater = config.NewOptionalInteger(64)\n\t\t\t})\n\t\t\tnode.UpdateUserSuppliedResourceManagerOverrides(func(overrides *rcmgr.PartialLimitConfig) {\n\t\t\t\t*overrides = rcmgr.PartialLimitConfig{\n\t\t\t\t\tSystem: rcmgr.ResourceLimits{StreamsInbound: 128},\n\t\t\t\t}\n\t\t\t})\n\n\t\t\tres := node.RunIPFS(\"daemon\")\n\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t})\n\t})\n}\n\nfunc unmarshalLimits(t *testing.T, b []byte) *libp2p.LimitsConfigAndUsage {\n\tlimits := &libp2p.LimitsConfigAndUsage{}\n\terr := json.Unmarshal(b, limits)\n\trequire.NoError(t, err)\n\treturn limits\n}\n" }, { "path": "test/cli/repo_verify_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// Well-known block file names in flatfs blockstore that should not be corrupted during testing.\n// Flatfs stores each block as a separate .data file on disk.\nconst (\n\t// emptyFileFlatfsFilename is the flatfs filename for an empty UnixFS file block\n\temptyFileFlatfsFilename = \"CIQL7TG2PB52XIZLLHDYIUFMHUQLMMZWBNBZSLDXFCPZ5VDNQQ2WDZQ\"\n\t// emptyDirFlatfsFilename is the flatfs filename for an empty UnixFS directory block.\n\t// This block has special handling and may be served from memory even when corrupted on disk.\n\temptyDirFlatfsFilename = \"CIQFTFEEHEDF6KLBT32BFAGLXEZL4UWFNWM4LFTLMXQBCERZ6CMLX3Y\"\n)\n\n// getEligibleFlatfsBlockFiles returns flatfs block files (*.data) that are safe to corrupt in tests.\n// Filters out well-known blocks (empty file/dir) that cause test flakiness.\n//\n// Note: This helper is specific to the flatfs blockstore implementation where each block\n// is stored as a separate file on disk under blocks/*/*.data.\nfunc getEligibleFlatfsBlockFiles(t *testing.T, node *harness.Node) []string {\n\tblockFiles, err := filepath.Glob(filepath.Join(node.Dir, \"blocks\", \"*\", \"*.data\"))\n\trequire.NoError(t, err)\n\trequire.NotEmpty(t, blockFiles, \"no flatfs block files found\")\n\n\tvar eligible []string\n\tfor _, f := range blockFiles {\n\t\tname := filepath.Base(f)\n\t\tif !strings.Contains(name, emptyFileFlatfsFilename) &&\n\t\t\t!strings.Contains(name, emptyDirFlatfsFilename) {\n\t\t\teligible = append(eligible, f)\n\t\t}\n\t}\n\treturn eligible\n}\n\n// corruptRandomBlock corrupts a random block file in the flatfs blockstore.\n// Returns the path to the corrupted file.\nfunc corruptRandomBlock(t *testing.T, node *harness.Node) string {\n\teligible := getEligibleFlatfsBlockFiles(t, node)\n\trequire.NotEmpty(t, eligible, \"no eligible blocks to corrupt\")\n\n\ttoCorrupt := eligible[0]\n\terr := os.WriteFile(toCorrupt, []byte(\"corrupted data\"), 0644)\n\trequire.NoError(t, err)\n\n\treturn toCorrupt\n}\n\n// corruptMultipleBlocks corrupts multiple block files in the flatfs blockstore.\n// Returns the paths to the corrupted files.\nfunc corruptMultipleBlocks(t *testing.T, node *harness.Node, count int) []string {\n\teligible := getEligibleFlatfsBlockFiles(t, node)\n\trequire.GreaterOrEqual(t, len(eligible), count, \"not enough eligible blocks to corrupt\")\n\n\tvar corrupted []string\n\tfor i := 0; i < count && i < len(eligible); i++ {\n\t\terr := os.WriteFile(eligible[i], fmt.Appendf(nil, \"corrupted data %d\", i), 0644)\n\t\trequire.NoError(t, err)\n\t\tcorrupted = append(corrupted, eligible[i])\n\t}\n\n\treturn corrupted\n}\n\nfunc TestRepoVerify(t *testing.T) {\n\tt.Run(\"healthy repo passes\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFS(\"add\", \"-q\", \"--raw-leaves=false\", \"-r\", node.IPFSBin)\n\n\t\tres := node.IPFS(\"repo\", \"verify\")\n\t\tassert.Contains(t, res.Stdout.String(), \"all blocks validated\")\n\t})\n\n\tt.Run(\"detects corruption\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFSAddStr(\"test content\")\n\n\t\tcorruptRandomBlock(t, node)\n\n\t\tres := node.RunIPFS(\"repo\", \"verify\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\tassert.Contains(t, res.Stdout.String(), \"was corrupt\")\n\t\tassert.Contains(t, res.Stderr.String(), \"1 blocks corrupt\")\n\t})\n\n\tt.Run(\"drop removes corrupt blocks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tcid := node.IPFSAddStr(\"test content\")\n\n\t\tcorruptRandomBlock(t, node)\n\n\t\tres := node.RunIPFS(\"repo\", \"verify\", \"--drop\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"should exit 0 when all corrupt blocks removed successfully\")\n\t\toutput := res.Stdout.String()\n\t\tassert.Contains(t, output, \"1 blocks corrupt\")\n\t\tassert.Contains(t, output, \"1 removed\")\n\n\t\t// Verify block is gone\n\t\tres = node.RunIPFS(\"block\", \"stat\", cid)\n\t\tassert.NotEqual(t, 0, res.ExitCode())\n\t})\n\n\tt.Run(\"heal requires online mode\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.IPFSAddStr(\"test content\")\n\n\t\tcorruptRandomBlock(t, node)\n\n\t\tres := node.RunIPFS(\"repo\", \"verify\", \"--heal\")\n\t\tassert.NotEqual(t, 0, res.ExitCode())\n\t\tassert.Contains(t, res.Stderr.String(), \"online mode\")\n\t})\n\n\tt.Run(\"heal repairs from network\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add content to node 0\n\t\tcid := nodes[0].IPFSAddStr(\"test content for healing\")\n\n\t\t// Wait for it to appear on node 1\n\t\tnodes[1].IPFS(\"block\", \"get\", cid)\n\n\t\t// Corrupt on node 1\n\t\tcorruptRandomBlock(t, nodes[1])\n\n\t\t// Heal should restore from node 0\n\t\tres := nodes[1].RunIPFS(\"repo\", \"verify\", \"--heal\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"should exit 0 when all corrupt blocks healed successfully\")\n\t\toutput := res.Stdout.String()\n\n\t\t// Should report corruption and healing with specific counts\n\t\tassert.Contains(t, output, \"1 blocks corrupt\")\n\t\tassert.Contains(t, output, \"1 removed\")\n\t\tassert.Contains(t, output, \"1 healed\")\n\n\t\t// Verify block is restored\n\t\tnodes[1].IPFS(\"block\", \"stat\", cid)\n\t})\n\n\tt.Run(\"healed blocks contain correct data\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add specific content to node 0\n\t\ttestContent := \"this is the exact content that should be healed correctly\"\n\t\tcid := nodes[0].IPFSAddStr(testContent)\n\n\t\t// Fetch to node 1 and verify the content is correct initially\n\t\tnodes[1].IPFS(\"block\", \"get\", cid)\n\t\tres := nodes[1].IPFS(\"cat\", cid)\n\t\tassert.Equal(t, testContent, res.Stdout.String())\n\n\t\t// Corrupt on node 1\n\t\tcorruptRandomBlock(t, nodes[1])\n\n\t\t// Heal the corruption\n\t\tres = nodes[1].RunIPFS(\"repo\", \"verify\", \"--heal\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"should exit 0 when all corrupt blocks healed successfully\")\n\t\toutput := res.Stdout.String()\n\t\tassert.Contains(t, output, \"1 blocks corrupt\")\n\t\tassert.Contains(t, output, \"1 removed\")\n\t\tassert.Contains(t, output, \"1 healed\")\n\n\t\t// Verify the healed content matches the original exactly\n\t\tres = nodes[1].IPFS(\"cat\", cid)\n\t\tassert.Equal(t, testContent, res.Stdout.String(), \"healed content should match original\")\n\n\t\t// Also verify via block get that the raw block data is correct\n\t\tblock0 := nodes[0].IPFS(\"block\", \"get\", cid)\n\t\tblock1 := nodes[1].IPFS(\"block\", \"get\", cid)\n\t\tassert.Equal(t, block0.Stdout.String(), block1.Stdout.String(), \"raw block data should match\")\n\t})\n\n\tt.Run(\"multiple corrupt blocks\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Create 20 blocks\n\t\tfor i := range 20 {\n\t\t\tnode.IPFSAddStr(strings.Repeat(\"test content \", i+1))\n\t\t}\n\n\t\t// Corrupt 5 blocks\n\t\tcorruptMultipleBlocks(t, node, 5)\n\n\t\t// Verify detects all corruptions\n\t\tres := node.RunIPFS(\"repo\", \"verify\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t// Error summary is in stderr\n\t\tassert.Contains(t, res.Stderr.String(), \"5 blocks corrupt\")\n\n\t\t// Test with --drop\n\t\tres = node.RunIPFS(\"repo\", \"verify\", \"--drop\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"should exit 0 when all corrupt blocks removed successfully\")\n\t\tassert.Contains(t, res.Stdout.String(), \"5 blocks corrupt\")\n\t\tassert.Contains(t, res.Stdout.String(), \"5 removed\")\n\t})\n\n\tt.Run(\"empty repository\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Verify empty repo passes\n\t\tres := node.IPFS(\"repo\", \"verify\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tassert.Contains(t, res.Stdout.String(), \"all blocks validated\")\n\n\t\t// Should work with --drop and --heal too\n\t\tres = node.IPFS(\"repo\", \"verify\", \"--drop\")\n\t\tassert.Equal(t, 0, res.ExitCode())\n\t\tassert.Contains(t, res.Stdout.String(), \"all blocks validated\")\n\t})\n\n\tt.Run(\"partial heal success\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\n\t\t// Start both nodes and connect them\n\t\tnodes.StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add 5 blocks to node 0, pin them to keep available\n\t\tcid1 := nodes[0].IPFSAddStr(\"content available for healing 1\")\n\t\tcid2 := nodes[0].IPFSAddStr(\"content available for healing 2\")\n\t\tcid3 := nodes[0].IPFSAddStr(\"content available for healing 3\")\n\t\tcid4 := nodes[0].IPFSAddStr(\"content available for healing 4\")\n\t\tcid5 := nodes[0].IPFSAddStr(\"content available for healing 5\")\n\n\t\t// Pin these on node 0 to ensure they stay available\n\t\tnodes[0].IPFS(\"pin\", \"add\", cid1)\n\t\tnodes[0].IPFS(\"pin\", \"add\", cid2)\n\t\tnodes[0].IPFS(\"pin\", \"add\", cid3)\n\t\tnodes[0].IPFS(\"pin\", \"add\", cid4)\n\t\tnodes[0].IPFS(\"pin\", \"add\", cid5)\n\n\t\t// Node 1 fetches these blocks\n\t\tnodes[1].IPFS(\"block\", \"get\", cid1)\n\t\tnodes[1].IPFS(\"block\", \"get\", cid2)\n\t\tnodes[1].IPFS(\"block\", \"get\", cid3)\n\t\tnodes[1].IPFS(\"block\", \"get\", cid4)\n\t\tnodes[1].IPFS(\"block\", \"get\", cid5)\n\n\t\t// Now remove some blocks from node 0 to simulate partial availability\n\t\tnodes[0].IPFS(\"pin\", \"rm\", cid3)\n\t\tnodes[0].IPFS(\"pin\", \"rm\", cid4)\n\t\tnodes[0].IPFS(\"pin\", \"rm\", cid5)\n\t\tnodes[0].IPFS(\"repo\", \"gc\")\n\n\t\t// Verify node 1 is still connected\n\t\tpeers := nodes[1].IPFS(\"swarm\", \"peers\")\n\t\trequire.Contains(t, peers.Stdout.String(), nodes[0].PeerID().String())\n\n\t\t// Corrupt 5 blocks on node 1\n\t\tcorruptMultipleBlocks(t, nodes[1], 5)\n\n\t\t// Heal should partially succeed (only cid1 and cid2 available from node 0)\n\t\tres := nodes[1].RunIPFS(\"repo\", \"verify\", \"--heal\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\n\t\t// Should show mixed results with specific counts in stderr\n\t\terrOutput := res.Stderr.String()\n\t\tassert.Contains(t, errOutput, \"5 blocks corrupt\")\n\t\tassert.Contains(t, errOutput, \"5 removed\")\n\t\t// Only cid1 and cid2 are available for healing, cid3-5 were GC'd\n\t\tassert.Contains(t, errOutput, \"2 healed\")\n\t\tassert.Contains(t, errOutput, \"3 failed to heal\")\n\t})\n\n\tt.Run(\"heal with block not available on network\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\n\t\t// Start both nodes and connect\n\t\tnodes.StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\n\t\t// Add unique content only to node 1\n\t\tnodes[1].IPFSAddStr(\"unique content that exists nowhere else\")\n\n\t\t// Ensure nodes are connected\n\t\tpeers := nodes[1].IPFS(\"swarm\", \"peers\")\n\t\trequire.Contains(t, peers.Stdout.String(), nodes[0].PeerID().String())\n\n\t\t// Corrupt the block on node 1\n\t\tcorruptRandomBlock(t, nodes[1])\n\n\t\t// Heal should fail - node 0 doesn't have this content\n\t\tres := nodes[1].RunIPFS(\"repo\", \"verify\", \"--heal\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\n\t\t// Should report heal failure with specific counts in stderr\n\t\terrOutput := res.Stderr.String()\n\t\tassert.Contains(t, errOutput, \"1 blocks corrupt\")\n\t\tassert.Contains(t, errOutput, \"1 removed\")\n\t\tassert.Contains(t, errOutput, \"1 failed to heal\")\n\t})\n\n\tt.Run(\"large repository scale test\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Create 1000 small blocks\n\t\tfor i := range 1000 {\n\t\t\tnode.IPFSAddStr(fmt.Sprintf(\"content-%d\", i))\n\t\t}\n\n\t\t// Corrupt 10 blocks\n\t\tcorruptMultipleBlocks(t, node, 10)\n\n\t\t// Verify handles large repos efficiently\n\t\tres := node.RunIPFS(\"repo\", \"verify\")\n\t\tassert.Equal(t, 1, res.ExitCode())\n\n\t\t// Should report exactly 10 corrupt blocks in stderr\n\t\tassert.Contains(t, res.Stderr.String(), \"10 blocks corrupt\")\n\n\t\t// Test --drop at scale\n\t\tres = node.RunIPFS(\"repo\", \"verify\", \"--drop\")\n\t\tassert.Equal(t, 0, res.ExitCode(), \"should exit 0 when all corrupt blocks removed successfully\")\n\t\toutput := res.Stdout.String()\n\t\tassert.Contains(t, output, \"10 blocks corrupt\")\n\t\tassert.Contains(t, output, \"10 removed\")\n\t})\n\n\tt.Run(\"drop with partial removal failures\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t// Create several blocks\n\t\tfor i := range 5 {\n\t\t\tnode.IPFSAddStr(fmt.Sprintf(\"content for removal test %d\", i))\n\t\t}\n\n\t\t// Corrupt 3 blocks\n\t\tcorruptedFiles := corruptMultipleBlocks(t, node, 3)\n\t\trequire.Len(t, corruptedFiles, 3)\n\n\t\t// Make one of the corrupted files read-only to simulate removal failure\n\t\terr := os.Chmod(corruptedFiles[0], 0400) // read-only\n\t\trequire.NoError(t, err)\n\t\tdefer func() { _ = os.Chmod(corruptedFiles[0], 0644) }() // cleanup\n\n\t\t// Also make the directory read-only to prevent deletion\n\t\tblockDir := filepath.Dir(corruptedFiles[0])\n\t\toriginalPerm, err := os.Stat(blockDir)\n\t\trequire.NoError(t, err)\n\t\terr = os.Chmod(blockDir, 0500) // read+execute only, no write\n\t\trequire.NoError(t, err)\n\t\tdefer func() { _ = os.Chmod(blockDir, originalPerm.Mode()) }() // cleanup\n\n\t\t// Try to drop - should fail because at least one block can't be removed\n\t\tres := node.RunIPFS(\"repo\", \"verify\", \"--drop\")\n\t\tassert.Equal(t, 1, res.ExitCode(), \"should exit 1 when some blocks fail to remove\")\n\n\t\t// Restore permissions for verification\n\t\t_ = os.Chmod(blockDir, originalPerm.Mode())\n\t\t_ = os.Chmod(corruptedFiles[0], 0644)\n\n\t\t// Should report both successes and failures with specific counts\n\t\terrOutput := res.Stderr.String()\n\t\tassert.Contains(t, errOutput, \"3 blocks corrupt\")\n\t\tassert.Contains(t, errOutput, \"2 removed\")\n\t\tassert.Contains(t, errOutput, \"1 failed to remove\")\n\t})\n}\n" }, { "path": "test/cli/routing_dht_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc waitUntilProvidesComplete(t *testing.T, n *harness.Node) {\n\tgetCidsCount := func(line string) int {\n\t\ttrimmed := strings.TrimSpace(line)\n\t\tcountStr := strings.SplitN(trimmed, \" \", 2)[0]\n\t\tcount, err := strconv.Atoi(countStr)\n\t\trequire.NoError(t, err)\n\t\treturn count\n\t}\n\n\tqueuedProvides, ongoingProvides := true, true\n\tfor queuedProvides || ongoingProvides {\n\t\tres := n.IPFS(\"provide\", \"stat\", \"-a\")\n\t\trequire.NoError(t, res.Err)\n\t\tfor _, line := range res.Stdout.Lines() {\n\t\t\tif trimmed, ok := strings.CutPrefix(line, \" Provide queue:\"); ok {\n\t\t\t\tprovideQueueSize := getCidsCount(trimmed)\n\t\t\t\tqueuedProvides = provideQueueSize > 0\n\t\t\t}\n\t\t\tif trimmed, ok := strings.CutPrefix(line, \" Ongoing provides:\"); ok {\n\t\t\t\tongoingProvideCount := getCidsCount(trimmed)\n\t\t\t\tongoingProvides = ongoingProvideCount > 0\n\t\t\t}\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n}\n\nfunc testRoutingDHT(t *testing.T, enablePubsub bool) {\n\tt.Run(fmt.Sprintf(\"enablePubSub=%v\", enablePubsub), func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(5).Init()\n\t\tnodes.ForEachPar(func(node *harness.Node) {\n\t\t\tnode.IPFS(\"config\", \"Routing.Type\", \"dht\")\n\t\t})\n\n\t\tvar daemonArgs []string\n\t\tif enablePubsub {\n\t\t\tdaemonArgs = []string{\n\t\t\t\t\"--enable-pubsub-experiment\",\n\t\t\t\t\"--enable-namesys-pubsub\",\n\t\t\t}\n\t\t}\n\n\t\tnodes.StartDaemons(daemonArgs...).Connect()\n\t\tt.Cleanup(func() { nodes.StopDaemons() })\n\n\t\tt.Run(\"ipfs routing findpeer\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tres := nodes[1].RunIPFS(\"routing\", \"findpeer\", nodes[0].PeerID().String())\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\n\t\t\tswarmAddr := nodes[0].SwarmAddrsWithoutPeerIDs()[0]\n\t\t\trequire.Equal(t, swarmAddr.String(), res.Stdout.Trimmed())\n\t\t})\n\n\t\tt.Run(\"ipfs routing get \", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\thash := nodes[2].IPFSAddStr(\"hello world\")\n\t\t\tnodes[2].IPFS(\"name\", \"publish\", \"/ipfs/\"+hash)\n\n\t\t\tres := nodes[1].IPFS(\"routing\", \"get\", \"/ipns/\"+nodes[2].PeerID().String())\n\t\t\tassert.Contains(t, res.Stdout.String(), \"/ipfs/\"+hash)\n\n\t\t\tt.Run(\"put round trips (#3124)\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tnodes[0].WriteBytes(\"get_result\", res.Stdout.Bytes())\n\t\t\t\tres := nodes[0].IPFS(\"routing\", \"put\", \"/ipns/\"+nodes[2].PeerID().String(), \"get_result\")\n\t\t\t\tassert.Greater(t, len(res.Stdout.Lines()), 0, \"should put to at least one node\")\n\t\t\t})\n\n\t\t\tt.Run(\"put with bad keys fails (issue #5113, #4611)\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\tkeys := []string{\"foo\", \"/pk/foo\", \"/ipns/foo\"}\n\t\t\t\tfor _, key := range keys {\n\t\t\t\t\tt.Run(key, func(t *testing.T) {\n\t\t\t\t\t\tt.Parallel()\n\t\t\t\t\t\tres := nodes[0].RunIPFS(\"routing\", \"put\", key)\n\t\t\t\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t\t\t\tassert.Contains(t, res.Stderr.String(), \"invalid\")\n\t\t\t\t\t\tassert.Empty(t, res.Stdout.String())\n\t\t\t\t\t})\n\t\t\t\t}\n\t\t\t})\n\n\t\t\tt.Run(\"get with bad keys (issue #4611)\", func(t *testing.T) {\n\t\t\t\tfor _, key := range []string{\"foo\", \"/pk/foo\"} {\n\t\t\t\t\tt.Run(key, func(t *testing.T) {\n\t\t\t\t\t\tt.Parallel()\n\t\t\t\t\t\tres := nodes[0].RunIPFS(\"routing\", \"get\", key)\n\t\t\t\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t\t\t\tassert.Contains(t, res.Stderr.String(), \"invalid\")\n\t\t\t\t\t\tassert.Empty(t, res.Stdout.String())\n\t\t\t\t\t})\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\n\t\tt.Run(\"ipfs routing findprovs\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\thash := nodes[3].IPFSAddStr(\"some stuff\")\n\t\t\twaitUntilProvidesComplete(t, nodes[3])\n\t\t\tres := nodes[4].IPFS(\"routing\", \"findprovs\", hash)\n\t\t\tassert.Equal(t, nodes[3].PeerID().String(), res.Stdout.Trimmed())\n\t\t})\n\n\t\tt.Run(\"routing commands fail when offline\", func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\t\t// these cannot be run in parallel due to repo locking\n\t\t\t// this seems like a bug, we should be able to run these without locking the repo\n\n\t\t\tt.Run(\"routing findprovs\", func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"routing\", \"findprovs\", testutils.CIDEmptyDir)\n\t\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t\tassert.Contains(t, res.Stderr.String(), \"this command must be run in online mode\")\n\t\t\t})\n\n\t\t\tt.Run(\"routing findpeer\", func(t *testing.T) {\n\t\t\t\tres := node.RunIPFS(\"routing\", \"findpeer\", testutils.CIDEmptyDir)\n\t\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t\tassert.Contains(t, res.Stderr.String(), \"this command must be run in online mode\")\n\t\t\t})\n\n\t\t\tt.Run(\"routing put\", func(t *testing.T) {\n\t\t\t\tnode.WriteBytes(\"foo\", []byte(\"foo\"))\n\t\t\t\tres := node.RunIPFS(\"routing\", \"put\", \"/ipns/\"+node.PeerID().String(), \"foo\")\n\t\t\t\tassert.Equal(t, 1, res.ExitCode())\n\t\t\t\tassert.Contains(t, res.Stderr.String(), \"can't put while offline: pass `--allow-offline` to override\")\n\t\t\t})\n\t\t})\n\t})\n}\n\nfunc testSelfFindDHT(t *testing.T) {\n\tt.Run(\"ipfs routing findpeer fails for self\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(1).Init()\n\t\tnodes.ForEachPar(func(node *harness.Node) {\n\t\t\tnode.IPFS(\"config\", \"Routing.Type\", \"dht\")\n\t\t})\n\n\t\tnodes.StartDaemons()\n\t\tdefer nodes.StopDaemons()\n\n\t\tres := nodes[0].RunIPFS(\"dht\", \"findpeer\", nodes[0].PeerID().String())\n\t\tassert.Equal(t, 1, res.ExitCode())\n\t})\n}\n\nfunc TestRoutingDHT(t *testing.T) {\n\ttestRoutingDHT(t, false)\n\ttestRoutingDHT(t, true)\n\ttestSelfFindDHT(t)\n}\n" }, { "path": "test/cli/rpc_auth_test.go", "content": "package cli\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/client/rpc/auth\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst rpcDeniedMsg = \"Kubo RPC Access Denied: Please provide a valid authorization token as defined in the API.Authorizations configuration.\"\n\nfunc TestRPCAuth(t *testing.T) {\n\tt.Parallel()\n\n\tmakeAndStartProtectedNode := func(t *testing.T, authorizations map[string]*config.RPCAuthScope) *harness.Node {\n\t\tauthorizations[\"test-node-starter\"] = &config.RPCAuthScope{\n\t\t\tAuthSecret: \"bearer:test-node-starter\",\n\t\t\tAllowedPaths: []string{\"/api/v0\"},\n\t\t}\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.API.Authorizations = authorizations\n\t\t})\n\t\tnode.StartDaemonWithAuthorization(\"Bearer test-node-starter\")\n\t\treturn node\n\t}\n\n\tmakeHTTPTest := func(authSecret, header string) func(t *testing.T) {\n\t\treturn func(t *testing.T) {\n\t\t\tt.Parallel()\n\t\t\tt.Log(authSecret, header)\n\n\t\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\t\"userA\": {\n\t\t\t\t\tAuthSecret: authSecret,\n\t\t\t\t\tAllowedPaths: []string{\"/api/v0/id\"},\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tapiClient := node.APIClient()\n\t\t\tapiClient.Client = &http.Client{\n\t\t\t\tTransport: auth.NewAuthorizedRoundTripper(header, http.DefaultTransport),\n\t\t\t}\n\n\t\t\t// Can access /id with valid token\n\t\t\tresp := apiClient.Post(\"/api/v0/id\", nil)\n\t\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\t\t// But not /config/show\n\t\t\tresp = apiClient.Post(\"/api/v0/config/show\", nil)\n\t\t\tassert.Equal(t, 403, resp.StatusCode)\n\n\t\t\t// create client which sends invalid access token\n\t\t\tinvalidApiClient := node.APIClient()\n\t\t\tinvalidApiClient.Client = &http.Client{\n\t\t\t\tTransport: auth.NewAuthorizedRoundTripper(\"Bearer invalid\", http.DefaultTransport),\n\t\t\t}\n\n\t\t\t// Can't access /id with invalid token\n\t\t\terrResp := invalidApiClient.Post(\"/api/v0/id\", nil)\n\t\t\tassert.Equal(t, 403, errResp.StatusCode)\n\n\t\t\tnode.StopDaemon()\n\t\t}\n\t}\n\n\tmakeCLITest := func(authSecret string) func(t *testing.T) {\n\t\treturn func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\t\"userA\": {\n\t\t\t\t\tAuthSecret: authSecret,\n\t\t\t\t\tAllowedPaths: []string{\"/api/v0/id\"},\n\t\t\t\t},\n\t\t\t})\n\n\t\t\t// Can access 'ipfs id'\n\t\t\tresp := node.RunIPFS(\"id\", \"--api-auth\", authSecret)\n\t\t\trequire.NoError(t, resp.Err)\n\n\t\t\t// But not 'ipfs config show'\n\t\t\tresp = node.RunIPFS(\"config\", \"show\", \"--api-auth\", authSecret)\n\t\t\trequire.Error(t, resp.Err)\n\t\t\trequire.Contains(t, resp.Stderr.String(), rpcDeniedMsg)\n\n\t\t\tnode.StopDaemon()\n\t\t}\n\t}\n\n\tfor _, testCase := range []struct {\n\t\tname string\n\t\tauthSecret string\n\t\theader string\n\t}{\n\t\t{\"Bearer (no type)\", \"myToken\", \"Bearer myToken\"},\n\t\t{\"Bearer\", \"bearer:myToken\", \"Bearer myToken\"},\n\t\t{\"Basic (user:pass)\", \"basic:user:pass\", \"Basic dXNlcjpwYXNz\"},\n\t\t{\"Basic (encoded)\", \"basic:dXNlcjpwYXNz\", \"Basic dXNlcjpwYXNz\"},\n\t} {\n\t\tt.Run(\"AllowedPaths on CLI \"+testCase.name, makeCLITest(testCase.authSecret))\n\t\tt.Run(\"AllowedPaths on HTTP \"+testCase.name, makeHTTPTest(testCase.authSecret, testCase.header))\n\t}\n\n\tt.Run(\"AllowedPaths set to /api/v0 Gives Full Access\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\"userA\": {\n\t\t\t\tAuthSecret: \"bearer:userAToken\",\n\t\t\t\tAllowedPaths: []string{\"/api/v0\"},\n\t\t\t},\n\t\t})\n\n\t\tapiClient := node.APIClient()\n\t\tapiClient.Client = &http.Client{\n\t\t\tTransport: auth.NewAuthorizedRoundTripper(\"Bearer userAToken\", http.DefaultTransport),\n\t\t}\n\n\t\tresp := apiClient.Post(\"/api/v0/id\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"API.Authorizations set to nil disables Authorization header check\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.API.Authorizations = nil\n\t\t})\n\t\tnode.StartDaemon()\n\n\t\tapiClient := node.APIClient()\n\t\tresp := apiClient.Post(\"/api/v0/id\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"API.Authorizations set to empty map disables Authorization header check\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.API.Authorizations = map[string]*config.RPCAuthScope{}\n\t\t})\n\t\tnode.StartDaemon()\n\n\t\tapiClient := node.APIClient()\n\t\tresp := apiClient.Post(\"/api/v0/id\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"Requests without Authorization header are rejected when auth is enabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\"userA\": {\n\t\t\t\tAuthSecret: \"bearer:mytoken\",\n\t\t\t\tAllowedPaths: []string{\"/api/v0\"},\n\t\t\t},\n\t\t})\n\n\t\t// Create client with NO auth\n\t\tapiClient := node.APIClient() // Uses http.DefaultClient with no auth headers\n\n\t\t// Should be denied without auth header\n\t\tresp := apiClient.Post(\"/api/v0/id\", nil)\n\t\tassert.Equal(t, 403, resp.StatusCode)\n\n\t\t// Should contain denial message\n\t\tassert.Contains(t, resp.Body, rpcDeniedMsg)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"Version endpoint is always accessible even with limited AllowedPaths\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\"userA\": {\n\t\t\t\tAuthSecret: \"bearer:mytoken\",\n\t\t\t\tAllowedPaths: []string{\"/api/v0/id\"}, // Only /id allowed\n\t\t\t},\n\t\t})\n\n\t\tapiClient := node.APIClient()\n\t\tapiClient.Client = &http.Client{\n\t\t\tTransport: auth.NewAuthorizedRoundTripper(\"Bearer mytoken\", http.DefaultTransport),\n\t\t}\n\n\t\t// Can access /version even though not in AllowedPaths\n\t\tresp := apiClient.Post(\"/api/v0/version\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"User cannot access API with another user's secret\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\"alice\": {\n\t\t\t\tAuthSecret: \"bearer:alice-secret\",\n\t\t\t\tAllowedPaths: []string{\"/api/v0/id\"},\n\t\t\t},\n\t\t\t\"bob\": {\n\t\t\t\tAuthSecret: \"bearer:bob-secret\",\n\t\t\t\tAllowedPaths: []string{\"/api/v0/config\"},\n\t\t\t},\n\t\t})\n\n\t\t// Alice tries to use Bob's secret\n\t\tapiClient := node.APIClient()\n\t\tapiClient.Client = &http.Client{\n\t\t\tTransport: auth.NewAuthorizedRoundTripper(\"Bearer bob-secret\", http.DefaultTransport),\n\t\t}\n\n\t\t// Bob's secret should work for Bob's paths\n\t\tresp := apiClient.Post(\"/api/v0/config/show\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\t// But not for Alice's paths (Bob doesn't have access to /id)\n\t\tresp = apiClient.Post(\"/api/v0/id\", nil)\n\t\tassert.Equal(t, 403, resp.StatusCode)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"Empty AllowedPaths denies all access except version\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\"userA\": {\n\t\t\t\tAuthSecret: \"bearer:mytoken\",\n\t\t\t\tAllowedPaths: []string{}, // Empty!\n\t\t\t},\n\t\t})\n\n\t\tapiClient := node.APIClient()\n\t\tapiClient.Client = &http.Client{\n\t\t\tTransport: auth.NewAuthorizedRoundTripper(\"Bearer mytoken\", http.DefaultTransport),\n\t\t}\n\n\t\t// Should deny everything\n\t\tresp := apiClient.Post(\"/api/v0/id\", nil)\n\t\tassert.Equal(t, 403, resp.StatusCode)\n\n\t\tresp = apiClient.Post(\"/api/v0/config/show\", nil)\n\t\tassert.Equal(t, 403, resp.StatusCode)\n\n\t\t// Except version\n\t\tresp = apiClient.Post(\"/api/v0/version\", nil)\n\t\tassert.Equal(t, 200, resp.StatusCode)\n\n\t\tnode.StopDaemon()\n\t})\n\n\tt.Run(\"CLI commands fail without --api-auth when auth is enabled\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\tnode := makeAndStartProtectedNode(t, map[string]*config.RPCAuthScope{\n\t\t\t\"userA\": {\n\t\t\t\tAuthSecret: \"bearer:mytoken\",\n\t\t\t\tAllowedPaths: []string{\"/api/v0\"},\n\t\t\t},\n\t\t})\n\n\t\t// Try to run command without --api-auth flag\n\t\tresp := node.RunIPFS(\"id\") // No --api-auth flag\n\t\trequire.Error(t, resp.Err)\n\t\trequire.Contains(t, resp.Stderr.String(), rpcDeniedMsg)\n\n\t\tnode.StopDaemon()\n\t})\n}\n" }, { "path": "test/cli/rpc_content_type_test.go", "content": "// Tests HTTP RPC Content-Type headers.\n// These tests verify that RPC endpoints return correct Content-Type headers\n// for binary responses (CAR, tar, gzip, raw blocks, IPNS records).\n\npackage cli\n\nimport (\n\t\"bytes\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestRPCDagExportContentType verifies that the RPC endpoint for `ipfs dag export`\n// returns the correct Content-Type header for CAR output.\nfunc TestRPCDagExportContentType(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon(\"--offline\")\n\n\t// add test content\n\tcid := node.IPFSAddStr(\"test content for dag export\")\n\n\turl := node.APIURL() + \"/api/v0/dag/export?arg=\" + cid\n\n\treq, err := http.NewRequest(http.MethodPost, url, nil)\n\trequire.NoError(t, err)\n\n\tresp, err := http.DefaultClient.Do(req)\n\trequire.NoError(t, err)\n\tdefer resp.Body.Close()\n\n\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\tassert.Equal(t, \"application/vnd.ipld.car\", resp.Header.Get(\"Content-Type\"),\n\t\t\"dag export should return application/vnd.ipld.car\")\n}\n\n// TestRPCBlockGetContentType verifies that the RPC endpoint for `ipfs block get`\n// returns the correct Content-Type header for raw block data.\nfunc TestRPCBlockGetContentType(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon(\"--offline\")\n\n\t// add test content\n\tcid := node.IPFSAddStr(\"test content for block get\")\n\n\turl := node.APIURL() + \"/api/v0/block/get?arg=\" + cid\n\n\treq, err := http.NewRequest(http.MethodPost, url, nil)\n\trequire.NoError(t, err)\n\n\tresp, err := http.DefaultClient.Do(req)\n\trequire.NoError(t, err)\n\tdefer resp.Body.Close()\n\n\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\tassert.Equal(t, \"application/vnd.ipld.raw\", resp.Header.Get(\"Content-Type\"),\n\t\t\"block get should return application/vnd.ipld.raw\")\n}\n\n// TestRPCProfileContentType verifies that the RPC endpoint for `ipfs diag profile`\n// returns the correct Content-Type header for ZIP output.\nfunc TestRPCProfileContentType(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon(\"--offline\")\n\n\t// use profile-time=0 to skip sampling profiles and return quickly\n\turl := node.APIURL() + \"/api/v0/diag/profile?profile-time=0\"\n\n\treq, err := http.NewRequest(http.MethodPost, url, nil)\n\trequire.NoError(t, err)\n\n\tresp, err := http.DefaultClient.Do(req)\n\trequire.NoError(t, err)\n\tdefer resp.Body.Close()\n\n\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\tassert.Equal(t, \"application/zip\", resp.Header.Get(\"Content-Type\"),\n\t\t\"diag profile should return application/zip\")\n}\n\n// TestHTTPRPCNameGet verifies the behavior of `ipfs name get` vs `ipfs routing get`:\n//\n// `ipfs name get `:\n// - Purpose: dedicated command for retrieving IPNS records\n// - Returns: raw IPNS record bytes (protobuf)\n// - Content-Type: application/vnd.ipfs.ipns-record\n//\n// `ipfs routing get /ipns/`:\n// - Purpose: generic routing get for any key type\n// - Returns: JSON with base64-encoded record in \"Extra\" field\n// - Content-Type: application/json\n//\n// Both commands retrieve the same underlying IPNS record data.\nfunc TestHTTPRPCNameGet(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon() // must be online to use routing\n\n\t// add test content and publish IPNS record\n\tcid := node.IPFSAddStr(\"test content for name get\")\n\tnode.IPFS(\"name\", \"publish\", cid)\n\n\t// get the node's peer ID (which is also the IPNS name)\n\tpeerID := node.PeerID().String()\n\n\t// Test ipfs name get - returns raw IPNS record bytes with specific Content-Type\n\tnameGetURL := node.APIURL() + \"/api/v0/name/get?arg=\" + peerID\n\tnameGetReq, err := http.NewRequest(http.MethodPost, nameGetURL, nil)\n\trequire.NoError(t, err)\n\n\tnameGetResp, err := http.DefaultClient.Do(nameGetReq)\n\trequire.NoError(t, err)\n\tdefer nameGetResp.Body.Close()\n\n\tassert.Equal(t, http.StatusOK, nameGetResp.StatusCode)\n\tassert.Equal(t, \"application/vnd.ipfs.ipns-record\", nameGetResp.Header.Get(\"Content-Type\"),\n\t\t\"name get should return application/vnd.ipfs.ipns-record\")\n\n\tnameGetBytes, err := io.ReadAll(nameGetResp.Body)\n\trequire.NoError(t, err)\n\n\t// Test ipfs routing get /ipns/... - returns JSON with base64-encoded record\n\troutingGetURL := node.APIURL() + \"/api/v0/routing/get?arg=/ipns/\" + peerID\n\troutingGetReq, err := http.NewRequest(http.MethodPost, routingGetURL, nil)\n\trequire.NoError(t, err)\n\n\troutingGetResp, err := http.DefaultClient.Do(routingGetReq)\n\trequire.NoError(t, err)\n\tdefer routingGetResp.Body.Close()\n\n\tassert.Equal(t, http.StatusOK, routingGetResp.StatusCode)\n\tassert.Equal(t, \"application/json\", routingGetResp.Header.Get(\"Content-Type\"),\n\t\t\"routing get should return application/json\")\n\n\t// Parse JSON response and decode base64 record from \"Extra\" field\n\tvar routingResp struct {\n\t\tExtra string `json:\"Extra\"`\n\t\tType int `json:\"Type\"`\n\t}\n\terr = json.NewDecoder(routingGetResp.Body).Decode(&routingResp)\n\trequire.NoError(t, err)\n\n\troutingGetBytes, err := base64.StdEncoding.DecodeString(routingResp.Extra)\n\trequire.NoError(t, err)\n\n\t// Verify both commands return identical IPNS record bytes\n\tassert.Equal(t, nameGetBytes, routingGetBytes,\n\t\t\"name get and routing get should return identical IPNS record bytes\")\n\n\t// Verify the record can be inspected and contains the published CID\n\tinspectOutput := node.PipeToIPFS(bytes.NewReader(nameGetBytes), \"name\", \"inspect\")\n\tassert.Contains(t, inspectOutput.Stdout.String(), cid,\n\t\t\"ipfs name inspect should show the published CID\")\n}\n" }, { "path": "test/cli/rpc_get_output_test.go", "content": "package cli\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestRPCGetContentType verifies that the RPC endpoint for `ipfs get` returns\n// the correct Content-Type header based on output format options.\n//\n// Output formats and expected Content-Type:\n// - default (no flags): tar (transport format) -> application/x-tar\n// - --archive: tar archive -> application/x-tar\n// - --compress: gzip -> application/gzip\n// - --archive --compress: tar.gz -> application/gzip\n//\n// Fixes: https://github.com/ipfs/kubo/issues/2376\nfunc TestRPCGetContentType(t *testing.T) {\n\tt.Parallel()\n\n\tnode := harness.NewT(t).NewNode().Init()\n\tnode.StartDaemon(\"--offline\")\n\n\t// add test content\n\tcid := node.IPFSAddStr(\"test content for Content-Type header verification\")\n\n\ttests := []struct {\n\t\tname string\n\t\tquery string\n\t\texpectedContentType string\n\t}{\n\t\t{\n\t\t\tname: \"default returns application/x-tar\",\n\t\t\tquery: \"?arg=\" + cid,\n\t\t\texpectedContentType: \"application/x-tar\",\n\t\t},\n\t\t{\n\t\t\tname: \"archive=true returns application/x-tar\",\n\t\t\tquery: \"?arg=\" + cid + \"&archive=true\",\n\t\t\texpectedContentType: \"application/x-tar\",\n\t\t},\n\t\t{\n\t\t\tname: \"compress=true returns application/gzip\",\n\t\t\tquery: \"?arg=\" + cid + \"&compress=true\",\n\t\t\texpectedContentType: \"application/gzip\",\n\t\t},\n\t\t{\n\t\t\tname: \"archive=true&compress=true returns application/gzip\",\n\t\t\tquery: \"?arg=\" + cid + \"&archive=true&compress=true\",\n\t\t\texpectedContentType: \"application/gzip\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\turl := node.APIURL() + \"/api/v0/get\" + tt.query\n\n\t\t\treq, err := http.NewRequest(http.MethodPost, url, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tresp, err := http.DefaultClient.Do(req)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer resp.Body.Close()\n\n\t\t\tassert.Equal(t, http.StatusOK, resp.StatusCode)\n\t\t\tassert.Equal(t, tt.expectedContentType, resp.Header.Get(\"Content-Type\"),\n\t\t\t\t\"Content-Type header mismatch for %s\", tt.name)\n\t\t})\n\t}\n}\n" }, { "path": "test/cli/rpc_unixsocket_test.go", "content": "package cli\n\nimport (\n\t\"context\"\n\t\"path\"\n\t\"testing\"\n\n\trpcapi \"github.com/ipfs/kubo/client/rpc\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/multiformats/go-multiaddr\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRPCUnixSocket(t *testing.T) {\n\tnode := harness.NewT(t).NewNode().Init()\n\n\tsockDir := node.Dir\n\tsockAddr := path.Join(\"/unix\", sockDir, \"sock\")\n\n\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t//cfg.Addresses.API = append(cfg.Addresses.API, sockPath)\n\t\tcfg.Addresses.API = []string{sockAddr}\n\t})\n\tt.Log(\"Starting daemon with unix socket:\", sockAddr)\n\tnode.StartDaemon()\n\n\tunixMaddr, err := multiaddr.NewMultiaddr(sockAddr)\n\trequire.NoError(t, err)\n\n\tapiClient, err := rpcapi.NewApi(unixMaddr)\n\trequire.NoError(t, err)\n\n\tvar ver struct {\n\t\tVersion string\n\t}\n\terr = apiClient.Request(\"version\").Exec(context.Background(), &ver)\n\trequire.NoError(t, err)\n\trequire.NotEmpty(t, ver)\n\tt.Log(\"Got version:\", ver.Version)\n\n\tvar res struct {\n\t\tID string\n\t}\n\terr = apiClient.Request(\"id\").Exec(context.Background(), &res)\n\trequire.NoError(t, err)\n\trequire.NotEmpty(t, res)\n\tt.Log(\"Got ID:\", res.ID)\n\n\tnode.StopDaemon()\n}\n" }, { "path": "test/cli/stats_test.go", "content": "package cli\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n)\n\nfunc TestStats(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"stats dht\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(2).Init().StartDaemons().Connect()\n\t\tdefer nodes.StopDaemons()\n\t\tnode1 := nodes[0]\n\n\t\tres := node1.IPFS(\"stats\", \"dht\")\n\t\tassert.NoError(t, res.Err)\n\t\tassert.Equal(t, 0, len(res.Stderr.Lines()))\n\t\tassert.NotEqual(t, 0, len(res.Stdout.Lines()))\n\t})\n}\n" }, { "path": "test/cli/swarm_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\n// TODO: Migrate the rest of the sharness swarm test.\nfunc TestSwarm(t *testing.T) {\n\ttype identifyType struct {\n\t\tID string\n\t\tPublicKey string\n\t\tAddresses []string\n\t\tAgentVersion string\n\t\tProtocols []string\n\t}\n\ttype peer struct {\n\t\tIdentify identifyType\n\t}\n\ttype expectedOutputType struct {\n\t\tPeers []peer\n\t}\n\n\tt.Parallel()\n\n\tt.Run(\"ipfs swarm peers returns empty peers when a node is not connected to any peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\tres := node.RunIPFS(\"swarm\", \"peers\", \"--enc=json\", \"--identify\")\n\t\tvar output expectedOutputType\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &output)\n\t\tassert.NoError(t, err)\n\t\tassert.Equal(t, 0, len(output.Peers))\n\t})\n\tt.Run(\"ipfs swarm peers with flag identify outputs expected identify information about connected peers\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\totherNode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer otherNode.StopDaemon()\n\t\tnode.Connect(otherNode)\n\n\t\tres := node.RunIPFS(\"swarm\", \"peers\", \"--enc=json\", \"--identify\")\n\t\tvar output expectedOutputType\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &output)\n\t\tassert.NoError(t, err)\n\t\tactualID := output.Peers[0].Identify.ID\n\t\tactualPublicKey := output.Peers[0].Identify.PublicKey\n\t\tactualAgentVersion := output.Peers[0].Identify.AgentVersion\n\t\tactualAddresses := output.Peers[0].Identify.Addresses\n\t\tactualProtocols := output.Peers[0].Identify.Protocols\n\n\t\texpectedID := otherNode.PeerID().String()\n\t\texpectedAddresses := []string{fmt.Sprintf(\"%s/p2p/%s\", otherNode.SwarmAddrs()[0], actualID)}\n\n\t\tassert.Equal(t, actualID, expectedID)\n\t\tassert.NotNil(t, actualPublicKey)\n\t\tassert.NotNil(t, actualAgentVersion)\n\t\tassert.Len(t, actualAddresses, 1)\n\t\tassert.Equal(t, expectedAddresses[0], actualAddresses[0])\n\t\tassert.Greater(t, len(actualProtocols), 0)\n\t})\n\n\tt.Run(\"ipfs swarm peers with flag identify outputs Identify field with data that matches calling ipfs id on a peer\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\t\totherNode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer otherNode.StopDaemon()\n\t\tnode.Connect(otherNode)\n\n\t\totherNodeIDResponse := otherNode.RunIPFS(\"id\", \"--enc=json\")\n\t\tvar otherNodeIDOutput identifyType\n\t\terr := json.Unmarshal(otherNodeIDResponse.Stdout.Bytes(), &otherNodeIDOutput)\n\t\tassert.NoError(t, err)\n\t\tres := node.RunIPFS(\"swarm\", \"peers\", \"--enc=json\", \"--identify\")\n\n\t\tvar output expectedOutputType\n\t\terr = json.Unmarshal(res.Stdout.Bytes(), &output)\n\t\tassert.NoError(t, err)\n\t\toutputIdentify := output.Peers[0].Identify\n\n\t\tassert.Equal(t, outputIdentify.ID, otherNodeIDOutput.ID)\n\t\tassert.Equal(t, outputIdentify.PublicKey, otherNodeIDOutput.PublicKey)\n\t\tassert.Equal(t, outputIdentify.AgentVersion, otherNodeIDOutput.AgentVersion)\n\t\tassert.ElementsMatch(t, outputIdentify.Addresses, otherNodeIDOutput.Addresses)\n\t\tassert.ElementsMatch(t, outputIdentify.Protocols, otherNodeIDOutput.Protocols)\n\t})\n\n\tt.Run(\"ipfs swarm addrs autonat returns valid reachability status\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init().StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\tres := node.RunIPFS(\"swarm\", \"addrs\", \"autonat\", \"--enc=json\")\n\t\tassert.NoError(t, res.Err)\n\n\t\tvar output struct {\n\t\t\tReachability string `json:\"reachability\"`\n\t\t\tReachable []string `json:\"reachable\"`\n\t\t\tUnreachable []string `json:\"unreachable\"`\n\t\t\tUnknown []string `json:\"unknown\"`\n\t\t}\n\t\terr := json.Unmarshal(res.Stdout.Bytes(), &output)\n\t\tassert.NoError(t, err)\n\n\t\t// Reachability must be one of the valid states\n\t\t// Note: network.Reachability constants use capital first letter\n\t\tvalidStates := []string{\"Public\", \"Private\", \"Unknown\"}\n\t\tassert.Contains(t, validStates, output.Reachability,\n\t\t\t\"Reachability should be one of: Public, Private, Unknown\")\n\n\t\t// For a newly started node, reachability is typically Unknown initially\n\t\t// as AutoNAT hasn't completed probing yet. This is expected behavior.\n\t\t// The important thing is that the command runs and returns valid data.\n\t\ttotalAddrs := len(output.Reachable) + len(output.Unreachable) + len(output.Unknown)\n\t\tt.Logf(\"Reachability: %s, Total addresses: %d (reachable: %d, unreachable: %d, unknown: %d)\",\n\t\t\toutput.Reachability, totalAddrs, len(output.Reachable), len(output.Unreachable), len(output.Unknown))\n\t})\n}\n" }, { "path": "test/cli/telemetry_test.go", "content": "package cli\n\nimport (\n\t\"encoding/json\"\n\t\"io\"\n\t\"maps\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"slices\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestTelemetry(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"opt-out via environment variable\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a new node\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Plugins.Plugins.telemetry.Disabled\", false)\n\n\t\t// Set the opt-out environment variable\n\t\tnode.Runner.Env[\"IPFS_TELEMETRY\"] = \"off\"\n\t\tnode.Runner.Env[\"GOLOG_LOG_LEVEL\"] = \"telemetry=debug\"\n\n\t\t// Capture daemon output\n\t\tstdout := &harness.Buffer{}\n\t\tstderr := &harness.Buffer{}\n\n\t\t// Start daemon with output capture\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithStdout(stdout),\n\t\t\t\tharness.RunWithStderr(stderr),\n\t\t\t},\n\t\t}, \"\")\n\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Get daemon output\n\t\toutput := stdout.String() + stderr.String()\n\n\t\t// Check that telemetry is disabled\n\t\tassert.Contains(t, output, \"telemetry disabled via opt-out\", \"Expected telemetry disabled message\")\n\n\t\t// Stop daemon\n\t\tnode.StopDaemon()\n\n\t\t// Verify UUID file was not created or was removed\n\t\tuuidPath := filepath.Join(node.Dir, \"telemetry_uuid\")\n\t\t_, err := os.Stat(uuidPath)\n\t\tassert.True(t, os.IsNotExist(err), \"UUID file should not exist when opted out\")\n\t})\n\n\tt.Run(\"opt-out via config\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a new node\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Plugins.Plugins.telemetry.Disabled\", false)\n\n\t\t// Set opt-out via config\n\t\tnode.IPFS(\"config\", \"Plugins.Plugins.telemetry.Config.Mode\", \"off\")\n\n\t\t// Enable debug logging\n\t\tnode.Runner.Env[\"GOLOG_LOG_LEVEL\"] = \"telemetry=debug\"\n\n\t\t// Capture daemon output\n\t\tstdout := &harness.Buffer{}\n\t\tstderr := &harness.Buffer{}\n\n\t\t// Start daemon with output capture\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithStdout(stdout),\n\t\t\t\tharness.RunWithStderr(stderr),\n\t\t\t},\n\t\t}, \"\")\n\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Get daemon output\n\t\toutput := stdout.String() + stderr.String()\n\n\t\t// Check that telemetry is disabled\n\t\tassert.Contains(t, output, \"telemetry disabled via opt-out\", \"Expected telemetry disabled message\")\n\t\tassert.Contains(t, output, \"telemetry collection skipped: opted out\", \"Expected telemetry skipped message\")\n\n\t\t// Stop daemon\n\t\tnode.StopDaemon()\n\n\t\t// Verify UUID file was not created or was removed\n\t\tuuidPath := filepath.Join(node.Dir, \"telemetry_uuid\")\n\t\t_, err := os.Stat(uuidPath)\n\t\tassert.True(t, os.IsNotExist(err), \"UUID file should not exist when opted out\")\n\t})\n\n\tt.Run(\"opt-out removes existing UUID file\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a new node\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Plugins.Plugins.telemetry.Disabled\", false)\n\n\t\t// Create a UUID file manually to simulate previous telemetry run\n\t\tuuidPath := filepath.Join(node.Dir, \"telemetry_uuid\")\n\t\ttestUUID := \"test-uuid-12345\"\n\t\terr := os.WriteFile(uuidPath, []byte(testUUID), 0600)\n\t\trequire.NoError(t, err, \"Failed to create test UUID file\")\n\n\t\t// Verify file exists\n\t\t_, err = os.Stat(uuidPath)\n\t\trequire.NoError(t, err, \"UUID file should exist before opt-out\")\n\n\t\t// Set the opt-out environment variable\n\t\tnode.Runner.Env[\"IPFS_TELEMETRY\"] = \"off\"\n\t\tnode.Runner.Env[\"GOLOG_LOG_LEVEL\"] = \"telemetry=debug\"\n\n\t\t// Capture daemon output\n\t\tstdout := &harness.Buffer{}\n\t\tstderr := &harness.Buffer{}\n\n\t\t// Start daemon with output capture\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithStdout(stdout),\n\t\t\t\tharness.RunWithStderr(stderr),\n\t\t\t},\n\t\t}, \"\")\n\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Get daemon output\n\t\toutput := stdout.String() + stderr.String()\n\n\t\t// Check that UUID file was removed\n\t\tassert.Contains(t, output, \"removed existing telemetry UUID file due to opt-out\", \"Expected UUID removal message\")\n\n\t\t// Stop daemon\n\t\tnode.StopDaemon()\n\n\t\t// Verify UUID file was removed\n\t\t_, err = os.Stat(uuidPath)\n\t\tassert.True(t, os.IsNotExist(err), \"UUID file should be removed after opt-out\")\n\t})\n\n\tt.Run(\"telemetry enabled shows info message\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Create a new node\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Plugins.Plugins.telemetry.Disabled\", false)\n\n\t\t// Capture daemon output\n\t\tstdout := &harness.Buffer{}\n\t\tstderr := &harness.Buffer{}\n\n\t\t// Don't set opt-out, so telemetry will be enabled\n\t\t// This should trigger the info message on first run\n\t\tnode.StartDaemonWithReq(harness.RunRequest{\n\t\t\tCmdOpts: []harness.CmdOpt{\n\t\t\t\tharness.RunWithStdout(stdout),\n\t\t\t\tharness.RunWithStderr(stderr),\n\t\t\t},\n\t\t}, \"\")\n\n\t\ttime.Sleep(500 * time.Millisecond)\n\n\t\t// Get daemon output\n\t\toutput := stdout.String() + stderr.String()\n\n\t\t// First run - should show info message\n\t\tassert.Contains(t, output, \"Anonymous telemetry\")\n\t\tassert.Contains(t, output, \"No data sent yet\", \"Expected no data sent message\")\n\t\tassert.Contains(t, output, \"To opt-out before collection starts\", \"Expected opt-out instructions\")\n\t\tassert.Contains(t, output, \"Learn more:\", \"Expected learn more link\")\n\n\t\t// Stop daemon\n\t\tnode.StopDaemon()\n\n\t\t// Verify UUID file was created\n\t\tuuidPath := filepath.Join(node.Dir, \"telemetry_uuid\")\n\t\t_, err := os.Stat(uuidPath)\n\t\tassert.NoError(t, err, \"UUID file should exist when daemon started without telemetry opt-out\")\n\t})\n\n\tt.Run(\"telemetry schema regression guard\", func(t *testing.T) {\n\t\tt.Parallel()\n\n\t\t// Define the exact set of expected telemetry fields\n\t\t// This list must be updated whenever telemetry fields change\n\t\texpectedFields := []string{\n\t\t\t\"uuid\",\n\t\t\t\"agent_version\",\n\t\t\t\"private_network\",\n\t\t\t\"bootstrappers_custom\",\n\t\t\t\"repo_size_bucket\",\n\t\t\t\"uptime_bucket\",\n\t\t\t\"reprovider_strategy\",\n\t\t\t\"provide_dht_sweep_enabled\",\n\t\t\t\"provide_dht_interval_custom\",\n\t\t\t\"provide_dht_max_workers_custom\",\n\t\t\t\"routing_type\",\n\t\t\t\"routing_accelerated_dht_client\",\n\t\t\t\"routing_delegated_count\",\n\t\t\t\"autonat_service_mode\",\n\t\t\t\"autonat_reachability\",\n\t\t\t\"swarm_enable_hole_punching\",\n\t\t\t\"swarm_circuit_addresses\",\n\t\t\t\"swarm_ipv4_public_addresses\",\n\t\t\t\"swarm_ipv6_public_addresses\",\n\t\t\t\"auto_tls_auto_wss\",\n\t\t\t\"auto_tls_domain_suffix_custom\",\n\t\t\t\"autoconf\",\n\t\t\t\"autoconf_custom\",\n\t\t\t\"discovery_mdns_enabled\",\n\t\t\t\"platform_os\",\n\t\t\t\"platform_arch\",\n\t\t\t\"platform_containerized\",\n\t\t\t\"platform_vm\",\n\t\t}\n\n\t\t// Channel to receive captured telemetry data\n\t\ttelemetryChan := make(chan map[string]any, 1)\n\n\t\t// Create a mock HTTP server to capture telemetry\n\t\tmockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif r.Method != \"POST\" {\n\t\t\t\thttp.Error(w, \"Method not allowed\", http.StatusMethodNotAllowed)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tbody, err := io.ReadAll(r.Body)\n\t\t\tif err != nil {\n\t\t\t\thttp.Error(w, \"Failed to read body\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tvar telemetryData map[string]any\n\t\t\tif err := json.Unmarshal(body, &telemetryData); err != nil {\n\t\t\t\thttp.Error(w, \"Invalid JSON\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t// Send captured data through channel\n\t\t\tselect {\n\t\t\tcase telemetryChan <- telemetryData:\n\t\t\tdefault:\n\t\t\t}\n\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer mockServer.Close()\n\n\t\t// Create a new node\n\t\tnode := harness.NewT(t).NewNode().Init()\n\t\tnode.SetIPFSConfig(\"Plugins.Plugins.telemetry.Disabled\", false)\n\n\t\t// Configure telemetry with a very short delay for testing\n\t\tnode.IPFS(\"config\", \"Plugins.Plugins.telemetry.Config.Delay\", \"100ms\")\n\t\tnode.IPFS(\"config\", \"Plugins.Plugins.telemetry.Config.Endpoint\", mockServer.URL)\n\n\t\t// Enable debug logging to see what's being sent\n\t\tnode.Runner.Env[\"GOLOG_LOG_LEVEL\"] = \"telemetry=debug\"\n\n\t\t// Start daemon\n\t\tnode.StartDaemon()\n\t\tdefer node.StopDaemon()\n\n\t\t// Wait for telemetry to be sent (configured delay + buffer)\n\t\tselect {\n\t\tcase telemetryData := <-telemetryChan:\n\t\t\treceivedFields := slices.Collect(maps.Keys(telemetryData))\n\t\t\tslices.Sort(expectedFields)\n\t\t\tslices.Sort(receivedFields)\n\n\t\t\t// Fast path: check if fields match exactly\n\t\t\tif !slices.Equal(expectedFields, receivedFields) {\n\t\t\t\tvar missingFields, unexpectedFields []string\n\t\t\t\tfor _, field := range expectedFields {\n\t\t\t\t\tif _, ok := telemetryData[field]; !ok {\n\t\t\t\t\t\tmissingFields = append(missingFields, field)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\texpectedSet := make(map[string]struct{}, len(expectedFields))\n\t\t\t\tfor _, f := range expectedFields {\n\t\t\t\t\texpectedSet[f] = struct{}{}\n\t\t\t\t}\n\t\t\t\tfor field := range telemetryData {\n\t\t\t\t\tif _, ok := expectedSet[field]; !ok {\n\t\t\t\t\t\tunexpectedFields = append(unexpectedFields, field)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tt.Fatalf(\"Telemetry field mismatch:\\n\"+\n\t\t\t\t\t\" Missing fields: %v\\n\"+\n\t\t\t\t\t\" Unexpected fields: %v\\n\"+\n\t\t\t\t\t\" Note: Update expectedFields list in this test when adding/removing telemetry fields\",\n\t\t\t\t\tmissingFields, unexpectedFields)\n\t\t\t}\n\n\t\t\tt.Logf(\"Telemetry field validation passed: %d fields verified\", len(expectedFields))\n\n\t\tcase <-time.After(5 * time.Second):\n\t\t\tt.Fatal(\"Timeout waiting for telemetry data to be sent\")\n\t\t}\n\t})\n}\n" }, { "path": "test/cli/testutils/asserts.go", "content": "package testutils\n\nimport (\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc AssertStringContainsOneOf(t *testing.T, str string, ss ...string) {\n\tfor _, s := range ss {\n\t\tif strings.Contains(str, s) {\n\t\t\treturn\n\t\t}\n\t}\n\tt.Errorf(\"%q does not contain one of %v\", str, ss)\n}\n" }, { "path": "test/cli/testutils/cids.go", "content": "package testutils\n\nconst (\n\tCIDWelcomeDocs = \"QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc\"\n\tCIDEmptyDir = \"QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\"\n)\n" }, { "path": "test/cli/testutils/files.go", "content": "package testutils\n\nimport (\n\t\"log\"\n\t\"os\"\n\t\"path/filepath\"\n)\n\nfunc MustOpen(name string) *os.File {\n\tf, err := os.Open(name)\n\tif err != nil {\n\t\tlog.Panicf(\"opening %s: %s\", name, err)\n\t}\n\treturn f\n}\n\n// Searches for a file in a dir, then the parent dir, etc.\n// If the file is not found, an empty string is returned.\nfunc FindUp(name, dir string) string {\n\tcurDir := dir\n\tfor {\n\t\tentries, err := os.ReadDir(curDir)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tfor _, e := range entries {\n\t\t\tif name == e.Name() {\n\t\t\t\treturn filepath.Join(curDir, name)\n\t\t\t}\n\t\t}\n\t\tnewDir := filepath.Dir(curDir)\n\t\tif newDir == curDir {\n\t\t\treturn \"\"\n\t\t}\n\t\tcurDir = newDir\n\t}\n}\n" }, { "path": "test/cli/testutils/floats.go", "content": "package testutils\n\nfunc FloatTruncate(value float64, decimalPlaces int) float64 {\n\tpow := 1.0\n\tfor range decimalPlaces {\n\t\tpow *= 10.0\n\t}\n\treturn float64(int(value*pow)) / pow\n}\n" }, { "path": "test/cli/testutils/httprouting/mock_http_content_router.go", "content": "package httprouting\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/ipns\"\n\t\"github.com/ipfs/boxo/routing/http/server\"\n\t\"github.com/ipfs/boxo/routing/http/types\"\n\t\"github.com/ipfs/boxo/routing/http/types/iter\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\t\"github.com/libp2p/go-libp2p/core/routing\"\n)\n\n// MockHTTPContentRouter provides /routing/v1\n// (https://specs.ipfs.tech/routing/http-routing-v1/) server implementation\n// based on github.com/ipfs/boxo/routing/http/server\ntype MockHTTPContentRouter struct {\n\tm sync.Mutex\n\tprovideBitswapCalls int\n\tfindProvidersCalls int\n\tfindPeersCalls int\n\tgetClosestPeersCalls int\n\tproviders map[cid.Cid][]types.Record\n\tpeers map[peer.ID][]*types.PeerRecord\n\tDebug bool\n}\n\nfunc (r *MockHTTPContentRouter) FindProviders(ctx context.Context, key cid.Cid, limit int) (iter.ResultIter[types.Record], error) {\n\tif r.Debug {\n\t\tfmt.Printf(\"MockHTTPContentRouter.FindProviders(%s)\\n\", key.String())\n\t}\n\tr.m.Lock()\n\tdefer r.m.Unlock()\n\tr.findProvidersCalls++\n\tif r.providers == nil {\n\t\tr.providers = make(map[cid.Cid][]types.Record)\n\t}\n\trecords, found := r.providers[key]\n\tif !found {\n\t\treturn iter.FromSlice([]iter.Result[types.Record]{}), nil\n\t}\n\tresults := make([]iter.Result[types.Record], len(records))\n\tfor i, rec := range records {\n\t\tresults[i] = iter.Result[types.Record]{Val: rec}\n\t\tif r.Debug {\n\t\t\tfmt.Printf(\"MockHTTPContentRouter.FindProviders(%s) result: %+v\\n\", key.String(), rec)\n\t\t}\n\t}\n\treturn iter.FromSlice(results), nil\n}\n\n// nolint deprecated\nfunc (r *MockHTTPContentRouter) ProvideBitswap(ctx context.Context, req *server.BitswapWriteProvideRequest) (time.Duration, error) {\n\tr.m.Lock()\n\tdefer r.m.Unlock()\n\tr.provideBitswapCalls++\n\treturn 0, nil\n}\n\nfunc (r *MockHTTPContentRouter) FindPeers(ctx context.Context, pid peer.ID, limit int) (iter.ResultIter[*types.PeerRecord], error) {\n\tr.m.Lock()\n\tdefer r.m.Unlock()\n\tr.findPeersCalls++\n\n\tif r.peers == nil {\n\t\tr.peers = make(map[peer.ID][]*types.PeerRecord)\n\t}\n\trecords, found := r.peers[pid]\n\tif !found {\n\t\treturn iter.FromSlice([]iter.Result[*types.PeerRecord]{}), nil\n\t}\n\n\tresults := make([]iter.Result[*types.PeerRecord], len(records))\n\tfor i, rec := range records {\n\t\tresults[i] = iter.Result[*types.PeerRecord]{Val: rec}\n\t\tif r.Debug {\n\t\t\tfmt.Printf(\"MockHTTPContentRouter.FindPeers(%s) result: %+v\\n\", pid.String(), rec)\n\t\t}\n\t}\n\treturn iter.FromSlice(results), nil\n}\n\nfunc (r *MockHTTPContentRouter) GetIPNS(ctx context.Context, name ipns.Name) (*ipns.Record, error) {\n\treturn nil, routing.ErrNotSupported\n}\n\nfunc (r *MockHTTPContentRouter) PutIPNS(ctx context.Context, name ipns.Name, rec *ipns.Record) error {\n\treturn routing.ErrNotSupported\n}\n\nfunc (r *MockHTTPContentRouter) NumFindProvidersCalls() int {\n\tr.m.Lock()\n\tdefer r.m.Unlock()\n\treturn r.findProvidersCalls\n}\n\n// AddProvider adds a record for a given CID\nfunc (r *MockHTTPContentRouter) AddProvider(key cid.Cid, record types.Record) {\n\tr.m.Lock()\n\tdefer r.m.Unlock()\n\tif r.providers == nil {\n\t\tr.providers = make(map[cid.Cid][]types.Record)\n\t}\n\tr.providers[key] = append(r.providers[key], record)\n\n\tpeerRecord, ok := record.(*types.PeerRecord)\n\tif ok {\n\t\tif r.peers == nil {\n\t\t\tr.peers = make(map[peer.ID][]*types.PeerRecord)\n\t\t}\n\t\tpid := peerRecord.ID\n\t\tr.peers[*pid] = append(r.peers[*pid], peerRecord)\n\t}\n}\n\nfunc (r *MockHTTPContentRouter) GetClosestPeers(ctx context.Context, key cid.Cid) (iter.ResultIter[*types.PeerRecord], error) {\n\tr.m.Lock()\n\tdefer r.m.Unlock()\n\tr.getClosestPeersCalls++\n\n\tif r.peers == nil {\n\t\tr.peers = make(map[peer.ID][]*types.PeerRecord)\n\t}\n\tpid, err := peer.FromCid(key)\n\tif err != nil {\n\t\treturn iter.FromSlice([]iter.Result[*types.PeerRecord]{}), nil\n\t}\n\trecords, found := r.peers[pid]\n\tif !found {\n\t\treturn iter.FromSlice([]iter.Result[*types.PeerRecord]{}), nil\n\t}\n\n\tresults := make([]iter.Result[*types.PeerRecord], len(records))\n\tfor i, rec := range records {\n\t\tresults[i] = iter.Result[*types.PeerRecord]{Val: rec}\n\t\tif r.Debug {\n\t\t\tfmt.Printf(\"MockHTTPContentRouter.GetPeers(%s) result: %+v\\n\", pid.String(), rec)\n\t\t}\n\t}\n\treturn iter.FromSlice(results), nil\n}\n" }, { "path": "test/cli/testutils/json.go", "content": "package testutils\n\nimport \"encoding/json\"\n\ntype JSONObj map[string]any\n\nfunc ToJSONStr(m JSONObj) string {\n\tb, err := json.Marshal(m)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn string(b)\n}\n" }, { "path": "test/cli/testutils/pinningservice/pinning.go", "content": "package pinningservice\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"reflect\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/julienschmidt/httprouter\"\n)\n\nfunc NewRouter(authToken string, svc *PinningService) http.Handler {\n\trouter := httprouter.New()\n\trouter.GET(\"/api/v1/pins\", svc.listPins)\n\trouter.POST(\"/api/v1/pins\", svc.addPin)\n\trouter.GET(\"/api/v1/pins/:requestID\", svc.getPin)\n\trouter.POST(\"/api/v1/pins/:requestID\", svc.replacePin)\n\trouter.DELETE(\"/api/v1/pins/:requestID\", svc.removePin)\n\n\thandler := authHandler(authToken, router)\n\n\treturn handler\n}\n\nfunc authHandler(authToken string, delegate http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tauthz := r.Header.Get(\"Authorization\")\n\t\tif !strings.HasPrefix(authz, \"Bearer \") {\n\t\t\terrResp(w, \"invalid authorization token, must start with 'Bearer '\", \"\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\ttoken := strings.TrimPrefix(authz, \"Bearer \")\n\t\tif token != authToken {\n\t\t\terrResp(w, \"access denied\", \"\", http.StatusUnauthorized)\n\t\t\treturn\n\t\t}\n\n\t\tdelegate.ServeHTTP(w, r)\n\t})\n}\n\nfunc New() *PinningService {\n\treturn &PinningService{\n\t\tPinAdded: func(*AddPinRequest, *PinStatus) {},\n\t}\n}\n\n// PinningService is a basic pinning service that implements the Remote Pinning API, for testing Kubo's integration with remote pinning services.\n// Pins are not persisted, they are just kept in-memory, and this provides callbacks for controlling the behavior of the pinning service.\ntype PinningService struct {\n\tm sync.Mutex\n\t// PinAdded is a callback that is invoked after a new pin is added via the API.\n\tPinAdded func(*AddPinRequest, *PinStatus)\n\tpins []*PinStatus\n}\n\ntype Pin struct {\n\tCID string `json:\"cid\"`\n\tName string `json:\"name\"`\n\tOrigins []string `json:\"origins\"`\n\tMeta map[string]any `json:\"meta\"`\n}\n\ntype PinStatus struct {\n\tM sync.Mutex\n\tRequestID string\n\tStatus string\n\tCreated time.Time\n\tPin Pin\n\tDelegates []string\n\tInfo map[string]any\n}\n\nfunc (p *PinStatus) MarshalJSON() ([]byte, error) {\n\ttype pinStatusJSON struct {\n\t\tRequestID string `json:\"requestid\"`\n\t\tStatus string `json:\"status\"`\n\t\tCreated time.Time `json:\"created\"`\n\t\tPin Pin `json:\"pin\"`\n\t\tDelegates []string `json:\"delegates\"`\n\t\tInfo map[string]any `json:\"info\"`\n\t}\n\t// lock the pin before marshaling it to protect against data races while marshaling\n\tp.M.Lock()\n\tpinJSON := pinStatusJSON{\n\t\tRequestID: p.RequestID,\n\t\tStatus: p.Status,\n\t\tCreated: p.Created,\n\t\tPin: p.Pin,\n\t\tDelegates: p.Delegates,\n\t\tInfo: p.Info,\n\t}\n\tp.M.Unlock()\n\treturn json.Marshal(pinJSON)\n}\n\nfunc (p *PinStatus) Clone() PinStatus {\n\treturn PinStatus{\n\t\tRequestID: p.RequestID,\n\t\tStatus: p.Status,\n\t\tCreated: p.Created,\n\t\tPin: p.Pin,\n\t\tDelegates: p.Delegates,\n\t\tInfo: p.Info,\n\t}\n}\n\nconst (\n\tmatchExact = \"exact\"\n\tmatchIExact = \"iexact\"\n\tmatchPartial = \"partial\"\n\tmatchIPartial = \"ipartial\"\n\n\tstatusQueued = \"queued\"\n\tstatusPinning = \"pinning\"\n\tstatusPinned = \"pinned\"\n\tstatusFailed = \"failed\"\n\n\ttimeLayout = \"2006-01-02T15:04:05.999Z\"\n)\n\nfunc errResp(w http.ResponseWriter, reason, details string, statusCode int) {\n\ttype errorObj struct {\n\t\tReason string `json:\"reason\"`\n\t\tDetails string `json:\"details\"`\n\t}\n\ttype errorResp struct {\n\t\tError errorObj `json:\"error\"`\n\t}\n\tresp := errorResp{\n\t\tError: errorObj{\n\t\t\tReason: reason,\n\t\t\tDetails: details,\n\t\t},\n\t}\n\twriteJSON(w, resp, statusCode)\n}\n\nfunc writeJSON(w http.ResponseWriter, val any, statusCode int) {\n\tb, err := json.Marshal(val)\n\tif err != nil {\n\t\tw.Header().Set(\"Content-Type\", \"text/plain\")\n\t\terrResp(w, fmt.Sprintf(\"marshaling response: %s\", err), \"\", http.StatusInternalServerError)\n\t\treturn\n\t}\n\tw.Header().Set(\"Content-Type\", \"application/json\")\n\tw.WriteHeader(statusCode)\n\t_, _ = w.Write(b)\n}\n\ntype AddPinRequest struct {\n\tCID string `json:\"cid\"`\n\tName string `json:\"name\"`\n\tOrigins []string `json:\"origins\"`\n\tMeta map[string]any `json:\"meta\"`\n}\n\nfunc (p *PinningService) addPin(writer http.ResponseWriter, req *http.Request, params httprouter.Params) {\n\tvar addReq AddPinRequest\n\terr := json.NewDecoder(req.Body).Decode(&addReq)\n\tif err != nil {\n\t\terrResp(writer, fmt.Sprintf(\"unmarshaling req: %s\", err), \"\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\tpin := &PinStatus{\n\t\tRequestID: uuid.NewString(),\n\t\tStatus: statusQueued,\n\t\tCreated: time.Now(),\n\t\tPin: Pin(addReq),\n\t}\n\n\tp.m.Lock()\n\tp.pins = append(p.pins, pin)\n\tp.m.Unlock()\n\n\twriteJSON(writer, &pin, http.StatusAccepted)\n\tp.PinAdded(&addReq, pin)\n}\n\ntype ListPinsResponse struct {\n\tCount int `json:\"count\"`\n\tResults []*PinStatus `json:\"results\"`\n}\n\nfunc (p *PinningService) listPins(writer http.ResponseWriter, req *http.Request, params httprouter.Params) {\n\tq := req.URL.Query()\n\n\tcidStr := q.Get(\"cid\")\n\tname := q.Get(\"name\")\n\tmatch := q.Get(\"match\")\n\tstatus := q.Get(\"status\")\n\tbeforeStr := q.Get(\"before\")\n\tafterStr := q.Get(\"after\")\n\tlimitStr := q.Get(\"limit\")\n\tmetaStr := q.Get(\"meta\")\n\n\tif limitStr == \"\" {\n\t\tlimitStr = \"10\"\n\t}\n\tlimit, err := strconv.Atoi(limitStr)\n\tif err != nil {\n\t\terrResp(writer, fmt.Sprintf(\"parsing limit: %s\", err), \"\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\tvar cids []string\n\tif cidStr != \"\" {\n\t\tcids = strings.Split(cidStr, \",\")\n\t}\n\n\tvar statuses []string\n\tif status != \"\" {\n\t\tstatuses = strings.Split(status, \",\")\n\t}\n\n\tp.m.Lock()\n\tdefer p.m.Unlock()\n\tvar pins []*PinStatus\n\tfor _, pinStatus := range p.pins {\n\t\t// clone it so we can immediately release the lock\n\t\tpinStatus.M.Lock()\n\t\tclonedPS := pinStatus.Clone()\n\t\tpinStatus.M.Unlock()\n\n\t\t// cid\n\t\tvar matchesCID bool\n\t\tif len(cids) == 0 {\n\t\t\tmatchesCID = true\n\t\t} else {\n\t\t\tfor _, cid := range cids {\n\t\t\t\tif cid == clonedPS.Pin.CID {\n\t\t\t\t\tmatchesCID = true\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tif !matchesCID {\n\t\t\tcontinue\n\t\t}\n\n\t\t// name\n\t\tif match == \"\" {\n\t\t\tmatch = matchExact\n\t\t}\n\t\tif name != \"\" {\n\t\t\tswitch match {\n\t\t\tcase matchExact:\n\t\t\t\tif name != clonedPS.Pin.Name {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\tcase matchIExact:\n\t\t\t\tif !strings.EqualFold(name, clonedPS.Pin.Name) {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\tcase matchPartial:\n\t\t\t\tif !strings.Contains(clonedPS.Pin.Name, name) {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\tcase matchIPartial:\n\t\t\t\tif !strings.Contains(strings.ToLower(clonedPS.Pin.Name), strings.ToLower(name)) {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\tdefault:\n\t\t\t\terrResp(writer, fmt.Sprintf(\"unknown match %q\", match), \"\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\n\t\t// status\n\t\tvar matchesStatus bool\n\t\tif len(statuses) == 0 {\n\t\t\tstatuses = []string{statusPinned}\n\t\t}\n\t\tfor _, status := range statuses {\n\t\t\tif status == clonedPS.Status {\n\t\t\t\tmatchesStatus = true\n\t\t\t}\n\t\t}\n\t\tif !matchesStatus {\n\t\t\tcontinue\n\t\t}\n\n\t\t// before\n\t\tif beforeStr != \"\" {\n\t\t\tbefore, err := time.Parse(timeLayout, beforeStr)\n\t\t\tif err != nil {\n\t\t\t\terrResp(writer, fmt.Sprintf(\"parsing before: %s\", err), \"\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !clonedPS.Created.Before(before) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\t// after\n\t\tif afterStr != \"\" {\n\t\t\tafter, err := time.Parse(timeLayout, afterStr)\n\t\t\tif err != nil {\n\t\t\t\terrResp(writer, fmt.Sprintf(\"parsing before: %s\", err), \"\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !clonedPS.Created.After(after) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\t// meta\n\t\tif metaStr != \"\" {\n\t\t\tmeta := map[string]any{}\n\t\t\terr := json.Unmarshal([]byte(metaStr), &meta)\n\t\t\tif err != nil {\n\t\t\t\terrResp(writer, fmt.Sprintf(\"parsing meta: %s\", err), \"\", http.StatusBadRequest)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tvar matchesMeta bool\n\t\t\tfor k, v := range meta {\n\t\t\t\tpinV, contains := clonedPS.Pin.Meta[k]\n\t\t\t\tif !contains || !reflect.DeepEqual(pinV, v) {\n\t\t\t\t\tmatchesMeta = false\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tif !matchesMeta {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\t// add the original pin status, not the cloned one\n\t\tpins = append(pins, pinStatus)\n\n\t\tif len(pins) == limit {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tout := ListPinsResponse{\n\t\tCount: len(pins),\n\t\tResults: pins,\n\t}\n\twriteJSON(writer, out, http.StatusOK)\n}\n\nfunc (p *PinningService) getPin(writer http.ResponseWriter, req *http.Request, params httprouter.Params) {\n\trequestID := params.ByName(\"requestID\")\n\tp.m.Lock()\n\tdefer p.m.Unlock()\n\tfor _, pin := range p.pins {\n\t\tif pin.RequestID == requestID {\n\t\t\twriteJSON(writer, pin, http.StatusOK)\n\t\t\treturn\n\t\t}\n\t}\n\terrResp(writer, \"\", \"\", http.StatusNotFound)\n}\n\nfunc (p *PinningService) replacePin(writer http.ResponseWriter, req *http.Request, params httprouter.Params) {\n\trequestID := params.ByName(\"requestID\")\n\n\tvar replaceReq Pin\n\terr := json.NewDecoder(req.Body).Decode(&replaceReq)\n\tif err != nil {\n\t\terrResp(writer, fmt.Sprintf(\"decoding request: %s\", err), \"\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\tp.m.Lock()\n\tdefer p.m.Unlock()\n\tfor _, pin := range p.pins {\n\t\tif pin.RequestID == requestID {\n\t\t\tpin.M.Lock()\n\t\t\tpin.Pin = replaceReq\n\t\t\tpin.M.Unlock()\n\t\t\twriter.WriteHeader(http.StatusAccepted)\n\t\t\treturn\n\t\t}\n\t}\n\terrResp(writer, \"\", \"\", http.StatusNotFound)\n}\n\nfunc (p *PinningService) removePin(writer http.ResponseWriter, req *http.Request, params httprouter.Params) {\n\trequestID := params.ByName(\"requestID\")\n\n\tp.m.Lock()\n\tdefer p.m.Unlock()\n\n\tfor i, pin := range p.pins {\n\t\tif pin.RequestID == requestID {\n\t\t\tp.pins = append(p.pins[0:i], p.pins[i+1:]...)\n\t\t\twriter.WriteHeader(http.StatusAccepted)\n\t\t\treturn\n\t\t}\n\t}\n\n\terrResp(writer, \"\", \"\", http.StatusNotFound)\n}\n" }, { "path": "test/cli/testutils/protobuf.go", "content": "package testutils\n\nimport \"math/bits\"\n\n// VarintLen returns the number of bytes needed to encode v as a protobuf varint.\nfunc VarintLen(v uint64) int {\n\treturn int(9*uint32(bits.Len64(v))+64) / 64\n}\n\n// LinkSerializedSize calculates the serialized size of a single PBLink in a dag-pb block.\n// This matches the calculation in boxo/ipld/unixfs/io/directory.go estimatedBlockSize().\n//\n// The protobuf wire format for a PBLink is:\n//\n//\tPBNode.Links wrapper tag (1 byte)\n//\t+ varint length of inner message\n//\t+ Hash field: tag (1) + varint(cidLen) + cidLen\n//\t+ Name field: tag (1) + varint(nameLen) + nameLen\n//\t+ Tsize field: tag (1) + varint(tsize)\nfunc LinkSerializedSize(nameLen, cidLen int, tsize uint64) int {\n\t// Inner link message size\n\tlinkLen := 1 + VarintLen(uint64(cidLen)) + cidLen + // Hash field\n\t\t1 + VarintLen(uint64(nameLen)) + nameLen + // Name field\n\t\t1 + VarintLen(tsize) // Tsize field\n\n\t// Outer wrapper: tag (1 byte) + varint(linkLen) + linkLen\n\treturn 1 + VarintLen(uint64(linkLen)) + linkLen\n}\n\n// EstimateFilesForBlockThreshold estimates how many files with given name/cid lengths\n// will fit under the block size threshold.\n// Returns the number of files that keeps the block size just under the threshold.\nfunc EstimateFilesForBlockThreshold(threshold, nameLen, cidLen int, tsize uint64) int {\n\tlinkSize := LinkSerializedSize(nameLen, cidLen, tsize)\n\t// Base overhead for empty directory node (Data field + minimal structure)\n\t// Empirically determined to be 4 bytes for dag-pb directories\n\tbaseOverhead := 4\n\treturn (threshold - baseOverhead) / linkSize\n}\n" }, { "path": "test/cli/testutils/random_deterministic.go", "content": "package testutils\n\nimport (\n\t\"crypto/sha256\"\n\t\"io\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"golang.org/x/crypto/chacha20\"\n)\n\ntype randomReader struct {\n\tcipher *chacha20.Cipher\n\tremaining int64\n}\n\nfunc (r *randomReader) Read(p []byte) (int, error) {\n\tif r.remaining <= 0 {\n\t\treturn 0, io.EOF\n\t}\n\tn := min(int64(len(p)), r.remaining)\n\t// Generate random bytes directly into the provided buffer\n\tr.cipher.XORKeyStream(p[:n], make([]byte, n))\n\tr.remaining -= n\n\treturn int(n), nil\n}\n\n// DeterministicRandomReader produces specified number of pseudo-random bytes\n// from a seed. Size can be specified as a humanize string (e.g., \"256KiB\", \"1MiB\").\nfunc DeterministicRandomReader(sizeStr string, seed string) (io.Reader, error) {\n\tsize, err := humanize.ParseBytes(sizeStr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn DeterministicRandomReaderBytes(int64(size), seed)\n}\n\n// DeterministicRandomReaderBytes produces exactly `size` pseudo-random bytes\n// from a seed. Use this when exact byte precision is needed.\nfunc DeterministicRandomReaderBytes(size int64, seed string) (io.Reader, error) {\n\t// Hash the seed string to a 32-byte key for ChaCha20\n\tkey := sha256.Sum256([]byte(seed))\n\t// Use ChaCha20 for deterministic random bytes\n\tvar nonce [chacha20.NonceSize]byte // Zero nonce for simplicity\n\tcipher, err := chacha20.NewUnauthenticatedCipher(key[:chacha20.KeySize], nonce[:])\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn &randomReader{cipher: cipher, remaining: size}, nil\n}\n" }, { "path": "test/cli/testutils/requires.go", "content": "package testutils\n\nimport (\n\t\"os\"\n\t\"os/exec\"\n\t\"runtime\"\n\t\"testing\"\n)\n\nfunc RequiresDocker(t *testing.T) {\n\tif os.Getenv(\"TEST_DOCKER\") != \"1\" {\n\t\tt.SkipNow()\n\t}\n}\n\nfunc RequiresFUSE(t *testing.T) {\n\t// Skip if FUSE tests are explicitly disabled\n\tif os.Getenv(\"TEST_FUSE\") == \"0\" {\n\t\tt.Skip(\"FUSE tests disabled via TEST_FUSE=0\")\n\t}\n\n\t// If TEST_FUSE=1 is set, always run (for backwards compatibility)\n\tif os.Getenv(\"TEST_FUSE\") == \"1\" {\n\t\treturn\n\t}\n\n\t// Auto-detect FUSE availability based on platform and tools\n\tif !isFUSEAvailable(t) {\n\t\tt.Skip(\"FUSE not available (no fusermount/umount found or unsupported platform)\")\n\t}\n}\n\n// isFUSEAvailable checks if FUSE is available on the current system\nfunc isFUSEAvailable(t *testing.T) bool {\n\tt.Helper()\n\n\t// Check platform support\n\tswitch runtime.GOOS {\n\tcase \"linux\", \"darwin\", \"freebsd\", \"openbsd\", \"netbsd\":\n\t\t// These platforms potentially support FUSE\n\tcase \"windows\":\n\t\t// Windows has limited FUSE support via WinFsp, but skip for now\n\t\treturn false\n\tdefault:\n\t\t// Unknown platform, assume no FUSE support\n\t\treturn false\n\t}\n\n\t// Check for required unmount tools\n\tvar unmountCmd string\n\tif runtime.GOOS == \"linux\" {\n\t\tunmountCmd = \"fusermount\"\n\t} else {\n\t\tunmountCmd = \"umount\"\n\t}\n\n\t_, err := exec.LookPath(unmountCmd)\n\treturn err == nil\n}\n\nfunc RequiresExpensive(t *testing.T) {\n\tif os.Getenv(\"TEST_EXPENSIVE\") == \"1\" || testing.Short() {\n\t\tt.SkipNow()\n\t}\n}\n\nfunc RequiresPlugins(t *testing.T) {\n\tif os.Getenv(\"TEST_PLUGIN\") != \"1\" {\n\t\tt.SkipNow()\n\t}\n}\n\nfunc RequiresLinux(t *testing.T) {\n\tif runtime.GOOS != \"linux\" {\n\t\tt.SkipNow()\n\t}\n}\n" }, { "path": "test/cli/testutils/strings.go", "content": "package testutils\n\nimport (\n\t\"bufio\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/netip\"\n\t\"net/url\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nvar (\n\tAlphabetEasy = []rune(\"abcdefghijklmnopqrstuvwxyz01234567890-_\")\n\tAlphabetHard = []rune(\"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890!@#$%^&*()-_+= ;.,<>'\\\"[]{}() \")\n)\n\n// StrCat takes a bunch of strings or string slices\n// and concats them all together into one string slice.\n// If an arg is not one of those types, this panics.\n// If an arg is an empty string, it is dropped.\nfunc StrCat(args ...any) []string {\n\tres := make([]string, 0)\n\tfor _, a := range args {\n\t\tif s, ok := a.(string); ok {\n\t\t\tif s != \"\" {\n\t\t\t\tres = append(res, s)\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\t\tif ss, ok := a.([]string); ok {\n\t\t\tfor _, s := range ss {\n\t\t\t\tif s != \"\" {\n\t\t\t\t\tres = append(res, s)\n\t\t\t\t}\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\t\tpanic(fmt.Sprintf(\"arg '%v' must be a string or string slice, but is '%T'\", a, a))\n\t}\n\treturn res\n}\n\n// PreviewStr returns a preview of s, which is a prefix for logging that avoids dumping a huge string to logs.\nfunc PreviewStr(s string) string {\n\tsuffix := \"...\"\n\tpreviewLength := 10\n\tif len(s) < previewLength {\n\t\tpreviewLength = len(s)\n\t\tsuffix = \"\"\n\t}\n\treturn s[0:previewLength] + suffix\n}\n\nfunc SplitLines(s string) []string {\n\tvar lines []string\n\tscanner := bufio.NewScanner(strings.NewReader(s))\n\tfor scanner.Scan() {\n\t\tlines = append(lines, scanner.Text())\n\t}\n\treturn lines\n}\n\n// URLStrToMultiaddr converts a URL string like http://localhost:80 to a multiaddr.\nfunc URLStrToMultiaddr(u string) multiaddr.Multiaddr {\n\tparsedURL, err := url.Parse(u)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\taddrPort, err := netip.ParseAddrPort(parsedURL.Host)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\ttcpAddr := net.TCPAddrFromAddrPort(addrPort)\n\tma, err := manet.FromNetAddr(tcpAddr)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn ma\n}\n\n// ForEachPar invokes f in a new goroutine for each element of s and waits for all to complete.\nfunc ForEachPar[T any](s []T, f func(T)) {\n\twg := sync.WaitGroup{}\n\twg.Add(len(s))\n\tfor _, x := range s {\n\t\tgo func(x T) {\n\t\t\tdefer wg.Done()\n\t\t\tf(x)\n\t\t}(x)\n\t}\n\twg.Wait()\n}\n" }, { "path": "test/cli/tracing_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/ipfs/kubo/test/cli/testutils\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nvar otelCollectorConfigYAML = `\nreceivers:\n otlp:\n protocols:\n grpc:\n\nprocessors:\n batch:\n\nexporters:\n file:\n path: /traces/traces.json\n\nservice:\n pipelines:\n traces:\n receivers: [otlp]\n processors: [batch]\n exporters: [file]\n`\n\nfunc TestTracing(t *testing.T) {\n\ttestutils.RequiresDocker(t)\n\tt.Parallel()\n\tnode := harness.NewT(t).NewNode().Init()\n\n\tnode.WriteBytes(\"collector-config.yaml\", []byte(otelCollectorConfigYAML))\n\n\t// touch traces.json and give it 777 perms in case Docker runs as a different user\n\tnode.WriteBytes(\"traces.json\", nil)\n\terr := os.Chmod(filepath.Join(node.Dir, \"traces.json\"), 0o777)\n\trequire.NoError(t, err)\n\n\tdockerBin, err := exec.LookPath(\"docker\")\n\trequire.NoError(t, err)\n\tnode.Runner.MustRun(harness.RunRequest{\n\t\tPath: dockerBin,\n\t\tArgs: []string{\n\t\t\t\"run\",\n\t\t\t\"--rm\",\n\t\t\t\"--detach\",\n\t\t\t\"--volume\", fmt.Sprintf(\"%s:/config.yaml\", filepath.Join(node.Dir, \"collector-config.yaml\")),\n\t\t\t\"--volume\", fmt.Sprintf(\"%s:/traces\", node.Dir),\n\t\t\t\"--net\", \"host\",\n\t\t\t\"--name\", \"ipfs-test-otel-collector\",\n\t\t\t\"otel/opentelemetry-collector-contrib:0.52.0\",\n\t\t\t\"--config\", \"/config.yaml\",\n\t\t},\n\t})\n\n\tt.Cleanup(func() {\n\t\tnode.Runner.MustRun(harness.RunRequest{\n\t\t\tPath: dockerBin,\n\t\t\tArgs: []string{\"stop\", \"ipfs-test-otel-collector\"},\n\t\t})\n\t})\n\n\tnode.Runner.Env[\"OTEL_TRACES_EXPORTER\"] = \"otlp\"\n\tnode.Runner.Env[\"OTEL_EXPORTER_OTLP_PROTOCOL\"] = \"grpc\"\n\tnode.Runner.Env[\"OTEL_EXPORTER_OTLP_ENDPOINT\"] = \"http://localhost:4317\"\n\tnode.StartDaemon()\n\tdefer node.StopDaemon()\n\n\tassert.Eventually(t,\n\t\tfunc() bool {\n\t\t\tb, err := os.ReadFile(filepath.Join(node.Dir, \"traces.json\"))\n\t\t\trequire.NoError(t, err)\n\t\t\treturn strings.Contains(string(b), \"go-ipfs\")\n\t\t},\n\t\t5*time.Minute,\n\t\t10*time.Millisecond,\n\t)\n}\n" }, { "path": "test/cli/transports_test.go", "content": "package cli\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/go-test/random/files\"\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestTransports(t *testing.T) {\n\tdisableRouting := func(nodes harness.Nodes) {\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Routing.Type = config.NewOptionalString(\"none\")\n\t\t\t\tcfg.Bootstrap = nil\n\t\t\t})\n\t\t})\n\t}\n\tcheckSingleFile := func(nodes harness.Nodes) {\n\t\ts := string(random.Bytes(100))\n\t\thash := nodes[0].IPFSAddStr(s)\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tval := n.IPFS(\"cat\", hash).Stdout.String()\n\t\t\tassert.Equal(t, s, val)\n\t\t})\n\t}\n\tcheckRandomDir := func(nodes harness.Nodes) {\n\t\trandDir := filepath.Join(nodes[0].Dir, \"foobar\")\n\t\trequire.NoError(t, os.Mkdir(randDir, 0o777))\n\t\trfCfg := files.DefaultConfig()\n\t\trfCfg.Dirs = 3\n\t\trfCfg.Files = 6\n\t\trfCfg.Depth = 4\n\t\trequire.NoError(t, files.Create(rfCfg, randDir))\n\n\t\thash := nodes[1].IPFS(\"add\", \"-r\", \"-Q\", randDir).Stdout.Trimmed()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tres := n.RunIPFS(\"refs\", \"-r\", hash)\n\t\t\tassert.Equal(t, 0, res.ExitCode())\n\t\t})\n\t}\n\n\trunTests := func(nodes harness.Nodes) {\n\t\tcheckSingleFile(nodes)\n\t\tcheckRandomDir(nodes)\n\t}\n\n\ttcpNodes := func(t *testing.T) harness.Nodes {\n\t\tnodes := harness.NewT(t).NewNodes(2).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Addresses.Swarm = []string{\"/ip4/127.0.0.1/tcp/0\"}\n\t\t\t\tcfg.Swarm.Transports.Network.QUIC = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.Relay = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.WebTransport = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.WebRTCDirect = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.Websocket = config.False\n\t\t\t\t// Disable AutoTLS since we're disabling WebSocket transport\n\t\t\t\tcfg.AutoTLS.Enabled = config.False\n\t\t\t})\n\t\t})\n\t\tdisableRouting(nodes)\n\t\treturn nodes\n\t}\n\n\tt.Run(\"tcp\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := tcpNodes(t).StartDaemons().Connect()\n\t\trunTests(nodes)\n\t\tnodes.StopDaemons()\n\t})\n\n\tt.Run(\"tcp with NOISE\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := tcpNodes(t)\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Swarm.Transports.Security.TLS = config.Disabled\n\t\t\t})\n\t\t})\n\t\tnodes.StartDaemons().Connect()\n\t\trunTests(nodes)\n\t\tnodes.StopDaemons()\n\t})\n\n\tt.Run(\"QUIC\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(5).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Addresses.Swarm = []string{\"/ip4/127.0.0.1/udp/0/quic-v1\"}\n\t\t\t\tcfg.Swarm.Transports.Network.TCP = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.QUIC = config.True\n\t\t\t\tcfg.Swarm.Transports.Network.WebTransport = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.WebRTCDirect = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.Websocket = config.False\n\t\t\t})\n\t\t})\n\t\tdisableRouting(nodes)\n\t\tnodes.StartDaemons().Connect()\n\t\trunTests(nodes)\n\t\tnodes.StopDaemons()\n\t})\n\n\tt.Run(\"QUIC+Webtransport\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(5).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Addresses.Swarm = []string{\"/ip4/127.0.0.1/udp/0/quic-v1/webtransport\"}\n\t\t\t\tcfg.Swarm.Transports.Network.TCP = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.QUIC = config.True\n\t\t\t\tcfg.Swarm.Transports.Network.WebTransport = config.True\n\t\t\t\tcfg.Swarm.Transports.Network.WebRTCDirect = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.Websocket = config.False\n\t\t\t})\n\t\t})\n\t\tdisableRouting(nodes)\n\t\tnodes.StartDaemons().Connect()\n\t\trunTests(nodes)\n\t\tnodes.StopDaemons()\n\t})\n\n\tt.Run(\"QUIC connects with non-dialable transports\", func(t *testing.T) {\n\t\t// This test targets specific Kubo internals which may change later. This checks\n\t\t// if we can announce an address we do not listen on, and then are able to connect\n\t\t// via a different address that is available.\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(5).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\t// We need a specific port to announce so we first generate a random port.\n\t\t\t\t// We can't use 0 here to automatically assign an available port because\n\t\t\t\t// that would only work with Swarm, but not for the announcing.\n\t\t\t\tport := harness.NewRandPort()\n\t\t\t\tquicAddr := fmt.Sprintf(\"/ip4/127.0.0.1/udp/%d/quic-v1\", port)\n\t\t\t\tcfg.Addresses.Swarm = []string{quicAddr}\n\t\t\t\tcfg.Addresses.Announce = []string{quicAddr, quicAddr + \"/webtransport\"}\n\t\t\t})\n\t\t})\n\t\tdisableRouting(nodes)\n\t\tnodes.StartDaemons().Connect()\n\t\trunTests(nodes)\n\t\tnodes.StopDaemons()\n\t})\n\n\tt.Run(\"WebRTC Direct\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnodes := harness.NewT(t).NewNodes(5).Init()\n\t\tnodes.ForEachPar(func(n *harness.Node) {\n\t\t\tn.UpdateConfig(func(cfg *config.Config) {\n\t\t\t\tcfg.Addresses.Swarm = []string{\"/ip4/127.0.0.1/udp/0/webrtc-direct\"}\n\t\t\t\tcfg.Swarm.Transports.Network.TCP = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.QUIC = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.WebTransport = config.False\n\t\t\t\tcfg.Swarm.Transports.Network.WebRTCDirect = config.True\n\t\t\t\tcfg.Swarm.Transports.Network.Websocket = config.False\n\t\t\t})\n\t\t})\n\t\tdisableRouting(nodes)\n\t\tnodes.StartDaemons().Connect()\n\t\trunTests(nodes)\n\t\tnodes.StopDaemons()\n\t})\n}\n" }, { "path": "test/cli/webui_test.go", "content": "package cli\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/config\"\n\t\"github.com/ipfs/kubo/test/cli/harness\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestWebUI(t *testing.T) {\n\tt.Parallel()\n\n\tt.Run(\"NoFetch=true shows not available error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.NoFetch = true\n\t\t})\n\n\t\tnode.StartDaemon(\"--offline\")\n\n\t\tapiClient := node.APIClient()\n\t\tresp := apiClient.Get(\"/webui/\")\n\n\t\t// Should return 503 Service Unavailable when WebUI is not in local store\n\t\tassert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)\n\n\t\t// Check response contains helpful information\n\t\tbody := resp.Body\n\t\tassert.Contains(t, body, \"IPFS WebUI Not Available\")\n\t\tassert.Contains(t, body, \"Gateway.NoFetch=true\")\n\t\tassert.Contains(t, body, \"ipfs pin add\")\n\t\tassert.Contains(t, body, \"ipfs dag import\")\n\t\tassert.Contains(t, body, \"https://github.com/ipfs/ipfs-webui/releases\")\n\t})\n\n\tt.Run(\"DeserializedResponses=false shows incompatible error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.DeserializedResponses = config.False\n\t\t})\n\n\t\tnode.StartDaemon()\n\n\t\tapiClient := node.APIClient()\n\t\tresp := apiClient.Get(\"/webui/\")\n\n\t\t// Should return 503 Service Unavailable\n\t\tassert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)\n\n\t\t// Check response contains incompatibility message\n\t\tbody := resp.Body\n\t\tassert.Contains(t, body, \"IPFS WebUI Incompatible\")\n\t\tassert.Contains(t, body, \"Gateway.DeserializedResponses=false\")\n\t\tassert.Contains(t, body, \"WebUI requires deserializing IPFS responses\")\n\t\tassert.Contains(t, body, \"Gateway.DeserializedResponses=true\")\n\t})\n\n\tt.Run(\"Both NoFetch=true and DeserializedResponses=false shows incompatible error\", func(t *testing.T) {\n\t\tt.Parallel()\n\t\tnode := harness.NewT(t).NewNode().Init()\n\n\t\tnode.UpdateConfig(func(cfg *config.Config) {\n\t\t\tcfg.Gateway.NoFetch = true\n\t\t\tcfg.Gateway.DeserializedResponses = config.False\n\t\t})\n\n\t\tnode.StartDaemon(\"--offline\")\n\n\t\tapiClient := node.APIClient()\n\t\tresp := apiClient.Get(\"/webui/\")\n\n\t\t// Should return 503 Service Unavailable\n\t\tassert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)\n\n\t\t// DeserializedResponses=false takes priority\n\t\tbody := resp.Body\n\t\tassert.Contains(t, body, \"IPFS WebUI Incompatible\")\n\t\tassert.Contains(t, body, \"Gateway.DeserializedResponses=false\")\n\t\t// Should NOT mention NoFetch since DeserializedResponses check comes first\n\t\tassert.NotContains(t, body, \"NoFetch\")\n\t})\n}\n" }, { "path": "test/dependencies/GNUmakefile", "content": "\nall: restore\n\nrestore:\n\t@echo \"*** $@ ***\"\n\twhich godep\n\tmkdir -p tmp_gopath\n\tOLD_GOPATH=\"$$GOPATH\"\n\texport GOPATH=$$(pwd)/tmp_gopath\n\tcd ../..\n\tgodep restore\n\tcd -\n\trm -rf tmp_gopath\n\texport GOPATH=\"$$OLD_GOPATH\"\n\n.PHONY: all restore\n" }, { "path": "test/dependencies/dependencies.go", "content": "//go:build tools\n\npackage tools\n\nimport (\n\t_ \"github.com/Kubuxu/gocovmerge\"\n\t_ \"github.com/golangci/golangci-lint/cmd/golangci-lint\"\n\t_ \"github.com/ipfs/go-cidutil/cid-fmt\"\n\t_ \"github.com/ipfs/go-test/cli/random-data\"\n\t_ \"github.com/ipfs/go-test/cli/random-files\"\n\t_ \"github.com/ipfs/hang-fds\"\n\t_ \"github.com/multiformats/go-multihash/multihash\"\n\t_ \"gotest.tools/gotestsum\"\n)\n" }, { "path": "test/dependencies/go.mod", "content": "module github.com/ipfs/kubo/test/dependencies\n\ngo 1.25.7\n\nreplace github.com/ipfs/kubo => ../../\n\nrequire (\n\tgithub.com/Kubuxu/gocovmerge v0.0.0-20161216165753-7ecaa51963cd\n\tgithub.com/golangci/golangci-lint v1.64.8\n\tgithub.com/ipfs/go-cidutil v0.1.1\n\tgithub.com/ipfs/go-log/v2 v2.9.1\n\tgithub.com/ipfs/go-test v0.2.3\n\tgithub.com/ipfs/hang-fds v0.1.0\n\tgithub.com/ipfs/iptb v1.4.1\n\tgithub.com/ipfs/iptb-plugins v0.5.1\n\tgithub.com/multiformats/go-multiaddr v0.16.1\n\tgithub.com/multiformats/go-multihash v0.2.3\n\tgotest.tools/gotestsum v1.13.0\n)\n\nrequire (\n\t4d63.com/gocheckcompilerdirectives v1.3.0 // indirect\n\t4d63.com/gochecknoglobals v0.2.2 // indirect\n\tgithub.com/4meepo/tagalign v1.4.2 // indirect\n\tgithub.com/Abirdcfly/dupword v0.1.3 // indirect\n\tgithub.com/Antonboom/errname v1.0.0 // indirect\n\tgithub.com/Antonboom/nilnil v1.0.1 // indirect\n\tgithub.com/Antonboom/testifylint v1.5.2 // indirect\n\tgithub.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect\n\tgithub.com/Crocmagnon/fatcontext v0.7.1 // indirect\n\tgithub.com/DataDog/zstd v1.5.7 // indirect\n\tgithub.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24 // indirect\n\tgithub.com/GaijinEntertainment/go-exhaustruct/v3 v3.3.1 // indirect\n\tgithub.com/Jorropo/jsync v1.0.1 // indirect\n\tgithub.com/Masterminds/semver/v3 v3.3.0 // indirect\n\tgithub.com/OpenPeeDeeP/depguard/v2 v2.2.1 // indirect\n\tgithub.com/RaduBerinde/axisds v0.1.0 // indirect\n\tgithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 // indirect\n\tgithub.com/alecthomas/go-check-sumtype v0.3.1 // indirect\n\tgithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect\n\tgithub.com/alexkohler/nakedret/v2 v2.0.5 // indirect\n\tgithub.com/alexkohler/prealloc v1.0.0 // indirect\n\tgithub.com/alingse/asasalint v0.0.11 // indirect\n\tgithub.com/alingse/nilnesserr v0.1.2 // indirect\n\tgithub.com/ashanbrown/forbidigo v1.6.0 // indirect\n\tgithub.com/ashanbrown/makezero v1.2.0 // indirect\n\tgithub.com/benbjohnson/clock v1.3.5 // indirect\n\tgithub.com/beorn7/perks v1.0.1 // indirect\n\tgithub.com/bitfield/gotestdox v0.2.2 // indirect\n\tgithub.com/bkielbasa/cyclop v1.2.3 // indirect\n\tgithub.com/blizzy78/varnamelen v0.8.0 // indirect\n\tgithub.com/bombsimon/wsl/v4 v4.5.0 // indirect\n\tgithub.com/breml/bidichk v0.3.2 // indirect\n\tgithub.com/breml/errchkjson v0.4.0 // indirect\n\tgithub.com/butuzov/ireturn v0.3.1 // indirect\n\tgithub.com/butuzov/mirror v1.3.0 // indirect\n\tgithub.com/caddyserver/certmagic v0.23.0 // indirect\n\tgithub.com/caddyserver/zerossl v0.1.3 // indirect\n\tgithub.com/catenacyber/perfsprint v0.8.2 // indirect\n\tgithub.com/ccojocar/zxcvbn-go v1.0.2 // indirect\n\tgithub.com/cespare/xxhash/v2 v2.3.0 // indirect\n\tgithub.com/charithe/durationcheck v0.0.10 // indirect\n\tgithub.com/chavacava/garif v0.1.0 // indirect\n\tgithub.com/ckaznocha/intrange v0.3.0 // indirect\n\tgithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b // indirect\n\tgithub.com/cockroachdb/errors v1.11.3 // indirect\n\tgithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect\n\tgithub.com/cockroachdb/pebble/v2 v2.1.4 // indirect\n\tgithub.com/cockroachdb/redact v1.1.5 // indirect\n\tgithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b // indirect\n\tgithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect\n\tgithub.com/cpuguy83/go-md2man/v2 v2.0.6 // indirect\n\tgithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf // indirect\n\tgithub.com/curioswitch/go-reassign v0.3.0 // indirect\n\tgithub.com/daixiang0/gci v0.13.5 // indirect\n\tgithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect\n\tgithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect\n\tgithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect\n\tgithub.com/denis-tingaikin/go-header v0.5.0 // indirect\n\tgithub.com/dnephin/pflag v1.0.7 // indirect\n\tgithub.com/dustin/go-humanize v1.0.1 // indirect\n\tgithub.com/ettle/strcase v0.2.0 // indirect\n\tgithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5 // indirect\n\tgithub.com/fatih/color v1.18.0 // indirect\n\tgithub.com/fatih/structtag v1.2.0 // indirect\n\tgithub.com/felixge/httpsnoop v1.0.4 // indirect\n\tgithub.com/filecoin-project/go-clock v0.1.0 // indirect\n\tgithub.com/firefart/nonamedreturns v1.0.5 // indirect\n\tgithub.com/flynn/noise v1.1.0 // indirect\n\tgithub.com/fsnotify/fsnotify v1.9.0 // indirect\n\tgithub.com/fzipp/gocyclo v0.6.0 // indirect\n\tgithub.com/gabriel-vasile/mimetype v1.4.13 // indirect\n\tgithub.com/gammazero/chanqueue v1.1.2 // indirect\n\tgithub.com/gammazero/deque v1.2.1 // indirect\n\tgithub.com/getsentry/sentry-go v0.27.0 // indirect\n\tgithub.com/ghostiam/protogetter v0.3.9 // indirect\n\tgithub.com/go-critic/go-critic v0.12.0 // indirect\n\tgithub.com/go-logr/logr v1.4.3 // indirect\n\tgithub.com/go-logr/stdr v1.2.2 // indirect\n\tgithub.com/go-toolsmith/astcast v1.1.0 // indirect\n\tgithub.com/go-toolsmith/astcopy v1.1.0 // indirect\n\tgithub.com/go-toolsmith/astequal v1.2.0 // indirect\n\tgithub.com/go-toolsmith/astfmt v1.1.0 // indirect\n\tgithub.com/go-toolsmith/astp v1.1.0 // indirect\n\tgithub.com/go-toolsmith/strparse v1.1.0 // indirect\n\tgithub.com/go-toolsmith/typep v1.1.0 // indirect\n\tgithub.com/go-viper/mapstructure/v2 v2.4.0 // indirect\n\tgithub.com/go-xmlfmt/xmlfmt v1.1.3 // indirect\n\tgithub.com/gobwas/glob v0.2.3 // indirect\n\tgithub.com/gofrs/flock v0.12.1 // indirect\n\tgithub.com/gogo/protobuf v1.3.2 // indirect\n\tgithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e // indirect\n\tgithub.com/golangci/dupl v0.0.0-20250308024227-f665c8d69b32 // indirect\n\tgithub.com/golangci/go-printf-func-name v0.1.0 // indirect\n\tgithub.com/golangci/gofmt v0.0.0-20250106114630-d62b90e6713d // indirect\n\tgithub.com/golangci/misspell v0.6.0 // indirect\n\tgithub.com/golangci/plugin-module-register v0.1.1 // indirect\n\tgithub.com/golangci/revgrep v0.8.0 // indirect\n\tgithub.com/golangci/unconvert v0.0.0-20240309020433-c5143eacb3ed // indirect\n\tgithub.com/google/go-cmp v0.7.0 // indirect\n\tgithub.com/google/gopacket v1.1.19 // indirect\n\tgithub.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect\n\tgithub.com/google/uuid v1.6.0 // indirect\n\tgithub.com/gordonklaus/ineffassign v0.1.0 // indirect\n\tgithub.com/gostaticanalysis/analysisutil v0.7.1 // indirect\n\tgithub.com/gostaticanalysis/comment v1.5.0 // indirect\n\tgithub.com/gostaticanalysis/forcetypeassert v0.2.0 // indirect\n\tgithub.com/gostaticanalysis/nilerr v0.1.1 // indirect\n\tgithub.com/hashicorp/go-immutable-radix/v2 v2.1.0 // indirect\n\tgithub.com/hashicorp/go-version v1.8.0 // indirect\n\tgithub.com/hashicorp/golang-lru v1.0.2 // indirect\n\tgithub.com/hashicorp/golang-lru/v2 v2.0.7 // indirect\n\tgithub.com/hashicorp/hcl v1.0.0 // indirect\n\tgithub.com/hexops/gotextdiff v1.0.3 // indirect\n\tgithub.com/huin/goupnp v1.3.0 // indirect\n\tgithub.com/inconshreveable/mousetrap v1.1.0 // indirect\n\tgithub.com/ipfs/bbloom v0.0.4 // indirect\n\tgithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422 // indirect\n\tgithub.com/ipfs/go-bitfield v1.1.0 // indirect\n\tgithub.com/ipfs/go-block-format v0.2.3 // indirect\n\tgithub.com/ipfs/go-cid v0.6.0 // indirect\n\tgithub.com/ipfs/go-datastore v0.9.1 // indirect\n\tgithub.com/ipfs/go-dsqueue v0.2.0 // indirect\n\tgithub.com/ipfs/go-ipfs-cmds v0.16.0 // indirect\n\tgithub.com/ipfs/go-ipfs-redirects-file v0.1.2 // indirect\n\tgithub.com/ipfs/go-ipld-cbor v0.2.1 // indirect\n\tgithub.com/ipfs/go-ipld-format v0.6.3 // indirect\n\tgithub.com/ipfs/go-ipld-legacy v0.3.0 // indirect\n\tgithub.com/ipfs/go-metrics-interface v0.3.0 // indirect\n\tgithub.com/ipfs/go-unixfsnode v1.10.3 // indirect\n\tgithub.com/ipfs/kubo v0.31.0 // indirect\n\tgithub.com/ipld/go-car/v2 v2.16.0 // indirect\n\tgithub.com/ipld/go-codec-dagpb v1.7.0 // indirect\n\tgithub.com/ipld/go-ipld-prime v0.22.0 // indirect\n\tgithub.com/ipshipyard/p2p-forge v0.7.0 // indirect\n\tgithub.com/jackpal/go-nat-pmp v1.0.2 // indirect\n\tgithub.com/jbenet/go-temp-err-catcher v0.1.0 // indirect\n\tgithub.com/jgautheron/goconst v1.7.1 // indirect\n\tgithub.com/jingyugao/rowserrcheck v1.1.1 // indirect\n\tgithub.com/jjti/go-spancheck v0.6.4 // indirect\n\tgithub.com/julz/importas v0.2.0 // indirect\n\tgithub.com/karamaru-alpha/copyloopvar v1.2.1 // indirect\n\tgithub.com/kisielk/errcheck v1.9.0 // indirect\n\tgithub.com/kkHAIKE/contextcheck v1.1.6 // indirect\n\tgithub.com/klauspost/compress v1.18.0 // indirect\n\tgithub.com/klauspost/cpuid/v2 v2.3.0 // indirect\n\tgithub.com/koron/go-ssdp v0.0.6 // indirect\n\tgithub.com/kr/pretty v0.3.1 // indirect\n\tgithub.com/kr/text v0.2.0 // indirect\n\tgithub.com/kulti/thelper v0.6.3 // indirect\n\tgithub.com/kunwardeep/paralleltest v1.0.10 // indirect\n\tgithub.com/lasiar/canonicalheader v1.1.2 // indirect\n\tgithub.com/ldez/exptostd v0.4.2 // indirect\n\tgithub.com/ldez/gomoddirectives v0.6.1 // indirect\n\tgithub.com/ldez/grignotin v0.9.0 // indirect\n\tgithub.com/ldez/tagliatelle v0.7.1 // indirect\n\tgithub.com/ldez/usetesting v0.4.2 // indirect\n\tgithub.com/leonklingele/grouper v1.1.2 // indirect\n\tgithub.com/libdns/libdns v1.0.0-beta.1 // indirect\n\tgithub.com/libp2p/go-buffer-pool v0.1.0 // indirect\n\tgithub.com/libp2p/go-cidranger v1.1.0 // indirect\n\tgithub.com/libp2p/go-doh-resolver v0.5.0 // indirect\n\tgithub.com/libp2p/go-flow-metrics v0.3.0 // indirect\n\tgithub.com/libp2p/go-libp2p v0.48.0 // indirect\n\tgithub.com/libp2p/go-libp2p-asn-util v0.4.1 // indirect\n\tgithub.com/libp2p/go-libp2p-kad-dht v0.39.0 // indirect\n\tgithub.com/libp2p/go-libp2p-kbucket v0.8.0 // indirect\n\tgithub.com/libp2p/go-libp2p-record v0.3.1 // indirect\n\tgithub.com/libp2p/go-libp2p-routing-helpers v0.7.5 // indirect\n\tgithub.com/libp2p/go-msgio v0.3.0 // indirect\n\tgithub.com/libp2p/go-netroute v0.4.0 // indirect\n\tgithub.com/libp2p/go-reuseport v0.4.0 // indirect\n\tgithub.com/macabu/inamedparam v0.1.3 // indirect\n\tgithub.com/magiconair/properties v1.8.7 // indirect\n\tgithub.com/maratori/testableexamples v1.0.0 // indirect\n\tgithub.com/maratori/testpackage v1.1.1 // indirect\n\tgithub.com/matoous/godox v1.1.0 // indirect\n\tgithub.com/mattn/go-colorable v0.1.14 // indirect\n\tgithub.com/mattn/go-isatty v0.0.20 // indirect\n\tgithub.com/mattn/go-runewidth v0.0.16 // indirect\n\tgithub.com/mattn/go-shellwords v1.0.12 // indirect\n\tgithub.com/mgechev/revive v1.7.0 // indirect\n\tgithub.com/mholt/acmez/v3 v3.1.2 // indirect\n\tgithub.com/miekg/dns v1.1.72 // indirect\n\tgithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 // indirect\n\tgithub.com/minio/sha256-simd v1.0.1 // indirect\n\tgithub.com/mitchellh/go-homedir v1.1.0 // indirect\n\tgithub.com/mitchellh/mapstructure v1.5.0 // indirect\n\tgithub.com/moricho/tparallel v0.3.2 // indirect\n\tgithub.com/mr-tron/base58 v1.2.0 // indirect\n\tgithub.com/multiformats/go-base32 v0.1.0 // indirect\n\tgithub.com/multiformats/go-base36 v0.2.0 // indirect\n\tgithub.com/multiformats/go-multiaddr-dns v0.5.0 // indirect\n\tgithub.com/multiformats/go-multiaddr-fmt v0.1.0 // indirect\n\tgithub.com/multiformats/go-multibase v0.2.0 // indirect\n\tgithub.com/multiformats/go-multicodec v0.10.0 // indirect\n\tgithub.com/multiformats/go-multistream v0.6.1 // indirect\n\tgithub.com/multiformats/go-varint v0.1.0 // indirect\n\tgithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect\n\tgithub.com/nakabonne/nestif v0.3.1 // indirect\n\tgithub.com/nishanths/exhaustive v0.12.0 // indirect\n\tgithub.com/nishanths/predeclared v0.2.2 // indirect\n\tgithub.com/nunnatsa/ginkgolinter v0.19.1 // indirect\n\tgithub.com/olekukonko/tablewriter v0.0.5 // indirect\n\tgithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect\n\tgithub.com/pelletier/go-toml/v2 v2.2.3 // indirect\n\tgithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 // indirect\n\tgithub.com/pion/datachannel v1.5.10 // indirect\n\tgithub.com/pion/dtls/v3 v3.1.2 // indirect\n\tgithub.com/pion/ice/v4 v4.0.10 // indirect\n\tgithub.com/pion/interceptor v0.1.40 // indirect\n\tgithub.com/pion/logging v0.2.4 // indirect\n\tgithub.com/pion/mdns/v2 v2.0.7 // indirect\n\tgithub.com/pion/randutil v0.1.0 // indirect\n\tgithub.com/pion/rtcp v1.2.16 // indirect\n\tgithub.com/pion/rtp v1.8.19 // indirect\n\tgithub.com/pion/sctp v1.8.39 // indirect\n\tgithub.com/pion/sdp/v3 v3.0.18 // indirect\n\tgithub.com/pion/srtp/v3 v3.0.6 // indirect\n\tgithub.com/pion/stun/v3 v3.1.1 // indirect\n\tgithub.com/pion/transport/v3 v3.0.7 // indirect\n\tgithub.com/pion/transport/v4 v4.0.1 // indirect\n\tgithub.com/pion/turn/v4 v4.0.2 // indirect\n\tgithub.com/pion/webrtc/v4 v4.1.2 // indirect\n\tgithub.com/pkg/errors v0.9.1 // indirect\n\tgithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect\n\tgithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a // indirect\n\tgithub.com/polyfloyd/go-errorlint v1.7.1 // indirect\n\tgithub.com/prometheus/client_golang v1.23.2 // indirect\n\tgithub.com/prometheus/client_model v0.6.2 // indirect\n\tgithub.com/prometheus/common v0.67.5 // indirect\n\tgithub.com/prometheus/procfs v0.20.1 // indirect\n\tgithub.com/quasilyte/go-ruleguard v0.4.3-0.20240823090925-0fe6f58b47b1 // indirect\n\tgithub.com/quasilyte/go-ruleguard/dsl v0.3.22 // indirect\n\tgithub.com/quasilyte/gogrep v0.5.0 // indirect\n\tgithub.com/quasilyte/regex/syntax v0.0.0-20210819130434-b3f0c404a727 // indirect\n\tgithub.com/quasilyte/stdinfo v0.0.0-20220114132959-f7386bf02567 // indirect\n\tgithub.com/quic-go/quic-go v0.59.0 // indirect\n\tgithub.com/raeperd/recvcheck v0.2.0 // indirect\n\tgithub.com/rivo/uniseg v0.4.7 // indirect\n\tgithub.com/rogpeppe/go-internal v1.14.1 // indirect\n\tgithub.com/russross/blackfriday/v2 v2.1.0 // indirect\n\tgithub.com/ryancurrah/gomodguard v1.3.5 // indirect\n\tgithub.com/ryanrolds/sqlclosecheck v0.5.1 // indirect\n\tgithub.com/sagikazarmark/locafero v0.6.0 // indirect\n\tgithub.com/sagikazarmark/slog-shim v0.1.0 // indirect\n\tgithub.com/sanposhiho/wastedassign/v2 v2.1.0 // indirect\n\tgithub.com/santhosh-tekuri/jsonschema/v6 v6.0.1 // indirect\n\tgithub.com/sashamelentyev/interfacebloat v1.1.0 // indirect\n\tgithub.com/sashamelentyev/usestdlibvars v1.28.0 // indirect\n\tgithub.com/securego/gosec/v2 v2.22.2 // indirect\n\tgithub.com/sirupsen/logrus v1.9.3 // indirect\n\tgithub.com/sivchari/containedctx v1.0.3 // indirect\n\tgithub.com/sivchari/tenv v1.12.1 // indirect\n\tgithub.com/sonatard/noctx v0.1.0 // indirect\n\tgithub.com/sourcegraph/conc v0.3.0 // indirect\n\tgithub.com/sourcegraph/go-diff v0.7.0 // indirect\n\tgithub.com/spaolacci/murmur3 v1.1.0 // indirect\n\tgithub.com/spf13/afero v1.12.0 // indirect\n\tgithub.com/spf13/cast v1.6.0 // indirect\n\tgithub.com/spf13/cobra v1.9.1 // indirect\n\tgithub.com/spf13/pflag v1.0.6 // indirect\n\tgithub.com/spf13/viper v1.19.0 // indirect\n\tgithub.com/ssgreg/nlreturn/v2 v2.2.1 // indirect\n\tgithub.com/stbenjam/no-sprintf-host-port v0.2.0 // indirect\n\tgithub.com/stretchr/objx v0.5.2 // indirect\n\tgithub.com/stretchr/testify v1.11.1 // indirect\n\tgithub.com/subosito/gotenv v1.6.0 // indirect\n\tgithub.com/tdakkota/asciicheck v0.4.1 // indirect\n\tgithub.com/tetafro/godot v1.5.0 // indirect\n\tgithub.com/timakin/bodyclose v0.0.0-20241017074812-ed6a65f985e3 // indirect\n\tgithub.com/timonwong/loggercheck v0.10.1 // indirect\n\tgithub.com/tomarrell/wrapcheck/v2 v2.10.0 // indirect\n\tgithub.com/tommy-muehle/go-mnd/v2 v2.5.1 // indirect\n\tgithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb // indirect\n\tgithub.com/ultraware/funlen v0.2.0 // indirect\n\tgithub.com/ultraware/whitespace v0.2.0 // indirect\n\tgithub.com/urfave/cli v1.22.16 // indirect\n\tgithub.com/uudashr/gocognit v1.2.0 // indirect\n\tgithub.com/uudashr/iface v1.3.1 // indirect\n\tgithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc // indirect\n\tgithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11 // indirect\n\tgithub.com/whyrusleeping/cbor-gen v0.3.1 // indirect\n\tgithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f // indirect\n\tgithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 // indirect\n\tgithub.com/wlynxg/anet v0.0.5 // indirect\n\tgithub.com/xen0n/gosmopolitan v1.2.2 // indirect\n\tgithub.com/yagipy/maintidx v1.0.0 // indirect\n\tgithub.com/yeya24/promlinter v0.3.0 // indirect\n\tgithub.com/ykadowak/zerologlint v0.1.5 // indirect\n\tgithub.com/zeebo/blake3 v0.2.4 // indirect\n\tgitlab.com/bosi/decorder v0.4.2 // indirect\n\tgo-simpler.org/musttag v0.13.0 // indirect\n\tgo-simpler.org/sloglint v0.9.0 // indirect\n\tgo.opentelemetry.io/auto/sdk v1.2.1 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 // indirect\n\tgo.opentelemetry.io/otel v1.42.0 // indirect\n\tgo.opentelemetry.io/otel/metric v1.42.0 // indirect\n\tgo.opentelemetry.io/otel/trace v1.42.0 // indirect\n\tgo.uber.org/automaxprocs v1.6.0 // indirect\n\tgo.uber.org/dig v1.19.0 // indirect\n\tgo.uber.org/fx v1.24.0 // indirect\n\tgo.uber.org/multierr v1.11.0 // indirect\n\tgo.uber.org/zap v1.27.1 // indirect\n\tgo.uber.org/zap/exp v0.3.0 // indirect\n\tgo.yaml.in/yaml/v2 v2.4.4 // indirect\n\tgolang.org/x/crypto v0.49.0 // indirect\n\tgolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90 // indirect\n\tgolang.org/x/exp/typeparams v0.0.0-20250210185358-939b2ce775ac // indirect\n\tgolang.org/x/mod v0.34.0 // indirect\n\tgolang.org/x/net v0.52.0 // indirect\n\tgolang.org/x/sync v0.20.0 // indirect\n\tgolang.org/x/sys v0.42.0 // indirect\n\tgolang.org/x/term v0.41.0 // indirect\n\tgolang.org/x/text v0.35.0 // indirect\n\tgolang.org/x/time v0.12.0 // indirect\n\tgolang.org/x/tools v0.43.0 // indirect\n\tgolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect\n\tgonum.org/v1/gonum v0.17.0 // indirect\n\tgoogle.golang.org/protobuf v1.36.11 // indirect\n\tgopkg.in/ini.v1 v1.67.0 // indirect\n\tgopkg.in/yaml.v2 v2.4.0 // indirect\n\tgopkg.in/yaml.v3 v3.0.1 // indirect\n\thonnef.co/go/tools v0.6.1 // indirect\n\tlukechampine.com/blake3 v1.4.1 // indirect\n\tmvdan.cc/gofumpt v0.7.0 // indirect\n\tmvdan.cc/unparam v0.0.0-20240528143540-8a5130ca722f // indirect\n)\n" }, { "path": "test/dependencies/go.sum", "content": "4d63.com/gocheckcompilerdirectives v1.3.0 h1:Ew5y5CtcAAQeTVKUVFrE7EwHMrTO6BggtEj8BZSjZ3A=\n4d63.com/gocheckcompilerdirectives v1.3.0/go.mod h1:ofsJ4zx2QAuIP/NO/NAh1ig6R1Fb18/GI7RVMwz7kAY=\n4d63.com/gochecknoglobals v0.2.2 h1:H1vdnwnMaZdQW/N+NrkT1SZMTBmcwHe9Vq8lJcYYTtU=\n4d63.com/gochecknoglobals v0.2.2/go.mod h1:lLxwTQjL5eIesRbvnzIP3jZtG140FnTdz+AlMa+ogt0=\ncloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=\ncloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=\ncloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=\ncloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=\ncloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=\ncloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=\ncloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=\ncloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=\ncloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=\ncloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=\ncloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=\ncloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=\ncloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=\ncloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=\ncloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=\ncloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=\ncloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=\ncloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=\ncloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=\ncloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=\ncloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=\ncloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=\ncloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=\ncloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=\ncloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=\ncloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=\ncloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk=\ncloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=\ncloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=\ncloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=\ncloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=\ncloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=\ncloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=\ncloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=\ncloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=\ncloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=\ndmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=\nfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5 h1:JA0fFr+kxpqTdxR9LOBiTWpGNchqmkcsgmdeJZRclZ0=\nfilippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5/go.mod h1:OjOXDNlClLblvXdwgFFOQFJEocLhhtai8vGLy0JCZlI=\nfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b h1:REI1FbdW71yO56Are4XAxD+OS/e+BQsB3gE4mZRQEXY=\nfilippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b/go.mod h1:9nnw1SlYHYuPSo/3wjQzNjSbeHlq2NsKo5iEtfJPWP0=\ngithub.com/4meepo/tagalign v1.4.2 h1:0hcLHPGMjDyM1gHG58cS73aQF8J4TdVR96TZViorO9E=\ngithub.com/4meepo/tagalign v1.4.2/go.mod h1:+p4aMyFM+ra7nb41CnFG6aSDXqRxU/w1VQqScKqDARI=\ngithub.com/Abirdcfly/dupword v0.1.3 h1:9Pa1NuAsZvpFPi9Pqkd93I7LIYRURj+A//dFd5tgBeE=\ngithub.com/Abirdcfly/dupword v0.1.3/go.mod h1:8VbB2t7e10KRNdwTVoxdBaxla6avbhGzb8sCTygUMhw=\ngithub.com/Antonboom/errname v1.0.0 h1:oJOOWR07vS1kRusl6YRSlat7HFnb3mSfMl6sDMRoTBA=\ngithub.com/Antonboom/errname v1.0.0/go.mod h1:gMOBFzK/vrTiXN9Oh+HFs+e6Ndl0eTFbtsRTSRdXyGI=\ngithub.com/Antonboom/nilnil v1.0.1 h1:C3Tkm0KUxgfO4Duk3PM+ztPncTFlOf0b2qadmS0s4xs=\ngithub.com/Antonboom/nilnil v1.0.1/go.mod h1:CH7pW2JsRNFgEh8B2UaPZTEPhCMuFowP/e8Udp9Nnb0=\ngithub.com/Antonboom/testifylint v1.5.2 h1:4s3Xhuv5AvdIgbd8wOOEeo0uZG7PbDKQyKY5lGoQazk=\ngithub.com/Antonboom/testifylint v1.5.2/go.mod h1:vxy8VJ0bc6NavlYqjZfmp6EfqXMtBgQ4+mhCojwC1P8=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=\ngithub.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=\ngithub.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=\ngithub.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=\ngithub.com/Crocmagnon/fatcontext v0.7.1 h1:SC/VIbRRZQeQWj/TcQBS6JmrXcfA+BU4OGSVUt54PjM=\ngithub.com/Crocmagnon/fatcontext v0.7.1/go.mod h1:1wMvv3NXEBJucFGfwOJBxSVWcoIO6emV215SMkW9MFU=\ngithub.com/DataDog/zstd v1.5.7 h1:ybO8RBeh29qrxIhCA9E8gKY6xfONU9T6G6aP9DTKfLE=\ngithub.com/DataDog/zstd v1.5.7/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=\ngithub.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24 h1:sHglBQTwgx+rWPdisA5ynNEsoARbiCBOyGcJM4/OzsM=\ngithub.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24/go.mod h1:4UJr5HIiMZrwgkSPdsjy2uOQExX/WEILpIrO9UPGuXs=\ngithub.com/GaijinEntertainment/go-exhaustruct/v3 v3.3.1 h1:Sz1JIXEcSfhz7fUi7xHnhpIE0thVASYjvosApmHuD2k=\ngithub.com/GaijinEntertainment/go-exhaustruct/v3 v3.3.1/go.mod h1:n/LSCXNuIYqVfBlVXyHfMQkZDdp1/mmxfSjADd3z1Zg=\ngithub.com/Jorropo/jsync v1.0.1 h1:6HgRolFZnsdfzRUj+ImB9og1JYOxQoReSywkHOGSaUU=\ngithub.com/Jorropo/jsync v1.0.1/go.mod h1:jCOZj3vrBCri3bSU3ErUYvevKlnbssrXeCivybS5ABQ=\ngithub.com/Kubuxu/gocovmerge v0.0.0-20161216165753-7ecaa51963cd h1:HNhzThEtZW714v8Eda8sWWRcu9WSzJC+oCyjRjvZgRA=\ngithub.com/Kubuxu/gocovmerge v0.0.0-20161216165753-7ecaa51963cd/go.mod h1:bqoB8kInrTeEtYAwaIXoSRqdwnjQmFhsfusnzyui6yY=\ngithub.com/Masterminds/semver/v3 v3.3.0 h1:B8LGeaivUe71a5qox1ICM/JLl0NqZSW5CHyL+hmvYS0=\ngithub.com/Masterminds/semver/v3 v3.3.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=\ngithub.com/OpenPeeDeeP/depguard/v2 v2.2.1 h1:vckeWVESWp6Qog7UZSARNqfu/cZqvki8zsuj3piCMx4=\ngithub.com/OpenPeeDeeP/depguard/v2 v2.2.1/go.mod h1:q4DKzC4UcVaAvcfd41CZh0PWpGgzrVxUYBlgKNGquUo=\ngithub.com/RaduBerinde/axisds v0.1.0 h1:YItk/RmU5nvlsv/awo2Fjx97Mfpt4JfgtEVAGPrLdz8=\ngithub.com/RaduBerinde/axisds v0.1.0/go.mod h1:UHGJonU9z4YYGKJxSaC6/TNcLOBptpmM5m2Cksbnw0Y=\ngithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 h1:bsU8Tzxr/PNz75ayvCnxKZWEYdLMPDkUgticP4a4Bvk=\ngithub.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54/go.mod h1:0tr7FllbE9gJkHq7CVeeDDFAFKQVy5RnCSSNBOvdqbc=\ngithub.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f h1:JjxwchlOepwsUWcQwD2mLUAGE9aCp0/ehy6yCHFBOvo=\ngithub.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f/go.mod h1:tMDTce/yLLN/SK8gMOxQfnyeMeCg8KGzp0D1cbECEeo=\ngithub.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=\ngithub.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=\ngithub.com/alecthomas/go-check-sumtype v0.3.1 h1:u9aUvbGINJxLVXiFvHUlPEaD7VDULsrxJb4Aq31NLkU=\ngithub.com/alecthomas/go-check-sumtype v0.3.1/go.mod h1:A8TSiN3UPRw3laIgWEUOHHLPa6/r9MtoigdlP5h3K/E=\ngithub.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=\ngithub.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=\ngithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0=\ngithub.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs=\ngithub.com/alexkohler/nakedret/v2 v2.0.5 h1:fP5qLgtwbx9EJE8dGEERT02YwS8En4r9nnZ71RK+EVU=\ngithub.com/alexkohler/nakedret/v2 v2.0.5/go.mod h1:bF5i0zF2Wo2o4X4USt9ntUWve6JbFv02Ff4vlkmS/VU=\ngithub.com/alexkohler/prealloc v1.0.0 h1:Hbq0/3fJPQhNkN0dR95AVrr6R7tou91y0uHG5pOcUuw=\ngithub.com/alexkohler/prealloc v1.0.0/go.mod h1:VetnK3dIgFBBKmg0YnD9F9x6Icjd+9cvfHR56wJVlKE=\ngithub.com/alingse/asasalint v0.0.11 h1:SFwnQXJ49Kx/1GghOFz1XGqHYKp21Kq1nHad/0WQRnw=\ngithub.com/alingse/asasalint v0.0.11/go.mod h1:nCaoMhw7a9kSJObvQyVzNTPBDbNpdocqrSP7t/cW5+I=\ngithub.com/alingse/nilnesserr v0.1.2 h1:Yf8Iwm3z2hUUrP4muWfW83DF4nE3r1xZ26fGWUKCZlo=\ngithub.com/alingse/nilnesserr v0.1.2/go.mod h1:1xJPrXonEtX7wyTq8Dytns5P2hNzoWymVUIaKm4HNFg=\ngithub.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=\ngithub.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=\ngithub.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=\ngithub.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=\ngithub.com/ashanbrown/forbidigo v1.6.0 h1:D3aewfM37Yb3pxHujIPSpTf6oQk9sc9WZi8gerOIVIY=\ngithub.com/ashanbrown/forbidigo v1.6.0/go.mod h1:Y8j9jy9ZYAEHXdu723cUlraTqbzjKF1MUyfOKL+AjcU=\ngithub.com/ashanbrown/makezero v1.2.0 h1:/2Lp1bypdmK9wDIq7uWBlDF1iMUpIIS4A+pF6C9IEUU=\ngithub.com/ashanbrown/makezero v1.2.0/go.mod h1:dxlPhHbDMC6N6xICzFBSK+4njQDdK8euNO0qjQMtGY4=\ngithub.com/benbjohnson/clock v1.3.5 h1:VvXlSJBzZpA/zum6Sj74hxwYI2DIxRWuNIoXAzHZz5o=\ngithub.com/benbjohnson/clock v1.3.5/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=\ngithub.com/bitfield/gotestdox v0.2.2 h1:x6RcPAbBbErKLnapz1QeAlf3ospg8efBsedU93CDsnE=\ngithub.com/bitfield/gotestdox v0.2.2/go.mod h1:D+gwtS0urjBrzguAkTM2wodsTQYFHdpx8eqRJ3N+9pY=\ngithub.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM=\ngithub.com/bkielbasa/cyclop v1.2.3 h1:faIVMIGDIANuGPWH031CZJTi2ymOQBULs9H21HSMa5w=\ngithub.com/bkielbasa/cyclop v1.2.3/go.mod h1:kHTwA9Q0uZqOADdupvcFJQtp/ksSnytRMe8ztxG8Fuo=\ngithub.com/blizzy78/varnamelen v0.8.0 h1:oqSblyuQvFsW1hbBHh1zfwrKe3kcSj0rnXkKzsQ089M=\ngithub.com/blizzy78/varnamelen v0.8.0/go.mod h1:V9TzQZ4fLJ1DSrjVDfl89H7aMnTvKkApdHeyESmyR7k=\ngithub.com/bombsimon/wsl/v4 v4.5.0 h1:iZRsEvDdyhd2La0FVi5k6tYehpOR/R7qIUjmKk7N74A=\ngithub.com/bombsimon/wsl/v4 v4.5.0/go.mod h1:NOQ3aLF4nD7N5YPXMruR6ZXDOAqLoM0GEpLwTdvmOSc=\ngithub.com/breml/bidichk v0.3.2 h1:xV4flJ9V5xWTqxL+/PMFF6dtJPvZLPsyixAoPe8BGJs=\ngithub.com/breml/bidichk v0.3.2/go.mod h1:VzFLBxuYtT23z5+iVkamXO386OB+/sVwZOpIj6zXGos=\ngithub.com/breml/errchkjson v0.4.0 h1:gftf6uWZMtIa/Is3XJgibewBm2ksAQSY/kABDNFTAdk=\ngithub.com/breml/errchkjson v0.4.0/go.mod h1:AuBOSTHyLSaaAFlWsRSuRBIroCh3eh7ZHh5YeelDIk8=\ngithub.com/butuzov/ireturn v0.3.1 h1:mFgbEI6m+9W8oP/oDdfA34dLisRFCj2G6o/yiI1yZrY=\ngithub.com/butuzov/ireturn v0.3.1/go.mod h1:ZfRp+E7eJLC0NQmk1Nrm1LOrn/gQlOykv+cVPdiXH5M=\ngithub.com/butuzov/mirror v1.3.0 h1:HdWCXzmwlQHdVhwvsfBb2Au0r3HyINry3bDWLYXiKoc=\ngithub.com/butuzov/mirror v1.3.0/go.mod h1:AEij0Z8YMALaq4yQj9CPPVYOyJQyiexpQEQgihajRfI=\ngithub.com/caddyserver/certmagic v0.23.0 h1:CfpZ/50jMfG4+1J/u2LV6piJq4HOfO6ppOnOf7DkFEU=\ngithub.com/caddyserver/certmagic v0.23.0/go.mod h1:9mEZIWqqWoI+Gf+4Trh04MOVPD0tGSxtqsxg87hAIH4=\ngithub.com/caddyserver/zerossl v0.1.3 h1:onS+pxp3M8HnHpN5MMbOMyNjmTheJyWRaZYwn+YTAyA=\ngithub.com/caddyserver/zerossl v0.1.3/go.mod h1:CxA0acn7oEGO6//4rtrRjYgEoa4MFw/XofZnrYwGqG4=\ngithub.com/catenacyber/perfsprint v0.8.2 h1:+o9zVmCSVa7M4MvabsWvESEhpsMkhfE7k0sHNGL95yw=\ngithub.com/catenacyber/perfsprint v0.8.2/go.mod h1:q//VWC2fWbcdSLEY1R3l8n0zQCDPdE4IjZwyY1HMunM=\ngithub.com/ccojocar/zxcvbn-go v1.0.2 h1:na/czXU8RrhXO4EZme6eQJLR4PzcGsahsBOAwU6I3Vg=\ngithub.com/ccojocar/zxcvbn-go v1.0.2/go.mod h1:g1qkXtUSvHP8lhHp5GrSmTz6uWALGRMQdw6Qnz/hi60=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=\ngithub.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/charithe/durationcheck v0.0.10 h1:wgw73BiocdBDQPik+zcEoBG/ob8uyBHf2iyoHGPf5w4=\ngithub.com/charithe/durationcheck v0.0.10/go.mod h1:bCWXb7gYRysD1CU3C+u4ceO49LoGOY1C1L6uouGNreQ=\ngithub.com/chavacava/garif v0.1.0 h1:2JHa3hbYf5D9dsgseMKAmc/MZ109otzgNFk5s87H9Pc=\ngithub.com/chavacava/garif v0.1.0/go.mod h1:XMyYCkEL58DF0oyW4qDjjnPWONs2HBqYKI+UIPD+Gww=\ngithub.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=\ngithub.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=\ngithub.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=\ngithub.com/ckaznocha/intrange v0.3.0 h1:VqnxtK32pxgkhJgYQEeOArVidIPg+ahLP7WBOXZd5ZY=\ngithub.com/ckaznocha/intrange v0.3.0/go.mod h1:+I/o2d2A1FBHgGELbGxzIcyd3/9l9DuwjM8FsbSS3Lo=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b h1:SHlYZ/bMx7frnmeqCu+xm0TCxXLzX3jQIVuFbnFGtFU=\ngithub.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b/go.mod h1:Gq51ZeKaFCXk6QwuGM0w1dnaOqc/F5zKT2zA9D6Xeac=\ngithub.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5 h1:UycK/E0TkisVrQbSoxvU827FwgBBcZ95nRRmpj/12QI=\ngithub.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5/go.mod h1:jsaKMvD3RBCATk1/jbUZM8C9idWBJME9+VRZ5+Liq1g=\ngithub.com/cockroachdb/errors v1.11.3 h1:5bA+k2Y6r+oz/6Z/RFlNeVCesGARKuC6YymtcDrbC/I=\ngithub.com/cockroachdb/errors v1.11.3/go.mod h1:m4UIW4CDjx+R5cybPsNrRbreomiFqt8o1h1wUVazSd8=\ngithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZeQy818SGhaone5OnYfxFR/+AzdY3sf5aE=\ngithub.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=\ngithub.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895 h1:XANOgPYtvELQ/h4IrmPAohXqe2pWA8Bwhejr3VQoZsA=\ngithub.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895/go.mod h1:aPd7gM9ov9M8v32Yy5NJrDyOcD8z642dqs+F0CeNXfA=\ngithub.com/cockroachdb/pebble/v2 v2.1.4 h1:j9wPgMDbkErFdAKYFGhsoCcvzcjR+6zrJ4jhKtJ6bOk=\ngithub.com/cockroachdb/pebble/v2 v2.1.4/go.mod h1:Reo1RTniv1UjVTAu/Fv74y5i3kJ5gmVrPhO9UtFiKn8=\ngithub.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30=\ngithub.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=\ngithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b h1:VXvSNzmr8hMj8XTuY0PT9Ane9qZGul/p67vGYwl9BFI=\ngithub.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b/go.mod h1:yBRu/cnL4ks9bgy4vAASdjIW+/xMlFwuHKqtmh3GZQg=\ngithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo=\ngithub.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06/go.mod h1:7nc4anLGjupUW/PeY5qiNYsdNXj7zopG+eqsS7To5IQ=\ngithub.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.5/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.6 h1:XJtiaUW6dEEqVuZiMTn1ldk455QWwEIsMIJlo5vtkx0=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=\ngithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf h1:dwGgBWn84wUS1pVikGiruW+x5XM4amhjaZO20vCjay4=\ngithub.com/crackcomm/go-gitignore v0.0.0-20241020182519-7843d2ba8fdf/go.mod h1:p1d6YEZWvFzEh4KLyvBcVSnrfNDDvK2zfK/4x2v/4pE=\ngithub.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=\ngithub.com/cskr/pubsub v1.0.2 h1:vlOzMhl6PFn60gRlTQQsIfVwaPB/B/8MziK8FhEPt/0=\ngithub.com/cskr/pubsub v1.0.2/go.mod h1:/8MzYXk/NJAz782G8RPkFzXTZVu63VotefPnR9TIRis=\ngithub.com/curioswitch/go-reassign v0.3.0 h1:dh3kpQHuADL3cobV/sSGETA8DOv457dwl+fbBAhrQPs=\ngithub.com/curioswitch/go-reassign v0.3.0/go.mod h1:nApPCCTtqLJN/s8HfItCcKV0jIPwluBOvZP+dsJGA88=\ngithub.com/daixiang0/gci v0.13.5 h1:kThgmH1yBmZSBCh1EJVxQ7JsHpm5Oms0AMed/0LaH4c=\ngithub.com/daixiang0/gci v0.13.5/go.mod h1:12etP2OniiIdP4q+kjUGrC/rUagga7ODbqsom5Eo5Yk=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c h1:pFUpOrbxDR6AkioZ1ySsx5yxlDQZ8stG2b88gTPxgJU=\ngithub.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c/go.mod h1:6UhI8N9EjYm1c2odKpFpAYeR8dsBeM7PtzQhRgxRr9U=\ngithub.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U0x++OzVrdms8=\ngithub.com/decred/dcrd/crypto/blake256 v1.1.0/go.mod h1:2OfgNZ5wDpcsFmHmCK5gZTPcCXqlm2ArzUIkw9czNJo=\ngithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 h1:5RVFMOWjMyRy8cARdy79nAmgYw3hK/4HUq48LQ6Wwqo=\ngithub.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=\ngithub.com/denis-tingaikin/go-header v0.5.0 h1:SRdnP5ZKvcO9KKRP1KJrhFR3RrlGuD+42t4429eC9k8=\ngithub.com/denis-tingaikin/go-header v0.5.0/go.mod h1:mMenU5bWrok6Wl2UsZjy+1okegmwQ3UgWl4V1D8gjlY=\ngithub.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=\ngithub.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=\ngithub.com/dnephin/pflag v1.0.7 h1:oxONGlWxhmUct0YzKTgrpQv9AUA1wtPBn7zuSjJqptk=\ngithub.com/dnephin/pflag v1.0.7/go.mod h1:uxE91IoWURlOiTUIA8Mq5ZZkAv3dPUfZNaT80Zm7OQE=\ngithub.com/dunglas/httpsfv v1.1.0 h1:Jw76nAyKWKZKFrpMMcL76y35tOpYHqQPzHQiwDvpe54=\ngithub.com/dunglas/httpsfv v1.1.0/go.mod h1:zID2mqw9mFsnt7YC3vYQ9/cjq30q41W+1AnDwH8TiMg=\ngithub.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=\ngithub.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/ettle/strcase v0.2.0 h1:fGNiVF21fHXpX1niBgk0aROov1LagYsOwV/xqKDKR/Q=\ngithub.com/ettle/strcase v0.2.0/go.mod h1:DajmHElDSaX76ITe3/VHVyMin4LWSJN5Z909Wp+ED1A=\ngithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5 h1:BBso6MBKW8ncyZLv37o+KNyy0HrrHgfnOaGQC2qvN+A=\ngithub.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5/go.mod h1:JpoxHjuQauoxiFMl1ie8Xc/7TfLuMZ5eOCONd1sUBHg=\ngithub.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=\ngithub.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=\ngithub.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=\ngithub.com/fatih/structtag v1.2.0 h1:/OdNE99OxoI/PqaW/SuSK9uxxT3f/tcSZgon/ssNSx4=\ngithub.com/fatih/structtag v1.2.0/go.mod h1:mBJUNpUnHmRKrKlQQlmCrh5PuhftFbNv8Ys4/aAZl94=\ngithub.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=\ngithub.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/filecoin-project/go-clock v0.1.0 h1:SFbYIM75M8NnFm1yMHhN9Ahy3W5bEZV9gd6MPfXbKVU=\ngithub.com/filecoin-project/go-clock v0.1.0/go.mod h1:4uB/O4PvOjlx1VCMdZ9MyDZXRm//gkj1ELEbxfI1AZs=\ngithub.com/firefart/nonamedreturns v1.0.5 h1:tM+Me2ZaXs8tfdDw3X6DOX++wMCOqzYUho6tUTYIdRA=\ngithub.com/firefart/nonamedreturns v1.0.5/go.mod h1:gHJjDqhGM4WyPt639SOZs+G89Ko7QKH5R5BhnO6xJhw=\ngithub.com/flynn/noise v1.1.0 h1:KjPQoQCEFdZDiP03phOvGi11+SVVhBG2wOWAorLsstg=\ngithub.com/flynn/noise v1.1.0/go.mod h1:xbMo+0i6+IGbYdJhF31t2eR1BIU0CYc12+BNAKwUTag=\ngithub.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=\ngithub.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=\ngithub.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=\ngithub.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=\ngithub.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=\ngithub.com/fzipp/gocyclo v0.6.0 h1:lsblElZG7d3ALtGMx9fmxeTKZaLLpU8mET09yN4BBLo=\ngithub.com/fzipp/gocyclo v0.6.0/go.mod h1:rXPyn8fnlpa0R2csP/31uerbiVBugk5whMdlyaLkLoA=\ngithub.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=\ngithub.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=\ngithub.com/gammazero/chanqueue v1.1.2 h1:dZEsxlyANZMyeTRemABqZF8QM9BnE4NBI43Oh3y5fIU=\ngithub.com/gammazero/chanqueue v1.1.2/go.mod h1:XDN1X/jjAbmSceNFOQbtKToeSkxtdVdpKu90LiEdBEE=\ngithub.com/gammazero/deque v1.2.1 h1:9fnQVFCCZ9/NOc7ccTNqzoKd1tCWOqeI05/lPqFPMGQ=\ngithub.com/gammazero/deque v1.2.1/go.mod h1:5nSFkzVm+afG9+gy0VIowlqVAW4N8zNcMne+CMQVD2g=\ngithub.com/getsentry/sentry-go v0.27.0 h1:Pv98CIbtB3LkMWmXi4Joa5OOcwbmnX88sF5qbK3r3Ps=\ngithub.com/getsentry/sentry-go v0.27.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY=\ngithub.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9 h1:r5GgOLGbza2wVHRzK7aAj6lWZjfbAwiu/RDCVOKjRyM=\ngithub.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9/go.mod h1:106OIgooyS7OzLDOpUGgm9fA3bQENb/cFSyyBmMoJDs=\ngithub.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=\ngithub.com/ghostiam/protogetter v0.3.9 h1:j+zlLLWzqLay22Cz/aYwTHKQ88GE2DQ6GkWSYFOI4lQ=\ngithub.com/ghostiam/protogetter v0.3.9/go.mod h1:WZ0nw9pfzsgxuRsPOFQomgDVSWtDLJRfQJEhsGbmQMA=\ngithub.com/go-critic/go-critic v0.12.0 h1:iLosHZuye812wnkEz1Xu3aBwn5ocCPfc9yqmFG9pa6w=\ngithub.com/go-critic/go-critic v0.12.0/go.mod h1:DpE0P6OVc6JzVYzmM5gq5jMU31zLr4am5mB/VfFK64w=\ngithub.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=\ngithub.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=\ngithub.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=\ngithub.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=\ngithub.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=\ngithub.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=\ngithub.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=\ngithub.com/go-quicktest/qt v1.101.0 h1:O1K29Txy5P2OK0dGo59b7b0LR6wKfIhttaAhHUyn7eI=\ngithub.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow=\ngithub.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=\ngithub.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=\ngithub.com/go-toolsmith/astcast v1.1.0 h1:+JN9xZV1A+Re+95pgnMgDboWNVnIMMQXwfBwLRPgSC8=\ngithub.com/go-toolsmith/astcast v1.1.0/go.mod h1:qdcuFWeGGS2xX5bLM/c3U9lewg7+Zu4mr+xPwZIB4ZU=\ngithub.com/go-toolsmith/astcopy v1.1.0 h1:YGwBN0WM+ekI/6SS6+52zLDEf8Yvp3n2seZITCUBt5s=\ngithub.com/go-toolsmith/astcopy v1.1.0/go.mod h1:hXM6gan18VA1T/daUEHCFcYiW8Ai1tIwIzHY6srfEAw=\ngithub.com/go-toolsmith/astequal v1.0.3/go.mod h1:9Ai4UglvtR+4up+bAD4+hCj7iTo4m/OXVTSLnCyTAx4=\ngithub.com/go-toolsmith/astequal v1.1.0/go.mod h1:sedf7VIdCL22LD8qIvv7Nn9MuWJruQA/ysswh64lffQ=\ngithub.com/go-toolsmith/astequal v1.2.0 h1:3Fs3CYZ1k9Vo4FzFhwwewC3CHISHDnVUPC4x0bI2+Cw=\ngithub.com/go-toolsmith/astequal v1.2.0/go.mod h1:c8NZ3+kSFtFY/8lPso4v8LuJjdJiUFVnSuU3s0qrrDY=\ngithub.com/go-toolsmith/astfmt v1.1.0 h1:iJVPDPp6/7AaeLJEruMsBUlOYCmvg0MoCfJprsOmcco=\ngithub.com/go-toolsmith/astfmt v1.1.0/go.mod h1:OrcLlRwu0CuiIBp/8b5PYF9ktGVZUjlNMV634mhwuQ4=\ngithub.com/go-toolsmith/astp v1.1.0 h1:dXPuCl6u2llURjdPLLDxJeZInAeZ0/eZwFJmqZMnpQA=\ngithub.com/go-toolsmith/astp v1.1.0/go.mod h1:0T1xFGz9hicKs8Z5MfAqSUitoUYS30pDMsRVIDHs8CA=\ngithub.com/go-toolsmith/pkgload v1.2.2 h1:0CtmHq/02QhxcF7E9N5LIFcYFsMR5rdovfqTtRKkgIk=\ngithub.com/go-toolsmith/pkgload v1.2.2/go.mod h1:R2hxLNRKuAsiXCo2i5J6ZQPhnPMOVtU+f0arbFPWCus=\ngithub.com/go-toolsmith/strparse v1.0.0/go.mod h1:YI2nUKP9YGZnL/L1/DLFBfixrcjslWct4wyljWhSRy8=\ngithub.com/go-toolsmith/strparse v1.1.0 h1:GAioeZUK9TGxnLS+qfdqNbA4z0SSm5zVNtCQiyP2Bvw=\ngithub.com/go-toolsmith/strparse v1.1.0/go.mod h1:7ksGy58fsaQkGQlY8WVoBFNyEPMGuJin1rfoPS4lBSQ=\ngithub.com/go-toolsmith/typep v1.1.0 h1:fIRYDyF+JywLfqzyhdiHzRop/GQDxxNhLGQ6gFUNHus=\ngithub.com/go-toolsmith/typep v1.1.0/go.mod h1:fVIw+7zjdsMxDA3ITWnH1yOiw1rnTQKCsF/sk2H/qig=\ngithub.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=\ngithub.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=\ngithub.com/go-xmlfmt/xmlfmt v1.1.3 h1:t8Ey3Uy7jDSEisW2K3somuMKIpzktkWptA0iFCnRUWY=\ngithub.com/go-xmlfmt/xmlfmt v1.1.3/go.mod h1:aUCEOzzezBEjDBbFBoSiya/gduyIiWYRP6CnSFIV8AM=\ngithub.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0=\ngithub.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=\ngithub.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=\ngithub.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=\ngithub.com/gofrs/flock v0.12.1 h1:MTLVXXHf8ekldpJk3AKicLij9MdwOWkZ+a/jHHZby9E=\ngithub.com/gofrs/flock v0.12.1/go.mod h1:9zxTsyu5xtJ9DK+1tFZyibEV7y3uwDxPPfbxeeHCoD0=\ngithub.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=\ngithub.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=\ngithub.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=\ngithub.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=\ngithub.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=\ngithub.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\ngithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e h1:4bw4WeyTYPp0smaXiJZCNnLrvVBqirQVreixayXezGc=\ngithub.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/golangci/dupl v0.0.0-20250308024227-f665c8d69b32 h1:WUvBfQL6EW/40l6OmeSBYQJNSif4O11+bmWEz+C7FYw=\ngithub.com/golangci/dupl v0.0.0-20250308024227-f665c8d69b32/go.mod h1:NUw9Zr2Sy7+HxzdjIULge71wI6yEg1lWQr7Evcu8K0E=\ngithub.com/golangci/go-printf-func-name v0.1.0 h1:dVokQP+NMTO7jwO4bwsRwLWeudOVUPPyAKJuzv8pEJU=\ngithub.com/golangci/go-printf-func-name v0.1.0/go.mod h1:wqhWFH5mUdJQhweRnldEywnR5021wTdZSNgwYceV14s=\ngithub.com/golangci/gofmt v0.0.0-20250106114630-d62b90e6713d h1:viFft9sS/dxoYY0aiOTsLKO2aZQAPT4nlQCsimGcSGE=\ngithub.com/golangci/gofmt v0.0.0-20250106114630-d62b90e6713d/go.mod h1:ivJ9QDg0XucIkmwhzCDsqcnxxlDStoTl89jDMIoNxKY=\ngithub.com/golangci/golangci-lint v1.64.8 h1:y5TdeVidMtBGG32zgSC7ZXTFNHrsJkDnpO4ItB3Am+I=\ngithub.com/golangci/golangci-lint v1.64.8/go.mod h1:5cEsUQBSr6zi8XI8OjmcY2Xmliqc4iYL7YoPrL+zLJ4=\ngithub.com/golangci/misspell v0.6.0 h1:JCle2HUTNWirNlDIAUO44hUsKhOFqGPoC4LZxlaSXDs=\ngithub.com/golangci/misspell v0.6.0/go.mod h1:keMNyY6R9isGaSAu+4Q8NMBwMPkh15Gtc8UCVoDtAWo=\ngithub.com/golangci/plugin-module-register v0.1.1 h1:TCmesur25LnyJkpsVrupv1Cdzo+2f7zX0H6Jkw1Ol6c=\ngithub.com/golangci/plugin-module-register v0.1.1/go.mod h1:TTpqoB6KkwOJMV8u7+NyXMrkwwESJLOkfl9TxR1DGFc=\ngithub.com/golangci/revgrep v0.8.0 h1:EZBctwbVd0aMeRnNUsFogoyayvKHyxlV3CdUA46FX2s=\ngithub.com/golangci/revgrep v0.8.0/go.mod h1:U4R/s9dlXZsg8uJmaR1GrloUr14D7qDl8gi2iPXJH8k=\ngithub.com/golangci/unconvert v0.0.0-20240309020433-c5143eacb3ed h1:IURFTjxeTfNFP0hTEi1YKjB/ub8zkpaOqFFMApi2EAs=\ngithub.com/golangci/unconvert v0.0.0-20240309020433-c5143eacb3ed/go.mod h1:XLXN8bNw4CGRPaqgl3bv/lhz7bsGPh4/xSaMTbo2vkQ=\ngithub.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=\ngithub.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=\ngithub.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=\ngithub.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=\ngithub.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo=\ngithub.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=\ngithub.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20250607225305-033d6d78b36a h1://KbezygeMJZCSHH+HgUZiTeSoiuFspbMg1ge+eFj18=\ngithub.com/google/pprof v0.0.0-20250607225305-033d6d78b36a/go.mod h1:5hDyRhoBCxViHszMt12TnOpEI4VVi+U8Gm9iphldiMA=\ngithub.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=\ngithub.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=\ngithub.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=\ngithub.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=\ngithub.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=\ngithub.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=\ngithub.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=\ngithub.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g=\ngithub.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k=\ngithub.com/gordonklaus/ineffassign v0.1.0 h1:y2Gd/9I7MdY1oEIt+n+rowjBNDcLQq3RsH5hwJd0f9s=\ngithub.com/gordonklaus/ineffassign v0.1.0/go.mod h1:Qcp2HIAYhR7mNUVSIxZww3Guk4it82ghYcEXIAk+QT0=\ngithub.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=\ngithub.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=\ngithub.com/gostaticanalysis/analysisutil v0.7.1 h1:ZMCjoue3DtDWQ5WyU16YbjbQEQ3VuzwxALrpYd+HeKk=\ngithub.com/gostaticanalysis/analysisutil v0.7.1/go.mod h1:v21E3hY37WKMGSnbsw2S/ojApNWb6C1//mXO48CXbVc=\ngithub.com/gostaticanalysis/comment v1.4.1/go.mod h1:ih6ZxzTHLdadaiSnF5WY3dxUoXfXAlTaRzuaNDlSado=\ngithub.com/gostaticanalysis/comment v1.4.2/go.mod h1:KLUTGDv6HOCotCH8h2erHKmpci2ZoR8VPu34YA2uzdM=\ngithub.com/gostaticanalysis/comment v1.5.0 h1:X82FLl+TswsUMpMh17srGRuKaaXprTaytmEpgnKIDu8=\ngithub.com/gostaticanalysis/comment v1.5.0/go.mod h1:V6eb3gpCv9GNVqb6amXzEUX3jXLVK/AdA+IrAMSqvEc=\ngithub.com/gostaticanalysis/forcetypeassert v0.2.0 h1:uSnWrrUEYDr86OCxWa4/Tp2jeYDlogZiZHzGkWFefTk=\ngithub.com/gostaticanalysis/forcetypeassert v0.2.0/go.mod h1:M5iPavzE9pPqWyeiVXSFghQjljW1+l/Uke3PXHS6ILY=\ngithub.com/gostaticanalysis/nilerr v0.1.1 h1:ThE+hJP0fEp4zWLkWHWcRyI2Od0p7DlgYG3Uqrmrcpk=\ngithub.com/gostaticanalysis/nilerr v0.1.1/go.mod h1:wZYb6YI5YAxxq0i1+VJbY0s2YONW0HU0GPE3+5PWN4A=\ngithub.com/gostaticanalysis/testutil v0.3.1-0.20210208050101-bfb5c8eec0e4/go.mod h1:D+FIZ+7OahH3ePw/izIEeH5I06eKs1IKI4Xr64/Am3M=\ngithub.com/gostaticanalysis/testutil v0.5.0 h1:Dq4wT1DdTwTGCQQv3rl3IvD5Ld0E6HiY+3Zh0sUGqw8=\ngithub.com/gostaticanalysis/testutil v0.5.0/go.mod h1:OLQSbuM6zw2EvCcXTz1lVq5unyoNft372msDY0nY5Hs=\ngithub.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=\ngithub.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=\ngithub.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=\ngithub.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=\ngithub.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=\ngithub.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=\ngithub.com/hashicorp/go-immutable-radix/v2 v2.1.0 h1:CUW5RYIcysz+D3B+l1mDeXrQ7fUvGGCwJfdASSzbrfo=\ngithub.com/hashicorp/go-immutable-radix/v2 v2.1.0/go.mod h1:hgdqLXA4f6NIjRVisM1TJ9aOJVNRqKZj+xDGF6m7PBw=\ngithub.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=\ngithub.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=\ngithub.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=\ngithub.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=\ngithub.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=\ngithub.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=\ngithub.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-version v1.2.1/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=\ngithub.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4=\ngithub.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=\ngithub.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=\ngithub.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=\ngithub.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=\ngithub.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=\ngithub.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=\ngithub.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=\ngithub.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=\ngithub.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=\ngithub.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=\ngithub.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=\ngithub.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=\ngithub.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc=\ngithub.com/huin/goupnp v1.3.0/go.mod h1:gnGPsThkYa7bFi/KWmEysQRf48l2dvR5bxr2OFckNX8=\ngithub.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=\ngithub.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=\ngithub.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=\ngithub.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=\ngithub.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=\ngithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422 h1:yY3ot/DU1bqTzHDBARACM76Tbx9s4xzcRbzifG1e/es=\ngithub.com/ipfs/boxo v0.37.1-0.20260317235537-851246983422/go.mod h1:8yyiRn54F2CsW13n0zwXEPrVsZix/gFj9SYIRYMZ6KE=\ngithub.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=\ngithub.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=\ngithub.com/ipfs/go-block-format v0.2.3 h1:mpCuDaNXJ4wrBJLrtEaGFGXkferrw5eqVvzaHhtFKQk=\ngithub.com/ipfs/go-block-format v0.2.3/go.mod h1:WJaQmPAKhD3LspLixqlqNFxiZ3BZ3xgqxxoSR/76pnA=\ngithub.com/ipfs/go-cid v0.6.0 h1:DlOReBV1xhHBhhfy/gBNNTSyfOM6rLiIx9J7A4DGf30=\ngithub.com/ipfs/go-cid v0.6.0/go.mod h1:NC4kS1LZjzfhK40UGmpXv5/qD2kcMzACYJNntCUiDhQ=\ngithub.com/ipfs/go-cidutil v0.1.1 h1:COuby6H8C2ml0alvHYX3WdbFM4F07YtbY0UlT5j+sgI=\ngithub.com/ipfs/go-cidutil v0.1.1/go.mod h1:SCoUftGEUgoXe5Hjeyw5CiLZF8cwYn/TbtpFQXJCP6k=\ngithub.com/ipfs/go-datastore v0.9.1 h1:67Po2epre/o0UxrmkzdS9ZTe2GFGODgTd2odx8Wh6Yo=\ngithub.com/ipfs/go-datastore v0.9.1/go.mod h1:zi07Nvrpq1bQwSkEnx3bfjz+SQZbdbWyCNvyxMh9pN0=\ngithub.com/ipfs/go-detect-race v0.0.1 h1:qX/xay2W3E4Q1U7d9lNs1sU9nvguX0a7319XbyQ6cOk=\ngithub.com/ipfs/go-detect-race v0.0.1/go.mod h1:8BNT7shDZPo99Q74BpGMK+4D8Mn4j46UU0LZ723meps=\ngithub.com/ipfs/go-ds-leveldb v0.5.2 h1:6nmxlQ2zbp4LCNdJVsmHfs9GP0eylfBNxpmY1csp0x0=\ngithub.com/ipfs/go-ds-leveldb v0.5.2/go.mod h1:2fAwmcvD3WoRT72PzEekHBkQmBDhc39DJGoREiuGmYo=\ngithub.com/ipfs/go-dsqueue v0.2.0 h1:MBi9w3oSiX98Xc+Y7NuJ9G8MI6mAT4IGdO9dHEMCZzU=\ngithub.com/ipfs/go-dsqueue v0.2.0/go.mod h1:8FfNQC4DMF/KkzBXRNB9Rb3MKDW0Sh98HMtXYl1mLQE=\ngithub.com/ipfs/go-ipfs-cmds v0.16.0 h1:Oq39Gzz3pWrPwP25SjbfBQugFjKyjFdsHlAMIXdzYzM=\ngithub.com/ipfs/go-ipfs-cmds v0.16.0/go.mod h1:iRNtY9ipM/vH0Yr+/0FY+JfT+trZDQIztDoesmmoTo4=\ngithub.com/ipfs/go-ipfs-delay v0.0.1 h1:r/UXYyRcddO6thwOnhiznIAiSvxMECGgtv35Xs1IeRQ=\ngithub.com/ipfs/go-ipfs-delay v0.0.1/go.mod h1:8SP1YXK1M1kXuc4KJZINY3TQQ03J2rwBG9QfXmbRPrw=\ngithub.com/ipfs/go-ipfs-pq v0.0.4 h1:U7jjENWJd1jhcrR8X/xHTaph14PTAK9O+yaLJbjqgOw=\ngithub.com/ipfs/go-ipfs-pq v0.0.4/go.mod h1:9UdLOIIb99IFrgT0Fc53pvbvlJBhpUb4GJuAQf3+O2A=\ngithub.com/ipfs/go-ipfs-redirects-file v0.1.2 h1:QCK7VtL91FH17KROVVy5KrzDx2hu68QvB2FTWk08ZQk=\ngithub.com/ipfs/go-ipfs-redirects-file v0.1.2/go.mod h1:yIiTlLcDEM/8lS6T3FlCEXZktPPqSOyuY6dEzVqw7Fw=\ngithub.com/ipfs/go-ipld-cbor v0.2.1 h1:H05yEJbK/hxg0uf2AJhyerBDbjOuHX4yi+1U/ogRa7E=\ngithub.com/ipfs/go-ipld-cbor v0.2.1/go.mod h1:x9Zbeq8CoE5R2WicYgBMcr/9mnkQ0lHddYWJP2sMV3A=\ngithub.com/ipfs/go-ipld-format v0.6.3 h1:9/lurLDTotJpZSuL++gh3sTdmcFhVkCwsgx2+rAh4j8=\ngithub.com/ipfs/go-ipld-format v0.6.3/go.mod h1:74ilVN12NXVMIV+SrBAyC05UJRk0jVvGqdmrcYZvCBk=\ngithub.com/ipfs/go-ipld-legacy v0.3.0 h1:7XhFKkRyCvP5upOlQfKUFIqL3S5DEZnbUE4bQmQ/tNE=\ngithub.com/ipfs/go-ipld-legacy v0.3.0/go.mod h1:Ukef9ARQiX+RVetwH2XiReLgJvQDEXcUPszrZ1KRjKI=\ngithub.com/ipfs/go-log/v2 v2.9.1 h1:3JXwHWU31dsCpvQ+7asz6/QsFJHqFr4gLgQ0FWteujk=\ngithub.com/ipfs/go-log/v2 v2.9.1/go.mod h1:evFx7sBiohUN3AG12mXlZBw5hacBQld3ZPHrowlJYoo=\ngithub.com/ipfs/go-metrics-interface v0.3.0 h1:YwG7/Cy4R94mYDUuwsBfeziJCVm9pBMJ6q/JR9V40TU=\ngithub.com/ipfs/go-metrics-interface v0.3.0/go.mod h1:OxxQjZDGocXVdyTPocns6cOLwHieqej/jos7H4POwoY=\ngithub.com/ipfs/go-peertaskqueue v0.8.3 h1:tBPpGJy+A92RqtRFq5amJn0Uuj8Pw8tXi0X3eHfHM8w=\ngithub.com/ipfs/go-peertaskqueue v0.8.3/go.mod h1:OqVync4kPOcXEGdj/LKvox9DCB5mkSBeXsPczCxLtYA=\ngithub.com/ipfs/go-test v0.2.3 h1:Z/jXNAReQFtCYyn7bsv/ZqUwS6E7iIcSpJ2CuzCvnrc=\ngithub.com/ipfs/go-test v0.2.3/go.mod h1:QW8vSKkwYvWFwIZQLGQXdkt9Ud76eQXRQ9Ao2H+cA1o=\ngithub.com/ipfs/go-unixfsnode v1.10.3 h1:c8sJjuGNkxXAQH75P+f5ngPda/9T+DrboVA0TcDGvGI=\ngithub.com/ipfs/go-unixfsnode v1.10.3/go.mod h1:2Jlc7DoEwr12W+7l8Hr6C7XF4NHST3gIkqSArLhGSxU=\ngithub.com/ipfs/hang-fds v0.1.0 h1:deBiFlWHsVGzJ0ZMaqscEqRM1r2O1rFZ59UiQXb1Xko=\ngithub.com/ipfs/hang-fds v0.1.0/go.mod h1:29VLWOn3ftAgNNgXg/al7b11UzuQ+w7AwtCGcTaWkbM=\ngithub.com/ipfs/iptb v1.4.1 h1:faXd3TKGPswbHyZecqqg6UfbES7RDjTKQb+6VFPKDUo=\ngithub.com/ipfs/iptb v1.4.1/go.mod h1:nTsBMtVYFEu0FjC5DgrErnABm3OG9ruXkFXGJoTV5OA=\ngithub.com/ipfs/iptb-plugins v0.5.1 h1:11PNTNEt2+SFxjUcO5qpyCTXqDj6T8Tx9pU/G4ytCIQ=\ngithub.com/ipfs/iptb-plugins v0.5.1/go.mod h1:mscJAjRnu4g16QK6oUBn9RGpcp8ueJmLfmPxIG/At78=\ngithub.com/ipld/go-car/v2 v2.16.0 h1:LWe0vmN/QcQmUU4tr34W5Nv5mNraW+G6jfN2s+ndBco=\ngithub.com/ipld/go-car/v2 v2.16.0/go.mod h1:RqFGWN9ifcXVmCrTAVnfnxiWZk1+jIx67SYhenlmL34=\ngithub.com/ipld/go-codec-dagpb v1.7.0 h1:hpuvQjCSVSLnTnHXn+QAMR0mLmb1gA6wl10LExo2Ts0=\ngithub.com/ipld/go-codec-dagpb v1.7.0/go.mod h1:rD3Zg+zub9ZnxcLwfol/OTQRVjaLzXypgy4UqHQvilM=\ngithub.com/ipld/go-ipld-prime v0.22.0 h1:YJhDhjEOvOYaqshd3b4atIWUoRg/rKrgmwCyUHwlbuY=\ngithub.com/ipld/go-ipld-prime v0.22.0/go.mod h1:ol7vKxOOVgEh0iAPuiDalM+0gScXVMA5ZZa4DVrTnEA=\ngithub.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20250821084354-a425e60cd714 h1:cqNk8PEwHnK0vqWln+U/YZhQc9h2NB3KjUjDPZo5Q2s=\ngithub.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20250821084354-a425e60cd714/go.mod h1:ZEUdra3CoqRVRYgAX/jAJO9aZGz6SKtKEG628fHHktY=\ngithub.com/ipshipyard/p2p-forge v0.7.0 h1:PQayexxZC1FR2Vx0XOSbmZ6wDPliidS48I+xXWuF+YU=\ngithub.com/ipshipyard/p2p-forge v0.7.0/go.mod h1:i2wg0p7WmHGyo5vYaK9COZBp8BN5Drncfu3WoQNZlQY=\ngithub.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=\ngithub.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=\ngithub.com/jbenet/go-temp-err-catcher v0.1.0 h1:zpb3ZH6wIE8Shj2sKS+khgRvf7T7RABoLk/+KKHggpk=\ngithub.com/jbenet/go-temp-err-catcher v0.1.0/go.mod h1:0kJRvmDZXNMIiJirNPEYfhpPwbGVtZVWC34vc5WLsDk=\ngithub.com/jgautheron/goconst v1.7.1 h1:VpdAG7Ca7yvvJk5n8dMwQhfEZJh95kl/Hl9S1OI5Jkk=\ngithub.com/jgautheron/goconst v1.7.1/go.mod h1:aAosetZ5zaeC/2EfMeRswtxUFBpe2Hr7HzkgX4fanO4=\ngithub.com/jingyugao/rowserrcheck v1.1.1 h1:zibz55j/MJtLsjP1OF4bSdgXxwL1b+Vn7Tjzq7gFzUs=\ngithub.com/jingyugao/rowserrcheck v1.1.1/go.mod h1:4yvlZSDb3IyDTUZJUmpZfm2Hwok+Dtp+nu2qOq+er9c=\ngithub.com/jjti/go-spancheck v0.6.4 h1:Tl7gQpYf4/TMU7AT84MN83/6PutY21Nb9fuQjFTpRRc=\ngithub.com/jjti/go-spancheck v0.6.4/go.mod h1:yAEYdKJ2lRkDA8g7X+oKUHXOWVAXSBJRv04OhF+QUjk=\ngithub.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=\ngithub.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=\ngithub.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=\ngithub.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=\ngithub.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=\ngithub.com/julz/importas v0.2.0 h1:y+MJN/UdL63QbFJHws9BVC5RpA2iq0kpjrFajTGivjQ=\ngithub.com/julz/importas v0.2.0/go.mod h1:pThlt589EnCYtMnmhmRYY/qn9lCf/frPOK+WMx3xiJY=\ngithub.com/karamaru-alpha/copyloopvar v1.2.1 h1:wmZaZYIjnJ0b5UoKDjUHrikcV0zuPyyxI4SVplLd2CI=\ngithub.com/karamaru-alpha/copyloopvar v1.2.1/go.mod h1:nFmMlFNlClC2BPvNaHMdkirmTJxVCY0lhxBtlfOypMM=\ngithub.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=\ngithub.com/kisielk/errcheck v1.9.0 h1:9xt1zI9EBfcYBvdU1nVrzMzzUPUtPKs9bVSIM3TAb3M=\ngithub.com/kisielk/errcheck v1.9.0/go.mod h1:kQxWMMVZgIkDq7U8xtG/n2juOjbLgZtedi0D+/VL/i8=\ngithub.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=\ngithub.com/kkHAIKE/contextcheck v1.1.6 h1:7HIyRcnyzxL9Lz06NGhiKvenXq7Zw6Q0UQu/ttjfJCE=\ngithub.com/kkHAIKE/contextcheck v1.1.6/go.mod h1:3dDbMRNBFaq8HFXWC1JyvDSPm43CmE6IuHam8Wr0rkg=\ngithub.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=\ngithub.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=\ngithub.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=\ngithub.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=\ngithub.com/koron/go-ssdp v0.0.6 h1:Jb0h04599eq/CY7rB5YEqPS83HmRfHP2azkxMN2rFtU=\ngithub.com/koron/go-ssdp v0.0.6/go.mod h1:0R9LfRJGek1zWTjN3JUNlm5INCDYGpRDfAptnct63fI=\ngithub.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=\ngithub.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=\ngithub.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=\ngithub.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=\ngithub.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=\ngithub.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=\ngithub.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=\ngithub.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=\ngithub.com/kulti/thelper v0.6.3 h1:ElhKf+AlItIu+xGnI990no4cE2+XaSu1ULymV2Yulxs=\ngithub.com/kulti/thelper v0.6.3/go.mod h1:DsqKShOvP40epevkFrvIwkCMNYxMeTNjdWL4dqWHZ6I=\ngithub.com/kunwardeep/paralleltest v1.0.10 h1:wrodoaKYzS2mdNVnc4/w31YaXFtsc21PCTdvWJ/lDDs=\ngithub.com/kunwardeep/paralleltest v1.0.10/go.mod h1:2C7s65hONVqY7Q5Efj5aLzRCNLjw2h4eMc9EcypGjcY=\ngithub.com/lasiar/canonicalheader v1.1.2 h1:vZ5uqwvDbyJCnMhmFYimgMZnJMjwljN5VGY0VKbMXb4=\ngithub.com/lasiar/canonicalheader v1.1.2/go.mod h1:qJCeLFS0G/QlLQ506T+Fk/fWMa2VmBUiEI2cuMK4djI=\ngithub.com/ldez/exptostd v0.4.2 h1:l5pOzHBz8mFOlbcifTxzfyYbgEmoUqjxLFHZkjlbHXs=\ngithub.com/ldez/exptostd v0.4.2/go.mod h1:iZBRYaUmcW5jwCR3KROEZ1KivQQp6PHXbDPk9hqJKCQ=\ngithub.com/ldez/gomoddirectives v0.6.1 h1:Z+PxGAY+217f/bSGjNZr/b2KTXcyYLgiWI6geMBN2Qc=\ngithub.com/ldez/gomoddirectives v0.6.1/go.mod h1:cVBiu3AHR9V31em9u2kwfMKD43ayN5/XDgr+cdaFaKs=\ngithub.com/ldez/grignotin v0.9.0 h1:MgOEmjZIVNn6p5wPaGp/0OKWyvq42KnzAt/DAb8O4Ow=\ngithub.com/ldez/grignotin v0.9.0/go.mod h1:uaVTr0SoZ1KBii33c47O1M8Jp3OP3YDwhZCmzT9GHEk=\ngithub.com/ldez/tagliatelle v0.7.1 h1:bTgKjjc2sQcsgPiT902+aadvMjCeMHrY7ly2XKFORIk=\ngithub.com/ldez/tagliatelle v0.7.1/go.mod h1:3zjxUpsNB2aEZScWiZTHrAXOl1x25t3cRmzfK1mlo2I=\ngithub.com/ldez/usetesting v0.4.2 h1:J2WwbrFGk3wx4cZwSMiCQQ00kjGR0+tuuyW0Lqm4lwA=\ngithub.com/ldez/usetesting v0.4.2/go.mod h1:eEs46T3PpQ+9RgN9VjpY6qWdiw2/QmfiDeWmdZdrjIQ=\ngithub.com/leonklingele/grouper v1.1.2 h1:o1ARBDLOmmasUaNDesWqWCIFH3u7hoFlM84YrjT3mIY=\ngithub.com/leonklingele/grouper v1.1.2/go.mod h1:6D0M/HVkhs2yRKRFZUoGjeDy7EZTfFBE9gl4kjmIGkA=\ngithub.com/libdns/libdns v1.0.0-beta.1 h1:KIf4wLfsrEpXpZ3vmc/poM8zCATXT2klbdPe6hyOBjQ=\ngithub.com/libdns/libdns v1.0.0-beta.1/go.mod h1:4Bj9+5CQiNMVGf87wjX4CY3HQJypUHRuLvlsfsZqLWQ=\ngithub.com/libp2p/go-buffer-pool v0.1.0 h1:oK4mSFcQz7cTQIfqbe4MIj9gLW+mnanjyFtc6cdF0Y8=\ngithub.com/libp2p/go-buffer-pool v0.1.0/go.mod h1:N+vh8gMqimBzdKkSMVuydVDq+UV5QTWy5HSiZacSbPg=\ngithub.com/libp2p/go-cidranger v1.1.0 h1:ewPN8EZ0dd1LSnrtuwd4709PXVcITVeuwbag38yPW7c=\ngithub.com/libp2p/go-cidranger v1.1.0/go.mod h1:KWZTfSr+r9qEo9OkI9/SIEeAtw+NNoU0dXIXt15Okic=\ngithub.com/libp2p/go-doh-resolver v0.5.0 h1:4h7plVVW+XTS+oUBw2+8KfoM1jF6w8XmO7+skhePFdE=\ngithub.com/libp2p/go-doh-resolver v0.5.0/go.mod h1:aPDxfiD2hNURgd13+hfo29z9IC22fv30ee5iM31RzxU=\ngithub.com/libp2p/go-flow-metrics v0.3.0 h1:q31zcHUvHnwDO0SHaukewPYgwOBSxtt830uJtUx6784=\ngithub.com/libp2p/go-flow-metrics v0.3.0/go.mod h1:nuhlreIwEguM1IvHAew3ij7A8BMlyHQJ279ao24eZZo=\ngithub.com/libp2p/go-libp2p v0.48.0 h1:h2BrLAgrj7X8bEN05K7qmrjpNHYA+6tnsGRdprjTnvo=\ngithub.com/libp2p/go-libp2p v0.48.0/go.mod h1:Q1fBZNdmC2Hf82husCTfkKJVfHm2we5zk+NWmOGEmWk=\ngithub.com/libp2p/go-libp2p-asn-util v0.4.1 h1:xqL7++IKD9TBFMgnLPZR6/6iYhawHKHl950SO9L6n94=\ngithub.com/libp2p/go-libp2p-asn-util v0.4.1/go.mod h1:d/NI6XZ9qxw67b4e+NgpQexCIiFYJjErASrYW4PFDN8=\ngithub.com/libp2p/go-libp2p-kad-dht v0.39.0 h1:mww38eBYiUvdsu+Xl/GLlBC0Aa8M+5HAwvafkFOygAM=\ngithub.com/libp2p/go-libp2p-kad-dht v0.39.0/go.mod h1:Po2JugFEkDq9Vig/JXtc153ntOi0q58o4j7IuITCOVs=\ngithub.com/libp2p/go-libp2p-kbucket v0.8.0 h1:QAK7RzKJpYe+EuSEATAaaHYMYLkPDGC18m9jxPLnU8s=\ngithub.com/libp2p/go-libp2p-kbucket v0.8.0/go.mod h1:JMlxqcEyKwO6ox716eyC0hmiduSWZZl6JY93mGaaqc4=\ngithub.com/libp2p/go-libp2p-record v0.3.1 h1:cly48Xi5GjNw5Wq+7gmjfBiG9HCzQVkiZOUZ8kUl+Fg=\ngithub.com/libp2p/go-libp2p-record v0.3.1/go.mod h1:T8itUkLcWQLCYMqtX7Th6r7SexyUJpIyPgks757td/E=\ngithub.com/libp2p/go-libp2p-routing-helpers v0.7.5 h1:HdwZj9NKovMx0vqq6YNPTh6aaNzey5zHD7HeLJtq6fI=\ngithub.com/libp2p/go-libp2p-routing-helpers v0.7.5/go.mod h1:3YaxrwP0OBPDD7my3D0KxfR89FlcX/IEbxDEDfAmj98=\ngithub.com/libp2p/go-libp2p-testing v0.12.0 h1:EPvBb4kKMWO29qP4mZGyhVzUyR25dvfUIK5WDu6iPUA=\ngithub.com/libp2p/go-libp2p-testing v0.12.0/go.mod h1:KcGDRXyN7sQCllucn1cOOS+Dmm7ujhfEyXQL5lvkcPg=\ngithub.com/libp2p/go-msgio v0.3.0 h1:mf3Z8B1xcFN314sWX+2vOTShIE0Mmn2TXn3YCUQGNj0=\ngithub.com/libp2p/go-msgio v0.3.0/go.mod h1:nyRM819GmVaF9LX3l03RMh10QdOroF++NBbxAb0mmDM=\ngithub.com/libp2p/go-netroute v0.4.0 h1:sZZx9hyANYUx9PZyqcgE/E1GUG3iEtTZHUEvdtXT7/Q=\ngithub.com/libp2p/go-netroute v0.4.0/go.mod h1:Nkd5ShYgSMS5MUKy/MU2T57xFoOKvvLR92Lic48LEyA=\ngithub.com/libp2p/go-reuseport v0.4.0 h1:nR5KU7hD0WxXCJbmw7r2rhRYruNRl2koHw8fQscQm2s=\ngithub.com/libp2p/go-reuseport v0.4.0/go.mod h1:ZtI03j/wO5hZVDFo2jKywN6bYKWLOy8Se6DrI2E1cLU=\ngithub.com/libp2p/go-yamux/v5 v5.0.1 h1:f0WoX/bEF2E8SbE4c/k1Mo+/9z0O4oC/hWEA+nfYRSg=\ngithub.com/libp2p/go-yamux/v5 v5.0.1/go.mod h1:en+3cdX51U0ZslwRdRLrvQsdayFt3TSUKvBGErzpWbU=\ngithub.com/macabu/inamedparam v0.1.3 h1:2tk/phHkMlEL/1GNe/Yf6kkR/hkcUdAEY3L0hjYV1Mk=\ngithub.com/macabu/inamedparam v0.1.3/go.mod h1:93FLICAIk/quk7eaPPQvbzihUdn/QkGDwIZEoLtpH6I=\ngithub.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=\ngithub.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=\ngithub.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=\ngithub.com/maratori/testableexamples v1.0.0 h1:dU5alXRrD8WKSjOUnmJZuzdxWOEQ57+7s93SLMxb2vI=\ngithub.com/maratori/testableexamples v1.0.0/go.mod h1:4rhjL1n20TUTT4vdh3RDqSizKLyXp7K2u6HgraZCGzE=\ngithub.com/maratori/testpackage v1.1.1 h1:S58XVV5AD7HADMmD0fNnziNHqKvSdDuEKdPD1rNTU04=\ngithub.com/maratori/testpackage v1.1.1/go.mod h1:s4gRK/ym6AMrqpOa/kEbQTV4Q4jb7WeLZzVhVVVOQMc=\ngithub.com/marcopolo/simnet v0.0.4 h1:50Kx4hS9kFGSRIbrt9xUS3NJX33EyPqHVmpXvaKLqrY=\ngithub.com/marcopolo/simnet v0.0.4/go.mod h1:tfQF1u2DmaB6WHODMtQaLtClEf3a296CKQLq5gAsIS0=\ngithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd h1:br0buuQ854V8u83wA0rVZ8ttrq5CpaPZdvrK0LP2lOk=\ngithub.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd/go.mod h1:QuCEs1Nt24+FYQEqAAncTDPJIuGs+LxK1MCiFL25pMU=\ngithub.com/matoous/godox v1.1.0 h1:W5mqwbyWrwZv6OQ5Z1a/DHGMOvXYCBP3+Ht7KMoJhq4=\ngithub.com/matoous/godox v1.1.0/go.mod h1:jgE/3fUXiTurkdHOLT5WEkThTSuE7yxHv5iWPa80afs=\ngithub.com/matryer/is v1.4.0 h1:sosSmIWwkYITGrxZ25ULNDeKiMNzFSr4V/eqBQP0PeE=\ngithub.com/matryer/is v1.4.0/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwMRU=\ngithub.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=\ngithub.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=\ngithub.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=\ngithub.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=\ngithub.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=\ngithub.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=\ngithub.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=\ngithub.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=\ngithub.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=\ngithub.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=\ngithub.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=\ngithub.com/mgechev/revive v1.7.0 h1:JyeQ4yO5K8aZhIKf5rec56u0376h8AlKNQEmjfkjKlY=\ngithub.com/mgechev/revive v1.7.0/go.mod h1:qZnwcNhoguE58dfi96IJeSTPeZQejNeoMQLUZGi4SW4=\ngithub.com/mholt/acmez/v3 v3.1.2 h1:auob8J/0FhmdClQicvJvuDavgd5ezwLBfKuYmynhYzc=\ngithub.com/mholt/acmez/v3 v3.1.2/go.mod h1:L1wOU06KKvq7tswuMDwKdcHeKpFFgkppZy/y0DFxagQ=\ngithub.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=\ngithub.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=\ngithub.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=\ngithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b h1:z78hV3sbSMAUoyUMM0I83AUIT6Hu17AWfgjzIbtrYFc=\ngithub.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b/go.mod h1:lxPUiZwKoFL8DUUmalo2yJJUCxbPKtm8OKfqr2/FTNU=\ngithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc h1:PTfri+PuQmWDqERdnNMiD9ZejrlswWrCpBEZgWOiTrc=\ngithub.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc/go.mod h1:cGKTAVKx4SxOuR/czcZ/E2RSJ3sfHs8FpHhQ5CWMf9s=\ngithub.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1/go.mod h1:pD8RvIylQ358TN4wwqatJ8rNavkEINozVn9DtGI3dfQ=\ngithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 h1:0lgqHvJWHLGW5TuObJrfyEi6+ASTKDBWikGvPqy9Yiw=\ngithub.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882/go.mod h1:qT0aEB35q79LLornSzeDH75LBf3aH1MV+jB5w9Wasec=\ngithub.com/minio/sha256-simd v0.1.1-0.20190913151208-6de447530771/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=\ngithub.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=\ngithub.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=\ngithub.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=\ngithub.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=\ngithub.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=\ngithub.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=\ngithub.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=\ngithub.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=\ngithub.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=\ngithub.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=\ngithub.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/moricho/tparallel v0.3.2 h1:odr8aZVFA3NZrNybggMkYO3rgPRcqjeQUlBBFVxKHTI=\ngithub.com/moricho/tparallel v0.3.2/go.mod h1:OQ+K3b4Ln3l2TZveGCywybl68glfLEwFGqvnjok8b+U=\ngithub.com/mr-tron/base58 v1.1.2/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=\ngithub.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=\ngithub.com/multiformats/go-base32 v0.1.0 h1:pVx9xoSPqEIQG8o+UbAe7DNi51oej1NtK+aGkbLYxPE=\ngithub.com/multiformats/go-base32 v0.1.0/go.mod h1:Kj3tFY6zNr+ABYMqeUNeGvkIC/UYgtWibDcT0rExnbI=\ngithub.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0=\ngithub.com/multiformats/go-base36 v0.2.0/go.mod h1:qvnKE++v+2MWCfePClUEjE78Z7P2a1UV0xHgWc0hkp4=\ngithub.com/multiformats/go-multiaddr v0.1.1/go.mod h1:aMKBKNEYmzmDmxfX88/vz+J5IU55txyt0p4aiWVohjo=\ngithub.com/multiformats/go-multiaddr v0.16.1 h1:fgJ0Pitow+wWXzN9do+1b8Pyjmo8m5WhGfzpL82MpCw=\ngithub.com/multiformats/go-multiaddr v0.16.1/go.mod h1:JSVUmXDjsVFiW7RjIFMP7+Ev+h1DTbiJgVeTV/tcmP0=\ngithub.com/multiformats/go-multiaddr-dns v0.5.0 h1:p/FTyHKX0nl59f+S+dEUe8HRK+i5Ow/QHMw8Nh3gPCo=\ngithub.com/multiformats/go-multiaddr-dns v0.5.0/go.mod h1:yJ349b8TPIAANUyuOzn1oz9o22tV9f+06L+cCeMxC14=\ngithub.com/multiformats/go-multiaddr-fmt v0.1.0 h1:WLEFClPycPkp4fnIzoFoV9FVd49/eQsuaL3/CWe167E=\ngithub.com/multiformats/go-multiaddr-fmt v0.1.0/go.mod h1:hGtDIW4PU4BqJ50gW2quDuPVjyWNZxToGUh/HwTZYJo=\ngithub.com/multiformats/go-multibase v0.2.0 h1:isdYCVLvksgWlMW9OZRYJEa9pZETFivncJHmHnnd87g=\ngithub.com/multiformats/go-multibase v0.2.0/go.mod h1:bFBZX4lKCA/2lyOFSAoKH5SS6oPyjtnzK/XTFDPkNuk=\ngithub.com/multiformats/go-multicodec v0.10.0 h1:UpP223cig/Cx8J76jWt91njpK3GTAO1w02sdcjZDSuc=\ngithub.com/multiformats/go-multicodec v0.10.0/go.mod h1:wg88pM+s2kZJEQfRCKBNU+g32F5aWBEjyFHXvZLTcLI=\ngithub.com/multiformats/go-multihash v0.0.8/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa63epEDmG8nTduyAew=\ngithub.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=\ngithub.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM=\ngithub.com/multiformats/go-multistream v0.6.1 h1:4aoX5v6T+yWmc2raBHsTvzmFhOI8WVOer28DeBBEYdQ=\ngithub.com/multiformats/go-multistream v0.6.1/go.mod h1:ksQf6kqHAb6zIsyw7Zm+gAuVo57Qbq84E27YlYqavqw=\ngithub.com/multiformats/go-varint v0.1.0 h1:i2wqFp4sdl3IcIxfAonHQV9qU5OsZ4Ts9IOoETFs5dI=\ngithub.com/multiformats/go-varint v0.1.0/go.mod h1:5KVAVXegtfmNQQm/lCY+ATvDzvJJhSkUlGQV9wgObdI=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=\ngithub.com/nakabonne/nestif v0.3.1 h1:wm28nZjhQY5HyYPx+weN3Q65k6ilSBxDb8v5S81B81U=\ngithub.com/nakabonne/nestif v0.3.1/go.mod h1:9EtoZochLn5iUprVDmDjqGKPofoUEBL8U4Ngq6aY7OE=\ngithub.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo=\ngithub.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM=\ngithub.com/nishanths/exhaustive v0.12.0 h1:vIY9sALmw6T/yxiASewa4TQcFsVYZQQRUQJhKRf3Swg=\ngithub.com/nishanths/exhaustive v0.12.0/go.mod h1:mEZ95wPIZW+x8kC4TgC+9YCUgiST7ecevsVDTgc2obs=\ngithub.com/nishanths/predeclared v0.2.2 h1:V2EPdZPliZymNAn79T8RkNApBjMmVKh5XRpLm/w98Vk=\ngithub.com/nishanths/predeclared v0.2.2/go.mod h1:RROzoN6TnGQupbC+lqggsOlcgysk3LMK/HI84Mp280c=\ngithub.com/nunnatsa/ginkgolinter v0.19.1 h1:mjwbOlDQxZi9Cal+KfbEJTCz327OLNfwNvoZ70NJ+c4=\ngithub.com/nunnatsa/ginkgolinter v0.19.1/go.mod h1:jkQ3naZDmxaZMXPWaS9rblH+i+GWXQCaS/JFIWcOH2s=\ngithub.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=\ngithub.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=\ngithub.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=\ngithub.com/onsi/ginkgo/v2 v2.23.4 h1:ktYTpKJAVZnDT4VjxSbiBenUjmlL/5QkBEocaWXiQus=\ngithub.com/onsi/ginkgo/v2 v2.23.4/go.mod h1:Bt66ApGPBFzHyR+JO10Zbt0Gsp4uWxu5mIOTusL46e8=\ngithub.com/onsi/gomega v1.36.3 h1:hID7cr8t3Wp26+cYnfcjR6HpJ00fdogN6dqZ1t6IylU=\ngithub.com/onsi/gomega v1.36.3/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=\ngithub.com/otiai10/copy v1.2.0/go.mod h1:rrF5dJ5F0t/EWSYODDu4j9/vEeYHMkc8jt0zJChqQWw=\ngithub.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=\ngithub.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=\ngithub.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=\ngithub.com/otiai10/curr v1.0.0/go.mod h1:LskTG5wDwr8Rs+nNQ+1LlxRjAtTZZjtJW4rMXl6j4vs=\ngithub.com/otiai10/mint v1.3.0/go.mod h1:F5AjcsTsWUqX+Na9fpHb52P8pcRX2CI6A3ctIT91xUo=\ngithub.com/otiai10/mint v1.3.1/go.mod h1:/yxELlJQ0ufhjUwhshSj+wFjZ78CnZ48/1wtmBH1OTc=\ngithub.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=\ngithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0=\ngithub.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y=\ngithub.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=\ngithub.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=\ngithub.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 h1:1/WtZae0yGtPq+TI6+Tv1WTxkukpXeMlviSxvL7SRgk=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9/go.mod h1:x3N5drFsm2uilKKuuYo6LdyD8vZAW55sH/9w+pbo1sw=\ngithub.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=\ngithub.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=\ngithub.com/pion/datachannel v1.5.10 h1:ly0Q26K1i6ZkGf42W7D4hQYR90pZwzFOjTq5AuCKk4o=\ngithub.com/pion/datachannel v1.5.10/go.mod h1:p/jJfC9arb29W7WrxyKbepTU20CFgyx5oLo8Rs4Py/M=\ngithub.com/pion/dtls/v3 v3.1.2 h1:gqEdOUXLtCGW+afsBLO0LtDD8GnuBBjEy6HRtyofZTc=\ngithub.com/pion/dtls/v3 v3.1.2/go.mod h1:Hw/igcX4pdY69z1Hgv5x7wJFrUkdgHwAn/Q/uo7YHRo=\ngithub.com/pion/ice/v4 v4.0.10 h1:P59w1iauC/wPk9PdY8Vjl4fOFL5B+USq1+xbDcN6gT4=\ngithub.com/pion/ice/v4 v4.0.10/go.mod h1:y3M18aPhIxLlcO/4dn9X8LzLLSma84cx6emMSu14FGw=\ngithub.com/pion/interceptor v0.1.40 h1:e0BjnPcGpr2CFQgKhrQisBU7V3GXK6wrfYrGYaU6Jq4=\ngithub.com/pion/interceptor v0.1.40/go.mod h1:Z6kqH7M/FYirg3frjGJ21VLSRJGBXB/KqaTIrdqnOic=\ngithub.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=\ngithub.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so=\ngithub.com/pion/mdns/v2 v2.0.7 h1:c9kM8ewCgjslaAmicYMFQIde2H9/lrZpjBkN8VwoVtM=\ngithub.com/pion/mdns/v2 v2.0.7/go.mod h1:vAdSYNAT0Jy3Ru0zl2YiW3Rm/fJCwIeM0nToenfOJKA=\ngithub.com/pion/randutil v0.1.0 h1:CFG1UdESneORglEsnimhUjf33Rwjubwj6xfiOXBa3mA=\ngithub.com/pion/randutil v0.1.0/go.mod h1:XcJrSMMbbMRhASFVOlj/5hQial/Y8oH/HVo7TBZq+j8=\ngithub.com/pion/rtcp v1.2.16 h1:fk1B1dNW4hsI78XUCljZJlC4kZOPk67mNRuQ0fcEkSo=\ngithub.com/pion/rtcp v1.2.16/go.mod h1:/as7VKfYbs5NIb4h6muQ35kQF/J0ZVNz2Z3xKoCBYOo=\ngithub.com/pion/rtp v1.8.19 h1:jhdO/3XhL/aKm/wARFVmvTfq0lC/CvN1xwYKmduly3c=\ngithub.com/pion/rtp v1.8.19/go.mod h1:bAu2UFKScgzyFqvUKmbvzSdPr+NGbZtv6UB2hesqXBk=\ngithub.com/pion/sctp v1.8.39 h1:PJma40vRHa3UTO3C4MyeJDQ+KIobVYRZQZ0Nt7SjQnE=\ngithub.com/pion/sctp v1.8.39/go.mod h1:cNiLdchXra8fHQwmIoqw0MbLLMs+f7uQ+dGMG2gWebE=\ngithub.com/pion/sdp/v3 v3.0.18 h1:l0bAXazKHpepazVdp+tPYnrsy9dfh7ZbT8DxesH5ZnI=\ngithub.com/pion/sdp/v3 v3.0.18/go.mod h1:ZREGo6A9ZygQ9XkqAj5xYCQtQpif0i6Pa81HOiAdqQ8=\ngithub.com/pion/srtp/v3 v3.0.6 h1:E2gyj1f5X10sB/qILUGIkL4C2CqK269Xq167PbGCc/4=\ngithub.com/pion/srtp/v3 v3.0.6/go.mod h1:BxvziG3v/armJHAaJ87euvkhHqWe9I7iiOy50K2QkhY=\ngithub.com/pion/stun/v3 v3.1.1 h1:CkQxveJ4xGQjulGSROXbXq94TAWu8gIX2dT+ePhUkqw=\ngithub.com/pion/stun/v3 v3.1.1/go.mod h1:qC1DfmcCTQjl9PBaMa5wSn3x9IPmKxSdcCsxBcDBndM=\ngithub.com/pion/transport/v3 v3.0.7 h1:iRbMH05BzSNwhILHoBoAPxoB9xQgOaJk+591KC9P1o0=\ngithub.com/pion/transport/v3 v3.0.7/go.mod h1:YleKiTZ4vqNxVwh77Z0zytYi7rXHl7j6uPLGhhz9rwo=\ngithub.com/pion/transport/v4 v4.0.1 h1:sdROELU6BZ63Ab7FrOLn13M6YdJLY20wldXW2Cu2k8o=\ngithub.com/pion/transport/v4 v4.0.1/go.mod h1:nEuEA4AD5lPdcIegQDpVLgNoDGreqM/YqmEx3ovP4jM=\ngithub.com/pion/turn/v4 v4.0.2 h1:ZqgQ3+MjP32ug30xAbD6Mn+/K4Sxi3SdNOTFf+7mpps=\ngithub.com/pion/turn/v4 v4.0.2/go.mod h1:pMMKP/ieNAG/fN5cZiN4SDuyKsXtNTr0ccN7IToA1zs=\ngithub.com/pion/webrtc/v4 v4.1.2 h1:mpuUo/EJ1zMNKGE79fAdYNFZBX790KE7kQQpLMjjR54=\ngithub.com/pion/webrtc/v4 v4.1.2/go.mod h1:xsCXiNAmMEjIdFxAYU0MbB3RwRieJsegSB2JZsGN+8U=\ngithub.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=\ngithub.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=\ngithub.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a h1:cgqrm0F3zwf9IPzca7xN4w+Zy6MC9ZkPvAC8QEWa/iQ=\ngithub.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a/go.mod h1:ocZfO/tLSHqfScRDNTJbAJR1by4D1lewauX9OwTaPuY=\ngithub.com/polyfloyd/go-errorlint v1.7.1 h1:RyLVXIbosq1gBdk/pChWA8zWYLsq9UEw7a1L5TVMCnA=\ngithub.com/polyfloyd/go-errorlint v1.7.1/go.mod h1:aXjNb1x2TNhoLsk26iv1yl7a+zTnXPhwEMtEXukiLR8=\ngithub.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=\ngithub.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g=\ngithub.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U=\ngithub.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=\ngithub.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=\ngithub.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=\ngithub.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=\ngithub.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=\ngithub.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=\ngithub.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=\ngithub.com/quasilyte/go-ruleguard v0.4.3-0.20240823090925-0fe6f58b47b1 h1:+Wl/0aFp0hpuHM3H//KMft64WQ1yX9LdJY64Qm/gFCo=\ngithub.com/quasilyte/go-ruleguard v0.4.3-0.20240823090925-0fe6f58b47b1/go.mod h1:GJLgqsLeo4qgavUoL8JeGFNS7qcisx3awV/w9eWTmNI=\ngithub.com/quasilyte/go-ruleguard/dsl v0.3.22 h1:wd8zkOhSNr+I+8Qeciml08ivDt1pSXe60+5DqOpCjPE=\ngithub.com/quasilyte/go-ruleguard/dsl v0.3.22/go.mod h1:KeCP03KrjuSO0H1kTuZQCWlQPulDV6YMIXmpQss17rU=\ngithub.com/quasilyte/gogrep v0.5.0 h1:eTKODPXbI8ffJMN+W2aE0+oL0z/nh8/5eNdiO34SOAo=\ngithub.com/quasilyte/gogrep v0.5.0/go.mod h1:Cm9lpz9NZjEoL1tgZ2OgeUKPIxL1meE7eo60Z6Sk+Ng=\ngithub.com/quasilyte/regex/syntax v0.0.0-20210819130434-b3f0c404a727 h1:TCg2WBOl980XxGFEZSS6KlBGIV0diGdySzxATTWoqaU=\ngithub.com/quasilyte/regex/syntax v0.0.0-20210819130434-b3f0c404a727/go.mod h1:rlzQ04UMyJXu/aOvhd8qT+hvDrFpiwqp8MRXDY9szc0=\ngithub.com/quasilyte/stdinfo v0.0.0-20220114132959-f7386bf02567 h1:M8mH9eK4OUR4lu7Gd+PU1fV2/qnDNfzT635KRSObncs=\ngithub.com/quasilyte/stdinfo v0.0.0-20220114132959-f7386bf02567/go.mod h1:DWNGW8A4Y+GyBgPuaQJuWiy0XYftx4Xm/y5Jqk9I6VQ=\ngithub.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=\ngithub.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=\ngithub.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=\ngithub.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=\ngithub.com/quic-go/webtransport-go v0.10.0 h1:LqXXPOXuETY5Xe8ITdGisBzTYmUOy5eSj+9n4hLTjHI=\ngithub.com/quic-go/webtransport-go v0.10.0/go.mod h1:LeGIXr5BQKE3UsynwVBeQrU1TPrbh73MGoC6jd+V7ow=\ngithub.com/raeperd/recvcheck v0.2.0 h1:GnU+NsbiCqdC2XX5+vMZzP+jAJC5fht7rcVTAhX74UI=\ngithub.com/raeperd/recvcheck v0.2.0/go.mod h1:n04eYkwIR0JbgD73wT8wL4JjPC3wm0nFtzBnWNocnYU=\ngithub.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=\ngithub.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=\ngithub.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=\ngithub.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=\ngithub.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=\ngithub.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=\ngithub.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=\ngithub.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=\ngithub.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=\ngithub.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=\ngithub.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=\ngithub.com/ryancurrah/gomodguard v1.3.5 h1:cShyguSwUEeC0jS7ylOiG/idnd1TpJ1LfHGpV3oJmPU=\ngithub.com/ryancurrah/gomodguard v1.3.5/go.mod h1:MXlEPQRxgfPQa62O8wzK3Ozbkv9Rkqr+wKjSxTdsNJE=\ngithub.com/ryanrolds/sqlclosecheck v0.5.1 h1:dibWW826u0P8jNLsLN+En7+RqWWTYrjCB9fJfSfdyCU=\ngithub.com/ryanrolds/sqlclosecheck v0.5.1/go.mod h1:2g3dUjoS6AL4huFdv6wn55WpLIDjY7ZgUR4J8HOO/XQ=\ngithub.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=\ngithub.com/sagikazarmark/locafero v0.6.0 h1:ON7AQg37yzcRPU69mt7gwhFEBwxI6P9T4Qu3N51bwOk=\ngithub.com/sagikazarmark/locafero v0.6.0/go.mod h1:77OmuIc6VTraTXKXIs/uvUxKGUXjE1GbemJYHqdNjX0=\ngithub.com/sagikazarmark/slog-shim v0.1.0 h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=\ngithub.com/sagikazarmark/slog-shim v0.1.0/go.mod h1:SrcSrq8aKtyuqEI1uvTDTK1arOWRIczQRv+GVI1AkeQ=\ngithub.com/sanposhiho/wastedassign/v2 v2.1.0 h1:crurBF7fJKIORrV85u9UUpePDYGWnwvv3+A96WvwXT0=\ngithub.com/sanposhiho/wastedassign/v2 v2.1.0/go.mod h1:+oSmSC+9bQ+VUAxA66nBb0Z7N8CK7mscKTDYC6aIek4=\ngithub.com/santhosh-tekuri/jsonschema/v6 v6.0.1 h1:PKK9DyHxif4LZo+uQSgXNqs0jj5+xZwwfKHgph2lxBw=\ngithub.com/santhosh-tekuri/jsonschema/v6 v6.0.1/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=\ngithub.com/sashamelentyev/interfacebloat v1.1.0 h1:xdRdJp0irL086OyW1H/RTZTr1h/tMEOsumirXcOJqAw=\ngithub.com/sashamelentyev/interfacebloat v1.1.0/go.mod h1:+Y9yU5YdTkrNvoX0xHc84dxiN1iBi9+G8zZIhPVoNjQ=\ngithub.com/sashamelentyev/usestdlibvars v1.28.0 h1:jZnudE2zKCtYlGzLVreNp5pmCdOxXUzwsMDBkR21cyQ=\ngithub.com/sashamelentyev/usestdlibvars v1.28.0/go.mod h1:9nl0jgOfHKWNFS43Ojw0i7aRoS4j6EBye3YBhmAIRF8=\ngithub.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=\ngithub.com/securego/gosec/v2 v2.22.2 h1:IXbuI7cJninj0nRpZSLCUlotsj8jGusohfONMrHoF6g=\ngithub.com/securego/gosec/v2 v2.22.2/go.mod h1:UEBGA+dSKb+VqM6TdehR7lnQtIIMorYJ4/9CW1KVQBE=\ngithub.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk=\ngithub.com/shurcooL/go v0.0.0-20200502201357-93f07166e636/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk=\ngithub.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041/go.mod h1:N5mDOmsrJOB+vfqUK+7DmDyjhSLIIBnXo9lvZJj3MWQ=\ngithub.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg=\ngithub.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=\ngithub.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546/go.mod h1:TrYk7fJVaAttu97ZZKrO9UbRa8izdowaMIZcxYMbVaw=\ngithub.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=\ngithub.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=\ngithub.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=\ngithub.com/sivchari/containedctx v1.0.3 h1:x+etemjbsh2fB5ewm5FeLNi5bUjK0V8n0RB+Wwfd0XE=\ngithub.com/sivchari/containedctx v1.0.3/go.mod h1:c1RDvCbnJLtH4lLcYD/GqwiBSSf4F5Qk0xld2rBqzJ4=\ngithub.com/sivchari/tenv v1.12.1 h1:+E0QzjktdnExv/wwsnnyk4oqZBUfuh89YMQT1cyuvSY=\ngithub.com/sivchari/tenv v1.12.1/go.mod h1:1LjSOUCc25snIr5n3DtGGrENhX3LuWefcplwVGC24mw=\ngithub.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY=\ngithub.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=\ngithub.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=\ngithub.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=\ngithub.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=\ngithub.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=\ngithub.com/sonatard/noctx v0.1.0 h1:JjqOc2WN16ISWAjAk8M5ej0RfExEXtkEyExl2hLW+OM=\ngithub.com/sonatard/noctx v0.1.0/go.mod h1:0RvBxqY8D4j9cTTTWE8ylt2vqj2EPI8fHmrxHdsaZ2c=\ngithub.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=\ngithub.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=\ngithub.com/sourcegraph/go-diff v0.7.0 h1:9uLlrd5T46OXs5qpp8L/MTltk0zikUGi0sNNyCpA8G0=\ngithub.com/sourcegraph/go-diff v0.7.0/go.mod h1:iBszgVvyxdc8SFZ7gm69go2KDdt3ag071iBaWPF6cjs=\ngithub.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=\ngithub.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=\ngithub.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=\ngithub.com/spf13/afero v1.12.0 h1:UcOPyRBYczmFn6yvphxkn9ZEOY65cpwGKb5mL36mrqs=\ngithub.com/spf13/afero v1.12.0/go.mod h1:ZTlWwG4/ahT8W7T0WQ5uYmjI9duaLQGy3Q2OAl4sk/4=\ngithub.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cast v1.6.0 h1:GEiTHELF+vaR5dhz3VqZfFSzZjYbgeKDpBxQVS4GYJ0=\ngithub.com/spf13/cast v1.6.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=\ngithub.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk=\ngithub.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=\ngithub.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=\ngithub.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=\ngithub.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=\ngithub.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=\ngithub.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=\ngithub.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns=\ngithub.com/spf13/viper v1.19.0 h1:RWq5SEjt8o25SROyN3z2OrDB9l7RPd3lwTWU8EcEdcI=\ngithub.com/spf13/viper v1.19.0/go.mod h1:GQUN9bilAbhU/jgc1bKs99f/suXKeUMct8Adx5+Ntkg=\ngithub.com/ssgreg/nlreturn/v2 v2.2.1 h1:X4XDI7jstt3ySqGU86YGAURbxw3oTDPK9sPEi6YEwQ0=\ngithub.com/ssgreg/nlreturn/v2 v2.2.1/go.mod h1:E/iiPB78hV7Szg2YfRgyIrk1AD6JVMTRkkxBiELzh2I=\ngithub.com/stbenjam/no-sprintf-host-port v0.2.0 h1:i8pxvGrt1+4G0czLr/WnmyH7zbZ8Bg8etvARQ1rpyl4=\ngithub.com/stbenjam/no-sprintf-host-port v0.2.0/go.mod h1:eL0bQ9PasS0hsyTyfTjjG+E80QIyPnBVQbYZyv20Jfk=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=\ngithub.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=\ngithub.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=\ngithub.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=\ngithub.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=\ngithub.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngithub.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=\ngithub.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=\ngithub.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=\ngithub.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=\ngithub.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=\ngithub.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=\ngithub.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=\ngithub.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=\ngithub.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=\ngithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d h1:vfofYNRScrDdvS342BElfbETmL1Aiz3i2t0zfRj16Hs=\ngithub.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d/go.mod h1:RRCYJbIwD5jmqPI9XoAFR0OcDxqUctll6zUj/+B4S48=\ngithub.com/tdakkota/asciicheck v0.4.1 h1:bm0tbcmi0jezRA2b5kg4ozmMuGAFotKI3RZfrhfovg8=\ngithub.com/tdakkota/asciicheck v0.4.1/go.mod h1:0k7M3rCfRXb0Z6bwgvkEIMleKH3kXNz9UqJ9Xuqopr8=\ngithub.com/tenntenn/modver v1.0.1 h1:2klLppGhDgzJrScMpkj9Ujy3rXPUspSjAcev9tSEBgA=\ngithub.com/tenntenn/modver v1.0.1/go.mod h1:bePIyQPb7UeioSRkw3Q0XeMhYZSMx9B8ePqg6SAMGH0=\ngithub.com/tenntenn/text/transform v0.0.0-20200319021203-7eef512accb3 h1:f+jULpRQGxTSkNYKJ51yaw6ChIqO+Je8UqsTKN/cDag=\ngithub.com/tenntenn/text/transform v0.0.0-20200319021203-7eef512accb3/go.mod h1:ON8b8w4BN/kE1EOhwT0o+d62W65a6aPw1nouo9LMgyY=\ngithub.com/tetafro/godot v1.5.0 h1:aNwfVI4I3+gdxjMgYPus9eHmoBeJIbnajOyqZYStzuw=\ngithub.com/tetafro/godot v1.5.0/go.mod h1:2oVxTBSftRTh4+MVfUaUXR6bn2GDXCaMcOG4Dk3rfio=\ngithub.com/timakin/bodyclose v0.0.0-20241017074812-ed6a65f985e3 h1:y4mJRFlM6fUyPhoXuFg/Yu02fg/nIPFMOY8tOqppoFg=\ngithub.com/timakin/bodyclose v0.0.0-20241017074812-ed6a65f985e3/go.mod h1:mkjARE7Yr8qU23YcGMSALbIxTQ9r9QBVahQOBRfU460=\ngithub.com/timonwong/loggercheck v0.10.1 h1:uVZYClxQFpw55eh+PIoqM7uAOHMrhVcDoWDery9R8Lg=\ngithub.com/timonwong/loggercheck v0.10.1/go.mod h1:HEAWU8djynujaAVX7QI65Myb8qgfcZ1uKbdpg3ZzKl8=\ngithub.com/tomarrell/wrapcheck/v2 v2.10.0 h1:SzRCryzy4IrAH7bVGG4cK40tNUhmVmMDuJujy4XwYDg=\ngithub.com/tomarrell/wrapcheck/v2 v2.10.0/go.mod h1:g9vNIyhb5/9TQgumxQyOEqDHsmGYcGsVMOx/xGkqdMo=\ngithub.com/tommy-muehle/go-mnd/v2 v2.5.1 h1:NowYhSdyE/1zwK9QCLeRb6USWdoif80Ie+v+yU8u1Zw=\ngithub.com/tommy-muehle/go-mnd/v2 v2.5.1/go.mod h1:WsUAkMJMYww6l/ufffCD3m+P7LEvr8TnZn9lwVDlgzw=\ngithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb h1:Ywfo8sUltxogBpFuMOFRrrSifO788kAFxmvVw31PtQQ=\ngithub.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb/go.mod h1:ikPs9bRWicNw3S7XpJ8sK/smGwU9WcSVU3dy9qahYBM=\ngithub.com/ultraware/funlen v0.2.0 h1:gCHmCn+d2/1SemTdYMiKLAHFYxTYz7z9VIDRaTGyLkI=\ngithub.com/ultraware/funlen v0.2.0/go.mod h1:ZE0q4TsJ8T1SQcjmkhN/w+MceuatI6pBFSxxyteHIJA=\ngithub.com/ultraware/whitespace v0.2.0 h1:TYowo2m9Nfj1baEQBjuHzvMRbp19i+RCcRYrSWoFa+g=\ngithub.com/ultraware/whitespace v0.2.0/go.mod h1:XcP1RLD81eV4BW8UhQlpaR+SDc2givTvyI8a586WjW8=\ngithub.com/urfave/cli v1.22.10/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=\ngithub.com/urfave/cli v1.22.16 h1:MH0k6uJxdwdeWQTwhSO42Pwr4YLrNLwBtg1MRgTqPdQ=\ngithub.com/urfave/cli v1.22.16/go.mod h1:EeJR6BKodywf4zciqrdw6hpCPk68JO9z5LazXZMn5Po=\ngithub.com/uudashr/gocognit v1.2.0 h1:3BU9aMr1xbhPlvJLSydKwdLN3tEUUrzPSSM8S4hDYRA=\ngithub.com/uudashr/gocognit v1.2.0/go.mod h1:k/DdKPI6XBZO1q7HgoV2juESI2/Ofj9AcHPZhBBdrTU=\ngithub.com/uudashr/iface v1.3.1 h1:bA51vmVx1UIhiIsQFSNq6GZ6VPTk3WNMZgRiCe9R29U=\ngithub.com/uudashr/iface v1.3.1/go.mod h1:4QvspiRd3JLPAEXBQ9AiZpLbJlrWWgRChOKDJEuQTdg=\ngithub.com/warpfork/go-testmark v0.12.1 h1:rMgCpJfwy1sJ50x0M0NgyphxYYPMOODIJHhsXyEHU0s=\ngithub.com/warpfork/go-testmark v0.12.1/go.mod h1:kHwy7wfvGSPh1rQJYKayD4AbtNaeyZdcGi9tNJTaa5Y=\ngithub.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0 h1:GDDkbFiaK8jsSDJfjId/PEGEShv6ugrt4kYsC5UIDaQ=\ngithub.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw=\ngithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc h1:BCPnHtcboadS0DvysUuJXZ4lWVv5Bh5i7+tbIyi+ck4=\ngithub.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc/go.mod h1:r45hJU7yEoA81k6MWNhpMj/kms0n14dkzkxYHoB96UM=\ngithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11 h1:5HZfQkwe0mIfyDmc1Em5GqlNRzcdtlv4HTNmdpt7XH0=\ngithub.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11/go.mod h1:Wlo/SzPmxVp6vXpGt/zaXhHH0fn4IxgqZc82aKg6bpQ=\ngithub.com/whyrusleeping/cbor-gen v0.3.1 h1:82ioxmhEYut7LBVGhGq8xoRkXPLElVuh5mV67AFfdv0=\ngithub.com/whyrusleeping/cbor-gen v0.3.1/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so=\ngithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f h1:jQa4QT2UP9WYv2nzyawpKMOCl+Z/jW7djv2/J50lj9E=\ngithub.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f/go.mod h1:p9UJB6dDgdPgMJZs7UjUOdulKyRr9fqkS+6JKAInPy8=\ngithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 h1:EKhdznlJHPMoKr0XTrX+IlJs1LH3lyx2nfr1dOlZ79k=\ngithub.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1/go.mod h1:8UvriyWtv5Q5EOgjHaSseUEdkQfvwFv1I/In/O2M9gc=\ngithub.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU=\ngithub.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA=\ngithub.com/xen0n/gosmopolitan v1.2.2 h1:/p2KTnMzwRexIW8GlKawsTWOxn7UHA+jCMF/V8HHtvU=\ngithub.com/xen0n/gosmopolitan v1.2.2/go.mod h1:7XX7Mj61uLYrj0qmeN0zi7XDon9JRAEhYQqAPLVNTeg=\ngithub.com/yagipy/maintidx v1.0.0 h1:h5NvIsCz+nRDapQ0exNv4aJ0yXSI0420omVANTv3GJM=\ngithub.com/yagipy/maintidx v1.0.0/go.mod h1:0qNf/I/CCZXSMhsRsrEPDZ+DkekpKLXAJfsTACwgXLk=\ngithub.com/yeya24/promlinter v0.3.0 h1:JVDbMp08lVCP7Y6NP3qHroGAO6z2yGKQtS5JsjqtoFs=\ngithub.com/yeya24/promlinter v0.3.0/go.mod h1:cDfJQQYv9uYciW60QT0eeHlFodotkYZlL+YcPQN+mW4=\ngithub.com/ykadowak/zerologlint v0.1.5 h1:Gy/fMz1dFQN9JZTPjv1hxEk+sRWm05row04Yoolgdiw=\ngithub.com/ykadowak/zerologlint v0.1.5/go.mod h1:KaUskqF3e/v59oPmdq1U1DnKcuHokl2/K1U4pmIELKg=\ngithub.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=\ngithub.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=\ngithub.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=\ngithub.com/zeebo/assert v1.1.0 h1:hU1L1vLTHsnO8x8c9KAR5GmM5QscxHg5RNU5z5qbUWY=\ngithub.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=\ngithub.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI=\ngithub.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE=\ngithub.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo=\ngithub.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4=\ngitlab.com/bosi/decorder v0.4.2 h1:qbQaV3zgwnBZ4zPMhGLW4KZe7A7NwxEhJx39R3shffo=\ngitlab.com/bosi/decorder v0.4.2/go.mod h1:muuhHoaJkA9QLcYHq4Mj8FJUwDZ+EirSHRiaTcTf6T8=\ngo-simpler.org/assert v0.9.0 h1:PfpmcSvL7yAnWyChSjOz6Sp6m9j5lyK8Ok9pEL31YkQ=\ngo-simpler.org/assert v0.9.0/go.mod h1:74Eqh5eI6vCK6Y5l3PI8ZYFXG4Sa+tkr70OIPJAUr28=\ngo-simpler.org/musttag v0.13.0 h1:Q/YAW0AHvaoaIbsPj3bvEI5/QFP7w696IMUpnKXQfCE=\ngo-simpler.org/musttag v0.13.0/go.mod h1:FTzIGeK6OkKlUDVpj0iQUXZLUO1Js9+mvykDQy9C5yM=\ngo-simpler.org/sloglint v0.9.0 h1:/40NQtjRx9txvsB/RN022KsUJU+zaaSb/9q9BSefSrE=\ngo-simpler.org/sloglint v0.9.0/go.mod h1:G/OrAF6uxj48sHahCzrbarVMptL2kjWTaUeC8+fOGww=\ngo.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=\ngo.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=\ngo.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ=\ngo.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=\ngo.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=\ngo.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=\ngo.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=\ngo.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=\ngo.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 h1:7iP2uCb7sGddAr30RRS6xjKy7AZ2JtTOPA3oolgVSw8=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0/go.mod h1:c7hN3ddxs/z6q9xwvfLPk+UHlWRQyaeR1LdgfL/66l0=\ngo.opentelemetry.io/otel v1.42.0 h1:lSQGzTgVR3+sgJDAU/7/ZMjN9Z+vUip7leaqBKy4sho=\ngo.opentelemetry.io/otel v1.42.0/go.mod h1:lJNsdRMxCUIWuMlVJWzecSMuNjE7dOYyWlqOXWkdqCc=\ngo.opentelemetry.io/otel/metric v1.42.0 h1:2jXG+3oZLNXEPfNmnpxKDeZsFI5o4J+nz6xUlaFdF/4=\ngo.opentelemetry.io/otel/metric v1.42.0/go.mod h1:RlUN/7vTU7Ao/diDkEpQpnz3/92J9ko05BIwxYa2SSI=\ngo.opentelemetry.io/otel/sdk v1.40.0 h1:KHW/jUzgo6wsPh9At46+h4upjtccTmuZCFAc9OJ71f8=\ngo.opentelemetry.io/otel/sdk v1.40.0/go.mod h1:Ph7EFdYvxq72Y8Li9q8KebuYUr2KoeyHx0DRMKrYBUE=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0 h1:mtmdVqgQkeRxHgRv4qhyJduP3fYJRMX4AtAlbuWdCYw=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0/go.mod h1:4Z2bGMf0KSK3uRjlczMOeMhKU2rhUqdWNoKcYrtcBPg=\ngo.opentelemetry.io/otel/trace v1.42.0 h1:OUCgIPt+mzOnaUTpOQcBiM/PLQ/Op7oq6g4LenLmOYY=\ngo.opentelemetry.io/otel/trace v1.42.0/go.mod h1:f3K9S+IFqnumBkKhRJMeaZeNk9epyhnCmQh/EysQCdc=\ngo.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=\ngo.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=\ngo.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=\ngo.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4=\ngo.uber.org/dig v1.19.0/go.mod h1:Us0rSJiThwCv2GteUN0Q7OKvU7n5J4dxZ9JKUXozFdE=\ngo.uber.org/fx v1.24.0 h1:wE8mruvpg2kiiL1Vqd0CC+tr0/24XIB10Iwp2lLWzkg=\ngo.uber.org/fx v1.24.0/go.mod h1:AmDeGyS+ZARGKM4tlH4FY2Jr63VjbEDJHtqXTGP5hbo=\ngo.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=\ngo.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=\ngo.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=\ngo.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=\ngo.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=\ngo.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=\ngo.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=\ngo.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=\ngo.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=\ngo.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=\ngo.uber.org/zap/exp v0.3.0 h1:6JYzdifzYkGmTdRR59oYH+Ng7k49H9qVpWwNSsGJj3U=\ngo.uber.org/zap/exp v0.3.0/go.mod h1:5I384qq7XGxYyByIhHm6jg5CHkGY0nsTfbDLgDDlgJQ=\ngo.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=\ngo.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=\ngolang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20200602180216-279210d13fed/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=\ngolang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=\ngolang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=\ngolang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=\ngolang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=\ngolang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=\ngolang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=\ngolang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=\ngolang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=\ngolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90 h1:jiDhWWeC7jfWqR9c/uplMOqJ0sbNlNWv0UkzE0vX1MA=\ngolang.org/x/exp v0.0.0-20260312153236-7ab1446f8b90/go.mod h1:xE1HEv6b+1SCZ5/uscMRjUBKtIxworgEcEi+/n9NQDQ=\ngolang.org/x/exp/typeparams v0.0.0-20220428152302-39d4317da171/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=\ngolang.org/x/exp/typeparams v0.0.0-20230203172020-98cc5a0785f9/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=\ngolang.org/x/exp/typeparams v0.0.0-20250210185358-939b2ce775ac h1:TSSpLIG4v+p0rPv1pNOQtl1I8knsO4S9trOxNMOLVP4=\ngolang.org/x/exp/typeparams v0.0.0-20250210185358-939b2ce775ac/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=\ngolang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=\ngolang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=\ngolang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=\ngolang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=\ngolang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=\ngolang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=\ngolang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=\ngolang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=\ngolang.org/x/mod v0.7.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=\ngolang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=\ngolang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=\ngolang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=\ngolang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=\ngolang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=\ngolang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=\ngolang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=\ngolang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=\ngolang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=\ngolang.org/x/net v0.16.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=\ngolang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=\ngolang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=\ngolang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=\ngolang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=\ngolang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=\ngolang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20211105183446-c75c47738b0c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=\ngolang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=\ngolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c h1:6a8FdnNk6bTXBjR4AGKFgUKuo+7GnR3FX5L7CbveeZc=\ngolang.org/x/telemetry v0.0.0-20260311193753-579e4da9a98c/go.mod h1:TpUTTEp9frx7rTdLpC9gFG9kdI7zVLFTFFlqaH2Cncw=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=\ngolang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=\ngolang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=\ngolang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=\ngolang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=\ngolang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=\ngolang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU=\ngolang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A=\ngolang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=\ngolang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=\ngolang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\ngolang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\ngolang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=\ngolang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=\ngolang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=\ngolang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=\ngolang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=\ngolang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=\ngolang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200324003944-a576cf524670/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=\ngolang.org/x/tools v0.0.0-20200329025819-fd4102a86c65/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=\ngolang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=\ngolang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200724022722-7017fd6b1305/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200820010801-b793a1359eac/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=\ngolang.org/x/tools v0.0.0-20201023174141-c8cfbd0f21e6/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=\ngolang.org/x/tools v0.1.1-0.20210205202024-ef80cdb6ec6d/go.mod h1:9bzcO0MWcOuT0tm1iBGzDVPshzfwoVvREIui8C+MHqU=\ngolang.org/x/tools v0.1.1-0.20210302220138-2ac05c832e1a/go.mod h1:9bzcO0MWcOuT0tm1iBGzDVPshzfwoVvREIui8C+MHqU=\ngolang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E=\ngolang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=\ngolang.org/x/tools v0.3.0/go.mod h1:/rWhSS2+zyEVwoJf8YAX6L2f0ntZ7Kn/mGgAWcipA5k=\ngolang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=\ngolang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s=\ngolang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=\ngolang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg=\ngolang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=\ngolang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=\ngolang.org/x/tools/go/expect v0.1.1-deprecated h1:jpBZDwmgPhXsKZC6WhL20P4b/wmnpsEAGHaNy0n/rJM=\ngolang.org/x/tools/go/expect v0.1.1-deprecated/go.mod h1:eihoPOH+FgIqa3FpoTwguz/bVUSGBlGQU67vpBeOrBY=\ngolang.org/x/tools/go/packages/packagestest v0.1.1-deprecated h1:1h2MnaIAIXISqTFKdENegdpAgUXz6NrPEsbIeWaBRvM=\ngolang.org/x/tools/go/packages/packagestest v0.1.1-deprecated/go.mod h1:RVAQXBGNv1ib0J382/DPCRS/BPnsGebyM1Gj5VSDpG8=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da h1:noIWHXmPHxILtqtCOPIhSt0ABwskkZKjD3bXGnZGpNY=\ngolang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=\ngonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=\ngonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=\ngoogle.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=\ngoogle.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=\ngoogle.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=\ngoogle.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=\ngoogle.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=\ngoogle.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=\ngoogle.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=\ngoogle.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=\ngoogle.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=\ngoogle.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=\ngoogle.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=\ngoogle.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=\ngoogle.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=\ngoogle.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\ngoogle.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=\ngoogle.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=\ngoogle.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=\ngoogle.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=\ngoogle.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=\ngoogle.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=\ngoogle.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\ngoogle.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=\ngoogle.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=\ngoogle.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\ngoogle.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=\ngoogle.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\ngoogle.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=\ngoogle.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=\ngopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=\ngopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=\ngopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\ngopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\ngopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\ngopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngotest.tools/gotestsum v1.13.0 h1:+Lh454O9mu9AMG1APV4o0y7oDYKyik/3kBOiCqiEpRo=\ngotest.tools/gotestsum v1.13.0/go.mod h1:7f0NS5hFb0dWr4NtcsAsF0y1kzjEFfAil0HiBQJE03Q=\ngotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=\ngotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=\nhonnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nhonnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nhonnef.co/go/tools v0.6.1 h1:R094WgE8K4JirYjBaOpz/AvTyUu/3wbmAoskKN/pxTI=\nhonnef.co/go/tools v0.6.1/go.mod h1:3puzxxljPCe8RGJX7BIy1plGbxEOZni5mR2aXe3/uk4=\nlukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg=\nlukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo=\nmvdan.cc/gofumpt v0.7.0 h1:bg91ttqXmi9y2xawvkuMXyvAA/1ZGJqYAEGjXuP0JXU=\nmvdan.cc/gofumpt v0.7.0/go.mod h1:txVFJy/Sc/mvaycET54pV8SW8gWxTlUuGHVEcncmNUo=\nmvdan.cc/unparam v0.0.0-20240528143540-8a5130ca722f h1:lMpcwN6GxNbWtbpI1+xzFLSW8XzX0u72NttUGVFjO3U=\nmvdan.cc/unparam v0.0.0-20240528143540-8a5130ca722f/go.mod h1:RSLa7mKKCNeTTMHBw5Hsy2rfJmd6O2ivt9Dw9ZqCQpQ=\nrsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=\nrsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=\nrsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=\n" }, { "path": "test/dependencies/iptb/iptb.go", "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\n\tcli \"github.com/ipfs/iptb/cli\"\n\ttestbed \"github.com/ipfs/iptb/testbed\"\n\n\tplugin \"github.com/ipfs/iptb-plugins/local\"\n)\n\nfunc init() {\n\t_, err := testbed.RegisterPlugin(testbed.IptbPlugin{\n\t\tFrom: \"\",\n\t\tNewNode: plugin.NewNode,\n\t\tGetAttrList: plugin.GetAttrList,\n\t\tGetAttrDesc: plugin.GetAttrDesc,\n\t\tPluginName: plugin.PluginName,\n\t\tBuiltIn: true,\n\t}, false)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\nfunc main() {\n\tcli := cli.NewCli()\n\tif err := cli.Run(os.Args); err != nil {\n\t\tfmt.Fprintf(cli.ErrWriter, \"%s\\n\", err)\n\t\tos.Exit(1)\n\t}\n}\n" }, { "path": "test/dependencies/ma-pipe-unidir/LICENSE", "content": "The MIT License (MIT)\n\nCopyright (c) 2017 Łukasz Magiera \n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n\n" }, { "path": "test/dependencies/ma-pipe-unidir/main.go", "content": "package main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"strconv\"\n\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nconst USAGE = \"ma-pipe-unidir [-l|--listen] [--pidFile=path] [-h|--help] \\n\"\n\ntype Opts struct {\n\tListen bool\n\tPidFile string\n}\n\nfunc app() int {\n\topts := Opts{}\n\tflag.BoolVar(&opts.Listen, \"l\", false, \"\")\n\tflag.BoolVar(&opts.Listen, \"listen\", false, \"\")\n\tflag.StringVar(&opts.PidFile, \"pidFile\", \"\", \"\")\n\tflag.Usage = func() {\n\t\tfmt.Print(USAGE)\n\t}\n\tflag.Parse()\n\targs := flag.Args()\n\n\tif len(args) < 2 { // \n\t\tfmt.Print(USAGE)\n\t\treturn 1\n\t}\n\n\tmode := args[0]\n\taddr := args[1]\n\n\tif mode != \"send\" && mode != \"recv\" {\n\t\tfmt.Print(USAGE)\n\t\treturn 1\n\t}\n\n\tmaddr, err := ma.NewMultiaddr(addr)\n\tif err != nil {\n\t\treturn 1\n\t}\n\n\tvar conn manet.Conn\n\n\tif opts.Listen {\n\t\tlistener, err := manet.Listen(maddr)\n\t\tif err != nil {\n\t\t\treturn 1\n\t\t}\n\n\t\tif len(opts.PidFile) > 0 {\n\t\t\tdata := []byte(strconv.Itoa(os.Getpid()))\n\t\t\terr := os.WriteFile(opts.PidFile, data, 0o644)\n\t\t\tif err != nil {\n\t\t\t\treturn 1\n\t\t\t}\n\n\t\t\tdefer os.Remove(opts.PidFile)\n\t\t}\n\n\t\tconn, err = listener.Accept()\n\t\tif err != nil {\n\t\t\treturn 1\n\t\t}\n\t} else {\n\t\tvar err error\n\t\tconn, err = manet.Dial(maddr)\n\t\tif err != nil {\n\t\t\treturn 1\n\t\t}\n\n\t\tif len(opts.PidFile) > 0 {\n\t\t\tdata := []byte(strconv.Itoa(os.Getpid()))\n\t\t\terr := os.WriteFile(opts.PidFile, data, 0o644)\n\t\t\tif err != nil {\n\t\t\t\treturn 1\n\t\t\t}\n\n\t\t\tdefer os.Remove(opts.PidFile)\n\t\t}\n\n\t}\n\n\tdefer conn.Close()\n\tswitch mode {\n\tcase \"recv\":\n\t\t_, err = io.Copy(os.Stdout, conn)\n\tcase \"send\":\n\t\t_, err = io.Copy(conn, os.Stdin)\n\tdefault:\n\t\treturn 1\n\t}\n\tif err != nil {\n\t\treturn 1\n\t}\n\treturn 0\n}\n\nfunc main() {\n\tos.Exit(app())\n}\n" }, { "path": "test/dependencies/pollEndpoint/main.go", "content": "// pollEndpoint is a helper utility that waits for a http endpoint to be reachable and return with http.StatusOK\npackage main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"os\"\n\t\"time\"\n\n\tlogging \"github.com/ipfs/go-log/v2\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\tmanet \"github.com/multiformats/go-multiaddr/net\"\n)\n\nvar (\n\thost = flag.String(\"host\", \"/ip4/127.0.0.1/tcp/5001\", \"the multiaddr host to dial on\")\n\ttries = flag.Int(\"tries\", 10, \"how many tries to make before failing\")\n\ttimeout = flag.Duration(\"tout\", time.Second, \"how long to wait between attempts\")\n\thttpURL = flag.String(\"http-url\", \"\", \"HTTP URL to fetch\")\n\thttpOut = flag.Bool(\"http-out\", false, \"Print the HTTP response body to stdout\")\n\tverbose = flag.Bool(\"v\", false, \"verbose logging\")\n)\n\nvar log = logging.Logger(\"pollEndpoint\")\n\nfunc main() {\n\tflag.Parse()\n\n\t// extract address from host flag\n\taddr, err := ma.NewMultiaddr(*host)\n\tif err != nil {\n\t\tlog.Fatal(\"NewMultiaddr() failed: \", err)\n\t}\n\n\tif *verbose { // lower log level\n\t\tlogging.SetDebugLogging()\n\t}\n\n\t// show what we got\n\tstart := time.Now()\n\tlog.Debugf(\"starting at %s, tries: %d, timeout: %s, addr: %s\", start, *tries, *timeout, addr)\n\n\tconnTries := *tries\n\tfor connTries > 0 {\n\t\tc, err := manet.Dial(addr)\n\t\tif err == nil {\n\t\t\tlog.Debugf(\"ok - endpoint reachable with %d tries remaining, took %s\", *tries, time.Since(start))\n\t\t\tc.Close()\n\t\t\tbreak\n\t\t}\n\t\tlog.Debug(\"connect failed: \", err)\n\t\ttime.Sleep(*timeout)\n\t\tconnTries--\n\t}\n\n\tif err != nil {\n\t\tgoto Fail\n\t}\n\n\tif *httpURL != \"\" {\n\t\tdialer := &connDialer{addr: addr}\n\t\thttpClient := http.Client{Transport: &http.Transport{\n\t\t\tDialContext: dialer.DialContext,\n\t\t}}\n\t\treqTries := *tries\n\t\tfor reqTries > 0 {\n\t\t\ttry := (*tries - reqTries) + 1\n\t\t\tlog.Debugf(\"trying HTTP req %d: '%s'\", try, *httpURL)\n\t\t\tif tryHTTPGet(&httpClient, *httpURL) {\n\t\t\t\tlog.Debugf(\"HTTP req %d to '%s' succeeded\", try, *httpURL)\n\t\t\t\tgoto Success\n\t\t\t}\n\t\t\tlog.Debugf(\"HTTP req %d to '%s' failed\", try, *httpURL)\n\t\t\ttime.Sleep(*timeout)\n\t\t\treqTries--\n\t\t}\n\t\tgoto Fail\n\t}\n\nSuccess:\n\tos.Exit(0)\n\nFail:\n\tlog.Error(\"failed\")\n\tos.Exit(1)\n}\n\nfunc tryHTTPGet(client *http.Client, url string) bool {\n\tresp, err := client.Get(*httpURL)\n\tif resp != nil && resp.Body != nil {\n\t\tdefer resp.Body.Close()\n\t}\n\tif err != nil {\n\t\treturn false\n\t}\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn false\n\t}\n\tif *httpOut {\n\t\t_, err := io.Copy(os.Stdout, resp.Body)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n\n\treturn true\n}\n\ntype connDialer struct {\n\taddr ma.Multiaddr\n}\n\nfunc (d connDialer) DialContext(ctx context.Context, network, addr string) (net.Conn, error) {\n\treturn (&manet.Dialer{}).DialContext(ctx, d.addr)\n}\n" }, { "path": "test/integration/GNUmakefile", "content": "# This Makefile provides a way to really simple way to run benchmarks in a\n# docker environment.\n\nIPFS_ROOT = ../..\nCONTAINER = go-ipfs-bench\nPACKAGE = integrationtest\nPACKAGE_DIR = test/integration\nBUILD_DIR = ./build/bench\nCONTAINER_WORKING_DIR = /go\nCPU_PROF_NAME = cpu.out\nEXTRA_TEST_ARGS =\n\nall: collect\n\ncollect: clean build_image run_profiler cp_pprof_from_container\n\ncp_pprof_from_container:\n\tdocker cp $(CONTAINER):$(CONTAINER_WORKING_DIR)/$(CPU_PROF_NAME) $(BUILD_DIR)\n\tdocker cp $(CONTAINER):$(CONTAINER_WORKING_DIR)/$(PACKAGE).test $(BUILD_DIR)\n\nbuild_image:\n\tcd $(IPFS_ROOT) && docker build -t $(IMAGE) .\n\nrun_profiler:\n\tdocker run --name $(CONTAINER) -it --entrypoint go $(IMAGE) test ./src/github.com/ipfs/go-ipfs/$(PACKAGE_DIR) --cpuprofile=$(CPU_PROF_NAME) $(EXTRA_TEST_ARGS)\n\n\nclean:\n\tdocker rm $(CONTAINER) || true\n\trm -rf $(BUILD_DIR)\n\nanalyze:\n\tgo tool pprof $(BUILD_DIR)/$(PACKAGE).test $(BUILD_DIR)/$(CPU_PROF_NAME)\n" }, { "path": "test/integration/addcat_test.go", "content": "package integrationtest\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"math\"\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/boxo/bootstrap\"\n\t\"github.com/ipfs/boxo/files\"\n\tlogging \"github.com/ipfs/go-log/v2\"\n\t\"github.com/ipfs/go-test/random\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\tmock \"github.com/ipfs/kubo/core/mock\"\n\t\"github.com/ipfs/kubo/thirdparty/unit\"\n\ttestutil \"github.com/libp2p/go-libp2p-testing/net\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\nvar log = logging.Logger(\"epictest\")\n\nconst kSeed = 1\n\nfunc Test1KBInstantaneous(t *testing.T) {\n\tconf := testutil.LatencyConfig{\n\t\tNetworkLatency: 0,\n\t\tRoutingLatency: 0,\n\t\tBlockstoreLatency: 0,\n\t}\n\n\tif err := DirectAddCat(RandomBytes(1*unit.KB), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestDegenerateSlowBlockstore(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{BlockstoreLatency: 50 * time.Millisecond}\n\tif err := AddCatPowers(conf, 128); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestDegenerateSlowNetwork(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{NetworkLatency: 400 * time.Millisecond}\n\tif err := AddCatPowers(conf, 128); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestDegenerateSlowRouting(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{RoutingLatency: 400 * time.Millisecond}\n\tif err := AddCatPowers(conf, 128); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc Test100MBMacbookCoastToCoast(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{}.NetworkNYtoSF().BlockstoreSlowSSD2014().RoutingSlow()\n\tif err := DirectAddCat(RandomBytes(100*1024*1024), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc AddCatPowers(conf testutil.LatencyConfig, megabytesMax int64) error {\n\tvar i int64\n\tfor i = 1; i < megabytesMax; i = i * 2 {\n\t\tfmt.Printf(\"%d MB\\n\", i)\n\t\tif err := DirectAddCat(RandomBytes(i*1024*1024), conf); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc RandomBytes(n int64) []byte {\n\trandom.SetSeed(kSeed)\n\treturn random.Bytes(int(n))\n}\n\nfunc DirectAddCat(data []byte, conf testutil.LatencyConfig) error {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\t// create network\n\tmn := mocknet.New()\n\tmn.SetLinkDefaults(mocknet.LinkOptions{\n\t\tLatency: conf.NetworkLatency,\n\t\t// TODO add to conf. This is tricky because we want 0 values to be functional.\n\t\tBandwidth: math.MaxInt32,\n\t})\n\n\tadder, err := core.NewNode(ctx, &core.BuildCfg{\n\t\tOnline: true,\n\t\tHost: mock.MockHostOption(mn),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer adder.Close()\n\n\tcatter, err := core.NewNode(ctx, &core.BuildCfg{\n\t\tOnline: true,\n\t\tHost: mock.MockHostOption(mn),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer catter.Close()\n\n\tadderAPI, err := coreapi.NewCoreAPI(adder)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcatterAPI, err := coreapi.NewCoreAPI(catter)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = mn.LinkAll()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbs1 := []peer.AddrInfo{adder.Peerstore.PeerInfo(adder.Identity)}\n\tbs2 := []peer.AddrInfo{catter.Peerstore.PeerInfo(catter.Identity)}\n\n\tif err := catter.Bootstrap(bootstrap.BootstrapConfigWithPeers(bs1)); err != nil {\n\t\treturn err\n\t}\n\tif err := adder.Bootstrap(bootstrap.BootstrapConfigWithPeers(bs2)); err != nil {\n\t\treturn err\n\t}\n\n\tadded, err := adderAPI.Unixfs().Add(ctx, files.NewBytesFile(data))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treaderCatted, err := catterAPI.Unixfs().Get(ctx, added)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// verify\n\tvar bufout bytes.Buffer\n\t_, err = io.Copy(&bufout, readerCatted.(io.Reader))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif !bytes.Equal(bufout.Bytes(), data) {\n\t\treturn errors.New(\"catted data does not match added data\")\n\t}\n\n\treturn nil\n}\n\nfunc SkipUnlessEpic(t *testing.T) {\n\tif os.Getenv(\"IPFS_EPIC_TEST\") == \"\" {\n\t\tt.SkipNow()\n\t}\n}\n" }, { "path": "test/integration/bench_cat_test.go", "content": "package integrationtest\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"math\"\n\t\"testing\"\n\n\t\"github.com/ipfs/boxo/bootstrap\"\n\t\"github.com/ipfs/boxo/files\"\n\t\"github.com/ipfs/kubo/core\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\tmock \"github.com/ipfs/kubo/core/mock\"\n\t\"github.com/ipfs/kubo/thirdparty/unit\"\n\ttestutil \"github.com/libp2p/go-libp2p-testing/net\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\nfunc BenchmarkCat1MB(b *testing.B) { benchmarkVarCat(b, unit.MB*1) }\nfunc BenchmarkCat2MB(b *testing.B) { benchmarkVarCat(b, unit.MB*2) }\nfunc BenchmarkCat4MB(b *testing.B) { benchmarkVarCat(b, unit.MB*4) }\n\nfunc benchmarkVarCat(b *testing.B, size int64) {\n\tdata := RandomBytes(size)\n\tb.SetBytes(size)\n\tfor n := 0; n < b.N; n++ {\n\t\terr := benchCat(b, data, instant)\n\t\tif err != nil {\n\t\t\tb.Fatal(err)\n\t\t}\n\t}\n}\n\nfunc benchCat(b *testing.B, data []byte, conf testutil.LatencyConfig) error {\n\tb.StopTimer()\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\t// create network\n\tmn := mocknet.New()\n\tmn.SetLinkDefaults(mocknet.LinkOptions{\n\t\tLatency: conf.NetworkLatency,\n\t\t// TODO add to conf. This is tricky because we want 0 values to be functional.\n\t\tBandwidth: math.MaxInt32,\n\t})\n\n\tadder, err := core.NewNode(ctx, &core.BuildCfg{\n\t\tOnline: true,\n\t\tHost: mock.MockHostOption(mn),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer adder.Close()\n\n\tcatter, err := core.NewNode(ctx, &core.BuildCfg{\n\t\tOnline: true,\n\t\tHost: mock.MockHostOption(mn),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer catter.Close()\n\n\tadderAPI, err := coreapi.NewCoreAPI(adder)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcatterAPI, err := coreapi.NewCoreAPI(catter)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = mn.LinkAll()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbs1 := []peer.AddrInfo{adder.Peerstore.PeerInfo(adder.Identity)}\n\tbs2 := []peer.AddrInfo{catter.Peerstore.PeerInfo(catter.Identity)}\n\n\tif err := catter.Bootstrap(bootstrap.BootstrapConfigWithPeers(bs1)); err != nil {\n\t\treturn err\n\t}\n\tif err := adder.Bootstrap(bootstrap.BootstrapConfigWithPeers(bs2)); err != nil {\n\t\treturn err\n\t}\n\n\tadded, err := adderAPI.Unixfs().Add(ctx, files.NewBytesFile(data))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tb.StartTimer()\n\treaderCatted, err := catterAPI.Unixfs().Get(ctx, added)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// verify\n\tvar bufout bytes.Buffer\n\t_, err = io.Copy(&bufout, readerCatted.(io.Reader))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif !bytes.Equal(bufout.Bytes(), data) {\n\t\treturn errors.New(\"catted data does not match added data\")\n\t}\n\treturn nil\n}\n" }, { "path": "test/integration/bench_test.go", "content": "package integrationtest\n\nimport (\n\t\"testing\"\n\n\t\"github.com/ipfs/kubo/thirdparty/unit\"\n\ttestutil \"github.com/libp2p/go-libp2p-testing/net\"\n)\n\nfunc benchmarkAddCat(numBytes int64, conf testutil.LatencyConfig, b *testing.B) {\n\tb.StopTimer()\n\tb.SetBytes(numBytes)\n\tdata := RandomBytes(numBytes) // we don't want to measure the time it takes to generate this data\n\tb.StartTimer()\n\n\tfor n := 0; n < b.N; n++ {\n\t\tif err := DirectAddCat(data, conf); err != nil {\n\t\t\tb.Fatal(err)\n\t\t}\n\t}\n}\n\nvar instant = testutil.LatencyConfig{}.AllInstantaneous()\n\nfunc BenchmarkInstantaneousAddCat1KB(b *testing.B) { benchmarkAddCat(1*unit.KB, instant, b) }\nfunc BenchmarkInstantaneousAddCat1MB(b *testing.B) { benchmarkAddCat(1*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat2MB(b *testing.B) { benchmarkAddCat(2*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat4MB(b *testing.B) { benchmarkAddCat(4*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat8MB(b *testing.B) { benchmarkAddCat(8*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat16MB(b *testing.B) { benchmarkAddCat(16*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat32MB(b *testing.B) { benchmarkAddCat(32*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat64MB(b *testing.B) { benchmarkAddCat(64*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat128MB(b *testing.B) { benchmarkAddCat(128*unit.MB, instant, b) }\nfunc BenchmarkInstantaneousAddCat256MB(b *testing.B) { benchmarkAddCat(256*unit.MB, instant, b) }\n\nvar routing = testutil.LatencyConfig{}.RoutingSlow()\n\nfunc BenchmarkRoutingSlowAddCat1MB(b *testing.B) { benchmarkAddCat(1*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat2MB(b *testing.B) { benchmarkAddCat(2*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat4MB(b *testing.B) { benchmarkAddCat(4*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat8MB(b *testing.B) { benchmarkAddCat(8*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat16MB(b *testing.B) { benchmarkAddCat(16*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat32MB(b *testing.B) { benchmarkAddCat(32*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat64MB(b *testing.B) { benchmarkAddCat(64*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat128MB(b *testing.B) { benchmarkAddCat(128*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat256MB(b *testing.B) { benchmarkAddCat(256*unit.MB, routing, b) }\nfunc BenchmarkRoutingSlowAddCat512MB(b *testing.B) { benchmarkAddCat(512*unit.MB, routing, b) }\n\nvar network = testutil.LatencyConfig{}.NetworkNYtoSF()\n\nfunc BenchmarkNetworkSlowAddCat1MB(b *testing.B) { benchmarkAddCat(1*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat2MB(b *testing.B) { benchmarkAddCat(2*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat4MB(b *testing.B) { benchmarkAddCat(4*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat8MB(b *testing.B) { benchmarkAddCat(8*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat16MB(b *testing.B) { benchmarkAddCat(16*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat32MB(b *testing.B) { benchmarkAddCat(32*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat64MB(b *testing.B) { benchmarkAddCat(64*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat128MB(b *testing.B) { benchmarkAddCat(128*unit.MB, network, b) }\nfunc BenchmarkNetworkSlowAddCat256MB(b *testing.B) { benchmarkAddCat(256*unit.MB, network, b) }\n\nvar hdd = testutil.LatencyConfig{}.Blockstore7200RPM()\n\nfunc BenchmarkBlockstoreSlowAddCat1MB(b *testing.B) { benchmarkAddCat(1*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat2MB(b *testing.B) { benchmarkAddCat(2*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat4MB(b *testing.B) { benchmarkAddCat(4*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat8MB(b *testing.B) { benchmarkAddCat(8*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat16MB(b *testing.B) { benchmarkAddCat(16*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat32MB(b *testing.B) { benchmarkAddCat(32*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat64MB(b *testing.B) { benchmarkAddCat(64*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat128MB(b *testing.B) { benchmarkAddCat(128*unit.MB, hdd, b) }\nfunc BenchmarkBlockstoreSlowAddCat256MB(b *testing.B) { benchmarkAddCat(256*unit.MB, hdd, b) }\n\nvar mixed = testutil.LatencyConfig{}.NetworkNYtoSF().BlockstoreSlowSSD2014().RoutingSlow()\n\nfunc BenchmarkMixedAddCat1MBXX(b *testing.B) { benchmarkAddCat(1*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat2MBXX(b *testing.B) { benchmarkAddCat(2*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat4MBXX(b *testing.B) { benchmarkAddCat(4*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat8MBXX(b *testing.B) { benchmarkAddCat(8*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat16MBX(b *testing.B) { benchmarkAddCat(16*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat32MBX(b *testing.B) { benchmarkAddCat(32*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat64MBX(b *testing.B) { benchmarkAddCat(64*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat128MB(b *testing.B) { benchmarkAddCat(128*unit.MB, mixed, b) }\nfunc BenchmarkMixedAddCat256MB(b *testing.B) { benchmarkAddCat(256*unit.MB, mixed, b) }\n" }, { "path": "test/integration/bitswap_wo_routing_test.go", "content": "package integrationtest\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\tblocks \"github.com/ipfs/go-block-format\"\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/core\"\n\tcoremock \"github.com/ipfs/kubo/core/mock\"\n\t\"github.com/ipfs/kubo/core/node/libp2p\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\nfunc TestBitswapWithoutRouting(t *testing.T) {\n\tctx := t.Context()\n\tconst numPeers = 4\n\n\t// create network\n\tmn := mocknet.New()\n\n\tvar nodes []*core.IpfsNode\n\tfor range numPeers {\n\t\tn, err := core.NewNode(ctx, &core.BuildCfg{\n\t\t\tOnline: true,\n\t\t\tHost: coremock.MockHostOption(mn),\n\t\t\tRouting: libp2p.NilRouterOption, // no routing\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer n.Close()\n\t\tnodes = append(nodes, n)\n\t}\n\n\terr := mn.LinkAll()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// connect them\n\tfor _, n1 := range nodes {\n\t\tfor _, n2 := range nodes {\n\t\t\tif n1 == n2 {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlog.Debug(\"connecting to other hosts\")\n\t\t\tp2 := n2.PeerHost.Peerstore().PeerInfo(n2.PeerHost.ID())\n\t\t\tif err := n1.PeerHost.Connect(ctx, p2); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t}\n\n\t// add blocks to each before\n\tlog.Debug(\"adding block.\")\n\tblock0 := blocks.NewBlock([]byte(\"block0\"))\n\tblock1 := blocks.NewBlock([]byte(\"block1\"))\n\n\t// put 1 before\n\tif err := nodes[0].Blockstore.Put(ctx, block0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// get it out.\n\tfor i, n := range nodes {\n\t\t// skip first because block not in its exchange. will hang.\n\t\tif i == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tlog.Debugf(\"%d %s get block.\", i, n.Identity)\n\t\tb, err := n.Blocks.GetBlock(ctx, cid.NewCidV0(block0.Multihash()))\n\t\tif err != nil {\n\t\t\tt.Error(err)\n\t\t} else if !bytes.Equal(b.RawData(), block0.RawData()) {\n\t\t\tt.Error(\"byte comparison fail\")\n\t\t} else {\n\t\t\tlog.Debug(\"got block: %s\", b.Cid())\n\t\t}\n\t}\n\n\t// put 1 after\n\tif err := nodes[1].Blockstore.Put(ctx, block1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// get it out.\n\tfor _, n := range nodes {\n\t\tb, err := n.Blocks.GetBlock(ctx, cid.NewCidV0(block1.Multihash()))\n\t\tif err != nil {\n\t\t\tt.Error(err)\n\t\t} else if !bytes.Equal(b.RawData(), block1.RawData()) {\n\t\t\tt.Error(\"byte comparison fail\")\n\t\t} else {\n\t\t\tlog.Debug(\"got block: %s\", b.Cid())\n\t\t}\n\t}\n}\n" }, { "path": "test/integration/three_legged_cat_test.go", "content": "package integrationtest\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"math\"\n\t\"testing\"\n\t\"time\"\n\n\tbootstrap2 \"github.com/ipfs/boxo/bootstrap\"\n\t\"github.com/ipfs/kubo/core/coreapi\"\n\tmock \"github.com/ipfs/kubo/core/mock\"\n\t\"github.com/ipfs/kubo/thirdparty/unit\"\n\n\t\"github.com/ipfs/boxo/files\"\n\ttestutil \"github.com/libp2p/go-libp2p-testing/net\"\n\t\"github.com/libp2p/go-libp2p/core/peer\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n)\n\nfunc TestThreeLeggedCatTransfer(t *testing.T) {\n\tconf := testutil.LatencyConfig{\n\t\tNetworkLatency: 0,\n\t\tRoutingLatency: 0,\n\t\tBlockstoreLatency: 0,\n\t}\n\tif err := RunThreeLeggedCat(RandomBytes(1*unit.MB), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestThreeLeggedCatDegenerateSlowBlockstore(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{BlockstoreLatency: 50 * time.Millisecond}\n\tif err := RunThreeLeggedCat(RandomBytes(1*unit.KB), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestThreeLeggedCatDegenerateSlowNetwork(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{NetworkLatency: 400 * time.Millisecond}\n\tif err := RunThreeLeggedCat(RandomBytes(1*unit.KB), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestThreeLeggedCatDegenerateSlowRouting(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{RoutingLatency: 400 * time.Millisecond}\n\tif err := RunThreeLeggedCat(RandomBytes(1*unit.KB), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestThreeLeggedCat100MBMacbookCoastToCoast(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{}.NetworkNYtoSF().BlockstoreSlowSSD2014().RoutingSlow()\n\tif err := RunThreeLeggedCat(RandomBytes(100*unit.MB), conf); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc RunThreeLeggedCat(data []byte, conf testutil.LatencyConfig) error {\n\tctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)\n\tdefer cancel()\n\n\t// create network\n\tmn := mocknet.New()\n\tmn.SetLinkDefaults(mocknet.LinkOptions{\n\t\tLatency: conf.NetworkLatency,\n\t\t// TODO add to conf. This is tricky because we want 0 values to be functional.\n\t\tBandwidth: math.MaxInt32,\n\t})\n\n\tbootstrap, err := mock.MockPublicNode(ctx, mn)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer bootstrap.Close()\n\n\tadder, err := mock.MockPublicNode(ctx, mn)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer adder.Close()\n\n\tcatter, err := mock.MockPublicNode(ctx, mn)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer catter.Close()\n\n\tadderAPI, err := coreapi.NewCoreAPI(adder)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcatterAPI, err := coreapi.NewCoreAPI(catter)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = mn.LinkAll()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbis := bootstrap.Peerstore.PeerInfo(bootstrap.PeerHost.ID())\n\tbcfg := bootstrap2.BootstrapConfigWithPeers([]peer.AddrInfo{bis})\n\tif err := adder.Bootstrap(bcfg); err != nil {\n\t\treturn err\n\t}\n\tif err := catter.Bootstrap(bcfg); err != nil {\n\t\treturn err\n\t}\n\n\tadded, err := adderAPI.Unixfs().Add(ctx, files.NewBytesFile(data))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Explicitly provide the root CID to the DHT so the catter can discover\n\t// the adder. Without this, the async reprovider may not have propagated\n\t// the record before the catter queries.\n\tif err := adder.Routing.Provide(ctx, added.RootCid(), true); err != nil {\n\t\treturn err\n\t}\n\n\treaderCatted, err := catterAPI.Unixfs().Get(ctx, added)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// verify\n\tvar bufout bytes.Buffer\n\t_, err = io.Copy(&bufout, readerCatted.(io.Reader))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif !bytes.Equal(bufout.Bytes(), data) {\n\t\treturn errors.New(\"catted data does not match added data\")\n\t}\n\treturn nil\n}\n" }, { "path": "test/integration/wan_lan_dht_test.go", "content": "package integrationtest\n\nimport (\n\t\"context\"\n\t\"encoding/binary\"\n\t\"fmt\"\n\t\"math\"\n\t\"math/rand\"\n\t\"net\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/ipfs/go-cid\"\n\t\"github.com/ipfs/kubo/core\"\n\tmock \"github.com/ipfs/kubo/core/mock\"\n\tlibp2p2 \"github.com/ipfs/kubo/core/node/libp2p\"\n\n\ttestutil \"github.com/libp2p/go-libp2p-testing/net\"\n\tcorenet \"github.com/libp2p/go-libp2p/core/network\"\n\tmocknet \"github.com/libp2p/go-libp2p/p2p/net/mock\"\n\n\tma \"github.com/multiformats/go-multiaddr\"\n)\n\nfunc TestDHTConnectivityFast(t *testing.T) {\n\tconf := testutil.LatencyConfig{\n\t\tNetworkLatency: 0,\n\t\tRoutingLatency: 0,\n\t\tBlockstoreLatency: 0,\n\t}\n\tif err := RunDHTConnectivity(conf, 5); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestDHTConnectivitySlowNetwork(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{NetworkLatency: 400 * time.Millisecond}\n\tif err := RunDHTConnectivity(conf, 5); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestDHTConnectivitySlowRouting(t *testing.T) {\n\tSkipUnlessEpic(t)\n\tconf := testutil.LatencyConfig{RoutingLatency: 400 * time.Millisecond}\n\tif err := RunDHTConnectivity(conf, 5); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\n// wan prefix must have a real corresponding ASN for the peer diversity filter to work.\nvar (\n\twanPrefix = net.ParseIP(\"2001:218:3004::\")\n\tlanPrefix = net.ParseIP(\"fe80::\")\n)\n\nfunc makeAddr(n uint32, wan bool) ma.Multiaddr {\n\tvar ip net.IP\n\tif wan {\n\t\tip = append(net.IP{}, wanPrefix...)\n\t} else {\n\t\tip = append(net.IP{}, lanPrefix...)\n\t}\n\n\tbinary.LittleEndian.PutUint32(ip[12:], n)\n\taddr, _ := ma.NewMultiaddr(fmt.Sprintf(\"/ip6/%s/tcp/4242\", ip))\n\treturn addr\n}\n\nfunc RunDHTConnectivity(conf testutil.LatencyConfig, numPeers int) error {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\t// create network\n\tmn := mocknet.New()\n\tmn.SetLinkDefaults(mocknet.LinkOptions{\n\t\tLatency: conf.NetworkLatency,\n\t\tBandwidth: math.MaxInt32,\n\t})\n\n\ttestPeer, err := core.NewNode(ctx, &core.BuildCfg{\n\t\tOnline: true,\n\t\tHost: mock.MockHostOption(mn),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer testPeer.Close()\n\n\twanPeers := []*core.IpfsNode{}\n\tlanPeers := []*core.IpfsNode{}\n\n\tconnectionContext, connCtxCancel := context.WithTimeout(ctx, 15*time.Second)\n\tdefer connCtxCancel()\n\tfor i := range numPeers {\n\t\twanPeer, err := core.NewNode(ctx, &core.BuildCfg{\n\t\t\tOnline: true,\n\t\t\tRouting: libp2p2.DHTServerOption,\n\t\t\tHost: mock.MockHostOption(mn),\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer wanPeer.Close()\n\t\twanAddr := makeAddr(uint32(i), true)\n\t\t_ = wanPeer.PeerHost.Network().Listen(wanAddr)\n\t\tfor _, p := range wanPeers {\n\t\t\t_, _ = mn.LinkPeers(p.Identity, wanPeer.Identity)\n\t\t\t_ = wanPeer.PeerHost.Connect(connectionContext, p.Peerstore.PeerInfo(p.Identity))\n\t\t}\n\t\twanPeers = append(wanPeers, wanPeer)\n\n\t\tlanPeer, err := core.NewNode(ctx, &core.BuildCfg{\n\t\t\tOnline: true,\n\t\t\tHost: mock.MockHostOption(mn),\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer lanPeer.Close()\n\t\tlanAddr := makeAddr(uint32(i), false)\n\t\t_ = lanPeer.PeerHost.Network().Listen(lanAddr)\n\t\tfor _, p := range lanPeers {\n\t\t\t_, _ = mn.LinkPeers(p.Identity, lanPeer.Identity)\n\t\t\t_ = lanPeer.PeerHost.Connect(connectionContext, p.Peerstore.PeerInfo(p.Identity))\n\t\t}\n\t\tlanPeers = append(lanPeers, lanPeer)\n\t}\n\tconnCtxCancel()\n\n\t// Add interfaces / addresses to test peer.\n\twanAddr := makeAddr(0, true)\n\t_ = testPeer.PeerHost.Network().Listen(wanAddr)\n\tlanAddr := makeAddr(0, false)\n\t_ = testPeer.PeerHost.Network().Listen(lanAddr)\n\t// The test peer is connected to one lan peer.\n\tfor _, p := range lanPeers {\n\t\tif _, err := mn.LinkPeers(testPeer.Identity, p.Identity); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\terr = testPeer.PeerHost.Connect(ctx, lanPeers[0].Peerstore.PeerInfo(lanPeers[0].Identity))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tstartupCtx, startupCancel := context.WithTimeout(ctx, time.Second*60)\nStartupWait:\n\tfor {\n\t\tselect {\n\t\tcase err := <-testPeer.DHT.LAN.RefreshRoutingTable():\n\t\t\tif err != nil {\n\t\t\t\tfmt.Printf(\"Error refreshing routing table: %v\\n\", err)\n\t\t\t}\n\t\t\tif testPeer.DHT.LAN.RoutingTable() == nil ||\n\t\t\t\ttestPeer.DHT.LAN.RoutingTable().Size() == 0 ||\n\t\t\t\terr != nil {\n\t\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak StartupWait\n\t\tcase <-startupCtx.Done():\n\t\t\tstartupCancel()\n\t\t\treturn fmt.Errorf(\"expected faster dht bootstrap\")\n\t\t}\n\t}\n\tstartupCancel()\n\n\t// choose a lan peer and validate lan DHT is functioning.\n\ti := rand.Intn(len(lanPeers))\n\tif testPeer.PeerHost.Network().Connectedness(lanPeers[i].Identity) == corenet.Connected {\n\t\ti = (i + 1) % len(lanPeers)\n\t\tif testPeer.PeerHost.Network().Connectedness(lanPeers[i].Identity) == corenet.Connected {\n\t\t\t_ = testPeer.PeerHost.Network().ClosePeer(lanPeers[i].Identity)\n\t\t\ttestPeer.PeerHost.Peerstore().ClearAddrs(lanPeers[i].Identity)\n\t\t}\n\t}\n\t// That peer will provide a new CID, and we'll validate the test node can find it.\n\tprovideCid := cid.NewCidV1(cid.Raw, []byte(\"Lan Provide Record\"))\n\tprovideCtx, cancel := context.WithTimeout(ctx, time.Second)\n\tdefer cancel()\n\tif err := lanPeers[i].DHT.Provide(provideCtx, provideCid, true); err != nil {\n\t\treturn err\n\t}\n\tprovChan := testPeer.DHT.FindProvidersAsync(provideCtx, provideCid, 0)\n\tprov, ok := <-provChan\n\tif !ok || prov.ID == \"\" {\n\t\treturn fmt.Errorf(\"Expected provider. stream closed early\")\n\t}\n\tif prov.ID != lanPeers[i].Identity {\n\t\treturn fmt.Errorf(\"Unexpected lan peer provided record\")\n\t}\n\n\t// Now, connect with a wan peer.\n\tfor _, p := range wanPeers {\n\t\tif _, err := mn.LinkPeers(testPeer.Identity, p.Identity); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\terr = testPeer.PeerHost.Connect(ctx, wanPeers[0].Peerstore.PeerInfo(wanPeers[0].Identity))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tstartupCtx, startupCancel = context.WithTimeout(ctx, time.Second*60)\nWanStartupWait:\n\tfor {\n\t\tselect {\n\t\tcase err := <-testPeer.DHT.WAN.RefreshRoutingTable():\n\t\t\t// if err != nil {\n\t\t\t//\tfmt.Printf(\"Error refreshing routing table: %v\\n\", err)\n\t\t\t// }\n\t\t\tif testPeer.DHT.WAN.RoutingTable() == nil ||\n\t\t\t\ttestPeer.DHT.WAN.RoutingTable().Size() == 0 ||\n\t\t\t\terr != nil {\n\t\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak WanStartupWait\n\t\tcase <-startupCtx.Done():\n\t\t\tstartupCancel()\n\t\t\treturn fmt.Errorf(\"expected faster wan dht bootstrap\")\n\t\t}\n\t}\n\tstartupCancel()\n\n\t// choose a wan peer and validate wan DHT is functioning.\n\ti = rand.Intn(len(wanPeers))\n\tif testPeer.PeerHost.Network().Connectedness(wanPeers[i].Identity) == corenet.Connected {\n\t\ti = (i + 1) % len(wanPeers)\n\t\tif testPeer.PeerHost.Network().Connectedness(wanPeers[i].Identity) == corenet.Connected {\n\t\t\t_ = testPeer.PeerHost.Network().ClosePeer(wanPeers[i].Identity)\n\t\t\ttestPeer.PeerHost.Peerstore().ClearAddrs(wanPeers[i].Identity)\n\t\t}\n\t}\n\n\t// That peer will provide a new CID, and we'll validate the test node can find it.\n\twanCid := cid.NewCidV1(cid.Raw, []byte(\"Wan Provide Record\"))\n\twanProvideCtx, cancel := context.WithTimeout(ctx, time.Second)\n\tdefer cancel()\n\tif err := wanPeers[i].DHT.Provide(wanProvideCtx, wanCid, true); err != nil {\n\t\treturn err\n\t}\n\tprovChan = testPeer.DHT.FindProvidersAsync(wanProvideCtx, wanCid, 0)\n\tprov, ok = <-provChan\n\tif !ok || prov.ID == \"\" {\n\t\treturn fmt.Errorf(\"Expected one provider, closed early\")\n\t}\n\tif prov.ID != wanPeers[i].Identity {\n\t\treturn fmt.Errorf(\"Unexpected lan peer provided record\")\n\t}\n\n\t// Finally, re-share the lan provided cid from a wan peer and expect a merged result.\n\ti = rand.Intn(len(wanPeers))\n\tif testPeer.PeerHost.Network().Connectedness(wanPeers[i].Identity) == corenet.Connected {\n\t\t_ = testPeer.PeerHost.Network().ClosePeer(wanPeers[i].Identity)\n\t\ttestPeer.PeerHost.Peerstore().ClearAddrs(wanPeers[i].Identity)\n\t}\n\n\tprovideCtx, cancel = context.WithTimeout(ctx, time.Second)\n\tdefer cancel()\n\tif err := wanPeers[i].DHT.Provide(provideCtx, provideCid, true); err != nil {\n\t\treturn err\n\t}\n\tprovChan = testPeer.DHT.FindProvidersAsync(provideCtx, provideCid, 0)\n\tprov, ok = <-provChan\n\tif !ok {\n\t\treturn fmt.Errorf(\"Expected two providers, got 0\")\n\t}\n\tprov, ok = <-provChan\n\tif !ok {\n\t\treturn fmt.Errorf(\"Expected two providers, got 1\")\n\t}\n\n\treturn nil\n}\n" }, { "path": "test/ipfs-test-lib.sh", "content": "# Generic test functions for go-ipfs\n\nansi_strip() {\n sed 's/\\x1b\\[[0-9;]*m//g'\n}\n\n# Quote arguments for sh eval\nshellquote() {\n\t_space=''\n\tfor _arg\n\tdo\n\t\t# On macOS, sed adds a newline character.\n\t\t# With a printf wrapper the extra newline is removed.\n\t\tprintf \"$_space'%s'\" \"$(printf \"%s\" \"$_arg\" | sed -e \"s/'/'\\\\\\\\''/g;\")\"\n\t\t_space=' '\n\tdone\n\tprintf '\\n'\n}\n\n# Echo the args, run the cmd, and then also fail,\n# making sure a test case fails.\ntest_fsh() {\n echo \"> $@\"\n eval $(shellquote \"$@\")\n echo \"\"\n false\n}\n\n# Same as sharness' test_cmp but using test_fsh (to see the output).\n# We have to do it twice, so the first diff output doesn't show unless it's\n# broken.\ntest_cmp() {\n\tdiff -q \"$@\" >/dev/null || test_fsh diff -u \"$@\"\n}\n\n# Same as test_cmp above, but we sort files before comparing them.\ntest_sort_cmp() {\n\tsort \"$1\" >\"$1_sorted\" &&\n\tsort \"$2\" >\"$2_sorted\" &&\n\ttest_cmp \"$1_sorted\" \"$2_sorted\"\n}\n\n# Same as test_cmp above, but we standardize directory\n# separators before comparing the files.\ntest_path_cmp() {\n\tsed -e \"s/\\\\\\\\/\\//g\" \"$1\" >\"$1_std\" &&\n\tsed -e \"s/\\\\\\\\/\\//g\" \"$2\" >\"$2_std\" &&\n\ttest_cmp \"$1_std\" \"$2_std\"\n}\n\n# Docker\n\n# This takes a Dockerfile, a tag name, and a build context directory\ndocker_build() {\n docker build --rm --tag \"$1\" --file \"$2\" \"$3\" | ansi_strip\n}\n\n# This takes an image as argument and writes a docker ID on stdout\ndocker_run() {\n docker run --detach \"$1\"\n}\n\n# This takes a docker ID and a command as arguments\ndocker_exec() {\n docker exec --tty \"$1\" /bin/sh -c \"$2\"\n}\n\n# This takes a docker ID as argument\ndocker_stop() {\n docker stop \"$1\"\n}\n\n# This takes a docker ID as argument\ndocker_rm() {\n docker rm --force --volumes \"$1\" > /dev/null\n}\n\n# This takes a docker image name as argument\ndocker_rmi() {\n docker rmi --force \"$1\" > /dev/null\n}\n\n# Test whether all the expected lines are included in a file. The file\n# can have extra lines.\n#\n# $1 - Path to file with expected lines.\n# $2 - Path to file with actual output.\n#\n# Examples\n#\n# test_expect_success 'foo says hello' '\n# echo hello >expected &&\n# foo >actual &&\n# test_cmp expected actual\n# '\n#\n# Returns the exit code of the command set by TEST_CMP.\ntest_includes_lines() {\n\tsort \"$1\" >\"$1_sorted\" &&\n\tsort \"$2\" >\"$2_sorted\" &&\n\tcomm -2 -3 \"$1_sorted\" \"$2_sorted\" >\"$2_missing\" &&\n\t[ ! -s \"$2_missing\" ] || test_fsh comm -2 -3 \"$1_sorted\" \"$2_sorted\"\n}\n\n# Depending on GNU seq availability is not nice.\n# Git also has test_seq but it uses Perl.\ntest_seq() {\n\ttest \"$1\" -le \"$2\" || return\n\ti=\"$1\"\n\tj=\"$2\"\n\twhile test \"$i\" -le \"$j\"\n\tdo\n\t\techo \"$i\"\n\t\ti=$(expr \"$i\" + 1)\n\tdone\n}\n\nb64decode() {\n case `uname` in\n Linux|FreeBSD) base64 -d ;;\n Darwin) base64 -D ;;\n *)\n echo \"no compatible base64 command found\" >&2\n return 1\n esac\n}\n" }, { "path": "test/sharness/.gitignore", "content": "# symlinks to lib/sharness\n/sharness.sh\n/lib-sharness\n# clone of sharness\nlib/sharness/\n# deps downloaded by lib/*.sh scripts\nlib/dependencies/\n# sharness files\ntest-results/\ntrash directory.*.sh/\n# makefile files\nplugins\n# macos files\n*.DS_Store\n" }, { "path": "test/sharness/GNUmakefile", "content": "# default target is to run all tests\nall: aggregate\n\nSH := $(wildcard t[0-9][0-9][0-9][0-9]-*.sh)\n\n.DEFAULT $(SH): ALWAYS\n\t$(MAKE) -C ../.. test/sharness/$@\n\nALWAYS:\n.PHONY: ALWAYS\n" }, { "path": "test/sharness/README.md", "content": "# ipfs whole tests using the [sharness framework](https://github.com/pl-strflt/sharness/tree/feat/junit)\n\n## Running all the tests\n\nJust use `make` in this directory to run all the tests.\nRun with `TEST_VERBOSE=1` to get helpful verbose output.\n\n```\nTEST_VERBOSE=1 make\n```\n\nThe usual ipfs env flags also apply:\n\n```sh\n# the output will make your eyes bleed\nGOLOG_LOG_LEVEL=debug TEST_VERBOSE=1 make\n```\n\nTo make the tests abort as soon as an error occurs, use the TEST_IMMEDIATE env variable:\n\n```sh\n# this will abort as soon the first error occurs\nTEST_IMMEDIATE=1 make\n```\n\n## Running just one test\n\nYou can run only one test script by launching it like a regular shell\nscript:\n\n```\n$ ./t0010-basic-commands.sh\n```\n\n## Debugging one test\n\nYou can use the `-v` option to make it verbose and the `-i` option to\nmake it stop as soon as one test fails.\nFor example:\n\n```\n$ ./t0010-basic-commands.sh -v -i\n```\n\n## Sharness\n\nWhen running sharness tests from main Makefile or when `test_sharness_deps`\ntarget is run dependencies for sharness\nwill be downloaded from its GitHub repo and installed in a \"lib/sharness\"\ndirectory.\n\nPlease do not change anything in the \"lib/sharness\" directory.\n\nIf you really need some changes in sharness, please fork it from\n[its canonical repo](https://github.com/mlafeldt/sharness/) and\nsend pull requests there.\n\n## Writing Tests\n\nPlease have a look at existing tests and try to follow their example.\n\nWhen possible and not too inefficient, that means most of the time,\nan ipfs command should not be on the left side of a pipe, because if\nthe ipfs command fails (exit non zero), the pipe will mask this failure.\nFor example after `false | true`, `echo $?` prints 0 (despite `false`\nfailing).\n\nIt should be possible to put most of the code inside `test_expect_success`,\nor sometimes `test_expect_failure`, blocks, and to chain all the commands\ninside those blocks with `&&`, or `||` for diagnostic commands.\n\n### Diagnostics\n\nMake your test case output helpful for when running sharness verbosely.\nThis means cating certain files, or running diagnostic commands.\nFor example:\n\n```\ntest_expect_success \".ipfs/ has been created\" '\n test -d \".ipfs\" &&\n test -f \".ipfs/config\" &&\n test -d \".ipfs/datastore\" &&\n test -d \".ipfs/blocks\" ||\n test_fsh ls -al .ipfs\n'\n```\n\nThe `|| ...` is a diagnostic run when the preceding command fails.\ntest_fsh is a shell function that echoes the args, runs the cmd,\nand then also fails, making sure the test case fails. (wouldn't want\nthe diagnostic accidentally returning true and making it _seem_ like\nthe test case succeeded!).\n\n\n### Testing commands on daemon or mounted\n\nUse the provided functions in `lib/test-lib.sh` to run the daemon or mount:\n\nTo init, run daemon, and mount in one go:\n\n```sh\ntest_launch_ipfs_daemon_and_mount\n\ntest_expect_success \"'ipfs add --help' succeeds\" '\n ipfs add --help >actual\n'\n\n# other tests here...\n\n# don't forget to kill the daemon!!\ntest_kill_ipfs_daemon\n```\n\nTo init, run daemon, and then mount separately:\n\n```sh\ntest_init_ipfs\n\n# tests inited but not running here\n\ntest_launch_ipfs_daemon\n\n# tests running but not mounted here\n\ntest_mount_ipfs\n\n# tests mounted here\n\n# don't forget to kill the daemon!!\ntest_kill_ipfs_daemon\n```\n" }, { "path": "test/sharness/Rules.mk", "content": "include mk/header.mk\n\nSHARNESS_$(d) = $(d)/lib/sharness/sharness.sh\n\nT_$(d) = $(sort $(wildcard $(d)/t[0-9][0-9][0-9][0-9]-*.sh))\n\nDEPS_$(d) := test/bin/multihash test/bin/pollEndpoint test/bin/iptb \\\n\t test/bin/go-sleep test/bin/random-data test/bin/random-files \\\n\t test/bin/go-timeout test/bin/hang-fds test/bin/ma-pipe-unidir \\\n\t test/bin/cid-fmt\nDEPS_$(d) += cmd/ipfs/ipfs\nDEPS_$(d) += $(d)/clean-test-results\nDEPS_$(d) += $(SHARNESS_$(d))\n\nifeq ($(OS),Linux)\nPLUGINS_DIR_$(d) := $(d)/plugins/\nORIGIN_PLUGINS_$(d) := $(plugin/plugins_plugins_so)\nPLUGINS_$(d) := $(addprefix $(PLUGINS_DIR_$(d)),$(notdir $(ORIGIN_PLUGINS_$(d))))\n\n$(PLUGINS_$(d)): $(ORIGIN_PLUGINS_$(d))\n\t@mkdir -p $(@D)\n\tcp -f plugin/plugins/$(@F) $@\n\nifneq ($(TEST_PLUGIN),0)\nDEPS_$(d) += $(PLUGINS_$(d))\nendif\nendif\n\nexport MAKE_SKIP_PATH=1\n\n$(T_$(d)): $$(DEPS_$(d)) # use second expansion so coverage can inject dependency\n\t@echo \"*** $@ ***\"\nifeq ($(CONTINUE_ON_S_FAILURE),1)\n\t-@(cd $(@D) && ./$(@F)) 2>&1\nelse\n\t@(cd $(@D) && ./$(@F)) 2>&1\nendif\n.PHONY: $(T_$(d))\n\n$(d)/aggregate: $(T_$(d))\n\t@echo \"*** $@ ***\"\n\t@(cd $(@D) && ./lib/test-aggregate-results.sh)\n.PHONY: $(d)/aggregate\n\n$(d)/test-results/sharness.xml: $(T_$(d))\n\t@echo \"*** $@ ***\"\n\t@(cd $(@D)/.. && ./lib/test-aggregate-junit-reports.sh)\n.PHONY: $(d)/test-results/sharness.xml\n\n$(d)/clean-test-results:\n\trm -rf $(@D)/test-results\n.PHONY: $(d)/clean-test-results\n\nCLEAN += $(wildcard $(d)/test-results/*)\n\n$(SHARNESS_$(d)): $(d) ALWAYS\n\t@clonedir=$(dir $(@D)) $&2 \"$@\"\n exit 1\n}\n\nif test -d \"$clonedir/$sharnessdir\"; then\n giturl=\"git@github.com:${gitrepo}.git\"\n echo \"Checking if $giturl is already cloned (and if its origin is correct)\"\n if ! test -d \"$gitdir\" || test \"$(git --git-dir \"$gitdir\" remote get-url origin)\" != \"$giturl\"; then\n echo \"Removing $clonedir/$sharnessdir\"\n rm -rf \"$clonedir/$sharnessdir\" || die \"Could not remove $clonedir/$sharnessdir\"\n fi\nfi\n\nif ! test -d \"$clonedir/$sharnessdir\"; then\n giturl=\"https://github.com/${gitrepo}.git\"\n echo \"Cloning $giturl into $clonedir/$sharnessdir\"\n git clone \"$giturl\" \"$clonedir/$sharnessdir\" || die \"Could not clone $giturl into $clonedir/$sharnessdir\"\nfi\n\n\necho \"Changing directory to $clonedir/$sharnessdir\"\ncd \"$clonedir/$sharnessdir\" || die \"Could not cd into '$clonedir/$sharnessdir' directory\"\n\necho \"Checking if $githash is already fetched\"\nif ! git show \"$githash\" >/dev/null 2>&1; then\n echo \"Fetching $githash\"\n git fetch origin \"$githash\" || die \"Could not fetch $githash\"\nfi\n\necho \"Resetting to $githash\"\ngit reset --hard \"$githash\" || die \"Could not reset to $githash\"\n\nexit 0\n" }, { "path": "test/sharness/lib/iptb-lib.sh", "content": "# iptb test framework\n#\n# Copyright (c) 2014, 2016 Jeromy Johnson, Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n\nexport IPTB_ROOT=\"$(pwd)/.iptb\"\n\nipfsi() {\n dir=\"$1\"\n shift\n IPFS_PATH=\"$IPTB_ROOT/testbeds/default/$dir\" ipfs \"$@\"\n}\n\ncheck_has_connection() {\n node=\"$1\"\n ipfsi \"$node\" swarm peers >\"swarm_peers_$node\" &&\n grep \"p2p\" \"swarm_peers_$node\" >/dev/null\n}\n\niptb() {\n if ! command iptb \"$@\"; then\n case \"$1\" in\n start|stop|connect)\n test_fsh command iptb logs\n ;;\n esac\n return 1\n fi\n}\n\nstartup_cluster() {\n num_nodes=\"$1\"\n shift\n other_args=\"$@\"\n bound=$(expr \"$num_nodes\" - 1)\n\n test_expect_success \"set Routing.LoopbackAddressesOnLanDHT to true\" '\n iptb run [0-$bound] -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true\n '\n\n if test -n \"$other_args\"; then\n test_expect_success \"start up nodes with additional args\" \"\n iptb start -wait [0-$bound] -- ${other_args[@]}\n \"\n else\n test_expect_success \"start up nodes\" '\n iptb start -wait [0-$bound]\n '\n fi\n\n test_expect_success \"connect nodes to each other\" '\n iptb connect [1-$bound] 0\n '\n\n for i in $(test_seq 0 \"$bound\")\n do\n test_expect_success \"node $i is connected\" '\n check_has_connection \"$i\" ||\n test_fsh cat \"swarm_peers_$i\"\n '\n done\n}\n\niptb_wait_stop() {\n while ! iptb run -- sh -c '! { test -e \"$IPFS_PATH/repo.lock\" && fuser -f \"$IPFS_PATH/repo.lock\" >/dev/null; }'; do\n go-sleep 10ms\n done\n}\n" }, { "path": "test/sharness/lib/test-aggregate-junit-reports.sh", "content": "#!/bin/sh\n#\n# Script to aggregate results using Sharness\n#\n# Copyright (c) 2014, 2022 Christian Couder, Piotr Galar\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\nSHARNESS_AGGREGATE_JUNIT=\"lib/sharness/aggregate-junit-reports.sh\"\n\ntest -f \"$SHARNESS_AGGREGATE_JUNIT\" || {\n echo >&2 \"Cannot find: $SHARNESS_AGGREGATE_JUNIT\"\n echo >&2 \"Please check Sharness installation.\"\n exit 1\n}\n\nls test-results/t*-*.sh.*.xml.part | \"$SHARNESS_AGGREGATE_JUNIT\" > test-results/sharness.xml\n" }, { "path": "test/sharness/lib/test-aggregate-results.sh", "content": "#!/bin/sh\n#\n# Script to aggregate results using Sharness\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\nSHARNESS_AGGREGATE=\"lib/sharness/aggregate-results.sh\"\n\ntest -f \"$SHARNESS_AGGREGATE\" || {\n echo >&2 \"Cannot find: $SHARNESS_AGGREGATE\"\n echo >&2 \"Please check Sharness installation.\"\n exit 1\n}\n\nls test-results/t*-*.sh.*.counts | \"$SHARNESS_AGGREGATE\"\n" }, { "path": "test/sharness/lib/test-lib-hashes.sh", "content": "# this file defines several useful hashes used across the test codebase.\n# thus they can be defined + changed in one place\n\nHASH_WELCOME_DOCS=\"QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc\"\nHASH_EMPTY_DIR=\"QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\"\n" }, { "path": "test/sharness/lib/test-lib.sh", "content": "# Test framework for go-ipfs\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n# We are using sharness (https://github.com/pl-strflt/sharness/tree/feat/junit)\n# which was extracted from the Git test framework.\n\n# use the ipfs tool to test against\n\n# add current directory to path, for ipfs tool.\nif test \"$MAKE_SKIP_PATH\" != \"1\"; then\n BIN=$(cd .. && echo `pwd`/bin)\n BIN2=$(cd ../.. && echo `pwd`/cmd/ipfs)\n PATH=${BIN2}:${BIN}:${PATH}\n\n # assert the `ipfs` we're using is the right one.\n if test `which ipfs` != ${BIN2}/ipfs; then\n echo >&2 \"Cannot find the tests' local ipfs tool.\"\n echo >&2 \"Please check test and ipfs tool installation.\"\n exit 1\n fi\nfi\n\n# set sharness verbosity. we set the env var directly as\n# it's too late to pass in --verbose, and --verbose is harder\n# to pass through in some cases.\ntest \"$TEST_VERBOSE\" = 1 && verbose=t\ntest \"$TEST_IMMEDIATE\" = 1 && immediate=t\ntest \"$TEST_JUNIT\" = 1 && junit=t\ntest \"$TEST_NO_COLOR\" = 1 && no_color=t\n# source the common hashes first.\n. lib/test-lib-hashes.sh\n\n\nln -sf lib/sharness/sharness.sh .\nln -sf lib/sharness/lib-sharness .\n\n. \"sharness.sh\" || {\n echo >&2 \"Cannot source: sharness.sh\"\n echo >&2 \"Please check Sharness installation.\"\n exit 1\n}\n\n# Please put go-ipfs specific shell functions below\n\n###\n# BEGIN Check for pre-existing daemon being stuck\n###\nwait_prev_cleanup_tick_secs=1\nwait_prev_cleanup_max_secs=5\ncur_test_pwd=\"$(pwd)\"\n\nwhile true ; do\n echo -n > stuck_cwd_list\n\n timeout 5 lsof -c ipfs -Ffn 2>/dev/null | grep -A1 '^fcwd$' | grep '^n' | cut -b 2- | while read -r pwd_of_stuck ; do\n case \"$pwd_of_stuck\" in\n \"$cur_test_pwd\"*)\n echo \"$pwd_of_stuck\" >> stuck_cwd_list\n ;;\n *)\n ;;\n esac\n done\n\n test -s stuck_cwd_list || break\n\n test \"$wait_prev_cleanup_max_secs\" -le 0 && break\n\n echo \"Daemons still running, waiting for ${wait_prev_cleanup_max_secs}s\"\n sleep $wait_prev_cleanup_tick_secs\n\n wait_prev_cleanup_max_secs=\"$(( $wait_prev_cleanup_max_secs - $wait_prev_cleanup_tick_secs ))\"\ndone\n\nif test -s stuck_cwd_list ; then\n test_expect_success \"ipfs daemon (s)seems to be running with CWDs of\n$(cat stuck_cwd_list)\nAlmost certainly a leftover from a prior test, ABORTING\" 'false'\n\n test_done\nfi\n###\n# END Check for pre-existing daemon being stuck\n###\n\n# Make sure the ipfs path is set, also set in test_init_ipfs but that\n# is not always used.\nexport IPFS_PATH=\"$(pwd)/.ipfs\"\n# Ask programs to please not print ANSI codes\nexport TERM=dumb\n\nTEST_OS=\"$(uname -s | tr '[a-z]' '[A-Z]')\"\n\n# grab + output options\ntest \"$TEST_FUSE\" = 1 && test_set_prereq FUSE\ntest \"$TEST_EXPENSIVE\" = 1 && test_set_prereq EXPENSIVE\ntest \"$TEST_DOCKER\" = 1 && type docker >/dev/null 2>&1 && groups | egrep \"\\bdocker\\b\" && test_set_prereq DOCKER\ntest \"$TEST_PLUGIN\" = 1 && test \"$TEST_OS\" = \"LINUX\" && test_set_prereq PLUGIN\n\n# this may not be available, skip a few dependent tests\ntype socat >/dev/null 2>&1 && test_set_prereq SOCAT\ntype unzip >/dev/null 2>&1 && test_set_prereq UNZIP\n\n\n# Set a prereq as error messages are often different on Windows/Cygwin\nexpr \"$TEST_OS\" : \"CYGWIN_NT\" >/dev/null || test_set_prereq STD_ERR_MSG\n\nif test \"$TEST_VERBOSE\" = 1; then\n echo '# TEST_VERBOSE='\"$TEST_VERBOSE\"\n echo '# TEST_IMMEDIATE='\"$TEST_IMMEDIATE\"\n echo '# TEST_FUSE='\"$TEST_FUSE\"\n echo '# TEST_DOCKER='\"$TEST_DOCKER\"\n echo '# TEST_PLUGIN='\"$TEST_PLUGIN\"\n echo '# TEST_EXPENSIVE='\"$TEST_EXPENSIVE\"\n echo '# TEST_OS='\"$TEST_OS\"\n echo '# TEST_JUNIT='\"$TEST_JUNIT\"\n echo '# TEST_NO_COLOR='\"$TEST_NO_COLOR\"\n echo '# TEST_ULIMIT_PRESET='\"$TEST_ULIMIT_PRESET\"\nfi\n\n# source our generic test lib\n. ../../ipfs-test-lib.sh\n\n# source iptb lib\n. ../lib/iptb-lib.sh\n\ntest_cmp_repeat_10_sec() {\n for i in $(test_seq 1 100)\n do\n test_cmp \"$1\" \"$2\" >/dev/null && return\n go-sleep 100ms\n done\n test_cmp \"$1\" \"$2\"\n}\n\ntest_run_repeat_60_sec() {\n for i in $(test_seq 1 600)\n do\n (test_eval_ \"$1\") && return\n go-sleep 100ms\n done\n return 1 # failed\n}\n\ntest_wait_output_n_lines() {\n for i in $(test_seq 1 3600)\n do\n test $(cat \"$1\" | wc -l | tr -d \" \") -ge $2 && return\n go-sleep 100ms\n done\n actual=$(cat \"$1\" | wc -l | tr -d \" \")\n test_fsh \"expected $2 lines of output. got $actual\"\n}\n\ntest_wait_open_tcp_port_10_sec() {\n for i in $(test_seq 1 100)\n do\n # this is not a perfect check, but it's portable.\n # can't count on ss. not installed everywhere.\n # can't count on netstat using : or . as port delim. differ across platforms.\n echo $(netstat -aln | egrep \"^tcp.*LISTEN\" | egrep \"[.:]$1\" | wc -l) -gt 0\n if [ $(netstat -aln | egrep \"^tcp.*LISTEN\" | egrep \"[.:]$1\" | wc -l) -gt 0 ]; then\n return 0\n fi\n go-sleep 100ms\n done\n return 1\n}\n\n\n# test_config_set helps us make sure _we really did set_ a config value.\n# it sets it and then tests it. This became elaborate because ipfs config\n# was setting really weird things and am not sure why.\ntest_config_set() {\n\n # grab flags (like --bool in \"ipfs config --bool\")\n test_cfg_flags=\"\" # unset in case.\n test \"$#\" = 3 && { test_cfg_flags=$1; shift; }\n\n test_cfg_key=$1\n test_cfg_val=$2\n\n # when verbose, tell the user what config values are being set\n test_cfg_cmd=\"ipfs config $test_cfg_flags \\\"$test_cfg_key\\\" \\\"$test_cfg_val\\\"\"\n test \"$TEST_VERBOSE\" = 1 && echo \"$test_cfg_cmd\"\n\n # ok try setting the config key/val pair.\n ipfs config $test_cfg_flags \"$test_cfg_key\" \"$test_cfg_val\"\n echo \"$test_cfg_val\" >cfg_set_expected\n ipfs config \"$test_cfg_key\" >cfg_set_actual\n test_cmp cfg_set_expected cfg_set_actual\n}\n\ntest_init_ipfs() {\n args=(\"$@\")\n\n # we set the Addresses.API config variable.\n # the cli client knows to use it, so only need to set.\n # todo: in the future, use env?\n\n test_expect_success \"ipfs init succeeds\" '\n export IPFS_PATH=\"$(pwd)/.ipfs\" &&\n ipfs init \"${args[@]}\" --profile=test > /dev/null\n '\n\n test_expect_success \"disable telemetry\" '\n test_config_set --bool Plugins.Plugins.telemetry.Disabled \"true\"\n '\n\n test_expect_success \"prepare config -- mounting\" '\n mkdir mountdir ipfs ipns mfs &&\n test_config_set Mounts.IPFS \"$(pwd)/ipfs\" &&\n test_config_set Mounts.IPNS \"$(pwd)/ipns\" &&\n test_config_set Mounts.MFS \"$(pwd)/mfs\" ||\n test_fsh cat \"\\\"$IPFS_PATH/config\\\"\"\n '\n\n}\n\ntest_init_ipfs_measure() {\n args=(\"$@\")\n\n # we set the Addresses.API config variable.\n # the cli client knows to use it, so only need to set.\n # todo: in the future, use env?\n\n test_expect_success \"ipfs init succeeds\" '\n export IPFS_PATH=\"$(pwd)/.ipfs\" &&\n ipfs init \"${args[@]}\" --profile=test,flatfs-measure > /dev/null\n '\n\n test_expect_success \"disable telemetry\" '\n test_config_set --bool Plugins.Plugins.telemetry.Disabled \"true\"\n '\n\n test_expect_success \"prepare config -- mounting\" '\n mkdir mountdir ipfs ipns &&\n test_config_set Mounts.IPFS \"$(pwd)/ipfs\" &&\n test_config_set Mounts.IPNS \"$(pwd)/ipns\" ||\n test_fsh cat \"\\\"$IPFS_PATH/config\\\"\"\n '\n\n}\n\ntest_wait_for_file() {\n loops=$1\n delay=$2\n file=$3\n fwaitc=0\n while ! test -f \"$file\"\n do\n if test $fwaitc -ge $loops\n then\n echo \"Error: timed out waiting for file: $file\"\n return 1\n fi\n\n go-sleep $delay\n fwaitc=`expr $fwaitc + 1`\n done\n}\n\ntest_set_address_vars() {\n daemon_output=\"$1\"\n\n test_expect_success \"set up address variables\" '\n API_MADDR=$(cat \"$IPFS_PATH/api\") &&\n API_ADDR=$(convert_tcp_maddr $API_MADDR) &&\n API_PORT=$(port_from_maddr $API_MADDR) &&\n\n GWAY_MADDR=$(sed -n \"s/^Gateway server listening on //p\" \"$daemon_output\") &&\n GWAY_ADDR=$(convert_tcp_maddr $GWAY_MADDR) &&\n GWAY_PORT=$(port_from_maddr $GWAY_MADDR)\n '\n\n if ipfs swarm addrs local >/dev/null 2>&1; then\n test_expect_success \"get swarm addresses\" '\n ipfs swarm addrs local > addrs_out\n '\n\n test_expect_success \"set swarm address vars\" '\n SWARM_MADDR=$(grep \"127.0.0.1\" addrs_out) &&\n SWARM_PORT=$(port_from_maddr $SWARM_MADDR)\n '\n fi\n}\n\ntest_launch_ipfs_daemon() {\n\n args=(\"$@\")\n\n test \"$TEST_ULIMIT_PRESET\" != 1 && ulimit -n 2048\n\n test_expect_success \"'ipfs daemon' succeeds\" '\n ipfs daemon \"${args[@]}\" >actual_daemon 2>daemon_err &\n IPFS_PID=$!\n '\n\n # wait for api file to show up\n test_expect_success \"api file shows up\" '\n test_wait_for_file 50 200ms \"$IPFS_PATH/api\"\n '\n\n test_set_address_vars actual_daemon\n\n # we say the daemon is ready when the API server is ready.\n test_expect_success \"'ipfs daemon' is ready\" '\n pollEndpoint -host=$API_MADDR -v -tout=1s -tries=60 2>poll_apierr > poll_apiout ||\n test_fsh cat actual_daemon || test_fsh cat daemon_err || test_fsh cat poll_apierr || test_fsh cat poll_apiout\n '\n}\n\ntest_launch_ipfs_daemon_without_network() {\n test_launch_ipfs_daemon --offline \"$@\"\n}\n\ndo_umount() {\n local mount_point=\"$1\"\n local max_retries=3\n local retry_delay=0.5\n \n # Try normal unmount first (without lazy flag)\n for i in $(seq 1 $max_retries); do\n if [ \"$(uname -s)\" = \"Linux\" ]; then\n # First attempt: standard unmount\n if fusermount -u \"$mount_point\" 2>/dev/null; then\n return 0\n fi\n else\n if umount \"$mount_point\" 2>/dev/null; then\n return 0\n fi\n fi\n \n # If not last attempt, wait before retry\n if [ $i -lt $max_retries ]; then\n go-sleep \"${retry_delay}s\"\n fi\n done\n \n # If normal unmount failed, try lazy unmount as last resort (Linux only)\n if [ \"$(uname -s)\" = \"Linux\" ]; then\n # Log that we're falling back to lazy unmount\n test \"$TEST_VERBOSE\" = 1 && echo \"# Warning: falling back to lazy unmount for $mount_point\"\n fusermount -z -u \"$mount_point\" 2>/dev/null\n else\n # On non-Linux, try force unmount\n umount -f \"$mount_point\" 2>/dev/null || true\n fi\n}\n\ntest_mount_ipfs() {\n\n # make sure stuff is unmounted first.\n test_expect_success FUSE \"'ipfs mount' succeeds\" '\n do_umount \"$(pwd)/ipfs\" || true &&\n do_umount \"$(pwd)/ipns\" || true &&\n do_umount \"$(pwd)/mfs\" || true &&\n ipfs mount >actual\n '\n\n test_expect_success FUSE \"'ipfs mount' output looks good\" '\n echo \"IPFS mounted at: $(pwd)/ipfs\" >expected &&\n echo \"IPNS mounted at: $(pwd)/ipns\" >>expected &&\n echo \"MFS mounted at: $(pwd)/mfs\" >>expected &&\n test_cmp expected actual\n '\n\n}\n\ntest_launch_ipfs_daemon_and_mount() {\n\n test_init_ipfs\n test_launch_ipfs_daemon\n test_mount_ipfs\n\n}\n\ntest_kill_repeat_10_sec() {\n # try to shut down once + wait for graceful exit\n kill $1\n for i in $(test_seq 1 100)\n do\n go-sleep 100ms\n ! kill -0 $1 2>/dev/null && return\n done\n\n # if not, try once more, which will skip graceful exit\n kill $1\n go-sleep 1s\n ! kill -0 $1 2>/dev/null && return\n\n # ok, no hope. kill it to prevent it messing with other tests\n kill -9 $1 2>/dev/null\n return 1\n}\n\ntest_kill_ipfs_daemon() {\n\n test_expect_success \"'ipfs daemon' is still running\" '\n kill -0 $IPFS_PID\n '\n\n test_expect_success \"'ipfs daemon' can be killed\" '\n test_kill_repeat_10_sec $IPFS_PID\n '\n}\n\ntest_curl_resp_http_code() {\n curl -I \"$1\" >curl_output || {\n echo \"curl error with url: '$1'\"\n echo \"curl output was:\"\n cat curl_output\n return 1\n }\n shift &&\n RESP=$(head -1 curl_output) &&\n while test \"$#\" -gt 0\n do\n expr \"$RESP\" : \"$1\" >/dev/null && return\n shift\n done\n echo \"curl response didn't match!\"\n echo \"curl response was: '$RESP'\"\n echo \"curl output was:\"\n cat curl_output\n return 1\n}\n\ntest_must_be_empty() {\n if test -s \"$1\"\n then\n echo \"'$1' is not empty, it contains:\"\n cat \"$1\"\n return 1\n fi\n}\n\ntest_should_contain() {\n test \"$#\" = 2 || error \"bug in the test script: not 2 parameters to test_should_contain\"\n if ! grep -q \"$1\" \"$2\"\n then\n echo \"'$2' does not contain '$1', it contains:\"\n cat \"$2\"\n return 1\n fi\n}\n\ntest_should_not_contain() {\n test \"$#\" = 2 || error \"bug in the test script: not 2 parameters to test_should_not_contain\"\n if grep -q \"$1\" \"$2\"\n then\n echo \"'$2' contains undesired value '$1'\"\n return 1\n fi\n}\n\ntest_str_contains() {\n find=$1\n shift\n echo \"$@\" | egrep \"\\b$find\\b\" >/dev/null\n}\n\ndisk_usage() {\n # normalize du across systems\n case $(uname -s) in\n Linux)\n DU=\"du -sb\"\n M=1\n ;;\n FreeBSD)\n DU=\"du -s -A -B 1\"\n M=512\n ;;\n Darwin | DragonFly | *)\n DU=\"du -s\"\n M=512\n ;;\n esac\n expr $($DU \"$1\" | awk \"{print \\$1}\") \"*\" \"$M\"\n}\n\n# output a file's permission in human readable format\ngeneric_stat() {\n # normalize stat across systems\n case $(uname -s) in\n Linux)\n _STAT=\"stat -c %A\"\n ;;\n FreeBSD | Darwin | DragonFly)\n _STAT=\"stat -f %Sp\"\n ;;\n *)\n echo \"unsupported OS\" >&2\n exit 1\n ;;\n esac\n $_STAT \"$1\" || echo \"failed\" # Avoid returning nothing.\n}\n\n# output a file's permission in human readable format\nfile_size() {\n case $(uname -s) in\n Linux)\n _STAT=\"stat --format=%s\"\n ;;\n FreeBSD | Darwin | DragonFly)\n _STAT=\"stat -f%z\"\n ;;\n *)\n echo \"unsupported OS\" >&2\n exit 1\n ;;\n esac\n $_STAT \"$1\"\n}\n\n# len 46: 2048-bit RSA keys, b58mh-encoded\n# len 52: ED25519 keys, b58mh-encoded\n# len 56: 2048-bit RSA keys, base36-encoded\n# len 62: ED25519 keys, base36-encoded\ntest_check_peerid() {\n peeridlen=$(echo \"$1\" | tr -dC \"[:alnum:]\" | wc -c | tr -d \" \") &&\n test \"$peeridlen\" = \"46\" -o \"$peeridlen\" = \"52\" -o \"$peeridlen\" = \"56\" -o \"$peeridlen\" = \"62\" || {\n echo \"Bad peerid '$1' with len '$peeridlen'\"\n return 1\n }\n}\n\ntest_check_rsa2048_b58mh_peerid() {\n peeridlen=$(echo \"$1\" | tr -dC \"[:alnum:]\" | wc -c | tr -d \" \") &&\n test \"$peeridlen\" = \"46\" || {\n echo \"Bad RSA2048 B58MH peerid '$1' with len '$peeridlen'\"\n return 1\n }\n}\n\ntest_check_ed25519_b58mh_peerid() {\n peeridlen=$(echo \"$1\" | tr -dC \"[:alnum:]\" | wc -c | tr -d \" \") &&\n test \"$peeridlen\" = \"52\" || {\n echo \"Bad ED25519 B58MH peerid '$1' with len '$peeridlen'\"\n return 1\n }\n}\n\ntest_check_rsa2048_base36_peerid() {\n peeridlen=$(echo \"$1\" | tr -dC \"[:alnum:]\" | wc -c | tr -d \" \") &&\n test \"$peeridlen\" = \"56\" || {\n echo \"Bad RSA2048 B36CID peerid '$1' with len '$peeridlen'\"\n return 1\n }\n}\n\ntest_check_ed25519_base36_peerid() {\n peeridlen=$(echo \"$1\" | tr -dC \"[:alnum:]\" | wc -c | tr -d \" \") &&\n test \"$peeridlen\" = \"62\" || {\n echo \"Bad ED25519 B36CID peerid '$1' with len '$peeridlen'\"\n return 1\n }\n}\n\nconvert_tcp_maddr() {\n echo $1 | awk -F'/' '{ printf \"%s:%s\", $3, $5 }'\n}\n\nport_from_maddr() {\n echo $1 | awk -F'/' '{ print $NF }'\n}\n\nfindprovs_empty() {\n test_expect_success 'findprovs '$1' succeeds' '\n ipfsi 1 routing findprovs -n 1 '$1' > findprovsOut\n '\n\n test_expect_success \"findprovs $1 output is empty\" '\n test_must_be_empty findprovsOut\n '\n}\n\nfindprovs_expect() {\n test_expect_success 'findprovs '$1' succeeds' '\n ipfsi 1 routing findprovs -n 1 '$1' > findprovsOut &&\n echo '$2' > expected\n '\n\n test_expect_success \"findprovs $1 output looks good\" '\n test_cmp findprovsOut expected\n '\n}\n\npurge_blockstore() {\n ipfs pin ls --quiet --type=recursive | ipfs pin rm &>/dev/null\n ipfs repo gc --silent &>/dev/null\n\n test_expect_success \"pinlist empty\" '\n [[ -z \"$( ipfs pin ls )\" ]]\n '\n test_expect_success \"nothing left to gc\" '\n [[ -z \"$( ipfs repo gc )\" ]]\n '\n}\n" }, { "path": "test/sharness/t0001-tests-work.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test sharness tests are correctly written\"\n\n. lib/test-lib.sh\n\nfor file in $(find .. -maxdepth 1 -name 't*.sh' -type f); do\n test_expect_success \"test in $file finishes\" '\n grep -q \"^test_done\\b\" \"$file\"\n '\n\n test_expect_success \"test in $file has a description\" '\n grep -q \"^test_description=\" \"$file\"\n '\n\n # We have some tests that manually kill.\n case \"$(basename \"$file\")\" in\n t0060-daemon.sh|t0023-shutdown.sh) continue ;;\n esac\n\n test_expect_success \"test in $file has matching ipfs start/stop\" '\n awk \"/^ *[^#]*test_launch_ipfs_daemon/ { if (count != 0) { exit(1) }; count++ } /^ *[^#]*test_kill_ipfs_daemon/ { if (count != 1) { exit(1) }; count-- } END { exit(count) }\" \"$file\"\n '\ndone\n\ntest_done\n" }, { "path": "test/sharness/t0002-docker-image.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test docker image\"\n\n. lib/test-lib.sh\n\n# if in travis CI on OSX, docker is not available\nif ! test_have_prereq DOCKER; then\n skip_all='skipping docker tests, docker not available'\n\n test_done\nfi\n\ntest_expect_success \"'docker --version' works\" '\n docker --version >actual\n'\n\ntest_expect_success \"'docker --version' output looks good\" '\n egrep \"^Docker version\" actual\n'\n\nTEST_TRASH_DIR=$(pwd)\nTEST_SCRIPTS_DIR=$(dirname \"$TEST_TRASH_DIR\")\nTEST_TESTS_DIR=$(dirname \"$TEST_SCRIPTS_DIR\")\nAPP_ROOT_DIR=$(dirname \"$TEST_TESTS_DIR\")\nIMAGE_TAG=kubo_test\n\ntest_expect_success \"docker image build succeeds\" '\n docker_build \"$IMAGE_TAG\" \"$TEST_TESTS_DIR/../Dockerfile\" \"$APP_ROOT_DIR\" ||\n test_fsh echo \"TEST_TESTS_DIR: $TEST_TESTS_DIR\" ||\n test_fsh echo \"APP_ROOT_DIR : $APP_ROOT_DIR\"\n'\n\ntest_expect_success \"write init scripts\" '\n echo \"ipfs config Mounts.IPFS Bar\" > 001.sh &&\n echo \"ipfs config Pubsub.Router Qux\" > 002.sh &&\n chmod +x 002.sh\n'\n\ntest_expect_success \"docker image runs\" '\n DOC_ID=$(docker run -d \\\n -p 127.0.0.1:5001:5001 -p 127.0.0.1:8080:8080 \\\n -v \"$PWD/001.sh\":/container-init.d/001.sh \\\n -v \"$PWD/002.sh\":/container-init.d/002.sh \\\n \"$IMAGE_TAG\")\n'\n\ntest_expect_success \"docker container gateway is up\" '\n pollEndpoint -host=/ip4/127.0.0.1/tcp/8080 -http-url http://localhost:8080/ipfs/bafkqaddimvwgy3zao5xxe3debi -v -tries 30 -tout 1s\n'\n\ntest_expect_success \"docker container API is up\" '\n pollEndpoint -host=/ip4/127.0.0.1/tcp/5001 -http-url http://localhost:5001/version -v -tries 30 -tout 1s\n'\n\ntest_expect_success \"check that init scripts were run correctly and in the correct order\" \"\n echo -e \\\"Sourcing '/container-init.d/001.sh'...\\nExecuting '/container-init.d/002.sh'...\\\" > expected &&\n docker logs $DOC_ID 2>/dev/null | grep -e 001.sh -e 002.sh > actual &&\n test_cmp actual expected\n\"\n\ntest_expect_success \"check that init script configs were applied\" '\n echo Bar > expected &&\n docker exec \"$DOC_ID\" ipfs config Mounts.IPFS > actual &&\n test_cmp actual expected &&\n echo Qux > expected &&\n docker exec \"$DOC_ID\" ipfs config Pubsub.Router > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"simple ipfs add/cat can be run in docker container\" '\n echo \"Hello Worlds\" | tr -d \"[:cntrl:]\" > expected &&\n HASH=$(docker_exec \"$DOC_ID\" \"echo $(cat expected) | ipfs add -q\" | tr -d \"[:cntrl:]\") &&\n docker_exec \"$DOC_ID\" \"ipfs cat $HASH\" | tr -d \"[:cntrl:]\" > actual &&\n test_cmp expected actual\n'\n\nread testcode <actual ; \\\n test -s actual ; \\\n docker exec -i \"$DOC_ID\" ipfs version --enc json \\\n | sed 's/^.*\"Commit\":\"\\\\\\([^\"]*\\\\\\)\".*$/\\\\\\1/g' >expected ; \\\n test -s expected ; \\\n test_cmp expected actual\nEOF\ntest_expect_success \"version CurrentCommit is set\" \"$testcode\"\n\ntest_expect_success \"stop docker container\" '\n docker_stop \"$DOC_ID\"\n'\n\ndocker_rm \"$DOC_ID\"\ndocker_rmi \"$IMAGE_TAG\"\ntest_done\n" }, { "path": "test/sharness/t0003-docker-migrate.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Whyrusleeping\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test docker image migration\"\n\n. lib/test-lib.sh\n\n# if in travis CI on OSX, docker is not available\nif ! test_have_prereq DOCKER; then\n skip_all='skipping '$test_description', docker not available'\n\n test_done\nfi\n\nif ! test_have_prereq SOCAT; then\n skip_all=\"skipping '$test_description': socat is not available\"\n test_done\nfi\n\nTEST_TRASH_DIR=$(pwd)\nTEST_SCRIPTS_DIR=$(dirname \"$TEST_TRASH_DIR\")\nTEST_TESTS_DIR=$(dirname \"$TEST_SCRIPTS_DIR\")\nAPP_ROOT_DIR=$(dirname \"$TEST_TESTS_DIR\")\nIMAGE_TAG=kubo_migrate\n\ntest_expect_success \"docker image build succeeds\" '\n docker_build \"$IMAGE_TAG\" \"$TEST_TESTS_DIR/../Dockerfile\" \"$APP_ROOT_DIR\"\n'\n\ntest_init_ipfs\n\ntest_expect_success \"configure migration sources\" '\n ipfs config --json Migration.DownloadSources \"[\\\"http://127.0.0.1:17233\\\"]\"\n'\n\ntest_expect_success \"setup http response\" '\n mkdir migration &&\n echo \"v1.1.1\" > migration/versions &&\n mkdir -p migration/fs-repo-6-to-7 &&\n echo \"v1.1.1\" > migration/fs-repo-6-to-7/versions &&\n CID=$(ipfs add -r -Q migration) &&\n echo \"HTTP/1.1 200 OK\" > vers_resp &&\n echo \"Content-Type: application/vnd.ipld.car\" >> vers_resp &&\n echo \"\" >> vers_resp &&\n ipfs dag export $CID >> vers_resp\n'\n\ntest_expect_success \"make repo be version 4\" '\n echo 4 > \"$IPFS_PATH/version\"\n'\n\ntest_expect_success \"startup fake dists server\" '\n ( socat tcp-listen:17233,fork,bind=127.0.0.1,reuseaddr \"SYSTEM:cat vers_resp\"!!STDERR 2> dist_serv_out ) &\n echo $! > netcat_pid\n'\n\ntest_expect_success \"docker image runs\" '\n DOC_ID=$(docker run -d -v \"$IPFS_PATH\":/data/ipfs -e IPFS_DIST_PATH=/ipfs/$CID --net=host \"$IMAGE_TAG\")\n'\n\ntest_expect_success \"docker container tries to pull migrations from netcat\" '\n sleep 4 &&\n cat dist_serv_out\n'\n\ntest_expect_success \"see logs\" '\n docker logs $DOC_ID\n'\n\ntest_expect_success \"stop docker container\" '\n docker_stop \"$DOC_ID\"\n'\n\ntest_expect_success \"kill the net cat\" '\n kill $(cat netcat_pid) || true\n'\n\ntest_expect_success \"correct version was requested\" '\n grep \"/fs-repo-6-to-7/v1.1.1/fs-repo-6-to-7_v1.1.1_linux-amd64.tar.gz\" dist_serv_out > /dev/null\n'\n\ndocker_rm \"$DOC_ID\"\ndocker_rmi \"$IMAGE_TAG\"\ntest_done\n" }, { "path": "test/sharness/t0012-completion-fish.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test generated fish completions\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"'ipfs commands completion fish' succeeds\" '\n ipfs commands completion fish > completions.fish\n'\n\ntest_expect_success \"generated completions completes 'ipfs version'\" '\n fish -c \"source completions.fish && complete -C \\\"ipfs ver\\\" | grep -q \\\"version.Show IPFS version information.\\\" \"\n'\n\ntest_done\n\n" }, { "path": "test/sharness/t0015-basic-sh-functions.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test some basic shell functions\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"shellquote works with simple stuff\" '\n var=$(shellquote one two)\n'\n\ntest_expect_success \"shellquote output looks good\" '\n test \"$var\" = \"'\\''one'\\'' '\\''two'\\''\" ||\n test_fsh echo \"var is \\\"$var\\\" instead of \\\"'\\''one'\\'' '\\''two'\\''\\\"\"\n'\n\n# The following two printf statements are equivalent:\n# printf \"%s\\n\" \\''\"foo\\\n# bar'\n# printf \"\\047\\042\\146\\157\\157\\134\\012\\142\\141\\162\\012\"\n# We use the second one to simplify quoting.\n\ntest_expect_success \"shellquote works with complex printf\" '\n eval \"$(shellquote printf \"\\047\\042\\146\\157\\157\\134\\012\\142\\141\\162\\012\")\" >actual\n'\n\ntest_expect_success \"shellquote output looks good\" '\n printf \"\\047\\042\\146\\157\\157\\134\\012\\142\\141\\162\\012\" >expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"shellquote works with many different bytes\" '\n bytes_sans_NUL=$(\n printf \"\\001\\002\\003\\004\\005\\006\\007\\010\\011\\013\\014\\015\\016\\017\\020\\021\\022\\023\\024\\025\\026\\027\\030\\031\\032\\033\\034\\035\\036\\037\\040\\041\\042\\043\\044%%\\046\\047\\050\\051\\052\\053\\054\\055\\056\\057\\060\\061\\062\\063\\064\\065\\066\\067\\070\\071\\072\\073\\074\\075\\076\\077\\100\\101\\102\\103\\104\\105\\106\\107\\110\\111\\112\\113\\114\\115\\116\\117\\120\\121\\122\\123\\124\\125\\126\\127\\130\\131\\132\\133\\134\\135\\136\\137\\140\\141\\142\\143\\144\\145\\146\\147\\150\\151\\152\\153\\154\\155\\156\\157\\160\\161\\162\\163\\164\\165\\166\\167\\170\\171\\172\\173\\174\\175\\176\\177\\200\\201\\202\\203\\204\\205\\206\\207\\210\\211\\212\\213\\214\\215\\216\\217\\220\\221\\222\\223\\224\\225\\226\\227\\230\\231\\232\\233\\234\\235\\236\\237\\240\\241\\242\\243\\244\\245\\246\\247\\250\\251\\252\\253\\254\\255\\256\\257\\260\\261\\262\\263\\264\\265\\266\\267\\270\\271\\272\\273\\274\\275\\276\\277\\300\\301\\302\\303\\304\\305\\306\\307\\310\\311\\312\\313\\314\\315\\316\\317\\320\\321\\322\\323\\324\\325\\326\\327\\330\\331\\332\\333\\334\\335\\336\\337\\340\\341\\342\\343\\344\\345\\346\\347\\350\\351\\352\\353\\354\\355\\356\\357\\360\\361\\362\\363\\364\\365\\366\\367\\370\\371\\372\\373\\374\\375\\376\\377\"\n ) &&\n eval \"$(shellquote printf \"%s\" \"$bytes_sans_NUL\")\" >actual\n'\n\ntest_expect_success \"shellquote output looks good\" '\n printf \"%s\" \"$bytes_sans_NUL\" >expected &&\n test_cmp expected actual\n'\n\ntest_done\n" }, { "path": "test/sharness/t0018-indent.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test sharness test indent\"\n\n. lib/test-lib.sh\n\nfor file in $(find .. -name 't*.sh' -type f); do\n if [ \"$(basename \"$file\")\" = \"t0290-cid.sh\" ]; then\n continue\n fi\n test_expect_success \"indent in $file is not using tabs\" '\n test_must_fail grep -P \"^ *\\t\" $file\n '\ndone\n\ntest_done\n" }, { "path": "test/sharness/t0021-config.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test config command\"\n\n. lib/test-lib.sh\n\n# we use a function so that we can run it both offline + online\ntest_config_cmd_set() {\n\n # flags (like --bool in \"ipfs config --bool\")\n cfg_flags=\"\" # unset in case.\n test \"$#\" = 3 && { cfg_flags=$1; shift; }\n\n cfg_key=$1\n cfg_val=$2\n test_expect_success \"ipfs config succeeds\" \"\n ipfs config $cfg_flags \\\"$cfg_key\\\" \\\"$cfg_val\\\"\n \"\n\n test_expect_success \"ipfs config output looks good\" \"\n echo \\\"$cfg_val\\\" >expected &&\n if [$cfg_flags != \\\"--json\\\"]; then\n ipfs config \\\"$cfg_key\\\" >actual &&\n test_cmp expected actual\n else \n ipfs config \\\"$cfg_key\\\" | tr -d \\\"\\\\n\\\\t \\\" >actual &&\n echo >>actual &&\n test_cmp expected actual\n fi\n \"\n}\n\ntest_profile_apply_revert() {\n profile=$1\n inverse_profile=$2\n\n test_expect_success \"save expected config\" '\n ipfs config show >expected\n '\n\n test_expect_success \"'ipfs config profile apply ${profile}' works\" '\n ipfs config profile apply '${profile}'\n '\n\n test_expect_success \"profile ${profile} changed something\" '\n ipfs config show >actual &&\n test_must_fail test_cmp expected actual\n '\n\n test_expect_success \"'ipfs config profile apply ${inverse_profile}' works\" '\n ipfs config profile apply '${inverse_profile}'\n '\n\n test_expect_success \"config is back to previous state after ${inverse_profile} was applied\" '\n ipfs config show >actual &&\n test_cmp expected actual\n '\n}\n\ntest_profile_apply_dry_run_not_alter() {\n profile=$1\n\n test_expect_success \"'ipfs config profile apply ${profile} --dry-run' doesn't alter config\" '\n cat \"$IPFS_PATH/config\" >expected &&\n ipfs config profile apply '${profile}' --dry-run &&\n cat \"$IPFS_PATH/config\" >actual &&\n test_cmp expected actual\n '\n}\n\ntest_config_cmd() {\n test_config_cmd_set \"Addresses.API\" \"foo\"\n test_config_cmd_set \"Addresses.Gateway\" \"bar\"\n test_config_cmd_set \"Datastore.GCPeriod\" \"baz\"\n test_config_cmd_set \"AutoNAT.ServiceMode\" \"enabled\"\n test_config_cmd_set \"--bool\" \"Discovery.MDNS.Enabled\" \"true\"\n test_config_cmd_set \"--bool\" \"Discovery.MDNS.Enabled\" \"false\"\n test_config_cmd_set \"--json\" \"Datastore.HashOnRead\" \"true\"\n test_config_cmd_set \"--json\" \"Datastore.HashOnRead\" \"false\"\n test_config_cmd_set \"--json\" \"Experimental.FilestoreEnabled\" \"true\"\n test_config_cmd_set \"--json\" \"Import.BatchMaxSize\" \"null\"\n test_config_cmd_set \"--json\" \"Import.UnixFSRawLeaves\" \"true\"\n test_config_cmd_set \"--json\" \"Routing.Routers.Test\" \"{\\\\\\\"Parameters\\\\\\\":\\\\\\\"Test\\\\\\\",\\\\\\\"Type\\\\\\\":\\\\\\\"Test\\\\\\\"}\"\n test_config_cmd_set \"--json\" \"Experimental.OptimisticProvideJobsPoolSize\" \"1337\"\n test_config_cmd_set \"--json\" \"Addresses.Swarm\" \"[\\\\\\\"test\\\\\\\",\\\\\\\"test\\\\\\\",\\\\\\\"test\\\\\\\"]\"\n test_config_cmd_set \"--json\" \"Gateway.PublicGateways.Foo\" \"{\\\\\\\"DeserializedResponses\\\\\\\":true,\\\\\\\"InlineDNSLink\\\\\\\":false,\\\\\\\"NoDNSLink\\\\\\\":false,\\\\\\\"Paths\\\\\\\":[\\\\\\\"Bar\\\\\\\",\\\\\\\"Baz\\\\\\\"],\\\\\\\"UseSubdomains\\\\\\\":true}\"\n test_config_cmd_set \"--bool\" \"Gateway.PublicGateways.Foo.UseSubdomains\" \"false\"\n\n test_expect_success \"'ipfs config show' works\" '\n ipfs config show >actual\n '\n\n test_expect_success \"'ipfs config show' output looks good\" '\n grep \"\\\"API\\\": \\\"foo\\\",\" actual &&\n grep \"\\\"Gateway\\\": \\\"bar\\\"\" actual &&\n grep \"\\\"Enabled\\\": false\" actual &&\n grep \"\\\"HashOnRead\\\": false\" actual\n '\n\n test_expect_success \"'ipfs config show --config-file' works\" '\n mv \"$IPFS_PATH/config\" \"$IPFS_PATH/config-moved\" &&\n ipfs config --config-file \"$IPFS_PATH/config-moved\" show >moved &&\n test_cmp moved actual &&\n mv \"$IPFS_PATH/config-moved\" \"$IPFS_PATH/config\"\n '\n\n test_expect_success \"setup for config replace test\" '\n cp \"$IPFS_PATH/config\" newconfig.json &&\n sed -i\"~\" -e /PrivKey/d -e s/10GB/11GB/ newconfig.json &&\n sed -i\"~\" -e '\"'\"'/PeerID/ {'\"'\"' -e '\"'\"' s/,$// '\"'\"' -e '\"'\"' } '\"'\"' newconfig.json\n '\n\n test_expect_success \"run 'ipfs config replace'\" '\n ipfs config replace - < newconfig.json\n '\n\n test_expect_success \"check resulting config after 'ipfs config replace'\" '\n sed -e /PrivKey/d \"$IPFS_PATH/config\" > replconfig.json &&\n sed -i\"~\" -e '\"'\"'/PeerID/ {'\"'\"' -e '\"'\"' s/,$// '\"'\"' -e '\"'\"' } '\"'\"' replconfig.json &&\n test_cmp replconfig.json newconfig.json\n '\n\n # SECURITY\n # Those tests are here to prevent exposing the PrivKey on the network\n\n test_expect_success \"'ipfs config Identity' fails\" '\n test_expect_code 1 ipfs config Identity 2> ident_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"Error: cannot show or change private key through API\" > ident_exp &&\n test_cmp ident_exp ident_out\n '\n\n test_expect_success \"'ipfs config Identity.PrivKey' fails\" '\n test_expect_code 1 ipfs config Identity.PrivKey 2> ident_out\n '\n\n test_expect_success \"output looks good\" '\n test_cmp ident_exp ident_out\n '\n\n test_expect_success \"lower cased PrivKey\" '\n sed -i\"~\" -e '\\''s/PrivKey/privkey/'\\'' \"$IPFS_PATH/config\" &&\n test_expect_code 1 ipfs config Identity.privkey 2> ident_out\n '\n\n test_expect_success \"output looks good\" '\n test_cmp ident_exp ident_out\n '\n\n test_expect_success \"fix it back\" '\n sed -i\"~\" -e '\\''s/privkey/PrivKey/'\\'' \"$IPFS_PATH/config\"\n '\n\n test_expect_success \"'ipfs config show' doesn't include privkey\" '\n ipfs config show > show_config &&\n test_expect_code 1 grep PrivKey show_config\n '\n\n test_expect_success \"'ipfs config replace' injects privkey back\" '\n ipfs config replace show_config &&\n grep \"\\\"PrivKey\\\":\" \"$IPFS_PATH/config\" | grep -e \": \\\".\\+\\\"\" >/dev/null\n '\n\n test_expect_success \"'ipfs config replace' with privkey errors out\" '\n cp \"$IPFS_PATH/config\" real_config &&\n test_expect_code 1 ipfs config replace - < real_config 2> replace_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"Error: setting private key with API is not supported\" > replace_expected\n test_cmp replace_out replace_expected\n '\n\n test_expect_success \"'ipfs config replace' with lower case privkey errors out\" '\n cp \"$IPFS_PATH/config\" real_config &&\n sed -i -e '\\''s/PrivKey/privkey/'\\'' real_config &&\n test_expect_code 1 ipfs config replace - < real_config 2> replace_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"Error: setting private key with API is not supported\" > replace_expected\n test_cmp replace_out replace_expected\n '\n\n test_expect_success \"'ipfs config Swarm.AddrFilters' looks good\" '\n ipfs config Swarm.AddrFilters > actual_config &&\n test $(cat actual_config | wc -l) = 1\n '\n\n test_expect_success \"copy ipfs config\" '\n cp \"$IPFS_PATH/config\" before_patch\n '\n\n test_expect_success \"'ipfs config profile apply server' works\" '\n ipfs config profile apply server\n '\n\n test_expect_success \"backup was created and looks good\" '\n test_cmp \"$(find \"$IPFS_PATH\" -name \"config-*\")\" before_patch\n '\n\n test_expect_success \"'ipfs config Swarm.AddrFilters' looks good with server profile\" '\n ipfs config Swarm.AddrFilters > actual_config &&\n test $(cat actual_config | wc -l) = 18\n '\n\n test_expect_success \"'ipfs config profile apply local-discovery' works\" '\n ipfs config profile apply local-discovery\n '\n\n test_expect_success \"'ipfs config Swarm.AddrFilters' looks good with applied local-discovery profile\" '\n ipfs config Swarm.AddrFilters > actual_config &&\n test $(cat actual_config | wc -l) = 1\n '\n\n test_profile_apply_revert server local-discovery\n\n # tests above mess with values this profile changes, need to do that before testing test profile\n test_expect_success \"ensure test profile is applied fully\" '\n ipfs config profile apply test\n '\n\n # need to do this in reverse as the test profile is already applied in sharness\n test_profile_apply_revert default-networking test\n\n test_profile_apply_dry_run_not_alter server\n\n test_profile_apply_dry_run_not_alter local-discovery\n\n test_profile_apply_dry_run_not_alter test\n\n test_expect_success \"'ipfs config profile apply local-discovery --dry-run' looks good with different profile info\" '\n ipfs config profile apply local-discovery --dry-run > diff_info &&\n test `grep \"DisableNatPortMap\" diff_info | wc -l` = 2\n '\n\n test_expect_success \"'ipfs config profile apply server --dry-run' looks good with same profile info\" '\n ipfs config profile apply server --dry-run > diff_info &&\n test `grep \"DisableNatPortMap\" diff_info | wc -l` = 1\n '\n\n test_expect_success \"'ipfs config profile apply server' looks good with same profile info\" '\n ipfs config profile apply server > diff_info &&\n test `grep \"DisableNatPortMap\" diff_info | wc -l` = 1\n '\n\n test_expect_success \"'ipfs config profile apply local-discovery' looks good with different profile info\" '\n ipfs config profile apply local-discovery > diff_info &&\n test `grep \"DisableNatPortMap\" diff_info | wc -l` = 2\n '\n\n test_expect_success \"'ipfs config profile apply test' looks good with different profile info\" '\n ipfs config profile apply test > diff_info &&\n test `grep \"DisableNatPortMap\" diff_info | wc -l` = 2\n '\n\n test_expect_success \"'ipfs config profile apply test --dry-run' doesn't include privkey\" '\n ipfs config profile apply test --dry-run > show_config &&\n test_expect_code 1 grep PrivKey show_config\n '\n\n test_expect_success \"'ipfs config profile apply test' doesn't include privkey\" '\n ipfs config profile apply test > show_config &&\n test_expect_code 1 grep PrivKey show_config\n '\n\n # won't work as it changes datastore definition, which makes ipfs not launch\n # without converting first\n # test_profile_apply_revert pebbleds\n\n test_expect_success \"cleanup config backups\" '\n find \"$IPFS_PATH\" -name \"config-*\" -exec rm {} \\;\n '\n}\n\ntest_init_ipfs\n\n# should work offline\ntest_config_cmd\n\n# should work online\ntest_launch_ipfs_daemon\ntest_config_cmd\ntest_kill_ipfs_daemon\n\n\ntest_done\n" }, { "path": "test/sharness/t0022-init-default.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test init command with default config\"\n\n. lib/test-lib.sh\n\ncfg_key=\"Addresses.API\"\ncfg_val=\"/ip4/0.0.0.0/tcp/5001\"\n\n# test that init succeeds\ntest_expect_success \"ipfs init succeeds\" '\n export IPFS_PATH=\"$(pwd)/.ipfs\" &&\n echo \"IPFS_PATH: \\\"$IPFS_PATH\\\"\" &&\n BITS=\"2048\" &&\n ipfs init >actual_init ||\n test_fsh cat actual_init\n'\n\ntest_expect_success \".ipfs/config has been created\" '\n test -f \"$IPFS_PATH\"/config ||\n test_fsh ls -al .ipfs\n'\n\ntest_expect_success \"ipfs config succeeds\" '\n ipfs config $cfg_flags \"$cfg_key\" \"$cfg_val\"\n'\n\ntest_expect_success \"ipfs read config succeeds\" '\n IPFS_DEFAULT_CONFIG=$(cat \"$IPFS_PATH\"/config)\n'\n\ntest_expect_success \"clean up ipfs dir\" '\n rm -rf \"$IPFS_PATH\"\n'\n\ntest_expect_success \"ipfs init default config succeeds\" '\n echo $IPFS_DEFAULT_CONFIG | ipfs init - >actual_init ||\n test_fsh cat actual_init\n'\n\ntest_expect_success \"ipfs config output looks good\" '\n echo \"$cfg_val\" >expected &&\n ipfs config \"$cfg_key\" >actual &&\n test_cmp expected actual\n'\n\ntest_done\n" }, { "path": "test/sharness/t0023-shutdown.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test shutdown command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"shutdown succeeds\" '\n ipfs shutdown\n'\n\ntest_expect_success \"daemon no longer running\" '\n for i in $(test_seq 1 100)\n do\n go-sleep 100ms\n ! kill -0 $IPFS_PID 2>/dev/null && return\n done\n'\n\ntest_launch_ipfs_daemon_without_network\n\ntest_expect_success \"shutdown succeeds\" '\n ipfs shutdown\n'\n\ntest_expect_success \"daemon no longer running\" '\n for i in $(test_seq 1 100)\n do\n go-sleep 100ms\n ! kill -0 $IPFS_PID 2>/dev/null && return\n done\n'\ntest_done\n" }, { "path": "test/sharness/t0024-datastore-config.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test datastore config\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_launch_ipfs_daemon\ntest_kill_ipfs_daemon\n\nSPEC_NOSYNC=$(cat ../t0024-files/spec-nosync)\n\nSPEC_NEWSHARDFUN=$(cat ../t0024-files/spec-newshardfun)\n\ntest_expect_success \"change runtime value in spec config\" '\n ipfs config --json Datastore.Spec \"$SPEC_NOSYNC\"\n'\n\ntest_launch_ipfs_daemon\ntest_kill_ipfs_daemon\n\ntest_expect_success \"change on-disk value in spec config\" '\n ipfs config --json Datastore.Spec \"$SPEC_NEWSHARDFUN\"\n'\n\ntest_expect_success \"can not launch daemon after on-disk value change\" '\n test_must_fail ipfs daemon\n'\n\ntest_done\n" }, { "path": "test/sharness/t0024-files/spec-newshardfun", "content": "{\n \"mounts\": [\n {\n \"child\": {\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/3\",\n \"sync\": true,\n \"type\": \"flatfs\"\n },\n \"mountpoint\": \"/blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"type\": \"measure\"\n },\n {\n \"child\": {\n \"compression\": \"none\",\n \"path\": \"datastore\",\n \"type\": \"levelds\"\n },\n \"mountpoint\": \"/\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"measure\"\n }\n ],\n \"type\": \"mount\"\n}\n" }, { "path": "test/sharness/t0024-files/spec-nosync", "content": "{\n \"mounts\": [\n {\n \"child\": {\n \"path\": \"blocks\",\n \"shardFunc\": \"/repo/flatfs/shard/v1/next-to-last/2\",\n \"sync\": false,\n \"type\": \"flatfs\"\n },\n \"mountpoint\": \"/blocks\",\n \"prefix\": \"flatfs.datastore\",\n \"type\": \"measure\"\n },\n {\n \"child\": {\n \"compression\": \"none\",\n \"path\": \"datastore\",\n \"type\": \"levelds\"\n },\n \"mountpoint\": \"/\",\n \"prefix\": \"leveldb.datastore\",\n \"type\": \"measure\"\n }\n ],\n \"type\": \"mount\"\n}\n" }, { "path": "test/sharness/t0025-datastores.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test non-standard datastores\"\n\n. lib/test-lib.sh\n\nprofiles=(\"flatfs\" \"pebbleds\" \"badgerds\")\nproot=\"$(mktemp -d \"${TMPDIR:-/tmp}/t0025.XXXXXX\")\"\n\nfor profile in \"${profiles[@]}\"; do\n test_expect_success \"'ipfs init --empty-repo=false --profile=$profile' succeeds\" '\n BITS=\"2048\" &&\n IPFS_PATH=\"$proot/$profile\" &&\n ipfs init --empty-repo=false --profile=$profile\n '\n test_expect_success \"'ipfs pin add' and 'pin ls' works with $profile\" '\n export IPFS_PATH=\"$proot/$profile\" &&\n echo -n \"hello_$profile\" | ipfs block put --pin=true > hello_cid &&\n ipfs pin ls -t recursive \"$(cat hello_cid)\"\n '\ndone\n\ntest_done\n" }, { "path": "test/sharness/t0026-id.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test to make sure our identity information looks sane\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_id_compute_agent() {\n local AGENT_SUFFIX\n AGENT_SUFFIX=$1\n AGENT_VERSION=\"$(ipfs version --number)\" || return 1\n AGENT_COMMIT=\"$(ipfs version --number --commit)\" || return 1\n if test \"$AGENT_COMMIT\" = \"$AGENT_VERSION\"; then\n AGENT_COMMIT=\"\"\n else\n AGENT_COMMIT=\"${AGENT_COMMIT##$AGENT_VERSION-}\"\n fi\n AGENT_VERSION=\"kubo/$AGENT_VERSION\"\n if test -n \"$AGENT_COMMIT\"; then\n AGENT_VERSION=\"$AGENT_VERSION/$AGENT_COMMIT\"\n fi\n if test -n \"$AGENT_SUFFIX\"; then\n AGENT_VERSION=\"$AGENT_VERSION/$AGENT_SUFFIX\"\n fi\n echo \"$AGENT_VERSION\"\n}\n\ntest_expect_success \"checking AgentVersion\" '\n test_id_compute_agent > expected-agent-version &&\n ipfs id -f \"\\n\" > actual-agent-version &&\n test_cmp expected-agent-version actual-agent-version\n'\n\ntest_expect_success \"checking ID of self\" '\n ipfs config Identity.PeerID > expected-id &&\n ipfs id -f \"\\n\" > actual-id &&\n test_cmp expected-id actual-id\n'\n\ntest_expect_success \"checking and converting ID of a random peer while offline\" '\n # Peer ID taken from `t0140-swarm.sh` test.\n echo k2k4r8ncs1yoluq95unsd7x2vfhgve0ncjoggwqx9vyh3vl8warrcp15 > expected-id &&\n ipfs id -f \"\\n\" --peerid-base base36 --offline QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N > actual-id &&\n test_cmp expected-id actual-id\n'\n\n# agent-version-suffix (local, offline)\ntest_launch_ipfs_daemon --agent-version-suffix=test-suffix\ntest_expect_success \"checking AgentVersion with suffix (local)\" '\n test_id_compute_agent test-suffix > expected-agent-version &&\n ipfs id -f \"\\n\" > actual-agent-version &&\n test_cmp expected-agent-version actual-agent-version\n'\n\n# agent-version-suffix (over libp2p identify protocol)\niptb testbed create -type localipfs -count 2 -init\nstartup_cluster 2 --agent-version-suffix=test-suffix-identify\ntest_expect_success \"checking AgentVersion with suffix (fetched via libp2p identify protocol)\" '\n ipfsi 0 id -f \"\\n\" > expected-identify-agent-version &&\n ipfsi 1 id \"$(ipfsi 0 config Identity.PeerID)\" -f \"\\n\" > actual-libp2p-identify-agent-version &&\n test_cmp expected-identify-agent-version actual-libp2p-identify-agent-version\n'\niptb stop\n\ntest_kill_ipfs_daemon\n\n# Version.AgentSuffix overrides --agent-version-suffix (local, offline)\ntest_expect_success \"setting Version.AgentSuffix in config\" '\n ipfs config Version.AgentSuffix json-config-suffix\n'\ntest_launch_ipfs_daemon --agent-version-suffix=ignored-cli-suffix\ntest_expect_success \"checking AgentVersion with suffix set via JSON config\" '\n test_id_compute_agent json-config-suffix > expected-agent-version &&\n ipfs id -f \"\\n\" > actual-agent-version &&\n test_cmp expected-agent-version actual-agent-version\n'\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0027-rotate.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test rotate command\"\n\n. lib/test-lib.sh\n\ntest_rotate() {\n FROM_ALG=$1\n TO_ALG=$2\n\n test_expect_success \"ipfs init (from $FROM_ALG, to $TO_ALG)\" '\n export IPFS_PATH=\"$(pwd)/.ipfs\" &&\n case $FROM_ALG in\n rsa)\n ipfs init --profile=test -a=rsa > /dev/null\n ;;\n ed25519)\n ipfs init --profile=test -a=ed25519 > /dev/null\n ;;\n *)\n ipfs init --profile=test > /dev/null\n ;;\n esac\n '\n\n test_expect_success \"Save first ID and key\" '\n ipfs id -f=\"\" > first_id &&\n ipfs id -f=\"\" > first_key\n '\n\n test_launch_ipfs_daemon\n\n test_kill_ipfs_daemon\n\n test_expect_success \"rotating keys\" '\n case $TO_ALG in\n rsa)\n ipfs key rotate -t=rsa -s=2048 --oldkey=oldkey\n ;;\n ed25519)\n ipfs key rotate -t=ed25519 --oldkey=oldkey\n ;;\n *)\n ipfs key rotate --oldkey=oldkey\n ;;\n esac\n '\n\n test_expect_success \"'ipfs key rotate -o self' should fail\" '\n echo \"Error: keystore name for back up cannot be named '\\''self'\\''\" >expected-self\n test_must_fail ipfs key rotate -o self 2>actual-self &&\n test_cmp expected-self actual-self\n '\n\n test_expect_success \"Compare second ID and key to first\" '\n ipfs id -f=\"\" > second_id &&\n ipfs id -f=\"\" > second_key &&\n ! test_cmp first_id second_id &&\n ! test_cmp first_key second_key\n '\n\n test_expect_success \"checking ID\" '\n ipfs config Identity.PeerID > expected-id &&\n ipfs id -f \"\\n\" > actual-id &&\n ipfs key list -l --ipns-base=b58mh | grep self | cut -d \" \" -f1 > keystore-id &&\n ipfs key list -l --ipns-base=b58mh | grep oldkey | cut -d \" \" -f1 | tr -d \"\\n\" > old-keystore-id &&\n test_cmp expected-id actual-id &&\n test_cmp expected-id keystore-id &&\n test_cmp old-keystore-id first_id\n '\n\n test_launch_ipfs_daemon\n\n test_expect_success \"publish name with new and old keys\" '\n echo \"hello world\" > msg &&\n ipfs add msg | cut -d \" \" -f2 | tr -d \"\\n\" > msg_hash &&\n ipfs name publish --offline --allow-offline --key=self $(cat msg_hash) &&\n ipfs name publish --offline --allow-offline --key=oldkey $(cat msg_hash)\n '\n\n test_kill_ipfs_daemon\n\n test_expect_success \"clean up ipfs dir\" '\n rm -rf \"$IPFS_PATH\"\n '\n\n}\ntest_rotate 'rsa' ''\ntest_rotate 'ed25519' ''\ntest_rotate '' ''\ntest_rotate 'rsa' 'rsa'\ntest_rotate 'ed25519' 'rsa'\ntest_rotate '' 'rsa'\ntest_rotate 'rsa' 'ed25519'\ntest_rotate 'ed25519' 'ed25519'\ntest_rotate '' 'ed25519'\n\ntest_done\n" }, { "path": "test/sharness/t0030-mount.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test mount command\"\n\n. lib/test-lib.sh\n\n# if in travis CI, don't test mount (no fuse)\nif ! test_have_prereq FUSE; then\n skip_all='skipping mount tests, fuse not available'\n\n test_done\nfi\n\n\n# echo -n \"ipfs\" > expected && ipfs add --cid-version 1 -Q -w expected\nexport IPFS_NS_MAP=\"welcome.example.com:/ipfs/bafybeicq7bvn5lz42qlmghaoiwrve74pzi53auqetbantp5kajucsabike\"\n\n# start iptb + wait for peering\nNUM_NODES=5\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count $NUM_NODES -init\n'\nstartup_cluster $NUM_NODES\n\n# test mount failure before mounting properly.\ntest_expect_success \"'ipfs mount' fails when there is no mount dir\" '\n tmp_ipfs_mount() { ipfsi 0 mount -f=not_ipfs -n=not_ipns -m=not_mfs >output 2>output.err; } &&\n test_must_fail tmp_ipfs_mount\n'\n\ntest_expect_success \"'ipfs mount' output looks good\" '\n test_must_be_empty output &&\n test_should_contain \"not_ipns\\|not_ipfs\\|not_mfs\" output.err\n'\n\ntest_expect_success \"setup and publish default IPNS value\" '\n mkdir \"$(pwd)/ipfs\" \"$(pwd)/ipns\" \"$(pwd)/mfs\" &&\n ipfsi 0 name publish QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\n'\n\n# make sure stuff is unmounted first\n# then mount properly\ntest_expect_success FUSE \"'ipfs mount' succeeds\" '\n do_umount \"$(pwd)/ipfs\" || true &&\n do_umount \"$(pwd)/ipns\" || true &&\n do_umount \"$(pwd)/mfs\" || true &&\n ipfsi 0 mount -f \"$(pwd)/ipfs\" -n \"$(pwd)/ipns\" -m \"$(pwd)/mfs\" >actual\n'\n\ntest_expect_success FUSE \"'ipfs mount' output looks good\" '\n echo \"IPFS mounted at: $(pwd)/ipfs\" >expected &&\n echo \"IPNS mounted at: $(pwd)/ipns\" >>expected &&\n echo \"MFS mounted at: $(pwd)/mfs\" >>expected &&\n test_cmp expected actual\n'\n\ntest_expect_success FUSE \"local symlink works\" '\n ipfsi 0 id -f\"\\n\" > expected &&\n basename $(readlink ipns/local) > actual &&\n test_cmp expected actual\n'\n\ntest_expect_success FUSE \"can resolve ipns names\" '\n echo -n \"ipfs\" > expected &&\n ipfsi 0 add --cid-version 1 -Q -w expected &&\n cat ipns/welcome.example.com/expected > actual &&\n test_cmp expected actual\n'\n\ntest_expect_success FUSE \"create mfs file via fuse\" '\n touch mfs/testfile &&\n ipfsi 0 files ls | grep testfile\n'\n\ntest_expect_success FUSE \"create mfs dir via fuse\" '\n mkdir mfs/testdir &&\n ipfsi 0 files ls | grep testdir\n'\n\ntest_expect_success FUSE \"read mfs file from fuse\" '\n echo content > mfs/testfile &&\n getfattr -n ipfs_cid mfs/testfile\n'\ntest_expect_success FUSE \"ipfs add file and read it back via fuse\" '\n echo content3 | ipfsi 0 files write -e /testfile3 &&\n grep content3 mfs/testfile3\n'\n\ntest_expect_success FUSE \"ipfs add file and read it back via fuse\" '\n echo content > testfile2 &&\n ipfsi 0 add --to-files /testfile2 testfile2 &&\n grep content mfs/testfile2\n'\n\ntest_expect_success FUSE \"test file xattr\" '\n echo content > mfs/testfile &&\n getfattr -n ipfs_cid mfs/testfile\n'\n\ntest_expect_success FUSE \"test file removal\" '\n touch mfs/testfile &&\n rm mfs/testfile\n'\n\ntest_expect_success FUSE \"test nested dirs\" '\n mkdir -p mfs/foo/bar/baz/qux &&\n echo content > mfs/foo/bar/baz/qux/quux &&\n ipfsi 0 files stat /foo/bar/baz/qux/quux\n'\n\ntest_expect_success \"mount directories cannot be removed while active\" '\n test_must_fail rmdir ipfs ipns mfs 2>/dev/null\n'\n\ntest_expect_success \"unmount directories\" '\n do_umount \"$(pwd)/ipfs\" &&\n do_umount \"$(pwd)/ipns\" &&\n do_umount \"$(pwd)/mfs\"\n'\n\ntest_expect_success \"mount directories can be removed after shutdown\" '\n rmdir ipfs ipns mfs\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0031-mount-publish.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test mount command in conjunction with publishing\"\n\n# imports\n. lib/test-lib.sh\n\n# if in travis CI, don't test mount (no fuse)\nif ! test_have_prereq FUSE; then\n skip_all='skipping mount tests, fuse not available'\n\n test_done\nfi\n\ntest_init_ipfs\n\n# start iptb + wait for peering\nNUM_NODES=3\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count $NUM_NODES -force -init &&\n startup_cluster $NUM_NODES\n'\n\n# pre-mount publish\nHASH=$(echo 'hello warld' | ipfsi 0 add -Q -w --stdin-name \"file\")\ntest_expect_success \"can publish before mounting /ipns\" '\n ipfsi 0 name publish \"$HASH\"\n'\n\n# mount\nIPFS_MOUNT_DIR=\"$PWD/ipfs\"\nIPNS_MOUNT_DIR=\"$PWD/ipns\"\ntest_expect_success FUSE \"'ipfs mount' succeeds\" '\n ipfsi 0 mount -f \"'\"$IPFS_MOUNT_DIR\"'\" -n \"'\"$IPNS_MOUNT_DIR\"'\" >actual\n'\ntest_expect_success FUSE \"'ipfs mount' output looks good\" '\n echo \"IPFS mounted at: $PWD/ipfs\" >expected &&\n echo \"IPNS mounted at: $PWD/ipns\" >>expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"cannot publish after mounting /ipns\" '\n echo \"Error: cannot manually publish while IPNS is mounted\" >expected &&\n test_must_fail ipfsi 0 name publish '$HASH' 2>actual &&\n test_cmp expected actual\n'\n\ntest_expect_success \"unmount /ipns out-of-band\" '\n fusermount -u \"'\"$IPNS_MOUNT_DIR\"'\"\n'\n\ntest_expect_success \"can publish after unmounting /ipns\" '\n ipfsi 0 name publish '$HASH'\n'\n\n# clean-up ipfs\ntest_expect_success \"unmount /ipfs\" '\n fusermount -u \"'\"$IPFS_MOUNT_DIR\"'\"\n'\niptb stop\n\ntest_done\n" }, { "path": "test/sharness/t0032-mount-sharded.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2021 Protocol Labs\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test mount command with sharding enabled\"\n\n. lib/test-lib.sh\n\nif ! test_have_prereq FUSE; then\n skip_all='skipping mount sharded tests, fuse not available'\n test_done\nfi\n\ntest_init_ipfs\n\ntest_expect_success 'force sharding' '\n ipfs config --json Import.UnixFSHAMTDirectorySizeThreshold \"\\\"1B\\\"\"\n'\n\ntest_launch_ipfs_daemon\ntest_mount_ipfs\n\n# we're testing nested subdirs which ensures that IPLD ADLs work\ntest_expect_success 'setup test data' '\n mkdir testdata &&\n echo a > testdata/a &&\n mkdir testdata/subdir &&\n echo b > testdata/subdir/b\n'\n\nHASH=QmY59Ufw8zA2BxGPMTcfXg86JVed81Qbxeq5rDkHWSLN1m\n\ntest_expect_success 'can add the data' '\n echo $HASH > expected_hash &&\n ipfs add -r -Q testdata > actual_hash &&\n test_cmp expected_hash actual_hash\n'\n\ntest_expect_success 'can read the data' '\n echo a > expected_a &&\n cat \"ipfs/$HASH/a\" > actual_a &&\n test_cmp expected_a actual_a &&\n echo b > expected_b &&\n cat \"ipfs/$HASH/subdir/b\" > actual_b &&\n test_cmp expected_b actual_b\n'\n\ntest_expect_success 'can list directories' '\n printf \"a\\nsubdir\\n\" > expected_ls &&\n ls -1 \"ipfs/$HASH\" > actual_ls &&\n test_cmp expected_ls actual_ls &&\n printf \"b\\n\" > expected_ls_subdir &&\n ls -1 \"ipfs/$HASH/subdir\" > actual_ls_subdir &&\n test_cmp expected_ls_subdir actual_ls_subdir\n'\n\ntest_expect_success \"unmount\" '\n do_umount \"$(pwd)/ipfs\" &&\n do_umount \"$(pwd)/ipns\"\n'\n\ntest_expect_success 'cleanup' 'rmdir ipfs ipns'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0040-add-and-cat.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test add and cat commands\"\n\n. lib/test-lib.sh\n\ntest_add_cat_file() {\n test_expect_success \"ipfs add --help works\" '\n ipfs add --help 2> add_help_err1 > /dev/null\n '\n\n test_expect_success \"stdin reading message doesn't show up\" '\n test_expect_code 1 grep \"ipfs: Reading from\" add_help_err1 &&\n test_expect_code 1 grep \"send Ctrl-d to stop.\" add_help_err1\n '\n\n test_expect_success \"ipfs help add works\" '\n ipfs help add 2> add_help_err2 > /dev/null\n '\n\n test_expect_success \"stdin reading message doesn't show up\" '\n test_expect_code 1 grep \"ipfs: Reading from\" add_help_err2 &&\n test_expect_code 1 grep \"send Ctrl-d to stop.\" add_help_err2\n '\n\n test_expect_success \"ipfs add succeeds\" '\n echo \"Hello Worlds!\" >mountdir/hello.txt &&\n ipfs add mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add output looks good\" '\n HASH=\"QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH\" &&\n echo \"added $HASH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --only-hash succeeds\" '\n ipfs add --only-hash mountdir/hello.txt > oh_actual\n '\n\n test_expect_success \"ipfs add --only-hash output looks good\" '\n test_cmp expected oh_actual\n '\n\n test_expect_success \"ipfs cat succeeds\" '\n ipfs cat \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat output looks good\" '\n echo \"Hello Worlds!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with offset succeeds\" '\n ipfs cat --offset 10 \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat from offset output looks good\" '\n echo \"ds!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat multiple hashes with offset succeeds\" '\n ipfs cat --offset 10 \"$HASH\" \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat from offset output looks good\" '\n echo \"ds!\" >expected &&\n echo \"Hello Worlds!\" >>expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat multiple hashes with offset succeeds\" '\n ipfs cat --offset 16 \"$HASH\" \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat from offset output looks good\" '\n echo \"llo Worlds!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat from negative offset should fail\" '\n test_expect_code 1 ipfs cat --offset -102 \"$HASH\" > actual\n '\n\n test_expect_success \"ipfs cat with length succeeds\" '\n ipfs cat --length 8 \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat with length output looks good\" '\n printf \"Hello Wo\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat multiple hashes with offset and length succeeds\" '\n ipfs cat --offset 5 --length 15 \"$HASH\" \"$HASH\" \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat multiple hashes with offset and length looks good\" '\n printf \" Worlds!\\nHello \" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with exact length succeeds\" '\n ipfs cat --length $(ipfs cat \"$HASH\" | wc -c) \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat with exact length looks good\" '\n echo \"Hello Worlds!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with 0 length succeeds\" '\n ipfs cat --length 0 \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat with 0 length looks good\" '\n : >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with oversized length succeeds\" '\n ipfs cat --length 100 \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs cat with oversized length looks good\" '\n echo \"Hello Worlds!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with negative length should fail\" '\n test_expect_code 1 ipfs cat --length -102 \"$HASH\" > actual\n '\n\n test_expect_success \"ipfs cat /ipfs/file succeeds\" '\n ipfs cat /ipfs/$HASH >actual\n '\n\n test_expect_success \"output looks good\" '\n echo \"Hello Worlds!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add -t succeeds\" '\n ipfs add -t mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add -t output looks good\" '\n HASH=\"QmUkUQgxXeggyaD5Ckv8ZqfW8wHBX6cYyeiyqvVZYzq5Bi\" &&\n echo \"added $HASH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --chunker size-32 succeeds\" '\n ipfs add --chunker rabin mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add --chunker size-32 output looks good\" '\n HASH=\"QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH\" &&\n echo \"added $HASH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --chunker size-64 succeeds\" '\n ipfs add --chunker=size-64 mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add --chunker size-64 output looks good\" '\n HASH=\"QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH\" &&\n echo \"added $HASH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --chunker=size-0 failed\" '\n test_expect_code 1 ipfs add -Q --chunker=size-0 mountdir/hello.txt\n '\n\n test_expect_success \"ipfs add --chunker rabin-36-512-1024 succeeds\" '\n ipfs add --chunker rabin-36-512-1024 mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add --chunker rabin-36-512-1024 output looks good\" '\n HASH=\"QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH\" &&\n echo \"added $HASH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --chunker rabin-12-512-1024 failed\" '\n test_expect_code 1 ipfs add -Q --chunker rabin-12-512-1024 mountdir/hello.txt\n '\n\n test_expect_success \"ipfs add --chunker buzhash succeeds\" '\n ipfs add --chunker buzhash mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add --chunker buzhash output looks good\" '\n HASH=\"QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH\" &&\n echo \"added $HASH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add on hidden file succeeds\" '\n echo \"Hello Worlds!\" >mountdir/.hello.txt &&\n ipfs add mountdir/.hello.txt >actual\n '\n\n test_expect_success \"ipfs add on hidden file output looks good\" '\n HASH=\"QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH\" &&\n echo \"added $HASH .hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"add zero length file\" '\n touch zero-length-file &&\n ZEROHASH=$(ipfs add -q zero-length-file) &&\n echo $ZEROHASH\n '\n\n test_expect_success \"zero length file has correct hash\" '\n test \"$ZEROHASH\" = QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH\n '\n\n test_expect_success \"cat zero length file\" '\n ipfs cat $ZEROHASH > zero-length-file_out\n '\n\n test_expect_success \"make sure it looks good\" '\n test_cmp zero-length-file zero-length-file_out\n '\n\n test_expect_success \"ipfs add --stdin-name\" '\n NAMEHASH=\"QmdFyxZXsFiP4csgfM5uPu99AvFiKH62CSPDw5TP92nr7w\" &&\n echo \"IPFS\" | ipfs add --stdin-name file.txt > actual &&\n echo \"added $NAMEHASH file.txt\" > expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --stdin-name -w\" '\n NAMEHASH=\"QmdFyxZXsFiP4csgfM5uPu99AvFiKH62CSPDw5TP92nr7w\" &&\n echo \"IPFS\" | ipfs add -w --stdin-name file.txt | head -n1> actual &&\n echo \"added $NAMEHASH file.txt\" > expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with stdin-name\" '\n NAMEHASH=$(echo \"IPFS\" | ipfs add -w --stdin-name file.txt -Q) &&\n ipfs cat /ipfs/$NAMEHASH/file.txt > expected &&\n echo \"IPFS\" > actual &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add -r .\" '\n mkdir test_current_dir &&\n echo \"Hey\" > test_current_dir/hey &&\n mkdir test_current_dir/hello &&\n echo \"World\" > test_current_dir/hello/world &&\n ( cd test_current_dir &&\n ipfs add -r -Q . > ../actual && cd ../ ) &&\n rm -r test_current_dir\n '\n\n test_expect_success \"ipfs add -r . output looks good\" '\n echo \"QmZQWnfcqJ6hNkkPvrY9Q5X39GP3jUnUbAV4AbmbbR3Cb1\" > expected\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add -r ./\" '\n mkdir test_current_dir &&\n echo \"Hey\" > test_current_dir/hey &&\n mkdir test_current_dir/hello &&\n echo \"World\" > test_current_dir/hello/world &&\n ( cd test_current_dir &&\n ipfs add -r -Q ./ > ../actual && cd ../ ) &&\n rm -r test_current_dir\n '\n\n test_expect_success \"ipfs add -r ./ output looks good\" '\n echo \"QmZQWnfcqJ6hNkkPvrY9Q5X39GP3jUnUbAV4AbmbbR3Cb1\" > expected\n test_cmp expected actual\n '\n\n # --cid-base=base32\n\n test_expect_success \"ipfs add --cid-base=base32 succeeds\" '\n echo \"base32 test\" >mountdir/base32-test.txt &&\n ipfs add --cid-base=base32 mountdir/base32-test.txt >actual\n '\n test_expect_success \"ipfs add --cid-base=base32 output looks good\" '\n HASHb32=\"bafybeibyosqxljd2eptb4ebbtvk7pb4aoxzqa6ttdsflty6rsslz5y6i34\" &&\n echo \"added $HASHb32 base32-test.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --cid-base=base32 --only-hash succeeds\" '\n ipfs add --cid-base=base32 --only-hash mountdir/base32-test.txt > oh_actual\n '\n test_expect_success \"ipfs add --cid-base=base32 --only-hash output looks good\" '\n test_cmp expected oh_actual\n '\n\n test_expect_success \"ipfs add --cid-base=base32 --upgrade-cidv0-in-output=false succeeds\" '\n echo \"base32 test\" >mountdir/base32-test.txt &&\n ipfs add --cid-base=base32 --upgrade-cidv0-in-output=false mountdir/base32-test.txt >actual\n '\n test_expect_success \"ipfs add --cid-base=base32 --upgrade-cidv0-in-output=false output looks good\" '\n HASHv0=$(cid-fmt -v 0 -b z %s \"$HASHb32\") &&\n echo \"added $HASHv0 base32-test.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --cid-base=base32 --upgrade-cidv0-in-output=false --only-hash succeeds\" '\n ipfs add --cid-base=base32 --upgrade-cidv0-in-output=false --only-hash mountdir/base32-test.txt > oh_actual\n '\n test_expect_success \"ipfs add --cid-base=base32 --upgrade-cidv0-in-output=false --only-hash output looks good\" '\n test_cmp expected oh_actual\n '\n\n test_expect_success \"ipfs cat with base32 hash succeeds\" '\n ipfs cat \"$HASHb32\" >actual\n '\n test_expect_success \"ipfs cat with base32 hash output looks good\" '\n echo \"base32 test\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat using CIDv0 hash succeeds\" '\n ipfs cat \"$HASHv0\" >actual\n '\n test_expect_success \"ipfs cat using CIDv0 hash looks good\" '\n echo \"base32 test\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add with multiple files succeeds\" '\n echo \"Helloo Worlds!\" >mountdir/hello2.txt &&\n ipfs add mountdir/hello.txt mountdir/hello2.txt >actual\n '\n\n test_expect_success \"ipfs add with multiple files output looks good\" '\n echo \"added QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH hello.txt\" >expected &&\n echo \"added Qmf35k66MZNW2GijohUmXQEWKZU4cCGTCwK6idfnt152wJ hello2.txt\" >> expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add with multiple files of same name and import dir succeeds\" '\n ipfs add mountdir/hello.txt mountdir/hello.txt >actual\n '\n\n test_expect_success \"ipfs add with multiple files of same name output looks good\" '\n echo \"added QmVr26fY1tKyspEJBniVhqxQeEjhF78XerGiqWAwraVLQH hello.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add with multiple files of same name but different dirs fails\" '\n mkdir -p mountdir/same-file/ &&\n cp mountdir/hello.txt mountdir/same-file/hello.txt &&\n test_expect_code 1 ipfs add mountdir/hello.txt mountdir/same-file/hello.txt >actual &&\n rm mountdir/same-file/hello.txt &&\n rmdir mountdir/same-file\n '\n\n ## --to-files with single source\n\n test_expect_success \"ipfs add --to-files /mfspath succeeds\" '\n mkdir -p mountdir && echo \"Hello MFS!\" > mountdir/mfs.txt &&\n ipfs add mountdir/mfs.txt --to-files /ipfs-add-to-files >actual\n '\n\n test_expect_success \"ipfs add --to-files output looks good\" '\n HASH_MFS=\"QmVT8bL3sGBA2TwvX8JPhrv5CYZL8LLLfW7mxkUjPZsgBr\" &&\n echo \"added $HASH_MFS mfs.txt\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs files read succeeds\" '\n ipfs files read /ipfs-add-to-files >actual &&\n ipfs files rm /ipfs-add-to-files\n '\n\n test_expect_success \"ipfs cat output looks good\" '\n echo \"Hello MFS!\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --to-files requires argument\" '\n test_expect_code 1 ipfs add mountdir/mfs.txt --to-files >actual 2>&1 &&\n test_should_contain \"Error: missing argument for option \\\"to-files\\\"\" actual\n '\n\n test_expect_success \"ipfs add --to-files / (MFS root) works\" '\n echo \"Hello MFS!\" >expected &&\n ipfs add mountdir/mfs.txt --to-files / &&\n ipfs files read /mfs.txt >actual &&\n test_cmp expected actual &&\n ipfs files rm /mfs.txt &&\n rm mountdir/mfs.txt\n '\n\n ## --to-files with multiple sources\n\n test_expect_success \"ipfs add file1 file2 --to-files /mfspath0 (without trailing slash) fails\" '\n mkdir -p test &&\n echo \"file1\" > test/mfs1.txt &&\n echo \"file2\" > test/mfs2.txt &&\n test_expect_code 1 ipfs add test/mfs1.txt test/mfs2.txt --to-files /mfspath0 >actual 2>&1 &&\n test_should_contain \"MFS destination is a file: only one entry can be copied to \\\"/mfspath0\\\"\" actual &&\n ipfs files rm -r --force /mfspath0\n '\n\n test_expect_success \"ipfs add file1 file2 --to-files /mfsfile1 (without trailing slash + with preexisting file) fails\" '\n echo test | ipfs files write --create /mfsfile1 &&\n test_expect_code 1 ipfs add test/mfs1.txt test/mfs2.txt --to-files /mfsfile1 >actual 2>&1 &&\n test_should_contain \"Error: to-files: cannot put node in path \\\"/mfsfile1\\\"\" actual &&\n ipfs files rm -r --force /mfsfile1\n '\n\n test_expect_success \"ipfs add file1 file2 --to-files /mfsdir1 (without trailing slash + with preexisting dir) fails\" '\n ipfs files mkdir -p /mfsdir1 &&\n test_expect_code 1 ipfs add test/mfs1.txt test/mfs2.txt --to-files /mfsdir1 >actual 2>&1 &&\n test_should_contain \"Error: to-files: cannot put node in path \\\"/mfsdir1\\\"\" actual &&\n ipfs files rm -r --force /mfsdir1\n '\n\n test_expect_success \"ipfs add file1 file2 --to-files /mfsdir2/ (with trailing slash) succeeds\" '\n ipfs files mkdir -p /mfsdir2 &&\n test_expect_code 0 ipfs add --cid-version 1 test/mfs1.txt test/mfs2.txt --to-files /mfsdir2/ > actual 2>&1 &&\n test_should_contain \"added bafkreihm3rktn5z33luic3youqdsn326toaq3ekesmdvsa53sbrd3f5r3a mfs1.txt\" actual &&\n test_should_contain \"added bafkreidh5zkhr2vnwa2luwmuj24xo6l3jhfgvkgtk5cyp43oxs7owzpxby mfs2.txt\" actual &&\n test_should_not_contain \"Error\" actual &&\n ipfs files ls /mfsdir2/ > lsout &&\n test_should_contain \"mfs1.txt\" lsout &&\n test_should_contain \"mfs2.txt\" lsout &&\n ipfs files rm -r --force /mfsdir2\n '\n\n test_expect_success \"ipfs add file1 file2 --to-files /mfsfile2/ (with trailing slash + with preexisting file) fails\" '\n echo test | ipfs files write --create /mfsfile2 &&\n test_expect_code 1 ipfs add test/mfs1.txt test/mfs2.txt --to-files /mfsfile2/ >actual 2>&1 &&\n test_should_contain \"Error: to-files: MFS destination \\\"/mfsfile2/\\\" is not a directory\" actual &&\n ipfs files rm -r --force /mfsfile2\n '\n\n ## --to-files with recursive dir\n\n # test MFS destination without trailing slash\n test_expect_success \"ipfs add with --to-files /mfs/subdir3 fails because /mfs/subdir3 exists\" '\n ipfs files mkdir -p /mfs/subdir3 &&\n test_expect_code 1 ipfs add -r test --to-files /mfs/subdir3 >actual 2>&1 &&\n test_should_contain \"cannot put node in path \\\"/mfs/subdir3\\\": directory already has entry by that name\" actual &&\n ipfs files rm -r --force /mfs\n '\n\n # test recursive import of a dir into MFS subdirectory\n test_expect_success \"ipfs add -r dir --to-files /mfs/subdir4/ succeeds (because of trailing slash)\" '\n ipfs files mkdir -p /mfs/subdir4 &&\n ipfs add --cid-version 1 -r test --to-files /mfs/subdir4/ >actual 2>&1 &&\n test_should_contain \"added bafkreihm3rktn5z33luic3youqdsn326toaq3ekesmdvsa53sbrd3f5r3a test/mfs1.txt\" actual &&\n test_should_contain \"added bafkreidh5zkhr2vnwa2luwmuj24xo6l3jhfgvkgtk5cyp43oxs7owzpxby test/mfs2.txt\" actual &&\n test_should_contain \"added bafybeic7xwqwovt4g4bax6d3udp6222i63vj2rblpbim7uy2uw4a5gahha test\" actual &&\n test_should_not_contain \"Error\" actual\n ipfs files ls /mfs/subdir4/ > lsout &&\n test_should_contain \"test\" lsout &&\n test_should_not_contain \"mfs1.txt\" lsout &&\n test_should_not_contain \"mfs2.txt\" lsout &&\n ipfs files rm -r --force /mfs\n '\n\n # confirm -w and --to-files are exclusive\n # context: https://github.com/ipfs/kubo/issues/10611\n test_expect_success \"ipfs add -r -w dir --to-files /mfs/subdir5/ errors (-w and --to-files are exclusive)\" '\n ipfs files mkdir -p /mfs/subdir5 &&\n test_expect_code 1 ipfs add -r -w test --to-files /mfs/subdir5/ >actual 2>&1 &&\n test_should_contain \"Error\" actual &&\n ipfs files rm -r --force /mfs\n '\n\n}\n\ntest_add_cat_5MB() {\n ADD_FLAGS=\"$1\"\n EXP_HASH=\"$2\"\n\n test_expect_success \"generate 5MB file using random-data\" '\n random-data -size=5242880 -seed=41 >mountdir/bigfile\n '\n\n test_expect_success \"sha1 of the file looks ok\" '\n echo \"11145b8c4bc8f87ea2fcfc3d55708b8cac2aadf12862\" >sha1_expected &&\n multihash -a=sha1 -e=hex mountdir/bigfile >sha1_actual &&\n test_cmp sha1_expected sha1_actual\n '\n\n test_expect_success \"'ipfs add $ADD_FLAGS bigfile' succeeds\" '\n ipfs add $ADD_FLAGS mountdir/bigfile >actual ||\n test_fsh cat daemon_err\n '\n\n test_expect_success \"'ipfs add bigfile' output looks good\" '\n echo \"added $EXP_HASH bigfile\" >expected &&\n test_cmp expected actual\n '\n test_expect_success \"'ipfs cat' succeeds\" '\n ipfs cat \"$EXP_HASH\" >actual\n '\n\n test_expect_success \"'ipfs cat' output looks good\" '\n test_cmp mountdir/bigfile actual\n '\n\n test_expect_success FUSE \"cat ipfs/bigfile succeeds\" '\n cat \"ipfs/$EXP_HASH\" >actual\n '\n\n test_expect_success FUSE \"cat ipfs/bigfile looks good\" '\n test_cmp mountdir/bigfile actual\n '\n\n test_expect_success \"remove hash\" '\n ipfs pin rm \"$EXP_HASH\" &&\n ipfs block rm \"$EXP_HASH\"\n '\n\n test_expect_success \"get base32 version of CID\" '\n ipfs cid base32 $EXP_HASH > base32_cid &&\n BASE32_HASH=`cat base32_cid`\n '\n\n test_expect_success \"ipfs add --cid-base=base32 bigfile' succeeds\" '\n ipfs add $ADD_FLAGS --cid-base=base32 mountdir/bigfile >actual ||\n test_fsh cat daemon_err\n '\n\n test_expect_success \"'ipfs add bigfile --cid-base=base32' output looks good\" '\n echo \"added $BASE32_HASH bigfile\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"'ipfs cat $BASE32_HASH' succeeds\" '\n ipfs cat \"$BASE32_HASH\" >actual\n '\n}\n\ntest_add_cat_raw() {\n test_expect_success \"add a small file with raw-leaves\" '\n echo \"foobar\" > afile &&\n HASH=$(ipfs add -q --raw-leaves afile)\n '\n\n test_expect_success \"cat that small file\" '\n ipfs cat $HASH > afile_out\n '\n\n test_expect_success \"make sure it looks good\" '\n test_cmp afile afile_out\n '\n\n test_expect_success \"add zero length file with raw-leaves\" '\n touch zero-length-file &&\n ZEROHASH=$(ipfs add -q --raw-leaves zero-length-file) &&\n echo $ZEROHASH\n '\n\n test_expect_success \"zero length file has correct hash\" '\n test \"$ZEROHASH\" = bafkreihdwdcefgh4dqkjv67uzcmw7ojee6xedzdetojuzjevtenxquvyku\n '\n\n test_expect_success \"cat zero length file\" '\n ipfs cat $ZEROHASH > zero-length-file_out\n '\n\n test_expect_success \"make sure it looks good\" '\n test_cmp zero-length-file zero-length-file_out\n '\n}\n\ntest_add_cat_derefargs() {\n test_expect_success \"create and hash zero length file\" '\n touch zero-length-file &&\n ZEROHASH=$(ipfs add -q -n zero-length-file)\n '\n\n test_expect_success \"create symlink and add with dereferenced arguments\" '\n ln -s zero-length-file symlink-to-zero &&\n HASH=$(ipfs add -q -n --dereference-args symlink-to-zero) &&\n test $HASH = $ZEROHASH\n '\n}\n\ntest_add_cat_expensive() {\n ADD_FLAGS=\"$1\"\n HASH=\"$2\"\n\n test_expect_success EXPENSIVE \"generate 100MB file using random-data\" '\n random-data -size=104857600 -seed=42 >mountdir/bigfile\n '\n\n test_expect_success EXPENSIVE \"sha1 of the file looks ok\" '\n echo \"11141e8c04d7cd019cc0acf0311a8ca6cf2c18413c96\" >sha1_expected &&\n multihash -a=sha1 -e=hex mountdir/bigfile >sha1_actual &&\n test_cmp sha1_expected sha1_actual\n '\n\n test_expect_success EXPENSIVE \"ipfs add $ADD_FLAGS bigfile succeeds\" '\n ipfs add $ADD_FLAGS mountdir/bigfile >actual\n '\n\n test_expect_success EXPENSIVE \"ipfs add bigfile output looks good\" '\n echo \"added $HASH bigfile\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success EXPENSIVE \"ipfs cat succeeds\" '\n ipfs cat \"$HASH\" | multihash -a=sha1 -e=hex >sha1_actual\n '\n\n test_expect_success EXPENSIVE \"ipfs cat output looks good\" '\n ipfs cat \"$HASH\" >actual &&\n test_cmp mountdir/bigfile actual\n '\n\n test_expect_success EXPENSIVE \"ipfs cat output hashed looks good\" '\n echo \"11141e8c04d7cd019cc0acf0311a8ca6cf2c18413c96\" >sha1_expected &&\n test_cmp sha1_expected sha1_actual\n '\n\n test_expect_success FUSE,EXPENSIVE \"cat ipfs/bigfile succeeds\" '\n cat \"ipfs/$HASH\" | multihash -a=sha1 -e=hex >sha1_actual\n '\n\n test_expect_success FUSE,EXPENSIVE \"cat ipfs/bigfile looks good\" '\n test_cmp sha1_expected sha1_actual\n '\n}\n\ntest_add_named_pipe() {\n test_expect_success \"Adding named pipes explicitly works\" '\n mkfifo named-pipe1 &&\n ( echo foo > named-pipe1 & echo \"added $( echo foo | ipfs add -nq ) named-pipe1\" > expected_named_pipes_add ) &&\n mkfifo named-pipe2 &&\n ( echo bar > named-pipe2 & echo \"added $( echo bar | ipfs add -nq ) named-pipe2\" >> expected_named_pipes_add ) &&\n ipfs add -n named-pipe1 named-pipe2 >actual_pipe_add &&\n rm named-pipe1 &&\n rm named-pipe2 &&\n test_cmp expected_named_pipes_add actual_pipe_add\n '\n\n test_expect_success \"useful error message when recursively adding a named pipe\" '\n mkdir -p named-pipe-dir &&\n mkfifo named-pipe-dir/named-pipe &&\n STAT=$(generic_stat named-pipe-dir/named-pipe) &&\n test_expect_code 1 ipfs add -r named-pipe-dir 2>actual &&\n printf \"Error: unrecognized file type for named-pipe-dir/named-pipe: $STAT\\n\" >expected &&\n rm named-pipe-dir/named-pipe &&\n rmdir named-pipe-dir &&\n test_cmp expected actual\n '\n}\n\ntest_add_pwd_is_symlink() {\n test_expect_success \"ipfs add -r adds directory content when ./ is symlink\" '\n mkdir hellodir &&\n echo \"World\" > hellodir/world &&\n ln -s hellodir hellolink &&\n ( cd hellolink &&\n ipfs add -r . > ../actual ) &&\n grep \"added Qma9CyFdG5ffrZCcYSin2uAETygB25cswVwEYYzwfQuhTe\" actual &&\n rm -r hellodir\n '\n}\n\ntest_launch_ipfs_daemon_and_mount\n\ntest_expect_success \"'ipfs add --help' succeeds\" '\n ipfs add --help >actual\n'\n\ntest_expect_success \"'ipfs add --help' output looks good\" '\n egrep \"ipfs add.*\" actual >/dev/null ||\n test_fsh cat actual\n'\n\ntest_expect_success \"'ipfs help add' succeeds\" '\n ipfs help add >actual\n'\n\ntest_expect_success \"'ipfs help add' output looks good\" '\n egrep \"ipfs add.*\" actual >/dev/null ||\n test_fsh cat actual\n'\n\ntest_expect_success \"'ipfs cat --help' succeeds\" '\n ipfs cat --help >actual\n'\n\ntest_expect_success \"'ipfs cat --help' output looks good\" '\n egrep \"ipfs cat.*\" actual >/dev/null ||\n test_fsh cat actual\n'\n\ntest_expect_success \"'ipfs help cat' succeeds\" '\n ipfs help cat >actual\n'\n\ntest_expect_success \"'ipfs help cat' output looks good\" '\n egrep \"ipfs cat.*\" actual >/dev/null ||\n test_fsh cat actual\n'\n\ntest_add_cat_file\n\ntest_expect_success \"ipfs cat succeeds with stdin opened (issue #1141)\" '\n cat mountdir/hello.txt | while read line; do ipfs cat \"$HASH\" >actual || exit; done\n'\n\ntest_expect_success \"ipfs cat output looks good\" '\n cat mountdir/hello.txt >expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"ipfs cat accept hash from built input\" '\n echo \"$HASH\" | ipfs cat >actual\n'\n\ntest_expect_success \"ipfs cat output looks good\" '\n test_cmp expected actual\n'\n\ntest_expect_success FUSE \"cat ipfs/stuff succeeds\" '\n cat \"ipfs/$HASH\" >actual\n'\n\ntest_expect_success FUSE \"cat ipfs/stuff looks good\" '\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs add -q' succeeds\" '\n echo \"Hello Venus!\" >mountdir/venus.txt &&\n ipfs add -q mountdir/venus.txt >actual\n'\n\ntest_expect_success \"'ipfs add -q' output looks good\" '\n HASH=\"QmU5kp3BH3B8tnWUU2Pikdb2maksBNkb92FHRr56hyghh4\" &&\n echo \"$HASH\" >expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs add -q' with stdin input succeeds\" '\n echo \"Hello Jupiter!\" | ipfs add -q >actual\n'\n\ntest_expect_success \"'ipfs add -q' output looks good\" '\n HASH=\"QmUnvPcBctVTAcJpigv6KMqDvmDewksPWrNVoy1E1WP5fh\" &&\n echo \"$HASH\" >expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs cat' succeeds\" '\n ipfs cat \"$HASH\" >actual\n'\n\ntest_expect_success \"ipfs cat output looks good\" '\n echo \"Hello Jupiter!\" >expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs add' with stdin input succeeds\" '\n printf \"Hello Neptune!\\nHello Pluton!\" | ipfs add >actual\n'\n\ntest_expect_success \"'ipfs add' output looks good\" '\n HASH=\"QmZDhWpi8NvKrekaYYhxKCdNVGWsFFe1CREnAjP1QbPaB3\" &&\n echo \"added $HASH $HASH\" >expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs cat' with built input succeeds\" '\n echo \"$HASH\" | ipfs cat >actual\n'\n\ntest_expect_success \"ipfs cat with built input output looks good\" '\n printf \"Hello Neptune!\\nHello Pluton!\" >expected &&\n test_cmp expected actual\n'\n\nadd_directory() {\n EXTRA_ARGS=$1\n\n test_expect_success \"'ipfs add -r $EXTRA_ARGS' succeeds\" '\n mkdir mountdir/planets &&\n echo \"Hello Mars!\" >mountdir/planets/mars.txt &&\n echo \"Hello Venus!\" >mountdir/planets/venus.txt &&\n ipfs add -r $EXTRA_ARGS mountdir/planets >actual\n '\n\n test_expect_success \"'ipfs add -r $EXTRA_ARGS' output looks good\" '\n echo \"added $MARS planets/mars.txt\" >expected &&\n echo \"added $VENUS planets/venus.txt\" >>expected &&\n echo \"added $PLANETS planets\" >>expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat accept many hashes from built input\" '\n { echo \"$MARS\"; echo \"$VENUS\"; } | ipfs cat >actual\n '\n\n test_expect_success \"ipfs cat output looks good\" '\n cat mountdir/planets/mars.txt mountdir/planets/venus.txt >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat accept many hashes as args\" '\n ipfs cat \"$MARS\" \"$VENUS\" >actual\n '\n\n test_expect_success \"ipfs cat output looks good\" '\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with both arg and stdin\" '\n echo \"$MARS\" | ipfs cat \"$VENUS\" >actual\n '\n\n test_expect_success \"ipfs cat output looks good\" '\n cat mountdir/planets/venus.txt >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs cat with two args and stdin\" '\n echo \"$MARS\" | ipfs cat \"$VENUS\" \"$VENUS\" >actual\n '\n\n test_expect_success \"ipfs cat output looks good\" '\n cat mountdir/planets/venus.txt mountdir/planets/venus.txt >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs add --quieter succeeds\" '\n ipfs add -r -Q $EXTRA_ARGS mountdir/planets >actual\n '\n\n test_expect_success \"ipfs add --quieter returns only one correct hash\" '\n echo \"$PLANETS\" > expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"cleanup\" '\n rm -r mountdir/planets\n '\n}\n\nPLANETS=\"QmWSgS32xQEcXMeqd3YPJLrNBLSdsfYCep2U7CFkyrjXwY\"\nMARS=\"QmPrrHqJzto9m7SyiRzarwkqPcCSsKR2EB1AyqJfe8L8tN\"\nVENUS=\"QmU5kp3BH3B8tnWUU2Pikdb2maksBNkb92FHRr56hyghh4\"\nadd_directory\n\nPLANETS=\"QmfWfQfKCY5Ukv9peBbxM5vqWM9BzmqUSXvdCgjT2wsiBT\"\nMARS=\"bafkreibmlvvgdyihetgocpof6xk64kjjzdeq2e4c7hqs3krdheosk4tgj4\"\nVENUS=\"bafkreihfsphazrk2ilejpekyltjeh5k4yvwgjuwg26ueafohqioeo3sdca\"\nadd_directory '--raw-leaves'\n\nPLANETS=\"bafybeih7e5dmkyk25up5vxug4q3hrg2fxbzf23dfrac2fns5h7z4aa7ioi\"\nMARS=\"bafkreibmlvvgdyihetgocpof6xk64kjjzdeq2e4c7hqs3krdheosk4tgj4\"\nVENUS=\"bafkreihfsphazrk2ilejpekyltjeh5k4yvwgjuwg26ueafohqioeo3sdca\"\nadd_directory '--cid-version=1'\n\nPLANETS=\"bafybeif5tuep5ap2d7zyhbktucey75aoacxufgt6i3v4gebmixyipnyp7y\"\nMARS=\"bafybeiawta2ntdmsy24aro35w3homzl4ak7svr3si7l7gesvq4erglyye4\"\nVENUS=\"bafybeicvkvhs2fr75ynebtdjqpgm4g2fc63abqbmysupwpmcjl4gx7mzrm\"\nadd_directory '--cid-version=1 --raw-leaves=false'\n\nPLANETS=\"bafykbzaceaptbcs7ik5mdfpot3b4ackvxlwh7loc5jcrtkayf64ukl7zyk46e\"\nMARS=\"bafk2bzaceaqcxw46uzkyd2jmczoogof6pnkqt4dpiv3pwkunsv4g5rkkmecie\"\nVENUS=\"bafk2bzacebxnke2fb5mgzxyjuuavvcfht4fd3gvn4klkujz6k72wboynhuvfw\"\nadd_directory '--hash=blake2b-256'\n\ntest_expect_success \"'ipfs add -rn' succeeds\" '\n mkdir -p mountdir/moons/jupiter &&\n mkdir -p mountdir/moons/saturn &&\n echo \"Hello Europa!\" >mountdir/moons/jupiter/europa.txt &&\n echo \"Hello Titan!\" >mountdir/moons/saturn/titan.txt &&\n echo \"hey you are no moon!\" >mountdir/moons/mercury.txt &&\n ipfs add -rn mountdir/moons >actual\n'\n\ntest_expect_success \"'ipfs add -rn' output looks good\" '\n MOONS=\"QmbGoaQZm8kjYfCiN1aBsgwhqfUBGDYTrDb91Mz7Dvq81B\" &&\n EUROPA=\"Qmbjg7zWdqdMaK2BucPncJQDxiALExph5k3NkQv5RHpccu\" &&\n JUPITER=\"QmS5mZddhFPLWFX3w6FzAy9QxyYkaxvUpsWCtZ3r7jub9J\" &&\n SATURN=\"QmaMagZT4rTE7Nonw8KGSK4oe1bh533yhZrCo1HihSG8FK\" &&\n TITAN=\"QmZzppb9WHn552rmRqpPfgU5FEiHH6gDwi3MrB9cTdPwdb\" &&\n MERCURY=\"QmRsTB5CpEUvDUpDgHCzb3VftZ139zrk9zs5ZcgYh9TMPJ\" &&\n echo \"added $EUROPA moons/jupiter/europa.txt\" >expected &&\n echo \"added $MERCURY moons/mercury.txt\" >>expected &&\n echo \"added $TITAN moons/saturn/titan.txt\" >>expected &&\n echo \"added $JUPITER moons/jupiter\" >>expected &&\n echo \"added $SATURN moons/saturn\" >>expected &&\n echo \"added $MOONS moons\" >>expected &&\n test_cmp expected actual\n'\n\ntest_expect_success \"random-data is installed\" '\n type random-data\n'\n\ntest_add_cat_5MB \"\" \"QmapAfmzmeWYTNztMQEhUXFcSGrsax22WRG7YN9xLdMeQq\"\n\ntest_add_cat_5MB --raw-leaves \"QmabWSFaPusmiZaaVZLhEUtHcj8CCvVeUfkBpKqAkKVMiS\"\n\n# note: the specified hash implies that internal nodes are stored\n# using CidV1 and leaves are stored using raw blocks\ntest_add_cat_5MB --cid-version=1 \"bafybeifwdkm32fmukqwh3jofm6ma76bcqvn6opxstsnzmya7utboi4cb2m\"\n\n# note: the specified hash implies that internal nodes are stored\n# using CidV1 and leaves are stored using CidV1 but using the legacy\n# format (i.e. not raw)\ntest_add_cat_5MB '--cid-version=1 --raw-leaves=false' \"bafybeifq4unep5w4agr3nlynxidj2rymf6dzu6bf4ieqqildkboe5mdmne\"\n\n# note: --hash=blake2b-256 implies --cid-version=1 which implies --raw-leaves=true\n# the specified hash represents the leaf nodes stored as raw leaves and\n# encoded with the blake2b-256 hash function\ntest_add_cat_5MB '--hash=blake2b-256' \"bafykbzacebxcnlql4oc3mtscqn32aumqkqxxv3wt7dkyrphgh6lc2gckiq6bw\"\n\n# the specified hash represents the leaf nodes stored as protoful nodes and\n# encoded with the blake2b-256 hash function\ntest_add_cat_5MB '--hash=blake2b-256 --raw-leaves=false' \"bafykbzacearibnoamkfmcagpfgk2sbgx65qftnsrh4ttd3g7ghooasfnyavme\"\n\ntest_add_cat_expensive \"\" \"Qma1WZKC3jad7e3F7GEDvkFdhPLyMEhKszBF4nBUCBGh6c\"\n\n# note: the specified hash implies that internal nodes are stored\n# using CidV1 and leaves are stored using raw blocks\ntest_add_cat_expensive \"--cid-version=1\" \"bafybeibdfw7nsmb3erhej2k6v4eopaswsf5yfv2ikweqa3qsc5no4jywqu\"\n\n# note: --hash=blake2b-256 implies --cid-version=1 which implies --raw-leaves=true\n# the specified hash represents the leaf nodes stored as raw leaves and\n# encoded with the blake2b-256 hash function\ntest_add_cat_expensive '--hash=blake2b-256' \"bafykbzaceduy3thhmcf6ptfqzxberlvj7sgo4uokrvd6qwrhim6r3rgcb26qi\"\n\ntest_add_named_pipe\n\ntest_add_pwd_is_symlink\n\ntest_add_cat_raw\n\ntest_expect_success \"ipfs add --cid-version=9 fails\" '\n echo \"context\" > afile.txt &&\n test_must_fail ipfs add --cid-version=9 afile.txt 2>&1 | tee add_out &&\n grep -q \"unknown CID version\" add_out\n'\n\ntest_kill_ipfs_daemon\n\n# should work offline\n\ntest_add_cat_file\n\ntest_add_cat_raw\n\ntest_expect_success \"ipfs add --only-hash succeeds\" '\n echo \"unknown content for only-hash\" | ipfs add --only-hash -q > oh_hash\n'\n\ntest_add_cat_derefargs\n\n#TODO: this doesn't work when online hence separated out from test_add_cat_file\ntest_expect_success \"ipfs cat file fails\" '\n test_must_fail ipfs cat $(cat oh_hash)\n'\n\ntest_add_named_pipe\n\ntest_add_pwd_is_symlink\n\n# Test daemon in offline mode\ntest_launch_ipfs_daemon_without_network\n\ntest_add_cat_file\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0042-add-skip.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test add and cat commands\"\n\n. lib/test-lib.sh\n\ntest_add_skip() {\n\n test_expect_success \"'ipfs add -r' with hidden file succeeds\" '\n mkdir -p mountdir/planets/.asteroids &&\n echo \"mars.txt\" >mountdir/planets/.gitignore &&\n echo \"Hello Mars\" >mountdir/planets/mars.txt &&\n echo \"Hello Venus\" >mountdir/planets/venus.txt &&\n echo \"Hello Pluto\" >mountdir/planets/.pluto.txt &&\n echo \"Hello Charon\" >mountdir/planets/.charon.txt &&\n echo \"Hello Ceres\" >mountdir/planets/.asteroids/ceres.txt &&\n echo \"Hello Pallas\" >mountdir/planets/.asteroids/pallas.txt &&\n ipfs add -r mountdir/planets >actual\n '\n\n test_expect_success \"'ipfs add -r' did not include . files\" '\n cat >expected <<-\\EOF &&\nadded QmZy3khu7qf696i5HtkgL2NotsCZ8wzvNZJ1eUdA5n8KaV planets/mars.txt\nadded QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt\nadded QmR8nD1Vzk5twWVC6oShTHvv7mMYkVh6dApCByBJyV2oj3 planets\nEOF\n test_cmp expected actual\n '\n\n test_expect_success \"'ipfs add -r --hidden' succeeds\" '\n ipfs add -r --hidden mountdir/planets >actual\n '\n\n test_expect_success \"'ipfs add -r --hidden' did include . files\" '\n cat >expected <<-\\EOF &&\nadded QmcAREBcjgnUpKfyFmUGnfajA1NQS5ydqRp7WfqZ6JF8Dx planets/.asteroids/ceres.txt\nadded QmZ5eaLybJ5GUZBNwy24AA9EEDTDpA4B8qXnuN3cGxu2uF planets/.asteroids/pallas.txt\nadded QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt\nadded QmPHrRjTH8FskN3C2iv6BLekDT94o23KSL2u5qLqQqGhVH planets/.gitignore\nadded QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt\nadded QmZy3khu7qf696i5HtkgL2NotsCZ8wzvNZJ1eUdA5n8KaV planets/mars.txt\nadded QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt\nadded Qmf6rbs5GF85anDuoxpSAdtuZPM9D2Yt3HngzjUVSQ7kDV planets/.asteroids\nadded QmczhHaXyb3bc9APMxe4MXbr87V5YDLKLaw3DZX3fK7HrK planets\nEOF\n test_cmp expected actual\n '\n\n test_expect_success \"'ipfs add -r --ignore-rules-path=.gitignore --hidden' succeeds\" '\n (cd mountdir/planets && ipfs add -r --ignore-rules-path=.gitignore --hidden .) > actual\n '\n\n test_expect_success \"'ipfs add -r --ignore-rules-path=.gitignore --hidden' did not include mars.txt file\" '\n cat >expected <<-\\EOF &&\nadded QmcAREBcjgnUpKfyFmUGnfajA1NQS5ydqRp7WfqZ6JF8Dx planets/.asteroids/ceres.txt\nadded QmZ5eaLybJ5GUZBNwy24AA9EEDTDpA4B8qXnuN3cGxu2uF planets/.asteroids/pallas.txt\nadded QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt\nadded QmPHrRjTH8FskN3C2iv6BLekDT94o23KSL2u5qLqQqGhVH planets/.gitignore\nadded QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt\nadded QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt\nadded Qmf6rbs5GF85anDuoxpSAdtuZPM9D2Yt3HngzjUVSQ7kDV planets/.asteroids\nadded QmaRsiaCYvc65RqHVAcv2tqyjZgQYgvaNqW1tQGsjfy4N5 planets\nEOF\n test_cmp expected actual\n '\n\n test_expect_success \"'ipfs add -r --ignore-rules-path=.gitignore --ignore .asteroids --ignore venus.txt --hidden' succeeds\" '\n (cd mountdir/planets && ipfs add -r --ignore-rules-path=.gitignore --ignore .asteroids --ignore venus.txt --hidden .) > actual\n '\n\n test_expect_success \"'ipfs add -r --ignore-rules-path=.gitignore --ignore .asteroids --ignore venus.txt --hidden' did not include ignored files\" '\n cat >expected <<-\\EOF &&\nadded QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt\nadded QmPHrRjTH8FskN3C2iv6BLekDT94o23KSL2u5qLqQqGhVH planets/.gitignore\nadded QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt\nadded QmemuMahjSh7eYLY3hbz2q8sqMPnbQzBQeUdosqNiWChE6 planets\nEOF\n test_cmp expected actual\n '\n\n test_expect_success \"'ipfs add' includes hidden files given explicitly even without --hidden\" '\n mkdir -p mountdir/dotfiles &&\n echo \"set nocompatible\" > mountdir/dotfiles/.vimrc\n cat >expected <<-\\EOF &&\nadded QmT4uMRDCN7EMpFeqwvKkboszbqeW1kWVGrBxBuCGqZcQc .vimrc\nEOF\n ipfs add mountdir/dotfiles/.vimrc >actual\n cat actual\n test_cmp expected actual\n '\n\n test_expect_success \"'ipfs add' with an unregistered hash and wrapped leaves fails without crashing\" '\n test_expect_code 1 ipfs add --hash poseidon-bls12_381-a2-fc1 --raw-leaves=false -r mountdir/planets\n '\n\n}\n\n# should work offline\ntest_init_ipfs\ntest_add_skip\n\n# should work online\ntest_launch_ipfs_daemon\ntest_add_skip\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0043-add-w.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test add -w\"\n\nadd_w_m='QmbDfuW3tZ5PmAucyLBAMzVeETHCHM7Ho9CWdBvWxRGd3i'\n\nadd_w_1='added QmP9WCV5SjQRoxoCkgywzw4q5X23rhHJJXzPQt4VbNa9M5 0h0r91\nadded Qmave82G8vLbtx6JCokrrhLPpFNfWj5pbXobddiUASfpe3 '\n\nadd_w_12='added QmP9WCV5SjQRoxoCkgywzw4q5X23rhHJJXzPQt4VbNa9M5 0h0r91\nadded QmNUiT9caQy5zXvw942UYXkjLseQLWBkf7ZJD6RCfk8JgP 951op\nadded QmWXoq9vUtdNxmM16kvJRgyQdi4S4gfYSjd2MsRprBXWmG '\n\nadd_w_d1='added QmQKZCZKKL71zcMNpFFVcWzoh5dimX45mKgUu3LhvdaCRn 3s78oa/cb5v5v\nadded QmPng2maSno8o659Lu2QtKg2d2L53RMahoyK6wNkifYaxY 3s78oa/cnd062l-rh\nadded QmX3s7jJjFQhKRuGpDA3W4BYHdCWAyL3oB6U3iSoaYxVxs 3s78oa/es3gm9ck7b\nadded QmSUZXb48DoNjUPpX9Jue1mUpyCghEDZY62iif1JhdofoG 3s78oa/kfo77-6i_hp0ttz\nadded QmdC215Wp2sH47aw6R9CLBVa5uxJB4zEag1gtsKqjYGDb5 3s78oa/p91vs5t\nadded QmSEGJRYb5wrJRBxNsse91YJSpmgf5ikKRtCwvGZ1V1Nc2 3s78oa\nadded QmS2ML7DPVisc4gQtSrwMi3qwS9eyzGR7zVdwqwRPU9rGz '\n\nadd_w_d1_v1='added bafkreibpfapmbmf55elpipnoofmda7xbs5spthba2srrovnchttzplmrnm fvmq97/0vz12t0yf\nadded bafkreihc5hdzpjwbqy6b5r2h2oxbm6mp4sx4eqll253k6f5yijsismvoxy fvmq97/2hpfk8slf0\nadded bafkreihlmwk6pkk7klsmypmk2wfkgijbk7wavhtrcvgrfxvug7x5ndawge fvmq97/nda000755cd76\nadded bafkreigpntro6bt4m6c5pcnmvk24qyiq3lwffhwry7k2hqtretqhfsfvqa fvmq97/nsz0wsonz\nadded bafkreieeznfvzr6742npktcn4ajzxujst6j2uztwfninhvic4bbvm356u4 fvmq97/pq3f6t0\nadded bafybeiatm3oos62mm5hu4cmq234wipw2fjaqflq2cdqgc6i6dcgzamxwrm fvmq97\nadded bafybeifp4ioszjk2377psexdhk7thcxnpaj2wls4yifsntbgxzti7ds4uy '\n\nadd_w_d2='added QmP9WCV5SjQRoxoCkgywzw4q5X23rhHJJXzPQt4VbNa9M5 0h0r91\nadded QmPpv7rFgkBqMYKJok6kVixqJgAGkyPiX3Jrr7n9rU1gcv 1o8ef-25onywi\nadded QmW7zDxGpaJTRpte7uCvMA9eXJ5L274FfsFPK9pE5RShq9 2ju9tn-b09/-qw1d8j9\nadded QmNNm9D3pn8NXbuYSde614qbb9xE67g9TNV6zXePgSZvHj 2ju9tn-b09/03rfc61t4qq_m\nadded QmUYefaFAWka9LWarDeetQFe8CCSHaAtj4JR7YToYPSJyi 2ju9tn-b09/57dl-1lbjvu\nadded QmcMLvVinwJsHtYxTUXEoPd8XkbuyvJNffZ85PT11cWDc2 2ju9tn-b09/t8h1_w\nadded QmUTZE57VoF7xqWmrrcDNtDXrEs6znTQaRwmwkawGDs1GA 2ju9tn-b09/ugqi0nmv-1\nadded QmfX5q9CMquL4JnAuG4H13RXjTb9DncMfu9pvpEsWkECJk fvmq97/0vz12t0yf\nadded Qmdr3jR1UATLFeuoieBTHLNNwhCUJbgN5oat7U9X8TtfdZ fvmq97/2hpfk8slf0\nadded QmfUKgXSiE1wCQuX3Pws9FftthJuAMXrDWhG5EhhnmA6gQ fvmq97/nda000755cd76\nadded QmYM35pgHvLdKH8ssw9kJeiUY5kcjhb5h3BTiDhAgbsYYh fvmq97/nsz0wsonz\nadded QmNarBSVwzYjLeEjGMJqTNtRCYGCLGo6TJqd21hPi7WXFT fvmq97/pq3f6t0\nadded QmUNhQpFBZvfH4JyNxiE8QY31bZDpQHMmjSRRnbRZYZ3be 2ju9tn-b09\nadded QmWtZu8dv4XRK8zPmwbNjS6biqe4bGEF9J5zb51sBJCMro fvmq97\nadded QmYp7QoL8wRacLn9pJftJSkiiSmNGdWb7qT5ENDW2HXBcu '\n\nadd_w_r='QmUerh2irM8cngqJHLGKCn4AGBSyHYAUi8i8zyVzXKNYyb'\n\n. lib/test-lib.sh\n\ntest_add_w() {\n\n test_expect_success \"random-files is installed\" '\n type random-files\n '\n\n test_expect_success \"random-files generates test files\" '\n random-files --seed 7547632 --files 5 --dirs 2 --depth 3 m &&\n echo \"$add_w_m\" >expected &&\n ipfs add -Q -r m >actual &&\n test_sort_cmp expected actual\n '\n\n # test single file\n test_expect_success \"ipfs add -w (single file) succeeds\" '\n ipfs add -w m/0h0r91 >actual\n '\n\n test_expect_success \"ipfs add -w (single file) is correct\" '\n echo \"$add_w_1\" >expected &&\n test_sort_cmp expected actual\n '\n\n # test two files together\n test_expect_success \"ipfs add -w (multiple) succeeds\" '\n ipfs add -w m/0h0r91 m/951op >actual\n '\n\n test_expect_success \"ipfs add -w (multiple) is correct\" '\n echo \"$add_w_12\" >expected &&\n test_sort_cmp expected actual\n '\n\n test_expect_success \"ipfs add -w (multiple) succeeds\" '\n ipfs add -w m/951op m/0h0r91 >actual\n '\n\n test_expect_success \"ipfs add -w (multiple) orders\" '\n echo \"$add_w_12\" >expected &&\n test_sort_cmp expected actual\n '\n\n # test a directory\n test_expect_success \"ipfs add -w -r (dir) succeeds\" '\n ipfs add -r -w m/9m7mh3u51z3b/3s78oa >actual\n '\n\n test_expect_success \"ipfs add -w -r (dir) is correct\" '\n echo \"$add_w_d1\" >expected &&\n test_sort_cmp expected actual\n '\n\n # test files and directory\n test_expect_success \"ipfs add -w -r succeeds\" '\n ipfs add -w -r m/9m7mh3u51z3b/1o8ef-25onywi \\\n m/vck_-2/2ju9tn-b09 m/9m7mh3u51z3b/fvmq97 m/0h0r91 >actual\n '\n\n test_expect_success \"ipfs add -w -r is correct\" '\n echo \"$add_w_d2\" >expected &&\n test_sort_cmp expected actual\n '\n\n # test -w -r m/* == -r m\n test_expect_success \"ipfs add -w -r m/* == add -r m succeeds\" '\n ipfs add -Q -w -r m/* >actual\n '\n\n test_expect_success \"ipfs add -w -r m/* == add -r m is correct\" '\n echo \"$add_w_m\" >expected &&\n test_sort_cmp expected actual\n '\n\n # test repeats together\n test_expect_success \"ipfs add -w (repeats) succeeds\" '\n ipfs add -Q -w -r m/9m7mh3u51z3b/1o8ef-25onywi m/vck_-2/2ju9tn-b09 \\\n m/9m7mh3u51z3b/fvmq97 m/0h0r91 m/9m7mh3u51z3b m/9m7mh3u51z3b m/0h0r91 \\\n m/0h0r91 m/vck_-2/0dl083je2 \\\n m/vck_-2/2ju9tn-b09/-qw1d8j9 >actual\n '\n\n test_expect_success \"ipfs add -w (repeats) is correct\" '\n echo \"$add_w_r\" >expected &&\n test_sort_cmp expected actual\n '\n\n test_expect_success \"ipfs add -w -r (dir) --cid-version=1 succeeds\" '\n ipfs add -r -w --cid-version=1 m/9m7mh3u51z3b/fvmq97 >actual\n '\n\n test_expect_success \"ipfs add -w -r (dir) --cid-version=1 is correct\" '\n echo \"$add_w_d1_v1\" >expected &&\n test_sort_cmp expected actual\n '\n\n test_expect_success \"ipfs add -w -r -n (dir) --cid-version=1 succeeds\" '\n ipfs add -r -w -n --cid-version=1 m/9m7mh3u51z3b/fvmq97 >actual\n '\n\n test_expect_success \"ipfs add -w -r -n (dir) --cid-version=1 is correct\" '\n echo \"$add_w_d1_v1\" > expected &&\n test_sort_cmp expected actual\n '\n}\n\ntest_init_ipfs\n\ntest_add_w\n\ntest_launch_ipfs_daemon\n\ntest_add_w\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0044-add-symlink.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test add -w\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"creating files succeeds\" '\n mkdir -p files/foo &&\n mkdir -p files/bar &&\n echo \"some text\" > files/foo/baz &&\n ln -s files/foo/baz files/bar/baz &&\n ln -s files/does/not/exist files/bad\n'\n\ntest_add_symlinks() {\n test_expect_success \"ipfs add files succeeds\" '\n ipfs add -Q -r files >filehash_out\n '\n\n test_expect_success \"output looks good\" '\n echo QmWdiHKoeSW8G1u7ATCgpx4yMoUhYaJBQGkyPLkS9goYZ8 > filehash_exp &&\n test_cmp filehash_exp filehash_out\n '\n\n test_expect_success \"ipfs add --cid-version=1 files succeeds\" '\n ipfs add -Q -r --cid-version=1 files >filehash_out\n '\n\n test_expect_success \"output looks good\" '\n # note this hash implies all internal nodes are stored using CidV1\n echo bafybeibyhlx64cklod6isy3h7tsmr4qvam3ae3b74n3hfes5bythjrwyua > filehash_exp &&\n test_cmp filehash_exp filehash_out\n '\n\n test_expect_success \"adding a symlink adds the link itself\" '\n ipfs add -q files/bar/baz > goodlink_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"QmdocmZeF7qwPT9Z8SiVhMSyKA2KKoA2J7jToW6z6WBmxR\" > goodlink_exp &&\n test_cmp goodlink_exp goodlink_out\n '\n\n test_expect_success \"adding a broken symlink works\" '\n ipfs add -q files/bad > badlink_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"QmWYN8SEXCgNT2PSjB6BnxAx6NJQtazWoBkTRH9GRfPFFQ\" > badlink_exp &&\n test_cmp badlink_exp badlink_out\n '\n\n test_expect_success \"adding with symlink in middle of path is same as\\\nadding with no symlink\" '\n mkdir -p files2/a/b/c &&\n echo \"some other text\" > files2/a/b/c/foo &&\n ln -s b files2/a/d\n ipfs add -rq files2/a/b/c > no_sym &&\n ipfs add -rq files2/a/d/c > sym &&\n test_cmp no_sym sym\n '\n}\n\ntest_init_ipfs\n\ntest_add_symlinks\n\ntest_launch_ipfs_daemon\n\ntest_add_symlinks\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0045-ls.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ls command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_ls_cmd() {\n test_expect_success \"'ipfs add -r testData' succeeds\" '\n mkdir -p testData testData/d1 testData/d2 &&\n echo \"test\" >testData/f1 &&\n echo \"data\" >testData/f2 &&\n echo \"hello\" >testData/d1/a &&\n random-data -size=128 -seed=42 >testData/d1/128 &&\n echo \"world\" >testData/d2/a &&\n random-data -size=1024 -seed=42 >testData/d2/1024 &&\n echo \"badname\" >testData/d2/`echo -e \"bad\\x7fname.txt\"` &&\n ipfs add -r testData >actual_add\n '\n\n test_expect_success \"'ipfs add' output looks good\" '\n cat <<-\\EOF >expected_add &&\nadded QmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN testData/d1/128\nadded QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN testData/d1/a\nadded QmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 testData/d2/1024\nadded QmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL testData/d2/a\nadded QmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn testData/d2/bad\\x7fname.txt\nadded QmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH testData/f1\nadded QmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M testData/f2\nadded QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j testData/d1\nadded Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW testData/d2\nadded QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc testData\nEOF\n test_cmp expected_add actual_add\n '\n \n test_expect_success \"'ipfs ls ' succeeds\" '\n ipfs ls QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j >actual_ls\n '\n\n test_expect_success \"'ipfs ls ' output looks good\" '\n cat <<-\\EOF >expected_ls &&\nQmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc:\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j - d1/\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW - d2/\nQmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH 5 f1\nQmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M 5 f2\n\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW:\nQmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 1024 1024\nQmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL 6 a\nQmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn 8 bad\\x7fname.txt\n\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j:\nQmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN 128 128\nQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN 6 a\nEOF\n test_cmp expected_ls actual_ls\n '\n\n test_expect_success \"'ipfs ls --size=false ' succeeds\" '\n ipfs ls --size=false QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j >actual_ls\n '\n\n test_expect_success \"'ipfs ls ' output looks good\" '\n cat <<-\\EOF >expected_ls &&\nQmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc:\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j d1/\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW d2/\nQmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH f1\nQmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M f2\n\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW:\nQmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 1024\nQmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL a\nQmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn bad\\x7fname.txt\n\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j:\nQmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN 128\nQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN a\nEOF\n test_cmp expected_ls actual_ls\n '\n\n test_expect_success \"'ipfs ls --headers ' succeeds\" '\n ipfs ls --headers QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j >actual_ls_headers\n '\n\n test_expect_success \"'ipfs ls --headers ' output looks good\" '\n cat <<-\\EOF >expected_ls_headers &&\nQmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc:\nHash Size Name\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j - d1/\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW - d2/\nQmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH 5 f1\nQmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M 5 f2\n\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW:\nHash Size Name\nQmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 1024 1024\nQmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL 6 a\nQmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn 8 bad\\x7fname.txt\n\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j:\nHash Size Name\nQmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN 128 128\nQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN 6 a\nEOF\n test_cmp expected_ls_headers actual_ls_headers\n '\n\n test_expect_success \"'ipfs ls --size=false --cid-base=base32 ' succeeds\" '\n ipfs ls --size=false --cid-base=base32 $(cid-fmt -v 1 -b base32 %s QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j) >actual_ls_base32\n '\n\n test_expect_success \"'ipfs ls --size=false --cid-base=base32 ' output looks good\" '\n cid-fmt -b base32 -v 1 --filter %s < expected_ls > expected_ls_base32\n test_cmp expected_ls_base32 actual_ls_base32\n '\n}\n\n\ntest_ls_cmd_streaming() {\n\n test_expect_success \"'ipfs add -r testData' succeeds\" '\n mkdir -p testData testData/d1 testData/d2 &&\n echo \"test\" >testData/f1 &&\n echo \"data\" >testData/f2 &&\n echo \"hello\" >testData/d1/a &&\n random-data -size=128 -seed=42 >testData/d1/128 &&\n echo \"world\" >testData/d2/a &&\n random-data -size=1024 -seed=42 >testData/d2/1024 &&\n echo \"badname\" >testData/d2/`echo -e \"bad\\x7fname.txt\"` &&\n ipfs add -r testData >actual_add\n '\n\n test_expect_success \"'ipfs add' output looks good\" '\n cat <<-\\EOF >expected_add &&\nadded QmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN testData/d1/128\nadded QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN testData/d1/a\nadded QmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 testData/d2/1024\nadded QmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL testData/d2/a\nadded QmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn testData/d2/bad\\x7fname.txt\nadded QmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH testData/f1\nadded QmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M testData/f2\nadded QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j testData/d1\nadded Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW testData/d2\nadded QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc testData\nEOF\n test_cmp expected_add actual_add\n '\n\n test_expect_success \"'ipfs ls --stream ' succeeds\" '\n ipfs ls --stream QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j >actual_ls_stream\n '\n\n test_expect_success \"'ipfs ls --stream ' output looks good\" '\n cat <<-\\EOF >expected_ls_stream &&\nQmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc:\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j - d1/\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW - d2/\nQmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH 5 f1\nQmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M 5 f2\n\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW:\nQmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 1024 1024\nQmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL 6 a\nQmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn 8 bad\\x7fname.txt\n\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j:\nQmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN 128 128\nQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN 6 a\nEOF\n test_cmp expected_ls_stream actual_ls_stream\n '\n\n test_expect_success \"'ipfs ls --size=false --stream ' succeeds\" '\n ipfs ls --size=false --stream QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j >actual_ls_stream\n '\n\n test_expect_success \"'ipfs ls --size=false --stream ' output looks good\" '\n cat <<-\\EOF >expected_ls_stream &&\nQmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc:\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j d1/\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW d2/\nQmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH f1\nQmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M f2\n\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW:\nQmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 1024\nQmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL a\nQmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn bad\\x7fname.txt\n\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j:\nQmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN 128\nQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN a\nEOF\n test_cmp expected_ls_stream actual_ls_stream\n '\n\n test_expect_success \"'ipfs ls --stream --headers ' succeeds\" '\n ipfs ls --stream --headers QmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc Qmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW QmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j >actual_ls_stream_headers\n '\n\n test_expect_success \"'ipfs ls --stream --headers ' output looks good\" '\n cat <<-\\EOF >expected_ls_stream_headers &&\nQmR5UuxvF2ALd2GRGMCNg1GDiuuvcAyEkQaCV9fNkevWuc:\nHash Size Name\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j - d1/\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW - d2/\nQmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH 5 f1\nQmNtocSs7MoDkJMc1RkyisCSKvLadujPsfJfSdJ3e1eA1M 5 f2\n\nQmapxr4zxxUjoUFzyggydRZDkcJknjbtahYFKokbBAVghW:\nHash Size Name\nQmZHVTX2epinyx5baTFV2L2ap9VtgbmfeFdhgntAypT5N3 1024 1024\nQmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL 6 a\nQmQSLRRd1Lxn6NMsWmmj2g9W3LtSRfmVAVqU3ShneLUrbn 8 bad\\x7fname.txt\n\nQmWWEQhcLufF3qPmmbUjqH7WVWBT9JrGJwPiVTryCoBs2j:\nHash Size Name\nQmWUixdcx1VJtpuAgXAy4e3JPAbEoHE6VEDut5KcYcpuGN 128 128\nQmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN 6 a\nEOF\n test_cmp expected_ls_stream_headers actual_ls_stream_headers\n '\n}\n\ntest_ls_cmd_raw_leaves() {\n test_expect_success \"'ipfs add -r --raw-leaves' then 'ipfs ls' works as expected\" '\n mkdir -p somedir &&\n echo bar > somedir/foo &&\n ipfs add --raw-leaves -r somedir/ > /dev/null &&\n ipfs ls '$1' QmThNTdtKaVoCVrYmM5EBS6U3S5vfKFue2TxbxxAxRcKKE > ls-actual\n echo \"bafkreid5qzpjlgzem2iyzgddv7fjilipxcoxzgwazgn27q3usucn5wlxga 4 foo\" > ls-expect\n test_cmp ls-actual ls-expect\n '\n}\n\ntest_ls_object() {\n test_expect_success \"ipfs add medium size file then 'ipfs ls --size=false' works as expected\" '\n random-data -size=500000 -seed=2 > somefile &&\n HASH=$(ipfs add somefile -q) &&\n echo \"QmWJuiG6dhfwo3KXxCc9gkdizoMoXbLMCDiTTZgEhSmyyo \" > ls-expect &&\n echo \"QmNPxtpjhoXMRVKm4oSwcJaS4fck5FR4LufPd5KJr4jYhm \" >> ls-expect &&\n ipfs ls --size=false $HASH > ls-actual &&\n test_cmp ls-actual ls-expect\n '\n\n test_expect_success \"ipfs add medium size file then 'ipfs ls' works as expected\" '\n random-data -size=500000 -seed=2 > somefile &&\n HASH=$(ipfs add somefile -q) &&\n echo \"QmWJuiG6dhfwo3KXxCc9gkdizoMoXbLMCDiTTZgEhSmyyo 262144 \" > ls-expect &&\n echo \"QmNPxtpjhoXMRVKm4oSwcJaS4fck5FR4LufPd5KJr4jYhm 237856 \" >> ls-expect &&\n ipfs ls $HASH > ls-actual &&\n test_cmp ls-actual ls-expect\n '\n}\n\n# should work offline\ntest_ls_cmd\ntest_ls_cmd_streaming\ntest_ls_cmd_raw_leaves\ntest_ls_cmd_raw_leaves --size\ntest_ls_object\n\n# should work online\ntest_launch_ipfs_daemon\ntest_ls_cmd\ntest_ls_cmd_streaming\ntest_ls_cmd_raw_leaves\ntest_ls_cmd_raw_leaves --size\ntest_kill_ipfs_daemon\ntest_ls_object\n\n#\n# test for ls --resolve-type=false\n#\n\ntest_expect_success \"'ipfs add -r' succeeds\" '\n mkdir adir &&\n # note: not using a seed as the files need to have truly random content\n random-data -size=1000 > adir/file1 &&\n random-data -size=1000 > adir/file2 &&\n ipfs add --pin=false -q -r adir > adir-hashes\n'\n\ntest_expect_success \"get hashes from add output\" '\n FILE=`head -1 adir-hashes` &&\n DIR=`tail -1 adir-hashes` &&\n test \"$FILE\" -a \"$DIR\"\n'\n\ntest_expect_success \"remove a file in dir\" '\n ipfs block rm $FILE\n'\n\ntest_expect_success \"'ipfs ls --resolve-type=false ' fails\" '\n test_must_fail ipfs ls --resolve-type=false $DIR > /dev/null\n'\n\ntest_expect_success \"'ipfs ls' fails\" '\n test_must_fail ipfs ls $DIR\n'\n\ntest_expect_success \"'ipfs ls --resolve-type=true --size=false' fails\" '\n test_must_fail ipfs ls --resolve-type=true --size=false $DIR\n'\n\ntest_launch_ipfs_daemon_without_network\n\ntest_expect_success \"'ipfs ls --resolve-type=false --size=false' ok\" '\n ipfs ls --resolve-type=false --size=false $DIR > /dev/null\n'\n\ntest_expect_success \"'ipfs ls' fails\" '\n test_must_fail ipfs ls $DIR\n'\n\ntest_expect_success \"'ipfs ls --resolve-type=false --size=true' fails\" '\n test_must_fail ipfs ls --resolve-type=false --size=true $DIR\n'\n\ntest_kill_ipfs_daemon\n\ntest_launch_ipfs_daemon\n\n# now we try `ipfs ls --resolve-type=false` with the daemon online It\n# should not even attempt to retrieve the file from the network. If\n# it does it should eventually fail as the content is random and\n# should not exist on the network, but we don't want to wait for a\n# timeout so we will kill the request after a few seconds\ntest_expect_success \"'ipfs ls --resolve-type=false --size=false' ok and does not hang\" '\n go-timeout 2 ipfs ls --resolve-type=false --size=false $DIR\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0046-id-hash.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test basic operations with identity hash\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nID_HASH0=bafkqaedknncdsodknncdsnzvnbvuioak\nID_HASH0_CONTENTS=jkD98jkD975hkD8\n\ntest_expect_success \"can fetch random identity hash\" '\n ipfs cat $ID_HASH0 > expected &&\n echo $ID_HASH0_CONTENTS > actual &&\n test_cmp expected actual\n'\n\ntest_expect_success \"can pin random identity hash\" '\n ipfs pin add $ID_HASH0\n'\n\ntest_expect_success \"ipfs add succeeds with identity hash\" '\n echo \"djkd7jdkd7jkHHG\" > junk.txt &&\n HASH=$(ipfs add -q --hash=identity junk.txt)\n'\n\ntest_expect_success \"content not actually added\" '\n ipfs refs local > locals &&\n test_should_not_contain $HASH locals\n'\n\ntest_expect_success \"but can fetch it anyway\" '\n ipfs cat $HASH > actual &&\n test_cmp junk.txt actual\n'\n\ntest_expect_success \"block rm does nothing\" '\n ipfs pin rm $HASH &&\n ipfs block rm $HASH\n'\n\ntest_expect_success \"can still fetch it\" '\n ipfs cat $HASH > actual\n test_cmp junk.txt actual\n'\n\ntest_expect_success \"ipfs add --inline works as expected\" '\n echo $ID_HASH0_CONTENTS > afile &&\n HASH=$(ipfs add -q --inline afile)\n'\n\ntest_expect_success \"ipfs add --inline uses identity multihash\" '\n MHTYPE=`cid-fmt %h $HASH`\n echo \"mhtype is $MHTYPE\"\n test \"$MHTYPE\" = identity\n'\n\ntest_expect_success \"ipfs add --inline --raw-leaves works as expected\" '\n echo $ID_HASH0_CONTENTS > afile &&\n HASH=$(ipfs add -q --inline --raw-leaves afile)\n'\n\ntest_expect_success \"ipfs add --inline --raw-leaves outputs the correct hash\" '\n echo \"$ID_HASH0\" = \"$HASH\" &&\n test \"$ID_HASH0\" = \"$HASH\"\n'\n\ntest_expect_success \"create 1000 bytes file and get its hash\" '\n random-data -size=1000 -seed=2 > 1000bytes &&\n HASH0=$(ipfs add -q --raw-leaves --only-hash 1000bytes)\n'\n\ntest_expect_success \"ipfs add --inline --raw-leaves works as expected on large file\" '\n HASH=$(ipfs add -q --inline --raw-leaves 1000bytes)\n'\n\ntest_expect_success \"ipfs add --inline --raw-leaves outputs the correct hash on large file\" '\n echo \"$HASH0\" = \"$HASH\" &&\n test \"$HASH0\" = \"$HASH\"\n'\n\ntest_expect_success \"enable filestore\" '\n ipfs config --json Experimental.FilestoreEnabled true\n'\n\ntest_expect_success \"can fetch random identity hash (filestore enabled)\" '\n ipfs cat $ID_HASH0 > expected &&\n echo $ID_HASH0_CONTENTS > actual &&\n test_cmp expected actual\n'\n\ntest_expect_success \"can pin random identity hash (filestore enabled)\" '\n ipfs pin add $ID_HASH0\n'\n\ntest_expect_success \"ipfs add succeeds with identity hash and --nocopy\" '\n echo \"djkd7jdkd7jkHHG\" > junk.txt &&\n HASH=$(ipfs add -q --hash=identity --nocopy junk.txt)\n'\n\ntest_expect_success \"content not actually added (filestore enabled)\" '\n ipfs refs local > locals &&\n test_should_not_contain $HASH locals\n'\n\ntest_expect_success \"but can fetch it anyway (filestore enabled)\" '\n ipfs cat $HASH > actual &&\n test_cmp junk.txt actual\n'\n\ntest_done\n" }, { "path": "test/sharness/t0047-add-mode-mtime.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test storing and retrieving mode and mtime\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"set Import defaults to ensure deterministic cids for mod and mtime tests\" '\n ipfs config --json Import.CidVersion 0 &&\n ipfs config Import.HashFunction sha2-256 &&\n ipfs config Import.UnixFSChunker size-262144\n'\n\nHASH_NO_PRESERVE=QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH\n\nPRESERVE_MTIME=1604320482\nPRESERVE_MODE=\"0640\"\nHASH_PRESERVE_MODE=QmQLgxypSNGNFTuUPGCecq6dDEjb6hNB5xSyVmP3cEuNtq\nHASH_PRESERVE_MTIME=QmQ6kErEW8kztQFV8vbwNU8E4dmtGsYpRiboiLxUEwibvj\nHASH_PRESERVE_LINK_MTIME=QmbJwotgtr84JxcnjpwJ86uZiyMoxbZuNH4YrdJMypkYaB\nHASH_PRESERVE_MODE_AND_MTIME=QmYkvboLsvLFcSYmqVJRxvBdYRQLroLv9kELf3LRiCqBri\n\nCUSTOM_MTIME=1603539720\nCUSTOM_MTIME_NSECS=54321\nCUSTOM_MODE=\"0764\"\nHASH_CUSTOM_MODE=QmchD3BN8TQ3RW6jPLxSaNkqvfuj7syKhzTRmL4EpyY1Nz\nHASH_CUSTOM_MTIME=QmT3aY4avDcYXCWpU8CJzqUkW7YEuEsx36S8cTNoLcuK1B\nHASH_CUSTOM_MTIME_NSECS=QmaKH8H5rXBUBCX4vdxi7ktGQEL7wejV7L9rX2qpZjwncz\nHASH_CUSTOM_MODE_AND_MTIME=QmUkxrtBA8tPjwCYz1HrsoRfDz6NgKut3asVeHVQNH4C8L\nHASH_CUSTOM_LINK_MTIME=QmV1Uot2gy4bhY9yvYiZxhhchhyYC6MKKoGV1XtWNmpCLe\nHASH_CUSTOM_LINK_MTIME_NSECS=QmPHYCxYvvHj6VxiPNJ3kXxcPsnJLDYUJqsDJWjvytmrmY\n\nmk_name() {\n tr -dc '[:alnum:]'\" /ipfs/$HASH) &&\n test \"$ACTUAL\" = \"lrwxrwxrwx\"\n '\n\n\n test_expect_success \"can set symlink modification time nanoseconds [$1]\" '\n touch -h -m -t 202011021234.42 \"$TESTLINK\" &&\n HASH=$(ipfs add -q --mtime=$CUSTOM_MTIME --mtime-nsecs=$CUSTOM_MTIME_NSECS \"$TESTLINK\") &&\n test \"$HASH_CUSTOM_LINK_MTIME_NSECS\" = \"$HASH\"\n '\n\n test_expect_success \"can get preserved mode and modification time [$1]\" '\n OUTFILE=\"$(mk_file $HASH_PRESERVE_MODE_AND_MTIME)\" &&\n ipfs get -o \"$OUTFILE\" $HASH_PRESERVE_MODE_AND_MTIME &&\n test \"$PRESERVE_MODE:$PRESERVE_MTIME\" = \"$(stat -c \"0%a:%Y\" \"$OUTFILE\")\"\n '\n\n test_expect_success \"can get custom mode and modification time [$1]\" '\n OUTFILE=\"$(mk_file $HASH_CUSTOM_MODE_AND_MTIME)\" &&\n ipfs get -o \"$OUTFILE\" $HASH_CUSTOM_MODE_AND_MTIME &&\n TIMESTAMP=$(date +%s%N --date=\"$(stat -c \"%y\" \"$OUTFILE\")\") &&\n MODETIME=$(stat -c \"0%a:$TIMESTAMP\" \"$OUTFILE\") &&\n printf -v EXPECTED \"$CUSTOM_MODE:$CUSTOM_MTIME%09d\" $CUSTOM_MTIME_NSECS &&\n test \"$EXPECTED\" = \"$MODETIME\"\n '\n\n test_expect_success \"can get custom symlink modification time [$1]\" '\n OUTFILE=\"$(mk_file $HASH_CUSTOM_LINK_MTIME_NSECS)\" &&\n ipfs get -o \"$OUTFILE\" $HASH_CUSTOM_LINK_MTIME_NSECS &&\n TIMESTAMP=$(date +%s%N --date=\"$(stat -c \"%y\" \"$OUTFILE\")\") &&\n printf -v EXPECTED \"$CUSTOM_MTIME%09d\" $CUSTOM_MTIME_NSECS &&\n test \"$EXPECTED\" = \"$TIMESTAMP\"\n '\n\n test_expect_success \"can change file mode [$1]\" '\n NAME=$(mk_name) &&\n HASH=$(echo testfile | ipfs add -q --mode=0600) &&\n OUTFILE=$(mk_file \"${NAME}\") &&\n ipfs files cp \"/ipfs/$HASH\" /$NAME &&\n ipfs files chmod 444 /$NAME &&\n HASH2=$(ipfs files stat /$NAME|head -1) &&\n ipfs get -o \"$OUTFILE\" $HASH2 &&\n test $(stat -c \"%a\" \"$OUTFILE\") = 444\n '\n\n # special case, because storing mode requires dag-pb envelope\n # and when dealing with CIDv1 we can have 'raw' block instead of 'dag-pb'\n # so it needs to be converted before adding attribute\n test_expect_success \"can add file mode to cidv1 raw block [$1]\" '\n NAME=$(mk_name) &&\n HASH=$(date | ipfs add -q --cid-version 1 --raw-leaves=true) &&\n OUTFILE=$(mk_file \"${NAME}\") &&\n ipfs files cp \"/ipfs/$HASH\" /$NAME &&\n ipfs files chmod 445 /$NAME &&\n HASH2=$(ipfs files stat /$NAME|head -1) &&\n ipfs get -o \"$OUTFILE\" $HASH2 &&\n test $(stat -c \"%a\" \"$OUTFILE\") = 445\n '\n\n test_expect_success \"can change file modification time [$1]\" '\n NAME=$(mk_name) &&\n OUTFILE=\"$(mk_file \"$NAME\")\" &&\n NOW=$(date +%s) &&\n HASH=$(echo testfile | ipfs add -q --mtime=$NOW) &&\n ipfs files cp \"/ipfs/$HASH\" /$NAME &&\n sleep 1 &&\n ipfs files touch /$NAME &&\n HASH=$(ipfs files stat /$NAME|head -1) &&\n ipfs get -o \"$OUTFILE\" \"$HASH\" &&\n test $(stat -c \"%Y\" \"$OUTFILE\") -gt $NOW\n '\n\n # special case, because storing mtime requires dag-pb envelope\n # and when dealing with CIDv1 we can have 'raw' block instead of 'dag-pb'\n # so it needs to be converted to dag-pb before adding attribute\n test_expect_success \"can add file modification time to cidv1 raw block [$1]\" '\n NAME=$(mk_name) &&\n OUTFILE=\"$(mk_file \"$NAME\")\" &&\n EXPECTED=\"$CUSTOM_MTIME\" &&\n HASH=$(date | ipfs add -q --cid-version 1 --raw-leaves=true) &&\n ipfs files cp \"/ipfs/$HASH\" /$NAME &&\n ipfs files touch --mtime=$EXPECTED /$NAME &&\n test $(ipfs files stat --format=\"\" \"/$NAME\") -eq $EXPECTED &&\n HASH=$(ipfs files stat /$NAME|head -1) &&\n ipfs get -o \"$OUTFILE\" \"$HASH\" &&\n test $(stat -c \"%Y\" \"$OUTFILE\") -eq $EXPECTED\n '\n\n test_expect_success \"can change file modification time nanoseconds [$1]\" '\n NAME=$(mk_name) &&\n echo test|ipfs files write --create /$NAME &&\n EXPECTED=$(date --date=\"yesterday\" +%s) &&\n ipfs files touch --mtime=$EXPECTED --mtime-nsecs=55567 /$NAME &&\n test $(ipfs files stat --format=\"\" /$NAME) -eq $EXPECTED &&\n test $(ipfs files stat --format=\"\" /$NAME) -eq 55567\n '\n\n ## TODO: update these tests if/when symbolic links are fully supported in go-mfs\n test_expect_success \"can change symlink modification time [$1]\" '\n NAME=$(mk_name) &&\n EXPECTED=$(date +%s) &&\n ipfs files cp \"/ipfs/$HASH_PRESERVE_LINK_MTIME\" \"/$NAME\" ||\n ipfs files touch --mtime=$EXPECTED \"/$NAME\" &&\n test $(ipfs files stat --format=\"\" \"/$NAME\") -eq $EXPECTED\n '\n\n test_expect_success \"can change symlink modification time nanoseconds [$1]\" '\n NAME=$(mk_name) &&\n EXPECTED=$(date +%s) &&\n ipfs files cp \"/ipfs/$HASH_PRESERVE_LINK_MTIME\" \"/$NAME\" ||\n ipfs files touch --mtime=$EXPECTED --mtime-nsecs=938475 \"/$NAME\" &&\n test $(ipfs files stat --format=\"\" \"/$NAME\") -eq $EXPECTED &&\n test $(ipfs files stat --format=\"\" \"/$NAME\") -eq 938475\n '\n}\n\nDIR_TIME=1655158632\n\nsetup_directory() {\n\n local TESTDIR=\"$(mktemp -d -p \"$FIXTURESDIR\" \"${1}XXXXXX\")\"\n mkdir -p \"$TESTDIR\"/{dir1,dir2/sub1/sub2,dir3}\n chmod 0755 \"$TESTDIR/dir1\"\n\n touch -md @$(($DIR_TIME+10)) \"$TESTDIR/dir2/sub1/sub2/file3\"\n ln -s ../sub2/file3 \"$TESTDIR/dir2/sub1/link1\"\n touch -h -md @$(($DIR_TIME+20)) \"$TESTDIR/dir2/sub1/link1\"\n\n touch -md @$(($DIR_TIME+30)) \"$TESTDIR/dir2/sub1/sub2\"\n touch -md @$(($DIR_TIME+40)) \"$TESTDIR/dir2/sub1\"\n touch -md @$(($DIR_TIME+50)) \"$TESTDIR/dir2\"\n\n touch -md @$(($DIR_TIME+60)) \"$TESTDIR/dir3/file2\"\n touch -md @$(($DIR_TIME+70)) \"$TESTDIR/dir3\"\n\n touch -md @$(($DIR_TIME+80)) \"$TESTDIR/file1\"\n touch -md @$(($DIR_TIME+90)) \"$TESTDIR/dir1\"\n touch -md @$DIR_TIME \"$TESTDIR\"\n\n echo \"$TESTDIR\"\n}\n\ntest_directory() {\n CUSTOM_DIR_MODE=0713\n TESTDIR=$(setup_directory $1)\n TESTDIR1=\"$TESTDIR/dir1\"\n OUTDIR=\"$(mk_dir \"${1}\")\"\n HASH_DIR_ROOT=QmSioyvQuXetxg7uo8FswGn9XKKEsisDq1HTMzGyWbw2R6\n HASH_DIR1_NO_PRESERVE=QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn\n HASH_DIR1_PRESERVE_MODE=QmRviohgafvCsbkiTgfQFipbuXJ6k1YtoiaQW4quttJPKu\n HASH_DIR1_PRESERVE_MTIME=QmYMy7CZGb498QFSQBF5ZFwv1FYbrAtYZMe4VxhDXxAcvf\n HASH_DIR1_CUSTOM_MODE=QmQ1ABnw2iip7sj23EzzBZ9T77KyyfESP6SUboiXPyzNQe\n HASH_DIR1_CUSTOM_MTIME=QmfWitW6F13WHFXLbJzXRYmwrS1p4gaAJAfucUSMytRPn3\n HASH_DIR1_CUSTOM_MTIME_NSECS=QmZFdCLJay31hT3Tx1LygJ7XfiLEs3qLCXtbeBfhf38aZg\n HASH_DIR_SUB1=QmeQwX5qAX18fcPDxDdkfM6ttuFCZetF5hgeUa6ov8D5oc\n\n HASH_DIR_MODE_AND_MTIME=(\n QmRCG3Pprg4jbhfYBzVzfJVyneFHnBquPGXwvXU3jSuf5j\n QmReHCn4BSJJdtd6Le8Hd8Puai6TmgpPCYb13wyM7FD9AD\n QmSioyvQuXetxg7uo8FswGn9XKKEsisDq1HTMzGyWbw2R6\n QmTMoVgJKhPrz9DfkvT132mxyBXNae5azXQ42WbM9abdSE\n QmVzXqpuQGCAgRwEbGuE9xe8Fidi1HEXaPKsQEFEbPJW9j\n QmW6Nqy2nziduAp3UGx2a52gtSUsYzhVcZMuPdxBRnwCyP\n QmeQwX5qAX18fcPDxDdkfM6ttuFCZetF5hgeUa6ov8D5oc\n QmefofUNwC2U3Xp87rB1x8Aws6AdsDuoXR7B9u2RkEZ4dQ\n Qmeu24TFarJwLzJgMTDYDJTr4BMGnzafoSnfxov1513abW\n Qmf82bbFg2e8HmcqiewutVVw5NoMpiXZD57LpLdC1poBuH)\n HASH_DIR_CUSTOM_MODE=(\n QmNZ5cyx3f6maXkczwhh3ufjDCh9f3k9zrDhX218ZZGvoV\n QmRqtFVLkXfWJuqWtYiCPthgomo3gouno8uvMeGAyCVaWS\n QmSkrWNcyDA7s1qiT6Ps7ey4zcB7uBH3sqGcKRfW4UMKhM\n QmSkrWNcyDA7s1qiT6Ps7ey4zcB7uBH3sqGcKRfW4UMKhM\n QmSkrWNcyDA7s1qiT6Ps7ey4zcB7uBH3sqGcKRfW4UMKhM\n QmZNAZXB6JyJ1cK9h1uJEK4XDo1CKsSuHMPGUUMrzDXCQz\n QmbSz6GyS8MNR4M9xtCteuGVJQRYkCXLbW174Fdy8jtaoZ\n QmccnAQQeJGtmtgZoi3hpEmgdxbuX1ao2hQmrKmmwQnCn9\n QmeTZoiAiduFY2hXaNQP4ehiE71BrQFEnrqduBZ5ZjHuFy\n Qmf13KNurvAHUfMBhMWvZuftmUikhhGY7ohWVaBDDndFMz)\n HASH_DIR_CUSTOM_MTIME=(\n QmPCGFZ8ZFowAwfWdCeGsr9wSbGXwZiHW3bZ7XSYcc1Zby\n QmT3aY4avDcYXCWpU8CJzqUkW7YEuEsx36S8cTNoLcuK1B\n QmT3aY4avDcYXCWpU8CJzqUkW7YEuEsx36S8cTNoLcuK1B\n QmT3aY4avDcYXCWpU8CJzqUkW7YEuEsx36S8cTNoLcuK1B\n QmUGMu9epCEz5HMsuJFgpJxxt3HoahsTQcC65Jje6LNqYF\n QmXhzoPKuqmkqbyr4kJFznFRXtGwriCXKGFPr4vviyK3aV\n QmZ5wKCcL11TckypuDTKLLNFP6JMCBJRCn385XKQQ6PCLt\n Qmdw3hiAxn6R5MRkkdzLdFvZUa2WJeLCTXXCyB8byFsHSA\n QmedF4m2Y8341azfkpvaHSkxbSrZa4fo6FT25h6sRUVkpq\n QmfWitW6F13WHFXLbJzXRYmwrS1p4gaAJAfucUSMytRPn3)\n\n test_expect_success \"feature on directory has no effect when not used [$1]\" '\n HASH=$(ipfs add -qr \"$TESTDIR1\") &&\n test \"$HASH_DIR1_NO_PRESERVE\" = \"$HASH\"\n '\n\n test_expect_success \"can preserve directory mode [$1]\" '\n HASH=$(ipfs add -qr --preserve-mode \"$TESTDIR1\") &&\n test \"$HASH_DIR1_PRESERVE_MODE\" = \"$HASH\"\n '\n\n test_expect_success \"can preserve directory modification time [$1]\" '\n HASH=$(ipfs add -qr --preserve-mtime \"$TESTDIR1\") &&\n test \"$HASH_DIR1_PRESERVE_MTIME\" = \"$HASH\"\n '\n\n test_expect_success \"can set directory mode [$1]\" '\n HASH=$(ipfs add -qr --mode=$CUSTOM_DIR_MODE \"$TESTDIR1\") &&\n test \"$HASH_DIR1_CUSTOM_MODE\" = \"$HASH\"\n '\n\n test_expect_success \"can set directory modification time [$1]\" '\n HASH=$(ipfs add -qr --mtime=$CUSTOM_MTIME \"$TESTDIR1\") &&\n test \"$HASH_DIR1_CUSTOM_MTIME\" = \"$HASH\"\n '\n\n test_expect_success \"can set directory modification time nanoseconds [$1]\" '\n HASH=$(ipfs add -qr --mtime=$CUSTOM_MTIME --mtime-nsecs=$CUSTOM_MTIME_NSECS \"$TESTDIR1\") &&\n test \"$HASH_DIR1_CUSTOM_MTIME_NSECS\" = \"$HASH\"\n '\n\n test_expect_success \"can recursively preserve mode and modification time [$1]\" '\n test \"700:$DIR_TIME\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR\")\" &&\n test \"644:$((DIR_TIME+10))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1/sub2/file3\")\" &&\n test \"777:$((DIR_TIME+20))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1/link1\")\" &&\n test \"755:$((DIR_TIME+30))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1/sub2\")\" &&\n test \"755:$((DIR_TIME+40))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1\")\" &&\n test \"755:$((DIR_TIME+50))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2\")\" &&\n test \"644:$((DIR_TIME+60))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir3/file2\")\" &&\n test \"755:$((DIR_TIME+70))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir3\")\" &&\n test \"644:$((DIR_TIME+80))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/file1\")\" &&\n test \"755:$((DIR_TIME+90))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir1\")\" &&\n HASHES=($(ipfs add -qr --preserve-mode --preserve-mtime \"$TESTDIR\"|sort)) &&\n test \"${HASHES[*]}\" = \"${HASH_DIR_MODE_AND_MTIME[*]}\"\n '\n\n test_expect_success \"can recursively set directory mode [$1]\" '\n HASHES=($(ipfs add -qr --mode=0753 \"$TESTDIR\"|sort)) &&\n test \"${HASHES[*]}\" = \"${HASH_DIR_CUSTOM_MODE[*]}\"\n '\n\n test_expect_success \"can recursively set directory mtime [$1]\" '\n HASHES=($(ipfs add -qr --mtime=$CUSTOM_MTIME \"$TESTDIR\"|sort)) &&\n test \"${HASHES[*]}\" = \"${HASH_DIR_CUSTOM_MTIME[*]}\"\n '\n\n test_expect_success \"can recursively restore mode and mtime [$1]\" '\n ipfs get -o \"$OUTDIR\" $HASH_DIR_ROOT &&\n test \"700:$DIR_TIME\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR\")\" &&\n test \"644:$((DIR_TIME+10))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir2/sub1/sub2/file3\")\" &&\n test \"777:$((DIR_TIME+20))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir2/sub1/link1\")\" &&\n test \"755:$((DIR_TIME+30))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir2/sub1/sub2\")\" &&\n test \"755:$((DIR_TIME+40))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir2/sub1\")\" &&\n test \"755:$((DIR_TIME+50))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir2\")\" &&\n test \"644:$((DIR_TIME+60))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir3/file2\")\" &&\n test \"755:$((DIR_TIME+70))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir3\")\" &&\n test \"644:$((DIR_TIME+80))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/file1\")\" &&\n test \"755:$((DIR_TIME+90))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIR/dir1\")\"\n '\n\n # basic smoke-test for cidv1 (we dont care about CID, just care about\n # mode/mtime surviving ipfs import and export if --cid-version 1 is at play)\n test_expect_success \"can recursively preserve and restore mode and mtime with CIDv1 [$1]\" '\n test \"700:$DIR_TIME\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR\")\" &&\n test \"644:$((DIR_TIME+10))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1/sub2/file3\")\" &&\n test \"777:$((DIR_TIME+20))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1/link1\")\" &&\n test \"755:$((DIR_TIME+30))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1/sub2\")\" &&\n test \"755:$((DIR_TIME+40))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2/sub1\")\" &&\n test \"755:$((DIR_TIME+50))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir2\")\" &&\n test \"644:$((DIR_TIME+60))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir3/file2\")\" &&\n test \"755:$((DIR_TIME+70))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir3\")\" &&\n test \"644:$((DIR_TIME+80))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/file1\")\" &&\n test \"755:$((DIR_TIME+90))\" = \"$(stat -c \"%a:%Y\" \"$TESTDIR/dir1\")\" &&\n CIDV1DIR=$(ipfs add -Qr --preserve-mode --preserve-mtime --cid-version 1 \"$TESTDIR\") &&\n OUTDIRV1=$(mk_dir cidv1roundtrip$1) &&\n ipfs get -o \"$OUTDIRV1\" $CIDV1DIR &&\n test \"700:$DIR_TIME\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1\")\" &&\n test \"644:$((DIR_TIME+10))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir2/sub1/sub2/file3\")\" &&\n test \"777:$((DIR_TIME+20))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir2/sub1/link1\")\" &&\n test \"755:$((DIR_TIME+30))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir2/sub1/sub2\")\" &&\n test \"755:$((DIR_TIME+40))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir2/sub1\")\" &&\n test \"755:$((DIR_TIME+50))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir2\")\" &&\n test \"644:$((DIR_TIME+60))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir3/file2\")\" &&\n test \"755:$((DIR_TIME+70))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir3\")\" &&\n test \"644:$((DIR_TIME+80))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/file1\")\" &&\n test \"755:$((DIR_TIME+90))\" = \"$(stat -c \"%a:%Y\" \"$OUTDIRV1/dir1\")\"\n '\n\n test_expect_success \"can change directory mode [$1]\" '\n NAME=$(mk_name) &&\n ipfs files cp \"/ipfs/$HASH_DIR_SUB1\" /$NAME &&\n ipfs files chmod 0710 /$NAME &&\n test $(ipfs files stat --format=\"\" /$NAME) = \"drwx--x---\"\n '\n\n test_expect_success \"can change directory modification time [$1]\" '\n NAME=$(mk_name) &&\n ipfs files cp \"/ipfs/$HASH_DIR_SUB1\" /$NAME &&\n ipfs files touch --mtime=$CUSTOM_MTIME /$NAME &&\n test $(ipfs files stat --format=\"\" /$NAME) -eq $CUSTOM_MTIME\n '\n\n test_expect_success \"can change directory modification time nanoseconds [$1]\" '\n NAME=$(mk_name) &&\n MTIME=$(date --date=\"yesterday\" +%s) &&\n ipfs files cp \"/ipfs/$HASH_DIR_SUB1\" /$NAME &&\n ipfs files touch --mtime=$MTIME --mtime-nsecs=94783 /$NAME &&\n test $(ipfs files stat --format=\"\" /$NAME) -eq $MTIME &&\n test $(ipfs files stat --format=\"\" /$NAME) -eq 94783\n '\n}\n\ntest_stat_template() {\n test_expect_success \"can stat $2 string mode [$1]\" '\n touch \"$STAT_TARGET\" &&\n HASH=$(ipfs add -qr --mode=\"$STAT_MODE_OCTAL\" \"$STAT_TARGET\") &&\n ACTUAL=$(ipfs files stat --format=\"\" /ipfs/$HASH) &&\n test \"$ACTUAL\" = \"$STAT_MODE_STRING\"\n '\n test_expect_success \"can stat $2 octal mode [$1]\" '\n touch \"$STAT_TARGET\" &&\n HASH=$(ipfs add -qr --mode=\"$STAT_MODE_OCTAL\" \"$STAT_TARGET\") &&\n ACTUAL=$(ipfs files stat --format=\"\" /ipfs/$HASH) &&\n test \"$ACTUAL\" = \"$STAT_MODE_OCTAL\"\n '\n\n test_expect_success \"can stat $2 modification time string [$1]\" '\n touch \"$STAT_TARGET\" &&\n HASH=$(ipfs add -qr --mtime=$CUSTOM_MTIME \"$STAT_TARGET\") &&\n ACTUAL=$(ipfs files stat --format=\"\" /ipfs/$HASH) &&\n test \"$ACTUAL\" = \"24 Oct 2020, 11:42:00 UTC\"\n '\n\n test_expect_success \"can stat $2 modification time seconds [$1]\" '\n touch \"$STAT_TARGET\" &&\n HASH=$(ipfs add -qr --mtime=$CUSTOM_MTIME \"$STAT_TARGET\") &&\n ACTUAL=$(ipfs files stat --format=\"\" /ipfs/$HASH) &&\n test $ACTUAL -eq $CUSTOM_MTIME\n '\n\n test_expect_success \"can stat $2 modification time nanoseconds [$1]\" '\n touch \"$STAT_TARGET\" &&\n HASH=$(ipfs add -qr --mtime=$CUSTOM_MTIME --mtime-nsecs=$CUSTOM_MTIME_NSECS \"$STAT_TARGET\") &&\n ACTUAL=$(ipfs files stat --format=\"\" /ipfs/$HASH) &&\n test $ACTUAL -eq $CUSTOM_MTIME_NSECS\n '\n}\n\ntest_stat() {\n STAT_TARGET=\"$FIXTURESDIR/statfile$1\"\n STAT_MODE_OCTAL=\"$CUSTOM_MODE\"\n STAT_MODE_STRING=\"-rwxrw-r--\"\n test_stat_template \"$1\" \"file\"\n\n STAT_TARGET=\"$FIXTURESDIR/statdir$1\"\n STAT_MODE_OCTAL=\"0731\"\n STAT_MODE_STRING=\"drwx-wx--x\"\n mkdir \"$STAT_TARGET\"\n test_stat_template \"$1\" \"directory\"\n\n STAT_TARGET=\"$FIXTURESDIR/statlink$1\"\n STAT_MODE_OCTAL=\"0777\"\n STAT_MODE_STRING=\"lrwxrwxrwx\"\n ln -s nothing \"$STAT_TARGET\"\n test_stat_template \"$1\" \"link\"\n\n\n STAT_TARGET=\"$FIXTURESDIR/statfile$1\"\n test_expect_success \"can chain stat template [$1]\" '\n HASH=$(ipfs add -q --mode=0644 --mtime=$CUSTOM_MTIME --mtime-nsecs=$CUSTOM_MTIME_NSECS \"$STAT_TARGET\") &&\n ACTUAL=$(ipfs files stat --format=\" \" /ipfs/$HASH) &&\n test \"$ACTUAL\" = \"24 Oct 2020, 11:42:00 UTC 1603539720 54321 -rw-r--r-- 0644\"\n '\n}\n\ntest_all() {\ntest_stat \"$1\"\ntest_file \"$1\"\ntest_directory \"$1\"\n}\n\n# test direct\ntest_all \"direct\"\n\n# test daemon\ntest_launch_ipfs_daemon_without_network\ntest_all \"daemon\"\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0050-block-data/testPut.pb", "content": "\n\u001btest json for sharness test" }, { "path": "test/sharness/t0050-block.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test block command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nHASH=\"bafkreibmlvvgdyihetgocpof6xk64kjjzdeq2e4c7hqs3krdheosk4tgj4\"\nHASHB=\"bafkreihfsphazrk2ilejpekyltjeh5k4yvwgjuwg26ueafohqioeo3sdca\"\n\nHASHV0=\"QmRKqGMAM6EZngbpjSqrvYzq5Qd8b1bSWymjSUY9zQSNDk\"\nHASHBV0=\"QmdnpnsaEj69isdw5sNzp3h3HkaDz7xKq7BmvFFBzNr5e7\"\n\n# \"block put tests\"\n#\n\ntest_expect_success \"'ipfs block put' succeeds\" '\n echo \"Hello Mars!\" >expected_in &&\n ipfs block put expected_out &&\n test_cmp expected_out actual_out\n'\n\ntest_expect_success \"'ipfs block put' with 2 files succeeds\" '\n echo \"Hello Mars!\" > a &&\n echo \"Hello Venus!\" > b &&\n ipfs block put a b | tee actual_out\n'\n\ntest_expect_success \"'ipfs block put' output looks good\" '\n echo \"$HASH\" >expected_out &&\n echo \"$HASHB\" >>expected_out &&\n test_cmp expected_out actual_out\n'\n\ntest_expect_success \"can set cid codec on block put\" '\n CODEC_HASH=$(ipfs block put --cid-codec=dag-pb ../t0050-block-data/testPut.pb)\n'\n\ntest_expect_success \"block get output looks right\" '\n ipfs block get $CODEC_HASH > pb_block_out &&\n test_cmp pb_block_out ../t0050-block-data/testPut.pb\n'\n\n#\n# \"block get\" tests\n#\n\ntest_expect_success \"'ipfs block get' succeeds\" '\n ipfs block get $HASH >actual_in\n'\n\ntest_expect_success \"'ipfs block get' output looks good\" '\n test_cmp expected_in actual_in\n'\n\n#\n# \"block stat\" tests\n#\n\ntest_expect_success \"'ipfs block stat' succeeds\" '\n ipfs block stat $HASH >actual_stat\n'\n\ntest_expect_success \"'ipfs block stat' output looks good\" '\n echo \"Key: $HASH\" >expected_stat &&\n echo \"Size: 12\" >>expected_stat &&\n test_cmp expected_stat actual_stat\n'\n\n#\n# \"block rm\" tests\n#\n\ntest_expect_success \"'ipfs block rm' succeeds\" '\n ipfs block rm $HASH >actual_rm\n'\n\ntest_expect_success \"'ipfs block rm' output looks good\" '\n echo \"removed $HASH\" > expected_rm &&\n test_cmp expected_rm actual_rm\n'\n\ntest_expect_success \"'ipfs block rm' block actually removed\" '\n test_must_fail ipfs block stat $HASH\n'\n\nRANDOMHASH=QmRKqGMAM6EbngbZjSqrvYzq5Qd8b1bSWymjSUY9zQSNDq\nDIRHASH=QmdWmVmM6W2abTgkEfpbtA1CJyTWS2rhuUB9uP1xV8Uwtf\nFILE1HASH=Qmae3RedM7SNkWGsdzYzsr6svmsFdsva4WoTvYYsWhUSVz\nFILE2HASH=QmUtkGLvPf63NwVzLPKPUYgwhn8ZYPWF6vKWN3fZ2amfJF\nFILE3HASH=Qmesmmf1EEG1orJb6XdK6DabxexsseJnCfw8pqWgonbkoj\nTESTHASH=QmeomffUNfmQy76CQGy9NdmqEnnHU9soCexBnGU3ezPHVH\n\ntest_expect_success \"add and pin directory\" '\n echo \"test\" | ipfs add --pin=false &&\n mkdir adir &&\n echo \"file1\" > adir/file1 &&\n echo \"file2\" > adir/file2 &&\n echo \"file3\" > adir/file3 &&\n ipfs add -r adir\n ipfs pin add -r $DIRHASH\n'\n\ntest_expect_success \"can't remove pinned block\" '\n test_must_fail ipfs block rm $DIRHASH 2> block_rm_err\n'\n\ntest_expect_success \"can't remove pinned block: output looks good\" '\n grep -q \"$DIRHASH: pinned: recursive\" block_rm_err\n'\n\ntest_expect_success \"can't remove indirectly pinned block\" '\n test_must_fail ipfs block rm $FILE1HASH 2> block_rm_err\n'\n\ntest_expect_success \"can't remove indirectly pinned block: output looks good\" '\n grep -q \"$FILE1HASH: pinned via $DIRHASH\" block_rm_err\n'\n\ntest_expect_success \"remove pin\" '\n ipfs pin rm -r $DIRHASH\n'\n\ntest_expect_success \"multi-block 'ipfs block rm' succeeds\" '\n ipfs block rm $FILE1HASH $FILE2HASH $FILE3HASH > actual_rm\n'\n\ntest_expect_success \"multi-block 'ipfs block rm' output looks good\" '\n grep -F -q \"removed $FILE1HASH\" actual_rm &&\n grep -F -q \"removed $FILE2HASH\" actual_rm &&\n grep -F -q \"removed $FILE3HASH\" actual_rm\n'\n\ntest_expect_success \"multi-block 'ipfs block rm '\" '\n test_must_fail ipfs block rm $RANDOMHASH $TESTHASH $RANDOMHASH &> actual_mixed_rm\n'\n\ntest_expect_success \"multi-block 'ipfs block rm ' output looks good\" '\n echo \"cannot remove $RANDOMHASH: ipld: could not find $RANDOMHASH\" >> expect_mixed_rm &&\n echo \"removed $TESTHASH\" >> expect_mixed_rm &&\n echo \"cannot remove $RANDOMHASH: ipld: could not find $RANDOMHASH\" >> expect_mixed_rm &&\n echo \"Error: some blocks not removed\" >> expect_mixed_rm\n test_cmp actual_mixed_rm expect_mixed_rm\n'\n\ntest_expect_success \"'add some blocks' succeeds\" '\n echo \"Hello Mars!\" | ipfs block put &&\n echo \"Hello Venus!\" | ipfs block put\n'\n\ntest_expect_success \"add and pin directory\" '\n ipfs add -r adir\n ipfs pin add -r $DIRHASH\n'\n\nHASH=QmRKqGMAM6EZngbpjSqrvYzq5Qd8b1bSWymjSUY9zQSNDk\nHASH2=QmdnpnsaEj69isdw5sNzp3h3HkaDz7xKq7BmvFFBzNr5e7\n\ntest_expect_success \"multi-block 'ipfs block rm' mixed\" '\n test_must_fail ipfs block rm $FILE1HASH $DIRHASH $HASH $FILE3HASH $RANDOMHASH $HASH2 2> block_rm_err\n'\n\ntest_expect_success \"pinned block not removed\" '\n ipfs block stat $FILE1HASH &&\n ipfs block stat $FILE3HASH\n'\n\ntest_expect_success \"non-pinned blocks removed\" '\n test_must_fail ipfs block stat $HASH &&\n test_must_fail ipfs block stat $HASH2\n'\n\ntest_expect_success \"error reported on removing non-existent block\" '\n grep -q \"cannot remove $RANDOMHASH\" block_rm_err\n'\n\ntest_expect_success \"'add some blocks' succeeds\" '\n echo \"Hello Mars!\" | ipfs block put &&\n echo \"Hello Venus!\" | ipfs block put\n'\n\ntest_expect_success \"multi-block 'ipfs block rm -f' with non existent blocks succeed\" '\n ipfs block rm -f $HASH $RANDOMHASH $HASH2\n'\n\ntest_expect_success \"existent blocks removed\" '\n test_must_fail ipfs block stat $HASH &&\n test_must_fail ipfs block stat $HASH2\n'\n\ntest_expect_success \"'add some blocks' succeeds\" '\n echo \"Hello Mars!\" | ipfs block put &&\n echo \"Hello Venus!\" | ipfs block put\n'\n\ntest_expect_success \"multi-block 'ipfs block rm -q' produces no output\" '\n ipfs block rm -q $HASH $HASH2 > block_rm_out &&\n test ! -s block_rm_out\n'\n\n# --format used 'protobuf' for 'dag-pb' which was invalid, but we keep\n# for backward-compatibility\ntest_expect_success \"can set deprecated --format=protobuf on block put\" '\n HASH=$(ipfs block put --format=protobuf ../t0050-block-data/testPut.pb)\n'\n\ntest_expect_success \"created an object correctly!\" '\n ipfs dag get $HASH > obj_out &&\n echo -n \"{\\\"Data\\\":{\\\"/\\\":{\\\"bytes\\\":\\\"dGVzdCBqc29uIGZvciBzaGFybmVzcyB0ZXN0\\\"}},\\\"Links\\\":[]}\" > obj_exp &&\n test_cmp obj_out obj_exp\n'\n\ntest_expect_success \"block get output looks right\" '\n ipfs block get $HASH > pb_block_out &&\n test_cmp pb_block_out ../t0050-block-data/testPut.pb\n'\n\ntest_expect_success \"can set --cid-codec=dag-pb on block put\" '\n HASH=$(ipfs block put --cid-codec=dag-pb ../t0050-block-data/testPut.pb)\n'\n\ntest_expect_success \"created an object correctly!\" '\n ipfs dag get $HASH > obj_out &&\n echo -n \"{\\\"Data\\\":{\\\"/\\\":{\\\"bytes\\\":\\\"dGVzdCBqc29uIGZvciBzaGFybmVzcyB0ZXN0\\\"}},\\\"Links\\\":[]}\" > obj_exp &&\n test_cmp obj_out obj_exp\n'\n\ntest_expect_success \"block get output looks right\" '\n ipfs block get $HASH > pb_block_out &&\n test_cmp pb_block_out ../t0050-block-data/testPut.pb\n'\n\ntest_expect_success \"can set multihash type and length on block put with --format=raw (deprecated)\" '\n HASH=$(echo \"foooo\" | ipfs block put --format=raw --mhtype=sha3 --mhlen=20)\n'\n\ntest_expect_success \"output looks good\" '\n test \"bafkrifctrq4xazzixy2v4ezymjcvzpskqdwlxra\" = \"$HASH\"\n'\n\ntest_expect_success \"can't use both legacy format and custom cid-codec at the same time\" '\n test_expect_code 1 ipfs block put --format=dag-cbor --cid-codec=dag-json < ../t0050-block-data/testPut.pb 2> output &&\n test_should_contain \"unable to use \\\"format\\\" (deprecated) and a custom \\\"cid-codec\\\" at the same time\" output\n'\n\ntest_expect_success \"can read block with different hash\" '\n ipfs block get $HASH > blk_get_out &&\n echo \"foooo\" > blk_get_exp &&\n test_cmp blk_get_exp blk_get_out\n'\n#\n# Misc tests\n#\n\ntest_expect_success \"'ipfs block stat' with nothing from stdin doesn't crash\" '\n test_expect_code 1 ipfs block stat < /dev/null 2> stat_out\n'\n\n# lol\ntest_expect_success \"no panic in output\" '\n test_expect_code 1 grep \"panic\" stat_out\n'\n\ntest_expect_success \"can set multihash type and length on block put without format or cid-codec\" '\n HASH=$(echo \"foooo\" | ipfs block put --mhtype=sha3 --mhlen=20)\n'\n\ntest_expect_success \"output looks good\" '\n test \"bafkrifctrq4xazzixy2v4ezymjcvzpskqdwlxra\" = \"$HASH\"\n'\n\ntest_expect_success \"can set multihash type and length on block put with cid-codec=dag-pb\" '\n HASH=$(echo \"foooo\" | ipfs block put --mhtype=sha3 --mhlen=20 --cid-codec=dag-pb)\n'\n\ntest_expect_success \"output looks good\" '\n test \"bafybifctrq4xazzixy2v4ezymjcvzpskqdwlxra\" = \"$HASH\"\n'\n\ntest_expect_success \"put with sha3 and cidv0 fails\" '\n echo \"foooo\" | test_must_fail ipfs block put --mhtype=sha3 --mhlen=20 --format=v0\n'\n\ntest_expect_success \"'ipfs block put' check block size\" '\n dd if=/dev/zero bs=2097153 count=1 > over-2MiB-file &&\n test_expect_code 1 ipfs block put over-2MiB-file >block_put_out 2>&1\n '\n\n test_expect_success \"ipfs block put output has the correct error\" '\n grep \"produced block is over 2MiB\" block_put_out\n '\n\n test_expect_success \"ipfs block put --allow-big-block=true works\" '\n test_expect_code 0 ipfs block put over-2MiB-file --allow-big-block=true &&\n rm over-2MiB-file\n '\n\ntest_done\n" }, { "path": "test/sharness/t0051-object.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Henry Bubert\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test object command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_patch_create_path() {\n root=$1\n name=$2\n target=$3\n\n test_expect_success \"object patch --create works\" '\n PCOUT=$(ipfs object patch $root add-link --create $name $target)\n '\n\n test_expect_success \"output looks good\" '\n ipfs cat \"$PCOUT/$name\" >tpcp_out &&\n ipfs cat \"$target\" >tpcp_exp &&\n test_cmp tpcp_exp tpcp_out\n '\n}\n\ntest_object_cmd() {\n EMPTY_DIR=$(echo '{\"Links\":[]}' | ipfs dag put --store-codec dag-pb)\n EMPTY_UNIXFS_DIR=$(echo '{\"Data\":{\"/\":{\"bytes\":\"CAE\"}},\"Links\":[]}' | ipfs dag put --store-codec dag-pb)\n\n test_expect_success \"'ipfs object patch' should work (no unixfs-dir)\" '\n OUTPUT=$(ipfs object patch $EMPTY_DIR add-link foo $EMPTY_DIR) &&\n ipfs dag stat $OUTPUT\n '\n\n test_expect_success \"'ipfs object patch' should work\" '\n OUTPUT=$(ipfs object patch $EMPTY_UNIXFS_DIR add-link foo $EMPTY_UNIXFS_DIR) &&\n ipfs dag stat $OUTPUT\n '\n\n test_expect_success \"'ipfs object patch' check output block size\" '\n DIR=$EMPTY_UNIXFS_DIR\n for i in {1..14}\n do\n DIR=$(ipfs object patch \"$DIR\" add-link \"$DIR.jpg\" \"$DIR\")\n done\n # Fail when new block goes over the BS limit of 2MiB, but allow manual override\n test_expect_code 1 ipfs object patch \"$DIR\" add-link \"$DIR.jpg\" \"$DIR\" >patch_out 2>&1\n '\n\n test_expect_success \"ipfs object patch add-link output has the correct error\" '\n grep \"produced block is over 2MiB\" patch_out\n '\n\n test_expect_success \"ipfs object patch --allow-big-block=true add-link works\" '\n test_expect_code 0 ipfs object patch --allow-big-block=true \"$DIR\" add-link \"$DIR.jpg\" \"$DIR\"\n '\n\n test_expect_success \"'ipfs object patch add-link' should work with paths\" '\n N1=$(ipfs object patch $EMPTY_UNIXFS_DIR add-link baz $EMPTY_UNIXFS_DIR) &&\n N2=$(ipfs object patch $EMPTY_UNIXFS_DIR add-link bar $N1) &&\n N3=$(ipfs object patch $EMPTY_UNIXFS_DIR add-link foo /ipfs/$N2/bar) &&\n ipfs dag stat /ipfs/$N3 > /dev/null &&\n ipfs dag stat $N3/foo > /dev/null &&\n ipfs dag stat /ipfs/$N3/foo/baz > /dev/null\n '\n\n test_expect_success \"'ipfs object patch add-link' allow linking IPLD objects\" '\n OBJ=$(echo \"123\" | ipfs dag put) &&\n N1=$(ipfs object patch $EMPTY_UNIXFS_DIR add-link foo $OBJ) &&\n\n ipfs dag stat /ipfs/$N1 > /dev/null &&\n ipfs resolve /ipfs/$N1/foo > actual &&\n echo /ipfs/$OBJ > expected &&\n\n test_cmp expected actual\n '\n\n test_expect_success \"object patch creation looks right\" '\n echo \"bafybeiakusqwohnt7bs75kx6jhmt4oi47l634bmudxfv4qxhpco6xuvgna\" > hash_exp &&\n echo $N3 > hash_actual &&\n test_cmp hash_exp hash_actual\n '\n\n test_expect_success \"multilayer ipfs patch works\" '\n echo \"hello world\" > hwfile &&\n FILE=$(ipfs add -q hwfile) &&\n EMPTY=$EMPTY_UNIXFS_DIR &&\n ONE=$(ipfs object patch $EMPTY add-link b $EMPTY) &&\n TWO=$(ipfs object patch $EMPTY add-link a $ONE) &&\n ipfs object patch $TWO add-link a/b/c $FILE > multi_patch\n '\n\n test_expect_success \"output looks good\" '\n ipfs cat $(cat multi_patch)/a/b/c > hwfile_out &&\n test_cmp hwfile hwfile_out\n '\n\n test_expect_success \"can remove the directory\" '\n ipfs object patch $OUTPUT rm-link foo > rmlink_output\n '\n\n test_expect_success \"output should be empty\" '\n echo bafybeiczsscdsbs7ffqz55asqdf3smv6klcw3gofszvwlyarci47bgf354 > rmlink_exp &&\n test_cmp rmlink_exp rmlink_output\n '\n\n test_expect_success \"multilayer rm-link should work\" '\n ipfs object patch $(cat multi_patch) rm-link a/b/c > multi_link_rm_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"bafybeicourxysmtbe5hacxqico4d5hyvh7gqkrwlmqa4ew7zufn3pj3juu\" > multi_link_rm_exp &&\n test_cmp multi_link_rm_exp multi_link_rm_out\n '\n\n test_patch_create_path $EMPTY a/b/c $FILE\n\n test_patch_create_path $EMPTY a $FILE\n\n test_patch_create_path $EMPTY a/b/b/b/b $FILE\n\n test_expect_success \"can create blank object\" '\n BLANK=$EMPTY_DIR\n '\n\n test_patch_create_path $BLANK a $FILE\n\n test_expect_success \"create bad path fails\" '\n test_must_fail ipfs object patch $EMPTY add-link --create / $FILE\n '\n}\n\n# should work offline\ntest_object_cmd\n\n# should work online\ntest_launch_ipfs_daemon\ntest_object_cmd\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0052-object-diff.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test object diff command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"create some objects for testing diffs\" '\n mkdir foo &&\n echo \"stuff\" > foo/bar &&\n mkdir foo/baz &&\n A=$(ipfs add -r -Q foo) &&\n AR=$(ipfs add --raw-leaves -r -Q foo) &&\n echo \"more things\" > foo/cat &&\n B=$(ipfs add -r -Q foo) &&\n BR=$(ipfs add --raw-leaves -r -Q foo) &&\n echo \"nested\" > foo/baz/dog &&\n C=$(ipfs add -r -Q foo)\n CR=$(ipfs add --raw-leaves -r -Q foo)\n echo \"changed\" > foo/bar &&\n D=$(ipfs add -r -Q foo) &&\n DR=$(ipfs add --raw-leaves -r -Q foo) &&\n echo \"\" > single_file &&\n SINGLE_FILE=$(ipfs add -r -Q single_file) &&\n SINGLE_FILE_RAW=$(ipfs add --raw-leaves -r -Q single_file) &&\n mkdir empty_dir\n EMPTY_DIR=$(ipfs add -r -Q empty_dir)\n EMPTY_DIR_RAW=$(ipfs add --raw-leaves -r -Q empty_dir)\n'\n\ntest_expect_success \"diff against self is empty\" '\n ipfs object diff $A $A > diff_out\n'\n\ntest_expect_success \"identity diff output looks good\" '\n printf \"\" > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) against self is empty\" '\n ipfs object diff $AR $AR > diff_raw_out\n'\n\ntest_expect_success \"identity diff (raw-leaves) output looks good\" '\n printf \"\" > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"diff against self (single file) is empty\" '\n ipfs object diff $SINGLE_FILE $SINGLE_FILE > diff_out &&\n printf \"\" > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) against self (single file) is empty\" '\n ipfs object diff $SINGLE_FILE_RAW $SINGLE_FILE_RAW > diff_raw_out &&\n printf \"\" > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"diff against self (empty dir) is empty\" '\n ipfs object diff $EMPTY_DIR $EMPTY_DIR > diff_out &&\n printf \"\" > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) against self (empty dir) is empty\" '\n ipfs object diff $EMPTY_DIR_RAW $EMPTY_DIR_RAW > diff_raw_out &&\n printf \"\" > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"diff added link works\" '\n ipfs object diff $A $B > diff_out\n'\n\ntest_expect_success \"diff added link looks right\" '\n echo + QmUSvcqzhdfYM1KLDbM76eLPdS9ANFtkJvFuPYeZt73d7A \\\"cat\\\" > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) added link works\" '\n ipfs object diff $AR $BR > diff_raw_out\n'\n\ntest_expect_success \"diff (raw-leaves) added link looks right\" '\n echo + bafkreig43bpnc6sjo6izaiqzzq5esapazosa3f3wt6jsflwiu3x7ydhq2u \\\"cat\\\" > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"verbose diff added link works\" '\n ipfs object diff -v $A $B > diff_out\n'\n\ntest_expect_success \"verbose diff added link looks right\" '\n echo Added new link \\\"cat\\\" pointing to QmUSvcqzhdfYM1KLDbM76eLPdS9ANFtkJvFuPYeZt73d7A. > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"verbose diff (raw-leaves) added link works\" '\n ipfs object diff -v $AR $BR > diff_raw_out\n'\n\ntest_expect_success \"verbose diff (raw-leaves) added link looks right\" '\n echo Added new link \\\"cat\\\" pointing to bafkreig43bpnc6sjo6izaiqzzq5esapazosa3f3wt6jsflwiu3x7ydhq2u. > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"diff removed link works\" '\n ipfs object diff -v $B $A > diff_out\n'\n\ntest_expect_success \"diff removed link looks right\" '\n echo Removed link \\\"cat\\\" \\(was QmUSvcqzhdfYM1KLDbM76eLPdS9ANFtkJvFuPYeZt73d7A\\). > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) removed link works\" '\n ipfs object diff -v $BR $AR > diff_raw_out\n'\n\ntest_expect_success \"diff (raw-leaves) removed link looks right\" '\n echo Removed link \\\"cat\\\" \\(was bafkreig43bpnc6sjo6izaiqzzq5esapazosa3f3wt6jsflwiu3x7ydhq2u\\). > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"diff nested add works\" '\n ipfs object diff -v $B $C > diff_out\n'\n\ntest_expect_success \"diff looks right\" '\n echo Added new link \\\"baz/dog\\\" pointing to QmdNJQUTZuDpsUcec7YDuCfRfvw1w4J13DCm7YcU4VMZdS. > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) nested add works\" '\n ipfs object diff -v $BR $CR > diff_raw_out\n'\n\ntest_expect_success \"diff (raw-leaves) looks right\" '\n echo Added new link \\\"baz/dog\\\" pointing to bafkreibxbkgajofglo2esqtv53bcp4nwstnqjr3nu2ylrlui5unldf4qum. > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_expect_success \"diff changed link works\" '\n ipfs object diff -v $C $D > diff_out\n'\n\ntest_expect_success \"diff looks right\" '\n echo Changed \\\"bar\\\" from QmNgd5cz2jNftnAHBhcRUGdtiaMzb5Rhjqd4etondHHST8 to QmRfFVsjSXkhFxrfWnLpMae2M4GBVsry6VAuYYcji5MiZb. > diff_exp &&\n test_cmp diff_exp diff_out\n'\n\ntest_expect_success \"diff (raw-leaves) changed link works\" '\n ipfs object diff -v $CR $DR > diff_raw_out\n'\n\ntest_expect_success \"diff（raw-leaves) looks right\" '\n echo Changed \\\"bar\\\" from bafkreidfn2oemjv5ns2fnc4ukgbjwt6bq5gdd4ciz4mpnehqi2dvwxfbde to bafkreid7rmo7yrtlmje7a3f6kxerotpsk6hhovg2pe755use55olukry6e. > diff_raw_exp &&\n test_cmp diff_raw_exp diff_raw_out\n'\n\ntest_done\n" }, { "path": "test/sharness/t0053-dag.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test dag command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"make a few test files\" '\n echo \"foo\" > file1 &&\n echo \"bar\" > file2 &&\n echo \"baz\" > file3 &&\n echo \"qux\" > file4 &&\n HASH1=$(ipfs add --pin=false -q file1) &&\n HASH2=$(ipfs add --pin=false -q file2) &&\n HASH3=$(ipfs add --pin=false -q file3) &&\n HASH4=$(ipfs add --pin=false -q file4)\n'\n\ntest_expect_success \"make an ipld object in dag-json\" '\n printf \"{\\\"hello\\\":\\\"world\\\",\\\"cats\\\":[{\\\"/\\\":\\\"%s\\\"},{\\\"water\\\":{\\\"/\\\":\\\"%s\\\"}}],\\\"magic\\\":{\\\"/\\\":\\\"%s\\\"},\\\"sub\\\":{\\\"dict\\\":\\\"ionary\\\",\\\"beep\\\":[0,\\\"bop\\\"]}}\" $HASH1 $HASH2 $HASH3 > ipld_object\n'\n\n# This data is in https://github.com/ipld/codec-fixtures/tree/master/fixtures/dagpb_Data_some\ntest_expect_success \"make the same ipld object in dag-cbor, dag-json and dag-pb\" '\n echo \"omREYXRhRQABAgMEZUxpbmtzgA==\" | base64 -d > ipld_object_dagcbor\n echo \"CgUAAQIDBA==\" | base64 -d > ipld_object_dagpb\n echo \"{\\\"Data\\\":{\\\"/\\\":{\\\"bytes\\\":\\\"AAECAwQ\\\"}},\\\"Links\\\":[]}\" > ipld_object_dagjson\n'\n\ntest_dag_cmd() {\n # Working with a plain IPLD hello-world object that's dag-json and dag-cbor compatible\n\n test_expect_success \"can add an ipld object using defaults (dag-json to dag-cbor)\" '\n IPLDHASH=$(cat ipld_object | ipfs dag put)\n '\n\n test_expect_success \"CID looks correct\" '\n EXPHASH=\"bafyreiblwimnjbqcdoeafiobk6q27jcw64ew7n2fmmhdpldd63edmjecde\"\n test $EXPHASH = $IPLDHASH\n '\n\ntest_expect_success \"'ipfs dag put' check block size\" '\n dd if=/dev/zero bs=2097153 count=1 > over-2MiB-file &&\n test_expect_code 1 ipfs dag put --input-codec=raw --store-codec=raw over-2MiB-file >dag_put_out 2>&1\n '\n\n test_expect_success \"ipfs dag put output has the correct error\" '\n grep \"produced block is over 2MiB\" dag_put_out\n '\n\n test_expect_success \"ipfs dag put --allow-big-block=true works\" '\n test_expect_code 0 ipfs dag put --input-codec=raw --store-codec=raw over-2MiB-file --allow-big-block=true &&\n rm over-2MiB-file\n '\n\n test_expect_success \"can add an ipld object using dag-json to dag-json\" '\n IPLDHASH=$(cat ipld_object | ipfs dag put --input-codec dag-json --store-codec dag-json)\n '\n\n test_expect_success \"CID looks correct\" '\n EXPHASH=\"baguqeera6gviseelmbzn2ugoddo5vulxlshqs3kw5ymgsb6w4cabnoh4ldpa\"\n test $EXPHASH = $IPLDHASH\n '\n\n test_expect_success \"can add an ipld object using dag-json to dag-cbor\" '\n IPLDHASH=$(cat ipld_object | ipfs dag put --input-codec dag-json --store-codec dag-cbor)\n '\n\n test_expect_success \"CID looks correct\" '\n EXPHASH=\"bafyreiblwimnjbqcdoeafiobk6q27jcw64ew7n2fmmhdpldd63edmjecde\"\n test $EXPHASH = $IPLDHASH\n '\n\n test_expect_success \"can add an ipld object using cid-base=base58btc\" '\n IPLDb58HASH=$(cat ipld_object | ipfs dag put -cid-base=base58btc)\n '\n\n test_expect_success \"CID looks correct\" '\n EXPHASH=\"zdpuAoN1XJ3GsrxEzMuCbRKZzRUVJekJUCbPVgCgE4D9yYqVi\"\n test $EXPHASH = $IPLDb58HASH\n '\n\n # Same object, different forms\n # (1) dag-cbor input\n\n test_expect_success \"can add a dag-cbor input block stored as dag-cbor\" '\n IPLDCBORHASH=$(cat ipld_object_dagcbor | ipfs dag put --input-codec dag-cbor --store-codec dag-cbor)\n '\n\n test_expect_success \"dag-cbor CID looks correct\" '\n EXPHASH=\"bafyreieculsmrexh3ty5jentbvuku452o27mst4h2tq2rb2zntqhgcstji\"\n test $EXPHASH = $IPLDCBORHASH\n '\n\n test_expect_success \"can add a dag-cbor input block stored as dag-pb\" '\n IPLDPBHASH=$(cat ipld_object_dagcbor | ipfs dag put --input-codec dag-cbor --store-codec dag-pb)\n '\n\n test_expect_success \"dag-pb CID looks correct\" '\n EXPHASH=\"bafybeibazl2z4vqp2tmwcfag6wirmtpnomxknqcgrauj7m2yisrz3qjbom\"\n test $EXPHASH = $IPLDPBHASH\n '\n\n test_expect_success \"can add a dag-cbor input block stored as dag-json\" '\n IPLDJSONHASH=$(cat ipld_object_dagcbor | ipfs dag put --input-codec dag-cbor --store-codec dag-json)\n '\n\n test_expect_success \"dag-json CID looks correct\" '\n EXPHASH=\"baguqeerajwksxu3lxpomdwxvosl542zl3xknhjgxtq3277gafrhl6vdw5tcq\"\n test $EXPHASH = $IPLDJSONHASH\n '\n\n # (2) dag-json input\n\n test_expect_success \"can add a dag-json input block stored as dag-cbor\" '\n IPLDCBORHASH=$(cat ipld_object_dagjson | ipfs dag put --input-codec dag-json --store-codec dag-cbor)\n '\n\n test_expect_success \"dag-cbor CID looks correct\" '\n EXPHASH=\"bafyreieculsmrexh3ty5jentbvuku452o27mst4h2tq2rb2zntqhgcstji\"\n test $EXPHASH = $IPLDCBORHASH\n '\n\n test_expect_success \"can add a dag-json input block stored as dag-pb\" '\n IPLDPBHASH=$(cat ipld_object_dagjson | ipfs dag put --input-codec dag-json --store-codec dag-pb)\n '\n\n test_expect_success \"dag-pb CID looks correct\" '\n EXPHASH=\"bafybeibazl2z4vqp2tmwcfag6wirmtpnomxknqcgrauj7m2yisrz3qjbom\"\n test $EXPHASH = $IPLDPBHASH\n '\n\n test_expect_success \"can add a dag-json input block stored as dag-json\" '\n IPLDJSONHASH=$(cat ipld_object_dagjson | ipfs dag put --input-codec dag-json --store-codec dag-json)\n '\n\n test_expect_success \"dag-json CID looks correct\" '\n EXPHASH=\"baguqeerajwksxu3lxpomdwxvosl542zl3xknhjgxtq3277gafrhl6vdw5tcq\"\n test $EXPHASH = $IPLDJSONHASH\n '\n\n # (3) dag-pb input\n\n test_expect_success \"can add a dag-pb input block stored as dag-cbor\" '\n IPLDCBORHASH=$(cat ipld_object_dagpb | ipfs dag put --input-codec dag-pb --store-codec dag-cbor)\n '\n\n test_expect_success \"dag-cbor CID looks correct\" '\n EXPHASH=\"bafyreieculsmrexh3ty5jentbvuku452o27mst4h2tq2rb2zntqhgcstji\"\n test $EXPHASH = $IPLDCBORHASH\n '\n\n test_expect_success \"can add a dag-pb input block stored as dag-pb\" '\n IPLDPBHASH=$(cat ipld_object_dagpb | ipfs dag put --input-codec dag-pb --store-codec dag-pb)\n '\n\n test_expect_success \"dag-pb CID looks correct\" '\n EXPHASH=\"bafybeibazl2z4vqp2tmwcfag6wirmtpnomxknqcgrauj7m2yisrz3qjbom\"\n test $EXPHASH = $IPLDPBHASH\n '\n\n test_expect_success \"can add a dag-pb input block stored as dag-json\" '\n IPLDJSONHASH=$(cat ipld_object_dagpb | ipfs dag put --input-codec dag-pb --store-codec dag-json)\n '\n\n test_expect_success \"dag-json CID looks correct\" '\n EXPHASH=\"baguqeerajwksxu3lxpomdwxvosl542zl3xknhjgxtq3277gafrhl6vdw5tcq\"\n test $EXPHASH = $IPLDJSONHASH\n '\n\n test_expect_success \"can get dag-cbor, dag-json, dag-pb blocks as dag-json\" '\n ipfs dag get $IPLDCBORHASH >& dag-get-cbor &&\n ipfs dag get $IPLDJSONHASH >& dag-get-json &&\n ipfs dag get $IPLDPBHASH >& dag-get-pb\n '\n\n test_expect_success \"can get dag-pb block transcoded as dag-cbor\" '\n ipfs dag get --output-codec=dag-cbor $IPLDPBHASH >& dag-get-dagpb-transcoded-to-dagcbor &&\n echo \"122082a2e4c892e7dcf1d491b30d68aa73ba76bec94f87d4e1a887596ce0730a534a\" >sha2_dagpb_to_dagcbor_expected &&\n multihash -a=sha2-256 -e=hex dag-get-dagpb-transcoded-to-dagcbor >sha2_dagpb_to_dagcbor_actual &&\n test_cmp sha2_dagpb_to_dagcbor_expected sha2_dagpb_to_dagcbor_actual\n '\n\n test_expect_success \"dag put and dag get transcodings match\" '\n ROUNDTRIPDAGCBOR=$(ipfs dag put --input-codec=dag-cbor --store-codec=dag-cbor dag-get-dagpb-transcoded-to-dagcbor) &&\n test $ROUNDTRIPDAGCBOR = $IPLDCBORHASH\n '\n\n # this doesn't tell us if they are correct, we test that better below\n test_expect_success \"outputs are the same\" '\n test_cmp dag-get-cbor dag-get-json &&\n test_cmp dag-get-cbor dag-get-pb\n '\n\n # Traversals using the original hello-world object\n\n test_expect_success \"various path traversals work\" '\n ipfs cat $IPLDHASH/cats/0 > out1 &&\n ipfs cat $IPLDHASH/cats/1/water > out2 &&\n ipfs cat $IPLDHASH/magic > out3\n '\n\n test_expect_success \"outputs look correct\" '\n test_cmp file1 out1 &&\n test_cmp file2 out2 &&\n test_cmp file3 out3\n '\n\n test_expect_success \"resolving sub-objects works\" '\n ipfs dag get $IPLDHASH/hello > sub1 &&\n ipfs dag get $IPLDHASH/sub > sub2 &&\n ipfs dag get $IPLDHASH/sub/beep > sub3 &&\n ipfs dag get $IPLDHASH/sub/beep/0 > sub4 &&\n ipfs dag get $IPLDHASH/sub/beep/1 > sub5\n '\n\n test_expect_success \"sub-objects look right\" '\n echo -n \"\\\"world\\\"\" > sub1_exp &&\n test_cmp sub1_exp sub1 &&\n echo -n \"{\\\"beep\\\":[0,\\\"bop\\\"],\\\"dict\\\":\\\"ionary\\\"}\" > sub2_exp &&\n test_cmp sub2_exp sub2 &&\n echo -n \"[0,\\\"bop\\\"]\" > sub3_exp &&\n test_cmp sub3_exp sub3 &&\n echo -n \"0\" > sub4_exp &&\n test_cmp sub4_exp sub4 &&\n echo -n \"\\\"bop\\\"\" > sub5_exp &&\n test_cmp sub5_exp sub5\n '\n\n test_expect_success \"traversals using /ipld/ work\" '\n ipfs dag get /ipld/$IPLDPBHASH/Data > ipld_path_Data_actual\n '\n\n test_expect_success \"retrieved node looks right\" '\n echo -n \"{\\\"/\\\":{\\\"bytes\\\":\\\"AAECAwQ\\\"}}\" > ipld_path_Data_expected\n test_cmp ipld_path_Data_actual ipld_path_Data_expected\n '\n\n test_expect_success \"can pin ipld object\" '\n ipfs pin add $IPLDHASH\n '\n\n test_expect_success \"can pin dag-pb object\" '\n ipfs pin add $IPLDPBHASH\n '\n\n test_expect_success \"can pin dag-cbor object\" '\n ipfs pin add $IPLDCBORHASH\n '\n\n test_expect_success \"can pin dag-json object\" '\n ipfs pin add $IPLDJSONHASH\n '\n\n test_expect_success \"after gc, objects still accessible\" '\n ipfs repo gc > /dev/null &&\n ipfs refs -r --timeout=2s $IPLDJSONHASH > /dev/null\n '\n\n test_expect_success \"can get object\" '\n ipfs dag get $IPLDHASH > ipld_obj_out\n '\n\n test_expect_success \"object links look right\" '\n grep \"{\\\"/\\\":\\\"\" ipld_obj_out > /dev/null\n '\n\n test_expect_success \"retrieved object hashes back correctly\" '\n IPLDHASH2=$(cat ipld_obj_out | ipfs dag put --input-codec dag-json --store-codec dag-cbor) &&\n test \"$IPLDHASH\" = \"$IPLDHASH2\"\n '\n\n test_expect_success \"add a normal file\" '\n HASH=$(echo \"foobar\" | ipfs add -q)\n '\n\n test_expect_success \"can view protobuf object with dag get\" '\n ipfs dag get $HASH > dag_get_pb_out\n '\n\n test_expect_success \"output looks correct\" '\n echo -n \"{\\\"Data\\\":{\\\"/\\\":{\\\"bytes\\\":\\\"CAISB2Zvb2JhcgoYBw\\\"}},\\\"Links\\\":[]}\" > dag_get_pb_exp &&\n test_cmp dag_get_pb_exp dag_get_pb_out\n '\n\n test_expect_success \"can call dag get with a path\" '\n ipfs dag get $IPLDHASH/cats/0 > cat_out\n '\n\n test_expect_success \"output looks correct\" '\n echo -n \"{\\\"Data\\\":{\\\"/\\\":{\\\"bytes\\\":\\\"CAISBGZvbwoYBA\\\"}},\\\"Links\\\":[]}\" > cat_exp &&\n test_cmp cat_exp cat_out\n '\n\n test_expect_success \"non-canonical dag-cbor input is normalized\" '\n HASH=$(cat ../t0053-dag-data/non-canon.cbor | ipfs dag put --store-codec dag-cbor --input-codec dag-cbor) &&\n test $HASH = \"bafyreiawx7ona7oa2ptcoh6vwq4q6bmd7x2ibtkykld327bgb7t73ayrqm\" ||\n test_fsh echo $HASH\n '\n\n test_expect_success \"cbor input can be fetched\" '\n EXPARR=$(ipfs dag get $HASH/arr)\n test $EXPARR = \"[]\"\n '\n\n test_expect_success \"add an ipld with pin\" '\n PINHASH=$(printf {\\\"foo\\\":\\\"bar\\\"} | ipfs dag put --input-codec dag-json --pin=true)\n '\n\n test_expect_success \"after gc, objects still accessible\" '\n ipfs repo gc > /dev/null &&\n ipfs refs -r --timeout=2s $PINHASH > /dev/null\n '\n\n test_expect_success \"can add an ipld object with sha3-512 hash\" '\n IPLDHASH=$(cat ipld_object | ipfs dag put --hash sha3-512)\n '\n\n test_expect_success \"output looks correct\" '\n EXPHASH=\"bafyriqforjma7y7akqz7nhuu73r6liggj5zhkbfiqgicywe3fgkna2ijlhod2af3ue7doj56tlzt5hh6iu5esafc4msr3sd53jol5m2o25ucy\"\n test $EXPHASH = $IPLDHASH\n '\n\n test_expect_success \"prepare dag-pb object\" '\n echo foo > test_file &&\n HASH=$(ipfs add -wQ test_file | ipfs cid base32)\n '\n\n test_expect_success \"dag put with json dag-pb works\" '\n ipfs dag get $HASH > pbjson &&\n cat pbjson | ipfs dag put --store-codec=dag-pb --input-codec=dag-json > dag_put_out\n '\n\n test_expect_success \"dag put with dag-pb works output looks good\" '\n echo $HASH > dag_put_exp &&\n test_cmp dag_put_exp dag_put_out\n '\n\n test_expect_success \"dag put with raw dag-pb works\" '\n ipfs block get $HASH > pbraw &&\n cat pbraw | ipfs dag put --store-codec=dag-pb --input-codec=dag-pb > dag_put_out\n '\n\n test_expect_success \"dag put with dag-pb works output looks good\" '\n echo $HASH > dag_put_exp &&\n test_cmp dag_put_exp dag_put_out\n '\n\n test_expect_success \"dag put with raw node works\" '\n echo \"foo bar\" > raw_node_in &&\n HASH=$(ipfs dag put --store-codec=raw --input-codec=raw -- raw_node_in) &&\n ipfs block get \"$HASH\" > raw_node_out &&\n test_cmp raw_node_in raw_node_out'\n\n test_expect_success \"dag put multiple files\" '\n printf {\\\"foo\\\":\\\"bar\\\"} > a.json &&\n printf {\\\"foo\\\":\\\"baz\\\"} > b.json &&\n ipfs dag put a.json b.json > dag_put_out\n '\n\n test_expect_success \"dag put multiple files output looks good\" '\n echo bafyreiblaotetvwobe7cu2uqvnddr6ew2q3cu75qsoweulzku2egca4dxq > dag_put_exp &&\n echo bafyreibqp7zvp6dvrqhtkbwuzzk7jhtmfmngtiqjajqpm6gtw55o7kqzfi >> dag_put_exp &&\n\n test_cmp dag_put_exp dag_put_out\n '\n\n test_expect_success \"prepare data for dag resolve\" '\n NESTED_HASH=$(echo \"{\\\"data\\\":123}\" | ipfs dag put) &&\n HASH=$(echo \"{\\\"obj\\\":{\\\"/\\\":\\\"${NESTED_HASH}\\\"}}\" | ipfs dag put)\n '\n\n test_expect_success \"dag resolve some things\" '\n ipfs dag resolve $HASH > resolve_hash &&\n ipfs dag resolve ${HASH}/obj > resolve_obj &&\n ipfs dag resolve ${HASH}/obj/data > resolve_data\n '\n\n test_expect_success \"dag resolve output looks good\" '\n printf $HASH > resolve_hash_exp &&\n printf $NESTED_HASH > resolve_obj_exp &&\n printf $NESTED_HASH/data > resolve_data_exp &&\n\n test_cmp resolve_hash_exp resolve_hash &&\n test_cmp resolve_obj_exp resolve_obj &&\n test_cmp resolve_data_exp resolve_data\n '\n\n test_expect_success \"get base32 version of hashes for testing\" '\n HASHb32=$(ipfs cid base32 $HASH) &&\n NESTED_HASHb32=$(ipfs cid base32 $NESTED_HASH)\n '\n\n test_expect_success \"dag resolve some things with --cid-base=base32\" '\n ipfs dag resolve $HASH --cid-base=base32 > resolve_hash &&\n ipfs dag resolve ${HASH}/obj --cid-base=base32 > resolve_obj &&\n ipfs dag resolve ${HASH}/obj/data --cid-base=base32 > resolve_data\n '\n\n test_expect_success \"dag resolve output looks good with --cid-base=base32\" '\n printf $HASHb32 > resolve_hash_exp &&\n printf $NESTED_HASHb32 > resolve_obj_exp &&\n printf $NESTED_HASHb32/data > resolve_data_exp &&\n\n test_cmp resolve_hash_exp resolve_hash &&\n test_cmp resolve_obj_exp resolve_obj &&\n test_cmp resolve_data_exp resolve_data\n '\n\n test_expect_success \"dag resolve some things with base32 hash\" '\n ipfs dag resolve $HASHb32 > resolve_hash &&\n ipfs dag resolve ${HASHb32}/obj > resolve_obj &&\n ipfs dag resolve ${HASHb32}/obj/data > resolve_data\n '\n\n test_expect_success \"dag resolve output looks good with base32 hash\" '\n printf $HASHb32 > resolve_hash_exp &&\n printf $NESTED_HASHb32 > resolve_obj_exp &&\n printf $NESTED_HASHb32/data > resolve_data_exp &&\n\n test_cmp resolve_hash_exp resolve_hash &&\n test_cmp resolve_obj_exp resolve_obj &&\n test_cmp resolve_data_exp resolve_data\n '\n\n\n}\n\n# should work offline\ntest_dag_cmd\n\n# should work online\ntest_launch_ipfs_daemon\ntest_dag_cmd\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0054-dag-car-import-export-data/README.md", "content": "# Dataset description/sources\n\n- lotus_testnet_export_256_multiroot.car\n - Export of the first 256 block of the testnet chain, with 3 tipset roots. Exported from Lotus by @traviperson on 2019-03-18\n\n\n- lotus_devnet_genesis.car\n - Source: https://github.com/filecoin-project/lotus/blob/v0.2.10/build/genesis/devnet.car\n\n- lotus_testnet_export_128.car\n - Export of the first 128 block of the testnet chain, exported from Lotus by @traviperson on 2019-03-24\n\n\n- lotus_devnet_genesis_shuffled_noroots.car\n- lotus_testnet_export_128_shuffled_noroots.car\n - versions of the above with an **empty** root array, and having all blocks shuffled\n\n- lotus_devnet_genesis_shuffled_nulroot.car\n- lotus_testnet_export_128_shuffled_nulroot.car\n - versions identical to the above, but with a single \"empty-block\" root each ( in order to work around go-car not following the current \"roots can be empty\" spec )\n\n- combined_naked_roots_genesis_and_128.car\n - only the roots of `lotus_devnet_genesis.car` and `lotus_testnet_export_128.car`,to be used in combination with the root-less parts to validate \"transactional\" pinning\n\n- lotus_testnet_export_128_v2.car\n- lotus_devnet_genesis_v2.car\n - generated with `car index lotus_testnet_export_128.car > lotus_testnet_export_128_v2.car`\n - install `go-car` CLI from https://github.com/ipld/go-car\n\n- partial-dag-scope-entity.car\n - unixfs directory entity exported from gateway via `?format=car&dag-scope=entity` ([IPIP-402](https://specs.ipfs.tech/ipips/ipip-0402/))\n - CAR roots includes directory CID, but only the root block is included in the CAR, making the DAG incomplete\n" }, { "path": "test/sharness/t0054-dag-car-import-export.sh", "content": "#!/usr/bin/env bash\n#\n\ntest_description=\"Test car file import/export functionality\"\n\n. lib/test-lib.sh\nexport -f ipfsi\n\nset -o pipefail\n\ntar -C ../t0054-dag-car-import-export-data/ --strip-components=1 -Jxf ../t0054-dag-car-import-export-data/test_dataset_car.tar.xz\ntab=$'\\t'\n\ntest_cmp_sorted() {\n # use test_cmp to dump out the unsorted file contents as a diff\n [[ \"$( sort \"$1\" | sha256sum )\" == \"$( sort \"$2\" | sha256sum )\" ]] \\\n || test_cmp \"$1\" \"$2\"\n}\nexport -f test_cmp_sorted\n\nreset_blockstore() {\n node=$1\n\n ipfsi \"$node\" pin ls --quiet --type=recursive | ipfsi \"$node\" pin rm &>/dev/null\n ipfsi \"$node\" repo gc &>/dev/null\n\n test_expect_success \"pinlist empty\" '\n [[ -z \"$( ipfsi $node pin ls )\" ]]\n '\n test_expect_success \"nothing left to gc\" '\n [[ -z \"$( ipfsi $node repo gc )\" ]]\n '\n}\n\n# hammer with concurrent gc to ensure nothing clashes\ndo_import() {\n node=\"$1\"; shift\n (\n touch spin.gc\n\n while [[ -e spin.gc ]]; do ipfsi \"$node\" repo gc &>/dev/null; done &\n while [[ -e spin.gc ]]; do ipfsi \"$node\" repo gc &>/dev/null; done &\n\n ipfsi \"$node\" dag import \"$@\" 2>&1 && ipfsi \"$node\" repo verify &>/dev/null\n result=$?\n\n rm -f spin.gc &>/dev/null\n wait || true # work around possible trigger of a bash bug on overloaded circleci\n exit $result\n )\n}\n\nrun_online_imp_exp_tests() {\n\n reset_blockstore 0\n reset_blockstore 1\n\n cat > basic_import_stats_expected < basic_import_expected\n\n # Explainer:\n # naked_root_import_json_expected output is produced by dag import of combined_naked_roots_genesis_and_128.car\n # executed when roots are already present in the repo - thus the BlockCount=0\n # (if blocks were not present in the repo, ipld: could not find would be returned)\n cat >naked_root_import_json_expected < basic_import_actual\n '\n\n test_expect_success \"basic import output as expected\" '\n test_cmp_sorted basic_import_expected basic_import_actual\n '\n\n test_expect_success \"basic import with --stats\" '\n do_import 0 --stats \\\n ../t0054-dag-car-import-export-data/combined_naked_roots_genesis_and_128.car \\\n ../t0054-dag-car-import-export-data/lotus_testnet_export_128_shuffled_nulroot.car \\\n ../t0054-dag-car-import-export-data/lotus_devnet_genesis_shuffled_nulroot.car \\\n > basic_import_actual\n '\n\n test_expect_success \"basic import output with --stats as expected\" '\n test_cmp_sorted basic_import_stats_expected basic_import_actual\n '\n\n test_expect_success \"basic fetch+export 1\" '\n ipfsi 1 dag export bafy2bzaced4ueelaegfs5fqu4tzsh6ywbbpfk3cxppupmxfdhbpbhzawfw5oy > reexported_testnet_128.car\n '\n test_expect_success \"export of shuffled testnet export identical to canonical original\" '\n test_cmp reexported_testnet_128.car ../t0054-dag-car-import-export-data/lotus_testnet_export_128.car\n '\n\n test_expect_success \"basic fetch+export 2\" '\n ipfsi 1 dag export bafy2bzaceaxm23epjsmh75yvzcecsrbavlmkcxnva66bkdebdcnyw3bjrc74u > reexported_devnet_genesis.car\n '\n test_expect_success \"export of shuffled devnet export identical to canonical original\" '\n test_cmp reexported_devnet_genesis.car ../t0054-dag-car-import-export-data/lotus_devnet_genesis.car\n '\n\n test_expect_success \"pinlist on node1 still empty\" '\n [[ -z \"$( ipfsi 1 pin ls )\" ]]\n '\n\n test_expect_success \"import/pin naked roots only, relying on local blockstore having all the data\" '\n ipfsi 1 dag import --stats --enc=json ../t0054-dag-car-import-export-data/combined_naked_roots_genesis_and_128.car \\\n > naked_import_result_json_actual\n '\n\n test_expect_success \"naked import output as expected\" '\n test_cmp_sorted naked_root_import_json_expected naked_import_result_json_actual\n '\n\n reset_blockstore 0\n reset_blockstore 1\n\n mkfifo pipe_testnet\n mkfifo pipe_devnet\n\n test_expect_success \"fifo import\" '\n (\n cat ../t0054-dag-car-import-export-data/lotus_testnet_export_128_shuffled_nulroot.car > pipe_testnet &\n cat ../t0054-dag-car-import-export-data/lotus_devnet_genesis_shuffled_nulroot.car > pipe_devnet &\n\n do_import 0 --stats \\\n pipe_testnet \\\n pipe_devnet \\\n ../t0054-dag-car-import-export-data/combined_naked_roots_genesis_and_128.car \\\n > basic_fifo_import_actual\n result=$?\n\n wait || true\t# work around possible trigger of a bash bug on overloaded circleci\n exit \"$result\"\n )\n '\n\n test_expect_success \"remove fifos\" '\n rm pipe_testnet pipe_devnet\n '\n\n test_expect_success \"fifo-import output as expected\" '\n test_cmp_sorted basic_import_stats_expected basic_fifo_import_actual\n '\n}\n\n\ntest_expect_success \"set up testbed\" '\n iptb testbed create -type localipfs -count 2 -force -init\n'\nstartup_cluster 2\n\nrun_online_imp_exp_tests\n\ntest_expect_success \"shut down nodes\" '\n iptb stop && iptb_wait_stop\n'\n\n\n# We want to just init the repo, without using a daemon for stuff below\ntest_init_ipfs --empty-repo=false\n\n\ntest_expect_success \"basic offline export of 'getting started' dag works\" '\n ipfs dag export \"$HASH_WELCOME_DOCS\" >/dev/null\n'\n\ntest_expect_success \"basic offline export of nonexistent cid\" '\n ! ipfs dag export QmYwAPJXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 2> offline_fetch_error_actual >/dev/null\n'\ntest_expect_success \"correct error\" '\n test_should_contain \"Error: block was not found locally (offline): ipld: could not find QmYwAPJXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\" offline_fetch_error_actual\n'\n\ncat >multiroot_import_json_stats_expected < multiroot_import_json_expected\n\ntest_expect_success \"multiroot import works (--enc=json)\" '\n ipfs dag import --enc=json ../t0054-dag-car-import-export-data/lotus_testnet_export_256_multiroot.car > multiroot_import_json_actual\n'\ntest_expect_success \"multiroot import expected output\" '\n test_cmp_sorted multiroot_import_json_expected multiroot_import_json_actual\n'\n\ntest_expect_success \"multiroot import works with --stats\" '\n ipfs dag import --stats --enc=json ../t0054-dag-car-import-export-data/lotus_testnet_export_256_multiroot.car > multiroot_import_json_actual\n'\ntest_expect_success \"multiroot import expected output\" '\n test_cmp_sorted multiroot_import_json_stats_expected multiroot_import_json_actual\n'\n\n\ncat >pin_import_expected << EOE\n{\"Stats\":{\"BlockCount\":1198,\"BlockBytesCount\":468513}}\nEOE\ntest_expect_success \"pin-less import works\" '\n ipfs dag import --stats --enc=json --pin-roots=false \\\n ../t0054-dag-car-import-export-data/lotus_devnet_genesis.car \\\n ../t0054-dag-car-import-export-data/lotus_testnet_export_128.car \\\n > no-pin_import_actual\n'\ntest_expect_success \"expected no pins on --pin-roots=false\" '\n test_cmp pin_import_expected no-pin_import_actual\n'\n\n\ntest_expect_success \"naked root import works\" '\n ipfs dag import --stats --enc=json ../t0054-dag-car-import-export-data/combined_naked_roots_genesis_and_128.car \\\n > naked_root_import_json_actual\n'\ntest_expect_success \"naked root import expected output\" '\n test_cmp_sorted naked_root_import_json_expected naked_root_import_json_actual\n'\n\ntest_expect_success \"'ipfs dag import' check block size\" '\n BIG_CID=$(dd if=/dev/zero bs=2097153 count=1 | ipfs dag put --input-codec=raw --store-codec=raw --allow-big-block) &&\n ipfs dag export $BIG_CID > over-2MiB-block.car &&\n test_expect_code 1 ipfs dag import over-2MiB-block.car >dag_import_out 2>&1\n'\ntest_expect_success \"ipfs dag import output has the correct error\" '\n grep \"block is over 2MiB\" dag_import_out\n'\n\ntest_expect_success \"ipfs dag import --allow-big-block works\" '\n test_expect_code 0 ipfs dag import --allow-big-block over-2MiB-block.car\n'\n\ncat > version_2_import_expected << EOE\n{\"Root\":{\"Cid\":{\"/\":\"bafy2bzaceaxm23epjsmh75yvzcecsrbavlmkcxnva66bkdebdcnyw3bjrc74u\"},\"PinErrorMsg\":\"\"}}\n{\"Root\":{\"Cid\":{\"/\":\"bafy2bzaced4ueelaegfs5fqu4tzsh6ywbbpfk3cxppupmxfdhbpbhzawfw5oy\"},\"PinErrorMsg\":\"\"}}\n{\"Stats\":{\"BlockCount\":1198,\"BlockBytesCount\":468513}}\nEOE\n\ntest_expect_success \"version 2 import\" '\n ipfs dag import --stats --enc=json \\\n ../t0054-dag-car-import-export-data/lotus_testnet_export_128_v2.car \\\n ../t0054-dag-car-import-export-data/lotus_devnet_genesis_v2.car \\\n > version_2_import_actual\n'\n\ntest_expect_success \"version 2 import output as expected\" '\n test_cmp_sorted version_2_import_expected version_2_import_actual\n'\n\ntest_expect_success \"'ipfs dag import' decode IPLD 'dag-json' codec works\" '\n NEW_HASH=$(echo \"{ \\\"test\\\": \\\"dag-json\\\" }\" | ipfs dag put --store-codec dag-json) &&\n ipfs dag export $NEW_HASH > dag-json.car &&\n ipfs dag import dag-json.car &&\n rm dag-json.car\n'\n\ntest_expect_success \"'ipfs dag import' decode IPLD 'dag-cbor' codec works\" '\n NEW_HASH=$(echo \"{ \\\"test\\\": \\\"dag-cbor\\\" }\" | ipfs dag put --store-codec dag-cbor) &&\n ipfs dag export $NEW_HASH > dag-cbor.car &&\n ipfs dag import dag-cbor.car &&\n rm dag-cbor.car\n'\n\ntest_expect_success \"'ipfs dag import' decode IPLD 'json' codec works\" '\n NEW_HASH=$(echo \"{ \\\"test\\\": \\\"json\\\" }\" | ipfs dag put --store-codec json) &&\n ipfs dag export $NEW_HASH > json.car &&\n ipfs dag import json.car &&\n rm json.car\n'\n\ntest_expect_success \"'ipfs dag import' decode IPLD 'cbor' codec works\" '\n NEW_HASH=$(echo \"{ \\\"test\\\": \\\"cbor\\\" }\" | ipfs dag put --store-codec cbor) &&\n ipfs dag export $NEW_HASH > cbor.car &&\n ipfs dag import cbor.car &&\n rm cbor.car\n'\n\n# IPIP-402\ncat > partial_nopin_import_expected << EOE\n{\"Stats\":{\"BlockCount\":1,\"BlockBytesCount\":1618}}\nEOE\ntest_expect_success \"'ipfs dag import' without pinning works fine with incomplete DAG (unixfs dir exported as dag-scope=entity from IPIP-402)\" '\n ipfs dag import --stats --enc=json --pin-roots=false ../t0054-dag-car-import-export-data/partial-dag-scope-entity.car >partial_nopin_import_out 2>&1 &&\n test_cmp partial_nopin_import_expected partial_nopin_import_out\n'\n\ntest_expect_success \"'ipfs dag import' with pinning errors due to incomplete DAG (unixfs dir exported as dag-scope=entity from IPIP-402)\" '\n ipfs dag import --stats --enc=json --pin-roots=true ../t0054-dag-car-import-export-data/partial-dag-scope-entity.car >partial_pin_import_out 2>&1 &&\n test_should_contain \"\\\"PinErrorMsg\\\":\\\"block was not found locally\" partial_pin_import_out\n'\n\ntest_expect_success \"'ipfs dag import' pin error in default CLI mode produces exit code 1 (unixfs dir exported as dag-scope=entity from IPIP-402)\" '\n test_expect_code 1 ipfs dag import ../t0054-dag-car-import-export-data/partial-dag-scope-entity.car >partial_pin_import_out 2>&1 &&\n test_should_contain \"Error: pinning root \\\"QmPDC11yLAbVw3dX5jMeEuSdk4BiVjSd9X87zaYRdVjzW3\\\" FAILED: block was not found locally\" partial_pin_import_out\n'\n\ntest_done\n" }, { "path": "test/sharness/t0055-dag-put-json-new-line.sh", "content": "#!/usr/bin/env bash\n\ntest_description='Test retrieval of JSON put as CBOR does not end with new-line'\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success 'create test JSON files' '\n WANT_JSON=\"{\\\"data\\\":1234}\"\n WANT_HASH=\"bafyreidbm2zncsc3j25zn7lofgd4woeh6eygdy73thfosuni2rwr3bhcvu\"\n printf \"${WANT_JSON}\\n\" > with_newline.json &&\n printf \"${WANT_JSON}\" > without_newline.json\n'\n\ntest_expect_success 'puts as CBOR work' '\n GOT_HASH_WITHOUT_NEWLINE=\"$(cat without_newline.json | ipfs dag put --store-codec dag-cbor)\"\n GOT_HASH_WITH_NEWLINE=\"$(cat with_newline.json | ipfs dag put --store-codec dag-cbor)\"\n'\n\ntest_expect_success 'put hashes with or without newline are equal' '\n test \"${GOT_HASH_WITH_NEWLINE}\" = \"${GOT_HASH_WITHOUT_NEWLINE}\"\n'\n\ntest_expect_success 'hashes are of expected value' '\n test \"${WANT_HASH}\" = \"${GOT_HASH_WITH_NEWLINE}\"\n test \"${WANT_HASH}\" = \"${GOT_HASH_WITHOUT_NEWLINE}\"\n'\n\ntest_expect_success \"retrieval by hash does not have new line\" '\n ipfs dag get \"${WANT_HASH}\" > got.json\n test_cmp without_newline.json got.json\n'\n\ntest_done\n" }, { "path": "test/sharness/t0060-daemon.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Juan Batiz-Benet\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test daemon command\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"create pebble config\" '\n ipfs init --profile=pebbleds,test > /dev/null &&\n cp \"$IPFS_PATH/config\" init-config\n'\n\ntest_expect_success \"cleanup repo\" '\n rm -rf \"$IPFS_PATH\"\n'\n\ntest_launch_ipfs_daemon --init --init-config=\"$(pwd)/init-config\" --init-profile=test\ntest_kill_ipfs_daemon\n\ntest_expect_success \"daemon initialization with existing config works\" '\n ipfs config \"Datastore.Spec.path\" >actual &&\n test $(cat actual) = \"pebbleds\" &&\n ipfs config Addresses > orig_addrs\n'\n\ntest_expect_success \"cleanup repo\" '\n rm -rf \"$IPFS_PATH\"\n'\n\ntest_launch_ipfs_daemon --init --init-config=\"$(pwd)/init-config\" --init-profile=test,randomports\ntest_kill_ipfs_daemon\n\ntest_expect_success \"daemon initialization with existing config + profiles works\" '\n ipfs config Addresses >new_addrs &&\n test_expect_code 1 diff -q new_addrs orig_addrs\n'\n\ntest_expect_success \"cleanup repo\" '\n rm -rf \"$IPFS_PATH\"\n'\n\ntest_init_ipfs\ntest_expect_success \"set Resource Manager variables showed at startup\" '\n ipfs config --json Swarm.ResourceMgr.MaxFileDescriptors 1024 &&\n ipfs config Swarm.ResourceMgr.MaxMemory 4GB\n'\ntest_launch_ipfs_daemon\n\n# this errors if we didn't --init $IPFS_PATH correctly\ntest_expect_success \"'ipfs config Identity.PeerID' works\" '\n PEERID=$(ipfs config Identity.PeerID)\n'\n\ntest_expect_success \"'ipfs swarm addrs local' works\" '\n ipfs swarm addrs local >local_addrs\n'\n\ntest_expect_success \"ipfs swarm addrs listen; works\" '\n ipfs swarm addrs listen >listen_addrs\n'\n\ntest_expect_success \"ipfs peer id looks good\" '\n test_check_peerid \"$PEERID\"\n'\n\n# this is for checking SetAllowedOrigins race condition for the api and gateway\n# See https://github.com/ipfs/go-ipfs/pull/1966\ntest_expect_success \"ipfs API works with the correct allowed origin port\" '\n curl -s -X POST -H \"Origin:http://localhost:$API_PORT\" -I \"http://$API_ADDR/api/v0/version\"\n'\n\ntest_expect_success \"ipfs gateway works with the correct allowed origin port\" '\n curl -s -X POST -H \"Origin:http://localhost:$GWAY_PORT\" -I \"http://$GWAY_ADDR/api/v0/version\"\n'\n\ntest_expect_success \"ipfs daemon output includes looks good\" '\n test_should_contain \"Initializing daemon...\" actual_daemon &&\n test_should_contain \"$(ipfs version --all)\" actual_daemon &&\n test_should_contain \"PeerID: $(ipfs config Identity.PeerID)\" actual_daemon &&\n test_should_contain \"Swarm listening on 127.0.0.1:\" actual_daemon &&\n test_should_contain \"RPC API server listening on '$API_MADDR'\" actual_daemon &&\n test_should_contain \"WebUI: http://'$API_ADDR'/webui\" actual_daemon &&\n test_should_contain \"Gateway server listening on '$GWAY_MADDR'\" actual_daemon &&\n test_should_contain \"Daemon is ready\" actual_daemon &&\n cat actual_daemon\n'\n\ntest_expect_success \".ipfs/ has been created\" '\n test -d \".ipfs\" &&\n test -f \".ipfs/config\" &&\n test -d \".ipfs/datastore\" &&\n test -d \".ipfs/blocks\" ||\n test_fsh ls -al .ipfs\n'\n\n# begin same as in t0010\n\ntest_expect_success \"ipfs version succeeds\" '\n ipfs version >version.txt\n'\n\ntest_expect_success \"ipfs version output looks good\" '\n egrep \"^ipfs version [0-9]+\\.[0-9]+\\.[0-9]\" version.txt >/dev/null ||\n test_fsh cat version.txt\n'\n\ntest_expect_success \"ipfs version deps succeeds\" '\n ipfs version deps >deps.txt\n'\n\ntest_expect_success \"ipfs version deps output looks good ( set \\$GOIPFSTEST_SKIP_LOCAL_DEVTREE_DEPS_CHECK to skip this test )\" '\n head -1 deps.txt | grep \"go-ipfs@(devel)\" &&\n [[ \"$GOIPFSTEST_SKIP_LOCAL_DEVTREE_DEPS_CHECK\" == \"1\" ]] ||\n [[ $(tail -n +2 deps.txt | egrep -v -c \"^[^ @]+@v[^ @]+( => [^ @]+@v[^ @]+)?$\") -eq 0 ]] ||\n test_fsh cat deps.txt\n'\n\ntest_expect_success \"ipfs help succeeds\" '\n ipfs help >help.txt\n'\n\ntest_expect_success \"ipfs help output looks good\" '\n egrep -i \"^Usage\" help.txt >/dev/null &&\n egrep \"ipfs .* \" help.txt >/dev/null ||\n test_fsh cat help.txt\n'\n\n# check transport is encrypted by default and no plaintext is allowed\n\ntest_expect_success SOCAT \"default transport should support encryption (TLS, needs socat )\" '\n socat -s - tcp:localhost:$SWARM_PORT,connect-timeout=1 > swarmnc < ../t0060-data/mss-tls &&\n grep -q \"/tls\" swarmnc &&\n test_must_fail grep -q \"na\" swarmnc ||\n test_fsh cat swarmnc\n'\n\ntest_expect_success SOCAT \"default transport should support encryption (Noise, needs socat )\" '\n socat -s - tcp:localhost:$SWARM_PORT,connect-timeout=1 > swarmnc < ../t0060-data/mss-noise &&\n grep -q \"/noise\" swarmnc &&\n test_must_fail grep -q \"na\" swarmnc ||\n test_fsh cat swarmnc\n'\n\ntest_expect_success SOCAT \"default transport should not support plaintext (needs socat )\" '\n socat -s - tcp:localhost:$SWARM_PORT,connect-timeout=1 > swarmnc < ../t0060-data/mss-plaintext &&\n grep -q \"na\" swarmnc &&\n test_must_fail grep -q \"/plaintext\" swarmnc ||\n test_fsh cat swarmnc\n'\n\ntest_expect_success \"output from streaming commands works\" '\n test_expect_code 28 curl -X POST -m 5 http://localhost:$API_PORT/api/v0/stats/bw\\?poll=true > statsout\n'\n\ntest_expect_success \"output looks good\" '\n grep TotalIn statsout > /dev/null &&\n grep TotalOut statsout > /dev/null &&\n grep RateIn statsout > /dev/null &&\n grep RateOut statsout >/dev/null\n'\n\n# end same as in t0010\n\ntest_expect_success \"daemon is still running\" '\n kill -0 $IPFS_PID\n'\n\ntest_expect_success \"'ipfs daemon' can be killed\" '\n test_kill_repeat_10_sec $IPFS_PID\n'\n\ntest_expect_success \"'ipfs daemon' should be able to run with a pipe attached to stdin (issue #861)\" '\n yes | ipfs daemon >stdin_daemon_out 2>stdin_daemon_err &\n DAEMON_PID=$!\n test_wait_for_file 20 100ms \"$IPFS_PATH/api\" &&\n test_set_address_vars stdin_daemon_out\n'\n\ntest_expect_success \"daemon with pipe eventually becomes live\" '\n pollEndpoint -host='$API_MADDR' -v -tout=1s -tries=10 >stdin_poll_apiout 2>stdin_poll_apierr &&\n test_kill_repeat_10_sec $DAEMON_PID ||\n test_fsh cat stdin_daemon_out || test_fsh cat stdin_daemon_err || test_fsh cat stdin_poll_apiout || test_fsh cat stdin_poll_apierr\n'\n\ntest_expect_success \"'ipfs daemon' cleans up when it fails to start\" '\n test_must_fail ipfs daemon --routing=foobar &&\n test ! -e \"$IPFS_PATH/repo.lock\"\n'\n\nulimit -S -n 512\nTEST_ULIMIT_PRESET=1\ntest_launch_ipfs_daemon\n\ntest_expect_success \"daemon raised its fd limit\" '\n test_should_not_contain \"setting file descriptor limit\" actual_daemon\n'\n\ntest_expect_success \"daemon actually can handle 2048 file descriptors\" '\n hang-fds -hold=2s 2000 '$API_MADDR' > /dev/null\n'\n\ntest_expect_success \"daemon didn't throw any errors\" '\n test_expect_code 1 grep \"too many open files\" daemon_err\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0060-data/mss-noise", "content": "\u0013/multistream/1.0.0\n\u0007/noise\n" }, { "path": "test/sharness/t0060-data/mss-plaintext", "content": "\u0013/multistream/1.0.0\n\u0011/plaintext/2.0.0\n" }, { "path": "test/sharness/t0060-data/mss-tls", "content": "\u0013/multistream/1.0.0\n\u000b/tls/1.0.0\n" }, { "path": "test/sharness/t0061-daemon-opts.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Juan Batiz-Benet\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test daemon command\"\n\n. lib/test-lib.sh\n\n\ntest_init_ipfs\n\ntest_launch_ipfs_daemon --disable-transport-encryption\n\ngwyaddr=$GWAY_ADDR\napiaddr=$API_ADDR\n\n# Odd. this fails here, but the inverse works on t0060-daemon.\ntest_expect_success SOCAT 'transport should be unencrypted ( needs socat )' '\n socat -s - tcp:localhost:$SWARM_PORT,connect-timeout=1 > swarmnc < ../t0060-data/mss-plaintext &&\n grep -q \"/plaintext\" swarmnc &&\n test_must_fail grep -q \"na\" swarmnc ||\n test_fsh cat swarmnc\n'\n\ntest_kill_ipfs_daemon\n\ntest_launch_ipfs_daemon_without_network\n\ngwyaddr=$GWAY_ADDR\napiaddr=$API_ADDR\n\ntest_expect_success 'gateway should work in offline mode' '\n echo \"hello mars :$gwyaddr :$apiaddr\" >expected &&\n HASH=$(ipfs add -q expected) &&\n curl -sfo actual1 \"http://$gwyaddr/ipfs/$HASH\" &&\n test_cmp expected actual1\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success 'daemon should not start with bad dht opt' '\n test_must_fail ipfs daemon --routing=fdsfdsfds > daemon_output 2>&1\n'\n\ntest_expect_success 'output contains info about dht option' '\n grep \"unrecognized routing option:\" daemon_output ||\n test_fsh cat daemon_output\n'\n\ntest_expect_success 'daemon should not start with supernode dht opt' '\n test_must_fail ipfs daemon --routing=supernode > daemon_output2 2>&1\n'\n\ntest_expect_success 'output contains info about supernode dht option' '\n grep \"supernode routing was never fully implemented\" daemon_output2 ||\n test_fsh cat daemon_output2\n'\n\ntest_done\n" }, { "path": "test/sharness/t0062-daemon-api.sh", "content": "#!/usr/bin/env bash\n#\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test daemon command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ndifferentport=$((API_PORT + 1))\napi_other=\"/ip4/127.0.0.1/tcp/$differentport\"\napi_unreachable=\"/ip4/127.0.0.1/tcp/1\"\n\ntest_expect_success \"config setup\" '\n peerid=$(ipfs config Identity.PeerID) &&\n test_check_peerid \"$peerid\"\n'\n\ntest_client() {\n opts=\"$@\"\n echo \"OPTS = \" $opts\n test_expect_success \"client must work properly $state\" '\n printf \"$peerid\" >expected &&\n ipfs id -f=\"\" $opts >actual &&\n test_cmp expected actual\n '\n}\n\ntest_client_must_fail() {\n opts=\"$@\"\n echo \"OPTS = \" $opts\n test_expect_success \"client should fail $state\" '\n echo \"Error: cannot connect to the api. Is the daemon running? To run as a standalone CLI command remove the api file in \\`\\$IPFS_PATH/api\\`\" >expected_err &&\n test_must_fail ipfs id -f=\"\" $opts >actual 2>actual_err &&\n test_cmp expected_err actual_err\n '\n}\n\ntest_client_suite() {\n state=\"$1\"\n cfg_success=\"$2\"\n diff_success=\"$3\"\n api_fromcfg=\"$4\"\n api_different=\"$5\"\n\n # must always work\n test_client\n\n # must always err\n test_client_must_fail --api \"$api_unreachable\"\n\n if [ \"$cfg_success\" = true ]; then\n test_client --api \"$api_fromcfg\"\n else\n test_client_must_fail --api \"$api_fromcfg\"\n fi\n\n if [ \"$diff_success\" = true ]; then\n test_client --api \"$api_different\"\n else\n test_client_must_fail --api \"$api_different\"\n fi\n}\n\n# first, test things without daemon, without /api file\n# with no daemon, everything should fail \n# (using unreachable because API_MADDR doesn't get set until daemon start)\ntest_client_suite \"(daemon off, no --api, no /api file)\" false false \"$api_unreachable\" \"$api_other\"\n\n\n# then, test things with daemon, with /api file\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"'ipfs daemon' creates api file\" '\n test -f \".ipfs/api\"\n'\n\ntest_client_suite \"(daemon on, no --api, /api file from cfg)\" true false \"$API_MADDR\" \"$api_other\"\n\n# then, test things without daemon, with /api file\n\ntest_kill_ipfs_daemon\n\n# again, both should fail\ntest_client_suite \"(daemon off, no --api, /api file from cfg)\" false false \"$API_MADDR\" \"$api_other\"\n\ntest_done\n" }, { "path": "test/sharness/t0063-daemon-init.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Juan Batiz-Benet\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test daemon --init command\"\n\n. lib/test-lib.sh\n\n# We don't want the normal test_init_ipfs but we need to make sure the\n# IPFS_PATH is set correctly.\nexport IPFS_PATH=\"$(pwd)/.ipfs\"\n\n# safety check since we will be removing the directory\nif [ -e \"$IPFS_PATH\" ]; then\n echo \"$IPFS_PATH exists\"\n exit 1\nfi\n\ntest_ipfs_daemon_init() {\n # Doing it manually since we want to launch the daemon with an\n # empty or non-existent repo; the normal\n # test_launch_ipfs_daemon does not work since it assumes the\n # repo was created a particular way with regard to the API\n # server.\n\n test_expect_success \"'ipfs daemon --init' succeeds\" '\n ipfs daemon --init --init-profile=test >actual_daemon 2>daemon_err &\n IPFS_PID=$!\n sleep 2 &&\n if ! kill -0 $IPFS_PID; then cat daemon_err; return 1; fi\n '\n\n test_expect_success \"'ipfs daemon' can be killed\" '\n test_kill_repeat_10_sec $IPFS_PID\n '\n}\n\ntest_expect_success \"remove \\$IPFS_PATH dir\" '\n rm -rf \"$IPFS_PATH\"\n'\ntest_ipfs_daemon_init\n\ntest_expect_success \"create empty \\$IPFS_PATH dir\" '\n rm -rf \"$IPFS_PATH\" &&\n mkdir \"$IPFS_PATH\"\n'\n\ntest_ipfs_daemon_init\n\ntest_done\n" }, { "path": "test/sharness/t0063-external.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test external command functionality\"\n\n. lib/test-lib.sh\n\n\n# set here so daemon launches with it\nPATH=`pwd`/bin:$PATH\n\ntest_init_ipfs\n\ntest_expect_success \"create fake ipfs-update bin\" '\n mkdir bin &&\n echo \"#!/bin/sh\" > bin/ipfs-update &&\n echo \"pwd\" >> bin/ipfs-update &&\n echo \"test -e \\\"$IPFS_PATH/repo.lock\\\" || echo \\\"repo not locked\\\" \" >> bin/ipfs-update &&\n chmod +x bin/ipfs-update &&\n mkdir just_for_test\n'\n\ntest_expect_success \"external command runs from current user directory and doesn't lock repo\" '\n (cd just_for_test && ipfs update) > actual\n'\n\ntest_expect_success \"output looks good\" '\n echo `pwd`/just_for_test > exp &&\n echo \"repo not locked\" >> exp &&\n test_cmp exp actual\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"external command runs from current user directory when daemon is running\" '\n (cd just_for_test && ipfs update) > actual\n'\n\ntest_expect_success \"output looks good\" '\n echo `pwd`/just_for_test > exp &&\n test_cmp exp actual\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0064-api-file.sh", "content": "#!/usr/bin/env bash\n#\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test api file\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n\ntest_launch_ipfs_daemon\ntest_kill_ipfs_daemon\n\ntest_expect_success \"version always works\" '\n ipfs version >/dev/null\n'\n\ntest_expect_success \"swarm peers fails when offline\" '\n test_must_fail ipfs swarm peers >/dev/null\n'\n\ntest_expect_success \"swarm peers fails when offline and API specified\" '\n test_must_fail ipfs swarm peers --api=\"$API_MADDR\" >/dev/null\n'\n\ntest_expect_success \"pin ls succeeds when offline\" '\n ipfs pin ls >/dev/null\n'\n\ntest_expect_success \"pin ls fails when offline and API specified\" '\n test_must_fail ipfs pin ls --api=\"$API_MADDR\" >/dev/null\n'\n\ntest_expect_success \"id succeeds when offline\" '\n ipfs id >/dev/null\n'\n\ntest_expect_success \"id fails when offline API specified\" '\n test_must_fail ipfs id --api=\"$API_MADDR\" >/dev/null\n'\n\ntest_expect_success \"create API file\" '\n echo \"$API_MADDR\" > \"$IPFS_PATH/api\"\n'\n\ntest_expect_success \"version always works\" '\n ipfs version >/dev/null\n'\n\ntest_expect_success \"id succeeds when offline and API file exists\" '\n ipfs id >/dev/null\n'\n\ntest_expect_success \"pin ls succeeds when offline and API file exists\" '\n ipfs pin ls >/dev/null\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"version always works\" '\n ipfs version >/dev/null\n'\n\ntest_expect_success \"id succeeds when online\" '\n ipfs id >/dev/null\n'\n\ntest_expect_success \"swarm peers succeeds when online\" '\n ipfs swarm peers >/dev/null\n'\n\ntest_expect_success \"pin ls succeeds when online\" '\n ipfs pin ls >/dev/null\n'\n\ntest_expect_success \"remove API file when daemon is running\" '\n rm \"$IPFS_PATH/api\"\n'\n\ntest_expect_success \"version always works\" '\n ipfs version >/dev/null\n'\n\ntest_expect_success \"swarm peers fails when the API file is missing\" '\n test_must_fail ipfs swarm peers >/dev/null\n'\n\ntest_expect_success \"id fails when daemon is running but API file is missing (locks repo)\" '\n test_must_fail ipfs pin ls >/dev/null\n'\n\ntest_expect_success \"pin ls fails when daemon is running but API file is missing (locks repo)\" '\n test_must_fail ipfs pin ls >/dev/null\n'\n\ntest_kill_ipfs_daemon\n\nAPIPORT=32563\n\ntest_expect_success \"Verify gateway file diallable while on unspecified\" '\n ipfs config Addresses.API /ip4/0.0.0.0/tcp/$APIPORT &&\n test_launch_ipfs_daemon &&\n cat \"$IPFS_PATH/api\" > api_file_actual &&\n echo -n \"/ip4/127.0.0.1/tcp/$APIPORT\" > api_file_expected &&\n test_cmp api_file_expected api_file_actual\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0065-active-requests.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test active request commands\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\ntest_launch_ipfs_daemon\n\ntest_expect_success \"command works\" '\n ipfs diag cmds > cmd_out\n'\n\ntest_expect_success \"invoc shows up in output\" '\n grep \"diag/cmds\" cmd_out > /dev/null\n'\n\ntest_expect_success \"start longer running command\" '\n ipfs log tail &\n LOGPID=$!\n go-sleep 100ms\n'\n\ntest_expect_success \"long running command shows up\" '\n ipfs diag cmds > cmd_out2\n'\n\ntest_expect_success \"output looks good\" '\n grep \"log/tail\" cmd_out2 | grep \"true\" > /dev/null\n'\n\ntest_expect_success \"kill log cmd\" '\n kill $LOGPID\n go-sleep 0.5s\n kill $LOGPID\n\n wait $LOGPID || true\n'\n\ntest_expect_success \"long running command inactive\" '\n ipfs diag cmds > cmd_out3\n'\n\ntest_expect_success \"command shows up as inactive\" '\n grep \"log/tail\" cmd_out3 | grep \"false\"\n'\n\ntest_kill_ipfs_daemon\ntest_done\n" }, { "path": "test/sharness/t0066-migration.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test migrations auto update prompt\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# Remove explicit AutoConf.Enabled=false from test profile to use implicit default\n# This allows daemon to work with 'auto' values added by v16-to-17 migration\nipfs config --json AutoConf.Enabled null >/dev/null 2>&1\n\nMIGRATION_START=7\nIPFS_REPO_VER=$(<.ipfs/version)\n\n# Generate mock migration binaries\ngen_mock_migrations() {\n mkdir bin\n i=$((MIGRATION_START))\n until [ $i -ge $IPFS_REPO_VER ]\n do\n j=$((i+1))\n echo \"#!/bin/bash\" > bin/fs-repo-${i}-to-${j}\n echo \"echo fake applying ${i}-to-${j} repo migration\" >> bin/fs-repo-${i}-to-${j}\n # Update version file to the target version for hybrid migration system\n echo \"if [ \\\"\\$1\\\" = \\\"-path\\\" ] && [ -n \\\"\\$2\\\" ]; then\" >> bin/fs-repo-${i}-to-${j}\n echo \" echo $j > \\\"\\$2/version\\\"\" >> bin/fs-repo-${i}-to-${j}\n echo \"elif [ -n \\\"\\$IPFS_PATH\\\" ]; then\" >> bin/fs-repo-${i}-to-${j}\n echo \" echo $j > \\\"\\$IPFS_PATH/version\\\"\" >> bin/fs-repo-${i}-to-${j}\n echo \"fi\" >> bin/fs-repo-${i}-to-${j}\n chmod +x bin/fs-repo-${i}-to-${j}\n ((i++))\n done\n}\n\n# Check for expected output from each migration\ncheck_migration_output() {\n out_file=\"$1\"\n i=$((MIGRATION_START))\n until [ $i -ge $IPFS_REPO_VER ]\n do\n j=$((i+1))\n grep \"applying ${i}-to-${j} repo migration\" \"$out_file\" > /dev/null\n ((i++))\n done\n}\n\n# Create fake migration binaries instead of letting ipfs download from network\n# To test downloading and running actual binaries, comment out this test.\ntest_expect_success \"setup mock migrations\" '\n gen_mock_migrations &&\n find bin -name \"fs-repo-*-to-*\" | wc -l > mock_count &&\n echo $((IPFS_REPO_VER-MIGRATION_START)) > expect_mock_count &&\n export PATH=\"$(pwd)/bin\":$PATH &&\n test_cmp mock_count expect_mock_count\n'\n\ntest_expect_success \"manually reset repo version to $MIGRATION_START\" '\n echo \"$MIGRATION_START\" > \"$IPFS_PATH\"/version\n'\n\ntest_expect_success \"ipfs daemon --migrate=false fails\" '\n test_expect_code 1 ipfs daemon --migrate=false > false_out 2>&1\n'\n\ntest_expect_success \"output looks good\" '\n grep \"Kubo repository at .* has version .* and needs to be migrated to version\" false_out &&\n grep \"Error: fs-repo requires migration\" false_out\n'\n\n# The migrations will succeed and the daemon will continue running\n# since the mock migrations now properly update the repo version number.\ntest_expect_success \"ipfs daemon --migrate=true runs migration\" '\n ipfs daemon --migrate=true > true_out 2>&1 &\n DAEMON_PID=$!\n # Wait for daemon to be ready then shutdown gracefully\n sleep 3 && ipfs shutdown 2>/dev/null || kill $DAEMON_PID 2>/dev/null || true\n wait $DAEMON_PID 2>/dev/null || true\n'\n\ntest_expect_success \"output looks good\" '\n check_migration_output true_out &&\n (grep \"Success: fs-repo migrated to version $IPFS_REPO_VER\" true_out > /dev/null ||\n grep \"Hybrid migration completed successfully: v$MIGRATION_START → v$IPFS_REPO_VER\" true_out > /dev/null)\n'\n\ntest_expect_success \"reset repo version for auto-migration test\" '\n echo \"$MIGRATION_START\" > \"$IPFS_PATH\"/version\n'\n\ntest_expect_success \"'ipfs daemon' prompts to auto migrate\" '\n test_expect_code 1 ipfs daemon > daemon_out 2>&1\n'\n\ntest_expect_success \"output looks good\" '\n grep \"Kubo repository at .* has version .* and needs to be migrated to version\" daemon_out > /dev/null &&\n grep \"Run migrations now?\" daemon_out > /dev/null &&\n grep \"Error: fs-repo requires migration\" daemon_out > /dev/null\n'\n\ntest_expect_success \"ipfs repo migrate succeed\" '\n test_expect_code 0 ipfs repo migrate > migrate_out\n'\n\ntest_expect_success \"output looks good\" '\n grep \"Migrating repository from version\" migrate_out > /dev/null &&\n (grep \"Success: fs-repo migrated to version $IPFS_REPO_VER\" migrate_out > /dev/null ||\n grep \"Hybrid migration completed successfully: v$MIGRATION_START → v$IPFS_REPO_VER\" migrate_out > /dev/null)\n'\n\ntest_expect_success \"manually reset repo version to latest\" '\n echo \"$IPFS_REPO_VER\" > \"$IPFS_PATH\"/version\n'\n\ntest_expect_success \"detect repo does not need migration\" '\n test_expect_code 0 ipfs repo migrate > migrate_out\n'\n\ntest_expect_success \"output looks good\" '\n grep \"Repository is already at version\" migrate_out > /dev/null\n'\n\n# ensure that we get a lock error if we need to migrate and the daemon is running\ntest_launch_ipfs_daemon\n\ntest_expect_success \"manually reset repo version to $MIGRATION_START\" '\n echo \"$MIGRATION_START\" > \"$IPFS_PATH\"/version\n'\n\ntest_expect_success \"ipfs repo migrate fails\" '\n test_expect_code 1 ipfs repo migrate 2> migrate_out\n'\n\ntest_expect_success \"output looks good\" '\n grep \"repo.lock\" migrate_out > /dev/null\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0067-unix-api.sh", "content": "#!/usr/bin/env bash\n#\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test unix API transport\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# We can't use the trash dir as the full name must be longer less than 108 bytes\n# long (because that's the max unix domain socket path length).\nSOCKDIR=\"$(mktemp -d \"${TMPDIR:-/tmp}/unix-api-sharness.XXXXXX\")\"\n\ntest_expect_success \"configure\" '\n peerid=$(ipfs config Identity.PeerID) &&\n ipfs config Addresses.API \"/unix/$SOCKDIR/sock\"\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"client works\" '\n printf \"$peerid\" >expected &&\n ipfs --api=\"/unix/$SOCKDIR/sock\" id -f=\"\" >actual &&\n test_cmp expected actual\n'\n\ntest_kill_ipfs_daemon\ntest_done\n" }, { "path": "test/sharness/t0070-user-config.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Brian Holder-Chow Lin On\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test user-provided config values\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"bootstrap doesn't overwrite user-provided config keys (top-level)\" '\n ipfs config Identity.PeerID >previous &&\n ipfs config Identity.PeerID foo &&\n ipfs bootstrap rm --all &&\n echo \"foo\" >expected &&\n ipfs config Identity.PeerID >actual &&\n ipfs config Identity.PeerID $(cat previous) &&\n test_cmp expected actual\n'\n\ntest_done\n" }, { "path": "test/sharness/t0080-repo.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs repo operations\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs --empty-repo=false\ntest_launch_ipfs_daemon_without_network\n\ntest_expect_success \"'ipfs repo gc' succeeds\" '\n ipfs repo gc >gc_out_actual\n'\n\ntest_expect_success \"'ipfs add afile' succeeds\" '\n echo \"some text\" >afile &&\n HASH=`ipfs add -q afile`\n'\n\ntest_expect_success \"added file was pinned\" '\n ipfs pin ls --type=recursive >actual &&\n grep \"$HASH\" actual\n'\n\ntest_expect_success \"'ipfs repo gc' succeeds\" '\n ipfs repo gc >gc_out_actual\n'\n\ntest_expect_success \"'ipfs repo gc' looks good (patch root)\" '\n test_should_not_contain \"removed $HASH\" gc_out_actual\n'\n\ntest_expect_success \"'ipfs repo gc' doesn't remove file\" '\n ipfs cat \"$HASH\" >out &&\n test_cmp out afile\n'\n\ntest_expect_success \"'ipfs pin rm' succeeds\" '\n ipfs pin rm -r \"$HASH\" >actual1\n'\n\ntest_expect_success \"'ipfs pin rm' output looks good\" '\n echo \"unpinned $HASH\" >expected1 &&\n test_cmp expected1 actual1\n'\n\ntest_expect_success \"ipfs repo gc fully reverse ipfs add (part 1)\" '\n ipfs repo gc &&\n random-data -size=100000 -seed=41 >gcfile &&\n find \"$IPFS_PATH/blocks\" -type f -name \"*.data\" | sort -u > expected_blocks &&\n hash=$(ipfs add -q gcfile) &&\n ipfs pin rm -r $hash &&\n ipfs repo gc\n'\ntest_expect_success \"'ipfs repo gc --silent' succeeds (no output)\" '\n echo \"should be empty\" >bfile &&\n HASH2=`ipfs add -q bfile` &&\n ipfs cat \"$HASH2\" >expected11 &&\n test_cmp expected11 bfile &&\n ipfs pin rm -r \"$HASH2\" &&\n ipfs repo gc --silent >gc_out_empty &&\n test_cmp /dev/null gc_out_empty &&\n test_must_fail ipfs cat \"$HASH2\" 2>err_expected1 &&\n grep \"Error: block was not found locally (offline): ipld: could not find $HASH2\" err_expected1\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"ipfs repo gc fully reverse ipfs add (part 2)\" '\n find \"$IPFS_PATH/blocks\" -type f -name \"*.data\" | sort -u > actual_blocks &&\n test_cmp expected_blocks actual_blocks\n'\n\ntest_launch_ipfs_daemon_without_network\n\ntest_expect_success \"file no longer pinned\" '\n ipfs pin ls --type=recursive --quiet >actual2 &&\n test_expect_code 1 grep $HASH actual2\n'\n\ntest_expect_success \"recursively pin afile(default action)\" '\n HASH=`ipfs add -q afile` &&\n ipfs pin add \"$HASH\"\n'\n\ntest_expect_success \"recursively pin rm afile (default action)\" '\n ipfs pin rm \"$HASH\"\n'\n\ntest_expect_success \"recursively pin afile\" '\n ipfs pin add -r \"$HASH\"\n'\n\ntest_expect_success \"pinning directly should fail now\" '\n echo \"Error: pin: $HASH already pinned recursively\" >expected3 &&\n test_must_fail ipfs pin add -r=false \"$HASH\" 2>actual3 &&\n test_cmp expected3 actual3\n'\n\ntest_expect_success \"'ipfs pin rm -r=false ' should fail\" '\n echo \"Error: $HASH is pinned recursively\" >expected4\n test_must_fail ipfs pin rm -r=false \"$HASH\" 2>actual4 &&\n test_cmp expected4 actual4\n'\n\ntest_expect_success \"remove recursive pin, add direct\" '\n echo \"unpinned $HASH\" >expected5 &&\n ipfs pin rm -r \"$HASH\" >actual5 &&\n test_cmp expected5 actual5 &&\n ipfs pin add -r=false \"$HASH\"\n'\n\ntest_expect_success \"remove direct pin\" '\n echo \"unpinned $HASH\" >expected6 &&\n ipfs pin rm \"$HASH\" >actual6 &&\n test_cmp expected6 actual6\n'\n\ntest_expect_success \"'ipfs repo gc' removes file\" '\n ipfs block stat $HASH &&\n ipfs repo gc &&\n test_must_fail ipfs block stat $HASH\n'\n\n# Convert all to a base32-multihash as refs local outputs cidv1 raw\n# Technically converting refs local output would suffice, but this is more\n# future proof if we ever switch to adding the files with cid-version 1.\ntest_expect_success \"'ipfs refs local' no longer shows file\" '\n EMPTY_DIR=QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn &&\n HASH_MH=`cid-fmt -b base32 \"%M\" \"$HASH\"` &&\n HARDCODED_HASH_MH=`cid-fmt -b base32 \"%M\" \"QmYCvbfNbCwFR45HiNP45rwJgvatpiW38D961L5qAhUM5Y\"` &&\n EMPTY_DIR_MH=`cid-fmt -b base32 \"%M\" \"$EMPTY_DIR\"` &&\n HASH_WELCOME_DOCS_MH=`cid-fmt -b base32 \"%M\" \"$HASH_WELCOME_DOCS\"` &&\n ipfs refs local | cid-fmt -b base32 --filter \"%M\" >actual8 &&\n grep \"$HARDCODED_HASH_MH\" actual8 &&\n grep \"$EMPTY_DIR_MH\" actual8 &&\n grep \"$HASH_WELCOME_DOCS_MH\" actual8 &&\n test_must_fail grep \"$HASH_MH\" actual8\n'\n\ntest_expect_success \"adding multiblock random file succeeds\" '\n random-data -size=1000000 >multiblock &&\n MBLOCKHASH=`ipfs add -q multiblock`\n'\n\ntest_expect_success \"'ipfs pin ls --type=indirect' is correct\" '\n ipfs refs \"$MBLOCKHASH\" >refsout &&\n ipfs refs -r \"$HASH_WELCOME_DOCS\" >>refsout &&\n sed -i\"~\" \"s/\\(.*\\)/\\1 indirect/g\" refsout &&\n ipfs pin ls --type=indirect >indirectpins &&\n test_sort_cmp refsout indirectpins\n'\n\ntest_expect_success \"pin something directly\" '\n echo \"ipfs is so awesome\" >awesome &&\n DIRECTPIN=`ipfs add -q awesome` &&\n echo \"unpinned $DIRECTPIN\" >expected9 &&\n ipfs pin rm -r \"$DIRECTPIN\" >actual9 &&\n test_cmp expected9 actual9 &&\n\n echo \"pinned $DIRECTPIN directly\" >expected10 &&\n ipfs pin add -r=false \"$DIRECTPIN\" >actual10 &&\n test_cmp expected10 actual10\n'\n\ntest_expect_success \"'ipfs pin ls --type=direct' is correct\" '\n echo \"$DIRECTPIN direct\" >directpinexpected &&\n ipfs pin ls --type=direct >directpinout &&\n test_sort_cmp directpinexpected directpinout\n'\n\ntest_expect_success \"'ipfs pin ls --type=recursive' is correct\" '\n echo \"$MBLOCKHASH\" >rp_expected &&\n echo \"$HASH_WELCOME_DOCS\" >>rp_expected &&\n echo \"$EMPTY_DIR\" >>rp_expected &&\n sed -i\"~\" \"s/\\(.*\\)/\\1 recursive/g\" rp_expected &&\n ipfs pin ls --type=recursive >rp_actual &&\n test_sort_cmp rp_expected rp_actual\n'\n\ntest_expect_success \"'ipfs pin ls --type=all --quiet' is correct\" '\n cat directpinout >allpins &&\n cat rp_actual >>allpins &&\n cat indirectpins >>allpins &&\n cut -f1 -d \" \" allpins | sort | uniq >> allpins_uniq_hashes &&\n ipfs pin ls --type=all --quiet >actual_allpins &&\n test_sort_cmp allpins_uniq_hashes actual_allpins\n'\n\ntest_expect_success \"'ipfs refs --unique' is correct\" '\n mkdir -p uniques &&\n echo \"content1\" > uniques/file1 &&\n echo \"content1\" > uniques/file2 &&\n ROOT=$(ipfs add -r -Q uniques) &&\n ipfs refs --unique $ROOT >expected &&\n ipfs add -q uniques/file1 >unique_hash &&\n test_cmp expected unique_hash\n'\n\ntest_expect_success \"'ipfs refs --unique --recursive' is correct\" '\n mkdir -p a/b/c &&\n echo \"c1\" > a/f1 &&\n echo \"c1\" > a/b/f1 &&\n echo \"c1\" > a/b/c/f1 &&\n echo \"c2\" > a/b/c/f2 &&\n ROOT=$(ipfs add -r -Q a) &&\n ipfs refs --unique --recursive $ROOT >refs_output &&\n wc -l refs_output | sed \"s/^ *//g\" >line_count &&\n echo \"4 refs_output\" >expected &&\n test_cmp expected line_count || test_fsh cat refs_output\n'\n\ntest_expect_success \"'ipfs refs --recursive (bigger)'\" '\n mkdir -p b/c/d/e &&\n echo \"content1\" >b/f &&\n echo \"content1\" >b/c/f1 &&\n echo \"content1\" >b/c/d/f2 &&\n echo \"content2\" >b/c/f2 &&\n echo \"content2\" >b/c/d/f1 &&\n echo \"content2\" >b/c/d/e/f &&\n cp -r b b2 && mv b2 b/b2 &&\n cp -r b b3 && mv b3 b/b3 &&\n cp -r b b4 && mv b4 b/b4 &&\n hash=$(ipfs add -r -Q b) &&\n ipfs refs -r \"$hash\" >refs_output &&\n wc -l refs_output | sed \"s/^ *//g\" >actual &&\n echo \"79 refs_output\" >expected &&\n test_cmp expected actual || test_fsh cat refs_output\n'\n\ntest_expect_success \"'ipfs refs --unique --recursive (bigger)'\" '\n ipfs refs -r \"$hash\" >refs_output &&\n sort refs_output | uniq >expected &&\n ipfs refs -r -u \"$hash\" >actual &&\n test_sort_cmp expected actual || test_fsh cat refs_output\n'\n\nget_field_num() {\n field=$1\n file=$2\n num=$(grep \"$field\" \"$file\" | awk '{ print $2 }')\n echo $num\n}\n\ntest_expect_success \"'ipfs repo stat' succeeds\" '\n ipfs repo stat > repo-stats\n'\n\ntest_expect_success \"repo stats came out correct\" '\n grep \"RepoPath\" repo-stats &&\n grep \"RepoSize\" repo-stats &&\n grep \"NumObjects\" repo-stats &&\n grep \"Version\" repo-stats &&\n grep \"StorageMax\" repo-stats\n'\n\ntest_expect_success \"'ipfs repo stat --human' succeeds\" '\n ipfs repo stat --human > repo-stats-human\n'\n\ntest_expect_success \"repo stats --human came out correct\" '\n grep \"RepoPath\" repo-stats-human &&\n grep -E \"RepoSize:\\s*([0-9]*[.])?[0-9]+\\s+?(B|kB|MB|GB|TB|PB|EB)\" repo-stats-human &&\n grep \"NumObjects\" repo-stats-human &&\n grep \"Version\" repo-stats-human &&\n grep -E \"StorageMax:\\s*([0-9]*[.])?[0-9]+\\s+?(B|kB|MB|GB|TB|PB|EB)\" repo-stats-human ||\n test_fsh cat repo-stats-human\n'\n\ntest_expect_success \"'ipfs repo stat' after adding a file\" '\n ipfs add repo-stats &&\n ipfs repo stat > repo-stats-2\n'\n\ntest_expect_success \"repo stats are updated correctly\" '\n test $(get_field_num \"RepoSize\" repo-stats-2) -ge $(get_field_num \"RepoSize\" repo-stats)\n'\n\ntest_expect_success \"'ipfs repo stat --size-only' succeeds\" '\n ipfs repo stat --size-only > repo-stats-size-only\n'\n\ntest_expect_success \"repo stats came out correct for --size-only\" '\n test_should_contain \"RepoSize\" repo-stats-size-only &&\n test_should_contain \"StorageMax\" repo-stats-size-only &&\n test_should_not_contain \"RepoPath\" repo-stats-size-only &&\n test_should_not_contain \"NumObjects\" repo-stats-size-only &&\n test_should_not_contain \"Version\" repo-stats-size-only\n'\n\ntest_expect_success \"'ipfs repo version' succeeds\" '\n ipfs repo version > repo-version\n'\n\ntest_expect_success \"repo version came out correct\" '\n egrep \"^ipfs repo version fs-repo@[0-9]+\" repo-version >/dev/null\n'\n\ntest_expect_success \"'ipfs repo version -q' succeeds\" '\n ipfs repo version -q > repo-version-q\n'\ntest_expect_success \"repo version came out correct\" '\n egrep \"^fs-repo@[0-9]+\" repo-version-q >/dev/null\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"remove Datastore.StorageMax from config\" '\n ipfs config Datastore.StorageMax \"\"\n'\ntest_expect_success \"'ipfs repo stat' still succeeds\" '\n ipfs repo stat > repo-stats\n'\n\ntest_done\n" }, { "path": "test/sharness/t0081-repo-pinning.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs repo pinning\"\n\n. lib/test-lib.sh\n\ntest_pin_flag() {\n object=$1\n ptype=$2\n expect=$3\n\n echo \"test_pin_flag\" \"$@\"\n\n if ipfs pin ls --type=\"$ptype\" \"$object\" >actual\n then\n test \"$expect\" = \"true\" && return\n test_fsh cat actual\n return\n else\n test \"$expect\" = \"false\" && return\n test_fsh cat actual\n return\n fi\n}\n\ntest_pin() {\n object=$1\n shift\n\n test_str_contains \"recursive\" $@\n [ \"$?\" = \"0\" ] && r=\"true\" || r=\"false\"\n\n test_str_contains \"indirect\" $@\n [ \"$?\" = \"0\" ] && i=\"true\" || i=\"false\"\n\n test_str_contains \"direct\" $@\n [ \"$?\" = \"0\" ] && d=\"true\" || d=\"false\"\n\n test_pin_flag \"$object\" \"recursive\" $r || return 1\n test_pin_flag \"$object\" \"indirect\" $i || return 1\n test_pin_flag \"$object\" \"direct\" $d || return 1\n return 0\n}\n\n\ntest_init_ipfs\n\n# test runs much faster without daemon.\n# TODO: turn this back on after:\n# https://github.com/ipfs/go-ipfs/issues/1075\n# test_launch_ipfs_daemon\n\nHASH_FILE6=\"QmRsBC3Y2G6VRPYGAVpZczx1W7Xw54MtM1NcLKTkn6rx3U\"\nHASH_FILE5=\"QmaN3PtyP8DcVGHi3Q2Fcp7CfAFVcVXKddWbHoNvaA41zf\"\nHASH_FILE4=\"QmV1aiVgpDknKQugrK59uBUbMrPnsQM1F9FXbFcfgEvUvH\"\nHASH_FILE3=\"QmZrr4Pzqp3NnMzMfbMhNe7LghfoUFHVx7c9Po9GZrhKZ7\"\nHASH_FILE2=\"QmSkjTornLY72QhmK9NvAz26815pTaoAL42rF8Qi3w2WBP\"\nHASH_FILE1=\"QmbgX4aXhSSY88GHmPQ4roizD8wFwPX8jzTLjc8VAp89x4\"\nHASH_DIR4=\"QmW98gV71Ns4bX7QbgWAqLiGF3SDC1JpveZSgBh4ExaSAd\"\nHASH_DIR3=\"QmRsCaNBMkweZ9vHT5PJRd2TT9rtNKEKyuognCEVxZxF1H\"\nHASH_DIR2=\"QmTUTQAgeVfughDSFukMZLbfGvetDJY7Ef5cDXkKK4abKC\"\nHASH_DIR1=\"QmNyZVFbgvmzguS2jVMRb8PQMNcCMJrn9E3doDhBbcPNTY\"\nHASH_NOPINDIR=\"QmWHjrRJYSfYKz5V9dWWSKu47GdY7NewyRhyTiroXgWcDU\"\nHASH_NOPIN_FILE1=\"QmUJT3GQi1dxQyTZbkaWeer9GkCn1d3W3HHRLSDr6PTcpx\"\nHASH_NOPIN_FILE2=\"QmarR7m9JT7qHEGhuFNZUEMAnoZ8E9QAfsthHCQ9Y2GfoT\"\n\nDIR1=\"dir1\"\nDIR2=\"dir1/dir2\"\nDIR4=\"dir1/dir2/dir4\"\nDIR3=\"dir1/dir3\"\nFILE1=\"dir1/file1\"\nFILE2=\"dir1/file2\"\nFILE3=\"dir1/file3\"\nFILE4=\"dir1/dir2/file4\"\nFILE6=\"dir1/dir2/dir4/file6\"\nFILE5=\"dir1/dir3/file5\"\n\ntest_expect_success \"'ipfs add dir' succeeds\" '\n mkdir dir1 &&\n mkdir dir1/dir2 &&\n mkdir dir1/dir2/dir4 &&\n mkdir dir1/dir3 &&\n echo \"some text 1\" >dir1/file1 &&\n echo \"some text 2\" >dir1/file2 &&\n echo \"some text 3\" >dir1/file3 &&\n echo \"some text 1\" >dir1/dir2/file1 &&\n echo \"some text 4\" >dir1/dir2/file4 &&\n echo \"some text 1\" >dir1/dir2/dir4/file1 &&\n echo \"some text 2\" >dir1/dir2/dir4/file2 &&\n echo \"some text 6\" >dir1/dir2/dir4/file6 &&\n echo \"some text 2\" >dir1/dir3/file2 &&\n echo \"some text 5\" >dir1/dir3/file5 &&\n ipfs add -Q -r dir1 >actual &&\n echo \"$HASH_DIR1\" >expected &&\n ipfs repo gc && # remove the patch chaff\n test_cmp expected actual\n'\n\ntest_expect_success \"objects are there\" '\n ipfs cat \"$HASH_FILE6\" >FILE6_a &&\n ipfs cat \"$HASH_FILE5\" >FILE5_a &&\n ipfs cat \"$HASH_FILE4\" >FILE4_a &&\n ipfs cat \"$HASH_FILE3\" >FILE3_a &&\n ipfs cat \"$HASH_FILE2\" >FILE2_a &&\n ipfs cat \"$HASH_FILE1\" >FILE1_a &&\n ipfs ls \"$HASH_DIR3\" >DIR3_a &&\n ipfs ls \"$HASH_DIR4\" >DIR4_a &&\n ipfs ls \"$HASH_DIR2\" >DIR2_a &&\n ipfs ls \"$HASH_DIR1\" >DIR1_a\n'\n\n# saving this output for later\ntest_expect_success \"ipfs dag get $HASH_DIR1 works\" '\n ipfs dag get $HASH_DIR1 | jq -r \".Links[] | .Hash | .[\\\"/\\\"]\" > DIR1_objlink\n'\n\n\ntest_expect_success \"added dir was pinned recursively\" '\n test_pin_flag $HASH_DIR1 recursive true\n'\n\ntest_expect_success \"rest were pinned indirectly\" '\n test_pin_flag \"$HASH_FILE6\" indirect true\n test_pin_flag \"$HASH_FILE5\" indirect true\n test_pin_flag \"$HASH_FILE4\" indirect true\n test_pin_flag \"$HASH_FILE3\" indirect true\n test_pin_flag \"$HASH_FILE2\" indirect true\n test_pin_flag \"$HASH_FILE1\" indirect true\n test_pin_flag \"$HASH_DIR3\" indirect true\n test_pin_flag \"$HASH_DIR4\" indirect true\n test_pin_flag \"$HASH_DIR2\" indirect true\n'\n\ntest_expect_success \"added dir was NOT pinned indirectly\" '\n test_pin_flag \"$HASH_DIR1\" indirect false\n'\n\ntest_expect_success \"nothing is pinned directly\" '\n ipfs pin ls --type=direct >actual4 &&\n test_must_be_empty actual4\n'\n\ntest_expect_success \"'ipfs repo gc' succeeds\" '\n ipfs repo gc >gc_out_actual\n'\n\ntest_expect_success \"objects are still there\" '\n cat FILE6_a FILE5_a FILE4_a FILE3_a FILE2_a FILE1_a >expected45 &&\n cat DIR3_a DIR4_a DIR2_a DIR1_a >>expected45 &&\n ipfs cat \"$HASH_FILE6\" >actual45 &&\n ipfs cat \"$HASH_FILE5\" >>actual45 &&\n ipfs cat \"$HASH_FILE4\" >>actual45 &&\n ipfs cat \"$HASH_FILE3\" >>actual45 &&\n ipfs cat \"$HASH_FILE2\" >>actual45 &&\n ipfs cat \"$HASH_FILE1\" >>actual45 &&\n ipfs ls \"$HASH_DIR3\" >>actual45 &&\n ipfs ls \"$HASH_DIR4\" >>actual45 &&\n ipfs ls \"$HASH_DIR2\" >>actual45 &&\n ipfs ls \"$HASH_DIR1\" >>actual45 &&\n test_cmp expected45 actual45\n'\n\ntest_expect_success \"remove dir recursive pin succeeds\" '\n echo \"unpinned $HASH_DIR1\" >expected5 &&\n ipfs pin rm -r \"$HASH_DIR1\" >actual5 &&\n test_cmp expected5 actual5\n'\n\ntest_expect_success \"none are pinned any more\" '\n test_pin \"$HASH_FILE6\" &&\n test_pin \"$HASH_FILE5\" &&\n test_pin \"$HASH_FILE4\" &&\n test_pin \"$HASH_FILE3\" &&\n test_pin \"$HASH_FILE2\" &&\n test_pin \"$HASH_FILE1\" &&\n test_pin \"$HASH_DIR3\" &&\n test_pin \"$HASH_DIR4\" &&\n test_pin \"$HASH_DIR2\" &&\n test_pin \"$HASH_DIR1\"\n'\n\ntest_expect_success \"pin some directly and indirectly\" '\n ipfs pin add -r=false \"$HASH_DIR1\" >actual7 &&\n ipfs pin add -r=true \"$HASH_DIR2\" >>actual7 &&\n ipfs pin add -r=false \"$HASH_FILE1\" >>actual7 &&\n echo \"pinned $HASH_DIR1 directly\" >expected7 &&\n echo \"pinned $HASH_DIR2 recursively\" >>expected7 &&\n echo \"pinned $HASH_FILE1 directly\" >>expected7 &&\n test_cmp expected7 actual7\n'\n\ntest_expect_success \"pin lists look good\" '\n test_pin $HASH_DIR1 direct &&\n test_pin $HASH_DIR2 recursive &&\n test_pin $HASH_DIR3 &&\n test_pin $HASH_DIR4 indirect &&\n test_pin $HASH_FILE1 indirect direct &&\n test_pin $HASH_FILE2 indirect &&\n test_pin $HASH_FILE3 &&\n test_pin $HASH_FILE4 indirect &&\n test_pin $HASH_FILE5 &&\n test_pin $HASH_FILE6 indirect\n'\n\ntest_expect_success \"'ipfs repo gc' succeeds\" '\n ipfs repo gc &&\n test_must_fail ipfs block stat $HASH_FILE3 &&\n test_must_fail ipfs block stat $HASH_FILE5 &&\n test_must_fail ipfs block stat $HASH_DIR3\n'\n\n# use object links for HASH_DIR1 here because its children\n# no longer exist\ntest_expect_success \"some objects are still there\" '\n cat FILE6_a FILE4_a FILE2_a FILE1_a >expected8 &&\n cat DIR4_a DIR2_a DIR1_objlink >>expected8 &&\n ipfs cat \"$HASH_FILE6\" >actual8 &&\n ipfs cat \"$HASH_FILE4\" >>actual8 &&\n ipfs cat \"$HASH_FILE2\" >>actual8 &&\n ipfs cat \"$HASH_FILE1\" >>actual8 &&\n ipfs ls \"$HASH_DIR4\" >>actual8 &&\n ipfs ls \"$HASH_DIR2\" >>actual8 &&\n ipfs dag get \"$HASH_DIR1\" | jq -r \".Links[] | .Hash | .[\\\"/\\\"]\" >>actual8 &&\n test_cmp expected8 actual8\n'\n\n# todo: make this faster somehow.\ntest_expect_success \"some are no longer there\" '\n test_must_fail ipfs cat \"$HASH_FILE5\" &&\n test_must_fail ipfs cat \"$HASH_FILE3\" &&\n test_must_fail ipfs ls \"$HASH_DIR3\"\n'\n\ntest_launch_ipfs_daemon_without_network\ntest_expect_success \"recursive pin fails without objects\" '\n test_must_fail ipfs pin add -r \"$HASH_DIR1\" 2>err_expected8 &&\n grep \"ipld: could not find\" err_expected8 ||\n test_fsh cat err_expected8\n'\n\n# Regression test for https://github.com/ipfs/go-ipfs/issues/4650\n# This test requires the daemon. Otherwise, the pin changes are reverted when\n# the pin fails in the previous test.\ntest_expect_success \"failed recursive pin does not remove direct pin\" '\n test_pin_flag \"$HASH_DIR1\" direct true\n'\ntest_kill_ipfs_daemon\n\ntest_expect_success \"test add nopin file\" '\n echo \"test nopin data\" > test_nopin_data &&\n NOPINHASH=$(ipfs add -q --pin=false test_nopin_data) &&\n test_pin_flag \"$NOPINHASH\" direct false &&\n test_pin_flag \"$NOPINHASH\" indirect false &&\n test_pin_flag \"$NOPINHASH\" recursive false\n'\n\n\ntest_expect_success \"test add nopin dir\" '\n mkdir nopin_dir1 &&\n echo \"some nopin text 1\" >nopin_dir1/file1 &&\n echo \"some nopin text 2\" >nopin_dir1/file2 &&\n ipfs add -Q -r --pin=false nopin_dir1 >actual &&\n echo \"$HASH_NOPINDIR\" >expected &&\n test_cmp actual expected &&\n test_pin_flag \"$HASH_NOPINDIR\" direct false &&\n test_pin_flag \"$HASH_NOPINDIR\" indirect false &&\n test_pin_flag \"$HASH_NOPINDIR\" recursive false &&\n test_pin_flag \"$HASH_NOPIN_FILE1\" direct false &&\n test_pin_flag \"$HASH_NOPIN_FILE1\" indirect false &&\n test_pin_flag \"$HASH_NOPIN_FILE1\" recursive false &&\n test_pin_flag \"$HASH_NOPIN_FILE2\" direct false &&\n test_pin_flag \"$HASH_NOPIN_FILE2\" indirect false &&\n test_pin_flag \"$HASH_NOPIN_FILE2\" recursive false\n\n'\n\nFICTIONAL_HASH=\"QmXV4f9v8a56MxWKBhP3ETsz4EaafudU1cKfPaaJnenc48\"\ntest_launch_ipfs_daemon\ntest_expect_success \"test unpinning a hash that's not pinned\" \"\n test_expect_code 1 ipfs pin rm $FICTIONAL_HASH --timeout=2s\n test_expect_code 1 ipfs pin rm $FICTIONAL_HASH/a --timeout=2s\n test_expect_code 1 ipfs pin rm $FICTIONAL_HASH/a/b --timeout=2s\n\"\ntest_kill_ipfs_daemon\n\n# test_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0082-repo-gc-auto.sh", "content": "#!/usr/bin/env bash\n#\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs repo auto gc\"\n\n. lib/test-lib.sh\n\nskip_all=\"skipping auto repo gc tests until they can be fixed\"\n\ntest_done\n\ncheck_ipfs_storage() {\n ipfs config Datastore.StorageMax\n}\n\ntest_init_ipfs\n\ntest_expect_success \"generate 2 600 kB files and 2 MB file using random-data\" '\n random-data -size=614400 -seed=41 >600k1 &&\n random-data -size=614400 -seed=42 >600k2 &&\n random-data -size=2097152 -seed=43 >2M\n'\n\ntest_expect_success \"set ipfs gc watermark, storage max, and gc timeout\" '\n test_config_set Datastore.StorageMax \"2MB\" &&\n test_config_set --json Datastore.StorageGCWatermark 60 &&\n test_config_set Datastore.GCPeriod \"20ms\"\n'\n\ntest_launch_ipfs_daemon --enable-gc\n\ntest_gc() {\n test_expect_success \"adding data below watermark doesn't trigger auto gc\" '\n ipfs add 600k1 >/dev/null &&\n disk_usage \"$IPFS_PATH/blocks\" >expected &&\n go-sleep 40ms &&\n disk_usage \"$IPFS_PATH/blocks\" >actual &&\n test_cmp expected actual\n '\n\n test_expect_success \"adding data beyond watermark triggers auto gc\" '\n HASH=`ipfs add -q 600k2` &&\n ipfs pin rm -r $HASH &&\n go-sleep 40ms &&\n DU=$(disk_usage \"$IPFS_PATH/blocks\") &&\n if test $(uname -s) = \"Darwin\"; then\n test \"$DU\" -lt 1400 # 60% of 2MB\n else\n test \"$DU\" -lt 1000000\n fi\n '\n}\n\n#TODO: conditional GC test is disabled due to files size bug in ipfs add\n#test_expect_success \"adding data beyond storageMax fails\" '\n# test_must_fail ipfs add 2M 2>add_fail_out\n#'\n#test_expect_success \"ipfs add not enough space message looks good\" '\n# echo \"Error: file size exceeds slack space allowed by storageMax. Maybe unpin some files?\" >add_fail_exp &&\n# test_cmp add_fail_exp add_fail_out\n#'\n\ntest_expect_success \"periodic auto gc stress test\" '\n for i in $(test_seq 1 20)\n do\n test_gc || return 1\n done\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0084-repo-read-rehash.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) Jakub Sztandera\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs blockstore repo read check.\"\n\n. lib/test-lib.sh\n\nrm -rf \"$IPF_PATH/*\"\n\ntest_init_ipfs\n\n\nH_BLOCK1=$(echo \"Block 1\" | ipfs add -q)\nH_BLOCK2=$(echo \"Block 2\" | ipfs add -q)\n\nBS_BLOCK1=\"XZ/CIQPDDQH5PDJTF4QSNMPFC45FQZH5MBSWCX2W254P7L7HGNHW5MQXZA.data\"\nBS_BLOCK2=\"CK/CIQNYWBOKHY7TCY7FUOBXKVJ66YRMARDT3KC7PPY6UWWPZR4YA67CKQ.data\"\n\n\ntest_expect_success 'blocks are swapped' '\n ipfs cat $H_BLOCK2 > noswap &&\n cp -f \"$IPFS_PATH/blocks/$BS_BLOCK1\" \"$IPFS_PATH/blocks/$BS_BLOCK2\" &&\n ipfs cat $H_BLOCK2 > swap &&\n test_must_fail test_cmp noswap swap\n'\n\nipfs config --bool Datastore.HashOnRead true\n\ntest_check_bad_blocks() {\n test_expect_success 'getting modified block fails' '\n (test_must_fail ipfs cat $H_BLOCK2 2> err_msg) &&\n grep \"block in storage has different hash than requested\" err_msg\n '\n\n test_expect_success \"block shows up in repo verify\" '\n test_expect_code 1 ipfs repo verify | cid-fmt --filter -b base32 \"%M\" > verify_out &&\n H_BLOCK2_MH=`cid-fmt -b base32 \"%M\" $H_BLOCK2` &&\n grep \"$H_BLOCK2_MH\" verify_out\n '\n}\n\ntest_check_bad_blocks\n\ntest_expect_success \"can add and cat a raw-leaf file\" '\n HASH=$(echo \"stuff\" | ipfs add -q --raw-leaves) &&\n ipfs cat $HASH > /dev/null\n'\n\ntest_launch_ipfs_daemon\ntest_check_bad_blocks\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0086-repo-verify.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n# NOTE: This is a legacy sharness test kept for compatibility.\n# New tests for 'ipfs repo verify' should be added to test/cli/repo_verify_test.go\n#\n\ntest_description=\"Test ipfs repo fsck\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nsort_rand() {\n case `uname` in\n Linux|FreeBSD)\n sort -R\n ;;\n Darwin)\n ruby -e 'puts STDIN.readlines.shuffle'\n ;;\n *)\n echo \"unsupported system: $(uname)\"\n esac\n}\n\ncheck_random_corruption() {\n # Exclude well-known blocks from corruption as they cause test flakiness:\n # - CIQL7TG2PB52XIZLLHDYIUFMHUQLMMZWBNBZSLDXFCPZ5VDNQQ2WDZQ.data: empty file block\n # - CIQFTFEEHEDF6KLBT32BFAGLXEZL4UWFNWM4LFTLMXQBCERZ6CMLX3Y.data: empty directory block (has special handling, served from memory even when corrupted on disk)\n to_break=$(find \"$IPFS_PATH/blocks\" -type f -name '*.data' | grep -v -E \"CIQL7TG2PB52XIZLLHDYIUFMHUQLMMZWBNBZSLDXFCPZ5VDNQQ2WDZQ.data|CIQFTFEEHEDF6KLBT32BFAGLXEZL4UWFNWM4LFTLMXQBCERZ6CMLX3Y.data\" | sort_rand | head -n 1)\n\n test_expect_success \"back up file and overwrite it\" '\n cp \"$to_break\" backup_file &&\n echo \"this is super broken\" > \"$to_break\"\n '\n\n test_expect_success \"repo verify detects failure\" '\n test_expect_code 1 ipfs repo verify\n '\n\n test_expect_success \"replace the object\" '\n cp backup_file \"$to_break\"\n '\n\n test_expect_success \"ipfs repo verify passes just fine now\" '\n ipfs repo verify\n '\n}\n\ntest_expect_success \"create some files\" '\n random-files -depth=3 -dirs=4 -files=10 foobar > /dev/null\n'\n\ntest_expect_success \"add them all\" '\n ipfs add -r -q foobar > /dev/null\n'\n\nfor i in `seq 20`\ndo\n check_random_corruption\ndone\n\ntest_done\n" }, { "path": "test/sharness/t0087-repo-robust-gc.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test robustness of garbage collector\"\n\n. lib/test-lib.sh\nset -e\n\nto_raw_cid() {\n ipfs cid format -b b --mc raw -v 1 \"$1\"\n}\n\ntest_gc_robust_part1() {\n\n test_expect_success \"add a 1MB file with --raw-leaves\" '\n random-data -size=1048576 -seed=56 > afile &&\n HASH1=`ipfs add --raw-leaves -q --cid-version 1 afile` &&\n REFS=`ipfs refs -r $HASH1` &&\n read LEAF1 LEAF2 LEAF3 LEAF4 < <(echo $REFS)\n '\n\n test_expect_success \"find data blocks for added file\" '\n HASH1MH=`cid-fmt -b base32 \"%M\" $HASH1` &&\n LEAF1MH=`cid-fmt -b base32 \"%M\" $LEAF1` &&\n LEAF2MH=`cid-fmt -b base32 \"%M\" $LEAF2` &&\n HASH1FILE=`find .ipfs/blocks -type f | grep -i $HASH1MH` &&\n LEAF1FILE=`find .ipfs/blocks -type f | grep -i $LEAF1MH` &&\n LEAF2FILE=`find .ipfs/blocks -type f | grep -i $LEAF2MH`\n '\n\n test_expect_success \"remove a leaf node from the repo manually\" '\n rm \"$LEAF1FILE\"\n '\n\n test_expect_success \"check that the node is removed\" '\n test_must_fail ipfs cat $HASH1\n '\n\n test_expect_success \"'ipfs repo gc' should still be fine\" '\n ipfs repo gc\n '\n\n test_expect_success \"corrupt the root node of 1MB file\" '\n test -e \"$HASH1FILE\" &&\n dd if=/dev/zero of=\"$HASH1FILE\" count=1 bs=100 conv=notrunc\n '\n\n test_expect_success \"'ipfs repo gc' should abort without removing anything\" '\n test_must_fail ipfs repo gc 2>&1 | tee gc_err &&\n grep -q \"could not retrieve links for $HASH1\" gc_err &&\n grep -q \"aborted\" gc_err\n '\n\n test_expect_success \"leaf nodes were not removed after gc\" '\n ipfs cat $LEAF3 > /dev/null &&\n ipfs cat $LEAF4 > /dev/null\n '\n\n test_expect_success \"unpin the 1MB file\" '\n ipfs pin rm $HASH1\n '\n\n # make sure the permission problem is fixed on exit, otherwise cleanup\n # will fail\n trap \"chmod 700 `dirname \"$LEAF2FILE\"` 2> /dev/null || true\" 0\n\n test_expect_success \"create a permission problem\" '\n chmod 500 `dirname \"$LEAF2FILE\"` &&\n test_must_fail ipfs block rm $LEAF2 2>&1 | tee block_rm_err &&\n grep -q \"permission denied\" block_rm_err\n '\n\n # repo gc outputs raw multihashes. We check HASH1 with block stat rather than\n # grepping the output since it's not a raw multihash\n test_expect_success \"'ipfs repo gc' should still run and remove as much as possible\" '\n test_must_fail ipfs repo gc 2>&1 | tee repo_gc_out &&\n grep -q \"could not remove $LEAF2\" repo_gc_out &&\n grep -q \"removed $(to_raw_cid $LEAF3)\" repo_gc_out &&\n grep -q \"removed $(to_raw_cid $LEAF4)\" repo_gc_out &&\n test_must_fail ipfs block stat $HASH1\n '\n\n test_expect_success \"fix the permission problem\" '\n chmod 700 `dirname \"$LEAF2FILE\"`\n '\n\n test_expect_success \"'ipfs repo gc' should be ok now\" '\n ipfs repo gc | tee repo_gc_out\n grep -q \"removed $(to_raw_cid $LEAF2)\" repo_gc_out\n '\n}\n\ntest_gc_robust_part2() {\n\n test_expect_success \"add 1MB file normally (i.e., without raw leaves)\" '\n random-data -size=1048576 -seed=56 > afile &&\n HASH2=`ipfs add -q afile`\n '\n\n LEAF1=QmcNNR6JSCUhJ9nyoVQgBhABPgcgdsuYJgdSB1f2g6BF5c\n LEAF1FILE=.ipfs/blocks/RA/CIQNA5C3BLRUX3LZ7X6UTOV3KSHLARNXVDK3W5KUO6GVHNRP4SGLRAY.data\n\n LEAF2=QmPvtiBLgwuwF2wyf9VL8PaYgSt1XwGJ2Yu4AscRGEQvqR\n LEAF2FILE=.ipfs/blocks/RN/CIQBPIKEATBI7TIHVYRQJZAKEWF2H22PXW3A7LCEPB6MFFL7IA2CRNA.data\n\n\n test_expect_success \"add some additional unpinned content\" '\n random-data -size=1000 -seed=3 > junk1 &&\n random-data -size=1000 -seed=4 > junk2 &&\n JUNK1=`ipfs add --pin=false -q junk1` &&\n JUNK2=`ipfs add --pin=false -q junk2`\n '\n\n test_expect_success \"remove a leaf node from the repo manually\" '\n rm \"$LEAF1FILE\"\n '\n\n test_expect_success \"'ipfs repo gc' should abort\" '\n test_must_fail ipfs repo gc 2>&1 | tee repo_gc_out &&\n grep -q \"could not retrieve links for $LEAF1\" repo_gc_out &&\n grep -q \"aborted\" repo_gc_out\n '\n\n test_expect_success \"test that garbage collector really aborted\" '\n ipfs cat $JUNK1 > /dev/null &&\n ipfs cat $JUNK2 > /dev/null \n '\n\n test_expect_success \"corrupt a key\" '\n test -e \"$LEAF2FILE\" &&\n dd if=/dev/zero of=\"$LEAF2FILE\" count=1 bs=100 conv=notrunc\n '\n\n test_expect_success \"'ipfs repo gc' should abort with two errors\" '\n test_must_fail ipfs repo gc 2>&1 | tee repo_gc_out &&\n grep -q \"could not retrieve links for $LEAF1\" repo_gc_out &&\n grep -q \"could not retrieve links for $LEAF2\" repo_gc_out &&\n grep -q \"aborted\" repo_gc_out\n '\n\n test_expect_success \"'ipfs repo gc --stream-errors' should abort and report each error separately\" '\n test_must_fail ipfs repo gc --stream-errors 2>&1 | tee repo_gc_out &&\n grep -q \"Error: could not retrieve links for $LEAF1\" repo_gc_out &&\n grep -q \"Error: could not retrieve links for $LEAF2\" repo_gc_out &&\n grep -q \"Error: garbage collection aborted\" repo_gc_out\n '\n\n test_expect_success \"unpin 1MB file\" '\n ipfs pin rm $HASH2\n '\n\n test_expect_success \"'ipfs repo gc' should be fine now\" '\n ipfs repo gc | tee repo_gc_out &&\n grep -q \"removed $(to_raw_cid $HASH2)\" repo_gc_out &&\n grep -q \"removed $(to_raw_cid $LEAF2)\" repo_gc_out\n '\n}\n\ntest_init_ipfs\n\ntest_gc_robust_part1\ntest_gc_robust_part2\n\ntest_launch_ipfs_daemon_without_network\n\ntest_gc_robust_part1\ntest_gc_robust_part2\n\ntest_kill_ipfs_daemon\n\ntest_done\n\n" }, { "path": "test/sharness/t0088-repo-stat-symlink.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 John Reed\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test 'ipfs repo stat' where IPFS_PATH is a symbolic link\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"create symbolic link for IPFS_PATH\" '\n mkdir sym_link_target &&\n ln -s sym_link_target .ipfs\n'\n\ntest_init_ipfs\n\n# ensure that the RepoSize is reasonable when checked via a symlink.\ntest_expect_success \"'ipfs repo stat' RepoSize is correct with sym link\" '\n reposize_symlink=$(ipfs repo stat | grep RepoSize | awk '\\''{ print $2 }'\\'') &&\n symlink_size=$(file_size .ipfs) &&\n test \"${reposize_symlink}\" -gt \"${symlink_size}\"\n'\n\ntest_done\n" }, { "path": "test/sharness/t0090-get.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Matt Bell\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test get command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_ipfs_get_flag() {\n ext=\"$1\"; shift\n tar_flag=\"$1\"; shift\n flag=\"$@\"\n\n test_expect_success \"ipfs get $flag succeeds\" '\n ipfs get \"$HASH\" '\"$flag\"' >actual\n '\n\n test_expect_success \"ipfs get $flag output looks good\" '\n printf \"%s\\n\" \"Saving archive to $HASH$ext\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs get $flag archive output is valid\" '\n tar \"$tar_flag\" \"$HASH$ext\" &&\n test_cmp \"$HASH\" data &&\n rm \"$HASH$ext\" &&\n rm \"$HASH\"\n '\n}\n\n# we use a function so that we can run it both offline + online\ntest_get_cmd() {\n\n test_expect_success \"'ipfs get --help' succeeds\" '\n ipfs get --help >actual\n '\n\n test_expect_success \"'ipfs get --help' output looks good\" '\n egrep \"ipfs get.*\" actual >/dev/null ||\n test_fsh cat actual\n '\n\n test_expect_success \"ipfs get succeeds\" '\n echo \"Hello Worlds!\" >data &&\n HASH=`ipfs add -q data` &&\n ipfs get \"$HASH\" >actual\n '\n\n test_expect_success \"ipfs get output looks good\" '\n printf \"%s\\n\" \"Saving file(s) to $HASH\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs get file output looks good\" '\n test_cmp \"$HASH\" data\n '\n\n test_expect_success \"ipfs get DOES NOT error when trying to overwrite a file\" '\n ipfs get \"$HASH\" >actual &&\n rm \"$HASH\"\n '\n\n test_expect_success \"ipfs get works with raw leaves\" '\n HASH2=$(ipfs add --raw-leaves -q data) &&\n ipfs get \"$HASH2\" >actual2\n '\n\n test_expect_success \"ipfs get output looks good\" '\n printf \"%s\\n\" \"Saving file(s) to $HASH2\" >expected2 &&\n test_cmp expected2 actual2\n '\n\n test_expect_success \"ipfs get file output looks good\" '\n test_cmp \"$HASH2\" data\n '\n\n test_ipfs_get_flag \".tar\" \"-xf\" -a\n\n test_ipfs_get_flag \".tar.gz\" \"-zxf\" -a -C\n\n test_ipfs_get_flag \".tar.gz\" \"-zxf\" -a -C -l 9\n\n test_expect_success \"ipfs get succeeds (directory)\" '\n mkdir -p dir &&\n touch dir/a &&\n mkdir -p dir/b &&\n echo \"Hello, Worlds!\" >dir/b/c &&\n HASH2=`ipfs add -r -Q dir` &&\n ipfs get \"$HASH2\" >actual\n '\n\n test_expect_success \"ipfs get output looks good (directory)\" '\n printf \"%s\\n\" \"Saving file(s) to $HASH2\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"ipfs get output is valid (directory)\" '\n test_cmp dir/a \"$HASH2\"/a &&\n test_cmp dir/b/c \"$HASH2\"/b/c &&\n rm -r \"$HASH2\"\n '\n\n # Test issue #4720: problems when path contains a trailing slash.\n test_expect_success \"ipfs get with slash (directory)\" '\n ipfs get \"$HASH2/\" &&\n test_cmp dir/a \"$HASH2\"/a &&\n test_cmp dir/b/c \"$HASH2\"/b/c &&\n rm -r \"$HASH2\"\n '\n\n test_expect_success \"ipfs get -a -C succeeds (directory)\" '\n ipfs get \"$HASH2\" -a -C >actual\n '\n\n test_expect_success \"ipfs get -a -C output looks good (directory)\" '\n printf \"%s\\n\" \"Saving archive to $HASH2.tar.gz\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"gzipped tar archive output is valid (directory)\" '\n tar -zxf \"$HASH2\".tar.gz &&\n test_cmp dir/a \"$HASH2\"/a &&\n test_cmp dir/b/c \"$HASH2\"/b/c &&\n rm -r \"$HASH2\"\n '\n\n test_expect_success \"ipfs get ../.. should fail\" '\n test_must_fail ipfs get ../.. 2>actual &&\n test_should_contain \"Error: invalid path \\\"../..\\\"\" actual\n '\n\n test_expect_success \"create small file\" '\n echo \"foo\" > small &&\n ipfs add -q small > hash_small\n '\n\n test_expect_success \"get small file\" '\n ipfs get -o out_small $(cat hash_small) &&\n test_cmp small out_small\n '\n\n test_expect_success \"create medium file\" '\n head -c 16000 > medium &&\n ipfs add -q medium > hash_medium\n '\n\n test_expect_success \"get medium file\" '\n ipfs get -o out_medium $(cat hash_medium) &&\n test_cmp medium out_medium\n '\n}\n\ntest_get_fail() {\n test_expect_success \"create an object that has unresolvable links\" '\n cat <<-\\EOF >bad_object &&\n{\"Data\":{\"/\":{\"bytes\":\"CAE\"}},\"Links\":[{\"Hash\":{\"/\":\"Qmd4mG6pDFDmDTn6p3hX1srP8qTbkyXKj5yjpEsiHDX3u8\"},\"Name\":\"bar\",\"Tsize\":56},{\"Hash\":{\"/\":\"QmUTjwRnG28dSrFFVTYgbr6LiDLsBmRr2SaUSTGheK2YqG\"},\"Name\":\"baz\",\"Tsize\":24266},{\"Hash\":{\"/\":\"QmZzaC6ydNXiR65W8VjGA73ET9MZ6VFAqUT1ngYMXcpihn\"},\"Name\":\"foo\",\"Tsize\":1897}]}\nEOF\n cat bad_object | ipfs dag put --store-codec dag-pb > put_out\n '\n\n test_expect_success \"output looks good\" '\n echo \"bafybeifrjjol3gixedca6etdwccnvwfvhurc4wb3i5mnk2rvwvyfcgwxd4\" > put_exp &&\n test_cmp put_exp put_out\n '\n\n test_expect_success \"ipfs get fails\" '\n test_expect_code 1 ipfs get QmaGidyrnX8FMbWJoxp8HVwZ1uRKwCyxBJzABnR1S2FVUr\n '\n}\n\n# should work offline\ntest_get_cmd\n\n# only really works offline, will try and search network when online\ntest_get_fail\n\n# should work online\ntest_launch_ipfs_daemon\ntest_get_cmd\n\ntest_expect_success \"empty request to get doesn't panic and returns error\" '\n curl -X POST \"http://$API_ADDR/api/v0/get\" > curl_out || true &&\n grep \"argument \\\"ipfs-path\\\" is required\" curl_out\n'\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0095-refs.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2018 Protocol Labs, Inc\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test 'ipfs refs' command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\ntest_launch_ipfs_daemon_without_network\n\n# This file performs tests with the following directory\n# structure.\n#\n# L0- _______ A_________\n# / | \\ \\\n# L1- B C D 1.txt\n# / \\ | |\n# L2- D 1.txt B 2.txt\n# | / \\\n# L3- 2.txt D 1.txt\n# |\n# L4- 2.txt\n#\n# 'ipfs add -r A' output:\n#\n# added QmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v A/1.txt\n# added QmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v A/B/1.txt\n# added QmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61 A/B/D/2.txt\n# added QmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v A/C/B/1.txt\n# added QmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61 A/C/B/D/2.txt\n# added QmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61 A/D/2.txt\n# added QmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS A/B/D\n# added QmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa A/B\n# added QmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS A/C/B/D\n# added QmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa A/C/B\n# added QmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH A/C\n# added QmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS A/D\n# added QmU6xujRsYzcrkocuR3fhfnkZBB8eyUFFq4WKRGw2aS15h A\n#\n# 'ipfs refs -r QmU6xujRsYzcrkocuR3fhfnkZBB8eyUFFq4WKRGw2aS15h' sample output\n# that shows visit order in a stable go-ipfs version:\n#\n# QmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v - 1.txt\n# QmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa - B (A/B)\n# QmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v - 1.txt (A/B/1.txt)\n# QmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS - D (A/B/D)\n# QmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61 - 2.txt (A/B/D/2.txt)\n# QmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH - C (A/C)\n# QmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa - B (A/C/B)\n# QmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v - 1.txt (A/C/B/1.txt)\n# QmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS - D (A/C/B/D)\n# QmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61 - 2.txt (A/C/B/D/2.txt)\n# QmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS - D (A/D)\n# QmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61 - 2.txt (A/D/2.txt)\n\n\nrefsroot=QmU6xujRsYzcrkocuR3fhfnkZBB8eyUFFq4WKRGw2aS15h\n\ntest_expect_success \"create and add folders for refs\" '\n mkdir -p A/B/D A/C/B/D A/D\n echo \"1\" > A/1.txt\n echo \"1\" > A/B/1.txt\n echo \"1\" > A/C/B/1.txt\n echo \"2\" > A/B/D/2.txt\n echo \"2\" > A/C/B/D/2.txt\n echo \"2\" > A/D/2.txt\n root=$(ipfs add -r -Q A)\n [[ \"$root\" == \"$refsroot\" ]]\n'\n\ntest_refs_output() {\n ARGS=$1\n FILTER=$2\n\n test_expect_success \"ipfs refs $ARGS -r\" '\n cat < expected.txt\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61\nQmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61\nEOF\n\n ipfs refs $ARGS -r $refsroot > refsr.txt\n test_cmp expected.txt refsr.txt\n '\n\n # Unique is like above but removing duplicates\n test_expect_success \"ipfs refs $ARGS -r --unique\" '\n cat < expected.txt\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61\nQmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH\nEOF\n\n ipfs refs $ARGS -r --unique $refsroot > refsr.txt\n test_cmp expected.txt refsr.txt\n '\n\n # First level is 1.txt, B, C, D\n test_expect_success \"ipfs refs $ARGS\" '\n cat < expected.txt\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nEOF\n ipfs refs $ARGS $refsroot > refs.txt\n test_cmp expected.txt refs.txt\n '\n\n # max-depth=0 should return an empty list\n test_expect_success \"ipfs refs $ARGS -r --max-depth=0\" '\n cat < expected.txt\nEOF\n ipfs refs $ARGS -r --max-depth=0 $refsroot > refs.txt\n test_cmp expected.txt refs.txt\n '\n\n # max-depth=1 should be equivalent to running without -r\n test_expect_success \"ipfs refs $ARGS -r --max-depth=1\" '\n ipfs refs $ARGS -r --max-depth=1 $refsroot > refsr.txt\n ipfs refs $ARGS $refsroot > refs.txt\n test_cmp refsr.txt refs.txt\n '\n\n # We should see the depth limit engage at level 2\n test_expect_success \"ipfs refs $ARGS -r --max-depth=2\" '\n cat < expected.txt\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61\nEOF\n ipfs refs $ARGS -r --max-depth=2 $refsroot > refsr.txt\n test_cmp refsr.txt expected.txt\n '\n\n # Here branch pruning and re-exploration come into place\n # At first it should see D at level 2 and don't go deeper.\n # But then after doing C it will see D at level 1 and go deeper\n # so that it outputs the hash for 2.txt (-q61).\n # We also see that C/B is pruned as it's been shown before.\n #\n # Excerpt from diagram above:\n #\n # L0- _______ A_________\n # / | \\ \\\n # L1- B C D 1.txt\n # / \\ | |\n # L2- D 1.txt B 2.txt\n test_expect_success \"ipfs refs $ARGS -r --unique --max-depth=2\" '\n cat < expected.txt\nQmdytmR4wULMd3SLo6ePF4s3WcRHWcpnJZ7bHhoj3QB13v\nQmNkQvpiyAEtbeLviC7kqfifYoK1GXPcsSxTpP1yS3ykLa\nQmSanP5DpxpqfDdS4yekHY1MqrVge47gtxQcp2e2yZ4UwS\nQmXXazTjeNCKFnpW1D65vTKsTs8fbgkCWTv8Em4pdK2coH\nQmSFxnK675wQ9Kc1uqWKyJUaNxvSc2BP5DbXCD3x93oq61\nEOF\n ipfs refs $ARGS -r --unique --max-depth=2 $refsroot > refsr.txt\n test_cmp refsr.txt expected.txt\n '\n}\n\ntest_refs_output '' 'cat'\n\ntest_refs_output '--cid-base=base32' 'ipfs cid base32'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0101-iptb-name.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs repo operations\"\n\n. lib/test-lib.sh\n\nnum_nodes=4\n\ntest_expect_success \"set up an iptb cluster\" '\n iptb testbed create -type localipfs -count $num_nodes -force -init\n'\n\nstartup_cluster $num_nodes\n\ntest_expect_success \"add an object on one node\" '\n echo \"ipns is super fun\" > file &&\n HASH_FILE=$(ipfsi 1 add -q file)\n'\n\ntest_expect_success \"publish that object as an ipns entry\" '\n ipfsi 1 name publish $HASH_FILE\n'\n\ntest_expect_success \"add an entry on another node pointing to that one\" '\n NODE1_ID=$(iptb attr get 1 id) &&\n ipfsi 2 name publish /ipns/$NODE1_ID\n'\n\ntest_expect_success \"cat that entry on a third node\" '\n NODE2_ID=$(iptb attr get 2 id) &&\n ipfsi 3 cat /ipns/$NODE2_ID > output\n'\n\ntest_expect_success \"ensure output was the same\" '\n test_cmp file output\n'\n\ntest_expect_success \"shut down iptb\" '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0109-gateway-web-_redirects.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test HTTP Gateway _redirects support\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\ntest_launch_ipfs_daemon\n\n## ============================================================================\n## Test _redirects file support\n## ============================================================================\n\n# Import test case\n# Run `ipfs cat /ipfs/$REDIRECTS_DIR_CID/_redirects` to see sample _redirects file\ntest_expect_success \"Add the _redirects file test directory\" '\n ipfs dag import --pin-roots ../t0109-gateway-web-_redirects-data/redirects.car\n'\nCAR_ROOT_CID=QmQyqMY5vUBSbSxyitJqthgwZunCQjDVtNd8ggVCxzuPQ4\n\nREDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/examples | cut -d \"/\" -f3)\nREDIRECTS_DIR_HOSTNAME=\"${REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/redirect-one redirects with default of 301, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/redirect-one\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /one.html\" response\n'\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/301-redirect-one redirects with 301, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/301-redirect-one\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /one.html\" response\n'\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/302-redirect-two redirects with 302, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/302-redirect-two\" > response &&\n test_should_contain \"302 Found\" response &&\n test_should_contain \"Location: /two.html\" response\n'\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/200-index returns 200, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/200-index\" > response &&\n test_should_contain \"my index\" response &&\n test_should_contain \"200 OK\" response\n'\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/posts/:year/:month/:day/:title redirects with 301 and placeholders, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/posts/2022/01/01/hello-world\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /articles/2022/01/01/hello-world\" response\n'\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/splat/one.html redirects with 301 and splat placeholder, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/splat/one.html\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /redirected-splat/one.html\" response\n'\n\n# ensure custom 4xx works and has the same cache headers as regular /ipfs/ path\nCUSTOM_4XX_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/examples/404.html | cut -d \"/\" -f3)\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/not-found/has-no-redirects-entry returns custom 404, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/not-found/has-no-redirects-entry\" > response &&\n test_should_contain \"404 Not Found\" response &&\n test_should_contain \"Cache-Control: public, max-age=29030400, immutable\" response &&\n test_should_contain \"Etag: \\\"$CUSTOM_4XX_CID\\\"\" response &&\n test_should_contain \"my 404\" response\n'\n\nCUSTOM_4XX_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/examples/410.html | cut -d \"/\" -f3)\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/gone/has-no-redirects-entry returns custom 410, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/gone/has-no-redirects-entry\" > response &&\n test_should_contain \"410 Gone\" response &&\n test_should_contain \"Cache-Control: public, max-age=29030400, immutable\" response &&\n test_should_contain \"Etag: \\\"$CUSTOM_4XX_CID\\\"\" response &&\n test_should_contain \"my 410\" response\n'\n\nCUSTOM_4XX_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/examples/451.html | cut -d \"/\" -f3)\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/unavail/has-no-redirects-entry returns custom 451, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/unavail/has-no-redirects-entry\" > response &&\n test_should_contain \"451 Unavailable For Legal Reasons\" response &&\n test_should_contain \"Cache-Control: public, max-age=29030400, immutable\" response &&\n test_should_contain \"Etag: \\\"$CUSTOM_4XX_CID\\\"\" response &&\n test_should_contain \"my 451\" response\n'\n\ntest_expect_success \"request for $REDIRECTS_DIR_HOSTNAME/catch-all returns 200, per _redirects file\" '\n curl -sD - --resolve $REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$REDIRECTS_DIR_HOSTNAME/catch-all\" > response &&\n test_should_contain \"200 OK\" response &&\n test_should_contain \"my index\" response\n'\n\n# This test ensures _redirects is supported only on Web Gateways that use Host header (DNSLink, Subdomain)\ntest_expect_success \"request for http://127.0.0.1:$GWAY_PORT/ipfs/$REDIRECTS_DIR_CID/301-redirect-one returns generic 404 (no custom 404 from _redirects since no origin isolation)\" '\n curl -sD - \"http://127.0.0.1:$GWAY_PORT/ipfs/$REDIRECTS_DIR_CID/301-redirect-one\" > response &&\n test_should_contain \"404 Not Found\" response &&\n test_should_not_contain \"my 404\" response\n'\n\n# With CRLF line terminator\nNEWLINE_REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/newlines | cut -d \"/\" -f3)\nNEWLINE_REDIRECTS_DIR_HOSTNAME=\"${NEWLINE_REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\ntest_expect_success \"newline: _redirects has CRLF line terminators\" '\n ipfs cat /ipfs/$NEWLINE_REDIRECTS_DIR_CID/_redirects | file - > response &&\n test_should_contain \"with CRLF line terminators\" response\n'\n\ntest_expect_success \"newline: request for $NEWLINE_REDIRECTS_DIR_HOSTNAME/redirect-one redirects with default of 301, per _redirects file\" '\n curl -sD - --resolve $NEWLINE_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$NEWLINE_REDIRECTS_DIR_HOSTNAME/redirect-one\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /one.html\" response\n'\n\n# Good codes\nGOOD_REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/good-codes | cut -d \"/\" -f3)\nGOOD_REDIRECTS_DIR_HOSTNAME=\"${GOOD_REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\ntest_expect_success \"good codes: request for $GOOD_REDIRECTS_DIR_HOSTNAME/redirect-one redirects with default of 301, per _redirects file\" '\n curl -sD - --resolve $GOOD_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$GOOD_REDIRECTS_DIR_HOSTNAME/a301\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /b301\" response\n'\n\n# Bad codes\nBAD_REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/bad-codes | cut -d \"/\" -f3)\nBAD_REDIRECTS_DIR_HOSTNAME=\"${BAD_REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\n# if accessing a path that doesn't exist, read _redirects and fail parsing, and return error\ntest_expect_success \"bad codes: request for $BAD_REDIRECTS_DIR_HOSTNAME/not-found returns error about bad code\" '\n curl -sD - --resolve $BAD_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$BAD_REDIRECTS_DIR_HOSTNAME/not-found\" > response &&\n test_should_contain \"500\" response &&\n test_should_contain \"status code 999 is not supported\" response\n'\n\n# if accessing a path that does exist, don't read _redirects and therefore don't fail parsing\ntest_expect_success \"bad codes: request for $BAD_REDIRECTS_DIR_HOSTNAME/found.html doesn't return error about bad code\" '\n curl -sD - --resolve $BAD_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$BAD_REDIRECTS_DIR_HOSTNAME/found.html\" > response &&\n test_should_contain \"200\" response &&\n test_should_contain \"my found\" response &&\n test_should_not_contain \"unsupported redirect status\" response\n'\n\n# Invalid file, containing \"hello\"\nINVALID_REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/invalid | cut -d \"/\" -f3)\nINVALID_REDIRECTS_DIR_HOSTNAME=\"${INVALID_REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\n# if accessing a path that doesn't exist, read _redirects and fail parsing, and return error\ntest_expect_success \"invalid file: request for $INVALID_REDIRECTS_DIR_HOSTNAME/not-found returns error about invalid redirects file\" '\n curl -sD - --resolve $INVALID_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$INVALID_REDIRECTS_DIR_HOSTNAME/not-found\" > response &&\n test_should_contain \"500\" response &&\n test_should_contain \"could not parse _redirects:\" response\n'\n\n# Invalid file, containing forced redirect\nINVALID_REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/forced | cut -d \"/\" -f3)\nINVALID_REDIRECTS_DIR_HOSTNAME=\"${INVALID_REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\n# if accessing a path that doesn't exist, read _redirects and fail parsing, and return error\ntest_expect_success \"invalid file: request for $INVALID_REDIRECTS_DIR_HOSTNAME/not-found returns error about invalid redirects file\" '\n curl -sD - --resolve $INVALID_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$INVALID_REDIRECTS_DIR_HOSTNAME/not-found\" > response &&\n test_should_contain \"500\" response &&\n test_should_contain \"could not parse _redirects:\" response &&\n test_should_contain \"forced redirects (or \\\"shadowing\\\") are not supported\" response\n'\n\n# if accessing a path that doesn't exist and _redirects file is too large, return error\nTOO_LARGE_REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/too-large | cut -d \"/\" -f3)\nTOO_LARGE_REDIRECTS_DIR_HOSTNAME=\"${TOO_LARGE_REDIRECTS_DIR_CID}.ipfs.localhost:$GWAY_PORT\"\ntest_expect_success \"invalid file: request for $TOO_LARGE_REDIRECTS_DIR_HOSTNAME/not-found returns error about too large redirects file\" '\n curl -sD - --resolve $TOO_LARGE_REDIRECTS_DIR_HOSTNAME:127.0.0.1 \"http://$TOO_LARGE_REDIRECTS_DIR_HOSTNAME/not-found\" > response &&\n test_should_contain \"500\" response &&\n test_should_contain \"could not parse _redirects:\" response &&\n test_should_contain \"redirects file size cannot exceed\" response\n'\n\ntest_kill_ipfs_daemon\n\n# disable wildcard DNSLink gateway\n# and enable it on specific DNSLink hostname\nipfs config --json Gateway.NoDNSLink true && \\\nipfs config --json Gateway.PublicGateways '{\n \"dnslink-enabled-on-fqdn.example.org\": {\n \"NoDNSLink\": false,\n \"UseSubdomains\": false,\n \"Paths\": [\"/ipfs\"]\n },\n \"dnslink-disabled-on-fqdn.example.com\": {\n \"NoDNSLink\": true,\n \"UseSubdomains\": false,\n \"Paths\": []\n }\n}' || exit 1\n\n# DNSLink test requires a daemon in online mode with precached /ipns/ mapping\n# REDIRECTS_DIR_CID=$(ipfs resolve -r /ipfs/$CAR_ROOT_CID/examples | cut -d \"/\" -f3)\nDNSLINK_FQDN=\"dnslink-enabled-on-fqdn.example.org\"\nNO_DNSLINK_FQDN=\"dnslink-disabled-on-fqdn.example.com\"\nexport IPFS_NS_MAP=\"$DNSLINK_FQDN:/ipfs/$REDIRECTS_DIR_CID\"\n\n# restart daemon to apply config changes\ntest_launch_ipfs_daemon\n\n# make sure test setup is valid (fail if CoreAPI is unable to resolve)\ntest_expect_success \"spoofed DNSLink record resolves in cli\" \"\n ipfs resolve /ipns/$DNSLINK_FQDN > result &&\n test_should_contain \\\"$REDIRECTS_DIR_CID\\\" result &&\n ipfs cat /ipns/$DNSLINK_FQDN/_redirects > result &&\n test_should_contain \\\"index.html\\\" result\n\"\n\ntest_expect_success \"request for $DNSLINK_FQDN/redirect-one redirects with default of 301, per _redirects file\" '\n curl -sD - --resolve $DNSLINK_FQDN:$GWAY_PORT:127.0.0.1 \"http://$DNSLINK_FQDN:$GWAY_PORT/redirect-one\" > response &&\n test_should_contain \"301 Moved Permanently\" response &&\n test_should_contain \"Location: /one.html\" response\n'\n\n# ensure custom 404 works and has the same cache headers as regular /ipns/ paths\ntest_expect_success \"request for $DNSLINK_FQDN/en/has-no-redirects-entry returns custom 404, per _redirects file\" '\n curl -sD - --resolve $DNSLINK_FQDN:$GWAY_PORT:127.0.0.1 \"http://$DNSLINK_FQDN:$GWAY_PORT/not-found/has-no-redirects-entry\" > response &&\n test_should_contain \"404 Not Found\" response &&\n test_should_contain \"Etag: \\\"Qmd9GD7Bauh6N2ZLfNnYS3b7QVAijbud83b8GE8LPMNBBP\\\"\" response &&\n test_should_not_contain \"Cache-Control: public, max-age=29030400, immutable\" response &&\n test_should_not_contain \"immutable\" response &&\n test_should_contain \"Date: \" response &&\n test_should_contain \"my 404\" response\n'\n\ntest_expect_success \"request for $NO_DNSLINK_FQDN/redirect-one does not redirect, since DNSLink is disabled\" '\n curl -sD - --resolve $NO_DNSLINK_FQDN:$GWAY_PORT:127.0.0.1 \"http://$NO_DNSLINK_FQDN:$GWAY_PORT/redirect-one\" > response &&\n test_should_not_contain \"one.html\" response &&\n test_should_not_contain \"301 Moved Permanently\" response &&\n test_should_not_contain \"Location:\" response\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0112-gateway-cors.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test CORS behavior on Gateway port\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# Default config\ntest_expect_success \"Default Gateway.HTTPHeaders is empty (implicit CORS values from boxo/gateway)\" '\ncat < expected\n{}\nEOF\n ipfs config --json Gateway.HTTPHeaders > actual &&\n test_cmp expected actual\n'\n\ntest_launch_ipfs_daemon\n\nthash='bafkqabtimvwgy3yk' # hello\n\n# Gateway\n\n# HTTP GET Request\ntest_expect_success \"GET to Gateway succeeds\" '\n curl -svX GET -H \"Origin: https://example.com\" \"http://127.0.0.1:$GWAY_PORT/ipfs/$thash\" >/dev/null 2>curl_output &&\n cat curl_output\n'\n\n# GET Response from Gateway should contain CORS headers\ntest_expect_success \"GET response for Gateway resource looks good\" '\n test_should_contain \"< Access-Control-Allow-Origin: \\*\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: GET\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: HEAD\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: OPTIONS\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: Content-Type\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: Range\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: User-Agent\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: X-Requested-With\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Range\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Length\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Chunked-Output\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Stream-Output\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Ipfs-Path\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Ipfs-Roots\" curl_output\n'\n# HTTP OPTIONS Request\ntest_expect_success \"OPTIONS to Gateway succeeds\" '\n curl -svX OPTIONS -H \"Origin: https://example.com\" \"http://127.0.0.1:$GWAY_PORT/ipfs/$thash\" 2>curl_output &&\n cat curl_output\n'\n\n# OPTION Response from Gateway should contain CORS headers\ntest_expect_success \"OPTIONS response for Gateway resource looks good\" '\n test_should_contain \"< Access-Control-Allow-Origin: \\*\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: GET\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: HEAD\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: OPTIONS\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: Content-Type\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: Range\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: User-Agent\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: X-Requested-With\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Range\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Length\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Chunked-Output\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Stream-Output\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Ipfs-Path\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Ipfs-Roots\" curl_output\n'\n\n# HTTP OPTIONS Request on path → subdomain HTTP 301 redirect\n# (regression test for https://github.com/ipfs/kubo/issues/9983#issuecomment-1599673976)\ntest_expect_success \"OPTIONS to Gateway succeeds\" '\n curl -svX OPTIONS -H \"Origin: https://example.com\" \"http://localhost:$GWAY_PORT/ipfs/$thash\" 2>curl_output &&\n cat curl_output\n'\n# OPTION Response from Gateway should contain CORS headers\ntest_expect_success \"OPTIONS response for subdomain redirect looks good\" '\n test_should_contain \"HTTP/1.1 301 Moved Permanently\" curl_output &&\n test_should_contain \"Location\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Origin: \\*\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Methods: GET\" curl_output\n'\n\ntest_kill_ipfs_daemon\n\n# Test CORS safelisting of custom headers\ntest_expect_success \"Can configure gateway headers\" '\n ipfs config --json Gateway.HTTPHeaders.Access-Control-Allow-Headers \"[\\\"X-Custom1\\\"]\" &&\n ipfs config --json Gateway.HTTPHeaders.Access-Control-Expose-Headers \"[\\\"X-Custom2\\\"]\" &&\n ipfs config --json Gateway.HTTPHeaders.Access-Control-Allow-Origin \"[\\\"localhost\\\"]\"\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"OPTIONS to Gateway without custom headers succeeds\" '\n curl -svX OPTIONS -H \"Origin: https://example.com\" \"http://127.0.0.1:$GWAY_PORT/ipfs/$thash\" 2>curl_output &&\n cat curl_output\n'\n# Range and Content-Range are safelisted by default, and keeping them makes better devexp\n# because it does not cause regressions in range requests made by JS\ntest_expect_success \"Access-Control-Allow-Headers extends the implicit list\" '\n test_should_contain \"< Access-Control-Allow-Headers: Range\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: X-Custom1\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Range\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Length\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Ipfs-Path\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Ipfs-Roots\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Custom2\" curl_output\n'\n\ntest_expect_success \"OPTIONS to Gateway with a custom header succeeds\" '\n curl -svX OPTIONS -H \"Origin: https://example.com\" -H \"Access-Control-Request-Headers: X-Unexpected-Custom\" \"http://127.0.0.1:$GWAY_PORT/ipfs/$thash\" 2>curl_output &&\n cat curl_output\n'\ntest_expect_success \"Access-Control-Allow-Headers extends the implicit list\" '\n test_should_not_contain \"< Access-Control-Allow-Headers: X-Unexpected-Custom\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: Range\" curl_output &&\n test_should_contain \"< Access-Control-Allow-Headers: X-Custom1\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: Content-Range\" curl_output &&\n test_should_contain \"< Access-Control-Expose-Headers: X-Custom2\" curl_output\n'\n\n# Origin is sensitive security perimeter, and we assume override should remove\n# any implicit records\ntest_expect_success \"Access-Control-Allow-Origin replaces the implicit list\" '\n test_should_contain \"< Access-Control-Allow-Origin: localhost\" curl_output\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0114-gateway-subdomains/README.md", "content": "# Dataset description/sources\n\n- fixtures.car\n - raw CARv1\n\n- QmUKd....ipns-record\n - ipns record, encoded with protocol buffer\n\n- 12D3K....ipns-record\n - ipns record, encoded with protocol buffer\n\nGenerated with:\n\n```sh\n# using ipfs version 0.21.0-dev (03a98280e3e642774776cd3d0435ab53e5dfa867)\n\n# CIDv0to1 is necessary because raw-leaves are enabled by default during\n# \"ipfs add\" with CIDv1 and disabled with CIDv0\nCID_VAL=\"hello\"\nCIDv1=$(echo $CID_VAL | ipfs add --cid-version 1 -Q)\nCIDv0=$(echo $CID_VAL | ipfs add --cid-version 0 -Q)\nCIDv0to1=$(echo \"$CIDv0\" | ipfs cid base32)\n# sha512 will be over 63char limit, even when represented in Base36\nCIDv1_TOO_LONG=$(echo $CID_VAL | ipfs add --cid-version 1 --hash sha2-512 -Q)\n\necho CID_VAL=${CID_VAL}\necho CIDv1=${CIDv1}\necho CIDv0=${CIDv0}\necho CIDv0to1=${CIDv0to1}\necho CIDv1_TOO_LONG=${CIDv1_TOO_LONG}\n\n# Directory tree crafted to test for edge cases like \"/ipfs/ipfs/ipns/bar\"\nmkdir -p testdirlisting/ipfs/ipns &&\necho \"hello\" > testdirlisting/hello &&\necho \"text-file-content\" > testdirlisting/ipfs/ipns/bar &&\nmkdir -p testdirlisting/api &&\nmkdir -p testdirlisting/ipfs &&\necho \"I am a txt file\" > testdirlisting/api/file.txt &&\necho \"I am a txt file\" > testdirlisting/ipfs/file.txt &&\nDIR_CID=$(ipfs add -Qr --cid-version 1 testdirlisting)\n\necho DIR_CID=${DIR_CID} # ./testdirlisting\n\nipfs files mkdir /t0114/\nipfs files cp /ipfs/${CIDv1} /t0114/\nipfs files cp /ipfs/${CIDv0} /t0114/\nipfs files cp /ipfs/${CIDv0to1} /t0114/\nipfs files cp /ipfs/${DIR_CID} /t0114/\nipfs files cp /ipfs/${CIDv1_TOO_LONG} /t0114/\n\nROOT=`ipfs files stat /t0114/ --hash`\n\nipfs dag export ${ROOT} > ./fixtures.car\n\n# Then the keys\n\nKEY_NAME=test_key_rsa_$RANDOM\nRSA_KEY=$(ipfs key gen --ipns-base=b58mh --type=rsa --size=2048 ${KEY_NAME} | head -n1 | tr -d \"\\n\")\nRSA_IPNS_IDv0=$(echo \"$RSA_KEY\" | ipfs cid format -v 0)\nRSA_IPNS_IDv1=$(echo \"$RSA_KEY\" | ipfs cid format -v 1 --mc libp2p-key -b base36)\nRSA_IPNS_IDv1_DAGPB=$(echo \"$RSA_IPNS_IDv0\" | ipfs cid format -v 1 -b base36)\n\n# publish a record valid for a 100 years\nipfs name publish --key ${KEY_NAME} --allow-offline -Q --ttl=876600h --lifetime=876600h \"/ipfs/$CIDv1\"\nipfs routing get /ipns/${RSA_KEY} > ${RSA_KEY}.ipns-record\n\necho RSA_KEY=${RSA_KEY}\necho RSA_IPNS_IDv0=${RSA_IPNS_IDv0}\necho RSA_IPNS_IDv1=${RSA_IPNS_IDv1}\necho RSA_IPNS_IDv1_DAGPB=${RSA_IPNS_IDv1_DAGPB}\n\nKEY_NAME=test_key_ed25519_$RANDOM\nED25519_KEY=$(ipfs key gen --ipns-base=b58mh --type=ed25519 ${KEY_NAME} | head -n1 | tr -d \"\\n\")\nED25519_IPNS_IDv0=$ED25519_KEY\nED25519_IPNS_IDv1=$(ipfs key list -l --ipns-base=base36 | grep ${KEY_NAME} | cut -d \" \" -f1 | tr -d \"\\n\")\nED25519_IPNS_IDv1_DAGPB=$(echo \"$ED25519_IPNS_IDv1\" | ipfs cid format -v 1 -b base36 --mc dag-pb)\n\n# ed25519 fits under 63 char limit when represented in base36\nIPNS_ED25519_B58MH=$(ipfs key list -l --ipns-base b58mh | grep $KEY_NAME | cut -d\" \" -f1 | tr -d \"\\n\")\nIPNS_ED25519_B36CID=$(ipfs key list -l --ipns-base base36 | grep $KEY_NAME | cut -d\" \" -f1 | tr -d \"\\n\")\n\n# publish a record valid for a 100 years\nipfs name publish --key ${KEY_NAME} --allow-offline -Q --ttl=876600h --lifetime=876600h \"/ipfs/$CIDv1\"\nipfs routing get /ipns/${ED25519_KEY} > ${ED25519_KEY}.ipns-record\n\necho ED25519_KEY=${ED25519_KEY}\necho ED25519_IPNS_IDv0=${ED25519_IPNS_IDv0}\necho ED25519_IPNS_IDv1=${ED25519_IPNS_IDv1}\necho ED25519_IPNS_IDv1_DAGPB=${ED25519_IPNS_IDv1_DAGPB}\necho IPNS_ED25519_B58MH=${IPNS_ED25519_B58MH}\necho IPNS_ED25519_B36CID=${IPNS_ED25519_B36CID}\n\n# CID_VAL=hello\n# CIDv1=bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am\n# CIDv0=QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\n# CIDv0to1=bafybeiffndsajwhk3lwjewwdxqntmjm4b5wxaaanokonsggenkbw6slwk4\n# CIDv1_TOO_LONG=bafkrgqhhyivzstcz3hhswshfjgy6ertgmnqeleynhwt4dlfsthi4hn7zgh4uvlsb5xncykzapi3ocd4lzogukir6ksdy6wzrnz6ohnv4aglcs\n# DIR_CID=bafybeiht6dtwk3les7vqm6ibpvz6qpohidvlshsfyr7l5mpysdw2vmbbhe # ./testdirlisting\n\n# RSA_KEY=QmVujd5Vb7moysJj8itnGufN7MEtPRCNHkKpNuA4onsRa3\n# RSA_IPNS_IDv0=QmVujd5Vb7moysJj8itnGufN7MEtPRCNHkKpNuA4onsRa3\n# RSA_IPNS_IDv1=k2k4r8m7xvggw5pxxk3abrkwyer625hg01hfyggrai7lk1m63fuihi7w\n# RSA_IPNS_IDv1_DAGPB=k2jmtxu61bnhrtj301lw7zizknztocdbeqhxgv76l2q9t36fn9jbzipo\n\n# ED25519_KEY=12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d\n# ED25519_IPNS_IDv0=12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d\n# ED25519_IPNS_IDv1=k51qzi5uqu5dk3v4rmjber23h16xnr23bsggmqqil9z2gduiis5se8dht36dam\n# ED25519_IPNS_IDv1_DAGPB=k50rm9yjlt0jey4fqg6wafvqprktgbkpgkqdg27tpqje6iimzxewnhvtin9hhq\n# IPNS_ED25519_B58MH=12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d\n# IPNS_ED25519_B36CID=k51qzi5uqu5dk3v4rmjber23h16xnr23bsggmqqil9z2gduiis5se8dht36dam\n```\n" }, { "path": "test/sharness/t0114-gateway-subdomains.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) Protocol Labs\n\ntest_description=\"Test subdomain support on the HTTP gateway\"\n\n\n. lib/test-lib.sh\n\n## ============================================================================\n## Helpers specific to subdomain tests\n## ============================================================================\n\n# Helper that tests gateway response over direct HTTP\n# and in all supported HTTP proxy modes\ntest_localhost_gateway_response_should_contain() {\n local label=\"$1\"\n local expected=\"$3\"\n\n # explicit \"Host: $hostname\" header to match browser behavior\n # and also make tests independent from DNS\n local host=$(echo $2 | cut -d'/' -f3 | cut -d':' -f1)\n local hostname=$(echo $2 | cut -d'/' -f3 | cut -d':' -f1,2)\n\n # Proxy is the same as HTTP Gateway, we use raw IP and port to be sure\n local proxy=\"http://127.0.0.1:$GWAY_PORT\"\n\n # Create a raw URL version with IP to ensure hostname from Host header is used\n # (removes false-positives, Host header is used for passing hostname already)\n local url=\"$2\"\n local rawurl=$(echo \"$url\" | sed \"s/$hostname/127.0.0.1:$GWAY_PORT/\")\n\n #echo \"hostname: $hostname\"\n #echo \"url before: $url\"\n #echo \"url after: $rawurl\"\n\n # regular HTTP request\n # (hostname in Host header, raw IP in URL)\n test_expect_success \"$label (direct HTTP)\" \"\n curl -H \\\"Host: $hostname\\\" -sD - \\\"$rawurl\\\" > response &&\n test_should_contain \\\"$expected\\\" response\n \"\n\n # HTTP proxy\n # (hostname is passed via URL)\n # Note: proxy client should not care, but curl does DNS lookup\n # for some reason anyway, so we pass static DNS mapping\n test_expect_success \"$label (HTTP proxy)\" \"\n curl -x $proxy --resolve $hostname:127.0.0.1 -sD - \\\"$url\\\" > response &&\n test_should_contain \\\"$expected\\\" response\n \"\n\n # HTTP proxy 1.0\n # (repeating proxy test with older spec, just to be sure)\n test_expect_success \"$label (HTTP proxy 1.0)\" \"\n curl --proxy1.0 $proxy --resolve $hostname:127.0.0.1 -sD - \\\"$url\\\" > response &&\n test_should_contain \\\"$expected\\\" response\n \"\n\n # HTTP proxy tunneling (CONNECT)\n # https://tools.ietf.org/html/rfc7231#section-4.3.6\n # In HTTP/1.x, the pseudo-method CONNECT\n # can be used to convert an HTTP connection into a tunnel to a remote host\n test_expect_success \"$label (HTTP proxy tunneling)\" \"\n curl --proxytunnel -x $proxy -H \\\"Host: $hostname\\\" -sD - \\\"$rawurl\\\" > response &&\n test_should_contain \\\"$expected\\\" response\n \"\n}\n\n# Helper that checks gateway response for specific hostname in Host header\ntest_hostname_gateway_response_should_contain() {\n local label=\"$1\"\n local hostname=\"$2\"\n local url=\"$3\"\n local rawurl=$(echo \"$url\" | sed \"s/$hostname/127.0.0.1:$GWAY_PORT/\")\n local expected=\"$4\"\n test_expect_success \"$label\" \"\n curl -H \\\"Host: $hostname\\\" -sD - \\\"$rawurl\\\" > response &&\n test_should_contain \\\"$expected\\\" response\n \"\n}\n\n## ============================================================================\n## Start IPFS Node and prepare test CIDs\n## ============================================================================\n\ntest_expect_success \"ipfs init\" '\n export IPFS_PATH=\"$(pwd)/.ipfs\" &&\n ipfs init --profile=test > /dev/null\n'\n\ntest_launch_ipfs_daemon_without_network\n\n# Import test case\n# See the static fixtures in ./t0114-gateway-subdomains/\nCID_VAL=hello\nCIDv1=bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am\nCIDv0=QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN\nCIDv0to1=bafybeiffndsajwhk3lwjewwdxqntmjm4b5wxaaanokonsggenkbw6slwk4\nCIDv1_TOO_LONG=bafkrgqhhyivzstcz3hhswshfjgy6ertgmnqeleynhwt4dlfsthi4hn7zgh4uvlsb5xncykzapi3ocd4lzogukir6ksdy6wzrnz6ohnv4aglcs\nDIR_CID=bafybeiht6dtwk3les7vqm6ibpvz6qpohidvlshsfyr7l5mpysdw2vmbbhe\n\nRSA_KEY=QmVujd5Vb7moysJj8itnGufN7MEtPRCNHkKpNuA4onsRa3\nRSA_IPNS_IDv0=QmVujd5Vb7moysJj8itnGufN7MEtPRCNHkKpNuA4onsRa3\nRSA_IPNS_IDv1=k2k4r8m7xvggw5pxxk3abrkwyer625hg01hfyggrai7lk1m63fuihi7w\nRSA_IPNS_IDv1_DAGPB=k2jmtxu61bnhrtj301lw7zizknztocdbeqhxgv76l2q9t36fn9jbzipo\n\nED25519_KEY=12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d\nED25519_IPNS_IDv0=12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d\nED25519_IPNS_IDv1=k51qzi5uqu5dk3v4rmjber23h16xnr23bsggmqqil9z2gduiis5se8dht36dam\nED25519_IPNS_IDv1_DAGPB=k50rm9yjlt0jey4fqg6wafvqprktgbkpgkqdg27tpqje6iimzxewnhvtin9hhq\nIPNS_ED25519_B58MH=12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d\nIPNS_ED25519_B36CID=k51qzi5uqu5dk3v4rmjber23h16xnr23bsggmqqil9z2gduiis5se8dht36dam\n\ntest_expect_success \"Add the test fixtures\" '\n ipfs dag import --pin-roots ../t0114-gateway-subdomains/fixtures.car &&\n ipfs routing put --allow-offline /ipns/${RSA_KEY} ../t0114-gateway-subdomains/${RSA_KEY}.ipns-record &&\n ipfs routing put --allow-offline /ipns/${ED25519_KEY} ../t0114-gateway-subdomains/${ED25519_KEY}.ipns-record\n'\n\n# ensure we start with empty Gateway.PublicGateways\ntest_expect_success 'start daemon with empty config for Gateway.PublicGateways' '\n test_kill_ipfs_daemon &&\n ipfs config --json Gateway.PublicGateways \"{}\" &&\n test_launch_ipfs_daemon_without_network\n'\n\n## ============================================================================\n## Test path-based requests to a local gateway with default config\n## (forced redirects to http://*.localhost)\n## ============================================================================\n\n# /ipfs/\n\n# IP remains old school path-based gateway\n\ntest_localhost_gateway_response_should_contain \\\n \"request for 127.0.0.1/ipfs/{CID} stays on path\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"$CID_VAL\"\n\n# 'localhost' hostname is used for subdomains, and should not return\n# payload directly, but redirect to URL with proper origin isolation\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{CIDv1} returns HTTP 301 Moved Permanently\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"301 Moved Permanently\"\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{CIDv1} returns Location HTTP header for subdomain redirect in browsers\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"Location: http://$CIDv1.ipfs.localhost:$GWAY_PORT/\"\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{DIR_CID} returns HTTP 301 Moved Permanently\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$DIR_CID\" \\\n \"301 Moved Permanently\"\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{DIR_CID} returns Location HTTP header for subdomain redirect in browsers\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$DIR_CID/\" \\\n \"Location: http://$DIR_CID.ipfs.localhost:$GWAY_PORT/\"\n\n# Kubo specific end-to-end test\n# (independent of gateway-conformance)\n\n# We return human-readable body with HTTP 301 so existing cli scripts that use path-based\n# gateway are informed to enable following HTTP redirects\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{CIDv1} includes human-readable link and redirect info in HTTP 301 body\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv1\" \\\n \">Moved Permanently\"\n\n# end Kubo specific end-to-end test\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{CIDv0} redirects to CIDv1 representation in subdomain\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv0\" \\\n \"Location: http://${CIDv0to1}.ipfs.localhost:$GWAY_PORT/\"\n\n# /ipns/\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipns/{CIDv0} redirects to CIDv1 with libp2p-key multicodec in subdomain\" \\\n \"http://localhost:$GWAY_PORT/ipns/$RSA_IPNS_IDv0\" \\\n \"Location: http://${RSA_IPNS_IDv1}.ipns.localhost:$GWAY_PORT/\"\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipns/{CIDv0} redirects to CIDv1 with libp2p-key multicodec in subdomain\" \\\n \"http://localhost:$GWAY_PORT/ipns/$ED25519_IPNS_IDv0\" \\\n \"Location: http://${ED25519_IPNS_IDv1}.ipns.localhost:$GWAY_PORT/\"\n\n# /ipns/\n\n# Kubo specific end-to-end test\n# (independent of gateway-conformance)\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipns/{fqdn} redirects to DNSLink in subdomain\" \\\n \"http://localhost:$GWAY_PORT/ipns/en.wikipedia-on-ipfs.org/wiki\" \\\n \"Location: http://en.wikipedia-on-ipfs.org.ipns.localhost:$GWAY_PORT/wiki\"\n\n# end Kubo specific end-to-end test\n\n## ============================================================================\n## Test subdomain-based requests to a local gateway with default config\n## (origin per content root at http://*.localhost)\n## ============================================================================\n\n# {CID}.ipfs.localhost\n\ntest_localhost_gateway_response_should_contain \\\n \"request for {CID}.ipfs.localhost should return expected payload\" \\\n \"http://${CIDv1}.ipfs.localhost:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\n# ensure /ipfs/ namespace is not mounted on subdomain\ntest_localhost_gateway_response_should_contain \\\n \"request for {CID}.ipfs.localhost/ipfs/{CID} should return HTTP 404\" \\\n \"http://${CIDv1}.ipfs.localhost:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"404 Not Found\"\n\n# ensure requests to /ipfs/* are not blocked, if content root has such subdirectory\ntest_localhost_gateway_response_should_contain \\\n \"request for {CID}.ipfs.localhost/ipfs/file.txt should return data from a file in CID content root\" \\\n \"http://${DIR_CID}.ipfs.localhost:$GWAY_PORT/ipfs/file.txt\" \\\n \"I am a txt file\"\n\n# Kubo specific end-to-end test\n# (independent of gateway-conformance)\n# This tests link to parent specific to boxo + relative pathing end-to-end tests specific to Kubo.\n\n# {CID}.ipfs.localhost/sub/dir (Directory Listing)\nDIR_HOSTNAME=\"${DIR_CID}.ipfs.localhost:$GWAY_PORT\"\n\ntest_expect_success \"valid file and subdirectory paths in directory listing at {cid}.ipfs.localhost\" '\n curl -s --resolve $DIR_HOSTNAME:127.0.0.1 \"http://$DIR_HOSTNAME\" > list_response &&\n test_should_contain \"hello\" list_response &&\n test_should_contain \"ipfs\" list_response\n'\n\ntest_expect_success \"valid parent directory path in directory listing at {cid}.ipfs.localhost/sub/dir\" '\n curl -s --resolve $DIR_HOSTNAME:127.0.0.1 \"http://$DIR_HOSTNAME/ipfs/ipns/\" > list_response &&\n test_should_contain \"..\" list_response &&\n test_should_contain \"bar\" list_response\n'\n\ntest_expect_success \"request for deep path resource at {cid}.ipfs.localhost/sub/dir/file\" '\n curl -s --resolve $DIR_HOSTNAME:127.0.0.1 \"http://$DIR_HOSTNAME/ipfs/ipns/bar\" > list_response &&\n test_should_contain \"text-file-content\" list_response\n'\n# end Kubo specific end-to-end test\n\n# *.ipns.localhost\n\n# .ipns.localhost\n\ntest_localhost_gateway_response_should_contain \\\n \"request for {CIDv1-libp2p-key}.ipns.localhost returns expected payload\" \\\n \"http://${RSA_IPNS_IDv1}.ipns.localhost:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\ntest_localhost_gateway_response_should_contain \\\n \"request for {CIDv1-libp2p-key}.ipns.localhost returns expected payload\" \\\n \"http://${ED25519_IPNS_IDv1}.ipns.localhost:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\ntest_localhost_gateway_response_should_contain \\\n \"localhost request for {CIDv1-dag-pb}.ipns.localhost redirects to CID with libp2p-key multicodec\" \\\n \"http://${RSA_IPNS_IDv1_DAGPB}.ipns.localhost:$GWAY_PORT\" \\\n \"Location: http://${RSA_IPNS_IDv1}.ipns.localhost:$GWAY_PORT/\"\n\ntest_localhost_gateway_response_should_contain \\\n \"localhost request for {CIDv1-dag-pb}.ipns.localhost redirects to CID with libp2p-key multicodec\" \\\n \"http://${ED25519_IPNS_IDv1_DAGPB}.ipns.localhost:$GWAY_PORT\" \\\n \"Location: http://${ED25519_IPNS_IDv1}.ipns.localhost:$GWAY_PORT/\"\n\n# .ipns.localhost\n\n# DNSLink test requires a daemon in online mode with precached /ipns/ mapping\ntest_kill_ipfs_daemon\nDNSLINK_FQDN=\"dnslink-test.example.com\"\nexport IPFS_NS_MAP=\"$DNSLINK_FQDN:/ipfs/$CIDv1\"\ntest_launch_ipfs_daemon\n\ntest_localhost_gateway_response_should_contain \\\n \"request for {dnslink}.ipns.localhost returns expected payload\" \\\n \"http://$DNSLINK_FQDN.ipns.localhost:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\n## ============================================================================\n## Test DNSLink inlining on HTTP gateways\n## ============================================================================\n\n# set explicit subdomain gateway config for the hostname\nipfs config --json Gateway.PublicGateways '{\n \"localhost\": {\n \"UseSubdomains\": true,\n \"InlineDNSLink\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\", \"/api\"]\n },\n \"example.com\": {\n \"UseSubdomains\": true,\n \"InlineDNSLink\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\", \"/api\"]\n }\n}' || exit 1\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipns/{fqdn} redirects to DNSLink in subdomain with DNS inlining\" \\\n \"http://localhost:$GWAY_PORT/ipns/en.wikipedia-on-ipfs.org/wiki\" \\\n \"Location: http://en-wikipedia--on--ipfs-org.ipns.localhost:$GWAY_PORT/wiki\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/{fqdn} redirects to DNSLink in subdomain with DNS inlining\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/en.wikipedia-on-ipfs.org/wiki\" \\\n \"Location: http://en-wikipedia--on--ipfs-org.ipns.example.com/wiki\"\n\n## ============================================================================\n## Test subdomain-based requests with a custom hostname config\n## (origin per content root at http://*.example.com)\n## ============================================================================\n\n# set explicit subdomain gateway config for the hostname\nipfs config --json Gateway.PublicGateways '{\n \"example.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\", \"/api\"]\n }\n}' || exit 1\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\n\n# example.com/ip(f|n)s/*\n# =============================================================================\n\n# path requests to the root hostname should redirect\n# to a subdomain URL with proper origin isolation\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipfs/{CIDv1} produces redirect to {CIDv1}.ipfs.example.com\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"Location: http://$CIDv1.ipfs.example.com/\"\n\n# error message should include original CID\n# (and it should be case-sensitive, as we can't assume everyone uses base32)\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipfs/{InvalidCID} produces useful error before redirect\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/QmInvalidCID\" \\\n 'invalid path \\\"/ipfs/QmInvalidCID\\\"'\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipfs/{CIDv0} produces redirect to {CIDv1}.ipfs.example.com\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv0\" \\\n \"Location: http://${CIDv0to1}.ipfs.example.com/\"\n\n# Support X-Forwarded-Proto\ntest_expect_success \"request for http://example.com/ipfs/{CID} with X-Forwarded-Proto: https produces redirect to HTTPS URL\" \"\n curl -H \\\"X-Forwarded-Proto: https\\\" -H \\\"Host: example.com\\\" -sD - \\\"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\\\" > response &&\n test_should_contain \\\"Location: https://$CIDv1.ipfs.example.com/\\\" response\n\"\n\n# Support ipfs:// in https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipfs/?uri=ipfs%3A%2F%2F.. produces redirect to /ipfs/.. content path\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/?uri=ipfs%3A%2F%2FQmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco%2Fwiki%2FDiego_Maradona.html\" \\\n \"Location: /ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Diego_Maradona.html\"\n\n# example.com/ipns/\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/{CIDv0} redirects to CIDv1 with libp2p-key multicodec in subdomain\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$RSA_IPNS_IDv0\" \\\n \"Location: http://${RSA_IPNS_IDv1}.ipns.example.com/\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/{CIDv0} redirects to CIDv1 with libp2p-key multicodec in subdomain\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$ED25519_IPNS_IDv0\" \\\n \"Location: http://${ED25519_IPNS_IDv1}.ipns.example.com/\"\n\n# example.com/ipns/\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/{fqdn} redirects to DNSLink in subdomain\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/en.wikipedia-on-ipfs.org/wiki\" \\\n \"Location: http://en.wikipedia-on-ipfs.org.ipns.example.com/wiki\"\n\n# DNSLink on Public gateway with a single-level wildcard TLS cert\n# \"Option C\" from https://github.com/ipfs/in-web-browsers/issues/169\ntest_expect_success \\\n \"request for example.com/ipns/{fqdn} with X-Forwarded-Proto redirects to TLS-safe label in subdomain\" \"\n curl -H \\\"Host: example.com\\\" -H \\\"X-Forwarded-Proto: https\\\" -sD - \\\"http://127.0.0.1:$GWAY_PORT/ipns/en.wikipedia-on-ipfs.org/wiki\\\" > response &&\n test_should_contain \\\"Location: https://en-wikipedia--on--ipfs-org.ipns.example.com/wiki\\\" response\n \"\n\n# Support ipns:// in https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/?uri=ipns%3A%2F%2F.. produces redirect to /ipns/.. content path\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/?uri=ipns%3A%2F%2Fen.wikipedia-on-ipfs.org\" \\\n \"Location: /ipns/en.wikipedia-on-ipfs.org\"\n\n# *.ipfs.example.com: subdomain requests made with custom FQDN in Host header\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.example.com should return expected payload\" \\\n \"${CIDv1}.ipfs.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.example.com/ipfs/{CID} should return HTTP 404\" \\\n \"${CIDv1}.ipfs.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"404 Not Found\"\n\n# Kubo specific end-to-end test\n# (independent of gateway-conformance)\n# HTML specific to Boxo/Kubo, and relative pathing specific to code in Kubo\n\n# {CID}.ipfs.example.com/sub/dir (Directory Listing)\nDIR_FQDN=\"${DIR_CID}.ipfs.example.com\"\n\ntest_expect_success \"valid file and directory paths in directory listing at {cid}.ipfs.example.com\" '\n curl -s -H \"Host: $DIR_FQDN\" http://127.0.0.1:$GWAY_PORT > list_response &&\n test_should_contain \"hello\" list_response &&\n test_should_contain \"ipfs\" list_response\n'\n\ntest_expect_success \"valid parent directory path in directory listing at {cid}.ipfs.example.com/sub/dir\" '\n curl -s -H \"Host: $DIR_FQDN\" http://127.0.0.1:$GWAY_PORT/ipfs/ipns/ > list_response &&\n test_should_contain \"..\" list_response &&\n test_should_contain \"bar\" list_response\n'\n\n# Note 1: we test for sneaky subdir names {cid}.ipfs.example.com/ipfs/ipns/ :^)\n# Note 2: example.com/ipfs/.. present in HTML will be redirected to subdomain, so this is expected behavior\ntest_expect_success \"valid breadcrumb links in the header of directory listing at {cid}.ipfs.example.com/sub/dir\" '\n curl -s -H \"Host: $DIR_FQDN\" http://127.0.0.1:$GWAY_PORT/ipfs/ipns/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_contain \"/ipfs/${DIR_CID}/ipfs/ipns\" list_response\n'\n\n# end Kubo specific end-to-end test\n\ntest_expect_success \"request for deep path resource {cid}.ipfs.example.com/sub/dir/file\" '\n curl -s -H \"Host: $DIR_FQDN\" http://127.0.0.1:$GWAY_PORT/ipfs/ipns/bar > list_response &&\n test_should_contain \"text-file-content\" list_response\n'\n\n# *.ipns.example.com\n# ============================================================================\n\n# .ipns.example.com\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CIDv1-libp2p-key}.ipns.example.com returns expected payload\" \\\n \"${RSA_IPNS_IDv1}.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CIDv1-libp2p-key}.ipns.example.com returns expected payload\" \\\n \"${ED25519_IPNS_IDv1}.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\ntest_hostname_gateway_response_should_contain \\\n \"hostname request for {CIDv1-dag-pb}.ipns.localhost redirects to CID with libp2p-key multicodec\" \\\n \"${RSA_IPNS_IDv1_DAGPB}.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"Location: http://${RSA_IPNS_IDv1}.ipns.example.com/\"\n\ntest_hostname_gateway_response_should_contain \\\n \"hostname request for {CIDv1-dag-pb}.ipns.localhost redirects to CID with libp2p-key multicodec\" \\\n \"${ED25519_IPNS_IDv1_DAGPB}.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"Location: http://${ED25519_IPNS_IDv1}.ipns.example.com/\"\n\n# DNSLink: .ipns.example.com\n# (not really useful outside of localhost, as setting TLS for more than one\n# level of wildcard is a pain, but we support it if someone really wants it)\n# ============================================================================\n\n# DNSLink test requires a daemon in online mode with precached /ipns/ mapping\ntest_kill_ipfs_daemon\nDNSLINK_FQDN=\"dnslink-subdomain-gw-test.example.org\"\nexport IPFS_NS_MAP=\"$DNSLINK_FQDN:/ipfs/$CIDv1\"\ntest_launch_ipfs_daemon\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {dnslink}.ipns.example.com returns expected payload\" \\\n \"$DNSLINK_FQDN.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"$CID_VAL\"\n\n# DNSLink on Public gateway with a single-level wildcard TLS cert\n# \"Option C\" from https://github.com/ipfs/in-web-browsers/issues/169\ntest_expect_success \\\n \"request for {single-label-dnslink}.ipns.example.com with X-Forwarded-Proto returns expected payload\" \"\n curl -H \\\"Host: dnslink--subdomain--gw--test-example-org.ipns.example.com\\\" -H \\\"X-Forwarded-Proto: https\\\" -sD - \\\"http://127.0.0.1:$GWAY_PORT\\\" > response &&\n test_should_contain \\\"$CID_VAL\\\" response\n \"\n\n## Test subdomain handling of CIDs that do not fit in a single DNS Label (>63chars)\n## https://github.com/ipfs/go-ipfs/issues/7318\n## ============================================================================\n\n# local: *.localhost\ntest_localhost_gateway_response_should_contain \\\n \"request for a ED25519 libp2p-key at localhost/ipns/{b58mh} returns Location HTTP header for DNS-safe subdomain redirect in browsers\" \\\n \"http://localhost:$GWAY_PORT/ipns/$IPNS_ED25519_B58MH\" \\\n \"Location: http://${IPNS_ED25519_B36CID}.ipns.localhost:$GWAY_PORT/\"\n\n# router should not redirect to hostnames that could fail due to DNS limits\ntest_localhost_gateway_response_should_contain \\\n \"request for a too long CID at localhost/ipfs/{CIDv1} returns human readable error\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv1_TOO_LONG\" \\\n \"CID incompatible with DNS label length limit of 63\"\n\ntest_localhost_gateway_response_should_contain \\\n \"request for a too long CID at localhost/ipfs/{CIDv1} returns HTTP Error 400 Bad Request\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv1_TOO_LONG\" \\\n \"400 Bad Request\"\n\n# direct request should also fail (provides the same UX as router and avoids confusion)\ntest_localhost_gateway_response_should_contain \\\n \"request for a too long CID at {CIDv1}.ipfs.localhost returns expected payload\" \\\n \"http://$CIDv1_TOO_LONG.ipfs.localhost:$GWAY_PORT\" \\\n \"400 Bad Request\"\n\n# public subdomain gateway: *.example.com\n\ntest_hostname_gateway_response_should_contain \\\n \"request for a ED25519 libp2p-key at example.com/ipns/{b58mh} returns Location HTTP header for DNS-safe subdomain redirect in browsers\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$IPNS_ED25519_B58MH\" \\\n \"Location: http://${IPNS_ED25519_B36CID}.ipns.example.com\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for a too long CID at example.com/ipfs/{CIDv1} returns human readable error\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1_TOO_LONG\" \\\n \"CID incompatible with DNS label length limit of 63\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for a too long CID at example.com/ipfs/{CIDv1} returns HTTP Error 400 Bad Request\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1_TOO_LONG\" \\\n \"400 Bad Request\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for a too long CID at {CIDv1}.ipfs.example.com returns HTTP Error 400 Bad Request\" \\\n \"$CIDv1_TOO_LONG.ipfs.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"400 Bad Request\"\n\n# Disable selected Paths for the subdomain gateway hostname\n# =============================================================================\n\n# disable /ipns for the hostname by not whitelisting it\nipfs config --json Gateway.PublicGateways '{\n \"example.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\"]\n }\n}' || exit 1\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\n# refuse requests to Paths that were not explicitly whitelisted for the hostname\ntest_hostname_gateway_response_should_contain \\\n \"request for *.ipns.example.com returns HTTP 404 Not Found when /ipns is not on Paths whitelist\" \\\n \"${RSA_IPNS_IDv1}.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"404 Not Found\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for *.ipns.example.com returns HTTP 404 Not Found when /ipns is not on Paths whitelist\" \\\n \"${ED25519_IPNS_IDv1}.ipns.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT\" \\\n \"404 Not Found\"\n\n## ============================================================================\n## Test path-based requests with a custom hostname config\n## ============================================================================\n\n# set explicit no-subdomain gateway config for the hostname\nipfs config --json Gateway.PublicGateways '{\n \"example.com\": {\n \"UseSubdomains\": false,\n \"Paths\": [\"/ipfs\"]\n }\n}' || exit 1\n\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\n# example.com/ip(f|n)s/* smoke-tests\n# =============================================================================\n\n# confirm path gateway works for /ipfs\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipfs/{CIDv1} returns expected payload\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"$CID_VAL\"\n\n# refuse subdomain requests on path gateway\n# (we don't want false sense of security)\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.example.com/ipfs/{CID} should return HTTP 404 Not Found\" \\\n \"${CIDv1}.ipfs.example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"404 Not Found\"\n\n# refuse requests to Paths that were not explicitly whitelisted for the hostname\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/ returns HTTP 404 Not Found when /ipns is not on Paths whitelist\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$RSA_IPNS_IDv1\" \\\n \"404 Not Found\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for example.com/ipns/ returns HTTP 404 Not Found when /ipns is not on Paths whitelist\" \\\n \"example.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$ED25519_IPNS_IDv1\" \\\n \"404 Not Found\"\n\n## ============================================================================\n## Test DNSLink requests with a custom PublicGateway (hostname config)\n## (DNSLink site at http://dnslink-test.example.com)\n## ============================================================================\n\ntest_kill_ipfs_daemon\n\n# disable wildcard DNSLink gateway\n# and enable it on specific NSLink hostname\nipfs config --json Gateway.NoDNSLink true && \\\nipfs config --json Gateway.PublicGateways '{\n \"dnslink-enabled-on-fqdn.example.org\": {\n \"NoDNSLink\": false,\n \"UseSubdomains\": false,\n \"Paths\": [\"/ipfs\"]\n },\n \"only-dnslink-enabled-on-fqdn.example.org\": {\n \"NoDNSLink\": false,\n \"UseSubdomains\": false,\n \"Paths\": []\n },\n \"dnslink-disabled-on-fqdn.example.com\": {\n \"NoDNSLink\": true,\n \"UseSubdomains\": false,\n \"Paths\": []\n }\n}' || exit 1\n\n# DNSLink test requires a daemon in online mode with precached /ipns/ mapping\nDNSLINK_FQDN=\"dnslink-enabled-on-fqdn.example.org\"\nONLY_DNSLINK_FQDN=\"only-dnslink-enabled-on-fqdn.example.org\"\nNO_DNSLINK_FQDN=\"dnslink-disabled-on-fqdn.example.com\"\nexport IPFS_NS_MAP=\"$DNSLINK_FQDN:/ipfs/$CIDv1,$ONLY_DNSLINK_FQDN:/ipfs/$DIR_CID\"\n\n# restart daemon to apply config changes\ntest_launch_ipfs_daemon\n\n# make sure test setup is valid (fail if CoreAPI is unable to resolve)\ntest_expect_success \"spoofed DNSLink record resolves in cli\" \"\n ipfs resolve /ipns/$DNSLINK_FQDN > result &&\n test_should_contain \\\"$CIDv1\\\" result &&\n ipfs cat /ipns/$DNSLINK_FQDN > result &&\n test_should_contain \\\"$CID_VAL\\\" result\n\"\n\n# DNSLink enabled\n\ntest_hostname_gateway_response_should_contain \\\n \"request for http://{dnslink-fqdn}/ PublicGateway returns expected payload\" \\\n \"$DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {dnslink-fqdn}/ipfs/{cid} returns expected payload when /ipfs is on Paths whitelist\" \\\n \"$DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"$CID_VAL\"\n\n# Test for a fun edge case: DNSLink-only gateway without /ipfs/ namespace\n# mounted, and with subdirectory named \"ipfs\" ¯\\_(ツ)_/¯\ntest_hostname_gateway_response_should_contain \\\n \"request for {dnslink-fqdn}/ipfs/file.txt returns data from content root when /ipfs in not on Paths whitelist\" \\\n \"$ONLY_DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/file.txt\" \\\n \"I am a txt file\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {dnslink-fqdn}/ipns/{peerid} returns 404 when path is not whitelisted\" \\\n \"$DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$RSA_IPNS_IDv0\" \\\n \"404 Not Found\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {dnslink-fqdn}/ipns/{peerid} returns 404 when path is not whitelisted\" \\\n \"$DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipns/$ED25519_IPNS_IDv0\" \\\n \"404 Not Found\"\n\n# DNSLink disabled\n\ntest_hostname_gateway_response_should_contain \\\n \"request for http://{dnslink-fqdn}/ returns 404 when NoDNSLink=true\" \\\n \"$NO_DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"404 Not Found\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {dnslink-fqdn}/ipfs/{cid} returns 404 when path is not whitelisted\" \\\n \"$NO_DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv0\" \\\n \"404 Not Found\"\n\n\n## ============================================================================\n## Test wildcard DNSLink (any hostname, with default config)\n## ============================================================================\n\ntest_kill_ipfs_daemon\n\n# enable wildcard DNSLink gateway (any value in Host header)\n# and remove custom PublicGateways\nipfs config --json Gateway.NoDNSLink false && \\\nipfs config --json Gateway.PublicGateways '{}' || exit 1\n\n# DNSLink test requires a daemon in online mode with precached /ipns/ mapping\nDNSLINK_FQDN=\"wildcard-dnslink-not-in-config.example.com\"\nexport IPFS_NS_MAP=\"$DNSLINK_FQDN:/ipfs/$CIDv1\"\n\n# restart daemon to apply config changes\ntest_launch_ipfs_daemon\n\n# make sure test setup is valid (fail if CoreAPI is unable to resolve)\ntest_expect_success \"spoofed DNSLink record resolves in cli\" \"\n ipfs resolve /ipns/$DNSLINK_FQDN > result &&\n test_should_contain \\\"$CIDv1\\\" result &&\n ipfs cat /ipns/$DNSLINK_FQDN > result &&\n test_should_contain \\\"$CID_VAL\\\" result\n\"\n\n# gateway test\ntest_hostname_gateway_response_should_contain \\\n \"request for http://{dnslink-fqdn}/ (wildcard) returns expected payload\" \\\n \"$DNSLINK_FQDN\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\n## ============================================================================\n## Test support for X-Forwarded-Host\n## ============================================================================\n\n# set explicit subdomain gateway config for the hostname\nipfs config --json Gateway.PublicGateways '{\n \"example.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\", \"/ipns\", \"/api\"]\n }\n}' || exit 1\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\ntest_expect_success \"request for http://fake.domain.com/ipfs/{CID} doesn't match the example.com gateway\" \"\n curl -H \\\"Host: fake.domain.com\\\" -sD - \\\"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\\\" > response &&\n test_should_contain \\\"200 OK\\\" response\n\"\n\ntest_expect_success \"request for http://fake.domain.com/ipfs/{CID} with X-Forwarded-Host: example.com match the example.com gateway\" \"\n curl -H \\\"Host: fake.domain.com\\\" -H \\\"X-Forwarded-Host: example.com\\\" -sD - \\\"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\\\" > response &&\n test_should_contain \\\"Location: http://$CIDv1.ipfs.example.com/\\\" response\n\"\n\ntest_expect_success \"request for http://fake.domain.com/ipfs/{CID} with X-Forwarded-Host: example.com and X-Forwarded-Proto: https match the example.com gateway, redirect with https\" \"\n curl -H \\\"Host: fake.domain.com\\\" -H \\\"X-Forwarded-Host: example.com\\\" -H \\\"X-Forwarded-Proto: https\\\" -sD - \\\"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\\\" > response &&\n test_should_contain \\\"Location: https://$CIDv1.ipfs.example.com/\\\" response\n\"\n\n# Kubo specific end-to-end test\n# (independent of gateway-conformance)\n# test configuration being wired up correctly end-to-end\n\n## ============================================================================\n## Test support for wildcards in gateway config\n## ============================================================================\n\n# set explicit subdomain gateway config for the hostnames\nipfs config --json Gateway.PublicGateways '{\n \"*.example1.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\"]\n },\n \"*.*.example2.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\"]\n },\n \"foo.*.example3.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\"]\n },\n \"foo.bar-*-boo.example4.com\": {\n \"UseSubdomains\": true,\n \"Paths\": [\"/ipfs\"]\n }\n}' || exit 1\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\n# *.example1.com\n\ntest_hostname_gateway_response_should_contain \\\n \"request for foo.example1.com/ipfs/{CIDv1} produces redirect to {CIDv1}.ipfs.foo.example1.com\" \\\n \"foo.example1.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"Location: http://$CIDv1.ipfs.foo.example1.com/\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.foo.example1.com should return expected payload\" \\\n \"${CIDv1}.ipfs.foo.example1.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\n# *.*.example2.com\n\ntest_hostname_gateway_response_should_contain \\\n \"request for foo.bar.example2.com/ipfs/{CIDv1} produces redirect to {CIDv1}.ipfs.foo.bar.example2.com\" \\\n \"foo.bar.example2.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"Location: http://$CIDv1.ipfs.foo.bar.example2.com/\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.foo.bar.example2.com should return expected payload\" \\\n \"${CIDv1}.ipfs.foo.bar.example2.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\n# foo.*.example3.com\n\ntest_hostname_gateway_response_should_contain \\\n \"request for foo.bar.example3.com/ipfs/{CIDv1} produces redirect to {CIDv1}.ipfs.foo.bar.example3.com\" \\\n \"foo.bar.example3.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"Location: http://$CIDv1.ipfs.foo.bar.example3.com/\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.foo.bar.example3.com should return expected payload\" \\\n \"${CIDv1}.ipfs.foo.bar.example3.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\n# foo.bar-*-boo.example4.com\n\ntest_hostname_gateway_response_should_contain \\\n \"request for foo.bar-dev-boo.example4.com/ipfs/{CIDv1} produces redirect to {CIDv1}.ipfs.foo.bar-dev-boo.example4.com\" \\\n \"foo.bar-dev-boo.example4.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"Location: http://$CIDv1.ipfs.foo.bar-dev-boo.example4.com/\"\n\ntest_hostname_gateway_response_should_contain \\\n \"request for {CID}.ipfs.foo.bar-dev-boo.example4.com should return expected payload\" \\\n \"${CIDv1}.ipfs.foo.bar-dev-boo.example4.com\" \\\n \"http://127.0.0.1:$GWAY_PORT/\" \\\n \"$CID_VAL\"\n\n## ============================================================================\n## Test support for overriding implicit defaults\n## ============================================================================\n\n# disable subdomain gateway at localhost by removing implicit config\nipfs config --json Gateway.PublicGateways '{\n \"localhost\": null\n}' || exit 1\n\n# restart daemon to apply config changes\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon_without_network\n\ntest_localhost_gateway_response_should_contain \\\n \"request for localhost/ipfs/{CID} stays on path when subdomain gw is explicitly disabled\" \\\n \"http://localhost:$GWAY_PORT/ipfs/$CIDv1\" \\\n \"$CID_VAL\"\n\n# =============================================================================\n# ensure we end with empty Gateway.PublicGateways\nipfs config --json Gateway.PublicGateways '{}'\ntest_kill_ipfs_daemon\n\ntest_expect_success \"clean up ipfs dir\" '\n rm -rf \"$IPFS_PATH\"\n'\n\ntest_done\n\n# end Kubo specific end-to-end test\n" }, { "path": "test/sharness/t0115-gateway-dir-listing/README.md", "content": "# Dataset description/sources\n\n- fixtures.car\n - raw CARv1\n\ngenerated with:\n\n```sh\n# using ipfs version 0.18.1\nmkdir -p rootDir/ipfs &&\nmkdir -p rootDir/ipns &&\nmkdir -p rootDir/api &&\nmkdir -p rootDir/ą/ę &&\necho \"I am a txt file on path with utf8\" > rootDir/ą/ę/file-źł.txt &&\necho \"I am a txt file in confusing /api dir\" > rootDir/api/file.txt &&\necho \"I am a txt file in confusing /ipfs dir\" > rootDir/ipfs/file.txt &&\necho \"I am a txt file in confusing /ipns dir\" > rootDir/ipns/file.txt &&\nDIR_CID=$(ipfs add -Qr --cid-version 1 rootDir) &&\nFILE_CID=$(ipfs files stat --enc=json /ipfs/$DIR_CID/ą/ę/file-źł.txt | jq -r .Hash) &&\nFILE_SIZE=$(ipfs files stat --enc=json /ipfs/$DIR_CID/ą/ę/file-źł.txt | jq -r .Size)\necho \"$FILE_CID / $FILE_SIZE\"\n\necho DIR_CID=${DIR_CID}\necho FILE_CID=${FILE_CID}\necho FILE_SIZE=${FILE_SIZE}\n\nipfs dag export ${DIR_CID} > ./fixtures.car\n\n# DIR_CID=bafybeig6ka5mlwkl4subqhaiatalkcleo4jgnr3hqwvpmsqfca27cijp3i # ./rootDir/\n# FILE_CID=bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm # ./rootDir/ą/ę/file-źł.txt\n# FILE_SIZE=34\n```\n" }, { "path": "test/sharness/t0115-gateway-dir-listing.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) Protocol Labs\n\ntest_description=\"Test directory listing (dir-index-html) on the HTTP gateway\"\n\n\n. lib/test-lib.sh\n\n## ============================================================================\n## Start IPFS Node and prepare test CIDs\n## ============================================================================\n\ntest_expect_success \"ipfs init\" '\n export IPFS_PATH=\"$(pwd)/.ipfs\" &&\n ipfs init --profile=test > /dev/null\n'\n\ntest_launch_ipfs_daemon_without_network\n\n# Import test case\n# See the static fixtures in ./t0115-gateway-dir-listing/\ntest_expect_success \"Add the test directory\" '\n ipfs dag import --pin-roots ../t0115-gateway-dir-listing/fixtures.car\n'\nDIR_CID=bafybeig6ka5mlwkl4subqhaiatalkcleo4jgnr3hqwvpmsqfca27cijp3i # ./rootDir/\nFILE_CID=bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm # ./rootDir/ą/ę/file-źł.txt\nFILE_SIZE=34\n\n## ============================================================================\n## Test dir listing on path gateway (eg. 127.0.0.1:8080/ipfs/)\n## ============================================================================\n\ntest_expect_success \"path gw: backlink on root CID should be hidden\" '\n curl -sD - http://127.0.0.1:$GWAY_PORT/ipfs/${DIR_CID}/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_not_contain \"..\" list_response\n'\n\ntest_expect_success \"path gw: redirect dir listing to URL with trailing slash\" '\n curl -sD - http://127.0.0.1:$GWAY_PORT/ipfs/${DIR_CID}/ą/ę > list_response &&\n test_should_contain \"HTTP/1.1 301 Moved Permanently\" list_response &&\n test_should_contain \"Location: /ipfs/${DIR_CID}/%C4%85/%C4%99/\" list_response\n'\n\ntest_expect_success \"path gw: Etag should be present\" '\n curl -sD - http://127.0.0.1:$GWAY_PORT/ipfs/${DIR_CID}/ą/ę/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_contain \"Etag: \\\"DirIndex-\" list_response\n'\n\ntest_expect_success \"path gw: breadcrumbs should point at /ipfs namespace mounted at Origin root\" '\n test_should_contain \"/ipfs/$DIR_CID/ą/ę\" list_response\n'\n\ntest_expect_success \"path gw: backlink on subdirectory should point at parent directory\" '\n test_should_contain \"..\" list_response\n'\n\ntest_expect_success \"path gw: name column should be a link to its content path\" '\n test_should_contain \"file-źł.txt\" list_response\n'\n\ntest_expect_success \"path gw: hash column should be a CID link with filename param\" '\n test_should_contain \"\" list_response\n'\n\n## ============================================================================\n## Test dir listing on subdomain gateway (eg. .ipfs.localhost:8080)\n## ============================================================================\n\nDIR_HOSTNAME=\"${DIR_CID}.ipfs.localhost\"\n# note: we skip DNS lookup by running curl with --resolve $DIR_HOSTNAME:127.0.0.1\n\ntest_expect_success \"subdomain gw: backlink on root CID should be hidden\" '\n curl -sD - --resolve $DIR_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DIR_HOSTNAME:$GWAY_PORT/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_not_contain \"..\" list_response\n'\n\ntest_expect_success \"subdomain gw: redirect dir listing to URL with trailing slash\" '\n curl -sD - --resolve $DIR_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DIR_HOSTNAME:$GWAY_PORT/ą/ę > list_response &&\n test_should_contain \"HTTP/1.1 301 Moved Permanently\" list_response &&\n test_should_contain \"Location: /%C4%85/%C4%99/\" list_response\n'\n\ntest_expect_success \"subdomain gw: Etag should be present\" '\n curl -sD - --resolve $DIR_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DIR_HOSTNAME:$GWAY_PORT/ą/ę/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_contain \"Etag: \\\"DirIndex-\" list_response\n'\n\ntest_expect_success \"subdomain gw: backlink on subdirectory should point at parent directory\" '\n test_should_contain \"..\" list_response\n'\n\ntest_expect_success \"subdomain gw: breadcrumbs should leverage path-based router mounted on the parent domain\" '\n test_should_contain \"/ipfs/$DIR_CID/ą/ę\" list_response\n'\n\ntest_expect_success \"subdomain gw: name column should be a link to content root mounted at subdomain origin\" '\n test_should_contain \"file-źł.txt\" list_response\n'\n\ntest_expect_success \"subdomain gw: hash column should be a CID link to path router with filename param\" '\n test_should_contain \"\" list_response\n'\n\n## ============================================================================\n## Test dir listing on DNSLink gateway (eg. example.com)\n## ============================================================================\n\n# DNSLink test requires a daemon in online mode with precached /ipns/ mapping\ntest_kill_ipfs_daemon\nDNSLINK_HOSTNAME=\"website.example.com\"\nexport IPFS_NS_MAP=\"$DNSLINK_HOSTNAME:/ipfs/$DIR_CID\"\ntest_launch_ipfs_daemon\n\n# Note that:\n# - this type of gateway is also tested in gateway_test.go#TestIPNSHostnameBacklinks\n# (go tests and sharness tests should be kept in sync)\n# - we skip DNS lookup by running curl with --resolve $DNSLINK_HOSTNAME:127.0.0.1\n\ntest_expect_success \"dnslink gw: backlink on root CID should be hidden\" '\n curl -v -sD - --resolve $DNSLINK_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DNSLINK_HOSTNAME:$GWAY_PORT/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_not_contain \"..\" list_response\n'\n\ntest_expect_success \"dnslink gw: redirect dir listing to URL with trailing slash\" '\n curl -sD - --resolve $DNSLINK_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DNSLINK_HOSTNAME:$GWAY_PORT/ą/ę > list_response &&\n test_should_contain \"HTTP/1.1 301 Moved Permanently\" list_response &&\n test_should_contain \"Location: /%C4%85/%C4%99/\" list_response\n'\n\ntest_expect_success \"dnslink gw: Etag should be present\" '\n curl -sD - --resolve $DNSLINK_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DNSLINK_HOSTNAME:$GWAY_PORT/ą/ę/ > list_response &&\n test_should_contain \"Index of\" list_response &&\n test_should_contain \"Etag: \\\"DirIndex-\" list_response\n'\n\ntest_expect_success \"dnslink gw: backlink on subdirectory should point at parent directory\" '\n test_should_contain \"..\" list_response\n'\n\ntest_expect_success \"dnslink gw: breadcrumbs should point at content root mounted at dnslink origin\" '\n test_should_contain \"/ipns/website.example.com/ą/ę\" list_response\n'\n\ntest_expect_success \"dnslink gw: name column should be a link to content root mounted at dnslink origin\" '\n test_should_contain \"file-źł.txt\" list_response\n'\n\n# DNSLink websites don't have public gateway mounted by default\n# See: https://github.com/ipfs/dir-index-html/issues/42\ntest_expect_success \"dnslink gw: hash column should be a CID link to cid.ipfs.tech\" '\n test_should_contain \"\" list_response\n'\n\n## ============================================================================\n## End of tests, cleanup\n## ============================================================================\n\ntest_kill_ipfs_daemon\ntest_expect_success \"clean up ipfs dir\" '\n rm -rf \"$IPFS_PATH\"\n'\ntest_done\n" }, { "path": "test/sharness/t0116-gateway-cache/README.md", "content": "# Dataset description/sources\n\n- fixtures.car\n - raw CARv1\n\ngenerated with:\n\n```sh\n# using ipfs version 0.21.0-dev (03a98280e3e642774776cd3d0435ab53e5dfa867)\n\nmkdir -p root2/root3/root4 &&\necho \"hello\" > root2/root3/root4/index.html &&\nROOT1_CID=$(ipfs add -Qrw --cid-version 1 root2)\nROOT2_CID=$(ipfs resolve -r /ipfs/$ROOT1_CID/root2 | cut -d \"/\" -f3)\nROOT3_CID=$(ipfs resolve -r /ipfs/$ROOT1_CID/root2/root3 | cut -d \"/\" -f3)\nROOT4_CID=$(ipfs resolve -r /ipfs/$ROOT1_CID/root2/root3/root4 | cut -d \"/\" -f3)\nFILE_CID=$(ipfs resolve -r /ipfs/$ROOT1_CID/root2/root3/root4/index.html | cut -d \"/\" -f3)\n\nTEST_IPNS_ID=$(ipfs key gen --ipns-base=base36 --type=ed25519 cache_test_key | head -n1 | tr -d \"\\n\")\n# publish a record valid for a 100 years\nipfs name publish --key cache_test_key --allow-offline -Q --ttl=876600h --lifetime=876600h \"/ipfs/$ROOT1_CID\"\nipfs routing get /ipns/${TEST_IPNS_ID} > ${TEST_IPNS_ID}.ipns-record\n\necho ROOT1_CID=${ROOT1_CID} # ./\necho ROOT2_CID=${ROOT2_CID} # ./root2\necho ROOT3_CID=${ROOT3_CID} # ./root2/root3\necho ROOT4_CID=${ROOT4_CID} # ./root2/root3/root4\necho FILE_CID=${FILE_CID} # ./root2/root3/root4/index.html\necho TEST_IPNS_ID=${TEST_IPNS_ID}\n\nipfs dag export ${ROOT1_CID} > ./fixtures.car\n\n# ROOT1_CID=bafybeib3ffl2teiqdncv3mkz4r23b5ctrwkzrrhctdbne6iboayxuxk5ui # ./\n# ROOT2_CID=bafybeih2w7hjocxjg6g2ku25hvmd53zj7og4txpby3vsusfefw5rrg5sii # ./root2\n# ROOT3_CID=bafybeiawdvhmjcz65x5egzx4iukxc72hg4woks6v6fvgyupiyt3oczk5ja # ./root2/root3\n# ROOT4_CID=bafybeifq2rzpqnqrsdupncmkmhs3ckxxjhuvdcbvydkgvch3ms24k5lo7q # ./root2/root3/root4\n# FILE_CID=bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am # ./root2/root3/root4/index.html\n# TEST_IPNS_ID=k51qzi5uqu5dlxdsdu5fpuu7h69wu4ohp32iwm9pdt9nq3y5rpn3ln9j12zfhe\n```\n" }, { "path": "test/sharness/t0116-gateway-cache.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test HTTP Gateway Cache Control Support\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\ntest_launch_ipfs_daemon_without_network\n\n# Cache control support is based on logical roots (each path segment == one logical root).\n# To maximize the test surface, we want to test:\n# - /ipfs/ content path\n# - /ipns/ content path\n# - at least 3 levels\n# - separate tests for a directory listing and a file\n# - have implicit index.html for a good measure\n# /ipns/root1/root2/root3/ (/ipns/root1/root2/root3/index.html)\n\n# Note: we cover important UnixFS-focused edge case here:\n#\n# ROOT3_CID - dir listing (dir-index-html response)\n# ROOT4_CID - index.html returned as a root response (dir/), instead of generated dir-index-html\n# FILE_CID - index.html returned directly, as a file\n#\n# Caching of things like raw blocks, CARs, dag-json and dag-cbor\n# is tested in their respective suites.\n\nROOT1_CID=bafybeib3ffl2teiqdncv3mkz4r23b5ctrwkzrrhctdbne6iboayxuxk5ui # ./\nROOT2_CID=bafybeih2w7hjocxjg6g2ku25hvmd53zj7og4txpby3vsusfefw5rrg5sii # ./root2\nROOT3_CID=bafybeiawdvhmjcz65x5egzx4iukxc72hg4woks6v6fvgyupiyt3oczk5ja # ./root2/root3\nROOT4_CID=bafybeifq2rzpqnqrsdupncmkmhs3ckxxjhuvdcbvydkgvch3ms24k5lo7q # ./root2/root3/root4\nFILE_CID=bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am # ./root2/root3/root4/index.html\nTEST_IPNS_ID=k51qzi5uqu5dlxdsdu5fpuu7h69wu4ohp32iwm9pdt9nq3y5rpn3ln9j12zfhe\n\n# Import test case\n# See the static fixtures in ./t0116-gateway-cache/\ntest_expect_success \"Add the test directory\" '\n ipfs dag import --pin-roots ../t0116-gateway-cache/fixtures.car\n ipfs routing put --allow-offline /ipns/${TEST_IPNS_ID} ../t0116-gateway-cache/${TEST_IPNS_ID}.ipns-record\n'\n\n# Etag\n test_expect_success \"GET for /ipfs/ unixfs dir listing succeeds\" '\n curl -svX GET \"http://127.0.0.1:$GWAY_PORT/ipfs/$ROOT1_CID/root2/root3/\" >/dev/null 2>curl_ipfs_dir_listing_output\n '\n\n test_expect_success \"GET for /ipns/ unixfs dir listing succeeds\" '\n curl -svX GET \"http://127.0.0.1:$GWAY_PORT/ipns/$TEST_IPNS_ID/root2/root3/\" >/dev/null 2>curl_ipns_dir_listing_output\n '\n\n ## dir generated listing\n test_expect_success \"GET /ipfs/ dir response has special Etag for generated dir listing\" '\n test_should_contain \"< Etag: \\\"DirIndex\" curl_ipfs_dir_listing_output &&\n grep -E \"< Etag: \\\"DirIndex-.+_CID-${ROOT3_CID}\\\"\" curl_ipfs_dir_listing_output\n '\n test_expect_success \"GET /ipns/ dir response has special Etag for generated dir listing\" '\n test_should_contain \"< Etag: \\\"DirIndex\" curl_ipns_dir_listing_output &&\n grep -E \"< Etag: \\\"DirIndex-.+_CID-${ROOT3_CID}\\\"\" curl_ipns_dir_listing_output\n '\n\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0119-prometheus-data/prometheus_metrics", "content": "exchange_bitswap_requests_in_flight\nexchange_bitswap_response_bytes_bucket\nexchange_bitswap_response_bytes_count\nexchange_bitswap_response_bytes_sum\nexchange_bitswap_wantlists_items_total\nexchange_bitswap_wantlists_seconds_bucket\nexchange_bitswap_wantlists_seconds_count\nexchange_bitswap_wantlists_seconds_sum\nexchange_bitswap_wantlists_total\nexchange_httpnet_request_duration_seconds_bucket\nexchange_httpnet_request_duration_seconds_count\nexchange_httpnet_request_duration_seconds_sum\nexchange_httpnet_request_sent_bytes\nexchange_httpnet_requests_body_failure\nexchange_httpnet_requests_failure\nexchange_httpnet_requests_in_flight\nexchange_httpnet_requests_total\nexchange_httpnet_response_bytes_bucket\nexchange_httpnet_response_bytes_count\nexchange_httpnet_response_bytes_sum\nexchange_httpnet_wantlists_items_total\nexchange_httpnet_wantlists_seconds_bucket\nexchange_httpnet_wantlists_seconds_count\nexchange_httpnet_wantlists_seconds_sum\nexchange_httpnet_wantlists_total\ngo_gc_duration_seconds\ngo_gc_duration_seconds_count\ngo_gc_duration_seconds_sum\ngo_gc_gogc_percent\ngo_gc_gomemlimit_bytes\ngo_goroutines\ngo_info\ngo_memstats_alloc_bytes\ngo_memstats_alloc_bytes_total\ngo_memstats_buck_hash_sys_bytes\ngo_memstats_frees_total\ngo_memstats_gc_sys_bytes\ngo_memstats_heap_alloc_bytes\ngo_memstats_heap_idle_bytes\ngo_memstats_heap_inuse_bytes\ngo_memstats_heap_objects\ngo_memstats_heap_released_bytes\ngo_memstats_heap_sys_bytes\ngo_memstats_last_gc_time_seconds\ngo_memstats_mallocs_total\ngo_memstats_mcache_inuse_bytes\ngo_memstats_mcache_sys_bytes\ngo_memstats_mspan_inuse_bytes\ngo_memstats_mspan_sys_bytes\ngo_memstats_next_gc_bytes\ngo_memstats_other_sys_bytes\ngo_memstats_stack_inuse_bytes\ngo_memstats_stack_sys_bytes\ngo_memstats_sys_bytes\ngo_sched_gomaxprocs_threads\ngo_threads\nhttp_server_request_body_size_bytes_bucket\nhttp_server_request_body_size_bytes_count\nhttp_server_request_body_size_bytes_sum\nhttp_server_request_duration_seconds_bucket\nhttp_server_request_duration_seconds_count\nhttp_server_request_duration_seconds_sum\nhttp_server_response_body_size_bytes_bucket\nhttp_server_response_body_size_bytes_count\nhttp_server_response_body_size_bytes_sum\nipfs_bitswap_active_block_tasks\nipfs_bitswap_active_tasks\nipfs_bitswap_bcast_skips_total\nipfs_bitswap_blocks_received\nipfs_bitswap_haves_received\nipfs_bitswap_pending_block_tasks\nipfs_bitswap_pending_tasks\nipfs_bitswap_recv_all_blocks_bytes_bucket\nipfs_bitswap_recv_all_blocks_bytes_count\nipfs_bitswap_recv_all_blocks_bytes_sum\nipfs_bitswap_recv_dup_blocks_bytes_bucket\nipfs_bitswap_recv_dup_blocks_bytes_count\nipfs_bitswap_recv_dup_blocks_bytes_sum\nipfs_bitswap_send_times_bucket\nipfs_bitswap_send_times_count\nipfs_bitswap_send_times_sum\nipfs_bitswap_sent_all_blocks_bytes_bucket\nipfs_bitswap_sent_all_blocks_bytes_count\nipfs_bitswap_sent_all_blocks_bytes_sum\nipfs_bitswap_want_blocks_total\nipfs_bitswap_wanthaves_broadcast\nipfs_bitswap_wantlist_total\nipfs_bs_cache_boxo_blockstore_cache_hits\nipfs_bs_cache_boxo_blockstore_cache_total\nipfs_fsrepo_datastore_batchcommit_errors_total\nipfs_fsrepo_datastore_batchcommit_latency_seconds_bucket\nipfs_fsrepo_datastore_batchcommit_latency_seconds_count\nipfs_fsrepo_datastore_batchcommit_latency_seconds_sum\nipfs_fsrepo_datastore_batchcommit_total\nipfs_fsrepo_datastore_batchdelete_errors_total\nipfs_fsrepo_datastore_batchdelete_latency_seconds_bucket\nipfs_fsrepo_datastore_batchdelete_latency_seconds_count\nipfs_fsrepo_datastore_batchdelete_latency_seconds_sum\nipfs_fsrepo_datastore_batchdelete_total\nipfs_fsrepo_datastore_batchput_errors_total\nipfs_fsrepo_datastore_batchput_latency_seconds_bucket\nipfs_fsrepo_datastore_batchput_latency_seconds_count\nipfs_fsrepo_datastore_batchput_latency_seconds_sum\nipfs_fsrepo_datastore_batchput_size_bytes_bucket\nipfs_fsrepo_datastore_batchput_size_bytes_count\nipfs_fsrepo_datastore_batchput_size_bytes_sum\nipfs_fsrepo_datastore_batchput_total\nipfs_fsrepo_datastore_check_errors_total\nipfs_fsrepo_datastore_check_latency_seconds_bucket\nipfs_fsrepo_datastore_check_latency_seconds_count\nipfs_fsrepo_datastore_check_latency_seconds_sum\nipfs_fsrepo_datastore_check_total\nipfs_fsrepo_datastore_delete_errors_total\nipfs_fsrepo_datastore_delete_latency_seconds_bucket\nipfs_fsrepo_datastore_delete_latency_seconds_count\nipfs_fsrepo_datastore_delete_latency_seconds_sum\nipfs_fsrepo_datastore_delete_total\nipfs_fsrepo_datastore_du_errors_total\nipfs_fsrepo_datastore_du_latency_seconds_bucket\nipfs_fsrepo_datastore_du_latency_seconds_count\nipfs_fsrepo_datastore_du_latency_seconds_sum\nipfs_fsrepo_datastore_du_total\nipfs_fsrepo_datastore_gc_errors_total\nipfs_fsrepo_datastore_gc_latency_seconds_bucket\nipfs_fsrepo_datastore_gc_latency_seconds_count\nipfs_fsrepo_datastore_gc_latency_seconds_sum\nipfs_fsrepo_datastore_gc_total\nipfs_fsrepo_datastore_get_errors_total\nipfs_fsrepo_datastore_get_latency_seconds_bucket\nipfs_fsrepo_datastore_get_latency_seconds_count\nipfs_fsrepo_datastore_get_latency_seconds_sum\nipfs_fsrepo_datastore_get_size_bytes_bucket\nipfs_fsrepo_datastore_get_size_bytes_count\nipfs_fsrepo_datastore_get_size_bytes_sum\nipfs_fsrepo_datastore_get_total\nipfs_fsrepo_datastore_getsize_errors_total\nipfs_fsrepo_datastore_getsize_latency_seconds_bucket\nipfs_fsrepo_datastore_getsize_latency_seconds_count\nipfs_fsrepo_datastore_getsize_latency_seconds_sum\nipfs_fsrepo_datastore_getsize_total\nipfs_fsrepo_datastore_has_errors_total\nipfs_fsrepo_datastore_has_latency_seconds_bucket\nipfs_fsrepo_datastore_has_latency_seconds_count\nipfs_fsrepo_datastore_has_latency_seconds_sum\nipfs_fsrepo_datastore_has_total\nipfs_fsrepo_datastore_put_errors_total\nipfs_fsrepo_datastore_put_latency_seconds_bucket\nipfs_fsrepo_datastore_put_latency_seconds_count\nipfs_fsrepo_datastore_put_latency_seconds_sum\nipfs_fsrepo_datastore_put_size_bytes_bucket\nipfs_fsrepo_datastore_put_size_bytes_count\nipfs_fsrepo_datastore_put_size_bytes_sum\nipfs_fsrepo_datastore_put_total\nipfs_fsrepo_datastore_query_errors_total\nipfs_fsrepo_datastore_query_latency_seconds_bucket\nipfs_fsrepo_datastore_query_latency_seconds_count\nipfs_fsrepo_datastore_query_latency_seconds_sum\nipfs_fsrepo_datastore_query_total\nipfs_fsrepo_datastore_scrub_errors_total\nipfs_fsrepo_datastore_scrub_latency_seconds_bucket\nipfs_fsrepo_datastore_scrub_latency_seconds_count\nipfs_fsrepo_datastore_scrub_latency_seconds_sum\nipfs_fsrepo_datastore_scrub_total\nipfs_fsrepo_datastore_sync_errors_total\nipfs_fsrepo_datastore_sync_latency_seconds_bucket\nipfs_fsrepo_datastore_sync_latency_seconds_count\nipfs_fsrepo_datastore_sync_latency_seconds_sum\nipfs_fsrepo_datastore_sync_total\nipfs_http_gw_concurrent_requests\nipfs_http_request_duration_seconds\nipfs_http_request_duration_seconds_count\nipfs_http_request_duration_seconds_sum\nipfs_http_request_size_bytes\nipfs_http_request_size_bytes_count\nipfs_http_request_size_bytes_sum\nipfs_http_requests_total\nipfs_http_response_size_bytes\nipfs_http_response_size_bytes_count\nipfs_http_response_size_bytes_sum\nipfs_info\nlibp2p_autonat_next_probe_timestamp\nlibp2p_autonat_reachability_status\nlibp2p_autonat_reachability_status_confidence\nlibp2p_autorelay_candidate_loop_state\nlibp2p_autorelay_candidates_circuit_v2_support_total\nlibp2p_autorelay_desired_reservations\nlibp2p_autorelay_relay_addresses_count\nlibp2p_autorelay_relay_addresses_updated_total\nlibp2p_autorelay_reservation_requests_outcome_total\nlibp2p_autorelay_reservations_closed_total\nlibp2p_autorelay_reservations_opened_total\nlibp2p_autorelay_status\nlibp2p_eventbus_events_emitted_total\nlibp2p_eventbus_subscriber_event_queued\nlibp2p_eventbus_subscriber_queue_full\nlibp2p_eventbus_subscriber_queue_length\nlibp2p_eventbus_subscribers_total\nlibp2p_holepunch_address_outcomes_total\nlibp2p_holepunch_outcomes_total\nlibp2p_identify_addrs_count\nlibp2p_identify_addrs_received_bucket\nlibp2p_identify_addrs_received_count\nlibp2p_identify_addrs_received_sum\nlibp2p_identify_identify_pushes_triggered_total\nlibp2p_identify_protocols_count\nlibp2p_identify_protocols_received_bucket\nlibp2p_identify_protocols_received_count\nlibp2p_identify_protocols_received_sum\nlibp2p_rcmgr_conn_memory_bucket\nlibp2p_rcmgr_conn_memory_count\nlibp2p_rcmgr_conn_memory_sum\nlibp2p_rcmgr_connections\nlibp2p_rcmgr_fds\nlibp2p_rcmgr_peer_connections_bucket\nlibp2p_rcmgr_peer_connections_count\nlibp2p_rcmgr_peer_connections_sum\nlibp2p_rcmgr_peer_memory_bucket\nlibp2p_rcmgr_peer_memory_count\nlibp2p_rcmgr_peer_memory_sum\nlibp2p_rcmgr_peer_streams_bucket\nlibp2p_rcmgr_peer_streams_count\nlibp2p_rcmgr_peer_streams_sum\nlibp2p_rcmgr_previous_conn_memory_bucket\nlibp2p_rcmgr_previous_conn_memory_count\nlibp2p_rcmgr_previous_conn_memory_sum\nlibp2p_rcmgr_previous_peer_connections_bucket\nlibp2p_rcmgr_previous_peer_connections_count\nlibp2p_rcmgr_previous_peer_connections_sum\nlibp2p_rcmgr_previous_peer_memory_bucket\nlibp2p_rcmgr_previous_peer_memory_count\nlibp2p_rcmgr_previous_peer_memory_sum\nlibp2p_rcmgr_previous_peer_streams_bucket\nlibp2p_rcmgr_previous_peer_streams_count\nlibp2p_rcmgr_previous_peer_streams_sum\nlibp2p_relaysvc_connection_duration_seconds_bucket\nlibp2p_relaysvc_connection_duration_seconds_count\nlibp2p_relaysvc_connection_duration_seconds_sum\nlibp2p_relaysvc_data_transferred_bytes_total\nlibp2p_relaysvc_status\nlibp2p_swarm_dial_ranking_delay_seconds_bucket\nlibp2p_swarm_dial_ranking_delay_seconds_count\nlibp2p_swarm_dial_ranking_delay_seconds_sum\notel_scope_info\nprocess_cpu_seconds_total\nprocess_max_fds\nprocess_network_receive_bytes_total\nprocess_network_transmit_bytes_total\nprocess_open_fds\nprocess_resident_memory_bytes\nprocess_start_time_seconds\nprocess_virtual_memory_bytes\nprocess_virtual_memory_max_bytes\nprovider_provides_total\ntarget_info\n" }, { "path": "test/sharness/t0119-prometheus-data/prometheus_metrics_added_by_enabling_rcmgr", "content": "libp2p_rcmgr_limit\n" }, { "path": "test/sharness/t0119-prometheus-data/prometheus_metrics_added_by_measure_profile", "content": "flatfs_datastore_batchcommit_errors_total\nflatfs_datastore_batchcommit_latency_seconds_bucket\nflatfs_datastore_batchcommit_latency_seconds_count\nflatfs_datastore_batchcommit_latency_seconds_sum\nflatfs_datastore_batchcommit_total\nflatfs_datastore_batchdelete_errors_total\nflatfs_datastore_batchdelete_latency_seconds_bucket\nflatfs_datastore_batchdelete_latency_seconds_count\nflatfs_datastore_batchdelete_latency_seconds_sum\nflatfs_datastore_batchdelete_total\nflatfs_datastore_batchput_errors_total\nflatfs_datastore_batchput_latency_seconds_bucket\nflatfs_datastore_batchput_latency_seconds_count\nflatfs_datastore_batchput_latency_seconds_sum\nflatfs_datastore_batchput_size_bytes_bucket\nflatfs_datastore_batchput_size_bytes_count\nflatfs_datastore_batchput_size_bytes_sum\nflatfs_datastore_batchput_total\nflatfs_datastore_check_errors_total\nflatfs_datastore_check_latency_seconds_bucket\nflatfs_datastore_check_latency_seconds_count\nflatfs_datastore_check_latency_seconds_sum\nflatfs_datastore_check_total\nflatfs_datastore_delete_errors_total\nflatfs_datastore_delete_latency_seconds_bucket\nflatfs_datastore_delete_latency_seconds_count\nflatfs_datastore_delete_latency_seconds_sum\nflatfs_datastore_delete_total\nflatfs_datastore_du_errors_total\nflatfs_datastore_du_latency_seconds_bucket\nflatfs_datastore_du_latency_seconds_count\nflatfs_datastore_du_latency_seconds_sum\nflatfs_datastore_du_total\nflatfs_datastore_gc_errors_total\nflatfs_datastore_gc_latency_seconds_bucket\nflatfs_datastore_gc_latency_seconds_count\nflatfs_datastore_gc_latency_seconds_sum\nflatfs_datastore_gc_total\nflatfs_datastore_get_errors_total\nflatfs_datastore_get_latency_seconds_bucket\nflatfs_datastore_get_latency_seconds_count\nflatfs_datastore_get_latency_seconds_sum\nflatfs_datastore_get_size_bytes_bucket\nflatfs_datastore_get_size_bytes_count\nflatfs_datastore_get_size_bytes_sum\nflatfs_datastore_get_total\nflatfs_datastore_getsize_errors_total\nflatfs_datastore_getsize_latency_seconds_bucket\nflatfs_datastore_getsize_latency_seconds_count\nflatfs_datastore_getsize_latency_seconds_sum\nflatfs_datastore_getsize_total\nflatfs_datastore_has_errors_total\nflatfs_datastore_has_latency_seconds_bucket\nflatfs_datastore_has_latency_seconds_count\nflatfs_datastore_has_latency_seconds_sum\nflatfs_datastore_has_total\nflatfs_datastore_put_errors_total\nflatfs_datastore_put_latency_seconds_bucket\nflatfs_datastore_put_latency_seconds_count\nflatfs_datastore_put_latency_seconds_sum\nflatfs_datastore_put_size_bytes_bucket\nflatfs_datastore_put_size_bytes_count\nflatfs_datastore_put_size_bytes_sum\nflatfs_datastore_put_total\nflatfs_datastore_query_errors_total\nflatfs_datastore_query_latency_seconds_bucket\nflatfs_datastore_query_latency_seconds_count\nflatfs_datastore_query_latency_seconds_sum\nflatfs_datastore_query_total\nflatfs_datastore_scrub_errors_total\nflatfs_datastore_scrub_latency_seconds_bucket\nflatfs_datastore_scrub_latency_seconds_count\nflatfs_datastore_scrub_latency_seconds_sum\nflatfs_datastore_scrub_total\nflatfs_datastore_sync_errors_total\nflatfs_datastore_sync_latency_seconds_bucket\nflatfs_datastore_sync_latency_seconds_count\nflatfs_datastore_sync_latency_seconds_sum\nflatfs_datastore_sync_total\nleveldb_datastore_batchcommit_errors_total\nleveldb_datastore_batchcommit_latency_seconds_bucket\nleveldb_datastore_batchcommit_latency_seconds_count\nleveldb_datastore_batchcommit_latency_seconds_sum\nleveldb_datastore_batchcommit_total\nleveldb_datastore_batchdelete_errors_total\nleveldb_datastore_batchdelete_latency_seconds_bucket\nleveldb_datastore_batchdelete_latency_seconds_count\nleveldb_datastore_batchdelete_latency_seconds_sum\nleveldb_datastore_batchdelete_total\nleveldb_datastore_batchput_errors_total\nleveldb_datastore_batchput_latency_seconds_bucket\nleveldb_datastore_batchput_latency_seconds_count\nleveldb_datastore_batchput_latency_seconds_sum\nleveldb_datastore_batchput_size_bytes_bucket\nleveldb_datastore_batchput_size_bytes_count\nleveldb_datastore_batchput_size_bytes_sum\nleveldb_datastore_batchput_total\nleveldb_datastore_check_errors_total\nleveldb_datastore_check_latency_seconds_bucket\nleveldb_datastore_check_latency_seconds_count\nleveldb_datastore_check_latency_seconds_sum\nleveldb_datastore_check_total\nleveldb_datastore_delete_errors_total\nleveldb_datastore_delete_latency_seconds_bucket\nleveldb_datastore_delete_latency_seconds_count\nleveldb_datastore_delete_latency_seconds_sum\nleveldb_datastore_delete_total\nleveldb_datastore_du_errors_total\nleveldb_datastore_du_latency_seconds_bucket\nleveldb_datastore_du_latency_seconds_count\nleveldb_datastore_du_latency_seconds_sum\nleveldb_datastore_du_total\nleveldb_datastore_gc_errors_total\nleveldb_datastore_gc_latency_seconds_bucket\nleveldb_datastore_gc_latency_seconds_count\nleveldb_datastore_gc_latency_seconds_sum\nleveldb_datastore_gc_total\nleveldb_datastore_get_errors_total\nleveldb_datastore_get_latency_seconds_bucket\nleveldb_datastore_get_latency_seconds_count\nleveldb_datastore_get_latency_seconds_sum\nleveldb_datastore_get_size_bytes_bucket\nleveldb_datastore_get_size_bytes_count\nleveldb_datastore_get_size_bytes_sum\nleveldb_datastore_get_total\nleveldb_datastore_getsize_errors_total\nleveldb_datastore_getsize_latency_seconds_bucket\nleveldb_datastore_getsize_latency_seconds_count\nleveldb_datastore_getsize_latency_seconds_sum\nleveldb_datastore_getsize_total\nleveldb_datastore_has_errors_total\nleveldb_datastore_has_latency_seconds_bucket\nleveldb_datastore_has_latency_seconds_count\nleveldb_datastore_has_latency_seconds_sum\nleveldb_datastore_has_total\nleveldb_datastore_put_errors_total\nleveldb_datastore_put_latency_seconds_bucket\nleveldb_datastore_put_latency_seconds_count\nleveldb_datastore_put_latency_seconds_sum\nleveldb_datastore_put_size_bytes_bucket\nleveldb_datastore_put_size_bytes_count\nleveldb_datastore_put_size_bytes_sum\nleveldb_datastore_put_total\nleveldb_datastore_query_errors_total\nleveldb_datastore_query_latency_seconds_bucket\nleveldb_datastore_query_latency_seconds_count\nleveldb_datastore_query_latency_seconds_sum\nleveldb_datastore_query_total\nleveldb_datastore_scrub_errors_total\nleveldb_datastore_scrub_latency_seconds_bucket\nleveldb_datastore_scrub_latency_seconds_count\nleveldb_datastore_scrub_latency_seconds_sum\nleveldb_datastore_scrub_total\nleveldb_datastore_sync_errors_total\nleveldb_datastore_sync_latency_seconds_bucket\nleveldb_datastore_sync_latency_seconds_count\nleveldb_datastore_sync_latency_seconds_sum\nleveldb_datastore_sync_total\nlibp2p_rcmgr_limit\n" }, { "path": "test/sharness/t0119-prometheus.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2020 Protocol Labs\n# MIT/Apache-2.0 Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test prometheus metrics are exposed correctly\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"enable ResourceMgr in the config\" '\n ipfs config --json Swarm.ResourceMgr.Enabled false\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"collect metrics\" '\n curl \"$API_ADDR/debug/metrics/prometheus\" > raw_metrics\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"filter metrics\" '\n sed -ne \"s/^\\([a-z0-9_]\\+\\).*/\\1/p\" raw_metrics | LC_ALL=C sort | uniq > filtered_metrics\n'\n\ntest_expect_success \"make sure metrics haven't changed\" '\n diff -u ../t0119-prometheus-data/prometheus_metrics filtered_metrics\n'\n\n# Check what was added by enabling ResourceMgr.Enabled\n#\n# NOTE: we won't see all the dynamic ones, but that is ok: the point of the\n# test here is to detect regression when rcmgr metrics disappear due to\n# refactor/human error.\n\ntest_expect_success \"enable ResourceMgr in the config\" '\n ipfs config --json Swarm.ResourceMgr.Enabled true\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"collect metrics\" '\n curl \"$API_ADDR/debug/metrics/prometheus\" > raw_metrics\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"filter metrics and find ones added by enabling ResourceMgr\" '\n sed -ne \"s/^\\([a-z0-9_]\\+\\).*/\\1/p\" raw_metrics | LC_ALL=C sort > filtered_metrics &&\n grep -v -x -f ../t0119-prometheus-data/prometheus_metrics filtered_metrics | LC_ALL=C sort | uniq > rcmgr_metrics\n'\n\ntest_expect_success \"make sure initial metrics added by setting ResourceMgr.Enabled haven't changed\" '\n diff -u ../t0119-prometheus-data/prometheus_metrics_added_by_enabling_rcmgr rcmgr_metrics\n'\n\n# Reinitialize ipfs with --profile=flatfs-measure and check metrics.\n\ntest_expect_success \"remove ipfs directory\" '\n rm -rf .ipfs mountdir ipfs ipns\n'\n\ntest_init_ipfs_measure\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"collect metrics\" '\n curl \"$API_ADDR/debug/metrics/prometheus\" > raw_metrics\n'\ntest_kill_ipfs_daemon\n\ntest_expect_success \"filter metrics and find ones added by enabling flatfs-measure profile\" '\n sed -ne \"s/^\\([a-z0-9_]\\+\\).*/\\1/p\" raw_metrics | LC_ALL=C sort > filtered_metrics &&\n grep -v -x -f ../t0119-prometheus-data/prometheus_metrics filtered_metrics | LC_ALL=C sort | uniq > measure_metrics\n'\n\ntest_expect_success \"make sure initial metrics added by initializing with flatfs-measure profile haven't changed\" '\n diff -u ../t0119-prometheus-data/prometheus_metrics_added_by_measure_profile measure_metrics\n'\n\ntest_done\n" }, { "path": "test/sharness/t0120-bootstrap.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\n# changing the bootstrap peers will require changing it in two places :)\nBP1=\"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN\"\nBP2=\"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa\"\nBP3=\"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb\"\nBP4=\"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt\"\nBP5=\"/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8\"\nBP6=\"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\nBP7=\"/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ\"\n\ntest_description=\"Test ipfs bootstrap operations\"\n\n# NOTE: For AutoConf bootstrap functionality (add default, --expand-auto, etc.)\n# see test/cli/bootstrap_auto_test.go and test/cli/autoconf/expand_test.go\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# we use a function so that we can run it both offline + online\ntest_bootstrap_list_cmd() {\n printf \"\" >list_expected\n\n for BP in \"$@\"\n do\n echo \"$BP\" >>list_expected\n done\n\n test_expect_success \"'ipfs bootstrap' succeeds\" '\n ipfs bootstrap >list_actual\n '\n\n test_expect_success \"'ipfs bootstrap' output looks good\" '\n test_cmp list_expected list_actual\n '\n\n test_expect_success \"'ipfs bootstrap list' succeeds\" '\n ipfs bootstrap list >list2_actual\n '\n\n test_expect_success \"'ipfs bootstrap list' output looks good\" '\n test_cmp list_expected list2_actual\n '\n}\n\n# we use a function so that we can run it both offline + online\ntest_bootstrap_cmd() {\n\n # remove all peers just in case.\n # if this fails, the first listing may not be empty\n ipfs bootstrap rm --all\n\n test_bootstrap_list_cmd\n\n test_expect_success \"'ipfs bootstrap add' succeeds\" '\n ipfs bootstrap add \"$BP1\" \"$BP2\" \"$BP3\" >add_actual\n '\n\n test_expect_success \"'ipfs bootstrap add' output looks good\" '\n echo \"added $BP1\" >add_expected &&\n echo \"added $BP2\" >>add_expected &&\n echo \"added $BP3\" >>add_expected &&\n test_cmp add_expected add_actual\n '\n\n test_bootstrap_list_cmd $BP1 $BP2 $BP3\n\n test_expect_success \"'ipfs bootstrap rm' succeeds\" '\n ipfs bootstrap rm \"$BP1\" \"$BP3\" >rm_actual\n '\n\n test_expect_success \"'ipfs bootstrap rm' output looks good\" '\n echo \"removed $BP1\" >rm_expected &&\n echo \"removed $BP3\" >>rm_expected &&\n test_cmp rm_expected rm_actual\n '\n\n test_expect_success \"'ipfs bootstrap rm' fails on bad peers\" '\n test_expect_code 1 ipfs bootstrap rm \"foo/bar\"\n '\n\n test_bootstrap_list_cmd $BP2\n\n test_expect_success \"'ipfs bootstrap rm --all' succeeds\" '\n ipfs bootstrap rm --all >rm2_actual\n '\n\n test_expect_success \"'ipfs bootstrap rm' output looks good\" '\n echo \"removed $BP2\" >rm2_expected &&\n test_cmp rm2_expected rm2_actual\n '\n\n test_bootstrap_list_cmd\n\n test_expect_success \"'ipfs bootstrap add' accepts args from stdin\" '\n echo $BP1 > bpeers &&\n echo $BP2 >> bpeers &&\n echo $BP3 >> bpeers &&\n echo $BP4 >> bpeers &&\n cat bpeers | ipfs bootstrap add > add_stdin_actual\n '\n\n test_expect_success \"output looks good\" '\n echo \"added $BP1\" > bpeers_add_exp &&\n echo \"added $BP2\" >> bpeers_add_exp &&\n echo \"added $BP3\" >> bpeers_add_exp &&\n echo \"added $BP4\" >> bpeers_add_exp &&\n test_cmp add_stdin_actual bpeers_add_exp\n '\n\n test_bootstrap_list_cmd $BP1 $BP2 $BP3 $BP4\n\n test_expect_success \"'ipfs bootstrap rm' accepts args from stdin\" '\n cat bpeers | ipfs bootstrap rm > rm_stdin_actual\n '\n\n test_expect_success \"output looks good\" '\n echo \"removed $BP1\" > bpeers_rm_exp &&\n echo \"removed $BP2\" >> bpeers_rm_exp &&\n echo \"removed $BP3\" >> bpeers_rm_exp &&\n echo \"removed $BP4\" >> bpeers_rm_exp &&\n test_cmp rm_stdin_actual bpeers_rm_exp\n '\n\n test_bootstrap_list_cmd\n}\n\n# should work offline\ntest_bootstrap_cmd\n\n# should work online\ntest_launch_ipfs_daemon\ntest_bootstrap_cmd\ntest_kill_ipfs_daemon\n\n\ntest_done\n" }, { "path": "test/sharness/t0121-bootstrap-iptb.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\n# changing the bootstrap peers will require changing it in two places :)\ntest_description=\"test node bootstrapping\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"disable mdns\" '\n ipfs config Discovery.MDNS.Enabled false --json\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"setup iptb nodes\" '\n iptb testbed create -type localipfs -count 5 -force -init\n'\n\ntest_expect_success \"start up iptb nodes\" '\n iptb start -wait\n'\n\ntest_expect_success \"check peers works\" '\n ipfs swarm peers >peers_out\n'\n\ntest_expect_success \"correct number of peers\" '\n test -z \"`cat peers_out`\"\n'\n\nbetterwait() {\n while kill -0 $1; do true; done\n}\n\ntest_expect_success \"bring down iptb nodes\" '\n PID0=$(cat \"$IPTB_ROOT/testbeds/default/0/daemon.pid\") &&\n PID1=$(cat \"$IPTB_ROOT/testbeds/default/1/daemon.pid\") &&\n PID2=$(cat \"$IPTB_ROOT/testbeds/default/2/daemon.pid\") &&\n PID3=$(cat \"$IPTB_ROOT/testbeds/default/3/daemon.pid\") &&\n PID4=$(cat \"$IPTB_ROOT/testbeds/default/4/daemon.pid\") &&\n iptb stop && # TODO: add --wait flag to iptb stop\n betterwait $PID0\n betterwait $PID1\n betterwait $PID2\n betterwait $PID3\n betterwait $PID4\n'\n\ntest_expect_success \"reset iptb nodes\" '\n # the api does not seem to get cleaned up in sharness tests for some reason\n iptb testbed create -type localipfs -count 5 -force -init\n'\n\ntest_expect_success \"set bootstrap addrs\" '\n bsn_peer_id=$(ipfs id -f \"\") &&\n BADDR=\"/ip4/127.0.0.1/tcp/$SWARM_PORT/p2p/$bsn_peer_id\" &&\n ipfsi 0 bootstrap add $BADDR &&\n ipfsi 1 bootstrap add $BADDR &&\n ipfsi 2 bootstrap add $BADDR &&\n ipfsi 3 bootstrap add $BADDR &&\n ipfsi 4 bootstrap add $BADDR\n'\n\ntest_expect_success \"start up iptb nodes\" '\n iptb start -wait\n'\n\ntest_expect_success \"check peers works\" '\n ipfs swarm peers > peers_out\n'\n\ntest_expect_success \"correct number of peers\" '\n test `cat peers_out | wc -l` = 5\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"bring down iptb nodes\" '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0131-multinode-client-routing.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test client mode dht\"\n\n. lib/test-lib.sh\n\ncheck_file_fetch() {\n node=$1\n fhash=$2\n fname=$3\n\n test_expect_success \"can fetch file\" '\n ipfsi $node cat $fhash > fetch_out\n '\n\n test_expect_success \"file looks good\" '\n test_cmp $fname fetch_out\n '\n}\n\nrun_single_file_test() {\n test_expect_success \"add a file on node1\" '\n random-data -size=1000000 > filea &&\n FILEA_HASH=$(ipfsi 1 add -q filea)\n '\n\n check_file_fetch 9 $FILEA_HASH filea\n check_file_fetch 8 $FILEA_HASH filea\n check_file_fetch 7 $FILEA_HASH filea\n check_file_fetch 6 $FILEA_HASH filea\n check_file_fetch 5 $FILEA_HASH filea\n check_file_fetch 4 $FILEA_HASH filea\n check_file_fetch 3 $FILEA_HASH filea\n check_file_fetch 2 $FILEA_HASH filea\n check_file_fetch 1 $FILEA_HASH filea\n check_file_fetch 0 $FILEA_HASH filea\n}\n\nNNODES=10\n\ntest_expect_success \"set up testbed\" '\n iptb testbed create -type localipfs -count $NNODES -force -init &&\n iptb run -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true\n'\n\ntest_expect_success \"start up nodes\" '\n iptb start -wait [0-7] &&\n iptb start -wait [8-9] -- --routing=dhtclient\n'\n\ntest_expect_success \"connect up nodes\" '\n iptb connect [1-9] 0\n'\n\ntest_expect_success \"add a file on a node in client mode\" '\n random-data -size=1000000 > filea &&\n FILE_HASH=$(ipfsi 8 add -q filea)\n'\n\ntest_expect_success \"retrieve that file on a node in client mode\" '\n check_file_fetch 9 $FILE_HASH filea\n'\n\nrun_single_file_test\n\ntest_expect_success \"shut down nodes\" '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0140-swarm.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs swarm command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_launch_ipfs_daemon\n\ntest_expect_success 'disconnected: peers is empty' '\n ipfs swarm peers >actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success 'disconnected: addrs local has localhost' '\n ipfs swarm addrs local >actual &&\n grep \"/ip4/127.0.0.1\" actual\n'\n\ntest_expect_success 'disconnected: addrs local matches ipfs id' '\n ipfs id -f=\"\\\\n\" | sort >expected &&\n ipfs swarm addrs local --id | sort >actual &&\n test_cmp expected actual\n'\n\ntest_expect_success \"ipfs id self works\" '\n myid=$(ipfs id -f=\"\") &&\n ipfs id --timeout=1s $myid > output\n'\n\ntest_expect_success \"output looks good\" '\n grep $myid output &&\n grep PublicKey output\n'\n\naddr=\"/ip4/127.0.0.1/tcp/9898/p2p/QmUWKoHbjsqsSMesRC2Zoscs8edyFz6F77auBB1YBBhgpX\"\n\ntest_expect_success \"can't trigger a dial backoff with swarm connect\" '\n test_expect_code 1 ipfs swarm connect $addr 2> connect_out\n test_expect_code 1 ipfs swarm connect $addr 2>> connect_out\n test_expect_code 1 ipfs swarm connect $addr 2>> connect_out\n test_expect_code 1 grep \"backoff\" connect_out\n'\n\ntest_kill_ipfs_daemon\n\nannounceCfg='[\"/ip4/127.0.0.1/tcp/4001\", \"/ip4/1.2.3.4/tcp/1234\"]'\ntest_expect_success \"test_config_set succeeds\" \"\n ipfs config --json Addresses.Announce '$announceCfg'\n\"\n\ntest_launch_ipfs_daemon\n\ntest_expect_success 'Addresses.Announce affects addresses' '\n ipfs swarm addrs local >actual &&\n test_should_contain \"/ip4/1.2.3.4/tcp/1234\" actual &&\n ipfs id -f\"\" | xargs -n1 echo >actual &&\n test_should_contain \"/ip4/1.2.3.4/tcp/1234\" actual\n'\n\ntest_kill_ipfs_daemon\n\n\nannounceCfg='[\"/ip4/127.0.0.1/tcp/4001\", \"/ip4/1.2.3.4/tcp/1234\"]'\ntest_expect_success \"test_config_set succeeds\" \"\n ipfs config --json Addresses.Announce '$announceCfg'\n\"\n# Include \"/ip4/1.2.3.4/tcp/1234\" to ensure we deduplicate addrs already present in Swarm.Announce\nappendAnnounceCfg='[\"/dnsaddr/dynamic.example.com\", \"/ip4/10.20.30.40/tcp/4321\", \"/ip4/1.2.3.4/tcp/1234\"]'\ntest_expect_success \"test_config_set Announce and AppendAnnounce succeeds\" \"\n ipfs config --json Addresses.Announce '$announceCfg' &&\n ipfs config --json Addresses.AppendAnnounce '$appendAnnounceCfg'\n\"\n\ntest_launch_ipfs_daemon\n\ntest_expect_success 'Addresses.AppendAnnounce is applied on top of Announce' '\n ipfs swarm addrs local >actual &&\n test_should_contain \"/ip4/1.2.3.4/tcp/1234\" actual &&\n test_should_contain \"/dnsaddr/dynamic.example.com\" actual &&\n test_should_contain \"/ip4/10.20.30.40/tcp/4321\" actual &&\n ipfs id -f\"\" | xargs -n1 echo | tee actual &&\n test_should_contain \"/ip4/1.2.3.4/tcp/1234/p2p\" actual &&\n test_should_contain \"/dnsaddr/dynamic.example.com/p2p/\" actual &&\n test_should_contain \"/ip4/10.20.30.40/tcp/4321/p2p/\" actual\n'\n\ntest_kill_ipfs_daemon\n\nnoAnnounceCfg='[\"/ip4/1.2.3.4/tcp/1234\", \"/ip4/10.20.30.40/tcp/4321\"]'\ntest_expect_success \"test_config_set succeeds\" \"\n ipfs config --json Addresses.NoAnnounce '$noAnnounceCfg'\n\"\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"Addresses.NoAnnounce affects addresses from Announce and AppendAnnounce\" '\n ipfs swarm addrs local >actual &&\n test_should_not_contain \"/ip4/1.2.3.4/tcp/1234\" actual &&\n test_should_not_contain \"/ip4/10.20.30.40/tcp/4321\" actual &&\n ipfs id -f\"\" | xargs -n1 echo >actual &&\n test_should_not_contain \"/ip4/1.2.3.4/tcp/1234\" actual &&\n test_should_not_contain \"/ip4/10.20.30.40/tcp/4321\" actual\n'\n\ntest_kill_ipfs_daemon\n\nnoAnnounceCfg='[\"/ip4/1.2.3.4/ipcidr/16\"]'\ntest_expect_success \"test_config_set succeeds\" \"\n ipfs config --json Addresses.NoAnnounce '$noAnnounceCfg'\n\"\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"Addresses.NoAnnounce with /ipcidr affects addresses\" '\n ipfs swarm addrs local >actual &&\n test_should_not_contain \"/ip4/1.2.3.4/tcp/1234\" actual &&\n ipfs id -f\"\" | xargs -n1 echo >actual &&\n test_should_not_contain \"/ip4/1.2.3.4/tcp/1234\" actual\n'\n\ntest_kill_ipfs_daemon\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"'ipfs swarm peering ls' lists peerings\" '\n ipfs swarm peering ls\n'\n\npeeringID='QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N'\npeeringID2='QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5K'\npeeringAddr='/ip4/1.2.3.4/tcp/1234/p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N'\npeeringAddr2='/ip4/1.2.3.4/tcp/1234/p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5K'\ntest_expect_success \"'ipfs swarm peering add' adds a peering\" '\n ipfs swarm peering ls > peeringls &&\n ! test_should_contain ${peeringID} peeringls &&\n ! test_should_contain ${peeringID2} peeringls &&\n ipfs swarm peering add ${peeringAddr} ${peeringAddr2}\n'\n\ntest_expect_success 'a peering is added' '\n ipfs swarm peering ls > peeringadd &&\n test_should_contain ${peeringID} peeringadd &&\n test_should_contain ${peeringID2} peeringadd\n'\n\ntest_expect_success \"'swarm peering rm' removes a peering\" '\n ipfs swarm peering rm ${peeringID}\n'\n\ntest_expect_success 'peering is removed' '\n ipfs swarm peering ls > peeringrm &&\n ! test_should_contain ${peeringID} peeringrm\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"set up tcp testbed\" '\n iptb testbed create -type localipfs -count 2 -force -init\n'\n\nstartup_cluster 2\n\ntest_expect_success \"disconnect work without specifying a transport address\" '\n [ $(ipfsi 0 swarm peers | wc -l) -eq 1 ] &&\n ipfsi 0 swarm disconnect \"/p2p/$(iptb attr get 1 id)\" &&\n [ $(ipfsi 0 swarm peers | wc -l) -eq 0 ]\n'\n\ntest_expect_success \"connect work without specifying a transport address\" '\n [ $(ipfsi 0 swarm peers | wc -l) -eq 0 ] &&\n ipfsi 0 swarm connect \"/p2p/$(iptb attr get 1 id)\" &&\n [ $(ipfsi 0 swarm peers | wc -l) -eq 1 ]\n'\n\ntest_expect_success \"/p2p addresses work\" '\n [ $(ipfsi 0 swarm peers | wc -l) -eq 1 ] &&\n ipfsi 0 swarm disconnect \"/p2p/$(iptb attr get 1 id)\" &&\n [ $(ipfsi 0 swarm peers | wc -l) -eq 0 ] &&\n ipfsi 0 swarm connect \"/p2p/$(iptb attr get 1 id)\" &&\n [ $(ipfsi 0 swarm peers | wc -l) -eq 1 ]\n'\n\ntest_expect_success \"ipfs id is consistent for node 0\" '\n ipfsi 1 id \"$(iptb attr get 0 id)\" > 1see0 &&\n ipfsi 0 id > 0see0 &&\n test_cmp 1see0 0see0\n'\n\ntest_expect_success \"ipfs id is consistent for node 1\" '\n ipfsi 0 id \"$(iptb attr get 1 id)\" > 0see1 &&\n ipfsi 1 id > 1see1 &&\n test_cmp 0see1 1see1\n'\n\ntest_expect_success \"addresses contain /p2p/...\" '\n test_should_contain \"/p2p/$(iptb attr get 1 id)\\\"\" 0see1 &&\n test_should_contain \"/p2p/$(iptb attr get 1 id)\\\"\" 1see1 &&\n test_should_contain \"/p2p/$(iptb attr get 0 id)\\\"\" 1see0 &&\n test_should_contain \"/p2p/$(iptb attr get 0 id)\\\"\" 0see0\n'\n\ntest_expect_success \"stopping cluster\" '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0141-addfilter.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs swarm command\"\n\nAF1=\"/ip4/192.168.0.0/ipcidr/16\"\nAF2=\"/ip4/127.0.0.0/ipcidr/8\"\nAF3=\"/ip6/2008:bcd::/ipcidr/32\"\nAF4=\"/ip4/172.16.0.0/ipcidr/12\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_swarm_filter_cmd() {\n printf \"\" > list_expected\n for AF in \"$@\"\n do\n echo \"$AF\" >>list_expected\n done\n\n test_expect_success \"'ipfs swarm filters' succeeds\" '\n ipfs swarm filters > list_actual\n '\n\n test_expect_success \"'ipfs swarm filters' output looks good\" '\n test_sort_cmp list_expected list_actual\n '\n}\n\ntest_config_swarm_addrfilters_cmd() {\n printf \"\" > list_expected\n for AF in \"$@\"\n do\n echo \"$AF\" >>list_expected\n done\n\n test_expect_success \"'ipfs config Swarm.AddrFilters' succeeds\" '\n ipfs config Swarm.AddrFilters > list_actual\n '\n\n printf \"\" > list_actual_cleaned\n if [ \"$( cat list_actual )\" != \"[]\" -a \"$( cat list_actual )\" != \"null\" ];\n then\n grep -v \"^\\]\" list_actual |\n grep -v \"^\\[\" |\n tr -d '\" ,' > list_actual_cleaned\n fi\n\n test_expect_success \"'ipfs config Swarm.AddrFilters' output looks good\" '\n test_sort_cmp list_expected list_actual_cleaned\n '\n}\n\ntest_swarm_filters() {\n\n # expect first address from config\n test_swarm_filter_cmd $AF1 $AF4\n\n test_config_swarm_addrfilters_cmd $AF1 $AF4\n\n ipfs swarm filters rm all\n\n test_swarm_filter_cmd\n\n test_config_swarm_addrfilters_cmd\n\n test_expect_success \"'ipfs swarm filter add' succeeds\" '\n ipfs swarm filters add $AF1 $AF2 $AF3\n '\n\n test_swarm_filter_cmd $AF1 $AF2 $AF3\n\n test_config_swarm_addrfilters_cmd $AF1 $AF2 $AF3\n\n test_expect_success \"'ipfs swarm filter rm' succeeds\" '\n ipfs swarm filters rm $AF2 $AF3\n '\n\n test_swarm_filter_cmd $AF1\n\n test_config_swarm_addrfilters_cmd $AF1\n\n test_expect_success \"'ipfs swarm filter add' succeeds\" '\n ipfs swarm filters add $AF4 $AF2\n '\n\n test_swarm_filter_cmd $AF1 $AF2 $AF4\n\n test_config_swarm_addrfilters_cmd $AF1 $AF2 $AF4\n\n test_expect_success \"'ipfs swarm filter rm' succeeds\" '\n ipfs swarm filters rm $AF1 $AF2 $AF4\n '\n\n test_swarm_filter_cmd\n\n test_config_swarm_addrfilters_cmd\n}\n\ntest_expect_success \"init without any filters\" '\n echo \"null\" >expected &&\n ipfs config Swarm.AddrFilters >actual &&\n test_cmp expected actual\n'\n\ntest_expect_success \"adding addresses to the config to filter succeeds\" '\n ipfs config --json Swarm.AddrFilters \"[\\\"$AF1\\\", \\\"$AF4\\\"]\"\n'\n\ntest_launch_ipfs_daemon\n\ntest_swarm_filters\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0142-testfilter.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2020 Protocol Labs\n# MIT/Apache-2.0 Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test swarm filters are effective\"\n\nAF=\"/ip4/127.0.0.0/ipcidr/24\"\n\n. lib/test-lib.sh\n\nNUM_NODES=3\n\ntest_expect_success \"set up testbed\" '\n iptb testbed create -type localipfs -count $NUM_NODES -force -init &&\n iptb run -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true\n'\n\ntest_expect_success 'filter 127.0.0.0/24 on node 1' '\n ipfsi 1 config --json Swarm.AddrFilters \"[\\\"$AF\\\"]\"\n'\n\nfor i in $(seq 0 $(( NUM_NODES - 1 ))); do\n test_expect_success \"change IP for node $i\" '\n ipfsi $i config --json \"Addresses.Swarm\" \\\n \"[\\\"/ip4/127.0.$i.1/tcp/0\\\",\\\"/ip4/127.0.$i.1/udp/0/quic\\\",\\\"/ip4/127.0.$i.1/tcp/0/ws\\\"]\"\n '\ndone\n\ntest_expect_success 'start cluster' '\n iptb start --wait\n'\n\ntest_expect_success 'connecting 1 to 0 fails' '\n test_must_fail iptb connect 1 0\n'\n\ntest_expect_success 'connecting 0 to 1 fails' '\n test_must_fail iptb connect 1 0\n'\n\ntest_expect_success 'connecting 2 to 0 succeeds' '\n iptb connect 2 0\n'\n\ntest_expect_success 'connecting 1 to 0 with dns addrs fails' '\n ipfsi 0 id -f \"\" | sed \"s|^/ip4/127.0.0.1/|/dns4/localhost/|\" > addrs &&\n test_must_fail ipfsi 1 swarm connect $(cat addrs)\n'\n\n\ntest_expect_success 'stopping cluster' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0150-clisuggest.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test ipfs cli cmd suggest\"\n\n. lib/test-lib.sh\n\ntest_suggest() {\n\n\n test_expect_success \"test command fails\" '\n test_must_fail ipfs kog 2>actual\n '\n\n test_expect_success \"test one command is suggested\" '\n grep \"Did you mean this?\" actual &&\n grep \"log\" actual ||\n test_fsh cat actual\n '\n\n test_expect_success \"test command fails\" '\n test_must_fail ipfs li 2>actual\n '\n\n test_expect_success \"test multiple commands are suggested\" '\n grep \"Did you mean any of these?\" actual &&\n grep \"ls\" actual &&\n grep \"log\" actual ||\n test_fsh cat actual\n '\n\n}\n\ntest_init_ipfs\n\ntest_suggest\n\ntest_launch_ipfs_daemon\n\ntest_suggest\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0151-sysdiag.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test output of sysdiag command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"ipfs diag sys succeeds\" '\n ipfs diag sys > output\n'\n\ntest_expect_success \"output contains some expected keys\" '\n grep \"virt\" output &&\n grep \"interface_addresses\" output &&\n grep \"arch\" output &&\n grep \"online\" output\n'\n\ntest_expect_success \"uname succeeds\" '\n UOUT=$(uname)\n'\n\ntest_expect_success \"output is similar to uname\" '\n case $UOUT in\n Linux)\n grep linux output > /dev/null\n ;;\n Darwin)\n grep darwin output > /dev/null\n ;;\n FreeBSD)\n grep freebsd output > /dev/null\n ;;\n CYGWIN*)\n grep windows output > /dev/null\n ;;\n *)\n test_fsh echo system check for $UOUT failed, unsupported system?\n ;;\n esac\n'\n\ntest_done\n" }, { "path": "test/sharness/t0152-profile.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test profile collection\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"profiling requires a running daemon\" '\n test_must_fail ipfs diag profile\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"test profiling (without sampling)\" '\n ipfs diag profile --profile-time=0 > cmd_out\n'\n\ntest_expect_success \"filename shows up in output\" '\n grep -q \"ipfs-profile\" cmd_out > /dev/null\n'\n\ntest_expect_success \"profile file created\" '\n test -e \"$(sed -n -e \"s/.*\\(ipfs-profile.*\\.zip\\)/\\1/p\" cmd_out)\"\n'\n\ntest_expect_success \"test profiling with -o\" '\n ipfs diag profile --profile-time=1s -o test-profile.zip\n'\n\ntest_expect_success \"test that test-profile.zip exists\" '\n test -e test-profile.zip\n'\n\ntest_expect_success \"test profiling with specific collectors\" '\n ipfs diag profile --collectors version,goroutines-stack -o test-profile-small.zip\n'\n\ntest_kill_ipfs_daemon\n\nif ! test_have_prereq UNZIP; then\n test_done\nfi\n\ntest_expect_success \"unpack profiles\" '\n unzip -d profiles test-profile.zip &&\n unzip -d profiles-small test-profile-small.zip\n'\n\ntest_expect_success \"cpu profile is valid\" '\n go tool pprof -top profiles/ipfs \"profiles/cpu.pprof\" | grep -q \"Type: cpu\"\n'\n\ntest_expect_success \"heap profile is valid\" '\n go tool pprof -top profiles/ipfs \"profiles/heap.pprof\" | grep -q \"Type: inuse_space\"\n'\n\ntest_expect_success \"goroutines profile is valid\" '\n go tool pprof -top profiles/ipfs \"profiles/goroutines.pprof\" | grep -q \"Type: goroutine\"\n'\n\ntest_expect_success \"mutex profile is valid\" '\n go tool pprof -top profiles/ipfs \"profiles/mutex.pprof\" | grep -q \"Type: delay\"\n'\n\ntest_expect_success \"block profile is valid\" '\n go tool pprof -top profiles/ipfs \"profiles/block.pprof\" | grep -q \"Type: delay\"\n'\n\ntest_expect_success \"goroutines stacktrace is valid\" '\n grep -q \"goroutine\" \"profiles/goroutines.stacks\"\n'\n\ntest_expect_success \"the small profile only contains the requested data\" '\n find profiles-small -type f | sort > actual &&\n echo -e \"profiles-small/goroutines.stacks\\nprofiles-small/version.json\" > expected &&\n test_cmp expected actual\n'\n\ntest_done\n" }, { "path": "test/sharness/t0160-resolve.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test resolve command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"resolve: prepare files\" '\n mkdir -p a/b &&\n echo \"a/b/c\" >a/b/c &&\n a_hash=$(ipfs add -Q -r a) &&\n b_hash=$(ipfs add -Q -r a/b) &&\n c_hash=$(ipfs add -Q -r a/b/c) &&\n a_hash_b32=$(cid-fmt -v 1 -b b %s $a_hash) &&\n b_hash_b32=$(cid-fmt -v 1 -b b %s $b_hash) &&\n c_hash_b32=$(cid-fmt -v 1 -b b %s $c_hash)\n'\n\ntest_expect_success \"resolve: prepare dag\" '\n dag_hash=$(ipfs dag put <<<\"{\\\"i\\\": {\\\"j\\\": {\\\"k\\\": \\\"asdfasdfasdf\\\"}}}\")\n'\n\ntest_expect_success \"resolve: prepare keys\" '\n self_hash=$(ipfs key list --ipns-base=base36 -l | grep self | cut -d \" \" -f1) &&\n alt_hash=$(ipfs key gen --ipns-base=base36 -t rsa alt)\n'\n\ntest_resolve_setup_name() {\n local key=\"$1\"\n local ref=\"$2\"\n\n # we pass here --ttl=0s to ensure that it does not get cached by namesys.\n # the alternative would be to wait between tests to ensure that the namesys\n # cache gets purged in time, but that adds runtime time for the tests.\n test_expect_success \"resolve: prepare $key\" '\n ipfs name publish --key=\"$key\" --ttl=0s --allow-offline \"$ref\"\n '\n}\n\ntest_resolve() {\n src=$1\n dst=$2\n extra=$3\n\n test_expect_success \"resolve succeeds: $src\" '\n ipfs resolve $extra \"$src\" >actual\n '\n\n test_expect_success \"resolved correctly: $src -> $dst\" '\n printf \"$dst\\n\" >expected &&\n test_cmp expected actual\n '\n}\n\ntest_resolve_cmd() {\n echo '-- starting test_resolve_cmd'\n test_resolve \"/ipfs/$a_hash\" \"/ipfs/$a_hash\"\n test_resolve \"/ipfs/$a_hash/b\" \"/ipfs/$b_hash\"\n test_resolve \"/ipfs/$a_hash/b/c\" \"/ipfs/$c_hash\"\n test_resolve \"/ipfs/$b_hash/c\" \"/ipfs/$c_hash\"\n test_resolve \"/ipld/$dag_hash/i/j/k\" \"/ipld/$dag_hash/i/j/k\"\n test_resolve \"/ipld/$dag_hash/i/j\" \"/ipld/$dag_hash/i/j\"\n test_resolve \"/ipld/$dag_hash/i\" \"/ipld/$dag_hash/i\"\n\n test_resolve_setup_name \"self\" \"/ipfs/$a_hash\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$a_hash\"\n test_resolve \"/ipns/$self_hash/b\" \"/ipfs/$b_hash\"\n test_resolve \"/ipns/$self_hash/b/c\" \"/ipfs/$c_hash\"\n\n test_resolve_setup_name \"self\" \"/ipfs/$b_hash\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$b_hash\"\n test_resolve \"/ipns/$self_hash/c\" \"/ipfs/$c_hash\"\n\n test_resolve_setup_name \"self\" \"/ipfs/$c_hash\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$c_hash\"\n\n # simple recursion succeeds\n test_resolve_setup_name \"alt\" \"/ipns/$self_hash\"\n test_resolve \"/ipns/$alt_hash\" \"/ipfs/$c_hash\"\n\n # partial resolve succeeds\n test_resolve \"/ipns/$alt_hash\" \"/ipns/$self_hash\" -r=false\n\n # infinite recursion fails\n test_resolve_setup_name \"self\" \"/ipns/$self_hash\"\n test_expect_success \"recursive resolve terminates\" '\n test_expect_code 1 ipfs resolve /ipns/$self_hash 2>recursion_error &&\n grep \"recursion limit exceeded\" recursion_error\n '\n}\n\ntest_resolve_cmd_b32() {\n echo '-- starting test_resolve_cmd_b32'\n # no flags needed, base should be preserved\n\n test_resolve \"/ipfs/$a_hash_b32\" \"/ipfs/$a_hash_b32\"\n test_resolve \"/ipfs/$a_hash_b32/b\" \"/ipfs/$b_hash_b32\"\n test_resolve \"/ipfs/$a_hash_b32/b/c\" \"/ipfs/$c_hash_b32\"\n test_resolve \"/ipfs/$b_hash_b32/c\" \"/ipfs/$c_hash_b32\"\n\n # flags needed passed in path does not contain cid to derive base\n\n test_resolve_setup_name \"self\" \"/ipfs/$a_hash_b32\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$a_hash_b32\" --cid-base=base32\n test_resolve \"/ipns/$self_hash/b\" \"/ipfs/$b_hash_b32\" --cid-base=base32\n test_resolve \"/ipns/$self_hash/b/c\" \"/ipfs/$c_hash_b32\" --cid-base=base32\n\n test_resolve_setup_name \"self\" \"/ipfs/$b_hash_b32\" --cid-base=base32\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$b_hash_b32\" --cid-base=base32\n test_resolve \"/ipns/$self_hash/c\" \"/ipfs/$c_hash_b32\" --cid-base=base32\n\n test_resolve_setup_name \"self\" \"/ipfs/$c_hash_b32\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$c_hash_b32\" --cid-base=base32\n\n # peer ID represented as CIDv1 require libp2p-key multicodec\n # https://github.com/libp2p/specs/blob/master/RFC/0001-text-peerid-cid.md\n local self_hash_b32protobuf=$(echo $self_hash | ipfs cid format -v 1 -b b --mc dag-pb)\n local self_hash_b32libp2pkey=$(echo $self_hash | ipfs cid format -v 1 -b b --mc libp2p-key)\n test_expect_success \"resolve of /ipns/{cidv1} with multicodec other than libp2p-key returns a meaningful error\" '\n test_expect_code 1 ipfs resolve /ipns/$self_hash_b32protobuf 2>cidcodec_error &&\n test_should_contain \"Error: peer ID represented as CIDv1 require libp2p-key multicodec: retry with /ipns/$self_hash_b32libp2pkey\" cidcodec_error\n '\n}\n\ntest_resolve_cmd_success() {\n test_resolve \"/ipfs/$a_hash\" \"/ipfs/$a_hash\"\n test_resolve \"/ipfs/$a_hash/b\" \"/ipfs/$b_hash\"\n test_resolve \"/ipfs/$a_hash/b/c\" \"/ipfs/$c_hash\"\n test_resolve \"/ipfs/$b_hash/c\" \"/ipfs/$c_hash\"\n test_resolve \"/ipld/$dag_hash\" \"/ipld/$dag_hash\"\n test_resolve \"/ipld/$dag_hash/i/j/k\" \"/ipld/$dag_hash/i/j/k\"\n test_resolve \"/ipld/$dag_hash/i/j\" \"/ipld/$dag_hash/i/j\"\n test_resolve \"/ipld/$dag_hash/i\" \"/ipld/$dag_hash/i\"\n\n test_resolve_setup_name \"self\" \"/ipfs/$a_hash\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$a_hash\"\n test_resolve \"/ipns/$self_hash/b\" \"/ipfs/$b_hash\"\n test_resolve \"/ipns/$self_hash/b/c\" \"/ipfs/$c_hash\"\n\n test_resolve_setup_name \"self\" \"/ipfs/$b_hash\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$b_hash\"\n test_resolve \"/ipns/$self_hash/c\" \"/ipfs/$c_hash\"\n\n test_resolve_setup_name \"self\" \"/ipfs/$c_hash\"\n test_resolve \"/ipns/$self_hash\" \"/ipfs/$c_hash\"\n}\n\n# should work offline\ntest_resolve_cmd\ntest_resolve_cmd_b32\n\n# should work online\ntest_launch_ipfs_daemon\ntest_resolve_cmd_success\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0165-keystore-data/README.md", "content": "# OpenSSL generated keys for import/export tests\n\nCreated with commands:\n\n```bash\nopenssl genpkey -algorithm ED25519 > openssl_ed25519.pem\nopenssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 > openssl_rsa.pem\n```\n\nsecp key used in the 'restrict import key' test.\nFrom: https://docs.openssl.org/1.1.1/man1/genpkey/\n```bash\nopenssl genpkey -genparam -algorithm EC -out ecp.pem \\\n -pkeyopt ec_paramgen_curve:secp384r1 \\\n -pkeyopt ec_param_enc:named_curve\nopenssl genpkey -paramfile ecp.pem -out openssl_secp384r1.pem\nrm ecp.pem\n```\nNote: The Bitcoin `secp256k1` curve which is what `go-libp2p-core/crypto`\nactually generates and would be of interest to test against is not\nrecognized by the Go library:\n```\nError: parsing PKCS8 format: x509: failed to parse EC private key embedded\n in PKCS#8: x509: unknown elliptic curve\n```\nWe keep the `secp384r1` type instead from the original openssl example.\n" }, { "path": "test/sharness/t0165-keystore-data/openssl_ed25519.pem", "content": "-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VwBCIEIJ2M1na2f3dRm4b1FcAQvsn7q08+XfBZcr4MgH4yiBdz\n-----END PRIVATE KEY-----\n" }, { "path": "test/sharness/t0165-keystore-data/openssl_rsa.pem", "content": "-----BEGIN PRIVATE KEY-----\nMIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDSaJB9EKnShOs6\nsbGkB40crn72yNKXj5OBPS2wBDTHWwxyhTB0qJirOT2QYW2DmR/4lPfVk5/f4CJ7\nxIHUBJRoC+NTwqHit24DQBd00tNG4EnKn2Dad/arZ/nEVshkKiGXn0qXxiHHsaCn\nX/pnVPU4+O7fdfUlz2EKf3Og/ocRCFrdMsULR2QwDc0YWsY8ngrcKegyFCbKjXjo\nzvfbGevCDPlhKaZLxRy0PHnON00YC4KO6d77XpbECFvsE1aG1RxYQX0Zjr+i8UvD\nUJp/YCoRNEX54/wKpGebMUrFse5K9hBsFen/wCsPnOsYPSb9g8qyoYRDBnr9sIe1\n9MxFTMy/AgMBAAECggEAKXu2KQI1CS1tlzfbdySJ/MKmg49afckv4sYmENLzeO6J\niLabtBRdbTyu151t0wlIlWEBb9lYJvJwuggnNJ7mh5D4c9YmxqU1imyDc2PxhcLI\nqas8lDYcqvSn+L7HaYAo+VTNhxjoJg/uRbGVk/PbGS1zIxmFiLvXPROdv3sPNBsf\nEYMDH9q7/8DI6dNBQPxtTKlTDLDsTezbkNFQ74znlXgQYcfY1mXljcRtbJqhQJT3\nuppktESPwLRmqtT9H+v9nCtQR6OLmAmLWNgMrSdGKBsSsgJwv2xfpNMffwd84dtT\nuGrS2K+BY0TH2q+Xx04r18GLCst3U5MBSklyHQ/mwQKBgQDqnxNOnK41/n/Q8X4a\n/TUnZBx/JHiCoQoa06AsMxFgOvV3ycR+Z9lwb5I5BsicH1GUcHIxSY3mCyd4fLwE\nFC0QIyNhPJ5oFKh0Oynjm+79VE8v7kK2qqRL4zUpaCXEsSOrhRsCY0/WQdMUPVsh\nokXDUIv37G9KUcjdrhNVpGK3oQKBgQDllK7augIhmlQZTdSLTgmuzhYsXdSGDML/\nBx48q7OvPhvZIIOsygLGhtcBk2xG6PN1yP44cx9dvcTnzxU6TEblO5P8TWY0BSNj\nZuC5wdxLwc3KUdLd9JLR7qcbjqndDruE01rQFVQ3MDbyB1+VrJgiVHIEomJJrKGm\nFQ+314moXwKBgQDL90sDlnZk/kED1k15DRN+kSus5HnXpkRwmfWvNx4t+FOZtdCa\ny5Fei8Akz17rStbTIwZDDtzLVnsT5exV52xdkQ6a4+YaOYtQsHZ0JwWXOgo1cv6Q\nary2NGns+1uKKS0HWYnng4rOix8Dg2uMS9Q2PfnQqLz/cSYcgc7RLz2awQKBgQDd\nHSaLYztKQeldtahPwwlwYuzYLkbSFNh559EnfffBgIAxzy8C7E1gB95sliBi61oQ\nx1SR6c776hoLaVd4np5picgt6B3XXFuJETy/rAcQr8gUZFpDi5sctk4cLHtNfTL9\n6tI8N061GKrS0GcvMNwVtF9cN0mSy8GkxAQvfFgI4QKBgQC4NVimIPptfFckulAL\n/t0vkdLhCRr1+UFNhgsQJhCZpfWZK4x8If6Jru/eiU7ywEsL6fHE2ENvyoTjV33g\nb9yJ7SV4zkz4VhBxc3p26SIvBgLqtHwH8IkIonlbfQFoEAg1iOneLvimPy0YGHsG\n+bTwwlAJJhctILkFtAbooeAQVQ==\n-----END PRIVATE KEY-----\n" }, { "path": "test/sharness/t0165-keystore-data/openssl_secp384r1.pem", "content": "-----BEGIN PRIVATE KEY-----\nMIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDC8DksLZwPKGQS8tuWI\nw+dNYiSHUyw30NkrK9YGmjgp84sVVa5NGrv0QniAnNWG1DqhZANiAATq0d5KV1MF\nIIpF4beNX+YsmFdqB2oDLhznO/4xNsFCFKE39oGQmuMwnQhDNZZ2CQA8csfgZmuF\nOSooe/Ru6ubyeGVKafcHJrOMBvl8hIg0tVWgIAhuXiTHq0UL0QTv9Vk=\n-----END PRIVATE KEY-----\n" }, { "path": "test/sharness/t0165-keystore.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test keystore commands\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_key_cmd() {\n# test key output format\ntest_expect_success \"create an RSA key and test B58MH/B36CID output formats\" '\nPEERID=$(ipfs key gen --ipns-base=b58mh --type=rsa --size=2048 key_rsa) &&\ntest_check_rsa2048_b58mh_peerid $PEERID &&\nipfs key rm key_rsa &&\nPEERID=$(ipfs key gen --ipns-base=base36 --type=rsa --size=2048 key_rsa) &&\ntest_check_rsa2048_base36_peerid $PEERID\n'\n\ntest_expect_success \"test RSA key sk export format\" '\nipfs key export key_rsa &&\ntest_check_rsa2048_sk key_rsa.key &&\nrm key_rsa.key\n'\n\ntest_expect_success \"test RSA key B58MH/B36CID multihash format\" '\nPEERID=$(ipfs key list --ipns-base=b58mh -l | grep key_rsa | head -n 1 | cut -d \" \" -f1) &&\ntest_check_rsa2048_b58mh_peerid $PEERID &&\nPEERID=$(ipfs key list --ipns-base=base36 -l | grep key_rsa | head -n 1 | cut -d \" \" -f1) &&\ntest_check_rsa2048_base36_peerid $PEERID &&\nipfs key rm key_rsa\n'\n\ntest_expect_success \"create an ED25519 key and test B58MH/B36CID output formats\" '\nPEERID=$(ipfs key gen --ipns-base=b58mh --type=ed25519 key_ed25519) &&\ntest_check_ed25519_b58mh_peerid $PEERID &&\nipfs key rm key_ed25519 &&\nPEERID=$(ipfs key gen --ipns-base=base36 --type=ed25519 key_ed25519) &&\ntest_check_ed25519_base36_peerid $PEERID\n'\n\ntest_expect_success \"test ED25519 key sk export format\" '\nipfs key export key_ed25519 &&\ntest_check_ed25519_sk key_ed25519.key &&\nrm key_ed25519.key\n'\n\ntest_expect_success \"test ED25519 key B58MH/B36CID multihash format\" '\nPEERID=$(ipfs key list --ipns-base=b58mh -l | grep key_ed25519 | head -n 1 | cut -d \" \" -f1) &&\ntest_check_ed25519_b58mh_peerid $PEERID &&\nPEERID=$(ipfs key list --ipns-base=base36 -l | grep key_ed25519 | head -n 1 | cut -d \" \" -f1) &&\ntest_check_ed25519_base36_peerid $PEERID &&\nipfs key rm key_ed25519\n'\n# end of format test\n\n\n test_expect_success \"create a new rsa key\" '\n rsahash=$(ipfs key gen generated_rsa_key --type=rsa --size=2048)\n echo $rsahash > rsa_key_id\n '\n\n test_key_import_export_all_formats rsa_key\n\n test_expect_success \"create a new ed25519 key\" '\n edhash=$(ipfs key gen generated_ed25519_key --type=ed25519)\n echo $edhash > ed25519_key_id\n '\n\n test_key_import_export_all_formats ed25519_key\n\n test_openssl_compatibility_all_types\n\n INVALID_KEY=../t0165-keystore-data/openssl_secp384r1.pem\n test_expect_success \"import key type we don't generate fails\" '\n test_must_fail ipfs key import restricted-type -f pem-pkcs8-cleartext $INVALID_KEY 2>&1 | tee key_exp_out &&\n grep -q \"Error: key type \\*crypto.ECDSAPrivateKey is not allowed to be imported\" key_exp_out &&\n rm key_exp_out\n '\n\n test_expect_success \"import key type we don't generate succeeds with flag\" '\n ipfs key import restricted-type --allow-any-key-type -f pem-pkcs8-cleartext $INVALID_KEY > /dev/null &&\n ipfs key rm restricted-type\n '\n\n test_expect_success \"test export file option\" '\n ipfs key export generated_rsa_key -o=named_rsa_export_file &&\n test_cmp generated_rsa_key.key named_rsa_export_file &&\n ipfs key export generated_ed25519_key -o=named_ed25519_export_file &&\n test_cmp generated_ed25519_key.key named_ed25519_export_file\n '\n \n test_expect_success \"key export can't export self\" '\n test_must_fail ipfs key export self 2>&1 | tee key_exp_out &&\n grep -q \"Error: cannot export key with name\" key_exp_out &&\n test_must_fail ipfs key export self -o=selfexport 2>&1 | tee key_exp_out &&\n grep -q \"Error: cannot export key with name\" key_exp_out\n '\n\n test_expect_success \"key import can't import self\" '\n ipfs key gen overwrite_self_import &&\n ipfs key export overwrite_self_import &&\n test_must_fail ipfs key import self overwrite_self_import.key 2>&1 | tee key_imp_out &&\n grep -q \"Error: cannot import key with name\" key_imp_out &&\n ipfs key rm overwrite_self_import &&\n rm overwrite_self_import.key\n '\n\n test_expect_success \"add a default key\" '\n ipfs key gen quxel\n '\n\n test_expect_success \"all keys show up in list output\" '\n echo generated_ed25519_key > list_exp &&\n echo generated_rsa_key >> list_exp &&\n echo quxel >> list_exp &&\n echo self >> list_exp\n ipfs key list > list_out &&\n test_sort_cmp list_exp list_out\n '\n\n test_expect_success \"key hashes show up in long list output\" '\n ipfs key list -l | grep $edhash > /dev/null &&\n ipfs key list -l | grep $rsahash > /dev/null\n '\n\n test_expect_success \"key list -l contains self key with peerID\" '\n PeerID=\"$(ipfs config Identity.PeerID)\"\n ipfs key list -l --ipns-base=b58mh | grep \"$PeerID\\s\\+self\"\n '\n\n test_expect_success \"key rm remove a key\" '\n ipfs key rm generated_rsa_key\n echo generated_ed25519_key > list_exp &&\n echo quxel >> list_exp &&\n echo self >> list_exp\n ipfs key list > list_out &&\n test_sort_cmp list_exp list_out\n '\n\n test_expect_success \"key rm can't remove self\" '\n test_must_fail ipfs key rm self 2>&1 | tee key_rm_out &&\n grep -q \"Error: cannot remove key with name\" key_rm_out\n '\n\n test_expect_success \"key rename rename a key\" '\n ipfs key rename generated_ed25519_key fooed\n echo fooed > list_exp &&\n echo quxel >> list_exp &&\n echo self >> list_exp\n ipfs key list > list_out &&\n test_sort_cmp list_exp list_out\n '\n\n test_expect_success \"key rename rename key output succeeds\" '\n key_content=$(ipfs key gen key1 --type=rsa --size=2048) &&\n ipfs key rename key1 key2 >rs &&\n echo \"Key $key_content renamed to key2\" >expect &&\n test_cmp rs expect\n '\n\n test_expect_success \"key rename can't rename self\" '\n test_must_fail ipfs key rename self bar 2>&1 | tee key_rename_out &&\n grep -q \"Error: cannot rename key with name\" key_rename_out\n '\n\n test_expect_success \"key rename can't overwrite self, even with force\" '\n test_must_fail ipfs key rename -f fooed self 2>&1 | tee key_rename_out &&\n grep -q \"Error: cannot overwrite key with name\" key_rename_out\n '\n\n test_launch_ipfs_daemon\n\n test_expect_success \"online import rsa key\" '\n ipfs key import generated_rsa_key generated_rsa_key.key > roundtrip_rsa_key_id &&\n test_cmp rsa_key_id roundtrip_rsa_key_id\n '\n\n # export works directly on the keystore present in IPFS_PATH\n test_expect_success \"prepare ed25519 key while daemon is running\" '\n edhash=$(ipfs key gen generated_ed25519_key --type=ed25519)\n echo $edhash > ed25519_key_id\n '\n\n test_key_import_export_all_formats ed25519_key\n\n test_openssl_compatibility_all_types\n\n test_expect_success \"key export over HTTP /api/v0/key/export is not possible\" '\n ipfs key gen nohttpexporttest_key --type=ed25519 &&\n curl -X POST -sI \"http://$API_ADDR/api/v0/key/export&arg=nohttpexporttest_key\" | grep -q \"^HTTP/1.1 404 Not Found\"\n '\n\n test_expect_success \"online rotate rsa key\" '\n test_must_fail ipfs key rotate\n '\n\n test_kill_ipfs_daemon\n\n}\n\ntest_check_rsa2048_sk() {\n sklen=$(ls -l $1 | awk '{print $5}') &&\n test \"$sklen\" -lt \"1600\" && test \"$sklen\" -gt \"1000\" || {\n echo \"Bad RSA2048 sk '$1' with len '$sklen'\"\n return 1\n }\n}\n\ntest_check_ed25519_sk() {\n sklen=$(ls -l $1 | awk '{print $5}') &&\n test \"$sklen\" -lt \"100\" && test \"$sklen\" -gt \"30\" || {\n echo \"Bad ED25519 sk '$1' with len '$sklen'\"\n return 1\n }\n}\n\ntest_key_import_export_all_formats() {\n KEY_NAME=$1\n test_key_import_export $KEY_NAME pem-pkcs8-cleartext\n test_key_import_export $KEY_NAME libp2p-protobuf-cleartext\n}\n\ntest_key_import_export() {\n local KEY_NAME FORMAT\n KEY_NAME=$1\n FORMAT=$2\n ORIG_KEY=\"generated_$KEY_NAME\"\n if [ $FORMAT == \"pem-pkcs8-cleartext\" ]; then\n FILE_EXT=\"pem\"\n else\n FILE_EXT=\"key\"\n fi\n\n test_expect_success \"export and import $KEY_NAME with format $FORMAT\" '\n ipfs key export $ORIG_KEY --format=$FORMAT &&\n ipfs key rm $ORIG_KEY &&\n ipfs key import $ORIG_KEY $ORIG_KEY.$FILE_EXT --format=$FORMAT > imported_key_id &&\n test_cmp ${KEY_NAME}_id imported_key_id\n '\n}\n\n# Test the entire import/export cycle with a openssl-generated key.\n# 1. Import openssl key with PEM format.\n# 2. Export key with libp2p format.\n# 3. Reimport key.\n# 4. Now exported with PEM format.\n# 5. Compare with original openssl key.\n# 6. Clean up.\ntest_openssl_compatibility() {\n local KEY_NAME FORMAT\n KEY_NAME=$1\n\n test_expect_success \"import and export $KEY_NAME with all formats\" '\n ipfs key import test-openssl -f pem-pkcs8-cleartext $KEY_NAME > /dev/null &&\n ipfs key export test-openssl -f libp2p-protobuf-cleartext -o $KEY_NAME.libp2p.key &&\n ipfs key rm test-openssl &&\n\n ipfs key import test-openssl -f libp2p-protobuf-cleartext $KEY_NAME.libp2p.key > /dev/null &&\n ipfs key export test-openssl -f pem-pkcs8-cleartext -o $KEY_NAME.ipfs-exported.pem &&\n ipfs key rm test-openssl &&\n\n test_cmp $KEY_NAME $KEY_NAME.ipfs-exported.pem &&\n\n rm $KEY_NAME.libp2p.key &&\n rm $KEY_NAME.ipfs-exported.pem\n '\n}\n\ntest_openssl_compatibility_all_types() {\n test_openssl_compatibility ../t0165-keystore-data/openssl_ed25519.pem\n test_openssl_compatibility ../t0165-keystore-data/openssl_rsa.pem\n}\n\n\ntest_key_cmd\n\ntest_done\n" }, { "path": "test/sharness/t0180-p2p.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test experimental p2p commands\"\n\n. lib/test-lib.sh\n\n# start iptb + wait for peering\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs --count 3 --init\n'\n\ntest_expect_success 'generate test data' '\n echo \"ABCDEF\" > test0.bin &&\n echo \"012345\" > test1.bin\n'\n\nstartup_cluster 3\n\ntest_expect_success 'peer ids' '\n PEERID_0=$(iptb attr get 0 id) &&\n PEERID_1=$(iptb attr get 1 id)\n'\ncheck_test_ports() {\n test_expect_success \"test ports are closed\" '\n (! (netstat -aln | grep \"LISTEN\" | grep -E \"[.:]10101 \")) &&\n (! (netstat -aln | grep \"LISTEN\" | grep -E \"[.:]10102 \")) &&\n (! (netstat -aln | grep \"LISTEN\" | grep -E \"[.:]10103 \")) &&\n (! (netstat -aln | grep \"LISTEN\" | grep -E \"[.:]10104 \"))\n '\n}\ncheck_test_ports\n\ntest_expect_success 'fail without config option being enabled' '\n test_must_fail ipfsi 0 p2p stream ls\n'\n\ntest_expect_success \"enable filestore config setting\" '\n ipfsi 0 config --json Experimental.Libp2pStreamMounting true\n ipfsi 1 config --json Experimental.Libp2pStreamMounting true\n ipfsi 2 config --json Experimental.Libp2pStreamMounting true\n'\n\ntest_expect_success 'start p2p listener' '\n ipfsi 0 p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101 2>&1 > listener-stdouterr.log\n'\n\ntest_expect_success 'cannot re-register p2p listener' '\n test_must_fail ipfsi 0 p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10103 2>&1 > listener-stdouterr.log\n'\n\n# Server to client communications\n\nspawn_sending_server() {\n test_expect_success 'S->C Spawn sending server' '\n ma-pipe-unidir --listen --pidFile=listener.pid send /ip4/127.0.0.1/tcp/10101 < test0.bin &\n\n test_wait_for_file 30 100ms listener.pid &&\n kill -0 $(cat listener.pid)\n '\n}\n\ntest_server_to_client() {\n test_expect_success 'S->C Connect and receive data' '\n ma-pipe-unidir recv /ip4/127.0.0.1/tcp/10102 > client.out\n '\n\n test_expect_success 'S->C Ensure server finished' '\n test ! -f listener.pid\n '\n\n test_expect_success 'S->C Output looks good' '\n test_cmp client.out test0.bin\n '\n}\n\nspawn_sending_server\n\ntest_expect_success 'S->C(/p2p/peerID) Setup client side' '\n ipfsi 1 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /p2p/${PEERID_0} 2>&1 > dialer-stdouterr.log\n'\n\ntest_expect_success 'S->C Setup(dnsaddr/addr/p2p/peerID) client side' '\n ipfsi 1 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10103 /dnsaddr/bootstrap.libp2p.io/p2p/${PEERID_0} 2>&1 > dialer-stdouterr.log\n'\n\ntest_expect_success 'S->C Setup(dnsaddr/addr) client side' '\n ipfsi 1 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10104 /dnsaddr/example-dnsaddr.multiformats.io 2>&1 > dialer-stdouterr.log\n'\n\n\ntest_expect_success 'S->C Output is empty' '\n test_must_be_empty dialer-stdouterr.log\n'\n\ntest_expect_success \"'ipfs p2p ls | grep' succeeds\" '\n ipfsi 1 p2p ls | grep \"/x/p2p-test /ip4/127.0.0.1/tcp/10104\"\n'\n\ntest_server_to_client\n\ntest_expect_success 'S->C Connect with dead server' '\n ma-pipe-unidir recv /ip4/127.0.0.1/tcp/10102 > client.out\n'\n\ntest_expect_success 'S->C Output is empty' '\n test_must_be_empty client.out\n'\n\nspawn_sending_server\n\ntest_server_to_client\n\ntest_expect_success 'S->C Close local listener' '\n ipfsi 1 p2p close -p /x/p2p-test\n'\n\ncheck_test_ports\n\n# Client to server communications\n\ntest_expect_success 'C->S Spawn receiving server' '\n ma-pipe-unidir --listen --pidFile=listener.pid recv /ip4/127.0.0.1/tcp/10101 > server.out &\n\n test_wait_for_file 30 100ms listener.pid &&\n kill -0 $(cat listener.pid)\n'\n\ntest_expect_success 'C->S Setup client side' '\n ipfsi 1 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /p2p/${PEERID_0} 2>&1 > dialer-stdouterr.log\n'\n\ntest_expect_success 'C->S Connect and receive data' '\n ma-pipe-unidir send /ip4/127.0.0.1/tcp/10102 < test1.bin\n'\n\ntest_expect_success 'C->S Ensure server finished' '\n go-sleep 250ms &&\n test ! -f listener.pid\n'\n\ntest_expect_success 'C->S Output looks good' '\n test_cmp server.out test1.bin\n'\n\ntest_expect_success 'C->S Close local listener' '\n ipfsi 1 p2p close -p /x/p2p-test\n'\n\ncheck_test_ports\n\n# Checking port\n\ntest_expect_success \"cannot accept 0 port in 'ipfs p2p listen'\" '\n test_must_fail ipfsi 2 p2p listen /x/p2p-test/0 /ip4/127.0.0.1/tcp/0\n'\n\ntest_expect_success \"'ipfs p2p forward' accept 0 port\" '\n ipfsi 2 p2p forward /x/p2p-test/0 /ip4/127.0.0.1/tcp/0 /p2p/$PEERID_0\n'\n\ntest_expect_success \"'ipfs p2p ls' output looks good\" '\n echo \"true\" > forward_0_expected &&\n ipfsi 2 p2p ls | awk '\\''{print $2}'\\'' | sed \"s/.*\\///\" | awk -F: '\\''{if($1>0)print\"true\"}'\\'' > forward_0_actual &&\n ipfsi 2 p2p close -p /x/p2p-test/0 &&\n test_cmp forward_0_expected forward_0_actual\n'\n\n# Listing streams\n\ntest_expect_success \"'ipfs p2p ls' succeeds\" '\n echo \"/x/p2p-test /p2p/$PEERID_0 /ip4/127.0.0.1/tcp/10101\" > expected &&\n ipfsi 0 p2p ls > actual\n'\n\ntest_expect_success \"'ipfs p2p ls' output looks good\" '\n test_cmp expected actual\n'\n\ntest_expect_success \"Cannot re-register app handler\" '\n test_must_fail ipfsi 0 p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101\n'\n\ntest_expect_success \"'ipfs p2p stream ls' output is empty\" '\n ipfsi 0 p2p stream ls > actual &&\n test_must_be_empty actual\n'\n\ncheck_test_ports\n\ntest_expect_success \"Setup: Idle stream\" '\n ma-pipe-unidir --listen --pidFile=listener.pid recv /ip4/127.0.0.1/tcp/10101 &\n\n ipfsi 1 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /p2p/$PEERID_0 &&\n ma-pipe-unidir --pidFile=client.pid recv /ip4/127.0.0.1/tcp/10102 &\n\n test_wait_for_file 30 100ms listener.pid &&\n test_wait_for_file 30 100ms client.pid &&\n kill -0 $(cat listener.pid) && kill -0 $(cat client.pid)\n'\n\ntest_expect_success \"'ipfs p2p stream ls' succeeds\" '\n echo \"3 /x/p2p-test /p2p/$PEERID_1 /ip4/127.0.0.1/tcp/10101\" > expected\n ipfsi 0 p2p stream ls > actual\n'\n\ntest_expect_success \"'ipfs p2p stream ls' output looks good\" '\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs p2p stream close' closes stream\" '\n ipfsi 0 p2p stream close 3 &&\n ipfsi 0 p2p stream ls > actual &&\n [ ! -f listener.pid ] && [ ! -f client.pid ] &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"'ipfs p2p close' closes remote handler\" '\n ipfsi 0 p2p close -p /x/p2p-test &&\n ipfsi 0 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"'ipfs p2p close' closes local handler\" '\n ipfsi 1 p2p close -p /x/p2p-test &&\n ipfsi 1 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ncheck_test_ports\n\ntest_expect_success \"Setup: Idle stream(2)\" '\n ma-pipe-unidir --listen --pidFile=listener.pid recv /ip4/127.0.0.1/tcp/10101 &\n\n ipfsi 0 p2p listen /x/p2p-test2 /ip4/127.0.0.1/tcp/10101 2>&1 > listener-stdouterr.log &&\n ipfsi 1 p2p forward /x/p2p-test2 /ip4/127.0.0.1/tcp/10102 /p2p/$PEERID_0 2>&1 > dialer-stdouterr.log &&\n ma-pipe-unidir --pidFile=client.pid recv /ip4/127.0.0.1/tcp/10102 &\n\n test_wait_for_file 30 100ms listener.pid &&\n test_wait_for_file 30 100ms client.pid &&\n kill -0 $(cat listener.pid) && kill -0 $(cat client.pid)\n'\n\ntest_expect_success \"'ipfs p2p stream ls' succeeds(2)\" '\n echo \"4 /x/p2p-test2 /p2p/$PEERID_1 /ip4/127.0.0.1/tcp/10101\" > expected\n ipfsi 0 p2p stream ls > actual\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs p2p close -a' closes remote app handlers\" '\n ipfsi 0 p2p close -a &&\n ipfsi 0 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"'ipfs p2p close -a' closes local app handlers\" '\n ipfsi 1 p2p close -a &&\n ipfsi 1 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"'ipfs p2p stream close -a' closes streams\" '\n ipfsi 0 p2p stream close -a &&\n ipfsi 0 p2p stream ls > actual &&\n [ ! -f listener.pid ] && [ ! -f client.pid ] &&\n test_must_be_empty actual\n'\n\ncheck_test_ports\n\ntest_expect_success \"'ipfs p2p close' closes app numeric handlers\" '\n ipfsi 0 p2p listen /x/1234 /ip4/127.0.0.1/tcp/10101 &&\n ipfsi 0 p2p close -p /x/1234 &&\n ipfsi 0 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"'ipfs p2p close' closes by target addr\" '\n ipfsi 0 p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101 &&\n ipfsi 0 p2p close -t /ip4/127.0.0.1/tcp/10101 &&\n ipfsi 0 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"'ipfs p2p close' closes right listeners\" '\n ipfsi 0 p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101 &&\n ipfsi 0 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10101 /p2p/$PEERID_1 &&\n echo \"/x/p2p-test /p2p/$PEERID_0 /ip4/127.0.0.1/tcp/10101\" > expected &&\n\n ipfsi 0 p2p close -l /ip4/127.0.0.1/tcp/10101 &&\n ipfsi 0 p2p ls > actual &&\n test_cmp expected actual\n'\n\ncheck_test_ports\n\ntest_expect_success \"'ipfs p2p close' closes by listen addr\" '\n ipfsi 0 p2p close -l /p2p/$PEERID_0 &&\n ipfsi 0 p2p ls > actual &&\n test_must_be_empty actual\n'\n\n# Peer reporting\n\ntest_expect_success 'start p2p listener reporting peer' '\n ipfsi 0 p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101 --report-peer-id 2>&1 > listener-stdouterr.log\n'\n\ntest_expect_success 'C->S Spawn receiving server' '\n ma-pipe-unidir --listen --pidFile=listener.pid recv /ip4/127.0.0.1/tcp/10101 > server.out &\n\n test_wait_for_file 30 100ms listener.pid &&\n kill -0 $(cat listener.pid)\n'\n\ntest_expect_success 'C->S Setup client side' '\n ipfsi 1 p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /p2p/${PEERID_0} 2>&1 > dialer-stdouterr.log\n'\n\ntest_expect_success 'C->S Connect and receive data' '\n ma-pipe-unidir send /ip4/127.0.0.1/tcp/10102 < test1.bin\n'\n\ntest_expect_success 'C->S Ensure server finished' '\n go-sleep 250ms &&\n test ! -f listener.pid\n'\n\ntest_expect_success 'C->S Output looks good' '\n echo ${PEERID_1} > expected &&\n cat test1.bin >> expected &&\n test_cmp server.out expected\n'\n\ntest_expect_success 'C->S Close listeners' '\n ipfsi 1 p2p close -p /x/p2p-test &&\n ipfsi 0 p2p close -p /x/p2p-test &&\n\n ipfsi 0 p2p ls > actual &&\n test_must_be_empty actual &&\n\n ipfsi 1 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ntest_expect_success \"non /x/ scoped protocols are not allowed\" '\n test_must_fail ipfsi 0 p2p listen /its/not/a/x/path /ip4/127.0.0.1/tcp/10101 2> actual &&\n echo \"Error: protocol name must be within '\"'\"'/x/'\"'\"' namespace\" > expected\n test_cmp expected actual\n'\n\ncheck_test_ports\n\ntest_expect_success 'start p2p listener on custom proto' '\n ipfsi 0 p2p listen --allow-custom-protocol /p2p-test /ip4/127.0.0.1/tcp/10101 2>&1 > listener-stdouterr.log &&\n test_must_be_empty listener-stdouterr.log\n'\n\nspawn_sending_server\n\ntest_expect_success 'S->C Setup client side (custom proto)' '\n ipfsi 1 p2p forward --allow-custom-protocol /p2p-test /ip4/127.0.0.1/tcp/10102 /p2p/${PEERID_0} 2>&1 > dialer-stdouterr.log\n'\n\ntest_server_to_client\n\ntest_expect_success 'C->S Close local listener' '\n ipfsi 1 p2p close -p /p2p-test\n ipfsi 1 p2p ls > actual &&\n test_must_be_empty actual\n'\n\ncheck_test_ports\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ncheck_test_ports\n\ntest_done\n\n" }, { "path": "test/sharness/t0181-private-network.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test private network feature\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"disable AutoConf for private network tests\" '\n ipfs config --json AutoConf.Enabled false\n'\n\nexport LIBP2P_FORCE_PNET=1\n\ntest_expect_success \"daemon won't start with force pnet env but with no key\" '\n test_must_fail go-timeout 5 ipfs daemon > stdout 2>&1\n'\n\nunset LIBP2P_FORCE_PNET\n\ntest_expect_success \"daemon output includes info about the reason\" '\n grep \"private network was not configured but is enforced by the environment\" stdout ||\n test_fsh cat stdout\n'\n\npnet_key() {\n echo '/key/swarm/psk/1.0.0/'\n echo '/bin/'\n random-data -size=32\n}\n\npnet_key > \"${IPFS_PATH}/swarm.key\"\n\nLIBP2P_FORCE_PNET=1 test_launch_ipfs_daemon\n\ntest_expect_success \"set up iptb testbed\" '\n iptb testbed create -type localipfs -count 5 -force -init &&\n iptb run -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true &&\n iptb run -- ipfs config --json \"Swarm.Transports.Network.Websocket\" false &&\n iptb run -- ipfs config --json Addresses.Swarm '\"'\"'[\"/ip4/127.0.0.1/tcp/0\"]'\"'\"' &&\n iptb run -- ipfs config --json AutoConf.Enabled false\n'\n\nset_key() {\n node=\"$1\"\n keyfile=\"$2\"\n\n cp \"$keyfile\" \"${IPTB_ROOT}/testbeds/default/${node}/swarm.key\"\n}\n\npnet_key > key1\npnet_key > key2\n\nset_key 1 key1\nset_key 2 key1\n\nset_key 3 key2\nset_key 4 key2\n\nunset LIBP2P_FORCE_PNET\n\ntest_expect_success \"start nodes\" '\n iptb start -wait [0-4]\n'\n\ntest_expect_success \"try connecting node in public network with priv networks\" '\n test_must_fail iptb connect --timeout=2s [1-4] 0\n'\n\ntest_expect_success \"node 0 (public network) swarm is empty\" '\n ipfsi 0 swarm peers &&\n [ $(ipfsi 0 swarm peers | wc -l) -eq 0 ]\n'\n\ntest_expect_success \"try connecting nodes in different private networks\" '\n test_must_fail iptb connect 2 3\n'\n\ntest_expect_success \"node 3 (pnet 2) swarm is empty\" '\n ipfsi 3 swarm peers &&\n [ $(ipfsi 3 swarm peers | wc -l) -eq 0 ]\n'\n\ntest_expect_success \"connect nodes in the same pnet\" '\n iptb connect 1 2 &&\n iptb connect 3 4\n'\n\ntest_expect_success \"nodes 1 and 2 have connected\" '\n ipfsi 2 swarm peers &&\n [ $(ipfsi 2 swarm peers | wc -l) -eq 1 ]\n'\n\ntest_expect_success \"nodes 3 and 4 have connected\" '\n ipfsi 4 swarm peers &&\n [ $(ipfsi 4 swarm peers | wc -l) -eq 1 ]\n'\n\n\nrun_single_file_test() {\n node1=$1\n node2=$2\n\n test_expect_success \"add a file on node$node1\" '\n random-data -size=1000000 > filea &&\n FILEA_HASH=$(ipfsi $node1 add -q filea)\n '\n\n check_file_fetch $node1 $FILEA_HASH filea\n check_file_fetch $node2 $FILEA_HASH filea\n}\n\ncheck_file_fetch() {\n node=\"$1\"\n fhash=\"$2\"\n fname=\"$3\"\n\n test_expect_success \"can fetch file\" '\n ipfsi $node cat $fhash > fetch_out\n '\n\n test_expect_success \"file looks good\" '\n test_cmp $fname fetch_out\n '\n}\n\nrun_single_file_test 1 2\nrun_single_file_test 2 1\n\nrun_single_file_test 3 4\nrun_single_file_test 4 3\n\n\ntest_expect_success \"stop testbed\" '\n iptb stop\n'\n\ntest_kill_ipfs_daemon\n\n# Test that AutoConf with default mainnet URL fails on private networks\ntest_expect_success \"setup test repo with AutoConf enabled and private network\" '\n export IPFS_PATH=\"$(pwd)/.ipfs-autoconf-test\" &&\n ipfs init --profile=test > /dev/null &&\n ipfs config --json AutoConf.Enabled true &&\n pnet_key > \"${IPFS_PATH}/swarm.key\"\n'\n\ntest_expect_success \"daemon fails with AutoConf + private network error\" '\n export IPFS_PATH=\"$(pwd)/.ipfs-autoconf-test\" &&\n test_expect_code 1 ipfs daemon > autoconf_stdout 2> autoconf_stderr\n'\n\ntest_expect_success \"error message mentions AutoConf and private network conflict\" '\n grep \"AutoConf cannot use the default mainnet URL\" autoconf_stderr > /dev/null &&\n grep \"private network.*swarm.key\" autoconf_stderr > /dev/null &&\n grep \"AutoConf.Enabled=false\" autoconf_stderr > /dev/null\n'\n\ntest_done\n" }, { "path": "test/sharness/t0182-circuit-relay.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test circuit relay\"\n\n. lib/test-lib.sh\n\n# start iptb + wait for peering\nNUM_NODES=3\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count $NUM_NODES -init &&\n iptb run -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true\n'\n\n# Network topology: A <-> Relay <-> B\ntest_expect_success 'start up nodes for configuration' '\n iptb start -wait -- --routing=none\n'\n\ntest_expect_success 'peer ids' '\n PEERID_0=$(iptb attr get 0 id) &&\n PEERID_1=$(iptb attr get 1 id) &&\n PEERID_2=$(iptb attr get 2 id)\n'\n\nrelayaddrs=$(ipfsi 1 swarm addrs local | jq --raw-input . | jq --slurp .)\nstaticrelay=$(ipfsi 1 swarm addrs local | sed -e \"s|$|/p2p/$PEERID_1|g\" | jq --raw-input . | jq --slurp .)\n\ntest_expect_success 'configure the relay node as a static relay for node A' '\n ipfsi 0 config Internal.Libp2pForceReachability private &&\n ipfsi 0 config --json Swarm.RelayClient.Enabled true &&\n ipfsi 0 config --json Swarm.RelayClient.StaticRelays \"$staticrelay\"\n'\n\ntest_expect_success 'configure the relay node' '\n ipfsi 1 config Internal.Libp2pForceReachability public &&\n ipfsi 1 config --json Swarm.RelayService.Enabled true &&\n ipfsi 1 config --json Addresses.Swarm \"$relayaddrs\"\n'\n\ntest_expect_success 'configure the node B' '\n ipfsi 2 config Internal.Libp2pForceReachability private &&\n ipfsi 2 config --json Swarm.RelayClient.Enabled true\n'\n\ntest_expect_success 'restart nodes' '\n iptb stop &&\n iptb_wait_stop &&\n iptb start -wait -- --routing=none\n'\n\ntest_expect_success 'connect A <-> Relay' '\n iptb connect 0 1\n'\n\ntest_expect_success 'connect B <-> Relay' '\n iptb connect 2 1\n'\n\ntest_expect_success 'wait until relay is ready to do work' '\n while ! ipfsi 2 swarm connect /p2p/$PEERID_1/p2p-circuit/p2p/$PEERID_0; do\n iptb stop &&\n iptb_wait_stop &&\n iptb start -wait -- --routing=none &&\n iptb connect 0 1 &&\n iptb connect 2 1 &&\n sleep 5\n done\n'\n\ntest_expect_success 'connect A <-Relay-> B' '\n ipfsi 2 swarm connect /p2p/$PEERID_1/p2p-circuit/p2p/$PEERID_0 > peers_out\n'\n\ntest_expect_success 'output looks good' '\n echo \"connect $PEERID_0 success\" > peers_exp &&\n test_cmp peers_exp peers_out\n'\n\ntest_expect_success 'peers for A look good' '\n ipfsi 0 swarm peers > peers_out &&\n test_should_contain \"/p2p/$PEERID_1/p2p-circuit/p2p/$PEERID_2$\" peers_out\n'\n\ntest_expect_success 'peers for B look good' '\n ipfsi 2 swarm peers > peers_out &&\n test_should_contain \"/p2p/$PEERID_1/p2p-circuit/p2p/$PEERID_0$\" peers_out\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0183-namesys-pubsub.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test IPNS pubsub\"\n\n. lib/test-lib.sh\n\n# start iptb + wait for peering\nNUM_NODES=5\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count $NUM_NODES -init\n'\n\nrun_ipnspubsub_tests() {\n\n test_expect_success 'peer ids' '\n PEERID_0_BASE36=$(ipfsi 0 key list --ipns-base=base36 -l | grep self | head -n 1 | cut -d \" \" -f1) &&\n PEERID_0_B58MH=$(ipfsi 0 key list --ipns-base=b58mh -l | grep self | head -n 1 | cut -d \" \" -f1)\n '\n\n test_expect_success 'check namesys pubsub state' '\n echo enabled > expected &&\n ipfsi 0 name pubsub state > state0 &&\n ipfsi 1 name pubsub state > state1 &&\n ipfsi 2 name pubsub state > state2 &&\n test_cmp expected state0 &&\n test_cmp expected state1 &&\n test_cmp expected state2\n '\n\n # These commands are *expected* to fail. We haven't published anything yet.\n test_expect_success 'subscribe nodes to the publisher topic' '\n ipfsi 1 name resolve /ipns/$PEERID_0_BASE36 --timeout=1s;\n ipfsi 2 name resolve /ipns/$PEERID_0_BASE36 --timeout=1s;\n true\n '\n\n test_expect_success 'check subscriptions' '\n echo /ipns/$PEERID_0_BASE36 > expected_base36 &&\n echo /ipns/$PEERID_0_B58MH > expected_b58mh &&\n ipfsi 1 name pubsub subs > subs1 &&\n ipfsi 2 name pubsub subs > subs2 &&\n ipfsi 1 name pubsub subs --ipns-base=b58mh > subs1_b58mh &&\n ipfsi 2 name pubsub subs --ipns-base=b58mh > subs2_b58mh &&\n test_cmp expected_base36 subs1 &&\n test_cmp expected_base36 subs2 &&\n test_cmp expected_b58mh subs1_b58mh &&\n test_cmp expected_b58mh subs2_b58mh\n '\n\n test_expect_success 'add an object on publisher node' '\n echo \"ipns is super fun\" > file &&\n HASH_FILE=$(ipfsi 0 add -q file)\n '\n\n test_expect_success 'publish that object as an ipns entry' '\n ipfsi 0 name publish $HASH_FILE\n '\n\n test_expect_success 'wait for the flood' '\n sleep 1\n '\n\n test_expect_success 'resolve name in subscriber nodes' '\n echo \"/ipfs/$HASH_FILE\" > expected &&\n ipfsi 1 name resolve /ipns/$PEERID_0_BASE36 > name1 &&\n ipfsi 2 name resolve /ipns/$PEERID_0_BASE36 > name2 &&\n test_cmp expected name1 &&\n test_cmp expected name2\n '\n\n test_expect_success 'cancel subscriptions to the publisher topic' '\n ipfsi 1 name pubsub cancel /ipns/$PEERID_0_BASE36 &&\n ipfsi 2 name pubsub cancel /ipns/$PEERID_0_BASE36\n '\n\n test_expect_success 'check subscriptions' '\n rm -f expected && touch expected &&\n ipfsi 1 name pubsub subs > subs1 &&\n ipfsi 2 name pubsub subs > subs2 &&\n test_cmp expected subs1 &&\n test_cmp expected subs2\n '\n\n test_expect_success \"shut down iptb\" '\n iptb stop\n '\n\n}\n\n# Test everything with ipns-pubsub enabled via config\ntest_expect_success 'enable ipns over pubsub' '\n iptb run -- ipfs config --json Ipns.UsePubsub true\n'\n\nstartup_cluster $NUM_NODES\nrun_ipnspubsub_tests\n\n# Test again, this time CLI parameter override the config\ntest_expect_success 'enable ipns over pubsub' '\n iptb run -- ipfs config --json Ipns.UsePubsub false\n'\nstartup_cluster $NUM_NODES --enable-namesys-pubsub\nrun_ipnspubsub_tests\n\n# Confirm negative CLI flag takes precedence over positive config\n\ntest_expect_success 'enable the pubsub-ipns via config' '\n iptb run -- ipfs config --json Ipns.UsePubsub true\n'\nstartup_cluster $NUM_NODES --enable-namesys-pubsub=false\n\ntest_expect_success 'ipns pubsub cmd fails because it was disabled via cli flag' '\n test_expect_code 1 ipfsi 1 name pubsub subs 2> pubsubipns_cmd_out\n'\n\ntest_expect_success \"ipns pubsub cmd produces error\" \"\n echo -e \\\"Error: IPNS pubsub subsystem is not enabled\\nUse 'ipfs name pubsub subs --help' for information about this command\\\" > expected &&\n test_cmp expected pubsubipns_cmd_out\n\"\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0184-http-proxy-over-p2p.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test http proxy over p2p\"\n\n. lib/test-lib.sh\n\nif ! test_have_prereq SOCAT; then\n skip_all=\"skipping '$test_description': socat is not available\"\n test_done\nfi\n\nWEB_SERVE_PORT=5099\nIPFS_GATEWAY_PORT=5199\nSENDER_GATEWAY=\"http://127.0.0.1:$IPFS_GATEWAY_PORT\"\n\nfunction show_logs() {\n\n echo \"*****************\"\n echo \" RECEIVER LOG \"\n echo \"*****************\"\n iptb logs 1\n echo \"*****************\"\n echo \" SENDER LOG \"\n echo \"*****************\"\n iptb logs 0\n echo \"*****************\"\n echo \"REMOTE_SERVER LOG\"\n echo $REMOTE_SERVER_LOG\n echo \"*****************\"\n cat $REMOTE_SERVER_LOG\n}\n\nfunction start_http_server() {\n REMOTE_SERVER_LOG=\"server.log\"\n rm -f $REMOTE_SERVER_LOG\n\n touch response\n socat tcp-listen:$WEB_SERVE_PORT,fork,bind=127.0.0.1,reuseaddr 'SYSTEM:cat response'!!CREATE:$REMOTE_SERVER_LOG &\n REMOTE_SERVER_PID=$!\n\n socat /dev/null tcp:127.0.01:$WEB_SERVE_PORT,retry=10\n return $?\n}\n\nfunction teardown_remote_server() {\n exec 7<&-\n kill $REMOTE_SERVER_PID > /dev/null 2>&1\n wait $REMOTE_SERVER_PID || true\n}\n\nfunction serve_content() {\n local body=$1\n local status_code=${2:-\"200 OK\"}\n local length=$((1 + ${#body}))\n echo -e \"HTTP/1.1 $status_code\\nContent-length: $length\\n\\n$body\" > response\n}\n\nfunction curl_check_response_code() {\n local expected_status_code=$1\n local path_stub=${2:-p2p/$RECEIVER_ID/http/index.txt}\n local status_code=$(curl -s --write-out %{http_code} --output /dev/null $SENDER_GATEWAY/$path_stub)\n\n if [[ \"$status_code\" -ne \"$expected_status_code\" ]];\n then\n echo \"Found status-code \"$status_code\", expected \"$expected_status_code\n return 1\n fi\n\n return 0\n}\n\nfunction curl_send_proxy_request_and_check_response() {\n local expected_status_code=$1\n local expected_content=$2\n\n #\n # make a request to SENDER_IPFS via the proxy endpoint\n #\n CONTENT_PATH=\"retrieved-file\"\n STATUS_CODE=\"$(curl -s -o $CONTENT_PATH --write-out %{http_code} $SENDER_GATEWAY/p2p/$RECEIVER_ID/http/index.txt)\"\n\n #\n # check status code\n #\n if [[ \"$STATUS_CODE\" -ne \"$expected_status_code\" ]];\n then\n echo -e \"Found status-code \"$STATUS_CODE\", expected \"$expected_status_code\n show_logs\n return 1\n fi\n\n #\n # check content\n #\n RESPONSE_CONTENT=\"$(tail -n 1 $CONTENT_PATH)\"\n if [[ \"$RESPONSE_CONTENT\" == \"$expected_content\" ]];\n then\n return 0\n else\n echo -e \"Found response content:\\n'\"$RESPONSE_CONTENT\"'\\nthat differs from expected content:\\n'\"$expected_content\"'\"\n return 1\n fi\n}\n\nfunction curl_send_multipart_form_request() {\n local expected_status_code=$1\n local FILE_PATH=\"uploaded-file\"\n FILE_CONTENT=\"curl will send a multipart-form POST request when sending a file which is handy\"\n echo $FILE_CONTENT > $FILE_PATH\n #\n # send multipart form request\n #\n STATUS_CODE=\"$(curl -o /dev/null -s -F file=@$FILE_PATH --write-out %{http_code} $SENDER_GATEWAY/p2p/$RECEIVER_ID/http/index.txt)\"\n #\n # check status code\n #\n if [[ \"$STATUS_CODE\" -ne \"$expected_status_code\" ]];\n then\n echo -e \"Found status-code \"$STATUS_CODE\", expected \"$expected_status_code\n return 1\n fi\n #\n # check request method\n #\n if ! grep \"POST /index.txt\" $REMOTE_SERVER_LOG > /dev/null;\n then\n echo \"Remote server request method/resource path was incorrect\"\n show_logs\n return 1\n fi\n #\n # check request is multipart-form\n #\n if ! grep \"Content-Type: multipart/form-data;\" $REMOTE_SERVER_LOG > /dev/null;\n then\n echo \"Request content-type was not multipart/form-data\"\n show_logs\n return 1\n fi\n return 0\n}\n\ntest_expect_success 'configure nodes' '\n iptb testbed create -type localipfs -count 2 -force -init &&\n iptb run -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true &&\n ipfsi 0 config --json Experimental.Libp2pStreamMounting true &&\n ipfsi 1 config --json Experimental.Libp2pStreamMounting true &&\n ipfsi 0 config --json Experimental.P2pHttpProxy true &&\n ipfsi 0 config --json Addresses.Gateway \"[\\\"/ip4/127.0.0.1/tcp/$IPFS_GATEWAY_PORT\\\"]\"\n'\n\ntest_expect_success 'configure a subdomain gateway with /p2p/ path whitelisted' \"\n ipfsi 0 config --json Gateway.PublicGateways '{\n \\\"example.com\\\": {\n \\\"UseSubdomains\\\": true,\n \\\"Paths\\\": [\\\"/p2p/\\\"]\n }\n }'\n\"\n\ntest_expect_success 'start and connect nodes' '\n iptb start -wait && iptb connect 0 1\n'\n\ntest_expect_success 'setup p2p listener on the receiver' '\n ipfsi 1 p2p listen --allow-custom-protocol /http /ip4/127.0.0.1/tcp/$WEB_SERVE_PORT &&\n ipfsi 1 p2p listen /x/custom/http /ip4/127.0.0.1/tcp/$WEB_SERVE_PORT\n'\n\ntest_expect_success 'setup environment' '\n RECEIVER_ID=$(ipfsi 1 id -f=\"\" --peerid-base=b58mh)\n RECEIVER_ID_CIDv1=$(ipfsi 1 id -f=\"\" --peerid-base=base36)\n'\n\ntest_expect_success 'handle proxy http request sends bad-gateway when remote server not available ' '\n curl_send_proxy_request_and_check_response 502 \"\"\n'\n\ntest_expect_success 'start http server' '\n start_http_server\n'\n\ntest_expect_success 'handle proxy http request propagates error response from remote' '\n serve_content \"SORRY GUYS, I LOST IT\" \"404 Not Found\" &&\n curl_send_proxy_request_and_check_response 404 \"SORRY GUYS, I LOST IT\"\n'\n\ntest_expect_success 'handle proxy http request ' '\n serve_content \"THE WOODS ARE LOVELY DARK AND DEEP\" &&\n curl_send_proxy_request_and_check_response 200 \"THE WOODS ARE LOVELY DARK AND DEEP\"\n'\n\ntest_expect_success 'handle proxy http request invalid request' '\n curl_check_response_code 400 p2p/DERPDERPDERP\n'\n\ntest_expect_success 'handle proxy http request unknown proxy peer ' '\n UNKNOWN_PEER=\"k51qzi5uqu5dlmbel1sd8rs4emr3bfosk9bm4eb42514r4lakt4oxw3a3fa2tm\" &&\n curl_check_response_code 502 p2p/$UNKNOWN_PEER/http/index.txt\n'\n\ntest_expect_success 'handle proxy http request to invalid proxy peer ' '\n curl_check_response_code 400 p2p/invalid_peer/http/index.txt\n'\n\ntest_expect_success 'handle proxy http request to custom protocol' '\n serve_content \"THE WOODS ARE LOVELY DARK AND DEEP\" &&\n curl_check_response_code 200 p2p/$RECEIVER_ID/x/custom/http/index.txt\n'\n\ntest_expect_success 'handle proxy http request to missing protocol' '\n serve_content \"THE WOODS ARE LOVELY DARK AND DEEP\" &&\n curl_check_response_code 502 p2p/$RECEIVER_ID/x/missing/http/index.txt\n'\n\ntest_expect_success 'handle proxy http request missing the /http' '\n curl_check_response_code 400 p2p/$RECEIVER_ID/x/custom/index.txt\n'\n\ntest_expect_success 'handle multipart/form-data http request' '\n serve_content \"OK\" &&\n curl_send_multipart_form_request 200\n'\n\n# OK: $peerid.p2p.example.com/http/index.txt\ntest_expect_success \"handle http request to a subdomain gateway\" '\n serve_content \"SUBDOMAIN PROVIDES ORIGIN ISOLATION PER RECEIVER_ID\" &&\n curl -H \"Host: $RECEIVER_ID_CIDv1.p2p.example.com\" -sD - $SENDER_GATEWAY/http/index.txt > p2p_response &&\n test_should_contain \"SUBDOMAIN PROVIDES ORIGIN ISOLATION PER RECEIVER_ID\" p2p_response\n'\n\n# FAIL: $peerid.p2p.example.com/p2p/$peerid/http/index.txt\ntest_expect_success \"handle invalid http request to a subdomain gateway\" '\n serve_content \"SUBDOMAIN DOES NOT SUPPORT FULL /p2p/ PATH\" &&\n curl -H \"Host: $RECEIVER_ID_CIDv1.p2p.example.com\" -sD - $SENDER_GATEWAY/p2p/$RECEIVER_ID/http/index.txt > p2p_response &&\n test_should_contain \"400 Bad Request\" p2p_response\n'\n\n# REDIRECT: example.com/p2p/$peerid/http/index.txt → $peerid.p2p.example.com/http/index.txt\ntest_expect_success \"redirect http path request to subdomain gateway\" '\n serve_content \"SUBDOMAIN ROOT REDIRECTS /p2p/ PATH TO SUBDOMAIN\" &&\n curl -H \"Host: example.com\" -sD - $SENDER_GATEWAY/p2p/$RECEIVER_ID/http/index.txt > p2p_response &&\n test_should_contain \"Location: http://$RECEIVER_ID_CIDv1.p2p.example.com/http/index.txt\" p2p_response\n'\n\ntest_expect_success 'stop http server' '\n teardown_remote_server\n'\n\ntest_expect_success 'stop nodes' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0185-autonat.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test autonat\"\n\n. lib/test-lib.sh\n\n# NOTE: This is currently a really dumb test just to make sure this service\n# starts. We need better tests but testing AutoNAT without public IP addresses\n# is tricky.\n\ntest_init_ipfs\n\ntest_expect_success \"enable autonat\" '\n ipfs config AutoNAT.ServiceMode enabled\n'\n\ntest_launch_ipfs_daemon\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"enable autonat\" '\n ipfs config AutoNAT.ServiceMode disabled\n'\n\ntest_launch_ipfs_daemon\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0190-quic-ping.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test ping over QUIC command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# start iptb + wait for peering\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count 2 -init\n'\n\naddr1='\"[\\\"/ip4/127.0.0.1/udp/0/quic-v1\\\"]\"'\naddr2='\"[\\\"/ip4/127.0.0.1/udp/0/quic-v1\\\"]\"'\ntest_expect_success \"add QUIC swarm addresses\" '\n ipfsi 0 config --json Addresses.Swarm '$addr1' &&\n ipfsi 1 config --json Addresses.Swarm '$addr2'\n'\n\nstartup_cluster 2\n\ntest_expect_success 'peer ids' '\n PEERID_0=$(iptb attr get 0 id) &&\n PEERID_1=$(iptb attr get 1 id)\n'\n\ntest_expect_success \"test ping other\" '\n ipfsi 0 ping -n2 -- \"$PEERID_1\" &&\n ipfsi 1 ping -n2 -- \"$PEERID_0\"\n'\n\ntest_expect_success \"test ping self\" '\n test_must_fail ipfsi 0 ping -n2 -- \"$PEERID_0\" &&\n test_must_fail ipfsi 1 ping -n2 -- \"$PEERID_1\"\n'\n\ntest_expect_success \"test ping 0\" '\n test_must_fail ipfsi 0 ping -n0 -- \"$PEERID_1\" &&\n test_must_fail ipfsi 1 ping -n0 -- \"$PEERID_0\"\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0191-webtransport-ping.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test ping over WebTransport command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# start iptb + wait for peering\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count 2 -init\n'\n\naddr1='\"[\\\"/ip4/127.0.0.1/udp/0/quic-v1/webtransport\\\"]\"'\naddr2='\"[\\\"/ip4/127.0.0.1/udp/0/quic-v1/webtransport\\\"]\"'\ntest_expect_success \"add WebTransport swarm addresses\" '\n ipfsi 0 config --json Addresses.Swarm '$addr1' &&\n ipfsi 0 config --json Swarm.Transports.Network.WebTransport true &&\n ipfsi 1 config --json Addresses.Swarm '$addr2' &&\n ipfsi 1 config --json Swarm.Transports.Network.WebTransport true\n'\n\nstartup_cluster 2\n\ntest_expect_success 'peer ids' '\n PEERID_0=$(iptb attr get 0 id) &&\n PEERID_1=$(iptb attr get 1 id)\n'\n\ntest_expect_success \"test ping other\" '\n ipfsi 0 ping -n2 -- \"$PEERID_1\" &&\n ipfsi 1 ping -n2 -- \"$PEERID_0\"\n'\n\ntest_expect_success \"test ping self\" '\n test_must_fail ipfsi 0 ping -n2 -- \"$PEERID_0\" &&\n test_must_fail ipfsi 1 ping -n2 -- \"$PEERID_1\"\n'\n\ntest_expect_success \"test ping 0\" '\n test_must_fail ipfsi 0 ping -n0 -- \"$PEERID_1\" &&\n test_must_fail ipfsi 1 ping -n0 -- \"$PEERID_0\"\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0195-noise.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test ping over NOISE command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# start iptb + wait for peering\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count 4 -init\n'\n\ntcp_addr='\"[\\\"/ip4/127.0.0.1/tcp/0\\\"]\"'\ntest_expect_success \"configure security transports\" '\niptb run < connect_error 2>&1 &&\n test_should_contain \"failed to negotiate security protocol\" connect_error ||\n test_fsh cat connect_error\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0220-bitswap.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test bitswap commands\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\ntest_launch_ipfs_daemon\n\ntest_expect_success \"'ipfs bitswap stat' succeeds\" '\n ipfs bitswap stat >stat_out\n'\n\ntest_expect_success \"'ipfs bitswap stat' output looks good\" '\n cat <expected &&\nbitswap status\n blocks received: 0\n blocks sent: 0\n data received: 0\n data sent: 0\n dup blocks received: 0\n dup data received: 0\n wantlist [0 keys]\n partners [0]\nEOF\n test_cmp expected stat_out\n'\n\ntest_expect_success \"ipfs peer id looks good\" '\n PEERID=$(ipfs config Identity.PeerID) &&\n test_check_peerid \"$PEERID\"\n'\n\ntest_expect_success \"'ipfs bitswap wantlist -p' works\" '\n ipfs bitswap wantlist -p \"$PEERID\" >wantlist_p_out\n'\n\ntest_expect_success \"'ipfs bitswap wantlist -p' output looks good\" '\n test_must_be_empty wantlist_p_out\n'\n\ntest_expect_success \"hash was removed from wantlist\" '\n ipfs bitswap wantlist > wantlist_out &&\n test_must_be_empty wantlist_out\n'\n\ntest_expect_success \"'ipfs bitswap stat' succeeds\" '\n ipfs bitswap stat >stat_out\n'\n\ntest_expect_success \"'ipfs bitswap stat' output looks good\" '\n cat <expected &&\nbitswap status\n blocks received: 0\n blocks sent: 0\n data received: 0\n data sent: 0\n dup blocks received: 0\n dup data received: 0\n wantlist [0 keys]\n partners [0]\nEOF\n test_cmp expected stat_out\n'\n\ntest_expect_success \"'ipfs bitswap wantlist -p' works\" '\n ipfs bitswap wantlist -p \"$PEERID\" >wantlist_p_out\n'\n\ntest_expect_success \"'ipfs bitswap wantlist -p' output looks good\" '\n test_cmp wantlist_out wantlist_p_out\n'\n\ntest_expect_success \"'ipfs bitswap stat --human' succeeds\" '\n ipfs bitswap stat --human >stat_out_human\n'\n\n\ntest_expect_success \"'ipfs bitswap stat --human' output looks good\" '\n cat <expected &&\nbitswap status\n blocks received: 0\n blocks sent: 0\n data received: 0 B\n data sent: 0 B\n dup blocks received: 0\n dup data received: 0 B\n wantlist [0 keys]\n partners [0]\nEOF\n test_cmp expected stat_out_human\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0230-channel-streaming-http-content-type.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Cayman Nava\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test Content-Type for channel-streaming commands\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_ls_cmd() {\n\n test_expect_success \"Text encoded channel-streaming command succeeds\" '\n mkdir -p testdir &&\n echo \"hello test\" >testdir/test.txt &&\n ipfs add -r testdir &&\n curl -X POST -i \"http://$API_ADDR/api/v0/refs?arg=QmTcJAn3JP8ZMAKS6WS75q8sbTyojWKbxcUHgLYGWur4Ym&stream-channels=true&encoding=text\" >actual_output\n '\n\n test_expect_success \"Text encoded channel-streaming command output looks good\" '\n printf \"HTTP/1.1 200 OK\\r\\n\" >expected_output &&\n printf \"Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length\\r\\n\" >>expected_output &&\n printf \"Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length\\r\\n\" >>expected_output &&\n printf \"Content-Type: text/plain\\r\\n\" >>expected_output &&\n printf \"Server: kubo/%s\\r\\n\" $(ipfs version -n) >>expected_output &&\n printf \"Trailer: X-Stream-Error\\r\\n\" >>expected_output &&\n printf \"Vary: Origin\\r\\n\" >>expected_output &&\n printf \"X-Chunked-Output: 1\\r\\n\" >>expected_output &&\n printf \"Transfer-Encoding: chunked\\r\\n\" >>expected_output &&\n printf \"\\r\\n\" >>expected_output &&\n echo QmRmPLc1FsPAn8F8F9DQDEYADNX5ER2sgqiokEvqnYknVW >>expected_output &&\n cat actual_output | grep -vE Date > cleaned_output &&\n test_cmp expected_output cleaned_output\n '\n\n test_expect_success \"JSON encoded channel-streaming command succeeds\" '\n mkdir -p testdir &&\n echo \"hello test\" >testdir/test.txt &&\n ipfs add -r testdir &&\n curl -X POST -i \"http://$API_ADDR/api/v0/refs?arg=QmTcJAn3JP8ZMAKS6WS75q8sbTyojWKbxcUHgLYGWur4Ym&stream-channels=true&encoding=json\" >actual_output\n '\n\n test_expect_success \"JSON encoded channel-streaming command output looks good\" '\n printf \"HTTP/1.1 200 OK\\r\\n\" >expected_output &&\n printf \"Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length\\r\\n\" >>expected_output &&\n printf \"Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length\\r\\n\" >>expected_output &&\n printf \"Content-Type: application/json\\r\\n\" >>expected_output &&\n printf \"Server: kubo/%s\\r\\n\" $(ipfs version -n) >>expected_output &&\n printf \"Trailer: X-Stream-Error\\r\\n\" >>expected_output &&\n printf \"Vary: Origin\\r\\n\" >>expected_output &&\n printf \"X-Chunked-Output: 1\\r\\n\" >>expected_output &&\n printf \"Transfer-Encoding: chunked\\r\\n\" >>expected_output &&\n printf \"\\r\\n\" >>expected_output &&\n cat <<-\\EOF >>expected_output &&\n{\"Ref\":\"QmRmPLc1FsPAn8F8F9DQDEYADNX5ER2sgqiokEvqnYknVW\",\"Err\":\"\"}\nEOF\n printf \"\\n\" >> expected_output &&\n perl -pi -e '\"'\"'chomp if eof'\"'\"' expected_output &&\n cat actual_output | grep -vE Date > cleaned_output &&\n test_cmp expected_output cleaned_output\n '\n}\n\n# should work online (only)\ntest_launch_ipfs_daemon\ntest_ls_cmd\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0231-channel-streaming.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test output of streaming json commands\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nget_api_port() {\n cat \"$IPFS_PATH/api\" | awk -F/ '{ print $5 }'\n}\n\ntest_ls_cmd() {\n test_expect_success \"make a file with multiple refs\" '\n HASH=$(random-data -size=1000000 | ipfs add -q)\n '\n\n test_expect_success \"can get refs through curl\" '\n PORT=$(get_api_port) &&\n curl http://localhost:$PORT/api/v0/refs/$HASH > output\n '\n\n # make sure newlines are printed between each object\n test_expect_success \"output looks good\" '\n test_expect_code 1 grep \"}{\" output > /dev/null\n '\n}\n\n# should work online (only)\ntest_launch_ipfs_daemon\ntest_ls_cmd\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0235-cli-request.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test http requests made by cli\"\n\n. lib/test-lib.sh\n\nif ! test_have_prereq SOCAT; then\n skip_all=\"skipping '$test_description': socat is not available\"\n test_done\nfi\n\n\ntest_init_ipfs\n\ntest_expect_success \"start nc\" '\n rm -f nc_out nc_outp nc_inp && mkfifo nc_inp nc_outp\n\n socat PIPE:nc_inp!!PIPE:nc_outp tcp-listen:5005,fork,max-children=1,bind=127.0.0.1 &\n NCPID=$!\n\n exec 6>nc_inp 7nc_out &&\n\n echo -e \"HTTP/1.1 200 OK\\r\" >&6 &&\n echo -e \"Content-Type: application/json\\r\" >&6 &&\n echo -e \"Content-Length: 21\\r\" >&6 &&\n echo -e \"\\r\" >&6 &&\n echo -e \"{\\\"Version\\\":\\\"0.23.0\\\"}\\r\" >&6 &&\n echo -e \"\\r\" >&6 &&\n\n # handle request for /api/v0/cat\n while read line; do\n if [[ \"$line\" == \"$(echo -e \"\\r\")\" ]]; then\n break\n fi\n echo \"$line\"\n done <&7 >nc_out &&\n\n echo -e \"HTTP/1.1 200 OK\\r\" >&6 &&\n echo -e \"Content-Type: text/plain\\r\" >&6 &&\n echo -e \"Content-Length: 0\\r\" >&6 &&\n echo -e \"\\r\" >&6 &&\n exec 6<&- &&\n\n # Wait for IPFS\n wait $IPFSPID\n'\n\ntest_expect_success \"stop nc\" '\n kill \"$NCPID\" && wait \"$NCPID\" || true\n'\n\ntest_expect_success \"output does not contain multipart info\" '\n test_expect_code 1 grep multipart nc_out\n'\n\ntest_expect_success \"request looks good\" '\n grep \"POST /api/v0/cat\" nc_out\n'\n\ntest_expect_success \"api flag does not appear in request\" '\n test_expect_code 1 grep \"api=/ip4\" nc_out\n'\n\ntest_expect_success \"host has dns name not ip address\" '\n grep \"Host: localhost:5005\" nc_out\n'\n\ntest_done\n" }, { "path": "test/sharness/t0236-cli-api-dns-resolve.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test dns resolution of api endpoint by cli\"\n\n. lib/test-lib.sh\n\nif ! test_have_prereq SOCAT; then\n skip_all=\"skipping '$test_description': socat is not available\"\n test_done\nfi\n\ntest_init_ipfs\n\ntest_expect_success \"start nc\" '\n rm -f nc_out nc_outp nc_inp && mkfifo nc_inp nc_outp\n\n socat PIPE:nc_inp!!PIPE:nc_outp tcp-listen:5006,fork,max-children=1,bind=127.0.0.1 &\n NCPID=$!\n\n exec 6>nc_inp 7nc_out &&\n\n echo -e \"HTTP/1.1 200 OK\\r\" >&6 &&\n echo -e \"Content-Type: application/json\\r\" >&6 &&\n echo -e \"Content-Length: 21\\r\" >&6 &&\n echo -e \"\\r\" >&6 &&\n echo -e \"{\\\"Version\\\":\\\"0.23.0\\\"}\\r\" >&6 &&\n echo -e \"\\r\" >&6 &&\n\n # handle request for /api/v0/cat\n while read line; do\n if [[ \"$line\" == \"$(echo -e \"\\r\")\" ]]; then\n break\n fi\n echo \"$line\"\n done <&7 >nc_out &&\n\n echo -e \"HTTP/1.1 200 OK\\r\" >&6 &&\n echo -e \"Content-Type: text/plain\\r\" >&6 &&\n echo -e \"Content-Length: 0\\r\" >&6 &&\n echo -e \"\\r\" >&6 &&\n exec 6<&- &&\n\n # Wait for IPFS\n wait $IPFSPID\n'\n\ntest_expect_success \"stop nc\" '\n kill \"$NCPID\" && wait \"$NCPID\" || true\n'\n\ntest_expect_success \"request was received by local nc server\" '\n grep \"POST /api/v0/cat\" nc_out\n'\n\ntest_done\n" }, { "path": "test/sharness/t0240-republisher.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test ipfs repo operations\"\n\n. lib/test-lib.sh\n\nexport DEBUG=true\n\nsetup_iptb() {\n num_nodes=\"$1\"\n bound=$(expr \"$num_nodes\" - 1)\n\n test_expect_success \"iptb init\" '\n iptb testbed create -type localipfs -count $num_nodes -init\n '\n\n for i in $(test_seq 0 \"$bound\")\n do\n test_expect_success \"set configs up for node $i\" '\n ipfsi \"$i\" config Ipns.RepublishPeriod 40s &&\n ipfsi \"$i\" config --json Ipns.ResolveCacheSize 0\n '\n done\n\n startup_cluster \"$num_nodes\"\n}\n\nteardown_iptb() {\n test_expect_success \"shut down nodes\" '\n iptb stop\n '\n}\n\nverify_can_resolve() {\n num_nodes=\"$1\"\n bound=$(expr \"$num_nodes\" - 1)\n name=\"$2\"\n expected=\"$3\"\n msg=\"$4\"\n\n for node in $(test_seq 0 \"$bound\")\n do\n test_expect_success \"$msg: node $node can resolve entry\" '\n ipfsi \"$node\" name resolve \"$name\" > resolve\n '\n\n test_expect_success \"$msg: output for node $node looks right\" '\n printf \"/ipfs/$expected\\n\" > expected &&\n test_cmp expected resolve\n '\n done\n}\n\nverify_cannot_resolve() {\n num_nodes=\"$1\"\n bound=$(expr \"$num_nodes\" - 1)\n name=\"$2\"\n msg=\"$3\"\n\n for node in $(test_seq 0 \"$bound\")\n do\n test_expect_success \"$msg: resolution fails on node $node\" '\n test_expect_code 1 ipfsi \"$node\" name resolve \"$name\"\n '\n done\n}\n\nnum_test_nodes=4\n\nsetup_iptb \"$num_test_nodes\"\n\ntest_expect_success \"publish succeeds\" '\n HASH=$(date +\"%FT%T.%N%z\" | ipfsi 1 add -q) &&\n ipfsi 1 name publish -t 10s $HASH\n'\n\ntest_expect_success \"get id succeeds\" '\n id=$(ipfsi 1 id -f \"\")\n'\n\nverify_can_resolve \"$num_test_nodes\" \"$id\" \"$HASH\" \"just after publishing\"\n\ngo-sleep 10s\n\nverify_cannot_resolve \"$num_test_nodes\" \"$id\" \"after 10 seconds, records are invalid\"\n\ngo-sleep 30s\n\nverify_can_resolve \"$num_test_nodes\" \"$id\" \"$HASH\" \"republisher fires after 30 seconds\"\n\n#\n\ntest_expect_success \"generate new key\" '\nKEY2=`ipfsi 1 key gen beepboop --type ed25519`\n'\n\ntest_expect_success \"publish with new key succeeds\" '\n HASH=$(date +\"%FT%T.%N%z\" | ipfsi 1 add -q) &&\n ipfsi 1 name publish -t 10s -k \"$KEY2\" $HASH\n'\n\nverify_can_resolve \"$num_test_nodes\" \"$KEY2\" \"$HASH\" \"new key just after publishing\"\n\ngo-sleep 10s\n\nverify_cannot_resolve \"$num_test_nodes\" \"$KEY2\" \"new key cannot resolve after 10 seconds\"\n\ngo-sleep 30s\n\nverify_can_resolve \"$num_test_nodes\" \"$KEY2\" \"$HASH\" \"new key can resolve again after republish\"\n\n#\n\nteardown_iptb\n\ntest_done\n" }, { "path": "test/sharness/t0250-files-api.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2015 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test the unix files api\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ncreate_files() {\n FILE1=$(echo foo | ipfs add \"$@\" -q) &&\n FILE2=$(echo bar | ipfs add \"$@\" -q) &&\n FILE3=$(echo baz | ipfs add \"$@\" -q) &&\n mkdir -p stuff_test &&\n echo cats > stuff_test/a &&\n echo dogs > stuff_test/b &&\n echo giraffes > stuff_test/c &&\n DIR1=$(ipfs add -r \"$@\" -Q stuff_test)\n}\n\nverify_path_exists() {\n # simply running ls on a file should be a good 'check'\n ipfs files ls $1\n}\n\nverify_dir_contents() {\n dir=$1\n shift\n rm -f expected\n touch expected\n for e in $@\n do\n echo $e >> expected\n done\n\n test_expect_success \"can list dir\" '\n ipfs files ls $dir > output\n '\n\n test_expect_success \"dir entries look good\" '\n test_sort_cmp output expected\n '\n}\n\ntest_sharding() {\n local EXTRA ARGS\n EXTRA=$1\n ARGS=$2 # only applied to the initial directory\n\n test_expect_success \"make a directory $EXTRA\" '\n ipfs files mkdir $ARGS /foo\n '\n\n test_expect_success \"can make 100 files in a directory $EXTRA\" '\n printf \"\" > list_exp_raw\n for i in `seq 100 -1 1`\n do\n echo $i | ipfs files write --create /foo/file$i || return 1\n echo file$i >> list_exp_raw\n done\n '\n # Create the files in reverse (unsorted) order (`seq 100 -1 1`)\n # to check the sort in the `ipfs files ls` command. `ProtoNode`\n # links are always sorted at the DAG layer so the sorting feature\n # is tested with sharded directories.\n\n test_expect_success \"sorted listing works $EXTRA\" '\n ipfs files ls /foo > list_out &&\n sort list_exp_raw > list_exp &&\n test_cmp list_exp list_out\n '\n\n test_expect_success \"unsorted listing works $EXTRA\" '\n ipfs files ls -U /foo > list_out &&\n sort list_exp_raw > sort_list_not_exp &&\n ! test_cmp sort_list_not_exp list_out\n '\n\n test_expect_success \"can read a file from sharded directory $EXTRA\" '\n ipfs files read /foo/file65 > file_out &&\n echo \"65\" > file_exp &&\n test_cmp file_out file_exp\n '\n\n test_expect_success \"can pin a file from sharded directory $EXTRA\" '\n ipfs files stat --hash /foo/file42 > pin_file_hash &&\n ipfs pin add < pin_file_hash > pin_hash\n '\n\n test_expect_success \"can unpin a file from sharded directory $EXTRA\" '\n read -r _ HASH _ < pin_hash &&\n ipfs pin rm $HASH\n '\n\n test_expect_success \"output object was really sharded and has correct hash $EXTRA\" '\n ipfs files stat --hash /foo > expected_foo_hash &&\n echo $SHARD_HASH > actual_foo_hash &&\n test_cmp expected_foo_hash actual_foo_hash\n '\n\n test_expect_success \"clean up $EXTRA\" '\n ipfs files rm -r /foo\n '\n}\n\ntest_files_api() {\n local EXTRA ARGS RAW_LEAVES\n EXTRA=$1\n ARGS=$2\n RAW_LEAVES=$3\n\n test_expect_success \"can mkdir in root $EXTRA\" '\n ipfs files mkdir $ARGS /cats\n '\n\n test_expect_success \"'files ls' lists root by default $EXTRA\" '\n ipfs files ls >actual &&\n echo \"cats\" >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"directory was created $EXTRA\" '\n verify_path_exists /cats\n '\n\n test_expect_success \"directory is empty $EXTRA\" '\n verify_dir_contents /cats\n '\n # we do verification of stat formatting now as we depend on it\n\n test_expect_success \"stat works $EXTRA\" '\n ipfs files stat / >stat\n '\n\n test_expect_success \"hash is first line of stat $EXTRA\" '\n ipfs ls $(head -1 stat) | grep \"cats\"\n '\n\n test_expect_success \"stat --hash gives only hash $EXTRA\" '\n ipfs files stat --hash / >actual &&\n head -n1 stat >expected &&\n test_cmp expected actual\n '\n\n test_expect_success \"stat with multiple format options should fail $EXTRA\" '\n test_must_fail ipfs files stat --hash --size /\n '\n\n test_expect_success \"compare hash option with format $EXTRA\" '\n ipfs files stat --hash / >expected &&\n ipfs files stat --format='\"'\"''\"'\"' / >actual &&\n test_cmp expected actual\n '\n test_expect_success \"compare size option with format $EXTRA\" '\n ipfs files stat --size / >expected &&\n ipfs files stat --format='\"'\"''\"'\"' / >actual &&\n test_cmp expected actual\n '\n\n test_expect_success \"check root hash $EXTRA\" '\n ipfs files stat --hash / > roothash\n '\n\n test_expect_success \"stat works outside of MFS\" '\n ipfs files stat /ipfs/$DIR1\n '\n\n test_expect_success \"stat compute the locality of a dag\" '\n ipfs files stat --with-local /ipfs/$DIR1 > output\n grep -q \"(100.00%)\" output\n '\n\n test_expect_success \"cannot mkdir / $EXTRA\" '\n test_expect_code 1 ipfs files mkdir $ARGS /\n '\n\n test_expect_success \"check root hash was not changed $EXTRA\" '\n ipfs files stat --hash / > roothashafter &&\n test_cmp roothash roothashafter\n '\n\n test_expect_success \"can put files into directory $EXTRA\" '\n ipfs files cp /ipfs/$FILE1 /cats/file1\n '\n\n test_expect_success \"file shows up in directory $EXTRA\" '\n verify_dir_contents /cats file1\n '\n\n test_expect_success \"file has correct hash and size in directory $EXTRA\" '\n echo \"file1\t$FILE1\t4\" > ls_l_expected &&\n ipfs files ls -l /cats > ls_l_actual &&\n test_cmp ls_l_expected ls_l_actual\n '\n\n test_expect_success \"file has correct hash and size listed with -l\" '\n echo \"file1\t$FILE1\t4\" > ls_l_expected &&\n ipfs files ls -l /cats/file1 > ls_l_actual &&\n test_cmp ls_l_expected ls_l_actual\n '\n\n test_expect_success \"file has correct hash and size listed with --long\" '\n echo \"file1\t$FILE1\t4\" > ls_l_expected &&\n ipfs files ls --long /cats/file1 > ls_l_actual &&\n test_cmp ls_l_expected ls_l_actual\n '\n\n test_expect_success \"file has correct hash and size listed with -l --cid-base=base32\" '\n echo \"file1\t`cid-fmt -v 1 -b base32 %s $FILE1`\t4\" > ls_l_expected &&\n ipfs files ls --cid-base=base32 -l /cats/file1 > ls_l_actual &&\n test_cmp ls_l_expected ls_l_actual\n '\n\n test_expect_success \"file shows up with the correct name\" '\n echo \"file1\" > ls_l_expected &&\n ipfs files ls /cats/file1 > ls_l_actual &&\n test_cmp ls_l_expected ls_l_actual\n '\n\n test_expect_success \"can stat file $EXTRA\" '\n ipfs files stat /cats/file1 > file1stat_orig\n '\n\n test_expect_success \"stat output looks good\" '\n grep -v CumulativeSize: file1stat_orig > file1stat_actual &&\n echo \"$FILE1\" > file1stat_expect &&\n echo \"Size: 4\" >> file1stat_expect &&\n echo \"ChildBlocks: 0\" >> file1stat_expect &&\n echo \"Type: file\" >> file1stat_expect &&\n echo \"Mode: not set (not set)\" >> file1stat_expect &&\n echo \"Mtime: not set\" >> file1stat_expect &&\n test_cmp file1stat_expect file1stat_actual\n '\n\n test_expect_success \"can stat file with --cid-base=base32 $EXTRA\" '\n ipfs files stat --cid-base=base32 /cats/file1 > file1stat_orig\n '\n\n test_expect_success \"stat output looks good with --cid-base=base32\" '\n grep -v CumulativeSize: file1stat_orig > file1stat_actual &&\n echo `cid-fmt -v 1 -b base32 %s $FILE1` > file1stat_expect &&\n echo \"Size: 4\" >> file1stat_expect &&\n echo \"ChildBlocks: 0\" >> file1stat_expect &&\n echo \"Type: file\" >> file1stat_expect &&\n echo \"Mode: not set (not set)\" >> file1stat_expect &&\n echo \"Mtime: not set\" >> file1stat_expect &&\n test_cmp file1stat_expect file1stat_actual\n '\n\n test_expect_success \"can read file $EXTRA\" '\n ipfs files read /cats/file1 > file1out\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo foo > expected &&\n test_cmp expected file1out\n '\n\n test_expect_success \"can put another file into root $EXTRA\" '\n ipfs files cp /ipfs/$FILE2 /file2\n '\n\n test_expect_success \"file shows up in root $EXTRA\" '\n verify_dir_contents / file2 cats\n '\n\n test_expect_success \"can read file $EXTRA\" '\n ipfs files read /file2 > file2out\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo bar > expected &&\n test_cmp expected file2out\n '\n\n test_expect_success \"can make deep directory $EXTRA\" '\n ipfs files mkdir $ARGS -p /cats/this/is/a/dir\n '\n\n test_expect_success \"directory was created correctly $EXTRA\" '\n verify_path_exists /cats/this/is/a/dir &&\n verify_dir_contents /cats this file1 &&\n verify_dir_contents /cats/this is &&\n verify_dir_contents /cats/this/is a &&\n verify_dir_contents /cats/this/is/a dir &&\n verify_dir_contents /cats/this/is/a/dir\n '\n\n test_expect_success \"dir has correct name\" '\n DIR_HASH=$(ipfs files stat /cats/this --hash) &&\n echo \"this/\t$DIR_HASH\t0\" > ls_dir_expected &&\n ipfs files ls -l /cats | grep this/ > ls_dir_actual &&\n test_cmp ls_dir_expected ls_dir_actual\n '\n\n test_expect_success \"can copy file into new dir $EXTRA\" '\n ipfs files cp /ipfs/$FILE3 /cats/this/is/a/dir/file3\n '\n\n test_expect_success \"can copy file into deep dir using -p flag $EXTRA\" '\n ipfs files cp -p /ipfs/$FILE3 /cats/some/other/dir/file3\n '\n\n test_expect_success \"file copied into deep dir exists $EXTRA\" '\n ipfs files read /cats/some/other/dir/file3 > file_out &&\n echo \"baz\" > file_exp &&\n test_cmp file_out file_exp\n '\n \n test_expect_success \"cleanup deep cp -p test $EXTRA\" '\n ipfs files rm -r /cats/some\n '\n\n test_expect_success \"can read file $EXTRA\" '\n ipfs files read /cats/this/is/a/dir/file3 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo baz > expected &&\n test_cmp expected output\n '\n\n test_expect_success \"file shows up in dir $EXTRA\" '\n verify_dir_contents /cats/this/is/a/dir file3\n '\n\n test_expect_success \"can remove file $EXTRA\" '\n ipfs files rm /cats/this/is/a/dir/file3\n '\n\n test_expect_success \"file no longer appears $EXTRA\" '\n verify_dir_contents /cats/this/is/a/dir\n '\n\n test_expect_success \"can remove dir $EXTRA\" '\n ipfs files rm -r /cats/this/is/a/dir\n '\n\n test_expect_success \"dir no longer appears $EXTRA\" '\n verify_dir_contents /cats/this/is/a\n '\n\n test_expect_success \"can remove file from root $EXTRA\" '\n ipfs files rm /file2\n '\n\n test_expect_success \"file no longer appears $EXTRA\" '\n verify_dir_contents / cats\n '\n\n test_expect_success \"check root hash $EXTRA\" '\n ipfs files stat --hash / > roothash\n '\n\n test_expect_success \"cannot remove root $EXTRA\" '\n test_expect_code 1 ipfs files rm -r /\n '\n\n test_expect_success \"check root hash was not changed $EXTRA\" '\n ipfs files stat --hash / > roothashafter &&\n test_cmp roothash roothashafter\n '\n\n # test read options\n\n test_expect_success \"read from offset works $EXTRA\" '\n ipfs files read -o 1 /cats/file1 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo oo > expected &&\n test_cmp expected output\n '\n\n test_expect_success \"read with size works $EXTRA\" '\n ipfs files read -n 2 /cats/file1 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n printf fo > expected &&\n test_cmp expected output\n '\n\n test_expect_success \"cannot read from negative offset $EXTRA\" '\n test_expect_code 1 ipfs files read --offset -3 /cats/file1\n '\n\n test_expect_success \"read from offset 0 works $EXTRA\" '\n ipfs files read --offset 0 /cats/file1 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo foo > expected &&\n test_cmp expected output\n '\n\n test_expect_success \"read last byte works $EXTRA\" '\n ipfs files read --offset 2 /cats/file1 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo o > expected &&\n test_cmp expected output\n '\n\n test_expect_success \"offset past end of file fails $EXTRA\" '\n test_expect_code 1 ipfs files read --offset 5 /cats/file1\n '\n\n test_expect_success \"cannot read negative count bytes $EXTRA\" '\n test_expect_code 1 ipfs read --count -1 /cats/file1\n '\n\n test_expect_success \"reading zero bytes prints nothing $EXTRA\" '\n ipfs files read --count 0 /cats/file1 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n printf \"\" > expected &&\n test_cmp expected output\n '\n\n test_expect_success \"count > len(file) prints entire file $EXTRA\" '\n ipfs files read --count 200 /cats/file1 > output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo foo > expected &&\n test_cmp expected output\n '\n\n # test write\n\n test_expect_success \"can write file $EXTRA\" '\n echo \"ipfs rocks\" > tmpfile &&\n cat tmpfile | ipfs files write $ARGS $RAW_LEAVES --create /cats/ipfs\n '\n\n test_expect_success \"file was created $EXTRA\" '\n verify_dir_contents /cats ipfs file1 this\n '\n\n test_expect_success \"can read file we just wrote $EXTRA\" '\n ipfs files read /cats/ipfs > output\n '\n\n test_expect_success \"can write to offset $EXTRA\" '\n echo \"is super cool\" | ipfs files write $ARGS $RAW_LEAVES -o 5 /cats/ipfs\n '\n\n test_expect_success \"file looks correct $EXTRA\" '\n echo \"ipfs is super cool\" > expected &&\n ipfs files read /cats/ipfs > output &&\n test_cmp expected output\n '\n\n test_expect_success \"file hash correct $EXTRA\" '\n echo $FILE_HASH > filehash_expected &&\n ipfs files stat --hash /cats/ipfs > filehash &&\n test_cmp filehash_expected filehash\n '\n\n test_expect_success \"can't write to negative offset $EXTRA\" '\n test_expect_code 1 ipfs files write $ARGS $RAW_LEAVES --offset -1 /cats/ipfs < output\n '\n\n test_expect_success \"verify file was not changed $EXTRA\" '\n ipfs files stat --hash /cats/ipfs > afterhash &&\n test_cmp filehash afterhash\n '\n\n test_expect_success \"write new file for testing $EXTRA\" '\n echo foobar | ipfs files write $ARGS $RAW_LEAVES --create /fun\n '\n\n test_expect_success \"write to offset past end works $EXTRA\" '\n echo blah | ipfs files write $ARGS $RAW_LEAVES --offset 50 /fun\n '\n\n test_expect_success \"can read file $EXTRA\" '\n ipfs files read /fun > sparse_output\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n echo foobar > sparse_expected &&\n echo blah | dd of=sparse_expected bs=50 seek=1 &&\n test_cmp sparse_expected sparse_output\n '\n\n test_expect_success \"cleanup $EXTRA\" '\n ipfs files rm /fun\n '\n\n test_expect_success \"cannot write to directory $EXTRA\" '\n ipfs files stat --hash /cats > dirhash &&\n test_expect_code 1 ipfs files write $ARGS $RAW_LEAVES /cats < output\n '\n\n test_expect_success \"verify dir was not changed $EXTRA\" '\n ipfs files stat --hash /cats > afterdirhash &&\n test_cmp dirhash afterdirhash\n '\n\n test_expect_success \"cannot write to nonexistent path $EXTRA\" '\n test_expect_code 1 ipfs files write $ARGS $RAW_LEAVES /cats/bar/ < output\n '\n\n test_expect_success \"no new paths were created $EXTRA\" '\n verify_dir_contents /cats file1 ipfs this\n '\n\n # Temporary check to uncover source of flaky test fail (see\n # https://github.com/ipfs/go-ipfs/issues/8131 for more details).\n # We suspect that sometimes the daemon isn't running when in fact we need\n # it to for the `--flush=false` flag to take effect. To try to spot the\n # specific error before it manifests itself in the failed test we explicitly\n # poll the damon API when it should be running ($WITH_DAEMON set).\n # Test taken from `test/sharness/lib/test-lib.sh` (but with less retries\n # as the daemon is either running or not but there is no 'bootstrap' time\n # needed in this case).\n test_expect_success \"'ipfs daemon' is running when WITH_DAEMON is set\" '\n test -z \"$WITH_DAEMON\" ||\n pollEndpoint -host=$API_MADDR -v -tout=1s -tries=3 2>poll_apierr > poll_apiout ||\n test_fsh cat actual_daemon || test_fsh cat daemon_err || test_fsh cat poll_apierr || test_fsh cat poll_apiout\n '\n\n test_expect_success \"write 'no-flush' succeeds $EXTRA\" '\n echo \"testing\" | ipfs files write $ARGS $RAW_LEAVES -f=false -e /cats/walrus\n '\n\n # Skip this test if the commands are not being run through the daemon\n # ($WITH_DAEMON not set) as standalone commands will *always* flush\n # after being done and the 'no-flush' call from the previous test will\n # not be enforced.\n test_expect_success \"root hash not bubbled up yet $EXTRA\" '\n test -z \"$WITH_DAEMON\" ||\n (ipfs refs local > refsout &&\n test_expect_code 1 grep $ROOT_HASH refsout)\n '\n\n test_expect_success \"changes bubbled up to root on inspection $EXTRA\" '\n ipfs files stat --hash / > root_hash\n '\n\n test_expect_success \"root hash looks good $EXTRA\" '\n export EXP_ROOT_HASH=\"$ROOT_HASH\" &&\n echo $EXP_ROOT_HASH > root_hash_exp &&\n test_cmp root_hash_exp root_hash\n '\n\n test_expect_success \"/cats hash looks good $EXTRA\" '\n export EXP_CATS_HASH=\"$CATS_HASH\" &&\n echo $EXP_CATS_HASH > cats_hash_exp &&\n ipfs files stat --hash /cats > cats_hash\n test_cmp cats_hash_exp cats_hash\n '\n\n test_expect_success \"flush root succeeds $EXTRA\" '\n ipfs files flush /\n '\n\n # test mv\n test_expect_success \"can mv dir $EXTRA\" '\n ipfs files mv /cats/this/is /cats/\n '\n\n test_expect_success \"can mv dir and dest dir is / $EXTRA\" '\n ipfs files mv /cats/is /\n '\n\n test_expect_success \"can mv dir and dest dir path has no trailing slash $EXTRA\" '\n ipfs files mv /is /cats\n '\n\n test_expect_success \"mv worked $EXTRA\" '\n verify_dir_contents /cats file1 ipfs this is walrus &&\n verify_dir_contents /cats/this\n '\n\n test_expect_success \"cleanup, remove 'cats' $EXTRA\" '\n ipfs files rm -r /cats\n '\n\n test_expect_success \"cleanup looks good $EXTRA\" '\n verify_dir_contents /\n '\n\n # test truncating\n test_expect_success \"create a new file $EXTRA\" '\n echo \"some content\" | ipfs files write $ARGS $RAW_LEAVES --create /cats\n '\n\n test_expect_success \"truncate and write over that file $EXTRA\" '\n echo \"fish\" | ipfs files write $ARGS $RAW_LEAVES --truncate /cats\n '\n\n test_expect_success \"output looks good $EXTRA\" '\n ipfs files read /cats > file_out &&\n echo \"fish\" > file_exp &&\n test_cmp file_out file_exp\n '\n\n test_expect_success \"file hash correct $EXTRA\" '\n echo $TRUNC_HASH > filehash_expected &&\n ipfs files stat --hash /cats > filehash &&\n test_cmp filehash_expected filehash\n '\n\n test_expect_success \"cleanup $EXTRA\" '\n ipfs files rm /cats\n '\n\n # test flush flags\n test_expect_success \"mkdir --flush works $EXTRA\" '\n ipfs files mkdir $ARGS --flush --parents /flushed/deep\n '\n\n test_expect_success \"mkdir --flush works a second time $EXTRA\" '\n ipfs files mkdir $ARGS --flush --parents /flushed/deep\n '\n\n test_expect_success \"dir looks right $EXTRA\" '\n verify_dir_contents / flushed\n '\n\n test_expect_success \"child dir looks right $EXTRA\" '\n verify_dir_contents /flushed deep\n '\n\n test_expect_success \"cleanup $EXTRA\" '\n ipfs files rm -r /flushed\n '\n\n test_expect_success \"child dir looks right $EXTRA\" '\n verify_dir_contents /\n '\n\n # test for https://github.com/ipfs/go-ipfs/issues/2654\n test_expect_success \"create and remove dir $EXTRA\" '\n ipfs files mkdir $ARGS /test_dir &&\n ipfs files rm -r \"/test_dir\"\n '\n\n test_expect_success \"create test file $EXTRA\" '\n echo \"content\" | ipfs files write $ARGS $RAW_LEAVES -e \"/test_file\"\n '\n\n test_expect_success \"copy test file onto test dir $EXTRA\" '\n ipfs files cp \"/test_file\" \"/test_dir\"\n '\n\n test_expect_success \"test /test_dir $EXTRA\" '\n ipfs files stat \"/test_dir\" | grep -q \"^Type: file\"\n '\n\n test_expect_success \"clean up /test_dir and /test_file $EXTRA\" '\n ipfs files rm -r /test_dir &&\n ipfs files rm -r /test_file\n '\n\n test_expect_success \"make a directory and a file $EXTRA\" '\n ipfs files mkdir $ARGS /adir &&\n echo \"blah\" | ipfs files write $ARGS $RAW_LEAVES --create /foobar\n '\n\n test_expect_success \"copy a file into a directory $EXTRA\" '\n ipfs files cp /foobar /adir/\n '\n\n test_expect_success \"file made it into directory $EXTRA\" '\n ipfs files ls /adir | grep foobar\n '\n\n test_expect_success \"test copy --force overwrites files\" '\n ipfs files cp /ipfs/$FILE1 /file1 &&\n ipfs files cp /ipfs/$FILE2 /file2 &&\n ipfs files cp --force /file1 /file2 &&\n test \"`ipfs files read /file1`\" = \"`ipfs files read /file2`\"\n '\n\n test_expect_success \"clean up\" '\n ipfs files rm /file1 &&\n ipfs files rm /file2\n '\n\n test_expect_success \"should fail to write file and create intermediate directories with no --parents flag set $EXTRA\" '\n echo \"ipfs rocks\" | test_must_fail ipfs files write --create /parents/foo/ipfs.txt\n '\n\n test_expect_success \"can write file and create intermediate directories $EXTRA\" '\n echo \"ipfs rocks\" | ipfs files write --create --parents /parents/foo/bar/baz/ipfs.txt &&\n ipfs files stat \"/parents/foo/bar/baz/ipfs.txt\" | grep -q \"^Type: file\"\n '\n\n test_expect_success \"can write file and create intermediate directories with short flags $EXTRA\" '\n echo \"ipfs rocks\" | ipfs files write -e -p /parents/foo/bar/baz/qux/quux/garply/ipfs.txt &&\n ipfs files stat \"/parents/foo/bar/baz/qux/quux/garply/ipfs.txt\" | grep -q \"^Type: file\"\n '\n\n test_expect_success \"can write another file in the same directory with -e -p $EXTRA\" '\n echo \"ipfs rocks\" | ipfs files write -e -p /parents/foo/bar/baz/qux/quux/garply/ipfs2.txt &&\n ipfs files stat \"/parents/foo/bar/baz/qux/quux/garply/ipfs2.txt\" | grep -q \"^Type: file\"\n '\n\n test_expect_success \"clean up $EXTRA\" '\n ipfs files rm -r /foobar /adir /parents\n '\n\n test_expect_success \"root mfs entry is empty $EXTRA\" '\n verify_dir_contents /\n '\n\n test_expect_success \"repo gc $EXTRA\" '\n ipfs repo gc\n '\n\n # test rm\n\n test_expect_success \"remove file forcibly\" '\n echo \"hello world\" | ipfs files write --create /forcibly &&\n ipfs files rm --force /forcibly &&\n verify_dir_contents /\n '\n\n test_expect_success \"remove multiple files forcibly\" '\n echo \"hello world\" | ipfs files write --create /forcibly_one &&\n echo \"hello world\" | ipfs files write --create /forcibly_two &&\n ipfs files rm --force /forcibly_one /forcibly_two &&\n verify_dir_contents /\n '\n\n test_expect_success \"remove directory forcibly\" '\n ipfs files mkdir /forcibly-dir &&\n ipfs files rm --force /forcibly-dir &&\n verify_dir_contents /\n '\n\n test_expect_success \"remove multiple directories forcibly\" '\n ipfs files mkdir /forcibly-dir-one &&\n ipfs files mkdir /forcibly-dir-two &&\n ipfs files rm --force /forcibly-dir-one /forcibly-dir-two &&\n verify_dir_contents /\n '\n\n test_expect_success \"remove multiple files\" '\n echo \"hello world\" | ipfs files write --create /file_one &&\n echo \"hello world\" | ipfs files write --create /file_two &&\n ipfs files rm /file_one /file_two\n '\n\n test_expect_success \"remove multiple directories\" '\n ipfs files mkdir /forcibly-dir-one &&\n ipfs files mkdir /forcibly-dir-two &&\n ipfs files rm -r /forcibly-dir-one /forcibly-dir-two &&\n verify_dir_contents /\n '\n\n test_expect_success \"remove nonexistent path forcibly\" '\n ipfs files rm --force /nonexistent\n '\n\n test_expect_success \"remove deeply nonexistent path forcibly\" '\n ipfs files rm --force /deeply/nonexistent\n '\n\n # This one should return code 1 but still remove the rest of the valid files.\n test_expect_success \"remove multiple files (with nonexistent one)\" '\n echo \"hello world\" | ipfs files write --create /file_one &&\n echo \"hello world\" | ipfs files write --create /file_two &&\n test_expect_code 1 ipfs files rm /file_one /nonexistent /file_two\n verify_dir_contents /\n '\n}\n\n# test with and without the daemon (EXTRA=\"with-daemon\" and EXTRA=\"no-daemon\"\n# respectively).\n# FIXME: Check if we are correctly using the \"no-daemon\" flag in these test\n# combinations.\ntests_for_files_api() {\n local EXTRA\n EXTRA=$1\n\n test_expect_success \"can create some files for testing ($EXTRA)\" '\n create_files\n '\n # default: CIDv0, dag-pb for all files (no raw-leaves)\n ROOT_HASH=QmcwKfTMCT7AaeiD92hWjnZn9b6eh9NxnhfSzN5x2vnDpt\n CATS_HASH=Qma88m8ErTGkZHbBWGqy1C7VmEmX8wwNDWNpGyCaNmEgwC\n FILE_HASH=QmQdQt9qooenjeaNhiKHF3hBvmNteB4MQBtgu3jxgf9c7i\n TRUNC_HASH=QmPVnT9gocPbqzN4G6SMp8vAPyzcjDbUJrNdKgzQquuDg4\n test_files_api \"($EXTRA)\"\n\n test_expect_success \"can create some files for testing with raw-leaves ($EXTRA)\" '\n create_files --raw-leaves\n '\n\n # partial raw-leaves: initial files created with --raw-leaves, test ops without\n if [ \"$EXTRA\" = \"with-daemon\" ]; then\n ROOT_HASH=QmTpKiKcAj4sbeesN6vrs5w3QeVmd4QmGpxRL81hHut4dZ\n CATS_HASH=QmPhPkmtUGGi8ySPHoPu1qbfryLJKKq1GYxpgLyyCruvGe\n test_files_api \"($EXTRA, partial raw-leaves)\"\n fi\n\n # raw-leaves: single-block files become RawNode (CIDv1), dirs stay CIDv0\n ROOT_HASH=QmTHzLiSouBHVTssS8xRzmfWGAvTGhPEjtPdB6pWMQdxJX\n CATS_HASH=QmPJkzbCoBuL379TbHgwF1YbVHnKgiDa5bjqYhe6Lovdms\n FILE_HASH=bafybeibkrazpbejqh3qun7xfnsl7yofl74o4jwhxebpmtrcpavebokuqtm\n TRUNC_HASH=bafybeigwhb3q36yrm37jv5fo2ap6r6eyohckqrxmlejrenex4xlnuxiy3e\n test_files_api \"($EXTRA, raw-leaves)\" '' --raw-leaves\n\n # cidv1 for mkdir: different from raw-leaves since mkdir forces CIDv1 dirs\n ROOT_HASH=QmTLdTaZNj8Mvq1cgYup59ZFJFv1KxptouFSZUZKeq7X3z\n CATS_HASH=bafybeihsqinttigpskqqj63wgalrny3lifvqv5ml7igrirdhlcf73l3wvm\n FILE_HASH=bafybeibkrazpbejqh3qun7xfnsl7yofl74o4jwhxebpmtrcpavebokuqtm\n TRUNC_HASH=bafybeigwhb3q36yrm37jv5fo2ap6r6eyohckqrxmlejrenex4xlnuxiy3e\n if [ \"$EXTRA\" = \"with-daemon\" ]; then\n test_files_api \"($EXTRA, cidv1)\" --cid-version=1\n fi\n\n test_expect_success \"can update root hash to cidv1\" '\n ipfs files chcid --cid-version=1 / &&\n echo bafybeiczsscdsbs7ffqz55asqdf3smv6klcw3gofszvwlyarci47bgf354 > hash_expect &&\n ipfs files stat --hash / > hash_actual &&\n test_cmp hash_expect hash_actual\n '\n\n # cidv1 root: root upgraded to CIDv1 via chcid, all new dirs/files also CIDv1\n ROOT_HASH=bafybeickjecu37qv6ue54ofk3n4rpm4g4abuofz7yc4qn4skffy263kkou\n CATS_HASH=bafybeihsqinttigpskqqj63wgalrny3lifvqv5ml7igrirdhlcf73l3wvm\n test_files_api \"($EXTRA, cidv1 root)\"\n\n if [ \"$EXTRA\" = \"with-daemon\" ]; then\n test_expect_success \"can update root hash to blake2b-256\" '\n ipfs files chcid --hash=blake2b-256 / &&\n echo bafykbzacebugfutjir6qie7apo5shpry32ruwfi762uytd5g3u2gk7tpscndq > hash_expect &&\n ipfs files stat --hash / > hash_actual &&\n test_cmp hash_expect hash_actual\n '\n # blake2b-256 root: using blake2b-256 hash instead of sha2-256\n ROOT_HASH=bafykbzaceaebvwrjdw5rfhqqh5miaq3g42yybnrw3kxxxx43ggyttm6xn2zek\n CATS_HASH=bafykbzaceaqvpxs3dfl7su6744jgyvifbusow2tfixdy646chasdwyz2boagc\n FILE_HASH=bafykbzaceca45w2i3o3q3ctqsezdv5koakz7sxsw37ygqjg4w54m2bshzevxy\n TRUNC_HASH=bafykbzaceadeu7onzmlq7v33ytjpmo37rsqk2q6mzeqf5at55j32zxbcdbwig\n test_files_api \"($EXTRA, blake2b-256 root)\"\n fi\n\n test_expect_success \"can update root hash back to cidv0\" '\n ipfs files chcid / --cid-version=0 &&\n echo QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn > hash_expect &&\n ipfs files stat --hash / > hash_actual &&\n test_cmp hash_expect hash_actual\n '\n}\n\ntests_for_files_api \"no-daemon\"\n\ntest_launch_ipfs_daemon_without_network\n\nWITH_DAEMON=1\n# FIXME: Used only on a specific test inside `test_files_api` but we should instead\n# propagate the `\"with-daemon\"` argument in its caller `tests_for_files_api`.\n\ntests_for_files_api \"with-daemon\"\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"enable sharding in config\" '\n ipfs config --json Import.UnixFSHAMTDirectorySizeThreshold \"\\\"1B\\\"\"\n'\n\ntest_launch_ipfs_daemon_without_network\n\n# sharding cidv0: HAMT-sharded directory with 100 files, CIDv0\nSHARD_HASH=QmPkwLJTYZRGPJ8Lazr9qPdrLmswPtUjaDbEpmR9jEh1se\ntest_sharding \"(cidv0)\"\n\n# sharding cidv1: HAMT-sharded directory with 100 files, CIDv1\nSHARD_HASH=bafybeiaulcf7c46pqg3tkud6dsvbgvlnlhjuswcwtfhxts5c2kuvmh5keu\ntest_sharding \"(cidv1 root)\" \"--cid-version=1\"\n\ntest_kill_ipfs_daemon\n\n# Test automatic sharding and unsharding\n\n# We shard based on size with a threshold of 256 KiB (see config file docs)\n# above which directories are sharded.\n#\n# The directory size is estimated as the size of each link. Links are roughly\n# the entry name + the CID byte length (e.g. 34 bytes for a CIDv0). So for\n# entries of length 10 we need 256 KiB / (34 + 10) ~ 6000 entries in the\n# directory to trigger sharding.\ntest_expect_success \"set up automatic sharding/unsharding data\" '\n mkdir big_dir\n for i in `seq 5960` # Just above the number of entries that trigger sharding for 256KiB\n do\n echo $i > big_dir/`printf \"file%06d\" $i` # fixed length of 10 chars\n done\n'\n\ntest_expect_success \"reset automatic sharding\" '\n ipfs config --json Import.UnixFSHAMTDirectorySizeThreshold null\n'\n\ntest_launch_ipfs_daemon_without_network\n\nLARGE_SHARDED=\"QmWfjnRWRvdvYezQWnfbvrvY7JjrpevsE9cato1x76UqGr\"\nLARGE_MINUS_5_UNSHARDED=\"QmbVxi5zDdzytrjdufUejM92JsWj8wGVmukk6tiPce3p1m\"\n\ntest_add_large_sharded_dir() {\n exphash=\"$1\"\n test_expect_success \"ipfs add on directory succeeds\" '\n ipfs add -r -Q big_dir > shardbigdir_out &&\n echo \"$exphash\" > shardbigdir_exp &&\n test_cmp shardbigdir_exp shardbigdir_out\n '\n\n test_expect_success \"can access a path under the dir\" '\n ipfs cat \"$exphash/file000030\" > file30_out &&\n test_cmp big_dir/file000030 file30_out\n '\n}\n\ntest_add_large_sharded_dir \"$LARGE_SHARDED\"\n\ntest_expect_success \"remove a few entries from big_dir/ to trigger unsharding\" '\n ipfs files cp /ipfs/\"$LARGE_SHARDED\" /big_dir &&\n for i in `seq 5`\n do\n ipfs files rm /big_dir/`printf \"file%06d\" $i`\n done &&\n ipfs files stat --hash /big_dir > unshard_dir_hash &&\n echo \"$LARGE_MINUS_5_UNSHARDED\" > unshard_exp &&\n test_cmp unshard_exp unshard_dir_hash\n'\n\ntest_expect_success \"add a few entries to big_dir/ to retrigger sharding\" '\n for i in `seq 5`\n do\n ipfs files cp /ipfs/\"$LARGE_SHARDED\"/`printf \"file%06d\" $i` /big_dir/`printf \"file%06d\" $i`\n done &&\n ipfs files stat --hash /big_dir > shard_dir_hash &&\n echo \"$LARGE_SHARDED\" > shard_exp &&\n test_cmp shard_exp shard_dir_hash\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0251-files-flushing.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test the unix files api flushing\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nverify_path_exists() {\n # simply running ls on a file should be a good 'check'\n ipfs files ls $1\n}\n\nverify_dir_contents() {\n dir=$1\n shift\n rm -f expected\n touch expected\n for e in $@\n do\n echo $e >> expected\n done\n\n test_expect_success \"can list dir\" '\n ipfs files ls $dir > output\n '\n\n test_expect_success \"dir entries look good\" '\n test_sort_cmp output expected\n '\n}\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"can copy a file in\" '\n HASH=$(echo \"foo\" | ipfs add -q) &&\n ipfs files cp /ipfs/$HASH /file\n'\n\ntest_kill_ipfs_daemon\ntest_launch_ipfs_daemon\n\ntest_expect_success \"file is still there\" '\n verify_path_exists /file\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0252-files-gc.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Kevin Atkinson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"test how the unix files api interacts with the gc\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"object not removed after gc\" '\n echo \"hello world\" > hello.txt &&\n cat hello.txt | ipfs files write --create /hello.txt &&\n ipfs repo gc &&\n ipfs cat QmVib14uvPnCP73XaCDpwugRuwfTsVbGyWbatHAmLSdZUS\n'\n\ntest_expect_success \"/hello.txt still accessible after gc\" '\n ipfs files read /hello.txt > hello-actual &&\n test_cmp hello.txt hello-actual\n'\n\nADIR_HASH=QmbCgoMYVuZq8m1vK31JQx9DorwQdLMF1M3sJ7kygLLqnW\nFILE1_HASH=QmX4eaSJz39mNhdu5ACUwTDpyA6y24HmrQNnAape6u3buS\n\ntest_expect_success \"gc okay after adding incomplete node -- prep\" '\n ipfs files mkdir /adir &&\n echo \"file1\" | ipfs files write --create /adir/file1 &&\n echo \"file2\" | ipfs files write --create /adir/file2 &&\n ipfs pin add --recursive=false $ADIR_HASH &&\n ipfs files rm -r /adir &&\n ipfs repo gc && # will remove /adir/file1 and /adir/file2 but not /adir\n test_must_fail ipfs cat $FILE1_HASH &&\n ipfs files cp /ipfs/$ADIR_HASH /adir &&\n ipfs pin rm $ADIR_HASH\n'\n\ntest_expect_success \"gc okay after adding incomplete node\" '\n ipfs dag get $ADIR_HASH &&\n ipfs repo gc &&\n ipfs dag get $ADIR_HASH\n'\n\ntest_expect_success \"add directory with direct pin\" '\n mkdir mydir/ &&\n echo \"hello world!\" > mydir/hello.txt &&\n FILE_UNPINNED=$(ipfs add --pin=false -q -r mydir/hello.txt) &&\n DIR_PINNED=$(ipfs add --pin=false -Q -r mydir) &&\n ipfs add --pin=false -r mydir &&\n ipfs pin add --recursive=false $DIR_PINNED &&\n ipfs cat $FILE_UNPINNED\n'\n\ntest_expect_success \"run gc and make sure directory contents are removed\" '\n ipfs repo gc &&\n test_must_fail ipfs cat $FILE_UNPINNED\n'\n\ntest_expect_success \"add incomplete directory and make sure gc is okay\" '\n ipfs files cp /ipfs/$DIR_PINNED /mydir &&\n ipfs repo gc &&\n test_must_fail ipfs cat $FILE_UNPINNED\n'\n\ntest_expect_success \"add back directory contents and run gc\" '\n ipfs add --pin=false mydir/hello.txt &&\n ipfs repo gc\n'\n\ntest_expect_success \"make sure directory contents are not removed\" '\n ipfs cat $FILE_UNPINNED\n'\n\ntest_done\n" }, { "path": "test/sharness/t0260-sharding.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2014 Christian Couder\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test directory sharding\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"set up test data\" '\n mkdir testdata\n for i in `seq 2000`\n do\n echo $i > testdata/file$i\n done\n'\n\ntest_add_dir() {\n exphash=\"$1\"\n test_expect_success \"ipfs add on directory succeeds\" '\n ipfs add -r -Q testdata > sharddir_out &&\n echo \"$exphash\" > sharddir_exp &&\n test_cmp sharddir_exp sharddir_out\n '\n test_expect_success \"ipfs get on directory succeeds\" '\n ipfs get -o testdata-out \"$exphash\" &&\n test_cmp testdata testdata-out\n '\n}\n\ntest_init_ipfs\n\nUNSHARDED=\"QmavrTrQG4VhoJmantURAYuw3bowq3E2WcvP36NRQDAC1N\"\n\ntest_expect_success \"force sharding off\" '\nipfs config --json Import.UnixFSHAMTDirectorySizeThreshold \"\\\"1G\\\"\"\n'\n\ntest_add_dir \"$UNSHARDED\"\n\ntest_launch_ipfs_daemon\n\ntest_add_dir \"$UNSHARDED\"\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"force sharding on\" '\n ipfs config --json Import.UnixFSHAMTDirectorySizeThreshold \"\\\"1B\\\"\"\n'\n\nSHARDED=\"QmSCJD1KYLhVVHqBK3YyXuoEqHt7vggyJhzoFYbT8v1XYL\"\ntest_add_dir \"$SHARDED\"\n\ntest_launch_ipfs_daemon\n\ntest_add_dir \"$SHARDED\"\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"sharded and unsharded output look the same\" '\n ipfs ls \"$SHARDED\" | sort > sharded_out &&\n ipfs ls \"$UNSHARDED\" | sort > unsharded_out &&\n test_cmp sharded_out unsharded_out\n'\n\ntest_expect_success \"ipfs cat error output the same\" '\n test_expect_code 1 ipfs cat \"$SHARDED\" 2> sharded_err &&\n test_expect_code 1 ipfs cat \"$UNSHARDED\" 2> unsharded_err &&\n test_cmp sharded_err unsharded_err\n'\n\ntest_expect_success \"'ipfs ls --resolve-type=false --size=false' admits missing block\" '\n ipfs ls \"$SHARDED\" | head -1 > first_file &&\n ipfs ls --size=false \"$SHARDED\" | sort > sharded_out_nosize &&\n read -r HASH _ NAME missing_out &&\n test_cmp sharded_out_nosize missing_out\n'\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"gateway can resolve sharded dirs\" '\n echo 100 > expected &&\n curl -sfo actual \"http://127.0.0.1:$GWAY_PORT/ipfs/$SHARDED/file100\" &&\n test_cmp expected actual\n'\n\ntest_expect_success \"'ipfs resolve' can resolve sharded dirs\" '\n echo /ipfs/QmZ3RfWk1u5LEGYLHA633B5TNJy3Du27K6Fny9wcxpowGS > expected &&\n ipfs resolve \"/ipfs/$SHARDED/file100\" > actual &&\n test_cmp expected actual\n'\n\ntest_kill_ipfs_daemon\n\ntest_add_dir_v1() {\n exphash=\"$1\"\n test_expect_success \"ipfs add (CIDv1) on directory succeeds\" '\n ipfs add -r -Q --cid-version=1 testdata > sharddir_out &&\n echo \"$exphash\" > sharddir_exp &&\n test_cmp sharddir_exp sharddir_out\n '\n\n test_expect_success \"can access a path under the dir\" '\n ipfs cat \"$exphash/file20\" > file20_out &&\n test_cmp testdata/file20 file20_out\n '\n}\n\n# this hash implies the directory is CIDv1 and leaf entries are CIDv1 and raw\nSHARDEDV1=\"bafybeibiemewfzzdyhq2l74wrd6qj2oz42usjlktgnlqv4yfawgouaqn4u\"\ntest_add_dir_v1 \"$SHARDEDV1\"\n\ntest_launch_ipfs_daemon\n\ntest_add_dir_v1 \"$SHARDEDV1\"\n\ntest_kill_ipfs_daemon\n\ntest_list_incomplete_dir() {\n test_expect_success \"ipfs add (CIDv1) on very large directory with sha3 succeeds\" '\n ipfs add -r -Q --cid-version=1 --hash=sha3-256 --pin=false testdata > sharddir_out &&\n largeSHA3dir=$(cat sharddir_out)\n '\n\n test_expect_success \"delete intermediate node from DAG\" '\n ipfs block rm \"/ipld/$largeSHA3dir/Links/0/Hash\"\n '\n\n test_expect_success \"can list part of the directory\" '\n ipfs ls \"$largeSHA3dir\" 2> ls_err_out\n echo \"Error: failed to fetch all nodes\" > exp_err_out &&\n cat ls_err_out &&\n test_cmp exp_err_out ls_err_out\n '\n}\n\ntest_list_incomplete_dir\n\ntest_done\n" }, { "path": "test/sharness/t0270-filestore.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test out the filestore nocopy functionality\"\n\n. lib/test-lib.sh\n\n\ntest_expect_success \"create a dataset\" '\n random-files -seed=483 -depth=3 -dirs=4 -files=6 -filesize=1000000 somedir > /dev/null\n'\n\nEXPHASH=\"QmXKtATsEt42CF5JoSsmzJstrvwEB5P89YQtdX4mdf9E3M\"\n\nget_repo_size() {\n disk_usage \"$IPFS_PATH\"\n}\n\nassert_repo_size_less_than() {\n expval=\"$1\"\n\n test_expect_success \"check repo size\" '\n test \"$(get_repo_size)\" -lt \"$expval\" ||\n { echo should be below \"$expval\" && test_fsh get_repo_size; }\n '\n}\n\nassert_repo_size_greater_than() {\n expval=\"$1\"\n\n test_expect_success \"check repo size\" '\n test \"$(get_repo_size)\" -gt \"$expval\" ||\n { echo should be above \"$expval\" && test_fsh get_repo_size; }\n '\n}\n\ntest_filestore_adds() {\n test_expect_success \"nocopy add succeeds\" '\n HASH=$(ipfs add --raw-leaves --nocopy -r -Q somedir)\n '\n\n test_expect_success \"nocopy add has right hash\" '\n test \"$HASH\" = \"$EXPHASH\"\n '\n\n assert_repo_size_less_than 1000000\n\n test_expect_success \"normal add with fscache doesn't duplicate data\" '\n ipfs add --raw-leaves --fscache -r -q somedir > /dev/null\n '\n\n assert_repo_size_less_than 1000000\n\n test_expect_success \"normal add without fscache duplicates data\" '\n ipfs add --raw-leaves -r -q somedir > /dev/null\n '\n\n assert_repo_size_greater_than 1000000\n}\n\ninit_ipfs_filestore() {\n test_expect_success \"clean up old node\" '\n rm -rf \"$IPFS_PATH\" mountdir ipfs ipns mfs\n '\n\n test_init_ipfs\n\n # Check the _early_ error message\n test_expect_success \"nocopy add errors and has right message\" '\n test_must_fail ipfs add --nocopy -r somedir 2> add_out &&\n grep \"either the filestore or the urlstore must be enabled\" add_out\n '\n\n assert_repo_size_less_than 1000000\n\n test_expect_success \"enable urlstore config setting\" '\n ipfs config --json Experimental.UrlstoreEnabled true\n '\n\n # Check the _late_ error message\n test_expect_success \"nocopy add errors and has right message when the urlstore is enabled\" '\n test_must_fail ipfs add --nocopy -r somedir 2> add_out &&\n grep \"filestore is not enabled\" add_out\n '\n\n assert_repo_size_less_than 1000000\n\n test_expect_success \"enable filestore config setting\" '\n ipfs config --json Experimental.UrlstoreEnabled true &&\n ipfs config --json Experimental.FilestoreEnabled true\n '\n}\n\ninit_ipfs_filestore\n\ntest_filestore_adds\n\ntest_debug '\n echo \"pwd=$(pwd)\"; echo \"IPFS_PATH=$IPFS_PATH\"\n'\n\n\ninit_ipfs_filestore\n\ntest_launch_ipfs_daemon\n\ntest_filestore_adds\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0271-filestore-utils.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test out the filestore nocopy functionality\"\n\n. lib/test-lib.sh\n\ntest_init_filestore() {\n test_expect_success \"clean up old node\" '\n rm -rf \"$IPFS_PATH\" mountdir ipfs ipns mfs\n '\n\n test_init_ipfs\n\n test_expect_success \"enable filestore config setting\" '\n ipfs config --json Experimental.FilestoreEnabled true\n '\n}\n\ntest_init_dataset() {\n test_expect_success \"create a dataset\" '\n rm -r somedir\n mkdir somedir &&\n random-data -size=1000 -seed=1 > somedir/file1 &&\n random-data -size=10000 -seed=2 > somedir/file2 &&\n random-data -size=1000000 -seed=3 > somedir/file3\n '\n}\n\ntest_init() {\n test_init_filestore\n test_init_dataset\n}\n\nEXPHASH=\"QmXqfraAT3U8ct14PPPXcFkWyvmqUZazLdo29GXTKSHkP4\"\n\ncat < ls_expect_file_order\nbafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu 1000 somedir/file1 0\nbafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4 10000 somedir/file2 0\nbafkreiemzfmzws23c2po4m6deiueknqfty7r3voes3e3zujmobrooc2ngm 262144 somedir/file3 0\nbafkreihgm53yhxn427lnfdwhqgpawc62qejog7gega5kqb6uwbyhjm47hu 262144 somedir/file3 262144\nbafkreigl2pjptgxz6cexcnua56zc5dwsyrc4ph2eulmcb634oes6gzvmuy 262144 somedir/file3 524288\nbafkreifjcthslybjizk36xffcsb32fsbguxz3ptkl7723wz4u3qikttmam 213568 somedir/file3 786432\nEOF\n\nsort < ls_expect_file_order > ls_expect_key_order\n\nFILE1_HASH=bafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu\nFILE2_HASH=bafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4\nFILE3_HASH=QmYEZtRGGk8rgM8MetegLLRHMKskPCg7zWpmQQAo3cQiN5\n\ncat < verify_expect_file_order\nok bafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu 1000 somedir/file1 0\nok bafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4 10000 somedir/file2 0\nok bafkreiemzfmzws23c2po4m6deiueknqfty7r3voes3e3zujmobrooc2ngm 262144 somedir/file3 0\nok bafkreihgm53yhxn427lnfdwhqgpawc62qejog7gega5kqb6uwbyhjm47hu 262144 somedir/file3 262144\nok bafkreigl2pjptgxz6cexcnua56zc5dwsyrc4ph2eulmcb634oes6gzvmuy 262144 somedir/file3 524288\nok bafkreifjcthslybjizk36xffcsb32fsbguxz3ptkl7723wz4u3qikttmam 213568 somedir/file3 786432\nEOF\n\nsort < verify_expect_file_order > verify_expect_key_order\n\ncat < verify_rm_expect\nok bafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4 10000 somedir/file2 0 keep\nok bafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu 1000 somedir/file1 0 keep\nchanged bafkreiemzfmzws23c2po4m6deiueknqfty7r3voes3e3zujmobrooc2ngm 262144 somedir/file3 0 remove\nchanged bafkreifjcthslybjizk36xffcsb32fsbguxz3ptkl7723wz4u3qikttmam 213568 somedir/file3 786432 remove\nchanged bafkreigl2pjptgxz6cexcnua56zc5dwsyrc4ph2eulmcb634oes6gzvmuy 262144 somedir/file3 524288 remove\nchanged bafkreihgm53yhxn427lnfdwhqgpawc62qejog7gega5kqb6uwbyhjm47hu 262144 somedir/file3 262144 remove\nEOF\n\ncat < verify_after_rm_expect\nok bafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4 10000 somedir/file2 0\nok bafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu 1000 somedir/file1 0\nEOF\n\nIPFS_CMD=\"ipfs\"\n\ntest_filestore_adds() {\n test_expect_success \"$IPFS_CMD add nocopy add succeeds\" '\n HASH=$($IPFS_CMD add --raw-leaves --nocopy -r -Q somedir)\n '\n\n test_expect_success \"nocopy add has right hash\" '\n test \"$HASH\" = \"$EXPHASH\"\n '\n\n test_expect_success \"'$IPFS_CMD filestore ls' output looks good'\" '\n $IPFS_CMD filestore ls | sort > ls_actual &&\n test_cmp ls_expect_key_order ls_actual\n '\n\n test_expect_success \"'$IPFS_CMD filestore ls --file-order' output looks good'\" '\n $IPFS_CMD filestore ls --file-order > ls_actual &&\n test_cmp ls_expect_file_order ls_actual\n '\n\n test_expect_success \"'$IPFS_CMD filestore ls HASH' works\" '\n $IPFS_CMD filestore ls $FILE1_HASH > ls_actual &&\n grep -q somedir/file1 ls_actual\n '\n\n test_expect_success \"can retrieve multi-block file\" '\n $IPFS_CMD cat $FILE3_HASH > file3.data &&\n test_cmp somedir/file3 file3.data\n '\n}\n\n# check that the filestore is in a clean state\ntest_filestore_state() {\n test_expect_success \"$IPFS_CMD filestore verify' output looks good'\" '\n $IPFS_CMD filestore verify | LC_ALL=C sort > verify_actual\n test_cmp verify_expect_key_order verify_actual\n '\n}\n\ntest_filestore_verify() {\n test_filestore_state\n\n test_expect_success \"$IPFS_CMD filestore verify --file-order' output looks good'\" '\n $IPFS_CMD filestore verify --file-order > verify_actual\n test_cmp verify_expect_file_order verify_actual\n '\n\n test_expect_success \"'$IPFS_CMD filestore verify HASH' works\" '\n $IPFS_CMD filestore verify $FILE1_HASH > verify_actual &&\n grep -q somedir/file1 verify_actual\n '\n\n test_expect_success \"rename a file\" '\n mv somedir/file1 somedir/file1.bk\n '\n\n test_expect_success \"can not retrieve block after backing file moved\" '\n test_must_fail $IPFS_CMD cat $FILE1_HASH\n '\n\n test_expect_success \"'$IPFS_CMD filestore verify' shows file as missing\" '\n $IPFS_CMD filestore verify > verify_actual &&\n grep no-file verify_actual | grep -q somedir/file1\n '\n\n test_expect_success \"move file back\" '\n mv somedir/file1.bk somedir/file1\n '\n\n test_expect_success \"block okay now\" '\n $IPFS_CMD cat $FILE1_HASH > file1.data &&\n test_cmp somedir/file1 file1.data\n '\n\n test_expect_success \"change first bit of file\" '\n dd if=/dev/zero of=somedir/file3 bs=1024 count=1\n '\n\n test_expect_success \"can not retrieve block after backing file changed\" '\n test_must_fail $IPFS_CMD cat $FILE3_HASH\n '\n\n test_expect_success \"'$IPFS_CMD filestore verify' shows file as changed\" '\n $IPFS_CMD filestore verify > verify_actual &&\n grep changed verify_actual | grep -q somedir/file3\n '\n\n # reset the state for the next test\n test_init_dataset\n}\n\ntest_filestore_rm_bad_blocks() {\n test_filestore_state\n\n test_expect_success \"change first bit of file\" '\n dd if=/dev/zero of=somedir/file3 bs=1024 count=1\n '\n\n test_expect_success \"'$IPFS_CMD filestore verify --remove-bad-blocks' shows changed file removed\" '\n $IPFS_CMD filestore verify --remove-bad-blocks > verify_rm_actual &&\n test_cmp verify_rm_expect verify_rm_actual\n '\n\n test_expect_success \"'$IPFS_CMD filestore verify' shows only files that were not removed\" '\n $IPFS_CMD filestore verify > verify_after &&\n test_cmp verify_after_rm_expect verify_after\n '\n \n # reset the state for the next test\n test_init_dataset\n}\n \ntest_filestore_dups() {\n # make sure the filestore is in a clean state\n test_filestore_state\n\n test_expect_success \"'$IPFS_CMD filestore dups'\" '\n $IPFS_CMD add --raw-leaves somedir/file1 &&\n $IPFS_CMD filestore dups > dups_actual &&\n echo \"$FILE1_HASH\" > dups_expect\n test_cmp dups_expect dups_actual\n '\n}\n\n#\n# No daemon\n#\n\ntest_init\n\ntest_filestore_adds\n\ntest_filestore_verify\n\ntest_filestore_dups\n\ntest_filestore_rm_bad_blocks\n\n#\n# With daemon\n#\n\ntest_init\n\n# must be in offline mode so tests that retrieve non-existent blocks\n# doesn't hang\ntest_launch_ipfs_daemon_without_network\n\ntest_filestore_adds\n\ntest_filestore_verify\n\ntest_filestore_dups\n\ntest_kill_ipfs_daemon\n\ntest_filestore_rm_bad_blocks\n\n##\n## base32\n##\n\nEXPHASH=\"bafybeienfbjfbywu5y44i5qm4wxajblgy5a6xuc4eepjaw5fq223wwsy3m\"\n\ncat < ls_expect_file_order\nbafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu 1000 somedir/file1 0\nbafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4 10000 somedir/file2 0\nbafkreiemzfmzws23c2po4m6deiueknqfty7r3voes3e3zujmobrooc2ngm 262144 somedir/file3 0\nbafkreihgm53yhxn427lnfdwhqgpawc62qejog7gega5kqb6uwbyhjm47hu 262144 somedir/file3 262144\nbafkreigl2pjptgxz6cexcnua56zc5dwsyrc4ph2eulmcb634oes6gzvmuy 262144 somedir/file3 524288\nbafkreifjcthslybjizk36xffcsb32fsbguxz3ptkl7723wz4u3qikttmam 213568 somedir/file3 786432\nEOF\n\nsort < ls_expect_file_order > ls_expect_key_order\n\nFILE1_HASH=bafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu\nFILE2_HASH=bafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4\nFILE3_HASH=bafybeietaxxjghilcjhc2m4zcmicm7yjvkjdfkamc3ct2hq4gmsb3shqsi\n\ncat < verify_expect_file_order\nok bafkreidx7ivgllulfkzyoo4oa7dfrg4mjmudg2qgdivoooj4s7lh3m5nqu 1000 somedir/file1 0\nok bafkreic2wqrsyr3y3qgzbvufen2w25r3p3zljckqyxkpcagsxz3zdcosd4 10000 somedir/file2 0\nok bafkreiemzfmzws23c2po4m6deiueknqfty7r3voes3e3zujmobrooc2ngm 262144 somedir/file3 0\nok bafkreihgm53yhxn427lnfdwhqgpawc62qejog7gega5kqb6uwbyhjm47hu 262144 somedir/file3 262144\nok bafkreigl2pjptgxz6cexcnua56zc5dwsyrc4ph2eulmcb634oes6gzvmuy 262144 somedir/file3 524288\nok bafkreifjcthslybjizk36xffcsb32fsbguxz3ptkl7723wz4u3qikttmam 213568 somedir/file3 786432\nEOF\n\nsort < verify_expect_file_order > verify_expect_key_order\n\nIPFS_CMD=\"ipfs --cid-base=base32\"\n\n#\n# No daemon\n#\n\ntest_init\n\ntest_filestore_adds\n\ntest_filestore_verify\n\ntest_filestore_dups\n\ntest_filestore_rm_bad_blocks\n\n#\n# With daemon\n#\n\ntest_init\n\n# must be in offline mode so tests that retrieve non-existent blocks\n# doesn't hang\ntest_launch_ipfs_daemon_without_network\n\ntest_filestore_adds\n\ntest_filestore_verify\n\ntest_filestore_dups\n\ntest_kill_ipfs_daemon\n\ntest_done\n\ntest_filestore_rm_bad_blocks\n\n##\n\ntest_done\n" }, { "path": "test/sharness/t0272-urlstore.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test out the urlstore functionality\"\n\n. lib/test-lib.sh\n\n\ntest_expect_success \"create some random files\" '\n random-data -size=2222 -seed=7 > file1 &&\n random-data -size=500000 -seed=7 > file2 &&\n random-data -size=50000000 -seed=7 > file3\n'\n\ntest_urlstore() {\n ADD_CMD=\"${@}\"\n\n test_init_ipfs\n \n test_expect_success \"add files using trickle dag format without raw leaves\" '\n HASH1a=$(ipfs add -q --trickle --raw-leaves=false file1) &&\n HASH2a=$(ipfs add -q --trickle --raw-leaves=false file2) &&\n HASH3a=$(ipfs add -q --trickle --raw-leaves=false file3)\n '\n \n test_launch_ipfs_daemon_without_network\n \n test_expect_success \"make sure files can be retrieved via the gateway\" '\n curl http://127.0.0.1:$GWAY_PORT/ipfs/$HASH1a -o file1.actual &&\n test_cmp file1 file1.actual &&\n curl http://127.0.0.1:$GWAY_PORT/ipfs/$HASH2a -o file2.actual &&\n test_cmp file2 file2.actual &&\n curl http://127.0.0.1:$GWAY_PORT/ipfs/$HASH3a -o file3.actual &&\n test_cmp file3 file3.actual \n '\n \n test_expect_success \"add files without enabling url store using $ADD_CMD\" '\n test_must_fail ipfs $ADD_CMD http://127.0.0.1:$GWAY_PORT/ipfs/$HASH1a &&\n test_must_fail ipfs $ADD_CMD http://127.0.0.1:$GWAY_PORT/ipfs/$HASH2a\n '\n \n test_kill_ipfs_daemon\n \n test_expect_success \"enable urlstore\" '\n ipfs config --json Experimental.UrlstoreEnabled true\n '\n \n test_launch_ipfs_daemon_without_network\n \n test_expect_success \"add files using gateway address via url store using $ADD_CMD\" '\n HASH1=$(ipfs $ADD_CMD --pin=false http://127.0.0.1:$GWAY_PORT/ipfs/$HASH1a) &&\n HASH2=$(ipfs $ADD_CMD http://127.0.0.1:$GWAY_PORT/ipfs/$HASH2a)\n '\n \n test_expect_success \"make sure hashes are different\" '\n test $HASH1a != $HASH1 &&\n test $HASH2a != $HASH2\n '\n \n test_expect_success \"get files via urlstore\" '\n rm -f file1.actual file2.actual &&\n ipfs get $HASH1 -o file1.actual &&\n test_cmp file1 file1.actual &&\n ipfs get $HASH2 -o file2.actual &&\n test_cmp file2 file2.actual\n '\n\n cat < ls_expect\nbafkreiconmdoujderxi757nf4wjpo4ukbhlo6mmxs6pg3yl53ln3ykldvi 2222 http://127.0.0.1:$GWAY_PORT/ipfs/QmUNEBSK2uPLSZU3Dj6XbSHjdGze4huWxESx2R4Ef1cKRW 0\nbafkreifybqsfcheqkxzlhuuvoi3u6wz42kic4yqohvkia2i5fg3mpkqt3i 262144 http://127.0.0.1:$GWAY_PORT/ipfs/QmTgZc5bhTHUcqGN8rRP9oTJBv1UeJVWufPMPiUfbP9Ghs 0\nbafkreigxuuyoickqhwxu4kjckmgfqb7ygd426qiakryvvstixy523imym4 237856 http://127.0.0.1:$GWAY_PORT/ipfs/QmTgZc5bhTHUcqGN8rRP9oTJBv1UeJVWufPMPiUfbP9Ghs 262144\nEOF\n\n test_expect_success \"ipfs filestore ls works with urls\" '\n ipfs filestore ls | sort > ls_actual &&\n test_cmp ls_expect ls_actual\n '\n\n cat < verify_expect\nok bafkreifybqsfcheqkxzlhuuvoi3u6wz42kic4yqohvkia2i5fg3mpkqt3i 262144 http://127.0.0.1:$GWAY_PORT/ipfs/QmTgZc5bhTHUcqGN8rRP9oTJBv1UeJVWufPMPiUfbP9Ghs 0\nok bafkreigxuuyoickqhwxu4kjckmgfqb7ygd426qiakryvvstixy523imym4 237856 http://127.0.0.1:$GWAY_PORT/ipfs/QmTgZc5bhTHUcqGN8rRP9oTJBv1UeJVWufPMPiUfbP9Ghs 262144\nok bafkreiconmdoujderxi757nf4wjpo4ukbhlo6mmxs6pg3yl53ln3ykldvi 2222 http://127.0.0.1:$GWAY_PORT/ipfs/QmUNEBSK2uPLSZU3Dj6XbSHjdGze4huWxESx2R4Ef1cKRW 0\nEOF\n \n test_expect_success \"ipfs filestore verify works with urls\" '\n ipfs filestore verify | sort > verify_actual &&\n test_cmp verify_expect verify_actual\n '\n \n test_expect_success \"garbage collect file1 from the urlstore\" '\n ipfs repo gc > /dev/null\n '\n \n test_expect_success \"can no longer retrieve file1 from urlstore\" '\n rm -f file1.actual &&\n test_must_fail ipfs get $HASH1 -o file1.actual\n '\n \n test_expect_success \"can still retrieve file2 from urlstore\" '\n rm -f file2.actual &&\n ipfs get $HASH2 -o file2.actual &&\n test_cmp file2 file2.actual\n '\n \n test_expect_success \"remove original hashes from local gateway\" '\n ipfs pin rm $HASH1a $HASH2a &&\n ipfs repo gc > /dev/null\n '\n \n test_expect_success \"gateway no longer has files\" '\n test_must_fail curl -f http://127.0.0.1:$GWAY_PORT/ipfs/$HASH1a -o file1.actual\n test_must_fail curl -f http://127.0.0.1:$GWAY_PORT/ipfs/$HASH2a -o file2.actual\n '\n\n cat < verify_expect_2\nerror bafkreifybqsfcheqkxzlhuuvoi3u6wz42kic4yqohvkia2i5fg3mpkqt3i 262144 http://127.0.0.1:$GWAY_PORT/ipfs/QmTgZc5bhTHUcqGN8rRP9oTJBv1UeJVWufPMPiUfbP9Ghs 0\nerror bafkreigxuuyoickqhwxu4kjckmgfqb7ygd426qiakryvvstixy523imym4 237856 http://127.0.0.1:$GWAY_PORT/ipfs/QmTgZc5bhTHUcqGN8rRP9oTJBv1UeJVWufPMPiUfbP9Ghs 262144\nEOF\n \n test_expect_success \"ipfs filestore verify is correct\" '\n ipfs filestore verify | sort > verify_actual_2 &&\n test_cmp verify_expect_2 verify_actual_2\n '\n \n test_expect_success \"files cannot be retrieved via the urlstore\" '\n test_must_fail ipfs cat $HASH1 > /dev/null &&\n test_must_fail ipfs cat $HASH2 > /dev/null\n '\n \n test_expect_success \"remove broken files\" '\n ipfs pin rm $HASH2 &&\n ipfs repo gc > /dev/null\n '\n \n test_expect_success \"add large file using gateway address via url store\" '\n HASH3=$(ipfs ${ADD_CMD[@]} http://127.0.0.1:$GWAY_PORT/ipfs/$HASH3a)\n '\n \n test_expect_success \"make sure hashes are different\" '\n test $HASH3a != $HASH3\n '\n \n test_expect_success \"get large file via urlstore\" '\n rm -f file3.actual &&\n ipfs get $HASH3 -o file3.actual &&\n test_cmp file3 file3.actual\n '\n \n test_expect_success \"check that the trickle option works\" '\n HASHat=$(ipfs add -q --cid-version=1 --raw-leaves=true -n --trickle file3) &&\n HASHut=$(ipfs $ADD_CMD --trickle http://127.0.0.1:$GWAY_PORT/ipfs/$HASH3a) &&\n test $HASHat = $HASHut\n '\n \n test_expect_success \"add files using gateway address via url store using --cid-base=base32\" '\n HASH1a=$(ipfs add -q --trickle --raw-leaves=false file1) &&\n HASH2a=$(ipfs add -q --trickle --raw-leaves=false file2) &&\n HASH1b32=$(ipfs --cid-base=base32 $ADD_CMD http://127.0.0.1:$GWAY_PORT/ipfs/$HASH1a) &&\n HASH2b32=$(ipfs --cid-base=base32 $ADD_CMD http://127.0.0.1:$GWAY_PORT/ipfs/$HASH2a)\n '\n \n test_kill_ipfs_daemon\n \n test_expect_success \"files cannot be retrieved via the urlstore\" '\n test_must_fail ipfs cat $HASH1 > /dev/null &&\n test_must_fail ipfs cat $HASH2 > /dev/null &&\n test_must_fail ipfs cat $HASH3 > /dev/null\n '\n \n test_expect_success \"check that the hashes were correct\" '\n HASH1e=$(ipfs add -q -n --cid-version=1 --raw-leaves=true file1) &&\n HASH2e=$(ipfs add -q -n --cid-version=1 --raw-leaves=true file2) &&\n HASH3e=$(ipfs add -q -n --cid-version=1 --raw-leaves=true file3) &&\n test $HASH1e = $HASH1 &&\n test $HASH2e = $HASH2 &&\n test $HASH3e = $HASH3\n '\n \n test_expect_success \"check that the base32 hashes were correct\" '\n HASH1e32=$(ipfs cid base32 $HASH1e)\n HASH2e32=$(ipfs cid base32 $HASH2e)\n test $HASH1e32 = $HASH1b32 &&\n test $HASH2e32 = $HASH2b32\n '\n\n test_expect_success \"ipfs cleanup\" '\n rm -rf \"$IPFS_PATH\" && rmdir ipfs ipns mountdir\n '\n}\n\ntest_urlstore add -q --nocopy --cid-version=1\n\ntest_done\n" }, { "path": "test/sharness/t0275-cid-security-data/CIQG6PGTD2VV34S33BE4MNCQITBRFYUPYQLDXYARR3DQW37MOT7K5XI.data", "content": "\n\u0005\b\u0002\u0018\u0001\u0012\u0007\n\u0005\u0001U\u0012\u0001m" }, { "path": "test/sharness/t0275-cid-security-data/EICEM7ITSI.data", "content": "testing\n" }, { "path": "test/sharness/t0275-cid-security.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jakub Sztandera\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Cid Security\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_expect_success \"adding using unsafe function fails with error\" '\n echo foo | test_must_fail ipfs add --hash shake-128 2>add_out\n'\n\ntest_expect_success \"error reason is pointed out\" '\n grep \"potentially insecure hash functions not allowed\" add_out || test_fsh cat add_out\n'\n\ntest_expect_success \"adding using too short of a hash function gives out an error\" '\n echo foo | test_must_fail ipfs block put -f protobuf --mhlen 19 2>block_out\n'\n\ntest_expect_success \"error reason is pointed out\" '\n grep \"digest too small\" block_out\n'\n\n\ntest_cat_get() {\n\n test_expect_success \"ipfs cat fails with unsafe hash function\" '\n test_must_fail ipfs cat bafksebhh7d53e 2>ipfs_cat\n '\n\n\n test_expect_success \"error reason is pointed out\" '\n grep \"potentially insecure hash functions not allowed\" ipfs_cat\n '\n\n\n test_expect_success \"ipfs get fails with too short function\" '\n test_must_fail ipfs get bafkreez3itiri7ghbbf6lzej7paxyxy2qznpw 2>ipfs_get\n\n '\n\n test_expect_success \"error reason is pointed out\" '\n grep \"digest too small\" ipfs_get\n '\n}\n\n\ntest_gc() {\n test_expect_success \"injecting insecure block\" '\n mkdir -p \"$IPFS_PATH/blocks/TS\" &&\n cp -f ../t0275-cid-security-data/EICEM7ITSI.data \"$IPFS_PATH/blocks/TS\"\n '\n\n test_expect_success \"gc works\" 'ipfs repo gc > gc_out'\n test_expect_success \"gc removed bad block\" '\n grep bafksebcgpujze gc_out\n '\n}\n\n\n# should work offline\ntest_cat_get\ntest_gc\n\n# should work online\ntest_launch_ipfs_daemon\ntest_cat_get\ntest_gc\n\ntest_expect_success \"add block linking to insecure\" '\n mkdir -p \"$IPFS_PATH/blocks/5X\" &&\n cp -f \"../t0275-cid-security-data/CIQG6PGTD2VV34S33BE4MNCQITBRFYUPYQLDXYARR3DQW37MOT7K5XI.data\" \"$IPFS_PATH/blocks/5X\"\n'\n\ntest_expect_success \"ipfs cat fails with code 1 and not timeout\" '\n test_expect_code 1 go-timeout 1 ipfs cat QmVpsktzNeJdfWEpyeix93QJdQaBSgRNxebSbYSo9SQPGx\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0276-cidv0v1.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jakub Sztandera\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"CID Version 0/1 Duality\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n#\n#\n#\n\ntest_expect_success \"create two small files\" '\n random-data -size=1000 -seed=7 > afile\n random-data -size=1000 -seed=9 > bfile\n'\n\ntest_expect_success \"add file using CIDv1 but don't pin\" '\n AHASHv1=$(ipfs add -q --cid-version=1 --raw-leaves=false --pin=false afile)\n'\n\ntest_expect_success \"add file using CIDv0\" '\n AHASHv0=$(ipfs add -q --cid-version=0 afile)\n'\n\ntest_expect_success \"check hashes\" '\n test \"$(cid-fmt %v-%c $AHASHv0)\" = \"cidv0-dag-pb\" &&\n test \"$(cid-fmt %v-%c $AHASHv1)\" = \"cidv1-dag-pb\" &&\n test \"$(cid-fmt -b z -v 0 %s $AHASHv1)\" = \"$AHASHv0\"\n'\n\ntest_expect_success \"make sure CIDv1 hash really is in the repo\" '\n ipfs block stat $AHASHv1\n'\n\ntest_expect_success \"make sure CIDv0 hash really is in the repo\" '\n ipfs block stat $AHASHv0\n'\n\ntest_expect_success \"run gc\" '\n ipfs repo gc\n'\n\ntest_expect_success \"make sure the CIDv0 hash is in the repo\" '\n ipfs block stat $AHASHv0\n'\n\ntest_expect_success \"make sure we can get CIDv0 added file\" '\n ipfs cat $AHASHv0 > thefile &&\n test_cmp afile thefile\n'\n\ntest_expect_success \"make sure the CIDv1 hash is not in the repo\" '\n ! ipfs refs local | grep -q $AHASHv1\n'\n\ntest_expect_success \"clean up\" '\n ipfs pin rm $AHASHv0 &&\n ipfs repo gc &&\n ! ipfs refs local | grep -q $AHASHv0\n'\n\n#\n#\n#\n\ntest_expect_success \"add file using CIDv1 but don't pin\" '\n ipfs add -q --cid-version=1 --raw-leaves=false --pin=false afile\n'\n\ntest_expect_success \"check that we can access the file when converted to CIDv0\" '\n ipfs cat $AHASHv0 > thefile &&\n test_cmp afile thefile\n'\n\ntest_expect_success \"clean up\" '\n ipfs repo gc\n'\n\ntest_expect_success \"add file using CIDv0 but don't pin\" '\n ipfs add -q --cid-version=0 --raw-leaves=false --pin=false afile\n'\n\ntest_expect_success \"check that we can access the file when converted to CIDv1\" '\n ipfs cat $AHASHv1 > thefile &&\n test_cmp afile thefile\n'\n\n#\n#\n#\n\ntest_expect_success \"set up iptb testbed\" '\n iptb testbed create -type localipfs -count 2 -init &&\n iptb run -- ipfs config --json \"Routing.LoopbackAddressesOnLanDHT\" true\n'\n\ntest_expect_success \"start nodes\" '\n iptb start -wait &&\n iptb connect 0 1\n'\n\ntest_expect_success \"add afile using CIDv0 to node 0\" '\n iptb run 0 -- ipfs add -q --cid-version=0 afile\n'\n\ntest_expect_success \"get afile using CIDv1 via node 1\" '\n iptb -quiet run 1 -- ipfs --timeout=2s cat $AHASHv1 > thefile &&\n test_cmp afile thefile\n'\n\ntest_expect_success \"add bfile using CIDv1 to node 0\" '\n BHASHv1=$(iptb -quiet run 0 -- ipfs add -q --cid-version=1 --raw-leaves=false bfile)\n'\n\ntest_expect_success \"get bfile using CIDv0 via node 1\" '\n BHASHv0=$(cid-fmt -b z -v 0 %s $BHASHv1)\n echo $BHASHv1 &&\n iptb -quiet run 1 -- ipfs --timeout=2s cat $BHASHv0 > thefile &&\n test_cmp bfile thefile\n'\n\ntest_expect_success \"stop testbed\" '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0280-plugin-dag-jose-data/dag-cbor/bafyreicxyzuqbx5yb7ytkgkuofwksbal3ygtswxuri25crxdxms55m5fki", "content": "bivpPSWIuAyO8CpevzCLctagvWZAMBblhzDCsQWOAKdlkSAiprotectedx'eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4R0NNIn0jciphertextx 3XqLW28NHP-raqW8vMfIHOzko4N3IRaR" }, { "path": "test/sharness/t0280-plugin-dag-jose-data/dag-cbor/bafyreihkt4u6euddfhofkutfzxwet7w7zm5qrjpop655yhnb5dnzqw26lm", "content": "bivpQ8xpPt_zZrfvHgR-ctagvjxHjcVusu0yrOBzw-Ex5zAiprotectedx3eyJhbGciOiJSU0EtT0FFUC0yNTYiLCJlbmMiOiJBMjU2R0NNIn0jciphertextx WTaw6WeqhaZDDhedzGYnsty4SMR-RzOwjrecipientsmencrypted_keyy\u0001VEqYaN4dFEH0vX4jU3d768hwOYSgZhElvVDzqdIKA6PFHsL4PPwJ7EIuebLrxwABJqXWBNG0kUBRjHuCv51VlxvX9WoH9ik7Qew0yROCGBj_AJef15PiZzUVUQwteHVDuSQs4OcsMfj18zc_ObskHvUMaN0PdCBA-G__7rGR2tcYSJOywbvxqqTENsCZNvasKxHSOuA_bjVsRmWloUMwLJkrbQxPAsVcwoPjAYF2agQ8D40AGFVEzGmhQDLI-OpXI-AfZYBurE7f_fU_NsYtqmFj5vZ9lvVCV1QsZa_HRhQlBBHxjTKyCBufY-0G4omt2nzYhyO-TaH44eUh81HFzww" }, { "path": "test/sharness/t0280-plugin-dag-jose-data/dag-jose/bagcqcerakjv2mmdlbai3urym22bw5kaw7nqov73yaxf6xjnp7e56sclsrooa", "content": "bivX\u0018;J:k5\tYi3J*ctagP\u00122B]\u000eAk\u0018 $(basename {}); \\\n diff {} $(basename {}) \\'\n '\n\n test_expect_success \"retrieve dag-jose in non-dag-jose encodings\" $'\n find ../t0280-plugin-dag-jose-data -type f | xargs -I {} sh -c \\' \\\n codec=$(basename $(dirname {})); \\\n joseHash=$(ipfs dag put --store-codec dag-jose --input-codec=$codec {}); \\\n ipfs dag get --output-codec dag-cbor $joseHash > /dev/null; \\\n ipfs dag get --output-codec dag-json $joseHash > /dev/null \\'\n '\n}\n\n# should work offline\ntest_dag_jose\n\n# should work online\ntest_launch_ipfs_daemon\ntest_dag_jose\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0280-plugin-data/example.go", "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/ipfs/kubo/plugin\"\n)\n\nvar Plugins = []plugin.Plugin{\n\t&testPlugin{},\n}\n\nvar _ = Plugins // used\n\ntype testPlugin struct{}\n\nfunc (*testPlugin) Name() string {\n\treturn \"test-plugin\"\n}\n\nfunc (*testPlugin) Version() string {\n\treturn \"0.1.0\"\n}\n\nfunc (*testPlugin) Init(env *plugin.Environment) error {\n\tfmt.Fprintf(os.Stderr, \"testplugin %s\\n\", env.Repo)\n\tfmt.Fprintf(os.Stderr, \"testplugin %v\\n\", env.Config)\n\treturn nil\n}\n" }, { "path": "test/sharness/t0280-plugin-fx.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test fx plugin\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nexport GOLOG_LOG_LEVEL=\"fxtestplugin=debug\"\nexport TEST_FX_PLUGIN=1\ntest_launch_ipfs_daemon\n\ntest_expect_success \"expected log entry should be present\" '\n fgrep \"invoked test fx function\" daemon_err >/dev/null\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0280-plugin-git.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jakub Sztandera\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test git plugin\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# from https://github.com/ipfs/go-ipld-git/blob/master/make-test-repo.sh\ntest_expect_success \"prepare test data\" '\n tar xzf ../t0280-plugin-git-data/git.tar.gz\n'\n\ntest_dag_git() {\n test_expect_success \"add objects via dag put\" '\n find objects -type f -exec ipfs dag put --store-codec=git-raw --input-codec=0x300078 --hash=sha1 {} \\; -exec echo -n \\; > hashes\n '\n\n test_expect_success \"successfully get added objects\" '\n cat hashes | xargs -I {} ipfs dag get -- {} > /dev/null\n '\n\n test_expect_success \"dag get works\" '\n echo -n \"{\\\"message\\\":\\\"Some version\\n\\\",\\\"object\\\":{\\\"/\\\":\\\"baf4bcfeq6c2mspupcvftgevza56h7rmozose6wi\\\"},\\\"tag\\\":\\\"v1\\\",\\\"tagger\\\":{\\\"date\\\":\\\"1497302532\\\",\\\"email\\\":\\\"johndoe@example.com\\\",\\\"name\\\":\\\"John Doe\\\",\\\"timezone\\\":\\\"+0200\\\"},\\\"type\\\":\\\"commit\\\"}\" > tag_expected &&\n ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi > tag_actual\n '\n\n test_expect_success \"outputs look correct\" '\n test_cmp tag_expected tag_actual\n '\n\n test_expect_success \"path traversals work\" '\n echo -n \"{\\\"date\\\":\\\"1497302532\\\",\\\"email\\\":\\\"johndoe@example.com\\\",\\\"name\\\":\\\"John Doe\\\",\\\"timezone\\\":\\\"+0200\\\"}\" > author_expected &&\n echo -n \"{\\\"/\\\":{\\\"bytes\\\":\\\"YmxvYiAxMgBIZWxsbyB3b3JsZAo\\\"}}\" > file1_expected &&\n echo -n \"{\\\"/\\\":{\\\"bytes\\\":\\\"YmxvYiA3ACcsLnB5Zgo\\\"}}\" > file2_expected &&\n ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/author > author_actual &&\n ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/tree/file/hash > file1_actual &&\n ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/parents/0/tree/dir2/hash/f3/hash > file2_actual\n '\n\n test_expect_success \"outputs look correct\" '\n test_cmp author_expected author_actual &&\n test_cmp file1_expected file1_actual &&\n test_cmp file2_expected file2_actual\n '\n}\n\n# should work offline\ntest_dag_git\n\n# should work online\ntest_launch_ipfs_daemon\ntest_dag_git\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0280-plugin-peerlog.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2017 Jakub Sztandera\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test peerlog plugin\"\n\n. lib/test-lib.sh\n\ntest_expect_success \"setup testbed\" '\n iptb testbed create -type localipfs -count 2 -force -init\n'\n\nstartup_cluster 2\n\ntest_expect_success \"peerlog is disabled by default\" '\n go-sleep 100ms\n iptb logs 0 >node0logs\n test_expect_code 1 grep peerlog node0logs\n'\n\ntest_expect_success 'stop iptb' 'iptb stop'\n\n\n\ntest_expect_success \"setup testbed\" '\n iptb testbed create -type localipfs -count 2 -force -init\n'\n\ntest_expect_success \"enable peerlog config setting\" '\n iptb run -- ipfs config --json Plugins.Plugins.peerlog.Config.Enabled true\n'\n\nstartup_cluster 2\n\ntest_expect_success \"peerlog plugin is logged\" '\n go-sleep 100ms\n iptb logs 0 >node0logs\n grep peerlog node0logs\n'\n\ntest_expect_success 'peer id' '\n PEERID_1=$(iptb attr get 1 id)\n'\n\ntest_expect_success \"peer id is logged\" '\n iptb logs 0 | grep -q \"$PEERID_1\"\n'\n\ntest_expect_success 'stop iptb' 'iptb stop'\n\ntest_done\n" }, { "path": "test/sharness/t0280-plugin.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2019 Protocol Labs\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test plugin loading\"\n\n. lib/test-lib.sh\n\nif ! test_have_prereq PLUGIN; then\n skip_all='skipping plugin tests, plugins not available'\n\n test_done\nfi\n\ntest_init_ipfs\n\ntest_expect_success \"ipfs id succeeds\" '\n ipfs id\n'\n\ntest_expect_success \"make a bad plugin\" '\n mkdir -p \"$IPFS_PATH/plugins\" &&\n echo foobar > \"$IPFS_PATH/plugins/foo.so\" &&\n chmod +x \"$IPFS_PATH/plugins/foo.so\"\n'\n\ntest_expect_success \"ipfs id fails due to a bad plugin\" '\n test_expect_code 1 ipfs id\n'\n\ntest_expect_success \"cleanup bad plugin\" '\n rm \"$IPFS_PATH/plugins/foo.so\"\n'\n\ntest_expect_success \"install test plugin\" '\n go build \\\n -asmflags=all=\"-trimpath=${GOPATH}\" -gcflags=all=\"-trimpath=${GOPATH}\" \\\n -buildmode=plugin -o \"$IPFS_PATH/plugins/example.so\" ../t0280-plugin-data/example.go &&\n chmod +x \"$IPFS_PATH/plugins/example.so\"\n'\n\ntest_plugin() {\n local loads=\"$1\"\n local repo=\"$2\"\n local config=\"$3\"\n\n rm -f id_raw_output id_output id_output_expected\n\n test_expect_success \"id runs\" '\n ipfs id 2>id_raw_output >/dev/null\n '\n\n test_expect_success \"filter test plugin output\" '\n sed -ne \"s/^testplugin //p\" id_raw_output >id_output\n '\n\n if [ \"$loads\" != \"true\" ]; then\n test_expect_success \"plugin doesn't load\" '\n test_must_be_empty id_output\n '\n else\n test_expect_success \"plugin produces the correct output\" '\n echo \"$repo\" >id_output_expected &&\n echo \"$config\" >>id_output_expected &&\n test_cmp id_output id_output_expected\n '\n fi\n}\n\ntest_plugin true \"$IPFS_PATH\" \"\"\n\ntest_expect_success \"disable the plugin\" '\n ipfs config --json Plugins.Plugins.test-plugin.Disabled true\n'\n\ntest_plugin false\n\ntest_expect_success \"re-enable the plugin\" '\n ipfs config --json Plugins.Plugins.test-plugin.Disabled false\n'\n\ntest_plugin true \"$IPFS_PATH\" \"\"\n\ntest_expect_success \"configure the plugin\" '\n ipfs config Plugins.Plugins.test-plugin.Config foobar\n'\n\ntest_plugin true \"$IPFS_PATH\" \"foobar\"\n\ntest_expect_success \"noplugin flag works\" '\n test_must_fail go run -tags=noplugin github.com/ipfs/go-ipfs/cmd/ipfs id > output 2>&1\n test_should_contain \"not built with plugin support\" output\n'\n\ntest_expect_success \"noplugin flag works\" '\n CGO_ENABLED=0 test_must_fail go run github.com/ipfs/go-ipfs/cmd/ipfs id > output 2>&1\n test_should_contain \"not built with cgo support\" output\n'\n\ntest_done\n" }, { "path": "test/sharness/t0290-cid.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test cid commands\"\n\n. lib/test-lib.sh\n\n# NOTE: Primary tests for \"ipfs cid\" commands are in test/cli/cid_test.go\n# These sharness tests are kept for backward compatibility but new tests\n# should be added to test/cli/cid_test.go instead. If any of these tests\n# break, consider removing them and updating only the test/cli version.\n\n# note: all \"ipfs cid\" commands should work without requiring a repo\n\nCIDv0=\"QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv\"\nCIDv1=\"zdj7WZAAFKPvYPPzyJLso2hhxo8a7ZACFQ4DvvfrNXTHidofr\"\nCIDb32=\"bafybeibxm2nsadl3fnxv2sxcxmxaco2jl53wpeorjdzidjwf5aqdg7wa6u\"\n\nCIDbase=\"QmYNmQKp6SuaVrpgWRsPTgCQCnpxUYGq76YEKBXuj2N4H6\"\nCIDb32pb=\"bafybeievd6mwe6vcwnkwo3eizs3h7w3a34opszbyfxziqdxguhjw7imdve\"\nCIDb32raw=\"bafkreievd6mwe6vcwnkwo3eizs3h7w3a34opszbyfxziqdxguhjw7imdve\"\nCIDb32dagcbor=\"bafyreievd6mwe6vcwnkwo3eizs3h7w3a34opszbyfxziqdxguhjw7imdve\"\n\ntest_expect_success \"cid base32 works\" '\n echo $CIDb32 > expected &&\n ipfs cid base32 $CIDv0 > actual1 &&\n test_cmp actual1 expected &&\n ipfs cid base32 $CIDv1 > actual2 &&\n test_cmp expected actual2\n'\n\ntest_expect_success \"cid format -v 1 -b base58btc\" '\n echo $CIDv1 > expected &&\n ipfs cid format -v 1 -b base58btc $CIDv0 > actual1 &&\n test_cmp actual1 expected &&\n ipfs cid format -v 1 -b base58btc $CIDb32 > actual2 &&\n test_cmp expected actual2\n'\n\ntest_expect_success \"cid format -v 0\" '\n echo $CIDv0 > expected &&\n ipfs cid format -v 0 $CIDb32 > actual &&\n test_cmp expected actual\n'\n\ncat < various_cids\nQmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo\n QmPhk6cJkRcFfZCdYam4c9MKYjFG9V29LswUnbrFNhtk2S\nbafybeihtwdtifv43rn5cyilnmkwofdcxi2suqimmo62vn3etf45gjoiuwy\nbafybeiek4tfxkc4ov6jsmb63fzbirrsalnjw24zd5xawo2fgxisd4jmpyq\nzdj7WgYfT2gfsgiUxzPYboaRbP9H9CxZE5jVMK9pDDwCcKDCR\nzdj7WbTaiJT1fgatdet9Ei9iDB5hdCxkbVyhyh8YTUnXMiwYi\nuAXASIDsp4T3Wnd6kXFOQaljH3GFK_ixkjMtVhB9VOBrPK3bp\n uAXASIDdmmyANeytvXUriuy4BO0lfd2eR0UjygabF6CAzfsD1\nEOF\n\ncat < various_cids_base32\nbafybeifgwyq5gs4l2mru5klgwjfmftjvkmbyyjurbupuz2bst7mhmg2hwa\nbafybeiauil46g3lb32jemjbl7yspca3twdcg4wwkbsgdgvgdj5fpfv2f64\nbafybeihtwdtifv43rn5cyilnmkwofdcxi2suqimmo62vn3etf45gjoiuwy\nbafybeiek4tfxkc4ov6jsmb63fzbirrsalnjw24zd5xawo2fgxisd4jmpyq\nbafybeifffq3aeaymxejo37sn5fyaf7nn7hkfmzwdxyjculx3lw4tyhk7uy\nbafybeiczsscdsbs7ffqz55asqdf3smv6klcw3gofszvwlyarci47bgf354\nbafybeib3fhqt3vu532sfyu4qnjmmpxdbjl7cyzemznkyih2vhanm6k3w5e\nbafybeibxm2nsadl3fnxv2sxcxmxaco2jl53wpeorjdzidjwf5aqdg7wa6u\nEOF\n\ncat < various_cids_v1\nzdj7WgefqQm5HogBQ2bckZuTYYDarRTUZi51GYCnerHD2G86j\nzdj7WWnzU3Nbu5rYGWZHKigUXBtAwShs2SHDCM1TQEvC9TeCN\nzdj7WmqAbpsfXgiRBtZP1oAP9QWuuY3mqbc5JhpxJkfT3vYCu\nzdj7Wen5gtfr7AivXip3zYd1peuq2QfKrqAn4FGiciVWb96YB\nzdj7WgYfT2gfsgiUxzPYboaRbP9H9CxZE5jVMK9pDDwCcKDCR\nzdj7WbTaiJT1fgatdet9Ei9iDB5hdCxkbVyhyh8YTUnXMiwYi\nzdj7WZQrAvnY5ge3FNg5cmCsNwsvpYjdtu2yEmnWYQ4ES7Nzk\nzdj7WZAAFKPvYPPzyJLso2hhxo8a7ZACFQ4DvvfrNXTHidofr\nEOF\n\ntest_expect_success \"cid base32 works from stdin\" '\n cat various_cids | ipfs cid base32 > actual &&\n test_cmp various_cids_base32 actual\n'\n\ntest_expect_success \"cid format -v 1 -b base58btc works from stdin\" '\n cat various_cids | ipfs cid format -v 1 -b base58btc > actual &&\n test_cmp various_cids_v1 actual\n'\n\ncat < bases_expect\n 0 identity\n0 48 base2\nb 98 base32\nB 66 base32upper\nc 99 base32pad\nC 67 base32padupper\nf 102 base16\nF 70 base16upper\nk 107 base36\nK 75 base36upper\nm 109 base64\nM 77 base64pad\nt 116 base32hexpad\nT 84 base32hexpadupper\nu 117 base64url\nU 85 base64urlpad\nv 118 base32hex\nV 86 base32hexupper\nz 122 base58btc\nZ 90 base58flickr\n🚀 128640 base256emoji\nEOF\n\ncat < codecs_expect\n 81 cbor\n 85 raw\n 112 dag-pb\n 113 dag-cbor\n 114 libp2p-key\n 120 git-raw\n 123 torrent-info\n 124 torrent-file\n 128 blake3-hashseq\n 129 leofcoin-block\n 130 leofcoin-tx\n 131 leofcoin-pr\n 133 dag-jose\n 134 dag-cose\n 144 eth-block\n 145 eth-block-list\n 146 eth-tx-trie\n 147 eth-tx\n 148 eth-tx-receipt-trie\n 149 eth-tx-receipt\n 150 eth-state-trie\n 151 eth-account-snapshot\n 152 eth-storage-trie\n 153 eth-receipt-log-trie\n 154 eth-receipt-log\n 176 bitcoin-block\n 177 bitcoin-tx\n 178 bitcoin-witness-commitment\n 192 zcash-block\n 193 zcash-tx\n 208 stellar-block\n 209 stellar-tx\n 224 decred-block\n 225 decred-tx\n 240 dash-block\n 241 dash-tx\n 250 swarm-manifest\n 251 swarm-feed\n 252 beeson\n 297 dag-json\n 496 swhid-1-snp\n 512 json\n46083 rdfc-1\n46593 json-jcs\nEOF\n\ncat < supported_codecs_expect\n 81 cbor\n 85 raw\n 112 dag-pb\n 113 dag-cbor\n 114 libp2p-key\n 120 git-raw\n 133 dag-jose\n 297 dag-json\n 512 json\nEOF\n\ncat < hashes_expect\n 0 identity\n 17 sha1\n 18 sha2-256\n 19 sha2-512\n 20 sha3-512\n 21 sha3-384\n 22 sha3-256\n 23 sha3-224\n 25 shake-256\n 26 keccak-224\n 27 keccak-256\n 28 keccak-384\n 29 keccak-512\n 30 blake3\n 86 dbl-sha2-256\n45588 blake2b-160\n45589 blake2b-168\n45590 blake2b-176\n45591 blake2b-184\n45592 blake2b-192\n45593 blake2b-200\n45594 blake2b-208\n45595 blake2b-216\n45596 blake2b-224\n45597 blake2b-232\n45598 blake2b-240\n45599 blake2b-248\n45600 blake2b-256\n45601 blake2b-264\n45602 blake2b-272\n45603 blake2b-280\n45604 blake2b-288\n45605 blake2b-296\n45606 blake2b-304\n45607 blake2b-312\n45608 blake2b-320\n45609 blake2b-328\n45610 blake2b-336\n45611 blake2b-344\n45612 blake2b-352\n45613 blake2b-360\n45614 blake2b-368\n45615 blake2b-376\n45616 blake2b-384\n45617 blake2b-392\n45618 blake2b-400\n45619 blake2b-408\n45620 blake2b-416\n45621 blake2b-424\n45622 blake2b-432\n45623 blake2b-440\n45624 blake2b-448\n45625 blake2b-456\n45626 blake2b-464\n45627 blake2b-472\n45628 blake2b-480\n45629 blake2b-488\n45630 blake2b-496\n45631 blake2b-504\n45632 blake2b-512\n45652 blake2s-160\n45653 blake2s-168\n45654 blake2s-176\n45655 blake2s-184\n45656 blake2s-192\n45657 blake2s-200\n45658 blake2s-208\n45659 blake2s-216\n45660 blake2s-224\n45661 blake2s-232\n45662 blake2s-240\n45663 blake2s-248\n45664 blake2s-256\nEOF\n\ntest_expect_success \"cid bases\" '\n cat <<-EOF > expect\n\tidentity\n\tbase2\n\tbase32\n\tbase32upper\n\tbase32pad\n\tbase32padupper\n\tbase16\n\tbase16upper\n\tbase36\n\tbase36upper\n\tbase64\n\tbase64pad\n\tbase32hexpad\n\tbase32hexpadupper\n\tbase64url\n\tbase64urlpad\n\tbase32hex\n\tbase32hexupper\n\tbase58btc\n\tbase58flickr\n\tbase256emoji\n\tEOF\n ipfs cid bases > actual &&\n test_cmp expect actual\n'\n\ntest_expect_success \"cid bases --prefix\" '\n cat <<-EOF > expect\n\t identity\n\t0 base2\n\tb base32\n\tB base32upper\n\tc base32pad\n\tC base32padupper\n\tf base16\n\tF base16upper\n\tk base36\n\tK base36upper\n\tm base64\n\tM base64pad\n\tt base32hexpad\n\tT base32hexpadupper\n\tu base64url\n\tU base64urlpad\n\tv base32hex\n\tV base32hexupper\n\tz base58btc\n\tZ base58flickr\n\t🚀 base256emoji\n\tEOF\n ipfs cid bases --prefix > actual &&\n test_cmp expect actual\n'\n\ntest_expect_success \"cid bases --prefix --numeric\" '\n ipfs cid bases --prefix --numeric > actual &&\n test_cmp bases_expect actual\n'\n\ntest_expect_success \"cid codecs\" '\n cut -c 8- codecs_expect > expect &&\n ipfs cid codecs > actual\n test_cmp expect actual\n'\n\ntest_expect_success \"cid codecs --numeric\" '\n ipfs cid codecs --numeric > actual &&\n test_cmp codecs_expect actual\n'\n\ntest_expect_success \"cid codecs --supported\" '\n cut -c 8- supported_codecs_expect > expect &&\n ipfs cid codecs --supported > actual\n test_cmp expect actual\n'\n\ntest_expect_success \"cid codecs --supported --numeric\" '\n ipfs cid codecs --supported --numeric > actual &&\n test_cmp supported_codecs_expect actual\n'\n\ntest_expect_success \"cid hashes\" '\n cut -c 8- hashes_expect > expect &&\n ipfs cid hashes > actual\n test_cmp expect actual\n'\n\ntest_expect_success \"cid hashes --numeric\" '\n ipfs cid hashes --numeric > actual &&\n test_cmp hashes_expect actual\n'\n\ntest_expect_success \"cid format -c raw\" '\n echo $CIDb32raw > expected &&\n ipfs cid format --mc raw -b base32 $CIDb32pb > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"cid format --mc dag-pb -v 0\" '\n echo $CIDbase > expected &&\n ipfs cid format --mc dag-pb -v 0 $CIDb32raw > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"cid format --mc dag-cbor\" '\n echo $CIDb32dagcbor > expected &&\n ipfs cid format --mc dag-cbor $CIDb32pb > actual &&\n test_cmp actual expected\n'\n\n# this was an old flag that we removed, explicitly to force an error\n# so the user would read about the new multicodec names introduced\n# by https://github.com/ipfs/go-cid/commit/b2064d74a8b098193b316689a715cdf4e4934805\ntest_expect_success \"cid format --codec fails\" '\n echo \"Error: unknown option \\\"codec\\\"\" > expected &&\n test_expect_code 1 ipfs cid format --codec protobuf 2> actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"cid format -b base256emoji \" '\n echo \"🚀🪐⭐💻😅❓💎🌈🌸🌚💰💍🌒😵🐶💁🤐🌎👼🙃🙅☺🌚😞🤤⭐🚀😃✈🌕😚🍻💜🐷⚽✌😊\" > expected &&\n ipfs cid format -b base256emoji bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"cid format -b base32 \" '\n echo \"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi\" > expected &&\n ipfs cid format -b base32 🚀🪐⭐💻😅❓💎🌈🌸🌚💰💍🌒😵🐶💁🤐🌎👼🙃🙅☺🌚😞🤤⭐🚀😃✈🌕😚🍻💜🐷⚽✌😊 > actual &&\n test_cmp actual expected\n'\n\n\ntest_done\n" }, { "path": "test/sharness/t0295-multibase.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test multibase commands\"\n\n. lib/test-lib.sh\n\n# note: all \"ipfs multibase\" commands should work without requiring a repo\n\ncat < bases_expect\n 0 identity\n0 48 base2\nb 98 base32\nB 66 base32upper\nc 99 base32pad\nC 67 base32padupper\nf 102 base16\nF 70 base16upper\nk 107 base36\nK 75 base36upper\nm 109 base64\nM 77 base64pad\nt 116 base32hexpad\nT 84 base32hexpadupper\nu 117 base64url\nU 85 base64urlpad\nv 118 base32hex\nV 86 base32hexupper\nz 122 base58btc\nZ 90 base58flickr\n 128640 base256emoji\nEOF\n\n# TODO: expose same cmd under multibase?\ntest_expect_success \"multibase list\" '\n cut -c 12- bases_expect > expect &&\n ipfs multibase list > actual &&\n test_cmp expect actual\n'\n\ntest_expect_success \"multibase encode works (stdin)\" '\n echo -n uaGVsbG8 > expected &&\n echo -n hello | ipfs multibase encode > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase encode works (file)\" '\n echo -n hello > file &&\n echo -n uaGVsbG8 > expected &&\n ipfs multibase encode ./file > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase encode -b (custom base)\" '\n echo -n f68656c6c6f > expected &&\n echo -n hello | ipfs multibase encode -b base16 > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase decode works (stdin)\" '\n echo -n hello > expected &&\n echo -n uaGVsbG8 | ipfs multibase decode > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase decode works (file)\" '\n echo -n uaGVsbG8 > file &&\n echo -n hello > expected &&\n ipfs multibase decode ./file > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase encode+decode roundtrip\" '\n echo -n hello > expected &&\n cat expected | ipfs multibase encode -b base64 | ipfs multibase decode > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"mutlibase transcode works (stdin)\" '\n echo -n f68656c6c6f > expected &&\n echo -n uaGVsbG8 | ipfs multibase transcode -b base16 > actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase transcode works (file)\" '\n echo -n uaGVsbG8 > file &&\n echo -n f68656c6c6f > expected &&\n ipfs multibase transcode ./file -b base16> actual &&\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase error on unknown multibase prefix\" '\n echo \"Error: failed to decode multibase: selected encoding not supported\" > expected &&\n echo -n ę-that-should-do-the-trick | ipfs multibase decode 2> actual ;\n test_cmp actual expected\n'\n\ntest_expect_success \"multibase error on a character outside of the base\" \"\n echo \\\"Error: failed to decode multibase: encoding/hex: invalid byte: U+007A 'z'\\\" > expected &&\n echo -n f6c6f6cz | ipfs multibase decode 2> actual ;\n test_cmp actual expected\n\"\n\ntest_done\n" }, { "path": "test/sharness/t0320-pubsub.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test pubsub command\"\n\n. lib/test-lib.sh\n\n# start iptb + wait for peering\nNUM_NODES=5\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count $NUM_NODES -init\n'\n\ntest_expect_success 'disable the DHT' '\n iptb run -- ipfs config Routing.Type none\n'\n\nrun_pubsub_tests() {\n test_expect_success 'peer ids' '\n PEERID_0=$(iptb attr get 0 id) &&\n PEERID_2=$(iptb attr get 2 id)\n '\n\n # ipfs pubsub sub\n test_expect_success 'pubsub' '\n echo -n -e \"test\\nOK\" | ipfs multibase encode -b base64url > expected &&\n touch empty &&\n mkfifo wait ||\n test_fsh echo init fail\n\n # ipfs pubsub sub is long-running so we need to start it in the background and\n # wait put its output somewhere where we can access it\n (\n ipfsi 0 pubsub sub --enc=json testTopic | if read line; then\n echo $line | jq -j .data > actual &&\n echo > wait\n fi\n ) &\n '\n\n test_expect_success \"wait until ipfs pubsub sub is ready to do work\" '\n go-sleep 500ms\n '\n\n test_expect_success \"can see peer subscribed to testTopic\" '\n ipfsi 1 pubsub peers testTopic > peers_out\n '\n\n test_expect_success \"output looks good\" '\n echo $PEERID_0 > peers_exp &&\n test_cmp peers_exp peers_out\n '\n\n test_expect_success \"publish something from file\" '\n echo -n -e \"test\\nOK\" > payload-file &&\n ipfsi 1 pubsub pub testTopic payload-file &> pubErr\n '\n\n test_expect_success \"wait until echo > wait executed\" '\n cat wait &&\n test_cmp pubErr empty &&\n test_cmp expected actual\n '\n\n test_expect_success \"wait for another pubsub message\" '\n echo -n -e \"test\\nOK\\r\\n2\" | ipfs multibase encode -b base64url > expected &&\n mkfifo wait2 ||\n test_fsh echo init fail\n\n # ipfs pubsub sub is long-running so we need to start it in the background and\n # wait put its output somewhere where we can access it\n (\n ipfsi 2 pubsub sub --enc=json testTopic | if read line; then\n echo $line | jq -j .data > actual &&\n echo > wait2\n fi\n ) &\n '\n\n test_expect_success \"wait until ipfs pubsub sub is ready to do work\" '\n go-sleep 500ms\n '\n\n test_expect_success \"publish something from stdin\" '\n echo -n -e \"test\\nOK\\r\\n2\" | ipfsi 3 pubsub pub testTopic &> pubErr\n '\n\n test_expect_success \"wait until echo > wait executed\" '\n cat wait2 &&\n test_cmp pubErr empty &&\n test_cmp expected actual\n '\n\n test_expect_success 'cleanup fifos' '\n rm -f wait wait2\n '\n\n}\n\n# Normal tests - enabled via config\n\ntest_expect_success 'enable the pubsub' '\n iptb run -- ipfs config --json Pubsub.Enabled true\n'\n\nstartup_cluster $NUM_NODES\nrun_pubsub_tests\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_expect_success 'disable the pubsub' '\n iptb run -- ipfs config --json Pubsub.Enabled false\n'\n\n# Normal tests - enabled via daemon option flag\n\nstartup_cluster $NUM_NODES --enable-pubsub-experiment\nrun_pubsub_tests\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\n# Test with some nodes not signing messages.\n\ntest_expect_success 'disable signing on nodes 1-3' '\n iptb run [0-3] -- ipfs config --json Pubsub.DisableSigning true\n'\n\nstartup_cluster $NUM_NODES --enable-pubsub-experiment\n\ntest_expect_success 'set node 4 to listen on testTopic' '\n rm -f node4_actual &&\n ipfsi 4 pubsub sub --enc=json testTopic > node4_actual &\n'\n\nrun_pubsub_tests\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_expect_success 'node 4 got no unsigned messages' '\n test_must_be_empty node4_actual\n'\n\n\n# Confirm negative CLI flag takes precedence over positive config\n\n# --enable-pubsub-experiment=false + Pubsub.Enabled:true\n\ntest_expect_success 'enable the pubsub via config' '\n iptb run -- ipfs config --json Pubsub.Enabled true\n'\nstartup_cluster $NUM_NODES --enable-pubsub-experiment=false\n\ntest_expect_success 'pubsub cmd fails because it was disabled via cli flag' '\n test_expect_code 1 ipfsi 4 pubsub ls 2> pubsub_cmd_out\n'\n\ntest_expect_success \"pubsub cmd produces error\" '\n echo \"Error: experimental pubsub feature not enabled, run daemon with --enable-pubsub-experiment to use\" > expected &&\n test_cmp expected pubsub_cmd_out\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0321-pubsub-gossipsub.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test pubsub with gossipsub\"\n\n. lib/test-lib.sh\n\n# start iptb + wait for peering\nNUM_NODES=5\ntest_expect_success 'init iptb' '\n iptb testbed create -type localipfs -count $NUM_NODES -init\n'\n\ntest_expect_success \"enable gossipsub\" '\n for x in $(seq 0 4); do\n ipfsi $x config Pubsub.Router gossipsub\n done\n'\n\n# this is just a copy of t0180-pubsub; smell.\nstartup_cluster $NUM_NODES --enable-pubsub-experiment\n\ntest_expect_success 'peer ids' '\n PEERID_0=$(iptb attr get 0 id) &&\n PEERID_2=$(iptb attr get 2 id)\n'\n\ntest_expect_success 'pubsub' '\n echo -n -e \"test\\nOK\" | ipfs multibase encode -b base64url > expected &&\n touch empty &&\n mkfifo wait ||\n test_fsh echo init fail\n\n # ipfs pubsub sub is long-running so we need to start it in the background and\n # wait put its output somewhere where we can access it\n (\n ipfsi 0 pubsub sub --enc=json testTopic | if read line; then\n echo $line | jq -j .data > actual &&\n echo > wait\n fi\n ) &\n'\n\ntest_expect_success \"wait until ipfs pubsub sub is ready to do work\" '\n go-sleep 500ms\n'\n\ntest_expect_success \"can see peer subscribed to testTopic\" '\n ipfsi 1 pubsub peers testTopic > peers_out\n'\n\ntest_expect_success \"output looks good\" '\n echo $PEERID_0 > peers_exp &&\n test_cmp peers_exp peers_out\n'\n\ntest_expect_success \"publish something from a file\" '\n echo -n -e \"test\\nOK\" > payload-file &&\n ipfsi 1 pubsub pub testTopic payload-file &> pubErr\n'\n\ntest_expect_success \"wait until echo > wait executed\" '\n cat wait &&\n test_cmp pubErr empty &&\n test_cmp expected actual\n'\n\ntest_expect_success \"wait for another pubsub message\" '\n echo -n -e \"test\\nOK2\" | ipfs multibase encode -b base64url > expected &&\n mkfifo wait2 ||\n test_fsh echo init fail\n\n # ipfs pubsub sub is long-running so we need to start it in the background and\n # wait put its output somewhere where we can access it\n (\n ipfsi 2 pubsub sub --enc=json testTopic | if read line; then\n echo $line | jq -j .data > actual &&\n echo > wait2\n fi\n ) &\n'\n\ntest_expect_success \"wait until ipfs pubsub sub is ready to do work\" '\n go-sleep 500ms\n'\n\ntest_expect_success \"publish something\" '\n echo -n -e \"test\\nOK2\" | ipfsi 1 pubsub pub testTopic &> pubErr\n'\n\ntest_expect_success \"wait until echo > wait executed\" '\n cat wait2 &&\n test_cmp pubErr empty &&\n test_cmp expected actual\n'\n\ntest_expect_success 'stop iptb' '\n iptb stop\n'\n\ntest_done\n" }, { "path": "test/sharness/t0322-pubsub-http-rpc.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Test pubsub command behavior over HTTP RPC API\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\ntest_launch_ipfs_daemon --enable-pubsub-experiment\n\n# Require topic as multibase\n# https://github.com/ipfs/go-ipfs/pull/8183\ntest_expect_success \"/api/v0/pubsub/pub URL arg must be multibase encoded\" '\n echo test > data.txt &&\n curl -s -X POST -F \"data=@data.txt\" \"$API_ADDR/api/v0/pubsub/pub?arg=foobar\" > result &&\n test_should_contain \"error\" result &&\n test_should_contain \"URL arg must be multibase encoded\" result\n'\n\n# Use URL-safe multibase\n# base64 should produce error when used in URL args, base64url should be used\ntest_expect_success \"/api/v0/pubsub/pub URL arg must be in URL-safe multibase\" '\n echo test > data.txt &&\n curl -s -X POST -F \"data=@data.txt\" \"$API_ADDR/api/v0/pubsub/pub?arg=mZm9vYmFyCg\" > result &&\n test_should_contain \"error\" result &&\n test_should_contain \"URL arg must be base64url encoded\" result\n'\n\ntest_kill_ipfs_daemon\ntest_done\n" }, { "path": "test/sharness/t0400-api-no-gateway/README.md", "content": "# Dataset description/sources\n\n- fixtures.car\n - raw CARv1\n\ngenerated with:\n\n```sh\n# using ipfs version 0.18.1\nHASH=$(echo \"testing\" | ipfs add -q)\nipfs dag export $HASH > fixtures.car\n\necho HASH=${HASH} # a file containing the string \"testing\"\n\n# HASH=QmNYERzV2LfD2kkfahtfv44ocHzEFK1sLBaE7zdcYT2GAZ # a file containing the string \"testing\"\n```\n" }, { "path": "test/sharness/t0400-api-no-gateway.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Lars Gierth\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test API security\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# Import test case\n# See the static fixtures in ./t0400-api-no-gateway/\ntest_expect_success \"Add the test directory\" '\n ipfs dag import ../t0400-api-no-gateway/fixtures.car\n'\nHASH=QmNYERzV2LfD2kkfahtfv44ocHzEFK1sLBaE7zdcYT2GAZ # a file containing the string \"testing\"\n\n# by default, we don't let you load arbitrary ipfs objects through the api,\n# because this would open up the api to scripting vulnerabilities.\n# only the webui objects are allowed.\n# if you know what you're doing, go ahead and pass --unrestricted-api.\n\ntest_launch_ipfs_daemon\ntest_expect_success \"Gateway on API unavailable\" '\n test_curl_resp_http_code \"http://127.0.0.1:$API_PORT/ipfs/$HASH\" \"HTTP/1.1 404 Not Found\"\n'\ntest_kill_ipfs_daemon\n\ntest_launch_ipfs_daemon --unrestricted-api\ntest_expect_success \"Gateway on --unrestricted-api API available\" '\n test_curl_resp_http_code \"http://127.0.0.1:$API_PORT/ipfs/$HASH\" \"HTTP/1.1 200 OK\"\n'\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0401-api-browser-security.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2020 Protocol Labs\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test API browser security\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\nPEERID=$(ipfs config Identity.PeerID)\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"browser is unable to access API without Origin\" '\n curl -sD - -X POST -A \"Mozilla\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 403 Forbidden\" curl_output\n'\n\ntest_expect_success \"browser is unable to access API with invalid Origin\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: https://invalid.example.com\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 403 Forbidden\" curl_output\n'\n\ntest_expect_success \"browser is able to access API if Origin is the API port on localhost (ipv4)\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: http://127.0.0.1:$API_PORT\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output && grep \"$PEERID\" curl_output\n'\n\ntest_expect_success \"browser is able to access API if Origin is the API port on localhost (ipv6)\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: http://[::1]:$API_PORT\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output && grep \"$PEERID\" curl_output\n'\n\ntest_expect_success \"browser is able to access API if Origin is the API port on localhost (localhost name)\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: http://localhost:$API_PORT\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output && grep \"$PEERID\" curl_output\n'\n\ntest_expect_success \"Random browser extension is unable to access RPC API due to invalid Origin\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: chrome-extension://invalidextensionid\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 403 Forbidden\" curl_output\n'\n\ntest_expect_success \"Companion extension is able to access RPC API on localhost\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: chrome-extension://nibjojkomfdiaoajekhjakgkdhaomnch\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n cat curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output && grep \"$PEERID\" curl_output\n'\n\ntest_expect_success \"Companion beta extension is able to access API on localhost\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: chrome-extension://hjoieblefckbooibpepigmacodalfndh\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output && grep \"$PEERID\" curl_output\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"setting CORS in API.HTTPHeaders works via CLI\" \"\n ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '[\\\"https://valid.example.com\\\"]' &&\n ipfs config --json API.HTTPHeaders.Access-Control-Allow-Methods '[\\\"POST\\\"]' &&\n ipfs config --json API.HTTPHeaders.Access-Control-Allow-Headers '[\\\"X-Requested-With\\\"]'\n\"\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"Companion extension is able to access RPC API even when custom Access-Control-Allow-Origin is set\" '\n ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin | grep -q valid.example.com &&\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: chrome-extension://nibjojkomfdiaoajekhjakgkdhaomnch\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n cat curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output &&\n grep \"$PEERID\" curl_output\n'\n\n# https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request\ntest_expect_success \"OPTIONS with preflight request to API with CORS allowlist succeeds\" '\n curl -svX OPTIONS -A \"Mozilla\" -H \"Origin: https://valid.example.com\" -H \"Access-Control-Request-Method: POST\" -H \"Access-Control-Request-Headers: origin, x-requested-with\" \"http://127.0.0.1:$API_PORT/api/v0/id\" 2>curl_output &&\n cat curl_output\n'\n\n# OPTION Response from Gateway should contain CORS headers, otherwise JS won't work\ntest_expect_success \"OPTIONS response for API with CORS allowslist looks good\" '\n grep \"< Access-Control-Allow-Origin: https://valid.example.com\" curl_output\n'\n\ntest_expect_success \"browser is able to access API with valid Origin matching CORS allowlist\" '\n curl -sD - -X POST -A \"Mozilla\" -H \"Origin: https://valid.example.com\" \"http://127.0.0.1:$API_PORT/api/v0/id\" >curl_output &&\n grep \"HTTP/1.1 200 OK\" curl_output && grep \"$PEERID\" curl_output\n'\n\ntest_kill_ipfs_daemon\ntest_done\n" }, { "path": "test/sharness/t0410-api-add.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Tom O'Donnell\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test API add command\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# Verify that the API add command returns size\n\ntest_launch_ipfs_daemon\ntest_expect_success \"API Add response includes size field\" '\n echo \"hi\" | curl -s -F file=@- \"http://localhost:$API_PORT/api/v0/add\" | grep \"\\\"Size\\\": *\\\"11\\\"\"\n'\ntest_kill_ipfs_daemon\n\ntest_done\n" }, { "path": "test/sharness/t0500-issues-and-regressions-offline.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Tests for various fixed issues and regressions.\"\n\n. lib/test-lib.sh\n\n# Tests go here\n\ntest_expect_success \"ipfs init with occupied input works - #2748\" '\n export IPFS_PATH=\"ipfs_path\"\n echo \"\" | go-timeout 10 ipfs init &&\n rm -rf ipfs_path\n'\ntest_init_ipfs\n\ntest_expect_success \"ipfs cat --help succeeds when input remains open\" '\n yes | go-timeout 1 ipfs cat --help\n'\n\ntest_expect_success \"ipfs pin ls --help succeeds when input remains open\" '\n yes | go-timeout 1 ipfs pin ls --help\n'\n\ntest_expect_success \"ipfs add on 1MB from stdin woks\" '\n random-data -size=1048576 -seed=42 | ipfs add -q > 1MB.hash\n'\n\ntest_expect_success \"'ipfs refs -r -e \\$(cat 1MB.hash)' succeeds\" '\n ipfs refs -r -e $(cat 1MB.hash) > refs-e.out\n'\n\ntest_expect_success \"output of 'ipfs refs -e' links to separate blocks\" '\n grep \"$(cat 1MB.hash) ->\" refs-e.out\n'\n\ntest_expect_success \"output of 'ipfs refs -e' contains all first level links\" '\n grep \"$(cat 1MB.hash) ->\" refs-e.out | sed -e '\\''s/.* -> //'\\'' | sort > refs-s.out &&\n ipfs refs \"$(cat 1MB.hash)\" | sort > refs-one.out &&\n test_cmp refs-s.out refs-one.out\n'\n\ntest_done\n" }, { "path": "test/sharness/t0600-issues-and-regressions-online.sh", "content": "#!/usr/bin/env bash\n\ntest_description=\"Tests for various fixed issues and regressions.\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs --empty-repo=false\n\ntest_launch_ipfs_daemon\n\n# Tests go here\n\ntest_expect_success \"commands command with flag flags works via HTTP API - #2301\" '\n curl -X POST \"http://$API_ADDR/api/v0/commands?flags\" | grep \"verbose\"\n'\n\ntest_expect_success \"ipfs refs local over HTTP API returns NDJOSN not flat - #2803\" '\n echo \"Hello World\" | ipfs add &&\n curl -X POST \"http://$API_ADDR/api/v0/refs/local\" | grep \"Ref\" | grep \"Err\"\n'\n\ntest_expect_success \"args expecting stdin don't crash when not given\" '\n curl -X POST \"$API_ADDR/api/v0/bootstrap/add\" > result\n'\n\ntest_expect_success \"no panic traces on daemon\" '\n test_must_fail grep \"nil pointer dereference\" daemon_err\n'\n\ntest_expect_success \"metrics work\" '\n curl -X POST \"$API_ADDR/debug/metrics/prometheus\" > pro_data &&\n grep \"ipfs_bs_cache_boxo_blockstore_cache_total\" < pro_data ||\n test_fsh cat pro_data\n'\n\ntest_expect_success \"pin add api looks right - #3753\" '\n HASH=$(date +\"%FT%T.%N%z\" | ipfs add -q) &&\n curl -X POST \"http://$API_ADDR/api/v0/pin/add/$HASH\" > pinadd_out &&\n echo \"{\\\"Pins\\\":[\\\"$HASH\\\"]}\" > pinadd_exp &&\n test_cmp pinadd_out pinadd_exp\n'\n\ntest_expect_success \"pin add api looks right - #3753\" '\n curl -X POST \"http://$API_ADDR/api/v0/pin/rm/$HASH\" > pinrm_out &&\n echo \"{\\\"Pins\\\":[\\\"$HASH\\\"]}\" > pinrm_exp &&\n test_cmp pinrm_out pinrm_exp\n'\n\ntest_expect_success SOCAT \"no daemon crash on improper file argument - #4003 ( test needs socat )\" '\n FNC=$(echo $API_ADDR | awk -F: '\\''{ printf \"%s:%s\", $1, $2 }'\\'') &&\n printf \"POST /api/v0/add?pin=true HTTP/1.1\\r\\nHost: $API_ADDR\\r\\nContent-Type: multipart/form-data; boundary=Pyw9xQLtiLPE6XcI\\r\\nContent-Length: 22\\r\\n\\r\\n\\r\\n--Pyw9xQLtiLPE6XcI\\r\\n\" | socat STDIO tcp-connect:$FNC | grep -m1 \"500 Internal Server Error\"\n'\n\ntest_kill_ipfs_daemon\n\ntest_expect_success \"ipfs daemon --offline --mount fails - #2995\" '\n test_expect_code 1 ipfs daemon --offline --mount 2>daemon_err &&\n grep \"mount is not currently supported in offline mode\" daemon_err ||\n test_fsh cat daemon_err\n'\n\ntest_launch_ipfs_daemon_without_network\n\ntest_expect_success \"'ipfs name resolve' succeeds after ipfs id when daemon offline\" '\n PEERID=`ipfs key list --ipns-base=base36 -l | grep self | cut -d \" \" -f1` &&\n test_check_peerid \"${PEERID}\" &&\n ipfs name publish --allow-offline -Q \"/ipfs/$HASH_WELCOME_DOCS\" >publish_out\n'\n\ntest_expect_success \"pubrmlish --quieter output looks good\" '\n echo \"${PEERID}\" >expected1 &&\n test_cmp expected1 publish_out\n'\n\ntest_expect_success \"'ipfs name resolve' succeeds\" '\n ipfs name resolve \"$PEERID\" >output\n'\n\ntest_expect_success \"resolve output looks good\" '\n printf \"/ipfs/%s\\n\" \"$HASH_WELCOME_DOCS\" >expected2 &&\n test_cmp expected2 output\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n\n" }, { "path": "test/sharness/t0701-delegated-routing-reframe/FindProvidersRequest", "content": "{\"FindProvidersRequest\":{\"Key\":{\"/\":\"bafybeigvgzoolc3drupxhlevdp2ugqcrbcsqfmcek2zxiw5wctk3xjpjwy\"}}}\n" }, { "path": "test/sharness/t0701-delegated-routing-reframe/FindProvidersResponse", "content": "{\"FindProvidersResponse\":{\"Providers\":[{\"Node\":{\"peer\":{\"ID\":{\"/\":{\"bytes\":\"EiAngCqwSSL46hQ5+DWaJsZ1SPV2RwrqwID/OEuj5Rdgqw\"}},\"Multiaddresses\":[{\"/\":{\"bytes\":\"NiJwZWVyLmlwZnMtZWxhc3RpYy1wcm92aWRlci1hd3MuY29tBgu43QM\"}}]}},\"Proto\":[{\"2304\":{}}]}]}}\n" }, { "path": "test/sharness/t0702-delegated-routing-http/FindProvidersResponse", "content": "{\"Providers\":[{\"Protocol\":\"transport-bitswap\",\"Schema\":\"bitswap\",\"ID\":\"12D3KooWARYacCc6eoCqvsS9RW9MA2vo51CV75deoiqssx3YgyYJ\",\"Addrs\":[\"/ip4/0.0.0.0/tcp/4001\",\"/ip4/0.0.0.0/tcp/4002\"]}]}" }, { "path": "test/sharness/t0800-blake3.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2020 Claudia Richoux\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test blake3 mhash support\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\n# the blake3 hash of \"foo\\n\" in UTF8 (which is what comes out of echo when you pipe into `ipfs`) starts with \"49dc870df1de7fd60794cebce449f5ccdae575affaa67a24b62acb03e039db92\"\n# without the newline it's \"04e0bb39f30b1a3feb89f536c93be15055482df748674b00d26e5a75777702e9\". so if you start seeing these values that's your problem\nBLAKE3RAWCID32BYTE=\"bafkr4icj3sdq34o6p7lapfgoxtset5om3lsxll72uz5cjnrkzmb6aoo3si\"\nBLAKE3RAWCID64BYTE=\"bafkr4qcj3sdq34o6p7lapfgoxtset5om3lsxll72uz5cjnrkzmb6aoo3sknmbprpe27pbfrb67tonydgfot5ixuq4skiva76ppbgpjlzc4ua4\"\nBLAKE3RAWCID128BYTE=\"bafkr5aabjhoiodpr3z75mb4uz26oispvztnok5np7kthujfwflfqhybz3ojjvqf6f4tl54eweh36nzxamyv2pvc6sdsjjcud7z54ez5fpelsqdtax2k3rvuq3wdl5a4blxv3gvsroa3nakfzzknamhu2apf3vvytyiobabrn2bfnfajq66ikjy5lewsp5jyddsg5l7u3emr2ancimryay\"\n\n### block tests, including for various sizes of hash ###\n\ntest_expect_success \"putting a block with an mhash blake3 succeeds (default 32 bytes)\" '\n HASH=$(echo \"foo\" | ipfs block put --mhtype=blake3 --cid-codec=raw | tee actual_out) &&\n test $BLAKE3RAWCID32BYTE = \"$HASH\"\n'\n\ntest_expect_success \"block get output looks right\" '\n ipfs block get $BLAKE3RAWCID32BYTE > blk_get_out &&\n echo \"foo\" > blk_get_exp &&\n test_cmp blk_get_exp blk_get_out\n'\n\ntest_expect_success \"putting a block with an mhash blake3 succeeds: 64 bytes\" '\n HASH=$(echo \"foo\" | ipfs block put --mhtype=blake3 --mhlen=64 --cid-codec=raw | tee actual_out) &&\n test $BLAKE3RAWCID64BYTE = \"$HASH\"\n'\n\ntest_expect_success \"64B block get output looks right\" '\n ipfs block get $BLAKE3RAWCID64BYTE > blk_get_out &&\n echo \"foo\" > blk_get_exp &&\n test_cmp blk_get_exp blk_get_out\n'\n\ntest_expect_success \"putting a block with an mhash blake3 succeeds: 128 bytes\" '\n HASH=$(echo \"foo\" | ipfs block put --mhtype=blake3 --mhlen=128 --cid-codec=raw | tee actual_out) &&\n test $BLAKE3RAWCID128BYTE = \"$HASH\"\n'\n\ntest_expect_success \"32B block get output looks right\" '\n ipfs block get $BLAKE3RAWCID128BYTE > blk_get_out &&\n echo \"foo\" > blk_get_exp &&\n test_cmp blk_get_exp blk_get_out\n'\n\n### dag tests ###\n\ntest_expect_success \"dag put works with blake3\" '\n HASH=$(echo \"foo\" | ipfs dag put --input-codec=raw --store-codec=raw --hash=blake3 | tee actual_out) &&\n test $BLAKE3RAWCID32BYTE = \"$HASH\"\n'\n\ntest_expect_success \"dag get output looks right\" '\n ipfs dag get --output-codec=raw $BLAKE3RAWCID32BYTE > dag_get_out &&\n echo \"foo\" > dag_get_exp &&\n test_cmp dag_get_exp dag_get_out\n'\n\n### add and cat tests ###\n\ntest_expect_success \"adding a file with just foo in it to ipfs\" '\n echo \"foo\" > afile &&\n HASH=$(ipfs add -q --hash=blake3 --raw-leaves afile | tee actual_out) &&\n test $BLAKE3RAWCID32BYTE = \"$HASH\"\n'\n\ntest_expect_success \"catting it\" '\n ipfs cat $BLAKE3RAWCID32BYTE > cat_out &&\n echo \"foo\" > cat_exp &&\n test_cmp cat_exp cat_out\n'\n\ntest_done\n" }, { "path": "test/sharness/x0601-pin-fail-test.sh", "content": "#!/usr/bin/env bash\n#\n# Copyright (c) 2016 Jeromy Johnson\n# MIT Licensed; see the LICENSE file in this repository.\n#\n\ntest_description=\"Test very large number of pins\"\n\n. lib/test-lib.sh\n\ntest_init_ipfs\n\ntest_launch_ipfs_daemon\n\ntest_expect_success \"pre-test setup\" '\n printf \"\" > pins &&\n ipfs pin ls --type=recursive -q > rec_pins_before\n'\n\n\nfor i in `seq 9000`\ndo\n test_expect_success \"ipfs add (and pin) a file\" '\n echo $i | ipfs add -q >> pins\n '\ndone\n\ntest_expect_success \"get pinset afterwards\" '\n ipfs pin ls --type=recursive -q | sort > rec_pins_after &&\n cat pins rec_pins_before | sort | uniq > exp_pins_after &&\n test_cmp rec_pins_after exp_pins_after\n'\n\ntest_kill_ipfs_daemon\n\ntest_done\n\n" }, { "path": "test/sharness_test_coverage_helper.sh", "content": "#!/bin/sh\n\nUSAGE=\"$0 [-h] [-v]\"\n\nusage() {\n echo \"$USAGE\"\n echo \"\tPrint sharness test coverage\"\n echo \"\tOptions:\"\n echo \"\t\t-h|--help: print this usage message and exit\"\n echo \"\t\t-v|--verbose: print logs of what happens\"\n exit 0\n}\n\nlog() {\n test -z \"$VERBOSE\" || echo \"->\" \"$@\"\n}\n\ndie() {\n printf >&2 \"fatal: %s\\n\" \"$@\"\n exit 1\n}\n\n# get user options\nwhile [ \"$#\" -gt \"0\" ]; do\n # get options\n arg=\"$1\"\n shift\n\n case \"$arg\" in\n\t-h|--help)\n\t usage ;;\n\t-v|--verbose)\n\t VERBOSE=1 ;;\n\t-*)\n\t die \"unrecognised option: '$arg'\\n$USAGE\" ;;\n\t*)\n\t die \"too many arguments\\n$USAGE\" ;;\n esac\ndone\n\nlog \"Create temporary directory\"\nDATE=$(date +\"%Y-%m-%dT%H:%M:%SZ\")\nTMPDIR=$(mktemp -d \"/tmp/coverage_helper.$DATE.XXXXXX\") ||\ndie \"could not 'mktemp -d /tmp/coverage_helper.$DATE.XXXXXX'\"\n\nlog \"Grep the sharness tests for ipfs commands\"\nCMD_RAW=\"$TMPDIR/ipfs_cmd_raw.txt\"\ngit grep -n -E '\\Wipfs\\W' -- sharness/t*-*.sh >\"$CMD_RAW\" ||\ndie \"Could not grep ipfs in the sharness tests\"\n\ngrep_out() {\n pattern=\"$1\"\n src=\"$TMPDIR/ipfs_cmd_${2}.txt\"\n dst=\"$TMPDIR/ipfs_cmd_${3}.txt\"\n desc=\"$4\"\n\n log \"Remove $desc\"\n egrep -v \"$pattern\" \"$src\" >\"$dst\" || die \"Could not remove $desc\"\n}\n\ngrep_out 'test_expect_.*ipfs' raw expect \"test_expect_{success,failure} lines\"\ngrep_out '^[^:]+:[^:]+:\\s*#' expect comment \"comments\"\ngrep_out 'test_description=' comment desc \"test_description lines\"\ngrep_out '^[^:]+:[^:]+:\\s*\\w+=\"[^\"]*\"\\s*(\\&\\&)?\\s*$' desc def \"variable definition lines\"\ngrep_out '^[^:]+:[^:]+:\\s*e?grep\\W[^|]*\\Wipfs' def grep \"grep lines\"\ngrep_out '^[^:]+:[^:]+:\\s*cat\\W[^|]*\\Wipfs' grep cat \"cat lines\"\ngrep_out '^[^:]+:[^:]+:\\s*rmdir\\W[^|]*\\Wipfs' cat rmdir \"rmdir lines\"\ngrep_out '^[^:]+:[^:]+:\\s*echo\\W[^|]*\\Wipfs' cat echo \"echo lines\"\n\ngrep_in() {\n pattern=\"$1\"\n src=\"$TMPDIR/ipfs_cmd_${2}.txt\"\n dst=\"$TMPDIR/ipfs_cmd_${3}.txt\"\n desc=\"$4\"\n\n log \"Keep $desc\"\n egrep \"$pattern\" \"$src\" >\"$dst\"\n}\n\ngrep_in '\\Wipfs\\W.*/ipfs/' echo slash_in1 \"ipfs.*/ipfs/\"\ngrep_in '/ipfs/.*\\Wipfs\\W' echo slash_in2 \"/ipfs/.*ipfs\"\n\ngrep_out '/ipfs/' echo slash \"/ipfs/\"\n\ngrep_in '\\Wipfs\\W.*\\.ipfs' slash dot_in1 \"ipfs.*\\.ipfs\"\ngrep_in '\\.ipfs.*\\Wipfs\\W' slash dot_in2 \"\\.ipfs.*ipfs\"\n\ngrep_out '\\.ipfs' slash dot \".ipfs\"\n\nlog \"Print result\"\nCMD_RES=\"$TMPDIR/ipfs_cmd_result.txt\"\nfor f in dot slash_in1 slash_in2 dot_in1 dot_in2\ndo\n fname=\"$TMPDIR/ipfs_cmd_${f}.txt\"\n cat \"$fname\" || die \"Could not cat '$fname'\"\ndone | sort | uniq >\"$CMD_RES\" || die \"Could not write '$CMD_RES'\"\n\nlog \"Get all the ipfs commands from 'ipfs commands'\"\nCMD_CMDS=\"$TMPDIR/commands.txt\"\nipfs commands --flags >\"$CMD_CMDS\" || die \"'ipfs commands' failed\"\n\n# Portable function to reverse lines in a file\nreverse() {\n if type tac >/dev/null\n then\n\ttac \"$@\"\n else\n\ttail -r \"$@\"\n fi\n}\n\nlog \"Match the test line commands with the commands they use\"\nGLOBAL_REV=\"$TMPDIR/global_results_reversed.txt\"\n\nprocess_command() {\n ipfs=\"$1\"\n cmd=\"$2\"\n sub1=\"$3\"\n sub2=\"$4\"\n sub3=\"$5\"\n\n if test -n \"$cmd\"\n then\n\tCMD_OUT=\"$TMPDIR/res_${ipfs}_${cmd}\"\n\tPATTERN=\"$ipfs(\\W.*)*\\W$cmd\"\n\tNAME=\"$ipfs $cmd\"\n\n\tif test -n \"$sub1\"\n\tthen\n\t CMD_OUT=\"${CMD_OUT}_${sub1}\"\n\t PATTERN=\"$PATTERN(\\W.*)*\\W$sub1\"\n\t NAME=\"$NAME $sub1\"\n\n\t if test -n \"$sub2\"\n\t then\n\t\tCMD_OUT=\"${CMD_OUT}_${sub2}\"\n\t\tPATTERN=\"$PATTERN(\\W.*)*\\W$sub2\"\n\t\tNAME=\"$NAME $sub2\"\n\n\t\tif test -n \"$sub3\"\n\t\tthen\n\t\t CMD_OUT=\"${CMD_OUT}_${sub3}\"\n\t\t PATTERN=\"$PATTERN(\\W.*)*\\W$sub3\"\n\t\t NAME=\"$NAME $sub3\"\n\t\tfi\n\t fi\n\tfi\n\n\tegrep \"$PATTERN\" \"$CMD_RES\" >\"$CMD_OUT.txt\"\n\treverse \"$CMD_OUT.txt\" | sed -e 's/^sharness\\///' | cut -d- -f1 | uniq -c >>\"$GLOBAL_REV\"\n fi\n}\n\nreverse \"$CMD_CMDS\" | while read -r line\ndo\n LONG_CMD=$(echo \"$line\" | cut -d/ -f1)\n SHORT_CMD=$(expr \"$line\" : \"[^/]*/*\\(.*\\)\")\n\n log \"Processing $LONG_CMD\"\n process_command $LONG_CMD\n LONG_NAME=\"$NAME\"\n\n log \"Processing $SHORT_CMD\"\n process_command $SHORT_CMD\n SHORT_NAME=\"$NAME\"\n\n test -n \"$SHORT_CMD\" && echo \"$SHORT_NAME\" >>\"$GLOBAL_REV\"\n test \"$LONG_CMD\" != \"ipfs\" && echo \"$LONG_NAME\" >>\"$GLOBAL_REV\"\n echo >>\"$GLOBAL_REV\"\ndone\n\n# The following will allow us to check that\n# we are properly excluding enough stuff using:\n# diff -u ipfs_cmd_result.txt cmd_found.txt\nlog \"Get all the line commands that matched\"\nCMD_FOUND=\"$TMPDIR/cmd_found.txt\"\ncat $TMPDIR/res_*.txt | sort -n | uniq >\"$CMD_FOUND\"\n\nlog \"Print results\"\nreverse \"$GLOBAL_REV\"\n\n# Remove temp directory...\n" }, { "path": "test/unit/.gitignore", "content": "gotest.json\ngotest.junit.xml\n" }, { "path": "test/unit/Rules.mk", "content": "include mk/header.mk\n\nCLEAN += $(d)/gotest.json $(d)/gotest.junit.xml\n\n# Convert gotest.json (produced by test_unit) to JUnit XML format\n$(d)/gotest.junit.xml: test/bin/gotestsum $(d)/gotest.json\n\tgotestsum --no-color --junitfile $@ --raw-command cat $(@D)/gotest.json\n\ninclude mk/footer.mk\n" }, { "path": "thirdparty/README.md", "content": "packages under this directory _must not_ import packages under\n`ipfs/kubo` that are not also under `thirdparty`.\n" }, { "path": "thirdparty/unit/unit.go", "content": "package unit\n\nimport \"fmt\"\n\ntype Information int64\n\nconst (\n\t_ Information = iota // ignore first value by assigning to blank identifier\n\tKB = 1 << (10 * iota)\n\tMB\n\tGB\n\tTB\n\tPB\n\tEB\n)\n\nfunc (i Information) String() string {\n\ttmp := int64(i)\n\n\t// default\n\td := tmp\n\tsymbol := \"B\"\n\n\tswitch {\n\tcase i > EB:\n\t\td = tmp / EB\n\t\tsymbol = \"EB\"\n\tcase i > PB:\n\t\td = tmp / PB\n\t\tsymbol = \"PB\"\n\tcase i > TB:\n\t\td = tmp / TB\n\t\tsymbol = \"TB\"\n\tcase i > GB:\n\t\td = tmp / GB\n\t\tsymbol = \"GB\"\n\tcase i > MB:\n\t\td = tmp / MB\n\t\tsymbol = \"MB\"\n\tcase i > KB:\n\t\td = tmp / KB\n\t\tsymbol = \"KB\"\n\t}\n\treturn fmt.Sprintf(\"%d %s\", d, symbol)\n}\n" }, { "path": "thirdparty/unit/unit_test.go", "content": "package unit\n\nimport \"testing\"\n\n// and the award for most meta goes to...\n\nfunc TestByteSizeUnit(t *testing.T) {\n\tif 1*KB != 1*1024 {\n\t\tt.Fatal(1 * KB)\n\t}\n\tif 1*MB != 1*1024*1024 {\n\t\tt.Fail()\n\t}\n\tif 1*GB != 1*1024*1024*1024 {\n\t\tt.Fail()\n\t}\n\tif 1*TB != 1*1024*1024*1024*1024 {\n\t\tt.Fail()\n\t}\n\tif 1*PB != 1*1024*1024*1024*1024*1024 {\n\t\tt.Fail()\n\t}\n\tif 1*EB != 1*1024*1024*1024*1024*1024*1024 {\n\t\tt.Fail()\n\t}\n}\n" }, { "path": "thirdparty/verifbs/verifbs.go", "content": "package verifbs\n\nimport (\n\t\"context\"\n\n\tbstore \"github.com/ipfs/boxo/blockstore\"\n\t\"github.com/ipfs/boxo/verifcid\"\n\tblocks \"github.com/ipfs/go-block-format\"\n\tcid \"github.com/ipfs/go-cid\"\n)\n\ntype VerifBSGC struct {\n\tbstore.GCBlockstore\n}\n\nfunc (bs *VerifBSGC) Put(ctx context.Context, b blocks.Block) error {\n\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, b.Cid()); err != nil {\n\t\treturn err\n\t}\n\treturn bs.GCBlockstore.Put(ctx, b)\n}\n\nfunc (bs *VerifBSGC) PutMany(ctx context.Context, blks []blocks.Block) error {\n\tfor _, b := range blks {\n\t\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, b.Cid()); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn bs.GCBlockstore.PutMany(ctx, blks)\n}\n\nfunc (bs *VerifBSGC) Get(ctx context.Context, c cid.Cid) (blocks.Block, error) {\n\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, c); err != nil {\n\t\treturn nil, err\n\t}\n\treturn bs.GCBlockstore.Get(ctx, c)\n}\n\ntype VerifBS struct {\n\tbstore.Blockstore\n}\n\nfunc (bs *VerifBS) Put(ctx context.Context, b blocks.Block) error {\n\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, b.Cid()); err != nil {\n\t\treturn err\n\t}\n\treturn bs.Blockstore.Put(ctx, b)\n}\n\nfunc (bs *VerifBS) PutMany(ctx context.Context, blks []blocks.Block) error {\n\tfor _, b := range blks {\n\t\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, b.Cid()); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn bs.Blockstore.PutMany(ctx, blks)\n}\n\nfunc (bs *VerifBS) Get(ctx context.Context, c cid.Cid) (blocks.Block, error) {\n\tif err := verifcid.ValidateCid(verifcid.DefaultAllowlist, c); err != nil {\n\t\treturn nil, err\n\t}\n\treturn bs.Blockstore.Get(ctx, c)\n}\n" }, { "path": "tracing/doc.go", "content": "// Package tracing contains the tracing logic for go-ipfs, including configuring the tracer and\n// helping keep consistent naming conventions across the stack.\n//\n// NOTE: Tracing is currently experimental. Span names may change unexpectedly, spans may be removed,\n// and backwards-incompatible changes may be made to tracing configuration, options, and defaults.\n//\n// Tracing is configured through environment variables, as consistent with the OpenTelemetry spec as possible:\n//\n// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/sdk-environment-variables.md\n//\n// OTEL_TRACES_EXPORTER: a comma-separated list of exporters:\n// - otlp\n// - zipkin\n// - file\n//\n// Different exporters have their own set of environment variables, depending on the exporter. These are typically\n// standard environment variables. Some common ones:\n//\n// OTLP HTTP/gRPC:\n//\n// - OTEL_EXPORTER_OTLP_PROTOCOL\n// one of [grpc, http/protobuf]\n// default: grpc\n// - OTEL_EXPORTER_OTLP_ENDPOINT\n// - OTEL_EXPORTER_OTLP_CERTIFICATE\n// - OTEL_EXPORTER_OTLP_HEADERS\n// - OTEL_EXPORTER_OTLP_COMPRESSION\n// - OTEL_EXPORTER_OTLP_TIMEOUT\n//\n// Zipkin:\n//\n// - OTEL_EXPORTER_ZIPKIN_ENDPOINT\n//\n// File:\n//\n// - OTEL_EXPORTER_FILE_PATH\n// file path to write JSON traces\n// default: `$PWD/traces.json`\n//\n// For example, if you run a local IPFS daemon, you can use the jaegertracing/all-in-one Docker image to run\n// a full Jaeger stack and configure Kubo to publish traces to it:\n//\n//\tdocker run -d --rm --name jaeger \\\n//\t -e COLLECTOR_OTLP_ENABLED=true \\\n//\t -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \\\n//\t -p 5775:5775/udp \\\n//\t -p 6831:6831/udp \\\n//\t -p 6832:6832/udp \\\n//\t -p 5778:5778 \\\n//\t -p 16686:16686 \\\n//\t -p 14250:14250 \\\n//\t -p 14268:14268 \\\n//\t -p 14269:14269 \\\n//\t -p 4317:4317 \\\n//\t -p 4318:4318 \\\n//\t -p 9411:9411 \\\n//\t jaegertracing/all-in-one\n//\tOTEL_EXPORTER_OTLP_INSECURE=true OTEL_TRACES_EXPORTER=otlp ipfs daemon --init\n//\n//\t# In this example the Jaeger UI is available at http://localhost:16686.\n//\n// Span names follow a convention of ., some examples:\n//\n// - component=Gateway + span=Request -> Gateway.Request\n// - component=CoreAPI.PinAPI + span=Verify.CheckPin -> CoreAPI.PinAPI.Verify.CheckPin\n//\n// We follow the OpenTelemetry convention of using whatever TracerProvider is registered globally.\npackage tracing\n" }, { "path": "tracing/tracing.go", "content": "package tracing\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/ipfs/boxo/tracing\"\n\tversion \"github.com/ipfs/kubo\"\n\t\"go.opentelemetry.io/otel\"\n\t\"go.opentelemetry.io/otel/sdk/resource\"\n\t\"go.opentelemetry.io/otel/sdk/trace\"\n\tsemconv \"go.opentelemetry.io/otel/semconv/v1.4.0\"\n\ttraceapi \"go.opentelemetry.io/otel/trace\"\n\t\"go.opentelemetry.io/otel/trace/noop\"\n)\n\n// shutdownTracerProvider adds a shutdown method for tracer providers.\n//\n// Note that this doesn't directly use the provided TracerProvider interface\n// to avoid build breaking go-ipfs if new methods are added to it.\ntype shutdownTracerProvider interface {\n\ttraceapi.TracerProvider\n\n\tTracer(instrumentationName string, opts ...traceapi.TracerOption) traceapi.Tracer\n\tShutdown(ctx context.Context) error\n}\n\n// noopShutdownTracerProvider adds a no-op Shutdown method to a TracerProvider.\ntype noopShutdownTracerProvider struct{ traceapi.TracerProvider }\n\nfunc (n *noopShutdownTracerProvider) Shutdown(ctx context.Context) error { return nil }\n\n// NewTracerProvider creates and configures a TracerProvider.\nfunc NewTracerProvider(ctx context.Context) (shutdownTracerProvider, error) {\n\texporters, err := tracing.NewSpanExporters(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif len(exporters) == 0 {\n\t\treturn &noopShutdownTracerProvider{TracerProvider: noop.NewTracerProvider()}, nil\n\t}\n\n\toptions := []trace.TracerProviderOption{}\n\n\tfor _, exporter := range exporters {\n\t\toptions = append(options, trace.WithBatcher(exporter))\n\t}\n\n\tr, err := resource.Merge(\n\t\tresource.Default(),\n\t\tresource.NewSchemaless(\n\t\t\tsemconv.ServiceNameKey.String(\"Kubo\"),\n\t\t\tsemconv.ServiceVersionKey.String(version.CurrentVersionNumber),\n\t\t),\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\toptions = append(options, trace.WithResource(r))\n\n\treturn trace.NewTracerProvider(options...), nil\n}\n\n// Span starts a new span using the standard IPFS tracing conventions.\nfunc Span(ctx context.Context, componentName string, spanName string, opts ...traceapi.SpanStartOption) (context.Context, traceapi.Span) {\n\treturn otel.Tracer(\"Kubo\").Start(ctx, fmt.Sprintf(\"%s.%s\", componentName, spanName), opts...)\n}\n" }, { "path": "version.go", "content": "package ipfs\n\nimport (\n\t\"fmt\"\n\t\"runtime\"\n\n\t\"github.com/ipfs/kubo/core/commands/cmdutils\"\n)\n\n// CurrentCommit is the current git commit, this is set as a ldflag in the Makefile.\nvar CurrentCommit string\n\n// taggedRelease is set via ldflag when building from a version-tagged commit\n// with a clean tree. When set, the commit hash is omitted from the libp2p\n// identify agent version and the HTTP user agent, since the version number\n// already identifies the exact source.\nvar taggedRelease string\n\n// CurrentVersionNumber is the current application's version literal.\nconst CurrentVersionNumber = \"0.41.0-dev\"\n\nconst ApiVersion = \"/kubo/\" + CurrentVersionNumber + \"/\" //nolint\n\n// RepoVersion is the version number that we are currently expecting to see.\nconst RepoVersion = 18\n\n// GetUserAgentVersion is the libp2p user agent used by go-ipfs.\nfunc GetUserAgentVersion() string {\n\t// For tagged release builds with a clean tree, the commit hash is\n\t// redundant since the version number identifies the exact source.\n\tcommit := CurrentCommit\n\tif taggedRelease != \"\" {\n\t\tcommit = \"\"\n\t}\n\n\tuserAgent := \"kubo/\" + CurrentVersionNumber\n\tif commit != \"\" {\n\t\tuserAgent += \"/\" + commit\n\t}\n\tif userAgentSuffix != \"\" {\n\t\tuserAgent += \"/\" + userAgentSuffix\n\t}\n\treturn cmdutils.CleanAndTrim(userAgent)\n}\n\nvar userAgentSuffix string\n\nfunc SetUserAgentSuffix(suffix string) {\n\tuserAgentSuffix = cmdutils.CleanAndTrim(suffix)\n}\n\ntype VersionInfo struct {\n\tVersion string\n\tCommit string\n\tRepo string\n\tSystem string\n\tGolang string\n}\n\nfunc GetVersionInfo() *VersionInfo {\n\treturn &VersionInfo{\n\t\tVersion: CurrentVersionNumber,\n\t\tCommit: CurrentCommit,\n\t\tRepo: fmt.Sprint(RepoVersion),\n\t\tSystem: runtime.GOARCH + \"/\" + runtime.GOOS, // TODO: Precise version here\n\t\tGolang: runtime.Version(),\n\t}\n}\n" }, { "path": "version_test.go", "content": "package ipfs\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\n// TestGetUserAgentVersion verifies the user agent string used in libp2p\n// identify and HTTP requests. Tagged release builds (where the commit matches\n// the tag) skip the commit hash from the agent version, since the version\n// number already identifies the exact source.\nfunc TestGetUserAgentVersion(t *testing.T) {\n\torigCommit := CurrentCommit\n\torigTagged := taggedRelease\n\torigSuffix := userAgentSuffix\n\tt.Cleanup(func() {\n\t\tCurrentCommit = origCommit\n\t\ttaggedRelease = origTagged\n\t\tuserAgentSuffix = origSuffix\n\t})\n\n\ttests := []struct {\n\t\tname string\n\t\tcommit string\n\t\ttagged string\n\t\tsuffix string\n\t\texpected string\n\t}{\n\t\t// dev builds without ldflags\n\t\t{\n\t\t\tname: \"no commit, no suffix\",\n\t\t\texpected: \"kubo/\" + CurrentVersionNumber,\n\t\t},\n\t\t// dev builds with commit set via ldflags\n\t\t{\n\t\t\tname: \"with commit\",\n\t\t\tcommit: \"abc1234\",\n\t\t\texpected: \"kubo/\" + CurrentVersionNumber + \"/abc1234\",\n\t\t},\n\t\t{\n\t\t\tname: \"with suffix, no commit\",\n\t\t\tsuffix: \"test-suffix\",\n\t\t\texpected: \"kubo/\" + CurrentVersionNumber + \"/test-suffix\",\n\t\t},\n\t\t{\n\t\t\tname: \"with commit and suffix\",\n\t\t\tcommit: \"abc1234\",\n\t\t\tsuffix: \"test-suffix\",\n\t\t\texpected: \"kubo/\" + CurrentVersionNumber + \"/abc1234/test-suffix\",\n\t\t},\n\t\t// tagged release builds: commit is redundant because the version\n\t\t// number already maps to an exact git tag, so it is omitted to\n\t\t// save bytes in identify and HTTP user-agent headers.\n\t\t{\n\t\t\tname: \"tagged release ignores commit\",\n\t\t\tcommit: \"abc1234\",\n\t\t\ttagged: \"1\",\n\t\t\texpected: \"kubo/\" + CurrentVersionNumber,\n\t\t},\n\t\t{\n\t\t\tname: \"tagged release with suffix ignores commit\",\n\t\t\tcommit: \"abc1234\",\n\t\t\ttagged: \"1\",\n\t\t\tsuffix: \"test-suffix\",\n\t\t\texpected: \"kubo/\" + CurrentVersionNumber + \"/test-suffix\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tCurrentCommit = tt.commit\n\t\t\ttaggedRelease = tt.tagged\n\t\t\tSetUserAgentSuffix(tt.suffix)\n\n\t\t\tassert.Equal(t, tt.expected, GetUserAgentVersion())\n\t\t})\n\t}\n}\n" } ]